Understanding Integrated Environmental Assessment in a Multi-Stakeholder Negotiation via Role-Play

Abstract

This article explains how a role-play game (RPG) can allow participants to get insights on the dynamics of incorporating scientific advice into adversarial policymaking. A RPG called GROUND-LEVEL OZONE (GLO) simulates a multi-stakeholder negotiation based on the recommendations of an integrated environmental assessment (IEA) for urban air quality management. Structured debriefing of the RPG used questionnaire survey of participants’ perceptions of IEA usefulness as well as participants’ feedback about the use of the IEA and performance of the facilitators during negotiations. By linking those perceptional and behavioral patterns to the negotiated outcomes, the debriefing drew much broader insights on the importance of upstream engagement for enhanced legitimacy of the IEA and constructive roles of a facilitator in a multi-stakeholder negotiation. This article also critically reviews the debriefing method and suggests a new design of more sophisticated debriefing method and a new RPG for more effective learning of the dynamics of the use of IEA in a multi-stakeholder negotiation in the future.

Keywords

debriefing facilitator integrated environmental assessment multi-stakeholder negotiation role-play game urban air pollution

Integrated environmental assessments (IEAs) have been developed as new analytic tools for decision-makers who often face uncertain forecasts, conflicting values of multi-actors, high stakes, and pressure to make quick choices (Funtowicz & Ravetz, 1992; Jäger, 1998). However, when the result of an IEA is to be used for a multi-stakeholder negotiation, the ideal of an IEA may obscure more than it clarifies due to inherent tensions between the realms of science and politics and the possibility of strategic use of the results of an IEA in the realm of adversarial policymaking.

Fundamental differences between science and politics limit their interactions (Caplan, 1979; Dunn, 1980). Scientific experts often try to maintain their integrity by being separated from turbulent politics. Natural scientists and engineers tend to interpret their assessments as purely technical ones rather than social and political ones. Such response is based on strong dichotomies between facts and values, and science and politics (Funtowicz, 2006). The lack of interactions between scientific experts and partisan stakeholders in an IEA process may cause the legitimacy deficit of the result of the IEA that is often the source of conflict among stakeholders in adversarial policymaking. Stakeholders make strategic use of their own scientific assessments and exploit the uncertainty in the data to delay a decision (Funtowicz & Strand, 2007; Susskind, 2008).

A growing literature has called for so-called upstream engagement as a solution where stakeholders participate in the framing decision prior to scientific assessment (Funtowicz & Strand, 2007; Karl, Susskind, & Wallace, 2007; Kates et al., 2001; Mayer, 2009; Nowotny, Scott, & Gibbons, 2001; Susskind et al., 2001; Toth, 2001; Wynne, 1992). Framing decisions that determine scopes of the assessment may affect in a crucial way the outcome of scientific advice. Also, a large body of literature on science-intensive environmental policy disputes demonstrates the importance of professional neutrals in joint fact-finding, environmental mediation, and consensus building (Adler et al., 2000; Karl et al., 2007; Ozawa & Susskind, 1985). In most instances, however, implementing such a participatory IEA with the involvement of facilitators from the outset is rarely easy. Thus, it matters to motivate scientific experts, decision-makers, and stakeholders to modify their conventional wisdoms on the relations between science and politics. Also, decision-makers and stakeholders need to build capacity to negotiate effectively based on the result of the IEA.

As a training tool for actors in science-politics interface, I developed a role-play game (RPG) called GROUND-LEVEL OZONE: A Negotiated Rule-Making Process on Varara’s Air Pollution Crisis and used it in an air pollution control workshop where relevant stakeholders could participate. GLO simulates a multi-stakeholder negotiation that is assisted by a facilitator and is based on the results of an IEA for urban air quality management.

This article mainly explains how GLO and its debriefing can allow participants to get insights on the dynamics of incorporating scientific advice into adversarial policymaking. After a brief overview of the uses of RPGs in relations to IEAs and environmental policymaking, I describe the characteristics of urban air pollution control in Mexico City as a basis for GLO. Then, this article focuses on how the debriefing was conducted with a questionnaire survey data on the perceived IEA usefulness, the usage of the IEA, and performance of facilitators during negotiations. Building on these experiences, I offer recommendations for improving the design of the RPG and debriefing method for more effective experiential learning.

RPG, IEA, and Environmental Policymaking

RPGs have proved to be useful in complex environmental policymaking. Their usefulness hinges on key characteristics of simulation and gaming. Simulations have unique power to capture and integrate both the technical-physical complexity and the social-political complexity of environmental problems (Crookall & Thorngate, 2009). Thus, some RPGs help participants to acquire substantive and holistic understanding of various complex environmental problems, such as water pollution management (Camargo, Jacobi, & Ducrot, 2007), irrigation projects (Dionnet, Kuper, Hammani, & Garin, 2008), biodiversity and conservation (Mathevet et al., 2007), energy policy (Wolf & Laessig, 1973), environmental cleanup in waste treatment sites (Applegate & Sarno, 1997), and climate change (Parson, 1997).

Its gaming component allows participants to be more creative and to learn from mistakes in a safe environment without risking their careers (Duke & Geurts, 2004; Mayer, 2008). Such gaming component enables RPGs to be a dominant tool to teach negotiation and consensus building skills in many decision-making contexts (Innes & Booher, 1999; Susskind & Corburn, 2000). Some RPGs were used as instrumental policy intervention tools, for example, to persuade natural resource users to adopt innovative practices (Kuipers, 1983) and to facilitate official group processes by bringing real stakeholders into an experiential table to discuss national energy policies (Dolin & Susskind, 1992).

RPGs have been developed in various associations with IEAs. Most literature on the roles of RPGs with IEAs discusses their instrumental benefits, such as improving the quality of models and generating proper scenarios. For example, “policy exercises” provide opportunities for scientists and stakeholders to generate plausible scenarios together and analyze the scenarios through the interactive formulation and testing of alternative policy options in workshop settings (Brewer, 1986; Mermet, 1993; Toth, 1988). Companion Modeling (ComMod) practice is another example, with a research team and local participants developing a model and a RPG together. Then, the RPG conducted in workshop settings helps to exchange points of views and adjust assumptions under the model to local stakeholders’ perspectives (Boissau & Castella, 2003; Gurung, Bousquet, & Trébuil, 2006). Sometimes, a RPG and a computerized model run simultaneously to support negotiation among stakeholders in more controlled experiment. In that case, the RPG is just a medium through which participants get acquainted with the contents of the computerized model, so that they may use it knowingly and follow the results of the model in real situation later (Barreteau, 2003).

In terms of its relation to IEA, GLO stands out in a few aspects. First, no RPG has been created so far in conjunction with IEAs for urban air quality management.¹ Only some interactive computer simulation games (e.g., Smog City 2, U.S. Environmental Protection Agency and Sacramento Air Quality Management District, n.d.) have been developed to teach the causal relationships between air pollution and other factors, such as energy consumption and price (for acid rain issue, see Baba, Uchida, & Sawaragi, 1984). Second, few literature discusses how a RPG can provide participants with insights about how to produce knowledge through an IEA for complex environmental policymaking, such as urban air quality management, and how to utilize them in a multi-stakeholder negotiation.

Urban Air Pollution in Mexico City and IEA

Managing urban air quality is a very complex task. Air pollutants from cars and industries do not observe jurisdictional boundaries and are the complex function of emissions, pollution control policies, energy, technical changes, and weather conditions (Portney, 1990). Governments, industries, and environmental groups are engaged in adversarial competitions to affect a policy outcome favorable to their own interests (Ackerman & Hassler, 1981; Jones, 1975; Morris, Yandle, & Dorchak, 2004).

Due to its technical-physical complexity, academic discourse on urban air pollution has been dominated by scientific and technological knowledge (Heymann, 2004). Accordingly, urban air quality management (UAQM) also depends mainly on heterogeneous set of computer-based scientific models from multiple disciplines in a single framework (Dennis et al., 1996; Schöpp, Amann, Cofala, Heyes, & Klimont, 1998). In turbulent policymaking area, industries and environmental groups have been regarded not as participants, but as regulatory targets and outsiders (Fiorino, 2001). Recently, however, cases of multi-stakeholder negotiation for UAQM based on scientific assessment are emerging in many parts of the world, but still rare (Kim, 2007; Weber & Khademian, 1997).

Although Mexico City, once considered the most polluted urban area in the world, had cleaned up its sky considerably since the 1980s (Tollefson, 2010), unhealthy concentrations of air pollutants, such as ground-level ozone (O₃), nitrogen oxides (NOx), and particulate matter (PM) have not been reduced over the past decade (Secretaría del Medio Ambiente, 2010). The Mexico City Metropolitan Area (MCMA) with more than 20 million people, 3.5 million vehicles, and 35,000 industries and services is home to the national political institutions and has concentration of economic investments (L. T. Molina & Molina, 2002).

The topography and meteorology of the MCMA contribute substantially to air pollution. The MCMA lies in an elevated plateau where abundant ultraviolet radiation triggers photochemical process involving volatile organic compounds (VOCs) or hydrocarbons (HC), and NOx to form O₃. Mountain ridges confine the MCMA on three sides to trap pollutants within the basin, especially in winter (M. J. Molina & Molina, 2004).

The transportation sector was a major source of air pollution in the MCMA, accounting for more than 80% of NOx, 40% of VOC, and 35% of PM₁₀ in 1998 (Comisión Ambiental Metropolitana, 2001). Rapid urban growth with increasing population in surrounding, but relatively poor jurisdictions creates more travel demand. However, the very demands was being met not by public transportation systems, such as buses and metro system, but by increasing number of private vehicles and relatively old, polluting colectivo (mini-buses). In particular, the colectivo system was competing for patronage on the same routes, frequently under conditions of limited regulation (Darido, 2001) and became dominant mode of mass transportation, causing a decrease in Metro ridership (Sistema de Transporte Colective del Gobierno del Distrito Federal, 1999).

Political and institutional complexities hindered collaboration and coordination among different levels of governments and different functional agencies in the MCMA. When each of the three levels of governments (city, state, and federal) involved in the Metropolitan Environmental Commission were administrated by a different political party, they had trouble in cooperating (L. T. Molina & Molina, 2002). While the population of the MCMA was split about equally between the adjacent two jurisdictions, the Federal District, and the State of Mexico, they had unequal financial resources and economic development. The per capita share of the federal tax revenue by the Federal District had been more than twice that of the State of Mexico (Krebs, 1999).

In February 2002, Metropolitan Environmental Commission unveiled a long-term air quality improvement program for the MCMA, called “PROAIRE III” (2002-2010). Upon the recommendations from research community, an IEA was developed to provide scientific underpinning for the long-term plan and to identify optimal policy strategies that could best accomplish the emission reduction goals of PROAIRE III.²

The scientific experts in the IEA used an analytic technique called “scenario-based multi-attribute trade-off analysis.” In order to avoid the pitfalls of deterministic planning (Schwartz, 1991), three plausible scenarios for the future of the MCMA by 2025 were formulated as “Keeping Up,” “Sustainable Growth,” and “Global Drag.” Each scenario was based primarily on economic and political trends, such as the integration of Mexico into both a regional and global trading regime, the health of the global economy, and institutional reforms and political fragmentation in Mexico (Connors et al., 2002). Also, the basic premise of the technique was to allow multi-stakeholders to negotiate with the information on superior portfolios of options and to make trade-offs across multiple attributes or performance variables, such as environmental performance and implementation costs (Bespolka, 1989). As an illustration in Figure 1, once an IEA provides the quantitative analysis of all the policy option strategies in terms of emission reductions and its implementation costs, decision-makers easily identify optimal policy option strategies (A, B, F, H, K) and then negotiate to choose one in the optimal line between A (highly effective, but costly) and K (ineffective, but cheap).

Figure 1.

Concept of trade-off analysis.

Development of GLO

Although the main model of the IEA had the premise for a multi-stakeholder negotiation, much uncertainty existed regarding the efficacy of unprecedented multi-stakeholder negotiation in the MCMA and the usability of the model in helping various partisan stakeholders negotiate. Despite no conspicuous public conflict on air pollution issue, stringent policy options proposed from the IEA might cause disputes among stakeholders and even among different government agencies. In such a context, development of a RPG was considered as a tool to enhance the legitimacy and the usability of the results of the IEA in a multi-stakeholder negotiation in the future.

However, the idea of a RPG came relatively late when the scenarios and quantitative models were almost finalized by experts. Without the possibility of using a RPG for upstream engagement, GLO was developed as a training tool with which decision-makers, stakeholders, and experts in the MCMA could experience in advance the dynamics of a multi-stakeholder negotiation using the results of scenario-based multi-attribute trade-off analysis and get valuable insights on the ways to produce and use the IEA for their decision-making.

The RPG simulated key characteristics of the model and natural, socio-political and institutional settings related to urban air quality management in the MCMA. The synopsis was crafted with crisis component of high level of ground-level ozone (O₃), so that participants could become more willing to discuss sensible solutions and learn from others (for more detailed contents of the RPG, log on the website at http://mce2.org/en/education/simulation-game). In order to make the synopsis and scientific information in the RPG look plausible, the draft of the RPG was reviewed through several test runs involving modeling experts and other Mexican researchers.

Conducting the RPG in a Workshop

GLO was conducted at the air quality management workshop in Mexico City on January 21, 2003,³ where many governmental officials at various levels and non-governmental stakeholders participated. The RPG was scheduled after the first three plenary sessions of the workshop, so that the participants could get more knowledge about technical and institutional issues of the air pollution in the MCMA before the exercise. The first plenary session discussed the facilitating roles of Metropolitan Environmental Commission in constructing political dialogues among multiple stakeholders in the metropolitan regulatory process. The second and the third plenary sessions addressed various topics of sustainable energy, diesel fuel quality, vehicle emission control technology, fuel/emission standards, air quality measurements and modeling, atmospheric chemistry, the impact of diesel emissions on public health, and the relationships among land use, mobility and urban development.

In the second afternoon of the workshop, 68 people participated in the RPG, making up eight negotiation tables. Around each table, sat eight players. Five Spanish-speaking tables were numbered from one to five, and three English-speaking tables, from six to eight.⁴ RPG organizers distributed seven roles randomly to the participants except the role of facilitator with the expectation that playing a different role from one in real life might help participants see a problem from another perspective (Etienne, 2003). Thus, although some participants played their real roles, most did not. Given the complexity of multi-party and multi-issue negotiation using much scientific data, the role of facilitator was extremely important. In order to provide good facilitators for the groups, workshop organizers nominated some participants who met the criteria of seniority, organizational position, and communication capacity.

Although the RPG started after the long and intensive plenary sessions, participants in the game were very serious in their 2-hour negotiation. At some tables, very serious, loud communications were observed. Even the earthquake of over 7 on the Richter scale that hit the Mexico City during the running time did not prevent them from hammering out agreements (Figure 2).

Figure 2.

A picture of negotiating tables in the workshop in 2003.

Right after the RPG, the participants were asked to fill out a simple questionnaire aimed at tracking their perceptions on the usefulness of the results of the IEA in shaping the group decisions and at evaluating the performance of the facilitator during their negotiation. Their answers were scaled from 1 (least useful and worst) to 10 (most useful and best). The facilitators were asked to report when and how their group members used the results of the IEA during their negotiation.

Debriefing

Perceived IEA Usefulness

The next morning a debriefing session began with a question of how useful the results of the IEA were for shaping the group decisions. After some feedback from the audience, the moderator presented a table that analyzed data from the questionnaire survey.⁵ It showed that participants perceived the usefulness of IEA very differently according to their roles (Table 1). For example, much lower scores from ALC and MBTA representatives implied that they did not perceive the IEA as useful in their negotiation. On the other hand, CPC and MTC negotiators considered them very useful.

Table 1.

Perceived IEA Usefulness by Each Role.

	Score (n = 7)
Role	M	SD
Camitia Petroleum Company (CPC)	8.97	0.88
Metropolitan Transportation Commission (MTC)	8.00	1.12
Varara Environmental Department (VED)	7.50	2.04
Camitia Auto Industry Association (CAIA)	7.12	1.45
Ministry of Finance (MOF)	5.85	3.33
Alliance for a Livable City (ALC)	5.50	3.84
Microbus Transportation Association (MBTA)	3.43	2.17
Overall	6.62	1.33

Note. IEA = integrated environmental assessments.

In order to identify key factors behind their perceptions, the moderator asked the participants the reasons for their specific scores. A MTC representative who gave very high scores mentioned two functional benefits of IEA. Thanks to the IEA, he was able to identify many superior policy packages with ease and to shift participants’ attention from subjective political issues toward more objective scientific findings. Interestingly, a CPC negotiator commented that she could spend much more time using the IEA during the negotiation than other negotiators from governmental agencies and ALC since CPC had fewer controversial issues than other parties did.

The degree to which the IEA supported participants’ positions and interests in negotiation was another factor. A VED representative found that the policy option of “Harmonization of Inspection and Maintenance” belonged to a few superior policy packages across the two scenarios. Thus, the VED representative tried to use the IEA actively in order to advance their position. However, a few MBTA representatives had to doubt the credibility of the IEA because the IEA listed “Regulated Competition for Microbuses” as an important component of several superior option packages. With that option adopted, microbus operators would be required to comply with stringent environmental standards in order to get their operating licenses. Doing so would incur considerable financial strains for them.

The scope of data used in the IEA and the way data were presented also affected participants’ perception on IEA usefulness. The data in the simple form of emission reductions and implementation costs caused some participants to suspect that the IEA had been prepared only for a few central government agencies that had implementation power. A MBTA negotiator complained that, in case of “I/M Harmonization” option, the IEA did not consider financial burdens for their members in its total costs. One ALC representative criticized that the IEA ignored the health benefits from improvements in air quality. Some participants did not understand why two scenarios were necessary in identifying robust policy packages. One MOF representative doubted the credibility of the IEA because it did not include uncertainty ranges for its forecasts.

The parties’ negotiation strategies driven by their political and economic situations also seemed to affect participants’ perceptions on IEA. For example, an ALC representative revealed that one of her negotiating strategies was to be very vocal with political arguments and maintain adversarial stance on a range of issues throughout negotiation processes. Consequently, she could not spare enough time to focus on the IEA. In the same vein, a MBTA representative found it difficult to shake off a sense of inferiority in terms of political and economic status, and relied upon aggressive bargaining strategies to defend his position.

Perception on the legitimacy of the IEA was very important factor, particularly for MBTA representatives. Only MBTA representatives were hinted in their confidential memo for negotiation that the IEA was prepared without their participation. Thus, MBTA representatives started their negotiation assuming that the IEA was illegitimate.

Use of IEA and Roles of Facilitator

For cross-group comparison, participants were asked to reflect on their negotiation process, particularly in terms of how they used the IEA within their negotiation. In order to facilitate discussion, the moderator prepared another presentation based on facilitators’ records on how negotiators used the IEA in their group and participants’ evaluation of facilitators’ performance (Table 2). However, Group 1 did not answer questionnaire. Groups 2, 3, and 5 formulated new policy options other than those given by the RPG in order to persuade disgruntled stakeholders into unanimous settlements. Thus, the data from those three groups could not be used for comparative analysis for debriefing.⁶

Table 2.

Cross-Group Comparison of Perception on the IEA, Usage of IEA, and Evaluation of Facilitator.

	Groups				Point (n = 4)
Categories	Group 4	Group 6	Group 7	Group 8	M	SD
Roles
CPC	9	10	8	10	9.25	0.83
MTC	6	8	8	8	7.50	0.87
CAIA	9	5	7	5	6.50	1.66
VED	4	7	9	5	6.25	1.92
MOF	4	10	9	1	6.00	3.67
ALC	1	1	10	2	3.50	3.77
MBTA	1	4	5	1	2.75	1.79
M	4.86	6.43	8.00	4.57	5.96	2.23
SD	3.34	3.31	1.63	3.51
Evaluation of facilitator (n = 7)
M	8.00	8.28	9.29	7.29	8.21
SD	0.58	1.70	0.76	1.80	0.83
Use of IEA data during negotiation	Not much	Only at the outset	Throughout negotiation process	Not much

Note. IEA = integrated environmental assessments; CPC = Camitia Petroleum Company; MTC = Metropolitan Transportation Commission; CAIA = Camitia Auto Industry Association; VED = Varara Environmental Department; MOF = Ministry of Finance; ALC = Alliance for a Livable City; MBTA = Microbus Transportation Association.

The data from Group 7 were remarkably different from those of other three groups. Group 7 regarded the IEA as very useful as a whole (8 points on average). Also, the group showed less variation in their perceptions on the IEA, while other groups were polarized in their opinions on the IEA. That difference could be largely explained by the relatively higher scores given by ALC and MBTA negotiators in Group 7. An intriguing question was whether negotiators for ALC and MBTA in Group 7 could be indifferent to the factors, such as aggressive negotiating strategies, that caused the same-role negotiators in other groups to perceive the IEA negatively.

ALC and MBTA negotiators in Group 7 offered a few interesting clues to the puzzle of their unusual perceptions. First, they appraised that their facilitator managed a complex negotiation process so effectively. For example, the facilitator enforced ground rules about participants’ negotiating behaviors, gave fair chances to speak, managed time wisely, and dealt with negative emotions among participants wisely. According to their comments, facilitator’s effort had an effect to soften their aggressive negotiating strategies and helped them focus more on the IEA during negotiation.

More specifically, their facilitator constantly referred to the IEA as guidance to their decision-making. This point was corroborated by another data about when and how negotiators used the IEA. According to the feedback from facilitators, Group 6 seemed to try to use the IEA primarily at the beginning of their negotiation when the parties introduced themselves and stated their main positions and interests. However, they did not maintain their attention on the IEA, but political positional bargaining dominated the later part of their negotiation. In case of Groups 4 and 8, facilitators reported that the parties did not use the IEA much in their negotiation. Furthermore, they observed that whenever some members tried to refer to the IEA, they were met with serious objections from others who downplayed the IEA. Large differences in perception on IEA usefulness within the two groups might cause conflicts on the use of IEA during the negotiation.

Interestingly, the facilitator in Group 7 reported that some negotiators referred to the IEA from the beginning of the negotiation in order to support their favorite policy options and others used the IEA to identify optimal option packages and make trade-off among those packages at the end of negotiation. A few members in Group 7 mentioned that their facilitator was helpful in ensuring that the IEA could be conveyed in a language that was readily comprehensible to all the participants. Overall, the facilitator in Group 7 received highest approval rate of 9.29 from their members.

The Results of Negotiations

Finally, the moderator put together the negotiated outcomes from the four groups across two scenarios in two graphs (Figures 3 and 4). As NOx/HC reduction and implementation costs are the only interests and criteria for the IEA, negotiated outcomes were compared in terms of the amounts of NOx/HC emission reductions and their implementation costs of the adopted policy packages. The dotted lines in the upper-left hand corner of each graph indicate the “optimal” line along which superior packages, identified by the IEA, would be located. With the help of the IEA, negotiators could have easily recognized superior policy option packages from the beginning and made a trade-off between achieving higher NOx/HC reduction and lowering costs along the “optimal” line.⁷

Figure 3.

Projected performance of the negotiated outcomes under “Divided City” scenario.

Figure 4.

Projected performance of the negotiated outcomes under “High Growth” scenario.

The mandate for negotiators was to adopt an effective, efficient, and robust agreement that could tackle serious urban air pollution. The agreement should be effective in terms of emission reductions, and efficient in terms of implementation costs. Facing uncertainty in terms of overall economic and social well-being in the metropolitan area, negotiators should adopt an agreement that would be robust across the two possible scenarios.

In terms of NOx emissions reduction only, Group 6 seemed to formulate the most effective agreement across the two scenarios. However, considering the financial resources available within a group was limited up to US$33.4 billion, Group 6 made a pledge to spend more money than they could afford under the “Divided City” scenario, which made their agreement unstable. On the other hand, Group 7 generated an agreement that could reduce HC emissions the most and relatively large amounts of NOx emissions at much lower or similar amounts of costs than other groups could do. In addition, the agreement from Group 7 seemed to be more robust and stable across the two scenarios than other agreements. All things considered, Group 7 achieved more effective, efficient, and robust agreement in terms of total emission reductions of NOx and HC and total costs across the two scenarios that other groups did.

Insights From GLO

This section explains what kinds of insights the moderator tried to draw by linking all the data and comments from the participants with their negotiated outcomes. First and foremost, the achievement of the Group 7 might be possibly linked to the members’ positive perception on the IEA, their persistent use of the IEA during negotiation, and the efforts of skillful facilitator. Then, suboptimal negotiated outcomes were associated with the factors that affected participants’ perceptions on the IEA negatively and the lack of use of the IEA during their negotiation. If the linkage is plausible, then it follows the questions of how to improve the perception on the IEA in the future and how to make them use it more effectively in a multi-party negotiation.

Negotiators often fail to achieve the best possible deal. They tend to be just satisfied with their agreement without knowing the existence of better deals. An abundance of literature on negotiation point that for “win-win” negotiation, parties should create values, or “move northeast” on a two axis graph, where each axis represents the value of an agreement for each party to an optimal line (Lax & Sebenius, 2006). The IEA in GLO was the tool to create values much easier by providing the list of optimal policy option packages for negotiators. However, this ideal situation could be realized only when the negotiators should believe first that the IEA was really worthwhile to follow.

Scientific experts in the real IEA project also observed and participated in the RPG in the workshop and experienced how an IEA could be utilized in a multi-stakeholder negotiation for urban air quality management. The debriefing manifested for them that, when an IEA is to be used for a multi-stakeholder negotiation, several factors might prevent parties from following the recommendations of the IEA. In other words, the legitimacy of IEA might not necessarily be earned by the optimality of a quantitative model.

The moderator opened the question of how to make stakeholders perceive the IEA positively to the extent that they could follow its recommendation in a negotiation. Then, the moderator introduced various mechanisms of upstream engagement that could improve the quality of the IEA. Discussion on the need for early engagement of partisan stakeholders in the IEA also prompted participants to deliberate on the tendency of experts and scientists to maintain their integrity by being separated from politics and on the challenges of incorporating conflicting values of partisan stakeholders in their assessment.

Partisan stakeholders could think about their roles in the IEA. They will not necessarily be passive recipients of the recommendations of the IEA at the end. If they just shift the responsibility for fact-finding to only scientific experts and blame them later for various reasons, the role of IEA as a decision-making tool will be very limited. Thus, even lay stakeholders have a good reason to bring their interests and concerns to the IEA from its early stage, so that they could utilize the final product of IEA very effectively for their negotiation.

The last, but not least, insight for both scientific experts and stakeholders was the roles of facilitator in science-intensive policy disputes. Participants experienced for themselves how skillful facilitators could help them negotiate complex issues in more organized and fairer ways, and affect their usage of the IEA and even final negotiated outcomes.

Discussion of the Debriefing Method

Getting the meaningful insights from the GLO hinges on the efficacy of the debriefing method that allows participants to buy into causal relationships between perceptional and behavioral variables and negotiated outcomes. The moderator in the debriefing effectively induced the participants to do that step-by-step by using data from questionnaire surveys and feedback from participants.

However, current debriefing method may be controversial due to fuzzy concept of IEA usefulness. The measurement of IEA usefulness by a single question may be too simplistic to construct proper causality between the variables. Thus, I discuss potential problems with the concept of IEA usefulness and suggest alternative concepts that can be measured for debriefing in a similar RPG in the future.

Reconstruction of the Concept of IEA Usefulness

Participants’ perceptions on IEA usefulness were measured mainly because the utility of the IEA as a decision-supporting tool for a multi-stakeholder negotiation was one of the key concerns for the IEA project. It was simply assumed at the outset that if the IEA could be utilized effectively by negotiators, they would make a better decision.

However, after the debriefing, I found the concept of usefulness is too broad to threat the validity of measurement. In other words, same scores on IEA usefulness may mean very different things. On the one hand, some parties may consider the IEA useful simply because it helps them to identify superior options and to trade-off among these options much faster. For others, the IEA may be seen as useful because it can shift parochial perspectives to integrated one and allow participants to see a greater connectivity of issues. Such concepts of IEA usefulness are relatively neutral and functional. In the debriefing, a few participants mentioned these functional utilities of the IEA. These functional concepts of usefulness may be conditional on a prior assumption that the IEA has a legitimate claim. Parties may not view the IEA as functionally useful if they doubt the legitimacy of the IEA.

Partisan stakeholders, on the other hand, may view the IEA through a lens of self-interests. Some participants perceived the IEA in this way in the RPG. Thus, it may be possible that some parties may not view the IEA as legitimate, but regard it as useful just because it helps to support their positions in a negotiation. For this kind of utility, I label as strategic utility of the IEA. Others may see the IEA as legitimate, but does not perceive it useful mainly because it is detrimental to their interests. Thus, high scores on its usefulness may not translate into what the IEA project cares the most, that is, high level of perceived legitimacy of the IEA or functional utilities of the IEA.

The other conceptual issue with IEA usefulness is that perception on the IEA may not be static, but dynamic through a negotiation processes. As the debriefing hinted, perception on the functional utilities of the IEA and even its strategic utilities may be altered during a negotiation due to change of negotiating strategies and enlightened use of the IEA with the help of facilitator. In other words, parties may come to perceive the IEA as more useful just because they have more chances to use it in more effective way during the process. Thus, current measurement of perceived IEA usefulness with a single question survey right after the RPG may not capture the dynamics of perceptional change during the negotiation process.

New Design of Debriefing Method for Learning and Research

Building upon the experience in the RPG and critical review of the debriefing method, I suggest a new design of debriefing method to understand better the dynamics of the use of IEA in a multi-stakeholder negotiation and provide more effective experiential learning about them. First, a new debriefing method needs to deal with the questions of what to measure, when to measure them, how to measure them, how to use them in a debriefing, and for which purposes. We are interested in knowing why and how various parties come to follow the recommendations of the IEA in their negotiation. Also, we are more interested in how and why some parties do that even if the IEA does not support their positions and interests in a negotiation.

Understand the pure effect of the functional utilities of an IEA on parties’ adoption of the recommendation of the IEA in a multi-party negotiation may require measurements on their perception on strategic utilities of the IEA as well as on legitimacy of the IEA before, during, and after the RPG with more sophisticated questionnaire survey questions. Also, qualitative feedbacks on the negotiating behaviors, use patterns of the IEA, and the roles of facilitators may be necessary to construct the causalities among all the variables in the dynamics.

Synthesizing and comparing the data from all the measurements may make it possible to test whether perception on the legitimacy of the IEA is necessary condition for functional utilities of the IEA. Also, we may understand more clearly the reason why parties decide to follow the recommendations of the IEA under the situation that the IEA does not have strategic utilities and parties plan to use aggressive negotiating strategies. It might be more interesting to know how facilitators could change their perceptions on legitimacy, functional utilities, strategic utilities, and even negotiating strategies by analyzing data from those measurements.

Regarding the measurement on legitimacy of the IEA, current RPG does not provide the opportunity for participants to judge the legitimacy of the recommendations of the IEA. Only MBTA negotiators were informed by their confidential information to discredit the IEA as illegitimate on the ground that they did not participate in the assessment process. Thus, a new RPG could be designed to have two phases where participants could negotiate for framing decision for the IEA (Part I) and negotiate with the results of the IEA later (Part II). For example, in Part I, parties including expert scientists may reconcile their different interests on the scope of the assessment, necessary data, contents of potential scenarios, and/or even format of the final report. With their input, an IEA model may shortly produce the recommendations with which the parties can negotiate for environmental decisions in Part II. If RPG organizers or researchers designate separately a non-treatment group who do not play Part I, but only negotiate with the results in Part II, then it would be possible to compare the negotiated outcomes from treatment groups who go through Parts I and II with those of the non-treatment group.

Finally, one-time exercise in a workshop setting, however, precludes a rigorous test on causality among any variables primarily due to the lack of data. Thus, causality test may require a combination of repeated exercises in a controlled laboratory setting and empirical case studies.

Conclusion

GLO was developed for mainly educational purpose in conjunction with the IEA on urban air quality management in Mexico City. By simulating the probable situation where the results of the IEA are used for a multi-stakeholder negotiation with the help of a facilitator, the RPG provided the participants with insights on the importance of upstream interaction between scientific experts and stakeholders and the roles of facilitators in managing the boundaries between science and politics.

In order to draw such insights, the debriefing used a simple questionnaire survey on participants’ perceptions on IEA usefulness as well as qualitative feedback from the participants on the usage of the IEA and performance of the facilitators during the negotiations. By linking these perceptional and behavioral variables with the negotiated outcomes from different tables, participants experientially learned that their perceptions on IEA and their negotiating strategies including how to use the IEA in a negotiation might affect negotiated outcomes in the end. Such insights made it possible to draw much broader lessons about how to produce scientific advice from the beginning.

However, current design of the debriefing method might be improved in a few ways in order to allow the participants in the RPG to buy into causal relationships among the variables much clearly. This article discusses the problems of fuzzy concepts of IEA usefulness and recommends a new design of debriefing method that may use more sophisticated questionnaire surveys and feedbacks to measure the legitimacy of the IEA, the functional utilities of the IEA, strategic utilities of the IEA, specific usage of the IEA, and the roles of facilitators before, during, and after the RPG. In order to measure the legitimacy of the RPG, a new RPG may have two phases of producing an IEA and utilizing the IEA in a multi-stakeholder negotiation, so that participants including scientific experts learn about the importance of upstream engagement in increasing the legitimacy of the IEA.

Footnotes

Acknowledgements

The author would like to offer special thanks to Professor Mario Molina and Dr. Luisa Molina for giving the opportunity to develop GLO as a part of an extraordinary interdisciplinary research team at MIT. He is also very grateful to Professor Lawrence Susskind at MIT for helping to design the RPG and giving critical comments on the draft of the article. He is also indebted to Dr. Olivier Barreteau at the Cemagref Irrigation Research Unit (CIRU), in France, and Dr. Mark Elder at the Institute for Global Environmental Strategies (IGES), in Japan, for their constructive suggestions to improve the manuscript, first as a blind reviewer, and then as a coaching reviewer.

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The author received a financial support directly from the Integrated Program on Urban, Regional and Global Air Pollution (IPURGAP) for his doctoral research in connection with this article.

Notes

Author Biography

Dong-Young Kim is an associate professor at Korea Development Institute (KDI) School of Public Policy and Management in South Korea. His research interests include theory and practice of public and environmental dispute resolution and negotiation. He received his PhD in public policy and environmental planning from MIT.

Contact: dykim@kdischool.ac.kr.

References

Ackerman

B. A.

Hassler

W. T.

(1981). Clean coal/dirty air or how the clean air act became a multi-million dollar bailout for high-sulfur coal producers and what should be done about it. New Haven, CT: Yale University Press.

Adler

P. S.

Barrett

R. C.

Bean

M. C.

Birkhoff

J. E.

Ozawa

C. P.

Rudin

E. B.

(2000). Managing scientific and technical information in environmental cases: Principles and practices for mediators and facilitators. Washington, DC: RESOLVE, The U.S. Institute for Environmental Conflict Resolution, and the Western Justice Center Foundation.

Applegate

J. S.

Sarno

D. J.

(1997). FUTURESITE: An environmental remediation game-simulation. Simulation & Gaming: An Interdisciplinary Journal, 28, 13-27.

Baba

Uchida

Sawaragi

(1984). A gaming approach to the acid rain problem. Simulation & Games, 15, 305-314.

Barreteau

(2003). The joint use of role-playing games and models regarding negotiation process: Characterization of associations. Journal of Artificial Societies and Social Simulation, 6(2). Retrieved from http://jasss.soc.surrey.ac.uk/6/2/3.html

Bespolka

C. G.

(1989). A multi attribute trade off and analysis framework for electric utility integrated resource planning (Energy Laboratory Report No. MIT-EL 89-005). Cambridge: Massachusetts Institute of Technology.

Boissau

Castella

J. C.

(2003). Constructing a common representation of local institutions and land use systems through simulation-gaming and multiagent modeling in rural areas of Northern Vietnam: The SAMBA-Week methodology. Simulation & Gaming: An Interdisciplinary Journal, 34, 342-357.

Brewer

(1986). Methods for synthesis: Policy exercises. In Clark

Munn

(Eds.), Sustainable development of the biosphere (pp. 455-475). Cambridge, UK: Cambridge University Press.

Camargo

M. E.

Jacobi

P. R.

Ducrot

(2007). Role-playing games for capacity building in water and land management: Some Brazilian experiences. Simulation & Gaming, 38, 472-493.

10.

Caplan

(1979). The two communities theory and knowledge utilization. American Behavioral Scientist, 22, 459-471.

11.

Comisión Ambiental Metropolitana. (2001, August). Inventario de Emisiones 1998 de la Zona Metropolitana del Valle México, Versión Preliminar [Emission inventory for Mexico City metropolitan area (1998). (Preliminary version)]. Mexico City, Mexico: Author.

12.

Connors

Dodder

Aoki

Mostashari

Bracamontes

Amano

. . . Roth

(2002). Preliminary assessment of long-term emissions reductions for reference cases incorporating alternative air quality program implementations and alternative urban transformations (Draft, Mexico City Project). Cambridge, MA: Integrated Program on Urban, Regional and Global Air Pollution, MIT.

13.

Crookall

Thorngate

(2009). Acting, knowing, learning, simulating, gaming. Simulation & Gaming: An Interdisciplinary Journal, 40, 8-26.

14.

Darido

G. B.

(2001). Managing conflicts between environment and mobility: The case of road-based transportation and air quality in Mexico City (MIT master’s thesis). MIT Integrated Program on Urban, Regional and Global Air Pollution (Report No. 23), Cambridge, MA.

15.

Dennis

R. L.

Byun

D. W.

Novak

J. H.

Galluppi

K. J.

Coats

Vouk

M. A.

(1996). The next generation of integrated air quality modeling: EPA’s models-3. Atmospheric Environment, 30, 1925-1938.

16.

Dionnet

Kuper

Hammani

Garin

(2008). Combining role-playing games and policy simulation exercises: An experience with Moroccan smallholder farmers. Simulation & Gaming: An Interdisciplinary Journal, 39, 498-514.

17.

Dolin

Susskind

(1992). A role for simulations in public policy disputes: The case of national energy policy. Simulation & Gaming: An Interdisciplinary Journal, 23, 20-44.

18.

Duke

Geurts

(2004). Policy games for strategic management: Pathways into the unknown. Amsterdam, The Netherlands: Dutch University Press.

19.

Dunn

(1980). The two communities metaphor and models of knowledge use. Knowledge: Creation Diffusion and Utilization, 3, 293-326.

20.

Eliassen

Saltbones

(1983). Modeling of long-range transport of sulphur over Europe: A two-year model run and some model experiments. Atmospheric Environment, 17, 1457-1473.

21.

Etienne

(2003). SYLVOPAST: A multiple target role-playing game to assess negotiation processes in sylvopastoral management planning. Journal of Artificial Societies and Social Simulation, 6(2). Retrieved from http://jasss.soc.surrey.ac.uk/6/2/5.html

22.

Fiorino

D. J.

(2001). Environmental policy as learning: A new view of an old landscape. Public Administration Review, 61, 322-334.

23.

Funtowicz

S. O.

(2006). Why knowledge assessment? In Pereira

A. G.

Vaz

S. G.

Tognetti

(Eds.), Interfaces between science and society (pp. 137-145). Sheffield, UK: Greenleaf Publishing.

24.

Funtowicz

S. O.

Ravetz

J. R.

(1992). Three types of risk assessment and the emergence of post-normal science. In Krimsky

Golding

(Eds.), Social theories or risk (pp. 251-274). Westport, CT: Greenwood.

25.

Funtowicz

S. O.

Strand

(2007). Models of science and policy. In Traavik

Lim

L. C.

(Eds.), Biosafety first: Holistic approaches to risk and uncertainty in genetic engineering and genetically modified organisms (pp. 263-278). Trondheim, Norway: Tapir Academic Press.

26.

Gurung

T. R.

Bousquet

Trébuil

(2006). Companion modeling, conflict resolution, and institution building: Sharing irrigation water in the Lingmuteychu Watershed, Bhutan. Ecology & Society, 11(2). Retrieved from http://www.ecologyandsociety.org/vol11/iss2/art36/

27.

Heymann

(2004). Air pollution control: Who are the experts? In Kurz-Milcke

Gigerenzer

(Eds.), Experts in science and society (pp. 159-177). Boston, MA: Kluwer Academic Publishers.

28.

Innes

J. E.

Booher

D. E.

(1999). Consensus building as role playing and bricolage: Toward a theory of collaborative planning. Journal of the American Planning Association, 65, 9-26.

29.

Jäger

(1998). Current thinking on using scientific finding in environmental policymaking. Environmental Modeling & Assessment, 3, 143-153.

30.

Jones

C. O.

(1975). Clean air: The policies and politics of pollution control. Pittsburgh, PA: University of Pittsburgh Press.

31.

Karl

H. A.

Susskind

L. E.

Wallace

K. H.

(2007). A dialogue, not a diatribe: Effective integration of science and policy through joint fact finding. Environment, 49(1), 20-34.

32.

Kates

R. W.

Clark

W. C.

Corell

Hall

J. M.

Jaeger

C. C.

Lowe

. . . Svedin

(2001). Sustainability science. Science, 292, 641-642.

33.

Kim

D. Y.

(2007). The challenges of consensus building in a consolidating democracy: Diesel vehicles and urban air pollution in Korea. Wiesbaden, Germany: VDM Verlag Dr. Mueller e.K.

34.

Krebs

D. J.

(1999). A tale of two cities: The regulation of particulate air pollution in Mexico City and Los Angeles (MIT master’s thesis). MIT Integrated Program on Urban, Regional and Global Air Pollution (Report No. 1), Cambridge, MA.

35.

Kuipers

(1983). The role of a game-simulation in a project of change—A case for deep-sea fishing. Simulation & Gaming: An Interdisciplinary Journal, 14, 275-296.

36.

Lax

D. A.

Sebenius

J. K.

(2006). 3D negotiation: Powerful tools to change the game in your most important deals. Boston, MA: Harvard Business School Press.

37.

Levy

M. A.

(1993). European acid rain: The power of tote-board diplomacy. In Keohane

R. O.

Hass

P. M.

Levy

M. A.

(Eds.), Institutions for the Earth: Sources of effective international environmental protection (pp. 75-132). Cambridge, MA: MIT Press.

38.

Mathevet

Le Page

Etienne

Lefebvre

Poulin

Gigot

. . . Mauchamp

(2007). BUTORSTAR: A role-playing game for collective awareness of wise reedbed use. Simulation & Gaming: An Interdisciplinary Journal, 38, 233-262.

39.

Mayer

(2008). Gaming for policy analysis. Learning about complex multi-actor systems. In de Caluwe

Hofstede

G. J.

Peters

(Eds.), Why do games work? In search of the active substance (pp. 31-40). Dordrecht, The Netherlands: Kluwer.

40.

Mayer

(2009). The gaming of policy and the politics of gaming: A review. Simulation & Gaming: An Interdisciplinary Journal, 40, 825-862.

41.

Mermet

(1993). A prospective method: Policy exercises. Nature, Sciences, Societies, 1, 34-46.

42.

Molina

L. T.

Molina

M. J.

(Eds.). (2002). Air quality in the Mexico megacity: An integrated assessment. Dordrecht, The Netherlands: Kluwer Academic Publishers.

43.

Molina

M. J.

Molina

L. T.

(2004). Improving air quality in megacities: Mexico City case study. Annals of the New York Academy of Sciences, 1023, 142-158.

44.

Morris

A. P.

Yandle

Dorchak

(2004, April). Choosing how to regulate (Case Research Paper Series in Legal Studies, Working Paper 04–3). Cleveland, OH: Case Western Reserve University.

45.

Nowotny

Scott

Gibbons

(2001). Re-thinking science: Knowledge and the public in an age of uncertainty. Cambridge, UK: Polity Press.

46.

Ozawa

C. P.

Susskind

L. E.

(1985). Mediating science-intensive policy disputes. Journal of Policy Analysis and Management, 5, 23-39.

47.

Parson

E. A.

(1997). Informing global environmental policy-making: A plea for new methods of assessment and synthesis. Environmental Modeling & Assessment, 2, 267-279.

48.

Portney

P. R.

(1990). Air pollution policy. In Portney

P. R.

(Ed.), Public policies for environmental protection. Washington, DC: Resources for the Future.

49.

Schöpp

Amann

Cofala

Heyes

Klimont

(1998). Integrated assessment of European air pollution emission control strategies. Environmental Modeling & Software, 14, 1-9.

50.

Schwartz

(1991). The art of the long view. New York, NY: Doubleday.

51.

Secretaría del Medio Ambiente. (2010). Calidad del aire en la Ciudad de México: Informe 2010 [Mexico City air quality: Information (2010)]. Retrieved from http://www.sma.df.gob.mx/sma/links/download/biblioteca/flippingbooks/informe_anual_calidad_aire_2010/

52.

Sistema de Transporte Colective del Gobierno del Distrito Federal. (1999). Plan de Empresa 2000-2006 [Business plan (2000-2006)]. México City, Mexico: Author.

53.

Smog City 2, U.S. Environmental Protection Agency and Sacramento Air Quality Management District. (n.d.). Available from http://www.smogcity2.org/index.html

54.

Susskind

L. E.

(2008). Arguing, bargaining, and getting agreement. In Moran

Rein

Goodin

R. E.

(Eds.), The Oxford handbook of public policy (pp. 269-295). Oxford, UK: Oxford University Press.

55.

Susskind

L. E.

Corburn

(2000). Using simulations to teach negotiation: Pedagogical theory and practice. In Wheeler

(Ed.), Teaching negotiation: Ideas and innovations. Cambridge, MA: Program on Negotiation at Harvard Law School.

56.

Susskind

L. E.

Jain

R. K.

Martyniuk

A. O.

(2001). Better environmental policy studies: How to design and conduct more effective analyses. Washington, DC: Island Press.

57.

Tollefson

(2010). Environment: Mexico’s scientist in chief. Nature, 467, 902-905.

58.

Toth

F. L.

(1988). Policy exercises: Objectives and design elements. Simulation & Gaming: An Interdisciplinary Journal, 19, 235-255.

59.

Toth

F. L.

(2001). Participatory integrated assessment methods: An assessment of their usefulness to the European Environmental Agency (Technical Report 64). Copenhagen, Denmark: European Environment Agency.

60.

van Daalen

C. E.

Thissen

W. A. H.

Berk

M. M.

(1998). Experiences with a dialogue process between policy makers and global modelers (Report No. 410 200 017). Delft, The Netherlands: Delft University of Technology.

61.

Weber

E. P.

Khademian

A. M.

(1997). From agitation to collaboration: Clearing the air through negotiation. Public Administration Review, 57, 396-410.

62.

Wolf

L. P.

Laessig

R. E.

(1973). ERG-Energy Resources Game: Simulation gaming of regional energy. Simulation & Games, 4, 315-323.

63.

Wynne

(1992). Uncertainty and environmental learning: Reconceiving science and policy in the preventive paradigm. Global Environmental Change, 2, 111-127.