Evaluating Networks for Collaborative Planning and Management

Relative Performance of ACTs

To assess the performance of ACTs, we considered both network goals and intermediate outcomes (see Table 1). We collected 178 surveys from individuals involved in ten different ACTs, but when we compared the mean responses grouped by each ACT, we found few significant differences between them. This is due the variability of the responses and the relatively small numbers of responses associated with each ACT (n = 12–24).

The one exception was the Cascades West ACT. An analysis of variance revealed higher scores for four variables (authority, resources, communication, and influence on the OTC) that were significantly different from one or more ACTs. Our analysis of surveys and interviews did not reveal obvious reasons for the higher evaluations, but several respondents noted “excellent leadership” and we speculate that geography may be another explanation. The Cascades West ACT is one of the smaller ACTs and encompasses the state capital of Salem where ACT members may have more opportunities to interact with state agencies and the OTC. Due to the lack of significant differences between ACTs, we focus on the overall performance of the statewide ACT network.

Network Structure

To assess network structure, we assessed issues such as roles, scope, support, and authority (see Table 1). Survey respondents were positive about many aspects of the ACT structure (see Table 2), but three factors revealed more dissatisfaction. Respondents evaluated the financial resources lower than other variables. However, interviews and written comments suggested that respondents were focused on the availability of resources for transportation generally rather than resources to support ACT operation.

There were also lower evaluations of the ACT scope and its authority, and the written comments and interviews revealed confusion about their authority. The OTC and ODOT use the ACT network to set funding priorities. ACTs were established as advisory bodies, but some participants appeared to believe that ACTs were empowered with decision-making authority—or believed that should be given that authority. For example, when asked about weaknesses, one respondent noted a “lack of authority,” and another stated “Not really sure what our role is. Advisory, suggestions, concurrence?” Interviewees noted that this came to light during statewide prioritization of projects. ODOT is organized into regional districts and there are several ACTs in each ODOT district. ACT recommendations are assembled and prioritized at both the district level and the state level. Therefore, while some ACT participants assumed that a project rated as a high priority by the ACT would receive funding, this was not always the case as the projects were amalgamated and ranked at the district and statewide level.

When asked about representation, almost 70 percent of respondents indicated that the appropriate interests were involved (n = 139). The OTC gave ACTs considerable flexibility to choose participants, allowing each one to tailor their structure to their regional circumstances. This flexibility has helped engender strong support for the ACT structure, but some of the representation approaches created complications that are discussed below.

In terms of resources and staffing, it was clear from interviews and surveys that ODOT provided consistent and substantial support for the ACTs. They have operated for over ten years, and ACT participants were very complimentary of ODOT staff in their survey responses and comments (see Table 2).

Network Process

To evaluate process, we considered factors such as facilitation, communication, participation, and decision-making (see Table 1). Facilitation, leadership, and clear membership roles all garnered very positive evaluations from participants (see Table 2). Respondents agreed that ACTs are well facilitated, have open communication, and participants felt heard and understood by one another. They gave ACTs positive evaluations for having clear decision-making processes and resolving conflicts effectively. For example, one ACT created an “equity resolution” and another agreed to “take turns” on large project proposals to help resolve funding conflicts. Among ACT strengths, respondents commonly cited their “collaborative approaches,” “hard work to build consensus,” and “willingness to compromise the needs of one jurisdiction for the welfare of another.” ACTs also received high marks from participants for clarity and flexibility. Respondents were less positive about the innovation of ACTs, with only 49 percent giving this a favorable assessment (see Table 2). Comments on this often related to the limited focus and role of ACTs relative to “other decision-making processes for the rest of the [transportation] system.”

There were higher levels of dissatisfaction with decision-making authority. As noted above, some participants were confused about ACT decisions relative to regional and state priority setting. Several ACTs also experienced difficulty representing the large number of local governments in their region, which is discussed in more detail below.

Interim Outcomes

In evaluating collaborative groups, factors such as influence, products, commitment, and improvements in communication, understanding, and coordination are all important interim outcomes for assessing effectiveness (see Table 1). Overall, respondents were very positive about communication and understanding, as well as commitment to the group (see Table 3). ACT respondents gave moderately positive ratings to the innovativeness of ACT products, which the interviewees attributed to the constraints of the project prioritization process and the limited opportunities to consider more wide-ranging concerns.

Network effectiveness is also evaluated by its defined roles of prioritizing transportation investment, regional coordination, and participation (OTC 2003). A primary ACT responsibility is to make recommendations on funding allocations for the STIP. Over 80 percent of respondents indicated that the ACT role in prioritization was effective or very effective and only 1 percent gave this role a negative evaluation (see Table 4). For seven ACTs, STIP prioritization was on the agenda for 75 to 100 percent of their meetings, while two ACTs covered this topic at 50 percent of their meetings and one ACT at 44 percent of their meetings. In interviews and focus groups, participants noted that this role also consumed the vast majority of their time and many interviewees indicated that ACTs were too focused on this role. A few interviewees with long involvement in transportation planning indicated that ACT funding allocations were a significant improvement compared to the pre-ACT approaches. The respondents were less positive about their advisory role in terms of ACT influence on the OTC and ODOT.

ACTs were also charged with coordinating regional transportation issues. Sixty-nine percent of respondents agreed that ACTs had improved transportation coordination (n = 137), but the assessment of coordination with some organizations was mixed. Although all ACTs with an MPO had an MPO representative at 100 percent of their meetings, over 20 percent were neutral on their coordination effectiveness. For most ACTs, there has been limited information exchange or joint projects with MPOs, except for project prioritization discussions. Improved communication and coordination between ACTs and MPOs was also one of the most commonly cited needs.

The third broad role of ACTs is to create a forum for public and stakeholder involvement in prioritizing transportation investment. The interviews and surveys indicated that this was the lowest rated of the defined roles (see Table 2). The committees ranged between 17 and 24 voting members, with the most representatives coming from cities, counties, ODOT, transit districts, and ports. Most ACTs also involve nonvoting members, including state agency staff, representatives from neighboring ACTs, legislators, local officials, and representatives from other states. Many ACT participants cited elected officials as the primary avenue for public input, and we found that only 15 percent of the meetings over a two-year period involved any public comment. A typical response noted by one respondent was that the “general public is clueless about the jobs that ACTs perform.”

The Shared Goal Weakness of Centralized Networks

The literature on collaborative networks emphasizes the importance of shared goals and interdependence created by participants who recognize they can accomplish activities collectively that they could not do on their own (Emerson, Nabatchi, and Balogh 2012; Innes and Booher 2009; Margerum 2011). Markovic (2017) suggests that networks will be more effective when they are integrated through a central core agency, but our findings highlight that this integration was limited.

While ACTs were created to serve a broad regional coordination role, they were defined by the authority shared by the OTC. The review of meeting agendas and feedback from participants all emphasized that the majority of activities centered on prioritizing ODOT transportation funding (Table 2). Participants were committed to ACTs because they had the opportunity to influence these important decisions, but they were not committed to the broader goal of collaborative transportation governance in their region.

Broader communication and coordination roles received far less attention and lower assessments of effectiveness even though our interviews and open-ended survey responses highlighted this as a need. Many participants did not envision ACTs as the means to achieve broader integration, because they viewed ACTs as a “creature of the OTC.” This broader role would require commitments from other agencies and participants to the collective mission of ACTs (Keast, Brown, and Mandell 2007; Provan and Kenis 2008). Therefore, we conclude that a broader coordination mandate is destined to underperform in a centralized network such as the ACTs, unless there is a broader (and probably more formal) joint decision-making role defined by the participating organizations.

Multilevel Networks Can Complicate Authority

Network authority is defined by the shared goals and scope, but it is also defined by the joint actions performed by the group (Emerson, Nabatchi, and Balogh 2012; Huxham and Vangen 2000; Keast, Brown, and Mandell 2007; Voets, Dooren, and Rynck 2008). This authority may be specified through formal agreement or emerge as informal norms, but it is defined by the issues offered to the group for consent and the extent to which the authority of group is binding on participants (Huxham and Vangen 2000; Voets, Van Dooren, and De Rynck 2008).

ACTs experienced confusion about their authority relative to ODOT and the OTC. Participants gave high marks to the prioritization role, but a much lower evaluation of ACT influence. ACTs were allocated authority by the OTC to make prioritization recommendations, but as part of a multilevel network, these recommendations had to be amalgamated at the regional and state level. The OTC was effectively offering ACTs the authority to recommend priorities on project funding. Some ACT members perceived their prioritizations to be final and did not fully understand how their recommendations were merged in each ODOT region and across the state. The literature distinguishes this shared influence as distinct from shared authority or joint decision-making (Emerson, Nabatchi, and Balogh 2012; Huxham and Vangen 2005). As Koontz et al. (2004) point out, government decision makes often retain control, but they can share power by integrating the input of stakeholders and operating as participants rather than dominant players.

The concerns about the amalgamation of ACT decisions at the regional basis led one regional ODOT manager to share additional authority. The Region 2 manager conducted “All Area” meetings that consisted of representatives from three ACTs and Lane County. All of the representatives then discussed these priorities at the meeting in an effort to reach consensus on Region 2 priorities. This multilevel structure is comparable to Goldstein and Butler’s (2010) analysis of the multilevel fire learning network, but with some key differences.

The all area meetings decreased some of the confusion about how ACT decisions were resolved within an ODOT region and highlighted the competition for projects. Consis-tent with Goldstein and Butler’s (2010) findings about sharing innovation, ACT found that sharing information with other ACTs helped participants understand how state prioritization and improved ACT-level processes. Implemented statewide, this approach could help reduce some of the dissatisfaction about decision-making and improve prioritization. Interviewees and survey respondents recommended a statewide meeting of ACTs to further improve OTC-ACT communication and cross-ACT information exchange.

However, unlike the fire learning networks studied by Goldstein and Butler (2010), ACT participants noted that the Region 2 process was “competitive rather than collaborative” because there was a fixed amount of limited resources. In other words, the primary issue being resolved was the distribution of a fixed sum of funding, which is well established in the literature as creating fewer opportunities for collaborative decision-making (Carpenter and Kennedy 1988; Hanemann and Dyckman 2009; O’Leary and Bingham 2007; Susskind and Cruikshank 1987).

Information Rules Can Define the Scope of Work

The literature on collaboration and networks emphasizes the importance of a deliberative process using high-quality information, open communication, and an exchange of creative and innovative ideas (Ansell and Gash 2008; Beierle 2002; Deyle and Wiedenman 2014). An institutional analysis perspective analyzes this interaction in terms of defined rules for information exchange (Imperial 2005; Ostrom 2005). In a network these protocols define the timing, extent, and procedures for exchanging information, whether it be through committee processes, referral procedures, or shared databases (Emerson, Nabatchi, and Balogh 2012; Imperial 2005; Klijn, Steijn, and Edelenbos 2010; Ostrom 2005). These types of procedures are described as “the ground rules for behavior and interaction in the network” (Klijn, Steijn, and Edelenbos 2010, 1070).

We found that in defining the interaction of the group, these protocols for information exchange had a significant effect on the scope of network activities because they defined clear procedures and built expertise among participants. In both surveys and interviews, participants were far more positive about transportation project prioritization than other coordination roles.

The OTC and ODOT gave the ACTs information at specific points in time and requested specific outputs in the form of project prioritization recommendations. There was little confusion about information procedures and the prioritization activities were the single most dominant agenda item. Furthermore, participants indicated that the regular timelines and focus on these activities built up an expertise within the group.

This approach created a fairly narrow set of decisions even though ACTs had broad flexibility in their scope. Many other activities relating to the roles of OTC and ODOT were not fulfilled by the ACTs, because there were no specified information procedures and these roles became effectively squeezed out. As one participant noted, “our decisions are only about [project] prioritization.” Interviewees noted that information on topics such as ODOT policy were not shared regularly or systematically, and participants noted that they did not feel they knew enough about these other issues to fully participate. Similarly, one of the common weaknesses cited by survey respondents was a lack of understanding of the bigger picture, such as state policy, neighboring ACT issues, and regional corridors.

From a network analysis standpoint, this suggests that explicit rules defining information exchange and decision points are just as important as the quality of interaction and the scope of network authority. These rules ensure more stable interaction (Imperial 2005), and they improve the ability of participants to interact. Participants build up experience with a subject matter and they are able to anticipate the topics and issues that will arise. These norms and rules also affect the scope of collaborative action (Emerson, Nabatchi, and Balogh 2012; Imperial 2005), even if the formal scope is much broader. The list of nine primary activities defined by the OTC for the ACTs were the things it would have liked for them to do. The rules that defined information exchange and communication defined what the ACTs actually did.

Representation Challenges

One of the commonly accepted tenants of collaborative decision-making is the need to select stakeholders that are broadly representative and have sufficient authority in their organization (Deyle and Wiedenman 2014; Gray 1989; Linden 2010; Ysa, Sierra, and Esteve 2014). Our research revealed that ACTs included the appropriate parties, but ACTs were limited by the time of elected officials and regional representation was challenging. While representation challenges are often noted in the literature, we suggest that these challenges may limit the effectiveness of a collaboration network.

First, many of the key participants on the ACTs are elected officials, which provides important high level participation. However, when asked about expanding the roles of ACTs, both staff and elected officials listed “lack of time” for elected officials as the chief limiting factor. ACTs cover large, multicounty areas that require significant travel time for participants, particularly elected officials who also attend numerous local meetings and events. When asked whether there were other primary activities or roles in which ACTS should be engaged, only 22 percent said “yes” and interviewees noted, “ACTs have a full agenda, and that adding issues would strain ACT members [and staff].” Put succinctly, when asked about expanding ACT activities, one elected official responded, “I think we do plenty.”

Second, the ability of individuals to represent a region presents challenges. Representatives are asked to assemble a regional perspective or make judgments about a position that their organization or entity may not have addressed (Margerum 2011). There may also not be consensus on the issues being discussed, which places the burden for a representative to negotiate at the table and negotiate among the set of constituencies the individual is representing (Colosi 1983; Huxham et al. 2000).

This two-table problem can be challenging for those representing organized interests and even more complex when there is no formal group. For example, some elected officials appointed to an MPO were asked to represent the MPO perspective. However, survey comments and focus group discussions highlighted that elected officials tended to rely more on personal judgment or represent their own jurisdiction’s perspective rather than presenting the MPO perspective. In another example, the high number of local governments in urbanized regions led ACTs to created representation arrangements. Some of these ACTs grouped representation into subregions; others created representatives for transportation corridors; several ACTs relied on counties to represent the perspectives of several smaller towns and cities within their jurisdiction. In all of these arrangements, participants cited dissatisfaction and difficulties. Representatives had difficulty presenting the perspective of these organizations, particularly since there was no organized forum for consulting with them. This remains an under-addressed topic in the literature. Even researchers who raise this difficulty (Laws 1999) do not offer clear strategies beyond suggestions for how the representative can work to gather perspectives from various constituencies.

Opportunities to Adapt a Centralized Network

The findings about ACT structure, authority, and information rules might lead to the conclusion that these shortcomings are insurmountable. However, within the ACT structure, there was one experimental program that was widely lauded for generating shared goals, joint collaboration, and innovation. In response to requests for transportation funding for a broader set of objectives, the OTC created the ConnectOregon program. This program provided new funding through a competitive, joint funding proposal process.

This program had several key features that distinguished it from its other primary activities. ODOT created a clear timeline and information procedures that allowed each ACT to initiate proposals or create a forum for discussing proposals. The scope of funding provided a much more open set of criteria that encouraged multiple objectives (transportation and economic development) and offered new opportunities for collaboration with public and private sector organizations. This contrasted with project prioritization, which is confined to a limited set of transportation projects and a relatively limited set of evaluation criteria. The program also allowed the ACTs to serve as a forum for new participants to come together to propose projects that met common goals and induced joint commitment of funding or action. ConnectOregon funded port upgrades, bike share programs, scenic railroad improvements, and freight rail infrastructure. The program also induced two ACTs to develop a “super ACT” proposal. ConnectOregon offered new funding that had not been available previously for projects that were often difficult to fund, allowed more creative proposals, and induced more joint action.

The results of the ConnectOregon program suggest that even within an organization-centered network, there may be opportunities to create programs and procedures that allow for stronger goal commitment and more creative forms of interaction. For the OTC, this also offers clues to how they could create new opportunities for ACTs. Many participants called for more transportation and land use coordination. A ConnectOregon type of program delivered with the state land use agency could be a pathway to address this growing need.