Toward a Watershed- and System of Systems–Oriented Perspective of Stormwater Management Enterprise Performance

Abstract

Public works decision and policy makers daily face myriad management problems. The problem of stormwater runoff management, for one, has for several decades received a great deal of attention, with much of that focused on reconciling the needs of natural watershed systems with those of the man-made physical and non-physicalsocietal systems that watersheds overlap. In other words, stormwater management has effectively been cast as a system of systems problem requiring a delicate balance among multiple natural and man-made systems. The authors propose a methodology with which stakeholders can first set and then realize expectations for stormwater management problems by using watershed- and system of systems–oriented perspectives. The methodology, Enterprise AID (assessment, improvement, and design) is particularly well suited to such a pairing of perspectives, and this article, therefore, shows how stakeholders in multiple and commonly disparate interests might best balance the stormwater management needs of watersheds and related societal constructs.

Keywords

performance measurement and management performance measurement and management system (PMMS)watersheds system of systems stormwater management

Introduction

The word enterprise typically prompts thoughts regarding corporations, executive boardrooms, and other representations of large organizations. Rouse (2005), for example, has offered one oft-repeated definition of enterprise: “An enterprise is a goal-directed organization of resources—human, information, financial, and physical—and activities, usually of significant operational scope, complication, risk, and duration” (p. 139). That definition’s emphasis on “significant operational scope . . . and duration” well aligns with the popular picture of enterprises as organizations, as does a following statement like “enterprises can range from corporations, to supply chains, to markets, to governments, to economies” (p. 139). However, this article’s authors believe it useful for public works management purposes to define enterprises much more broadly than is commonly done. Boardrooms are well and good, but rather than ranging from corporations through economies, the authors suggest that enterprises be seen to range “from barbeques to boardrooms.” This broader characterization is one better suited to everyman pursuits, such as barbeques, and holding an authority that can only be granted by the everyman source of definitions, the dictionary: an enterprise is “a project or undertaking that is especially difficult, complicated, or risky” (Mish, 2009, p. 416). Relative terms such as difficult, complicated, and risky allow for applications running the gamut from barbeques to boardrooms. Can the many strategic decisions demanded of Fortune 500 corporate executives be difficult, complicated, or risky? Surely, they can and are, as a matter of course. Can the backyard barbeques undertaken by millions of households worldwide on any given day be just as hazardous? Yes, they can, and perhaps they represent an all-too-common occurrence as suggested by U.S. Consumer Product Safety Commission (CPSC; 2008) records of fire damage to homes and personal injuries commonly suffered by persons attempting to deep fry the turkey dinners traditional for Americans’ annual Thanksgiving Day holiday. Most significant about the dictionary definition is its treatment of “project or undertaking” as central to the notion of enterprise. Large and small organizations can certainly represent the enterprises, or projects or undertakings, that they pursue; however, those organizations need not be viewed as “enterprises” per se, and certainly not as the only types. In the grand scheme of things, individuals and groups of individuals are always pursuing some enterprise, and individuals tasked with public works management and policy efforts are no exception; it stands to reason that those pursuits demand measurement and an accompanying management effort. The aim of this article is to provide individuals in the public works management and policy arena a new approach with which to undertake performance measurement and management efforts aimed at evaluating and improving their enterprises. Following a brief history of means for measuring and managing enterprise performance, this article’s following sections will therefore demonstrate the author-developed Enterprise AID’s utility in addressing and resolving a specific public works problem, stormwater management. It is with this in mind that we next address the means with which performance measurement and management systems (PMMSs) might be determined for all manner of enterprise.

Enterprise PMMS Development Approaches

Prior to roughly 1990, organizational performance was measured and managed using “backward looking accounting based” (Bourne, Wilcox, Neely, & Platts, 2000, p. 754) perspectives, other financial gauges (Bean, 1995; Bodner & Rouse, 2007; Hultink & Robben, 1995; Kaplan & Norton, 1992; Tangen, 2004; Wilcox & Bourne, 2003), or with respect to indicators such as the quantity of new products developed, patents granted, or like evaluations (W. B. Brown & Gobeli, 1992; M. G. Brown & Svenson, 1988; Graves, Ringuest, & Case, 2000) defensibly characterized more as organizational attributes than as reflections of organization stakeholders’ desires presumably behind the attributes. The 1990s saw calls for PMMSs incorporating complementary concern for measures like customer satisfaction (Eccles, 1991), and those calls were met with popular PMMS design and deployment approaches such as Balanced Scorecard (Kaplan & Norton, 1992), Performance Prism (Neely, Adams, & Crowe, 2001; Neely, Adams, & Kennerley, 2002), and numerous others (Taticchi, Balachandran, & Tonelli, 2012).

These PMMS approaches, however, have proven to offer less than hoped to the organizations they were intended to support. Neely (1999) draws from numerous sources in noting traditional performance measures and measurement systems to be historically rather than strategically focused (Skinner, 1974) or predictive (Dixon, Nanni, & Vollmann, 1990), as encouraging “short termism” (Banks & Wheelwright, 1979; Hayes & Abernathy, 1980), as promoting “local optimisation” (Goldratt & Cox, 1986; Hall, 1983) at the expense of “what customers [really] want” (Camp, 1989; Kaplan & Norton, 1992), and as too often prompting “comments such as ‘we measure everything that walks and moves, but nothing that matters’” (p. 206). Furthermore, the vast majority of even the past two decades’ approaches seem to have featured a rather formal sense of “organization” at the expense of “enterprise” as we have described it. Few, if any, have emphasized the latter in its broadest sense, generating a gap for a PMMS design and deployment approach to the risky undertakings that barbeques can represent every bit as well as responses to boardroom decisions. Enterprise AID—or simply AID, for assessment, improvement, and design—is a methodology designed to avoid such issues by drawing from systems science, operational test and evaluation, and multicriteria decision analysis disciplines to allow for assessments or forecasts of extant or envisioned enterprise performance in the face of some specified problem.

AID is by design problem-centric and, equivalently, focused on a “project or undertaking” (Mish, 2009) of “risk” generated in response to some problem (p. 416). It neither presumes nor depends on the existence of a formal organization, most certainly not one of any standing characteristics. Instead, it straightforwardly arms stakeholders with means to set and realize expectations for any specific enterprise faced with its own set of specific problems; and it does so using one or two facilitators who support deliberations and decisions of typically three to five subject matter experts (SMEs) pre-selected for their knowledge of stakeholder aims for some problem of interest. The SMEs observe their stakeholder-given charge in large part by leveraging the following definitions and implied relationships:

Problem. An issue that enterprise stakeholders wish to see resolved.

Problem Essentials (PEs). A synthesis of stakeholder expectations related to some problem and that must each be realized for problem resolution.

Problem Measures (PMs). Scales of goodness (Dockery, 1986) derived by stakeholders from PEs and with which stakeholders can determine contributions made to expectations by solutions proposed to realize them.

Solution Measures (SMs). Scales of performance characterizing the contributions made or to be made toward expectations by proposed solutions’ key attributes.

Enterprise Utility. A relative measure of value assigned to a particular enterprise configuration of key attributes, meaningful only with respect to other configurations considered for the same enterprise and problem.

Figures 1 and 2 display relationships among these and what are AID’s two phases of “set . . . (and) realize stakeholder expectations.” The Figure 2 Enterprise AID application flowchart portrays the methodology in terms of its two phases and 12 subordinate steps, including the Enterprise Strategy Framework (ESF) developed as a by-product of Phase 1’s 4 steps 1a through 1d. Although provided at this juncture with little explanation, the Figures 1 and 2 portrayals will become more understandable with this article’s next three sections.

Figure 1.

Enterprise AID’s major elements.

Figure 2.

Enterprise AID’s application flowchart.

Enterprise AID is not only indifferent to whatever might be the organizational conventions under which it is applied, it is also equally robust to applications within single system settings as uncomplicated as backyard barbeques or as complex as a watershed’s stormwater management or other public works enterprise spread across multiple jurisdictions in system of systems–type environments (Meyers & Hester, 2013). Moreover, Sage and Cuppan’s (2001) Table 1 descriptions strongly suggest that system of systems–type problems should be addressed only with methodological approaches that, like AID, can truly promote such problems’ resolution. When coupled with what the literature reveals as advantages held by two of AID’s three primary conceptual components—systems science (McGarity, 2013) and multicriteria decision analysis (Lai, Lundie, & Ashbolt, 2008)—for stormwater management, a strong case can be made for the methodology’s usefulness in resolving problems prominently characterized in terms of watershed and system of systems perspectives. To serve that suggestion, we next offer a notional example aimed more at the barbeque than boardroom end of AID’s application spectrum, and we follow that with a more complex example directly focused on this article’s watershed and system of systems perspectives theme.

Table 1.

System of Systems Distinguishing Characteristics (Sage & Cuppan, 2001, p. 326).

Characteristic	Description
Operational Independence	A system of systems is composed of systems that are independent and useful in their own right. If a system of systems is disassembled into its component systems, these component systems are capable of independently performing useful operations.
Managerial Independence	The component systems not only can operate independently, they generally do operate independently to achieve an intended purpose. The component systems are typically individually acquired and integrated, and they maintain a continuing operational existence independent of the system of systems.
Geographic Distribution	Geographic dispersion of component systems can be large. They can often readily exchange only information and knowledge with one another, not substantial quantities of physical mass or energy.
Emergent Behavior	A system of systems performs functions and carries out purposes that do not reside in any component system. This behavior represents emergent properties of the entire system of systems and not the behavior of any component system. Purposes that prompt the engineering of systems of systems are fulfilled by such emergent behavior.
Evolutionary Development	A system of systems is never fully formed or complete. It evolves over time, with structure, function, and purpose added, removed, or modified over time.

A Notional Application

Enterprise AID’s applicability across a spectrum of “barbeques to boardrooms” prompts a barbeque-related example exploring a limited number of the methodology’s key application features. Precisely, the example will develop a notional ESF that could support individuals hopeful of tasty Thanksgiving Day turkeys but not wishing to burn down their homes!

In 2011, actor William Shatner teamed with U.S. insurance company, State Farm® to produce a public service announcement (PSA) video (YouTube.com, 2011) humorously describing a too personal history with outdoor gas-fueled turkey fryers and how others might avoid the same. It seems that Captain Kirk of television and movie Star Trek fame had finally met an adversary he could not dispatch with the same swiftness he had routinely demonstrated against Klingons and other foes from the final frontier. Indeed, the fryer might have sent him into cosmological history. “I am not immune to frying accidents,” Shatner has said, further noting that “Several years ago I was even burned on my arms after accidentally dropping the turkey in the hot oil. People need to remember that hot oil and turkey can be a dangerous combination” (StateFarm.com, 2011). Yes, they can, but the following abridged yet instructive example of Enterprise AID in use will show how the methodology can help fans of fried turkey or other barbeque-like delights pursue their chosen backyard enterprises in complete safety and with appetite-pleasing results.

Restricting for illustration purposes the description of Shatner’s clearly risky, turkey-frying undertaking to what Figures 1 and 2 show as an enterprise problem, PEs, PMs, and SMs can yield an ESF of the sort identified in Figure 3. Had Shatner developed and used such a framework before initiating his turkey-frying enterprise, that very powerful construct could have greatly depersonalized his too close encounter with the fryer. For that reason, Figure 3 draws from the State Farm PSA (2011) and related sources (CPSC, 2008; StateFarm.com, 2011; Underwriters Laboratories, n.d.) to show a notional but plausible ESF, while Table 2 describes certain of the framework’s major features.

Figure 3.

William Shatner’s turkey-frying ESF.

Table 2.

William Shatner’s Turkey-Frying Enterprise Strategy Framework (ESF).

ESF Element	Description
Problem	A succinct statement of the issue at hand and for which Shatner, as stakeholder and chief cook, will pursue the risky undertaking of deep frying a turkey he wishes to feed his family on Thanksgiving Day.
PEs	A synthesis of the cook’s expectations for his turkey-frying venture, with each PE expressed as a “yes” or “no” question so that Shatner can declare his expectations met if and only if the answer to both is truly “yes.”
PMs	Succinct expressions of the scales of goodness with which Captain Kirk and his stakeholding crew can determine the “yes” or “no” answers demanded by PEs. Here Shatner is reasoned to argue that a “pleasing” turkey is largely determined by “moistness,” “tastiness,” and “appearance”; and that a pleasing and safe frying enterprise will be one that gets a turkey to the table with no harm done to a person or thing.
SMs	The succinctly expressed attributes of Shatner’s intended frying operation are those he believes will most significantly influence the PMs to which his ESF links them. For example, in this case, he is shown to logically tie attributes such as oil temperature and fryer stability with some common (“risk of personal harm”) and some distinct (“tastiness” vs. “risk of harm to structures”) PMs.

Note. PEs = Problem Essentials; PMs = Problem Measures; SMs = Solution Measures.

The Enterprise AID features that suit it to this turkey fryer–based example also allow it to support more sophisticated enterprises such as that pursued by a particular stormwater management consortium. The following section therefore reviews stormwater management as a representative enterprise that can be used to explore AID’s use in resolving public works problems. Following a discussion of stormwater management are additional sections that describe how AID can first serve the design of appropriate PMMSs and, once designed, their deployment by those charged with stormwater management.

Stormwater Management Problems

Stormwater management is an ideal example of a public works enterprise critical to myriad interests, both formal and informal. According to Daigger (2009), the functions of stormwater management and its corollary activity of rainwater harvesting represent

a diverse set of natural and man-made processes generally intended to capture stormwater runoff and treat it for introduction to the environment or capture it for later use. These processes can slow stormwater flow to allow natural mechanisms to remove pollutants, reduce peak flow to moderate flooding, and direct portions of runoff into the groundwater supply or toward vegetation’s evapotranspiration processes that return it to the atmosphere. (p. 813; Strecker et al., 2005)

In everyday life, the notion of a watershed is plainly linked to and at least as prominent as that of stormwater management. Homeowners, for example, can correctly view their yards or even portions of them as watersheds, just as municipal or other societal constructs equivalently can regard as watersheds the natural and man-made domains within which they commonly exercise authority for stormwater management. Both the citizen- and society-centric viewpoints owe to a simple and widely accepted definition: a watershed is “an area of land that water flows across as it moves toward a common body of water” (Southwest Florida Management District, 2012, p. 1).

That watershed boundaries can and do encompass areas as large as the authors’ own Chesapeake Bay region, for example, guarantees that they will also routinely encompass multiple political jurisdictions with authority to address watershed-related and other societal needs. Therefore, watersheds comprise systems of natural and man-made physical systems—such as natural geographic contours or man-made culverts—almost always intertwined with multiple man-made but non-physical systems such as those for societal governance. A “system of systems” notion, then, joins that of “watershed” as an important aspect of stormwater management, and Sage and Biemer (2007) have proposed what may be among the former notion’s most widely accepted definitions:

(A system of systems) is a large-scale, complex system involving a combination of technologies, humans, and organizations, and consisting of components which are systems themselves, achieving a unique end-state by providing synergistic capability from its component systems, and exhibiting a majority of the following characteristics: operational and managerial independence, geographic distribution, emergent behavior, [and] evolutionary development . . . (p. 7)

Mother Nature, however, does not recognize the many civil constructs so tied to everyday life. For example, the flow of stormwater in any particular watershed traverses state, county, city, or any other jurisdictional boundaries on its way toward natural destinations such as lakes, rivers, wetlands, groundwater, or evapotranspiration; but in so doing, the stormwater’s course marks the overlapping systems of the stormwater management systems traversed. Would it not make sense, then, for those charged with stormwater management to execute their charge in a manner respecting the very systems of systems identified by Mother Nature for the task? Pertinent case literature strongly suggests that it does (Abel & Hennessey, 1997; Edwards, 2007; Grigg, 2013; Kellogg, 1997; Kraft, 2006; Pisano, Mazmanian, Little, Linder, & Perry, 2010; Water Environment Research Foundation, 2010), and the authors’ home state of Virginia provides an example adding to the case history.

Like other U.S. states, what is officially the Commonwealth of Virginia relies in varying degree on county, city, or similar entities for local governance. Again like other states, however, the Commonwealth also acknowledges that certain types of issues should be addressed using regional perspectives best able to accommodate complex issues that affect multiple governmental jurisdictions. Virginia codified this position in 1968 with its Regional Cooperation Act that established 21 planning districts chartered in broad terms to spearhead enterprises focused on multijurisdictional problems within their boundaries. One of these is the Middle Peninsula Planning District that exercises jurisdiction over a subregion of the Chesapeake Bay watershed. Among the roles encouraged for the district’s governing commission is one pertaining to “water supply and other environmental management” (Regional Cooperation Act, Virginia Code, 1968), a role for which has been de facto adopted a system of systems approach to stormwater problems. Just how this can be done is the focus of the remainder of this article.

Using Enterprise AID to Address Stormwater Management Problems

Stakeholders of stormwater management enterprises can use the Enterprise AID methodology to address myriad problems, whether those problems derive from system of systems–type settings or not. However, because system of systems–derived problems are so prevalent within stormwater management settings, the following two subsections will illustrate how a small group of stakeholder representatives—an AID application’s participating SMEs—can help stormwater management program stakeholders first set and then realize expectations for the system of systems–derived problems they do typically encounter. The methodology supports that sequence with two respective phases.

Phase 1: Setting Expectations for Stormwater Management Enterprise Performance

Figure 2 shows an AID application’s Phase 1 to generally set stakeholder expectations for some particular enterprise, or work effort. In accordance with the methodology’s protocol, the phase expressly enables development of what is termed an ESF suited to two purposes:

Once developed by a facilitated application’s participating SMEs, an ESF may be used by enterprise stakeholders as a stand-alone construct able to serve a broad set of needs typically linked to an also broadly defined problem. For example, stakeholders can pursue this first purpose to emplace a guidance-type standard, such as a strategic plan, for their general endeavors or when they simply may not be prepared for the detailed effort that would accompany AID’s Phase 2; or

An ESF may provide the springboard to AID’s Phase 2 and so more directly serve the methodology’s three principal functions of enterprise assessment, improvement, and design. Phase 2 extends Phase 1’s ESF product to produce a fully operational PMMS with which enterprise stakeholders can meet current or anticipated problems.

A setting like that established for the Middle Peninsula Planning District Commission (MPPDC) could catalyze either purpose, so following arguments address the second as much as allowed by the article’s scope. Readers will find that the nature of the Phase 1 activities better lends itself to description than does the set of generally more tedious Phase 2 proceedings, but all Phase 2 and other methodology application details deliberately ignored here may be found elsewhere (Hester & Meyers, 2012).

Step 1a: Define the problem

An earlier section defined a problem, for AID purposes, to be an issue that stakeholders wish to see resolved. The SMEs who drive AID applications must always begin their endeavor by asking “What problem are we trying to solve?” A bit more precisely, they must “Define and generalize the needs, . . . Consider the total problem environment” (Haimes & Schneiter, 1996, p. 484), and “Clearly identify the problem” (Haimes, 2004, p. 8). Application participants must also formally define their problems of interest in accordance with AID protocols. MPPDC SMEs, for example, might plausibly define the following problem statement in accordance with those protocols: “MPPDC requires a stormwater management program balancing its watershed and societal needs.” That succinct assessment does, indeed, respect AID’s definition of a problem. The MPPDC concedes that it currently has no stormwater management program able to balance, for the common good, the needs of a healthy local watershed against other needs pressing on the constituents and societal constructs represented by the district. The lack of such a program represents an issue to be resolved. Defining a problem as shown in Figure 4 can be a chore much more difficult than meets the eye; and so, although fully describing how an AID application can promote problem definition would reach beyond the scope of this article, suffice to say that the methodology’s Step 1a exists to acknowledge the primacy of defining problems in terms agreeable to an application’s participants.

Figure 4.

MPPDC stormwater management program problem essential identification.

Step 1b: Identify PEs

Problem definition is certainly key to eventual problem resolution, but each of AID’s assessment, improvement, and design purposes demands that an agreed-upon problem definition be reconfigured as some number of what the methodology formally terms Problem Essentials, or PEs. PEs are stakeholder-identified elements of problems that may be considered resolved only if each of their constituent PEs is resolved. Under Enterprise AID protocols, then, PEs are always posed as “yes” or “no” type questions that must all be answered as “yes” before stakeholders may legitimately claim that a specific enterprise undertaken to resolve a specific problem has succeeded in doing so. By way of illustration, Figure 4 continues the MPPDC example with the posited stormwater management program problem and a set of PEs that SMEs could logically determine as a necessary and sufficient representation of the problem.

Note that SMEs will have here determined, perhaps with the help of their effort’s facilitators, that the earlier-defined problem be reconfigured and equivalently expressed in terms of three PEs, each couched as a question that the experts hope an enterprise of stormwater program development will eventually allow them to answer with a “yes.” Recalling, too, that the acronym AID represents the purposes of enterprise, “assessment,” “improvement,” and “design” that the methodology can support, it should be clear that all PEs must reflect the same purpose as that of the problem from which they are identified. For the MPPDC example, that purpose is to “design” a system, precisely a PMMS, able to prompt and gauge progress toward implementing a stormwater management program of qualities to be expressed with a completed ESF.

Step 1c: Derive PMs

PMs are the scales of goodness (Dockery, 1986) that an Enterprise AID application’s participants derive from PEs and with which SMEs can determine contributions to expectations by the enterprises intended to realize them. Numerical values that AID processes will eventually assign the PMs of particular applications will bear completely on stakeholder claims of resolution (or not) regarding respective problems of interest. Figure 5 therefore suggests a set of five PMs that a MPPDC application’s SMEs might derive from the set of three PEs they will have previously identified as minimally but wholly representing the stormwater management program development problem of this article’s continuing example.

Figure 5.

MPPDC stormwater management program problem measure derivation.

It should be noted that PMs and the PEs from which they derive represent emergent properties that will only be recognizable once a system for developing MPPDC’s stormwater management program has been made operational. That property of emergence evokes the Table 1 description of “emergent behavior” that Sage and Cuppan (2001, p. 326) linked to systems of systems, and it is quite appropriate when viewed from the perspective of expectations that MPPDC stormwater management program stakeholders would hope to realize. The connotations of terminologies such as “have we characterized,” “are stakeholders working,” “engagement,” “management,” and “support” therefore make perfect sense as part of a properly developed ESF.

Step 1d: Select SMs

Similar to the PM-to-PE relationships shown in Figure 5, the following figure displays what could be SME-assigned linkages among the five PMs and a total of six of what AID terms solution measures, or SMs. Unlike the links portrayed in Figure 5, the upward-pointing arrows of Figure 6 show SMs to be not at all derived from PMs or any other ESF element. They are instead selected strictly for what SMEs view as their bearing on certain of a framework’s previously accepted PMs. The Phase 2-related section that immediately follows will slightly expand on the evaluation of SM-to-PM relationships, just as it will on the evaluations of relationships among PMs and PEs.

Figure 6.

MPPDC stormwater management program solution measure selection.

Phase 2: Realizing Expectations for Stormwater Management Enterprise Performance

The first phase of an AID application involves what the approach counts as four successive but possibly also iterative steps toward development of an ESF. Phase 2 commences once that framework is established, and the second phase of activity is always strictly guided by the ESF construct, even if an original construct has been or continues to be altered in a manner allowed under the Figure 2 portrayal. Succeeding subsections highlight work to be performed and results to be had with each of what Figure 2 also portrays as Enterprise AID’s Steps 2a to 2h.

Step 2a: Assign PE threshold values

PE threshold values are necessary for the AID approach because they “set the bar” against which the “yes” or “no” judgments regarding PE, and hence problem resolution, can be made. Threshold values must be set as soon as possible during an application so that they represent truly unbiased needs and cannot deliberately or inadvertently be set to favor results of other evaluations that constitute an AID effort. Therefore, with Step 2a, Enterprise AID protocols solicit from an application’s SMEs their answers to the following pair of questions:

Do you grade this PE’s resolution as “likely” or not? and

Do you grade the consequence of this PE’s resolution to be on a par with other PEs, or do you judge it to be more “prominent” than that of other PEs nevertheless essential in their own right?

SME responses are then converted to numeric threshold values appropriate for later use with the Step 2f that will determine whether or not an application’s PEs have been satisfied. PEs calculated to hold values at least that of the SME-prescribed thresholds are considered to have been satisfied; and those ultimately assigned values less than their respective thresholds are considered unsatisfied and therefore open to SME deliberations possibly to occur under AID’s Step 2g.

Step 2b: Assign PM and SM weights

The ESF development that catalyzes a Phase 2 effort has participating SMEs derive from each PE some number of PMs they believe to logically and efficiently promote evaluations of the PEs from which they derive. It likewise has those experts link to every PM a set of SMs selected for their appropriateness in promoting the PM evaluations that AID also requires. However, to operationalize the ESF, it remains for Phase 2 participants to determine the strength of influence exerted by each PM on its parent PE and by each SM on the PM to which SMEs have linked it. Figure 2 shows Phase 2 participants to determine those respective influences—or equivalently, to assign collectively considered weights to all ESF PM-to-PE and SM-to-PM relationships—during an application’s Step 2b.

Participating SMEs determine PM-to-PE weighting using one of five linguistic descriptions of the influence they believe each PM to exert on resolution of the PE from which it was derived. The descriptive, linguistic terms from which SMEs select strength of PM-to-PE influences are as follows:

Essential

Significant

Moderate

Minimal

Insignificant

In a manner like that used for Step 2a’s linguistic-to-numeric value conversions, AID protocols convert the above terms to numeric values appropriate for the associated PM, a process yielding results later required for Step 2e. An application’s expert group next uses the same term set to characterize what it feels are the influences of SMs on the PMs with which the ESF associates them. While exceeding the scope of this particular document, the mathematical basis of influence coding as well as the threshold assignment process related in the previous section are elsewhere provided by this article’s authors (Hester & Meyers, 2012).

Step 2c: Assign SM value functions

Like the eggs, flour, or sugar of a cake mix, the SMs called for with Enterprise AID are the key ingredients of some problem’s hoped-for solution either already operating and being assessed for its success or proposed to resolve a current or even envisioned issue. An AID application’s SME group selects a set of qualitative or quantitative SMs that it believes will most satisfactorily represent a specific solution proposed for their problem of interest. Group members then characterize each selected SM in terms of the value they perceive it to offer the proposed problem solution, or enterprise. For example, if members of a SME group select a proposed environmental control system’s dehumidifying feature as one to consider with an assessment of the system’s ability to meet some building’s environmental control needs, they will next collectively characterize what AID protocols would define as a dehumidifying value function to be used with other significant features’ value functions to ascertain overall system worth to the building’s environmental control.

Under AID protocols, SMEs link SMs’ raw qualitative or quantitative characteristics of performance to value function parameters ranging from 0 to 100. For example, in serving a stormwater management program development–related application like that posited here, SMEs could define a value function for the SM, “Watershed complexity” by using qualitative terms such as minimally, moderately, and significantly through exceedingly complex to describe what they anticipate as the difficulty of characterizing the Middle Peninsula–area watershed in a way appropriate for stakeholders’ stormwater management program desires. They would next associate with their qualitative descriptor what they felt to be the appropriate quantitative value drawn from the integer scale of 0 to 100. SMs linked by SMEs to multiple PMs would bear on those PMs through a common value function. Step 2c concludes with SME consensus on each SM’s value function, a circumstance allowing commencement of Step 2d data collection processes.

Step 2d: Collect data

The data collection needs of any AID application can be satisfied in two ways. Data representing qualitative judgments of participating SMEs are normally elicited during application sessions involving the entire SME group and supporting facilitators. Largely quantitative data may be collected without involving the SME group but by facilitators or other application support persons who retrieve information suited to SMEs’ SM selections and value function assignments. For an effort like that of this section’s example, which incorporates both qualitative and quantitative SMs, SMEs and facilitators would then, as appropriate for data type, either generate and collect or simply collect requisite data before exiting Step 2d for Steps 2e through 2f.

Step 2e: Evaluate PMs

Figure 2 shows the Enterprise AID methodology to prescribe needs to “evaluate performance measures”—with “performance measures” representing all PEs, PMs, and SMs—and to compare values of the most important of them, the PEs, against established PE thresholds. Steps 2e through 2f can be executed in rapid succession, without facilitator or SME intervention, using software tools appropriate for methodology applications. Figure 7 shows a hypothetical evaluation of each of the MPPDC problem PEs, together with the respective contribution made by each SM to PE resolution.

Figure 7.

Stormwater management PMMS PE evaluation.

Step 2f: Compare PE values to thresholds

The AID methodology’s Step 2f provides the “yes” or “no” answers that the earlier article sections noted to be required for each of some problem of interest’s PEs. The step’s criticality prompts SMEs to address the equally critical question of whether or not to explore improvements in an enterprise that will then have been shown to meet or not meet stakeholder expectations for it. As with its depiction of Step 2e-related proceedings, Figure 7 offers a sense of how an AID application’s results support SME judgments to be made in response to calculated PE values. PEs determined to hold values less than those of their assigned thresholds obviously demand improvement. For the MPPDC example, Figure 7 shows the PE of “Have we characterized watershed needs?” to possess a value less than its pre-assigned threshold and therefore necessarily garnering a “no” response to the question of its resolution.

Step 2g: Determine desired improvements

SMEs decide whether or not to explore improvements to an enterprise that may or may not already satisfy expectations of the stakeholding parties that SMEs represent. An SME group may, of course, elect not to explore improvements to an enterprise shown with Step 2f as satisfying pertinent expectations; however, it may pursue any degree of analysis aimed at enhancing stakeholder satisfaction. Given the charge to do so, such a group would certainly be obliged to explore possible improvements to enterprises failing to meet stakeholder desires, and they could do that with the support of their facilitators and appropriate software. For example, and although there certainly seem many avenues through which improvements to some stormwater management enterprise could be made, Figure 7 suggests an immediate need to improve the value of the “ . . .watershed needs” PE. This might be done by SMEs exploring related SMs to determine which of them offer the best chance for the PE’s improvement.

Step 2h: Calculate optimal improvements

The Enterprise AID methodology and its developers have so far chosen not to incorporate any optimisation algorithms with AID protocols because they view such a step as a risky departure from AID’s paramount concern for SME deliberations. Although mathematical optimisation techniques could, if used wisely, enhance such deliberations, methodology designers recognize a potential for more harm than good. They have therefore chosen to presently ignore traditional optimisation approaches in favor of Step 2g’s multicriteria decision analysis–based schemes better aligned with AID’s emphasis on faithful representations of stakeholder expectations.

Conclusions and Recommendations

Stakeholders in watersheds and the man-made physical and non-physical constructs that watersheds typically overlap can benefit by pursuing related problems from perspectives appropriate for the systems of systems that the watersheds and man-made features together constitute. Indeed, those same stakeholders and the societies of which they are a part deserve the benefits to be found in problem approaches equally respectful of watershed and societal needs. This article suggests that the Enterprise AID methodology offers what societies and their natural environments deserve.

The heart of Enterprise AID is its use of pertinent tenets of systems science, operational test and evaluation, and multicriteria decision analysis disciplines, two of which—systems science and multicriteria decision analysis—the literature has revealed as possibly quite useful in addressing the stormwater management problems (Lai et al., 2008; McGarity, 2013) on which this article has focused. The methodology’s robustness to system of systems–related problems, too, promotes it as a means of particular value to interests confronted with the types of issues that almost by definition accompany stormwater management enterprises in particular and other public works management enterprises in general. Complex as it is, stormwater management represents only the tip of an iceberg comprising much more than that singular pursuit, and so this article’s authors encourage stakeholders of all public works management enterprises to consider Enterprise AID’s utility whenever they address system of systems–type problem needs like the one shown here to confront the Virginia’s MPPDC.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Thomas J. Meyers holds a PhD degree in systems engineering and is an adjunct assistant professor at Old Dominion University’s Department of Engineering Management and Systems Engineering. His research interests include complex system performance measurement and management, risk assessment, group decision making, and elicitation of expert opinion. He is a member of the Institute of Electrical and Electronics Engineers (IEE), the International Test and Evaluation Association, and the Performance Management Association.

Patrick T. Hester is an associate professor of engineering management and systems engineering at Old Dominion University. He received a PhD in risk and reliability engineering from Vanderbilt University. His research interests include multi-criteria decision analysis, enterprise performance measurement and management, systems of systems, and systemic thinking, and he is a member of the IEE, Performance Management Association, Society for Judgment and Decision Making, and International Society on Multiple Criteria Decision Making.

James C. Pyne, PhD, PE, BCEE, is the chief of small communities for Hampton Roads Sanitation District (HRSD) and an adjunct assistant professor of engineering management and system engineering at Old Dominion University in Norfolk Virginia. He has more than 35 years of experience in wastewater management at the technical, managerial, and policy level. His expertise includes sociotechnical systems as well as complex systems analysis. In his role as the HRSD chief of small communities, he has used this knowledge to assist numerous and various stakeholders deal with complex issues.

References

Abel

T. D.

Hennessey

J. T.

Jr. (1997). State and local opportunities in environmental policy: Cleaning up the Potomac and Anacostia rivers. Public Works Management & Policy, 2, 159-170.

Banks

R. L.

Wheelwright

S. C.

(1979). Operations versus strategy: Trading tomorrow for today. Harvard Business Review, 57(3), 112-120.

Bean

A. S.

(1995). Why some R&D organizations are more productive than others. Research Technology Management, 38, 25-29.

Bodner

D. A.

Rouse

W. B.

(2007). Understanding R&D value creation with organizational simulation. Systems Engineering, 10, 64-82.

Bourne

Wilcox

Neely

Platts

K. W.

(2000). Designing, implementing, and updating performance measurement systems. International Journal of Operations & Production Management, 20, 754-771.

Brown

M. G.

Svenson

R. A.

(1988). Measuring R&D productivity. Research Technology Management, 31(4), 11-15.

Brown

W. B.

Gobeli

(1992). Observations on the measurement of R&D productivity: A case study. IEEE Transactions on Engineering Management, 39, 325-331.

Camp

R. C.

(1989). Benchmarking: The search for industry best practices that lead to superior performance. Milwaukee, WI: ASQC Quality Press.

Consumer Product Safety Commission. (2008, November). The safety review. Washington, DC: Author.

10.

Daigger

G. T.

(2009). Evolving urban water and residuals management paradigms: Water reclamation and reuse, decentralization, and resource recovery. Water Environment Research, 81, 809-823.

11.

Dixon

J. R.

Nanni

A. J.

Vollmann

T. E.

(1990). The new performance challenge: Measuring operations for world-class competition. Homewood, IL: Dow Jones-Irwin.

12.

Dockery

J. T.

(1986). Why not fuzzy measures of effectiveness. Signal, 40, 171-176.

13.

Eccles

R. G.

(1991). The performance measurement manifesto. Harvard Business Review, 69(1), 131-137.

14.

Edwards

T. L.

(2007). Sustainable intergovernmental agreements: A case study of civil infrastructure design and regional cooperation. Public Works Management & Policy, 12, 370-377. doi:10.1177/1087724x07302587

15.

Goldratt

E. M.

Cox

(1986). The goal: Beating the competition. Hounslow, UK: Creative Output Books.

16.

Graves

S. B.

Ringuest

J. L.

Case

R. H.

(2000). Formulating optimal R&D portfolios. Research Technology Management, 43, 47-51.

17.

Grigg

N. S.

(2013). Stormwater programs: Organization, finance, and prospects. Public Works Management & Policy, 18, 5-22.

18.

Haimes

Y. Y.

(2004). Risk modeling, assessment, and management (2nd ed.). Hoboken, NJ: Wiley.

19.

Haimes

Y. Y.

Schneiter

(1996). Covey’s seven habits and the systems approach: A comparative analysis. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 26, 483-487.

20.

Hall

R. W.

(1983). Zero inventories. Homewood, IL: Dow Jones-Irwin.

21.

Hayes

R. H.

Abernathy

W. J.

(1980). Managing our way to economic decline. Harvard Business Review, 58(4), 67-77.

22.

Hester

P. T.

Meyers

T. J.

(2012). Enterprise AID: A performance measurement system for enterprise assessment, improvement, and design (Report NCSOSE-TR-2012-001 of January 23, 2012). Norfolk, VA: National Centers for System of Systems Engineering, Old Dominion University.

23.

Hultink

E. J.

Robben

H. S. J.

(1995). Measuring new product success: The difference that time perspective makes. Journal of Product Innovation Management, 12, 392-405.

24.

Kaplan

R. S.

Norton

D. P.

(1992). The balanced scorecard—Measures that drive performance. Harvard Business Review, 70(1), 71-79.

25.

Kellogg

W. A.

(1997). Local government and environmental quality: A case study. Public Works Management & Policy, 2, 94-111.

26.

Kraft

M. E.

(2006). Sustainability and water quality: Policy evolution in Wisconsin’s Fox-Wolf River basin. Public Works Management & Policy, 10, 202-213.

27.

Lai

Lundie

Ashbolt

N. J.

(2008). Review of multi-criteria decision aid for integrated sustainability assessment of urban water systems. Urban Water Journal, 5, 315-327.

28.

McGarity

A. E.

(2013). Watershed systems analysis for urban storm-water management to achieve water quality goals. Journal of Water Resources Planning and Management, 139, 464-477.

29.

Meyers

T. J.

Hester

P. T.

(2013). Measuring system of systems performance. International Journal of System of Systems Engineering, 3, 277-289.

30.

Mish

F. C.

(Ed.). (2009). Merriam-Webster’s collegiate dictionary (11th ed.). Springfield, MA: Merriam-Webster.

31.

Neely

(1999). The performance measurement revolution: Why now and where next? International Journal of Operations & Production Management, 19, 205-228.

32.

Neely

Adams

Crowe

(2001). The Performance prism in practice. Measuring Business Excellence, 5(2), 6-12.

33.

Neely

Adams

Kennerley

(2002). The performance prism: The scorecard for measuring and managing stakeholder relationships. London, England: Financial Times.

34.

Pisano

Mazmanian

D. A.

Little

R. G.

Linder

Perry

(2010). Toward a national strategic investment framework. Public Works Management & Policy, 14, 246-262.

35.

Regional Cooperation Act, Virginia Code § 15.2-4200-4222 (1968).

36.

Rouse

W. B.

(2005). Enterprises as systems: Essential challenges and approaches to transformation. Systems Engineering, 8, 138-150.

37.

Sage

A. P.

Biemer

S. M.

(2007). Processes for system family architecting, design, and integration. IEEE Systems Journal, 1, 5-16.

38.

Sage

A. P.

Cuppan

C. D.

(2001). On the systems engineering and management of systems of systems and federations of systems. Information, Knowledge, Systems Management, 2, 325-345.

39.

Skinner

(1974). The decline, fall, and renewal of manufacturing plants. Industrial Engineering, 6(10), 32-38.

40.

Southwest Florida Management District. (2012). Watershed management: A comprehensive approach to managing water resources in the southwest Florida water management district. Retrieved from http://www.swfwmd.state.fl.us/publications/files/watershed_management.pdf

41.

StateFarm.com . (2011). William Shatner saves the world from . . . exploding turkeys!? Retrieved from http://www.statefarm.com/aboutus/_pressreleases/2011/william-shatner-saves-the-world-from-exploding-turkeys.asp

42.

Strecker

E. W.

Huber

W. C.

Heaney

J. P.

Bodine

Sansalone

J. J.

Quigley

Thayumanavan

(2005). Critical assessment of stormwater treatment and control selection issues (WERF Report 02-SW-1). Alexandria, VA: Water Environment Research Foundation.

43.

Tangen

(2004). Performance measurement: From philosophy to practice. International Journal of Productivity and Performance Management, 53, 726-737.

44.

Taticchi

Balachandran

Tonelli

(2012). Performance measurement and management systems: State of the art, guidelines for design and challenges. Measuring Business Excellence, 16, 41-54.

45.

Underwriters Laboratories. (n.d.). Product safety tips: Turkey fryers. Retrieved from http://www.ul.com/global/eng/pages/offerings/perspectives/consumer/productsafety/turkeys/

46.

Water Environment Research Foundation. (2010). Sustainable integrated water management: State of the knowledge. Retrieved from https://www.werf.org/c/2011Challenges/SIWM.aspx

47.

Wilcox

Bourne

(2003). Predicting performance. Management Decision, 41, 801-816.

48.

YouTube.com . (2011). William Shatner & State Farm present “Eat, Fry, Love,” a turkey fryer fire cautionary tale. Retrieved from http://youtube.com/watch?v=EYkRF_FmD40