Does the Program Manager Matter? New Public Management and Defense Acquisition

Abstract

Past efforts to reform defense acquisition have been based in New Public Management assumptions that certain attributes of program managers (PMs), such as their training and experience levels, are important for improving outcomes. This article documents an effort to examine the relationship between such PM attributes and program outcomes using data drawn from annual Department of Defense Selected Acquisition Reports for major defense acquisition programs between 1997 and 2010. The findings provide little support for these assumptions. They point instead to the potential for institutionalist theories to explain acquisition outcomes, which can enable more nuanced reform policies in the future.

Keywords

defense acquisition New Public Management program manager institutionalism

Introduction

Few areas of American government entail such costly, complex, and critical activities as defense acquisition.¹ Over one third of the annual budget for the Department of Defense (DOD)—an average of more than US$150 billion annually since 2005—buys materiel, systems, and services (Comptroller, 2014). Acquiring expensive weapons through large contracts with industry is inherently complicated, risky, and political (Brown, Potoski, & Van Slyke, 2009; Fox & Miller, 2006; Mayer, 1991). Acquisition’s criticality is evident in what it seeks to provide: capabilities to defend the nation and its interests, and to protect its men and women in uniform. Defense acquisition is also highly problematical, fraught with complexity, waste, and inefficiency, with individual weapons programs such as the F-35 Joint Strike Fighter (JSF) or the Littoral Combat Ship often experiencing large cost overruns, schedule delays, and performance shortfalls (see, for example, Farrell, 1997; Government Accountability Office [GAO], 2008b; 2010a; Lebovic, 1996; McNaugher, 1989).² A history of continuing problems with unfavorable outcomes in major DOD programs has led to a long series of acquisition reform recommendations and measures since the mid-20th century (Fox, 2012; General Accounting Office, 1979; Lockwood, 1990; Rich & Dews, 1986).

For the most part, these late 20th-century reforms are reflective of, and in fact fostered the development of, what has come to be called New Public Management (NPM; Hood, 1991). Reforms were often driven by an assumption that acquisition lacked an efficiency focus, inherent in public management, and illustrated well by President Reagan’s appointment of the Packard Commission in 1986 following several high-profile procurement “horror stories” including US$435 hammers and US$640 toilet seats (Fox, 1988). This tended to result in calls for a more professionalized approach to acquisition under the assumption that the problem was with the lack of business acumen in program management—a problem that could be solved with better training and more adept business management, ideas very much in line with NPM (Hood & Lodge, 2004; Taylor, 2009). However, there are many critics who suggest that public management is fundamentally different from business management owing to its distinct context and requirements (Perry, 2007), and in particular, due to the distinct institutional environment of defense acquisition (Kelman, 1990; Mayer & Khademian, 1996; Thompson, 1993). In this research, we assess the effects of NPM-inspired policy reforms in defense acquisition. Recent data (e.g., GAO, 2014, 2015) suggest that these reforms have not been successful. Here we subject NPM assumptions to empirical study, assessing whether they are justified by improvements in acquisition effectiveness, or whether institutionalist arguments suggesting that the organizational environment will be resistant to change hold sway.

NPM and Institutionalism

Although use of the term NPM is relatively recent, the ideas underlying NPM are not necessarily new. Ideas of professionalization, an efficiency focus and performance measurement can be traced back to the likes of Wilson, Taylor, and other managerialists (Terry, 1998). NPM ideas focus on the pursuit of managerial efficiency by focusing on outcomes (Hood & Lodge, 2004; Osborne & Gaebler, 1992), performance measurement (Bouckaert & Peters, 2002), market incentives (Gansler, 1989, 2011a), privatization (Savas, 1982), or deregulation (Adelman & Augustine, 1990). The fundamental premise is that better policy outcomes can be achieved by more measurement and more adherence to market philosophies, utilizing not only incentives and standardization but also innovation and discretion. These ideas have had a profound influence on public management reform efforts for decades, but especially the past 30 years (Pollitt & Bouckaert, 2011).

The enthusiasm for NPM-oriented reform in practice has not generally been shared by those who study public organizations. Scholarship tends to critique the NPM philosophy, noting the mismatch between market-oriented approaches and the context of public management (Hood, 1991) and particularly on the implications for democracy and representation (Goodsell, 2003; Perry, 2007). In fact, critiques of market-oriented approaches have also been prominent for decades, reaching back to the behavioral and institutionalist arguments of March and colleagues (Cyert & March, 1963; March & Olsen, 1984, 2005). These scholars argued that management, particularly in complex environments characterized by uncertain outcomes and political complications, is about bargaining and navigating through institutional complexity.

There is a fundamental incongruence between these two theoretical approaches. The rational reforms that would be suggested by an NPM philosophy are not likely to be effective in the political environment described by institutionalists. However, NPM proponents may suggest that rationalization and professionalization may help buffer organizational practices from political interference and foster efficiency in both processes and outcomes. In that sense, we may not expect immediate returns from NPM reforms, as professionalization may require time to mature to the point where reforms achieve their intended goals. Fortunately, we see an arena, largely neglected in evaluations of NPM reforms, where changes have had ample time to reach a level of maturity, but yet which also includes a relevant counterfactual group by which to compare outcomes: defense acquisition and the professionalization of the program manager (PM).

Defense Acquisition, PMs, and Reform

A perennial focus of defense acquisition policy reform is the PM—the public official appointed to manage the effort to acquire a defense system (Baumgartner, 1979; Denny, 1985; Fox, 1974, 1984, 1988; Ladner, 1983; Peck & Scherer, 1962). Academic studies, presidential commissions, and government reports identify PMs with attributes such as advanced education, training, and experience as important contributors to improved program outcomes (Blue Ribbon Commission, 1986; General Accounting Office, 1986; Lockwood, 1985). The DOD and Congress have pursued a number of reforms over the years designed to enhance the quality of the acquisition workforce, most notably the Defense Acquisition Workforce Improvement Act (DAWIA) of 1990 (Lockwood, 1986, 1990; Mavroules, 1991; Snider, 1996). These policies all share a common frame of reference predicated on a simple operational assumption: that program outcomes can be improved by appointing PMs with more and more relevant training, education, and experience. This assumption reflects a stream of managerialism in 20th-century public administration (Terry, 1998) and, more recently, fits well within the norms of NPM (Gansler, 2003; Osborne & Gaebler, 1992; Thompson & Jones, 1994).

Yet, despite such reforms, acquisition outcomes have not improved, at least on the aggregate level; programs continue to miss their cost, schedule, and performance targets (Fox, 2012; GAO, 2010a, 2011, 2014, 2015; Munechika, 1997). The institutionalist argument is well represented in defense acquisition, with works that argue that significant institutional factors restrict implementation options and thus limit policy efforts toward broad-based acquisition reform (Chin, 2004; Kelman, 1990; Mayer & Khademian, 1996; Thompson, 1993). In general, these authors hold little optimism for wholesale reforms, especially those premised on NPM. The institutional theorists call instead for heightened attention to the essentially political nature of defense acquisition (Burnett & Kovacic, 1989). Each acquisition program is different, and there is only so much that an individual PM can do for highly complex and politically charged programs (Mayer & Khademian, 1996).

According to this line of thinking, the NPM ideas regarding PM reform are wrongheaded: Although it may be desirable to enhance a PM’s professional attributes, improvements in program outcomes—at least in an aggregate way—might not be obtainable (Pollitt & Bouckaert, 2011). From this perspective, the contributions of a PM to program outcomes are dwarfed by larger institutional forces that the PM cannot influence, such as the complex DOD structure, an uncertain geopolitical environment or partisan control of the budget. Despite the conceptually appealing idea that better trained managers should foster better outcomes, institutionalists would expect to see little to no correlation between PM attributes and program outcomes, and little reason to expect program outcomes to improve at all.

Our purpose in this article is to examine the competing perspectives of the institutionalist and NPM arguments. We argue that the reforms to defense acquisition are NPM in orientation and we subject to empirical scrutiny the underlying NPM acquisition reform assumption that improving PM attributes improves program outcomes, or whether, as the institutionalists suggest, characteristics of the PM show little to no correlation with either improved or worsened program outcomes. We do so by comparing defense program outcomes that vary by the professional status and experience of the PM, and by the commitment to professionalization by the DOD agency that manages the program. Our analysis begins by exploring the basis for the assumption that PM attributes and agency commitment to NPM ideas matter for program outcomes. We draw our outcome variables from outcomes that have been established to be important through Congressional legislation. In other words, we evaluate the reform policies on the basis of the outcomes for which the reforms were intended to improve. We describe general trends of DOD acquisition programs and PMs, using data from the largest programs from 1997 to 2010.

This article has two key goals. First, we aim to inform future acquisition reform policy through an evaluation of current policy. In questioning the underlying assumptions concerning PMs and the extent of their influence, we seek to promote more thoughtful and analytically based policies on future acquisition reform. Second, we inform broader public management theory and practice, by offering insights into the management of complex public projects that rival defense acquisition in their cost, complexity, and criticality. Examples of problematical “mega-projects” include Boston’s Central Artery/Third Harbor Tunnel project (the “Big Dig”; Allison & Aloisi, 2004), the Denver International Airport project (Dempsey, Goetz, & Szyliowicz, 1997), and the Department of Homeland Security’s Secure Border Initiative (“Border Fence”; GAO, 2010b).³ NPM assumptions have prevailed in developing the roles, responsibilities, authorities, and expectations for leaders of these projects; however, our findings challenge those views and offer more appropriate interpretations of the position of “public project manager.”

The Origins of Project Management and NPM

Although firm-based project management has long been a staple of business management, public project management emerged during the mid-20th century as a discipline for managing complex and unique acquisition efforts, such as the Manhattan Project and development of the first U.S. intercontinental ballistic missiles (Baumgartner, 1979). On the cutting edge of ideas that would come to be known as NPM, the Second Hoover Commission (MacNeil & Metz, 1956) found that DOD’s organizations and processes were ill-suited for the acquisition of such weaponry. In particular, specialized skills and knowledge were needed, the Commission argued, highlighting three areas of deficiency:

The need for specialized PM training, education, and career paths

The need for longer tenure-in-office durations among PMs to build expertise and for managerial stability

The need to decrease the proportion of uniformed PMs in favor of civilian civil service PMs who, the Commission held, generally had better business skills and thus were better suited for complex acquisition managerial roles

Throughout the 1970s and 1980s, these remained persistent targets for reform by Congress and various commissions and panels (Table 1).

Table 1.

PM Reforms.

Year	Reform entity	Findings/recommendations
1970	President’s Blue Ribbon Defense Panel (Fitzhugh Commission; Lockwood, 1985)	Recommended to Establish career specialty tracks for military PMs, along with selection and training criteria; Increase the number of qualified civilian PMs; Increase the duration of PM assignments.
1972	Commission on Government Procurement (McKinney, Gholz, & Sapolsky, 1994)	Recommended to increase the number of civilians as major PMs as a means to increase managerial expertise and reduce turnover in that position
1981	Congress (Defense Acquisition Improvement Program; Munechika, 1997)	Directed increased PM tenure
1984	Reagan Administration (Lockwood, 1985)	Directed the establishment of career management programs for professional federal employees
1984, 1985	Congress (National Defense Authorization Acts; Lockwood, 1985)	Set requirements for PM assignment durations to 4 years or the completion of a major program milestone; Minimum PM training and education.
1986	President’s Blue Ribbon Commission on Defense Management (Packard Commission; Blue Ribbon Commission, 1986)	Found that, compared with that of private industry, the defense acquisition workforce is “undertrained, underpaid, and inexperienced”; Affirmed the importance of prior legislation and further improvements to training and career development for both military and civilian PMs.
1990	Congress (DAWIA, 1990)	Established a designated PM career field, career managers, advisory boards, and promotion paths; Established PM certification levels and requirements; Favored increased numbers of civilian PMs; Reinforced requirements for minimum PM tenure duration, training, and experience.

Note. PM = program manager; DAWIA = Defense Acquisition Workforce Improvement Act.

NPM Reform and PM Attributes

We see three related key reforms to the PM position over this time that reflect the NPM philosophy: professionalization of the PM, including increasing the number of civilian PMs; ensuring that PMs have more experience in the position, whether active duty or civilian; and changing agency PM development processes to foster a cultural commitment to these goals.

Professionalization

By the late 1980s, as NPM ideas gained influence and in light of continuing poor acquisition outcomes, interest in the quality of the acquisition workforce had grown to the point that Congress was considering major reform legislation (Etherton, 2011; Preston, 2011). As a prelude, the House Armed Services Committee (HASC) discovered that the PM environment appeared to be resistant to the managerial changes, finding the following in 1990:

Fewer than half of major program PMs met minimum training requirements.

Since 1984, only 6 of 94 major programs PMs had satisfied minimum tenure requirements.

The DOD’s career programs for PMs lacked advanced specialized education in acquisition-related disciplines.

Only about 10% of major program PM positions were filled by civilians (Mavroules, 1991).

However, the report also noted differing levels of attention to PM attributes among the three military departments. For example, in line with the Air Force’s approach to professionalization (discussed below), the Air Force had more experienced PMs (averaging 17 years of experience) than the Army or Navy, as well as a higher percentage (97%) of PMs with at least 8 years of acquisition experience than either the Army (81%) or the Navy (71%; HASC, 1990). In other words, the problem was not with the NPM-oriented reforms but with insufficient and uneven agency commitment to reform within DOD.

Congress took action with the DAWIA (1990), premised on NPM principles of outcome-orientation and professionalism. DAWIA required the Secretary of Defense to establish policies and procedures for acquisition career management and to ensure their uniformity throughout the DOD. Some of its major provisions regarding PMs are listed in Table 1.

Compared with the years leading up to the DAWIA, relatively little policy attention was paid to the PM issue through the early 2000s. Since then, however, the issue has once again emerged as a topic of reform emphasis, as shown in Table 2.

Table 2.

Post-2000 PM Reform.

Year	Reform entity	Findings/recommendations
2001	Office of the Secretary of Defense	Reported that, on average since passage of the DAWIA, PMs had failed to meet statutory tenure requirements (Office of the Director of Acquisition Education, Training, and Career Development, 2001).
2005	Defense Acquisition Performance Assessment Project	Called for specific PM tenure requirements (Assessment Panel of the Defense Acquisition Performance Assessment Project for the Acting Deputy Secretary of Defense, 2005)
2008	GAO	Reported that for 39 major programs started since March 2001, the average PM tenure was only 17 months—less than half of what was required by statute (GAO, 2008a)
2009	BENS	Called for improved PM training and experience, and lengthened tenure durations; Noted that the acquisition career field is not viewed as a profession (BENS, 2009).
2011	DBB	Found that, compared with private sector PMs, DOD’s military PMs were inexperienced; Offered two choices for reform: Professionalize PMs (in the sense of creating a “career destination” as in law or medicine) or reserve major PM positions for civilians who serve for longer tenure durations (DBB, 2011).

Note. PM = program manager; DAWIA = Defense Acquisition Workforce Improvement Act; GAO = Government Accountability Office; BENS = Business Executives for National Security; DBB = Defense Business Board; DOD = Department of Defense.

PM tenure

Throughout the 1980s, reformers saw the requirement for longer PM tenure durations as the main issue for PM improvement (Etherton, 2011; Fox, 2012). Longer tenures were thought to contribute to PM experience and program stability, as well as enhance PM accountability; short tenures made it difficult to hold a PM accountable for a program’s shortcomings (Fox, 1988; Preston, 2011).

Efforts to increase the duration of PM tenures met with little success for several reasons that illustrate the institutionalists’ perspective. First, assignment and promotion policies for uniformed officers favored frequent career broadening rotations among a variety of positions (Fox, 1984; Gansler, 2011b). Second, acquisition was often perceived as a less than desirable career field: Active duty PMs saw themselves as having nominal charge over their programs, with little real authority because of the many overseers and stakeholders who held sway over their programs’ direction (Etherton, 2011; Fox, 1984). PMs were perceived to have fewer promotion opportunities than officers in operational (e.g., combat-related) career fields, and career planning and management for PMs received little emphasis (Gansler, 2011b; HASC, 1990). For such reasons, active duty PMs had incentives to leave their positions for other more career-enhancing assignments at the earliest opportunity. Finally, the personnel subcommittees of the Armed Services Committees in Congress, which held sway on military personnel matters, did not place the same priority on PM assignment durations as did other acquisition reformers in Congress (Etherton, 2011).

The issue remains unresolved, fraught by lack of consensus about strategies for extending PM tenures: by mandating minimum tenure durations, by increasing the proportion of civilian PMs (who are not subject to the same career influences as uniformed PMs), or by establishing dedicated career fields to allow uniformed PMs to serve for longer durations. In short, the institutional constraints of defense acquisition make it appear that the process is resistant to change. Nevertheless, the focus on increasing PM tenure remains an important goal of acquisition reform.

Differences among the military departments

During the key reform years, Fox (1974, 1988) critiqued acquisition cultures that steadfastly resisted reform, including reforms directed toward improving PM attributes. Nevertheless, he asserted the importance of those improvements, claiming, “The capability of the program manager and his staff obviously determines the ultimate success of each weapon acquisition program” (1974, p. 180), and “the reality that an effective and efficient acquisition program requires advanced program management skills, based on extensive practical training and years of program management experience” (1988, p. 196).

Within DOD, the military departments took Fox’s (1984) advice with varying levels of institutional commitment. The Air Force developed the most robust career management policies and procedures, essentially training and assigning officers in PM-related jobs from the most junior levels (Etherton, 2011; Preston, 2011). The Army generally took midlevel officers from the operational ranks and provided developmental assignments and training before assigning them as PMs (Baumgartner, 1979). The Navy had the least developed PM career program. Senior officers were often taken directly from operational (i.e., “at sea”) positions, assigned to the one PM training courses mentioned earlier, and then assigned as PM (Etherton, 2011; Preston, 2011). Navy policy held that its PMs should be proven leaders with recent operational experience in their programs’ respective domains (i.e., a submariner should serve as PM of a submarine program; a pilot should serve as PM of an aircraft program). To the extent that business expertise and program continuity were desired in program management, the Navy held that these were properly provided by civilian deputies to the uniformed PMs (Fox, 1984). The Navy’s reaction in particular illustrates the institutional resistance that emanates from a military service’s norms and culture and which opposes reforms based in managerialism and NPM.

Measuring Outcomes

Along with the focus on professionalism, performance measurement is an integral aspect of the NPM philosophy (Taylor, 2009), and defense acquisition reform included this outcome and measurement orientation. The fundamental idea is that instrumental improvements in program outcomes cannot be achieved without regular monitoring (Behn, 2003). To wit, early project management literature established three parameters for successfully managing any project: its cost, its schedule or time to complete, and the performance (or quality) of its results (Baumgartner, 1963). Because these parameters are, at least intuitively, connected but interdependent (e.g., achieving higher quality typically requires investing additional resources), they require tradeoffs by the PM to balance them appropriately to achieve an acceptable outcome for each parameter. For example, if project costs begin to rise, the PM may recommend deleting some lower priority features of the end product so as to satisfy at least minimal outcomes in both cost and performance (Rendon & Snider, 2008).

As NPM gained prominence, these three parameters, although possibly subjective and not necessarily grounded in instrumental utility, were institutionalized in DOD policy requiring that each acquisition program when initiated must establish an approved baseline of minimally acceptable, or threshold, values for cost, schedule, and performance (Defense Acquisition University [DAU], 2011).⁴ This acquisition program baseline (APB) serves as an important management tool throughout the life of a program. PMs and other managers track variances between current values and baseline values. As long as the current estimates of all parameter values are favorable in relation to their thresholds, the program is generally viewed as on track. If an unfavorable variance becomes sufficiently large, a “breach” occurs. In this context, a breach is simply a deviation from the program’s project path. There is not necessarily a causal assumption of fault for a breach; a breach is intended to alert officials above the PM of potentially critical problems with the acquisition program.

In the 1982 DOD Authorization Act, Senators Nunn and McCurdy sponsored a provision that elevated the importance of program unit cost by requiring that the Secretary of Defense report to Congress whenever the unit cost for any major program has an unfavorable variance large enough to constitute a breach. As with an APB breach, a “Nunn-McCurdy breach” of unit cost signals the potential for serious problems in a program, and hence the need for attention by senior officials and/or Congress.

Although, in line with the reform policies, we utilize variance and breach as our outcome variables of interest, we do note their limitations. The difficulty in defining and measuring outcomes for project success, especially for complex projects with diverse stakeholders like DOD acquisition programs, generates much attention (Pinto & Slevin, 1988). Agreement on specific success criteria is elusive, and most agree on only general standards like overall mission accomplishment, stakeholder satisfaction, and conformance to quality, cost, and schedule targets (Baker, Murphy, & Fisher, 1988; Murphy, Baker, & Fisher, 1974). This problem is exacerbated by the lack of data on achievement of cost, schedule, and quality targets; as a result, most studies rely on stakeholders’ subjective assessments to determine outcomes (Crawford, 2002).⁵ Furthermore, uncertainty in forecasting creates incentives for “strategic optimism” (Flyvbjerg, Holm, & Buhl, 2002)⁶ and creates an incentive to either game the performance objectives (Bevan & Hood, 2006) or strategically buffer the program from scrutiny (Oliver, 1991). Faulty initial estimates, from which the APB thresholds are derived, contribute significantly to variances (Bertisen & Davis, 2008; Feuring, 2007; Harrington, Morgenstern, & Nelson, 2000; Quirk & Terasawa, 1986). Thus, the program baseline may constitute a figurative “rigged deck” when it comes to evaluating program management.

Institutionalism and Reform Skepticism

Critics of these PM and outcome reforms argue that program outcomes are shaped by powerful institutional dynamics that PMs have little ability to control. As Preston (2011) described the Peacekeeper (MX) missile program,

It was deemed as one of the most successful programs that the Air Force had ever had. Why was that? It was because Congress supported the program, [DOD] supported the program, it got all the funding it needed, and the technology was developed in time . . . . You could have the best program manager in the world [but] if the contractor can’t get the technology breakthroughs that they need [or] if Congress changes its mind halfway through a program and cuts the funding . . . you are going to have an unsuccessful program.

Brown et al.’s (2009) study of the U.S. Coast Guard’s “Deepwater” program, by contrast, depicts a deeply troubled program beset by increasing costs and complexity that began to “spiral away” from a government–industry partnership toward increasingly legal, formal, and adversarial relations. Reform skeptics see the MX missile and Deepwater programs as evidence that the institutional dynamics shaping program outcomes will likely not yield to improved management, and broad-based acquisition reform offers little hope of altering systemic problems rooted in politics and culture (Burnett & Kovacic, 1989; Kelman, 1990; Mayer & Khademian, 1996; Thompson, 1993).

There are significant challenges to considering what is important for assessing PM performance (Geoghegan & Dulewicz, 2008; Posner, 1987; Thamhain, 1991), including specifically the performance of acquisition PMs (Cullen & Gadeken, 1990; Gadeken, 2002; McVeigh, 1995). Business theorists debate the extent to which managerial competence contributes to even such a well-defined and accepted outcome as profitability (Roquebert, Phillips, & Westfall, 1996; Skandalis, Liargovas, & Merika, 2008). When considering a domain where the very nature of outcome definition is contested, like defense acquisition, additional challenges arise. While efficiency in acquisition is clearly a concern, the determination of a successful outcome measure to assess PM effectiveness is fraught with complexity.

Viewed from a human resource management (HRM) perspective, PM reform proponents call for investments in the human capital (Becker, 1964; Flamholtz & Lacey, 1981) of PMs with the expectation of future returns, that is, favorable program outcomes. The HR literature documents, however, several major difficulties with such return-on-investment (ROI) calculations; these include training validity (treatment effects from the program), performance validity (transferring performance from the program to the job), and the lack of metrics and data (Chmielewski & Phillips, 2002; Goldstein, 1979). Furthermore, institutional theorists are skeptical of human capital’s economic basis, noting that it ignores the complex of internal and external normative pressures that lead organizations to adopt such rational-managerial approaches in spite of these significant ROI measurement issues (Meyer & Rowan, 1977; Tolbert & Zucker, 1983).

Nevertheless, the prevailing assumption over the past decades is that improving PM experience (through training, tenure, or agency commitment) should result in better organizational outcomes. The conventional wisdom that we describe is no mere NPM straw man but rather has been the basis for acquisition reform over the past several decades. Yet, these reforms have met with less than satisfactory results (GAO, 2015), a circumstance that institutionalists may have predicted. An opportunity thus exists for systematic empirical inquiry into this important issue, which can provide a basis in evidence rather than in governing philosophies for makers of future acquisition reform policy. In short, we ask whether reforms that have aimed to professionalize the PM position, by increasing the use of civilian PMs, increasing PM training and agency commitment, and increasing PM tenure in position, are associated with improved organizational outcomes.

Analysis

Data

Using the Defense Acquisition Management Information Retrieval (DAMIR) Purview⁷ system, we constructed a panel data set including all major defense acquisition program⁸ (MDAP) Selected Acquisition Reports⁹ (SARs) submitted between 1997—the first year for which Purview SAR data are available—and 2010. These data include all acquisition programs that were designated as MDAP and which submitted a SAR during any of these years.¹⁰ The data include measures of the key outcomes variables that have been identified as signaling program success (or perhaps more accurately, failure) and several key characteristics of the PMs’ experience.

Outcome variables

We rely on two measures that, as described previously, DOD and Congress have clearly defined as outcomes measures that indicate whether a program is functioning successfully or not—unit cost variance (hereafter, cost variance) and breach, both of which are reported explicitly in the data set. We acknowledge that these measures are not necessarily ideal measures of organizational outcomes, but rather than delve into a more detailed discussion of public organization outcomes (see Radin, 2006, for a detailed discussion), we opted to rely on the measures that were identified in the reform policies as important. In other words, given the data limitations and difficulty adequately measuring aspects of the institutional environment, we opted to evaluate the NPM reforms on the basis of the goals that the reforms were intended to achieve.

Cost variance

Cost variance is the annual percentage change in an acquisition program’s unit cost from its original estimates. For example, if in Year 1 the estimated baseline unit cost for a program was US$1 million, and in Year 2 the estimate rose to US$1.1 million, the cost variance was a positive 10%¹¹; such a result would be considered unfavorable. Ideally, subsequent cost estimates would be, at worst, in line with initial estimates, or more favorably, lower. Thus, positive values of unit cost variance are indicative of poor performance, whereas negative values are indicative of good performance.

Within our data set, there are several programs with extremely high cost variance observations, and the unit cost estimates across all programs are widely variable—There are many comparatively low price programs and a handful of very expensive programs. As this situation is not atypical of dollar value variables generally, we operationalized this performance variable by taking the difference between the natural log of the actual unit cost and the natural log of the target unit cost. Thus, a positive value is indicative of a program that exceeded its targeted costs and a negative value indicates a program is performing better than its targets.

Program breach

A breach occurs when an estimate for any major cost, schedule, or performance parameter is determined to be significantly less favorable than its baseline estimate; these include, as discussed earlier, both APB and Nunn-McCurdy breaches. For this study, the breach variable will take on two nominal values: yes (if a program has experienced any type of breach in a year) or no (if not). Obviously, this dependent variable is very general and accounts for almost any adverse circumstance of sufficient importance to be reported. Furthermore, the breach outcome is expected to be a salient indicator of program performance both to agency superiors and also potentially to Congress (see, for example, Blickstein, Nemfakos, & Sollinger, 2013). Although there is a positive correlation between program breach and unit cost variance in a given year, the correlation is weak (.175); thus, we see these variables as assessing different outcomes.

NPM variables

Robust indicators of program specifics are not publicly available, thus, although our data unfortunately do not include detailed unique characteristics of particular programs or PMs, we do know two key aspects about the PM that are foci of the NPM reforms—whether the PM is military or civilian, and the PM’s tenure. In addition, we can approximate agency support for and with respect to both training and institutional effects; we can note which military service the program belonged to and was managed by. Tenure is measured as the number of months that a PM has been in the position, and according to NPM arguments, the longer the PM is in the position, the better the program outcomes should be. Military PM is a dichotomous variable indicating if the PM is an active duty military officer (rather than civilian). As civilian PMs are presumed to be trained professionals in program management, programs with civilian PMs are presumed to have more favorable outcomes than those with military PMs.¹² We also include indicator variables for the service the project belonged to, noting that the Air Force has tended to provide the most robust professional PM training, the Navy the least, and the Army a moderate amount. The data also include an indicator for multiservice (joint) general DOD programs, for which we have no particular expectations.

Institutional variables

Unfortunately, many of the key institutional factors that affect the programs, such as variations in political interest in a project or constraints against program changes (like a member of Congress supporting a program that DOD has recommended canceling), are not available in our data. We thus can only roughly approximate aspects of the institutional argument with service branch indicators, which in addition to signaling organizational commitment to PM professionalization, also serve as proxies for the institutional environment. We also include indicator variables for the type of commodity being purchased and the phase of the acquisition process. Commodities include aircraft, ground systems, missile systems, munitions, ships, space-based systems, and C3I (command, control, communications, and intelligence) systems. Program phase is an indicator variable differentiating programs that are in the design/development phase from those in the production phase. Summary statistics and proportions are provided in Table 4.

Control variables

Finally, as a control variable we use a program’s unit cost as a proxy for the complexity of the system to be acquired. Systems with higher unit costs (e.g., ships, aircraft) tend to have higher levels of complexity. Because this is a rough approximation of complexity and the very high variance of costs across all programs, each program will take on the nominal value of the quartile in which its unit cost falls (1-4).

As can be seen in Table 3, breach is common across each of the services and program characteristics, as are high unit cost variances. In any given year, approximately half of all programs have a positive unit cost variance (i.e., missed cost projections) and roughly 45% of all programs are in breach.

Table 3.

Performance Outcomes.

	Percentage of programs with positive unit cost variance	Mean unit cost variance (logged)	Percentage of programs in breach	Proportion of cases
PM characteristics
Military PM	53.59	2.83	44.69	86.07
Nonmilitary PM	35.76	−1.26	35.76	13.93
Services
Air Force	55.49	1.13	44.81	30.26
Army	47.12	3.71	42.71	27.21
Navy	50.50	2.54	61.40	5.26
Multiservice DOD	50.88	−1.82	40.35	37.27
Program characteristics
Low complexity	42.15	−1.77	52.49	24.08
Mid-low complexity	50.38	2.46	36.92	23.99
Mid-high complexity	60.31	2.99	40.84	24.17
High complexity	51.50	5.32	43.52	27.77
Overall	51.11	2.25	43.45

Note. PM = program manager; DOD = Department of Defense.

The full set of summary statistics for these variables is provided in Table 4. We do not have specific hypotheses because, as we noted above, there are theoretical arguments that could be made for either the absence or presence of associations between our independent variables and outcome variables. In short, if the institutionalist arguments hold sway, we would expect to see no correlation between PM characteristics and program outcomes, but clear trends between the different service branches and commodities (although we acknowledge that we do not have data available to assess specific reasons for such trends should they occur). If the NPM argument that undergirds existing policy is correct, we would expect to see better program outcomes as PM tenure increases, when PMs are civilians, and when the project is part of the Air Force (relative to the Army and especially the Navy, given the different approaches the three branches have taken).

Table 4.

Summary Statistics.

	M	SD	Minimum	Maximum
Log unit cost difference	0.14	0.16	−0.402	2.27
In breach	0.43	0.50	0	1
PM change from military to civilian	0.02	0.14	0	1
PM change from civilian to military	0.09	0.28	0	1
Air Force program	0.30	0.45	0	1
Army program	0.27	0.45	0	1
Navy program	0.37	0.48	0	1
General DOD program	0.05	0.22	0	1
Aircraft	0.31	0.46	0	1
Ground system	0.06	0.24	0	1
Missile system	0.14	0.34	0	1
Munitions system	0.09	0.28	0	1
Ship	0.11	0.31	0	1
Space system	0.11	0.31	0	1
C3I	0.17	0.38	0	1
Proportion of military PMs	0.86	0.34	0	1
Proportion of programs in development phase	0.47	0.50	0	1
PM tenure in months	17.75	12.96	0	76
Program cost category	2.56	1.13	1	4

Note. PM = program manager; DOD = Department of Defense; C3I = command, control, communications, and intelligence.

Method

As our data are from a panel of programs over a 13-year period, we utilized a generalized least squares (GLS) model and a panel logistic regression. We ran two different sets of models with our dependent variables corresponding to the measurements described above: the logged difference between the actual unit cost and the targeted unit cost, and an indicator variable for a program in breach. Each conforms to the basic model shown in Equation 1, where $Y_{i t}$ is the respective performance measurement (i.e., the difference in unit cost variance or the logged probability of breach) at time t for program i, $T_{i t}$ is the tenure in months of the program’s PM, $M_{i t}$ indicates if the PM is an active duty member of the military, $S_{i t}$ is a matrix of indicator variables for the service (Air Force, Navy, DOD with Army as the referent category) of program t, $P_{i t}$ is a matrix of indicator variables for the type of commodity being purchased, and $C_{i t}$ is a categorical measure of the total cost of the program. Finally, $v_{i t}$ is made up of three separate error components. First, there are likely to be estimation errors $μ_{i}$ that are common across the multiple observations of individual programs, i. Second, there are likely to be estimation errors, $λ_{t}$ that are common across the years included. Finally, there are also purely random errors $ε_{i t}$ associated with each unit and time period. For these models, we utilized a random effects logistic regression for the dichotomous breach indicator outcome variables, and a random effects regression for the model corresponding to the cost variance outcome variable.

Y_{i t} = T_{i t} + M_{i t} + S_{i t} + P_{i t} + C_{i t} + v_{i t} .

Results

The results are provided in Table 5. There is no association between breach or high unit cost variance and the type of PM (civilian or military). Programs managed by civilian PMs are neither more nor less likely to breach or have relatively high unit cost variances. Similarly, the length of time a PM has been on the job shows no association with either problematic outcomes or, inferentially, positive outcomes, and there are no significant differences in either breaches or cost variance problems across the different military services.

Table 5.

Performance GLS Regression Results.

	Program in breach		Logged unit cost variance
	Odds ratio	SE	β	SE
Civilian PM	0.618	0.157	−.024	0.015
PM tenure in months	1.00	0.006	−.001	0.001
Air Force program^a	1.10	0.402	−.010	0.017
Navy program^a	0.995	0.336	−.017	0.015
DOD program^a	2.81	1.60	−.004	0.029
Aircraft^b	7.38*	6.83	−.078*	0.039
Ground system^b	10.46*	10.7	−.050	0.045
Missile system^b	4.52	4.35	−.059	0.041
Munitions system^b	3.03	3.09	−.009	0.044
Ship^b	9.52*	9.66	−.092*	0.044
Space system^b	13.40*	13.3	−.095*	0.042
C3I^b	4.14	3.81	−.111*	0.040
Program cost category	0.762*	0.101	.027*	0.007
Program in development	1.08	0.202	.025*	0.011
N (groups)	189		179
N (total)	1,073		1,014
Wald	20.9		44.29*

Note. PM = program manager; DOD = Department of Defense; C3I = command, control, communications, and intelligence.

Army is reference service.

Other is the referent commodity type.

p < .05.

Institutional variables also show little correlation with performance outcomes, and results are mixed when they do. Aircraft, ships, and space systems are not only more likely to be in breach but also have more favorable unit cost variances. Finally, as programs become more complex, they tend to have slightly less favorable unit cost variances, but again conversely, are less likely to be in breach. Thus, there is a tendency to see mixed results between both breaches and high unit cost variances, which may not be surprising given the weak correlation between the two outcomes, but this suggests further that the two outcome measures are tending to measure different, not necessarily congruent, outcomes.

More broadly speaking, as seen in Figure 1, when aggregated across all cases, there do not seem to be any particular trends between unit cost variance and time nor, as seen in Figure 2, between time and the propensity of programs to be in breach.

Figure 1.

Aggregate unit cost variance over time.

Figure 2.

Aggregate proportion of breaches over time.

Discussion and Case Examples

In general, we take these trends as evidence that the institutional argument, that PMs are limited in their ability to affect program outcomes due to the complexities and politics of the organizational environment, holds sway (Mayer & Khademian, 1996). Although the data that we were able to access did not include finely grained details that could have been used to further assess the particular characteristics of the institutional environment, like PM perceptions of the political influences of their environment, a firm conclusion that can be made from our results is that efforts to professionalize the PM position have had no aggregate effect on policy-dictated program outcomes. The NPM assumptions that underlie these reforms, that increased PM training and experience will lead to better program outcomes, appear questionable. Although we are careful to note that our data do not permit us to explore the reasons for this lack of connection between PMs and program outcomes, we think it is important to discuss some of the possible reasons for our results, and we illustrate these conjectures by delving into detail on several of the programs from our data.

PMs Don’t Really Manage Programs

Both Preston (2011) and Soloway (2011) suggested that the term program manager is a misnomer. In this view, PMs exercise only nominal influence and little authority over program performance amid many powerful program stakeholders (Etherton, 2011; Fox, 1984, 156). Preston noted that defense contractors manage DOD acquisition programs, and DOD PMs manage the contractors. Soloway recalled a conversation with a PM who stated, “We’re not doing program management anymore . . . We’re managing people, budget, and politics . . . I don’t even do what I was really trained to do . . . from the technical side of program management.” If valid, these comments help explain why the independent variables in this study were poor predictors of program outcomes, as none of the factors to which Preston and Soloway refer were available in our data. Accounting for these factors in future analysis, however, would entail new challenges, for example, in making them operational in a reasonable model (e.g., how to operationalize an acquisition program’s political factors and the PM’s involvement in those) and actually acquiring the data to do so. In any event, the thrust of these interview comments, illustrated by the description of the CH-47F program below, challenge NPM-based assumptions of the traditional role of a defense acquisition PM in favor of an understanding of that role from an institutionalist perspective.

CH-47F

The Army’s CH-47F helicopter program was initiated in 1997 as an upgrade to the existing Boeing’s Chinook helicopter. Considered relatively low-risk, the initial CH-47F unit cost estimate was approximately US$10 million (GAO, 2003). Between 1999 and 2001, however, both total production and unit cost estimates roughly doubled, triggering a Nunn-McCurdy breach . Army decisions altering aircraft quantities and capabilities added unexpected production costs, as did increases in the cost of labor and materials for manufacturer Boeing (GAO, 2003). In 2009, the CH-47F program unit cost increased to about US$23 million, again breaching the program’s APB threshold. Looking back at the program’s development, the Army’s decision to upgrade Boeing’s Chinook locked-in existing technologies where a PM might otherwise have found innovative, cost-effective solutions for its heavy lift helicopter needs. Moreover, in the absence of technical and vendor competition and in light of the Army’s heavy demand for lift capabilities in Iraq and Afghanistan, the likelihood for significant cost growth was foreseeable.

Other Factors Outweigh the PM’s Influence

Scholars of defense acquisition (see, for example, Fox & Miller, 2006; Kronenberg, 1990) have noted the high level of complexity in defense acquisition programs. Several knowledgeable practitioners (Berteau, 2011; Etherton, 2011; Preston, 2011; Soloway, 2011) supported this view, suggesting that the range and influence of “non-PM” factors in major acquisition programs are large in comparison with PM-related factors. This may help explain why our analysis failed to show any significant influences by PMs on program outcomes, which is consistent with the inconclusive findings in the business literature, mentioned above, on the relationship between managerial competency and firm profitability. Events in the Excalibur program show how requirements beyond the PM’s managerial sphere of influence may have profound effects on a program’s outcomes. Put another way, there is a possibility that even the best PM can still have a troubled program.

Excalibur

The Army’s Excalibur program supplies Global Positioning System (GPS) guided artillery with enhanced range and accuracy over traditional artillery. Between 2002 and 2010, program development proceeded with minor difficulties. An initial 500 projectiles delivered in 2009 to American forces in Iraq and Afghanistan performed as well or better than expected. In light of the projected reductions in U.S. deployments abroad, however, a 2010 review of future requirements recommended a much smaller portfolio of precision munitions than previously anticipated (Wilson, 2011). The number of Excalibur rounds to be procured was reduced from 30,000 to 6,200—less than a quarter of expected production quantities. As a consequence, in 2011, the program’s unit cost rose by 200% causing a Nunn-McCurdy breach (DAMIR, 2011). A program that otherwise met cost and schedule goals and performed well in operations generated at once large unfavorable unit cost variances triggering a breach and favorable total cost variances because of decisions made above the PM’s pay grade.

Politics and law trump management

In the introduction to this article, we described acquisition’s cost, complexity, and criticality. We consider it axiomatic that, as these three characteristics of an acquisition program grow, so too do the needs for good management. At the same time, however, the influences of politics and law (Rosenbloom, 1998) also grow stronger and begin to overshadow those of management. For example, the decision-making authority for the largest and highest cost programs is vested in an acquisition executive who is a political appointee subject to Senate confirmation, and the great majority of statutes governing acquisition management apply only to those largest programs. The F-35 program highlights these powerful institutional forces in a current major acquisition program.

Alternate engine program

The F-35 JSF alternate engine program began in 1996, when Congress directed DOD to ensure that the JSF “provides for adequate engine competition,” thus requiring DOD to develop an alternative to the Pratt and Whitney engine that currently powers the JSF. The alternate engine, developed jointly by General Electric and Rolls-Royce, was funded in DOD’s budgets until 2006. Each year from 2007 to 2010, however, DOD proposed to terminate the program, citing cost savings and operational advantages accruing from a single engine supplier. Congress instead appropriated the full amount each of those years and directed that the program continue, with members claiming that engine competition would save money and result in higher engine reliability. Through 2010, when Congress finally agreed to end the program, more than US$2.5 billion had been spent on the alternate engine. Critics of the termination argue that immediate budget pressures drove the decision rather than rational analysis of long-term costs and benefits, and they also cite adverse impacts on relations with the United Kingdom, which had stakes in the alternate engine program (Gertler, 2012).

JSF demonstrates the prominence of political and legal influences in some major acquisition programs. These actions and decisions are taken at levels—for example, the Congress, the military branch hierarchy and DOD headquarters, and among industry executives—well beyond any PM’s managerial reach.

Conclusion

In this research, we found that PMs seem to have little relationship with defense acquisition program outcomes. Despite an underlying logic in defense acquisition policy that better PM retention and training will improve program performance, we find little evidence to support this assumption. In our study, there was no relationship between improving PM experience and training and program performance. These findings are in line with other recent research that has found a disconnect between the logic of NPM and the context of implementing policies rooted in NPM thinking (Eckerd & Heidelberg, 2015; Hood & Lodge, 2004; Pollitt & Bouckaert, 2011). In this research, we note further that the institutional environment of public management is often too complex and too political to be amenable to NPM managerial reforms.

We conclude with two points. First, regarding accountability: If PMs indeed can have little influence over program outcomes, it is unreasonable to hold them fully accountable for those outcomes; however, this opposes NPM’s approach that has guided defense acquisition for more than two decades. Reformers have treated the PM as an object for improvement, believing that, if the PM’s experience and training could be improved, then program outcomes would improve. This has the effect of deflecting accountability from its proper place—those in the realms of politics and law—and onto an easy target—the PM.

Second, regarding costs and benefits: It seems inherently logical that better trained PMs should have a positive influence in some way on outcomes. However, our analysis did not find much of a relationship. Our results suggest that any improvement based on PM attributes is small at best, but it is also worth noting that our analysis suggests that PM improvements do not worsen outcomes either. Thus, policy makers should perhaps choose to invest scarce reform resources on other factors of acquisition beyond just the PM.

Footnotes

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: One contributor (Snider) is an employee of the U.S. Government (Department of the Navy) and prepared the Contribution as part his official duties.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

Adam Eckerd is an Assistant Professor at the University of Tennessee. His research areas include social and environmental policy, public and nonprofit performance evaluation, and organizational decision making.

Keith Snider is Professor of Public Administration and Management at the Naval Postgraduate School, where he teaches courses and conducts research in public procurement and defense acquisition management.

References

Adelman

Augustine

(1990). The defense revolution: Intelligent downsizing of America’s military. San Francisco, CA: Institute for Contemporary Studies.

Allison

Aloisi

(2004). The big dig. Beverly, MA: Commonwealth Editions.

Assessment Panel of the Defense Acquisition Performance Assessment Project for the Acting Deputy Secretary of Defense. (2005). Defense acquisition performance assessment: Executive summary. Retrieved from http://www.defense.gov/pubs/pdfs/DAPA%2012-22%20WEB%20Exec%20Summary.pdf

Baker

Murphy

Fisher

(1988). Factors affecting project success. In Cleland

King

(Eds.), Project management handbook (2nd ed., pp. 902-919). New York, NY: Van Nostrand Reinhold.

Baumgartner

(1963). Project management. Homewood, IL: Irwin.

Baumgartner

(1979). Systems management. Washington, DC: Bureau of National Affairs.

Becker

(1964). Human capital: A theoretical and empirical analysis. New York, NY: Columbia University Press.

Behn

(2003). Why measure performance? Different purposes require different measures. Public Administration Review, 63, 586-606.

Berteau

(2011, April 4). Personal interview by author [Digital recording]. Washington, DC.

10.

Bertisen

Davis

G. A.

(2008). Bias and error in mine project capital cost estimation. The Engineering Economist, 53, 118-139.

11.

Bevan

Hood

(2006). What’s measured is what matters: Targets and gaming in the English public health care system. Public Administration, 84, 517-538.

12.

Biery

(1992). The effectiveness of weapon system acquisition reform efforts. Journal of Policy Analysis and Management, 11, 637-664.

13.

Blickstein

Nemfakos

Sollinger

(2013). Digging out the root cause: Nunn-McCurdy breaches in major defense acquisition programs. Defense Acquisition Review Journal, 20, 128-153.

14.

Blue Ribbon Commission. (1986). Report to the president on defense management. Washington, DC: U.S. Government Printing Office.

15.

Bouckaert

Peters

(2002). Performance measurement and management: The Achilles’ Heel in administrative modernization. Public Performance & Management Review, 25, 359-362.

16.

Brown

Potoski

Van Slyke

(2009). Contracting for complex products. Journal of Public Administration Research and Theory, 20, i41-i58.

17.

Burnett

Kovacic

(1989). Reform of U.S. weapons acquisition policy: Competition, teaming agreements, and dual-sourcing. Yale Law Review, 6, 249-317.

18.

Business Executives for National Security. (2009). Getting to best: Reforming the defense acquisition enterprise. Washington, DC: Author. Retrieved from http://www.bens.org/document.doc?id=44

19.

Chin

(2004). British weapons acquisition policy and the futility of reform: 1945 to the present. Aldershot, UK: Ashgate.

20.

Chmielewski

Phillips

(2002). Measuring return-on-investment in government: Issues and procedures. Public Personnel Management, 31, 225-237.

21.

Christensen

(1994). Cost overrun optimism. Acquisition Review Quarterly, 1994, 25-38.

22.

Christensen

Searle

Vickery

(1999). The impact of the Packard Commission’s recommendations on reducing cost overruns on defense acquisition contracts. Acquisition Review Quarterly, 1999, 251-262.

23.

Cohen

Eimicke

(2008). The responsible contract manager: Protecting the public interest in an outsourced world. Washington, DC: Georgetown University Press.

24.

Comptroller. (2014). United States Department of Defense fiscal year 2014 budget request: Overview. Retrieved from http://comptroller.defense.gov/budgetmaterials/budget2014.aspx

25.

Crawford

(2002). Profiling the competent project manager. In Slevin

Cleland

Pinto

(Eds.), The frontiers of project management research (pp. 151-176). Newtown Square, PA: Project Management Institute.

26.

Cullen

Gadeken

(1990). A competency model of program managers in the DoD acquisition process. Fort Belvoir, VA: Defense Systems Management College.

27.

Cyert

March

(1963). A behavioral theory of the firm. Englewood Cliffs, NJ: Prentice-Hall.

28.

Defense Acquisition University. (2011). Defense acquisition guidebook. Retrieved from https://dag.dau.mil/Pages/Default.aspx

29.

Defense Acquisition Workforce Improvement Act. (1990). National defense authorization act, fiscal year 1991. 10 U.S.C. §1701.

30.

Defense Business Board. (2011), Review of DoD’s program managers. Report FY11-03. Retrieved from http://dbb.defense.gov/Portals/35/Documents/Reports/2011/FY11-3_Review_Of_DoD’s_Program_Managers_2011-4.pdf

31.

Dempsey

Goetz

Szyliowicz

(1997). Denver International Airport: Lessons learned. New York, NY: McGraw-Hill.

32.

Denny

(1985, May). Better buyers, better weapons. Military Logistics Forum, pp. 16-25.

33.

Department of Defense. (1995). Glossary: Defense acquisition acronyms and terms (6th ed). Fort Belvoir, VA: Defense Systems Management College Press.

34.

Eckerd

Heidelberg

(2015). Public incentives, market motivations, and contaminated properties: New Public Management and Brownfield liability reform. Public Administration Review, 75(2), 252-261.

35.

Etherton

(2011, March 23). Personal interview by author [Digital recording]. McClean, VA.

36.

Farrell

(1997). Weapons without a cause. New York: St. Martin’s Press.

37.

Feuring

(2007). The impact of human capital on the cost of Air Force acquisition programs (Master’s thesis). School of Engineering and Management, Air Force Institute of Technology, Wright-Patterson Air Force Base, OH.

38.

Flamholtz

E. G.

Lacey

(1981). The implications of the economic theory of human capital for personnel management. Personnel Review, 10, 30-40.

39.

Flyvbjerg

Holm

Buhl

(2002). Underestimating cost in public works: Error or lie? Journal of the American Planning Association, 68, 279-295.

40.

Fox

J. R.

(1974). Arming America: How the U.S. buys weapons. Cambridge, MA: Harvard University Press.

41.

Fox

J. R.

(1984, September-October). Revamping the business of national defense. Harvard Business Review, 50, 63-70.

42.

Fox

J. R.

(1988). The defense management challenge: Weapons acquisition. Boston, MA: Harvard Business School Press.

43.

Fox

J. R.

(2012). Defense acquisition reform, 1960-2009: An elusive goal. Washington, DC: U.S. Army Center of Military History.

44.

Fox

J. R.

Miller

(2006). Challenges in managing large projects. Fort Belvoir, VA: Defense Acquisition University.

45.

Gadeken

(2002). What the United States Defense Systems Management College has learned from ten years of project leadership research. In Slevin

Cleland

Pinto

(Eds.), The frontiers of project management research (pp. 97-114). Newtown Square, PA: Project Management Institute.

46.

Gansler

(1989). Affording defense. Cambridge, MA: MIT Press.

47.

Gansler

(2003). Moving toward market-based government: The changing role of government as the provider. Arlington, VA: IBM Endowment for the Business of Government.

48.

Gansler

(2011a). Democracy’s arsenal: Creating a twenty-first-century defense industry. Cambridge, MA: MIT Press.

49.

Gansler

(2011b, April 8). Personal interview by author [Digital recording]. College Park, MD.

50.

General Accounting Office. (1979). Recommendations of the commission on government procurement: A final assessment (GAO/PSAD-79-80). Washington, DC: U.S. Government Printing Office.

51.

General Accounting Office. (1986). DOD acquisition: Strengthening capabilities of key personnel in systems acquisition (GAO-NSIAD-86-45). Washington, DC: U.S. Government Printing Office.

52.

Geoghegan

Dulewicz

(2008). Do project managers’ leadership competencies contribute to project success? Project Management Journal, 39(4), 58-67.

53.

Gertler

(2012). F-35 alternate engine program: Background and issues for congress (Report for Congress 7-5700). Washington, DC: Congressional Research Service.

54.

Goldstein

(1979). The pursuit of internal and external validity in the evaluation of training programs. Public Personnel Management, 8(6), 416.

55.

Goodsell

(2003). The case for bureaucracy: A public administration polemic. Washington, DC: CQ Press.

56.

Government Accountability Office. (2003). Defense acquisitions: Assessments of major weapon programs (GAO-03-476). Washington, DC: Author.

57.

Government Accountability Office. (2008a). Defense acquisitions: Department of Defense actions on program manager empowerment and accountability (GAO-08-62R). Washington, DC: Author.

58.

Government Accountability Office. (2008b). Defense acquisitions: Results of annual assessment of DoD weapon programs (GAO-08-674T). Washington, DC: Author.

59.

Government Accountability Office. (2010a). Defense acquisitions: Assessments of selected weapon programs (GAO-10-388SP). Washington, DC: Author.

60.

Government Accountability Office. (2010b). Secure border initiative: DHS needs to reconsider its proposed investment in key technology program (GAO-10-340). Washington, DC: Author.

61.

Government Accountability Office. (2011). Defense acquisitions: Assessments of selected weapon programs (GAO-11-233SP). Washington, DC: Author.

62.

Government Accountability Office. (2014). Defense acquisitions: Assessments of selected weapon programs (GAO-14-340SP). Washington, DC: Author.

63.

Government Accountability Office. (2015). Defense acquisitions: Assessments of selected weapon programs (GAO-15-342SP). Washington, DC: Author.

64.

Harrington

Morgenstern

R. D.

Nelson

(2000). On the accuracy of regulatory cost estimates. Journal of Policy Analysis and Management, 19, 297-322.

65.

Hefetz

Warner

(2004). Privatization and its reverse: Explaining the dynamics of the government contracting process. Journal of Public Administration Research and Theory, 14, 171-190.

66.

Hood

(1991). A public management for all seasons. Public Administration, 69, 3-19.

67.

Hood

Lodge

(2004). Competency, bureaucracy, and public management reform: A comparative analysis. Governance, 17, 313-333.

68.

House Armed Services Committee. (1990). The quality and professionalism of the acquisition workforce. Washington, DC: U.S. Government Printing Office.

69.

Ibbs

Reginato

(2002). Measuring project management’s value. In Slevin

Cleland

Pinto

(Eds.), The frontiers of project management research (pp. 177-186). Newtown Square, PA: Project Management Institute.

70.

Kausal

(1999). A comparison of the defense acquisition systems of France, Great Britain, Germany, and the United States. Fort Belvoir, VA: Defense Systems Management College Press.

71.

Kausal

(2000). A comparison of the defense acquisition systems of Australia, Japan, South Korea, Singapore, and the United States. Fort Belvoir, VA: Defense Systems Management College Press.

72.

Kelman

(1990). Procurement and public management: The fear of discretion and the quality of government performance. Washington, DC: American Enterprise Institute Press.

73.

Kronenberg

(1990). Public administration and the defense department: Examination of a prototype. In Wamsley

. (Eds.), Refounding public administration (pp. 274-306). Newbury Park, CA: SAGE.

74.

Ladner

(1983, March-April). The program management office: Who’s really running the store? Program Manager, pp. 33-35.

75.

Lebovic

(1996). Foregone conclusions: U.S. weapons acquisition in the post-cold war transition. Boulder, CO: Westview.

76.

Lockwood

(1985). Defense procurement managers and weapons acquisition reform (CRS Report 86-13F). Washington, DC: Congressional Research Service.

77.

Lockwood

(1986). The Packard Commission report and defense acquisition organization (CRS Report 86-717). Washington, DC: Congressional Research Service.

78.

Lockwood

(1990). Defense procurement reform (CRS Issue Brief IB89015). March 2. Washington, DC: Congressional Research Service.

79.

MacNeil

Metz

(1956). The Hoover report, 1953–1955. New York, NY: Macmillan.

80.

March

J. G.

Olsen

J. P.

(1984). The new institutionalism: Organizational factors in political life. American Political Science Review, 78, 734-749.

81.

March

J. G.

Olsen

J. P.

(2005). Elaborating the new institutionalism (Working Paper No. 11), March. Center for European Studies, University of Oslo.

82.

Mavroules

(1991). Creating a professional acquisition workforce. National Contract Management Journal, 24(2), 15-33.

83.

Mayer

(1991). The political economy of defense contracting. New Haven, CT: Yale University Press.

84.

Mayer

Khademian

(1996). Bringing politics back in: Defense policy and the theoretical study of institutions and processes. Public Administration Review, 56, 180-190.

85.

McKinney

Gholz

Sapolsky

(1994). Acquisition reform. MIT Lean Aircraft Initiative Policy Working Group, Working Paper #1, Cambridge, MA.

86.

McNaugher

(1989). New weapons, old politics: America’s procurement muddle. Washington, DC: Brookings Institution.

87.

McVeigh

(1995, January-February). The right stuff—revisited: A competency perspective of Army program managers. Program Manager, pp. 30-34.

88.

Meyer

Rowan

(1977). Institutionalized organizations: Formal structure as myth and ceremony. American Journal of Sociology, 83, 340-363.

89.

Morris

(2002). Research trends in the 1990s. In Slevin

Cleland

Pinto

(Eds.), The frontiers of project management research (pp. 31-56). Newtown Square, PA: Project Management Institute.

90.

Munechika

(1997). Acquisition reform: This, too, shall pass (Research Paper AU/ACSC/ 0385/97-03). Maxwell Air Force Base, AL: Air Command and Staff College.

91.

Murphy

Baker

Fisher

(1974). Determinants of project success. Boston, MA: National Aeronautics and Space Administration.

92.

Office of the Director of Acquisition Education, Training, and Career Development. (2001). Program manager (PM) and program executive officer (PEO) tenure: Military services/joint/DoD programs (Unpublished report). Washington, DC: Office of the Secretary of Defense.

93.

Oliver

(1991). Strategic responses to institutional processes. Academy of Management Review, 16, 145-179.

94.

Osborne

Gaebler

(1992). Reinventing government: How the entrepreneurial spirit is transforming the public sector. Reading, MA: Addison-Wesley.

95.

Peck

Scherer

(1962). The weapons acquisition process: An economic analysis. Boston, MA: Harvard Business School.

96.

Perry

(2007). Democracy and the new public service. The American Review of Public Administration, 37, 3-16.

97.

Pinto

Slevin

(1988). Project success: Definitions and measurement techniques. Project Management Journal, 19(1), 67-72.

98.

Pollitt

Bouckaert

(2011). Public management reform: A comparative analysis of new public management, governance, and the neo-Weberian state. New York, NY: Oxford University Press.

99.

Posner

(1987). What it takes to be a good project manager. Project Management Journal, 28 (March), 51-54.

100.

Preston

(2011, April 1). Telephonic interview by author [Digital recording]. Alexandria, VA.

101.

Quirk

Terasawa

(1986). Sample selection and cost underestimation bias in pioneer projects. Land Economics, 62, 192-200.

102.

Radin

(2006). Challenging the performance movement: Accountability, complexity, and democratic values. Washington, DC: Georgetown University Press.

103.

Rendon

Snider

(2008). Project management concepts. In Rendon

Snider

(Eds.), Management of defense acquisition projects (pp. 1-16). Reston, VA: American Institute of Aeronautics and Astronautics.

104.

Rich

Dews

(1986). Improving the military acquisition process: Lessons from RAND research (R-3373-AF/RC). Santa Monica, CA: RAND Corporation.

105.

Ritschel

J. D.

(2013). Impact of political-economy variables on cost growth in military weapon system contracts. Journal of Public Procurement, 13(4), 516-538.

106.

Roquebert

Phillips

Westfall

(1996). Markets vs. management: What “drives” profitability? Strategic Management Journal, 17, 653-664.

107.

Rosenbloom

(1998). Public administration: Understanding management, politics, and law in the public sector. New York, NY: McGraw-Hill.

108.

Savas

(1982). Privatizing the public sector: How to shrink government. Chatham, NJ: Chatham House.

109.

Skandalis

Liargovas

Merika

(2008). Firm management competence: Does it matter? International Journal of Business and Economics, 7, 167-180.

110.

Snider

(1996). DAWIA and the price of professionalism. Acquisition Review Quarterly 3(2): 97–108.

111.

Soloway

(2011, April 6). Personal interview by author [Digital recording]. Arlington, VA.

112.

Taylor

(2009). Strengthening the link between performance measurement and decision making. Public Administration, 87, 853-871.

113.

Terry

(1998). Administrative leadership, neo-managerialism, and the public management movement. Public Administration Review, 58, 194-200.

114.

Thamhain

(1991). Developing project management skills. Project Management Journal, 22(3), 39-44.

115.

Thompson

Jones

(1994). Reinventing the Pentagon: How the New Public Management can promote institutional renewal. San Francisco, CA: Jossey-Bass.

116.

Thompson

(1993). Matching responsibilities with tactics: Administrative controls and modern government. Public Administration Review, 53, 303-318.

117.

Tolbert

P. S.

Zucker

L. G.

(1983). Institutional sources of change in the formal structure of organizations: The diffusion of civil service reform, 1880-1935. Administrative Science Quarterly, 28, 22-39.

118.

Wilson

J. R.

(2011). Steel on target: JM&L LCMC delivers precision munitions for the Warfighter (U.S. Army Materiel Command 2010-2011). Fort Belvoir, VA: U.S. Army Materiel Command.