Managing Performance for Capital Projects

Abstract

This study contributes to our understanding of public performance with original research on Earned Value Management (EVM), a practice used for managing cost and schedule performance for capital projects throughout execution. Recently, EVM has emerged in some states as part of their performance management strategy for major Information Technology acquisitions but research has yet to study implementation. Overall, results indicate that EVM can support proactive cost and schedule management and help achieve performance objectives but that it is not fully embedded into work routines. Findings also illustrate that successful implementation is contingent on several organizational, human capital, and policy variables.

Keywords

performance management project management policy implementation performance budgeting capital budgeting earned value management

Introduction

In public administration, the challenges of infrastructure project management are hardly a contemporary phenomenon. The historical record overflows with high-profile blunders that illustrate the tendency of major projects to experience benefit shortfalls, cost overruns, and schedule delays (Flyvbjerg 2017). Still, because the landscape of government acquisitions continues to grow in terms of cost, complexity, and diversity, effective project management remains a top priority for improving performance (Blair 2015; Grandage 2021; U.S. Government Accountability Office 2021). Yet, despite these well-documented challenges, academic research on public sector project management remains underdeveloped (Grandage 2021; Kassel 2016).

Research on public management and performance has focused on organizations, agencies, and programs as units of analysis (Bouckaert and Halligan 2008; Moynihan 2008; O’Toole and Meier 1999; Rainey and Steinbauer 1999; Talbot 2010; Walker, Boyne and Brewer 2010). A plethora of studies have examined multiple dimensions of performance and potential sources of public service improvement using a variety of methodological approaches (Boyne 2003; Walker, Boyne and Brewer 2010). However, relatively few studies featuring projects as units of analysis have been conducted (Grandage 2021; Mitchell 2019), contributing to a knowledge gap on what strategies are being implemented, which ones are successful, and what makes them effective.

The purpose of this study is to help fill this knowledge gap with original research on Earned Value Management (EVM). EVM is a proactive strategy for managing some of the headline dimensions of performance for capital projects in real-time. Specifically, it provides an integrated solution for establishing project milestones, analyzing cost and schedule performance on a monthly basis, and considering corrective action plans based on early warning indicators (Fleming and Koppelman 2010; Grandage 2021; U.S. Government Accountability Office 1997, 2009a).

EVM has been used in the United States (U.S.) federal government for over 50 years, evolving to become an internationally recognized best practice in project management (Fleming and Koppelman 2010; Project Management Institute 2011; Song 2010; U.S. Government Accountability Office 2009a). Overall, research indicates that EVM is an effective performance management tool, helping improve communication, provide early warning of performance issues, and facilitate corrective action (Kim, Wells and Duffey 2003; Song 2010). However, research on EVM in state and local governments has yet to receive adequate coverage (Grandage 2021). As a result, we lack specific information on the contributions, if any, of EVM to project management within this situational context and factors critical to effective implementation.

This research analyzes EVM implementation for major Information Technology (IT) acquisitions in U.S. state governments. Successfully delivering IT projects has proved to be very challenging in both the private and public sector, with reports of up to 25 percent considered failures and cost overruns of 50 percent not uncommon for completed projects (Flyvbjerg 2017; Standish Group 2015). As state governments have modernized their IT infrastructure and navigated these risks, they have formed statewide Project Management Offices (PMOs) to improve project delivery for major agency and enterprise IT systems, with some using EVM as part of their performance management strategy (Grandage 2021).

This study has two primary research questions. First, what are the contributions to project management, if any, from EVM? Second, what factors are critical for effective conduct of EVM? A multi-method research design including surveys, interviews, case studies, and focus groups is utilized. Results from 31 states indicate that roughly half currently use EVM, with the majority adopting it within the last ten years. States that use EVM report several contributions to IT capital project management, such as improved communication, proactive management of risk, early warning of performance issues, and more consistently delivering projects on time and within budget. However, several organizational, human capital, and policy factors are decisive for realizing benefits. Ultimately, the findings align with previous research, but illustrate that EVM practices in U.S. state governments are a relatively new addition to the performance management arena that are not fully embedded into existing work routines.

Earned Value Management

Before introducing EVM, it is helpful to provide a general description of a project monitoring system. A project monitoring system is designed to capture metrics throughout execution by establishing performance objectives, recognizing progress as activities are completed, and analyzing results. For example, when constructing a new facility, time and cost objectives are set for specific activities, such as land clearing and completing the foundation. A simple way of monitoring cost performance would be to compare the planned and actual costs once activities are completed. However, a reactive approach such as this may obscure early warning signals and reduce the likelihood of successfully implementing corrective actions when they are needed, which is problematic for risky acquisitions.

A distinctive aspect of EVM is that it offers a solution to this problem. Specifically, it provides visibility of cost and schedule performance for activities while they are in the process of being completed (Fleming and Koppelman 2010; Grandage 2021). Returning to the construction example, a manager using EVM devises ways of tracking cost performance in real-time, such as comparing the percentage of land clearing work completed with the cost of performing that work to date. For example, such comparisons may reveal the work is only 30 percent complete but that 50 percent of the budget was used to achieve that production, identifying the overrun to date for that activity and risk for cost growth in completing it. Regarding schedule performance, EVM is equipped to analyze it too by comparing planned and actual production in units of time or money (Khamooshi and Golafshani 2014). In sum, one of the key working assumptions of EVM is that early detection is beneficial but requires real-time information on project status (Grandage 2021).

To provide this information, EVM specifies rules for recognizing progress on at least a monthly basis and procedures for analyzing cost and schedule performance. Furthermore, its reporting structure provides granular data on the performance of different functional groups and contractors, meaning that project stakeholders can more easily identify mediocre contributors. Finally, EVM is more than simply a monitoring system because it prescribes best practices for planning, scheduling, cost-planning, baseline revision, etc. (Project Management Institute 2011; U.S. Government Accountability Office 2009a). Therefore, it is considered a comprehensive strategy for managing some of the headline dimensions of performance in real-time.

The primary appeal of EVM for managers should be understood in terms of information quality, specifically timeliness and specificity. Simply put, without in-progress evaluation for deliverables and a granular reporting structure, the data generated for project status can be too stale and aggregated to facilitate swift adjustments (Grandage 2021). However, it is also important to note for the purposes of oversight that prescribing best practices helps ensure disciplined execution more generally and that a common set of metrics eases portfolio analysis (Christensen 1998; Grandage 2021; U.S. Government Accountability Office 1997, 2009a).

EVM has a long history in the public sector, having been adopted by the U.S. Department of Defense (DoD) in the 1960s to improve project management for major acquisitions (Fleming and Koppelman 2010; Grandage 2021; Kwak and Anbari 2012). Following the DoD's adoption, the practice diffused to several U.S. federal agencies and other national governments (Abba 2000; Song 2010). As EVM evolved in the U.S. federal government, concerns grew that too many burdensome reporting requirements were imposed on project managers by the financial community that governed the process (Christensen 1998; Grandage 2021; U.S. Government Accountability Office 1997). Therefore, amidst the general reinventing government movement and specific “insight not oversight” philosophy of acquisition reforms in the 1990s, the process was reformed to be more compatible with the needs of project managers (Christensen 1998; Fleming and Koppelman 2010; Grandage 2021; U.S. Government Accountability Office 1997).

Project management research has explored the potential benefits of EVM application and key success factors. Overall, the literature indicates that EVM can serve as an effective performance management tool but that successful implementation is contingent on numerous factors (Grandage 2021). Specifically, studies on potential benefits have found that EVM can help improve communication, proactively manage risks, and more consistently deliver projects on time and within budget (Batselier and Vanhoucke 2015a, 2015b; Kim, Wells and Duffey 2003; Song 2010). In particular, quantitative studies have provided evidence of early warning capabilities and qualitative research using surveys, interviews, and case studies have been used to understand improvements in cost and schedule performance (Grandage 2021).

Regarding key success factors, the literature has illustrated the importance of leadership and organizational culture, human capital, conduct of oversight, training programs, policy standards and guidelines, reporting tools, tailoring of practices to fit project context, and other tactical details (Christensen 1998; Fleming and Koppelman 2010; Grandage 2021; Kim, Wells and Duffey 2003; Song 2010; U.S. Government Accountability Office 1997, 2009a, 2009b, 2020). Here too, qualitative studies have provided rich insight into EVM practices. However, with relatively few exceptions (Grandage 2021; Kim, Wells and Duffey 2003; Song 2010; U.S. Government Accountability Office 2009a), these factors are not integrated into conceptual frameworks to inform policy implementation in different settings. Furthermore, industry standards for EVM are generic in nature and not intended to be applied uniformly in an organization. Therefore, it is worthwhile to review the EVM literature and apply it to new implementation environments, especially since much of the knowledge on public sector use has been derived from national defense.

EVM Framework

As Figure 1 shows, the framework identifies several EVM success factors and places them into three groupings: organization, human capital, and policy. The framework can be considered a top-down model of policy implementation because it focuses considerable attention on factors that can be structured by policymakers (Mazmanian and Sabatier 1989; O’Toole 2000). Specifically, following Mazmanian and Sabatier (1989), it recognizes the importance of delegation to loyal units with sufficient hierarchical integration (organization), skillful implementation (human capital), and clear and consistent policy objectives (policy).

Figure 1.

EVM Implementation Factors.

The organizational factors address the need for traditional governance functions, such as policy development and oversight. However, they also recognize the challenges of modifying existing work routines when introducing a new performance management system (Moynihan and Pandey 2010), especially one understood to benefit from project management maturity (Grandage 2021). In particular, a significant precondition for effective implementation is proficiency with core project management competencies and EVM specific training (Grandage 2021; Kim, Wells and Duffey 2003; Song 2010; U.S. Government Accountability Office 2009a, 2012, 2016, 2021). Therefore, to develop these capabilities, PMOs can provide training, education, and consulting services. Ideally, when implementing these changes, the PMO and agency assess current capabilities and future needs to determine if EVM training is appropriate (Grandage 2021; Project Management Institute 2018). As Grandage (2021) explains, this attention to knowledge generation and change management in the PMO setting squares with Moynihan’s (2008) perspective on the role central offices can play in fostering the effective use of performance information at the agency level.

Consistent with ideas from top-down models of policy implementation, performance budgeting research in state government has indicated that specific guidance on the development, reporting, and use of performance information can contribute to stronger use (Lu, Willoughby and Arnett 2011), and the same applies to EVM (Grandage 2021; U.S. Government Accountability Office 2009a). Specifically, organizations must establish industry standards, clearly define criteria for EVM use, and tailor practices according to project context (Grandage 2021; Song 2010; U.S. Government Accountability Office 2009a). Industry standards provide an authoritative source for defining project management guidelines, benchmarking, and specifying practices that contractors are expected to use when performing work on behalf of the government (Grandage 2021). To support clarity of process, organizations often specify when EVM should be applied based on cost, duration, contract type, and overall assessments of risk (Grandage 2021; National Defense Industrial Association 2016; Song 2010).

In addition to supporting clarity of process, attention to these parameters helps with tailoring practices according to project context. Ultimately, tailoring ensures that practices fit with situational context, both in terms of general risk and particulars of the implementation environment (Grandage 2021; Mitchell 2019; Project Management Institute 2018). For example, analysis of cost, duration, contract type, and uncertainty can be used to assign projects into different risk categories, with guidance given on how practices should vary among them. Regarding EVM, it may be applied more intensively for high-risk projects, less intensively for medium-risk ones, and perhaps not at all for low-risk efforts. For example, an intensive application of EVM may entail independent reviews of the performance baseline, bi-weekly status reports, and sensitive tolerance limits for variance reporting and corrective action plans. Lastly, tailoring can also address tactics that are sector specific by illustrating how EVM can be embedded within existing work routines (Grandage 2021).

The final policy components of the framework concern independent reviews of the performance baseline and development of corrective action plans. Typically, the U.S. federal government requires an Integrated Baseline Review (IBR) to be conducted within six months of a major contract award (National Defense Industrial Association 2019). The purpose of doing an IBR is to establish a mutual understanding of key risks and the strategies that will be used during execution to manage them. A key benefit of performing an IBR is that it may uncover specific project risks or management weaknesses that otherwise may only be discovered later on (National Defense Industrial Association 2019; U.S. Government Accountability Office 2009a). Finally, policy must be clear in specifying that corrective action plans should be developed for major risks and be considered based on EVM indicators. In sum, the independent reviews help set realistic baselines upfront and continuous EVM monitoring is used to track deviations from plan during execution, ensuring proactive cost and schedule management throughout the project lifecycle.

Methods and Data

In developing the framework, concepts from grounded theory were applied. In short, grounded theory is an inductive technique for building theory from the data of social research (Glaser and Strauss 1967). The strategy is considered appropriate for this research, given the exploratory nature and need for theoretical synthesis. To discover factors critical to effective EVM implementation, the study performed a comprehensive literature review, categorized industry practice guidance, examined policies issued by federal agencies and state governments, and conducted content analysis of governmental reports. This allowed EVM implementation tactics to emerge inductively and for their properties to be defined iteratively as new information was collected. The analysis was completed once a point of theoretical saturation had been met, meaning that additional sources of information did not identify new tactics or properties of them.

Next, steps were taken to delimit the framework to a reasonable number of items that EVM users in state government could be surveyed on while maintaining content validity. Specifically, an initial strategic framework was developed and a focus group was held with several EVM experts at The U.S. Government Accountability Office (GAO). The experts provided comments on the literature review and helped refine the set of components outlined in the framework and properties of them. Once the theoretical development was complete, a multi-method design was used to collect the data.

A multi-method design involves the use of multiple qualitative approaches. Here, a combination of surveys, interviews, case studies, and focus groups were utilized. To help guide the development of the questionnaire, previous EVM survey research thoroughly piloted in the project management community was used (Song 2010). Following the tailored design method, a social exchange strategy was implemented to establish trust with participants and incentive response (Dillman, Smyth and Christian 2014). Specifically, participants were initially contacted through the physical mail using official University envelopes and letterhead. The letters informed subjects on the nature of the study and that they had been carefully selected based on their position and expertise. To encourage their participation, they were promised a summary report and an invitation to a collaborative webinar to share lessons learned with other officers in state government.

In most states, letters were sent to the Director of the statewide PMO. For states that a PMO Director could not be identified, Chief Technology Officers were contacted via mail. Each of the 50 states were contacted and a total of 31 participated in the study, of which roughly half currently use EVM. Regarding the individual participants, 60 percent have worked in project management for more than a decade and 25 percent have more than 20 years of experience. Over half of the participants have more than 10 years of experience managing IT projects in state government in particular. Thus, the perspectives of these PMO Directors offer rich insight, both in managing their own enterprise projects and overseeing agency ones. Concerning the composition of the states, they are representative in terms of population, region, and political orientation. In passing, it should be noted that states currently using EVM for major IT acquisitions tend to have greater capital expenditures and more regularly deal with costly and complex investments (Grandage 2021).

To collect perceptual data on contributions to project management and factors affecting implementation, the survey uses a series of Likert scale questions. The question stems were designed to ensure that responses captured experiences specific to IT project management in state government. Five-point response scales were used, with labels indicated for each point. In determining the number of points to include, five was selected because research indicates that data from Likert items may lose validity if the number of points is less than five or greater than seven (Colman, Norris and Preston 1997; Miller 1956).

To get additional explanatory context on the survey results, interviews, case studies, and focus groups were conducted. A total of 11 semi-structured interviews were done as follow-up to the survey and lasted from 30 to 60 minutes. Three case studies were conducted that chronicled the origin and evolution of PMOs and their use of EVM. To complete the social exchange strategy, a summary report was sent to the 31 participating states and three webinars were held that served as focus groups to disseminate knowledge of PMO operations and EVM application from the study. Finally, a previous version of this paper was reviewed by practitioners at GAO for feedback on the final product.

Results

What Contributions, if any, Does EVM Provide to IT Project Management in State Government?

Table 1 provides a frequency tabulation for each potential contribution of EVM evaluated by the 16 officers from states that currently use it. Specifically, Table 1 shows statements the participants were asked, response categories, and distributions of the ordinal responses. The median responses are bolded. The ordinal responses were converted to numerical scores to provide another measure of central tendency. Overall, most users agree that EVM is an effective performance management tool and point to specific benefits.

Table 1.

Frequency Tabulation for Contributions of EVM (N = 16)

Based on your experiences in state government managing IT projects, indicate your perception regarding potential contributions to project management from EVM:
	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Mean (Range 1–5)
EVM is an effective early warning system for cost overruns			4	8	4	4
EVM is an effective early warning system for schedule delays		2		12	2	3.9
EVM helps prevent scope creep		5	4	7		3.1
EVM indicators are effective in communicating project status in portfolio management		1	2	10	3	3.9
EVM helps facilitate corrective action		1	2	11	2	3.9
EVM helps achieve cost objectives			3	13		3.8
EVM helps achieve schedule objectives		1		14	1	3.9
Overall, EVM is an effective performance management tool			2	12	2	4

Regarding early warning capabilities, participants were asked about using EVM indicators for analyzing performance to date and forecasting total project costs and duration. Most often, if cost indicators show a potential overrun of 10 percent or higher, then projects are flagged and adjustments are considered. Generally speaking, officers indicated that the timely and detailed structure of EVM reporting helps catch problems early, pinpoint the source of variance, and facilitate discussions regarding its root case, potential impacts, and adjustments that may be needed. As one PMO Director explained regarding their enterprise projects:

We’ve had several projects that we successfully achieved all objectives. Those projects were successful because of proper planning. When performance measures for the cost criteria or schedule criteria reach thresholds, then adjustments were made.

However, users are hesitant to apply EVM indicators in the early stages of projects for the purposes of forecasting, preferring for projects to stabilize first. Furthermore, officers indicated that EVM was one of many techniques used for forecasting, with one of its chief purposes being a simple sanity check on bottoms-up estimates given from project managers.

One area that received a lukewarm evaluation was prevention of scope creep. As Table 1 shows, the median response was neutral and it has the lowest mean. Some officers did indicate that variance reporting may help detect work being performed that is not authorized. However, most users simply did not view EVM as a scope management tool, considering it more useful for managing cost and schedule performance. In place of EVM, other strategies are used to ensure that project stakeholders have a common understanding of what needs to be delivered, such as change control from a contracting approach.

Table 1 shows that users mostly consider EVM to be effective for communicating status in portfolio management. Several participants explained the importance of an objective and common set of metrics being used for portfolio reporting. For example, prior to EVM reforms in one state, project managers were given discretion in how they reported status using stoplight charts. In a few cases, indiscretion obscured early warning signals that EVM data would have easily provided, leading to missed opportunities for timely corrective action. In decentralized states, PMO directors were asked to explain how their experiences differed with EVM users and non-users. One officer gave a comment that was reflective of others:

A project using EVM is more likely to be aware of their project status, communicate, and meet its technical, schedule, and cost objectives than a project that does not.

Participants also favorably evaluated EVM's ability to facilitate corrective action and help achieve cost and schedule objectives. In discussing examples with participants, a few themes emerged. First, at the most basic level, officers indicated that EVM was critical in meeting performance objectives simply because it enabled real-time tracking of them. Second, the timely and targeted information enables problems to be caught earlier and for specific corrective action plans to be implemented swiftly. Third, it has been introduced to deal with problems on specific projects when they arise and help moving forward. An illustrative example comes from a state where a vendor failed to meet acceptance for a major deliverable, resulting in a costly stalemate between the agency and vendor. As an officer explained, EVM played a pivotal role in restarting the program and providing a path going forward:

EVM provided us with some measurements that both the State and vendor could agree on during the renegotiation of the contract. In the process, we were able to educate the agency on how EVM can serve as an early warning for project and vendor performance. It should serve the agency well in avoiding this type of situation on future projects.

Fourth, EVM indicators have been used to help inform a variety of decisions, such as adding time or money to a project, or in select cases, cancelling projects because they will clearly be too costly in relation to their benefits. For example, a Director from a state that recently adopted EVM noted:

In major enterprise initiatives, we have been able to use EVM as a way to justify additional procurement activity or budgetary needs in planning as opposed to leveraging the emergency process.

Finally, another officer shared that EVM was used to inform a key decision on project cancellation:

In one case, the use of EVM helped the project oversight team decide that a project was unsustainable and that it was better to cancel a project rather than to add millions more in budget and extension of schedule.

What Factors are Critical to EVM Implementation?

Table 2 provides a frequency tabulation for factors affecting EVM implementation. Results indicate that several organizational, human capital, and policy factors are very important for effective conduct of EVM. State officials explained that, in the early stages of PMO existence, their offices focused on setting the foundation for effective project management statewide and managing change as new practices were implemented. Importantly, many officers stressed the importance of evolving from an initial focus on standards and compliance to becoming a trusted partner in supporting agency driven changes.

Table 2.

Frequency Tabulation for Factors Affecting EVM Implementation (N = 16)

Based on your experiences in state government managing IT projects, indicate your perception of the potential importance of the following factors to successful EVM implementation:
	Not at all important	Slightly important	Moderately important	Very important	Extremely important	Mean (Range 1–5)
Establishment of a centralized project management office			7	6	3	3.8
EVM training programs		1	6	7	2	3.6
Project Management Professionals (PMP)	3	4	5	1	3	2.8
Contractor experience with EVM		3	11	1	1	3
Adoption of EVM industry standards		2	6	7	1	3.4
Tailoring of EVM procedures for project context		1	3	11	1	3.8
Independent reviews of the performance baseline		1	6	9		3.5
Development of corrective action plans based on EVM indicators		1	6	9		3.5

Regarding EVM, results converged on the importance of knowledge and change management. Simply put, teams need to be proficient with core project management competencies and have a plan for embedding EVM into their existing work routines. For example, one state attempted to introduce EVM statewide following a high-profile project failure and scathing audit findings. In fact, an officer from the state described the rollout as “guns-blazing…across the board”. Unsurprisingly, this shotgun marriage with EVM failed, with the main outcome being the PMO learning which competencies they would need to develop before using EVM in the future. To this point, an officer provided a comment reflective of others:

If implemented and supported by leadership, PMOs can effectively establish the disciplines and guidelines required for projects to use EVM practices. PMOs should also be prepared to create effective training programs and conduct periodic reviews of projects to ensure correct usage of EVM.

Finally, some officers shared that they are considering pulling major projects out of agencies that irregularly deal with them and managing the projects themselves. In short, this helps avoid building capacity where it is not needed and ensures that experienced project managers selected by the PMO are responsible for major projects.

Ultimately, these results illustrate the significant impact PMOs can have in ensuring skillful implementation. However, officers provided mixed reviews on the importance of certified Project Management Professionals (PMP) and contractor experience with EVM. Regarding PMPs, officers emphasized the general importance of this certification. However, for EVM in particular, they considered EVM specific training, especially on-the-job training, to be more important than formal certifications. For contractor experience, one officer explained that reforms requiring all vendors to comply with EVM requirements has improved their ability to manage deliverables. Nevertheless, officers did not consider contractors to be especially problematic and placed more emphasis on their internal use of EVM than external challenges.

Regarding the policy factors, results are consistent with expectations. Specifically, industry standards were considered important for prescribing best practices for core functions, such as scope definition, scheduling, and cost-planning. Furthermore, tailoring of practices was reported to be critical for both the acceptance of EVM and its effective use. Thus, results point to the importance of both standardization and attention to situational context, but with caveats. That is, standardization helped provide needed structure in the early stages of reform for defining simple and repeatable core practices. However, attention to situational context takes on great importance for embedding more complex and flexible solutions into routines, such as EVM, and is perhaps even more important than standardization itself in the long-run.

Conducting independent reviews of performance baselines was found to be an important complement to EVM. Officers reported use of these reviews to be helpful in identifying major risks before projects begin and advance to subsequent phases. Furthermore, consideration of these risks has assisted in anticipating potential corrective action plans that may need to be implemented. For example, an officer explained benefits they have realized from “Independent Verification and Validation” (IV&V), which serves many of the same purposes of an IBR:

IV&V helps reduce the likelihood of project failure by identifying risks early and ensuring that management is aware of these risks and what the potential impact is for things like schedule, cost, and quality.

However, at the same time, officers reiterated the importance of internal capabilities and continuous monitoring, recognizing the limitation of periodic outside reviews and their costs. As an officer explained to support this point:

IV&V can be an effective compliment to project management to improve the likelihood of project success, but it isn't right for every project and isn't the most efficient approach in many cases….establishing sound PM practices/standards and engaged leadership are often more efficient.

Finally, during the course of the study, a promising strategy emerged. In Georgia, the PMO surveys project team members and stakeholders for their assessments on project status. Specifically, they use a series of Likert-scale questions that capture the extent to which there is agreement that EVM indicators accurately reflect project status, while also gathering information on other dimensions of performance. If EVM indicators and survey results align, this lends confidence to the metrics reported by the project manager. If not, this provides the PMO an opportunity to hear concerns earlier than they otherwise would communicating strictly with the project manager. As an officer from Georgia explained, this can be a major change for project managers who are accustomed to owning project status. However, this participatory evaluation technique can easily gain acceptance, given that it values the input of team members and provides greater visibility to project stakeholders.

Discussion

The contributions of this research span public performance, project management, and budgeting. Research on public performance has extensively studied organizations, agencies, and programs (Bouckaert and Halligan 2008; Lu and Willoughby 2018; Moynihan 2008; O’Toole and Meier 1999; Rainey and Steinbauer 1999; Talbot 2010; Walker, Boyne and Brewer 2012). However, studies featuring projects as units of analysis are lacking, with relatively few providing context on sources of public service improvement (Grandage 2021; Mitchell 2019).

This study helps fill this knowledge gap with original research on EVM in U.S. state governments for major IT acquisitions. Overall, results indicate that EVM can be an effective performance management tool but that successful implementation is contingent on several organizational, human capital, and policy variables. In particular, officers from 16 states that currently use EVM report that it improves communication, provides early warning of cost overruns and schedule delays, supports corrective action, and helps achieve cost and schedule objectives. However, officers gave mixed reviews on its potential to prevent scope creep and reported limitations of using EVM indicators in the early stages of projects for the purposes of forecasting. Therefore, results support the observation that EVM can be an effective cost and schedule performance management tool, especially when used in conjunction with other sound contracting and financial management practices (Grandage 2021; U.S. Government Accountability Office 2009a; Song 2010).

Importantly, the study found several factors to be critical for effective implementation, some dealing with general PMO operations such as policy formulation, knowledge development, and change management. For policy formulation, both standardization and attention to situational context matter in their own ways, the benefits of the former mostly limited to defining simple and repeatable practices based on industry standards in the early stages of PMO existence. In contrast, attention to situational context fosters acceptance of change and the effective use of more complex processes needing flexibility, such as EVM. Furthermore, knowledge management efforts have provided foundational skills, with some PMOs describing their journey as evolving to focus on organizational learning and supporting agency driven changes. Thus, at the organizational level, results point to the relative importance of knowledge development, flexibility of process, and change management in comparison to standardization and other command-and-control tactics.

Regarding more specific practices, the findings on independent reviews of performance baselines are also nuanced. That is, officers report that they contribute to proactive management of risk but recognize the shortcomings of relying on periodic external reviews, the cost of conducting them, and the relative importance of constant engagement. Of the EVM practices observed in this study, the most intriguing was arguably the participatory evaluation technique used by Georgia where team members have a formal say in reporting project status. From a purely technical perspective, this process provides greater visibility of project performance. However, recognizing that use of performance information is a type of organizational behavior (Moynihan and Pandey 2010), the unique potential of this approach is that it respects the viewpoints of team members, facilitates open communication, and, therefore, can build trust and legitimacy in the system itself. In sum, the study indicates that EVM may function most effectively when used in combination with external reviews at major checkpoints and participatory evaluation throughout execution.

The findings from this study also contribute to project management research, with novel analysis of EVM implementation in U.S. state governments. As noted in the literature review, much of the research on public sector EVM has focused on the U.S. federal government, with knowledge of subnational governments practices underdeveloped (Grandage 2021). In addition, the study sheds light on how PMO operations can support effective EVM implementation. Overall, results align with the EVM body of knowledge, both in terms of contributions to project management and factors critical to effective implementation (Kim, Wells and Duffey 2003; Song 2010; U.S. Government Accountability Office 2009a).

Given EVM's role in managing cost for capital projects, this research also contributes to public budgeting. Specifically, normative models of capital budgeting emphasize the importance of project management for efficiently delivering capital projects (Srithongrung, Ermasova and Yusuf 2019). Yet, empirical studies of implementation strategies are lacking (Grandage 2021; Mitchell 2019). Moreover, research has found that performance information is arguably most useful for execution when considering each phase of the budgeting process (Ho 2018; Lu and Willoughby 2018; Moynihan 2008). Thus, the study contributes with original research on the use of real-time metrics for capital projects but also squares with foundational literature relating to the relative importance of performance information for budget execution.

Finally, an impressive body of scholarship has emerged on managing complex contracts in the public sector (Brown, Potoski and Van Slyke 2013, 2018). In brief, this research has been important for providing context on the challenges of acquisition management from a contracting perspective and guidance for structuring rules and managing relationships. However, relatively little research has been conducted on the project management tactics used when purchasing these complex products in conjunction with contracting strategies (Grandage 2021). Therefore, for public management research to further inform the practice of government acquisitions, project management literature can integrated into forthcoming studies.

Conclusion

It is hard to imagine a government that could not benefit from improved project management. Unfortunately, public administration research has yet to treat the topic seriously, with inadequate coverage of what practices are being implemented, which ones are successful, and what makes them work. This study contributed our understanding of managing performance for capital projects with original research on EVM in state government. Although EVM has been a major feature of project management research, studies of use at the subnational level are lacking. Therefore, we have lacked evidence on the results of implementation within this situational context and key success factors.

For state and local governments, the key takeaway is that EVM can support proactive cost and schedule management and help achieve project performance objectives. However, to effectively manage scope creep and develop accurate forecasts, other contracting and financial management practices are necessary. Findings also show the relative importance of knowledge development, change management, and flexibility of process when compared to standardization and other command-and-control design features. For optimal results, governments should consider use of participatory evaluation techniques to provide greater visibility of project performance and facilitate open communication. That said, the practice of EVM is not right for every organization, especially those that are still building foundational project management skills or irregularly deal with major acquisitions. Nevertheless, all things considered, the potential applications of EVM in state and local government are vast, especially for transportation, water and sewer, and construction of major facilities.

Future research can build on some of the limitations of this study. Given the low number of participants, surveys of additional users are needed. Furthermore, studies must be conducted in different sectors to better understand the generalizability of these results. Heeding the policy implementation literature, this small-N study should be considered a primer for large-N studies (Goggin 1986). Specifically, the contributions of EVM represent a set of outcomes to be studied with the key success factors serving as independent variables. In particular, quantitative studies would help improve upon the subjective nature of this study, but it is important for researchers to understand they need data on tactics used for specific projects to avoid ecological fallacies (Mitchell 2019), especially given the decentralized nature of project delivery (Grandage 2021). Of course, project performance is a multidimensional concept and there are many potential sources of public service improvement aside from EVM needing research. After all, there are more important things for government performance than performance management systems (Moynihan 2008).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Andrew J. Grandage

References

Abba

Wayne

. 2000. “How Earned Value Got to Primetime: A Short Look Back and a Glance Ahead.” Project management institute seminars and symposium, Houston, TX.

Batselier

Jordy

Vanhoucke

Mario

. 2015a. “Empirical Evaluation of Earned Value Management Forecasting Accuracy for Time and Cost.” Journal of Construction Engineering and Management 141 (11): 05015010. https://ascelibrary.org/doi/abs/10.1061/(ASCE)CO.1943-7862.0001008

Batselier

Jordy

Vanhoucke

Mario

. 2015b. “Evaluation of Deterministic State-of-the-art Forecasting Approaches for Project Duration Based on Earned Value Management.” International Journal of Project Management 33 (7): 1588–96. https://doi.org/10.1016/j.ijproman.2015.04.003

Blair

Dan G

. 2015. “Enhancing Trust in Government: Adoption of Program Management Discipline Can Help Government Improve Program Delivery and Service.” Public Administration Review 75 (6): 789–90. https://onlinelibrary-wiley-com-443.web.bisu.edu.cn/doi/10.1111/puar.12460

Bouckaert

Geert

Halligan

John

. 2008. Managing Performance: International Comparisons. London: Routledge.

Boyne

George A

. 2003. “Sources of Public Service Improvement: A Critical Review and Research Agenda.” Journal of Public Administration Research and Theory 13 (3): 367–94. https://doi.org/10.1093/jpart/mug027

Brown

Trevor L.

Potoski

Matthew

Van Slyke

David M.

. 2013. Complex Contracting. Government Purchasing in the Wake of the US Coast Guard’s Deepwater Program. Cambridge, United Kingdom: Cambridge University Press.

Brown

Trevor L.

Potoski

Matthew

Van Slyke

David M.

. 2018. “Complex Contracting: Management Challenges and Solutions.” Public Administration Review 78 (5): 739–47. https://doi.org/10.1111/puar.12959

Christensen

David

. 1998. “The Costs and Benefits of The Earned Value Management Process.” Journal of Parametrics 18 (2): 1–16. https://doi.org/10.1080/10157891.1998.10462568

10.

Colman

Andrew M.

Norris

Claire E.

Preston

Carolyn C.

. 1997. “Comparing Rating Scales of Different Lengths: Equivalence of Scores from 5-Point and 7-Point Scales.” Psychological Reports 80 (2): 355–62. https://doi.org/10.2466/pr0.1997.80.2.355

11.

Dillman

Don A.

Smyth

Jolene D.

Christian

Leah M.

. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 4th ed. Bibliographies. Hoboken, New Jersey: John Wiley and Sons, Inc.

12.

Fleming

Quentin W.

Koppelman

Joel M.

. 2010. Earned Value Project Management. 4th ed. Newton Square, Pennsylvania: Project Management Institute.

13.

Flyvbjerg

Bent

. 2017. The Oxford Handbook of Megaproject Management. Oxford: Oxford University Press.

14.

Glaser

Barney G.

Strauss

Anselm L.

. 1967. Discovery of Grounded Theory: Strategies for Qualitative Research. New Yok: Routledge.

15.

Goggin

Malcolm L

. 1986. “The “Too Few Cases/Too Many Variables” Problem in Implementation Research.” The Western Political Quarterly 39 (2): 328–47. https://doi.org/10.1177/106591298603900210

16.

Grandage

Andrew J

. 2021. “Advancing Capital Project Management.” Journal of Public Budgeting, Accounting & Financial Management 33 (5): 618–34. https://doi.org/10.1108/JPBAFM-01-2021-0003

17.

Alfred Tat-Kei

. 2018. “From Performance Budgeting to Performance Budget Management: Theory and Practice.” Public Administration Review 78 (5): 748–58. https://onlinelibrary-wiley-com-443.web.bisu.edu.cn/doi/10.1111/puar.12915

18.

Kassel

David S.

2016. Managing Public Sector Projects: A Strategic Framework for Success in an era of Downsized Government. New York: Routledge.

19.

Khamooshi

Homayoun

Golafshani

Hamed

. 2014. “EDM: Earned Duration Management, a new Approach to Schedule Performance Management and Measurement.” International Journal of Project Management 32 (6): 1019–41. https://doi.org/10.1016/j.ijproman.2013.11.002

20.

Kim

Wells

W. G.

Jr Duffey

M. R.

. 2003. “A Model for Effective Implementation of Earned Value Management Methodology.” International Journal of Project Management 21 (5): 375–82. https://doi.org/10.1016/S0263-7863(02)00049-2

21.

Kwak

Y. H.

Anbari

F. T.

. 2012. “History, Practices, and Future of Earned Value Management in Government: Perspectives from NASA.” Project Management Journal 43 (1): 77–90. https://doi.org/10.1002/pmj.20272

22.

Elaine

Willoughby

Katherine

. 2018. Public Performance Budgeting: Principles and Practice. New York: Routledge.

23.

Elaine

Willoughby

Katherine

Arnett

Sarah

. 2011. “Performance Budgeting in the American States: What’s Law Got to do with it?” State & Local Government Review 43 (2): 79. https://journals-sagepub-com-s.web.bisu.edu.cn/doi/10.1177/0160323X11407523

24.

Mazmanian

Daniel A.

Sabatier

Paul A.

. 1989. Implementation and Public Policy. New York: University Press of America.

25.

Miller

George A

. 1956. “The Magical Number Seven, plus or minus two: Some Limits on our Capacity for Processing Information.” Psychological Review 63 (2): 81. https://psycnet.apa.org/doiLanding?doi=10.1037%2Fh0043158

26.

Mitchell

David

. 2019. “It all depends, but on What? Testing Contingency in Public Strategic Implementation.” Public Performance & Management Review 42 (4): 783–807. https://www-tandfonline-com-s.web.bisu.edu.cn/doi/full/10.1080/15309576.2018.1523063

27.

Moynihan

Donald P.

2008. The Dynamics of Performance Management: Constructing Information and Reform. Washington, D.C.: Georgetown University Press.

28.

Moynihan

Donald P.

Pandey

Sanjay K.

. 2010. “The big Question for Performance Management: Why do Managers use Performance Information?” Journal of Public Administration Research and Theory 20 (4): 849–66. https://doi.org/10.1093/jopart/muq004

29.

National Defense Industrial Association. 2016. U.S. Federal Agency EVMS Policy Summary. Arlington, Virginia: National Defense Industrial Association

30.

National Defense Industrial Association. 2019. Guide to the Integrated Baseline Review (IBR). 3rd ed. Arlington, Virginia: National Defense Industrial Association

31.

O’Toole

Laurence J

. 2000. “Research on Policy Implementation: Assessment and Prospects.” Journal of Public Administration Research & Theory 10 (2): 263–88. https://doi.org/10.1093/oxfordjournals.jpart.a024270

32.

O’Toole

Laurence J.

Meier

Kenneth J.

. 1999. “Modeling the Impact of Public Management: Implications of Structural Context.” Journal of Public Administration Research & Theory 9 (4): 505–26. https://doi.org/10.1093/oxfordjournals.jpart.a024421

33.

Project Management Institute. 2011. Practice Standard for Earned Value Management. 2nd ed. Newtown Square, Pennsylvania. Project Management Institute.

34.

Project Management Institute. 2018. The Standard for Organizational Project Management. Newtown Square, Pennsylvania: Project Management Institute.

35.

Rainey

Hal G.

Steinbauer

Paula

. 1999. “Galloping Elephants: Developing Elements of a Theory of Effective Government Organizations.” Journal of Public Administration Research & Theory 9 (1): 1–32. https://doi.org/10.1093/oxfordjournals.jpart.a024401

36.

Song

Lingguang

. 2010. Earned Value Management: A Global and Cross-Industry Perspective on Current EVM Practice. Newtown Square, Pennsylvania. Project Management Institute.

37.

Srithongrung

Arwiphawee

Ermasova

Natalia B.

Yusuf

Juita-Elena Wie

. 2019. Capital Management and Budgeting in the Public Sector. Hershey, Pennsylvania: IGI Global.

38.

Standish Group. 2015. Chaos Report 2015. Standish Group: Retrieved from: https://standishgroup.com/sample_research_files/CHAOSReport2015-Final.pdf

39.

Talbot

Colin

. 2010. Theories of Performance: Organizational and Service Improvement in the Public Domain. Oxford: Oxford University Press.

40.

U.S. Government Accountability Office. 1997. Significant Changes Underway in DOD’s Earned Value Management Process: GAO/NSIAD-97-108.

41.

U.S. Government Accountability Office. 2012. Earned Value Management Implementation Across major Spaceflight Projects is Uneven; GAO-13–22.

42.

U.S. Government Accountability Office. 2016. NASA: Preliminary Observations On Major Acquisition Projects And Management Challenges: GAO-16-461T.

43.

U.S. Government Accountability Office. 2020. Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Program Costs.

44.

U.S. Government Accountability Office. 2021. High-Risk Series: Dedicated Leadership Needed to Address Limited Progress in Most High-Risk Areas.

45.

U.S. Government Accountability Office. 2009a. Agencies Need to Improve the Implementation and use of Earned Value Techniques to Help Manage major System Acquisitions; GAO-10-2.

46.

U.S. Government Accountability Office. 2009b. Best Practices for Developing and Managing Capital Program Costs (Supersedes GAO-07-1134SP).

47.

Walker

Richard M.

Boyne

George A.

Brewer

Gene A.

. 2010. Public Management and Performance: Research Directions. Cambridge: Cambridge University Press.