Public-Private Partnerships and Value for Money

Abstract

The implementation of infrastructure projects is one element that drives economic development. When considering the high costs and complexity of these projects, the importance of analysis is highlighted a little more than traditional value for money when such projects are carried out in a public-private partnership. This article examines the state of the art of research on evaluating public-private partnership performance considering evidence of value for money. For this purpose, a systematic literature review was conducted based on the scientific production available in the Scopus and Web of Science databases. The main contributions of this study consist of presenting (1) a set of papers on performance evaluation that considers VfM as a complement to its evaluation systems, comparatively or inclusively in its analyses; (2) a methodological framework; and (3) guidelines for future research. This study provides insights for developing empirical studies in a wide range of contexts or sectors.

Keywords

public-private partnership evaluation value for money systematic review

Introduction

Public-private partnerships (PPPs) are defined as contracts between the private and public sectors to provide services to a particular community (Yescombe, 2007). In a document from the World Bank, the public-private partnership is defined as “a long-term contract between a private party and a government entity, to provide a public asset or service” (Bank, 2017, p. 5). This definition is based on the British model, established in 1992, whose central character is the private sector financing and building the asset, and selling the flow of services generated to the public sector, which has a direct or indirect obligation to acquire these services during the contract (Grout, 1997). Since PPPs have become popular around the globe, studying their performance from different perspectives is important, as their use tends to be more constant (Hodge & Greve, 2017). In the recent literature, Dolla and Laishram (2019) argued the need for effective procurement, since infrastructure projects contracted through partnerships should generate Value for Money (VfM), while at the same time focusing criticism of the PPP institution that results in higher costs.

Infrastructure projects involve installations necessary for society to function, such as transport, sanitation, and public lighting, in addition to schools, hospitals, and prisons, as well as other essential arrangements for the structure of a society (Yescombe, 2007). It is known, therefore, that the government is not always a specialist in all areas and agreement with private companies is a way of guaranteeing these services for society and, moreover, with quality, speed and at a lower cost. Despite the criticism that PPPs may not generate better results than traditional contracts (Verweij & van Meerkerk, 2021), it is through monitoring and evaluation methods in the various stages of a project that greater security of the proposed objectives is obtained. Assessing performance is a necessary condition to ensure the delivery of the objectives established in the PPP and guarantee VfM (Love et al., 2015). In this context, the following research question is asked: What is the state of the art of research on the evaluation of PPP performance using VfM?

To answer the research question, it was necessary to conduct a systematic literature review (SLR), based on the research protocols suggested by authors such as de Almeida Biolchini et al. (2007), Kitchenham (2004), and Tranfield et al. (2003). The SLR was conducted in the Scopus and Web of Science databases on issues related to performance evaluation of partnership projects between the government and the private sector, considering the cost-benefit – VfM of this undertaking.

VfM is a presumptive estimate of performance in the delivery of public sector works and services through private contracting, in which the higher the measure, the better the works and the lower the expenses (Hu et al., 2014; Peterson, 2019). To determine VfM, the Public Sector Comparator (PSC) is usually used to compare the cost of the proposed PPPs and the reference cost of a traditional acquisition (Grimsey & Lewis, 2005). Thus, according to the authors, VfM can be described as an ideal combination of cost and quality for the project's entire life that meets the user's needs. Therefore, seeking the best result in terms of cost-benefit makes sense, because if it does not occur, other forms of government takeover are preferable. Thus, performance evaluation methods must take this philosophy into account, since the PPP institution exists to achieve better results. In this context, this article assumes that PPPs incorporate principles of efficiency, speed and lower costs into the public sector, supported by the recommendations of Hodge and Greve (2007) regarding a way to gauge and provide advice in advance on partnership projects between government and companies.

This article contributes to the literature by presenting a typology emphasizing (1) a methodological framework, (2) the scope of PPPs and (3) VfM components. In this respect, authors like Grimsey and Lewis (2002, 2005) addressed relevant issues related to PPPs and VfM, as did Song et al. (2016), who influenced the field by highlighting trends for studies on PPP performance evaluation. It is hoped that this article can help researchers and PPPs with regard to the creation and application of evaluation methods, especially in the bidding phase for the selection of new partners and public policy makers in decision making to minimize future problems.

Methodological Procedures

This article presents the state of the art in relation to methods of evaluating the performance and cost-benefit of public-private partnerships. The SLR is a technique that allows the mapping and analysis of relevant research, enabling discussions based on those provided by traditional reviews (de Almeida Biolchini et al., 2007; Kitchenham, 2004; Tranfield et al., 2003).

The SLR protocol is shown in Figure 1. For a better analysis and replicability of this study, the SLR was divided into three stages, as suggested by Tranfield et al. (2003): (1) SLR planning, (2) SLR conduct and (3) SLR knowledge dissemination.

Figure 1.

Research protocol systematization.

Stage I: Planning the Systematic Literature Review

Stage I was based on the guidance of experts in the field, in addition to support from the literature to develop the search string and ensure the originality of the research. After defining the keywords and Boolean operators, articles were selected from the Scopus and Web of Science databases, specifically on reviews, as shown in Table 1.

Table 1.

Strings and Number of Articles Resulting From Searches for Reviews.

Database	String	Result^a
Scopus	TITLE-ABS-KEY ( “public-private partnership” OR “public-private partnership” ) AND “value for money” AND (“evaluation” OR “performance evaluation” OR “performance assessment” ) AND ( LIMIT-TO ( DOCTYPE , “ re” ) ) AND ( LIMIT-TO ( LANGUAGE , “English” ) )	33
Web of science	TS=(( “public-private partnership” OR “public-private partnership” ) AND “value for money” AND ( “evaluation” OR “performance evaluation” OR “performance assessment” ))) AND LANGUAGE: (English) AND DOCUMENT TYPES : (Review) Timespan: All years. Indexes: SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, ESCI.	1

^aThe data exportation/collect occurred on July 19th 2020.

Table 1 presents 34 previously selected articles, six of which focus on the literature review. Among the 34 documents, the six that focus on reviews are those by Jayasuriya et al. (2019) and Zhang et al. (2020), developed from a specific set of periodicals identified a priori, focused on public-private partnerships in the context of construction. On the other hand, Ma et al. (2019) provided an overview of the state of the art of public-private partnerships between 2008 and 2018. Sanni and Hashim (2016) conducted a review of the practice of public-private partnerships procurement in Malaysia and the study of Ullah et al. (2016) was directed at aspects involving the concession of public-private partnerships. Liu et al. (2014) provided a narrative review of the literature on measuring the performance of public-private partnerships.

This study represents a significant contribution to the literature on Public-Private Partnerships, considering the important role of the PPP institution in providing public services to society with regard to risk sharing with the private sector, as well as taking advantage of the experience, effectiveness and resources that the private sector has to offer in its area of expertise.

Stage II: Conducting the Systematic Literature Review

The second stage consisted of applying the search string in the Scopus and Web of Science databases. Core Collection databases are chosen for this kind of research due to the wide coverage of scientific literature (Chadegani et al., 2013; Chersan et al., 2020). Table 2 shows the string and the results according to the database used.

Table 2.

Number of Articles Found in Scopus and Web of Science.

Database	String	Result^a
Scopus	TITLE-ABS-KEY ( “public-private partnership” OR “public-private partnership” ) AND ”value for money” AND ( “evaluation” OR “performance evaluation” OR “performance assessment” ) AND PUBYEAR > 1999 AND ( LIMIT-TO ( DOCTYPE , “ar” ) ) AND ( LIMIT-TO ( LANGUAGE , “English” ) )	452
Web of science	TS=(( “public-private partnership” OR “public-private partnership” ) AND “value for money” AND ( “evaluation” OR “performance evaluation” OR “performance assessment” ))) AND LANGUAGE: (English) AND DOCUMENT TYPES : (Article) Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, ESCI Timespan=2000-2020	19

^aThe data exportation/collection occurred on July 19th 2020.

As shown in Figure 1, after the selection of the articles, the textual corpus selection process was begun. The first was to exclude duplicate studies (17) from the 471 articles that were identified, leaving a total of 454 articles.

Regarding reputation, the articles were selected from journals in the first and second citation quartiles (Q1 and Q2) of the Scimago Journal & Country Ranking (SJR, 2019). Given the lower acceptance rates of journals ranked in the first quartiles, compared to journals ranked in the third and fourth quartiles, according to Gu and Blackmore (2019), these journals are more demanding, reflecting on the quality of the articles. Thus, of the 454 articles, 207 are Q1 and 157 are Q2.

In the remaining 364 articles, the presence of the string words in the title, abstract and keywords fields were then analyzed. Thus, owing to the functions of the OR and AND operators and adherence to the research theme, the final textual corpus of 117 articles was selected.

Stage III: Reporting and Dissemination

In the third and last stage, the analyses and discussions were presented in three stages: (1) descriptive analysis; (2) typology analysis; and (3) bibliographic coupling analysis. These analyses were performed with the support of RStudio 1.0.143, IRaMuTeQ 0.7 and Gephi software (Bastian et al., 2009; Perianes-Rodriguez et al., 2016; Souza et al., 2018; Team, 2021).

Analysis of the Results and Discussions

This section presents (i) a descriptive analysis of the textual corpus, (ii) typology, and (iii) a bibliographic coupling analysis with a focus on the research proposals.

Descriptive Analysis

Figure 2 shows the annual distribution of the textual corpus composed of a sample of 117 articles. The selected articles are part of the collection of a total of 65 journals and 263 authors and co-authors. The average number of citations per article was 25.95 in the twenty-year period under analysis (2000 to July 2020), the period in which PPPs grew around the world, given the laws and regulations that emerged in the 1990s and in the United Kingdom, where they were named Private Finance Initiatives (PFIs) (Grout, 1997). In the early 2000s, the 2003 African Treasury Regulation established rules for their conduct. In Brazil, Law 11.079/2004 instituted norms for contracting PPPs nationwide. In Chile, Law 20,410 of 2010 regulated Public-Private Partnerships. The period ended in July 2020, when the collection began, and it was not possible to collect data after the research had begun.

Figure 2.

Annual distribution of the textual corpus.

Between 2008 and 2018 a growing trend can be seen in the volume of articles published, which may indicate the growing interest of the international scientific community in the subject. From 2010 onwards, the variation in quantity constantly increased year by year. In this respect, considering that the inclusion of articles in the corpus is a reflection of the selection criteria defined a priori, it is noteworthy that in the years 2010, 2013, 2015 and 2018, the number of articles from journals with publication areas aligned with the research in question and classified in the first and second citation quartile of the SJR (2019) was higher. Among the areas of publication are: Development, Urban Studies, Energy Engineering and Power Technology, Health Policy, Medicine, Public Health, and Environmental and Occupational Health.

Table 3 presents the 10 most cited articles in the textual corpus, in which the discussion of PPPs in different contexts is well known.

Table 3.

Rank	Title	Authors	Year	Quotes
1	“Evaluating the risks of public private partnerships for infrastructure projects	Grimsey & Lewis	2002	465
2	Are Public Private Partnerships value for money?: Evaluating alternative approaches and comparing academic and practitioner views”	Grimsey & Lewis	2005	275
3	“Selection of performance objectives and key performance indicators in public-private partnership projects to achieve value for Money”	Yuan, Zeng, Skibniewski & Li	2009	176
4	“Developing a risk assessment model for PPP projects in China-A fuzzy synthetic evaluation approach”	Xu Y., Yeung JFY, Chan APC, Chan DWM, Wang SQ, Ke Y.	2010	171
5	“A critical financial analysis of the Private Finance Initiative: Selecting a financing method or allocating economic wealth?”	Shaoul J.	2005	137
6	“A review of emerging trends in global PPP research: analysis and visualization”	Song J., Zhang H., Dong W.	2016	102
7	“Factors influencing the success of PPP at feasibility stage - A tripartite comparison study in Hong Kong”	Ng ST, Wong YMW, Wong JMW	2012	84
8	“Paving the way for public-private partnerships in infrastructure development”	Zhang X.	2005	83
9	“Value for money and risk in public-private partnerships”	Siemiatycki & Farooqi	2012	67
10	“Future proofing PPPs: Life-cycle performance measurement and Building Information Modeling”	Love, Liu, Matthews, Sing & Smith	2015	64

Proposed Typology

For a robust analysis of the textual corpus, the Reinert Method, also known as Descending Hierarchical Classification (DHC), was used to present a typology when interpreting words grouped into classes.

A typology is a systematic grouping of words that, in addition to the frequency of occurrence, considers the lexical field. When considering the meaning of words, the interpretation of the classes that make up the typology enables a contextualization and debate of the themes in question and, in this case, contributes to the development of the literature on public-private partnerships (Loo, 2020).

The textual corpus composed of 117 abstracts was submitted for classification, comprising 684 text segments. Of these 684 text segments, 625 were classified, which is equivalent to 91.37%. It is noteworthy that the classification must be at least 70% of the text segments for the DHC to be considered statistically significant (Camargo & Justo, 2013).

Regarding categorization, a total of three classes were obtained (Figure 3), which were renamed from the content analysis: Class 1 includes 261 text segments (41.76%); Class 2 contains 149 text segments (23.84%); and Class 3 includes 215 text segments (34.40%).

Figure 3.

Classes of the typology.

The discussion of each typology class is based on the occurrence of words whose chi-square value is greater than 3.80 (χ² > 3.80), or alternatively, where the probability value is less than 5% (p-value < .05), eliminating those words that do not meet this statistical criterion. We emphasize that a p-value < .05 portrays a significance level in terms of the association between words and classes (Reinert, 1990). Words included in Class 1, Class 2, and Class 3 are shown in Figure 3, as well as, an author associated with each word.

Class 1 presents the methodological procedures considered significant in terms of chi-square (χ²) and p-value statistical metrics. However, in order to provide a complete methodological framework, we conducted a more detailed survey with regard to data collection and analysis techniques (Figure 4). The studies were categorized according to the nature of the research: (1) quantitative research, which represents 41.03% of the textual corpus; and, (2) qualitative research, which represents 58.97% of the textual corpus. In addition, the first authors of each study were considered based on the frequency of occurrences in which the methods were used in the articles. It is also noteworthy that some studies used more than one method of collection and analysis in the same research to achieve better results.

Figure 4.

Methods used in the corpus articles.

Qualitative approaches were predominant. Of the qualitative data analysis methods, the recurrent ones were descriptive analysis, prescriptive analysis, VfM analysis, comparative analysis and exploratory data analysis. Despite the variety of methods used, no evidence was found of studies of a bibliographic nature that focused on the same topic discussed in the scope of this work and that used metrics of citation, co-citation networks or bibliographic coupling networks together, or even, that used both the Scopus and Web of Science databases as sources of data. With regard to quantitative studies, the recurrent methods of analysis were structural equation modeling and exploratory factor analysis, in addition to the relational method of regression analysis.

Regarding the data collection methods used in qualitative research, the most frequently used metrics were document analysis and interviews. On the other hand, in the case of quantitative collection methods, structured questionnaires through survey research were predominant.

The words that comprise Class 2 (Figure 3) address terms regarding the elements of the PPP contract. In this class, we highlight the work of Javed et al. (2013), in which the authors reinforced the importance of establishing robust specifications for output requirements, particularly in the initial acquisition phase. Moeover, Willems et al. (2017) showed that it is necessary to recognize the diversity of projects and develop a common vocabulary, and contracting authorities must have a clear vision of their objectives before beginning the competition.

Meanwhile, Love et al. (2010) developed a conceptual model represented by the public sector based on trust between alliances in which the use of incentive payments through risk/reward models provide the impetus for better project performance. To these authors, the growing search for a better cost-benefit ratio for infrastructure projects raises the need to identify Success Factors (SF) for an alliance with competitive prices.

In the Class 3 (Figure 3), the words portray the VfM components. The Value for Money measurement can estimate the ideal cost allocation for the project life cycle, comparing it with another contracting modality, that is, the PSC. In this class, Hu and Han (2018) discussed a quantitative VfM assessment of PPP mode road projects in China using the PSC. With the basis of the construction cost and the operating cost, as well as the toll revenues and the revenues from the services and advertising area, the authors identified that a higher discount rate is profitable for the private sector, whereas, in contrast, a lower discount rate is favorable for the government.

This class also highlights the work developed by Aldrete et al. (2012), which provided details on the incorporation of risk factors in the VfM analysis, with this analysis as the central point for decision-making by managers. According to the aforementioned authors, the use of objective measures of costs and quantification of risks was recommended to maximize VfM, while they recommended more methodological approaches through an assessment framework from the VfM perspective.

Bibliographic Coupling Analysis

The bibliographic coupling analysis (BCA), presented in Figure 5, measures the similarity between pairs of documents from shared references (Kessler, 1963) and, according to Grácio (2016), makes it possible to study the development of research lines, as it allows the identification of the most important research centers, researchers and articles in a scientific domain.

Figure 5.

Coupled documents network.

Figure 5 shows an excerpt of the textual corpus from the period between 2016 and 2020, with each article (node) represented by the main author. The average clustering coefficient is 0.607 and the modularity index of 0.194 suggests that the network has a modular structure (Newman, 2006) and, in this case, it can be analyzed from three clusters.

Using the betweenness centrality, it is possible to highlight some authors in the network. This index measures the mediation intensity in order to enable the connection between pairs of nodes, establishing the network information flow. According to Zhang and Luo (2017), as a node represents the only way to connect with other pairs of nodes, this node possibly has high betweenness centrality.

From this perspective, it is observed that the betweenness centrality index highlights authors willing to interact, as it quantifies their contributions as a source option to establish relationships between other pairs of authors (Newman, 2005). Table 4 presents 10 linked articles from each cluster with the best betweenness centrality indices.

Table 4.

Details of the Coupled Documents.

Cluster (Color)	Node	Betweeness Centrality	Article
Cluster 1 (Green)	Song (2016)	109.90	Song, J., Zhang, H., & Dong, W. (2016). “A review of emerging trends in global PPP research: analysis and visualization”. Scientometrics, 107(3), 1111–1147
	Liu (2018c)	84.62	Liu, HJ, Love, PED, Smith, J., Irani, Z., Hajli, N., & Sing, MCP (2018). “From design to operations: a process management life-cycle performance measurement system for Public-Private Partnerships”, Production Planning & Control, 29(1), 68–83
	Liu (2016)	82.32	Liu, J., Love, PED, Smith, J., Matthews, J., & Sing, C.-P. (2016). “Praxis of Performance Measurement in Public-Private Partnerships: Moving beyond the Iron Triangle”. Journal of Management in Engineering, 32(4), 04016004.
	Swanson (2020)	51.52	Swanson, R., & Sakhrani, V. (2020). “Appropriating the Value of Flexibility in PPP Megaproject Design”. Journal of Management in Engineering, 36(5), 05020010
	Liu (2018a)	37.34	Liu, HJ, Love, PE, Smith, J., Sing, MC, & Matthews, J. (2018). “Evaluation of public–private partnerships: A life-cycle Performance Prism for ensuring value for money”. Environment and Planning C: Politics and Space, 239965441775087
	Budayan (2020)	32.48	Budayan, C., Okudan, O., & Dikmen, I. (2020). “Identification and prioritization of stage-level KPIs for BOT projects – evidence from Turkey”. International Journal of Managing Projects in Business
	Xiong (2019)	25.94	Xiong, W., Chen, B., Wang, H., & Zhu, D. (2019). “Transaction Hazards and Governance Mechanisms in Public-Private Partnerships: A Comparative Study of Two Cases”. Public Performance & Management Review, 1–26
	Hossain (2019)	23.75	Hossain, M., Guest, R., & Smith, C. (2019). “Performance indicators of public private partnership in Bangladesh”. International Journal of Productivity and Performance Management, 68(1), 46–68
	Babatunde (2017)	14.95	Babatunde, SO, & Perera, S. (2017). “Public-private partnership in university female students' hostel delivery”. Facilities, 35(1/2), 64–80.
	Rouhani (2016)	7.80	Rouhani, OM, Geddes, RR, Gao, HO, & Bel, G. (2016). “Social welfare analysis of public–private investment partnership approaches to transportation projects”. Transportation Research Part A: Policy and Practice, 88, 86–103
Cluster 2 (Red)	Yuan (2018b)	70.55	Yuan, J., Xu, W., Xia, B., & Skibniewski, MJ (2018). “Exploring Key Indicators of Residual Value Risks in China's Public–Private Partnership Projects”. Journal of Management in Engineering, 34(1), 04017046
	Simon (2020)	55.40	Simon, L., Jefferies, M., Davis, P., & Newaz, MT (2020). “Developing a theoretical success factor framework for the tendering phase of social infrastructure PPPs”. International Journal of Construction Management, 1–15
	Kavishe (2019)	47.35	Kavishe, N., Jefferson, I., & Chileshe, N. (2019). “Evaluating issues and outcomes associated with public–private partnership housing project delivery: Tanzanian practitioners' preliminary observations”, International Journal of Construction Management, 19:4, 354–369
	Cui (2019)	40.72	Cui, C., Wang, J., Liu, Y., & Coffey, V. (2019). “Relationships among Value-for-Money Drivers of Public–Private Partnership Infrastructure Projects”. Journal of Infrastructure Systems, 25(2), 04019007
	Chen (2019)	38.09	Chen, C., Yu, Y., Osei-Kyei, R., Chan, APC, & Xu, J. (2019). “Developing a project sustainability index for sustainable development in transnational public–private partnership projects”. Sustainable Development
	Wang (2018)	24.93	Wang, L., Zhang, P., Zhang, P., Li, R., Zhang, Y., & Wu, Y. (2018). “A systematic approach for public-private partnership projects CSFs evaluation”. Engineering, Construction and Architectural Management
	Chou (2016)	24.88	Chou, J.-S., & Leatemia, GT (2016). “Critical Process and Factors for Ex-Post Evaluation of Public-Private Partnership Infrastructure Projects in Indonesia”. Journal of Management in Engineering, 32(5), 05016011.
	Wang (2019)	17.80	Wang, N., Chen, X., & Wu, G. (2019). “Public Private Partnerships, a Value for Money Solution for Clean Coal District Heating Operations”. Sustainability, 11(8), 2386
	Osei-Kyei (2018b)	17.00	Osei-Kyei, R., Chan, APC, Dansoh, A., Ofori-Kuragu, JK, & Oppong, GD (2018). “Strategies for Effective Management of Unsolicited Public–Private Partnership Proposals”. Journal of Management in Engineering, 34(3), 04018006.
	Osei-Kyei (2018a)	16.38	Osei-Kyei, R., & Chan, APC (2018). “Evaluating the project success index of public-private partnership projects in Hong Kong”. Construction Innovation, 18(3), 371–391
Cluster 3 (Orange)	Santandera (2016)	42.82	Santandrea, M., Bailey, S., & Giorgino, M. (2016). “Value for money in UK healthcare public–private partnerships: A fragility perspective”. Public Policy and Administration, 31(3), 260–279
	Makovšek (2018)	41.42	Makovšek, D., & Moszoro, M. (2018). “Risk pricing inefficiency in public–private partnerships”. Transport Reviews, 38(3), 298–321
	Burke (2019)	34.02	Burke, R., & Demirag, I. (2019). “Risk management by SPV partners in toll road public private partnerships”. Public Management Review, 21(5), 711–731
	Boardman (2017)	28.83	Boardman, A., & Hellowell, M. (2016). “A Comparative Analysis and Evaluation of Specialist PPP Units' Methodologies for Conducting Value for Money Appraisals”. Journal of Comparative Policy Analysis: Research and Practice, 19(3), 191–206
	Opara (2017)	20.95	Opara, M., Elloumi, F., Okafor, O., & Warsame, H. (2017). “Effects of the institutional environment on public-private partnership (P3) projects: Evidence from Canada”. Accounting Forum, 41(2), 77–95
	Willems (2017b)	16.56	Willems, T., Van Dooren, W., & van den Hurk, M. (2017). “PPP policy, depoliticisation, and anti-politics. Partecipazione e Conlitto”, 10(2), 448–471
	Zwalf (2017)	16.39	Zwalf, S., Hodge, G., & Alam, Q. (2017). “Choose Your Own Adventure: Finding a Suitable Discount Rate for Evaluating Value for Money in Public-Private Partnership Proposals*”. Australian Journal of Public Administration, 76(3), 301–315. doi:10.1111/1467–8500.12242
	Kweun (2018)	16.15	Kweun, JY, Wheeler, PK, & Gifford, JL (2018). “Evaluating highway public-private partnerships: Evidence from US value for money studies”. Transport Policy, 62, 12–20.
	Xiong (2016)	12.50	Xiong, W., Zhang, X., & Chen, H. (2016). “Early-Termination Compensation in Public–Private Partnership Projects”. Journal of Construction Engineering and Management, 142(4), 04015098
	Jasiukevicius (2018)	10.09	Jasiukevičius, L., & Vasiliauskaitė, A. “The Assessment of Public-Private Partnership's Possibilities to Optimize Investments in Public Infrastructure”. Inzinerine Ekonomika-Engineering Economics, 29(1), 32–45

Source: Research data.

Table 4 shows that Song et al. (2016) occupy a prominent place in the network, which can be explained by considering that their research provides theoretical support for PPPs.

In Cluster 1, we also highlight the research by Liu, Love, Smith, Irani et al. (2018), Liu, Love, Smith, Sing et al. (2018) and Swanson and Sakhrani (2020), who, based on the developed PPP performance measurement system, suggested greater efforts to adjust payment mechanisms through an explicit function to enable the use of the system in practice. According to the authors, this adjustment to the model structure, combined with the monitoring of service delivery, increases the chances of PPP contracts to provide long-term value for interested parties.

Liu et al. (2016) recommended validating the proposed theoretical framework, which encompasses the real life cycle VfM assessment, stakeholder-oriented performance measures and a systematic review mechanism of operational key performance indicators (KPI). Budayan et al. (2020) suggested studies focusing on the validation of specific KPIs for build-operate-transfer projects.

In general, in Cluster 1, suggestions for future research focus on validating the developed structure or models, inserting variables and developing specific indices. The content of the articles reflects the importance of the review that was conducted, as they demonstrate performance appraisals and how they can aid the VfM assessment. For Hossain et al. (2019), the PSC analysis, on which the VfM assessment depends, is not frequently conducted in Bangladesh. The authors' contribution in the context of the study was to identify a prioritized set of performance indicators and key performance areas (KPAs) for Bangladeshi PPPs by applying the AHP method, establishing weights for indicators identified by a review of the related literature.

This work is a constant example in Cluster 1, developing a method that helps the decision process without disregarding VfM. The results show that “funding” is the most significant KPA with regard to measuring PPP performance, followed in order of importance by “planning and initiation,” and “transparency and accountability” (Hossain et al., 2019).

In the same perspective, the study of Liu, Love, Smith, Sing, et al. (2018) reported that there are very few ways of evaluating the performance of PPPs, which makes evaluations difficult, especially in the initial phases, and restricts the verification of VfM. The authors proposed a Performance Prism that can be applied throughout the lifecycle to fill this gap. This prism extends the assessment of VfM with qualitative and quantitative approaches. This mechanism for reviewing performance measurement systems throughout the lifecycle helps to ensure project success.

In Cluster 2, the studies suggest a more in-depth search for greater accuracy of models, practical applications and validity tests in different types of PPP, distinct regions and different sample sizes (Chou & Leatemia, 2016; Kavishe et al., 2019; Simon et al., 2020; Wang et al., 2019).

Regarding the use of VfM to choose between PPPs and traditional acquisitions of infrastructure projects, it is recommended that the PPP format be chosen when it provides the best cost-benefit ratio. Therefore, it is important to choose indicators that go beyond the mathematical and financial definition of present value, according to Cui et al. (2019), who studied the impacts of VfM drivers and developed structural equation model that indicated that cost and effectiveness drivers contribute to the project's sustainability and performance and the better the cooperation environment, the better the participants' capacities. In Cluster 3, the research recommends a wider range of variables and the development of new tools (Burke & Demirag, 2019; Kweun et al., 2018; Makovšek & Moszoro, 2018; Opara et al., 2017; Santandrea et al., 2016). This review contributes by finding works that analyze risk factors for PPP assessment, for example, despite the controversy over the discount rate to be used for financial estimation between the PSC and PPP. Zwalf et al. (2017), from Cluster 3, claimed that PSC-based valuations must guarantee a discount rate that reflects the real VfM. Hence, an adequate discount rate, especially concerning risk inclusion, is an important issue to consider so that distortions do not occur.

Final Considerations

This article aimed to investigate the state of the art of research on the evaluation of PPP performance using VfM. To achieve the proposed objective, an SLR was performed based on the research protocol developed by Tranfield et al. (2003).

The state of the art was analyzed based on a set of 117 articles from QI and Q2 SJR (2019), available in the Scopus and Web of Science databases. The results showed the behavior of the annual distribution of publications and revealed the growing interest of the academic community in the topic in question, with this interest being more focused on infrastructure projects (construction and highways) and less focused on the area of health and the environment. Among the most cited articles are “Evaluating the risks of public private partnerships for infrastructure projects” (Grimsey & Lewis, 2002) and “Are Public Private Partnerships value for money?: Evaluating alternative approaches and comparing academic and practitioner views” (Grimsey & Lewis, 2005). Both works addressed the evaluation of PPPs with VfM integration. In particular, the latter article highlighted in the title the question of whether Public-Private Partnerships really generate VfM, suggesting measurement alternatives from both academic and practical viewpoints.

In relation to the typology, three classes showed the methodological procedures, elements of the PPP contract and the VfM components. Among the methods, those that seek to model PPP performance assessment instruments through structural equations of exploratory or confirmatory analysis were evident. This purpose, of creating an assessment instrument or applying an assessment tool, was evident in the articles that were analyzed. It is recommended that the VfM approach encompass criteria that address different stakeholders and thus improve the analysis of PPP performance compared with other forms of contracting.

This article can help researchers in future empirical studies, both quantitative and qualitative, through theoretical support for the formulation of research problems, especially when considering the set of possible trends provided in the coupling analysis. In addition, we suggest the development of modeling studies of the multicriteria evaluation methods of Public-Private Partnerships, considering the life cycle phases, especially in the bidding phase, to evaluate private proposals, thereby optimizing long-term results through more assertive choices. Another suggestion is the possibility of bidding proposals using indicators that represent the different stakeholders involved in the public-private partnership.

Although this work presents a systematic and exhaustive review of the literature, there are some limitations in relation to our findings, which can be considered in future research: (1) the defined search strings limited the results to articles published in English; (2) only journals placed in the first and second citation quartiles according to the SJR (2019) were included. Therefore, it cannot be ignored that relevant research has been published in other formats, such as books and conference proceedings, or in other languages.

Footnotes

Acknowledgments

The authors wish to express their gratitude to the editor Richard G. Little and reviewers for their constructive input and kind feedback.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Claudimar Pereira da Veiga

Author Biographies

Robson de Faria Silva is a Ph.D. and Master's in Administration from the Pontifical Catholic University of Paraná. He is a Specialist in Controllership and Financial Management. He is Coordinator and Professor in the Business Administration course at the Cesul Higher Education Center and Finance Columnist - Jornal de Beltrão and has worked as a financial consultant. He has experience in Administration, emphasizing Financial and Budget Management, working mainly on the following topics: Corporate Governance, Stock Exchange, Cooperatives, Capital Markets, Strategic Cost Management, and Game Theory. Member of the Research Group Decision-making processes at the Federal University of Alagoas. .

Alceu Souza is a full professor in the Business School and Postgraduate program in Business Administration (PPAD) at the Pontifical Catholic University of Paraná. Has experience in Business Administration, focusing on Financial Management, acting on the following subjects: investment decisions, feasibility indicators, Multi-Index Methodology for project risk assessment, costing methods, and strategic cost management.

Fabíola Kaczam is a Ph.D. student in the Graduate Program in Administration at the Federal University of Santa Maria, working in the MUTARE Research Group (Organizational Change, Innovation, and Organizational Behavior). Production Engineer from the Federal Technological University of Paraná. Master in Production Engineering from the Graduate Program in Production Engineering at the Federal University of Santa Maria, Campus Sede. Degree in Professional Education (PEG-UFSM 2019). .

Luciano Luiz Dalazen is a Ph.D. in Business Administration from the Pontifical Catholic University of Parana in the Strategic Administration area. In addition, he holds a degree in Food Technology from the Federal Technological University of Parana. He has experience in food science and technology, including industrial processes. He prefers to study Behavioral Finance, Strategy, and Quantitative Methods in Management. Member of the Research Group on Decision Processes FEAC-UFAL. Evaluator of periodicals and events in the themes of Strategy and Quantitative Methods.

Wesley Vieira da Silva is a Ph.D. in Production Engineering from the Federal University of Santa Catarina. He has a level 1D productivity scholarship from the National Council for Scientific and Technological Development-CNPq. He is a professor in the Faculty of Economics, Administration, and Accounting (FEAC) at the Federal University of Alagoas. He works in the major areas of Administration, Economics, Accounting, and Production Engineering, emphasizing the following themes: Quantitative and Econometric Methods, Behavioral Finance, Risk Management, Technical Efficiency, and Innovation Analysis. He is a permanent professor in the Graduate Program in Administration at the Federal Rural University of the Semi-Árido, in addition to acting as a professor and vice-coordinator of the Professional Master's Degree in Public Administration in Networks. Furthermore, he is the Leader of the Research Group on Decision-Making Processes at the Federal University of Alagoas. .

Claudimar Pereira da Veiga is a professor and researcher in the field of marketing at Fundação Dom Cabral (FDC). His work aims to link research and professional practice in business administration, integrating the following themes: marketing, strategy, and healthcare management. Professor Veiga has published more than 100 articles in journals recognized by the Financial Times 50 (FT50), ABDC A, CABS, and QUALIS A. He is an associate editor of BMC Health Services Research (Springer) and a member of the editorial board of Humanities and Social Sciences Communications (Nature). He served as Head of the Department of General and Applied Administration and as an advisory member of the Social and Applied Sciences sector at the Federal University of Parana (UFPR). .

References

Aldrete

Bujanda

Valdez

G. A.

(2012). Valuing public-sector revenue risk exposure in transportation public–private partnerships. Transportation Research Record: Journal of the Transportation Research Board, 2297(1), 88–96. https://doi.org/10.3141/2297-11

Bank

(2017). Public-private partnerships: Reference guide version 3. World Bank.

Bastian

Heymann

Jacomy

(2009). Gephi: An open source software for exploring and manipulating networks. In: International AAAI Conference on Weblogs and Social Media, San Jose, CA, May 17–20, 2009.

Budayan

Okudan

Dikmen

(2020). Identification and prioritization of stage-level KPIs for BOT projects – evidence from Turkey. International Journal of Managing Projects in Business, 13(6), 1311–1337. https://doi.org/10.1108/IJMPB-11-2019-0286

Burke

Demirag

(2019). Risk management by SPV partners in toll road public private partnerships. Public Management Review, 21(5), 711–731. https://doi.org/10.1080/14719037.2018.1523450

Camargo

B. V

Justo

A. M.

(2013). IRAMUTEQ: A free software for textual data analysis. Temas Em Psicologia, 21(2), 513–518. https://doi.org/10.9788/tp2013.2-16

Chadegani

A. A.

Salehi

Yunus

M. M.

Farhadi

Fooladi

Farhadi

Ebrahim

N. A.

(2013). A comparison between two main academic literature collections: Web of science and Scopus databases. Asian Social Science, 9(5), 18–26. https://doi.org/10.5539/ass.v9n5p18

Chersan

I. C.

Valentin Florentin

Gorgan

(2020). Green public procurement in the academic literature. Amfiteatrueconomic, 22(53), 82. https://doi.org/10.24818/EA/2020/53/82

Chou

J.-S.

Leatemia

G. T.

(2016). Critical process and factors for ex-post evaluation of public-private partnership infrastructure projects in Indonesia. Journal of Management in Engineering, 32(5), 05016011. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000436

10.

Cui

Wang

Liu

Coffey

(2019). Relationships among value-for-money drivers of public–private partnership infrastructure projects. Journal of Infrastructure Systems, 25(2), 04019007. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000479

11.

de Almeida Biolchini

J. C.

Mian

P. G.

Natali

A. C. C.

Conte

T. U.

Travassos

G. H.

(2007). Scientific research ontology to support systematic review in software engineering. Advanced Engineering Informatics, 21(2), 133–151. https://doi.org/10.1016/j.aei.2006.11.006

12.

Dolla

Laishram

(2019). Bundling in public–private partnership projects – A conceptual framework. International Journal of Productivity and Performance Management, 69(6), 1177–1203. https://doi.org/10.1108/IJPPM-02-2019-0086

13.

Grácio

M. C. C.

(2016). Acoplamento bibliográfico e análise de cocitação: Revisão teórico-conceitual. Encontros Bibli: Revista Eletrônica de Biblioteconomia e Ciência Da Informação, 21(47), 82. https://doi.org/10.5007/1518-2924.2016v21n47p82

14.

Grimsey

Lewis

M. K.

(2002). Evaluating the risks of public private partnerships for infrastructure projects. International Journal of Project Management, 20(2), 107–118. https://doi.org/10.1016/S0263-7863(00)00040-5

15.

Grimsey

Lewis

M. K.

(2005). Are public private partnerships value for money? Accounting Forum, 29(4), 345–378. https://doi.org/10.1016/j.accfor.2005.01.001

16.

Grout

(1997). The economics of the private finance initiative. Oxford Review of Economic Policy, 13(4), 53–66. https://doi.org/10.1093/oxrep/13.4.53

17.

Blackmore

K. L.

(2019). Towards a Broader Understanding of Journal Impact: Measuring Relationships between Journal Characteristics and Scholarly Impact. International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, 11(10), 2127–2132.

18.

Hodge

Greve

(2007). Public-private partnerships: An international performance review. Public Administration Review, 67(3), 545–558. https://doi.org/10.1111/j.1540-6210.2007.00736.x

19.

Hodge

G. A.

Greve

(2017). On public–private partnership performance. Public Works Management & Policy, 22(1), 55–78. https://doi.org/10.1177/1087724X16657830

20.

Hossain

Guest

Smith

(2019). Performance indicators of public private partnership in Bangladesh. International Journal of Productivity and Performance Management, 68(1), 46–68. https://doi.org/10.1108/IJPPM-04-2018-0137

21.

Han

(2018). Highway project value of money assessment under PPP mode and its application. Journal of Advanced Transportation, 2018(3), 1–11. https://doi.org/10.1155/2018/1802671

22.

Chen

Zhang

(2014). Value for money and its influential factors: An empirical study of PPP projects in Japan. Built Environment Project and Asset Management, 4(2), 166–179. https://doi.org/10.1108/BEPAM-12-2012-0055

23.

Javed

A. A.

Lam

P. T. I.

Chan

A. P. C.

(2013). A model framework of output specifications for hospital PPP/PFI projects. Facilities, 31(13/14), 610–633. https://doi.org/10.1108/f-02-2012-0014

24.

Jayasuriya

Zhang

Jing Yang

(2019). Challenges in public private partnerships in construction industry. Built Environment Project and Asset Management, 9(2), 172–185. https://doi.org/10.1108/BEPAM-01-2018-0024

25.

Kavishe

Jefferson

Chileshe

(2019). Evaluating issues and outcomes associated with public–private partnership housing project delivery: Tanzanian practitioners’ preliminary observations. International Journal of Construction Management, 19(4), 354–369. https://doi.org/10.1080/15623599.2018.1435154

26.

Kessler

M. M.

(1963). Bibliographic coupling between scientific papers. American Documentation, 14(1), 10–25. https://doi.org/10.1002/asi.5090140103

27.

Kitchenham

(2004). Procedures for performing systematic reviews.

28.

Kweun

J. Y.

Wheeler

P. K.

Gifford

J. L.

(2018). Evaluating highway public-private partnerships: Evidence from U.S. value for money studies. Transport Policy, 62(2018), 12–20. https://doi.org/10.1016/j.tranpol.2017.03.009

29.

Liu

H. J.

Love

P. E.

Smith

Sing

M. C.

Matthews

(2018a). Evaluation of public–private partnerships: A life-cycle performance prism for ensuring value for money. Environment and Planning C: Politics and Space, 36(6), 1133–1153. https://doi.org/10.1177/2399654417750879

30.

Liu

H. J.

Love

P. E. D.

Smith

Irani

Hajli

Sing

M. C. P.

(2018b). From design to operations: A process management life-cycle performance measurement system for public-private partnerships. Production Planning & Control, 29(1), 68–83. https://doi.org/10.1080/09537287.2017.1382740

31.

Liu

Love

E.D.P.

Smith

Regan

Sutrisna

(2014). Public-private partnerships: A review of theory and practice of performance measurement. International Journal of Productivity and Performance Management, 63(4), 499–512. https://doi.org/10.1108/IJPPM-09-2013-0154

32.

Liu

Love

P. E. D.

Smith

Matthews

Sing

C.-P.

(2016). Praxis of performance measurement in public-private partnerships: Moving beyond the iron triangle. Journal of Management in Engineering, 32(4), 04016004. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000433

33.

Loo

(2020). Building users’ search skills for systematic reviews: Development of self-directed learning through qualitative synthesis of guidelines (pp. 1–47). UC San Diego Library. https://escholarship.org/uc/item/0q53r0rg

34.

Love

P. E. D.

Liu

Matthews

Sing

C.-P.

Smith

(2015). Future proofing PPPs: Life-cycle performance measurement and building information modelling. Automation in Construction, 56(2015), 26–35. https://doi.org/10.1016/j.autcon.2015.04.008

35.

Love

P. E. D.

Mistry

Davis

P. R.

(2010). Price competitive alliance projects: Identification of success factors for public clients. Journal of Construction Engineering and Management, 136(9), 947–956. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000208

36.

Jin

(2019). A holistic review of public-private partnership literature published between 2008 and 2018. Advances in Civil Engineering, 2019(2019), 1–18. https://doi.org/10.1155/2019/7094653

37.

Makovšek

Moszoro

(2018). Risk pricing inefficiency in public–private partnerships. Transport Reviews, 38(3), 298–321. https://doi.org/10.1080/01441647.2017.1324925

38.

Newman

(2005). A measure of betweenness centrality based on random walks. Social Networks, 27(1), 39–54. https://doi.org/10.1016/j.socnet.2004.11.009

39.

Newman

M. E. J.

(2006). Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23), 8577–8582. https://doi.org/10.1073/pnas.0601602103

40.

S. T.

Wong

Y. M. W.

Wong

J. M. W.

(2012). Factors influencing the success of PPP at feasibility stage – a tripartite comparison study in Hong Kong. Habitat International, 36(4), 423–432. https://doi.org/10.1016/j.habitatint.2012.02.002

41.

Opara

Elloumi

Okafor

Warsame

(2017). Effects of the institutional environment on public-private partnership (P3) projects: Evidence from Canada. Accounting Forum, 41(2), 77–95. https://doi.org/10.1016/j.accfor.2017.01.002

42.

Perianes-Rodriguez

Waltman

van Eck

N. J.

(2016). Constructing bibliometric networks: A comparison between full and fractional counting. Journal of Informetrics, 10(4), 1178–1195. https://doi.org/10.1016/j.joi.2016.10.006

43.

Peterson

H. O.

(2019). Evaluating the costs, quality, and value for money of infrastructure public-private partnerships: A systematic literature review. Annals of Public and Cooperative Economics, 90(2), 227–244. https://doi.org/10.1111/apce.12243

44.

Reinert

(1990). ALCESTE, une méthodologie d’analyse des données textuelles et une application: Aurélia de G. de Nerval. Bulletin de Méthodologie Sociologique, 28(1), 24–54. https://doi.org/10.1177/075910639002600103

45.

Sanni

A. O.

Hashim

(2016). A review of public private partnership procurement practice in Malaysia. International Journal of Technological Learning, Innovation and Development, 8(1), 20. https://doi.org/10.1504/IJTLID.2016.075178

46.

Santandrea

Bailey

Giorgino

(2016). Value for money in UK healthcare public–private partnerships: A fragility perspective. Public Policy and Administration, 31(3), 260–279. https://doi.org/10.1177/0952076715618003

47.

Shaoul

(2005). A critical financial analysis of the private finance initiative: Selecting a financing method or allocating economic wealth? Critical Perspectives on Accounting, 16(4), 441–471. https://doi.org/10.1016/j.cpa.2003.06.001

48.

Siemiatycki

Farooqi

(2012). Value for money and risk in public–private partnerships. Journal of the American Planning Association, 78(3), 286–299. https://doi.org/10.1080/01944363.2012.715525

49.

Simon

Jefferies

Davis

Newaz

M. T.

(2020). Developing a theoretical success factor framework for the tendering phase of social infrastructure PPPs. International Journal of Construction Management, 20(6), 613–627. https://doi.org/10.1080/15623599.2020.1720343

50.

SJR (2019). Scimago journal and country rank. https://www.scimagojr.com

51.

Song

Zhang

Dong

(2016). A review of emerging trends in global PPP research: Analysis and visualization. Scientometrics, 107(3), 1111–1147. https://doi.org/10.1007/s11192-016-1918-1

52.

Souza

M. A. R. de

Wall

M. L.

Thuler

A. C. de M. C.

Lowen

I. M. V.

Peres

A. M.

(2018). O uso do software IRAMUTEQ na análise de dados em pesquisas qualitativas. Revista Da Escola de Enfermagem Da USP, 52(2), 1–2. https://doi.org/10.1590/s1980-220x2017015003353

53.

Swanson

Sakhrani

(2020). Appropriating the value of flexibility in PPP megaproject design. Journal of Management in Engineering, 36(5), 1–16. https://doi.org/10.1061/(asce)me.1943-5479.0000770

54.

Team (2021). RStudio. Integrated Development Environment for R. RStudio Inc.

55.

Tranfield

Denyer

Smart

(2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222. https://doi.org/10.1111/1467-8551.00375

56.

Ullah

Ayub

Siddiqui

S. Q.

Thaheem

M. J.

(2016). A review of public-private partnership: Critical factors of concession period. Journal of Financial Management of Property and Construction, 21(3), 269–300. https://doi.org/10.1108/JFMPC-02-2016-0011

57.

Verweij

van Meerkerk

(2021). Do public–private partnerships achieve better time and cost performance than regular contracts? Public Money & Management, 41(4), 286–295. https://doi.org/10.1080/09540962.2020.1752011

58.

Wang

Chen

(2019). Public private partnerships, a value for money solution for clean coal district heating operations. Sustainability, 11(8), 2386. https://doi.org/10.3390/su11082386

59.

Willems

Verhoest

Voets

Coppens

Van Dooren

Hurk

(2017). Ten lessons from ten years PPP experience in Belgium. Australian Journal of Public Administration, 76(3), 316–329. https://doi.org/10.1111/1467-8500.12207

60.

Yeung

J. F. Y.

Chan

A. P. C.

Chan

D. W. M.

Wang

S. Q.

(2010). Developing a risk assessment model for PPP projects in China — a fuzzy synthetic evaluation approach. Automation in Construction, 19(7), 929–943. https://doi.org/10.1016/j.autcon.2010.06.006

61.

Yescombe

E. R.

(2007). Public–private partnerships: Principles of policy and finance. Elsevier.

62.

Yuan

Zeng

A. Y.

Skibniewski

M. J.

(2009). Selection of performance objectives and key performance indicators in public–private partnership projects to achieve value for money. Construction Management and Economics, 27(3), 253–270. https://doi.org/10.1080/01446190902748705

63.

Zhang

Luo

(2017). Degree centrality, betweenness centrality, and closeness centrality in social network. In: Proceedings of the 2017 2nd international conference on modelling, simulation and applied mathematics (MSAM2017), Bangkok, Thailand, March 26–27, 2017. https://doi.org/10.2991/msam-17.2017.68

64.

Zhang

(2005). Paving the way for public–private partnerships in infrastructure development. Journal of Construction Engineering and Management, 131(1), 71–80. https://doi.org/10.1061/(ASCE)0733-9364(2005)131:1(71)

65.

Zhang

Y.-C.

Luo

W.-Z.

Shan

Pan

D.-W.

W.-J.

(2020). Systematic analysis of PPP research in construction journals: From 2009 to 2019. Engineering, Construction and Architectural Management, 27(10), 3309–3339. https://doi.org/10.1108/ECAM-03-2020-0178

66.

Zwalf

Hodge

Alam

(2017). Choose Your Own Adventure: Finding a Suitable Discount Rate for Evaluating Value for Money in Public-Private Partnership Proposals. Australian Journal of Public Administration, 76(3), 301–315. https://doi.org/10.1111/1467–8500.12242