Abstract
This study conducts a systematic literature review on several aspects regarding the success of initiatives of digital transformation in the public sector (DTPS), focusing on case studies, processes, and success factors of the DTPS projects. The research was carried out using several scientific databases and indexers (such as ACM Digital Library, IEEE Xplore, and ISI Web of Science). The literature review process enabled the identification of a total of 38 case studies, four processes, and 51 success factors, all of which are presented and discussed, contributing both to theory and practice, to minimizing risks and improving the success of DTPS projects. The review process also enabled the identification of new research avenues.
Keywords

Introduction
Digital transformation emerges as a process of evolution to a new digital reality by taking advantage of opportunities stemming from technological advances to automate processes, achieve productivity gains and increase efficiency (Bleicher & Stanley, 2017; Costa, 2020). In this sense, Digital Transformation in the Public Sector (DTPS) should follow a holistic approach, as it is an opportunity to review key government processes and services, move forward and go beyond traditional digitalization efforts by focusing, among others, on meeting citizens’ needs, new ways of providing services, and expanding user bases (Mergel et al., 2019).
Digital transformation is also crucial for thoroughly modernize public administration (Schedler et al., 2019). DTPS “evolves along a continuum of transition from analog to digital, to a full-stack review of policies, current processes, and user needs, and results in a complete revision of the existing and the creation of new digital services” (Mergel et al., 2019). However, as Clarke (2020) pointed out, despite all this plethora of possibilities, digital service failures abound, and citizens’ trust in government continues to wane. Despite the well-founded efforts towards DTPS, public organizations struggle to realize their full potential (Hackney & McBride, 1995). One of the reasons for this is that the public sector IT departments – and consequently public organizations – are not always properly prepared to face this challenge (Torres et al., 2005).
Therefore, the public management community cannot ignore or neglect digital transformation, related best practices, and contemporary governance roles (Clarke, 2020; Renteria et al., 2019). By driving a compelling response, a truly holistic DTPS approach can play a critical role in reducing failures and increasing the success of digital transformation initiatives, thus resulting in modulating citizens’ trust (Mahmood, 2016; Morgeson III et al., 2011; Tolbert & Mossberger, 2006).
The success of digital transformation initiatives in the public sector rests fundamentally on the adoption of structured processes that thoroughly consider the factors that influence the results. It is also important to look at successful cases of digital transformation so that they can be used as a benchmark – “learning from the best practice in e-governments will minimize the implementation risks and time” (Al-Shuaili et al., 2019). Although several studies in the current literature report success cases, processes applied, and success factors, to the best of our knowledge, no studies have yet compiled these investigations in a way that facilitates easy consultation of the results obtained. This article seeks to fill this gap.
Our study provides a systematic literature review on DTPS initiatives by addressing the success of projects and focusing on case studies and processes applied. It seeks to (1) catalog the case studies of digital transformation projects in the public sector and their main results; (2) to classify and detail the work processes reported in related studies; (3) to list the factors that can impact success in the public sector in the form of a unified framework of success factors for digital transformation projects in the public sector. This threefold set of goals can serve as a guide for future DTPS initiatives, helping to reduce inherent risks, prevent failures, and improve public trust in the success of DTPS initiatives.
The findings are valuable for public organizations as they attempt to realize the potential of their digital transformation initiatives. Our study provides both a theoretical and a practical contribution. The theoretical contribution stems from the compilation of digital transformation processes in the public sector and the proposal for a unified framework of success factors for digital transformation projects in the public sector. Practitioners can find in this article several alternative processes that can be applied to their specific contexts and a framework of success factors to take into consideration. Furthermore, the identified cases can be used as a benchmark.
In the ensuing Section 2 we present the background regarding digital transformation in general and digital transformation in the specific context of public sector organizations. Section 3 presents the research method, Section 4 details the results, and Section 5 discusses these results. The paper ends with conclusions, limitations, and suggestions for further work.
Background
Digital Transformation (DT) is defined as “the use of new digital technologies (social media, mobile, analytics or embedded devices) to enable major business improvements” (Fitzgerald et al., 2014). According to the European Commission, DT and the advancement of technologies are inextricably linked and “are giving rise to disruptive business models that profoundly impact industries, businesses, and citizens” (Probst et al., 2017).
DT is not only about technology. It also involves strategy, leadership, and the effort to change the mindset of teams, organizational culture, and processes before selecting a specific technology and how to implement it (Rogers, 2016; Tabrizi et al., 2019). In order to consolidate concepts, given the multiple extant definitions, DT can be regarded as “an evolutionary process that leverages digital capabilities and technologies to enable business models, operational processes and customer experiences to create value” (Morakanyane et al., 2017).
Digital transformation is not only driving innovation in the business sector, but also seems to be influencing developments in the public sector (Kokkinakos et al., 2016). Real-world digital transformation experiences are changing citizens’ expectations of governments’ ability to deliver high-value, real-time digital services (Mergel et al., 2019). Clarke (2020) and Jonathan (2019) corroborate this type of pressure on governments and argue that it stems from demands both from the politicians themselves and the citizens. Hence, to deal with this extended scenario, particularly in the public sector, DT must foster new ways of working with stakeholders, build new service delivery structures, and create new stakeholder relationships (Mergel et al., 2019).
In line with this, Clarke (2020) argues that government digital transformation comes with accountability challenges, since digitalization is increasingly not a mere add-on to a government’s daily work, but rather underpins all of its operations, emerging both as the front and back end of government operations.
Method
This section presents the literature review methodological approach. This systematic review aims to catalog DTPS case studies and reported processes, focusing particularly on the factors that can impact the success of digital transformation projects in the public sector.
Systematic literature review process.
Figure 1 shows the overall process and respective steps, which are described next.
The main research question is:
MQ. What factors can impact the success of digital transformation initiatives in the public sector? For complete coverage, we also address DTPS case studies and DTPS processes, since they represent important lessons learned and good practices that are critical for achieving success. The complementary questions are: CQ1. What are the case studies and respective lessons learned from digital transformation projects in the public sector? CQ2. What processes are followed in digital transformation projects in the public sector?
The articles were gathered from the three databases that cover the leading journals and conferences (Carrizo & Rojas, 2018; dos Santos et al., 2021; M Al-Eidan et al., 2020), aiming to bring together a broad volume of quality works related to the research: ACM Digital Library (dl.acm.org), IEEE Xplore (ieeexplore.ieee.org), and ISI Web of Science (isiwebofknowledge.com).
The databases were consulted from January 1, 2015 to March 31, 2020. This five-year period accounts for much of the emphasis on DTPS and allows avoiding bias from the first wave of the Sars-Covid-2 pandemic. The search expressions were (“digital transformation framework” or “digital transformation model” or “digital transformation” or “digitization” or “digitalization”) AND (“public sector” or “public administration” or “public service” or “government” or “public sphere” or “public organization”). The queries applied to each database are described in Appendix A.
The inclusion criteria were the following: IC1 – Academic or industry study; IC2 – Describe successful cases of DTPS projects; IC3 – Describe processes that guided DTPS initiatives; IC4 – Present success factors for DTPS projects; IC5 – Quantitative or qualitative study; IC6 – Written in English; and IC7 – Published from 2015 onwards. In turn, the exclusion criteria were the following: EC1 – Inaccessibility (not entirely accessible online or totally inaccessible); EC2 – Repeated article; EC3 – Not written in English; EC4 – Published as a short paper or a poster; EC5 – Lacking an abstract; EC6 – Not focused on DTPS; EC7 – Preface, editorial, article summary, book chapter, book, interview, workshop, short paper, panel, or poster; and EC8 – Present a process without its application description. An article should match at least one inclusion criterion and none exclusion criterion to be part of our selection.
After applying the search strings on the databases, a total of 3,804 articles were obtained, distributed among ACM Digital Library (1,600), IEEE Xplore (1,563), and ISI Web of Science (641). After removing the duplicates, with Mendeley software’s support for cataloging the resulting articles’ list, the total was 3,772 unique articles (398 in 2015, 702 in 2016, 714 in 2017, 819 in 2018, 1,055 in 2019, and 84 in 2020). Then, each researcher performed a selection based on reading the title and the abstract, using the Rayyan software, with the blind option activated. This analysis yielded 85 articles.
The full versions of these 85 articles were subsequently retrieved, and the Zotero software was used for storing the articles to be read. During this step, 2 of 85 papers proved to be inaccessible and were excluded.
In order to organize the systematic review and the resulting analysis, a spreadsheet was created to classify the 83 retrieved articles in terms of case study description, critical success factors, and processes implemented in the initiative. In addition, the country of the publication and the scope of evaluation (government level and government branch) were indicated for each article (when available). Finally, abstract, limitations, and future work were organized for each article. In the end, each article received a summary. As a result, in addition to the features described here, every article under study was submitted to the reviewers’ decision to be included or not. And finally, the inclusion and exclusion criteria were applied. In the end, a total of 56 papers remained, distributed as shown in Table 1.
Analyzed articles
Analyzed articles
The objective of the quality assessment phase was to obtain relevant information related to the research questions. The articles were classified by type (case study, process, or success factor) to try to answer the questions by gathering more information about them, considering the methodological aspects and their contribution to the research questions (Kitchenham & Charters, 2007). For this, we applied a checklist of quality criteria according to the following items: QC1 – Does the paper present a case study on DTPS projects?; QC2 – Does the paper present a process applied to DTPS projects?; QC3 – Does the paper present success factors for DTPS projects?; QC4 – Does the paper mention its research limitations?; QC5 – Does the paper indicate opportunities for further research?
According to Petticrew and Roberts (2008), checklists applied in a systematic review “provide one means of ensuring that included studies contain sufficient information about the sample, the question, the data analysis”. To create a relevancy ranking, each paper received one point whenever it met one of the five criteria. If the paper covers a case study, process, or success factor, it is scored according to each of these categories. We also considered the key limitations to contextualize the research findings, to interpret the validity of the scientific work, and assign a level of credibility to the conclusions (Ioannidis, 2007); therefore, one point was assigned whenever this aspect was present. An additional point was assigned if the paper under analysis described future studies, since this aspect is essential for complementing or continuing the published studies.
Appendix B lists all of the 56 studies according to their ranking. Noteworthy are the studies from Carrasqueiro et al. (2018); Lappi et al. (2019); Marzooqi et al. (2017); and Sundberg (2018), since they all matched four criteria.
Results
This section presents the results organized by case studies, processes, and success factors regarding DTPS projects.
Case studies
A total of 38 case studies were identified in the literature and were subsequently analyzed and classified according to country of publication, governmental level of the initiative (Central, Regional, and/or Local), and governmental branch (Executive, Legislative, and/or Judiciary). Table 2 presents the 38 case studies involving DTPS projects and a summary of the main results of each case study.
Case studies
Case studies
A total of four processes for digital transformation initiatives in the public sector were identified in the literature, and were analyzed regarding their origin (literature review or case study) and the structure they presented. Furthermore, following the typology proposed by Hatsu and Ngassam (2017), the type of process was also identified. Hatsu and Ngassam (2017) proposed a typification for e-Gov frameworks by considering: (1) e-Readiness and strategy-based, (2) Maturity stage-based, and (3) Project life cycle-centric. e-Readiness and strategy-based sets how to convert an e-Gov vision through objectives and principles into an implementation plan – how the implementation plan would be executed is out of the scope of this type of framework (Hatsu & Ngassam, 2017; Rabaiah & Vandijct, 2011). Maturity stage-based views e-Gov as a multistep/multistage process, emphasizing the development level of the e-Gov project (Concha et al., 2012; Hatsu & Ngassam, 2017; Klievink & Janssen, 2009). All four processes analyzed fit better in the typology project life cycle-centric, since they are concerned with DT implementation activities. Table 3 presents the articles that describe processes for DTPS projects.
Processes articles
Processes articles
The success factors for digital transformation initiatives in the public sector were collected, consolidated, and systematized from the works analyzed. Given the diversity and number of factors, a categorization by dimensions was adopted to facilitate a better organization.
Ivančić et al. (2019) studied ways to group and categorize success factors based on the analysis of DT cases in different activity segments and concluded that, in addition to the adoption of technology, other factors are relevant for successful DT. This finding is also aligned with the work by the European Commission (Barcevičius et al., 2019; Williams & Valayer, 2018). The work conducted by Ivančić et al. (2019) lists seven dimensions (strategy, people, organization, customers, ecosystem, technology, and innovation) that, in their view, are the basis for digital transformation.
To categorize the critical success factors for DTPS projects, our work adopts the dimensions proposed by Ivančić et al. (2019), albeit with minor adaptations (we replaced customer with citizen, and created the Legal/Normative dimension). This type of organization is in line with arguments that state that DT goes beyond technology and also covers strategy, organization, teams, and more (Rogers, 2016; Tabrizi et al., 2019). The success factors gathered from the analyzed articles were subsequently consolidated and systematized and can be seen in Table 4, with respective references, clustered according to the following dimensions: strategy, people, organization, citizen, ecosystem, technology, innovation, and legal/normative. In each dimension, the success factors are ordered according to the number of references that have cited it (or a homonym).
Success factors
Success factors
Case studies
Regarding the first complementary research question, the case studies identified and analyzed provide evidence of the predominance of DTPS cases related to the executive branch at the central government level.
Denmark and its Faroe Islands territory are noteworthy cases. Denmark boasts the best UN-EGDI 2020 and also ranks in the third position in EU Digital Economy and Society Index – DESI 2020. The Danish case studies have focused on (1) a service; (2) the country as a whole; and (3) enablers of the Danish approach. Besides, as a microstate, the Faroe Islands had two studies evaluating ICT adoption in the territory.
In turn, India has the highest number of published cases analyzed: three in 2017 and three in 2019. The cases from 2017 report (1) the automation of two public services, and (2) a large-scale service implemented via an agile approach at the central and regional levels. Moreover, the cases published in 2019 address (1) a framework for advising, evaluating, and accelerating DT; (2) ten automation projects that reduced service time, among other benefits; (3) DT as a means for reducing corruption.
Another aspect worth mentioning is the description of interaction mechanisms between the government levels in studies that assess cases implemented in Norway, Denmark, and Sweden.
Valuable lessons learned can be identified from the main results presented in each case, and the case list constitutes a benchmark index for practitioners.
Processes
The second complementary research question regards the processes applied to digital transformation projects in the public sector. The articles focused on processes are fewer in number than those that describe cases or success factors. Nonetheless, due to its transformational character and scope of application, processes are an important object of study.
Regarding the classification by typology, despite the idealized classification, all the identified processes were classified as project life cycle-centric type, with more significant concern for implementation. In this sense, all project life cycle-centric studies describe the stages of a DT project.
Another key point worth highlighting in this type of project is the concern not only with closure but also with the assessment of post-implantation impact; and, in some cases, also concerns with maintenance, support, and updating. This post-implantation concern seems to be focused on the effectiveness of actions, and directly contributes to the realization of the strategy. It also aligns with more modern concepts of project management (Varajão, 2018), such as the extension of the project’s life cycle to cover the impact analysis and promote any necessary adjustments.
Practitioners find in the reported processes several alternatives for defining their digital transformation projects in the public sector.
Success factors
The main research question regards the factors that can influence the success of digital transformation initiatives in the public sector. After consolidation and systematization, the analysis of the success factors, based on the number of references to each one, allowed identifying the most reported success factors in DTPS.
Regarding the strategy dimension, the most cited success factors refer to the need of governance structures and practices (factors 1.1 and 1.2) to enable DT, followed by adoption of portfolio, program, and project management (factor 1.3), as well as IT leadership (factor 1.4), and the support of top management (factor 1.5). The next most relevant factors are securing political support and sustaining IT investments (factors 1.6 and 1.7) to provide stability and sustainability to the initiative. It is also worth highlighting the factor of promoting trust in e-Gov services (factor 1.10), which is in line with the arguments by Clarke (2020) regarding increasing public trust via the application of ICT (Mahmood, 2016; Morgeson III et al., 2011; Tolbert & Mossberger, 2006).
Parallel to this, people play a central role in DT initiatives in the public sector, as they will conceive, plan and execute projects, as well as sustain the initiatives. The most mentioned success factor is ensuring team awareness and training (factor 2.1), via generic training or via complementary training with specialization on the subject (factors 2.2, 2.3, and 2.4).
Organization is the most populated dimension. In this dimension, the most mentioned success factor regards multilevel governance and management structures (factor 3.1). This factor can be perceived as an enabler of another one – governance structures to enable DT (the most cited regarding the strategic dimension), as both seem critical to the success of initiatives. The second position in this dimension – Promote transparency and openness (factor 3.2) – enables the accountability necessary for the social control of expenses and actions carried out by public agents. The next most mentioned success factor concerns ensuring that the right positions are assigned to leaders and decision-makers (factor 3.3). Still, in the dimension of organization, mention should be made of the adoption of agile approaches (factor 3.4), GDPR compliance (factor 3.5), and, from a more operational perspective, concerns about organizational and process changes, by promoting integration and rationalization, as argued by Mergel et al. (2019).
Within the scope of DTPS initiatives, the citizen is no longer a mere consumer but a co-creator for the successful operationalization of broad digital transformation initiatives in the public sector, as evidenced by the most cited success factor within the citizen dimension (factor 4.1). The citizen involvement is also related to the post-production stage, as it fosters communication and encourages feedback. Still within this dimension, promoting acceptance, adoption, and use of e-Gov feature among the most cited success factors (factor 4.2).
In this sense, the government should collaborate with citizens and partners – including other public agencies, academia, non-profit organizations, and companies – in co-creating value and establishing an ecosystem. This collaborative orientation is patent in the success factors within the ecosystem dimension (factors 5.1 and 5.3); moreover, reducing the digital divide (factor 5.2) is critical to promote acceptance and adoption of e-Gov initiatives.
As mentioned before, DTPS should not be focused on the technology but rather on how governments interact with stakeholders (Barcevičius et al., 2019; Mergel et al., 2019; Williams & Valayer, 2018). However, technology is still an important part and it essentially plays the role of a commodity that enables new services and promotes usability (factor 6.2), being required a proper technological infrastructure (factor 6.1). Despite the non-technological focus, considering emerging digital technologies as an innovation driver (factor 6.3) is also a success factor.
The next dimension, the innovation dimension, features only two factors, each with a single citation, focused on encouraging people to innovate (factor 7.1) and understanding the continuous cycle arising from innovation (factor 7.2). For instance, innovation can generate a new service, which, in turn, when used, will recursively generate pressure for a new innovation cycle. This example illustrates the importance of this dimension.
Finally, the legal and normative dimension is critically focused on reducing the legal complexity of services (factor 8.1), which, together with the legal and regulatory framework (factor 8.3), form the legal foundations for digital transformation initiatives. Still, regarding this dimension, it is worth mentioning the explicit goal of fighting corruption (factor 8.2), which stems as one of the benefits derived from DTPS initiatives.
The presented unified framework of success factors for digital transformation projects in the public sector can be used by practitioners as a checklist of factors in their projects, aiming to reduce the risk of failure and leverage success.
Conclusions, limitations, and further work
From the results presented, it is possible to conclude that the extant literature on DTPS offers a wealth of case studies and success factors. In contrast, there is a scarcity of studies describing the processes that underpinned those initiatives or cases.
From a geographical point of view, i.e., taking the countries under study as parameter, the analyzed studies cover the main countries featuring in the UN-EGDI ranking, and are therefore a valuable resource to learn about the best practices. Besides showcasing the experience of countries well ranked in the UN-EGDI, the case studies under analysis also describe the reality of other countries on the African, Asian, and American continents. However, the studies focused on some developed countries are scarce, such as the United States of America, New Zealand, the Netherlands, or Japan, to name a few. The general lack of information regarding this reality does not allow us to infer the cause.
Regarding the processes applied within the scope of DTPS initiatives, the most commonly implemented are project life cycle-centric, compared to e-readiness and strategy-based. It is noticeable a concern with the ex-ante and ex-post phases of projects, regardless of the type of process implemented, seeking effectiveness and alignment with strategy.
By presenting cases, processes, and success factors, there is both a theoretical and practical contribution. In terms of practical contribution, to highlight that professionals in the area can find in our study a compilation of several alternative processes already adopted within the scope of other initiatives, which can be taken into consideration for organizing their digital transformation projects. Practitioners also find a list of success factors that can be used to minimize risks and improve the success of DT projects. In addition, the several cases identified can be used as a benchmark for good practices and to identify new project opportunities.
In terms of theoretical contributions, researchers can find in our study a unified framework of success factors for digital transformation projects in the public sector that enriches the extant body of literature. The compilation of alternative digital transformation processes already implemented in the public sector can serve as a starting point for a new research strand focusing on the development of a unified process based on good practices. The catalog of cases is also useful to identify research gaps and the need for further research.
Our study has some limitations. The search terms used for collecting the articles for the purpose of this literature review may be limited in scope despite having been thought through comprehensively. Thus, synonyms or related concepts may be addressed in future research to build on this work. The focus of our review was on research literature, and future research may focus on grey literature. Moreover, private sector initiatives were not considered, since our study aimed primarily to consolidate the factors that can impact the success of digital transformation initiatives in the public sector.
As future work, beyond exploring the limitations mentioned above, the legal framework for DTPS in the top-ranked countries in UN-EGDI could be analyzed to identify a possible relationship between the legal framework and success in adoption; moreover, a future update of this systematic review will be useful to allow comparing the results pre, during, and post-pandemic. Additionally, regarding success factors, an instrument (e.g., a checklist) could be derived to create a DTPS project readiness index, supporting the evaluation of the conditions to carry out a DT project and assessing the gap between reality and the required environment. Also, a unified process could be derived from the consolidation of the processes analyzed, both by considering the pre-project, planning, execution, and closure phases and the post-implantation phase, and by relating the success factors to the different phases.
Footnotes
Authors biographies
Fernando Escobar is a PhD Student at the University of Minho. He holds a Master’s in Applied Computing from UNB, with research in risk management. He also has Postgraduate in Public Administration from the National School of Public Administration; MBA in Electronic Government from UNA-MG; Bachelor’s Technologist in Data Processing from State University (FATEC-SP). Since August 2016, he is Federal Servant, in the position of IT Judicial Analyst at Tribunal Regional Federal – TRF1, where since May 2018 he has been Supervisor of the Information Systems Management Department. Escobar has over 25 years of experience and is also a PMP.
Washington H. C. Almeida is a PhD Student at the Center of Advanced Studies and Systems of Recife – C.E.S.A.R. Master Degree in Software Engineering at C.E.S.A.R. Specialist in Information Security Management at the University of Brasília (2011). Holds a degree in Information Technology from the Federal Institute of Piauí – IFPI (2004). He is currently a Federal Judicial Analyst (TRF1). He was a Substitute Professor at the University of Brasília – UNB in the Computer Science Department. ISF Certificate ISO/IEC 27002. Has experience in the area of IT, mainly in topics: Information Security, Software Engineering, IT Management and IT Governance.
João Varajão is currently a professor of Information Systems (IS) and Project Management (PM) at the University of Minho. He is also a researcher of the ALGORITMI center. His current research interests are in IS, PM, and IS Development (addressing, particularly, IS and PM success). Before joining academia, he worked as an IS consultant, project manager, IS analyst and software developer, for private companies and public institutions. He has published numerous refereed publications, authored books, edited books, as well as book chapters and communications at conferences. He serves as editor-in-chief, associate editor and member of committees for international journals and conferences.
Appendix A: Queries applied
Base
Query applied
ACM
[[[All: “digital transformation framework”] OR [All: “digital transformation model”] OR [All: “digital transformation”] OR [All: digitization] OR [All: digitalization]] AND [All: “public sector”]] OR [All: “public administration”] OR [All: “public service”] OR [All: “Government”] OR [All: “public sphere”] OR [All: “public organization”] AND [Publication Date: (01/01/2015 TO 03/31/2020)]
IEEE
((((((((((“All Metadata”: “Digital transformation framework”) OR “All Metadata”: “Digital transformation model”) OR “All Metadata”: “Digital transformation”) OR “All Metadata”: Digitization) OR “All Metadata”: Digitalization) AND “All Metadata”: “Public sector”) OR “All Metadata”: “Public administration”) OR “All Metadata”: “public service”) OR “All Metadata”: “Government”) OR “All Metadata”: “Public sphere”)OR “All Metadata”: “Public Organization”)
ISI
TS
((Digital transformation framework OR Digital transformation model OR Digital transformation OR Digitization OR Digitalization) AND Public sector OR Public administration OR public service OR Government OR Public sphere OR Public Organization)
Appendix B: Articles,contents,and quality criteria
Ref
Case study
Process
Success factor
Limitations
Future works
Final score
(Marzooqi et al., 2017)
X
X
X
X
4
(Carrasqueiro et al., 2018)
X
X
X
X
4
(Sundberg, 2018)
X
X
X
X
4
(Lappi et al., 2019)
X
X
X
X
4
(Pereira et al., 2017)
X
X
X
3
(Singh et al., 2017)
X
X
X
3
(Almukhlifi et al., 2019)
X
X
X
3
(Mergel et al., 2019)
X
X
X
3
(Lindgren & van Veenstra, 2018)
X
X
X
3
(Kompella, 2019)
X
X
X
3
(Lessa & Tsegaye, 2019)
X
X
X
3
(Rehouma & Hofmann, 2018)
X
X
X
3
(Schedler et al., 2019)
X
X
X
3
(Madsen & Kraemmergaard, 2016)
X
X
X
3
(Avila-Maravilla & Luna-Reyes, 2018)
X
X
X
3
(Putri & Sensuse, 2018)
X
X
X
3
(Kaya et al., 2020)
X
X
X
3
(McBride, 2019)
X
X
X
3
(Sangolt & Keitsch, 2016)
X
X
X
3
(Lessa, 2019)
X
X
X
3
(Müller et al., 2018)
X
X
X
3
(Ngonzi & Sewchurran, 2019)
X
X
X
3
(Tchao et al., 2017)
X
X
X
3
(Tambotoh et al., 2017)
X
X
X
3
(Choi et al., 2016)
X
X
X
3
(Cahlikova, 2017)
X
X
2
(Nerurkar & Das, 2017a)
X
X
2
(Hatsu & Ngassam, 2017)
X
X
2
(Al-Shuaili et al., 2019)
X
X
2
(Nerurkar & Das, 2017b)
X
X
2
Ref
Case study
Process
Success factor
Limitations
Future works
Final score
(Filgueiras et al., 2019)
X
X
2
(Rakotonirina & Raoelson, 2018)
X
X
2
(Meyerhoff Nielsen, 2019)
X
X
2
(Paletti, 2018)
X
X
2
(Reascos et al., 2019)
X
X
2
(Enaw et al., 2016)
X
X
2
(Fajar Surya & Amalia, 2017)
X
X
2
(Renteria et al., 2019)
X
X
2
(Defriani & Resmi, 2019)
X
X
2
(Avramchikova et al., 2019)
X
1
(Marino & Pariso, 2019)
X
1
(Scupola, 2018)
X
1
(Prakash & Gunalan, 2019)
X
1
(Ivanova et al., 2019)
X
1
(Shin & Rakhmatullayev, 2019)
X
1
(Mayakul et al., 2019)
X
1
(Ayquipa et al., 2019)
X
1
(Nielsen, 2016)
X
1
(Agarwal et al., 2017)
X
1
(Osifo, 2018)
X
1
(Pappel et al., 2019)
X
1
(Carrillo et al., 2017)
X
1
(Roy, 2019)
X
1
(Anand & Khemchandani, 2019)
X
1
(Rammea & Grobbelaar, 2017)
X
1
(Kim et al., 2016)
X
1
