A framework to support Robotic process automation

Abstract

With the increasing demand for digitalization, organizations look to emerging technologies such as Robotic Process Automation (RPA) to increase their business performance. This makes it essential to identify and select the most suitable processes to maximize the benefits for organizations. However, despite the increasing interest of academics and professionals in RPA, the literature lacks a study on the main criteria organizations should consider to decide which processes to automate. Therefore, this research lists the main criteria for process automation based on scientific and professional know-how. A systematic literature review was performed, followed by a Delphi study with RPA professionals, to tune the former insights collected from the scientific literature. Our findings point to 32 criteria that organizations and decision-makers should consider before choosing which processes to automate. Feasibility, process description, and input and output data are the most voted. The criteria are evaluated with 18 processes in six organizations with positive results. While professionals may find valuable information in this document to help them decide which processes must be automated first, academics are now aware of which areas deserve further investigation.

Keywords

Robotic process automation selection robotic process automation criteria Delphi method process automation

Introduction

Robotic Process Automation (RPA) is a topic that has been getting more and more attention in the past few years, both academically and in organizations (Ratia et al., 2018). Despite the absence of a consensual definition of RPA, there is a common idea that most researchers share, which characterizes RPA as a technology allowing human users to be entirely or partially disengaged from business processes that are performed by software robots. These software robots mimic the actions the same way a human user would but faster (Santos et al., 2019) and avoid human error (Riedl and Beetz, 2019).

Currently, increasingly more attention is devoted to the digitization of operations and business processes. The concept of digitization covers service enterprises, including industries such as finance, banking, insurance, marketing, accounting, public administration, logistics, and others (Ratia et al., 2018; Willcocks et al., 2017). Therefore, the RPA industry is proliferating, driven by digital business demands

However, the amount of processes that can be automated is still limited due to the lack of cognitive ability that RPAs still have (Lacity et al., 2015). But, in the future, with the help of Artificial Intelligence (AI) and Machine Learning (ML), RPAs are expected to be more intelligent (Santos et al., 2019).

As with any emerging new technology, there will be many benefits that organizations would like to take advantage of. RPA is not an exception. However, adverse effects are also significant when bad decisions are taken (Madakam et al., 2019). Therefore, selecting the most suitable processes to automate is vital to bring more value and maximize the benefits. Consequently, it is essential to know which criteria organizations should consider automating, given their context and strategy.

As far as the authors could reach, there is no literature proposing a list of criteria organizations could use to assist decision-makers. Therefore, this research aims to identify the criteria for selecting the best-suited processes to automate. Since academics' and professionals’ opinions are relevant, the adopted methodology integrates a systematic literature review (SLR), then adjusted through a Fuzzy Delphi with 34 experts to produce the final list of criteria, which may then be evaluated with real-world cases.

Robotic Process Automation

What is Robotic Process Automation

The objective of RPA is to disengage human intervention from repetitive processes and replace them with software robots. Robots will take care of more administrative tasks while the employees can focus on demanding cognitive tasks (Nawaz, 2019). Furthermore, software robots use the same interfaces that a human does while executing the business process (Riedl and Beetz, 2019). This implies that the flow performed by a software robot is the same that a human would use (Riedl and Beetz, 2019).

Software robots can be split into two categories: unattended and attended. Unattended software robots are categorized as running 24 hours, seven days a week, without stopping (Hofmann et al., 2020).

Typically, these software robots do not have any human intervention except punctual exceptions, which the software robot cannot solve by himself. They also require inputs with a suitable data structure to properly manage the data and carry with the standard workflow of the everyday business process transactions, which helps reduce exceptions while performing the tasks (Santos et al., 2019).

On the other hand, unattended projects typically take more time and expertise to develop and bigger development teams due to the hidden complexity that a business process could have, even if the business process seems very simple on the surface (Hofmann et al., 2020). The other type of software robot is attended software robots. These software robots can do the same processes as the unattended ones but work alongside humans.

Therefore, they do not run constantly, and they only are used when a human decides that there are transactions that the software robot can perform. As a result, attended robots are faster to develop, and unlike unattended robots, they are cheaper to produce (Hofmann et al., 2020).

Organizations can target one type or combine both types depending on the objectives. For example, unattended robots are used for high amounts of transaction processes that have not changed in the last 12 months. On the other hand, attended robots might offer more protection to the organization because a human user will always oversee what the robot is doing (Hofmann et al., 2020).

In summary, RPA is a new technical approach to process automation that can enable a technology-induced digital transformation (Lacity et al., 2015; Willcocks et al., 2017).

Robotic Process Automation in Practice

With the constant digitalization in organizations, there will be a value creation increase due to the utilization and deployment of RPA tools. In addition, RPA appears to help organizations automate their processes in a way that is faster and reduces their employees' workload on unnecessary time-consuming tasks (Nawaz, 2019).

Robots can work without stopping, while humans need a work schedule and breaks, therefore not producing the same as a robot (Ansari et al., 2019), reducing the employees' manual work. This way, employees can focus on tasks that a robot cannot do or even spend time learning new competencies (Madakam et al., 2019).

Reviewing business processes to automate and automating those processes will help the organization standardize its ways of working, resulting in a better data management procedure between processes and making them more efficient.

Robots will perform the same rules for a business process that removes human error from the transaction flow, which a human cannot consistently achieve (Itpa, 2013).

Software robots are much faster to develop than other IT tools and do not require a big team of specialists to build them.

RPA can help create new jobs, such as robot management. However, even though the robots can work by themselves and handle a few exceptions, it will always be necessary to have a support team to take care of these tools despite being attended to or unattended (Santos et al., 2019).

In any technology, there are many advantages and disadvantages. Even though the RPA advantages are much more significant and can overshadow the weaknesses, knowing them is still very important when deciding if RPA is the best approach for a case.

The most significant disadvantage of an RPA adoption is job loss, a valid fear that any organization should consider. However, the job loss that RPA represents can be transformed in new hires because RPA solutions will need constant support from human workers. For example, a human worker will do that task if the robot cannot handle an exception.

RPA solutions are still short and medium-term solutions. These tools are not yet prepared to work on a business process for the long term, discouraging its development and encouraging management to look for other solutions (Nawaz, 2019).

Research methodology

The authors perform an SLR to unveil and synthesize the criteria reported by other researchers into a list of criteria. Then, this list is used as the input for a Delphi study and tuned by RPA experts. Figure 1 presents the research methodology followed in this investigation.

Figure 1.

Research Methodology Steps.

Systematic literature review

The structure used to conduct the SLR (Figure 2) follows the guidelines proposed by Kitchenham (2004) and Okoli (2015), which keeps this review scientifically rigorous and transparent, which ultimately improves the literature review (Okoli, 2015). This approach is composed of three phases that should be done sequentially, and each stage has its steps to ensure the results will have the required quality.

Figure 2.

Systematic Literature Review Steps.

Outlining systematic literature review

Since this research focuses on identifying the criteria to select the most suitable business processes to automate, this SLR aims to analyze RPA implementation studies with information regarding the process selection and the criteria used. Then, six electronic repositories were used: IEEE, ACM, SCHOLAR, SpringerLink, Web of Science, and Scopus. The following search string was used in each repository: (“Robotic Process Automation” OR “RPA”) “AND” (“selection” or “criteria”). Only English publications were considered.

Conducting a systematic literature review

Two filters were applied to reach the most relevant articles (Table 1). The first filter aims to gather the articles with keywords in the abstract or title to ensure that only relevant articles were selected. The second filter seeks to remove any duplicate articles. Then, a snowballing technique was applied to include all pertinent studies missing the search string. The papers found with the snowballing technique were important to further support the conclusions of this research. In the end, 46 final articles were collected. The entire filtration process is visible in Figure 3.

Table 1.

Amount Articles resulting from the filtering process.

Digital Library	No Filter	First Filter	Second Filter	Snowballing
IEEE	237	18	2	2
ACM	346	5	3	3
Scholar	7859	31	14	18
SpringerLink	12,544	10	10	10
Web of Science	792	5	5	8
Scopus	874	31	4	5
Total	22,652	100	38	46

Figure 3.

Filtration Flow.

After the conclusion of the filtration process, an analysis of the resulting articles is conducted. The published year, business sector, model type or process criteria, geographical location, and other characteristics are extracted for each article.

Most of the articles collected were published in journals between 2019 and 2021. This reinforces the relevance of the topic.

As shown in Table 2, there is a lack of literature surrounding this research topic. This can be justified by the fact that RPA is still a recent technology or because authors do not explain the development of selected models to select the best processes to automate - either because the projects are small, have low complexity, or the push for RPA in the organizations is not substantial.

Table 2.

Extracted articles analysis vectors.

Vector Ref	Country	Provides a model or criteria	Year	Specifies the business sector
(Ratia et al., 2018)	Germany	Criteria	2020	—
(Santos et al., 2019)	Portugal	Criteria	2019	—
(Riedl and Beetz, 2019)	Australia, Estonia, Italy	Criteria	2020	—
(Hofmann et al., 2020)	Germany	Model and Criteria	2019	—
(Madakam et al., 2019)	India	Criteria	2019	Banking, Insurance, Healthcare, Manufacturing, Telecom, Energy
(Nawaz, 2019)	Malaysia	Criteria	2014	HR
(Ansari et al., 2019)	India	Criteria	2019	HR
(Itpa, 2013)	Germany	Criteria	2019	—
(Wellmann et al., 2020)	Germany	Criteria	2020	—
(Leno et al., 2020)	Australia	Criteria	2008	—
(Pranav, Manish Kumar, and Srinivasan, 2020)	India	Criteria	2020	—
(Liu et al., 2020)	Bahrain	Criteria	2019	—
(Chacón-Montero, Jiménez-Ramírez, and Enríquez, 2019)	Spain	Criteria	2020	—
(Chacón-Montero, Jiménez-Ramírez, and Enríquez, 2019)	Indonesia	Criteria	2021	Audit
(Zhang and Liu, 2018)	China	Criteria	2018	Economy
(Ratia et al., 2018)	Finland	Criteria	2018	Healthcare
(Rhouati, et al.,2021)	France	Criteria	2021	Supply Chain
(Enriquez et al., 2020)	Spain	Criteria	2020	Industrial
(Wewerka and Reichert, 2020)	Germany	Criteria	2020	Banking, Telecom
(Moffitt, Rozario, and Vasarhelyi, 2018)	USA	Criteria	2018	Audit
(Ivančić, Suša Vugec and Bosilj Vukšić, 2019)	Croatia	Criteria	2019	—
(Siderska, 2020)	Poland	Criteria	2020	IT
(Huang and Vasarhelyi, 2019)	USA	Criteria	2019	Audit
(Syed et al., 2020)	Australia	Criteria	2019	—
(Osman, 2019)	Romania	Criteria	2019	Telecom, Insurance, Finance
(Leopold, van der Aa, and Reijers, 2018)	Netherlands	Criteria	2018	IT
(Wewerka and Reichert, 2021)	Germany	Criteria	2012	—
(Völker, Siegert, and Weske, 2021)	Germany	Criteria	2021	—
(Lau et al., 2000)	Hong Kong	-	2000	—
(Herm et al., 2020)	Germany	Criteria	2020	—
(Axmann et al., 2021)	Germany	Criteria	2021	—
(Griffiths and Pretorius, 2021)	South Africa	Criteria	2021	Audit
(Viehhauser and Doerr, 2021)	Germany	Criteria	2021	—
(Ghose, Mahala, Pulawski and Dam, 2021)	Australia	Criteria	2021	HR, Banking, Telecom
(Choi, , 2021)	South Korea	Criteria	2021	—
(Fominykh, et al., 2018)	Russia	-	2018	Construction
(Rose, 2013)	USA	-	2017	Military
(Marciniak and Stanisławski, 2021)	Poland	Criteria	2021	Banking, Telecom, HR
(Lacity and Willcocks, 2021)	USA and UK	Criteria	2021	—
(Davenport, 2018)	USA	Criteria	2018	—
(Davenport and Ronanki, 2018)	USA	Criteria	2018	—
(Gosh, Prasad and Pallail, 2022)	-	Criteria	2022	Enterprise Tech Investment
(Hallikainen et al., 2018)	Australia, Finland	Criteria	2018	Business Process Outsourcing
(Lacity et al., 2015)	USA and UK	Criteria	2015	Telecommunications
(Lacity et al., 2021)	USA, UK and Australia	Criteria	2021	Information Systems
(Lacity and Willcocks, 2016)	USA and UK	Criteria	2016	—

The analysis of each article is presented in Table 2, following the concept theory (Kent and Saffer, 2014). In addition, some vectors were used to help our classification: country and year of the study, if the article provides a model or criteria, and if it specifies any business sector where the criteria mentioned should be used.

Reporting the findings

Several criteria were elicited from the literature. Below, one can find a brief description of each criteria definition and why it matters. Table 3 summarizes the criteria.

Table 3.

Business and Technical Criteria for business process selection.

Condition	Ref
Amount of human intervention	(Ratia, Myllärniemi, and Helander, 2018; Santos, Pereira, and Vasconcelos, 2019; Riedl and Beetz, 2019)
Data structure of inputs	(Santos, Pereira, and Vasconcelos, 2019; Riedl and Beetz, 2019; Wellmann, et al., 2020; Rhouati, et al.,2021; Wewerka and Reichert, 2020; Siderska, 2020; Syed, et al., 2020; Lau et al., 2000; Griffiths and Pretorius, 2021; Viehhauser and Doerr, 2021; Marciniak and Stanisławski, 2021; Lacity and Willcocks, 2021; Lacity et al., 2021)
Rule-based process	(Santos et al., 2019; Riedl and Beetz, 2019; Hofmann et al., 2020; Wellmann et al.,, 2020; Rhouati, 2021; Enriquez, 2020; Wewerka and Reichert, 2020; Wewerka and Reichert, 2020; Ivančić et al., 2019; Siderska 2020; Syed et al., 2020; Osman 2019; Völker, Siegert, and Weske,, 2021; Ghose et al., 2021; Marciniak and Stanisławski, 2021; Lacity and Willcocks, 2021; Davenport, 2018; Hallikainen et al., 2018; Lacity, Willcocks and Gozman, 2021)
Environments	(Santos, Pereira, and Vasconcelos, 2019; Riedl and Beetz, 2019; Lau et al., 2000; Davenport and Ronanki, 2018; Hallikainen et al., 2018)
Process standardized	(Santos et al., 2019; Riedl and Beetz, 2019; Hofmann et al., 2020; Wellmann et al., 2020; Wewerka and Reichert, 2020; Wewerka and Reichert, , 2021; Herm et al., 2020; Viehhauser and Doerr, 2021; Davenport, 2018; Lacity et al., 2015; Lacity and Willcocks, 2016)
Process stability	(Santos et al., 2019; Syed et al., 2020; Wewerka and Reichert, 2021; Lau et al., 2000)
Number of exceptions	(Santos et al., 2019; Riedl and Beetz, 2019; Wellmann et al., 2020; Rhouati et al., 2021)
Business value	(Wellmann et al., 2020; Lau et al., 2000; Gosh, Prasad and Pallail, 2022; Lacity et al., 2021)
Number of transactions	(Ratia et al., 2018; Santos et al., 2019; Wellmann et al., 2020; Rhouati et al., 2021; Wewerka and Reichert, 2020; Huang and Vasarhelyi, 2019), Völker etal., 2021; Lau et al., 2000; Herm et al., 2020; Viehhauser and Doerr, 2021; Marciniak and Stanisławski, 2021; Hallikainen et al., 2018; Lacity et al., 2015; Lacity et al., 2021)
None or low cognitive capabilities	(Santos et al., 2019; Wellmann et al., 2020)
Repetitive	(Santos et al., 2019; Wellmann et al., 2020; Rhouati et al., 2021; Wewerka and Reichert, 2020; Ivančić, Suša Vugec and Bosilj Vukšić, 2019; Siderska, 2020; Syed et al., 2020; Leopold, van der Aa, and Reijers, 2018; Griffiths and Pretorius, 2021; Marciniak and Stanisławski, 2021; Lacity and Willcocks, 2021; Gosh, Prasad and Pallail, 2022; Hallikainen et al., 2018; Lacity et al., 2021; Lacity and Willcocks, 2016)
Existing stable environments	(Riedl and Richard Beetz, 2019)
Process cost	(Ratia et al., 2018; Riedl and Beetz, 2019)
Human Effort	(Wewerka and Reichert, 2020; Lau et al., 2000)
Data Digitalization	(Zhang and Liu, 2018; Syed et al., 2020; Griffiths and Pretorius, 2021; Viehhauser and Doerr, 2021)
Duration	(Rhouati et al., 2021; Wewerka and Reichert, 2020; Syed et al., 2020), (Lau et al., 2000; Choi, 2021; Lacity and Willcocks, 2021)
Human Error	(Lau et al., 2000)
Complexity	(Wewerka, and Reichert, 2020; Syed et al., 2020; Lacity, Willcocks and Craig, 2015)

With the list of the most relevant criteria present in the scientific literature for process automation identified, it is now time to tune it with inputs from practitioners.

The Delphi methodology

The Delphi method is characterized as a way of structuring group communication and is essential to gather the feedback of each contributor, assess the group opinion, and have anonymity for the participants' responses. Thus, questionnaires are used with controlled feedback, avoiding confrontation among the participants (Okoli and Pawlowski, 2004) so that every participant can fully express their ideas. It is essential to be a group of experts on the research matter (Okoli and Pawlowski, 2004) to provide credibility to the artifact.

However, the consensus calculation in the Delphi method can be extensive, especially if there is a big group of experts. Moreover, this calculation happens every round since the opinions of each expert on each item can be different, which results in many rounds, and therefore increase the drop rate of the questionnaires, finally making the result of the Delphi less substantial for the subject matter study (Kuo, 1998). Consequently, it is required to use a method that allows an easier way to handle the consensus between each item and convert the subjective evaluation to a quantitative measure. The Fuzzy Delphi method, a subset of the Delphi method, was used based on the requirements already mentioned (Rejab et al., 2019).

There is a significant advantage with the Fuzzy Delphi method when evaluating an extensive set of items. Each item can be evaluated by itself to obtain consensus. Such evaluation can help the researcher discard the item in question to get the consensus as intended (Rigby et al., 2012).

The Delphi is developed based on the list of criteria mentioned earlier as input and follows the flow of the Delphi method (Figure 4). By using the Delphi methodology, the authors aim to tune the former list of criteria using RPA experts and reach a suitable list of criteria to assess processes to be automated.

Figure 4.

Delphi Methodology.

Expert selection

Okoli and Pawlowski (2004) discussed how experts should be selected to increase rigor in the study, where the focus is to identify the kind of expertise required by the participant initiating the study.

In this research, 55 potential participants were identified. These participants were selected based on their expertise regarding RPAs; 34 participants accepted to continue the study, of which 17 were RPA developers, and 17 were RPA analysts.

Communication protocol

The Delphi study requires communication with these experts to be maintained anonymously. The invitations were sent via electronic email with an overview of the Delphi study and the start date of the first round

Survey design

This Delphi study aims to get the expert panel’s consensus on the criteria that should be used to evaluate business processes to automate.

In this Delphi study, the surveys can be divided into two types. The first one used in the first round of the survey was compounded by open questions to retrieve order criteria not present on the initial criteria list obtained from the SLR in Section 2.

Delphi study instruments validation

An essential process in a Delphi study is the development of instruments, and these instruments can be questionnaires used to collect data on a particular subject matter being investigated. The questionnaires can have one or two questions, so it is possible to understand the general opinion of the participants in the study or multiple questions that the participants need to respond to based on their expertise.

The responses from the participants are analyzed at the end of each round. Typically, the first questionnaire is designed with a few questions to give an objective perspective to the researchers for the subsequent questionnaires. The objective is to attain the group's consensus from the second questionnaire. The steps for this calculation are demonstrated in Figure 5 (Nurul et al., 2019).

Figure 5.

Consensus calculation steps for Fuzzy Delphi.

After analyzing the round results, it is necessary to evaluate the next step. If a consensus is not obtained, it must send another questionnaire to verify the participants' opinions. Although this process can result in multiple rounds, which can cause the drop rate per round to increase, the Delphi study can stop if the consensus is achieved or if the study is at a point of saturation of results.

The first round of the Delphi study aimed to collect a set of criteria in the form of open questions such as: “Which is the most important criterion to consider when evaluating business processes for automation, please provide as many criteria as you can.”

For the first questionnaire, 34 experts answered, which resulted in a drop rate for that round of 0%. Of the 34 experts, 17 had a management-type role, and the other 17 had a developer role regarding RPA. Figure 6 summarizes the experience of the experts in the field.

Figure 6.

Distribution of How Many Years the Experts Have Been Working in the RPA Field.

From the unstructured responses of the experts, with the addition of the 18 criteria collected from the SLR, it was possible to retrieve a list with 33 criteria presented in Table 4. This table showcases the name of the criterion retrieved and how many mentions of that criterion were obtained from the experts.

Table 4.

Criterion list obtained from the Delphi study.

Item Number	Criterions obtained from the Delphi	Mentions in Delphi	Corresponding SLR criterion
1	Accurate process description	9
2	Application Access	1
3	Automation Type	3
4	Data security	2
5	Efficiency	2
6	Feasibility	3
7	Input and Output data	19	The data structure of inputs
8	Human effort	1	Human effort
9	Manual involvement	2
10	Cognitive requirements	2	Amount of human intervention; None or low cognitive capabilities
11	Number of exceptions	17	Number of exceptions
12	Number of process steps	1
13	Number of robots allowed	1
14	Number of applications involved	14	Environments
15	Number of users	3
16	OCR involved	1
17	Predictability of outcomes	1
18	Process complexity	6	Complexity
19	Process cost	1	Process cost
20	Data Digitalization	1	Data Digitalization
21	Process stability	1	Process stability
22	Process standardized	8	Process standardized
23	Repetitive	15	Repetitive
24	Reusability	2
25	Human error	3	Human Error
26	Rule-based process	5	Rule-based process
27	Savings	17	Business value
28	Applications similarity	1
29	SLA impact	3
30	Applications maturity	8	Existing stable environments
31	Test data	1
32	Time-consuming	5	Duration
33	Number of transactions	7	Number of transactions

Fuzzy Delphi second round

From the second round onwards, the objective was to achieve the consensus of the group study. Therefore, a questionnaire was created where the experts evaluated each criterion detailed in Table 4 by a Likert scale with values between 1 and 5. As a result, of the 34 participants in the second round, 31 answered with a drop rate of 8.82%, which respects the 30% drop rate per round stated in the literature (Rigby et al., 2012).

The Fuzzy Delphi method was used, as mentioned in Section 3. This method facilitates consensus calculation because it can be calculated by item, not by round. Therefore, it was followed a set of specific steps to calculate the consensus, starting with the definition of the Fuzzy scale selection, presented in Table 5, which then will be used to translate the Likert values from the questionnaires to values between 0 and 1 to be able to perform all the calculations for the group consensus.

Table 5.

Fuzzy Scale selection.

Approval level	Fuzzy Scale
Extremely High (5)	0,6	0,8	1
High (4)	0,4	0,6	0,8
Fair (3)	0,2	0,4	0,6
Low (2)	0	0,2	0,4
Very Low (1)	0	0	0,2

The following step was to calculate the average values of m1, m2, and m3, which represent the minimum (m1) value, the reasonable value (m2), and the maximum value (m3) from the Fuzzy Scale. In this step, the values obtained from the questionnaire between 1 and 5 are translated according to Table 6. Thus, each item from one evaluation will have three different m (m1, m2, and m3) values. To calculate the value of the expert agreement level for each item d per item, the equation (3.1) was used:

d = \sqrt{(\frac{1}{3}} * {(m 1 - c 1)}^{2} + {(m 2 - c 2)}^{2} + {(m 3 - c 3)}^{2})

(3.1)

Table 6.

Criterion consensus calculation for rounds 2 and 3.

Item	Item Round 2				Round 3
	Average			Value of d	Average			Value of d	Item Number d ≤ 0.2	Percent of Each Item d ≤ 0.2	Fuzzy Evaluation	Score
	m1	m2	m3	Value of d	m1	m2	m3	Value of d	Item Number d ≤ 0.2	Percent of Each Item d ≤ 0.2	Fuzzy Evaluation	Score
1	0,522581	0,722581	0,922581	0,155	0,550000	0,750000	0,950000	0,079	27	96%	0,750000	2
2	0,477419	0,683871	0,877419	0,201	0,521429	0,714286	0,921429	0,111	25	89%	0,719048	4
3	0,419355	0,619355	0,819355	0,273	0,385714	0,585714	0,785714	0,093	23	82%	0,585714	27
4	0,490323	0,690323	0,890323	0,201	0,514286	0,714286	0,914286	0,098	28	100%	0,714286	5
5	0,503226	0,703226	0,903226	0,152	0,500000	0,700000	0,900000	0,100	28	100%	0,700000	7
6	0,535484	0,735484	0,935484	0,159	0,564286	0,764286	0,964286	0,059	28	100%	0,764286	1
7	0,490323	0,690323	0,890323	0,213	0,535714	0,735714	0,935714	0,092	27	96%	0,735714	3
8	0,425806	0,625806	0,825806	0,209	0,385714	0,585714	0,785714	0,093	23	82%	0,585714	28
9	0,387097	0,587097	0,787097	0,265	0,407143	0,607143	0,807143	0,096	24	86%	0,607143	26
10	0,361290	0,554839	0,754839	0,273	0,342857	0,535714	0,735714	0,133	23	82%	0,538095	31
11	0,412903	0,606452	0,806452	0,259	0,450000	0,650000	0,850000	0,129	24	86%	0,650000	17
12	0,316129	0,509677	0,709677	0,342	0,342857	0,542857	0,742857	0,145	20	71%	X	X
13	0,348387	0,535484	0,735484	0,312	0,371429	0,571429	0,771429	0,124	21	75%	0,571429	30
14	0,412903	0,606452	0,806452	0,282	0,442857	0,642857	0,842857	0,090	26	93%	0,642857	20
15	0,335484	0,529032	0,729032	0,282	0,307143	0,507143	0,707143	0,142	22	79%	0,507143	32
16	0,348387	0,535484	0,735484	0,306	0,421429	0,621429	0,821429	0,115	23	82%	0,621429	22
17	0,393548	0,593548	0,793548	0,192	0,414286	0,614286	0,814286	0,080	24	86%	0,614286	24
18	0,425806	0,625806	0,825806	0,238	0,485714	0,685714	0,885714	0,106	27	96%	0,685714	12
19	0,393548	0,593548	0,793548	0,216	0,471429	0,671429	0,871429	0,092	28	100%	0,671429	14
20	0,432258	0,632258	0,832258	0,259	0,442857	0,642857	0,842857	0,101	25	89%	0,642857	20
21	0,451613	0,651613	0,851613	0,223	0,500000	0,700000	0,900000	0,107	27	96%	0,700000	7
22	0,445161	0,645161	0,845161	0,217	0,492857	0,692857	0,892857	0,107	27	96%	0,692857	10
23	0,438710	0,638710	0,838710	0,235	0,450000	0,650000	0,850000	0,107	25	89%	0,650000	17
24	0,367742	0,561290	0,761290	0,244	0,371429	0,571429	0,771429	0,122	22	79%	0,571429	29
25	0,458065	0,658065	0,858065	0,205	0,464286	0,664286	0,864286	0,107	26	93%	0,664286	15
26	0,438710	0,632258	0,832258	0,218	0,514286	0,714286	0,914286	0,104	27	96%	0,714286	5
27	0,445161	0,645161	0,845161	0,205	0,500000	0,700000	0,900000	0,107	27	96%	0,700000	7
28	0,380645	0,574194	0,774194	0,216	0,414286	0,614286	0,814286	0,119	22	79%	0,614286	24
29	0,438710	0,638710	0,838710	0,24	0,450000	0,650000	0,850000	0,139	22	79%	0,650000	19
30	0,419355	0,619355	0,819355	0,204	0,478571	0,678571	0,878571	0,113	26	93%	0,678571	13
31	0,445161	0,645161	0,845161	0,199	0,492857	0,692857	0,892857	0,115	26	93%	0,692857	10
32	0,400000	0,593548	0,793548	0,203	0,457143	0,657143	0,857143	0,112	25	89%	0,657143	16
33	0,361290	0,541935	0,741935	0,309	0,421429	0,621429	0,821429	0,089	24	86%	0,621429	23

The values of m1, m2, and m3 were calculated in the previous step for each item. Thus, the values c1, c2, and c3 are translated for the Fuzzy scale values per item from the Likert scale. For this step, the values of d per item can be seen in Table 6.

The value of d per item and overall acceptance must be ≤ 0.2 in Table 6. Only five items respect this threshold: 1, 5, 6, 17, and 31.

Value d is the average of the values of d per item that is ≤0.2, representing a value of d=0,171 overall.

Since only five items have values lower than 0.2, the researchers opted to perform another round to reach more consensus.

Fuzzy Delphi third round

In the third and final round, the same second-round questionnaire was used to re-evaluate the same criteria to achieve better results than in the second round. Of the 34 participants, only 28 responded to the questionnaire, resulting in a drop rate of 17.65%, which is still lower than the threshold of 30% per round that should be upheld.

This round focused on re-evaluating the criteria in round 2, aiming to improve the results of the values for the variable d per item. As well as manage it to calculate the consensus for all criteria, create a new set of criteria ranked by the experts’ opinions, and discard any item that does not respect the thresholds set in the Fuzzy Delphi method.

The same equation to calculate the value of d per item was used in round 2, resulting in the average values of d per item presented in Table 6.

The average values of d in round 3 all respected the threshold ≤ 0.2; therefore, all of them were used to calculate the overall value of d, which is equal to 0,107.

In calculating the percentage per item, the number of times the value d per item is ≤ 0.2 will be divided by the number of participants in each round to get the percentage (Table 6).

An item needs to have a percentage ≥ 75% to be accepted. Otherwise, that item is discarded from the set of items. For example, according to Table 6, from the 33 item percentages calculated, only item 12 (Number of process steps) did not respect the threshold required. This way, this item was discarded from the criteria pool.

Equation (3.2) calculates the overall acceptance percentage. All the percentages of the items that respected the threshold of 75% are added and divided by the total number of items minus the discarded items.

O v e r a l l p e r c e n t a g e = \frac{s u m o f i t e m s p e r c e n t a g e}{t o t a l n u m b e r o f i t e m s - n u m b e r o f i t e m s d i s c a r d e d}

(3.2)

The minimum threshold value for the overall percentage is 90%, which was the resulting value from the calculation of the overall percentage from the 32 accepted items.

The process of Defuzzification will determine the position/scoring of each item, which results in calculating the average of the m1, m2, and m3 values. Then, the m1, m2, and m3 values will be used in equation (3.3) to calculate the Fuzzy evaluation:

f u z z y e v a l u t i o n = (\frac{1}{3}) * (m 1 a v e r a g e + m 2 a v e r a g e + m 3 a v e r a g e)

(3.3)

According to the values calculated from the equation of the Fuzzy evaluation per item, the higher the value, the better position the item will have in Table 6 - indicating that the item had a high level of consensus among the participants. Consequently, it is an important criterion to be included in the criteria to analyze possible business processes for automation.

The scoring can be equal for multiple items. For example, items 4 and 26 have an equal score of 5. The percentage per item was used to determine the scoring order demonstrated in Table 6. The item with a higher percentage would be in a higher position in Table 7. Also, the value α-cut for this calculation was 0,5, which means that any item below 0,5 in the Fuzzy evaluation column of Table 7 would also be discarded as it means the experts agree to reject the item from the set of criteria in the study. Based on this threshold value for α-cut, no items were discarded since all the items had Fuzzy evaluation values higher than 0,5.

Table 7.

List of criteria ranked based on experts’ opinion.

Item number	Item	Fuzzy Evaluation	Score
6	Feasibility	0,764286	1
1	Accurate process description	0,750000	2
7	Input and output data	0,735714	3
2	Application access	0,719048	4
4	Data security	0,714286	5
26	Rule-based	0,714286	5
5	Efficiency	0,700000	7
27	Savings	0,700000	7
21	Process stability	0,700000	7
22	Process standardization and stability	0,692857	10
31	Test data	0,692857	10
18	Process complexity	0,685714	12
30	Applications maturity	0,678571	13
19	Process cost	0,671429	14
25	Human error	0,664286	15
32	Time-consuming	0,657143	16
23	Repetitive	0,650000	17
11	Number of exceptions	0,650000	17
29	SLA impact	0,650000	19
14	Number of applications involved	0,642857	20
20	Data digitalization	0,642857	20
16	OCR involved	0,621429	22
33	Volume of items per transaction	0,621429	23
17	Predictability of outcomes	0,614286	24
28	Applications similarity	0,614286	24
9	Human involvement	0,607143	26
3	Automation type	0,585714	27
8	Human effort	0,585714	28
24	Reusability	0,571429	29
13	Number of robots allowed	0,571429	30
10	Cognitive requirements	0,538095	31
15	Number of users	0,507143	32

Based on the results of the Delphi method, it was possible to create a new set of criteria (Table 7) where the different criteria are ranked based on their scoring value.

Demonstration and evaluation

This section presents the results of the evaluations performed on business processes from different organizations with the criteria list achieved in Section 4.

Using the rank present in Table 8, it was possible to perform multiple tests to observe if the ranked list of criteria would satisfy real cases. The ranking of the criteria list was done by three primary thresholds, if the value of d per item was below 0,2, if the percentage per item was higher or equal to 75% and if the Fuzzy evaluation value was higher than 0,5. Therefore, six tests were conducted with experts to rank business cases based on the new list of criteria.

Table 8.

Factor values per item.

Position	Item	Definition	Factor
1	Feasibility	Whether an idea is doable	0,32
2	Accurate process description	If the process is well-detailed and described	0,31
3	Input and Output data	If there are structured and digital Inputs and outputs	0,3
4	Applications requirements	Amount and types of applications access	0,29
5	Data security	What concerns exist regarding data security	0,28
6	Rule-based process	If a process is based on rules	0,27
7	Efficiency	How efficient is a process	0,26
8	Savings	How much savings can the automation bring	0,25
9	Process stability	Did the process change in the last 12 months	0,24
10	Process standardize and stability	Is the process stable and is not going to change	0,23
11	Test data	If data exists to perform tests	0,22
12	Process complexity	How complex is a process to be automized	0,21
13	Applications maturity	Are the systems related to the process stable	0,2
14	Process cost	How much costs the process (manually)	0,19
15	Human error	Is the process prone to risks	0,18
16	Time-consuming	How much time a process consumes before automatization	0,17
17	Repetitive	Is the process repetitive	0,16
18	Number of exceptions	How many exceptions does the process have	0,15
19	SLA impact	Does the automation of the process satisfy the SLAs	0,14
20	Number of systems involved	How many systems are involved in the process	0,13
21	Data Digitalization	If there is data being digitally managed	0,12
22	OCR involved	Is there any OCR involved in the process	0,11
23	Volume of items per transaction	How many items per transaction	0,1
24	Predictability of outcomes	Are the outcomes of the process predictable	0,09
25	Applications similarity	Are the systems involved similar	0,08
26	Human involvement	How much manual involvement is required to complete a transaction	0,07
27	Automation type	What is the type of automation required	0,06
28	Human effort	Is the process labor-intense on the employee	0,05
29	Reusability	Can the process be reused once automated	0,04
30	Number of robots allowed	Number of robots that can run in the same instance	0,03
31	Cognitive requirements	Is there any cognitive ability required to complete a transaction	0,02
32	Number of users	Number of users using the automation	0,01

The organizations selected three business processes in each test that could be already automated, in development, or should be automated. In the initial test phase, the interviewee would give his opinion on which order they would automate the business processes based on their ranking system. Later the experts were asked to evaluate the business processes with values between 1 and 5 based on each criterion listed in Table 7. The experts did not know each criterion's factor values, presented in Table 8, so the evaluation can be unbiased.

As seen in Table 8, the items evaluated as the most important to classify business processes have higher Factor numbers. This gives these criteria higher importance in the Score per item equation (4.1), which calculates the weight of each criterion.

S c o r e p e r i t e m = e x p e r t v a l u e * f a c t o r

(4.1)

F a c t o r v a l u e = \frac{(33 - I t e m p o s i t i o n)}{100}

(4.2)

Equation (4.2) shows how the factor for each item in Table 8 was calculated. The interview values were between 1 and 5. The highest this value meant that the criterion for that business process was very relevant, which would increase the final score for that business process.

It was necessary to calculate the average of all the scores per item to calculate the score per business process. The score per business process was a value that only varied between 0 and 1. Closer to 1 would mean that the business process based on the criteria used was a good candidate for automation. Table 9 presents the final evaluation of the six tests performed.

Table 9.

Test results based on the criteria list.

Units of analysis	Order by which processes were implemented	Organizational processes assessed			Order advised by the artifact	Match with the organization's decision?
Units of analysis	Order by which processes were implemented	Process 1	Process 2	Process 3	Order advised by the artifact	Match with the organization's decision?
1	1->2->3	0,65687	0,64656	0,53437	1->2->3	✓
2	1->2->3	0,69656	0,73562	0,69687	2->3->1	×
3	3->2->1	0,64812	0,70343	0,71437	3->2->1	✓
4	3->2->1	0,69093	0,69906	0,75281	3->2->1	✓
5	2->1->3	0,65375	0,73031	0,64468	2->1->3	✓
6	2->3->1	0,65968	0,68406	0,67218	2->3->1	✓

As shown in Table 9, from the six tests performed, in 5 out of 6 tests, the result order based on the list of criteria matches the same order as the expert would choose to automate the business processes.

Only in the second test did the result between the expert order and the list of criteria not match, resulting in an inconclusive test. In this case, the expert would typically use a small and fixed list of criteria. Only those criteria would matter for their evaluation. From this test, it was even possible to retrieve some feedback from the expert.

This feedback included some key points such as knowing who will receive the output of the business process, the urgency of the automation, situations where the automation could potentially replace or extinct departments, and a more significant focus on calculating the savings.

Discussion and analysis

This investigation lies on the premise that organizations struggle to decide which processes they should automate since their context may vary. The results indicate 32 criteria relevant to both scientific and practitioners viewpoints. For easier understanding, the authors have grouped the criteria into main categories and added a brief description of each criterion in Table 10.

Table 10.

Categories and criteria description.

Category	Criterion	Description
Data	Input and output data	The data structure used as inputs and outputs should be standardized and semi-structured
	Data security	Data access and manipulation already comply with security best practices.
	Data digitalization	If the data is being digitally managed
	Test data	If there is any data to be used to test before deployment
Environment	Number of applications involved	Number of applications used by the process
	Applications similarity	If the involved applications have a similar structure or differ a lot. For instance, similar APIs or programming languages
	Applications stability	Involved applications are stable, and fewer updates are expected throughout the time
	Applications requirements	The complexity of requirements to connect with applications
Human resources	Human effort	How much human effort is needed to complete a transaction before automatization
	Human involvement	How much human involvement is needed to complete a transaction after automatization
	Number of users	How many users are involved in the process may represent significant savings in resources relocation
	Cognitive requirements	How many human interventions/decisions are required to complete a transaction in a process
	Human error	The amount of human error that a business process has
Governance	Process cost	How much is the cost of the process
	Savings	Any benefit that can come with the automatization
	SLA impact	If the automation will increase the SLA compliance rate
	Time-consuming	How much time is required to finish a transaction
	Accurate process description	If all the process documentation exists and is clear
	Number of robots allowed	Each enterprises contract with the RPA software provided may limit the number of possible robots in parallel
	Reusability	If it is possible to reuse part of other RPA developments, or if the current development may be easily reused
	Efficiency	If it is expected to increase the productivity and quality of the process
	Feasibility	How complex is the automatization given the overall process complexity
Structure	Number of process steps	If the process has a considerable amount of steps or not
	Process complexity	How complex the process is considering its entire context like human involvement, applications involved, and process steps, among others.
	Number of exceptions	Number of possible exceptions that a process can have
	Process stability	If the process did not suffer any significant change in the past 12–18 months, indicate that it is stable and fewer updates are expected throughout the time
	Process standardized	The process should already be standardized; otherwise, the development will take a lot longer, and the robot will face a bigger number of exceptions that were not mentioned while in the development
	Repetitive	Usually already standardized, but the process always follows the same ruled-based workflow
	Number of transactions	Processes that originate high amounts of transactions (number of times the bot does a task) are good candidates for automation
	Predictability of outcomes	Despite the some known output being structured, others may be unpredictable
	Rule-based process	The process is dominated by business rules which are already contemplated in the process
	Automation type	Attended processes run in user machines and are more dependent on human intervention. Unattended processes run in virtual machines and work 24/7

Having the final list of criteria, the authors have then developed a framework to assist scientists and practitioners in the future. This must be seen as a pioneer artefact that must be further evolved and updated in future investigations. The framework can be seen in Figure 7. Please note that the n^o of robots allowed does not have classification since it depends on the process intended to automate. For instance, a process with several instances running in parallel may require more robots; otherwise, a few robots are enough.

Figure 7.

Framework for Process Selection for Automation.

The data suggest that the decision process is not simple, so different questions must be answered before moving on with automation. Deciding on the correct process may be the difference between success and failure. The elicited criteria touch several process domains. Some of the most known and used are, for example, labor intensity, the volume of transactions, process cost, or a repetitive process. However, some less explored criteria were also revealed. For instance: data security or human error, process complexity, systems maturity or even if the possible input data is identified and output data is predictable.

The results indicate that the problem might be more complex to solve than one may have thought. With requirements from strategic to operational level and involving all the process environment increases the decision-making complexity. The framework proposed in the study was successfully demonstrated in practice and is the first step in a domain that deserves further investigation.

Conclusion

This study aimed to elicit criteria to assist decision-makers in evaluating which business processes are suitable for automation. The objective was successfully achieved. This research has two main contributions:

• Synthesize the knowledge about process automation criteria that already exist in the literature. This contribution was achieved by performing an SLR, resulting in a list of criteria that served as a basis for the Delphi study.

• Create a list of process automation criteria tuned by RPA experts. This contribution was achieved by performing a Delphi study, resulting in a set of tuned criteria. The proposed criteria were then used to assess a set of processes from real organizations to understand if the proposal is aligned with workers’ decisions.

While professionals may find valuable information in this document to help them decide which processes must be automated first, academics are now aware of which areas deserve further investigation. Each criterion points to a different area, most of which is in an early stage of development. For instance, how to increase process standardization, turn data testable, or even human error, to name a few.

This investigation also has some limitations that are worth to be mentioned. First, an SLR was performed, but some relevant studies may have been left out despite a rigorous methodology. Second, since RPA is a relatively recent technology, it cannot yet be considered a mature topic in literature. Therefore the authors have worked with what was available so far. Finally, it was challenging to identify people with a high level of expertise on the subject matter.

Considering the limitations, some future work paths are advised. First, the authors are already evolving this investigation by developing a multi-criteria decision model. Second, with RPA being increasingly adopted by organizations, it is expected to raise some risks that may impact business continuity. Third, the possible integration of RPA with other technologies is yet to be explored. So far, the most referred to in literature are AI and ML to enhance the level of cognitive abilities not currently present in RPA. The inclusion of AI and ML in process automation is referred to in the literature as intelligent process automation. Finally, it should be analyzed how process mining can accelerate the implementation of RPA and guide RPA initiatives, helping in that way to define the most eligible processes for automation. There are already a few studies in this area, such as the one carried out by Geyer-Klingeberg et al. (2018), but it is still a topic that requires further research.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Rafael Almeida

Author biographies

Diogo Farinha is a RPA Developer at Arpa Elastic Solutions. He has a Master’s degree in Telecommunications Engineering from ISCTE. He also graduated in Telecommunications Engineering from ISCTE. He has been a consultant in several industries, such as Telecommunications and Contact Centers, acting as an Optimization engineer and in sectors such as Telecommunications, Banking, Advocacy, supply chains and the Automotive Industry, acting as an RPA developer focused on the development, maintenance and analysis of new projects.

Ruben Pereira is an Assistant Professor at ISCTE. He has a PhD in Computer Science and Information Systems from Instituto Superior Técnico, where he also graduated with a Master's in Computer Engineering and Computer Science. He has been a consultant in several industries, such as services, banking, telecommunications, Healthcare, and Ecommerce, among others. Author of dozens of scientific papers, his areas of scientific interest are IT Risk, IT Strategy, IT Value, IT Governance, Business Process Management, Innovation, Digital Transformation, DevOps, and Robotic Process Automation.

Rafael Almeida is a seasoned professional with a PhD. in Computer Science and Engineering from Instituto Superior Técnico. Their research in IT Governance, Enterprise Architecture, and Risk Management has resulted in numerous scientific papers widely cited in academic circles. In recent years, Rafael has worked as a Product Manager and Product Owner for startups and corporate organizations, bringing their technical expertise to drive innovation and growth. Overall, Rafael Almeida has a proven track record in both academia and industry.

References

Ansari

Diya

Patil

. A Review on Robotic Process Automation - The Future of Business Organizations. SSRN Electronic Journal. 2019. doi:10.2139/ssrn.3372171.

Axmann

Harmoko

Herm

Janiesch

. A framework of cost drivers for robotic process automation projects. Lecture Notes in Business Information Processing. 2021; 428: 7–22. doi:10.1007/978-3-030-85867-4_2.

Chacón-Montero

Ramirez

Enríquez

, et al. (2019) ‘Towards a method for automated testing in robotic process automation projects’. IEEE. pp. 42–47, doi: 10.1109/AST.2019.00012.

Choi

R’bigui

Cho

. Candidate digital tasks selection methodology for automation with robotic process automation. Sustainability (Switzerland). 2021; 13(16): 8980. doi: 10.3390/su13168980.

Davenport

. From analytics to artificial intelligence. Journal of Business Analytics. 2018; 1(2): 73–80.

Davenport

Ronanki

. Artificial intelligence for the real world. Harvard business review. 2018; 96(1): 108–116.

Enriquez

Jimenez-Ramirez

Dominguez-Mayo

Garcia-Garcia

. Robotic process automation: a scientific and industrial systematic mapping study. IEEE Access. 2020; 8: 39113–39129. doi: 10.1109/ACCESS.2020.2974934.

Fominykh

Rezchikov

Kushnikov

, et al. Problem of quality assurance during metal constructions welding via robotic technological complexes. Journal of Physics: Conference Series. 2018; 1015(3): 32169. doi: 10.1088/1742-6596/1015/3/032169.

Geyer-Klingeberg

Nakladal

Balduf

, et al. Process Mining and Robotic Process Automation: A Perfect Match. BPM (Dissertation/Demos/Industry); 2018: 124–131.

10.

Ghose

The Future of Robotic Process Automation (RPA)', Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12632. LNCS; 2021. 273–280. doi:10.1007/978-3-030-76352-7_28.

11.

Griffiths

Pretorius

. Implementing robotic process automation for auditing and fraud control. Communications in Computer and Information Science. 2021; 1477: 26–36. doi:10.1007/978-3-030-86761-4_3.

12.

Hallikainen

Bekkhus

Pan

. How opus capita used internal RPA capabilities to offer services to clients. MIS Quarterly Executive. 2018; 17: 1.

13.

Herm

Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12168. LNCS; 2020. 471–488. doi: 10.1007/978-3-030-58666-9_27.

14.

Hofmann

Samp

Urbach

. Robotic process automation. Electronic Markets. 2020; 30(1): 99–106. doi:10.1007/s12525-019-00365-8.

15.

Huang

Vasarhelyi

. Applying robotic process automation (RPA) in auditing: a framework. International Journal of Accounting Information Systems. 2019;35:100433. doi: 10.1016/j.accinf.2019.100433.

16.

Ivančić

Susa Vugec

Bosilj Vuksic

. Robotic process automation: systematic literature review. Business Process Management: Blockchain and Central and Eastern Europe Forum. 2019; 361: 280–295. doi: 10.1007/978-3-030-30429-4_19.

17.

Itpa

. Criteria, use cases and effects of information technology process automation (ITPA). Adv Robot Autom. 2013; 03: 03. doi:10.4172/2168-9695.1000124.

18.

Kent

Saffer

. A Delphi study of the future of new technology research in public relations. Public Relations Review. 2014; 40(3): 568–576. doi:10.1016/j.pubrev.2014.02.008.

19.

Kitchenham

(2004) ‘Procedures for Performing Systematic Reviews.’ Keele, UK, Keele University. 2004; 33: 1–26.

20.

Kuo

. A decision support system for the stock market through integration of Fuzzy neural networks and Fuzzy Delphi. Applied Artificial Intelligence. 1998;12(6):501–520. doi:10.1080/088395198117640.

21.

Lacity

. Robotic process automation at Telefonica O2. London: The London School of Economics and Political Science; 2015.

22.

Lacity

Willcocks

. A new approach to automating service. MIT Sloan Management Review. 2016; 58(1): 41–49.

23.

Lacity

Willcocks

(2021) ‘Becoming strategic with intelligent automation’, MIS Quarterly Executive 20 (2021): 169-182.

24.

Lacity

Willcocks

Gozman

. Influencing information systems practice: The action principles approach applied to robotic process and cognitive automation. Journal of Information Technology. 2021; 36(3): 216–240.

25.

Lau

HYK

. An Intelligent Robotic Framework for Automated Assembly; 2000.

26.

Leno

. Identifying Candidate routines for Robotic Process Automation From Unsegmented UI Logs. 2020. http://arxiv.org/abs/2008.05782 http://arxiv.org/abs/2008.05782.

27.

Leopold

van der Aa

Reijers

. Identifying candidate tasks for robotic process automation in textual process descriptions. Enterprise, Business-Process and Information Systems Modeling. 2018; 318: 67–81. doi: 10.1007/978-3-319-91704-7_5.

28.

Liu

Zhang

Dutta

Goh

. Digital twinning for productivity improvement opportunities with robotic process automation: case of greenfield hospital. International Journal of Mechanical Engineering and Robotics Research. 2020; 9(2): 258–263. doi: 10.18178/ijmerr.9.2.258-263.

29.

Madakam

Holmukhe

Kumar Jaiswal

. The future digital work force: robotic process automation (RPA). Journal of Information Systems and Technology Management. 2019; 16: 1–17. doi:10.4301/s1807-1775(2019)16001.

30.

Marciniak

Stanisławski

. Internal determinants in the field of RPA technology implementation on the example of selected companies in the context of industry 4.0 assumptions. Information (Switzerland). 2021; 12(6): 222. doi: 10.3390/info12060222.

31.

Moffitt

, Robotic process automation for auditing. Journal of Emerging Technologies in Accounting 2018; 15(1): 1–10. doi: 10.2308/jeta-10589.

32.

Nawaz

. Robotic process automation for recruitment process. INTERNATIONAL JOURNAL OF ADVANCED RESEARCH IN ENGINEERING & TECHNOLOGY. 2019;10(2):608-611. doi:10.34218/IJARET.10.2.2019.057.

33.

Nurul

Wan

Kadir

, et al. The application of the fuzzy Delphi technique on a component of development of form four STEM-based physics interactive laboratory (I-Lab). International Journal of Scientific and Technology Research. 2019; 8: 12. www.ijstr.org

34.

Okoli

Pawlowski

. The Delphi method as a research tool: an example, design considerations, and applications. Information and Management. 2004; 42(1): 15–29. doi: 10.1016/j.im.2003.11.002.

35.

Okoli

A guide to conducting a standalone systematic literature review. Communications of the Association for Information Systems. 2015; 37(1): 879–910. doi: 10.17705/1cais.03743.

36.

Osman

(2019) ‘Robotic process automation: lessons learned from case studies’, Informatica Economica, 23, 66–71. doi: 10.12948/issn14531305/23.4.2019.06.

37.

Pranav

. Dementia detection for elderly people using. Robotic Process Automation. 2020; 7(1): 502–506.

38.

Ratia

Myllarniemi

Helander

. Robotic process automation - creating value by digitalizing work in the private healthcare? Proceedings of the 22nd International Academic Mindtrek Conference. 2018; 18: 222–227. doi:10.1145/3275116.3275129.

39.

Rejab

Firdaus

Chuprat

. Fuzzy Delphi method for evaluating HyTEE model (hybrid software change management tool with test effort estimation). International Journal of Advanced Computer Science and Applications. 2019; 10(4): 529–535. doi:10.14569/ijacsa.2019.0100465.

40.

Rhouati

Ettifouri

Dahhane

, et al. Impact of robotic process automation in supply chain: a model for task selection. 2021 the 3rd International Conference on Robotics Systems and Automation Engineering (RSAE). 2021; 3: 17–20. doi: 10.1145/3475851.3475865.

41.

Riedl

Beetz

(2019) ‘Robotic process automation: developing a multi-criteria evaluation model for the selection of automatable business processes’, 25th Am. Conf. Inf. Syst. AMCIS. 2019, pp. 1–10.

42.

Rigby

Schofield

Mann

, et al. Education research: an exploration of case-based learning in neuroscience grand rounds using the Delphi technique. Neurology. 2012; 79(3): 19–26. doi: 10.1212/WNL.0b013e31825fdfa2.

43.

Rose

Arnold

Howse

. Unmanned aircraft systems selection practices: current research and future directions. Military Psychology. 2013; 25(5): 413–427. doi: 10.1037/mil0000008.

44.

Santos

Pereira

Vasconcelos

. Toward robotic process automation implementation: an end-to-end perspective. Business Process Management Journal. 2019; 26(2): 405–420. doi:10.1108/BPMJ-12-2018-0380.

45.

Siderska

. Robotic process automation-a driver of digital transformation? Engineering Management in Production and Services. 2020; 12(2): 21–31. doi: 10.2478/emj-(2020)-0009.

46.

Syed

Suriadi

Adams

, et al. Robotic process automation: contemporary themes and challenges. Computers in Industry. 2020; 115: 103162. doi:10.1016/j.compind.2019.103162.

47.

Viehhauser

Doerr

. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), In: Digging for Gold in RPA Projects – A Quantifiable Method to Identify and Prioritize Suitable RPA Process Candidates. LNCS; 2021: 313–327. doi: 10.1007/978-3-030-79382-1_19.

48.

Völker

Siegert

Weske

. Adding decision management to robotic process automation. Lecture Notes in Business Information Processing. 2021; 428: 23–37. doi: 10.1007/978-3-030-85867-4_3.

49.

Wellmann

(2020). ‘A framework to evaluate the viability of robotic process automation for business process activities’, pp. 1–15, doi: 10.1007/978-3-030-58779-6_14.

50.

Wewerka

Reichert

. Robotic Process Automation -- A Systematic Literature Review and Assessment Framework; 2020. http://arxiv.org/abs/2012.11951

51.

Wewerka

Reichert

. Robotic process automation in the automotive industry - lessons learned from an exploratory case study. Research Challenges in Information Science. 2021; 415: 3–19. doi: 10.1007/978-3-030-75018-3_1.

52.

Willcocks

Lacity

Craig

. Robotic process automation: strategic transformation lever for global business services? Journal of Information Technology Teaching Cases. 2017; 7: 17–28.

53.

Zhang

Liu

. The Key Factors Affecting RPA-Business Alignment. ACM International Conference Proceeding Series; 2018. doi: 10.1145/3265689.3265699.