Ranking of asset maintenance process KPIs using Fermatean fuzzy Delphi and Fermatean fuzzy ELECTRE method

Abstract

The paper aims to present a hybrid model for measuring the performance of business processes in complex organizations based on the subjective decision-making of expert teams. The subject of the research is finding ways to measure, analyze and improve the key performance indicators (KPIs) process. Obtaining the values of KPIs, which reflect the real state of the process, creates a basis for their ranking, i.e. insight into KPIs that are extremely important for the process as well as KPIs that are of lesser importance, but as such are not excluded from consideration because they are necessary for the beginning, realization and completion of the process. The model was compiled through five phases and was tested through a case study in a real business organization, which deals with the maintenance of complex combat systems. The obtained results helped the management to take certain measures in order to improve the performance of the maintenance process. In the model, it is proposed to form two expert teams, which make assessments based on experience and express them in linguistic terms according to a predefined scale. Modeling of linguistic expressions is realized using intuitive fuzzy sets of a higher order, more precisely Fermatean fuzzy sets (FFS). Selecting KPIs, decomposing the process into sub-processes and assessing the relative importance of sub-processes is carried out by one team of experts, while another team carries out the assessment of KPIs at the level of each sub-process. Determining the relative importance of sub-processes is realized using the Delphi method extended to FFS while reaching a consensus. The measurement of process performance, i.e. the value of KPIs, is realized using Multi-Criteria Group Decision-Making (MCGDM), such as the ELECTRE method extended with FFS. The sensitivity analysis of the developed model is realized by uncertainty modeling with q-rung orthopair fuzzy sets.

Keywords

Fermatean fuzzy set (FFSs)ELECTRE method Delphi method maintenance performance

1 Introduction

The goal of every organization is to operate with the greatest possible profit with a rational use of resources. In order to determine whether the organization operates effectively and efficiently, there must be a certain indicator, measure, price or criterion, based on which the success of the system’s functioning can be assessed [13, 32]. Therefore, certain approaches are applied in practice that enable the organization’s management to achieve the set goal through successful management. One of the approaches is the application of a model to manage the key performance indicators (KPIs) of the process/system. In order to facilitate this approach, certain models have been developed, and the most commonly used in the literature are: Balanced Scorecard model (BSC) developed by Kaplan & Norton in 1993 [21], Performance Pyramid developed by Judson in 1990 [20], European Foundation for Quality Management (EFQM) [14] and others.

However, the results of the existing models indicate the existence of shortcomings such as: the relative importance of KPIs are equal or predetermined, the values of KPIs are described with precise numbers, with the BSC model the values are expressed in percentages, with the EFQM model there is a predefined value scale. In order to overcome the weaknesses of existing models, to effectively measure the performance of the process itself, it is necessary to develop an adequate Performance Measurement System (PMS), which will support strategies and plans for process improvement, as well as the identification of the KPIs themselves.

KPIs represent the set of measures that are critical to the current and future success of an organization, and the basis for measuring the performance of an organization or process. Pritchard [36] is presented the key performance indicators as qualitative or quantitative indicators that show the success of achieving the given goal. Hronec [18] shows performance indicators as the success of activities within a process, that is, as a result of the realization (output) of the process in relation to the planned outcome. According to Harringtonton [17], indicators are quantitative or qualitative indicators by which, directly or indirectly, the level or degree of achievement of a certain goal could be assessed or measured, as well as the speed or time or deadline of goal achievement. According to Parmenter [34], KPIs represent the quantitative measure, which is designed/redesigned in advance and reflects the critical success factors of an organization. Therefore, KPIs allows the organization’s management to measure progress in achieving set goals and meeting the needs of stakeholders.

According to Rosemann [39], KPIs represents the valuable source of information on how strategic goals are translated into process goals and encourage effective process control. The goal is to determine as few key indicators as possible, in order to maintain focus only on the most important activities n the process/system. Achieving the set goals requires constant monitoring of the process, early detection and preventive action to correct the activities in the process that are dominant in the system. The complexity of the system in terms of organization, technology, variety of resources (personnel, equipment, infrastructure, technological time, etc.) creates a potential space for unplanned losses that are sometimes invisible to system management and directly affect the achievement of set goals.

In order to measure the performance of the process in the system, it is necessary to decompose the process into sub-processes, identify KPIs and establish a PMS, which will most closely display the results in the form of output values of KPIs. Next, it is necessary to rank the KPIs, in order to determine which KPIs have the greatest impact on the technological process, and which KPIs have the least impact. Therefore, there was a need to develop hybrid models for measuring the performance of processes in complex systems, which will enable management to achieve target values that are of strategic importance for the functioning of the system in a faster and simpler way.

In order to make the system performance measurement process more transparent and objective, especially in cases where subjectivity cannot be completely excluded, the organization’s management should include an independent group of decision-makers (experts) with different skills, experience and knowledge capable of evaluating all sub-processes and key performance indicators, which will contribute to greater objectivity in deciding on the final measures for business improvement.

In general, the problem of process performance measurement gives a lot of space for research and constant search for answers to numerous questions, such as: How to decompose complex processes? What indicators are suitable for measuring performance? How to objectively determine the value of the indicator? Which information formats are suitable for use by decision-makers during the subjective assessment and evaluation of KPIs and sub-processes? How to effectively determine the competence of decision makers? How to effectively achieve consensus or agreement among decision makers? How to build an effective decision-making approach when determining the final order of KPIs and sub-processes? What are the most suitable methods and techniques of multi-criteria decision making?

The aim of this paper is to expand the limited literature on process performance measurement in complex systems and to point out the need to improve existing models. In this sense, the paper proposed a hybrid model for ranking of process KPIs using Fermatean fuzzy Delphi and Fermatean fuzzy ELECTRE method, which will enable management to achieve target values that are of strategic importance for system functioning in a faster and simpler way. The justification for applying the hybrid model is obtaining more objective and measurable values of KPIs and sub-processes during subjective decision-making, as well as minimizing the risk of careless, incompetent and irresponsible decision-making. In addition, the application of the proposed model is justified because it points to invisible problems that occur in the process, on the basis of which the management system takes measures to improve the process and business of the organization.

The proposed model was tested through a case study, on the example of a military overhaul system whose basic process is the maintenance of complex combat equipment.

Besides the introduction, where the problem and aim of the work are given, the paper is organized as follows. In Section 2, the theoretical framework for the development of the model is given, in Section 3 the model for process performance measurement is shown, in Section 4 the application of the performance measurement model is realized on the example of a complex overhaul system, in Section 5 the analysis of the results is presented in Section 5 the analysis of the results is presented as well as model testing when uncertainty modeling is realized by other fuzzy sets. In this research, q-rung orhopair fuzzy sets are used, which is from the set of intuitive fuzzy sets as well as FFS. Based on the obtained results, it is shown whether the model is sensitive to changes, as well as whether the results obtained using FFS are more suitable for decision makers or not. In Section 6 the conclusion is given, and in the last Section 7, a detailed review of the literature that was used for research and development of the model is presented.

In order to make it easier to follow the development of the model, the following abbreviations are used in the paper: Hierarchy –Input –Output - Processing (HIPO), key performance indicators (KPIs), performance measurement system (PMS), Fermatean fuzzy sets (FFS), Multi-criteria group decision-making (MAGDM), Elimination and Choice translating reality (ELECTRE), Fermatean fuzzy weighted power average (FFWPA), Fermatean fuzzy weighted power geometric (FFWGA), sub-processes (PP), experts (E).

2 Background

In this part, the theoretical framework that was applied for the development of the proposed model in Section 3 is presented.

2.1 Process decomposition approach

The complexity of organizations requires a good knowledge of the process, as it involves various inputs, outputs and stakeholders. In order to measure the performance of the process, it is necessary to realize the decomposition of the process into sub-processes. One of the models for process decomposition is Hierarchy - Input - Output - Processing (HIPO) [11]. The HIPO method was chosen because it corresponds to hierarchically structured processes, follows the flow of information through the process, identifies the procedural flow from the input to the output of the process, and monitors the relationships between sub-processes and within the sub-processes themselves.

2.2 Approaches to evaluating KPIs

Evaluation of KPIs can be objective and subjective. The objective approach gives values that are measured and expressed in some measurement unit, while the subjective approach of evaluation gives values expressed in the form of linguistic expressions, based on the experience and intuition of the evaluator. In the development of the proposed model, a subjective approach for evaluating KPIs was used. Modeling values that are uncertain by nature require the application of a fuzzy set. The Fermatean fuzzy set was chosen for this model, for the reason that it has justified its application in previous research, as shown in the papers analyzed in Section 2.3.

The complexity of the organization requires the engagement of experts, who will contribute to verifying the obtained results of the model practically on the example of a complex organization. The formation of a group of experts who will evaluate KPIs belongs to the domain of group decision-making, which causes the use of models with the achievement of agreement on the obtained values of KPIs. The Delphi method was used, which, with its concept, can meet the stated requirements. The suitability of using the Delphi method in current practice has been confirmed in the papers analyzed in Section 2.5.

To obtain values for each KPI, ranking and determining priority KPIs for the process, it is most appropriate to use multi-criteria decision-making methods. The ELECTRE method extended with FFS is proposed in the model. Previous research and application of the ELECTRE method are analyzed in detail in Section 2.4.

2.3 Analysis of Fermatean fuzzy set

Higher-order fuzzy sets arose from the need to, in addition to the existing fuzzy sets, model uncertainties more qualitatively, taking into account the certainty and uncertainty of the decision-maker when evaluating or assessing the situation. The basis of FFS is intuitive fuzzy sets developed by Atanassov [7], from the need to include certainty or uncertainty in the decision made when making a decision in uncertainty modeling. The upgrade of intuitive fuzzy sets consists of Pythagorean fuzzy sets [54] as fuzzy sets of the second order, and then Fermatean fuzzy sets as fuzzy sets of the third order, all with the aim of quantifying uncertainty modeling as precisely as possible. In the literature review, FFS has recently played a major role in scientific research, where uncertainty modeling is used in various fields, such as: analysis of the advantages of the MABAC method that uses the r-rung orthopair fuzzy environment [50], research on the evolution of pattern recognition techniques [15], a statistical concept for medical decision-making applications [25], application of the Hernian mean operator on the example of electronic surveillance [40], analysis of hospital preparedness in the event of a disaster [24], medical waste disposal planning for health centers [1], pattern recognition based on similarity measures [9], extended capital budgeting technique [12], selection of antivirus mask [46], assessment of challenges for Industry 4.0 adoption for sustainable digital transformation [42], occupational risk assessment of flight school [16], warehouse location selection for the automotive industry [41], evaluation of green suppliers in a complex uncertain environment [56], multi-criteria decision making dealing with the lack of preference order of similarity to the ideal solution [5], using Fermatte type-2 phase sets in group decision making [2], analysis of trading performance of the European Union and Serbia [27], supplier selection [35]. From the previously analyzed literature, there is the author’s conviction that uncertainty modeling using FFS in practice becomes justified.

Based on [43 –45] in the world scientific literature, Senapati & Yagwer introduced FFS in 2019, where they defined the following algebra:

Definition 1. Let X be a fixed set, a FFS in X is defined as, F ={ 〈 x _j, μ _F (x _j) , ν _F (x _j) 〉 | x _j ∈ X } where μ _F (x _j) (0 ⩽ μ _F (x _j) ⩽ 1) represent the membership and ν _F (x _j) (0 ⩽ ν _F (x _j) ⩽ 1) nonmembersship degrees of x _j ∈ X to the set F, respectively and they satisfy the following condition: 0 ⩽ (μ _F (x _j)) ³ + (ν _F (x _j)) ³ ⩽ 1. For all x _j ∈ X, if $π_{F} (x_{j}) = \sqrt[3]{1 - {(μ_{F} (x_{j}))}^{3} - {(ν_{F} (x_{j}))}^{3}}$ , then π _F (x _j) is the indeterminacy degree of x _j ∈ X to the set F.

Definition 5. Let F _i = (μ _i, ν _i) i = 1, 2, . . , n be a number of FFNs and ω = (ω ₁, ω ₂, . . . , ω _n) ^T be weight vector of F _i with $\sum_{i = 1}^{n} ω_{i} = 1$ , then a Fermatean fuzzy weighted power average (FFWPA) operator is a function , where

$\begin{matrix} FFWPA (F_{1}, F_{2}, . . ., F_{n}) = \\ ({(\sum_{i = 1}^{n} ω_{i} μ_{i}^{3})}^{\frac{1}{3}}, {(\sum_{i = 1}^{n} ω_{i} υ_{i}^{3})}^{\frac{1}{3}}) \end{matrix}$ (1)

Definition 6. Let F _i = (μ _i, ν _i) i = 1, 2, . . , n be a number of FFNs and ω = (ω ₁, ω ₂, . . . , ω _n) ^T be weight vector of F _i with $\sum_{i = 1}^{n} ω_{i} = 1$ then a Fermatean fuzzy weighted power geometric (FFWGA) operator is a function , where $\begin{matrix} FFWPG (F_{1}, F_{2}, . . ., F_{n}) = \\ ({(1 - \prod_{i = 1}^{n} {(1 - μ_{i}^{3})}^{ω_{i}})}^{\frac{1}{3}}, {(1 - \prod_{i = 1}^{n} {(1 - υ_{i}^{3})}^{ω_{i}})}^{\frac{1}{3}}) \end{matrix}$ (2)

Definition 7. Let F ₁ = (μ ₁, ν ₁) and F ₂ = (μ ₂, ν ₂) two FFS. Then the Euclidean distance between F ₁ and F ₂ is defined as: $\begin{matrix} d_{e} (F_{1}, F_{2}) = \\ \sqrt{\frac{1}{2} ({[μ_{1}^{3} - μ_{2}^{3}]}^{2} + {[υ_{1}^{3} - υ_{2}^{3}]}^{2} + {[π_{1}^{3} - π_{2}^{3}]}^{2})} \end{matrix}$ (3)

Definition 8. Calculating the standard the deviation based on the expression [42]: $σ^{2} = \frac{1}{m} \sum_{i = 1, . . ., e} {(ω_{j} - ω_{j}^{e})}^{2},$ (4) where: $ω_{j} - ω_{j}^{e} = d_{e}$ , and ω _j is represents the aggregated value of subprocess relative importance (PPi) obtained by FFWPG, while $ω_{j}^{e}$ represents the relative importance (PPi) assigned by the ith expert. The variance σ is a measure of the consensus for the obtained relative importances.

2.4 Analysis of fuzzy ELECTRE algorithm extended with Fermatean fuzzy number

There are a number of methods and approaches for alternative ranking. In this, the multi-criteria decision-making method ELECTRE (Elimination and Choice translating reality) developed by Bernard Roy in 1986 with his collaborators was used. The ELECTRE method enables the decision-maker to choose an alternative with maximum advantage and minimum conflicts in function of various criteria, that is, it provides a clear picture of the value of each alternative. The method is based on pairwise comparisons of alternatives, which means that each alternative is compared with all other alternatives. The main advantage of this method is that it does not allow compensation between criteria and any normalization procedure, because normalization distorts the original data. The ELECTRE method is often used in research, especially in complex areas. The ELECTRE method has found application in the case of consumer decision-making [53], choice of alternative filling in bed production [49], evaluation of performance and team selection [23], performance analysis of food suppliers [28], selection of the most optimal supplier of drones [30], management of a project to remove construction obstacles for schools where there are children with developmental disabilities [38], selection of means of transport for the needs of the logistics system [48], comparative selection of a bicycle path for sustainable tourism in Franciacorta [8], application of the AHP-ELECTRE decision-making method for solving health and safety problems [37], assigning a machine maintenance strategy [26], improved ELECTRE II method with linguistic m-polar fuzzy sets [4], risk assessment with hesitation [3], evaluation the contractor selection [47], etc.

According to Wu [52] and Zhou [57], the fuzzy Electra extended FFS method algorithm is presented in seven steps.

Step 1. Formation of the decision-making matrix X _ij, where the decision-maker evaluates alternatives, i.e. KPIs according to all criteria, i.e. sub-processes PP _i. Grades are expressed using Fermatean phase numbers X _ij = (μ _ij, v _ij) $\begin{matrix} X_{i j} = \begin{matrix} K P I_{1} \\ K P I_{2} \\ ⋮ \\ K P I_{i} \end{matrix} [\begin{matrix} P P_{1} & P P_{2} & \dots & P P_{j} \\ (μ_{11}, v_{11}) & (μ_{12}, v_{12}) & \dots & (μ_{1 j}, v_{1 j}) \\ (μ_{21}, v_{21}) & (μ_{22}, v_{22}) & \dots & (μ_{2 j}, v_{2 j}) \\ (μ_{31}, v_{31}) & (μ_{32}, v_{32}) & \dots & (μ_{3 j}, v_{3 j}) \\ (μ_{i 1}, v_{m 1}) & (μ_{i 2}, v_{m 2}) & \dots & (μ_{i j}, v_{i j}) \end{matrix}] \end{matrix}$ (5)

Step 2. Different methods can be used to determine the weight or importance of criteria ω _j, usually the decision maker decides which method will be used to determine the importance of each criterion, while the condition must be met that $\sum_{i = 1}^{n} ω_{i} = 1$ , where 0 ⩽ ω _j ⩽ 1. In this paper, the Defley method extended with Fematean fuzzy set was used for the first time.

Step 3. Construct the Concordance and Discordance sets:

a) strong concordance set is portrayed as C _kl, medium concordance set is portrayed as , weak concordance set is portrayed as :

$\begin{array}{l} C_{k l} = {j | μ_{k j} \geq μ_{i j}, v_{k j} < v_{i j} i π_{k j} < π_{i j}} \\ C_{k l}^{'} = {j | μ_{k j} \geq μ_{i j}, v_{k j} < v_{i j} i π_{k j} \geq π_{i j}} \\ C_{k l}^{''} = {j | μ_{k j} \geq μ_{i j}, v_{k j} \geq v_{i j}} \end{array}$ (6)

b) strong discordance set is portrayed as D _kl, strong discordance set is portrayed as , strong discordance set is portrayed as :

$\begin{array}{l} D_{k l} = {j | μ_{k j} < μ_{i j}, v_{k j} \geq v_{i j}, π_{k j} \geq π_{i j}} \\ D_{k l}^{'} = {j | μ_{k j} < μ_{i j}, v_{k j} \geq v_{i j}, π_{k j} < π_{i j}} \\ D_{k l}^{''} = {j | μ_{k j} < μ_{i j}, v_{k j} < v_{i j}} \end{array}$ (7)

Step 4: Identify the weights of concordance and discordance sets: $W^{'} = {ω_{c}, ω_{c}^{'}, ω_{c}^{''}, ω_{D}^{}, ω_{D}^{'}, ω_{D}^{''}}$ .

a) the weights of strong concordance set ω _c, the weights of medium concordance set ω _c′, the weights of weak concordance set ω _{c
^′′} is computed with as follows: $\begin{array}{l} ω_{c} = \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times {\sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C^{'}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) + \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C^{''}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j})}^{- 1} \\ ω_{c'} = \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C'_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times {\sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C^{'}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) + \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C^{''}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j})}^{- 1} \\ ω_{c''} = \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C''_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times {\sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C^{'}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) + \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in C^{''}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j})}^{- 1} \end{array}$ (8)

b) the weights of strong discordance set ω _D, the weights of medium discordance set ω _D′, the weights of weak discordance set ω _{D
^′′} is computed with as follows: $\begin{array}{l} ω_{D} = \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times {\sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D^{'}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) + \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D^{''}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j})}^{- 1} \\ ω_{D'} = \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D'_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times {\sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D^{'}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) + \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D^{''}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j})}^{- 1} \\ ω_{D''} = \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D''_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times {\sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) \times \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D^{'}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j}) + \sum_{\begin{matrix} f, g = 1 \\ f \neq g \end{matrix}}^{m} \sum_{j \in D^{''}_{f g}} ω_{j} \times \bar{d} (A_{f j}, A_{g j})}^{- 1} \end{array}$ (9)

Step 5. Formation of the consent matrix $C_{pq}^{}$ is the result of approvals and their weights; it is realized in accordance with the following: $G = [\begin{matrix} - g_{12} . . . g_{1 m} \\ g_{21} - . . . g_{2 m} \\ g_{(m - 1) 1} . . . . - g_{(m - 1) m} \\ g_{m 1} g_{m 2} g_{m (m - 1)} - \end{matrix}]$

where $g_{k i} = ω_{c} \times \sum_{j \in C_{k i}} ω_{j} + ω_{c}^{'} \times \sum_{j \in C'_{k i}} ω_{j} + ω_{c}^{''} \times \sum_{j \in C''_{k i}} ω_{j}$ (10)

so that the consent matrix can be written: $K = [\begin{matrix} - k_{12} . . . k_{1 m} \\ k_{21} - . . . k_{2 m} \\ k_{(m - 1) 1} . . . . - k_{(m - 1) m} \\ k_{m 1} k_{m 2} k_{m (m - 1)} - \end{matrix}]$ (11)

where k _ki = g ^* - g _ki, g ^* is the maximum consensus index.

Step 6. Formation of the matrix of discrepancies $D_{pq}^{}$ is the result of discrepancies and their weights, realized in accordance with the following: $H = [\begin{matrix} - h_{12} . . . h_{1 m} \\ h_{21} - . . . h_{2 m} \\ h_{(m - 1) 1} . . . . - h_{(m - 1) m} \\ h_{m 1} h_{m 2} h_{m (m - 1)} - \end{matrix}]$ (12)

where $h_{ki} = \frac{{max}_{j \in D_{ki}} ω_{D}^{*} \times d (X_{kj}, X_{ij})}{{max}_{j \in J} \times d (X_{kj}, X_{ij})},$ $ω_{D}^{*} = {ω_{D}, ω_{D}^{'}, ω_{D}^{''}}$ (13)

the discrepancy matrix is: $L = [\begin{matrix} - l_{12} . . . l_{1 m} \\ l_{21} - . . . l_{2 m} \\ l_{(m - 1) 1} . . . . - l_{(m - 1) m} \\ l_{m 1} l_{m 2} l_{m (m - 1)} - \end{matrix}],$ (14)

where l _ki = h ^* - h _ki,

h ^* is the maximum discrepancy index.

Step 7. Forming a dominance matrix Rwhich represents the aggregation of the safety matrix and the uncertainty matrix, and it has the following form: $R = [\begin{matrix} - r_{12} . . . r_{1 m} \\ r_{21} - . . . r_{2 m} \\ r_{(m - 1) 1} . . . . - r_{(m - 1) m} \\ r_{m 1} r_{m 2} r_{m (m - 1)} - \end{matrix}],$ (15)

where $r_{ki} = \frac{l_{ki}}{k_{ki} + l_{ki}}$

Ranking of alternatives is realized in accordance with the following: $T_{ki} = \frac{1}{m - 1} \sum_{i = 1, l \neq k}^{m} r_{ki}, k = 1, 2, . . ., m$ (16) where the best alternative is the one with the maximum value A = max{ T _k }.

2.5 Delphi method analysis

The Delphi method is a group decision-making process based on the anonymous and iterative collection of opinions by experts in a specific field. The advantage of the method is the anonymity of the experts, which frees them from the pressure to agree or oppose the opinions of the majority of the group. This creates an environment where experts freely express their opinions. The next advantage of the method is that it enables multiple iterations of gathering the opinions of experts, because after each iteration, the experts are given feedback on the collective opinion of the group. This phase allows the experts to revise and adjust their answers. This act enables the group’s consensus to be reached. Different methods for reaching consensus can be found in the literature, such as the adaptive approach [31]. In this paper, the consensus check is based on testing the hypothesis about the difference of means combined with the algebra of Fermatean fuzzy sets. The Delphi method usually uses professionals from a certain field, experts, which results in high-quality and reliable results of the decision made. Another advantage of the Delphi method is the efficient collection of opinions using a survey, which enables quick data collection and analysis. Bearing in mind that it is necessary to make a decision based on the opinion of experts, where the assessment is formed on the basis of experience in the form of linguistic statements, which causes uncertainty, the existing Delphi method will be expanded with an intuitive fuzzy set, more precisely Fermatean fuzzy set (FFSDelfi), which until now, based on the reviewed works, it was not used. The traditional Delphi method with modifications was used to select an intermodal route for cargo transportation from Korea to Central Asia [51], assessment of key competencies for lifelong learning [6], development of an optimal water supply plan using integrated fuzzy Delphi and fuzzy ELECTRE III methods—Basin case study Gamasiab [33], the selection of performance factors in many service industries [22], the fuzzy evaluation method is particularly suitable for expressing opinions and preferences of experts characterized by uncertainty, ambiguity, unobservability and scarcity [55], evaluation of forest roads [10].

3 Hybrid model for measuring the performance of business processes in complex organizations

The specificity of the organization and work technology of complex organizations requires the development of a hybrid model for performance measurement. Figure 1 shows the algorithm of the developed model, which is presented through the following stages:

Fig. 1

A hybrid model for performance measurement.

Phase 1. Formation of two teams of experts. The first team of experts is tasked with decomposing processes into sub-processes, selecting KPIs and determining the relative importance of PPi. Another team is tasked with evaluating each KPI at the level of each sub-process. Two teams were formed to reduce subjectivity, increase independence and fairness in evaluation.

Phase 2. Decomposing the process into sub processes. Based on the technological process and the application of the HIPO method, the experts identified the key sub-processes.

Also, from a large number of factors that influence the process, it is necessary to select the key performance indicators of the process. Experts define the KPIs on the basis of which the performance of the process will be measured.

Phase 3. Determining the relative importance of sub-processes by experts. The Delphi method is used for this activity, and the results are collected using a survey. The evaluation is carried out on the basis of the proposed linguistic expressions. Modeling is done using FFS. Agreement on relative importance values is checked by consensus. Until a consensus is reached, the results of all experts are returned to the experts through a survey with the possibility to reconsider their assessment, based on which they can give a different assessment or confirm the previous assessment. The value of the relative importance of the subprocess for which a consensus is reached is adopted.

Phase 4. Evaluation of KPIs at the level of each sub-process. Based on their competences, insight into the documentation that is kept in the process, review of resources, experts evaluate each KPI at the level of each PPi. The assessment is based on the proposed linguistic expressions according to a pre-determined linguistic scale. Linguistic expression modeling is realized using intuitive fuzzy sets of the third order (q = 3) or FFS.

Phase 5. To determine the value of the KPIs of the process, a phased method of multi-attribute decision-making is used, such as the method of conflicts (ELECTRE), extended with the Fermatean fuzzy set [52, 57]. The input values to the model are the relative importance of the PPi and the KPI value at the level of each PPi. The output values represent the values of the KPIs, the ranking of which determines the KPIs with greater or lesser importance for the process. Based on the value of KPIs, the organization’s management analyzes the results and takes measures to improve them.

4 Case study

The developed model was tested on the example of a overhaul organization that deals with the maintenance of complex combat systems. The application of the model is given through the following steps:

Step 1: Decomposing the maintenance process into sub-processes. A team of experts, based on the technological process prescribed for the overhaul of complex combat systems, using the HIPO method [11], proposed the following nine sub-processes: PP₁ - Previous defection, PP₂ - Disassembly of assemblies, sub-assemblies and spare parts, PP₃ - Repair of aggregates and equipment, PP₄ - Assembly, adjustments and partial tests, PP₅ - Final checks, PP₆ - Painting, PP₇ - Ground and flight tests, PP₈ - Overhaul process verification and PP₉ - Risk identification and assessment.

KPI identification is realized by a team of experts, who recognized the following performances as key for the maintenance process: KPI₁ - Availability of protective equipment needed in the process of asset maintenance, KPI₂ - User satisfaction with the maintenance service, KPI₃ - Material provision of spare parts and consumables, KPI₄ - Degree of application of safety at work measures in the overhaul process, KPI₅ - Availability of general and special tools and equipment, KPI₆ - Level of necessary knowledge and skills of employees, KPI₇ - Degree of coordination with other parts of the overhaul organization, KPI₈ - Ability to follow modern maintenance trends, KPI₉ - Degree of conformity of procedures, KPI₁₀ - Availability of experts, KPI₁₁ - Availability of technical literature and KPI₁₂ - Availability of workshop capacities.

Step 2: Determining the relative importance of sub-processes at the level of each KPI. Ten experts (e = E₁,...,E₁₀) from the overhaul organization participate in determining the weight of the subprocess. Suppose expert’s weighting vector of equal importance is (0.1). The experts evaluated the KPI at the level of each sub-process with linguistic statements based on Table 1.

Table 1
Performance ratings of sub-process as linguistic values [57]

Linguistic variables FFS

Absolutely Good (AG) (0.98, 0.02)

Very Good (VG) (0.9, 0.6)

Good (G) (0.8, 0.65)

Medium Good (MG) (0.75, 0.6)

Average (A) (0.5, 0.5)

Medium Bad (MB) (0.6, 0.7)

Bad (B) (0.7, 0.8)

Very bad (VB) (0.6, 0.9)

Absolutely Bad (AB) (0.02, 0.98)

Linguistic variables	FFS
Absolutely Good (AG)	(0.98, 0.02)
Very Good (VG)	(0.9, 0.6)
Good (G)	(0.8, 0.65)
Medium Good (MG)	(0.75, 0.6)
Average (A)	(0.5, 0.5)
Medium Bad (MB)	(0.6, 0.7)
Bad (B)	(0.7, 0.8)
Very bad (VB)	(0.6, 0.9)
Absolutely Bad (AB)	(0.02, 0.98)

The collection of evaluations by experts was carried out using a survey. On the basis of experts’ ratings given in the form of linguistic expressions, linguistic expressions were modeled using FFS. Then, the unification of the relative importance of ω _i at the level of each subprocess is realized using the FFWPA operator according to equation (1). The obtained value FFWPA-represents the relative importance of the subprocess ω _ppi. In order to examine whether the experts agree on the obtained value of ω _ppi for each subprocess, that is, whether a consensus has been reached, the variance σ is used, which represents a measure of consensus, according to equation (4). The experts unanimously decided that a consensus has been reached regarding the value of the relative importance of subportions ω _ppi if the variance is σ≤0.25. After the first iteration, the experts did not reach a consensus about the values of the relative importance of the ω _ppi subprocesses.

In the second iteration, the survey was returned to the experts as well as the results obtained from the previous iteration, with the idea of converging opinions. After the second iteration, a consensus was reached, the variance was δ≤0.25, and the results of the survey are given in Table 2.

Table 2

Evaluation of the importance of sub-processes by experts

Exp.	PP₁	PP₂	PP₃	PP₄	PP₅	PP₆	PP₇	PP₈	PP₉
E₁	VG	G	MG	MG	AG	VG	G	G	VB
E₂	AG	MG	MG	VG	MB	B	G	G	G
E₃	VG	MG	G	MG	MB	MG	B	MG	G
E₄	AG	A	G	G	G	VG	VG	MG	G
E₅	VG	MG	MG	MG	MG	A	G	A	VG
E₆	AG	MB	MG	G	G	B	VB	VG	MB
E₇	B	AG	VG	MG	AG	G	VG	G	VG
E₈	MG	MG	G	VG	G	VB	G	MB	MB
E₉	VG	AG	G	MG	MG	VG	G	MG	G
E₁₀	AG	MG	VG	VB	MG	G	VG	MB	G

The experts reached a consensus and decided that the sub-processes have the following relative importance: $\begin{matrix} ω_{{pp}_{1}} = 0.26; ω_{{pp}_{2}} = 0.13; ω_{{pp}_{3}} = 0.12; \\ ω_{{pp}_{4}} = 0.10; ω_{{pp}_{5}} = 0.12; ω_{{pp}_{6}} = 0.06; \\ ω_{{pp}_{7}} = 0.09; ω_{{pp}_{8}} = 0.07; ω_{{pp}_{9}} = 0.06; \end{matrix}$

Based on the obtained results, the experts assigned the greatest weight to the “previous defection” sub-process, which aims to accurately determine the failure of the asset as well as to establish the reason for the failure. The sub-processes “disassembly”, “repair of aggregates and equipment”, “assembly, adjustments and partial tests”, “final checks”, “ground and flight tests” have approximate relative importance because they are almost equally important to experts in the maintenance process. The sub-processes “painting the asset”, “verification of the repair process” and “risk identification and assessment” have less relative importance according to the opinion of the experts.

Step 3: The KPI evaluation at the level of each sub-process. Five experts from the overhaul organization participated in this activity (e = E₁, E₂, E₃, E₄ and E₅). Assume that the vector of equal importance expert weights is (0.2, 0.2, 0.2, 0.2, 0.2). The evaluation was carried out through a survey, where experts gave ratings according to Table 4, using the linguistic statements given in Table 3.

Table 3

Evalution values describes as linguistic variables by five experts

E	KPI_i/PP_i	PP₁	PP₂	PP₃	PP₄	PP₅	PP₆	PP₇	PP₈	PP₉
E₁	KPI₁	E	VE	E	VE	E	VE	VE	E	M
E₂		VE	ME	E	E	E	ME	VE	M	E
E₃		E	M	E	M	E	E	VE	VE	E
E₄		VE	U	ME	U	VE	ME	VE	VE	MU
E₅		E	E	VE	ME	VE	ME	VE	E	ME
E₁	KPI₂	E	E	VE	E	E	E	VE	E	E
E₂		E	E	ME	VE	VE	VE	ME	E	U
E₃		VE	E	M	U	E	ME	ME	M	MU
E₄		E	M	E	M	VE	VE	ME	M	VU
E₅		VE	VE	VE	E	MU	ME	ME	E	VE
E₁	KPI₃	VE	VE	VE	M	ME	ME	E	VE	E
E₂		VE	VE	VE	E	E	VE	ME	VE	E
E₃		VE	VE	E	VE	E	ME	ME	VE	E
E₄		VE	VE	E	VE	E	VE	ME	VE	E
E₅		VE	VE	U	ME	E	E	E	E	E
E₁	KPI₄	VE	VE	ME	VE	E	M	E	ME	E
E₂		E	M	VE	VE	E	VE	E	E	VE
E₃		VE	VU	ME	E	E	ME	E	ME	ME
E₄		E	VE	VE	ME	E	E	E	E	E
E₅		MU	E	E	ME	E	VE	E	M	ME
E₁	KPI₅	VE	VE	VE	VE	VE	E	VE	VE	VE
E₂		ME	M	ME	E	VE	ME	VE	ME	M
E₃		VE	E	VE	VE	E	ME	VE	VE	VE
E₄		ME	ME	E	E	VE	VE	VE	VE	E
E₅		E	M	VE	ME	VE	M	VE	ME	VE
E₁	KPI₆	E	E	ME	VE	E	ME	VE	ME	E
E₂		ME	VE	M	E	E	VE	VE	ME	ME
E₃		E	ME	VE	ME	E	M	VE	VE	VE
E₄		E	VE	M	VE	E	ME	VE	ME	VE
E₅		VE	VE	VE	E	M	M	VE	VE	ME
E₁	KPI₇	ME	E	M	VE	E	E	E	E	MU
E₂		E	E	M	E	E	ME	E	E	M
E₃		E	M	M	E	E	VE	E	E	E
E₄		E	E	M	VE	E	E	E	E	E
E₅		ME	M	M	VE	ME	ME	E	E	E
E₁	KPI₈	E	ME	VE	E	M	ME	E	MU	ME
E₂		E	VE	E	E	E	E	E	E	E
E₃		ME	ME	E	M	ME	VE	E	E	M
E₄		E	VE	VE	E	E	E	E	MU	ME
E₅		E	ME	VE	ME	E	M	E	E	M
E₁	KPI₉	VE	VE	E	VE	VE	MU	ME	VE	E
E₂		VE	E	E	E	VE	VE	VE	E	E
E₃		VE	E	E	E	VE	E	VE	ME	E
E₄		E	E	M	E	M	VE	VE	M	E
E₅		E	ME	E	VE	ME	U	VE	VE	E
E₁	KPI₁₀	VE	VE	VE	ME	VE	MU	VE	VE	VE
E₂		VE	M	VE	E	E	ME	ME	E	E
E₃		VE	E	VE	E	VE	VE	M	ME	ME
E₄		VE	E	VE	ME	M	VE	VE	ME	M
E₅		VE	VE	VE	E	E	E	E	ME	E
E₁	KPI₁₁	E	VE	VE	VE	E	ME	ME	M	VE
E₂		E	ME	VE	ME	VE	VE	VE	M	M
E₃		E	ME	VE	ME	ME	E	VE	ME	MU
E₄		ME	VE	E	VE	E	VE	E	M	E
E₅		E	VE	VE	VE	VE	MU	VE	VE	VE
E₁	KPI₁₂	E	VE	E	VE	E	E	VE	ME	M
E₂		VE	ME	VE	E	ME	VE	ME	VE	ME
E₃		VE	M	E	M	E	E	VE	VE	E
E₄		VE	VE	ME	VE	VE	VE	VE	E	VE
E₅		E	E	VE	ME	VE	VE	VE	VE	ME

Table 4

Performance ratings of subtprocess as linguistic values [56]

Linguistic variables	FFS
Very Eligible (VE)	(0.9, 0.2)
Eligible (E)	(0.8, 0.3)
Medium Eligible (ME)	(0.7, 0.5)
Medium (G)	(0.6, 0.6)
Medium Unqualified (MU)	(0.5, 0.7)
Unqualified (U)	(0.3, 0.8)
Very Unqualified (VU)	(0.2, 0.9)

Step 4: Unification of the KPI score at the level of each subprocess was realized using operator FFWPG Equation (3), where the decision matrix X _ij was formed based on expression Equation (2).

Step 5: On the basis of expression (6), sets of agreement were formed, namely: strong concordance ω _c, medium concordance ω _c′, weak concordance set ω _{c
^′′}, that is, based on expression (7), sets of strong discordance ω _D, medium discordance ω _D′, and weak discordance ω _{D
^′′} were obtained. After the formation of concordance and discordance sets, based on expression (8), the weights of strong concordance set ω _c; the weights of medium concordance set ω _c′; the weights of weak concordance set ω _{c
^′′} were obtained. Based on expression (9), the relative importance of the strong discordance set ω _D; the weights of medium discordance set ω _D′; the weights of weak discordance set ω _D″ are computed. In this way, the following values were calculated: ω _c = 0.753 ; ω _c′ = 0.116 ; ω _c″ = 0.131and ω _D = 0.755 ; ω _D′ = 0.115 ; ω _D″ = 0.129.

Step 6: Based on expressions (10) and (11), a matrix of agreement was constructed K: $K = (\begin{matrix} - 0, 45 0, 46 0, 38 0, 41 0, 21 0, 15 0, 22 0, 40 0, 49 0, 22 0, 46 \\ 0, 44 - 0, 59 0, 54 0, 29 0, 28 0, 18 0, 34 0, 60 0, 60 0, 41 0, 62 \\ 0, 10 0, 24 - 0, 35 0, 24 0, 21 0, 09 0, 02 0, 25 0, 42 0, 27 0, 26 \\ 0, 43 0, 58 0, 44 - 0, 53 0, 36 0, 33 0, 31 0, 53 0, 53 0, 57 0, 56 \\ 0, 24 0, 31 0, 37 0, 20 - 0, 24 0, 11 0, 26 0, 32 0, 29 0, 33 0, 25 \\ 0, 39 0, 39 0, 47 0, 37 0, 50 - 0, 12 0, 00 0, 44 0, 29 0, 21 0, 34 \\ 0, 46 0, 48 0, 47 0, 48 0, 50 0, 49 - 0, 42 0, 54 0, 54 0, 54 0, 51 \\ 0, 49 0, 50 0, 54 0, 51 0, 40 0, 49 0, 23 - 0, 53 0, 62 0, 59 0, 53 \\ 0, 28 0, 18 0, 46 0, 25 0, 25 0, 32 0, 16 0, 13 - 0, 41 0, 28 0, 48 \\ 0, 18 0, 10 0, 42 0, 24 0, 23 0, 20 0, 18 0, 02 0, 31 - 0, 28 0, 20 \\ 0, 31 0, 28 0, 29 0, 41 0, 42 0, 38 0, 08 0, 20 0, 50 0, 36 - 0, 34 \\ 0, 03 0, 13 0, 39 0, 20 0, 24 0, 15 0, 04 0, 09 0, 41 0, 44 0, 25 - \end{matrix})$

Step 7: On the basis of expressions (12), (13) and (14) the disagreement matrix L was constructed: $L = (\begin{matrix} - 0, 52 0, 03 0, 23 0 0 0, 17 0, 37 0 0 0 0 \\ 0 - 0, 16 0, 64 0, 15 0, 12 0, 26 0, 49 0, 28 0, 31 0, 21 0, 15 \\ 0 0, 71 - 0, 52 0 0 0, 53 0, 52 0 0, 12 0, 19 0 \\ 0 0, 65 0 - 0 0 0, 50 0, 42 0, 26 0, 18 0, 30 0 \\ 0, 61 0, 53 0, 09 0, 47 - 0, 15 0, 64 0, 49 0, 33 0, 06 0, 18 0 \\ 0, 24 0, 40 0, 29 0, 31 0 - 0, 48 0, 27 0, 19 0 0 0 \\ 0 0 0 0 0 0 - 0 0 0 0 0 \\ 0 0, 41 0 0, 24 0 0 0, 36 - 0, 04 0 0 0 \\ 0, 49 0, 71 0, 20 0, 41 0 0, 26 0, 61 0 - 0 0 0, 27 \\ 0, 08 0, 70 0 0, 30 0 0, 15 0, 44 0, 76 0, 30 - 0, 23 0 \\ 0, 06 0, 36 0 0, 68 0 0, 16 0, 68 0, 76 0, 17 0 - 0 \\ 0, 53 0, 76 0, 26 0, 60 0, 12 0, 44 0, 54 0, 55 0, 49 0, 07 0, 43 - \end{matrix})$

Step 8: Based on expression (15), the dominance matrix R was obtained, which has the following form: $R = (\begin{matrix} - 0, 54 0, 05 0, 38 0 0 0, 53 0, 63 0 0 0 0 \\ 0 - 0, 21 0, 54 0, 34 0, 29 0, 59 0, 59 0, 31 0, 34 0, 34 0, 20 \\ 0 0, 75 - 0, 60 0 0 0, 86 0, 96 0 0, 23 0, 41 0 \\ 0 0, 53 0 - 0 0 0, 61 0, 58 0, 33 0, 25 0, 34 0 \\ 0, 71 0, 64 0, 19 0, 70 - 0, 39 0, 86 0, 65 0, 51 0, 16 0, 35 0 \\ 0, 38 0, 51 0, 38 0, 46 0 - 0, 80 1 0, 30 0 0 0 \\ 0 0 0 0 0 0 - 0 0 0 0 0 \\ 0 0, 45 0 0, 32 0 0 0, 61 - 0, 07 0 0 0 \\ 0, 63 0, 79 0, 31 0, 62 0 0, 45 0, 79 0 - 0 0 0, 36 \\ 0, 31 0, 87 0 0, 56 0 0, 43 0, 73 0, 97 0, 49 - 0, 46 0 \\ 0, 15 0, 56 0 0, 62 0 0, 29 0, 89 0, 79 0, 26 0 - 0 \\ 0, 95 0, 85 0, 40 0, 75 0, 33 0, 75 0, 93 0, 86 0, 54 0, 13 0, 63 - \end{matrix})$

Step 9: Based on the expression (16), the ranking of KPIs was performed, where the following results were obtained:

T_k1 = 0.193; T_k2 = 0.342; T_k3 = 0.347; T_k4 = 0.240; T_k5 = 0.469; T_k6 = 0.347; T_k7 = 0; T_k8 = 0.132; T_k9 = 0.360; T_k10 = 0.438; T_k11 = 0.324; T_k12 = 0.648

By comparing the KPIs based on the obtained values, the following rank is obtained:

KPI₁₂ > KPI₅ > KPI₁₀ > KPI₉ > KPI₆ > KPI₃ > KPI₂ > KPI₁₁ > KPI₄ > KPI₁ > KPI₈ > KPI_7 .

5 Discussion and results

5.1 Discussion and results

Based on the obtained results, it can be considered that KPIs with a higher priority, that is, KPIs that have the greatest impact on the technological maintenance process, are KPIs whose values are in the interval from 0.3 to 0.7 and that they can be called special KPIs. While for KPIs whose values are in the interval from 0 to 0.2999, they will be KPIs that do not directly affect the maintenance process, but are not excluded from consideration because their importance is a necessary prerequisite for the initiation, implementation and completion of the technological maintenance process, and as such they can be called general KPIs. The model generated the following special

KPIs: availability of workshop capacities (KPI₁₂), availability of general and special tools and equipment (KPI₅), availability of experts (KPI₁₀), degree of conformity of procedures (KPI₉), level of required knowledge and skills (KPI₆), material provision of spare parts and consumables (KPI₃), user satisfaction (KPI₂) and availability of technical literature (KPI₁₁). The general KPIs that represent a prerequisite for the maintenance process are: the degree of application of occupational safety measures (KPI₄), the possibility of monitoring modern trends (KPI₈), the availability of protective equipment in the overhaul process (KPI₁) and the degree of coordination with other organizational units in the overhaul institute (KPI₇). General KPIs provide, through their processes, other units of the overhaul system that are not in the direct technological process. Based on the results, it can be concluded that for professionals, general KPIs do not represent a focus for the maintenance process, because protective equipment is provided up to compliance standards, all safety measures at work are prescribed to prevent injury to persons at work, coordination with other organizational units in terms of finances, procurement of required funds, as well as infrastructural support of workshop capacities functions without interruption. The possibility of monitoring modern repair trends as an indicator depends on the availability of literature for new means, the possibility of participating in seminars, visiting other repair systems that deal with the maintenance of more modern means, is specific and more difficult to achieve due to the reason of providing access to such activities and repair systems abroad. Ensuring the aforementioned conditions for monitoring modern overhaul trends is the responsibility of organizational units outside the technological process, so KPI₈ is also classified as general KPIs in the real system. Special KPIs require constant measurement, monitoring and improvement. The availability of workshop capacities (KPI₁₂) has the greatest impact on the maintenance process, and as such can represent a bottleneck in the maintenance process. It depends on infrastructural support, compliance with the line of the technological maintenance process, and most of all it depends on the allocation of capacity according to the requirements of the users of the funds. The problem arises when maintenance requirements exceed workshop capacities. Availability of general and special tools and equipment (KPI₅) indicator that presents the availability of tools and equipment in the maintenance process, its correctness and attestation. Disadvantages are manifested in the appearance of tool malfunctions, retention of tools and equipment in the certification process, unavailability of certain tools on the market. Availability of experts (KPI₁₀) indicator that represents the occupancy of experts. The lack of professionals is present in most companies, due to various trends (planned and natural outflows, migrations, leaving for better positions, retraining for other jobs, etc.), the impossibility of replacement in case of absence, as well as the possibility of young people entering the technological process. The degree of conformity of the procedures (KPI₉) presents the conformity of the procedures of the technological maintenance process with standards, rules, regulations, normatively regulated. The problems affecting this indicator are the up-to-dateness of procedures, the availability of rules and regulations for new assets coming for overhaul. Material provision with spare parts and consumables (KPI₃) represents the basis of the maintenance process. The problems affecting the indicator are mostly related to the area of procurement of spare parts and consumables, i.e. the reliability of purchased spare parts. Global problems in the form of the Covid pandemic have not bypassed the industrial sector for the production of spare parts. Deliveries of spare parts are delayed, smaller quantities are delivered compared to the required quantities. The reliability of the delivered spare parts is questionable because the vehicle often fails after overhaul. The level of required knowledge and skills (KPI₆) requires knowledge of tools, failure defects, assembly disassembly and assembly. What can reduce the value of this indicator is the ability to overhaul new assets. The availability of technical literature (KPI₁₁) is one of the tools of experts. The variety of assets puts experts in a disadvantageous situation when they do not have technical literature describing the maintenance of the asset, diagrams and images of assemblies and subassemblies, factory numbers of spare parts, etc. User satisfaction (KPI₂) presents an assessment of the work of the unit that deals with maintenance, indicates failures in the work on the asset itself, which can be useful to management.

In order to improve the indicators, the obtained results were analyzed, potential problems affecting individual KPIs were considered, and a proposal for measures to improve them was made. Workshop capacities need to be expanded and equipped with tools and equipment for maintenance in accordance with the introduction of new resources, worn-out tools should be replaced with new and more modern ones. For the layout of tools and equipment, use the Lean concept, such as “5 s principles” [19], send tools and equipment for certification at intervals so that the workshop does not remain without them for a long period of time. KPI₁₀ should be increased through the training of experts in the maintenance of related assets so that replaceability is possible, then the earlier entry of young experts into the maintenance process before older people end their working life. This results in the transfer of experience to young people as well as continuity in the maintenance process. By introducing new means of requiring and training maintenance personnel. Therefore, before new assets are put into use, it is necessary to carry out: training of maintenance professionals, which will increase the value of KPI₆, procurement of technical literature that follows the asset maintenance process that affects KPI₁₁, procurement of those spare parts whose operational reliability is rated as lower value, thereby creating a base that will influence KPI₃. Fulfillment of the aforementioned proposals will also improve user satisfaction with maintenance services (KPI₂).

A deeper analysis of KPIs can lead to the conclusion that even if KPIs are considered individually, there is still an interdependence between them, because the consequence of one KPI chain transfers the problem to other KPIs.

5.2 Managerial implications

Measuring the performance of business organizations is a complex and time-consuming process. Existing performance measurement models require modifications, which may lead to a violation of the basic concept of the performance measurement model. In order to depict the state of the process as realistically as possible, a hybrid model is proposed that can be widely applied in various areas where the subjective opinion of the decision maker plays a key role. Evaluating KPIs that are qualitative in size requires a subjective assessment, with the primary focus on harmonizing or reaching consensus. Reducing bias and influence in decision-making requires finding a mode, which in this model is presented as a solution with several teams of experts, where one of the teams focuses on the KPIs and PPi that will be measured, and the other team performs their evaluation. Uncertainty modeling is represented in the model by an intuitive fuzzy approach of the q^th order (where q = 3), in order to reduce fluctuations in subjective assessment.

In order to check whether and how uncertainty modeling by changing the q^th order affects the results of the proposed model, with the same input data, the model is tested for the values q = (4,5,6,7,8,9,10). Algebra for applying q-rung orthopair fuzzy set is given in the work of Pinar and Boran [35], and the obtained results are given in Table 5 and graphically shown in Fig. 2.

Table 5
Rank KPIs using q-rung orthopair fuzzy set

KPIs q = 3 q = 4 q = 5 q = 6 q = 7 q = 8 q = 9 q = 10

KPI₁ 10 8 8 8 8 8 8 8

KPI₂ 7 10 10 10 10 10 10 10

KPI₃ 6 4 4 4 4 4 4 4

KPI₄ 9 9 9 9 9 9 9 9

KPI₅ 2 2 2 2 2 2 2 3

KPI₆ 5 6 5 5 6 6 7 7

KPI₇ 12 12 12 12 12 12 12 12

KPI₈ 11 11 11 11 11 11 11 11

KPI₉ 4 5 6 6 5 5 5 5

KPI₁₀ 3 3 3 3 3 3 3 2

KPI₁₁ 8 7 7 7 7 7 6 6

KPI₁₂ 1 1 1 1 1 1 1 1

KPIs	q = 3	q = 4	q = 5	q = 6	q = 7	q = 8	q = 9	q = 10
KPI₁	10	8	8	8	8	8	8	8
KPI₂	7	10	10	10	10	10	10	10
KPI₃	6	4	4	4	4	4	4	4
KPI₄	9	9	9	9	9	9	9	9
KPI₅	2	2	2	2	2	2	2	3
KPI₆	5	6	5	5	6	6	7	7
KPI₇	12	12	12	12	12	12	12	12
KPI₈	11	11	11	11	11	11	11	11
KPI₉	4	5	6	6	5	5	5	5
KPI₁₀	3	3	3	3	3	3	3	2
KPI₁₁	8	7	7	7	7	7	6	6
KPI₁₂	1	1	1	1	1	1	1	1

Fig. 2

Displaying the rank of KPIs for the values of q = (3,4,5,6,7,8,9,10).

Comparing the results obtained using the Fermatean fuzzy set (q = 3) with the results when q takes values from 4 to 10, it can be seen that the model gives the same or similar rank for KPI₄, KPI₅, KPI₇, KPI₈, KPI₁₀ and KPI₁₂, that is, less rank changes for KPI₁, KPI₂, KPI₃, KPI₆, KPI₉ and KPI₁₁.

In comparing the obtained results for q > 3, with the results from the business organization, the importance of KPI₄, KPI₅, KPI₇, KPI₈, KPI₁₀ and KPI₁₂ to the decision makers is in accordance with the obtained ranks, while the importance of KPI₂, KPI₆, KPI₉ is higher and lower for KPI₁, KPI₃ and KPI₁₁ in relation to the obtained results. The importance of KPIs when q = 3, for the organization’s management, corresponds more to the real situation.

Looking at Fig. 2, it can be concluded that the model is less sensitive to changes in the value of q, because KPIs in most cases kept their ranks and remained in the groups of more important and less important KPIs.

Uncertainty modeling using FFS (q = 3) is more suitable for this model than the application of q-rung orthopair fuzzy set (q > 3), which does not mean that the opposite results would be achieved in other examples.

The proposed model can be implemented in a software application, so that decision-makers, after collecting surveys from experts, can easily enter the KPI values into the model, and get performance values at the output of the model. In this way, with less effort and time invested, results would be reached faster, which the management of the organization would further use as a basis for taking measures to improve the organization’s processes.

6 Conclusions

The developed model was tested on the example of an overhaul organization that deals with the maintenance of complex combat systems. The obtained results presented performances that are key as well as performances that are less important for the maintenance process, but their influence is not excluded, because they do not generate problems that affect the maintenance process. The proposed model had imprecise data for the input results, which required the application of fuzzy sets. Higher-order fuzzy sets, such as FFS, were chosen for uncertainty modeling. The advantage of FFS uncertainty modeling is that, in addition to the decision made, the assessment includes the degree of certainty and uncertainty in the decision made, thus achieving better uncertainty modeling. Determining the relative importance of sub-processes, the Delphi method was chosen. Compared to the traditional approach, the model was modified for the first time with FFS. The Delphi method gained an advantage over other methods for determining relative importance because it allowed experts to express their opinions without the influence of the environment, as well as to have an insight into the evaluations of other experts through subsequent iterations, which gives them the opportunity to reconsider their decisions. Harmonizing the opinions and attitudes of several persons in practice is very difficult. The Delphi method made it possible to reach consensus on certain KPIs in a simpler way. The ranking of KPIs required the application of some of the MGDM methods. The conflict method (ELECTRE) was chosen for the model. Using the ELECTRE method extended FFS, a comparison of KPIs was realized at the level of each PPi subprocess, where the output result gives a realistic ranking. Also, the ELECTRE method provides a clear picture of the values of KPIs, that is, the preferences between KPIs. Applying the ELECTRE method enables a realistic assessment of the efficiency of the maintenance process. Based on the results, the ranked KPIs are classified into general and special. General KPIs do not have a direct impact on the technological process of maintenance, but they are not excluded from consideration, because they are a prerequisite for the start, implementation and completion of the technological process. Special KPIs verified the real picture of the technological maintenance process. There is room for their improvement, which is given in the paper.

Analysis of the sensitivity of the model to the application of other fuzzy sets is realized using q-rung orhopair fuzzy sets. Values from 4 to 10 are taken for q, the model showed that it is less sensitive to changes, and that the results obtained with FFS (q = 3) in this case correspond more to the real situation than the results obtained for q > 3. KPIs remained in the group of more important and less important for the maintenance process.

In future research, it is desirable to use other models of multi-criteria decision-making, and based on the obtained results, perform a comparative analysis with the results obtained on the basis of developed model.

The developed model has verified the results from a real system and as such can be used in similar organizations that mainly deal with the maintenance of assets. Due to its complexity, the developed model requires application improvement, which makes it easier for the organization’s management to access and use it more easily. The model can be used for various organizations regardless of the field of interest.

References

Akin

and Hatice

C.A.

, Medical waste disposal planning for healthcare units using spherical fuzzy CRITIC-WASPAS, Applied Soft Computing 144 (2023), 110480. https://doi.org/10.1016/j.asoc.2023.110480.

Akram

, Bibi

and Devec

, An outranking approach with 2-tuple linguistic Fermatean fuzzy sets for multi-attribute group decision-making, Engineering Applications of Artificial Intelligence 121 (2023), 105992.https://doi.org/10.1016/j.engappai.2023.105992.

Akram

, Luqman

and Alcantud

J.C.R.

, An integrated ELECTRE-I approach for risk evaluation with hesitant Pythagorean fuzzy information, Expert Systems With Applications 200 (2022). https://doi.org/10.1016/j.eswa.2022.116945.

Akram

, Noreen

and Deveci

, Enhanced ELECTRE II method with 2-tuple linguistic m-polar fuzzy sets multi-criteria group decision making, Expert Systems with Applications 213 (2023),part C, https://doi.org/10.1016/j.eswa.2022.119237.

Alahmadi

R.A.

, Ganie

A.H.

, Al-Qudah

, Khalaf

M.M.

and Ganie

A.H.

, Multi-attribute decision-making based on novel Fermatean fuzzy similarity measure and entropy measure, Granular Computing, 2023. https://doi.org/10.1007/s41066-023-00378-x.

Aleksić

, Nestić

, Huber

and Ljepava

, The Assessment of the Key Competences for Lifelong Learning—The Fuzzy Model Approach for Sustainable Education, Sustainability 14 (2022), 2686. https://doi.org/10.3390/su14052686.

Atanassov

K.T.

, Intuitionistic fuzzy sets, Fuzzy Sets and Systems 20(1) (1986), 87–96.

Carra

, Botticini

, Pavesi

F.C.

, Maternini

, Pezzagno

and Barabino

, A comparative cycling path selection for sustainable tourism in Franciacorta. An integrated AHP-ELECTRE method, Transportation Research Procedia 69 (2023), 448–455. https://doi.org/10.1016/j.trpro.2023.02.194.

Changlin

and Juhong

, Multi-criteria decision making and pattern recognition based on similarity measures for Fermatean fuzzy sets, Journal of Intelligent & Fuzzy Systems 41 (2021), 5847–5863. 10.3233/JIFS-201557.

10.

Dimić

and Ljubojević

, Decision Making Model In Forest Road Network Management, Vojnotehnički glasnik/Military technical courier 67(1) (2019)pp.93–115. doi: 10.5937/vojtehg67-18446; https://doi.org/10.5937/vojtehg67-18446.

11.

Durugbo

, Tiwari

and Alcock

J.R.

, A review of information flow diagrammatic models for product–service systems, International Journal of Advanced Manufacturing Technology 52(9-12) (2011),pp.1193–1208.

12.

Duygu

, Sarib

I.U.

and Senapati

, Extension of capital budgeting techniques using interval-valued Fermatean fuzzy sets, Journal of Intelligent & Fuzzy Systems 42 (2022), 365–376. DOI:10.3233/JIFS-219196.

13.

Enos

D.D.

, Performance improvement: Makingithappen, CRC Press 2007.

14.

European Founation for Quality Management: https://efqm.org/the-efqm-model/#.

15.

Ganai

S.A.

, Bhardwaj

and Padder

R.A.

, From Fuzzy Sets to Deep Learning: Exploring the Evolution of Pattern Recognition Techniques, International Journal of Science, Mathematics and Technology Learning 31(1) (2023),ISSN: 2327-7971 (Print) ISSN: 2327-915X (Online)

16.

Gul

and Fatih

A.M.

, Occupational Risk Assessment for Flight Schools: A 3,4-Quasirung Fuzzy Multi-Criteria Decision Making-Based Approach, Sustainability 14 (2022), 9373. https://doi.org/10.3390/su14159373.

17.

Harrington

H.J.

, Business process improvement: The break through strategy for total quality, productivity, and competitiveness, McGraw Hill Professional 1991.

18.

Vital Signs: Using Quality. Time andcost performance measurements to chart your company’s future, Amacom, New York, NY. 1993.

19.

http://www.cimlss.rs/5s/.

20.

Judson

A.S.

, Making Strategy Happen, Transforming Plans into Reality. London: Basil Blackwell. 1990

21.

Kaplan

S.R.

and Norton

D.P.

, Putting the Balanced Scorecard to Work, Harvard Business Review 71(5) (1993), 134–147.

22.

Kardaras

D.K.

, Karakostas

and Mamakou

X-J.

, Content Presentation Personalization and Media Adaptation in Tourism Web Sites Using Fuzzy Delphi Method and Fuzzy Cognitive Maps, Expert Systems with Applications 40(6) (2013), 2331–2342. 10.1016/j.eswa.2012.10.031.

23.

Khatrouch

, Kermad

, el Mhamedi

and Boujelbene,

, A hybrid AHP-ELECTRE I multicriteria model for performance assessment and team selection. Organizational productivity and performance measurements using predictive modeling and analytics, IGI Global (2017), 115–127.

24.

Kirisci

and Necip

, A novel kernel principal component analysis with application disaster preparedness of hospital: interval-valued Fermatean fuzzy set approach,;, The Journal of Supercomputing (2023). https://doi.org/10.1007/s11227-023-05395-w.

25.

Kirisci

, Fermatean Fuzzy Type Statistical Concepts with Medical, Fuzzy Optimization and Modelling 4(1) (2023), 1–14. E-ISNN: 2676-7007,https://doi.org/10.30495/fomj.2023.1982256.1082.

26.

Liang

, Fu

and Ishizaka

, A consensual group ELECTRE-SORT approach considering the incomparable classes with the application of machine maintenance strategy assignment, Omega 118 (2023). https://doi.org/10.1016/j.omega.2023.102849.

27.

Lukic

, Analysis of the Trade Performance of the European Union and Serbia on the Base of FF-WASPAS and WASPAS Methods, International Comparative Management 24(2) (2023). 10.24818/RMCI.2023.2.228.

28.

Lukic

, Application of ELECTRE method in performance analysis of food retailers in Serbia, Business Excellence and Management 1(3) (2021a)84–102. DOI:https://doi.org/10.24818/beman/2021.11.3-05.

29.

Milenkov

, Milovanović

, Sokolović

and Epler

, A methodology for the bidders evaluation and selection in the public procurement process based on heterogeneous information and adaptive consensus approaches, International Journal of Industrial Engineering 30(19) (2023), 203–231. DOI:10.23055/ijietap.2023.30.1.8479.

30.

Milovanović

, Aleksić

, Sokolović

and Milenkov

, Uncertainty modeling using intuitionistic fuzzy number. Vojnotehnički glasnik/Military technical courier, 69(4) (2021), 905–929. DOI: 10.5937/vojtehg69-33301; https://doi.org/10.5937/vojtehg69-33301.

31.

Mohamad

S-N-A.

, Embi

M-A.

and Nordin

, Determining E-Portfolio Elements in Learning Process Using Fuzzy Delphi Analysis, International Education Studies 8 (2015), 171–176.

32.

Neely

, Business Performance Measurement –Theory and Practice, Cambride University Press (2002), pp. 63.

33.

Noori

, Bonakdari

, Ki

K.M.

and Gharabaghi

, Development of optimal water supply plan using integrated fuzzy Delphi and fuzzy ELECTRE III methods—Case study of the Gamasiab basin, Expert Systems (2020), e12568. https://doi.org/10.1111/exsy.12568, DOI: 10.1111/exsy.12568.

34.

Parmenter

, Key performance indicators: Developing, implementing, and using winning KPIs. JohnWiley&Sons. 2015.

35.

Pinar

and Boran

E.F.

, A q-rung orthopair fuzzy multi-criteria group decision making method for supplier selection based on a novel distance measure, International Journal of Machine Learning and Cybernetics 11 (2020), 1749–1780.

36.

Pritchard

R.D.

, Roth

P-L.

, Jones

S.D.

and Roth

P.G.

, Implementing feedback systems to enhance productivity: A practical guide, Global Business and Organizational Excellence 10(1) (1990), 57–67.

37.

Rocha

, Application of AHP and ELECTRE I decision-making methods to solve a health and safety problem, Procedia Computer Science 219 (2023), 28–35. https://doi.org/10.1016/j.procs.2023.01.260.

38.

Rogulj

and Kilić-Pamuković

, Environmental Adaptation of Construction Barriers under Intuitionistic Fuzzy Theory, , Technical Journal 15 (2021), 1,1–10. https://doi.org/10.31803/tg-20210215210742.

39.

Rosemann

, Brocke

V.J.

, The six core elements of business process management, U J. vom Brocke & M. Rosemann (Editori), Handbook on Business Process Management. Introduction, methods and informations system. (2010), 107–122. Berlin: Springer.

40.

Ruan

C.Y.

, Chen

X.J.

and Han

L.N.

, Fermatean Hesitant Fuzzy Prioritized Heronian Mean Operator and Its Application in Multi-Attribute Decision Making, Computers, Materials & Continua 75(2) (2023),pp.3204–3222.19p.

41.

Saha

, Pamucar

, Gorcun

O.F.

and Mishra

A.R.

, Warehouse site selection for the automotive industry using a Fermatean fuzzy-based decision-making approach, Expert Systems With Applications 211 (2023), 118497. https://doi.org/10.1016/j.eswa.2022.118497.

42.

Saraji

M.K.

, Streimikiene

D.D.

and Kyriakopoulos

G.L.

, Fermatean Fuzzy CRITIC-COPRAS Method for Evaluating the Challenges to Industry 4.0 Adoption for a Sustainable Digital Transformation, Sustainability, 13 (2021), 9577. https://doi.org/10.3390/su13179577.

43.

Senapati

and Yager

R.R.

, Some new operations over Fermatean fuzzy numbers and application of Fermatean fuzzy WPM in multiple criteria decision making,b), Informatica 30(2) (2019), 391–412.

44.

Senapati

and Yagwer

R.R.

, Fermatean fuzzy sets, Journal of Ambient Intelligence and Humanized Coputing 11 (2020), 663–674.

45.

Senapati

and Yagwer

R.R.

, Fermatean fuzzy weighted averaging/geometric operators its application in multi-criteria decision-making methods, Engineering Applications of Artificial Intelligence 85 (2019a)112–121.

46.

Shahzadi

and Akram

, Group decision-making for the selection of an antivirus mask under Fermatean fuzzy soft information, Journal of Intelligent & Fuzzy Systems 40 (2021), 1401–1416. DOI:10.3233/JIFS-201760.

47.

Shen

Z.S.

, Zhang

, Rodriqueiz

R.M.

, Perdycz

and Martinez

, Expertise-based bid evaluation for construction-contractor selection with generalized comparative linguistic ELECTRE III, Automation in Construction 125 (2021), 103578.

48.

Sokolova

M.L.

and Georgievich

V.Ch.

, Choice of the means of transport for the creation of a logistic system relying on uncertainties,, Vojnotehnički glasnik/Military technical courier 64(2) (2016). 10.5937/vojtehg64-10304.

49.

Vural

and Köse

, Selection of Alternative Filling Material in The Bed Production with AHP and ELECTRE Methods, Journal of Applied Research on Industrial Engineering 7(2) (2020), 163–176.

50.

Wang

, Wei

, MABAC method for multiple attribute group decision making under q-rung orthopair fuzzy environment, Defence Technology 2019;https://doi.org/10.1016/j.dt.2019.06.019.

51.

Wang

, Yeo

G.T.

, Intermodal route selection for cargo transportation from Korea to Central Asia by adopting Fuzzy Delphi and Fuzzy ELECTRE I methods, Maritime Policy & Management 2017; ISSN: 0308-8839, Journal homepage:https://www-tandfonline-com.web.bisu.edu.cn/loi/tmpm20.

52.

M.C.

and Chen

T.Y.

, The ELECTRE Multicriteria Analysis Approach Based on Atanassov’s Intuitionistic Fuzzy Sets. Available at:, Expert Systems with Applications 38(10) (2011),pp.12318–12327. https://doi.org/10.1016/j.eswa.2011.04.010.

53.

M.C.

, Comparative study of ELECTRE methods with intuitionistic fuzzy sets applied on consumer decision making case, European Journal of Engineering Research and Science 4(10) (2019), 103–110.

54.

Yager

R.R.

, Pythagorean fuzzy subsets. In Joint IFSA World,AB, Congress and NAFIPS Annual Meeting (IFSA/NAFIPS), Edmonton (2013), 57–61. http://doi.org/10.1109/IFSA-NAFIPS.2013.6608375.

55.

Yeo

G.T.

, Wang

and Chou

C.C.

, Evaluating the Competitiveness of the Aerotropolises in East Asia, Journal of Air Transport Management 32 (2013), 24–31 10.1016/j.jairtraman.2013.06.004.

56.

Zeng

, Chen

, Gu

and Zhang

, An Integrated EDAS Model for Fermatean Fuzzy Multi-Attribute Group Decision Making and Its Application in Green-Supplier Selection, Systems 11 (2023), 162. https://doi.org/10.3390/systems11030162.

57.

Zhou

L.P.

, Wan

S.P.

and Dong

J-Y.

, A Fermatean fuzzy ELECTRE method for multi-criteria group decision-making, Informatica 33(1) (2022), 181–224.