Abstract
With the economic growth of the world, sustainable development is a popular issue in recent years. Sustainable assessment is an important part of sustainable development. There are many previous scholars have used multiple-criteria decision-making (MCDM) to develop different evaluation frameworks in different fields. Elimination et Choix Traduisant la Realite II (ELECTRE II) is one of the most commonly used methods for MCDM. ELECTRE II uses alternatives, criteria, and criteria weighting from decision-makers to calculate the concordance and discordance indices. These two indices are used to rank the alternatives. The concordance and discordance indices in ELECTRE II are important because they are the key to make accurate decisions. Previous scholars have failed to make comprehensive calculations for these indices, nor make their units of measure comparable, which negatively affected their results. This study improved the approach in calculating these indices and illustrated it using three case studies: (1) university examination results, (2) a sustainability assessment of groundwater remediation and (3) an assessment of power generation technologies. This improved ELECTRE II method offers decision-makers an objective basis for decision-making.
Introduction
With the development of human economy and the consumption of natural resources, many environments have been destroyed. Therefore, sustainable development has become the global goal in recent years. The goal is to achieve a balance between society and environment. There are many related fields of academic research and challenges [6], including environment, energy, climate, economy, society and engineering. In order to achieve sustainable development, the process of development should be measured [12, 42]. There are many popular issues in the process of sustainable development, including integration of sustainable alternatives and criteria [6], and development the framework for sustainable decision-making [18].
Decision-making is the process of finding the best solution among many feasible solutions based on a given set of criteria [19]. For multiple-criteria decision-making (MCDM), mathematical methods are often employed to rank the alternatives to find the best solution in the context of limited resources [2, 34]. Because conflict among the criteria is common, MCDM techniques can be applied in these scenarios to make objective assessments.
Dozens of MCDM methods have been developed, including simple additive weighting (SAW), analytic hierarchy process (AHP), data envelopment analysis (DEA), and technique for order preference by similarity to an ideal solution (TOPSIS), VlseKriterijuska Optimizacija I Komoromisno Resenje (VIKOR), decision making and trial evaluation laboratory (DEMATEL), elimination et choice traduisant la realite (ELECTRE), multi-objective optimization on the basis of ratio analysis (MOORA), weighted aggregated sum product assessment (WASPAS), multi-attributive border approximation area comparison (MABAC), evaluation based on distance from average solution (EDAS), combined compromise solution (CoCoSo), measurement of alternatives and ranking according to compromise solution (MARCOS).
Sarker et al. [27] combined VIKOR, TOPSIS, SAW, and balanced scorecard to evaluate sustainable performance assessment in the leather industry. Rao [37] used DEMATEL to find the interaction between the criteria, and ANP to find the important factors in the corporate social responsibility impact of the Taiwan High Speed Rail. Mohammed et al. [1] considered the changes in demand and supply in the supply chain. They used DEMATEL to find out the importance of the criteria through and then selected suppliers using TOPSIS. Saraswat et al. [38] found suitable locations for wind and solar power plants in India using GIS and AHP. Ghorui et al. [28] identified the weights of COVID-19 transmission risk factors by AHP and then ranked the risk factors by TOPSIS. Seker and Kahraman [35] integrated AHP and Multi MOORA to evaluate the manufacturer for solar power plants. Bhadra and Dhar [10] identified the weights by using AHP and select the material in aerospace cabins by using TOPSIS, EDAS, and COPRAS. Huang [30] used VIKOR to select the technology of renewable energy and compared the results with WSM and TOPSIS.
Among the many MCDM methods, Elimination et Choice Traduisant la Realite (ELECTRE) is the most typical [45] and widely used [44]. This method was proposed by Benayoun et al. [31]. Decision-makers must quantify qualitative data, then determine the pros and cons of the alternatives using subjective evaluation [7]. In the ELECTRE method, the concordance and discordance indices are used to rank the alternatives. However, only the core solution of the alternatives can be excluded, and there is no way to know the complete order of all alternatives [39]. Therefore, Roy and Bertier [4] proposed ELECTRE II to exclude the complete order of the alternatives.
Lin et al. [21] used entropy theory to determine the weights of alternatives and combined ELECTRE II with probabilistic linguistics to solve the problem of the computing network. Liu and Wan [44] analyzed the sensitivity of ELECTRE I and ELECTRE II and used the example of location selection. Singh et al. [41] combined entropy and ELECTR II to select the material for natural fibers. They compared the results with COPRAS, TOPSIS, VIKOR, SAW, and MOORA. Reddy et al. [17] used AHP to obtain the weights of the alternatives and used ELECTRE II to find the important factors of product design. Akram et al. [20] believed that the feature of ELECTRE is to evaluate alternatives by outranking relations. Therefore, ELECTRE II and hesitant Pythagorean fuzzy are combined and applied to the electronic commerce problem. Shen et al. [14] combined ELECTRE II with probabilistic linguistic term sets to evaluate the risk of project investment. Sarwar et al. [23] combined ELECTRE II with the theory of rough numbers to find the risk factors. Zhang et al. [33] proposed double hierarchy hesitant fuzzy linguistic term set ELECTRE II for performance evaluation. Liu et al. [45] integrated Z-numbers and ELECTRE II for performance evaluation.
In the process of ranking alternatives in ELECTRE II, the concordance index only considers which alternative is better based on the criterion. The degree to which it is better is not measured. Similarly, the discordance index finds the worst alternative and average loss incurred by various alternatives. An et al. [9] pointed out that in traditional ELECTRE II, the concordance and discordance indices cannot distinguish between alternatives with similar evaluation values. Therefore, they redefined the concordance and discordance indices. Weighting factors are added to the concordance index, and refined the discordance index as the average loss of the evaluation value.
However, the unit of measure for the concordance index is the weight of the criteria, while the unit of measure for the discordance index is the evaluation value. Therefore, these two indices cannot be used for any direct comparisons or calculations. This study thus sought to improve ELECTRE II by exploring the relationship between the alternatives and the criteria to construct an accurate ranked list and the meaning of the concordance and discordance indices. This improved decision-making tool provides an objective basis for decision-making.
The remainder of this paper is organized as follows. Section 2 introduces ELECTRE II. Section 3 discusses the limitation of traditional ELECTRE II and the improvements contributed by this study. Section 4 introduces the case study. Section 5 discusses and analyzes our findings. Finally, the research draws conclusions in Section 6.
Preliminaries
Basic concepts for MCDM techniques
MCDM can be divided into two categories: multiple-objective decision-making (MODM) and multi-attribute decision-making (MADM) [15]. MODM approaches can be regarded as an extension of the operation research (OR) model, a decision process to find the best solution from the continuous decision space by algorithms. It is suitable for the case where there are infinite and continuous feasible alternatives, and only for the state where only the objective function and restrictive conditions are known, but the feasible alternatives are unknown. MADM is to establish a causal relationship evaluation model with expert experience from a limited number of feasible solutions in the discrete solution space. The best alternative is found by comparing the same assessment attribute within each alternative.
The basic element in the process of MCDM is the decision matrix. The decision matrix includes (1) all alternatives considered by the decision-makers; (2) criteria (i.e., factors affecting the choice of the decision-makers; (3) weights for each criterion to determine their relative importance; (4) evaluation values quantifying the alternatives based on the criteria.
Churchman and Ackoff [8] proposed SAW based on the utility function. The evaluations for each alternative and criteria are reasonably obtained from decision-makers or experts. Then the score of the alternative is the sum of each alternative calculated by its evaluation multiplied by the weight for each criterion. Opricovic [36] believed that no alternative that can satisfy all the criteria because there are often conflicting criteria in any given situation of decision-making. VIKOR was proposed based on the concept of compromise to deal with conflicting criteria while ranking alternatives. The feature VIKOR is that it can pursue the maximization of group utility and the minimization of individual regret at the same time. Brauers and Zavadskas [43] believed that each alternative should proposed be compared with the target. Therefore, they proposed MOORA with the concept of ratio. Zavadskas et al. [13] proposed WASPAS based on weighted product model (WPM) and weighted sum model (WSM). They thought that the performance of their proposed method could be better than WPM and WSM. Pamucar and Cirovic [11] believed that the distance between each alternative and the boundary approximation region must be calculated in order to find the best alternative appreciably. They proposed MABAC to solve the problem of investment decisions for logistics centers. Ghorabaee et al. [25] considered the average solution with every alternative. Therefore, they proposed EDAS and verified the performance of the proposed by comparing it with VIKOR, TOPSIS, SAW, and COPRAS. Yazdani et al. [24] proposed CoCoSo by combining the concepts of WASPAS, SAW, and multiplicative exponential weighting (MEW). They verified the stability of their proposed method by comparing the results with TOPSIS, VIKOR, COPRAS, WASPAS, MOORA, EDAS, and CODAS. Stevic et al. [46] proposed MARCOS by considering the aggregations of each alternative and the distances from the ideal and anti-ideal solutions. They compared their result with MABAC, SAW, ARAS, WASPAS, TOPSIS, and EDAS.
Traditional ELECTRE II
ELECTRE II ranks all feasible alternatives. This study details its procedure in the following:
After identifying the problem to be solved, we need to collect the basic information, including alternatives, criteria, weights of criteria, and the evaluation of each alternatives. A generic form of the decision matrix for MCDM is shown in Table 1, where x mn is the evaluation value associated with the m th alternative based on the n th criterion.
Decision matrix
Decision matrix
Some of the criteria in the decision matrix represent benefits, while others represent cost. Furthermore, each criterion has its own unit of measure, so they cannot be directly compared. It thus becomes necessary to normalize these evaluation values. Normalization converts the evaluation value of each criterion into a number 0 and 1. The normalization of benefit criteria is represented by Equation (1); the normalization of cost criteria is represented by Equation (2):
For both concordance and discordance indices, it is necessary to first classify the evaluation values of the alternative into one of three subsets. Under criterion C
k
, if the evaluation value of the alternative A
x
is superior to the alternative A
y
, the criterion C
k
is classified into set I+, as in Equation (3); if the evaluation value of the alternative A
x
is equal to the alternative A
y
, the criterion C
k
is classified into set I=, as in Equation (4); if the evaluation value of the alternative A
x
is inferior to the alternative A
y
, the criterion C
k
is classified into set I-, as in Equation (5).
The concordance index measures the degree of satisfaction of the decision-makers with an alternative. That is, this ratio is the sum of the weights of the satisfaction criteria to the sum of the weights of all the criteria. The weight of the satisfaction criteria includes sets I+ and I=. If when alternative A
x
is not worse than alternative A
y
, it includes in the concordance index, as described by Equation (6):
The discordance index measures the degree of dissatisfaction of the decision-makers with an alternative. Calculation of this index only includes criteria from set I-. Equation (11) is the discordance index that wants to know which criterion has the worst value of the evaluation.
Before calculating the degree of credibility, we subtract the discordance matrix from the concordance matrix, as follow in Equation (13):
Credibility is determined as follows in Equation (14):
The alternative can be ranked after the credibility degree matrix is obtained. Starting from the degree of credibility
An et al. [9] pointed out that in traditional ELECTRE II, the concordance and discordance indices cannot distinguish between alternatives with similar evaluation values. Therefore, they redefined the concordance index as follows in Equation (16):
The discordance index in traditional ELECTRE II is the lowest evaluation value in set I-. An et al. [9] refined the discordance index as the average value of set I-, as follows in Equation (17).
An et al. [9] refined the concordance and discordance indices. In the concordance index, they gave the appropriate weighting factors to distinguish alternative A x from alternative A y . In the discordance index, they used the average value to improve the worst evaluation value in the traditional ELECTRE II to represent the degree of dissatisfaction. However, this study believes that there are still insufficient considerations in the ELECTRE II so that we find and improve it. The method proposed by this study is described in the following section.
In this study, we set out to improve step 3 of the traditional ELECTRE II. The key equation for decision-makers is presented in the form of Equation (13), in which the discordance index is subtracted from the concordance index. It is in Equations (6)–(9) which the concordance index is calculated, and Equation (11) calculates the discordance index. The other steps of the algorithm remain unchanged, in order to preserve the spirit of decision-making in ELECTRE II and provide a more objective decision-making method to assist decision-makers in decision-making. The flowchart of proposed assessment framework of ELECTRE II is presented in Fig. 1. Section 3.1 introduced the formula of the purposed concordance index. Then used example 1 (examination results of students) and example 2 (the sustainability evaluation for groundwater remediation) with purposed concordance index and traditional discordance index to verify and illustrate the unreasonable parts of traditional ELECTRE II. The modified discordance index in this study is purposed in section 3.2. We used example 2 with purposed discordance and traditional concordance index to verify and illustrate that even if the formula has changed to the easier one which is purposed in this study, the rank of the alternatives will not change. It means that our formula simplified the computational burden imposed on the other methods. Since we have already demonstrated the concordance and discordance indices proposed in this study in Sections 3.1 and 3.2, the two formulas are combined in Section 3.3 to verify the ELECTRE II method proposed in this study.

The flowchart of proposed assessment framework of ELECTRE II.
According to the improved ELECTRE II proposed by An et al. [9], to calculates the concordance index of alternative A x compared to alternative A y , a binary decision is made as to whether the sum of the weights of alternative A x based on criterion C k is more than that of alternative A y based on criterion C k . The size of the difference between these values is not taken into account.
Examination results and concordance indices of two students in three subjects of differing importance
Examination results and concordance indices of two students in three subjects of differing importance
#: by weight of subject multiplied by gap between two students.
From the perspective, the performance of student B is better than that of student A. If we look at the overall high scores of student B (i.e., 90, 79, and 90), this result makes more sense intuitively.
Therefore, this study redefined the concordance index. While the equation of the index is the same as the traditional one, we define the component parts differently. As long as alternative A
x
is not worse than the alternative A
y
, we use W
I
+
, W
I
=
, and W
I
-
to calculate the concordance index, as in Equation (18):
However, rather than simply summing the weights of the criteria in the sets, we sum the weights multiplied by the difference between the evaluation values. Therefore, Equations (7)–(9) are modified as presented in Equations (19)–(21).
In set I-, because X xC k is worse than X yC k , X xC k will produce a negative value when X yC k is subtracted. Therefore, the absolute value is added to Equation (21). In the traditional and modified ELECTRE II proposed by An et al. [9], only the weights of the criteria in each set are considered and used in W I + , W I = and W I - . The method proposed by this study calculates the size of the difference between alternative A x and alternative A y . That is, the distance between alternative A x and alternative A y is multiplied by the weight of the criterion.
Decision matrix for groundwater remediation
The rankings determined by traditional ELECTRE II and the method of An et al. [9] are presented in Table 4.
Ranking of alternatives (round 1)
We then kept the same definition of the discordance index and implemented the proposed definition for the concordance index. The results are shown in Table 5. The proposed method returns a different ranking.
Ranking of alternatives (round 2)
All methods suggest P & T as the optimal choice. Therefore, in this study, the evaluation value was changed to a more extreme value under the condition that the ranking of the evaluation values of P & T and MNA remains unchanged. The evaluation values that make the evaluation of P & T has only a few points to win the evaluation of MNA, but as long as P & T is worse, it will lose more points. The modified decision matrix is shown in Table 6. The values in bold were changed for the purpose of this illustration.
Decision matrix for groundwater remediation
The results are shown in Table 7. The rankings of traditional ELECTRE II and the method proposed by An et al. [9] remain unchanged. The proposed method, however, reverses the ranking of the first and last ranked, which illustrates the profound effects of this new definition for the concordance index.
Ranking of alternatives (round 2), including proposed method
For the discordance index, traditional ELECTRE II seeks the worst evaluation value in set I- to present the degree of dissatisfaction with alternative A
i
compared to alternative A
j
. The method proposed by An et al. [9] looks for the average value in set I-. To calculate credibility, the discordance matrix is subtracted from the concordance matrix. However, the units of measure for the two indices differ, which compromises the result. The concordance matrix considers the percentage of satisfaction across the whole set, while the discordance matrix only considers the scores lose in set I-. Therefore, for the discordance index, this study considers the percentage of dissatisfaction across the whole set, Equations (17) are modified as presented in Equation (22):
In order to illustrate the advantages of the proposed definition, we fix the discordance index and implement the proposed calculation for the discordance index. The results are shown in Table 8. It can be seen that if the proposed definition for the discordance index is applied, the ranking is returned by both the traditional and modified ELECTRE II. In addition, Equation (22) simplified the computational burden imposed on the other methods.
Ranking of alternatives using proposed definition of discordance index
For the case of groundwater pollution remediation, we ranked the alternatives using Equations (19)–(21) for the concordance index and Equation (22) for the discordance index. In addition, we also used the traditional ELECTRE II, modified ELECTRE II [9], VIKOR, SAW, MOORA, WASPAS, EDAS, CoCoSo, and MARCOS to solve this problem. Table 9 is a comparison table of the ranking of the solutions.
Ranking of alternatives for groundwater pollution remediation
Ranking of alternatives for groundwater pollution remediation
This problem is solved by ten multi-criteria decision-making methods of previous scholars and the proposed method in this study. We found that the best alternative in the traditional and the modified ELECTRE II is P & T, while the best alternative in other methods and the proposed methods is MNA. P & T ranks the last in the traditional and modified ELECTREE II. Therefore, it is impossible to make an objective decision if we only discuss loss or win without exploring the gap between the alternatives.
Background description
Environmental issues are important to discuss all over the world, and energy is not only an issue in developed countries but also a popular issue in developing countries [26]. With the economic growth, Bangladesh was promoted to a developing country in 2018. Therefore, the demand for the power in Bangladesh has also increased rapidly [3, 29]. The energy is mainly divided into renewable energy and non-renewable energy. There are 86.66% of current power generation in Bangladesh is non-renewable energy, which includes gas, oil, and coal [5]. However, the Bangladeshi government has formulated a plan that hopes to realize the net-zero carbon emission policy in 2041. Therefore, Bangladesh should reevaluate all current power generation technologies and explore the factors more comprehensively that affect power generation technologies.
This study refers to the case study of Ali et al. [40] evaluating power generation technologies in Bangladesh. There are six power generation technologies including three non-renewable energy technologies (gas, oil, and coal) and three renewable energy technologies, which are clean energy technologies (hydro, wind, and solar). There are eight criteria are presents in Table 10.
Criteria for power generation technologies
Criteria for power generation technologies
We select the best alternative in this case study according to the steps in Section 2.1, but step 3 of tradition ELECTRE II is based on the method proposed in this study in Section 3.
This problem is prioritizing the best power generation technology of the six alternatives and considering eight criteria. The decision matrix is presented in Table 11.
Decision matrix for power generation technologies
Decision matrix for power generation technologies
We used Equations (2) to normalize and convert the evaluation value of each criterion into a number 0 and 1 The normalized decision matrix is shown in Table 12.
Decision matrix for power generation technologies with normalized evaluation values
According to Equations (18)–(21), concordance can be calculated, shown as follow:
According to Equation (22), discordance can be calculated, shown as follow:
According to Equations (13)–(14), discordance can be calculated, shown as follow:
In addition to the method proposed in this study, we also used the traditional ELECTRE II, modified ELECTRE II [9], VIKOR, SAW, MOORA, WASPAS, MABAC, EDAS, CoCoSo, and MARCOS to solve this problem. Table 13 is a comparison table of the ranking of the solutions.
Ranking of alternatives for power generation technologies
This problem is solved by eleven multi-criteria decision-making methods of previous scholars and the proposed method in this study. We found that the best alternative in the traditional and the modified ELECTRE II is Gas, while the best alternative in other methods and the proposed methods is Oil. Oil ranks fourth in the traditional ELECTREE II and even cannot distinguish whether it is ranked 5th or 6th place in the modified ELECTREE II. It illustrates that the traditional and modified ELECTRE II are not comprehensive enough in decision-making. Therefore, there is no way to distinguish between alternatives that are very close to each other, and it is impossible to know the real difference between the alternatives.
There are many methods of MCDM for sustainable assessment, and ELECTRE II is one of the popular methods. The ELECTRE II method compares various alternatives according to a given set of criteria to determine which alternative is superior. However, the calculation procedures for the concordance and discordance indices are flawed. Therefore, this study offers new definitions for these indices and illustrated the proposed approach with examples related to examination results, groundwater remediation, and power generation technologies. From the results of the two examples, it can be found that the results of decision-making of the proposed method are the same as those of the method proposed by scholars in recent years. It also proves that the decision-making process of traditional and modified ELECTRE II by An et al. [9] have some irrationalities. Important points are discussed in the following: In the previous methods, the concordance index considers only the sum of the weights and uses this to determine the percentage of satisfaction. It does not take into account the size of the difference between evaluation values. Therefore, this study included two factors in the calculation of the concordance index, which can be considered more objective. In traditional ELECTRE II, the discordance index uses the worst evaluation value to represent the degree of dissatisfaction. In the method proposed by An et al. [9], the average value is used. However, to calculate the degree of credibility, the discordance matrix is subtracted from the concordance matrix. The unit of measure for the concordance index is the criterion weight, while the unit of measure for the discordance index is the evaluation value. In the concordance index, the whole set is considered, while the discordance index only considers set I-. The two matrices therefore cannot be compared. Therefore, this study redefined the discordance index as the percentage of dissatisfaction to ensure the subtraction would not be distorted. In our example, this new definition does not affect the ranking returned by the traditional and modified ELECTRE II. Therefore, the proposed simplified definition of the discordance index is verified. In the discordance index, An et al. [9] improved the “most loss of evaluation value” of traditional ELECTRE II by using the “average loss of the evaluation value”. It no longer only considers the single worst performance criterion. Based on this concept, this study changed the concordance index into the total difference in the value of evaluation of each alternative to win over another alternative, while the discordance index was changed to the difference in the value of evaluation of each alternative to lose against another alternative; therefore, a more objective decision calculation index was proposed. This study proposed a calculation method to improve the concordance index and discordance index in ELECTRE II. In addition to considering the weights of the criteria, the difference between the alternatives is also considered, making the decision-making process more objective. It will not cause unreasonable sequencing of alternatives because it only considers the weights of the criteria and the evaluation value of the alternatives lost, and thus decision-makers can have a more objective basis for judgment.
Conclusions
The frameworks for sustainable decision-making have developed rapidly in recent years and applied in different fields. The most important key to decision-making is whether each step is reasonable. Therefore, the aim purpose of this study was to pinpoint the limitations of ELECTRE II to provide a more objective decision-making method. Our contributions can be divided into two aspects: theoretical and practical contributions.
(1) Theoretical contributions
More and more scholars have developed different frameworks for sustainable evaluation. Although the selection of sustainable alternatives and criteria is very important, this study emphasizes the rationality and objectivity of evaluation methods.
This study explores the significance of each step in the ELECTRE II method and proposes new definitions of the concordance and discordance indices to overcome the limitations of previous approaches. We provide examples to illustrate the superiority of the proposed approach. Regarding the calculation method of the concordance index, previous scholars did not consider the evaluation value of each alternative for the criterion of winning or losing and only considered the weight of the criterion, so the satisfaction of the alternatives could not be calculated. In addition, in the discordance index, previous scholars did not consider whether the concordance matrix and the discordance matrix were fair when making comparisons nor consider whether the unit of measure was the same. Therefore, there would be a problem of distortion in the calculation, and the degree of dissatisfaction could not be calculated. In this study, we proposed a modified method to redefine the calculation method of the concordance index and the discordance index to improve the ELECTRE II method. We illustrate the implications of the redefinition in this study through a simple example, which improves the unreasonable parts in the previous research, making the concordance index and the discordance index in ELECTRE II more accurate in decision-making results.
(2) Management Implications
There are many issues of sustainability evaluation are discussed. This study takes the selection of groundwater remediation technologies and power generation technologies as the example, which can provide decision-makers with more reasonable consideration of all alternatives and criteria, and will not ignore the alternatives with similar performance. Decision-makers rely on subjective experience to determine the best choice among a range of alternatives. This can lead to variation in the results. The improved ELECTRE II method proposed in this study provides decision-makers with a more objective basis for decision-making that is more in line with intuitive assessment.
(3) Future directions and limitations
This study proposed an improved ELECTRE II sustainability assessment framework. In terms of methodology, the proposed ELECTRE II method can be extended to other ELECTRE family methods in the future. It can also combine with other MCDM methods into a hybrid decision-making model to solve more complex problems. In terms of application, this framework can also be applied to engineering or manufacturing problems. There are many kinds of MCDM methods. The feasible alternatives that are unknown is the limitations of MODM. Oppositely, the feasible alternative, criterion, and weights are known in MADM. Therefore, if only the alternatives, criteria, and weights are known, any problem can use the proposed method in this study to solve it.
