A net-flow based approach to detect outliers in multicriteria decision problems

Abstract

Detecting outliers in multicriteria decision aid (MCDA) field is a new research direction that has not been enough explored. This paper tackles this problem by proposing a statistical approach based on PROMETHEE method net-flow. To consider the multicriteria character of the problem, the net-flow of each object is computed by applying PROMETHEE method. According to the normal distribution of the net-flow values, the standard deviation (SD) or interquartile range (IQR) statistical methods are used to detect outliers. To prove its applicability, the proposed approach is evaluated on real life problem.

Keywords

Multicriteria decision aid outlier detection PROMETHEE net-flow standard deviation method interquartile range method

1. Introduction

In data mining and knowledge discovery, outlier detection (anomaly detection) is the task aiming at identifying rare items, event or observations that diverge from the majority of the data. Hawkins [1] defines an outlier as an observation that deviates so much from other observations as to arouse suspicion that it was generated by a different mechanism. The presence of such outliers or anomalous items will translate to some kind of problems such as structural defect, medical problems or bank fraud.

Outliers detection methods have been applied in several domains such as network intrusion [2, 3], credit card fraud [4], email spam [5] and customer activity monitoring [6]. Many algorithms have been developed to find outliers. These algorithms can be classified as either distance-based [7, 8, 9, 10], density-based [11], clustering [12], depth-based [13] or distribution-based [14]. Nevertheless, detecting outliers in multicriteria decision aid (MCDA) field has not been enough explored.

Several number of MCDA methods have been developed and applied in a large number of domains [15, 16, 17, 18, 19, 20]. Most of these methods are based on subjective parameters which represent decision makers’ preferences (like preference, indifference and veto thresholds). The robustness of these methods has not been sufficiently tackled [21]. For example, changes in the values of one of these parameters could affect the results of these methods. For this reason, it is important to study the robustness of these methods and for consequence it is also important to study the presence of outliers. This new research direction has not been enough tackled in the literature. The main concern of this research direction is how to integrate the multicriteria character of the problem (defined by conflicting criteria and subjective parameters) in the process of identifying outliers because the outlier detection methods cannot be applied directly to the initial data of a multicriteria decision problem.

Table 1
The data of the example of the hydroelectric facilities

Criterion	Min/Max	$a_{1}$	$a_{2}$	$a_{3}$	$a_{4}$	$a_{5}$	$a_{6}$	Criterion type	Parameters
$g_{1}$	Min	80	65	83	40	52	94	II	$I=10$
$g_{2}$	Max	90	58	60	80	72	96	III	$m=30$
$g_{3}$	Min	600	200	400	1000	600	700	V	$s=50;r=450$
$g_{4}$	Min	5.4	9.7	7.2	7.5	2.0	3.6	IV	$q=1;p=5$
$g_{5}$	Min	8	1	4	7	3	5	I	–
$g_{6}$	Max	5	1	7	10	8	6	VI	$\sigma=5$

In the literature, to the best of our knowledge, two research works have focused on this problem. The first one has been developed by De Smet et al. [22] in which the authors proposed a distance-based model to detect multicriteria outliers. To consider the multicriteria character of the problem, the authors extended the multicriteria distance proposed by Smet and Guzman [23] to different samplings of the set of objects. Outliers were detected by the identification of bi-modal distributions of the distance values. In the second one, Rouba and Nait Bahloul [24] proposed an outlier detection method based on binary outranking relations and Local Outlier Factor (LOF) algorithm [19]. The outlier is detected by applying LOF algorithm on the distribution of the outranking relations generated by a multicriteria outranking method. The multicriteria nature of the problem has been taken into account by the use of outranking relations.

In this paper, we introduce a statistical based approach to detect outliers in multicriteria decision aid problem. The multicriteria nature of the problem is considered using PROMETHEE net-flow [18]. The idea is to test the normality distribution of the net-flow values of the objects, if the net-flow values follow the normal distribution, the standard deviation (SD) method is used [25]. If the values of the net-flow do not follow the normal distribution, the interquartile range method [26] is used.

The rest of the paper is organized as follows. In Section 2, we present some basic concepts about MCDA field and PROMETHEE method net-flow. Statistical methods for outlier detection are presented in Section 3. The algorithm of the proposed approach is presented in Section 4. The proposed approach is evaluated on a real life problem in Section 5 before concluding with some general remarks and future works.

2. Multicriteria decision aid

In MCDA, the problem is defined by a set of actions noted $A=\{a_{1},a_{2},\ldots,a_{n}\}$ evaluated according to a set of criteria $G=\{g_{1},g_{2},\ldots,g_{m}\}$ . These actions can be potential decisions, candidates, projects, feasible solutions, items, units, alternatives, etc.

In the multicriteria context, actions are compared by defining a preference structure. A decision maker (DM) who compares two actions $a_{i},a_{j}$ of $A$ may expresses a preference ( $P$ ), an indifference ( $I$ ) or an incomparability ( $R$ ) statement between these two actions [27]. These relations are built from the comparison of every couple of actions with the respect of the set of criteria $G$ using a given multicriteria outranking method. Several multicriteria method have been developed such as MAUT [28], UTA [29], AHP [30], ELECTRE [31] and PROMETHEE [18].

PROMETHEE is an outranking multicriteria method developed by Brans et al, in 1982. It aims to rank the actions from the best to the worst. According to the decision maker perspective, PROMETHEE can provide either a partial or a complete ranking. For this purpose the outranking flows for each action are computed.

The basic data of a multicriteria problem consist of an evaluation table (see Table 1). In the PROMETHEE methodology, each criterion is characterized by subjective parameters defined by the decision maker. For example, let’s consider a multicriteria problem defined by six actions evaluated on six criteria. The data of the example are derived from a real example. The space of actions $A$ consists of 6 projects of hydroelectric facilities evaluated over the following criteria [18]:

•
$g_{1}$ : The number of staff required to operate the facility.
•
$g_{2}$ : The electrical power in Mega Watt.
•
$g_{3}$ : The construction cost expressed in millions of Dollars.
•
$g_{4}$ : The annual maintenance costs in millions of Dollars.
•
$g_{5}$ : The number of villages to be evacuated to allow the water reservoir.
•
$g_{6}$ : A measure of degree of security.

For each criterion $g_{k}$ , the DM evaluates the preference of an action $a_{i}$ over an action $a_{j}$ by measuring the difference of their evaluation on $g_{k}$ .

$\displaystyle d_{k}(a_{i},a_{j})=g_{k}(a_{i})-g_{k}(a_{j})$ (1)

This pairwise comparison allows the DM to quantify how the action $a_{i}$ performs on $g_{k}$ compared to the action $a_{j}$ . Then, a preference function $P_{k}$ is used to transform this value into a preference degree. Such a function should be defined separately for each criterion. Each preference function takes its values between 0 and 1. Decision maker must choose a preference function associated with each criterion. Six different types are available. For example, the preference function for a criterion type IV is computed as follows (A detailed description of all the types is available in [18]).

$\displaystyle P_{k}(a_{i},a_{j})=\begin{cases}1&\text{if }d_{k}(a_{i},a_{j})>q% +p\\ {\displaystyle\frac{1}{2}}&\text{if }q<d_{k}(a_{i},a_{j})\leqslant q+p\\ 0&\text{if }d_{k}(a_{i},a_{j})\leqslant q\end{cases}$ (2)

To quantify the global preference of $a_{i}$ over $a_{j}$ , the preference index $\pi(a_{i},a_{j})$ is computed as the average of all the unicriterion preference degree $P_{k}(a_{i},a_{j})$ .

$\displaystyle\pi(a_{i},a_{j})=\frac{1}{m}\sum^{m}_{k=1}{{P}_{k}}(a_{i},a_{j})$ (3)

In the last step of the PROMETHEE I methods, the outranking flows of each action are computed. The outgoing $\o^{+}(a_{i})$ flow allows the DM to quantify how an action $a_{i}$ is preferred to all the remaining actions of the set A. The incoming $\o^{-}(a_{i})$ flow quantify how all the actions of A are preferred to $a_{i}$ .

$\displaystyle{\varphi}^{+}(a_{i})=\sum_{a_{j}\in A}{\pi(a_{i},a_{j})}$ (4) $\displaystyle{\varphi}^{-}(a_{i})=\sum_{a_{j}\in A}{\pi(a_{j},a_{i})}$ (5)

The outgoing and incoming flows could be combined into the outranking net-flow ${\o}(a_{i})$ which is used in PROMETHEE II.

$\displaystyle\varphi(a_{i})={\varphi}^{+}(a_{i})-{\varphi}^{-}(a_{i})$ (6)

Based on the outgoing and incoming flow scores, the PROMETHEE I method generates a partial ranking of the actions. In PROMETHEE II, a complete ranking is generated based on the net-flow values of the actions. In Table 2 are summarized the rank and the outranking flows for each action of the considered example.

Table 2
Rank and outranking flows of the example

Action Rank ${\varphi}^{+}(a_{i})$ ${\varphi}^{-}(a_{i})$ $\varphi(a_{i})$

$a_{5}$ 1 0.4648 0.1934 0.2714

$a_{2}$ 2 0.4911 0.2922 0.1989

$a_{6}$ 3 0.3010 0.3596 $-$ 0.0586

$a_{1}$ 4 0.2363 0.3575 $-$ 0.1212

$a_{3}$ 5 0.2375 0.3684 $-$ 0.1308

$a_{4}$ 6 0.2722 0.4318 $-$ 0.1595

As stated in the introduction, the proposed approach aims at applying statistical methods on the distribution of the net-flow values in order to identify outliers. In the next section, we present some statistical methods for outliers detection.
3. Statistical methods for outliers detection

Action	Rank	${\varphi}^{+}(a_{i})$	${\varphi}^{-}(a_{i})$	$\varphi(a_{i})$
$a_{5}$	1	0.4648	0.1934	0.2714
$a_{2}$	2	0.4911	0.2922	0.1989
$a_{6}$	3	0.3010	0.3596	$-$ 0.0586
$a_{1}$	4	0.2363	0.3575	$-$ 0.1212
$a_{3}$	5	0.2375	0.3684	$-$ 0.1308
$a_{4}$	6	0.2722	0.4318	$-$ 0.1595

There are several ways to detect outliers in statistics. We present here the most frequently used

3.1 Parametric methods

In order to use a parametric method, we must first ensure that the data follow a normal distribution. To perform this, the use of a normality test such as the Kolmogorov-Smirnov test [32] is essential. Once assured that the data follow the normal distribution, the standard deviation (SD) [25] method can be used. The observations two standard deviations (SD) away from the mean (M) may be considered as an outlier i.e., are identified as outliers values which are greater than $M+2*\text{SD}$ , and those which are less than $M-2*\text{SD}$ .

3.2 Non-parametric methods

In most cases, the data obtained does not follow the normal distribution. It is therefore necessary to use a non-parametric method called the interquartile range (IQR) method [20]. This method is based on the lower quartile $Q_{1}$ (which represents the 25th percentile of the data), and the upper quartile $Q_{3}$ (which represents the 75th percentile of the data). The inter-quartile range (IQR) is then defined as the interval between $Q_{1}$ and $Q_{3}$ . Are considered as outliers, observations which are greater than $Q_{3}+[1.5*\text{IQR}]$ and those which are less than $Q_{1}-[1.5*\text{IQR}]$ .

4. The proposed approach

The proposed approach is inspired from statistical outlier detection approaches. In fact, the idea is to test if the net-flow values follow the normal distribution. To perform this, the well-konown Kolmogorov-Smirnov [15] test of normality is used. The result of this test is a probability value called $p$ -value. The net-flow values are considered as following the normal distribution if $p$ -value is greater than 0.05. In this case, the SD method is used. If the values of the net-flow do not follow the normal distribution, the interquartile range method is used. In the following, the proposed algorithm is presented.

Outlier_Detection_MCDA

A: $\{a_{1},a_{2},\ldots,a_{n}\}$ a set of $n$ actions. G: $\{g_{1},g_{2},\ldots,g_{m}\}$ a set of $m$ criteria.

O: a set of actions considered as outliers.

Begin

O $\leftarrow$ {}

$\text{PHI}=\{\text{phi}_{1},\text{phi}_{2},\ldots,\text{phi}_{n}\}\leftarrow$ PROMETHEEII(A, G)

M $\leftarrow$ mean (PHI)

SD $\leftarrow$ Standard_Deviation(PHI)

$p$ -value $\leftarrow$ Kolmogorov-Smirnov_test(PHI, M, SD)

( $p$ -value $>$ 0.05) $d_{1}\leftarrow M+2*\text{SD}$

$d_{2}\leftarrow M-2*\text{SD}$

phi ${}_{k}$ in PHI (phi ${}_{k}>d_{1}$ ) ou (phi ${}_{k}<d_{2}$ ) O $\leftarrow$ OU $\{a_{k}\}$

$Q_{1}\leftarrow$ quartile(PHI, 1)

$Q_{3}\leftarrow$ quartile(PHI, 3)

$v_{1}\leftarrow Q_{3}+(1,5*(Q_{3}-Q_{1}))$

$v_{2}\leftarrow Q_{1}-(1,5*(Q_{3}-Q_{1}))$

phi ${}_{k}$ in PHI

(phi ${}_{k}>v_{1}$ ) ou (phi ${}_{k}<v_{2}$ )

O $\leftarrow$ OU $\{a_{k}\}$

Let’s execute the proposed algorithm on the example presented in the Section 2. First, we test the normality of the net-flow values (Table 2) using ks.test function of the stats package in R language. The test generates a $p$ -value of 0.5979 which means that the net-flow values follow the normal distribution. In Fig. 1 are presented the histogram and the boxplot of the net-flow values.

Figure 1.

Data representation of the example without the artificial action.

To test the ability of the proposed algorithm to detect outliers, an artificial action ( $a_{7}$ ) has been added to the example. The evaluation of the artificial action over the criteria is presented on Table 3. It is important to notice that the artificial action is characterized by the best evaluation over all the criteria which makes it possible to be considered as a “super outlier”. The new values of the net-flow are presented in Table 4.

Table 3

Evaluation table of the artificial action

	$g_{1}$	$g_{2}$	$g_{3}$	$g_{4}$	$g_{5}$	$g_{6}$
$a_{7}$	36	100	150	1, 5	1	1

Table 4

The new values of the net-flows

Action	$\varphi(a_{i})$
$a_{7}$	0.6993
$a_{5}$	0.0995
$a_{2}$	0.0824
$a_{6}$	$-$ 0.1607
$a_{1}$	$-$ 0.2151
$a_{3}$	$-$ 0.2483
$a_{4}$	$-$ 0.2571

In Fig. 2 are presented the histogram and the boxplot of the new net-flow values. It is easy to see that the artificial action is outside the boxplot which means that it can be considered as an outlier. The presence of this action can be easily observed on the histogram. To confirm this observation, the normality of the new values has been tested. The resulting $p$ -value is 0.6803 which means the new net-flows values still follow the normal distribution. According to the algorithm, the outlier is the action for which the net-flow value is greater than $M+2*\text{SD}=\textit{0.68584}$ or less then $M-2*\text{SD}=-\textit{0.68584}$ . It easy to show that the only action that satisfies this condition is the artificial action $a_{7}$ .

Figure 2.

Data representation of the example with the artificial action.

Table 5

Criterion list associated the regional planning problem

Criterion	Type	Associated factors	Subjective parameters
			Preference threshold	Indifference threshold	Veto threshold	weight
Harm	Natural	Air pollutions, odors	1	0.3	2	0.14
Noise	Social	Motorways, railways	0.6	0.3	0.6	0.17
Impacts	Social	Underground water, sectoral plan	1	0	2	0.17
Geographical and natural risks	Natural	Landslides, flood, seism, fire	2	0	4	0.17
Equipment	Economic	Distance to equipment: Gas, electricity, water, access road	130	55	500	0.2
Accessibility	Social	Distance to localities	15	10	0	0.2
Climate	Natural	Sun, fog, temperature, dampness	0.6	0.3	1.3	0.04

Table 6

Net-flow values of the first experimentation

Action	Rank	$\varphi(a_{i})$
action259	1	0.4192
action337	2	0.4109
action338	3	0.4073
action260	4	0.3858
action135	5	0.3816
action132	6	0.3614
action595	7	0.3613
action340	8	0.357
action258	9	0.3468
action134	10	0.3388
action629	567	$-$ 0.2571

Table 7

Data of the second experimentation

Action	Rank	$\varphi(a_{i})$
action259	1	0.4192
action337	2	0.4109
action338	3	0.4073
action260	4	0.3858
action135	5	0.3816
action132	6	0.3614
action595	7	0.3613
action340	8	0.357
action258	9	0.3468
action134	10	0.3388
action491	11	0.3325
action91	200	0.0897
action598	600	$-$ 0.2447

5. Experimentation

In this section, the proposed approach is experimented on a regional planning problem. It is an MCDA problem that concerns the selection of propitious sites in order to construct buildings. The aim is to search a better surface satisfying certain criteria. The problem has been treated by Joerin [16] and Rouba and Nait Bahloul [33]. The area of study is in the canton of Vaud, to approximately 15 km in the north of Lausanne. A set of 650 geographical zones has been chosen. A set of seven criteria covering natural, social and economic factors have been considered to describe each geographical zone. For each criterion, a list of subjective parameters used by the decision maker is presented (Table 5).

Firstly, PROMETHEE II method has been executed on the data of the problem in order to compute the net-flow of each action (the net-flow value of each action is presented in appendix A). Then, several experimentations have been conducted to test the proposed approach.

We considered, in the first experimentation, a sample of the 10 best ranked actions. This sample followed the normal distribution with a $p$ -value of 0.2449. We added to the sample the action “action629” ranked 567th (see Table 6) and we tested the normality again. We found that the new sample did not follow the normal distribution because the $p$ -value was 1.267e-06. We computed $Q3+[1.5*\text{IQR}]=$ 0.463525 and $Q1-[1.5*\text{IQR}]=$ 0.284925. Thus, it was easy to verify that the net-flow of the “action629” was less then 0.284925 which means that this actions represents an outlier for the sample of the 10 best ranked actions. The net-flow histogram and boxplot of the sample are presented in Fig. 3.

Figure 3.

Data representation of the first experimentation. (a) without “action629”, (b) with “action629”.

To test the sensitivity of the proposed approach, we took, in the second experimentation, a sample of the 10 best ranked actions for which we added 3 other actions (Table 7). The first one is ranked 11th (“action491”), the second one (“action91”) is ranked 200th and the third one is ranked 600th (“action598”). The normality test was negative because the $p$ -value of the ks.test was 1.659e-06. The values of v1 and v2 were respectively 0.4563 and 0.2683. For consequence, the actions “action91” and “action598” were detected as outliers. The action “action491” was not detected because it is close to the 10 best ranked actions. This result is confirmed graphically in Fig. 4.

Figure 4.

Data representation of the second experimentation.

Figure 5.

Data representation of the third experimentation. (a) before adding the four actions and (b) after adding the four actions.

In the third experimentation, the ability to detect multiple outliers has been evaluated. We considered a larger sample composed of 50 actions ranked from 301th to 350th. Then, we added to this sample a set of 4 actions ranked respectively 9th, 10th, 649th and 650th. The normality test was negative with a $p$ -value less than 2.2e-16. The proposed algorithm detected the four added actions as outliers. The result is confirmed graphically in Fig. 5.

In the last experimentation, the set of all the actions has been considered. The normality test was positive with a $p$ -value of 0.2099. Then, we added two artificial actions with extreme values for each criterion (see Table 8), so that they are ranked the best one and lowest one respectively. With the new added actions, the normality test was negative with a $p$ -value of 0.001844 and the algorithm has detected these two actions as outliers which proves the effectiveness of the proposed algorithm. Figure 6 represents graphically the result of the last experimentation before.

Table 8

Evaluation table of the two added artificial actions

	g1	g2	g3	g4	g5	g6
art1	0	0	0	9	2400	34
art2	1	1	6	1	38	1

Figure 6.

Data representation of the last experimentation. (a) before adding artificial actions and (b) after adding the artificial actions.

6. Conclusion

This paper focuses on the identification of outliers in multicriteria decision aid field, which is a new research direction that has not been enough studied in the literature. The main concern in this research direction is how to take into account the multicriteria character of the problem while detecting outliers. To solve this problem, we proposed an approach that uses the net-flow values, as information provided by the multicriteria PROMETHEE method, to detect outliers. The normal distribution of net-flow values is then verified and outliers are detected using either the standard deviation (SD) or interquartile range (IQR) method. The proposed approach has been experimented on a regional planning problem.

Compared to the previous works in the field, the proposed approach has two main advantages.

The proposed approach is characterized its ability to detect multiple outliers which is not the case for the other approaches.

The proposed approach does not require any input parameters, which increases the stability of its results. Unlike the other approaches, where a tuning phase has to be addressed to find the optimal values for the parameters (see Table B.1).

However, the main disadvantage of our approach is related to the normality test. In fact, Kolmogorov-Smirnov normality test (ks-test) is very sensitive to the presence of ties. The presence of indifferent alternatives (alternatives which have the same value of the net-flow) can lead to a lack of normality. This problem will be investigated in the future. Furthermore, we are investigating the experimentation of the proposed approach on other problems to confirm its coherence. The experimentation of other statistical methods (like z-score, Median Absolute Deviation (MAD), Median Rule, (and studying their impacts on the result of the proposed approach is another interesting issue. Finally, the use of multicriteria clustering approaches seems also a promising issue to identify and detect multicriteria outliers.

Footnotes

Appendix A

Table A1

Alternatives’ rank form 1st to 100th

Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$
action259	1	0.4192	action54	26	0.2843	action128	51	0.2264	action370	76	0.2095
action337	2	0.4109	action198	27	0.2789	action191	52	0.2264	action544	77	0.2093
action338	3	0.4073	action650	28	0.2749	action130	53	0.2261	action103	78	0.2093
action260	4	0.3858	action66	29	0.2660	action547	54	0.2258	action126	79	0.2053
action135	5	0.3816	action104	30	0.2604	action620	55	0.2255	action272	80	0.2035
action132	6	0.3614	action43	31	0.2599	action362	56	0.2248	action399	81	0.2021
action595	7	0.3613	action68	32	0.2587	action612	57	0.2228	action400	82	0.2020
action340	8	0.3570	action29	33	0.2554	action586	58	0.2195	action247	83	0.2010
action258	9	0.3468	action197	34	0.2519	action548	59	0.2194	action58	84	0.1997
action134	10	0.3388	action133	35	0.2506	action423	60	0.2192	action494	85	0.1993
action491	11	0.3325	action118	36	0.2498	action122	61	0.2187	action341	86	0.1973
action327	12	0.3255	action361	37	0.2497	action584	62	0.2183	action249	87	0.1963
action342	13	0.3254	action69	38	0.2464	action649	63	0.2181	action535	88	0.1949
action47	14	0.3249	action131	39	0.2464	action38	64	0.2168	action398	89	0.1920
action635	15	0.3248	action510	40	0.2428	action539	65	0.2165	action594	90	0.1888
action71	16	0.3156	action328	41	0.2409	action585	66	0.2157	action433	91	0.1885
action299	17	0.3147	action284	42	0.2390	action65	67	0.2150	action45	92	0.1834
action577	18	0.3139	action28	43	0.2382	action542	68	0.2144	action335	93	0.1823
action125	19	0.3083	action568	44	0.2359	action329	69	0.2144	action451	94	0.1813
action199	20	0.3049	action397	45	0.2329	action25	70	0.2133	action281	95	0.1811
action67	21	0.3049	action129	46	0.2313	action545	71	0.2132	action64	96	0.1797
action46	22	0.2959	action339	47	0.2302	action540	72	0.2128	action602	97	0.1795
action520	23	0.2947	action546	48	0.2285	action51	73	0.2127	action391	98	0.1792
action70	24	0.2944	action543	49	0.2283	action541	74	0.2107	action406	99	0.1788
action257	25	0.2888	action549	50	0.2269	action127	75	0.2102	action402	100	0.1774

In Tables below, are presented the net-flow values of the actions of the regional planning problem used in the experimentation section.

Table A2

Alternatives’ rank form 101st to 200th

Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$
action123	101	0.1758	action297	126	0.1517	action578	151	0.1332	action619	176	0.1120
action324	102	0.1758	action121	127	0.1515	action372	152	0.1322	action533	177	0.1117
action101	103	0.1748	action519	128	0.1511	action607	153	0.1313	action323	178	0.1090
action330	104	0.1746	action512	129	0.1510	action447	154	0.1299	action603	179	0.1068
action120	105	0.1738	action41	130	0.1497	action334	155	0.1295	action592	180	0.1066
action392	106	0.1735	action61	131	0.1488	action107	156	0.1286	action188	181	0.1065
action407	107	0.1718	action369	132	0.1475	action171	157	0.1281	action162	182	0.1052
action424	108	0.1699	action27	133	0.1471	action282	158	0.1272	action22	183	0.1051
action124	109	0.1697	action405	134	0.1465	action493	159	0.1272	action375	184	0.1048
action593	110	0.1690	action513	135	0.1463	action511	160	0.1270	action280	185	0.1020
action617	111	0.1665	action194	136	0.1454	action492	161	0.1270	action377	186	0.1016
action63	112	0.1649	action298	137	0.1453	action532	162	0.1252	action117	187	0.1011
action621	113	0.1648	action534	138	0.1451	action82	163	0.1248	action57	188	0.1007
action62	114	0.1645	action193	139	0.1446	action448	164	0.1248	action106	189	0.0997
action401	115	0.1637	action403	140	0.1430	action283	165	0.1239	action616	190	0.0982
action363	116	0.1623	action428	141	0.1405	action192	166	0.1237	action116	191	0.0977
action591	117	0.1610	action427	142	0.1404	action371	167	0.1228	action40	192	0.0960
action196	118	0.1594	action404	143	0.1403	action343	168	0.1201	action55	193	0.0959
action273	119	0.1593	action579	144	0.1364	action59	169	0.1186	action189	194	0.0955
action449	120	0.1572	action168	145	0.1363	action373	170	0.1158	action618	195	0.0954
action186	121	0.1570	action119	146	0.1361	action326	171	0.1148	action56	196	0.0952
action435	122	0.1549	action270	147	0.1360	action609	172	0.1142	action271	197	0.0942
action44	123	0.1535	action518	148	0.1359	action42	173	0.1142	action648	198	0.0903
action195	124	0.1522	action60	149	0.1356	action524	174	0.1136	action318	199	0.0897
action465	125	0.1521	action18	150	0.1347	action85	175	0.1133	action91	200	0.0896

Table A3

Alternatives’ rank form 201st to 300th

Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$
action263	201	0.0888	action99	226	0.0676	action590	251	0.0518	action220	276	0.0290
action26	202	0.0887	action244	227	0.0665	action464	252	0.0509	action112	277	0.0286
action248	203	0.0885	action589	228	0.0659	action37	253	0.0488	action625	278	0.0275
action190	204	0.0884	action634	229	0.0654	action374	254	0.0484	action454	279	0.0273
action39	205	0.0878	action255	230	0.0647	action242	255	0.0482	action445	280	0.0263
action376	206	0.0871	action294	231	0.0647	action425	256	0.0462	action608	281	0.0247
action115	207	0.0861	action243	232	0.0645	action380	257	0.0443	action631	282	0.0246
action246	208	0.0839	action23	233	0.0630	action489	258	0.0443	action20	283	0.0241
action331	209	0.0838	action615	234	0.0616	action254	259	0.0434	action110	284	0.0235
action325	210	0.0830	action113	235	0.0612	action241	260	0.0433	action185	285	0.0225
action279	211	0.0804	action53	236	0.0608	action84	261	0.0414	action83	286	0.0217
action49	212	0.0799	action613	237	0.0597	action310	262	0.0414	action346	287	0.0214
action365	213	0.0796	action456	238	0.0596	action488	263	0.0395	action486	288	0.0209
action98	214	0.0781	action446	239	0.0580	action21	264	0.0382	action111	289	0.0203
action102	215	0.0773	action167	240	0.0565	action336	265	0.0379	action463	290	0.0200
action149	216	0.0758	action487	241	0.0561	action253	266	0.0371	action576	291	0.0190
action256	217	0.0752	action187	242	0.0554	action100	267	0.0368	action166	292	0.0189
action296	218	0.0747	action434	243	0.0551	action180	268	0.0365	action236	293	0.0188
action408	219	0.0746	action24	244	0.0550	action633	269	0.0358	action348	294	0.0181
action523	220	0.0714	action431	245	0.0548	action484	270	0.0347	action455	295	0.0181
action333	221	0.0713	action378	246	0.0539	action345	271	0.0343	action165	296	0.0180
action642	222	0.0697	action430	247	0.0532	action614	272	0.0333	action347	297	0.0177
action114	223	0.0695	action632	248	0.0532	action575	273	0.0331	action410	298	0.0171
action490	224	0.0690	action174	249	0.0532	action344	274	0.0326	action356	299	0.0158
action245	225	0.0685	action238	250	0.0523	action379	275	0.0302	action239	300	0.0148

Table A4

Alternatives’ rank form 301st to 400th

Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$
action628	301	0.0147	action385	326	$-$ 0.004	action384	351	$-$ 0.018	action567	376	$-$ 0.033
action52	302	0.0135	action364	327	$-$ 0.005	action182	352	$-$ 0.018	action560	377	$-$ 0.034
action351	303	0.0133	action409	328	$-$ 0.006	action81	353	$-$ 0.019	action482	378	$-$ 0.034
action383	304	0.0127	action411	329	$-$ 0.006	action252	354	$-$ 0.019	action95	379	$-$ 0.034
action485	305	0.0125	action278	330	$-$ 0.006	action509	355	$-$ 0.019	action96	380	$-$ 0.034
action19	306	0.0108	action50	331	$-$ 0.007	action517	356	$-$ 0.020	action559	381	$-$ 0.035
action462	307	0.0107	action352	332	$-$ 0.007	action4	357	$-$ 0.020	action515	382	$-$ 0.035
action36	308	0.0090	action514	333	$-$ 0.009	action251	358	$-$ 0.021	action563	383	$-$ 0.035
action627	309	0.0082	action17	334	$-$ 0.009	action508	359	$-$ 0.023	action558	384	$-$ 0.035
action350	310	0.0072	action163	335	$-$ 0.010	action16	360	$-$ 0.024	action606	385	$-$ 0.036
action240	311	0.0070	action237	336	$-$ 0.011	action268	361	$-$ 0.025	action557	386	$-$ 0.036
action349	312	0.0057	action183	337	$-$ 0.012	action538	362	$-$ 0.025	action537	387	$-$ 0.036
action97	313	0.0054	action353	338	$-$ 0.013	action15	363	$-$ 0.026	action181	388	$-$ 0.036
action184	314	0.0050	action386	339	$-$ 0.013	action48	364	$-$ 0.026	action461	389	$-$ 0.036
action516	315	0.0022	action569	340	$-$ 0.014	action158	365	$-$ 0.027	action536	390	$-$ 0.037
action588	316	0.0020	action354	341	$-$ 0.014	action429	366	$-$ 0.027	action160	391	$-$ 0.037
action332	317	0.0013	action412	342	$-$ 0.015	action143	367	$-$ 0.028	action555	392	$-$ 0.037
action250	318	$-$ 6E-04	action570	343	$-$ 0.016	action312	368	$-$ 0.030	action235	393	$-$ 0.038
action159	319	$-$ 8E-04	action355	344	$-$ 0.017	action566	369	$-$ 0.031	action14	394	$-$ 0.038
action164	320	$-$ 0.001	action426	345	$-$ 0.017	action309	370	$-$ 0.031	action34	395	$-$ 0.038
action381	321	$-$ 0.002	action277	346	$-$ 0.018	action565	371	$-$ 0.031	action561	396	$-$ 0.039
action109	322	$-$ 0.002	action522	347	$-$ 0.018	action564	372	$-$ 0.032	action553	397	$-$ 0.039
action319	323	$-$ 0.003	action172	348	$-$ 0.018	action562	373	$-$ 0.033	action142	398	$-$ 0.040
action641	324	$-$ 0.003	action382	349	$-$ 0.018	action276	374	$-$ 0.033	action275	399	$-$ 0.041
action393	325	$-$ 0.003	action450	350	$-$ 0.018	action483	375	$-$ 0.033	action141	400	$-$ 0.041

Table A5

Alternatives’ rank form 401st to 500th

Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$
action161	401	$-$ 0.041	action611	426	$-$ 0.053	action157	451	$-$ 0.077	action230	476	$-$ 0.099
action583	402	$-$ 0.042	action108	427	$-$ 0.056	action320	452	$-$ 0.077	action630	477	$-$ 0.106
action274	403	$-$ 0.043	action531	428	$-$ 0.057	action366	453	$-$ 0.079	action92	478	$-$ 0.106
action556	404	$-$ 0.043	action604	429	$-$ 0.057	action285	454	$-$ 0.080	action437	479	$-$ 0.107
action605	405	$-$ 0.044	action234	430	$-$ 0.059	action530	455	$-$ 0.081	action504	480	$-$ 0.109
action322	406	$-$ 0.044	action521	431	$-$ 0.060	action269	456	$-$ 0.081	action395	481	$-$ 0.112
action35	407	$-$ 0.044	action205	432	$-$ 0.060	action93	457	$-$ 0.081	action477	482	$-$ 0.112
action295	408	$-$ 0.045	action457	433	$-$ 0.061	action506	458	$-$ 0.082	action290	483	$-$ 0.113
action554	409	$-$ 0.045	action394	434	$-$ 0.061	action11	459	$-$ 0.082	action218	484	$-$ 0.114
action321	410	$-$ 0.046	action178	435	$-$ 0.062	action529	460	$-$ 0.083	action416	485	$-$ 0.114
action552	411	$-$ 0.047	action480	436	$-$ 0.063	action479	461	$-$ 0.083	action438	486	$-$ 0.114
action551	412	$-$ 0.047	action458	437	$-$ 0.065	action177	462	$-$ 0.084	action8	487	$-$ 0.115
action460	413	$-$ 0.047	action459	438	$-$ 0.065	action221	463	$-$ 0.084	action229	488	$-$ 0.116
action550	414	$-$ 0.047	action94	439	$-$ 0.066	action610	464	$-$ 0.087	action156	489	$-$ 0.117
action582	415	$-$ 0.048	action233	440	$-$ 0.066	action79	465	$-$ 0.087	action478	490	$-$ 0.117
action292	416	$-$ 0.048	action33	441	$-$ 0.069	action267	466	$-$ 0.088	action32	491	$-$ 0.118
action308	417	$-$ 0.049	action2	442	$-$ 0.069	action266	467	$-$ 0.089	action288	492	$-$ 0.119
action13	418	$-$ 0.049	action223	443	$-$ 0.071	action9	468	$-$ 0.090	action289	493	$-$ 0.120
action140	419	$-$ 0.049	action12	444	$-$ 0.072	action231	469	$-$ 0.092	action155	494	$-$ 0.120
action436	420	$-$ 0.050	action505	445	$-$ 0.074	action10	470	$-$ 0.093	action646	495	$-$ 0.122
action601	421	$-$ 0.051	action481	446	$-$ 0.075	action105	471	$-$ 0.094	action313	496	$-$ 0.122
action179	422	$-$ 0.051	action387	447	$-$ 0.075	action415	472	$-$ 0.096	action219	497	$-$ 0.125
action213	423	$-$ 0.052	action413	448	$-$ 0.075	action78	473	$-$ 0.096	action647	498	$-$ 0.127
action80	424	$-$ 0.052	action232	449	$-$ 0.076	action291	474	$-$ 0.096	action7	499	$-$ 0.128
action293	425	$-$ 0.053	action432	450	$-$ 0.076	action414	475	$-$ 0.099	action572	500	$-$ 0.129

Table A6

Alternatives’ rank form 501st to 600th

Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$
action77	501	$-$ 0.130	action644	526	$-$ 0.160	action573	551	$-$ 0.179	action314	576	$-$ 0.212
action73	502	$-$ 0.131	action645	527	$-$ 0.160	action265	552	$-$ 0.180	action388	577	$-$ 0.212
action217	503	$-$ 0.131	action419	528	$-$ 0.161	action441	553	$-$ 0.181	action89	578	$-$ 0.214
action228	504	$-$ 0.131	action317	529	$-$ 0.161	action286	554	$-$ 0.182	action571	579	$-$ 0.217
action31	505	$-$ 0.132	action507	530	$-$ 0.165	action624	555	$-$ 0.185	action150	580	$-$ 0.220
action30	506	$-$ 0.132	action173	531	$-$ 0.165	action170	556	$-$ 0.187	action211	581	$-$ 0.221
action574	507	$-$ 0.134	action439	532	$-$ 0.167	action1	557	$-$ 0.188	action224	582	$-$ 0.223
action396	508	$-$ 0.137	action501	533	$-$ 0.167	action599	558	$-$ 0.188	action87	583	$-$ 0.223
action176	509	$-$ 0.138	action581	534	$-$ 0.168	action139	559	$-$ 0.190	action444	584	$-$ 0.226
action226	510	$-$ 0.138	action368	535	$-$ 0.169	action169	560	$-$ 0.190	action86	585	$-$ 0.226
action418	511	$-$ 0.140	action154	536	$-$ 0.170	action264	561	$-$ 0.191	action74	586	$-$ 0.227
action502	512	$-$ 0.140	action421	537	$-$ 0.171	action151	562	$-$ 0.192	action358	587	$-$ 0.228
action367	513	$-$ 0.141	action475	538	$-$ 0.171	action214	563	$-$ 0.193	action305	588	$-$ 0.228
action227	514	$-$ 0.142	action76	539	$-$ 0.172	action316	564	$-$ 0.193	action145	589	$-$ 0.231
action175	515	$-$ 0.144	action3	540	$-$ 0.172	action152	565	$-$ 0.194	action148	590	$-$ 0.231
action88	516	$-$ 0.147	action287	541	$-$ 0.175	action474	566	$-$ 0.197	action626	591	$-$ 0.231
action6	517	$-$ 0.148	action304	542	$-$ 0.177	action629	567	$-$ 0.199	action357	592	$-$ 0.235
action476	518	$-$ 0.148	action499	543	$-$ 0.177	action443	568	$-$ 0.199	action453	593	$-$ 0.236
action420	519	$-$ 0.149	action207	544	$-$ 0.178	action315	569	$-$ 0.199	action306	594	$-$ 0.238
action442	520	$-$ 0.149	action215	545	$-$ 0.178	action500	570	$-$ 0.200	action210	595	$-$ 0.243
action503	521	$-$ 0.151	action225	546	$-$ 0.178	action587	571	$-$ 0.201	action307	596	$-$ 0.244
action90	522	$-$ 0.153	action417	547	$-$ 0.178	action212	572	$-$ 0.207	action360	597	$-$ 0.244
action5	523	$-$ 0.158	action153	548	$-$ 0.178	action311	573	$-$ 0.208	action359	598	$-$ 0.244
action216	524	$-$ 0.159	action440	549	$-$ 0.179	action75	574	$-$ 0.210	action138	599	$-$ 0.245
action580	525	$-$ 0.159	action600	550	$-$ 0.179	action262	575	$-$ 0.211	action598	600	$-$ 0.245

Table A7

Alternatives’ rank form 601st to 650th

Action	Rank	$\varphi(a_{i})$	Action	Rank	$\varphi(a_{i})$
action300	601	$-$ 0.245	action203	626	$-$ 0.295
action147	602	$-$ 0.246	action144	627	$-$ 0.299
action470	603	$-$ 0.246	action468	628	$-$ 0.301
action472	604	$-$ 0.248	action525	629	$-$ 0.309
action643	605	$-$ 0.250	action526	630	$-$ 0.310
action473	606	$-$ 0.251	action469	631	$-$ 0.312
action623	607	$-$ 0.253	action202	632	$-$ 0.313
action303	608	$-$ 0.255	action640	633	$-$ 0.320
action261	609	$-$ 0.255	action136	634	$-$ 0.329
action302	610	$-$ 0.256	action201	635	$-$ 0.331
action528	611	$-$ 0.258	action466	636	$-$ 0.333
action452	612	$-$ 0.261	action497	637	$-$ 0.339
action301	613	$-$ 0.262	action206	638	$-$ 0.340
action146	614	$-$ 0.262	action390	639	$-$ 0.342
action209	615	$-$ 0.265	action467	640	$-$ 0.343
action222	616	$-$ 0.266	action498	641	$-$ 0.345
action527	617	$-$ 0.272	action639	642	$-$ 0.366
action72	618	$-$ 0.273	action389	643	$-$ 0.369
action597	619	$-$ 0.277	action637	644	$-$ 0.372
action596	620	$-$ 0.281	action496	645	$-$ 0.372
action204	621	$-$ 0.282	action638	646	$-$ 0.377
action471	622	$-$ 0.286	action636	647	$-$ 0.388
action622	623	$-$ 0.287	action495	648	$-$ 0.392
action208	624	$-$ 0.288	action422	649	$-$ 0.405
action137	625	$-$ 0.289	action200	650	$-$ 0.422

Appendix B

Table B1

Comparative study between the proposed approach and the previous works in the field

	Proposed approach	Rouba and Nait Bahloul [2]	De Smet et al. [33]
Input data	A net-flow vector	A matrix of outranking relations	A matrix of outranking relations
Multicriteria method	PROMETHEE II	Any multicriteria outranking method	Any multicriteria outranking method
Method to detect outliers	Normal distribution (SD/IQR methods)	Lof algorithm	Bi-modal distribution
Input parameter	No parameter	Pct: a percentage of alternatives to which the outlier is compared. K: to determine k-th nearest neighbour.	m:the size of the sample of alternatives K: the number of repetitions.
Detection of multiple outliers	Yes	Not always (according to the pct parameter)	No

References

Hawkins

DM.

,Identification of outliers, London; Chapman and Hall; 1980.

Ding

Katenka

Barford

Kolaczyk

Crovella

. Intrusion as (anti)social communication: Characterization and detection. in: Proceedings of the 18th ACM International Conference on Knowledge Discovery and Data Mining. 2012; 12-16, Beijing, China. Association for Computing Machinery, New York NY, United States 2012. Available from https://dl.acm.org/doi/proceedings/10.1145/2339530.

Johnson

Kwok

. Fast computation of 2-dimensional depth contours. in: Proceedings of the 4rd International Conference on Knowledge Discovery and Data Mining KDD98. 1998; 27-31 New York, USA. AAAI Press 1998. Available from https://dl.acm.org/doi/proceedings/10.5555/3000292.

Richard

Hand

. Unsupervised profiling methods for fraud detection. in: Proceedings of the 7th Conference of Credit Scoring and Credit Control. 2001; Edinburgh, UK. 5-7.

Castillo

Donato

Gionis

Murdock

Silvestri

. Know your neighbors: Web spam detection using the web topology. in: Proceedings of the 30th International Conference on Research and Development in Information Retrieval (SIGIR). 2007; 23-27; Amsterdam, Netherlands. New York, NY, United States: Association for Computing Machinery; 2007. Available from https://dl.acm.org/doi/proceedings/10.1145/1277741.

Fawcett

Provost

. Activity monitoring: Noticing interesting changes in behavior. in: Proceedings of the 5th ACM International Conference on Knowledge Discovery and Data Mining. 1999; 15-18, San Diego, CA, USA. Association for Computing Machinery, New York NY United States 1999. Available from https://dl.acm.org/doi/proceedings/10.1145/312129.

Knorr

. A unified notion of outliers: Properties and computation. in: Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining KDD97. 1997; 14-17, Newport Beach, California, USA. AAAI Press 1997. Available from https://dl.acm.org/doi/proceedings/10.5555/3001392.

Knorr

. Algorithms for mining distance-based outliers in large datasets. in: Proceedings of the 24th International Conference on Very Large Databases VLDB98. 1998; 24-27, New York, USA. Morgan Kaufmann 1998.

Knorr

. Finding intentional knowledge of distance-based outliers. in: Proceedings of the 25th International Conference on Very Large Databases VLDB99. 1999; 7-10, Edinburgh, Scotland, Morgan Kaufmann 1999. Available from https://dl.acm.org/doi/proceedings/10.5555/645925.

10.

Knorr

Tucakov

. Distance-based outliers: Algorithms and applications. VLDB Journal. 2000; 8(3): 237.

11.

Breunig

Kriegel

Sander

. Lof: Identifying density-based local outliers. in: Proceedings of the ACM SIGMOD International Conference on Management of Data. 2000; 16-18, Dallas, Texas, USA. Association for Computing Machinery, New York NY, United States 2000. Available from https://dl.acm.org/doi/proceedings/10.1145/342009.

12.

Jain

Murty

Flynn

. Data clustering: A review. ACM Computing Surveys. 1999; 31(3): 264.

13.

Johnson

Kwok

14.

Barnett

Lewis

. Outliers in statistical data. New York; John Wiley and Sons; 1994.

15.

Zopounidis

. Multicriteria decision aid in financial management. European Journal of Operational Research. 1999; 119(2): 404.

16.

Joerin

. Décider sur le territoire: Proposition d’une approche par l’utilisation de SIG et de MMC [dissertation]. Ecole Polytechnique Federale de Lausanne; 1997.

17.

Figueira

Mousseau

Roy

. ELECTRE methods. in: Multiple Criteria Decision Analysis: State of the Art Surveys, Springer, Boston. Figueira

Greco

Ergott

, eds, 2005; 133-162.

18.

Brans

Vincke

. A preference ranking organization method: The Promethee method for multiple criteria decision-making. Management Science. 1985; 31: 647.

19.

Dyer

. MAUT multiattribute utility theory. in: Multiple Criteria Decision Analysis: State of the Art Surveys, Springer, Boston. Figueira

Greco

Ergott

, eds, 2005; 265-292.

20.

Belacel

. Multicriteria assignment method PROAFTN: Methodology and medical applications. European Journal of Operational Research. 2000; 125(1): 175.

21.

Hites

De Smet

, Risse Salazar-Neumann

Vincke

. About the applicability of MCDA to some robustness problems. European Journal of Operational Research. 2006; 174: 322.

22.

De Smet

Hubinont

Rosenfeld

. A note on the detection of outliers in a binary outranking relation. in: Evolutionary Multi-Criterion Optimization. Lecture Notes in Computer Science Trautmann

, et al., eds. 10173. Springer, Cham. 2017; 151-159.

23.

De Smet

Guzman

. Towards multicriteria clustering: An extension of the k-means algorithm. European Journal of Operational Research. 2004; 158(2): 390.

24.

Rouba

Nait Bahloul

. Towards identifying multicriteria outliers: An outranking relation-based approach. International Journal of Decision support system technology. 2018; 10(3): 27.

25.

Freedman

Pisani

Purves

. Statistics. New York; WWNorton; 1978.

26.

Tukey

. Exploratory data analysis, Boston, USA; Addison-Wesely; 1977.

27.

Vincke

. Multicriteria Decision-aid. New York; John Wiley and Sons; 1992.

28.

Keeney

Raiffa

. Decision with multiple objectives: Preferences and value tradeoffs. New York; John Wiley and Sons; 1976.

29.

Jacquet-Lagrèze

Siskos

. Assessing a set of additive utility functions for multicriteria decision-making, the UTA method. European Journal of Operations Research. 1992; 10(2): 151.

30.

Saaty

. How to make a decision. The Analytic Hierarchy Process. European Journal of Operational Research. 1990; 48(1): 9.

31.

Roy

Bouyssou

. Aide Multicritère à la Décision: Méthodes et Cas. Paris: Economica; 1993.

32.

Chakravarti

Laha

Roy

. Handbook of methods of applied statistics. New York; John Wiley and Sons; 1967.

33.

Rouba

Nait Bahloul

. A multicriteria clustering approach based on similarity indices and clustering ensemble techniques. International Journal of Information Technology and Decision Making. 2014; 13(4): 811.

A net-flow based approach to detect outliers in multicriteria decision problems

Abstract

Keywords

1. Introduction

Table 1 The data of the example of the hydroelectric facilities

3.1 Parametric methods

3.2 Non-parametric methods

4. The proposed approach

Footnotes

Appendix A

Appendix B

References

Table 1
The data of the example of the hydroelectric facilities