Abstract
A scenario that often encounters in the event of aggregating options of different experts for the acquisition of a robust overall consensus is the possible existence of extremely large or small values termed as outliers in this paper, which easily lead to counter-intuitive results in decision aggregation. This paper attempts to devise a novel approach to tackle the consensus outliers especially for non-uniform data, filling the gap in the existing literature. In particular, the concentrate region for a set of non-uniform data is first computed with the proposed searching algorithm such that the domain of aggregation function is partitioned into sub-regions. The aggregation will then operate adaptively with respect to the corresponding sub-regions previously partitioned. Finally, the overall aggregation is operated with a proposed novel consensus measure. To demonstrate the working and efficacy of the proposed approach, several illustrative examples are given in comparison to a number of alternative aggregation functions, with the results achieved being more intuitive and of higher consensus.
Introduction
When dealing with real-world problems, the opinions of different experts are usually aggregated in order to provide more robust solutions. Aggregation functions that aim to combine multiple arguments for the production of a representative consensus are of significant importance in decision making, especially in the presence of the uncertain and noisy settings. It has attracted wide attention for both researchers and practitioners with a number of important approaches and their variations [4, 32] proposed in recent literature. Although methods constructed on the basis of fuzzy sets [19] and evidence theory [14] could deal with scenarios under uncertainties, more dedicated approaches are in demand to perform decision aggregation where outliers exist. A common problem that often exists in dealing with decision aggregation is whether the consensus obtained is representative of the majority, reflecting a unbiased universal opinion. It is known that the aggregation of multiple judgements from different sources are often more accurate and reliable than individuals’ dominating judgements. This is also known as the wisdom of crowds [26], with a number of significant applications such as political and economic forecasting, nuclear and chemical safety evaluation and even possible responses to restless volcanoes [26]. A scenario that often encounters when aggregating multiple judgements is the existence of individuals whose judgements may significantly deviate from their peers’, which are termed outliers in this paper.
As a numerical example, suppose there are 10 witnesses who give their opinions with respect to the probability of Perter being guilty of a crime as shown in Table 1. Out of 10 different witnesses, 6 tend to believe Peter is more guilty with probability beliefs over 0.5, 1 is totally neutral, while the remaining three opinions favour Peter being innocent. In case of the aggregated result (0.46) calculated over purely arithmetic mean, the final judgement may be counter-intuitive especially considering that the majority vote to the opposite. A conventional approach to improve this situation is to remove the highest and lowest values. For Peter’s example again, the probability of being guilty will be 0.481, still leading to a counter-intuitive result, if both the maximal value (0.7) and the minimal value (0.05) are removed.
Criminal suspect possibilities given by witnesses
Criminal suspect possibilities given by witnesses
A very recent review is conducted in [6], which analyses the most popular aggregation operators out of 50 most notable studies. The interrelationship between input argues is typically considered in numerous popular aggregation operators including the discrete Choquet (Ch) integral [15], the Bonferroni Mean (BM) [24], the Muirhead mean [21], the Maclaurin symmetric mean [22] and the Heronian mean (HM) [10]. Additionally, it’s also common to observe aggregation operators extended in combination with fuzzy concepts with the aim to tackle uncertainty and imprecision. For instance, the Muirhead mean, which is commonly employed in decision making, is combined with intuituiionistic fuzzy numbers in [21], and the Maclaurin symmetric mean has been studied in [22] in combination with linguistic information and intuitionistic fuzzy numbers. In other literature, the Robust Estimators of Location (REL) [27] is another means to aggregate data with outliers, where the averages are computed using the shortest contiguous sub-sample containing at least half of the data. However, apart from the outliers, non-outliers may also be discarded when aggregation. Another popular set of approaches [30, 31] propose to only accept the aggregated result if the consensus measure meets a predefined threshold. However, a possible pitfall of these approaches lies in the mechanism that the original values requires to be modified in order to pass the consensus measure, which may not be desirable for many cases. Alternatively the density-based means (DBM) [2] has been proposed for data with outliers which are weighted by their density. However, in the event of outliers with extremely large or small values, DBM still aggregates counter-intuitive results. Moreover, in the event of no outliers, the aggregated results may also be counter-intuitive, as it may differ from the pure arithmetic mean.
Another interesting and more general type of aggregation operator is the family of Ordered Weighted Averaging (OWA) operators [33], based on which a number of aggregation functions have been proposed [35, 36]. OWA is a parameterized operator based on the ordering of extraneous variables that it is applied to. The fundamental aspect of this family of operators is the reordering step in which the extraneous variables are rearranged in descending order, with their values subsequently integrated into a single aggregated one. In particular, the Power Average (PA) function and Power OWA (POWA) function [34] have been proposed, whose weights depend upon the inputs and allow values being aggregated to support and reinforce each other. A number of aggregation functions [32] have been proposed based on the PA function, which allow adjacent values to support and reinforce each other by considering correlations of the underlying data, providing potential benefit to aggregating data with outliers. The above review suggests that although popular aggregation operators such as the Ch, BM, HM, PA and their extensions take into account the relationship of input arguments, but without explicitly considering scenarios where outliers may exist. While methods such as the REL and DBM may work with outliers, they could still suffer from undesired pitfalls including discarding non-outliers and generating counter-intuitive results in case of extremely large or small outliers. Inspired by the above observation, this paper proposes a novel approach using a mixed strategy to aggregate consensus where extremely deviated outliers exist. The concentrate region of a set of non-uniform data is first computed such that the domain of aggregation function is partitioned into sub-regions. The aggregation will then operate dynamically with respect to the corresponding sub-regions. In order to demonstrate the working and efficacy of the proposed approach, the illustrated examples are given in comparison to a number of alternative aggregation functions, with results achieved by the proposed method being more intuitive and of higher consensus. The remainder of this article is organized as follows. In Section 2, the background preliminaries including definition of aggregation function and the methods that have been previously reviewed and later utilised for experimental comparison are introduced. The novel aggregation approach for data with outliers are introduced in Section 3. This is followed by illustrative examples in comparison to several popular state-of-the-arts in Section 4. The comparison with alternative methods is introduced in Section 5. Section 5 provides the conclusion of the paper with future work.
In order to demonstrate the proposed approach better, preliminary background is given in this section, which reviews the basic definition of aggregation function as well as a number of related works in dealing with outliers.
Aggregation function
The aggregation functions that take real arguments from a closed interval
where a, b ∈ [0, 1] are two constants.
The formal definitions of the common aggregations could be found below, which have been proposed for generic decision aggregations, but may not fit for scenarios where extreme outliers exist.
Robust estimators of location
REL delivers values representative of the majority of the data when a specified proportion of data seen as potential outliers is removed [12, 27]. In the following, the brief descriptions about the methods including in REL are given. Firstly, the values are ordered, such as
The LMS estimator is the midpoint of
Density based means
DBM is a weighted arithmetic mean and the weights depend on the density of the data [2, 5]. The weights of inputs that are closer to the main group of data are higher than the weights of those far away. Let d ij denote the distance between inputs x i and x j , the density based mean is defined as follows.
Power average aggregation function
Yager [34] introduced a non linear weighted-average aggregation function, which is called Power Average (PA) aggregation function and can be defined as follows
Sup (x
i
, x
j
) ∈ [0, 1] Sup (x
i
, x
j
) = Sup (x
j
, x
i
) Sup (x
i
, x
j
) ≥ Sup (x, y), if |x
i
- x
j
| ≤ |x - y|
Obviously, the support (i.e., Sup) measure is essentially a similarity index. The more the similarity, the closer the two values are, and the more they support each other. In this article, Sup (x i , x j ) =1 - |x i - x j |, i ≠ j.
Discrete choquet integral
Choquet Integral (Ch) is a powerful aggregation function especially in merging finite real inputs. Ch focus on capturing the interrelationship among the inputs by adopting strategies to generate the weights of the inputs.The standard Choquet Integral formulas can not accommodate inputs which exist in continuum. So the Discrete Choquet integral based on fuzzy measure m is proposed [15]. The characteristics of inputs can then be captured by the fuzzy measure, which makes it able to work with outliers. The Discrete Choquet integral is defined as follows
where m is a monotone set-function satisfying two boundary conditions m (∅) = 0 and m ({ 1, 2, . . . , n }) =1.
Bonferroni mean
Bonferroni Mean (BM) assumes homogeneous relation among the inputs, i.e., each data x
i
is related to the rest of the inputs X∖ { x
i
}, where X ={ x1, x2, . . . , x
n
} denotes the inputs. BM focuses directly on aggregated arguments to capture the interrelationships among them [24]. The degree of interrelationships in between input arguments is measured on the basis of their magnitudes. That is, the interrelationship of input arguments that come with small values tend to be high with respect to those with large values, but low with respect to those also with smaller values. BM is defined as follows
Heronian mean
Heronian Mean (HM) [10] is a function that can reflect the interrelationship of the input arguments and is defined as follows:
Mixed aggregation functions decreasing effects of outliers
Despite most of the consensus lies in the so-called Concentrate Region (CR), the information coming from the minority outliers, i.e., the extremely large or small values, should not be simply ignored. In order to tackle this challenge, this paper proposes to utilise a mixed strategy so that appropriate aggregation functions may operate on argument values if they fall into the corresponding sub-regions. In particular, the arguments may be aggregated with averaging function such as arithmetic mean if they fall into the region of CR. Whereas in case of the aggregation for outliers, the key idea is to make the aggregated results as close as those located in the CR, in an attempt to avoid counter-intuitive aggregation behaviour overall.
Algorithm searching CR for non-uniform data
In order to perform different aggregation strategies depending on the characteristics of the data that may fall into different sub-regions within the universe, it is of significant importance that a method is devised such that it is able to detect the consensus outliers efficiently without human intervention. Instead of assuming any priori distribution of the underlying data which is not always available [16], nor involving high computational calculation by fitting the data into some existing models [13, 18], the set of argument values are considered as one-dimensional data, and the sub-region that contains fewer arguments will be iteratively combined into the Concentrate Region if it is closer to the existing Concentrate Region than the data subsets containing large or small values.
Let
Three subsets are then initialized as Initialize Compute the distances between x[i] and S
min
, S
concentrated
, S
max
, respectively, which is presented as d (x[i], S
min
) , d (x[i], S
concentrated
) and d (x[i], S
max
); Assign x[i] to the set with minimal distance such that x[i] ∈ S
k
0
given that Update i = i + 1, if i < n, go to Step 2, otherwise go to Step 5; Let
Step 2, d (0.2, S min ) =0.1, d (0.2, S concentrated ) =0.3875 and d (0.2, S max ) =0.78. Step 3, d (0.2, S min ) =0.1 is the minimum, 0.2 is added to S min , S min = {0.1, 0.2} , distance [j] =0.1, j = j + 1. Step 4, i = 3 <12, go to step 2. Repeat steps 2, 3, 4 until i = 4, x i = 0.6, go to step 2. Step 2, d (0.6, S min ) =0.4, d (0.6, S concentrated ) =0.0125 and d (0.6, S max ) =0.38, where S min = {0.1, 0.2, 0.2} and S concentrated , S max are not changed. Step 3, d (0.6, S concentrated ) =0.0125 is the minimum, 0.6 is added to S concentrated , S concentrated = {0.5875, 0.6} , distance [j] =0.0125, j = j + 1. Step 4, i = 5 <12, go to step 2. Repeat steps 2, 3, 4 untili = 12, go to step 5. Step 5, maxdis = max {0.1, 0, 0.0125, 0, 0, 0, 0.1, 0, 0.1, 0.01} =0.1, max {S min } = max {0.1, 0.2, 0.2} =0.2, min {S max } = min {0.97, 0.98} =0.97, max {S concentrated } = max {0.5875, 0.6, 0.6, 0.6, 0.6, 0.7, 0.7, 0.8} =0.8, min {S concentrated } = min {0.5875, 0.6, 0.6, 0.6, 0.6, 0.7, 0.7, 0.8} =0.5875. As |min {S concentrated } - max {S min } | > maxdis and |max {S concentrated } - min {S max } | > maxdis, the Concentrate Region is CR = [0.6, 0.8].
Analysing and classifying partitioned sub-regions of domains
Based on relative positions between CR and the range of data, there are generally three cases as shown in Fig. 1, where [a, b] is CR, and [u, v] is the range of data. In particular, points of CR do not overlap with points of the range of data in Fig. 1(a). Whereas, point a of CR overlaps with the point u in Fig. 1(b), and point b of CR overlaps with point v in Fig. 1(c). As a result, the partition of the domain may be classified into four cases in the bivariate situation as show in Fig. 2, where the axis represents two attributes whose values are combined into a single number.

Distributions of CR of data.

Classifications for partition of domains.
In particular, case one presented as 1 in Fig. 2 is when both values are smaller than CR. Similarly, the second and third cases are when both values are within the range of CR or greater than above CR as shown in 2 and 3 of Fig. 2, respectively. In the event of one value is within with the other does not, this is represented as 4-1 in Fig. 2. Whereas when one value is above CR and the other is below CR, it is presented as 4-2 in Fig. 2. The average of two values cannot be very large or small and may be a value located within CR in the cases 4-1 and 4-2. Therefore, these two cases can be combined into one and it is case 4. In Fig. 2, the other sub-graphs can be considered as the special ones of the sub-graph in Fig. 2 (a).
The behaviours of mixed aggregation functions depend on the inputs, these functions exhibit conjunctive, disjunctive or averaging behaviours on different parts of the domains and there are many different families [4]. Averaging is the most common method to combine information into a single numerical value. The basic rule is that the total score cannot be above or below any of the inputs. The aggregated value is seen as some sort of representative value of all the inputs.
Conjunctive aggregation functions do not allow for compensation: low scores for some criteria cannot be compensated by other scores. The final aggregated result is the smallest value of any of the inputs that bounds the output value. Comparing with conjunctive aggregation functions, disjunctive aggregation functions are the opposite.
The most popular mixed aggregation functions, uninorms [37] and nullnorms [7], are related to triangular norms and conorms [4]. Uninorms make conjunctive operator when dealing with low input values (these below a given value e), and have a disjunctive operator for high values (these above e) and are averaging for otherwise [4]. Nullnorms are contrary [4]. The characteristics of MADEs (Mixed Aggregation Functions Decreasing Effects of Outliers) were constructed according to Nullnorms.
Without loss of generality, once the partition of domain of the aggregation function done based on the sub-graph shown in Fig. 2 (a), this section discusses the proposed bivariate MADEs. To make the descriptions simpler, let CRs of the aggregated values be the same in the following definitions, with the behaviours of bivariate MADEs shown in Fig. 3.

Behaviors of bivariate MADEs.
In this paper, the Conjunctive and Disjunctive functions satisfying the above conditions are denoted as MC (x, y) and MD (x, y), respectively. The averaging function usually means arithmetic or geometric mean.
It is then necessary to discuss how to obtain a concrete MADE. Two special classes of conjunctive and disjunctive aggregation functions are the triangular norms and conorms. Hamacher t-norms and t-conorms [4] are the most popular continuous Archimedean t-norm and t-conorm [25] in many practices, with well-known Algebraic and Einstein t-conorms and t-norms as the special cases [28]. For simplicity, Hamacher t-norms and t-conorms were adopted as the special instances in this article, the following Conjunctive and Disjunctive functions are introduced to construct the bivariate MADEs.
Based on this proposition, the simple strategy to choose the appropriate functions to construct the MADE can be described as follows. That is, if the effects of extremely small outliers on aggregating result are more significant than the ones with extremely large outliers, for instance, the number of small outliers may be greater than the one with larger outliers or there are no large outliers, then the larger values should be chosen for γ. Otherwise, the smaller values should be chosen for γ. Usually, the averaging aggregation function adopted here is the arithmetic mean. A simplistic example is given below to explain how the bivariate MADEs is constructed.
Let the aggregated values be a vector Let symbolic (i) represent the index of the ith largest value and [i] denote the index of the ith smallest value for
In the aforementioned definition, M may be any aggregation functions. However, the main contribution of the proposed MADEs is to decrease the possible side effects of outliers that are located outside of CR. As a result, M is the weighted arithmetic mean and the weights are defined as the proportions of outliers and the normal argument values are defined as:
One of the biggest advantages of using the proposed searching algorithm lies in the avoidance of manually adjusting associated parameters, which could have made the algorithm sensitive to human errors. For the given one-dimensional data, the distribution of data by searching algorithm is unique, that is, the distribution of data is uniform or the outliers can be found. When aggregating one-dimensional data, the Extended Conjunction function will be used for outliers with larger values. In particular, such function will result in an aggregation, which is less than min { X i |i = 1, 2, …, n } and is more than values lying in the concentrate region, thereby reducing the impact of outliers with high values. Conversely, the application of the extended disjunctive function to outliers with low values will lead to an aggregated result greater than max { X i |i = 1, 2, …, n } and smaller than any value in the concentrate region.
Let X ∈ [0, 1]
n
be the input vector and y be the output value. In the case where y is known, the constructor F aims to approach y such that F (X) ≈ y. In the case where y cannot be obtained in advance, or simply not available, the consensus measure may be utilised to check whether the final result obtained by the aggregation function is consistent with the majority of inputs. For instance, the consensus degree of the preference value may be measured by [29] as
In group decision making (GDM), the arithmetic mean of the data located in the concentrate region CR is the value that reflects the closest opinion to the universal one. In order to avoid the overall aggregated result significantly affected by outlier values, it is important for the aggregated results of outliers to approximate the CR mean. A consensus measure is therefore proposed for the final aggregation of argument values with outliers.
Fuzzy implication [3, 23] is a function which is monotone non-increasing in the first argument, monotone non-decreasing in the second argument, and satisfies the boundary conditions, I (0, 0) =1, I (1, 1) =1 and I (1, 0) =0. In this study, the Łukasiewicz implication (I L (x, y) = min (1, 1 - x + y)) is used to construct the consensus measure.
For the above definition, For For the special case, Let π (i) denote all permutations on {1, . . . , n} and
The formula
According to this definition, CRI is the average of CERs.
In this subsection, the application is introduced to demonstrate the procedures where MADEs are used to aggregate multiple values and are compared with other methods, such as, Least Squares Trimmed estimator (LTS) [27], Density Based Means (DBM) [2], Power Average aggregation functions (PA) [35], Discret Choquet Integral (Ch) [15], Bonferroni Mean (BM) [24], Heronian Mean(HM) [10] and so on.
The example is on the background of job shop scheduling, where the executive board consisting of a number of experts votes for a certain plan. As it is not a good practice to simply discard or give lowers weights to any one, their votes are usually equally treated.
Experts given possibilities on possible plan
Experts given possibilities on possible plan
In comparison to SM, DBM, TM, Mid, PA, LTS, Ch, BM, HM with aggregated results shown in Table 3, it can be observed that MADE computed result (0.67299) is very close to SM (0.65714) than the others computed ones. The CER for each possibility is depicted in Fig. 4 in comparison to MADE with TM, DBM, Ch, BM, HM and PA. Although Mid and LTS have higher CRIs than TM as shown in Table 3, they discard too much data including those seen as non-outliers, the comparing results for them with MADE are shown in Fig. 5. It can be observed that, for all possibilities, CERs for SM and MADE are very close to each other. Possibilities p i (1 ≤ i ≤ 7) are in CR ([0.6, 0.8]), CERs for DBM, PA, Ch, HM and TM are less than the ones for MADE. There are same results on possibilities p i (i = 11, 12), but the results are contrary, for p i (i = 8, 9, 10). Possibilities p i (i = 8, 9, 10) have smaller values and are not in CR, the differences between them and SM are larger. MADE computed results are very close to SM and far from p i (i = 8, 9, 10). CERs between p i (i = 8, 9, 10) and MADE computed result are lower. DBM, PA, HM and Ch computed results are smaller and far from SM, but MADE computed result is large and very close to SM. Possibilities p i (i = 11, 12) have larger values and are not in CR, CERs for MADE are higher. Together, it can be concluded that CRI of MADE is closer to the one of SM and is higher than the others (Table 3). Although the partial result of MADE is similar to the BM, the CRI of MADE is closer to the one of SM than BM.
Aggregated results and CRIs
In Fig. 5, it can be observed that, for possibilities (p i , 1 ≤ i ≤ 7) which are in CR, CERs for Mid are lower than the ones for MADE. CERs for LTS are lower than the ones for MADE for p2, p4, p6, p7, but contrary for p1, p3, p5. Possibilities (p i , i = 8, 9, 10) have smaller values and are not in CR, CERs for MADE are less than the ones for Mid, but are greater than the ones for LTS. For large values (p i , i = 11, 12) that are not in CR, CERs for MADE are greater than the ones for Mid, but less than the CERs for LTS. CERs for 58.33% of possibilities and MADE computed result are higher than ones for these possibilities and LTS obtained result.

Comparing results of CERs for MADE with DBM, PA, Ch, BM, HM and TM to possibilities on possible plan.

Comparing results of CERs for MADE with Mid, lTS and TM to possibilities on possible plan.
In general, CRI for MADE is higher than CRIs for Mid and LTS. The Mid discards most data except the single midpoint, its performance is likely to be affected by outliers. The same problems exist for LTS. Based on the discussions given above, it can be concluded that MADE has higher consensus with inputs and it does not discard any input maximally utilising all available information. DBM, PA, Ch, HM and BM do not ignore any argument values, but their consensus degree is lower than that of MADE. SM discards the outliers and is regarded as the standard for comparison. Based on MADE computed result (0.67299), it can be concluded that the APS given plan can be accepted.
The MADE method can also be used in the group decision-making.
In terms of computation complexity for the proposed method, it is attributed to the computation of its two components. The first step of the proposed method divides the input data into three sub regions based on the distance between data and each sub region, resulting in the computation complexity of O (n2). The second step aggregates values of the three sub regions, resulting in the computation complexity of O (n). Overall, the complexity of MADE is therefore O (n2). Although tradition averaging methods such as TM and SM come with O (n), they generally cannot work well to generate unbiased result with extreme outliers. Whereas more advanced methods such as DBM, PA, Ch, HM and BM also come with the complexity of O (n2).
In summary, integrating the aforementioned analysis, the differences between different methods in this study can be summarized as follows: In Mid and LTS, a fairly large proportion of original argument values may be removed, therefore leading to the conclusion that are not the representative, and their performance of both methods are not stable as shown in Fig. 5 and 1. The outliers which are far from the concentrate area of the data have smaller weights in DBM and PA. Although BM, HM and Ch methods consider the distribution of input data, the overall aggregated result is still not satisfactory as demonstrated by whether Peter is the criminal. In case where outliers are either extremely small or large (instead of a mixture of both), results aggregated by DBM, PA, HM and Ch may still be counter-intuitive, which could be corrected using the proposed MADEs as shown in Section 1. The results aggregated by the proposed method have achieved higher consensus than those of TM, DBM, PA, Ch, HM and BM as demonstrated in Table 3. Moreover, for each aggregated value, MADEs has also achieved higher consensus with the aggregated values that are in the CR, and they have lower consensus with the values that are not in CR as demonstrated in Figs. 4 and 5.
Conclusion
In group decision making, extremely very large or small values that are isolated from others may lead to counter-intuitive aggregated result when data is simply aggregated by averaging. In order to tackle this challenge, this paper proposes a novel approach without discarding any exiting argument value. A search algorithm is first proposed to search the concentrate region for non-uniform data resulting in a number of partitioned domains. Based on the partition of domain, MADEs are constructed and their basic properties are studied. To measure the consensus of aggregated result and the inputs, CRI is proposed, which reflects the consensus degree between aggregated result and the set of all original argument values. Furthermore, CER is proposed, which reflects the consensus degree between the aggregated result and each individual input. Based on computed CERs, more details on consensus between aggregated result and inputs can be obtained. The illustrative examples are given in comparison to a number of alternative aggregation functions, with the results achieved being more intuitive and of higher consensus. Whilst promising results achieved, it is proposed to put this approach into real-world applications such as those from the wisdom of crowds [26]. In addition, the proposed method could be incorporated in conjunction with multi-attribute group decision-making [11], multi-source information fusion [20], fuzzy system construction [8, 9].
Footnotes
Acknowledgment
This work is sponsored by Science and Technology Project in Shaanxi Province of China (Program No.2019ZDLGY07-08), and Xi’an Project for College Talent Providing Services to Enterprise (No. GXYD17.15).
