An analysis of consensus approaches based on different concepts of coincidence

2.1 GDM problem framework

GDM problem is modelled by assuming a collection of possible alternatives X = {x _i , i = 1, …, n} (n ≥ 2) which are assessed by the members of the group, i.e. experts, E = {e _i , i = 1, …, m} (m ≥ 2), and then, the goal is to get a consensus resolution in accordance with the most of the preferences expressed by the experts [7, 56]. Fuzzy preference relations are widely utilized in the literature to depict the expert preferences [15, 43].

Definition 1. Let X be a collection of alternatives. A fuzzy preference relation on X, P, is defined by η _P: X × X → [0, 1], depicting p _ij  = η (x _i , x _j ) the different preference levels over the alternatives: p _ij  = 1 stands for the highest level of preference for one alternative x _i on another x _j , p _ij  ∈   (0.5, 1) stands for a certain preference for the alternative x _i on the alternative x _j and p _ij  = 0.5 stands for nonchalance.

Sometimes, some rational criteria are required, as for example, the additive reciprocity property: p _ji  + p _ij  = 1 for all i, j in {1, …, n}.

In this situation, the solution process of a GDM problem consists of obtaining a set of solution alternatives, X _sol  ⊂ X, from the preferences expressed by the experts. This solution process is found on two dissimilar processes [9, 35]: selection and consensus. The solution collection of alternatives is deduced thanks to the selection process while the consensus process is employed to increase the level of accord amongst experts before obtaining that solution.

2.2 Selection process

Two procedures configure the selection process when dealing with a GDM problem [3, 38]: an aggregation procedure of expert preferences and an exploitation procedure of that aggregated preferences.

By aggregating all single fuzzy preference relations, namely, {P ⁱ ,  i = 1, … , m }, one collective relation of preference, namely, P c = ( p ij c ) , is obtained. P ^c stands for the collective preference on every pair of alternatives. Many aggregation operators could be used, however we should point out that the ordered weighted averaging (OWA) operator [51] is extensively utilized in GDM frameworks.

Definition 2. An ordered weighted averaging operator is a function φ: ℝ m → ℝ , which uses a vector of weights, W = (w₁, …, w _m ), being w _i in the unit interval, [0, 1], and ∑ i = 1 m w i = 1 , so as to perform the aggregation of a set of values {p₁, … , p _m  } through the expression: φ ( p 1 , … , p m ) = ∑ k = 1 m w k · p σ ( k ) ,(1) with σ being a permutation function, σ: {1, … , m} → { 1, …, m }, according to which p_σ(k) ≥ p_σ(k+1), ∀ k = 1, …, m - 1, this way being p_σ(k) the k greatest value in the collection {p₁, i = 1, … , n }.

Yager defines an interesting procedure to determine the vector W through fuzzy linguistic quantifiers Q [52]. Each weight w _k could be computed by the following expression: w k = Q ( k m ) - Q ( k - 1 m ) , ∀ k ∈ { 1 , … , m }(2)

Then, we could compute every collective preference value p ij c in the collective preference relation as p ij c = φ Q ( p ij 1 , … , p ij m ) where φ _Q denotes the OWA operator found on the notion of fuzzy majority represented by Q ( s ) = { 0 if 0 ≤ s < l s - l u - l if l ≤ s ≤ u 1 if u < s ≤ 1(3)

In this paper we use three fuzzy linguistic quantifiers: “at least half”, “most of” and “as many as possible”, with the parameters (0, 0.5), (0.3, 0.8) and (0.5, 1) for (l, u), respectively.

By exploiting the collective preferences on the alternatives we achieve a whole ranking of them, which allows us to attain the solution collection of alternatives. To do it, we could apply any degrees of choice of alternatives. We employ the following two degrees of choice of alternatives [12, 20]:

Quantifier guided dominance degree: QGDD i = φ Q ( p i 1 c , … , p in c ) .(4)

The solution X _sol is then achieved by means of the application of the two choice degrees in such a way that those alternatives with higher choice degrees are chosen.

In a consensus process we have to define a coincidence criterion to calculate those consensus measures that allows us to lead to the process of reaching consensus. This is a dynamic process (see Figs.1 and 2) in which a previously agreed consensus value and/or number of rounds serves as control of the process. If any of these values is exceeded, the process ends and, if not, a feedback mechanism provides advice to experts in order to approximate their positions.

OPEN IN VIEWER

Fig.1

Consensus Model based on distance functions.

OPEN IN VIEWER

Fig.2

Consensus Model for C _x method.

Two of the main approaches that can be used in a consensus process are the notion of soft coincidence among preferences and the concept of the coincidence among solutions [4, 19].

3.1 Wilcoxon signed-ranks matched-pairs test

Let X₁, X₂, …, X _n denote a n-size random sample from a distribution function F which is continuous, let p denote a value in ( 0 , 1 ) ∈ ℝ , and let ξ _p  (F) indicate the p-quantile of F, that is, ξ _p  (F) is that value which provides a solution of the equation F (x) = p. When p = 0.5, ξ_0.5 (F) is called median of the distribution function F.

A problem of symmetry and location consists in testing the hypothesis H₀ : ξ_0.5 (F) = ξ₀ and F is symmetric against the hypothesis H₁ : ξ_0.5 (F) ≠ ξ₀ and F is not symmetric. A nonparametric statistical technique known as Wilcoxon signed ranks test supplies a hypothesis test that considers the measure of the differences between the observed values and the quantile in H₀ with the aim of performing a problem of symmetry and location.

Let H₀ : ξ_0.5 (F) = ξ₀ be the null hipothesis and let D _i  = X _i  - ξ₀, i = 1, 2, …, n, be the differences to the hypothesized value. Under H₀ positive and negative differences are expected to be dispersed, so that the expected number of negative differences will be n/2 and negative and positive differences of equal absolute magnitude should take place with the same probability.

Let |D _i |, i = 1, 2, …, n, be the absolute values of D _i , i = 1, 2, …, n, and let us rank them from 1 (for the smallest) to n (for the largest). If T₊ denotes the sum of ranks assigned to those D i ′ s that are positive and T_- denotes the sum of ranks assigned to those D i ′ s that are negative, it follows that T + +: T - = ∑ k = 1 n k = n ( n + 1 ) 2

So, T₊ and T_- are linearly related and provide equivalent rules. A large value of T₊ indicates that most of the larger ranks are assigned to positive D i ′ s , so that they support H₁ : ξ_0.5 (F) > ξ₀. The same assertion applies to the alternative H₁ : ξ_0.5 (F) < ξ₀ and H₁ : ξ_0.5 (F) ≠ ξ₀.

This way the test rejects H₀ : ξ_0.5 (F) = ξ₀ to accept H₁ : ξ_0.5 (F) > ξ₀ if T₊ > c₁; equivalently, if t₀ denotes the observed value of T₊, it rejects H₀ if p₀ = P _{H 0}  [T₊ ≥ t₀] ≤ α, being α the significance level of the test. The test rejects H₀ to accept H₁ : ξ_0.5 (F) < ξ₀ if T_- > c₂ or p₀ = P _{H 0}  [T_- ≥ t₀] ≤ α, being t₀ the observed value of T_-. And the test rejects H₀ to accept H₁ : ξ_0.5 (F) ≠ ξ₀ if T₊ > c₃ or T_- > c₄ being values c _i the critical region borders, or, equivalently, if p₀ = 2(smaller tail probability).

Under H₀, the common distribution of T₊ and T_- is symmetric about the mean E [T₊] = n (n + 1)/4 with variance var [T₊] = n (n + 1) (2n + 1)/24. For large n, the standardized T₊ has approximately a standard normal distribution.

If the available data are matched-paired, {(X _i , Y _i ) , i = 1, …, n}, being derived from applying two treatments to the same set of subjects, to test H₀ : ξ_0.5 (F_{X i -Y i} ) = ξ₀ against the possible alternatives Wilcoxon’s test is carried out in the same way as in the case of one sample by taking D _i  = X _i  - Y _i  - ξ₀ [12].

3.2 Friedman mean ranks

The analysis of data resulting from k-related samples can be performed using various nonparametric techniques. One of them addresses the problem as an extension of the two-way analysis of variance for a randomized block design when the assumption that distributions are continuous replaces the assumption of normality.

Let (X_i1, …, X _ik ), i = 1, …, n, be a random sample from a k-variate continuous type distribution function. The data may be arranged in n rows (blocks) and k columns (treatment/measure). The observations in different rows are independent and those in different columns are dependent. The observation x _ij then corresponds to the ith block and jth treatment/measure, j = 1, …, k and i = 1, …, n.

In order to test the hypothesis that measure effects are all equal against the hypothesis that measure effects are not all equal, the Friedman procedure [37] involves replacing each observation in a block by its rank. The rank of the jth observation in the ith block, R (x _ij ) = R _ij , is the value from 1 to k obtained by consecutively numbering the observations X _ij , j = 1, …, k. So, for each block, the observed values are sorted for each measure and ranked from 1 (the lowest value in the block) up to k (the highest value of the block). Hence ranks are assigned separately for each block.

Let R _j denotes the sum of ranks for jth measure, j = 1, …, k, R j = ∑ i = 1 n R ij . The mean rank for the jth measure, j = 1, …, k, is defined as R ¯ j = R j n

4.1 Statistical test results

Table 1 shows the p-value for each one of the distances used in our experimental study versus C _x . To better understand this table and the following ones, from now on, d₁ is denoted by “Ma”, d₂ by “Eu”, d₃ by “Co”, d₄ by “Di” and, finally, d₅ is denoted by “Ja”. In view of the results, it follows that the hypothesis tested and displayed in Section 3 is rejected.

The comparison between different distance functions and C _x to measure consensus produces significantly different results in all possible combinations used in the experiment. In particular, we observe a p-value less or equal than 0, 05 (predetermined significance level α) in all cases. In fact, in four of the cases the p-value is lower than 0, 001  (α), except for the Jaccard distance function, where the p-value is 0,013.

We note that, at the relation level, measurement of consensus is affected significantly by the use of a different distance function in comparison with C _x . Results differ significantly depending on the distance function to be compared. Obviously, the comparison of different distance functions, for which significant variation has been established, could affect the convergence of the consensus process at this level.

Moreover, it can be observed that the results of the C _x consensus measure and the Jaccard distance function are similar. This fact leads us to think that the consensus measure C _x can be used as an alternative to the Jaccard distance function in the considered situation, and also that a consensus model based on the coincidence among solutions could replace a model based on the among experts preferences and vice versa in the calculation of the degree of consensus when dealing with a situation like this one.

Let us now analyze the results derived from Friedman mean ranks according to quantifiers, experts and alternatives.

Table 2 shows mean ranks of all methods according to the linguistic quantifiers considered in this study: “at least half”, “most of” and “as many as possible”. As regards to the first linguistic quantifier, we observe that the Jaccard distance function is closer to C _x than the rest of distance functions. In this case the Jaccard distance function achieves bigger values than C _x . We also note that as for the linguistic quantifier “most of”, C _x is positioned between the Jaccard and Dice distance functions. In this situation the Jaccard distance function is nearer to C _x than the Dice distance function. Regarding to the linguistic quantifier “as many as possible”, it can be seen that C _x is positioned between the Jaccard and Dice distance functions as in the previous situation but, in this case, C _x is nearer to the Dice distance function.

Table 3 shows mean ranks for different methods and different number of experts considered in this study. In this case C _x is situated between the Jaccard and Dice distance functions except for the 4 experts case where the results are different from the ones obtained from 6 experts. We also note that, in all cases except the above-mentioned 4 experts case, C _x gets values closer to the Jaccard distance function than the Dice distance function.

On the other hand Table 4 shows mean ranks of different methods for the three possible number of alternatives proposed in the study. We observe that the behavior of C _x is very stable in all cases. It can be pointed out that the values of C _x are situated between the ones of the Jaccard and Dice distance functions.

Table 5 shows mean ranks for all methods and samples. It can be observed that the Jaccard and Dice distance functions are closer to C _x in their mean rank positions than the rest of functions, being Jaccard the closest one.

In summary, we perceive that at the level of relation the evaluation of consensus depend on the distance functions which are used instead of C _x . Therefore, the utilization of distinct distance functions could influence the convergence of the consensus reaching process at the level of relation. Furthermore, the results obtained suggest that the Jaccard distance function is the best option versus C _x method, and the second option should be the Dice distance function.

The experimental study performed in this paper allows us to visualize a ranking of the different distance functions useful for their application as it is illustrated in Fig. 3. Most of metrics reach a consensus speed faster than C _x , so if we want a faster convergence to consensus we would measure consensus on preferences with Manhattan, Cosine or Euclidian metric.

OPEN IN VIEWER

Fig.3

Global consensus speed.

If required in a consensus process, the values of C _x model on the set of solutions could be substituted for those provided by the the appropriate distance function on the alternatives.

These results are coherent with the ones obtained in [12] but in this paper we also obtain a quantitative classification for the considered methods.

Finally, note that the values obtained for the consensus measure C _x are placed between the values obtained for the Dice and Jaccard distance functions. This way, it seems to be confirmed that C _x can be considered a different measure but comparable with the measures already analyzed [12], being the values of C _x closer to those of the Jaccard distance functuion and being able to replace or be replaced by them.

5.1 First round

At the relation level, the Jaccard distance function provides a consensus degree of 0.43, C _x of 0.49 and the Cosine distance function of 0.66. The global consensus degree is lower than the threshold consensus level, so that experts receive feedback to modify their preference relations.

5.2 Second round

The new fuzzy preference relations are the same that appear in [12]. Using the Cosine distance function the consensus degree results in 0.81 which is greater than the threshold consensus level and the consensus process ends. However, if the Jaccard distance function or C _x are used, it would be necessary to continue with the consensus reaching process since the consensus degree levels are 0.53 and 0.61 respectively.

5.3 Third round

The new fuzzy preference relations are the ones that appear in [12]. The Jaccard distance function results in a consensus degree level of 0.60 and C _x of 0.65.

5.4 Fourth round

The new fuzzy preference relations appear in [12]. Using C _x we would have had a consensus degree level of 0.79 which is greater than the threshold consensus level and the consensus process ends. However, using again the Jaccard distance function we need to continue with the consensus reaching process since the consensus degree level would be 0.69.

5.5 Fifth round

The new fuzzy preference relations areappear in [12]. The Jaccard distance function results in a consensus degree level of 0.78, the consensus reaching process stops and the selection process is activated to derive the solution of consensus.