An influence analysis of the number of members on the quality of knowledge in a collective

Abstract

Collective knowledge is understood as the common knowledge state of a collective consisting of autonomous units. The knowledge states referred from these autonomous units to some degree reflect the real knowledge state of a subject in the real world, but it is not known to what degree because of incompleteness and uncertainty. Although collective knowledge determination is an important task because these knowledge states can be different from each other, there exists another important issue with its quality. The quality of collective knowledge is based on the difference between the real knowledge state and the collective knowledge. In this study, we investigate the influence of the number of collective members on the quality of collective knowledge. Through experimental analysis, the larger collective we use, the better the quality of collective knowledge will be. In other words, the large number of collective members positively affects the quality of collective knowledge. Besides, some theorems about the relationship between the collective knowledge and the knowledge states in a collective, the influence of adding or removing members on the quality of collective knowledge are also proved.

Keywords

Collective knowledge consensus choice inconsistency knowledge

1 Introduction

The problem of using multi-autonomous units (collective members) such as agents, experts, etc., for solving some common subjects in the real world is increasing popular. For example, in multi-agent technology a set of autonomous sources are used in making decision processes [5 , 20]; or in clinical decision support systems where tasks are performed by taking into account a number of sources consisting of a set of databases and a set of data sources [47]. On the one hand, this phenomenon is useful in giving a proper solution [49, 51]. Since a large collective may have additional knowledge that a collective with small members do not possess on their own. This knowledge may be relevant to the solving subject [52] and it can be reused for solving similar subjects in the future [48]. On the other hand, this phenomenon also causes conflicts because collective members can have different (or inconsistent) elements of knowledge about the same subject in the real word. This is due to the fact that each collective member can be a good specialist in one or two dimensions whereas subjects in the real world aremulti-dimensional.

The concept of collective knowledge has remained controversial [44]. There exist many concepts about the collective knowledge such as belief justified true or acceptable by most members in a collective [45]; the sum of shared contributions among collective members [42] or the knowledge of a collective as a whole [37]. In this study, a collective is understood as a set of knowledge states referred from collective members on the same subject. The real knowledge state of the subject exists but is not known by collective members when they are asked to give their knowledge states about the subject. Thus, the knowledge states in a collective to some degree reflect the real knowledge because of incompleteness and uncertainty. The knowledge of a collective as a whole (collective knowledge) is determined on the basis of the knowledge states in a collective [37]. In collective knowledge determination, based on the way in which the real knowledge state exists, there may be two possible cases:

The real knowledge state exists dependently on the knowledge states in a collective.

The real knowledge state exists independently of the knowledge states in a collective.

The first case is related to the problem of election committee, group decision making, etc. In this case the collective knowledge and the real knowledge state are identical. Thus, we name this case as subjective case. For the second case, however, the existing of the real knowledge state is independent of the knowledge states in a collective such as in the problem of weather forecast for a future day, the currency rate in the next month, etc. The collective knowledge of a collective to some degree reflects the real knowledge state of a subject in the real world. Thus, we name it as objective case.

In [35] Nguyen has worked out many consensus-based algorithms which serve for determining collective knowledge of a collective for different knowledge representations such as logical, relational structures, ontology, etc. Consensus choice has usually been understood as a general agreement in situations where participants have not agreed on some matters [10]. It does not require a full degree of agreement i.e. unanimity [23] because unanimity is difficult to achieve in practice. In this study, however, we don’t aim to propose a consensus model to reach a high level agreement between collective members’ knowledge states as in [11 , 56]. Instead, a consensus model is used to determine a knowledge state which is considered as the representative of the knowlege states in a collective. Consensus models have been proved useful in solving conflicts arising in the process of collective knowledge determination [3 , 32].

The general research problem of the quality of collective knowledge for objective case is described in Fig. 1 as follows:

For a given subject in the real world, a set of collective members are asked to give their opinions (knowledge states). These opinions to some degree reflect the real knowledge state of the subject. Then collective knowledge is determined from the basis of these knowledge states. There exist two most important factors affecting the quality of collective knowledge: the inconsistency degree of a collective and the number of collective members. Inconsistency degree presents the coherence and density levels of the knowledge states in a collective [35]. In the previous works [35, 36] based on the Euclidean space, we have proved that the collective knowledge is better than the worst knowledge state in a collective; in case the distances from all knowledge states to the real state are identical, the collective knowledge is the best one in comparison with all knowledge states in a collective. In addition, through simulation [39], the authors have concluded that the larger the number of collective members, the better the quality of collective knowledge. In [41] with some restrictions, the hypothesis “the higher the inconsistency degree, the better the quality of collective knowledge” is true. In other words, the inconsistency degree positively affects the quality of collective knowledge.In [16, 25], the inconsistency degree also has positive influence on new knowledge elements which arisen in the process of collective knowledge determination. It is due to the fact that a homogeneous collective involving members with the same knowledge states about a real world subject can not generate any new additional knowledge elements.

Beyond inconsistency degree, the problem of the influence of the number of collective members on the quality of collective knowledge is also an important issue. In some situations small collectives can give better solutions as in [14, 21]. In other situations, however, large collectives play an important role in giving better solutions as in [7 , 53]. For example, we consider the following collectives when collective members are invited to give their answers for the question “Will it be rainy tomorrow in Wroclaw, Poland?” as follows: X = {yes, no}, Y = {yes, no, yes}, Z = {yes, no, yes, yes}. With collective X, it is difficult to infer which one can be chosen as the representative of the collective. However, for collective Y, “yes” is more probable and it is the most probable in the case of collective Z. In this example, the number of collective members plays an important role not only in determining the representative of a collective but also in evaluating the quality of that representative. Thus, in this work, we will investigate the influence of the number of collective members on the quality of collective knowledge for objective case (with assumption the cost for inviting/implementing more collective members is acceptable). This approach can be considered as a revised and expanded version of the papers from ACIIDS 2015, 2016 [39, 40] with three separate datasets in the field of wisdom of crowds. Through experimental analysis, the large number of collective members has positive impact on the quality of collective knowledge. However, the experiments also reveal that the difference between the real knowledge state and the collective knowledge of a collective becomes insignificant when the number of collective is large enough. This means that the increase of the number of collective members should not lead the collective knowledge of a collective to be closer to the real knowledge state. In addition, some theorems about the relationship between the collective knowledge and the knowledge states in a collective, the influence of adding or removing members on the quality of collective knowledge are also proved. These are the main contributions of the paper.

The remaining part of this paper is organized as follows. In Section 2, some related works to the influence of the number of collective members on the quality of collective knowledge are briefly introduced. Section 3 revises some basic notions related to consensus choice, collective of knowledge states, knowledge of a collective, and a method for measuring the quality of collective knowledge. Section 4 presents the proposed method of the paper, the experimental results and their evaluation. Finally, some conclusions and future works are pointed out in Section 6.

2 Related works

The problem of using a large collective to solve some common problems in the real world has been being more popular in recent years, after the influential book The Wisdom of Crowds has been published by Surowiecki [51]. However, maybe it has existed for a very long time, especially in the field of psychology. For instance, in 1907 Galton conducted a contest about guessing the weight of an ox [13]. Through experimental analysis, the collective guess was only 9 pounds off the actual weight. Similarly, in case of ranking the weights of the objects, introduced by Gordon in [15], the average of all collective members rankings is 0.79 for collectives of ten; 0.86 for collectives of twenty; and for collectives of fifty it was 0.94.

Recently, there exist a lot of statistical results which have shown that a collective is better than a single member in solving some common problems in the real world. According to [49], a statistical result from the game show “Who wants to be a millionaires?” only 65% of answers given by experts were on target, while 91% of answers given by the public were correct. This is due to the fact that a single member can be a good specialist in one or two dimensions, whereas problems in the real world are multi-dimensional. Then the author has put forward a hypothesis that a collective is more intelligent than an individual. Later, this hypothesis has been confirmed by Surowiecki in [51]. In that work the author has described several experiments which served for proving the statement the reconciled decision of a collective is in general more proper than individual decisions. For instance, a number of collective members have been asked to guess how many beans are in a jar. The average of results given by the collective members is 871 grains and the actual number of beans in the jar is 850; the error is only 2.5%. In [53] the authors have invited a collective of 500 members (478 valid guesses) for solving some common problems such as forecast temperature, guessing the weights of specific amounts of coffee, milk, gasoline, air, and gold. Through experimental analysis with different numbers of collective members such as 10, 20, 50, 100, and 200, the authors have concluded that the large collective has positive influence on the collective judgment quality. In [7] through experiments using prediction markets the authors have highlighted the important role of the number of collective members in producing a good forecast. Concretely, the large collectives (18 members) perform substantially better than small collectives (8 members). The success of these works is considered as based on the Law of Large Numbers which has been proved useful in error reduction in collective intelligence [22 , 50]. Besides, in [55] the authors have stated that both speed and accuracy of decision making increase with the number of collective members.

In addition to these related works, this study presents another approach to determine the influence of the number of collective members on the quality of collective knowledge. For this aim, we will increases the number of collective members from minimal value (n) up to maximal value (m), each step by k members. This is helpful in determining a number of collective members such that its increase does not lead the difference between the collective knowledge and the real knowledge state becomes significant. In addition, the knowledge states in the collective at step t are involved in the collective at step (t + 1). In addition, based on Manhattan distance space, the influence of adding or removing members on the quality of collective knowledge is also formally proved.

3 Background

3.1 Collective of knowledge states

By U we denote a set of objects representing the potential elements of knowledge referring to a concrete real-world subject. The elements of U can represent, for example, logic expressions, tuples, etc. Symbol 2^U denotes the powerset of U that is the set of all subsets of U. By $\prod_{k} (U)$ we denote the set of all k-element subsets (with repetitions) of set U for k ∈ N (N is a set of natural numbers), and let $\prod (U) = ⋃_{k = 1}^{\infty} \prod_{k} (U)$ (1)

Thus ∏ (U) is the set of all non-empty finite subsets with repetitions of set U. A set X ∈ ∏ (U) can represent the knowledge states in a collective given by collective members on the same subject where each element x ∈ X represents the knowledge state of a collective member. We also call X a collective. Set U can contain elements which are inconsistent with each other. In this work, a collective is described as follows: $X = {x_{i} = (x_{i 1}, x_{i 2}, \dots, x_{im}) : i = 1, 2, \dots, n}$ (2)

where x_ik ∈ R, k = 1, 2, …, m, m is the number of dimensions.

It is a multi-dimensional vector including a set of collective members’ opinions about a real-world subject. Concretely, we consider the following example about the forecasts of Euro to Dollar Exchange Rate (EUR/USD).

Example 1. The forecasts of Euro to Dollar Exchange Rate (EUR/USD) for end-of-quarter 2015 through 2016 1 from institutions as described in Table 1 as follows:

Definition 1. For a given collective X ∈ ∏ (U) as follows: $X = {x_{1}, x_{2}, \dots, x_{n}}$ (3)

Collective X is called:

homogeneous, if all its knowledge states are identical, that isx_i = x_jforalli, j ∈ 1, 2, …, n(nis a natural number,n ∈ N).

heterogeneous, if it is not homogeneous. That is ∃x_i, x_j ∈ X : x_i ≠ x_j.

Elements of U have two structures. They are: macrostructure and microstructure. The microstructure is considered as the structure of elements of the set U such as: linear orders [1 , 57], semi-lattices [4], n-tree [10, 54], ordered partitions and coverings [8], incomplete ordered partitions [17], non-ordered partitions [2], weak hierarchies [26], time interval [30, 38], etc. The macrostructure is understood as the relationship between elements in a collective. In general, the macrostructure is often defined by distance functions for measuring difference between elements in a collective. The definition of the macrostructure is as follows:

Definition 2. The macrostructure of the set U is a distance function $d : U \times U \to [0, 1]$ (4)

which is:

Nonnegative, i.e. $\forall x, y \in U : d (x, y) \geq 0,$

Reflexive, i.e. $\forall x, y \in U : d (x, y) = 0 if x = y,$

Symmetrical, i.e. $\forall x, y \in U : d (x, y) = d (y, x)$

where [0,1] is the closed interval of real numbers between 0 and 1. Pair (U, d) is called a distance space. The definition of a distance function is independent of the structure of elements of U.

In this work, the Manhattan distance function is used for determining a representative of each collective (collective knowledge) and proving some theorems about the relationship between the collective knowledge and the knowledge states in a collective; the influence of adding or removing members on the quality of collective knowledge.

3.2 Knowledge of a collective

As aforementioned, the knowledge of a collective (or collective knowledge) is considered as the consensus of members’ knowledge states in a collective. It is considered as the representative of a collective. In [33, 35] the author has presented many criteria for collective knowledge determination [35] such as: reliability, unanimity, simplification, quasi-unanimity, consistency, condorcet consistency, general consistency, proportion, 1-optimality and 2-optimality (or O₁ and O₂ for short). Criteria O₁ and O₂ are the most popular criteria used in collective knowledge determination because of satisfying these criteria it implies satisfying the majority of other criteria [32, 35]. There exist a lot of consensus-based algorithms which have been proposed for determining the knowledge of a collective for different knowledge representations such as ordered partitions [9], relational structure [33, 35], interval numbers [30, 31], hierarchical structure [24], ontology [35], etc.

Definition 3.For a given collectiveX, the collective knowledge ofXis determined by:

criterion O ₁ if: $d (x^{*}, X) = min_{y \in U} d (y, X)$

criterion O ₂ if: $d^{2} (x^{*}, X) = min_{y \in U} d^{2} (y, X)$

where x^* is the collective knowledge X, d (x^*, X) is the sum of distances from x^* to elements in collective X and d² (x^*, X) is the sum of squared distances from x^* to elements in collective X.

In case of satisfying criterion O₂, x^* has the following form: $x^{*} = \frac{1}{n} (\sum_{i = 1}^{n} x_{i 1}, \sum_{i = 1}^{n} x_{i 2}, \dots, \sum_{i = 1}^{n} x_{im})$ (5)

In this study, criterion O₁ will be used to determine the collective knowledge of a collective while some theorems as mentioned in Introduction section will be proved based on criterion O₂.

3.3 Quality of collective knowledge

For objective case, the real knowledge state of a subject is independent of the knowledge states in a collective and the collective knowledge reflects the real knowledge state to some degree. Thus, the quality of collective knowledge is based on the distance from the knowledge of a collective to the real knowledge state [35, 36]. In this case, the best collective knowledge is the closest one to the real knowledge state.

From Fig. 2, the quality of collective knowledge is measured by taking into account the distance from the collective knowledge to the real knowledge state. Its definition is as follows: $Q_{X} = 1 - d (r, x^{*})$ (6) where x^* represent the collective knowledge of collective X and r represents the real knowledge state. According to above definition, Q_X belongs to [0, 1]. The closer the collective knowledge to the real knowledge state is, the higher quality of collective knowledge will be. The following theorem presents the relationship between the collective knowledge and the knowledge states in a collective.

Theorem 1. For a given collective including n members $X = {x_{i} = (x_{i 1}, x_{i 2}, \dots, x_{im}) : i = 1, 2, \dots,, n}$

Let vector $x^{*} = \frac{1}{n} (\sum_{i = 1}^{n} x_{i 1}, \sum_{i = 1}^{n} x_{i 2}, \dots, \sum_{i = 1}^{n} x_{im})$

be its knowledge satisfying criterion O ₂ ; then $d (r, x^{*}) \leq \frac{d (r, X)}{n}$

Proof. For proving this theorem, we consider the Manhattan distance space. $\begin{matrix} d (r, x^{*}) & = & \frac{1}{n} (| \sum_{i = 1}^{n} x_{i 1} - {nr}_{1} | + | \sum_{i = 1}^{n} x_{i 2} - {nr}_{2} | \\ + \dots + | \sum_{i = 1}^{n} x_{im} - {nr}_{m} |) \\ = & \frac{1}{n} (\sum_{j = 1}^{m} (| \sum_{i = 1}^{n} x_{ij} - {nr}_{j} |)) \end{matrix}$

For any dimension j, we know that: $| \sum_{i = 1}^{n} x_{ij} - {nr}_{j} | \leq \sum_{i = 1}^{n} | x_{ij} - r_{j} |$

Then, $\begin{matrix} d (r, x^{*}) & \leq & \frac{1}{n} (\sum_{i = 1}^{n} | x_{i 1} - r_{1} | + \sum_{i = 1}^{n} | x_{i 2} - r_{2} | \\ + \dots + \sum_{i = 1}^{n} | x_{im} - r_{m} |) \end{matrix}$

In conclusion, the following dependency is true: $d (r, x^{*}) \leq \frac{d (r, X)}{n}$

According to Theorem 1, the distance from the collective knowledge to the real knowledge state does not exceed the average of the distances from the knowledge states in a collective to the real knowledge state. In other words, if collective X is not homogeneous, then the collective knowledge is closer to the real knowledge state than the average of distances from the elements in a collective to the real knowledge state. Thus, the average of distances from the knowledge states in a collective to the real knowledge state can be considered as the upper limit of the difference between the real knowledge state and the collective knowledge. In the next section, we will propose a method to determine the influence of the number of collective members on the quality of collective knowledge for objective case.

4 The influence of the number of members on the quality of collective knowledge

4.1 The proposed method

The problem of analyzing the influence of the number of collective members on the quality of collective knowledge is described as follows: for a given subject in the real world, a set of collective members are asked to give their opinions (knowledge states). These knowledge states reflect the real knowledge state (which is not known to them) of the subject to some degree. To determine the influence of the number of collective members on the quality of collective knowledge, a collective consisting of minimal members will be increased up to maximal members, each step by k members. The detailed procedure of the proposed method is described in Fig. 3 as follows:

We start with a collective consisting of the minimal number of collective members (n). Then the number of collective members is increased up to the maximal number of collective members (m). Y_j is an added collective with k members. At the end of this process we have (m - n)/k collectives. Then criterion O₁ is used to determine a representative (collective knowledge) of each collective. As aforementioned, the quality is based on the distance from the collective knowledge to the real knowledge state. The best collective knowledge is the closest one to the real knowledge state. Based on the difference between the real knowledge state and a set of collective knowledge of (m - n)/k collectives we will determine whether the large number of collective members positively affects the quality of collective knowledge?

4.2 Experimental results

4.2.1 Datasets

In this study, we use three separate datasets from the field of wisdom of crowds to determine the influence of the number of collective members on the quality of collective knowledge. The first dataset has done by Gideon Rosenblatt 2 about guessing the total pieces of cereal in a weird-looking glass vase. The second one is guessing the weight of a cow 3 . The last one is about guessing how many coins were in a huge jar 4 . The details of these datasets are described in Table 2.

In these datasets, of course, the real state (actual value) is not known by collective members when they are asked to give their opinions. Thus, the collective opinion (collective knowledge) of the collective consisting of these opinions may reflect the real state to some degree.

4.2.2 Experimental results and their evaluation

In this section, we will present the experimental results of the proposed method with above datasets. Since the problem of choosing the number of collective members should be enough for a given subject is a difficult and complicated task. Thus, the minimal number of collective members is beginning with an odd number as mentioned in [34, 35]. First, the experiments are performed with collectives (from 3 to 331 members, k = 1). For each collective size we repeat with 50 collectives. Then with each collective we calculate the difference between the collective knowledge and the real knowledge state (d (r, x^*)) and the average of the differences accross each 50 collectives. Criterion O₁ is used to determine the collective knowledge of each collective.

In [35] the distance between elements of set U is normalized to interval [0,1]. However, according to Table 2, the difference between the min and max values is too large (such as 1 and 14,555 in case of cow weight dataset). Thus, the normalization of whole dataset leading the difference between members in a collective or between collective knowledge and the real knowledge state are too small. In addition, the difference seems to become insignificant when the number of collective members is increased. According to Fig. 2, the better quality of collective knowledge is considered as the closer one to the real knowledge state. Thus, in this study, the distance between two elements of set U belongs to $R$ . In the following figures (Figs. 4–7), d_rx presents the distance from the collective knowlege of a collective to the real knowledge state (on the y-axis). The x-axis presents the number of collectivemembers.

From these figures we can state that the number of collective members has influence on the quality of collective knowledge. Concretely, if we increase the number of members in a collective, then the difference between the real knowledge state and the collective knowledge tends to be down. However, with dataset cow weight, when the number of collective members increases up to 331, the influence of the number of collective members to the quality of collective knowledge seems insignificant in comparison to the case of increasing the number of collective members up to 100.

Another problem is whether the relationship between the number of collective members and the quality of collective knowledge is significant? For this aim, a statistical test (correlation coefficient) is used to evaluate the strength of the relationship. The data from our experiments do not come from a normal distribution (according to the Shapiro-Wilk tests). Therefore the Spearman correlation coefficient is used for measuring the strength of the relationship.

From Table 3 the relationship between the number of collective members and the difference between the collective knowledge of a collective and the real knowledge state is very strong (–0.957 for Cereal vase, –0.818 for Coin jar, and –0.746 for Cow weight). In addition, the p-values are very small. Therefore, we can conclude that the correlation coefficient in this case is statistically significant. Generally, with some restrictions, the hypothesis the large number of collective members positively affects the quality of collective knowledge. In other words, the large number of collective members is helpful in reducing the difference between the collective knowledge of a collective and the real knowledge state.

In addition, the experiments also reveal that the difference between the real knowledge state and the collective knowledge of a collective become insignificant when the number of collective is large enough. It means that the increase of the number of collective members does not lead the collective knowledge of a collective to be closer to the real knowledge state. Concretely, we consider the following figures:

Figures 8–10 present the experiments with the number of collective members from 3 to 353 and in each step the number of collective members is increased by k = 50 members (not 1 member as above figures). According to these figures the difference between the real knowledge state and the collective knowledge seems become insignificant in case the number of collective members is larger than 100. However, this phenomenon should be formally proved and should be the subject of future work. Owning to it we can determine the number of collective members should be enough for a given subject in the real world. This problem has very high practical impact.

In the next section, we will prove some theorems about the influence of increasing or decreasing the number of collective members on the quality of collective knowledge in case of satisfying criterion O₂.

Theorem 2.For collectivesX, Y, Zas follows: $\begin{matrix} X & = & {x_{i} = (x_{i 1}, x_{i 2}, \dots, x_{im}) : i = 1, 2, \dots, n} \\ Y & = & {X \cup {y} : y = (y_{1}, y_{2}, \dots, y_{m})} \\ Z & = & {X ∖ {z} : z = (z_{1}, z_{2}, \dots, z_{m})} \end{matrix}$

Let x^*, y^*, and z^* are collective knowledge of collectives (satisfying criterionO₂) X, Y, Z respectively.

If d (r, y) ≤ d (r, x^*), then d (r, y^*) ≤ d (r, x^*).

If d (r, z) ≤ d (r, x^*), then d (r, z^*) ≥ d (r, x^*).

Proof. In case of satisfying criterion O₂x^*, y^*, andz^* have the following forms: $\begin{matrix} x^{*} & = & \frac{1}{n} (\sum_{i = 1}^{n} x_{i 1}, \sum_{i = 1}^{n} x_{i 2}, \dots, \sum_{i = 1}^{n} x_{im}) \\ y^{*} & = & \frac{1}{n + 1} (\sum_{i = 1}^{n} x_{i 1} + y_{1}, \sum_{i = 1}^{n} x_{i 2} + y_{2}, \\ \dots, \sum_{i = 1}^{n} x_{im} + y_{m}) \\ z^{*} & = & \frac{1}{n - 1} (\sum_{i = 1}^{n} x_{i 1} - z_{1}, \sum_{i = 1}^{n} x_{i 2} - z_{2}, \\ \dots, \sum_{i = 1}^{n} x_{im} - z_{m}) \end{matrix}$

(a) For 2a): $\begin{matrix} d (r, x^{*}) & = & \sum_{j = 1}^{m} | \frac{1}{n} \sum_{i = 1}^{n} x_{ij} - r_{j} | \\ = & \frac{1}{n} \sum_{j = 1}^{m} | \sum_{i = 1}^{n} x_{ij} - {nr}_{j} | \\ d (r, y^{*}) & = & \sum_{j = 1}^{m} | \frac{1}{n + 1} (\sum_{i = 1}^{n} x_{ij} + y_{j}) - r_{j} | \\ = & \frac{\sum_{j = 1}^{m} | (\sum_{i = 1}^{n} x_{ij} - {nr}_{j}) + (y_{j} - r_{j}) |}{n + 1} \end{matrix}$

We need to prove: $\begin{matrix} \sum_{j = 1}^{m} | \frac{1}{n} \sum_{i = 1}^{n} x_{ij} - r_{j} | \\ \geq \sum_{j = 1}^{m} | \frac{1}{n + 1} (\sum_{i = 1}^{n} x_{ij} + y_{j}) - r_{j} | \end{matrix}$

From d (r, y) ≤ d (r, x^*), we have: $\begin{matrix} \sum_{j = 1}^{m} | y_{j} - r_{j} | & \leq & \sum_{j = 1}^{m} | \frac{1}{n} \sum_{i = 1}^{n} x_{ij} - r_{j} | \\ \leq & \sum_{j = 1}^{m} | \frac{1}{n} (\sum_{i = 1}^{n} x_{ij} - {nr}_{j}) | \end{matrix}$

By adding $\sum_{j = 1}^{m} | \sum_{i = 1}^{n} (x_{ij} - r_{j}) |$ to both sides of above formula, then we have: $\begin{matrix} \sum_{j = 1}^{m} | y_{j} - r_{j} | + \sum_{j = 1}^{m} | \sum_{i = 1}^{n} (x_{ij} - r_{j}) | \\ \leq (1 + \frac{1}{n}) \sum_{j = 1}^{m} | (\sum_{i = 1}^{n} x_{ij} - {nr}_{j}) | \end{matrix}$

By applying |a| + |b| ≥ |a + b|, the left side of above formula is: $\begin{matrix} \sum_{j = 1}^{m} | y_{j} - r_{j} | + \sum_{j = 1}^{m} | \sum_{i = 1}^{n} (x_{ij} - r_{j}) | \\ \geq \sum_{j = 1}^{m} | \sum_{i = 1}^{n} (x_{ij} - r_{j}) + (y_{j} - r_{j}) | \end{matrix}$

In conclusion, we have: $\begin{matrix} \frac{\sum_{j = 1}^{m} | \sum_{i = 1}^{n} (x_{ij} - r_{j}) + (y_{j} - r_{j}) |}{n + 1} \\ \leq \frac{\sum_{j = 1}^{m} | (\sum_{i = 1}^{n} x_{ij} - {nr}_{j}) |}{n} \end{matrix}$

In other words, the following dependency is true: d (r, y^*) ≤ d (r, x^*)

(b) For 2b): $\begin{matrix} d (r, z^{*}) & = & \sum_{j = 1}^{m} | \frac{1}{n - 1} (\sum_{i = 1}^{n} x_{ij} - z_{j}) - r_{j} | \\ = & \frac{\sum_{j = 1}^{m} | (\sum_{i = 1}^{n} x_{ij} - {nr}_{j}) - (z_{j} - r_{j}) |}{n - 1} \end{matrix}$

We need to prove: $\begin{matrix} \sum_{j = 1}^{m} | \frac{1}{n - 1} (\sum_{i = 1}^{n} x_{ij} - z_{j}) - r_{j} | \\ \geq \sum_{j = 1}^{m} | \frac{1}{n} \sum_{i = 1}^{n} x_{ij} - r_{j} | \end{matrix}$

By applying |a - b| ≥ |a| - |b| we have the following dependency: $\begin{matrix} (n - 1) d (r, z^{*}) \\ \geq \sum_{j = 1}^{m} (| (\sum_{i = 1}^{n} x_{ij} - {nr}_{j}) | - | z_{j} - r_{j} |) \end{matrix}$

In addition, $\begin{matrix} d (r, x^{*}) - d (r, z) \\ = \sum_{j = 1}^{m} (| (\sum_{i = 1}^{n} x_{ij} - {nr}_{j}) | - | z_{j} - r_{j} |) \end{matrix}$

According to the condition of removing member z, d (r, z) ≤ d (r, x^*), we have: $nd (r, x^{*}) - d (r, z) \geq nd (r, x^{*}) - d (r, x^{*})$

In conclusion, the following dependency is true: $d (r, z^{*}) \geq d (r, x^{*})$

From Theorem 2 if we add members which are closer to the real knowledge state than the collective knowledge of a collective, then the quality of new collective (after adding that member) will be better. Conversely, the quality is worse in case of removing members which are closer to the real knowledge state than the collective knowledge of the collective. In addition, if we add a member which is the collective knowledge of a collective to that collective, then the quality of collective knowledge is unchanged. Concretely, for a given collective X, Y as follows: $\begin{matrix} X & = & {x_{i} = (x_{i 1}, x_{i 2}, \dots, x_{im}) : i = 1, 2, \dots, n} \\ Y & = & X \cup {x^{*}} \end{matrix}$ where x^* and y^* are their collective knowledge respectively.

According to [35], x^* and y^* are identical. That is: $d (r, x^{*}) = d (r, y^{*})$

Thus, the quality of collective knowledge is not better or worse in case of adding the collective knowledge to the collective. In other words, in case of objective case, this member is not helpful in improving the quality of collective knowledge. It is only useful in improving the consistency degree of a collective.

5 Conclusions and future works

In this paper, we have investigated the influence of the number of collective members on the quality of collective knowledge for objective case. Through experimental analysis, the number of collective members positively influences the quality of collective knowledge. However, the experiments also reveal that the difference between the real knowledge state and the collective knowledge of a collective seems to become insignificant when the number of collective is large enough. In this case, the increase of the number of collective members should not lead the collective knowledge of a collective to be closer to the real knowledge state. In addition, the following statements have been formally proved. The distance from the collective knowledge to the real knowledge state does not exceed the average of distances from the knowledge states in a collective to the real knowledge state. The quality of collective knowledge is better in case of adding members which are closer to the real knowledge state than the collective knowledge of a collective. Conversely, the quality is worse in case of removing this kind of members from the collective.

For future work, we will investigate the problem of analyzing the influence of the number of collective members on the quality of collective knowledge by taking into account the inconsistency degree of a collective. The number of collective members should be enough for solving some common problems in the real world are also considered as the future work. For these problems, paraconsistent logics could be useful [27].

Footnotes

Acknowledgments

This work was supported by the 2014 Yeungnam University Research Grant. We would like to thank Gideon Rosenblatt, Quoc Trung Bui, Erik Steiner for sharing datasets; anonymous reviewers for their valuable comments.

References

Arrow

K.J.

, Social choice and individual values. Wiley, New York, 1963.

Barth’elemy

and Leclerc

, The median procedure for partitions, 1995.

Barthelemy

J.P.

, Guenoche

and Hudry

, Median linear orders: Heuristics and a branch and bound algorithm, European Journal of Operational Research42 (1989), 313–325.

Barthélemy

J.P.

and Janowitz

M.F.

, A formal theory of consensus, SIAM Journal on Discrete Mathematics4 (1991), 305–322.

Bosse

, Jonker

C.M.

, Schut

M.C.

and Treur

, Collective representational content for shared extended mind, Cognitive Systems Research7 (2006), 151–174.

Castelfranchi

, Modelling social action for AI agents, Artif Intell103 (1998), 157–182.

Cui

, Gallino

, Moreno

and Zhang

D.J.

, The operational value of social media information, Available at SSRN (2015).

Daniłowicz

, Nguyen

H.C.

and Nguyen

N.T.

, Model of User Profiles and Personalization for Web-Based Information Retrieval System. In Abramowicz

, (Ed.), Knowledge-Based Information Retrieval and Filtering from the Web, 746, 2003, pp. 121–136.

Danilowicz

and Nguyen

N.T.

, Consensus-based partitions in the space of ordered partitions, Pattern Recognition21 (1988), 269–273.

10.

Day

W.H.E.

, The Consensus Methods as Tools for Data Analysis. In Block

H.H.

(Ed.), IFC’87: Classifications and Related Methods of Data Analysis, 1988, pp. 317–324.

11.

Eklund

, Rusinowska

and de Swart

, A consensus model of political decision-making, Annals of Operations Research158 (2008), 5–20.

12.

Fishburn

P.C.

, Condorcet social choice functions, SIAM Journal on Applied Mathematics33 (1977), 469–489.

13.

Galton

, Vox populi (The wisdom of crowds), Nature75 (1907), 450–451.

14.

Goldstein

D.G.

, McAfee

R.P.

and Suri

, The wisdom of smaller, smarter crowds. In Proceedings of the Fifteenth ACM Conference on Economics and Computation, 2014, pp. 471–488.

15.

Gordon

, Group judgments in the field of lifted weights, Journal of Experimental Psychology7 (1924), 398–400.

16.

Hecker

, Knowledge beyond the individual? Making sense of a notion of collective knowledge in organization theory, Organization Studies33 (2012), 423–445.

17.

Hernes

and Nguyen

N.T.

, Deriving consensus for hierarchical incomplete ordered partitions and coverings, J UCS13 (2007), 317–328.

18.

Herrera-Viedma

, Cabrerizo

F.J.

, Kacprzyk

and Pedrycz

, A review of soft consensus models in a fuzzy environment, Information Fusion17 (2014), 4–13.

19.

Herrera-Viedma

, Herrera

and Chiclana

, A consensus model for multiperson decision making with different preference structures, Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on32 (2002), 394–402.

20.

Hoen

P.J.T.

and Bohte

S.M.

, COllective INtelligence with Sequences of Actions. In Lavrač

, Gamberger

, Blockeel

& Todorovski

(Eds.), Machine Learning: ECML 2003, 28372003, pp. 181–192.

21.

Kao

A.B.

and Couzin

I.D.

, Decision accuracy in complex environments is often maximized by small group sizes, In Proc R Soc B, 2812014, 20133305.

22.

Kelley

T.L.

, The applicability of the Spearman-Brown formula for the measurement of reliability, Journal of Educational Psychology16 (1925), 300–303.

23.

Kline

J.A.

, Orientation and group consensus, Central States Speech Journal23 (1972), 44–47.

24.

Maleszka

, Mianowska

and Nguyen

N.T.

, A method for collaborative recommendation using knowledge integration tools and hierarchical structure of user profiles, Knowledge-Based Systems47 (2013), 1–13.

25.

Maleszka

and Nguyen

N.T.

, Integration computing and collective intelligence, Expert Syst Appl42 (2015), 332–340.

26.

McMorris

and Powers

, The median function on weak hierarchies, DIMACS Series in Discrete Mathematics and Theoretical Computer Science37 (1997), 265–269.

27.

Nakamatsu

and Abe

, The paraconsistent process order control method, Vietnam Journal of Computer Science1 (2014), 29–37.

28.

Nguyen

N.T.

, Using consensus methods for determining the representation of expert information in distributed systems. In Artificial Intelligence: Methodology, Systems, and Applications, 2000, pp. 11–20.

29.

Nguyen

N.T.

, Using Consensus Methods for Solving Conflicts of Data in Distributed Systems. In Hlaváč

, Jeffery

& Wiedermann

(Eds.), SOFSEM 2000: Theory and Practice of Informatics, 2000, pp. 411–419.

30.

Nguyen

N.T.

, Consensus-based timestamps in distributed temporal databases, The Computer Journal44 (2001), 398–409.

31.

Nguyen

N.T.

, Representation choice methods as the tool for solving uncertainty in distributed temporal database systems with indeterminate valid time, In 2070, 2001, pp. 445–454.

32.

Nguyen

N.T.

, Using consensus for solving conflict situations in fault-tolerant distributed systems. In Cluster Computing and the Grid, 2001 Proceedings First IEEE/ACM International Symposium on, 2001, pp. 379–384.

33.

Nguyen

N.T.

, Consensus system for solving conflicts in distributed systems, Information Sciences147 (2002), 91–122.

34.

Nguyen

N.T.

, Criteria for Consensus Susceptibility in Conflicts Resolving. In Inuiguchi

, Hirano

& Tsumoto

(Eds.), Rough Set Theory and Granular Computing, 125, 2003, pp. 223–232.

35.

Nguyen

N.T.

, Advanced Methods for Inconsistent Knowledge Management. London: Springer-Verlag, 2008.

36.

Nguyen

N.T.

, Inconsistency of knowledge and collective intelligence, Cybernetics and Systems39 (2008), 542–562.

37.

Nguyen

N.T.

, Processing inconsistency of knowledge in determining knowledge of collective, Cybernetics and Systems40 (2009), 670–688.

38.

Nguyen

N.T.

and Danilowicz

, Deriving Consensus for Conflict Data in Web-Based Systems. In Chung

P.H.

, Hinde

& Ali

(Eds.), Developments in Applied Artificial Intelligence, 2718, 2003, pp. 254–263.

39.

Nguyen

V.D.

and Nguyen

N.T.

, A Method for Improving the Quality of Collective Knowledge. In ACIIDS, 2015, pp. 75–84.

40.

Nguyen

V.D.

and Nguyen

N.T.

, An Influence Analysis of the Inconsistency Degree on the Quality of Collective Knowledge for Objective Case. In ACIIDS, 2016, pp. 23–32.

41.

Nguyen

V.D.

, Nguyen

N.T.

and Truong

H.B.

, A preliminary analysis of the influence of the inconsistency degree on the quality of collective knowledge, Cybernetics and Systems (2016), 69–87.

42.

Padula

, Reggiori

and Capetti

, Managing Collective Knowledge in the Web 3.0. In Evolving Internet, 2009 INTERNET ’09 First International Conference on, 2009, pp. 33–38.

43.

Page

S.E.

, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton, NJ: Princeton University Press, 2007.

44.

Ridder

, Epistemic dependence and collective scientific knowledge, Synthese (2013), 1–17.

45.

Rolin

, Science as collective knowledge, Cognitive Systems Research9 (2008), 115–124.

46.

Saint

and Lawson

J.R.

, Rules for Reaching Consensus: A Modern Approach to Decision Making: Jossey-Bass, 1994.

47.

Sanchez

, Toro

, Artetxe

, Graña

, Sanin

, Szczerbicki

, Carrasco

and Guijarro

, Bridging challenges of clinical decision support systems with a semantic approach. A case study on breast cancer, Pattern Recognition Letters34 (2013), 1758–1768.

48.

Sanin

and Szczerbicki

, Towards the construction of decisional DNA: A set of experience knowledge structure jave class within an ontology system, Cybernetics and Systems38 (2007), 859–878.

49.

Shermer

, The science of good and evil: New York: Henry Holt, 2004.

50.

Simons

A.M.

, Many wrongs: The advantage of group navigation, Trends in Ecology & Evolution19 (2004), 453–455.

51.

Surowiecki

, The wisdom of crowds: Anchor, 2005.

52.

Vroom

V.H.

, Leadership and the decision-making process, Organizational Dynamics28 (2000), 82–94.

53.

Wagner

and Suh

, The Wisdom of Crowds: Impact of Collective Size and Expertise Transfer on Collective Performance, In 2014 47th Hawaii International Conference on System Sciences, 2014, pp. 594–603.

54.

Wang

J.T.L.

and Zhang

, Identifying consensus of trees through alignment, Journal of Information Sciences126 (2000), 165–189.

55.

Ward

A.J.W.

, Herbert-Read

J.E.

, Sumpter

D.J.T.

and Krause

, Fast and accurate decisions through collective vigilance in fish shoals, Proceedings of the National Academy of Sciences108 (2011), 2312–2315.

56.

and Xu

, Consensus reaching models of linguistic preference relations based on distance functions, Soft Computing16 (2012), 577–589.

57.

Young

H.P.

, An axiomatization of Borda’s rule, Journal of Economic Theory9 (1974), 43–52.