MSIF: Multi-source information fusion based on information sets

Abstract

Multi-source information fusion is a sophisticated estimating technique that enables users to analyze more precisely complex situations by successfully merging key evidence in the vast, varied, and occasionally contradictory data obtained from various sources. Restricted by the data collection technology and incomplete data of information sources, it may lead to large uncertainty in the fusion process and affect the quality of fusion. Reducing uncertainty in the fusion process is one of the most important challenges for information fusion. In view of this, a multi-source information fusion method based on information sets (MSIF) is proposed in this paper. The information set is a new method for the representation of granularized information source values using the entropy framework in the possibilistic domain. First, four types of common membership functions are used to construct the possibilistic domain as the information gain function (or agent). Then, Shannon agent entropy and Shannon inverse agent entropy are defined, and their summation is used to evaluate the total uncertainty of the attribute values and agents. Finally, an MSIF algorithm is designed by infimum-measure approach. The experimental results show that the performance of Gaussian kernel function is good, which provides an effective method for fusing multi-source numerical data.

Keywords

Multi-source information fusion information sets Shannon entropy uncertainty fuzzy membership degree

1 Introduction

With the continuous enhancement and development of sensor and Internet technologies, the acquisition of data is no longer limited to a single form of data source, but is stored and described in the form of multiple sources [1, 2]. The samples of data from multiple information sources have varied knowledge structures that express information from different points of view [3]. Data from different sources or description information from different angles of the same data will enrich the knowledge structure contained in the data. The information of knowledge structure reflects different perspectives of the learning task in different applications, which facilitates a comprehensive understanding of the multiple information embedded in the data [4 –9]. In view of this, it is necessary to design reasonable and effective multi-source information fusion (MsIF) models and algorithms.

Up to now, lots of MsIF methods have been proposed for complex multi-source data. For example, Yager provided a general framework for MsIF based on a voting like process that tries to adjudicate conflict among the data [10]. Xu and Yu investigated two types of confidence degrees to estimate the reliability of each information source in multi-source information systems [11]. Li and Zhang introduced an MsIF method in a multi-source incomplete information system based on information entropy [12]. To make full use of the information from multiple sources, Sang et al. proposed three kinds of multi-source decision models in multi-source information system [13]. Based on rough learning techniques, Wei and Liang et al. came up with a survey of existing information fusion models and methods from five perspectives, i.e., multi-source, multi-modality, multi-scale, and multi-view information systems [14]. Che et al. employed three methods to solve the information fusion and numerical characterization of uncertain information in multi-source information systems [15]. Huang et al. put forward a new fusion technology based on fuzzy information granulation, which can translate multi-source interval-valued data into trapezoidal fuzzy information granules [16]. From the perspective of granular counting, Zhang et al. studied multi-granularity fusion multi-source information fusion model and multi-source homogeneous information fusion [3 , 18].

Information is the source of knowledge for human beings to understand and transform the world. Because the real world is diverse, complex, and dynamic, people’s expressions of things and information are often imprecise, uncertain, and vague. The variety of information people are exposed to is sometimes deterministic, but more often, it is uncertain. Whether the information is certain or uncertain, it does not matter whether it is good or bad. The problem lies in correctly understanding certainty and uncertainty and grasping the essential law of knowledge uncertainty. In recent years, research on information uncertainty has drawn more and more attention. For example, uncertainty and information system were handled using rough set theory [19 –22], measuring fuzzy and relation uncertainty [23 –25], and a variety of entropy measures uncertainty [26 –28], etc. However, even though there are various fusion methods and models, dealing with the uncertainty in multi-source information fusion is still a challenging problem. In general, the uncertainty resulting from the attribute values (data itself) and the distinctions across information sources is included into the process of fusing numerous information sources. It is worthwhile to research how to decrease uncertainty in the information fusion process.

Information sets [29], proposed by Aggarwal, which is a useful tool for the representation of uncertainty in the granulized attribute values by making use of the entropy framework. In addition, the method is complementary to fuzzy sets to a certain extent, and has more advantages in dealing with uncertainty problems. The main notion of the information sets gives a universal formalism for representing uncertainty to link the value of an information source with the agent’s appraisal of it, which produces an entropy value that we refer to as the “information value”. An information set is a collection of various information values, the sum of which quantifies the uncertainty of the information source. The fundamental idea enshrined in the information set is to unravel uncertainty by the parametric gain function by the values of the information source. It can also give the uncertainty in the possibility distribution (provided by membership function) by capturing this distribution through the information gain. At present, research on information set has been applied in many fields, i.e., decision making [30 –32], face recognition [33, 34] and others [35]. To the best of our knowledge, the fusion of multiple information sources using the information set method has not been reported in any existing literature.

Based on this, this paper proposes the multi-source information fusion model based on information sets. Our main contributions are as follows.

Introducing the idea of information set into MsIF, which is beneficial to multi-source numerical data fusion.

Four different membership degree functions are used as the components of information sets when calculating the entropy functions.

Shannon agent entropy and Shannon inverse agent entropy are defined, their summation can be used to quantify the uncertainty of information sources.

An MSIF algorithm is designed on the basis of infimum-measure strategy. The experimental results show that the proposed algorithm is reasonable and effective.

The rest of this paper is organized as follows. Section 2 briefly introduces some basic concepts about information systems, multi-source information systems and information sets. In Section 3, membership degree functions and uncertainty measurements for agents and attribute values are defined. Multi-source information fusion algorithm based on infimum-measure is designed in Section 4. Section 5 shows experimental results and analyses. Conclusion and future work are given in Section 6.

2 Preliminary

This section shades light on the preliminary definition of information systems, multi-source information systems and information sets. In addition, the symbols and abbreviations involved in the article are shown in Table 1.

Table 1
Abbreviations

Symbols Explanations

U A finite universe U = {x₁, x₂, ⋯ , x_n}

A A non-empty finite attribute set

a (x_j) A attribute value

IS An information system

MsIS A multi-source information system

E _S Shannon entropy

E _a The uncertainty of attribute value

S _a Information set

S_a (x_j) Information value

g_a (x_j) Information gain

μ (x_j) Membership

FS Fuzzy set

SGE Shannon agent entropy

SIGE Shannon inverse agent entropy

TUT The total uncertainty

N The new attribute set

AIS Average information sources

GMD Gaussian membership degree

ZMD Trapezoidal membership degree

TMD Triangular membership degree

SMD Sigmoid membership degree

NIA Neighborhood information amount

NIE Neighborhood information entropy

MsIF Multi-source information fusion

MSIF Multi-source information fusion

based on information sets

Symbols	Explanations
U	A finite universe U = {x₁, x₂, ⋯ , x_n}
A	A non-empty finite attribute set
a (x_j)	A attribute value
IS	An information system
MsIS	A multi-source information system
E _S	Shannon entropy
E _a	The uncertainty of attribute value
S _a	Information set
S_a (x_j)	Information value
g_a (x_j)	Information gain
μ (x_j)	Membership
FS	Fuzzy set
SGE	Shannon agent entropy
SIGE	Shannon inverse agent entropy
TUT	The total uncertainty
N	The new attribute set
AIS	Average information sources
GMD	Gaussian membership degree
ZMD	Trapezoidal membership degree
TMD	Triangular membership degree
SMD	Sigmoid membership degree
NIA	Neighborhood information amount
NIE	Neighborhood information entropy
MsIF	Multi-source information fusion
MSIF	Multi-source information fusion
based on information sets

Definition 1. ([36]) Let (U, A, V) be an information system, where U is a non-empty finite sample set and A is a non-empty finite attribute set. V is called a union of attribute domain, namely, V = ∑_a∈AV_a, where V_a is the attribute domain of the attribute a. An information function a : U × A ⟶ V that satisfies any a ∈ A and x ∈ U, with a (x) ∈ V_a.

Definition 2. ([3]) Let MsIS = {IS_i|IS_i = (U, A, V_i) , i = 1, 2, ⋯ , m} be a multi-source information system, where i) U is a set of objects (non-empty finite set); ii) IS_i is the i-th data table in the MsIS and m is the number of information sources; iii) A is a set of attributes of the i-th IS_i (non-empty finite set); iv) V_a is the value of the attribute a ∈ A; and v) U × A → V_i is an information function for each x ∈ U and a ∈ A, a (x) ∈ V_i in the i-th IS_i where a (x) is the information value with respect to x under attribute a. Moreover, the MsIS can also be expressed by $\begin{matrix} (U, MsIS) = {{IS}_{1}, {IS}_{2}, \dots, {IS}_{m}} . \end{matrix}$ (1)

Definition 3. ([37]) For any x_j ∈ U, the Shannon entropy can be defined as $\begin{matrix} E_{S} = - \sum_{j} p (x_{j}) \log (p (x_{j})), \end{matrix}$ (2) where p (x_j) is the probability of x_j.

Definition 4. ([38]) For any x_j ∈ U, the uncertainty of attribute value a (x_j) w.r.t. attribute a ∈ A is denoted as $\begin{matrix} E_{a} = \sum_{j} a (x_{j}) g_{a} (x_{j}), \end{matrix}$ (3) where g_a (x_j) is called information gain or agent [29].

Definition 5. ([29]) For any x_j ∈ U, the information set can be denoted a collection of attribute values corresponding to the original attribute values, it is given by $\begin{matrix} S_{a} = {a (x_{j}) g_{a} (x_{j})}, \forall x_{j} \in U, \end{matrix}$ (4) where S_a (x_j) = a (x_j) g_a (x_j) is called an information value.

3 Membership degree functions and uncertainty measurements for agent and attribute values in an MsIS

According to [39], anything that can be considered to be sensing its environment qualifies as an agent. Just like animals use their eyes, nose, ears, or other organs to perceive information from the outside world. In the age of artificial intelligence, many machine tools, such as aircraft, robots, and controller stations, mostly use sensors to sense and transmit information. In fuzzy set (FS) theory, a FS can be characterized by a membership function that maps attribute value a (x_j) to the degrees of membership μ (x_j). Therefore, the membership degree can be viewed as the role of an agent. In [29], the term information gain g_a (x_j) and agent are both used interchangeably. For example, Agarwal and Hanmandlu used Hanman-Anirban entropy function to represent the uncertainty of attribute values, and is denoted as $\begin{matrix} g_{a} (x_{j}) = e^{- α (a (x_{j})^{3} + β (a (x_{j}))^{2} + γ (a (x_{j})) + δ)^{λ}}, \end{matrix}$ (5) where α, β, γ, δ and λ are the real-valued parameters belong to k-th fuzzy set [29]. Equation (5) has multiple parameters as an information gain function, which makes the uncertainty representation adaptable. Thereby the gain function can be implemented in any way, including the commonly used membership functions.

Let MsIS = {IS_i|IS_i = (U, A, V_i)} be an MsIS where U = {x₁, x₂, ⋯ , x_n} and A = {a₁, a₂, ⋯, a_l}. For any a ∈ A, the mean attribute values can be denoted as a_mean = $\frac{1}{| U |} \sum_{j = 1}^{n} a (x_{j})$ , x_j ∈ U. σ_a is the standard deviation of attribute values under the attribute a. According to Equation (5), if α = 0, β = 0, $γ = \frac{1}{\sqrt{2} σ_{a}}$ , $δ = - \frac{a_{mean}}{\sqrt{2} σ_{a}}$ , λ = 2, then we have $\begin{matrix} g_{a} (x_{j}) = e^{- (\frac{a (x_{j}) - a_{mean}}{\sqrt{2} σ_{a}})^{2}}, \end{matrix}$ (6) where g_a (x_j) is Gaussian membership degree function. There are many types of membership degree functions, we list several representative membership degree functions, i.e., Gaussian membership degree function, triangular membership degree function, trapezoidal membership degree function and sigmoid membership degree function, as shown in Table 2.

Table 2

Four types of representative membership degree functions

Membership degree functions	Formulas
Gaussian membership degree	$f (x; σ, c) = e^{- \frac{(x - c)^{2}}{2 σ^{2}}}$
Trapezoidal membership degree	$f (x; a, b, c, d) = {\begin{matrix} 0, \\ \frac{x - a}{b - a}, & a \leq x \leq b \\ 1, & b \leq x \leq c \\ \frac{d - x}{d - c}, & c \leq x \leq d \\ 0 & d \leq x \end{matrix}}$
Triangular membership degree	$f (x; a, b, c) = {\begin{matrix} 0, & x \leq a \\ \frac{x - a}{b - a}, & a \leq x \leq b \\ \frac{c - x}{c - b}, & b \leq x \leq c \\ 0 & c \leq x \end{matrix}}$
Sigmoid membership degree	$f (x; α, c) = \frac{1}{1 + e^{- α (x - c)}}$

In what follows, three uncertainty measures are defined to evaluate agents and attribute values.

Definition 6. Let MsIS = {IS_i|IS_i = (U, A, V_i) , i = 1, 2, ⋯ , m} be an MsIS. For any x_j ∈ U, the Shannon agent entropy (SGE) of agent (g_a (x_j)) considering information set of attribute value a (x_j) in i-th NIS_i is defined as $\begin{matrix} {SGE}_{a}^{i} = - \sum_{j}^{n} g_{a} (x_{j}) \log (S_{a} (x_{j})), \end{matrix}$ (7) where S_a (x_j) = a (x_j) g_a (x_j).

Definition 7. Let MsIS = {IS_i|IS_i = (U, A, V_i) , i = 1, 2, ⋯ , m} be an MsIS. For any x_j ∈ U, the Shannon inverse agent entropy (SIGE) of agent (g_a (x_j)) considering information set of attribute value a (x_j) in i-th NIS_i is defined as $\begin{matrix} {SIGE}_{a}^{i} = - \sum_{j}^{n} g_{a} (x_{j}) \log (S_{a}^{'} (x_{j})), \end{matrix}$ (8) where $S_{a}^{'} (x_{j}) = a (x_{j}) g_{a}^{'} (x_{j})$ , and the complement agent is $g_{a}^{'} (x_{j}) = 1 - g_{a} (x_{j})$ .

Definition 8. Let MsIS = {IS_i|IS_i = (U, A, V_i) , i = 1, 2, ⋯ , m} be an MsIS. For any a ∈ A, the total uncertainty (TUT) of the attribute value and the evaluating agent can be denoted by $\begin{matrix} {TUT}_{a}^{i} = {SGE}_{a}^{i} + {SIGE}_{a}^{i}, \end{matrix}$ (9) where $S_{a}^{'} (x_{j}) = a (x_{j}) g_{a}^{'} (x_{j})$ , and the complement agent is $g_{a}^{'} (x_{j}) = 1 - g_{a} (x_{j})$ .

4 Multi-source information fusion based on infimum-measure method

By Definition 8, the larger the value of ${TUT}_{a}^{i}$ , the greater the fuzziness of attribute values and the evaluating agent. Namely, the greater the uncertainty of attribute values and the evaluating agent. Therefore, we always wish to reduce the uncertainty of the attribute values in the multi-source information fusion process to acquire deterministic information. In this context, the infimum-measure method [17] is used to fuse multi-source data to reduce the total uncertainty of fusion process as the following definition.

Definition 9. Let MsIS = {IS_i|IS_i = (U, A, V_i) , i = 1, 2, ⋯ , m} be an MsIS. For any a_k ∈ A, the k-th attribute of new information system after fusion is denoted as $\begin{matrix} Inf (F ({IS}_{1} (a_{k})), F ({IS}_{2} (a_{k})), \dots, \\ F ({IS}_{m} (a_{k}))) \to select V_{a_{k}}^{{IS}_{i}}, \end{matrix}$ (10) where F is called the infimum-measure function where $F = {TUT}_{a}^{i}$ .

There are discrepancies in order of magnitude and dimensionality in real-world datasets. As a result, the numerical attributes of the original information sources are normalized before data processing. In this paper, min-max normalization is used as follows. For any x_j ∈ U, then

$Norm (a (x_{j})) = \frac{a (x_{j}) - {min}_{a (x_{j})}}{{max}_{a (x_{j})} - {min}_{a (x_{j})}},$ (11) where $max_{a (x_{j})}$ and $min_{a (x_{j})}$ are the maximum and minimum attribute values, respectively.

The next step is to perform fuzzy modeling on the normalized data by different membership degree functions, which allow mapping from fuzzy sets to information set. According to the aforementioned definition, a multi-source information fusion algorithm is designed as follows.

Algorithm 1

Multi-source information fusion based on information sets (MSIF)

Require: A MsIS MsIS = {IS_i|IS_i = (U, A, V_i) , i = 1, 2, ⋯ , m}, where |U| = n, |A| = l;

Ensure: A new information system (U, N, V).;

1: Normalizing the original information sources by Equation (11);

2: Calculating the membership degrees by Table 2;

3: Calculating information sets by Equation (4);

4: Calculating SGE by Equation (7);

5: Calculating SIGE by Equation (8);

6: Calculating TUT by Equation (9);

7: Calculating the TUT of the k-th attribute of all information sources by Equation (10), A′ ← ∅ , N ← N ∪ {_{a
_k}};

8: return: N;

By Algorithm 1, Steps 1-4 are to compute the membership degrees of each information source, whose time complexity is O (n × l). Then, for all information sources, the time complexity is O (n × l × m). Steps 5-9 focus on computing different uncertainty measures, whose time complexity is O (n × l). The time complexity of Steps 10-15 is O (l × m). Based on this, the total time complexity of Algorithm 1 is O (n × l × m + n × l + l × m).

Example 1. First, a toy MsIS is randomly generated using Iris 1 data from UCI, as shown in Table 3. Let MsIS = {IS₁, IS₂, IS₃}, where U = {x₁, x₂, x₃, x₄, x₅} denotes six objects and A = {a₁, a₂, a₃, a₄} expresses attribute set.

Table 3

An MsIS

U	IS ₁				IS ₂				IS ₃
	a ₁	a ₂	a ₃	a ₄	a ₁	a ₂	a ₃	a ₄	a ₁	a ₂	a ₃	a ₄
x ₁	5.1	3.5	1.4	0.2	5.1	3.7	1.5	0.4	4.8	3	1.4	0.1
x ₂	4.9	3	1.4	0.2	5	3.5	1.3	0.3	4.3	3	1.1	0.4
x ₃	7	3.2	4.7	1.4	6.2	2.9	4.3	1.3	5	2	3.5	1
x ₄	6.4	3.2	4.5	1.5	5.1	2.5	3	1.1	5.9	3	4.2	1.5
x ₅	6.3	3.3	6	2.5	6.5	3.2	5.1	2	6.9	3.1	5.4	2.1

According to Algorithm 1, the raw information sources in Table 3 are normalized by Equation (11), which is shown in Table 4.

Table 4

The normalized MsIS

U	IS ₁				IS ₂				IS ₃
	a ₁	a ₂	a ₃	a ₄	a ₁	a ₂	a ₃	a ₄	a ₁	a ₂	a ₃	a ₄
x ₁	0.0952	1.0000	0.0000	0.0000	0.0667	1.0000	0.0526	0.0588	0.1923	0.9091	0.0698	0.0000
x ₂	0.0000	0.0000	0.0000	0.0000	0.0000	0.8333	0.0000	0.0000	0.0000	0.9091	0.0000	0.1500
x ₃	1.0000	0.4000	0.7174	0.5217	0.8000	0.3333	0.7895	0.5882	0.2692	0.0000	0.5581	0.4500
x ₄	0.7143	0.4000	0.6739	0.5652	0.0667	0.0000	0.4474	0.4706	0.6154	0.9091	0.7209	0.7000
x ₅	0.6667	0.6000	1.0000	1.0000	1.0000	0.5833	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000

In this example, Gaussian membership degree is used as the agent. Then, by Equation (6), Table 5 lists the Gaussian membership degrees for all attribute values.

Table 5

Gaussian membership degrees of all attribute values in the MsIS

U	IS ₁				IS ₂				IS ₃
	a ₁	a ₂	a ₃	a ₄	a ₁	a ₂	a ₃	a ₄	a ₁	a ₂	a ₃	a ₄
x ₁	0.5813	0.2780	0.5001	0.5464	0.7527	0.4496	0.5899	0.6106	0.8196	0.9089	0.5793	0.4470
x ₂	0.4354	0.3359	0.5001	0.5464	0.6605	0.7284	0.5097	0.5141	0.5017	0.9089	0.4710	0.6937
x ₃	0.4216	0.9702	0.8409	0.9629	0.6226	0.8308	0.7023	0.9043	0.9182	0.1378	0.9737	0.9996
x ₄	0.8499	0.9702	0.8905	0.9270	0.7527	0.3030	0.9996	0.9918	0.8522	0.9089	0.8064	0.8032
x ₅	0.9052	0.9341	0.4383	0.3080	0.3523	0.9956	0.3889	0.2915	0.2551	0.7936	0.3832	0.3297

Subsequently, by Definition 5, all information values of each information source can be calculated as in Table 6.

Table 6

All information values of each information source

U	IS ₁				IS ₂				IS ₃
	a ₁	a ₂	a ₃	a ₄	a ₁	a ₂	a ₃	a ₄	a ₁	a ₂	a ₃	a ₄
x ₁	0.0554	0.2780	0.0000	0.0000	0.0502	0.4496	0.0310	0.0359	0.1576	0.8263	0.0404	0.0000
x ₂	0.0000	0.0000	0.0000	0.0000	0.0000	0.6070	0.0000	0.0000	0.0000	0.8263	0.0000	0.1041
x ₃	0.4216	0.3881	0.6033	0.5024	0.4981	0.2769	0.5545	0.5319	0.2472	0.0000	0.5435	0.4498
x ₄	0.6071	0.3881	0.6001	0.5240	0.0502	0.0000	0.4472	0.4667	0.5245	0.8263	0.5813	0.5622
x ₅	0.6034	0.5605	0.4383	0.3080	0.3523	0.5808	0.3889	0.2915	0.2551	0.7936	0.3832	0.3297

It is important to note that the 0 values have a very small ɛ= 0.0001 to prevent issues with the logarithmic function. In this paper, the base of the logarithmic function is 2. According to Definitions 6-8, the SGE, SIGE and TUT are calculated as in Table 7.

Table 7

The calculation results of SGE, SIGE and TUT

A	IS ₁			IS ₂			IS ₃
	${SGE}_{a_{k}}^{1}$	${SIGE}_{a_{k}}^{1}$	${TUT}_{a_{k}}^{1}$	${SGE}_{a_{k}}^{2}$	${SIGE}_{a_{k}}^{2}$	${TUT}_{a_{k}}^{2}$	${SGE}_{a_{k}}^{3}$	${SIGE}_{a_{k}}^{3}$	${TUT}_{a_{k}}^{3}$
a ₁	10.0094	2.7339	12.7433	16.4321	0.9517	17.3839	11.9991	1.6013	13.6004
a ₂	8.4069	2.6132	11.0201	7.3886	3.0063	10.3949	2.8457	8.6921	11.5379
a ₃	15.0802	2.6641	17.7443	12.0160	1.9772	13.9933	10.9577	2.4160	13.3736
a ₄	16.8648	2.1260	18.9908	12.1941	2.0673	14.2614	10.5516	2.1192	12.6708

Finally, driven by Definition 9, we have

$Inf (12.7433, 17.3839, 13.6004) = 12.7433 \to select V_{a_{1}}^{{IS}_{1}}$ . Therefore, attribute a₁ of the first information source IS₁ can be selected as the attribute $a_{1}^{N}$ of a new information system. Similarly, $Inf (11.0201, 13.9933, 11.5379) = 11.0201 \to select V_{a_{2}}^{{IS}_{1}}$ ; Inf (17.7443, 13.9933, 13.3736) =13.3736 $\to select V_{a_{3}}^{{IS}_{3}}$ , and Inf (18.9908, 14.2614, 12.6708) =12.6708 $\to select V_{a_{4}}^{{IS}_{3}}$ . Let (U, N, V) be a new information system. Then, we have $(U, N, V) = (V_{a_{1}}^{{IS}_{1}}, V_{a_{2}}^{{IS}_{1}}, V_{a_{2}}^{{IS}_{3}}, V_{a_{3}}^{{IS}_{3}}}$ and $N = {a_{1}^{N},$ $a_{2}^{N}, a_{3}^{N}, a_{4}^{N}}$ . It is shown as in Table 8.

Table 8

The new information system (U, N, V)

U	$a_{1}^{N}$	$a_{2}^{N}$	$a_{3}^{N}$	$a_{4}^{N}$
x ₁	5.1	3.5	3	1.4
x ₂	4.9	3	3	1.1
x ₃	7	3.2	2	3.5
x ₄	6.4	3.2	3	4.2
x ₅	6.3	3.3	3.1	5.4

Therefore, a multi-source information system is fused into a new information system, which contains the information of agent and information source itself with minimum uncertainty.

5 Experimental results and analysis

In this section, to demonstrate more clearly the correctness and validity of the conclusions drawn from the cases in the previous section. We conducted some experiments to show the performance of the proposed fusion method in terms of classification accuracy. Six datasets are selected from UCI database 2 , which are shown in Table 9.

Table 9
The description of datasets

Datasets Number of objects Number of attributes Classes

Iris 150 4 3

Ecoli 336 7 8

Glass 214 10 7

Wine 178 13 3

Wpbc 198 34 2

Yeast 1484 8 9

Datasets	Number of objects	Number of attributes	Classes
Iris	150	4	3
Ecoli	336	7	8
Glass	214	10	7
Wine	178	13	3
Wpbc	198	34	2
Yeast	1484	8	9

Given that current public databases rarely involve multi-source data, we employ the technique of introducing white noise and random noise into the initial datasets to get the necessary multi-source data for the experiment. First, q numbers (N₁, N₂, ⋯ , N_q) are generated that satisfy the (N (O, σ)) distribution where σ is the standard deviation. The white noise is added by following formula.

${IS}_{i} (x, a) = {\begin{matrix} IS (x, a) + N_{i}, & if 0 \leq | N_{i} | \leq 1, \\ IS (x, a), & otherwise . \end{matrix}$ In addition, s random numbers (M₁, M₂, ⋯ , M_s) between -M and M are generated, where M is a parameter of random error, it can be denoted as $I S_{i} (x, a) = {\begin{matrix} IS (x, a) + M_{i}, & if 0 \leq | M_{i} | \leq 1, \\ IS (x, a), & otherwise, \end{matrix}$ where IS (x, a) refers to the value of object x with attribute a in the original data, and IS_i (x, a) refers to object x with attribute a in the i-th information source. Then, we add random noise to the remaining 20% of the data in the original information system, add white noise to 40% of the data at random, and leave the remaining 60% of the data untouched. Finally, each original data source can generate m information sources. In this paper, let n = 5. Namely, (U, MsIS) = {IS₁, IS₂, IS₃, IS₄, IS₅}. The fusion process is displayed in Fig. 1.

Fig. 1

The fusion process of the proposed method.

To reflect the effective of the proposed fusion model, seven fusion methods are compared, including AIS, GMD, TMD, ZMD, SMD, NIA [17] and NIE [17]. The seven comparison methods are summarized as follows.

AIS: Average information sources. It can be understood as the average performance of original information sources.

GMD: Gaussian membership degree as the agent. Using GMD (f (x ; σ, c)) for fuzzy modeling, the specific formula is shown in Table 2, where σ and c represent the standard deviation and mean value of objects under attribute a, respectively.

ZMD: Trapezoidal membership degree as the agent. Using ZMD (f (x ; a, b, c, d)) for fuzzy modeling, the specific formula is shown in Table 2, where a,b,c and d represent the minimum, mean, median and maximum values of the objects under attribute a, respectively.

TMD: Triangular membership degree as the agent. Using ZMD (f (x ; a, b, c)) for fuzzy modeling, the specific formula is shown in Table 2, where a,b and c represent the minimum, mean, and maximum values of the objects under attribute a, respectively.

SMD: Sigmoid membership degree as the agent. Using ZMD (f (x ; α, c)) for fuzzy modeling, the specific formula is shown in Table 2, where α = 1, and c represents the mean value of the objects under attribute a.

NIA: Neighborhood information amount. Using NIA to fuse multiple information sources.

NIE: Neighborhood information entropy. Using NIE to fuse multiple information sources.

Since there is no unified method to test the performance of fusion, this paper uses the classification accuracy to evaluate the performance. The classification performance of the fused information source is compared by using three classifiers, i.e., classification and regression trees (CART), k-nearest neighbor (KNN, k=3) and support vector machine (SVM). All classification experiments were derived from ten-fold cross validation. It randomly divides a sample into ten subsets, nine of them are treated as the training set, and the remaining one is regarded as the test set. After 10 rounds, the final performance is obtained by calculating the average and standard deviations of the classification accuracy. All information sources in the experimental datasets were normalized in terms of Equation (11).

The experimental results are displayed in Tables 10-12. “•” indicates that the performance of the fused data is better than that of the average original information source before fusion. From the data in Tables 10-12, we have the following findings:

In Table 10, the classification performance of most fusion methods outperforms AIS. TMD and ZMD perform slightly worse on Wine dataset. SMD, NIA and NIE methods are inferior to AIS on the Yeast dataset. Compared with TMD, ZMD, SMD, NIA and NIE, the performance of the GMD method is the best. This demonstrates the benefits of using GMD as an agent to fuse multiple information sources.

In Table 11, the classification performance of most fusion methods outperforms AIS. The classification performance of ZMD is poor on Wine and Wpbc datasets. However, NIA and NIE perform poorly on the Iris dataset. Overall, the performance of SMD and GMD is the closest, with the average classification performance of 0.7889 and 0.7861, respectively.

In Table 12, GMD, TMD, ZMD and SMD methods perform better in most cases, except for the Glass and Wpbc datasets. The performances of NIA and NIE on Glass and Wpbc datasets are slightly worse than other methods.

In general, the performance of GMD is the best among these three classifiers. This is probably due to the characteristics of the normal distribution of the Gaussian membership function, and it has a good smoothness and symmetry graph. Additionally, the function has no zeros and an obvious physical meaning, which makes it a very good approximation of the membership function. The experimental results show that it is appropriate to use Gaussian membership functions to describe the fuzzy concept of agents. Therefore, it is suggested that the Gaussian membership function can be selected to fuse multiple information sources in real applications.

Table 10

Classification results of seven methods on CART classifier

U	AIS	GMD	TMD	ZMD	SMD	NIA	NIE
Iris	0.9413±0.0061	0.9460±0.0123•	0.9427±0.0173•	0.9433 ± 0.0169•	0.9413± 0.0100•	$\underline{0.9447 \pm 0.0147}$ •	0.9407±0.0097
Ecoli	0.7532±0.0156	$\underline{0.7707 \pm 0.0238}$ •	0.7693±0.0089•	0.7571±0.0113•	0.7799±0.0168•	0.7592±0.0080•	0.7699±0.0151•
Glass	0.8257±0.0110	0.8397±0.0096•	0.8294±0.0150•	0.8318±0.0101•	0.7939±0.0166•	$\underline{0.8425 \pm 0.0115}$ •	0.8463±0.0111•
Wine	0.8783±0.0143	0.9174±0.0231•	0.8674±0.0231	0.8573±0.0262	0.8792±0.0133•	$\underline{0.9101 \pm 0.0161}$ •	0.8949±0.0143•
Wpbc	0.6700±0.0240	0.6833±0.0141•	0.6833±0.0141•	0.6843±0.0282•	$\underline{0.6869 \pm 0.0368}$ •	0.6500±0.0310	0.7111±0.0231•
Yeast	0.4304±0.0105	0.4365±0.0100•	$\underline{0.4349 \pm 0.0111}$ •	0.4321±0.0101•	0.4272±0.0117	0.4276±0.0090	0.4233±0.0097
Avg.	0.7498	0.7656	0.7545	0.7510	0.7514	0.7557	$\underline{0.7644}$

Table 11

Classification results of seven methods on KNN classifier

U	AIS	GMD	TMD	ZMD	SMD	NIA	NIE
Iris	0.9047±0.0077	0.9447±0.0045•	0.9073±0.0091•	0.9047±0.0063•	$\underline{0.9440 \pm 0.0056}$ •	0.8980±0.0055	0.9027±0.0064
Ecoli	0.7955±0.0067	0.8245±0.0061•	0.8015±0.0082•	0.8035±0.0071•	0.7904±0.0044	0.8077±0.0070•	$\underline{0.8113 \pm 0.0060}$ •
Glass	0.8170±0.0073	$\underline{0.8645 \pm 0.0099}$ •	0.8379±0.0085•	0.8332±0.0088•	0.8790±0.0078•	0.8287±0.0079•	0.8355±0.0112•
Wine	0.9361±0.0072	$\underline{0.9455 \pm 0.0076}$ •	$\underline{0.9455 \pm 0.0046}$ •	0.9337±0.0064	0.9478±0.0103•	0.9376±0.0041•	$\underline{0.9455 \pm 0.0070}$ •
Wpbc	0.7035±0.0172	$\underline{0.7056 \pm 0.0180}$ •	$\underline{0.7056 \pm 0.0180}$ •	0.7024±0.0150	0.7449±0.0153•	0.7035±0.0151•	0.6864±0.0169
Yeast	0.4254±0.0052	0.4316±0.0055•	0.4271±0.0050•	0.4293±0.0073•	0.4275±0.0043•	0.4369±0.0031•	$\underline{0.4334 \pm 0.0062}$ •
Avg.	0.7637	$\underline{0.7861}$	0.7708	0.7678	0.7889	0.7688	0.7691

Table 12

Classification results of seven methods on SVM classifier

U	AIS	GMD	TMD	ZMD	SMD	NIA	NIE
Iris	0.9393±0.0038	0.9610±0.0057•	0.9420±0.0082•	0.9487±0.0053•	$\underline{0.9513 \pm 0.0055}$ •	0.9447±0.0045•	0.9393±0.0054•
Ecoli	0.8213±0.0064	0.8224±0.0069•	0.8220±0.0052•	$\underline{0.8279 \pm 0.0048}$ •	0.8388±0.0072•	0.8217±0.0047•	0.8217±0.0051•
Glass	0.7950±0.0067	0.7946±0.0060	0.7977±0.0054 •	$\underline{0.7967 \pm 0.0071}$ •	0.7911±0.0066	0.7864±0.0062	0.7930±0.0070
Wine	0.9596±0.0048	$\underline{0.9713 \pm 0.0018}$ •	$\underline{0.9713 \pm 0.0018}$ •	0.9685±0.0047•	0.9736±0.0065•	0.9589±0.0064	0.9592±0.0061
Wpbc	0.7737±0.0113	0.7747±0.0078•	0.7747±0.0078•	0.7732±0.0060	0.7702±0.0131	0.7843±0.0119•	$\underline{0.7763 \pm 0.0072}$ •
Yeast	0.5302±0.0026	0.5398±0.0037 •	0.5316±0.0037•	$\underline{0.5374 \pm 0.0027}$ •	0.5337±0.0039•	0.5276±0.0030	0.5362±0.0023•
Avg.	0.8032	0.8106	0.8066	0.8087	$\underline{0.8098}$	0.8039	0.8043

6 Conclusion

In this paper, a multi-source information fusion method was proposed based on information sets. The information sets can describe both the information source value and the uncertainty information of the agent. We tested the effect of four common agents (GMD, TMD, ZMD and SMD) on the proposed MSIF algorithm. The experimental results showed that GMD has certain advantages in fusing multi-source numerical data. This study can provide a method for possibility modeling and facilitate the further development of multi-source information fusion. This paper only provides a multi-source numerical data fusion model from the perspective of minimum uncertainty. However, related theories and methods still need to be improved. The following three directions need further discussion: 1) How to develop a unified multi-source information fusion model. 2) How to fuse multi-source heterogeneous information. 3) How to develop performance evaluation criteria for multi-source information fusion methods.

Footnotes

Acknowledgements

The authors would like to thank the editors, the anonymous reviewers. This work was supported in part by Guangxi University Young and Middle-aged Teachers Basic Ability Improvement Project under grant 2021KY0648.

References

Pan

, Multi-soure information fusion theory and its applications, Tsinghua University Press, 2013.

Llinas

and Waltz

, Multisensor data fusion, Artech Housse Publisher, 1990.

Zhang

, Li

, Wang

, Luo

, Chen

, Zhang

, Wang

and Yu

, Multi-source information fusion based on rough set theory: A review, Information Fusion 68 (2021), 85–117.

Chen

, Li

, Wang

and Yang

, Domain sentiment dictionary construction and optimization based on multi-source information fusion, Intelligent Data Analysis 24(2) (2020), 229–251.

Liang

, Liang

, Cai

and Bertoni

, The influence factors of the stability of tailings dam based on multi-source information fusion method, Journal of Intelligent & Fuzzy Systems 37(3) (2019), 3365–3372.

, He

, Cao

, Wu

, McCauley

, Balas

V.E.

and Shi

, Multi-source information fusion model in rule-based gaussian-shaped fuzzy control inference system incorporating gaussian density function, Journal of Intelligent & Fuzzy Systems 29(6) (2015), 2335–2344.

Zhang

, Xiao

, Deng

and Jiang

, A multi-source information fusion method for ship target recognition based on bayesian inference and evidence theory, Journal of Intelligent & Fuzzy Systems Preprint (2022), 1–16.

Wang

, Tian

and Jia

, Driving fatigue detection based on feature fusion of information entropy, Journal of Computational Methods in Sciences and Engineering 18(4) (2018), 977–988.

Ah-Pine

, On data fusion in information retrieval using different aggregation operators, Web Intelligence and Agent Systems: An International Journal 9(1) (2011), 43–55.

10.

Yager

R.R.

, A framework for multi-source data fusion, Information Sciences 163(1-3) (2004), 175–200.

11.

and Yu

, A novel approach to information fusion in multi-source datasets: A granular computing viewpoint, Information sciences 378 (2017), 410–423.

12.

and Zhang

, Information fusion in a multi-source incomplete information system based on information entropy, Entropy 19(11) (2017), 570.

13.

Sang

, Guo

, Shi

and Xu

, Decision-theoretic rough set model of multi-source decision systems, International Journal of Machine Learning and Cybernetics 9(11) (2018), 1941–1954.

14.

Wei

and Liang

, Information fusion in rough set theory: An overview, Information Fusion 48 (2019), 107–118.

15.

Che

, Mi

and Chen

, Information fusion and numerical characterization of a multi-source information system, Knowledge-Based Systems 145 (2018), 121–133.

16.

Huang

, Li

, Luo

, Fujita

and Horng

S.-J.

, Dynamic fusion of multisource interval-valued data by fuzzy granulation, IEEE Transactions on Fuzzy Systems 26(6) (2018), 3403–3417.

17.

Zhang

, Li

, Yuan

, Chuan

, Wang

, Liu

and Du

, A data-level fusion model for unsupervised attribute selection in multi-source homogeneous data, Information Fusion 80 (2022), 87–103.

18.

Zhang

, Li

, Luo

and Wang

, Amg-dtrs: Adaptive multi-granulation decision-theoretic rough sets, International Journal of Approximate Reasoning 140 (2022), 7–30.

19.

Al-shami

T.M.

, An improvement of rough sets’ accuracy measure using containment neighborhoods with a medical application, Information Sciences 569 (2021), 110–124.

20.

Al-Shami

T.M.

, Maximal rough neighborhoods with a medical application, Journal of Ambient Intelligence and Humanized Computing (2022), 1–12.

21.

Al-shami

T.M.

, Topological approach to generate newrough set models, Complex & Intelligent Systems (2022), 1–13.

22.

Al-shami

T.M.

, Improvement of the approximations and accuracy measure of a rough set using somewhere dense sets, Soft Computing 25(23) (2021), 14449–14460.

23.

Pal

N.R.

and Bezdek

J.C.

, Measuring fuzzy uncertainty, IEEE Transactions on Fuzzy Systems 2(2) (1994), 107–118.

24.

Z.W.

, Zhang

P.F.

, Ge

, Xie

N.X.

, Zhang

G.Q.

and Wen

C.-F.

, Uncertainty measurement for a fuzzy relation information system, IEEE Transactions on Fuzzy Systems 27(12) (2019), 2338–2352.

25.

Wang

C.Z.

, Huang

, Shao

M.W.

and Chen

D.G.

, Uncertainty measures for general fuzzy relations, Fuzzy Sets and Systems 360 (2019), 82–96.

26.

Zhang

, Li

, Yuan

, Luo

, Liu

and Yang

, Heterogeneous feature selection based on neighborhood combination entropy, IEEE Transactions on Neural Networks and Learning Systems (2022), 1–14.

27.

Liang

J.Y.

and Shi

Z.Z.

, The information entropy, rough entropy and knowledge granulation in rough set theory, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 12(01) (2004), 37–46.

28.

Beaubouef

, Petry

F.E.

and Arora

, Information-theoretic measures of uncertainty for rough sets and rough relational databases, Information Sciences 109(1-4) (1998), 185–195.

29.

Aggarwal

and Hanmandlu

, Representing uncertainty with information sets, IEEE Transactions on Fuzzy Systems 24(1) (2015), 1–15.

30.

Aggarwal

, Rough information set and its applications in decision making, IEEE Transactions on Fuzzy Systems 25(2) (2017), 265–276.

31.

Aggarwal

, Hesitant information sets and application in group decision making, Applied Soft Computing 75 (2019), 120–129.

32.

Deng

and Cui

, An improved belief structure satisfaction to uncertain target values by considering the overlapping degree between events, Information Sciences 580 (2021), 398–407.

33.

Sayeed

and Hanmandlu

, Properties of information sets and information processing with an application to face recognition, Knowledge and Information Systems 52(2) (2017), 485–507.

34.

Singhal

, Hanmandlu

, Vasikarla

, et al., Video-based face recognition with new classifiers, Journal of Modern Physics 12(03) (2021), 361.

35.

Medikonda

, Bhardwaj

and Madasu

, An information set-based robust text-independent speaker authentication, Soft Computing 24(7) (2020), 5271–5287.

36.

Pawlak

, Rough sets, International Journal of Computer & Information Sciences 11(5) (1982), 341–356.

37.

Shannon

C.E.

, A mathematical theory of communication, The Bell System Technical Journal 27(3) (1948), 379–423.

38.

Hanmandlu

and Das

, Content-based image retrieval by information theoretic measure, Defence Science Journal 61(5) (2011), 415.

39.

Russell

S.J.

, Artificial intelligence a modern approach. Pearson Education, Inc., 2010.

MSIF: Multi-source information fusion based on information sets

Abstract

Keywords

1 Introduction

2 Preliminary

Table 9 The description of datasets Datasets Number of objects Number of attributes Classes Iris 150 4 3 Ecoli 336 7 8 Glass 214 10 7 Wine 178 13 3 Wpbc 198 34 2 Yeast 1484 8 9

Footnotes

Acknowledgements

References

Table 9
The description of datasets

Datasets Number of objects Number of attributes Classes

Iris 150 4 3

Ecoli 336 7 8

Glass 214 10 7

Wine 178 13 3

Wpbc 198 34 2

Yeast 1484 8 9