Fuzzy comprehensive evaluation based on multi-attribute group decision making for business intelligence system

Abstract

Business Intelligence System (BIS) has become an important tool for enterprises to make decision timely and effectively. However there are many differences in the quality and performance of the BIS on the market, it is necessary for enterprise managers to evaluate the BIS before buying, so that they could choose the right BIS. This study provided a fuzzy comprehensive evaluation method based on multi-attribute group decision making for selecting BIS. Eight evaluation criteria about BIS were firstly determined through literature review with three judgment methods being presented to score these criteria. Then the ordered pairwise comparison method was used to determine the weight of criteria, and the entropy measure method of interval-valued intuitionistic fuzzy sets was applied to determine the weights of experts. A fuzzy comprehensive evaluation algorithm based on multi-attribute group decision making was proposed, which we used to select suitable supplier of BIS. Finally an illustrative supplier selection problem was described to demonstrate the practicality and effectiveness of the proposed method.

Keywords

Business intelligence system interval-valued intuitionistic fuzzy sets multi-attribute group decision making

1 Introduction

In the 21st century, with the deepening globalization and the changing market environment, enterprises are facing with fierce competition and challenges. Information technology as the essential tool to improve the level of production, operation and management in an enterprise, to strengthen the core competitiveness of enterprises, has been identified and adopted by more and more enterprises [1]. In order to improve the level of information management, many enterprises spent a lot of money and resources to build online transaction processing and enterprise resource planning. However continuous cumulative information and data stored in the data warehouse has come to a astonishing scale, the growing volume of data caused the traditional data analysis tools and methods not to be equal to its ambition in processing massive data timely and making business analysis accurately, at this right moment BIS came into being. It was first introduced by Gartner Group in 1989 as the technology to improve business decision-making [2]. Although the study about BIS has been going on for more than 20 years, there was still no uniformity in its definition. We believe that BIS is a complete set of solutions, which utilizes modern data warehouse technology, online analyzing and processing technology, data mining and data display technology to effectively integrate and analyze the existing data of the enterprise, provides statements quickly and accurately, puts forward the basis of decision-making, helps the enterprise to make wise decisions. Therefore, capturing the valuable information, responding promptly to business events, and making accurate decision are the main goal of BIS [3].

In the recent years, the development and application of BIS received wide attention from industries at home and abroad, on the one hand, many well-known enterprises have learnt of its great development potential, more large enterprises involved in the research and development of BIS. Some outstanding software manufacturers launched diverse software systems to support development and application of BIS, such as IBM, Microsoft, oracle, SAP and so on. On the other, enterprises put forth numerous demands for BIS, regard BIS as one of the effective ways to help them achieve their business objectives.

Many enterprises spent a lot of money on BIS and related technology each year. It was estimated that the capital spent on BIS was approximately 20 billion dollar in 2006, and is expected to grow 10% annually in the future [4, 5].

Presently, the market is flooded with a wide variety of BIS, different system has different performance, the price of each system is very high. When confronted with a huge investment, it is necessary for enterprises to carefully evaluate the performance of each BIS, thus how to choose the right BIS system and avoid a blind purchase becomes their key problem.

Lönnqvist and Pirttimäki firstly assessed evaluation approaches of BIS [6]. After that, Lin et al. constructed a performance evaluation model of BIS using analytic network process, and acquired the most critical factors that influence the effectiveness of BIS [7]. Yeoh and Koronios identified the critical success factors that impact BIS implementation by adopting an interpretivist paradigm and Delphi method [8]. Farrokhi and Pokorádi analyzed the necessities that enterprise builds a model to evaluate BIS [9]. Rouhani et al. proposed an evaluation model of BIS based on fuzzy technique for order preference by similarity to ideal solution (TOPSIS) technique [10]. Fedouaki, et al. proposed a maturity model for BIS using the critical success factors, then validated the model by pilot test and empirical investigation [11]. Jooste et al. addressed the usability evaluation criteria of BIS, which were used on the evaluation of a coal mining organization by indirect observation, expert evaluation and user based evaluation [12]. Ghazanfari et al. presented thirty-four criteria for BIS, and evaluated the BIS of Port using a fuzzy TOPSIS model [13]. Rouhani and Ravasan addressed a framework to evaluate BIS and three ERPs of an offshore engineering and construction company in Iran’s oil industry using fuzzy ANP approach [14]. Soloukdara and Parpanchi provided key criteria for selecting BIS vendors, evaluating and ranking BIS of six different vendors using fuzzy analytical hierarchy process (AHP) and fuzzy TOPSIS [15].

Although the above mentioned literature have introduced a series of research on the selection and evaluation of BIS, these evaluation indicators proposed by the literature are hard to be applied to the practical BIS evaluation because of the huge quantity. Furthermore, these literatures did not select evaluation methods on the basis of the characteristics of the evaluation indicator, only used one evaluation method to evaluate all the indicators, which would cause evaluation results inaccurate. Additionally, these literatures assumed that the expert weight and indicator weight had been given in whole or in part, but in the actual BIS evaluation, these are not certain necessarily in advance. Based on this, through merging and reducing attributes, constructing appropriate quantity of BIS evaluation indictors, selecting different evaluation methods according to characteristics of the indicator, this paper proposes the hybrid interval-valued intuitionistic fuzzy multi-attribute group decision-making model based on the ordered pairwise comparison method and entropy measure method of interval-valued intuitionistic fuzzy sets, helps enterprises to make a correct assessment to the BIS which would be purchased.

The rest of this paper is arranged as follows. Section 2 provides the key evaluation criteria of BIS and the fuzzy comprehensive evaluation methods based on multi-attribute group decision making. Section 3 illustrates a real example. Section 4 comes to the conclusion.

2 Materials and methods

This section is consisted of two parts. The first part introduces the key evaluation criteria of BIS selection. The second part explains fuzzy comprehensive evaluation method based on multi-attribute group decision making and clarifies the fuzzy comprehensive evaluation algorithm which shall be applied in this paper.

2.1 Identification of evaluation criteria of BIS

BIS criteria system is an integrity constituted by a series of characteristics of BIS criteria; which reflects the BIS in all aspects. The evaluation criteria are not the more the better; there may be repeatability and interference. They are also not the less the better, the selected criteria may not be typical and easy to be one-sided. So it is a prerequisite to set BIS criteria system correctly and comprehensively in order to make a comprehensive evaluation of BIS. With proper evaluation criteria the assessors could provide effective evaluation and analysis to the BIS which shall be purchased and the decision makers could choose the right BIS.

Therefore, the design of BIS evaluation criteria system needs to follow the following principles

Advancement and Scientificity. It should have scientific theoretical basis in selecting the BIS evaluation criteria, which can reasonably reflect the effectiveness of the BIS application.

Systematicness. it should be able to fully reflect the comprehensive situation of BIS.

The combination of qualitative and quantitative. It should choose the easy qualitative evaluation criteria, also should choose the ones which can reflect the actual effectiveness of BIS, using a method of combination of quantitative criteria and qualitative criteria, effectively reflect the fundamental characteristics of BIS.

Measurability. It refers to that each criteria of BIS has a definite meaning, the collection of data and material is convenient, measurement method can used to analyze the criteria.

Independence. The BIS criteria have no inclusion relation, or approximately the same meaning, each criteria should be an individual.

Comparability. The stronger the comparability of BIS criteria system is, the greater the credibility of the evaluation results is. The BIS evaluation criteria should be objective and practical, and easy to compare. In the process of standardization of BIS evaluation criteria, we should keep the same trend to ensure the comparability between the criteria.

Based on the above principles, we reviewed the relevant literature and consulted with some experts, then identified eight evaluation criteria. Table 1 lists the identified criteria to evaluate BIS.

2.2 Fuzzy comprehensive evaluation method

The fuzzy set theory was introduced by Zadeh to cope with the vagueness and uncertainty relating to information about several parameters [16], then it was widely used in many fields [17 –21]. Atanassov proposed the definition of intuitionistic fuzzy set in 1986. The intuitionistic fuzzy set extended the application area of the fuzzy set, it could depict the fuzzynature of the objective world more delicately by adding a new parameter on the basis of fuzzy set, which was the non-membership degree [22]. In 1989, Atanassov and Gargov extended the notion of intuitionistic fuzzy set, used interval numbers to represent the membership degree and non-membership degree, and the concept of interval-valued intuitionistic fuzzy set was put forward [23]. After that the intuitionistic fuzzy numbers and interval intuitionistic fuzzy numbers have been widely applied in the fields of decision programming [24, 25], logic programming [26], pattern recognition [27] and so on.

In the recent years, the research on the application of intuitionistic fuzzy sets and interval intuitionistic fuzzy sets in the field of the multi attribute decision-making methods has become one of the research focuses.

Zhao et al. presented a decision-making method based on the hybrid decision matrix which integrated three kinds of evaluation information, as intuitionistic fuzzy set, interval-valued intuitionistic fuzzy set and linguistic evaluation set, the decision result was obtained by means of the triangular fuzzy numbers which was transformed from the linguistic evaluation information [28]. Park et al. also proposed a multi-attribute group decision-making method with incomplete attribute weights information by defining the geometric operator and score function of interval-valued intuitionistic fuzzy set [29]. Furthermore, Ye developed a multi-attribute decision-making model in which intuitionistic trapezoidal fuzzy numbers was used to describe the information of attributes and weights [30]. Recently, Li et al. proposed a multi-attribute decision-making technique about order preference by similarity to an ideal solution base on intuitionistic fuzzy set when the weights were known [31].

At present, the study about the interval valued intuitionistic fuzzy multiple attribute group decision either never considers the expert weights and attribute weights, or partially considers the expert weights and attribute weights, it is not common to study the multiple group decision on the condition that the expert weights and attribute weights are completely unknown.

In this paper, a hybrid multiple attribute group decision making method is proposed based on the attributes determining method of ordered pairwise comparison method and expert weights determining method of entropy measure to help the enterprise to evaluate BIS.

2.2.1 Interval-valued intuitionistic fuzzy set

Atanassov first gave the definition of intuitionistic fuzzy set as follows [22].

Definition 1. Let X be a non-empty set, the set A = {< x, u_A (x) , v_A (x) >/x ∈ X} is called an intuitionistic fuzzy set, where u_A (x) ∈ [0, 1], v_A (x) ∈ [0, 1], and u_A (x) + v_A (x) ∈ [0, 1].

u_A (x) and v_A (x) are called the membership function and the non-membership of x to A, respectively.

Atanassov and Gargov gave the definition of interval-valued intuitionistic fuzzy set as follows [23].

Definition 2. The set $\tilde{A} = {< x, u_{\tilde{A}} (x), v_{\tilde{A}} (x) > / x \in X} = {< x, [u_{\tilde{A} L} (x), u_{\tilde{A} U} (x)], [v_{\tilde{A} L} (x), v_{\tilde{A} U} (x)] > / x \in X}$ is called an interval-valued intuitionistic fuzzy set, where $u_{\tilde{A} L} (x) \in [0, 1]$ , $u_{\tilde{A} U} (x) \in [0, 1]$ , $v_{\tilde{A} L} (x) \in [0, 1]$ , $v_{\tilde{A} U} (x) \in [0, 1]$ , and $u_{\tilde{A} U} (x) + v_{\tilde{A} U} (x) \in [0, 1]$ .

When $u_{\tilde{A} L} (x) = u_{\tilde{A} U} (x)$ and $v_{\tilde{A} L} (x) = v_{\tilde{A} U} (x)$ , the interval-valued intuitionistic fuzzy set $\tilde{A}$ is an intuitionistic fuzzy set. To simplify, $\tilde{A}$ is usually denoted as $< [u_{\tilde{A} L} (x), u_{\tilde{A} U} (x)], [v_{\tilde{A} L} (x), v_{\tilde{A} U} (x)] >$ . Let $π_{\tilde{A} U} (x) = 1 - u_{\tilde{A} U} (x) - v_{\tilde{A} U} (x)$ and $π_{\tilde{A} L} (x) = 1 - u_{\tilde{A} L} (x) - v_{\tilde{A} L} (x)$ , $π_{\tilde{A}} (x) = [π_{\tilde{A} U} (x), π_{\tilde{A} L} (x)]$ is called an hesitancy degree of x to $\tilde{A}$ .

Xu introduced the definition of interval-valued intuitionistic fuzzy number and gave the method of comparing two interval-valued intuitionistic fuzzy numbers [32].

Definition 3. Let a = ([u_L, u_U] , [v_L, v_U]), where [u_L, u_U] ⊂ [0, 1], [v_L, v_U] ⊂ [0, 1] and u_U + v_U ≤ 1, a is called as an interval-valued intuitionistic fuzzy number. Let $s (a) = \frac{u_{L} - v_{L} + u_{U} - v_{U}}{2}$ and $h (a) = \frac{u_{L} + v_{L} + u_{U} + v_{U}}{2}$ , then s (a) and h (a) are called the score degree and the accuracy degree of a respectively.

Definition 4. Let a₁ = ([u_1L, u_1U] , [v_1L, v_1U]), a₂ = ([u_2L, u_2U] , [v_2L, v_2U]), s (a₁) and s (a₂) are the score degrees of a₁ and a₂ respectively; h (a₁) and h (a₂) are the score degrees of a₁ and a₂, respectively. Then, we have

If s (a₁) > s (a₂), then a₁ is bigger than a₂, denoted by a₁ > a₂;

If s (a₁) = s (a₂), then the following results are true:

If h (a₁) > h (a₂), then a₁ is bigger than a₂, denoted by a₁ > a₂;

If h (a₁) < h (a₂), then a₁ is smaller than a₂, denoted by a₁ < a₂;

If h (a₁) = h (a₂), then a₁ is the same as a₂, denoted by a₁ ∼ a₂.

Definition 5. Let the interval-valued intuitionistic fuzzy number a_i = ([u_iL, u_iU] , [v_iL, v_iU])(i = 1, 2, ⋯ , n), let $\bar{A} = {a_{i} / i = 1, \dots, n}$ ,

$\begin{matrix} A_{w} & = & ([1 - Π_{i = 1}^{n} (1 - u_{iL})^{w_{i}}, \\ 1 - Π_{i = 1}^{n} (1 - u_{iU})^{w_{i}}], [Π_{i = 1}^{n} v_{iL}^{w_{i}}, Π_{i = 1}^{n} v_{iU}^{w_{i}}]) \end{matrix}$ (1) is called an interval-valued intuitionistic fuzzy weighted averaging operator (IIFWAO), where w_i ∈ [0, 1] is the weight value of a_i and $\sum_{i = 1}^{n} w_{i} = 1$ .

Suppose S = {s₁, ⋯ , s_2k−1}( $k \in ℤ^{+}$ ) is any linguistic evaluation set, using a method similar to the method proposed by Zhao et al. [28], linguistic evaluation set S can be transformed into a set consisting of the corresponding interval-valued intuitionistic number $A_{S} = {([u_{iL}, u_{iU}], [v_{iL}, v_{iU}]) / i = 1, 2, \dots, 2 k - 1}$ (2) where $u_{iL} = {(0.5 - \frac{1}{4 k - 2})}^{k - i + 1}$ , $u_{iU} = 0 . 5^{k - i + 1}, v_{iL} = 1 - {(0.5 + \frac{1}{4 k - 2})}^{k - i + 1}$ , v_iU = 1 −0 . 5^k−i+1

when i ≤ k; $u_{iL} = {(0.5 - \frac{1}{4 k - 2})}^{1 / (i - k + 1)}, u_{iU} = 0 . 5^{1 / (i - k + 1)}, v_{iL} = 1 - {(0.5 + \frac{1}{4 k - 2})}^{1 / (i - k + 1)}$ ,v_iU = 1 − 0.5^1/(i−k+1) when i ≥ k.

2.2.2 Ordered pairwise comparison method

The ordered pairwise comparison method is usually used to evaluate the weight of the attribute [31]. We introduce this method as follows. Let C = {C₁, ⋯ , C_m} be an attribute set, R = (r_ij) _m×m be a pairwise comparison matrix of C about fuzzy concept “importance”. Let $r_{i} = \sum_{j = 1}^{m} r_{ij}$ , we can determine the consistency ranking order of C about “importance” on the basis of the size of r_i. In order to simplify the problem, it supposes C₁≻ C₂ ≻ ⋯ ≻ C_m−1 ≻ C_m, where C_i ≻ C_i+1 indicates that the attribute C_i−1 is more important than C_i. The expert then gives the rational evaluation score λ_i (i = 1, 2, ⋯ , m − 1) of the attribute C_i to C_i+1. λ_i > 1 represents that C_i is more important than C_i+1, λ_i < 1 represents that C_i is less important than C_i+1, and λ_i = 1 represents that C_i is as important as C_i+1. Obviously, λ_i > 1 for all i = 1, 2, ⋯ , m − 1.

Assume that α_i was the weight of the attribute C_i, then the weight of all attribute are given as follow: $α_{i} = {\begin{matrix} {(1 + Σ_{i = 1}^{m - 1} Π_{j = i}^{m - 1} λ_{j})}^{- 1} & i = m \\ α_{m} Π_{j = i}^{m - 1} λ_{j} & i = 1, \dots, m - 1 \end{matrix}$ (3)

2.2.3 Determination method of expert weights based on entropy measure

Entropy measure of interval-valued intuitionistic fuzzy sets was given by Wei and zhang and was applied to determine the expert weights [33]. We also apply this method to determine the weights of experts who evaluate BIS.

Assume that there are n experts to evaluate BIS, D = {d₁, ⋯ , d_n} is the set of experts. Let X = {x₁, ⋯ , x_m} be an alternative set. The comprehensive evaluation value given by the ith expert to all the alternatives is ${\tilde{Z}}^{(i)} = {([u_{jL}^{(i)}, u_{jU}^{(i)}], [v_{jL}^{(i)}, v_{jU}^{(i)}]) / j = 1, \dots m}$ , $E ({\tilde{Z}}^{(i)}) = \frac{1}{m} \sum_{j = 1}^{m} cos \frac{| u_{jL}^{(i)} - v_{jL}^{(i)} | + | u_{jU}^{(i)} - v_{jU}^{(i)} |}{2 (2 + π_{jL}^{(i)} + π_{jU}^{(i)})}$ is an entropy measure of ${\tilde{Z}}^{(i)}$ . Then, the weights of experts are given as follows: $β_{i} = \frac{1 - e_{i}}{\sum_{i = 1}^{n} (1 - e_{i})}$ (4)

where $e_{i} = \frac{E_{i}}{\sum_{i = 1}^{n} E_{i}}$ , i = 1, 2, ⋯ , n.

2.2.4 Fuzzy comprehensive evaluation method based on multi-attribute group decision making

According to the above analysis, we give the fuzzy comprehensive evaluation method based on multi-attribute group decision making, the main steps of the method are listed as follow:

Step 1: Clarify the alternative set X = {x₁, ⋯ , x_n}, attribute set C = {C₁, ⋯ , C_m} and expert set D = {d₁, ⋯ , d_s};

Step 2: Collect all experts’ evaluation value on the basis of intuitionistic fuzzy set, interval-valued intuitionistic fuzzy set and linguistic evaluation set, establish the hybrid decision matrix $F^{(k)} = {(f_{ij}^{(k)})}_{n \times m}$ , where $f_{ij}^{(k)}$ is the evaluation value of the ith alternative given by the kth expert to the jth attribute;

Step 3: Transform the linguistic evaluation value into the interval-valued intuitionistic fuzzy number by Equation (2);

Step 4: Transform the hybrid decision matrix F^(k) into an interval-valued intuitionistic fuzzy decision matrix ${\tilde{F}}^{(k)} = {({\tilde{f}}_{ij}^{(k)})}_{n \times m}$ ;

Step 5: Determine the weight α = (α₁, ⋯ , α_m) of all attributes given by each expert by means of the ordered pairwise comparison method;

Step 6: Calculate the comprehensive evaluation value $z^{(k)} = (z_{1}^{(k)}, \dots, z_{n}^{(k)})$ (k = 1, ⋯ , s) of every expert to all alternatives by Equation (1);

Step 7: Calculate the weight β = (β₁, ⋯ , β_s) of all experts by Equation (4);

Step 8: Calculate the group evaluation value z = (z₁, ⋯ , z_n) of all expert to all alternatives by Equation (1);

Step 9: Compare the comprehensive evaluation value z₁, ⋯ , z_n, and rank all alternatives x₁, ⋯ , x_n, then determine the best alternative.

3 Results

This section applies the proposed approach to evaluate and select the suppliers of BIS. Now we suppose a company is going to purchase a set of BIS, five suppliers intend to compete for the tender, the five suppliers are the alternative set X = {x₁, ⋯ , x₅}. And in order to ensure to purchase the most appropriate BIS, the purchaser invites five experts to make an analysis on the BIS of the five enterprises, and meanwhile to make an evaluation on the BIS using the eight indicators of Table 1, the five experts are the expert set D = {d₁, ⋯ , d₅}. The 8 indicators are the attribute set C = {C₁, ⋯ , C₈}: System response time (C₁), Output information quality (C₂), Reliability and accuracy (C₃), Maintainability (C₄), Data analysis and delivery (C₅), Price of product (C₆), Support services (C₇), compatibility and expansibility (C₈). Among these criteria, C₁ and C₆ are two quantitative criteria, C₂, C₃, C₄, C₅, C₇, C₈ are six qualitative criteria. The five experts use intuitionistic fuzzy numbers to evaluate system response time (C₁), use interval valued intuitionistic fuzzy number to evaluate price of product (C₆), use 7 scale language level {verypoor s₁, poorer s₂, poor s₃, average s₄, goods₅, better s₆, verygood s₇} to evaluate the five qualitative indicators {C₂, C₃, C₄, C₅, C₇, C₈}. According to the BIS evaluation given by the 5 experts, the hybrid decision matrix is constructed as follows: $\begin{matrix} F^{(1)} \\ = (\begin{matrix} (0.7, 0.3) & s_{5} & s_{6} & s_{6} & s_{4} & ([0.8, 0.9], [0, 0.1]) & s_{4} & s_{5} \\ (0.6, 0.3) & s_{6} & s_{7} & s_{5} & s_{6} & ([0.7, 0.8], [0, 0.2]) & s_{6} & s_{6} \\ (0.8, 0.2) & s_{3} & s_{6} & s_{4} & s_{6} & ([0.5, 0.7], [0.1, 0.3]) & s_{5} & s_{4} \\ (0.9, 0.1) & s_{4} & s_{5} & s_{6} & s_{5} & ([0.4, 0.6], [0.2, 0.3]) & s_{3} & s_{6} \\ (0.5, 0.3) & s_{7} & s_{6} & s_{6} & s_{7} & ([0.6, 0.7], [0.1, 0.2]) & s_{4} & s_{6} \end{matrix}) \\ F^{(2)} \\ = (\begin{matrix} (0.6, 0.3) & s_{4} & s_{5} & s_{7} & s_{3} & ([0.6, 0.7], [0.1, 0.3]) & s_{5} & s_{6} \\ (0.7, 0.2) & s_{5} & s_{6} & s_{6} & s_{7} & ([0.5, 0.7], [0.2, 0.3]) & s_{5} & s_{6} \\ (0.7, 0.2) & s_{3} & s_{5} & s_{5} & s_{6} & ([0.5, 0.6], [0.1, 0.3]) & s_{4} & s_{5} \\ (0.8, 0.1) & s_{4} & s_{6} & s_{5} & s_{3} & ([0.4, 0.6], [0.2, 0.3]) & s_{3} & s_{7} \\ (0.4, 0.3) & s_{7} & s_{6} & s_{5} & s_{6} & ([0.7, 0.8], [0.1, 0.2]) & s_{5} & s_{5} \end{matrix}) \end{matrix}$ $\begin{matrix} F^{(3)} \\ = (\begin{matrix} (0.8, 0.2) & s_{4} & s_{7} & s_{5} & s_{5} & ([0.5, 0.6], [0.2, 0.3]) & s_{6} & s_{6} \\ (0.5, 0.4) & s_{5} & s_{6} & s_{4} & s_{5} & ([0.5, 0.6], [0.1, 0.2]) & s_{7} & s_{5} \\ (0.7, 0.1) & s_{4} & s_{5} & s_{3} & s_{7} & ([0.6, 0.7], [0, 0.2]) & s_{5} & s_{6} \\ (0.7, 0.2) & s_{3} & s_{4} & s_{5} & s_{4} & ([0.7, 0.8], [0.1, 0.2]) & s_{4} & s_{5} \\ (0.6, 0.2) & s_{5} & s_{5} & s_{7} & s_{6} & ([0.6, 0.7], [0.1, 0.3]) & s_{5} & s_{5} \end{matrix}) \\ F^{(4)} \\ = (\begin{matrix} (0.6, 0.2) & s_{6} & s_{4} & s_{6} & s_{3} & ([0.6, 0.7], [0, 0.2]) & s_{5} & s_{4} \\ (0.7, 0.3) & s_{5} & s_{5} & s_{4} & s_{4} & ([0.5, 0.6], [0, 0.1]) & s_{5} & s_{7} \\ (0.8, 0.1) & s_{2} & s_{5} & s_{5} & s_{5} & ([0.6, 0.7], [0.1, 0.2]) & s_{6} & s_{5} \\ (0.8, 0.2) & s_{3} & s_{6} & s_{4} & s_{4} & ([0.4, 0.5], [0.2, 0.4]) & s_{5} & s_{4} \\ (0.6, 0.2) & s_{5} & s_{4} & s_{5} & s_{5} & ([0.6, 0.7], [0, 0.1]) & s_{5} & s_{5} \end{matrix}) \\ F^{(5)} \\ = (\begin{matrix} (0.8, 0.2) & s_{6} & s_{7} & s_{5} & s_{5} & ([0.7, 0.8], [0, 0.1]) & s_{5} & s_{6} \\ (0.7, 0.2) & s_{6} & s_{5} & s_{6} & s_{6} & ([0.8, 0.9], [0, 0.1]) & s_{7} & s_{6} \\ (0.7, 0.1) & s_{5} & s_{6} & s_{6} & s_{5} & ([0.6, 0.7], [0.1, 0.2]) & s_{6} & s_{6} \\ (0.7, 0.2) & s_{6} & s_{5} & s_{5} & s_{6} & ([0.6, 0.7], [0.1, 0.2]) & s_{5} & s_{5} \\ (0.6, 0.3) & s_{6} & s_{5} & s_{6} & s_{6} & ([0.7, 0.8], [0.1, 0.2]) & s_{5} & s_{7} \end{matrix}) \end{matrix}$

According to Equation (2), the linguistic evaluation value of the hybrid decision matrix can be converted into interval-valued intuitionistic fuzzy number. Therefore, the hybrid decision matrix F⁽¹⁾, F⁽²⁾, F⁽³⁾, F⁽⁴⁾, F⁽⁵⁾ are transformed into the interval-valued intuitionistic fuzzy decision matrix ${\tilde{F}}^{(1)}$ , ${\tilde{F}}^{(2)}$ , ${\tilde{F}}^{(3)}$ , ${\tilde{F}}^{(4)}$ , ${\tilde{F}}^{(5)}$ .

In order to simplify the discussion, the five experts analyze the importance of the relationship of the eight indicators together, and determine the pairwise comparison matrix R of the eight indicators. $R = (r_{ij})_{8 \times 8} = (\begin{matrix} 0.5 & 0 & 0 & 0 & 0.5 & 0 & 0.5 & 0 \\ 1 & 0.5 & 0.5 & 0.5 & 1 & 0.5 & 1 & 1 \\ 1 & 0.5 & 0.5 & 0.5 & 0 & 0.5 & 1 & 1 \\ 1 & 0.5 & 0.5 & 0.5 & 1 & 0 & 1 & 1 \\ 0.5 & 0 & 1 & 0 & 0.5 & 0 & 1 & 1 \\ 1 & 0.5 & 0.5 & 1 & 1 & 0.5 & 1 & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0.5 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0.5 \end{matrix})$

By calculation, we get r₁ = 1.5, r₂ = 6, r₃ = 5, r₄ = 5.5, r₅ = 4, r₆ = 6.5, r₇ = 0.5, r₈ = 1.5, so the importance of the eight indicators is in the order as follows: $C_{6} ≻ C_{2} ≻ C_{4} ≻ C_{3} ≻ C_{5} ≻ C_{1} \sim C_{8} ≻ C_{7}$ where C₁ ∼ C₈ indicates that C₁ is as important as C₈.

The rational evaluation score are given by 5 experts as follows: λ₁ = 1.2, λ₂ = 1.1, λ₃ = 1.3, λ₄ = 1.1, $\begin{matrix} {\tilde{F}}^{(1)} \\ = (\begin{matrix} ([0.7, 0.7], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.7539, 0.7937], & ([0.4286, 0.5], & ([0.8, 0.9], & ([0.4286, 0.5], & ([0.6547, 0.7071], \\ [0.3, 0.3]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.1702, 0.2063]) & [0.4286, 0.5]) & [0, 0.1]) & [0.4286, 0.5]) & [0.2441, 0.2929]) \\ ([0.6, 0.6], & ([0.7539, 0.7937], & ([0.8091, 0.8409], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.7, 0.8], & ([0.7539, 0.7937], & ([0.7539, 0.7937], \\ [0.3, 0.3]) & [0.1702, 0.2063]) & [0.1306, 0.1591]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0, 0.2]) & [0.1702, 0.2063]) & [0.1702, 0.2063]) \\ ([0.8, 0.8], & ([0.1837, 0.25], & ([0.7539, 0.7937], & ([0.4286, 0.5], & ([0.7539, 0.7937], & ([0.5, 0.7], & ([0.6547, 0.7071], & ([0.4286, 0.5], \\ [0.2, 0.2]) & [0.6735, 0.75]) & [0.1702, 0.2063]) & [0.4286, 0.5]) & [0.1702, 0.2063]) & [0.1, 0.3]) & [0.2441, 0.2929]) & [0.4286, 0.5]) \\ ([0.9, 0.9], & ([0.4286, 0.5], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.6547, 0.7071], & ([0.4, 0.6], & ([0.1837, 0.25], & ([0.7539, 0.7937], \\ [0.1, 0.1]) & [0.4286, 0.5]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.2441, 0.2929]) & [0.2, 0.3]) & [0.6735, 0.75]) & [0.1702, 0.2063]) \\ ([0.5, 0.5], & ([0.8091, 0.8409], & ([0.7539, 0.7937], & ([0.7539, 0.7937], & ([0.8091, 0.8409], & ([0.6, 0.7], & ([0.4286, 0.5], & ([0.7539, 0.7937], \\ [0.3, 0.3]) & [0.1306, 0.1591]) & [0.1702, 0.2063]) & [0.1702, 0.2063]) & [0.1306, 0.1591]) & [0.1, 0.2]) & [0.4286, 0.5]) & [0.1702, 0.2063]) \end{matrix}) \\ {\tilde{F}}^{(2)} \\ = (\begin{matrix} ([0.6, 0.6], & ([0.4286, 0.5], & ([0.6547, 0.7071], & ([0.8091, 0.8409], & ([0.1837, 0.25], & ([0.6, 0.7], & ([0.6547, 0.7071], & ([0.7539, 0.7937], \\ [0.3, 0.3]) & [0.4286, 0.5]) & [0.2441, 0.2929]) & [0.1306, 0.1591]) & [0.6735, 0.75]) & [0.1, 0.3]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) \\ ([0.7, 0.7], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.7539, 0.7937], & ([0.8091, 0.8409], & ([0.5, 0.7], & ([0.6547, 0.7071], & ([0.7539, 0.7937], \\ [0.2, 0.2]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.1702, 0.2063]) & [0.1306, 0.1591]) & [0.2, 0.3]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) \\ ([0.7, 0.7], & ([0.1837, 0.25], & ([0.6547, 0.7071], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.5, 0.6], & ([0.4286, 0.5], & ([0.6547, 0.7071], \\ [0.2, 0.2]) & [0.6735, 0.75]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.1, 0.3]) & [0.4286, 0.5]) & [0.2441, 0.2929]) \\ ([0.8, 0.8], & ([0.4286, 0.5], & ([0.7539, 0.7937], & ([0.6547, 0.7071], & ([0.1837, 0.25], & ([0.4, 0.6], & ([0.1837, 0.25], & ([0.8091, 0.8409], \\ [0.1, 0.1]) & [0.4286, 0.5]) & [0.1702, 0.2063]) & [0.2441, 0.2929]) & [0.6735, 0.75]) & [0.2, 0.3]) & [0.6735, 0.75]) & ([0.6547, 0.7071], \\ ([0.4, 0.4], & ([0.8091, 0.8409], & ([0.7539, 0.7937], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.7, 0.8], & ([0.6547, 0.7071], & ([0.6547, 0.7071], \\ [0.3, 0.3]) & [0.1306, 0.1591]) & [0.1702, 0.2063]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.1, 0.2]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) \end{matrix}) \\ {\tilde{F}}^{(3)} \\ = (\begin{matrix} ([0.8, 0.8], & ([0.4286, 0.5], & ([0.8091, 0.8409], & ([0.6547, 0.7071], & ([0.6547, 0.7071], & ([0.5, 0.6], & ([0.7539, 0.7937], & ([0.7539, 0.7937], \\ [0.2, 0.2]) & [0.4286, 0.5]) & [0.1306, 0.1591]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) & [0.2, 0.3]) & [0.1702, 0.2063]) & [0.1702, 0.2063]) \\ ([0.5, 0.5], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.4286, 0.5], & ([0.6547, 0.7071], & ([0.5, 0.6], & ([0.8091, 0.8409], & ([0.6547, 0.7071], \\ [0.4, 0.4]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.4286, 0.5]) & [0.2441, 0.2929]) & [0.1, 0.2]) & [0.1306, 0.1591]) & [0.2441, 0.2929]) \\ ([0.7, 0.7], & ([0.4286, 0.5], & ([0.6547, 0.7071], & ([0.1837, 0.25], & ([0.8091, 0.8409], & ([0.6, 0.7], & ([0.6547, 0.7071], & ([0.7539, 0.7937], \\ [0.1, 0.1]) & [0.4286, 0.5]) & [0.2441, 0.2929]) & [0.6735, 0.75]) & [0.1306, 0.1591]) & [0, 0.2]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) \\ ([0.7, 0.7], & ([0.1837, 0.25], & ([0.4286, 0.5], & ([0.6547, 0.7071], & ([0.4286, 0.5], & ([0.7, 0.8], & ([0.4286, 0.5], & ([0.6547, 0.7071], \\ [0.2, 0.2]) & [0.6735, 0.75]) & [0.4286, 0.5]) & [0.2441, 0.2929]) & [0.4286, 0.5]) & [0.1, 0.2]) & [0.4286, 0.5]) & [0.2441, 0.2929]) \\ ([0.6, 0.6], & ([0.6547, 0.7071], & ([0.6547, 0.7071], & ([0.8091, 0.8409], & ([0.7539, 0.7937], & ([0.6, 0.7], & ([0.6547, 0.7071], & ([0.6547, 0.7071], \\ [0.2, 0.2]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) & [0.1306, 0.1591]) & [0.1702, 0.2063]) & [0.1, 0.3]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) \end{matrix}) \end{matrix}$

$\begin{matrix} {\tilde{F}}^{(4)} \\ = (\begin{matrix} ([0.6, 0.6], & ([0.7539, 0.7937], & ([0.4286, 0.5], & ([0.7539, 0.7937], & ([0.1837, 0.25], & ([0.6, 0.7], & ([0.6547, 0.7071], & ([0.4286, 0.5], \\ [0.2, 0.2]) & [0.1702, 0.2063]) & [0.4286, 0.5]) & [0.1702, 0.2063]) & [0.6735, 0.75]) & [0, 0.2]) & [0.2441, 0.2929]) & [0.4286, 0.5]) \\ ([0.7, 0.7], & ([0.6547, 0.7071], & ([0.6547, 0.7071], & ([0.4286, 0.5], & ([0.4286, 0.5], & ([0.5, 0.6], & ([0.6547, 0.7071], & ([0.8091, 0.8409], \\ [0.3, 0.3]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) & [0.4286, 0.5]) & [0.4286, 0.5]) & [0, 0.1]) & [0.2441, 0.2929]) & [0.1306, 0.1591]) \\ ([0.8, 0.8], & ([0.0787, 0.125], & ([0.6547, 0.7071], & ([0.6547, 0.7071], & ([0.6547, 0.7071], & ([0.6, 0.7], & ([0.7539, 0.7937], & ([0.6547, 0.7071], \\ [0.1, 0.1]) & [0.8134, 0.875]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) & [0.1, 0.2]) & [0.1702, 0.2063]) & [0.2441, 0.2929]) \\ ([0.8, 0.8], & ([0.1837, 0.25], & ([0.7539, 0.7937], & ([0.4286, 0.5], & ([0.4286, 0.5], & ([0.4, 0.5], & ([0.6547, 0.7071], & ([0.4286, 0.5], \\ [0.2, 0.2]) & [0.6735, 0.75]) & [0.1702, 0.2063]) & [0.4286, 0.5]) & [0.4286, 0.5]) & [0.2, 0.4]) & [0.2441, 0.2929]) & [0.4286, 0.5]) \\ ([0.6, 0.6], & ([0.6547, 0.7071], & ([0.4286, 0.5], & ([0.6547, 0.7071], & ([0.6547, 0.7071], & ([0.6, 0.7], & ([0.6547, 0.7071], & ([0.6547, 0.7071], \\ [0.2, 0.2]) & [0.2441, 0.2929]) & [0.4286, 0.5]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) & [0, 0.1]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) \end{matrix}) \\ {\tilde{F}}^{(5)} \\ = (\begin{matrix} ([0.8, 0.8], & ([0.7539, 0.7937], & ([0.8091, 0.8409], & ([0.6547, 0.7071], & ([0.6547, 0.7071], & ([0.7, 0.8], & ([0.6547, 0.7071], & ([0.7539, 0.7937], \\ [0.2, 0.2]) & [0.1702, 0.2063]) & [0.1306, 0.1591]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) & [0, 0.1]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) \\ ([0.7, 0.7], & ([0.7539, 0.7937], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.7539, 0.7937], & ([0.8, 0.9], & ([0.8091, 0.8409], & ([0.7539, 0.7937], \\ [0.2, 0.2]) & [0.1702, 0.2063]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.1702, 0.2063]) & [0, 0.1]) & [0.1306, 0.1591]) & [0.1702, 0.2063]) \\ ([0.7, 0.7], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.7539, 0.7937], & ([0.6547, 0.7071], & ([0.6, 0.7], & ([0.7539, 0.7937], & ([0.7539, 0.7937], \\ [0.1, 0.1]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.1702, 0.2063]) & [0.2441, 0.2929]) & [0.1, 0.2]) & [0.1702, 0.2063]) & [0.1702, 0.2063]) \\ ([0.7, 0.7], & ([0.7539, 0.7937], & ([0.6547, 0.7071], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.6, 0.7], & ([0.6547, 0.7071], & ([0.6547, 0.7071], \\ [0.2, 0.2]) & [0.1702, 0.2063]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.1, 0.2]) & [0.2441, 0.2929]) & [0.2441, 0.2929]) \\ ([0.6, 0.6], & ([0.7539, 0.7937], & ([0.6547, 0.7071], & ([0.7539, 0.7937], & ([0.7539, 0.7937], & ([0.7, 0.8], & ([0.6547, 0.7071], & ([0.8091, 0.8409], \\ [0.3, 0.3]) & [0.1702, 0.2063]) & [0.2441, 0.2929]) & [0.1702, 0.2063]) & [0.1702, 0.2063]) & [0.1, 0.2]) & [0.2441, 0.2929]) & [0.1306, 0.1591]) \end{matrix}) \end{matrix}$

λ₅ = 1.2, λ₆ = 1, λ₇ = 1.1.

According to Equation (3), we can get the weight of 8 criteria, they are $\begin{matrix} α_{6} & = & 0.2004, α_{2} = 0.1671, α_{4} = 0.1519, α_{3} = 0.1169, \\ α_{5} & = & 0.1062, α_{1} = 0.0885, α_{8} = 0.0885, α_{7} = 0.0805 . \end{matrix}$

By Equation (1), we can work out the comprehensive evaluation value of software given by each expert as z⁽¹⁾, z⁽²⁾, z⁽³⁾, z⁽⁴⁾, z⁽⁵⁾. $z^{(1)} = (\begin{matrix} ([0.6934, 0.762], & ([0.7268, 0.7776], & ([0.5718, 0.6499], & ([0.6285, 0.6927], & ([0.7114, 0.7594], \\ [0, 0.238]) & [0, 0.2168]) & [0.251, 0.3501]) & [0.2369, 0.2901]) & [0.1612, 0.212]) \end{matrix}),$ $z^{(2)} = (\begin{matrix} \begin{matrix} ([0.6191, 0.677], \\ [0.2241, 0.3149]) \end{matrix} & \begin{matrix} ([0.6944, 0.7561], \\ [0.1896, 0.2353]) \end{matrix} & \begin{matrix} ([0.574, 0.6323], \\ [0.2393, 0.3349]) \end{matrix} & \begin{matrix} ([0.5718, 0.6427], \\ [0.2611, 0.3172]) \end{matrix} & \begin{matrix} ([0.7039, 0.7586], \\ [0.1728, 0.2271]) \end{matrix} \end{matrix}),$

$z^{(3)} = (\begin{matrix} \begin{matrix} ([0.6603, 0.7107], \\ [0.2214, 0.273]) \end{matrix} & \begin{matrix} ([0.6198, 0.676], \\ [0.2118, 0.2765]) \end{matrix} & \begin{matrix} ([0.6032, 0.662], \\ [0, 0.2828]) \end{matrix} & \begin{matrix} ([0.5539, 0.6257], \\ [0.2819, 0.3611]) \end{matrix} & \begin{matrix} ([0.6824, 0.7343], \\ [0.1755, 0.2499]) \end{matrix} \end{matrix}),$ $z^{(4)} = (\begin{matrix} \begin{matrix} ([0.607, 0.6663], \\ [0, 0.2893]) \end{matrix} & \begin{matrix} ([0.6031, 0.6602], \\ [0, 0.2574]) \end{matrix} & \begin{matrix} ([0.6115, 0.6679], \\ [0.224, 0.288]) \end{matrix} & \begin{matrix} ([0.5143, 0.5739], \\ [0.3182, 0.4075]) \end{matrix} & \begin{matrix} ([0.6179, 0.6779], \\ [0, 0.243]) \end{matrix} \end{matrix},)$ $z^{(5)} = (\begin{matrix} \begin{matrix} ([0.7263, 0.7767], \\ [0, 0.1944]) \end{matrix} & \begin{matrix} ([0.7551, 0.8118], \\ [0, 0.1815]) \end{matrix} & \begin{matrix} ([0.6972, 0.747], \\ [0.1611, 0.2116]) \end{matrix} & \begin{matrix} ([0.6798, 0.732], \\ [0.1817, 0.2384]) \end{matrix} & \begin{matrix} ([0.7206, 0.7723], \\ [0.1687, 0.2219]) \end{matrix} \end{matrix}) .$

By Equation (4), we can get the weights of experts: $\begin{matrix} β_{1} & = & 0.2002, β_{2} = 0.1998, β_{3} = 0.1997, \\ β_{4} & = & 0.1997, β_{5} = 0.2006 . \end{matrix}$

By Equation (1), we can get the group evaluation value of 5 alternatives: $\begin{matrix} z_{1} & = & ([0.6098, 0.7222], [0, 0.2583]), \\ z_{2} & = & ([0.6854, 0.743], [0, 0.231]), \\ z_{3} & = & ([0.6146, 0.6745], [0, 0.2891]), \\ z_{4} & = & ([0.5941, 0.6579], [0.2515, 0.3174]), \\ z_{5} & = & ([0.6893, 0.7426], [0, 0.2303]) . \end{matrix}$

From Definition 3, we get the score degrees of 5 alternatives:

s (z₁) =0.5369, s (z₂) =0.5987, s (z₃) =0.5, s (z₄) =0.3416, s (z₅) =0.6008 .

According to the ranking rule of interval-valued intuitionistic fuzzy number, we have $x_{5} ≻ x_{2} ≻ x_{1} ≻ x_{3} ≻ x_{4}$

Therefore, the fifth supplier is on the top after the five experts made evaluation on the BIS of the five enterprises.

4 Conclusions

Business intelligence can be used to collect the high quality information by making full use of various information systems, then to analyze and handle the information effectively, and acquire the key factors that affect the running of enterprises. This can help enterprises to make decision quickly and rightly. Now business intelligence has become a vigorous and promising field, BIS is also used in more industries. BIS with new functionality are released every year. When facing with a variety of BIS, we have to consider how to select the most suitable BIS for enterprise.

This study first provides the evaluation criteria of BIS by means of literature review and expert consultation. A comprehensive multi-attribute group decision making evaluation method is presented based on the ordered pairwise comparison method and entropy measure method of interval-valued intuitionistic fuzzy sets, in which the weights of criteria and experts were unknown completely. This method could score these criteria by different judgment methods on the basis of different characteristics of criteria, then the scores of different criteria can be calculated and ranked by transforming them into interval-valued intuitionistic fuzzy number. The evaluation criteria and method was used to evaluate five suppliers of BIS, the ranking result wasobtained.

This study still has some limitations. For example, the above evaluation criteria are too general, the more specific and differential criteria need to be provided for different enterprises. Furthermore, although the effectiveness of the comprehensive evaluation method is validated by a case, the strengths and weakness between the comprehensive evaluation method and other evaluation methods are still unknown. These problems shall be studied in our future research.

Footnotes

Acknowledgments

This research is supported by National Science & Technology Supporting Program of China (2014BA H23F01) and Scientific Research Project of Beijing Educational Committee (SQKM201610016016).

References

M.C.

, Sharif

, and Baark

, Information technology services: A key knowledge-intensive business service industry in Hong Kong SAR, China, Science, Technology and Society19(1) (2014), 27–55.

Nylund

, Tracing the BI family tree, Knowledge Management7 (1999), 70–71.

Guran

, Mehanna

, and Hussein

, Real time on-line analytical processing for business intelligence, UPB Scientific Bulletin, Series C71(3) (2009), 71–88.

Howson

, Successful business intelligence: Secrets to making bi a killer app, McGraw-Hill, 2008.

Gantz

, Morris

, Vesset

, and Manfrediz

Taming Information Chaos, IDC White Paper, 2007.

Lönnqvist

, and Pirttimäki

, The measurement of business intelligence, Information Systems Management23(1) (2006), 32–40.

Lin

Y.H.

, Tsai

K.M.

, Shiang

W.J.

, Kuo

T.C.

, and Tsai

C.H.

, Research on using ANP to establish a performance assessment model for business intelligence systems, Expert Systems with Applications36(2) (2009), 4135–4146.

Yeoh

, and Koronios

, Critical success factors for business intelligence systems, Journal of Computer Information Systems50(3) (2010), 23–32.

Farrokhi

, and Pokorádi

, The necessities for building a model to evaluate Business Intelligence projects- Literature Review, International Journal of Computer Science & Engineering Survey3(2) (2012), 1–10.

10.

Rouhani

, Ghazanfari

, and Jafari

, Evaluation model of business intelligence for enterprise systems using fuzzy TOPSIS, Expert Systems with Applications39 (2012), 3764–3771.

11.

Fedouaki

, Okar

, and Alami

S.E.

, A maturity model for Business Intelligence System project in Small and Medium-sized Enterprises: An empirical investigation, International Journal of Computer Science Issues10(6) (2013), 61–69.

12.

Jooste

, Biljon

J.V.

, and Mentz

, Usability evaluation for Business Intelligence applications: A user support perspective, South African Computer Journal53 (2014), 32–44.

13.

Ghazanfari

, Rouhani

, and Jafari

, A fuzzy TOPSIS model to evaluate the Business Intelligence competencies of Port Community Systems, Polish Maritime Research21(2) (2014), 86–96.

14.

Rouhani

, and Ravasan

A.Z.

, A practical framework for assessing business intelligence competencies of enterprise systems using fuzzy ANP approach, International Journal Applied Decision Sciences8(1) (2015), 52–82.

15.

Soloukdar

, and Parpanchi

S.A.

, Comparing fuzzy AHP and fuzzy TOPSIS for evaluation of business intelligence vendors, Decision Science Letters4 (2015), 137–164.

16.

Zadeh

L.A.

, Fuzzy sets, Information and Computation8(3) (1965), 338–353.

17.

Praščević

, and Praščević

, Application of fuzzy AHP method based on eigenvalues for decision making in construction industry, Technical Gazette23(1) (2016), 57–64.

18.

Chiali

E.M.

, Massoum

, Hanafi

, and Taib

, Fuzzy logic controller for sensorless direct torque control of an induction motor driven by a matrix, Journal of Engineering Science and Technology Review9(1) (2016), 177–181.

19.

Ren

, and Ren

Y.F.

, Kernel Fuzzy c-means clustering for word sense disambiguation in biomedical texts, Journal of Digital Information Management13(6) (2015), 411–420.

20.

, Qi

, Jin

L.L.

, and Zhang

G.L.

, Integrated batch planning optimization based on fuzzy genetic and constraint satisfaction for steel production, International Journal of Simulation Modelling15(1) (2016), 133–143.

21.

Ríos

L.J.S.

, Ortiz

A.T.

, Colomo

R.J.A.

, and Gonzalez

B.G.

, Expert system for decision- making based on fuzzy logic and analytic hierarchy process to adverse weather, DYNA90 (2015), 503–512.

22.

Atanassov

, Intuitionistic fuzzy sets, Fuzzy Sets and Systems20(1) (1986), 87–96.

23.

Atanassov

, and Gargov

, Interval valued intuitionistic fuzzy sets, Fuzzy Sets and Systems31(3) (1989), 343–349.

24.

Atanassov

, Pasi

, and Yager

, Intuitionistic fuzzy interpretations of multi-person multi-criteria decision making, International Journal of Systems Science36(14) (2005), 859–868.

25.

, Multicriteria fuzzy decision-making method using entropy weights-based correlation coefficients of interval-valued intuitionistic fuzzy sets, Applied Mathematical Modeling34(12) (2010), 3864–3870.

26.

Atanassov

, and Georgiev

, Intuitionistic fuzzy Prolog, Fuzzy Sets and Systems53(2) (1993), 121–128.

27.

Khatibi

, and Montazer

, Intuitionistic fuzzy set vs. fuzzy set application in medical pattern recognition, Artificial Intelligence in Medicine47(1) (2009), 43–52.

28.

Zhao

, Xu

Z.S.

, Ni

M.F.

, and Cui

, Hybrid fuzzy multiple attribute decision making, Information12 (2009), 1033–1044.

29.

Park

D.G.

, Kwun

Y.C.

, and Park

J.H.

, Correlation coefficient of interval-valued intuitionistic fuzzy sets and its application to multiple attribute group decision making problem, Mathematical and Computer Modeling50(1) (2009), 1279–1293.

30.

, Expected value method for intuitionistic trapezoidal fuzzy multicriteria decision-making problems, Expert Systems with Applications38(9) (2011), 11730–11734.

31.

, and Nan

, Extension of the TOPSIS for multi-attribute group decision making under atanassov ifs environments, International Journal of Fuzzy System Applications1(4) (2011), 47–61.

32.

, Methods for aggregating interval-valued intuitionistic fuzzy information and their application to decision making, Control and Decision22(2) (2007), 215–219(in Chinese).

33.

Wei

, and Zhang

, Entropy measures for interval-valued intuitionistic fuzzy sets and their application in group decision-making, Mathematical Problems in Engineering2015 (2015), 1–13.