An intelligent approach for the evaluation of innovation projects

Abstract

In this study, an intelligent approach is presented for the evaluation and selection of innovation projects. Selecting the best innovation project is a complicated multiple criteria decision making (MCDM) problem with several potentially competing quantitative and qualitative criteria. In this paper, two hesitant fuzzy MCDM methods; hesitant fuzzy Analytic Hierarchy Process (hesitant F-AHP) and hesitant fuzzy VIsekriterijumska optimizacija i KOmpromisno Resenje (hesitant F-VIKOR) are integrated to evaluate and rank innovation projects. In the hesitant fuzzy AHP-VIKOR, hesitant F-AHP is used to find fuzzy evaluation criteria weights and hesitant F-VIKOR is implemented to rank innovation project alternatives. A numerical example is given where five innovation projects are evaluated based on nine criteria by three decision makers.

Keywords

Innovation project selection fuzzy multiple-criteria decision making AHP VIKOR

1 Introduction

In today’s business environment, innovation has been a critical point for companies under fierce world competition. With the effect of globalization winds, companies are increasingly forced to design and manufacture more innovative products, and compete with their competitors in national, as well as international level since consumers demand more sophisticated and innovative products [61]. On the other hand, innovation studies help strengthen the economies with a sustainable growth and prevent the economic crises [33]. As a result of this, most companies invest more sources in innovation studies to develop their competitive advantage (i.e. increasing market share and productivity), by spending on around 1-2% of turnover on innovation activities.

A recent global survey by the BCG on innovation practices in companies, reveals that innovation is one of their company’s top-three priorities [2]. Moreover, in near future, around 60% of companies plan to increase their innovation spending [2]. Spending on innovation-related activities are also encouraged by governments in various ways: for instance; implementing policies to stimulate R&D projects directly through grants or loans [33]. R&D projects are main elements of innovation and they help improve competitive advantage [18]. Hence, government funding and support of innovation projects by assisting companies results in designing and manufacturing more innovative products and better financial performance [47].

An intelligent method is needed to evaluate and prioritize innovation projects, especially when there are several competing qualitative and quantitative criteria to consider in the evaluation. In this research, an intelligent MCDM method is presented to evaluate innovation projects at the beginning stages of innovation in a context of limited resources. As the MCDM method, hesitant F-AHP and hesitant F-VIKOR are integrated. In hesitant fuzzy AHP-VIKOR, hesitant F-AHP is used to determine the fuzzy weights of the evaluation criteria since with hesitant F-AHP, using pairwise comparisons of both quantitative and qualitative criteria, dependable weights can be determined. However, when there are many alternatives and criteria, hesitant F-AHP, without the integration, can be tiresome since a great deal of pairwise comparisons are needed. Therefore, hesitant F-VIKOR is employed to evaluate alternatives, using the obtained fuzzy criteria weights. Hesitant F-VIKOR does not provide guidelines to determine criteria weights, so a reliable method such as hesitant F-AHP is needed in this integration.

AHP, introduced by Saaty in 1980 s [42], has a hierarchical and multi-level structure. In AHP, alternatives are evaluated based on various criteria and then ranked according to a total weighted score. VIKOR is mostly for problems with competing or non-commensurable criteria. In VIKOR, compromise ranking is performed between alternatives based on closeness to the ideal solution [36 , 52]. To capture the uncertainty and vagueness on judgments of decision-makers (DMs), their fuzzy extensions, F-AHP and F-VIKOR and integrated fuzzy AHP-VIKOR have been used in many MCDM problems in the literature. F-VIKOR is broadly used to find compromise solution(s) to problems with many alternatives and conflicting criteria. With F-VIKOR, an established compromise solution is determined with a maximum group utility of the majority and a minimum of individual regret of the opponent [23 , 35–38].

Traditional fuzzy AHP-VIKOR integration has been used in several applications in the literature such as public transportation bus selection [7], tyre recycling process selection [42], renewable energy and production site selection in Istanbul [19], material selection for pipes in sugar industry [1], fiscal performance evaluation of Iranian companies [14], manufacturing machine tool selection [16], biodiesel blend selection for IC engines [37], compliant polishing tool selection [2], the best knowledge flow practicing organization selection [8], evaluation of sectoral investments for sustainable development [46], quantifying risk ratings in occupational health and safety risk assessment [16] and selection of sustainable energy conversion technologies for agricultural residues [54]. In traditional F-AHP and F-VIKOR, DMs give a single linguistic expression to evaluate criteria and alternatives, respectively. However, this does not reflect the degrees of hesitation DMs might have in reality. For example, while evaluating an alternative with respect to a criterion in F-VIKOR, a DM might want to asses as “this alternative’s performance is between fair and medium good with respect to the considered criterion”. In this research, to allow these kind of “hesitant” expressions, to enrich the evaluation linguistic expressions and to represent the opinions of DMs more accurately in a more flexible manner, hesitant fuzzy linguistic term sets (HFLTS) are employed in fuzzy AHP-VIKOR.

The concept of hesitant fuzzy sets and HFLTS were introduced by Torra and Narukawa [49] and Rodrigez et al. [41] and in HFLTS, multi-linguistic expressions can be considered with linguistic variables. Usage of HFLTS in decision making improves the flexibility and enriches the multiple linguistic evaluations [22]. In the literature, HFLTS are used in a few applications. Tüysüz and Şimşek [50] developed a hesitant F-AHP to analyse and prioritize the factors affecting the performance of a cargo company in Turkey. Umamaheswari and Kumari [53] implemented triangular hesitant fuzzy sets in F-TOPSIS and F-VIKOR and presented a numerical example about teacher quality evaluation. Öztayşi et al. [39] developed a hesitant F-AHP and used it for supplier selection. Ayhan [6] applied hesitant F-AHP to determine criteria weights and then TOPSIS to evaluate and rank summer sport schools. Başar [9] used hesitant fuzzy pairwise comparisons for software cost estimation in a Turkish company. Dong et al. [14] developed a linguistic hesitant F-VIKOR approach and presented an intelligent transportation system evaluation example to demonstrate the effectiveness of the approach. Li et al. [22] implemented hesitant F-AHP to determine weights of criteria and then F-VIKOR to rank power generation enterprises. Camci et al. [12] applied hesitant F-AHP to select CNC router in small and medium-sized enterprises in woodwork manufacturing. Acar et al. [1] implemented hesitant F-AHP to evaluate sustainability of different hydrogen production methods. Narayanamoorthy et al. [32] used interval-valued intuitionistic hesitant fuzzy entropy based VIKOR for selection of industrial robots and presented a numerical example related to a material handling operation.

At present, there does not appear to be a research paper in the literature about the integration of hesitant F-AHP and hesitant F-VIKOR, and its application to evaluation of innovation projects. With the proposed hesitant fuzzy AHP-VIKOR, innovation projects are evaluated and ranked without too many tiresome pairwise comparisons and complex calculations. The MCDM literature related to project selection is given in Section 2 and steps of hesitant fuzzy AHP-VIKOR are explained in Section 3. In Section 4, a numerical example is given to explain the steps further, along with conclusions in Section 5.

2 Literature review

In the literature, there are many methods used for project selection [55]. Badri et al. [7] made a list of methods that are used for information system project selection problem. Dey [13] also listed project selection methods such as; fuzzy logic, GP with LP and AHP. Various kinds of project selection methods and corresponding decision problems can be found in Wang et al.’s research [55]. Wang et al. [55] also compared project selection methods such as cost analysis, linear and integer programming, fuzzy logic, AHP, ANP and Grey Target Decision in terms of their advantages and disadvantages.

Specifically, innovation project selection has been studied in various research papers. Mikkola [30] used an R & D Project Portfolio matrix that links the company’s competitive advantages to the benefits of projects to its customers and analyzed the portfolios of R & D projects, as an example, demonstrating battery samples for electric vehicles and hybrid electric vehicles. Melon et al. [29] implemented AHP to evaluate educational innovation projects for the Institute of Educational Sciences of the Politechnical University of Valencia. Zimmer, Yannou and Stal-Le Cardinal [60], as one of the foundations of radical innovation design methodology, created a selection model that measures innovative ideas, concepts or projects according to evidence of value, concept and innovation and they used this model in an innovation competition about development of new products and elderly services. Lerch and Spieth [21] worked on the cause-effect of innovation project portfolio management (IPPM) performance and have shown that performance should be considered as an integrated system consisting of portfolio balance, strategic compliance and value maximization, simultaneously. Later, Spieth and Lerch [45] tested hypotheses on the use of methods, the use of criteria, the IPPM design, the existence of knowledge, and the interactions between internal perceptions on management perception and satisfaction, and on IPPM performance and innovation. As a result, the methods and criteria are determined to have an insignificant role, however IPPM design elements such as transparency and formality are found to be the key elements in IPPM performance. They also determined that the existence of knowledge and management perception and satisfaction are key structures in IPPM performance. Mitchell, Phaal and Athanassopoulou [31] presented a multi-factor scoring system design approach to evaluate and select early stage innovation projects. Brook and Pagnanelli [11] focused on the automotive industry and prioritized the innovation projects. First, they determined the weights of AHP and then with the weighted average technique SMART, they calculated the total values of the projects with and prioritized the projects according to these values. Bin et al. [10] used a two-phase method for the selection of R & D and innovation projects in the Brazilian electricity sector, solving a backpack problem in the first phase, choosing projects that would provide the greatest benefit considering the limited resources, and then sorting these projects using the PROMETHEE II method. Barcic et al. [8] deconstructed innovation in the Croatian furniture industry to three components: product innovation, process innovation, human resource innovation. Then they evaluated the relationship between these components and 4 company management activities / factors (R & D investments, company flexibility, export activity and internet use). The links between innovation components and business practices have revealed the importance of management activities in the furniture industry for innovation studies and development. Meifort [28] conducted a large literature review on innovation portfolio management and discussed the study in 4 perspectives: the perspective of optimization, strategic perspective, decision-making perspective and organizational perspective. Rathi et al. [40] used F-VIKOR to evaluate and prioritize six sigma projects in the Indian auto sector. Lin et al. [24] evaluated the critical success factors affecting project execution, integrating F-AHP and VIKOR. In their research, first F-AHP was used to determine weights and then VIKOR was used to rank factors in environmental consultancy service industry in Taiwan. He and Xu [17] implemented three MCDM methods: the probabilistic hesitant fuzzy reference ideal method (PHFRIM), the technique for order preference by similarity to a reference ideal solution (TOPSRIS) and the reference ideal VIKOR (RI-VIKOR) to the total dissolved gas evaluation problem of water conservancy projects and made a comparison analysis.

To the best of authors knowledge, in the literature, fuzzy AHP-VIKOR or hesitant fuzzy AHP-VIKOR methods have never been applied to an innovation project selection problem. Next, the steps of the proposed approach; hesitant fuzzy AHP-VIKOR are explained more in detail.

3 Hesitant fuzzy AHP-VIKOR Approach

3.1 Definitions

In fuzzy set theory, there are classes with unsharp boundaries [20, 27] and crisp theories can be fuzzified by applying the concept of a fuzzy set [59] to the set within that theory. In the proposed hesitant fuzzy AHP-VIKOR approach, triangular fuzzy numbers (TFNs) are utilized due to its simplicity. A fuzzy number is a special fuzzy set F ={ (x, μ_F (x)) , x ∈ R } where x is a real number, R : -∞ < x < + ∞ and μ_F (x) is a continuous mapping from R to [0, 1]. A TFN denoted as $\tilde{M} = (l, m, ut)$ , where l ≤ m ≤ u, has the following triangular type membership function; $μ_{F} (x) = {\begin{matrix} 0 \\ x - l / m - l \\ u - x / u - m \\ 0 \end{matrix} \begin{matrix} x < l \\ l \leq x \leq m \\ m \leq x \leq u \\ x > u \end{matrix}$ (1)

Basic operations between two positive TFNs $\tilde{A} = (l_{1}, m_{1}, u_{1})$ , $\tilde{B} = (l_{2}, m_{2}, u_{2}) l_{1} \leq m_{1} \leq u_{1,} l_{2} \leq m_{2} \leq u_{2}$ and a crisp number C are explained as [3, 56]: $\tilde{A} + \tilde{B} = (l_{1} + l_{2}, m_{1} + m_{2}, u_{1} + u_{2})$ (2) $\tilde{A} + C = (l_{1} + C, m_{1} + C, u_{1} + C)$ (3) $\tilde{A} - \tilde{B} = (l_{1} - u_{2}, m_{1} - m_{2}, u_{1} - l_{2})$ (4) $\tilde{A} - C = (l_{1} - C, m_{1} - C, u_{1} - C)$ (5) $\tilde{A} * \tilde{B} = (l_{1} * l_{2}, m_{1} * m_{2}, u_{1} * u_{2})$ (6)

$\begin{matrix} \tilde{A} * C = (l_{1} * C, m_{1} * C, u_{1} * C) \\ forC ⩾ 0 \end{matrix}$ (7) $\frac{\tilde{A}}{\tilde{B}} = (\frac{l_{1}}{u_{2}}, \frac{m_{1}}{m_{2}}, \frac{u_{1}}{l_{2}})$ (8) $\frac{\tilde{A}}{\tilde{B}} = (min (\frac{l_{1}}{l_{2}}, \frac{l_{1}}{u_{2}}, \frac{u_{1}}{l_{2}}, \frac{u_{1}}{u_{2}}), \frac{m_{1}}{m_{2}}, max (\frac{l_{1}}{l_{2}}, \frac{l_{1}}{u_{2}}, \frac{u_{1}}{l_{2}}, \frac{u_{1}}{u_{2}}))$ (9) if $\tilde{A}$ and $\tilde{B}$ are two TFNs (not necessarily positive TFNs). $\frac{\tilde{A}}{C} = (\frac{l_{1}}{C}, \frac{m_{1}}{C}, \frac{u_{1}}{C}) forC > 0$ (10) ${\tilde{A}}^{- 1} = (\frac{1}{u_{1}}, \frac{1}{m_{1}}, \frac{1}{l_{1}})$ (11)

$\begin{matrix} MAX (\tilde{A} + \tilde{B}) = \\ (max (l_{1}, l_{2}), max (m_{1}, m_{2}), max (u_{1}, u_{2})) \end{matrix}$ (12)

$\begin{matrix} MIN (\tilde{A} + \tilde{B}) = \\ (min (l_{1}, l_{2}), min (m_{1}, m_{2}), min (u_{1}, u_{2})) \end{matrix}$ (13)

As the defuzzification method of TFNs, the graded mean integration approach [57] is used as seen below: $Crisp (\tilde{A}) = (4 m_{1} + l_{1} + u_{1}) / 6$ (14)

In determining the membership degree of an element, a DM might hesitate while making judgements since there may be more than one possible value. The concept of hesitant fuzzy sets (HFS) was introduced to address this problem [48, 49]. In Triangular Fuzzy Hesitant Fuzzy Sets (TFHFS), the membership degree of an element to a given set is expressed by several possible TFNs. In the literature, several aggregation operators for TFHFS were proposed by Yu [58] and Yu applied these concepts to a teaching quality evaluation problem.

If X is a fixed set, the HFS on X returns a subset of [0, 1] by: devendran

E = {〈 x, h_{E} (x) 〉 | x \in X}

(15)

where h_E (x)is the possible membership degrees of element x ∈ Xto set E by taking values in [0, 1]. The lower and upper bounds are calculated as: $h_{(x)}^{-} = min h (x), h_{(x)}^{+} = max h (x)$ (16)

Basic operations for 3 HFS h, h₁, h₂ are as follows: $h^{ℷ} = \cup_{γ \in h} {γ^{ℷ}}$ (17) $ℷ h = \cup_{γ \in h} {1 - {(1 - γ)}^{ℷ}}$ (18) $h_{1} \pm h_{2} = \cup_{γ 1 \in h 1, γ 2 \in h 2} {γ^{1} + γ^{2} - γ^{1} γ^{2}}$ (19) $h_{1} \cap h_{2} = \cup_{γ 1 \in h 1, γ 2 \in h 2} \min {γ^{1}, γ^{2}}$ (20) $h_{1} \cup h_{2} = \cup_{γ 1 \in h 1, γ 2 \in h 2} \max {γ^{1}, γ^{2}}$ (21) $h_{1} \otimes h_{2} = \cup_{γ 1 \in h 1, γ 2 \in h 2} {γ^{1} γ^{2}}$ (22)

An Ordered Weighting Averaging (OWA) operator that can be used in the process is as follows: $OWA (a_{1}, a_{2}, \dots, a_{n}) = \sum_{j = 1}^{n} w_{j} b_{j}$ (23) where, b_j is the jth largest of a₁, a₂, …, a_n, $w_{j} \in [0, 1] \forall jand \sum_{j = 1}^{n} w_{j} = 1 .$ [9]

3.2 Combining DM evaluations in hesitant fuzzy AHP-VIKOR

DM evaluations are combined by fuzzy envelope approach [26] in hesitant F-AHP and hesitant F-VIKOR steps. Scales given for DM evaluations are sorted from the lowest s_oto the highest s_g so if the DM’s evaluations are between s_i and s_j, then s_o ≤ s_i ≤ s_j ≤ s_g.

Based on the hesitant fuzzy linguistic term sets, linguistic expressions can be represented by a triangular fuzzy membership function $\tilde{A} =$ (a,b,c), where a, b and c are calculated as: $a = \min {a_{L}^{i}, a_{M}^{i}, a_{M}^{i + 1}, \dots, a_{M}^{j}, a_{R}^{j}} = a_{L}^{i}$ (24) $b = {\begin{matrix} a_{M}^{i} if i + 1 = j \\ {OWA}_{W} {a_{M}^{i}, a_{M}^{i + 1}, \dots, a_{M}^{j}} otherwise \end{matrix}$ (25) $c = \max {a_{L}^{i}, a_{M}^{i}, a_{M}^{i + 1}, \dots, a_{M}^{j}, a_{R}^{j}} = a_{R}^{j}$ (26)

Weight vector in OWA operator is defined in Filev and Yager’s [15] research as: $w_{1} = α^{n - 1}, w_{2} = (1 - α) α^{n - 2}, \dots,$ (27) $w_{n} = (1 - α), where$ $α = (g - j + i) / (g - 1)$

Here, g depends on the number of terms in DM’s evaluation scale (in Table 1 or Table 2), j is the rank of the highest, and i is the rank of the lowest evaluation value. i and j can take ranks starting from 0 to g and n = j-i. [9]

Table 1

Scale for the evaluation of criteria in hesitant F-AHP [44]

Linguistic terms	Triangular fuzzy number (TFN)
Absolutely strong (AS)	(2, 5/2, 3)
Very strong (VS)	(3/2, 2, 5/2)
Fairly strong (FS)	(1, 3/2, 2)
Slightly strong (SS)	(1, 1, 3/2)
Equal (E)	(1, 1, 1)
Slightly weak (SW)	(2/3, 1, 1)
Fairly weak (FW)	(1/2, 2/3, 1)
Very weak (VW)	(2/5, 1/2, 2/3)
Absolutely weak (AW)	(1/3, 2/5, 1/2)

Table 2

Scale for the ratings of alternatives in hesitant F-VIKOR [44]

Linguistic terms	Triangular fuzzy number (TFN)
Very poor (VP)	(0,0,1)
Poor (P)	(0,1,3)
Medium poor (MP)	(1,3,5)
Fair (F)	(3,5,7)
Medium good (MG)	(5,7,9)
Good (G)	(7,9,10)
Very good (VG)	(9,10,10)

3.3 Finding the fuzzy weights of criteria with hesitant F-AHP

In hesitant F-AHP, the DMs make pairwise comparisons of criteria to determine the fuzzy importance weights using the linguistic terms given in Table 1.

Computational Steps of F-AHP: [3–5 , 43]

Step 1: Identify K DMs, n criteria; and linguistic terms and scale for the pairwise comparison of criteria. Based on the scale used in Table 1 and Equations (23)-(27), combine DM’s evaluations with fuzzy envelope approach and obtain TFNs corresponding to the evaluation of each DM. Calculate ${\tilde{x}}_{ij} = \frac{1}{K} ({\tilde{x}}_{ij}^{1} (+) {\tilde{x}}_{ij}^{2} (+) . . . (+) {\tilde{x}}_{ij}^{K})$ where ${\tilde{x}}_{ij}^{K} = (a_{ij}^{K}, b_{ij}^{K}, c_{ij}^{K}) \forall i, j, k$ is the TFN corresponding to the evaluation of the K^th DM.

Step 2: $\tilde{X} = [\begin{matrix} (1, 1, 1) & {\tilde{x}}_{12} & . . & . . & {\tilde{x}}_{1 n} \\ {\tilde{x}}_{21} & (1, 1, 1) & . . & . . & {\tilde{x}}_{2 n} \\ . . & . . & . . & . . & . . \\ . . & . . & . . & . . & . . \\ {\tilde{x}}_{n 1} & {\tilde{x}}_{n 2} & . . & . . & (1, 1, 1) \end{matrix}]$ with elements ${\tilde{x}}_{ij} = (a_{ij}, b_{ij}, c_{ij})$ is normalized and $\tilde{S}$ is obtained. $\tilde{S} = [\begin{matrix} {\tilde{s}}_{11} & {\tilde{s}}_{12} & . . & . . & {\tilde{s}}_{1 n} \\ {\tilde{s}}_{21} & {\tilde{s}}_{22} & . . & . . & {\tilde{s}}_{2 n} \\ . . & . . & . . & . . & . . \\ . . & . . & . . & . . & . . \\ {\tilde{s}}_{n 1} & {\tilde{s}}_{n 2} & . . & . . & {\tilde{s}}_{nn} \end{matrix}]$ , where ${\tilde{s}}_{ij} = (\frac{a_{ij}}{\sum_{i} c_{ij}}, \frac{b_{ij}}{\sum_{i} b_{ij}}, \frac{c_{ij}}{\sum_{i} a_{ij}})$ . Fuzzy weight vector ${\tilde{w}}_{criteria} = ({\tilde{w}}_{1}, {\tilde{w}}_{2}, . . ., {\tilde{w}}_{n})$ is determined by averaging the entries on each row of $\tilde{s}$ .

Step 3: $\tilde{X}$ is defuzzified with Equation (14) and w_cr = (w₁, w₂, . . . , w_n) (approximate crisp criteria weights) is determined by averaging the entries on each row of normalized X. So the normalized principal eigen vector is w_cr. The largest eigen value, called the principal eigen value (λ_max) is determined with Equation (28). ${Xw}_{cr}^{T} = λ_{max} w_{cr}^{T}$ (28)

The consistency index (CI) is calculated as: $CI = \frac{λ_{max} - n}{n - 1}$ (29)

The consistency ratio (CR) in Equation (30) is used to estimate the consistency of pairwise comparisons. $CR = \frac{CI}{RI}$ (30)

RI is the average index for randomly generated weights [42]. If CR < 0.10, the comparisons are acceptable, otherwise they are not [42].

3.4 Ranking Alternatives with hesitant F-VIKOR

After finding ${\tilde{w}}_{criteria}$ with hesitant F-AHP, hesitant F-VIKOR is used to rank the alternatives. In hesitant F-VIKOR, the alternatives are evaluated using the linguistic terms given in Table 2.

Computational Steps of hesitant F-VIKOR: [19 , 56]

Step 1: Identify m alternatives; and linguistic terms for the evaluations of alternatives with respect to each criterion.

Based on the scale used in Table 2 and Equations (23)-(27), combine DM’s evaluations with fuzzy envelope approach and obtain TFNs corresponding to the evaluation of each DM. Calculate ${\tilde{r}}_{ij} = \frac{1}{K} ({\tilde{r}}_{ij}^{1} (+) {\tilde{r}}_{ij}^{2} (+) . . . (+) {\tilde{r}}_{ij}^{K})$ , where ${\tilde{r}}_{ij}^{K} = (a_{ij}^{K}, b_{ij}^{K}, c_{ij}^{k})$ is the TFN corresponding to the evaluation of the K^th DM. After this aggregation, the fuzzy matrix, including m alternatives that are evaluated in terms of n criteria, is given as: $\tilde{D} = [\begin{matrix} {\tilde{r}}_{11} & {\tilde{r}}_{12} & . . & . . & {\tilde{r}}_{1 n} \\ {\tilde{r}}_{21} & {\tilde{r}}_{22} & . . & . . & {\tilde{r}}_{2 n} \\ . . & . . & . . & . . & . . \\ . . & . . & . . & . . & . . \\ {\tilde{r}}_{n 1} & {\tilde{r}}_{n 2} & . . & . . & {\tilde{r}}_{mn} \end{matrix}]$ ,

where ${\tilde{r}}_{ij} = (a_{ij}, b_{ij}, c_{ij}) \forall i, j$ are positive TFNs.

Step 2: Determine the fuzzy best value (FBV, ${\tilde{f}}_{j}^{*}$ ) and the fuzzy worst value (FWV, ${\tilde{f}}_{j}^{-}$ ) for each criterion. ${\tilde{f}}_{j}^{*} = max_{i} {\tilde{r}}_{ij} \forall j, {\tilde{f}}_{j}^{-} = min_{i} {\tilde{r}}_{ij} \forall j$ (31)

Step 3: Calculate the separation measures of each alternative from the FBV ( ${\tilde{S}}_{i}$ ) and FWV ( ${\tilde{R}}_{i}$ ). ${\tilde{S}}_{i} = \sum_{j = 1}^{n} {\tilde{w}}_{j} ({\tilde{f}}_{j}^{*} - {\tilde{r}}_{ij}) / ({\tilde{f}}_{j}^{*} - {\tilde{f}}_{j}^{-}) \forall i$ (32) ${\tilde{R}}_{i} \underset{j}{= max} [w˜j (f˜j* - r˜ij) / (f˜j* - f˜j-)] \forall i$ (33)

Step 4: Calculate ${\tilde{S}}^{*}, {\tilde{S}}^{-}, {\tilde{R}}^{*}, {\tilde{R}}^{-}$ values as: ${\tilde{S}}^{*} = min_{i} {\tilde{S}}_{i}, {\tilde{S}}^{-} = max_{i} {\tilde{S}}_{i}$ (34) ${\tilde{R}}^{*} = min_{i} {\tilde{R}}_{i}, {\tilde{R}}^{-} = max_{i} {\tilde{R}}_{i}$ (35)

Step 5: Calculate ${\tilde{Q}}_{i}$ values for each alternative. $\begin{matrix} {\tilde{Q}}_{i} = ν ({\tilde{S}}_{i} - {\tilde{S}}^{*}) / ({\tilde{S}}^{-} - {\tilde{S}}^{*}) + \\ (1 - ν) ({\tilde{R}}_{i} - {\tilde{R}}^{*}) / ({\tilde{R}}^{-} - {\tilde{R}}^{*}) \forall i \end{matrix}$ (36)

Here, ν is the weight of the strategy of the maximum group utility (majority of criteria) and 1 - ν is the weight of the individual regret. ν is usually assumed to be 0.5 (by consensus). [3, 19].

Step 6: Defuzzify the ${\tilde{Q}}_{i}$ values with Equation (14) and rank the alternatives based on crisp Q_i values so that the alternative with the minimum Q_i value is the best.

4 Numerical example

In this study, 9 criteria (C1, C2, ... , C9) are determined for the evaluation of innovation projects. These are listed in Table 3. Five alternative projects (A, B, C, D, E) are going to be evaluated with respect to these criteria by three DMs (DM1, DM2, DM3).

Table 3
Evaluation criteria for innovation projects

C1 Experienced labor force existence C6 Innovativeness

C2 Infrastructure possibilities qualifications C7 Social influence

C3 Existing technological adequacy C8 Available use of resources

C4 Feasibility C9 Commercial success potential, profitability and return on investment

C5 Completion time

First, fuzzy criteria weights are obtained with hesitant F-AHP. In hesitant F-AHP, DMs make pairwise comparison of criteria with the linguistic terms in Table 1 and these comparisons are presented in Table 4.

Table 4

Pairwise comparison of evaluation criteria by 3 DMs

	C1	C2	C3	C4	C5	C6	C7	C8	C9
C1	E	FS	between VW,E	SW	between SS,VS	E	SS	SW	FS
	E	SS	E	E	VS	E	SS	E	SS
	E	E	FW	FS	E	FS	E	FS	E
C2		E	AS	FS	between SS, AS	between FW,SS	E	VS	FS
		E	E	E	SS	VS	FW	VS	E
		E	E	SW	E	E	SS	E	SW
C3			E	E	VW	E	VS	FS	VW
			E	SW	VS	VW	VS	between SS, VS	E
			E	between E, FS	E	between VW, SW	SS	E	FW
C4				E	AS	VS	E	FS	VS
				E	E	E	SS	E	VS
				E	AW	E	VS	SW	SS
C5					E	E	E	VS	between E, VS
					E	VW	FW	E	E
					E	FS	SS	E	AW
C6						E	E	AS	VW
						E	between FW, E	E	E
						E	E	E	FW
C7						E	AS	SW
							E	E	E
							E	AW	between FW, FS
C8								E	VW
								E	E
								E	FW
C9									E
									E
									E

After the combination of each DM’s evaluations with fuzzy envelope approach and aggregation of the corresponding TFNs of 3 DMs evaluations, the fuzzy evaluation matrix for the criteria weights ( $\tilde{X}$ ) in Table 5 is determined. Then, ${\tilde{w}}_{criteria} = ({\tilde{w}}_{1}, {\tilde{w}}_{2}, . . ., {\tilde{w}}_{n})$ presented in Table 6, is calculated by averaging the entries on each row of normalized $\tilde{X}$ ( $\tilde{S}$ ). To check the CR of $\tilde{X}$ , Equation (14) is used for defuzzification. Here, CI = (9.7231–9)/8 = 0.0904 and CR = CI/RI = 0.0904/1.45 = 0.0623, implementing Equations (28)-(30). Since CR < 0.1, the comparisons are determined to be consistent.

Table 5

The fuzzy evaluation matrix for the criteria weights ( $\tilde{X}$ )

	C1	C2	C3	C4	C5
C1	(1.000,1.000,1.000)	(1.000,1.167,1.500)	(0.633,0.902,1.000)	(0.890,1.167,1.333)	(1.167,1.643,2.000)
C2	(0.723,0.890,1.000)	(1.000,1.000,1.000)	(1.333,1.500,1.667)	(0.890,1.371,1.667)	(0.833,1.000,1.333)
C3	(1.000,1.155,1.833)	(0.777,0.800,0.833)	(1.000,1.000,1.000)	(0.890,1.143,1.333)	(0.967,1.167,1.390)
C4	(0.833,0.890,1.167)	(0.778,0.824,1.167)	(0.833,0.900,1.167)	(1.000,1.000,1.000)	(1.110,1.300,1.500)
C5	(0.600,0.673,0.890)	(0.779,1.000,1.333)	(0.967,1.167,1.390)	(1.110,1.300,1.500)	(1.000,1.000,1.000)
C6	(0.833,0.890,1.000)	(0.800,1.000,1.223)	(1.167,1.350,2.000)	(0.800,0.833,0.890)	(1.000,1.223,1.500)
C7	(0.780,1.000,1.000)	(0.890,1.167,1.333)	(0.490,0.667,0.780)	(0.690,0.833,0.890)	(0.890,1.167,1.333)
C8	(0.833,0.890,1.167)	(0.600,0.667,0.780)	(0.633,0.730,1.000)	(0.833,0.890,1.167)	(0.800,0.833,0.890)
C9	(0.723,0.890,1.000)	(0.833,0.890,1.167)	(1.167,1.500,1.833)	(0.490,0.667,0.780)	(1.133,1.373,1.667)
	C6	C7	C8	C9
C1	(1.000,1.167,1.333)	(1.000,1.000,1.333)	(0.890,1.167,1.333)	(1.000,1.167,1.500)
C2	(1.000,1.223,1.500)	(0.833,0.890,1.167)	(1.333,1.667,2.000)	(0.890,1.167,1.333)
C3	(0.600,0.817,0.890)	(1.333,1.667,2.167)	(1.000,1.476,1.833)	(0.633,0.723,0.890)
C4	(1.167,1.333,1.500)	(1.167,1.333,1.667)	(0.890,1.167,1.333)	(1.333,1.667,2.167)
C5	(0.800,1.000,1.223)	(0.833,0.890,1.167)	(1.167,1.333,1.500)	(0.777,1.004,1.333)
C6	(1.000,1.000,1.000)	(0.833,1.000,1.000)	(1.333,1.500,1.667)	(0.633,0.723,0.890)
C7	(1.000,1.000,1.333)	(1.000,1.000,1.000)	(1.110,1.300,1.500)	(0.723,1.031,1.333)
C8	(0.777,0.800,0.833)	(1.110,1.300,1.500)	(1.000,1.000,1.000)	(0.633,0.723,0.890)
C9	(1.167,1.500,1.833)	(0.833,0.972,1.500)	(1.167,1.500,1.833)	(1.000,1.000,1.000)

Table 6

Fuzzy criteria weights ${\tilde{w}}_{criteria} = ({\tilde{w}}_{1}, {\tilde{w}}_{2}, . . ., {\tilde{w}}_{n})$ determined with hesitant F-AHP

Criteria	Fuzzy weights	Criteria	Fuzzy weights
C1	(0.083,0.119,0.166)	C6	(0.080,0.108,0.149)
C2	(0.084,0.122,0.170)	C7	(0.072,0.105,0.141)
C3	(0.078,0.113,0.163)	C8	(0.069,0.090,0.125)
C4	(0.088,0.119,0.171)	C9	(0.081,0.117,0.167)
C5	(0.077,0.107,0.153)

${\tilde{w}}_{criteria} = ({\tilde{w}}_{1}, {\tilde{w}}_{2}, . . ., {\tilde{w}}_{n})$ , that is obtained with hesitant F-AHP, is used in hesitant F-VIKOR to rank innovation projects. In hesitant F-VIKOR, DMs evaluate alternatives with respect to each criterion using the linguistic terms in Table 2 and these are given in Table 7. After the combination of each DM’s evaluations with fuzzy envelope approach and aggregation of the corresponding TFNs of 3 DMs evaluations, the fuzzy evaluation matrix $\tilde{D}$ is presented in Table 8. The FBV ( ${\tilde{f}}_{j}^{*}$ ) and FWV ( ${\tilde{f}}_{j}^{-}$ ) for each criterion are also presented in Table 8. The separation measures of each alternative ${\tilde{S}}_{i}$ and ${\tilde{R}}_{i}$ are given in Table 9, along with ${\tilde{S}}^{*}, {\tilde{S}}^{-}, {\tilde{R}}^{*}, {\tilde{R}}^{-}$ values. Based on these, ${\tilde{Q}}_{i}$ value for each alternative is calculated and ${\tilde{Q}}_{i}$ values are defuzzified with Equation (14) (Q_i) to rank the alternatives. The results of the hesitant fuzzy AHP-VIKOR are presented in Table 10. As seen from these results, the innovation projects are ranked as follows; D, E, A, B and C.

Table 7

3 DMs’ evaluation scores of the innovation projects with respect to each criterion

	C1	C2	C3	C4	C5	C6	C7	C8	C9
A	between P,MG	F	VP	G	P	MG	P	G	VP
	MP	F	F	VG	F	MP	G	VG	F
	VP	between MP,G	G	MP	F	VP	F	MP	G
B	P	VP	P	VP	P	P	G	P	P
	P	F	VP	F	P	VP	VG	VP	VG
	G	G	between VP,F	G	G	F	MP	F	G
C	G	P	MG	P	between P,F	F	G	MG	P
	VG	F	MP	G	F	between F,G	G	between F,VG	VP
	between MP,G	F	VP	F	F	G	F	VP	F
D	VP	P	P	G	P	P	P	P	VP
	F	G	F	between VP,P	VP	P	VP	G	F
	G	G	VG	MP	F	G	between MP,MG	F	G
E	F	VG	P	MG	G	G	MG	G	P
	F	F	P	MP	F	VG	MP	VG	F
	G	G	G	VP	G	MP	VP	MP	between MP,G

Table 8

Fuzzy evaluation matrix ( $\tilde{D}$ ) for the alternatives and fuzzy best values (FBV) and fuzzy worst values (FWV)

	C1	C2	C3	C4	C5
A	(0.333,2.117,5.000)	(2.333,5.117,8.000)	(3.333,4.667,6.000)	(5.667,7.333,8.333)	(2.000,3.667,5.667)
B	(2.333,3.667,5.333)	(3.333,4.667,6.000)	(0.000,1.307,3.667)	(3.333,4.667,6.000)	(2.333,3.667,5.333)
C	(5.667,8.117,10.00)	(2.000,3.667,5.667)	(2.000,3.333,5.000)	(3.333,5.000,6.667)	(2.000,4.627,7.000)
D	(3.333,4.667,6.000)	(4.667,6.333,7.667)	(4.000,5.333,6.667)	(2.667,4.333,6.000)	(1.000,2.000,3.667)
E	(4.333,6.333,8.000)	(6.333,8.000,9.000)	(2.333,3.667,5.333)	(2.000,3.333,5.000)	(5.667,7.667,9.000)
FBV	(5.667,8.117,10.00)	(6.333,8.000,9.000)	(4.000,5.333,6.667)	(5.667,7.333,8.333)	(5.667,7.667,9.000)
FWV	(0.333,2.117,5.000)	(2.000,3.667,5.667)	(0.000,1.307,3.667)	(2.000,3.333,5.000)	(1.000,2.000,3.667)
	C6	C7	C8	C9
A	(2.000,3.333,5.000)	(3.333,5.000,6.667)	(5.667,7.333,8.333)	(3.333,4.667,6.000)
B	(1.000,2.000,3.667)	(5.667,7.333,8.333)	(1.000,2.000,3.667)	(5.333,6.667,7.667)
C	(4.333,7.293,9.000)	(5.667,7.667,9.000)	(2.667,4.640,6.667)	(1.000,2.000,3.667)
D	(2.333,3.667,5.333)	(0.333,2.293,4.333)	(3.333,5.000,6.667)	(3.333,4.667,6.000)
E	(5.667,7.333,8.333)	(2.000,3.333,5.000)	(5.667,7.333,8.333)	(1.333,3.784,6.667)
FBV	(5.667,7.333,9.000)	(5.667,7.667,9.000)	(5.667,7.333,8.333)	(5.333,6.667,7.667)
FWV	(1.000,2.000,3.667)	(0.333,2.293,4.333)	(1.000,2.000,3.667)	(1.000,2.000,3.667)

Table 9

${\tilde{S}}_{i}$ , ${\tilde{R}}_{i}$ , ${\tilde{S}}^{*}, {\tilde{S}}^{-}, {\tilde{R}}^{*}, {\tilde{R}}^{-}$ values

	${\tilde{S}}_{i}$	${\tilde{R}}_{i}$	${\tilde{Q}}_{i}$
A	(–1.198,0.478,8.680)	(0.007,0.119,2.412)	(–1.347,0.603,1.063)
B	(–0.249,0.655,10.045)	(0.020,0.113,3.254)	(–1.687,0.706,1.000)
C	(–1.277,0.468,8.684)	(0.020,0.122,2.278)	(–1.306,0.685,1.063)
D	(–0.915,0.580,8.298)	(0.019,0.107,1.663)	(–1.095,0.375,1.041)
E	(–1.317,0.358,7.881)	(0.009,0.119,2.115)	(–1.207,0.403,1.069)
${\tilde{S}}^{*}$	(–1.317,0.358,7.881)	${\tilde{R}}^{*}$	(0.007,0.107,1.663)
${\tilde{S}}^{-}$	(–0.249,0.655,10.045)	${\tilde{R}}^{-}$	(0.020,0.122,3.254)

Table 10

Hesitant fuzzy AHP-VIKOR results for innovation project selection

	${\tilde{Q}}_{i}$	Q _i	Rank
A	(–1.347,0.603,1.063)	0.354	3
B	(–1.687,0.706,1.000)	0.356	4
C	(–1.306,0.685,1.063)	0.416	5
D	(–1.095,0.375,1.041)	0.241	1
E	(–1.207,0.403,1.069)	0.246	2

5 Conclusions

In this paper, an integrated hesitant fuzzy AHP-VIKOR method is presented for the evaluation of innovation projects. In this method, hesitant F-AHP is used to determine the fuzzy criteria weights, and hesitant F-VIKOR is used to rank alternatives utilizing these fuzzy weights.

At present, there does not appear to be a study in the literature that integrates hesitant F-AHP and hesitant F-VIKOR and also an application of this integtation to the evaluation of innovation projects. Here, to model uncertainty and vagueness of judgments of decision makers, linguistic terms and fuzzy numbers are used, and in the presence of competing criteria, alternatives are ranked without too many repetitive pairwise comparisons and complex calculations. Moreover, usage of hesitant fuzzy linguistic terms in decision making provides DMs flexibility and also reflects the degree of hesitation DMs might have in reality. In this research, outer-dependence, inner-dependence, and feedback relations that might occur between criteria are not taken into consideration due to the limitations of hesitant F-AHP, however, for future research, these relations between criteria can be investigated with hesitant F-ANP and hesitant F-ANP can be integrated with hesitant F-VIKOR for evaluation and ranking problems.

References

Acar

, Beskese

and Temur

G.T.

, Sustainability analysis of different hydrogen production options using hesitant fuzzy AHP, Int J Hydrogen Energy 43 (2018), 18059–18076.

Andrew

J.P.

, Manget

, Michael

D.C.

, Taylor

and Zablit

, M2: Innovation 2010: A return to prominence - and the emergence of a New World Order, 2010.

Anojkumar

, Ilangkumaran

and Sasirekha

, Comparative analysis of MCDM methods for pipe material selection in sugar industry, Expert Syst Appl 41 (2014), 2964–2980.

Ayağ

, A fuzzy AHP-based simulation approach to concept evaluation in a NPD environment, IIE Trans 37 (2005), 827–842.

Ayağ

and Özdemir

R.G.

, A Fuzzy AHP Approach to Evaluating Machine Tool Alternatives, J Intell Manuf 17 (2006), 179–190.

Ayhan

M.B.

, An Integrated Hesitant Fuzzy AHP and TOPSIS Approach for Selecting Summer Sport School, Sak Univ J Sci 22 (2018), 1–1.

Badri

M.A.

, Davis

and Davis

, A comprehensive 0–1 goal programming model for project selection, Int J Proj Manag 19 (2001), 243–252.

Barčič

A.P.

, Motik

, Oblak

and Vlosky

, Management activity linkages to innovation deconstruction: An exploratory study of the furniture industry in Croatia, BioResources 11 (2016), 3987–4005.

Başar

, Hesitant fuzzy pairwise comparison for software cost estimation: A case study in Turkey, Turkish J Electr Eng Comput Sci 25 (2017), 2897–2909.

10.

Bin

, Azevedo

, Duarte

, Salles-Filho

and Massaguer

, R&D and innovation project selection: Can optimization methods be adequate? Procedia Comput Sci 55 (2015), 613–621.

11.

Brook

J.W.

and Pagnanelli

, Integrating sustainability into innovation project portfolio management - A strategic perspective, J Eng Technol Manag - JET-M 34 (2014), 46–62.

12.

Camci

, Temur

G.T.

and Beskese

, CNC router selection for SMEs in woodwork manufacturing using hesitant fuzzy AHP method, J Enterp Inf Manag 31 (2018), 529–549.

13.

Dey

P.K.

, Integrated project evaluation and selection using multiple-attribute decision-making technique, Int J Prod Econ 103 (2006), 90–103.

14.

Ying Dong

, Fang

Yuan F.

and Ping

Wan S.

, Extended VIKOR method for multiple criteria decision-making with linguistic hesitant fuzzy information, Comput Ind Eng 112 (2017), 305–319.

15.

Filev

and Yager

R.R.

, On the issue of obtaining OWA operator weights, Fuzzy Sets Syst 94 (1998), 157–169.

16.

Gul

and Ak

M.F.

, A comparative outline for quantifying risk ratings in occupational health and safety risk assessment, J Clean Prod 196 (2018), 653–664.

17.

and Xu

, Multi-attribute decision making methods based on reference ideal theory with probabilistic hesitant information, Expert Syst Appl 118 (2019), 459–469.

18.

Heneric

, Licht

, Sofka

, eds., Europe’s Automotive Industry on the Move, Competitiveness in a Changing World 32(1), Physica-Verlag Heidelberg, 2005.

19.

Kaya

and Kahraman

, Multicriteria renewable energy planning using an integrated fuzzy VIKOR & AHP methodology: The case of Istanbul, Energy 35 (2010), 2517–2527.

20.

Klir

G.J.

, Yuan

, Fuzzy Sets and Fuzzy Logic: Theory and Applications, 1st ed., Springer US, 1995.

21.

Lerch

and Spieth

, Innovation project portfolio management: A qualitative analysis, IEEE Trans Eng Manag 60 (2013), 18–29.

22.

, Dong

and Wang

, Competition ability evaluation of power generation enterprises using a hybrid MCDM method under fuzzy and hesitant linguistic environment, J Renew Sustain Energy 10 (2018).

23.

Liang

, Zhang

, Xu

and Jamaldeen

, Pythagorean fuzzy VIKOR approaches based on TODIM for evaluating internet banking website quality of Ghanaian banking industry, Appl Soft Comput J 78 (2019), 583–594.

24.

Lin

, Chang

T.H.

, Fu

H.P.

, Yeh

H.P.

, Hsu

K.Y.

and Tsai

H.Y.

, Analysis of the CSFs affecting project execution: A case application, Total Qual Manag Bus Excell 29 (2018), 848–865.

25.

Liu

and Stewart

T.J.

, Integrated object-oriented framework for MCDM and DSS modelling, Decis Support Syst (2004).

26.

Liu

and Rodríguez

R.M.

, A fuzzy envelope for hesitant fuzzy linguistic term set and its application to multicriteria decision making, Inf Sci (Ny) 258 (2014), 220–238.

27.

Lootsma

F.A.

, Fuzzy Logic for Planning and Decision Making, Applied Optimization, Springer US, 1997.

28.

Meifort

, Innovation Portfolio Management: A Synthesis and Research Agenda, Creat Innov Manag 25 (2016), 251–269.

29.

Melón

M.G.

, Aragonés Beltran

and Carmen

González Cruz M.

, An AHP-based evaluation procedure for Innovative Educational Projects: A face-to-face vs. computer-mediated case study, Omega 36 (2008), 754–765.

30.

Mikkola

J.H.

, Portfolio management of R & D projects: Implications for innovation management, Technovation 21 (2001), 423–435.

31.

Mitchell

, Phaal

and Athanassopoulou

, Scoring methods for prioritizing and selecting innovation projects, PICMET 2014 - Portl Int Cent Manag Eng Technol Proc Infrastruct Serv Integr (2014), 907–920.

32.

Narayanamoorthy

, Geetha

, Rakkiyappan

and Joo

Y.H.

, Interval-valued intuitionistic hesitant fuzzy entropy based VIKOR method for industrial robots selection, Expert Syst Appl 121 (2019), 28–37.

33.

OECD, OECD Innovation Strategy 2015 An Agenda for Policy Action, Meet OECD Counc Minist Level, Paris, 3-4 June 2015 (2015), 395–423.

34.

Opricovic

, Fuzzy VIKOR with an application to water resources planning, Expert Syst Appl 38 (2011), 12983–12990.

35.

Opricovic

and Tzeng

G.-H.

, Fuzzy Multicriteria Model for Postearthquake Land-Use Planning, Nat Hazards Rev 4 (2003), 59–64.

36.

Opricovic

and Tzeng

G.-H.

, Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS, Eur J Oper Res 156 (2004), 445–455.

37.

Opricovic

and Tzeng

G.-H.

, Extended VIKOR method in comparison with outranking methods, Eur J Oper Res 178 (2007), 514–529.

38.

Opricovic

and Tzeng

G.H.

, Multicriteria planning of post-earthquake sustainable reconstruction, Comput Civ Infrastruct Eng (2002).

39.

Oztaysi

, Onar

S.C.

, Bolturk

and Kahraman

, Hesitant fuzzy analytic hierarchy process, IEEE Int Conf Fuzzy Syst 2015-Novem (2015).

40.

Rathi

, Khanduja

and Sharma

S.K.

, A fuzzy-MADM based approach for prioritising Six Sigma projects in the Indian auto sector, Int J Manag Sci Eng Manag 12 (2017), 133–140.

41.

Rodriguez

R.M.

, Martinez

and Herrera

, Hesitant fuzzy linguistic term sets for decision making, IEEE Trans Fuzzy Syst 20 (2012), 109–119.

42.

Saaty

T.L.

, The Analytic Hierarchy Process, McGraw-Hill Inc, New York, 1980.

43.

Samanlioglu

and Aya

, A fuzzy AHP-PROMETHEE II approach for evaluation of solar power plant location alternatives in Turkey, J Intell Fuzzy Syst 33 (2017), 859–871.

44.

Samanlioglu

, Taskaya

Y.E.

, Gulen

U.C.

and Cokcan

, A Fuzzy AHP–TOPSIS-Based Group Decision-Making Approach to IT Personnel Selection, Int J Fuzzy Syst 20 (2018), 1576–1591.

45.

Spieth

and Lerch

, Augmenting innovation project portfolio management performance: The mediating effect of management perception and satisfaction, R D Manag 44 (2014), 498–515.

46.

Suganthi

, Multi expert and multi criteria evaluation of sectoral investments for sustainable development: An integrated fuzzy AHP, VIKOR / DEA methodology, Sustain Cities Soc 43 (2018), 144–156.

47.

Technopolis Group & Mioir, Evaluation of Innovation Activities: Guidance on methods and practices, Eval Innov Act Guid Methods Pract Study Funded by Eur Comm Dir Reg Policy (2012), 1–64.

48.

Torra

, Hesitant fuzzy sets, Int J Intell Syst 25 (2010), 529–539.

49.

Torra

and Narukawa

, On hesitant fuzzy sets and decision, in: 2009 IEEE Int Conf Fuzzy Syst 2009, pp. 1378–1382.

50.

Tüysüz

and Şimşek

, A hesitant fuzzy linguistic term sets-based AHP approach for analyzing the performance evaluation factors: An application to cargo sector, Complex Intell Syst 3 (2017), 167–175.

51.

Tzeng

G.-H.

, Lin

C.-W.

and Opricovic

, Multi-criteria analysis of alternative-fuel buses for public transportation, Energy Policy 33 (2005), 1373–1383.

52.

Tzeng

G.-H.

, Teng

M.-H.

, Chen

J.-J.

and Opricovic

, Multicriteria selection for a restaurant location in Taipei, Int J Hosp Manag 21 (2002), 171–187.

53.

Umamaheswari

and Kumari

, Fuzzy TOPSIS and Fuzzy VIKOR Methods Using the Triangular Fuzzy Hesitant Sets, Int J Comput Sci Eng Inf Technol Res 4 (2014), 15–24.

54.

Wang

, Song

, Ren

, Li

, Duan

and Wang

, Selecting sustainable energy conversion technologies for agricultural residues: A fuzzy AHP-VIKOR based prioritization from life cycle perspective, Resour Conserv Recycl 142 (2019), 78–87.

55.

Wang

, Xu

and Li

, Research on project selection system of pre-evaluation of engineering design project bidding, Int J Proj Manag 27 (2009), 584–599.

56.

, Ahmad

and Xu

, A group decision making framework based on fuzzy VIKOR approach for machine tool selection with linguistic information, Appl Soft Comput 42 (2016), 314–324.

57.

Yong

, Plant location selection based on fuzzy TOPSIS, Int J Adv Manuf Technol 28 (2005), 839–844.

58.

, Triangular Hesitant Fuzzy Set and Its Application to Teaching Quality Evaluation, J Inf Comput Sci 10 (2013), 1925–1934.

59.

Zadeh

L.A.

, Fuzzy logic, neural networks, and soft computing, Commun ACM 37 (1994), 77–84.

60.

Zimmer

, Yannou

and Stal-Le Cardinal

, Proposal of a radical innovation project selection model based on proofs of value, innovation, and concept, Proc Int Des Conf Des DS 70 (2012), 141–150.

61.

Measuring Innovation, OECD Publishing, 2010.

C1	Experienced labor force existence	C6	Innovativeness
C2	Infrastructure possibilities qualifications	C7	Social influence
C3	Existing technological adequacy	C8	Available use of resources
C4	Feasibility	C9	Commercial success potential, profitability and return on investment
C5	Completion time