An ERP selection using a hybrid methodology

3.1 MMASSI/TI-multi-criteria methodology to assess and select an information system

The MMASSI/TI is a group decision support system (GDSS) that aims to support decision making in the selection of IS/IT towards alternatives in complex cases with conflicting goals [7, 8], which differs from other MCDA decision support systems because of the consistent and complete set of features/attributes that characterise a set of predefined alternatives. Despite this methodological feature, it is a flexible GDSS inasmuch as it allows making changes according to context by adjusting the set of criteria to a coherent and consistent family of criteria. The number of criteria and sub-criteria is not limited, despite being already defined and despite the suggestion for a comprehensive and coherent way to operationalise them according to a given context. It allows the not selection, modification and addition of new criteria and their operationalisation [8]. The MMASSI/TI is a multi-criteria methodology to support the selection of alternatives and choices that has been designed to be easy to understand and use without the specific support of an analyst, to offer the GDM an effective support decision-making tool and to act as an enhancer of the specification accuracy. This methodology purposefully intends to be simple so that the GDM can be led through the ten steps that it is divided into [8, 9].

3.2 AHP-analytic hierarchy process

The Analytic Hierarchy Process (AHP), developed by Saaty [1], provides a comprehensive and rational framework for structuring a decision problem by allowing the representation and quantification of its elements. It is addressed to tackle complex and ill-structured problems for which a discrete set of possible solutions has been identified. The solutions are analysed and ranked according to their value, regarding a set of relevant characteristics previously identified [3].

The main reasons behind the wide applicability and acceptability of AHP are that: i) it is easy to understand – it is simple and its intuitive nature allows the decision-makers (experts and stakeholders) to understand how the recommendations are obtained; ii) it allows active participation – the decision makers are involved in every step of the decision analysis; iii) its provides hierarchy – helps in identifying the importance of the factors involved in the problem; iv) it addresses subjectivity – its ability to deal with subjective assessments (feelings and judgments), which are then converted into numerical values that reflect the DMs’ opinion and preferences; v) it is a pairwise comparison method – it is much easier to perform and it helps articulate the relative importance of criteria and quantify the relative contributions of the alternatives to the criteria; vi) it measures inconsistencies – it helps avoid inconsistent judgments as, by identifying them, the decision makers can repeatedly work through their inconsistent judgments until they obtain acceptable results. In addition, it leads to better communication, clearer understanding, and easier consensus in GDM settings, and thus to greater commitment to the chosen alternative; vii) there are group decision support systems (GDSS) to support AHP [7, 10]. Nevertheless, some drawbacks are pointed out [10]: i) rank reversal – the ranking of the alternatives may be altered by the addition of another alternative; ii) compensating effect – it admits trade-offs among the various elements of the model; and iii) time consuming – to perform the comparisons. The rank reversal is not an issue in the present application since we have considered all possible ERPs suppliers that are up to company’s required standards. The compensatory nature is only an issue if the problem under analysis does not allow for trade-offs, which clearly is not the case for the ERP selection problem. Finally, the time required to perform pairwise comparisons may indeed become very long if many alternatives and/or criteria exist.

3.3 Hybrid MMASSI/TI-AHP methodology

As said, the MCDA comprises several steps. Although the practice has been to choose a single method to carry out all the steps, it is also possible to harness the strengths of the different methods and use each of them to perform the steps they are best at. By doing so, a more powerful approach is expected [10].

To define the MCDA context, the criteria and their weight, we used the MMASSI/TI methodology as it presents a predefined set of criteria, and its operationalisation covers the IS scope, expediting the analysis of the context of application of the MCDA model. For DMs’ preference elicitation, aggregation values, ranking and sensitivity analysis we used the AHP methodology.

For alternative pairwise comparison on each criterion matrix, the DMs are asked to perform pairwise comparisons for the chosen criteria, using the nine-point intensity scale proposed by Saaty [1], where 1-equal importance; 3-moderate importance; 5-strong importance; 7-very strong importance; 9-extreme importance. Intensities of 2, 4, 6, and 8 are used for intermediate values [10].

For GDM, a geometric mean is used to compute the consensus judgments for k = number of participants, according Eq. (1) [4]:

b i ⁢ j = ( a 1 ⁢ i ⁢ j ⋅ a 2 ⁢ i ⁢ j ⋅ … ⋅ a k ⁢ i ⁢ j ) 1 k .(1)

These judgments result in a square matrix that the AHP converts into weights for each alternative of the assessed criterion, being the consensus values checked. Judgments are checked for consistency, and if consistency is found to be unacceptable, i.e. higher than 10% for consistency ratio, Eq. (2), the pairwise comparison is repeated. If any of the criteria is divided into sub-criteria, then they need to be pairwise compared in the same manner the criteria have been. Let C = [ C j | j = 1 , 2 , … , n ] be the set of criteria and A = [ a i ⁢ j ] the pairwise comparison matrix:

This matrix is then normalized by dividing each element a i ⁢ j by the sum of the column in which it appears. The consistency is then checked by finding if the consistency ratio (CR) is under 10%:

𝐶𝑅 = 𝐶𝐼 𝑅𝐼 ,(2)

with 𝐶𝐼 = λ 𝑚𝑎𝑥 - n n - 1 , where RI is the Random Index, see Table 1, obtained from a randomly generated pairwise comparison matrix of size n × n , and λ 𝑚𝑎𝑥 denotes the maximum eigenvalue of the pairwise comparison matrix [1].

A combined MMASSI/TI-AHP methodology applied to the same DM problem addressed in [9] was used. MMASSI/TI for the two first steps of an MCDA. The AHP method was explain to the DM.

Based on previous research, the MMASSI/TI pre-defined criteria, a coherent family of relevant criteria comprised of eleven criteria was defined only from the second-phase, see Table 2.

Criteria operationalisation was validated by the decision makers so that the three DM had the same understanding of the criterion and could evaluate it in the same way. This is crucial for the alternatives’ elicitation values in each criterion.

Then, the criteria weighted values were computed using the same MMASSI/TI procedure [9].

For the rest of the MCDA phases, the AHP method was used for alternatives assessment in each criterion. For group DM a geometric mean was used to compute each DM preferences.

Table 3 presents the criteria weight computation using the MMASSI/TI procedure.

To perform the remaining steps, the AHP group decision method was used. The assessment was done by three DMs, k = 3 . In each iteration the consistency ratio was validated.

As an example, for the criterion A3-cost of ERP, the APH process is presented. First, the calculation of geometric mean values is presented, see Table 4, computed by Eq. (1), from DM1, DM2 and DM3 elicited values for the alternatives, using the semantic Saaty scale.

Table 5 presents the group DM values, which is the geometric mean computed in Table 4.

The sum of columns computed in Table 5 is used to normalise the group DM values, by dividing each value by the sum of the respective column, which are showed in Table 6, as well as the normalised eigenvector.

With the values computed in Table 6, we needed to compute the CR using Eq. (2). The CR is less than 10%, presented in Table 7, which gives confidence in the previous AHP elicitation process.

This process, showed in Tables 4–7, was repeated for the rest of the criteria and the results are presented in Table 7.