Abstract
With this work, we developed a multi-criteria decision-making model to assess and select an Enterprise Resource Planning (ERP), using a multi-criteria decision analysis (MCDA). A hybrid multi-criteria methodology is used for the assessment and selection of an ERP, combining the MMASSI/IT methodology, which is used to both define the relevant family of criteria, based on their features and flexibility to change and adapt to a given scope, and the weight of criteria. Then, the well-known Analytic Hierarchy Process (AHP) methodology is used to perform the decision-makers’ value function elicitation preference of alternatives pairwise comparison in each criterion. The additive aggregation is used to compute the alternatives global score. The proposed hybrid model was validated in an industrial context by three Decision-makers.
Introduction
The selection of an Enterprise Resource Planning (ERP) System is a complex problem due to the number of existing market solutions, the need of customisation, and the need to fit the business’ specifications and requirements. New paradigms, as industry 4.0, require the integration of Information Systems (IS), Information Technologies (IT), and interoperability of the IS, as well as the equipment’s integration with the IS. All ERP commercial solutions need to be customised to the company’s needs. The latter is the main issue related to the system’s impact on the business performance and global cost of the solution adopted by a company. This is a time-consuming task involving the IT supplier resources and the company which must have an active role in creating the necessary conditions to adjust its ERP solution, allowing the business to become much stronger and more competitive. IS have the power to change the way businesses work, preparing organisations to better operate in a competitive market.
This paper presents a novel approach which involves hybridisation of different methodologies, MMASSI and AHP, to assess and develop an IS/IT selection model.
Theoretical framework and prior research
MCDA is a problem-solving methodology that organises and synthesises the information regarding a given decision problem in a way that provides the decision makers (DM) with a coherent overall view of the problem [1]. MCDA methods assist DMs in the process of identifying the most preferred action(s), from a set of possible alternative actions, when there are multiple, complex, incommensurable and often conflicting objectives, measured in terms of different evaluation criteria. The alternative actions distinguish themselves by the extent to which they achieve the objectives [2]. The implementation of an MCDA methodology is a nonlinear recursive process comprised of several stages: the first is to reach a common understanding of the decision context and the identification of the decision problem. The second and third comprises the identification of decision alternatives, as well as of the decision criteria that are relevant to assess them. The fourth is the assignment of relative importance weights to the criteria and the fifth is the elicitation of DMs’ preference, or assessment, for each alternative in each criterion that can be done using different methods [3].
There are several different MCDA methods, all supported by decision support systems (DSS). The most worldwide known are the AHP [4] from the American school, the PROMETHEE [5] and ELETRE families [6] from the European schools. Different in methodological nature, all present advantages and disadvantages to the DM process [7].
Model, methodology and data
MMASSI/TI-multi-criteria methodology to assess and select an information system
The MMASSI/TI is a group decision support system (GDSS) that aims to support decision making in the selection of IS/IT towards alternatives in complex cases with conflicting goals [7, 8], which differs from other MCDA decision support systems because of the consistent and complete set of features/attributes that characterise a set of predefined alternatives. Despite this methodological feature, it is a flexible GDSS inasmuch as it allows making changes according to context by adjusting the set of criteria to a coherent and consistent family of criteria. The number of criteria and sub-criteria is not limited, despite being already defined and despite the suggestion for a comprehensive and coherent way to operationalise them according to a given context. It allows the not selection, modification and addition of new criteria and their operationalisation [8]. The MMASSI/TI is a multi-criteria methodology to support the selection of alternatives and choices that has been designed to be easy to understand and use without the specific support of an analyst, to offer the GDM an effective support decision-making tool and to act as an enhancer of the specification accuracy. This methodology purposefully intends to be simple so that the GDM can be led through the ten steps that it is divided into [8, 9].
AHP-analytic hierarchy process
The Analytic Hierarchy Process (AHP), developed by Saaty [1], provides a comprehensive and rational framework for structuring a decision problem by allowing the representation and quantification of its elements. It is addressed to tackle complex and ill-structured problems for which a discrete set of possible solutions has been identified. The solutions are analysed and ranked according to their value, regarding a set of relevant characteristics previously identified [3].
The main reasons behind the wide applicability and acceptability of AHP are that: i) it is easy to understand – it is simple and its intuitive nature allows the decision-makers (experts and stakeholders) to understand how the recommendations are obtained; ii) it allows active participation – the decision makers are involved in every step of the decision analysis; iii) its provides hierarchy – helps in identifying the importance of the factors involved in the problem; iv) it addresses subjectivity – its ability to deal with subjective assessments (feelings and judgments), which are then converted into numerical values that reflect the DMs’ opinion and preferences; v) it is a pairwise comparison method – it is much easier to perform and it helps articulate the relative importance of criteria and quantify the relative contributions of the alternatives to the criteria; vi) it measures inconsistencies – it helps avoid inconsistent judgments as, by identifying them, the decision makers can repeatedly work through their inconsistent judgments until they obtain acceptable results. In addition, it leads to better communication, clearer understanding, and easier consensus in GDM settings, and thus to greater commitment to the chosen alternative; vii) there are group decision support systems (GDSS) to support AHP [7, 10]. Nevertheless, some drawbacks are pointed out [10]: i) rank reversal – the ranking of the alternatives may be altered by the addition of another alternative; ii) compensating effect – it admits trade-offs among the various elements of the model; and iii) time consuming – to perform the comparisons. The rank reversal is not an issue in the present application since we have considered all possible ERPs suppliers that are up to company’s required standards. The compensatory nature is only an issue if the problem under analysis does not allow for trade-offs, which clearly is not the case for the ERP selection problem. Finally, the time required to perform pairwise comparisons may indeed become very long if many alternatives and/or criteria exist.
Hybrid MMASSI/TI-AHP methodology
As said, the MCDA comprises several steps. Although the practice has been to choose a single method to carry out all the steps, it is also possible to harness the strengths of the different methods and use each of them to perform the steps they are best at. By doing so, a more powerful approach is expected [10].
To define the MCDA context, the criteria and their weight, we used the MMASSI/TI methodology as it presents a predefined set of criteria, and its operationalisation covers the IS scope, expediting the analysis of the context of application of the MCDA model. For DMs’ preference elicitation, aggregation values, ranking and sensitivity analysis we used the AHP methodology.
For alternative pairwise comparison on each criterion matrix, the DMs are asked to perform pairwise comparisons for the chosen criteria, using the nine-point intensity scale proposed by Saaty [1], where 1-equal importance; 3-moderate importance; 5-strong importance; 7-very strong importance; 9-extreme importance. Intensities of 2, 4, 6, and 8 are used for intermediate values [10].
For GDM, a geometric mean is used to compute the consensus judgments for
These judgments result in a square matrix that the AHP converts into weights for each alternative of the assessed criterion, being the consensus values checked. Judgments are checked for consistency, and if consistency is found to be unacceptable, i.e. higher than 10% for consistency ratio, Eq. (2), the pairwise comparison is repeated. If any of the criteria is divided into sub-criteria, then they need to be pairwise compared in the same manner the criteria have been. Let
This matrix is then normalized by dividing each element
with
Random index
Criteria and sub-criteria for validated decisional context
A combined MMASSI/TI-AHP methodology applied to the same DM problem addressed in [9] was used. MMASSI/TI for the two first steps of an MCDA. The AHP method was explain to the DM.
Based on previous research, the MMASSI/TI pre-defined criteria, a coherent family of relevant criteria comprised of eleven criteria was defined only from the second-phase, see Table 2.
Criteria operationalisation was validated by the decision makers so that the three DM had the same understanding of the criterion and could evaluate it in the same way. This is crucial for the alternatives’ elicitation values in each criterion.
Then, the criteria weighted values were computed using the same MMASSI/TI procedure [9].
For the rest of the MCDA phases, the AHP method was used for alternatives assessment in each criterion. For group DM a geometric mean was used to compute each DM preferences.
Table 3 presents the criteria weight computation using the MMASSI/TI procedure.
MMASSITI’s Results of aggregation model
MMASSITI’s Results of aggregation model
To perform the remaining steps, the AHP group decision method was used. The assessment was done by three DMs,
As an example, for the criterion A3-cost of ERP, the APH process is presented. First, the calculation of geometric mean values is presented, see Table 4, computed by Eq. (1), from DM1, DM2 and DM3 elicited values for the alternatives, using the semantic Saaty scale.
AHP group elicitation process
Table 5 presents the group DM values, which is the geometric mean computed in Table 4.
Dm’s preferences elicitation process
The sum of columns computed in Table 5 is used to normalise the group DM values, by dividing each value by the sum of the respective column, which are showed in Table 6, as well as the normalised eigenvector.
AHP normalisation matrix process
With the values computed in Table 6, we needed to compute the CR using Eq. (2). The CR is less than 10%, presented in Table 7, which gives confidence in the previous AHP elicitation process.
AHP Consistency Ratio computation
This process, showed in Tables 4–7, was repeated for the rest of the criteria and the results are presented in Table 7.
The DMs’ elicitation values were performed using an AHP DSS, based on the nine-point fundamental scale by Saaty [1]. Table 8 summarizes the results obtained.
Results of aggregation model
Results of aggregation model
The results ranked Primavera ERP with best score, 0.34, followed by PHC ERP with 0.25, and SAP ERP with 0.22 and NAV ERP with 0.18. Next, sensitivity analysis was performed; the same order was maintained, except for PHC and SAP presenting similar values. In [9] the MAASSI/TI methodology was used to support the four MCDA phases. In this case, as the MAASSI/TI methodology was used in the 1st two phases and group AHP methodology in the last to phases, i.e. value function elicitation and value aggregation, the normalization procedure of the elicited values is different in nature. The last three alternatives were very close in the analysis performed with both methods as, on the one hand, the alternatives present similar features as they are expected to already meet the requirements, and, on the other hand, both methods have a compensating effect as they allow the use of a large number of criteria.
The sensitivity analysis performed showed no change in the ranking of the alternatives. Thus, it can be stated that PRIMAVERA is the software that best fits the requirements set by decision makers.
The selection of an ERP is a complex problem because of the need to analyse how the ERP requirements meet the company’s needs, address any technical issues, and the ERP impact on the company’s business performance. New industry paradigms such as Industry 4.0 demand for integrated solutions, hence the increasing investment in this area. Because of increased competition among peers in the present economic climate, businesses need IT support to develop automated systems that reduce waste and thereby increase profit margins to foster their sustainable economic growth.
A hybrid methodology combining MMASI/TI and AHP was used to analyse several ERP available on the market. Using MMASI/TI’s first and second steps, the alternatives and the relevant criteria were defined and weighted. It was possible reduce the number of ERP to four, which met the company’s requirements, with the analysis of each alternative, based on the suppliers’ demonstrations and portfolio, essential information regarding the features of each ERP, such as functionality, compatibility, limitations, technical support and other information. To perform the remaining MCDA steps, we used AHP supported by a GDSS. The results ranked Primavera ERP with best score, i.e. 0.34. Sensitivity analysis did not change the ranking order, although PHC and SAP presented a small difference of values. Comparing with the results obtained in [9] the same ranking was obtained.
Footnotes
Acknowledgments
We thank the anonymous reviewers whose comments/suggestions helped improve and clarify this paper. Fernanda A. Ferreira acknowledges the financial support of UNIAG, R&D unit funded by the FCT – Portuguese Foundation for the Development of Science and Technology, Ministry of Science, Technology and Higher Education, under the Project UID/GES/04752/2019. Teresa Pereira acknowledges the financial support of CIDEM, R&D unit funded by the FCT – Portuguese Foundation for the Development of Science and Technology, Ministry of Science, Technology and Higher Education, under the Project UID/EMS/0615/2019.
