Abstract
Evaluation and analysis of failures which occur in the products or services in different economic areas are an important task of operational management. Solution of this problem leads to the increase of product’s/service’s quality, but in the same time it also increases business processes effectiveness and business goal’s realization. The treated problem is especially important in the information technologies domain. In this paper, the risk factors that may cause failures of the software are defined in compliance with the Failure Mode and Effect Analysis (FMEA) framework and they are assessed during the development and the maintenance phase. The relative importance of these risk factors and their values at the level of each identified failure are described by pre-defined linguistic terms which are modelled by the interval type-2 trapezoidal fuzzy numbers (IT2TrFNs). The weights vector is calculated by using the Fuzzy Analytic Hierarchy Process (FAHP) with interval type-2 fuzzy sets. The rank of failures is obtained by using the complex proportional assessment (COPRAS) method. The example with real life data is illustrated to demonstrate the potential and applicability of the adopted methods.
Introduction
The failures that can occur in the product or process can cause catastrophic consequences for companies, such as: decrease of profit, less competitiveness in the market, loss of customers, etc. Therefore, risk assessment problem has become interesting research topic. This problem is very important in the information technology domain, because this sector has a major impact on the development of the economy of each country. Risk assessment problem is considered in many papers [23, 37].
One of the mostly used methods for identifying and eliminating known and/or potential failures of products or processes problems is Failure Mode and Effect Analysis (FMEA) [7]. In the last half of the last century this method was adopted and promoted by Ford Motor and since then, it has become a powerful tool extensively used for safety and reliability analysis of products and processes especially, in automotive industries [33, 34]. It should be noted that this method can be applied for the failure analysis in a wide range of industries, such as in the information technologies sector [36]. In conventional FMEA, each failure can be evaluated by three risk factors (RFs) as severity, occurrence of failure realization and difficulty of detection of the failure. The values of these RFs can be determined with respects to constructed measurement scale, which is defined with a set of natural numbers, at interval from 1 to 10 (with 1 being the best and 10 being the worst case). The rank of failure may be determined according to risk priority number (RPN) which is calculated as product of risk factors.
In the literature, there are many papers in which different extensions of FMEA are proposed, such as those where integrated FMEA and multi-criteria decision making (MCDM) methods [9], which lead to the elimination of conventional FMEA lacks. In some papers an assumption is introduced that the relative importance of RFs and their values for the identified failure can be described in a sufficiently good way with linguistic expressions [29, 32]. The application of fuzzy sets theory [6, 21] allows for the different types of uncertainties to be quantitatively sufficiently reliably and described. In the literature, there are many papers in which imprecise and uncertain data are modelled by type 1 fuzzy sets [22, 40]. Utilizing type-1 fuzzy sets is not suitable in these cases so introduction of the type-2 fuzzy sets occurred [27] with a motivation to provide higher degree of freedom and flexibility. Hence, type-2 fuzzy sets are more accurate in the modelling of uncertainty in comparison to type-1 fuzzy sets. Type-2 fuzzy sets require complex and enormous computational operations so that they do not have wide spread application in modelling of real uncertainties [24]. Many researchers have developed fuzzy decision making approaches using the interval type 2 fuzzy sets within the context of risk, which can be found in the literature [25, 35].
It is believed that decision makers can assess the relative importance more accurately if each pair of RFs is watched separately, than when using a direct estimation method. In this paper, determination of RF weights is based on Fuzzy Analytic Hierarchy Process (FAHP) with the interval type-2 trapezoidal fuzzy numbers (IT2TrFNs) (by analogy [3]).
Ranking of the identified failures by using RNP has many shortcomings [14, 42]. In literature, there is a large number of papers where different improvements of conventional FMEA are made, such as: the values of RFs are modelled by fuzzy sets theory [17, 32], the ranking of identified failures is stated as fuzzy MCDM problem and rank of failures is given by using modified fuzzy MCDM methods [2, 17]. One of MCDM which is easy for understanding and application is complex proportional assessment (COPRAS) [10]. In respect to suggestions of many authors [11, 31], COPRAS is used in this paper.
Motivation for this research comes from the fact that there are no research papers where analysis of failures in the software development phase and the phase of software maintenance (furthermore, only software) is considered in the exact way.
The wider objective of this research may be interpreted as an integration of the FMEA, the IT2TrFNs and MCDM. The mentioned integration includes: a) identification of software failures and causes that can lead to failure occurrence, b) determination of linguistic expressions which in the exact way describe the relative importance of RFs and their values, c) modelling of pre-defined linguistic terms by using the fuzzy sets theory [24], d) determination of weights of the risk RFs by using AHP with IT2FNs [3] e) definition of management initiatives which should lead to the improvement of software efficiency; the order of taking initiatives is based on the obtained rank of the identified failures. It can be said that by using the proposed model, all marked deficiency of conventional FMEA can be substantially eliminated.
The paper is organized in the following way: in Section 2 there is a literature review of FMEA, FMEA integrated with fuzzy sets theory and the model which integrates FMEA, fuzzy set theory and MCDM is presented in detail. In Section 3, the modelling of the relative importance of RFs and their values at the level of each identified failure of software is presented. The proposed model is presented in Section 4. Case study is presented in Section 5 and Conclusion is given in Section 6.
Literature review
Risk assessment can be defined as the systematic use of information to identify failures and to estimate the risk that can occur due to the realization of the identified failures. Determination of risk priority by using procedure which is suggested in conventional FMEA [3] has many lacks: (1) defined measurement scale has some non-intuitive statistical properties [28], (2) it is known that it is very difficult for decision makers to give good enough estimates by means of numerical measurement scale, so it can be concluded that usage of proposed measurement scale is not quite appropriate [32], (3) the relative importance among the RFs is not considered as they are accepted to have an equally importance [5, 20], (4) there is no clear explanation of why RNP is calculated as product of three considered RFs, therefore the proposed mathematical formula is questionable and debatable [14, 42], (5) risk implication due to occurrence of failure to whom the same RNP value is associated, can be quite different [3, 16].
The short retrospective of papers where different approaches for improvement of conventional FMEA are suggested is further shown.
The values of RFs are described by linguistic expressions which are modelled by gaussmf function which is defined in MATLAB in [26], triangular fuzzy numbers (TFNs) in [17, 32], with trapezoidal fuzzy numbers (TrFNs) in [34], interval-valued intuitionistic fuzzy sets [34], hesitant fuzzy sets in [35]. Fuzzy numbers domains are defined by different measurement scales, [0–10] in [32, 34], [1–10] in [26], [0-1] in [13, 15].
Comparative analysis of modelling of the RFs values with other papers from the relevant literature is presented in Table 1.
Summarized comparison analysis for the risk factor values
Summarized comparison analysis for the risk factor values
Based on the analysis of papers which can be found in relevant literature it can be clearly concluded that the determination of relative importance of RFs can be considered as a separate problem. Liu et al. [17] suggest that the determination of the weights of RFs should be stated as fuzzy group decision making problem. Each decision maker uses one of the seven pre-defined linguistic terms for description of RF weights. These linguistic variables are modelled by TFNs whose domains are defined at interval [0-1]. Aggregation of individual assessments of decision makers into group consensus is performed according to procedure which is proposed in [17]. The fuzzy pair-wise comparison matrix of the relative importance of risk factors is stated in [2]. The elements of these matrices are described by nine linguistic variables which are modelled by TFNs. The weights of these RFs are determined by extent analysis which is developed in [8]. The domain of these fuzzy numbers defined in [2] is defined on the real line at interval [1/3-3]. In [5] the criteria weights are determined by using the Shannon’s entropy concept that is used in [1]. In this paper, the RF weights are determined by using the fuzzy AHP with IT2TrFNs, analogy [3, 28].
There are many papers in which authors suggested different methods for failure rank [2, 17]. These methods are summarized in Table 2. Liu et al. [17] considered the problem of risk assessment under fuzzy environment. The weighted fuzzy decision matrix was constructed and transformed into decision matrix by applying the centroid defuzzification method [6]. Rank of failures was given by using Multi-criteria Optimization and Compromise Solution (VIKOR) method [39]. Kutlu and Ekmekçioğlu [2] used fuzzy Technique for Order of Preference by Similarity to Ideal Solution (FTOPSIS) which was proposed in [4] for ranking of identified failures. Adhikary et al. [5] considered problem of failure analysis for coal-fired thermal power plants. These authors, besides RFs which are considered in conventional FMEA, introduced other RFs too, such as environmental, economic, downtime reduction, and human risk factors. The rank of failure was obtained by using the conventional COPRAS [10].
Summarized comparative analysis for risk factor weights and failures ranking
Comparative analysis of modeling of the relative importance of RFs, determination of RF weights and MCDM method that is used for software failures ranking with other papers from the relevant literature is presented in Table 2.
In this paper, modelling and handling of existing uncertainties is based on IT2TrFNs which presents a special case of type-2 fuzzy sets and its operations [24]. Complex and enormous computational onerous operations are associated with type 2 fuzzy sets, so that there are not to many applications in modelling of real uncertainties. The computational effort with the interval type-2 fuzzy sets is reduced and their use is increased to solve different decision-making problems [38].
Granularity is defined as the number of fuzzy sets assigned to the existing uncertainties. According to [12] human being can have only seven categories at the most. Therefore, in this paper, authors have defined the linguistic expressions with respects to presented recommendation.
In order to understand the problem, the review of some background of IT2TrFNs is given in Section 3.1.
Preliminaries
In this Section, some basic definitions related to fuzzy algebra rules of the IT2TrFNs are presented [15].
Definition 1. A type 2 fuzzy set,
Definition 2. Let
The upper membership function (
Definition 3. If X is a set of real numbers, then a type-2 fuzzy set and an interval type-2 fuzzy set in X are called a type-2 fuzzy number and an interval type-2 fuzzy number, respectively.
Definition 4. If the upper membership function and lower membership function of
The lower and upper bound in the domain are denoted as
Definition 5. Let us two IT2TrFNs,
The the arithmetic operations are introduced by [3]:
The addition operation, which is denoted as,
The subtraction operation, which is denoted as,
The multiplication operation, which is denoted as,
The division operation, which is denoted as,
Definition 6. Let us trapezoidal interval type-2 fuzzy number,
Definition 7. The defuzzificated Trapezoidal type-2 fuzzy numbers approach (DTraT) is proposed [18]:
All the risk factors, according to which possible software’s failures are assessed, can lead to the reduction of data flow and speed and they are not commonly of the same relative importance. Many authors consider that it best suits human-decision nature to assess the relative importance of each pair of criteria by using linguistic expressions [3, 22].
The fuzzy pair-wise comparison matrix of the relative importance of considered RFs is stated. The elements of this fuzzy matrix are defined as the relative importance of RF k over RF:
These linguistic terms are modelled by IT2TrFNs,
If strong relative importance of evaluation RF, k′ over evaluation RF k holds, then pair-wise comparison scale can be represented by IT2TrFN:
If RF k and k′ (k, k′ = 1, . . . , K) have equal relative importance then the values of the fuzzy pair-wise comparison matrix is represented by single point 1 which is IT2TrFN:
In this paper, the relative importance of each pair of treated RFs are described by one of three linguistic terms which are given in the following way:
low importance (L):
high importance (H):
The domain values of these IT2TrFNs are defined at interval [1–5]. Value 1 denotes that relative importance of the RF k over RF k′ is equal. Value 5 denotes that risk factor k is extremely important over the RF, k′ ; k, k′ = 1, . . . , K
Modelling of the uncertain RF values
As it is known, RF values in the automotive industry should be assessed by using pre-defined measurement scales (standard IATF16949). In others economic domains, the RF values are assessed by decision makers which use the different measurement scales or linguistic expressions. As it is mentioned, it is closer to the human decision making process that estimates are expressed by words rather than by precise numbers. Their estimates are based primarily on the knowledge about the considered problem, experience as well as evidence data. It is proposed that decision makers use pre-defined linguistic expressions which are modelled by IT2TrFNs (Tables 3 and 4).
The severity of consequences and detection of failures during the process realization
The severity of consequences and detection of failures during the process realization
In respect to the considered problem, it is known that there are not enough accurate evidence data according to which the possibility of failure occurrence can be determined. It is assumed that decision makers can use one of five pre-defined linguistic expressions for describing these software failures. These linguistic expressions are determined by decision makers and based on results of analyzing the causes that lead to their realization. In this paper, the five linguistic expressions are defined and presented in Table 4.
The occurrence of failures
The domain values of IT2TrFNs are defined by common measurement scale [0, 1]. Value 0 denotes that severity of consequence on data flow is almost negligible, i.e. the failure is absolutely impossible to detect, i.e. the failure is not realized during the time period, respectively. Value 1 denotes that the severity of consequence on data flow is extremely large, i.e. it is absolutely possible to detect the failure, i.e. the failure occurs at each startup of the software, respectively.
The software failure analysis is considered at the level of primitive processes from the hierarchical tree of the process that is created by using structured systems analysis and design method. The possible failures can be formally presented as i = {1, . . . , i, . . . , I}. The index for failure is denoted as i = 1, . . . , I, and the total number of failures at the level of considered primitive processes is denoted as I.
In general, the analysis of the identified failures can be performed with respects to many RFs (by analogy [29]), which can be presented by set of indices γ = {1, . . . , k, . . . , K}. The index for a RF is denoted as a k, and K is the total number of considered RFs. In this paper, determination of RFs is based on FMEA framework [7].
It can be said that, the aspects of the consideration and evaluation of consequence severities depend on the type of the problem. In this paper, the severity of consequence which occurs due to the realization of each identified failure is considered from the aspect of the data flow. The possibility of occurrence of these errors is determined based on estimations of decision makers. It can be said that the decision makers have almost the same experience and knowledge in development and monitoring software domain. They base their assessments on the available data from database as well as analysis of the causes that lead to the occurrence of these failures. Failure can be detected: (1) by using characteristic of considered software, or the existence of software controls for automatic checking or (2) by the users themselves, in situations where the user is active in the data processing, as it was done in this paper. In this paper, the criteria values for each identified failure are described by one of the pre-defined linguistic expressions which are modelled by IT2TrFNs. Determining the membership function is subject to decision maker’s subjectivity. Therefore, using IT2TrFNs in a better way describes the imprecise estimations of decision makers.
The relative importance of considered RFs is stated by the fuzzy pair-wise comparison matrix. The elements of this fuzzy matrix are modelled by IT2TrFNs. It can be considered that by using defuzzification procedure [18], as is applied in this paper, are obtained more accuracy the representative scalar values of IT2TrFNs. In this way, the consistency check is based on accuracy data. The handling of uncertainties is performed by using the FAHP with IT2TrFNs which is presented in [3].
The rank of identified software’s failures is determined by applying conventional COPRAS [10]. The priorities of management actions which should lead to elimination of identified failures, and therefore to improving of application of considered software, are determined according to the obtained rank.
The proposed algorithm
Rank of the identified failures is given by using the procedure which is presented by the following the Algorithm.
Step 1. The fuzzy pair-wise comparison matrix of the relative importance of risk factors is constructed:
Step 2. Mapping the fuzzy pair-wise comparison matrix,
Defuzzification is performed by using the procedure which is proposed in [3]:
Checking of assessment’s consistency of decision maker by using the eigenvector [41]. If the consistency index is less than or equal to 0.1 it can be considered that the mistakes made by the decision makers do not have impact on accuracy of the assessment.
Step 3. Calculating the weights vector,
The lower and upper bounds and modal values of lower trapezoidal membership function of IT2TrFNs
Step 4. The fuzzy rating of the uncertain risk factors,
Step 5. The weighted fuzzy decision matrix,
Step 6. The representative scalars of IT2TrFN
Step 7. The aggregated values of benefit type risk factor P
i
and cost type risk factor Q
i
are determined according to the procedure which is developed in conventional COPRAS [10]:
Step 8. Calculate the overall aggregated values of each failure i, i = 1, . . . , I so that:
Where
Step 9. Determine degree of utility for each failure i, i = 1, . . . , I so that:
Where:
Step 10. Rank of the identified failures at the level of each process is determined according to the obtained values of ζ i which are sorted into increasing order. The first place in the rank takes the failure to which the lowest degree of utility value is joined.
Step 11. According to the obtained rank, the priority of measures is defined.
At the Faculty of Mechanical Engineering of the University of Belgrade, Republic of Serbia, the software for supporting the teaching process was developed and implemented in order to increase its efficiency. The essential activities of this process are attending lectures on those courses that are in the book of the subjects, taking an exam, realization of all planned obligations as well as graduation activities. During the maintenance phase, the Software Development Team - SDT (software developer and software maintenance engineer) identifies software failures, that have been realized and as well as potential software failures. It is common practice that SDT based on experience determines which failures most affect the software’s efficiency. A priority action that needs to bring to the increase the efficiency of software needs to be based on the rank of failures. First of all, the software risk analysis (identification, evaluation and ranking of failures) was done in the usual way by SDT. After the implementation of the management actions, a software re-testing was performed. The obtained results showed that software efficiency did not significantly improve. Therefore, it can be concluded that the actions taken were not sufficiently adequate, but further, among other things, indicates that the ranking of errors was very burdened by the subjective attitudes of SDT.
An application
In order to improve software risk analysis, which is further propagated to increase software efficiency, this paper proposes the model that is presented in this paper. The proposed model is tested on the real life data which come from the considered process.
The fuzzy rating of the relative importance of RFs as well as their values are performed by SDT. Their assessments are based on the data from the records and experience by panel discussion.
By applying the procedure of the proposed Algorithm (Step 1 to Step 2), the fuzzy pair-wise comparison matrix of the relative importance of RFs, as well as the consistency index is given:
The weights vector is given according to [18] (Step 3 of the proposed Algorithm). The proposed procedure is illustrated in the following example:
In a similar way it is counted:
Let us:
The RF weights are calculated according to the procedure which is developed in [3]. This procedure is illustrated by the following example:
The weights of the other considered risk factors are obtained in the same way:
Estimation of treated RFs for identified failures (Step 4 of the proposed Algorithm) are shown in Table 3 (see Appendix).
By using the procedure (Step 5 to Step 6 of the proposed Algorithm), the fuzzy decision matrix and crisp decision matrix are constructed. The proposed procedure is illustrated by example:
The defuzzification procedure which is proposed in [3] and applied in this paper is illustrated by the following example:
The representative scalar of the IT2TrFN,
The decision matrix
The decision matrix
The proposed Algorithm (Step 7 to Step 8), the aggregated value for benefit-type RFs, P
i
and cost-type RFs, Q
i
as well as overall aggregated value, Z
i
are calculated and associated to each identified failure i, i = 1, . . . , 41. This procedure is illustrated by the following example:
The obtained results are presented in Table 6.
The aggregated values and the overall aggregated values
The degree of utility for each failure i, i = 1, . . . , 41 and the rank of failures are calculated according to proposed procedure (Step 9) and illustrated by example:
The degree of utility of all failures and their rank (Step 10 of the proposed Algorithm) is presented in Table 7.
The degree of utility and rank of failures
Reduction or elimination of identified failure’s impact to functioning of incremental software model can be achieved by undertaking appropriate management initiatives. In this paper, sequence of application of management initiatives is based on the obtained rank of identified failures..
Based on the obtained results, it can be concluded that the failure which has the greatest impact on the considered process realization, is a failure that is marked as an incomplete set of students’ requirements for selecting subjects (i = 12). Due to the realization of this failure, students’ requests will be partially processed or completely unprocessed. This consequence can lead to the inability to complete process realization which is denoted as collection and processing of students’ requests. Output of this process is invalid and incomplete list of subjects and students for the current school year. Irregularities created in this way are remedied by corrective processes and with alternative procedures which are systemically foreseen. The causes that may lead to the occurrence of the considered failure are: (1) insufficiently informative user interface for the implementation of the collecting of student requests procedure and (2) the inability of a complete description of the request that exists in the analysis phase. The actions that can be taken to prevent the occurrence of this failure are: (1) redefining the request, (2) modification of the design of the corresponding part of the system, (3) coding of a new user interface mask so that it contains and displays the necessary information, as well as to implement the necessary restrictions. At the second place in the rank is a failure unresolved desires for student’s subjects (i = 11). Materialization of this failures leads to the fact that all subjects are not assigned to within the envisaged number of ECTS credits, so that students can not listen to and taking exams, and at the end they cannot finish the school year. The cause that can lead to occurrence of this failure is insufficiently defined requirement by faculty management and other stakeholders. The treated failure can be removed if a change of procedure is made and in this way the space of incompatible states has been reduced.
Of these two analyzed failures which have the largest impact on functioning of considered software, three failures (i = 11, and i = 12) are materialized in a process that is denoted as Collection and analysis of students’ desires, software developer should pay special attention to a part of the information system which refers to this process. This further means that it is necessary to check all phases of the process development.
In order to validate the proposed model, the software is tested again after implemented predefined management actions. The test results have shown that failures (i = 11, and i = 12) have a significantly lower impact on the functionality of the developed software. Based on the obtained results, it is clear that the proposed model is an adequate software risk analysis tool. Using a developed model, SDT is able to spend less resource (money, time...) while at the same time increasing the efficiency of the software.
Conclusion
Digitalization of the work process in organizations is one of the goals of National strategy of development and has a significant influence on the effectiveness, competitiveness and long-term sustainability of these organizations. One way of achieving this goal is failure assessment and taking actions in order to eliminate them in processes of software development and maintenance that present support to many processes in any organization.
The mostly used method for analyses failures in any organization is FMEA. In respect to suggestions by many authors that conventional FMEA has many lacks, in this paper a model is proposed by which the rank of identified failures can be determined in an exact way as well as the priority of management initiatives.
The proposed model was tested on real-life data from Faculty of Mechanical Engineering, University of Belgrade. In this paper, identification of failures of software in education sector is based on user remarks as well as on the basis of data analysis results. Determination of the causes that can lead to the materialization of each failure is defined by development management team based on their knowledge and experience.
The main advantages of the presented model for information technology sectors, which at the same time represent its main contributions, are:
Adequate linguistic expressions by which the relative importance are described and values of RFs for IT sector are invented.
The modelling of existing uncertainties is performed by IT2TrFNs.
The relative importance of defined RFs is determined by AHP with IT2TrFNs.
The considered problem may be described by formal language that enables determination of the software failures rank in an exact manner.
Sequence of application of management activities that should lead to elimination of identified failures is determined according to the obtained rank. In this way, significantly fewer resources are consumed during software improvement process.
The proposed method is flexible to the changes in: (1) the numbers of failures, (2) the relative importance and values of RFs, (3) the membership functions shape of interval type 2 fuzzy numbers can be easily incorporated into the proposed model, and (4) can be easily extended to the analysis of other management decision problems in different areas of research.
Some limitations of the proposed model are: (1) subjectivity in the assessment of input data based on which the rank of failures is obtained, (2) requires greater complexity of computation.
Future research should include: (1) the analysis of software failures used in different service organizations, (2) development of the user friendly software tool that corresponds to the proposed model.
