A fuzzy rule induction algorithm for discovering classification rules

Abstract

Recently, the topic of data mining has attracted considerable attention from both academia and industry. Data mining is the process of extracting useful knowledge from large amounts of data. Among the types of knowledge to be mined, classification knowledge is the most widely exploited in engineering applications. A variety of methods exist for inductive learning of crisp classification knowledge. This paper presents a new inductive learning algorithm called FuzzyRULES that extracts fuzzy classification rules from a database of examples. The use of fuzzy sets and fuzzy logic methods not only provides a powerful, flexible approach to handling vagueness and uncertainty, but also increases the expressive power and comprehensibility of the induced classification knowledge. An example involving the induction of process planning rules is used to illustrate the operation of FuzzyRULES. The algorithm has also been compared against conventional crisp and fuzzy rule induction algorithms on several benchmark data sets. The results obtained have shown that FuzzyRULES induces more compact and more accurate classification rule sets.

Keywords

Fuzzy sets rule induction inductive learning classification data mining

1 Introduction

Data mining is the process of discovering useful knowledge from data [1 –3]. The goal is to analyse the data to reveal regularities and relationships that are non-trivial and previously unknown. Data mining consists of a series of steps including data preparation, data modelling and model evaluation. The most important step in data mining concerns applying appropriate mining algorithms to the prepared data. Classification learning is the most common data mining technique. It employs a set of pre-categorised instances to develop a model that can classify new instances from the same population. Classification is a problem frequently encountered in engineering [4 –8]. Among the various learning approaches developed for classification, inductive learning is perhaps the most commonly adopted in real-world applications [9, 10]. Inductive learning techniques are simple and fast. Another advantage is that they generate models that are easy to understand. Finally, inductive learning classifiers are more accurate compared with other classification techniques.

Inductive learning techniques can be divided into two main categories, namely, decision tree induction and rule induction. There are a variety of algorithms for building decision trees [11 –13]. These learningsystems are categorised as “divide-and-conquer” inductive systems. The knowledge induced by these systems is represented as decision trees which can be transformed to a set of If-Then rules. As with decision tree learning, there are many rule induction algorithms [14 –17], which can be categorised as separate-and-conquer inductive systems. Unlike decision tree learning, rule induction directly generates If-Then rules.

Fuzzy logic has been increasingly used in data mining systems performing classification, due to its similarity with some aspects of human reasoning [18, 19]. A key feature of fuzzy logic models is that they can handle vagueness and uncertainty. Fuzzy logic allows one to deal with fuzzy and high-level notions (e.g. being small) while processing low-level information (e.g. size is 84.5 mm) [20].

Various learning methods have been developed for inducing fuzzy decision trees [21 –28]. Fuzzy rules can be indirectly learned by converting fuzzy decision trees to fuzzy rule sets [29], or by extracting fuzzy rules from neural networks [30 –35]. Genetic algorithms [36 –40] and Bayesian classifiers [41 –43] were also employed to obtain fuzzy rule sets.

There exist very few fuzzy rule learners that directly induce fuzzy classification rules from data [44 –48]. Hühn and Hüllermeier [47] introduced a fuzzy rule-based classification method called FURIA, which extends the RIPPER crisp rule induction algorithm [15]. FURIA learns fuzzy unordered rule sets and employs a rule stretching method to deal with the “no match” problem that occurs when using the induced rules to infer the class of a new instance. Experimental results demonstrated that FURIA significantly outperforms the original RIPPER, as well as other classifiers such as C4.5 [12], in terms of classification accuracy. However, the improved performance of FURIA comes at the price of an increase in the execution time. One problem with FURIA is that it first generates crisp rules and then fuzzifies these rules by replacing the crisp intervals of the numerical attributes appearing in the original rules by fuzzy intervals. Therefore, much work is wasted in learning crisp rules and subsequently fuzzifying these rules. Also, fuzzifying rules increases the number of covered instances, which entails the repetition of the whole induction process. This is because, in rule learning, the induction of the second rule is based on the instances remaining after the removal of the instances covered by the first rule. If the first rule is to be fuzzified, this would affect the induction of the second rule. Another problem is that FURIA fuzzifies rules in a greedy fashion, and uses the rule purity metric to measure the quality of fuzzification. Greedy search does not guarantee to find the best fuzzification of each rule. The purity metric obtains its optimal values when no negative instances are covered. Also, it does not aim to cover many positive instances. As a result, the purity metric tends to select very specific rules covering only a small number of instances. This is undesirable since rules covering few instances are unreliable, especially where there is noise in the data.

RULES-F Plus is a fuzzy rule induction algorithm for continuous output handling that is based on the structure of the RULES-5 Plus crisp rule learner [16, 48]. RULES-F Plus iteratively employs a specific rule-forming process in order to create a complete rule set covering all instances. One of the main problems of RULES-F Plus is that it learns a complete and consistent rule set that tries to cover all of the positive and none of the negative training instances. In the case of noisy data, specific rules based on a few training instances tend to be selected. Although these rules perform perfectly on the training instances, their predictive accuracy on future test instances is often lower because rules formed on the basis of small numbers of instances are susceptible to noise. Consequently, the rule set is both large and not of the highest accuracy.

This paper introduces a novel algorithm, FuzzyRULES (Fuzzy RULe Extraction System), for the induction of fuzzy rules based on the principles of fuzzy sets and fuzzy logic. The use of fuzzy sets increases the expressiveness of the rules, allowing semantically more natural conditions to be described. Furthermore, fuzzy sets provide a natural mechanism for dealing with continuous data. Finally, they offer a mechanism for reasoning in the presence of uncertainty, vagueness and ambiguity. One of the advantages of FuzzyRULES compared with most existing fuzzy induction algorithms is that it integrates the fuzzy set techniques into the rule induction process and allows the automatic creation of membership functions. FuzzyRULES also employs a fast and noise-tolerant search method for fuzzy rules generation. Finally, it uses simple and effective methods for fuzzy rule evaluation. The outcome is an efficient algorithm that produces accurate and compact fuzzy rule sets.

The remainder of this paper is organised as follows. Section 2 reviews some basic concepts of fuzzy logic relevant to fuzzy rule generation. Section 3 describes the proposed FuzzyRULES algorithm. An example illustrating the learning process of FuzzyRULES is described in Section 4. Section 5 presents an experimental evaluation of FuzzyRULES. Conclusions and future work are provided in Section 6.

2 Basic fuzzy set concepts

Introduced by Zadeh [49], fuzzy logic enables the creation of mathematical models for domains with boundaries that are not sharp. This section covers the basic concepts of fuzzy sets needed to develop the FuzzyRULES algorithm.

Let U be the universe of discourse of a variable u. A fuzzy set A in U is defined by a membership function μ_A (u) ∈ [0, 1] which describes the degree to which u belongs to the fuzzy set A. For u ∈ U, μ_A (u) =0 means that u does not belong to the fuzzy set and μ_A (u) ∈ ]0, 1] means that u belongs to the fuzzy set with a degree of membership μ_A (u).

Let A and B be two fuzzy sets in U with membership functions μ_A (u) and μ_B (u), respectively. The following fuzzy operators and measures can be defined [50 –52].

1. Union operator: The union of two fuzzy sets is a fuzzy superset where the membership degree of a member is the larger of its degrees of membership of the individual sets. Formally, the operator is defined as follows: $μ_{A \cup B} (u) = max (μ_{A} (u), μ_{B} (u)) \forall u \in U$ (1)

2. Intersection operator: The intersection of two fuzzy sets results in a new fuzzy set to which a member belongs with a membership degree that is the smaller of its degrees of membership of the individual fuzzy sets. Formally, the operator is defined as follows: $μ_{A \cap B} (u) = min (μ_{A} (u), μ_{B} (u)) \forall u \in U$ (2)

3. α-cut operator: The α-cut of a fuzzy set A, A_α, contains all elements in the universal set U with membership degrees in A greater than or equal to the specified value of α. Formally, the operator is defined as follows: $A_{α} (U) = {u \in U | μ_{A} (u) \geq α}$ (3)

4. Cardinality measure: The cardinality measure, also referred to as the sigma count, of a fuzzy set A, M (A), is the sum of all membership degrees of the set A. Formally, the measure is defined as follows: $M (A) = \sum_{u \in U} μ_{A} (u)$ (4)

3 The FuzzyRULES algorithm

FuzzyRULES is based on the learning strategy used in the RULES-6 crisp rule induction algorithm [53], previously developed by the author. RULES-6 belongs to the RULES family of simple inductive learning algorithms which have proven useful in several manufacturing and engineering applications [54 –56]. This section describes the proposed FuzzyRULES algorithm and highlights its distinctive features.

3.1 Representation

FuzzyRULES extracts fuzzy If-Then rules directly from a set of instances called the training set T. Each instance I is described by its nominal class value C_I and by a vector of n_a attributes (A¹, …, Aⁱ, …, A^{n
_a}). Each attribute value $v_{I}^{i}$ in instance I is either nominal or continuous. An instance I can therefore be formally defined asfollows:

$\begin{matrix} I = (A^{1} = v_{I}^{1}, \dots, A^{i} = v_{I}^{i}, \dots, \\ A^{n_{a}} = v_{I}^{n_{a}}, Class = C_{I}) \end{matrix}$ (5)

A fuzzy rule R is described by a conjunction of conditions on each attribute ( ${Cond}_{R}^{i}$ ) and the value (C_t) of an associated target class (the class to be learned). It can be represented as follows:

$\begin{matrix} R = {Cond}_{R}^{1} \land, \dots, \land {Cond}_{R}^{i} \\ \land, \dots, \land {Cond}_{R}^{n_{a}} \to C_{t} \end{matrix}$ (6)

For the ith attribute Aⁱ, a fuzzy condition has the form [Aⁱ is $L_{R}^{i}$ ], where $L_{R}^{i}$ is the linguistic value (term) of the ith attribute in rule R. Each linguistic value represents a fuzzy set for which every instance in the training set has a corresponding membership value. For continuous attributes, linguistic values can be obtained by defining membership functions on the domains of the attributes. Nominal attributes values can be converted to linguistic terms by crisp functions with membership degrees of either 0 or 1.

3.2 Fuzzy matching of rules

Due to the notion of fuzziness, an instance I will have a particular “degree of match” (μ_R (I)) with each rule R. The degree of match is obtained by first assessing the membership degree $μ_{L_{R}^{i}} (v_{I}^{i})$ of each attribute value ( $v_{I}^{i}$ ) of the instance with regard to the corresponding linguistic term ( $L_{R}^{i}$ ) in a rule R. Using these membership degrees, μ_R (I) can be computed using the intersection operator (Equation (2)) as follows:

$\begin{matrix} μ_{R} (I) = min (μ_{L_{R}^{1}} (I), \dots \dots, μ_{L_{R}^{i}} (I), \\ \dots \dots, μ_{L_{R}^{n_{a}}} (I)) \end{matrix}$ (7) where $μ_{L_{R}^{i}} (I)$ is the membership degree of instance I to the linguistic term $L_{R}^{i}$ . Clearly, if one membership degree is equal to zero, μ_R (I) will also be equal to zero. Also, if μ_R (I) =0, the instance is said to be not covered by the rule. If μ_R (I) >0, the instance is covered by the rule, i.e. it matches the rule.

During the rule induction process, an evaluation function is used to score rules directly based on the membership of instances to them, and this has an important effect on the search for good rules. In practice, many of these instances can match a rule antecedent to a small degree, and the summation of all these small degrees of membership can give the false impression that the rule is good, which can undermine the reliability of the evaluation function. To prevent such instances from being covered, a user-defined α-cut threshold can be applied to the instance memberships. The membership of an instance to a rule, μ_R (I), is defined to be zero when it lies below the α-cut threshold value. After the generation of rules, defuzzification must be performed when the rules are used to classify instances. For this process, the α-cut threshold can be applied again, i.e. a rule fires when the antecedent membership is above the threshold value.

When the value of μ_R (I) is greater than or equal to the specified value of α, rule R is said to α-cover instance I. The set of instances in the training set T that are α-covered by a rule R can be defined using the α-cut operator (Equation (3)) as follows: $R_{α} (T) = {I \in T | μ_{R} (I) \geq α}$ (8) where α is a user-defined threshold.

A rule set is a disjunction of a number of rules {R₁, …… , R_l, …… , R_{n
_r}}, and is denoted as RS = {R₁ ∨ …… ∨ R_l ∨ …… ∨ R_{n
_r}} . The degree of cover of instance I by the rule set RS can thus be evaluated using the union operator (Equation (1)) as follows:

$\begin{matrix} μ_{RS} (I) & = & max (μ_{R_{1}} (I), \dots \dots, μ_{R_{l}} (I), \\ \dots \dots, μ_{R_{n_{r}}} (I)) \end{matrix}$ (9) This concept of fuzzy matching is used in the FuzzyRULES algorithm to handle vagueness.

3.3 Fuzzy rules construction

A simplified description of the FuzzyRULES algorithm is given in Fig. 1. The algorithm induces fuzzy rules in an iterative fashion. In each iteration, it takes a “seed” instance not covered by previously created rules to form a new rule, as outlined in Fig. 2. Having found a rule, FuzzyRULES marks those instances that are α-covered by the rule and appends the new rule to its rule set. This process continues until all instances in the training set are covered. Note that the instances covered by previously formed rules are only marked in order to stop FuzzyRULES from repeatedly finding the same rule. However, these instances continue to be used to guide the rule specialisation process (i.e. the process of adding conditions to an existing conjunction of conditions) and to assess the accuracy and generality of newly formed rules. Also, FuzzyRULES does not work on a class-per-class basis. The class of the selected seed instance is considered “positive” (i.e. the target class) and all the remaining classes are regarded as “negative”.

The generated rules are simplified using a post pruning procedure based on the minimum description length (MDL) principle [57]. The pruning procedure is aimed at finding the rule set for each class that minimises the total coding length. It first examines each rule in turn starting from the last rule added to see whether it could be omitted or not. Then, if a rule cannot be omitted, it is pruned using a greedy strategy by repeatedly deleting a single condition starting from the last condition, if this improves (decreases) the total coding length, until no further improvements in the coding length is possible.

To construct a rule, FuzzyRULES performs a pruned general-to-specific beam search. The search aims to generate rules which cover as many instances from the target class and as few instances from the other classes as possible, while ensuring that the seed instance remains covered. As a consequence, simpler rules that are not consistent, but are more accurate for unseen data, can be learned. To increase the efficiency of the learning process, several search-space pruning tests are used to reduce the size of the search space and to stop further specialisation of a candidate rule when it cannot improve the classification performance of the rule set. The pruning tests can be implemented by associating two sets of linguistic terms, ValidTerms and InvalidTerms, to each candidate rule. The first set contains the valid linguistic terms that may still be used to specialise a rule. Initially, this set includes the linguistic terms constructed for all attributes values in the selected seed instance. The second set contains the invalid terms that will not lead to improved rules.

The learning of single rules starts with the most general rule whose body is empty. Specialisation of the rule involves incrementally adding conditions to its body. Possible conditions are linguistic terms defined on the attributes values of the selected seed instance. In the case of continuous attributes, conditions of the form [Aⁱ is $L_{s}^{i}$ ] are created, where $L_{s}^{i}$ is the linguistic term corresponding to the continuous value ( $v_{s}^{i}$ ) of attribute Aⁱ in the selected seed instance s. Membership functions have to be designed in order to specify the degree to which each attribute value ( $v_{s}^{i}$ ) belongs to the corresponding linguistic term ( $L_{s}^{i}$ ) of that attribute. A method that can automatically generate fuzzy membership functions for continuous attributes is described in Section 3.5. In the case of nominal attributes, the above fuzzy representation is also employed. However, the values of nominal attributes are converted to linguistic terms by assigning crisp functions with membership degrees of either 0 or 1. For each condition, a new ChildRule is formed by adding the condition to the current ParentRule. The term used to form the appended condition is removed from the ValidTerms set of the ChildRule to prevent repetition of specialisation effort. The score or quality measure of each new ChildRule is then computed and the rule with the best score is remembered.

The new ChildRule is examined to determine if it can be pruned away. A ChildRule that does not cover a minimum number of positive instances(i.e. instances belonging to the class defined by the rule) is removed from the search since no improved rules will be obtained from additional specialisations of this rule. Also, a ChildRule that does not exclude any new negative instances (i.e. instances not belonging to the class defined by the rule) is deemed ineffective since either it leaves out positive instances only, or it keeps the covered instances unchanged. A ChildRule that has become consistent is prevented from further specialisation as this will only decrease the number of positive instances covered by this rule and therefore yield lower values for the quality measure. If the ChildRule is discarded, the last linguistic term used to form it is added to the InvalidTerms set of its ParentRule so as to ensure that it will be removed from other specialisations of the same ParentRule. Otherwise, the ChildRule is added to a set of newly formed expressions representing fragments of potential rules, the ChildRules set. Thus, the ChildRules set only contains useful expressions that can be employed for further specialisation. This process is repeated until there are no rules remaining to be specialised in the ParentRules set which is the complete set of partial rules.

Another test that allows sections of the search space to be pruned away is now applied to each expression in the ChildRules set after the best expression overall in the current specialisation step is identified. Expressions are removed from the ChildRules set if their optimistic values of the quality measure cannot improve on that of the current BestRule. A simple optimistic value is obtained by determining the quality measure of a rule with the same positive cover as the current rule but with zero negative cover. The last linguistic terms used to generate these rules are added to the InvalidTerms set of their parents. All InvalidTerms are then deleted from the corresponding set of ValidTerms for each expression in the ChildRules set. Such terms cannot lead to a viable specialisation from any point in the search space that can be reached via identical sets of specialisations and thus excluding them will prevent the unnecessary construction of ineffective specialisations at subsequent specialisation steps.

After all duplicate rules have been eliminated, the best w expressions from the ChildRules set are chosen to replace all expressions in the ParentRules set. The comparison between expressions is based on the quality measure defined in the next subsection. If two expressions have equal quality measures, the simpler rule, in other words, the one with fewer conditions is selected. If both the quality measure and the number of conditions of the expressions are the same, the more general expression which covers more instances is chosen. The specialisation process is then repeated until the ParentRules set becomes empty. During specialisation, the best expression obtained is stored and returned at the end of the procedure.

3.4 Fuzzy computation of the rule quality measure

Given that the rule induction process could be conceived as a search process, a metric is needed to estimate the quality of rules found in the search space and to direct the search towards the best rule. The rule quality measure is the most influential bias in rule induction. In real-world applications, a typical objective of a learning system is to find rules that optimise a rule quality criterion that takes both training accuracy and rule coverage into account so that the rules learned are both accurate and reliable.

A quality measure must be estimated from the available data. All common measures are based on the number of positive and negative instances covered by a rule. Several different metrics are used in existing algorithms [58 –60]. For some of these, fuzzy variations were designed and used to score rules [61, 62]. A rule quality measure that performs very well, especially in the presence of noise, is the m-probability-estimate [63]. For a rule R, this measure can be defined as follows: $mAccuracy (R) = \frac{p + {mP}_{0} (C_{t})}{p + n + m}$ (10) where p and n are the numbers of positive and negative instances covered by the rule, P₀ (C_t) is the a priori probability of the target class C_t and m is a domain dependent parameter.

FuzzyRULES employs a fuzzy version of them-probability-estimate to score the performance of rules on the training set during the learning process, and to select the set of best candidates for further exploration. Let A be the antecedent of rule R and C its consequent. Then the fuzzy version of them-probability-estimate function can be defined using the α-cut operator (Equation (3)) and the cardinality measure (Equation (4)) as follows:

$\begin{matrix} mAccuracy (R) = \frac{M (A \cap C) + {mP}_{0} (C_{t})}{M (A) + m} \\ = \frac{\sum_{I \in R_{α} (P)} μ_{R} (I) + {mP}_{0} (C_{t})}{\sum_{I \in R_{α} (T)} μ_{R} (I) + m} \end{matrix}$ (11) where M (A ∩ C) is the fuzzified version of the number p of positive instances α-covered by the rule, and M (A) is the fuzzified version of the number (p + n) of instances α-covered by the rule. This paper only considers crisp class attributes and therefore M (A ∩ C) can be calculated as ∑_{I∈R_α(P)}μ_R (I), where μ_R (I) is a rule membership value specifying the degree to which rule R covers instance I and R_α (P) , defined as {I ∈ P|μ_R (I) ≥ α} , is the set of positive instances α-covered by rule R. Similarly, M (A) can be calculated as ∑_{I∈R_α(T)}μ_R (I), where R_α (T) is the set of instances in the training set that are α-covered by R and is defined as R_α (T) = {I ∈ T|μ_R (I) ≥ α} .

In FuzzyRULES, m is set to n_c, the number of classes, and the a priori probability P₀ (C_t) is assumed to be equal to the training accuracy of the empty rule that predicts the target class, namely: $P_{0} (C_{t}) = \frac{n_{C_{t}}}{N}$ (12) where n_{C
_t} is the number of instances belonging to the target class C_t and N is the total number of instances in the training data set.

3.5 Fuzzy discretisation of continuous attributes

When there are both continuous and nominal attributes in a data set, most rule induction techniques discretise continuous attributes into intervals and the discretised intervals are treated in a similar way to nominal values during induction. In the crisp case, discretisation results in “hard” intervals and an attribute value either belongs to a certain interval or not. In the fuzzy case, however, an attribute value belongs to an interval to a certain degree. Discretised intervals, therefore, should be fuzzily interpreted. An intuitive way to obtain fuzzy intervals for each continuous attribute is to discretise its domain into several crisp intervals and fuzzify these intervals accordingly. This section describes how the fuzzy intervals of continuous attributes are defined and how membership functions are constructed for every associated fuzzy interval.

A simple method of crisp discretisation is to divide the domain of each attribute into intervals with equal length or equal frequency [64]. This method, however, generally results in poor system performance. On the other hand, it is also difficult and time consuming to find an optimal discretisation [65]. An efficient method is to find several possible cut points and perform partitioning for a limited number of times. Fayyad and Irani [66] proved that with instances sorted in ascending order by the values of a continuous attribute, possible cut points exist in the boundary where two adjacent instances belong to different classes. A cut point with the highest information gain [67] is then used to split the range of the attribute. This process is iteratively applied until astopping criterion based on the MDL principle [68] is met. This method is employed in this study to obtain crisp intervals for each continuous attribute in the data being mined. A crisp interval can then be fuzzified by associating an appropriate fuzzy membership function with it. An attribute value can then be classified into different intervals with different degrees at the same time. Membership functions can be assigned by domain experts or derived from statistical data [69]. Fuzzy clustering can also be used to determine membership functions [70]. This section proposes a new method that generates membership functionsautomatically.

Typical shapes of membership functions are triangular, trapezoidal, and bell-shaped. Triangular membership functions are chosen in this study as they are simple and often used in fuzzy sets. These functions can be calculated from the boundary points of the crisp intervals using the algorithm detailed below. The example given in Fig. 3 is used to illustrate the steps of the algorithm. Figure 3(a) shows the cut points obtained by applying the crisp discretisation method described above. These points can be used to form a set of disjoint intervals which can be described by means of crisp membership functions as shown in Fig. 3(b). The value 1 is assigned to each membership function if the attribute value is placed inside the corresponding interval. The remaining membership values are equal to 0. In Fig. 3(c), overlapping intervals described by fuzzy membership functions are shown. A point near to a cut point belongs to two fuzzy sets with a membership degree less than 1 and greater than 0 for both membership functions. Assuming that the sum of two adjacent membership functions is always one and the crossing points of these functions coincide with the cut point in the interval partition, the triangular membership functions can be generated as follows.

$\begin{matrix} μ_{L_{1}} (v) \\ = {\begin{matrix} 1, & v \leq a_{1}, \\ (a_{2} - v) / (a_{2} - a_{1}), & a_{1} < v < a_{2}, \\ 0, & v \geq a_{2} . \end{matrix} \end{matrix}$ (13) $\begin{matrix} μ_{L_{j}} (v) \\ = {\begin{matrix} 0, & v \geq a_{j + 1}, \\ (a_{j + 1} - v) / (a_{j + 1} - a_{j}), & a_{j} < v < a_{j + 1}, \\ 1, & v = a_{j} & 1 < j < n_{l} \\ (v - a_{j - 1}) / (a_{j} - a_{j - 1}), & a_{j - 1} < v < a_{j}, \\ 0, & v \leq a_{j - 1} . \end{matrix} \end{matrix}$ (14) $\begin{matrix} μ_{L_{n_{l}}} (v) \\ = {\begin{matrix} 1, & v \geq a_{n_{v}}, \\ (v - a_{n_{v} - 1}) / (a_{n_{v}} - a_{n_{v} - 1}), & a_{n_{l} - 1} < v < a_{n_{l}}, \\ 0, & v \leq a_{n_{l} - 1} . \end{matrix} \end{matrix}$ (15)

where v is a value in the domain of a continuous attribute A, L_j is the jth fuzzy linguistic term associated with attribute A and μ_{L
_j} (v) is a fuzzy membership function specifying the degree to which the original value v of attribute A belongs to the respective linguistic term of the attribute.

In this case, the only parameters that need to be determined are the a_j (j = 1, …, n_l) values. Those values can be obtained from the set of c_k (k = 1, …, n_l - 1) cut points as follows: $\begin{matrix} a_{j} \\ = {\begin{matrix} c_{k} - (c_{k + 1} - c_{k}) / 2, & j = k = 1, \\ a_{j - 1} + 2 * (c_{j - 1} - a_{j - 1}), & \forall j = k, c_{j - 1} > a_{j - 1}, & 1 < j \leq n_{l}, 1 < k \leq n_{l} - 1 . \end{matrix} \end{matrix}$ (16)

3.6 Classification of new instances

When the induced rule set is used to classify a new instance, three outcomes are possible:

One rule or multiple rules having the same consequent covers the new instance,

More than one rule covers the new instance and the matched rules have different consequents, or

No rules cover the new instance.

Each case requires a different classification procedure to predict a label for the new instance. The FuzzyRULES algorithm applies its fuzzy rule set to classify new instances in the same way as the classical rule sets created by RULES-6. The difference is that the degree of match between each instance I and rule R (Equation (7)), and the α-cut threshold (Equation (8)) will be used when computing the coverage of rules. In the first case, FuzzyRULES simply assigns the class predicted by the matched rule(s) to the new instance. To resolve the conflict between rules in the second case, FuzzyRULES selects the rule with the highest membership value to classify the new instance (Equation (9)). In the third case, the generally adopted solution is to use a default rule which simply assigns the most frequent class in the entire training set to the new instance to be classified, independently of its attributes. This method is appropriate in cases where the number of classes in a training set is small and one of the classes contains a predominant number of instances. However, the results deteriorate when the number of classes grows, and instances are more evenly distributed. FuzzyRULES classifies the new instance by assigning to it the class of the nearest rule in the rule set. To find the nearest rule, a simple distance measure introduced by Bigot [71] is employed.

4 Illustrative example

This section provides an example of applying the FuzzyRULES algorithm to support process planning by proposing feasible process routes to produce a part. All possible technological routes in a given manufacturing environment are stored in a database. When it is required to select a route for a new part, the values of the part’s attributes are entered and an appropriate route can be retrieved from the database. The selected route can be further modified ifnecessary.

Table 1 shows a sample training data used for the development of process routes. Each instance is described by four attributes of the part with the following values: Heat Treatment (Yes, No), Material (Steel_3135, Aluminium, Steel_1045), Tolerance (6.5, 8, 8.5, 10, 11, 13, 14, 15), and Surface Finish (Low, Medium, High). There are four classes with values (R1, R2, R3, R4) denoting the possible technological routes to produce the part.

FuzzyRULES starts inducing rules for the first seed instance (Heat Treatment = Yes, Material =Steel_1045, Tolerance = 8, Surface Finish = Medium, Route = R1). The prespecified values of the algorithm parameters MinPositives (the minimum number of positive instances covered), MinNegatives (the minimum number of negative instances excluded), w (the beam width), and the α-cut threshold are 1, 1, 1, and 0.5, respectively. The post pruning procedure is disabled.

Figure 4 shows the entire set of rules that could be derived from the selected seed instance. Each node represents a rule, where the terms present in the rule are shown in square brackets. The first two values in the parentheses are the numbers of positive and negative instances covered by the rule, respectively. The third value in the parentheses is the evaluation obtained by the respective rule. The circled nodes indicate the rules examined by FuzzyRULES and the bold ones show the best rules chosen in each specialisation level.

The search process starts with the default most general rule having null condition “If [No Condition] THEN Route is R1”. This rule has a score of 0.145 as computed by the m-probability-estimate measure defined in Equation (11). The default rule is used to initialise the BestRrule and the ParentRules set. The ChildRules set is initialised to empty.

Linguistic terms are then used to create new child rules by specialising the ChildRule(s) in the ParentRules set. Four linguistic terms are constructed for the values of the attributes in the selected seed instance. These are [Heat Treatment is Yes], [Material is Steel_1045], [Tolerance is $L_{1}^{3}$ ], and [Surface Finish is Medium]. The membership functions generated by FuzzyRULES to define the linguistic terms of the numerical attribute Tolerance are shown in Fig. 5. The values 9.25 and 12 in the figure are the cut points generated by the crisp discretisation method of FuzzyRULES to split the range of the Tolerance attribute. From these values, the parameters a_j which define the boundaries of the membership functions can be computed using Equation (16). The linguistic term $L_{1}^{3}$ is chosen to represent the value 8 of the Tolerance attribute in the chosen seed instance since the membership degree $μ_{L_{1}^{3}} (8)$ is maximum. $μ_{L_{1}^{3}} (8)$ can be computed using Equation (13) and its value is 0.95.

The formed linguistic terms are used to generate the following four child rules:

ChildRule 1: IF [Heat Treatment is Yes] THEN Route is R1,

ChildRule 2: IF [Material is Steel_1045] THEN Route is R1,

ChildRule 3: IF [Tolerance is $L_{1}^{3}$ ] THEN Route is R1,

ChildRule 4: IF [Surface Finish is Medium] THEN Route is R1.

These four rules have better scores than the BestRule and the first three rules have the same evaluation. ChildRule 1 is generated first and it is therefore chosen to replace the BestRule. The new child rules are then examined to determine if they can be pruned away based on the search-space pruning tests. Since the new child rules are not consistent, their MinPositives and MinNegatives values are higher than the predefined values, and their optimistic scores are better than the current BestRule, they are added to the ChildRules set for further specialisation.

The best w rules from the ChildRules set are chosen to replace all rules in the ParentRules set. Since w is equal to one, only the ChildRule “IF [Heat Treatment is Yes] THEN Route is R1” with the best evaluationis chosen, and the specialization process is then repeated. New linguistic terms except the term “[Heat Treatment is Yes]” are added to the current rule. The following three child rules are generated:

ChildRule 1: IF [Heat Treatment is Yes] AND [Material is Steel_1045] THEN Route is R1,

ChildRule 2: IF [Heat Treatment is Yes] AND [Tolerance is $L_{1}^{3}$ ] THEN Route is R1,

ChildRule 3: IF [Heat Treatment is Yes] AND [Surface Finish is Medium] THEN Route is R1.

ChildRules 1 and 2 obtained the same scores and their evaluation is higher than that of the BestRule. Again, ChildRule 1 is selected to replace the BestRule as it is generated first. ChildRules 1 and 2 are consistent as they cover no negative instances. Also, ChildRule 3 does not exclude (uncover) more negative instances. Although FuzzyRULES can create further specialisations to these rules, this can only cause their evaluation to become worse because only the same number or fewer positive instances will be covered. Therefore, ChildRules 1, 2 and 3 are removed from the search process and they are not added to the ChildRules set. As the ChildRules set does not have any new rules to specialise, FuzzyRULES halts the search and returns the BestRule.

The result of the process is the following rule “IF [Heat Treatment is Yes] AND [Material is Steel_1045] THEN Route is R1”. This rule is added to the rule set, and the training instances that are α-covered by it are marked as covered. Using Equations (7) and (8), it can be easily determined that the generated rule α-covers instance 7 in addition to instance 1. Since there are still instances in the training set that are not covered by the formed rule, the learning procedure must generate new rules to cover these instances. The four rules produced by the FuzzyRULES algorithm are listed in Fig. 6. The classification accuracy of these rules on the training data is 100% .

5 Experimental results

This section compares FuzzyRULES against other crisp and fuzzy rule learning algorithms. FuzzyRULES was compared first with the RULES-6 crisp rule induction algorithm on which the fuzzy version was based. FuzzyRULES was compared also against the well-known crisp inductive learner C5.0 [13] which is probably the best performing induction algorithm commercially available. Finally, FuzzyRULES was compared against the recently developed RULES-F Plus fuzzy rule learning algorithm [48].

Three criteria were used to evaluate the performance of the tested algorithms, namely, classification accuracy, rule set complexity and number of rules evaluated during the search process. The complexity of a rule set is measured by the total number of rules or total number of conditions in that rule set. In order to draw reliable conclusions about the behaviour of the learning algorithms, 30 data sets (listed in Table 2) were considered. All data sets were obtained from the University of California at Irvine (UCI) repository of machine learning databases [73]. In the experiments conducted in this study, the hold-out approach was used to partition the data into training and test data [74, 75]. Each large data set with more than 1000 instances was randomly divided once into a training set with two-thirds of the data and a test set with the remaining one-third. For small data sets with fewer than 1000 instances, this partitioning procedure and the induction process were repeated ten times, and the results were averaged. RULES-6, C5.0, RULES-F Plus, and FuzzyRULES each has a number of parameters whose values determine the quality of their induced rule sets. The default parameters of RULES-6, and C5.0 were used. For RULES-F Plus, the experiments reported in [48] set the beam width to 2 and the noise level (NL) to 0.1, 0.2,or 0.3.

Therefore, these parameter values were used in this study. For FuzzyRULES, the α-cut threshold was set to 0.5.

5.1 Comparison with RULES-6

Table 3 shows the results obtained. As can be seen from the table, the performance obtained by FuzzyRULES was impressive. There were considerable improvements in the classification accuracy for 17 data sets. The accuracy degraded for 9 data sets. For the remaining 4 data sets, equivalent results were obtained. It can also be noted from the table that FuzzyRULES produced much more compact rule sets than RULES-6. The total number of rules decreased by 25.5 per cent (from 781 to 582). Also, the total number of conditions dropped by 19.3 per cent (from 2332 to 1883). The capability of the FuzzyRULES algorithm to produce simple and accurate rules can be attributed to the use of fuzzy linguistic terms in its rules antecedents, which supports the generation of more natural and more comprehensible rules, and makes FuzzyRULES more robust in tolerating imprecise, conflict, and missing information. Despite using a more complex rule-forming process, the fewer and more general rules created by the FuzzyRULES algorithm made it much faster than RULES-6 as indicated in Table 3. The total number of evaluations fell by 15.7 per cent (from 176369 to 148704). These results confirm that FuzzyRULES obtains rules that are less complex, but more accurate than those extracted by RULES-6.

5.2 Comparison with C5.0

FuzzyRULES was compared with C5.0 for the data sets listed in Table 2. C5.0 has a facility to generate a set of pruned production rules from a decision tree. Table 4 presents the results for each algorithm on each data set. In each case, the accuracy of the test data and the complexity of the resulting rule sets are given. The number of rules was taken as a measure of the complexity of the rule set. A complexity of 1 was assigned to the default rule. It is clear from Table 4 that the accuracy obtained by FuzzyRULES was, in total, higher than that of C5.0. In addition, FuzzyRULES achieved higher accuracies for 19 out of 30 data sets, whereas C5.0 yielded better accuracies for 11 out of 30 data sets. It is also clear from the table that, in total, FuzzyRULES created fewer rules than C5.0. The number of rules was lower for 25 data sets and higher for 5 data sets for FuzzyRULES when compared with C5.0. Taking into account both the number of rules created and the test accuracy obtained, it can be concluded that the FuzzyRULES algorithm outperformed C5.0 in the classification experiments conducted. Although in most situations the C5.0 decision tree induction method works well, it has a number of problems due to its representation of rules and its strategy for induction. First, it often happens that identical subtrees have to be learned at various places in a decision tree. Second, by minimising the average entropy of a set of instances, C5.0 disregards the fact that some attributes or attribute values may be irrelevant to a particular classification. Finally, C5.0 performs a kind of hill-climbing search for rules, by exploring just one alternative at a time, which makes it very sensitive to the problem of local maxima. The FuzzyRULES algorithm, on the other hand, does not suffer from these problems. It uses attribute-value pairs instead of attributes when inducing rules and thus avoids generating inappropriate and overspecialised models. Also, FuzzyRULES has the advantage of being less sensitive to the local maxima problem, since it explores w alternatives at a time, where w is the beam width.

The problems of the C5.0 algorithm can be demonstrated by applying it to the data of the process planning problem. Figure 7 displays the decision tree constructed from this data and Fig. 8 shows the rules derived from this tree. C5.0 generated rules that are more complex than those of FuzzyRULES. Some of the rules and conditions produced by C5.0 are redundant such as the first condition in the first rule, and the second rule which does not cover anyinstances.

5.3 Comparison with RULES-F Plus

The performance of FuzzyRULES has been compared against that of RULES-F Plus. Table 5 presents the results obtained. It is clear from Table 5 that FuzzyRULES achieved a substantially lower complexity without sacrificing accuracy for most data sets. The accuracy obtained by FuzzyRULES over all the data sets was in total higher than that produced by RULES-F Plus with the three noise levels considered. Also, it produced significantly fewer rules in total than RULES-F Plus for all noise levels. Moreover, FuzzyRULES gives better performance results than the best results obtained by RULES-F Plus with the optimal noise level. In terms of classification accuracy, FuzzyRULES outperformed RULES-F Plus for 20 out of 30 data sets. On 3 data sets, FuzzyRULES developed rule sets with accuracies similar to RULES-F Plus. Only on 7 data sets, did RULES-F Plus outperform FuzzyRULES. In terms of the number of rules created, FuzzyRULES obtained fewer rules 13 times and RULES-F Plus 15 times. Both algorithms produced equal number of rules for the remaining 2 data sets. However, RULES-FPlus created only one rule (“IF anything THEN predicts the class with the largest number of training instances”) for the data sets Anneal (NL = 0.3), German (NL = 0.3), Hepatitis (NL = 0.3), Hypothyroid (NL = 0.1, 0.2, and 0.3), Shuttle (NL = 0.3), and Sick-euthyroid (NL = 0.1, 0.2, and 0.3). This demonstrates a deficiency of the RULES-F Plus algorithm as it fails to learn any interesting pattern in the above-mentioned data sets. It could therefore be concluded that FuzzyRULES gives the best performance when considering both the classification accuracy and the number of rules. The improved performance of the FuzzyRULES algorithm compared to the RULES-F Plus algorithm can be attributed to the ability of FuzzyRULES to handle noise in the data, which is achieved by employing a search method that tolerates inconsistency in the rule specialisation process. Also, FuzzyRULES adopts a very simple criterion for evaluating the quality of rules and a robust method for handling attributes with continuous values, which further improves the performance of the algorithm.

6 Conclusions and future work

This paper has presented FuzzyRULES, a novel algorithm for inducing fuzzy classification rules from data. FuzzyRULES uses an approach similar to that employed by the RULES-6 crisp rule induction algorithm. However, unlike RULES-6 and other crisp rule induction procedures, FuzzyRULES is able to handle fuzzy data, and thus has the all benefits associated with fuzzy sets. Continuous attributes, for example,are processed in a much more natural way, and vagueness and ambiguity can be readily handled. The operation of FuzzyRULES has been illustrated in an industrial application concerning the automatic identification of possible process routes to produce a part. FuzzyRULES has also been compared against RULES-6 and C5.0 crisp inductive learners, as well as the recently developed RULES-F Plus fuzzy learning algorithm. The results obtained have clearly shown the strong performance of the new algorithm.

In this research, all output values were of the discrete, nominal type. However, it is common for real-world data to have continuous values. Future work should focus on developing fuzzy rule induction algorithms that can predict continuous output values. Also, this study only considered the triangular membership function and one simple method for describing it. Future work should investigate other membership functions such as trapezoidal functions and other methods of determining the cut points including those based on the density of attributevalues such as clustering techniques.

Footnotes

Acknowledgments

The author wishes to thank the Industrial Engineering Department at Zagazig University, Egypt for providing a good environment, facilities and financial means to complete this paper.

References

Witten

I.H.

and Frank

, Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations, Morgan Kaufmann Publishers, USA, 2000.

Han

and Kamber

, Data Mining: Concepts and Techniques, Morgan Kaufmann Publishers, San Francisco, California, USA, 2001.

Klösgen

and Żytkow

J.M.

, Handbook of Data Mining and Knowledge Discovery, Oxford University Press, New York, USA, 2002.

Braha

, Data Mining for Design and Manufacturing: Methods and Applications, Kluwer Academic Publishers, Boston, Massachusetts, USA, 2001.

Monostori

, AI and machine learning techniques for managing complexity, changes and uncertainties in manufacturing, Engineering Applications of Artificial Intelligence16 (2003), 227–291.

Pham

D.T.

and Afify

A.A.

, Machine-learning techniques and their applications in manufacturing, Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture219 (2005), 395–412.

Harding

J.A.

, Shahbaz

, Srinivas

and Kusiak

, Data mining in manufacturing: A review, Journal of Manufacturing Science and Engineering128 (2006), 969–976.

Kusiak

and Smith

, Data mining in design of products and production systems, Annual Reviews in Control1 (2007), 147–156.

Apté

and Weiss

, Data mining with decision trees and decision rules, Future Generation Computer Systems13 (1997), 197–210.

10.

Pham

D.T.

and Afify

A.A.

, Machine learning: Techniques and trends, In: Proceedings of the 9th International Workshop on Systems, Signals and Image Processing (IWSSIP-02), Manchester Town Hall, UK, 2002, pp. 12–36.

11.

Breiman

, Friedman

J.H.

, Olshen

R.A.

and Stone

C.J.

, Classification and Regression Trees, Belmont, 1984.

12.

Quinlan

J.R.

, C4.5: Programs for Machine Learning, Morgan Kaufmann, San Mateo, California, USA, 1993.

13.

Data

RuleQuest.

, Data Mining Tools C5.0. Pty Ltd, 30 Athena Avenue, St IvesNSW2075, Australia.Available from: http://www.rulequest.com/see5-info.html [Accessed: 20 March 2015].

14.

Clark

and Boswell

, Rule induction with CN2: Some recent improvements, In: Proceedings of the 5th European Conference on Artificial Intelligence, Porto, Portugal, 1991, pp. 151–163.

15.

Cohen

W.W.

, Fast effective rule induction, In: Proceedings of the 12th International Conference on Machine Learning, Lake Tahoe CityCalifornia, USA, 1995, pp. 115–123.

16.

Pham

D.T.

, Bigot

and Dimov

S.S.

, RULES-5: A rule induction algorithm for problems involving continuous attributes, Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science217 (2003), 1273–1286.

17.

Pham

D.T.

and Afify

A.A.

, SRI: A scalable rule induction algorithm, Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical EngineeringScience220 (2006), 537–552.

18.

Janikow

C.Z.

, Fuzzy decision trees: Issues and methods, IEEE Transactions on Systems, Man, and Cybernetics,Part B: Cybernetics28 (1998), 1–14.

19.

Hüllermeier

, Fuzzy methods in machine learning and data mining: Status and prospects, Fuzzy Sets and Systems156 (2005), 387–406.

20.

Sebag

and Schoenauer

, Inductive learning of membership functions and fuzzy rules, In: Uncertainty Modelling and Analysis: Theory and Applications, Ayyub

and Gupta

M.M.

, ed., North-Holland-Elsevier Scientific Publishers, Amsterdam, The Netherlands, 1994, pp. 275–283.

21.

Cios

K.J.

and Sztandera

L.M.

, Continuous ID3 algorithm with fuzzy entropy measures, In: Proceedings of The First IEEE Conference on Fuzzy Systems, San Diego, California, USA, 1992, pp. 469–476.

22.

Yuan

and Shaw

M.J.

, Induction of fuzzy decision trees, Fuzzy Sets and Systems69 (1995), 125–139.

23.

Jeng

, Jeng

Y.-M.

and Liang

T.-P.

, FILM: A fuzzy inductive learning method for automated knowledge acquisition, Decision Support Systems21 (1997), 61–73.

24.

Lee

K.-M.

, Lee

K.-M.

, Lee

J.-H.

and Hyung

L.-K.

, A fuzzy decision tree induction method for fuzzy data, In: Proceedings of the IEEE International Conference on Fuzzy Systems, Seoul, Korea, vol. 1, 1999, pp. 16–21.

25.

Suárez

and Lutsko

J.F.

, Globally optimal fuzzy decision trees for classification and regression, IEEE Transactions on Pattern Analysis and Machine Intelligence21 (1999), 1297–1311.

26.

Dong

and Kothari

, Look-ahead based fuzzy decision tree induction, IEEE Transactions on Fuzzy Systems9 (2001), 461–468.

27.

Olaru

and Wehenkel

, A complete fuzzy decision tree technique, Fuzzy Sets and Systems138 (2003), 221–254.

28.

Crockett

, Bandar

, Mclean

and O’Shea

, On constructing a fuzzy inference framework using crisp decision trees, Fuzzy Sets and Systems157 (2006), 2809–2832.

29.

Mitra

, Konwar

K.M.

and Pal

S.K.

, Fuzzy decision tree, linguistic rules and fuzzy knowledge-based network: Generation and evaluation, IEEE Transactions on Systems, Man, and Cybernetics –Part C: Applications and Reviews32 (2002), 328–339.

30.

Halgamuge

S.K.

and Glesner

, Neural networks in designing fuzzy systems for real world applications, Fuzzy Sets and Systems65 (1994), 1–12.

31.

Zhang

and Hoshi

, A fuzzy neural network model (FNN Model) for classification using Landsat-TM image data, Pasadena, In: Proceedings of the IEEE Interpretation Geoscience and Remote Sensing Symposium (IGARSS-94), California, USA, vol. 3, (1994), 1416–1418.

32.

Abe

and Lan

M.-S.

, A method for fuzzy rules extraction directly from numerical data and its application to pattern classification, IEEE Transactions on Fuzzy Systems3 (1995), 18–28.

33.

Tschichold-Gürman

, Generation and improvement of fuzzy classifiers with incremental learning using Fuzzy RuleNet, In: Applied Computing (Proceedings of the ACM Symposium on Applied Computing, Tennessee, USA), George

K.M.

, Carrol

J.H.

, Deaton

, Oppenheim

and Hightower

, ed., ACMPress, New York, USA, 1995, pp. 466–470.

34.

Nauck

and Kruse

, NEFCLASS –a neuro-fuzzy approach for the classification of data, In: Applied Computing (Proceedings of the ACM Symposium on Applied Computing, Nashville, Tennessee, USA), George

K.M.

, Carrol

J.H.

, Deaton

, Oppenheim

and Hightower

, ed., ACMPress, New York, USA, 1995, pp. 461–465.

35.

Quteishat

and Lim

C.P.

, A modified fuzzy min–max neural network with rule extraction and its application to fault detection and classification, Applied Soft Computing8 (2008), 985–995.

36.

Ishibuchi

, Nozaki

, Yamamoto

and Tanaka

, Selecting fuzzy if-then rules for classification problems using genetic algorithms, IEEE Transactions on Fuzzy Systems3 (1995), 260–270.

37.

Yuan

and Zhuang

, A genetic algorithm for generating fuzzy classification rules, Fuzzy Sets and Systems84 (1996), 1–19.

38.

Ishibuchi

, Nakashima

and Kuroda

, A hybrid fuzzy GBML algorithm for designing compact fuzzy rule-based classification systems, In: Proceedings of the 9th IEEE International Conference on Fuzzy Systems, San Antonio, Texas, USA, vol. 2, 2000, pp. 706–711.

39.

Hoffmann

, Combining boosting and evolutionary algorithms for learning of fuzzy classification rules, Fuzzy Sets and Systems141 (2004), 47–58.

40.

Ishibuchi

, Nojima

and Kuwajima

, Fuzzy data mining by heuristic rule extraction and multiobjective genetic rule selection, In: Proceedings of the IEEE International Conference on Fuzzy Systems, Vancouver, Canada, 2006, pp. 1633–1640.

41.

Borgelt

, Timm

and Kruse

, Probabilistic networks and fuzzy clustering as generalizations of Naive Bayes classifiers, In: Computational Intelligence in Theory and Practice (Advances in Soft Computing), Reusch

and Temme

K.-H.

, ed., Physica-Verlag, Heidelberg, Germany, 2001, pp. 121–138.

42.

Störr

H.P.

, A compact fuzzy extension of the Naive Bayesian classification algorithm, In: Proceedings of the 3rd International Conference on Intelligent Technologies and Vietnam-Japan Symposium on Fuzzy Systems and Applications, Hanoi, Vietnam, 2002, pp. 172–177.

43.

Tang

, Pan

, Li

and Xu

, Fuzzy Naive Bayes classifier based on fuzzy clustering, In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics5 (2002).

44.

Wang

L.X.

and Mendel

J.M.

, Generating fuzzy rules by learning from examples, IEEE Transactions on Systems, Man, and Cybernetics22 (1992), 1414–1427.

45.

Fertig

C.S.

, Freitas

A.A.

, Arruda

L.V.R.

and Kaestner

, A fuzzy beam-search rule induction algorithm, In: Principles of Data Mining and Knowledge Discovery (Proceedings of the 3rd European Conference, Prague, Czech Republic), Żytkow

J.M.

and Rauch

, ed, Springer, UK, 1999, pp. 341–347.

46.

Cloete

and van Zyl

, Fuzzy rule induction in a set covering framework, IEEE Transactions on Fuzzy Systems14 (2006), 93–110.

47.

Hühn

and Hüllermeier

, FURIA: An algorithm for unordered fuzzy rule induction, Data Mining and Knowledge Discovery19 (2009), 293–319.

48.

Bigot

, A new rule space representation scheme for rule induction in classification and control applications, Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering225 (2011), 1018–1038.

49.

Zadeh

L.A.

, Fuzzy sets, Information and Control8 (1965), 338–353.

50.

Klir

G.J.

and Yuan

, Fuzzy Sets and Fuzzy Logic: Theory and Applications, Prentice-Hall, Upper Saddle River, New Jersey, USA, 1995.

51.

Cios

K.J.

, Pedrycs

and Swiniarski

, Data Mining Methods for Knowledge Discovery, Kluwer Academic Publishers, Boston, Massachusetts, USA, 1998.

52.

Cox

, The Fuzzy Systems Handbook: A Practitioner’s Guide to Building, Using, and Maintaining Fuzzy Systems, 2nd Edition, Academic Press, London, UK, 1998.

53.

Pham

D.T.

and Afify

A.A.

, RULES-6: A simple rule induction algorithm for handling large data sets, Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science219 (2005), 1119–1137.

54.

Pham

D.T.

and Dimov

S.S.

, An approach to concurrent engineering, Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture212 (1998), 13–27.

55.

Jennings

N.R.

, Automated Visual Inspection of Engine Valve Stem Seals, Internal Report, Cardiff University, UK, 1996.

56.

Afify

A.A.

, Dimov

, Naim

M.M.

, Valeva

and Shukla

, Data mining: A tool for detecting cyclical disturbances in supply networks, Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture221 (2007), 1771–1785.

57.

Pham

D.T.

and Afify

A.A.

, A new minimum description length based pruning technique for rule induction algorithms, Proceedings of the Institution of Mechanical Engineers,Part C: Journal of Mechanical Engineering Science222 (2008), 1339–1352.

58.

Lavrač

, Flach

P.A.

and Zupan

, Rule evaluation measures: A unifying view, In: Proceedings of the 9th International Workshop on Inductive Logic Programming, Bled, Slovenia, Springer-Verlag, Berlin, Germany, 1999, pp. 174–185.

59.

Fürnkranz

and Flach

P.A.

, An analysis of rule evaluation metrics, In: Proceedings of the 20th International Conference on Machine Learning, Washington, USA, 2003, pp. 202–209.

60.

Fürnkranz

and Flach

P.A.

, ROC ’n’ rule learning –Towards a better understanding of covering algorithms, Machine Learning58 (2005), 39–77.

61.

Cloete

and van Zyl

, Evaluation function guided search for fuzzy set covering, In: Proceedings of the IEEE International Conference on Fuzzy Systems, Budapest, Hungary, 2004, pp. 1007–1012.

62.

van Zyl

and Cloete

, Heuristic functions for learning fuzzy conjunctive rules, In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, The Hague, The Netherlands, 2004, pp. 2332–2337 .

63.

Cestnik

, Estimating probabilities: A crucial task in machine learning, Proceedings of the 3rd European Conference on Artificial Intelligence, Sweden, 1990, pp. 147–149.

64.

Wong

A.K.C.

and Chiu

D.K.Y.

, Synthesizing statistical knowledge from incomplete mixed-mode data, IEEE Transactions on Pattern Analysis and Machine Intelligence9 (1987), 796–805.

65.

Rousu

, Efficient Range Partitioning in Classification Learning, Ph. D. dissertation, Department of ComputerScience, University of Helsinki, Finland, 2001.

66.

Fayyad

U.M.

and Irani

K.B.

, Multi-interval discretization of continuous-valued attributes for classification, In: Proceedings of the 13th International Joint Conference on Artificial Intelligence, Chambery, France, 1993, pp. 1022–1027.

67.

Quinlan

J.R.

, Learning efficient classification procedures and their application to chess endgames, In: Machine Learning: An Artificial Intelligence Approach, Michalski

R.S.

, Carbonell

J.G.

and Mitchell

T.M.

, ed., Tioga Publishing Co., Palo Alto, California, USA, vol. I, 1983, pp. 463–482.

68.

Barron

, Rissanen

and Yu

, The Minimum Description Length principle in coding and modelling, IEEE Transactions on Information Theory44 (1998), 2743–2760.

69.

Civanlar

M.R.

and Trussell

H.J.

, Constructing membership functions using statistical data, Fuzzy Sets and Systems18 (1986), 1–13.

70.

Klawonn

and Keller

, Fuzzy clustering and fuzzy rules, In: Proceedings of the 7th International Fuzzy Systems Association World Congress, Prague, 1997, pp. 193–198.

71.

Bigot

, New Techniques for Handling Continuous Values in Inductive Learning, Ph. D. dissertation, Systems Engineering Division, Cardiff University, Cardiff, UK, 2003.

72.

Hou

T.-H.

, Liu

W.-L.

and Lin

, Intelligent remote monitoring and diagnosis of manufacturing processes using an integrated approach of neural networks and rough sets, Journal of Intelligent Manufacturing14 (2003), 239–253.

73.

Blake

C.L.

and Merz

C.J.

, UCI Repository of Machine Learning Databases. University of California, Department of Information and Computer Science, USA, 1998.

74.

Devijver

P.A.

and Kittler

, Pattern Recognition: A Statistical Approach, Prentice Hall, Englewood Cliffs, London, 1982.

75.

Efron

and Tibshirani

, An Introduction to the Bootstrap, Chapman & Hall, USA, 1993.