Intelligent Data Analysis: An International Journal

Holonic multi-agent system (HMAS) offers a promising approach to model complex systems. HMAS is based on self-similar entities defined in a holarchical organization. Although some models and frameworks have been proposed for holonic systems, there is no general reinforcement learning method that can be easily implemented in HMAS. This paper presents a reinforcement learning method for HMAS. The holons in different levels have direct effect on learning process of each other through communication. For hierarchical communications between holons, abstract data flows are defined that are used for state estimation, action selection and reward calculation. The proposed learning method includes a self-similar structure in which the learning processes of the holons are independent of their actual positions in the holarchy. A real-world application is also used to show that how the holons can be implemented in practice. Experimental results show that the proposed holonic reinforcement learning method improves the performance.

Bayesian Multi-nets (BMNs) are a special kind of Bayesian network (BN) classifiers that consist of several local Bayesian networks, one for each predictable class, to model an asymmetric set of variable dependencies given each class value. Deterministic methods using greedy local search are the most frequently used methods for learning the structure of BMNs based on optimizing a scoring function. Ant Colony Optimization (ACO) is a meta-heuristic global search method for solving combinatorial optimization problems, inspired by the behavior of real ant colonies. In this paper, we propose two novel ACO-based algorithms with two different approaches to build BMN classifiers: ABC - Miner l m n and ABC - Miner g m n . The former uses a local learning approach, in which the ACO algorithm completes the construction of one local BN at a time. The latter uses a global approach, which involves building a complete BMN classifier by each single ant in the colony. We experimentally evaluate the performance of our ant-based algorithms on 33 benchmark classification datasets, where our proposed algorithms are shown to be significantly better than other commonly used deterministic algorithms for learning various Bayesian classifiers in the literature, as well as competitive to other well-known classification algorithms.

Evolutionary algorithms are often applied to solve multi-objective optimization problems. Such algorithms effectively generate solutions of wide spread, and have good convergence properties. However, they do not provide any characteristics of the found optimal solutions, something which may be very valuable to decision makers. By performing a post-analysis of the solution set from multi-objective optimization, relationships between the input space and the objective space can be identified. In this study, decision trees are used for this purpose. It is demonstrated that they may effectively capture important characteristics of the solution sets produced by multi-objective optimization methods. It is furthermore shown that the discovered relationships may be used for improving the search for additional solutions. Two multi-objective problems are considered in this paper; a well-studied benchmark function problem with on a beforehand known optimal Pareto front, which is used for verification purposes, and a multi-objective optimization problem of a real-world production system. The results show that useful relationships may be identified by employing decision tree analysis of the solution sets from multi-objective optimizations.

In this paper, a new co-training algorithm based on modified Fisher's Linear Discriminant Analysis (FLDA) is proposed for semi-supervised learning, which only needs a small set of labeled samples to train classifiers and is thus very useful in applications like brain-computer interface (BCI) design. Two classifiers, one aiming to maximize the normalized between-class diversity and the other to minimize the normalized within-class diversity, are proposed for the co-training process. A method with a confidence criterion is also proposed for selecting unlabeled data to expand training data set. The co-training algorithm is compared with a static FLDA method and a FLDA based on self-training algorithm on the data set 2a for BCI Competition IV, with statistical significance test. Experimental results show that the new co-training algorithm outperformed the other two methods and its average classification accuracy was improved iteration by iteration, demonstrating the convergence of the co-training process.

In this paper we address the problem of clustering interval data, adopting a model-based approach. To this purpose, parametric models for interval-valued variables are used which consider configurations for the variance-covariance matrix that take the nature of the interval data directly into account. Results, both on synthetic and empirical data, clearly show the well-founding of the proposed approach. The method succeeds in finding parsimonious heterocedastic models which is a critical feature in many applications. Furthermore, the analysis of the different data sets made clear the need to explicitly consider the intrinsic variability present in interval data.

Financial time series analysis is crucial to a successful assets allocation. Applying a matrix factorization technique can generate genuine grouping knowledge for the allocation of assets according to their association with a number of underlying bases. A constrained nonnegative matrix factorization NSMF is proposed to incorporate three penalties in order to compute a solution which can maximize between-base disjointness and volatility difference. A series of quantitative measures are designed for evaluation of bases and their volatility. Different types of real data are used in the experiments and compared regarding clustering consistency. Experimental analysis of historical prices of US blue chip stocks indicates that NSMF is superior to agglomerative clustering and independent component analysis and NSMF can extract bases with a higher discrepancy of volatility. The non-stochasticity constraint increases the dissimilarity of bases and it governs basis deviation over smoothness and sparseness. The clustering results of bases and persistent pairs, which are gained from NSMF, can consolidate our understanding of financial data properties and they provide meaningful knowledge in the construction of a well risk-balanced and diversified portfolio.

A batch-mode active learning technique taking advantage of the cluster assumption was proposed. It focused on binary classification tasks adopting SVM (support vector machine). In each active learning iteration, unlabeled instances in the SVM margin were first grouped into two clusters. Then from each cluster, points most similar to the other cluster were selected for labeling. Such points lying near the boundary between clusters were expected to become support vectors in the final classification model with high probability. The clustering process was performed in the same kernel space as SVM. With semi-supervised K-medoids, labeled instances were also used to improve the clustering performance. Experiments showed that the proposed method was efficient and robust (to poor initial samples).

During the last years, many works focused on the exploitation and the extraction of rare patterns. In fact, these patterns allow conveying knowledge on rare and unexpected events. They are hence useful in several application fields. Nevertheless, a main moan addressed to rare pattern extraction approaches is, on the one hand, their high number and, on the other hand, the low quality of several mined patterns. The latter can indeed not present strong correlations between the items they contain. In order to overcome these limits, we propose to integrate the correlation measure bond aiming at only mining the set of rare patterns fulfilling this measure. A characterization of the resulting set, of rare correlated patterns, is then carried out based on the study of constraints of distinct types induced by the rarity and the correlation. In addition, based on the equivalence classes associated to a closure operator dedicated to the bond measure, we introduce new concise representations of rare correlated patterns as well as the derivation process of the generic bases of the rare correlated association rules. We then design the RcprMiner algorithm allowing an efficient extraction of the proposed concise representations. Carried out experimental studies highlight the very encouraging compactness rates offered by the proposed concise representations and show the good performance of the RcprMiner algorithm.

Information security is an important and growing need. The most common schemes used for detection systems include pattern- or signature-based and anomaly-based. Anomaly-based schemes use a set of metrics, which outline the normal system behavior and any significant deviation from the established profile will be treated as an anomaly. This paper contributes with an anomaly-based scheme that monitors the bandwidth consumption of a subnetwork, at the Universidad Michoacana, in Mexico. A normal behavior model is based on bandwidth consumption of the subnetwork. The presence of an anomaly indicates that something is misusing the network (viruses, worms, denial of service, or any other kind of attack). This work also presents a scheme for an automatic architecture design and parameters optimization of Hidden Markov Models (HMMs), based on Evolutionary Programming (EP). The variables to be used by the HMMs are: the bandwidth consumption of network (IN and OUT), and the associated time where the network activity occurs. The system was tested with univariate and bivariate observation sequences to analyze and detect anomaly behavior. The HMMs, designed and trained by EP, were compared against semi-random HMMs trained by the Baum-Welch algorithm. On a second experiment, the HMMs, designed and trained by EP, were compared against HMMs created by an expert user. The HMMs outperformed the other methods in all cases. Finally, we made the HMMs time-aware, by including time as another variable. This inclusion made the HMMs capable of detecting activity patterns that are normal during a period of time but anomalous at other times. For instance, a heavy load on the network may be completely normal during working times, but anomalous at nights or weekends.

Weighted itemset mining has been a widely studied topic in data mining. The reason is that weighted itemset mining considers not only the occurrence of items in transactions but also the individual importance of items. The traditional upper-bound model can be used to handle the weighted itemset mining problem, but a large number of candidates are generated by the model. This work thus presents an improved model to enhance the effectiveness of reducing unpromising candidates. Besides, an effective strategy, projection-based pruning, is proposed as well to tighten upper-bounds of weighted supports for itemsets in the mining process, thus reducing the execution time further. Through a series of experimental evaluation, the results on synthetic and real datasets show that the proposed approach has good performance in both pruning effectiveness and execution efficiency under various parameter settings when compared to some other approaches.

With the development of new technologies more and more information is stored in log files. Analyzing such logs can be very useful for the decision maker. One of the probably best known example is the Web log file analysis where lots of efficient tools have been proposed to extract the top-k accessed pages, the best users or even the patterns describing the behaviors of users on a Web site. These tools take advantages of the well-formed structures of the data. Unfortunately, logs files from the industrial world have very heterogeneous complex structures (e.g., tables, lists, data blocks). For experts, analyzing logs to find messages helping to better understand causes of a failure, if a problem have already occurred in the past or even knowing the main consequences of a failure is a hard, tedious, time-consuming and error-prone task. There is thus a need for new tools helping the experts to easily recognize the appropriate part in logs. Passage retrieval methods have proved to be very useful for extracting relevant parts in documents. In this paper we propose a new approach for automatically split logs files into relevant segments based on their logical units. We characterize the complex logical units found in logs according to their syntactic characteristics. We also introduce the notion of generalized vs-grams which is used to automatically extract the syntactic characteristics of special structures found in log files. Conducted experiments are performed on real datasets from the industrial world to demonstrate the efficiency of our proposal on the recognition of complex logical units.

Incremental learning is a learning algorithm that can get new information from new training sets without forgetting the acquired knowledge from the previously used training sets. In this paper, an incremental learning algorithm based on ensemble learning is proposed. Then, an application of the proposed algorithm for spam filtering is discussed. The proposed algorithm called incremental RotBoost, assumes the environment is stationary. It trains new weak classifiers for newly arriving data, which are added to the ensemble of classifiers. To evaluate the performance of the proposed algorithm, several computer experiments are conducted. The results of computer experiments show the ability of our proposed algorithm for different tasks in the incremental learning. The results also demonstrate that the proposed algorithm can learn incrementally, and it can learn new classes, as well.