Intelligent Data Analysis: An International Journal

Conventional clustering algorithms are designed for a single independent dataset, i.e. Fuzzy C-Means (FCM) clustering algorithm. In the real world, a dataset is independent of other datasets but sometimes can be cooperative with others by exchanging information, such as the relationship between subsidiary companies. We should therefore consider the influence from other relative collaborative datasets while performing clustering learning under such collaborative circumstances. In this paper, three different collaborative models are discussed and new correct methods are proposed to quantitatively measure such collaboration between datasets, i.e. information gain. The corresponding collaborative clustering algorithms are presented accordingly and the theoretical analysis shows that the new cooperative clustering algorithms can finally converge to a local minimum. Experimental results demonstrate that the clustering structures obtained by new cooperative algorithms are different from those of conventional algorithms for the consideration of collaboration and the performances of these collaborative clustering algorithms can be much better than those conventional “single” clustering algorithms under the cooperating circumstances.

A large volume of transaction data is generated everyday in a number of applications. These dynamic data sets have immense potential for reflecting changes in customer behaviour patterns. One of the strategies of data mining is association rule discovery which correlates the occurrence of certain attributes in the database leading to the identification of large data itemsets. This paper seeks to generate large itemsets in a dynamic transaction database using the principles of Genetic Algorithms. Intra Transactions, Inter Transactions and Distributed Transactions are considered for mining Association Rules. Further, we analyze the time complexities of single scan technique DMARG (Dynamic Mining of Association Rules using Genetic Algorithms), with Fast UPdate (FUP) algorithm for intra transactions and E-Apriori for inter transactions. Our study shows that the algorithm DMARG outperforms both FUP and E-Apriori in terms of execution time and scalability, without compromising the quality or completeness of rules generated.

Regression problems occur in many data analysis applications. The aim of regression is to approximate a function from which measurements were taken. When considering a regression problem, we have to take a number of aspects into account: How noisy the data are, whether they cover the domain sufficiently in which we want to find the regression function and what kind of regression function we should choose. However, the underlying assumption is always that the data actually are (noisy) samples of a single function. In some cases, this might not be true. For instance, when we consider data from a technical process that is controlled by human operators, these operators might use different strategies to reach a particular goal. Even a single operator might not stick to the same strategy all the time. Thus, the dataset containing a mixture of samples from different strategies, do not represent (noisy) samples from a single function. Therefore, there exists an ambiguity of selecting data from a large dataset for regression problems to fit a single model.

In this paper, we suggest an approach using a modified mountain method (MMM) to select data from a jumble of large data samples that come from different functions, in order to cope with the ambiguities in the underlying regression problem. The proposed method may also serve to identify the best local (approximation) function(s). These are determined using a weighted regression analysis method. The proposed methodology is explained with a one-dimensional problem, a single input single output system, and later performance of the proposed approach is analysed with artificial data of a two-dimensional case study.

Exception mining in large datasets is an important task in traditional data mining with numerous applications in credit card fraud detection, weather prediction, intrusion detection, and cellular phone cloning fraud detection; among other applications. Sifting through the dynamic, unstructured, and ever-growing web data for outliers is more challenging than finding outliers in numeric datasets. Interestingly, existing outlier mining algorithms are restricted to finding outliers in numeric datasets leaving web outlier mining as an open research issue. Web outliers are web data that show significantly different characteristics than other web data taken from the same category. Although the presence of web outliers appears obvious, algorithms for mining them are currently unavailable. Secondly, traditional outlier mining algorithms designed solely for numeric datasets cannot be used on web datasets because they typically contain multimedia. This paper establishes the presence of outliers on the web called web outliers and proposes a general framework for mining them. A web outlier taxonomy is reported that supports the development of content-specific algorithms for mining web outliers. Finally, we propose the WCO-Mine algorithm for mining web content outliers. Experimental results demonstrate that WCO-Mine is capable of finding web outliers from web datasets.

The poor quality of a training dataset can have untoward consequences in software quality estimation problems. The presence of noise in software measurement data may hinder the prediction accuracy of a given learner. A filter improves the quality of training datasets by removing data that is likely noise. We evaluate the Ensemble Filter against the Partitioning Filter and the Classification Filter. These filtering techniques combine the predictions of base classifiers in such a way that an instance is identified as noisy if it is misclassified by a given number of these learners. The Partitioning Filter first splits the training dataset into subsets, and different base learners are induced on each subset. Two different implementations of the Partitioning Filter are presented: the Multiple-Partitioning Filter and the Iterative-Partitioning Filter. In contrast, the Ensemble Filter uses base classifiers induced on the entire training dataset. The filtering level and/or the number of iterations modify the filtering conservativeness: a conservative filter is less likely to remove good data at the expense of retaining noisy instances. A unique measure for comparing the relative efficiencies of two filters is also presented. Empirical studies on a high assurance software project evaluate the relative performances of the Ensemble Filter, Multiple-Partitioning Filter, Iterative-Partitioning Filter, and Classification Filter. Our study demonstrates that with a conservative filtering approach, using several different base learners can improve the efficiency of the filtering schemes.