Abstract
Data resampling techniques are increasingly used for assigning confidence to clustering results, in particular for tumor class discovery based on genomic data. One factor that determines the success of this approach is the capability of a resampling scheme to simulate the sampling variability by using the information of sparse sample data. We present a method for evaluating resampling performance based on model simulations. This method was applied to results of 40 cluster validity indices and one partition stability index obtained from 12 clustering procedures including different distance measures. The results were generated for benchmark data of five statistical models, gene expression profiles of three multi-class tumor sample data sets, four data sets of the widely used UCI repository, and spatiotemporal neuroimaging data. The results suggest a ranking of the three resampling techniques analyzed: perturbation (adding noise to the data) was more effective than subsampling and both clearly outperformed the bootstrapping technique in the detection of correct clustering consensus results. Due to the consistency of the results this ranking may have impact on the selection of a resampling method for the cluster validation in future studies. Moreover, intelligent control of the resampling parameters can increase the achievable confidence in clustering results.