Finite mixture models for clustering multilevel data with multiple cluster structures

Abstract

Finite mixture models are useful tools for clustering two-way datasets within a sound statistical framework which can assess some important questions, such as how many clusters are there in the data. Models that can also be used for clustering multilevel data have been proposed, with the intent to produce clusterings of units at every level on the basis of all the available variables, considering the hierarchical structure of the dataset. This paper introduces a new class of mixture models for datasets with two levels that makes it possible to discover a clustering of level 2 units and different clusterings of level 1 units corresponding to different subsets of the variables (multiple cluster structures). This new class is obtained by adapting a mixture model proposed to identify multiple cluster structures in a data matrix to the multilevel situation. The usefulness of the new method is shown using simulated data and a real example.

Keywords

cluster analysis cluster structure mixture model model selection multilevel data

Get full access to this article

View all access options for this article.

References

Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In Petrov BN and Csaki BF (eds) Second international symposium on information theory. Budapest: Academiai Kiado , 267–81.

Belitskaya-Levy I (2006) A generalized clustering problem with application to DNA microarrays . Statistical Applications in Genetics and Molecular Biology, 5, Article 2.

Bozdogan H (1987) Model selection and Akaike’s information criterion (AIC): the general theory and its analytical extensions . Psychometrika, 52, 345–70 .

Bozdogan H (1994) Mixture-model cluster analysis using model selection criteria and a new informational measure of complexity. In Bozdogan H (ed.). Proceedings of the first US/Japan conference on the frontiers of statistical modelling: an informational approach. Boston: Kluwer Academic Publishers , 69–113.

Brusco MJ and Cradit JD (2001) A variable selection heuristic for k-means clustering . Psychometrika, 66, 249–70 .

Dempster AP , Laird NM and Rubin DB (1979) Maximum likelihood for incomplete data via the EM algorithm . Journal of the Royal Statistical Society, Series B, 39, 1–22 .

Desarbo WS , Carroll JD , Clark LA and Green PE (1984) Synthesized clustering: a method for amalgamating alternative clustering bases with differential weighting of variables . Psychometrika, 49, 57–78 .

Fraley C and Raftery AE (2002a) Model-based clustering, discriminant analysis and density estimation . Journal of the American Statistical Association, 97, 611–31 .

Fraley C and Raftery AE (2002b) MCLUST: software for model-based clustering, density estimation and discriminant analysis. Technical Report No. 415, Department of Statistics, University of Washington .

10.

Fraley C and Raftery AE (2003) Enhanced software for model-based clustering . Journal of Classification, 20, 263–86 .

11.

Friedman JH and Meulman JJ (2004) Clustering objects on subsets of attributes . Journal of the Royal Statistical Society B, 66, 815–49 .

12.

Galimberti G and Soffritti G (2007) Model-based methods to identify multiple cluster structures in a data set . Computational Statistics and Data Analysis, 52, 521–36 .

13.

Gnanadesikan R , Kettenring JR and Tsao SL (1995) Weighting and selection of variables for cluster analysis . Journal of Classification, 12, 113–36 .

14.

Gordon AD (1999) Classification, 2nd edition. Boca Raton: Chapman & Hall .

15.

Hubert L and Arabie P (1985) Comparing partitions . Journal of Classification, 2, 193–218 .

16.

Magidson J and Vermunt JK (2001) Latent class factor and cluster models, biplots and related graphical displays . Sociological Methodology, 31, 223–64 .

17.

McLachlan GJ and Krishnan T (1997) The EM algorithm and extensions. Chichester: Wiley .

18.

McLachlan GJ and Peel D (2000) Finite mixture models. Chichester: Wiley .

19.

Raftery AE and Dean N (2006) Variable selection for model-based clustering . Journal of the American Statistical Association, 101, 168–78 .

20.

Rand WM (1971) Objective criteria for the evaluation of clustering methods . Journal of the American Statistical Association, 66, 846–50 .

21.

Schwartz G (1978) Estimating the dimension of a model . The Annals of Statistics, 6, 461–64 .

22.

Soffritti G (2003) Identifying multiple cluster structures in a data matrix . Communications in Statistics: Simulation and Computation, 32, 1151–77 .

23.

Vermunt JK (2003) Multilevel latent class models . Sociological Methodology, 33, 213–39 .

24.

Vermunt JK (2008) Latent class and finite mixture models for multilevel data sets . Statistical Methods in Medical Research, 17, 33–51 .

25.

Vermunt JK and Magidson J (2005) Hierarchical mixture models for nested data structures. In Weihs C and Gaul W (eds). Classification: the ubiquitous challenge. Heidelberg: Springer , 176–83.