Intelligent Data Analysis: An International Journal

The volume of data being produced for administrative purposes is increasing rapidly. Data must be analysed in order to extract useful information to support decision making. The demand for evidence-based information means that the analysis must be conducted according to the principles of scientific research. Unfortunately, the massive second-hand data sets seem not to fit very well into the traditional methodological paradigm. A secondary data source imposes limitations on the formulation of a problem and concepts, because the measurement can only be based on existing data. The aim of this paper is to present a methodological framework for the utilisation of administrative registers in the creation of scientifically valid information. This is done by discussing fruitful methodological aspects encountered in the practical knowledge-discovery process. The ideas presented originate from many different fields, such as statistics, data mining and sociology. The emphasis lies on understanding connections between problem, data and analysis in the case of massive secondary administrative data sources.

The paper describes a system for spatial data mining illustrating its features by an application to spatial census data. Using census data for data mining includes specific challenges. Because of data privacy regulations, census data are generally available for analysis only in aggregated form. Primary data (responses of persons) are aggregated in many cross tabulations for small geographical units. Thus the target objects of secondary analysis are small areas (enumeration districts or wards). Any cell or marginal of a cross tabulation can be used as variable on these target objects. The target objects can be linked with other spatial objects (e.g. rivers, roads, railway lines) for spatial analyses. In this paper we discuss the special problems that occur for this type of aggregate data mining including spatial analyses. We show an application of SubgroupMiner, which is an advanced subgroup mining system supporting multirelational hypotheses, efficient data base integration, discovery of causal subgroup structures, and visualization based interaction options. The application explores if transportation lines (e.g. roads, railway lines) increase mortality for those persons that live near such objects because of a possible higher occurrence of some disease.

Census data mining has great potential both in business development and in good public policy, but still must be solved in this field a number of research issues. In this paper, problems related to the geo-referenciation of census data are considered. In particular, the accommodation of the spatial dimension in census data mining is investigated for the task of discovering spatial association rules, that is, association rules involving spatial relations among (spatial) objects. The formulation of a new method based on a multi-relational data mining approach is proposed. It takes advantage of the representation and inference techniques developed in the field of Inductive Logic Programming (ILP). In particular, the expressive power of predicate logic is profitably used to represent both spatial relations and background knowledge, such as spatial hierarchies and rules for spatial qualitative reasoning. The logical notions of generality order and of the downward refinement operator on the space of patterns are profitably used to define both the search space and the search strategy. The proposed method has been implemented in the ILP system SPADA (Spatial Pattern Discovery Algorithm). SPADA has been interfaced both to a module for the extraction of spatial features from a spatial database and to a module for numerical attribute discretization. The three modules have been used in an application to urban accessibility of a hospital in Stockport, Greater Manchester. Results obtained through a spatial analysis of geo-referenced census data are illustrated.

Politicians, planners and social scientists have an increasing need for tools clarifying the spatial distribution of relevant features. Special interest is in predicting changes in a what-if analysis: what would happen if we change some features in a specific way. To predict future developments requires a statistical model with inherent modelling uncertainty. In this paper we investigate Bayesian models which on the one hand are able to represent complex relations between geo-referenced variables and on the other hand estimate the inherent uncertainty in predictions. For solution the models require Markov-Chain Monte Carlo techniques.

The data descriptions of the units are called “symbolic” when they are more complex than standard ones, due to the fact that they contain internal variations and are structured. Symbolic data arise from many sources, for instance when summarizing huge Relational Data Bases by their underlying concepts. “Extracting knowledge” means obtaining explanatory results, and for this reason, “symbolic objects” are introduced and studied in this paper. They model concepts and constitute an explanatory output for data analysis. Moreover, they can be used to define queries of a Relational Data Base and propagate concepts between Data Bases. We define “Symbolic Data Analysis” (SDA) as the extension of standard Data Analysis to symbolic data tables as input in order to find symbolic objects as output. Any SDA is based on four spaces: the space of individuals, the space of concepts, the space of descriptions modelling individuals or classes of individuals, the space of symbolic objects modelling concepts. New problems arise from these four spaces, such as the quality, robustness and reliability of the approximation of a concept given by a symbolic object, the symbolic description of a class, the consensus between symbolic descriptions, and so on. In this paper we give an overview of recent developments in SDA. We briefly describe some SDA tools and methods and, in particular, we describe some dissimilarity methods for symbolic objects which are central to the majority of symbolic data analysis methods. Finally, we introduce the software prototype, developed by 17 teams from nine countries involved in the SODAS EUROSTAT project.

In the competitive environments, in which all sorts of organisations move it is of utmost importance to have information about clients. Public databases offer information about households and families. However, the non-crossed and non-georeferenced format of these databases often makes it difficult to extract typologies and information.

There are only two public databases from which to get information at the household or family level in Spain: Population and Housing Censuses, which provide aggregated and georeferenced information, and the Family Expenditure Surveys, which provide information on household consumption, both published by the National Statistics Institute. The two databases cannot be directly cross-referenced, because the Family Expenditure Surveys offer a detailed description of the families, whereas the Census provides the same data but aggregated without cross-references.

In this paper, we define a procedure for cross-referencing these DBs and calculating the economic household indexes for Spanish censal sections that define the average quarterly economic behaviour of the households located in each censal section. The necessary analysis procedure is based on neural networks and provides an estimate of the trend in these indexes over a series of years. The procedure can be easily extrapolated to similar problems with official data sources from other countries.