Improving the measurement of material deprivation at the European Union level

Abstract

In June 2010, European Union (EU) Heads of State and Government adopted a social inclusion target as part of the new ‘Europe 2020 Strategy’: to lift at least 20 million people in the EU from the risk of poverty and exclusion by 2020. One of the three indicators used to monitor progress towards this target is the EU indicator of severe material deprivation (MD). A main limitation of this indicator is the weak reliability of some of the items it is based on. For this reason, a thematic module on MD was included in the 2009 wave of the EU Statistics on Income and Living Conditions (EU-SILC) survey. This article assesses the 2009 EU-SILC MD data and proposes an analytical framework for developing robust EU MD indicators. It carries out a systematic item by item analysis at both EU and country levels to identify the MD items which satisfactorily meet suitability, validity, reliability and additivity criteria across the EU. This approach has resulted in a proposed 13-item MD indicator covering some key aspects of living conditions which are customary across the whole EU covering a broad range of basic (food, clothes, shoes, etc.) as well as social (Internet, regular leisure activities, etc.) necessities.

Keywords

EU-SILC Europe 2020 material deprivation poverty social indicators

Introduction

Since 2000, European Union (EU) cooperation in the field of social policy is organised around a set of key objectives commonly agreed by all EU Member States and the European Commission (Council of the European Union, 2011). In order to monitor progress towards these EU social objectives, EU Heads of State and Government endorsed a number of methodological principles for the construction of the EU indicators for social inclusion (the so-called ‘Laeken indicators’) and, more broadly, the EU social indicators (Atkinson et al., 2002; Marlier et al., 2007; Social Protection Committee, 2001). This methodological framework specifies that EU social indicators should (a) capture the essence of the problem and have a clear and accepted normative interpretation, (b) be robust and statistically validated, (c) provide a sufficient level of cross-country comparability, (d) be timely and susceptible to revision and (e) be responsive to policy interventions but not subject to manipulation (European Commission, 2009).

In 2009, two material deprivation (MD) indicators were adopted by all 27 EU Member States and the European Commission in order to complement the Laeken indicators on income poverty and to better reflect differences in actual standards of living across the EU (especially since the 2004 and 2007 enlargements (see Appendix 1)). Based on the work by Guio (2009) and the limited information available from the EU Statistics on Income and Living Conditions (EU-SILC) dataset, they are defined on the basis of nine items:¹

Coping with unexpected expenses;

One week’s annual holiday away from home;

Avoiding arrears (in mortgage or rent, utility bills or hire purchase instalments);

A meal with meat, chicken, fish or vegetarian equivalent every second day;

Keeping the home adequately warm;

A washing machine;

A colour TV;

A telephone;

A personal car.

The first indicator (the EU MD rate) is the proportion of people living in households who cannot afford at least three of these items. The second indicator, which measures the intensity of MD, is the average number of items lacked by those identified as deprived.

Since June 2010, the importance of MD indicators has grown significantly as a result of the adoption of the Europe 2020 Strategy on smart, sustainable and inclusive growth, with its five ‘headline targets’ to be achieved by 2020 (Marlier et al., 2010). These headline objectives include a specific and time-bound social inclusion target for the EU as a whole: ‘promoting social inclusion, in particular through the reduction of poverty, by aiming to lift at least 20 million people out of the risk of poverty and social exclusion in the EU’.

The Europe 2020 social inclusion target is measured using three indicators:

The EU ‘at-risk-of-poverty’ indicator, that is, people living in households whose net equivalised income is below 60 percent of the national median equivalised household income (the equivalence scale is the Organisation for Economic Co-operation and Development (OECD) modified scale);²

An indicator of ‘severe MD’ (based on the EU MD indicator, with the threshold raised from three to four out of the nine items);

A measure of ‘very low household work intensity’ also referred to as ‘(quasi-)joblessness’ (i.e. people aged 0–59 years living in households where, on average, adult members aged 18–59 years have worked less than 20% of their total work potential during the income reference period (i.e. the year prior to the survey in most countries))

When the target was adopted, EU countries agreed that, in the context of the 2014–2015 mid-term review of Europe 2020, the target and its three sub-indicators should be revised and improved measures of MD should be worked on.

The main limitation of the current EU MD index is the weak reliability of some of its items, particularly in ‘richer’ EU Member States. This is a primary reason why a thematic module on deprivation was included in the 2009 wave of EU-SILC. This module was developed by an EU Task Force on MD, set up in 2007 by Eurostat (the statistical office of the EU), drawing upon the deprivation questions from the British 1999 Poverty and Social Exclusion (PSE) survey (Gordon et al., 2000; Pantazis et al., 2006) and the Irish survey used for building the Irish ‘consistent’ poverty measure (Maître et al., 2006).

The purpose of this article is to identify deprivation items among those collected by the 2009 wave of EU-SILC that would allow the construction of an improved robust aggregate MD index/indicator for use by all EU Member States and the European Commission. For this, we propose a framework to test all the 33 relevant deprivation items included in the 2009 wave of EU-SILC – that is, both ‘core items’ (i.e. those collected each year in the core part of EU-SILC) and ‘module items’ (collected in the 2009 thematic MD module). Online Appendix presents the list of 33 items analysed.

The article starts with a description of the conceptual and methodological aspects of this framework. It then considers, in turn, the four criteria we use to ensure a robust selection of MD items among the 33 items available (suitability, validity, reliability and additivity). Finally, it presents the resulting 13-item list and concludes.

Conceptual framework

In building MD indicators, what we are looking for are measures covering some key aspects of living conditions which appear to be customary across the whole EU and from which some people are excluded due to a lack of resources, that is, a concept of MD which is consistent with Peter Townsend’s (1979) theory of relative deprivation:

Poverty can be defined objectively and applied consistently only in terms of the concept of relative deprivation. […] Individuals, families and groups in the population can be said to be in poverty when they lack the resources to obtain the type of diet, participate in the activities and have the living conditions and amenities which are customary, or at least widely encouraged or approved, in the societies to which they belong. Their resources are so seriously below those commanded by the average individual or family that they are, in effect, excluded from ordinary living patterns, customs or activities. (p. 31)

Thus, Townsend defines ‘poverty’ as a lack of command of sufficient resources over time, and ‘deprivation’ is an outcome of poverty. In addition, deprivation is a relative phenomenon which encompasses both a lack of material goods and social activities:

Deprivation takes many different forms in every known society. People can be said to be deprived if they lack the types of diet, clothing, housing, household facilities and fuel and environmental, educational, working and social conditions, activities and facilities which are customary, or at least widely encouraged and approved, in the societies to which they belong. (Townsend, 1987: 126)

Methodological framework

The methods used in this article draw extensively on the 1999 PSE survey deprivation indicator construction methodology (Gordon et al., 2000; Pantazis et al., 2006). This PSE methodology has been used to develop robust and comparable measures of deprivation for several British poverty surveys (Fahmy et al., 2011), in Northern Ireland (Hillyard et al., 2003) and in other countries. An important aspect of this methodology is that it facilitates the identification and selection of robust deprivation items from the initial list of available items.

We have ensured this robust selection of items at both national and EU levels, by considering four aspects:

The suitability of each deprivation item: We check that citizens in the different Member States (as well as the different population sub-groups within each Member State) consider it necessary to have ‘the living conditions and amenities which are customary, or at least widely encouraged or approved, in the societies to which they belong’ (Townsend, 1979: 31). Here, ‘suitability’ is understood as a measure of face validity among the EU population.

The validity of individual items: We ensure that each item exhibits statistically significant relative risk ratios with independent variables known to be correlated with MD (health problems and ‘lack of resources’ (see section ‘Conceptual framework’) as measured by low income and difficulty in making ends meet).

The reliability of the MD scale: We assess the reliability of the scale as a whole, that is, its internal consistency using Classical Test Theory (CTT). We complement this with Item Response Theory (IRT) that provides additional information on the reliability of each individual item in the scale.

The additivity of items: We test that someone with an MD indicator score of ‘2’ is suffering from more severe MD than someone with a score of ‘1’, that is, that the MD indicator’s components add up. Additivity is measured for the MD items that successfully passed the suitability, validity and reliability tests.

In each country, the deprivation items that successfully pass these four steps can thus be considered to be suitable, valid, reliable and additive candidates for being aggregated into a MD indicator. At EU level, we have considered that an item does not pass a test (suitability, validity, reliability or additivity) as soon as it fails in 3 of 26 EU Member States (Sweden was excluded from this criterion because of significant missing data for all MD module items).³ For EU indicators to be fit for purpose (i.e. to ensure that they will provide robust tools for assessing and monitoring MD not only at EU level but also at individual Member State’s level), a key methodological principle they have to meet (see above) is that they should exhibit ‘a sufficient level of cross-country comparability’.

A measure which is suitable, reliable, valid and additive in each EU country and also for the EU as a whole is likely to be measuring the same latent construct across countries.⁴ We have further assessed the extent of cross-national measurement equivalence using exploratory factor analysis, cluster analysis, multidimensional scaling (MDS) and correspondence analysis in order to compare the dimensional structure of the items across countries (for EU results, see Guio et al., 2012).⁵ A clear dimensional structure is evident among the full 33 potential deprivation items across EU Member States. The five environmental deprivation items form a distinct dimension as do the two access to services variables (bank/post office and public transport). The remaining 26 deprivation variables are more closely related and group together although sub-dimensional structure is evident, which could be used to further sub-divide these variables. When new data become available, confirmatory factor analysis modelling will be run to explore further configural, metric and scalar equivalence.

Suitability

The 1983 Breadline Britain study pioneered this ‘consensual’ or ‘perceived deprivation’ approach to measuring poverty by investigating the public’s perceptions of minimum needs:

This study tackles the question ‘how poor is too poor?’ by identifying the minimum acceptable way of life for Britain in the 1980s. Those who have no choice but to fall below this minimum level can be said to be ‘in poverty’. This concept is developed in terms of those who have an enforced lack of socially perceived necessities. This means that the ‘necessities’ of life are identified by public opinion and not by, on the one hand, the views of experts or, on the other hand, the norms of behaviour per se. (Mack and Lansley, 1985)

Mack and Lansley defined necessities as possessions and activities that every family (or person) should be able to afford and that nobody should have to live without. An item regarded as necessary by at least 50 percent of respondents was seen as a ‘socially perceived necessity’.

An EU-wide Eurobarometer survey on the perception of poverty and social exclusion was carried out in 2007 (Dickes et al., 2010; Guio et al., 2009; TNS, 2007), which showed that while the proportion of people considering each item to be a necessity varies across Member States, there is little variation between countries in the rank order of deprivation items, that is, in general the items which are ranked highly in one country will also be ranked highly in all other countries, and vice versa (low-ranked items in one country are low ranked in all other countries).

There can also be differences between what people consider necessary for the whole of society compared to their own needs, so we examined the actual behaviour of people using the 2009 EU-SILC data. Perry (2002) argues that the degree of ‘importance’ of each item, at EU and country levels, can be measured by the proportion of people ‘wanting’ an item (i.e. the proportion of people who have the item PLUS the proportion of people who would like it but cannot afford it). A high proportion of people having and/or wanting an item provides a measure of the ordinary living patterns, customs or activities, which is a key criteria in Townsend’s sociological definition of poverty (see above).

Only EU-SILC questions that distinguish between a ‘simple’ lack of an item (people who do not possess/have access to the item) and an ‘enforced’ lack of an item (people would like to possess/have access to an item but cannot afford it) could be analysed in this way (i.e. durables in the core part of EU-SILC and the majority of items collected in the module). For these items, there were three answer categories:

Have the item;

Do not have the item because cannot afford it;

Do not have the item for any other reason.

At EU level, all items were either possessed or ‘wanted’ by more than 70 percent of the population, except adult leisure activity. In fact, most items were wanted by a large majority of the population (at least 90%). The failure of adult leisure activity to attain the 70 percent threshold needs to be interpreted cautiously as with social activities, the ‘No, for any other reason’ category is likely to include people who may want to do this activity but are prevented from doing so by other constraints beyond a lack of income, for example, poor health, lack of time due to caring responsibilities or work, lack of transport, problem of physical access and feeling unwelcome (see Gordon et al., 2000; Pantazis et al., 2006).

Analyses for each of the 27 EU Member States and by social group (in order to test the ‘homogeneity of preferences’) show that most items are wanted by large majorities. Even for the more problematic items/countries, the proportion of people ‘wanting’ the item is not lower than 60 percent. This provides face validity support for the use of the same set of MD items across the EU.

Validity

All items in a deprivation index/indicator need to be valid measures of deprivation. An individual MD item can be considered to be valid if it exhibits statistically significant relative risk ratios with a set of independent variables known to be correlated with the latent construct of deprivation. We tested this by running binary logistic regressions for each MD item (dependent variable) against independent variables known to be correlated with MD.

Three indicators of validity were used:

At-risk-of-poverty, which is known to be closely related to MD although the overlap is far from perfect for a variety of reasons (Fusco et al., 2010; Gordon, 2000; Halleröd et al., 2006). Although the cross-sectional association between low income and deprivation is often lower than might be expected (Perry, 2002), there is a long tradition of using this association to validate deprivation indicators. Both Peter Townsend (1979) and Mack and Lansley (1985) used the size of the correlation between income and deprivation to select their items.

Economic strain (‘great difficulties’ or ‘difficulties’ with making ends meet), which is often used as a measure of financial stress, is closely related to MD (Fahmy and Gordon, 2005; Nolan and Whelan, 2011). It would be expected from Townsend’s theory of relative deprivation and Mack and Lansley’s concept of ‘consensual poverty’ that someone who is ‘deprived’ would also be more likely to consider themselves to be subjectively poor (Bradshaw and Finch, 2003).

Self-reported health status (‘limitations’ or ‘strong limitations’ in activities because of health problems, after controlling for age and gender effects). Many scientific studies have shown that people suffering from deprivation have a higher probability of developing health problems and that there is a strong association between living/growing up in poverty and deprivation and having worse health. The relationship can go in both directions, that is, health problems can also cause poverty. However, while ill health may sometimes be a cause of poverty/deprivation, the available evidence suggests that in EU countries, ‘poor’ people are much more likely to become ‘sick’ than ‘sick’ people are to become ‘poor’. The direction of causality is strongly weighted towards deprivation/poverty causing ill health rather than ill health causing poverty/deprivation (Commission on Social Determinants of Health, 2008; Gordon et al., 1999; Leon and Walt, 2001).

Here, we consider that an item has validity problems if the results of the logistic regressions are not significant in two out of our three validity tests. We reject an item as ‘invalid’ if it has validity problems in more than 2 out of the 26 Member States (Sweden not being included; see above).

These analyses involved over 2000 tests of validity (i.e. 30 items × 26 countries × 3 validity indicators = 2340). Using the less-than-5 percent significance level, it would be expected that 1 in 20 tests will yield incorrect results due to random chance, hence the need not to reject a deprivation item as invalid if it fails a single test in a single country.

Table 1 summarises the results from the validity tests.⁶ Two items are rejected on validity grounds: ‘a colour TV’ (Cyprus (CY), Greece (EL), Ireland (IE), Luxembourg (LU)) and ‘a telephone’ (CY, Denmark (DK), LU, The Netherlands (NL)). All the other items appear valid, that is, they have either no validity problems in any Member State or they have validity problems in no more than 2 out of the 26 Member States. All the items that pass the validity tests are valid to measure MD across the whole EU as they fulfil the demanding criterion we have imposed – that is, to be valid in at least 24 Member States. This means that all these deprivation items exhibit a clear social gradient with income, economic strain and health in virtually all EU countries.

Table 1.

Number of Member States (MSs) with validity problems (i.e. non-significant relation between a given item and at least two of the following: income poverty, subjective poverty and health problems).

	Income poverty	Subjective poverty	Health	MSs with two validity problems (out of three)
The person cannot afford (but would like to have (enforced lack))
A mobile phone	0	1	1
Some new clothes (not second-hand)	0	0	0
Two pairs of shoes	0	1	0
Spend a small amount of money each week on oneself	0	0	0
Meeting with friends/family for a drink/meal monthly	0	0	0
Leisure activities	0	0	0
The household’s dwelling suffers from
Lack of basic amenities (toilet, hot running water, bath)	2	3	3	MT
Shortage of space	4	0	11
Darkness	3	0	0
Leaky roof, damp, etc.	0	0	0
Litter lying around	8	0	2	MT
Vandalism	9	0	6	MT
Noise	5	1	2
Pollution	8	0	5	IE, MT
Crime	8	1	3	AT
Overcrowding	1	0	8
High housing costs	0	0	12
The household cannot afford
To replace worn-out furniture	0	0	0
Meat, chicken, fish (or vegetarian equivalent) every second day	0	0	0
To face unexpected expenses	0	0	0
To keep home adequately warm	0	0	0
One week annual holiday away from home	0	0	0
Avoiding arrears	0	0	0
A washing machine (enforced lack)	0	0	10
A colour TV (enforced lack)	3	3	10	CY, EL, IE, LU
A telephone (enforced lack)	5	3	11	CY, DK, LU, NL
A computer/Internet access (enforced lack)	0	0	4
A personal car (enforced lack)	0	0	0
The household has a (very) difficult access to
Public transport	7	4	1	AT, MT
Postal/banking services	5	2	3	FR, LU

Source: EU-SILC 2009 cross-sectional data, Users’ database – August 2011, authors’ computation.

CY: Cyprus; EL: Greece; IE: Ireland; LU: Luxembourg; DK: Denmark; NL: The Netherlands; MT: Malta; AT: Austria; FR: France; EU-SILC: European Union Statistics on Income and Living Conditions.

The figures are in bold for items with at least two validity problems (out of three), i.e. items which failed the validity test.

Reliability

Reliability was tested using CTT (Nunally, 1978) and IRT models. The Cronbach’s alpha statistic measures the internal consistency of a scale, that is, how closely related a set of items are as a group. A ‘high’ value of alpha is often used as evidence that the set of items measures an underlying (or ‘latent’) construct. An alpha of 0.70 or higher is considered as ‘satisfactory’ in most social science research situations (Nunally, 1978).

We tested the reliability of the deprivation items using all MD items (the full set) related to the whole population (0+), that is, including those items that failed our validity tests. We identified which items if omitted (one by one) would increase the reliability of the deprivation index (i.e. increase Cronbach’s alpha – analysis performed at both country and EU levels). In line with our validity tests, the criterion we applied is that an item is considered unreliable if it is unreliable in three countries or more.

The 14 items which failed our reliability test are as follows:

Some basic durables (TV, telephone, washing machine) and basic amenities;

The two items related to problems of accessibility, that is, access to postal/banking services and to public transport;

Items which measure local environment problems (crime, noise, pollution, litter lying around, vandalism);

Three items related to housing: overcrowding, dark dwelling and high housing costs.

It should be noted that this pattern is very consistent across countries. The number of reliable items per country does not vary much (between 21 (Poland, PL) and 27 (Belgium [BE], CY, Finland [FI]); 22 at EU level), and the reliability problems tend to occur repeatedly with the same items.

CTT provides information on the reliability of an MD scale/index as a whole. This can usefully be complemented with IRT that provides additional information on the reliability of each individual item in the scale/index. IRT, also known as Latent Trait Analysis, is a set of statistical models which describes the relationship between a person’s response to questionnaire items and an unobserved latent trait such as knowledge of biology, level of happiness or amount of MD. IRT is often used for the selection of questions in educational assessment and for psychological testing. It has also been used for developing measures of poverty (e.g. Cappellari and Jenkins, 2007; Fusco and Dickes, 2008; Martini and Vanin, 2013; Raileanu Szeles and Fusco, 2013).

The IRT model assumes that ‘deprivation’ is an unobservable latent trait which cannot be measured directly, as it is a concept rather than a physical entity. It is, however, assumed that this concept of ‘deprivation’ can be measured indirectly using social survey questions about respondent’s ability to afford certain consumer durables and to participate in social activities.

It is assumed that the items included in the deprivation index measure only one latent trait (unidimensionality). Unidimensional IRT models are robust to moderate degrees of multidimensionality, however, as defined by factor analyses, particularly where the dimensions are highly correlated and/or where the test/index length is more than 20 items and/or the sample size is more than 250 (Kirisci et al., 2001). Local independence is also an important assumption, that is, responses to a test item do not depend on other test item responses once trait is taken into account.

We have applied a two-parameter IRT test (using Mplus and R) to each of the MD items that were not excluded on the basis of the validity and reliability (Cronbach’s alpha) tests, including some of the ‘borderline’ housing items. The two parameters are the severity and the discrimination of the item.

The ‘severity’ of an item is the likelihood that the person/household will lack/not be able to afford that item; thus, it is desirable that a deprivation indicator should include items with a range of different ‘severity’ scores, that is, some low severity items, some medium severity items and some high severity items.⁷ We set the severity threshold at 3 standard deviations from the mean (in line with the ‘three-sigma rule’), that is, we drop all items with a severity greater than 3 standard deviations (deprivation which affects less than 0.13% of the EU population on average). Items with a severity of more than 3 standard deviations from the mean MD capture levels of severe deprivation endured by only a very small minority of EU-SILC respondents. They are therefore considered statistically unreliable (although they may be valid). In the EU level analysis, only the item ‘dwelling suffers from darkness’ fails this test. Although this deprivation groups together in the same dimension as the other deprivation items, the problem of a dark dwelling only affects people with a very high level of deprivation (those who have an MD of 4 standard deviations), making it unsuitable for the reliable measurement of MD in surveys with limited sample sizes.

The ‘discrimination’ of a deprivation item measures how well this item differentiates between the deprived and the non-deprived; thus, high ‘discrimination’ scores are desirable. The purpose of a deprivation index/scale is to measure how much of the latent trait ‘deprivation’ a person/household suffers from. The discrimination parameter indicates how well each item discriminates between deprived and non-deprived respondents. This can also be transformed into correlations (ranging between −1 and +1) between items and MD. The discrimination criterion we use is to drop all items whose correlation is lower than 0.4. At EU level, three items do not fulfil this condition: dwelling with a leaking roof, dark dwelling and dwelling with shortage of space. Overcrowding also has a relatively low correlation when compared to the other items, but it is above 0.4.

Item Characteristic Curves provide a useful visual display of the severity and discrimination of each deprivation item. The ability of each item to measure severity is shown by the position of each asymptotic (i.e. ‘S’-shaped) curve along the x-axis – the further to the right, the more severe the deprivation. The ability of each item to discriminate between the deprived and non-deprived people/households is shown by how vertical each curve is with respect to the y-axis; the more upright, the better the discriminating ability of the item and the higher its correlation with MD.

Ideally, a ‘good’ MD index would be illustrated by a series of fairly vertical ‘S’-shaped curves spread out along the x-axis. The inflection point of each curve, that is, half the distance between the upper and lower asymptotes, where the slope is steepest, should lie between 0 and +3 on the x-axis or, in other words, have a severity of between 0 and +3 standard deviations. As shown in Figure 1, darkness, shortage of space and leaky roof stand out as items which conform less to the ideal pattern, darkness being the strongest outlier.

Similarly, at EU Member State level, the four housing items (leaking roof, overcrowding, shortage of space and dark dwelling) had severities above 3 standard deviations (i.e. our threshold value) in many EU Member States, which means that these items do not satisfactorily measure MD in many EU countries. In addition to these housing items, ‘mobile phone’ is also associated with high severity levels of MD in many EU countries, and computer/Internet has severity problems in three Member States, that is, just above our severity threshold.

Figure 1.

Item Response Theory results for the 18 items retained for the MD indicator after the validity and reliability (Cronbach’s alpha) tests. See colour figure online.

Table 2 summarises the results of the various tests. It flags the items with suitability, validity or reliability problems in more than two countries.

Table 2.

Summary of suitability, validity and reliability tests.

	Problems of
At least half of the adults living in the household cannot afford (but would like to have)
A mobile phone (M)	√ (Borderline IRT)
Some new clothes (M)	√
Two pairs of shoes (M)	√
Some money for oneself (M)	√
Drink/meal monthly (M)	√
Leisure activities (M)	√ (Suitability)
The household’s dwelling suffers from
Basic amenities	Reliability
Shortage of space	Reliability
Darkness	Reliability
Leaky roof, damp, etc.	Reliability
Litter lying around (M)	Reliability
Vandalism (M)	Reliability
Noise	Reliability
Pollution	Reliability
Crime	Reliability
Overcrowding	Reliability
High housing costs (>40% income)	Reliability
The household cannot afford
To replace worn-out furniture	√
A meal with meat, chicken or fish	√
To face unexpected expenses	√
To keep home adequately warm	√
One week annual holiday away from home	√
To avoid arrears	√
A washing machine (enforced lack)	Reliability
A colour TV (enforced lack)	Validity and reliability
A telephone (enforced lack)	Validity and reliability
A computer/Internet (enforced lack)	√
A personal car (enforced lack)	√
The household has a (very) difficult access to
Public transport (M)	Reliability
Postal/banking services (M)	Reliability

Source: EU-SILC 2009 cross-sectional data, Users’ database – August 2011, authors’ computation.

M: MODULE; √: successful on all four criteria; IRT: Item Response Theory; EU-SILC: European Union Statistics on Income and Living Conditions; CTT: Classical Test Theory.

We consider that an item has validity problems if the results of the logistic regressions are not significant in two out of the three validity tests. We consider an item ‘invalid’ if it has validity problems in more than 2 out of the 27 Member States. Due to the large proportion of missing cases for all module items in Sweden (around 40%), this country is not taken into account in the count. For reliability tests, the same logic is followed. The reliability tests are considered successful if reliability problems are observed for maximum two Member States. An item is kept in the proposed indicator if it does not violate any of the four criteria we have retained in our framework (suitability, validity, reliability (CTT) and reliability (IRT)).

Mobile phone was a ‘borderline’ item according to the IRT analysis and was dropped from the final list to help ensure consistency of the scale over time. All the items for which one or more of these robustness criteria are not met have been dropped from the lists of items retained for our proposed indicator.

Additivity

Additivity tests aim to ensure that the MD indicator’s components add up, that is, to check that someone with a MD indicator score of ‘2’ is in reality suffering from more severe MD than someone with a score of ‘1’ or a score of ‘0’. This was checked using an analysis of variance (ANOVA) model (second-order interactions of MD items by level of equivalised disposable household income). These models assume that people who suffer from two deprivations (e.g. those who cannot afford both clothes and shoes) should live in households with (on average) significantly lower net equivalised incomes than those who only suffer from one deprivation (clothes or shoes deprivation only) or no deprivations. Similarly, those people suffering from one deprivation should have lower incomes than those with no deprivations. This should hold for all possible combinations of deprivation items.

These additivity tests can show that, in the EU as a whole or in any of the 27 EU Member States, those with a higher deprivation score are ‘poorer’ on average than those with a lower deprivation score. For the 13 valid, suitable and reliable MD items (see Table 2), 2184 two by two tables with 10,920 interactions (i.e. the difference in the mean income of all possible deprivation scores of 0, 1 and 2) were calculated. There were only seven statistically significant (but still fairly minor) interaction problems. Five of these problems were with the ‘Meat’ MD item in DK, one was ‘Car by PC/Internet’ in DK and the final problem was the interaction of ‘Expenses’ and ‘Arrears’ in CY. No MD items were dropped on the basis of these fairly minor interaction problems. The interaction problems for the ‘Meat’ item in DK may be related to the fact that according to the United Nations (UN) Food and Agriculture Organization, the people of DK eat more meat per capita than any other nation in the world.

Final deprivation index

The final list of items retained for the MD indicator consists of the following five ‘adult’ and eight ‘household’ items:

Adult: Some new clothes (enforced lack) – NEW;

Adult: Two pairs of shoes (enforced lack) – NEW;

Adult: Some money for oneself (enforced lack) – NEW;

Adult: Leisure activities (enforced lack) – NEW;

Adult: Drink/meal monthly (enforced lack) – NEW;

Household: Replace worn-out furniture (enforced lack) – NEW;

Household: Meat, chicken, fish (or vegetarian equivalent) – Retained from current MD indicator;

Household: Unexpected expenses – Retained from current MD indicator;

Household: Holiday – Retained from current MD indicator;

Household: Arrears – Retained from current MD indicator;

Household: Computer and Internet (enforced lack) – NEW;

Household: Home adequately warm – Retained from current MD indicator;

Household: Car (enforced lack) – Retained from current MD indicator;

Among the nine items used in the current EU MD indicator, six are retained. The enforced lack of a washing machine, a TV and a telephone failed in our tests and has no impact on the proportion of people deprived in most EU Member States. The seven ‘NEW’ items are from the 2009 MD module and are not currently included in the core questionnaire of EU-SILC.

The Cronbach’s alpha for the 13-item deprivation indicator is 0.85 for the pooled EU-27 dataset. The national alphas range from 0.75 in Sweden to 0.86 in Belgium. Thus, our proposed deprivation indicator has a high reliability for the EU-27 as a whole and also for each of the 27 EU Member States. In each country, the reliability exceeds the 0.70 acceptability threshold. This is a major improvement on the current (9-item) EU deprivation index for which the alpha varies from 0.50 to 0.60 in LU, CY, Sweden (SE), the United Kingdom, DK and NL to 0.70 in Bulgaria (BG) (0.69 for the EU as a whole).

To summarise, the current 9-item MD indicator has a low reliability in almost all EU Member States and also for the pooled EU-27 data. By contrast, our proposed 13-item MD indicator is highly reliable for the EU as a whole and in all EU countries and, therefore, measures deprivation with much greater precision. The proposed scale is also highly reliable for different age groups, with an alpha of 0.86 for children aged 0–17 years and 0.83 for older people (aged 65+ years).

We have also tested different thresholds (common to all countries). As illustrated in Figure 2 (for the EU-27 as a whole), a threshold of at least five items lacked (out of 13) leads to an MD rate for the EU-27 average (17.7%) which is quite close to that provided by the current EU-27 MD indicator (17.1%, 3+ items out of nine (EU MD)). A threshold of at least seven items lacked (out of 13) leads to an MD rate for the EU-27 average (9.2%) that is slightly higher than the current EU severe MD indicator: 8.1 percent (4+ items out of 9 (EU severe MD [SMD])).

Figure 2.

The 95 percent confidence interval for MD rates (%) according to different MD thresholds (comparison between the proposed 13-item MD indicator and the current MD indicators), EU-27.

The choice of the threshold(s) will need to be made according to a clear and transparent methodology. It is beyond the scope of this article to provide a detailed sensitivity analysis of the different potential thresholds. A threshold of at least seven items lacked (out of 13) leads to an MD rate for the EU-27 that is slightly higher than the current EU severe MD indicator (4+ items out of nine – EU SMD) used in the Europe 2020 social inclusion target (see Figure 2). As shown by Guio and Marlier (2013), moving from the current EU severe MD indicator to the alternative MD 7+ indicator has no statistically significant impact on the proportion of people severely deprived in five Member States (Czech Republic (CZ), Estonia (EE), DK, NL, Austria (AT)). Among those countries where differences are statistically different from zero, this impact is statistically higher than 2 percentage points in CY (MD 7+ lower than EU SMD) and in Lithuania (LT), Latvia (LV), Hungary (HU), Portugal (PT) and Romania (RO) (MD 7+ higher than EU severe MD).

‘MD 5+’ to ‘MD 9+’ refer to a lack of 5+ to 9+ items in our proposed 13-item indicator. ‘EU MD’ refers to the current indicator of MD (three lacks out of nine), whereas ‘SMD’ refers to the current indicator of ‘severe’ MD used in the definition of the Europe 2020 social inclusion target (4 lacks out of 9).

The very high level of reliability of our proposed indicator needs to be highlighted. CTT assumes that there are an infinite (or very large) number of MD measures. If we could have answers to this infinite number of MD questions, then we would have ‘perfect knowledge’ (we would know everything) about each person’s deprivation. No set of weights could add any additional information as we would already know everything: that is to say, the infinite deprivation index is self-weighting. The square root of the Cronbach’s alpha statistic can be considered to be the correlation between the index and the ‘perfect’ index made from the answers to the infinite set of deprivation questions (Nunally, 1978). The Cronbach’s alpha is 0.85. The correlations with the perfect infinite set of deprivation indicators are therefore impressive (0.92), so there is little additional information that any differential weights could add. Even if perfect, error-free differential weights could be developed, the results from the current MD indicator and the weighted indicator would be essentially identical. In view of these results and because of the simplicity and transparency of this approach, an equal weighting approach seems to be well suited for the construction of EU MD indicators.

Conclusion

The primary objective of this article was to propose a framework for developing sound aggregate indicators of MD that can be used for social monitoring purposes at both national and EU levels. Based on an analysis of the MD data collected in the 2009 wave of EU-SILC, we have made proposals for a new MD indicator.

In order to address the limitations of the current EU MD indicator, we opted for a broad approach to the concept of MD that makes full use of the richness of the 2009 EU-SILC wave. This approach has resulted in a proposed 13-item MD indicator covering some key aspects of living conditions which are customary across the whole EU. This measure is consistent with Townsend’s theory of relative deprivation.

We carried out a systematic item by item analysis at both EU and country levels to identify the MD items which satisfactorily meet our suitability, validity, reliability and additivity criteria across the EU. Such items, which satisfy each of these four criteria in each EU Member State and also in the EU as a whole, are likely to be measuring the same latent construct across countries. We have further assessed the extent of cross-national measurement equivalence using exploratory factor analysis, cluster analysis, MDS and correspondence analysis in order to compare the dimensional structure of the items across countries. Our results indicate that the same optimal sub-set of the available MD items can be used in all the Member States.

Yet, the analyses presented in this article are based on one single dataset. In 2013, almost all EU countries collected the seven MD items that are not included in the core part of EU-SILC and that were retained as a result of our analyses. When they become available, these data will allow confirmatory factor analysis modelling to provide additional information about configural, metric and scalar equivalence. They will also allow for in-depth change-over-time analysis. Additional questions such as the distribution of MD among adults living in the same household will be further investigated. Once the 2014 data become available, longitudinal analysis will also be possible. We believe that this will contribute to further advancing the ‘state of the art’ of poverty and MD measurement in the EU.

Footnotes

Appendix

The Online Appendix is titled - List of MD items available in EU-SILC 2009 for the whole population.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work has been supported by the second Network for the analysis of EU-SILC (Net-SILC2) - an international research project funded by Eurostat and coordinated by LISER - and by the UK Economic and Social Research Council (Grant RES-060-25-0052; 'Poverty and Social Exclusion in the United Kingdom').

Notes

References

Atkinson

Cantillon

Marlier

Nolan

(2002) Social Indicators: The EU and Social Inclusion. Oxford: Oxford University Press.

Bradshaw

Finch

(2003) ‘Overlaps in Dimensions of Poverty’, Journal of Social Policy 32(4): 513–25.

Cappellari

Jenkins

S.P.

(2007) ‘Summarizing Multiple Deprivation Indicators’, in Jenkins

S.P.

Micklewright

(eds) Inequality and Poverty: Re-examined, pp.166–84. Oxford: Oxford University Press.

Commission on Social Determinants of Health (2008) Final Report: Closing the Gap in a Generation: Health Equity through Action on the Social Determinants of Health. Geneva: WHO.

Council of the European Union (2011) Opinion of the Social Protection Committee on: Reinvigorating the Social OMC in the Context of the Europe 2020 Strategy. 10405/11, 23 May. Brussels: European Council.

Dickes

Fusco

Marlier

(2010) ‘Structure of National Perceptions of Social Needs across EU Countries’, Social Indicators Research 95(1): 143–67.

European Commission (2009) Portfolio of Indicators for the Monitoring of the European Strategy for Social Protection and Social Inclusion (2009 update). Brussels: European Commission.

Fahmy

Gordon

(2005) ‘La pauvreté et l’exclusion sociale en Grande-Bretagne’, Economie et Statistique 383–385: 109–30.

Fahmy

Gordon

Dorling

Rigby

Wheeler

(2011) ‘Poverty and Place in Britain, 1968–1999’, Environment and Planning A 43(3): 594–617.

10.

Fusco

Dickes

(2008) ‘The Rasch Model and Multidimensional Poverty Measurement’, in Kakwani

Silber

(eds), Quantitative Approaches to Multidimensional Poverty Measurement, pp.49–62. Houndmills: Palgrave Macmillan.

11.

Fusco

Guio

A.-C.

Marlier

(2010) ‘Characterizing the Income Poor and the Materially Deprived in European Countries’, in Atkinson

A.B.

Marlier

(eds) Income and Living Conditions in Europe, pp.133–53. Luxembourg: Office for Official Publications of the European Communities (OPOCE).

12.

Gordon

(2000) ‘The Scientific Measurement of Poverty: Recent Theoretical Advances’, in Bradshaw

Sainsbury

(eds) Researching Poverty, pp.37–58. Aldershot: Ashgate.

13.

Gordon

Adelman

Ashworth

Bradshaw

Levitas

Middleton

. (2000) Poverty and Social Exclusion in Britain. York: Joseph Rowntree Foundation.

14.

Gordon

Pantazis

(eds) (1997) Breadline Britain in the 1990s. Aldershot: Ashgate.

15.

Gordon

Davey Smith

Dorling

Shaw

(eds) (1999) Inequalities in Health: The Evidence Presented to the Independent Inquiry into Inequalities in Health. Bristol: Policy Press.

16.

Guio

A.-C.

(2009) ‘What can be Learned from Deprivation Indicators in Europe?’, Eurostat Methodologies and Working Papers. Luxembourg: Office for Official Publications of the European Communities (OPOCE).

17.

Guio

A.-C.

Marlier

(2013) ‘Alternative versus Current Measures of Material Deprivation at EU Level: What Difference Does it Make?’, Improve Working Papers No. 13/07, August 2013. Antwerp: Herman Deleeck Centre for Social Policy.

18.

Guio

A.-C.

Pomati

(2016) ‘How do European Citizens Cope with Economic Shocks? The Longitudinal Order of Deprivation’, in Atkinson

A.B.

Guio

A.-C.

Marlier

(eds) Monitoring Social Europe. Luxembourg: Publications Office of the European Union (OP).

19.

Guio

A.-C.

Fusco

Marlier

(2009) ‘A European Union Approach to Material Deprivation using EU-SILC and Euro-barometer data’. IRISS Working Paper 2009–19, 23 December. Luxembourg: CEPS/INSTEAD.

20.

Guio

A.-C.

Gordon

Marlier

(2012) ‘Measuring Material Deprivation in the EU: Indicators for the Whole Population and Child-Specific Indicators’, Eurostat Methodologies and Working Papers. Luxembourg: Office for Official Publications of the European Communities (OPOCE).

21.

Halleröd

Gordon

Larsson

Ritakallio

V.M.

(2006) ‘Relative Deprivation: A Comparative Analysis of Britain, Finland and Sweden’, Journal of European Social Policy 16(4): 328–45.

22.

Hillyard

Kelly

G.P.

McLaughlin

Patsios

Tomlinson

(2003) Bare Necessities: Poverty and Social Exclusion in Northern Ireland. Belfast: Democratic Dialogue.

23.

Kirisci

Hsu

T.C.

(2001) ‘Robustness of Item Parameter Estimation Programs to Assumptions of Unidimensionality and Normality’, Applied Psychological Measurement 25: 146–62.

24.

Leon

Walt

(2001) Poverty, Inequality and Health: An International Perspective. Oxford: Oxford University Press.

25.

Mack

Lansley

(1985) Poor Britain. London: George Allen & Unwin.

26.

Maître

Nolan

Whelan

(2006) Reconfiguring the Measurement of Deprivation and Consistent Poverty in Ireland. Dublin: ESRI Series Policy Research Series 58.

27.

Marlier

Natali

(eds), with Van Dam

(2010) Europe 2020: Towards a More Social EU? Brussels: PIE-Peter Lang.

28.

Marlier

Atkinson

A.B.

Cantillon

Nolan

(2007) The EU and Social Inclusion: Facing the Challenges. Bristol: Policy Press.

29.

Martini

M.C.

Vanin

(2013) ‘A Measure of Poverty Based on the Rasch Model’ in Torelli

Pesarin

Bar-Hen

(eds) Advances in Theoretical and Applied Statistics, pp.327–37. Berlin; Heidelberg: Springer.

30.

Nolan

Whelan

C.T.

(2011) Poverty and Deprivation in Europe. Oxford: Oxford University Press.

31.

Nunally

J.C.

(1978) Psychometric Theory, 2nd edn. New York: McGraw-Hill.

32.

Pantazis

Townsend

Gordon

(2006) ‘The Necessities of Life’, in Pantazis

Gordon

Levitas

(eds) Poverty and Social Exclusion in Britain: The Millennium Survey, pp.89–122. Bristol: Policy Press.

33.

Perry

(2002) ‘The Mismatch between Income Measures and Direct Outcome Measures of Poverty’, Social Policy Journal of New Zealand 19: 101–27.

34.

Raileanu Szeles

Fusco

(2013) ‘Item Response Theory and the Measurement of Deprivation: Evidence from Luxembourg Data’, Quality & Quantity 47(3): 1545–60.

35.

Social Protection Committee (2001) Report on Indicators in the Field of Poverty and Social Exclusion. Brussels: Council of the European Union.

36.

TNS (2007) Poverty and Social Exclusion. Report on the Special Eurobarometer 279/Wave 67.1, September 2007. Brussels: European Commission.

37.

Townsend

(1979) Poverty in the United Kingdom. Harmondsworth: Penguin Books.

38.

Townsend

(1987) ‘Deprivation’, Journal of Social Policy 16(2): 125–46.