Record linkage to advance Indigenous mortality statistics in Australia – sources of error and bias

1. Introduction

This paper describes efforts made in Australia in the use of data linkage to enhance Indigenous mortality statistics. Reliable mortality measures require good death statistics and good population estimates. The quality of Australian Indigenous death and population statistics is known to be poor. This paper addresses the quality of Indigenous death statistics and efforts to improve them; it does not discuss in any detail the important issue of the quality of Indigenous population estimates.

There has always been a need to know the conditions of the Australian Indigenous population at national and sub-national levels. The most important data collection that includes Indigenous people is the regular Australian census of population and housing. The census has always counted Indigenous Australians to some extent, but it was not until a constitutional change in 1967 that all Indigenous people were included in official population estimates [1, 2]. Population census counts of Indigenous people have not been consistent, with censuses since 1971 showing large increases that cannot be explained through demographic accounting of births, deaths and migration [3]. The estimated size of the Indigenous population was 517,000 in 2006 based on the 2006 Census. It was estimated to be 670,000 in 2011 based on the 2011 Census, a very large increase of some 30% in only 5 years [4, 5].

While the population census sheds some light on the conditions of the Indigenous people, the content of the census is limited and the coverage of the Indigenous population is incomplete. In particular, census data alone are insufficient to calculate quality mortality measures.

Indigenous disadvantage is of increasing concern in Australia. In 2002, the Council of Australian Governments, comprising the heads of the state, territory and federal governments, commissioned a regular report against key indicators of Indigenous disadvantage [6]. In 2005, the Aboriginal and Torres Strait Islander Social Justice Commissioner urged Australian governments to commit to achieving equality for Indigenous people in health and life expectancy within 25 years. In response, non-government organisations launched a National Indigenous Health Equality Campaign in 2006 with targets for closing the gap, including closing of the life expectancy gap within a generation, and halving the gap in mortality rates for Indigenous children under five within a decade [7]. In 2007 the then Prime Minister, Kevin Rudd, announced that he would present a report to Parliament each year on the gap between the Indigenous and non-Indigenous people. This annual report ‘Closing the Gap’ has continued until today [8]. The indicators used for measuring the health gap are life expectancy at birth and under-five child mortality. There has therefore been an increased urgency to improve Indigenous data to respond to these government reporting requirements.

2. Quality of Australian Indigenous death registration data

Before the 1980s various definitions were used by state and federal governments to identify Aboriginal and Torres Strait Islander Australians (Indigenous Australians). In 1978, the federal Government adopted a three-part definition of an Indigenous Australian and this definition was soon adopted also by all state and territory governments – an Aboriginal or Torres Strait Islander is a person of Aboriginal or Torres Strait Islander descent who identifies as an Aboriginal or Torres Strait Islander and is accepted as such by the community in which he (she) lives [9]. This three-part definition is currently used for most government administrative purposes that are designed specifically for Indigenous people such as Indigenous health services, Indigenous student financial assistance, Indigenous housing assistance etc. However, in most statistical data collections such as censuses and surveys, in practice, only self-identification is used. The standard question used in statistical collections is: “Are you (is the person) of Aboriginal or Torres Strait Islander origin” [10]?

While it is believed the almost all deaths in Australia are registered, Indigenous identification in deaths is known to be incomplete. In Australia, the legal registration of deaths is the responsibility of state and territory governments. Funeral directors and doctors are responsible for completing the relevant documents for the registration of the deaths. Death notification forms used by funeral directors across all states and territories contain the standard Indigenous question about the deceased. Similarly, the standard question is also used in the doctor’s medical certificate of cause of death. The states began asking for Indigenous identification in deaths only in the 1980s – the most recent state to do so being Queensland in 1997 [11].

Even when the question was included, not all Indigenous deaths were identified. In 2000–2004, using indirect demographic methods, the Australian Bureau of Statistics (ABS) estimated that only 57% of Indigenous deaths were identified, and that the level of identification varied greatly between States and Territories [4, 12]. Northern Territory had the highest identification level of 94%, followed by Western Australia (72%), South Australia (66%), Queensland (53%), New South Wales (46%) and Victoria (35%) [4]. New South Wales and Queensland which have low levels of Indigenous identification are where the highest numbers of Indigenous people live (31% and 28% respectively). Between 1990 and 1997, ABS considered Indigenous death data were of sufficient quality in Northern Territory, Western Australia and South Australia to be officially published. In 1997 Queensland and in 2010 New South Wales data were included [13], even thought it was recognised that full identification had not been achieved in all the included States.

OPEN IN VIEWER

Indigenous population estimates and projections are important inputs to government policy considerations and program evaluation. The Australian Bureau of Statistics is responsible for their production. Accurate and reliable Indigenous mortality information is required for population estimation and projection. Indigenous life tables are needed to produce population projections using the demographic component method. Life expectancy estimates from life tables are used as indicators of health status of the Indigenous people.

Given the known deficiencies of Indigenous death data, the need to adjust death data to correct for under-identification is well recognised. The method used initially by the ABS in the 1990s to calculate the required adjustment factors was the census survival method developed by Preston and Hill [14] and for the period 2001–2006 the indirect method developed by Bhat [15]. The Bhat method is a reformulation of the General Growth Balanced method first developed by Brass in the 1970s [16].

These indirect methods use jointly population data in two consecutive censuses and death registration data in the interval between the two censuses. The methods are generally based on four assumptions:

Accurate census counts and age and sex identification.

The Bhat method, however, relaxes the second assumption of no migration between the two censuses and incorporates migration into the estimation equation. Australian censuses since 1971 have shown large growth in the Indigenous population that cannot be accounted for by natural increase and migration. In the use of the Bhat method, the ABS treated the unexplained growth of population between the two censuses as migration and incorporated this unexplained growth into the estimation.

The Australian Bureau of Statistics clearly recognised the weaknesses of the methods and conceded that some of the assumptions were not necessarily met. The life tables and life expectancy estimates were called “experimental” to acknowledge that they were work in progress and subject to revision.

Results for the 1990s and the early 2000s based on the indirect methods showed very large gaps in life expectancy between Indigenous and non-Indigenous Australians – around 18 years based either on the Preston and Hill method [17] or on the Bhat method [4, 12]. These are much larger than the gaps estimated for Indigenous people in New Zealand (about 9 years), Canada (about 7 years) and the USA (about 6 years) [18, 19].

Although not fully comparable, because of differences in data source and methodology [20], there have been serious doubts about the size of the Australian Indigenous gaps compared with Indigenous people in the other countries that have a similar colonial history and level of social deprivation. Both the results and the methods used by the ABS were subject to intense debate in the mid 2000s. The assumption of invariant coverage across ages in both population census and death data was questioned, and the circularity in the use of the Bhat method was also considered problematic [19, 21].

In 2004–2005, new estimates of Indigenous life expectancy were made by Hill and his colleagues as part of the 2006 Australian Indigenous Burden of Disease study, using another re-formulation of the Brass General Growth Balance method [22]. This method does not treat unexplained population growth independently as migration but incorporates it into coverage error in the census counts. Results of this estimation gave a gap in life expectancy between Indigenous and non-Indigenous Australians of about 12–13 years for the period 1996–2001 [19]. This result demonstrated that adopting different assumptions under the general indirect estimation methodology produced very different results.

At about the same time, in view of the debate and the range of estimates, the National Advisory Group on Aboriginal and Torres Strait Islander Health Information and Data (consisting of representatives of the health departments of all Australian governments) commissioned a review of indirect methods for estimating Indigenous life expectancy. This review tested the impact of defective data on the indirect estimation methods by introducing different types and levels of errors into the relatively good quality population and death data of the Northern Territory. The review essentially measures the sensitivities of the assumptions that underlie these methods.

The report of the review [23] found that the indirect method advocated by Hill and his colleagues and the Bhat method used by the ABS are very sensitive to minor variations in the input data – in particular, variations in the age structure of the population and deaths. The report also found that it is not immediately obvious in which direction (increasing or lowering the estimated life expectancy) the methods would react to simulated variations. These findings led to the conclusion that estimates from these indirect methods cannot be considered reliable.

After each census, the Australian Bureau of Statistics reviews the methods it had used to produce estimates of Indigenous death coverage for use in calculating life expectancy. In 2008, it tested the efficacy of the Bhat and the Hill methods, and found that the methods produced death coverage rates for the periods 1996–2001 and 2001–2006 which were difficult to explain and did not seem plausible, in terms of both the level of coverage for some jurisdictions and the changes over the two periods [24]. The 2008 review concluded that the assumptions behind the indirect methods were difficult to justify. As part of this review, the ABS experimented with linking the 2006 census records to death records as an alternative to the indirect methods to develop death coverage rates and recommended this as the preferred way to adjust deaths data to overcome under-identification [24].

The first ABS census-death linkage project was developed as part of the ABS 2006 census data quality study. It was repeated after the 2011 Census and is currently being conducted in conjunction with the recent 2016 Census.

In the 2006 and 2011 linkages, death records after census date were linked to census records during the census processing phase. Census processing lasts 11 months. This allows death records of the 11 months following the census to be linked to census records to determine the accuracy of Indigenous identification in the death records. The limit of 11 months was due to the ABS practice of destroying identifiable information of census records (names and addresses) immediately after census processing [25, 26].

Australia does not have a national personal identification system, so there is no unique personal ID number that can be used for data linkage. The ABS census-death linkage relies on name, address, date of birth (or age) and other characteristics that exist in both census and death records. Sophisticated established data linkage algorithms are used for automated matching. Clerical review is performed to confirm automatic linkages and to assess cases that cannot be automatically assigned [27]. Linkage rates of 74% and 80% of death registration records were achieved in 2006 and 2011 respectively [28, 29].

Because the census suffers from some under-enume- ration, census undercount is also taken into consideration in the estimation of death coverage rates. Estimates of census undercount for the Indigenous population are derived from a census post-enumeration survey (PES) that is conducted after each census.

The ABS estimation of death coverage involves a two-part adjustment process – the first is adjusting for census Indigenous under-enumeration and the second is adjusting for under-identification of Indigenous deaths in death registration. The basic assumptions are: (1) Indigenous identification is superior in the PES data to the Census data and (2) Indigenous identification is superior in the census data to the death data.

Indigenous death identification rates and adjustment factors were calculated based on ABS census-death linked data. The adjustment factors were + 9% for the period 2005–07 and + 21% for the period 2011–12 [28, 29]. Life expectancy estimates resulting from the ABS census-death linkage study now show life expectancy gaps of around 10 years between Indigenous and non-Indigenous people. These are more comparable with those of New Zealand, Canada and the USA than earlier ABS estimates.

The ABS estimation algorithm in the use of census-death linked data for the 2005–07 periods has been examined in some detail by Madden et al. [30]. Issues concerning the accuracy of the linked census and death data and the reasonableness of some of the results have been raised. In particular, the large differences in the estimated adjustment factors between States and Territories and the negative adjustment factors for Western Australia and the Northern Territory were questioned.

The AIHW is an agency in the federal health portfolio that deals mainly with health and welfare administrative data. Among its data holdings are three national data sets that contain death records with Indigenous identification that can be used for linkage to death registration records – namely hospital separation records, residential aged care records and peri-natal (mid-wives) records. A project was completed in 2012 to test the feasibility of linking death registration data to death records in these data sets for the period 2001–2006 for the enhancement of Indigenous death data [31].

Deaths in Australia occur mainly outside the home. Hospital deaths account for about 55–60% of all deaths, and deaths in residential aged care facilities account for a little more than 30%. These two data collections, therefore, account for a large majority of all deaths. While there may be a small amount of duplications between the two data collections, these are identified during linkage and resolved.

Early infant deaths in the peri-natal death data set were used to link with death registration records. Additional information from the peri-natal data set is important as Indigenous infant deaths are more likely to be under-identified than adult deaths when they are registered.

One of the responsibilities of the AIHW is to link national disease register data (such as cancer and diabetes registers) with other data sets, including death register data, to support medical and health research. Based on this experience, algorithms were developed to link death registration to the three administrative data sets. In the absence of a unique identification system in Australia, other identifying variables were used. Names are available to the AIHW for linkage on the death registration records and the residential aged care records, but not on hospitals and peri-natal death records. However, date of death and date of birth are available and are important variables for linkage. A linkage rate of 86% of death registration records was achieved.

An “ever-Indigenous” approach was adopted in determining the Indigenous status from the various datasets. Unlike the ABS census-death registration study, this approach does not make a judgement on the relative reliability of Indigenous identification in the data sets and accepts that the deceased was Indigenous if this is indicated in any of the data sets.

The Indigenous death adjustment factor for the 2001–2006 period calculated from the AIHW data linkage study was 12% [31]. Life expectancy estimates resulting from the AIHW 2001–2006 study show life expectancy almost identical to the ABS 2005–2007 study. Therefore, the life expectancy gap between Indigenous and non-Indigenous people are also similar between the AIHW and the ABS studies.

While the AIHW and the ABS studies have shown similar results, this is not evidence that the ABS and AIHW approaches are both correct, and it certainly does not indicate that the linkages are free of errors and bias.

The linkage rate achieved in both the ABS and the AIHW studies are acceptable, but they are not very high. The ABS linkage rate (census with death registration) is 74% in 2006 and 80% in 2011; that is, 20% or more of death registration records were not linked [28, 29]. The AIHW study achieved a slightly better linkage result – 86% of death records in 2001–06 were linked [31]. In both the ABS and the AIHW studies, additional adjustments to the linkage results were made to cover the un-linked records.

Data linkage requires good identifying information for linkage. The lack of a unique identifying system in Australia necessitates the reliance on a combination of other identifying information such as name, date of birth, address etc. It is generally known that this identifying information is not perfect. It is also known that the quality is lower among Indigenous records than non-Indigenous records. This affects both the ABS and the AIHW studies.

In addition, there is also the issue of incomplete coverage of the data sets to be linked. In the case of the ABS study, the post enumeration survey showed that the 2011 population census missed some 17% of Indigenous people [5]. Included in this were records that did not have the Indigenous question answered, some of whom were Indigenous. The number of 2011 census forms returned which did not have the Indigenous questioned answered was large – 5%, or about 1.06 million records – almost twice the number of Indigenous persons counted at the 2011 census – about 550,000 [3]. Census under-enumeration and census records with missing Indigenous information would affect the linkage with death records. The ABS relied on the census post enumeration survey to make adjustments to correct for such deficiencies, but the post enumeration survey is designed to assess census data quality in general, and the Indigenous component is very small with a correspondingly high error range. In addition to small sample size problems, the coverage of the PES is also limited. Before 2011, remote areas and discrete Indigenous communities were not included in the PES [32]. Also, non-private dwellings, such as prisons and boarding houses, were not in scope. These non-inclusions would have some effect on the use of the PES as a linkage data source.

In the case of the AIHW study, hospital and residential aged care data sets together do not have 100% coverage of all deaths. It is not known whether Indigenous people are over or under represented in the missing category.

7. Consistency of Indigenous identification in the different data sets

Another data issue is the consistency of Indigenous identification between the different datasets. The consistency in identification between the data sets affects the quality of the identification rates that result from comparing records in the data sets.

The primary aim of the ABS and AIHW studies is to correct for the under identification of Indigenous people in the death data set. The ABS estimated that 92% of the 2006–2007 Indigenous deaths [6] and 82% of the 2011–2012 Indigenous deaths [29] were identified in the death registration system. The AIHW estimated that 89% of the 2001–2006 Indigenous deaths were so identified [31]. These identification rates were derived by comparing the Indigenous identification in the death registration data set with that of the alternative comparison datasets

A consistent response is where an Indigenous identification is recorded on the matched record in both datasets. An inconsistent response occurs when Indigenous identification is recorded on one of the matched records and not the other, or vice versa. Either of these inconsistent patterns can occur, although not with equal frequency. This is shown in the ABS data linkage comparison of death registrations with 2011 census responses. In this linkage, 1550 pairs of records (62%) were consistent, 606 pairs (24%) had Indigenous identified in the census but non-Indigenous or not stated in death registration, and 334 pairs (13%) had Indigenous identified in death registration but non-Indigenous or not stated in the census [29].

To understand the extent of consistency, a consistency index is calculated to show consistent Indigenous responses in both datasets as a percentage of Indigenous responses in either of the two datasets. An index of 1.0 shows complete consistency, an index of 0.0 shows no consistency and an index of 0.5 shows that half of the records are inconsistent with each other.

Indices obtained by comparing the various dataset pairs show varying inconsistencies. Consistency between both 2006 and 2011 censuses and PES data is high – 86% for Australia – and is highest for the Northern Territory (around 95%). But consistency for New South Wales is much lower (around 70–75%). Consistency in Indigenous identification between census and death data is much lower – in the low 60% nationally and just over 50% for New South Wales. Northern Territory again shows much higher consistency.

Data for South Australia, Victoria, Tasmania, and the Australian Capital Territory were not released by the ABS because of the very small number of Indigenous people and deaths included in the data linkage. The geographic pattern of identification consistency is similar to the pattern for earlier censuses estimated by the ABS using the Preston/Hill indirect method.

The AIHW has conducted two personal interview surveys in 2007–08 and 2011–12 to assess the quality of Indigenous identification in hospital separation records. These surveys compare personal interviews of patients with hospital patient records [33, 34]. Consistency indices based on the survey shows high consistency between hospital and survey data, especially for remote and very remote areas

Overall, apart from hospital-survey data comparison, the consistency indices in the other data sets show that inconsistency is substantial, reaching nearly 50% for the New South Wales census-death registration comparison. Inconsistency varies greatly between locations – data for the Northern Territory are much more consistent than the eastern states – with New South Wales the least consistent. Data for major cities are less consistent than rural and remote areas. Inconsistency also varies between data sets, with hospital data showing more consistency than census data compared with PES data. Death registration data seem most inconsistent when compared with census data.

There are two broad sets of requirements that need to be met before data linkage can be effectively used to correct for the deficiencies in Indigenous death data. One concerns the quality of the identifying information used for record matching and the extent of coverage of the datasets. The other concerns inadequacy and inconsistency in Indigenous identification and the relative superiority of datasets. An assessment of these is required for a decision to be made on which dataset(s) to rely on to benchmark Indigenous identification.

In both the ABS and the AIHW studies, the lack of a unique personal identification system in Australia impedes the linkage process. Names, dates of birth and addresses are not unique and can be mis-reported and mis-recorded easily. This has given rise to the relatively low although acceptable match rates in both studies. In addition, the data sets used by both the ABS and the AIHW to enhance death data do not have full coverage. The ABS population census suffers from under-enumeration and non-responses to the Indigenous question; the PES is a sample survey and captures only small numbers of Indigenous people. Similarly, hospitals and residential aged care data used by the AIHW do not have full coverage. It can be expected that those who are not covered in the datasets are different from those covered, thereby contributing some bias in the result. This bias still exists even if adjustments are made under assumptions to correct for the under coverage.

Enhancements to question design, data collection and processing may improve the quality of the matching datasets. But these are not easily made in official data collections. The quality of identifying information and census under-enumerations may not improve enough to lift the linkage rates significantly.

The second broad set of requirements relates to Indigenous identification. For over 100 years, Australian native policy was dedicated to the ultimate disappearance of Indigenous Australians as an identifiable race. People of mixed descent were required to ‘assimilate’ and identify as non-Indigenous. As a result, many such people became statistically invisible. The assimilation policy has now been rejected, but its legacy still impacts on the way people identify and are identified in statistical collections [2]. Indigenous people who were once statistically invisible are increasingly identifying themselves, resulting for instance in the rapid non-demographic increase in the census population referred to above.

It is shown that Indigenous identification is not always consistent between datasets. The ABS in choosing to rely on census identification essentially ignored the identification in the death registration, arguing that doing so improved numerator-denominator consistency. Madden et al. [30] questioned the implied judgement that identification in the census is superior, and identification in death registration without substantiation in census data is faulty.

The AIHW linkage study uses an “ever-Indigenous” method that accepts the Indigenous identification in any of the data sets and discounts any inconsistent non-Indigenous identification. While data collection and processing problems may obviously give rise to false Indigenous records, and the way individuals are identified may change from time to time and from occasion to occasion, the overall assumption is that if a person once identifies as Indigenous, that will override any inconsistent identification as non-Indigenous in other sources.

All indirect and data linkage methods for estimating death coverage and life expectancies rely on assumptions. They all improve mortality estimates by varying degrees. The experience in Australia so far indicates that data linkage methods are the most promising and are not too sensitive to violations of assumptions. However, further improvement of the basic traditional data sources – that is Indigenous death registration and population estimates – will facilitate and improve linkage method results. Significant improvements to basic source of data would hopefully make indirect or linkage base estimates unnecessary. Meanwhile, the results of these linkage studies could be verified by comparing them with data sources which incorporate all three elements of the official definition – not just Indigenous identification but also descent and endorsement by Indigenous community groups.