Extending TSE to Administrative Data: A Quality Framework and Case Studies from Stats NZ

Abstract

Many national statistics offices acknowledge that making better use of existing administrative data can reduce the cost of meeting ongoing statistical needs. Stats NZ has developed a framework to help facilitate this reuse. The framework is an adapted Total Survey Error (TSE) paradigm for understanding how the strengths and limitations of different data sets flow through a statistical design to affect final output quality. Our framework includes three phases: 1) a single source assessment, 2) an integrated data set assessment, and 3) an estimation and output assessment. We developed a process and guidelines for applying this conceptual framework to practical decisions about statistical design, and used these in recent redevelopment projects. We discuss how we used the framework with data sources that have a non-statistical primary purpose, and how it has helped us spread total survey error ideas to non-methodologists.

Keywords

Total survey error multiple data sources official statistics survey design

References

Australian Bureau of Statistics. 2009. The Australian Bureau of Statistics Data Quality Framework. Canberra: Australian Bureau of Statistics. Available at: http://www.abs.gov.au/ausstats/abs@.nsf/mf/1520.0. (accessed June 2013).

Bakker, B. 2012. “Estimating the Validity of Administrative Variables.” Statistica Neerlandica 66: 8–17. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1111/j.1467-9574.2011.00504.x.

Biemer, P. 2010. “Total Survey Error: Design, Implementation and Evaluation.” Public Opinion Quarterly 74: 817–848. Doi: http://poq.oxfordjournals.org/content/74/5/817.full.pdf+html10.1093/poq/nfq058.

Biemer, P. and L. Lyberg. 2003. Introduction to Survey Quality. New York: Wiley.

Black, A. 2016. The IDI prototype spine’s creation and coverage. Available at: http://www.stats.govt.nz/methods/research-papers/working-papers-original/idi-prototype-spine. (accessed August 2016).

Bryant, J., K. Dunstan, P. Graham, N. Matheson-Dunning, E. Shrosbree, and R. Speirs. 2016. Measuring Uncertainty in the 2013-Base Estimated Resident Population (Stats NZ Working Paper No 16-04). Available at: http://www.stats.govt.nz/methods/research-papers/working-papers-original/measure-uncertainty-2013-erp.aspx (accessed March 2017).

Bryant, J. and P. Graham. 2015. “A Bayesian Approach to Population Estimation with Administrative Data.” Journal of Official Statistics 31: 475–487. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1515/JOS-2015-0028.

Burger, J., J. Davies, D. Lewis, A. van Delden, P. Daas, and J.-M. Frost. 2013. Deliverable 6.5/2011: Final List of Quality Indicators and Associated Guidance, Report for Work Package 6 of the ESSnet on the Use of Administrative and Accounts Data for Business Statistics. Luxembourg: Eurostat. Available at: https://ec.europa.eu/eurostat/cros/system/files/SGA%202011_Deliverable_6.5.pdf_en. (accessed August 2016).

Daas, P.J.H., S.J.L. Ossen, and M. Tennekes. 2010. “Determination of Administrative Data Quality: Recent Results and New Developments.” In Proceedings of the Q2010 European Conference on Quality in Official Statistics, May 4–6, 2010. Available at: http://www.pietdaas.nl/beta/pubs/pubs/Q2010_Session34_presentation.pdf. (accessed June 2012).

10.

Daas, P., S. Ossen, M. Tennekes, L.-C. Zhang, C. Hendriks, K. Foldal Haugen, A. Bernardi, F. Cerroni, T. Laitila, A. Wallgren, and B. Wallgren. 2011. Deliverable 4.1: List of Quality Groups and Indicators Identified for Administrative Data Sources, Report for Work Package 4 of the European Commission 7th Framework program BLUE-ETS. Brussels: European Commission. Available at: http://www.blue-ets.istat.it/index.php?id=7. (accessed December 2015).

11.

Daas, P., S. Ossen, and M. Tennekes. 2012. Deliverable 4.3: Quality Report Card for Administrative Data Sources Including Guidelines and Prototype of an Automated Version, Report for Work Package 4 of the European Commission 7th Framework program BLUE-ETS. Brussels: European Commission. Available at: http://www.blue-ets.istat.it/index.php?id=7. (accessed December 2015).

12.

Gibb, S., C. Bycroft, and N. Matheson-Dunning. 2016. Identifying the New Zealand Resident Population in the Integrated Data Infrastructure (IDI). Available at: http://www.stats.govt.nz/methods/research-papers/topss/identifying-nz-resident-pop-in-idi.aspx. (accessed August 2016).

13.

Groves, R.M. and L. Lyberg. 2010. “Total Survey Error: Past, Present, and Future.” Public Opinion Quarterly 74: 849–879. Doi: http://poq.oxfordjournals.org/content/74/5/849.full.pdf+html10.1093/poq/nfq065.

14.

Laitila, T. and A. Holmberg. 2010. “Comparison of Sample and Register Survey Estimators via MSE Decomposition.” In Proceedings of the Q2010 European Conference on Quality in Official Statistics, May 4–6, 2010. Available at: http://q2010.stat.fi/sessions/special-session-34/. (accessed December 2015).

15.

Office for National Statistics. 2013. London: Office for National Statistics. Guidelines for Measuring Statistical Quality. Newport: Office for National Statistics. Available at: http://webarchive.nationalarchives.gov.uk/20160105160709/http://www.ons.gov.uk/ons/guide-method/method-quality/quality/guidelines-for-measuring-statistical-quality/index.html. (accessed February 2017).

16.

Organization for Economic Cooperation and Development. 2007. OECD Glossary of Statistical Terms. Paris: OECD. Available at: https://stats.oecd.org/glossary/index.htm. (accessed August 2016).

17.

Roemer, M. 2002. Using Administrative Earnings Records to Assess Wage Data Quality in the March Current Population Survey and the Survey of Income and Program Participation. Maryland: U.S. Census Bureau. (Technical paper No. TP-2002-22). Available at: https://www2.census.gov/ces/tp/tp-2002-22.pdf. (accessed September 2016).

18.

Scholtus, S. and B.F.M. Bakker. 2013. Estimating the Validity of Administrative and Survey Variables Through Structural Equation Modelling: A Simulation Study on Robustness. The Hague / Heerlen: Statistics Netherlands. (1572-0314, no - 201302).

19.

Smith, T.W. 2011. “Refining the Total Survey Error Perspective.” International Journal of Public Opinion Quarterly 23: 464–484. Doi: http://ijpor.oxfordjournals.org/content/23/4/464.short/10.1093/ijpor/edq052.

20.

Statistics Canada. 2009. Statistics Canada Quality Guidelines. Ontario: Statistics Canada. Available at: http://www.statcan.gc.ca/pub/12-539-x/12-539-x2009001-eng.pdf. (accessed June 2013).

21.

Statistics NZ. 2015a. Implementing Classification and Other Changes to Building Consent Statistics. Available at: http://www.stats.govt.nz/browse_for_stats/industry_sectors/Construction/building-consent-changes-2015.aspx. (accessed January 2016).

22.

Statistics NZ. 2015b. Methodology and Classification Changes to Value of Building Work Put in Place Statistics. Available at: http://www.stats.govt.nz/browse_for_stats/industry_sectors/Construction/methodology-classification-changes-value-buildingwork.aspx (accessed January 2016).

23.

Statistics NZ. 2015c. Retail Trade Survey: September 2015 Quarter, Data Quality Section. Available at: http://www.stats.govt.nz/browse_for_stats/industry_sectors/RetailTrade/RetailTradeSurvey_HOTPSep15qtr/Data%20Quality.aspx (accessed January 2016).

24.

Statistics NZ. 2015d. Value of Building Work Put in Place: June 2015 Quarter. Available at: http://www.stats.govt.nz/browse_for_stats/industry_sectors/Construction/ValueOfBuildingWork_HOTPJun15qtr/Data%20Quality.aspx. (accessed August 2016).

25.

Statistics NZ. 2016a. Guide to Reporting on Administrative Data Quality. Available at: http://www.stats.govt.nz/methods/data-integration/guide-to-reporting-on-admin-data-quality.aspx (accessed at August 2016).

26.

Statistics NZ. 2016b. Our Strategic Direction. Available at: http://www.stats.govt.nz/about_us/who-we-are/our-strategic-direction.aspx. (accessed August 2016).

27.

Statistics NZ. 2016c. How Accurate are Population Estimates and Projections? An Evaluation of Statistics New Zealand Population Estimates and Projections, 1996–2013. Available at: http://www.stats.govt.nz/browse_for_stats/population/estimates_and_projections/how-accurate-pop-estimates-projns-1996-2013.aspx (accessed March 2017).

28.

Statistics NZ. 2016d. Standard for population terms. Available at: http://www.stats.govt.nz/browse_for_stats/population/standard-pop-terms.aspx (accessed May 2017).

29.

United Nations Economic Commission for Europe. 2011a. Canberra Group Handbook on Household. Income Statistics. New York and Geneva: United Nations. Available at: http://www.unece.org/index.php?id=28894 (accessed December 2015).

30.

United Nations Economic Commission for Europe. 2011b. Using Administrative and Secondary Sources for Official Statistics – A Handbook of Principles and Practices. New York and Geneva: United Nations. Available at: http://www.unece.org/fileadmin/DAM/stats/publications/Using_Administrative_Sources_Final_for_web.pdf. (accessed December 2015).

31.

Wallgren, A. and B. Wallgren. 2014. Register-Based Statistics:Sstatistical Methods for Administrative Data, 2nd ed. Chichester: Wiley.

32.

Zhang, L.-C. 2012. “Topics of Statistical Theory for Register-Based Statistics and Data Integration.” Statistica Neerlandica 66: 41–63. https://dx-doi-org.web.bisu.edu.cn/10.1111/j.1467-9574.2011.00508.x.