Accuracy of Mixed-Source Statistics as Affected by Classification Errors

Abstract

Publications in official statistics are increasingly based on a combination of sources. Although combining data sources may result in nearly complete coverage of the target population, the outcomes are not error free. Estimating the effect of nonsampling errors on the accuracy of mixed-source statistics is crucial for decision making, but it is not straightforward. Here we simulate the effect of classification errors on the accuracy of turnover-level estimates in car-trade industries. We combine an audit sample, the dynamics in the business register, and expert knowledge to estimate a transition matrix of classification-error probabilities. Bias and variance of the turnover estimates caused by classification errors are estimated by a bootstrap resampling approach. In addition, we study the extent to which manual selective editing at micro level can improve the accuracy. Our analyses reveal which industries do not meet preset quality criteria. Surprisingly, more selective editing can result in less accurate estimates for specific industries, and a fixed allocation of editing effort over industries is more effective than an allocation in proportion with the accuracy and population size of each industry. We discuss how to develop a practical method that can be implemented in production to estimate the accuracy of register-based estimates.

Keywords

Accuracy editing administrative data short-term business statistics bootstrap resampling

References

Biemer, P. and L. Lyberg. 2003. Introduction to Survey Quality. New York: John Wiley and Sons. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1002/0471458740.

Bishop, Y.M., S.E. Fienberg, and P.W. Holland. 1975. Discrete Multivariate Analysis: Theory and Practice. Cambridge, MA: MIT Press.

Bryant, J.R. and P. Graham. 2015. “A Bayesian Approach to Population Estimation with Administrative Data.” Journal of Official Statistics 31: 475–487. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1515/JOS-2015-0028.

Burger, J., A. van Delden, and S. Scholtus. 2015. “Sensitivity of Mixed-Source Statistics to Classification Errors.” Journal of Official Statistics 31: 489–506. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1515/jos-2015-0029.

Chipperfield, J. and R. Chambers. 2015. “Using the Bootstrap to Analyse Binary Data Obtained via Probabilistic Linkage.” Journal of Official Statistics 31: 397–414. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1515/JOS-2015-0024.

Christensen, J.L. 2008. “Questioning the Precision of Statistical Classification of Industries.” Paper presented at the DRUID Conference on Entrepreneurship and Innovation, 17–20 June 2008, Copenhagen. Available at: http://www2.druid.dk/conferences/viewpaper.php?id=3419&cf=29 (accessed April 2016).

Cochran, W.G. 1977. Sampling Techniques, 3rd ed. New York: John Wiley and Sons.

Delden, A. van, S. Scholtus, and J. Burger. 2015a. “Quantifying the Effect of Classification Errors on the Accuracy of Mixed-Source Statistics.” Discussion Paper 2015-10. Available at: https://www.researchgate.net/publication/281450992_Quantifying_the_effect_of_classification_errors_on_the_accuracy_of_mixed-source_statistics (accessed April 2016).

Delden, A. van, S. Scholtus, and J. Burger. 2015b. “Effect of Classification Errors on the Accuracy of Business Statistics.” Paper presented at the European Establishment Statistics Workshop, 7 – 9 September 2015, Poznan. Available at: https://enbes.wikispaces.com/file/view/Effect+of+classification+errors+on+the+accuracy+of+business+statistics+Arnout+van+Delden,+Sander+Scholtus+and+Joep+Burger.pdf (accessed April 2016).

10.

Delden, A. van and P.P. de Wolf. 2013. “A Production System for Quarterly Turnover Levels and Growth Rates Based on VAT Data.” Paper presented at the New Techniques and Technologies for Statistics (NTTS) conference, 5–7 March 2013, Brussels. Available at http://ec.europa.eu/eurostat/cros/sites/crosportal/files/NTTS2013%20Proceedings_0.pdf (accessed April 2016).

11.

Delden, A. van, S. Scholtus, P.P. de Wolf, and J. Pannekoek. 2014. “Methods to Assess the Quality of Mixed-Source Estimates.” Internal report PPM-2014-09-26-ADLN-SSHSPWOF-JPNK, Statistics Netherlands, The Hague.

12.

Efron, B. and R.J. Tibshirani. 1993. An Introduction to the Bootstrap. London: Chapman & Hall/CRC. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1007/978-1-4899-4541-9.

13.

Groves, R., F. Fowler, M. Couper, J. Lepkowski, E. Singer, and R. Tourangeau. 2009. Survey Methodology, 2nd ed. New York: John Wiley and Sons.

14.

Hall, P. and T. Maiti. 2006. “On Parametric Bootstrap Methods for Small Area Prediction.” Journal of the Royal Statistical Society B 68: 221–238.

15.

Ilves, M. and T. Laitila. 2009. “Probability-Sampling Approach to Editing.” Austrian Journal of Statistics 38: 171–182. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1111/j.1467-9868.2006.00541.x.

16.

Kreuter, F., ed. 2013. Improving Surveys with Paradata: Analytic Uses of Process Information. New York: John Wiley and Sons. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1002/9781118596869.

17.

Kuha, J. and C. Skinner. 1997. “Categorical Data Analysis and Misclassification.” In Survey Measurement and Process Quality, edited by L.E. Lyberg, P.P. Biemer, M. Collins, E.D. de Leeuw, C. Dippo, N. Schwarz, and D. Trewin, 633–670. New York: John Wiley and Sons. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1002/9781118490013.

18.

Lumme, S., R. Sund, A.H. Leyland, and I. Keskmäki. 2015. “A Monte Carlo Method to Estimate the Confidence Intervals for the Concentration Index Using Aggregated Population Register Data.” Health Services and Outcomes Research Methodology 15: 82–98. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1007/s10742-015-0137-1.

19.

McCullagh, P. and J.A. Nelder. 1989. Generalized Linear Models, 2nd ed. London: Chapman & Hall. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1007/978-1-4899-3242-6.

20.

Stoer, J. and R. Bulirsch. 2002. Introduction to Numerical Analysis, 3rd ed. New York: Springer. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1007/978-0-387-21738-3.

21.

Waal, T. de, J. Pannekoek, and S. Scholtus. 2011. Handbook of Statistical Data Editing and Imputation. New York: John Wiley and Sons.

22.

Zhang, L.-C. 2005. “On the Bias in Gross Labour Flow Estimates Due to Nonresponse and Misclassification.” Journal of Official Statistics 21: 591–604.

23.

Zhang, L.-C. 2011. “A Unit-Error Theory for Register-Based Household Statistics.” Journal of Official Statistics 27: 415–432.

24.

Zhang, L.-C. 2012a. “Topics of Statistical Theory for Register-Based Statistics and Data Integration.” Statistica Neerlandica 66: 41–63. Doi: https://dx-doi-org.web.bisu.edu.cn/10.1111/j.1467-9574.2011.00508.x.

25.

Zhang, L.-C. 2012b. “On the Accuracy of Register-Based Census Employment Statistics.” Paper presented at the European Conference on Quality in Official Statistics, May 30–June 1 2012, Athens. Available at: http://www.q2012.gr/articlefiles/sessions/23.4_Zhang_AaccuracyRegisterStatistics.pdf (accessed April 2016).