Statistical Journal of the IAOS

This paper describes international collaboration activities to help the official statistical community to better understand the phenomenon of ``Big Data'', and how it might impact on statistical production. This work is taking the form of annual projects, overseen by the High-Level Group for the Modernisation of Statistical Production and Services, a group of ten heads of national and international statistical organisations. The results obtained during 2014 are presented, and the paper looks forward to the planned activities in 2015.

The paper is focused on the results of testing web scraping techniques in the field of consumer price surveys with specific reference to consumer electronics products (goods) and airfares(services). The paper takes as starting point the work done by Italian National Statistical Institute (Istat), in the context of the European project ``Multipurpose Price Statistics'' (MPS). Among the different topics covered by MPS are the modernization of data collection and the use of web scraping techniques. Included are the topic of quality (in terms of efficiency and reduction of error) and some preliminary comments about the usability of big data for statistical purposes. The general aims of the paper are described in the introduction (Section 1). In Section 2 the choice of products to test web scraping procedures are explained. In Sections 3 and 4, after a description of the survey for consumer electronics and airfares, the results and/or the issues of testing web scraping techniques are conveyed and discussed. Section 5 stresses some comments about the possible improvements in terms of quality deriving from web scraping for inflation measures. Some conclusive remarks (in Section 6) are drawn with a specific attention to big data issue. In two fact boxes centralised collection of consumer prices in Italy and the IT solutions adopted for web scraping are presented.

The national information communications technology (ICT) industry association (popularly known as PIKOM) in collaboration with the largest online job recruitment service provider, namely Jobstreet.com, has been publishing salary profile of ICT professionals in Malaysia annually since 2006. The series has been publishing average salary of ICT professionals by industry, job category, employment size, geographical locations and top paying industry. The online job registration system that Jobstreet.com biannually updates constituted the data source for the statistical activity. The reported data are of high quality as there is an inherent tendency for jobseekers to furnish accurate information in search of new jobs or career advancements. The past series also demonstrated consistency and stability in data trends that official statistics is equally concerned about. Not only private sector, the mainstream policy and planning agencies also have become vivid user of PIKOM data, in particular ICT salary records at three digit occupation and five digit industry levels. Typically such levels of data dissemination are not feasible under national sample surveys. More importantly, the ICT salary data are published as official statistics in the Digital Economy Satellite Account (DESA) system that consolidates all ICT data in the country including private sector initiatives. As next level collaboration, the paper also discusses big data analytics (BDA) targeted at semi-structured and unstructured data that the online job registration system entails.

Big data offers many opportunities for official statistics: for example increased resolution, better timeliness, and new statistical outputs. But there are also many challenges: uncontrolled changes in sources that threaten continuity, lack of identifiers that impedes linking to population frames, and data that refers only indirectly to phenomena of statistical interest. We discuss two approaches to deal with these challenges and opportunities.

First, we may accept big data for what they are: an imperfect, yet timely, indicator of phenomena in society. These data exist and that's why they are interesting. Secondly, we may extend this approach by explicit modelling. New methods like machine-learning techniques can be considered alongside more traditional methods like Bayesian techniques.

National statistical institutes have always been reluctant to use models, apart from specific cases like small-area estimates. Based on the experience at Statistics Netherlands we argue that NSIs should not be afraid to use models, provided that their use is documented and made transparent to users. Moreover, the primary purpose of an NSI is to describe society; we should refrain from making forecasts. The models used should therefore rely on actually observed data and they should be validated extensively.

Many statistical offices have been moving towards an increased use of administrative data sources for statistical purposes, both as a substitute and as a complement to survey data. Moreover, the emergence of big data constitutes a further increase in available sources. As a result, statistical output in official statistics is increasingly based on complex combinations of sources. The quality of such statistics depends on the quality of the primary sources and on the ways they are combined.

This paper analyses the appropriateness of the current set of output quality measures for multiple source statistics, it explains the need for improvement and outlines directions for further work. The usual approach for measuring the quality of the statistical output is to assess quality through the measurement of the input and process quality. The paper argues that in multisource production environment this approach is not sufficient. It advocates measuring quality on the basis of the output itself - without analysing the details of the inputs and the production process - and proposes directions for further development.

To assess census coverage the Brazilian Institute of Geography and Statistics - IBGE has been conducting a post enumeration survey - PES since 70's census. In 2010 the survey was conducted in a sample of enumeration areas in each of the 27 federation units, matching was performed for data from Census and PES and a reconciliation work was conducted on the unmatched housing units and persons. Finally, dual-system estimation was applied to estimate the 2010 Census net coverage, omission and erroneous inclusion. One of the biggest improvements of the 2010 Brazilian Census is the incorporation of new methodologies and technologies. Use of handheld devices in the 2010 Census and PES allowed improvement of quality and timeliness in the data collection process, and facilitated automatic matching of PES to the Census. An automatic matching step, based on the probabilistic linkage theory of Fellegi and Sunter, was added to the assisted matching and reconciliation already performed in the 2000 Census PES, making the three-step 2010 matching system. The paper gives an overview on the process and methods used to carry on the 2010 Brazilian Post Enumeration Survey, bringing key information on the preparation, data collection, sampling, matching and estimation. Results allow knowing the performance of the matching system in addition to 2010 Census coverage measuring as a hole.

Social media can play a critical role in the dissemination of the information as well as collection of relevant data during natural disasters. The idea of leveraging social media data such as Twitter is intuitively attractive, given their natural ties to mobile devices with obvious disaster response implications.

However, as with any data analysis, the design of the analysis, from the objective definition, the data collection specifics, data management, valuation of the data contents, and analysis methodologies, can determine the ultimate effectiveness and validity of the analysis activities. This is exasperated in the context of disaster response given its time-sensitive nature. The disaster response resources and the analytical resources must work closely together to ensure that the analysis is fit for purpose and meets the ultimate objective. This study discusses the key considerations for such collaboration through an analysis of Twitter data surrounding the 2013 landfall of Typhoon Haiyan in the Philippines.

This paper examines various aspects of data fabrication, or ``curbstoning'' in field surveys. Our ability to detect and control such behavior is limited by the costs of the most effective instruments, the weakness of most of our instruments and our limited understanding of what drives such behavior. Culture, an emergent pattern of thoughts and behaviors from a larger complex of systems and behaviors, operates in spaces where direct incentives or control cannot reach. Monitoring is important, but fostering a healthy culture among field staff may be the most efficient and the most humane approach to controlling curbstoning.

The National Statistical Offices (NSO) in the countries in Northern Europe (Norway, Sweden, Finland and Denmark) have developed production systems that are based on statistical registers. Almost all surveys done by the Nordic NSOs are based on these systems of registers. Besides all register surveys that use these registers also sample surveys and censuses use the register system for creating frames and utilizing register variables as statistical variables and as auxiliary variables used for estimation.

These ways of using registers for statistics production are well-known and many countries are developing their production systems to become more and more based on registers as in the Nordic countries. However, the system of registers has features that have not been generally recognized: it opens possibilities to work with quality assessment and nonsampling errors in a new way.

All registers can be compared at the microdata level and also all sample surveys can be compared at the microdata level with all registers in the system. Systematic comparisons between samples surveys and registers in the system will give new knowledge of quality in different surveys and also give new possibilities to redesign surveys and improve the quality of the surveys in the system.

During the last 15 years labour market statistics produced by Statistics Denmark have increasingly become more integrated. For example, the Statistics on People Receiving Public Benefits have been joined into an integrated statistical system. In this way, the quality of the statistics has been enhanced and the published figures have become logically consistent. However, the statistical users request a more cohesive statistical system covering the entire population's attachment to the labour market. The system should include volume information and provide the possibility of analyzing longitudinal labour market data.

Against this background, in the beginning of 2012 Statistics Denmark initiated work on developing an integrated statistical system for analyzing the entire population's attachment to the labour market. The statistical system is called Labour Market Account (LMA). It is intended to publish statistics from the LMA in 2014. In addition to being an important source of the future labour market statistics, the LMA will also be a direct or indirect input source to a number of other statistics within social, business and economic statistics.

This presentation gives a description of the new statistical system and of the user requirements with regard to the system. Data from the various source registers frequently contain non-permissible overlaps or inconsistent start and end dates concerning the individual states. Subsequently, the presentation describes the core of the new statistical system which is the harmonization of data from a great variety of input sources and the longitudinal data processing conducted by the rule driven engine developed for this purpose.

Postscript. In April 2015 Statistics Denmark has published figures of the populations attachment to the labour market measured in full-time equivalents and as labour market status based on the new Labour Market Accounts (LMA). Longitudinal micro data for researcher will be available soon. The Working Time Account which provide data to the National Account will be revised in 2016 to use data from the Labour Market Account. The interest from users on the new possibilities is big.

Macro-integration techniques are used for the reconciliation of macro figures, usually in the form of large multi-dimensional tabulations, obtained from different sources. Traditionally these techniques have been extensively applied in the area of macro-economics, especially in the compilation of the National Accounts. Methods for macro-integration have developed over the years to become very versatile techniques for integration of data from different sources at the macro level. Applications in other domains than macro-economics seem promising. In this paper we present an application to labour market data from two sources, an administrative one and a survey, with slightly different definitions and different frequencies of reporting (monthly, quarterly). The purpose is to combine these estimates to form a single monthly estimate. Depending on the specification of the macro-integration model several alternatives for obtaining such estimates are derived.

This paper explores how information is generated about killings in conflict, and how the process of generation shapes the statistical patterns in the observed data. The difference between the observed patterns and the true patterns is called bias, two examples of which will be examined. First, we compare multiple individual sources reporting identifiable killings in Syria, highlighting variations in the likely probabilities of reporting for events of different sizes. Second, we conduct a similar analysis examining the number of sources reporting events of varying sizes in the Iraq Body Count public dataset. In both cases we explore how depending on the observed data without accounting for bias caused by missing data could mislead policy. The paper closes with recommendations about the use of data and analysis in the development of policy.

In 2008 the Council of Australian Governments agreed to national reforms to address homelessness through the National Agreement on Affordable Housing (NAHA) and National Partnership Agreement on Homelessness(NPAH). Integral to meeting the information needs to support these reforms was the development of a Specialist Homelessness Services data collection. Working in collaboration with Commonwealth and state and territory governments, NGO service providers, peak homelessness bodies and a private sector information systems supplier, the Australian Institute of Health and Wealth (AIHW) developed a new national data collection and introduced an innovative data collection instrument and client information management system, known as SHIP. SHIP enables NGO case workers to easily capture client information, case manage clients and monitor their progress. SHIP allows seamless collection of data conforming to national data standards and submission to AIHW each month via a secure website.

This paper describes the strong partnership arrangements that successfully delivered the new information system and data collection to 1500 NGO service providers across Australia and is providing valuable information to support measurement of policy effectiveness under the NAHA and NPAH in achieving successful outcomes for the clients of homelessness services.

An important strategic goal of Destatis is to continuously collect information about the customer satisfaction and the perception of important stakeholders and target groups. We conduct frequent customer surveys since 2007. But not all important stakeholders and target groups are necessarily registered customers. To learn more about their demands a reputation analysis was conducted in 2013 in cooperation with a market researcher. To determine a manageable frame for the study, we focused on three target groups: Respondents (households and enterprises), fast multipliers (online and data journalists) and young multipliers (young academics). The analysis was mainly based on the ``Kano-Model'', a methodological approach, which is often used in quality management and product development. In the following article the survey design and the main results will be presented.

Since many influences bear on customers' decisions to buy, an ongoing multivariate approach to understanding customers' needs is advocated for every organization. It is demonstrated here. Influences on art lovers' behavior in choosing a price to pay for a piece of art, on their art-buying habits, on their volunteering time to work at an art museum, and on their making financial donations to an art museum are examined. Factor analysis, the method used for data analysis, is tracked from its invention to current use. Folding insights about customers' needs into accomplishing and tracking appropriate innovations in an organization's operations, then assessing how the innovation affects organizational outcomes, is outlined. The methods and advice apply to organizations of every kind.

There are many methods (for example, expert reviews, cognitive interviews, and experiments) to test questionnaires and other data collection instruments. In practice, all surveys cannot be tested with all methods; there has to be a balance with regard to survey importance, consequences of data errors, available resources, and costs. Statistics Sweden has developed a strategy how to test questionnaires in different surveys, taking the abovementioned factors into account. This strategy is based on a small set of survey characteristics which mainly are taken from a database with classifications for many surveys. Based on how a survey is classified in the chosen characteristics, the strategy proposes different levels of testing. These levels vary in both ambition and the methods included. The strategy has, since implemented, increased the amount of testing but in a differentiated way, based on risk and resources.

The dramatic decline in survey response rates over the past three decades raises significant concerns about the possibility of bias in survey results. Current theory emphasizes that it is the relationship between response propensity and variables of interest that determines the extent of the bias, and that a low response rate in itself does not necessarily imply a high level of bias. This assertion is supported by a number of studies which have shown that response rate alone is a fairly poor predictor of nonresponse bias. However, most of these studies suffer from methodological features that in some way compromise their attempts to isolate the relationship between response rate and bias. This paper describes the results of a pair of studies which allow for a near-ideal examination of this relationship. The results support the conclusions of prior research, showing that even achieved samples with response rates as low as 10 percent may produce highly accurate estimates in certain cases.

In First World colonised nations such as Aotearoa New Zealand and Australia, population statistics form the evidentiary base for how Indigenous peoples are known and `managed' through state policy approaches. Yet, population statistics are not a neutral counting. Decisions of what and how to count reflect particular assumptions about Indigenous identity, ways of life and wellbeing. More often than not, the requirements and priorities of government take precedence over the informational needs and priorities of Indigenous communities. Whereas National Statistics Offices (NSOs) once rendered Indigenous peoples invisible in official statistics through non-recognition, the more pressing problem in the 21st century is that of misrecognition. In seeking to move beyond statistical misrecognition, we propose a set of guiding principles for bringing government reporting frameworks and Indigenous concepts of identity and wellbeing into closer proximity. We argue that a principled approach to collecting, disseminating and analysing Indigenous data not only avoids misrecognising Indigenous peoples but enhances the functionality of official statistics for Indigenous peoples and NSOs alike.