Celebrating JOS Forty Years: Future Research Needs in the New Era of Official Statistics

1. Introduction

The Journal of Official Statistics (JOS) was first published in March 1985. For this Fortieth Anniversary Issue, we revisited the original Editorial Statement from that inaugural year. When JOS was established, its founder, Lars Lyberg, envisioned an internationally respected journal that would address both methodological and policy challenges faced by National Statistical Institutes (NSIs) and other producers of official statistics. The journal was intended for a broad audience, including professionals in statistical agencies, academia, and private organizations engaged in the field of official statistics. The mission statement, shown below, continues to guide the journal’s purpose and scope today:

Producers of official statistics use, or should use, sound methods from many disciplines (statistics, economics, computer science, social science, etc.), and from many fields within these disciplines. Therefore the new Journal of Official Statistics will cover wider methodological areas than most other statistical journals.

Over the past century, the production of statistics has evolved—from relying primarily on censuses, to the widespread use of surveys, and more recently, to the increasing integration of administrative data. Now, we are entering what is sometimes referred to as a new era of statistics, characterized by incorporation of diverse types of digital data sources and advanced methods to produce multisource statistics. This shift is driven by rapid technological advancements, changing user needs, rising survey costs, and declining response rates. The transition to this new paradigm is gradual rather than abrupt. Traditional and emerging methods are likely to coexist, each serving different audiences and purposes. Hence, our field has become perhaps even more multidisciplinary than before and maintaining scientific rigor in methods used remains essential, though it is an increasingly challenging task.

The fortieth anniversary of JOS provides an opportunity to look back on its journey over these years, in which respect we are particularly grateful to Risto Lehtonen for his excellent review (Lehtonen 2025). Moreover, this special anniversary issue brings together twenty-two invited articles from leaders and experts in our field, on future challenges and research needs, whose common objective is to keep official statistics relevant, accurate, and trustworthy. We hope that this special issue will inspire our readers and that in ten years, upon JOS fiftieth anniversary, we might be able to draw conclusions about the progress made on the exciting research avenues proposed herein.

Given the wide range of topics covered in this issue, we introduce them in thematic groups highlighting some connections that may help the reader navigating and digesting the complex methodological landscape of official statistics as we see it.

2. National Statistical Systems

The first set of articles shed light on the critical institution-wide challenges faced by the NSIs.

Starting with user needs, Santos (2025) highlights the tension between accuracy and timeliness, a discussion that seems to have intensified by the seemingly quickening pace of changes in societies, which generate new user needs challenging the relevance of existing data collections and the underlying concepts. The new System of National Accounts is an important example, that has consequences across a wide range of outputs of official statistics, as MacFeely (2025) discusses the dilemmas of moving beyond GDP.

Increasing the use of non-survey data is necessary to deliver for the increasing user needs, which cannot be satisfied by survey data alone and generally raises the need for appropriate methods beyond survey sampling, as pointed out by Santos. Combining data from multiple sources, or data integration, is a large topic area when it comes to reusing non-survey data for statistical purposes, in order to achieve the high standard that is required of official statistics. Holmberg (2025) reflects on several related challenges for methods, quality, and modernization approaches. We should also like to refer the readers to the special JOS issue on data integration published in June 2025.

Ultimately, perhaps even more than any specific methodology adopted, the public trust in official statistics depends on the fundamental values and principles, as well as the professional independence and accountability of the statistical agencies. How to deliver in terms of professional ethics and scientific integrity is a critical question to be addressed, and Eltinge (2025) discusses the complexity of empirical, nuanced considerations of relevant tradeoffs.

Protecting confidentiality is another related important issue that can be raised, where the ultimate aim is to get consent and maintain public trust. For potentially sensitive non-survey data, such as health information, mobile phone positions, payment transactions, privacy enhancing technology during data collection and processing has become a necessary development one needs to and should embrace, as Ricciato (2025) argues. Meanwhile, as reviewed and discussed by Domingo-Ferrer et al. (2025), traditional disclosure limitation for dissemination of statistical data is faced with renewed challenges, conceptually and operationally, due to the emergence of the differential privacy model.

Lastly and perhaps our “greatest challenge” (Santos 2025) concerns our staff. Pfeffermann (2025) calls for active policies on recruitment and training, in terms of competence and skills, to address the imbalance of university curricula as well as the increasing competition for talents from related data science fields. Santos (2025) reminds us of the importance of fostering a culture of innovation and a mindset open to change throughout statistical agencies.

3. Survey Data Collection

This group of articles focus on survey data collection, which still occupies a central position among all the raw data that feed the national statistical systems. As survey methodology keeps evolving over time, new problems and difficulties have kept arising; indeed, some may even argue that the quality-cost ratio of surveys has declined over the past decades and, for this reason, many fear that survey as a mode of soliciting relevant information may not be sustainable.

Schouten (2025) reflects on the history of survey response enhancement, from chasing the last respondents to targeting for better representativity to the hunt of engaged respondents. Olson (2025) describes the challenges and look to the future of interview surveys and Christian (2025) in a similar fashion looks at mixed mode surveys. Kreuter (2025) discusses the broader modernization of survey data collection, including the integration of non-survey sources such as administrative and big data, as well as the emerging potentials of AI technology—particularly large language models. All three authors, Christian, Kreuter, and Olson raise the compelling question of whether AI could eventually assist or even replace human interviewers, highlighting a shift in methodology and ethics of data collection.

Innovation and development concerning these challenges are important, as survey data is likely to continue playing a vital role in official statistics for a foreseeable future. Indeed, while it may be possible to produce more and more fit-for-purpose statistics exclusively based on non-survey data, their validation and quality assurance would likely require survey data based on periodic audit sampling, as for example mentioned by Holmberg (2025).

4. Statistical Production, Inference and Analysis

The articles in this section delve into more details of many other topics important to statistical production, inference, and analysis, although any of them can as easily be considered in terms of their connections to multisource statistics or survey sampling or other topics discussed earlier.

Population census or census-like statistics remain the cornerstone product of official statistics. Brown and Chipperfield (2025) summarize the challenges for census transformation, in countries without a population register that has the sufficient quality for “direct” enumeration, where one needs to rely on administrative or sign-of-life data sources, possibly combined with coverage surveys.

Non-probability samples are another topical issue for data integration, regardless the sizes or sources of such samples. Kim (2025) proposes a framework of generalized entropy calibration to non-probability samples, in a unified manner to calibration estimation based on probability samples. Liao and Biemer (2025) describe the experience and lessons learned from a large study of data blending methods in the US.

Probability sampling has a unique value to official statistics due to its transparent scientific foundation and its target-agnostic nature of statistical inference. Chauvet (2025) draws attention to the challenges of sampling-based estimation in the context of mixed mode survey and data integration.

Scholtus (2025) reflects on statistical data editing, which commands a considerable amount of resources in the production of official statistics. The traditional strive of data editing, to automate and prevent future errors, is likely to assume an even greater, indeed necessary, role when it comes to quality control of non-survey big data sources.

Use of machine learning (ML) models and algorithms has attracted much attention in recent years. Not only can various ML tools often achieve improved performance compared to the traditional statistical models, but they can also greatly extend the range of tasks and the types of data that can be handled, as showcased by Ranalli (2025) and Tzavidis (2025) in their respective articles. At the same time, however, they both point out that applying any black-box-like ML algorithm would clearly increase the risk of misleading results, unless the statistician can provide appropriate inference framework of the associated uncertainty and robust quality assurance procedures.

Aside from the large amount of descriptive statistics that are familiar to many users, NSIs also have an important role when it comes to analytic inference, the goals of which are directed at unobservable or theoretic characteristics of a population.

A broadly relevant concern arises for analytic or causal inference from the perspective of secondary analysts, who need to rely on micro data provided by statistical agencies. In cases where record linkage is required to create the micro data, one may need to account for the potential linkage errors. Chambers (2025) delineates several challenging issues given that the secondary analysts have neither access to the source data nor the details of the record linkage procedure. In addition to integrating data, Breidt et al. (2025) highlight the needs for privacy protection and detailed metadata, as well as pointing out the opportunities that may be helpful to both the statistical agencies and the researchers.

Next, Münnich (2025) focuses on microsimulation as an intriguing approach to what-if analysis, where the base population and its evolution should ideally conform to all the relevant official statistics and other reliable data sources, in order to make the simulations as realistic as possible.

Finally, regarding international and cross-cultural comparability, De Jong (2025) reflects on the challenges that stem from comparison errors, due to differences in cultural norms, methods, and languages, making it difficult to ensure consistent interpretations across diverse populations. She points to the fact that ethnic diversity is significantly increasing in numerous countries. This growing diversity underscores the urgency for many NSIs to adopt more inclusive and adaptive methodologies from the “Hard-to-Reach” and “Hard-to-Survey” literature to capture more accurate data from traditionally underrepresented populations.

5. Reflections on Quality in a Changing Landscape

Drawing on the contributions to this special issue, as well as ongoing discussions in various forums within the statistical community, a clear picture emerges of the rapidly evolving landscape of official statistics. On the output side, the user needs will become more demanding in terms of scope as well as speed. On the input side, a growing share of data will come from non-probability and non-survey sources complementing traditional designed surveys. On the throughput side, the integration of new methods such as machine learning will shape how data is processed and transformed. AI will also impact the way data is collected, processed, and communicated; it is however less clear at this stage exactly how. Regardless how precarious it may be to tell the future of research against this background, one thing is clear: quality remains a key concern and a unifying theme across all contributions and ongoing discussions.

Today’s Quality Management Systems (QMS) at NSIs are built on a set of interconnected components that together supports the accuracy, reliability, and credibility of statistical outputs. These systems typically include both quality assurance and quality control functions. Quality assurance focuses on preventing errors before or during production through proactive and systematic efforts—such as staff training, adherence to best practices, and the use of standardized methods. In contrast, quality control involves reactive measures like interviewer monitoring, coding control, and data editing, aimed at detecting and correcting errors during or after production. Ideally, quality control also contributes to root cause analysis, feeding insights back into the system for continuous improvement.

Beyond these core functions, QMS frameworks often incorporate broader elements such as total error frameworks, which help identify and manage different sources of error; evaluation studies to assess and refine processes and products; metadata management to ensure transparency and traceability; and user engagement to align outputs with user needs. As new digital data sources, AI, and advanced analytical methods are introduced into statistical production, all these QMS components will need to adapt to new conditions. In fact, work to adjust and update them has already begun in several areas.

Consider, for example, compiling statistics on production of goods in the European Union (PRODCOM, Production Communautaire) entirely based on tax records and transactions data provided by relevant payment service providers, which uses ML algorithms for data linkage, business unit identification, text/digital content processing, and product classification, all completed in a confidential computing environment sanctioned for official statistics. Even though one may achieve state-of-art performance of all the relevant processes, it is obviously impossible to avoid errors related to for example, coverage, linkage, coding, and classification. This raises some questions: How do we validate that the resulting statistics are fit-for-purpose? Is there a quality standard one can call on?

As discussed by Zhang (2021, 2023), audit sampling has been used in the past to address such questions, typically in connection with census or register-based statistics, and the approach can be extended and enhanced to provide a cost-effective, design-based quality standard for error evaluation of non-survey multisource official statistics. Unlike quality control that operates at the level of underlying processes, auditing can be applied to the final statistics without isolating different error sources. The inference of error is based on the probability design of the audit survey, regardless the models and assumptions necessary to the statistics being audited. Because the goal is to estimate error—not to produce the full statistics—audit samples can be smaller in size, stratified to focus on high-risk areas, and conducted less frequently than the statistics themselves. The idea aligns with modern quality management principles, particularly risk-based thinking, by directing resources to areas with the highest potential for error.

Many articles in this special issue can be viewed from the perspective of future research on quality for official statistics. Our discussion above is intended to supplement their contributions and to emphasize quality as a constant despite the changing eras. Moreover, we would like to encourage our readers to cross-examine the future challenges and research needs presented in this issue, and to draw further inspiration from the topics that have been explored here.

Finally, it is with gratitude and pride that we take this opportunity to thank the authors, reviewers, and all editors for their invaluable effort and support to JOS over the past forty years. Looking ahead, we’re excited about the future of official statistics and the role JOS will continue to play in sharing important and inspiring research in the field.