The quality assurance of official statistics in Japan: Framework and practice

Abstract

Assuring the quality of official statistics is critical as the information based on official statistics is used for reasonable decision making not only by the government but also by citizens. In order to assure the quality of official statistics in Japan, a multi-layered framework and continual practice under the framework play essential roles. The legal part of the framework consists of several elements such as the Statistics Act, the Master Plan, Examination of plans and budgets for censuses and statistical surveys, and the Statistics Commission, while the practical part of the framework is implemented based on the Guidelines for Quality Assurance of Official Statistics to form a PDCA cycle. This paper describes how the quality of official statistics in Japan is assured in terms of its framework and practical implementation, which are well compatible with the Fundamental Principles of Official Statistics (FPOS) and the National Quality Assurance Framework (NQAF) of the United Nations.

Keywords

Official statistics statistics act national quality assurance framework guidelines PDCA cycle

1. Introduction

The initiation of the activities for quality assurance of official statistics in Japan dates back 70 years, when the original Statistics Act was established in 1947. Since then, the examination of plans and budgets for censuses and statistical surveys has been performed by the statistical authority concerned, which has served to assure the quality of official statistics.

In 2007, the Statistics Act [1] was fully amended to put more emphasis on the usefulness of official statistics, which necessarily enhanced the scope of the quality assurance of official statistics in Japan. The amended Statistics Act (hereinafter called “Act”) is well compatible with the Fundamental Principles of Official Statistics (FPOS) of the United Nations [2], which was adopted by the UN Statistical Commission in 1994 and by the General Assembly in 2014. Then, more tangible measures for assuring the quality of official statistics started in 2010, when the “Guidelines for Quality Assurance of Official Statistics” [3] were formulated based on the “Master Plan” which was decided by the Cabinet according to the Act.

In 2012, the Statistics Commission of the United Nations endorsed the generic national quality assurance framework (NQAF) [4] that was developed by the Expert Group on National Quality Assurance Frameworks. In the course of the development, experts from Japan not only participated in the Group and contributed to the development, but also were inspired to promote activities on quality assurance of official statistics in Japan.

In recent years, the Statistics Commission of Japan suggested a PDCA cycle on the accuracy of official statistics with quality checks as the starting point. The actual implementation of these checks has just started in 2017.

This paper aims to describe the quality assurance of official statistics in Japan thus developed from the viewpoints of its framework and practical implementation.

2. Quality assurance framework

The quality assurance framework of the Japanese official statistical system has a hierarchical structure with multiple layers, comprising two legal layers and two practical ones (see Fig. 1).

The first layer is the Statistics Act. The Japanese Statistics Act was established in 1947 and thoroughly amended in 2007 in order to place greater value on statistics as an information source for citizens in their reasonable decision making. The Act serves as an important basis for the Japanese statistical activities including quality assurance.

The second layer consists of three elements: the Master Plan, Examination of plans and budgets for censuses and statistical surveys, and the Statistics Commission. The Master Plan, officially called the “Master Plan Concerning the Development of Official Statistics,” is a basic plan stipulated by the Act to promote measures comprehensively and systematically concerning the development of official statistics. The second element, the Examination of plans and budgets for censuses and statistical surveys, is also based on the Act. This examination is performed by the Minister for Internal Affairs and Communications before he/she grants an approval to the head of an administrative organ who intends to obtain approval from the Minister on conducting fundamental or general statistical surveys. Thirdly, the Statistics Commission, established based on the Act, is set up in the Ministry of Internal Affairs and Communications (MIC) as a professional advisory board. It consists of 13 learned and experienced experts. The Commission is in charge of deliberating and offering opinions on the Master Plan and Fundamental Statistics.2

The third layer is the Guidelines for Quality Assurance of Official Statistics, which were agreed at the Conference of the Directors in charge of Statistics of Line Ministries on March 31, 2010, and amended lastly on February 23, 2016. These Guidelines provide practical guides to promote review and efficiency of official statistics through the indication and evaluation of the quality of official statistics while maintaining and improving the reduction of response burden and the quality of statistics. They also provide the guides on evaluating the quality of the implementation process of statistical surveys such as planning, implementation, editing, data processing, and dissemination of statistical surveys.

The fourth layer is the PDCA practice to assure quality of statistics, which is realized, based on the Guidelines, by the effective application and operation of the legal part of the quality assurance framework.

These elements in the four layers are combined organically and provide the framework for assuring the quality of Japanese official statistics. The following sections describe how these elements play their respective roles in assuring the quality of official statistics.

Figure 1.

Multi-layered structure of the quality assurance framework of official statistics in Japan.

3. Quality assurance based on regulations

The elements in the legal domain of the quality assurance framework of official statistics are well compatible with the FPOS and the NQAF of the United Nations. In the following subsections, how those elements such as the Act and the Master Plan are related with the FPOS and the NQAF is described (see Annex 1).

3.1 Statistics Act

The Statistics Act plays an essential role in the Japanese Statistical system. The present Act, which went fully into effect in 2009, puts greater value on ensuring the usefulness of official statistics and necessarily aims to assure the quality of official statistics in the broader sense of the term. Assuring such quality of official statistics is essential as official statistics are crucial information for citizens in their reasonable decision making, which is described at the very beginning of the Act. The next subsections describe a number of characteristics of the Act in relation to quality assurance.

3.1.1 Fundamental Principles

The fundamental principles of the Statistics Act are stipulated in Article 3 of the Act, which are provided as follows.

(Fundamental Principles)

Article 3

Official statistics shall be developed systematically through mutual cooperation and the appropriate sharing of roles among administrative organs, etc.

Official statistics shall be produced with appropriate and reasonable methodologies, so as to ensure neutrality and reliability.

Official statistics shall be provided widely for the citizens so that they are easily accessible for their effective use.

Confidential matters concerning individuals or juridical persons, or other organizations that were used for producing official statistics shall be protected.

These principles are well compatible with the FPOS and the philosophy of the NQAF of the UN. In concrete terms, Paragraph 1 of Article 3 is compatible with Principle 8 of FPOS, which emphasizes the coordination among statistical agencies within countries. Paragraph 2 is in line with Principles 1, 2, and 5 of FPOS which stipulate impartiality, trustworthiness, and appropriateness of sources, respectively. Paragraph 3 states effective use, and this is compatible with Principle 1 and also Principle 3 of FPOS, the latter of which claims the necessity to present information for correct interpretation of the data. Lastly, Paragraph 4 is in line with Principle 6 of FPOS, which stipulates confidentiality of individual data.

The next several subsections describe how articles of the Act correspond to NQAF lines with respect to each grouping (category) of the NQAF.

3.1.2 Managing the statistical system

Articles of the Act that relate to managing the statistical system, the first category of NQAF, are Articles 3, 4, and 28. Paragraph 1 of Article 3 corresponds to coordinating the national statistical system prescribed in NQAF 1. Paragraph 5 of Article 4 which stipulates the Minister for Internal Affairs and Communications to take necessary measures to reflect the opinions of the citizens when intending to prepare a draft Master Plan relates to managing relationships with data users and data providers stipulated in NQAF 2. Article 28 that is relevant to the establishment of Statistical Standards is compatible with managing statistical standards in NQAF 3.

3.1.3 Managing the institutional environment

In this category of NQAF, Paragraph 2 of Article 3 of the Act is compatible with NQAF 4 to 5 and 8 to 9. Using appropriate and reasonable methodologies stipulated in Paragraph 2 of Article 3 leads to assuring professional independence (NQAF 4), assuring the quality commitment (NQAF 8), and assuring adequacy of resources (NQAF 9). This paragraph is also to ensure neutrality and reliability, which are connected to assuring impartiality and objectivity (NQAF 5).

According to Article 8 and 23 of the Act, the head of an administrative organ shall, when having produced fundamental/general statistics, promptly publicize the statistics and related matters through the Internet, etc., of which description is in line with assuring transparency (NQAF 6).

Protection of confidential matters and its punitive clauses are prescribed in Paragraph 4 of Article 3 and Articles 39, 41, and 57, all of which are compatible with assuring statistical confidentiality and security of NQAF 7.

3.1.4 Managing statistical processes

Managing statistical processes, the third category of NQAF lines, consists of assuring methodological soundness (NQAF 10), assuring cost-effectiveness (NQAF 11), assuring soundness of implementation (NQAF 12), and managing the respondent burden (NQAF 13).

Again, Paragraph 2 of Article 3 of the Act is compatible with this category. This Paragraph states the necessity of using appropriate and reasonable methodologies, which directly relates to NQAF 10. It also leads to cost-effectiveness (NQAF 11) and managing the respondent burden (NQAF 13). Besides, assuring soundness of implementation (NQAF 12) is the prerequisite for ensuring neutrality and reliability stipulated in this Paragraph.

Article 29 of the Act is the provision that a ministry may request other ministries and agencies (hereinafter called “ministries”) to provide administrative record information when finding that utilizing such information would contribute significantly to the accurate and efficient production of statistics or to the reduction of the burden on respondents of statistical surveys. This provision is compatible with assuring cost-effectiveness (NQAF 11) and managing the response burden (NQAF 13).

3.1.5 Managing statistical outputs

The last category of NQAF lines is managing statistical outputs. Intimately associated with this category is Paragraph 3 of Article 3 of the Act. Especially, effectiveness of use stated in this Paragraph relates to assuring relevance (NQAF 14), assuring accuracy and reliability (NQAF 15), and managing metadata (NQAF 19). Accessibility also prescribed in this Paragraph is compatible with assuring accessibility and clarity (NQAF 17).

From the viewpoint of punctuality and clarity, Articles 8 and 23 of the Act stipulate prompt publicity and easy access of official statistics. These are in line with assuring timeliness and punctuality (NQAF 16) as well as accessibility and clarity (NQAF 17).

In addition, Article 28 of the Act, which concerns establishment of statistical standards, assures coherence and comparability (NQAF 18).

3.2 Master Plan

The Master Plan is to be drawn up approximately every five years and the first Plan was decided by the Cabinet in 2009 based on the Statistics Act. The present Master Plan, decided by the Cabinet in 2014, is called the Second Term Master Plan (hereinafter simply called “Master Plan”) [5] which also plays an important role in the quality assurance framework.

In the first part of the Master Plan, five basic perspectives and policies are described: (1) Securing and improving compatibility between statistics, (2) Securing and improving international comparability of statistics, (3) Adequate responses toward environmental changes in the economy and society, (4) Promotion of accurate and efficient production of statistics, and (5) Promotion of open statistical data and transparency of statistical production process.

These perspectives and policies are highly relevant to quality assurance and also compatible with the NQAF lines. For example, the first perspective is relevant to NQAF 15 (Assuring accuracy and reliability) and NQAF 18 (Assuring coherence and comparability). The second perspective is in line with NQAF 3 (Managing statistical standards) and NQAF 18, and the third one NQAF 14 (Assuring relevance). The fourth perspective, promotion of accurate and efficient production of statistics, is highly related to the “Managing statistical processes” category of NQAF (NQAF10-13) as well as NQAF 15. The last perspective, which is on openness and transparency of statistical data, relates to NQAF 6 (Assuring transparency), NQAF 17 (Assuring accessibility and clarity), and NQAF 19 (Managing metadata).

The Master Plan includes concrete plans and measures that respective statistics should follow. Especially, in connection with the promotion of statistical quality assurance activity of official statistics, the following concrete measures are described.

Measures for the promotion of planned self-evaluation and publishing of the results, etc. shall be strengthened by sharing information about the status of implementing the measures related to statistical quality assurance activities, and their effective and efficient practices, etc.

With reference to international trends and the findings of related academic research, etc., the Guidelines for Quality Assurance of Official Statistics shall be revised, aiming at introducing process assurance.

As the above two measures are closely associated with the “Guidelines for Quality Assurance of Official Statistics,” detailed explanation on how we have dealt with these measures will be given in later sections concerning the guidelines and their practical implementation.

In this way, the Master Plan promotes various concrete measures emphasizing the above-mentioned basic perspectives and policies, which are highly relevant to quality assurance of official statistics, in order to develop statistics systematically aiming at securing and improving statistical usefulness.

Table 1
Points to be checked and correspondence between the NQAF lines and them

Matters	Points	NQAF lines
Need for surveys	1. Are the statistics produced by the existing survey data or administrative records?	10, 11, 12,13, 14
Respondents	2. Is the population set out definitely?	10, 13, 14
	3. Is the survey frame adequate?	10, 13
	4. Are sample surveys used instead of censuses when possible?	10, 11
	5. Is the response burden kept reasonable?	10, 13
Questionnaire	6. Is there a need for each question?	10, 14, 15
	7. Are the questions easy to understand?	13, 15
	8. Is there an explanation for respondents to understand the legal basis?	6
Methodology	9. Are the procedures taken to protect confidentiality adequate?	7
Dissemination	10. Can the statistics be more promptly released?	16
	11. Are all the statistics released?	17
Standards to be used	12. Are the standards used appropriately?	18
Preservation of the statistical data	13. Are the statistics archived properly?	17

Source: Sawamura Y, Kubo S. “The Quality Assurance Framework in Japan”.

3.3 Examination of plans and budgets for censuses and statistical surveys

Based on the Statistics Act, any governmental organization which intends to conduct a census or a statistical survey must submit a plan thereon to the Minister for Internal Affairs and Communications in order to seek approval of the Minister. Then, the plan is examined by the Director-General for Policy Planning in charge of Statistical Standards (DGSS) who is under the direction of the Minister. The examination is performed from the aspects stipulated by the Act. Such aspects consist of three major points: (1) the contents of the composition element of the plan for the census or the survey are necessary and sufficient in light of the purpose of producing official statistics, (2) the official statistical survey is reasonable and appropriate in terms of statistical techniques, and (3) duplication with other official statistical surveys is within the limit considered to be reasonable. In addition to the above examination and approval, the DGSS of the MIC offers opinions to the Budget Bureau of the Ministry of Finance about statistical activities so that the opinions are reflected in the budget, if necessary.

This examination of censuses and statistical surveys, which has been performed since 1947, is a key tool for the coordination of statistical surveys, and has played and is playing an important role in quality assurance of official statistics in Japan.

Examples of points to be checked in the examination include the following: necessity of the census or the survey, relevance of the survey frame and response burden, appropriateness of questions on the questionnaire, confidentiality, promptness of dissemination, appropriateness of statistical standards used, and preservation of the statistical data. These points are compatible with the lines of NQAF and their correspondence relationships are shown in Table 1, which were presented by Sawamura and Kubo [6].

3.4 Statistics Commission

In assuring quality of official statistics, the role of the Statistics Commission is quite pivotal.

The Statistics Commission is authorized to state an opinion when the Minister for Internal Affairs and Communications makes the Master Plan, approves/disapproves an administrative organ’s plan to conduct an official statistical survey such as a fundamental statistical survey, intends to establish statistical standards, reports on the situation of the enforcement of the Act, etc.

By means of the above activities, the Statistics Commission bears the function of ensuring quality of official statistics, which is specified by other elements for assuring quality of statistics such as the Master Plan, approval of official statistical surveys, and establishing statistical standards.

3.5 PDCA cycle in the framework

The above-mentioned legal elements of the framework for the quality assurance of official statistics are closely associated with each other and statistical activities are carried out within the framework to form a PDCA cycle.

The first step of the PDCA cycle is to “Plan” to produce statistics, which corresponds to drawing up a Master Plan and more detailed plan by ministries concerned to achieve the Master Plan. As mentioned earlier, when a ministry or an agency intends to conduct an official statistical survey based on the Master Plan and other related regulations, it must submit a plan for the survey to receive the approval of the Minister for Internal Affairs and Communications.

Based on such plans, each ministry or agency produces statistics e.g. by conducting a statistical survey or by compiling statistical data. This is the “Do” phase of the PDCA cycle.

The next step is to “Check” the situation of the enforcement of the Statistics Act including the progressing condition of the Master Plan. This is performed based on Article 55 of the Act, in which the Minister for Internal Affairs and Communications may request the heads of administrative organs, etc. to report on the situation of the enforcement of the Act. The Minister then compiles those reports every year, publicizes their outline and reports the outline to the Statistics Commission.

Table 2
Quality elements and their definitions in official statistics

Elements	Definitions
Major Elements
Relevance	The degree to which the statistics, as fundamental public information to be widely and effectively used by a variety of social entities, meet users’ needs.
Accuracy	The degree to which the statistics correctly describe social and economic values or conditions.
Timeliness	Whether the statistics are published or made available in time to be useful for users’ objectives.
Interpretability/Clarity	The degree of the availability of necessary information such as the methods for data collection, processing, storage and publication, in order that users can appropriately understand and effectively use it.
Supplementary Elements
Reliability	The degree to which the method of producing the statistics is determined based on expertise and they are disseminated, and to which the measures for confidentiality protection are taken appropriately.
Coherence/Comparability	The degree to which the statistics are coherent or consistent in concepts, definitions, and classifications among different statistics so that users can compare these statistics between regions and over time.
Accessibility	The degree to which both the data and basic information are made easily available in response to users’ needs.
Effectiveness	The degree to which the statistics are produced by appropriate sources and methods in view of cost and burden on respondents.

Source: “The Guidelines for Quality Assurance of Official Statistics”.

The Commission then deliberates the reports and may state its opinion on the enforcement of this Act to the Prime Minister, the Minister for Internal Affairs and Communications, or the heads of the administrative organs concerned. In accordance with the opinion stated or suggestion made by the Statistics Commission, Ministers concerned will “Act” to improve their statistical activities. Particularly, the Minister for Internal Affairs and Communications may draft a revised version of the Master Plan incorporating the opinion of the Statistics Commission.

In this way, the PDCA cycle for quality assurance in the legal domain of official statistical activities is systematically embedded in the Statistics Act and it is soundly implemented every year (Fig. 2).

Figure 2.

The PDCA cycle in the quality assurance framework of official statistics.

4. Quality assurance in practice

4.1 Guidelines for quality assurance

On March 31, 2010, “The Guidelines for Quality Assurance of Official Statistics” was agreed at the Conference of the Directors in charge of Statistics of Line Ministries in order to provide a standard guidance to promote quality assurance aiming to assure and improve the usefulness and reliability of official statistics. In the Guidelines, the quality assurance is considered to be promoted through the production and dissemination of official statistics which meet user needs, and also through the indication, evaluation and improvement of their quality.

The following subsections present the contents of the Guidelines including their basic principles and how to implement quality assurance of official statistics by means of the Guidelines.

4.1.1 Basic principles of the Guidelines

The quality of official statistics consists of various elements including accuracy, relevance, and timeliness. Accuracy means that official statistics show the state of the society and economy as accurately as possible. Relevance indicates that official statistics meet the users’ needs as far as possible, and timeliness means that the produced official statistics are disseminated in a timely fashion in accordance with users’ needs and purposes.

The Guidelines assume that the quality of official statistics comprises the elements listed in Table 2, and these elements are used as indexes for indicating and evaluating the quality of statistics. The elements are divided into two categories: Major Elements and Supplementary Elements. Those elements which are considered to be universal and are adopted by a number of counties and international organizations are classified as the Major Elements. The other elements which are necessary for quality assurance are classified as the Supplementary Elements.

Ministries concerned are expected to improve the quality indications for their official statistics, and carry out self-evaluation of the quality, through which they improve the official statistics premeditatedly. In doing so, “Relevance” is regarded as a core element paying attention to the relation with the other elements, because official statistics are regarded as critical information for the citizens and it is important to take into account the needs of them widely.

These Guidelines are constantly reviewed based on the following matters in relation to quality assurance of official statistics: results of approaches by the ministries, results of research by the learned societies related, international trend, etc.

4.1.2 Implementation of the Guidelines

Based on the Guidelines, ministries concerned are expected to improve the indications of the statistics qualities under their control according to the “Quality indication items of official statistics” presented in Table 3, which are for statistics based on surveys. There is another table of quality indication items for statistics not dependent on surveys, but it is omitted here.

Table 3
Quality indication items of official statistics – For statistics based on surveys

Menu		Common publishing items
1	Outline of the survey	Purpose of the survey History of the survey Legal basis of the survey Survey population Sampling method Items surveyed Questionnaire Reference date Survey method (others)
2	Results of the survey	Explanation of terms Outline of the results Method of data processing and estimation Notes for users Notices of corrections List of statistical tables Utilization practices (others)
3	Release schedule
4	FAQ
5	Contact information
6	(Information in the past)
7	(Other notes)

Source: “The Guidelines for Quality Assurance of Official Statistics”.

In addition to the above quality indications, ministries are expected to carry out premeditated self-evaluations of the quality of the official statistics under their control based on the “Quality evaluation items of official statistics” in the Guidelines, of which items are categorized according to the quality elements listed in Table 2.

Furthermore, based on the latest Guidelines which were amended on February 23, 2016, ministries concerned are also expected to implement self-evaluations of the quality for the implementation process of statistical surveys based on the “Quality evaluation items for the implementation process of statistical surveys” in the Guidelines. These items are divided into five processes: I Basic principles, II Management of survey planning, III Data collection, IV Managing and processing data, and V Report of the survey. As an example of the contents for evaluation in the processes, the items for III Data collection are shown in Table 4.

Table 4

Quality evaluation items for the implementation process of statistical surveys – For items on data collection

Items
1	Grasping implementation status of the collection work of questionnaires
2	Data collection using ICT
14	Record on the results of questionnaires collection
3	Publicity and explanation for survey	3.1	Publicity and explanation for survey respondents about confidentiality
	respondents	3.2	Publicity and explanation for survey respondents about the purpose and importance of
			the statistical survey
4	Consideration for survey respondents
5	Manifestation of the utilization status of administrative record information to users
6	Recruiting and appointing enumerators and their supervisors
7	Instruction and training for enumerators and supervisors	7.1	Basic instruction and training for newly appointed enumerators	7.1.1	Basic instruction and training for newly appointed enumerators
				7.1.2	Support to newly appointed enumerators
		7.2	Instruction and training for enumerators and supervisors
8	Issuance of ID cards for enumerators and supervisors
9	Explanation of duties for enumerators and supervisors
10	Individual instruction for enumerators and supervisors
11	Record on the collection status of questionnaires
12	Notes on rewards
13	Confirmation of the activity status of	13.1	Confirmation of the activity status of enumerators and supervisors
	enumerators and supervisors	13.2	Correction and recurrence prevention of problems on the work of enumerators and
			supervisors

Source: “The Guidelines for Quality Assurance of Official Statistics”.

When carrying out self-evaluations of the quality for the implementation process of statistical surveys, ministries are to receive reports concerning the implementation status on the items necessary to secure accuracy and reliability of official statistics from organizations such as local branches, local governments and private companies to which statistical work is entrusted from the ministries.

Ministries then utilize the results of the self- evaluation for reviewing and improving statistical surveys.

4.2 Assuring impartiality and transparency

As part of the efforts to assure impartiality and transparency of official statistics, the DGSS of the Ministry of Internal Affairs and Communications decided the “Guidelines on a range and a procedure for sharing statistical information before the release date” on May 12, 2010.

Leaking the data of official statistics before the release date leads to a loss of the people’s trust in the overall official statistics. To avoid this, ministries are recommended to follow the above guidelines and determine the range of persons who can share the statistical information before the date of release and make the list of the persons. Ministries are then expected to make public the range of persons by e.g. posting it on their website.

4.3 PDCA practice for quality assurance

The necessity to check the statistical accuracy of official statistics periodically was first suggested by the Statistics Commission on March 22, 2016. During fiscal 2016, which starts on April 1, 2016, the Commission deliberated about how to perform a premeditated check on statistical accuracy, which forms an important part of PDCA practice for quality assurance of official statistics.

On March 31, 2017, the Commission issued a report on the results of the deliberation [7], in which the Commission suggested a detailed method of checking the accuracy of statistics produced by each ministry. The outline of the suggestion is described below.

4.3.1 Outline of accuracy checks

With the objective of supporting the autonomous efforts of ministries that produce statistics, the MIC performs premeditated checks on the accuracy of statistics and reports the results of the checking to the Statistics Commission every fiscal year. These checks are done based on Article 55 of the Statistics Act. The procedure taken every year is as follows.

By the end of March every year, the Statistics Commission suggests the names of the statistics which should be checked during the next fiscal year starting in April. Then, from April through September, the MIC performs the checks based on the suggestion of the Commission. Lastly, around October, the MIC reports the result of the checks to the Statistics Commission.

The Commission then deliberates on the results and issues a report indicating a policy for solving the problem, if necessary. In response to the report of the Statistics Commission, ministries are encouraged to make plans to solve the issue and implement them.

This again forms a PDCA cycle of assuring the accuracy of official statistics in a practical manner (Fig. 3). In this case, however, the cycle starts with the “Check” stage, so this cycle could be called a “CAPDo” cycle.

Figure 3.

The CAPDo cycle for assuring accuracy of official statistics in practice.

4.3.2 Contents of accuracy checks

Accuracy checks consist of standard checks and optional ones.

The standard checks are to examine the status of publicity of information, so called “visualization” status, on the accuracy of each statistics based on the common standard. In concrete terms, for six items on statistical accuracy such as sampling design, survey method (data collection method), data processing and estimation method, sampling error, non-sampling error, and comparison and analyses with other statistics, the statuses of publicity such as being posted on the website are evaluated at four levels, e.g. unexplained, briefly explained, explained, and precisely explained. These standard checks are done in a manner consistent with the Guidelines for Quality Assurance of Official Statistics.

The optional checks are performed for the statistics which the MIC acknowledges the need to examine based on the suggestion of the Statistics Commission. The menu for the optional checks is as follows.

Check on the status of fitting to population

This check is done by comparing the composition ratios of basic attributes such as sex and age based on samples with those based on the population. If large differences are detected, the influences on the statistics are judged.

Analysis of difference from other statistics

In this analysis, the statistics concerned are compared with other statistics which seem to indicate a similar level or trend. If the level or trend shows notable discrepancy, the cause of it is clarified.

Check on missing data

The status of missing data occurrence and how to impute the missing values of the statistics concerned are checked and the influences of them on the released data are examined. In addition, the possibility to adopt an alternative imputation method is studied and its effect on the statistics, etc. is evaluated, if feasible.

Check by various simulations

This check is performed by running simulations such as resampling using individual data from questionnaires of the statistics concerned, in order to produce reference series data or clarify the effect of rotation sampling, etc.

Overall check

Overall check is to examine the implementation status of statistical surveys in light of comprehensive standpoints, of which examples are as follows: the number of responses, bias of responses, imputation and post-stratification in tabulation, follow-up survey and alternative samples, online survey, estimation of weighted response rates, budgets, response burden, survey objects, and re- porting period.

Special check

Based on the suggestion by the Statistics commission, the target statistics and the contents of the special checks are determined as necessary.

After implementing these checks, the MIC may make a proposal on a feasible improvement method where necessary, as these checks aim to promote the improvement of official statistics.

In the future, the contents of the standard checks, which are designed for mainly statistics based on surveys, will be reviewed so that they can be applied to other types of statistics such as those produced by compiling various statistics or produced through administration.

5. Concluding remarks

The quality of official statistics in Japan is assured by a multi-layered framework and continual practice under it. The legal part of the framework consists of several elements such as the Statistics Act, the Master Plan, examination of plans and budgets for censuses and statistical surveys, and the Statistics Commission, all of which are organically combined to form a PDCA cycle of statistical activities. In the practical part of the framework, the Guidelines for Quality Assurance of Official Statistics play a pivotal role in assuring the quality of statistics. In addition, PDCA practice for accuracy has been implemented in conformity with the Guidelines through systematic checking of statistics produced by censuses and statistical surveys.

It should be noted that perpetual implementation of the PDCA cycle is absolutely essential for the quality assurance of official statistics.

Footnotes

Fundamental Statistics are those designated by the Minister for Internal Affairs and Communications as particularly important official statistics for planning national policies and implementing them, etc. Official Statistics other than Fundamental Statistics, which are produced by the central government, are called General Statistics.

Acknowledgments

The author would like to thank his colleagues who provided him with comments and suggestions, in particular Mr. Yasunori Sawamura, Mr. Tatsuya Miyauchi and Mr. Naoki Makita of MIC. The author, however, takes responsibility for any errors or inaccuracies.

Appendix

Annex 1

Summary mapping of the UN generic national quality assurance framework template to Japanese framework

NQAF (UN)		Statistics Act		Master Plan (Basic perspectives)		Examination of plans and budgets for
						censuses and statistical surveys
3A. MANAGING THE STATISTICAL SYSTEM
NQAF 1	Coordinating the national statistical system	3-1	Mutual cooperation and the appropriate sharing of roles among administrative organs
NQAF 2	Managing relationships with data users and data providers	4-5	To reflect the opinions of the citizens (Master Plan)
NQAF 3	Managing statistical standards	28	Establishment of Statistical Standards	2	Securing and improving international comparability
3B. MANAGING THE INSTITUTIONAL ENVIRONMENT
NQAF 4	Assuring professional independence	3-2	Appropriate and reasonable methodologies
NQAF 5	Assuring impartiality and objectivity	3-2	To ensure neutrality and reliability
NQAF 6	Assuring transparency	8, 23	Publicize the fundamental/general statistics and related matters	5	Promotion of open statistical data and transparency of the statistical production process	8	Is there an explanation for respondents to understand the legal basis?
NQAF 7	Assuring statistical confidentiality and security	3-4	Confidential matters shall be protected			9	Are the procedures taken to protect confidentiality adequate?
		39	Appropriate management of questionnaire information
		41	Confidentiality
		57	Confidentiality (penalty)
NQAF 8	Assuring the quality commitment	3-2	Appropriate and reasonable methodologies
NQAF 9	Assuring adequacy of resources	3-2	Appropriate and reasonable methodologies
3C. MANAGING STATISTICAL PROCESSES
NQAF 10	Assuring methodological soundness	3-2	Appropriate and reasonable methodologies	4	Promotion of accurate and efficient production	1	Are the statistics produced by the existing survey data or administrative records?
						2	Is the population set out definitely?
						3	Is the survey frame adequate?
						4	Are sample surveys used instead of censuses when possible?
						5	Is the response burden kept reasonable?
						6	Is there a need for each question?
NQAF 11	Assuring cost-effectiveness	3-229	Appropriate and reasonable methodologies Accurate and efficient production of statistics	4	Promotion of accurate and efficient production	1 4	Are the statistics produced by the existing survey data or administrative records? Are sample surveys used instead of censuses when possible?

Annex 1, continued
NQAF (UN)		Statistics Act		Master Plan (Basic perspectives)		Examination of plans and budgets for
						censuses and statistical surveys
3C. MANAGING STATISTICAL PROCESSES
NQAF 12	Assuring soundness of implementation	3-2	To ensure neutrality and reliability	4	Promotion of accurate and efficient production	1	Are the statistics produced by the existing survey data or administrative records?
NQAF 13	Managing the respondent burden	3-2 29	Appropriate and reasonable methodologies Reduction of the burden on respondents	4	Promotion of accurate and efficient production	1 2 3 5 7	Are the statistics produced by the existing survey data or administrative records? Is the population set out definitely? Is the survey frame adequate? Is the response burden kept reasonable? Are the questions easy to understand?
3D. MANAGING STATISTICAL OUTPUTS
NQAF 14	Assuring relevance	3-3	Provided widely for the citizens so that they are easily accessible for their effective use	3	Adequate responses toward environmental changes in the economy and society	1 2 6	Are the statistics produced by the existing survey data or administrative records? Is the population set out definitely? Is there a need for each question?
NQAF 15	Assuring accuracy and reliability	3-3 29	Provided widely for the citizens so that they are easily accessible for their effective use Accurate and efficient production of statistics	1 4	Securing and improving compatibility between statistics Promotion of accurate and efficient production	6 7	Is there a need for each question? Are the questions easy to understand?
NQAF 16	Assuring timeliness and punctuality	3-3 8, 23	Provided widely for the citizens so that they are easily accessible for their effective use Punctuality and easy access			10	Can the statistics be more promptly released?
NQAF 17	Assuring accessibility and clarity	3-3 8, 23	Provided widely for the citizens so that they are easily accessible for their effective use Punctuality and easy access	5	Promotion of open statistical data and transparency of the statistical production process	11 13	Are all the statistics released? Are the statistics archived properly?
NQAF 18	Assuring coherence and comparability	3-3 28	Provided widely for the citizens so that they are easily accessible for their effective use Establishment of Statistical Standards	1 2	Securing and improving compatibility between statistics Securing and improving international comparability	12	Are the standards used appropriately?
NQAF 19	Managing metadata	3-3	Provided widely for the citizens so that they are easily accessible for their effective use	5	Promotion of open statistical data and transparency of the statistical production process

References

Japanese Government [homepage on the Internet]. Statistics Act. Act No. 53 of May 23, 2007 [cited 2017 Oct. 17]. Available from: http://www.japaneselawtranslation.go.jp/law/detail/%3Fvm=04&re=01&id=148.

United Nations Statistics Division [homepage on the Internet]. Fundamental Principles of Official Statistics (A/RES/ 68/261 from 29 January 2014) [cited 2017 Oct. 17]. Available from: https://unstats.un.org/unsd/dnss/gp/fundprinciples.aspx.

Conference of the Directors in charge of Statistics of Line Ministries [homepage on the Internet]. Guidelines for Quality Assurance of Official Statistics. Japanese Government [revised 2016 February 23; cited 2017 Oct. 17]. Available from: http://www.soumu.go.jp/main_content/000467813.pdf.

United Nations Statistics Division [homepage on the Internet]. National Quality Assurance Frameworks [cited 2017 Oct. 17]. Available from: https://unstats.un.org/unsd/dnss/QualityNQAF/nqaf.aspx.

Japanese Government [homepage on the Internet]. Master Plan Concerning the Development of Official Statistics [cited 2017 Oct. 17]. Available from: http://www.soumu.go.jp/english/dgpp_ss/seido/masterplan2014.htm.

Sawamura

Kubo

[homepage on the Internet]. The Quality Assurance Framework in Japan. Presented at European Conference on Quality in Official Statistics – Q2012; 2012 May 29 – June 1; Athens (Greece) [cited 2017 June 28]. Available from: http://www.q2012.gr/articlefiles/sessions/1.1_Kubo_Quality%20assuance%20framework%20in%20japan.pdf.

Statistics Commission [homepage on the Internet]. 2015 Report of the Deliberation Results on the Implementation Status of the Statistics Act (Deliberated in the second half of the fiscal year 2016); March 31, 2017 (in Japanese only) [cited 2017 Oct. 17]. Available from: http://www.soumu.go.jp/main_content/000476196.pdf.

The quality assurance of official statistics in Japan: Framework and practice

Abstract

Keywords

1. Introduction

2. Quality assurance framework

3.1 Statistics Act

3.1.1 Fundamental Principles

3.1.2 Managing the statistical system

3.1.3 Managing the institutional environment

3.1.4 Managing statistical processes

3.1.5 Managing statistical outputs

3.2 Master Plan

Table 1 Points to be checked and correspondence between the NQAF lines and them

3.4 Statistics Commission

3.5 PDCA cycle in the framework

Table 2 Quality elements and their definitions in official statistics

4.1 Guidelines for quality assurance

4.1.1 Basic principles of the Guidelines

4.1.2 Implementation of the Guidelines

Table 3 Quality indication items of official statistics – For statistics based on surveys

4.3 PDCA practice for quality assurance

4.3.1 Outline of accuracy checks

5. Concluding remarks

Footnotes

Acknowledgments

Appendix

References

Table 1
Points to be checked and correspondence between the NQAF lines and them

Table 2
Quality elements and their definitions in official statistics

Table 3
Quality indication items of official statistics – For statistics based on surveys