The quality assurance of official statistics in Japan: Framework and practice

3.1.1 Fundamental Principles

The fundamental principles of the Statistics Act are stipulated in Article 3 of the Act, which are provided as follows.

(Fundamental Principles)

Article 3

Official statistics shall be developed systematically through mutual cooperation and the appropriate sharing of roles among administrative organs, etc.

These principles are well compatible with the FPOS and the philosophy of the NQAF of the UN. In concrete terms, Paragraph 1 of Article 3 is compatible with Principle 8 of FPOS, which emphasizes the coordination among statistical agencies within countries. Paragraph 2 is in line with Principles 1, 2, and 5 of FPOS which stipulate impartiality, trustworthiness, and appropriateness of sources, respectively. Paragraph 3 states effective use, and this is compatible with Principle 1 and also Principle 3 of FPOS, the latter of which claims the necessity to present information for correct interpretation of the data. Lastly, Paragraph 4 is in line with Principle 6 of FPOS, which stipulates confidentiality of individual data.

The next several subsections describe how articles of the Act correspond to NQAF lines with respect to each grouping (category) of the NQAF.

3.1.2 Managing the statistical system

Articles of the Act that relate to managing the statistical system, the first category of NQAF, are Articles 3, 4, and 28. Paragraph 1 of Article 3 corresponds to coordinating the national statistical system prescribed in NQAF 1. Paragraph 5 of Article 4 which stipulates the Minister for Internal Affairs and Communications to take necessary measures to reflect the opinions of the citizens when intending to prepare a draft Master Plan relates to managing relationships with data users and data providers stipulated in NQAF 2. Article 28 that is relevant to the establishment of Statistical Standards is compatible with managing statistical standards in NQAF 3.

3.1.3 Managing the institutional environment

In this category of NQAF, Paragraph 2 of Article 3 of the Act is compatible with NQAF 4 to 5 and 8 to 9. Using appropriate and reasonable methodologies stipulated in Paragraph 2 of Article 3 leads to assuring professional independence (NQAF 4), assuring the quality commitment (NQAF 8), and assuring adequacy of resources (NQAF 9). This paragraph is also to ensure neutrality and reliability, which are connected to assuring impartiality and objectivity (NQAF 5).

According to Article 8 and 23 of the Act, the head of an administrative organ shall, when having produced fundamental/general statistics, promptly publicize the statistics and related matters through the Internet, etc., of which description is in line with assuring transparency (NQAF 6).

Protection of confidential matters and its punitive clauses are prescribed in Paragraph 4 of Article 3 and Articles 39, 41, and 57, all of which are compatible with assuring statistical confidentiality and security of NQAF 7.

3.1.4 Managing statistical processes

Managing statistical processes, the third category of NQAF lines, consists of assuring methodological soundness (NQAF 10), assuring cost-effectiveness (NQAF 11), assuring soundness of implementation (NQAF 12), and managing the respondent burden (NQAF 13).

Again, Paragraph 2 of Article 3 of the Act is compatible with this category. This Paragraph states the necessity of using appropriate and reasonable methodologies, which directly relates to NQAF 10. It also leads to cost-effectiveness (NQAF 11) and managing the respondent burden (NQAF 13). Besides, assuring soundness of implementation (NQAF 12) is the prerequisite for ensuring neutrality and reliability stipulated in this Paragraph.

Article 29 of the Act is the provision that a ministry may request other ministries and agencies (hereinafter called “ministries”) to provide administrative record information when finding that utilizing such information would contribute significantly to the accurate and efficient production of statistics or to the reduction of the burden on respondents of statistical surveys. This provision is compatible with assuring cost-effectiveness (NQAF 11) and managing the response burden (NQAF 13).

3.1.5 Managing statistical outputs

The last category of NQAF lines is managing statistical outputs. Intimately associated with this category is Paragraph 3 of Article 3 of the Act. Especially, effectiveness of use stated in this Paragraph relates to assuring relevance (NQAF 14), assuring accuracy and reliability (NQAF 15), and managing metadata (NQAF 19). Accessibility also prescribed in this Paragraph is compatible with assuring accessibility and clarity (NQAF 17).

From the viewpoint of punctuality and clarity, Articles 8 and 23 of the Act stipulate prompt publicity and easy access of official statistics. These are in line with assuring timeliness and punctuality (NQAF 16) as well as accessibility and clarity (NQAF 17).

In addition, Article 28 of the Act, which concerns establishment of statistical standards, assures coherence and comparability (NQAF 18).

4.1.1 Basic principles of the Guidelines

The quality of official statistics consists of various elements including accuracy, relevance, and timeliness. Accuracy means that official statistics show the state of the society and economy as accurately as possible. Relevance indicates that official statistics meet the users’ needs as far as possible, and timeliness means that the produced official statistics are disseminated in a timely fashion in accordance with users’ needs and purposes.

The Guidelines assume that the quality of official statistics comprises the elements listed in Table 2, and these elements are used as indexes for indicating and evaluating the quality of statistics. The elements are divided into two categories: Major Elements and Supplementary Elements. Those elements which are considered to be universal and are adopted by a number of counties and international organizations are classified as the Major Elements. The other elements which are necessary for quality assurance are classified as the Supplementary Elements.

Ministries concerned are expected to improve the quality indications for their official statistics, and carry out self-evaluation of the quality, through which they improve the official statistics premeditatedly. In doing so, “Relevance” is regarded as a core element paying attention to the relation with the other elements, because official statistics are regarded as critical information for the citizens and it is important to take into account the needs of them widely.

These Guidelines are constantly reviewed based on the following matters in relation to quality assurance of official statistics: results of approaches by the ministries, results of research by the learned societies related, international trend, etc.

4.1.2 Implementation of the Guidelines

Based on the Guidelines, ministries concerned are expected to improve the indications of the statistics qualities under their control according to the “Quality indication items of official statistics” presented in Table 3, which are for statistics based on surveys. There is another table of quality indication items for statistics not dependent on surveys, but it is omitted here.

In addition to the above quality indications, ministries are expected to carry out premeditated self-evaluations of the quality of the official statistics under their control based on the “Quality evaluation items of official statistics” in the Guidelines, of which items are categorized according to the quality elements listed in Table 2.

Furthermore, based on the latest Guidelines which were amended on February 23, 2016, ministries concerned are also expected to implement self-evaluations of the quality for the implementation process of statistical surveys based on the “Quality evaluation items for the implementation process of statistical surveys” in the Guidelines. These items are divided into five processes: I Basic principles, II Management of survey planning, III Data collection, IV Managing and processing data, and V Report of the survey. As an example of the contents for evaluation in the processes, the items for III Data collection are shown in Table 4.

When carrying out self-evaluations of the quality for the implementation process of statistical surveys, ministries are to receive reports concerning the implementation status on the items necessary to secure accuracy and reliability of official statistics from organizations such as local branches, local governments and private companies to which statistical work is entrusted from the ministries.

Ministries then utilize the results of the self- evaluation for reviewing and improving statistical surveys.

4.3.1 Outline of accuracy checks

With the objective of supporting the autonomous efforts of ministries that produce statistics, the MIC performs premeditated checks on the accuracy of statistics and reports the results of the checking to the Statistics Commission every fiscal year. These checks are done based on Article 55 of the Statistics Act. The procedure taken every year is as follows.

By the end of March every year, the Statistics Commission suggests the names of the statistics which should be checked during the next fiscal year starting in April. Then, from April through September, the MIC performs the checks based on the suggestion of the Commission. Lastly, around October, the MIC reports the result of the checks to the Statistics Commission.

The Commission then deliberates on the results and issues a report indicating a policy for solving the problem, if necessary. In response to the report of the Statistics Commission, ministries are encouraged to make plans to solve the issue and implement them.

This again forms a PDCA cycle of assuring the accuracy of official statistics in a practical manner (Fig. 3). In this case, however, the cycle starts with the “Check” stage, so this cycle could be called a “CAPDo” cycle.

OPEN IN VIEWER

Accuracy checks consist of standard checks and optional ones.

The standard checks are to examine the status of publicity of information, so called “visualization” status, on the accuracy of each statistics based on the common standard. In concrete terms, for six items on statistical accuracy such as sampling design, survey method (data collection method), data processing and estimation method, sampling error, non-sampling error, and comparison and analyses with other statistics, the statuses of publicity such as being posted on the website are evaluated at four levels, e.g. unexplained, briefly explained, explained, and precisely explained. These standard checks are done in a manner consistent with the Guidelines for Quality Assurance of Official Statistics.

The optional checks are performed for the statistics which the MIC acknowledges the need to examine based on the suggestion of the Statistics Commission. The menu for the optional checks is as follows.

Check on the status of fitting to population

This check is done by comparing the composition ratios of basic attributes such as sex and age based on samples with those based on the population. If large differences are detected, the influences on the statistics are judged.

After implementing these checks, the MIC may make a proposal on a feasible improvement method where necessary, as these checks aim to promote the improvement of official statistics.

In the future, the contents of the standard checks, which are designed for mainly statistics based on surveys, will be reviewed so that they can be applied to other types of statistics such as those produced by compiling various statistics or produced through administration.