Diverse ideas about the value of Official Statistics systems

1. Introduction

As with other forms of public service activity, there is an ongoing need to understand the value of Official Statistics. One inevitable driver is the importance of being able to demonstrate that value in order to protect or extend public investment in Official Statistics – recognising that funds allocated to Official Statistics are funds that are therefore not (directly) allocated to other public goods, including those goods whose value might be more easily asserted or quantified (such as healthcare or education). A further driver is the benefit gained from having clarity about the value to be delivered so that decisions made about and within Official Statistics systems can be focused on maximising that value. These two factors are neatly summarised as the requirements to both ‘prove’ and ‘improve’ in the important work on the value of Official Statistics steered by the Conference of European Statisticians (CES) [1].

That CES report concludes, correctly I believe, that user perspectives are a fundamental component of any work to express or to quantify the value of Official Statistics. Looking through this lens may lead to a focus on the value of specific sets of statistics to specific sets of users, given that most users do not tend to access the full suite of any given Official Statistics portfolio.

The purpose of this paper is to offer a complementary perspective, gathering ideas about the broad society-level value that might be generated by, or linked to, Official Statistics systems as a whole. Official Statistics systems are complex groupings of organisations – those who produce Official Statistics, those engaged in signposting and disseminating, those who use Official Statistics in different ways and those concerned with the overall direction and governance of these systems.

One way of gathering ideas about the different potential types of value of these systems is to look at the formal statements made about the outcomes that they aim to deliver or enable. Hence this paper draws on a sample of recent corporate documents (for example Strategies and Corporate Plans) produced by National Statistical Offices (NSOs) and other relevant bodies and some complementary documents produced by international statistical organisations.

In reviewing these documents, the aim has been to seek out any statements of commitment or ambition about the tangible, real-world outcomes that a well-functioning Official Statistics system might deliver – or assist in enabling. Hence the aim is to focus externally – on concepts that might be recognised by those outside the world of Official Statistics rather than on more internally facing outcomes (high quality statistics, better statistical capability, trust in Official Statistics etc). The aim is not to rigorously analyse the frequency of use of the different ideas but to draw out some of the diversity of the different types of outcome that are currently referenced.

This paper firstly touches on some of the foundational, high-level ideas about value that inform many formal narratives about Official Statistics. It then looks in more detail at further expressions of tangible outcomes – for example, for citizens, for governments and more broadly. It then concludes with some observations about the resulting set of ideas.

2. Foundational ideas about the value of Official Statistics systems

There are two high-level ideas that are commonly seen across the various ambitions set out for Official Statistics systems – or for NSOs more specifically.

There is a regular framing, or assertion, of Official Statistics as an essential component of a modern state. Official Statistics are presented as “an indispensable element for the creation of a democratic and informed society” in the UN Fundamental Principles of Official Statistics [2]. In a similar vein, Paris 21 [3] states that “the capacity to produce official statistics is a core function of the modern state”.

This view of the essential nature of Official Statistics is also reflected in some NSO strategy documents where Official Statistics is often presented as an element of national infrastructure. For example:

•

Statistics Japan [4] refers to Official Statistics as an “information platform for society”.

•

Statistics Finland [5] talks about Official Statistics as “our shared national capital.”

•

Statistics Netherlands [6] describes its task as “creating an independent and objective map of society.”

The other common theme replicated across many corporate documents is the reference to the central purpose of Official Statistics being to enable decision-making – both within government and beyond. For example:

•

Statistics Canada [7] sets out the aim of ensuring that “Canadians have the key information they need to make evidence-based decisions about the economy, society and the environment.”

•

The Singapore Department of Statistics [8] states that their role is to “deliver insightful statistics and trusted statistical services that empower decision making.”

•

The ambition of Stats South Africa [9] is that their work will have impact through “evidence-based decisions that promote citizenry and inform policy development, planning, monitoring and evaluation to create a better life for all.”

•

The Philippine Statistics Authority’s vision [10] is for a “Responsive, relevant, and innovative statistical system that provides quality statistics for evidence-based planning and decision-making towards improved and sustained quality of life for all.”

None of these statements claim exclusivity – ie that Official Statistics are the only form of evidence that might (or should) inform decisions. There is, however, a consistent underlying assertion which is that statistics-informed decisions – whether one-off policy/programme decisions or those that emerge from regular monitoring – will be (on average) “better” decisions and that in turn these better decisions will lead to better (social, economic, environmental) outcomes.

3. Value for citizens

Some statements of ambition and strategy for Official Statistics systems are explicit about the intention to create specific outcomes for citizens of nation states. These ambitions are expressed in a range of ways.

In some countries, the core vision for the Official Statistics system is simply that citizens are informed – about their world and the way that it is changing. For example:

•

Statistics Iceland [11] aims to enable an “informed society”

•

Statistics Estonia [12] state that “We help people understand the world better through data.”

Beyond the broad ambition of an informed society, some ambitions capture the concept of different groups of citizens being represented or ‘seen’ in the statistics that seek to describe their country. Hence:

•

The UK Statistics Authority [13] sets out the ambition for Official Statistics to “reflect the experiences of everyone in our society so that everyone counts, and is counted, and no one is forgotten”.

•

Statistics Sweden [14] notes their commitment that “Our statistics and data describe Sweden so that citizens can also identify at the regional and local level.”

Probably the most common active role envisaged for citizens (i.e. what individual citizens might actually do with the Official Statistics that are available to them) tends to be reflected in the term ‘holding government to account’. Hence:

•

The UK Statistics Authority [13] notes that “Evidence …informs decision-makers to make choices and lets others hold the powerful to account”.

•

Stats NZ [15] notes the aim that “government is held to account through a robust and transparent data system”.

4. Value for (national) governments

The historical role of Official Statistics has been to enable a state to carry out what it determines to be its basic functions – for example, allocating resources (via population estimates) or estimating taxation returns (through data about businesses and individuals). Indeed statistics have been described [16] as “tools designed to simplify the job of government, for better or worse.” In essence this value for governments centres on the decision-making rationale covered earlier.

Beyond referencing the contribution of Official Statistics in enabling the state to make better decisions, some statements of ambition reference the role of Official Statistics in enabling the overall process through which policy is made and implemented. For example:

•

Statistics Japan [4] aims to support “the planning, formulation and evaluation of governmental policies.”

•

The National Bureau of Statistics (Tanzania) [17] notes the importance of good statistics in facilitating “evidence-based policy formulation, planning, monitoring and evaluation which are key in the realization of development aspirations.”

These statements of ambition suggest a distinction between the value of Official Statistics in informing better decisions – by which we might mean decisions that are more likely to lead to the outcomes intended by the decision-maker – and the value of Official Statistics in enabling a better overall governmental decision-making process. In the latter case, the implication is that a more consistent and resilient process (through which governments understand the circumstances and needs of citizens, businesses and places and seek to develop and implement appropriate policies, programmes and services in response) is itself of value – not least in building wider legitimacy and trust in decision-making.

Some statements of ambition also touch on the related role of Official Statistics in providing transparency (about the evidence used in decision-making and about resulting outcomes) and hence in enabling decision-makers to be held to account:

•

The United Nations [18] presents Official Statistics as “an essential element of the accountability of governments and public bodies to the public in a democratic society.”

•

The National Statistical Office of Papua New Guinea [19] sets out its commitment to enable Government to “promote good Governance and be held accountable on its performance with statistical information.”

•

The Ghana Statistical Service [20] notes the aspiration to be positioned “as the trusted provider of Official Statistics for good governance.”

5. Value for public debate

A linked ambition for Official Statistics, which combines outcomes for both governments and for citizens, is that they enable healthy, well-informed public debate. For example:

•

Ireland’s Central Statistics Office [21] sets out its aim to “support and promote understanding and debate across government, business and society”.

•

The Australian Bureau of Statistics [22] states that “ABS data and statistics …support a strong well-functioning democracy and provide reliable information on a range of matters critical to public debate.”

•

One of the aims of the Tonga Strategy for the Development of Statistics [23] is to “Encourage the development and practice of informative debate.”

•

Statistics Norway [24] see Official Statistics as “providing a basis for …general discussion in society.”

As with formal decision-making, there is perhaps a distinction to be drawn between the ‘correctness’ of otherwise of any final conclusions reached as a result of a public debate and the process of that debate, where the intended value of Official Statistics is seen as lying in ensuring that such debates utilise, and are grounded in, trustworthy information.

6. Value for the national economy

At the macro level, the existence and credibility of a national Official Statistics system has been linked to positive outcomes for the wider (national) economy. Stats NZ [25] notes that:

“The integrity of official data can have wide-ranging implications, from such intangible factors as a country’s standing on the world stage through to more measurable effects, such as the interest charged on government borrowing.”

The implication is that any such economic value – for example the ability of a nation state to seek more affordable borrowing – comes not from any specific economic statistic but from the wider existence of a well-functioning Official Statistics system.

Similarly, at a lower level, a well-functioning system may be one factor in encouraging specific private sector investment in specific countries, localities or business sectors. Hence, for example:

The National Statistical Office of Papua New Guinea [19] note their intended focus on “private sector investors who want to use data and statistics to aid investment decisions”.

7. Observations

There are clearly a number of limitations to the above summary of potential ambitions expressed for Official Statistics in different national and international documents. The analysis draws on a relatively small sample of Official Statistics Strategies, Corporate Plans and the like (accessed from sources making these documents available in the English language).

Clearly any analysis of these written commitments or ambitions needs to acknowledge that the source documents will reflect a range of political, cultural and administrative influences and that a number of the documents studied have been translated from their original language. Recognising these limitations, there are, nonetheless a few provisional observations that can be made on the basis of this initial exercise.

•

There is, almost uniformly, a high level of ambition for Official Statistics systems. Although no individual strategy or corporate plan explicitly commits to the full suite of outcomes listed above, there is, nonetheless, a common theme of seeking to deliver important (and challenging) outcomes in the ‘real’ world.

•

There is strong level of coherence in the stated ambitions across different countries – which may reflect the ongoing influence of the Fundamental Principles of Official Statistics [2].

•

There is however some diversity of ambition to recognise – for example, some corporate documents do explicitly reference the intended contributions to well-informed public debate or to national economic growth while others do not. This may reflect the fact that, at the national level, there may well be ongoing and legitimate choices (perhaps related in part to levels of funding) that individual countries, or individual organisations, are making – for example about the relative balance of focus on governmental or citizen needs.

•

In most cases, Official Statistics organisations have not, to date, committed to ongoing measurement of the sort of higher-level system-wide outcomes listed above – hence, for example, there are no instances captured to date of producers seeking to regularly measure the extent to which, for example, their work has successfully enabled better public debate or has improved national economic performance.

•

That absence of quantification may also relate to the broad nature of some of these outcomes and to the fact that the contribution of an Official Statistics system (to outcomes such as “an informed society” or “well-informed public debate”) may only be one contribution amongst many. Hence the corporate documents studied tend not to suggest that, in broad outcome terms, the impact of Official Statistics is necessarily distinctive or even unique.

8. Next steps

The purpose of this initial paper is simply to offer an initial exploration of some of the ideas that we might consider when seeking to express the value of Official Statistics at a broader, system level. This initial exploration could, of course, be expanded – both in terms of the range of sources considered and in terms of the rigour of the framework used to collate and present themes. Any such expansion might also be valuable in exploring further outcome areas beyond the initial set listed here – for example looking more closely at expressions of tangible international-level outcomes.

Any such broadening of this sort of analysis could be part of, or linked to, existing work on the value of Official Statistics, for example the work currently led by the CES. There is a potentially positive combination of the analysis of desired high-level outcomes and the collation of more specific expressions of value – notably of the value of specific groups of products to specific groups of users.

User-driven analyses of value may generate more tangible quantifications of impact – as the logic chain of Official Statistics enabling better decisions which in turn enable better societal, economic or environmental outcomes may be better demonstrated in specific case studies or for segments of any given Official Statistics portfolio. A system level categorisation of value could then provide some context – to understand the types of value that are being captured by that user-focused work and those that are not. This in turn might inform the development of any broader framework describing and categorising the different forms of value that Official Statistics systems might seek to deliver.

A further contribution that this sort of analysis of potential outcomes might provide is to suggest ways in which Official Statistics bodies might seek to gather feedback on the wider impact of the Official Statistics system as a whole. There are already exercises in different countries to gather public and stakeholder feedback on issues such as the trustworthiness of Official Statistics. There may be an opportunity to gather views on wider ambitions – for example about the extent to which citizens (or stakeholders) feel better informed, represented or empowered as a result of the Official Statistics that they access or use.

Finally there is the question of who should decide on the balance of outcomes (or priorities) set for any given Official Statistics system at any given point in time. In general, legislation will often provide the overall framing for any subsequent Strategies and Corporate Plans. Thereafter, as highlighted above, there may be a rich set of different areas of potential focus – which may suggest that Official Statistics organisations might, from time to time, consider wider public engagement about the outcomes that the system as a whole should seek to deliver.