Training official statisticians for adaptive statistical practice

1. Introduction

During the past two decades, the world has experienced momentous change in the way it produces, processes, and uses information. This transformation has been fuelled by technological progress in data storage, electronic communications, and computational power, and its effects are manifest in our everyday lives, from targeted advertising to personalized medicine. All together these developments are often characterized as the Data Revolution.

The digital and data revolutions have had important implications for official statistics. The monopoly held by official statistics, to provide timely and high-quality statistics for free is being challenged by the proliferation of readily available, ad-hoc and fragmentary data [1]. This and other social, environmental and economic changes, such as, globalisation, digitalisation, climate change, ideological battles as to whether data are a public or a proprietary good and changing attitudes to data privacy pose significant challenges for official statistics [2]. The continued relevance of official statistics will rely on its capacity to adapt in the midst of constant change, and a critical factor in this is the training of new official statisticians to ensure they understand the political – social – economic environment in which they operate.

University training of official statisticians has traditionally focused on mathematical statistics, especially core topics such as probability theory, statistical modelling, and methods for sampling and inference. This is typically performed in the context of a Bachelors’ and/or Masters’ degree in Statistics, although specialized programs such as the recent European Masters in Official Statistics (EMOS) have more targeted curricula [3]. Although the term ‘statistics’ started as synonymous with official demographic and economic data, the scientific field of Statistics eventually assumed a much broader scope, becoming the foremost discipline tasked with extracting knowledge from general data. This primacy, however, is now being challenged from emerging fields such as Machine Learning, Data Mining, Data Science, and Data Analytics, which claim similar goals. This has caused considerable soul-searching within the discipline, as Statistics is trying to rediscover its place in this new competitive world. But there is also a growing consensus that Statistics needs to embrace recent developments, such as algorithmic learning, while upholding its traditional strengths and, most importantly, its principled approach to data collection and inference. For example, the American Statistical Association (ASA) issued a statement on the role of Statistics in Data Science [4] calling for a ‘sustained and substantial collaborative effort’ in which ‘statisticians must engage, learn from, teach, and work with ‘researchers with expertise in data management and computation.’ In their epilogue, Efron and Hastie [5] offer a historical perspective of Statistics’ shifting focus between applications, mathematics, and computation, and provide examples of recent developments fuelled by applications and computation, the type of which will guarantee the sustained importance of the field.

Besides recent developments in statistical research and practice, there are significant efforts underway to update statistical education. There have been calls to broaden undergraduate curricula going back to at least Cobb [6], who recommends emphasizing statistical thinking using real data and active learning. More recently, Horton and Hardin [7] recognized that ‘the traditional statistics curriculum with mathematical foundations has not kept up with pressing demands for students who can make sense of data’ and call for curricula that ‘prepare students to engage in the entire data analysis process’. Specifically, Nolan and Temple Lang [8] argue that ‘computational literacy and programming are as fundamental to statistical practice and research as mathematics’ and propose a more central role for computing in teaching statistics. While these calls have been around for some time, the emergence of ‘Data Science’ has given new impetus and urgency to their adoption. Many academic departments are now implementing reforms aimed at balancing theory and methodology with computing and statistical practice, along the lines summarised in the most recent ASA curriculum guidelines [9]. These reforms are facilitated by the ready availability of data, computational power, and open-source software, which have opened up new ways of teaching statistics (e.g., simulation-based inference) and integrating empirical work in the classroom (e.g., collaborative projects).

A major motivation for the ongoing reforms is not just to teach students new skills and procedures, but to fundamentally develop their ability to understand, evaluate, produce, and communicate statistical results and arguments. The terms statistical literacy, statistical reasoning, and statistical thinking have all been used to describe overlapping aspects of this ability; see Garfield [10] for definitions and delMas [11] for a comparison. For the purposes of this paper, we adopt an expanded view of literacy, which is closer to that put forward by Gal [12], who uses literacy to suggest ‘a broad cluster not only of factual knowledge and certain formal and informal skills, but also of desired beliefs, habits of mind, or attitudes, as well as general awareness and a critical perspective’. Obviously, statistical literacy is a primary learning outcome for educational programmes in statistics, and a critical requirement for any aspiring official statistician working in a National Statistical Office (NSO) or as part of the National Statistical System (NSS). But statistical literacy also relies upon other abilities, more fundamentally on general literacy and numeracy, as well as technological and information literacy. General literacy includes reading and writing, but also a wider appreciation of history, geography, legal issues, ethics and culture. Gore [13] describes reading and writing as ‘democracy’s calisthenics’, and argues that the decline in literacy has impacted our ability to reason and think. More generally, a statistician should understand the broader context of what is being measured – including the history and the appropriateness of the measurement tool. Walter Radermacher [14], former Director General of Eurostat, correctly notes that ‘statistical competence and literacy must not be reduced to mathematical-technical skills. Rather, it also requires an understanding of the fundamentals of the humanities.’ Moreover, information and technological literacy, or the ability to access, evaluate, and use information while making effective use of technology, are becoming essential prerequisites for other forms of literacy.

Even with such bold reforms underway, the rapidly changing landscape of Statistics and Data Science, coupled with fundamental social, economic, environmental and other changes, guarantee that new generations of official statisticians will face conditions and challenges never seen before. In such a dynamic environment, it is important to anticipate the direction of future developments, but also to be able to adapt to unforeseen conditions; in this paper, we try to address both challenges. The following section identifies some of the core skills and central themes that will be essential in the training of official statisticians going forward. Next, we claim that alongside new content and skills, our programs should also cultivate an adaptive mindset, one that will prepare students to apply their knowledge flexibly and creatively, and to learn on their own. Finally, we present strategies for cultivating both core skills and an adaptive mindset, with examples in the context of university courses in Statistics.

2. Requirements of official statistics

It is very difficult to anticipate what specific skills NSOs and NSSs will require in the future. But at this juncture it is hard to imagine that the ‘hard’ statistical skills required of a statistician today will not be as relevant tomorrow. Arguably, these core statistical skills will become more important, rather than less, as graduates become more accustomed to using ‘blackbox’ software packages, but perhaps less accustomed to thinking about the underlying concepts and methodologies. While hard skills will most likely remain an essential ingredient for any statistician in the future, perhaps the necessary skills mix will change. We anticipate that three essential skills will be required: numerical skills; statistical skills; and increasingly, technological skills. Irrespective of whether we are discussing a data scientist, an official statistician or a professional statistician in another field, the requirement for these basic skills is universal. Mathematical and numerical skills should be, more or less, self-explanatory, but crucially a statistician should be able to spot patterns, understand differences between stocks and flows and be comfortable reading and writing in scientific notation. Having statistical skills means being able to work with real, often messy or incomplete data and understanding bias; both the likely sources and what remedial actions can be taken. Statisticians should understand the subtle but important differences between accuracy and precision and also develop a good understanding of concepts like uncertainty and risk. A competent statistician should be able to select and use appropriate statistical techniques and models. Future technological skills are the area hardest to predict. Technology is changing rapidly, with consequences not only for the applications we will use, but also the types of data we may have access to. Here it is very hard for a university to prepare courses for the future and for statistical offices to say with any certainty what will be required. If current trends have anything useful to say, then they suggest a greater use to ‘freeware’ and combining packages. They also suggest a commitment to lifelong learning will be essential.

OPEN IN VIEWER

Any discussion of future requirements should be broadened beyond skills to also include competencies. Statisticians, like any other professionals will need to continually update their skills over the lifetime of their careers. What are less likely to change over time are the basic characteristics or competencies necessary to be a good statistician (see Fig. 1). Specifically, a statistician must be creative, curious, critical, skeptical and resourceful. These are self-explanatory and do not require any elaboration. Perhaps less obvious, a statistician should also be aware of the cultural and civic or political environment in which they operate. It is very important that a statistician not only understands the context in which previous indicators and statistics were compiled, but also that they properly understand the opportunities and constraints that their environment imposes on their practice. For example, when contemplating the use of big data, NSOs may be forced to confront issues before the law is clear or cultural norms have been established. Given the importance of public trust for an NSO, it is essential that statisticians are sensitive to these issues and understand what is acceptable by the public they serve. The challenge for universities is how to nurture and develop competencies.

Statisticians must understand the underlying logic of theory, so that having acquired skills, they can apply them and put theory into practice in a variety of real-life situations (all invariably more complex and messy than the scenarios presented in text books). Other skills, perhaps neglected in the past, but now universally recognized as important, is the ability to communicate well and to present statistics in context. In a world awash with data and cluttered, incoherent messaging, the ability to translate data into comprehensible statistics and digestible messages is absolutely essential. Data visualisation is an important element subset of this skill, but perhaps one where too much emphasis is being placed at the moment. A statistician can only design an effective visualization if they are clear themselves what the key messages are.

A university course that prepares statisticians for a career in official statistics would ideally cover four broad domains: statistical systems; statistical techniques; policy issues; and for want of a better heading, the political economy of data. Thus, ‘official statistics literacy’ incorporates a wider set of domains than the more general concept of statistical literacy presented above. Including this wider mix of issues would enrich the content of university courses, stimulate discussion and give statistics students a better flavor of what their future career might involve.

Statistical Systems. A modeule in statistical systems would include: an introduction to the official statistics eco-system that comprises national, regional and global statistical systems; an outline of the mandate and functions of the UN Statistical Commission, the UN regional Commissions and UN specialised agencies, funds and programmes and other important international organisations, such as, the World Bank Group, the International Monetary Fund (IMF), the Organisation for Economic Cooperation and Development (OECD) and Eurostat. The module would explain how these institutions coordinate their work; it would also provide an explanation of the importance of internationally recognized classifications, such as ISIC, CPC, SITC, COICOP, or EBOPS1 and framework methodologies, such as, the System of National Accounts (SNA), Balance of Payments (BoP), System of Environmental-Economic Accounting (SEEA) or International Merchandise Trade Statistics (IMTS), and how they have been developed and continue to evolve.

Statisticians should also be introduced to: the UN Fundamental Principles of Official Statistics [16], the Principles Governing International Statistical Activities [17] and any regionally or nationally appropriate adaptations; professional principles, such as, the ISI Declaration of Professional Ethics [18]; and internationally recognized statistical quality assurance frameworks and codes of practice, such as, the UN Statistical Quality Assurance Framework [19], or the UN National QAF [20] and any regionally or nationally appropriate adaptations. An important element of any statistician’s work (whether official or otherwise) will be dealing with legislation. Official statisticians will need to be conversant with the national statistical law (and those in Europe will also need to know their way around European statistical legislation and GDPR2). Increasingly, concerns regarding privacy and confidentiality (terms often treated as synonymous but in fact very different things) mean that all statisticians and data scientists must understand the legal environment in which they work, in particular, data protection and freedom of information legislation and how they interrelate. But equally, statisticians must understand the human rights arguments regarding access to information (Resolution 59 of the UN General Assembly adopted in 1946 and Article 19 of the 1948 Universal Declaration of Human Rights). As we expect more and more secondary digital data to be generated, understanding of the legal issues of accessing and using these data are critical to ensuring a statistician does no harm [21]. Statistical courses could also usefully devote some time to discussing terms like ‘harm’ and ‘good’ and what they mean.

Many other interesting debates are underway, not least, the ‘End of Theory’ argument posited by Anderson [44] that with the emergence of big data, correlation has superseded causation. In other words, hypothesis driven science is redundant. Whether one agrees or not, this is an argument worth exploring. Should ‘open data’ exclusively target public sector data, or should it also push for more openness with private sector data. Are there risks to not pursuing a more balanced policy? Is ‘Fake News’ undermining the credibility of science and official statistics and how can NSOs tackle it without themselves becoming politicised? Has official statistics failed in its role – why for example, in the face of overwhelming statistical evidence is the climate crisis not being taken as seriously as it should? There are interesting ideas, such as data infrastructure, to be discussed where state information can be organized into registers supported by unique identifiers [45]. This is a very logical, effective and efficient approach but requires some sacrifice of privacy to the state. What are the cultural or other barriers to this? There are so many fascinating debates that students could and should be introduced to, so that they understand the fascinating range of opportunities available to them. For example, there is much talk today of Data Revolution [46], but what does that mean? Has there really been a revolution or is the data deluge just another step in the natural evolution of data [2]?

We have seen how advances in the production and processing of data are catalysing changes in statistical education, placing increased emphasis on computation and applications. And in the last section, we identified specific requirements for training of official statisticians, both traditional and with an eye on the future. Nevertheless, as Zheng [47] points out, ‘for the everexpanding skill set and application of Data Science, we can never be able to teach our students enough “coping skills” to handle each situation’. In a world that is always changing, the objectives and function of Official Statistics need constant readjustment [48]. Of course, curriculum renewal is also a continuous process, but it is typically slow and trails developments in the field. So instead of trying to anticipate change, it is better to prepare students for future learning, that is to cultivate their ‘capacity to learn new information, to use resources effectively and innovatively, and to invent new strategies for learning and problem solving in practice’ [49]. This should be an obvious requirement as statisticians, like any other professional, will need to continually update their practice and skills over the lifetime of their career [15]. Nevertheless, this becomes critically important in a fast-changing discipline. In this section we examine how to prepare for lifelong development through the lens of adaptive expertise, a notion that will help us identify key traits and how to foster them.

The concept of adaptive expertise was introduced by Hatano and Inagaki [50] to describe the ability to aptly address new types of problems. It is used in juxtaposition to routine expertise, which is the ability to solve familiar problems in an optimal way. Adaptive expertise encompasses routine expertise, that is, any expert must be able to perform optimally and consistently in common representative domain tasks. This requires domain-specific knowledge as well as procedural competency, accumulated through repeated practice with standard tasks. But the defining characteristic of adaptive experts is that they can exhibit superior performance in new and unfamiliar tasks as well. This requires a deeper conceptual understanding of their area, combined with ingenuity and creativity. Undoubtedly, these are desirable characteristics, so the main question is how to cultivate them within programs of study or training in statistics. We first review the literature for a general framework of developing adaptive expertise, before proposing specific strategies for statistical education.

OPEN IN VIEWER

Adaptive expertise calls for superior performance in novel situations, something that requires both domain-specific proficiency and general inventiveness or creativity/resourcefulness (see Fig. 1). Schwartz et al. [51] discuss the trade-offs involved in developing such skills with the help of a two-dimensional space of Efficiency and Innovation, an adapted version of which appears in Fig. 3. The authors consider different trajectories to adaptive expertise: one which prioritizes efficiency, one which prioritizes innovation, and one which combines the two, which we refer to as an “integrated” trajectory. Most formal education has traditionally been geared towards efficiency, or what Hatano and Inagaki [50] call “procedural knowledge”. This certainly rings true for Statistics, which is still frequently taught as a collection of methodological solutions, accompanied by a set of decision rules for when to use them. Thus, a typical trajectory for a university student is to first acquire procedural skills and knowledge in standard settings, and to then develop adaptive expertise through projects, internships, capstones, or, eventually, work experience. On the other hand, Schwartz et al. [51] note that content-free activities in critical thinking and problem-solving (i.e., innovation) can help develop the relevant skills, but these might be ineffective for tackling larger, more complex problems. This trajectory is typical for those approaching Statistics from other disciplines and learning on-the-job, often at the cost of having to reinvent the wheel. Schwartz et al. advocate for a more balanced approach, one that supports simultaneous development along both dimensions. We also believe in fostering progress along both dimensions, even at the expense of delaying proficiency, since adaptability is at least as important as specialized skills in a rapidly changing environment [52].

Setting an integrated approach as the preferred way of developing adaptive expertise, the next question is how to attain it. Since educational programs are conventionally geared towards efficiency, we examine how to promote progress along the innovation dimension. As a starting point, Hatano [53] lists three generic factors that support the development of adaptive vs routine expertise:

Exposure to variation: Learners should face “natural randomness” in the types of problems they encounter.

Hatano advocates a process model for adaptive expertise, whereby learners are regularly confronted with new situations which do not perfectly conform to prior knowledge/skills. They are then prompted to engage in experimentation without the need or expectation of extrinsic rewards, which typically favour efficiency. Lastly, students are encouraged to reflect on, compare, and assess their approach, refining their understanding and mental models of the subject at hand. This process model reflects aspects of other known strategies for building expertise, such as problem-based learning [54] and deliberate practice [55].

The factors advocated by Hatano are rather generic, so other researchers have built upon these ideas, especially in the context of professional education in fields such as medicine and engineering. Bohle-Carbonell et al. [56] provide a comprehensive review of characteristics and learning environments for adaptive expertise, while Mylopoulos et al. [57] offer twelve practical recommendations for designing curricula that support adaptive expertise. Alongside Hatano’s original principles and generic methods for adaptive expertise in the literature, we identify three relevant traits and attitudes that are particular to Statistics:

Inquisitiveness: Statistics is a domain-free scientific discipline, or as Tukey famously put it “[statisticians] get to play in everyone’s backyard”. Therefore, statisticians should be characterized by intellectual curiosity and a spirit of inquiry (see Fig. 1). This is especially true of official statisticians, who are called upon to work on an ever-expanding range of social, economic, and scientific issues. In order to engage productively in such diverse applications, statisticians must have broad interests and an appetite for knowledge. This will help them identify important questions and motivate them to pursue answers beyond conventional beliefs and practices.

Taken together, these three features constitute what we call an adaptive statistical mindset, i.e. a way of thinking that is essential to the independent and sustained development of learners and practitioners of statistics. The term describes the desired mindset with which a statistician should approach new situations. Specifically, they should first develop a deep understanding of the problem at hand, appreciating its context, requirements, and peculiarities. Then, they should be willing to try out new approaches beyond off-the-shelf solutions, while maintaining clear objectives and applying methods for comparing, assessing, and adapting these approaches. The latter is facilitated by the effective exchange of information and feedback among the various collaborators and stakeholders. Lastly, a statistician with an adaptive mindset should be able to constructively reflect on their experiences, which will result in continually improving their analytical and critical thinking, as well as their creativity and confidence.

The traits and attitudes of an adaptive statistical mindset parallel the model investigation process as outlined in Wild and Pfannkuch [59], from problem definition, to planning, collecting, and analyzing data, until drawing conclusions. The core of the process relies on statistical thinking, the need for which is well documented and dates to at least Cobb [6]. The other two characteristics of inquisitiveness and extroversion are complementary to the statistical investigation process, in that they incite and promote its development. More specifically, they help identify interesting problems, interpret and communicate the results, as well as iterate and refine the investigation process. All three features include higher-order thinking skills which, although difficult to teach, should nevertheless be integrated and cultivated throughout the curriculum. In the next section, we propose relevant teaching strategies in the context of statistics courses that promote such an adaptive statistical mindset.

Adaptive expertise in any field is developed through sustained engagement in activities that promote efficiency and innovation, and for official statisticians, like most professionals, such experiences will span their entire career. In this section, we focus on post-secondary education and in particular university courses offered as part of study or training programs in Statistics. We present five strategies that are generic enough to be employed at any stage of the curriculum, from introductory to advanced methodological courses, at either the undergraduate or graduate level. The goal of these strategies is to encourage progress along the innovation dimension, which is to be pursued in tandem with the goal of developing procedural skills and knowledge. Table 1 lists the five teaching strategies and highlights their correspondence to the three components of an adaptive statistical mindset. While these strategies are not new and some have appeared in similar contexts [9, 58], we have not seen them compiled or examined through the lens of adaptivity before. Below we provide more details on each strategy, and its relation to adaptive statistical practice.

Open-ended statistical investigations: Students should be exposed to problems that are intentionally vague and under-specified, and which require them to think about and formulate meaningful statistical questions. Problems should lend themselves to different perspectives, inciting students to invent and evaluate alternative approaches. The context of the investigation is also important for coming up with good questions, and topics that are close to students’ experiences and interests, and those with broader social significance typically work better at capturing student interest. Such investigations can form the basis for other strategies, and their benefits span all three components.

Specific examples of these strategies in the context introductory courses at two campuses of the University of Toronto are provided in Gibbs and Damouras [60] and an overview of a complete program designed with such considerations in mind is given in Gibbs [61]. Sustained progress along the path of adaptive statistical practice will require integration of novel situations and learning that emphasizes understanding, exploration and discovery.

5. Conclusion

We have argued that statisticians require a broad range of skills. One can anticipate that technological skills will play a greater role in the toolbox of a future statistician, while all the core statistical skills will remain essential. In a rapidly changing environment, the requirement for particular technological skills may change or evolve, whereas the core competencies will not. We have also argued that a statistician, whether today or in fifty years’ time, will need to be curious and inquisitive. They must naturally critically analyse trends and be sceptical of results. The world is a complex, messy and resource constrained place, and so they must be creative, inventive and resourceful if they are to get the job done.

The role of universities should not only be to impart knowledge and skills, but also ignite curiosity and stimulate discussion. Too often statistics is presented or taught as a black and white set of rules or solutions, but in reality the application of statistics to the complex world we inhabit is at best grey. Official statistics, in particular, has not been historically well covered in academia – perhaps as many academics themselves do not fully appreciate or understand their role. We have argued that, in addition to teaching the traditional elements of statistics, more practical, applied and contextual statistics could be taught.

The bigger challenge for universities is how to cultivate and nurture competencies. This might be tackled by introducing students to what we have termed policy issues and the political economy of data. These issues are fascinating and could be used to encourage debate, stimulating students to think about the complex role that data and statistics play in today’s data driven world. The role of the Professional Diploma in Official Statistics and Policy Evaluation [62] in Ireland or the European Master in Official Statistics [3] are interesting case studies as they were designed with input from official statisticians and take a broader perspective than traditional statistics courses.

Another way to meet current and future demands for our profession is by fostering an adaptive statistical mindset through university education and training programs. The concept comprises three traits and attitudes, namely inquisitiveness, statistical thinking, and extroversion, which promote adaptive practice and lifelong professional development in the field. We describe both general principles for promoting adaptivity, as well as five specific teaching strategies tailored to statistics. All our suggestions combined can widen the scope of statistics courses, making them more interesting, linking them to the great debates of the day, better preparing students to tackle new problems as they arise, and showcasing what a fascinating and rewarding career may be awaiting graduates.