Abstract
Can you tell me a bit about how the OECD is dealing with the COVID-19 crisis?
The OECD historically is not focused on education in emergency, so what we are doing is concentrating on our value added: the depth of data collection, research and the expertise around that. In addition, OECD plays an important convening role to bring different and very disparate stakeholders together from across the world.
In response to the COVID crisis we’ve done several things: repurposing some of our existing large-scale survey data, drilling down on key questions in Programme for International Student Assessment (PISA) and The Teaching and Learning International Survey (TALIS), for example, on digital infrastructure access, use etc. for both students and teachers. Of course, these are data gathered in 2018, before the pandemic.
In addition, we’ve put together a couple of quick flash questionnaires on countries’ immediate responses, when the first wave of school closures hit, and again now as countries are reopening their schools. It’s very light touch, for an immediate sense of what is happening.
Another thing we’ve done is to think through the key priorities on two timescales: the immediate return to school in the autumn (for the Northern Hemisphere) and the next 12–24 months, which could potentially see a series of rolling school closures. The priorities there are not only to ensure learning and student physical and emotional well-being in this potentially quite uncertain time, but also to really take advantage of a lot of the innovation that we’ve seen emerging as a response to the first wave; really building on the efforts of students, teachers, schools and using it to challenge ourselves to do better and rethink where we can.
And how do you see the ongoing challenges affecting surveys such as PISA?
There are big conceptual issues and more practical issues. From a practical perspective, many of our surveys were going to be field tested this year: PISA and also PIAAC, the Programme for International Assessment of Adult Competencies, which tests adult competencies, and which only runs every 10 years. The teacher study (TALIS) is a little bit luckier in terms of timing because it is not scheduled for another 4 years. So both PIAAC and PISA have been delayed by a year, and the data collection for PISA will take place in 2022, which means that field testing will happen next year conditions permitting. PIAAC has the same kind of timeline.
There are complicating issues that go along with these changes, for both the OECD and participating countries. For example, countries have employed contractors to handle the national data collection; for them it’s a huge operation to rejig the schedule and think about what compensation to their contractors might look like. There are many other practical details being handled by my colleagues on the big surveys and their counterparts in capitols at this moment. And of course, there’s still a lot of uncertainty because we don’t know what the situation will be like in a year, so we are partially blinkered in terms of that. But we are doing the best we can, like everyone else.
So are there new questions arising due to the pandemic?
Conceptually, there are a lot of issues that are quite interesting to think through. For example, we’d really like to know the answers to many more questions around access and use of digital devices but also teacher digital knowledge and teaching practices, which may or may not be digital. Also, countries using non-digital means: many countries are using television and radio to make sure that instruction reaches those least likely to have access to digital devices and the Internet. What learning has happened, how well do these efforts work and what might we expect the impact to be, academically, but also in terms of physical and emotional well-being?
So there are those questions, but then at the same time, we have to consider that the survey structure is built on following trends across time. There’s a tension between gathering data on themes that have always been collected, to track ongoing trends, and the issues that have gained particular relevance due to COVID. However, there’s not a lot of room for extra questions timewise, otherwise it is too heavy a load on the respondents. So in the end, quite practically, at least for PISA, there will likely only be a very small number of questions that are added that are specific to this crisis. And hopefully, the answers to many of the other questions will allow us to fill in some of the missing pieces.
So digital access, is a real worry here in the UK, how is this playing out across other OECD countries?
This is a key concern, and the UK is not the only place that has that worry. If you look for example at PISA data, over 90% of 15-year-olds in 2018 said that they have access to a computer to do homework. Those students might also have siblings who also theoretically had access and who would have all happily said yes in a survey. But then when you’re faced with the reality of 100% transition to online education, you might not have the access that you need after all, because you also have siblings and parents working from home, who would also potentially be using the device.
So there were many questions that simply weren’t designed to capture the reality of lockdown and massive school closures. Teasing out these issues is a full-time job, and I’m still not sure we’re going to be able to really adequately capture them to be very honest, because, not only is the situation changing constantly, devices age poorly and so access, and skills required will be continuously shifting. That’s something many large-scale surveys can’t really capture adequately, as they take time to do well across so many countries and contexts.
But this is not just an issue for survey research, it is also exactly the difference between designing and implementing a policy, and the reality of what happens when that policy is under duress. That’s where we saw the cracks coming in, in the UK but also across the whole world.
In terms of teachers’ expertise in delivering digital as you know, I work for an online university and the challenges that we have in preparing materials for good online learning are considerable, yet I know that there’s an awful lot of creativity going on across the piece in schools: Do you see this changing the approach to online learning; do you see a recognition among school leaders that this is not just for the present crisis, but a sense of ‘we’ve got to take this forward?’
I think there is. I mean the thing to be careful with at this point when you’re measuring responses based on these very rapid surveys is the people who are willing to respond, who have the time to respond, are not a random group of people. Certainly, the people who are working with us and responding to these surveys now are absolutely convinced: as they are responding to an online survey they are already fluent with digital environments. I do think it’s been made quite clear that digital is not just a ‘nice to have’ but is pretty fundamental. Many countries have been starting to move towards declaring access to the Internet as a human right, and some countries, for example, Costa Rica, have had that for years. But more countries are now understanding it, not just as a technological tool, which is useful, but as fundamental to participating as an active citizen in a democracy. Never more so as when you are developing and growing and learning.
In terms of the professionals, there’s long been a tension between the uptake of digital tools for teaching and whether or not they’re actually achieving their goals. And it’s quite easy to blame teachers who may not be using technologies as much as they should. But, at the same time, teachers can rightfully say that they haven’t got the training they need, or that the tools are not up to pedagogical standards. And I think all of those things are true. So in my opinion, what we’re seeing is really a need to revision and rethink both initial teacher education and continuing professional development to include knowledge of technology and digital skills for teaching. This means pedagogies, including classroom management etc., but also on the other side, to really involve learning science and pedagogical experts in the development of digital learning and teaching tools. Because too often, many of the platforms and many of the services were developed by marketing people, and they’re not based on learning sciences or pedagogical principles that teachers can use. So I feel there’s been a fundamental mismatch there, and now is the time to have that conversation.
And do you see a role for the tech giants here?
I think so. What we’ve seen is they’ve really stepped in in the COVID emergency, and they’ve rolled out a lot of platforms and services available free of charge. There was, during lockdown, a distinction between the angels and the vultures, where you’re hoping everyone is being an angel and showing up and providing free services, but everybody’s worried that they’re secretly vultures, capturing student data that they shouldn’t be capturing etc.
I think there are real questions there around the role of such giants. They are uniquely placed; they have the reach and the tools and the expertise to really help build not just as a stopgap, but really help build and redesign a system of learning and teaching. But at the same time of course they have their own vested interests and they are private companies. Their job is to make money and that’s legitimate. So in terms of a public good and keeping education as a public good, we need to go beyond conversations about goodwill and really start thinking through the regulatory mechanisms that we want to set up in order to make sure that everybody’s on the same page – that everybody’s compliant with the various restrictions that are needed, to safeguard student data, for example, or to protect children from any cyber dangers they might be exposed to.
And do you see a role for inspectorates in this? Because I see there are, I mean some inspectorates are making noises at the moment about inspecting the quality of provision, but that would seem to me to be putting the cart before the horse.
Yes, one of the things I’ve been thinking about, is that in many ways the inspectorates, at least on the international level, have been surprisingly quiet. It’s perfectly possible I’ve missed it, but I feel like I’m not hearing them enough in the conversation. And I do think they play a very important role, but I do wonder what their positioning is in terms of assessing the quality of these things.
If you’re assessing digital teaching and performance and assessment and services and platforms, what is the knowledge base that’s needed for accurate assessment of that? Do inspectors have it? Are they supported to develop the knowledge base and continuously update it? So there’s a capacity question there as well, which I’d be really interested to hear more about because I to be honest haven’t heard enough about it.
PISA’s important in terms of policymaking etc. for national governments – if this crisis is having a negative impact on the PISA results, how’s that going to affect it as a tool to drive policy in the future?
I would be extremely cautious in interpreting a drop of PISA results to this particular crisis. It’s a snapshot of a moment in time, and what is driving that is incredibly hard to distinguish. The rigorous way to do it would be to say: let’s take a look at the length of school closure and the real number of students that have access to learning at that time, and the quality of that learning. And let’s compare the results of the countries that did not close schools, for example, to those that did. Or students who had poor quality versus high quality resources. If empirically there are differences, we can then look at these questions very carefully. But all of those things are difficult to measure and define, and by the time PISA is in the field in 2022, it’s a different cohort of kids.
So, a year-and-a-half later, what is the impact on their learning and all the other elements that PISA’s testing? I’m cautiously curious, but I’d also like to embed the PISA findings within a nuanced discussion of what’s going on and trying to understand associations as well as the causal mechanisms behind the associations.
This is not just a PISA issue, by the way – that’s always the struggle with policy: how do you act at the speed that policy requires, with the certainty that policy needs, when research can very rarely operate at that speed or deliver that kind of certainty. And so the one thing that we would like to avoid is people trying to solve what are in effect are quite complex issues with rather simple measures.
I mean of course at this moment we’re concerned because of the pandemic. But we need to remember that the most at risk are exactly the same students we are always concerned about. Who are the most disadvantaged, the poorest, who don’t have proper housing or nutrition? These are not pandemic-specific problems. What the pandemic has done is helped us to not just identify new weaknesses in the system, but it’s also really highlighted those weaknesses which we already knew were there. And those questions have been hard policy questions for a century or more. That’s where we need to engage most, quite frankly. I mean the pandemic is, let’s say, it’s shining a light at the cracks. And I honestly think that those bigger issues are the ones we really need to challenge ourselves to address.