Editorial: Opening for debate and contestation: OECD’s International Early Learning and Child Well-being Study and the testing of children’s learning outcomes

The International Early Learning and Child Well-being Study: What, why, how and who

This special issue aims to bring critical perspectives to bear on a growing phenomenon in education: comparative assessment of educational performance using standardized measures of outcomes or ‘international large-scale assessments’ – ILSAs. We focus on one of its latest examples: the International Early Learning and Child Well-being Study (IELS). Proposed by the Organisation for the Economic Co-operation and Development (OECD, n.d.a) in 2012, this study is now being put into practice, targeting early childhood education and young children in particular. The articles in this edited collection offer varied critiques of this project, as well as critiques of the influential role that the OECD is playing in how member countries design, implement and assess their early childhood education.

The IELS is a cross-national standardized assessment of early learning outcomes involving the testing of 5-year-old children in officially registered early childhood centres and/or schools in participating countries (OECD, n.d.b). The IELS aims to assess early learning domains which include emerging literacy, emerging numeracy, self-regulation, empathy and trust. Tablets are given to children who have approximately 15 minutes to answer a number of questions in each domain (up to a total of 60 minutes for the testing of the four domains over two days).

The IELS aims to improve early childhood services by providing policymakers, researchers and educators in early education with valid and comparable information on children’s early learning (OECD, n.d.b). The proponents of this study have argued that the IELS will help countries to improve later educational performance – and thus countries’ economic growth – by giving earlier and more specific recommendations of what skills and competencies its young children need to improve. The IELS is expected to provide an overview of how to improve young children’s learning outcomes and increase test results 10 years later in the Programme for International Student Assessment (PISA), the centre piece of OECD’s international web of measurement in education.

In September 2016, ¹ the OECD appointed the Australian Council for Educational Research and the International Association for the Evaluation of Educational Achievement (IEA) to design, develop and pilot the study (Moss, 2017b), including the software development, data management and study coordination. Another IELS partner includes cApStAn, which is responsible for ensuring that all countries participating in IELS are using comparable language in their instruments. These international partners are already responsible for several international assessments in areas such as math, science and reading, including Trends in International Mathematics and Science Study (TIMMS), ² Progress in International Reading Literacy Study (PIRLS) ³ and PISA.

Development and piloting work and field trials of IELS assessment measures took place in 2017, with the main study, or ‘first wave’, implemented in the autumn of 2018. Just three countries (out of OECD’s 36 member states) signed up for this: England, US and Estonia. Each country appointed a national centre to undertake the study locally. In the US, the government has appointed Westat, a professional service corporation for the coordination of the data collection and analysis, while in England, the government has appointed the National Foundation of Education Research (NFER, n.d.), a non-profit charitable organization, to administer the IELS across the country. It has been planned that the study administrators will comply with a so-called ‘international representative sample’ of at least 3000 children in 200 settings per country, with up to 15 children per early childhood centre or school, which will be randomly selected. Once this sample is finalized, children will also be randomly selected to participate. The contractors are responsible for making sure that a rigorous sample process is in place to provide reliable data for international comparative assessment.

At the local level, the study will involve two assessments. The direct assessment, or the so-called ‘play-based’ assessment, consists of a number of activities on a tablet that the child is invited to perform. The indirect assessment involves a collection of reports filled out by staff, educators, parents or primary caregivers and the study administrator. Based on the adults’ observations, the study aims to gather information regarding the child’s background (i.e. age, gender, language, immigrant background, parental socioeconomic status [SES], family composition), home-learning environment (i.e. relations with the child, activities, and home-learning resources) and early childhood education and care experiences (how long the child has been enrolled in a service, frequency, continuity, type, etc.).

Despite the OECD’s wide membership, the number of countries participating in the first wave study is small. Most member countries chose not to participate, and although there is no comprehensive record of why countries declined to take part, a number did so because of their opposition to the effects that standardized testing may have on children, their educators and families, and the whole system of early childhood education overall. For example, the German government declined its participation in the IELS, taking into account a statement written by a coalition of service providers, trade unions, parent groups, and researchers. This coalition contended that participation in the IELS would deny children’s rights to an early childhood education that is diverse and socioculturally contextualized (Urban and Swadener, 2016).

The IELS: Part of something much bigger

The implementation of the IELS can be seen as the latest piece in a whole framework of international standardized testing running from early years to adulthood, in which the OECD plays a leading role. The OECD’s web of measurement, either under development or in operation, includes PISA, PISA for Development, PISA-based Test for Schools, the Assessment of Higher Education Learning Outcomes, the Programme for the International Assessment of Adult Competencies, and the Study on Social and Emotional Skills of 10 to 15-year-olds. As such, the IELS can be seen as part of what has been termed the Global Education Reform Movement (GERM) (Robertson, 2015), a widespread neoliberal model of education that aims to transfer the principles and strategies of the market economy into public education to improve the quality of students’ learning. Following this market economy model of education, GERM fosters competition among schools, autonomy (minimizing the ability of education departments to make their own decisions regarding systemic support to schools), choice (enabling parents to choose where their child goes to school) and accountability (understood as a managerial emphasis on child and school performance). Standardized testing plays a major role in the pursuit of these ends, stimulating competition and providing parent-consumers with information to aid making choices (Vincent and Ball, 2006) and for managers and funders to assess performance.

The OECD has played an important role in the spread of GERM, not least through its championing and implementation of ILSAs, stimulating competition between countries and heightening a performativity agenda (Ball, 2003). The organization’s preoccupation with standardization in and of education, predetermined results, and test-based accountability policies (Sahlberg, 2011) has contributed to the reduction of education to a technical practice, prioritizing a search for effective methods. The big question is what works to achieve what David Larabee describes as ‘a single overarching goal for education … human capital uber [sic] alles’ (2017: 218)? In this triumph of technical practice, political and ethical practices have faded, and with them the asking and contesting of fundamental political questions about the meaning, purpose and values of education.

Within this broader educational context, the OECD through the IELS is now seeking to develop a universal standard for early childhood education, a goal which has been strongly criticized around the world for its lack of consideration of the diversity of educational projects (Biesta, 2010, 2017; Moss, 2014; Urban and Swadener, 2016). In adopting this approach to the comparative study of early childhood education, the OECD has chosen to turn away from more comprehensive and contextualized approaches and, indeed, its past work in this field. From 1996 to 2006, the OECD undertook Starting Strong I and Starting Strong II (OECD, 2001, 2006), which was a review of early childhood policies and provisions in 20 countries through case studies that portrayed the complex diversity of systems and pedagogies.

It is not clear why the OECD changed course; why for example it opted for the IELS rather than taking the innovative work of Starting Strong I and II further. One of the repeated critiques that the IELS’s proponents have been unable or unwilling to answer satisfactorily is the paucity of contextual information about early childhood education in participating countries. While the study includes some limited contextual information about the children being assessed, it has nothing to say about the wider political, social, cultural and pedagogical contexts of early childhood policy and provision, without which it is even harder to know what to make of the data that the IELS will generate.

The IELS: Quiet beginning, growing dissent

The OECD and member state governments first started discussing the IELS in 2012. Since then, there has been no attempt to engage with the international early childhood community, nor to inform or consult them about the desirability, purpose, design and implementation of this study.

In 2016, as news about the proposed IELS began to leak out, several responses from the international early childhood education community appeared in academic journals, websites, colloquiums, and in non-governmental organizations, voicing concern about the design and implementation of this study (see for example: Dahlberg et al., 2016; Moss, 2017a, 2017b; Moss and Urban, 2017, 2018; Moss et al., 2016; Urban and Swadener, 2016). For instance, in a statement on behalf of the international Reconceptualizing Early Childhood Education network, Mathias Urban and Beth Swadener (2016: 7) argued that the IELS will be ‘largely meaningless due to [its] disconnect with and disrespect for diverse, locally embedded approaches to early childhood education and care’, and that the focus on the early learning function of ECEC ignores the important contribution of ECEC services to a wide range of ‘societal and political challenges of our time’ (Urban and Swadener, 2016: 8). In this statement, Swadener and Urban (2016: 1) put forward ‘a call for supporting competent systems, democratic accountability, and systemic evaluation’. To date, this statement has been signed by more than 150 scholars, senior academic researchers, policymakers and practitioners.

Until now, the whole exercise of initiating, developing and implementing the IELS has been treated by the OECD and participating countries as a technical exercise requiring no democratic engagement. In their most recent critical article, Moss and Urban (2018: 3) note that ‘neither the OECD nor any of the three participating countries nor any of the organisations contracted to implement the IELS have offered any response to widespread criticisms’. In their opinion, the OECD’s disregard of the early childhood community reflects an ingrained culture of secrecy and indifference to the international response of initiatives like the IELS. Referring to the lack of contextual information and the arbitrary inclusion of countries in the study, they conclude that as ‘the IELS progresses and we come to know more about it, the more we are struck by its superficiality and pointlessness’ (Moss and Urban, 2018: 3).

Special issue

This special issue provides an additional avenue in which to critically examine the policy implications of the international standardized measurement regimes, with particular attention to the IELS. As some of the contributors in this special issue demonstrate, the IELS is part of a much broader international educational agenda. The articles aim to challenge the possibility that the IELS may become the next ILSA, following in the footsteps of PISA (Wasmuth, 2018), by showing its problematic effects on national early childhood education systems. We are pleased to contribute to the debate about ILSAs in general and the IELS in particular, with contributions by authors with diverse perspectives and from countries that are already participating in the study (England) as well as countries that are not (Canada, New Zealand, Portugal). In addition to each of the contributions, this special issue would not have been possible without the generous contributions of each of the reviewers who, from across the world, offered their time and expertise to make each of these articles a strong piece for a much-needed international debate today.

In the first article, Euan Auld and Paul Morris take a critical look at PISA, the most well-established, extensive and influential comparative, standardized assessment to date. They do so to explore the potential effects that the implementation of IELS might have on early childhood education systems and ultimately on children’s early learning. The authors take PISA as their starting point to situate the IELS within the OECD’s broader agenda in education governance. As Auld and Morris (2016) have demonstrated elsewhere, there is no clear evidence of the assumed relationships between PISA and national economic performance. Despite the fact that PISA’s proponents have not been able to refute the critiques regarding the test’s inability to identify the necessary skills for the job market, the OECD has decided to move forward with its version for younger children: the IELS. If the authors’ critiques are ignored, we may witness a radical paradigm shift: ‘one which stresses cognitive skills and children’s role as future sources of human capital’ (Auld and Morris, 2019).

Guy Roberts-Holmes offers a visionary perspective of what might happen if the IELS becomes an established standardized testing of young children’s learning in England and elsewhere. The author accounts for the reasons why England has signed the country up to taking part in the IELS despite the context of sustained austerity in government spending. The IELS is in line with a wider national neoliberal educational agenda, including early childhood education, that has at its core a web of measurement of teachers’ performance and children’s achievement. Roberts-Holmes makes us think about some of the implications that the further development of the IELS, as a neoliberal governance, may bring into early childhood education. On one hand, the IELS may come to reinforce a logic of datafication of children’s learning and dataveillance of teachers’ performance, and on the other, it may help to expand edu-business opportunities at the expense of treating early childhood education as a purely technical practice. Being aware of these possibilities provides more arguments with which to resist the homogenization of early childhood education that the OECD seems to be proposing.

Diana Sousa, Laura Oxley and Sue Grey examine the potential effects that the introduction of the IELS might have on Portugal’s aspiration to foster a more democratic pedagogical system through early childhood education. Their discussion is situated within a study (Sousa, 2017)of three different early childhood education settings, in which educators’ values and pedagogical practices regarding democratic principles and practices in ECE were analyzed. As the authors show, the educators’ narratives regarding the purposes of early childhood education in a democratic society contrasted with the outcome-based approach that is at play in the IELS. Key divergences among these discourses include: (a) different orientations towards pedagogy (from child-centred in the Portuguese context to adult-centred/‘expert’ in the IELS context); and (b) different notions of the child; on one hand, the child who must achieve measurable skills and competencies for IELS, and on the other, the child as a competent social and democratic citizen with capacity for critical thought for Portuguese educators. The authors warn us that the (potential) introduction of the IELS would lead to a ‘democratic deficit’ in which a universal model would infringe on existing plurality of methods and approaches within early childhood education settings in Portugal. While Portugal is not yet participating in the IELS, the authors of this paper present some very real challenges that may follow if the IELS does become a part of Portugal’s early childhood education context, and highlight the need to continue to contest many of the discourses and practices that are part and parcel of the IELS.

Andrea Delaune contributes to this special issue with an analysis of the IELS as a technology of governance. She draws on the work of Michel Foucault to critique the IELS’s narrative of ‘what is best’ in early childhood education. Along with OECD’s other technologies of governance like PISA, the IELS is expected to reconfigure the relationships among teachers, families and children around what is ‘best’ in the education of young children. This singular truth threatens to homogenize what is valued as children’s learning, overlooking the context where learning takes place and the vision that countries and communities have about the ultimate purpose of education. Delaune argues that through the OECD’s tools of standardized testing like the IELS, the young child is becoming a mere measure of countries’ market values. Nonetheless, she concludes by insisting on a hopeful message: technologies of governance can be turned into tools of resistance.

Pei-Ying Lin and Yu-Cheng Lin offer a strong argument for why it is important to turn away from standardized assessment and move instead towards an assessment for and as learning in the case of exceptional learners or children with special needs or disabilities. The authors argue that ILSAs have failed to include exceptional learners in their instruments’ design and implementation. Exceptional learners potentially benefit from being included in the IELS as long as the study takes account of principles of test accommodation in relation to educational measurement and special education theories. Based on an analysis of PISA and how this international assessment tool has been implemented in the past, Lin and Lin point out the caveats and challenges for the IELS to be inclusive in its assessment. These challenges include the collection, interpretation and communication of data in ways that, first, identify the gaps in special education policies and practices at local, national and international levels, and, second, avoid the reinforcement of a culture of achievement and competition (e.g. countries’ ranking) where these learners do not belong.

Grounded in a substantial critique of GERM, Mathias Urban offers an overview of the IELS’s implementation from its early conception to its current state. This overview lays down the pillars for the author to draw on post-colonial and neo-colonial perspectives and demonstrates how the actual administration of the IELS reinforces hegemonic discourses about what counts as children’s learning. Children who will be taking the IELS will come across ‘Tom’ and ‘Mia’, the two characters featured in the IELS testing material. Urban suggests that they work as agents of neo-colonization by enacting one way of being a child. Along with other authors in this special issue, Urban concludes with a call to resist current global hegemonic narratives of learning. Urban reminds us that experiences made at the margin of society offer alternatives to resist global imaginaries like Tom and Mia.

Conclusion

António Nóvoa (2018: 5) has suggested that over the last 50 years, comparative education has increasingly moved towards a phase of prescription, with ‘a celebration of “big data”, allowing experts to prescribe the best solutions for the different educational systems’; perhaps, he adds, ‘we can call it “dataism”, the religion of data’. This ‘solutionist drift’ entails a belief in ‘global solutions imposed by data and evidence on “what works” and “where the best results are” … [an approach] based on the false idea of consensus on the aims of education and the paths to achieving them’ (Nóvoa, 2018: 4). He calls, instead, for a comparative education that seeks to revitalize the commons, contesting a world of hyper-individualization in which we only interact with what is similar to us; to strengthen the public space instead of contributing to the authority of experts as if they alone possessed the type of knowledge that can be transformed into policy; and to build a science of difference, rather than a ‘solution’ that tends to homogenize educational directions throughout the world.

The homogenization of education, fuelled by the spread of ILSAs, with their assumption that everyone should be heading in the same direction, has a particular character or bias, drawing on ‘examples, images and models of the “Anglo-Saxon North”’ (Nóvoa, 2018: 6), and leading to the dominance of the English-speaking world. That bias is evident, too, in the IELS, with a strong representation from Australia, England and the US, either as participant countries or in organizations and individuals involved in the development and implementation of the study. The words of Loris Malaguzzi are also brought to mind, when he decried the spread of what he called ‘Anglo-Saxon testology’.

[with] its rush to categorise … where it is enough to do some tests on an individual and immediately the individual has been defined and measured in some way … which is nothing but a ridiculous simplification of knowledge, and a robbing of meaning from individual histories. (Cagliari et al., 2016: 331, 378)