Since 2000 when the Organization for Economic Co-operation and Development (OECD) launched a global academic benchmark for measuring student outcomes by testing 15-year-olds, many global education systems have been impacted by what sometimes looks and feels like a race to rank high.

When the OECD launched the Programme for International Student Assessment — PISA — the idea was to enable countries to make cross-national comparisons of student achievement using a common/standard metric to increase human capital. In other words, higher academic achievement should corelate with earning in the future and a country’s standard of living. As PISA states, it publishes the results of the test a year after the students are tested to help governments shape their education policies.

As PISA has developed, through seven global testing rounds every three years, with the first in 2000 and the most recent in 2018, for some it has gained a reputation as the “Olympics of education” given the widespread attention that country rankings receive following the release of results.

Recent cross-cultural research suggests the influence of PISA is growing around the world. Indeed, in countries such as Germany and Canada assessment systems have been developed that mirror the PISA test. Further, governments look to PISA results twinned with other social outcome measures such as equity in education and social mobility or immigrant success.

Now, partly in the face of criticisms, PISA is looking at expanding how and what it tests. Collectively, changes to PISA will likely spur a shift in priorities by national governments — particularly since countries are keen to achieve good outcomes and to rank high.

As this process unfolds, policy-makers must remember that the social consequences of a test are just as important as the test’s content. Putting a new face on PISA will undoubtedly present various opportunities and challenges.

What PISA now tests

To date, PISA has been restricted to what is generally called the “cognitive” side of learning, focusing on reading, mathematics and scientific literacy. In addition to test questions, students and school principals fill out questionnaires to provide contextual information on student and school environment characteristics that can be associated with more or less favourable performance.

Countries that excel in PISA tests, such as Finland, a country with less than six million people, have become regarded by policy-makers as a “global reference society” — an ideal to aspire to — due to their high performance in PISA rankings.

Asian countries or jurisdictions like Singapore, Hong Kong (China) and Japan tend to consistently achieve exceptional PISA performances and hence get a lot of attention from other countries wishing to emulate their success via borrowing policy. For example, England flew teachers out to China to study mathematics teaching.

Yet even as countries strive to keep pace with preparing students for PISA, criticism — which largely follows any large-scale test that has important implications — has emerged. Some policy-makers, educators and researchers decry that keeping up with rankings drives an emphasis on a narrow, rather than broad, range of measurable aspects of education. Crude interpretations of country rankings and policy lessons can easily be used to promote rigid reforms.

Perhaps the most notable resistance to PISA was expressed in an open letter in The Guardian headlined “OECD and PISA tests are damaging education worldwide — academics,” signed by a group of more than 80 high-profile global academics.

Clearly, the OECD has been aware of criticisms of PISA. In 2015 and in 2018, the test covered new areas respectively of collaborative problem solving and financial literacy and then global competence. In 2015, the OECD also changed the model on which previous PISA scores were based with the aim of reducing measurement error.

More recent statements from the director of PISA, Andreas Schleicher, suggest significant changes are on the horizon because the modern world doesn’t reward us for what we know but for “how we apply knowledge.”

Schleicher recently acknowledged “PISA is, at the moment, a partial picture of what is important.” But future testing innovations seem likely to fall flat if they are taken up by governments in a prescriptive manner.

Not ‘back to basics’

In the next administration in 2021, PISA will tackle creative thinking, trying to find ways to assess, and have students assess, flexibility in thinking and habits of creativity such as being inquisitive and persistent. The PISA team is also developing a way of testing students’ digital learning, which should be ready in time for the 2024 assessment.

Recently, and somewhat ironically, the highest-ranking global country on PISA tests, Singapore, made waves when its education minister announced that it would be scaling back testing in the country to try to find balance with the “joy of learning.”

Collectively, these statements run somewhat against a dominant standards-based reform agenda that has emphasized “back to basics” in math and language since the late 1980s and early 1990s in the United Kingdom and then the United States.

Policy learning and borrowing?

The OECD has stressed “better policies for better lives.” For the most part, policy-makers have largely agreed with Schleicher’s position that the PISA can help facilitate public education system improvements through the lessons learned from top performing nations or economies.

However, it should be remembered that education policies from high achieving nations don’t migrate across international boundaries without consideration given to national and cultural contexts. Rather, innovations and changes in education require teachers to have the time and opportunity to re-educate themselves in relation to more recent insights in what it means to get the best out of children.

The OECD will need to respond to previous critiques and provide greater transparency around newer test instruments and the choices made to arrive at rankings. The latter is no small challenge since the future focus of PISA is based on topics which seem more difficult to evaluate than math, science or reading skills.

Author Bios: Louis Volante is a Professor at Brock University, John Jerrim is a Lecturer in Economics and Social Statistics at UCL, Jo Ritzen is Professor of International Economics of Education, Science and Technology at Maastricht University and Sylke Schnepf is a Senior researcher at the European Commission’s Joint Research Centre