EDUCATION

The problem with averages

In the aftermath of the exams disaster it should sound obvious that the individual matters, says Dan Corry – but it is a point missed out far too often in policy analysis.

The whole exam fiasco has been unedifying at best and the destroyer of lives and wellbeing at worst. There were never going to be any easy or simple answers once we knew exams would not be used to assess this cohort of teenagers. It is hard to imagine how the Government, in both Westminster, the devolved nations, and the regulators, could have made any more of a shambles than that which they inflicted upon children, many of whom are already about to be thrown head first into an appalling job market.

One could wallow in countless aspects of this hapless story, but a lesson that comes out loud and clear for all policy- makers and leaders is that individuals matter, not just averages.

In the case of exams, ministers told us (before they promptly jumped ship on the algorithm) that on average they would not treat poorer kids any less fairly than the better-off kids. That claim is debatable, given what we came to know about the bias favouring smaller groups who were found in private schools.

Now we can see the evidence that under the iron rule of the algorithm, a child from a more deprived background was more likely to have their marks downgraded because they are more likely to be at a school that historically gets poor results.

Presumably, if it weren't for the constraint of having to stop grade inflation, there could have been an algorithm that worked more fairly. But even then, the aim needs not so much to be about average scores but about fairness – actual and perceived – to individual students.

On many occasions during my time in Government, I was involved in work to bring in a new policy or amend an old one, where one could show that the vast majority will gain from the change. ‘Seventy-eight per cent will gain minister', might be the refrain from the eager civil servant analyst. Why not enact this policy? Why not raise aggregate human utility in this way? Why is the special adviser sounding worried?

In the first instance, 78% gaining means that the 22% in my stylised case – are probably going to lose (unless a lot are neither better nor worse off afterwards). We know from our own experience that those who lose shout louder than those who win. This is something behavioural science has given an academic gloss, showing that risk aversion means we greatly prefer avoiding losses to acquiring equivalent gains. So, the policy-makers are far more cautious than those who produce this material.

But even if the losers are few, the issue remains. No amount of statistics trying to show that it is overall a good change will help when universal credit delivers outcomes to some individuals that just feels wrong to reasonable people.

This is not just about the politics, the media attacks and so on, but the reality that we need to weigh up the burden put on individuals, families or groups when we propose policy change. It is the right and moral thing to do.

Standing up for the individual is one of the reasons we elect politicians – to make decisions, instead of leaving it all to statistical tables and analysis – or even algorithms.

I recall analysis being carried out for ministers trying to look at the ‘hard cases' thrown up by different policy options, but such analysis was not routinely delivered by officials in Whitehall. I doubt they are at local level either. It was the connected minister or MP that asked for that analysis. I suspect it is the same with councillors. The connection they have to their constituents and the public at large gives them a perspective that usually goes beyond averages, even if that did not seem to be the case with many of the key ministers in the exams fiasco.

So, if ministers learn anything from this crisis it must be that they should be wary of making policy decisions exclusively on the basis of averages. The real impact of their decisions will often be felt far more by those at the margins, not the midpoint.

Dan Corry is chief executive of NPC – a think tank and consultancy on third sector issues. He is a former Treasury and Downing Street economic adviser

EDUCATION

Homelessness: bold new AI-powered thinking is needed

By Alex Stefany | 29 July 2024

AI could be Starmer's secret weapon in solving homelessness and rethinking the welfare state, says Alex Stefany

EDUCATION

The Covid Inquiry needs a local voice

By Henry Bermingham | 29 July 2024

Henry Bermingham calls for the Covid Inquiry to consider giving a greater voice to local government leaders and says stronger regional involvement remains a ...

EDUCATION

Appetite for construction

By Jonathan Werran | 29 July 2024

Jonathan Werran ponders whether the Labour Government’s headline-grabbing housebuilding proposals will ultimately achieve their goals

EDUCATION

Planning, planet and participation

By Graham Duxbury | 26 July 2024

Graham Duxbury explains why the new Government needs to put people and nature at the heart of its decision-making

Dan Corry

Popular articles by Dan Corry