It's a funny phrase – Best Value. What does it mean? In the language of everyday consumerism it implies the best balance between quality and price. But in the public sector, the short answer is that it means whatever the Government says it means. It does, however, sit a little odd against the concepts of public value and social value.

In 2019, the HM Treasury published The Public Value Framework. It provides guidance on how to pursue goals, manage resource inputs, engage with users and citizens, and develop overall system capacity. This framework consists of four pillars, themselves composed of 13 factors and 35 sub-factors.

In 2020, the Cabinet Office produced guidance on how Government contracts ought to incorporate social value. Based on the notions established in the Public Services (Social Value) Act of 2012, which also applies to councils, the model has five themes, including ‘fighting climate change' and ‘equal opportunity'.

So where does ‘Best Value' come from? It is rooted in the regime that preceded it – compulsory competitive tendering. Under this regime a range of frontline functions had to be tendered to the private sector, although in-house teams could also bid.

Best Value replaced this enforced tendering regime in the year 2000. It was first introduced during the differential devolution process to Scotland, Wales and Northern Ireland. This means it has taken slightly different forms in the four nations.

In Scotland, it included partnerships, sustainability and fairness among its themes. In Wales it was wholly replaced in 2011 by the Wales Improvement Programme. This places emphasis on local ownership of improvement planning.

In its original form it was predicated on the requirements of extensive consultation and listening to service users, citizens and taxpayers prior to making changes to services and investments. But in England since 2010, the Best Value regime has been used as a means of intervening in councils when things appear to have gone badly wrong.

Best Value is a duty to ‘make arrangements to secure continuous improvement in the way in which its functions are exercised, having regard to a combination of economy, efficiency and effectiveness'. The term pivots on the concept of ‘continuous improvement'.

In the 1990s this was a cornerstone of the then total quality management movement. Indeed, the Japanese business philosophy of Kaizen (change for the better), became integral to the six sigma and the lean management movement of that time.

Over the past 30 years, this approach has added considerable discipline to the management of change in local government – based as it is on detailed operational step-by-step improvement.

The new Best Value draft guidance targets the right areas. It expands on the meaning of continuous improvement by saying that, ‘authorities are not expected to be perfect, but rather they should prioritise learning and development throughout the organisation and always strive to learn from past mistakes, address under-performance, and avoid continuing in a direction where failure is evident'.

It underscores the importance for councils to be open to independent external challenge and scrutiny. It outlines seven important Best Value themes – although environmental sustainability and fairness are notably absent. In its Appendix it tabulates the ‘characteristics of a well-functioning authority' as well as ‘indicators of potential failure'.

To date, Government has found cause to intervene in the light of severe ethical breaches in governance; in the event of extreme service failure; and, more recently, in the context of extraordinary financial failures involving borrowing, investments and budgeting. These are the main reasons why this new guidance has been drafted.

The reasons for Government intervention have been as varied as the local causes as to why things have gone wrong. Not having a clear and sequenced framework for intervention is damaging to the Government as much as it is to the councils concerned. But in too many areas it suggests a search for quantifiable answers to qualitative questions. There is a sense that it is fixated on finding reliable forebodings of future failure. If only we could tell who will fail next, perhaps we could prevent them failing. A laudable aim and, perhaps in respect of financial failure, this may be feasible.

In these cases, the key ratios, metrics, and measures may be good enough. But every manager knows ‘you don't fatten the pig, simply by weighing it'. Failure comes from wrong choices, made for wrong reasons, or on wrong bases. Sometimes it stems from simple inattentiveness or neglect.

Of course it can also stem from error, mistake or worse – persistent ethical breaches. Mostly it stems from a mixture of poor management, poor professional judgement, poor advice-giving, poor political judgement or a closed culture bloated by hubris. Failure can come from a range of contingent circumstances – even from good people trying to do the right thing, but doing it wrong.

That is why what is needed is a continual dialogue between Government and local government about how to pursue success and avoid failure. Guidance documents that emanate from the Department for Levelling Up, Housing and Communities once a decade are not enough to deal with the complexity of delivering services – especially as two-thirds of council expenditure is on functions sponsored by other Government departments.

Barry Quirk CBE is a former council chief executive and local government adviser, and a former president and chairman of Solace

@BarryQuirk1