Left column

Middlecolumn

Since we first published our Framework to review programmes in 2017 there has been no let-up in NAO reports on major projects and programmes, most recently on Crossrail, the Emergency Services Network and the Stonehenge by-pass road. From the need to manage the risks of untried approaches to signs warning of unrealistic cost estimates, this […]

Article

Major programmes – what are we learning?

Posted on July 4, 2019 by

Image of a suited man considering options

Since we first published our Framework to review programmes in 2017 there has been no let-up in NAO reports on major projects and programmes, most recently on Crossrail, the Emergency Services Network and the Stonehenge by-pass road. From the need to manage the risks of untried approaches to signs warning of unrealistic cost estimates, this blog highlights some of the themes emerging from our recent work on major programmes, which have been incorporated into our updated guidance.

In 2017 we launched our Framework to review programmes, as I introduced in my post A systematic look at major programmes. Since then we’ve published another 36 reports on major projects, gaining further insights that we’ve used to update the Framework. These are in addition to our work on government’s preparedness for Exiting the EU, which illustrates some of the same challenges. We’ve also published a Survival guide to challenging costs in major projects.

Our reports cover the implementation of policy and business change as well as traditional infrastructure projects, and the findings we have brought together here are drawn from both. There are common themes across types of programme: the importance of being clear about the intended benefits and developing and delivering an approach that achieves them; the need to choose realistic and deliverable options; and the importance of ensuring there are the resources to put programmes into practice.

Signs of improvement

Over the last few years we’ve seen better project discipline in government. For instance:

Outputs delivered, but outcomes unclear

Our report Projects leaving the Government Major Projects Portfolio showed that most projects produced the outputs they set out to, but it was more difficult to say they had achieved the planned benefits. In some cases it was not clear what the intended benefits were, as some projects did not set out their intended outcomes in a business case.   

Image from NAO's Smart Meters report

Example from new Framework: In Rolling out smart meters we found not only that the original ambition of installing smart meters in every home by 2020 will not be met, but that gaps in monitoring are impacting on the programme’s ability to achieve the intended energy savings and pass these on to consumers.

Risky approaches being chosen

Many of our reports are about programmes where a risky way forward has been chosen. Our option appraisal examples show ambitious options being chosen without evidence that their benefits would justify the extra risk.

Image from NAO's electronic monitoring report

Example from new framework: In The new generation electronic monitoring programme we found that the Ministry of Justice (MoJ) chose to develop a new, ‘world-leading’ ankle tag and then also selected the highest-risk approach to the procurement. The process led to disputes with suppliers and two failed procurements. The MoJ finally concluded that abandoning the original plan was the least-bad option. After learning costly lessons and incurring significant delay, the MoJ changed its approach, purchasing off-the-shelf tags and bringing the integration function back in-house.

Consequences of trying to do too much

We have previously blogged on the essential need to prioritise. In addition, recent examples have made it clear how departments cannot fulfil their objectives if projects cost more than they can afford. For instance the original intention of the NHS vanguard programme to integrate health and social care services was not realised because funding was reallocated to deal with NHS Trusts’ financial deficits. The Ministry of Defence’s Equipment Plan is forecast to cost £7 billion more than its budget, but the Ministry does not have much scope to reduce funds for other budgets.

Optimism in one area has consequences elsewhere. Cost increases and delays leave departments facing unpalatable options. They may have to reduce the scope, delay the project or take funding from other priorities. For example, cost increases on Network Rail’s Great Western Railway modernisation and other rail programmes meant work was deferred and ultimately electrification projects on the Great Western, Trans-Pennine and Midland mainline routes were cancelled, as we set out in our investigation. Our report on Early progress in transforming courts and tribunals also showed how the programme reduced its scope and therefore its planned benefits because its initial timetable was too ambitious.

Cost estimates need to be challenged

With increasingly complex projects, it is not easy to reach a realistic initial cost estimate, especially when under pressure to make firm commitments early on. Those responsible for programmes can be well aware of the risks, but with many factors to consider it can be hard to spot the warning signs and to challenge enthusiastic backers of projects. To help them, our Survival guide to challenging costs in major projects sets out warning signs, dos and don’ts, useful questions to ask about cost estimates and management, and a glossary of costing terminology.

Image from NAO's guide to challenging costs in major projects

Some warning signs

  • It’s a “ground breaking” project”
  • Alternatives are ruled out too early
  • Soft benefits that take decades to deliver
  • “Don’t worry, the contractor bears all the risk”
  • The project is delayed to deal with cost overruns

Complex programmes are difficult to put into practice

Ambitious approaches also create challenges for later phases of programmes. Our recent blog on getting contracts right highlighted the difficulties of achieving workable contractual arrangements for complex programmes such as Crossrail and the Emergency Services Network programme. By contrast, the London 2012 programme, justifiably regarded as a successful programme, had contingency plans that allowed it to overcome last-minute difficulties.

Image from NAO's Update on Thameslink report

Example from new framework: In Update on the Thameslink Programme we found that inefficiencies and service delays arose because Network Rail had not initially set up its processes to deal with the volume of design change that turned out to be needed when conditions differed from those expected. Network Rail subsequently improved its financial control of the programme, introduced measures to improve the way it managed design changes and used more sophisticated cost forecasting techniques.

Building project management capability in the civil service remains a challenge

The government’s portfolio of major projects would be huge and complex, even without departmental commitments to EU Exit work. Our March 2017 report, Capability in the civil service, showed that government had identified skill shortages in digital, commercial and project delivery skills. And in People and skills: The role of the centre of government we reported that the Infrastructure and Project Authority was helping departments to build up their project and programme management capability, but they are reliant on a limited pool of experienced people.

Even the best designed programme can founder if there aren’t the skills to manage and deliver it in practice.


We hope that our framework of questions and costing guide help programme managers to address the unquestionably huge challenges they face.

Major programmes will undoubtedly continue to feature in the NAO’s work. The NAO always welcomes your feedback on our guides and our posts and we invite you to contact us directly.

Sandy GordonAbout the author: Sandy Gordon is an Audit Manager with over 25 years’ experience in conducting value-for-money studies, specialising in examining the implementation of new services and major change programmes across government. Sandy has led the NAO’s Project Delivery team, working to improve the NAO’s approach to examining programmes, including by supporting NAO teams’ use of this Framework to review programmes. He is leaving the NAO, after 33 years, at the end of July.



Share this article on social media:

Tagged:                     


2 responses to “Major programmes – what are we learning?”

  1. @JagPatel3 says:

    Earlier this year, the Public Accounts Committee released its report on the Ministry of Defence’s 10-year forward Equipment Plan. It concluded that for a second year in a row, the Equipment Plan is not affordable and went on to list failings in MoD’s procurement processes which have led to this sorry state of affairs.

    There is a desperate need for concerted leadership at the Ministry of Defence which has been plagued by whole range of problems – from a budget which is out of kilter with its Equipment Plan, to a defence industrial supplier base that has consistently failed to deliver within time and cost constraints.

    When it comes to procuring defence equipment for the Armed Forces, the government has no option but to rely on the Private Sector, because it no longer has the ability to produce military equipment – as it used to do.

    This is because the means of defence production, distribution and exchange is now exclusively in the hands of private interests, that is to say, the State is entirely dependent on for-profit organisations for the design, development, manufacture and delivery of new military equipment to the Armed Forces. Consequently, the government has become reliant on the Private Sector for all its military equipment needs, including its subsequent upkeep, when in-service with the user. The harsh reality is that, no department of state in Whitehall is as dependent on the Private Sector, as is the Ministry of Defence. Likewise, these private interests are completely hooked on a steady flow of taxpayer funds for their very existence.

    Consequently, as the only customer of defence equipment, the government is in a powerful position to dictate the terms of trade. But it has failed to leverage this enormous purchasing power to its advantage.

    The main reason for this failure is that people in Whitehall have little or no understanding of the forces at work and commercial pressures that exist within for-profit organisations which are ready to be harnessed for the benefit of taxpayers – not least, because they have not spent a single day of their lives in the Private Sector – and yet, they have been put in charge of spending taxpayers’ money to the tune of £15 billion per year to buy defence equipment, outsourced services and labour from the Private Sector.

    What’s more, the judgements made by these people, as it relates to the expenditure of public funds, are distorted by the fact that they will end up in the Private Sector to pursue a second career, later on. So, it comes as no surprise that MoD employees are, without exception, favourably disposed towards the defence industry because they are completely dependent on it for their subsequent career choices (via the revolving door), when their time in public service comes to an end, or their employment contract is terminated abruptly by political edict because they have been found wanting. Indeed, it is very hard to find anyone at MoD who will aggressively defend taxpayers’ interests, once they have enjoyed a cosy relationship with defence contractors. It is fair to say that they know which side their ‘bread is buttered’!

    This mass migration would explain why staff on defence contractors’ payroll is made-up entirely of people who were previously in the pay of the State.

    So, how did this total lack of diversity in the workforce of publicly-quoted companies, right across the full spectrum of defence engineering businesses and government outsourcing contractors, large and small, come about? The answer is clear, through the interfering hand of the State – in the shape of the Defence Secretary, who has been actively encouraging these defence contractors to hire people who were most recently in the pay of the State – thereby discriminating against other, equally deserving groups.

    Defence contractors may very well call themselves public companies and have their shares quoted on the stock market, but they really are Private Sector organisations in name only, not least, because they are ‘created in the image’ of public sector institutions like MoD Abbey Wood (warts ’n all) displaying all the tell-tale features of: a workforce made-up entirely of people who were previously in the pay of the State (where they developed an unhealthy penchant for rules, regulations and processes), who have succeeded in transplanting a work culture characterised by failed practices of management by committee & PowerPoint presentations and groupthink that disallows external challenge.

    What’s more, instead doing the right thing and educating people in the pay of the State about the ways of the Private Sector, these new arrivals (in association with those who have gone before) then set about exploiting the ignorance of their former colleagues at the Ministry of Defence, for one purpose only – relieving them of taxpayers’ money – which has, in itself, left the public finances in pretty bad shape.

    There is something very disturbing about people who have previously, as public servants sworn undying allegiance to Her Majesty Queen Elizabeth II, only to then engage in defrauding Her Majesty’s Government of taxpayers’ money on behalf of vested interests.

    The appallingly poor performance of indigenous defence contractors over the last several decades – characterised by persistent delays and cost overruns – has convinced some people to conclude that they pose the single biggest threat to the financial security of this country, aided and abated by people who were previously in the pay of the State.

    Remember the much-vaunted principles of selflessness, integrity, objectivity, accountability, openness, honesty and leadership in public life which are supposed to guide the conduct of public servants? Well, there is pitifully little sign of them right now. It seems that these values have been left behind in the public sector for others to cherish!
    @JagPatel3

    • Jo Stanwell says:

      Well said Jag Patel.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Right column

  • About the NAO blog

    Our experts share their views about issues and common challenges facing government, what public sector leaders should look out for and how organisations have addressed issues. Our posts draw together threads from across our reports, share secrets spilled in events and reveal our experts’ expectations for the future.

    We encourage comments that support the exchange of ideas for improvement, but ask that those posting are respectful of others.

  • Sign up for automatic feeds

    Sign up to receive email alerts:




    RSS IconSubscribe in an RSS Reader
  • Recent posts