Major project delivery
Posted on July 7, 2021 by Emma Willson
At the National Audit Office, we come across many business cases when looking at government programmes. A strong business case is vital for effective decision making and for successfully delivering intended outcomes.
The foundation of a business case is a clear understanding of what success will look like for a programme – the strategic case. But when it’s not clear what a programme is trying to achieve, it’s hard for decision makers to know if this programme is the right thing to do, or to plan and focus resources. It creates the risk that different stakeholders have different expectations about what will be achieved. It makes it harder to spot where other programmes may contribute to similar goals or where there may be adverse impacts. And for the public, parliament and us as auditors, it makes it hard to understand if the programme has delivered good value for money.
Promoting the strategic case
The November 2020 update to HM Treasury’s Green Book (its guidance on how to appraise and evaluate spending proposals) introduces a stronger requirement to establish clear objectives up front. Proposals should be assessed primarily on how well they deliver policy objectives, and cannot be considered value for money without delivering these.
But for proposals to be assessed this way, the strategic case needs to be robust. Therefore, when auditing major programmes, we ask the seemingly simple question – is it clear what objective the programme is intended to achieve?
Our recent learning from COVID-19 re-emphasised the importance of government being clear and transparent about what it is trying to achieve, so that it can assess if it is making a difference. For example, HM Revenue & Customs agreed clear principles for its employment support schemes. Although the Bounce Back Loans Scheme achieved its initial objective of quickly supporting small businesses, a lack of more detailed scheme-specific objectives will make it difficult to measure its ultimate success.
The government’s commitment to ‘levelling up,’ and uncertainty over what this means, may create difficulties for programmes to set out what they will achieve. They will need clarity to produce a business case. It could be interpreted as giving everyone access to the same opportunities, or at least to the same minimum standards – say of health outcomes or broadband access. This prioritises spreading prosperity to deprived areas. However, it also could be framed as addressing gaps in potential by, for example, investing where an area should be showing higher productivity. This prioritises value for money investments. As these different goals require different policy solutions, it can be challenging to set out how an intervention will achieve ‘levelling up’. Later this year, government will publish a levelling up White Paper setting out how new policies will improve opportunity and livelihoods across the UK.
Whilst defending the economic case…
A strong strategic case alone does not mean an intervention is justified. There might be other ways to meet an objective which could be better value for money. We often see business cases that seem to justify a pre-selected solution, rather than exploring a range of options for meeting the objectives – what the Green Book calls ‘the long list’.
Our report on Hinkley Point C found that alternative ways of the government providing support for the planned nuclear power station could have resulted in lower costs to consumers over the life of the project, but weren’t considered. We have also seen departments not considering different options when thinking about how to deliver policy – nine out of the 24 business cases we reviewed as part of our report on arm’s length bodies did not consider a long-list of options.
The economic case is important in setting out value for money, often through formal modelling, the results of which will need to be considered alongside the strategic case. Our early work on High Speed 2 found that the relationship between savings (with the Department for Transport putting a high emphasis on journey time savings) and the strategic reasons for doing the programme, such as rebalancing regional economies, was unclear.
So, what do we expect from strategic cases?
Throughout a programme, the strategic case needs to help ensure effective decision-making. As well as specifying what should be achieved (with a clear, logical set of assumptions) it needs to:
- Be easily understandable so effective trade-offs can be made. Our lessons from major programmes describes how objectives need to be clear enough to be translated into a programme scope (what will be required and when). For example, government has been considering which objectives to prioritise for the roll-out of gigabit-capable broadband. In our report, we found that prioritising the speed of programme delivery over other objectives posed a risk to value for money.
- Help prioritise cross-government objectives. We see cases where objectives are neither coherent when taken together, nor clearly prioritised when tensions emerge between them. In November 2020 we considered progress in achieving government’s long-term environmental goals. The government set out 10 overarching goals but did not provide a clear and coherent set of objectives, with, for example, varying and often unclear timescales.
- Be measurable (where possible). The strategic case will capture those assumptions that cannot be equated to a monetary equivalent. And, the easier assumptions are to quantify, the easier it will be to assess progress. Our early High Speed 2 review found the strategic case should have been more developed. For example, it included limited evidence on forecast passenger demand which provided a weak foundation for securing and demonstrating success. The Department was working to strengthen its analysis. Also, our Hinkley Point C report found the Department put more weight on the wider, unquantified strategic case when the economic case weakened but had little control over these benefits and at the time of our report no plan to realise them.
Government plans to invest heavily in programmes, with £100 billion expected investment in 2021-22 alone. For government to secure best value from this it must set out clearly and logically what it wants, how to best deliver this and how it will show what has been achieved for the investment.
Authors: Ruth Kelly and Emma Willson
Emma Willson leads our Major Projects hub. She has worked at the NAO for almost 20 years, auditing a wide range of government programmes, from welfare reform to large-scale defence equipment projects. She is a qualified chartered accountant and holds an International Association for Contract and Commercial Management (IACCM) qualification.
Ruth Kelly is our Chief Analyst and has wide experience of applying economics and other analytical approaches to support policy evaluation, investment decisions and risk management. Prior to joining the NAO, she held business evaluation and risk management roles for a global resources company, and advised clients on carbon and energy issues for a Big 4 economic consultancy practice.
Posted on April 20, 2021 by Emma Willson
We are keen to support those responsible for both delivering and overseeing government programmes. In that spirit, we today publish our Framework to review programmes summarising what we ask when auditing major programmes. It draws from our experience of around 200 studies and includes examples of what we have seen.
Our work provides insights on the challenges in getting government programmes right. Programmes often encounter difficulties, take longer and cost more than planned, and don’t deliver the intended aims, with significant and high-profile consequences. Our unique remit means we can reflect on this across the lifecycle of different programmes, whether they aim to deliver long-term government ambitions (e.g. net zero); infrastructure (e.g. High Speed Two); or processes (e.g. offender rehabilitation).
We repeatedly see similar problems, many of which have their root causes within the programme scope; cost and schedule planning; interdependencies; and oversight. In November, we published Lessons learned from Major Programmes which examined these root causes and what government can learn from them. This included ensuring the management approach evolves over time and the need to consider operational planning from the start. These lessons will relate, to a lesser or greater extent, to all programmes. For example, ambitious technology-enabled business change may mean digital programmes need to handle greater uncertainty, which we will report on later this year.
These lessons are not new. You will have heard them before and organisations like the Infrastructure and Projects Authority and Association for Project Management share similar insights. For example, the IPA’s Principles for project success outlines the need to plan realistically, tell it like it is and control the scope. However, given this consistency of thinking, why are things so hard to change? What can be done?
Although not necessarily the golden bullet, we see real value in those managing and overseeing programmes asking themselves the seemingly simple and straightforward questions, particularly early in a programme. Our framework shares 18 possible questions across four themes – purpose; value; set-up; and delivery and variation management.
In terms of purpose, you want to ask – Is it clear what objective the programme is intended to achieve? Does the programme make sense in relation to the organisation’s strategic priorities?
HM Treasury guidance requires departments to establish a strategic case setting out how a programme will meet an underlying rationale and objectives. However, we often see bodies struggle to maintain a clear focus on a programme’s objectives. Government major programmes are likely to have multiple objectives, sometimes involving more than one department, and we have seen a lack of coherency and prioritisation. For example, in November 2020 we considered progress in Achieving government’s long-term environmental goals. Government’s plan bringing together commitments and aspirations did not provide a clear and coherent set of objectives so it was hard to determine how the ambitions related to pre-existing targets and each other.
A clear scope and objectives enable government to make trade-offs and better understand the impact of other projects and programmes. Our report on improving the A303 between Amesbury and Berwick Down found that the project could only fully deliver its strategic objectives as part of a completed A303/ A358 corridor. This required further projects, with five of these being individually appraised as low to poor value for money.
In terms of value, you may want to ask – Are cost and duration estimates appropriate to the stage of the programme, with risks and uncertainties appropriately reflected?
Does the programme have a plan to deliver benefits and is this being implemented?
We have reported how the relative lack of information early in a programme means estimates will be highly uncertain. In many programmes we reviewed, governments have not sufficiently recognised these inherent uncertainties and risks. We have seen how using these estimates as targets can drive behaviours detrimental to successful delivery.
We highlighted this in our report on High Speed Two progress, whilst the framework references our early review of the risks faced by Parliament’s Restoration and Renewal. This recognised that uncertainties needed to be understood and recommended developing evidence based cost and time ranges with milestones setting out when estimates could be reassessed and the ranges narrowed. This drew from our survival guide which offers senior decision makers factors to consider when challenging costs.
Our set-up questions include – Does the programme have the right culture and leadership with the necessary authority and influence? Are key risks identified, understood and addressed? Whilst on delivery and variation management – Is the programme sufficiently flexible to deal with setbacks and changes in the operating context? Does the programme have a clear plan for transfer to operations/business as usual?
With COVID-19, EU Exit and net zero, government is undertaking more programmes. And as the complexity of these, and the overarching landscape increases, getting the basics right has never been more important. There have been notable improvements, and the government’s aim to set out requirements and best practice guidance is a good thing. Now is the time to reflect and learn from the past. Programme delays and increased costs have repercussions for the programme itself, other programmes and overall expenditure at a time when resources are constrained.