Major project delivery
Posted on January 7, 2022 by Jemma Dunne
Ambulances need to travel fast! Ambulance drivers must take risks that regular drivers do not. This includes running red lights and travelling at high speeds through busy roads. However, to avoid accidents, precautions are taken to manage risks. The driver is trained, there are flashing blue lights and loud sirens.
Delivering programmes at speed requires a similar assessment of risks. In our recent lessons learned report we show that some programmes have successfully delivered quickly but not all – just as not all vehicles can be driven like ambulances. Speed creates greater risks which will not be appropriate or sustainable for every programme or organisation.
Should the risks of speed be taken?
Programmes may need to be delivered at speed for various reasons, including in an emergency or where there is a fixed deadline. We recently reported on the Kickstart Scheme launched by the Department for Work and Pensions (DWP). In response to a significant forecast rise in youth unemployment given the COVID-19 pandemic, DWP wanted to set up support quickly. It launched Kickstart on 2 September 2020, after only around six weeks of work, in time for the expected end of furlough in October 2020. We have also seen programmes delivered at speed as government simply wants to achieve outcomes sooner. A clear rationale for speed, can make it easier to get wider support and justify taking risks. Other drivers understand an ambulance’s need for speed and often make way.
Decision-makers need to understand ‘why speed’ to assess if the risks of speed are necessary and justifiable. Risks can include cost increases, not achieving outcomes, or people being diverted into a programme at the expense of other work. Our recent report on bounce back loans highlights the impact when risks are not managed – the Scheme facilitated faster lending by removing credit and affordability checks and allowing businesses to self-certify their application documents. Prioritising speed contributed to high levels of estimated fraud.
Monitoring and managing risks in practice
Where decision-makers choose to take the risks of delivering a programme quickly, they must proactively monitor and manage these increased and different risks. In November 2021, we shared insights from our lessons learned report with the Ministry of Justice team responsible for the Probation Reform Programme and the creation of the unified Probation Service to understand how this resonated with their practical experience. In June 2021, the Lord Chancellor had written to Parliament confirming probation services had been unified.
The team told us that they consciously chose to deliver at speed and identified a clear narrative for the reforms being at pace. As such, everyone was clear on the reasons for the reforms. The team also made clear that there was zero contingency beyond the expected delivery date. Alongside setting a minimum expectation of the requirements needed for Day 1, this helped force the pace and prioritisation of effort.
The programme team also highlighted the importance of strong leadership, with a culture of accountability and responsibility, to deal with any uncertainties or issues. In particular, they spoke of a culture which encouraged people to raise any problems they’d encountered, rather than hide them or focus on the ‘good news’.
Additionally, the programme team said they had built a strong internal assurance team, comprised of former senior operations staff, to carry out site visits and desktop reviews to ensure the programme was on track.
Alongside this, the programme team outlined the advantages of a flexible programme structure. The team recognised that it was difficult to plan everything up front, and instead ensured they had the required processes and information needed to respond quickly. This was done through regional teams, with a dedicated senior manager, tasked with identifying risks as soon as possible. This meant that the central programme team could deal with ‘unknown unknowns’ effectively when they arose.
Many of the points raised by the Probation Reform Programme team align with our insights. In particular:
- Including speed as a specific programme objective to provide a clear framework for decision-making and help make trade-offs between speed, cost and outcomes.
- Building teams with the right leadership, skills and experience to make clear, timely and reliable decisions.
- Tailoring processes to add value and momentum to programme decision-making.
- Recognising the uncertainties of delivering at speed and managing these.
As speed remains important for ambulances, so it will for some programmes, particularly with commitments to achieving ‘net zero’ greenhouse gas emissions by the fixed deadline of 2050. Our report helps those deciding whether to deliver at speed ask questions to determine when or how this should be done and then continually test whether a programme will achieve its outcomes.
- Lessons learned: Delivering programmes at speed
- Employment support: The Kickstart Scheme
- The Bounce Back Loan Scheme: an update
About the authors
Josh Perks is a qualified accountant with experience of working on the NAO’s transport team. His work has included audits of the main government transport bodies and value-for-money studies of major rail programmes. Recently, he has taken an active role in the NAO’s Major Projects Delivery Hub.
Jemma Dunne is an Audit Manager and has delivered value for money reports across areas such as health and defence, including those on government programmes. She is a qualified chartered accountant (FCA) and holds the APM Project Management Qualification (PMQ).
Posted on July 7, 2021 by Emma Willson
At the National Audit Office, we come across many business cases when looking at government programmes. A strong business case is vital for effective decision making and for successfully delivering intended outcomes.
The foundation of a business case is a clear understanding of what success will look like for a programme – the strategic case. But when it’s not clear what a programme is trying to achieve, it’s hard for decision makers to know if this programme is the right thing to do, or to plan and focus resources. It creates the risk that different stakeholders have different expectations about what will be achieved. It makes it harder to spot where other programmes may contribute to similar goals or where there may be adverse impacts. And for the public, parliament and us as auditors, it makes it hard to understand if the programme has delivered good value for money.
Promoting the strategic case
The November 2020 update to HM Treasury’s Green Book (its guidance on how to appraise and evaluate spending proposals) introduces a stronger requirement to establish clear objectives up front. Proposals should be assessed primarily on how well they deliver policy objectives, and cannot be considered value for money without delivering these.
But for proposals to be assessed this way, the strategic case needs to be robust. Therefore, when auditing major programmes, we ask the seemingly simple question – is it clear what objective the programme is intended to achieve?
Our recent learning from COVID-19 re-emphasised the importance of government being clear and transparent about what it is trying to achieve, so that it can assess if it is making a difference. For example, HM Revenue & Customs agreed clear principles for its employment support schemes. Although the Bounce Back Loans Scheme achieved its initial objective of quickly supporting small businesses, a lack of more detailed scheme-specific objectives will make it difficult to measure its ultimate success.
The government’s commitment to ‘levelling up,’ and uncertainty over what this means, may create difficulties for programmes to set out what they will achieve. They will need clarity to produce a business case. It could be interpreted as giving everyone access to the same opportunities, or at least to the same minimum standards – say of health outcomes or broadband access. This prioritises spreading prosperity to deprived areas. However, it also could be framed as addressing gaps in potential by, for example, investing where an area should be showing higher productivity. This prioritises value for money investments. As these different goals require different policy solutions, it can be challenging to set out how an intervention will achieve ‘levelling up’. Later this year, government will publish a levelling up White Paper setting out how new policies will improve opportunity and livelihoods across the UK.
Whilst defending the economic case…
A strong strategic case alone does not mean an intervention is justified. There might be other ways to meet an objective which could be better value for money. We often see business cases that seem to justify a pre-selected solution, rather than exploring a range of options for meeting the objectives – what the Green Book calls ‘the long list’.
Our report on Hinkley Point C found that alternative ways of the government providing support for the planned nuclear power station could have resulted in lower costs to consumers over the life of the project, but weren’t considered. We have also seen departments not considering different options when thinking about how to deliver policy – nine out of the 24 business cases we reviewed as part of our report on arm’s length bodies did not consider a long-list of options.
The economic case is important in setting out value for money, often through formal modelling, the results of which will need to be considered alongside the strategic case. Our early work on High Speed 2 found that the relationship between savings (with the Department for Transport putting a high emphasis on journey time savings) and the strategic reasons for doing the programme, such as rebalancing regional economies, was unclear.
So, what do we expect from strategic cases?
Throughout a programme, the strategic case needs to help ensure effective decision-making. As well as specifying what should be achieved (with a clear, logical set of assumptions) it needs to:
- Be easily understandable so effective trade-offs can be made. Our lessons from major programmes describes how objectives need to be clear enough to be translated into a programme scope (what will be required and when). For example, government has been considering which objectives to prioritise for the roll-out of gigabit-capable broadband. In our report, we found that prioritising the speed of programme delivery over other objectives posed a risk to value for money.
- Help prioritise cross-government objectives. We see cases where objectives are neither coherent when taken together, nor clearly prioritised when tensions emerge between them. In November 2020 we considered progress in achieving government’s long-term environmental goals. The government set out 10 overarching goals but did not provide a clear and coherent set of objectives, with, for example, varying and often unclear timescales.
- Be measurable (where possible). The strategic case will capture those assumptions that cannot be equated to a monetary equivalent. And, the easier assumptions are to quantify, the easier it will be to assess progress. Our early High Speed 2 review found the strategic case should have been more developed. For example, it included limited evidence on forecast passenger demand which provided a weak foundation for securing and demonstrating success. The Department was working to strengthen its analysis. Also, our Hinkley Point C report found the Department put more weight on the wider, unquantified strategic case when the economic case weakened but had little control over these benefits and at the time of our report no plan to realise them.
Government plans to invest heavily in programmes, with £100 billion expected investment in 2021-22 alone. For government to secure best value from this it must set out clearly and logically what it wants, how to best deliver this and how it will show what has been achieved for the investment.
Authors: Ruth Kelly and Emma Willson
Emma Willson leads our Major Projects hub. She has worked at the NAO for almost 20 years, auditing a wide range of government programmes, from welfare reform to large-scale defence equipment projects. She is a qualified chartered accountant and holds an International Association for Contract and Commercial Management (IACCM) qualification.
Ruth Kelly is our Chief Analyst and has wide experience of applying economics and other analytical approaches to support policy evaluation, investment decisions and risk management. Prior to joining the NAO, she held business evaluation and risk management roles for a global resources company, and advised clients on carbon and energy issues for a Big 4 economic consultancy practice.
Posted on April 20, 2021 by Emma Willson
We are keen to support those responsible for both delivering and overseeing government programmes. In that spirit, we today publish our Framework to review programmes summarising what we ask when auditing major programmes. It draws from our experience of around 200 studies and includes examples of what we have seen.
Our work provides insights on the challenges in getting government programmes right. Programmes often encounter difficulties, take longer and cost more than planned, and don’t deliver the intended aims, with significant and high-profile consequences. Our unique remit means we can reflect on this across the lifecycle of different programmes, whether they aim to deliver long-term government ambitions (e.g. net zero); infrastructure (e.g. High Speed Two); or processes (e.g. offender rehabilitation).
We repeatedly see similar problems, many of which have their root causes within the programme scope; cost and schedule planning; interdependencies; and oversight. In November, we published Lessons learned from Major Programmes which examined these root causes and what government can learn from them. This included ensuring the management approach evolves over time and the need to consider operational planning from the start. These lessons will relate, to a lesser or greater extent, to all programmes. For example, ambitious technology-enabled business change may mean digital programmes need to handle greater uncertainty, which we will report on later this year.
These lessons are not new. You will have heard them before and organisations like the Infrastructure and Projects Authority and Association for Project Management share similar insights. For example, the IPA’s Principles for project success outlines the need to plan realistically, tell it like it is and control the scope. However, given this consistency of thinking, why are things so hard to change? What can be done?
Although not necessarily the golden bullet, we see real value in those managing and overseeing programmes asking themselves the seemingly simple and straightforward questions, particularly early in a programme. Our framework shares 18 possible questions across four themes – purpose; value; set-up; and delivery and variation management.
In terms of purpose, you want to ask – Is it clear what objective the programme is intended to achieve? Does the programme make sense in relation to the organisation’s strategic priorities?
HM Treasury guidance requires departments to establish a strategic case setting out how a programme will meet an underlying rationale and objectives. However, we often see bodies struggle to maintain a clear focus on a programme’s objectives. Government major programmes are likely to have multiple objectives, sometimes involving more than one department, and we have seen a lack of coherency and prioritisation. For example, in November 2020 we considered progress in Achieving government’s long-term environmental goals. Government’s plan bringing together commitments and aspirations did not provide a clear and coherent set of objectives so it was hard to determine how the ambitions related to pre-existing targets and each other.
A clear scope and objectives enable government to make trade-offs and better understand the impact of other projects and programmes. Our report on improving the A303 between Amesbury and Berwick Down found that the project could only fully deliver its strategic objectives as part of a completed A303/ A358 corridor. This required further projects, with five of these being individually appraised as low to poor value for money.
In terms of value, you may want to ask – Are cost and duration estimates appropriate to the stage of the programme, with risks and uncertainties appropriately reflected?
Does the programme have a plan to deliver benefits and is this being implemented?
We have reported how the relative lack of information early in a programme means estimates will be highly uncertain. In many programmes we reviewed, governments have not sufficiently recognised these inherent uncertainties and risks. We have seen how using these estimates as targets can drive behaviours detrimental to successful delivery.
We highlighted this in our report on High Speed Two progress, whilst the framework references our early review of the risks faced by Parliament’s Restoration and Renewal. This recognised that uncertainties needed to be understood and recommended developing evidence based cost and time ranges with milestones setting out when estimates could be reassessed and the ranges narrowed. This drew from our survival guide which offers senior decision makers factors to consider when challenging costs.
Our set-up questions include – Does the programme have the right culture and leadership with the necessary authority and influence? Are key risks identified, understood and addressed? Whilst on delivery and variation management – Is the programme sufficiently flexible to deal with setbacks and changes in the operating context? Does the programme have a clear plan for transfer to operations/business as usual?
With COVID-19, EU Exit and net zero, government is undertaking more programmes. And as the complexity of these, and the overarching landscape increases, getting the basics right has never been more important. There have been notable improvements, and the government’s aim to set out requirements and best practice guidance is a good thing. Now is the time to reflect and learn from the past. Programme delays and increased costs have repercussions for the programme itself, other programmes and overall expenditure at a time when resources are constrained.