Posted on October 10, 2017 by Sandy Gordon
Major programmes are expensive, high profile and carry great uncertainties and risks. For most government bodies, undertaking a major programme will involve doing something new, with relatively little organisational experience. Many fall short of their objectives, in terms of cost and/or outcomes. So it’s not surprising that they are the focus of many NAO reports – about 100 since 2010. Our new Framework to review programmes shows the questions we typically ask, and brings together many of our recent findings. We hope it will show what we are looking for and what we expect to see when we examine major programmes.
Our reports illustrate how risky programmes can be. If they’re innovative, those risks are higher, and it’s harder to learn from past experience. If they’re complex they are affected by many unpredictable factors. And the scale of challenge is increasing as government bodies support major new infrastructure projects, introduce new technology and reorganise to make the most of scarce resources as well as implementing the changes necessary as a result of leaving the European Union.
We tend to look at the biggest and riskiest programmes at key points during their life, or when a particular event prompts a review. Our examinations may seek to address different questions depending on the context, but our Framework to review programmes sets out 18 key questions that are likely to be important, with links to examples from our past work and to further more detailed guidance on specific issues.
Learning from successes and failures
Our framework of questions is based on lessons we’ve learnt from our previous reports. These are just some of the examples highlighted in our Framework.
Managing interdependencies: In our review, Modernising the Great Western railway, we found that Network Rail had not worked out the minimum feasible schedule for the work, including dependencies between key stages. This led us to focus our questions more how on interdependencies are managed within programmes.
Learning from early experience: Some of our examples highlight progress made in projects after we identified problems at the early stages. For instance our report on Progress with preparations for High Speed 2 said the Department for Transport had taken steps to address weaknesses in the business case we reported on in 2013, and had learned from High Speed 1, where the benefits had not materialised as expected, as we reported in The completion and sale of HS1.
Sharing lessons across a portfolio: In our report Welfare reform – lessons learned, we found that the Department for Work and Pensions (DWP) implemented around 30 distinct programmes over five years with few operational problems. This was largely because DWP learnt from early failings such as: sticking too rigidly to fixed deadlines; thinking too late about the management information and the leading indicators it needed to understand progress and performance; and relying too much on dealing with difficulties as they emerged rather than anticipating what might go wrong.
Sharing lessons more widely
From our review of DWP’s transformation on this huge scale we produced this Briefing: Lessons for major service transformation (pdf – 116KB), which details 11 lessons:
- Transformation programmes raise the greatest risks of failure
- Set realistic goals and be honest about what really matters
- Policy development must take account of implementation
- Don’t be tempted to score benefits early
- Do identify tangible short-term gains
- Recognise the (senior) organisational cost of transformation
- Don’t underestimate what you can learn from engagement
- Recognise the value of learning and market development
- Do anticipate the need to make changes in live running
- Recognise the opportunities and limits of technology
- Set out clear decision-making and challenge
Lessons from successful programmes: Our review of the London 2012 Olympics found that the successful delivery of the Games owed much to: improvements made to portfolio management in 2009; the governance and oversight structure; and a risk management approach that included contingency planning and intensive scenario testing.
Focusing on the key issues
To deal with the range and complexity of issues we might look at, the Framework allows us to go into plenty of detail if needed: it contains 107 detailed sub-questions and also points to other resources that help with more specific aspects of programmes, such as contract management or payment-by-results schemes.
Some of these tools are also mentioned in our previous post, The challenges of major projects, which outlines five key areas in which poor performance is commonly found. As that post notes, further useful NAO publications can be found on our Managing major projects web-page.
We have found it useful always to consider the four key areas shown in this diagram, which are good questions to ask about any project or programme.
The Framework is just a starting point, and not intended to be a checklist. We ask the key questions first, then use the detailed questions and more specific tools to probe deeper into areas of particular interest for each programme.
We expect to update the Framework and refresh it with new examples as we learn from ongoing and future work. So we strongly encourage your feedback, and invite you to comment on this post or contact us.
About the author: Sandy Gordon is an Audit Manager with over 25 years’ experience in conducting value-for-money studies, specialising in examining the implementation of new services and major change programmes across government. Sandy currently leads the NAO’s Project Delivery team and is working to improve the NAO’s approach to examining programmes, including by supporting NAO teams’ use of this Framework to review programmes.
Posted on September 29, 2017 by Paul Wright-Anderson
Government is being transformed through changes ranging from digital delivery to downsizing and devolution. To match new delivery approaches, buildings need modernisation, organisations need to be co-located, and appealing locations should help to attract the necessary skills. In response, government property is undergoing a revolution, with regional “Hubs” and an integrated public sector estate. But there are big challenges to be addressed. Can the public sector work together to achieve the expected benefits? more… The future of government offices: early lessons
Posted on August 18, 2017 by Jacob Holliday
High profile incidents remind individuals and organisations just how important it is to manage potential conflicts of interest. In March 2017, the Court of the Bank of England commissioned a review of the institution’s approach to managing conflicts of interest. The review was prompted by the resignation of Charlotte Hogg who had been Deputy Governor for Markets and Banking. A report from the House of Commons Treasury Select Committee published in March 2017 had raised concerns about her compliance with the Bank’s rules governing conflicts of interest. As part of its Review the Bank invited the NAO to contribute a briefing paper on managing conflicts of interest. It’s a timely opportunity for the whole public sector to consider whether it’s managing conflicts of interest effectively. more… Managing conflicts of interest and keeping public trust
Tagged: Accountability, Business operations, Conflict of interest, Contract management, Cross-government, Employee engagement, Good practice principles, International, Investigations, Local government, Regulation & consumer protection, Reporting, Risk management
Posted on July 20, 2017 by George Crockford
The civil service is under pressure, as we found in our recent report Capability in the Civil Service. It has lost one in four civil servants since 2006 – with no reduction in workload, there’s a growing number of major projects to implement, greater public demand for services, new technologies – bringing both opportunities and threats, new ways of delivering public services, and action needed to leave the European Union. How can public sector organisations get or develop the people and skills they need? The first thing is: prioritise; it simply must do less. more… Stretching civil servants’ capability
Posted on July 13, 2017 by Alison Taylor
Billions of pounds of taxpayers’ money is lost every year through fraud. To put this money to its proper use, in the report Eliminating Public Sector Fraud, the government identified the need to adopt a zero tolerance culture and a coordinated approach to tackling fraud. But fraud is complex and many organisations lack the required skills, making it even more important that public sector organisations collaborate, innovate and share lessons. Our recent Investigation into Department for International Development’s approach to tackling fraud has lessons for all organisations seeking to tackle fraud, as well as exploring some of the challenges in delivering international aid. more… Lessons in fraud prevention, detection and recovery
Tagged: Accountability, Cost reduction, Cross-government, Fraud and error, Good practice principles, International, Investigations, Overseas aid, Process management, Public finances, Reporting, Risk management
Posted on July 7, 2017 by Lee Summerfield
MPs have queried the way grants are distributed; multiple programme changes have left many questioning aims and effectiveness; growing backlogs have raised issues about public service delivery; and failed contracts losing millions of pounds have implied serious mismanagement. Concerns raised by MPs, members of the public and our own staff as a result of their work often need the facts set out – clearly and quickly. They call for an NAO ‘Investigation’. more… NAO Investigates
Posted on June 29, 2017 by Andy Morrison
Do you pay all the tax you should? Chances are, this depends on your values, your familiarity with tax rules, opportunities for evasion, and your attitudes. These may be affected by how much help you get from HM Revenue and Customs (HMRC) – and could potentially be influenced by how happy you are with HMRC’s service. In our recent report on The quality of service for personal taxpayers, we said we would work with HMRC to explore whether service quality affects tax revenues. The way we have done this is a great example of how the NAO can help to drive public service improvement. more… Can better service bring in more tax revenue? How the NAO is helping government investigate
Posted on June 23, 2017 by Tom McDonald
WannaCry, the 12th of May global cyber attack, brought home clearly one of the key cyber security risks to government services: loss of access to data. This ransomware attack didn’t target the NHS, but the NHS was particularly affected by it, causing extensive disruption to patients and healthcare for a week. With digital transformation of public services a key government priority, what lessons from this episode can the government learn to protect public services from cyber attacks? more… WannaCry: what does it mean for government?
Posted on April 27, 2017 by Administrator
The NAO Blog must take a break in the period before the General Election. Meanwhile, we would like to highlight our interactive guide to Good practice from the ‘Excellence in Reporting’ in the Public Sector Award, sponsored by the NAO and part of the wider PwC-sponsored Building Public Trust in Corporate Reporting Awards. Kate Mathers, one of our Executive Leaders, sits on the judging panel for the public sector awards. We set out below the criteria for which we looked. Our publication provides examples of good practice from the 50 reports we reviewed. more… Excellence in Public Sector Reporting and the Election break
Posted on April 7, 2017 by Emma Willson
When it comes to public sector contracts, running proper competitions simply isn’t enough. Government needs to act intelligently to take advantage of competition between potential suppliers, weighing up options and considering who carries the risks, and at what cost. Since launching Commercial and contract management – insights and emerging best practice we’ve reported on two contracts that shed further light on these issues. We explore the new insights from these reports in this blog-post, the latest in our series on key issues for contracts and commercial relationships. more… Intelligent contract competition and risk management