Left column

Middlecolumn

Latest posts

    Improving government data: what needs to change? 

  • Posted on July 25, 2022 by

    If the government wants to modernise, it must start with its data – but addressing the government’s data issues isn’t easy. 

    For 25 years, the government has wanted to improve and make better use of its data. Using data well is essential for a cheaper, more efficient government and better services for citizens. But progress in this area has been slow.  

    Part of this is due to the government being weighed down by the baggage of legacy systems and ways of working. This puts it at a disadvantage to newer organisations like Amazon or Netflix which were designed and built for effective use of data from the outset. 

    What are the questions to resolve? 

    There is work to do, but what exactly? Our new guide to improving government data aims to help decision-makers understand the current issues in more detail, and the ways they can be addressed. 

    The government now has new plans to improve data, with the Central Digital and Data Office and the Office for National Statistics are now  doing some good work. in this area. For these plans to have the best chance of success, all areas of government must understand the barriers that have held back progress in the past. 

    Our guide addresses some of the key issues, such as:  

    • Data sharing: Why the answer is more than just individual data sharing agreements 
    • Data quality: Why it matters, and why problems with data quality persist 
    • Data standards: Why they are so hard to implement in practice 
    • Creating cross-government data sets: Why further questions arise when using each other’s data to create single data sets 
    • Data analytics: Why advanced technologies can’t solve all the problems  

    The government needs to acknowledge and address these issues and understand why previous initiatives have not had the success they hoped for. 

    Where might the answers lie? 

    Our guide sets out the steps needed to create the foundations for improving government data. Managing and improving data cannot be achieved without focused effort and prioritisation and having the right amount of funding to back it up. In the past, when the difficulties with data initiatives have got too hard, they have petered out. 

    This needs to change. It’s a big challenge, but addressing these issues could unlock substantial benefits for the public and the government alike. 

    About the author: 

    Yvonne Gallagher

    Yvonne Gallagher

    Yvonne is our digital transformation expert, focused on assessing the value for money of the implementation of digital change programmes. Yvonne has over 25 years’ experience in IT, business change, digital services and cyber and information assurance, including as CIO in two government departments and senior roles in private sector organisations, including the Prudential and Network Rail.

    Comment on this post...

  • Achieving value for money for government’s environmental goals

  • Posted on June 20, 2022 by

    Gareth Davies

    2021 was a big year for government’s commitments to the environment. The statutory target to achieve net zero greenhouse gas emissions by 2050 was backed up with a strategy and the Environment Bill gives legal force to nature recovery targets.  The challenge now lies in turning targets and strategies into actions that will combat rising temperatures and restore our natural environment in a way which represents both short- and long-term value for money for taxpayers at a time when households and businesses are under intense financial pressure.

    The Treasury acknowledged in its recent Net Zero Review that ‘the transition has implications for current and future taxpayers’ but ‘policy to support the transition can help make the most of the opportunities and keep costs down’.

    The scale and complexity of the task is daunting which is why external scrutiny is critical. Government has recognised this by creating the Climate Change Committee and the Office for Environmental Protection to help monitor progress towards net zero and restoring the natural environment. Meanwhile, the National Audit Office’s role will be to scrutinise how efficiently and effectively public money is spent to achieve those goals.

    The risks of waste, escalating costs, and projects failing to deliver significant environmental benefit are high, while failing to prepare for a changing climate is likely to impose a heavy cost on future generations. We will be keeping a close eye on how well government is spending to achieve both short-term and long-term value for money.

    Our long experience of scrutinising government spending already offers many lessons. We see three particularly significant challenges: the need for a whole-economy response; the critical role of effective public engagement; and the importance of costed programmes with disciplined management.

    The changes required won’t be achieved unless government action supports effective market responses, engaging public, private and voluntary sectors. We have seen the cost of failing to do this with the Green Homes Voucher Scheme where the complexity and certification requirements hindered energy-efficiency installers from mobilising to meet demand. But when government gets this coordination right, as with the covid vaccine roll-out, it is a force multiplier.

    The next phase of decarbonising the economy needs buy-in from consumers and citizens. With energy and fuel costs already high, government will need to be clear on how the cost of replacing gas boilers or switching to electric cars will be managed. Our work has shown that interventions designed to promote behaviour change need careful testing if they are to be successful. The mandatory charge for plastic shopping bags has significantly reduced their use. But the roll-out of smart meters saw energy companies find it difficult to get customers to commit to installation appointments.

    New technologies and innovative policies will be needed to limit the impact of climate change. However, the novelty of the problem must not distract from getting the basics right. Implementing these new solutions will require costed strategies with clear objectives, active risk management, and ways to evaluate progress so it can correct course when needed. We have often seen this go wrong, such as with the Emergency Services Network which has been beset with delays and cost overruns. A lack of robust planning failed to consider first-responder needs and how the new communications technology will integrate with other systems. This is why, as with our report yesterday on tackling local air quality breaches, it is important we look at programmes while they are underway to test that they are set up for success.

    We must also consider the longer term. Poor investment decisions for infrastructure such as flood defences could result in far higher costs if they do not make us resilient in dealing with a changing climate. When we looked at the deal government struck to build Hinkley Point C we were concerned that it had not fully assessed the long-term costs for consumers. Government will need to learn from that as it seeks to deliver on its energy security strategy.

    The COVID pandemic has vividly shown the cost of inadequate preparation for whole system emergencies. At a time of rising prices for consumers and inflation eating away at government spending power, government must not only spend wisely now but also for future generations.

    This article was first published in the Daily Telegraph

    Tagged:

    Comment on this post...

  • Mapping the Justice landscape 

  • Posted on May 3, 2022 by

    James Gjertsen explains how the NAO uses spatial analytics to bring audit teams closer to the data and visualise the complex geographic relationships at work behind the scenes in the justice system 

    From public transport to healthcare, we use services and infrastructure provided by central and local government all the time. Multiple Government departments have to coordinate long term decisions about provision of services and land use – and these all come together in a place. Bringing a place-based focus to decisions is key to maximising the positive impact government can have on our lives. This is why the NAO has an increasing interest in the power of spatial analysis to provide insight.   

    Even within a single policy area, such as health or justice, one department or ministry can be responsible for managing multiple different types of organisations that need to work together to deliver a coordinated service for users. One area we recently looked at was the criminal justice system.  

    This involves a range of organisations, from police forces to courts and probation. Mapping the spatial relationships between these organisations helps us answer deeper questions about the co-ordination and delivery of those services. We can ask more insightful questions about where backlogs are occurring or how the user experiences their linked-up journey through the system, and spot good practice. But all this starts with how these organisations relate to one another spatially. 

    To explore this, we developed an interactive tool visualising how all these various organisations interact with each other. You can use it to see at a glance where those interactions are simple and where they may be more complicated, along with summaries for each body.  

    This quickly shows that the number of organisations interacting with each other can add up fast: for police forces the Metropolitan Police tops the list for the most interactions with 63 local interacting organisations! This includes one prison region, HM Courts and Tribunals Service cluster and youth Justice Board, two Crown Prosecution regions, nine local justice areas, eighteen Probation Delivery Units and thirty-one Youth offending teams.  

    The tool can help raise questions about the efficiency of the criminal justice system by providing geographic context around the interactions between the various bodies.  

    Under the hood 

    The tool uses local authority boundaries as the building blocks for each organisation’s map layer. As an example: For probation delivery units, Cardiff, Newport and the Vale of Glamorgan are listed under the same unit, “Cardiff and the Vale”. The tool joins these local authorities’ boundaries together to produce a boundary that represents the probation delivery unit.  

    Converting local authority boundaries to probation delivery units in Wales 

    There were benefits to using this approach. Firstly, a few of the organisations didn’t have a set of predetermined boundaries to provide us, so using this approach let us construct them from scratch.  

    Secondly, using the same building blocks for each layer makes comparing them much easier. Boundaries will often be simplified to shrink the size of files being dealt with. Imagine trying to record the borders of a town. If you took measurements every 500 meters you would need far fewer measurements than if you took them every ten centimetres, and for some purposes the result would be just as useful.  

    However, it can be hard to compare boundaries that may have been recorded differently. Using the same base boundaries to build all the layers simplifies things. They will all be recorded in the same way and any differences that get flagged will be genuine, no extra work needed. 

    We’ve been building our capacity at the NAO to use data and spatial analytics to derive deeper insights around the topics we audit. This tool is just one of the ways we are supporting our current and future work by bringing teams closer to the data, by clearly articulating the complex geographic relationships at work behind the scenes. Why not see for yourself? 

    About the author

    James Gjertsen

    James Gjertsen is a Senior Analyst in the Analysis Hub at the NAO. He joined the NAO in 2018 and leads the Analytical Insights Team which aims to support the NAO’s VFM work programme by  drawing deeper insights from the data it collects.

    1 Comment

  • Taking on the challenge of public sector cyber security

  • Posted on March 8, 2022 by

    The government recently published its new Cyber Security Strategy specifically aimed at building a cyber resilient public sector. Resilience is key in underpinning its vision to make the UK a cyber power in a world increasingly shaped by technologies that offer many benefits but also pose risks. The strategy reiterates that government remains an attractive target for a broad range of malicious actors with 40% of incidents 2020-21 affecting the public sector. 

    The main benefits highlighted are the need to protect key UK assets and the uninterrupted continuation of vital services. The strategy also aims to enable the development of skills and capability in cyber awareness and risk management.   

    This has been a major theme in our work, we recently published our good practice guide, aimed at Audit Committees, on cyber and information security where we set out the type of risk and capability management in relation to cyber security we would expect to see in organisations. 

    In order to harden government to cyber-attack and build the required resilience in the public sector by 2030, the Cyber Security strategy has two main pillars and five objectives:  

    Pillar 1Build organisational cyber resilience Pillar 2 – ‘Defend as one’
    Objective 1

    Manage cyber security risk
    Objective 2

    Protect against cyber attack
    Objective 3

    Detect cyber security events
    Objective 4

    Minimise the impact of cyber security incidents
    Objective 5 – Develop the right cyber security skills, knowledge, and culture

    Each objective has a range of outcomes to be achieved in two stages, the first tranche by 2025, and the next by 2030. The government plans to invest £2.6 bn in cyber and legacy IT over the spending review 2021 period and will devise a number of key performance indicators to measure progress. 

    The strategy is ambitious and welcomed given the increasing threat environment the UK government is facing. In order to succeed, it will need to overcome a range of challenges that we have come across in our work on digital and cyber security. From our point of view, two of the key ones are: 

    • The public sector will need to overcome known legacy and data issues in a situation where IT assets are not always catalogued or risk assessed; and where data quality varies with expanding and interconnecting supplier systems that increase the likelihood of vulnerabilities. 
    • Cyber risk management with effective escalation and mitigation, in and across departments, will need to be established – whilst also aligning disparate central and arms-length bodies across government to focus on the right things, in the right way at the right time. 

    Our Cyber and information security: Good practice guide addresses these and a number of other challenges. It enables Audit Committees to ask the right questions of organisations to help them start aligning themselves to the new Cyber Security Strategy.  

    About the author

    Daniel Lambauer joined the NAO in 2009 as a performance measurement expert and helped to establish our local government value for money (performance audit) team. He is the Executive Director with responsibility for Strategy and Resources. As part of his portfolio, he oversees our international work at executive and Board level and has represented the NAO internationally at a range of international congresses. He is also the NAO’s Chief Information Officer and Senior Information Responsible Owner (SIRO). Before joining the NAO, Daniel worked in a range of sectors in several countries, including academia, management consultancy and the civil service.

    Comment on this post...

  • Step back and see the full picture: lessons learned in risk management   

  • Posted on March 1, 2022 by

    Confessions of a risk manager

    A few years ago, I decided to renovate my bathroom, it wasn’t a small feat and required all new electrics, plumbing, new boiler, the works. I was reliant on contracted experts to get the results I wanted. I decided to handle the project management myself, I was confident, I’m a risk manager after all! Once the project was underway little things began to go wrong, delays, disruptions and scheduling conflicts cascaded, and I found myself in the middle firefighting. I could manage some of the problems myself, but most of the uncertainty was coming from the people and expertise outside of my direct control. I’ve made a resolution this year to start the next renovation project and I know that to succeed, I will need to learn the lessons from the past.  

    Enterprise thinking

    Uncertainty is at the heart of risk management, and without a doubt we have been living in very uncertain times over the last two years. The impact of the pandemic has been felt across all sectors and has redefined the risk landscape. Here at the NAO, the increased level of uncertainty has influenced our programme of value for money and insight work. It sharpened our focus on the arrangements in place for government to identify, evaluate, and respond to risks. In our latest preparedness report: The government’s preparedness for the COVID-19 pandemic: lessons for government on risk management we found the pandemic has exposed vulnerabilities in government’s approach to managing whole-systems risks and that lessons, that would have helped prepare for a pandemic like COVID-19, were not fully implemented.

    Enterprise thinking in risk management allows us to integrate the practice of risk management across the whole system, from strategic decision making to execution and delivery. However, looking at uncertainties inside the organisation won’t give us the full picture about what is happening outside and across other organisations. We need to step outside and look out into the extended enterprise. If we think of an organisation as a castle, the extended enterprise refers to anything outside of the castle walls. To go back to my project, I was on the inside and close to the project, I wanted the project to succeed, clouded by optimism and missing the full picture. I’d forgotten to account for what might be happening outside of my “castle walls” and how uncertainty would impact what I was trying to achieve.

    I am of course not alone in having optimism bias. Being close to the detail is not a bad thing, in fact it’s often vital, but when we’re on the inside it’s much harder to cast our view out to the horizon and to see the uncertainties just out of focus. We need to see the whole system in order to anticipate, coordinate and prepare for what might happen, even if we’re really hoping it won’t.

    Connecting the dots

    By taking an enterprise approach to identify, evaluate, and respond to risks, we get a better understanding of the full picture. We can see the interdependencies and connections between the various risks facing the delivery of objectives. The NAO’s reports NHS backlogs and waiting times in England and Reducing the backlogs in criminal courts are both clear examples where identifying the complex interdependencies and taking a whole-systems approach will be needed to tackle and improve outcomes. For instance, understanding the inherent risks of harm to patients as a result of longer wait times, and the cascading impact this could have on local partnerships, community support and organisations outside of the NHS.

    Yet, applying this thinking to the extended enterprise of government will also be necessary to tackle and achieve some of the most complex risks of today and of the future. In our report Achieving net zero we concluded that the all-encompassing nature of net zero means that all government bodies, including departments, arm’s-length bodies, and executive agencies, have a role to play. This is perhaps the clearest example of the importance of whole-systems thinking and enterprise-wide risk management.

    Opportunities

    We mustn’t forget that uncertainty can generate both threats and opportunities. We’re often taught to see risks only as threats. However, those threats can also present us with opportunities to improve, providing we have the desire, agility, and resilience to respond and act. I’ve already started planning for my next project and I know that by applying the lessons learned from last time I can increase my chances of success.

    As we continue to recover from what we hope is the worst of the pandemic, it’s important to look at the full picture, identify the lessons and apply improvements where we can. Our lessons learned programme of work at the NAO has highlighted opportunities to strengthen government’s approach to risk management, to ensure that it includes a clearer view of whole system risks. Applying this learning will require collaboration not only within and across government but also across sectors and the entire extended enterprise. The challenge questions is: who is providing the enterprise view of risks across the whole of government and what other lessons are there to be learned?

    You can read more about our findings and insights on our website.  Links to the specific reports and topics explored in this blog are set out below:

    Please feel free to comment and share your thoughts, your views are very welcome.  

    About the author

    Russell Heppleston

    Russell Heppleston

    Russell Heppleston is a Risk Manager for the Financial and Risk Management hub at the NAO. He joined the NAO in 2021 as an experienced risk manager with over 15 years experience working in Local Government, specialising in internal assurance, risk and governance. He is a Chartered Internal Audit Leader (QIAL) and Certified Risk Manager (CMIRM).

    1 Comment

  • What makes a super model? Using innovative approaches to audit departments’ models

  • Posted on February 14, 2022 by

    Setting targets for carbon emissions is a crucial part of government’s plans to tackle climate change. To do this government uses its UK TIMES model, a model of the whole UK energy system, to provide important evidence supporting decisions like the net zero target. That’s just one example of the hundreds of models that government relies on.  

    Models are used for activities like estimating costs, distributing funding within organisations, and testing policy options – and they underpin decisions that affect people’s lives. In recent years departments have used models to plan NHS test and trace services, set allocations for teacher training places, and estimate the cost of the financial settlement when leaving the EU. So it’s really important that people who depend on outputs from models can feel confident in the quality and robustness of these models.  

    How the NAO uses models 

    At the NAO, part of our financial audit work involves scrutinising the models that underpin significant estimates in departments’ accounts. Our expert Modelling Team looks for innovative ways to do this, and to support departments in improving the way they produce and use models.  

    Building an independent copy or reproduction of a model is one of the most comprehensive ways of quality assuring a model.  We applied this ‘gold standard’ approach to one of the most technically complex and inherently uncertain models that we audit – HMRC’s Oil and Gas Decommissioning model. This is a micro simulation model for oil and gas activity in the North Sea, which generates an estimate of the total provision of revenue from the Petroleum Revenue Tax and Ring Fence Corporation Tax.  

    The complexity of micro simulation models makes traditional approaches to auditing models challenging and amplifies potential errors that can be easy to miss. To help us audit the estimate, we built an independent reproduction of the model in the R software language. Running the reproduction separately allows us to produce an independent estimate and helps us to identify and investigate any discrepancies to the original model. This has enhanced confidence in the outputs for key stakeholders.  

    How we managed uncertainty 

    Modelled outputs are inherently uncertain. As well as checking that the central estimate is reasonable, we also wanted to understand the full range of plausible outcomes. We built in fully automated uncertainty analysis to our reproduction, which lets us stress test the estimate under extreme scenarios. It also lets us test what happens to the estimate when several inputs change at the same time, by running thousands of simulations to generate a likely range of outcomes.  This is something not carried out in many of the models we audit and is an area where our independent model assurance can provide additional value. It gives us confidence that the estimate will not be materially wrong, even when economic shocks are considered.  

    This fully working model reproduction has transformed the way we audit the estimate and is a great example of what is possible in terms of model quality assurance.  It’s enhanced the quality of our work: quality assurance checks are automated, including more advanced sensitivity analysis. And it’s helped us to be more efficient: the quality assurance checks in the reproduction are quicker to produce, freeing up our analysts to focus on creating greater insights. 

    What next? 

    We think there are opportunities to replicate this approach across the portfolio of models that we audit and help enhance our quality assurance work. We want our audit work to help build confidence in the quality of government’s models and support government in making plans that don’t place value for money at risk. 

    Our recently published report on Financial Modelling in government looks at how government produces and uses models and identifies the systemic issues that can lead to value for money risks. 

    To find out more about the way we audit models, see our Framework to Review Models is framework is aimed at people commissioning, carrying out or assuring analysis. It provides a structured approach to review models, which organisations can use to determine whether the modelling outputs they produce are reasonable and robust.  

    How do you think this framework could help you or your organisation? Tell us in the comment section below.  

    About the author

    Ruth Kelly

    Ruth Kelly

    Ruth Kelly is our Chief Analyst and has wide experience of applying economics and other analytical approaches to support policy evaluation, investment decisions and risk management. Prior to joining the NAO, she held business evaluation and risk management roles for a global resources company, and advised clients on carbon and energy issues for a Big 4 economic consultancy practice.

    2 Comments

  • Cyber security: has the pandemic changed anything?

  • Posted on January 14, 2022 by

    The start of a new year brings the opportunity to look back and reflect on the challenges we faced in dealing with COVID-19 during the last year. One of the many impacts of the pandemic we did not foresee was moving many aspects of our social and economic life online to try and keep them going through lockdowns. This came with considerable advantages, keeping many businesses, social networks and relationships going. But it also came with a significant downside, as we all became more vulnerable to the risks associated with operating online. In addition to the major attacks like WannaCry and SolarWinds, which have affected organisations in the UK and overseas, it is now increasingly likely that each of us has either personally suffered from some kind of online crime or know someone else who has.

    In its latest Annual Report, government’s National Cyber Security Centre (NCSC) is clear about the nature of the risks we have faced during the pandemic, noting the startling finding that “From household goods to vaccine appointments, there have been few avenues criminals have not tried to exploit”. And the move to living more of our lives online has resulted in some shifts in criminal activity.

    The major trend identified by the NCSC is the growth in criminal groups using ransomware to extort organisations of all kinds. The NCSC describes ransomware as the most immediate cyber security threat to UK businesses: this obviously makes it a threat to the resilience and performance of the economy. But it is also a risk to both central and local government and the wide range of services which they support. So, whether we are taxpayers or service users, we should be concerned at this increased use of ransomware being added to the existing list of cyber threats.

    Unfortunately, the other threats on that list haven’t gone away. The March 2021 Microsoft Exchange Servers incident, in which a sophisticated attacker used zero-day vulnerabilities to compromise at least 30,000 separate organisations, highlighted the dangers posed by supply chain attacks. And there are plenty of examples in the news of other incidents, both malicious and accidental, which have put data, operations and organisational resilience at risk in both private and public sectors.

    In its new National Cyber Strategy, government has set out some of the things it wants to do to make the UK more resilient to cyber-attack. Like its predecessors, the Strategy is painted on a broad canvas, setting out high-level objectives: it says that the UK should strengthen its grasp of technologies that are critical to cyber security and that it should limit its reliance on individual suppliers or technologies which are developed under regimes that do not share its values. These objectives are aimed at the structural factors behind cyber security. And in the meantime, government is developing its Active Cyber Defence programme – which seeks to reduce the risk of high-volume cyber-attacks ever reaching UK citizens – and pressing ahead with other work on skills, resilience and partnerships across different industries and sectors.

    So, it seems clear that, despite the efforts of public and private sectors, the pandemic has exacerbated some of the threats we face online. But one thing that most experts agree on is that our best defence is getting the basics right. Many of the attacks which we have seen during the pandemic could have been avoided if individuals and organisations had followed recognised good practice. This includes actions like implementing formal information security regimes, avoiding unsupported software and adopting good password practices. We have specific guidance to help Audit Committees think about these sorts of issues in our updated Cyber and Information Security Good Practice Guide.

    So, if you are still thinking about your New Year’s resolutions, how about refreshing your cyber security practices? That may help you avoid becoming the next victim of a cyber-attack.

    Tom McDonaldAbout the author

    Tom McDonald is the Director responsible for the NAO’s work on cyber security. Tom has worked at the NAO since 2001 and has focused his career on the defence, overseas, health and national security sectors. He has degrees in modern languages, international relations and management from Bristol University and Ashridge Business School.

    Comment on this post...

  • Delivering programmes at speed? What you need to consider

  • Posted on January 7, 2022 by

    Ambulances need to travel fast! Ambulance drivers must take risks that regular drivers do not. This includes running red lights and travelling at high speeds through busy roads. However, to avoid accidents, precautions are taken to manage risks. The driver is trained, there are flashing blue lights and loud sirens.  

    Delivering programmes at speed requires a similar assessment of risks. In our recent lessons learned report we show that some programmes have successfully delivered quickly but not all – just as not all vehicles can be driven like ambulances. Speed creates greater risks which will not be appropriate or sustainable for every programme or organisation.  

    Should the risks of speed be taken? 

    Programmes may need to be delivered at speed for various reasons, including in an emergency or where there is a fixed deadline. We recently reported on the Kickstart Scheme launched by the Department for Work and Pensions (DWP). In response to a significant forecast rise in youth unemployment given the COVID-19 pandemic, DWP wanted to set up support quickly. It launched Kickstart on 2 September 2020, after only around six weeks of work, in time for the expected end of furlough in October 2020. We have also seen programmes delivered at speed as government simply wants to achieve outcomes sooner. A clear rationale for speed, can make it easier to get wider support and justify taking risks. Other drivers understand an ambulance’s need for speed and often make way. 

    Decision-makers need to understand ‘why speed’ to assess if the risks of speed are necessary and justifiable. Risks can include cost increases, not achieving outcomes, or people being diverted into a programme at the expense of other work. Our recent report on bounce back loans highlights the impact when risks are not managed – the Scheme facilitated faster lending by removing credit and affordability checks and allowing businesses to self-certify their application documents. Prioritising speed contributed to high levels of estimated fraud. 

    Given the risks decision-makers need to ask:
    • Can I justify taking the risks?
    • Have I thought about things enough?
    • Is the end result worth it?

    Monitoring and managing risks in practice 

    Where decision-makers choose to take the risks of delivering a programme quickly, they must proactively monitor and manage these increased and different risks. In November 2021, we shared insights from our lessons learned report with the Ministry of Justice team responsible for the Probation Reform Programme and the creation of the unified Probation Service to understand how this resonated with their practical experience. In June 2021, the Lord Chancellor had written to Parliament confirming probation services had been unified. 

    The team told us that they consciously chose to deliver at speed and identified a clear narrative for the reforms being at pace. As such, everyone was clear on the reasons for the reforms. The team also made clear that there was zero contingency beyond the expected delivery date. Alongside setting a minimum expectation of the requirements needed for Day 1, this helped force the pace and prioritisation of effort. 

    The programme team also highlighted the importance of strong leadership, with a culture of accountability and responsibility, to deal with any uncertainties or issues. In particular, they spoke of a culture which encouraged people to raise any problems they’d encountered, rather than hide them or focus on the ‘good news’. 

    Additionally, the programme team said they had built a strong internal assurance team, comprised of former senior operations staff, to carry out site visits and desktop reviews to ensure the programme was on track. 

    Alongside this, the programme team outlined the advantages of a flexible programme structure. The team recognised that it was difficult to plan everything up front, and instead ensured they had the required processes and information needed to respond quickly. This was done through regional teams, with a dedicated senior manager, tasked with identifying risks as soon as possible. This meant that the central programme team could deal with ‘unknown unknowns’ effectively when they arose.  

    Our insights 

    Many of the points raised by the Probation Reform Programme team align with our insights. In particular:  

    • Including speed as a specific programme objective to provide a clear framework for decision-making and help make trade-offs between speed, cost and outcomes. 
    • Building teams with the right leadership, skills and experience to make clear, timely and reliable decisions. 
    • Tailoring processes to add value and momentum to programme decision-making. 
    • Recognising the uncertainties of delivering at speed and managing these. 

    As speed remains important for ambulances, so it will for some programmes, particularly with commitments to achieving ‘net zero’ greenhouse gas emissions by the fixed deadline of 2050. Our report helps those deciding whether to deliver at speed ask questions to determine when or how this should be done and then continually test whether a programme will achieve its outcomes.  

    Further reading 


    About the authors

    Josh Perks is a qualified accountant with experience of working on the NAO’s transport team. His work has included audits of the main government transport bodies and value-for-money studies of major rail programmes. Recently, he has taken an active role in the NAO’s Major Projects Delivery Hub.

    Jemma Dunne is an Audit Manager and has delivered value for money reports across areas such as health and defence, including those on government programmes. She is a qualified chartered accountant (FCA) and holds the APM Project Management Qualification (PMQ).

    Comment on this post...

  • Effective governance and accountability is a mainstay for successful contract management

  • Posted on December 15, 2021 by

    This is the last in a series of posts on our good practice guidance for managing the commercial lifecycle. In that guide we shared fresh insights from our extensive body of work on government’s commercial activities.  

    It’s not surprising that many of the examples in our guidance related directly to responding to the COVID-19 pandemic. Just as everyday purchases we normally take for granted like buying pasta from the local supermarket, became complex thanks to the effects of the pandemic, so did government’s processes for buying the things it needed.  

    Of the supporting elements in our guidance which apply across the whole contract letting spectrum, governance and accountability may be the most crucial for responding to big changes and extreme circumstances. Risks are more likely to be borne out at any stage without the right people to make appropriate decisions and provide adequate scrutiny. 

    Explaining governance and accountability 

    When considering governance, we look at the oversight arrangements at an organisational level – poring over what risk reviews have taken place, and the adoption of learning from previous lessons.  When we consider ‘accountability’, we reference the support to contract managers; others charged with responsibility for the governance of contracts and the organisation as a whole (including the Accounting Officer for whom the duties for managing public money and reporting to Parliament falls to).  

    To be effective, these arrangements should facilitate open discussion and continuous improvement. Public bodies should demonstrate robust, independent oversight of both their contractual arrangements and overall commercial portfolios. We have often found that organisations do oversight well at a ‘big picture’ level or in fine detail but often cannot achieve both. 

    Accountability in complex systems 

    In 2020, we reported on the value for money of the local bus service system overseen by the Department for Transport. The report looked at the effectiveness of the government’s support for local bus services and whether the tools to improve local bus services were in place. The Department is not accountable for delivering bus services, but it has a national policy responsibility. We found that, during the COVID-19 pandemic, the Department came together with local authorities and operators, intervening rapidly to target the weakest areas and keep buses running – a good learning point for future programmes. 

    Using governance to support speed 

    Effective governance is important at all times but crucially so when risks are higher and goods and services are required to be delivered at greater speed. The NAO’s work on Lessons Learned: Delivering Programmes at Speed highlights the contribution governance can make in such situations. This publication references the need for governance structures to be tailored to support the pace of the work. We have seen different structures work in different circumstances, but the main principles are to have clear accountability lines and to involve the right people at the right time. 

    An example of the importance of clear accountability comes from the NHS Test and Trace Service. This was created to lead the government’s COVID-19 test and trace programme, which was delivered in part by contracted providers. The NHS Test and Trace Service was part of the Department of Health & Social Care (the Department) and was subject to the Department’s financial, information and staffing controls. However, the executive chair of the NHS Test and Trace Service did not initially report to the Department’s ministers or Permanent Secretary, but to the Prime Minister and the Cabinet Secretary. This unusual organisational relationship created dual reporting lines, which brought risks of unclear accountability. The relationship subsequently changed so that the executive chair reported to the Secretary of State for Health. 

    What good looks like 

    In defining how to achieve good governance and accountability to support successful contracts, our good practice guide sets out areas of improvement and outlines our expectations of best practice. They include:  

    • Accountability is defined and responsible officers are appropriately empowered. 
    • There is effective independent scrutiny of commercial activity. 
    • Lessons learned promptly feed into the wider strategy and plans for other contracts, and are integrated into needs assessment and the list of strategic suppliers. 
    • Reliable, timely management information is used for rapid diagnosis of issues and prompt action. 
    • The organisation’s own performance against obligations and the supplier’s view of performance are known and considered. 
    • Industry expertise is valued and used, subject to robust independence controls. 

    Wishing you all successful governance and accountability arrangements for your contracting activities!  

    About the author

    Iain Forrester

    Iain Forrester is a qualified accountant with long experience of working on the NAO’s commercial and contracting related work. This has included cross-government work on grants, shared services, EU Exit, and the government’s response to COVID-19. He also worked on the commercial and contract management insights guide published in 2016.


    Comment on this post...

  • Commercial strategy needs to be joined up to achieve best value

  • Posted on November 25, 2021 by

    Last December, the government published a Green Paper on Transforming Public Procurement. It  stressed that investments should be subject to consideration of the public good, including supporting national priorities. It discussed leveraging commercial activity to achieve social and environmental value.

    For our good practice guidance for managing the commercial lifecycle, we examined similar opportunities and how to support them. We shared fresh insights and learning from our extensive body of work on government’s commercial activities. In this latest post, I will share some of our insights on commercial strategy.

    Commercial strategy means thinking about the overall approach to ensure that procurement and other commercial activities provide the outcomes that government wants and benefit us all. This is the part of our guidance which really focuses on the context around what government does when it runs a competition or revises a contract, and the importance of joining up different elements.

    Joining up commercial strategy is vital if government wants to achieve its wider aims as well as value for money. That includes establishing a consistent approach to risk management, and the organisational capacity and capability to respond to uncertainty. It is also where other considerations come in, including where procurement can be a lever for larger goals like encouraging innovation or diversifying the landscape of suppliers to government. Commercial strategies should demonstrate how each commercial agreement aligns with wider strategic objectives and how this is then reflected in the approach for managing commercial risks and incentives throughout the commercial lifecycle.

    A couple of our past reports give good examples of the importance of joining up strategy at the programme level and more widely.

    Aligning timetables

    In 2018 we reported on The Ministry of Defence’s arrangement with Annington Property Limited, a sale and leaseback arrangement for accommodation. As part of our review, we found that the timetable for developing the Ministry of Defence’s wider estate strategies was not aligned with the timetable for rent reviews. The department was developing a ‘Future Accommodation Model’ intended to provide personnel with more flexible accommodation options. However, the timing meant that its negotiations on the sites with Annington would begin before a decision was taken on the wider model. This affected its ability to develop negotiating strategies for these sites. We recommended that the department align the timetables to use realistic scenarios in its negotiations, giving it a clearer strategic view.

    Leveraging procurement across government 

    On a wider scale, we reported in 2016 on the government’s spending with small and medium-sized enterprises (SMEs). The government recognised that SMEs could offer many benefits to the public sector, including flexibility, innovation and better value for money due to lower overhead costs, as well as increasing local investment and improving social outcomes. We recommended that the government should take a more focused approach, identifying where SMEs could bring the most benefit, and look into an integrated cross-government procurement platform to support its commercial strategy. The government has since introduced guidance for SMEs applying for contracts and promised to invest in joining up the different procurement systems. The intention is that this will help drive the commercial benefits from better data sharing – as part of changes to procurement processes following exit from the European Union.

    That is an example of the kind of strategic approach to identifying risks and opportunities which we want to see applied consistently across organisations, and we look forward to seeing how government’s follow-up to Transforming Public Procurement would help to further encourage this.

    What good looks like

    Our good practice guide sets out areas of improvement and outlines our expectations of best practice, with specific case study examples that demonstrate some of these expectations of a joined-up commercial strategy. They include:

    • Commercial, policy, operational and business teams work together to develop a clear understanding of the contracts and produce required outcomes
    • Each contract staffing model is developed early, regularly reviewed and tailored to different contract stages
    • Capability plans include operational resilience to address unplanned demands
    • Knowledge and experience of underlying contract issues is retained throughout the lifecycle of a commercial relationship
    • There is investment in the organisation and its people to ensure adequate access to training and development to support commercial awareness and expertise.

    About the author

    Iain Forrester

    Iain Forrester is a qualified accountant with long experience of working on the NAO’s commercial and contracting related work. This has included cross-government work on grants, shared services, EU Exit, and the government’s response to COVID-19. He also worked on the commercial and contract management insights guide published in 2016.


    1 Comment

Right column

  • About the NAO blog

    Our experts share their views about issues and common challenges facing government, what public sector leaders should look out for and how organisations have addressed issues. Our posts draw together threads from across our reports, share secrets spilled in events and reveal our experts’ expectations for the future.

    We encourage comments that support the exchange of ideas for improvement, but ask that those posting are respectful of others.

  • Sign up for automatic feeds

    Sign up to receive email alerts:




    RSS IconSubscribe in an RSS Reader