Left column

Middlecolumn

Latest posts

    Mapping the Justice landscape 

  • Posted on May 3, 2022 by

    James Gjertsen explains how the NAO uses spatial analytics to bring audit teams closer to the data and visualise the complex geographic relationships at work behind the scenes in the justice system 

    From public transport to healthcare, we use services and infrastructure provided by central and local government all the time. Multiple Government departments have to coordinate long term decisions about provision of services and land use – and these all come together in a place. Bringing a place-based focus to decisions is key to maximising the positive impact government can have on our lives. This is why the NAO has an increasing interest in the power of spatial analysis to provide insight.   

    Even within a single policy area, such as health or justice, one department or ministry can be responsible for managing multiple different types of organisations that need to work together to deliver a coordinated service for users. One area we recently looked at was the criminal justice system.  

    This involves a range of organisations, from police forces to courts and probation. Mapping the spatial relationships between these organisations helps us answer deeper questions about the co-ordination and delivery of those services. We can ask more insightful questions about where backlogs are occurring or how the user experiences their linked-up journey through the system, and spot good practice. But all this starts with how these organisations relate to one another spatially. 

    To explore this, we developed an interactive tool visualising how all these various organisations interact with each other. You can use it to see at a glance where those interactions are simple and where they may be more complicated, along with summaries for each body.  

    This quickly shows that the number of organisations interacting with each other can add up fast: for police forces the Metropolitan Police tops the list for the most interactions with 63 local interacting organisations! This includes one prison region, HM Courts and Tribunals Service cluster and youth Justice Board, two Crown Prosecution regions, nine local justice areas, eighteen Probation Delivery Units and thirty-one Youth offending teams.  

    The tool can help raise questions about the efficiency of the criminal justice system by providing geographic context around the interactions between the various bodies.  

    Under the hood 

    The tool uses local authority boundaries as the building blocks for each organisation’s map layer. As an example: For probation delivery units, Cardiff, Newport and the Vale of Glamorgan are listed under the same unit, “Cardiff and the Vale”. The tool joins these local authorities’ boundaries together to produce a boundary that represents the probation delivery unit.  

    Converting local authority boundaries to probation delivery units in Wales 

    There were benefits to using this approach. Firstly, a few of the organisations didn’t have a set of predetermined boundaries to provide us, so using this approach let us construct them from scratch.  

    Secondly, using the same building blocks for each layer makes comparing them much easier. Boundaries will often be simplified to shrink the size of files being dealt with. Imagine trying to record the borders of a town. If you took measurements every 500 meters you would need far fewer measurements than if you took them every ten centimetres, and for some purposes the result would be just as useful.  

    However, it can be hard to compare boundaries that may have been recorded differently. Using the same base boundaries to build all the layers simplifies things. They will all be recorded in the same way and any differences that get flagged will be genuine, no extra work needed. 

    We’ve been building our capacity at the NAO to use data and spatial analytics to derive deeper insights around the topics we audit. This tool is just one of the ways we are supporting our current and future work by bringing teams closer to the data, by clearly articulating the complex geographic relationships at work behind the scenes. Why not see for yourself? 

    About the author

    James Gjertsen

    James Gjertsen is a Senior Analyst in the Analysis Hub at the NAO. He joined the NAO in 2018 and leads the Analytical Insights Team which aims to support the NAO’s VFM work programme by  drawing deeper insights from the data it collects.

    1 Comment

  • Taking on the challenge of public sector cyber security

  • Posted on March 8, 2022 by

    The government recently published its new Cyber Security Strategy specifically aimed at building a cyber resilient public sector. Resilience is key in underpinning its vision to make the UK a cyber power in a world increasingly shaped by technologies that offer many benefits but also pose risks. The strategy reiterates that government remains an attractive target for a broad range of malicious actors with 40% of incidents 2020-21 affecting the public sector. 

    The main benefits highlighted are the need to protect key UK assets and the uninterrupted continuation of vital services. The strategy also aims to enable the development of skills and capability in cyber awareness and risk management.   

    This has been a major theme in our work, we recently published our good practice guide, aimed at Audit Committees, on cyber and information security where we set out the type of risk and capability management in relation to cyber security we would expect to see in organisations. 

    In order to harden government to cyber-attack and build the required resilience in the public sector by 2030, the Cyber Security strategy has two main pillars and five objectives:  

    Pillar 1Build organisational cyber resilience Pillar 2 – ‘Defend as one’
    Objective 1

    Manage cyber security risk
    Objective 2

    Protect against cyber attack
    Objective 3

    Detect cyber security events
    Objective 4

    Minimise the impact of cyber security incidents
    Objective 5 – Develop the right cyber security skills, knowledge, and culture

    Each objective has a range of outcomes to be achieved in two stages, the first tranche by 2025, and the next by 2030. The government plans to invest £2.6 bn in cyber and legacy IT over the spending review 2021 period and will devise a number of key performance indicators to measure progress. 

    The strategy is ambitious and welcomed given the increasing threat environment the UK government is facing. In order to succeed, it will need to overcome a range of challenges that we have come across in our work on digital and cyber security. From our point of view, two of the key ones are: 

    • The public sector will need to overcome known legacy and data issues in a situation where IT assets are not always catalogued or risk assessed; and where data quality varies with expanding and interconnecting supplier systems that increase the likelihood of vulnerabilities. 
    • Cyber risk management with effective escalation and mitigation, in and across departments, will need to be established – whilst also aligning disparate central and arms-length bodies across government to focus on the right things, in the right way at the right time. 

    Our Cyber and information security: Good practice guide addresses these and a number of other challenges. It enables Audit Committees to ask the right questions of organisations to help them start aligning themselves to the new Cyber Security Strategy.  

    About the author

    Daniel Lambauer joined the NAO in 2009 as a performance measurement expert and helped to establish our local government value for money (performance audit) team. He is the Executive Director with responsibility for Strategy and Resources. As part of his portfolio, he oversees our international work at executive and Board level and has represented the NAO internationally at a range of international congresses. He is also the NAO’s Chief Information Officer and Senior Information Responsible Owner (SIRO). Before joining the NAO, Daniel worked in a range of sectors in several countries, including academia, management consultancy and the civil service.

    Comment on this post...

  • Step back and see the full picture: lessons learned in risk management   

  • Posted on March 1, 2022 by

    Confessions of a risk manager

    A few years ago, I decided to renovate my bathroom, it wasn’t a small feat and required all new electrics, plumbing, new boiler, the works. I was reliant on contracted experts to get the results I wanted. I decided to handle the project management myself, I was confident, I’m a risk manager after all! Once the project was underway little things began to go wrong, delays, disruptions and scheduling conflicts cascaded, and I found myself in the middle firefighting. I could manage some of the problems myself, but most of the uncertainty was coming from the people and expertise outside of my direct control. I’ve made a resolution this year to start the next renovation project and I know that to succeed, I will need to learn the lessons from the past.  

    Enterprise thinking

    Uncertainty is at the heart of risk management, and without a doubt we have been living in very uncertain times over the last two years. The impact of the pandemic has been felt across all sectors and has redefined the risk landscape. Here at the NAO, the increased level of uncertainty has influenced our programme of value for money and insight work. It sharpened our focus on the arrangements in place for government to identify, evaluate, and respond to risks. In our latest preparedness report: The government’s preparedness for the COVID-19 pandemic: lessons for government on risk management we found the pandemic has exposed vulnerabilities in government’s approach to managing whole-systems risks and that lessons, that would have helped prepare for a pandemic like COVID-19, were not fully implemented.

    Enterprise thinking in risk management allows us to integrate the practice of risk management across the whole system, from strategic decision making to execution and delivery. However, looking at uncertainties inside the organisation won’t give us the full picture about what is happening outside and across other organisations. We need to step outside and look out into the extended enterprise. If we think of an organisation as a castle, the extended enterprise refers to anything outside of the castle walls. To go back to my project, I was on the inside and close to the project, I wanted the project to succeed, clouded by optimism and missing the full picture. I’d forgotten to account for what might be happening outside of my “castle walls” and how uncertainty would impact what I was trying to achieve.

    I am of course not alone in having optimism bias. Being close to the detail is not a bad thing, in fact it’s often vital, but when we’re on the inside it’s much harder to cast our view out to the horizon and to see the uncertainties just out of focus. We need to see the whole system in order to anticipate, coordinate and prepare for what might happen, even if we’re really hoping it won’t.

    Connecting the dots

    By taking an enterprise approach to identify, evaluate, and respond to risks, we get a better understanding of the full picture. We can see the interdependencies and connections between the various risks facing the delivery of objectives. The NAO’s reports NHS backlogs and waiting times in England and Reducing the backlogs in criminal courts are both clear examples where identifying the complex interdependencies and taking a whole-systems approach will be needed to tackle and improve outcomes. For instance, understanding the inherent risks of harm to patients as a result of longer wait times, and the cascading impact this could have on local partnerships, community support and organisations outside of the NHS.

    Yet, applying this thinking to the extended enterprise of government will also be necessary to tackle and achieve some of the most complex risks of today and of the future. In our report Achieving net zero we concluded that the all-encompassing nature of net zero means that all government bodies, including departments, arm’s-length bodies, and executive agencies, have a role to play. This is perhaps the clearest example of the importance of whole-systems thinking and enterprise-wide risk management.

    Opportunities

    We mustn’t forget that uncertainty can generate both threats and opportunities. We’re often taught to see risks only as threats. However, those threats can also present us with opportunities to improve, providing we have the desire, agility, and resilience to respond and act. I’ve already started planning for my next project and I know that by applying the lessons learned from last time I can increase my chances of success.

    As we continue to recover from what we hope is the worst of the pandemic, it’s important to look at the full picture, identify the lessons and apply improvements where we can. Our lessons learned programme of work at the NAO has highlighted opportunities to strengthen government’s approach to risk management, to ensure that it includes a clearer view of whole system risks. Applying this learning will require collaboration not only within and across government but also across sectors and the entire extended enterprise. The challenge questions is: who is providing the enterprise view of risks across the whole of government and what other lessons are there to be learned?

    You can read more about our findings and insights on our website.  Links to the specific reports and topics explored in this blog are set out below:

    Please feel free to comment and share your thoughts, your views are very welcome.  

    About the author

    Russell Heppleston

    Russell Heppleston

    Russell Heppleston is a Risk Manager for the Financial and Risk Management hub at the NAO. He joined the NAO in 2021 as an experienced risk manager with over 15 years experience working in Local Government, specialising in internal assurance, risk and governance. He is a Chartered Internal Audit Leader (QIAL) and Certified Risk Manager (CMIRM).

    1 Comment

  • What makes a super model? Using innovative approaches to audit departments’ models

  • Posted on February 14, 2022 by

    Setting targets for carbon emissions is a crucial part of government’s plans to tackle climate change. To do this government uses its UK TIMES model, a model of the whole UK energy system, to provide important evidence supporting decisions like the net zero target. That’s just one example of the hundreds of models that government relies on.  

    Models are used for activities like estimating costs, distributing funding within organisations, and testing policy options – and they underpin decisions that affect people’s lives. In recent years departments have used models to plan NHS test and trace services, set allocations for teacher training places, and estimate the cost of the financial settlement when leaving the EU. So it’s really important that people who depend on outputs from models can feel confident in the quality and robustness of these models.  

    How the NAO uses models 

    At the NAO, part of our financial audit work involves scrutinising the models that underpin significant estimates in departments’ accounts. Our expert Modelling Team looks for innovative ways to do this, and to support departments in improving the way they produce and use models.  

    Building an independent copy or reproduction of a model is one of the most comprehensive ways of quality assuring a model.  We applied this ‘gold standard’ approach to one of the most technically complex and inherently uncertain models that we audit – HMRC’s Oil and Gas Decommissioning model. This is a micro simulation model for oil and gas activity in the North Sea, which generates an estimate of the total provision of revenue from the Petroleum Revenue Tax and Ring Fence Corporation Tax.  

    The complexity of micro simulation models makes traditional approaches to auditing models challenging and amplifies potential errors that can be easy to miss. To help us audit the estimate, we built an independent reproduction of the model in the R software language. Running the reproduction separately allows us to produce an independent estimate and helps us to identify and investigate any discrepancies to the original model. This has enhanced confidence in the outputs for key stakeholders.  

    How we managed uncertainty 

    Modelled outputs are inherently uncertain. As well as checking that the central estimate is reasonable, we also wanted to understand the full range of plausible outcomes. We built in fully automated uncertainty analysis to our reproduction, which lets us stress test the estimate under extreme scenarios. It also lets us test what happens to the estimate when several inputs change at the same time, by running thousands of simulations to generate a likely range of outcomes.  This is something not carried out in many of the models we audit and is an area where our independent model assurance can provide additional value. It gives us confidence that the estimate will not be materially wrong, even when economic shocks are considered.  

    This fully working model reproduction has transformed the way we audit the estimate and is a great example of what is possible in terms of model quality assurance.  It’s enhanced the quality of our work: quality assurance checks are automated, including more advanced sensitivity analysis. And it’s helped us to be more efficient: the quality assurance checks in the reproduction are quicker to produce, freeing up our analysts to focus on creating greater insights. 

    What next? 

    We think there are opportunities to replicate this approach across the portfolio of models that we audit and help enhance our quality assurance work. We want our audit work to help build confidence in the quality of government’s models and support government in making plans that don’t place value for money at risk. 

    Our recently published report on Financial Modelling in government looks at how government produces and uses models and identifies the systemic issues that can lead to value for money risks. 

    To find out more about the way we audit models, see our Framework to Review Models is framework is aimed at people commissioning, carrying out or assuring analysis. It provides a structured approach to review models, which organisations can use to determine whether the modelling outputs they produce are reasonable and robust.  

    How do you think this framework could help you or your organisation? Tell us in the comment section below.  

    About the author

    Ruth Kelly

    Ruth Kelly

    Ruth Kelly is our Chief Analyst and has wide experience of applying economics and other analytical approaches to support policy evaluation, investment decisions and risk management. Prior to joining the NAO, she held business evaluation and risk management roles for a global resources company, and advised clients on carbon and energy issues for a Big 4 economic consultancy practice.

    2 Comments

  • Cyber security: has the pandemic changed anything?

  • Posted on January 14, 2022 by

    The start of a new year brings the opportunity to look back and reflect on the challenges we faced in dealing with COVID-19 during the last year. One of the many impacts of the pandemic we did not foresee was moving many aspects of our social and economic life online to try and keep them going through lockdowns. This came with considerable advantages, keeping many businesses, social networks and relationships going. But it also came with a significant downside, as we all became more vulnerable to the risks associated with operating online. In addition to the major attacks like WannaCry and SolarWinds, which have affected organisations in the UK and overseas, it is now increasingly likely that each of us has either personally suffered from some kind of online crime or know someone else who has.

    In its latest Annual Report, government’s National Cyber Security Centre (NCSC) is clear about the nature of the risks we have faced during the pandemic, noting the startling finding that “From household goods to vaccine appointments, there have been few avenues criminals have not tried to exploit”. And the move to living more of our lives online has resulted in some shifts in criminal activity.

    The major trend identified by the NCSC is the growth in criminal groups using ransomware to extort organisations of all kinds. The NCSC describes ransomware as the most immediate cyber security threat to UK businesses: this obviously makes it a threat to the resilience and performance of the economy. But it is also a risk to both central and local government and the wide range of services which they support. So, whether we are taxpayers or service users, we should be concerned at this increased use of ransomware being added to the existing list of cyber threats.

    Unfortunately, the other threats on that list haven’t gone away. The March 2021 Microsoft Exchange Servers incident, in which a sophisticated attacker used zero-day vulnerabilities to compromise at least 30,000 separate organisations, highlighted the dangers posed by supply chain attacks. And there are plenty of examples in the news of other incidents, both malicious and accidental, which have put data, operations and organisational resilience at risk in both private and public sectors.

    In its new National Cyber Strategy, government has set out some of the things it wants to do to make the UK more resilient to cyber-attack. Like its predecessors, the Strategy is painted on a broad canvas, setting out high-level objectives: it says that the UK should strengthen its grasp of technologies that are critical to cyber security and that it should limit its reliance on individual suppliers or technologies which are developed under regimes that do not share its values. These objectives are aimed at the structural factors behind cyber security. And in the meantime, government is developing its Active Cyber Defence programme – which seeks to reduce the risk of high-volume cyber-attacks ever reaching UK citizens – and pressing ahead with other work on skills, resilience and partnerships across different industries and sectors.

    So, it seems clear that, despite the efforts of public and private sectors, the pandemic has exacerbated some of the threats we face online. But one thing that most experts agree on is that our best defence is getting the basics right. Many of the attacks which we have seen during the pandemic could have been avoided if individuals and organisations had followed recognised good practice. This includes actions like implementing formal information security regimes, avoiding unsupported software and adopting good password practices. We have specific guidance to help Audit Committees think about these sorts of issues in our updated Cyber and Information Security Good Practice Guide.

    So, if you are still thinking about your New Year’s resolutions, how about refreshing your cyber security practices? That may help you avoid becoming the next victim of a cyber-attack.

    Tom McDonaldAbout the author

    Tom McDonald is the Director responsible for the NAO’s work on cyber security. Tom has worked at the NAO since 2001 and has focused his career on the defence, overseas, health and national security sectors. He has degrees in modern languages, international relations and management from Bristol University and Ashridge Business School.

    Comment on this post...

  • Delivering programmes at speed? What you need to consider

  • Posted on January 7, 2022 by

    Ambulances need to travel fast! Ambulance drivers must take risks that regular drivers do not. This includes running red lights and travelling at high speeds through busy roads. However, to avoid accidents, precautions are taken to manage risks. The driver is trained, there are flashing blue lights and loud sirens.  

    Delivering programmes at speed requires a similar assessment of risks. In our recent lessons learned report we show that some programmes have successfully delivered quickly but not all – just as not all vehicles can be driven like ambulances. Speed creates greater risks which will not be appropriate or sustainable for every programme or organisation.  

    Should the risks of speed be taken? 

    Programmes may need to be delivered at speed for various reasons, including in an emergency or where there is a fixed deadline. We recently reported on the Kickstart Scheme launched by the Department for Work and Pensions (DWP). In response to a significant forecast rise in youth unemployment given the COVID-19 pandemic, DWP wanted to set up support quickly. It launched Kickstart on 2 September 2020, after only around six weeks of work, in time for the expected end of furlough in October 2020. We have also seen programmes delivered at speed as government simply wants to achieve outcomes sooner. A clear rationale for speed, can make it easier to get wider support and justify taking risks. Other drivers understand an ambulance’s need for speed and often make way. 

    Decision-makers need to understand ‘why speed’ to assess if the risks of speed are necessary and justifiable. Risks can include cost increases, not achieving outcomes, or people being diverted into a programme at the expense of other work. Our recent report on bounce back loans highlights the impact when risks are not managed – the Scheme facilitated faster lending by removing credit and affordability checks and allowing businesses to self-certify their application documents. Prioritising speed contributed to high levels of estimated fraud. 

    Given the risks decision-makers need to ask:
    • Can I justify taking the risks?
    • Have I thought about things enough?
    • Is the end result worth it?

    Monitoring and managing risks in practice 

    Where decision-makers choose to take the risks of delivering a programme quickly, they must proactively monitor and manage these increased and different risks. In November 2021, we shared insights from our lessons learned report with the Ministry of Justice team responsible for the Probation Reform Programme and the creation of the unified Probation Service to understand how this resonated with their practical experience. In June 2021, the Lord Chancellor had written to Parliament confirming probation services had been unified. 

    The team told us that they consciously chose to deliver at speed and identified a clear narrative for the reforms being at pace. As such, everyone was clear on the reasons for the reforms. The team also made clear that there was zero contingency beyond the expected delivery date. Alongside setting a minimum expectation of the requirements needed for Day 1, this helped force the pace and prioritisation of effort. 

    The programme team also highlighted the importance of strong leadership, with a culture of accountability and responsibility, to deal with any uncertainties or issues. In particular, they spoke of a culture which encouraged people to raise any problems they’d encountered, rather than hide them or focus on the ‘good news’. 

    Additionally, the programme team said they had built a strong internal assurance team, comprised of former senior operations staff, to carry out site visits and desktop reviews to ensure the programme was on track. 

    Alongside this, the programme team outlined the advantages of a flexible programme structure. The team recognised that it was difficult to plan everything up front, and instead ensured they had the required processes and information needed to respond quickly. This was done through regional teams, with a dedicated senior manager, tasked with identifying risks as soon as possible. This meant that the central programme team could deal with ‘unknown unknowns’ effectively when they arose.  

    Our insights 

    Many of the points raised by the Probation Reform Programme team align with our insights. In particular:  

    • Including speed as a specific programme objective to provide a clear framework for decision-making and help make trade-offs between speed, cost and outcomes. 
    • Building teams with the right leadership, skills and experience to make clear, timely and reliable decisions. 
    • Tailoring processes to add value and momentum to programme decision-making. 
    • Recognising the uncertainties of delivering at speed and managing these. 

    As speed remains important for ambulances, so it will for some programmes, particularly with commitments to achieving ‘net zero’ greenhouse gas emissions by the fixed deadline of 2050. Our report helps those deciding whether to deliver at speed ask questions to determine when or how this should be done and then continually test whether a programme will achieve its outcomes.  

    Further reading 


    About the authors

    Josh Perks is a qualified accountant with experience of working on the NAO’s transport team. His work has included audits of the main government transport bodies and value-for-money studies of major rail programmes. Recently, he has taken an active role in the NAO’s Major Projects Delivery Hub.

    Jemma Dunne is an Audit Manager and has delivered value for money reports across areas such as health and defence, including those on government programmes. She is a qualified chartered accountant (FCA) and holds the APM Project Management Qualification (PMQ).

    Comment on this post...

  • Effective governance and accountability is a mainstay for successful contract management

  • Posted on December 15, 2021 by

    This is the last in a series of posts on our good practice guidance for managing the commercial lifecycle. In that guide we shared fresh insights from our extensive body of work on government’s commercial activities.  

    It’s not surprising that many of the examples in our guidance related directly to responding to the COVID-19 pandemic. Just as everyday purchases we normally take for granted like buying pasta from the local supermarket, became complex thanks to the effects of the pandemic, so did government’s processes for buying the things it needed.  

    Of the supporting elements in our guidance which apply across the whole contract letting spectrum, governance and accountability may be the most crucial for responding to big changes and extreme circumstances. Risks are more likely to be borne out at any stage without the right people to make appropriate decisions and provide adequate scrutiny. 

    Explaining governance and accountability 

    When considering governance, we look at the oversight arrangements at an organisational level – poring over what risk reviews have taken place, and the adoption of learning from previous lessons.  When we consider ‘accountability’, we reference the support to contract managers; others charged with responsibility for the governance of contracts and the organisation as a whole (including the Accounting Officer for whom the duties for managing public money and reporting to Parliament falls to).  

    To be effective, these arrangements should facilitate open discussion and continuous improvement. Public bodies should demonstrate robust, independent oversight of both their contractual arrangements and overall commercial portfolios. We have often found that organisations do oversight well at a ‘big picture’ level or in fine detail but often cannot achieve both. 

    Accountability in complex systems 

    In 2020, we reported on the value for money of the local bus service system overseen by the Department for Transport. The report looked at the effectiveness of the government’s support for local bus services and whether the tools to improve local bus services were in place. The Department is not accountable for delivering bus services, but it has a national policy responsibility. We found that, during the COVID-19 pandemic, the Department came together with local authorities and operators, intervening rapidly to target the weakest areas and keep buses running – a good learning point for future programmes. 

    Using governance to support speed 

    Effective governance is important at all times but crucially so when risks are higher and goods and services are required to be delivered at greater speed. The NAO’s work on Lessons Learned: Delivering Programmes at Speed highlights the contribution governance can make in such situations. This publication references the need for governance structures to be tailored to support the pace of the work. We have seen different structures work in different circumstances, but the main principles are to have clear accountability lines and to involve the right people at the right time. 

    An example of the importance of clear accountability comes from the NHS Test and Trace Service. This was created to lead the government’s COVID-19 test and trace programme, which was delivered in part by contracted providers. The NHS Test and Trace Service was part of the Department of Health & Social Care (the Department) and was subject to the Department’s financial, information and staffing controls. However, the executive chair of the NHS Test and Trace Service did not initially report to the Department’s ministers or Permanent Secretary, but to the Prime Minister and the Cabinet Secretary. This unusual organisational relationship created dual reporting lines, which brought risks of unclear accountability. The relationship subsequently changed so that the executive chair reported to the Secretary of State for Health. 

    What good looks like 

    In defining how to achieve good governance and accountability to support successful contracts, our good practice guide sets out areas of improvement and outlines our expectations of best practice. They include:  

    • Accountability is defined and responsible officers are appropriately empowered. 
    • There is effective independent scrutiny of commercial activity. 
    • Lessons learned promptly feed into the wider strategy and plans for other contracts, and are integrated into needs assessment and the list of strategic suppliers. 
    • Reliable, timely management information is used for rapid diagnosis of issues and prompt action. 
    • The organisation’s own performance against obligations and the supplier’s view of performance are known and considered. 
    • Industry expertise is valued and used, subject to robust independence controls. 

    Wishing you all successful governance and accountability arrangements for your contracting activities!  

    About the author

    Iain Forrester

    Iain Forrester is a qualified accountant with long experience of working on the NAO’s commercial and contracting related work. This has included cross-government work on grants, shared services, EU Exit, and the government’s response to COVID-19. He also worked on the commercial and contract management insights guide published in 2016.


    Comment on this post...

  • Commercial strategy needs to be joined up to achieve best value

  • Posted on November 25, 2021 by

    Last December, the government published a Green Paper on Transforming Public Procurement. It  stressed that investments should be subject to consideration of the public good, including supporting national priorities. It discussed leveraging commercial activity to achieve social and environmental value.

    For our good practice guidance for managing the commercial lifecycle, we examined similar opportunities and how to support them. We shared fresh insights and learning from our extensive body of work on government’s commercial activities. In this latest post, I will share some of our insights on commercial strategy.

    Commercial strategy means thinking about the overall approach to ensure that procurement and other commercial activities provide the outcomes that government wants and benefit us all. This is the part of our guidance which really focuses on the context around what government does when it runs a competition or revises a contract, and the importance of joining up different elements.

    Joining up commercial strategy is vital if government wants to achieve its wider aims as well as value for money. That includes establishing a consistent approach to risk management, and the organisational capacity and capability to respond to uncertainty. It is also where other considerations come in, including where procurement can be a lever for larger goals like encouraging innovation or diversifying the landscape of suppliers to government. Commercial strategies should demonstrate how each commercial agreement aligns with wider strategic objectives and how this is then reflected in the approach for managing commercial risks and incentives throughout the commercial lifecycle.

    A couple of our past reports give good examples of the importance of joining up strategy at the programme level and more widely.

    Aligning timetables

    In 2018 we reported on The Ministry of Defence’s arrangement with Annington Property Limited, a sale and leaseback arrangement for accommodation. As part of our review, we found that the timetable for developing the Ministry of Defence’s wider estate strategies was not aligned with the timetable for rent reviews. The department was developing a ‘Future Accommodation Model’ intended to provide personnel with more flexible accommodation options. However, the timing meant that its negotiations on the sites with Annington would begin before a decision was taken on the wider model. This affected its ability to develop negotiating strategies for these sites. We recommended that the department align the timetables to use realistic scenarios in its negotiations, giving it a clearer strategic view.

    Leveraging procurement across government 

    On a wider scale, we reported in 2016 on the government’s spending with small and medium-sized enterprises (SMEs). The government recognised that SMEs could offer many benefits to the public sector, including flexibility, innovation and better value for money due to lower overhead costs, as well as increasing local investment and improving social outcomes. We recommended that the government should take a more focused approach, identifying where SMEs could bring the most benefit, and look into an integrated cross-government procurement platform to support its commercial strategy. The government has since introduced guidance for SMEs applying for contracts and promised to invest in joining up the different procurement systems. The intention is that this will help drive the commercial benefits from better data sharing – as part of changes to procurement processes following exit from the European Union.

    That is an example of the kind of strategic approach to identifying risks and opportunities which we want to see applied consistently across organisations, and we look forward to seeing how government’s follow-up to Transforming Public Procurement would help to further encourage this.

    What good looks like

    Our good practice guide sets out areas of improvement and outlines our expectations of best practice, with specific case study examples that demonstrate some of these expectations of a joined-up commercial strategy. They include:

    • Commercial, policy, operational and business teams work together to develop a clear understanding of the contracts and produce required outcomes
    • Each contract staffing model is developed early, regularly reviewed and tailored to different contract stages
    • Capability plans include operational resilience to address unplanned demands
    • Knowledge and experience of underlying contract issues is retained throughout the lifecycle of a commercial relationship
    • There is investment in the organisation and its people to ensure adequate access to training and development to support commercial awareness and expertise.

    About the author

    Iain Forrester

    Iain Forrester is a qualified accountant with long experience of working on the NAO’s commercial and contracting related work. This has included cross-government work on grants, shared services, EU Exit, and the government’s response to COVID-19. He also worked on the commercial and contract management insights guide published in 2016.


    1 Comment

  • A small leak will sink a great ship: the need to counter fraud and error in government

  • Posted on November 16, 2021 by

    My outdoor tap leaks. Not very much, just a small drip. And though I put a bucket underneath to catch the drips, I’ll admit that sometimes the bucket overflows before I can use the water in my garden. I know I should find out if it’s just a dodgy washer or I need a replacement tap but somehow it never makes it to the top of my to-do list.

    Out of interest, the other day I looked up how much a leaky tap costs the average household. Potentially hundreds of pounds each month, so safe to say it makes financial sense to get it fixed. Then I saw that leaky taps cost UK householders an estimated £3 million every year. Thousands of little drips adding up to a big chunk of money being washed away.

    The Government Counter Fraud Function (GCFF) faces a similar challenge when trying to tackle fraud against the public purse. Government estimates that £26.8 billion a year is lost to fraud and error in the tax and welfare system, but for me, the most surprising thing is that GCFF estimate that up to £25 billion a year more is lost through fraud and error in other areas of government spending. The measurement data available suggests that most departments are losing a relatively small amount to fraud and error every year, but these hundreds of small leaks add up to an eye watering cost to the taxpayer.

    Part of the problem government faces is this sheer diversity of risk – fraud and error impacts everything from grants and procurement to income collection. Fraud and error has also traditionally been the sole responsibility of each department to manage leading to considerable variations in approach to similar risks. Although this has clear benefits for accountability, focusing on risks by organisation rather than type and across multiple organisations leads to missed opportunities. Most government grant programmes are likely to face similar challenges when it comes to managing fraud and error risks, even if their exact nature varies.

    The GCFF was established in 2018 as a means to bridge these gaps and bring together the 16,000 or so people working in public sector counter-fraud to share knowledge and best practice. Though progress has been made, in June 2021 the Committee of Public Accounts reported that the

    Cabinet Office and HM Treasury’s central mechanisms for managing fraud and error are still in their infancy

    There are also less than 7,000 recognised counter-fraud professionals working across central and local government and policing, with more than 75% of these working in tax and welfare. That doesn’t leave many qualified specialists to tackle all the other fraud risks across government.

    The GCFF recognises that the information it holds on fraud and error losses outside the tax and welfare system needs improvement – its fraud loss estimate of 0.5% to 5% of expenditure is a massive potential range. In our Good Practice Guide on Fraud & Error, we set out how departments need to ensure they have a cost effective control environment. This means doing everything they reasonably can to minimise fraud and error, to the point where doing anything more would have a detrimental impact on wider objectives. In a world of limited resources is it enough to replace a washer or do they need a brand new tap?

    The Good Practice Guide also includes our Fraud and Error Audit Framework, developed over several years based on best practice in government and the private sector. Fraud and error risk is continuously evolving, and the Framework provides a structure for assessing how management uses an iterative approach to measure the effectiveness of its counter-fraud and error activities and to continuously improve its controls.

    Ultimately, without more precise information on the scale and causes of fraud and error outside the tax and welfare system, government risks large amounts of fraud and error remaining unidentified or untackled. Ensuring that more effort goes into improving government’s understanding of exactly where money is leaking from the system  is a key focus of our ongoing work on fraud and error. Now, more than ever, it’s important to make sure that vital taxpayers’ money isn’t being washed down the drain.

    About the author

    Katie Dixon manages our work on fraud and error within the Financial and risk Management Hub. She joined us as a trainee in 2011 and, after qualifying as an accountant, completed a masters degree in counter fraud and counter corruption studies. She has experience auditing fraud and error risks across financial, investigative and value for money audits and represents the NAO on several public sector fraud expert panels.

    Follow Katie on LinkedIn


    2 Comments

  • Capability is for contract life, not just for procurement

  • Posted on November 2, 2021 by

    The NAO’s work includes looking at a huge range of government activities, and the setting up and managing of commercial arrangements are central to many of them. This became very clear when looking back at twenty years of our work auditing government’s spending and reporting on its value for money for taxpayers. Over this time, we have assessed over 350 of government’s agreements with commercial partners to deliver services and goods for the public.  

    Government has come a long way in developing its expertise in this space, but the pandemic and the need to procure services quickly has highlighted the extent to which further improvements are needed. Our perspective looking right across government and its programmes puts us in a unique position to draw together lessons and the common themes which have kept coming up in our work. 

    One example is our recently published Good practice guidance: managing the commercial lifecycle. In it we share fresh insights and learning from our extensive body of work on government’s commercial activities. The guidance has 10 sections – six procedural steps and four supporting elements. I previously wrote about one of those supporting elements, data, and this time I want to focus on another: capability.  

    Capability covers both personal effectiveness and organisational capability. It means having people within commercial teams and elsewhere with the appropriate commercial skills, at the right time. It also means supporting them with appropriate organisational leadership, systems and levers to deliver the required outcomes. Achieving all of this will require some measure of collaboration across organisations – be they cross government, or the wider public and private sectors.  

    Getting these capability aspects right is crucial throughout the entire process of the commercial lifecycle, from requirement through to transition at the end of the contract and is a key area for improving outcomes. An important aspect of managing contracts is being able to respond to change.  

    Substantial additions to government’s programmes, like the purchasing of COVID-19 vaccines and the associated equipment to roll out the vaccine programme, have shown just how much change can occur in a short time. This comes alongside the pandemic’s disruption of long-held assumptions and traditions, with huge impacts to our ways of life. 

    If that much change can take place within a couple of years, it’s clear that contracts, which sometimes span decades, must often react to very significant changes. That means that organisations need to make sure they maintain the appropriate skills and capability to manage contracts and commercial activities as a whole.  

    Some of our past reports highlight the importance of capability approaches. For instance, in our progress report on Terminating the Magnox contract, we highlighted the importance of reassessing capability throughout the life of a contract.  

    In March 2017, the Nuclear Decommissioning Authority (NDA) decided, based on legal advice, to terminate a 14-year reactor decommissioning contract due to a “significant mismatch” between the work specified in the tendered contract and the work that needed to be done. The NDA decided to renegotiate the contract with the incumbent, with the contract ending in 2019, nine years earlier than originally planned. To react to the change in circumstance and to better equip itself, the NDA commissioned a review of the delivery plan to improve its intelligent client capability before changing contract arrangements. It also strengthened its executive team, including a new commercial director, and increased the capacity of its contract management team. These changes were an example of meeting the expectation that contract staffing models should be regularly reviewed and tailored to different contract stages.  

    In our report on the BBC’s TV licence fee collection, we recognised how the BBC had benefited from introducing contract governance and reporting, supported by a multi-disciplinary team model and a wider strategic contracts infrastructure. We also recommended that it should maintain information on commercial skills to enable it to adapt to changes, such as upgrades to technology and ICT, which require different skill sets. This point was particularly important because we had identified that the BBC did not routinely assess its commercial skills and future requirements. 

    In our good practice guide we emphasise ways that improvements can be made to organisational and people capability, and their application throughout the commercial lifecycle.  

    There is an opportunity to make projects more successful in the short and long-term. We include expectations that:  

    • Commercial, policy, operational and business teams work together to develop a clear understanding of the contracts and produce required outcomes 
    • Each contract staffing model is developed early, regularly reviewed and tailored to different contract stages 
    • Capability plans include operational resilience to address unplanned demands 
    • Knowledge and experience of underlying contract issues is retained throughout the lifecycle of a commercial relationship 
    • There is investment in the organisation and its people to ensure adequate access to training and development to support commercial awareness and expertise. 

    The guidance also includes our expectations of good practice for all other stages of the commercial lifecycle and draws attention to some of the most important things for government to improve in its commercial activities. Stay tuned for further entries in this blog series, the next of which will be on commercial strategy. 

    About the author

    Iain Forrester

    Iain Forrester is a qualified accountant with long experience of working on the NAO’s commercial and contracting related work. This has included cross-government work on grants, shared services, EU Exit, and the government’s response to COVID-19. He also worked on the commercial and contract management insights guide published in 2016.

    Comment on this post...

Right column

  • About the NAO blog

    Our experts share their views about issues and common challenges facing government, what public sector leaders should look out for and how organisations have addressed issues. Our posts draw together threads from across our reports, share secrets spilled in events and reveal our experts’ expectations for the future.

    We encourage comments that support the exchange of ideas for improvement, but ask that those posting are respectful of others.

  • Sign up for automatic feeds

    Sign up to receive email alerts:




    RSS IconSubscribe in an RSS Reader