Posted on June 1, 2021 by Daniel Lambauer
In our ever-increasing digital and automatized world, certain buzzwords are becoming more centre stage in the public sector. One of them is “artificial intelligence”. While the concept, and development, of artificial intelligence is not new (artificial intelligence was first recognised as a formal discipline in the mid-1950s), it is a word that has been casually thrown around more in recent years in the public sector, and sometimes carelessly.
Traditional algorithms vs machine learning models
These days modern data scientists normally associate with artificial intelligence systems that are based on machine learning models. Machine learning models deploy methods that develop rules from input data to achieve a given goal.1 There is a difference to, what you may call, traditional algorithms. Traditional algorithms don’t need data to learn, they just churn out results based on the rules inherent to them.
Traditional algorithms have been used in the public sector for some time to make decisions. The latest example making the headline was the model determining A level exam results last summer. From an auditing perspective, as the basis of the algorithms are usually transparent, auditing them is something we as a public audit institution are used to.2
But artificial intelligence that is based on machine learning is different – it has only been (cautiously) employed in the public sector in recent years.
It is different because, firstly, for a machine learning model to learn it needs good, quality data – and often a lot of it. Our report on the challenges of using data across government has shown that that condition is not always given.
Secondly, it can be quite costly to develop and deploy them. Moreover, the benefits are not always guaranteed and immediately realisable. In a public sector context with tight budgets, the willingness to put money behind them may not always be there.
The reason for this is related to a third point. It is not always certain from the outside what the machine will learn and therefore what decision-making rules it will generate. This makes it hard to state the immediate benefits. Much of the progress in machine learning has been in models that learn decision-making rules that are difficult to understand or interrogate.
Lastly, many decisions affecting people’s lives that artificial intelligence models would support pertain to personal circumstances and involve personal data, such as health, benefit or tax data. Whilst the personal data protection landscape has strengthened in recent years, there are not always the organisational regulatory structures and relevant accountabilities in place of the use of personal data in machine learning models.3 Public sector organisations are therefore at risk of inadvertently falling foul of developing data protection standards and expectations.
How to audit public sector machine learning models
Given all these challenges, it may not be surprising that in our public audit work, we are not coming across a lot of examples of the use of machine learning models in decision-making. But there are examples4 and we foresee that they may be growing in the future.
We have therefore teamed up with other public audit organisations in Norway, the Netherlands, Finland and Germany, and produced a white paper and audit catalogue on how to audit machine learning models. You can find it here: Auditing machine learning algorithms (auditingalgorithms.net).
As the paper outlines in more detail, we identified the following key problem areas and risk factors:
- Developers of machine learning models often focus on optimising specific numeric performance metrics. This can lead them to neglect other requirements, most importantly around compliance, transparency and fairness.
- The developers of the machine learning models are almost always not the same people who own the model within the decision-making process. But the ‘product owners’ may not communicate their requirements to the developers – which can lead to machine learning models that increase costs and make routine tasks more, rather than less time-consuming.
- Often public sector organisations lack the resources and/or competence to develop machine learning applications internally and therefore rely on external commercial support. As a result they may take on a model without understanding how to maintain it and how to ensure it is compliant with relevant regulations.
We also highlighted the implications for auditors to meaningfully audit artificial intelligence applications:
- They need a good understanding of the high-level principles of machine learning models
- They need to understand common coding languages and model implementations, and be able to use appropriate software tools
- Due to the high demand on computing power, machine learning supporting IT infrastructure usually includes cloud-based solutions. Auditors therefore also need a basic understanding of cloud services to properly perform their audit work.
Our audit catalogue sets out a series of questions that we suggest auditors should use when auditing machine learning models. We believe it will also be of interest to the public sector bodies we audit that employ machine learning models. It will help them understand what to focus on when developing or running machine learning models. As a minimum, it gives fair warning what we as auditors will be looking for when we are coming to audit your models!
1 In fact there are two main classes of machine learning models. Supervised machine learning models attempt to learn from known data to make predictions; unsupervised machine learning models try to find patterns within datasets in order to group or cluster them.
2 See for example our Framework to review models – National Audit Office (NAO) Report to understand more about what we look out for when auditing traditional models and algorithms. We currently have some work in progress that aims to take stock of current practices and identify the systemic issues in government modelling which can lead to value for money risks
3 In the UK the Information Commissioner Office has published guidance on the use of personal data in artificial intelligence: : Guidance on AI and data protection | ICO
4 For some UK example see: https://www.gov.uk/government/collections/a-guide-to-using-artificial-intelligence-in-the-public-sector
About the author:
Daniel Lambauer joined the NAO in 2009 as a performance measurement expert and helped to establish our local government value for money (performance audit) team. He is the Executive Director with responsibility for Strategy and Resources. As part of his portfolio, he oversees our international work at executive and Board level and has represented the NAO internationally at a range of international congresses. He is also the NAO’s Chief Information Officer and Senior Information Responsible Owner (SIRO). Before joining the NAO, Daniel worked in a range of sectors in several countries, including academia, management consultancy and the civil service.
Posted on April 29, 2021 by Yvonne Gallagher
The shielding programme was a swift government wide response to identify and protect clinically extremely vulnerable (CEV) people against COVID-19.
Our recent report on Protecting and supporting the clinically extremely vulnerable during lockdown, shows how government quickly recognised the need to provide food, medicines and basic care to those CEV people shielding. This had to be pulled together rapidly as there were no detailed contingency plans.
But there was a problem. In order to do this, government was faced with the urgent task of identifying the people who needed support based on existing, disparate data sources.
Difficulties in extracting and combining data
The urgency of this exercise was recognised by all involved, but difficulties in extracting, matching and validating data from across many different systems meant that it took time for people to be identified as CEV.
At the start of the pandemic, there was no mechanism to allow a fast ‘sweep’ across all patients to identify, in real time, those who fell within a defined clinical category.
It was a major challenge to identify and communicate with 1.3 million people by extracting usable data from a myriad of different NHS and GP IT systems all holding data differently.
This lack of joined-up data systems meant NHS Digital had to undertake the task of accessing and extracting GP patient data, stored in different ways in each practice and holding specific details about people’s medical conditions to merge with their own databases. It took a huge effort by the team to complete this task in three weeks.
Data issues were not resolved by the time of the second lockdown
Government had identified systems were not capable of ‘speaking’ to each other across hospital, primary care, specialist and adult social care services following the first iteration of shielding (March – August 2020), and sought to apply them to the second lockdown towards the end of 2020. However, our report highlighted resolving the data issues was not an area where significant progress had been or could be made.
This reflects the wider issues of data across government
These challenges are examples of broader issues that we have previously highlighted in our report on Challenges in using data across government. People often talk about better use of data as if this is a simple undertaking. But there are significant blockers and constraints that require sustained effort to overcome, which apply to all areas of government trying to use and share data other than for the single purpose it was originally created for.
The basic issues are widely known and acknowledged:
- Huge variability in the quality and format of data across government organisations
- Lack of standardisation within departmental families and across organisational boundaries making it difficult for systems to interoperate
- The extent of legacy IT systems across government further compounding the difficulties
- Ownership and accountability aren’t easily agreed where a shared dataset of personal data is brought together and has equal value to different services.
It’s unclear to us how calls to establish and enforce data standards are going to work in practice if existing systems can’t be modified to support them and there is no firm timetable, road map or funding commitment for replacing them.
In our report Digital transformation in the NHS, we reported that 22% of trusts did not consider that their digital records were reliable, based on a self-assessment undertaken in 2017. The average replacement cycle for a patient records system is something in the region of once every 15 years so this change isn’t going to happen overnight.
Our aim is to support government in tackling these issues, and not to be critical of past failings, because we recognise that it is hard. We set out a number of recommendations in our data report and they are summarised in our accompanying data blog.
Some are aimed at the centre of government and others are steps that individual organisations can take. Our cross-government recommendations were primarily around accountabilities, governance, funding and developing rules and common ways of doing things.
Our recommendations for individual organisations are:
- Put in place governance for data, including improving the executive team’s understanding of the issues associated with the underlying data and the benefits of improving that data
- Set out data requirements in business cases. This should include an assessment of the current state of the data, and the improvements or new data that are necessary. These assessments should have an explicit consideration of ethics and safe use
- Implement guidance for front-line staff for handling data, including standardisation, data ethics and quality.
Organisations that hold a cohesive view of their citizen/patient data must address this issue in a managed and incremental way, rather than having to resort to one-off costly exercises which have to be repeated when the next need arises. This will require sustained effort and perseverance.
Unfortunately, there are no easy shortcuts, but with a will to put in the necessary effort progress can be made one step at a time.
Yvonne is our digital transformation expert, focused on assessing the value for money of the implementation of digital change programmes. Yvonne has over 25 years’ experience in IT, business change, digital services and cyber and information assurance, including as CIO in two government departments and senior roles in private sector organisations, including the Prudential and Network Rail.
Posted on April 22, 2021 by Gareth Davies
Like governments around the world, ours has committed unprecedented amounts of public money to the fight against coronavirus. By the end of 2020, this reached £271 billion in the UK and will continue to increase.
As the UK’s independent public spending watchdog, the National Audit Office has been tracking the Government’s pandemic spending commitments, reporting to Parliament and the public on whether that money has been accounted for correctly and spent as intended. The Committee of Public Accounts has also used our findings as the basis for taking evidence from senior civil servants and for making its own recommendations.
One year on from the start of the pandemic, what can be learned from the way government has responded to it?
Like many countries, the UK was not well prepared for this pandemic. While we recognise that government cannot be expected to plan for every eventuality, we have repeatedly found that there was no contingency plan to deal with the unfolding situation. And, where plans were in place, these did not anticipate this type of pandemic.
Reflecting on this difficult starting point, we have independently assessed each element of the Government’s response based on what was reasonable to expect in the circumstances.
The urgency and scale of action required meant that ensuring value for money for the public did not always take priority. Trade-offs were necessary, which increased the risk of financial losses. So, the question we looked to answer in our audits was how well those trade-offs were understood and managed.
The furlough scheme, designed and implemented by HMRC and HM Treasury, and the scaling up of Universal Credit payments by the Department of Work and Pensions, were delivered at impressive speed and against a sudden and huge increase in demand.
Yet, even with this sterling effort by the Civil Service, the speedy response has come at a cost – higher levels of fraud and error than government would have otherwise expected.
This increased risk of financial losses is seen most clearly in the Bounce Back Loan Scheme. The scheme has provided vital cash flow support to small- and medium-size businesses – by the end of March 2021, more than 1.5 million loans had been issued with a total value of £47 billion.
However, when initially launched, the scheme proved slow and cumbersome for smaller businesses. In response, credit and affordability checks were removed from the process for loans of up to £50,000 and government guaranteed 100 per cent of the loans. This sped up loans and proved a lifeline for thousands of smaller businesses, but is likely to come at a cost.
When we reported on this in November 2020, government estimated that 35 per cent to 60 per cent of borrowers may default on the loans. A better indication of the true cost to the public purse will begin to emerge from May when the first repayments are due, although businesses are able to apply to defer this.
Those involved in the procurement of personal protective equipment (PPE) faced the considerable challenge of an overheated global market for PPE and an inadequate UK stockpile. Necessary trade-offs, to allow for rapid acquisition of life-saving equipment, were less well managed. At the height of the emergency, it was reasonable to place urgent orders directly with suppliers, rather than use slower competitive tendering methods. But even allowing for the urgency of the situation, essential standards of government transparency were not consistently met. This includes how some suppliers were picked and how a high priority channel for considering certain suppliers was created.
In emergency situations where the assurance provided by open competition is not available, it is even more important to provide prompt and full transparency to maintain public trust in how taxpayers’ money is being used.
The public health response to the virus has required government to create and deliver a vaccine programme and a test and trace operation at a scale and pace never seen before. The success of the UK vaccine programme is based on shrewd investments in candidate vaccines, brilliant scientists, effective commercial agreements with industry and the delivery power of a National Health Service bolstered by an army of volunteers. Public money had to be committed when there was no guarantee of vaccine effectiveness and this risk was managed well, making good use of relevant scientific and commercial expertise.
As the vaccine roll-out progresses and as lockdown measures lift, NHS Test and Trace’s role in identifying and suppressing outbreaks will become more vital. Our initial work on Test and Trace in December found that it had achieved a rapid scale-up in activity and had built much new infrastructure and capacity from scratch. However, we also highlighted value for money concerns and weak evidence of the effectiveness of the service.
Test and Trace’s operations will transfer to the new UK Health Security Agency, which is expected to become fully operational later in the year. We will report again on the progress Test and Trace has made in the summer.
There is much to be learned already from the pandemic. To promote transparency, government must clearly define its appetite and tolerance of risk, particularly under emergency spending conditions. Uncompetitive procurement practices must not be allowed to become a new norm. It should also monitor how Covid-19 programmes are operating, dynamically updating demand forecasts, and ensuring it has the ability to flex its response.
Reporting on the Government’s ongoing response to the pandemic will remain a priority for the NAO. Our upcoming work will include a review of the role of Greensill Capital in Covid-19 loan schemes. We will also publish a series of lessons learned reports starting in May, designed to be of value for the remainder of this pandemic and to help the UK better prepare for future emergencies.
This article was first published in the Daily Telegraph
Tagged with: COVID-19
Posted on February 1, 2021 by Gareth Davies
Since the COVID-19 pandemic began, the NAO’s role has been to provide Parliament and the public with evidence-based reports on how public money has been used to tackle the crisis. So far we have published 12 reports on different elements of the pandemic response, including for example the test and trace programme; the procurement of PPE and ventilators; the furlough scheme; and loans to businesses. All of our COVID-19 work is on our website, along with details of the pipeline of COVID-19 work in progress, due for publication in 2021.
Our work so far has highlighted the challenge faced by government of responding effectively to an unprecedented public health and economic emergency whilst maintaining control over how (and how well) public money is spent. Our reports show how the trade-off between speed, effectiveness, cost and control has been managed in the different elements of the COVID-19 response and provide important learning for the rest of this pandemic and any future public health emergencies. The Public Accounts Committee has held public sessions on each of the topics covered by our COVID-19 reports, taking evidence from the officials responsible and issuing its own reports.
To support transparency and the effective scrutiny of government spending, we are continuing to update our COVID-19 cost tracker, with the latest update made on 29 January 2021. As well as providing the latest estimate of the cost of every significant government commitment as part of its pandemic response, the tracker shows spend to date where that is available. It also allows the data to be downloaded and analysed by type of support, department responsible and date of commitment.
Although the pandemic has rightly required significant audit attention, our work programme has also covered other important areas of public spending. In November, we reported on the state of preparations at the UK border for the end of the EU Exit transition period on 31 December 2020. We will follow up how the new border arrangements are working in practice later this year.
In December, our first report on the government’s progress in meeting its commitment to a net zero carbon economy by 2050 looked at the governance and management arrangements being put in place to deliver this big shift in how we generate power, heat our homes, use our land and travel. We are following up with audits of specific elements of the net zero strategy, such as government’s role in encouraging the transition to ultra-low emission cars.
COVID-19 has also impacted the NAO’s other major area of work, the audit of government department and arm’s-length body accounts. Finance and audit teams alike adjusted well to fully remote accounts preparation and audit for the 2019-20 annual reports and accounts. Overall, audits took longer to complete, partly due to the logistical impact of lockdown but also because of the impact of the pandemic on 31 March 2020 asset and liability valuations and on the going concern status of organisations facing significant loss of income. I had to include ‘emphasis of matter’ paragraphs in 84 of my audit reports, drawing attention to significant uncertainty in these areas.
Looking ahead, government’s 2020-21 accounts present significant accounting and audit challenges. For the departments in the front line of the pandemic response, they must account for tens of billions of unplanned spending during the year, often in risky control environments. Our audits will assess the robustness of the estimates and judgements made by departments in accounting for this spending.
As well as responding to the pandemic in the last year, we have also been making progress on the new strategic priorities we set for the NAO for the five years 2020 to 2025. This is already visible in our new series of lessons learned reports, bringing together good practice, warning signs and tips for success on important areas of public spending. The first in this series, Lessons learned from major programmes, was published in November. As part of our focus on audit quality, we’ve also embarked on an overhaul of our audit methodology and the procurement of a new audit software platform, which will incorporate powerful new data analytics.
All of this work has only been possible thanks to the commitment and professionalism of my colleagues who, like so many others, have continued to deliver our work programmes whilst managing the impact of lockdowns, home schooling and other pressures. We’re conscious that others, notably healthcare and other front line workers, are handling much greater challenges. That thought focuses us on helping government to extract as much learning as possible from this experience so that the country is prepared for any future emergency of this kind.
Tagged with: COVID-19
Posted on December 14, 2020 by Simon Bittlestone
Achieving net zero greenhouse gas emissions will be a colossal challenge for the UK and one that will require all parts of government to work together. The NAO has audited numerous cross-government projects and programmes in the past, ranging from the 2012 Olympics to preparations for EU Exit. This experience has given us a deep insight into the factors that are likely to be critical to the government achieving its climate change ambitions, which are outlined in our recent report Achieving net zero.
We found that although the government has reorganised its approach to tackling climate change since setting the net zero target, it still needs to put in place effective cross-government working to ensure it can step up to the challenge. These findings are even more relevant now that the government has announced the UK’s Nationally Determined Contribution, which commits us to reducing our emissions by 68 per cent on 1990 levels by 2030.
Of course, we recognise the scale of the task ahead for government. Achieving net zero means finding ways to reduce greenhouse gas emissions from all sectors of the economy, including those where progress to date has been limited, such as transport and buildings. It’s also not yet known what role new technologies will play in reducing emissions, or how much individuals will change habits and lifestyles. Adding to the complexity, many of the activities required to reach the goal are interdependent; for example, increasing the number of electric vehicles creates demands on the power system. Achieving net zero will require careful sequencing and planning.
Net zero is a truly cross-government endeavour. The Climate Change Committee, which advises government on how to achieve net zero, made recommendations to 15 different government departments in its most recent progress report. This means departments need to have finely-tuned ways of working together that mean everyone is pulling in the same direction. Government’s coordination arrangements for net zero are still evolving and some have been delayed because of COVID-19. It is still too early to say how well the arrangements are working.
Government wants to create a sense of collective ownership for achieving net zero, rather than responsibility resting with one central ‘owner’. But there are certain risks to consider for this to happen, such as ensuring net zero takes sufficient priority in relation to government’s other aims. There also needs to be enough capability across government, and sharing of information across departments, including relevant insights that could be applied across different sectors.
The challenge of achieving net zero stretches beyond central government and into a wide range of other public bodies, including regulators, arm’s-length bodies and local authorities. However, the responsibility of these public bodies in meeting the net zero target is not clear. For example, local authorities do not have a clearly defined role in the achievement of net zero, despite the important part they could play in decarbonising transport and housing, and encouraging businesses and individuals to take action. More can be done to ensure public bodies reduce their own emissions. Decarbonising the public sector is an important signal that government is taking the net zero target seriously, as well as setting an example for businesses to follow.
We looked at government’s plans over the next 12 months to establish a net zero strategy and the challenges it will face beyond. Establishing the strategy – which government plans to do before it hosts the global climate conference “COP26” in November 2021 – is a critical step for clarifying its plans around how net zero will be achieved. A clearer pathway towards achieving target is more likely to build private sector investors’ confidence to back the technologies and infrastructure that will be needed. Government needs to find a way to provide this greater clarity, whilst at the same time, giving itself the flexibility to react to unexpected events or new solutions emerging over the next three decades.
Once the government’s plan is in place, it needs to ensure it has a way to track progress on the policies aimed at achieving net zero. While there is good annual reporting on greenhouse emissions, which give a broad sense of where the UK is on the road to net zero, currently government doesn’t have a way of assessing the overall performance of its key emissions policies. This means senior decision makers could lack an understanding of where key problems might be emerging, or where resources need to be reprioritised. There is also a lack of collated information on the costs and benefits of policies aimed at achieving net zero, which could hamper resourcing decisions and preclude full public accountability.
There is a need to reach out of Whitehall and engage businesses and individuals, who will be critical to achieving net zero. Almost every individual will also have some role to play, whether that is changing the type of car they drive, updating their household heating system or even reducing the amount of meat and dairy products they consume. Currently, there is a mismatch between the public’s support for action to tackle climate change and an understanding of the changes that everyone will be required to make.
You can read our report in full here: Achieving net zero. The next year is set to be a critical period in the net zero journey, and we plan to continue working with government departments over that time as it considers and implements our recommendations. We are undertaking a programme of work to assess the government’s activities in particular aspects of achieving net zero: one on environmental tax measures and another on reducing carbon emissions from cars.
Simon Bittlestone, manager of net zero studies at the National Audit Office.
Posted on September 24, 2020 by Leena Mathew
Given the wide-reaching impact of membership of the EU, preparing for the UK’s departure from the EU represented a significant challenge to government, with considerable potential consequences for businesses and citizens.
Our programme of work on EU Exit
Over the last four years, the NAO has carried out a substantial programme of work on various aspects of government’s EU Exit preparations. We have published 28 reports so far, which have provided a detailed picture of significant EU Exit projects and programmes in real time, looking from planning into implementation stages. Using these reports, we compared different approaches taken by departments and considered the common barriers or enablers for achieving outcomes. We also returned to the same issues over time – for example, in our suite of reports on the border – to provide a point-in-time assessment of progress and risks, and to understand how and why things had changed.
Our work highlighted the large amount of new activity departments were undertaking. This included developing understanding in new areas such as the Department for Health and Social Care’s work on supply chains; getting involved in new markets such as the Department for Transport’s procurement of freight capacity; or setting up new structures such as the cross-government Border Delivery Group. We recognised that new ways of working were necessary for government to meet the challenges it faced, and risks could be better managed if actively considered and planned for.
Insights from government’s EU Exit preparations
Our latest report Learning for government from EU Exit preparations brings together our insights from how government has tackled the challenge of preparing for EU Exit to date. We draw on our reports to highlight areas of good practice, such as the initial work the Department for Exiting the European Union asked departments to do to identify all the areas where EU Exit would have an impact. This provided a solid basis for assessing the scale of the challenge across government as a whole and across individual departments, and for identifying where an issue required a coordinated response across government. We also highlight areas where government’s approach could have been improved, such as in the slow move from policy to addressing implementation challenges, and the lack of timely and clear communication from the centre to departments.
We saw that government had taken action to understand what was not working and to take steps to improve. The insights we set out are intended to help government identify and implement improvements on existing work at a faster pace.
These fall into four broad areas that encompass important aspects of any government programme.
- Plan for all possible scenarios, with robust contingency plans.
- Identify the scale, nature and complexity of the task at the outset.
- Recognise the opportunities and increased risks from working at speed or in new ways.
- Develop clear structures for oversight and decision-making.
- Draw on expertise in implementation early on, to expose delivery risks.
- Develop effective structures to facilitate cross-government working.
- Establish a culture of clear and timely communication across departments.
- Engage early with key stakeholders, and understand their role in delivering the outcome.
- Encourage strong financial management, for informed decision-making and accountability.
Why our insights are important now
While at the time EU Exit was a challenge with little historical precedent, the insights we have gathered continue to have relevance. Preparations for the end of the transition period in December 2020 are ongoing (we will be reporting on these in the near future), and the impact of the UK’s Exit from the EU will continue to be felt on the work of government as it plans the future policy direction of areas such as immigration, agriculture, consumer protection and the environment.
Government is now also responding to the demands of a global pandemic which similarly requires a fast-paced response, innovative policy solutions, coordinated action across government and effective, external transparency and communication. And, as we highlighted in our five-year strategy, there are significant challenges for government ahead such as the transition to a net zero carbon economy, how demographic changes will continue to drive higher demand for public services, and how technological innovation will continue to reshape whole industries and public service delivery.
Back in 2017, we raised the need for government to prioritise what it could manage from its existing business-as-usual work alongside the demands of EU Exit. The government of today is no less busy, and the Civil Service itself is undergoing reform, with a new Cabinet Secretary and a new Chief Operating Officer at the helm. It is more important than ever that government learns from its recent experiences to ensure that it is as well-placed as it can be to manage the scale and complexity of the many challenges it faces.
Authors: Leena Mathew and Sarah Pearcey
Leena has responsibility for the NAO’s work on EU Exit and the border, and Sarah has led many EU Exit reports. They both worked on the NAO’s report on the learning government can draw from its EU Exit preparations.
Tagged with: Brexit
Posted on August 27, 2020 by Daniel Lambauer
COVID-19 continues to have a significant impact on the work of the National Audit Office, including our international work. Our team has continued to complete our international assignments successfully from the UK. We are actively exchanging experiences with other national audit agencies (also known as Supreme Audit Institutions – SAIs) on how to audit governments’ response to COVID-19 across the world, which provides us with valuable insight to strengthen our UK audit response.
The objectives of our international work are three-fold – we want to use international good practice to improve our UK focused work; we want to enhance the UK’s reputation by showcasing the quality of our public audit work worldwide; and we want to protect the UK taxpayer’s interests overseas by bidding to audit international organisations that receive UK funds, or by providing training to SAIs in countries that receive UK aid.
The NAO published its new strategy in June , so we were already planning how international activities could best contribute to our mission as the UK’s independent public spending watchdog. As Gareth Davies, the Comptroller and Auditor General, recently described, the NAO is undertaking a substantial audit programme on the government’s response to the COVID-19 pandemic. From the start, we were clear that UK audit coverage of the pandemic would be stronger by including global insights. People who read our reports see media coverage contrasting how other countries are responding and want an objective comparison between these responses. We can also learn lessons from other countries that can help the UK to better respond to this pandemic.
The NAO has a global reputation as a leading SAI, but we strongly believe there is always more we can learn from others on how best to provide a modern public audit service. Having good links with other SAIs also allows us to make better use of international comparisons in our reports – particularly in key areas such as delivering major infrastructure and defence projects – providing Parliament with insights on how other governments approach the same challenges facing our own.
As we contacted other SAIs it was obvious our situation was not unique. As well as considering the specific nature and unprecedented scale of the pandemic, SAIs around the world were all thinking about how to audit the same thing at the same time. SAIs have a unique cross-government perspective and an independent evidence base that many other commentators do not. We are expected to audit government’s use of public funds but there is a lot to consider. How do we time our interventions so as not to compromise the emergency response, whilst ensuring full accountability for the use of public money. What type of audit product is useful at different stages of the response?
NAO staff have shared insights on the pandemic in webinars organised by the International Organisation of Supreme Audit Institutions (INTOSAI ). We also established a new European Organisation of Supreme Audit Institutions (EUROSAI) Project Group on Auditing the response to the COVID-19 pandemic, which we are co-chairing with SAI Finland. As part of this, SAIs from 30 European countries have agreed to coordinate and communicate COVID-19 work; share audit approaches, information and outputs; and scope content for any future lessons learned reports. In June and July we co-hosted eight online meetings where 30 SAIs set out the impact of COVID-19, their audit response, and what they wanted to share and learn. We will use these insights, and further information exchanges on specific topics, in our own reports to Parliament.
This isn’t easy, and each SAI has a different answer as every country’s context is different. However, what has quickly become apparent is that government responses around the world are similar: preparing, responding and then handling the recovery and long-term impacts of the pandemic. This involves significant expenditure in healthcare, wider emergency response measures and supporting individuals, businesses and the economy.
Since the pandemic started, operating internationally hasn’t been straightforward. In March we recalled our people working overseas at short notice and like everyone we have had to adapt to working online. Our work has continued successfully. We completed our international audits, including the World Intellectual Property Organisation, the Organisation for the Prohibition of Chemical Weapons, and the Pan American Health Organisation. The international bodies we audit will have to consider new ways of working in response to the pandemic. To support them in this, our audit reports provided an important and independent perspective on the decisions they have to make on future strategic planning, more efficient and effective processes, and on making better use of the resources provided to them. We will also share experiences with other SAIs auditing UN organisations at the UN Panel of External Auditors this November.
Learning how public servants around the world have responded to the pandemic, many in countries with fewer resources than the UK, provides valuable comparisons. It makes us even more grateful to those on the front line of the UK’s response.
The aim of everything we do at the NAO is to help Parliament hold government to account for its use of public money and to help improve public services. The nature of public audit means we first need to look backwards to understand what happened, so we can then look forwards to recommend how to make things better. By working internationally, the NAO is also looking outwards to help us understand how others are meeting the same challenges the UK faces.
Authors: Daniel Lambauer, Kevin Summersgill, Damian Brewitt
Daniel Lambauer joined the NAO in 2009 as a performance measurement expert and helped to establish our local government value for money (performance audit) team. He is the Executive Director with responsibility for Strategy and Resources. As part of his portfolio, he oversees our international work at executive and Board level and has represented the NAO internationally at a range of international congresses. Before joining the NAO, Daniel worked in a range of sectors in several countries, including academia, management consultancy and the civil service.
Kevin Summersgill joined the NAO’s value for money (performance audit) specialism in 2005. He has audited a wide range of public policy areas and as our Head of International Relations and Technical Cooperation routinely represents the NAO internationally. A specialist in continuous improvement and management systems thinking, he has advised governments and United Nations organisations around the world on how to increase effectiveness, transparency and accountability.
Damian Brewitt is a Chartered Accountant and Director of our international audit portfolio, leading financial and performance audit teams across the NAO’s portfolio of international external audit engagements. He has spent 16 of his 28 years of public sector external audit undertaking financial and performance audit of organisations across the international system. He has specific expertise in IPSAS, international governance and risk management. He supports the NAO and our stakeholders in providing sector insight, leveraged from his experience and the work of our teams across our international client portfolio. He has chaired the Technical Group of the UN Panel of External Auditors for the last two years.
Tagged with: COVID-19
Posted on July 23, 2020 by Gareth Davies
I last posted to this blog in late April as the country was in the teeth of the COVID-19 pandemic, explaining how we were maintaining our operations and adjusting our work programme in the light of the government response to the virus.
Now, in late July, most of the UK is gradually emerging from lockdown following a devastating period for many families, an enormous effort by health and care workers and many others in crucial roles and an unprecedented set of government spending interventions to mitigate some of the economic effects of the crisis.
Like every organisation, the NAO has had to adapt to the new working environment, but throughout this period we have continued to deliver our core programmes of work: the audit of the 2019/20 accounts of more than 400 government departments and public bodies and a substantial programme of value for money reviews and investigations into important spending programmes.
A substantial proportion of our effort has been redirected to the audit of government’s response to COVID-19. We have published reports on the scale and nature of the public spending commitments and on readying the NHS and social care for the pandemic. We have a further seven studies already underway to begin publishing in the Autumn, and more in the planning phases. We’ve developed our COVID-19 hub for people to follow our work on this.
Looking back, at the point that Parliament rose for its summer recess, we had completed the audits of 171 departments and public bodies, including some of our larger audits such as the Department for Work and Pensions and the Home Office. Our work on the remainder continues over the summer and those audited accounts will be laid in Parliament from its resumption in September. I’m grateful to both my audit teams and the finance teams of our audited bodies for their determined approach to obtaining the audit evidence we need when working remotely, and for addressing the financial reporting challenges posed by the pandemic including volatile valuations and going concern risks.
We have continued to complete and publish value for money reports on vital areas of public interest, including universal credit, the MoD’s aircraft carrier programme, asylum accommodation and support and digital transformation in the NHS. Our factual investigations have included reports on progress in removing dangerous cladding from residential blocks, the selection of towns to bid for money from the £3.6bn Towns Fund and government’s response to the collapse of Thomas Cook. This work has supported sessions of the Public Accounts Committee which has been meeting virtually throughout.
Our Overview of the UK government’s response to the COVID-19 pandemic brought together in one place all of the main activities, costs and funding undertaken by government in the initial months and laid the foundation for us to build our programme of more in depth studies. We also looked further into the action the Department of Health & Social Care and other bodies undertook during March and April to ready the NHS and adult social care for a rapid increase in the number of infected people.
Looking forward, we have a broad and varied programme of work on COVID-19 to come once Parliament reconvenes in the Autumn. We have several reports planned on the important issue of government procurement. They include an audit of government buying in the pandemic, as well as specific studies into the efforts to increase the number of ventilators available to the NHS and on supplying the NHS and adult social care sector with PPE.
We will also examine measures aimed at protecting businesses and individuals from the economic impact of the coronavirus pandemic. This includes reports into the Bounce Back Loan Scheme and the Coronavirus Job Retention Scheme. Other studies will look into the government’s work to protect and support the vulnerable during lockdown and the Free School Meals voucher scheme. Our website has full details of our work in progress related to COVID-19, and we will continue to add further reviews to this list as they are approved.
In all of this work, we will examine how government adapted its approach to reflect the need for urgency in the first phase of the pandemic, and how it is managing the attendant risks to value for money and probity in public spending. Our reports will be published, laid in Parliament and be available to the Public Accounts Committee for its programme of inquiries in the normal way.
We will continue to respond to the risks to public money posed by this unprecedented time for the country to provide the assurance required by Parliament and the public and to draw out the lessons for future phases of this pandemic and future emergencies.
Tagged with: COVID-19
Posted on June 25, 2020 by Abdool Kara
In January of this year, I took on senior level responsibility for leading the NAO’s people agenda as we reviewed and reset our strategy. I quickly recognised that that this was to be a significant undertaking for the organisation, with much work to do if we are to become the exemplar employer that we aspire to be.
What I did not and could not know at the time was how our world would be transformed over the course of a few months. Firstly, the arrival of the COVID-19 pandemic, having to deal with its impact on our people, as well as, in the fullness of time, covering it in our programme of work. Secondly, the consequences of the killing of George Floyd, highlighting the injustices faced by black people not only in the USA but here in the UK as well.
I have long been interested in the issues of racism, from witnessing the demonstrations and riots of the 1980s in my teens. My heritage on my father’s side, whilst not drawn directly from slavery, is the result of the importation of indentured ‘coolie’ labour brought from India to replace the slaves who had previously worked the sugar cane fields of Mauritius. And I have experienced both direct and indirect prejudice, and yes racism, on many occasions throughout my career.
On the one hand, we have been here before, witnessing tragic events which shine a spotlight on the devastating effects of racism on our communities and reminding us of our country’s history of slavery and colonialism. What feels different this time is the acceptance that racism exists throughout our society, rather than the issue being swept under the carpet. It gives me hope – less ‘it doesn’t happen here’ and more ‘this is an important conversation for the nation’. Change may, finally, be upon us in a new wave of civil rights that feels essential, immediate and in step with the times we live in.
But change is not an external force – all change starts with people taking personal responsibility for their actions, and organisations putting their own houses in order. I and my senior colleagues are committed to accelerating the NAO’s progress in becoming a genuinely inclusive organisation which embraces and values difference and supports our people to bring the full breadth of their thinking, experience and skills to work.
At the forefront of our new strategy is our people, it is our commitment to bringing about genuine change in the lived experience of colleagues from all backgrounds, but especially our black and minority ethnic colleagues who have not always been well served by our ability to embrace their talents and enhance their opportunities.
As an organisation whose very purpose is to hold government and public bodies to account for their progress in improving public services for all communities, we recognise that we cannot speak with authority if our own record on diversity and inclusion is open to question. Therefore, our aspiration is to be an exemplar employer in all respects, but with a focus on key areas, such as BAME representation at senior levels in our business, where we have the most progress to make.
We also need to ensure that our work, such as our Windrush report, continues to highlight inequalities in the way public services are delivered and secures deep and sustained improvement in the outcomes for the most vulnerable groups in society. The vast statistical differences in the impact of COVID-19 on different ethnicities and socio-economic groups is raising important questions that need examining.
We are evidence-led when assessing the services of others and must be equally evidence-led in our assessment of ourselves. Doing so will help us to openly acknowledge the challenges we face in achieving sustained improvement in our approach to equality, diversity and inclusion for all our people. As an organisation we are particularly dissatisfied with the progression rates for BAME colleagues in recent years. We need to make change quickly and remove barriers to progression wherever we find them.
This is a serious commitment which requires deep and permanent change and we are putting in place a range of measures to realise our ambitions. We are building on recent progress in improving the diversity of our graduate intake by applying rigorous requirements for diverse shortlists in all our recruitment and promotion campaigns. We have also launched our first diversity mentoring scheme which is giving our senior leaders much richer insight into the experiences of female, black, disabled, LGBT+ and underprivileged colleagues. Also, each member of our executive team takes an active role in sponsoring one of our diversity networks so that their voice is heard at the most senior levels of our business.
We will shortly be publishing our 2019-20 Diversity and Inclusion Annual Report which provides an honest and transparent assessment of where we see that tangible progress has been made, but also sets out clearly where we have more to do.
The recent launch of our new five-year strategy highlights our commitment of having a diverse, inclusive and healthy workforce as a key enabler in delivering our strategic priorities. We will be consulting on our new Diversity and Inclusion Strategy for 2021-23, which will support us in realising our ambition of being an exemplar employer, where all colleagues can realise their full potential irrespective of background.
I am conscious that statements of aspiration can often sound hollow without clear actions and demonstrable improvements, but as senior leader for the NAO’s people agenda I am fully committed to seeing genuine, sustained and beneficial change for our people and know that I have the support of my colleagues to make this happen.
Reshaping our culture and bringing to life our new values, which include respect and inclusion at their heart, will be key to this. So too will practical actions to revise our HR and other policies and practices which we know can reflect or enable unconscious bias.
So, there is plenty for us to get on with. But my personal passion and commitment to the equalities agenda, born from my own heritage and life experiences, both professional and personal, mean that I won’t be satisfied until the NAO is a fantastic place to work for all.
Posted on April 22, 2020 by Gareth Davies
With Parliament returning yesterday, I wanted to take the opportunity to reflect on the last few extraordinary weeks and set out what it means for the National Audit Office and its work.
Firstly though, on behalf of the whole NAO, I would like to pay tribute to everyone who is working so hard to see our country through this crisis. That of course includes our courageous health and social care workers and others on the front line of the response, but also all the public servants behind the scenes at the national and local level keeping our country going. As the organisation responsible for scrutinising so many of these public bodies, we have a privileged insight into how vital they are to everyone’s lives every day – and even more so at a time like this.
To all of the public servants rising to this unprecedented challenge, thank you.
Like many other organisations, the NAO has been home-based for a month now. In infrastructure terms, the NAO was well-prepared for homeworking as our systems are designed to support secure remote auditing. We are working hard to support our staff as they grapple with the practical and wellbeing challenges of the current situation. We are of course not the only ones. And as an organisation that supports Parliament, that has been especially brought home to us as we see the House of Commons resume business in a manner we have never seen before.
As Parliament adapts, so too are we in order to ensure that we can help it to hold government to account. The response to the global pandemic will have implications for many years for public spending and public service delivery. It is too early to tell exactly what the impact will be, but it will be profound.
What is already clear is that MPs, and the public that they represent, will expect us to carry out a substantial programme of work on the COVID-19 response so we can learn for the future. This will include looking at government spending on the direct health response as well as the wider emergency response. We will also look at the spending on the measures to protect businesses and individuals from the economic impact.
It will take us time to develop and produce our work, more importantly it will take time for the public sector to be in a place where it can learn from our findings. Our challenge is to try and provide the appropriate level of evidence-based reporting to support accountability and provide insight at the most suitable time. We must not get in the way of public servants working hard to save lives, but we must also ensure that our reporting is sufficiently prompt to support proper accountability for public money.
We have decided to begin with a factual summary of the significant government spending commitments and programmes relating to COVID-19 which we hope to publish next month. We will use this to identify a risk-based series of evaluative studies where we think there is most to learn.
What is also important to Parliament is that we do not lose sight of the wider picture. There are many other challenges facing the UK including EU Exit; progress in meeting government’s net zero carbon emissions target; major infrastructure projects and the financial sustainability of key public services. Our work programme will have to balance this with the demands of COVID-19. That is why we will be continuing to publish reports already in train. We are also working hard to meet our key statutory duty to audit the accounts of over 450 public bodies.
My colleagues and I are committed to providing Parliament and the public with the evidence they need to understand how public money has been used in tackling this crisis. We will also help ensure that the appropriate lessons are learned for the future.
Tagged with: COVID-19