The challenges in implementing digital change

Cross-government

REPORT

by the Comptroller
and Auditor General

Session 2021-22

21 July 2021

HC 575

The National Audit Office (NAO) scrutinises public spending for Parliament and is independent of government and the civil service. We help Parliament hold government to account and we use our insights to help people who manage and govern public bodies improve public services.

The Comptroller and Auditor General (C&AG), Gareth Davies, is an Officer of the House of Commons and leads the NAO. We audit the financial accounts of departments and other public bodies. We also examine and report on the value for money of how public money has been spent.

In 2020, the NAO’s work led to a positive financial impact through reduced costs, improved service delivery, or other benefits to citizens, of £926 million.

We are the UK’s independent public spending watchdog.

We support Parliament in holding government to account and we help improve public services through our high-quality audits.

The challenges in implementing digital change

Ordered by the House of Commons
to be printed on 19 July 2021

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Gareth Davies
Comptroller and Auditor General
National Audit Office

13 July 2021

Lessons learned reports

Our lessons learned reports bring together what we know on important recurring issues to make it easier for others to understand and apply the lessons from our work.

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.org.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

009122 07/21 NAO

This report can be found on the National Audit Office website at www.nao.org.uk

If you need a version of this report in an alternative format for accessibility reasons, or any of the figures in a different format, contact the NAO at enquiries@nao.org.uk

The National Audit Office study team consisted of:

John Bell, Caroline Harper, Daniel Kintoff and Jonathan Pownall, under the direction of Yvonne Gallagher and Tom McDonald

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

020 7798 7400

www.nao.org.uk

@NAOorguk

Summary

1 Our way of life is now increasingly digital, and technology is almost always a feature of large-scale government business change programmes. Current and future public services are dominated by digital change. This is clear in much of government’s thinking about how to build back public services following the COVID-19 pandemic, as well as in longer-term policies and strategies. In addition, the public increasingly expects the government to make effective use of technology, so public bodies have little choice but to deliver high-quality digital services.

2 When large digital business change programmes run into difficulty, the technology solution is often cast as the primary reason for failure. There is rarely a single, isolated reason which causes critical programmes to fail. Many of these programmes face intrinsic business challenges as well as technical challenges. Our findings point to a range of problems, including: shifting business requirements; over-optimism; supplier performance; and lack of capability at the senior and operational level. Only a small proportion of permanent secretaries and other senior officials have first-hand experience of digital business change and as a result many lack sufficient understanding of the business, technical and delivery risks for which they are responsible. This means that many of the problems stem from the inability of senior decision-makers to engage effectively with the difficult decisions required to implement technology-enabled business change.

3 Pressures on public finances mean there is an urgent imperative for those designing and delivering digital business change programmes to learn from the mistakes and experiences of their predecessors. If they do not do so, these programmes will continue to fail. This report sets out the lessons for the centre of government and departments to learn from the experience of implementing digital change. It will be particularly useful for senior decision-makers who may not have direct technical experience or who have not yet grasped the scale of the challenge. We have focused on operational business change programmes with a significant technical component, by which we mean programmes which deliver a service that users interact with electronically. In pulling together these lessons, we have reviewed our published reports and interviewed senior digital leaders across government and the private sector. We have assessed good practice and consulted with experts from industry, academia and think tanks to highlight the nature of the challenges and understand why government has found it hard to apply the lessons of experience. Our scope and evidence base are set out in Appendix One.

Findings: lessons for government

4 Having consulted widely across government and its commercial suppliers, we found a high level of agreement among digital leaders regarding the challenges they face in delivering digital business change programmes. These programmes share characteristics and challenges with all major programmes, but added complexities make the difficulties even more acute and have often been poorly understood. We hope that our report will add further impetus to the work being carried out in government and support practical improvements in digital change programmes.

5 We have identified lessons for government digital programmes in six categories, which are essential to get right at the outset. These are critical in any major project or programme, but in digital change the initial and pre-deployment stages are even more pivotal than usual because of the increased uncertainties which typically characterise them, including ‘unknown unknown’ risks. If the delivery implications are poorly understood the level of ambition can be unrealistic from the outset. Successful delivery of digital business change programmes requires organisations to equip non-technical leaders with the right skills, and design suitable approval and governance frameworks.

Concluding remarks

6 Initiating digital change involves taking a difficult set of decisions about risk and opportunity, but these decisions often do not reflect the reality of the legacy environment and do not fit comfortably into government’s standard mechanisms for approval, procurement, funding and assurance. We found that digital leaders understand these issues well and bring much needed expertise to the public sector, but they often struggle to get the attention, understanding and support they need from senior decision-makers.

7 Despite 25 years of government strategies and countless attempts to deliver digital business change successfully, our reports show a consistent pattern of underperformance. This underperformance can often be the result of programmes not being sufficiently thought through before key decisions on technology solutions are made. This means that there is a gap between what government intends to achieve and what it delivers to citizens and service users, which wastes taxpayers’ money and delays improvements in public services. If government is to improve its track record in delivering digital business change, it must learn the hard-won lessons of experience and equip its leaders to act effectively.

Recommendations: Actions for government

8 We do not underestimate the challenge involved in digital change, particularly given government’s vast legacy IT estate and the need for government to deliver services where there is no counterpart model in the private sector from which government can draw. But there is widespread support from stakeholders for the centre of government to learn from the lessons we have identified in this report and make the required changes. The new Central Digital and Data Office, along with the Government Digital Service and the Cabinet Office, should work to provide clear leadership for this agenda, in particular:

a revise existing training programmes to better equip and train all decision-makers with responsibility for digital transformation programmes. This should include education on legacy systems, the importance of data and the risks of ‘build before buy’ and of opting for unproven technology;

b work with HM Treasury to review existing business case funding and approval processes for digital programmes to: remove the incentives to state with full confidence those things which are still unknown; ensure that uncertainties associated with assumptions are made clear, together with when these uncertainties will be better understood; understand what the final product should look like, and the path to get there; be clear on what risks represent ‘unknown unknowns’; and ensure professional independent technical assurance mechanisms are in place, to support those responsible for approving programmes; and

c disseminate and apply lessons learned from the successes and failures of the past and seek to understand why digital strategies have made poor progress.

9 Individual departments and public bodies should:

d carry out proper evaluation and assurance in the early stages of a digital programme to understand its complexity and scope, assess how realistic the chance of success is and reflect this in the programme approach;

e ensure senior digital, data and technology colleagues have wider influence on all change programmes with digital components, by providing strategic direction and oversight at key decision points in the process;

f strengthen their intelligent client function for digital change including identifying and developing key requirements before tenders and bid processes commence and taking the lead on supplier engagement;

g maximise the chances of effective digital delivery by ensuring that business leaders have sufficient skills, commitment and time to engage in all aspects of governance and decision-making;

h produce departmental strategies and plans for how to manage the legacy IT estate so that maintenance, support and decommissioning are systematically addressed and required funding is ringfenced; and

i ensure that agile principles and approaches are appropriately applied within the context of significant business programme change, for example by developing interim and target operating models, and having appropriate business and technical architecture in place.

Part One

Introduction

1.1 This part introduces the rationale for this ‘lessons learned’ report. In it we describe the challenge of delivering digital change and government’s response to that challenge.

The challenge of delivering digital change

1.2 Digital transformation is one of the most influential drivers of organisational change today, challenging public sector organisations to come up with new ways of serving their customers and the public. Our way of life is increasingly digital, and technology is now widespread in large-scale government change programmes. This is clear in much of government’s thinking about how to ‘build back better’ following the COVID-19 pandemic and the planned use of technology in policy documents such as Global Britain in a Competitive Age: the Integrated Review of Security, Defence, Development and Foreign Policy and Build Back Better: our plan for growth.1 These plans suggest that the need to deliver successful digital change will be even more crucial in future.

1.3 As a result, those working to develop public services need to understand how to manage digital change, by which we mean how government brings together data, processes, technology and people to deliver high-quality and effective services to, for and with citizens. This requirement creates a challenge for the skills and capabilities of everyone involved, including senior decision-makers, senior responsible owners, delivery teams, digital leaders, those providing assurance and suppliers and delivery partners.

1.4 However, government is not a ‘green field’ site and added complexity arises from the need to transform or change existing services. The difficulties lie in understanding and determining how to make changes to these, often ageing, systems, known as the ‘legacy environment’, and what it means to build new systems on top of them. This type of change requires a level of analysis before making decisions that does not fit comfortably into government’s standard mechanisms for approval, procurement, funding and assurance.

The nature of the problem

1.5 Government has many major projects and programmes which include some form of digital transformation. The government’s Major Projects Portfolio has 125 projects worth £448 billion, many of which have digital elements. We regularly report on major programmes, and this has included some digital change programmes that did not achieve all their intended benefits. Some digital change programmes deliver successfully but they may not have the typical characteristics of the programmes we are considering, for example because they do not have to deal with the legacy environment. Figure 1 shows some of the digital projects which have not met their planned timetables. Figure 2 sets out some of the financial costs and lost benefits arising as a result of poor performance.

The government’s response

1.6 Over the past 25 years, the government has published a series of strategies to improve its digital performance (Figure 3). But these strategies have failed to change the pattern of performance for individual projects and programmes.

1.7 Responsibility for improving government’s overall performance rests with both the centre of government (Figure 4) – setting strategic direction and supporting capability development – and departments, who have day-to-day responsibility for delivery.

1.8 We found that digital leaders understand the challenges set out above well and bring much needed expertise to the public sector, but they often struggle to get the attention, understanding and support they need from senior decision-makers. The digital leaders we spoke to revealed their frustration with the status quo and with their inability to influence change programmes with digital components. Only a small proportion of permanent secretaries and other senior officials have first-hand experience of digital change and as a result many lack sufficient understanding of the technical and delivery risks for which they are responsible. Outside government, there is an increasing focus on equipping senior executives with the understanding they need to provide effective leadership to programmes with a strong technical element.

1.9 The centre of government remains committed to improving performance in various ways. The Central Digital and Data Office within the Cabinet Office, created in 2021, together with the Government Digital Service, is responsible for addressing many of the issues we have identified. In the remainder of the report we discuss the areas that can be improved by the centre and departments. In some places we include specific examples from our published work, which reflect our findings at the time of the original report and not the current status of each programme.

1 Cabinet Office, Global Britain in a Competitive Age: the Integrated Review of Security, Defence, Development and Foreign Policy, CP 403, March 2021 and HM Treasury, Build Back Better: our plan for growth, CP 401, March 2021.

Part Two

Initiating for success

2.1 Digital programmes and projects can deliver beneficial change but require a great deal of up-front thinking and planning to succeed. There needs to be a thorough understanding of the business problem to be solved and an overall plan and design for how the business can move from its current state to its changed state. Delivery planning should include a realistic timetable, engagement with end users, an aligned supply chain and a shared goal, recognised by all stakeholders.2

2.2 This part discusses some of the key challenges that the government faces when first setting up large digital programmes. Our evidence highlights three lessons that departments have found particularly challenging. These are:

understanding aims, ambition and risk;

engaging commercial partners; and

approach to legacy systems and data.

Understanding aims, ambition and risk

2.3 The government has significant ambitions for digital business change, and this has led to rapid, large-scale business transformation attempts, often using untested technology. This is a complex and risky approach and the government needs to assess from the outset if these programmes are realistic. Digital leaders highlighted to us the need for the government to take a longer-term view of transformation, delivering more manageable levels of change in incremental steps and managing the risk of ‘scope creep’ – when a programme or project’s scope extends beyond what was originally agreed.

2.4 Government has begun to use more pragmatic and good-practice approaches to digital innovation, by looking for opportunities to test out ideas on a small scale. This is the approach taken by the £20 million GovTech Catalyst fund, created in 2018 by the Government Digital Service, which tries to solve public sector problems using innovative digital technology via a two-stage approach. During the first stage, the public sector proposes complex ‘challenges’ and will fund up to five suppliers to work on the challenge for three months. At the end of the first stage a panel of government experts evaluate progress. If the results look promising, the government grants additional funding to one or two of the suppliers to work on their solutions for a further 12 months, with the aim of developing a working product that addresses the challenge.

2.5 Although parts of government experiment with smaller-scale innovation, our evidence shows that some digital programmes have tried to implement untested technologies immediately on a larger scale. The Ministry of Justice’s programme to develop ankle-tags for monitoring offenders (Figure 5) showed how difficult this is to achieve. Instead, the government can benefit from being a fast follower of innovation in the private sector by making better use, where possible, of proven, off-the-shelf solutions.

2.6 If departments do not think sufficiently up-front about the complexities of a digital programme, they risk failing to meet the business need or needing to de-scope or abandon programmes as understanding grows. Digital leaders told us programme teams often rush to a solution because of pressure to deliver quickly, and do not spend enough time understanding the business need, the existing system or what business improvement the programme team wants to deliver. Figure 6 shows how in developing the Verify programme, the team did not fully understand the end-to-end needs of users or the changes needed to the business environment.

2.7 Delivering digital change is challenging and departments have often taken longer to deliver programmes than originally scheduled, due to insufficient or unrealistic scoping and planning (Figure 1). Our report on the Emergency Services Network (Figure 7) found that despite the high inherent risks, the Home Office set an over-ambitious timeline for delivery, with no contingency, and fell significantly behind schedule. We found this problem is widespread, as international public sector digital programmes also often overrun and exceed their budget.

2.8 A range of digital leaders told us that senior decision-makers and operational and policy teams lack an understanding of digital change. This lack of understanding is one potential reason why digital programmes appear to under-deliver against their business cases. Senior leaders need to be able to make credible and informed decisions about the scope and timescales of digital change and provide large digital change programmes with active leadership. The Infrastructure and Projects Authority (IPA) runs a ministerial training course on how to sponsor major projects effectively but has not been commissioned to provide specific training for digital projects. In May the Government Digital Service created a new Digital Leaders course for senior civil servants. Digital, data and technology leaders, who do understand these issues, need to be able to influence programme decisions to give implementing teams the best chance of success.

Assuring changes to programme scope

2.9 It is difficult to define the scope and costs of large digital programmes until programme teams perform detailed exploratory work and build their understanding. Programmes need business cases early to secure funding, and digital leaders perceive there is an incentive to show a high return on investment and give a false impression of certainty. The Green Book: Central government guidance on appraisal and evaluation produced by HM Treasury requires departments to make adjustments to deal with the challenge of uncertainty.3 Despite this guidance, digital leaders told us the current business case process does not work well for digital programmes in practice because it locks in assumptions too early, which can lead to scope creep.

2.10 The difficulty of getting detailed specifications right early means that the full extent of a programme’s requirements can emerge over time. This can cause scope creep, and programme teams need to be able to respond. It is important to consider if deviations from the initial business case are a necessary response to new information, rather than a failure of execution. However, our published reports show programme teams can be slow to effectively acknowledge and address underlying scope challenges.

Engaging commercial partners

2.11 Commercial partners are an important source of expertise and bring valuable experience of working with complexity and scale.4 It is hard to define and write detailed specifications for complex digital programmes, yet our evidence shows departments have not spent enough time exploring requirements with commercial partners at an early-enough stage. Early discussions with a range of suppliers before settling on a solution can help departments to de-risk programmes and explore what is possible. Lengthening that early engagement can improve collaboration, facilitate innovation and improve quality requirements. Our experience has shown, however, that departments ask suppliers to commit to contracts without a reasonable understanding of what to deliver. NHS England’s management of the primary care support services contract (Figure 8) is an example of the problems that can arise from failure to understand what is being contracted.

2.12 Programmes and requirements change over time, but we found that departments do not incorporate sufficient flexibility into their contracts to allow for change and uncertainty. Suppliers told us that it is difficult to introduce assumptions or flexibility into initial contracts, and our evidence suggests departments do not typically revisit, renegotiate and update contracts, except in the event of failure. Government may not get the best possible outcome if the requirements delivered by suppliers are not the right ones. This can also result in financial losses for suppliers if the government holds them accountable for unrealistic contractual obligations.

2.13 Our investigation into the British Army’s Recruiting Partnering Project (Figure 9) highlighted that inflexible contracts can result in poor outcomes for both the department and the supplier. The government is unlikely to get the quality of service it needs if suppliers need to minimise their losses. A more mutually beneficial arrangement for departments and suppliers would involve commercial negotiation that recognises that scope and requirements may change. This would avoid government’s tendency to over-specify and then fail to adapt to other emerging key dependencies.

2.14 An important part of being an intelligent client is cultivating open relationships with suppliers. Where there is a lapse in intelligent buying, this can lead to adversarial relationships and poor outcomes. To act as an intelligent client when contracting out digital change programmes, departments need individuals who can combine commercial and digital skills. The IPA’s Major Projects Leadership Academy teaches senior project leaders across government to work together with suppliers, as experience shows that complex projects benefit from a partnership model based on collaboration. Figure 10 shows how challenges can arise in the relationships between departments and suppliers if departments do not appreciate the commercial realities suppliers may face.

Approach to legacy systems and data

2.15 The government relies on legacy systems for many important services.5 Older legacy systems are often difficult and expensive to support, lacking in resilience, and can be vulnerable to cyber-attacks. If the underlying legacy systems fail, key government services, such as tax and benefit systems, will not run effectively. Changing legacy systems can be a complex and risky undertaking. The Financial Conduct Authority’s report Implementing Technology Change found that in the financial services sector, “firms with a lower proportion of legacy infrastructure and applications had a higher change success rate”.6

2.16 Legacy systems need a significant level of resource to make more regular, incremental changes over time. The legacy system environment exposes government to what is likely to be an uncertain but high level of financial risk from potential operational and cyber-related incidents. The government established the Legacy IT Programme to address this problem and it reported its findings internally in November 2020. The programme made recommendations on how the government can make progress in reducing its reliance on legacy systems. It is important that the government responds effectively to these recommendations. As part of the 2020 Spending Review HM Treasury prioritised investment in legacy IT and agreed funding for approximately £600 million to invest in modernising legacy systems. Although these are positive developments, maintenance of legacy systems is often one of the costs most likely to be cut or delayed, and business cases do not always include these maintenance costs. The size of the legacy environment means that additional investment and further work may be needed.

2.17 Failure to understand the complexity and dependencies associated with replacing legacy IT has undermined government’s attempts to move away from legacy systems. Making the transition from legacy systems to modern replacements is complex and difficult, especially if the legacy system has many dependencies. Replacement systems often need extended transition time, with dual running of the legacy IT alongside its replacement for years, until departments are confident in the new systems.

2.18 The government has at times found it hard to manage these transitions and complexities. For example, Figure 11 sets out that the Home Office has been attempting to upgrade its legacy border systems since 2003. Departments typically do not have a good understanding of their IT estate and its interdependencies, and legacy systems are often poorly understood because of their age. This can add to the time, risk and cost of the transition from legacy systems. Departmental IT strategies are an important way to help appraise and set priorities for replacing legacy IT. Departmental IT strategies and business cases could also better consider how small, iterative updates to departments’ IT landscape can keep the IT estate up to date and prevent new legacy issues building up over time. Until departments have a good understanding of their IT estate, it is difficult to manage the legacy IT issues at a whole-of-government level effectively.

2.19 Many departments are using cloud technology to modernise, but this only partly addresses some of the legacy IT challenges. As we note in our Guidance for audit committees on cloud services, the cloud can be a potentially cheaper and more secure place to hold systems and data. This is because cloud providers can use economies of scale and concentration of expertise to offer a level of security that would be economically or operationally difficult for many organisations to provide on their own.7 Some organisations may, however, lack the capacity and expertise to select the right services for their needs, implement them securely, and manage and optimise them effectively. Moving to the cloud can fix IT infrastructure, but some applications are too old to transfer to new infrastructure and it is not always clear where the risks lie and who will remediate these applications for transfer. This continues to build up ‘technical debt’.8 Implementation of cloud services is not a ‘once and done’ endeavour and simply moving legacy systems into the cloud without other improvements will not resolve all the complexity, costs and risks associated with legacy systems.

2.20 Despite a high-level acknowledgement that data are key assets, the government still has a poor appreciation of the state of the data in legacy systems and its impact on the transformation of operational services. Data issues include the data’s age, quality and consistency across different systems. Building a new system from scratch starts with the data requirement, data model and data architecture. Migration from legacy infrastructure is dependent on having these in place, but government transformation programmes and business cases often fail to explicitly address data at the start, and instead it becomes an area of concern and delay further into the project. In our report Challenges in using data across government we recommended that business cases should include an assessment of data requirements.9

2.21 Government also has an ambition to join up data but has not yet addressed the underlying barriers and constraints that make this such a difficult undertaking. In our report on the Challenges in using data across government, we said there are no shortcuts to resolving the issue of lack of data consistency and poor-quality data across government.10 Recent efforts have borne this out, such as creating the COVID-19 clinically extremely vulnerable list (Figure 12), with significant effort in combining data from different NHS and GP IT systems.

2 Prepared for the Association for Project Management (APM) by BMG Research, Factors in project success, November 2014.

3 HM Treasury, The Green Book: Central government guidance on appraisal and evaluation, December 2020.

4 By commercial partners we mean external suppliers of digital technology who have an ongoing relationship with government through extensive contractual involvement.

5 We define legacy systems as systems and applications that have been operationally embedded within a business function but have been overtaken by newer technologies or no longer meet changed business needs.

6 Financial Conduct Authority, Implementing technology change, February 2021. Available at:
www.fca.org.uk/publications/multi-firm-reviews/implementing-technology-change (accessed 15 July 2021).

7 National Audit Office, Guidance for audit committees on cloud services, April 2021.

8 Technical debt refers to the future costs that will be incurred in ensuring that a system continues to remain operational and fit for purpose.

9 Comptroller and Auditor General, Challenges in using data across government, Session 2017–2019, HC 2220, National Audit Office, June 2019.

10 See footnote 9.

Part Three

Setting up for effective delivery

3.1 Before digital change programmes and projects can move into the delivery stage, the right conditions for success need to be in place. What sets digital programmes apart from others is that they need a design framework within which to plan their transition activities from the existing environment to the new. It also needs the right capability and experience in its leaders to shape the change, and the technical skills to deliver that change effectively.

3.2 This part discusses some of the key areas that the government needs to get right when setting up for digital delivery. Our evidence highlights three topics that departments have found particularly challenging. These are:

using the right mix of capability;

choice of delivery method; and

effective funding mechanisms.

3.3 Digital leaders told us that departments need to have in place a target operating model, enterprise architecture, data model and a roadmap.11 The roadmap should show: the sequence of changes; the transitional states between changes; the output expected at key points in the programme; and should act as a communication tool to ensure all involved are well informed. Organisations particularly need to manage their enterprise architecture. This is vital to ensure that they can deploy people, processes, systems and data as effectively as possible in support of business objectives. In this way, organisations can avoid the operational complexity and decline in performance that can occur when too much change happens too quickly and incoherently.

Using the right mix of capability

3.4 Many departments face a large capacity gap for people with digital skills. This skills shortage is replicated globally, which makes this challenging to overcome. Specialists in scarce supply, such as digital architects, service designers and technical managers, earn significantly more money in the private sector than the government is willing to pay, so it can be hard for the civil service to attract and retain them at appropriate levels. Figure 13 shows that in 2015, digital leaders in government viewed civil service salaries, recruitment processes and external market conditions as the primary barriers to recruitment and retention. In the interviews we carried out for this report, digital leaders continued to raise these as significant barriers.

3.5 Government needs to set out clearly what skills it wants to develop and retain, and what skills are more efficient to contract out. Some departments have built in-house teams but supplemented them with suitable supplier capability. Chief digital and information officers told us they experienced better results by maintaining core capabilities within their departments. For example, the former Department for International Development (DFID) developed an aid management platform effectively that delivered the intended benefits.12 DFID’s in-house capability meant it avoided the need to rely on contractors.

3.6 If the government is to strengthen its digital profession, it must quickly expand the pool of people with digital skills. Our report on the Common Agricultural Policy Delivery programme provides an example of how the government has not always had the internal skills and capacity needed to deliver the scale and complexity of its ambitious digital programmes. The government is seeking to expand its skills capacity through the No.10 Innovation Fellowship Programme, jointly run by 10 Downing Street and the Central Digital and Data Office. The programme aims to bring senior talent from the digital and technology sector into the civil service for one year to work on projects that can be delivered in that time.

3.7 Political announcements, policy design and programme teams’ ability to deliver do not always fully align. Many interviewees emphasised the need for closer working between policy, operational and technical colleagues, to give policies a better chance of successful implementation. Where programmes are not going well, departmental officials need a safe space to hold early and honest discussions with ministers. Teams of experts and practitioners need to be able to challenge unrealistic goals and prevent programmes from becoming too big to deliver, but too big to fail.

3.8 Technology expertise is often under-represented at the permanent secretary and director general level compared with other types of expertise, such as policy and finance. This makes it difficult for departmental leadership to comprehend the complexities of legacy systems, data and dependencies.

Choice of delivery method

3.9 Delivery approaches to digital change vary but the issues set out in Parts One and Two will negatively impact both agile and traditional ‘waterfall’ approaches.13 Regardless of the delivery approach, digital change needs proper programme management, with risk management and assurance activities built in. Success or failure lies more often with the skills and experience of the programme team and board than the delivery method.

3.10 Agile methods (Figure 14) have become the default choice for delivery of government digital programmes but are not the answer to all programme delivery challenges.14 Agile methods cannot solve the early stage issues that we have highlighted and can exacerbate problems when the complexity of the programme is not sufficiently understood. Nor can simple iterative approaches provide practical solutions in cases such as where architectural foundations are complex, missing or insufficiently developed, or back-office integration is not achievable. In our report Early review of the new farming programme, we said, “designing and developing technology solutions ahead of key business decisions may lead to … the IT solution costing more, taking longer and creating a sub-optimal outcome based on an incomplete architecture and design, leading to integration issues”.15

3.11 Agile is an excellent approach when used appropriately. When programme teams get agile right, they: target specific software deliverables using the right expertise; have a small budget and short duration; consider the user perspective; and deliver in a controlled and managed way. However, Figure 15 highlights the difficulties in introducing agile methods into challenging programmes such as Universal Credit. Where policies need to be agreed up-front, the business need and the change destination must be clear and use of the legacy environment and other service dependencies needs thorough analysis.

3.12 Some departments do not have appropriate controls in place when using agile methods. Digital leaders told us that programme teams often neglect long-term planning and programme management because of the flexibility offered by agile methods. Agile methods can be harder than waterfall methods to manage, still need detailed planning, and can sometimes make it harder to see what progress is really being made. Programme leaders need to bring together multiple teams, sprints and dependencies to deliver value through each product release and monitor progress against budget, timetable and scope. For example, our report Early review of the Common Agricultural Policy Delivery Programme (Figure 16) found that inconsistent and incomplete management information and assurance prevented effective monitoring and risk management. Without the necessary planning and programme management, departments will not make agile work at scale or intervene early if progress is not on track.

Effective funding mechanisms

3.13 Digital leaders told us that the government’s largely annual funding model hinders departments from building up strategic capabilities over time and makes it difficult to maintain them when they enter live service. Programmes often last more than five years, but the government’s Spending Review cycle varies in length and the funding model is largely annual. This makes long-term planning harder because there remain concerns that funding for digital programmes could be reallocated elsewhere. For some digital change, such as the transition from legacy IT, the government needs to prepare for multi-year investment over several Spending Review periods. Departments expressed a desire for more multi-year funding. However, they could help by setting out their priorities better in IT strategies. The Financial Conduct Authority’s report Implementing Technology Change found that financial service firms “that allocated a higher proportion of their technology budget to change experienced fewer change related incidents”.16

3.14 Many people in government see new digital ways of working and the management of data and technology as separate from the core business of government and reflect this in the distinction between capital and resource spend. Interviewees highlighted the comparative ease of getting capital funding to invest in non-current assets, as opposed to revenue funding, which is needed to maintain those assets or consume services and is under more pressure. This can lead to departments delivering digital services without the funding to maintain them. Digital requires a continued resource budget investment in business capability, and this need will increase as the government moves toward the cloud. The cloud uses the internet to access systems and data stored outside an organisation’s own premises, using software and technology ‘as a service’. While cloud pricing models reduce capital expenditure, they replace it with operational costs as use of cloud services is in effect ‘renting’ the infrastructure of the cloud provider. This requires a different approach to financial management across departments.

3.15 The government also needs to see technology as part of a service that involves people, processes and systems. Often there is an unmeasured ‘people cost’ to not modernising operational services. In the legacy landscape, large numbers of civil servants need to knit together data and find workaround solutions to compensate for the inadequacies of the legacy systems. Our Challenges in using data across government report highlighted that departments do not measure the extra people and process costs from managing inconsistent and poor-quality data, but informal estimates are that this can take 60% to 80% of some civil servants’ time.17 The government does not properly consider the strategic business case for replacing outdated technology. Digital programmes do not always save costs or have cashable benefits, especially in the short term. It can be hard to make the economic case for investment in legacy IT.

11 Enterprise architecture is the framework which shows how the different parts of an organisation, including the technology, fit together.

12 DFID merged with the Foreign & Commonwealth Office in September 2020 to form the Foreign, Commonwealth & Development Office.

13 Waterfall is a linear approach to project delivery consisting of sequential stages where each stage must be fully completed before moving to the next.

14 Agile methodology is an iterative and incremental approach to delivery frequently used in software development projects. The main alternative to agile delivery is the waterfall model. Both methods are valid, and their effectiveness will depend on the need of the project, the stage of the project lifecycle and the skills, experience and culture of the programme team.

15 Comptroller and Auditor General, Early review of the new farming programme, Session 2017–2019, HC 2221, National Audit Office, June 2019, paragraph 2.22.

16 Financial Conduct Authority, Implementing technology change, February 2021. Available at:
www.fca.org.uk/publications/multi-firm-reviews/implementing-technology-change (accessed 15 July 2021).

17 Comptroller and Auditor General, Challenges in using data across government, Session 2017–2019, HC 2220, National Audit Office, June 2019.

Appendix One

Our scope and evidence base

Scope

1 We prepared this report to provide insights from our previous work auditing large programmes that feature digital change, set out the challenges that we see and provide recommendations we think the government should focus on to help improve its performance.

2 For the purposes of this report, we define a digital change programme as a large-scale business change programme, which contains an aspect of technology or digital vital to achieving the business change. This definition includes programmes such as Crossrail, which is often considered an infrastructure programme, but will be the first fully digital railway to be built and operated in the UK. Bringing it into service requires Crossrail Ltd and its contractors to complete and integrate around 500,000 physical and digital assets, such as fire safety systems or platform screen doors. The work to bring the railway into service was made more complex by bespoke designs and a lack of standardisation throughout the programme, as well as needing to integrate three different signalling systems with trains.

3 Although many major programmes are not purely digital in nature, it is increasingly common for all types of major programmes to feature digitally enabled change. The report focuses on the challenges specific to operational digital change, primarily for citizen services, or which – although applicable to all types of programmes – digitally enabled change programmes experience more acutely.

4 In some places we include specific examples from our published work, which are illustrative examples and are not indicative of the overall performance of a specific department. Nor do all the programmes featured, or in government, suffer all the issues we identify.

Evidence base

5 We analysed evidence collected between January and April 2021 to reach our conclusions on the challenges of implementing digital change in government.

Interviews

6 We conducted around 35 interviews with:

chief digital and information officers from government departments;

other senior digital leaders from across the public sector;

government suppliers; and

other stakeholders such as professional service firms.

7 We carried out these interviews to understand what stakeholders see as the biggest barriers preventing successful digital change in government, how these challenges can be overcome, and any existing examples of good or bad practice government can learn from.

Published National Audit Office reports and document review

8 This report draws on our experience of auditing large programmes featuring digitally enabled change across different parts of government and at different stages over many years. We provide the case examples to illustrate the challenges that we have identified throughout this report and how they have manifested in digital change programmes. The summaries provided in this report reflect our findings at the time of the original report. They do not reflect the current status of each programme.

9 Our previous reports on digital issues and major programmes can be found on our website (www.nao.org.uk/search/pi_area/managing-major-projects/ and www.nao.org.uk/search/pi_area/digital-service-delivery/). We have used the following National Audit Office reports to inform our findings.

Comptroller and Auditor General, Universal Credit: early progress, Session 2013-14, HC 621, National Audit Office, September 2013.

Comptroller and Auditor General, Update on preparations for Smart Metering, Session 2014-15, HC 167, National Audit Office, June 2014.

Comptroller and Auditor General, Universal Credit: progress update, Session 2014-15, HC 786, National Audit Office, November 2014.

Comptroller and Auditor General, E-borders and successor programmes, Session 2015-16, HC 608, National Audit Office, December 2015.

Comptroller and Auditor General, Early review of the Common Agricultural Policy Delivery Programme, Session 2015-16, HC 606, National Audit Office, December 2015.

National Audit Office, The digital skills gap in government: Survey findings, December 2015.

Comptroller and Auditor General, Upgrading emergency service communications: the Emergency Services Network, Session 2016-17, HC 627, National Audit Office, September 2016.

Comptroller and Auditor General, Digital transformation in government, Session 2016-17, HC 1059, National Audit Office, March 2017.

Comptroller and Auditor General, The new generation electronic monitoring programme, Session 2017–2019, HC 242, National Audit Office, July 2017.

Comptroller and Auditor General, NHS England’s management of the primary care support services contract with Capita, Session 2017–2019, HC 632, National Audit Office, May 2018.

Comptroller and Auditor General, Rolling out Universal Credit,
Session 2017–2019, HC 1123, National Audit Office, June 2018.

Comptroller and Auditor General, Rolling out smart meters, Session 2017–2019, HC 1680, National Audit Office, November 2018.

Comptroller and Auditor General, Investigation into the British Army Recruiting Partnering Project, Session 2017–2019, HC 1781, National Audit Office, December 2018.

Comptroller and Auditor General, Investigation into Verify, Session 2017–2019, HC 1926, National Audit Office, March 2019.

Comptroller and Auditor General, Completing Crossrail, Session 2017–2019, HC 2106, National Audit Office, May 2019.

Comptroller and Auditor General, Progress delivering the Emergency Services Network, Session 2017–2019, HC 2140, National Audit Office, May 2019.

Comptroller and Auditor General, Early review of the new farming programme, Session 2017–2019, HC 2221, National Audit Office, June 2019.

Comptroller and Auditor General, Challenges in using data across government, Session 2017–2019, HC 2220, National Audit Office, June 2019.

Comptroller and Auditor General, Transforming courts and tribunals – a progress update, Session 2017–2019, HC 2638, National Audit Office, September 2019.

Comptroller and Auditor General, Digital transformation in the NHS, Session 2019–2021, HC 317, National Audit Office, May 2020.

Comptroller and Auditor General, Universal Credit: getting to first payment, Session 2019–2021, HC 376, National Audit Office, July 2020.

Comptroller and Auditor General, Digital Services at the Border,
Session 2019–2021, HC 1069, National Audit Office, December 2020.

Comptroller and Auditor General, Protecting and supporting the clinically extremely vulnerable during lockdown, Session 2019–2021, HC 1131, National Audit Office, February 2021.

National Audit Office, Guidance for audit committees on cloud services, April 2021.

External panel

10 We held a virtual panel, which 10 external experts attended. The purpose of the panel was to assess if we had identified the most important issues the government needs to tackle, and to seek views on how the government can best improve its performance in these areas. The panel included representation from both academia and industry, and we sought varied expertise to cover the different issues identified by our fieldwork.

Literature review feasibility study

11 We commissioned RAND to perform a feasibility study that considered the value of a wider literature review of research on digital change programmes that succeeded or failed. However, RAND concluded that existing literature did not add significant value to our report, as the majority of literature focused on high-level summaries of good practice or approaches for public and private sector digital change projects, analyses of national policies, or literature on small public sector projects at the local government level.

Information from the supreme audit institutions of other countries

12 We identified literature from other countries’ supreme audit institutions as a potential source of relevant information. We asked for information from seven countries, received responses from five and reviewed reports or case examples from the other two. We asked the supreme audit institutions to suggest reports and resources relevant to the issues identified by our fieldwork.

Design and Production by NAO External Relations
DP Ref: 009122-001