Improving the performance of major equipment contracts

Ministry of Defence

REPORT

by the Comptroller
and Auditor General

Session 2021-22

24 June 2021

HC 298

The National Audit Office (NAO) scrutinises public spending for Parliament and is independent of government and the civil service. We help Parliament hold government to account and we use our insights to help people who manage and govern public bodies improve public services.

The Comptroller and Auditor General (C&AG), Gareth Davies, is an Officer of the House of Commons and leads the NAO. We audit the financial accounts of departments and other public bodies. We also examine and report on the value for money of how public money has been spent.

In 2020, the NAO’s work led to a positive financial impact through reduced costs, improved service delivery, or other benefits to citizens, of £926 million.

We are the UK’s independent public spending watchdog.

We support Parliament in holding government to account and we help improve public services through our high-quality audits.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 23 June 2021

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Gareth Davies
Comptroller and Auditor General
National Audit Office

22 June 2021

© National Audit Office 2021

Value for money reports

Our value for money reports examine government expenditure in order to form a judgement on whether value for money has been achieved. We also make recommendations to public bodies on how to improve public services.

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.org.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

009120 06/21 NAO

This report can be found on the National Audit Office website at www.nao.org.uk

If you need a version of this report in an alternative format for accessibility reasons, or any of the figures in a different format, contact the NAO at enquiries@nao.org.uk

The National Audit Office study team consisted of:

Kaye Dunnet, Jeremy Gostick, Harry Hagger Johnson and Michael Slater with assistance from Huzaifa Abid, Jemma Dunne, Jeremy Heywood, Matthew Sharland and Ronan Smyth under the direction of Jeremy Lonsdale.

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

020 7798 7400

www.nao.org.uk

@NAOorguk

Key facts

Summary

1 The Ministry of Defence (the Department) is responsible for some of the most technically complex, risky and costly procurement programmes in government. It has the largest number of programmes in the Government Major Projects Portfolio, with a total budgeted whole-life cost of £162.6 billion in 2019-20. In that year, it paid some 44% of its £26.6 billion procurement expenditure to 10 major providers, nine of which are involved directly in the supply of military equipment.1 Due to the Department’s complex requirements for its major equipment programmes, it relies on a limited specialist supplier base to meet its needs. Equipment programmes place high demands on the technical skills of both departmental and supplier staff. They also require a high level of expertise in commercial negotiation, and programme and contract management.

2 Responsibility for managing and delivering defence programmes and contracts is shared between the Commands (Army, Navy, Air and Strategic Command), the Department’s Head Office, and its delivery agents – these include Defence Equipment & Support (DE&S) and Defence Digital. Teams within the Commands are accountable for delivery of the programmes. Scrutiny teams in the Commands and the Department’s Head Office provide assurance on programme teams’ business cases, such as the commercial and financial cases. The delivery agents (most often DE&S) lead on commercial negotiations and day-to-day relationships with suppliers and are accountable for delivery of equipment to the programme.

Strategic context

3 In March 2021, the government published its Integrated Review of Security, Defence, Development and Foreign Policy, a defence command paper, and the Defence and Security Industrial Strategy. The command paper set out the ambition to build a more strategic relationship with industry and a more sustainable industrial base, by providing greater certainty about future workflow. The Department stated that competition will no longer be the “default” position and put a new emphasis on delivering with UK-based skills, technologies and capabilities. This stance sits alongside recent changes to government guidance that now encourages departments to think more broadly about the “social value” that can be generated from departmental interventions, such as defence equipment programmes. In recent years, the Department has been reforming its approach to equipment acquisition, management and oversight of programmes, and developing its staff to tackle ongoing challenges with delivery.

4 Our annual reports on the Department’s 10-year budget for equipment procurement and support (the Equipment Plan) have highlighted the unaffordability of its forward programme, with an estimated affordability gap in the period 2020–2030 of at least £7.3 billion.2 Partly to address this gap, in November 2020, the Spending Review provided an additional £16.5 billion of capital funding over the next four years for the Department, including to modernise and invest in new technologies.

5 The Department’s performance in delivering major defence programmes has been mixed. In 2020, we reported there was an average forecast delay of more than two years to achieve full operating capability for the most significant capabilities.3 Such problems are not unique to UK defence; other countries experience similar issues.

6 Against this background, we sought to identify the causes, and explain the consequences, of cost overruns and schedule delays in the contracts for some of the most significant equipment programmes and to examine how the Department is working to improve delivery. After setting out the policy and administrative context (Part One), this report examines:

schedule delays and cost increases (Part Two);

contract and programme management (Part Three);

shortages of skilled staff (Part Four); and

delivering value for money through the life of the contract (Part Five).

Appendix Two summarises our work, which focused on 20 major programmes (Figure 10) for the delivery of equipment for use in the air, on land and at sea, and in space. Together these programmes have a whole-life budgeted cost of more than £120 billion. For the purposes of this report, we did not consider whether individual programmes have achieved value for money.

Key findings

Schedule delays and cost increases

7 The Department and its suppliers have both contributed to schedule delays across the contracted programmes we examined, resulting in shortfalls in defence capability. While several of the Department’s programmes were delayed prior to contract award, delays were more pronounced afterwards. The Department faces cumulative forecast net delays to equipment entering into service of 254 months across 13 of the programmes we examined. These delays were due to a variety of factors, including setting over-optimistic schedules early in projects and programmes, supplier performance, and contract management. Wider departmental affordability also contributed, leading, for example, to delays in order to make short-term savings (paragraphs 2.9 to 2.11 and Figures 5 and 6).

8 Programme cost estimates we examined have increased, but mostly before contracts were let. Where comparable information was available, we found that the forecast cost of nine out of 12 programmes increased between the initial business case and the main investment decision, on three occasions by 59% or more. In some cases, the Department responded to cost increases by reducing the number of units to be procured to remain within budget. Cost increases have been less likely and relatively less significant after contracts were let (although some programmes are still at an early stage). This may be attributable to more certainty around cost estimates following engagement with suppliers, the use of firm-fixed price contracts and improved project controls in DE&S (paragraphs 2.5 to 2.8 and Figure 4).

Managing challenges to contract delivery

9 The Department’s short-term approach to the financial management of its equipment portfolio has affected suppliers’ ability to deliver contracts effectively, although it is now seeking to address this through its industrial strategy. Our recent report The Equipment Plan 2020–2030 found that the Department’s focus on short-term financial management and delaying expenditure into future years is increasingly restricting Front-Line Commands’ ability to develop the capabilities they need. The Department depends on its suppliers to deliver defence capability. However, a lack of clarity over the future programme, and the use of departmental savings measures designed to manage short-term affordability challenges, mean suppliers lose skills, and are reluctant to take on risk when contracts are let. The Department is now seeking to rebuild capacity in the UK defence industry to address gaps in capability, including through long-term strategies for shipbuilding and combat aircraft (paragraphs 2.2 to 2.4).

10 The Department has not opened to competition the majority of the programmes we examined. In the past, the Department has found it more difficult to secure value for money from procurement where there is no competition. In March 2021, the Department announced it is moving away from using competition as the default option for defence equipment programmes, but in fact competition has not been the norm. Out of the 20 projects and programmes that we examined, 14 are being wholly or partly procured non-competitively. The Department only made this choice because of government national security policy in a minority of cases. In other cases, the Department chose not to let the contract through competition, either because of a lack of alternative suppliers for particularly complex requirements, or as a result of its own internal analysis of the most cost-effective solution. Cost overruns and schedule delays have affected programmes adopting both competitive and non-competitive procurements. In the absence of competition to help demonstrate value for money, delivery teams and the Department report that the Single Source Contract Regulations 2014 have proved effective at driving out unnecessary cost from contracts covered by the legislation and in strengthening the Department’s negotiating position, although the full benefits will take some time to become apparent (paragraphs 3.2 and 5.5 to 5.8).

11 Suppliers have failed to deliver contracted levels of performance on a number of programmes. Figure 6 on pages 34 to 38 sets out challenges to successful delivery faced by the projects and programmes we examined, including the areas in which suppliers were not delivering to the contract. Problems with supplier performance referenced by delivery teams in the projects and programmes that had experienced problems with supplier performance included:

the technical ability of suppliers to execute complex design work and meet the requirements of defence standards. The ability to provide safety cases that satisfy the various military safety regulators is a recurring problem; and

the suppliers’ ability to manage the programme effectively, including their ability to oversee other parts of the supply chain.

On occasions – such as on the Ajax armoured vehicle programme – the Department has changed its requirements after the contract was let, making it more difficult for suppliers to achieve cost and schedule milestones. This has led to renegotiation of aspects of the contract (paragraphs 3.7 to 3.11 and Figure 6).

12 The Department’s lack of accurate information on the progress of some programmes makes it more difficult to monitor suppliers’ performance, but it is taking steps to address this. For example, in the case of the Crowsnest radar system, neither the Department nor the prime contractor identified the lack of progress being made by the key sub-contractor. Staffing issues described in paragraphs 16 to 18 also contribute to a lack of oversight. The Department is now introducing earlier support and scrutiny to programmes in order to identify gaps in knowledge and is measuring supplier progress more accurately using Earned Value Management techniques. Once problems are identified, we saw evidence of the Department addressing issues, such as through engagement by senior leaders or by embedding staff with the supplier (paragraphs 1.5, 3.7 and 3.11 to 3.15).

13 The Department does not meet its own obligations to suppliers under some contracts. Government Furnished Assets (GFA) are assets purchased separately by the Department, or existing departmental assets, which are integrated into a programme and used by the supplier. Problems with the supply and availability of GFA persist despite programme teams recognising the risks involved and trying to minimise GFA use. The Department has also experienced difficulties when attempting to take on the role of integrating GFA with the work of suppliers (paragraphs 3.16 and 3.17 and Figure 6).

14 The Department uses different contracting approaches to better control costs and, less successfully, to speed up delivery. The Department adopts different contracting approaches depending on the risks and challenges of a project or programme. The Department has sometimes imposed financial penalties on suppliers for poor performance and failing to meet milestones and has contracted with suppliers in a way that will reward them for over-achieving against cost and schedule targets. Despite this, schedule delays remain common. The Department wants to speed up procurements by deploying ‘agile’ programme delivery more widely, but this does not sit comfortably with contracting approaches designed to minimise costs, or the Department‘s existing culture and skills, and requires better transparency on progress (paragraphs 3.3 to 3.5).

15 The Department needs strong leadership to embed accepted commercial good practice. The Department is adopting accepted commercial good practice, which it estimates could realise benefits of £788 million over the next 10 years. This includes development of a Strategic Partnering Programme with 19 of its main suppliers. The Department and BAE Systems, the pathfinder company for the programme, have together developed a shared view of programme performance, and are carrying out specific interventions to improve contract delivery. The Department is drawing on its experience of individual projects and programmes to inform its interventions. Under new government proposals, it will also be easier to take past performance by suppliers into account when letting new contracts. The Department is also implementing Category Management, a best-practice approach to realising greater efficiencies from procurement, which it has previously tried, and failed, to embed. Strong leadership will be needed to manage the risks to realising the benefits of these initiatives, including: that adequate levels of skilled resources will not be available to implement the initiatives properly; that cultural barriers in the Department and suppliers will block progress; and that failure to capture evidence of benefits will undermine the case for change and lead to withdrawal of funding (paragraphs 3.17 to 3.20).

Capacity and capability to manage the delivery of contracts

16 The Department’s ability to effectively manage contracts, identify risks and oversee suppliers has sometimes been hampered by shortages in experienced and skilled staff. In the programmes we examined, risks around the availability of sufficient qualified and experienced staff were a recurring concern for teams. The cost of such staff is small compared with the costs of the programmes they deliver. DE&S has ‘pinch point’ shortages in areas vital to programme delivery. Both it, and the Submarine Delivery Agency (SDA) have made increased use of contractor support to fill in-house posts. This is costly and reduces corporate memory. When recruiting contractors, organisations may find themselves competing with each other, other parts of government and the private sector. To address this, the Department has plans to increase the skills and capabilities of its workforce across commercial management and programme delivery. In the meantime, it has set up frameworks for private sector partners to provide additional commercial and cost assurance skills more efficiently (paragraphs 4.2 to 4.5, 4.7, 4.12 and Figures 6 and 7).

17 The Department faces particularly acute problems in recruiting and retaining staff to oversee digital programmes and is now seeking to reduce reliance on contractors. The New Style of IT (Deployed) programme and the Morpheus project rely on outside contractors for 79% and 58% of their complements respectively. The Skynet 6 satellite programme is finding it difficult to rebuild capacity in space-based capability after the Department let its predecessor under a private finance initiative contract, thereby losing in-house skills. Defence Digital is disadvantaged when recruiting compared with DE&S and the SDA, which can diverge from civil service pay rates. The Department’s chief information officer has developed a Strategic Workforce Plan to reduce reliance on outside contractors and improve digital skill levels (paragraphs 4.6 to 4.8).

18 Senior members of programme delivery teams can lack support and sufficient time in post to effectively manage the programme and supplier relationships. Despite the long-term nature of defence programmes, there is considerable churn among those leading them, with individuals frequently in post for only a fraction of the contract lifecycle. This is particularly true for service personnel, where there is inherent tension between the duration of these programmes and their career paths. For example, the programme managers for our sample had typically been in place for just over one year. Senior responsible owners (SROs) report a lack of key skills commonly associated with effective contract management and a tension between their accountability for programme delivery and lack of budgetary control (paragraphs 4.9 to 4.11).

Ensuring value for money through the life of the contract

19 The Department does not rigorously review whether it is continuing to secure value for money across the lifecycle of its projects and programmes as costs and schedules change. The Department requires programme teams to present a value-for-money (VFM) case to justify the chosen contracting approach for each programme. In the programmes we examined, affordability and a limited choice of contracting solutions frequently determined the definition of VFM. Once a programme is on contract, the operational need for delivery as soon as possible means that issues with cost increases and delays are rarely acknowledged within the Department as compromising VFM. In eight of the projects and programmes we examined, the Department’s accounting officer has formally reported to Parliament where programme cost and/or time parameters have been breached, or are likely to be breached. None of these submissions stated that VFM was affected, although not all addressed the issue directly. The Department does not currently evaluate the extent to which contracts where the equipment has entered service have delivered VFM, although it has plans to upgrade its approach to programme evaluation (paragraphs 5.2, 5.3 and 5.9 to 5.11).

20 Although Departmental teams have identified learning from their own experience, many had not systematically learned lessons from other contracts. Teams’ current analysis of the risks they face indicate that they are aware of the challenges discussed in this report but can find it difficult to mitigate them. We saw good examples of teams identifying and collating learning from their own and others’ experience, but the depth and maturity of this work varies significantly. In some cases, it appears the same lessons are learned separately by different teams; for example, on providing GFA to the programme. Embedding good practice more widely would offer the opportunity to reduce the assurance processes currently in place and enable teams to demonstrate that they have acted to avoid past mistakes and reduce programme risks (paragraphs 5.12 and 5.13).

Conclusion on value for money

21 The Department has regularly experienced difficulties in effectively managing its major equipment contracts, with frequent delays and cost increases. These stem from supplier under-performance; weaknesses in departmental contract management; the Department and suppliers underestimating the scope and technical complexity; and the Department prioritising short-term solutions because of its affordability challenges. Consequently, the Department has not been able to optimise value for money from the contracts for its largest, most complex equipment programmes. The recent Integrated Review and the announcement of £16.5 billion of additional expenditure, much of it for future defence equipment and support work, emphasises the urgency of strengthening how the Department manages key contracts.

22 To improve value for money the Department must follow through on its initial efforts to embed wider good practice in its commercial relationships and project delivery. These are promising steps, but it is early days for the initiatives, some of which have been tried unsuccessfully before and do not necessarily fit easily with the existing departmental culture. Strong leadership and sustained resources will be needed to fully embed these changes and deliver real benefits. A key part of the Department’s agenda must also be to learn lessons routinely across the portfolio, including being honest in acknowledging and learning from examples of poor value for money when they occur.

Recommendations

23 The Department is developing a programme of important reforms, some of which are at a relatively early stage. To support this agenda and to complement the recommendations in our recent report on the Department’s 2020–2030 Equipment Plan,4 we recommend that:

a Programme teams should state explicitly in their initial business cases how they have applied lessons learned from other programmes. To avoid repetition of past mistakes and embed learning from experience into the approvals processes, teams should demonstrate to the Investment Approvals Committee and the Department’s scrutiny teams how they have drawn on comparable programmes, including past experience of working with suppliers on other programmes.

b While the Department has introduced initiatives to secure better value from its contracts, such as the Strategic Partnering Programme, it must ensure they become embedded in departmental practice. We support the enhanced approaches to joint working with suppliers but have seen previous efforts falter. To avoid this, the Department should ensure sufficient resources are available, and make use of existing governance arrangements to maintain a focus on progress and yielding anticipated benefits.

c The Department should be prepared to penalise suppliers for past poor performance when letting new contracts. While the Department is doing more to improve working relationships with suppliers, there remains the challenge of what to do if suppliers fail to deliver. In the government green paper on procurement it is proposing to make it easier to exclude suppliers with records of poor performance from future procurements. As and when rules change, the Department should consider how it could use this power to incentivise better performance.

d The Department should work with the Cabinet Office and HM Treasury to address shortfalls in vital contract and programme management skills. Differing levels of remuneration and terms of employment between different parts of the Department, between the Department and other parts of government, and between government and other sectors create long-term skills gaps. Reliance on buying in temporary support is expensive and inefficient. The Department’s Industrial Strategy promises to develop required skills in the defence industry and should be used as an opportunity to develop an approach that helps all parties.

e The Department should do more to keep key personnel in place on contracted programmes for as long as is needed to meet specific milestones. SROs and senior members of their team are crucial to the delivery of contracted programmes and strong supplier relationships, but there is evidence that they are insufficiently supported and too often move before key milestones have been achieved. The Department should examine how it can align movement of personnel more closely with programme progress.

f The Department should pay greater consideration throughout a programme as to whether it remains value for money. Business cases set out whether the proposed option is VFM, but major changes in delivery dates, for example to make the portfolio more affordable in the short-term, and increases in forecast costs, can have a significant impact on whether a programme remains so. Currently, it is not clear that at this point there is appropriate reflection on options or remedial action. Each business case should include specific performance, cost and time criteria for when the programme ceases to be VFM, based on better baselines and benchmarks derived from improved programme evaluations. Breaching these criteria should require more rigorous and transparent consideration of how VFM can be achieved.

1 Airbus Group SE; Babcock International Group PLC; BAE Systems PLC; General Dynamics Corporation; Leonardo SpA; Lockheed Martin Corporation; QinetiQ Group PLC; Rolls-Royce Holdings PLC; The Boeing Company.

2 Comptroller and Auditor General, The Equipment Plan 2020–2030, Session 2019–2021, HC 1037, National Audit Office, January 2021.

3 Comptroller and Auditor General, Defence capabilities – delivering what was promised, Session 2019–2021, HC 106, National Audit Office, March 2020.

4 See footnote 2.

Part One

Contracting for defence equipment

1.1 In our 2020 report Defence capabilities – delivering what was promised,5 we reported on the challenges the Ministry of Defence (the Department) faces when setting the requirements for equipment, through to the capability being ready for full deployment. This report examines the underlying causes of problems in contracting for defence equipment programmes. This part describes the policy and administrative context.

1.2 The Department requires technically complex military equipment to meet the government’s strategic requirements for modern warfare. Equipment must also be integrated with existing infrastructure and personnel. As a result, there is inherent risk at all stages of the acquisition process. Given the long lead times for its major acquisition programmes, the Department faces a major challenge in dealing with an external environment where the capability of potential adversaries is constantly evolving and presenting new threats.

1.3 The Department has the largest number of programmes in the Government Major Projects Portfolio, with a total budgeted whole-life cost of £162.6 billion in 2019-20. Some 44% of its £26.6 billion annual procurement expenditure goes to its 10 largest suppliers. Due to the Department’s complex requirements for its major equipment programmes, it relies on a limited specialist supplier base to meet its needs.

1.4 Contracts deliver military capability. Capability needs are identified through the strategic planning process, most recently outlined in the 2021 Integrated Review of Security, Defence, Development and Foreign Policy (the Integrated Review).6 Figure 1 shows that responsibility for delivering the required defence equipment programmes and managing contracts with suppliers is shared between the Commands (Army, Navy, Air and Strategic Command) and the Department’s delivery organisations, while Head Office provides strategic direction and approves and monitors major programmes.

1.5 We examined 20 of the Department’s largest, most complex equipment programmes in depth (see Appendix Two). They are at different stages of their programme lifecycle, from concept design through to being in service, and some have been under development since the 1990s. For this reason, at various points in this report, our analysis focuses as appropriate on sub-sets of the 20 projects and programmes.

Performance of defence contracts

1.6 In the past 50 years, there have been at least eight major defence reviews. We have reported on many occasions on the Department’s programme of major projects and, since 2013, on the Equipment Plan, as well as individual programmes. Figure 2 summarises the factors affecting the performance of the Department’s contracts and programmes identified in this report and illustrates how contract performance may be influenced by factors outside the immediate contractual relationship between the Department and its suppliers. Failure to bring equipment into service as expected means the Department must rely on ageing platforms and technology for longer than it anticipates, or manage gaps in military capability.7 For example, in our 2020 report Carrier Strike – Preparing for deployment, we reported that an 18-month delay in the Crowsnest radar system would affect Carrier Strike capabilities in its first two years (see paragraph 3.7).8

1.7 The Department’s senior responsible owners (SROs) have regularly expressed concerns over aspects of the Department’s delivery of equipment programmes and suppliers’ performance. Figure 3 shows that as at March 2021, among 19 of the 20 programmes we examined,9 SROs rated:

delivery confidence as ‘amber/red’ or ‘red’ in eight;

supplier engagement and delivery performance as ‘amber/red’ or ‘red’ in four, involving three prime suppliers. These programmes have been on contract for between 59 and 149 months; and

programme skills and capabilities as ‘amber/ red’ or ‘red’ in five cases.

1.8 During 2020-21, the programmes we examined have been affected to varying degrees by the COVID-19 pandemic, mainly where suppliers are manufacturing equipment in industrial facilities with large numbers of workers on site. We have not discussed the extent to which COVID-19 has affected programme delivery in our report, as commercial negotiations are ongoing on the extent of the disruption and its financial consequences.

1.9 The Department is not alone in facing challenges with procuring defence equipment. The US Government Accountability Office has published many reports on defence equipment acquisitions. Its findings are consistent with a number of those in this report. For example, it has identified cost and schedule uncertainty when programmes are approved, compounded by incentives for the Department of Defense to be over-optimistic about delivery, and a lack of competition among suppliers.

Recent developments

1.10 Since 2018, the Department has been developing and implementing its Acquisition and Approvals Transformation Programme to improve the outcomes from its acquisition system. We commented on the early stages of this in March 2020.10 Figure 2 shows how these activities are intended to mitigate some of the issues we have identified. We have referred to relevant parts of the transformation programme throughout the report.

1.11 The performance of the Department’s contracts must also be seen against a background of its unaffordable Equipment Plan. In January 2021, we reported that the Department estimates that costs will be £7.3 billion higher than its £190 billion equipment budget for procurement and support to 2030, although this figure could be significantly higher if certain risks materialise.11 In November 2020, the Spending Review provided an additional £16.5 billion of defence funding over the next four years. In March 2021, the government published the Integrated Review,12 a defence command paper,13 and the Defence Industrial Strategy.14 The Strategy states that competition will remain an important tool to drive value for money, but a more flexible approach will be applied in designing capability and acquisition strategies to deliver and grow UK-based industrial skills, technologies and capabilities.15 The Department also stated that it would make procurement policies and processes more agile.

5 Comptroller and Auditor General, Defence capabilities – delivering what was promised, Session 2019–2021, HC 106, National Audit Office, March 2020.

6 HM Government, Global Britain in a competitive age: The Integrated Review of Security, Defence, Development and Foreign Policy, CP 403, March 2021.

7 The main milestones discussed in this report are in-service date, initial operating capability (IOC) and full operating capability (FOC).

 In-service date is the date on which a programme enters service.

 IOC is the minimum level at which the capability or service is usefully deployable.

 FOC is the level of military capability which is intended for a particular programme.

8 Comptroller and Auditor General, Carrier Strike – Preparing for deployment, Session 2019–2021, HC 374, National Audit Office, June 2020.

9 The Department ceased reporting on the Warrior vehicle upgrade programme following its cancellation (see Figure 8). As at December 2020, the SRO rated the programme’s delivery confidence as ‘amber/red’, supplier engagement and performance as ‘amber/red’ and skills and capabilities as ‘amber/green’.

10 Comptroller and Auditor General, Defence capabilities – delivering what was promised, Session 2019–2021, HC 106, National Audit Office, March 2020, Figure 10.

11 See footnote 5. This does not include the additional funding announced in the November 2020 Spending Review.

12 See footnote 6.

13 Ministry of Defence, Defence in a competitive age, CP 411, March 2021

14 Ministry of Defence, Defence and Security Industrial Strategy: A strategic approach to the UK’s defence and security industrial sectors, CP 410, March 2021.

15 The Department has reported that departmental expenditure supports almost 200,000 jobs in the UK.

Part Two

Schedule delays and cost increases

2.1 This Part examines: the budgetary pressures that provide the context for the letting and delivery of equipment contracts; and the extent to which cost increases and delays among the programmes that we examined have occurred since contracts for the programmes were let.

Budgetary pressures and cost increases

2.2 Due to the long-standing pressures on its equipment budget, examined in our annual reports on the Equipment Plan, the Ministry of Defence (the Department) initiates all its major programmes within tight budget constraints. As a result, any cost increase will have consequences for the programme’s affordability, and potentially that of others in the portfolio. The National Audit Office (NAO) and the Committee of Public Accounts have commented regularly on the Department’s approach of reducing programme funding through in-year underspends where the programme is not making the expected level of progress; in-year savings measures (against either capital or resource budgets) to avoid spending more than Parliament has authorised; and strategic defence reviews and spending reviews altering priorities.16 These pressures were evident in the programmes we examined.

2.3 Focusing on managing short-term financial pressures in this way leads to higher overall costs, and deferring programmes has created larger funding shortfalls in later years. In 2020-21, the Department continued to defer programmes into future years, despite the additional funding announced in November 2020. Faced with a lack of clarity about the defence programme pipeline, suppliers lose skills and are reluctant to take on risk when contracts are let (see paragraph 3.3).17 The Department has acknowledged this through strategies for shipbuilding (2017) and combat aircraft (2018), and states in its March 2021 industrial strategy that it will give UK companies the confidence to increase investment in their skills and equipment.18

2.4 Within the context of a constrained overall budget, the cost estimates used by the Department in the preparatory stages of programmes are often understated, causing budget pressures once more mature estimates show programmes will exceed their allocated budgets. Reasons for this include:

the technically complex nature of large defence programmes means that there is a high degree of uncertainty about future costs and requirements;

companies specialising in supplying UK defence may have an incentive to understate costs owing to the need to gain work from their only customer; and

the Commands are conscious that they are in competition for funding, and there is a risk they present an overly optimistic scenario to gain approval to proceed.

Much of this behaviour was noted by RAND Europe in a discussion paper that we commissioned. RAND also noted the ‘moral hazard’ that results when under-performing programmes receive more funding.19

Pre-contract cost estimate increases

2.5 Figure 4 shows how cost estimates for programmes covered in this report changed prior to the Department committing to the full costs of the programme. It is to be expected that programme costs will evolve as better information is available. However, it means that the Department may be committed to a programme before the true costs become clear. Nine programmes out of the 12 with available data saw forecast cost increases between their early business cases and main investment decision, with the costs of three growing by 59% or more. In addition, in four of the nine programmes with available data, the cost estimate approved at the main investment decision point exceeded the worst-case scenario modelled at the point of initial approval. Analysis carried out by the Department in October 2020 confirmed that large increases during this stage were common. The efforts necessary to accommodate these increased cost estimates within the programme and across the portfolio can have a significant influence on the contracting approach, by prioritising cost avoidance over the other elements, and on relationships between the Department and suppliers.

Post-contract cost increases

2.6 To date, cost increases after programmes have gone on contract occurred less often. Four of the 12 reported significant cost increases of between 18% and 60%. These were the Warrior armoured vehicle upgrade (recently cancelled) and procurements for the Protector unmanned aerial vehicle, A400M transport aircraft and Astute Boat 5 attack submarine. Protector has seen the largest cost increase since approval. The single largest element of the forecast cost increase for Protector was the decision to defer the programme for short-term affordability reasons, and not due to contract management issues.

2.7 Possible reasons for the lower levels of increase in cost estimates once the programme is on contract may include:

that, at this point, there is a much higher degree of cost certainty, although the Department may continue to change its requirements, as described in paragraph 3.9. Two of the programmes – the Boxer armoured vehicle and P-8A Poseidon maritime patrol aircraft – are purchasing equipment already in service with other nations, which should significantly de-risk acquisition;

improved programme management practices within Defence Equipment & Support (DE&S) (see paragraph 3.11); and

that the Department has limited the cost implications of problems by using firm-fixed price contracts (see paragraph 3.3).

However, some programmes have not been on contract for long, and cost pressures are more likely to emerge further into the contract. Two of the 12 programmes have been on contract from early in 2021, and two others since late 2019.

Impact of cost increases on capability

2.8 Cost increases or funding cuts to programmes affect the ability of the Department to meet the equipment requirements identified by Commands. We identified examples where the original capability requirement was modified, sometimes significantly, to remain within cost parameters. For example:

funding cuts in 2012 reduced the number of Ajax armoured vehicle numbers from an initial requirement of 761 to a contractual commitment of 589;

the Department is proposing to upgrade 148 Challenger 3 tanks against a requirement of 190, based on revised costs to properly reflect user requirements;

the Department reduced the number of P-8A Poseidon maritime patrol aircraft it will purchase from 12 to nine, the minimum it believed necessary; and

when the Type 26 frigate programme team received approval to enter the ‘assessment’ phase in 2010, it expected to gain approval for manufacture in 2013. Approval for the assessment phase was delayed until March 2010. The estimated cost increased from £8.2 billion in 2009 to £12.1 billion by 2012. Contributory factors included an increase in the planned number of ships, from 10 in 2009 to 13 in 2012, and the need to re-plan in 2010 to reflect a change in requirement for an increased level of capability for the class as a whole. Despite efforts by the programme team and supplier, costs could not be reduced to a level deemed affordable. Eventually, the Department reduced its requirement to eight ships, and will also now purchase five Type 31e frigates, capable of a more limited range of tasks.

Schedule delays

2.9 Where comparable data are available, nine of the programmes we examined had experienced delays of between three and 47 months in reaching the main investment decision point against early forecasts. Delays in the programmes we examined were more pronounced after award of contract, where the Department faces cumulative forecast net delays to entry into service of 254 months across 13 of the programmes we examined (Figure 5). A number of these programmes will incur further delays as a result of the impact of the COVID-19 pandemic.

2.10 Delays may be attributable to the Department, its suppliers, or a combination of both (see Figure 6 in Part Three, on pages 34 to 38). For example:

on A400M, the supplier’s delivery of aircraft to the partner countries has been severely delayed, including a six-year delay to UK entry into service;

in the case of the Skynet 6 satellite, the Department delayed this programme by three years as a savings measure. As a result, the Department had to develop a sub-project within the Skynet programme to maintain existing Skynet 5 capability during the three-year delay until the introduction of Skynet 6;

on the Type 26 frigates, joint efforts by the Department and supplier to make them affordable (paragraph 2.8) led to delays of more than three years in approval to manufacture; and

on the Marshall air traffic management system, delays have resulted from various factors including the supplier’s and Department’s under-estimation of the technical complexities, and programme and contract management demands of the work.

Consequences of schedule delays

2.11 Delaying the entry into service of new capabilities can mean the Department has to maintain equipment that is ageing, and in some cases becoming obsolete. For example, the Protector unmanned aerial vehicle was originally intended to enter service in 2018 to coincide with retirement of the Reaper predecessor capability. The Department’s forecast date for initial operating capability (IOC) was July 2021 by the time the decision was taken to delay the programme because of budgetary pressures.20 IOC is now expected in November 2023, requiring extension of Reaper at an additional cost of £50 million. In addition, delays in the preliminary stages of the Type 26 frigate, and an acceptance of a more realistic timetable for the Type 31e Frigate as part of the competition process, mean that the first ships of each class are forecast to start to enter service in 2026 and 2027 respectively. This requires the predecessor Type 23 to undergo upgrade work to stay in service until the new ships are available.

16 Comptroller and Auditor General, The Equipment Plan 2020 to 2030, Session 2019–2021, HC 1037, National Audit Office, January 2021.

17 For example, see the Offshore Patrol Vessel case study in Comptroller and Auditor General, Defence capabilities – delivering what was promised, Session 2019–2021, HC 106, March 2020.

18 Ministry of Defence, National Shipbuilding Strategy: The Future of Naval Shipbuilding in the UK, September 2017; Ministry of Defence, Combat Air Strategy: An ambitious vision for the future, July 2018.

19 RAND Europe, Persistent Challenges in UK Defence Equipment Acquisition, June 2021. The discussion paper can be found at: www.rand.org/pubs/research_reports/RRA1174-1.html

20 IOC is the minimum level at which the capability or service is usefully deployable.

Part Three

Contract and programme management

3.1 Contract management is at the heart of successful programme delivery. Its effectiveness is influenced by a range of factors including budgetary issues discussed in Part Two and the supply of skilled staff discussed in Part Four. This part looks at:

the impact of different contracting approaches;

the challenge of managing the suppliers’ work; and

the Ministry of Defence’s (the Department’s) initiatives to improve contract and programme delivery.

Contracting approaches

Competition versus non-competitive (‘single source’)

3.2 In the past the Department has found it more difficult to secure value for money from procurement where there is no competition. Between 2012 and 2021, the Department’s policy was to use competition, and to also look for proven existing products to meet its requirements ‘off-the-shelf’ where possible.21 However, in 2017 we reported that around half of equipment contracts were still let non-competitively.22 Of the 20 programmes we examined in this study, the Department is procuring 11 non-competitively, and three partly non-competitively. Although UK sovereignty requirements are often cited as a reason why non-competitive contracts are so common in defence procurement, only four of the 14 non-competitive programmes we examined were subject to this constraint. In several programmes we examined, the Department is seeking to introduce competition into areas which were previously ‘single source’.

However, the government has recently announced that it is moving away from a “competition by default” approach. In the case of the Challenger 3 tank and Boxer armoured vehicle programmes, and the Poseidon P-8A maritime patrol aircraft, the Department chose not to compete the requirements, either because of a lack of alternative suppliers for particularly complex requirements or as a result of its own internal analysis of the most cost-effective solutions. For example, the Poseidon P-8A maritime patrol aircraft team identified that the chosen option was the lowest risk on grounds of technical maturity and was twice as effective as alternative aircraft at carrying out its core task.

Types of contracts used

3.3 The selection of the contracting approach appropriate to the risks and challenges of a programme can set the groundwork for successful delivery. This is especially important given the frequent absence of competition to put downward pressure on costs. For all contract types, the Department may also incur additional costs if it chooses to change its requirements during the contract period. The Department uses a range of contracting approaches in its portfolio of major equipment programmes. It tailors its commercial approach according to programme requirements, sectors and suppliers. Certain types of contract predominate:

Contracts where the Department pays the allowable costs incurred by the supplier, plus a profit percentage (‘cost plus’). Since this approach only permits the Department to monitor costs and profits, rather than incentivising the supplier to minimise costs, the Department seeks to limit such arrangements to the design phase of major programmes, where scope and parameters are still under development. An extreme example is the Type 26 frigate, where approval for manufacture, and a move to a more cost-controlled contract, was delayed while the Department worked with the supplier to make the overall programme affordable. The programme was due to spend four years on a ‘cost plus’ contract at the design and development phase, at a cost of £158 million, but instead this lasted more than seven years, at a cost of £853 million.

Contracts which incentivise suppliers and the Department to do what is necessary to deliver within the target cost. This differs from a ‘cost plus’ approach as it offers suppliers a financial incentive to deliver below target cost, with both parties sharing the financial risks of failing to achieve it. Historically, high-profile contracts of this type have incurred cost overruns, for example the Astute attack submarine programme and Queen Elizabeth Carriers. Such contracts move away from the Department’s standard model of costing programmes on the basis of the 50th percentile (that is, the programme’s cost model indicates that the actual cost has an equal chance of being higher or lower than the estimate). For the Type 26 frigate, the negotiated target cost was at the 85th percentile, and for Astute Boat 5, the 76th percentile. This means that there is more scope in these cases for costs to increase without penalty to the suppliers than would normally be the case.

Contracts where the supplier agrees to meet the requirement for a set, all-inclusive price, or where variation is limited to an element for inflation. These firm-fixed price contracts are desirable for the Department when budgetary pressures increase the need for certainty over costs. We have seen several examples where this approach has controlled costs to date, such as the Crowsnest and Spearfish programmes, although future increases cannot be ruled out given the problems encountered in both cases. This type of contract means that the supplier will bear the risk of failing to accurately cost the work.

The impact of ‘agile’ acquisition on contract delivery

3.4 The Department’s plans to transform defence procurement include using ‘agile’ approaches. Agile is a methodology widely used for delivery of digital services, where the speed of technological change would overtake traditional procurement approaches. While such programmes still have a defined outcome, requirements and solutions are developed iteratively. Therefore, the balance of cost, time and capability is subject to constant change.

3.5 Delivery teams have, however, identified that agile is not compatible with those types of contract favoured by the Department which emphasise certainty of output and cost (see paragraph 3.3). In addition, teams delivering the New Style of IT (Deployed) programme and the Morpheus project identified other issues with the early adoption of agile. These included:

project and programme teams being left to learn about agile by trial and error;

a lack of shared understanding of what is to be delivered among Departmental stakeholders and suppliers;

customers’ reluctance to accept trade-offs in capability to improve deliverability;

lack of experience of the technique within some suppliers; and

problems for agile programmes interfacing with related programmes which are being procured conventionally.

Contract and programme management

3.6 As we saw in Parts One and Two, the major programmes we examined have a mixed record of delivery against time and cost milestones, despite the scrutiny of these programmes at key stages. Contracts may fail to deliver for a range of reasons. These include loss of skills due to gaps between programmes, and differing assumptions between the Department and suppliers about aspects such as design maturity, workforce productivity, or the appreciation of risk and who carries it. Figure 6 sets out the main issues encountered in the programmes we examined which were on contract. We have drawn on these examples to illustrate points made throughout this report.

Underestimating programme complexity

3.7 For several of the programmes we examined, major problems resulted from the Department and suppliers jointly underestimating the complexity of upgrading or building upon equipment that is already in service. For example:

at the start of the programme in 2010 the programme team reported that the Spearfish torpedo upgrade was considered ‘low risk’ by the supplier, Defence Equipment & Support (DE&S) and subject matter experts, because the technology and the approach to integration were seen as well-established. However, this significantly underestimated the level of interdependencies with other programmes, leading to: too narrow a programme scope; use of immature cost estimates; underestimation of the complexities of securing safety accreditation; and overestimation of supplier ability to manage the technical challenges; and

the Crowsnest radar system was developed to be fitted to the Department’s existing Merlin helicopter fleet. An internal ‘lessons learned’ review concluded that neither the Department nor industry understood the complexities of delivering the capability. Changes to funding, scope and timetable, together with a fixed price contract, contributed to subcontractor under-performance, which the prime contractor and the Department did not detect until it was too late to meet the target delivery date.

Supplier performance and contract management

3.8 Figure 6 sets out challenges to successful delivery faced by the projects and programmes we examined, including the areas in which suppliers were not delivering to the contract. Problems with supplier performance referenced by delivery teams included:

the technical ability of suppliers to execute complex design work within the agreed timetable, and meet the requirements of defence standards, including the ability to provide safety cases that satisfy the various military safety regulators, for example the Spearfish torpedo upgrade; and

suppliers’ ability to project manage effectively. This was sometimes due to a wider under-resourcing of the work by the supplier. A particular issue was the management of other parts of the supply chain, for example in the case of the Type 26 frigate.

The Department has made achievement of milestones more difficult in some cases by changing user requirements after the contract was let. For example, on the Ajax armoured vehicle, user changes required renegotiation of the contract with the supplier.

3.9 Once a project or programme has experienced difficulties, we saw examples of the Department intervening, with engagement at senior levels; staff being embedded on site to improve relationships; improved programme controls; sharing of management information; and, where relationships had broken down, staff changes on both sides. These interventions are reported as yielding benefits but would have been better put in place before problems occurred. The Department has also mitigated some of the consequences of poor performance by imposing financial penalties, or withholding payments, where allowed in the contract.

3.10 Some suppliers have under-performed on a range of contracts over time, and anticipation of this happening again shows up on risk registers and in teams seeking to identify lessons from other programmes. However, even when under-performance is significant, the Department appears reluctant to remove a supplier from a contract because of the knock-on effects of ‘resetting’ a programme. Under current regulations, the Department is unable to bar suppliers from future contracts unless it has removed them from a previous contract for poor performance. In December 2020, the government published a green paper on transforming public procurement.23 It proposes that the best-performing suppliers should be rewarded with a greater share of government contracts, and conversely that it should be easier to take into account past supplier performance when awarding future contracts.

Improvements to programme management

3.11 Programme teams maintain risk registers for each programme to record the main risks to successful delivery. Programmes often live with high risk. Across the programmes we examined, 41% of the main risks identified by programme delivery teams had a ‘high’ or ‘very high’ probability of occurring, even after proposed mitigations. Risks relating to the availability of sufficient qualified and experienced staff (see Part Four) were among the highest risks for more than half of the programmes we examined.24

3.12 The Department is attempting to manage these risks in a number of ways. For instance, DE&S has improved its project controls, including gathering enhanced data on supplier performance and risks across individual programmes and portfolios. It also monitors supply chain capacity, performance and risk tolerance, as well as suppliers’ exposure to market risks and opportunities. This work is feeding into the Department’s Strategic Partnering Programme (paragraph 3.17).

3.13 One way to ensure that the supplier is performing in line with milestones and is, therefore, only being paid for work done, is the programme management technique known as Earned Value Management (EVM). In 2015, we reported that DE&S was failing to make use of the technique even though it had been mandated since the 1990s.25 During our current work, we found that EVM is now being adopted widely on the programmes we examined.

3.14 Partly in response to the problems described above, from April 2019, the Department introduced a new approach to the approval of programmes to better identify risks and mitigate actions at an early stage of the programme. A Strategic Outline Case is produced to provide more assurance that the critical success factors, such as the level of complexity and the main risks, are being considered at an early stage of the programme. The benefits of this approach will only become apparent over time. In late 2020, the Department conducted a review of how the process had worked in its early days and identified a need to clarify and simplify the process in order to reduce the demands on programme resources.

Management of programme dependencies

3.15 Most major defence equipment programmes involve engagement of a prime contractor to act as a single point of contact between the client and the supply chain. This reduces the demands on departmental teams. In two digital programmes we examined, programme teams have taken on this role due to dissatisfaction with the way that suppliers have carried it out in the past, but staff involved reported they lacked the resources and experience to do so effectively.

3.16 One of the areas providing the most frequent challenges to delivery teams is provision of Government Furnished Assets (GFA). This refers to any Departmental asset such as equipment, information or resources made available to the contractor by the Department, where it carries liability for not providing the assets at the right time and in the right condition. We found recurring problems with GFA, despite teams recognising it as a significant risk. For example, on the Type 31e frigate, to gain financial recompense, the supplier is required to show that late or defective GFA has had a detrimental impact on the delivery of the ship. The Department had still not finalised its schedule of GFA 18 months into the contract, and classed the probability of delay as high.

Departmental initiatives to improve delivery

The Strategic Partnering Programme

3.17 In 2018, the Department established a Strategic Partnering Programme (SPP) to maximise commercial leverage with its 19 most important suppliers by improving contract performance and managing strategic risks. BAE Systems is furthest into the programme. The Department and BAE Systems have jointly identified shared shortcomings in programme scheduling, pace of delivery and approval timescales, and in aligning objectives and incentives. The parties are working together on a number of workstreams intended to improve the performance of an identified programme or thematic area, for example, looking at the international supply chain feeding into the Type 26 frigate programme.

3.18 In doing this, the Department is applying well established good-practice principles, such as the need for transparency between customer and supplier, and enablers such as co-location of staff and shared access to data. These principles were incorporated in a ‘gold standard’ of how to manage the complex cultural and systemic drivers of defence programme performance developed by the National Audit Office in 2005.26

3.19 The challenge for the Department will be to ensure the programme has the capability and capacity to deliver the SPP across all 19 strategic suppliers, and to consider how the programme manages changes arising from the Integrated Review of Security, Defence, Development and Foreign Policy and the Defence and Security Industrial Strategy. To ensure widespread support for the approach, the Department will also need to develop a process to clearly track and communicate the benefits such as financial savings, schedule improvements, risks and equipment availability. It estimates that its SPP could deliver benefits exclusively attributable to this programme of £160 million over the next 10 years. In order to have a significant effect on the issues identified in this report, the SPP must deliver successful interventions across the range of strategic suppliers.

Category Management

3.20 Category Management is a strategic approach to procurement whereby organisations group together related products they buy into market ‘segments’ to generate efficiencies. It is used widely in the private sector and government. In 2020, the Department’s chief commercial officer proposed applying it across the Department with start-up funding of £51 million. According to the Department, an earlier attempt to introduce category management into the Department in 2004 yielded significant benefits but failed because the processes needed were not embedded by the time start-up funding was withdrawn. A challenge this time will be purchasing in categories that cut across the Commands (which now hold the budgets), for example where each Command holds part of the budget for that category. If the initiative is successful, the Department estimates risk-adjusted efficiencies of £628 million over the next 10 years.

21 Ministry of Defence, National Security Through Technology: Technology, Equipment, and Support for UK Defence and Security, Cm 8278, February 2012.

22 Comptroller and Auditor General, Improving value for money in non-competitive procurement of defence equipment, Session 2017–2019, HC 412, National Audit Office, October 2017.

23 Cabinet Office, Transforming public procurement, CP 353, December 2020.

24 Our analysis of programme risks included 17 programmes which reported in March 2021 and two programmes which reported in December 2020. Of the two, the first was the Warrior armoured vehicle programme, which ceased reporting following its cancellation (see Figure 8), and the second was a programme which subsequently classified its risk reporting. A further programme’s risk reporting remained classified over both reporting periods.

25 Comptroller and Auditor General, Reforming defence acquisition, Session 2014-15, HC 946, National Audit Office, February 2015.

26 Comptroller and Auditor General, Driving the Successful Delivery of Major Defence Projects: Effective Project Control is a Key Factor in Successful Projects, Session 2005-06, HC 30, National Audit Office, May 2005.

Part Four

Shortages of skilled staff

4.1 Having sufficient staff with the right skills is crucial for managing contracts and programmes effectively but is a long-standing problem for the Ministry of Defence (the Department), even though the cost of such personnel is small compared to the programmes they deliver. This part looks at:

how shortages in suitably qualified and experienced personnel (SQEP) hinder delivery of contracts;

why these problems persist, and what the Department is doing to address them; and

the challenges faced by senior responsible owners (SROs) and senior team members.

Recruiting and retaining key staff

4.2 The Department has historically struggled to staff programme teams with the personnel required to deliver programmes successfully. In 2019, we reported on skills shortfalls in the Department’s civilian workforce, including commercial and portfolio and programme management staff.27 Within Defence Equipment & Support (DE&S) and the Submarine Delivery Agency (SDA), the number of outside contractors taken on to fill vacant posts temporarily increased from 230 people in 2018-19 to 730 in 2019-20. DE&S accounts for a majority of the staff involved in programme delivery. During 2019-20, 94% of its posts were filled, but the project management and commercial functions were prominent among posts vacant. The number of people who leave each year is also above industry norms.

4.3 As at March 2021, the availability of SQEP to programme delivery teams was mixed among 19 projects and programmes we examined. Six teams reported they had no vacant posts, with all filled by either a public servant or temporary contractor.28 Figure 7 shows to what degree projects and programme delivery teams are reliant on contractors to fill their posts. Around half of the projects and programmes had little or no reliance on contractors, but some were dependent on them. In particular, all three of the digital programmes in our sample face significant challenges (paragraph 4.6). Among the programmes we examined, lack of SQEP was the risk most frequently cited (across nine programmes on 12 occasions) as being ‘high’ or ‘very high’ probability; five programmes expected seven SQEP risks would remain ’high’ or ‘very high’, even after proposed mitigating actions.

Responding to shortfalls in staffing

4.4 When seeking to recruit and retain personnel, the Department must comply with wider public sector pay constraints. Defence organisations can pay allowances if they cannot recruit to particular posts, but the burden of proof is high and the need regularly reassessed. DE&S and the SDA are both able to set pay strategies independent of the civil service. However, this does not mean that they can compete with the private sector or are immune to wider departmental cost constraints.

4.5 The Department will continue to rely on contractors for the foreseeable future, and it will always require the use of external contractors to some degree, for example for specialist skillsets for which it would not be cost-effective to maintain permanent in-house capacity. To reduce costs, it is establishing relationships with private sector partners. For example:

in August 2020, it let a four-year contract for commercial resources across a range of activities, including procurement and contract management, at a cost of up to £216 million; and

Defence Digital is producing a business case for a Digital and IT Professional Services Framework to develop long-term relationships with firms able to supply suitable staff.

Particular challenges for digital programmes

4.6 Under the previous Defence Information Strategy, Defence Digital intended to take more responsibility for system architecture, design and integration, and reduce its dependence on large prime contractors. However, evidence from the three digital programmes we examined indicated that the Department has failed to recruit enough people with the skills and experience required to manage the various suppliers in this way.29 Only 26% of funded delivery team posts were occupied by public sector staff in March 2021. Contractors filled 48% of funded posts, while 27% were vacant. For example, the Skynet 6 satellite programme is finding it difficult to rebuild capacity for space-based capability after the Department let its predecessor under a private finance initiative contract, thereby losing in-house capability. Delivery teams have also raised concerns that Defence Digital is not currently set up to provide programmes with the commercial support that is available to the Department’s strategic suppliers.

4.7 Defence Digital programme staff told us that they suffered in comparison with other delivery agents by not having the same pay freedoms, and their main site finds it difficult to attract staff, especially as it is close to DE&S’s Bristol headquarters. However, they also told us the recent experience of remote working has shown this is a viable business model which could aid recruitment. DE&S has developed its own digital strategy and it has a plan to collaborate with Defence Digital on the development of digital skills.

4.8 As part of a wider digital transformation programme, the chief information officer has developed a Strategic Workforce Plan. The objectives are to reskill and redeploy the existing workforce; recruit staff with missing critical skills; and rationalise the use of bought-in contractors, with contracts linked to outcomes. The estimated cost is £34 million over three years, with anticipated benefits of hundreds of millions of pounds. In mid-2020, Defence Digital also began a programme of programme delivery professionalisation.

Role of the SRO and team

4.9 The SRO has overall accountability to Parliament for ensuring a programme meets its objectives and delivers projected benefits. SROs oversee governance of the programme and are responsible for steering it through the various key decision points, assisted by a delivery team, led by the programme director (PD) and programme manager (PM). Day-to-day management of the supplier rests with the delivery agents, most commonly DE&S, which commits to deliver to the SRO the elements of the programme for which they are responsible as set out in the business case, such as physical or digital assets.

4.10 Our analysis of 19 of the 20 programmes we examined showed that, as at March 2021, the median time in post for an SRO was 22 months and SROs were typically expected to spend 25% of their time on the programme.30 PDs had, on average, been in post for 15 months and spent 60% of their time on the role. The median PM had been in post for 13 months, and most are full-time. Separately, project managers within DE&S had been in post for 24 months. The tenures of the key officials contrast with the median running time of 77 months for the programmes we examined. This rate of turnover reflects the career path requirements of the armed forces, of which most postholders are members. The Infrastructure and Projects Authority (IPA) commented during a 2018 review that many programmes had suffered through the career development of individuals being put before success of a major programme.

4.11 In 2020, the Department’s Project Delivery Centre of Excellence carried out a survey of 66 defence SROs as part of a review of the challenges SROs face. Of those who responded, 72% said they do not feel empowered to make decisions about their programme’s funding, and 46% do not feel empowered to make decisions about scope. Some were uncertain about the relative responsibilities of the delivery team, delivery agents and other stakeholders. Moreover, SROs reported that they felt least confident in areas important to supplier management such as conflict resolution, collaboration and influencing. These are areas which are important in building and maintaining effective relationships with suppliers and other stakeholders. The review also identified that there had been a high turnover of SROs in some major programmes, and that time devoted to each defence programme was on average lower than in other government departments. Further work identified a lack of career management for SROs within the military appointments process. Despite the IPA’s expectation that departments should support professional capability training, some SROs had difficulty accessing the Major Projects Leadership Academy courses underpinning preparation for the role.

Increasing skills in the specialist workforce

4.12 The Department has plans under way to increase the skills and capabilities of its workforce in key aspects of commercial management and programme delivery. In particular:

the Department’s Commercial Function is rolling out commercial training. It believes the Department needs 362 accredited commercial ‘experts’ and a further 1,383 ‘practitioners’. As at January 2021, 42% of the required staff had completed ‘expert-level’ training and 48% had completed ‘practitioner-level’ training;

the Department has let a contract for a commercial delivery partner, operational from August 2020, to provide a pool of SQEP available to address urgent staffing requirements and longer-term recruitment and resourcing difficulties;

the Department’s Project Delivery Function is undertaking work to identify delivery professionals’ skills and competencies in order to develop career paths supported by training; and

the Department’s Cost Assurance and Analysis Service is seeking to improve the quality of cost assurance and cost management capability available to delivery agents and the Commands.

27 Comptroller and Auditor General, Reforming the civilian workforce, Session 2017–2019, HC 1925, National Audit Office, March 2019.

28 The six programmes without any vacant posts were the Ajax armoured vehicle, Crowsnest radar system, Morpheus, Type 26 frigate, Poseidon P-8A maritime patrol aircraft and Complex Weapons.

29 Skynet 6, New Style of IT (Deployed) and Morpheus (part of Land Environment Tactical Communications and Information Systems programme).

30 The Department ceased reporting on the Warrior vehicle upgrade programme following its cancellation (see Figure 8). This programme was therefore not included in this analysis.

Part Five

Delivering value for money through the life of the contract

5.1 Achieving value for money in defence acquisition involves getting as close as possible to achieving the optimum balance between cost, schedule and performance. This involves clearly defining what a good outcome would look like up-front, and monitoring whether this is actually achieved as the programme goes on contract and, ultimately, is delivered. This part examines:

how the value for money of programmes is defined and monitored, including taking account of the government’s social value agenda; and

whether the Ministry of Defence (the Department) uses evaluations to improve value for money in the future.

Defining and monitoring value for money during the procurement process

Defining value for money

5.2 At key decision points, each project or programme must satisfy the Department’s senior approving authority, the Investment Approvals Committee (IAC) and its scrutiny teams, that proposed solutions will be ‘value for money’ (VFM) in terms of cost, time to enter service, and ability to meet capability requirements. The programme team should reaffirm that the programme remains VFM if it has breached the originally approved parameters. Defining clear parameters for VFM on a programme will become more challenging with the wider adoption of ‘agile’ contracting techniques, where cost, time and performance parameters are always negotiable.

5.3 We found that business cases for the programmes we examined did not always set out a clear definition of VFM against which progress could be measured. In practice, an acceptable solution involves trade-offs between resources available and the desired outcome. As discussed in Part Two, the affordability of the programme within the overall portfolio is often a key constraint, with schedule and capability traded around it. For example, in 2015, the senior responsible owner (SRO) for the Challenger 3 tank upgrade programme put forward a solution which fitted the existing budget but did not meet key user requirements. The IAC approved a solution that did meet key user requirements for each tank in March 2021, but at a cost 80% higher than the original estimate, and for the upgrade of a reduced number of hulls.

5.4 A recent review by HM Treasury of how government approaches business cases requires the Department to take account of wider government strategic priorities, when considering whether bids from suppliers will deliver value for money.31 The competition to build the Fleet Solid Support ship fits with these aspirations. Tender evaluation will include how bidders’ proposals will add to UK ‘social value’, without being prescriptive as to the approach to achieving this.32 At the same time, the government has also directed that a significant proportion of the build and assembly work be carried out in the UK. Quantified claims of job creation need to take into account wider economic effects, such as the fact that in most cases the highly skilled people involved would find employment elsewhere in the economy.

Contract cost and VFM

5.5 Although in theory, competition between suppliers can provide evidence that the Department is achieving the optimum cost for the programme, for various reasons it is often absent for the largest, most complex programmes (as explained in paragraph 3.2). Competition is useful for exposing unrealistic assumptions by the Department. Teams for both the Type 31 frigate and the Fleet Solid Support ship had to rethink their first attempts at a competition, as suppliers did not consider the required level of capability deliverable within the cost parameters set by the Department.

5.6 In the absence of competition, the Department carries out comparisons against VFM benchmarks, but this is not an exact science. In two cases we examined – the Skynet 6 military satellite and the Fleet Solid Support programme – the proposed cost was higher than the VFM benchmark. In both cases, the SROs questioned the quality and completeness of the benchmark, for example where benefits are not easily monetised. The Department subsequently approved both business cases (in the latter case, subject to satisfactory completion of negotiations with suppliers). In addition, a benchmark is not always available. For example, supplier ownership of intellectual property rights, or the nature of international agreements, might mean the Department does not have access to actual costs.

Single source contract regulations

5.7 To achieve assurance about the bids submitted in non-competitive situations, the government introduced the Single Source Contract Regulations (the Regulations) under the Defence Reform Act 2014. These were designed to enhance the Department’s ability to challenge suppliers’ non-competitive bids by improving their transparency, except where – notably with international procurements – they are exempt. In 2017, we reported that the Regulations offered considerable opportunities to improve contract management if implemented and applied effectively.33

5.8 In some of the programmes we examined, we saw teams gain assurance on costs provided by suppliers through the Regulations as evidence that single-source bids are VFM. The Department also used the Regulations to test and remove costs, and to challenge suppliers’ indirect costs such as labour rates and corporate overheads, which underpin prices on individual contracts across a number of years. The full benefits of the Regulations will take some time to become apparent. The Department is currently reviewing how to further develop the Regulations and is considering how they could be used to incentivise suppliers to innovate, take on more programme risk and support wider government objectives.

Monitoring VFM once the programme is on contract

5.9 Despite the delays, cost increases and other challenges faced by the programmes we examined, none were subsequently seen as poor VFM. In 2016, departmental scrutineers did comment that the VFM of the Type 26 frigate had been ‘eroded’ due to the inability to find an affordable solution for the original requirement of 13 ships. Typically, once the Department has entered into a relationship with a supplier, ending the arrangement and restarting the process would provide additional delays and affordability problems, so the current arrangement is often described as offering the ‘best’ VFM in the circumstances. Figure 8 illustrates that programmes can incur considerable expenditure over a number of years without clarity about VFM.

5.10 As described in paragraph 2.9, the programmes we examined were prone to slippages in their estimated dates of entry into service once on contract. The Department has not indicated that such slippage affected VFM, despite the loss of value to the Department from the absence of these capabilities, leading to a need to manage obsolescence or capability gaps for longer periods. Our sample of programmes included the armoured vehicles reported on by the House of Commons Defence Committee in March 2021. The Committee stated that the absence of these new and upgraded vehicles would leave the Army seriously overmatched in any conflict.34

5.11 HM Treasury directs that accounting officers should confirm to Parliament that procurements remain VFM where there are significant changes to cost and time parameters. Besides the Warrior armoured vehicle, there were seven instances of accounting officer assessments among the programmes we examined:

The assessments for the Future Maritime Support Programme (July 2019), Type 31e frigate (January 2020), and Fleet Solid Support ships (November 2020) all stated that VFM would be delivered in the future.

The assessment for the Skynet 6 military satellite (October 2019) stated that VFM would be determined when each element of the programme reached its main investment decision. We found the programme had previously stated that VFM would be achieved through competition, but that Departmental delays to the subsequent declaration of a UK-sovereign capability only allowed for a single source solution.

The assessment for the Protector unmanned aerial vehicle (November 2019) stated the programme remained VFM despite a two-year delay and increased costs of £326 million.

The assessment for the Ajax armoured vehicle (October 2020), stated the programme remained a VFM solution despite slippage of entry into service from July 2020 to June 2021, with a worst-case scenario of slippage to December 2022.

The assessment for the A400M transport aircraft (March 2021) stated only that the programme remains within the most recent cost boundaries set by the approving authority.

Learning from experience

5.12 The main purpose of evaluation is to ensure good practice is perpetuated, lessons learned and costly mistakes avoided. Existing guidance is clear on its importance and benefits. Despite this, the Department has not systematically gathered and distributed lessons from experience (LFE) from programmes. In December 2020, it established a central register of LFE. The Department told us that, by April 2022, it plans to enhance central assurance of major programmes, improve evaluation capability, embed cross-Department lesson learning and learn more from NATO allies.

5.13 Programme teams’ current analysis of the risks they face indicates that they are aware of many of the challenges discussed in this report. In the programmes we examined, most had identified lessons, although some much more systematically than others, including from relevant programmes in other Commands. The Warrior upgrade team reflected that earlier collaboration to learn lessons between different parts of the programme team and with the Ajax armoured vehicle team would have improved the delivery of the programme. We identified examples of the same lessons being identified across programme teams, including a lack of appreciation of programme complexity, establishing requirements, users changing their requirements, the need to have sufficient resourcing and skills in place (Part Four) and shortcomings in delivering Government Furnished Assets (paragraph 3.16). We also identified some good practice in learning from previous programmes, including how the Challenger 3 tank programme drew from the experience of the Warrior and Ajax armoured vehicle teams around control of the design phase. In addition, the Fleet Solid Support programme is capturing and incorporating LFE from within the Department and 70 external sources, with 25% of lessons resulting in changes to the specification.

31 HM Treasury, Green Book Review 2020: Findings and response, CP331, November 2020.

32 ‘Social value’ represents the additional social benefits that can be delivered through government contracts, to achieve policy outcomes aligned with government priorities.

33 Comptroller and Auditor General, Improving value for money in non-competitive procurement of defence equipment, Session 2017–2019, HC 412, National Audit Office, October 2017.

34 House of Commons Defence Committee, Obsolescent and outgunned: the British Army’s armoured vehicle capability, Fifth Report of Session 2019–2021, HC 659, March 2021.

Appendix One

Our audit approach

1 Our study examined whether the Ministry of Defence (the Department) optimises value for money (VFM) through its commercial and programme management arrangements for its equipment. We considered:

whether the Department is working with suppliers at the strategic level to improve the performance of its major equipment contracts;

whether the Department is following good practice in programme and supplier management in executing its major equipment programmes; and

whether the Department clearly defines what VFM looks like in equipment contracts and monitors VFM throughout the equipment lifecycle.

2 This report follows on from our report Defence Capabilities – delivering what was promised, which reported that programme senior responsible owners identified supplier performance as a major threat to delivery. Issues have persisted for decades and the Department has introduced many initiatives to improve commercial capability and engagement.

3 Our audit approach is summarised in Figure 9. Our evidence base is described in Appendix Two.

Appendix Two

Our evidence base

1 We reached our conclusions based on evidence gathering and analysis conducted between October 2020 and May 2021. The main approaches to evidence gathering are set out below.

Review of existing literature

2 In designing our study, we drew on knowledge and insights gained from our previous reports, particularly Defence Capabilities – delivering what was promised,35 as well as our annual reports on the Department’s Equipment Plan; Carrier Strike – preparing for deployment;36 and past National Audit Office (NAO) work on major programmes delivered elsewhere in government.

3 To draw on a wider range of literature, we commissioned RAND Europe, a not-for-profit research institute with considerable experience of working in the defence field, to prepare a paper for us on ‘Persistent Challenges in UK Defence Equipment Acquisition’. This was designed to shed light on the underlying causes of equipment procurement problems and the reasons why they persist despite repeated efforts to learn from the past. This short paper draws on decades’ worth of public and non-public RAND analyses of US, UK and European defence acquisition programmes and input from senior subject matter experts. This paper, which provided confirmation of many of the themes identified in our own work, is available at: www.rand.org/pubs/research_reports/RRA1174-1.html

Interviews

4 We undertook interviews with senior staff in many parts of the Ministry of Defence (the Department). These included: Air Command, Army Command, Commercial Function, Defence Digital, Defence Equipment & Support (DE&S), Defence Portfolio and Approvals Secretariat, Defence Safety Authority, Finance Function, Navy Command, Submarine Delivery Agency and the Department’s Cost Analysis and Assurance Service.

5 These interviews provided a comprehensive overview of the challenges to effective contracting in defence, including commercial capability, management information and safety accreditation. They also provided us with an understanding of recent developments in the Department such as the Strategic Partnering Programme and Approvals and Acquisition Transformation Programme. In addition, we met with the chair and non-executive members of DE&S to gain their perspective on contract performance.

6 We met with senior staff with the following defence contractors: Airbus Defence and Space, BAE Systems, Lockheed Martin UK, MBDA UK, MSI Defence Systems and Thales UK. These interviews were designed to give us an understanding of the challenges from a supplier’s point of view of contracting with the Department. They informed the development of our questions for the Department. We provided suppliers mentioned in this report with the opportunity to comment on factual accuracy.

7 We also spoke to a number of other organisations to gain their perspectives on defence contracting. These were: Association of Defence Suppliers; the Infrastructure and Projects Authority; HM Treasury; and the Single Source Regulations Office.

Examination of 20 major equipment programmes

8 We examined 20 of the Department’s 35 major programmes in the Government Major Projects Portfolio to examine in detail the challenges faced (Figure 10). The programmes we examined included:

competitive procurements (for example, the Type 31e frigate), UK single source procurements (for example, the Astute attack submarine) and single source foreign military sales (for example, the P-8A Poseidon maritime patrol aircraft);

innovative digital and research and development programmes, as well as more traditional military platforms;

programmes from across all the Commands; and

some high- and lower-performing programmes, as well as programmes at different stages of the acquisition life-cycle.

9 We engaged with all 20 programme teams. The main areas covered varied but included: the current state of the programme; the main challenges and risks to delivery; relations with suppliers; the impact of recent departmental reforms; resourcing and affordability challenges; success factors; and good practice.

10 Our approach to examining each programme varied depending on the extent of our existing knowledge and the nature of the programme (for example, whether competitive or single source, or sourced through foreign governments). We requested a common list of documents and data from programme teams, as well as further documents depending on the specific issues on each programme. Common documentation included documents/logs covering issues such as early market engagement, commercial strategies, investment appraisals, supplier management and risk identification and management.

11 In this study we looked only at a selection of the most significant and complex programmes within the Department’s Major Projects Portfolio as at July 2020, which were classed by the Department as equipment capabilities. Within this group we excluded the following equipment programmes:

F-35 Lightning programme – covered in depth in 2020 in the NAO reports on Defence Capabilities and Carrier Strike.

Queen Elizabeth carriers – covered in depth in 2020 in the NAO report on Carrier Strike. This programme also left the portfolio during the fieldwork stage of our study.

Dreadnought ballistic missile submarine – national security sensitivities would prevent disclosure of information. We intend to revisit the Department’s nuclear enterprise work in the future.

12 Not all of the programmes we examined are included in each piece of analysis within the report as they are at different stages of development and delivery, and some do not follow the standard departmental procurement cycle with comparable decision points.

13 Since the programmes we have examined only constitute a small proportion of the number of projects and programmes which are being delivered by the Department, we have not sought to use our findings to comment on the Department’s delivery of its whole equipment portfolio. However, given the fact that the programmes covered account for a significant proportion of the value of the whole portfolio, as well as their importance to national security, what happens to these programmes has considerable significance for the Department and taxpayer.

35 Comptroller and Auditor General, Defence Capabilities – delivering what was promised, Session 2019–2021, HC 106, National Audit Office, March 2020.

36 Comptroller and Auditor General, Carrier Strike – preparing for deployment, Session 2019–2021, HC 374, National Audit Office, June 2020.

Design and Production by NAO External Relations
DP Ref: 009120-001