Report

by the Comptroller
and Auditor General

Ministry of Defence

Defence capabilities – delivering what was promised

HC 106 Session 2019–2021 18 march 2020

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office (NAO) helps Parliament hold government to account for the way it spends public money. It is independent of government and the civil service. The Comptroller and Auditor General (C&AG), Gareth Davies, is an Officer of the House of Commons and leads the NAO. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether government is delivering value for money on behalf of the public, concluding on whether resources have been used efficiently, effectively and with economy. The NAO identifies ways that government can make better use of public money to improve people’s lives. It measures this impact annually. In 2018 the NAO’s work led to a positive financial impact through reduced costs, improved service delivery, or other benefits to citizens, of £539 million.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 16 March 2020

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Gareth Davies
Comptroller and Auditor General
National Audit Office

12 March 2020

This study examined whether the Ministry of Defence (the Department) gets the capabilities it requires when it needs them to meet its current and future defence objectives.

© National Audit Office 2020

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.org.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

006572 03/20 NAO

The National Audit Office study team consisted of:
Loreen Ash, Rebecca Brooks, Jemma Dunne, David Evans, Matthew Fright, Jeremy Gostick, Emma Helland, Jake Hutt, Anna Kennedy-O’Brien, Liam Schmidtke, and Rowena Thomas, under the direction of Jeremy Lonsdale.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key facts

Summary

1 The Ministry of Defence (the Department) develops and operates military capabilities in order to meet its strategic requirements and objectives. A military capability is not simply a piece of equipment such as a tank. Rather, it is a tank with a trained crew that: can communicate with others on the battlefield; can meet identified threats; and can be properly maintained and repaired during its lifetime. The Department estimates that around 20,000 civilians and military personnel within the Department are involved in delivering such defence capabilities.

2 The military capabilities are intended to meet the strategic requirements that underpin UK defence policy, as set out in the National Security Strategy and five-yearly Strategic Defence and Security Reviews. Supporting this work are processes for identifying strategic threats and continuously analysing the UK’s ability to meet them. Where the Department concludes there are current or future gaps in its ability to counter these threats, it must decide whether to fill the gap or carry the risk. However, once the Department has decided the gap must be filled with a new or replacement capability, it has historically struggled to deliver fully functioning capabilities to schedule.

3 Where the Department decides to fill a capability gap (or prevent one appearing), a Senior Responsible Owner (SRO) in one of the Commands (Air, Army, Navy, Strategic (formerly Joint Forces)), or the Defence Nuclear Organisation (DNO), oversees delivery of the capability.1 The user requirements are tracked across eight ‘defence lines of development’, including the equipment itself, the relevant training and the infrastructure necessary to support it. A series of key milestones to delivering the capability underpin the project or programme.

4 As at 30 September 2019, there were 32 programmes defined as capabilities in the Department’s Defence Major Projects Portfolio (DMPP) – those programmes the Department considers the most complex and strategically significant, its top priorities. These programmes have an estimated total procurement cost of £196.2 billion. Thirty of these programmes were also in the Government Major Projects Portfolio as at 31 March 2019. The DMPP programmes are overseen by the centre of the Department (known as ‘Head Office’). Less significant programmes are monitored at Command level under the Department’s delegated organisational model. The Department has a range of initiatives under way that are intended to transform the system to approve and monitor acquisitions.

5 The focus of our study is on the delivery of defence capabilities, from the point at which the Department sets the requirement and starts the acquisition process through to the capability being declared fully operational. The work complements our review of the Department’s Equipment Plan, which focuses on the cost of these capabilities.2 In particular, this report considers whether the Department secures the capabilities it requires when it needs them. To do this, we examined:

the extent and causes of delays and shortfalls in bringing capabilities into service (Part One);

the completeness of the Department’s system to monitor the delivery of capabilities (Part Two); and

whether the Department is putting in place appropriate arrangements to transform its capability delivery in the future (Part Three).

6 The study does not examine why and how the Department derived a particular user requirement. We have also not looked in detail at the role of delivery organisations, such as Defence Equipment & Support, which act on behalf of the Commands (the customer). We set out our audit approach and evidence in Appendices One and Two.

7 Our report combines an overview of the Department’s system for delivering the capabilities, with a detailed review of a sample of eight programmes from across the Commands (Figure 1 and Appendix Three). We selected these programmes to enable us to examine the processes in place in more detail. Six of them sit outside the DMPP, thus providing us with insight into the majority of capability acquisitions which are not covered by the Department’s central monitoring. They are not intended to be representative of all the Department’s programmes. Instead, given the limited number of projects which passed through milestones during the sample period, they do illustrate how the Department’s processes are currently being applied.

Key findings

Delays to capability delivery

8 The Department currently identifies major risks to the timely delivery of nearly one-third of its most significant capabilities. Delayed delivery can affect the Department’s ability to meet external threats to national security, leave gaps in capability and require use of existing capabilities beyond their normal lifetime at increased cost. As part of the background work to its transformation initiatives, the Department carried out research that showed delays to capabilities are now less than they were 10 years ago. However, 10 of the 32 current most significant capabilities are rated by their SROs as either requiring ‘urgent action’ to address major risks to delivery, or being ‘undeliverable’ to the approved timescale. In contrast, the Department forecasts successful delivery is ‘probable’ or ‘highly likely’ for five of the 32 capabilities. Across the most significant capabilities there is an average delay of 12 months for those projects yet to reach Initial Operating Capability (IOC) and an average delay of 26 months for capabilities which are not yet fully operational. Other countries also experience significant delays in the delivery of capabilities (paragraphs 1.4, 1.5 and 1.8 to 1.10 and Figures 2 and 3).

9 The Department faces specific recurring challenges which affect the delivery of capabilities, of which the most persistent is late or faulty equipment delivery by the supplier. Our case studies and other analysis show that a range of factors affect delivery of each capability. In some cases, poor performance by suppliers has persisted over a number of years and has compromised the operation of capabilities. SROs of 10 of the 32 most significant capabilities currently being delivered report serious concerns about supplier engagement or delivery performance, from poor quality control to lack of transparency about progress. The suppliers involved in delivering these capabilities were predominantly a mix of UK firms and subsidiaries of non-UK companies (paragraphs 1.11 to 1.14 and Figures 4 and 5).

10 Commands and delivery teams lack capacity and skills, which affects delivery of the capability. Under-resourced project and delivery teams contribute to delays in delivery. Six of the 32 DMPP capabilities face shortfalls of more than 20% in their programme teams. Our 2019 report Reforming the civilian workforce drew attention to the issue of a lack of suitably qualified and experienced personnel in the Department.3 There are particular shortages of engineering and commercial staff, and some teams are very reliant on consultancy support. SROs for nine of the 32 DMPP capabilities consider that a lack of team capacity and skills is affecting capability delivery. SROs themselves are crucial, being personally responsible for delivering the capability. A shortfall in the number of SROs has led to some individuals being spread across multiple projects. The Department is trying to address these gaps (paragraphs 1.15 and 1.17).

11 The Department’s introduction of capabilities is also hindered by an inability to provide appropriate training when required. The delivery of four of our case studies was affected by a lack of capacity and capability in providing necessary training. Reasons included a lack of appropriate personnel, funding shortages or insufficient equipment. For example, in our Falcon Early Entry Capability case study, reduced capacity within the organisation intended to deliver training led to additional expenditure on a short-term alternative (paragraph 1.18).

12 The Department’s funding challenges affect individual capabilities and the whole portfolio. Our case studies included upgrades to existing capabilities which had been delayed to fund other priorities or where implementation was hindered by funding shortages. For example, a shortfall in training funds affected the upgraded Base Inventory Warehouse Management System, and infrastructure funding delays affected the F-35 fighter jet. We have reported for the past three years that the Department’s overall equipment portfolio is unaffordable (paragraphs 1.19 and 1.20).4

Under-delivery of capability at key milestones

13 The Department declares key project milestones as achieved, without the intended capability always being delivered at that point. Departmental guidance permits the declaration of a milestone even if performance does not meet acceptance criteria, or if testing to confirm criteria have been met is incomplete. The Department allows exceptions for a variety of reasons, but the most frequently used in our case studies was that progress was “good enough”, despite criteria not being met. In some cases, this affected the Department’s ability to use the capability in the way intended. For example, in the case of the F-35 fighter jet (which accounted for two-thirds of exceptions in our case studies), delays to the provision of synthetic training facilities affected the availability of trained pilots and maintainers. Exceptions should be granted on the basis that there is a timebound plan for their resolution, but this was not the case in most of our sampled case studies. It was apparent that several of the capabilities needed further development work due to the original specification not meeting the user’s capability needs (paragraphs 1.21, 1.23 and 1.24 and Figure 7).

14 We identified some examples where the definitions of milestones left the Department unclear what level of capability had actually been achieved against the specified requirements. A lack of clarity about the level of capability generated during acquisition can be caused by the milestone criteria not being sufficiently well defined. For example, in the case of the Type 45 destroyer, the SRO reported that it was unclear what level of capability had been achieved when the ships were declared in service between 2010 and 2013. This was because there was no single statement of the capability required at a particular stage. Even when a capability has technically reached Full Operational Capability (FOC), this does not mean it can do all that is required of it. For example, our case study of Watchkeeper, the Department’s unmanned aerial vehicle, illustrates how significant issues remained to be addressed following declaration of FOC (paragraphs 1.21 to 1.23 and Figure 6).

Monitoring progress of capability delivery

15 Head Office currently does not have the information it needs to hold Commands fully to account and make strategic decisions. Easy access to consistent information across the entire portfolio is vital for holding Commands to account and for making strategic decisions. Head Office has a strategic role in capability delivery and therefore concentrates on the DMPP. The Department has a central data system for collecting information on this subset of programmes, but it does not have ready access itself to information on all capability projects across the Department and has not ensured that the Commands have it. The DMPP accounts for more than 40% of the Equipment Plan over the next 10 years. It is not always clear why particular capability projects are excluded from the DMPP. There are limitations with the central data collected, which can be out of date when presented to senior decision-makers (paragraphs 2.1, 2.6 to 2.8 and Figure 8).

16 Commands’ portfolio management arrangements are at different stages of development, which could affect their ability to hold SROs to account. The portfolio offices within the Commands vary in their maturity and the roles they carry out. Commands are upgrading the capability of their portfolio offices (or in the case of the DNO, developing one for the first time), but all have staffing gaps. Commands do not all have processes for measuring capability acquired against original requirements, although all have a forum for reviewing the progress of the Command portfolio. Nor do they all have information on all the capability projects and programmes within their portfolios, and they use different tools to collect this information, which may duplicate the centrally approved system. Not all Commands centrally manage the relationships and linkages between different projects, systems and activities. In selecting our case study examples, we found errors in Command-level information which, if repeated more widely, could provide a misleading picture of capability delivery (paragraphs 2.11 to 2.14 and Figure 9).

Transforming capability delivery

17 The Department has introduced capability transformation initiatives, but these will not address all the recurring issues with delivery. The Department has a portfolio of transformation projects, including several linked to capability delivery. Some of these initiatives are designed to improve the quality of thinking around milestone setting and monitoring, as well as to speed up the acquisition process. Potentially the initiatives could have prevented problems in our case studies where, for example, significant delays in delivering capabilities threatens their continued relevance. However, the initiatives would not in themselves have prevented some of the problems identified in our analysis, such as inadequate contractor performance or failure to deliver timely training. The Department’s transformation initiatives are constantly evolving and run the risk of not being coordinated (paragraphs 3.1, 3.3 to 3.8 and 3.10 and Figure 10).

18 There is a risk that the Department’s understandable desire to respond to the challenges of modern capability acquisition, encourages a culture in which capability levels are overstated or opaque. In late 2018, the Department’s acquisition system review identified that its organisational culture prioritised the passing of milestones and securing approvals over the delivery of outputs and outcomes. The Department has concluded that the speed of technological change and the evolving nature of the threats it faces mean that its current acquisition processes are no longer fit for purpose. The Department wants to make capability acquisition quicker and more responsive through the introduction of a more rigorous business case process and by providing more flexibility in procurement routes for new technologies. The latter will involve a constant cycle of capability upgrades, which is not compatible with the current milestone-setting process. These objectives are important, but it is essential that it does not create a culture that encourages premature declarations of achievement of capability, or leaves it unclear what level of capability is available (paragraphs 3.4, 3.8 and 3.9).

19 The Department has introduced new processes to identify shortfalls in capability readiness, but their success is dependent on addressing the wider affordability challenge. Head Office has processes in place to assess both current and future capability, and the level of readiness of that capability. The additional mechanisms introduced, such as the Capability Readiness Assessment Framework, are intended to strengthen the ability of Head Office to monitor capability challenges and ensure they are addressed. However, addressing capability readiness issues is contingent upon having funding available. The Department already faces an affordability gap for its existing capability commitments and no new capabilities can be funded currently without reducing existing capability commitments further (paragraphs 3.11 and 3.12 and Figure 11).

Conclusion on value for money

20 The Department delivers complex, long-term capability programmes to meet the threats which it has identified. However, at a time of fast-paced technological and political change, it is essential that it can make swift and full use of these capabilities as planned. Failure to do so is likely to undermine the Department’s ability to carry out its key tasks, and lead to existing assets being used for longer and additional costs. To achieve value for money, the Department must deliver capabilities to performance, cost and time consistently within a challenging funding envelope. While the Department may be able to deliver some individual capabilities in ways that deliver value for money, the frequent delays, problems with the quality of what is being delivered and poor monitoring information mean the Department has not achieved it for the portfolio as a whole.

21 In response to the challenges, the Department is currently implementing changes to its acquisitions and approvals systems. For these to be successful, the Department must change the culture around capability delivery in a number of ways, including ensuring that pressure to be seen to deliver capabilities quickly does not distort accurate reporting of progress. The Department also needs to address the affordability gap in the overall defence budget, as this affects its ability to maintain and enhance capabilities.

Recommendations

22 The Department needs to improve its delivery of defence capabilities to ensure it can meet the demands of modern warfare. We recommend:

a Head Office should ensure it has the capacity and capability to hold the Commands to account more effectively for capability delivery through a fully developed central portfolio office function. Management information systems should be aligned with need, provide an overall view of performance which is timely and consistent, and promote transparency at all levels.

b The portfolio office should carry out a root cause analysis to establish the causes of time slippages across the capability acquisition portfolio, and the extent to which particular initiatives will address endemic issues in capability acquisition. The lessons from this work should be considered by the Department’s Board.

c The Department should be clear about what good capability delivery will look like following implementation of its current transformation programmes, and set out when benefits will be expected to appear. Guidance, particularly around milestones, should be updated to reflect any transition to a faster, more flexible procurement approach in a way that ensures that the current status of capabilities is clearly understood at all levels of the Department.

d The Department should instil a culture where exceptions are seen as just that – exceptional. The Department should require the communication of any exceptions on the most significant capabilities to the central approving authority prior to declaration of the milestone being achieved, together with an explanation of the implications for the future delivery timetable and budget for the capability. The approving authority would then be able to evaluate whether what has been achieved to date is consistent with what was approved.

e In its guidance and training to SROs and their teams, the Department should emphasise the importance of recognising and avoiding optimism bias in reporting of progress. It should also ensure SROs have enough time to carry out their roles when appointing them and look at how they can balance competing demands.

1 Where we refer to Commands this also includes DNO.

2 Comptroller and Auditor General, The Equipment Plan 2019 to 2029, Session 2019–2020, HC 111, National Audit Office, February 2020.

3 Comptroller and Auditor General, Reforming the civilian workforce, Session 2017–2019, HC 1925, National Audit Office, March 2019.

4 See footnote 2.

Part One

Delays bringing capabilities into service

1.1 This report examines how effectively the Ministry of Defence (the Department) is delivering the military capabilities it has decided it needs against the milestones that it sets itself.5 It is intended to complement our work on the affordability of those capabilities delivered through the Department’s 10-year Equipment Plan.6 Our report on the 2019–2029 Equipment Plan concluded that the Department continues to manage budgets to address urgent in-year affordability pressures, and is introducing new commitments without fully understanding the impact on the affordability of the Plan.7

1.2 This report covers:

current departmental performance in delivering capabilities and acceptance of key milestones (this Part);

the Department’s oversight of the challenges to delivery (Part Two); and

how the Department is transforming capability delivery (Part Three).

Why this matters

1.3 In order to maintain or deliver future capabilities, the Department has allocated a budget of £181 billion to equipment and support projects alone for the period from 2019 to 2029, 42% of its entire budget for the period. The Department has estimated that around 20,000 civilian and military personnel are involved in delivering defence capabilities. In 2018-19, the Department’s accounts stated that it had spent £32.4 billion (85% of the £38.1 billion total defence spend) on all activities linked to the provision of defence capabilities.8,9 The Department cannot provide a more specific figure for the costs of bringing into service new capabilities.

1.4 It is crucial for the security of the UK that the Department can bring capabilities into service as planned, and that they work as intended. The need for effective capability delivery is becoming more urgent due to pressures on resources and rapid developments in technology. Failure to respond to these pressures and deliver on time is likely to lead to capability gaps, overextended use of existing assets and additional costs (for example, the additional cost of the Offshore Patrol Vessels due to delays in the Type 26 programme – see Figure 5). The Department itself has identified issues with delivery to time and is currently undertaking a number of reviews designed to improve capability delivery, including bringing the Department more into line with approaches across government (see Part Three).

1.5 For many years, National Audit Office (NAO) reports have noted long delays in the Department delivering new or upgraded capabilities in line with key milestones. For example, in The Major Projects Report 2010 we highlighted delays to the delivery of the Queen Elizabeth Class aircraft carriers and reductions in the level of capabilities being delivered.10 The Committee of Public Accounts has also reported on these issues, and in March 2010 concluded that “Delaying projects once they have started increases costs, postpones the delivery of military capability and puts the Department at a disadvantage as it tries to secure value from its commercial partners.”11 The Department believes that its performance in delivering the capability it needs has improved in the last 10 years, although it is difficult to compare current performance with past eras on a like-for-like basis. In 2009, a report commissioned by the Department reported that programmes were delivered 80% later than initially expected.12 In 2019, the Department’s analysis indicated that the average had reduced to 46%.

Current departmental performance in delivering the most complex and strategically significant capabilities

1.6 The Department monitors those programmes it considers its top priorities – the most complex and strategically significant – through its Defence Major Projects Portfolio (DMPP).13 As at 30 September 2019, there were 32 programmes in the DMPP which the Department classed as ‘capabilities’, with a forecast total procurement cost of £196.2 billion.14 Thirty of these were also in the Government Major Projects Portfolio as at 31 March 2019.15

1.7 Progress in delivering capabilities is monitored through key milestones. The two most common milestones, which we discuss in this report, are Initial Operating Capability (IOC) and Full Operating Capability (FOC). IOC is the minimum level at which the capability or service is usefully deployable. The Department expects this to be the same as the In Service Date (ISD), although they can be different for certain capabilities, for example, in our HMS Forth case study. FOC is the level of military capability which is intended for a particular project. Responsibility lies with the Command to decide what level of military deployable capability this will represent.16

1.8 As at 30 September 2019, 10 of the 32 capabilities in the DMPP (31%) were rated in the categories of ‘urgent action is needed to address major risks to delivery’ or ‘delivery appears to be unachievable’ (Figure 2).17 This proportion stayed largely constant during the first nine months of 2019. Five capabilities record successful delivery as ‘probable’ or ‘highly likely’, notwithstanding any past slippages. However, two of these have already achieved the FOC milestone.

1.9 Twelve of the 17 most significant capabilities yet to achieve IOC are delayed. The average forecast delay is 12 months against the baseline date reported by the Senior Responsible Owner (SRO). The Department expects 14 of 20 projects to deliver the FOC late, with an average delay of 26 months across all projects delivering FOC (Figure 3). In its Equipment Plans for 2018 and 2019,18 the Department showed three capabilities as due to enter service in 2019. These are:

Astute-class submarine HMS Audacious. This submarine was due for operational handover into service in August 2019 but this has been delayed;

the Marshall air traffic control system. This was due to enter service in June 2019, but is also delayed for technical reasons, with a forecast IOC date of May 2020; and

integration of the Sea Ceptor missile on the Type 26 frigate. This was due in December 2019. This date actually represents the delivery of equipment for integration and this was delivered on schedule.

1.10 These challenges are not unique to the United Kingdom. The Australian National Audit Office reported a schedule slippage of 691 months across 26 major projects as at June 2019.19 The US Government Accountability Office reported an average delay of more than 27 months in delivering the initial capabilities of the Department of Defense’s 2018 portfolio of 82 major programmes.20

The main causes of delays to capability delivery

1.11 The Department’s management information and our case studies show that a range of factors impact upon the delivery of capabilities. The most common problems are set out below.

Late or faulty equipment delivery

1.12 Late or faulty equipment delivery by the supplier is the major contributor to forecast and actual delays in capability delivery. This has been a longstanding concern for the Department, as evidenced in past NAO reports21 and the Department’s Equipment Plan.22 Figure 4 shows that 10 of the 32 capabilities within the DMPP report serious issues with the supplier’s engagement or delivery performance as at 30 September 2019. The suppliers involved in delivering these capabilities were predominantly a mix of UK firms and subsidiaries of non-UK companies. SROs refer to a range of factors influencing this rating, both in the management information returns and in communication with the approving authority within the Department’s Head Office, although all are linked to the non-delivery or defective nature of the equipment and systems involved. SROs refer variously to a lack of capacity and skills within suppliers, poor internal quality control, a failure of supplier senior management to engage with problems promptly, and a lack of transparency in supplier reporting of progress.

1.13 Within our case studies, equipment issues featured prominently as the cause of delays in several cases. For example:

problems with the build quality of HMS Forth delayed IOC for more than a year;

after a re-baselining of the delivery timetable in response to changes to air safety regulations, Watchkeeper’s FOC was further delayed by several years due to equipment problems, some of which have led to loss of vehicles due to crashes; and

emergent technical issues continue to affect the availability of the F-35 fighter jet, and training is affected by technical problems with simulators.

1.14 A supplier issue of a different kind emerged in our Materiel Distribution Land case study, which consisted of two separate vehicle projects. The contractor in this case was the existing provider, who was selected to take the solution forward through a non-competitive tender. However, it refused to be bound by the Single Source Contracting Regulations as it considered the profit rate was too low, halting progress in that aspect of the programme. We reported on resistance to compliance with the Regulations in October 2017,23 and in March 2018 the Committee of Public Accounts recommended that: “Where suppliers refuse to comply with the regime, the Department should develop and apply appropriate sanctions.”24

Lack of capacity and skills, and the effect on delivery of the capability and training in its use

Availability of skilled personnel

1.15 Deliverability can also be undermined by a lack of capacity and capability in the departmental teams responsible for delivery. Under-resourced teams in the Commands acting as the customer, as well as in the delivery organisations, contribute to delays in delivery. Skills and capabilities are a serious concern for nine of the 32 DMPP capabilities. Our 2019 report Reforming the civilian workforce drew attention to the shortages of suitably qualified and experienced personnel in the Department.25 There are particular shortages of engineering and commercial staff, and some delivery teams are very reliant on consultancy support. Concerns about this challenge are most common in Strategic Command.26 Six of the 32 DMPP capabilities face shortfalls of more than 20% in their programme teams. Three have a delivery confidence assessment of amber, and three have an assessment of amber/red or worse. Digital and nuclear programme teams display a particular reliance on external consultants to provide expert input to programme teams.

1.16 The Department also has to manage skills issues with suppliers. In one of our case studies, HMS Forth and the other Offshore Patrol Vessels were ordered with the primary intention of sustaining shipyard skills on the Clyde. However, quality assumptions of both the Department and the supplier proved invalid, causing delays to HMS Forth and her sister ships (Figure 5 and Appendix Three).

1.17 SROs are responsible for ensuring projects deliver their expected objectives and benefits, often alongside other important line management roles. In 2018 the Department examined SRO capacity and nine SROs for capability programmes reported not having enough capacity to carry out their tasks effectively. The Department calculated at this time that 12 extra SROs would be needed if each DMPP programme was to have its own SRO. This figure included transformation as well as capability programmes, but the large majority of these additional SROs would work on capabilities. Separately, the Department sets out how much time SROs should spend on each capability, in the context of the total SRO capacity available. As at 30 September 2019 SROs spend less than the mandated amount of time in six cases.

Training

1.18 Four of our case studies – Falcon Early Entry Capability, F-35 fighter jet, the upgraded Base Inventory Warehouse Management System and Watchkeeper – experienced problems at the time of milestone acceptance because of a limited capacity and capability to deliver training. For the F-35, a lack of operational availability in the run-up to declaration of the milestone hampered the ability to deliver training. Also, the F-35 programme’s delivery of training using simulators has been frustrated by technical difficulties and delays in security vetting. For Falcon Early Entry Capability, a lack of previously promised capacity in the training solution delayed IOC while the delivery team funded an alternative solution using the contractor. The Department’s Accounting Officer informed the Committee of Public Accounts in November 2019 that training drone operators is the main threat to achievement of the Protector unmanned air vehicle’s (UAV’s) IOC date.27

Impact of affordability issues

1.19 Since 2012 we have reported on the annual Defence Equipment Plan (the Department’s 10-year portfolio of equipment and support projects). Our most recent report concluded that for the third successive year, the Equipment Plan remains unaffordable and noted the Department’s continuing inability to meet its commitments from the funds available.28 The consequences of this inevitably feed through into capability delivery. For example, the Department reported to the Committee of Public Accounts in November 2019 that it had delayed the delivery of the Protector UAV by two years due to wider affordability challenges across the Department.29

1.20 In our case studies we found several examples where financial pressures had affected the timing and scope of capability delivery. For example:

the Navy secured funding for the Commando Training Centre Royal Marines (CTCRM) Tactical Engagement System capability at Exmouth from the Education & Skills Funding Agency, as the Navy Command could not fund it;

the upgraded Base Inventory Warehouse Management System was postponed for funding reasons until a key component of the legacy system was identified as life-expired, and then funding was provided. Even then, funding shortfalls created several problems during delivery of the capability;

the Falcon Early Entry Capability battlefield communication system struggled to fund a training solution, which delayed progress; and

there was delayed approval for funding of aspects of air base infrastructure for the F-35 fighter jet.

Underachievement against capability milestones

1.21 It is not only the speed of delivery against capability milestones that is important; it is also the extent to which promised capability is actually delivered. In some cases, this remains uncertain. A milestone should represent a specified level of capability and is declared by the SRO. Departmental guidance permits SROs to use their judgement to declare a milestone with limitations, termed ‘exceptions’. These are granted to allow acceptance of the capability where performance does not meet the criteria for that milestone, or if testing to confirm achievement of the milestone criteria is incomplete. Exceptions are granted not only for interim milestones, but also for FOC.

1.22 Departmental guidance suggests a number of acceptable reasons for SROs to report exceptions. Most involve a trade-off for some aspect of capability, user changes to capability requirements, or acknowledgement that the original requirement was unrealistic. In addition, exceptions are acceptable:

to avoid the withdrawal from service of the old capability without an operational successor. The Type 45 destroyer is an example where capability declarations were driven by a need to avoid a capability gap, as the older Type 42 destroyers were increasingly expensive to maintain in service and did not meet modern capability requirements. However, this led to a failure to acknowledge the extent of problems with the Type 45’s innovative propulsion system (Figure 6); and

because the current level of capability offered is considered “good enough”. Departmental guidance does not clarify what this term means.

1.23 There were 102 exceptions granted against milestones for five of our six case studies where the option of granting exceptions was available.30 Of these, 67 applied to declaration of the F-35 fighter jet IOC (Figure 7 and Appendix Three). A large majority reflected non-achievement of the required level of performance at the relevant milestone. In these cases, the SRO had deemed performance was “good enough” (although for the F-35 this was based on the assumption that deployability would be demonstrated over the following 12 months). In addition:

five of the eight exceptions for HMS Forth (the Offshore Patrol Vessel) reflected the fact that the ship had yet to be tested against some of the requirements;

more than half of exceptions for the Falcon Early Entry Capability battlefield communication system involved the need to address shortfalls to the capability as provided, or enhance it beyond the original requirement; and

limitations on the deployment of the Watchkeeper unmanned reconnaissance vehicle in certain weather conditions and low levels of aircraft availability for the F-35, were not noted as formal exceptions when approving milestones, although they affected performance.

1.24 According to the guidance, granting an exception should be conditional on a plan being in place to resolve the exception by a certain date. Most of the five case studies with exceptions lacked full documentation showing the dates by which they should be cleared

5 Capability – this is defined as the combination of equipment, trained personnel and support that gives the armed forces the capacity to achieve the tasks they are given.

6 Comptroller and Auditor General, The Equipment Plan 2019 to 2029, Session 2019–2020, HC 111, National Audit Office, February 2020.

7 Affordability challenges are also a feature of some of our case studies – see paragraph 1.20.

8 This estimate includes a proportion of costs which are not directly applicable to capabilities being brought into service, due to the way the costs are recorded within the Department’s accounting systems. For example, it includes £10.5 billion of personnel costs under the heading ‘provision of defence capability’, which will include the costs of operating in-service capabilities.

9 Ministry of Defence, Annual Report and Accounts 2018-19, HC 2347, July 2019.

10 Comptroller and Auditor General, Ministry of Defence: The Major Projects Report 2010, Session 2010–11, HC 489-I, National Audit Office, October 2010.

11 HC Committee of Public Accounts, Ministry of Defence: The Major Projects Report 2009, Twenty-third Report of Session 2009–10, HC 338, March 2010.

12 Bernard Gray, Review of Acquisition for the Secretary of State for Defence: An independent report by Bernard Gray, October 2009.

13 We describe these as the 32 most significant defence capabilities.

14 These costs include amounts already spent, and, for some capabilities, will extend beyond the 10-year period covered by the Department’s Equipment Plan. This analysis includes the budgeted costs for one project where no forecast costs were available.

15 The Government Major Projects Portfolio is composed of the largest, most innovative and highest-risk projects and programmes delivered by the government, requiring HM Treasury approval.

16 There are four Commands (Air, Army, Navy, Strategic (formerly Joint Forces)) and the Defence Nuclear Organisation (DNO) who hold the responsibility and the budget for delivery of the portfolio of defence capabilities. Where we refer to Commands this also includes DNO.

17 The Infrastructure and Projects Authority sets the delivery confidence ratings definitions. Infrastructure and Projects Authority, Annual Report on Major Projects 2018–19, July 2018, available at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/817654/IPA_AR_MajorProjects2018-19_web.pdf

18 Ministry of Defence, The Defence Equipment Plan 2018, November 2018, and Ministry of Defence, The Defence Equipment Plan 2019, February 2020.

19 Australian National Audit Office, 2018-19 Major Projects Report, ANAO Report Number 19 2019-20, December 2019, available at: https://www.anao.gov.au/work/major-projects-report/major-projects-report-2018-19

20 United States Government Accountability Office, Weapons Systems Annual Assessment, GAO-19-336SP, May 2019, available at: https://www.gao.gov/assets/700/698933.pdf

21 Comptroller and Auditor General, Delivering Carrier Strike, Session 2016-17, HC 1057-I, National Audit Office, March 2017.

22 Ministry of Defence, The Defence Equipment Plan 2019, February 2020.

23 Comptroller and Auditor General, Improving value for money in non-competitive procurement of defence equipment, Session 2017–2019, HC 412, National Audit Office, October 2019.

24 HC Committee of Public Accounts, Ministry of Defence: Acquisition and support of defence equipment, Twenty-eighth Report of Session 2017–2019, HC 724, March 2018.

25 Comptroller and Auditor General, Reforming the civilian workforce, Session 2017–2019, HC 1925, National Audit Office, March 2019.

26 In December 2019, Joint Forces Command was renamed Strategic Command.

27 Ministry of Defence, Protector programme Accounting Officer assessment, November 2019, available at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/854362/20191105_-_Meg_Hillier_Protector_AOA.pdf

28 See footnote 6.

29 See footnote 27.

30 Two of the case studies were not delivered through an approach where exceptions would be applicable, and one is delayed in achieving its milestone.

Part Two

Oversight of the challenges to capability delivery

2.1 Responsibility for identifying capability requirements and delivering them is split within the Ministry of Defence (the Department). In response to a 2011 report by Lord Levene, the Department undertook a transformation programme which led to a delegated organisational model from April 2014.31 The centre of the Department, known as ‘Head Office’, directs activity. Head Office takes a strategic role and has retained the role of approving and monitoring, through its Investments Approvals Committee (IAC), the progress of capabilities with a high value (more than £400 million) as well as others deemed to be novel or contentious. Under this model, the Commands – Air, Army, Navy, Strategic Command (formerly Joint Forces Command) and, more recently, the Defence Nuclear Organisation (DNO) – identify the capabilities they need, and hold the responsibility and the budget for delivery of the portfolio of defence capabilities.32 Senior Responsible Owners (SROs) make sure individual acquisitions meet their objectives. Delivery organisations manage the equipment and support projects on behalf of the Commands. Further detail on the role of each is provided in Figure 8.

2.2 Under the Department’s devolved system, Commands should know how programmes are performing and be able to address emerging issues in a timely manner. Head Office should be fully aware of the performance of all significant projects and programmes. At the end of a project, the Department is required to establish whether the capability is delivering what was required.

2.3 In this Part we look at oversight of capability delivery:

across the Department by Head Office; and

within the Commands.

Head Office oversight

2.4 The system set up to deliver capabilities is inevitably complex. Descriptions of the system are embedded within central guidance, but the Department has reduced the amount of resource devoted to maintenance of this guidance in recent years, so parts of it are out of date. The structure of the guidance is not always easy to follow, nor is it clear which parts need to be followed. The acquisition system guidance has become longer and more complex over time; for example, the acronym glossary contains more than 1,900 entries and even then is not exhaustive.

2.5 Under its delegated model for capability delivery, the Department established the Finance and Military Capability (FMC) organisation within Head Office to, among other things, maintain the balance of investment controls over the link between resources and capability. The Department now considers this has not worked as well as intended. It acknowledges that it has further to go in establishing cross-cutting business functions such as commercial, project delivery, finance and capability management and ensuring these work coherently with one another.

2.6 Easy access to consistent information across the entire portfolio is vital for holding Commands to account and for making strategic decisions, but is not currently available. The Department produces and presents senior staff with a variety of different types of information in relation to the Department’s capability programmes, for example through the approvals and scrutiny process. In 2013, a review of implementation of the delegated model found there was “a lack of clarity around the future responsibility and resources to deliver the management information [element of the reforms]”, and steps to improve it were not expected to be complete until 2017. The Department acknowledges that under the delegated model it has reduced central oversight, and data on all projects and programmes which the Department is undertaking are still not held in a single location within the system.

2.7 As explained in paragraph 1.6, Head Office reviews information on the most significant capabilities – those included in the Defence Major Projects Portfolio (DMPP). The accuracy and completeness of the information provided is the responsibility of the SRO. The DMPP accounts for more than 40% of the Equipment Plan over the next 10 years.33 It is not always clear why particular capability deliveries are excluded from the DMPP. For example, work for our HMS Forth case study showed that the Offshore Patrol Vessel (OPV) programme would have benefited from closer central review. Its initial budget was around £378 million34 (since increased to five ships costing £683 million), and the programme has important implications for the ship-building skills needed for the Type 26 frigate programme. However, it was not included in the DMPP.

2.8 There are limitations with the DMPP data collected within Head Office, which are presented to senior staff at quarterly DMPP review meetings. Management information can be several months out of date by the time it is presented to senior decision-makers. Senior staff use different financial information (sourced from the central Contract Purchasing and Finance system) to that provided in SRO returns as they consider the former to be more accurate.

Consistency of reporting

2.9 We looked at the progress ratings applied by SROs to capabilities in the DMPP. We were not able to examine these capabilities in as much detail as our case studies, but we looked at consistency between the rating and other information contained in the reports. Most assessments for 30 September 2019 appeared to show a reasonable approach to the assessment of delivery confidence. One anomaly concerns the introduction of the Marshall air traffic control system. The SRO did not declare IOC on schedule in June 2019, and the Department informed us that the project is not expected to reach IOC until May 2020. Despite this, during 2019 the delivery confidence rating did not indicate that successful delivery of the milestone was in doubt or had become unachievable.

2.10 For each capability, the customer’s project team tracks user requirements across the eight ‘defence lines of development’, which together provide a detailed picture of progress across all aspects of the capability. Each line of development addresses a specific aspect of capability, such as equipment, training, infrastructure, and so forth.35 However, this internal information on performance is not monitored by Head Office. The categories in Head Office management information – the delivery confidence assessment and its associated sub-categories – overlap little with the lines of development. Where there is overlap on equipment delivery, projects often provided different ratings within the same report. Our analysis showed there was only a weak correlation for projects between the Department’s rating of supplier confidence and the ‘equipment’ line of development. It was clear from the commentary in the data returns that, while most personnel considered that the two indicators were measuring the same thing, some did not. Even in the former cases, there were some wide differences in the assessment between the two indicators.

Oversight of capability delivery within the Commands

2.11 As part of the implementation of the delegated model, the Commands monitor delivery of their capabilities through their own portfolio offices, which are designed to meet their capability information needs. The Department has mandated good-practice elements of project, programme and portfolio management, and provided a standard model for capability planning supported by guidance.

2.12 We found that, five years after being delegated responsibility, the Commands progress in developing their management of their portfolio varied,36 and that Command portfolio offices vary in the role they carry out (Figure 9). There were staffing gaps in all the offices, and for some Commands this was a significant challenge. All Commands told us they have plans to upgrade the capability of their portfolio offices, and in the case of the DNO, to develop one for the first time.

2.13 Around 100 capability programmes are actively monitored at Command level.37,38 Currently, Command portfolio offices do not have a complete picture of all the capabilities they are acquiring; only Air Command holds information on all its programmes and projects. While all Commands use the mandated Project Management Reporting System to report delivery performance of those capabilities on the DMPP, some use other systems for their internal monitoring, most notably Army Command. There is variation in the amount of information that is collected for minor capabilities. For example, Navy Command holds little information for those capabilities below its ‘top 25’ programmes. Some Commands struggled to provide us with a complete or accurate list of all their projects and programmes, and we found errors in some Command-level information when selecting our case study examples. If repeated more widely, this could provide a misleading picture of capability delivery.

2.14 All Commands have a forum for reviewing the progress of the Command portfolio, although these differ in format and name. All perform some form of analysis to identify common challenges to capability delivery but share the insights differently. Air Command, Army Command and DNO centrally manage interdependencies, but in Strategic Command and Navy Command it is up to the SRO or programme directors. Not all Commands have processes for measuring capability against original requirements. Strategic Command does not currently have a process for gathering lessons learned, and Air Command has struggled to implement a process due to staff shortages. All Commands produce their own guidance on portfolio and project management, in addition to that provided by Head Office. However, only limited checks are made to test if guidance has been followed. Strategic Command faces particular challenges because of the responsibilities it holds with regards to capabilities that will be used across defence, and where the SROs are located in other Commands.

31 Ministry of Defence, Defence Reform: An independent report into the structure and management of the Ministry of Defence, June 2011, available at: https://www.gov.uk/government/publications/defence-reform-an-independent-report-into-the-structure-and-management-of-the-ministry-of-defence--2

32 Where we refer to Commands this also includes DNO.

33 This estimate includes all elements of the capability delivery including non-equipment figures such as infrastructure costs. There are also differences in the reporting period across DMPP projects and the Department’s annual Equipment Plan report, which covers equipment procurement and support.

34 Procurement cost of the three Batch 2 OPV ships originally ordered (including HMS Forth).

35 Defence Lines of Development (DLODs) are training, equipment, personnel, information, doctrine and concepts, organisation, infrastructure and logistics.

36 The Defence Nuclear Organisation has only existed since 1 April 2018.

37 Of these, 32 are included in the DMPP.

38 This figure represents capabilities across the Navy, Army, Air and Strategic Commands. As not all Commands monitor small projects this figure may be higher.

Part Three

Transforming capability delivery

3.1 In January 2018, the Ministry of Defence (the Department) set up a dedicated central transformation team reporting to the chief operating officer. Transformation is taking place across four key functional areas of defence business, of which the first is most relevant for the purposes of this report. The areas are:

Acquisition – creating a more agile, adaptable and streamlined acquisition system to increase the pace of procurement.

Support of military capabilities.

Embracing modern technologies.

Delivering a sustainable workforce with the required skills.

3.2 In this part of the report we explain the steps being taken by the Department to transform capability acquisition, and the extent to which these changes address the issues identified in Parts One and Two.

Transformation projects

3.3 The Department set out in the Levene report its intentions for capability planning and financial management, which included better decision-making, less bureaucracy and simplified processes, with the centre of the Department (known as ‘Head Office’) having visibility and assurance to ensure financial control.39 The new initiatives are an acknowledgement that this has not happened as intended. The Department’s key objectives for the acquisition system are to:

create a more agile, adaptable and streamlined acquisition system to increase the pace of procurement; and

strengthen the role of Head Office in overseeing the system, in line with the original intent of the Levene reforms (see paragraph 2.1). We described the current oversight regime in Part Two.

3.4 In 2018, the Department introduced Project MAID (MoD’s Approach to Investment Decisions) to improve the quality of investment decisions, based on a more rigorous business case justification in the early stages of the project (akin to that seen elsewhere in government departments). In addition, in September 2019, the Department completed an Acquisition Review. This is intended to increase the speed of delivery by tailoring the acquisition process to suit each programme, and increasing the quality of oversight by Head Office, partly through improvements to management information. In October 2019, Project MAID and the Acquisition Review were brought together under one programme board. The Department told us it was developing a detailed plan for implementation of the Acquisition Review later in 2019-20. More information on both initiatives is shown in Figure 10.

3.5 The Department has set itself an ambitious challenge of taking two years out of the average ‘time-to-front-line process’, and to move from 75% of projects being delayed to 75% being on time, although it is unclear what the baseline is for these metrics. The Department has run a number of pathfinder projects to test its new acquisition approach. One pathfinder project – covering the Interim Surface to Surface Guided Weapon – showed how delivery of the capability could be accelerated by three years. Despite some potential cost avoidance and savings, it could now cost around £200 million over three, rather than six, years – with an annual cost increase of £33 million. However, speeding up delivery and so compressing cost into a shorter time span could increase the Department’s difficulties with the near-term unaffordability of its equipment portfolio, as set out in our work on the Department’s Equipment Plan.40

3.6 In addition to the two reviews mentioned above, other ongoing initiatives will affect how the Department delivers capabilities. In particular:

the Department has reviewed the operating model it introduced in 2014 with the aim of ensuring all parts of the Department work together effectively. The new model aims to clarify roles, responsibilities, authorities, accountabilities and the Department’s processes for taking key decisions. It has been published internally within the Department during 2019-20; and

the Department has undertaken a specific review of how Joint Forces Command (now called Strategic Command) manages its functions and delivers its outputs, including capabilities. The review recommends that the Command should have an enhanced role in supporting Head Office. The review’s recommendations are now being implemented.

3.7 As part of developing the Defence Operating Model, the Department has also introduced a new ‘functional leadership’ approach to oversee cross-cutting functions such as project delivery, which is a key element of capability delivery. Delivery of the project delivery functional strategy will be facilitated by a centre of excellence, to replace a previous one disbanded in 2012. One of the roles of the centre of excellence is to ensure that lessons are learned.

Addressing the challenges we have identified

3.8 We looked at the extent to which the challenges to capability delivery identified from our analysis and case studies would be addressed by the Department’s various initiatives:

Milestone declarations

We found examples of milestones being declared where there were potentially significant shortcomings against the agreed criteria and which had nothing to do with changes in capability requirements during delivery. The Acquisition Review recommended establishing firm time parameters for capability delivery, monitoring against them, and adhering to them. Updated approvals guidance references aspects of capability delivery and scheduling which should be considered.

Flexibility

The Department identified that technological change will continue to increase at a pace that outstrips the agility of its current capability planning and acquisition system. Therefore, it set out that it needs a system that supports a more flexible approach to procurement that can respond to the speed of technological change by subjecting core capabilities to a process of continual upgrades. We consider that changes could potentially have mitigated examples in our case studies where individual capabilities were obstructed by a failure to keep pace with technological change. For example, in our Falcon Early Entry Capability battlefield communication system case study the new capability is constrained by obsolete technology in other parts of the system. However, the danger of moving away from the traditional approach whereby a capability is generated against a set of fixed, clearly defined milestones, is that stakeholders across the defence environment will be unclear what level of capability a system has achieved at any given point.

Better information

As identified in Part Two, there is also a need for greater transparency about performance across the Commands and within Head Office.41 As part of the implementation of the Acquisition Review, the Department will roll out a system of portfolio and performance management, initially for equipment, then other aspects of capability delivery such as infrastructure, personnel and training, from 2020.

Management of programmes

In our work we came across issues with the current lead command model, particularly around prioritisation and flexibility of funding, and the relationships between Strategic Command (formerly Joint Forces Command) and the other Commands. In a new approach, where Strategic Command is leading cross-command work, this Command will seek to transfer programme ownership to another at an early stage, while retaining capability sponsorship responsibilities. This increases the importance of having mature portfolio management in each of the Commands, and Strategic Command has told us that it is working hard to address this, for example through increasing the numbers working on the management of the portfolio and introducing a portfolio Centre of Excellence.

Capacity and capability

The Department faces challenges around capacity and capability, and in Part One we highlighted issues affecting Senior Responsible Owners (SROs), and more general skills shortages in project and delivery teams. The Acquisition Review recommended more expert support for SROs to understand time, cost and performance trade-offs and tailor their acquisition route. Work is being undertaken on SROs by the Project Delivery function.

3.9 The Department’s Acquisition Review also highlighted that there is a culture within the Department which prioritises passing review points and approvals above delivery of outputs and outcomes. One of the key aims of acquisition transformation is to increase the pace of delivery. It is important that reforms do not further encourage declaring milestones in the face of capability shortfalls in order to create the perception of fast progress. Developing the implementation plan to shape the culture will therefore be important.

3.10 The initiatives that are being undertaken by the Department are intended to address challenges identified through our work. Overall, they do address many of them but they would not necessarily prevent the occurrence of some issues identified in our analysis, such as contractor performance or failure to deliver timely training. It is also too early to assess the impact the changes will have as the Department does not yet have an implementation plan. The Department has also still to develop a plan that brings the initiatives together and shows how they are being coordinated, to ensure they do not cut across one another or some become deprioritised.

Force readiness

3.11 The Levene reforms were intended to introduce a system that would enable strategic balance of investment decisions and requirement setting across all defence lines of development. In 2019-20 the Department introduced additional mechanisms to support assessments of both current and future capability, and the level of readiness of that capability (Figure 11). The initial work done in these areas is intended to strengthen the ability of Head Office to be aware of capability issues and ensure they are addressed. It is too early to assess the effectiveness of these new processes. These mechanisms sit alongside existing processes, notably the Defence Capability Assessment Register (DCAR). This sets out for decision-makers the risk to delivery of the Defence Tasks over the next 20 years. It provides a baseline from which the requirement, and priority, for further strategic capability investment can be assessed.

3.12 Although these measures will generate a coherent picture of capability readiness against defence objectives, addressing capability challenges is contingent upon having the funding available to implement the results. As part of its submission to HM Treasury for the 2019 Spending Review, the Department carried out a detailed ‘balance of investment’ analysis using multi-criteria decision analysis. This generated a number of options to prioritise enhancements to existing capabilities, and for savings, based on the best ratio of benefits to costs. None of the enhancements can be funded currently without reducing existing capability commitments. In February 2019, the Committee of Public Accounts concluded: “The Department’s Equipment Plan remains unaffordable as government continues to delay decisions on its priorities, and on whether to increase funding or stop, delay or scale back programmes.”42

39 Ministry of Defence, Defence Reform: An independent report into the structure and management of the Ministry of Defence, June 2011, available at: https://www.gov.uk/government/publications/defence-reform-an-independent-report-into-the-structure-and-management-of-the-ministry-of-defence--2

40 Comptroller and Auditor General, The Equipment Plan 2019 to 2029, Session 2019-20, HC 111, National Audit Office, February 2020.

41 There are four Commands (Air, Army, Navy, Strategic (formerly Joint Forces)) and the Defence Nuclear Organisation (DNO) who hold the responsibility and the budget for delivery of the portfolio of defence capabilities. Where we refer to Commands this also includes DNO.

42 HC Committee of Public Accounts, Defence Equipment Plan 2018–28, Seventy-seventh Report of Session 2017–2019, HC 1519, February 2019.

Appendix One

Our audit approach

1 This study examined whether the Ministry of Defence (the Department) gets the capabilities it requires when it needs them to meet its current and future defence objectives. We examined the Department’s approaches to:

identifying and managing the challenges of delivering required capabilities into service;

having a complete system in place to monitor the delivery of capabilities; and

putting in place appropriate arrangements to transform its capability delivery in the future.

2 We drew on our previous reports on major projects across government to determine good practice when reviewing the Department’s approach. We also applied insights on industry standards on portfolio management, for example the Infrastructure and Projects Authority’s Project Delivery Functional Standard,43 alongside the National Audit Office’s guidance frameworks for project delivery.

3 Our audit approach is summarised in Figure 12. Our evidence base is described in Appendix Two.

43 Infrastructure and Projects Authority, Government Functional Standard GovS 002: Project delivery, August 2018, available at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/746400/Project_Delivery_Standard_1.2.pdf

Appendix Two

Our evidence base

1 Our independent conclusions on whether the Ministry of Defence (the Department) gets the capabilities it requires when it needs them were reached following our analysis of evidence collected between May and December 2019.

2 We applied an evaluative framework to consider what arrangements would be optimal for capability delivery. Our audit approach is outlined in Appendix One.

3 In examining if the Department identifies and manages the challenges of delivering required capabilities into service we:

reviewed central management information on project performance delivery for the 32 capability programmes on the Defence Major Projects Portfolio (DMPP), which the Department considers to be the most complex and strategically significant capabilities, monitored through the Department’s Project Management Reporting System (PMRS). These are projects the Department has defined as ‘capabilities’ rather than ‘business change’ or ‘IT’ projects.44 We analysed data between quarter one 2015-16 and quarter two 2019-20 to identify trends; and

undertook in-depth analysis of eight case study projects across four military Commands (Navy, Army, Air and Strategic) which had achieved Initial Operating Capability (IOC) or Full Operating Capability (FOC) within the past three years. We selected one IOC and one FOC per Command. We selected one smaller and one larger capability per Command (in cost terms). We selected from project listings supplied by the Commands. Given the limited number of cases passing through some Commands (Air and Army Commands), project choice was limited, especially for the most complex and significant projects. Where more cases were available, choices were narrowed on the basis of secondary criteria, such as balance of project sizes and whether the project’s security classification made it too sensitive to report on. In one case, we selected a project where IOC was not yet declared due to inaccuracies in the information supplied to us. We retained this project in the sample as the actual IOC date was due to fall within the study fieldwork period. Delivery teams for the selected capabilities were contacted and asked to supply documentation relevant to the declaration of the milestone. This was then evaluated against a proforma developed from good practice on project delivery. Supplementary information and interviews with the Senior Responsible Owner and project team were used to support our analysis. In the case of two capabilities, a site visit was undertaken to see the final capability (the High G pilot training facility and HMS Medway – a sister ship to HMS Forth).

4 In considering the system in place to deliver defence capabilities we:

interviewed departmental staff within central teams involved in capability delivery. This included the Finance and Military Capability group, the Defence Portfolio and Approvals Secretariat and the Project Delivery Function. We reviewed central departmental documentation relating to capability delivery and guidance held on the Department’s Knowledge in Defence (KiD) system. In scoping our work we spoke to the government’s Infrastructure and Projects Authority (IPA); and

we prepared a Command questionnaire informed by good practice and the National Audit Office’s guidance frameworks for project delivery.45 The questionnaire was issued to the portfolio offices of all Commands (Air, Army, Navy, Strategic) and the Defence Nuclear Organisation (DNO) to inform our understanding of the maturity of their portfolio function. We also held interviews and reviewed documents supplied by the Commands and DNO to inform our understanding of portfolio processes at the Command level. We used this information to assess Command and DNO portfolio offices against a set of criteria desirable in an efficient and effective portfolio office, which has been informed by departmental guidance. Performance against these criteria has been summarised on a five-point ‘traffic light’ rating scale to identify the relative progress across the different Commands and DNO. The overall rating is calculated using an average from all the categories. We received feedback from the Project Delivery Centre of Excellence on our assessment. We did not assess the skills and performance of individual staff members.

5 In assessing the Department’s arrangements in place to transform capability delivery we:

reviewed the Department’s documents on the transformation of acquisition and approvals, as set out through the Acquisition Review and Project MAID (MoD’s Approach to Investment Decisions). These were supported by semi-structured interviews with staff within the transformation teams;

enquired about wider transformation initiatives and reviewed some documentation in relation to the project delivery function and the new operating model. We also reviewed documents and interviewed staff in relation to the Joint Forces Command review; and

interviewed departmental staff involved in force readiness work and reviewed documentation associated with this, as well as attending associated board meetings.

44 The Department acknowledges that the boundaries between these categories may not be clear-cut. For example, some IT projects are classed as capabilities.

45 National Audit Office, Framework to review programmes, April 2019, available at: https://www.nao.org.uk/report/framework-to-review-programmes/, and National Audit Office, Initiating successful projects, December 2011, available at: https://www.nao.org.uk/wp-content/uploads/2011/12/NAO_Guide_Initiating_successful_projects.pdf

Appendix Three

Case studies

Case study summary sheets

HMS Forth Batch 2 Offshore Patrol Vessel (OPV) (Initial Operating Capability (IOC))

Commando Training Centre Royal Marines (CTCRM) Tactical Engagement System (TES) (Full Operating Capability (FOC))

Materiel Distribution Land (IOC/In Service Date (ISD))

Watchkeeper unmanned aerial vehicle (FOC)

F-35 Lightning II Joint Strike Fighter (IOC Land (IOC(L)/ISD))

High G test and training facility (FOC)

Falcon Early Entry Capability (FEEC) battlefield communication system (IOC)

Upgraded Base Inventory Warehouse Management System (BIWMS) – Release 2 (FOC)

Design and Production by NAO External Relations
DP Ref: 006572-001