Report

by the Comptroller
and Auditor General

Ministry of Defence

The Equipment Plan 2016 to 2026

HC 914 Session 2016-17 27 January 2017

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO, which employs some 785 people. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund have used their resources efficiently, effectively, and with economy. Our studies evaluate the value for money of public spending, nationally and locally. Our recommendations and reports on good practice help government improve public services, and our work led to audited savings of £1.21 billion in 2015.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 25 January 2017

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

13 January 2017

In this report we give an overview of the Ministry of Defence’s 2016 to 2026 Equipment Plan for procurement and support.

© National Audit Office 2017

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.gsi.gov.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

11408 01/17 NAO

The National Audit Office study team consisted of:
Jeremy Gostick, Thomas White, Alice Foster, Matt Ryder, Rohan Soni and Kaye Sullivan, under the direction of Keith Lloyd.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key facts

Summary

Scope of the report

1 In 2012, the Ministry of Defence (the Department) adopted a new approach to generate greater stability in its procurement activity. They called this the Equipment Plan (the Plan). The Plan involves developing a budget for a ‘core programme’ of key equipment projects and an additional sum set aside for contingency. The Plan covers forecast spend for 10 years and is updated annually. For the period 2016 to 2026, the equipment budget is £178 billion, made up of procurement (£82 billion) and support (£91 billion) budgets, and a central contingency reserve (£5 billion). The Plan is funded from the Department’s overall budget, and makes up more than 40% of its planned spend.

2 The Department publishes an annual Statement on the affordability of this 10-year plan to deliver and support the equipment that the Armed Forces require to meet their objectives.1 At the request of the Secretary of State, we report on the robustness of the assumptions underlying the Statement. We examine the bottom-up costings of a sample of the largest projects in the Plan (nine procurement projects and seven support projects), and a top-down review of assumptions about expenditure and funding at Departmental level.

3 We have not set out to offer a definitive view on the affordability of the Plan, as it is, by its nature, based on assumptions about the future that will inevitably change. Rather, we review the assumptions that underpin the forecast costs and funding to assess whether they were reasonable and consistently applied when they were made. Neither do we evaluate the value for money of the various projects mentioned in this report.

4 We explain the background to our work in Part One. We look at changes to this year’s Plan and the assumptions underpinning future funding (Part Two); and the Department’s assumptions underpinning the forecast costs of the Plan (Part Three). Parts Two and Three set out the challenges to the affordability of the Plan. In Part Four we review whether the disclosures in the Department’s Statement are sufficient for the reader to understand fully the sensitivities of the affordability position. Appendices One and Two contain full details of our audit procedures. Summaries of the projects in our sample are included at Appendix Three.

Key findings

Increased size of the Plan in 2016

5 The cost of the 2016 to 2026 Plan has risen by 7% since the previous year to £178 billion. This rate of increase over last year’s Plan compares to an increase of only 1.2% between the 2013 to 2015 Plans. The Plan is made up of £82 billion for equipment procurement, £91 billion of support costs and a contingency of £5 billion (Figure 1 and paragraph 2.2).

6 The Strategic Defence and Security Review (the Review) added £24.4 billion of new commitments, the majority of which are to be funded within the existing Plan. Significant new procurements include the Mechanised Infantry Vehicle and Poseidon Maritime Patrol Aircraft, which alone add £5.5 billion of additional spend over 10 years. The Review also announced additions to current and future capabilities, including life extension of the first production tranche of Typhoon fast jets and the acceleration of purchases of the F-35 Joint Strike Fighter aircraft (paragraphs 2.3, 2.4 and 3.4).

7 The requirement to support new equipment entering service puts increasing pressure on the £91 billion support budget. The Department will face an increased challenge in controlling its support cost budget following the Review as new procurement commitments will require support in future Plans, while some equipment due to be retired is now being retained with an extended life (paragraph 2.5).

Sources of funding for the Plan

8 The Department has allocated all headroom previously set aside in the Plan, removing its flexibility to accommodate additional capability requirements. In previous years the Department created ‘headroom’ to provide scope to meet emerging priority requirements, thus increasing flexibility within the Plan. To help meet its new commitments, the Department redistributed £9.5 billion of headroom from the 2015 Plan and £1.2 billion that would otherwise have been rolled forward as headroom in the 2016 Plan. With the redistribution of all headroom, any further capability requirements during the lifetime of the Plan period will have to be met through a reprioritisation of existing commitments, or a reallocation of funding from the wider Defence budget (paragraph 2.8).

9 The Department must generate £5.8 billion of new savings from projects within the Plan to meet its new commitments. The Department’s plans are challenging, with £3.3 billion to be generated from transformational activities within Defence Equipment and Support, £1.7 billion from the application of the Single Source Contracts Regulations and a further £0.8 billion from other sources. The Department apportioned the savings target between front-line commands but had not identified which projects would provide these savings before finalising the Plan (paragraphs 2.9 and 2.10).

10 Further savings of £1.5 billion to fund new commitments are to be met from within the wider Defence budget, which is already under increasing cost pressure. At a time when an increasing proportion of the Defence budget is taken up by the Plan, generating savings from the wider Defence budget will be challenging. The source of the bulk of these savings is understood, for example through five years of military and civilian pay restraint, but further work is needed to refine the detail. This is against a background of wider funding pressures, such as those affecting the Defence estate (paragraphs 2.12 and 2.13).

11 Since finalising the Plan, the Department has put in place a Delivery Board to oversee efforts to deliver efficiencies from across the Department. During 2016-17 Commands and other budget holders have identified the sources of these savings. If savings are not achieved it will fall to Commands to propose to Head Office how they plan to reprioritise commitments or transfer funds from other parts of their budget to fund projects. The Department’s internal audit function found that the Department had set up a strong governance framework to track savings, but recommended a number of improvements to the framework to ensure that savings are deliverable and measured accurately (paragraphs 2.14 to 2.16)

12 The affordability of the Plan also depends on the realisation of £7.1 billion of brought forward savings already assumed in the Plan. The Department’s progress against these plans varies significantly. The Department estimates that it has achieved only £4.6 billion of this target to date, and it faces a challenge in generating the remainder of the £2.5 billion, the majority of which has to come from within the Complex Weapons area (paragraphs 2.17 and 2.18).

13 Additional external funding supports only £6.4 billion of the new commitments. This funding will come from the newly created Joint Security Fund for the Armed Forces and Security and Intelligence Agencies, the sole source of new funding associated with the Review (paragraph 2.19).

Confidence in the costings of the Plan

14 New commitments in the Review have considerably increased cost uncertainty in the Plan, with the number of immature cost estimates increasing. The stability of the costs in the Plan in recent years has reflected the relative maturity of the project portfolio. Review-related projects such as the Mechanised Infantry Vehicle have immature cost estimates, which reflect their early stage of development and will be revised going forward. Of the £24.4 billion of additional commitments in this year’s Plan, £3.9 billion (15%) has yet to go through detailed costing at project level (paragraphs 3.3 and 3.4).

15 Cost estimates for the Type 26 Global Combat Ship, the largest non-nuclear procurement project in the Plan, could not reflect fully decisions made in the Review. The Review made significant changes, including the introduction of a new class of frigate to replace part of the Type 26 requirement. The Department’s re-costing and rescheduling of the Type 26 project, together with the development of the new frigate design, mean that a full costing of all the elements of the new requirement will not be available until the 2018 Plan (paragraph 3.5).

16 Cost growth in the existing nuclear programme continues, with the potential to destabilise the Plan. The scale of spend and life cycle of the programme makes costs particularly uncertain. The forecast 10-year costs of the Dreadnought and Astute nuclear submarine programmes increased by £836 million in 2015-16. The Department’s Cost Assurance and Analysis Service (CAAS) estimates that costs of these programmes are still understated by £1.3 billion over the 10 years of the Plan. Elsewhere in the Nuclear Enterprise, forecast costs of the Atomic Weapons Establishment management contract fell by £0.7 billion following contract re-negotiations. Given its complexity and inherent cost uncertainty, the Department is enhancing its management of the programme through the introduction of a new Director General Nuclear post and a separate submarine delivery body (paragraphs 3.6 to 3.8 and Appendix Three).

17 The Department’s current costing practice can lead to significant understatement in the likely cost of some projects at an early stage of development, at a time when the Review has resulted in a higher proportion of large, early-stage projects in the Plan. Analysis by CAAS has shown that the costs of a project can increase significantly as it moves from its design stage into manufacture and point of service entry. The analysis implies that the current costing policy is likely to understate the costs of early-stage projects currently included in the Plan (paragraphs 3.10 to 3.14).

18 Changes in foreign exchange rates, such as those that happened after the EU referendum, can pose a significant risk to the Plan’s affordability in the future. As at 10 January 2017, the pound was 21.4% below the exchange rate with the US dollar and 4.2% below the exchange rate for the euro used in the Department’s planning assumptions. Approximately £18.6 billion of the Plan is denominated in US dollars and £2.6 billion in euros over 10 years. This illustrates the significant risk to affordability currently faced by the Department (paragraphs 3.17 to 3.19).

19 Although the Department’s contingency has risen slightly in both absolute and percentage terms, it has little room for manoeuvre if costs grow. The reallocation of ‘headroom’ into the core Plan this year means that cost growth must be constrained within the 3% contingency provision in the Plan. The CAAS estimate of the variance between the 50th percentile estimates and the most likely project outturn for this year’s Plan as a whole is £4.8 billion, within the Department’s £5.3 billion contingency provision. Due to timing restrictions CAAS was not able to carry out specific cost estimates of the additional commitments to the Plan arising from the Review during 2015-16 (paragraphs 3.20 to 3.22).

20 The Department’s Statement provides sufficient information on the components of the Plan, but lacks detail on sources of efficiency savings and the sensitivity of the Plan to changes in project cost assumptions. The Affordability Statement provides detail on the value of procurement, support and contingency; however, there is insufficient detail to understand the Department’s progress in identifying the source of the savings required to fund the commitments from the Review. There is no detail on future variability of the Plan resulting in fundamental changes to assumptions such as foreign exchange fluctuations (paragraphs 4.3 to 4.5).

Conclusion

21 The affordability of the Plan is now at greater risk than at any time since reporting was introduced in 2012 and the Department faces the risk that in future it may have to return to a situation where affordability of the portfolio is maintained by delaying or reducing the scope of projects. The cost of the new commitments included in the Review considerably exceeds the net increase in funding for the Plan. The Department has agreed to fund these new commitments partly through demanding efficiency targets, from both within the existing Plan and from the wider Defence budget. All existing headroom has had also to be allocated to meet the new commitments arising from the Review, with the result that this money is not now available to fund newly emerging requirements during the period covered by the Plan.

22 Moreover, the risk of cost growth is still evident in the Plan, both in existing projects and also because a greater proportion of large projects are at an early stage of development (largely due to a number of new high-value commitments introduced by the Review). This risk is further increased as the Department’s current costing policy has historically underestimated the cost of projects in their early stage of development. The Department also faces a significant potential threat to affordability as a result of exchange rate movements against the pound.

Recommendations

a The Department should identify the current projects most at risk from cost growth, and using the Cost Assurance and Analysis Service, assess how these projects could be costed more realistically in the Plan. Projects in the early stages will have more immature costings than those that are more advanced, and are particularly prone to optimism bias. The Department should have a clear view of where and when the main cost pressures might occur in future years and a plan for how they might be accommodated. Such analysis should consider whether more flexibility should be introduced in the application of the current costing policy for estimating the cost of immature projects, to reflect the increased level of uncertainty.

b The Department should ensure that any commitments or savings targets arising from the 2016 Review that are not reflected in project-level costings are programmed at project level in the 2017 Plan, and identify which projects will generate the efficiency savings required to fund new commitments. When the Department finalised the 2016 Equipment Plan it was still working through the financial implications of the commitments entered into in the Review. This work is ongoing at the time of publication. It is vital that the Department concludes this exercise promptly to increase confidence in the affordability of the Plan.

c The Department should assess the impact of future exchange rate fluctuations on the affordability of the Equipment Plan. A significant proportion of spend within the Plan is in US dollars and euros. Project teams should re-model future costs based on updated forecast exchange rate values. The Department should better understand the total affordability of the Plan under a range of different exchange rate scenarios, which should also be used to inform future policy on forward purchase of currencies.

d The Department should ensure that it has in place suitable mechanisms for prioritising spend and removing or deferring projects from the Plan should affordability be compromised to the extent that Commands are unable to accommodate cost growth within their budgets, and central contingency is insufficient. The Commands, who now have day-to-day responsibility for managing the equipment budget, will face new challenges in delivering the post-Review portfolio. It is important that the Department has in place a robust central process for reprioritising commitments in the Plan that balances operational need with the requirement to protect value for money, and that decisions are supported by suitable business cases that address both requirements.

e The Department should improve the transparency of its Affordability Statement on the Plan by providing greater insight into the range of potential outturn costs across the portfolio. Uncertainties within projects mean that many teams provide a range of potential costs, in line with the Department’s guidance. However, this range is not reflected at portfolio level. We have made this recommendation for several years and it is more relevant now than ever.

1 These objectives are set out in HM Government, National Security Strategy and Strategic Defence and Security Review 2015, Cm 9161, November 2015.

Part One

Affordability of defence equipment

1.1 Since 2012, the Ministry of Defence (the Department) has published an annual Equipment Plan (the Plan). This sets out its plans for the delivery and support of equipment that the Armed Forces need to meet their objectives over the next 10 years (although many of these projects will be delivered over a longer period). This process began in 2012 as a way of assuring Parliament that the Department’s spending plans are affordable. From 1 April 2016 to 31 March 2026, the Plan has a total budget of £177.9 billion for:

equipment procurement (£81.9 billion);

equipment support (£90.7 billion); and

a contingency provision (£5.3 billion, of which £1.4 billion is ring-fenced for use by the Nuclear Enterprise).2

1.2 Fiscal responsibility for the Plan is delegated to the four front-line Commands of Air, Army and Navy, and Joint Forces Command, and the Strategic Programmes Directorate within the Department’s Head Office (collectively known as ‘the Commands’). Responsibility for programmes and projects remains principally with the Defence Equipment and Support organisation. Information Systems and Services is the responsibility of the Joint Forces Command, which carries out the project delivery role for IT projects. The Commands review changes in project costs through quarterly reporting by the project teams. The Department’s board approves funding for equipment projects (including changes to previously approved funding), and is kept regularly informed of progress on major projects. Cost pressures are designed to be managed wherever possible within and between Commands.

Our review of the Plan

1.3 In this report we give an overview of the Department’s 2016 to 2026 Equipment Plan for procurement and support. We examine whether:

the Department’s management of the Plan is helping to maintain the Plan’s affordability – in particular, how secure is the funding necessary to meet the Department’s increased equipment commitments (Part Two);

the costings in the Plan are based on assumptions that are reasonable and consistent (Part Three); and

the Department’s Affordability Statement contains appropriate information to make the reader aware of key assumptions and risks (Part Four).

Strategic Defence and Security Review 2015

1.4 In November 2015, the government published its latest Strategic Defence and Security Review (the Review).3 This set out the government’s defence policy and the methods for achieving its strategic objectives, along with the resources needed to do so. The Spending Review conducted by HM Treasury, and published alongside the Review, set a minimum Plan budget for the next five years.

1.5 We published a memorandum on the potential impact of the Review on the Plan where we noted that the Department faced significant challenges in the next 10 years in addressing capability gaps and enhancing current capabilities, because of the need to:

realise significant efficiencies;

manage cost growth; and

balance the costs of existing and future requirements.4

Our approach

1.6 Our audit approach is set out at Appendix One and our evidence base at Appendix Two. To support our review of the assumptions that underpin the forecast costs of the Plan, we reviewed in detail the cost estimates and underlying cost assumptions for nine of the Department’s largest procurement projects and seven support projects. Together, they constitute 41% by value of the spending in the Plan. Summaries of our findings on the projects in our sample are included at Appendix Three.

1.7 To support our review of the sources of funding for the Plan, we interviewed senior finance officers in the Department and reviewed papers relating to efficiency initiatives, business case submissions to the Defence Board, and explanations of central adjustments to the Plan.

1.8 We also worked closely with the Department’s Cost Assurance and Analysis Service (CAAS) to understand the work it has done through its independent cost reviews and estimates. CAAS is part of the Defence Equipment and Support organisation and provides independent assurance to the Department on costing and pricing work carried out by project teams and others.

2 This covers a range of related projects, which includes the Dreadnought and Astute submarine programmes.

3 HM Government, National Security Strategy and Strategic Defence and Security Review 2015, Cm 9161, November 2015.

4 Comptroller and Auditor General, Ministry of Defence: Impact of the Strategic Defence and Security Review on the Equipment Plan, Session 2016-17, HC 319, National Audit Office, June 2016.

Part Two

Cost of the Equipment Plan and sources of funding

2.1 In this part of the report we explain how the 2016 Equipment Plan (the Plan) has a much larger year-on-year increase in spend than its predecessors, and how this increase relates to the new commitments announced in the 2015 Strategic Defence and Security Review (the Review). We explain that most of the money required to fund the new commitments within this year’s Plan will come from within existing resources, including ‘headroom’ and efficiency savings. Finally, we comment on the progress the Ministry of Defence (the Department) is making to achieve these required efficiency savings.

Changes to the 2016 Equipment Plan

2.2 The 10-year budget profile of the 2016 to 2026 Plan is £177.9 billion, split between equipment procurement of £81.9 billion, equipment support of £90.7 billion and a £5.3 billion contingency provision (Figure 2). This represents a net increase of £11.6 billion from the 2015 to 2025 Plan (Figure 3). This 6.9% increase since 2015 contrasts with the 1.2% increase in the value of the Plan between 2013 and 2015 (Figure 4), and reflects decisions reached during the Review.

2.3 There are £24.4 billion of additional commitments arising from the Review. These commitments include new equipment, such as the Poseidon Maritime Patrol Aircraft, as well as extensions to current capabilities, such as the life extension of the first production tranche of Typhoon fast jets. In addition to the project-level commitments, £3.9 billion of commitments have not yet been costed in enough detail to permit distribution to individual project budgets, and are held in the Plan at Command level. These costs will change as the Department develops its plans during 2016-17. The £24.4 billion of commitments are largely funded from re-allocation of budget and efficiency savings from within the Plan itself, which results in a net increase of £11.6 billion of the Plan overall (paragraphs 2.8 to 2.11, 2.17 and 2.18).

2.4 The increases within the Plan are not evenly spread across the different Commands. Commitments resulting from the 2015 Review fall mainly within Joint Forces Command, Army Command and Air Command (Figure 5). The budgets of Navy Command and Strategic Programmes (which includes the Nuclear Enterprise) have small increases due to some existing projects increasing their level of spend at the end of the 10-year period covered by the Plan.

2.5 As we reported last year, there is increasing pressure on the Department to find room in the £91 billion support budget for the costs of supporting new equipment entering service.5 Since 2013 the value of the overall support budget has increased by £1.2 billion (1.3%), but within that the estimated cost of supporting new equipment has risen by 58%, from £14.8 billion to £23.4 billion.6 The Department will face an increased challenge in controlling its support cost budget following the Review as new procurement commitments will require support in future Plans, while some equipment due to be retired is now being retained with an extended life.

Source of funding for the 2016 Plan

2.6 To ensure the Plan remains affordable following the additional commitments announced within the 2015 Review, the Department has identified several sources of increased funding. Maintaining affordability depends on re-allocating funding from elsewhere in the Plan, making efficiency savings from both within the Plan and across the wider defence budget, and obtaining additional funding from the government.

2.7 Figure 3 shows how the size of the 2016 Plan’s core programme has increased since last year and the source of the new funding. The different elements of these funding plans are discussed below.

Re-allocation of funds previously held in the Plan as ‘headroom’

2.8 In previous years the Department maintained ‘headroom’ to fund extra projects beyond the core programme according to its military priorities. In the 2015 Plan the Department made provision for headroom of £7.3 billion at Command level and also held an additional £2.2 billion of headroom centrally. In addition, the Department originally intended to increase headroom by £1.2 billion within the 2016 Plan. In the 2016 Plan all £10.7 billion of this headroom has been used to fund commitments in the Review and other projects within the core programme. Some of these projects would previously have been earmarked for headroom funding. While the Review has attempted to anticipate future requirements, at least for the next five years, funding for any emerging requirements in the future will have to be found from within Commands’ existing budgets. This will significantly reduce the flexibility of the Department to respond to future defence procurement needs.

Required efficiency savings from within the Plan

2.9 To ensure the affordability of the Plan, the Department must achieve £7.3 billion of savings – £5.8 billion of savings must come from within the Plan itself, with the remaining £1.5 billion coming from elsewhere within the defence budget. This is in addition to savings initiatives already incorporated into the Plan from previous years.

2.10 At the time of the 2016 Plan, the Department had, at a high level, identified that the £5.8 billion of savings forecast to be generated from within the core programme would come from three broad categories:

£1.7 billion from reviews of contracts under the Single Source Contract Regulations. The Department’s Single Source Advisory Team calculated this amount based on the likely outcomes from negotiations on a projected pipeline of single-source procurements. There was no input from the Single Source Regulations Office itself. Only just over one-third of this amount is expected to be generated in the first five years;

£3.3 billion from transformation activities within Defence Equipment and Support carried out by the organisation’s Managed Service Providers; and

£0.8 billion from several other sources, including continuing reviews of support projects, testing and evaluation work.

2.11 In addition, the Plan now also contains £897 million of savings over the next 10 years from the Department’s IT programme, including the savings from disaggregation of the Defence Core Network Services programme mentioned in our 2015 report.

Required efficiency savings from the wider defence budget

2.12 The Department must make £1.5 billion of savings from outside the Plan budget. These savings will be achieved from within three current savings initiatives:

£2.4 billion of savings from achieving a 30% reduction in the size of the defence estate by 2040;

a 30% reduction in the size of the civilian workforce by 2020; and

£2 billion of savings from military and civilian pay restraint by 2020.

2.13 Figure 6 shows that the Department currently spends 42% of its core budget on equipment and support. Prior to the Review, the government announced in the 2015 Summer Budget that the defence budget would rise by 0.5% above inflation for the rest of the Parliament.7 Figure 7 shows that the percentage of the defence budget devoted to the Plan will increase from 42% in 2016-17 to 49% in 2021-22, before falling back to 43% at the end of the period, assuming that the Spending Review commitment is extended through to 2025.8 As the Plan takes up a greater proportion of the defence budget going forward, the scope to generate savings from within non-equipment budgets becomes increasingly difficult.

Progress towards delivering the required efficiency savings

2.14 When setting the new targets for efficiency savings from within the 2016 Plan and the wider defence budget, the Department did not identify which specific projects would contribute savings before finalising the Plan. It has subsequently begun developing detailed savings plans to achieve these targets during 2016-17 as part of a wider programme of efficiency savings targets across the Department. To support this, the Department has put in place structures to oversee the savings programme.

2.15 An Efficiencies Delivery Board is coordinating work to monitor savings delivery across the Department, including those within the Plan. The Board initially focused on defining baselines and agreeing reporting mechanisms, and is now using these reporting mechanisms both to hold top-level budget holders (TLBs) to account for delivery, and to help TLBs identify and mitigate delivery risks as early as possible.9 While the Board will continue to monitor progress, the Commands and other TLBs holders are responsible for delivering the savings. Commands have, in turn, spent the first part of 2016-17 identifying the sources of these savings at project level, which will be incorporated into project costings in the 2017 Plan.

2.16 The Department’s internal audit function reviewed the processes put in place for monitoring and reporting of savings. They concluded that while the Department had set up a strong governance framework, to facilitate the delivery of the required savings the Department will need to:

establish clear baselines against which savings are to be measured;

establish a robust audit trail;

put in place reporting arrangements which mitigate the risk of double-counting;

report on risks to deliverability and sustainability of savings; and

create clear lines of accountability for delivery within budget holders.

Departmental progress in achieving existing savings initiatives within the Plan

2.17 Affordability of the 2016 Plan depends on £7.1 billion of savings targets that have been carried forward from previous Plans (Figure 8). The target achieved to date is £4.6 billion, leaving £2.5 billion still to be achieved.

2.18 The Department’s progress towards its existing savings targets varies significantly. The Department estimates that it is more than 80% of the way to achieving its support budget savings target (compared with 61% last year), although it no longer monitors progress against this target at project level. By comparison, it estimates that it has definitely secured 27% of its 10-year savings target contained within the Complex Weapons pipeline agreement with the contractor after six years of the programme. This raises concerns about whether the target will be achieved, although a further £458 million of spend has been ‘avoided’ to date, but is ‘at risk’ until the relevant project phase has completed. These savings have already been removed from the project team budgets, so if savings are not met the shortfall has to be met from elsewhere in Command budgets. The Submarine Enterprise Performance Programme project team estimates that it has achieved 73% of its target amount, but currently forecasts a shortfall against the overall target. This is a snapshot and the various parties will continue to seek savings opportunities. The predicted shortfall is due to savings which are the responsibility of the Department rather than contractors.

New funding for the Plan

2.19 The majority of funding for the 2016 Plan comes from re-allocation of existing funding within the Plan and achieving efficiency savings. The sole source of new funding associated with the Review comes from the newly created Joint Security Fund (the Fund). The 2015 Spending Review stated that £6.4 billion will be available to the Department from the Fund. Of this, £2.1 billion will be available up to 2020, with the Department planning for an additional £4.3 billion from the Fund in the following five years.

5 Comptroller and Auditor General, Major Projects Report 2015 and the Equipment Plan 2015 to 2025, Session 2015-16, HC 488, National Audit Office, November 2015.

6 The Department did not provide an equipment support budget until 2013.

7 HM Treasury, Summer Budget 2015, HC 264, July 2015.

8 This excludes additional funding from the Joint Security Fund announced in the Spending Review, the funding profile of which is still uncertain.

9 Top-level budget holder refer to the four front-line Commands of Air, Army and Navy, and Joint Forces Command, and the Strategic Programmes Directorate within the Department’s Head Office, who hold fiscal responsibility for the Plan.

Part Three

Accuracy of cost estimates in the Equipment Plan

3.1 In this part of the report we explain that the level of uncertainty of cost estimates in the Equipment Plan (the Plan) has increased. This is due to an increased proportion of new projects, pressure on costs in existing projects, and because the Ministry of Defence’s (the Department’s) costing policy is particularly prone to under-estimation when projects are in their early stage. We explain:

the context for these estimates – how the Department estimates the costs of equipment projects, and how and why these cost estimates increase;

the quality of cost forecasting by project teams based on our review of the costing assumptions in 16 of the largest projects by value, nine procurement and seven support projects; and

the contingency levels within the Plan and whether they appear adequate.

3.2 More detail about movements in forecast costs for projects in our sample is given at Appendix Three.

Changes to cost estimates in the 2016 Plan

Impact of new commitments in the Strategic Defence and Security Review

3.3 The new equipment commitments introduced by the 2016 Strategic Defence and Security Review (the Review) have contributed to significant movements in forecast costs of projects in our sample, as shown by Figure 9. Change on this scale makes it more difficult to manage the portfolio from year to year.

3.4 Changes in forecast costs within our sample since 2015 are detailed in Appendix Three and fall into four categories which are representative of the £24.4 billion of overall commitments arising from the Review:

Two projects were added to the Plan following the Review (the Poseidon Maritime Patrol Aircraft and Mechanised Infantry Vehicle), together valued at £5.5 billion over 10 years. The cost estimate for the former may be vulnerable to exchange rate fluctuations while the latter is based on assumptions that could change significantly as plans develop, depending on the choice of procurement route.

‘Future Beyond Line of Sight’ communications system procurement and Grapevine Global Connectivity projects’ budget increases reflected changes of assumptions about the procurement approach, adding £4.2 billion to the Plan. The latter is part of a change of approach to the provision of defence information communications technology services to improve deliverability. The cost estimates for both of these projects may change significantly in the future.

Increases of £2.3 billion in six other projects are largely due to the Nuclear Enterprise (paragraph 3.6) and additional Review commitments for the F-35 and Typhoon projects.

Reductions of £2.2 billion in six projects are the result of scope reductions (Ajax armoured vehicle and Atomic Weapons Establishment), re-profiling of costs in the light of a change of requirement (Type 26 Global Combat Ship) and the running-down of a current contract (Skynet 5 satellite).

Obsolescence of cost estimate for Type 26 Global Combat Ship

3.5 The Type 26 Global Combat Ship is the highest-value non-nuclear procurement in the Plan. The Review significantly changed the requirement, reducing the number of Type 26s from 13 to eight, the shortfall in ship numbers to be filled by the creation of a new class of frigate. Two additional offshore patrol vessels were also ordered. The forecast in the 2016 Plan could not fully reflect this change of requirement, due to the need for further refinement of the costings for the remaining Type 26 ships and the need to develop a design for the new class of frigate. The Department expects to have completed this re-costing work in time for the 2018 Plan. As a result, these costs could change significantly. One of the questions to be resolved is how the new class of frigate will be funded. The Department has re-profiled the delivery schedule of the Type 26 programme, resulting in the removal of £487 million from the early years of the cost forecast. Work is still ongoing to profile the cost of delivering the revised number of eight Type 26 ships across the later years of the programme. The Department expects this to be completed in order to support the main investment decision in summer 2017.

Changes to cost estimates of existing projects in the Plan

3.6 Cost increases of £836 million in the Nuclear Enterprise programme (specifically the Dreadnought submarine platform, its nuclear propulsion unit and boats 4-7 of the Astute submarine programme) are not linked to the Review. The Department attributes the cost increases to contractors not achieving planned levels of activity, and their risk-averse pricing of bids. The cost estimates for the period 2016 to 2026 have increased by 6%, 20%, and 10% for these three projects since 2012. As part of the Review, the post of Director General Nuclear was created within the Department to draw all elements of the Nuclear Enterprise Programme together into a coherently managed portfolio and the Department is in the process of creating a Submarine Delivery Body dedicated to the delivery of submarine procurement.

3.7 Within the Nuclear Enterprise programme these increases were partly offset by a £691 million reduction in the forecast for the next 10 years of the Atomic Weapons Establishment management and operations contract. The Department signed a new contract with the managing contractors on 31 March 2016, which revised downwards its expectations of future activity levels on their sites, both in terms of capital works and running costs. This was partially offset by an increase in the management fee rate. The project will be required to make further savings over time.

3.8 During the year, the Department became aware of concerns about the accuracy of contractor-supplied data incorporated into the costings of a number of projects within the Nuclear Enterprise programme. While the contractor is making efforts to improve the quality of its models, the implications of this for project costings are still unclear. This illustrates the significant uncertainty associated with a project with such large spend and timescale.

How project teams account for risk

3.9 Generating a range of probable costs is good practice as a means of allowing for the uncertainties around projects under development, and helpful in understanding risk. Projects have to take account of several types of risk:

‘Risk inside costing’. These are risks which the project teams include within their cost model. The project teams will identify a number of different risks to the project, an estimate of the cost of each risk and the likelihood of each risk occurring. Together, these risks undergo statistical analysis to generate a range of the possible cost outcomes from 10th to the 90th percentiles. Project teams will use the 50th percentile risk for inclusion in their overall project cost estimate.

‘Risk outside costing’, which occurs in two forms:

a modelled risks, where a project uses a cost model that generates a range of possible cost outcomes from the 10th to the 90th percentiles, the project teams take the difference of the 50th and 90th percentile estimates and calls this the ‘risk outside cost’. Eight projects in our sample did this, with a total value of £1.7 billion; and

b un-modelled risks – these are risks which the project teams have decided not to include within their cost model, for instance if the risk has a low probability of occurring but high financial impact if it did. Project teams may nevertheless calculate the potential cost of these low-probability risks. For 14 projects in our sample the combined value of both categories of ‘risk outside costing’ was £8.6 billion.

Basis of cost estimates and historic performance of the Department against them

Cost estimating at the 50th percentile

3.10 The Department requires project cost estimates in the Plan to be forecast at the median of the potential cost range; this is referred to as the ‘50th percentile cost’. At this point, each project is considered to be equally likely to cost less or more than this estimate. Some variation against this estimate can therefore be expected. Forecasting requires judgement, so costs are not absolute and can be over- or under-stated. Nevertheless, the Plan is predicated on variations in the 50th percentile costs for each project balancing out across the portfolio, within the tolerances provided by the Department’s contingency provision. If this does not happen, the Department can intervene to bring forward projects or accept delays in order to remain within the budget.

3.11 Costing a portfolio at the 50th percentile irrespective of complexity and level of maturity may not always be appropriate. This approach is not followed universally across government. For example, Network Rail costs its project portfolio at the 80th percentile. We have seen evidence in our assurance work of contractors refusing to price bids at the 50th percentile as they regard this as exposing themselves to too much risk.

3.12 The Cost Assurance and Analysis Service (CAAS) does not consider it appropriate to cost all projects at the 50th percentile irrespective of the status of cost estimates. For 66 projects where it conducted an independent cost estimate in 2015-16, its ‘realistic outturn’ project cost was based on an estimate above the 50th percentile in 24 cases.

3.13 CAAS has also carried out two pieces of analysis based on data it has been collecting since 2009 to examine how project cost estimates change between the start of the design phase of a project and either the most recent estimate, or the point of service entry. CAAS analysed:

the overall performance of a historic dataset of 331 procurement projects; and

a more recent subset of 129 of these projects which entered the design phase after 1999, or entered service after 2008.

The analyses show that cost estimates had increased by an average of 23% for the former dataset and 10% for the latter.

3.14 The earlier a project is in its lifecycle the greater the scope for cost estimates to grow at later stages. Figure 10 shows the stage of development of the largest procurement projects, which between them constitute 50% of the value of the Plan in 2012 and 2016. It shows that a higher proportion of these large high-value projects are at an early stage of the project life cycle in 2016, as you would expect following the inclusion of new commitments in the Review. This significantly increases the exposure of the Plan to cost growth of this type.

Impact of corporate planning assumptions on cost forecasts

3.15 It is important to incorporate realistic estimates of inflation and foreign exchange rates within cost models as these can be significant elements of future costs. Project teams are provided with guidance on the inflation and foreign exchange rates to be used when estimating future costs. In past reports, we have expressed concerns about the failure by project teams to follow this guidance consistently.

Treatment of inflation

3.16 In last year’s sample we found that seven projects demonstrated poor practice in this area. We considered that none of the 16 projects in this year’s sample took an unreasonable approach to the treatment of inflation. However, partly prompted by our work in previous years, analysis by CAAS suggests that cost estimates for the Dreadnought programme may be overstated by £1.7 billion over the period to 2033-34 due to the cost model not following more usual practice in calculating inflation.

Foreign exchange

3.17 The international nature of defence procurement means that many of the Department’s largest procurements are paid for in foreign currency, examples being the F-35 Joint Strike Fighter and the Poseidon Maritime Patrol Aircraft. The Department forecasts that it will spend £18.6 billion in US dollars and £2.6 billion in euros over the 10-year period. Using the Department’s foreign exchange rate planning assumptions this equates to $28.8 billion and €3.2 billion respectively. Within this foreign currency spend, approximately $8.5 billion and €1 billion have been added as a result of new commitments following the Review.

3.18 The Department manages the risk of exchange rate exposure for the US dollar and euro using forward purchase contracts with the Bank of England and covered 89% and 99% respectively of 2015-16 in-year expenditure in US dollars and euros. When forward purchase does not cover the whole difference between the corporate rates used in the cost estimate for a project and the actual rate paid, the Department may chose to fund the variance from the wider defence budget.

3.19 Following the decision to leave the European Union, the pound has significantly lost value against other currencies. If planning is undertaken using unrealistic rates there could be serious affordability issues for those projects, and possibly for the Plan as a whole. As at 10 January 2017, the pound was 21.4% below the exchange rate with the US dollar and 4.2% below the exchange rate for the euro used in the Department’s planning assumptions.

The Department’s own assessment of the sufficiency of contingency to cope with cost growth

3.20 The Department has set aside contingency funding of £5.3 billion within the Plan to cover the risk of Commands being unable to cover cost growth due to:

failures to meet savings targets;

risks materialising;

unexpected events that have financial impacts; and

over-optimism in project teams’ costings.

3.21 Within the £5.3 billion contingency (representing 3% of Plan expenditure), £1.4 billion is set aside for cost growth within the Nuclear Enterprise programme. This element of the contingency is available from this year, while the more general contingency is not expected to be required until 2019-20 at the earliest. The Department has stated, however, that the nuclear contingency would be available for wider use if not required for its original purpose. The total contingency represents 3% of the value of the Plan, compared with 2.6% in 2015. The total contingency for the Dreadnought programme is £10 billion over the life of the programme and is based on the cost of the four Dreadnought nuclear submarines.10

3.22 Each year, CAAS estimates the extent to which project teams may be underestimating the real costs of projects in the cost forecasts that together make up the 10-year Plan. Figure 11 shows how the estimates have changed since last year. CAAS considers that the cost of the Plan as a whole is likely to be under-estimated by £4.8 billion (within the Department’s contingency provision of £5.3 billion). Due to the timing of the Review, CAAS was not able to undertake cost estimates for additional commitments in the Review during 2015-16. This contributed to a fall in the proportion of Procurement Plan costs covered by specific cost estimates to 60% compared with 71% in 2015. The proportion of the Support Plan covered increased to 56%.

10 Calculation based on the results of leading research into cost estimating on past public infrastructure projects.

Part Four

Disclosures within the Department’s affordability statement

4.1 The Ministry of Defence’s (the Department’s) annual Statement on the affordability of the Equipment Plan (the Statement) should:

aid transparency;

show whether the Equipment Plan (the Plan) is affordable and achievable; and

give the defence industry more information for planning.

4.2 We examined whether the Statement contains adequate and sufficient disclosures for users of the Plan to fully understand the key assumptions that:

have been used to generate the Plan’s costs and the sensitivity of the costs to changes in those assumptions; and

the Department has made about the level of available future funding, and the sensitivity of affordability of the Plan to changes in those assumptions.

Disclosure regarding cost and funding assumptions

4.3 The Statement for 2015-16 is similar in format and content to those of earlier years. It breaks down the Plan budget into its component parts. This enables the reader of the Statement to identify:

the costs related to procurement and support;

the unallocated budget; and

the contingency provision detailed on a year-by-year basis for the reported 10-year period.

4.4 There is sufficient discussion for the reader to understand the nature of and rationale for the different components of the Plan, including the contingency provision and nuclear provision.

4.5 The Statement’s disclosures have not, however, noticeably built on the improvements we noted last year in a way that would help readers to fully understand the fundamental assumptions that underpin the Plan, and the risks and sensitivities in implementing it within budget. Specifically, there is still scope to explain:

the range of possible amounts that the total Plan could cost, for example through the provision of information on the greater level of uncertainty of costing at an early stage of a project’s development. We have made recommendations about this in the past and it is particularly relevant this year, in the light of changes to the Plan resulting from the 2016 Strategic Defence and Security Review (the Review). Also, an indication of which projects present the greatest threat to the Plan’s overall affordability should be highlighted;

the total value of ‘risks outside costing’ and the impact on affordability should these risks materialise;

the approach to foreign exchange assumptions and the sensitivity to these assumptions. Planning assumptions no longer reflect actual foreign exchange rates and this represents an increased risk to affordability. The Department’s approach to mitigating this risk should also be explained in more detail; and

the source of the efficiency savings required to fund the Plan, both from within the Plan itself and the wider defence budget. In addition, progress against the previous efficiency savings, including key milestones, should be clarified.

Appendix One

Our audit approach

Affordability of the Equipment Plan

1 As in our previous reports, we constructed a programme of work to test the Ministry of Defence’s (the Department’s) assertions within its assessment of the cost of the Equipment Plan (the Plan) and the funding available. The model breaks these assertions down into a set of hypotheses, as set out in Figure 12. Appendix Two sets out the evidence we used to test these hypotheses.

2 To support our review of the assumptions that underpin the forecast costs of the Plan, we looked in detail at a sample of nine of the Department’s largest procurement projects and seven support projects, which together constitute 41% by value of the expenditure in the Plan. Appendix Three sets out the results of this review.

3 Our judgements about affordability are informed by the work of the Department’s Cost Assurance and Analysis Service (CAAS). In particular, CAAS compiles a report in which they set out their view on the accuracy of the cost estimates in the equipment procurement and support plans. This is partly based on ‘independent cost estimates’, which this year constitute 59% by value of the Equipment Procurement Plan and 56% of the Equipment Support Plan. CAAS produces an overall view on affordability by modelling the remainder of the project procurement portfolio, and applying a cost adjustment factor to the support portfolio.

4 This year we have varied the degree of testing on projects in our sample based on the CAAS view of the accuracy of each project’s cost estimates. To assure ourselves that this was the correct approach we drew on the principles set out in the professional standard ‘ISAE 3000: Assurance Engagements Other Than Audits or Reviews of Historical Financial Information’ and utilised work undertaken in 2014 and 2015 to test and conclude upon the independence and capability of CAAS.

5 To support our review of the sources of funding for the Plan, we interviewed senior finance officers in the Department and reviewed papers relating to efficiency initiatives and business case submissions to the Defence Board.

Appendix Two

Our evidence base

1 We reached our conclusions based on our analysis of evidence collected during fieldwork between July and October 2016.

The sample of projects

2 Selection of the sample of projects is now based on the size of the projects, since failure to control cost growth on these projects would present the greatest threat to the affordability of the Equipment Plan (the Plan). This means that the sample can place more focus on support projects. The 16 projects in the sample – nine procurement and seven support projects – are set out in Appendix Three. Together, they represent 41% of the value of the Plan.

3 We built a model to test the Ministry of Defence’s (the Department’s) assertions within its assessment of the cost of the Plan and the funding available. The model breaks these assertions down into a set of hypotheses. Our audit approach is set out in Appendix One. We referred to the following sources of evidence to test these hypotheses on our sample of projects:

We reviewed alternative cost estimates generated by the Department’s internal Cost Assurance and Analysis Service (CAAS). Where there were significant differences between the CAAS and the project teams’ estimates we evaluated the risk to the affordability assertion.

We reviewed the cost models and cost-estimating techniques used in generating cost forecasts; risk management and how uncertainty and risk are built into costings. We also matched actual in-year spend to contracted amounts.

We reviewed the application of central government guidance on how to treat inflation and foreign exchange, and assessed the reasonableness of alternative approaches.

We reviewed historical data on actual costs against planned spending. This enabled us to assess the Department’s ability to forecast costs accurately.

Affordability of the Equipment Plan

4 CAAS supplies us with an overall view of the accuracy of cost estimates at Plan-level, based partly on its independent cost estimates of major projects and partly on modelling of future costs for the balance of the portfolio.

5 In 2016, CAAS carried out in-depth independent cost estimates of 76 of the major projects in the portfolio (including 11 of the projects in our sample). It produces an estimate for the extent to which the costs of the Plan may be understated by combining these estimates with a modelled value for the rest of the project population using its historic project performance model.

6 The Department has supplied us with a detailed breakdown and explanation for central adjustments to the Plan at Operating Centre and top-level budget holder-level.

Availability of funding

7 We interviewed senior finance officers within the Department, who supplied us with documentation about the governance arrangements for delivering the efficiency savings required to achieve funding targets and the extent of current plans at Command level for their identification and achievement.

Appendix Three

Our findings: cost estimates of a sample of projects

Design and Production by NAO External Relations
DP Ref: 11408-001