Report

by the Comptroller
and Auditor General

Ministry of Defence, Defence Equipment and Support

Reforming defence acquisition

HC 946 Session 2014-15 26 February 2015

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO, which employs some 820 employees. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund have used their resources efficiently, effectively, and with economy. Our studies evaluate the value for money of public spending, nationally and locally. Our recommendations and reports on good practice help government improve public services, and our work led to audited savings of £1.1 billion in 2013.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 24 February 2015

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

23 February 2015

We examined progress in tackling the accepted weaknesses within Defence Equipment and Support (DE&S), the Ministry of Defence’s progress in carrying out organisational change and the challenges that remain within DE&S.

© National Audit Office 2015

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.gsi.gov.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

10597 02/15 NAO

The National Audit Office study team consisted of:
Nigel Vinson, Lee Nicholson and Lee Staley, under the direction of Lee Summerfield.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key facts

Summary

1 In 2013-14, the Ministry of Defence (the Department) spent £13.9 billion buying and supporting military equipment. Over the next 10 years it plans to spend some £162.9 billion, as set out in its Equipment Plan 2014−2024.

2 Providing military equipment relies on a partnership between:

the Department’s head office, which gives strategic oversight;

the Armed Forces (the commands), who say what equipment they need; and

Defence Equipment and Support (DE&S), which buys and maintains military equipment, known as defence acquisition.

3 Defence acquisition is complex and demanding. It requires close partnering with industry and often working with other governments on international collaborative defence projects.

4 In 2013-14 DE&S employed some 16,000 permanent members of staff, an estimated 3,400 contractors, and had running costs of £1.3 billion. DE&S needs specialist skills, such as commercial, financial, engineering, project and programme management and supplier management.

Defence acquisition reform

5 The Department has long wrestled with problems in defence acquisition. By 2010 poor performance led to an ‘overheated’ equipment programme, where a gap of £38 billion emerged in the defence budget. Building on an independent report commissioned by the Department, DE&S estimated £1.5 billion was lost annually because of three systemic issues:

to offset cost overruns in the equipment programme, major projects were routinely delayed or reduced in scope as well as impacts across other areas of defence spending;

the Department struggled to make strategic investment decisions because of blurred roles and accountabilities between head office, the commands and DE&S; and

poor skills and management freedoms within DE&S to maximise the value of funds allocated to buying equipment and manage projects through their life.

6 The need for change was widely acknowledged and made in a number of Department commissioned reviews, as well as in reports by the House of Commons’ Committee of Public Accounts and Defence Committee. There are three strands to the Department’s strategy for reforming defence acquisition. The Department has sought to:

develop an affordable equipment programme, in the form of the Department’s ten-year Equipment Plan;

strengthen the roles and responsibilities of head office, DE&S and the commands, through a new operating model in line with Lord Levene’s 2011 review; and

improve DE&S’s skills and capabilities through the Department’s Materiel Strategy.

7 To improve acquisition within DE&S, the Department initiated the Materiel Strategy, designed to:

give DE&S greater management freedoms to introduce private sector practices, skills and tools;

formalise and introduce rigour in dealings between DE&S, the commands and head office; and

improve DE&S’s efficiency and deliver the Equipment Plan for less.

8 The Department evaluated three organisational structures to meet these objectives:

Trading fund

A public corporation, government-owned and operated, financed from its trading activity.

Executive non-departmental public body with a strategic partner
(ENDPB/SP)

A special-purpose body that manages part of the government’s work, contracting with the private sector for business support.

Government-owned, contractor-operated model (GoCo)

A company where the government owns the strategic direction and assets, but the GoCo would be operated on a for-profit basis by a private company, accountable to its shareholders.

9 The Department opted for the structural reform that offered the highest evaluated benefits, but carried greater assessed implementation risks, the GoCo. This was actively pursued between 2012 and 2013, and has now been discontinued in favour of establishing DE&S as a bespoke trading entity. The Department chose the bespoke trading entity when terminating the GoCo competition in December 2013. The Department evaluated the bespoke trading entity as a lower-risk option, but with the potential to bring the functional improvements necessary without requiring complex risk transfer to the private sector.

Scope of the report

10 We examined progress in tackling the accepted weaknesses within DE&S, the Department’s progress in carrying out organisational change and the challenges that remain within DE&S. The Department has set initial key performance targets for DE&S, but is yet to set out fully how it will measure whether the bespoke trading entity is improving defence acquisition. We have examined progress against the Department’s objectives for reforming DE&S, which have remained the same since 2011. They are to improve:

capabilities and skills of DE&S staff;

systems and tools available within DE&S; and

the interface between DE&S and the commands.

Key findings

11 The Department has made progress in stabilising the equipment programme, and in clarifying the roles and responsibilities of head office, DE&S and the commands. There are several positive features in the 2014−24 Equipment Plan, not least the relative stability of forecast project costs of core equipment. A new operating model came into effect in 2014, and the Department continues to embed this approach. Responsibilities and accountabilities for defence acquisition are clearer, with greater separation between those requesting equipment (the commands) and DE&S as deliverer of military equipment (paragraphs 1.6 to 1.11).

12 The Department’s preferred option for securing change in DE&S through a government-owned, contractor-operated (GoCo) organisation proved undeliverable. The Department chose the higher-risk option of changing DE&S into a GoCo. It assessed that greater benefits would be secured through the direct involvement of the private sector, with mutual obligations defined by contracts, as well as the capacity to offer enhanced reward packages to attract and retain key skill groups. That proposal proved undeliverable and the Department halted it in late 2013. To the point the GoCo was terminated the Department had spent £33 million and two and a half years developing the Materiel Strategy (paragraphs 1.12 to 1.19).

13 Although the GoCo option was halted, the Department gained a better insight into its business needs within the context of an affordable equipment programme. The Department’s experience of dealing with the private sector and its approach to transferring financial risk during the GoCo competition has given DE&S a better understanding of its business needs. For example, DE&S has a better understanding of how staff spend their time and where key skills are needed (paragraph 2.3).

14 The bespoke trading entity structure allows the Department to retain greater control of DE&S, while meeting the objectives of the reform. The Department changed DE&S into a bespoke trading entity in April 2014. DE&S remains in the public sector but has freedoms and flexibilities from civil service pay rules. Departmental and DE&S governance structures have been enhanced, with a new board structure and initial key performance targets for DE&S. The private sector has been engaged to support continued reform in several areas within DE&S. The Department has also set DE&S a target to make savings from operating costs (paragraphs 2.7 to 2.8, 2.14, 2.16 to 2.21).

15 To assess whether the bespoke trading entity is delivering improvements to DE&S’s performance, it will be important for the Department to set robust measures of success. Being able to track benefits will be essential both to establish whether the bespoke trading entity is delivering acquisition reform at DE&S, and to allow the Department to take any additional measures to improve performance should it be necessary (paragraphs 2.22 to 2.26).

Conclusion on value for money

16 Success in all three elements in the Department’s strategy to reform defence acquisition is necessary to improve long-term value for money. The Department has made progress in improving the affordability of the equipment programme and clarifying roles and responsibilities. Reforming DE&S remains the most challenging of the three, but is essential nonetheless.

17 The failure of the GoCo competition and shelving of that option has cost time and money, but it has yielded some learning. If realised, the Department expected it to have given more rigour through the direct involvement of the private sector, with mutual obligations defined by contracts, as well as the capacity to offer enhanced reward packages to attract and retain key skill groups. Many of these benefits appear achievable through the bespoke trading entity. As the entity remains within the Department, a greater level of sponsorship and continuous rigour in measuring DE&S’s progress against clear goals will be required from the Ministry of Defence’s leadership, if the entity is to be a successful vehicle for acquisition reform.

Recommendations

18 Our recommendations are designed to help the Department gain assurance that the bespoke trading entity will address systemic weaknesses in defence acquisition:

a The Department needs to show how the bespoke trading entity will address systemic weaknesses in defence acquisition. The affordability of the Equipment Plan critically relies on improving DE&S’s performance to deliver equipment within existing budgets, including more than £6 billion of savings already factored into the Plan. The Department has agreed an overarching plan for DE&S and set out expectations for the organisation. However, it has not yet demonstrated the value-for-money case for the entity. Neither has it defined what becoming the leading defence acquisition organisation will look like. We would expect the Department to set out the value-for-money case once detailed plans are available in summer 2015.

b The Department should set realistic performance measures to ensure that the bespoke trading entity makes genuine improvements in acquisition. Having defined the characteristics of a leading defence acquisition organisation, the Department must have robust metrics to monitor DE&S’s progress towards achieving this goal by 2017. DE&S’s corporate plan includes initial high-level performance targets, and more detailed metrics should be set by summer 2015. Head office and the commands will need to be watchful and respond quickly if sufficient improvements do not materialise.

c DE&S should set metrics to ensure managed service providers are offering value for money by measuring their progress in enhancing skills and capabilities. DE&S is placing great reliance on managed service providers to define and make improvements in projects and programmes, human resources and management information systems. In return, DE&S expects to pay around a quarter of a billion pounds over three and a half years for this business support. DE&S has agreed contracts with the managed service providers whereby part of their fees are, in principle, linked to milestones and a set of metrics. It is essential, therefore, that DE&S establish these metrics and a baseline to track progress in implementing these reforms and transferring skills and capabilities to DE&S staff. This will help it keep on schedule to become the leading defence acquisition organisation by 2017.

Part One

Defence acquisition

How acquisition is organised

1.1 The Ministry of Defence (the Department) publishes annually an Equipment Plan. The Plan sets out how it will buy and support military equipment the Armed Forces (the commands) require over the next 10 years to meet the objectives of the National Security Strategy1 and the Strategic Defence and Security Review.2 From 1 April 2014 to 31 March 2024 the Plan is forecast to cost £162.9 billion.

1.2 Providing military equipment relies on a partnership between the Department’s head office, which gives strategic oversight, the commands that specify their requirements, and Defence Equipment and Support (DE&S), which buys and maintains military equipment (or ‘defence acquisition’). Defence acquisition is complex and demanding work that requires close partnering with industry. It often involves working with other governments on international collaborative defence projects.

1.3 In 2013-14 DE&S employed some 16,000 permanent members of staff, an estimated 3,400 contractors and technical support staff (which we call ‘contractors’) and had running costs of £1.3 billion.3 To perform its role, DE&S needs specialist skills, such as commercial, financial, engineering, project and programme management and supplier management.

The need for reform

1.4 The Department has long wrestled with problems in defence acquisition. By 2010 poor performance had led to an ‘overheated’ equipment programme, where a gap of £38 billion emerged in the defence budget. Building on an independent report commissioned by the Department,4 DE&S estimated £1.5 billion was lost annually to defence, because of three systemic issues:5

Overheated equipment programme

The Department often delayed or reduced the scope of major projects to find savings to tackle the overheated equipment programme. Too broad a range of tasks, too many types of equipment ordered, and at too high a specification, contributed to planned requirements significantly exceeding planned funding.

Blurred roles and responsibilities

Difficulties in taking strategic investment decisions on defence equipment due to blurred roles and responsibilities between head office, the commands and DE&S. The Equipment Plan was dominated by a ‘bottom up’ approach, making it hard for ‘top down’ strategic guidance to control the balance of investment.

Lack of skills and management freedoms

DE&S lacked skills and management freedoms to maximise the value of the funds allocated to buying equipment and managing these projects through their life. DE&S’s business capabilities were assessed as minimal for an organisation managing complex programmes.

1.5 The Department prioritised having an affordable plan for defence equipment and has made progress in Departmental reform to improve defence acquisition.

Tackling the overheated equipment programme

1.6 Through the 2010 Strategic Defence and Security Review the Department tried to address some immediate causes of the overheated budget.6 It cancelled some equipment projects and took some equipment out of service earlier than originally planned. In 2012 it used a new approach of setting a budget for a ‘core programme’ of key equipment projects, with extra sums for contingency and emerging requirements.

1.7 At £162.9 billion, the cost of the 2014−2024 Equipment Plan is £1.4 billion less than the previous Plan. It comprises:

a procurement budget (£68.854 billion);

a support budget (£80.576 billion);

a central contingency reserve (£4.600 billion);7

headroom (£8.003 billion);8

central provision (£1.152 billion);9 and

central over-programming (-£300 million).10

1.8 The Department’s management of the Equipment Plan has improved over time. There are several positive features in the 2014−2024 Equipment Plan, not least the relative stability of the forecast project costs of core equipment. However, risks remain, and affordability relies on DE&S having the skills and capacity to make more than £6 billion of savings, a significant proportion of which it has yet to identify. Our annual report on the assumptions underpinning the Equipment Plan gives greater detail.11

Strengthening roles and responsibilities

1.9 Recognising the need to address weaknesses in defence acquisition, the Department implemented a new organisational model. Based on a review by Lord Levene, the new model:12

has a smaller, stronger, more strategically focused head office;

streamlines the boards’ and committees’ structure;

defines clear roles and relationships between various parts of the Department; and

emphasises individual accountability.

1.10 In his third and final annual progress review in December 2014, Lord Levene noted that:13

“good progress… has been made in the last year around the future organisation and operation of DE&S: its new status as a bespoke trading entity within MoD, and the strengthened governance arrangements under an independently chaired, commercial-style board, seem right; the quality and morale of the top DE&S management team is good: and I congratulate MoD on securing extensive new pay flexibilities. These reforms augur well for the future.”

Lord Levene also noted, however, that the reforms were still at an early stage and the focus must now be on properly developing and implementing them.

1.11 These structural changes have clarified responsibilities for acquisition. Since April 2014, the commands manage equipment and support budgets and hold DE&S to account for project delivery (Figure 1).

Improving DE&S’s capabilities and skills

1.12 The Department has made progress in implementing strategic reform, particularly in creating a more affordable Equipment Plan. However, the Plan’s long-term affordability depends on making more than £6 billion in savings. DE&S has a role in delivering these. The Department remains committed to enhancing DE&S’s performance by addressing weaknesses in skills and capabilities.

1.13 Within DE&S, the Department set out to improve:

capability and skills within DE&S;

the systems and tools available; and

the interface between DE&S and the commands.

1.14 To make these improvements, in 2011 the Department launched the Materiel Strategy.

Materiel Strategy

1.15 The Materiel Strategy identified 14 different organisational structures that could potentially transform DE&S, to enhance its performance. Each option varied in how far it offered radical change.

1.16 In November 2011 DE&S’s executive board14 shortlisted three options: a Trading fund;15 an executive non-departmental public body with a strategic partner (ENDPB/SP);16 and a government-owned, contractor-operated (GoCo) organisation.17 The Department assessed how closely each option would meet DE&S’s business needs and the issues for implementing each option (Figure 2).

1.17 In July 2012 the Department selected to pursue a GoCo, to maximise the benefits of organisational change at DE&S (see Appendix Three). The GoCo proposition offered more than £1.6 billion in extra benefits over the other option − the ENDPB. Also, the Department developed a public sector comparator, known as DE&S plus (DE&S+), as a benchmark to measure GoCo bids and give a fall-back option should the competition to secure a GoCo fail.

1.18 Compared to other options, the Department recognised that there were higher risks with setting up the GoCo:

The Department wanted a 9.5-year contract covering £100 billion of Departmental expenditure, which would make it the largest GoCo of its type globally.

No single company could give all the business capabilities needed from a GoCo, and the Department expected companies to form into consortia to bid, limiting competition.

Implementing a GoCo would be complex and require transferring elements of DE&S’s business to head office and changes in the commands to ensure they had the capacity to work with the GoCo.

The Department would need to identify and mitigate conflicts of interest between GoCo bidders and existing Departmental contracts.

Other countries would need to agree that a GoCo could act as the Department’s agent on sensitive defence contracts.

A GoCo required significant Departmental financial investment, which increased the focus on getting a return on that investment. Internal scrutiny raised concerns that it would take between 8 and 9 years to make the financial savings needed to pay back the investment, compared to the GoCo’s planned 9.5-year contract term.

1.19 In December 2013 the Department halted the GoCo proposal because of a lack of competition − it received one detailed bid from industry (see Appendix Three).18 Up to that date, the Department had spent £33 million on the Materiel Strategy, including £25 million on contractors and advisers. Despite this setback, there remained consensus that radical reform of DE&S was still necessary.

1.20 Building on the previous work in the Materiel Strategy to prepare for the GoCo and DE&S+, the Department announced in December 2013 that DE&S would become a bespoke trading entity (the entity) on 1 April 2014.19

1.21 Under this structure, DE&S remains within the public sector, although now as an arm’s-length part of the Department, with its own board, chaired by a non-executive director. DE&S would then contract with the private sector for business support.

1.22 In Part Two we explore how the Department expects the bespoke trading entity to address each of DE&S’s business needs. As well as the incremental changes secured to DE&S’s capability in recent years alongside the Materiel Strategy.

1 HM Government, A Strong Britain in an Age of Uncertainty: The National Security Strategy, Cm 7953, October 2010.

2 HM Government, Securing Britain in an Age of Uncertainty: The Strategic Defence and Security Review, Cm 7948, October 2010.

3 Running costs include staff, consultancy, IT and communications, infrastructure, transport and movement costs.

4 The Review of Acquisition for the Secretary of State for Defence (the Gray Review) 2009, p. 6, estimated those loses to be in the range of £900 million to £2.2 billion per annum. Based on this review, DE&S took a figure of £1.5 billion per annum as a working level assumption.

5 Materiel Strategy Outline Business Case, January 2013, paragraph 1.52.

6 HM Government, Securing Britain in an Age of Uncertainty: The Strategic Defence and Security Review, Cm 7948, October 2010.

7 The Department has set a contingency to mitigate potential cost increases within the core equipment programme.

8 The Department intends to use this money to fund extra projects beyond the current core programme according to their military priority (and assuming they are affordable), so that it can reach the full equipment capability needed to meet its longer-term objectives.

9 An amount of budget, held as part of the Equipment Plan, which is not allocated to any project but is available to fund potential cost growth in the existing core programme.

10 The Department is planning work above the budgeted level to compensate for predicted underspending in-year.

11 Comptroller and Auditor General, Ministry of Defence, Major Projects Report 2014 and the Equipment Plan 2014 to 2024, Session 2014-15, HC 941-I, National Audit Office, January 2015.

12 Lord Levene of Portsoken KBE, Defence reform: an independent report into the structure and management of the Ministry of Defence, June 2011.

13 Letter from Lord Levene to the Secretary of State for Defence, 5 December 2014. Available at: www.gov.uk/government/uploads/system/uploads/attachment_data/file/388383/Levene_Third_Annual_Review_5_December_20140001.pdf

14 At the time the most senior leadership board in DE&S.

15 Trading funds are public corporations, government-owned and operated. They must finance their operations from trading activity.

16 An ENDPB is a special-purpose body responsible for part of the process of government. Each has a sponsor department with general oversight of its activity. A strategic partner is a private sector company contracted to give business support.

17 Under a GoCo, the government would own the organisation’s strategic direction but the GoCo would be operated on a for-profit basis by a private company accountable to its shareholders.

18 The detailed stage was the second of three stages of bidding and required the consortia to provide a comprehensive response against all areas of the Department’s requirements.

19 A concept made more explicit in HM Treasury, Managing Public Money, July 2013.

Part Two

Acquisition reform at DE&S

2.1 Department-wide changes in the Ministry of Defence (the Department) to ensure affordability and stability of equipment projects depend on tackling weaknesses within Defence Equipment and Support (DE&S). In this Part we examine progress in carrying out reform at DE&S. In particular, we have examined progress against the reform objectives to improve:

staff capabilities and skills giving DE&S a flexible workforce with access to all skills and tools required to do its work cost-effectively;

systems and tools to give a timely, accurate and agreed view of its business, to respond to challenges rapidly, and to maximise opportunities; and

the interface between DE&S and the commands ensuring clear accountabilities on what is required, by when and at what cost.

Improving staff capabilities and skills

2.2 DE&S has long-standing skills gaps, including airworthiness, engineering roles, commercial, logistics and project and programme management. Building on the work of an independent review for the Department,20 DE&S estimated that enhancing DE&S’s skills and improving its interface with commands could deliver between £1.03 billion and £1.45 billion of financial savings from the Equipment Plan each year.21 Skills gaps emerged primarily because DE&S could not recruit and retain enough skilled staff, exacerbated by a requirement to cut staff to meet Departmental efficiency savings.

2.3 DE&S must use existing skilled staff more effectively to eliminate gaps. DE&S has no system to routinely collect accurate and timely information on what skills it has and requires, or how staff spend their time. A DE&S report noted that in 2011 of the 300 accountancy posts it required, 25% were filled by unqualified staff and a further 9% vacant, despite having 400 qualified accountants.22 In late 2013 DE&S employed a specialist private sector company to do a ‘zero-baseline review’ of its organisational structure. It found a structure that was not transparent, with inefficient allocation of skills to tasks and unclear accountabilities among staff.

2.4 DE&S reduced permanent staff numbers by nearly half between 2007 and 2014 to try to reduce operating costs. Potential savings, however, have been offset by employing contractors who, on average, cost between three and four times more than permanent DE&S staff. The overall costs of DE&S’s permanent staff and contractors fell by £9 million between 2008-09 (£1,290 million) and 2013-14 (£1,281 million) (Figure 3). In 2013-14 we estimate that contractors’ costs comprised 37% (£481 million) of overall operating costs.23

2.5 Employing contractors can be beneficial, allowing more flexible working practices as well as providing essential skills that are not available in-house. Contractors have been employed to address gaps in specialist roles, such as technical and engineering.24 However, DE&S lacks robust information on how project teams are using contractors, and the cost of doing so. DE&S has also had to reduce permanent staff numbers to meet Departmental efficiency savings. These two points prevented management from getting a strategic view, and risks achieving value for money from how it gets skills from the private sector.

2.6 To address continued skills gaps at DE&S, the Materiel Strategy set out a need for greater freedoms, including:

paying market rates to compete with industry. For some specific skills, DE&S assessed it was offering salaries between one-fifth and three-quarters less than industry;

incentivising people to improve performance;

allocating staff to tasks where their skills are most required; and

better matching the workforce to suit the tasks demanded of them.

2.7 Since May 2014, DE&S has gained more freedom from a number of standard civil service rules to offer greater financial rewards to attract and retain certain skills. In addition, they no longer have to meet the headcount reduction target. DE&S now has a target operating cost of £1.3 billion for 2014-15, reducing to £1.2 billion by 2016-17.

2.8 DE&S has contracted industry to provide a project and programme service provider role for £215 million over 3.5 years. DE&S has agreed contracts with the managed service providers so part of their fees are, in principle, linked to milestones and a set of metrics. However, DE&S has not defined these metrics and agreed a baseline to track progress in carrying out reforms and transferring skills and capabilities to DE&S staff. It has not yet decided whether different pay scales will increase staff costs without a net increase in capability.

Improving the systems and tools

2.9 DE&S faces several challenges in reforming its systems and processes, including:

relatively weak business capabilities and business management;

poor financial information;

lack of integration between different information systems; and

poor contract and inventory management.

2.10 A 2011 independent assessment of DE&S’s business capabilities in five categories scored DE&S in the lower quartile against 160 capital-intensive projects in government and international private sector organisations (Figure 4).25 There is a difference between how the review assessed DE&S’s skills and DE&S’s self-assessment. The former was based on the reviewer’s established methodology, the latter on a DE&S staff survey.26

2.11 Poor integrated systems hinder effective scrutiny of equipment project costs.27 DE&S uses its management information system to monitor project performance, including costs. However, it does not reconcile these costs with the same projects reported on the Department-wide planning budget and forecasting system used for in-year budget monitoring and Departmental scrutiny of its financial position. The poor quality of the business information the Department had, which consortia were required to base their bids on, was partly why one of two consortia withdrew from the bidding process in November 2013.28

2.12 DE&S also lacks systems and expertise on most equipment projects to link industry payments to progress, even though this technique, known as earned value management, has been mandated since the 1990s. Similar techniques have been used on Crossrail.29 Lack of such a system reduces critical financial information, such as whether projects are on budget and managing in-year budgets. DE&S has also struggled to quantify why there have been underspends in the Equipment Plan.30 There may be several reasons for underspends, such as delays or delivering equipment more cheaply, but a lack of information makes it more challenging to respond effectively. Our review of a sample of major defence projects found that most underspend had not come from efficiencies, but from poor project team or contractor performance.31

2.13 DE&S is now implementing a forecasting improvement programme to address poor financial management in project teams.32 It has introduced several other changes to improve internal processes, including:

A new board-level quarterly review of performance, finance and risk has been instigated to increase scrutiny. The resources of DE&S’s Cost Assurance and Analysis Service, responsible for providing semi-independent advice on project costings has also been bolstered. DE&S has enhanced its finance function by appointing a director-general of resources from industry and four financial controllers working with the chiefs of materiel.

DE&S has sought to enhance its commercial expertise. A director of commercial operations was appointed from industry, and there is a greater focus on contract management and establishing strategic engagement with key suppliers. DE&S has set up a graduate recruitment scheme for commercial skills, one of the first government organisations to do so.

DE&S has enhanced its inventory management by setting up a dedicated inventory management team, commissioned a specialist private sector company to review holdings, and put in place a new inventory management system.33 DE&S is currently running a competition to contract out parts of its inventory management system to improve value for money.34

DE&S let a £43 million, three-year contract in November 2014 with a managed service provider to identify an optimum organisational structure for DE&S and support its implementation by creating an in-house human resources function.

2.14 DE&S has postponed securing a managed service provider to deliver enhanced financial controls and a single source of accurate and timely management information until summer 2015. It has postponed this part of reform so it can better understand the requirement as it develops support in other areas.

Improving the interface between DE&S and the commands

2.15 The Department found unclear responsibilities and accountabilities between the commands, who specify equipment requirements and hold budgets, and DE&S as ‘deliverer’ of these requirements, was a root cause of poor performance in DE&S. Individual projects gained approval for significant investment without full account of their impact on the Department’s overall affordability position. As such, the Department believed that the future organisational design for DE&S must include:

increased clarity around responsibilities and accountabilities, and a separation of duties between DE&S and the commands, plus stronger governance arrangements;

hard-charging the commands for DE&S’s services to ensure workload is defined; and

setting a baseline to measure performance against, and to ensure change control process for new work is better regulated.

2.16 Responsibilities and accountabilities throughout defence have been formalised. Much of the improvements to the interface between the commands and DE&S has come through the new defence delegated operating model (paragraph 1.9), as well as a requirement from the Materiel Strategy team to better define accountabilities in preparation for implementing a GoCo. In May 2014 the Department published a framework document reaffirming DE&S’s purpose and role, with the governance, policy, financial and human resources parameters that guide its day-to-day work.

2.17 Organisational changes from the new operating model have streamlined DE&S and focused it on delivery of the Equipment Plan. From April 2013, 120 capability planning staff transferred to the commands. In April 2014, 1,200 staff responsible for managing UK naval bases transferred to the Royal Navy and 2,000 Information Systems and Services’ staff transferred to Joint Forces Command.

2.18 To improve accountability, head office no longer manages the Equipment Plan budgets. In April 2014 budgeting responsibilities moved to the commands, who are now responsible for setting budgets and also holding DE&S to account for delivery.

2.19 To make greater efficiencies from DE&S’s £1.3 billion annual operating costs, the Department expects to implement a ‘hard-charging regime’ in a limited form from April 2015. DE&S will recover its operating costs from the commands, who hold the budget, rather than DE&S receiving budgets from head office.

2.20 The commands are developing and agreeing with DE&S quasi-contracts specifying the equipment and support DE&S will provide and the resources the commands will provide.35 Although these contracts are not yet in force, the Department plans to further develop them and expects these will influence its corporate key performance indicators from 2015-16.

2.21 DE&S now has greater autonomy as a departmental arms-length body. A DE&S board has been established, including a non-executive chair and members to advise and constructively challenge the chief executive, supported by an executive board (Figure 5).36 The chief executive is responsible to Parliament for stewardship of DE&S’s resources as an accounting officer.

Measuring DE&S’s performance

2.22 Under the new delegated model, the commands and head office will require performance measures to gain confidence in DE&S’s ability to deliver the Equipment Plan. The focus of measuring DE&S’s performance has previously been through equipment project performance.

2.23 The Department recognised that existing DE&S metrics were insufficient to assess DE&S’s overall performance. Under the bespoke trading entity, a new performance measurement regime for DE&S has been established.

2.24 DE&S has published its Corporate Plan 2014–2017, setting out its priorities and high-level objectives over the next three years (Figure 6).37

2.25 Underpinning these objectives is a revised set of key performance targets. The Department will hold DE&S to account for its performance against these targets through new governance structures (paragraph 2.21). The 2014-15 targets are largely a continuation of previous internal indicators to allow comparison with prior years, with some extra metrics:38

equipment support performance;

operating cost-efficiency;forecast accuracy;

customer satisfaction;39 and

delivery of a transformation plan and contracting for managed service providers.

2.26 To assess whether the bespoke trading entity structure is improving DE&S’s performance, the Department must set more detailed measures. They must track benefits to show whether the bespoke trading entity can reform acquisition at DE&S, and for the Department to take extra measures to improve performance if necessary.

20 Review of Acquisition for the Secretary of State for Defence (the Gray Review) 2009.

21 Materiel Strategy Outline Business Case, January 2013. Financial savings include both current costs saved and future costs avoided. The remaining savings would come from addressing the overheated equipment programme.

22 DE&S, Way forward on DE&S Organisation Design: The Materiel Strategy supporting report, January 2012.

23 DE&S estimates that if it had recruited all posts in-house the cost would have been between £111 million and £148 million.

24 Chief of Defence Materiel evidence to the Defence Select Committee, 12 December 2013, Q22.

25 This work built on previous assessments of DE&S’s capabilities and skills. For example, the Department commissioned a review in 2010 which benchmarked DE&S’s project and programme management performance against similar companies. This review, although limited in scope, found DE&S was relatively strong compared with peer organisations.

26 Categories were: engineering, finance, project management, supplier management and talent management. Evidence taken before the Defence Committee of the House of Commons, 4 September 2013. Available at: www.publications.parliament.uk/pa/cm201314/cmselect/cmdfence/652/c65201.htm

27 The corporate management information system only reports on new projects, not equipment in-service (around half of the Equipment Plan), with this being monitored via offline spreadsheets.

28 The third bidder had previously withdrawn from the GoCo competition as it was involved in another Departmental transformation programme and had insufficient resources to bid on both.

29 Comptroller and Auditor General, Department for Transport: Crossrail, Session 2013-14, HC 965, National Audit Office, January 2014.

30 The equipment programme underspent by £850 million in 2011-12 and £1.355 billion in 2012-13. To avoid a 2013-14 underspend the Department used over-programming by planning work above the budgeted level to compensate for predicted underspending in-year. When the Department became concerned that an underspend against the Defence budget would emerge in-year, a further £213 million of additional work was added to the equipment programme. Taken together, this resulted in an £185 million overspend against the equipment programme. Against the combined work programme (including the additional programming added in-year), the Department underspent by £948 million in 2013-14.

31 Comptroller and Auditor General, Major Projects Report 2014 and the Equipment Plan 2014 to 2024, Session 2014-15, HC 941-I, National Audit Office, January 2015, paragraph 2.12.

32 Comptroller and Auditor General, Major Projects Report 2014 and the Equipment Plan 2014 to 2024, Session 2014-15, HC 941-I, National Audit Office, January 2015, paragraph 2.13.

33 DE&S estimates that it has reduced inventory purchases by £1 billion between 2011-12 and 2013-14, and disposed of more than £10 billion of inventory during this period.

34 DE&S’s logistics and commodity service.

35 Command acquisition and support plans (CASPs).

36 The executive team leads day-to-day management of DE&S. It is made up of the chief of defence materiel as chief executive of DE&S, the director general resources as chief finance officer, the chiefs of materiel for Fleet, Land, Air and Joint Enablers, the human resources director, the commercial director and the exports director.

37 Defence Equipment and Support, Corporate Plan 2014–2017, May 2014.

38 Procurement projects (performance), procurement projects (cost), procurement project (time), and inventory management.

39 This indicator is a work in progress.

Appendix One

Our audit approach

1 This report examined the Ministry of Defence’s (the Department’s) progress in reforming Defence Equipment and Support (DE&S) to tackle the known systemic weaknesses in defence acquisition. We assessed:

how the Department managed its key programme to improve DE&S performance − the Materiel Strategy − up to the point where it halted its preferred option of contracting for DE&S as a government-owned, contractor-operated organisation (GoCo) in December 2013;

the Department’s latest plan for reform through changing DE&S into a bespoke trading entity; and

the main incremental improvements DE&S has made alongside pursuing the Materiel Strategy.

2 We applied an analytical framework to assess value for money. The framework considers whether reforms the Department said it requires at DE&S are being secured in a timely and efficient manner. During our review, implementation of the entity was at an early stage, and we did not draw a value-for-money conclusion here. However, we examined the Department’s plans to change DE&S into a bespoke trading entity and the challenges in using the entity to improve DE&S’s performance.

3 Our framework acknowledges expressed or implied restrictions or constraints. One constraint is DE&S’s poor management information, including full operating cost data. Another is the lack of documentation supporting some key decisions taken while developing the Materiel Strategy.

4 Our audit approach is summarised in Figure 7 and our evidence base is outlined in Appendix Two.

Appendix Two

Our evidence base

1 We reached our independent conclusions on the Ministry of Defence’s (the Department’s) progress in reforming defence acquisition at Defence Equipment and Support (DE&S) after analysing evidence collected between February and December 2014. Our audit approach is outlined in Appendix One.

2 We assessed the Department’s approach to addressing systemic weaknesses in defence acquisition through the Materiel Strategy.

3 We reviewed reports commissioned by the Department, and those of the House of Commons’ Committee of Public Accounts and Defence Committee, setting out the context for defence acquisition reform.

4 We conducted semi-structured interviews with the senior officials responsible for developing the Materiel Strategy, including private sector companies in support. We discussed how the evidence underpinning the Strategy was developed, including options appraisals for organisational change, cost−benefit analyses, and how the Department worked with the private sector to test its plans.

5 Where available, we reviewed official documents showing how the Department took key decisions and the information used to inform these decisions, including cross-government and ministerial correspondence. We also reviewed reports that the Department used to track its progress.

6 We reviewed official documents that set out the results of internal scrutiny of the Department’s management of the Materiel Strategy, including internal audit, the Cabinet Office’s major project review group and Departmental internal scrutiny reports.

7 To examine how robust and complete the baseline information for government-owned, contractor-operated organisation (GoCo) bidders was, we reviewed official documentation shared with the bidders.

8 We conducted semi-structured interviews to gather stakeholders’ views from outside the Materiel Strategy, including the commands, Cabinet Office and HM Treasury, on the Department’s engagement throughout the process.

9 We conducted semi-structured interviews with 10 of the 12 companies that formed into three consortia to bid to operate a GoCo, to gather perspectives on how the Department managed the competition. While we did not meet with the remaining two companies, we believe we have an adequate insight into the consortia’s views. We also met two of the companies involved in the early stages of the Department’s GoCo preparations, who ultimately did not take part in the GoCo bidding process, to understand their views on the GoCo proposition.

10 We reviewed official documents that summarised the Department’s analysis of the 1,200-plus page GoCo bid received. We reviewed the in-house benchmark and back-up option (DE&S+) against which the Department had planned to compare a winning GoCo bid.

11 We conducted semi-structured interviews with senior DE&S officials responsible for change within DE&S to understand the incremental changes being pursued alongside the Materiel Strategy. We also met with the specialist private sector company leading on the work to identify savings from equipment support.

12 We conducted semi-structured interviews with senior Departmental officials responsible for overseeing DE&S and financial management within the Department.

13 We reviewed and analysed equipment project performance data to explore how DE&S’s performance has changed.

14 Where available, we analysed DE&S’s expenditure data to find out DE&S’s operating costs since it was formed in 2007, including spend on consultancy support.

15 We examined the Department’s rationale for changing DE&S into a bespoke trading entity and how it planned and managed processes to allow the entity to reach full operating capability.

16 We reviewed the limited official documents available, such as key planning materials, DE&S’s Framework Document40 and Corporate Plan41 for the bespoke trading entity, to explore how the Department decided on a bespoke trading entity structure.

17 We reviewed official documentation available at the time of our review of the process to contract with managed service providers, including a statement of requirements and procurement strategy.

18 We supplemented this review and analysis with semi-structured interviews with senior officials who implemented the bespoke trading entity and wider change programmes across the Department.

40 Defence Equipment and Support, Framework document, May 2014.

41 Defence Equipment and Support, Corporate Plan 2014—2017, May 2014.

Appendix Three

Implementing a GoCo

1 In 2011 the Ministry of Defence (the Department) launched the Materiel Strategy to reform defence acquisition. It consisted of several business models for running Defence Equipment and Support (DE&S). Ultimately, the Department chose to contract out the running of DE&S to the private sector, with the Department retaining strategic direction and ownership of DE&S’s assets. This is known as a government-owned, contractor-operated organisation (GoCo) (Figure 8).

2 The Department expected the GoCo to address long-standing weaknesses in DE&S by:

giving DE&S greater freedoms to introduce private sector practices, skills and tools;

formalising and introducing rigour in dealings between DE&S, the Armed Forces (the commands) and head office; and

improving DE&S’s efficiency, as the private sector would be incentivised to manage the organisation and deliver the Equipment Plan for less.

3 Contracting with the private sector, however, put extra responsibilities on the Department. It had to act as an intelligent customer of the GoCo, and the deal would have been complex to secure.

4 Lower-risk options, including an incremental approach to improvement used elsewhere in the Department, were dismissed. The Department’s internal scrutiny team, as part of their routine assessment of investment business cases, advised that the Department had pre-judged its preference for a GoCo and there were major weaknesses in the evidence supporting the proposal’s benefits.

5 The timeline setting out the key events during the Department’s implementation of a GoCo is outlined in Figure 9.

6 In April 2013 the Department launched a competition to secure a proposal to operate DE&S as a GoCo. Three consortia met the Department’s requirement and the Department invited them to participate in the competition. In August 2013 a consortia withdrew from the competition because a constituent member lacked resources to bid. This company was already participating in another Department transformation programme. In November 2013 a further consortia withdrew from the competition, and in December 2013 the GoCo was halted due to a lack of competition, as only one detailed bid was received from industry. This was partly because:

poor governance arrangements exacerbated the different expectations that had emerged within the Department and DE&S on the timing and nature of financial savings from the proposition; and

there remained a lack of clarity on the level of risk being transferred to the private sector.

Governance arrangements

7 Poor governance arrangements had allowed different expectations of the GoCo proposition to develop within the Department. Unusually, the key oversight board for the Materiel Strategy was also chaired by the senior responsible owner for the Materiel Strategy and reported directly to the Defence board, rather than through the Department’s accounting officer.42 This complicated how head office oversaw the Materiel Strategy.

8 Some key decisions were not clearly documented or evidenced while developing the Materiel Strategy. Although the acquisition architecture board met fortnightly from March 2013, no minutes were taken until after the GoCo competition was stopped.

9 The lack of clear formal evidence hinders effective scrutiny of the process followed. However, because of the different expectations that had emerged around what was being asked of industry and in the GoCo’s purpose, the Department had to change the proposition significantly three weeks before being issued to the market. This prioritised greater cost savings, rather than focusing on improving DE&S’s performance.

10 The Department chose to pursue the most challenging timetable that was potentially achievable, which broadly equated to a 10% confidence of achieving, although it recognised this timetable was likely to slip. The late changes to the GoCo proposition made achieving this timetable even less likely. Although the Department’s internal scrutiny team raised serious concerns about the proposition, it was still issued to the market, rather than delaying the timetable.

Reaching agreement on the level of risk being transferred to the private sector

11 The late change to the GoCo deal also placed greater risk on industry as the chosen consortium to operate the GoCo needed to earn its fee from savings generated from the Equipment Plan and DE&S operating costs. Before the change, the consortia expected to earn a base fee and incentive fee without delivering any cash savings. Consortia had limited opportunity to verify that the Equipment Plan costs that the Department had forecast were appropriate. This made it extremely challenging for any consortia to calculate its likely profit from operating DE&S as a GoCo.

12 The business information the Department had, which consortia were required to base their bids on, was poor. This was partly why one of the two remaining consortia withdrew from the bidding process in November 2013.43

13 Additionally, there were differences in views between the Department and the consortium that withdrew in November 2013 over whether their concerns regarding the deal were sufficiently escalated prior to their withdrawal.

Developing an alternative approach to reform

14 In the event that the Department could not reach agreement on the transfer of risk, the Department planned to implement an in-house proposal, known as DE&S Plus (DE&S+). It would decide how to improve DE&S within the public sector should the GoCo proposal not proceed. Moving to a DE&S+ organisation would present far lower risk to the Department. However, it was unlikely to maximise the commercial benefits compared with the GoCo option.

15 When the GoCo competition was halted in December 2013, the DE&S+ proposition was not implemented. Although scrutiny had raised concerns up to October 2013 that DE&S+ was insufficiently resourced, it remains unclear why the Department did not complete development of DE&S+ and implement it as the anticipated fall-back option. Instead, the Department built on previous work undertaken for the Materiel Strategy and developed a third option, the bespoke trading entity.

16 At the point the GoCo competition was halted, the Department had spent £33 million on the Materiel Strategy, including £25 million on contractors and advisers.

42 Acquisition architecture board.

43 The third bidder had previously withdrawn from the GoCo competition as it was involved in another Departmental transformation programme and had insufficient resources to bid on both.

Design and Production by NAO Communications
DP Ref: 10597-001