Report

by the Comptroller
and Auditor General

Department for Environment, Food & Rural Affairs

Early review of the Common Agricultural Policy Delivery Programme

HC 606 Session 2015-16 1 December 2015

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO, which employs some 810 people. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund have used their resources efficiently, effectively, and with economy. Our studies evaluate the value for money of public spending, nationally and locally. Our recommendations and reports on good practice help government improve public services, and our work led to audited savings of £1.15 billion in 2014.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 30 November 2015

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

26 November 2015

The report examines whether the Department for Environment, Food & Rural Affairs has managed the Common Agricultural Policy Delivery Programme well so far and, in doing so whether it delivered value for money.

© National Audit Office 2015

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.gsi.gov.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

10835 12/15 NAO

The National Audit Office study team consisted of:
Michelle Bull, Richard Davis, Ravinder Pahal, Rebecca Sidell and Freddie Wong with support from colleagues from the Digital Team, under the direction of Simon Helps.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key facts

Summary

1 Since 2012, the Department for Environment, Food & Rural Affairs (the Department) has been leading the CAP Delivery Programme (the Programme) involving its main delivery bodies, the Rural Payments Agency (RPA), Natural England and the Forestry Commission, and with support from the Government Digital Service (GDS) in the Cabinet Office, to deliver the new Common Agricultural Policy (CAP). The CAP is the European Union (EU) framework of subsidies and rural development programmes. The Programme was originally forecast to cost £155 million and expected to bring benefits of £274 million, supporting payments to farmers from 2015.

Structure of the report

2 Our report sets out the key events since the Programme began and how these have affected the development of the Programme and its likelihood of success. The report covers:

the background and rationale for the Programme (Part One);

the design and vision for the Programme (Part Two);

system development (Part Three); and

the revised plan, introduced in March 2015 (Part Four).

The Common Agricultural Policy

3 In England, the Department has overall responsibility for the CAP. It pays out £1.8 billion each year to English farmers and landowners under two funding ‘pillars’:

Pillar 1 primarily provides direct support to farmers through the Basic Payment Scheme (BPS); and

Pillar 2 provides funding for rural development programmes, such as the Environmental Stewardship Scheme and the Countryside Stewardship Scheme that replaces it.

4 The EU reforms the CAP every seven years or so. The current regulations came into force in 2014 and are expected to be in place until 2020. They require more complex schemes to be implemented in 2015. The Department aimed to have a new service in operation in time to accept BPS applications by the EU deadline of 15 May 2015, and begin making BPS payments to farmers in December 2015. Some farmers continue to receive payments for Pillar 2 schemes from the previous CAP regime and payments for these legacy schemes began in October 2015.

5 The reform of the CAP took many years. Policy, primarily at the European level, was finalised late in the process. This led to delays in finalising detailed control requirements in England, in some cases until after implementation had started, adding to the complexity of the Programme.

The case for change

6 Systems developed to support the previous CAP regime in 2005 were beset by problems.1 Under the previous delivery arrangements, farmers had to apply separately for different elements of the CAP, creating a disjointed user experience. The systems used ageing software and were increasingly costly to maintain. Staff had to carry out many processes manually. This increased the risk of errors, which, in many cases, led to the European Commission (EC) imposing penalties, known as disallowance. During the previous CAP period (2005 to 2014) the Department incurred disallowance penalties of £642 million, primarily related to issues with the implementation of the system in 2005. Given the complexity of the new CAP, the Department decided that the existing systems could not be enhanced to support the new service.

7 The Programme was established to address these failings. It aimed to:

reduce delivery costs;

reduce exposure to disallowance penalties;

improve customer experience;

provide flexibility to accommodate last-minute decisions and rule changes by the EC; and

improve capacity to achieve environmental outcomes.

Programme delivery failures

8 The new service did not operate as originally intended at two key stages during its first year:

Registration

The Department expected applicants to start using Verify, the government’s identity assurance system, to register for the new service from October 2014. However it quickly became apparent that Verify was not sufficiently developed to assure the identity of a significant proportion of farmers, and did not therefore work as the Department expected. Although a small number of farmers were able to register through Verify, the majority of customers registered using the RPA’s existing customer registration process, supported by drop-in centres and RPA’s telephone helpline.

Applications

In March 2015, in response to serious failings of the system, the online application system was withdrawn and replaced by ‘paper-assisted digital’ applications for the 2015 schemes.

9 As a result of the withdrawal of online applications, the RPA had to enter many application forms and land data changes manually. Manual data entry and the late development of interfaces with old systems that continue to be used, along with a one-month extension granted by the EU to the BPS application deadline, impacted the RPA’s ability to make interim payments for the Environmental Stewardship Scheme. As a result, payments totalling £200 million that otherwise could have been made from August 2015 were made from October 2015.

10 By March 2015, when the online application system was withdrawn, there was limited functionality within other components of the system. They were not fully integrated; development of links to legacy systems required to make payments had ceased; accounting and payment systems had been deprioritised and delayed; and there were other significant technical issues and problems with land data. A comprehensive recovery plan was therefore created to develop the infrastructure required, mitigate the significantly increased risk of disallowance penalties and minimise delays of payments to farmers. The budget contingency had been used up and the forecast cost of the Programme at this stage was £183 million.

11 The Department submitted a revised business case to HM Treasury in September 2015. By this point, the cost had risen to £215 million, 40% higher than originally planned, despite reductions in the planned level of functionality. This reflects cost overruns leading up to the withdrawal of the applications system in March 2015, and subsequent recovery actions.

Key findings

Strategic design

12 The original vision and design of the Programme was narrow. It focused on procuring IT systems and did not set out the wider organisational transformation required. There was no clear vision at the outset of the desired state of business operations nor a route map showing how the organisations involved should move towards it. In the outline business case, the Department did not clearly set out how it would achieve the expected savings from changes to the delivery bodies’ business processes (paragraphs 2.2 to 2.3).

13 There are differences in strategic direction and vision between the Cabinet Office, the Department, the RPA and other delivery bodies. The different priorities of the organisations involved were not resolved and caused disruption and delay at the beginning of the Programme. The RPA’s priority was to pay farmers accurately and on time, and minimise the risk of disallowance penalties, while the focus for Natural England and the Forestry Commission was to achieve environmental outcomes and build relationships with their customers. The Cabinet Office aimed to encourage innovation, reduce costs and develop learning across government as part of the government’s digital strategy to build digital services based on user needs (paragraph 2.4).

14 Many fundamental changes were made to the Programme, significantly increasing the level of innovation and risk. The Cabinet Office applied its ICT spending controls to the Programme and proposed changes that would bring it into line with the government’s digital strategy. Although the Department did not have previous experience of many of the changes, and they were relatively new across government, the Department did not consider that they increased the overall risk of failure for the Programme, but did identify risk mitigation plans for individual Programme elements. It is not clear that the changes were all appropriate or that the Department and Cabinet Office fully understood the implications of implementing them all at once. For example, the Department took on the systems integration role without having the necessary skills or knowing that they would be able to obtain them (paragraphs 2.5 to 2.9 and 3.28 to 3.29).

15 The Department did not give timely consideration to how it would reduce the risk of disallowance penalties. The outline business case did not provide sufficient detail about the specific steps it would take to reduce the risk of disallowance penalties, or which elements of disallowance are within the Programme’s control. In January 2014, it became clear that the EC intended to apply much higher rates of disallowance in some cases, but the Department did not begin developing its detailed response to the increased risk until later in the year when further details were released by the EC. As a result of the EC’s higher rates and insufficient mitigation, the Department expects disallowance penalties to exceed its 2% target by some margin (paragraphs 2.10 to 2.15).

System development

16 The Programme has been set back by numerous changes in direction, senior responsible owner and governance. Since the start of the Programme, there have been four different senior responsible owners (SROs), each from a different background and each bringing their own style, vision and priorities. Repeated changes to governance arrangements have disrupted the Programme and caused uncertainty and confusion for staff. SROs have not been formally trained or enrolled in the Major Projects Authority’s Leadership Academy and not all had experience of delivering government major projects of this scale and complexity (paragraphs 3.3 to 3.5).

17 The Department failed to prevent counterproductive behaviours by senior leaders. There were deep rifts between Programme leaders at many stages of the Programme’s history. These went beyond the healthy tension that could be expected in a multi-organisational programme, not only hindering progress but also impacting on staff morale and stress. Despite attempts to resolve these behaviours, the Department was not able to stop them (paragraphs 3.10 to 3.12).

18 Inconsistent and incomplete management information and assurance, alongside a good news culture, prevented effective progress monitoring and risk management. Management information has at times been patchy and inconsistent. Producing and sharing good quality, complete and meaningful management information was not always given sufficient priority. The Department did not consistently and transparently acknowledge and respond to findings from external assurance. The pervasive good news culture meant that early warnings of possible failure were unvoiced or not adequately addressed, nor appropriate contingency plans developed. This began to improve at the beginning of 2015, but the consistency and continuity of management information continues to be an issue in progress updates produced for the Secretary of State. These reports have recently been supplemented by a weekly dashboard that provides a factual view of achievements against an expected profile (paragraphs 3.14 to 3.22).

19 GDS did not provide the support the Department needed. GDS committed to reducing overall costs, improving delivery confidence and building the Department’s digital capability to support approval of the business case and the adoption of new approaches unfamiliar to the Department. Through the Programme:

formal estimates of cost reductions were not provided;

the Major Projects Authority’s delivery confidence assessments did not improve; and

although GDS provided resources to the Department, its support was reported to be patchy. GDS provided limited continuity and insufficient insight into how to adopt agile on this scale. It was not able to identify and provide the systems integration skills required (paragraphs 3.23 to 3.25).

Damage limitation

20 In March 2015, in response to serious failings of the system, the requirement for all customers to apply online was withdrawn and replaced by ‘paper-assisted digital’ applications for the 2015 scheme. The Department acknowledged that it would no longer be possible to achieve a fully online approach in 2015. To ensure that farmers could apply by the deadline, an alternative approach involving paper-based applications for some customers was needed, using paper forms that were entered manually into the claims processing system, which connected with the other planned supporting systems. This was a speedy and effective change to the Programme that has increased the likelihood that the majority of farmers will receive their BPS payments in December (paragraphs 4.2 to 4.7).

21 The focus on resolving immediate issues has diverted attention from longer-term goals of improving the service to farmers, minimising disallowance penalties and achieving other intended benefits. The Programme is intended to support the delivery of CAP for ten years starting in 2015. Efforts to enable and handle paper-based applications have necessarily changed short-term priorities for the Programme diverting resources from the longer-term priorities of achieving efficiencies and addressing the land data issues that are causing current disallowance penalties (paragraph 4.12).

Conclusion on value for money

22 The Programme is a combined effort between the Department, the RPA, other delivery bodies and GDS to develop new systems and processes to support the implementation of the new CAP in England. But ineffective collaboration between these bodies undermined their ability to deliver a successful rural payments service. The Department and the Cabinet Office did not ensure a clear and consistent vision for the Programme with a manageable level of innovation. Nor did they effectively manage competing priorities. The result is that the Department expects higher levels of disallowance penalties, increased Programme costs, poorer customer experience and difficulties paying farmers accurately at the earliest opportunity. The Programme has therefore not provided value for money at this early stage.

23 The Department took action in March 2015 to revert to paper-based applications for some customers and ensure the infrastructure was in place to support this. Implementing these recovery actions and containing the costs of the revised approach is a significant challenge, and the Department is making good progress to meeting its target of paying BPS claims for the majority of farmers in December. However, significant challenges remain to make sure this year’s payments are accurate, to prepare for future years, maximise Programme benefits and minimise disallowance penalties.

Recommendations

a The Department must ensure it focuses on longer-term priorities to continue to support efficiencies and minimise disallowance penalties in 2016 and beyond. The pressure to resolve immediate issues and ensure farmers are paid on time has diverted resources away from longer-term objectives. The Department should provide greater clarity about the Programme’s benefits, in particular the desired state of the delivery bodies, the benefits the Programme will bring and how their realisation will be monitored.

b The Department and the RPA should ensure that payments made are accurate. Manual data entry generated a high proportion of errors. The Department is implementing controls to mitigate these errors, but should ensure end-to-end checks and associated assurance is in place and comprehensive. It should quantify the risk of inaccuracies arising from gaps in mitigating controls, and must not allow the target to pay in December and January to lead to payment inaccuracies and significant disallowance penalties.

c The Department should maintain a strong and consistent leadership for the Programme. The frequent changes to leadership have led to shifts in focus and governance arrangements. To deliver the longer-term benefits and ensure everyone across the Programme is working towards the same goals, the Department must develop a clear vision that withstands any future change of SRO. The Department should also ensure that SROs have appropriate experience and training relevant to the role.

d The Department should review how it sought to address personal rifts in the Programme. It should identify relevant lessons to improve future performance and ensure that such behaviours can be addressed and seen to be addressed.

e The Department must ensure it has meaningful and consistent management information and that clear milestones allow progress to be tracked. The leadership has at times been unable to adequately manage the Programme or measure progress due to the poor quality and inconsistency of management information and an absence of clear milestones. The Department must guard against a good news culture by encouraging honest discussion, ensuring reporting is balanced and that concerns can be raised openly and risks more easily identified.

f The Department must establish a comprehensive understanding of the full costs of delivering the CAP. The Department does not know the full cost of the Programme across all the organisations involved. Its current estimates do not include all immediate costs of recovering the Programme, nor longer-term costs of disallowance penalties and ongoing maintenance and support.

g The Department must provide a range of application methods that suit the diverse needs and abilities of its customers. The drive to enable online registrations and applications for all those who wanted it led to a failure to accommodate the different levels of digital literacy of the Programme’s customers and the differing levels of complexity in their claims. The Department must now ensure that it delivers the functionality to make multiple channels available to meet the needs of all of the different customer groups.

h The centre of government should learn from what has gone wrong, and for future programmes develop a more flexible approach to dealing with departments implementing them. The changes to the Programme created gaps in skills and experience within the Programme that neither the Department nor GDS could resolve and therefore added to the risk that the Programme would fail to deliver its aims. In response, the Cabinet Office should:

provide stronger written guidance and capability building for departments on agile management and governance for major projects and how it fits with traditional governance structures;

take into account departments’ capability and capacity to apply new methods and ensure sufficient flexibility and support in their adoption;

adopt an approach that is more sensitive to the risks and culture of the organisations and teams involved, in order to ensure that the best use is made of GDS’s limited resources and that cost savings are delivered;

support departments in acquiring the management and technical skills required to apply agile at scale;

ensure it has appropriate accountability for programme delivery when significant changes to programmes are enforced; and

step in to resolve conflicts quickly when departments are not able to do so themselves.

1 Comptroller and Auditor General, Department for Environment, Food & Rural Affairs, and Rural Payments Agency, The delays in administering the 2005 Single Payment Scheme in England, Session 2005-06, HC 1631, National Audit Office, October 2006.

Part One

Background

1.1 Since 2012, the Department for Environment, Food & Rural Affairs (the Department) has been leading the CAP Delivery Programme (the Programme) involving its main delivery bodies, the Rural Payments Agency (RPA), Natural England and the Forestry Commission, and with assistance from the Government Digital Service (GDS) to improve delivery of the new Common Agricultural Policy (CAP). This part describes the CAP, the Programme and the two key failures that impacted on delivery.

The Common Agricultural Policy

1.2 The CAP is the European Union (EU) framework of subsidies and rural development programmes. It pays £1.8 billion each year to farmers and landowners in England under two funding ‘pillars’:

Pillar 1 primarily provides direct support to farmers through the Basic Payment Scheme (BPS); and

Pillar 2 provides funding for rural development programmes, such as the Environmental Stewardship Scheme and Countryside Stewardship Scheme that replaces it, aimed at achieving positive environmental outcomes and enhancing rural growth.

1.3 Figure 1 sets out the current responsibilities for delivering the CAP. In England, the CAP is delivered by the RPA, Natural England and the Forestry Commission with the Department retaining overall policy responsibility. The RPA’s principal role is to act as the paying agency for all CAP payments. It administers Pillar 1 while Pillar 2 is delegated by the RPA to Natural England, the Forestry Commission and local delivery groups.

1.4 The EU reforms the CAP every seven years or so. The current regulations came into force in 2014 and are expected to run until 2020. They required more complex schemes to be implemented by 2015.

1.5 CAP reform takes many years to finalise and, despite the UK government’s efforts at simplification, the current round of reform has resulted in the most complex CAP regime so far. Policy, primarily at European level, was finalised late in the process, delaying the finalisation of detailed control requirements in England, in some cases, until after implementation had started. To allow for last minute decisions and rule changes, significant flexibility needed to be built into the development of new schemes, systems and processes, adding to the complexity of the Programme.

The case for change

1.6 After considering a number of delivery options for the new CAP, the Department decided it was not feasible to adapt existing systems within the time available. This was because:

the delivery arrangements under the previous CAP had created multiple points of contact for customers, leading to a disjointed and poor user experience;

the systems comprised a number of ageing and unsupported IT components spread across the delivery bodies, which were becoming increasingly expensive to maintain;

there were difficulties in connecting information, decisions, data and outcomes through the user journey. This increased costs and the risk of fraud and disallowance penalties, because of errors arising from the difficulty of checking information across systems;

the new CAP schemes were more complex leading to higher anticipated administration costs; and

the new systems needed to be flexible to enable changes to policy, scheme rules and processes to be incorporated easily and quickly.

1.7 During the previous CAP period (2005 to 2014), the European Commission imposed penalties – known as disallowance – for failing to comply with the regulations to the satisfaction of EU auditors. These amounted to £642 million, around four times the original planned cost of the Programme. Disallowance penalties can arise, for example, as a result of delays in payments, member states misinterpreting the regulations, customer error or the Commission identifying control issues that it considers a risk to EU funds.2

The CAP Delivery Programme

1.8 In February 2012, the RPA published its five-year plan. A key part of the plan was the Future Options Programme, which intended to stabilise the organisation, procure replacement systems to meet its CAP obligations and to achieve value for money in implementing those systems. As a digital project involving significant spending, the Programme was subject to the Cabinet Office ICT spending controls. As a result, the Cabinet Office proposed a number of fundamental changes and, in July 2012, the Programme became known as the CAP Delivery Programme, to be led by the Department.

1.9 The Programme aimed to address many of the problems associated with the previous CAP scheme by providing a single IT solution to process and deliver payments across both pillars of the reformed CAP. In doing so, it was expected to deliver a range of benefits, including:

reducing delivery costs through automating processes, providing more efficient, accurate and fit-for-purpose systems, and decommissioning expensive legacy systems;

delivery structures that were more adaptable to changes in policy or legislation;

reduced exposure to disallowance penalties through fewer manual controls and automated cross-validation checks;

a simple and joined-up approach for customers to get information and services from the Department and the other delivery bodies; and

better environmental outcomes.

1.10 The system was envisaged as separate but integrated components that together would deliver the new service. The main components and their suppliers are set out in Figure 2.

1.11 Procurement and development of the system took place during 2013 and 2014. The Programme was working towards immovable deadlines: the service had to be ready to accept BPS claims by 15 May 2015 and to make BPS payments to customers between 1 December 2015 and 30 June 2016. Failure to meet these deadlines would incur disallowance penalties. Figure 3 sets out the key events.

Programme delivery failures

1.12 The new service did not operate as originally intended at two key stages during its first year:

Registrations

The Department expected applicants to register using Verify, the government’s identity assurance system, from October 2014. Although GDS’s Verify team had advised the Department that they would need to make alternative means available to access the service, no alternative was initially put in place. In summer 2014, farmers and landowners trialling the new system began reporting problems in using Verify, which was still in the early stages of development. Verify was not sufficiently ready for widespread use by farmers when the Department and the SRO launched its use for all applicants in October 2014. In November 2014, although a small number of farmers had managed to register using Verify, the Department decided to allow applicants to use its existing customer registration process, supported by drop-in centres and the RPA’s telephone helpline. At this stage, the forecast cost of the Programme rose to £183 million.

Applications

In February 2015, performance issues with the new IT system that had been evident for some time grew more serious as the number of users on the system grew. Programme staff, GDS staff and suppliers all put significant effort and resources into resolving these issues but could not resolve them in the time available. In March 2015, a revised plan was implemented, involving sending partially pre-populated paper claim forms and maps to some customers for them to send back with any amendments. At the same time, responsibility for running the Programme passed from the Department to the RPA.

1.13 In September 2015, the Department submitted a revised business case to HM Treasury. By this point, the cost had risen to £215 million, 40% higher than originally planned, in part to cover the additional costs of the revised plan.

1.14 Because of the complexity of implementing the new scheme, the EU allowed a one-month extension of the BPS deadline. This, together with the increased manual entry required and the late development of interfaces with the systems in place for legacy schemes, delayed the application of required controls and checks, and consequently impacted the RPA’s ability to pay interim Environmental Stewardship Scheme payments. As a result, payments totalling £200 million that otherwise could have been made from August 2015 started in October 2015.3

2 Comptroller and Auditor General, Department for Environment, Food & Rural Affairs and the Rural Payments Agency, Managing Disallowance Risk, Session 2015-16, HC 306, National Audit Office, July 2015.

3 Farmers were informed in 2013 that these interim payments should only be expected by December 2015. This was due to the need to change grant periods to comply with EC rules, and was unrelated to the new CAP or the Programme.

Part Two

Strategic design

2.1 For a programme of this size and complexity, it is essential that there is a clear shared vision and a blueprint to manage priorities, establish a consistent basis for decisions about service design and assess progress against objectives. This should be able to withstand any changes to programme leadership. In this part we consider:

the vision for the Programme;

changes to the Programme required by the Cabinet Office; and

how the risk of disallowance was built into the Programme.

The Department’s vision

2.2 The Department’s vision for the Programme focused on procuring an IT system, rather than aspiring to wider organisational transformation to deliver further efficiencies. The business case considered different options for procuring the IT systems, but not for delivering the service more widely. In the summer of 2014, the then SRO observed that “the plans for business transformation lag behind the technology development rather than leading it which, regrettably, is often the case in government projects”.

2.3 The Department did not develop a suitable operating model until January 2015, nearly two years after the outline business case had been approved. For a programme of this complexity and ambition, a target operating model is a basic requirement, setting out the transition to a future position and linking services, customers, processes, technology, organisational design and governance arrangements. The target operating model developed in the early stages was a single-page reference document that only described the high-level design principles of the proposed IT solution. The Major Projects Authority (MPA) described the absence of a target operating model as a critical failing in December 2014.

Differences in strategic priorities

2.4 There were a number of differences in strategic vision and cultural and management style across organisations. The RPA’s priority was to pay farmers accurately and on time, while minimising the risk of disallowance penalties. The other main delivery bodies, Natural England and the Forestry Commission, have wider objectives of achieving environmental outcomes and building ongoing relationships with their customers. The Cabinet Office aimed to innovate and disseminate good practice and learning across government to promote the government’s digital strategy to build digital services based on user needs.

Cabinet Office conditions for funding approval

2.5 The Programme was ‘reset’ during 2013 as a result of the Cabinet Office ICT spending controls review. It was established as a ‘digital exemplar’ under the Cabinet Office Transformation Programme.

2.6 The Cabinet Office imposed several conditions on the Department, designed to encourage innovation, reduce costs and bring the Programme in line with the government’s digital strategy, as a prerequisite for funding approval. In addition the Department agreed to use the Verify programme for identity assurance. Each of the changes had potential value in improving digital development across government but, taken together, the seven key changes significantly increased delivery risk. Many were new for the Department but also relatively untested across government. We are not convinced that all of the changes were appropriate or deliverable for the service and its users, nor that the Department and the Cabinet Office fully understood the implications of taking them on board all at once. The Cabinet Office did not provide evidence that these changes would reduce costs or that the Programme would be deliverable.

2.7 Figure 4 summarises the changes made to the Programme, and why we view them as adding risk.

Additional risk of the reset

2.8 At the time of the reset, the Department did not believe that these changes added to the overall Programme risk but did identify mitigating actions in an attempt to manage the individual risks associated with each change. However, many staff were concerned that the cumulative risk of introducing seven fundamental changes was more than the Department could be expected to accommodate. The RPA Chief Executive told the Environment, Food & Rural Affairs Select Committee in September 2015, that there was too much focus on “being first on a number of key development areas, so compounding the difficulties of being a digital exemplar for Digital by Default, being one of the first to use some of the new contracts on government’s G Cloud and being the first to try the identity verification service called Verify in a live environment”.4

2.9 The Department did not include additional allowance for optimism bias to mitigate the increased risk associated with the seven changes. The Department had identified some risks and put in place risk mitigation plans for individual Programme elements. However, we saw no evidence of an overarching contingency plan at the Programme level. This was raised as an issue by the Programme Sub-Committee in January 2013, but was not subsequently followed up. There appeared to be a widespread view across the Programme that contingency planning was an admission of a failure that could not be allowed to happen.

Disallowance risk

2.10 The scale of disallowance penalties incurred during the previous CAP and anticipated over the coming years far exceeds the total cost of the Programme. A key objective of the outline business case was “to minimise the exposure of the UK taxpayer to disallowance penalties”. It assumed that, if the Programme was delivered successfully, disallowance would be contained at 2% of total scheme value or £44 million each year.

2.11 The outline business case provides little detail on how the Programme would achieve this 2% level. It deliberately avoided including disallowance in the economic assessment due to the uncertain nature of the risk, and because the Department could not separate Programme-related from wider disallowance penalties. An external review that formed the first stage of the Department’s understanding of risks from the Programme impacting potential disallowance was not produced until March 2014. As a result, the Department did not have the required information to effectively prioritise efforts within the Programme to minimise disallowance. It has not focused enough on critical areas that impact on disallowance penalties, such as the development, maintenance and implementation of a single data architecture to ensure data quality.

2.12 A key element of the Department’s approach to minimising the risk of disallowance penalties was its choice of Abaco, an Italian IT supplier, to deliver the system’s ‘rules-based engine’. This applies the complicated scheme rules to calculate payments based on claim and mapping data. The Department selected Abaco, against GDS advice, because they had a successful track record in preventing disallowance in other countries, although this was recognised as presenting challenges for systems integration and meeting Digital by Default standards. The Department chose to use only some elements of the Abaco product and try to integrate them with other parts of its system. It was not able to successfully integrate Abaco with the online application portal for 2015 applications.

2.13 In January 2014, it became clear that the EC was changing how it calculated disallowance penalties, greatly increasing the value of penalties compared with those that would have been applied for similar failures under the previous regime.

2.14 The Department responded by starting to develop a CAP Disallowance Strategy from late 2014. In developing the strategy, the Department has taken steps to improve its understanding of the causes of disallowance and developed other measures, including improving its mapping capabilities. These are described in our recent report on disallowance.5 It has also submitted an invest-to-save proposal to reduce future disallowance for consideration within the 2015 Comprehensive Spending Review.

2.15 Although the full extent of the increase in potential penalties was not known until late 2014, the Department could have responded much earlier by reassessing the disallowance risk and potential Programme benefits at the point at which they were known to be likely. Enhancements to the Programme design to reduce disallowance were not fully developed until September 2015 when an addendum to the full business case was submitted to HM Treasury. Disallowance is now expected to exceed the 2% target by some margin.

4 Environment, Food & Rural Affairs Select Committee, Oral evidence: Common Agricultural Policy, HC 405, September 2015.

5 Comptroller and Auditor General, Department for Environment, Food & Rural Affairs and the Rural Payments Agency, Managing Disallowance Risk, Session 2015-16, HC 306, National Audit Office, July 2015.

Part Three

System development

3.1 Digital transformation programmes need strong and experienced leaders that understand how to achieve digital transformation, as well as established and effective governance mechanisms that hold management to account. Transparent and consistent performance monitoring arrangements, based on complete and accurate management information, are essential for keeping a programme on track and keeping the leadership informed and able to make effective decisions.

3.2 In this part we consider:

Programme leadership;

governance arrangements;

working relationships;

Programme assurance;

support from the Government Digital Service (GDS); and

the challenges of applying seven fundamental changes.

Frequent changes in Programme leadership

3.3 There has been a rapid turnover in the Programme’s senior leadership with four senior responsible owners (SROs) since the start of the Programme in 2012. Each SRO came from a different background (the Department, the Major Projects Authority, GDS and the RPA) and each new SRO made changes to the priorities of the Programme as well as to the governance arrangements (Figure 5). This created disruption for Programme staff and suppliers, as well as contributing to a lack of clarity across the Programme.

3.4 While demonstrating a range of leadership experience, not all had previous experience of leading complex government major projects on this scale. The Cabinet Office introduced a requirement in October 2012 that SROs should be enrolled in its Major Projects Leadership Academy (MPLA) by the end of 2014. The Programme Director, who has been in post throughout the Programme, was one of the first graduates of the MPLA but none of the SROs enrolled or had graduated.

Programme governance

3.5 At the start of the Programme, governance structures were cumbersome with over 30 different committees, in part reflecting the complexity of needing to align policy with delivery (Figure 6). The Programme was complex and made more so by the number of delivery bodies and suppliers involved. The changes in SRO led to further disruption to the arrangements, but governance was rationalised considerably in mid-2014 as the Programme moved towards implementation. These arrangements have largely remained in place.

3.6 Programme and departmental governance was not well aligned to the agile approach. The Department and its delivery bodies were accustomed to more traditional, structured governance arrangements with detailed planning and large staggered releases. We were told by key stakeholders that the Department’s decision to adopt an agile approach, with its more fluid iterative approach and greater freedom for decision-makers, brought new challenges. The Department told us it sought guidance in 2013 from GDS on best practice for agile governance, but guidance on this was not published until June 2014.

3.7 A Unified Plan for the Programme was only introduced in September 2014. This was designed to incorporate business and operational needs across the Programme and to focus on end-to-end integration and testing of the BPS by December 2014 and testing of Pillar 2 by April 2015. In November 2014, the Plan fell out of use with a move to phased releases. A number of key staff reported becoming unsighted on overall progress and delivery as a result.

Service Manager role

3.8 In October 2014, a Service Manager role was created, with autonomy to determine release timing and content, to address perceived deficiencies in obtaining and integrating user feedback. Following this appointment, the Programme Executive and Programme Director were not always clearly sighted on key decisions on the design and delivery of the Programme. There was not always a clear trail to show a detailed consideration of the financial impacts. The SRO is responsible for supporting the Programme Director but in our view, by appointing a Service Manager who at times operated independently, he undermined the Programme Director who retained financial and overall Programme delivery responsibility. From this point, some key staff and delivery bodies reported that they were not sufficiently involved in, or aware of, key decisions that they would have been party to under previous arrangements.

3.9 This situation persisted until January 2015 when, at the request of the Permanent Secretaries of the Department and the Cabinet Office, steps were taken by the Deputy SRO and Programme Director to restore financial controls to their previous position. The Unified Plan and financial controls were re-established and signed off by the Department’s Network Executive Committee in February 2015.6

Personal rifts at the top

3.10 There were deep and persistent personal rifts at senior levels and at times these led to counter-productive behaviour by the Programme’s leaders. The differences in strategic vision in the design phase (paragraph 2.4) were not resolved. Senior people told us that they found it almost impossible to work together at times. Interviewees reported confrontational behaviour between senior Programme staff at the RPA and GDS.

3.11 Rifts between senior Programme officials went beyond the creative tension that is to be expected in a multi-organisational programme, and impacted on implementation and delivery as well as staff morale and stress, especially from mid-2014. The dysfunction and inappropriate behaviour at the top was very apparent to Programme staff at this time, and created a frustrating working environment for them, preventing the culture of trust and collaboration needed to deliver a large and complex programme.

3.12 The Department made several attempts to tackle these behaviours but proved unable to do so. The Permanent Secretary became aware of the conflict at various stages, but the issue repeatedly resurfaced and was never properly resolved. GDS also escalated concerns around this issue within the Cabinet Office from February 2014, but with little impact.

Programme assurance

3.13 The Programme followed HM Treasury’s assurance model based around ‘three lines of defence’, but we found weaknesses in each of the components of assurance (Figure 7).

Management information and planning

3.14 We found that the Programme’s management information (MI) was patchy and inconsistent at times and was not always shared in the best way to support the effective running and oversight of the Programme. Challenges arose in the sharing of information between RPA and GDS staff within the Programme, and in one case the Secretary of State had to intervene for information to be released from RPA to GDS staff within the Programme.

3.15 Between November 2014 and February 2015, there was a break from the established MI reporting. Information that was previously collected and distributed by the Programme Management Office was no longer made available to the Programme Executive. The SRO wanted information at a faster pace to keep up with the challenges of developing the new service. He therefore asked for information directly from technical teams to feed into weekly update reports to the Permanent Secretary, Ministers (the Department and the Cabinet Office), Chief Executive of the RPA and Director Generals in the Department. These reports were shared with the SRO Board but not with Programme Executive members or the Programme Director, who continued to receive the more traditional, but by then incomplete, MI reports from the Programme Management Office. Programme Executive members raised specific concerns about this at board meetings in January and February 2015.

3.16 Suppliers and delivery bodies told us that the lack of a detailed plan agreed by all parties hampered the ability to plan, resource and identify critical paths and dependencies. The Unified Plan was introduced in September 2014 (paragraph 3.7), alongside an end-to-end service map, but there was no reporting against its milestones until February 2015.

3.17 MI needed to be consolidated. From March 2015, a new approach was adopted for reporting on progress and performance to the Programme Executive. This involved reporting against Unified Plan milestones, provision of metrics on progress by the Service Manager’s delivery teams, reinstating monthly financial reports, sharing daily MI on registrations for the BPS and a regular review of the Secretary of State weekly progress reports. This new package of MI was intended to provide increased confidence and understanding to the Programme Executive. Figure 8 sets out the nature of the available MI and how this changed during the course of the Programme.

Assurance activity

3.18 The Programme’s Assurance Director was responsible for ensuring effective assurance arrangements were in place, providing monthly reports to the Programme Board and specific internally and externally commissioned reviews and assurance. The Assurance Director did not always have the influence required to ensure sufficient challenge, especially from October 2014, despite escalating some issues to the Permanent Secretary.

3.19 The SROs occasionally commissioned external reviews on key areas of risk, such as systems integration and disallowance. Early warnings from these reviews were not always heeded by the Programme’s leadership. The Programme provided evidence of how it tracked its resolution of recommendations from external reviews, although governance bodies were not always sighted on resolution. The reporting of some reviews was also significantly delayed.

Good news culture

3.20 Changes to governance and escalation routes were a particular concern of many Programme officials and other stakeholders. We were repeatedly told that issues escalated to the Programme Board, the Programme Executive (which replaced the Board) and successive SROs were often not followed up. Some formal governance meetings were viewed as being stage-managed, and officials reported that they felt inhibited from raising their concerns to the Programme Executive for fear of being seen as ‘not on board’ with the Programme. This began to improve at the beginning of 2015, but the culture had previously restricted the Programme leadership’s ability to recognise when elements of the Programme were going wrong.

3.21 Examples of concerns that were escalated, but not adequately addressed, included the readiness and suitability of Verify for farmers and the Programme’s lack of capacity to fulfil the systems integration role. The Programme’s user insight team and scrum teams, for example, escalated their serious concerns about the readiness of Verify to the Programme Executive from early 2014, several months before this new service was expected to be accessible to all customers. This was not raised in the Programme’s risk register until June 2014, and not addressed until after registrations had been opened to all applicants. Similarly, major concerns about the Programme’s ability to carry out the systems integration role in-house were voiced both inside and outside the Programme from as early as February 2013. A specific concern was raised in the Programme’s risk register, but was not adequately addressed.

3.22 Weekly update reports for the Secretary of State to show progress towards being ready to make payments in December 2015 did not present a balanced view. The reports show actions completed each week and actions planned for the following week, but do not show weaknesses or failures to complete planned tasks. Actions planned for one week often do not appear as completed the following week, or only appear several weeks later as completed. Reports have been recently supplemented by a weekly dashboard that provides a factual view of achievements against an expected profile.

GDS support for the Programme

3.23 A number of fundamental changes were made to the Programme’s design as a result of the ICT spending controls process, but the Cabinet Office did not provide the support that the Department believed it needed to accommodate them. The number of approaches introduced that were new to the Department and the complexity of the Programme presented a number of challenges, including obtaining sufficient expertise and resources. Although GDS did provide resources to the Department, its own limited resources prevented it from being able to provide all the support that the Department required. The Department and the RPA described GDS support as patchy. There was little continuity in personnel and GDS staff were reported to have provided insufficient insight into the use of agile at this scale.

3.24 GDS committed to reducing overall costs and improving delivery confidence. However, no formal estimates of cost reductions were provided and the Major Projects Authority’s delivery confidence assessments did not improve. The Memorandum of Understanding agreed between the Department and GDS in January 2013 did not set out the level of support the Department could expect from GDS and this left the Programme exposed. It states that “GDS will provide key people resources to work with Defra to ensure success of the Programme”. Many of the commitments GDS made to the Department are vague. For example, it did not quantify the savings that the use of agile would achieve: “no formal estimates of cost savings will be offered but previous experience of operating in an agile manner would suggest a significant cost reduction can be expected from traditional approaches to large scale IT procurement”. It was agreed that the Memorandum of Understanding would be reviewed every six months at Programme Board level, but this did not happen.

3.25 A number of Programme officials told us that they would find written guidance on agile programme management and governance useful. GDS provides guidance through its Service Design Manual and ad hoc blogs and has provided other additional guidance, for example, its advice on the process for gaining approval for agile projects as a supplement to HM Treasury’s Green Book. More comprehensive guidance on agile management would help departments align governance for major projects with traditional governance structures.

Key difficulties applying the seven fundamental changes

3.26 The challenges the Department experienced in applying all of the seven fundamental changes are detailed in Figure 4, but Verify and systems integration gave rise to particular difficulties, requiring significant changes to the registration and application processes.

Adoption of Verify

3.27 A range of stakeholders raised concerns about the suitability of Verify, the government’s identity assurance system, because they believed that customers did not have the documentary evidence and credit history required. While some farmers successfully used Verify, an alternative should have been offered from the start. Verify was not ready, despite the Department’s expectation that it would be. The Department had to move to an alternative telephone identity assurance service. Considerable time and resources trying to make the other system components ready to work with Verify had been diverted away from other pressing priorities.

Systems integration

3.28 The Department experienced significant difficulties obtaining the systems integration expertise it needed. Initially the Department outsourced the systems integration role, but this arrangement was terminated as a result of the Cabinet Office ICT spending controls. These controls aimed to limit the day rate paid, and avoid the risk of being too dependent on a single large supplier. We reported in January 2013 that the pace, breadth and depth of change required by the Cabinet Office’s ICT reforms is opening up capacity and capability gaps across government.7 As a result, the Department brought the systems integration function in-house, but underestimated what this meant in practice and did not foresee the difficulty it would have in getting the necessary skills in place. In its December 2014 review, the MPA reported that the lack of technical resources remained a major issue for the Department.8

3.29 By December 2014, the Programme had not completed a walk-through of the end-to-end process to ensure it worked. Because of the failings in systems integration and lack of prioritisation, some of the major components of the system were not sufficiently developed to enable integration testing.

6 This committee has high-level oversight of the operational running of the Department and cross-network issues.

7 Comptroller and Auditor General, The impact of government’s ICT savings initiatives, Session 2012-13, HC 887, National Audit Office, January 2013.

8 Major Projects Authority, Project Validation Review (December 2014).

Part Four

Damage limitation

4.1 The Department for Environment, Food & Rural Affairs (the Department) revised its plans in March 2015 when it recognised there was a high risk that systems would not be ready to allow farmers to apply by the EU deadline for their CAP payments. The manner and speed of the Department’s response was crucial to limiting reputational damage, but more importantly to ensuring farmers could submit their claims on time. This part of the report looks at:

what the Department did to recover the situation;

what the Programme has delivered; and

remaining risks and challenges.

Actions to recover the Programme

4.2 In February and March 2015, the mounting difficulties encountered culminated in three failed attempts to fix the link between the web portal and claims processing system over successive weekends. Programme and GDS staff and suppliers all put significant effort and resources into resolving these issues but were unable to do so in the time available.

4.3 The Department responded quickly by reverting to a ‘paper-assisted digital’ approach. A range of approaches were introduced for claims, accommodating the complexity of customers’ claims:

agents representing larger customers and those with more complex claims were trained in, and given access to, an alternative online interface allowing them to enter claims online;

customers that were not represented by an agent, but who needed to submit a new or changed claim, were required to supply details on a partially pre-populated paper claim form; and

customers whose claim was unchanged from the previous year were sent an email with details of their previous claim and were asked to click to confirm that no changes were required.

4.4 The RPA used its existing call centre and strengthened its network of drop-in centres to provide help with paper and online applications. Drop-in centres were open until midnight on 15 June, the deadline for submitting claims.

4.5 The RPA Chief Executive was given responsibility for stabilising the Programme by delivering a revised plan, and ensuring the majority of farmers receive BPS payments in December 2015. There are still significant challenges and, in September 2015, he told the Environment, Food and Rural Affairs Select Committee that, while advance payments are permitted, the RPA was not in a position to pay before 1 December 2015, but he was confident that the majority of farmers would be paid in December and the ‘vast majority’ by the end of January.

4.6 To minimise disallowance penalties, the Department will need to assure itself that payments made will be accurate, and expected controls robustly applied. Manual data entry has created a large number of errors. According to the RPA’s best estimate, these affect 21% of claims. Not all errors impact payment, and the RPA expects verification and validation checks to reduce this to 13% of claims, and further work is under way to identify and mitigate these data entry errors. However, further validation and mitigation is required to ensure BPS payments made from 1 December are accurate.

4.7 The Department and its suppliers have now partially or fully delivered many of the key components of the service (Figure 9). For several of the components that have been delivered, work is still ongoing to enhance and develop the products and services. Of the nine key system components, three have been fully delivered. Others such as the Land Management System and the customer portal are in use but with reduced functionality.

Financial impact of change

4.8 A number of direct financial impacts have arisen from the Programme’s failings:

the Department reviewed its systems in March 2015, and identified an impairment charge of £5 million.9 This figure will be reviewed again in March 2016 and we expect it to increase;

the move to paper-based applications incurred direct costs of between £3 million and £4 million for the Programme, with additional operational resource and infrastructure costs of around £7.5 million being absorbed by delivery bodies;

old agreements from Pillar 2 will not now be migrated on to the rural payments system in order to de-risk the Programme. This will necessitate extending the use and associated cost of legacy systems. The Department has not estimated the cost of this;

HMRC provided 150 government apprentices to input manual claims. The current forecast of these costs, which the Department will have to cover, is £2.1 million; and

GDS provided additional resources to the Programme.

Future development

4.9 A more staggered approach to development is planned for future years. An addendum to the full business case was submitted to HM Treasury for approval in September 2015, setting out a move to a ‘minimum credible service’. It is anticipated this will reintroduce online applications through the web portal from 2016, with increasing functionality and automation over time, alongside the continuing opportunity for paper applications. The service will also be adapted to apply new priorities and policy decisions.

4.10 Some functionality will no longer be delivered. New plans will leave more legacy systems in place at Natural England, increasing ongoing support costs. The new service aims to offer different application routes for customers to choose from. This change will meet farmer preferences that were not fully taken into account in the Programme, but will also increase long-term running costs.

4.11 There is a lack of transparency over the total costs and benefits of the Programme. The business case did not fully quantify benefits as it had deliberately excluded potential reductions in disallowance. This was due to the uncertainty in estimating future risks and the fact that it is not possible to quantify the extent of disallowance penalties that are directly attributable to the Programme. The Department is currently seeking to understand the full costs of the Programme, but has not yet fully quantified the ongoing costs of all the organisations involved.

Impact of immediate pressures on achieving longer-term goals

4.12 Programme resources have rightly focused on resolving immediate priorities to recover the Programme and ensure farmers are paid accurately and on time. In the longer term, it will be important for the Department to focus on developing the systems capability that is most needed to ensure customers can make their application, achieve efficiencies and address the land data issues that currently cause disallowance penalties.

Disallowance costs will be significant

4.13 The Department does not expect to achieve the disallowance reductions set out in the outline business case. This is partly as a result of changes to the way disallowance is calculated by the EC (see paragraph 2.13). But disallowance risk has also increased due to delays implementing the Land Management System and the decision to revert to a paper-assisted digital approach for 2015, with an increased risk of error arising from manual input of changes to land data. The combination of these factors could lead to disallowance penalties for the early years of the new CAP of 10%.

4.14 However, future Programme-related disallowance penalties will depend on the funding and implementation of the planned Disallowance Strategy, final design of the system, the extent of any further delays in implementation and the Department’s ability to challenge EC audit findings.

4.15 Failures in the Programme have led to the displacement of staff that were addressing historic causes of disallowance penalties. Manual input of BPS and the Countryside Stewardship Scheme applications, and associated forms, displaced staff that were previously implementing controls to mitigate the risk of disallowance and updating land data. Land data must be updated before the EC stops imposing ongoing disallowance penalties of £27 million per year. Therefore their displacement presents an increased risk of disallowance. The Department had anticipated this would be complete by 2016, but now expects only to achieve the required updates for mapping data by 2017.

9 An impairment charge is incurred when an asset is revalued downward. In this case, the charge is incurred as a result of the reduced functionality of the system.

Appendix One

Our audit approach

1 This report examined how the Department for Environment, Food & Rural Affairs (the Department) planned and implemented its Common Agricultural Policy Delivery Programme (the Programme). It looks at key events from 2012 and then evaluates the Department’s handling of the Programme when it hit problems in early 2015. We designed the study to address four key questions:

Was the Programme well set up?

Was management information adequate to monitor progress and identify emerging issues?

Was the Programme sufficiently flexible to deal with setbacks and delays?

Are appropriate governance and oversight arrangements in place?

2 We applied an analytical framework with evaluative criteria, which included considering what would be optimal for planning and implementing a programme of this kind. By optimal we mean the most desirable possible, while acknowledging expressed or implied restrictions.

3 Our audit approach is summarised in Figure 10.

Appendix Two

Our evidence base

1 Our independent conclusions on whether the Department’s design and implementation of the Programme achieved value for money were reached following our analysis of evidence collected in September and October 2015.

2 We applied an analytical framework with evaluative criteria, which included consideration of what would be optimal for planning and implementing a programme of this kind. Our audit approach is outlined in Appendix One.

3 Our approach relied heavily on evidence collated from 32 semi-structured interviews with Programme officials and wider stakeholders and observers. We also asked suppliers and stakeholder bodies to submit written responses to pre-set questions.

We assessed how well the Programme was set up.

4 We reviewed key internal documents as well as published and unpublished external reviews in order to assess whether there were clear and consistent Programme objectives, including:

papers for the Future Options Programme (the forerunner to the Programme);

the business cases;

board papers and minutes from the early stages of the Programme;

audit committee papers;

internal audit reports;

framework documents with Government Digital Service (GDS); and

Major Project Authority reviews.

5 We interviewed key Programme officials including:

former Permanent Secretary of the Department;

former and current senior officials at GDS;

the four senior responsible owners (SROs) of the Programme;

Programme Director;

Service Manager;

two Programme Assurance Directors;

two Technology Directors;

representatives from the delivery bodies;

head of the Programme Management Office; and

Internal Assurance officials.

6 In order to understand the customer perspective and how far their needs and views were understood by the Programme, we invited four written submissions from:

the National Farmers’ Union;

the Central Association of Agricultural Valuers;

the Tenant Farmers Association; and

the Country Land Association.

7 We also obtained written submissions from key suppliers (Abaco, Kainos and Sopra Steria). We triangulated these submissions with other sources including interviews with the Programme’s user insight team and by reviewing key documents related to customer engagement.

We assessed whether the Programme’s management information was appropriate.

8 We reviewed key management information, sent to the Programme
Board/Executive.

9 We interviewed the head of the Programme Management Office, responsible for compiling management information to understand how the information was compiled and how this changed between SROs.

10 In our interviews with delivery bodies and written consultation with suppliers we asked about the information they provided to the Programme.

We assessed whether the Programme had sufficient flexibility to deal with setbacks.

11 We examined relevant documents including:

business cases and planning documents to understand the level of contingency;

Major Project Authority assessments; and

budgets and forecasting papers.

12 We interviewed Programme officials to understand how specific setbacks had been managed.

We assessed whether appropriate governance and oversight arrangements were in place.

13 We reviewed internal Programme papers to understand the changes made to the governance structures under each SRO.

14 We interviewed key Programme officials to ask about governance arrangements in order to get their perceptions of strengths and weaknesses. We asked about escalation routes and whether roles and accountabilities were clear.

Design and Production by NAO Communications
DP Ref: 10835-001