Report

by the Comptroller
and Auditor General

Department for Transport

Lessons from major rail infrastructure programmes

HC 267 Session 2014-15 29 October 2014

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO, which employs some 820 employees. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund have used their resources efficiently, effectively, and with economy. Our studies evaluate the value for money of public spending, nationally and locally. Our recommendations and reports on good practice help government improve public services, and our work led to audited savings of £1.1 billion in 2013.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 27 October 2014

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

24 October 2014

This review looks at a number of issues that the Department has faced, as identified at the time of our reports, in sponsoring major rail infrastructure programmes, and sets out the lessons learned from its experience.

© National Audit Office 2014

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.gsi.gov.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

10547 10/14 NAO

The National Audit Office study team consisted of:
Hedley Ayres, Naomi Flood, Alison Hood, Bridget Jackson and Tracey Payne under the direction of Geraldine Barker.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key facts

Summary

Introduction

1 The Department for Transport (the Department) sponsors a number of significant rail infrastructure programmes with which it aims to deliver improved transport benefits to the public. These programmes are large in scale and complex, with construction either expected to, or having taken, 9 to 10 years, and cost up to £21.4 billion. The NAO and Committee of Public Accounts reported on five of these during the last decade, including the now completed Channel Tunnel Rail Link (later called High Speed 1) and the modernisation of the West Coast Mainline. We have also reported on the progress of Crossrail and Thameslink which are under construction, and High Speed 2 which is still being planned.

2 Our reports have focused on key issues and risks to future delivery covering: the business case; planning for delivery; securing funding; setting up delivery arrangements; construction; and the delivery of benefits and evaluation. We intend to revisit the three ongoing programmes (High Speed 2, Thameslink and Crossrail) as they progress.

3 This review looks at a number of issues that the Department has faced, as identified at the time of our reports, in sponsoring major rail infrastructure programmes, and sets out the lessons learned from its experience. The lessons will also be of interest to other departments which sponsor or invest in major infrastructure although they will need to be adapted to the specific circumstances of each programme.

4 To date we have not examined, and therefore do not comment upon in detail, the transition to running train services. We highlight where the Department has carried out further work following our reports, but we have not carried out additional audit work to evaluate the Department’s progress for this review. We are likely to examine these areas when we revisit the three ongoing programmes.

Our report

5 Overall our reports reflect an improvement in the Department’s sponsorship of major rail programmes. It took over responsibility for oversight of programmes following the abolition of the Strategic Rail Authority in 2005. This marked a change in the Department’s role because, prior to that point, it had taken an arm’s-length role on programmes which were largely delivered and overseen by the private sector.

6 In Part One we summarise the five rail programmes and the scope of our reports from which these lessons are drawn. Figure 1 shows issues the Department has encountered, identified from past NAO reports on individual programmes arranged according to the programme management cycle. As this report shows, the Department has responded to many of these issues. The remaining parts of the report describe the Department’s progress against these issues in four main stages in the cycle:

business case (Part Two);

programme delivery arrangements (Part Three);

planning and monitoring progress (Part Four); and

benefits realisation (Part Five).

7 We summarise below the lessons from the Department’s experience with the issues we have identified.

Business case

8 Clear strategic objectives are needed to build support for a programme. The Department’s objectives for programmes such as High Speed 1 and High Speed 2 go beyond meeting an immediate transport need. They include transforming the transport system, supporting economic growth and other national aspirations. These objectives were more challenging to explain because there has been a lack of definitive research on the wider impacts of transport. A programme can be delayed if sponsors cannot explain what they are trying to achieve, and build public and political support. For example, it took a long time for the Crossrail programme to get the support it needed to go ahead. The first bill was defeated in Parliament in 1994 and, after a pause in the programme, the second bill took 3 years to be approved. On High Speed 2, we believe criticism of the programme from taxpayers and politicians occurred, in part, because the strategic case was unclear. Since our report, the Department published, in October 2013, The Strategic Case for High Speed 2 and Parliament has voted to support the programme (paragraphs 2.2 to 2.4).

9 Economic analysis must be sense-checked to ensure results are realistic. The Department’s experiences show it needs to sense-check the results of its analysis particularly where there is strong external pressure to go ahead with a programme. Such scrutiny might have identified errors in early analysis of phase 1 of High Speed 2 which led to an initial benefit–cost ratio of 2.4:1 (restated to 2.6:1 in our 2013 report). After correcting these errors and changes to assumptions, the current ratio, of 1.4:1, is more in line with equivalent programmes. The Department is improving quality assurance of its analysis but the reduction in the ratio undermined public confidence in the decisions it made on the basis of the higher ratio. Confidence can also be reduced if key assumptions are perceived or challenged as unrealistic. For this reason it is important that the Department explains changes to a benefit–cost ratio which can be a normal consequence of updating assumptions or data (paragraphs 2.5 to 2.7).

Programme delivery arrangements

10 The role of programme sponsor should be separated from delivery. The Department has separated its sponsor role from programme delivery since the West Coast Mainline modernisation programme where Railtrack held both roles. It uses different approaches to sponsorship to reflect the needs of the programme. The separation allows the Department to oversee and challenge progress, while the delivery body concentrates on delivering the programme. It is easier to distinguish between sponsor and delivery body roles during delivery than when a programme is being developed. For example on High Speed 2, the Department, as sponsor, established HS2 Limited to develop proposals for a high speed line but it also has a role in promoting the programme to obtain Parliamentary approval (paragraphs 3.2 to 3.3).

11 Governance arrangements will need to adapt during programme delivery. The Department and Transport for London planned changes to the governance of the Crossrail programme at the outset. The delivery body, Crossrail Limited, earned autonomy to let contracts without prior approval from programme sponsors by passing a series of review points which tested its ability to deliver. This has meant key decisions can be made more quickly. The Department also expects Transport for London, to take a more leading role as the programme moves from construction to fitting systems, testing and operating the railway. On Thameslink, the Department was sensible in changing the governance structure when original arrangements did not allow it to manage the interdependencies between different elements of the programme. When we reported in 2013, the Department was strengthening its systems integration team to better manage interdependencies on the programme, and also set up the interface steering group to manage interfaces between the infrastructure and other programmes (paragraphs 3.4 to 3.8).

12 The Department needs more programme management capacity and skills. The Department has limited programme management skills for the scale of its investment programme. It has sought to manage its pool of experienced programme managers by rotating senior staff between programmes, putting the most experienced people on the highest risk programmes and appointing experts to do detailed reviews. These actions demonstrate that it has a shortage of skilled staff and this is an issue which we and the Committee of Public Accounts have repeatedly commented upon. The Department also needs to continue to develop the capability of its senior responsible owners and use its experienced staff to build skills and capability so that it has sufficient capacity for the number and scale of programmes for which it is responsible (paragraphs 3.9 to 3.12).

Planning and monitoring progress

13 Programme plans must be realistic and responsive to unexpected events. The Department is often subject to pressure to start construction as quickly as possible and build momentum on a programme. We saw this on Thameslink where the Department approved the programme budget when plans for the second stage of the programme were immature. The Department responded sensibly by adjusting the scope of some work and rescheduling completion for 3 years later to keep costs for phase two within budget. The Department’s understanding of the need to integrate the various elements of complex programmes at an early stage has improved but it will always be a source of risk for starting rail services. Partly in response to the 2010 Spending Review, the sponsors and Crossrail Limited decided to change and extend the schedule for opening Crossrail, and to extend the deadline for completing the central tunnelled section by a year to reduce integration risks. Delaying the start of full operations contributed to cost savings which allowed sponsors to reduce committed funding by £1.1 billion during the spending review (paragraphs 4.3 to 4.4).

14 Beneficiaries of new transport could contribute more funding and finance. Past finance choices have been based on what the government can afford or on policy rather than value for money. With section 1 of High Speed 1, the Department used private finance which we calculated in 2001 cost £80 million more than HM Treasury funding even though it guaranteed the loans. In 2010, the Department sold a 30-year concession to run the high speed line for £2.0 billion, with a further sale possible after 2040. However, it is responsible for repaying project debt valued at £4.8 billion at the time of our report. On Crossrail, sponsors have more successfully used a mixed model of public and private funding. Beneficiaries of the line are providing funding through a business rate supplement and negotiated contributions from private sources (paragraphs 4.8 to 4.11).

Benefits realisation and evaluation

15 Benefits and outcomes need to be evaluated to inform future programmes. Evaluation of completed programmes would provide evidence of the impact of transport investment. This could also help the Department negotiate larger contributions for future programmes from those who benefit most from proposed investment. Responding to criticisms from the Committee of Public Accounts, the Department has commissioned an evaluation of High Speed 1 which we understand will be published shortly. This information has not been available to help the Department develop High Speed 2. The Department is considering how it will evaluate programmes such as Crossrail and High Speed 2. This early planning will allow the Department to collect data it needs for the evaluation and monitor progress in achieving benefits during programme delivery (paragraphs 5.5 to 5.7).

16 A single organisation should be responsible for delivering wider benefits. We have seen that transport investment can stimulate redevelopment around stations at some locations. However, expected developments around the new High Speed 1 station at Ebbsfleet have not yet occurred several years after the line opened. The Department partly attributes this to the economic downturn and the availability of other viable development opportunities. Regeneration and development are not the Department’s core responsibility, and no other organisation was responsible for coordinating activities at Ebbsfleet. The government announced plans in the 2014 budget to address this. It is important that wider benefits are delivered to achieve the benefit–cost ratio originally expected from a programme (paragraphs 5.2 to 5.4).

Our view on the Department’s progress

17 Our reports show that the Department has made progress in its management of rail infrastructure programmes. In particular, it has established governance structures which separate its role as sponsor during delivery and allow construction to progress. It has also responded well on programmes such as Thameslink and Crossrail to manage risks and control costs, using risk-based assessments of likely costs and completion dates. Our reports also indicate that there are areas where the Department has not made as much progress as in others or where it will need to focus as current and future programmes develop:

developing clear strategic business cases and scrutinising economic analysis of the estimated benefits of new railways;

building its capacity and capability as a sponsor;

developing plans that are mature and realistic before construction begins;

understanding the impact of transport investment so that it can seek a greater contribution from those who will directly benefit from a programme; and

monitoring and evaluating benefits against the programme’s original objectives and using evaluation to inform future programmes.

Part One

Introduction

1.1 During the last 20 years or so, the government has overseen several major programmes to improve UK rail infrastructure, and deliver improvements in transport benefits to the public. In the last decade we have reported on the progress of 5 of these programmes. These include 3 reports on the Channel Tunnel Rail Link (later referred to as High Speed 1); reports on the modernisation of the West Coast Mainline, Thameslink and Crossrail; and early preparations for High Speed 2. Figure 2 and Figure 3 indicate the scale and length of the programmes. They show that for several years, the Department for Transport (the Department) has been either developing or implementing multiple programmes at the same time.

1.2 In these reports, we examined:

the business case;

planning for delivery;

securing funding;

setting up delivery arrangements;

progress during construction; and

in our last report on High Speed 1, achieving benefits and evaluation.

1.3 In past reports, and therefore in these lessons, we examined the delivery of the infrastructure rather than running services. We intend to revisit the 3 ongoing programmes (High Speed 2, Thameslink and Crossrail) at vital stages, including the transition to running services.

1.4 This review looks at a number of issues that the Department has faced (Figure 4) in delivering its major rail infrastructure programmes, and sets out the lessons learned from its experience. As this report shows, the Department has responded to many of these issues. The lessons will also be of interest to other departments which sponsor or invest in major infrastructure. We do not consider construction and operational issues in detail as these are primarily the responsibility of delivery bodies and the private sector. Instead, we concentrate on the Department’s oversight and how it manages the risks to which it is exposed.

1.5 We and the Committee of Public Accounts examined the 5 programmes at different stages of their development, focusing on the issues relevant to each programme at the time. This review does not therefore compare the 5 programmes against each other. Figure 5 summarises our assessment of the Department’s performance in each of these programmes when we reported. As can be seen with the 3 Channel Tunnel Rail Link (later referred to as High Speed 1) reports, the focus of our assessment changed as the programme progressed. In this review, we highlight where the Department has sought to address issues since our reports, but we have not carried out additional audit work to evaluate the Department’s progress.

1.6 Of the two completed programmes, High Speed 1 was completed within the overall available funding and timescale but at a higher cost and later than more challenging programme targets. The West Coast Mainline modernisation programme was completed to its revised timescale at a cost of nearly £9 billion. Both the Thameslink and Crossrail programmes are on track to be delivered within budget and revised timescales. As part of the Spending Review 2010, the Department extended the timetables for both programmes to reduce costs and risks. On High Speed 2, the Department met its challenging milestone to introduce the hybrid bill for phase one to Parliament in 2013 within the budget for bill preparation.

1.7 In Parts Two to Five, we set out the issues the Department has faced in its major rail infrastructure projects and comment on its response to these issues in four areas:

Business case (Part Two);

Programme delivery arrangements (Part Three);

Planning and monitoring progress (Part Four); and

Benefits realisation (Part Five).

Part Two

Business case

2.1 This part sets out issues the Department faces when it is developing business cases on which it bases its investment decisions. Interested parties also use these cases to understand the Department’s decisions. We have focused here on the parts of business cases which are most developed when the Department makes its investment decision.1 These are: the strategic case, which provides evidence supporting the need to intervene, and the economic case, which quantifies the benefits and costs of options.

Issue 1: Strategic case for investment

2.2 The Department’s reasons for investing vary between programmes. For programmes such as High Speed 1 and High Speed 2 the strategic objectives go beyond meeting transport needs and include transformational, wider economic or national aspirations. The case for these programmes is inevitably more complex to assess and articulate than programmes such as Thameslink, where the rationale is based only on transport factors. A criticism we and the Committee of Public Accounts made about the Department’s early planning on High Speed 2 was that the Department needed to set out its strategic case more clearly. The Committee found that the Department had yet to demonstrate that the improved connectivity between London and regional cities would enhance growth and activity in the regions rather than drawing more activity into London. Since then, the Department has further articulated its case by publishing The Strategic Case for HS2 in October 2013.2

2.3 The Department needs to explain to taxpayers and politicians approving a programme the reason for it, to secure the support they need for it to go ahead. For example, Crossrail did not have strong political support initially. It was rejected by Parliament in 1994 during a recession. A new Crossrail Bill was not introduced until 2005 and received Royal Assent in 2008. The programme underwent significant revision and scrutiny in the interim. The updated business case shows that Crossrail is needed to meet forecasts of employment growth and demand in peak-time public transport in London. On High Speed 2, the Department believes that its 2013 strategic case has helped change the public debate on the programme and Parliament has voted to support it.3

2.4 One of the problems with drawing up business cases for programmes that aim to bring wider national and economic benefits has been the lack of definitive research and evidence to demonstrate such impacts. This is a problem not only in this country but worldwide. We comment on how the Department is starting to develop its body of knowledge in Part Five. The lack of evidence meant that the Department has had to apply its judgement on what to include in a business case, making it more likely that issues were subject to public debate. Nevertheless, clearly articulating the problem that the Department or wider government is trying to resolve and how the programme will seek to do that is vital to help make investment decisions.

Issue 2: Use of economic analysis

2.5 Economic analysis centres on the production of quantified benefit–cost ratios. However, these ratios are necessarily uncertain as they are based on estimates often for several decades ahead and are inherently subject to change as time passes and new data become available. The Department has a good understanding of costs and some benefits on its programmes. However, it does not always recognise the limitations of its analysis nor scrutinise outputs to make sure they make sense. This means that its analysis has been open to challenge and, in some cases, decisions have been made using unrealistic analysis.

2.6 We have identified areas of weakness in how the Department uses its economic analysis where there is strong pressure and lobbying for a programme to go ahead:

Challenging and ‘sense-checking’ its economic analyses

High Speed 2 and High Speed 1 each provide examples:

A sense-check on how many High Speed 1 passengers would be needed to achieve forecasted benefits for the new line might have identified they were over-optimistic. Actual passenger numbers in 2010 were around 30% below the forecasts the Department commissioned before agreeing to restructure the Channel Tunnel Rail Link deal in 1998. These forecasts were equivalent to a full train arriving or leaving St Pancras station every 17 minutes.

The original benefit–cost ratio for the first phase of the High Speed 2 programme at 2.4:1 (restated to 2.6:1 in our 2013 report)4 was considerably higher than ratios for other programmes (Figure 6) but the Department did not query why this was the case. Later on, it identified that errors in modelling had the effect of some benefits being double counted. After this was corrected and changes made to some other assumptions the ratio is now 1.4:1. This is more in line with the equivalent ratio on High Speed 1. In the summer of 2012, the Department increased its oversight of the economic analysis for High Speed 2 and HS2 Limited commissioned a detailed, thorough independent audit of the passenger demand forecasts and analysis behind the benefit–cost ratio. We understand quality assurance work is ongoing.

Updating assumptions to reflect real-life behaviour

The Department is well regarded in government and internationally for its economic analysis in which it uses simplifying assumptions to forecast future passenger journeys. Nonetheless, we and the Committee of Public Accounts have criticised the Department for being slow to take account of potentially significant changes to passenger behaviour in its economic analysis for High Speed 2. Since then, in October 2013, the Department published a revised business case for High Speed 2 which used new evidence on the value of business travellers’ time. It has a programme of further research to understand how passenger behaviour has changed when travelling including the impact of new technology such as laptops, tablets and internet connections on trains.

Communicating inherent uncertainty

On High Speed 2, we recommended that the Department present a range of possible benefit–cost ratios as the programme is in its early planning stage and will change during the life of the programme. The 2013 economic case did present the sensitivity of the benefit–cost ratio to key assumptions as ranges and the Department’s value for money guidance now includes the requirement to present ranges around ratios.

2.7 The good practice which the Department has shown in the economic analysis of its rail infrastructure programmes include:

Benefit–cost analysis methodology

The Department has a clearly-defined method for developing a benefit–cost ratio for transport user benefits, such as faster journey times and less crowding. It estimates a second ratio which includes wider economic impacts from, for example, increased productivity and more market competition, although this is more challenging to forecast accurately. We commented in our report on Reducing Costs in the Department for Transport that:5

“The Department had a generally good understanding of the relationships between costs and benefits on specific transport projects, such as Crossrail and national roads schemes, because it had economic appraisals to test various options for reductions.”

Updating analysis

In general the Department updates its analysis at key decision-points, so that its investment decisions are based on the latest data and assumptions. For example, on the Thameslink programme the Department did so in 2013 before procuring new rolling stock; in 2010 as part of the spending review; and before starting work on the second phase of infrastructure.

Sensitivity testing

The Department has used sensitivity testing to understand the range of possible benefits and costs of a programme. When the Committee of Public Accounts first examined the High Speed 1 project, in 2002, it recommended that the Department should examine a wide range of sensitivities, particularly around forecasts for new ventures as they are inherently less certain.6 The Department used such techniques to assess the potential variation in the benefit–cost ratio for High Speed 2 before it decided to go ahead with the project in January 2012. It also carried out and published detailed sensitivity analysis in October 2013.

Quality assurance

Since cancelling the Inter-City West Coast franchise competition, the Department has taken steps to strengthen its assurance of business models, and the analysis it develops from them. For example, models the Department uses for policy decisions have a ‘senior model owner’ who is responsible for quality assurance of the model, its outputs and for ensuring that decision-makers understand the assumptions. The Department’s decision-makers are given a statement of the strengths, risks and limitations of analysis supporting key decisions by the model owner. The statement is independently reviewed.

1 The other parts in HM Treasury’s 5-case model cover financial, commercial and programme management issues.

2 Department for Transport, The Strategic Case for HS2, October 2013.

3 Royal Asset was granted for the HS2 Paving Bill, which allows preparatory work to be carried out, in November 2013. The Hybrid Bill which gives full Parliamentary approval to build phase one of the programme was supported at its first reading in November 2013 and second reading in April 2014.

4 We restated the benefit–cost ratio so that it is comparable with subsequent economic appraisals of High Speed 2 by subtracting indirect tax impacts from benefits rather than costs.

5 Comptroller and Auditor General, Reducing Costs in the Department for Transport, Session 2010–2012, HC 1700, National Audit Office, December 2011.

6 The project was then called the Channel Tunnel Rail Link.

Part Three

Programme delivery arrangements

3.1 Several programmes we have examined have shown the importance of getting programme delivery arrangements right. Since the failure of the West Coast Mainline modernisation programme under Railtrack, the Department has done this well using a variety of arrangements during the delivery phase. However, the Department could also do more to consider integration of all elements of the programme from the outset. The Department will need to remember lessons on governance and assurance to provide current and future programmes such as High Speed 2 with the firm foundations needed for success.

Issue 3: Governance roles

3.2 The initial delivery models used for High Speed 1 and the West Coast Mainline modernisation programme, which were established by the private sector delivery bodies, contributed to difficulties those programmes faced in the early stages:

High Speed 1

Governance sat largely with the private sector under an arm’s-length approach established for the original private finance contract. The Department did not change these arrangements after it agreed to guarantee the debt the delivery body, London and Continental Railways (LCR), needed to fund construction. The Department had to step in when costs rose above contract targets and milestones were at risk on the second phase of construction. This required the release of contingency funding held by LCR.

West Coast Mainline modernisation

At the outset of the programme in 1998, Railtrack, the then private owner of the UK’s rail infrastructure, was both the sponsor and the programme’s delivery body. This resulted in unclear programme governance arrangements, which contributed to the programme’s failure. In 2002, after the wind-up of Railtrack, the Strategic Rail Authority became the programme sponsor (replaced by the Department after the Authority was wound up in 2005). Network Rail, the body that replaced Railtrack, became the main delivery body. The advantage of this model is that it allows the programme sponsor to challenge and oversee the programme, and a separate body to focus on delivery.

3.3 There have been clear developments in the Department’s approach to delivery models and programme governance. The respective roles of the Department as sponsor and the delivery body are set out in agreements between the parties:

Thameslink

The Department became the sponsor of the Thameslink programme in 2005, taking over from the Strategic Rail Authority, and Network Rail is the main delivery body. A specific ‘regulatory protocol’ defines how they work together to implement the programme. This is a departure from the usual arrangements for Network Rail’s infrastructure programmes, where the Office of Rail Regulation is responsible for regulating efficiency and safety. The protocol gives the Department a more direct monitoring role, designed to reduce its exposure to the risk of the programme overrunning.

Crossrail

The delivery model on the Crossrail programme is somewhat more complex, but works similarly. The Department for Transport is joint programme sponsor with Transport for London and the sponsors established Crossrail Limited in 2009 to deliver the programme.7 A project development agreement signed by the 3 parties gives, for example, a high level description of what will be delivered, the roles and responsibilities of each party, the controls that sponsors have over the programme, and the governance arrangements. Network Rail is upgrading the surface sections of the railway and is not party to this agreement but has a separate regulatory protocol with the Department and Crossrail Limited. These founding documents have provided the programme with a stable foundation, which has been important for its success so far.

High Speed 2

The Department sponsors the programme and set up HS2 Limited in 2009 to develop and promote the programme. Working arrangements are set out in a development agreement, which has been agreed but not yet formally approved. It is easier to distinguish between sponsor and delivery body roles during delivery than when a programme is being developed. During programme planning, the Department has to both promote the scheme to get Parliamentary approval and protect taxpayer interests whereas in delivery it can act solely as programme sponsor.

3.4 Crossrail Limited has earned considerable autonomy to, for example, let contracts without requiring sponsor approval. This feature of the programme has proved particularly beneficial to allow delivery to progress to schedule. Under the development agreement the programme had to pass 4 review points to give sponsors assurance that it was set up well and to set out an exit route if sponsors’ expectations were not met. The review points tested, for example, the robustness of the programme’s cost estimates and Crossrail Limited’s capability to deliver the programme. The Major Projects Review Group was involved in review points 2 to 4. HS2 Limited and the Department have included a system of review points in the development agreement for High Speed 2.

Issue 4: Impact of governance arrangements on decision-making

3.5 In response to significant delay and cost increases on the West Coast Mainline modernisation programme, the Department instructed the Strategic Rail Authority to intervene in 2002. The Strategic Rail Authority established new arrangements to enable decisions to be made more quickly and effectively. The main components of these arrangements were:

A Project Board for making key decisions. This board had senior level representation. It was chaired by the chief executive of the Strategic Rail Authority, and included board members from the Authority, Network Rail and the Office of the Rail Regulator.

A Project Development Group took detailed decisions on delivery, resource and operational issues. This was chaired by the Authority’s West Coast Director and attended by Network Rail, the Strategic Rail Authority, with the Office of Rail Regulation in attendance as an observer.

A West Coast Mainline Joint Board which included representation from train and freight operators, where operational and maintenance issues were discussed.

3.6 On Crossrail, the Department for Transport and Transport for London have effective governance and oversight arrangements. The Joint Sponsor Board is the top level oversight body for the programme, where sponsors approve major decisions and challenge Crossrail Limited. The Department and Transport for London are both represented on the Joint Sponsor Board by two voting members and one of the Department’s representatives is the Director General for Rail.

3.7 The Department expects its role to change when the civil engineering elements of Crossrail are completed in 2016 and the programme moves into fitting the railway systems and testing the railway. Then, Transport for London will take more of a lead role. This change is not described in the Crossrail programme agreements, but the Department, Transport for London and Crossrail Limited know that they need to develop transition plans.

Issue 5: Integration of all elements of the programme

3.8 The Department has not always focused early enough on integrating the various elements of a programme but there are signs of improvement. Integrated planning is challenging but crucial to manage the risks around introducing new train services. Rail infrastructure programmes include complex, interrelated elements such as new track, maintenance depots and signalling systems, and are often linked to associated improvements in rolling stock and changes to how the railway will operate:

Thameslink

Despite the programme’s size and complexity, the Department did not devote enough attention to managing interdependencies between the infrastructure, train and franchise early on. The 3-year delay in procuring trains has made delivering other parts of the programme more complex. When we reported in 2013, the Department was expanding the programme management role of the Thameslink systems integration team, and establishing an ‘interface steering group’ to address interfaces between the infrastructure and other department-led programmes.

Crossrail

In contrast, Crossrail Limited’s plans for integrating the programme were well advanced relative to other rail projects we reviewed. A director of operations reporting to the chief executive was in place from 2006 to 2008 during the development of Crossrail plans. In addition, operations staff have been in place throughout the programme. Crossrail Limited recruited the current operations director in early 2013, in advance of the appointment of the operator. The Joint Sponsor Team also worked closely with the Department’s Crossrail and franchising teams to align Crossrail with other rail services.

Issue 6: Programme management skills

3.9 The Department needs to continue to develop the capability of its senior responsible owners. Our reports, and those of the Committee of Public Accounts, have found that the Department lacks a sufficient number of suitably skilled and experienced staff. For example, the Committee concluded that the move of the Thameslink senior responsible owner to the High Speed 2 programme in 2013 was due to the scarcity of project management skills in the Department when compared to the scale of its investment programme. Several early Major Projects Authority reviews of Thameslink and High Speed 2 also noted concerns about the skills and capability of the Department’s teams.

3.10 The Department has tried to maintain continuity on its programmes despite these constraints:

the Thameslink programme benefited from having stability of senior management, including having the same senior responsible owner between 2008 and 2013;

the Crossrail programme has had frequent changes in the Department’s senior representatives overseeing the programme. However, the impact was lessened as a small number of the Department’s staff had rotated on and off the programme’s Joint Sponsor Board; and

the Committee of Public Accounts recommended, in October 2013, that the Department have a clear plan to build skills in the organisation and plan when moving staff will have minimal impact on a programme.

3.11 The Department has also appointed a project representative to support individual programme sponsor teams. The project representative is a team of independent experts which reviews progress and management information. The project representative also carries out detailed reviews of parts of the programme and reports the results to sponsors.

3.12 As well as having the right skills in the sponsor team, the delivery body needs the right skills and experience. The Crossrail programme shows how valuable this is. Crossrail Limited is led by a highly capable senior team with a track record of success in the public and private sectors worldwide. Securing such skills may become more difficult in the future given the number of large infrastructure projects currently in progress and planned.

7 Crossrail Limited replaced Cross London Rail Links, which was established in 2002 to develop the business case
and carry out early planning for the Crossrail programme.

Part Four

Planning and monitoring progress

4.1 Before construction begins, successful programmes need clear, detailed plans including programme schedules, budgets, sources of funding, and a clear understanding of the risks. Getting value for money depends on the Department and delivery bodies striking the right balance between cost, schedule and risks to successful delivery. Long-term, complex programmes are often, however, subject to changing circumstances, and encounter unexpected challenges to which departments and delivery bodies must adapt.

4.2 Our reports show that the Department tends to be either over-optimistic about programme schedules initially or, as with Thameslink, to approve and begin the programme before plans have been fully developed. However, the Department has a good track record of reviewing programmes as circumstances change or risks emerge and adapting plans to reduce costs and increase delivery confidence.

Issue 7: Setting milestones and realistic plans

4.3 To get value for money the Department and delivery bodies have to plan a programme schedule to deliver the specified scope within budget, and with a level of risk that does not create delays or increase costs. Two of the programmes that we have examined – Thameslink and the West Coast Mainline modernisation programme – had plans which were originally over-optimistic and incomplete. The sponsor and delivery body sensibly extended their programme schedules part way through construction to reduce risks, and through that, control programme costs:

Thameslink

The Department approved the programme budget before Network Rail had identified the full scope of the infrastructure work, particularly for the second phase. A 2010 Office of Rail Regulation review also found that programme milestones were optimistic with insufficient contingency. This meant that Network Rail had to work hard to control the programme later on. Phase one was ultimately completed on time and to budget despite scope changes to Farringdon and Blackfriars stations. To keep costs within budget, in response to the 2010 Spending Review, the Department re-scoped phase two and extended the schedule by 3 years.

West Coast Mainline modernisation

The Rail Regulator reviewed the revised West Coast Mainline modernisation programme in December 2003. The regulator reported that the programme’s schedule-driven approach was creating inefficiencies and decided that Network Rail should delay, by 18 months, planned delivery of some major work. This would give Network Rail more time to develop improved scheme designs and more efficient arrangements. Other works to improve the speed and frequency of train services were brought forward.

4.4 Even with thorough planning, it is unlikely that the final programme will perfectly match the initial plan. Recently the Department has been good at working with its delivery bodies to respond to risks and unexpected events as they emerge. For example, on Crossrail forecast costs exceeding £17 billion in some scenarios and the disciplines of the 2010 Spending Review prompted sponsors and Crossrail Limited to revisit scope, schedule and risk. The Department, Transport for London and Crossrail Limited had managed to reduce the cost estimates to £14.5 billion at the time of our report by, for example, agreeing more favourable contracts in a more competitive market. Risk mitigation was also a significant factor. In particular, deciding to open the railway over a longer time period reduced integration risks associated with complex work. This contributed to £1.6 billion savings which enabled sponsors to reduce committed funding by £1.1 billion during the spending review. The full railway will now be operational in December 2019 rather than May 2018 as originally planned. Other than this, for a programme of its size, Crossrail has had relatively few scope changes, a useful indicator of good planning at the outset.

4.5 The Department has improved its approach to developing cost estimates and setting budgets, and is now using more sophisticated costing techniques. On the West Coast Mainline modernisation, costs spiralled out of control. On both Crossrail and High Speed 2, on the other hand, the delivery bodies have used a quantified risk analysis approach both in establishing early cost estimates and, in the case of Crossrail, in monitoring the programme. On Crossrail, the sophisticated approach gives the Department assurance that the final cost with be within the available funds, thus protecting its financial exposure.

Issue 8: Using management information

4.6 On High Speed 1, the Department did not obtain all the financial information to which it was entitled under the original Private Finance deal. This hampered the Department’s ability to monitor Eurostar revenues on which the plans to finance the new railway depended. Shortfalls in expected revenue meant that the Department first guaranteed the debt for the programme, and is now responsible for its repayment.

4.7 The Department more actively monitors Thameslink and Crossrail to manage its exposure to financial risks. This is partly the result of the formal protocols and project delivery agreements described in paragraph 3.3:

Thameslink

The Department responded well in 2009 and 2010 to rising costs on phase two of the Thameslink infrastructure programme. It increased its focus on cost control, working with industry to develop options to deliver at lower cost and using an independent cost consultant to provide assurance over Network Rail’s revised cost estimates. The Department also refined its monitoring approach over time.

Crossrail

The Department uses Crossrail Limited’s reports, along with expert review of them by its ‘Project Representative’, to monitor progress against schedule and budget, and identify where action needs to be taken. Our 2014 report showed how the Department, Transport for London and Crossrail Limited used this information to make changes when its confidence in delivering on schedule slipped to 65%. High quality information has also enabled the Department to protect the benefits that it wants to gain from programmes. Its economic analysis process allowed it to protect the value of its programmes during the 2010 comprehensive spending review. For example, the Department rejected two possible changes to the scope of Crossrail because they would have reduced the benefit–cost ratio of the programme.

Issue 9: Choosing appropriate finance

4.8 In our opinion, the Department’s choice of financing approach tends not to be based on what will provide best value for money. Rather, it is linked to the level of public funding available and to government policy on public or private finance at the time of the project. In the case of High Speed 1, for example, the Department wanted to achieve the government’s aim of keeping the project off the public balance sheet and so required the delivery body, London and Continental Railways (LCR), to raise finance to build the line. However, LCR could not raise all the finance it needed without government help. In 1998, therefore, the Department provided a guarantee to its lenders that it would cover the repayments if LCR could not. This cost the Department £80 million more than if it had used HM Treasury funding.

4.9 The guarantees provided by the Department on High Speed 1 meant that the Department was exposed to shortfalls in Eurostar revenue as a result of lower than expected passenger numbers and, therefore, fare income. At the time the Department did not expect its guarantees to be called upon. However, in 2009 it assumed responsibility for servicing and repaying the debt, which then had a value of £4.8 billion, as part of preparations to sell a 30-year concession to run the completed line. The sale raised £2.0 billion, more than the Department had expected, which it achieved by removing areas of uncertainty and maintaining competitive tension between bidders. A further sale will be possible after 2040.

4.10 Since then the Department has used a variety of funding models on its programmes:

Thameslink

The infrastructure programme has been funded through ring-fencing part of the Department’s Network Grant payments to Network Rail.

Crossrail

The Department and Transport for London have used a combination of types of funding: local taxation from those who stand to benefit from the programme (business rate supplement in the Greater London area); and the sale of surplus land and property it acquired for the project. The Department also entered negotiations with individual organisations, but did not secure all the contributions that it initially expected, in part due to regulatory decisions. It may have to contribute an extra £160 million.

High Speed 2

To overcome uncertainty, HM Treasury have approved in-principle funding on High Speed 2 since we reported in 2013. It is also looking to get private sector contributions but there is uncertainty as to how much it is seeking and for which elements it would wish others to pay.

4.11 To increase contributions from the beneficiaries of future investment, the Committee of Public Accounts concluded that the Department needs a better understanding of the benefits from new transport. Part Five discusses the issues the Department is facing in evaluating its programmes to build the knowledge it needs in this area.

Part Five

Benefits realisation

5.1 This part covers the work to achieve the wider benefits anticipated from improvements to existing rail infrastructure and building new lines. We also examine the difficulties the Department faces in evaluating its programmes so it can understand and compare the outcomes they achieve against its initial expectations.

Issue 10: Focus on benefits realisation

5.2 Achieving the full range of potential benefits from investment in rail infrastructure requires a wider focus than successfully delivering the construction work. The Department has recognised the wider benefits its rail programmes can deliver but achieving regeneration is not always straight forward. For many programmes however, such wider benefits are critical for achieving value for money. For example, on High Speed 1 we estimated that the value of benefits to passengers are likely to be around £7 billion while the net cost to taxpayers will be around £10 billion.8 Substantial other benefits need to be demonstrated, such as from an evaluation of development around stations, for the original benefit–cost ratio of 1.5:1 to be achieved.

5.3 Expected developments around the new High Speed 1 station at Ebbsfleet have not yet occurred several years after the line opened. The Committee of Public Accounts attributed the slow progress in regeneration at Ebbsfleet to a lack of leadership from central government. The government has recently sought to address this by announcing, in March 2014, plans to create a garden city of 15,000 new homes at Ebbsfleet. These plans will be supported by public sector investment, and coordinated by a dedicated development corporation. In contrast, the Department’s delivery body, London and Continental Railways, have worked for many years with private sector developers to develop land around the High Speed 1 stations at St Pancras and Stratford.

5.4 Bringing about regeneration and economic growth does not sit in the Department for Transport’s core area of expertise but it is now focusing on how these will be achieved while its major rail programmes are still being planned and built:

It is developing and implementing a benefits realisation plan for Crossrail with Transport for London covering both transport and wider benefits.

On Thameslink it is overseeing industry plans and actions to make sure benefits are achieved.

It set up the HS2 Growth Taskforce to advise on how to achieve maximum potential  for economic growth from the High Speed 2 programme, during its current planning stage. In response to the Taskforce’s recommendations, the Department has committed to fund the development of local growth strategies and develop a business case for setting up a national regeneration company to help local areas develop and implement their strategies.

Issue 11: Evaluation of programme benefits and outcomes

5.5 In the past, the Department has not systematically evaluated its major rail programmes to determine the outcomes they have achieved and to validate the original business case assumptions. The Committee of Public Accounts has repeatedly raised a concern that, as a result, the Department does not fully understand the wider economic benefits transport brings. This information could provide strong evidence when developing business cases and making investment decisions on future programmes, including negotiating with potential private-sector contributors.

5.6 The Department established its evaluation framework for High Speed 1 after the line had been built and high speed services were running. In its 2012 report on the programme, the Committee of Public Accounts was concerned that this risked developing a framework based on what had actually occurred rather than identifying whether original expectations have been met and understanding the reasons for any under or over-performance. The Department commissioned an evaluation of High Speed 1 in 2013 which we understand will be published shortly. However, this information was not available to inform the Department’s assessment of High Speed 2. It could have provided, for example, useful evidence for the programme business case.

5.7 We recognise that it can be difficult to measure impacts and show causality on rail infrastructure programmes due to the length of time they take to plan and build. Unforeseen external events such as changes in the economic climate, new technology developments, or the influence of other major programmes (such as the London 2012 Olympics at Stratford) can influence outcomes. The historical data and information required to quantify outcomes may no longer be available unless it has been identified and collected at an early stage. The Department responded to the Committee of Public Accounts’ concerns by publishing, in 2013, a monitoring and evaluation strategy and a forward programme for evaluating all major projects for which it is responsible. This means that it has thought early on about data collection, monitoring and evaluation for programmes still in planning and delivery. For example, the Department plans to:

collect baseline data before Crossrail and the Thameslink infrastructure enhancement are opened in 2018 to evaluate whether or not expected benefits were realised;

capture the commercial lessons from these schemes; and

integrate monitoring and evaluation on High Speed 2 with benefits realisation. This should allow the Department to respond during project delivery if the benefits it expects from High Speed 2 are at risk of not being achieved.

8 Over the Department’s standard 60-year appraisal period from 2010, the first year of full services, to 2070.

Design and Production by NAO Communications
DP Ref: 10547-001