Report

by the Comptroller
and Auditor General

Department for Work & Pensions

Welfare reform – lessons learned

HC 77 Session 2015-16 29 May 2015

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO, which employs some 820 employees. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund have used their resources efficiently, effectively, and with economy. Our studies evaluate the value for money of public spending, nationally and locally. Our recommendations and reports on good practice help government improve public services, and our work led to audited savings of £1.1 billion in 2013.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 27 May 2015

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

26 May 2015

This report examines lessons from the Department’s implementation of recent welfare reforms.

© National Audit Office 2015

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.gsi.gov.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

10724 05/15 NAO

The National Audit Office study team consisted of:
Catherine Becker, George French, Vinh Nguyen, Sam Piper, Rebecca Sidell, Andrew Tuffin and Emma Willson under the direction of Max Tse.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key findings

Summary

1 What can we learn from recent welfare reforms? In the past five years the Department for Work & Pensions (the Department) has changed the support it offers to jobseekers, pensioners, disabled people and separated parents. The Department has had a mixed record of implementing reforms, raising questions on how to manage individual reforms and how to assess its overall capacity for change.

2 In this report we examine lessons from the Department’s implementation of recent welfare reforms. The implementation of a reform matters for several reasons. First, it is important in its own right and can affect large numbers of people. Second, it can signal problems with a scheme’s design or operation. And finally, it gives the Department a chance to test policies and learn from experience.

3 We consider a few specific aspects of implementation and explore them in detail. There is much guidance for programme managers and policy-makers on setting up programmes or avoiding common pitfalls. In this report we consider lessons from how the Department has managed reforms in practice. For example, many people recognise that phasing in a new programme is a good idea. But there are many ways to phase programmes including by geographic area, household size, or type of claim.

4 We take a broad view of what constitutes welfare, and include the 2012 child maintenance scheme and welfare-to-work schemes as well as more conventional benefit changes. For detailed comparisons of early implementation we draw on our previous reports on: Universal Credit, Employment and Support Allowance, Personal Independence Payment, the Work Programme, child maintenance and Housing Benefit. Building on these examples we consider:

the context for welfare reforms (Part One);

lessons for anticipating challenges in welfare reforms (Part Two); and

lessons for responding to challenges in welfare reforms (Part Three).

5 To organise our findings we use our core management cycle, which shows the importance of learning and feedback in major programmes. We describe our approach in more detail in Appendix One. Lessons from welfare reform should also resonate widely across government. Jobseekers react in uncertain ways to welfare-to-work support, just as energy users react unpredictably to efficiency incentives. And benefit claimants criticise delays in processing, as do people renewing passports.

Lessons learned

Recent context

6 The Department has had to deal with an unprecedented number of major programmes and reforms. The Department has introduced some change to almost every benefit it manages. We estimate that over the past five years it has introduced around 30 distinct programmes, including changes to benefit entitlements and major programmes such as Universal Credit and Personal Independence Payment.

7 The Department has reduced its spending, while dealing with the challenges of an economic downturn. It has reduced departmental administrative and programme spending by 18% between 2010-11 and 2013-14. At the same time it has had to deal with greater demands on jobcentres through the recent economic downturn. We have found previously that the Department sustained headline performance levels through the downturn.

8 The Department has introduced many reforms without significant operational problems. Despite some important and high-profile early failings, the Department has introduced a large number of reforms over the past five years with few signs of operational problems. We have not been able to review these programmes in depth, but the Department deserves credit for managing so many of these changes and continuing to make progress across its portfolio of reforms.

9 The Department has also continued to make progress in major programmes. It has expanded Universal Credit to over 270 jobcentres for single unemployed claimants and nearly 100 sites for couples and families. In June 2014 it introduced phase 2 of the child maintenance 2012 scheme including charging and case closure for legacy cases. Published statistics also show some improvements in programme performance. For example, Work Programme outcomes have continued to improve for recent cohorts, and claim clearance times have come down since 2014 for Personal Independence Payment.

Anticipating challenges

10 The Department often has a clear, high-level vision and specific plans for reform, but needs to develop a strategic approach to managing uncertainty. By identifying the core elements of major reforms, the Department would be able to anticipate what might happen during planned implementation, and could better manage programme and policy development iteratively. The Department did integrate policy changes with programme and IT development on child maintenance reforms. But it initially struggled to develop Universal Credit contracts, governance and assurance structures using an iterative approach.

11 The Department has relied too heavily on uncertain and insufficiently challenged assumptions without understanding what it means for programme risks. For Employment and Support Allowance and Personal Independence Payment the Department initially made optimistic assumptions about the assessment process. The Department also assumed greater responsiveness by third-party providers, without fully understanding whether they could anticipate, or cope with, changes to operating assumptions.

12 The Department did not have sufficient understanding of its portfolio of programmes or overall capacity. The Department began several large reforms (and many smaller ones) without fully assessing the organisation’s capacity to manage these programmes. It continued to develop these programmes despite recognising the risks of doing so at the same time as reducing costs and reorganising the Department. In 2011 it created a team to start to manage and provide assurance over the whole portfolio. The Department’s approach is still evolving and it is now reorganising responsibilities for portfolio management.

Responding to challenges

13 The Department has thought too late about the management information and leading indicators it needs to monitor progress and performance. The Department should consider information requirements when designing how programmes will work. It has not always developed or interpreted leading indicators for major risks within programmes. This meant the Department took several weeks to identify backlogs in Personal Independence Payment claims. For Universal Credit it had no adequate measure of programme progress and had not sufficiently considered how to initially track progress under agile development methods.

14 In some cases, the Department has responded to uncertainty and adapted plans based on operational advice. Departments should be able to set out plans with specific timetables, costs and impacts. They should also have clear processes for revising plans against changing circumstances or expectations. For example, the child maintenance 2012 scheme programme team postponed roll-out dates several times, based on clear operational criteria. However, for Universal Credit the Department initially held too rigidly to fixed deadlines. It has now adopted a more flexible approach and will need to balance this with clear milestones against which to monitor and assess progress.

15 The Department has learnt to introduce reforms gradually and now needs to develop a more structured approach to phasing programmes. It has clearly learnt from past government experience by using phased or staged starts. Sometimes the Department has failed to use phasing successfully. When it introduced Personal Independence Payment in 2013 the Department did not leave enough time to review performance across the complete assessment process before increasing volumes.

Concluding remarks

16 The Department has accomplished a great deal in the past five years. It has taken on an unprecedented number of reforms while cutting costs and managing a surge in demand following the economic downturn. The Department has shown that it can introduce and adapt programmes flexibly in the face of uncertainty. And it has continued to make progress in difficult major programmes despite early failings.

17 We should expect that any large portfolio of reforms will experience problems. And the Department’s incremental approach allows it to respond to some of the challenges of implementing these programmes. But the Department has relied too heavily on reacting to problems and has not always been able to anticipate likely points of failure or set up leading indicators for performance and progress.

18 With continued pressure to reduce costs and improve services, the Department will not be able to revert to a slow incremental approach whenever it encounters problems, or to delay or reduce planned savings whenever it fails to plan for the uncertainty of assumptions. The Department has had to manage a broad and radical change programme, and learned some hard lessons with significant financial and human costs. It is important that the Department incorporates these lessons into an enhanced approach to managing change quickly and based on more developed forward planning and anticipation of risks.

Recommendations

19 How can the Department address the lessons in this report? In most of these cases, individual reforms have been in their early stages and in the Department’s view it has already addressed some of the issues raised in individual reforms. It has access to a range of guidance for managing programmes and is also developing its overall business strategy alongside identifying ways to exploit common services across the Department and government more widely.

20 In our view, the Department can learn from recent lessons by developing a more resilient anticipatory approach to major reforms. Rather than relying on its ability to react to events, the Department should:

a Plan more openly for the possibility of failure. Departmental planning should set out critical working assumptions, the consequences of any variances, and the Department’s possible response. The Department should anticipate that assumptions are likely to be wrong, beyond simply cautioning that impacts are uncertain. Planning for the possibility of failure shifts management attention to the core programme the Department is trying to preserve, and sets clear responsibilities for managing risks. It should think through contingency arrangements rather than assuming risks can be mitigated or dealt with after the fact.

b Design management information into programmes from the start. Most programme guidance recommends early development of management information. Interim arrangements may be enough for narrow operational purposes. But the value of management information is to clarify processes, flush out inconsistent interpretations (or gaps in responsibilities) and identify risks. Departments should also develop leading indicators of performance at likely points of risk across a process.

c Build an integrated view of portfolio risks and capacity. The Department cannot reliably assess its overall tolerance for risk without clear measures of programme needs and available capacity. There is no hard and fast rule on the number of reforms the Department can manage – after all, many of the recent reforms covered different operational areas and customer groups – but the Department has clearly struggled with gaps in key roles. Based on recent experience the Department should also be extremely wary of undertaking major business reorganisation at the same time as introducing several large programmes.

Part One

The recent context for welfare reforms

A large reform programme

1.1 The Department for Work & Pensions (the Department) spent £168 billion in 2014-15, largely in the form of payments to individuals and households. When we include spending on tax credits and Child Benefit, the government spent £208 billion on benefits and pensions, or 28% of total government spending.1 It was therefore unsurprising when the Department announced several initiatives to reduce benefit spending in 2010 as part of the government’s austerity programme.

1.2 Between 2010 and 2015 the Department has undertaken an unprecedented number of welfare reforms. We estimate that it has continued or introduced more than 30 distinct reforms over this period, which the Department has described as the “most fundamental change to the social security system in 60 years”. Almost no part of the Department, or benefits managed by the Department, has been untouched by reform (Figure 1).

1.3 The Department’s reforms vary in their nature, scale and difficulty. Several reforms have restricted existing entitlements to benefits, for example by capping annual household benefits at £26,000. Others have changed claim processes and eligibility criteria, for example in incapacity and disability benefits. The challenges and risks have varied significantly across these initiatives.

Tight departmental spending

1.4 At the same time as introducing new reforms and dealing with the operational consequences of increased unemployment during the economic downturn, the Department has faced tight restrictions on administrative and programme spending. Between 2010-11 and 2013-14 the Department reduced its spending by 18% in current terms. Of these reductions, 46% related to reduced expenditure on employment programmes and 29% to reduced National Insurance Fund benefit caseloads. Based on plans for 2015-16, the Department needs to identify a further 13% of savings (Figure 2).

1.5 In order to reduce spending and staff numbers the Department started an organisational design review in April 2010 and a further review of corporate functions in early 2011. In our report Reducing costs in the Department for Work & Pensions, we highlighted the risks of undertaking business reorganisation at a time of significant reform without a clear view of the departmental operating model.2 The Committee of Public Accounts raised concerns about the ability of the Department to manage major reforms at the same time as reducing costs.3

1.6 Despite the downturn and spending reductions the Department has sustained its headline performance levels for its main operational areas. For example, in December 2014, 94% of claimants had stopped claiming Jobseeker’s Allowance within 1 year (compared with 92% in December 2010). We have found previously that the Department sustained headline performance levels through the recent economic downturn.4

Mixed progress implementing welfare reform

1.7 In many cases the Department has introduced reforms quickly and without significant operational problems. Within a short space of time it has managed to bring in changes to Housing Benefit entitlements, a household benefit cap and several new welfare-to-work initiatives. It has also introduced new legislation, the Welfare Reform Act 2012, for reforms including Universal Credit and Personal Independence Payment.

1.8 We have not assessed the Department’s performance across smaller reforms, and we have not reviewed whether these reforms have achieved their objectives. Nonetheless the Department deserves credit for managing a large number of smaller changes under ambitious timescales and tight resources.

1.9 In some larger programmes, the Department has experienced high-profile difficulties. Claimants have had to wait months for assessments as part of their Personal Independence Payment or Employment and Support Allowance claims. Working-age households will move over to Universal Credit several years later than originally planned, due to serious problems with the early management of the programme. In response to these problems the Office for Budget Responsibility has reduced and delayed its forecasts of savings to benefit spending.5

1.10 The Department has continued to make progress in major programmes. It has expanded Universal Credit to over 270 jobcentres for single unemployed claimants and nearly 100 sites for couples and families. In June 2014 it introduced phase 2 of the child maintenance 2012 scheme including charging and case closure for legacy cases. Published statistics also show some improvements in programme performance. For example Work Programme outcomes have continued to improve for recent cohorts, and claim clearance times have come down since 2014 for Personal Independence Payment. In this report we have not reviewed programme performance in depth. We plan to update on performance on specific programmes in future work.

Learning from recent experience

1.11 Given the number of reforms and the difficulties they have faced, we can identify important lessons for the future. Our reports have reviewed the early progress of several individual reforms and identified specific successes, failings and challenges. In this report we consider common themes from the Department’s recent experience implementing welfare reform.

1.12 The lessons may reflect some of the specific challenges of welfare programmes. People dealing with homelessness, disability or family break-up will often be vulnerable and can respond unpredictably to policies and changes to processes. Reforms need to take account of the impact of new processes, and use expert and stakeholder input to understand likely responses. The Department often cannot know how programmes will affect people and must anticipate challenges in introducing reforms.

1.13 The Department handles millions of interactions with households. The success of reforms often relies on encouraging small hard-to-measure responses across large numbers of people. The Department’s experience during early implementation allows it to modify and change processes as it goes along. It needs good information and should set up programmes so it can respond effectively.

1.14 To organise our findings we use the National Audit Office’s core management cycle (Figure 3). It sets out the main stages that departments should consider in designing and implementing programmes, and highlights the importance of using evaluation and feedback. In Part Two and Part Three of the report we structure the lessons from recent reforms around the core management cycle.

1 HM Revenue & Customs administers tax credits and Child Benefit.

2 Comptroller and Auditor General, Reducing costs in the Department for Work & Pensions, Session 2010–2012, HC 1089, National Audit Office, June 2011.

3 HC Committee of Public Accounts, Reducing costs in the Department for Work & Pensions, Forty-seventh report of Session 2010–2012, HC 1351, 13 September 2011.

4 Comptroller and Auditor General, Responding to change in jobcentres, Session 2012-13, HC 955, National Audit Office, February 2013.

5 Office for Budget Responsibility, Welfare trends report, October 2014.

Part Two

Anticipating challenges

2.1 Many people face complicated situations around their jobs, health, families and day-to-day living arrangements. They rely on a variety of government support, and can be affected in unpredictable ways by a change to any part of the system. Welfare reforms have uncertain impacts both in isolation and in how they affect people cumulatively.

2.2 Given the uncertain and systemic nature of welfare reforms, the Department for Work & Pensions (the Department) needs to set realistic expectations for how processes will work, understand its own capacity to manage reforms, and anticipate how to re-shape programmes to achieve its core objectives when problems arise.

Taking a strategic perspective

2.3 Any programme needs clear aims. Lack of clarity underlies many common problems, such as prioritisation and the management of interdependencies. Most guidance on programme management – whether for government or the private sector – highlights the need for clear aims or an overarching vision for change.

2.4 The trouble is that having a clear vision is not enough for programme leaders to rely on when problems arise. Many people would recognise the aims of Universal Credit (such as making work pay) or Personal Independence Payment (matching support to an assessment of need). The difficulty will be developing the strategy that converts the Department’s overarching aims into a detailed operating model for how a programme is going to work.

2.5 In developing its strategy, the Department needs to consider how it will handle and respond to the unexpected challenges, the constraints it faces and the trade-offs it will need to make. What does it mean for Universal Credit to make work pay? How important is online take-up? What kind of risks should we accept to introduce reforms quickly?

2.6 Recent reforms suggest there are three important ways in which the Department could benefit from a more strategic approach:

Protecting the core objectives of reform. The more transformational and ambitious the reform, the harder it can be to distinguish achievable short-term benefits and break a programme up into manageable steps. For example, with Universal Credit the Department struggled to set out how the detailed design of systems and processes fitted together and related to objectives. This meant that by mid-2012 the Department could not agree what security it needed to protect claimant transactions, it had adopted a demanding interpretation of the principle of ‘digital-by-default’, and was unclear about how Universal Credit would integrate with other programmes. After a reset of the programme in 2013, the Department formed a more coherent plan of what it wanted to achieve and was better able to distinguish between necessary and desirable aspects of the programme.

Exploring different ways to implement policies. The Department recognises the value of developing policy regulations, processes and technology in an iterative way. This has led to developing some simpler policies which have reduced operational complexity. For example, for the child maintenance 2012 scheme the Department uses simpler measures to assess parents’ income, which reduces the complexity of the calculations required. By contrast, in Universal Credit the Department struggled to incorporate an agile approach into contracts, governance and assurance structures in 2011 and 2012.

Identifying wider opportunities for government. A strategic approach goes beyond simply managing dependencies between programmes and considers the wider opportunities across government. For example, the Department has now identified opportunities to use HM Revenue & Customs’ real-time information to simplify the process of validating claims from providers on the Work Programme.

2.7 One important way in which the Department can develop a more strategic approach is to actively plan for the possibility of failure and develop clear scenario and contingency planning. Unlike risk registers and mitigation plans, which try to defend against problems and keep to a pre-determined path, contingency planning forces programme leaders to challenge assumptions about their core aims and constraints. This can be particularly valuable in complicated transformation programmes where risks are hard to anticipate.

Using assumptions

2.8 When planning reforms, the Department must specify how processes will work. It needs to make assumptions about how claimants, staff, other parts of government and third-party providers will interact with new processes and systems. All assumptions are imperfect and some assumptions will be more uncertain than others. Departments need to understand the impact these assumptions can have on their processes and then reduce or manage those risks.

2.9 Recent experience suggests that the Department should pay particular attention to the way it manages uncertainty around major programmes including:

Identifying critical operational assumptions. The Department needs to identify and monitor the assumptions which pose the greatest risks to the delivery and success of the programme. Some optimistic assumptions for Personal Independence Payment claims led to the Department exceeding expected timeframes for processing claims and backlogs for claimants.

Testing and modelling the impact of assumptions. The Department often recognises that assumptions are uncertain and has considered sensitivities and the financial impact of changes to assumptions. But it has not always assessed the impact of uncertainties on its processes to, for example, identify potential bottlenecks (Figure 4).

Building in appropriate learning curves. Given the large volumes of transactions that the Department handles it is important to understand the impact of variation and ‘bedding in’ in the early stages of roll-out. This process can take a long time. For example, we recommended that for the Work Programme the Department factor both slow starts and uncertain volumes into timing, roll-out and design. The Department did not meet its minimum expectations for the first 450,000 claimants referred to the main Jobseeker’s Allowance groups. After two years
it has now met or exceeded minimum performance levels.

Allowing users and delivery partners to challenge assumptions. In some cases the Department has not discussed and developed assumptions with stakeholders or delivery partners. For Personal Independence Payment, providers signed contracts with the Department based on a 42 calendar day processing target, but told us that they were not consulted on how this target was chosen.

Setting clear responsibilities for managing the consequences of variation. The Department has introduced mechanisms to transfer risks to third-party providers, for example through ‘payment by results’ programmes such as the Work Programme. However, as the Department recognises, there are limits to transferring risks to third-parties. Departments bear ultimate responsibility for service standards and are affected by third-party under-performance. For some programmes the Department assumed providers could respond more quickly to changing circumstances, leading to poor performance and impacts on benefit spending.

Understanding interactions across government. In a range of settings the Department has a wider obligation to be informed about the systems and processes it supports, and needs to fight against the risk of interpreting its own role (and accompanying risks) too narrowly. Our report Housing Benefit fraud and error found that the Department (which ultimately pays for Housing Benefit) had weak incentives in place for local authorities (which administer payments) to tackle fraud and claimant error after a claim had been awarded.6

2.10 By identifying and testing critical assumptions at the start of programmes, the Department can reduce risks of optimism bias and better anticipate problems. Building on this better understanding of operational risks, the Department is also then in a better position to use controlled starts or phasing to learn from early experience, and to design appropriate monitoring and management information. We cover these issues in Part Three.

Understanding capacity and capability

2.11 In the context of wider austerity, the Department has reduced budgets and headcount. Between 2011 and 2015 the Department reduced staff by 23%, or around 23,500 full-time equivalent employees. Reductions have been evenly spread across the business (Figure 5).

2.12 The Committee of Public Accounts has raised concerns about the risks of introducing major reforms at the same time as a business reorganisation.7 Reducing numbers does not necessarily impinge on the Department’s ability to implement reform, but it is increasingly important for the Department to assess capacity across the organisation and plan ahead when introducing new programmes.

2.13 Recent reforms have stretched the Department’s ability to manage programmes. In 2011, after it had made policy decisions on the welfare reform programme, the Department created a portfolio management team to manage and provide assurance over the change portfolio. This included monitoring its capacity and identifying gaps. In 2014-15 it estimated that demand exceeded supply by 9% for central service expertise and 20% for IT skills. To reduce demand on resources, the Department only considered programme changes that were both exceptionally urgent and important.

2.14 The challenge for the Department is that it cannot easily compare its available capacity or capability with the demands of reform programmes. It is difficult to assess demand consistently across programmes, particularly as the Department can influence capacity and capability in various ways such as changing staff capability, overall staff numbers, or its programme requirements (such as simplifying a programme). This challenge has been compounded by the Department not having a comprehensive understanding of capacity across the Department or a clear set of prioritisation criteria. Partly in response to these difficulties, and after making limited progress implementing its intentions for portfolio management, the Department reorganised this function in summer 2014.

2.15 Recent experience suggests the capacity and capability of leadership is particularly important for programmes with significant uncertainty, such as welfare reforms. The child maintenance 2012 scheme had a stable leadership team and we found it had developed a culture of openness and challenge. By contrast, insufficient capacity and capability at a senior level – in some cases due to unforeseeable events – created a lack of oversight in the early years of Universal Credit (Figure 6).

2.16 The scale of recent reform has put inevitable pressure on programme leaders. Combining programme leadership and day-to-day operational roles (for example, on Personal Independence Payment until May 2014) has concerned the Major Projects Authority. Similarly the Department has closed down programme teams and handed over new processes to business-as-usual management fairly quickly after implementation (for example on the Work Programme) only to have to revisit this design and establish new management structures. Also, without effective succession planning, moving more experienced leaders around the Department risks a loss of knowledge and continuity in programmes. The Department has since re-organised its Senior Responsible Owner functions and is aiming to have full-time Senior Responsible Owners for all major programmes.

6 Comptroller and Auditor General, Housing Benefit fraud and error, Session 2014-15, HC 720, National Audit Office, October 2014.

7 HC Committee of Public Accounts, Reducing Costs in the Department for Work & Pensions, Forty-seventh Report of Session 2010–2012, HC 1351, September 2011.

Part Three

Responding to challenges

3.1 The Department for Work & Pensions (the Department) handles millions of transactions every week – taking new claims, processing changes and making payments. It has valuable opportunities to learn from experience within this repeated high-volume environment. When it introduces new reforms it can refine services and processes in response to early performance.

3.2 The opportunity to respond to challenges should not make departments complacent about preparing for reforms, nor should it undermine the use of clear milestones and accountability. Recent experience shows that a responsive approach puts an even higher premium on good management information, clear communication of plans and well-planned implementation.

Developing management information early

3.3 Departments require timely management information from the outset of a programme to identify problems and make informed decisions. They need to invest time in thinking about what ‘early warning’ data and leading indicators for major risks they need at each stage of the process. Thinking through the assumptions and risks around processes should help clarify the information required.

3.4 Across recent welfare reforms the Department has often deprioritised management information in order to meet go-live dates. The Department has not always had planned systems in place when services start (Figure 7). The Department has relied on interim information, which has not been complete and can take time to prepare. For example, in the very early testing stages of the child maintenance 2012 scheme, the Department had to gather information manually every day to track the progress of cases. By the time it had to deal with larger case volumes the Department had introduced an interim case management system which it believed was sufficient for phase 1 of the programme. In our report Child maintenance 2012 scheme: early progress we recommended that for later stages of the programme the Department shows it has introduced adequate management to monitor performance, the achievement of outcomes and progress in closing cases.8

3.5 Recent experiences implementing welfare reform illustrate the problem with failing to anticipate management information requirements or delaying them. Early programme implementation provides a critical opportunity to learn about processes. Given the uncertainty inherent in welfare reforms the Department should be placing more not less emphasis on having adequate information and leading indicators of major risks in place from the early stages of implementation.

3.6 Variation will always be expected with new systems and processes. Some variances can be explained by an expected improvement period as people become familiar with the system, others will be early indicators of more significant flaws. Without understanding the reasons behind variances and having the right leading indicators in place the Department will find it difficult to determine whether it can cope with variances.

3.7 Medical assessments are an example of where a better understanding of management information and leading indicators might have helped to identify emerging problems. The Department monitored the total claims outstanding against its expectations if processes operated as planned for Employment and Support Allowance claims (Figure 8). It did not adopt a similar indicator or monitor how long claims were taking for Personal Independence Payment until June 2014. Interim management information had begun to show trends in performance, such as an increasing number of outstanding cases in June 2013. The Department took several weeks to identify delays when it realised assessment providers were not meeting expected performance levels.

3.8 When the Department introduced Personal Independence Payment it also did not collect data on staff productivity or break down data between claims following different processes. It has subsequently introduced these measures and identified ways to make processes more efficient and provide better information for claimants.

3.9 As well as monitoring programme performance, the Department needs to understand how programmes are getting on and develop indicators to assess progress. This will inform decision-making and encourage accountability. During the early years of Universal Credit the Department did not initially have any adequate measures of programme progress. It had not considered how to track progress under agile methods with incremental development, which requires more sophisticated measurement and evaluation to demonstrate value in the early stages of a project, alongside existing measures.

Developing clear milestones while acknowledging risks

3.10 With uncertain operational performance and behavioural responses, the Department will often need to evaluate progress and reassess plans. The early expectations and timetables that the Department sets out in business cases and operational plans will change. The use of agile project management techniques also means Departments need to think differently about governance and iterative project development.

3.11 On first glance there is a tension here. How can the Department reflect the uncertainty of welfare reform programmes when it presents detailed business cases? In our view the Department should be able both to develop transparent plans and also to reflect appropriate points where flexibility may be needed. The Department should be able to set out clear milestones, recognise openly it has not met expectations, have clear criteria for changes to plans and track where and why its plans have changed. The more uncertain and complicated the programme, the more important it becomes for the Department to be clear about what it is trying to achieve at any point in time and to identify tangible milestones.

3.12 Recent experience shows how the Department can balance clear plans with flexible responses (Figure 9). The Department initially struggled to monitor the progress of child maintenance 2012 scheme reforms. Our report Child Maintenance and Enforcement Commission: Cost reduction found that the Department could not explain changes in its budgeted costs.9 After reorganising the programme management the Department developed much clearer control over plans. When the Department postponed and altered the introduction of the scheme several times further, it made these decisions upfront and based on clear operational criteria.

3.13 By contrast our report Universal Credit: early progress found that the Department allowed early indicative timetables to become fixed targets with no significant consideration of alternatives.10 The Department subsequently adopted more realistic plans and, in September 2014, the strategic outline business case was approved. But in our update report we found that the Department’s new ‘test and learn’ approach was still evolving. The programme board recognised the need for clearer milestones against which to plan and assess progress.11

3.14 The challenge for the Department is not to avoid ambitious targets and introduce all programmes slowly, but it should have an appropriate assessment of the risks that timetable decisions create and be explicit on the trade-offs resulting from decisions to change timetables. The Department introduced Personal Independence Payment quickly, partly to achieve projected savings to benefit spending. This limited the time available to engage with stakeholders and test assumptions. Even where the Department slows down implementation to reduce operational risks, the impact of these decisions needs to be evidence-based and transparent.

Reducing risks through phasing

3.15 The Department has clearly recognised that ‘big bang’ starts will not always be appropriate. It has tried to stagger the implementation of most major programmes, and only introduced ‘big bang’ reforms for changes where it is operationally appropriate or required under legislation. Other programmes have adopted a variety of different approaches to implementing reforms (Figure 10). A challenge for the Department is to fully understand each implementation approach so it can determine the most appropriate way to introduce each reform.

3.16 Phased implementation (or controlled starts) can help reduce specific implementation risks. Recent experience shows the Department has not always managed the substance of phasing implementation successfully. In our report on the child maintenance 2012 scheme we found that the Department had introduced the new scheme carefully and safely through using different stages of implementation.12 In introducing Personal Independence Payment, the Department did not identify the most significant risks to test. It used a phased start to test early parts of the process but did not allow enough time between phases to assess performance across the full benefit process before it increased volumes (Figure 11). Backlogs developed undetected early in the programme and the Department did not have sufficient time to resolve problems before the next stage of implementation.

3.17 In setting out a phased start to a programme, the Department should consider what it is hoping to learn from the process rather than assuming phasing will allow a reactive response to any problems. Its particular approach should be tailored to the programme risks identified. The Department should build on the work it has done – to set out its strategic objectives, identify critical assumptions, and manage capacity risks – to identify critical decisions throughout implementation. Recent experience suggests that alongside legislative requirements, the Department should consider several specific factors including:

The length of the operating cycle. The time between claimants or applicants making an application and receiving a decision varies across benefits. For example, child maintenance applicants do not require a face-to-face assessment and should receive a decision within a month, compared with an initial expectation of 74 days for Personal Independence Payment claims. The Department can learn more quickly through phasing when the end-to-end process is shorter.

The local variations in capacity. The Department has experienced particular problems with Employment and Support Allowance and Personal Independence Payment assessments in specific geographic areas where there are shortages of trained assessors.

The speed of response to problems. Disability benefit claims involve assessments conducted by contracted-out healthcare practitioners. A limited pool of practitioners and a 6-month training time mean capacity cannot increase as quickly as in, for example, the Work Programme. Here, caseworkers take less time to train and the frequency and nature of services is more adaptable. The Department will need greater assurance over less flexible processes before implementing for large claimant numbers.

Alternative ways to manage process uncertainties. Phased implementation is only one way to reduce uncertainties. Claimant consultation and system testing could reduce uncertainties by improving the understanding of how processes work. Using more staff during implementation will mitigate the risks of uncertainties. During the early implementation of Universal Credit live service in September 2013, the Department had 1 staff member for every 7 claimants. As caseloads increased, and systems were developed and tested, this ratio rose to 16 in September 2014 and is expected to rise to 37 claimants per staff member by September 2015.

3.18 The Department has also used other implementation approaches such as ‘test and learn’ to introduce reforms gradually. In contrast to ‘controlled starts’ (which test and confirm that developed processes work as intended), ‘test and learn’ allows for the incremental design and development of polices and systems during live running. In our report Universal Credit: progress update we noted that the uncertainty and ambiguity of this approach created additional implementation risks.13 The Department expected this approach to help it learn from live running and inform the development of a digital service alongside achieving some benefits as early as possible.

8 Comptroller and Auditor General, Child Maintenance 2012 scheme: early progress, Session 2014-15, HC 173, National Audit Office, June 2014

9 Comptroller and Auditor General, Child Maintenance and Enforcement Commission: Cost reduction, Session 2010–2012, HC 1793, National Audit Office, February 2012.

10 Comptroller and Auditor General, Universal Credit: early progress, Session 2013-14, HC 621, National Audit Office, September 2013.

11 Comptroller and Auditor General, Universal Credit: progress update, Session 2014-15, HC 786, National Audit Office, November 2014.

12 Comptroller and Auditor General, Child maintenance 2012 scheme: early progress, Session 2014-15, HC 173, National Audit Office, June 2014

13 Comptroller and Auditor General, Universal Credit: progress update, Session 2014-15, HC 786, National Audit Office, November 2014.

Appendix One

Our audit approach

1 This report drew together lessons based on the Department for Work & Pensions’ (the Department) recent experience implementing welfare reform. It considered the recent context for welfare reforms (Part One), how the Department anticipated challenges implementing welfare reform (Part Two), and how the Department responded to challenges (Part Three).

2 We highlighted what we have learnt about implementing welfare reform based on our recent assessments of welfare reforms and past value-for-money reports. We did not assess the value for money of the overall, or individual, welfare reform programmes; or the extent to which the Department has responded to issues identified in previous reports. Nor did we provide an update on progress or evaluate recent performance. We did not provide a value-for-money conclusion, but concluding remarks outlining areas for the Department to consider going forward.

3 The report considered the practical implications of implementing reforms rather than providing a prescriptive set of guidance or ‘golden rules’. It sets out some of the reasons why widely accepted good practice standards have been difficult to implement and achieve across the welfare reform programme. The lessons identified do not provide an exhaustive list but explore specific aspects of implementation in more detail.

4 To organise our findings we used the National Audit Office’s core management cycle. The cycle provides a high-level indication of the activities we would expect at each stage of a programme or management cycle. The cycle can apply across individual programmes or repeatedly across different stages of a programme. Our audit approach is summarised in Figure 12.

Appendix Two

Our evidence base

1 From October 2014 to April 2015 we developed our lessons learned through:

analysing audit evidence from our recent reports and work on welfare reform. We drew on our work on Housing Benefit, the child maintenance 2012 scheme, Work Programme, Universal Credit, Personal Independence Payment and Employment and Support Allowance;14

consulting with stakeholders to develop broader insights into how welfare reform implemented and refine our insights. This included a range of claimant support groups, think tanks and interest groups; and

considering programme management guidance to understand how our lessons fit with existing principles. This includes internal guidance, such as Initiating Successful Projects and Common Causes of Project Failure, and external guidance, for example PRINCE2 and Directing Change.15

discussions with senior officials at the Department, who shared their views of the main lessons learned from recent programmes. In particular we held several open discussions with the senior responsible owners for each of the programmes mentioned in this report and the Accounting Officer for the Department.

2 We have relied largely on our past audit reports to understand the implementation of welfare reform. Past work provides a sufficiently strong and comparable evidence base to identify different approaches and what works well and less well. We requested audit evidence from the Department for Work & Pensions (the Department) in August and November 2014. This would have allowed us to validate or comment on the Department’s performance more broadly and how they have subsequently addressed issues identified in previous reports. However, the Department failed to send the evidence despite requests.

3 Following receipt of a draft report in April 2015, the Department provided some evidence on how it has tried to address issues identified in previous reports and how other welfare reforms have considered these themes. In the Department’s view it has made progress across programmes since the time of our initial reports. This includes addressing concerns about programme management on Universal Credit and reducing the time taken to process Personal Independence Payment claims.

4 Given the selective nature of the evidence provided and the limited time to review a wide range of programmes, we were unable to audit the evidence or consider the additional information in detail. We have acknowledged some recent developments based on published information and statistics in Part One of the report. And we will review several elements of the Department’s progress implementing welfare reforms through our audit programme over the next few years.

14 Comptroller and Auditor General reports: Housing Benefit fraud and error, Session 2014-15, HC 720, October 2014; Child Maintenance 2012 scheme: early progress, Session 2014-15, HC 173, June 2014; The Work Programme, Session 2014-15, HC 266, July 2014; Universal Credit: early progress, Session 2013-14, HC 621, September 2013; Universal Credit: progress update, Session 2014-15, HC 786, November 2014; Personal Independence Payment: early progress, Session 2013-14, HC 1070, February 2014.

15 National Audit Office, Initiating successful projects, December 2011; National Audit Office, Common Causes of Project Failure; Association for Project Management, Directing Change: A Guide to Governance and Project Management, October 2011; Axelos, Managing Successful Projects with PRINCE2 Manual, March 2002.

Design and Production by NAO Communications
DP Ref: 10724-001