Report

by the Comptroller
and Auditor General

HM Treasury, Department for Work & Pensions and HM Revenue & Customs

Managing the welfare cap

HC 952 Session 2015-16 27 April 2016

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO, which employs some 810 people. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund have used their resources efficiently, effectively, and with economy. Our studies evaluate the value for money of public spending, nationally and locally. Our recommendations and reports on good practice help government improve public services, and our work led to audited savings of £1.15 billion in 2014.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 25 April 2016

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

22 April 2016

This report examines the government’s approach to forecasting and managing the welfare cap.

© National Audit Office 2016

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.gsi.gov.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

11067 04/16 NAO

The National Audit Office study team consisted of:
Alastair Bowman, Caroline Harper, Tracey Payne and Andrew Tuffin,
under the direction of Max Tse.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key facts

Summary

1 In April 2014, the government introduced a cap on elements of benefit spending, which it calls the welfare cap. The welfare cap aims to improve spending control and financial management and to ensure that benefit spending remains affordable. It sets overall annual limits on certain types of spending, including many benefits and tax credits. In 2016-17, the cap is £115 billion with a further 2% margin for allowable changes in forecast assumptions. The welfare cap is not to be confused with the ‘benefit cap’ that limits the benefits a household can receive each week.

2 HM Treasury most recently set the welfare cap at the Summer Budget 2015. It sets the cap for a rolling five-year period, with an additional year added at the first fiscal event of the financial year. The Department for Work & Pensions (DWP) manages most of the spending within the cap, with HM Revenue & Customs (HMRC) and HM Treasury being responsible for tax credits and Child Benefit. The Department for Business, Innovation & Skills and the Government of Northern Ireland are also responsible for small amounts of the cap (see Appendix Three).

3 The welfare cap is a self-imposed cap on forecast spending, but not actual spending. A breach occurs when forecast spending in any future year exceeds the cap; actual spending is not assessed against the cap. The welfare cap is therefore a control on changes to government spending plans, while still allowing in-year spending to vary in response to demand and entitlements.

4 The welfare cap is new and processes are developing. But the introduction of the cap has highlighted that forecasts are necessarily uncertain and can contain errors. Given the significance of decisions related to the cap – both for claimants and government spending – departments need reliable forecasts and clear processes for analysing and responding to changes in spending.

Scope of this report

5 In this report we examine government’s approach to forecasting and managing the welfare cap. We do not evaluate whether the cap is an effective or desirable policy, nor do we assess specific options for reducing spending when breaches are identified. We set out our recommendations for government as it develops its forecasts and management of the cap, and highlight limitations or risks to this new control on public spending.

6 In our past work we have highlighted the importance of five criteria for frameworks to control public spending: clear coverage; reliable forecasts; prompt reporting; robust controls; and clear governance.1 In this report we:

review the development of the welfare cap (Part One); and

assess how well the government is managing the cap, against the criteria outlined above (Part Two).

Key findings

The development of the cap

7 The cap has increased departments’ oversight of spending on benefits and tax credits. Since this spending is largely determined by individual entitlements, departments are not able to exercise control in-year in the way they might for other areas of spending. As a consequence spending on benefits and tax credits is not included in departments’ expenditure limits. The introduction of the cap has increased the level of attention to benefit spending forecasts across government and at the Office for Budget Responsibility (OBR) (paragraphs 1.6, 2.18, 2.19 and 2.22).

8 The welfare cap is new and processes for managing it are evolving. In the Autumn Statement 2015 the OBR identified a breach of the terms of the welfare cap, only four months after HM Treasury had set the cap. This is the first time a breach has been identified since the cap was introduced (paragraphs 1.16 and 2.16–2.19).

9 Recent fiscal events have exposed weaknesses in the way sensitive policy changes are costed. There have been two notable errors in recent policy costings. In the Summer Budget 2015 the government made errors in its calculation of the savings from its proposed tax credit measures and a further error occurred when the tax credit measures were reversed in the Autumn Statement 2015. The OBR outlined the source of the Summer Budget 2015 errors as mostly due to a tight timescale combined with a large number of policy measures late in the process. It attributed the Autumn Statement 2015 errors to a restriction of knowledge about sensitive policy changes (paragraphs 1.16, 1.17, 2.16, 2.17, 2.27 and 2.28).

The management of the cap

10 The coverage of the cap is clear and takes into account different factors affecting spending. It includes spending on most benefits and tax credits, but has two significant exclusions. The government has chosen to exclude the State Pension, which forms the largest share of all benefit spending. The cap also excludes spending which is highly dependent on the state of the economy, including Jobseeker’s Allowance and related Housing Benefit payments (and the equivalent elements of Universal Credit). This reduces the level of uncertainty around unemployment assumptions in forecasts but there is a risk that the current scope of the cap may create an incentive for government to keep claimants on Jobseeker’s Allowance, instead of encouraging claimants into work and increasing entitlement to in-work benefits, or on to alternative out-of-work benefits, both of which are inside the cap. Departments told us they believe this risk to be small and have no intention of using the cap in such a way (paragraphs 1.7, 1.8, 2.3, 2.4 and Appendix Three).

11 While the welfare cap builds on relatively well-established forecasts of social security and tax credit spending, there are several areas where analysis and forecasts could be improved:

There is ambiguity in the way some changes in spending are classified. The cap allows for a 2% margin for forecasting errors such as changes in economic conditions. The OBR is responsible for distinguishing forecasting errors from policy changes or reclassification of spending. These distinctions are not always clear-cut, creating uncertainty about the scope of the cap (paragraphs 2.5, 2.6 and 2.23).

Interactions between benefits are not dealt with systematically. At the moment DWP’s forecasts are developed for individual benefit streams with separate manual adjustments to account for interactions between benefits. DWP has a model – known as INFORM – to account for interactions, but recent welfare reforms mean that the model is no longer considered fit for this purpose. DWP plans to invest in upgrading its models (paragraphs 2.10 and 2.11).

There is limited independent model review outside the OBR process. The departments’ analytical and policy teams and HM Treasury have informal links between individuals within and across departments. But in DWP there is heavy reliance on self-review with limited peer-review or independent review of these models. And it is not clearly documented how and when concerns about forecasts of the welfare cap as a whole should be escalated (paragraphs 2.12, 2.13 and 2.31).

Analysis and reporting of forecast uncertainty is limited. Some benefit forecasts do not include quantitative estimates of uncertainty. Policy costings must include an assessment of uncertainty, and the OBR discusses risks in its reports (paragraph 2.26).

There is not always time for the OBR to review all the elements of the forecasts fully – in particular the government’s policy costings – given the complicated reporting environment. To make clear how it has made its assessments the OBR reports where it has challenged estimates, where it has departed from departmental assumptions, and where it has not had time to check estimates. The risk of errors occurring increases when timescales are tightly compressed (paragraphs 2.27 and 2.28).

The OBR believes the process around its scrutiny of policy impacts is improving generally but it has been hampered in some circumstances. Information about the most sensitive policy measures is sometimes restricted to individual lead departments. This limits OBR’s ability to call directly on the expertise of officials across DWP and HMRC. These restrictions make it more difficult to assess the impacts of policy changes across departmental boundaries and have led to errors which the OBR has not identified prior to certification (paragraphs 2.16, 2.17, 2.19, 2.27 and 2.34).

12 Governance and decision-making around controls for the welfare cap are evolving. Departments develop policy options and support ministers to make these policy decisions. It is not yet clear how the welfare cap could affect incentives for departments to control spending through discretionary or operational changes, or how to treat certain reclassifications of spending (paragraphs 2.29–2.33).

Concluding remarks

13 The welfare cap is encouraging greater oversight and understanding of spending. Forecasts will always be uncertain and there are many areas where modelling, processes and controls need to improve and are evolving. But the welfare cap is leading to better challenge of departments’ modelling assumptions, and greater transparency about the factors and limitations that affect forecasts.

14 While the level of the welfare cap and any decisions about social security and tax credit spending are ultimately policy choices, it is important that processes for managing the cap are reliable and support informed and transparent decision-making. When – as is currently the case – spending is projected to be close to or over the cap, any weaknesses in forecasts are particularly important as they may affect real policy or operational decisions. All departments involved should therefore move quickly to improve their processes, including more complete assurance of models, better working across departments and greater clarity over how changes in forecasts are classified.

Recommendations

15 As departments continue to refine forecasts and processes for the welfare cap, we make the following recommendations:

a HM Treasury should support the OBR better by increasing its access to expertise within departments and increasing the time it has to consider the impact of new policy on its forecasts. The adjustments required to forecasts following the Summer Budget 2015 and Autumn Statement 2015 could have been avoided if the OBR had been able to discuss policy costings more openly with the departments involved and if the departments had more time to model the impacts properly. HM Treasury should review its process surrounding policy costings to support better forecasting of interdependencies of benefits.

b DWP should introduce more rigorous quality assurance of models. DWP has already adopted good practice in some areas recommended by HM Treasury’s Aqua Book: guidance on producing quality analysis for government, March 2015. It should build on this by implementing periodic peer review and external review of all its business critical forecast models.

c Improve model integration and analysis of uncertainty. We encourage all departments to discuss uncertainty more openly and try to quantify this to an appropriate level. They should continue to increase transparency over the assumptions in models. The departments should consider publishing models or more details on their methodologies and their core assumptions as has been done in other areas of government.

d DWP should introduce better documentation of models and assumptions. There is currently too much reliance on self knowledge of policy areas which increases the risks of errors and loss of knowledge if staff move elsewhere. We recommend that all policy assumptions, models, interactions and quality assurance arrangements should be fully documented.

e HM Treasury should set out clearer cross-departmental governance arrangements. Better decision-making would be supported by strengthening and clarifying the role of the AME Risk Group which is the only body with attendance from all departments. The Group should have clearer terms of reference and should be able to call on experts across departments in all circumstances so that it can provide informed advice on the combined impact of proposed policy measures to ministers.

1 For example, Comptroller and Auditor General, The Levy Control Framework, Session 2013-14, HC 815, National Audit Office, November 2013.

Part One

Introduction

1.1 In April 2014, the government introduced a cap on elements of social security and tax credit spending as part of its plans to reduce public sector net borrowing. The welfare cap aims to improve spending control and financial management and to ensure that benefits and tax credits remain affordable. This part sets out the key features of the welfare cap and how it has developed. We do not evaluate whether the cap is an effective or desirable policy, nor do we assess specific options for reducing forecast spending when breaches of the cap are identified.

Capping benefits spending

1.2 The government has committed to achieving an overall budget surplus on the public finances in 2019-20. To do this it has set out its plans for ‘fiscal consolidation’, including cutting spending and increasing tax. This fiscal consolidation process was introduced in 2010 when the coalition government set up the Office for Budget Responsibility (OBR).

1.3 The coalition government published the first Charter for Budget Responsibility in April 2011.2 The Charter sets out the government’s commitment to managing public finances. This includes:

ensuring sustainable public finances that support confidence in the economy, and to promote ‘intergenerational fairness’, and ensure the effectiveness of wider government policy; and

supporting and improving the effectiveness of monetary policy in stabilising economic fluctuations.

1.4 In 2014, the government amended the Charter to create the welfare cap. It defines this as:

“a cap on welfare spending, at a level set out by the Treasury in the most recently published Budget report, over a five-year forecast period, to ensure expenditure on welfare is contained within a predetermined ceiling.”

1.5 The welfare cap is not to be confused with the ‘benefit cap’ that limits the benefits a household can receive each week. The first welfare cap was set by the coalition government in March 2014. A new cap was set in July 2015 at the first budget of the current Parliament.

1.6 Benefits and tax credits can be more difficult to forecast than other departmental spending.3 They depend on statutory entitlements and vary according to the level of unemployment or the age and size of the population, and trends in the level and distribution of wages and housing costs. HM Treasury sets limits for departmental spending but has not previously done so for spending on benefits and tax credits. The welfare cap is a step towards exerting greater control over this area.

1.7 The welfare cap is set at £115 billion for 2016-17. Forecast spending is £120 billion.4 The Department for Work & Pensions (DWP) is responsible for £76 billion of spending within the cap which includes most housing and disability benefits (see Appendix Three for details). HM Revenue & Customs (HMRC) is responsible for administering £40 billion of the cap through tax credits, Child Benefit and tax-free child care. Social security spending in Northern Ireland accounts for a further £3 billion.

1.8 There are two significant exclusions from the welfare cap. It does not include the State Pension, (forecast expenditure of £92 billion in Great Britain in 2016-17), which represents the largest individual spending area. It also excludes Jobseeker’s Allowance, (forecast expenditure £2 billion – £3 billion annually in Great Britain).

A cap for forecasts not actual spending

1.9 The welfare cap is set in cash terms for the five years ahead. It is a limit on forecast spending rather than actual spending. The OBR updates its forecasts at each budget and autumn statement. It formally compares the forecasts with the welfare cap at each autumn statement to see whether forecast spending remains within the cap. There is no formal process for comparing actual spending with the cap. The cap is not intended to set a limit on actual spending but rather to encourage greater attention to the impacts on future spending when policy decisions are made.

1.10 The government uses the welfare cap as a financial management planning tool by taking a longer term view of how much it will spend on benefits and tax credits. This helps the government to understand, and transparently report on, the factors that influence social security spending and how its policy decisions will affect spending in the future.

Multiple departments involved in forecasts

1.11 The welfare cap reinforces the need for departments and the OBR to work together to finalise forecasts at budgets and autumn statements:

HM Treasury has overall responsibility for controlling spending and ensuring public finances are sustainable. It leads the processes for producing the budget and autumn statement. It also published the Charter for Budget Responsibility, which outlines the OBR’s roles and requirements.5 HM Treasury also commissions policy-costing notes from departments to support policy decisions in budgets and autumn statements.

DWP is accountable for the welfare cap and must lead a debate in the House of Commons if the cap is breached. It is also responsible for around two-thirds of the spending within the cap. Analysts within DWP submit five-year forecasts to OBR and policy costing notes to HM Treasury before budgets and autumn statements.

HMRC has no explicit responsibilities for the welfare cap. It provides forecasts and policy costings for tax credits, Child Benefit and tax-free childcare and has administrative (but not policy) responsibility for these areas of spending.6 It also provides operational advice to ministers if required.

The Northern Ireland Executive and the Department for Business, Innovation & Skills are also each responsible for small amounts of spending within the cap (see Appendix Three).

The OBR examines and reports on whether the public finances are sustainable. To do this it produces its own forecasts for public spending based on the forecasts provided by departments. The OBR’s Budget Responsibility Committee is responsible to Parliament for all judgements and assumptions in this ‘pre-measures’ forecast. The OBR also provides independent scrutiny and certification of the government’s costings of its new policies. At each autumn statement, the OBR formally assesses whether forecast spending (including the impact of new policy measures) remains within the welfare cap.

1.12 There is a statutory process for setting and managing the welfare cap which is set out in the Charter for Budget Responsibility (Figure 1).

1.13 Two processes come together at each budget and autumn statement to form the basis for the OBR’s independent assessment of the government’s spending plans: a forecast before the new policy measures; and the costs of the new policies themselves (Figure 2).

1.14 Forecasts and policy costings are discussed and challenged by OBR through a series of meetings with departments. The OBR reviews the underlying assumptions and methodologies for the forecasts, checks that these are consistent with its own forecasts for the economy, and has ultimate responsibility for final assumptions and judgements. Additionally, it certifies the departments’ policy costings as ‘reasonable and central estimates’, unless it decides that a government estimate is not ‘reasonable and central’. In these circumstances it will use its own forecast instead. The OBR finalises its overall forecast once the government has clarified the exact policy measures that it will announce in that budget or autumn statement (Figure 2).

1.15 Once a year, at the autumn statement, the OBR formally assesses whether the government has breached the welfare cap by comparing its forecasts for the next five years with the limits of the welfare cap. The OBR judges whether any changes in forecast spending are due to ‘forecast’ changes (for example inflation or increased caseload) or due to policy action (for example reversing the tax credit changes announced in the summer of 2015). The government sets a 2% margin of error for forecasting changes (approximately £2.3 billion each year). Forecast changes which are within this margin do not constitute a breach of the cap.

Breach at Autumn Statement 2015

1.16 At the Autumn Statement 2015 the OBR announced that the government was forecast to breach the terms of the cap for the first three years of the welfare cap period (Figure 3). The breach was due to the government’s decision to reverse two of the changes to tax credits that it had announced in the Summer Budget 2015. On 16 December 2015, the government proposed that the House of Commons should accept that the breach was justified for the three years. The motion was debated and carried. In the Budget 2016, the OBR’s forecasts show spending exceeding the cap in 2019-20 and 2020-21 as well. The OBR is not due to make an official assessment against the welfare cap until the Autumn Statement 2016.

1.17 The reversal of the tax credit measures also highlighted that the government had overestimated the savings that it would achieve from changes to tax credits. At the time of the Summer Budget 2015, the OBR certified savings of £4.4 billion in 2016-17 from changes to tax credits and Universal Credit. In the Autumn Statement 2015, the OBR published revised figures, which showed the estimated costs of reversing the tax credit (but not Universal Credit) measures were in fact £3.4 billion in 2016-17.7 At Budget 2016, the OBR reported that it had identified further errors in the revised figures provided in advance of the Autumn Statement 2015.8 These are discussed further in part two.

2 HM Treasury, Charter for Budget Responsibility, April 2011.

3 Departmental spending is classified as either within departmental expenditure limits (DEL) or as annually managed expenditure (AME). DEL covers areas such as employing staff, paying for buildings and buying equipment. The majority of benefit spending is classified as AME.

4 Based on latest estimates at Budget 2016.

5 HM Treasury, Charter for Budget Responsibility, October 2015.

6 Tax-free childcare is being introduced from early 2017.

7 Office for Budget Responsibility, Economic and fiscal outlook, November 2015.

8 Office for Budget Responsibility, Economic and fiscal outlook, March 2016.

Part Two

How the processes for managing the welfare cap are working

2.1 In this part we look at how well the government manages the welfare cap against five evaluative criteria (Figure 4).

Clear overall coverage

2.2 The welfare cap should improve the government’s understanding of spending and financial control. To achieve this the spending within the cap needs to be unambiguous, well understood and aligned to strategic objectives.

2.3 HM Treasury defines the scope of the welfare cap. It has published a list showing which benefits and credits are included in the cap (see Appendix Three). The Office for Budget Responsibility (OBR) publishes forecast spending for each benefit in its economic and fiscal outlook at each budget and autumn statement. This allows departments, politicians and taxpayers to assess the relative contribution of each benefit to forecast spending.

2.4 The welfare cap includes most benefit expenditure with two clear exceptions. It excludes the State Pension (£92 billion in 2016-17 in Great Britain). The government believes that pensions are better managed over a longer term than five years. It also excludes Jobseeker’s Allowance and other benefits for jobseekers such as Housing Benefit (and the corresponding payments within Universal Credit). The government argues that Jobseeker’s Allowance is highly dependent on the state of the economy and needs to fluctuate accordingly. There is a risk that the current scope of the cap may create an incentive to keep claimants on Jobseeker’s Allowance, instead of encouraging claimants into work and increasing entitlement to in-work benefits, or on to alternative out-of-work benefits, both of which are inside the cap. We have not seen any evidence that departments have assessed the likelihood or size of the risk, although all departments told us they believe this risk to be small and have no intention of using the cap in such a way.

2.5 HM Treasury must seek the approval of the House of Commons for any changes to the list of benefits within the welfare cap, the level of the cap or the margin. The exceptions are ‘classification changes’. Classification changes are changes that the OBR has certified as ‘fiscally neutral’. For example, in the Autumn Statement 2014, write-offs of overpaid benefits were reclassified as benefit expenditure and the welfare cap was increased by £300 million accordingly. The legislation sets out that the cap will always be adjusted when such neutral classification changes are made but does not set out whether the OBR or HM Treasury is responsible for making the adjustment.9

2.6 In some cases what counts as a classification change is an area for careful judgement. For example, in the Autumn Statement 2015, the government announced that it would change the way it funds local authorities for managing temporary accommodation. The funding (between £200 million and £260 million a year) was removed from DWP’s welfare cap spending and instead will be provided directly to local authorities by grants from the Department for Communities and Local Government’s departmental expenditure. This did not result in a reduction of the welfare cap. The OBR agreed with HM Treasury in this case that this was not a classification change.

2.7 In our view, the cap’s effectiveness as a financial control would be enhanced by maximum transparency in the treatment of classification changes. This would be easier if either HM Treasury or OBR were clearly responsible for adjusting the cap when such changes are made, and if they publish more information about the reasons for these decisions.

Well-established forecasts but reliance on some individuals

2.8 Forecasts form the basis of assessments against the welfare cap and subsequent spending decisions. While forecasts can never eliminate uncertainty, they should build on clear processes, good quality data, realistic assumptions, reliable models and be appropriately quality assured.10

2.9 Forecasting for the welfare cap is an extension of DWP’s well-established medium-term forecasting process:

DWP updates its in-year forecasts monthly and its analysts and forecasters explain any differences in spending to date and the forecast for the year end against the Department’s main social security spending estimate.

The forecasts are based on data and information from several established sources, including: national statistics; internal caseload information; latest assumptions about implementation of welfare reforms; and also data from local authorities.

Senior staff review the work of junior staff, and go through the forecasts with policy owners, who sign off the forecast assumptions as reasonable. This suggests DWP has adopted the good practice principles from the Macpherson review.

At budgets and autumn statements DWP prepares forecasts on behalf of the OBR. It has an extra process where the head of financial analysis and the chief economist, (who together lead its policy costings and forecasting scrutiny committee), review all benefit forecasts, and draw them together to check that they are reasonable. The judgements underpinning the forecasts are the responsibility of the OBR.

2.10 DWP has correctly identified that its integrated benefits forecasting model (known as INFORM) has not kept pace with the rate of welfare reform and was producing forecasts for some benefits that did not appear plausible. As a result, DWP now bases its forecasts on individual, simpler models for each benefit (Figure 5).

2.11 DWP’s move to using individual models for each benefit may have increased the risk that the interaction between benefits may not be captured fully in the forecasts.11 It has, however, reduced the risks that INFORM presented due to not being suitable for the current high rate of welfare reform and its base data being drawn from a period which is considered to be unrepresentative of the longer-term situation. DWP has agreed in principle to invest in new modelling software to help address these issues and is pursuing this option.

2.12 DWP could improve its forecasting arrangements as its models for individual benefits are not regularly reviewed by either peers within the department or by external experts. The models were developed by forecasters who also maintain the models and use them to monitor spending in their area. These same forecasters review their own forecasts, by checking whether spending is reasonable compared with previous periods and analysing any variances. Although occasional peer review takes place, and external advice has been sought, DWP could extend and formalise such arrangements beyond the initial production of models and its own analysts. However, DWP is aware of the risks that inadequate models present and reflects this on its Analytical Community risk register.

2.13 A further risk to DWP’s forecasts comes from relying too much on a small number of individuals. DWP depends on the expertise of individual staff on how benefits interact. Its dependence on individuals is exacerbated by a lack of documentation to support handovers to other staff. DWP’s analysts told us that they have plans to document the models for Housing Benefit, Employment and Support Allowance and Personal Independence Payment but have not yet done so.

2.14 These issues are further heightened as some analysts rely on manual calculations to adjust for interactions between benefits. Off-model calculations can help to ensure that models are kept appropriately up to date, but it reduces transparency. In the March 2016 Economic and fiscal outlook, the OBR commented that it had significant concerns over the Universal Credit model because it had not been able to keep pace with recent policy changes and had become even less transparent due to the increasing use of off-model adjustments.12

2.15 HMRC has well-documented integrated forecasting models that are calibrated using up-to-date outturn data. Forecasts are reviewed by a manager and the relevant deputy director within HMRC’s Knowledge, Analysis and Intelligence directorate. Its guidance also recommends a biannual review of business critical models. HMRC also relies on a few individuals to produce its welfare cap forecasts, and some manual adjustments. However, this is less of an issue than it is at DWP because HMRC only has a small number of models and because it has more complete documentation of its models and of its forecasting processes.13

2.16 There are many reasons why spending forecasts change between fiscal events including revised economic assumptions and new data. However, forecasts have also had to be revised due to mistakes. Risks of mistakes increase when there is insufficient time to run new policy measures through models; and those policy measures have knock-on effects on other benefits. In the Summer Budget 2015 there was very little time: as a result HMRC did not have enough time to arrange for detailed modelling by DWP of the interactions between the proposed tax credit reforms and Housing Benefit. Instead it used a ‘ready reckoner’, which was designed to deal with small changes to the system, rather than significant reforms such as the proposed reduction to the working tax credit threshold. This resulted in an underestimate of the impact of tax credit changes.

2.17 In the March 2016 Economic and fiscal outlook, the OBR referenced a further error in the November 2015 costings of the effect on DWP’s benefits spending of reversing July’s tax credit measures. It commented that this was the second successive scorecard containing errors in welfare spending measures that affect both HMRC and DWP administered benefits. It cited a combination of insufficient time for scrutiny and HMRC’s analysts not being permitted to discuss the costings with their DWP counterparts. The OBR has stated that at future fiscal events, if similar circumstances were to arise it would  be unlikely to certify the packages of measures as ‘reasonable and central’.

Controls over spending on benefits and tax credits are evolving

2.18 The welfare cap supports tighter control over spending by formalising scrutiny of forecasts and setting out a process for dealing with breaches. The welfare cap complements existing controls over public spending by:

Setting longer-term fiscal objectives. Because spending on benefits and tax credits depends on statutory entitlements there is a limit to how much departments can control spending without changing legislation. In the past departments had forecasts of benefit spending but these did not act as direct controls over spending.

Increasing transparency of changes to forecasts. By setting out the different types of changes the OBR’s assessment supports more informed discussion of potential responses.

Establishing a process for dealing with changes in forecasts. While departments have always monitored forecasts and considered policy responses the welfare cap sets out a more consistent and prescribed process. HM Treasury also provides the OBR with formal notes on interactions between key benefits.

2.19 Transparency over spending on benefits and tax credits has increased since the introduction of the OBR in 2010. The welfare cap has helped improve transparency further. Greater attention is paid across government and at the OBR to spending that falls within the cap. The OBR’s reports set a clear baseline against which future spending forecasts can be compared and distinguish more clearly between different factors affecting forecasts.14 The OBR and HM Treasury have told us that the forecasts are improving as a result and that independent forecasts reduce the risk of optimism bias in departments.

2.20 At the Autumn Statement 2015, the government decided to reverse some of its reforms to tax credits. This resulted in an increase in forecast spending on benefits, which meant that it breached the limits in the cap for three years: 2016-17, 2017-18 and 2018-19. The House of Commons debated these breaches as required by the Charter for Budget Responsibility.15

2.21 The OBR is responsible for deciding why welfare cap forecasts have changed. There are two main possibilities:

A forecast change, for example inflation that is higher than forecast. The cap would not automatically increase to reflect forecast changes as it incorporates a 2% margin to accommodate forecast changes.

A policy change, for example reducing the household benefit cap to £20,000.16 The cap would not automatically reduce to reflect policy changes.

2.22 Departments and HM Treasury continue to have discussions with ministers about possible policy actions leading up to budgets and autumn statements. It is unclear to us whether the process and supporting analysis have been affected by the introduction of the welfare cap although we have seen evidence that spending discussions within DWP are considering the impacts of the cap in the longer term.

2.23 There are areas where the processes related to the welfare cap could be improved to make it a more effective control on spending forecasts:

The departments and HM Treasury could extend their processes for overseeing the welfare cap as a whole. Currently HMRC and DWP develop some sensitive policy costings independently of each other in the run up to budgets and autumn statements. Officials from HM Treasury bring together the forecasts and policy costings from DWP and HMRC (together with those from Northern Ireland and the Department for Business, Innovation & Skills) on a spreadsheet, and check that they are reasonable, and advise their ministers of the overall position.

Spending decisions should clearly distinguish between those that reduce spending overall and those that move spending in and out of the cap. For example, DWP has extended the period that someone who is ill can stay on Jobseeker’s Allowance (outside the cap) before they must claim Employment Support Allowance (inside the cap) instead.

Decisions about whether a forecast has changed due to a policy decision or for some other reason can be very finely balanced. This can lead to apparent inconsistency. For example changes to the timetables for implementing welfare reform affect the spending forecasts. The OBR judged that deferred roll-out of Universal Credit was a policy change while it decided that the delay in implementing Personal Independence Payment was a forecast change, as the delay was due to more people being eligible than was originally forecast.

Complicated reporting environment with time pressures

2.24 Having forecast benefit spending, the departments involved in the welfare cap need to submit those forecasts to the OBR. Once the OBR has decided its forecast it assesses forecast benefit and tax credit spending against the cap and outlines why they have changed since the last assessment. The departments need to ensure that: they report and aggregate forecasts consistently; their operational assumptions are realistic; they explain changes clearly; and they clearly communicate risks or limitations.

2.25 The budget responsibility process, of which the welfare cap is part, has strengthened government reporting by introducing independent scrutiny of forecasts by the OBR. The OBR produces the final authoritative forecasts of the public finances against which government is held to account at each budget and autumn statement. This process has encouraged departments to produce robust forecasts that stand up to the challenge provided by the OBR.

2.26 The OBR assesses the uncertainty of the costings for each new policy announced at budgets and autumn statements, which increases transparency. But as the OBR’s assessments are published after policy decisions are made, they cannot replace departments’ own assessments of uncertainty in internal reports and advice to ministers. DWP’s forecasts do not formally quantify uncertainty. DWP has had well-established and reliable forecasts over time but recently its in-year forecasts increased by £0.5 billion (0.6%) between the Summer Budget 2015 and March 2016, showing short-term volatility of its forecasts. The scale of welfare reform has also made DWP’s forecasts less certain.

2.27 There are three key areas of reporting on the welfare cap that could be improved:

Greater access to expertise. The OBR has felt unable to call on all of the official expertise it needs. It has attributed some of the errors in the Summer Budget 2015 forecasts to its inability to explore fully the impact of the tax credit measures with expert officials from both HMRC and DWP in the same room at the same time.17 HM Treasury told us that it has responded and increased the OBR’s access to officials for subsequent fiscal events, although this was raised as an issue again by the OBR in relation to the Autumn Statement 2015.

More time for considering policies. The OBR did not have the time it needed to complete its review of the departments’ estimates of the costs of all the policies in the Summer Budget 2015. This was because the government did not provide it with details for a large proportion of significant policy measures until just before the agreed deadlines.18

Better transparency and consistency in reporting costing notes. In Autumn Statement 2015 HM Treasury changed the ordering of its scorecard so that the effects of the delayed Universal Credit migration were considered before the reversal of the tax credits measures. Since Universal Credit was relatively more generous before the tax credit reversal, the effect of this was that Universal Credit delays were shown to save money in the short term. If HM Treasury had used its regular approach to ordering the impacts of policy changes it would have demonstrated that the policy to delay Universal Credit migration would cost more and the cost of the tax credit reversal less. Overall the net fiscal impact of the two measures would be the same. The numbers in the costing were the same as those certified by the OBR so it was the reordering of the measures that was potentially misleading.19

2.28 It is particularly important that the OBR gets the details relating to welfare cap policy measures in good time for two reasons: first, the OBR has to judge whether changes to the welfare cap forecasts are due to policy decisions or forecasting changes. This is the responsibility of the OBR’s three-person Budget Responsibility Committee, so it cannot be done more quickly by allocating more people to the task. Second, this is a complex area with many interdependent benefits and credits. It takes time to fully understand and assure the combined effect of the policy measures that the government decides to introduce.

Cross-departmental governance is still evolving

2.29 Governance is important for developing forecasts, reporting and then supporting decisions. Roles, responsibilities and accountabilities for managing the welfare cap need to be clearly defined, agreed and understood to support effective decision-making. DWP has established internal governance structures to manage its forecasts of its own welfare cap spending. HMRC relies on HM Treasury for policy decisions about spending (Figure 6).

2.30 Internal governance arrangements are well-established at DWP. It has reviewed and revised its internal governance arrangements since the Summer Budget 2015, and has replaced its welfare cap management committee with an AME (annually managed expenditure) board with new terms of reference.20 The AME Board focuses on DWP’s own spending forecasts rather than the welfare cap as a whole.

2.31 Cross-departmental governance arrangements are less clear than DWP’s internal governance arrangements. This is partly because the welfare cap is managed alongside and as part of HM Treasury’s usual process for budgets and autumn statements. The cap is managed through policy decisions which we fully recognise are a matter for ministers. But departments need clearer processes for handling escalation and supporting decision-making relating to the cap as a whole. The welfare cap forecasts are particularly challenging to manage because the spending in the cap straddles HMRC and DWP. DWP is accountable for the welfare cap, although it has no direct control over one third of the spending within it.

2.32 At present, information about the welfare cap forecasts is shared across departments informally at working level and via HM Treasury’s AME Risk Group. The AME Risk Group is a cross-departmental forum hosted by HM Treasury where officials from all three departments discuss potential risks to the welfare cap throughout the year; maintain a risk register; monitor the evolution of the welfare cap forecasts as they progress through the budget and autumn statement processes; and refer any concerns they may have to HM Treasury’s more senior Fiscal Risks Group.

2.33 Risks to cross-departmental governance persist. This is because HM Treasury’s AME Risk Group is the only cross-departmental structure but its terms of reference do not have sufficient clarity with respect to membership, roles, responsibilities, accountabilities and criteria for escalating concerns. Cross-departmental governance of the welfare cap as a whole has operated effectively except for the errors at the Summer Budget 2015 (which HM Treasury then used to set the welfare cap) and at the Autumn Statement 2015. These errors were not identified by OBR until after the relevant fiscal events.

2.34 Expert forecasters from DWP and HMRC work together to support the advice given to ministers and to respond to the OBR’s queries and challenges. But there are no formal arrangements to enable this collaboration to override traditional arrangements for keeping sensitive policy measures confidential within individual departments. This increases the risk of mistakes in the final stages of preparation of budgets and autumn statements when decisions are made very quickly and when the free flow of information through informal channels is prevented to avoid leaks.

2.35 The official assessment of the cap at Autumn Statement 2015 showed forecasts were only just inside the welfare cap plus margin for 2019-20 and 2020-21. At the Budget 2016, the OBR’s spending forecasts showed the forecasts were in excess of the cap plus margin for these years as well. The next official assessment against the welfare cap will be at the Autumn Statement 2016. It will be important that departments are able to make decisions that balance value for money with controlling benefit spending. HM Treasury told us that it was clear that there must be another debate in the House of Commons if the OBR were to decide the cap has been breached in any year. The only exception would be if the breach were identical (in size and cause) to the specific breach which was debated in December 2015. There is no minimum size for a breach.

9 HM Treasury, Charter for Budget Responsibility, Summer Budget 2015 Update, July 2015.

10 In October 2012, the government commissioned a review of the quality assurance of analytical models across government (the Macpherson review). The government expects departments to use the best practice principles outlined in the review and in HM Treasury, Aqua Book: guidance on producing quality analysis for government, March 2015. We have used the high level principles set out in these documents in our assessment of quality of forecasts and models for the welfare cap.

11 ‘Interaction’ is when receiving one benefit makes getting another one more or less likely, or changes the amount payable.

12 Office for Budget Responsibility, Economic and fiscal outlook, March 2016.

13 Personal tax credits, Child Benefit and tax-free childcare (which is not yet operational).

14 The economic and fiscal outlook produced at each budget and autumn statement.

15 Hansard HC, 16 December 2015, vol. 603, cols 1633-1650.

16 In London £23,000.

17 There is a full explanation of these forecast changes in Tax Credit Costings, Office for Budget Responsibility, November 2015.

18 Office for Budget Responsibility, Economic and fiscal outlook, July 2015, p. 183.

19 Letter from Robert Chote to the Rt Hon Frank Field MP on Universal Credit and policy costing, 4 February 2016. Available at: http://budgetresponsibility.org.uk/docs/dlm_uploads/RC-to-FF.pdf

20 Departmental spending is classified as either within departmental expenditure limits (DEL) or as annually managed expenditure (AME). DEL covers areas such as employing staff, paying for buildings and buying equipment. The majority of benefit spending is classified as AME.

Appendix One

Our audit approach

1 This report examined the government’s approach to forecasting and managing the welfare cap. We do not evaluate whether the cap is an effective or desirable policy, nor do we assess specific options for reducing spending when breaches are identified.

2 We applied our analytical framework for assessing spending control mechanisms. In our past work we have highlighted the importance of five criteria for frameworks to control public spending:

clear coverage;

reliable forecasts;

robust controls;

prompt reporting; and

clear governance.

3 Our audit approach is summarised in Figure 7. Our evidence base is described in Appendix Two.

Appendix Two

Our evidence base

1 Our independent conclusions on the government’s approach to managing and forecasting the welfare cap are based on our analysis of evidence collected between July 2015 and March 2016. This includes the Autumn Statement 2015, when the Office for Budget Responsibility (OBR) declared a breach of the welfare cap for the first time.

2 Our analytical framework has been used previously in our report on the Department for Energy & Climate Change’s Levy Control Framework.21 We have also used our toolkit for assessing government models.

3 To assess whether the coverage of the welfare cap had a clear rationale and fitted with the government’s main financial control regime, and to understand how the cap works:

We reviewed published documentation and legislation on the scope of the welfare cap, available from HM Treasury and the OBR, including budget and autumn statements, the Charter for Budget Responsibility, the OBR’s economic and fiscal outlooks and its welfare trends reports.

We spoke to officials within the Department for Work & Pensions (DWP) and HM Treasury to understand the rationale for inclusion or exclusion of benefits in the cap.

4 To assess processes for forecasting and reporting against the welfare cap:

We met with officials and reviewed documents at DWP and HM Revenue & Customs (HMRC) to understand how the internal forecasting and modelling processes were organised and how these fed into the budget and autumn statements.

We assessed how well the processes and documentation within DWP and HMRC measured against the Macpherson review of quality assurance of government analytical models and the government’s Aqua Book.22

We also met officials at the OBR and reviewed its published documents to understand how all forecasts are brought together as part of the standard budget and autumn statements processes.

5 We carried out some interviews jointly with DWP’s internal audit team who were undertaking a review of internal processes for the welfare cap around the same time. We have had reference to their report on the welfare cap while finalising our own. We would like to thank the internal audit team for their assistance in undertaking interviews and providing documentation.

6 To assess whether departments have effective controls for costs of, and outcomes from, policy decisions:

We interviewed officials within HM Treasury and the OBR to understand how decisions are made with regard to policy changes. We do not assess the merits of such policy decisions.

We reviewed documentation relating to policy decisions and the effects these have on the welfare cap.

7 To assess whether governance arrangements were appropriate and engaged skilled and empowered individuals:

We reviewed documentation regarding roles and responsibilities for the welfare cap, both within and across departments.

We discussed roles and responsibilities with officials at the OBR and HM Treasury, to understand how oversight committees were organised and their terms of reference.

8 To understand the errors in the Summer Budget 2015 forecasts and any lessons learnt:

We reviewed relevant documentation.

We discussed the causes with officials at DWP, HMRC, HM Treasury and the OBR.

21 Comptroller and Auditor General, The Levy Control Framework, Session 2013-14, HC 815, National Audit Office, November 2013.

22 HM Treasury, Aqua Book: guidance on producing quality analysis for government, March 2015.

Appendix Three

What is in the welfare cap

Appendix Four

The Office for Budget Responsibility’s process for developing its forecasts

Design and Production by NAO Communications
DP Ref: 11067-001