NAO blog

The problem’s identified; the policy-fix is designed; but then the public doesn’t respond as expected. It’s an old story, and not unique to the public sector. But the consequences of inadequate consumer understanding and lack of early-warning signs can be costly – as our recent report, Green Deal and Energy Company Obligation, shows.

For these schemes to succeed in improving the energy efficiency of homes depended, as with many policies, on people reacting as government predicted – and they didn’t. Moreover, there weren’t adequate monitoring measures to give early warning of trouble.

Many of our studies have shown the importance of understanding, testing, monitoring and evaluating consumer behaviours when designing and implementing policies. This is of particular importance for issues requiring a degree of citizen behaviour change such as moving to digital delivering of public services, discouraging tax avoidance, increasing savings, reducing carbon emissions, addressing offender behaviour and reducing lifestyle diseases. (There are links to some relevant reports at the end of this post.)

There are seven key questions that organisations should ask themselves when considering consumer behaviour:

1. Targeting: Are we clear about what our approach is trying to achieve?
2. Analysis: Do we have an evidence-based understanding of behavioural insights?
3. Planning: Have we integrated our knowledge of behaviour into policy design?
4. Testing: Have we tested our approach to changing behaviour?
5. Implementation: Have we implemented the approach effectively?
6. Measurement: Are we able to measuring the results?
7. Evaluation and adaptation: Have we reviewed the results, and made changes as necessary? Is there a plan for on going monitoring?

In the case of the Green Deal, we found that the Department of Energy & Climate Change (DECC, as it was) came up short in a number of these areas.

The Green Deal and ECO scheme: some background

People are not used to paying for energy efficiency measures themselves – the historic trend for government to rely on supplier obligations – like Energy Company Obligation (ECO) – to improve homes’ energy efficiency means that households often received measures for free. But DECC did little to test whether the Green Deal would persuade people of the value of paying for energy efficiency measures. It did not try out its final design with consumers, and relied primarily on a stated-preference survey as assurance that people would take it up after roll-out. Even where there was evidence of what worked, such as low interest rates on loans in similar locally-run schemes, DECC did not incorporate this into its design.

Common measures for improving energy efficiency in semi-detached homes

Consumer understanding wasn’t applied to the scheme’s promotion, either. DECC’s early marketing of the scheme focused on the financial benefits for households, although consumer research showed people were interested in other benefits, such as a warmer home.

Uptake of Green Deal loans was low from the outset and by July 2015 DECC effectively brought the scheme to a halt. By this time, DECC had spent £240 million on the scheme, and with only 14,000 loans taken out; that’s £17,000 spent per loan. And it now thinks that the scheme has not saved any additional CO₂. We therefore concluded that DECC did not achieve value for money with the Green Deal.

ECO was more successful, with suppliers meeting their obligations for CO₂ savings and reducing bills. The scheme has been responsible for 97% of the measures we considered in our report. But we think it would have cost suppliers – and therefore bill-payers – less if DECC had not designed it to support the Green Deal.

Wider lessons can also be drawn from the Green Deal and ECO scheme about designing and implementing major projects. DECC did not set clear success criteria for the Green Deal, meaning it could not compare early performance to its expectations, and therefore couldn’t identify early warning signs that things were off track. It does not have monitoring information that would let it measure progress towards each of its objectives, meaning it cannot tell whether the schemes have actually achieved what it wanted. We also think that a lack of long-term vision for how to improve household energy efficiency and the inconsistency in DECC’s approach could lead to increased costs later on.

DECC has an important challenge to reduce carbon emissions by improving the energy efficiency of existing homes in the UK which, like with many of its policies, depends on people reacting in a certain way. As DECC develops approaches to this and its other activities, such as the roll-out of Smart Meters, we encourage it – like we do all public sector bodies – to focus on understanding consumers’ behaviours and continuing to monitor it as external circumstances change.

I’ve provided some links below to other NAO reports addressing aspects of behaviour change. Please contact us if you would like to discuss any aspect of our work.

Some NAO reports covering consumer behaviour issues:

• Digital Britain 2: Putting users at the heart of government’s digital services
• Tackling tax credits error and fraud
• Tax reliefs
• Financial services mis-selling: regulation and redress
• Update on preparations for Smart Metering
• The government’s long-term plans to deliver secure, low carbon and affordable electricity
• The Help to Buy equity loan scheme