Evaluation of regulatory impact assessments 2006-07

Executive Summary

National Audit Office Value for Money Report

1. The aim of Regulatory Impact Assessments (RIAs) is to assess the need for, and impact of, proposed regulations and amendments to existing regulations. They are a tool to help policy makers understand the consequences of possible Government regulation. RIAs are required for all forms of government intervention that impose, or reduce, costs on businesses, the third sector or the public sector.[Footnote 1] They are a central element of the Governments objective to regulate only where necessary and reduce the burden of regulation on business and the third sector.
    
   
2. The number of RIAs produced by government departments has increased steadily since 2003 (Figure 1). The National Audit Office (NAO) has reviewed the quality of RIAs for the last three years. Last year we concluded that the standard was disappointing and RIAs had not yet altered the way that Government thinks about regulation (see Appendix 2). This report evaluates the quality of RIAs produced by the Department of Health (DH) and the Department for Communities and Local Government (CLG), and considers how these departments are seeking to raise standards and improve their use. We also consider the extent to which RIAs can deliver their aims within the realities of policy making.
   
   Figure 1 ("Number of published RIAs 2003-06") is unavailable in this version of the executive summary.
   
   

   Overall conclusion

   
   
3. RIAs should be a cornerstone of evidence-based policy making but our results indicate that they were not always being used effectively. The majority of RIAs were competent, with fewer cases of poor quality analysis, although there were continued weaknesses in the quality of economic analysis and insufficient consideration of the impact of proposed changes. All too often, however, RIAs were not an integral part of the policy making process as they were not used to inform and facilitate all stages of policy formation from initial development through to implementation and review.
   
   

   Key findings

   
   

   The quality of Regulatory Impact Assessments at the Department of Health and the Department for Communities and Local Government
   
   

4. The results of our assessment of this years sample of RIAs show that the quality was mixed (Figure 2).[Footnote 2] The majority of RIAs were assessed as competent and consultation was again the strongest area in the RIAs assessed. The assessment of costs and benefits was, however, again the weakest area, with deficiencies in the standard of the evidence base and limited use of recognised economic appraisal methods. There was also insufficient consideration of the impact of regulations following implementation. Only one quarter of RIAs considered compliance and enforcement issues fully; and under half contained sufficient details of how the new legislation would be monitored and evaluated.
   
   Figure 2 ("NAO analysis of RIAs") is unavailable in this version of the executive summary. 

   
   The role of Regulatory Impact Assessments in the policy making process
   
   

5. The quality of RIAs, and the extent to which they influence policy decisions, must be viewed within the wider context of the realities of policy making. RIAs were often not commissioned or used early enough in policy formation to really challenge the need for new regulations. The do nothing was not considered as a viable option in 18 of the 19 final RIAs we examined.
    
6. RIAs were not widely used in the Parliamentary process. RIAs were only occasionally used by Parliamentary Committees and to inform Parliamentary debate. A lack of awareness and Committee clerks perceptions of weaknesses in the quality of analysis prevented RIAs from playing a greater role in informing the Parliamentary process. Our analysis showed that eight of the 19 RIAs included predominantly good quality analysis and would, potentially, have provided valuable information.[Footnote 3]
    
7. There continues to be an unstructured and ad hoc approach to post-implementation review across all departments. The Department of Health and the Department for Communities and Local Government have begun to develop a more systematic approach to evaluating the impact of policy changes, but have not yet begun a rolling programme of reviews. Our census of departments highlighted resource constraints and time pressure as the main reasons why reviews were not more widespread.
   
   

   Improving the use and quality of RIAs across Government

   
   
8. The relationship between the Better Regulation Executive (BRE) and departments has been strengthened. There have been tensions with the BRE, particularly around the adequacy of communication, but changes to the BREs role have helped clarify respective responsibilities and strengthen working relationships.
    
9. Revisions to the RIA guidance created uncertainty during the period of transition. Between July 2006 and April 2007 the BRE undertook a public consultation and revised the Impact Assessment Guidance. The new requirements and a standard template aimed at improving the presentation of results and encouraging impact assessment earlier in the policy making process will be phased in from May 2007. There was a lack of clarity on the future direction and coverage of RIAs during this transitional phase.
    
10. The adequacy of governance arrangements to encourage high quality RIAs is varied. The departments in our sample are strengthening the scrutiny of RIAs. The Department of Health has taken a number of positive steps to integrate the principles of better regulation into its approach to policy making. The Department for Communities and Local Government is also strengthening procedures to ensure timely expert input into RIAs and robust challenge on the evidence base.
    
    

    Recommendations

    
    
11. RIAs increase Government accountability as they improve the transparency of policy decisions and, if used well, can encourage evidence-based policy making that helps deliver more effective regulation. Achieving this will require action to improve the quality and influence of RIAs and, in addition to providing clarification on the content of RIAs, our recommendations encourage departments to review the adequacy of their processes and the support they provide to policy officials. This years assessment is based on the Department of Health and the Department for Communities and Local Government but the recommendations are applicable to all departments.
     

Improving the content of RIAs

1. Departments should ensure their guidance, training and procedures emphasise the need for high quality analysis and early engagement with departmental economists. In doing so, they should promote the importance of quantification and a renewed emphasis on analytical techniques.
    
2. RIAs should more explicitly consider the impacts of legislation when it comes into force. There should be a stronger emphasis on compliance and enforcement issues, including the distribution of compliance and associated cost estimates of the proposed enforcement regime.
    
3. Departments need to ensure that RIAs contain explicit statements on how and when post-implementation reviews will be conducted. In addition, departments should develop a systematic programme of reviews of regulations and assign responsibility for reporting results to Senior Management.
   
   

   Strengthening processes

   
   

4. Departments should give consideration to potential regulatory implications at the early stages of policy formation, including the establishment of PSA targets. This will assist in the achievement of the Government’s Better Regulation agenda – to regulate only where necessary and provide early challenge to regulatory proposals.
    
5. Departments should strengthen processes and provide adequate incentives for the development of high quality RIAs by:
   • integrating expertise into RIA development at appropriate stages of the process; 
   • strengthening scrutiny processes – for example, peer review – to provide robust challenge on RIAs throughout the process. Challenge panels would be strengthened by including external stakeholders; 
   • ensuring principles of better regulation and impact assessment are integrated into a consistent, department-wide approach to policy making; and 
   • making much more systematic use of post-implementation reviews and evaluations to improve the process of impact assessment. 
   
    
6. The BRE and departments should promote flexibility and proportionality when applying the new RIA guidance, in order to take account of the varied and complex nature of policy development.
    
7. To encourage consistent and robust scrutiny, the BRE and departments should work together to identify the key measures of good quality RIAs. The tests should be based on the requirements of the new guidance and incorporate the evaluative criteria applied by the NAO.
    

   Providing support to Policy Makers

8. Departments need to change the culture of regulation amongst policy officials, by creating stronger incentives and provide appropriate support through a combination of:
   • on-going senior management commitment which provides proactive rather than passive support, and promotes the importance of better regulation. Departments should also ensure better regulation initiatives are included in departmental publications; for example, business plans
      
   • greater awareness and an improved understanding of the requirements of RIAs – by communicating with all levels and parts of the department. Policy officials should ‘buy-in’ to the benefits of using RIAs and understand how they can assist policy development; and
      
   • a targeted package of training and support which integrates RIAs into mainstream policy development.
      

1.  [back from footnote 1] RIAs need to be completed for primary and secondary legislation, codes of practice and guidance. RIAs are required for proposals affecting the public sector if expected costs are over 5 million.
2.  [back from footnote 2] We assessed a random sample of 19 RIAs: 10 from the Department for Communities and Local Government and nine from the Department of Health.
3.  [back from footnote 3] Eight of the 19 RIAs contained at least four green assessments out of a possible six, and no red assessments.