Report

by the Comptroller
and Auditor General

Home Office, Ministry of Justice, Attorney General’s Office, covering:
HM Inspectorate of Constabulary, The Independent Chief Inspector of Borders and Immigration, HM Inspectorate of Prisons, HM Inspectorate of Probation and HM Crown Prosecution Service Inspectorate

Inspection: A comparative study

HC 1030 Session 2014-15 13 February 2015

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO, which employs some 820 employees. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund have used their resources efficiently, effectively, and with economy. Our studies evaluate the value for money of public spending, nationally and locally. Our recommendations and reports on good practice help government improve public services, and our work led to audited savings of £1.1 billion in 2013.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 11 February 2015

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

10 February 2015

The government and Parliament see inspection as a tool to give assurance, enable accountability and improve public services. There are five inspectorates in home affairs and justice: HM Inspectorate of Constabulary; The Independent Chief Inspector of Borders and Immigration; HM Inspectorate of Prisons; HM Inspectorate of Probation; and HM Crown Prosecution Service Inspectorate. Around £35 million a year is spent on these inspectorates, with around £20 billion a year spent on the sectors they inspect.

© National Audit Office 2015

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.gsi.gov.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

10631 02/15 NAO

The National Audit Office study team consisted of:
Andrew Batey, Julie Bouchard, Chris Groom, Howard Revill, and Shivani Shah, under the direction of Louise Bladen.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key facts

Summary

1 Around £20 billion of public money is spent on the home affairs and justice sectors each year. Parliament, government and the public have an appetite for independent, robust information about how well this money is used, the performance of organisations delivering services, and the experience of people using these services, sharpened by the context of resource constraints and rising expectations.

2 Through public reporting and scrutiny, effective inspection can provide benefits of:

independent assurance, by periodic, objective, targeted checks on public functions;

valuable and often unique insights into how public services are run;

identifying where service performance is at risk or failing;

transparency;

promoting public confidence in safety and quality of services provided;

identifying opportunities for efficiencies and service improvements; and

preventing ill-treatment of vulnerable people.

Findings from independent inspections are therefore a significant knowledge asset with substantial potential benefits for citizens and taxpayers.

3 Inspectorate focus varies, with legislation setting out each inspectorate’s different and distinctive remit. Broadly, inspectorates focus on performance improvement and transparency. For example, Her Majesty’s Inspectorate of Prisons’ human rights focus provides a window into the treatment and conditions in which people are detained and, in doing so, aims to prevent their ill-treatment. There are similarities between inspectorates’ roles and performance auditors (such as the National Audit Office) and regulators (such as the Care Quality Commission). However, these activities are distinct and complementary. Importantly, inspectorates only have the power of their voice to draw attention to what they find, informing accountability through objective information, insight and independent judgement, whereas regulators, for instance, may also apply powers of intervention, prescription and direction in the sectors they regulate.

4 The five home affairs and justice inspectorates that are the focus of this report inspect functions which place restriction on the freedom of individuals. Therefore, inspection is important and prominent, given the need for transparency and accountability. The inspectorates have a combined annual spend of around £35 million, with £142 million spent over the past five years. In order of 2014-15 budgets, these are:

Her Majesty’s Inspectorate of Constabulary (HMIC, £21.4 million);

Her Majesty’s Inspectorate of Prisons (HMI Prisons, £4.5 million);

Her Majesty’s Inspectorate of Probation (HMI Probation, £3.4 million);

Her Majesty’s Crown Prosecution Service Inspectorate (HMCPSI, £3 million); and,

The Independent Chief Inspector of Borders and Immigration (ICI Borders and Immigration, £2.4 million).

5 Inspectorates have existed for a long time, with the first Inspector of Constabulary appointed in 1856. Most recently, the Home Secretary appointed the first Chief Inspector of Borders and Immigration in 2007. Ministers appoint Chief Inspectors, usually after a consultation process. Inspectorate staff have various backgrounds, with inspectorates aiming to balance their needs for technical insight, sector knowledge, and inspection skills.

6 The agencies and organisations within the criminal justice system have evolved differently in their history, approach and governance structures. This complex system has shaped how inspectorates approach inspection, individually and collectively. Patterns and trends in inspection approach have evolved. For example there has been a convergence towards four-grade ratings scales when inspecting single bodies. Single establishment inspections (83% of reports) have been complemented by thematic inspections (17% of reports) looking at services end-to-end and across institutions. There has been longstanding cooperation between the inspectorates. Under the Criminal Justice Joint Inspection (CJJI) regime, four inspectorates (excluding ICI Borders and Immigration) work to a rolling two-year programme. Considerable joint inspection takes place outside this framework, including, for example input from Ofsted and the Care Quality Commission to both prison and probation inspections.

7 In 2003, the Cabinet Office set out ten inspection principles, but inspection has received little central attention in recent years. We used the principles as a starting point for our examination. We considered developments for inspection regimes arising from, for instance, the Optional Protocol to the United Nations Convention Against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment (OPCAT), Ofsted’s work, and the Francis reports on failings at the mid-Staffordshire NHS Trust. These called for more emphasis on inspection rather than self-assessment, and drew attention to the importance of not ignoring bodies that have failed to appear on the radar of concern.

Scope of the report

8 The extent to which inspection is effective and provides public value and impact is affected by factors such as how the inspectorates’ role is set up and determined, the nature and practice of inspection regimes, and constraints in the environment within which inspectorates operate. We consider it is relevant and timely to report on inspection, given substantial changes in public service configuration and delivery (including contracting out), pressures on resources, and developments in accountability in this sector.

9 Across the five criminal justice inspectorates, we focus on three areas of comparison that are key to the optimal use of the knowledge asset which inspection findings generate. First we consider the most important aspect which is:

whether the impact of inspection is maximised (Part Two);

which is in turn influenced by:

whether the strategic framework for inspection supports the work of inspectorates (Part Three); and

whether inspectorates carry out their work effectively (Part Four).

Key findings

Maximising the impact of inspection

10 The act of inspection has a direct cultural impact on sector performance. Inspected bodies pointed to the burden of inspection. However, they recognised the profound cultural effect on behaviour of the act of inspection and knowing the inspector would, or could, visit. Having inspection standards, such as HMI Prisons has, that the sector knows, has an impact, as often managers will aim to meet these standards before they are inspected. As well as their reports, inspectorates also use several routes to influence sector improvement, such as dialogue with ministers, promoting their work at conferences and to Parliament. They may also be active in oversight groups assessing compliance with inspection reports and policy more generally (paragraphs 2.2 and 2.3).

11 Making improvements arising from recommendations is the responsibility of bodies delivering services, but visibility about whether recommendations have been implemented can be poor for all parties involved. The main way inspectorates intend to have impact is through their published reports and recommendations, usually directed at inspected bodies. For maximum impact and value from inspection findings, there at least needs to be risk-based monitoring and follow-up. Responsibility for monitoring and following up recommendations rests, however, with different bodies and at different levels, reflecting a variety of accountability mechanisms. Assurance about implementation can be weak without follow-up work or an inspector re-visiting. For example, HMCPSI emphasises follow-up in its work programme. It sees action plans addressing its recommendations, and regularly follows up inspections to monitor implementation. In contrast, ICI Borders and Immigration’s follow-up is limited. It relies on information from the Home Office about recommendation implementation. Some inspected bodies and departments said it can be difficult to implement a large number of recommendations. Audit and risk committees were unclear how departments were prioritising recommendations (paragraphs 2.4 to 2.8 and 2.11 to 2.15).

12 Poor inspection results can trigger intervention action by oversight bodies, but these bodies have flexibility in deciding whether to intervene. Although a snapshot of performance, inspection findings are an independent source of information. They give indicators that, when considered with other factors, can trigger an oversight body to intervene. For example, the Ministry of Justice uses inspection ratings when assessing prison governor performance, and its judgement of performance may result in the replacement of a prison governor. Frameworks for intervention need to allow for discretion about what action to take in the light of inspection reports. However, not all oversight bodies have clear mechanisms linking inspection findings to priorities for action (paragraphs 2.9 and 2.10).

13 The benefits of the knowledge and good practice identified by inspection findings and recommendations are not always maximised. Independent inspection findings generate a substantial knowledge asset. Individual inspection reports identify examples of good practice, and some inspectorates have published compendium reports pointing to or informing good practice, for example HMIC on policing in austerity. HMI Prisons has mined knowledge from prison inspections by linking findings on treatment of ex-service personnel. We found, however, that the benefits of inspection are not always maximised. Inspectorates, limited by resource constraints, do not generally link findings and recommendations across their reports to create extra value from the significant knowledge base they have generated (paragraphs 2.16 and 2.17).

14 Some inspectorates include indicators of sector performance improvement in annual reports, but do not collate systematic evidence of impact. Most inspectorates’ annual reports summarise findings from their inspection reports published that year, but none included detailed examples of how their work had led to better performance. HMI Prisons includes trends in inspection ratings to indicate the sector’s health, which mirrors Ofsted in its annual reports. It also analyses the number of recommendations accepted and achieved. There is, however, no systematic measurement or evaluation of inspection impact, which means the benefits of inspection  are not always visible to the sector (paragraphs 2.18 to 2.20).

The strategic framework for inspectorates

15 Inspectorates’ independence can be perceived as limited. Chief Inspectors’ remits are statutory, and their appointment processes typically include pre-appointment hearings by parliamentary committees. However, effectively, Secretaries of State decide the appointments. They also determine tenure length which, if short, can be a constraint on establishing independence. Departments also control the size of inspectorates’ budgets. They can therefore exert considerable influence on the strategic direction of inspectorates. Once appointed, however, Chief Inspectors act independently in carrying out inspections. The historical context, personal style and authority of the Chief Inspector can be crucial in setting the tone for the inspectorate. In doing so, Chief Inspectors may tread a delicate line in demonstrating independence and shining a light on the impact of reforms. For example, HM Chief Inspector of Prisons has publicly used inspection findings to make observations and recommendations about proposed legislative and policy changes (paragraphs 3.2 to 3.7).

16 Inspectorates mainly determine their own work programmes, but the Department’s influence varies, according to legislation and historical context. Inspectorates determine their work programmes using their own sector risk profiles (from inspections) and sector intelligence (such as performance data and stakeholder feedback). Ministers approve HMIC’s inspection programme, but are only consulted on the work programmes of other inspectorates. Secretaries of State can also direct inspectorates to take on extra work. In practice, the Secretaries of State have rarely used this power of direction, but have reached agreement with inspectorates on commissioning extra work. The Home Office has made more requests of the inspectorates it sponsors than the Ministry of Justice and the Attorney General’s Office. We found, nonetheless, inspections are carried out independently of departments regardless of how programme content is established (paragraph 3.6).

17 Departments’ sponsorship arrangements for inspectorates risk perceived or actual conflicts of interest. We found no agreed approach for departmental sponsorship arrangements for inspectorates, for example from departments working together or from the Cabinet Office. No government department currently considers itself responsible for overall policy in relation to inspection. The Home Office and Ministry of Justice sponsors are in the policy directorates responsible for the inspected sectors. ICI Borders and Immigration is unique in that, since the Home Office brought the activities of the UK Border Agency back into the Department in 2012 and 2013, ICI Borders and Immigration now inspects part of the Home Office. There is also limited independent challenge about how Chief Inspectors run their inspectorates internally; only HMCPSI has a non-executive member on its management board (paragraphs 3.9 to 3.11).

How inspectorates carry out their work

18 Inspectorates have flexed their inspection regimes to adapt to evolving demands, resource pressures and policy developments. Inspectorates’ spending (excluding HMIC) has on average reduced by 9% from 2010-11 to 2014-15. Inspectorates have responded flexibly to funding reductions. With fewer resources, they have mostly developed programmes aiming inspection resource where risks to services are greater. They also reduced thematic work, used electronic media more, and reduced costs. From 2014, however, HMIC is carrying out annual inspections of the effectiveness, efficiency and legitimacy of all police forces, requiring a 66% increase in funding. This is to inform the government’s model for public and democratic accountability of the police. HMI Probation is developing a new outcome-focused inspection approach as the government reforms offender rehabilitation, creating 21 Community Rehabilitation Companies and a new National Probation Service (paragraphs 3.4, 3.7, 4.2 to 4.4, and 4.8).

19 Despite developing their approaches to inspection, inspectorates’ identification and sharing of how they could learn from each other has been limited. We found some differences in inspection practice. For example most inspectorates now use a four-grade rating system for their inspection judgements. However, each inspectorate uses different grades, which can be confusing for those unfamiliar with the sector. We found, however, many similarities in inspection practice. Many of the methodologies and types of skills applied to inspection work are common. Greater sharing and learning could help establish good practice and efficiencies, given the limited resources for inspection. Each inspectorate develops training for its staff, and for joint inspections, and, in the case of HMI Prisons, with other partners, but inspectorates have not collaborated to develop training programmes for common skills. There is methodological guidance for joint work, but good inspection practice is not captured, shared or amplified in a systematic way. Benchmarking of the efficiency of inspectorates’ activity is also limited (paragraphs 3.8, 4.6 to 4.10, and 4.18).

20 Inspectorates publish their inspection reports themselves, with the exception of the Chief Inspector of Borders and Immigration. Before publishing their reports (which can be from two to eight months after the end of fieldwork), all inspectorates feed back to senior management and staff of inspected bodies. This is either at the end of the inspection visit or, for ICI Borders and Immigration and HMCPSI, two weeks later. Most inspectorates then publish their own inspection reports. However, since January 2014, the reports of the Chief Inspector of Borders and Immigration have been published by the Home Secretary, reflecting legal advice to the Home Office that this was the process required under the UK Borders Act 2007 which set up the inspectorate. During 2014, the Home Secretary has published the reports between 43 and 163 days after the Chief Inspector provided the factually agreed report. The Chief Inspector of Borders and Immigration considered that delays in publishing reports risked reducing the effectiveness of independent inspection, and contributed to a sense that his independence was being compromised. The Home Secretary also has powers to redact his reports on national security grounds, and has done so on four occasions (paragraphs 3.5 and 4.11 to 4.15).

21 Inspectorates and departments agree on the importance and potential of joint inspection work, but the Criminal Justice Joint Inspection (CJJI) regime has struggled to make this work effectively and maximise its impact. Joint inspection activity can inspect themes that cut across remits in home affairs and justice, for example bureaucracy in the criminal justice system. It can therefore play a vital role in identifying where systemic change is needed. The Chief Inspectors are committed to this regime, providing resources from their annual inspection budgets to deliver a joint programme. However, there is no dedicated budget for this work. It has been challenging for the different inspectorates to work together, with tone and approach to joint inspections determined by a lead inspecting body and no agreed style for reporting. The Chief Inspectors are accountable to ministers for the direction and delivery of the CJJI programme. We found, however, no overall accountability for the programme’s impact, and a lack of clarity among inspectorates and departments about how to track and follow-up recommendations from the CJJI programme, particularly where these had been directed at bodies where the inspectorates had no remit (paragraphs 4.16 to 4.20).

Conclusion

22 The government has spent around £142 million in the past five years to develop the knowledge asset that inspection findings create. Inspection brings benefits of public scrutiny and assurance. It gives departments, Parliament and the public valuable signs of where performance can be improved. However, there is inconsistency in the governance, extent of independence, and reporting arrangements of inspectorates which can limit their impact. Different routes for communicating findings and, sometimes, unclear systems for following up recommendations can further weaken the effectiveness of external scrutiny. To optimise the £35 million annual investment in inspection activity, inspectorates and their sponsors should consider whether current inspection arrangements are consistent and adequate. This includes whether they best meet the purposes they are publicly intended to serve, and how they can generate more value from the significant knowledge asset created from inspection findings, particularly in the context of constrained and reduced resources.

Recommendations

23 Overall, we recommend that government should identify which department leads overall policy in relation to inspection. That Department should then re-engage with the inspection agenda. It should revisit how the inspection principles set down in 2003 have developed to test if they still hold given the considerable developments in the configuration and delivery of public services in the past ten years.

24 In addition, from our analysis of the five inspectorates across the home affairs and justice sectors, we recommend that inspectorates and departments should do the following:

Maximising impact

a Inspectorates should do more to exploit the knowledge created from their inspections by identifying and disseminating those findings, recommendations and good practice examples which have had most impact, using existing joint working arrangements and liaison with other bodies where appropriate.

b Establish a coherent mechanism and accountability for tracking the impact of joint inspection work carried out under the Criminal Justice Joint Inspection regime.

In sponsorship

c Clarify what the sponsor role should be in relation to inspectorates, explaining any differences between inspectorates in sponsorship arrangements.

d Separate the sponsor roles for inspectorates in departments from areas with policy or operational responsibility for the inspected sector, to avoid perceived or actual conflicts of interest.

e Satisfy themselves that those charged with governance and overseeing inspectorates’ recommendations, for example audit committees and non-executives, have sufficient direct contact with inspectorates to discuss their findings and progress with implementing their recommendations.

f The Home Office should satisfy itself that the reporting arrangements for the Independent Chief Inspector of Borders and Immigration are appropriate to underpin the independence of the Chief Inspector. It should consider whether legislation needs to be changed to make these arrangements consistent with other inspectorates.

Carrying out inspection activity

g Examine the impact of different rating scales on inspected bodies, the executive and the public.

h Share experience of common inspection practice to learn from each other and hone practices, especially where inspectorates have common elements in their missions and objectives.

i Devise an accredited inspection training programme to develop common inspection skills.

Part One

Introduction

1.1 Each year around £20 billion is spent in the areas of home affairs and justice. This part examines the approach, history and context for inspection work in these areas. The work covered by this spending is scrutinised by internal and external auditors (such as the National Audit Office), regulators, and inspectorates for ministers, Parliament, stakeholders and the public. For departments, information from such scrutiny aids accountability. It can also help to oversee provision, highlighting service shortcomings, such as provider failure, and identifying good sector practice and opportunities for improvement.

1.2 The scrutiny bodies complement one another but have distinct roles. Importantly, inspectorates have only the power of their voice, through reporting on periodic, objective checks of inspected bodies against standards and criteria. In recent decades inspection has developed significantly. Most recently, in July 2003, the Office of Public Services Reform published Inspecting for improvement: Developing a customer focused approach including ten key principles of inspection. We used these as a starting point to review arrangements for, and activity of, five inspectorates in the home affairs and justice sector (Figure 1). Taken together, they cost £35 million in 2014-15. We also considered developments for inspection regimes arising from, for instance, Ofsted’s work, and the Francis reports on mid-Staffordshire NHS Trust failings.

1.3 HMI Prisons’ work, and the custodial parts of other inspectorates’ work, are subject to the UK’s obligations from the United Nations protocol against torture, known as the ‘Optional Protocol to the Convention Against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment (OPCAT)’. The Protocol requires state parties to have a national preventive mechanism to carry out independent preventive inspections of all places of detention.1 The work of this mechanism is scrutinised by relevant UN bodies.

1.4 The nature and scope of inspection activity evolves. Inspectorates use several approaches to determine inspection programmes and carry out their work. They may: select case files for review, analyse samples of cases, examine documentation held by the inspected body, and use observation, interviews, focus groups, questionnaires and surveys. They may talk to staff at different levels in inspected bodies and seek views from service users. In doing their work inspectorates need to consider the burden being placed on inspected bodies. Inspections may also be pre-announced or unannounced. Whatever techniques are used, they need to be appropriate to inspection’s scope and goals. Although the nature of the cycle will vary between inspectors, their work typically follows a plan-inspect-report-recommend-follow-up cycle (Figure 2).

Our approach

1.5 In this comparative review we selected topics to provide insight into ongoing arrangements for inspection, rather than comparing the inspectorates directly on every aspect of their work. We make comparisons to Ofsted and the Care Quality Commission, a regulator, where appropriate to illustrate our points. We do not examine the sectors inspected in any detail. We intend that any lessons will also be relevant to other departments that sponsor or invest in inspection activity, and also to the centre of government (Cabinet Office). Our comparative examination covers three topic areas, the most important of which is:

whether the impact of inspection is maximised (Part Two);

This is in turn influenced by:

whether the strategic framework for inspection supports the work of
inspectorates (Part Three); and

whether inspectorates carry out their work effectively (Part Four).

1 Inspectorates within this mechanism include HMI Prisons, HMIC, Ofsted, and the Care Quality Commission (a regulator).

Part Two

Maximising the impact of inspection

2.1 Getting public value from inspection findings depends on several factors. First, how far inspection findings shed light to inform stakeholders. Second, whether findings and inspection standards are known about, and the impact they have in improving the way services are provided or experienced. This part examines: the implementation of inspection recommendations and inspectorates’ visibility over this; sharing and informing good practice to improve sector performance; and how inspectorates in home affairs and justice assess their impact.

2.2 Inspected bodies pointed to the burden of inspection. However, they recognised the profound cultural effect on their behaviour of the act and presence of inspection and knowing the inspector would, or could, visit. The existence of inspection standards, such as those of HMI Prisons, has an impact as often managers will aim to meet these standards before they are inspected. Furthermore, having inspectors on site provides an opportunity for staff and others, such as service users, to raise concerns about the inspected body.

2.3 As well as published reports and inspection standards, inspectorates use other routes to influence and drive sector improvement. For instance:

Ongoing dialogue. Chief Inspectors talk to their respective ministers, respond to formal policy consultations, speak at conferences, and appear before parliamentary committees.

Oversight groups. Inspectorates may be active in groups assessing compliance with inspection reports and policy more generally. For example, HMIC chairs the Rape Monitoring Group, which brings together agencies to promote a more joined-up approach to rape and to provide expert opinion and recommendations to the government.

Routine monitoring. For example, HMIC monitors police forces’ performance so that Chief Constables and local policing bodies can identify where corrective action may be needed.

Management reports. Inspectorates may produce internal management reports to encourage improvement. For example, in 2013-14, HMCPSI produced 18 management reports for the Crown Prosecution Service (CPS) on CPS areas’ and units’ performance, as part of its Annual Casework Examination Programme (ACEP).

Inspectorate recommendations

Implementation of inspectorate recommendations

2.4 The main way inspectorates intend to have impact is by reporting their findings, and inspection reports include recommendations for improvement. Figure 3 shows examples of reports inspectorates consider have had a strong impact. Reports are either on individual establishments (around 83%) or are thematic reports (17%). The latter examine services end-to-end and across institutions.

2.5 The volume of reports produced varies from HMI Prisons’ annual average of 98 to HMCPSI’s 10 (Figure 4), although this excludes HMCPSI’s unpublished management reports providing assurance on the performance of the CPS (paragraph 2.3). Each year HMCPSI allocates 10–20% of its inspector time to ACEP, examining a sample of 800–2,200 files from the simplest to the most complex prosecution cases. In reaching its judgement on decisions made in these cases, it deploys specialist legal inspection expertise to assess, among other things, the accuracy of CPS decision-making.

2.6 Inspectorates mainly focus their recommendations on the bodies being directly inspected. For example, HMI Prisons directs over 80% of its recommendations at the prison’s governor. HMI Prisons’ reports on individual establishments make far more recommendations than other inspectorates (Figure 5). In contrast, inspectorates’ thematic reports tend to make fewer recommendations, aimed at more bodies.

2.7 Responsibility for implementing inspection recommendations principally rests with the inspected bodies. Addressing performance of inspected bodies – for example enforcing implementation or taking intervention action – is the responsibility of bodies overseeing the inspected bodies. Oversight responsibilities lie at different levels, depending on the accountability landscape the inspected bodies operate in (Figure 6). Inspectorates said that the absence of an obvious implementation route for recommendations that affect whole systems is a significant challenge for them.

2.8 The Home Office (for borders and immigration) and the Crown Prosecution Service monitor their own implementation of recommendations (Figure 7). Where delivery of policy and services is local, detailed oversight of recommendations is also performed locally, with some central government departments having less visibility over implementation progress, depending on the accountability arrangements. For example, local Police and Crime Commissioners are responsible for their forces’ provision of effective and efficient policing in their area, including making sure that action is taken on HMIC recommendations, with the Home Office having a limited role. In contrast, while the National Offender Management Service (NOMS) has delegated responsibility for monitoring implementation of recommendations made by HMI Prisons to its regions, its Regional Directors feed back to NOMS HQ on their progress and on any concerns they have about recommendations that are not being addressed, allowing NOMS HQ to identify any system-wide issues or trends of concern.

2.9 A poor inspection can trigger intervention by oversight bodies (Figure 7). For example, NOMS considers inspection ratings when assessing prison governor performance, and its judgement of performance may result in the replacement of a prison governor. A poor inspection report does not automatically trigger a prison governor’s removal; this would depend on circumstances, such as time in post and the central support they have previously received. NOMS may also change an establishment’s population or capacity in response to a poor inspection report.

2.10 Where the accountability for service delivery rests at the local level, local oversight bodies usually take the lead in intervening, with the national oversight body having only limited powers of intervention. For example, only local Police and Crime Commissioners have the power to remove a Chief Constable. The Home Secretary does have back-stop intervention powers, although these consist of directing a Commissioner to take action, and the Home Secretary cannot directly intervene. The Home Office considers the use of these back-stop powers on a case-by-case basis.

2.11 Some inspected bodies and departments told us that it can be difficult to implement a large number of recommendations or ones that are ill-judged. NOMS also told us that inspection standards do not always match operational standards.

2.12 We found that those charged with governance in departments struggled with the number of recommendations made, how to address accountability for them, and whether they were directed where action could be taken. Departments could be overwhelmed with the volume of recommendations made, and, when assessing reports from several different inspectorates, lacked clarity about how to prioritise the recommendations and make sense of the different rating systems for inspection judgements. In turn, the audit and risk committees of the Ministry of Justice and Home Office were not clear how their departments were prioritising recommendations nor about the systems in place to track them. Chief Inspectors are not routinely invited to attend these committees.

Inspectorate visibility of recommendation implementation

2.13 The inspectorates have no formal powers to intervene to enforce the implementation of their recommendations. However, they can use their voice to exert pressure by publicly commenting on inspected bodies’ progress in addressing their findings. How far each inspectorate uses this voice varies. Some inspectorates have less visibility of bodies’ progress in implementing recommendations and some do less follow-up work (Figure 8).

2.14 All inspectorates receive copies of the action plans prepared by the inspected bodies in response to each inspection report. This contrasts with Ofsted which, due to the large number of schools it inspects, only sees the plans of those rated as ‘requiring improvement’ or ‘inadequate’. These action plans state which recommendations have been accepted, partially accepted or rejected and, in the case of accepted recommendations, how the inspected body plans to address these. Their quality is variable, with the formal responses from some Police and Crime Commissioners containing little detail on how recommendations will be taken forward.

2.15 Inspectorates’ visibility of the progress made in implementing these action plans varies (Figure 8). All inspectorates check to see how their recommendations have been implemented when they fully re-inspect a body. However, there may be long periods between inspections. Inspectorates may choose to follow up sooner, to see how recommendations have been implemented. Inspectorates vary in the amount of effort they direct at such work. For example, HMCPSI carries out shorter, follow-up inspections for each report within up to two years of publication. They focus purely on progress in implementing recommendations. It then publishes these follow-up reports. In contrast, the Chief Inspector of Borders and Immigration has chosen not to do formal follow-up. But, for one year only, he piloted spot-checks of up to one day, producing short published reports. In comparison, Ofsted undertakes monitoring inspections of schools judged to be ‘inadequate’, and of those schools judged as “requires improvement” where leadership and management also requires improvement.

Sharing and informing good practice

2.16 Another way to secure impact is through inspectorates sharing good practice identified during inspections with inspected bodies, or by informing good practice developed by others. For instance, HMIC reports may inform the College of Policing, which has responsibility for setting out national standards, identifying, developing and promoting good practice, and authorising professional policing practice. While the inspectorates highlight good practice in their published reports on individual establishments, their analysis of such reports to identify and disseminate examples of good practice is extremely limited (Figure 9). When it is done, inspectorates usually mine individual reports to inform thematic reports, rather than to generate practical advice, where appropriate, for providers. For example, HMI Prisons has analysed individual prison inspection reports to identify findings on the treatment of ex-service personnel. HMI Prisons told us that it would like to do more improvement work, but faced resourcing constraints.

2.17 In contrast, Ofsted has developed from its findings a searchable, publicly accessible, database on its website, with examples of practice that have worked for providers in achieving successful outcomes for children, young people and learners. HMIC is planning to work with the College of Policing to develop a similar web-based facility. In the absence of good practice guides from inspectorates, some bodies, such as the Youth Justice Board and the Home Office’s borders and immigration directorates, have started to analyse inspection reports themselves to identify good practice.

Assessing the inspectorates’ impact

2.18 Assessing the impact of inspection is inherently challenging due to the difficulty in establishing the causal link between an inspection report and any subsequent improvement in an inspected body’s performance. We found inspectorates made only limited assessment of their impact (Figure 10). No inspectorate annual reports included detailed examples of how their work had led to better performance. Both ICI Borders and Immigration and HMI Prisons reported on the number of accepted recommendations. However, only HMI Prisons then reported on the number of recommendations implemented. HMI Prisons was also the only inspectorate to report on trends in inspection ratings in its annual reports, mirroring Ofsted’s practice.

2.19 Inspectorates also monitor stakeholder satisfaction with their performance, using several methods:

regular dialogue (especially Chief Inspectors) with ministers, sponsoring departments and senior management of inspected bodies;

feedback from inspected bodies at the end of each inspection;

HMI Prisons undertakes annual stakeholder surveys, covering a range of bodies, although not users of the service inspected; and

ICI Borders and Immigration holds three annual forums (Sea Ports, Aviation, and Refugee and Asylum) to seek stakeholders’ views.

2.20 No home affairs and justice inspectorate or sponsoring department has undertaken a systematic measurement or evaluation of its actual impact. In contrast, the Care Quality Commission, a regulator, is attempting to evaluate and measure its success across four levels, impact, outcomes, quality and effectiveness, and internal capability (Figure 11).

Part Three

The strategic framework for inspectorates

3.1 The strategic framework for inspectorates is important as it contributes directly to the credibility of inspection, the scope of the work inspection covers, and how the independence and objectivity of inspection is perceived. This can all maximise the impact of inspection as discussed in Part Two. This part therefore compares inspectorates’:

direction, and how it is set;

work programming and budgets; and

governance arrangements.

Setting the direction of inspectorates

3.2 Legislation sets out the remits of each inspectorate and for criminal justice joint inspection work (Part Four). Their statements of purpose cover a spectrum of performance improvement, monitoring, advising, and promoting efficiency, effectiveness and better outcomes (Figure 12). Their detailed focus varies, but, broadly, inspectorates focus on performance improvement and transparency. And, while all inspection brings transparency, to meet obligations arising from OPCAT (paragraph 1.3), HMI Prisons’ activity in particular focuses on human rights, to report on the treatment of people detained and conditions of places of detention.

3.3 For all Chief Inspectors, the decisions as to who is appointed to the post and the length of their tenure are effectively taken by the relevant Secretaries of State (Figure 13). A pre-appointment process usually includes candidates being scrutinised by parliamentary committees, although their recommendations are non-binding. By appointing the Chief Inspector, departments can therefore exert considerable influence over an inspectorate’s strategic direction. The choice of a short tenure can also act as a constraint on a new Chief Inspector establishing their independence. In November 2014, the Public Administration Committee recommended that HMI Prisons should be more fully independent of government and report to Parliament.2

3.4 The Chief Inspector’s voice can be a significant influence on the inspected sectors and on departments. All except one of the current Chief Inspectors in post in December 2014 had been appointed from outside the sector they inspect,3 bringing the benefit of independence from the sector they are inspecting. Appointments incorporate setting expectations about the role of the Chief Inspector in the context of system reforms departments are making. For example, the current HM Chief Inspector of Constabulary (the first to be appointed who was not formerly a police officer) was appointed in recognition of the complex challenges facing policing in the context of greater democratic accountability of police forces, after elected Police and Crime Commissioners were introduced under the Police Reform and Social Responsibility Act 2011. Similarly HM Chief Inspector of Probation was appointed in February 2014, as radical reforms to offender rehabilitation are being introduced, with 21 Community Rehabilitation Companies and a new National Probation Service being created. HMI Probation is expected “to play a vital role in maintaining high standards and shining a light across the new system, helping the government to get the best from its reforms.” 4

3.5 Tensions can arise between Chief Inspectors and departments where the Chief Inspector uses their personal style and authority to set a clear tone for their independence. For example, HM Chief Inspector of Prisons has publicly used inspection findings to make observations and recommendations about proposed legislative and policy changes, and to draw attention to pressures on inspected establishments from reduced resources. The Chief Inspector of Borders and Immigration also wrote to the chair of the Committee of Public Accounts in November 2014 highlighting that the majority of his reports since January 2014 had been subject to significant delays between submission to the Home Secretary for publication and their laying in Parliament. He considered that lengthy delays in publishing reports risked reducing the effectiveness of independent inspection, which depends to a large extent on timely publication of findings, and contributed to a sense that the independence of his role was being compromised.

Work programmes and budgets

3.6 The inspectorates determine their work programmes using their own risk profiles, and intelligence from inspection reports, sector performance data, and stakeholder feedback.5 Legislation provides for slightly different determination of the work programmes, and how far departments make extra requests for inspection work varies. Inspectorates consult ministers and others about work programmes, but only HMIC’s programme requires ministerial approval (Figure 13). All inspectorates can be directed by the relevant Secretaries of State to take on extra work. In practice, the Secretaries of State have rarely used this power of direction, but have reached agreement with inspectorates to commission additional work. The Home Secretary has commissioned more work from the inspectorates sponsored by the Home Office than the Justice Secretary and the Attorney General. For example, in March 2014, HMIC laid its approved programme before Parliament. Shortly after, the Home Secretary commissioned two pieces of work, on domestic violence and undercover officers. ICI Borders and Immigration’s inspection programme does not require approval. However, it must prioritise in-year commissions by the Home Secretary (four since 2008). We found, nonetheless, inspections are carried out independently of departments regardless of how programme content is established.

3.7 The sponsor departments allocate inspectorates’ budgets. Being mostly small in size, inspectorates rely on their sponsoring department for back-office functions (such as IT, human resources and finance). The spending of four of the inspectorates has on average reduced by 9% between 2010-11 and 2014-15 (Figure 14). Inspectorates have handled funding reductions in several ways. For example, more efficient use of staff, reducing thematic work, using electronic media more, sharing of accommodation with other bodies, generating extra income, and reducing travel and subsistence costs. However, HMIC’s shift to annual PEEL inspections has increased its annual budget by 66%.6

3.8 Inspectorates have tried to assess their efficiency. However, indicators are not on a common basis, and cannot be used to compare their relative efficiency. Only limited benchmarking has been carried out. HMIC benchmarked its overall budget, comparing relative costs of other inspectorates as a percentage of the sector inspected, as part of its business case for extra resources for annual force inspections (PEEL). HMI Probation prepared a case study in 2005 of the costs and benefits of its inspection, and HMCPSI has monitored its unit costs. Two inspectorates publish their deployable inspection hours, but only HMI Prisons publishes the cost per deployable inspection hour (which it has reduced year-on-year). Two inspectorates also publish inspection effort indicators – the percentage of resources directed at inspection activity (ICI Borders and Immigration, 70% and HMIC, 88%).

Governance arrangements

3.9 For inspectorates sponsored by both the Home Office and Ministry of Justice, the sponsorship function lies in the policy areas being inspected. Since the UK Border Agency was abolished in March 2013, responsibility for the delivery of borders and immigration policy rests with the Home Office itself. ICI Borders and Immigration is therefore in a unique position in that it directly inspects its own sponsoring department. Within the Home Office, the sponsorship function does not reside, however, with one of the three directorates responsible for providing borders and immigration services, but with the Director General, International and Immigration Policy.

3.10 We found no agreed approach for what the role of a sponsor of an independent inspectorate should be, for example from departments working together or from Cabinet Office. There have been no developments in central thinking about the role of inspection or the sponsor role for inspectorates in recent years. No government department currently considers itself responsible for overall policy in relation to inspection.

3.11 For internal governance of inspectorates, only one of the inspectorates had a non-executive member of its management board (HMCPSI). There is therefore limited independent challenge about how Chief Inspectors run their inspectorates internally, for example, their operational efficiency. Inspectorates have drawn on external expertise for specific advice. For example, HMIC for developing its PEEL programme and HMI Prisons for developing its inspection standards for women’s prisons (Figure 13).

2 Public Administration Select Committee Who’s accountable? Relationships between Government and arm’s-length bodies, First Report of Session 2014-15, HC 110, November 2014.

3 Paul McDowell was the Chief Executive Officer of NACRO, the charity for reducing crime and reoffending at the time of his appointment as HM Chief Inspector of Probation in 2014.

4 Ministry of Justice, New Chief Inspector of Probation announced. Press Release, 8 November 2013.

5 The Criminal Justice Joint Inspection programme is determined in a similar way, by Chief Inspectors (Part Four).

6 In December 2013, the Home Office decided to fund a new annual programme of all-force inspections – PEEL (Police Efficiency Effectiveness Legitimacy) Assessments. On 27 November 2014, HMIC published the first PEEL assessment. HMIC continues to develop its PEEL methodology.

Part Four

How inspectorates carry out their work

4.1 Inspectorates carry out inspections using different approaches. They have developed their own ways of working, according to: their objectives, appropriateness of practices and methods as determined by inspectorates, and legislation. Differences also arise from the context the inspectorates work in, such as service user needs, and how service providers are configured. This part examines similarities and differences in how the inspectorates conduct inspections, reach their judgements, and report their findings. This, in turn, can maximise the impact achieved. We also examine the Criminal Justice Joint Inspection regime.

Conducting inspections

4.2 Inspectorates have flexed their inspection regimes to adapt to evolving demands and resource pressures. In recent years, the government and others have called for inspectorates to adopt a more light-touch approach to inspection to reduce the administrative burden on bodies. In response, home affairs and justice inspectorates have mostly adopted a more risk-based approach. They select areas, bodies or themes to inspect, rather than planning to cover the entire population that could be inspected (Figure 15). With such an approach, the frequency with which they inspect a body or issue depends upon the risk they assess is posed. This is determined on the basis of previous inspection findings and an inspected body’s reaction to these, and in terms of risks to services. This is in contrast to a cyclical approach, where inspectorates inspect each delivery body to a set cycle. This change has also reflected the inspectorates’ desire to protect their core inspection activity in the face of funding reductions (paragraph 3.7).

4.3 Running counter to this risk-based approach, from 2014-15, the Home Office decided to provide significant extra funding to HMIC (paragraph 3.7) to enable it to carry out PEEL assessments covering all police forces annually. The Home Office wanted an inspection approach that allowed the public to see from a few easy-to-understand categories, whether their local police force was performing well in cutting crime and providing value for money. HMIC published its first PEEL assessments in November 2014, and continues to develop the PEEL methodology.

4.4 The February 2013 Francis Report into the problems at the mid-Staffordshire NHS Trust, called for more emphasis on inspection than on self-reporting and assessment. There are now calls on all inspectorates to take an approach which is still risk-based but is less light-touch and more rigorous.

4.5 The inspectorates vary in how much notice of an inspection they give to an inspected body, with only HMI Prisons extensively using unannounced inspections (Figure 15). HMI Prisons does this to ensure that it gets an accurate picture of prison conditions and the treatment of prisoners when it inspects. In comparison, HMCPSI, HMI Probation and ICI Borders and Immigration do mostly announced inspections. Their inspections include examining samples of case files, which can take inspected bodies time to provide. Despite being announced inspections, the inspectorates’ random selection of samples aims to give an accurate picture of a body’s performance. Differences in the extent of announced or unannounced inspection is significant only insofar as inspectors need to determine the most appropriate approach to getting an accurate and timely view of the inspected services.

4.6 There is much in common in how the inspectorates carry out inspections. However, we found that there is no systematic sharing of good inspection practice among inspectors at a working level, which could help establish good practice and efficiencies, given constrained resources. Links between inspectorates tend to be at a senior level. For example, the Chief Inspectors meet each quarter under the Criminal Justice Joint Inspection regime (Figure 1), and practice is shared and developed through the cross-inspectorate Criminal Justice Development Group, made up of senior managers from each inspectorate, which also meets quarterly. Inspectors also share knowledge when undertaking joint inspection work. However, there is no working level cross-inspectorates forum, nor a virtual portal where inspectors can exchange knowledge and experience.

4.7 The inspectorates commonly use a mix of staff to carry out inspections, including people seconded from the inspected sector, in order to obtain the required balance of technical insight, sector knowledge, and inspection skills. Each inspectorate offers training for its staff. For example, HMIC has developed a training programme for the large number of staff it has recruited to carry out the PEEL assessment and it intends to have this programme accredited. Training is also provided for joint inspections and, in the case of HMIP, with other partners. The inspectorates have not, however, collaborated to develop training programmes, even though the types of skills applied to inspection work are common.

Reaching inspection judgements

4.8 All the inspectorates inspect in the same way in that they each have a set of pre-agreed criteria against which they assess an inspected body and which they make available to these bodies. However, the focus of these criteria varies widely, reflecting the inspectorates’ different statutory remits (Figure 15). Thus, HMI Prisons’ criteria reflects its statutory focus on the treatment of prisoners and prison conditions. ICI Borders and Immigration’s criteria probe compliance with proper border and immigration procedures. HMI Probation is revising its detailed inspection criteria to focus more on reoffending outcomes.

4.9 All inspectorates use similar methods to collect evidence to assess inspected bodies against their criteria. For example, interviews with management and staff, analysis of performance data, and observation. HMCPSI, HMI Probation and ICI Borders and Immigration use document review more, through examining case files. HMIC, HMI Prisons and HMI Probation seek feedback from service users.

4.10 In recent years, most inspectorates have converged on the use of a four-grade rating system to assess performance against their criteria, although the grades used by each vary (Figure 15). Different grades makes comparison between the types of bodies involved in the criminal justice sector difficult, and it can be confusing for those unfamiliar with the sector, such as the general public. In its PEEL assessments, HMIC has deliberately adopted the grade labels used by Ofsted as these are ratings with
which the public are familiar.

Reporting inspection findings

4.11 All inspectorates check the factual accuracy of their reports with the inspected body and relevant third parties before publication. The period between the end of fieldwork and report publication in 2013-14 has been, on average, approximately two to four months, depending on the inspectorate, and eight months for CJJI reports (Figure 16).

4.12 Delays in publishing reports can limit their impact. The adverse effects of delayed publication can be partly offset by the practice all inspectorates undertake of giving senior management and staff of the inspected body initial feedback on findings after the inspection. According to the Home Office, those inspected sometimes found significant differences between feedback sessions and the draft inspection report, when received. Three inspectorates provide their feedback on site on the last day of the inspection visit,7 while ICI Borders and Immigration and HMCPSI do this up to two weeks after the inspection visit.

4.13 All but one of the inspectorates publish their own individual inspection reports and are free to decide when they do this. The exception is ICI Borders and Immigration, whose reports the Home Secretary lays before Parliament. The Home Secretary decides, once an inspection report has been factually agreed, when the report should be laid, and publishes the report. During 2014, the Home Secretary has published ICI Borders and Immigration reports between 43 and 163 days after the Chief Inspector provided the factually agreed version of its report. As a result, the average time taken to publish ICI Borders and Immigration reports from April to December 2014 has increased to 5.5 months, compared to 4.3 months in 2013-14 (Figure 16). Delays are partly because the reports cannot be laid before Parliament when it is in recess. In December 2014, the Home Office and ICI Borders and Immigration agreed that the Department would aim to lay reports within eight weeks of receiving them, excluding recess.

4.14 The Home Secretary started to publish these reports from January 2014, after the Home Office had legal advice that this was the process required under the UK Borders Act 2007 which established ICI Borders and Immigration. Up to December 2013, the Home Office was content for ICI Borders and Immigration to publish its own reports. The Home Office’s statutory framework for ICI Borders and Immigration contrasts to that for HMIC. Until 2011, the Home Office similarly arranged to publish HMIC reports. However, under the Police Reform and Social Responsibility Act 2011, it then gave this power to HMIC.

4.15 Inspection reports produced by ICI Borders and Immigration and HMIC can be redacted in cases where publication of the relevant part of the report would be against the interests of national security or might jeopardise anyone’s safety. In the case of ICI Borders and Immigration reports, the Secretary of State has the powers to redact. Until the Police Reform and Social Responsibility Act 2011, the Secretary of State had similar powers to redact HMIC reports, but these were transferred to HMIC by the Act. The powers have only been used to make redactions to sections of four ICI Borders and Immigration reports in total, and none since November 2013.

Criminal Justice Joint Inspection

4.16 The Criminal Justice Joint Inspection (CJJI) regime (Figure 1) complements the inspection of individual delivery bodies by the individual inspectorates. It enables inspectorates to examine thematic issues across the justice system and end-to-end processes that involve more than one criminal justice agency. This can identify where systemic change is needed.8 The agencies and organisations within the criminal justice system have evolved differently in their history, approach and governance structures. The complexity of this system has shaped how inspectorates approach inspection, both individually and collectively.

4.17 The Chief Inspectors put together a rolling two-year programme of CJJI work annually, and consult on this with a number of parties, including ministers. The Chief Inspectors meet quarterly to monitor delivery, discuss the programme and share good practice, supported by the Criminal Justice Development Group (paragraph 4.6). Chief Inspectors also report on progress with the programme and its latest findings to ministers bi-annually. There is no dedicated budget for CJJI work, and scheduling of resources can be problematic. Nevertheless the Chief Inspectors are committed to this regime, providing resources from their annual inspection budgets for this joint programme. Inspectorates have allocated 36,500 hours for CJJI work in 2013-14, equating to 1,000 weeks over a year and about 6% of all the inspectorates’ planned work.

4.18 The delivery of agreed CJJI work is ‘singly-led but jointly owned’, with an individual inspectorate identified as the lead for each piece of work. The approach to planning, scoping and carrying out the inspection and the tone and style of the final report are determined by this lead inspector. Their experience of inspecting different parts of the criminal justice system brings complementary perspectives and the benefit of cross-inspectorate learning during joint inspection work. The different inspectorates have nevertheless found it challenging to work together, and the presentation of CJJI reports to March 2014 varies. In March 2013, Chief Inspectors published their first standard joint methodology, providing guidance on conducting CJJI inspections and drafting CJJI reports, which lead inspectors were free to adopt. They produced a revised version in April 2014.

4.19 Monitoring the implementation of recommendations in CJJI reports and measuring their impact is difficult. The nature of thematic recommendations is that they are often about system-wide change, rather than being addressed at individual delivery bodies. As mentioned in paragraph 2.7, the absence of an obvious implementation route for recommendations that affect whole systems is a significant challenge for inspectorates. Recommendations are also addressed at many different types of body. We identified that the CJJI reports published between April 2010 and March 2014 contained recommendations for 35 different types of bodies, some of which were organisations, such as NHS bodies and local authorities, where the criminal justice inspectorates have no remit.

4.20 Individual inspectorates are left to monitor the implementation of any CJJI recommendations aimed at the bodies they inspect and the lead inspector constructs measures to assess an inspection’s impact. Inspectorates use the same procedures to track CJJI recommendations as they do for their own recommendations, with each adopting its normal follow-up practice. The quality of these procedures is variable (paragraph 2.15) and there is no consistency in the way recommendations from CJJI reports are followed up. The Chief Inspectors are accountable to ministers for the direction and delivery of the CJJI programme via their bi-annual meetings (paragraph 4.17). There is, however, no overall accountability for the programme’s impact, and no central monitoring by the Chief Inspectors Group or Development Group to track CJJI recommendations or the impact CJJI inspections have achieved. The CJJI is committed to improving follow-up to identify where its recommendations have been the catalyst for change, and in February 2014, introduced a ‘close-down’ process for its reports to see whether a follow-up inspection was needed.

7 HMI Prisons additionally provides a written ‘script’ of its key findings to managers.

8 CJJI reports are published on its website at www.justiceinspectorates.gov.uk/cjji/

Appendix One

Our audit approach

1 We examined if inspection in the criminal justice system is giving assurance on service quality and encouraging improvements in public service provision. The study considers five inspectorates sponsored by the Ministry of Justice, the Home Office and the Attorney General’s Office to give a comparative assessment. In particular we reviewed the following:

the inspectorates’ accountability and governance arrangements, objectives, resources, and work programmes;

their inspection regimes including the frameworks, reporting arrangements feedback mechanisms and the joint inspection approach; and

the stakeholders’ satisfaction towards the inspection process, the systems in place to encourage sector improvement, and arrangements to evaluate impact.

2 We analysed what would maximise value in:

the impact of inspection work;

the strategic framework setting the direction and focus of inspection; and

how inspection activity is carried out.

3 Our audit approach is summarised in Figure 17. Our evidence base is described in Appendix Two.

Appendix Two

Our evidence base

1 We reached our conclusions after analysing evidence collected between August and November 2014.

2 We applied an analytical framework with evaluative criteria to examine how the impact of the inspectorates’ work is maximised, the strategic framework setting the direction and focus of inspection, and how inspection activity is carried out. Our audit approach is outlined in Appendix One.

3 We examined whether the impact of the inspectorates’ work is maximised:

We reviewed key inspectorate and sponsoring department documents to identify how they monitor the implementation of inspectorates’ recommendations, communication of findings to inspected bodies, measurement of impact, and stakeholder feedback on their performance. We also examined if mechanisms to drive and influence sector improvements were in place.

We reviewed legislation and departmental documentation to identify intervention regimes.

We carried out semi-structured interviews with the inspectorates, their sponsoring departments, and stakeholders to confirm our understanding.

We held four focus groups with inspectorates’ staff, inspected bodies, and other stakeholders to seek their views on the adequacy of the follow-up and intervention regimes and the extent of the inspectorates’ impact.

We reviewed inspectorates’ key performance data to identify how they have performed against their objectives and the impact they have had.

4 We analysed how the strategic framework for inspectorates supports their work:

We reviewed governance documents, key strategic documentation, management data, and annual reports held by the inspectorates and their sponsoring departments to compare how the inspectorates plan their work and to identify any limits on their independence.

We reviewed the joint inspection documentation to identify how inspectorates determine the joint inspection work programmes.

We reviewed external documentation such as legislation and select committee appointment reports.

We interviewed the inspectorates’ senior management and their sponsoring departments.

We held four focus groups with inspectorates’ staff, inspected bodies, and other stakeholders.

5 We examined how inspectorates carry out inspection activity:

We reviewed each inspectorate’s key documentation which set out their inspection methods, frameworks and judgements.

We reviewed joint inspection documents to identify how joint inspections are carried out.

We reviewed academic literature and external evaluations of inspection in the home affairs and criminal justice areas.

We interviewed the inspectorates to confirm our understanding from the work above.

We held four focus groups with inspectorates’ staff, inspected bodies, and other stakeholders to seek their views on the adequacy of the inspectorates’ approaches to inspection.

We reviewed reporting arrangements, and analysed inspectorates’ reports to identify the length of time between the end of fieldwork and report publication.

We attended an inspection visit with each one of the five inspectorates to confirm their approach to inspection.

We analysed the reporting and activities of the CJJI programme, and how it has developed.

6 In addition, we reviewed literature and consulted experts in the field. In particular we were grateful for the advice of Professor Stephen Shute of University of Sussex and Professor Rod Morgan of University of Bristol.

Design and Production by NAO Communications
DP Ref: 10631-001