Report

by the Comptroller
and Auditor General

Care Quality Commission

Care Quality Commission – regulating health and social care

HC 409 Session 2017–2019 13 October 2017

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund have used their resources efficiently, effectively, and with economy. Our studies evaluate the value for money of public spending, nationally and locally. Our recommendations and reports on good practice help government improve public services, and our work led to audited savings of £734 million in 2016.

Care Quality Commission

Care Quality Commission – regulating health and social care

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 11 October 2017

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

4 October 2017

This report examines whether the Care Quality Commission is taking appropriate action to address the risks to people’s care.

© National Audit Office 2017

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.gsi.gov.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

11628 10/17 NAO

The National Audit Office study team consisted of:

Sergiu Cociu, Fran Duke, Natalie Low, Alex Pullen, Laura Scott and Dan Varey, under the direction of Robert White.

This report can be found on the National Audit Office website at www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157-197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key facts

Summary

1 The Care Quality Commission (the Commission) is the independent regulator of health and adult social care in England. It is a non-departmental public body accountable to Parliament, sponsored by the Department of Health (the Department). The Commission has two main purposes: to make sure health and social care services provide people with safe, effective, compassionate, high-quality care; and to encourage providers to improve the quality of care.

2 The Commission regulates providers across three sectors: hospitals, adult social care and primary medical services. It registers, monitors and inspects providers, and publishes its assessments and provider ratings. The Commission can also take enforcement action when care falls below fundamental standards.

3 We have reported on the Commission twice before, in 2011 and 2015. Our 2015 report found that the Commission had made progress in updating its regulatory model. However, challenges remained around its ability to assess its overall impact, establish a stable workforce, improve its data on regulated bodies and monitor its own performance. Following the Committee of Public Accounts’ (the Committee) report in December 2015, the Committee requested further work to assess the progress the Commission was making.

4 Since our 2015 report, the Commission has introduced a new five-year strategy, which includes a move to a more intelligence-driven regulatory approach, and its funding will reduce by 13% between 2015-16 and 2019-20. These changes come at a time when health and social care providers are facing very high levels of demand and financial challenge. The Commission has also implemented new responsibilities, including from April 2015 the market oversight of ‘difficult-to-replace’ providers of adult social care.

5 This report looks at whether the Commission is taking appropriate action to address the risks to people’s care through examining:

the extent to which the Commission’s current performance is ensuring high-quality care and encouraging improvement;

how the Commission uses its resources and measures its performance; and

how the Commission is preparing to implement its new strategy with a smaller budget and for potential longer-term changes in the delivery of care.

6 In looking at the Commission’s regulatory activity we focus on its core functions of: registration; monitoring; inspecting and rating; and responding to concerns and taking enforcement action.

Key findings

Undertaking the Commission’s core regulatory functions

7 The Commission has completed its inspection and rating programme, comprising more than 28,000 provider locations. The Commission fulfilled its programme to inspect and rate NHS hospitals, adult social care providers and GP practices in January 2017. For the first time, this provides a baseline assessment of the quality of services across England. The Commission completed the programme in line with its 2016-17 business plan commitments, having missed earlier target dates due to difficulties in recruiting inspectors (paragraph 2.14).

8 There remain a number of operational issues with the Commission’s registration of providers. In 2016, the Commission established a registration improvement programme to address a number of issues, including the speed and burden of registration. For all three sectors the Commission regulates, it did not meet its key performance indicator to complete the registration process for 90% of cases within 50 days during 2016-17, although performance on the indicator did improve (paragraph 2.2).

9 The Commission introduced a new key performance indicator in 2017-18 for when it inspects newly registered providers, which in quarter one it met for primary medical services providers but did not meet for adult social care providers. Registration is the point at which the Commission assesses a provider’s potential to provide a good-quality service; inspection provides assurance about the actual quality of services. For 2017-18, the Commission has introduced a key performance indicator of 90% (100% for adult social care) of newly registered locations to be inspected within specified time periods based on the date of registration. Figures for quarter one 2017-18 show that 100% of newly registered primary medical services providers received a first inspection within the target timescales, with 94% of newly registered adult social care providers inspected within targets. In this quarter, 49% of the primary medical services and 42% of adult social care first inspections were undertaken within one year of the provider being registered (paragraphs 2.3 and 2.4).

10 The Commission’s systems for bringing information together on the quality of services are not supporting inspectors effectively. Inspection staff must deal with a high volume of both centrally collated data and local intelligence to assess the quality of services and decide what action to take. A major part of the Commission’s plan to improve its central support for inspectors is its new Insight model. This brings together key indicators on provider performance and service quality. Our focus groups with inspection staff highlighted concerns about how well the broader information systems currently supported them, in particular the main system (the Customer Relationship Management system) used to collate information. Our focus groups and interviews with inspection staff and local stakeholders also highlighted variations in local working relationships and information-sharing (paragraphs 2.6 to 2.12).

11 Most providers and inspectors think that the Commission’s judgements are fair but some stakeholders have concerns about consistency. The Commission’s surveys show that the majority of providers and inspectors think inspection judgements and ratings are fair. However, some stakeholders we spoke to raised concerns about inconsistency in the Commission’s regulatory judgements. They cited examples of individual inspectors being subjective or inconsistent and questioned the consistency and profile of ratings within and across sectors. The Commission seeks to address consistency issues through its quality assurance processes and training (paragraphs 2.15 and 2.16).

12 The Commission has not met its timeliness targets for publishing reports. Across the three sectors it regulates, the Commission did not meet its overall target of publishing 90% of inspection reports within 50 days during 2016-17 and quarter one 2017-18, although performance did improve. The biggest gap between target (90% in quarter one 2017-18) and performance (25%) was for hospitals with less than three core services. It is taking a range of actions to improve its performance (paragraph 2.17).

13 The Commission introduced new key performance indicators in 2017-18 for when it re-inspects providers, which it met for primary medical services and did not meet for adult social care in the first quarter. The Commission introduced a key performance indicator in 2017-18 to undertake 90% of re-inspections within agreed maximum time periods. Performance in quarter one 2017-18 against the new performance indicator was 84% for adult social care and 93% for primary medical services. Prior to this, it committed to re-inspect adult social care providers within specified guidelines for 2016-17, of which 84% were undertaken in line with the guidelines (paragraph 2.18).

14 The Commission meets its target for referring safeguarding alerts within one day, but does not meet its timeliness target for taking further action. The Commission receives a significant volume of concerns about the quality of services from people using services, carers and staff (whistleblowers). Safeguarding alerts are particularly important as the Commission is the first organisation to receive the information, and the Commission met its target for referring safeguarding alerts to the appropriate authority within one day during 2016-17 and quarter one 2017-18. However, although performance is improving, it missed its 95% target for taking further action within five days following a safeguarding alert or concern during 2016-17 and quarter one 2017-18 (89% in quarter one 2017-18) (paragraphs 2.19 to 2.21).

15 The Commission increasingly takes action when care falls below fundamental standards. The number of completed enforcement actions increased over 2015-16 and 2016-17, while the number of providers entering special measures remained steady. The Commission links the increase with a focus on improving its inspectors’ skills and knowledge about enforcement. However, poor recording means the Commission cannot be assured that enforcement action is always completed (paragraph 2.22).

Encouraging providers to improve

16 There is evidence that the Commission is influencing providers to improve. Over the course of 2016-17 and quarter one 2017-18, most of the providers rated either ‘inadequate’ or ‘requires improvement’ improved their rating on re-inspection. The Commission does not provide direct support to providers to improve but seeks to influence quality through other routes. The Commission’s 2017 provider survey shows that most hospitals and adult social care providers think the Commission is helping them to improve, but GPs do not value the Commission’s regulation as highly (paragraphs 1.2, 2.23 to 2.25).

The Commission’s use of resources

17 The Commission is focusing more on savings and has a better understanding of its own costs. The Commission underspent on its budget between 2012-13 and 2016-17. Until 2016-17, this was mainly because it did not meet target staffing levels. In 2016-17 most of its underspend (£8 million out of £14 million) was made up of non-pay cost savings. For the first time since 2011, the Commission reduced its year-on-year spending, by £13 million, or 6%. The Commission has a better understanding of its own costs, through developing a more comprehensive costing model. The Commission is also moving to recover the full cost of its core regulation activities from provider fees, and will need to manage future relationships with providers carefully (paragraphs 3.2 to 3.5).

18 The Commission’s staff vacancy rates have fallen. The Commission has run successful recruitment campaigns including the ‘project 600’ campaign for inspectors. By the end of June 2017, vacancy rates were 6% for inspectors, 0% for inspection managers and 16% for senior analysts, compared with 34%, 35% and 36% respectively in April 2015. Overall staff turnover rates increased from 7.6% at the end of March 2015 to an average of 12.2% for the 12 months ending June 2017, so the Commission must continue to manage the risk of staff shortages (paragraphs 3.6 and 3.7).

Performance measurement and accountability

19 The Commission has improved how it measures its performance and takes action to correct poor performance. Since our 2015 report, the Commission has: introduced targets and baselines for its operational key performance indicators; published a report of its impact on quality and improvement; and identified a set of indicators to measure the impact of its new strategy. The Commission is transparent about its performance and publishes a wide range of information. We found examples where the Commission has taken action to correct poor performance, for example setting up the registration improvement programme (paragraphs 3.8 to 3.10).

20 The Commission is held to account appropriately by its board and the Department. A recent independent review of the Commission’s board concluded that it was effective. We assessed that the Department has put in place an appropriate framework for holding the Commission to account, although it will need to maintain an adequate level of oversight, challenge and support with reduced resources (paragraph 3.11).

21 The Commission has made reasonable progress against the Committee’s recommendations. In 2015, the Committee made six recommendations covering areas such as staffing, measuring performance, publishing inspection reports and engaging with people who use care and the public. The Commission has made good progress against three recommendations, good/adequate progress against one, adequate progress against one and adequate/poor progress against one (paragraph 3.11).

Developments in the Commission’s regulatory approach

22 The Commission made progress in implementing its new strategy during 2016-17, but missed some important milestones. During 2016-17, the Commission made good progress against many of the activities set out in its business plan. However, it missed milestones on rolling out use of resources assessments, designing its approach to the next phase of inspection and improving the way it collects information on providers (paragraphs 4.5 and 4.6).

23 Within the risks that the Commission has identified, we have highlighted three areas where effective management is particularly important for its implementation of a more intelligence-driven approach to regulation and meeting savings targets. These comprise:

programme management and governance of digital transformation;

aligning developments in collecting and collating information with other elements of the Commission’s strategy; and

maintaining flexibility to adapt to changes in the external environment. (paragraph 4.7)

24 The Commission is preparing for new ways that care might be delivered. New models of care could have implications for the Commission. For example, providers may work more in partnership and potentially change their legal structure. The Commission and the Department are confident that the current legislative framework is sufficiently flexible to cater for emerging new models of care. The Commission is taking action to ensure that it remains engaged with providers as new care models emerge. It is already responding to changes such as online primary medical services (paragraphs 4.8 to 4.12).

Conclusion on value for money

25 The Commission has improved as an organisation. It has completed its inspection and rating programme, which provides a benchmark of the quality of health and social care services. It has significantly reduced staff vacancies and is increasing its focus on cost savings. It has improved how it measures its performance. There is evidence that it influences providers to improve quality. There remain some concerns about the consistency of its regulatory judgements. Value for money is improving and the Commission can secure further improvement, if it continues its current direction of travel.

26 The Commission needs to overcome some persistent issues with the timeliness of some of its regulation activities if it is to sustain further improvement. Its ambition to base more of its regulatory activities on intelligence and risk-based information introduces significant challenges. These must be carefully managed and supported by digital systems and capabilities if it is to minimise the risk of missing poor care. The Department and Commission must also be realistic about its capacity to take on new responsibilities in this period of change. It must fulfil and improve upon its core responsibilities, in an environment of changing health and social care delivery, and continuing pressures on service quality.

Recommendations

27 Our recommendations are designed to reinforce the current actions the Commission is taking, and to help it refine and adapt its regulation approach.

28 The Commission should:

a Clarify key dependencies within its new strategy and the impact any delays in development might have on other aspects of its strategy. Two key areas are its development of its digital capacity and its work to develop information collection and systems.

b Ensure that digital systems effectively support inspection staff by bringing information together and helping to identify emerging risks to people’s care.

c Assess how inspection staff engage with other local stakeholders and share information. The results should be used to develop approaches that will support staff in improving local engagement and maximise local intelligence.

d Review the activities it currently uses to test and demonstrate consistency in inspection approaches and judgements. This review should include: discussions with providers, provider representative organisations and its own inspection staff to understand the concerns they have about consistency; and engagement with other regulators to understand how they approach this issue.

e Set out how it will get assurance that its inspection staff are taking consistent and appropriate decisions about regulatory action in response to intelligence. This might involve, for example, in-depth review of a sample of concerns or providers.

Part One

The Care Quality Commission

1.1 The Care Quality Commission (the Commission) is the independent regulator of health and adult social care in England. It began operating on 1 April 2009 and is a non-departmental public body accountable to Parliament, sponsored by the Department of Health (the Department). It is funded by grant-in-aid from the Department and fees charged to regulated organisations.

1.2 The Commission has two main purposes:

To make sure that health and social care services provide people with safe, effective, compassionate, high-quality care. This is addressed through the Commission’s core regulation functions (see Figure 1).

To encourage providers to improve the quality of care. The Commission does not provide direct support to providers to improve but seeks to influence quality through other routes.1 These include: making recommendations to providers following an inspection; highlighting good practice; and publishing reports and thematic reviews.2

1.3 In 2013, the Commission introduced its new regulatory model, which is built around five key questions to assess the quality of care and test whether providers are meeting fundamental standards of care (Figure 1).3 It tests providers at three stages: when it registers them; as it monitors and reviews performance data; and when it carries out inspections. After inspections, the Commission publishes its assessments and rates providers on a four-point scale (outstanding, good, requires improvement, inadequate).4 The Commission can also take enforcement action when care falls below the fundamental standards.

1.4 A major change involved appointing chief inspectors and creating specialist inspection directorates in each of the three sectors that the Commission regulates:

hospitals – including NHS acute, community and mental health hospitals; ambulance services; and independent sector hospitals;

primary medical services – including GP practices; GP out-of-hours services; dental practices; prison healthcare services; urgent care centres; and independent consulting doctors; and

adult social care services – including nursing homes; residential care homes; domiciliary care services; hospices; and supported living services.

1.5 We have reported on the Commission twice before, in 2011 and 2015.5,6 Our 2011 report highlighted the “considerable” difficulties the Commission was having in its transition from the merger of three former regulators. Our 2015 report focused on the Commission’s progress in implementing its 2013–2016 strategy and new regulatory approach. It found that the Commission had made progress in changing its regulatory model. However, challenges remained around its ability to assess its overall impact, establish a stable workforce, improve its data on regulated bodies and monitor its own performance. We intended to produce a later report to examine how the Commission’s regulatory model works in practice. Following the Committee of Public Accounts’ (the Committee) hearing and report in December 2015, the Committee expressed a wish to return to the subject to assess the progress the Commission was making.7

1.6 Since our 2015 report, there have been several developments in the way the Commission operates:

In May 2016, the Commission introduced its new five-year strategy for the period 2016–2021. Its overall ambition was to move to a more “targeted, responsive and collaborative approach” to regulation (see Figure 13, Part Four). The Commission will also be operating with a reduced budget (see paragraph 4.3), with an increasing proportion of funding from provider fees (see paragraph 3.5).

The Commission has taken on new functions and responsibilities, including:

Since April 2015, it has undertaken the market oversight of ‘difficult-to-replace’ providers of adult social care.

From April 2016, the Commission became responsible for assessing use of resources by NHS trusts (see paragraph 4.6).

Also from April 2016, the Commission is hosting the Office of the National Guardian to promote the freedom to speak up in the NHS.

In addition, Healthwatch England’s back-office functions have become more integrated with the Commission.

At the Department’s request, in 2017-18 the Commission will undertake system reviews in 20 local authority areas, examining the interface between health and social care. This is to better understand the pressures and challenges and identify any areas for improvements in the provision of health and social care within a local system, so that people using services are provided with safe, timely and high-quality care.

1.7 The changes to the Commission come at a time when health and social care providers are facing extremely high levels of demand and financial challenge, as set out in the Commission’s State of Care report and other publications.8,9

1.8 Based on the Commission’s inspection ratings, many health and social care providers continue to provide good care, with 82% of all providers (hospitals, adult social care providers and primary medical services providers) rated as either ‘good’ or ‘outstanding’, although there remains variation between sectors (Figure 2).

1 NHS Improvement provides support for NHS trusts and foundation trusts. NHS England and the Royal College of GPs support GP practices. There is no formal mechanism to support adult social care providers; local authorities may provide support to improve via commissioning and contract management.

2 For example: Care Quality Commission, Celebrating good care, championing outstanding care, March 2015.

3 While outside the scope of this report, the Commission performs a range of other regulatory activities. These include: independent voice – publishing its views on major quality issues in health and social care, for example through its annual State of Care reports; its responsibilities for monitoring and reporting on the use of the Mental Health Act 1983; and its joint inspection work with other regulators and inspectorates, such as Ofsted or HM Inspectorate of Prisons.

4 Rating powers do not apply to some inspected services, for example dental practices.

5 Comptroller and Auditor General, The Care Quality Commission: regulating the quality and safety of health and adult social care, Session 2010–2012, HC 1665, National Audit Office, December 2011.

6 Comptroller and Auditor General, Capacity and capability to regulate the quality and safety of health and adult social care, Session 2015-16, HC 271, National Audit Office, July 2015.

7 HC Committee of Public Accounts, Care Quality Commission, Twelfth Report of Session 2015-16, HC 501, December 2015.

8 Care Quality Commission, The state of health care and adult social care in England 2015-16, HC 706, October 2016.

9 See for example, Age UK, Briefing: the health and care of older people in England 2015, October 2015; Comptroller and Auditor General, Financial sustainability of the NHS, Session 2016-17, HC 785, National Audit Office, November 2016; Association of Directors of Adult Social Services, ADASS Budget Survey 2017, June 2017.

Part Two

Ensuring high-quality care and encouraging improvement

2.1 In this part we examine how the Care Quality Commission (the Commission) undertakes its core regulatory functions, designed to ensure high-quality care. We also examine whether the Commission is encouraging providers to improve quality.

Undertaking the Commission’s core regulatory functions

Registration

2.2 Following the publication of A fresh start for registration in 2015, the Commission introduced a registration improvement programme in 2016 to address a number of operational issues – including timeliness, the level of burden for different registration changes, and efficiency of the current process.10 This included: developing specialist sector teams; mapping the registration process; and improving management information. In the longer term, the Commission intends to move to a more risk-based approach to registration and a fully online system. However, there remain several issues:

We heard from a number of provider and staff representative organisations about frustrations with the bureaucracy of the registration process. In particular, the British Medical Association and Royal College of GPs cited the complexity of the process for relatively simple changes such as changes in the partnership at a GP practice.

Although performance is improving, the Commission consistently missed its own key performance indicator on completing registrations within 50 days during 2016-17 across all three sectors it regulates (Figure 3).

In quarter one 2017-18, 50% of registration enquiries were still rejected or withdrawn (down from 61% in quarter one 2015-16) at the first screening point within the Commission’s National Customer Service Centre, which handles initial registration enquiries. This represents wasted effort and cost for both the Commission and applicants. The rejection/withdrawal rate for enquiries made through the Commission’s online registration portal was lower than for emailed forms (34% compared with 55%). However, 74% of enquiries were still made via an emailed form. The share of portal enquiries increased from 7% in quarter one 2015-16 to 24% in quarter one 2017-18.

2.3 Registration is the first point at which the Commission assesses providers and their potential to provide a good quality of services, by looking at their systems, processes and premises and by interviewing applicants. In the Commission’s post-registration survey for January to March 2017, more than 90% of providers agreed the registration process provided a robust assessment of their ability to deliver a high-quality service. Inspection provides assurance about the actual quality of services, as inspectors can speak to staff and people who use services, observe care and review people’s records. During 2016-17, 33% of newly registered providers that received a first inspection were rated as ‘requires improvement’ or ‘inadequate’.11 The Commission told us that inspectors monitor provider risks using a range of information, which for newly registered providers would include any concerns highlighted by the registration team and establishing whether the location is brand new or taken over from an existing provider. Where there are concerns, inspectors can bring inspections forward.

2.4 The Commission has introduced a key performance indicator for 2017-18 of 90% (100% for adult social care) of newly registered locations to be inspected within specified time periods based on the date of registration (Figure 4). In quarter one 2017-18, 100% of primary medical services providers and 94% of adult social care providers receiving a first inspection were completed within target.12 In this quarter, 49% of the primary medical services and 42% of adult social care first inspections were undertaken within one year of the provider being registered.

Monitoring

2.5 The Commission has a long-standing objective to develop the way it uses information to identify risks to people’s care. As set out in Figure 13 (Part Four) a key priority for its 2016–2021 strategy is to deliver an intelligence-driven approach to regulation, including the development of its Insight model.

Information and intelligence sources

2.6 Inspection staff use a range of information and intelligence, including both centrally collated data and intelligence gathered from local sources, to assess the quality of services and prioritise activity (Figure 5).

2.7 The available information that the Commission collates centrally varies across and within sectors, both in terms of its scope and its timeliness:

For NHS hospitals, there is a wide range of information relating to the quality of services. In contrast, information about primary medical services, and particularly adult social care, is much more limited.

There is also variation across all sectors in how quickly different information is made available and therefore its usefulness in highlighting emerging risks to people’s care. For example, safeguarding alerts and concerns can be available quickly, while patient satisfaction surveys may only take place once or twice a year.

2.8 The information the Commission collates centrally has not changed significantly since we reported in 2015. The Commission’s work has found some weak linkages between indicators and ratings for a number of indicators across different sectors. The Commission supplements its indicators by analysing very high or low values. For example, in adult social care, inspectors can be alerted to a potential risk in care homes based on higher-than-expected death notifications from providers and other corroborating information.

2.9 The Commission is seeking to improve and increase the information available to it. Balanced against this, in its strategy it also committed to working with other regulators and oversight bodies to develop a “single shared view of quality”. It is taking a number of actions, for example:

Engaging with national bodies on initiatives that include enhancing and streamlining information. Initiatives include the National Quality Board, the National Information Board, the model hospital programme (coordinated by NHS Improvement); and the Quality Matters initiative in adult social care. For example, part of Quality Matters involves developing common measures of quality and a set of core data requirements to reduce the burden on providers.

Increasing the sources of information it has available. For example, it is using national clinical audit information relevant to hospitals’ core services.

Improving the way it collects information from providers. For example, the Commission is looking to move to an online system where providers can update their information regularly rather than only providing it before an inspection, with an initial roll-out to adult social care providers. In our 2015 report, we noted the Commission’s plans for improving provider information in adult social care. There has been little progress since then, and the current work to improve provider information collections is behind schedule (paragraph 4.6, Part Four).

Improving information about people’s experience of care. It has established ‘tell us about your care’ partnerships with a number of third-sector organisations (for example, The Patients Association), which now pass on information of concern to the Commission. These were being piloted at the point of our 2015 report. In addition, the volume of ‘share your experience’ information provided by the public through the Commission’s website has increased by 21% between 2015-16 and 2016-17. Generally, public (prompted) awareness of the Commission’s role increased significantly from 22% in 2012 to 51% in 2016, according to the Commission’s annual public awareness survey, although it has reduced a little since 2014 (55%).

2.10 Our focus groups with local inspection and Healthwatch staff, and interviews with regional NHS England and NHS Improvement teams, emphasised the importance of good local intelligence and strong relationships and how local intelligence can prompt action such as bringing forward inspections.

2.11 The Commission has established a national policy for inspection teams to use for their engagement work with local stakeholders. However, we found variations in local intelligence-gathering, for example:

Our focus groups with inspection staff highlighted variation in information-sharing between the Commission and other bodies such as clinical commissioning groups and local authorities. For example, available information from local authorities can vary depending on the level of activity of their contract compliance teams.

Our interviews with regional NHS England and NHS Improvement teams indicated that there were generally good working relationships and information-sharing with the Commission and Commission staff were generally engaged in their local quality surveillance group. More generally, a recent NHS England review highlighted that groups can vary in terms of who attends, the level of information-sharing and their assessment and response to risk.13

An important partnership is with local Healthwatch organisations, as they gather local people’s views on services. In Healthwatch England’s 2016 survey, 65% of local Healthwatch organisations said that they had a very or fairly good relationship with their local inspection team, while 55% felt that the Commission used their information to inform its work. Our focus groups indicated that relationships varied greatly between local inspectors and Healthwatch staff. In 2016, the Commission, Healthwatch England and local Healthwatch organisations co-produced guidance for working together.

Bringing together information to identify risks to people’s care

2.12 Local inspection staff are responsible for assessing all the information and intelligence they receive to identify emerging risks and decide what action to take with individual providers. With the volume of information the Commission receives, it is vital that it has effective systems in place to support its inspectors to bring together different sources of information. Figure 6 sets out the main ways the Commission draws information together for inspectors, the effectiveness of these arrangements and the plans for future development.

2.13 The Commission receives a large amount of qualitative (text-based) information, such as information from people on their experience of care and the narrative sections of notifications from providers. This information is difficult to analyse. The Commission manually assesses some lower-volume information, such as coroner’s reports. However, most of this information is not analysed centrally or brought together to support other information. In February 2016 the Commission purchased software to help it analyse large amounts of text. It is currently training staff to use this software, which will be installed by August 2017, with data uploaded in stages through to early 2018.

Inspection and rating

2.14 In 2014-15, the Commission set out its intention to inspect and rate all NHS hospitals, adult social care providers and GP practices.14 This would provide a baseline of the quality of services across England. In January 2017, the Commission completed the programme, which comprised comprehensive inspections at more than 28,000 provider locations between October 2014 and January 2017. Although the Commission missed earlier target completion dates due to difficulties in recruiting inspectors, it completed the programme in line with its 2016-17 business plan commitments (Figure 7).

2.15 The majority of stakeholders that we interviewed raised concerns about inconsistency in the Commission’s regulatory judgements.15 They cited examples of individual inspectors being subjective or inconsistent. Others questioned the consistency and profile of ratings within and across sectors. Other evidence shows a mixed picture on consistency:

In the Commission’s post-inspection survey (January to March 2017), 85% of adult social care providers, 75% of primary medical services providers and 73% of hospitals agreed that the inspection judgement was fair and based on the evidence.

In the Commission’s survey of inspection teams in April 2017, 60% of inspectors stated that, completely or to a large extent, inspection judgements and ratings were good quality and consistent, and 57% that there is consistency in approach from inspection to inspection.

2.16 The Commission has specific measures in place to address consistency:

It provides training for inspectors – in its inspection team survey for quarter two 2016-17, 74% of inspectors said they completely, or to a large extent, had an appropriate level of training. It has developed key lines of enquiry and guidance for inspectors.

Each inspection directorate has its own quality assurance process to review reports and ratings. For hospitals, a national panel reviews all reports. Between October 2016 and February 2017, this panel reviewed 2,638 ratings and changed 9% of ratings: of these, 59% were downgraded and 41% upgraded. In adult social care, due to the larger number of inspections, reviews focus on reports with either ‘outstanding’ or ‘inadequate’ ratings.

The Commission has a quality framework supported by a quality sampling process, which covers different aspects of the Commission’s regulatory activities, including inspections.

2.17 There remain issues with the timeliness of publishing inspection reports. Inspectors give feedback to providers on areas of concern immediately after an inspection. However, delays in publishing reports mean that information is not provided to the public in a timely way. During 2016-17 and the first quarter of 2017-18, the Commission consistently missed its own key performance indicator for publishing reports although performance did improve (Figure 3). Compared with a target of 90% for publishing reports within set times, the quarter one 2017-18 figures were 83% for adult social care and 64% for primary medical services; the equivalent figures for hospitals with less than three core services were 25% and for NHS hospitals with three or more core services were 38%. Reasons for the delays included: inefficiencies in the process; time taken to process factual accuracy comments from providers; delays because of enforcement actions; and issues with the technology to support report writing. The Commission is taking action to improve the timeliness and quality of inspection reports, including: introducing training in report writing; streamlining the quality assurance process; and redesigning reports. Further work is planned with an improvement plan presented to the Commission’s board in July 2017.

2.18 The Commission introduced a key performance indicator in 2017-18 to undertake 90% of re-inspections within agreed maximum time periods. Performance in quarter one 2017-18 against the new indicator was 84% for adult social care and 93% for primary medical services.16 Prior to this, the Commission did not have any set targets in place, although it committed to re-inspect adult social care providers within specified guidelines for 2016-17. During 2016-17, 84% of adult social care re-inspections were undertaken in line with these guidelines.

Responding to concerns and taking action

2.19 The Commission receives a significant number of concerns about the quality of services from a range of sources, including people who use services, carers and staff (whistleblowers). During 2016-17, the Commission received 153,000 contacts that it classified as relating to safeguarding issues and 7,452 contacts from whistleblowers.17 Safeguarding contacts have increased slightly (1%) since 2015-16, with whistleblower contacts falling by 16%.

2.20 During 2016-17 and quarter one 2017-18, while the Commission met its key performance indicator for referring safeguarding alerts to the safeguarding authority, it did not meet the indicator for taking further action, although performance is improving (Figure 3). The Commission met its internal target for passing on 95% of whistleblower enquiries to inspectors within one day during most of 2016-17, but missed its target in March, April and May 2017. The Commission stated that this was due to the introduction of a new operating model in its National Customer Service Centre. Performance was above target in June 2017.

2.21 In 2016, inspectors took a range of actions after receiving safeguarding alerts and whistleblowing enquiries.18 For 15% of safeguarding alerts, inspection staff took enforcement action, and for 11% of whistleblowing enquiries, they carried out a responsive inspection or brought forward a planned inspection. Most frequently, inspectors noted the issue for future inspections (47% of whistleblower enquiries and 29% of safeguarding alerts).

2.22 When care falls below fundamental standards, the Commission has powers to take enforcement action. This is underpinned by its enforcement policy and handbook for inspectors, including an enforcement action decision tool. The numbers of completed enforcement actions increased over 2015-16 to 1,910 in 2016-17 (Figure 8), while the number of providers entering special measures has remained steady. The mix of actions is also changing: the proportion of warning notices decreased from 76% of actions in 2015-16 to 71% in 2016-17 as the use of other actions (such as varying the conditions of registration) increased. The Commission links the increase in enforcement actions with a focus on improving its inspectors’ skills and knowledge about enforcement and the introduction of the decision tool to assist inspectors. The Commission’s quality assurance has raised concerns about how consistently inspectors apply the decision tool and the timeliness of enforcement action. It has also raised concerns about how well inspectors record each step in the enforcement process. In particular, poor recording of the latter stages means the Commission cannot be assured that enforcement action is always completed. In the Commission’s 2017 provider survey, 74% of adult social care providers and 72% of hospitals agreed that enforcement action is effective in encouraging compliance, compared with 42% of primary medical services providers.

Encouraging improvement

2.23 There is evidence that the Commission is influencing providers to make improvements. The Commission does not provide direct support to providers to improve, but seeks to influence quality through other routes (paragraph 1.2). One measure of its influence is whether providers improve their ratings on re-inspection (although other factors such as finances and commissioning decisions will also play a part). Figure 9 shows that, over 2016-17 and quarter one 2017-18, most providers rated ‘inadequate’ or ‘requires improvement’ improved their rating on re-inspection.

2.24 In the Commission’s 2017 provider survey, most adult social care and hospital providers thought that the Commission’s inspections or inspection reports helped them to identify or make improvements. This was only the case for a minority of primary medical services. In June 2017, the Commission published the results of eight case studies of NHS hospital trusts with a significant improvement in ratings. The report suggests that the Commission’s inspections help improvement by: identifying problems; helping trusts develop improvement plans; and giving a rigour and discipline to improvement work.19 During our stakeholder interviews, we heard a range of views:

Most provider representative organisations thought that the Commission’s ratings were important and that providers wanted to improve their ratings.

Adult social care provider representative groups felt that there was not enough support generally to help providers in their sector to improve. They contrasted this with the level of support available to the NHS.

Our discussions with the Royal College of GPs and the British Medical Association indicated that there remains a significant issue with how GP practices perceive the value added from the Commission’s regulation. These concern the cost-effectiveness and perceived administrative burden of the current approach and also that the Commission is not looking at the right things to make judgements about the quality of GP services.

2.25 The Commission’s report on its impact cited further evidence that it influences providers.20 For example, providers are put into special measures when they receive a rating of ‘inadequate’.21 Of the 551 providers exiting special measures across hospitals, adult social care and primary medical services in 2016, 386 had improved sufficiently to come out of special measures, 93 were de-registered and 72 had their registration cancelled. Improvements are often achieved with support from organisations such as NHS Improvement (see footnote 1).

10 Care Quality Commission, A fresh start for registration, August 2015.

11 Providers registered after 1 October 2014 and inspected in 2016-17.

12 The Commission counts those inspections that are late within a 10% tolerance level as meeting this key performance indicator. The Commission has included this tolerance as it is the first year of frequency-based inspections and this allows inspectors flexibility to respond to risk and make changes to schedules. Without this tolerance, 75% of adult social care and all primary medical services inspections would have met the target in quarter one 2017-18.

13 NHS England, National Quality Board review of Quality Surveillance Groups and Risk Summits – final report, June 2017. Quality surveillance groups were established in 2013 to bring local organisations together with the aim of identifying risks to the quality of care as early as possible.

14 Registered as at October 2014.

15 Comprising provider, staff and commissioner representative organisations and providers.

16 The Commission counts those re-inspections that are late within a 10% tolerance level as meeting this key performance indicator. The Commission has included this tolerance as it is the first year of frequency-based inspections and this allows inspectors flexibility to respond to risk and make changes to schedules. Without this tolerance, 63% of adult social care and 90% primary medical services re-inspections would have met the target in quarter one 2017-18.

17 Safeguarding means protecting people’s health, well-being and human rights, and enabling them to live free from harm, abuse and neglect.

18 The Commission defines a safeguarding alert as when it is the first body to be informed of a safeguarding issue.

19 Care Quality Commission, Driving improvement – Case studies from eight NHS trusts, June 2017.

20 Care Quality Commission, Review of CQC’s impact on quality and improvement in health and social care, April 2017.

21 The Commission’s Chief Inspector of Hospitals will normally recommend that an NHS trust or foundation trust is placed in special measures when it is rated ‘inadequate’ in the ‘well-led’ key question and ‘inadequate’ in one or more of the other key questions. NHS Improvement formally places trusts in special measures.

Part Three

Using resources and measuring performance

3.1 This part of the report examines whether the Care Quality Commission (the Commission):

uses its resources effectively;

has enough staff to carry out its regulatory functions;

measures its performance effectively; and

is held to account.

The Commission’s use of resources

3.2 Historically, the Commission has underspent against its budget (Figure 10). Until 2016-17, most of the Commission’s underspend was on staff costs, with staffing below planned levels.

3.3 The Commission is focusing more on cost savings, introducing a cost reduction programme since our last report. In 2016-17, the Commission’s underspend against its budget was £14 million, with £8 million coming from non-pay cost savings. For the first time since 2011 the Commission reduced year-on-year spending, by £13 million (6%) compared with 2015-16. The largest year-on-year saving (£4.1 million) was on travel and subsistence. In addition, the Commission has seen improvements in inspectors’ productivity between 2015-16 and 2016-17. As a result, the unit cost of inspection has fallen.

3.4 Over the past two years, the Commission has gained a better understanding of its own costs. It has developed a more detailed costing model, which provides information on variations in cost between directorates and can break down costs between those funded by grant-in-aid versus provider fees. It plans further developments to allow, for example, comparisons of inspection costs in different parts of the country.

3.5 The Commission is moving to recover the full cost of its core regulation activities (excluding enforcement) from provider fees. The proportion of its funding covered by fees rose from 45% in 2015-16 to 66% in 2016-17 (and is projected to rise to 92% by 2019-20). The Commission will need to manage future relationships with providers carefully, as this remains a sensitive issue. In particular, our stakeholder interviews highlighted how important it will be for the Commission to demonstrate its value for money and the value of its regulatory activities, and some dissatisfaction with communications and consultations around fees.22 If there are future underspends on fee-funded activities, the Commission told us it plans either to reduce fees in future years, use the surplus to avoid future fee increases, or invest in projects with demonstrable added value to providers.

Staffing

3.6 The Commission has made progress in recruiting staff. Figure 11 shows a significant fall in the vacancy rates for inspectors, inspection managers and senior analysts since our last report. At the end of June 2017, vacancy rates were 6% for inspectors, 0% for inspection managers and 16% for senior analysts. These rates compared with 34%, 35% and 36% respectively in April 2015.23 The Commission has run successful recruitment campaigns, including the ‘project 600’ campaign to recruit 600 additional inspectors. There are areas where recruitment remains a challenge; for example, the Commission has to compete with the private sector when recruiting analysts. It is looking to bridge the differential in salaries through initiatives such as a graduate analyst scheme. The Commission recognises that analysts are increasingly important as it moves to a more intelligence-driven approach to regulation.

3.7 Staff turnover rates for the Commission as a whole have increased from 7.6% at end of March 2015 to an average of 12.2% for the 12 months ending June 2017 with the turnover rate for inspectors at around 9%. The Commission’s planned turnover rate is 10%. Increasing turnover of staff means the Commission will need to continue to manage the risk of staff shortages. The Commission’s own analysis shows that the number of people leaving in the first two years following recruitment and the disengagement of longer serving staff are key issues. The Commission recognises that, while turnover will help it to reach staff reduction targets, there are risks around losing staff in specific roles and geographic areas.

Performance measurement

3.8 The Commission has improved its performance measurement since our 2015 report:

Since 2015-16, the Commission has used a broadly consistent set of indicators to measure its operational performance with a small number of changes to reflect changing business priorities. All indicators now have a baseline and target, compared with the small number that had these when we reported in 2015. The Commission’s board monitors performance with additional scrutiny from the Audit and Corporate Governance Committee and Regulatory Governance Committee.

In April 2017, the Commission published a report on its impact on the quality of services and encouraging improvement.24 While the report shows positive evidence of impact, the Commission sees this as the starting point of a longer-term process to understand its impact.

To measure the impact of its new strategy, the Commission has identified a set of existing and new indicators. These include, for example, the number of services that improve their ratings on re-inspection.

3.9 The Commission is transparent about its performance. It publishes the minutes and papers from, and recordings of, its monthly board meeting on its website, including a monthly report on performance. Each quarter, it publishes a more extensive performance report and has improved the information within this report over time.

3.10 In our fieldwork, we identified a number of examples where the Commission has aimed to learn from its past performance and taken steps to improve, including: the registration improvement programme; timelier publication of inspection reports; and work to understand poor staff survey results. Figure 12 provides one example.

Holding the Commission to account

3.11 This section looks at the Commission’s accountability arrangements across three areas:

Board accountability

A recent independent report concluded that the Commission’s board was effective and that a small number of changes in operation would improve its effectiveness further.25

Departmental accountability

Using a National Audit Office framework setting out principles for assessing oversight arrangements, we assessed that the Department of Health (the Department) has put in place an appropriate structure for holding the Commission to account. The key challenge for the Department is how to maintain adequate levels of oversight, challenge and support to the Commission, as the capacity of its sponsorship team reduces (by approximately 30%) as part of wider departmental restructuring.

Parliamentary accountability

The Commission tracks progress against the Committee of Public Accounts’ (the Committee) recommendations, with each recommendation owned by an executive director. Of the six recommendations that the Committee made in 2015, the Commission has made good progress against three, good/adequate progress against one, adequate progress against one and adequate/poor progress against one (see Appendix Three).

22 Comprising provider, staff and commissioner representative organisations and providers.

23 Comptroller and Auditor General, Capacity and capability to regulate the quality and safety of health and adult social care, Session 2015-16, HC 271, National Audit Office, July 2015.

24 Care Quality Commission, Review of CQC’s impact on quality and improvement in health and social care, April 2017.

25 Deloitte, Care Quality Commission: Independent review of Board effectiveness – Final Report, April 2017.

Part Four

Developments in the Commission’s regulatory approach

4.1 This part examines how the Care Quality Commission (the Commission) is preparing for the future. It looks at the implementation of and risks to the Commission’s new strategy and how it is preparing for new ways of providing care.

The Commission’s 2016–2021 strategy

4.2 In May 2016, the Commission introduced its five-year strategy for 2016–2021. It aims to deliver “a more targeted, responsive and collaborative approach to regulation”, including more targeted use of inspections and more reliance on intelligence (Figure 13). The strategy does not fundamentally change the Commission’s purpose, role or regulatory model.

4.3 The Commission’s budget will reduce by 13% from £249 million in 2015-16 to £217 million in 2019-20, and overall staff numbers by approximately 300 (9%) between 2016-17 and 2019-20. It is managing the reduction through a range of measures, including not filling vacant posts. It is also introducing new roles (for example, assistant inspectors) based on the capabilities and skills it thinks are needed to deliver its strategy. The Commission’s internal audit report on the cost reduction programme recognised it was achieving savings targets, but would require stronger governance and operational processes in future as making savings becomes more difficult. In response the Commission has established a board-level Finance Committee and Workforce Planning Group.

4.4 Responses to the Commission’s consultations on the strategy show broad agreement with its main aims and direction of travel.26 Between December 2016 and February 2017, the Commission also consulted on the next phase of its regulatory model including: principles for regulating new models of care; changes to its assessment frameworks; and changes to its regulation of NHS hospital trusts. An independent review of responses found that, overall, most respondents were supportive of the proposals and thought that changes could improve regulation and ultimately service quality.27 Some respondents raised issues around: the clarity of proposals and implementation plans; how the Commission would ensure consistency, transparency and flexibility in its regulation; reducing the provider burden; and the importance of the Commission working with others, including commissioners.

Progress to date

4.5 The Commission has developed three-year implementation and finance plans, setting out key milestones and saving plans for 2017-18 to 2019-20. An internal audit report on the implementation plan highlighted that it would need updating to cover all the ambitions set out in the strategy and to include the required cultural changes and any future digital upgrades.

4.6 The Commission’s 2016-17 business plan set out more than 60 activities, with around half directly related to implementing its strategy. Our review of the Commission’s documents and interviews with its staff indicate that it has made good progress against many of these activities, such as development of the Insight model. However, there are some important areas where the Commission has not met deadlines:

Use of resources assessment

The target date for finalising its approach was January 2017. It carried out a joint consultation with NHS Improvement in December 2016, with piloting scheduled up to December 2017. The general election delayed this work further. In August 2017, the Commission indicated that it would produce a combined quality and use of resources rating from 2018, following further consultation in autumn 2017. Consultation responses indicated that 65% of respondents supported combining ratings but some had concerns about diluting the existing quality rating and the increased complexity of the new overall rating.28

Design approach to next phase of inspection

The target date for publishing its approach for each sector was March 2017. It held consultations in December 2016 (hospitals) and June 2017 (adult social care and primary medical services). Roll-out is scheduled for 2017-18. The general election delayed this work further.

Improving information management and technology systems including provider information collections

The target date for this work was March 2017. The new chief digital officer has reviewed and reshaped plans for this work, which will be presented to the Commission’s board in October 2017.

Risks to delivery of the strategy

4.7 The Commission has identified and monitors risks to delivering its new strategy as part of its overall risk management process. Within the risks it has identified, our work has highlighted three areas where we think effective management will be particularly important for the Commission’s implementation of a more intelligence-driven approach to regulation:

Programme management and governance of digital transformation

Key assumptions in the strategy rely on improvements to the Commission’s digital technology and systems. For example, planned reductions in hospital inspection times (from 15.5 days in 2016-17 to 12 days in 2019-20) assume that the Commission will improve its electronic capture and storage of information, and make the inspection report-writing process more automated.

The Commission recognises the need to strengthen its digital capability and is planning to move to an agile development approach. It has made a number of senior appointments, including a new chief digital officer in January 2017, who was appointed jointly with NHS Improvement. In April 2017, the new officer’s assessment of the Commission’s current digital systems highlighted the need for fundamental transformation to ensure that the Commission delivers its strategy successfully. Over the past year, the Commission’s rating of its strategic risk arising from technology has ranged between amber and red after mitigation action. Our recent reports have highlighted that government bodies can struggle with the challenges of undertaking major digital programmes.29

Aligning developments in collecting and collating information with other elements of the Commission’s strategy

As set out in Part Two (paragraphs 2.5 to 2.13), there are still limitations in the Commission’s centrally collated information, variability in local information-sharing and issues with the systems the Commission uses to bring information together. There have also been delays in making improvements to provider information collections. We have not seen a clear description of the level of development required in, for example, the Insight model and collection of provider information, to allow the Commission to change the frequency and depth of its inspections (Figure 14). The Commission has told us that it is currently working on plans to align its intelligence outputs with changes in inspection frequency.

Maintaining flexibility to adapt to changes in the external environment

The Commission expects around half of its cost savings to come from reducing staff. This is based on a number of planning assumptions, including that the profile of ratings across providers will remain broadly unchanged. If ratings deteriorate, this will have implications for inspector numbers, as the number of inspections (see Figure 14) and enforcement actions required would increase. In planning staff requirements, the Commission has held workshops for senior teams in the inspection directorates to identify the implications of different scenarios (including changes in ratings), although we have not seen any detailed sensitivity analysis of key assumptions.

New models of care

4.8 NHS England’s Five Year Forward View set out the need for the NHS to develop ‘new models of care’, with greater emphasis on integrated services and out-of-hospital care.30 Five care models are being tested across 50 ‘vanguard’ areas: for example, integrated primary and acute care systems that join up GP, hospital, community and mental health services.

4.9 New models of care could have implications for the Commission. Providers will work more in partnership, changing how they deliver care and potentially their legal structure. The Five Year Forward View envisaged the development of ‘accountable care organisations’ which, in their most integrated form, involve a single organisation having responsibility for commissioning and providing health and social care services across a local area. The Five Year Forward View next steps document signalled the introduction of the first accountable care systems, in which local areas choose to take on clear collective responsibility for resources and population health in return for more control and freedom over the operation of the health system in their area.31 NHS England expects several accountable care systems and a small number of accountable care organisations to be established by 2018-19. In line with the NHS England Mandate, the Commission estimates that 50% of health and social care services will be delivered in different ways by 2020.

4.10 The Commission and the Department of Health (the Department) have reviewed the current legislative framework and are confident that it is sufficiently flexible to cater for emerging models of care. The Commission is, however, aware that there are challenges to its regulatory model, including: registering complex organisations appropriately, taking account of the organisation’s “guiding mind”; coordinating inspection teams when an organisation’s activities cover a range of sectors; and providing organisation-level ratings for complex providers. Other interviews with NHS England, NHS Improvement and vanguard sites highlighted the need for the Commission to remain flexible in its approach and consider how it will assess care delivered across pathways rather than by individual organisations.

4.11 Our interviews with NHS England, NHS Improvement and two vanguard sites indicate that the Commission is taking appropriate action to engage with the vanguard programme and keep abreast of developments. Actions taken by the Commission include:

publishing a ‘statement of intent’ that sets out its approach to new models of care and encourages innovation in local areas;

working with NHS England and NHS Improvement in local areas, with more advanced plans for accountable care organisations, to identify how to coordinate oversight and regulation activities;

sitting on NHS England’s new models of care advisory board, and establishing individual leads for each vanguard area;

consulting on proposals to revise its approach to regulating new and complex providers and to increase coverage of integration of services in inspections; and

reporting on health and social care services across local populations.

4.12 The Commission is adapting to other changes in the delivery of care. For example, as at June 2017 there were 39 providers of online primary medical services (where people can obtain a prescription and purchase medicines) registered with the Commission. Its initial inspections of 15 services raised concerns about quality, and the Commission took a range of enforcement actions. The Commission has now inspected all of these services.

26 OPM Group, Shaping the future – CQC strategy consultation, April 2016.

27 OPM Group, CQC’s next phase of regulation consultation, June 2017.

28 Care Quality Commission and NHS Improvement, Summary of consultation responses: use of resources and well-led assessments, August 2017.

29 For example: Comptroller and Auditor General, Early review of the Common Agricultural Policy Delivery Programme, Session 2015-16, HC 606, National Audit Office, December 2015.

30 NHS England, Five Year Forward View, October 2014.

31 NHS England, Next steps on the NHS five year forward view, March 2017.

Appendix One

Our audit approach

1 This report examines whether the Care Quality Commission (the Commission) is taking appropriate action to address the risks to people’s care. We reviewed:

the Commission’s current performance in ensuring high-quality care and encouraging providers to improve;

whether the Commission uses its resources effectively to undertake its regulatory functions; and

how the Commission is preparing to implement its new strategy and cater for longer-term changes in the delivery of care.

2 In reviewing these issues, we applied an analytical framework, with evaluative criteria, of the characteristics we would expect to find as evidence that the Commission is taking appropriate action to address risks to people’s care. Our audit approach is summarised in Figure 15. Our evidence base is described in Appendix Two.

Appendix Two

Our evidence base

1 We reached our independent conclusions on whether the Care Quality Commission (the Commission) is taking appropriate action to address the risks to people’s care after analysing evidence that we collected between December 2016 and June 2017. Our audit approach is outlined in Appendix One.

2 We carried out semi-structured interviews with senior officials at the Commission. This included the chief executive, the chair of the Commission’s board and the chief inspectors of hospitals, adult social care and primary medical services. We also interviewed executive directors and directors covering a wide range of the Commission’s activities.

3 We reviewed a range of the Commission’s documents, including:

published documents – performance reports; strategy and business planning documents; consultation documents; guidance to providers; board papers; public and provider engagement documents; and the accountability framework agreement with the Department of Health (the Department); and

unpublished documents – internal audit reports; quality assurance documents; planning documents; documents related to specific programmes and projects; and progress-tracking of actions against National Audit Office (NAO) and Committee of Public Accounts recommendations.

4 We conducted three focus groups with the Commission’s inspectors and inspection managers. We ran groups with inspection staff across the hospital, adult social care and primary medical services inspection directorates. The groups helped us to understand the sources of local information available to inspectors and its importance, and how inspectors work with other organisations locally to gather information. They also gave us feedback on various aspects of an inspector’s role, for example how effectively the Commission’s technology and systems support them.

5 We analysed the Commission’s data across a range of areas:

regulatory performance information: to understand the Commission’s current performance and trends against key performance indicators, for example the timeliness of publication of inspection reports and registration of new providers;

financial information: to understand past financial performance and management;

staffing information: to understand staff vacancy and turnover rates; and

activity information: to understand progress in completing the Commission’s inspection and rating programme; changes in the use of the Commission’s website; trends in the information the Commission has received from the public; the actions the Commission takes following receipt of safeguarding alerts and whistleblowing enquiries; and trends in enforcement actions and organisations in special measures.

6 We analysed the results from a range of surveys carried out by the Commission.

External surveys:

provider surveys for 2015 and 2017: we used this to understand the extent to which the Commission encourages providers to improve (in 2017 the Commission received 6,905 responses);

post-inspection and post-registration surveys for 2017: we used these to understand providers’ views on the Commission’s registration and inspection activities (for the post-registration survey, the Commission received 1,007 responses between January and March 2017, a response rate of 31%; for the post-inspection survey, the Commission received 1,317 responses between January and March 2017, a response rate of 25%);

website user satisfaction surveys for 2014 and 2017: we used these to assess user satisfaction with the Commission’s website (in 2017 the Commission received 874 survey responses);

inspection report user satisfaction surveys for 2014 and 2017: we used these to understand public satisfaction with, and usefulness of, the Commission’s inspection reports (in 2017 the Commission received 1,645 survey responses); and

public awareness surveys for 2012 to 2016: we used these to understand the levels of public awareness of the Commission and its role (in 2016 results were based on a sample of 1,000 members of the public).

Internal surveys:

inspection team surveys for 2014-15 and April 2017: we used these to understand how useful the data packs are for inspections and to understand how inspectors view the consistency of the inspection approach (in April 2017, the Commission received 502 responses from inspectors, a response rate of 36%).

7 We analysed the assumptions that the Commission used to plan its staffing requirements under its medium-term finance plan. This was a high-level review to assess the reasonableness of the Commission’s assumptions relating to inspection activity and inspection staff requirements.

8 We spoke to staff across the Department, NHS England and NHS Improvement. We conducted semi-structured interviews with a range of senior officials working both centrally and also in regional teams. Our interviews with the Department were based on an internal NAO framework to examine oversight arrangements. We used these to assess whether the Department has put in place appropriate accountability arrangements. We interviewed central NHS England and NHS Improvement staff to understand: the broad working relationships between the organisations; how they share information; how they are working together to understand the implications of new models of care; and how joint working in areas such as developing the use of resources assessment have progressed. Our interviews with NHS England and NHS Improvement regional staff were used to understand how local quality surveillance arrangements are working. We also reviewed documentation that we requested to support statements made during the interviews.

9 We interviewed individuals from a range of stakeholder organisations. These were used to get stakeholders’ views across a wide range of areas covered by this report. Interviews were carried out with:

provider representative organisations – NHS Confederation, NHS Providers, Care England, Registered Nursing Home Association, United Kingdom Homecare Association;

oversight and staff representative organisations – General Medical Council, the British Medical Association, the Nursing and Midwifery Council, the Royal College of GPs, the Royal College of Physicians, the Royal College of Nursing, the Local Government and Social Care Ombudsman, the Parliamentary and Health Service Ombudsman;

patient/service user representative organisations – Healthwatch England, Rethink Mental Health, Independent Age, the Patients Association, Carers UK;

commissioners’ representative organisations – NHS Clinical Commissioners, the Association of Directors of Adult Social Services; and

providers – Care UK, HC-One, Stewart Lodge Care Home, Diana Princess of Wales Hospital.

10 We conducted a focus group with local Healthwatch organisations. The focus group included representatives from four local Healthwatch organisations and was used to understand: the sources of local information available on the quality of services and its importance; how local quality surveillance groups are working; how local Healthwatch organisations work with the Commission; and the views of local Healthwatch organisations on the Commission’s performance.

11 We analysed the responses from Healthwatch England’s 2016 survey of local Healthwatch organisations. We used this to understand how the Commission is working with local Healthwatch organisations. Healthwatch England received responses from 120 of the 152 local Healthwatch organisations.

12 We spoke with senior staff from two new care model vanguard sites. One was a multispecialty community provider site, and the other was an integrated primary and acute care systems site. The interviews helped us to understand how well the Commission is engaging with vanguard sites and, more generally, preparing for potential changes in how care is delivered.

Appendix Three

Progress against Committee of Public Accounts’ recommendations

1 Figure 16 provides our assessment of the Care Quality Commission’s (the Commission’s) progress against the recommendations that the Committee of Public Accounts made in December 2015. The Commission agreed to all of the recommendations.

Design and Production by NAO External Relations
DP Ref: 11628-003