Report

by the Comptroller
and Auditor General

Department for Education

Follow-up on alternative higher education providers

HC 411 Session 2017–2019 18 October 2017

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund have used their resources efficiently, effectively, and with economy. Our studies evaluate the value for money of public spending, nationally and locally. Our recommendations and reports on good practice help government improve public services, and our work led to audited savings of £734 million in 2016.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 17 October 2017

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

13 October 2017

This follow-up report examines whether the Department for Education has made sufficient progress in improving its oversight and control of alternative higher education providers.

© National Audit Office 2017

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.gsi.gov.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

11499 10/17 NAO

The National Audit Office study team consisted of:
Heather Thompson, Sonia Sousa and Colm Molloy, under the direction of Siân Jones.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

Key facts

Summary

1 In December 2014, we published our Investigation into financial support for students at alternative higher education providers.1 The report examined the arrangements for oversight and funding of alternative higher education providers. These are institutions that offer higher education qualifications but that do not receive direct funding from the Higher Education Funding Council for England (HEFCE). They do have students who access public funds through student support. At September 2017, there were 112 alternative providers accessing student support funding.

2 Our 2014 report focused on four specific concerns:

students at some alternative providers had claimed support for which they were not eligible;

some providers had recruited students who did not have the capability or motivation to complete their courses;

some providers had recruited students in receipt of student support onto courses that the former Department for Business, Innovation & Skills (BIS) had not approved; and

some providers had given the former BIS inaccurate information about student attendance.

3 The Committee of Public Accounts held evidence sessions in December 2014 and March 2015. In February 2015, it published a report which made recommendations for strengthening oversight of the alternative provider sector.

4 In July 2016, the Department for Education (the Department) assumed responsibility for higher education, including the alternative provider sector. Previously, BIS (now the Department for Business, Energy & Industrial Strategy) had responsibility. In this report, we refer to the department with responsibility for higher education as ‘the Department’.

Scope and approach

5 This follow-up report focuses on the issues we and the Committee of Public Accounts reported on in 2014 and 2015. Specifically, we examine the following:

whether there are signs that the Department has resolved the major issues we identified in 2014 (Part Two);

whether the evolving oversight arrangements offer protection to taxpayers and students (Part Three); and

whether the Department has assessed the effectiveness of the policy of expanding the alternative provider sector (Part Four).

6 This report does not examine the full range of processes that the Department and its partner organisations use to oversee alternative providers or the financial sustainability of alternative providers. It focuses on students from England and the European Union who attend alternative providers and receive student support from the Student Loans Company (SLC).

Key findings

Size of the sector

7 The number of students at alternative providers receiving student loans has declined from a peak of 51,000 in 2013/14 to 34,000 in 2015/16. In November 2013, following rapid expansion in the number of students accessing support since 2010/11, the Department instructed 23 of the fastest-growing alternative providers to stop recruiting funded students. In 2014/15, the Department imposed additional student number controls. The total support paid to students has declined, from £724 million in 2013/14 to £417 million in 2015/16. The Department told us that it does not have a target size for the alternative provider sector as its aim is to manage a demand-led sector, by routinely using student number controls both to enable controlled growth in quality degree-level providers and as a sanction to limit the size of poorly performing providers (paragraph 2.9).

Ineligible payments

8 The level of ineligible payments has fallen as a proportion of all payments made to students attending alternative providers, from 4% in 2012/13 to 0.5% in 2015/16. The number of ineligible students receiving student support follows a similar trend. The Department attributes this reduction to new controls it implemented in response to the issues we raised in 2014 and analysing applications data on a daily basis to detect new problems emerging (paragraphs 2.2 and 2.3).

9 However, the Department does not on a regular basis measure and analyse the scale of or reasons behind why ineligible payments are made to students attending alternative providers, because of limitations in how the SLC records ineligible payments data. The Department considers that its annual audit to measure the overall rate of fraudulent or incorrect payments across the higher education sector is a proportionate and risk-based approach, given the drop in ineligible payments to students at alternative providers since our 2014 report. Nevertheless, improved management information and analysable data would help the Department measure the level of ineligible payments in a timely manner, explore what may be behind trends and further tighten controls where needed (paragraphs 2.4 to 2.7).

10 Furthermore, the Department has not recovered £36 million of the £45 million ineligible payments it identified in the six years to 2015/16. This total includes around £10 million of ineligible payments made in 2014/15 and 2015/16 (paragraph 2.6).

Non-continuation rates

11 The proportion of students who drop out of their studies (indicative non-continuation rate) has reduced over time.2 Between 2012/13 (the most recent data available when we reported in 2014) and 2014/15 (the most recent data currently available), the overall indicative non-continuation rate for the alternative provider sector reduced from 38% to 25% (paragraphs 2.10 to 2.12).

12 However, the indicative non-continuation rate at alternative providers is three percentage points higher than the sector benchmark, and 15 percentage points higher than for the rest of the higher education sector. The benchmark non-continuation rates take into account the characteristics of each individual provider’s student population such as age and subjects studied. For the alternative provider sector as a whole, the non-continuation rate in 2014/15 was 25% compared with a sector benchmark of 22%. The non-continuation rate for HEFCE-funded providers is 10%. These providers have fewer students with characteristics associated with high non-continuation (paragraphs 2.12 and 2.13).

13 The three largest providers have non-continuation rates above their benchmarks. The Department has used the benchmark non-continuation rates published by the Higher Education Statistics Agency (HESA) to identify under-performing providers. For students beginning their studies in 2014/15, 11 providers had non-continuation rates that were between five and 55 percentage points higher than their individual benchmarks. Ten providers had non-continuation rates that were between four and 17 percentage points lower than their individual benchmarks. The remaining providers either had non-continuation rates that were not significantly different from their benchmarks, or had too few students for meaningful measurement of non-continuation rates (paragraph 2.15).

14 The Department has taken action against all 11 providers with high non-continuation rates in 2014/15. To make consistent decisions on sanctions where providers had high non-continuation rates in 2014/15, the Department used a framework comparing providers’ non-continuation rates with individual benchmarks based on each provider’s student population. The framework also considered the absolute non-continuation performance of each provider. Of the 11 providers with high non-continuation rates relative to benchmarks, three are no longer designated for student support and the Department has issued improvement notices to the remaining eight. It issued an improvement notice to one further provider where its non-continuation rate was considered unacceptably high in absolute terms (paragraphs 2.18 and 2.19 and Figure 6).

15 More timely and reliable non-continuation data would enable the Department to impose sanctions more promptly. The Department uses end-of-year data on non-continuation rates as the basis for sanctions on under-performing providers. However, depending on when a student stopped attending, end-of-year data may not become available until more than a year after the student dropped out. Although the Department also uses termly attendance data recorded by the SLC to investigate issues emerging and to inform engagement with providers, these attendance data are not sufficiently reliable to be the basis of sanctions. A HESA project to improve timeliness by collecting reliable data during the academic year is expected to be fully implemented in 2019/20 (paragraph 2.21).

Student registration

16 The Department took action in five cases where new data-matching checks identified non-compliance with requirements to register students with the awarding body for Higher National courses, Edexcel. Between 2014/15 and 2016/17, the Department identified five non-compliant providers (out of between 18 and 35 providers within the scope of the data-matching). The Department suspended payments for four providers while checks were being undertaken, and issued one with an improvement notice (paragraph 2.25).

Quality of provision

17 The Department and partner organisations have taken action to address issues on quality of provision. Since 2015, all alternative providers must have had a successful Higher Education Review by the Quality Assurance Agency (QAA) in order to receive student support payments.3 The Department has revoked designation for student support payments from three providers on the basis of concerns raised by the QAA. In addition, since January 2015, the Department has investigated 14 providers where whistleblowers, data or engagement with partner organisations have raised concerns about quality of teaching (paragraphs 2.27 to 2.29).

Oversight of the alternative provider sector

18 Since we reported in 2014, the Department has taken steps to strengthen its oversight framework. In particular, providers’ designation for student funding is limited to one year and the Department undertakes annual checks to reconfirm designation. Previously, designation for student support was not time limited. Alternative providers must now meet quality review requirements that are consistent with those met by HEFCE-funded providers, and provide comparable data to HESA. The Department has also developed a range of sanctions that it imposes on under-performing providers and has expanded counter-fraud activity (paragraphs 3.2 and 3.3 and Figure 9).

19 In addition, the Department has increased its engagement with providers and its investigative capacity, and has taken action where it has had concerns. In 2015 the Department, along with HEFCE, established a joint Alternative Provider Intelligence Unit, which as at September 2017 had 16 staff members. In 2016, it developed risk-based engagement plans with all providers. Between January 2015 and November 2016, the unit considered 32 potential investigations, of which 31 are now closed. Overall, the Department has taken action against providers in more than 30 cases where its investigations or data on, for example, non-continuation rates, have identified providers who are not performing at expected levels (paragraphs 3.2 to 3.6).

Assessing whether government is achieving its policy objectives

20 The government’s policy of expanding the alternative provider sector appears to be widening access to under-represented groups. The Department is developing its data collection approach and does not yet have full data to allow it to evaluate the policy impact. Nevertheless, data for 2015/16 from HESA show that 55% of alternative provider students are over 24 years old (compared with 22% at HEFCE-funded providers) and 52% come from ethnically diverse groups compared with 22% at HEFCE-funded providers (paragraph 4.5).

21 However, early indications are that graduates who studied at alternative providers have lower rates of progression into employment or further study, and lower salaries than graduates of HEFCE-funded providers. In July 2017, the Destinations of Leavers from Higher Education (DLHE) survey included, for the first time, data for all alternative providers with undergraduate designated courses. For 2015/16, 84% of alternative provider graduates were working or studying six months after graduating, with a median salary of £20,500. In comparison, 90% of graduates who studied at HEFCE-funded providers were working or studying six months after graduating, with a median salary of £21,500 (paragraph 4.9).

22 Appendix Three presents the actions taken by the Department to address the recommendations the Committee of Public Accounts made in 2015 on the basis of our 2014 report.

Conclusion

23 The Department has made progress towards addressing weaknesses in its oversight of the alternative provider sector in response to our 2014 report and the Committee of Public Accounts’ 2015 recommendations. For example, the Department has strengthened its oversight framework and it has suspended or revoked payments to providers where intelligence from third parties or its own analysis has identified under-performing or non-compliant providers. In addition, there are indications that expansion of the alternative provider sector has helped widen access to higher education for unrepresented groups of students.

24 However, the Department still has important issues to address before it can provide assurance that it has a firm grip on current or emerging problems. While the Department has reduced the number of ineligible payments to students at alternative providers, it does not have sufficiently timely and specific data to allow prompt measurement of the level of ineligible payments or analysis of trends. The rate of students at alternative providers who drop out of their courses has fallen, but it remains much higher than in the rest of the higher education sector. Furthermore, the Department also lacks reliable and timely data on non-continuation rates so does not impose prompt sanctions on under-performers.

Recommendations

25 The Higher Education and Research Act, which received Royal Assent in April 2017, will change the way higher education is regulated. In future, English higher education providers, including alternative providers, will be registered by the new Office for Students (OfS). The OfS will take a risk-based approach to regulation, focusing its attention on providers which represent most risk to student interest and value to the taxpayer.

26 Our recommendations are aimed at supporting the Department and the new OfS as part of continuing efforts to take a data-driven approach to manage the alternative provider sector and to develop a data strategy for alternative providers, a group where some providers present risks. As responsibility for regulating alternative providers transfers from the Department to the OfS, there will continue to be a need for timely, reliable and specific data that allows early warning of issues emerging in the sector. We recommend:

a As part of wider action to improve the SLC’s information systems, the Department should work with the SLC to strengthen analysis of applications and payments data for students attending alternative providers to support early detection of issues emerging and enable analysis of the reasons ineligible payments are made, so that controls can be tightened where needed.

b The Department should work with HESA to ensure that its project to improve data on higher education providers will result in timely and reliable in-year non-continuation data that will allow the OfS to impose prompt sanctions on under-performing providers and on providers that do not submit accurate and timely data.

c In developing and implementing the new regulatory framework, the OfS should drive further reduction in non-continuation rates. It should take steps to address the underlying reasons why students with certain characteristics and backgrounds are less likely to complete their studies, by encouraging providers to take appropriate action and make effective support available to students at all providers where evidence suggests they are less likely to complete their studies.

d The Department should consider using alternative sources of data and new techniques to promptly identify problems in the sector. For example, the OfS could develop the National Student Survey further or work with HESA to develop methodologies that allow it to identify specific or emerging issues with quality.

1 Comptroller and Auditor General, Department for Business, Innovation & Skills, Investigation into financial support for students at alternative higher education providers, Session 2014-15, HC 861, National Audit Office, December 2014.

2 The Department calculated indicative non-continuation rates using similar methodology to that used by the Higher Education Statistics Agency when publishing non-continuation rates for HEFCE-funded providers.

3 The QAA is a not-for-profit body independent of government that monitors and advises the public on standards and quality in UK higher education.

Part One

Background

1.1 Higher education in England is provided by institutions that receive a proportion of funding from the Department for Education (the Department) via the Higher Education Funding Council for England (HEFCE). They are referred to in this report as ‘HEFCE-funded providers’. Higher education is also provided by institutions that do not receive any direct funding from HEFCE but may receive substantial public funding through student support. These institutions are referred to as ‘alternative providers’. As at September 2017, there were 112 alternative providers accessing student support funding.

1.2 Students in higher education can apply for student support, provided from public funds by the Student Loans Company (SLC). Student support from SLC is available only to students enrolled on courses approved or ‘designated’ by the Secretary of State. Eligible courses at HEFCE-funded providers are automatically designated, while alternative providers must apply to have their courses designated for student support. Each designation at an alternative provider relates to a specific course delivered at a specific location.

1.3 On 14 July 2016, the Department for Education assumed responsibility for higher education taking over from the then Department for Business, Innovation & Skills (which has since become the Department for Business, Energy & Industrial Strategy). In this report, we refer to the department with responsibility for higher education as ‘the Department’.

Oversight of alternative providers

1.4 Alternative providers need to meet certain requirements to access public funding (Figure 1).

1.5 In June 2011, the government published a white paper that introduced measures designed to encourage diversity and competition in the higher education sector.4 Details of the resulting growth in the number of students at alternative providers receiving student support can be found in our 2014 report, Investigation into financial support for students at alternative higher education providers.5

1.6 Since 2011, the expansion of the alternative provider sector has led to concerns about whether the current regulatory arrangements are fit for purpose. In its 2011 report on regulating higher education, the Committee of Public Accounts recommended that the Department should provide new powers to regulate institutions that receive little or no direct public funding, but whose students have access to publicly provided loans.6 However, in 2012 the Department informed the Committee of Public Accounts that it would work within existing legislation rather than bring forward new legislation, and took steps to strengthen its processes for designating courses.

1.7 Although the Department still has limited regulatory powers over alternative providers, it has made access to premises, records and staff following fair notice a condition of designation for student support funding, with immediate access if it suspects fraudulent activity.

1.8 In 2016, the government introduced a new Higher Education and Research Act that received Royal Assent on 27 April 2017. It will dissolve HEFCE and establish a new regulatory framework for higher education. In particular, a new Office for Students (OfS) will take on responsibility for funding and regulating higher education providers. The OfS will regulate all registered English higher education providers and therefore the Department will stop directly regulating alternative providers.

Concerns about support provided to students at some alternative providers

1.9 Our 2014 report examined the arrangements for oversight and funding of alternative higher education providers. It established a number of issues:

European Union students at some alternative providers had claimed or attempted to claim support they were not entitled to.

In 2012/13, non-continuation rates at nine alternative providers were higher than 20%, compared with an average non-continuation rate of 4% in the rest of the higher education sector.

About 20% of students enrolled in Higher National courses at alternative providers and claiming student support did not appear to have been registered with the qualification awarding body.

There were instances of students on undesignated courses receiving support payments and providers supplying the SLC with incorrect information about student attendance.

1.10 In December 2014 and March 2015, the Committee of Public Accounts questioned the Accounting Officer about the evidence in the report. In February 2015, it published a report which recommended that the Department should:7

systematically assess and control the specific risks identified by the National Audit Office and at its evidence session, and provide the Committee with a clear explanation on how it will manage these risks in future;

report back to the Committee with an assessment of how much public money is at risk of being wasted;

ensure that it has a much firmer grip on the quality of teaching and the standard students can expect in private sector higher education colleges, identify poor performers and take appropriate action to protect students and the sector as a whole; and

set specific, measurable objectives for its policy [of expanding the alternative provider sector], and collect and analyse the right data in order to evaluate the full impact (see also Appendix Three).

1.11 This report follows up the issues we reported on in 2014 and the progress that the Department has made in implementing the Committee of Public Accounts’ recommendations. Specifically, we examine the following:

whether there are signs that the Department has resolved the specific issues we examined in 2014 (Part Two);

whether the evolving oversight arrangements offer protection to taxpayers and students (Part Three); and

whether the Department has assessed the effectiveness of the policy of expanding the alternative provider sector (Part Four).

4 Department for Business, Innovation & Skills, Higher Education: Students at the Heart of the System, white paper, CM 8122, June 2011.

5 Comptroller and Auditor General, Department for Business, Innovation & Skills, Investigation into financial support for students at alternative higher education providers, Session 2014-15, HC 861, National Audit Office, December 2014.

6 HC Committee of Public Accounts, Regulating financial sustainability in higher education, Thirty-sixth Report of Session 2010-12, HC 914, June 2011.

7 HC Committee of Public Accounts, Financial support for students at alternative higher education providers, Forty-first Report of Session 2014-15, HC 811, February 2015.

Part Two

Follow-up on major issues

2.1 This part of the report examines the current position on the major issues we identified in our 2014 report.

2.2 Since our previous report, the level of ineligible payments has fallen as a proportion of all payments to students attending alternative providers, from 4% in 2012/13 to 0.5% in 2015/16, in line with the rate of ineligible payments across the higher education sector (0.8%).

2.3 The Department for Education (the Department) attributes this drop to new controls to prevent ineligible payments. In particular, it requires additional evidence to confirm that students meet residency requirements, and has tightened controls to prevent providers from recruiting students claiming student support onto undesignated courses. In addition, the Student Loans Company (SLC) analyses applications data on a daily basis to detect new problems emerging and aims to prevent ineligible payments from being made. In 2016, the Department established the Higher Education Fraud and Error Resilience Board to strengthen counter-fraud capability.

2.4 The Department considers that it has a proportionate and risk-based approach to monitoring the level of ineligible payments across the higher education sector. In particular, it carries out an annual audit to measure the overall rate of fraudulent or incorrect payments made to all providers. It considers that it would be disproportionate to design a system specifically for measuring ineligible payments to alternative providers, not least because the proportion of these payments has fallen substantially.

2.5 Therefore, the Department does not on a regular basis measure or analyse the level of ineligible payments made to students at alternative providers. Limitations in how the SLC records ineligible payments to students at alternative providers mean that it cannot readily analyse these data. With improved management information, the Department would increase its capacity to measure the level of ineligible payments in a timely manner, explore what might be behind trends, or act promptly to further tighten controls if there is a spike in ineligible payments.

2.6 At our request, the Department analysed ineligible payments to students at alternative providers as at January 2017:

The SLC made ineligible payments totalling about £10 million in 2014/15 and 2015/16.

In the six years to 2015/16, the SLC made ineligible payments totalling £45 million.

The Department has not recovered £36 million of the £45 million ineligible payments in the six years to 2015/16.

2.7 The Department could not, without further analysis, tell us the reasons why £19 million was paid to ineligible students, mostly since 2013/14 (Figure 2). Although the SLC records the reason why each individual payment is ineligible, limitations in its information systems prevent it from routinely analysing the data without substantial data manipulation.

2.8 However, available evidence suggests a changing pattern in the reasons why ineligible payments to alternative providers are made. There are indications that payments to students who do not meet residency requirements is no longer the main reason for ineligible payments. Available data show that ineligible payments made to EU nationals as a proportion of all payments to EU nationals reduced from 13% in 2012/13 to 0.5% in 2015/16.

2.9 Since the Department imposed student number controls, student support payments and numbers of students at alternative providers have reduced. Support payments to full-time students fell from a peak of £724 million in 2013/14 to £417 million in 2015/16. The number of full-time students at alternative providers receiving student support also dropped from a peak of 51,000 in 2013/14 to 34,000 in 2015/16 (Figure 3). The Department told us that it does not have a target size for the alternative provider sector as its aim is to manage a demand-led sector, routinely using student number controls, both to enable controlled growth in quality degree-level providers, and as a sanction to limit the size of poorly performing providers.

Trend in non-continuation rates

2.10 Since we reported in 2014, HEFCE has developed an analysis of indicative non-continuation rates which addresses some of the limitations we noted at that time.8 HEFCE’s analysis identifies students who did not return in the following academic year and is broadly consistent with how HESA calculates non-continuation rates for HEFCE-funded providers. HEFCE indicative non-continuation rates use SLC data, whereas HESA uses data reported to it by each provider. HEFCE developed its analysis in the absence of HESA non-continuation data for alternative providers, which would not be available until March 2017. More information is available in Appendix Two.

2.11 HEFCE has also calculated the non-continuation rate which would be expected for the alternative provider sector given overall non-continuation rates and certain characteristics of the alternative provider student population, such as students’ age and the subjects they are studying. This benchmark rate allows the Department to understand the extent to which differences from the average can be explained by factors that are known to affect non-continuation.

2.12 Indicative non-continuation rates at alternative providers have reduced over time, on average, but remain higher than the benchmark rate for the sector. Since 2012/13 (the previous position we reported), the non-continuation rate for the alternative provider sector has dropped by 13 percentage points, from 38% in 2012/13 to 25% in 2014/15. However, the non-continuation rate in 2014/15 was three percentage points higher than the benchmark for the alternative provider sector (22%) (Figure 4).9

2.13 The benchmark indicative non-continuation rate is much higher for alternative providers (22%) than for providers funded by HEFCE (10%). This is because alternative providers have more students with characteristics associated with high non-continuation which has pushed up the sector benchmark rate in the alternative provider sector.

Non-continuation rates by provider

2.14 In March 2017, HESA published non-continuation data for alternative providers for the first time in line with data published for the HEFCE-funded sector.10 For each of 57 alternative providers, HESA published the actual non-continuation rate and the benchmark non-continuation rate that takes into account the characteristics of the individual provider’s student population (including age and subjects studied).11 The Department compared each provider’s actual performance with its benchmark to identify poorly performing providers.

2.15 The three largest alternative providers have non-continuation rates significantly worse than their benchmark rates. In 2014/15, analysis of 57 providers showed that:

10 providers (18%) had non-continuation rates significantly better than their benchmarks. Two of these providers had more than 500 undergraduate students starting courses. Differences from their benchmarks ranged from four to 17 percentage points (purple and green markers in Figure 5);

11 providers (19%) had non-continuation rates significantly worse than their benchmarks. Three of these providers were among the largest alternative providers. Differences from their benchmarks ranged from five to 55 percentage points (red and orange markers in Figure 5); and

36 providers (63%) had non-continuation rates that were not significantly different from their benchmarks.

2.16 In the HEFCE-funded sector, 12 providers (8%) had non-continuation rates significantly worse than their benchmarks in 2014/15, with differences ranging from three to 27 percentage points.

Sanctions imposed on providers with high non-continuation rates

2.17 The Department uses data on non-continuation rates as the basis for sanctions on under-performing providers. It issued improvement notices to 14 providers with high indicative non-continuation rates in 2012/13 and 2013/14.

2.18 To make consistent decisions on sanctions in relation to non-continuation rates for 2014/15, the Department used a framework comparing providers’ non-continuation rates with individual benchmarks based on each provider’s student population. The framework also considered the absolute non-continuation performance of each provider.

2.19 The Department has taken action against all 11 providers with non-continuation rates in 2014/15 that were higher than their individual benchmarks (Figure 6). In addition, the Department issued an improvement notice in one case where the provider’s non-continuation rate was not significantly different from its benchmark but was deemed unacceptably high in absolute terms.

2.20 The HESA data do not allow us to analyse and confirm the reasons for non-continuation in each case, which will differ by student and provider. Non-continuation rates reflect a range of potential factors:

Student characteristics such as age and subjects studied. These are taken into account by the benchmarking analysis.

Student characteristics such as students’ socio-economic background and working status. These are not taken into account by the benchmarking analysis.

Non-continuation rates may reflect factors besides differences in student characteristics. High non-continuation rates may also be an indication of the quality of the education and support at the institution, the capability and motivation of students, or inappropriate recruitment by the provider.

Limitations in using end-of-year data

2.21 The Department is not able to impose sanctions on under-performing providers promptly because it uses end-of-year non-continuation rate data as the basis of sanctions. Depending on when a student stopped attending, this end-of-year non-continuation data may not be available for more than a year after the student dropped out. This is because information on students who did not return for a subsequent year of their studies can be derived only after providers have submitted enrolment data to HESA at the end of that subsequent academic year. The Department has told us that HESA has a project under way to enable data collection during the academic year, which should help address reporting time-lags. Full implementation is expected for 2019/20.

2.22 The Department informed us that it uses termly data on attendance held by the SLC to monitor providers and investigate emerging issues. The Department also told us that these data are not sufficiently reliable to be the basis of sanctions, because they only capture in-year withdrawals, and because some providers are better than others at providing attendance data to the SLC in a timely manner.

2.23 In response to a recommendation from the Committee of Public Accounts, the Department has, since July 2016, required that the level of English language spoken by students attending funded courses is at least ‘competent user’. We checked compliance by reviewing the websites of a risk-based sample of 21 alternative providers and found that 19 were compliant. However, we identified two providers whose websites indicated they would recruit students whose English-language proficiency falls below ‘competent user’ (4th column of Figure 7). They are Mont Rose College and The London College UCK. In early 2017, the Department contacted the providers who said that their websites were in error and agreed to update their websites immediately.

2.24 Students at alternative providers attending designated courses and wishing to claim student support from the SLC must register with a qualification awarding body.

2.25 In 2015, in response to a recommendation from the Committee of Public Accounts, the Department put in place an annual exercise to check whether students claiming student support for Higher National courses were registered with the awarding body Edexcel. In 2015/16, one-third of the student population at alternative providers were taking Higher National qualifications. The Department took action in the five cases where data checks identified discrepancies in the three academic years to 2016/17 (out of between 18 and 35 providers within the scope of the data-matching). The Department suspended payments to four providers while checks were being undertaken and issued one with an improvement notice.

2.26 The Department does not perform any checks to determine whether supported students are registered with awarding bodies other than Edexcel. Around two-thirds of undergraduates at alternative providers in 2015/16 attended courses other than Higher Nationals which are qualified by, for example, City & Guilds or HEFCE-funded providers. These students must register with the relevant awarding bodies in order to attain their qualification but there are no mechanisms for checking that students claiming student support have been registered. However, the Department considers the risk of non-registration to be lower where courses are qualified by HEFCE-funded providers, who will ensure that alternative providers operate in line with their established arrangements and expectations.

2.27 Quality assessment is one of the key criteria the Department requires alternative providers to satisfy in order to be designated to receive public funds through student support payments. Since September 2015, all providers must have had a successful Higher Education Review by the Quality Assurance Agency (QAA), have maintained their relationship with the QAA and have met expected standards during annual monitoring.

2.28 In 2015 and 2016 the QAA conducted annual reviews of 53 alternative providers. It also conducts full risk-based reviews of alternative providers on a four-yearly basis. To decide timing and length of the full review, the QAA takes into account the provider’s size, complexity of delivery and concerns it has identified through desk review. QAA engages directly with students as part of its reviews, inviting a student submission, meeting with students and including students from a different provider in most review teams. QAA completed full reviews of 23 alternative providers between 2013 and 2015. Since 2015, the Department has revoked designation for student support payments from three providers on the basis of concerns raised by the QAA.

2.29 Since January 2015, the Department has investigated 14 providers where whistleblowers, data and engagement with partner organisations raised concerns about quality of teaching. The Department revoked designation for student support payments from three of these providers where these concerns were substantiated.

2.30 In 2017, participation in the National Student Survey was mandatory for the first time for alternative providers. This survey asked final-year students whether they were satisfied with the quality of their course. Each provider was then measured against a benchmark which took into account the characteristics of the provider’s student population such as age and subjects studied:

20% of alternative providers had satisfaction rates significantly better than their benchmark compared with 9% of HEFCE-funded providers;

9% of alternative providers had satisfaction rates significantly worse than their benchmark compared with 15% of HEFCE-funded providers; and

71% of alternative providers had satisfaction rates not significantly different from their benchmarks, compared with 76% of HEFCE-funded providers.

2.31 There is scope for the Department to supplement the current mechanisms for detecting quality issues (QAA reviews and the National Student Survey) with more timely indicators that could act as real-time warning flags of problems emerging. HESA’s project to enable data collection within the academic year could be an opportunity to develop more timely data on student satisfaction or quality.

2.32 Since 2015, the Department has investigated seven cases of potentially inappropriate recruitment practices. In four cases it concluded that the allegations were unsubstantiated and in three cases applied sanctions.

2.33 Since our 2014 report there has been a changing trend in the nature of recruitment advertising at alternative providers. We examined advertising posted on social media in 2015 and 2016 for a risk-based sample of 32 alternative providers. We identified 18 advertisements emphasising student support or offering incentives to students (Figure 8).

2.34 The Department informed us that it responds on a case-by-case basis where providers are offering incentives such as iPads or vouchers. Use of such incentives is common for all providers across both the HEFCE-funded and alternative provider sector and the Department considers it to be a legitimate marketing tool if a provider is otherwise performing well (for example, passing quality reviews, reporting low non-continuation rates). The Department told us that where offering recruitment incentives is part of a pattern of issues, it would consider what action is needed.

8 Our 2014 analysis allowed us to identify providers with high in-year non-continuation rates but it had known limitations, which we acknowledged but could not address with the information available at that time. In particular, we identified students who dropped out during the academic year but the analysis did not capture students who did not return the following year. We also knew that the analysis did not account for differences in student characteristics between alternative providers and HEFCE-funded providers. For example, alternative providers have higher proportions of students from under-represented groups, such as older students and students from black, asian and minority ethnic groups. Data across the higher education sector show that these groups are more likely to withdraw from higher education.

9 For completeness, and despite the methodological limitations, we re-performed the analysis in our 2014 report and found similar trends. Details are at Appendix Two.

10 HESA non-continuation rates supersede HEFCE indicative non-continuation rates, which HEFCE developed as an interim measure in the absence of HESA analysis for the alternative provider sector.

11 HESA performed this analysis for the 57 providers that had more than 23 students starting and who were required to submit non-continuation data for 2014/15.

Part Three

Oversight of the alternative provider sector

3.1 This part of the report examines how the Department for Education’s (the Department’s) oversight of the alternative provider sector has evolved since we reported in 2014.

Oversight and sanctions

3.2 Since we reported in 2014, the Department has taken steps to strengthen its oversight of alternative providers to help improve the quality of education they offer (Figure 9). In particular, in 2015 it established the joint Alternative Provider Intelligence Unit (the Intelligence Unit) with the Higher Education Funding Council for England (HEFCE) to increase engagement with providers and investigative capacity. The Intelligence Unit has made more than 150 formal visits to providers at their premises since 2015, which primarily aimed to provide support and ensure understanding of the regulatory framework.

3.3 Since our 2014 report, the Department has also developed its approach to imposing sanctions on alternative providers. Sanctions now include:

Improvement notice. The Department formally puts the provider under notice that it must bring aspects of performance up to required standards or face further sanctions.

Suspension of payments. Payments are suspended while the Department investigates or until the issues have been addressed.

Partial re-designation. Specific courses are no longer available for student support.

Student number controls. Additional limits on the numbers of students able to access student support funding can be imposed on under-performing providers and lifted when performance improves.

Full de-designation. The alternative provider is no longer eligible for student support.

3.4 There are a range of scenarios where the Department may take action against providers. These include responding to major issues set out in Part Two such as high non-continuation rates, providers that have not registered students with qualification awarding bodies, concerns about quality, or other breaches of designation. Since 2015, the Department has suspended student support payments to 19 providers under investigation. Overall, it has taken action in more than 30 cases where its investigations, data or other engagement have identified providers who are not performing at expected levels.

3.5 Of 32 potential investigations the Intelligence Unit considered between January 2015 and November 2016, 31 are now closed, 13 of which were unsubstantiated. More than half of the investigations were triggered by whistleblowers. In considering the handling of investigations, the Intelligence Unit works closely with the Department’s Student Finance Investigations Team, the Government Internal Audit Agency and the QAA. The Intelligence Unit draws on other information including SLC data on attendance and also benefits from its engagement with partner organisations including Edexcel and the QAA and with individual providers (Figure 10). Figure 11 shows the various concerns investigated.

3.6 We carried out a detailed review of five out of 32 investigations considered by the Department since 2015. These provide a range of issues and information sources to allow us to examine what had triggered the investigation, how it was investigated and the outcomes. A summary of three of these investigations that were concluded at the point we carried out our fieldwork is at Figure 12.

Part Four

Achieving the policy objectives of expanding the alternative provider sector

4.1 This part of the report examines the progress government has made in achieving policy objectives through expanding the alternative provider sector.

The policy objectives

4.2 In 2011, the Higher Education: Students at the Heart of the System white paper set out the policy objectives for the higher education sector.12 Its objectives include promoting fair access by attracting under-represented groups including part-time students. It also set out a desire to open up the higher education market, including to alternative providers, to meet the changing needs of employers, individuals and their communities.

4.3 In 2015, the government consulted on steps to boost competition and choice in higher education and strengthen regulation. The Higher Education and Research Act 2017 received Royal Assent in April 2017 and provided the legislative basis for:

a single regulator, the Office for Students, to oversee all types of higher education providers, including alternative providers (see paragraph 1.8);

sector regulation to drive competition and choice; and

a risk-based approach to the regulation of all higher education providers to focus attention where it is needed to improve quality.

4.4 The Department for Education (the Department) has taken steps to bring data collection for alternative providers in line with the data collected for the Higher Education Funding Council for England (HEFCE)-funded sector on students’ demographics and outcomes. The Higher Education Statistics Agency (HESA) published the first data on employment outcomes in July 2017 for students at alternative providers who graduated in 2015/16. HESA has also collected statistics on demographics of students at alternative providers for academic years 2014/15 and 2015/16. We examined this information to assess whether there are early signs that government is achieving its policy objectives.

Increasing diversity

4.5 In 2015/16, undergraduate student population at alternative providers, as compared with the HEFCE-funded sector was:

more ethnically diverse: black, Asian and minority ethnic students represented 52% of undergraduates at alternative providers, compared with 22% at HEFCE-funded providers;

older: 55% of undergraduate student at alternative providers were over 24 years old, compared with 22% at HEFCE-funded providers; and

more gender balanced: 50% of undergraduates at alternative providers were male, compared with 44% at HEFCE-funded providers.

4.6 These statistics can be illustrated by the example in Figure 13.

Subjects studied

4.7 Alternative providers tend not to offer the same range of subjects as providers in the rest of the sector. In 2015/16:

10% of undergraduates at alternative providers studied science courses (46% in the HEFCE-funded sector);

48% of undergraduates at alternative providers attended business and administrative courses (13% in the HEFCE-funded sector); and

20% of undergraduates at alternative providers studied creative arts and design (9% in the HEFCE-funded sector).

4.8 Specific examples illustrating these statistics are given at Figure 14. While the Department does not have a specific policy objective for the subjects that students in receipt of support from the SLC should study, the trend may be worthy of further examination.

Outcomes

4.9 The Destinations of Leavers from Higher Education (DLHE) survey published in July 2017 presents, for the first time, information about destinations and salaries for undergraduate students who graduated from alternative providers in 2015/16 comparable with the HEFCE-funded sector:13

About 84% of alternative provider graduates were working or studying six months after graduating, compared with 90% for graduates who studied at HEFCE-funded providers.

The overall median salary reported by alternative provider graduates with a full-time employment in the UK six months after graduating is £20,500 compared with £21,500 for graduates who attended HEFCE-funded providers.

12 Department for Business, Innovation & Skills, Higher Education: Students at the Heart of the System, white paper, Cm 8122, June 2011.

13 The results exclude non-EU domiciled students.

Appendix One

Our audit approach

1 This report follows up the issues we and the Committee of Public Accounts reported on in 2014. Alternative providers are institutions offering higher education qualifications that do not receive direct funding from the Higher Education Funding Council for England (HEFCE) but their students can access public funds through student support from the Student Loans Company (SLC).

2 Specifically, our follow-up report examines the following:

Whether there are signs that the Department for Education (the Department) has resolved the major issues we identified in 2014.

Whether the evolving oversight arrangements offer protection to taxpayers and students.

Whether the Department has assessed the effectiveness of the policy of expanding the alternative provider sector.

3 We examined whether the Department has made sufficient progress in improving its control and oversight of the alternative higher education provider sector.

4 Our audit approach is summarised in Figure 15. Our evidence base is described in Appendix Two.

Appendix Two

Our evidence base

1 We reached our independent conclusions on whether the Department for Education (the Department) has made sufficient progress in improving its control and oversight of the alternative higher education provider sector after analysing evidence collected between November 2016 and August 2017.

2 We drew on a variety of sources and methods. Our analytical audit approach is summarised in Appendix One.

3 We engaged with relevant individuals from the Department including the Alternative Providers Intelligence Unit (the Intelligence Unit) to establish how concerns come to their attention and how the Department responded.

4 We also consulted with the Quality Assurance Agency for Higher Education (QAA) and the awarding body Edexcel to understand their roles in overseeing alternative providers and checking compliance with requirements for course designation for student support.

5 We reviewed case studies of eight alternative providers developed by the Department and conducted semi-structured interviews with representatives from two alternative providers selected from that group. This gave us a deeper understanding of the impact of alternative providers on (a) widening access to higher education to under-represented groups and on (b) students’ outcomes.

6 We analysed data on the 32 investigations the Intelligence Unit conducted from January 2015 to November 2016. These allowed us to understand what concerns the Department investigated and the nature of actions and sanctions applied.

7 We examined a sample of five investigations. This gave us a deeper understanding of what triggered the investigations and the extent of the Department’s response to concerns and allegations. We limited the number of in-depth reviews to five due to the time available and the volume of documentation available for review. The five investigations were broadly representative of (a) the sources of information the Department drew upon; (b) the issues under investigation; and (c) the action the Department took following the investigation.

8 We examined documents from the Department and oversight bodies that set out (a) the requirements for course designation for student support, and (b) controls and procedures the Department had in place to check compliance.

9 We analysed data on students’ demographics, destinations and salaries earned, and student satisfaction with their learning experience. This allowed us to examine the extent to which the Department is achieving the objectives for its policy of expanding the alternative provider sector and how it compares with the Higher Education Funding Council for England (HEFCE)-funded sector. We analysed data on student’s demographics published by the Higher Education Statistics Agency (HESA). We analysed the data from the 2015/16 Destination of Leavers from Higher Education Survey and data from the National Student Survey 2017, which for the first time included data for all alternative providers with undergraduate courses designated for student support, comparable with similar data for the HEFCE-funded sector.

10 We analysed data on non-continuation rates, or the proportion of students who did not return for their second year. The data available covered full-time undergraduate students. We used data prepared by HESA on behalf of HEFCE for students who entered in 2014/15, the only year available. We also used indicative non-continuation rates prepared by HEFCE for the period from 2012/13 to 2014/15 (now superseded by HESA analysis).14 The HESA non-continuation rates are official statistics submitted by the providers, which is subject to HESA’s quality assurance process.

11 HEFCE and HESA derived benchmark non-continuation rates for each alternative provider, but HESA uses a more sophisticated methodology which is in line with what it uses in the HEFCE-funded sector. Benchmark rates correspond to the non-continuation rates expected for each provider after correcting for differences in certain student characteristics. HESA’s benchmark rates correct for differences in students’ age, subjects studied and qualifications at entry.15 HEFCE indicative benchmark rates (now superceded by HESA’s) corrected for differences in students’ age, subjects studied and qualification aim (as opposed to qualification at entry).

12 The non-continuation data allowed us to understand trends and spot outlier providers in both the alternative provider sector and HEFCE-funded providers. We identified as outliers the alternative providers for which non-continuation rates were above the provider’s benchmark using the HESA data for 2014/15. We identified trends using the HEFCE indicative non-continuation rates.

13 We re-performed our 2014 analysis of in-year non-continuation rates using data from the SLC by individual alternative provider for the academic years from 2011/12 to 2015/16. The results show a trend similar to that we found in the data analysis presented in Figure 4. We used the same methodology as explained in our 2014 report (Appendix One, paragraphs 7 and 8).16

14 We analysed unpublished SLC management data on payments to ineligible students. This allowed us to understand the amount of ineligible payments that the Department did not recover.

15 We analysed data on students receiving support at alternative providers and the amount of support over time. We analysed published SLC official statistics on the amount of support paid to full-time students at alternative providers and the amount of support paid from academic year 2011/12 to 2015/16. This analysis allowed us to understand growth in the alternative provider sector.

16 We reviewed the websites of a risk-based sample of 21 alternative providers to check their compliance with the minimum entry requirements for student proficiency in the English language. The risk-based sample included the providers identified in our 2014 report, as well as further providers that the Intelligence Unit investigated in 2015 and 2016, regardless of the issue investigated.

17 We undertook online research to identify recruitment practices in a risk-based sample of 32 alternative providers in 2015 and 2016. We searched recruitment advertisements posted between January 2015 and December 2016, on the providers’ websites, as well as on Facebook and Twitter. The risk-based sample included the providers we identified in our 2014 report, as well as further providers investigated by the Intelligence Unit in 2015 and 2016, regardless of the issue investigated.

14 HEFCE indicative non-continuation rates used the proportion of students who stopped receiving support from the Student Loan Company (SLC) in their second year as a proxy for non-continuation. These were produced as an interim measure in the absence of such analysis for the alternative provider sector and have now been superseded by HESA non-continuation rates.

15 HESA published guidance on UK performance indicators and benchmarks, available at: www.hesa.ac.uk/data-and-analysis/performance-indicators/benchmarks

16 Comptroller and Auditor General, Department for Business, Innovation & Skills, Investigation into financial support for students at alternative higher education providers, Session 2014-15, HC 861, National Audit Office, December 2014.

Appendix Three

Summary of progress against previous recommendations

Design and Production by NAO External Relations
DP Ref: 11499-001