Report

by the Comptroller
and Auditor General

Home Office

Investigation into the response to cheating in English language tests

HC 2144 Session 2017–2019 24 May 2019

Our vision is to help the nation spend wisely.

Our public audit perspective helps Parliament hold government to account and improve public services.

The National Audit Office scrutinises public spending for Parliament and is independent of government. The Comptroller and Auditor General (C&AG), Sir Amyas Morse KCB, is an Officer of the House of Commons and leads the NAO. The C&AG certifies the accounts of all government departments and many other public sector bodies. He has statutory authority to examine and report to Parliament on whether departments and the bodies they fund, nationally and locally, have used their resources efficiently, effectively, and with economy. The C&AG does this through a range of outputs including value-for-money reports on matters of public interest; investigations to establish the underlying facts in circumstances where concerns have been raised by others or observed through our wider work; landscape reviews to aid transparency; and good-practice guides. Our work ensures that those responsible for the use of public money are held to account and helps government to improve public services, leading to audited savings of £741 million in 2017.

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 23 May 2019

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Sir Amyas Morse KCB
Comptroller and Auditor General
National Audit Office

22 May 2019

In 2014 a BBC Panorama documentary drew attention to fraud in the UK visa system, including cheating in English language tests. The Home Office revoked visas where there was evidence of cheating, but its decisions have come under renewed public and Parliamentary scrutiny. This investigation looks at the information held by the Home Office on its action to date.

Investigations

We conduct investigations to establish the underlying facts in circumstances where concerns have been raised with us, or in response to intelligence that we have gathered through our wider work.

© National Audit Office 2019

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.org.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

006629 05/19 NAO

The National Audit Office study team consisted of:
Andy Morrison and Tosin Omole, with assistance from Benjamin Bernard and Robert Cooper, under the direction of Tom McDonald.

This report can be found on the
National Audit Office website at
www.nao.org.uk

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

Tel: 020 7798 7400

Enquiries: www.nao.org.uk/contact-us

Website: www.nao.org.uk

Twitter: @NAOorguk

What this investigation is about

1 In 2014 the BBC’s Panorama programme uncovered examples of organised fraud taking place during Secure English Language Tests (SELTs) required under immigration rules for students and other visas. The Home Office (the Department) responded by suspending the licences of test centres and revoking the visas of people accused of cheating. A number of people have protested their innocence. The Home Affairs Select Committee has carried out an inquiry, and the House of Commons has debated the matter, without achieving a clear resolution.

2 The Department works with a number of different bodies regarding SELTs. This includes:

SELT providers, who develop, administer and mark the tests;

student sponsors such as private colleges, who must have a licence with the Department to admit students with General student visas (Tier 4) and must remain compliant with the Department’s terms; and

visa applicants, including students who apply to study with Tier 4 sponsors and must demonstrate their English language proficiency before doing so, in some cases by passing a SELT.

3 This investigation examines the Department’s response to suspected cheating in the Test of English for International Communication (TOEIC), a SELT administered by Educational Testing Service Global BV, a subsidiary of Educational Testing Service USA (ETS), a non-profit educational testing and assessment organisation. We consider how the Department assessed the number of people who may have cheated and what action it took.

4 We looked at the Department’s approach to identifying cheats and at its quality assurance work. In particular, we looked at its estimates of the number of people cheating, the steps it took to assess the evidence, and what happened to people subsequently. We examined the Department’s data and documents, reviewed published material and met with a number of stakeholders. Figure 1 sets out the chronology of major events. Figure 2 sets out what happened to the people affected. Appendix One sets out our methodology.

5 A number of factors make it challenging to provide a consistent picture over time about what happened to the individuals concerned. These factors include: the different kinds of monitoring the Department undertook between 2014 and the present, the evolution of the Department’s approach to legal cases and changes in individuals’ circumstances which have in some cases altered their immigration status.

Summary

Key findings

On the number of people identified as cheating

1 In 2014, the Home Office (the Department) obtained evidence of large-scale organised fraud from multiple sources. The Department already had longstanding concerns about the student visa system, and had taken action against hundreds of colleges. Following the Panorama programme in February 2014, the Department concluded that the system was too open to abuse and moved to investigate all Secure English Language Test (SELT) providers. It obtained data on exams and categories of cheating from US-based Educational Testing Service (ETS), the provider implicated by Panorama. It used this information to investigate sponsors, test centres and individual students. It found widespread failings in sponsors and test centres, including evidence of organised criminality. The Department had other data and intelligence which pointed to large-scale deception within the system (paragraphs 1.13 and 1.14, 2.1, 2.2 and 3.8).

2 ETS identified virtually every test in the UK as suspicious. It identified that 97% of all UK tests were invalid or questionable. ETS had not used voice recognition technology before to detect cheating in Test of English for International Communication (TOEIC) exams. As a precaution, ETS assigned two human listeners, based in the US, to each independently verify the results of each test. They confirmed that 58% of all tests were ‘invalid’ and marked the rest as either ‘questionable’ or ‘cleared of suspicion’. ETS then reviewed the overall patterns for each test centre and reclassified some cases as invalid, questionable or cleared of suspicion. It identified 58% of 58,361 UK test participants as having invalid results and 39% as questionable. The Department could not tell us how many tests ETS reclassified following initial checks. ETS’s lists suggest that it classified 462 more tests as invalid than the total number confirmed by human listeners (paragraphs 2.5 and 2.6).

3 For two years the Department revoked the visas of anyone with an invalid test, without expert assurance of the validity of voice recognition evidence. The Department normally relies on SELT providers for assurance when awarding visas. It visited ETS, the US test provider, in June 2014 to understand the process and listen to some recordings. Based on legal advice at the time the Department considered ETS’s evidence was sufficient to take action against students without the need for expert review. To support an appeal in 2015 the National Union of Students (NUS) commissioned an expert review, which said that voice recognition software could have made errors in up to 20% of cases and human listeners could have made errors in up to 30% of cases. On hearing the evidence, the courts criticised the Department staff’s lack of expertise in challenging ETS’s assessment but still concluded that the Department’s overall evidence was sufficient to support its allegations. The Department instructed an eminent expert on speaker identification in 2016 to form an independent opinion on how likely it was that someone ETS identified as invalid may in fact be the innocent victim of a false positive (paragraphs 2.8 to 2.10).

4 In 2016, the Department’s independent expert estimated that voice recognition checks would have identified substantially fewer than 1% of people of cheating incorrectly, based on a series of assumptions. ETS faced a considerable workload to check the voice recordings from 66,000 tests. Initially 19 assessors experienced in voice biometrics had worked intensively carrying out checks. Voice recognition technology had not been used before with TOEIC tests and ETS did not use piloting or a control group. The expert judgement is based on more information than the expert the NUS commissioned, but it still needed to make a series of assumptions to estimate how many people may have been incorrectly identified. The estimate begins with the assumption that an error rate of 2% from a pilot with a similar ETS language test, the Test of English as a Foreign Language (TOEFL), also applied to the TOEIC tests. The expert said that because additional human checks rejected 42% of results it was likely that the number of innocent people would be very substantially less than 1%, with substantially more people given the benefit of the doubt. Neither expert had all the information they wanted to make their assessment. They did not know what software had been used, have access to many voice recordings, or know the performance of human verifiers. A judicial review in January 2017 accepted that the Department’s expert opinion provided support for ETS’s assessment. However, representatives of the Department conceded that neither the expert evidence nor ETS’s assessment of invalidity could prove on their own whether a claimant cheated (paragraphs 2.11 to 2.14).

5 Most but not all people identified as cheating by ETS had very high marks. Many people found to have invalid tests had scores indicating most were very fluent English language speakers. The distribution of scores indicates 92% (30,000) were fluent English speakers, of which 16,000 (49%) were highly fluent. People with questionable tests also had markedly skewed results. People cleared by voice recognition checks had a normal distribution of marks, similar to that of other English language tests (paragraph 2.16 and Figure 5).

6 It is difficult to estimate accurately how many people may have been wrongly identified. The degree of error is difficult to determine accurately without piloting or a control group for TOEIC tests. Analysis of the distribution of marks can help to identify anomalies compared with similar tests and people whose scores are not easily explained by known methods of cheating. We identified anomalies which the Department had not investigated. For example:

In speaking tests deemed invalid, 100 people (0.3% of invalid results) had lower scores than the level required for study in the UK, meaning supposed proxies were actually people with limited English language ability.

Thousands of people suspected of cheating by voice recognition checks had low scores in multiple choice tests. Panorama had shown evidence that test centres read out multiple choice answers to all candidates sitting a test. ETS cleared only 2,200 people outright of cheating but 5,900 multiple choice reading test results and 3,500 listening tests did not meet the level required for study, suggesting they were not given the answers.

ETS cleared only 93 people with the very highest levels of fluency. In similar English language tests, a greater number of highly fluent speakers score top marks, but this pattern was not evident for people cleared of cheating (paragraphs 2.18 and 2.19).

7 Thousands of people accused of cheating have still won the right to remain in the UK. The courts have taken a case-by-case approach, having accepted that the Department’s evidence places the burden of proof on individuals. Individuals’ testimonies can be the deciding factor, and data indicate that thousands have succeeded in obtaining immigration status despite the Department’s decision to revoke their visa for deception in TOEIC tests. People flagged with invalid test results on the Department’s systems have:

won 2,664 First-tier appeals against immigration decisions since 2014, equivalent to 8% of all those with invalid test results; people usually had to appeal on human rights grounds because they could not appeal the decision directly; and

gained leave to remain in the UK in 4,157 cases: 532 have indefinite leave to remain, 477 are now British citizens and 2,682 have leave to remain of more than a year. Numbers represent people with some form of leave to remain as at March 2019 but may not represent the overall number of people that gained leave, because they are based on a reporting tool originally designed in 2014 and it may not capture some changes to people’s immigration status (paragraphs 2.9, 3.30 and Figure 11).

8 We saw no evidence that the Department considered whether ETS had misclassified individuals or looked for anomalies. The Department provided us with evidence of the steps it had taken to analyse ETS data to check for errors and explained the processes it followed to correctly identify people on Home Office systems. It had not investigated the reasons why people with invalid scores had low marks, won appeals or gained leave to remain. The Department suggested that people only required low scores for some visas, such as for civil partnerships, and that it had used statistical analysis as far as is possible to check for errors. It said it would have needed to do a manual review of case files to reach a definitive view. The Department told us it could not provide accurate numbers on appeals which people won specifically against allegations of TOEIC deception because it had not carried out significant analysis on this since 2016, and HM Courts and Tribunals Service records these as human rights appeals. The Department said it investigated issues raised at appeal and that these were specific to each case. It does not see a significant risk of innocent people being wrongly caught up in action because the 2016 independent expert report and its other evidence indicated the number of errors was very low (paragraphs 2.17 and 3.32).

9 It was not possible for the Department to directly check the accuracy of ETS classifications. The Department has not had access to check ETS assessments of cheating. Instead it analysed the distribution of results by test centre and by test session and compared these with scores from other SELT providers. It concluded that ETS had not made systematic errors. We reviewed ETS’s data and identified one systematic error. Some 6,000 people had been incorrectly categorised as ‘UK nationals’. The Department did not know the reason for this and told us it might have been down to individuals stating their nationality incorrectly, test centres faking results, or IT systems defaulting to UK nationalities where no entry is recorded. The error does not demonstrate whether other data in the ETS lists are correct or not. In July 2016 experts for the Department and defendants said that they could not be completely confident in ETS because audio files did not have the original time, date and location stamps to enable cross-checking. In 2018 the courts considered ETS evidence on the security of data as part of an overall package of evidence that proved sufficient for criminal convictions (paragraphs 2.23 to 2.25).

On action taken against people accused of cheating

10 The Department estimated it had spent £21 million dealing with TOEIC as at November 2017. The Department did not keep detailed records of its spending but put together an estimate as part of preparations for possible legal action in 2017. It estimates it spent £9 million on immigration enforcement, £5.5 million on appeals and judicial reviews, £3 million dealing with temporary visa applications and £2 million on criminal and financial investigations. The Department made £624,000 of confiscations and cash seizures (paragraph 3.7 and Figure 8).

11 The Department took a number of actions against education and testing institutions, including ETS. In 2014, the Department revoked the sponsor licences of 75 colleges and one university. It suspended all TOEIC test centres and allowed the ETS testing licence to expire in April 2014. In 2017, the Department concluded a three-year investigation to determine whether ETS and/or any of its employees were criminally complicit in undermining the TOEIC test. It did not find evidence of complicity. The Department also launched civil proceedings in 2017 against ETS Global BV (who administered the tests in the UK) rather than ETS US (who provided the analysis that the Department relied on to take action). The Department reached a financial settlement with ETS Global BV in 2018 for £1.6 million after legal advice that it did not have significant assets, and that Department estimates were extremely vulnerable in successfully demonstrating costs incurred. The Department told us decisions, both to initiate litigation and to reach settlement (in early 2018), were taken in consultation with ministers and in line with legal advice that it represented the best value for the taxpayer (paragraph 3.12 and 3.16).

12 The Department has supported successful prosecutions against 25 organised criminals. So far, 31 defendants from six organised crime groups have stood trial and 25 have been convicted. Twenty-one have received prison sentences totalling 70 years, and four more await sentencing. All those identified as having planned and organised the offences have been convicted. Fourteen further suspects are awaiting a decision on charging. The Department investigated 20 colleges in total, which it prioritised based on evidence from ETS and other information (paragraphs 3.8 to 3.13).

13 Students at the colleges that had their licences revoked were affected irrespective of whether they sat TOEIC or another English language test. Students had to find courses elsewhere if their college had its Home Office licence revoked. Some have struggled to find sponsors. The Department set up a sponsorship working group to look at how students could be supported to find other sponsors. It established an online tool containing details of 62 sponsors with courses available and wrote to 4,795 students in need of assistance. Some 837 students used the portal between November 2014 and January 2015. Stakeholders including Universities UK, the NUS and the UK Council for International Student Affairs (UKCISA) raised concerns about the effectiveness and progress of the sponsorship working group (paragraphs 3.20 to 3.22).

14 The Department needed to identify thousands of individuals on its systems before it could take action against them. As at February 2016 the Department had identified 51,000 potential matches on its system. Further case work reduced the number of potential matches to around 42,000, with the Department confirming it had accurately matched names on its systems to ETS lists in 16,000 cases and a further 26,000 matches unconfirmed. The Department issued letters to all people with invalid tests saying their visa had expired and they would need to leave the country immediately. People with questionable results were told they would need to retake the test if they wished to extend their visa (paragraphs 3.24 and 3.25).

15 As at the end of March 2019 data showed that at least 11,000 people who had taken TOEIC tests had left the country following detection of mass fraud. The number is based only on 11,000 invalid and 5,000 questionable cases with confirmed matches and people may have left for reasons not related to allegations of deception in TOEIC tests. Around 7,200 left voluntarily after April 2014. Around 2,500 people were forced to leave and almost 400 were refused re-entry to the UK on arrival at a port. UKCISA gave us evidence of several students detained at the border in relation to taking a SELT in the past or their English language ability. The Department told us it would only have taken action against people with an invalid certificate or other evidence against them (paragraph 3.28 and Figure 10).

16 Around 12,500 appeals involving individuals matched to invalid or questionable TOEIC certificates have been heard up to March 2019. The Immigration Act 2014 removed the right of appeal against applications for leave to remain in the UK. Students could challenge refusals of leave through judicial review. Where students made a separate human rights claim to the Department, and that claim was refused, the student could lodge a human rights appeal. Some people refused under the previous appeals system only had an out-of-country right of appeal. Stakeholders told us some people left the UK only to find they lacked access to the legal skills needed for UK courts. In December 2017, the courts ruled that out-of-country appeals did not offer sufficient protection for human rights and that appeals must be heard in the UK. The Department is investing in video-conferencing so that out-of-country appeals can be heard in future (paragraphs 3.30 and 3.31, 3.36 and 3.37).

17 Individuals matched to invalid or questionable TOEIC certificates won 3,600, First-tier appeals but the Department has not tracked the reasons why. In most cases, appeals were made on human rights grounds. Individuals suspected of cheating won some 40% of First-tier appeals heard in court compared with 60% won by the Department. The Department only analysed the reasons why it lost the first 385 appeals up to September 2016, because case law was not yet established. In 17% of these cases people winning were proven innocent of cheating in TOEIC tests and in 57% cheating was not proven. The Department does not hold readily available data on how many people proved their innocence of cheating in TOEIC cases and how many people won for other reasons. A detailed review of the case files would be required to establish this. Human rights grounds, such as the right to family life, will have become more significant with the passage of time. The most recent available data for 28 appeals in May 2018 indicated almost all appellants won where they had a British child or spouse. Appellants without British relatives tended to be unsuccessful (paragraphs 3.32, 3.34 and 3.35 and Figure 11).

Concluding remarks

18 Evidence from ETS, Departmental analysis and successful prosecutions against those involved in administering test centres strongly suggests that there was widespread abuse of the Tier 4 visa system. The exact scale of that abuse is difficult to determine because experts drew different conclusions over the reliability of voice recognition checks. It may be right to say that the Department expert was more informed than the NUS expert, but not that he was sufficiently well informed to provide a definitive view. The Department nonetheless relied on his judgement to assume that collateral damage would be low. The data environment is weak and the Department has not been able to independently validate ETS judgements about cheating. It is reasonable based on the balance of probabilities to conclude there was cheating on a large scale because of the unusual distribution of marks, and high numbers of invalid tests in test centres successfully prosecuted for cheating.

19 The Department decided to react vigorously to the evidence of widespread cheating that was reported by Panorama, and which confirmed their pre-existing concerns. Widespread action to close colleges meant students who did not sit TOEIC exams have been affected, some of whom have struggled to find replacement sponsors. The courts found in 2016 that the Department’s evidence was sufficient to make the accusations it did. Nonetheless, the Department’s course of action against TOEIC students carried with it the possibility that a proportion of those affected might have been branded as cheats, lost their course fees, and been removed from the UK without being guilty of cheating or adequate opportunity to clear their names. To date the Home Office has considered its evidence sufficient to tell people that the number of people wrongly affected would be very small, and that they can appeal from outside the UK or lodge a Judicial Review.

Part One

Policy and operational context

1.1 In 2014, the BBC’s Panorama programme uncovered examples of organised fraud taking place so that people passed English language tests required under immigration rules. The Home Office (the Department) responded by suspending the licences of test centres and revoking the visas of students suspected of cheating in the Test of English for International Communication (TOEIC) administered by ETS Global BV, a wholly-owned subsidiary of Educational Testing Service (ETS), a non-profit educational testing and assessment organisation headquartered in the USA. A number of people have protested their innocence. The Home Affairs Select Committee has carried out an inquiry and the House of Commons has debated the matter, without achieving a clear resolution.

1.2 This investigation examines what happened to people implicated in cheating in TOEIC tests, with a particular focus on establishing how the Department responded and what safeguards existed to protect innocent people.

1.3 This part covers the immigration regime in place up to 2014 and concerns raised. Part Two looks at the evidence of cheating. Part Three sets out the Department’s response.

Points-based immigration system and Tier 4

1.4 Successive governments have introduced more stringent criteria to the student visa system. In February 2008, the government began to phase in a points-based system (PBS) for immigration. The system’s objectives were to better identify and attract migrants with the most to contribute to the UK, improve compliance and reduce the scope for abuse. The UK Border Agency (UKBA, a former executive agency of the Department) implemented Tier 4 of the PBS on 31 March 2009 for students and their dependants from outside the European Economic Area (EEA).

1.5 Before the PBS, non-EEA students could apply to one of 15,000 colleges registered with the then Department for Innovation, Universities and Skills to obtain a student visa. There was no limit to the number of students whom a college could enrol. Under Tier 4, the Department and its agencies licensed each educational institution allowed to sponsor international students. It granted some 2,500 licences initially.

1.6 In 2011, the government introduced revised policy objectives and strengthened the criteria on who could come to the UK to study, what they could study and for how long. Students had to pass Secure English Language Tests (SELTs) and colleges had to demonstrate a good compliance record and standard of education.

English language tests required under immigration rules

1.7 Under immigration rules, a student’s English language ability must be assessed before they are allowed to study in the UK. The level of English a student requires and how they are assessed depends on the level of course they are following and the type of institution. Figure 3 provides an overview of the SELT regime.

1.8 Higher education institutions such as universities can determine the English  language ability of students, but students applying to other institutions must demonstrate their knowledge of English by passing a SELT at a Department-approved test centre.

The Department sub-contracted English language testing

1.9 UKBA’s PBS team first appointed ETS in 2007-08, along with 18 other suppliers, when English language provisions were introduced for some types of visa application. The appointment was not formalised through a procurement process because there was no contractual agreement between the parties. UKBA assessed the equivalence of each tester’s English test to a recognised European Standard and security arrangements around testing.

1.10 In 2010-11, UKBA conducted an open competition with limited commercial involvement apart from advice on potential market participants and use of eSourcing technology. As in 2007-08, it envisaged using sub-contractors and did not identify any risks.

1.11 The arrangement did not go through any other formal governance in UKBA or the wider Department. Commercial controls required approvals for contracts in excess of £1 million. UKBA did not seek approvals because SELTs did not carry a direct cost to the Department.

Concerns about the Tier 4 regime

1.12 In March 2012 we reported on the Tier 4 visa system. We concluded that the UKBA had implemented the system with predictable flaws, which it could have avoided if it had introduced key controls at the same time. We estimated that 40,000 to 50,000 individuals might have entered the UK through Tier 4 in its first year of operation to work rather than study. In August 2012, the Committee of Public Accounts concluded that UKBA had implemented Tier 4 of the PBS for students without effective controls to prevent abuse. The government agreed that it should put in place adequate controls and identify and actively manage risks before implementing changes in the future. It noted that it had controls in place, such as the licensing system and visa letters, but in hindsight it had not achieved the right balance.

1.13 Between 2011 and 2014 the Department suspended and revoked hundreds of sponsor colleges from the register for failing to meet the basic education and immigration requirements and established an ongoing programme of work to reduce abuse in Tier 4 (Figure 4).

1.14 In January 2014, having become aware of Panorama’s findings, the Department identified a need to rethink its policies on sponsors. It was concerned that compliance action alone was not sufficient and that the franchising and partnership arrangements were allowing a number of disreputable colleges to flourish. It identified a range of issues, including systemic abuse of Tier 4 arrangements by organised criminals.

Part Two

The number of people identified as cheating

BBC’s Panorama programme showed evidence of systematic cheating in English language tests

2.1 In February 2014, the BBC broadcast a Panorama documentary showing compelling video evidence of three types of immigration fraud. These were:

the use of proxies for all candidates sitting Test of English for International Communication (TOEIC) speaking tests;

invigilators reading out multiple choice answers from reading and listening tests to all TOEIC candidates; and

using other people’s bank statements to provide evidence that applicants had sufficient funds to study in the UK.

2.2 The footage showed organised cheating taking place in two TOEIC test centres administered by third parties on behalf of US firm, Educational Testing Service (ETS). The brazen nature of the frauds raised fundamental questions about whether the Home Office (the Department) could depend on the ETS testing regime.

Voice recognition checks

2.3 Over the course of 2014 ETS gave the Department every candidate’s results since 2011 (reading, writing, listening and speaking), as well as data on where and when they sat the tests, their names and their nationality.

2.4 ETS also reviewed audio files from speaking tests for evidence of proxy test-takers. It used new voice recognition software to identify whether proxies had sat tests on behalf of multiple people. The results of each test were checked by two trained listeners, based in the US, who independently confirmed or rejected the result.

2.5 ETS determined that almost everyone taking tests in the UK had cheated or may have cheated. It identified that 97% of all UK tests taken between 2011 and 2014 were invalid or questionable. Human checks confirmed that 33,663 UK tests (58% of all UK tests) were invalid because voices matched to more than one identity. They rejected 42% of matches where there was any doubt about the validity of the match.

2.6 We understand that ETS reviewed the overall patterns for each test centre for anomalies and reclassified some cases as invalid, questionable or cleared. It classified 22,476 UK tests (39%) as questionable. It cleared just 2,222 UK tests of cheating outright. The Department could not tell us how many tests ETS reclassified following voice recognition checks. ETS lists suggest it may have classified 462 more tests as invalid than the total number confirmed by human listeners.

Assessments of the reliability of voice recognition checks

2.7 The Department did not seek expert assurance over the results of the voice recognition checks for two years. In 2014, a delegation of five civil servants visited the US to understand the process that ETS used and to listen to a small number of recordings. None of the people in the delegation had expertise in the technology or techniques used. Instead, they relied on assurances from ETS that the voice recognition technology was suitable for the task.

2.8 In 2015 the National Union of Students (NUS) commissioned an expert report for a court hearing. The report raised doubts about the accuracy of the method used by ETS. It could not say how many false positives existed without access to ETS information. However, it suggested voice recognition could have miscategorised up to 2,000 people, based on software miscategorising between 1% and 20% of recordings and human checks having a typical error rate of up to 30%. Giving evidence in court, the expert testified that human voice analysis systems, when performing at their best, can be expected to generate an error rate of some 20%. The expert reported that half of the small number of recordings he had listened to did not meet the minimum level required for voice recognition analysis.

2.9 The court was critical that the Department’s witnesses did not have sufficient expertise to ascertain the reliability of the voice recognition process. It acquitted the defendants based on their testimonies. Even so, the court’s omnibus finding (applicable to all people implicated) still concluded that the totality of the Department’s evidence (including, for example, analysis of exam results and other intelligence) meant the burden of proof had been narrowly discharged and the onus remained on individuals to demonstrate their innocence.

2.10 In 2016, the Department commissioned an eminent expert in the UK in speaker identification, who was also a senior practitioner from the same company as the expert the NUS employed, to form an independent opinion on how likely it was that someone ETS identified as having an invalid result may in fact be the innocent victim of a false positive. The expert had access to additional information, including the approximate amount of training provided to staff checking voice recognition results and the tools and methods at their disposal. He noted staff underwent ‘multi-day training’ and they had a mentoring period with staff experienced in using voice recognition checks.

2.11 The expert judgement was based on a series of assumptions, which may have been reasonable. The expert noted that a voice recognition pilot for a similar exam, the Test of English as a Foreign Language (TOEFL), had an error rate of 2%. Assuming this applied to TOEIC, the expert concluded the addition of human checks would reduce the error rate well below 1% because the trained listeners had rejected 42% of the results using:

stringent criteria for verification;

potentially more speech available from the tests than that processed by the voice recognition system; and

a much wider range of speech features available on which to base their decisions than just vocal tract resonances performed by the voice recognition system.

2.12 Both experts’ reports had limited access to information and relied on assumptions. They did not know what software had been used nor the performance of human verifiers. Neither expert was able to listen to a significant number of recordings. The Department expert noted that there were no data on the average duration of the speech files.

2.13 ETS had a huge workload to check the voice recordings from 66,000 tests. In December 2018, a witness statement from the head of ETS testing said that the voice biometric checks were carried out by 19 members of staff experienced in assessing TOEFL exams using voice biometric data. These staff worked intensively to cope with the workload, and ETS needed to bring in around 70 experienced staff from the assessment division, the programme area and ETS evaluators of TOEIC and TOEFL tests. Staff attended a training event and had an experienced ‘buddy’. ETS assessed newly trained reviewers and released a few who did not meet the required standard.

2.14 At a judicial review in January 2017 representatives of the Department conceded that neither the expert evidence nor ETS’s assessment of invalidity could prove on their own whether a claimant cheated. The court accepted both expert reports supported claims against students and that the expert evidence commissioned by the Department allowed real weight to be given to ETS’s assessment.

Analysis of ETS data

2.15 The Department has not had access to check ETS assessments of cheating. Instead it carried out extensive analysis of ETS data including looking at the patterns of marks in every test centre. It identified unusual patterns, including groupings of people with the same marks and unusual distributions of scores towards very high marks. It identified that the marks TOEIC students achieved were much better than those achieved in similar tests run by other providers. We saw reports the Department produced on around 20 test centres to support court hearings. Each showed unusual distributions of marks and concluded the scores did not reflect the genuine ability of candidates in the majority of tests.

2.16 We looked at the distribution of marks and found the scores for tests deemed invalid and questionable were very high and not distributed normally (Figure 5). The distribution of scores indicates 92% (30,000) were fluent English speakers, of which 16,000 (49%) were highly fluent. Some 2,654 people (8%) scored at the level required for study in the UK. This differed from people cleared of cheating by the voice recognition software, with 45% scoring at the level required for study in the UK and 28% demonstrating fluency. The distribution of scores cleared by voice recognition checks was similar to the distribution of scores for similar English language tests run by different providers (Figure 6).

2.17 We could find no evidence that the Department had actively looked at whether innocent people were wrongly assessed as cheats. The Department provided us with evidence of the steps it had taken to analyse ETS data to check for errors and explained the processes it followed to correctly identify people on Home Office systems. It did not check whether the ETS classification was correct or investigate anomalies for people wrongly implicated. The Department told us it had used statistical analysis as far as is possible to check for errors and it would have needed to do a manual review of case files to reach a definitive view. It did not see a significant risk of innocent people being wrongly caught up in action because the 2016 independent expert report and its other evidence indicated that the number of errors was very low. The Department told us that it investigated issues raised at appeal and that these were specific to the facts of each case.

2.18 It is difficult to estimate accurately how many innocent people may have been wrongly identified as cheating. Voice recognition technology is new, and it had not been used before with TOEIC tests. The degree of error is difficult to determine accurately because there was no piloting or control group established for TOEIC tests.

2.19 Analysis of the distribution of marks can help to identify people whose scores are not easily explained by known methods of cheating. We identified anomalies which the Department had not investigated. For example:

In speaking tests deemed invalid, 100 people (0.3% of invalid results) had lower scores than the level required for study in the UK, meaning supposed proxies were actually people with limited English language ability. The Department suggested that people only required low scores for some visas, such as for civil partnerships, and that it could not have acted against people who did not meet visa requirements.

Thousands of people suspected of cheating by voice recognition checks had low scores in multiple choice tests. Panorama had shown evidence that test centres read out multiple choice answers to all candidates sitting a test. ETS cleared only 2,200 people outright of cheating but at least 5,900 multiple choice reading tests and 3,500 multiple choice listening tests did not meet the level required for study. It suggests ETS treated at least 3,700 multiple choice tests as suspicious (6% of tests detected by the software) even though the marks suggest candidates were not fed the answers.

ETS cleared only 93 people with the very highest levels of fluency. Figure 6 shows a spike in the number of highly fluent people scoring top marks with a similar English language test. This pattern was not evident for people cleared of TOEIC cheating.

Questions about the handling of data

2.20 Some appeals have challenged the handling of data by ETS and the test centres it employed, particularly because some test centres handling data were criminal enterprises. Students attempting to clear their names have encountered a number of problems in accessing data from their tests. Some students have obtained access to voice recognition data and found the recordings were not of their voice, indicating either that a proxy was used or the wrong voice file was assigned to their name.

2.21 Not all students have been able to obtain information from ETS, who did not release voice recordings initially. We saw evidence that ETS refused to provide one student with any data relating to their exam results on the basis that ETS destroyed it after two years. Even when students have had legal representation they have had difficulty obtaining all of their personal data, including the original recordings and other materials from the day of the test. Some students also had difficulties obtaining the voice clips that were used as evidence against them. Bindmans, the legal firm which has represented dozens of TOEIC students, told us that ETS had failed to provide information and documents to which the individuals are entitled. The speaking test responses have not been provided in their entirety in the original format, with meta-data demonstrating the time and location of the recording. Instead, ETS provided short voice clips that were supposedly taken from the original files to be subjected to the voice analysis. ETS told the Department its current policy is to provide test takers with copies of audio recordings utilised in voice biometrics on request. It confirmed it has refused requests for different formats of the files because the requests were not made on the grounds that the audio files could not be played or for any other technical reason.

2.22 We heard that it was hard for some students to prove their innocence when accused of using a proxy. One student told us that the location of his test centre had been recorded incorrectly in the evidence provided. He had taken the test in London but the Department’s letter to him said he had taken the test in Leicester on the same day. He had bank transactions showing he was in London. He was told this was not evidence of his innocence because he could have still used a proxy to sit the test on his behalf.

2.23 In July 2016 experts for the Department and defendants said that they could not be completely confident in ETS because audio files did not have the original time, date and location stamps to enable cross-checking. A court hearing a case in August 2016, found that this expert evidence highlighted enduring unanswered questions and uncertainties relating in particular to systems, processes and procedures concerning the TOEIC testing, the subsequent allocation of scores and the later conduct and activities of ETS. The court still found that the appellant had engaged in deception based on his testimony. It concluded that fluency in English did not remove incentives to cheat, which included lack of confidence, fear of failure, lack of time and commitment and contempt for the immigration system. In 2018 the courts considered the security of data in a successful criminal prosecution. Representatives from ETS provided evidence outlining the steps taken to ensure security of data as part of a package that was sufficient to secure criminal convictions (meeting the higher criminal standard of evidence). The evidence heard in court has not been decisive in immigration hearings or criminal trials.

2.24 We reviewed the Department’s work to validate ETS data. The Department queried any data that were obviously incorrect. It cleansed the data for obvious errors and checked the total entries provided to them in case of duplicates. It identified 293 duplicate entries, which ETS corrected. Otherwise, the Department did not carry out any independent checking or testing of the data, trusting that ETS had correctly categorised individuals as having invalid or questionable results.

2.25 We reviewed an anonymised dataset and found errors in the data. ETS had listed some 6,000 people as ‘UK nationals’ but the Department’s evidence showed that none of these were confirmed matches to British Citizens with invalid or questionable TOEIC certificates. The Department did not know the reason for the misclassification in ETS lists and told us it might have been down to individuals stating their nationality incorrectly, test centres faking results, or IT systems defaulting to UK nationalities where no entry is recorded. The error does not demonstrate whether other data in the ETS lists are correct or not.

Part Three

Action taken against people accused of cheating

3.1 In this part we set out action the Home Office (the Department) took in response to cheating in the Test of English for International Communication (TOEIC). We report:

early action the Department took following the Panorama programme;

the latest position of the groups acted against; and

appeals and challenges to Department action.

Early action the Department took following the Panorama programme

3.2 Faced with a risk of systemic organised fraud, the Department took immediate action in February 2014 following the Panorama documentary. It set up a ‘Gold Command’ unit to lead the response and report regularly to ministers. It suspended two colleges identified by Panorama and undertook immediate compliance visits.

3.3 In February 2014, the Department placed on hold all Tier 4 visa applications supported by an English language qualification from US firm Educational Testing Service (ETS). The Department contacted people with current applications using a TOEIC certificate. As a precautionary measure it placed on hold all visa applications from people using other English language test providers, pending investigation. The Department took action across the sector, including against other providers of Secure English Language Tests (SELTs), Tier 4 sponsors and individuals (Figure 7).

3.4 The Department immediately suspended the use of ETS SELTs in immigration applications and all TOEIC test centres. It contacted ETS for information on its operating model and control practices and asked it to explain their failure in these test centres. It asked ETS to suspend all further testing in the UK until it could demonstrate it had adequate control arrangements in place.

3.5 The Department had intended to replace the self-regulatory regime with its own inspection regime, and it decided to accelerate that process following detection of fraud by Panorama. The Department allowed the licence with ETS to expire in April 2014. It commissioned Moore Stephens to review the remaining four providers and extended their existing licences until November 2014, pending the findings of the review.

3.6 In August 2014 Moore Stephens reported that it had not identified any fundamental issues requiring immediate suspension or termination of activities. It noted that the Department’s changes had resulted in a stepped improvement in the compliance regime, but that the pace of the providers’ response to the updated requirements was not commensurate with the gravity of malpractices at ETS. It considered that the Department still faced a significant risk that the scope for cheating and malpractice, while potentially reduced, remained higher than it should have been. Following a procurement exercise, the Department issued only two SELT licences from April 2015.

3.7 The Department did not keep detailed records of its costs other than for the very early period of its activity. In November 2017 the Department estimated it had spent £21 million dealing with the repercussions of TOEIC cheating (Figure 8).

Criminal investigations

3.8 In 2014, the Department’s Criminal and Financial Investigation (CFI) teams began investigating organised criminal activity in TOEIC test centres. Figure 9 shows that 26 test centres had invalid tests in more than 60% of cases. Based on ETS data, the Department carried out preliminary inquiries into around 50 test centres with high numbers of invalid and questionable results. The Department prioritised 20 test centres for further investigation using analysis of ETS data, and other intelligence, including previous audit visits.

3.9 As of April 2019, the CFI teams’ work has supported six prosecutions in relation to criminal abuse of the TOEIC test:

thirty-one defendants from six organised crime groups have stood trial;

twenty-five people have been convicted;

twenty-one people have been sentenced, to a total of 70 years in prison. There were four convictions in April 2019 where the sentence is yet to be handed down;

fourteen further suspects await a charging decision; and

the Department made £559,000 of confiscations and £65,000 of cash seizures.

3.10 One investigation examined abuse of the TOEIC test by an organised crime group with control of three colleges in Manchester. The colleges administered 2,039 tests over 12 months between 2012 and 2013. ETS analysis found that 1,052 tests were invalid and 987 were questionable. Evidence from unannounced inspection visits, computers and documents proved the fraud had taken place. The investigation found the crime group had used a set of proxy test-takers to take the TOEIC speaking test on behalf of people willing to pay for the service. The group was making considerable income from the fraud, charging around £750 for each TOEIC test, which would ordinarily cost £180. Ultimately, seven people were charged and convicted of criminal offences.

3.11 Investigators told us that there was no master criminal organising all TOEIC fraud. Each enterprise had identified loopholes that they could exploit and all had offered slightly different arrangements.

3.12 In 2017 the Department investigated ETS to determine whether it and/or any of its employees were criminally complicit in undermining the TOEIC test. It did not uncover evidence of complicity. In March 2017 it became clear the Department could not take criminal action against ETS. It launched civil proceedings against ETS Global BV (who administered the tests in the UK) rather than ETS US (who provided the analysis that the Department relied on to take action). The Department reached a financial settlement with ETS Global BV in 2018 for £1.6 million after receiving advice that it did not have significant assets, and that Home Office estimates were extremely vulnerable in successfully demonstrating costs incurred. The Department told us both decisions to initiate litigation and reach settlement (in early 2018) were taken in consultation with ministers and in line with legal advice that it represented the best value for the taxpayer.

3.13 The number of convictions appears relatively low compared with the number of people who have been removed from the UK. The Department took the decision at the outset to focus cases on people facilitating deception rather than the beneficiaries. In addition, the high-profile nature of the Panorama broadcast meant that criminal gangs were forewarned and able to shut down their operations and dismantle evidence. The Department had to prioritise its resources. It told us that cases can take a long time to prosecute and, overall, it had secured a high number of convictions for this type of fraud. At least one further prosecution is planned.

Action against sponsor colleges and universities

3.14 As well as demonstrating proficiency in the English language, international students also need sponsorship from an academic institution with a Tier 4 licence. Universities and colleges must verify the student’s capability and intention to complete the course before they can issue a Confirmation of Acceptance for Studies. They must also comply with other Department regulations or risk having their licence suspended or revoked. At the time of the Panorama documentary, the Department was already concerned about abuse of the Tier 4 visa system, including private colleges selling sponsorships as part of a package to help people emigrate to or remain within the UK. These packages could include providing proxies to pass English language tests.

3.15 In total, 1,051 colleges and universities had sponsored students with invalid tests. Before the Panorama programme, the Department had already revoked the licences of 349 of these sponsors. ETS and Department data indicated that a small group of around 50 private colleges were responsible for the majority of students with invalid scores. Of the remaining colleges and universities, most had only a small number of students with invalid tests.

3.16 In total, the Department suspended 96 sponsor licences. Of these:

seventy-nine licences were revoked, meaning the institution was no longer allowed to sponsor Tier 4 students;

ten licences were surrendered by the sponsor, meaning the institution chose to stop sponsoring students;

seven licences were reinstated, meaning the institution could resume sponsorship of students. In five cases this was subject to an action plan being in place; and

the Department received 25 legal challenges to its decisions to revoke licences. Of these, only one ended in the claimant’s favour.

3.17 In February 2014, the Department began ‘Operation Firewall’ to investigate abuse within Tier 4 of the points-based system. It expanded Operation Firewall following the Panorama programme to identify which sponsors were complicit in the deception or non-compliant with Department guidance. Operation Firewall identified sponsors it considered to be high-risk, based on evidence from ETS on the number of invalid TOEIC certificates. It also used other evidence, including compliance visits, inspections and attendance and performance data of individual students to determine whether sponsors had been compliant.

3.18 In 2015, the Independent Chief Inspector of Borders and Immigration (the Inspectorate) reported on the effectiveness of the Department’s monitoring of Tier 4 sponsor compliance and its handling of Operation Firewall.1 The Inspectorate concluded that Operation Firewall had been handled well, but noted that weaknesses in the Department’s sponsor compliance checks before 2014 may have contributed to the scale of the deception. For instance, in the Inspectorate’s sample of 50 Operation Firewall cases, there were 12 in which the Operation identified a number of sponsor compliance failings but the previous visit had found that the sponsor had fully ‘met’ its duties.

3.19 Universities UK, the Immigration Law Practitioners’ Association (ILPA) and those we spoke to during our focus groups told us the Department informed sponsors if they had students ETS identified as having an invalid or questionable TOEIC test certificate. These and other stakeholders raised concerns about the Department’s engagement with sponsors:

The IPLA and Universities UK told us some sponsors felt particularly aggrieved by the Department’s actions because they had accepted students with TOEIC test certificates in good faith based on it being a Department-approved SELT.

Universities UK told us that in some cases the Department had given sponsors a ‘clear’ audit rating shortly before investigating them again with a view to suspending or revoking their licence. The Department told the institutions it could suspend, revoke or apply a zero-sponsorship allocation if the sponsor “posed an immigration risk”.

The UK Council for International Student Affairs (UKCISA) and Universities UK told us that some sponsors were asked to withdraw students without explaining why or providing proof that they had cheated. ILPA told us that in some cases the Department asked institutions to withdraw sponsorship from students, but some institutions refused. We heard similar examples from students, with some finding out about the accusations of cheating from their sponsor rather than the Department.

Sponsorship working group

3.20 Students who did not sit TOEIC exams were affected if their college had its licence revoked. They had to find courses elsewhere and some struggled to find new sponsors. In June 2014, the Department established a sponsorship working group (SWG) to look at how it could help students find other sponsors. The SWG included 17 representatives from the education sector, including the National Union of Students (NUS), UKCISA, the then Higher Education Funding Council for England and Universities UK.

3.21 The Department told us that during the SWG’s tenure it identified and wrote to 4,795 students in need of assistance. It made an online Course Information Tool available between November 2014 and March 2015, containing details of 62 licensed institutions with courses available. As of 22 January 2015, 837 students had used the portal.

3.22 Stakeholders, including Universities UK and UKCISA, raised concerns about the effectiveness and progress made by the SWG and the lack of senior sponsorship from within the Department. Universities UK highlighted that officials running the SWG were also involved in enforcement action against students accused of cheating.

Action against individuals

3.23 The Department’s analysis of exam results, knowledge of the sector, evidence from investigations into test centres and colleges and the findings of Panorama convinced it that there had been mass cheating. During 2014, it began matching people from ETS lists to Department systems so it could confirm it had the right people and could take action against them.

3.24 The Department flagged anyone with invalid or questionable results for some form of investigation. This included confirming someone’s identity and wider work to check what other information was held on its systems. As at February 2016 the Department had identified 51,000 potential matches on its system. Further case work reduced the number of potential matches to around 42,000. Its investigations confirmed it had the correct identity for 11,231 invalid cases and 5,038 questionable cases. It had 16,430 unconfirmed matches for invalid cases and 9,477 unconfirmed matches for questionable cases.

3.25 The Department cancelled the visas and refused all pending applications of everyone with an invalid test. The Department did not allow exam retakes for those with invalid tests because it assumed they all used deception to obtain their visa. The Department issued letters to people telling them their visas had been revoked and to leave the country immediately.

3.26 People with questionable results were given the benefit of the doubt but told they would need to retake the test if they wished to extend their existing visa. The Department required anyone in this group wishing to extend their visa to take a new SELT with another provider and attend an interview with the Department. Its policy was to take no action against people in this group unless their leave to remain in the UK had expired.

3.27 By the end of March 2019, Department records showed that at least 11,356 people associated with an invalid or questionable TOEIC test result had left the UK. Most of these left voluntarily (Figure 10). The data indicate that at least 5,000 people left within a year of being flagged on Department systems, and at least 8,000 people left within two years. Some 2,468 people were subject to enforced removals by the Department, and 391 were refused re-entry to the UK on arrival at a port. People may have departed for reasons other than allegations of using deception in TOEIC tests. The numbers are based only on individuals matched to invalid or questionable TOEIC certificates and may be underestimates.

3.28 UKCISA provided evidence from several institutions of cases where border staff had detained students at the border, sometimes for several hours, for suspicious TOEIC tests. Two students claimed they had never sat a TOEIC test. UKCISA told us it had passed the examples to UK Visas and Immigration, who agreed to clarify instructions to Border Force staff. The Department told us it was not aware of any such examples and that only people with an invalid certificate or other evidence against them should have been refused entry.

3.29 As at March 2019, data indicated 25,000 people had been subject to some form of Department action. Of these, 17,500 had an invalid flag and 7,500 had a questionable flag. Around 7,000 people had been granted leave to remain, including 4,157 people with an invalid flag. Department action may have been related to reasons other than allegations of using deception in TOEIC tests.

Appeals

3.30 Thousands of people accused of cheating have still won the right to remain in the UK. Data indicate 4,157 people with invalid cases had some form of leave to remain in the UK as at March 2019. Some 532 people had indefinite leave to remain, 477 were now British citizens and 2,682 had leave to remain of more than a year. Numbers represent people with some form of leave to remain as at March 2019 but may not represent the overall number of people that gained leave, because they are based on a reporting tool originally designed in 2014 and it may not capture some changes to people’s immigration status. The Home Office has not tracked the reasons why people have since been allowed to stay in the country.

3.31 Around 12,500 appeals involving individuals matched to invalid or questionable TOEIC certificates have been heard between April 2014 and March 2019. In most cases, appeals had to be made on human rights grounds. The Immigration Act 2014, which became law in May 2014, removed international students’ right of appeal against applications for leave to remain in the UK. Students could challenge refusals of leave through judicial review. Where students made a separate human rights claim to the Department, and that claim was refused, the student could lodge a human rights appeal.

3.32 Individuals with TOEIC flags have won some 40% of First-tier appeals compared with 60% by the Department up to March 2019 (Figure 11). The Department told us it could not provide accurate numbers on the total number of appeals people had won specifically against allegations of TOEIC deception because it had not carried out significant analysis on this in recent years, and HM Courts and Tribunals Service records these appeals as human rights appeals.

3.33 Up to May 2019 the Upper Tribunal had heard 1,720 appeals against First-tier tribunal appeals. Figure 12 shows that the Home Office overturned 167 (20%) of the cases it appealed while 231 (27%) were remitted for a fresh decision. Individuals overturned 178 (20%) of the cases they appealed with 182 (21%) remitted for a new hearing. The Department appealed three-quarters of the cases it lost up to September 2016 as it sought to establish case law. It told us that since mid-2016 its policy has been to only appeal cases where the court has not followed case law.

3.34 The Department has not formally analysed the reasons it lost appeals since September 2016 or considered whether its losses in appeals may indicate that significant numbers of innocent people had been wrongly accused. It did analyse the reason it lost 385 appeals up to September 2016. Figure 13 shows that 17% of people won appeals because they demonstrated they took a test legitimately. A further 57% of people won appeals because cheating was not proven. The Department told us that it undertook this analysis before case law was settled on handling appeals in this area. It estimated that the proportion of people demonstrating they took the test legitimately would probably have increased since mid-2016 but the numbers where cheating was not proven would have declined because the courts had found that the Department’s evidence was sufficient to make the accusation of deception.

3.35 People’s rate of success in appeals would have improved over time as human rights factors become more significant when people have been in the UK for a long time, for example because they have British children. The Department provided us with limited analysis it ran in May 2018. This showed 28 appeals involving TOEIC cases decided in the month. Of these, 18 were allowed and 10 dismissed. Of those allowed, the appellant had a British child or spouse in almost every case. In three cases the appellant did not have a British relative. None of the dismissed cases involved a British relative.

3.36 The removal of students’ right to appeal in the UK is likely to have reduced the total number of appeals from 2015. Stakeholders told us that international students found making appeals from overseas difficult because of:

high costs: legal recourse can be expensive and some students are not entitled to legal aid. The students we spoke to explained that travelling to the UK to study was a significant financial investment. Migrant Voice told us it was aware that some students had spent more than £40,000 challenging the Department’s decision in court, and that this was typical of many students; and

limited access to UK-based litigation from abroad: the Joint Council for the
Welfare of Immigrants (JCWI) told us that instructing lawyers and engaging with the UK legal system is much harder from overseas. International practitioners may be less familiar with the UK legal system and liaising with UK-based individuals may prove challenging due to time-zone differences and issues with technology. JCWI noted that getting witnesses to give statements is much harder when trying to contact them from abroad, and that reliable technology for video conferencing does not exist in the courts, which can make it difficult to give evidence or to engage in hearings.

3.37 In December 2017, the courts ruled that out-of-country appeals did not currently offer sufficient protection to human rights given the importance of oral testimony. The Department told us it was investing in video conferencing and had successfully demonstrated this in a number of cases in recent months. It told us it was, however, taking a pragmatic approach with people who remained in the UK. Given the passage of time the Department viewed it as likely that many individuals who had not left the UK would by now have some arguable human rights claim. The Department said it was usually willing to consider such a claim in the course of existing litigation action. Even if this claim was refused it would ordinarily attract an in-country right of appeal which would allow an independent member of the Judiciary to consider an accusation of TOEIC deception.

1 Independent Chief Inspector of Borders and Immigration, An inspection of the effectiveness of the monitoring of Tier 4 sponsor licences, July 2015.

Appendix One

Our investigative approach

Scope

1 In 2014, the BBC’s Panorama programme uncovered examples of organised fraud within the Tier 4 points-based visa system. This included cheating on the Test of English for International Communication (TOEIC), a Secure English Language Test (SELT). We investigated the Home Office’s (the Department’s) response to the issue. Our report:

describes the Tier 4 system and SELT arrangements at the time (Part One);

examines the Department’s efforts to determine the scale of fraud in SELTs (Part Two); and

describes the action the Department took in response (Part Three).

Methods

2 In examining these issues, we drew on a variety of evidence sources:

We interviewed key individuals from the Department to understand their approach to investigating and determining the level of fraud, and subsequent action, including the Department’s operational response, criminal investigations and litigation.

We reviewed:

a range of Departmental documents to understand the Department’s approach, analysis and quality assurance, including: quantitative analysis, operational reports, ministerial submissions and independent reviews;

reports by forensic experts on the methods used by the Educational Testing Service to determine the scale of cheating; and

evidence from other independent commentators on the Department’s response, including the Independent Chief Inspector of Borders and Immigration, the National Union of Students and Migrant Voice.

3 We spoke to stakeholder groups to understand their views on the Department’s response, including:

Bindmans LLP;

English UK;

Immigration Law Practitioners’ Association;

Joint Council for the Welfare of Immigrants;

Migrant Voice;

National Union of Students;

UK Council for International Student Affairs; and

Universities UK.

4 We ran focus groups with students who have been accused of cheating in their TOEIC test and claim they are innocent.

Design and Production by NAO External Relations
DP Ref: 006629-001