Nothing wrong with official evidence of English language test cheating, Upper Tribunal holds

This decision forms the next episode in the saga of cases arising from the Test of English for International Communication (“TOEIC”) certificates obtained from test centres in the United Kingdom administering tests set by the Educational Testing Service (“ETS”).

The BBC Panorama programme broadcast on 10 February 2014 exposed widespread cheating in tests of English needed for immigration purposes. Subsequent investigation revealed that many thousands of results were obtained by fraud, many of them by impersonation of the candidate at the speaking test…

In this decision it is possible to examine the evidence on which the Secretary of State relies to establish the frauds in individual cases. We conclude that despite the general challenges made, both in judicial proceedings and elsewhere, there is no good reason to conclude that the evidence does not accurately identify those who cheated. It is amply sufficient to prove the matter on the balance of probabilities, which is the correct legal standard.

In open the judgment in DK and RK (ETS: SSHD evidence, proof) India [2022] UKUT 112 (IAC). The Upper Tribunal found that “there is no reason to suppose that the voice recognition process was substantially defective”. Anyone seeking to challenge voice recognition evidence that they probably cheated on the test will need to have something concrete to show that a mistake has been made in their individual case.

Background

The Home Office has never seemed particularly interested in sifting the innocent from the guilty in its response to undoubtedly widespread cheating on English language tests. As the National Audit Office found, “the Department’s course of action against TOEIC students carried with it the possibility that a proportion of those affected might have been branded as cheats, lost their course fees, and been removed from the UK without being guilty of cheating or adequate opportunity to clear their names”.

These concerns were canvassed in some detail in a July 2019 report by the All-Party Parliamentary Group on TOEIC. The MPs heard evidence from lawyers and technical experts in voice recognition and concluded that there were “fundamental flaws in the evidence” that ETS gave to the Home Office as proof of cheating in individual cases. The appellants in this case wanted the Upper Tribunal to engage with that report and, quite possibly, rule that the whole notion of revoking people’s immigration statuses off the back of the ETS evidence was flawed.

The tribunal refused to admit the MPs’ report itself but did consider the transcript of what the experts had said. It also heard directly from Professor Peter Sommer, a digital forensics expert.

Expert evidence

The judges were less impressed than the politicians. Their starting point was the “overwhelming” evidence of “widespread fraud” at ETS-run testing centres. Assertions to the contrary are “simply not open to anybody with any regard for the rule of law or the value of truth”.

While this didn’t prove that “any individual certificate was obtained fraudulently”, it made any individual allegation “more plausible” than if there hadn’t been a BBC exposé and raft of criminal convictions.

… the general evidence changes the starting point. The possible response to the assertion of fact moves from disbelief that such a thing could ever – or could regularly – happen, to a specific enquiry about whether one of the events that certainly did happen was associated with the individual under investigation.

They then examined the techniques used by ETS to spot cheating. This involved voice recognition software, backed up by human listeners, identifying the same voice appearing in multiple oral exams. Test results found “invalid” for impersonation then had to be matched to the particular candidate: a matter of record-keeping rather than anything else. The tribunal heard from a Home Office analyst on the integrity of these results and found him “wholly reliable”.

By contrast, the latest expert evidence casting doubt on the ETS results appeared less convincing in the cold light of judicial scrutiny:

The difference between the caution employed by Professor Sommer, in particular, in expert opinions for court use and in what he said at the APPG session is striking. In the former circumstances he is cautious about reaching conclusions on the basis of the evidence available to him… In the APPG proceedings, however, he is content to begin his narrative by saying that ‘it was unsafe for anybody to be relying upon computer files generated by ETS and used by the Home Office as a sole means of making a decision’. On the other hand, he was not able, he admits even in that context, to ‘identify specific points at which things had gone wrong’, but that ‘the administrative arrangements and the audit trails were simply not present’.

Even allowing for the more relaxed format of the MPs’ evidence session, the tribunal found, “we cannot find anything in the way of facts in the transcript substantially to undermine the existing evidence adduced by the Secretary of State”.

Overall, 

the truth of the matter is that although those who have their own process for voice recognition examination, can suggest other ways in which this examination of the data could have been made, there is no reason to suppose that the voice recognition process was substantially defective… We conclude that the voice recognition process is clearly and overwhelmingly reliable in pointing to an individual test entry as the product of a repeated voice. By ‘overwhelmingly reliable’ we do not mean conclusive, but in general there is no good reason to doubt the result of the analysis.

Similarly, fundamental errors in the record-keeping “remains only a theoretical possibility not supported by any detailed evidence”. While the possibility of error “prevent[s] the data conclusively proving fraud in a scientific sense”, it did not prevent a conclusion that someone had cheated on the balance of probabilities, in which context “showing that the case is not watertight is not sufficient”.

What does all this mean for others trying to appeal against adverse TOEIC findings in future? The tribunal found at paragraph 126:

Where the evidence derived from ETS points to a particular test result having been obtained by the input of a person who had undertaken other tests, and if that evidence is uncontradicted by credible evidence, unexplained, and not the subject of any material undermining its effect in the individual case, it is in our judgment amply sufficient to prove that fact on the balance of probabilities.

It is hard to think of a judgment less sympathetic to the idea that there is anything fundamentally wrong with the standard evidence against TOEIC appellants. In describing it as “amply sufficient”, the Upper Tribunal goes further than in SM and Qadir [2016] UKUT 229 (IAC)., in which the ETS analysis was considered only just good enough. It also describes the approach taken to the evidential burden of proof in Muhandiramge [2015] UKUT 675 (IAC) as “wrong”, a point reflected in the terse third paragraph of the headnote.

The official headnote

1. The evidence currently being tendered on behalf of the Secretary of State in ETS cases is amply sufficient to discharge the burden of proof and so requires a response from any appellant whose test entry is attributed to a proxy.

2. The burden of proving the fraud or dishonesty is on the Secretary of State and the standard of proof is the balance of probabilities.

3. The burdens of proof do not switch between parties but are those assigned by law.