Groups of students could be disadvantaged by the marking system this summer

Some students cram hard just before the exam, needing the adrenaline to sharpen their senses and give them a boost. Others do well in course work but crumble under the pressure of a public exam. Anecdotally we all know of someone who tanked in mock exams but then came away with A*s the following summer. 

My own GCSE mocks weren’t great. Actually, even that’s an understatement. And if I ended up with the grades the schools said I’d get, my life would have been very different. FrankIy, I was lucky my dad took an interest and found a way to help me. The budding lawyer in me wrote down the things my dad said he’d do to encourage me.

Results day was an interesting experience and on seeing my grades - enough to get me into my first-choice college - my dad quickly tried to change the deal and, in particular, the money part. What happened? Well that’s a whole other story, but I did get to take the path I wanted and to eventually become an education lawyer. As you’ve probably guessed, my mock exams weren’t a true indicator of my ability and as a result I was “underestimated”. Thankfully, my mocks cost my father – yes, he paid up - and not me. Thank you, dad.

Twenty or so years on, we’re in lockdown and students are hoping that their mock results won’t cost them their future. In these unique times, the Department for Education has cancelled the summer exams and has set out a system for allocating final grades. The new process relies on teacher assessment for both the final grade awarded and a student’s rank in relation to peers.

Teacher based assessment is nothing new …

Teacher assessment is not a new tool for awarding marks to school students - it has been used in primary schools for assessing 7-year olds. The difference is that 7-year olds do not need to rely on allocated grades for access to sixth form college, university courses or the workplace. Teacher-allocated grading favours some students over others and disadvantages certain groups of students.

EDSK is a think tank providing a new perspective on education and skills. Their mission is to design new and better ways for policymakers and educators to help every learner succeed, particularly those from disadvantaged backgrounds. I spoke with Tom Richmond, the founder and director of EDSK. Tom says there is certainly evidence that some groups of students could be disadvantaged by the marking system being put into place this summer.

Tom cites evidence from the government itself. In 2011, it conducted a review of testing in primary schools which discussed the weakness of teacher led assessment. The Bew review was an independent review of key stage two testing. 

Published studies from the UK and abroad have also looked at the drawback of teacher assessment, particularly for minority groups. A study by Professor Berger suggests that on average black Caribbean and black African pupils are under-assessed relative to white peers. By contrast, Indian, Chinese, mixed Asian and white pupils are sometimes over-assessed. This study proves that relying solely on teacher assessment could disproportionately affect black pupils.

In 2016, the Australian Journal of Education published Bias in Grading: A Meta-analysis of Experimental Research Findings. The researchers looked at the existence of bias in the subjective grading of student work, such as essay writing. All studies involved the graders being exposed to a specific type of information about a student other than the student’s performance on a task. The biasing characteristics included different race or ethnic backgrounds, education-related deficiencies, physical unattractiveness and poor quality of prior performance. The results suggest that bias can occur in subjective grading when graders are aware of irrelevant information about the students, just as many schoolteachers are. 

More than just a statistic …

As the pandemic hit, in late March 2020 teachers actively lobbied to be able to predict their students’ grades. A teacher started a petition on change.org requesting the government to allow teachers to set predicted grades for their students. The teacher explained that as teachers, they know their students ‘as more than just statistics’ and that they were not going to try to ‘cheat the system’. In response, many students and some adults affected by an unequal system took to Twitter to air their concerns and share experiences of bias. Many told stories of how they were held back by low expectation or how they had obtained a better grade than they were predicted. 

It is surely because teachers know their students that some students are not happy with the prospect of teacher assessment. Assumptions about students’ abilities are, as the research shows, affected by favouritism and racial bias.

We also know that bias impacting certain groups of pupils is not limited to teacher assessment outcomes. Research by LKMCo in 2018 looked at underachievement within specific groups of boys in London and identified teacher bias in every aspect of their school life, from disciplinary measures such as school exclusions to everyday matters such as setting expectations and class allocation. 

Alarm bells

Precisely because some groups of students are already impacted by unconscious bias, a system relying on teachers to set final grades this year set off alarm bells among some student and community groups.  

Ofqual sent out letters and guidance to students and exam centres setting out how the final marks would be awarded, as well confirming a mechanism for appealing the final grade. The tone was undoubtedly reassuring, confirming that students would be able to progress to the next stage of their education or vocation as expected in the autumn, lockdown permitting. 

The details in relation to the appeal process are vague, but the right of appeal is there in black and white. Now it seems this right of appeal will be extremely limited. Currently, the regulator has said that appeals should only be allowed on the grounds that the school or college made ‘a data error’ when submitting its information; or similarly, that the exam board made a mistake when calculating, assigning or communicating a grade. 

Uncertainty ahead…

It has been reported that college heads and headteachers might want to limit the autumn exams to core subjects only, because holding a full set of exams will be disruptive. The scope of the autumn exam series is yet to be decided and discussions are ongoing. We live in a rapidly changing world – and the process may change again – and fast.

Acknowledging bias …

Ofqual published an equality impact assessment on 15 April 2020 and acknowledged that ‘some research studies of potential bias in teacher assessment saw that differences between teacher assessment and exam assessment results can sometimes be linked to student characteristics like gender, special educational needs, ethnicity and age’ but countered that ‘the effects, when present, are small and inconsistent across subjects.’

The equality impact assessment also looked at data held by UCAS which includes A level grade predictions made by teachers for their students' university admission applications, as well as reviewed studies that made use of individual exam boards' data to examine the accuracy of the GCSE and A level grades that exam boards used to collect from teachers. Of great concern was Ofqual’s conclusion that ‘there are likely some effects on prediction accuracy of ethnicity (specifically, more over-prediction for some ethnic minority groups) and disadvantage (specifically, more over-prediction for the more disadvantaged in general, and less over-prediction for the more disadvantaged among high attainers)’ but acknowledged that ‘those effects have not been properly estimated’. So, the regulator is aware of the bias but not the effects. 

When in doubt consult …

The solution to the many questions seems to be to offer a formal consultation, so on 15 April 2020 Ofqual announced a consultation in relation to this summer’s exams, seeking views on key aspects of its published guidance. That consultation closes today.

Appeal in name only …

Ofqual is specifically consulting on how the appeal should operate and has highlighted the limited nature of appeal on offer, pointing out that a student’s grades can go down as well as up following an appeal. 

• Ofqual’s view is that a student can ask their exam centre to ‘review’ the accuracy of data that they submitted to the exam board. Ofqual say that if there is an error in the way of the exam board ‘processed’ the data submitted then the exam centre itself can appeal to the exam board. This offers an arm’s length appeal limited to data processing, only, to students who feel that they were under assessed by their teachers. 
• Ofqual states that appeals should focus on whether the ‘right’ data was used and correctly applied. But it qualifies this by saying the appeal should not focus on the teachers’ professional judgement or the pupil’s likely performance in the exams had they gone ahead. 

One of the unknowns of the process is how results will be standardised across the country, from exam centre to exam centre. The regulator’s guidance to students and exam centres says that a yet to be determined mathematical formula will be used. However, Ofqual does not think that it is appropriate for exam centres or for students to appeal on the grounds of the standardisation model. 

So how would we challenge a teacher’s judgement that we disagree with? 

The consultation paper is frank and states that students who believe they would have received a better grade had they taken their exams can go on to do just that in the autumn, essentially saying ‘if you think that you can do better than your allocated grade, prove it!' That’s all well and good, except the dates for the autumn exam series haven’t yet been set. Most college and university places are confirmed in August and start in September. Ofqual has already said in previous guidance that both the allocated grade from the summer and the attained grade from the autumn exams will stand. We are not sure how that will look on your exam certificates, but what is clear is that at present there isn’t a mechanism for removing an allocated grade from a student’s list of final grades. 

Summertime sadness?

Hopefully, by results day in August the lockdown will be over. Many students will get the grades they deserve; however, we know that unconscious bias can and will affect teacher-based assessment and education attainment generally amongst certain groups of students. I suspect I may be busy challenging final marks on behalf of students who feel their grades aren’t a true reflection of their ability… and who knows, I may also have to challenge the adequacy of the appeals process itself.