Matthew informed me of the mess in England about school exams, which is causing huge difficulties there and leaving a lot of students at their wit’s end. I asked him to tell us a bit about it, as reading online only confused me. So here’s Matthew’s explanation.
by Matthew Cobb
The future of a generation of English school-leavers is being blighted by the pandemic and by government incompetence. Their end-of-school exams—A-levels—did not take place, so the government devised an algorithm to determine their marks (used for university entry, but also by future employers). The algorithm took into account the teachers’ predictions of the grades (teachers do this every year for the weird UK university entrance system, whereby students are made offers partly on the basis of these predictions), but then moderated them on the basis of the school’s bad performance. They checked the algorithm against last year’s results, and found that it made mistakes in over 2/3 of cases, and that state school pupils were systematically downgraded while private school pupils were not, but they went ahead with it anyway.
As you might expect, there has been a massive outcry and a huge sense of injustice as many teenagers have been deemed to fail exams they did not even take, and promised university futures have evaporated through no fault of the students. A new slogan has emerged on demonstrations that we will undoubtedly hear more of as the 21st century progresses: ‘Fuck the algorithm’. The government said students would be able to appeal against the grading, starting today, but over the weekend that option was removed without explanation. The grisly details of all this—in particular the incompetent and unqualified cronies who are in charge of various decisions—are even more grisly than I have set out here, but maybe UK-based readers would like to add to this in the comments.
Later this week the malfunctioning algorithm will be used to determine the marks of 16 year olds, for their GCSE exams. Already Northern Ireland (which has control over its education system) has said it will ignore the algorithm and will simply use teacher’s predicted grades for the GSCEs. Exactly the same thing happened in Scotland (which has an entirely separate education system and local control over it) a couple of weeks ago with their Highers. Faced with massive protest, the Scottish government had to backtrack and use teacher’s predictions. It seems probable that the English government will do the same, but slacker Johnson is on holiday, and anyway isn’t interested in anything resembling responsibility. For the moment, the algorithm results stand.
[EDIT: As expected, the English government has backed down and agreed that students in England will have their exam results based on their teachers’ predictions (‘centre assessment grades’). Ofqual, the body in charge of this utter fiasco, has said:
We understand this has been a distressing time for students, who were awarded exam results last week for exams they never took. The pandemic has created circumstances no one could have ever imagined or wished for. We want to now take steps to remove as much stress and uncertainty for young people as possible – and to free up heads and teachers to work towards the important task of getting all schools open in two weeks. After reflection, we have decided that the best way to do this is to award grades on the basis of what teachers submitted. The switch to centre assessment grades will apply to both AS and A-levels and to the GCSE results which students will receive later this week.
Good. Now comes a few more days of chaos as students, schools and universities try to sort out the mess and, in some cases, find places for all the extra students who they will now have to take on board.]
What a messed up “system”!
In hindsight, they should have held the exams.
Now that we are where we are, I can’t see any way out of this that isn’t a mess.
I think you’re right.
It is remarkable that both the US and the UK managed to get catastrophic leadership at the same time. I suppose it is a measure of the “special relationship” between our countries.
I am a parent of one of those unfortunate 16yos. This is going to be a challenging week. But my daughter already has her 6th form place, so it may be that her grades will have a minimal impact on her future, but it’s the lack of confidence in our government that will live on that will in turn impact our education system.
I think the A level students had it worse, with their university places being at risk.
I think it is good that NI (and Wales too) have outright said that they will take the predicted grades. We need these kinds of firm decisions from more areas (like the universities for the A Level students) because good firm decisions like shine a bright light on Boris and his Balls up.
Well the problem is that, if you accept the predicted grades from teachers, everybody does better because predicted grades are almost always optimistic. Apparently, in NI, 40% of students were predicted to get an A or an A* and overall, results would have been 12% better than last year.
I do not know how that would have affected the universities, but I would have thought they would have the resources for only a certain number of students, so you probably would find students with qualifying grades missing out.
As I said above, I can see no way out of this that isn’t a mess.
Yeah, they should definitely use the American system where rich people pay* to have their kids put into the finest schools and everyone else has to be in the top 1% of their graduating class to get a chance to owe tens of thousands of dollars for their education.
*pay someone to take their kid’s SAT; buy a building; create an endowed professorship; buy new equipment for the University football team; etc
+1
The Times are reporting a U-turn today at 4pm.
https://twitter.com/thetimes/status/1295355264523173894
This Government simply goes from one disaster to another, but still the electorate supports it. I despair for our democracy at times like these.
And yet there’s a reasonable argument that people thought a Corbyn government would have been worse, and voted accordingly in the last General Election. If enough people come to think differently by the next General Election then they will vote differently. That is democracy – but it is not as fast as some people want.
Rather to the point, as slogans go.
What it lacks in politesse, it makes up for in pithiness.
Surprise, surprise the algorithm wasn’t used in the same way for schools where only a small group of students were studying a subject, and in those cases the results were pretty much based on the teachers’ predicted grades. By an amazing coincidence, the vast majority of these small groups are made up of pupils at public (which, in the UK, confusingly means private!) schools.
The exams regulator Ofqual has just announced A-levels and GCSE results will be based on teachers’ predicted grades in England. Northern Ireland and Wales announced the same decision earlier in the day, and Scotland did so a while ago.
Ah, the age of the algorithm. If only HAL 9000 knew.
His response to “Fuck the algorithm” would probably be something along the lines of, “I’m sorry, Dave. I’m afraid I can’t do that.”
Hahaha. Oh man, too bad you weren’t on the screenwriting team! Where is that franchise sequel when you need it!
Update:
They are going to use teacher predicted grades….
Govt climbdown after protests
Here’s a nice piece explaining the situation. It was written shortly before the government’s u-turn: https://www.bbc.co.uk/news/education-53787203
The algorithm took into account the teachers’ predictions of the grades . . . but then moderated them on the basis of the school’s bad performance. They checked the algorithm against last year’s results, and found that it made mistakes in over 2/3 of cases . . . .”
I wonder on what if any objective (and also surely unavoidably subjective) bases teachers arrive at their predictions. (Will consider putting in-depth research on this topic on my long To Do list.) I gather that the best objective basis would be some kind of test. But since tests were not given, the next best measure would be past objectively-documented student performance. (But then it seems that one should not totally ignore the type of disclaimer given by investment management types, to the effect that past performance does not insure similar future performance.)
I suppose students who have mediocre academic track records might protest this approach, saying they have no doubt that they would do better. Would that they had so thought earlier in their academic careers. C’est la vie.
Re: ” . . . moderated them on the basis of the school’s bad performance . . . .” I gather that that bad performance is a statistical average. If a student has consistently outstandingly performed, why should her/his prediction, however arrived at by the teacher, be “moderated” (lowered!) by some tyrant of an algorithm on account of the school’s overall bad performance, to which the student hardly if at all contributed one iota of “badness”?
(For some reason I’m reminded of student academic group work, where 1-2 of the 4-5 in the group do the vast majority of work yielding an outstanding grade, but everyone is treated as having equally contributed to the result.)
Teachers’ predicted grades are almost always a bit optimistic. You assume the student will do his or her best work in the exam and so forth.
Also, a BBC story that came out just before the results quoted a figure of 16% for accurate prediction of grades by teachers in previous years.
On one level, the approach was quite sensible: the grades are going to be optimistic, so we’ll add a correction. If your sole objective is to get the right distribution of grades, that’s fine and probably the same number of students didn’t get their grades as in any normal year. It’s gone wrong because, in normal years, students fail to get the right grades through their own fault.
There were also some outright mistakes. The BBC managed to find one student predicted to get A’s and B’s who got D, E and U. That’s not the algorithm, that’s a straight up human error. Somebody typed in incorrect grades for that student.
Most students would have taken mock exams in the spring, before schools closed. I guess that is some basis for the teachers’ estimates.
Why not do it backwards? You have the placement results; you have data on the students’ grades. Run a principle component analysis or multifactorial analysis and see what component(s) of a students’ resume best predict their GCSE’s/placement.
I’m not necessarily advocating using that instead of teacher’s predicted grades, but look, if you’re going to make an algorithm for something like this, and you have data about who went where in the last year, use the freaking data to help develop your algorithm.
The concept of failing an exam that one hasn’t even taken has a certain resonance. It reminds me of Orson Welles’ parable of
The Gate of the Law at:
What about all those who already went into Clearing (where those who fail to get into top choices get other possibilities)?
The UK government is as Matthew says, run as a way of offering contracts to their finders… sadly more Americanisation of British politics as we move towards a more powerful government with less collective responsibility & more power in the hands of the PM.
I hate Johnson…
Finders = finders!
Funders!!! Effing iPhone!!!
The situation is massively complicated because the government didn’t bite the bullet and make this decision before the A-level results were announced. Students who have accepted places at their second choice university, possibly on a different course, will now have met the entry requirements for their first choice after all. What do they and both universities involved do now? The government has lifted the cap on student numbers, but for many courses with restricted access in terms of spaces in labs and numbers for work placements (e.g. medical students who will need hospital training places) that creates its own problems. And some universities promise all first year students accommodation on campus – assuming that’s a thing, for the new academic year – but again, that’s based on the expected intake with the government limit on new students. This mess is going to take a while to sort out. And the universities are going to want extra cash to deal with it.
For efficiency, you can’t go past the Staircase Method of grading exams.
https://media-exp1.licdn.com/dms/image/C5612AQHtLvcrbzeCNg/article-inline_image-shrink_1000_1488/0?e=1603324800&v=beta&t=Zz3pvZZM5J-aIptwqB49q9WAU4mGy9FkIvJR4C_qMnY
https://www.linkedin.com/pulse/20140414044726-2259773-a-guide-to-grading-exams
See, it is great contributions from readers like this that are one of the reasons why I keep coming back!
Yes, the whole thing is a complete dog’s breakfast; but it is not just a question of a dodgy algorithm.
It is not ‘the government’ that devised the algorithm in question, but the Office of Qualifications and Examinations Regulation (Ofqual). They were under pressure from a number of vested interests, including MPs and others who were concerned about what they saw as unconstrained grade inflation over recent years.
Now that the resulting formula has been shown to curtail grade inflation, but at the expense of huge inequity in the results of individual students, people are changing their tune. Ofqual have not exactly taken a sophisticated approach to the issue; but they have had an impossible job to react to the conflicting pressures of central Government, MPs, schools and parents.
We are now back to relying on teacher predictions, which are hugely variable and therefore unreliable. Stand by for GCSEs next week!