by Greg Mayer
The title of this second post in what I hope will be a continuing series comes from the title of a new article in The Atlantic by Joe Pinsker. In the article he discusses an unpublished paper by Jeffrey Denning and colleagues at Brigham Young University. It highlights one of the chief ills of American higher education: that the goal of the system is not education. As noted in my first post in this series, the neoliberal consensus in higher education holds that the goal of higher education is maximization of monetary value. For businesses and legislators, this means colleges should train workers that industry needs in order to maximize profits; for labor advocates, it means training workers for jobs that pay well.
Though it might seem that achievement of this goal can be measured in dollars earned (by businesses or workers), such economic data can be difficult to gather, and suffer from the fact that it can take many years for the payoff of an investment to be evident. In lieu of this, some measure of success in reaching this goal must be agreed upon, and businessmen, legislators, bureaucrats, and educational think tanks, with the ready assent of college administrators, have settled upon graduation rate as the metric by which to measure the results of higher education.
It is to these rates, and how and why they are changing, that Denning et al. address themselves. Noting that graduation rates have gone up recently, but that there are no demographic or academic reasons to readily explain this, they consider that maybe it’s just easier to graduate. Money quote from Pinsker:
If grades are improving but there’s no reason to think that students have become better students, an interesting possibility is raised: The unassuming, academic way Denning puts it in a recent paper (co-authored with his BYU colleague Eric Eide and Merrill Warnick, an incoming Stanford doctoral student) is that “standards for degree receipt” may have changed. A less measured way of saying what that implies: College may have gotten easier… altering what’s necessary to get a degree is “the lowest cost way to increase graduation rates.”
Now, on the face of it, a high graduation rate would seem to be a good thing, and it is; but when you make a metric the goal, then the metric can be gamed, which is what Denning et al. suggest is being done by colleges. This is a well known phenomenon, nicely summarized by Abhishek Chakraborty:
It has been established that when you measure effectiveness solely based on quantitative indicators, people involved have a high incentive to demonstrate less ethical behaviour, and most likely less effective results as well. This is called Campbell’s Law. [emphasis added]
Donald Campbell was an academic psychologist, who expressed his law this way:
The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. [paywalled]
Frederick Hess gives a nice example:
The first time I heard of Campbell’s Law was in a college class in public policy. The professor asked, “Can data ever cause problems? Can it ever hurt?” It seemed like a trick question. Pretty much in unison, the class uncertainly mumbled a version of, “I don’t think so.”
The professor then asked, “What if a police department decides to evaluate officers based on the number of traffic tickets they write? Could anything go wrong?” Someone observed that cops would try to write lots of tickets—including for people who might not deserve them.
The professor asked, “Okay, so what if they flip it? What if they reward cops who issue fewer tickets?” Well, duh. Police might turn a blind eye to real problems.
The instructor smiled and said, “See, you can think of lots of ways where data might hurt.”
Another term for essentially the same phenomenon is Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.”
It seems to me that much of what passes for “reform” in higher education is an instantiation of Campbell’s and Goodhart’s Laws. Under pressure to show results responsive to the neoliberal consensus, measures, which might even have merit, are turned into the purpose of the system, and almost any method may be resorted to to achieve it. It is ironic that use of this measure may pervert even the neoliberal goal: if graduates are required to learn less and have fewer skills, will they really be able to maximize business’s profits, or their own salaries?
sub
I graduated with BS from Tulsa in 77 and MS from Northwestern in 79. A lot of my friends children in Engineering schools seem to have a harder time than I had in my BS degree.
When I was in college in the mid 80s, my father used to tell me that the things that I was learning on college, he learned in high school. I don’t see any reason to think that that hasn’t progressed even further.
IIRC from reading, the need for remedial coursework has increased markedly in colleges. I don’t necessarily blame teachers for this, in that we live in a markedly intellectually indifferent (if not willfully ignorant and anti-intellectual) culture in the U.S. Re: Hofstadter’s “Anti-Intellectualism in American Life” and Jacoby’s “The Age of American Unreason.”
Truth.
I think I was aware of Campbell’s law, but I never heard it put so clear and concrete before – very interesting.
I think I was aware of Campbell’s law, but I never heard it put so clear and concrete before – very interesting.
Yes, I’ve always preferred Goodhart’s formulation, as it’s much more concise and, I think, clearer. It also makes clear how it applies to numerous situations and basically anything that involves bureaucracy, where success is almost always measured by metrics these days and the goal is to achieve the “correct” result as measured by those metrics.
This has been apparent for decades. It arose in big organisations as soon as ‘key indicators’ started being used to assess the performance of work groups.
It certainly used to skew our project management processes – and not in a good way. It actually cost money to achieve arbitrary ‘targets’ – both ours, and construction contractors who would have to jump through hoops and rush parts of their jobs.
As another example, I fell off a pipe and sprained my shoulder. Subsequent visits to a physiotherapist (she was gorgeous, by the way) were paid for by the company but I booked my physio time to ‘the project’ so it didn’t count as a ‘lost-time accident’ and spoil their statistics…
cr
Here in the UK there seems to be a marked increase in the number of first class degrees being awarded and concern that this rise cannot be accounted for. https://www.bbc.co.uk/news/education-48951653
The cost of tuition increased significantly at British universities back in 2012, so you can take your pick of explanations for the increase in the award of top degrees. Either students are working harder because college is more expensive and they want to maximise their investment. Or students now regard themselves as customers, and universities feel obliged to award them the class of degree they paid for. Or maybe all of our students are now above average.
My degree was almost 20 years ago – as a mature student – & I was not a star student – but I was consistent. My most satisfying claim is not my high marks, but that I got no mark below 60%. I dropped zero subjects, & put effort into all course units, & I think you can make a case for seeing the spread of marks as you do on one part of the certificates one gets.
However, having interviewed & short-listed, I can say that just having ANY degree ticks a box. Shortlisting & interviewing is another part of the same issue – I am unconvinced that it gets you the right people.
“… and concern that this rise cannot be accounted for.”
It’s very easy to account for the rise!
1) Universities started getting ranked in league tables, where one of the metrics was what fraction of first-class degrees they awarded. So they started awarding more of them.
2) Universities started getting judged according to indicators such as surveys of student satisfaction. Students are more satisfied if they get higher marks and higher degree classes, so the universities started giving them what they want.
When I went to University in 1984, I think the percentage of school children that left school and took a first degree was somewhere around 10%. I think it’s now closer to 50%. If the percentage of first class degrees was to go anything but down, then there would have to be a dumbing down of standards (Disclaimer, I got a Desmond).
I have an anecdote that panders to my prejudice. Part of my job is to do technical interviews for people who want to work at my company. I was speaking to a fresh graduate of the University of West of England and I asked him to tell me about his final year project which he did in enthusiastic detail. There were several obvious ways in which he could have extended his project work, so I asked him which ones he had done. His response was that the lecturer absolutely forbade them from going beyond the tough syllabus in any way. Once he had done the set task for the project, he was under the impression he would have been penalised for going any further.
The article linked shows that math scores have fallen in the last 30 years. It does not mention the word science, physics, or biology.
There are a lot more online services for those willing to look for help. I would have gotten better math and physics grades had YouTube existed when I was an undergrad and graduate. And I would have spent less time struggling.
It’s going to be complicated. If rates of graduation are climbing, it probably is in part because of Campbell’s Law as that seems to be a fairly unstoppable force. But there are so many more programs to choose from, so one can navigate through a program that is finely tailored to your interests and abilities. I think another big factor is that it is easier to “pay” for college through student loans. So more can graduate by taking on debt.
My somewhat recent (and second hand) experience is that reputable degrees are probably not easier, but if a student begins to struggle there will be an army of advisors who will try to sweep them into an easier program or reschedule the remaining courses over a longer period.
A sufficiently well funded student could easily pay 7-10 years of tuition to get a 4 year degree.
Hardly a ‘law’ but perhaps relevant:
Since an institution’s prime directive is to stay open, any means are warranted to do so.
In my long professorial life at a small liberal arts college I witnessed over the decades a steady inflation of grades. That is, a ‘C’ became a ‘B’ became an ‘A.’ Then pluses and minuses were introduced. While the notion was to give faculty a finer tool for evaluation, the effect was to make a ‘B-‘ into a ‘B,’ etc.
Each academic year saw higher grade-point averages and an increase in the number of students graduating ‘cum laude,’ and up. The old ‘gentlemen’s C’–a la G.W. Bush–became a ‘C+’ or disappeared altogether. And the only failing grades were for students’ almost complete abdication of academic responsibilities.
I can say with authority why all this happened: to keep enrollment solid. My school was heavily dependent on tuition, and parents and students (most of whom were A-students in high school) were not interested in paying the high prices if they weren’t virtually guaranteed an academic success that would lead to ‘a good job.’
To hell with the ‘mission statement,’ the curriculum and academic standards: give them a ‘good college experience’ and a credential.
The saddest part: intellectually and educationally, college as I knew it and taught it is over. Probably forever.
I remember looking in my aunt’s 1959 high school annual at the valedictorian’s and salutatorian’s pictures. Their GPA’s were listed, expressed in terms of 100 per cent. That is, the valedictorian’s GPA was say, 98.34, measured to the hundredth’s place. It strikes me that that is always as it should have been, as compared to letter grades, and to the 4.0 system. Nowadays one hears of a high school graduate having a GPA of 5.46. Fatuous!
Neo-Liberal? How did that happen? My understanding is that liberals tend to emphasize education for the way it broadens one’s mind and mindset — literature, history, science,the study of other cultures — whereas conservatives are all about the employment and the income. The increasing tendency to think of a university education as job training on a path to wealth really got started in the 80’s as rebellion against idealistic hippies — and was associated with being Republican. In general, the stereotype is that Democrats focus on minorities being able to earn a living wage, and become social workers themselves.
I don’t argue that liberal colleges aren’t interested in jobs. Sure. But I’m going to have to be persuaded that when it comes to the goal of education, “maximization of monetary value” is going to be ticked by liberals, neo or otherwise, over “ to become an informed citizen of the world.”
In the last high school I taught in our principal bragged to whoever would listen that something like 93% of the freshman had an A average. She could see nothing wrong with it. She said it’s all about “success”.She Gave me grief for not having similar percentages of A’s in my calculus classes. Aargh.
Lake Wobegon High School?
So you think grade inflation might partly be coming from the “Everyone’s a winner!” Self-Esteem faction of the Liberal Agenda? Maybe. But that doesn’t necessarily square with the get-a-high-paying-job mentality. You’re a winner no matter what level you’re employed at: Proud to Be Me.
I wonder how “proficient” the principal was in calculus? (It absolutely grates on me to have to hear a Romneyesque English/Political Science undergrad major – MBA/JD/Wall Street type presume to lecture students on the need to enter STEM fields.)
I think every person running for public office (at least at the federal level – the state and local level may be a lost cause), as a matter of self-respect, ought to know how to derive the quadratic formula from ax^2 + bx + c = y. There are far more complicated things than that in life.
Do I correctly recall that it is possible to get a non-STEM degree at one of the allegedly “prestigious” U.S. Ivy League schools without taking even one science and/or math course?
And this principal wouldn’t have known a datum
if it bit her in the ass.Worse
typing than usual as I broke my right arm yesterday and I am dictating and typing with my left fingers.
As for prestigious universities, when I was at Stanford in the 60s liberal arts majors had to take 17 units of math or science and everyone had to take some language. It may be quite different now.
What is it about quantitative reasoning that is so different in practice from other subjects?
The mathematician James Simon (I think that’s his name) claims that mathematically trained teachers – as in actuality being good at it – aren’t paid enough in public schools so they go elsewhere. So he started Math For America- a foundation that boosts pay for math teachers in New York.
I mention Goodhart’s (Campbell’s) law in many of my introductory statistics classes, and I’m just generally on a personal crusade to make it better known by mentioning it in conversation whenever possible.
The article is 100% true, and it’s old news in engineering. There is zero incentive to fail a student. Student course evaluations are, I think, correlated with the graduation rate incentive. Instructors want to maximize their numerical scores on student evaluations, and the way to do that is to ensure everyone gets a good grade and make them feel like they earned it. This means cutting back on the breadth and depth of topics and skills.
I think an aspect of that is there are more non-tenure track lecturer positions and they especially feel pressure to be kept on by their university.
Specifically by the non-STEM MBA/JD types running universities now.
This is fascinating. The Campbell/Goodhart Law sums up a good deal that is happening around us. For one thing, it summarizes how the well-intentioned policy of Affirmative Action led to the racket and then religion of Holy Diversity. A classic instance of the law was enacted in the late-lamented Soviet Union. Wiki explains as follows:
“Shturmovshchina (Russian: штурмовщина, IPA: [ʂtʊrmɐfˈɕːinə], last-minute rush, lit. storming)[1] was a common Soviet work practice of frantic and overtime work at the end of a planning period in order to fulfill the planned production target. The practice usually gave rise to products of poor quality at the end of a planning cycle.”
This seems quite evidently true to me. I understand that formally quantifying it is a difficult thing to do, but it is very obvious. Education has become highly commodified. Today any job above flipping burgers requires a college degree in order to be a competitive prospect. Such an increase in demand always attracts providers whose primary goal is profits. Especially when government subsidizes are also involved.
In most cases the employer doesn’t care about qualifying the pedigree of the diploma and for many jobs just about any kind of diploma will do. In many cases the requirement is primarily a means of reducing the stack of applications to a more manageable size.
These pressures combined with human nature and a lack reasonable oversight and regulation at numerous levels leads to a lowering of quality, a lowering of standards and an increase in wealth for a relative few. And not the students.
I think it’s safe to say that degrees in the physical sciences have suffered less than other categories. You do actually have to know some pretty technical stuff to earn your wage at a typical STEM job and you are more likely to be pretty visible if you don’t. But a large number of the jobs outside STEM fields are much easier to get by in and all to nearly all the useful knowledge necessary to be an adequate employee can and are learned on the job.
It occurs to me that, although there is often pressure to lower standards, there are some teachers who resist that corruption. I had several professors who were quite principled in teaching and grading – which I admired, even if some other instructor might have given me a higher grade. I suspect there are counter pressures from some segment of the faculty and students (who are interested in learning for it’s own sake) that might, over time, reverse the current trend. It will probably involve changing the way success is measured.
Campbell’s/Goodhart’s law is a special case of more the general problem: Be careful what you measure and which incentives you set.
People here may also be interested in this book (which contains case studies on K-12 education in US and colleges):
Jerry Muller: The tyranny of metrics. Princeton UP, 2018
https://www.amazon.com/Tyranny-Metrics-Jerry-Z-Muller/dp/0691174954
I presume this is a spirited defence of the Imperial/US system of measurement? (Feet/pounds/gallons…)
😉
cr
“Frederick Hess gives a nice example:…”
The professor in the example actually gives a logically weird example. In the example of speeding tickets, the data is the number of tickets issued and simply knowing it does not hurt. The application of the data to evaluate performance is what hurts, but that is not the same thing.
Knowing the data rarely what hurts (assuming it is correct), but how it is used.
So I would tell the professor that based on the examples given I still maintain my answer that data does not hurt.
(I could give theoretical examples where this is not so obvious, like in the case of “self-fulfilling prophecies”, but that is already a very liberal use of the term data.)
This being said, the metrics of performance evaluation are a decades old pain of the corporate world. Obsessing with metrics almost inevitably hurts quality in some ways.
The cops knowing how the data is going to be collected and evaluated is what causes the harm.
But I think that was obvious from the original example. With respect, I think you’re splitting a hair that isn’t there.
cr
The harm is still caused by the cops knowing that they are evaluated by the data (usage), not the cops knowing that the data is there (existence).
And if we say that the question must be understood as inherently referring to the usage of the data too, then the answer should be obvious to most of the students in the example (data can be used to hurt).
I am splitting a hair that is true (I do not think this is a cardinal issue), also this is rather beside the point made by the professor and the article, but the hair is nevertheless there.
Okay, maybe I’m the one splitting hairs…
😉
cr
Most people seem to say it’s gotten easier. And when I ask them when it began to get easier the answer is always ‘after I left’.
Funny that; it never got easier while people were getting their qualifications. It’s only after they got them that standards were dramatically relaxed…
My boyfriend is returning to school in preparation for a career change. Compared to when he was in college 15 years ago, his classes are a joke, the teachers are incompetent, and he’s shocked that the fellow students passed elementary school given their poor grammar and lack of critical thinking. This “everyone gets a trophy” mentality is creating a pervasive sense of entitlement, a poorly skilled workforce, and I suspect it will only continue to get worse across time.
“he’s shocked that the fellow students passed elementary school given their poor grammar”
And that’s why I have to watch ads for products like Grammarly every d**n time I watch a YouTube video.
I’m grateful that my primary school teachers inculcated the importance of good grammar and spelling into me. Of course, that was almost fifty years ago, when the teachers themselves still possessed those skills.
Good point about current teachers’ knowledge of grammer…
Grammar
I’m 30 and my parents made many sacrifices to put their three children through private school. The more people I meet, the more grateful I am that they recognized the shortcomings of public education and tried to give us a leg up, both during the school day and at home. The country (and world) are in desperate need of change, yet every change comes back to education… politics won’t change if people aren’t taught civics, the military-industrial complex will continue until people learn about other possible models, and health will decline for as long as people are ignorant to both biology and capitalism. And we live in a world where, for most, ignorance truly is bliss and the consequences are few.
I reasonably assume that in your private school a student who didn’t sufficiently perform academically was shown the door. The public schools do not have that option.
“the goal of the system is not education”, what’s “education”? The Catholic Church has a very different educational system. Is that education? Does it produce better people?
This does not surprise me. Recently, our state higher education system went to basing subsidy solely on graduation and passage rates, rather than headcount. If you don’t have enough students passing your class, the instructor (me or my colleagues) gets targeted as doing a poor job. We’ve been told to NOT recommend that a student drop a course. The school has put programs in place to help students – which is great, don’t get me wrong, but they don’t ever say “it’s about the money.” ‘Cause it doesn’t really seem about an education, else they’d’ve done these programs earlier.
Nice to have a name for this phenomenon!
Right. Student motivation and self-discipline can’t possibly have anything to do with it.
Amen😬
I wish I got a tiny percentage raise added to my base for every time I’ve raised these very same points in committee and Faculty Senate meetings. And on every occasion, I’ve been looked at as if I was some crazy nutter crank going on about aliens, only to have people tell me I’m right in private after the meeting. This really hits the nail on the head.
Goodhart’s law has hidden depths. A dozen examples spring to mind. Thank you for that!