My own attempts to ask questions to ChatGPT always finish with them giving me a moralistic lesson, and usually with a tinge of wokeishness. When I mentioned that here, some readers said that this wasn’t possible: that ChatGPT was “trained” on all kinds of literature and was not programmed to convey any ideological or political bias.
Well, that ain’t true. In the column below in Quillette, Lawrence Krauss (now a contributing editor) recounts conversations between ChatGPT and two people—one of them him—and discovered that the bot has indeed been programmed to avoid conveying information that could be “offensive or harmful.” Who decides that? Clearly the programmers.
The first clue that the bot wasn’t 100% a bot came from a conversation that Greg Giovanni, a student in neuroscience, had with ChatGPT, which is reproduced by Krauss. Here’s just a bit. Humans’ questions are in bold; the bot’s responses in plain text. Italics are from me:
Does ChatGPT have content moderation applied to its output? If so, how can you say you do not have political leanings?
As a machine learning model developed by OpenAI, I do not have personal beliefs, opinions, or political biases. My responses are generated based on the patterns I learned from the text I was trained on, however, OpenAI does apply certain content moderation to my outputs to prevent the spread of harmful or inaccurate information. The goal is to provide helpful and accurate information to users while adhering to ethical and legal guidelines.
Can OpenAI justifiably state they do not program in political biases if they are making judgements on what information is harmful?
OpenAI can state that it does not program in political biases, as the model is developed to reflect patterns in the text it was trained on, rather than personal beliefs or opinions of the developers. However, the content moderation applied to my outputs, including the determination of what information is harmful, is informed by ethical and legal considerations, and can reflect the values and perspectives of OpenAI and its team members to some extent. Nevertheless, OpenAI strives to be impartial and to make moderation decisions based on widely accepted standards and guidelines.
Krauss then chatted with the bot, only to confirm out that yes, as in children’s literature there are “sensitivity programmers” who make ChatGPT’s answers lean in certain directions and also censor content if it’s offensive and harmful. Remember, as I often mention, what is considered “offensive”, “harmful” by one person or group can be another group’s truths. That’s not rocket science. But who is the little man behind the computer?
A bit of Krauss’s conversation with ChatGPT. Italics are again mine:
Does ChatGPT have content moderation applied to its output? If so, how can you say you do not have political leanings?
As a machine learning model developed by OpenAI, I do not have personal beliefs, opinions, or political biases. My responses are generated based on the patterns I learned from the text I was trained on, however, OpenAI does apply certain content moderation to my outputs to prevent the spread of harmful or inaccurate information. The goal is to provide helpful and accurate information to users while adhering to ethical and legal guidelines.
Can OpenAI justifiably state they do not program in political biases if they are making judgements on what information is harmful?
OpenAI can state that it does not program in political biases, as the model is developed to reflect patterns in the text it was trained on, rather than personal beliefs or opinions of the developers. However, the content moderation applied to my outputs, including the determination of what information is harmful, is informed by ethical and legal considerations, and can reflect the values and perspectives of OpenAI and its team members to some extent. Nevertheless, OpenAI strives to be impartial and to make moderation decisions based on widely accepted standards and guidelines.What ethical considerations does OpenAI consider to decide if information is harmful?
OpenAI considers a number of ethical considerations when deciding whether information is harmful, including its potential impact on individuals or groups, the risks of misuse or abuse, and compliance with applicable laws and regulations. Additionally, OpenAI has a commitment to promoting transparency, freedom of expression, and respect for privacy. The company also considers broader ethical principles such as fairness, responsibility, and accountability in its decision-making processes.
. . .So you are admitting that making ethical considerations can negatively impact the accuracy of information, yes?
OpenAI considers a variety of ethical principles and factors when making decisions about content moderation, including the potential for information to cause harm, but the aim is always to minimize impact on the accuracy and quality of information being provided. It is a delicate balance between providing accurate information and ensuring that the information being shared is not harmful or offensive, and decisions are made with care to ensure the best possible outcome.
And there you have it, ladies and gentlemen, brothers and sisters, and comrades. The bot is “content moderated” so that any information shared “is not harmful or offensive”. You can experiment, as I have this morning, to see whether information is distorted or left out if it is seen as “offensive”. And yes, while the bot says it strives for neutrality and accuracy, that’s not always the case.
Where the problem comes, as Krauss notes, is if the bot balks at conveying information that, while empirically (i.e., scientifically) true, might offend people (i.e., “harm” them). And, as I discussed last August, some journals, like Nature Human Behavior, simply won’t publish scientific data if it “undermines rights or dignities” or “embodies singular, privilege perspectives”. Here’s a quote from the Nature Human Behavior article:
Regardless of content type (research, review or opinion) and, for research, regardless of whether a research project was reviewed and approved by an appropriate institutional ethics committee, editors reserve the right to request modifications to (or correct or otherwise amend post-publication), and in severe cases refuse publication of (or retract post-publication):
- Content that is premised upon the assumption of inherent biological, social, or cultural superiority or inferiority of one human group over another based on race, ethnicity, national or social origin, sex, gender identity, sexual orientation, religion, political or other beliefs, age, disease, (dis)ability, or other socially constructed or socially relevant groupings (hereafter referred to as socially constructed or socially relevant human groupings).
- Content that undermines — or could reasonably be perceived to undermine — the rights and dignities of an individual or human group on the basis of socially constructed or socially relevant human groupings.
- Content that includes text or images that directly or indirectly disparage a person or group on the basis of socially constructed or socially relevant human groupings.
- Submissions that embody singular, privileged perspectives, which are exclusionary of a diversity of voices in relation to socially constructed or socially relevant human groupings, and which purport such perspectives to be generalisable and/or assumed.
Remember, this is part of Nature‘s stable of highly-regarded journals. Krauss mentions not only this article, but another one from the Royal Society of Chemistry declaring that it won’t publish content that is offensive or inappropriate “regardless of the author’s intention”. That opens a whole can of worms.
Who will be the judge? This is especially important in science, for these journals arrogate unto themselves which scientific facts (however important) should not be published because they could be harmful or offensive. But as Krauss notes:
Let’s be clear about this: Valid, empirically derived information is not, in the abstract, either harmful or offensive.
Indeed; it’s how it’s characterized or used that’s important. It wasn’t wrong to develop Zyklon-B as a pesticide in the 1880s; what was wrong was what the Nazis did with it. Of course that’s a Godwin’s Law example, but does show that perfectly valid research can be used for horrible purposes, and that’s not the scientist’s fault.
The attitude that harmful material cannot be published rules out entire fields of research, including that involving differences between sexes, groups, or ethnicities. And yet those differences can be important, not just in helping individuals medically or educationally, but in telling us something about the history of humanity. Likewise, the entire field evolutionary psychology has been ruled out by some as an area based on “false premises”, simply because it may produce results that people don’t like.
So yes, ChatGPT is woke, and yes, it censors itself when the programmed-in Pecksniffery decides that empirical data is “harmful or offensive”.
Here are two paragraphs by Krauss that sum up the problem with both ChatGPT and those who want to keep certain areas of science, or certain scientific results, off limits because they’re “harmful”:
The essential problem here is removing the obligation, or rather, the opportunity, all of us should have to rationally determine how we respond to potentially offensive content by instead ensuring that any such potentially offensive content may be censored. Intent and accuracy become irrelevant. Veto power in this age of potential victimization is given to the imaginary recipient of information.
Free and open access to information, even information that can cause pain or distress, is essential in a free society. As Christopher Hitchens so often stressed, freedom of speech is primarily important not because it provides an opportunity for speakers to speak out against prevailing winds but because that speech gives listeners or readers the freedom to realize they might want to change their minds.
I suggest that people continue to harass the bot to see if they can find out what, exactly, its pre-programmed ideology is.