Are your letters of recommendation gender-biased?

September 15, 2021 • 10:45 am

It was pointed out to me that Lehigh University in Pennsylvania has a website from 2016 that discusses the content of letters of recommendation written by academics. Part of our job is to write recommendations for our students or technicians—letters to go to graduate school, to medical school, for industry, for jobs as technicians, and so on. These are quite hard to write, especially if the applicant isn’t a star but is decent.

My policy has been that if a student is hopeless, I tell them that I simply cannot write a letter (without saying, “because I don’t want to ruin your career if others feel differently”). For borderline students who have both virtues and problems, I will agree to write a letter, and try to be as honest as possible. For uniformly excellent people who I really want to get the position, I’m famous for my long letters of recommendation that go into great detail about the person’s accomplishments, figuring that the length demonstrates how well I know the person. (It’s not unusual for such a letter to run six single-spaced pages, and of course nobody reads them in their entirety because there are always many applicants.) I think many faculty have a policy like mine.

Until now, I never worried about the specific words I used in my letters, but then I saw this website (click on it):

It says, and there’s research to show this, that some adjectives are associated with letters written females, and others for males. As you’ll see, adjectives about “competence” or “diligence” are female-associated words, while indications of “excellence” or “intelligence” are associated with letters for male applicants. This much we know. The Lehigh site says this:

Have you wondered if the letter you are READING- or the letters you are WRITING – are inadvertently perpetuating implicit biases that could reduce the likelihood of the candidate getting a fair chance at the new opportunity?  This one pager summarizes some facts and ideas about letters of recommendation. You can also put your own letters through this online gender bias calculator.The calculator was inspired by presentations on research organized by AWIS; several articles and blog posts share personal reactions to learning of this phenomenon as well as the tool.

It’s worth taking this into account, with three caveats. But it does helps to know what words are perceived in what way by recipients, so the “one pager” is useful to read.

The problems are these. First, given that academia is now preferentially looking for female applicants (this will change as the proportion of male students in college keeps shrinking), a letter with female-biased words may not “perpetuate implicit bias”.  More important, suppose you run one of your letters thorough the linked “gender bias calculator”,  which you can find at the link above or by clicking on the screenshot below. It was made by Tom Force:

As a test, I ran through it a long letter I wrote a while back for one of my female undergraduate research assistants, who wanted to go to medical school. I got this result:

Here you can see the kind of words associated with female letters of recommendation (left column), emphasizing diligence and reliability. On the right are the words associated with letters for male applicants, and they’re about smarts and curiosity and high ranking. This bespeaks sexism, as far as I’m concerned. But I was happy to see that in this letter, for a women, I had roughly equal numbers in each column, with slightly more of the male words. (They don’t say whether letters with more female-associated words reduce the applicant’s chance of getting the position.)

And here’s from a letter in which I recommended a female technician for medical school (again an acceptance); most of the words are male-associated.

Again, what is the recommendation? Is this what you want for a letter for a female, or should I have added some more words about diligence which, after all, is an important characteristic for a future doctor? I don’t know, but she’s now an excellent doctor as well.

The second caveat is this: What are you supposed to do if the letter is imbalanced? For example, in the above letter, should I have cut out some of the female-associated words? (It turns out that the undergrad did get into medical school and is now a fine oncologist.) Are you supposed to ensure that the male-associated words are the predominant ones for either sex?  They don’t tell you.

That leads to the third issue: what if you’re writing for a male whose prime virtues are diligence, reliability, and responsibility. There are some science jobs, like a technician, where some of the most important qualities are showing up, following orders properly, and doing the job diligently and well. Initiative is desirable, too, but that’s a bonus. In fact, one of my colleagues wrote a letter for a guy applying for a technician position, ran the letter through the calculator, and found out that it was imbalanced in favor of female-associated words. (This was after the fact, just like my letter.) What was my colleagues supposed to do: insert more “male associated” words? At any rate, this guy got the job, turned out to be a great technician, and improved in the “male associated” traits.

The main issue is this: what are we supposed to do about “balance” in such a letter? Is imbalance bad? Are “male-associated” words good? That’s the implication. But here we have a woman applicant with male-associated words.

I realize that these lists are based on data, and that one has to be cognizant of how adjectives are perceived with respect to sex. But I think there are some problems with this method that weren’t explicated.

If you write letters of recommendation yourself, you may try running one of them through the calculator, and letting us know how the result came out (the letters of course should not be shown).

32 thoughts on “Are your letters of recommendation gender-biased?

  1. Let’s be controversial: why would we expect that — on average (and the “on average” is important here), male and female “styles” of learning and working and researching would be the same? Many school teachers attest that, on average, boys and girls tend to have rather different styles of learning. We can call these “stereotypes” but many stereotypes arise because they have truth behind them.

    Now, if there were a systematic, on-average difference, then the fact that certain words are somewhat more likely to be used for women and other words for men, could reflect actual reality rather than being a “bias”.

    1. The problem, I think, is that stereotypes tend to be absolutist (she’s a woman, therefore her style will be diligent) when the stats probably aren’t (there’s a small but statistical difference based on gender).

      1. Agreed, we should not act on stereotypes, we should treat people on their individual merits. But if analysing stats shows a difference in line with a stereotype then that is not necessarily a problem.

    1. That’s what I came down here to ask too. I can’t imagine that it’s not important.

      As an aside, a lot of institutions/systems now seem to ask you to cap letters at two pages because nobody (least of all me) wants to read a six pager.

      1. It’s hard to say how important the gender of the recommender is….but it is worth considering. I have noticed my own biased thinking that is probably in line with my male colleagues simply because we are sharing the same culture (spoiler alert: I was shocked at my internal reaction to a female in a certain role and realized its because I never saw females in that role. I also, at that same moment, realized that others most likely see me differently than I see myself vis a vis gender roles). But there may be other biases each of us brings toward our own gender and toward other genders.

      2. I always limited my letters of recommendation to one page. (So many of the word examples I ‘ve never used).

  2. I’m thinking the letters of recommendation process is kind of old school, meaning it has been around for a long time. I’m not all that sure it still is but do not know what may have replaced it. It got me remembering what system was once used within our company to assist in the promotion process. It was called the PER or Personal Evaluation Report and it was an annual requirement for every employee. It was borrowed from the military who had used the system for years. It was basically your boss writing the review on you and it went into the HR or personnel file. The way ours worked at the time everyone of your grade would be in the pile and ranked for promotion time. Again, very much a copy of the military system. It was a very bad (subjective) system. Almost everyone hated it. The system changed a little every few years but it was still a very poor system. Whether you ever got promoted depended mostly on your boss and how he wrote PERs. Some bosses were so poor at writing PERs they often let the employ write his own. It could be a real clown show. Finally, a few years before I retired the PER system was thrown out and the new replacement was a goals accomplished or based system. It took a few years to be fully defined but eventually worked out pretty well and was much better than the old system.

    1. Annually? That’s Paradise. I have to enter a trimestrial report. Yes , I can get worked up about this. Don’t these ffing burocrats have no idea we actually have real work to do? A ffing nuisance, if anything. And since bonuses depend on it it is even worse, a ffing plague, it borders on harassment.

      1. Please note what I refer to here is an old system. One used for many years but prior to the 90s. The only thing that system did concerned promotion, not annual pay raises and step increases that were in play at that time. Now the new system used at this company, the one wrapped around meeting stated goals and all of that is involved with promotion and pay increases. It determines any and all increase in pay you may get. If your review is poor, you get nothing. There is no more time in grade increases in pay like the old days. They call the new system Pay for Performance now.

  3. interesting

    I checked the most recent rec letters in support of job applications that I wrote for 5 of my female and 5 of my male PhD students. Male-associated words dominate greatly in all letters, regardless of the applicant’s gender. Eyeballing the data, the magnitude of that preponderance does not seem to differ between the 5 females and 5 males. Letters written for jobs that involved both teaching and research seem to have a slightly reduced preponderance of male-associate words (because teaching-associated words seem to be scored as female-associated by that website). This could mean that letters written in support of teaching jobs will be scored by that website as more female-associated, letters written in support of research jobs will be scored as more male-associated, and that this will be true for the two kinds of letters written for the same applicant. Not sure such implicit bias is that meaningful for those kinds of letters.

    I hope that that website did not store all the writing that I just had checked, because there is a lot of personal information in these letters. Better be careful before entrusting any writing to that website.

    1. I’ll bet there is a huge difference anyway between individuals & the choice of those words. We all use language and words at different frequencies- if we can ID a Shakespeare section in another play from the use of vocabulary, so just because you use ‘intelligence’ instead of ‘diligence’ (or work!) it may be down to ones reading & background rather than being male or female?

  4. I would be curious, too, to see whether there is a difference between disciplines. Do letters from the humanities have more of the female associated words, while stem recommendations have more male associated words?

  5. The problem is much deeper than just letters of recommendation: dread gender bias afflicts the usage of the English language in every situation. Think of the relative use of such words as “beautiful” and
    “handsome”. And it is even worse in other languages. In the Romance languages, all nouns are divided into masculine and feminine categories, generally with different endings. And there is no provision by which a word can change its ending, demonstrating that transphobia is embedded in the language. Only
    one solution to this burning problem is possible; cancel language.

  6. Commendation is due your diligence. I don’t think I ever wrote a health science committee letter reaching six pages, let alone an individual recommendation. Your students have been well served by you.

  7. Unless I missed a link, I couldn’t see the methodology behind the assertion that some words are predominantly used to describe male applicants rather than female ones and vice versa? Intelligent v diligent – really?!

    1. I know. And honestly, if it were an either/or situation, I think I’d usually pick diligent over intelligent. It that a gender bias, or just a bias for one personal characteristic over another.

      1. I guess the diligent/intelligent balance would depend on the role – though I’m likely to be described as neither anyway…

  8. As our host pointed out, much depends for what position the recommendee is applying:
    for a technician, a registrar post or for head of department? I also (like our host, I mean) note that for a position of technician, the ‘female’ words are used disproprtionally, for a registrar some more ‘male’ words come in, and for a head of department ‘male’ words may even be more than half. (the ‘higher’ the post, the more ‘male’ words? Sexism after all?)
    I see there is no words directly relating to managerial skills, interpersonal relations, teamwork or organizational talent. Skills I generally value highly, but I wouldn’t know if they are particularly ‘male’ or ‘female’.

  9. I was external reviewer for two tenure packages at about the same time this year. Both were glowing recommendations in favor. For the male applicant, my letter included 24 male-associated words and 20 female-associated words; for the female applicant, the ratio was 41 male-associated words and 18 female-associated words.

  10. I just checked a letter I wrote for a female grad student yesterday for a job application. I used 47 female-associated words and 61 male-associated words. I wonder how big a problem this apparent bias really is.

  11. I do not recall whether PCC has ever discussed this book on WEIT. It is quite possible as he is so well read, and frequently shares his literary discoveries. In any case, for everyone like myself who is called upon to write too many letters of recommendation for their own good, I warmly recommend this book, one of the funniest novels I have read in a very long time: Dear Committee Members, by Julie Schumacher (https://www.penguinrandomhouse.com/books/232129/dear-committee-members-by-julie-schumacher/).

    From the Penguin Random House website: “Jason Fitger is a beleaguered professor of creative writing and literature at Payne University, a small and not very distinguished liberal arts college in the midwest … his life is a tale of woe, and the vehicle this droll and inventive novel uses to tell that tale is a series of hilarious letters of recommendation that Fitger is endlessly called upon by his students and colleagues to produce, each one of which is a small masterpiece of high dudgeon, low spirits, and passive-aggressive strategies.”

    1. I didn’t read this but, based on reviews of it, did read The Shakespeare Requirement. Entertaining and clever, but a tad overwrought. As the author no doubt intended.

  12. I checked a couple that I did, both were women, and there was something of a slant towards female words but when I had a look part of that effect seems to be due to the fact that I was emphasising the person’s teaching (which in context is what the reference required) but also because I was using the wrong words around research as the person concerned had been doing various joint projects with me and while that was quite clearly stressed it seems I didn’t use enough male words as part of that. This may also have been influenced by the fact that I knew the recipients would have a full CV in front of them. The good part is that the person got the job and is now going like a rocket than her academic career.

Leave a Reply