Flam on Bayes’s Theorem

September 30, 2014 • 11:07 am

Official Website Science Journalist™ Faye Flam, who gave us a nice guest post a while back about how to talk to reporters, has her first article up in today’s Science Times at the New York Times, “The odds, continually updated.” It’s about Bayes’s Theorem, a way to calculate probabilities if you have some prior information about related probabilities, and I have a feeling that a lot of our science readers know about it. But even if you do, you should check out Faye’s piece, as it has some nifty ways the theorem has been used, like narrowing the search for a lost fisherman. (She makes the case, and it’s pretty strong, that the theorem actually saved that fisherman’s life.)

It’s a nice summary, covering the origin of the theorem, what it means, how it’s used and how probabilities calculated from that theorem differ from conventionally calculated probabilities. It’s a lot of information packed into a readable and concise format. Here’s one bit about the Monty Hall problem. (Try telling one of your friends about it and then tell them that they should always switch doors. Odds are they won’t believe you!):

A famously counterintuitive puzzle that lends itself to a Bayesian approach is the Monty Hall problem, in which Mr. Hall, longtime host of the game show “Let’s Make a Deal,” hides a car behind one of three doors and a goat behind each of the other two. The contestant picks Door No. 1, but before opening it, Mr. Hall opens Door No. 2 to reveal a goat. Should the contestant stick with No. 1 or switch to No. 3, or does it matter?

A Bayesian calculation would start with one-third odds that any given door hides the car, then update that knowledge with the new data: Door No. 2 had a goat. The odds that the contestant guessed right — that the car is behind No. 1 — remain one in three. Thus, the odds that she guessed wrong are two in three. And if she guessed wrong, the car must be behind Door No. 3. So she should indeed switch.

Jasen Rosenhouse wrote a whole book about the Monty Hall problem, which you can find at this link.

Anyway, Faye manages to write the whole article without once showing the theorem itself, but nevertheless captures its essence. It’s good popular science writing. And here’s the theorem:

bayes-rule

Addendum: Faye also has a new piece in Forbes which will interest many of us: it’s about why plants make caffeine. The piece is called,”Though it may taste divine, coffee DNA tells a Darwinian tale.” I won’t give you the punchline, but here’s a teaser:

But caffeine appears to be extra beneficial to plants since it evolved through different genetic mutations in coffee, chocolate and tea – a phenomenon called convergent evolution. We know why we like coffee, but what’s in it for these different plants?

120 thoughts on “Flam on Bayes’s Theorem”

  1. Must a good writer, because this is the first time I finally understand the Monty Hall problem, and it only took a couple paragraphs lol.

  2. I’ll head over to read Flam’s column, and I’m curious what she’s saying about the Monty Hall problem. I’ve been involved in online discussions about it since about 1996, before I knew what the web thing was (using Usenet).

    I’ve found that not only do many people not understand it at first, but many people who will tell you that you should always switch don’t even understand it fully.

    If the problem statement tells you that Monty is always forced to offer you the switch, then the “2/3 if you switch” is indeed the correct answer, but most versions I’ve seen in the wild don’t include that. And if you don’t know that, you can’t tell the probability unless you make an assumption about Monty’s motivation. In the extreme case, assume Monty is trying to get you to lose – he will offer the choice of switching only when you’ve picked the right door on your first guess. If you picked wrong at first, then he’ll show you a goat and offer the switch. So making the assumption that Monty is evil, you will lose every time by switching.

    1. Yes, per Wikipedia, to clarify the underlying assumptions about Monty’s behavior:

      1. The host must always open a door that was not originally picked by the contestant.

      2. The host must always open a door to reveal a goat and never the car.

      3. The host must always offer the chance to switch between the originally chosen door and the remaining closed door.

      1. That is why the correct answer to the Monty Hall problem is “it depends”. If Monty only let you switch if you had pick a winning door, then you would never switch. So what if Monty always let you switch if you have a winning door and sometimes when you have a loser? It depends on how often he lets you switch a loser. If it is half the time, you would be indifferent. Less than half, you don’t switch, more than half you would switch.

        It is also wrong to say the choice was between a car and two goats. This applies only to the last deal of the show – the Big Deal. No goats – just prizes of different value. The current incarnation of the show usually has prizes worth $25K, $8K and $3K. What if the door with the $3K prize is opened. And you are told that the Big Deal is a trip. Unfortunately, the trip is valued – and taxed – at the listed prices, not what you could actually buy it for. Airfares are not discounted. You cannot trade or transfer trip prizes. A lot of trip prizes on LMAD and the Price is Right are not claimed. Often, prizes like hot tubs and boats are also not claimed. You have to pay the income tax on your prize before you get it.

        1. You say “it depends”, but that’s hardly the interesting part of the problem here. Dependence on the caveats that you’ve given is not at all interesting.

          The pure problem (once it IS understood correctly) is what’s interesting, because the answer is so counterintuitive.

          1. His point is that you must state the problem correctly.

            The host must know which door contains the car, and he must always offer the switch, and the contestant must know this.

            Otherwise the answer is indeed “depends” – and that is indeed the correct response to the problem as posed by the author of that article.

            Lesson: State the problem correctly.

          2. Yes, fair enough. Given the checkered history of misunderstandings about this thing, the author should have known better. The article that Joe Felsenstein linked to below on the history of this is entertaining.

            To be honest, I don’t think Monty Hall is a great example for an article about Bayesian reasoning. I don’t think Bayes helps much with the failure of intuition in Monty Hall.

          3. I am in a small minority but I don’t think the answer is counterintuitive. Suppose after choosing a door but before a door is opened, you were offered the opportunity to exchange your door for the other two – both of them. You would take that trade. You are still being offered the same thing after one door has been opened. Nothing has changed except you are now certain where one goat is. But you knew that at least one of the two doors had a goat.

          4. Yes, that’s the way I described it in comment 4 – and I agree, once you think of it that way, it’s obvious.

            Perhaps on reflection much of the problem does lie with “deceptive framing”. Given that Monty Hall is certainly not something that is likely to occur in everyday life, perhaps it’s the structure of the problem itself that’s non-intuitive. So that it’s particularly easy for somebody who encounters the problem for the first time simply to misunderstand the rules.

            It’s been so long since I first encountered it, that I actually can’t remember if I still struggled with the probabilities once I fully understood the rules.

          5. There’s another well known problem described here

            http://en.wikipedia.org/wiki/Boy_or_Girl_paradox

            Here, I think the difficulty lies entirely with counterintuitive framing of the problem, rather than counterintuitive probabilities. It is extremely difficult, in everyday life, to imagine a plausible context in which you would be given precisely the information “at least one of the two children is a boy”.

          6. The easiest way to understand the Monty Hall problem goes like this: If you play the game 99 times and never switch doors, then you win 33 times. But that means: If you had switched in every round, you would have won 66 times. So by switching you double your chance to win.

  3. I understand the symbolism &c. “P(A|B)” but can someone step through the process of supplying numbers for the various probabilities, please.

    I presume the first step is P(A), the prior probability that the car is behind a selected door and it equals 1/3. But what is “B” and P(B) and P(B|A)?

    1. I’m happy to be corrected but I think this is the right way to apply it:

      P(A) = probability of you selected right.
      P(B) = probability Monty reveals a goat behind door #2
      P(B|A) = probability Monty reveals a goat behind door #2 given that you selected right.
      P(A|B) = proability you selected right given Monty reveals a goat behind door #2.

      It’s critically important to remember that Monty always picks a goat. He is required to do so. So it turns into:

      P(A) = 0.33
      P(B) = 1
      P(B|A) = 1
      P(A|B) = 1 * 0.33 / 1.

      You can do the same calculation substituting “probability I selected wrong” and you’ll end up with a 2/3 probability that the car is behind door C.

      The real power in using Bayes theorem on this problem is not in the numbers, which are very simple, but in showing why you should switch: it’s because of Rule #2, “Monty always picks a goat.” That’s the rule that screws up people’s intuitive logic. That’s the information most people don’t or can’t mathematically account for when they try and intuit the answer. Because if Monty instead flipped a coin to decide whether to reveal door B or C, then most people’s standard intuition (“I have a 50/50 shot of being right) would be correct. In that case, P(B) = 0.66 (because there are two goats behind three doors), P(B|A) = 1 (if you picked right, he’s 100% likely to randomly pick a goat), and P(A|B) = 0.50

      1. It is a way to apply it. But it is not really meaningful. Since you defined P(B|A) to be one you don’t gain any additional information so you will always end up with the same prior as you started with.

        You can define the random variables in any way you want. Here is a different application that demonstrates how information can be gained through observation.

        A – choosing the door with the car
        B – a car is observed after the door has been opened

        The prior probability of choosing the correct door is 1/3 since no information is given:
        p(A) = 1/3

        Given the fact that the correct door was chosen the probability that we will observe the car is zero since another door than the one containing the car will be opened)
        p(B|A) = 0

        The marginal probability of observing a car is 1/3 since the there is only one car and if one door was opened at random we would see the car with 1/3 probability.

        P(A|B) is the probability of having chosen the door with the car after our observation

        P(A|B) = (0 * 1/3) / (1/3) = 0

        Which is consistent with the intuition. Once we have observed the car we know for certain that we initially chose the wrong door.

        You can play the same game for different meanings of the random variables.

        @Jerry: Sorry for the long comment

        1. Damn it. I just realized that my definition of the variables are not exactly in line with the rules of the game.

          Forget about it…

        2. Since you defined P(B|A) to be one you don’t gain any additional information so you will always end up with the same prior as you started with.

          It’s not defined as 1, it is 1 because of the rule “monty always chooses a goat.” If, for example, you use P(A) as the probability that you picked wrong, and Monty flipped a coin, then P(B|A) under my definitions would be 0.50. What’s creating all the 1.0 probabilities is the fact that Monty always chooses a goat.

          Here is a different application that demonstrates how information can be gained through observation.

          A – choosing the door with the car
          B – a car is observed after the door has been opened

          Since a car is only revealed after the game is over, this does not provide information on which the player can revise their probabilities.

  4. I’m not sure that Bayesian analysis helps much with intuition for Monty Hall. But I found this does:

    You initially choose door #1.
    Monty now tells you: “You can either keep door #1, or you can have the car if it is behind EITHER of the other doors #2 & #3”

    Obviously you would switch, two doors is better than one.

    With a little reflection, I think it’s quite intuitive to realize that this is precisely the choice that Monty offers you. By eliminating one of the two doors #2 and #3, he is effectively allowing you to switch to the “best of” those two doors, i.e. giving you the car if it’s behind EITHER #2 or #3.

      1. The other classic way to gain intuition is to increase the number of doors. Suppose 100 doors, only 1 has the grand prize. You choose #1. He then eliminates 98 of the other 99 doors (showing you goats). It’s intuitive that you switch, because he’s clearly giving you the “best of” all the other 99 doors.

        1. I tried that one, but with three doors there is only one remaining empty door (if you want to keep a choice for the contestant) whereas with a hundred doors you have to open 98 doors. You are going to get the obvious question: “hey, you are only supposed to open one extra door!”

          1. Yes, I agree that it’s not really obvious that the 3-door and 100-door setups are similar. I find the description in comment #4 more satisfying.

    2. This does not justify the idea that you should change the initial choice you made. You are now faced with a *new* situation, and finding out that one of the three doors was in front of a goat does not affect the chance finding the car behind one of the two remaining doors.

      1. I’m not sure what to tell you. We probably don’t want to get into an extended discussion of this – it is incontrovertibly true that by switching you DO win 2/3 of the time. If you don’t believe that, you can easily prove it to yourself by simulation (actually DO it).

        So the result is not in question.

        It’s just a question of what explanation gives good intuition.

        I think my explanation above helps to show that what he is effectively doing by opening one of the two doors is giving you the “best of” the other two doors.

        1. “I think my explanation above helps to show that what he is effectively doing by opening one of the two doors is giving you the “best of” the other two doors.”

          One of the three doors, not two doors. Of course, if he opened the door with the car, but if not, how would it affect the probability of finding the car behind one of the two other doors?

          1. Sorry for being tetchy, but every time this problem comes up, it can expand into thousands of comments from people who have not come across the problem before.

            First, please make some effort to understand the framing of the problem correctly:

            Monty is the host, he knows beforehand what is behind each door.

            You choose (say) door 1.

            According to the rules of the game, Monty MUST now open either door 2 or door 3, and the door that he opens MUST reveal a goat, not the car.

            You now have the opportunity to keep door 1, or to switch from door 1 to the other unopened door.

            —–

            The answer is that if you keep door 1, you win the car 1/3 or the time. If you switch, you win the car 2/3 of the time.

            —–

            One way of to see why it’s asymmetrical, is to realize that under the rules of the game, by choosing door 1, you barred Monty from opening door 1.

    4. It illustrates the vital difference between Bayes theorem, which is a theorem in (effectively) any theory of probability worthy of the name, and Bayesian approaches to statistics or probability, which *use it differently*. One can agree with the usefulness of the former and find the latter wrongheaded. One debate there, for example, is the domain of the probability function- there are some of us, myself included, which suggest that only *events* (the ontological category, not the generic name used sometimes, confusingly) are probable or improbable in the sense of the probability calculus.

  5. A classic example of conditional probability was the OJ Simpson trial. One of the defense attorneys said something like:

    “The probability that a husband murders his wife is only 1 in 100,000!”

    Of course, that’s for a randomly chosen married couple, including all those where the wife is still alive!

    In fact, the probability that a husband murdered his wife, GIVEN THAT WE KNOW SHE HAS BEEN MURDERED BY SOMEONE, is about 1 in 2.

    1. That’s very similar to the crude anti-marijuana propaganda we used to hear that “99% of all heroin addicts started on marijuana” (which implied that marijuana smokers were sure to end up on hard stuff). The reduction ad absurdum is to note that of course, 99% of heroin addicts also started on coffee, too…

  6. There’s also a German book on the Monty Hall problem, called “Das Ziegenproblem” (goat problem).

  7. The Monty Hall problem can also be counter intuitive since it assumes you don’t want the goat.

  9. I first heard of Bayes’ Theorem from Richard Carrier, who has been brought up on this website a few times already, by Jerry Coyne as well, since Carrier argues that there was no historical Jesus.
    He has a few youtube videos on Jesus and Bayes, but this Skepticon Video is probably most apt.

    1. Forgot to mention, Carrier uses a Bayesian analysis to reach the conclusion that there probably was no historical Jesus.

  11. Bayes theorem should be of interest to everyone who wants to be an efficient skeptic. Many skeptics’ popular phrases and heuristics – like extraordinary claims require extraordinary evidence and Occam’s Razor – can be expressed using Bayes Theorem. You can even use it to demonstrate that a claim is unfalsifiable.

  12. So, if I don’t trust Monty Hall to be either for or against me, it doesn’t matter if I switch or not.

    2. He is always for you, which is why you should switch.
      If he randomly picked one of the other doors to open, then you’re right, it wouldn’t matter. But Monty’s choice is not random, it’s always “goat.”

    3. So long as the rules of the game are clear, trusting Monty doesn’t come into it. He MUST always reveal a door with a goat, and he MUST always give you the opportunity to switch. He’s an automaton.

  13. And of course Bayes’ theorem can also be used to prove that God (most likely) does not exist…

    For instance, see:

    Can Science Test Supernatural Worldviews?

    http://link.springer.com/article/10.1007/s11191-007-9108-4

    http://www.naturalism.org/Can%20Science%20Test%20Supernatural%20Worldviews-%20Final%20Author%27s%20Copy%20(Fishman%202007).pdf

    Does Science Presuppose Naturalism (or Anything at all?)

    http://www.academia.edu/3799490/Does_Science_Presuppose_Naturalism_or_Anything_at_All_

    https://sites.google.com/site/maartenboudry/teksten-1

  17. The Monty Hall Problem becomes intuitive when you think about it not as 3 doors, but 100. The player would again pick one door at random. Monty would again reveal all false doors, except one. At this point most people intuitively think it was 50/50 to keep or to switch and with this example probably see that it would be foolish to hold on to one’s choice.

    1. The way it seems to me, by revealing the false doors, Monty is improving the odds that the other door is correct but he is EQUALLY increasing the odds that your already chosen door is correct, so it’s a wash.

      1. Wanna play for money?

        The asymmetry arises from the fact that if you initially choose Door #1, you prohibit Monty from opening Door #1. You do not gain any additional information about Door #1, but you do gain additional information about the other doors.

    2. This is exactly analogous to the final contestant option in the mind-numbingly stupid game, Deal or No Deal, where near the end the contestant has the option of swapping his/her originally-selected briefcase for the one remaining on the floor. Given the odds, and if a large prize is still outstanding, swapping is always the correct thing to do.

  18. Before this grows to the inevitable thousands of comments from people who haven’t come across it before, this is the Monty Hall Problem:

    ————–

    Monty is the host of a TV show, you are the competitor. There are 3 doors, behind two doors are booby prizes (goats) behind one door is the grand prize (a car).

    You pick a door, say door 1.

    Monty now MUST ALWAYS open one of the OTHER two doors to reveal a goat. He MUST open a door that reveals a goat – he knows what’s behind the doors, he does not open at random, he will never reveal the car.

    Monty MUST now offer you the choice to either keep door 1, or to switch to the other unopened door.

    —————

    To be clear – Monty knows what’s behind the doors, he does not pick at random; and his motivations are irrelevant, he is effectively an automaton, he must follow the rules of the game.

      1. No, the competitor’s state of mind is not relevant. The rules of the game fully determine the probabilities.

        Whether a mathematician or a monkey rolls a dice, the probabilities are still the same.

        1. The competitor’s knowledge does not affect the odds, up to the point where he has to decide whether to switch. At that point he needs to know in order to take advantage of the odds.

          2. I think it depends on how you define ‘the odds’. If you’re talking about his chances of actually winning the car, then his knowledge certainly affects the outcome since he has the opportunity to switch or not.

            (But as we’ve seen, how you define the conditions is highly relevant to the answer).

          4. BUT switching doors (or not) certainly does affect your chances of winning.
            Obviously switching doors can’t influence where the car is, BUT you’ve just been given some additional information (i.e. one place it isn’t) which allows you to refine your bet.
            And – if the problem is stated correctly – the odds at the start assume that you WILL refine your bet accordingly.

            If you’re NOT allowed to switch then showing you the goat behind one door would irrelevant as it doesn’t improve your chances. And the odds at the start will be worse than the Monty Hall odds.

          6. Agreed, BUT it changes your chances of choosing the right door.

            Your initially chosen door is 1/3 probability. There’s 2/3 probability that it’s behind one or other of the other 2 doors – and the host has kindly shown you which one it isn’t, so the remaining door is 2/3 probability. Your host has changed the odds in your favour.

          8. I think what we seem to be getting at here is that the host’s strategy/knowledge doesn’t change the odds. If he were to only offer you the switch when you picked the right one initially, it means he only offers you the switch 1/3 of the time. The other 2/3 of the time you picked the wrong door and he didn’t offer you the switch (but that’s not how the scenario is spelled out, it’s stated that the switch is offered).

            Assuming an evil host just means that you know you won when offered a switch and you know you lost when not offered a switch, thus there is no strategy to even consider.

          9. I think the quantum consciousness of the host working at the universal level has an imperceivable effect on what the contestant chooses. Also, the doors disappear if you don’t keep looking at them because quantum!

            Ha ha! I had you there for a minute, didn’t I? 😀

          11. Damn, I should have used that line “because quantum” for the very annoying air duct cleaners that seem to call every week. Will try to remember next time. I have quacked at them and also told them to duck off, but doesn’t seem to work.

          12. I believe you had me for a moment but you didn’t know where you had me. Now, I’m posting, so you’ve found me, but you don’t have me anymore. Your comment was polarizing though, and the odds of all of this were simultaneously 0 and 1 which proves we’ve found a being that is both full potential and full act at once, thus greater than God. God doesn’t exist, because quantum.

            Q.E.D.

          15. infiniteimprobabilit,

            The option to switch doors is stated as one of the rules of the game.

            The problem is usually framed as
            “is it a better strategy to switch?”
            or
            “what are the probabilities that the car is behind each of the two available doors”

            The probability calculation to answer this question depends only upon the rules of the game, the state of mind of the competitor is irrelevant.

            You seem to be contemplating scenarios where the competitor is unwilling to switch because he’s either ignorant of the rules, or not smart enough to figure out the optimum strategy. Of course that may happen in real life, but it’s not relevant to the probability calculation that’s at the heart of the Monty Hall problem.

          16. I think we’re having a problem of terminology. I understand what the odds are and how they work. But I count the competitor’s knowledge as part of the relevant circumstances – if he doesn’t know to switch, (or rather if he is sure to refuse to switch), his chances of winning are halved. He is in a position to change the odds.
            But I think we’re starting to go round in circles.

          17. Oh, and you’re correct – the way you’ve framed the problem, the competitor’s state of mind IS irrelevant.

  19. “Bayesian statistics, in short, can’t save us from bad science.”

    I’m very glad to read a much more measured and accurate piece about the use and power of Bayesian statistics (including its limitations). Too often it seems like people like to sell it as some panacea to scientific and classical problems in statistics, but it is simply another tool that makes the overall field more robust, one that has gotten more useful with the rapid rise in computing power over the past few decades. This article also doesn’t fall into the trap of playing up some supposed war between Bayesians and frequentists (although certainly some controversy did exist decades ago).

    As another reader pointed out above, Andrew Gelman has some good revisions on his website: http://goo.gl/rwojDy. The first revision is an important one.

    A good article overall!

    1. Well, it’s great if you know P(A), P(B), and P(B|A) with a high degree of confidence! But that’s the rub, isn’t it? If we knew those things in most circumstances, we probably wouldn’t need Bayesian statistics.

      AIUI (and this is not my field), the main advantage it has is that normal statistics doesn’t typically use P(B|A) to try and help get P(A|B). But P(B|A) is often much much easier to figure out than P(A|B) is just from P(A) and P(B), so Bayes’ theorem in many circumstances substitutes an easier problem for a harder problem. But if you don’t have good estimates of P(A) or P(B), you’re stuck either way.

      1. It’s more useful than that because you can substitute a range of values for the probabilities to see how the outcome varies. This can be useful for testing your model of some system. Ultimately you’re forced to improve your actual probability values if you require better predictions, but in many cases that’s not a big problem (and of course it’s trivial to come up with situations where improving the values is impossible).

        1. There’s nothing Bayesian about that, frequentists can do that too. So…not an advantage of the technique.

      2. You’re right that the mathematical difference tends to be on which conditional probability we are trying to make inferences. In the classical (frequentist) framework, we generally make inferences around Pr(data | hypothesis), whereas in a Bayesian setup we go directly for Pr(hypothesis | data). In my opinion, the Bayesian approach is usually far more appealing philosophically.

        But that is only a small part of the story. It is always the individual problem that one is tackling that should be the ultimate arbiter as to which method (or combination of methods) is most useful. As somewhat of a general rule, if you have lots of reliable data that was collected in a very controlled way, then frequentist methods are the way to go. Bayesian methods will give you the same results but will usually require more computation and greater obscurity, since most scientists are only really familiar with frequentist methods. But if you have a problem with little and/or very poorly controlled data, then Bayesian methods may be far more desirable (i.e. the assumptions one has to make to do Bayesian modelling in these cases may be much more reasonable).

        Of course, even that general rule is subject to myriad exceptions. As a practising statistician, it is my first priority to make sure the analysis is understood. It does the client no good if I give them a fancy analysis that they are incapable of understanding or using. This means that I usually rely on frequentist methods, even when they may not be completely ideal from the statistician’s perspective.

        1. I agree with most of that, but not –

          “Bayesian methods will give you the same results but will usually require more computation”

          One of the principal reasons that Bayesian statistics is flourishing is because we now have so much computing power, and Bayesian methods are often much better suited to computational approaches. For example, Bayesian MCMC can solve problems that are intractable for Max Likelihood methods.

          1. You’re right, naturally – that was a very lazy thing for me to say! It would have been better to say that Bayesian methods will lead you to the same conclusions in instances where the frequentist framework is satisfied; in these cases, frequentism wins over Bayes usually just because people are more familiar with the methods. That’s a practical assertion though (and an opinion), not a philosophical one.

    1. The simulation is easy really, you can see how it works out without doing it:

      Let’s say you always pick door 1 initially (since, by symmetry, it doesn’t matter). And suppose you follow the correct strategy, “always switch”.

      1/3 of the time the car is behind door 1.
      You choose door 1. Monty reveals either door 2 or door 3 (both contain goats), you switch to the other one and lose.

      1/3 of the time the car is behind door 2.
      You choose door 1. Monty reveals the goat behind door 3. You switch to door 2 and win.

      1/3 of the time the car is behind door 3.
      You choose door 1. Monty reveals the goat behind door 2. You switch to door 3 and win.

      So “always switch” wins 2/3 of the time.

    2. I recommend you repeat your experiment! That distribution is a bit atypical, but not extremely unlikely (a quick calculation shows such a distribution is likely to be observed about 2% of the time). If you see that same distribution (or something more extreme) again, then maybe you could start to question how well the game is programmed on that site.

      (FYI, I ran a replication myself on that site and got the following distribution: Door 1, 8 cars; Door 2, 5 cars; Door 3, 7 cars.)

  22. I am intrigued by Richard Carrier’s use of Bayes Theorem to address the question of whether there was or was not an historical Jesus. I am looking forward to reading his rather lengthy book, On the Historicity of Jesus: Why We Might Have Reason for Doubt, in which he applies Bayes Theorem to this question.

  24. “Thus, the odds that she guessed wrong are two in three. And if she guessed wrong, the car must be behind Door No. 3. So she should indeed switch.”

    Don’t understand the conclusion. “MUST be”?Just how much more likely (and by how much) is it that it is behind No. 3 than No. 1? But, have the hard copy beckoning me so will look at it in a couple of hours.

    1. Flam is just saying that in this iteration of the game, Door 2 has been eliminated (Monty opened it to show a goat). So IF the car is not behind Door 1, then it MUST be behind Door 3. So in this iteration, if Door 1 has a 2/3 chance of being wrong, then Door 3 has a 2/3 chance of being right.

    2. Here’s another way to look at the situation.

      The asymmetry between your initial pick (Door 1) and the last remaining door lies in the fact that your initial pick of Door 1 prohibited Monty from opening it.

      When you make the initial choice of Door 1, what you’re really doing is partitioning the doors as follows:

      [ Door 1 ] – you WILL NOT get any additional information about this door

      [ Door 2, Door 3 ] – you WILL get additional information about these doors

      In picking door 1, you prohibit Monty from opening it. So door that Monty subsequently chooses to open tells you nothing directly about Door 1. It’s also impossible to INFER anything about Door 1 from the door that Monty opens, because it’s always possible for Monty to find at least one goat elsewhere to show you.

      Therefore, the probability that Door 1 contains the car is 1/3 at the beginning, and still 1/3 even AFTER Monty opens one of the other doors.

  26. From Flam’s article: “One downside of Bayesian statistics is that it requires prior information — and often scientists need to start with a guess or estimate.”

    From the little I know of Bayesian stats, one can use a so-called “uninformative prior probability” and so need not rely on making an unfounded guess or estimate to start. I welcome any explanation by those who know better.

    1. I’m not an expert by any means, but because Bayesean analysis is iterative and brings in real-world results once the process has started it will eventually self-correct.

      1. Obviously that’s an advantage of Bayesian stats. My point is that before you have information, you need not start with a guess, as Flam’s article says, you can use uninformative prior probability distributions, so it would effectively be the equivalent of standard frequentist statistics, if I understand correctly.

    2. At least in Bayesian phylogenetics–which is the only application of Bayesian statistics with which I have any familiarity–the researcher will not necessarily have reliable prior information about the parameters in question, but he or she can input a reasonable distribution of values. Values will then be sampled from this distribution during the Markov chain Monte Carlo analysis, and those values that increase the posterior probability of the result will (usually, depending on the setup of the analysis) be provisionally accepted until a better parameter estimate is sampled.

      In the end, you get a distribution of posterior probabilities for your tree topology (and other parameter estimates) instead of a single point estimate like you would get in a Maximum Likelihood (i.e., frequentist) approach, which IMHO is useful because it better reflects the uncertainty of your hypothesis.

      That is my understanding, at least. I am by no means an expert at this stuff.

    3. The use of an uninformative prior is fine if you have enough data. As Chris said, Bayesian analysis is formally iterative (although science performed using classical statistical methods should be iterative as well – it just happens that sometimes that part is overlooked), so under certain assumptions the process will approach the “truth.” But it’s the process that requires data, maybe lots of it. An uninformative prior does not save you from having to make a poor initial guess. In a way, it replaces making more formal assumptions (as in a classical/frequentist framework) with greatly increasing the uncertainty in your analysis. That increase in uncertainty can only be offset if you have enough data (i.e. enough iterations of the process). Unfortunately, it’s getting more of the data that is often the hard part – or even perhaps impossible.

  27. A puzzle about the M.H. problem: a contestant gets to play the game only ONCE – so why is it relevant to ponder what would happen when it’s played multiple times? Sure, the calculations et al make it clear that if one plays it even as few as 5-10 times, never mind many times, you’ll come out ahead if you switch. But why does that apply at all if one plays it only once? Why is the assumed frequency of similar events at all relevant to a unique event?

    1. Could you clarify your question? Are you questioning some specific detail of Monty Hall, like whether we are really sure that Monty is playing fair? Or are you raising a more general philosophical question about whether probability theory is ever relevant to an isolated event?

    2. That was already plainly stated: if you keep your original guess, your chances are 1/3, but if you change then your chances are 1/2. You can verify that with the website people have been playing with, but some patience is required because you’ve got to do a *lot* of trials to show the trend clearly. Now some people repeatedly chose a single door, but you will get the same results if you choose a random door.

    3. Well of course probability is relevant to an isolated event. If I’m picking just one marble out of a bag with two-thirds red, one-third blue then I’d be silly to bet on blue, wouldn’t I?

      (I could establish the ratio of marbles either by opening the bag and counting them, or by taking out a marble and replacing it a hundred times and noting which colours I picked – either of those would give the probability and it would be relevant to the unique event when I did it ‘for real’).

    4. Well in the frequentist approach, what happens if you play many times is (I believe) equivalent to the odds of it happening if you only play once. Saying “66 of 100 times, you’re better off switching” is the same as saying “if you only play once, switching gives you a 66% chance of winning the car.”

  28. Jason Rosenhouse (of Evolution Blog fame) has a book on the Monty Hall Problem and its variations, entitled (unsurprisingly) “The Monty Hall Problem: The Remarkable Story of Math’s Most Contentious Brain Teaser”. It’s a good read on probability theory, including a lot on Bayesian statistics. It discusses all sorts of Monty Hall variations, including where Monty opens the door without knowing what is behind each door – are you better off switching?

    1. Apologies to all – I missed Prof. CC’s reference to Jason’s book in the original post.

    1. I can recommend a theology class instead. To score well in the test, just remember that the probability is always 1.

  30. It’s been a long time since I’ve read a paper employing statistics, but I really enjoyed the articles where the authors used the (inverse) Bayes Theorem to pick out flawed papers (some of which were honest mistakes and a few of which were outright fraud). I even have vague memories of one paper suggesting that the technique be used to check the lab reports of people studying electronics, physics, and chemistry. Beware: the statisticians are watching!

  31. This was auspicious. I just posted the following comment on Bayes probabilities elsewhere, in response to “frequentist vs bayesian” probability:

    Bayesian inferences are only promoted to probabilities that all agree on if the bets can be tested (HMM modeled).

    Physics and biology use frequentist and bayesian probability methods of course, but mainly they use testing, so likelihoods:

    “To avoid the introduction of prior probabilities, physicists are usually satisfied with
    the information contained in the likelihood function. In most cases the MLE and
    the likelihood ratio error interval are sufficient to summarize the result. Contrary
    to the frequentist confidence interval this concept is compatible with the maximum
    likelihood point estimation as well as with the likelihood ratio comparison of discrete
    hypotheses and allows to combine results in a consistent way.”

    [ http://www-library.desy.de/preparch/books/vstatmp_engl.pdf ]

  32. Bayes’s theorem is one of those ideas that’s so powerful once you grasp it. It’s good to see good communication of the idea.

  33. Another way to view this problem is the following:

    Your initial pick has a 1/3 chance of being right. When the incorrect pick is revealed and you are given the choice to switch, you can, 1) Do nothing, thus retaining your 1/3 chance of being correct; 2) Switch doors. On this pick, your odds are 1/2 (2 remaining doors, 1 is right). The odds you chose incorrectly on both picks is 2/3 * 1/2 = 1/3. Thus, the odds you are correct when you switch is 1 – 1/3 = 2/3.

  34. Jason Rosenhouse, who wrote the book The Monty Hall Problem (and who I would nominate to be the website’s official mathematician, should Jerry find the need for one), has just put up an article criticizing Flam’s article at his blog

Comments are closed.