Math Formula of the Week

Bayes' theorem

Simply put, Bayes’ theorem gives the probability of a random event A occurring given that we know a related event B occurred. This probability is noted P(AB), and is read "probability of A given B". This measure is sometimes called the "posterior", since it is computed after all other information on A and B is known.

According to Bayes’ theorem, the probability of A occurring given B will be dependent on three things:The probability of A occurring on its own, regardless of B. This is noted P(A) and read "probability of A". This measure is sometimes called the "prior", meaning it precedes any other information – as opposed to the posterior, defined above, which is computed after all other information is known.

The probability of B occurring on its own, regardless of A. This is noted P(B) and read "probability of B". This measure is sometimes called the normalising constant, since it will always be the same, regardless of which event A one is studying.

The probability of B occurring given that A occurred. This is noted P(BA) and is read "probability of B given A". This measure is sometimes called the likelihood, since it is the likelihood of A occurring given that B occurred. It is important not to confuse the likelihood of A given B and the probability of A given B. Even though both notions may seem similar and are related, they are quite different.

Example

To illustrate, suppose there are two bowls full of cookies. Bowl #1 has 10 chocolate chip cookies and 30 plain cookies, while bowl #2 has 20 of each. Our friend Fred picks a bowl at random, and then picks a cookie at random. We may assume there is no reason to believe Fred treats one bowl differently from another, likewise for the cookies. The cookie turns out to be a plain one. How probable is it that Fred picked it out of bowl #1?

Intuitively, it seems clear that the answer should be more than a half, since there are more plain cookies in bowl #1. The precise answer is given by Bayes' theorem. But first, we can clarify the situation by rephrasing the question to "what’s the probability that Fred picked bowl #1, given that he has a plain cookie?” Thus, to relate to our previous explanation, the event A is that Fred picked bowl #1, and the event B is that Fred picked a plain cookie. To compute Pr(AB), we first need to know:

Pr(A), or the probability that Fred picked bowl #1 regardless of any other information. Since Fred is treating both bowls equally, it is 0.5.

P(B), or the probability of getting a plain cookie regardless of any information on the bowls. In other words, this is the probability of getting a plain cookie from each of the bowls. It is computed as the sum of the probability of getting a plain cookie from a bowl multiplied by the probability of selecting this bowl. We know from the problem statement that the probability of getting a plain cookie from bowl #1 is 0.75, and the probability of getting one from bowl #2 is 0.5, and since Fred is treating both bowls equally the probability of selecting any one of them is 0.5. Thus, the probability of getting a plain cookie overall is 0.75×0.5 + 0.5×0.5 = 0.625.

Pr(BA), or the probability of getting a plain cookie given that Fred has selected bowl #1. From the problem statement, we know this is 0.75, since 30 out of 40 cookies in bowl #1 are plain. Given all this information, we can compute the probability of Fred having selected bowl #1 given that he got a plain cookie, as such.

As we expected, it is more than half. ((.75 * .5) / .625)