What is Bayesian Inference

Bayes' theorem

Bayes' theorem is one of the most important theorems of probability. It says that there is a relationship between the conditional probability of two events P (A | B) and the inverse form P (B | A).


For two events A and B, for B ≠ 0, Bayes’s theorem reads:

  • P (A | B) is the (conditional) probability of event A under the condition that B has occurred
  • P (B | A) is the (conditional) probability of event B under the condition that A has occurred
  • P (A) is the probability (initial probability) that event A will occur
  • P (B) is the probability (initial probability) that event B will occur

Initial probability means that one event is viewed independently of another.

example 1

An example from the August 5, 2011 issue of the New York Times (freely quoted):

Assume that you are given three coins: two of them are fair (50:50 according to Laplace) and one is rigged. This always lands head up. You choose one of the three coins at random, the probability that it is the manipulated one is 1/3. This is the previous probability of the hypothesis that it is the manipulated coin. Now we choose a coin at random and toss it three times. We find that the coin has turned upside down every time. With these new findings, we now want to know whether the previous probability of whether it is a manipulated coin is still 1/3 is. The answer to this question can be answered with Bayes' theorem: the probability that the coin is the manipulated one is now of 1/3 on 4/5 gone up.

Example 2

A drug test has a specificity of 99% and a sensitivity of 98.5%. This means that the results of the test will be 99% correct for drug addicts and 98% correct for non-drug addicts. If we know that 0.5% of the people tested have used the drug, what is the probability that a randomly selected person who tested positive actually used the drug?

The following results from Bayes’s theorem:

('+' indicates that the test was positive, '-' that it was negative)

Despite the seemingly very accurate accuracy of the test, someone who tests positive is more likely to take the drug Not has consumed (≈ 75%).


This surprising result comes about because the number of non-drug addicts is very large in relation to drug addicts. The probability of a false positive result (person is not drug addict, but test is positive) is 1.4925% greater than the probability of a correct result (person is addicted to drugs and test is positive) (0.495%). To illustrate with numbers, if 1000 people were tested, we would statistically expect 5 drug addicts and 995 non-drug addicts. We expect about 15 of the 995 (995 1.5% = 14.925 ≈ 15) to test positive (false positive test result). From the 5 drug addicts we expect that all (5 · 99% = 4.95 ≈ 5) will test positive. Out of a total of 20 who tested positive, however, only 5 are actually drug addicts, therefore just under 25%.


Bayes' theorem can be derived from the formula for conditional probability:

Bayes' theorem vivid and interactive