Probability is a numerical measure of likelihood. If an event has a probability equal to 1 (or 100%), then it is certain to occur. If it has a probability equal to 0, then it will definitely not occur. And if it has a probability equal to 1/2 (or 50%), then it is as likely as not to occur.

You will know that tossing a fair coin has probability 1/2 to yield heads, and that casting a fair die has probability 1/6 to yield a 1. How do we know this?

There is a principle known as the principle of indifference, which states: if there are n mutually exclusive and jointly exhaustive possibilities, and if, as far as we know, there are no differences between the n possibilities apart from their names (such as "heads" or "tails"), then each possibility should be assigned a probability equal to 1/n.

Mutually exclusive: only one possibility can be realized in a single trial. Jointly exhaustive: at least one possibility is realized in a single trial. Mutually exclusive and jointly exhaustive: exactly ony possibility is realized in a single trial.

Since this principle appeals to what we know, it concerns epistemic probabilities (a.k.a. subjective probabilities) or degrees of belief. If you are certain of the truth of a proposition, then you assign to it a probability equal to 1. If you are certain that a proposition is false, then you assign to it a probability equal to 0. And if you have no information that makes you believe that the truth of a proposition is more likely (or less likely) than its falsity, then you assign to it probability 1/2. Subjective probabilities are therefore also known as ignorance probabilities: if you are ignorant of any differences between the possibilities, you assign to them equal probabilities.

If we assign probability 1 to a proposition because we believe that it is true, we assign a subjective probability, and if we assign probability 1 to an event because it is certain that it will occur, we assign an objective probability. Until the advent of quantum mechanics, the only objective probabilities known were relative frequencies.

The advantage of the frequentist definition of probability is that it allows us to measure probabilities, at least approximately. The trouble with it is that it refers to ensembles. You can't measure the probability of heads by tossing a single coin. You get better and better approximations to the probability of heads by tossing a larger and larger number N of coins and dividing the number N_H of heads by N. The exact probability of heads is obtained in the theoretical limit in which N becomes infinite.

The probability that m events from a mutually exclusive and jointly exhaustive set of n possible events happen is the sum of the probabilities of the m events. Suppose, for example, you win if you cast either a 1 or a 6. The probability of winning is

p(1 or 6) = p(1) + p(6) = (1/6) + (1/6) = 1/3.

In frequentist terms this is virtually self-evident. N(1)/N approximates p(1), N(6)/N approximates p(6), and [N(1)+N(6)]/N approximates p(1 or 6).

The probability that two independent events happen is the product of the probabilities of the individual events. Suppose, for example, you cast two dice and you win if the total is 12. Then

p(6 and 6) = p(6) x p(6) = (1/6) x (1/6) = 1/36.

By the principle of indifference, there are now 6 x 6 = 36 equiprobable possibilities, and casting a total of 12 with two dice is one of them.

The general rule for the joint probability of two events is

p(A,B) = p(B|A) p(A) = p(A|B) p(B).

p(B|A) is a conditional probability: the probability of B given that A.

Let's see why. If N(A,B) is the number of trials in which both A and B happen, then N(A,B)/N approximates p(A,B), N(A,B)/N(A) approximates p(B|A), and N(A)/N approximates p(A). Evidently

N(A,B)/N = [N(A,B)/N(A)] x [N(A)/N]

In the limit N → ∞ this becomes p(A,B) = p(B|A) p(A). An immediate consequence of this is Bayes' theorem:

p(B|A) = p(A,B)/p(A).

The following is just as readily established:

p(X) = p(X|Y) p(Y) + p(X|¥) p(¥),

where ¥ is what happens whenever Y does not happen.