**De Finetti's theorem** is named in honor of

Bruno de Finetti. One of the differences between

Bayesian and

frequentist methods in

statistical inference is that frequentists often treat observations as

independent that Bayesians treat as

*exchangeable*. The concept of exchangeability, explicated below, was introduced by de Finetti. De Finetti's theorem explains the mathematical relationship between independence and exchangeability. The too-short version says that exchangeable observations are

*conditionally* independent given some (usually) unobservable quantity to which an

epistemic probability distribution would then be assigned. A Bayesian statistician will often seek the conditional probability distribution of that unobservable quantity given the observable data.

A random variable *X* has a "Bernoulli distribution" if P(*X* = 0 or *X* = 1) = 1. An infinite sequence

- <math>X_1, X_2, X_3, \dots</math>

of such random variables is said to be "exchangeable" if for any finite cardinal number

*n* and any two finite sequences

*i*_{1}, ...,

*i*_{n} and

*j*_{1}, ...,

*j*_{n}, the two sequences

- <math>X_{i_1},\dots,X_{i_n}</math>

and

- <math>X_{j_1},\dots,X_{j_n}</math>

both have the same probability distribution.
The condition of exchangeability is stronger than the assumption of identical distribution of the individual random variables in the sequence, and weaker than the assumption that they are

independent and identically distributed.

**De Finetti's theorem** says that the probability distribution of any infinite exchangeable sequence of Bernoulli random variables is a "mixture" of the probability distributions of independent and identically distributed sequences of Bernoulli random variables. "Mixture", in this sense, means a weighted average, but this need not mean a finite or countably infinite (i.e., discrete) weighted average; it can be an integral rather than a sum.

Here is a concrete example. Suppose *p* = 2/3 with probability 1/2 and *p* = 9/10 with probability 1/2. Suppose the conditional distribution of the sequence

- <math>X_1, X_2, X_3, \dots</math>

given the event that

*p* = 2/3, is described by saying that they are independent and indentically distributed and

*X*_{1} = 1 with probability 2/3 and

*X*_{1} = 0 with probability 1 - (2/3). Further, the conditional distribution of the same sequence given the event that

*p* = 9/10, is described by saying that they are independent and identically distributed and

*X*_{1} = 1 with probability 9/10 and

*X*_{1} = 0 with probability 1 - (9/10). The independence asserted here is

*conditional* independence, i.e., the Bernoulli random variables in the sequence are conditionally independent given the event that

*p* = 2/3, and are conditionally independent given the event that

*p* = 9/10. But they are not unconditionally independent; they are positively

correlated.
In view of the

strong law of large numbers, we can say that

- <math>\lim_{n\rightarrow\infty}(X_1+\cdots+X_n)/n = \left\{\begin{matrix}

2/3 & \mbox{with probability }1/2 \\
9/10 & \mbox{with probability }1/2
\end{matrix}\right\}.</math>
Rather than concentrating probability 1/2 at each of two points between 0 and 1, the "mixing distribution" can be any

probability distribution supported on the interval from 0 to 1; which one it is depends on the joint distribution of the infinite sequence of Bernoulli random variables.

Another way of stating the conclusion of de Finetti's theorem is that the Bernoulli random variables are conditionally independent given the tail sigma-field.

The conclusion of the first version of the theorem above makes sense if the sequence of exchangeable Bernoulli random variables is finite, but the theorem is not generally true in that case. It is true if the sequence can be extended to an exchangeable sequence that is infinitely long. The very simplest example of an exchangeable sequence of Bernoulli random variables that cannot be so extended is the one in which *X*_{1} = 1 - *X*_{2} and *X*_{1} is either 0 or 1, each with probability 1/2. This sequence is exchangeable, but cannot be extended to an exchangeable sequence of length 3, let alone an infinitely long one.

All Wikipedia text
is available under the
terms of the GNU Free Documentation License