Law of large numbers

In probability theory, the weak law of large numbers states that if X₁, X₂, X₃, ... is an infinite sequence of random variables, all of which have the same expected value μ and the same finite variance σ², and they are uncorrelated (i.e., the correlation between any two of them is zero), then the sample average

<math>\overline{X}_n=(X_1+\cdots+X_n)/n</math>

<math>\lim_{n\rightarrow\infty}P\left(\left|\overline{X}_n-\mu\right|<\varepsilon\right)=1.</math>

A consequence of the weak law of large numbers is the Asymptotic Equipartition Property.

The strong law of large numbers states that if X₁, X₂, X₃, ... is an infinite sequence of random variables that are independent and identically distributed, and have a common expected value μ then

<math>P\left(\lim_{n\rightarrow\infty}\overline{X}_n=\mu\right)=1,</math>

This law justifies the intuitive interpretation of the expected value of a random variable as the "long-term average when sampling repeatedly".