Convergence of random variables

In probability theory, several different notions of convergence of random variables are investigated. These will be presented here. Throughout, we assume that (X_n) is a sequence of random variables, and X is a random variable, and all of them are defined on the same probability space (Ω, P).

Table of contents 1 Convergence in distribution 2 Convergence in probability 3 Almost sure convergence 4 Convergence in mean 5 Convergence in mean square

Convergence in distribution

We say that the sequence X_n converges towards X in distribution, if

<math>\lim_{n\rightarrow\infty}P(X_n\leq a)=P(X\leq a)</math>

for every real number a at which the cumulative distribution function of the limiting random variable X is continuous.

Essentially, this means that the probability that the value of X is in a given range is very similar to the probability that the value of X_n is in that range, if only n is large enough. This notion of convergence is used in the central limit theorems.

Convergence in distribution is also called convergence in law, since the word "law" is sometimes used as a synonym of "probability distribution." Another name is weak convergence.

Convergence in probability

We say that the sequence X_n converges towards X in probability if

<math>\lim_{n\rightarrow\infty}P\left(\left|X_n-X\right|\geq\varepsilon\right)=0</math>

for every ε > 0.

This means that if you pick a tolerance ε and choose n large enough, then the value of X_n will be almost guaranteed to be within that tolerance of the value of X. This notion of convergence is used in the weak law of large numbers.

Convergence in probability implies convergence in distribution.

Almost sure convergence

We say that the sequence X_n converges almost surely or almost everywhere or with probability 1 or strongly towards X if

<math>P\left(\lim_{n\rightarrow\infty}X_n=X\right)=1.</math>

This means that you are virtually guaranteed that the values of X_n approach the value of X. This notion of convergence is used in the strong law of large numbers.

Almost sure convergence implies convergence in probability.

Convergence in mean

We say that the sequence X_n converges towards X in mean or in the L¹ norm if

<math>\lim_{n\rightarrow\infty}E\left(\left|X_n-X\right|\right)=0</math>

where E denotes the expected value.

This means that the expected difference between X_n and X gets as small as desired if n is chosen big enough. This convergence is considered in L^p spaces (where p = 1).

Convergence in the mean implies convergence in probability. There is no general relation between convergence in mean and almost sure convergence however.

Convergence in mean square

We say that the sequence X_n converges towards X in mean square or in the L² norm if

<math>\lim_{n\rightarrow\infty}E\left(\left|X_n-X\right|^2\right)=0.</math>

This means that the expected squared difference between X_n and X gets as small as desired if n is chosen big enough. This convergence is considered in L^p spaces (where p = 2).

Convergence in mean square implies convergence in mean.