Encyclopedia > Likelihood function

  Article Content


Redirected from Likelihood function

In the English language, the word likelihood means the same as, or almost the same as, the word 'probability'.

In statistics likelihood is a concept related to, but distinct from, that of probability. (Confusingly, likelihood is also used in its informal English sense in some discussions of the interpretation of mathematical probability.)

Unlike probability, which estimates the frequency of or degree of belief in unknown consequences of known causes, likelihood works backwards, from observed results to hypothetical models and parameters.

The likelihood function is the function which specifies the probability of the sample observed on the basis of a known model, as a function of the model's parameters.

Note: This is not the same as the probability that those parameters are the right ones, given the observed sample. Attempting to interpret the likelihood of a hypothesis given observed evidence as the probability of the hypothesis is a common error, with potentially disastrous real-world consequences in medicine, engineering or jurisprudence.


For example, if I toss a coin, with a probability pH of landing heads up ('H'), the probability of getting two heads in two trials ('HH') is pH2. If pH = 0.5, then the probability of seeing two heads is 0.25.

In symbols, we can say the above as

<math>P(\mbox{HH} \mid p_H = 0.5) = 0.25</math>

Another way of saying this is to reverse it and say that "the likelihood of pH = 0.5 given the observation 'HH' is 0.25", i.e.,

<math>L(p_H=0.5 \mid \mbox{HH}) = P(\mbox{HH}\mid p_H=0.5) =0.25</math>.

But this is not the same as saying that the probability of pH = 0.5 given the observation is 0.25.

To take an extreme case, on this basis we can say "the likelihood of pH = 1 given the observation 'HH' is 1". But it is clearly not the case that the probability of pH = 1 given the observation is 1: the event 'HH' can occur for any pH > 0 (and often does, in reality, for pH roughly 0.5).

The likelihood function does not in general follow all the axioms of probability: for example, the integral over the likelihood density function does not in general sum to 1.

This is because integration of the likelihood density function <math>L</math> is performed over all possible values of the model parameters (in this case, <math>p_H</math>), while integration of a probability density function <math>P</math> is performed over the random variables (which in this case take on the four pairs of values 'TT', 'TH', 'HT' and 'HH').

In this example, the integral of the likelihood density over the interval [0, 1] in pH is 1/3, demonstrating again that the likelihood density function cannot be interpreted as a probability density function for pH.

On the other hand, given any particular value of pH, e.g. pH=0.5, the integral of the probability density function over the domain of the random variables is 1.

See also:

All Wikipedia text is available under the terms of the GNU Free Documentation License

  Search Encyclopedia

Search over one million articles, find something about almost anything!
  Featured Article
U.S. list of state sponsors of international terrorism

... sponsors of international terrorism U.S. list of state sponsors of international terrorism The list, which imposes strict sanctions, has remained unchanged since ...