Bayesian probability

Bayesianism is the philosophical tenet that the mathematical theory of probability applies to the degree of plausibility of statements, or to the degree of belief of rational agents in the truth of statements. It is opposed to frequentism, which rejects degree-of-belief interpretations of mathematical probability, and assigns probabilities instead to random events according to their relative frequencies of occurrence. Whereas a frequentist might assign probability 1/2 to the event of getting a head when a coin is tossed (but only if the frequentist knows that that is the relative frequency) a Bayesian might assign probability 1/2 to the proposition that there was life on Mars a billion years ago, without intending that assignment to assert anything about any relative frequency.

The terms subjectivism, subjective probability, personal probability, epistemic probability and logical probability are used to describe what Bayesians believe in. Not all of these terms are synonymous, however.

Advocates of "logical probability" would like to codify techniques whereby if two people have the same prior information relevant to the truth of an uncertain proposition, then they would assign the same probability. No one has any idea how to do that except in simple cases, and then the validity of proposed methods is subject to philosophical controversy. The most sophisticated proponents of this view have been Sir Harold Jeffreys[?] and Edwin Jaynes.

"Subjective probability" is supposed to measure how sure someone is of an uncertain proposition.

The Bayesian approach is in contrast to frequency probability where probability is held to be derived from observed or imagined frequency distributions or proportions of populations. The difference has many implications for the methods by which statistics is practiced when following one model or the other.

History of Bayesian probability

Bayesianism is named after Thomas Bayes, the originator of Bayes' theorem. This theorem is often used to update the plausibility of a given statement in light of new evidence. Laplace (1812) rediscovered the theorem and put it to good use in solving problems in celestial mechanics, medical statistics and, by some accounts, even jurisprudence.

For instance, he estimated the mass of Saturn, given orbital data that were available to him from various astronomical observations. He presented the result together with an indication of its uncertainty, stating it like this: `It is a bet of 11000 to 1 that the error in this result is not within 1/100th of its value'. He would have won the bet, as another 150 years' accumulation of data has changed the estimate by only 0.63%. According to the frequency probability definition, however, we are not permitted to use probability theory to tackle this problem. This is because the mass of Saturn is a constant and not a random variable, therefore, it has no frequency distribution and so probability theory cannot be used.

Bayesianism has been championed by L. J. Savage[?], Bruno de Finetti, Edwin Jaynes, Frank P. Ramsey, and others. They created the idea of defining rational belief[?] as an abstraction of betting behavior subject to the constraint that one doesn't want to be inconsistent in his behavior. A series of critiques of statistical methods was based on this concept and formed the basis of debate from the 1950s and statisticians remain divided on the issue.

See uncertainty

Applications of Bayesian probability

Today, there are a variety of applications of personal probability that have gained wide acceptance. Some schools of thought emphasise Cox's theorem and Jaynes' principle of maximum entropy as cornerstones of the theory, while others may claim that Bayesian methods are more general and give better results in practice than frequency probability.

There is growing interest in using Bayesian probability to filter spam. For example: Bogofilter