Bayes' Theorem

Bayes' theorem is a rule in probability theory that describes how to revise the probability of a hypothesis in light of new evidence. It combines a prior belief, how likely something seemed before the evidence, with the likelihood of seeing that evidence if the hypothesis were true, to produce an updated or posterior probability. It is named after Thomas Bayes, an eighteenth-century English minister and mathematician, whose essay on the subject was published in 1763 after his death; Pierre-Simon Laplace developed the idea further.

The theorem underpins Bayesian statistics, an approach to inference that treats probability as a degree of belief. It is used in medicine, law, engineering and econometrics, and in finance for credit scoring, fraud detection, portfolio construction and forecasting. Many spam filters, anomaly detection tools and fraud detection systems apply Bayesian reasoning, as do machine learning models that must combine past knowledge with new data.

Imagine a smoke alarm that almost always sounds when there is a fire, but also sometimes goes off when you burn toast. If it beeps, should you assume the house is on fire? That depends on how often there are real fires compared with how often you burn toast. Because real fires are rare and burnt toast is common, most beeps are false alarms, even though the alarm is good at detecting fires.

Bayes' theorem is the arithmetic behind that common sense. It tells you to start with how likely something was beforehand, then adjust for what the new clue tells you. A clue that is common even when nothing is wrong should not change your mind much; a clue that almost never appears unless something is wrong should change it a lot.

The theorem is written as:

• $P(H \mid E)$ is the posterior: the probability of hypothesis $H$ given evidence $E$.
• $P(E \mid H)$ is the likelihood: the probability of seeing the evidence if the hypothesis is true.
• $P(H)$ is the prior: the probability of the hypothesis before seeing the evidence.
• $P(E)$ is the overall probability of the evidence, across all cases.

Consider a payment fraud alert. Suppose 1 in 1,000 card transactions is fraudulent, so $P(H) = 0.001$. The alert flags 95% of frauds, so $P(E \mid H) = 0.95$, but it also flags 2% of genuine transactions. The overall chance of an alert is $0.95 \times 0.001 + 0.02 \times 0.999 = 0.00095 + 0.01998 = 0.02093$. The probability that a flagged transaction is actually fraud is therefore $0.00095 \div 0.02093 \approx 0.045$, or about 4.5%. Despite a 95% detection rate, more than 95% of alerts are false positives, because genuine transactions vastly outnumber frauds. Lowering the threshold to catch more fraud raises false alerts further, while raising it pushes up the false negative rate, so the base rate shapes every design choice.

The fraud example explains why banks tune alert thresholds carefully and why compliance teams face large volumes of false alerts in transaction monitoring. Bayesian reasoning also appears in:

• Credit scoring. Lenders update a borrower's estimated probability of default as new payment data arrives, a core task in managing credit risk and setting a credit score.
• Portfolio construction. The Black-Litterman model blends market-implied returns with an investor's own views, weighting each by confidence.
• Forecasting. Central banks and economists use Bayesian models to combine economic theory with noisy data.
• Insurance. Actuaries use credibility methods, closely related to Bayesian updating and to expected value calculations, to blend a policyholder's own claims history with wider experience.
• Trading. Quantitative traders update signals as prices and news arrive, and prediction markets can be read as crowds performing a form of Bayesian updating.

Ignoring the prior, known as base-rate neglect, is the classic mistake: people judge a test or signal by its accuracy alone and overlook how rare the condition is. The prosecutor's fallacy, which confuses the probability of evidence given innocence with the probability of innocence given evidence, has led to wrongful convictions. In markets, investors may overreact to dramatic news or underreact to steady evidence, patterns studied in behavioral finance. Bayesian methods also need a prior, which can be subjective, and critics argue that results can be steered by the choice of prior. Supporters reply that making assumptions explicit is better than hiding them, and that with enough data the influence of the prior fades.

Classical, or frequentist, statistics treats probability as the long-run frequency of events and avoids priors, relying on tools such as confidence intervals and significance tests used in regression analysis. Bayesian statistics treats unknown quantities as having probability distributions, such as a normal distribution, and updates them with data. For much of the twentieth century frequentist methods dominated, partly because Bayesian calculations were hard to perform. Cheap computing and simulation methods developed from the 1990s made Bayesian models practical, and both approaches are now used side by side in finance, with the choice often depending on how much data and prior knowledge are available.