Posterior probability

Part of a series on
Bayesian statistics
Posterior = Likelihood × Prior ÷ Evidence
Background
Bayesian inference Bayesian probability Bayes' theorem Bernstein–von Mises theorem Coherence Cox's theorem Cromwell's rule Principle of indifference Principle of maximum entropy
Model building
Strong prior Conjugate prior Linear regression Empirical Bayes Hierarchical model
Posterior approximation
Markov chain Monte Carlo Laplace's approximation Integrated nested Laplace approximations Variational inference Approximate Bayesian computation
Estimators
Bayesian estimator Credible interval Maximum a posteriori estimation
Evidence approximation
Evidence lower bound Nested sampling
Model evaluation
Bayes factor Model averaging Posterior predictive
Mathematics portal
v t e

The posterior probability is a type of

In variational Bayesian methods, the posterior probability is the probability of the parameters ${\displaystyle \theta }$ given the evidence ${\displaystyle X}$, and is denoted ${\displaystyle p(\theta |X)}$.

It contrasts with the likelihood function, which is the probability of the evidence given the parameters: ${\displaystyle p(X|\theta )}$.

The two are related as follows:

Given a prior belief that a probability distribution function is ${\displaystyle p(\theta )}$ and that the observations ${\displaystyle x}$ have a likelihood ${\displaystyle p(x|\theta )}$, then the posterior probability is defined as

${\displaystyle p(\theta |x)={\frac {p(x|\theta )}{p(x)}}p(\theta )}$,^[6]

where ${\displaystyle p(x)}$ is the normalizing constant and is calculated as

${\displaystyle p(x)=\int p(x|\theta )p(\theta )d\theta }$

for continuous ${\displaystyle \theta }$, or by summing ${\displaystyle p(x|\theta )p(\theta )}$ over all possible values of ${\displaystyle \theta }$ for discrete ${\displaystyle \theta }$.^[7]

The posterior probability is therefore

• ${\displaystyle P(G)}$, or the probability that the student is a girl regardless of any other information. Since the observer sees a random student, meaning that all students have the same probability of being observed, and the percentage of girls among the students is 40%, this probability equals 0.4.
• ${\displaystyle P(B)}$, or the probability that the student is not a girl (i.e. a boy) regardless of any other information (${\displaystyle B}$ is the complementary event to ${\displaystyle G}$). This is 60%, or 0.6.
• ${\displaystyle P(T|G)}$, or the probability of the student wearing trousers given that the student is a girl. As they are as likely to wear skirts as trousers, this is 0.5.
• ${\displaystyle P(T|B)}$, or the probability of the student wearing trousers given that the student is a boy. This is given as 1.
• ${\displaystyle P(T)}$, or the probability of a (randomly selected) student wearing trousers regardless of any other information. Since ${\displaystyle P(T)=P(T|G)P(G)+P(T|B)P(B)}$ (via the law of total probability), this is ${\displaystyle P(T)=0.5\times 0.4+1\times 0.6=0.8}$.

Given all this information, the posterior probability of the observer having spotted a girl given that the observed student is wearing trousers can be computed by substituting these values in the formula:

${\displaystyle P(G|T)={\frac {P(T|G)P(G)}{P(T)}}={\frac {0.5\times 0.4}{0.8}}=0.25.}$

An intuitive way to solve this is to assume the school has N students. Number of boys = 0.6N and number of girls = 0.4N. If N is sufficiently large, total number of trouser wearers = 0.6N+ 50% of 0.4N. And number of girl trouser wearers = 50% of 0.4N. Therefore, in the population of trousers, girls are (50% of 0.4N)/(0.6N+ 50% of 0.4N) = 25%. In other words, if you separated out the group of trouser wearers, a quarter of that group will be girls. Therefore, if you see trousers, the most you can deduce is that you are looking at a single sample from a subset of students where 25% are girls. And by definition, chance of this random student being a girl is 25%. Every Bayes-theorem problem can be solved in this way.^[9]

Calculation

The posterior probability distribution of one

gives the posterior probability density function for a random variable ${\displaystyle X}$ given the data ${\displaystyle Y=y}$, where

• ${\displaystyle f_{X}(x)}$ is the prior density of ${\displaystyle X}$,
• ${\displaystyle {\mathcal {L}}_{X\mid Y=y}(x)=f_{Y\mid X=x}(y)}$ is the likelihood function as a function of ${\displaystyle x}$,
• ${\displaystyle \int _{-\infty }^{\infty }f_{X}(u){\mathcal {L}}_{X\mid Y=y}(u)\,du}$ is the normalizing constant, and
• ${\displaystyle f_{X\mid Y=y}(x)}$ is the posterior density of ${\displaystyle X}$ given the data ${\displaystyle Y=y}$.^[10]

Credible interval

Posterior probability is a conditional probability conditioned on randomly observed data. Hence it is a random variable. For a random variable, it is important to summarize its amount of uncertainty. One way to achieve this goal is to provide a credible interval of the posterior probability.^[11]

Classification

In

See also

• Prediction interval
• Bernstein–von Mises theorem
• Probability of success
• Bayesian epistemology
• Metropolis–Hastings algorithm

References