Pseudolikelihood

This article's factual accuracy is disputed. Relevant discussion may be found on the talk page. Please help to ensure that disputed statements are reliably sourced. (October 2016) (Learn how and when to remove this message)

In statistical theory, a pseudolikelihood is an approximation to the joint probability distribution of a collection of random variables. The practical use of this is that it can provide an approximation to the likelihood function of a set of observed data which may either provide a computationally simpler problem for estimation, or may provide a way of obtaining explicit estimates of model parameters.

The pseudolikelihood approach was introduced by

Definition

Given a set of random variables ${\displaystyle X=X_{1},X_{2},\ldots ,X_{n}}$ the pseudolikelihood of ${\displaystyle X=x=(x_{1},x_{2},\ldots ,x_{n})}$ is

${\displaystyle L(\theta ):=\prod _{i}\mathrm {Pr} _{\theta }(X_{i}=x_{i}\mid X_{j}=x_{j}{\text{ for }}j\neq i)=\prod _{i}\mathrm {Pr} _{\theta }(X_{i}=x_{i}\mid X_{-i}=x_{-i})}$

in discrete case and

${\displaystyle L(\theta ):=\prod _{i}p_{\theta }(x_{i}\mid x_{j}{\text{ for }}j\neq i)=\prod _{i}p_{\theta }(x_{i}\mid x_{-i})=\prod _{i}p_{\theta }(x_{i}\mid x_{1},\ldots ,{\hat {x}}_{i},\ldots ,x_{n})}$

in continuous one. Here ${\displaystyle X}$ is a vector of variables, ${\displaystyle x}$ is a vector of values, ${\displaystyle p_{\theta }(\cdot \mid \cdot )}$ is conditional density and ${\displaystyle \theta =(\theta _{1},\ldots ,\theta _{p})}$ is the vector of parameters we are to estimate. The expression ${\displaystyle X=x}$ above means that each variable ${\displaystyle X_{i}}$ in the vector ${\displaystyle X}$ has a corresponding value ${\displaystyle x_{i}}$ in the vector ${\displaystyle x}$ and ${\displaystyle x_{-i}=(x_{1},\ldots ,{\hat {x}}_{i},\ldots ,x_{n})}$ means that the coordinate ${\displaystyle x_{i}}$ has been omitted. The expression ${\displaystyle \mathrm {Pr} _{\theta }(X=x)}$ is the probability that the vector of variables ${\displaystyle X}$ has values equal to the vector ${\displaystyle x}$. This probability of course depends on the unknown parameter ${\displaystyle \theta }$. Because situations can often be described using state variables ranging over a set of possible values, the expression ${\displaystyle \mathrm {Pr} _{\theta }(X=x)}$ can therefore represent the probability of a certain state among all possible states allowed by the state variables.

The pseudo-log-likelihood is a similar measure derived from the above expression, namely (in discrete case)

${\displaystyle l(\theta ):=\log L(\theta )=\sum _{i}\log \mathrm {Pr} _{\theta }(X_{i}=x_{i}\mid X_{j}=x_{j}{\text{ for }}j\neq i).}$

One use of the pseudolikelihood measure is as an approximation for inference about a Markov or Bayesian network, as the pseudolikelihood of an assignment to ${\displaystyle X_{i}}$ may often be computed more efficiently than the likelihood, particularly when the latter may require marginalization over a large number of variables.

Properties

Use of the pseudolikelihood in place of the true likelihood function in a