Heavy-tailed distribution

This article may be too technical for most readers to understand. Please help improve it to make it understandable to non-experts, without removing the technical details. (May 2020) (Learn how and when to remove this template message)

In probability theory, heavy-tailed distributions are probability distributions whose tails are not exponentially bounded:^[1] that is, they have heavier tails than the exponential distribution. In many applications it is the right tail of the distribution that is of interest, but a distribution may have a heavy left tail, or both tails may be heavy.

There are three important subclasses of heavy-tailed distributions: the

There is still some discrepancy over the use of the term heavy-tailed. There are two other definitions in use. Some authors use the term to refer to those distributions which do not have all their power

log-normal

that possess all their power moments, yet which are generally considered to be heavy-tailed. (Occasionally, heavy-tailed is used for any distribution that has heavier tails than the normal distribution.)

Definitions

Definition of heavy-tailed distribution

The distribution of a

That means

${\displaystyle \int _{-\infty }^{\infty }e^{tx}\,dF(x)=\infty \quad {\mbox{for all }}t>0.}$ [4]

This is also written in terms of the tail distribution function

${\displaystyle {\overline {F}}(x)\equiv \Pr[X>x]\,}$

as

${\displaystyle \lim _{x\to \infty }e^{tx}{\overline {F}}(x)=\infty \quad {\mbox{for all }}t>0.\,}$

Definition of long-tailed distribution

The distribution of a random variable X with distribution function F is said to have a long right tail^[1] if for all t > 0,

${\displaystyle \lim _{x\to \infty }\Pr[X>x+t\mid X>x]=1,\,}$

or equivalently

${\displaystyle {\overline {F}}(x+t)\sim {\overline {F}}(x)\quad {\mbox{as }}x\to \infty .\,}$

This has the intuitive interpretation for a right-tailed long-tailed distributed quantity that if the long-tailed quantity exceeds some high level, the probability approaches 1 that it will exceed any other higher level.

All long-tailed distributions are heavy-tailed, but the converse is false, and it is possible to construct heavy-tailed distributions that are not long-tailed.

Subexponential distributions

Subexponentiality is defined in terms of

${\displaystyle X_{1},X_{2}}$ with a common distribution function ${\displaystyle F}$, the convolution of ${\displaystyle F}$ with itself, written ${\displaystyle F^{*2}}$ and called the convolution square, is defined using Lebesgue–Stieltjes integration by:

${\displaystyle \Pr[X_{1}+X_{2}\leq x]=F^{*2}(x)=\int _{0}^{x}F(x-y)\,dF(y),}$

and the n-fold convolution ${\displaystyle F^{*n}}$ is defined inductively by the rule:

${\displaystyle F^{*n}(x)=\int _{0}^{x}F(x-y)\,dF^{*n-1}(y).}$

The tail distribution function ${\displaystyle {\overline {F}}}$ is defined as ${\displaystyle {\overline {F}}(x)=1-F(x)}$.

A distribution ${\displaystyle F}$ on the positive half-line is subexponential^[1][5][2] if

${\displaystyle {\overline {F^{*2}}}(x)\sim 2{\overline {F}}(x)\quad {\mbox{as }}x\to \infty .}$

This implies^[6] that, for any ${\displaystyle n\geq 1}$,

${\displaystyle {\overline {F^{*n}}}(x)\sim n{\overline {F}}(x)\quad {\mbox{as }}x\to \infty .}$

The probabilistic interpretation^[6] of this is that, for a sum of ${\displaystyle n}$

${\displaystyle X_{1},\ldots ,X_{n}}$ with common distribution ${\displaystyle F}$,

${\displaystyle \Pr[X_{1}+\cdots +X_{n}>x]\sim \Pr[\max(X_{1},\ldots ,X_{n})>x]\quad {\text{as }}x\to \infty .}$

This is often known as the principle of the single big jump^[7] or catastrophe principle.^[8]

A distribution ${\displaystyle F}$ on the whole real line is subexponential if the distribution ${\displaystyle FI([0,\infty ))}$ is.^[9] Here ${\displaystyle I([0,\infty ))}$ is the indicator function of the positive half-line. Alternatively, a random variable ${\displaystyle X}$ supported on the real line is subexponential if and only if ${\displaystyle X^{+}=\max(0,X)}$ is subexponential.

All subexponential distributions are long-tailed, but examples can be constructed of long-tailed distributions that are not subexponential.

Common heavy-tailed distributions

All commonly used heavy-tailed distributions are subexponential.^[6]

Those that are one-tailed include:

• the Pareto distribution;
• the Log-normal distribution;
• the Lévy distribution;
• the Weibull distribution with shape parameter greater than 0 but less than 1;
• the Burr distribution;
• the log-logistic distribution;
• the log-gamma distribution;
• the Fréchet distribution;
• the q-Gaussian distribution
• the log-Cauchy distribution, sometimes described as having a "super-heavy tail" because it exhibits logarithmic decay producing a heavier tail than the Pareto distribution.^[10][11]

Those that are two-tailed include:

• The Cauchy distribution, itself a special case of both the stable distribution and the t-distribution;
• The family of
  stable distributions,^[12] excepting the special case of the normal distribution within that family. Some stable distributions are one-sided (or supported by a half-line), see e.g. Lévy distribution. See also financial models with long-tailed distributions and volatility clustering
  .
• The t-distribution.
• The skew lognormal cascade distribution.^[13]

Relationship to fat-tailed distributions

A fat-tailed distribution is a distribution for which the probability density function, for large x, goes to zero as a power ${\displaystyle x^{-a}}$. Since such a power is always bounded below by the probability density function of an exponential distribution, fat-tailed distributions are always heavy-tailed. Some distributions, however, have a tail which goes to zero slower than an exponential function (meaning they are heavy-tailed), but faster than a power (meaning they are not fat-tailed). An example is the log-normal distribution^{[contradictory]}. Many other heavy-tailed distributions such as the log-logistic and Pareto distribution are, however, also fat-tailed.

Estimating the tail-index

There are parametric^[6] and non-parametric^[14] approaches to the problem of the tail-index estimation.^{[when defined as?]}

To estimate the tail-index using the parametric approach, some authors employ

; they may apply the maximum-likelihood estimator (MLE).

Pickand's tail-index estimator

With ${\displaystyle (X_{n},n\geq 1)}$ a random sequence of independent and same density function ${\displaystyle F\in D(H(\xi ))}$, the Maximum Attraction Domain[15] of the generalized extreme value density ${\displaystyle H}$, where ${\displaystyle \xi \in \mathbb {R} }$. If ${\displaystyle \lim _{n\to \infty }k(n)=\infty }$ and ${\displaystyle \lim _{n\to \infty }{\frac {k(n)}{n}}=0}$, then the Pickands tail-index estimation is^[6][15]

${\displaystyle \xi _{(k(n),n)}^{\text{Pickands}}={\frac {1}{\ln 2}}\ln \left({\frac {X_{(n-k(n)+1,n)}-X_{(n-2k(n)+1,n)}}{X_{(n-2k(n)+1,n)}-X_{(n-4k(n)+1,n)}}}\right),}$

where ${\displaystyle X_{(n-k(n)+1,n)}=\max \left(X_{n-k(n)+1},\ldots ,X_{n}\right)}$. This estimator converges in probability to ${\displaystyle \xi }$.

Hill's tail-index estimator

Let ${\displaystyle (X_{t},t\geq 1)}$ be a sequence of independent and identically distributed random variables with distribution function ${\displaystyle F\in D(H(\xi ))}$, the maximum domain of attraction of the generalized extreme value distribution ${\displaystyle H}$, where ${\displaystyle \xi \in \mathbb {R} }$. The sample path is ${\displaystyle {X_{t}:1\leq t\leq n}}$ where ${\displaystyle n}$ is the sample size. If ${\displaystyle \{k(n)\}}$ is an intermediate order sequence, i.e. ${\displaystyle k(n)\in \{1,\ldots ,n-1\},}$, ${\displaystyle k(n)\to \infty }$ and ${\displaystyle k(n)/n\to 0}$, then the Hill tail-index estimator is^[16]

${\displaystyle \xi _{(k(n),n)}^{\text{Hill}}=\left({\frac {1}{k(n)}}\sum _{i=n-k(n)+1}^{n}\ln(X_{(i,n)})-\ln(X_{(n-k(n)+1,n)})\right)^{-1},}$

where ${\displaystyle X_{(i,n)}}$ is the ${\displaystyle i}$-th order statistic of ${\displaystyle X_{1},\dots ,X_{n}}$. This estimator converges in probability to ${\displaystyle \xi }$, and is asymptotically normal provided ${\displaystyle k(n)\to \infty }$ is restricted based on a higher order regular variation property^[17] .^[18] Consistency and asymptotic normality extend to a large class of dependent and heterogeneous sequences,^[19][20] irrespective of whether ${\displaystyle X_{t}}$ is observed, or a computed residual or filtered data from a large class of models and estimators, including mis-specified models and models with errors that are dependent.^[21][22][23] Note that both Pickand's and Hill's tail-index estimators commonly make use of logarithm of the order statistics.^[24]

Ratio estimator of the tail-index

The ratio estimator (RE-estimator) of the tail-index was introduced by Goldie and Smith.^[25] It is constructed similarly to Hill's estimator but uses a non-random "tuning parameter".

A comparison of Hill-type and RE-type estimators can be found in Novak.^[14]

Software

• aest Archived 2020-11-25 at the Wayback Machine, C tool for estimating the heavy-tail index.^[26]

Estimation of heavy-tailed density

Nonparametric approaches to estimate heavy- and superheavy-tailed probability density functions were given in Markovich.^[27] These are approaches based on variable bandwidth and long-tailed kernel estimators; on the preliminary data transform to a new random variable at finite or infinite intervals, which is more convenient for the estimation and then inverse transform of the obtained density estimate; and "piecing-together approach" which provides a certain parametric model for the tail of the density and a non-parametric model to approximate the mode of the density. Nonparametric estimators require an appropriate selection of tuning (smoothing) parameters like a bandwidth of kernel estimators and the bin width of the histogram. The well known data-driven methods of such selection are a cross-validation and its modifications, methods based on the minimization of the mean squared error (MSE) and its asymptotic and their upper bounds.^[28] A discrepancy method which uses well-known nonparametric statistics like Kolmogorov-Smirnov's, von Mises and Anderson-Darling's ones as a metric in the space of distribution functions (dfs) and quantiles of the later statistics as a known uncertainty or a discrepancy value can be found in.^[27] Bootstrap is another tool to find smoothing parameters using approximations of unknown MSE by different schemes of re-samples selection, see e.g.^[29]

See also

• Leptokurtic distribution
• Generalized extreme value distribution
• Generalized Pareto distribution
• Outlier
• Long tail
• Power law
• Seven states of randomness
• Fat-tailed distribution
  • Taleb distribution and Holy grail distribution

References