In statistics, the Wishart distribution is a generalization of the gamma distribution to multiple dimensions. It is named in honor of John Wishart, who first formulated the distribution in 1928. Other names include Wishart ensemble, or Wishart–Laguerre ensemble, or LOE, LUE, LSE.
Article
In statistics, the Wishart distribution is a generalization of the gamma distribution to multiple dimensions. It is named in honor of John Wishart, who first formulated the distribution in 1928. Other names include Wishart ensemble (in random matrix theory, probability distributions over matrices are usually called "ensembles"), or Wishart–Laguerre ensemble (since its eigenvalue distribution involves Laguerre polynomials), or LOE, LUE, LSE (in analogy with GOE, GUE, GSE).
It is a family of probability distributions defined over symmetric, positive-definite random matrices (i.e. matrix-valued random variables). These distributions are of great importance in the estimation of covariance matrices in multivariate statistics. In Bayesian statistics, the Wishart distribution is the conjugate prior of the inverse covariance-matrix of a multivariate-normal random vector.
Contents
Definition
Suppose G is a p × n matrix, each column of which is independently drawn from a p-variate normal distribution with zero mean:
The Wishart distribution arises as the distribution of the sample covariance matrix for a sample from a multivariate normal distribution. It occurs frequently in likelihood-ratio tests in multivariate statistical analysis. It also arises in the spectral theory of random matrices and in multidimensional Bayesian analysis. It is also encountered in wireless communications, while analyzing the performance of Rayleigh fading MIMO wireless channels.
Probability density function
The Wishart distribution can be characterized by its probability density function as follows:
Let X be a p × p symmetric matrix of random variables that is positive semi-definite. Let V be a (fixed) symmetric positive definite matrix of size p × p.
Then, if n ≥ p, X has a Wishart distribution with n degrees of freedom if it has the probability density function
In Bayesian statistics, in the context of the multivariate normal distribution, the Wishart distribution is the conjugate prior to the precision matrix Ω = Σ−1, where Σ is the covariance matrix. The use of Normal-Wishart conjugate priors (for mean and precision) is particularly common for vector autoregression models.
Choice of parameters
The least informative, proper Wishart prior is obtained by setting n = p.
A common choice for V leverages the fact that the mean of X ~Wp(V, n) is nV. Then V is chosen so that nV equals an initial guess for X. For instance, when estimating a precision matrix Σ−1 ~ Wp(V, n) a reasonable choice for V would be n−1Σ0−1, where Σ0 is some prior estimate for the covariance matrix Σ.
Properties
Log-expectation
The following formula plays a role in variational Bayes derivations for Bayes networks
involving the Wishart distribution. From equation (2.63),
where E[⋅] denotes expectation. (Here Θ is any matrix with the same dimensions as V, 1 indicates the identity matrix, and i is a square root of −1). Properly interpreting this formula requires a little care, because noninteger complex powers are multivalued; when n is noninteger, the correct branch must be determined via analytic continuation.
Theorem
If a p × p random matrix X has a Wishart distribution with m degrees of freedom and variance matrix V — write
gives the marginal distribution of each of the elements on the matrix's diagonal.
George Seber points out that the Wishart distribution is not called the “multivariate chi-squared distribution” because the marginal distribution of the off-diagonal elements is not chi-squared. Seber prefers to reserve the term multivariate for the case when all univariate marginals belong to the same family.
Estimator of the multivariate normal distribution
The Wishart distribution is the sampling distribution of the maximum-likelihood estimator (MLE) of the covariance matrix of a multivariate normal distribution. A derivation of the MLE uses the spectral theorem.
Bartlett decomposition
The Bartlett decomposition of a matrix X from a p-variate Wishart distribution with scale matrix V and n degrees of freedom is the factorization:
This set is named after Simon Gindikin, who introduced it in the 1970s in the context of gamma distributions on homogeneous cones. However, for the new parameters in the discrete spectrum of the Gindikin ensemble, namely,
Λ
p
∗
:=
{
0
Wishart–Laguerre ensembles and β-extensions
In random matrix theory, the Wishart family is often studied through its Laguerre ensembles. For the real case (orthogonal symmetry, β=1), the joint density of the eigenvalues
. This relationship may be derived by noting that the absolute value of the Jacobian determinant of this change of variables is |C|p+1, see for example equation (15.15) in.
In Bayesian statistics, the Wishart distribution is a conjugate prior for the precision parameter of the multivariate normal distribution, when the mean parameter is known.
A generalization is the multivariate gamma distribution.
Article from Wikipedia (CC BY-SA 4.0), where it is maintained by volunteer editors.
known as the scatter matrix. One indicates that S has that probability distribution by writing
S
∼
W
p
(
V
,
n
)
.
{\displaystyle S\sim W_{p}(V,n).}
The positive integer n is the number of degrees of freedom. Sometimes this is written W(V, p, n). For n ≥ p the matrix S is invertible with probability 1 if V is invertible.
If p = V = 1 then this distribution is a chi-squared distribution with n degrees of freedom.
The density above is not the joint density of all the
p
2
{\displaystyle p^{2}}
elements of the random matrix X (such
p
2
{\displaystyle p^{2}}
-dimensional density does not exist because of the symmetry constrains
X
i
j
=
X
j
i
{\displaystyle X_{ij}=X_{ji}}
), it is rather the joint density of
p
(
p
+
1
)
/
2
{\displaystyle p(p+1)/2}
elements
X
i
j
{\displaystyle X_{ij}}
for
i
≤
j
{\displaystyle i\leq j}
(, page 38). Also, the density formula above applies only to positive definite matrices
x
;
{\displaystyle \mathbf {x} ;}
for other matrices the density is equal to zero.
In fact the above definition can be extended to any real n > p − 1. If n ≤ p − 1, then the Wishart no longer has a density—instead it represents a singular distribution that takes values in a lower-dimension subspace of the space of p × p matrices.
The diagonal elements, most evidently in the first element, follow the χ2 distribution with n degrees of freedom (scaled by σ2) as expected. The off-diagonal element is less familiar but can be identified as a normal variance-mean mixture where the mixing density is a χ2 distribution. The corresponding marginal probability density for the off-diagonal element is therefore the variance-gamma distribution
It is also possible to write down the moment-generating function even in the noncentral case (essentially the nth power of Craig (1936) equation 10) although the probability density becomes an infinite sum of Bessel functions.
A different type of generalization is the normal-Wishart distribution, essentially the product of a multivariate normal distribution with a Wishart distribution.
