In probability theory, the Gram–Charlier A series (named in honor of Jørgen Pedersen Gram and Carl Charlier), and the Edgeworth series (named in honor of Francis Ysidro Edgeworth) are series that approximate a probability distribution over the real line
(
−
∞
,
∞
)
{\displaystyle (-\infty ,\infty )}
in terms of its cumulants. The series are the same; but, the arrangement of terms (and thus the accuracy of truncating the series) differ. The key idea of these expansions is to write the characteristic function of the distribution whose probability density function f is to be approximated in terms of the characteristic function of a distribution with known and suitable properties, and to recover f through the inverse Fourier transform.
Contents
Gram–Charlier A series
We examine a continuous random variable. Let
f
^
{\displaystyle {\hat {f}}}
be the characteristic function of its distribution whose density function is f, and
κ
r
{\displaystyle \kappa _{r}}
its cumulants. We expand in terms of a known distribution with probability density function ψ, characteristic function
ψ
^
{\displaystyle {\hat {\psi }}}
, and cumulants
γ
r
{\displaystyle \gamma _{r}}
. The density ψ is generally chosen to be that of the normal distribution, but other choices are possible as well. By the definition of the cumulants, we have (see Wallace, 1958)
f
^
(
t
)
=
exp
[
∑
r
=
1
∞
κ
r
(
i
t
)
r
r
!
]
{\displaystyle {\hat {f}}(t)=\exp \left[\sum _{r=1}^{\infty }\kappa _{r}{\frac {(it)^{r}}{r!}}\right]}
and
ψ
^
(
t
)
=
exp
[
∑
r
=
1
∞
γ
r
(
i
t
)
r
r
!
]
,
{\displaystyle {\hat {\psi }}(t)=\exp \left[\sum _{r=1}^{\infty }\gamma _{r}{\frac {(it)^{r}}{r!}}\right],}
which gives the following formal identity:
f
^
(
t
)
=
exp
[
∑
r
=
1
∞
(
κ
r
−
γ
r
)
(
i
t
)
r
r
!
]
ψ
^
(
t
)
.
{\displaystyle {\hat {f}}(t)=\exp \left[\sum _{r=1}^{\infty }(\kappa _{r}-\gamma _{r}){\frac {(it)^{r}}{r!}}\right]{\hat {\psi }}(t)\,.}
By the properties of the Fourier transform,
(
i
t
)
r
ψ
^
(
t
)
{\displaystyle (it)^{r}{\hat {\psi }}(t)}
is the Fourier transform of
(
−
1
)
r
[
D
r
ψ
]
(
−
x
)
{\displaystyle (-1)^{r}[D^{r}\psi ](-x)}
, where D is the differential operator with respect to x. Thus, after changing
x
{\displaystyle x}
with
−
x
{\displaystyle -x}
on both sides of the equation, we find for f the formal expansion
f
(
x
)
=
exp
[
∑
r
=
1
∞
(
κ
r
−
γ
r
)
(
−
D
)
r
r
!
]
ψ
(
x
)
.
{\displaystyle f(x)=\exp \left[\sum _{r=1}^{\infty }(\kappa _{r}-\gamma _{r}){\frac {(-D)^{r}}{r!}}\right]\psi (x)\,.}
If ψ is chosen as the normal density
φ
(
x
)
=
1
2
π
σ
exp
[
−
(
x
−
μ
)
2
2
σ
2
]
{\displaystyle \varphi (x)={\frac {1}{{\sqrt {2\pi }}\sigma }}\exp \left[-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right]}
with mean and variance as given by f, that is, mean
μ
=
κ
1
{\displaystyle \mu =\kappa _{1}}
and variance
σ
2
=
κ
2
{\displaystyle \sigma ^{2}=\kappa _{2}}
, then the expansion becomes
f
(
x
)
=
exp
[
∑
r
=
3
∞
κ
r
(
−
D
)
r
r
!
]
φ
(
x
)
,
{\displaystyle f(x)=\exp \left[\sum _{r=3}^{\infty }\kappa _{r}{\frac {(-D)^{r}}{r!}}\right]\varphi (x),}
since
γ
r
=
0
{\displaystyle \gamma _{r}=0}
for all r > 2, as higher cumulants of the normal distribution are 0. By expanding the exponential and collecting terms according to the order of the derivatives, we arrive at the Gram–Charlier A series. Such an expansion can be written compactly in terms of Bell polynomials as
exp
[
∑
r
=
3
∞
κ
r
(
−
D
)
r
r
!
]
=
∑
n
=
0
∞
B
n
(
0
,
0
,
κ
3
,
…
,
κ
n
)
(
−
D
)
n
n
!
.
{\displaystyle \exp \left[\sum _{r=3}^{\infty }\kappa _{r}{\frac {(-D)^{r}}{r!}}\right]=\sum _{n=0}^{\infty }B_{n}(0,0,\kappa _{3},\ldots ,\kappa _{n}){\frac {(-D)^{n}}{n!}}.}
Since the n-th derivative of the Gaussian function
φ
{\displaystyle \varphi }
is given in terms of Hermite polynomial as
φ
(
n
)
(
x
)
=
(
−
1
)
n
σ
n
He
n
(
x
−
μ
σ
)
φ
(
x
)
,
{\displaystyle \varphi ^{(n)}(x)={\frac {(-1)^{n}}{\sigma ^{n}}}\operatorname {He} _{n}\left({\frac {x-\mu }{\sigma }}\right)\varphi (x),}
this gives us the final expression of the Gram–Charlier A series as
f
(
x
)
=
φ
(
x
)
∑
n
=
0
∞
1
n
!
σ
n
B
n
(
0
,
0
,
κ
3
,
…
,
κ
n
)
He
n
(
x
−
μ
σ
)
.
{\displaystyle f(x)=\varphi (x)\sum _{n=0}^{\infty }{\frac {1}{n!\sigma ^{n}}}B_{n}(0,0,\kappa _{3},\ldots ,\kappa _{n})\operatorname {He} _{n}\left({\frac {x-\mu }{\sigma }}\right).}
Integrating the series gives us the cumulative distribution function
F
(
x
)
=
∫
−
∞
x
f
(
u
)
d
u
=
Φ
(
x
)
−
φ
(
x
)
∑
n
=
3
∞
1
n
!
σ
n
−
1
B
n
(
0
,
0
,
κ
3
,
…
,
κ
n
)
He
n
−
1
(
x
−
μ
σ
)
,
{\displaystyle F(x)=\int _{-\infty }^{x}f(u)du=\Phi (x)-\varphi (x)\sum _{n=3}^{\infty }{\frac {1}{n!\sigma ^{n-1}}}B_{n}(0,0,\kappa _{3},\ldots ,\kappa _{n})\operatorname {He} _{n-1}\left({\frac {x-\mu }{\sigma }}\right),}
where
Φ
{\displaystyle \Phi }
is the CDF of the normal distribution.
If we include only the first two correction terms to the normal distribution, we obtain
f
(
x
)
≈
1
2
π
σ
exp
[
−
(
x
−
μ
)
2
2
σ
2
]
[
1
+
κ
3
3
!
σ
3
H
e
3
(
x
−
μ
σ
)
+
κ
4
4
!
σ
4
He
4
(
x
−
μ
σ
)
]
,
{\displaystyle f(x)\approx {\frac {1}{{\sqrt {2\pi }}\sigma }}\exp \left[-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right]\left[1+{\frac {\kappa _{3}}{3!\sigma ^{3}}}He_{3}\left({\frac {x-\mu }{\sigma }}\right)+{\frac {\kappa _{4}}{4!\sigma ^{4}}}\operatorname {He} _{4}\left({\frac {x-\mu }{\sigma }}\right)\right]\,,}
with
H
e
3
(
x
)
=
x
3
−
3
x
{\displaystyle He_{3}(x)=x^{3}-3x}
and
He
4
(
x
)
=
x
4
−
6
x
2
+
3
{\displaystyle \operatorname {He} _{4}(x)=x^{4}-6x^{2}+3}
.
Note that this expression is not guaranteed to be positive, and is therefore not a valid probability distribution. The Gram–Charlier A series diverges in many cases of interest—it converges only if
f
(
x
)
{\displaystyle f(x)}
falls off faster than
exp
(
−
(
x
2
)
/
4
)
{\displaystyle \exp(-(x^{2})/4)}
at infinity (Cramér 1957). When it does not converge, the series is also not a true asymptotic expansion, because it is not possible to estimate the error of the expansion. For this reason, the Edgeworth series (see next section) is generally preferred over the Gram–Charlier A series.
The Edgeworth series
Edgeworth developed a similar expansion as an improvement to the central limit theorem. The advantage of the Edgeworth series is that the error is controlled, so that it is a true asymptotic expansion.
Let
{
Z
i
}
{\displaystyle \{Z_{i}\}}
be a sequence of independent and identically distributed random variables with finite mean
μ
{\displaystyle \mu }
and variance
σ
2
{\displaystyle \sigma ^{2}}
, and let
X
n
{\displaystyle X_{n}}
be their standardized sums:
X
n
=
1
n
∑
i
=
1
n
Illustration: density of the sample mean of three χ² distributions
Take
X
i
∼
χ
2
(
k
=
2
)
,
i
=
1
,
2
,
3
(
n
=
3
)
{\displaystyle X_{i}\sim \chi ^{2}(k=2),\,i=1,2,3\,(n=3)}
and the sample mean
X
¯
=
1
3
∑
i
=
1
Discussion of results
For finite samples, an Edgeworth expansion is not guaranteed to be a proper probability distribution as the CDF values at some points may go beyond
[
0
,
1
]
{\displaystyle [0,1]}
.
They guarantee (asymptotically) absolute errors, but relative errors can be easily assessed by comparing the leading Edgeworth term in the remainder with the overall leading term.