Isserlis' Theorem

$2$ $x_1, x_2, \cdots, x_d$ $d$

\begin{matrix} (1) & E [x_{1} x_{2} \dots x_{d}] = \sum_{p \in P_{d}^{2}} \prod_{{i, j} \in p} E [x_{i} x_{j}] \end{matrix}

$p$ $\{1,2, \cdots , d\}$ $d$ ${\mathrm E} [x_1 x_2 \cdots x_d] =0$ $2$ ${\mathrm E} [x_i x_j] = {\mathrm {Cov}} [x_i x_j]$ .

$4$ -th order moments goes like this

\begin{matrix} (2) & E [x_{1} x_{2} x_{3} x_{4}] = E [x_{1} x_{2}] E [x_{3} x_{4}] + E [x_{1} x_{3}] E [x_{2} x_{4}] + E [x_{1} x_{4}] E [x_{2} x_{3}] \end{matrix}

Proof

$\mathbf x \in \mathbb R^d$ $\mathbf x \sim {\mathcal N}(\mathbf 0, \mathbf \Sigma)$ .

We use a quadratic identity equation

\begin{matrix} (3) & (x - Σ v)^{⊤} Σ^{- 1} (x - Σ v) = x^{⊤} Σ^{- 1} x - 2 x^{⊤} v + v^{⊤} Σ v \end{matrix}

$1$ , we have

\begin{aligned} (4) & 1 = \int \frac{e^{- \frac{1}{2} (x - Σ v)^{⊤} Σ^{- 1} (x - Σ v)}}{| 2 π Σ |} d x & = \int \frac{e^{- \frac{1}{2} x^{⊤} Σ^{- 1} x + x^{⊤} v - \frac{1}{2} v^{⊤} Σ v}}{| 2 π Σ |} d x \\ (5) & \Rightarrow e^{\frac{1}{2} v^{⊤} Σ v} & = \int \frac{e^{- \frac{1}{2} x^{⊤} Σ^{- 1} x + x^{⊤} v}}{| 2 π Σ |} d x \end{aligned}

$(4)$ $v_1, v_2, \cdots, v_d$ $v_1, v_2, \cdots, v_d=0$ . Differentiating the right-hand side, we get exactly the moments we want

\begin{aligned} \frac{\partial \int \frac{1}{| 2 π Σ |} e^{- \frac{1}{2} x^{⊤} Σ^{- 1} x + x^{⊤} v} d x}{\partial v_{1} \partial v_{2} \dots \partial v_{d}} |_{v_{1}, v_{2}, \dots, v_{d} = 0} & = \frac{1}{| 2 π Σ |} \int \frac{\partial e^{- \frac{1}{2} x^{⊤} Σ^{- 1} x + x^{⊤} v}}{\partial v_{1} \partial v_{2} \dots \partial v_{d}} |_{v_{1}, v_{2}, \dots, v_{d} = 0} d x \\ = \frac{1}{| 2 π Σ |} \int x_{1} x_{2} \dots x_{d} e^{- \frac{1}{2} x^{⊤} Σ^{- 1} x} d x \\ (6) & \equiv E [x_{1} x_{2} \dots x_{d}] \end{aligned}

Differentiating the left-hand side, we get

\begin{matrix} (7) & \begin{matrix} \frac{\partial e^{\frac{1}{2} v^{⊤} Σ v}}{\partial v_{1} \partial v_{2} \dots \partial v_{d}} |_{v_{1}, v_{2}, \dots, v_{d} = 0} = \frac{\partial (1 + \frac{1}{2} v^{⊤} Σ v + \frac{1}{2!} (\frac{1}{2} v^{⊤} Σ v)^{2}) + \dots}{\partial v_{1} \partial v_{2} \dots \partial v_{d}} |_{v_{1}, v_{2}, \dots, v_{d} = 0} = {\begin{cases} 0, odd d \\ \frac{\partial \frac{1}{(d / 2)!} (\frac{1}{2} v^{⊤} Σ v)^{d / 2}}{\partial v_{1} \partial v_{2} \dots \partial v_{d}} |_{v_{1}, v_{2}, \dots, v_{d} = 0}, even d \end{cases} \end{matrix} \end{matrix}

$d=4$ $d$ $d=4$

\begin{matrix} (8) & \frac{\partial \frac{1}{2!} (\frac{1}{2} v^{⊤} Σ v)^{2}}{\partial v_{1} \partial v_{2} \partial v_{3} \partial v_{4}} = \frac{\partial}{\partial v_{1} \partial v_{2} \partial v_{3} \partial v_{4}} \frac{1}{8} \sum_{i, j, k, l}^{4} v_{i} v_{j} v_{k} v_{l} Σ_{i j} Σ_{k l} \end{matrix}

$v_1 v_2 v_3 v_4$ $v_1, v_2, v_3, v_4$ $4!=24$ $\{v_1, v_2, v_3, v_4\}$ $C_4^2/2=3$ $\mathbf \Sigma_{ij} \mathbf \Sigma_{kl}$ $\mathbf \Sigma_{12} \mathbf \Sigma_{34}, \mathbf \Sigma_{13} \mathbf \Sigma_{24}, \mathbf \Sigma_{14} \mathbf \Sigma_{23}$ $\mathbf \Sigma_{ij} \mathbf \Sigma_{kl}$ $24/3=8$ $(8)$ as

\begin{aligned} \frac{\partial}{\partial v_{1} \partial v_{2} \partial v_{3} \partial v_{4}} \frac{1}{8} \sum_{i, j, k, l}^{4} v_{i} v_{j} v_{k} v_{l} Σ_{i j} Σ_{k l} & = \frac{1}{8} (8 Σ_{12} Σ_{34} + 8 Σ_{13} Σ_{24} + 8 Σ_{14} Σ_{23}) \\ (9) & = Σ_{12} Σ_{34} + Σ_{13} Σ_{24} + Σ_{14} Σ_{23} \end{aligned}

By the definition of the covariance matrix, we finish the proof

\begin{aligned} E [x_{1} x_{2} x_{3} x_{4}] & = Σ_{12} Σ_{34} + Σ_{13} Σ_{24} + Σ_{14} Σ_{23} \\ (10) & = E [x_{1} x_{2}] E [x_{3} x_{4}] + E [x_{1} x_{3}] E [x_{2} x_{4}] + E [x_{1} x_{4}] E [x_{2} x_{3}] \end{aligned}

$d=4$ $d$ $d=6$ $2$ $4$ $4$ .

Comments

Isserlis' theorem is specific to zero-mean Gaussian random variables. It does not extend to other distributions.

$x_1 \perp x_2$ ${\mathrm E}[x_1 x_2] = {\mathrm E}[x_1] {\mathrm E}[x_2]$ $x_1$ $x_2$ follow.