Let \(\pmb{X}\sim N_n(\pmb{\mu}, \pmb{\Sigma})\), then
\[(\pmb{X}-\pmb{\mu})'\pmb{\Sigma}^{-1}(\pmb{X}-\pmb{\mu})\sim \chi^2(n)\]
proof
\[ \begin{aligned} &(\pmb{X}-\pmb{\mu})'\pmb{\Sigma}^{-1}(\pmb{X}-\pmb{\mu}) = \\ &(\pmb{X}-\pmb{\mu})'\pmb{\Sigma}^{-1/2}\pmb{\Sigma}^{-1/2}(\pmb{X}-\pmb{\mu}) = \\ &\left[\pmb{\Sigma}^{-1/2}(\pmb{X}-\pmb{\mu}) \right]'\left[\pmb{\Sigma}^{-1/2}(\pmb{X}-\pmb{\mu}) \right] =\\ &\pmb{Z}'\pmb{Z} \\ \end{aligned} \]
where
\[\pmb{\Sigma}^{-1/2}(\pmb{X}-\pmb{\mu}) = \pmb{Z}\sim N_n(\pmb{0}, \pmb{I})\]
because of (5.2.2)
Let \(\pmb{X}\sim N_n(\pmb{\mu}, \pmb{\Sigma})\), let \(\pmb{A}\) be a symmetric matrix of constants with rank r, and let \(\lambda=\frac12\pmb{\mu'A\mu}\), then
\[\pmb{X}'\pmb{A}\pmb{X}\sim \chi^2(r, \lambda)\] iff \(\pmb{A\Sigma}\) is idempotent.
proof
By (5.3.7) the moment generating function of \(\pmb{X}'\pmb{A}\pmb{X}\) is given by
\[\psi(t)=\vert \pmb{I}-2t\pmb{A}\pmb{\Sigma} \vert^{-1/2}\exp \left\{ - \pmb{\mu}' \left[ \pmb{I}- (\pmb{I}-2t\pmb{A}\pmb{\Sigma})^{-1} \right] \pmb{\Sigma}^{-1} \pmb{\mu}/2 \right\}\]
The eigenvalues of \(\pmb{I}-2t\pmb{A}\pmb{\Sigma}\) are \(1-2t\lambda_i\), i=1,..,p, where \(\lambda_i\) are the eigenvalues of \(\pmb{A}\pmb{\Sigma}\). In the proof of (5.3.8) we showed that
\[\vert \pmb{I}-2t\pmb{A}\pmb{\Sigma} \vert=\prod_{i=1}^p (1-2t\lambda_i)\]
and that
\[(\pmb{I}-2t\pmb{A}\pmb{\Sigma})^{-1} = \pmb{I} + \sum_{i=1}^\infty (2t)^i(\pmb{A}\pmb{\Sigma})^i\]
if \(|2t\lambda_i|<1\) for all i. Therefore
\[\psi(t)=\prod_{i=1}^p (1-2t\lambda_i)^{-1/2}\exp \left\{ - \pmb{\mu}'\left[-\sum_{i=1}^\infty (2t)^i(\pmb{A}\pmb{\Sigma})^i \right]\pmb{\Sigma}^{-1}\pmb{\mu}/2\right\}\]
Suppose that \(\pmb{A\Sigma}\) is idempotent with rank r, then r of the \(\lambda_i's\) are equal to 1 and n-r are equal to 0, and \((\pmb{A\Sigma})^i=\pmb{A\Sigma}\), so
\[\psi(t)=(1-2t)^{-r/2}\exp \left\{ - \pmb{\mu}'\left[-\sum_{i=1}^\infty (2t)^i \right]\pmb{A}\pmb{\Sigma}\pmb{\Sigma}^{-1}\pmb{\mu}/2\right\}=\]
\[(1-2t)^{-r/2}\exp \left\{ -\pmb{\mu}' \left[1-\sum_{i=0}^\infty (2t)^i \right]\pmb{A}\pmb{\mu}/2\right\}=\]
\[(1-2t)^{-r/2}\exp \left\{ -\frac12 \pmb{\mu}' \pmb{A}\pmb{\mu}[1-1/(1-2t)]\right\}=\]
if(\(\vert 2 t\vert<1\). This is the moment generating function of a chi-square with r degrees of freedom and non-centrality parameter \(\frac12\pmb{\mu'A\mu}\).
The proof of the reverse is omitted.
If \(\pmb{X}\sim N_p(\pmb{0}, \pmb{I})\), then \(\pmb{X}'\pmb{A}\pmb{X}\sim \chi^2(r)\) iff \(\pmb{A}\) is symmetric and idempotent of rank r.
If \(\pmb{X}\sim N_p(\pmb{\mu}, \sigma^2\pmb{I})\), then \(\pmb{X}'\pmb{A}\pmb{X}/\sigma^2\sim \chi^2(r, \pmb{\mu'A\mu}/(2\sigma^2))\) iff \(\pmb{A}\) is symmetric and idempotent of rank r.
Let \(X,Y\sim N(0,1)\) and independent. We want to find \(P\left((X+Y)^2<3\right)\).
Easy: \(X+Y\sim N(0,\sqrt 2)\) and so
\[P\left((X+Y)^2<3\right)=P\left(\vert X+Y\vert <\sqrt3\right)\]
round(2*pnorm(sqrt(3), 0, sqrt(2))-1, 4)## [1] 0.7793A little more complicated we have
\[[(X+Y)/\sqrt 2]^2\sim \chi(1)\]
and so
\[P\left((X+Y)^2<3\right)=P\left((X+Y)^2/2<3/2\right)\]
round(pchisq(3/2, 1), 4)## [1] 0.7793Let’s also check with R:
B=1e4
x=rnorm(B)
y=rnorm(B)
u=(x+y)^2
round(sum(u<3)/B, 4)## [1] 0.7772but note that if \(\pmb{A} =\frac12 \begin{pmatrix} 1&1 \\ 1&1 \end{pmatrix}\), then
\[ \begin{aligned} &\pmb{X'AX} = \begin{pmatrix} x & y \end{pmatrix} \frac12 \begin{pmatrix} 1&1 \\ 1&1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}=(x+y)^2/2\\ &(x+y)^2=2\pmb{X'AX}\\ &P\left((X+Y)^2<3\right)=P(2\pmb{X'AX}<3)=P(\pmb{X'AX}<3/2) \end{aligned} \] also \(\pmb{A}\) is symmetric and idempotent, so by the corrolary part i \(\pmb{X'AX}\sim \chi^2(1)\).
Now say \(X,Y\sim N(1,1)\), independent, and again we want to find \(P\left((X+Y)^2<3\right)\). Now \(X+Y\sim N(2, \sqrt 2)\), and so
\[P\left((X+Y)^2<3\right)=P\left(-\sqrt3< X+Y<\sqrt3\right)\]
round(pnorm(sqrt(3), 2, sqrt(2))-pnorm(-sqrt(3), 2, sqrt(2)), 4)## [1] 0.4207Also according to the corrolary part ii now \(\pmb{X'AX}\sim \chi^2(1,\lambda)\) where
\[\lambda=\frac12 \pmb{\mu'A'\mu}=\frac12(1+1)^2/2=1\]
and so
\[P\left((X+Y)^2<3\right)=P\left(\pmb{X'AX}<3/2\right)\]
round(pchisq(3/2, 1, 2), 4)## [1] 0.4207Let’s also check with R:
B=1e5
x=rnorm(B, 1)
y=rnorm(B, 1)
u=(x+y)^2
round(sum(u<3)/B, 4)## [1] 0.4222Say X and Y are two normal random variables. Is it possible that XY has a chi-square distribution?
Say \(X\sim N(\mu_x,\sigma_x^2)\), \(Y\sim N(\mu_y,\sigma_y^2)\) and \(cor(X,Y)=\rho\). Let
\[ \pmb{A} = \begin{pmatrix} a & b \\ b & c\\ \end{pmatrix} \]
then \(\pmb{x'Ax}=ax^2+2bxy+cy^2\). So we need \(a=c=0,b=\frac12\). Now
\[\pmb{A\Sigma} = \begin{pmatrix} 0&\frac12 \\ \frac12 & 0 \end{pmatrix} \begin{pmatrix} \sigma_x^2&\sigma_x\sigma_y\rho \\ \sigma_x\sigma_y\rho & \sigma_y^2 \end{pmatrix}= \begin{pmatrix} \sigma_x\sigma_y\rho&\sigma_y^2 \\ \sigma_x^2&\sigma_x\sigma_y\rho \end{pmatrix}/2\]
By theorem 5.5.2 XY has a chi-square distribution if and only if \(\pmb{A\Sigma}\) is idempotent. So
\[ \begin{aligned} &(\pmb{A\Sigma})^2 = \\ &\begin{pmatrix} \sigma_x\sigma_y\rho&\sigma_y^2 \\ \sigma_x^2&\sigma_x\sigma_y\rho \end{pmatrix}/2 \begin{pmatrix} \sigma_x\sigma_y\rho&\sigma_y^2 \\ \sigma_x^2&\sigma_x\sigma_y\rho \end{pmatrix}/2 =\\ &\frac14\begin{pmatrix} (\sigma_x\sigma_y\rho)^2+\sigma_x^2\sigma_y^2&2\sigma_x\sigma_y^3\rho \\ 2\sigma_x^3\sigma_y\rho&(\sigma_x\sigma_y\rho)^2+\sigma_x^2\sigma_y^2 \end{pmatrix} = \\ &\frac14\begin{pmatrix} \sigma_x^2\sigma_y^2(1+\rho^2)&2\sigma_x\sigma_y^3\rho \\ 2\sigma_x^3\sigma_y\rho&\sigma_x^2\sigma_y^2(1+\rho^2) \end{pmatrix} \end{aligned} \] and so we would need
\[ \begin{aligned} &\sigma_x^2\sigma_y^2(1+\rho^2) =4\sigma_x\sigma_y\rho\\ &\sigma_x\sigma_y^3\rho=4\sigma_y^2\\ &\sigma_x^3\sigma_y\rho=4\sigma_x^2\\ &\sigma_x^2\sigma_y^2(1+\rho^2) =4\sigma_x\sigma_y\rho\\ \end{aligned} \]
or
\[ \begin{aligned} &\sigma_x\sigma_y(1+\rho^2) =4\rho\\ &\sigma_x\sigma_y\rho=4\\ &4\rho=\sigma_x\sigma_y+\sigma_x\sigma_y\rho^2 =\sigma_x\sigma_y+4\rho\\ &\sigma_x\sigma_y=0 \end{aligned} \]
and that implies that either \(\sigma_x\) or \(\sigma_y\) is zero, which is impossible.
Of course there is another, much simpler argument!
Say \(\pmb{X} = (X_1,..,X_n)'\) where \(X_i\sim N(\mu,\sigma^2)\) and independent. Recall that we can write \(\pmb{X}\sim N_n(\mu \pmb{j}, \sigma^2\pmb{I})\) and that \(\sum_{i=1}^n (x_i-\bar{x})^2 = \pmb{x}'[\pmb{I}-(1/n)\pmb{J}]\pmb{x}\). From theorem (5.3.2) we know that \(\pmb{I}-(1/n)\pmb{J}\) is idempotent. Also
\[rank(\pmb{I}-(1/n)\pmb{J}) = tr(\pmb{I}-(1/n)\pmb{J})=n(1-1/n) = n-1\].
Now
\[ \begin{aligned} &\lambda = \pmb{\mu'A\mu}/(2\sigma^2) = \\ &\mu\pmb{j}'(\pmb{I}-(1/n)\pmb{J})\mu\pmb{j}/(2\sigma^2) = \\ &\mu^2(\pmb{j}'\pmb{j}-(1/n)\pmb{j}'\pmb{J}\pmb{j}/(2\sigma^2) = \\ &\mu^2(\pmb{j}'\pmb{j}-(1/n)\pmb{j}'\pmb{j}\pmb{j}'\pmb{j}/(2\sigma^2) = \\ &\mu^2(n-(1/n)n^2)/(2\sigma^2) =0 \end{aligned} \]
and so
\[(n-1)s^2/\sigma^2 = \pmb{x}'[\pmb{I}-(1/n)\pmb{J}]\pmb{x}/\sigma^2\sim \chi^2(n)\]
Suppose \(\pmb{B}\) is a \(k\times p\) matrix of constants, \(\pmb{A}\) is a \(p\times p\) symmetric matrix of constants and \(\pmb{X}\sim N_p(\pmb{\mu}, \pmb{\Sigma})\). Then \(\pmb{BX}\) and \(\pmb{X'AX}\) are independent iff \(\pmb{B\Sigma A}=\pmb{O}\).
proof
Suppose \(\pmb{B\Sigma A}=\pmb{O}\). We will also assume that \(\pmb{A}\) is idempotent. The general case can be found in textbooks.
We can write
\[\pmb{X'AX} =\pmb{X'A'AX} = \pmb{(AX)'(AX)}\]
If \(\pmb{B\Sigma A}=\pmb{O}\) we have \(cov(\pmb{BX, AX})=0\). Because they are multivariate normal random vectors, uncorrelated implies independence.
For the reverse assume \(\pmb{BX}\) and \(\pmb{X'AX}\) are independent. Therefore \(cov(\pmb{BX,X'AX})=\pmb{0}\) and so
\[2\pmb{B\Sigma A\mu}=0\]
this holds for all \(\pmb{\mu}\) and so \(\pmb{B\Sigma A}=0\)
\(\pmb{X}\sim N_p(\pmb{\mu}, \sigma^2\pmb{I})\). Then \(\pmb{BX}\) and \(\pmb{X'AX}\) are independent iff \(\pmb{BA}=\pmb{O}\).
Say \(\pmb{X}\sim N_n(\mu\pmb{j}, \sigma^2\pmb{I})\). We can write
\[ \begin{aligned} &\bar{X} = (1/n)\pmb{j}'\pmb{X} \\ &s^2 = \pmb{X}'[\pmb{I}-(1/n)\pmb{J}]\pmb{X}/(n-1) \end{aligned} \]
now
\[ \begin{aligned} &(1/n)\pmb{j}'[\pmb{I}-(1/n)\pmb{J}] = \\ &(1/n)\pmb{j}'-(1/n)(1/n)\pmb{j}'\pmb{J} = \\ &(1/n)\pmb{j}'-(1/n)(1/n)n\pmb{j}' = 0\\ \end{aligned} \]
and so \(\bar{X}\) and \(s^2\) are independent.
Suppose \(\pmb{A}\) and \(\pmb{B}\) are symmetric matrices of constants and \(\pmb{X}\sim N_p(\pmb{\mu}, \pmb{\Sigma})\). Then \(\pmb{X'AX}\) and \(\pmb{X'BX}\) are independent iff \(\pmb{A\Sigma B}=\pmb{O}\).
proof
Suppose \(\pmb{A\Sigma B}=\pmb{O}\). Again we will also assume that \(\pmb{A}\) and \(\pmb{B}\) are idempotent, so we can write \(\pmb{X'AX}=\pmb{(AX)'AX}\) and \(\pmb{X'BX}=\pmb{(BX)'BX}\). \(\pmb{A\Sigma B}=\pmb{O}\) we have \(cov(\pmb{AX, BX})=\pmb{A\Sigma B}=\pmb{O}\). Because they are multivariate normal random vectors, uncorrelated implies independence and vice versa.
We have previously partitioned
\[\sum_{i=1}^n x_i^2 = \sum_{i=1}^n (x_i-\bar{x})^2 + n \bar{x}^2\]
(see (5.3.1)), which we can write as
\[\pmb{x}'\pmb{x} = \pmb{x}'[\pmb{I}-(1/n)\pmb{J}]\pmb{x}+\pmb{x}'(1/n)\pmb{J}\pmb{x}\]
now \(\pmb{x}'[\pmb{I}-(1/n)\pmb{J}]\pmb{x}\) and \(\pmb{x}'(1/n)\pmb{J}\pmb{x}\) are independent iff \([\pmb{I}-(1/n)\pmb{J}][(1/n)\pmb{J}]=\pmb{0}\), which is easy to show.
(James’ Theorem)
Let \(\pmb{X}\sim N_n(\pmb{\mu}, \sigma^2\pmb{I})\), let \(\pmb{A}_i\) be symmetric with rank \(r_i\), i=1,..,k. Set \(\pmb{A}=\sum_{i=1}^k \pmb{A}_i\) and assume \(\pmb{A}\) is symmetric of rank r. Then
i-iii are true iff at least two of the following are true
or if c and d are true, where
proof omitted