8.2 Semi-Circle Law and Marchenko–Pastur Law
8.2.1 Trace Calculation
Proof. We present for \(\beta = 1\). By linear algebra, we can choose an orthogonal matrix \(U_1\), and transform the matrix \(M\) to \(U_1MU_1^\top \), whose the first row is \[ (x_{11}, \sqrt {\sum _{k=2}^n x_{ik}^2}, 0,...,0). \] Note that the eigenvalues do not change after the transformation.
We can inductively do the same operation on the rest sub-matrix. □
Now we begin to prove the semi-circle law. First we give the key result of trace calculation.
Proof. By SLLN, we have \begin{align*} &\lim _{N\to \infty } \frac {\chi _{\beta (N-1)}}{\sqrt {2N}} = \sqrt {\frac {\beta }{2}},\\ &\lim _{N\to \infty } \frac {\chi _{\beta (N-\alpha N)}}{\sqrt {2N}} = \sqrt {\frac {\beta (1-\alpha )}{2}}. \end{align*}
So by the definition of the trace, we have \begin{align*} {\operatorname{tr}} \ \left (\frac {T_\beta }{\sqrt {N}}\right )^k &= \sum _{m=1}^N \sum _{\substack {\text {k-step paths} \\ \text {from } m \text { to } m}} \left (\ \textcolor {gray}{ \text {some} \ \frac {\chi _{\beta (N-i)}}{\sqrt {2N}} } \right ) \left (\ \textcolor {gray}{ \text {some} \ \frac {\mathcal {N}(0,1)}{\sqrt {N}} }\right )\\ &=\sum _{m=1}^N \left ( \# {\text {k-step paths from } m \text { to } m} \right ) \sqrt {\frac {\beta }{2}\left ( \frac {N-m}{N} \right )^k} + o(N). \end{align*}
The second equation follows from that if there exists "some \(\frac {\mathcal {N}(0,1)}{\sqrt {N}}\)", the term vanishes. When \(2 \nmid k\), \(\# {\text {k-step paths from } m \text { to } m}\) is zero. When \(2\mid k\), \(\# {\text {k-step paths from } m \text { to } m} = \binom {k}{\frac {k}{2}}\). Approximate the summation by integration and we have \[ \frac {1}{N} {\operatorname{tr}} \ \left (\frac {T_\beta }{\sqrt {N}}\right )^k \to \binom {k}{\frac {k}{2}} \left (\frac {\beta }{2}\right )^{\frac {k}{2}} \int _0^1 x^{\frac {k}{2}}{\mathop{}\!\mathrm{d}} x = {\operatorname{Cat}} _{\frac {k}{2}} \left (\frac {\beta }{2}\right )^{\frac {k}{2}} \] □
Now we introduce the semi-circle distribution.
We use moment method to recover the proof and give the definition.
Proof. (Method I) We just calculate \[ m_k = \int \frac {1}{\sqrt {2\pi }} \sqrt {4-x^2}x^k {\mathop{}\!\mathrm{d}} x. \] Let \(x = 2\cos \theta \), and by inductive calculation, we derive the results. □
However, we are not satisfied for the calculation is not so intuitive. We then introduce another proof.
Proof. (Method II) Introduce generating function: \[ G(z) = \sum _{k=0}^\infty m_k z^{-k-1}, \] \(m_0 =1\). By classical results, we have \[ \sum _{n=0}^\infty {\operatorname{Cat}} _n x^n = \frac {1-\sqrt {1-4x}}{2x} =: C(x). \] Now we can derive the expression of \(G(z)\) using \(C(x)\). \[ G(z) = \sum _{k=0}^\infty m_k z^{-k-1} = \sum _{l=0}^\infty {\operatorname{Cat}} _l z^{-2l-1} = \frac {z-\sqrt {z^2-4}}{2}. \] Next we introduce two propositions.
Proof. By Taylor expansion, we have \begin{align*} \int \frac {\mu (x)}{z-x} {\mathop{}\!\mathrm{d}} x &= \frac {1}{z} \int \frac {\mu (x)}{1-\frac {x}{z}}{\mathop{}\!\mathrm{d}} x\\ &= \frac {1}{z}\int \mu (x)\sum _{k=0}^\infty \left ( \frac {x}{z}\right )^k\\ &=\sum _{k=0}^\infty m_k z^{-k-1} \\ &= G(z) \end{align*} □
Proof. By proposition 8.11, we have \begin{align} -\frac {1}{\pi } \Im (G(x_0+iy_0)) &= -\frac {1}{\pi } \int \Im \frac {1}{x_0+iy_0-x} \mu (x){\mathop{}\!\mathrm{d}} x\\ &= -\frac {1}{\pi } \int \Im \frac {x_0-x-iy_0}{(x-x_0)^2+y_0^2}\mu (x) {\mathop{}\!\mathrm{d}} x\\ &= \int \frac {1}{\pi } \frac {y_0}{(x-x_0)^2+y_0^2}\mu (x) {\mathop{}\!\mathrm{d}} x. \end{align}
Notice that \(\frac {1}{\pi } \frac {y_0}{(x-x_0)^2+y_0^2}\) is a "good" kernel, and thus as \(y_0\to 0\) \[ \int \frac {1}{\pi } \frac {y_0}{(x-x_0)^2+y_0^2}\mu (x) {\mathop{}\!\mathrm{d}} x \to \mu (x_0). \] □
Return to the Theorem 8.10, by proposition 8.12 \[ \mu (x_0) = -\frac {1}{\pi } \lim _{y\to 0^+} \Im \left ( \frac {1}{2}\left [(x+iy) - \sqrt {(x+iy)^2 - 4)}\right ] \right ) = \begin {cases} 0 &, \quad |x|>2;\\ \frac {\sqrt {4-x^2}}{2\pi } &,\quad |x|\le 2. \end {cases} \] □
Now we are prepared to formally state and prove the semi-circle law.
Proof. We prove the result in four steps.
Step 1 The results hold for \(f(x) = x^k\).
Step 2 The results hold for any polynomials.
Step 3 Take \(L>2\), the results hold for \(f(x) = \mathbb {1}_{|x|>L} x^k\). We have \begin{align*} \left |\int f(x)\mu _N({\mathop{}\!\mathrm{d}} x)\right | &= \left |\int x^k\mu _N({\mathop{}\!\mathrm{d}} x)\right |\\ &\le L^{-2m} \int _{|x|\ge L} x^{2k+2m}\mu _N({\mathop{}\!\mathrm{d}} x)\\ &\to L^{-2m} \int x^{2k+2m} \mu ({\mathop{}\!\mathrm{d}} x)\\ &\le L^{-2m} 2^{2k+2m} = 2^{2k} \left (\frac {2}{L}\right )^m \to 0. \end{align*}
Step 4 By step 3, we can restrict the support set of \(f\) on a compact set, i.e. \([-4, 4]\). Apply the Weierstrass theorem and we get the proof. □
At last, we give three generalizations of the semi-circle law. Next theorem tells us the gaussian assumption of the semi-circle law is not neccessary.
8.2.2 Marchenko–Pastur Law
Now we consider the case where the matrix is not square. Let \(X\) be an \(N\times S\) random matrix whose elements are Gaussian with parameter \(\beta > 0\) (different form according to \(\beta \)). Assume that \(N, S\to \infty \) in such a way that \(N/S \to \gamma ^2 \in (0,1)\). Let \(\lambda _1, \lambda _2,\ldots , \lambda _N\) denote the eigenvalues of \(\frac {1}{S}XX^*\), and define the empirical spectral measure \[ \rho ^N = \frac {1}{N}\sum _{i=1}^N \delta _{\lambda _i}. \]
Proof. We calculate the moments for \(\rho ^N\) and \(\mu _{\mathrm {MP}}\) separately
Step 1 We claim that \[ \lim _{N,S\to \infty } \frac {1}{NS^k}{\operatorname{tr}} [(XX^*)^k] = \beta ^k \sum _{r=0}^{k-1} \frac {\gamma ^{2r}}{r+1} \binom {k}{r} \binom {k-1}{r}. \]
Notice that as \(S\to \infty \) \[ \frac {\chi _{aS}}{\sqrt {S}} \to \sqrt {a}, \] we have \begin{align*} \operatorname {LHS} &= \lim _{S,N\to \infty }\frac {1}{NS^k} \sum _{m=1}^N\sum _{\substack {\text {k-step paths} \\ \text {from } m \text { to } m}} (\text {diagonal entries})(\text {sub-diagonal entries})\\ &= \lim _{S,N\to \infty } \frac {1}{NS^k} \sum _{m=1}^N \sum _{i=0}^{[k/2]} \left ( \chi _{\beta (N-m)}\chi _{\beta (S-m+1)} \right )^{2i}\chi _{S+N-2m-2}^{2(k-2i)} \binom {k}{2i}\binom {2i}{i} \end{align*}
By replacing \(\chi \) by its limit, we have \[ \frac {\chi _{\beta (N-m)}\chi _{\beta (S-m+1)}}{S} \to \beta \sqrt {(\gamma ^2-\frac {m}{S})(1-\frac {m}{S})} \] \[ \frac {\chi ^2_{\beta (S+N-2m+2)}}{S} \to \beta (1+\gamma ^2 - 2\frac {m}{S}) \] Then we approximate the sum by integral \begin{align*} \operatorname {LHS} &= \frac {\beta ^k}{\gamma ^2} \sum _{i=0}^{[k/2]} \binom {k}{2i}\binom {2i}{i} \int _0^{\gamma ^2} (\gamma ^2-t)^i(1-t)^i (1+\gamma ^2-2t)^{k-2i}{\mathop{}\!\mathrm{d}} t\\ &=\frac {\beta ^k}{\gamma ^2} \sum _{i=0}^{[k/2]} \binom {k}{2i}\binom {2i}{i} \int _0^{\gamma ^2} u^i(1-\gamma ^2+u)^i (1-\gamma ^2+2u)^{k-2i}{\mathop{}\!\mathrm{d}} u \end{align*}
We claim that \[ \frac {1}{\gamma ^2}\sum _{i=0}^{[k/2]} \binom {k}{2i}\binom {2i}{i} \int _0^{\gamma ^2} u^i(1-\gamma ^2+u)^i (1-\gamma ^2+2u)^{k-2i}{\mathop{}\!\mathrm{d}} u = \sum _{r=0}^{k-1} \frac {\gamma ^{2r}}{r+1} \binom {k}{r} \binom {k-1}{r}. \] and we obtain the proof.
The proof of the claim: Denote the left hand side as A. First, we observe that \begin{equation} \operatorname {A} = \text {The sum of coefficients of all the }x^iy^i \text { in} \int _{0}^{\gamma ^2} [xu+y(1-\gamma ^2 +u)+(1-\gamma ^2+2u)]^k{\mathop{}\!\mathrm{d}} u \end{equation} Denote B as the integral of (25) \begin{align*} B &= \frac {1}{\gamma ^2}\frac {[(x+1)\gamma ^2+(y+1)]^{k+1} - [(y+1)(1-\gamma ^2)]^{k+1}}{(x+y+2)(k+1)}\\ &=\frac {1}{k+1}\sum _{l=0}^k [(y+1)(1-\gamma ^2)]^l [(x+1)\gamma ^2+y+1]^{k-l}. \end{align*}
To calculate the coefficients, we can let \(x = \frac {1}{y}\) and calculate the constant coefficient. \begin{align*} \operatorname {B} &= \frac {1}{k+1} \sum _{l=0}^k [(y+1) (1-\gamma ^2)]^l \left (\left (\frac {1}{y}+1\right )\gamma ^2 + y+1\right )^{k-l}\\ &=\frac {(y+1)^k}{k+1} \sum _{l=0}^k(1-\gamma ^2)^l(\frac {1}{y}\gamma ^2+1)^{k-l}\\ &=\frac {(y+1)^k}{k+1} \frac {(\frac {1}{y}\gamma ^2+1)^{k+1} - (1-\gamma ^2)^{k+1}} {\frac {1}{y}\gamma ^2 + \gamma ^2}\\ &=\frac {y(y+1)^{k-1}}{(k+1)\gamma ^2} \left (\left (\frac {1}{y}\gamma ^2+1\right )^{k+1} - (1-\gamma ^2)^{k+1}\right ) \end{align*}
Then the constant coefficient is \[ \frac {1}{(k+1)\gamma ^2} \sum _{r=0}^{k-1} \binom {k-1}{r} \binom {k+1}{r+1} \gamma ^{2r+2} = \sum _{r=0}^{k-1} \frac {\gamma ^{2r}}{r+1} \binom {k}{r} \binom {k-1}{r}. \]
Step 2 We claim that \(\lambda _{+} = \beta (1+\gamma )^2\) and \(\lambda _{-} = \beta (1-\gamma )^2\).
Define the density of the limit distribution \[ \mu (x) = \frac {1}{2\pi \gamma ^2}\frac {\sqrt {(\lambda _{+}-x)(x-\lambda _{-})}}{x} \mathbb {1}_{[\lambda _{-}, \lambda _{+}]} \] and \(\mu _N = \rho ^N\).
The calculation below is inspired by Zhidong Bai’s book. First, we calculate the moment of \(\mu (x)\), for \(m\ge 1\), \begin{align*} {\mathbb{E}} _{\mu } x^m &= \int x^m \mu (x){\mathop{}\!\mathrm{d}} x\\ &= \frac {1}{2\pi \gamma ^2}\int x^{m-1} \sqrt {(\lambda _{+}-x)(x-\lambda _{-})} {\mathop{}\!\mathrm{d}} x\\ &= \frac {1}{2\pi \gamma ^2}(2\gamma )^2\int _{-1}^1 (1+\gamma ^2+2\gamma u)^{m-1}\sqrt {1-u^2}{\mathop{}\!\mathrm{d}} u \\ &(\text {by setting }x =\beta (1+\gamma ^2+2\gamma u))\\ &= \beta ^m\frac {1}{\pi }\sum _{l=0}^{[(m-1)/2]} \binom {m-1}{2l}(1+\gamma ^2)^{m-1-2l} (4\gamma ^2)^l \int _{-1}^1 u^{2l}\sqrt {1-u^2}{\mathop{}\!\mathrm{d}} u\\ &= \beta ^m \sum _{l=0}^{[(m-1)/2]} \binom {m-1}{2l} \frac {1}{l+1}\binom {2l}{l} (1+\gamma ^2)^{m-1-2l} \gamma ^{2l}\\ &= \beta ^m\sum _{l=0}^{[(m-1)/2]}\sum _{s=0}^{m-1-l} \frac {(m-1)!}{l!(l+1)!s!(m-1-2l-s)!}\gamma ^{2(l+s)}\\ &=\beta ^m \frac {1}{m}\sum _{r=0}^{m-1} \gamma ^{2r} \binom {m}{r} \sum _{l=0}^{r\wedge (m-1-r)}\binom {r}{l} \binom {m-r}{m-r-l-1}\\ &=\beta ^m \frac {1}{m}\sum _{r=0}^{m-1}\binom {m}{r}\binom {m}{r+1}\gamma ^{2r}\\ &=\beta ^m \sum _{r=0}^{m-1}\frac {1}{r+1}\binom {m}{r}\binom {m-1}{r}\gamma ^{2r}. \end{align*}
Next, we notice that \[ {\mathbb{E}} _{\mu _N} x^m =\frac {1}{N} \sum _{i=1}^N \lambda _i^N = \frac {1}{N}{\operatorname{tr}} \left [\left (\frac {XX^*}{S}\right )^m\right ] \] and \[ \lim _{N\to \infty } \frac {1}{N}{\operatorname{tr}} \left [\left (\frac {XX^*}{S}\right )^m\right ] = \beta ^m \sum _{r=0}^{m-1}\frac {1}{r+1}\binom {m}{r}\binom {m-1}{r}\gamma ^{2r}. \] So we have \[ \lim _{N\to \infty }{\mathbb{E}} _{\mu _N} x^m = {\mathbb{E}} _{\mu } x^m. \] Similar to the proof of semi-circle law, we have \(\rho ^N\) converges (weakly, in probability) to the Marchenko-Pastur law. □