Proof. \begin{align*} ES_n^4 &=\sum _i \sum _j \sum _k \sum _l E[X_i X_j X_k X_l]\\ &= nEX^4 + \binom {4}{2}\binom {n}{2} E[X_1^2 X_2^2] \\ &= nEX^4 + 3n(n-1) \underbrace {(EX^2)^2}_{\leq EX^4} \end{align*}

since \((EY)^2 \leq E(Y^2)\). Fix \(\varepsilon > 0\). \begin{align*} P\left (\left |\frac {S_n}{n}\right | \geq \varepsilon \right ) &\leq E\left |\frac {S_n}{n}\right |^4 \cdot \frac {1}{\varepsilon ^4} \\ &\leq \varepsilon ^{-4} n^{-4} \cdot 3n^2 EX^4 \\ &\leq 3\varepsilon ^{-4} EX^4 n^{-2} \end{align*}

This implies that \[ \sum _n P\left (\left |\frac {S_n}{n}\right | \geq \varepsilon \right ) \leq \sum _n 3\varepsilon ^{-4} EX^4 n^{-2} < \infty \] By Borel-Cantelli Lemma 1.19, \(S_n/n \to 0\) a.s. We used the fact that \(s^4 = |s|^4\) and \(s^2 = |s|^2\), but this does not work for the third moment: \(s^3 \neq |s|^3\). □

Proof. Since \(\operatorname {var}(S_n) \leq nB\), Chebyshev’s inequality implies \[ P\left (\frac {|S_n|}{n} \geq \varepsilon \right ) \leq \frac {nB}{n^2 \varepsilon ^2} = \frac {B}{n\varepsilon ^2} \] Take \(n(j) = j^2\). \[ P\left (\left |\frac {S_{n(j)}}{n(j)}\right | \geq \varepsilon \right ) \leq \frac {B}{\varepsilon ^2} \frac {1}{j^2} \] Use Borel-Cantelli 1.19. \[ \frac {S_{n(j)}}{n(j)} \to 0 \quad \text {a.s.} \quad \text {as } j \to \infty \] It is enough to prove \(D_j / j^2 \to 0\) a.s., for \[ D_j = \max _{j^2 \leq n < (j+1)^2} |S_n - S_{j^2}| \] Then \[ D_j^2 = \max _{j^2 \leq n < (j+1)^2} (S_n - S_{j^2})^2 \] \[ ED_j^2 \underset {\text {crude}}{\leq } \sum _{n=j^2}^{(j+1)^2 - 1} E(S_n - S_{j^2})^2 \] Since \[ E(S_n - S_{j^2})^2 = \operatorname {var}\left (\sum _{j^2+1}^{n} X_i\right ) \leq B(n - j^2) \] Letting \(n = j^2 + i\), we have \[ ED_j^2 \leq B \sum _{i=1}^{2j+1} i = \frac {1}{2}(2j+1)(2j+2)B \] We have \[ P\left (\frac {D_j}{j^2} \geq \varepsilon \right ) \leq \frac {ED_j^2}{\varepsilon ^2 j^4} \in O(j^{-2}) \] Borel-Cantelli Lemma 1.19 implies that \(D_j / j^2 \to 0\) as \(j \to \infty \). □

Proof. Define \(M_k = \sup _{n > k} \left |\sum _{i=k+1}^{n} X_i\right |\). It is enough to show that \(M_k \to 0\) a.s. as \(k \to \infty \). Define also \(W_k = \sup _{n_2 > n_1 > k} \left |\sum _{i=n_1+1}^{n_2} X_i\right |\) and note that \(M_k \leq W_k \leq 2M_k\) and \(W_k\) decreases as \(k\) increases. \[ P\left (\sup _{k < n \leq N} \left |\sum _{i=k+1}^{N} X_i\right | \geq \varepsilon \right ) \underset {martingale \ maximal \ ineq.}{\leq } \varepsilon ^{-2} \operatorname {var}\left (\sum _{i=k+1}^{N} X_i\right ) = \varepsilon ^{-2} \sum _{i=k+1}^{N} \sigma _i^2 \]

Taking \(N \to \infty \), \(P(M_k > \varepsilon ) \leq \varepsilon ^{-2} \sum _{i=k+1}^{\infty } \sigma _i^2\). \[ P(W_k > \varepsilon ) \leq P\left (M_k > \frac {\varepsilon }{2}\right ) \leq 4\varepsilon ^{-2} \sum _{i=k+1}^{\infty } \sigma _i^2 \to 0 \text { as } k \to \infty \]

Taking \(k \to \infty \), then \(W_k \downarrow W_\infty \) for some \(W_\infty \) a.s. Then \(P(W_\infty > \varepsilon ) = 0\), which implies that \(W_\infty = 0\) a.s., which implies that \(W_k \downarrow 0\) a.s. and \(M_k \to 0\) a.s. □

Proof. Theorem 1.29 implies that \(\sum _n X_n / a_n\) converges a.s. Then Lemma 1.30 implies that \(S_n / a_n \to 0\) a.s. □

Proof of Theorem 1.28. The idea is to truncate, center, and then apply 9.4.

If \(Z \geq 0\), then \[ EZ^k = \int _0^{\infty } kz^{k-1} P(Z \geq z) \, dz. \]

Define \(Y_k = X_k 1_{(|X_k| \leq k)}\). Then \[ \sum _k P(Y_k \neq X_k) = \sum _{k=1}^{\infty } P(|X| > k) \leq \int _0^{\infty } P(|X| > x) \, dx = E|X| < \infty . \]

Then Borel-Cantelli Lemma 1.19 implies that \(P(Y_k = X_k, \text { ultimately}) = 1\). It is enough to prove that \((1/n) \sum _{k=1}^{n} Y_k \to EX\) a.s.

Center: define \(X_k' = Y_k - EY_k\). Claim: \(\sum _k \operatorname {var}(X_k')/k^2 < \infty \). \begin{align*} EY_k^2 &= \int _0^{\infty } 2y P(|Y_k| > y) \, dy = \int _0^{\infty } \underbrace {2y P(k \geq |X_k| \geq y) 1_{(y \leq k)}}_{\text {Check this!}} \, dy \\ &\leq \int _0^{\infty } 2y P(|X_k| \geq y) 1_{(y \leq k)} \, dy \end{align*}

Claim: \(G(y) \leq 4\), for all \(0 < y < \infty \). Since \(G(y) \leq \sum _k 1/k^2 \leq 2\) for \(y \leq 1\), this is true for \(y \leq 1\). Take \(y > 1\). \[ \frac {1}{k^2} \leq \int _{k-1}^{k} \frac {1}{x^2} \, dx \] so \[ \sum _k \frac {1}{k^2} 1_{(y \leq k)} = \sum _{k \geq \lceil y \rceil } \frac {1}{k^2} \leq \int _{\lceil y \rceil - 1}^{\infty } \frac {1}{x^2} \, dx = \frac {1}{\lceil y \rceil - 1} \]

Since \(y > 1\), \[ G(y) \leq \frac {2y}{\lceil y \rceil - 1} \leq 4. \] Then \[ \sum _k \frac {\operatorname {var}(X_k')}{k^2} \leq 4 \int _0^{\infty } P(|X| \geq y) \, dy = 4E|X| < \infty \]

Apply 1.31 to \((X_n')\): \((1/n) \sum _{i=1}^{n} X_i' \to 0\) a.s., so \((1/n) \sum _{i=1}^{n} (Y_i - EY_i) \to 0\) a.s. Note that \[ EY_i = EX 1_{(|X| \leq i)} \to EX \] as \(i \to \infty \). By dominated convergence, \((1/n) \sum _{i=1}^{n} (EY_i - EX) \to 0\) a.s. Add the two equations to get \((1/n) \sum _{i=1}^{n} (Y_i - EX) \to 0\) a.s., which implies that \((1/n) \sum _{i=1}^{n} Y_i \to EX\) a.s. □

Proof. WLOG take \(\mu = 0\). It is enough to show \[ \underbrace {\phi _{S_n/\sqrt {n}}(t)}_{\text {Left}} \to \exp \left (-\frac {\sigma ^2 t^2}{2}\right ). \]

Also, \[ \phi _{S_n/\sqrt {n}}(t) = \left (\phi _X\left (\frac {t}{\sqrt {n}}\right )\right )^n = \left (1 + \frac {n(\phi _X(t/\sqrt {n}) - 1)}{n}\right )^n. \]

It is enough to show \(n(\phi _X(t/\sqrt {n}) - 1) \to \sigma ^2 t^2 / 2\). The bound for \(n = 2\) and \(EX = 0\) is \[ \left |\phi _X(s) - \left (1 - \frac {s^2 \sigma ^2}{2}\right )\right | = o(s^2). \]

Then, with \(s = t/\sqrt {n}\), \[ \text {Left} = n\left (\frac {t^2}{n}\frac {\sigma ^2}{2} + o\left (\frac {t^2}{n}\right )\right ) = \frac {t^2 \sigma ^2}{2} + n \cdot o\left (\frac {t^2}{n}\right ) \to \frac {t^2 \sigma ^2}{2}. {\qquad\square} \] □