4 Classical Statistical Model
Up to this point, we have acquired many statistical tools, and it is now time to step back and look at the whole picture. First, we collect a set of data points \((x_i, y_i)\). We then select an appropriate statistical model for the data, typically a parameterized one, and find the optimal parameters. Finally, we validate the model and use it to make predictions at new data points. In this section, we will discuss several statistical models. The section is organized as follows:
- 1.
- Regression
- 2.
- Classification
- 3.
- Trees and Weak Learners
- 4.
- Cross-Validation
- 5.
- Time Series
This section mainly follows STAT254 (UC Berkeley), Statistics Learning (PKU, taught by Fang Yao) and STAT153 (UC Berkeley). I would also like to extend my heartfelt thanks to my STAT254 professor, Ryan Giordano, who has offered me countless insights and inspiration in the field of statistics.
4.1.1 Fundamentals
4.1.2 Ridge (\(L_2\)) and Lasso (\(L_1\)) Regression
4.1.3 Kernel Methods
4.2 Classification
4.2.1 Fundamentals
4.2.2 Support Vector Machines
4.3 Trees and Weak Learners
4.3.1 Classification and Regression Trees
4.3.2 Bootstrapping
4.3.3 Bagging
4.3.4 Boosting
4.4 Cross-Validation
4.4.1 Cross-Validation
4.4.2 Influence Function
4.5 Time Series