Conference 2022
Post-training
LoRA
LoRA: Low-Rank Adaptation of Large Language Models
ICLR 2022 · Important historical work on parameter-efficient adaptation (DBLP checked 2026-10-29) ↗ONE PAPER A DAY
Daily classicReading the foundations.
One earlier work each day: its motivation, background, key equations, and a question to pursue.
Papers read
16 papersConference 2021
RNN / linear attention
Performer
Rethinking Attention with Performers
ICLR 2021 · important historical work in linear attention ↗Conference 2019
Deep learning theory
Lottery Ticket Hypothesis
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
ICLR 2019 Best Paper Award ↗Conference 2021
Diffusion models
DDIM
Denoising Diffusion Implicit Models
ICLR 2021 · Foundational diffusion-sampling work ↗Conference 2015
Post-training
TRPO
Trust Region Policy Optimization
ICML 2015 · foundational historical policy-optimization work ↗Conference 2024
RNN / linear attention
Mamba
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
COLM 2024 Outstanding Paper Award ↗Conference 2018
Deep learning theory
GD max-margin bias
The Implicit Bias of Gradient Descent on Separable Data
JMLR 2018 · Foundational historical work on implicit bias ↗Conference 2020
Scaling laws
Kaplan scaling laws
Scaling Laws for Neural Language Models
Foundational scaling-laws work; OpenAlex record: 1,745 citations (checked 2026-10-22) ↗Conference 2022
Post-training
InstructGPT
Training Language Models to Follow Instructions with Human Feedback
NeurIPS 2022; ACM Digital Library displayed 33,248 citations on 2026-10-21 ↗Conference 2021
Diffusion models
Score-SDE
Score-Based Generative Modeling through Stochastic Differential Equations
ICLR 2021 Outstanding Paper Award ↗Conference 2020
RNN / linear attention
Linear Transformers
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
ICML 2020 · Foundational work in linear attention ↗Conference 2018
Deep learning theory
Neural Tangent Kernel
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
NeurIPS 2018 · foundational deep-learning theory ↗Conference 2022
Scaling laws
Chinchilla
Training Compute-Optimal Large Language Models
NeurIPS 2022 · historical scaling-laws landmark ↗Conference 2023
Post-training
DPO
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
NeurIPS 2023 Oral · Outstanding Main Track Runner-Up ↗Conference 2020
Diffusion models
DDPM
Denoising Diffusion Probabilistic Models
Highly cited · 5,441 citations in the checked OpenAlex record (2026-10-15) ↗Conference 2022
RNN / linear attention
S4
Efficiently Modeling Long Sequences with Structured State Spaces
ICLR 2022 Oral · Outstanding Paper Honorable Mention ↗Priority goes to highly cited papers, best paper awards, and oral presentations, with verifiable selection evidence. Classics are labeled separately from current research.
← Issue archive