No. 016
Latest issueDaily research notes
Predicting mixtures, identifying gains
Meta Models · Limits of data curation · Reading LoRA
Daily classic · LoRA (2022) →ZIXUN HUANG / RESEARCH NOTES
Research DigestNotes on how learning works.
Understand mechanisms through equations. Calibrate intuition with experiments.
Issue archive
16 issuesNo. 015
Daily research notes
Norm growth and environment composition
MuonGS · SynthAPI · MoMuSAM · Reading Performer
Daily classic · Performer (2021) →No. 014
Daily research notes
Last iterates and ambiguous extrapolation
Finite-step Muon · Rashomon fits · IV prediction · Lottery Ticket
Daily classic · Lottery Ticket Hypothesis (2019) →No. 013
Daily research notes
Perturbations, state, and sampling paths
SOMA · Scheduler tensor aliasing · Data breadth · DDIM
Daily classic · DDIM (2021) →No. 012
Daily research notes
Which policy generated this trajectory?
Search-agent RL · Mixed-version rollouts · TRPO
Daily classic · TRPO (2015) →No. 011
Daily research notes
Accurate Judges, Useful Teachers
Generative reward models · Rollout consistency · Reading Mamba
Daily classic · Mamba (2024) →No. 010
Daily research notes
Error Geometry and the Teacher's Advantage
PolarGD · AOPD · Safety-probe scaling · Reading GD's max-margin bias
Daily classic · GD max-margin bias (2018) →No. 009
Daily research notes
Timing, Geometry, and the Limits of Gaussianity
Data × learning rate · Muon initialization · Graph tests · Kaplan scaling laws
Daily classic · Kaplan scaling laws (2020) →No. 008
Daily research notes
Step Size, Scale, and Pareto Frontiers
MuPACE · Constant-step Muon · Pareto frontier · InstructGPT
Daily classic · InstructGPT (2022) →No. 007
Daily research notes
Which Compute, Which Data?
Polar Switch · Loop depth · Tabular data scaling · Score-SDE
Daily classic · Score-SDE (2021) →No. 006
Daily research notes
Budgets Shape Geometry; States Compress History
BMuon · Small-scale extrapolation · Async RL · Linear Transformers
Daily classic · Linear Transformers (2020) →No. 005
Daily research notes
Same Policy, Different Trajectories
Consistency in long-horizon RL · Selectors and budgets · Reading NTK
Daily classic · Neural Tangent Kernel (2018) →No. 004
Daily research notes
What to prune, what to preserve
Data pruning · Elastic models · Post-training blogs · Chinchilla close reading
Daily classic · Chinchilla (2022) →No. 003
Daily research notes
Budgets, truncation, and transfer
Frontier pretraining · Muon budget comparisons · STOPD · DPO close reading
Daily classic · DPO (2023) →No. 002
Daily research notes
Stabilizers, precision, and noise
Muon’s asymptotic geometry · Mixed-precision scaling · Spork · DDPM close reading
Daily classic · DDPM (2020) →No. 001
Pilot edition
Noise, Direction, and the Cost of Experiments
Muon's batch-size effects · Active experimental design for scaling laws
Daily classic · S4 (2022) →About these notes
New papers, technical blogs, and substantive updates from selected authors and research areas. Each issue includes essential background, equations, experimental comparisons, and the limits of the evidence. One daily classic covers scaling laws, post-training, deep learning theory, diffusion, or RNN / linear attention.
Subscribe by email →