Three main readings connect spectral geometry, distillation signals, and label budgets; 11 briefs cover inference allocation, data selection, and technical blogs. The classic derives GD's max-margin direction from logistic regression.
Proposition 2: the error matrix determines preconditioner conditioning
A=(HEE⊤H)−1/2H,Q=AE,A⊤A=(EE⊤)−1,κ(A)=κ(E)
If EE^T=cI, p(E)=1 gives one-step convergence. Even with H=I, nearly rank-deficient errors can make PolarGD much slower than GD. Finite NS amplifies tiny singular values approximately linearly instead of lifting them directly to one.[1]
Evidence
Setting / observation
Quadratic counterexample
H=I, d=10, initial singular values 1 and 10^-4; exact-polar loss ratio 0.64 after its first step; GD finishes in one
Measured spectrum
90M Transformer, 1.8B tokens; roughly 1/4 of attention-momentum singular values lie in the five-step NS linear regime
The latter establishes that this spectral regime exists, without testing end-to-end gains from fewer NS steps.[1]
Main reading · New Wharton-author preprintRead Section 3, Lemma 3.1, external-teacher experiments, and Table 1
AOPD: teacher-preferred tokens may not help the student
Let S be the frozen student rollout policy, T the teacher, R terminal reward, and H trajectory length. Q^S is expected reward after action a followed by student continuations; V^S averages over student actions. Weight teacher supervision by student advantage:[2]
An irrelevant fixed trajectory-length constant is omitted
Separate continuations for every teacher action are unnecessary. Student trajectories and a baseline b independent of that trajectory, such as a leave-one-out mean, estimate the entire objective:[2]
Equation 3.6: unbiased if T>0 implies S>0; S, T, and history distributions are frozen
Yuchen Wu (University of Washington) · Jerry Wei (Anthropic)
First posted 2026-10-21 23:43 EDT (October 22 on arXiv) · v1
Probes read a fixed model's activations to detect safety attributes. Monitored-model size and labels form distinct budget axes. Six harder datasets reduce saturation seen on existing small datasets.[3]
Empirical fits: N is billions of parameters, D labels; CC++ uses g=1, while last-token probes fit g
Maximizing pass@k of retained reasoning prefixes combines quality and redundancy in one objective. Across eight open-weight models it beats quality- or diversity-based heuristics, especially for moderate retained pools; the abstract gives no budget-matched numerical comparison.[4]
BriefBased on the original abstract
R-squared-D: remask-denoise as an inference-scaling axis
A Markov chain over clean sequences motivates inference-only R-squared-D; TCT additionally trains on trajectory errors. Logical tasks show near-log-linear scaling, over 97% on hard Sudoku and over 99% on 3-SAT. These results cannot all be attributed to training-free R-squared-D alone.[5]
BriefBased on the original abstract
Dataweights: transfer learned weighting rules to larger models
Metagradients learn example weights against a small high-quality target set. Parametric weighting rules extend to new or streaming examples and larger models, with reward-modeling experiments on human/synthetic mixtures. The abstract establishes no universal cross-scale transfer guarantee.[6]
BriefBased on the original abstract
Replace unverifiable subset optimality with full-data faithfulness
Ahmed Mehdi Inane · Mahdi Haghifam · Gintare Karolina Dziugaite · Ioannis Mitliagkas
First posted October 21, 16:51 EDT · v1
Population-optimal selection requires a validation set larger in order than training data. Under ULA training, accumulated gradient discrepancy bounds KL between subset- and full-data-trained parameter distributions. Accepted swaps reduce that bound, without guaranteeing monotonic improvements in population error.[7]
BriefBased on the original abstract
Width-depth training limits: clipping changes the convergence mode
For linear ResNets with 1/sqrt(depth) Depth-muP scaling and muP width parameterization, any fixed clipped-GD horizon has the same L2 limit along all joint growth paths. Without clipping, only convergence in probability is established; nonlinear networks and long horizons are outside this result.[8]
BriefBased on the original abstract
A graph-search scaling wall can depend on supervision and tokenization
Arvid Frydenlund · FuTe Wong · Katherine Lambert · Frank Rudzicz
First posted October 22, 11:35 EDT · v1
Multi-token prediction and alternative tokenizations address separate barriers to shortest-path generation on unseen graphs. Authors report learning at larger graph scales than prior limits. This controlled latent graph-search result is not a general law of natural-language reasoning.[9]
BriefBased on the original abstract
One matched spectral statistic does not predict synthetic-data transfer
Edric Castel C. Hao · Juan Raphael S. Seña · Neon S. De Jesus
First posted October 22, 02:27 EDT · v1
Across LSTM, PatchTST, and TCN with three seeds, Omega-matched sine-plus-noise pretraining improves mean ETTm1 MSE by 2.3%, whereas matched chaotic-ODE sources worsen it by 13.2%. The negative result concerns source selection, not synthetic pretraining in general.[10]
BriefBased on the original abstract
Before routing inference, compare the predictor with self-confidence
A predictor's advantage over chosen-answer confidence transfers exactly to log-loss improvement through recalibration. Across 16 models and four benchmarks, predictor-based prediction and cascading gains shrink after finetuning. Predictor capacity must be counted in the comparison.[11]
BriefRead training, task-selection, and evaluation sections of the original blog
ThinkingBox: repeatable success rises while coverage slips
Tuhin Kundu · Microsoft and Hugging Face collaborators
Blog published October 22
After 50 GRPO steps on a separate training pool, Qwen3.8-27B is tested 20 times on each of 507 tasks. Pass@1 rises 51.70% to 60.78%; observed 20/20 tasks rise 38 to 107, but pass@20 falls 89.35% to 87.97%. Observed 20/20 is not a deployment-reliability lower bound.[12]
BriefRead original-blog methods and token accounting; the older paper is not counted as a new preprint
JOLT: train the teacher to meet the student's capabilities
New blog October 21; paper arXiv:2610.10447 v1 first posted October 7
One model jointly learns as a privileged teacher and unprivileged student. Teacher rewards are regularized toward the student; JOLT+ adds student rewards. At 63.44% MATH accuracy, GRPO's observed 8.40M processed tokens are compared with JOLT's fitted 1.30M. The 6.5-fold token comparison is not wall-clock acceleration.[13]
BriefRead blog experimental settings, update logs, and cost discussion
OPAL: rewrite Muon / AdamW recipes during a training run
Xingjian Bai · Jia-Hao Liang · Jayden Teoh · Vincent Sitzmann
Page datePublished / dateModified both October 21
An agent rewrites groupwise update code from training statistics, including Muon switches, momentum changes, and loss-spike guards. Multilingual NanoGPT and diffusion experiments compare available one-run recipes; RAEv2-B FID improves 3.39 to 3.19. Agent tokens and decision overhead must be counted; uniform compute superiority is not established.[14]
Daily classic · Historical workRead the setting, Theorem 3, proof sketch, and degenerate-data discussion; no multiclass result is used
Hard-margin SVM explicitly selects small norm; separable logistic regression has no finite minimizer. Soudry et al. (2018) connect them: GD's norm diverges while its direction is selected.[15][16]
Finite, separable data; homogeneous linear predictor; X has columns xi; summed logistic loss has beta=1/4
This is the paper's asymptotic intuition, not a convergence proof. Assuming a limiting direction u, exponential tails suppress larger-margin examples, leaving support vectors.[15]
Logistic derivative's exponential tail; the ratio compares dominance along a fixed direction