Two close readings separate static judging accuracy, online learning outcomes, and implementation mismatch; seven briefs cover compute, data, and models from the overlap window. The classic is Mamba: choosing what to write and computing it in parallel.
Scaling lawsMuonPost-trainingRNN / linear attention
First posted October 22, 01:37 EDT · v1; overlap-window catch-up
Following reward overoptimization (Gao et al., 2022), correctly judging story preferences on a fixed distribution need not guide an evolving policy. The paper compares SFT Answer-Only and GRPO Thinking judge pipelines in creative writing.[1]
Fixed Gold-oriented response pair; + uses the Gold label and − its flip
Appendix B.3 identity: which pairs are mislabeled, their gradient directions, and magnitudes matter
∇θℓ−−∇θℓ+=β∇θΔθ
Online sampling and filtering invalid or conflicting judge votes also change the retained-pair distribution. The identity is an explanation; the paper does not measure a per-pair gradient decomposition.[1]
Setting
Observation
Qwen3 judges 0.6B–14B; policies 0.6B–8B
Thinking has higher static accuracy
Fixed learned 32B Gold evaluator
Answer-Only has higher peak/top-five Elo in 7 of 8 matched settings
Another learned judge, 80 pairs
Answer-Only wins 42; inconclusive overall
The ranking reversal is conditional on these training and evaluation pipelines.[1]
Descriptive scaling fit: P and R are billions of parameters; M indicates Thinking
Y=b0+bPlogP+bRlogR+bMM+bMPMlogP+bMRMlogR+ϵ
Close readingRead Day-0 RL support and adjacent implementation details; not reproduced
Low-precision RL: three checks before treating implementations as one policy
The training section combines an NVFP4 checkpoint with veRL, FSDP2, and LoRA, freezing base experts and syncing adapters. Before training it checks weight transfer, expert-routing replay, and same-token logprobs: equal weights can still yield different execution.[2]
Explanatory notation for token-level IS; the source specifies a ratio cap of 2
Each batch receives one optimizer step, using rollout logprobs as old logprobs and skipping a recomputation forward pass. This condition does not directly transfer to multiple reuse epochs or more stale training.[2]
Setup / metric
Start → end
DAPO-Math-17K; 16×GB300
200 GRPO steps
AIME-2024 accuracy
36.3% → 67.5%
Train/rollout KL
0.040 → 0.024
Original blog report; no separate ablation of these consistency measures is provided here.[2]
BriefRead abstract and pretraining overview; not all evaluations audited
Muon, μP, and data scheduling in a 320B MoE recipe
320B total / 15B active parameters; pretraining combines Stable SplitNorm, Muon, and μP, increasing the coding mixture later. The report emphasizes a margin between safe and optimal learning rates; it does not isolate Muon’s causal contribution.[3]
BriefBased on the original abstract
Joint scaling of continuous semantics and discrete generation
Hongcan Guo · Yian Zhao · Feng Wang · Shen Nie · Guoqiang Wei · Yan Zeng · Hengshuang Zhao
October 22, 10:04 EDT · v1
Dynamic semantic segmentation lets token prediction update the encoder while stop-gradient decouples the flow prior. Experiments at about 210B tokens examine design and scaling; an unconverged model with 346.03B source tokens averages 60.10 over eight tasks. Cross-model token differences do not establish a compute-matched advantage.[4]
BriefBased on the original abstract
Data sources: control priors without robot trajectories
Three procedural control abstractions train a Transformer with nontrivial zero-shot action prediction on unseen robot tasks. A compute-matched real-data baseline is stronger only on its training datasets. This compares data sources and transfer, not a universal robot scaling law.[5]
BriefBased on the original abstract
Data quality and trainable parameters at fixed token budgets
At a matched 1B tokens, curated data improves the four-task mean by 3.51 points over raw web. At 5B tokens, partial-parameter training recovers 94% of full-CPT gain in 52% of wall time; means are 73.19 vs 73.58. Recovery is the incremental gain over base, not retention of 94% of all capabilities.[6]
BriefBased on the original abstract
VANE: next-frame prediction in a fixed semantic space
With a frozen semantic tokenizer, a causal encoder predicts the next full-frame embedding and an auxiliary decoder preserves detail. The paper reports model/data scaling and a 7.1-point gain over V-JEPA 2.1 in EPIC-KITCHENS-100 anticipation mean-class recall@5; strict budget matching is not specified in the abstract.[7]
BriefBased on the original abstract
Scene coverage and data scaling for closed-loop navigation
94 reconstructed scenes, 94K demonstrations, and 46.5M observation-state-action samples. Scaling ACT training from 5 to 70 scenes raises seen/unseen median success from 30.6% to 41.2% and 16.8% to 29.4%. Offline prediction error is not interchangeable with closed-loop success.[8]
BriefAbstract and version dates verified; substantive version differences unverified
Author watch: collaborative compute in software worlds
Multi-agent collaboration is evaluated through hidden downstream efficiency in dependent Python packages: three worlds, eight models, and over 10,000 agent-hours, with reported gains up to 31%. Included for Luke author tracking; these conclusions are not claimed as additions in v2.[9]
Daily classic · Historical workRead §2, §§3.1–3.5, Theorem 1, and §4.1 task setups; kernel benchmarks not reproduced
S4 (2022) compresses history with a fixed linear system; linear attention (2020) accumulates a finite state. Which language tokens deserve memory depends on content. Mamba (2024) makes writing, reading, and forgetting input-dependent.[10]
Single-channel notation: N-dimensional state and fixed diagonal A; the layer runs across D channels
A small gate retains state and ignores the current input; a large gate resets toward it. This scalar interpretation is not a convergence theorem for the full network. Input-dependent coefficients break fixed convolution but preserve associativity of affine composition.[10]
Equivalent algebraic explanation of scan; associativity follows from affine composition
A parallel prefix scan is therefore possible. The implementation fuses discretization and recurrence in SRAM and recomputes states in backward, avoiding the full B×L×D×N state tensor in HBM. Selection and hardware-aware execution jointly support linear sequence complexity.[10]