Stabilizers, precision, and noise

A stabilizer changes Muon’s asymptotic geometry

For separable matrix-linear classification with logistic loss, fixed finite NS iterations, a positive stabilizer and a sufficiently small step, Theorem 1 gives Frobenius max-margin bias. We use the momentum-free case.[1]

Let $P_K$ apply the K-fold odd polynomial $p$ to singular values, with $a_K=(p^{\circ K})^{\prime}(0)>0$. The assumption $p^{\circ K}(s)>0$ for $0<s\le1$ includes the standard cubic and quintic maps. Near zero, the key is local linearization:[1]

Descent makes squared gradients summable, hence the GD perturbation is summable. Bounded trajectory distance transfers the normalized limit. A sufficient step is $\eta\le4\varepsilon/(R^2u_K)$, where $R=\max_i\|X_i\|_F$ and $u_K=\sup_{0<s\le1}p^{\circ K}(s)/s$.[1]

Low-precision error is not just fewer effective tokens

Write forward/backward quantization schemes as $\psi_f,\psi_b$, and parameters/tokens as $N,D$. The empirical law separates effective capacity, effective data and persistent optimization error:[2]

$N$ excludes embedding parameters; $\rho_N,\rho_D$ rescale effective size, $\epsilon_f,\epsilon_b$ are persistent errors, and $E$ is irreducible loss.[2]

Convex, differentiable-loss analysis with bounded second moments motivates nonvanishing forward-rounding and backward-gradient errors. With schemes fixed, the fitted law approaches $E+\epsilon_f+\epsilon_b$ as model and data grow.[2]

Spork: share capabilities, specialize, then distill

Low-success-rate GRPO has sparse signal. Spork warms up on the full dataset, specializes partition teachers, then consolidates into a base-initialized student on student-generated trajectories.[38]

$D_i$ denotes a partition, $w$ its sampling weight, and $T_\tau$ student tokens only. Implementation uses top-k logits and excludes environment tokens. Warmup uses 40 steps with 8 prompts × 8 rollouts.[38]

Long-horizon RL: Muon within an asynchronous recipe

Combines single-rollout asynchronous learning, token-averaged loss, truncated importance sampling and auxiliary penalties for repository repair across two model families. The abstract supports a full-recipe training-success gain, not attribution entirely to Muon.[8]

Subspace Muon with orthogonal residual correction

PMRC applies matrix sign to projected momentum and restores the orthogonal gradient residual. The authors analyze fixed and refreshed bases; we omit the abstract’s rate because its assumptions and convergence metric have not been checked.[9]

Replica scaling depends on the inner optimizer

Under fixed-budget DiLoco replica scaling, pseudo-gradient covariance and batch-dependent noise distinguish inner optimizers including Muon and AdamW. Whitening-based methods perform strongly over 8–96 replicas; this does not eliminate communication costs.[10]

Optimizer implicit bias as a privacy variable

Relates KKT conditions to training-data reconstruction for SGD, Muon and Adam on MNIST/CIFAR-10. Muon’s reconstructability depends on architecture and regime; Adam’s resistance to the evaluated attacks is not a general privacy guarantee.[11]

Compound p-values: aggregate rather than individual validity

Zhimei Ren and coauthors connect heterogeneous/shared-control competitions to compound p-values. Bonferroni controls FWER; BH may incur FDR inflation, which the paper bounds. These are different guarantees.[12]

What do spatial association axes estimate?

Nancy R. Zhang and coauthors study skrCCA targets, identifiability and stability. Conditional on locations under a mean-zero model, targets are kernel-weighted cross-covariances; the fixed-direction null variance assumes independence between blocks.[13]

Agent communication: probabilities or actions?

Aaron Roth and coauthors compare action messages with calibrated beliefs in synthetic and maze tasks. Biased probabilities can erase the advantage; post-hoc conversational calibration addresses that mismatch.[14]

L3M: a verifiable agent runtime

Dean Foster and coauthors use Lean 4 for a runtime and checked invariants, with recoverable state in Git. The abstract reports Terminal-Bench results with safe tools; formal coverage is unaudited here. The manuscript is dated September 30; this is a first-arXiv listing.[15]

Flow matching: more steps need not mean deeper reasoning

Kaifeng Lyu and coauthors show a constant-round parallel simulation of vanilla flow sampling under output regularity assumptions; self-conditioning changes this limitation. The claim is model-dependent, not a blanket impossibility result for diffusion reasoning.[16]

Best-of-N is a competition between reward tails

Sampling a correct answer is insufficient if incorrect answers dominate the reward tail. Extreme-value analysis separates correct-dominant, incorrect-dominant and critical regimes; the abstract does not provide a universal safe sampling threshold.[17]

Capacity-driven scaling in a bigram model

Fixed random embeddings and Zipf token statistics yield tractable width/vocabulary-dependent regret laws, checked on WikiText2/OpenWebText. This isolates capacity, not full Transformer optimization.[18]

Size generalization can have an exponential sample cost

For sets/graphs, polynomial source decay can yield polynomial in-domain error but only logarithmic size-generalization error. An exponential source condition restores polynomial rates; functional assumptions behind the claimed matching minimax bounds remain unchecked.[19]

Multilingual MoE: language coverage is a budget axis

Fits active parameters, tokens per language, expert count and language coverage, accounting for repetition, on 26,300 per-language losses from 64 configurations. Reported mitigation from more experts is conditional on the matched settings.[20]

Count tokenizer and generative-model FLOPs together

Generative representation learning uses information differences between conditional predictions to build tokenizers and studies joint compute efficiency. ImageNet/GPIC FLOP extrapolations should not be read as measured end-to-end speedups.[21]

Visual gains need not lower text-only loss

Content-free padding replaces image embeddings as a control. Visual content improves image-text prediction with approximately power-law compute gains, without materially changing the text-only loss floor or optimal model/data balance; gains vary by task.[22]

Repeating a smaller dataset can sometimes win

Image self-distillation depends jointly on unique examples and repetition. A modified Chinchilla-style fit helps in some regimes but inconsistently across datasets; deduplication is not uniformly beneficial.[23]

Extrapolating cheap synthetic data for radio-map prediction

Pretrains ResNet-101/Swin-T on up to 128M synthetic examples. Laws fitted at 2M–16M predict downstream normalized RMSE at 32M–128M; simulator mismatch remains a transfer limitation.[24]

Scaling curves for concept transmissibility

Optimized messages and prompt recovery probe 1.2k concepts as model capacity grows. Apparent plateaus are observational and sender/receiver-relative, not absolute limits of human language.[25]

PGLA: correct voting logits with pairwise comparisons

Combines pairwise evidence and vote support in a KL-regularized objective, evaluated on seven models and ten reasoning benchmarks. Full equal-inference-budget comparisons have not been checked.[26]

Recursive reasoning: change the update map

ICS searches input conditions at the same latent state to induce alternative update maps without training. Six-task FLOP comparisons do not imply a universal 100-fold speedup.[27]

SHELDON: search hypotheses and observations jointly

Separates observation discovery from current hypotheses, using unexplained observations to guide search across diagnosis, abstract reasoning and optimization. The abstract does not fully specify comparator inference budgets.[28]

ARBITER: counterexamples for code selection and refinement

Trains an agentic verifier to produce checkable counterexamples for code selection and revision. The IOI case uses official submission limits; this is evidence under a specific inference-budget protocol, not a universal scaling exponent.[29]

COUNT: move capacity into host-memory n-gram counts

Uses offline n-gram counts to combine existing unembeddings without added learned parameters. Fixed-compute 1B pretraining and table-swapping adaptation separate host-storage capacity from GPU compute.[30]

LatentLoop: unequal budgets across shared-block visits

Compresses the first visit’s sequence, retains full resolution on the second and reallocates compute to width. Across fifteen 1.3B–7B configurations, 19.0%/7.9% fixed-NLL savings are fitted predictions against two baselines.[31]

Negative result: more tokens do not replace regime diversity

In synthetic limit-order books with known latent states and Bayes predictors, capacity/tokens saturate early and regime diversity governs transfer. The conclusion concerns the controlled simulator family.[32]

Data filtering: selection versus redistribution

A four-arm budget-matched design separates within-support selection, redistribution and interaction. Finite pools may make the design unconstructible; unresolved components in retrieval experiments are not evidence of zero effect.[33]

Softmax geometry: Hilbert distance connects RowNorm and Muon

Linear-output RMS movement need not track softmax distribution change. Hilbert-distance constraints connect a RowNorm approximation for LM heads with Compositional Muon for query–key updates; this is a unifying account, not a new large-scale benchmark.[34]

The diffusion-LM “16× gap” depends on the protocol

A roughly 1,700-model study changes likelihood evaluation, block diffusion and encoder–decoder architecture. Reported compute gaps depend on block size; training savings at smaller blocks cannot simply be combined with full-parallel decoding benefits.[35]

Memory reuse is constrained by verification budgets

Treats cached reasoning as contraction and studies stale memories after facts change. Under outcome-only verification per memory, certification load and Zipf demand determine safe-reuse scaling; this is not an unconditional bound for arbitrary agent memory.[36]

PDE surrogates: budget the compute used to generate data

Jointly chooses sample count and simulation fidelity across six PDE problems, four surrogate architectures and over 7,000 runs. The reported up-to-6.3-fold saving concerns data-generation compute, not the full training pipeline.[37]

GEM: jointly scaling context, capacity and systems efficiency

Reports longer-history scaling, sparse attention and structured representations transferred downstream. End-to-end MFU is 20–25%; CompSkip’s 43% refers to FLOPs reduction. Architecture, precision, kernels and parallelism change together, preventing single-algorithm attribution.[39]