ZHZixun Huang
Research DigestNo. 005
Daily research notes14 min read

Same Policy, Different Trajectories

Consistency in long-horizon RL · Selectors and budgets · Reading NTK

This issue examines training–sampling consistency in long-horizon RL and the budget boundaries of selection and feedback in 3D generation. Five papers/blogs and one technical follow-up are overlap-window additions, alongside two version records with unverified changes; none is presented as first released today. Our classic is NTK: from Jacobians to spectral filtering.

Post-trainingInference scalingData scalingNTK
Daily classic · 2018Neural Tangent Kernel
Focus · technical blogFull blog read; checked QwenGyre Appendix B and the prefix invariant in renderers

Same weights, different policy

Mo’ turns, mo’ problems
Ian Barber
2026-10-15 · overlap-window addition; QwenGyre and renderers below are background

Long-horizon RL mismatch extends beyond stale weights: tool parsing can change tokens, trajectory reconstruction can change contexts, and numerical kernels can change probabilities. The blog connects these failure modes across execution.[1]

Explanatory notation: cᵤ is the original context; equality also requires matching inputs, sampling distributions, and computation
ρu(θ)=πθ(yu∣cu)puroll,θ=θroll ⇏ ρu=1\rho_u(\theta)=\frac{\pi_\theta(y_u\mid c_u)}{p_u^{\rm roll}},\qquad \theta=\theta_{\rm roll}\ \not\Rightarrow\ \rho_u=1

QwenGyre retains recorded behavior probabilities instead of recomputing the denominator on the trainer, and deduplicates trainable tokens across an execution tree. It normalizes by execution rather than rewarding tasks with more branches:[2]

Original Appendix B; Lᵢ counts deduplicated target tokens, and ℓᵤ includes the advantage, clipping, and shared mask
si=sg⁡exp⁡ ⁣(1Li∑u∈Tilog⁡ρu),ρˉu=sg⁡min⁡(ρu,5),Li=1Li∑u∈Tiℓus_i=\operatorname{sg}\exp\!\left(\frac1{L_i}\sum_{u\in\mathcal T_i}\log\rho_u\right),\quad \bar\rho_u=\operatorname{sg}\min(\rho_u,5),\qquad \mathcal L_i=\frac1{L_i}\sum_{u\in\mathcal T_i}\ell_u
Focus · new preprintRead Section 5.5, Table 4, and Appendix G.2–G.5

More candidates, or more reliable feedback?

Grounding Generative Verification in 3D Content
Shuanglin Li · Jiaying Lin · Shuquan Ye
First posted 2026-10-15 13:59 EDT · v1 · overlap-window addition

Best-of-N gains depend on both the pool and the selector, not just an oracle. For each of 40 inputs, Pixal3D and Hunyuan3D-2.1 each generate eight candidates; seven factual requirements are fixed beforehand.[4]

Original G.2: a marks satisfied facts; d marks a severe defect and r source fidelity
fij=∏k=17aijk,hij=fij(1−dij)rijf_{ij}=\prod_{k=1}^{7}a_{ijk},\qquad h_{ij}=f_{ij}(1-d_{ij})r_{ij}
Same pool of 16Full (%)Strict (%)
Uniform random26.716.6
GPT-6 Astra selection55.045.0
Human oracle90.085.0
Full requires all seven facts; Strict additionally requires no severe defects and source fidelity.[4]
Brief · technical follow-upRead task generation, reward audits, and training results

Nemotron follow-up: make the reward measure the task

Post-Training Nemotron 3.5 Super VL for Long-Horizon Enterprise Work
Radhika Gaonkar · Florian Brand · Anirudh Ravichandran · Christian Munley · Daniele Molinari
2026-10-15 · task-construction experiment for an already covered model, not another model release

Write an executable solution before deriving checks; test both shortcuts and equivalent valid solutions. With 2,500 tasks, 128×GB200, and about 11 hours / 100 GRPO steps, public held-out AutomationBench-AA scores rise from 34.0 to 57.7. The report selects the best checkpoint; this is not the private leaderboard.[5]

Original score: partial objective credit gated by constraints; 57.7% is not full-task success
Ri=#{objectives met}#{objectives}1{no guardrail violated}R_i=\frac{\#\{\text{objectives met}\}}{\#\{\text{objectives}\}}\mathbf1\{\text{no guardrail violated}\}
Brief · new preprintAbstract only

At matched data and compute, does tactile supervision help?

Spark-0: Scaling Spatially Aligned Tactile Pretraining for Robot Dexterous Manipulation
Tianxing Chen · Wenwei Lin · Chuqiao Lyu et al.
First posted 2026-10-15 13:56 EDT · v1 · overlap-window addition

Using 1,000 hours of HumanTouch, retaining tactile pretraining supervision raises vision-only Ego Humanoid success from 52.5 to 71.7% against a same-data/compute baseline that discards touch. This tests supervision quality, not a fitted data-scaling exponent. Real-robot results also use deployment-time touch and have a different attribution.[6]

Brief · new preprintAbstract only

One pretraining corpus, different saturation points

Data-Efficient ConvNeXt Pretraining for High-Resolution Sentinel-1 GRD SAR Representation Learning
Yuki Natsume · Sang-Ho Yun · Chai Kiat Yeo
First posted 2026-10-15 10:40 EDT · v1 · overlap-window addition

In SparK-SAR data sweeps, classification saturates around 10–25% of SSL4EO-S12 v1.1, while dense prediction still benefits up to 25–50%. Task differences should not be compressed into one exponent. Comparing a fine-tuned 27.8M encoder with a frozen 139M encoder also changes the adaptation protocol.[7]

Brief · new preprintAbstract only

Spend extra inference budget at critical states

VeriVLA: Adaptive Test-Time Verification for Flow-Based VLAs with Latent World Models
Xueli Sun · Yuxiang Lu · Yantong Liu · Xianzhe Fan · Xi Chen · Zhe Liu · Hengshuang Zhao
First posted 2026-10-15 10:34 EDT · v1 · overlap-window addition

Temporal variation and cross-sample variance from two initial action chunks trigger extra sampling, ranked by a JEPA latent world model. Mean completion score across three real tasks rises from 50.8 to 65.5%, adding 326 ms per triggered decision on an RTX4090. This is conditional latency, not a fixed per-step cost or full-task success rate.[8]

Version record · changes unverifiedAbstract and version history only

MIXAR v2: version record

MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts
Chen Hu · Yintao Tai · Antonio Vergari · Frank Keller · Alessandro Suglia
First posted 2026-04-13; v2 updated 2026-10-15 13:46 EDT

Multilingual pixel language modeling, model-scale comparisons, and GRPO generation training. The v2 date is verified, but v1/v2 changes have not been checked item by item; abstract claims are not presented as new results.[9]

Version record · changes unverifiedAbstract and version history only

GRAM v4: recursive depth and parallel inference

Generative Recursive Reasoning
Junyeob Baek · Mingyu Jo · Minsu Kim · Mengye Ren · Yoshua Bengio · Sungjin Ahn
First posted 2026-05-19; v4 updated 2026-10-15 11:32 EDT

Stochastic latent trajectories support both recursive depth and parallel sampling. Substantive changes from October 7 v3 to v4 remain unverified; this is a version record, not a new performance claim.[10]

Daily classic · 005Read Sections 2–5, 6.1–6.2, and the appendix limit setup; checked the PDF theorem page

NTK: nonlinear training becomes a linear flow

Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot · Franck Gabriel · Clément Hongler
Historical classic: first posted 2018-06-20; read v4 dated 2020-02-10; NeurIPS 2018
NeurIPS 2018 · foundational deep-learning theory ↗

NNGP (Lee et al., ICLR 2018) describes the function distribution at initialization.[13] NTK asks how functions move once optimization starts, separating nonlinear parameterization from dynamics on a finite training set.[11][12]

Scalar output and full-batch gradient flow; an exact finite-width chain-rule identity
ft=(fθt(xi))i=1n,rt=ft−y,Jt=∂ft∂θ,L=∥rt∥22n,r˙t=−1nKtrt,Kt=JtJt⊤f_t=(f_{\theta_t}(x_i))_{i=1}^n,\quad r_t=f_t-y,\quad J_t=\frac{\partial f_t}{\partial\theta},\quad \mathcal L=\frac{\|r_t\|^2}{2n},\qquad \dot r_t=-\frac1nK_t r_t,\quad K_t=J_tJ_t^\top

Theorem 2 assumes fixed depth, Gaussian initialization, NTK parameterization, a Lipschitz twice-differentiable activation with bounded second derivative, sequential width limits, and a stochastically bounded time-integrated training direction. The kernel then becomes constant on finite time intervals.[11]

Original parameterization and a matrix formulation of the theorem on a fixed finite training set; T is fixed
α~(ℓ+1)=W(ℓ)α(ℓ)mℓ+βb(ℓ),sup⁡t≤T∥Kt−K∞∥op→P0\widetilde\alpha^{(\ell+1)}=\frac{W^{(\ell)}\alpha^{(\ell)}}{\sqrt{m_\ell}}+\beta b^{(\ell)},\qquad \sup_{t\le T}\|K_t-K_\infty\|_{\rm op}\xrightarrow{\mathbb P}0

With the kernel fixed, squared-loss dynamics form a linear constant-coefficient system. The next two formulas solve it directly, using orthonormal eigenvectors:[11]

Residuals with λ=0 persist; positive semidefiniteness alone does not ensure fitting every label
K∞vj=λjvj,rt=e−tK∞/nr0=∑j=1ne−λjt/n⟨vj,r0⟩vjK_\infty v_j=\lambda_jv_j,\qquad r_t=e^{-tK_\infty/n}r_0=\sum_{j=1}^n e^{-\lambda_jt/n}\langle v_j,r_0\rangle v_j
Direct consequence: early stopping is spectral filtering; large kernel eigenvalues learn first, not universally low input frequencies
gt(λ)=1−e−λt/n,K∞≻0 ⟹ ∥rt∥2≤e−λmin⁡t/n∥r0∥2g_t(\lambda)=1-e^{-\lambda t/n},\qquad K_\infty\succ0\ \Longrightarrow\ \|r_t\|_2\le e^{-\lambda_{\min}t/n}\|r_0\|_2

Let k(x,X)k(x,X) be the limiting kernel row from a test point to the training set. When invertible, off-training predictions retain an initialization term:[11]

Obtained by integrating the training-set solution; scalar-output rewrite
ft(x)=f0(x)+k(x,X)K∞−1(I−e−tK∞/n)(y−f0(X))f_t(x)=f_0(x)+k(x,X)K_\infty^{-1}(I-e^{-tK_\infty/n})(y-f_0(X))

READ THE ORIGINALS

Original sources

  1. Ian Barber · 2026-10-15 · technical synthesis
  2. QwenGyre · 2026-09-27 · background, Section 5 and Appendix B
  3. renderers · 2026-05-12 · background
  4. 3DVerifier · v1 · Section 5.5 and Appendix G
  5. Prime Intellect / NVIDIA · 2026-10-15 · task construction and training
  6. Spark-0 · v1 · abstract
  7. SparK-SAR · v1 · abstract
  8. VeriVLA · v1 · abstract
  9. MIXAR · v2 · version record and abstract
  10. GRAM · v4 · version record and abstract
  11. NTK · Jacot, Gabriel, Hongler · read v4
  12. NeurIPS 2018 official proceedings · NTK
  13. Jaehoon Lee et al. · ICLR 2018 · historical background
Search window

2026-10-15 08:37 — 2026-10-18 08:37 · America/New_York

READ · THINK · DISCUSS

Discussion & likes

Discuss a derivation, add a reference, or suggest the next classic.

Comment, reply, or like below. Sign in with GitHub to participate; comments are public.

Loading discussion…

Open discussion on GitHub ↗

← Back to all issues