全部 AI 动态

(1/N) 🧵 Chinchilla assumes you'll never run out of fresh data. That era is ending! Compute keeps growing exponentially, but high-quality tokens don't. So what's the exchange rate between extra compute and fresh, high-quality data? We propose Compute-Data (CD) scaling laws, which measure the exchange rate between extra compute and fresh data.

We asked an unreleased research version of Claude to take a stab at the Riemann hypothesis. It didn’t solve it, but it did make strides on a related problem: it increased the lower bound for the fraction of zeros of the Riemann zeta function that satisfy the hypothesis from 41.6% to 67.2%. https://www.anthropic.com/research/riemann-zeta

We asked an unreleased research version of Claude to take a stab at the Riemann hypothesis. It didn’t solve it, but it did make strides on a related problem: it increased the lower bound for the fraction of zeros of the Riemann zeta function that satisfy the hypothesis from 41.6% to 67.2%. https://www.anthropic.com/research/riemann-zeta

We asked an unreleased research version of Claude to take a stab at the Riemann hypothesis. It didn’t solve it, but it did make strides on a related problem: it increased the lower bound for the fraction of zeros of the Riemann zeta function that satisfy the hypothesis from 41.6% to 67.2%. https://www.anthropic.com/research/riemann-zeta

I optimized SQLite 3. The optimized build achieves a verified geometric-mean speedup of 1.59x over the pristine latest trunk across four benchmarks: - 2.06x on the official speedtest1 (~30k statements) - 1.90x on TATP (OLTP mix: 400k transactions, 100k subscribers) - 1.30x on the Star Schema Benchmark (13 queries x 2, 1.5M-row lineorder) - 1.25x on kvtest (blob I/O: 40k x 10 KB, seq + random + update) All 1,032,940 cases in the full SQLite test suite pass, and every benchmark run produces checksum-verified identical results. For context: the SQLite team has spent nearly 20 years tuning this code; their own measurements show ~3.5x total CPU improvement since 2008, earned a few percent at a time. Tested so far on standard Linux provided by GCP. An agent called KISS Sorcar did it in under 8 hours for under $150 in API cost. I wrote no code: 1 main short prompt plus a couple of steering prompts. Why I trust the results: 1. No cheating in the speedup. The biggest win is defaulting to WAL journaling, the configuration SQLite's own docs recommend. Re-measured durability-neutral (synchronous=FULL: committed transactions survive power loss exactly as strongly as the baseline), it is still 1.54x faster. 2. Adversarial testing. A separate attacking agent tried to break the changes: a 37-script differential SQL corpus (recursive CTEs, window functions, triggers, UPSERT, JSON, FTS5, rtree, corrupt inputs) compared byte-for-byte against a pristine build under ASan/UBSan, plus WAL-file corruption, multi-process mptest, fd-exhaustion, symlink, and read-only-media attacks. 3. Security hardening and independent review. Two hardening rounds with kimi-k3 as the model: in-tree fuzzers (fuzzcheck over all 8 corpora, sessionfuzz) came back clean; 14 hostile-WAL corruption scenarios - no crashes; OOM injection and page-size sweeps; one real bug found and fixed. 1.59x is what an agent can honestly find in one of the most heavily optimized codebases in the world overnight. Optimized SQLite Repository: https://github.com/ksenxx/sqlite-optimized/ Blog: https://kisssorcar.github.io/blog/sqlite-optimization-blog.html

Zero Train–Inference Mismatch — now for linear attention, and under async RL 🎯 We got bitwise-exact trainer/generator parity for Gated DeltaNet (Qwen3.5-9B / 35B-A3B) on TorchTitan RL + vLLM, then asked the question nobody had actually tested in open source: does it help async RL? Two firsts (as far as we know): 1️⃣ First open-source stack to hit train/inference logprob diff = 0 on a linear-attention model 2️⃣ First to measure what zero mismatch is worth under async off-policy RL How — smaller than it sounds: • trainer and generator share one model definition (TorchTitan unified model). No more aligning trainer & generator op by op • recurrent kernel everywhere on the forward (prefill + decode + trainer), chunked only for backward • split-K off, batch-invariant GEMM, + a batch-invariant bmm for the MoE router • we never touch FLA kernel internals — the recurrent kernel is intrinsically batch-invariant: one sequence's own state, fixed order, no cross-sequence reductions • vLLM's linear-cache management + prefix caching keep working as-is 📊 Qwen3.5-9B + DAPO-Math, async at offpolicy = 4 / 12 / 32: • logprob_diff = exactly 0 at step 0 ✅ every nonzero after that is staleness, not precision • at offpolicy=32 the stock stack runs away past 0.065; ours holds ~0.035 • the unified model is free — the entire bill is the kernels • also verified on MoE 35B-A3B and a 64-turn / 64K-context terminal agent 🤔 Now the honest part: it depends on the config. On math at offpolicy=12, BI leads on train reward and peaks highest on held-out AIME. On the terminal agent it's slightly ahead. On search, and on other math configs, the curves are indistinguishable. What is consistent is the numerics — BI keeps the gap from exploding as you widen the window. Cost: 2–3× trainer throughput (≈5× on the agent). Our take: bitwise parity is a debugging tool, not a production default. Flip BI on for 20 on-policy steps — if the gap is 0 and your run is still broken, the infra is exonerated and the bug is in your data or your algorithm. That's worth 20 steps, not a whole run. Next: • make BI cheap (kill the 2–3×) • more workloads, more seeds • stay tuned for more complex agentic training recipes/tests 👀 Full writeup 🔗 https://yichuan-w.github.io/blog/GDN-train-inference-mismatch-asyncRL/ We collaborated closely with the TorchTitan team, and we sincerely encourage everyone to check out TorchTitan RL. With its clean, efficient unified model architecture and native async RL design, we believe it is a strong foundation for your next RL engine. #LLM #RL #agentic #pytorch #torchtitan

New paper, led by Agatha Duzan: your CoT monitorability numbers are too optimistic. Models are bad at hiding on demand: instructions to do a bad thing leaks into the CoT. How much does this inflate the numbers? Our new benchmark measures it 🧵