全部 AI 动态

There's not enough buzz around Claude Mods. Read this Claude Mods article if you want to: - Understand Claude Mods basics - Install and use the most useful mods today - Ponder where the agent-human interface will go For @latentspacepod.

When our AI code reviewer's cost per run doubled, we dug into the token usage to find out why. Learn how we used prompt caching and intelligent model routing to slash our inference bill by 68% while speeding up reviews. https://cfl.re/4isj2NU

I just published my roadmap for building evals on highly subjective tasks. Here are the key steps involved and practical tricks for reducing noise / increasing confidence… Blog is here: https://cameronrwolfe.substack.com/p/subjective-evals Why is this difficult? Evaluating agents is already hard due to their broad scope, emphasis on expensive / long-horizon tasks, and non-determinism. Subjective tasks make this even harder because we cannot deterministically verify solutions and must rely on human judgments (or LLM judges). This introduces additional noise into an already difficult evaluation process. Evaluation roadmap. We can build reliable evaluations using a repeatable framework with three components: - Human evaluation: Our North Star for quality. Create quality guidelines that clearly define quality with the help of subject-matter experts, calibrate human evaluators to ensure agreement, and continually monitor evaluation quality. - Offline evaluation: Use calibrated LLM judges or task-specific rubrics as an efficient proxy for human evaluation. This allows us to scale experimentation without relying on expensive human evaluation for every new model. - Production monitoring: Track quality signals from real-world agent usage and use A/B testing to validate improvements before fully deploying a new agent. Multi-stage evaluation. These components form a tiered evaluation process: Offline evaluation → Human evaluation → Production A/B test → Full deployment We can test many ideas automatically, send promising candidates to human evaluation, and validate improvements through production A/B testing. Continual production monitoring then allows us to discover new use cases / failure cases in production that we can add into the earlier stages of evaluation to create a feedback loop. Sampling useful data. Evaluation quality depends heavily on which tasks we select. We want tasks that are diverse, realistic, difficult, discriminative, and accurate. Techniques like structured task taxonomies, model-based difficulty filtering, Item Response Theory (IRT), and LLM-based auditing help us construct more informative evaluation sets. Handling noise. We should leverage statistics throughout the evaluation process: - Power analysis: Estimate the number of evaluation samples needed to detect meaningful improvements. - Confidence intervals: Quantify uncertainty and provide clear confidence signals alongside evaluation results. - Re-sampling: Repeat agent / judge evaluations to reduce within-task variance and potentially tighten confidence intervals. - Paired comparisons: Evaluate models on the same tasks to more reliably detect performance differences. Key takeaway. An improved benchmark score does not always mean our agent is more capable. We need well-designed evaluation tasks, calibrated human / model evaluators, and statistical confidence signals to distinguish meaningful capability improvements from noise (especially on subjective tasks). By combining these techniques into a repeatable evaluation process / roadmap, we can make building and maintaining evaluation suites more systematic, reliable, and scalable.

看过谭谭交通么兄弟们，这个节目很搞笑，但是说话实说里边很多对话不是很能听懂，比如福建司机找医院这个。我一直想知道，这个福建人，到底说的是什么，他朋友的翻译到底对不对。现在好了，Ming-3.0语音系列模型来了，解决了我的问题，你们也可以去试试！ https://x.com/i/article/2113042697183465472

Grok Bot can now build reports, memos, and dashboards. Ask Bot for a page, then edit it together or have Bot keep it up to date.

Talking at 3pm at @nvidia GTC Berlin on how @Shopify post-trains open source models and optimizes inference using Nvidia's models and tools. Read our blog post about it here: https://shopify.engineering/open-model-task-experts

Grok Bot is now one step closer. Set it as your iPhone's Action Button and press it to talk with Bot.

https://x.com/i/article/2112617637763047425

https://x.com/i/article/2103456568209768449

完成了《火炬之光 2》在 Mac 上的现代化改造。让十几年前的引擎和各种组件在 M 芯片上原生运行。支持了 Retina 的高清显示，游戏窗口能够任意拖动、最大化。 下一步准备把主机板的手柄支持接回来，让一个版本能同时支持键鼠和手柄。做完后再移植手机，交互就比较简单了。

pstack v0.15.27 ships with a new skill /create-formal-verification-skill! it's still in beta, and similar to /create-verification-skill, this is a meta-skill that creates a new skill and formal verification suite based on the characteristics you want to verify. for example, you may want to model state machines and systems with concurrency using TLA+, or use kani in rust to check for undefined behavior. if you're new to formal verification, think of it as mathematically proving properties and invariants (something that must always be true) or "laws" of your system. usually, this means modeling your programs in a separate language like with TLA+ and then running the solver on that spec. newer approaches like with kani and @bendlang take a different approach where there is no separate language: you write kani in rust, and bend in bend (it's also a programming language, and the proofs are written as laws and expressed in its type system). historically formal verification never reached mainstream status because of the difficulty in modeling and writing proofs. it's quite difficult to model your business requirements and keep them in sync! but i expect that we will see more innovation in formal verification in the future as being able to formally verifiy that the code does what you intended gives your agents superpowers in writing correct software. in the meantime though, i've been experimenting with modeling our code with TLA+ and have already found several bugs with the skill created by /create-formal-verification-skill. try it out and let me know how it goes!

Over 1 million lines of dead code removed. 🧹 Meet AutoClean, Uber's AI-powered system that's cleaned up 16,000+ stale feature flags using AI agents, deterministic code transformations, and validation. https://x.com/i/article/2112273498110660608

New blog post 🎉: How To Use Decision Models For Evals In this post, we show you how to choose a decision model for your task, tune its threshold (don't assume the 0.5 default is good!), and interpret its scores. Lots of cool interactive charts here. https://x.com/i/article/2112552453422104576

New blog post 🎉: How To Use Decision Models For Evals In this post, we show you how to choose a decision model for your task, tune its threshold (don't assume the 0.5 default is good!), and interpret its scores. Lots of cool interactive charts here. https://x.com/i/article/2112552453422104576

@bot check replies to this thread, add all to a new bot (ask dr egg bot) for anime/tv/movies watch list. sort them into tv and movies and then sub-genre (ie. action/comedy/animated). do this once a day for the next 5 days. include imdb links for each one.

「2Dを3Dに繋ぐ」 生成した動画にアイデアを重ねたらどこまでやれるんだろう、と頑張ってみた軌跡です。 左上が元の生成、右上が完成、下がBlenderの画面。 ■やったこと 1. MidjourneyでKVを作る 2. MiniMax H3で10秒の走りを生成 3. 同じ動きのまま、素材だけ変えて7本に（絵の具／クローム／粉／墨／花／水／糸） 4. 前半の2Dは、Pythonでポーズごとに素材を切り替える 5. 2Dと同じ見え方の3D空間をBlenderに作り、つなぐ 6. 3Dでは、素材の軌跡の中にカメラが入っていく 7. 抜けて引くと、欠片が集まって巨大な走者の絵になる 8. 光の筋や床の映り込みも、Blenderの中で足す ■わかったこと ・生成とツールを組み合わせるとクオリティUP ・2Dを3Dに繋げば多少リッチな表現に AI生成で元からかっこいい素材を出せばいいのかもしれませんが、こういったアプローチもありそうだなと参考まで。

「音と動きをビジュアライズ」 Opus 5.5×Blender×Python ■素材 ・人物：MiniMax H3 ・キービジュアル：Midjourney ・音楽：MiniMax Music 3.0 ■Blenderとコードでやらせたこと ・音が大きいほど、絵が立体になって手前に飛び出す ・速く動いた所ほど、粒になって浮く ・打った瞬間に時間が止まり、カメラだけが回り込む ・止まっている間、空間ごと色と照明が切り替わる ■わかったこと ・生成だけでは出ない絵も組み合わせるとできる ・やりすぎるとだめ ・映像の動きにも反応させたら、映像そのものが鳴っているように見えてきた ・飛び出す量は曲の音と、素材の動きに任せた 実験的な映像ではありますが、何かの表現に使えそうだなと思います。AIをより拡張できる方法を探っていきたいです。 音と動きの両方で絵が動くと、なんか面白いですね。

斯坦福 @stanfordnlp 新课 CME 295，双胞胎兄弟 @afshinea @shervinea 主讲，9 讲走完 LLM 全部版图：从 Transformer 架构、注意力变体、RoPE，到 LoRA 微调、RLHF/GRPO 训练，再到 Flash Attention 推理优化、MCP 与 AI Agent、扩散语言模型。 课程和知名速查表全部公开。 https://x.com/i/article/2100019396664336384

我把给孩子做的键盘打字游戏开源了 Keyboard Island · 键盘小岛 https://github.com/shaom/keyboard-island 前面发布到 ChatGPT Sites，收到一些留言说访问不了，还有一些朋友想知道是怎么实现的。 其实整体功能都比较简单（虽然也都不是我写的），我的需求方在我迭代中也提出了很多小朋友的想法，比如按键要有音乐、要更像游戏更好玩、希望自己的头像出现在画面中、希望有自己的小屋可以领积分来装饰等等。所以就有了现在的版本。 感兴趣的朋友，欢迎多多交流，当然也欢迎 ⭐️ 一下 😄

I built an automated report generator that runs entirely on my own machine. Input: structured data + charts. Output: a full long-form report — earnings interpretation, weekly summaries, whatever you need. No API. No cloud. No monthly bill. -> Here's MiniCPM-V 4.7 35B-A3B doing exactly that 👇 It read the charts AND the dense tables in one pass, then streamed out analyst-grade markdown: scorecard, segment breakdown, margin drivers, cash flow, even a bull/bear case. The part that got me: it didn't just OCR numbers. It connected them — input costs → margin pressure → guidance risk. That's the jump from "text extraction" to "actual analysis." Why this works locally: → 35B total params, only ~3B active (MoE) → Efficient visual encoding keeps multi-chart inputs cheap → Structured markdown output — goes straight into your doc pipeline Try it GitHub: http://github.com/OpenBMB/MiniCPM-V Hugging Face: https://huggingface.co/collections/openbmb/minicpm-v-47

昨天我老婆问我 App Store 怎么付费，看了一下才知道她要给儿子买一个打字 App，收费 128 块。。 这。。小孩和女人钱这么好赚？？ 我说等我半个小时，打开 Codex，把打字游戏的需求输入进去，一个 Goal 跑起来，20 分钟就做好部署到 ChatGPT Sites 了。花了几分钟看了一遍，还不错，顺手再加了一个实时拍摄人像，让头像作为游戏主角的功能，30 分钟准时交活儿。 效果怎么样呢？我第一版做了 36 关，儿子一连玩了 18 关 😁😁😁，我边看他玩，边看有什么问题，边让 Codex 改，顺便增加了几十关，和无限模式。。 以后儿子再有什么想玩但花钱的游戏和应用，自己做就行了，大家感兴趣可以看这: https://keyboard-island-doudou.groovy-river-7567.chatgpt.site