全部 AI 动态

This new-ish 35B multimodal model went from 3.38 → 20.51 tps on an iPhone. MiniCPM-V 4.7 35B-A3B was updated to decode faster on the iPhone by improving how it moves the model between storage and memory. Constraints 🧠 35B total / ~3B active 📱 iPhone 18 Pro Max — 12GB RAM 👁️ image + multi-image + video understanding Most of those 35B weights don't sit in RAM. The experts can stay in the phone's flash storage and get loaded when they're actually needed. Normally that's slow 🐌 SSD expert loading → 3.38 t/s So OpenBMB built Lookahead Expert Scheduling (LES) It predicts which experts the next layer will probably need and starts loading them while the current layer is still computing. Then they stacked the optimizations: 💾 baseline SSD loading → 3.38 tps 👀 expert lookahead → 6.57 ⚙️ optimized scheduling/loading → 11.34 🗜️ mixed-precision quantization → 17.16 🚀 + DSpark speculative decoding → 20.51 tps That's a 6.07× speedup over the same-phone unoptimized baseline. ⚠️ And these are still OpenBMB's own measurements. Link in ALT

CS 329Z: Engineering AI Agents Stanford / Fall 2026 @stanfordnlp https://cs329z.stanford.edu/ 课程定位：从"模型"到"系统"的工程学 覆盖：简单 LLM 流水线 → 复合 AI 系统 → 自主 Agent。三位讲师的背景也高度互补： @Diyi_Yang（斯坦福 NLP 教授，人机交互与社会计算方向） @michaelryan207（DSPy 核心贡献者，自动评估 AutoMetrics 作者） @jyangballin（SWE-agent / SWE-bench / SWE-smith 作者，软件工程 Agent 领域最重要的研究者之一） # 课程主线：三大工程挑战 贯穿全课的三个核心问题——分解（decomposition）、数据（data）、评估（evaluation）。11 周的内容基本围绕这三条线展开，可以分为五个模块： 模块 1：构建基元（Week 2–3） · LLM 作为构建材料：API/SDK（litellm）、结构化输出、约束生成、解码策略、test-time compute、上下文工程、模型选型与成本/延迟权衡 · RAG：embedding、向量库、分块策略、混合检索、cross-encoder 与 ColBERT 后期交互 · 工具调用：函数调用 API、MCP（Model Context Protocol）、工具设计、代码沙箱、错误处理与重试 模块 2：框架与设计模式（Week 3–5） · 框架层：DSPy（signature / module / optimizer）、LangChain/LangGraph、LlamaIndex，重点是"框架抽象了什么 vs. 你手写了什么" · 设计模式：workflow vs. agent 的分类学，五种可组合 workflow 模式，ReAct / plan-and-execute / reflection，"scaffold（脚手架）本身就是设计决策" · 记忆架构：短期/长期记忆、记忆作为工具动作、文件系统作为外化记忆、跨 Agent 记忆（MemGPT、Mem0、Generative Agents） · 多 Agent 系统：编排模式、handoff 与状态传递，以及一个很有态度的对照阅读——既读 AutoGen，也读《Why Do Multi-Agent LLM Systems Fail?》和 Neubig 的《Don't Sleep on Single-agent Systems》 模块 3：优化（Week 5） · 从提示词到微调的全景：GEPA、MIPROv2、OPRO、TextGrad（提示优化）；LoRA/QLoRA、蒸馏、RLHF/DPO（权重优化）；test-time scaling（推理算力） · 核心问题是决策框架：什么时候优化 prompt、什么时候优化 weights、什么时候堆推理算力 模块 4：数据与评估（Week 6–8）——最有分量的部分 · 数据：trace、demonstration、feedback 三类数据；训练数据 vs. 评估数据；数据飞轮；合成数据；从 Agent 轨迹构建数据集（SWE-smith） · 评估基础：为什么 eval 难；4 元组框架（request / environment / stopping criteria / scorer）；好 benchmark 的性质；tinyBenchmarks · 评估基础设施：三类 grader、LLM-as-judge 的 prompt 设计与已知偏差、pairwise vs. pointwise、非确定性指标 pass@k vs. pass^k、harness 设计 模块 5：安全与前沿（Week 8–11） · 安全：工具访问的隐私风险、prompt injection（含间接注入）、红队、沙箱与权限模型、输出护栏、human-in-the-loop · Coding Agent：SWE-agent、Claude Code、OpenHands 的端到端架构对比 · 主动式 Agent：从 reactive 到 proactive，General User Models（GUM）、Next Action Prediction，以及"Agent 何时应主动、何时应等待"的 mixed-initiative 问题 · 开放问题：多模态/web/计算机使用 Agent、科学 Agent、长时运行架构、生产可观测性（tracing、monitoring、成本管理） # 作业设计：一手建、一手评 HW1：从零构建 Agent 系统（10%） 给定论文库，构建能检索并推理回答科学问题的 Agent。 · Part A：只用 litellm 手写 RAG + 工具调用 + ReAct 式 Agent 循环 · Part B：用 DSPy 重建关键组件，并反思框架抽象了什么 HW2：评估一个 Agent（10%） 给定一个预构建 Agent，设计完整评估套件：代码型 grader、至少一个 LLM-as-judge、用 4 元组框架构建 benchmark 任务、错误分析。

I've gone full Grokbot. It checks email, DMs, mentions, and LinkedIn every hour. It flags anything interesting, drafts replies, and tosses the other 99%. It literally feels like having a personal secretary 🤯 And 2 days in I'm like how did I live without this 😂

Fun fact: `--dangerously-skip-permissions` doesn't actually approve EVERYTHING—we still block some obvious things like `rm -rf ~`. But today's release will let you disable this if you really want, by setting a new env var `CLAUDE_CODE_EVEN_MORE_DANGEROUSLY_SKIP_PERMISSIONS=1`

As our customers scale to training robot models using millions of hours of our recordings, we needed to build a new kind of data infrastructure. Terabytes of data delivered in minutes, and the capacity to deliver over a Petabyte per day to each of our customers, to multiple customers at the same time. That’s more than 10 Gigabytes a second, delivered into any cloud, wherever our customers need it. Using our customer portal, customers select precisely the data they want and it starts arriving in minutes at their destination, including AWS, GCP, Cloudflare, Hugging Face and any cloud provider who can handle the ingest speed. We’ve written about how we worked with @huggingface and @Cloudflare to build this secure, scalable infrastructure. Link in comments.

progress on my task-agotchi

so many gems and great tips in this talk from @yminsky about how to improve your codebase for agents • agents thrive on feedback, and types are one of the best kinds • make illegal states unrepresentable with types, and it enforces global rules across your codebase • property based tests, fuzzing, and even full-system deterministic simulation testing • expect tests: just print stuff out and capture it in the test. makes behavior legible. it's very similar to how compilers have fixture tests, it reduces the review surface area • formal methods used to be very expensive: seL4 is about 8,700 lines of C and took around 25 person-years, with 23 lines of proof for every line of C code, but agents change the roi https://www.youtube.com/watch?v=rUYP4C29yCw

https://x.com/i/article/2114435487905808384

A lot of people ask me how I build games. Well, it's pretty simple and I call it "vibecoder dream" Before going to bed, I give a simple game idea to Tripo CLI + Unreal CLI, then, using ORCA CLI, let Fable orchestrate everything at max effort, with 10+ Opus 5.5 agents running at high effort. By the time I wake up, I usually have something like this.

How I build apps these days: 1. Record a 1-5 minute prompt with @WisprFlow The more detail, the better. 2. Add visual inspo so the agent knows what I have in mind (screenshots or Inspo MCP). 3. Use @mattpocockuk's grill-me skill to have the agent ask clarifying questions. 4. Get back a v1, then send a list of feedback, feature requests, and design tweaks (via another voice recording usually). 5. For each new feature, ask for a quick screen recording to verify it works before merging the PR. 6. Keep iterating until I'm happy, then post it on @X or throw it away. For my harness, I use a mix of Claude Code with Opus 5.5 (high) and OpenCode with DeepSeek V4.1 Flash / GLM 5.3. I also give my agents some CLIs & skills: • Vercel + GitHub CLIs so they can push code and deploy apps. • Together skills so they can use chat, image, & voice models from Together • Design/animation skills so they have guidance on spacing & interactions

Finally summarized what I've been sending to friends in DMs https://x.com/i/article/2114385531811319808

The complete reasoning-from-scratch video series is now up (and linked) here: https://magazine.sebastianraschka.com/p/building-reasoning-models (Taking a short break but will be back with LLM model and research updates in November!)

@dexhorthy How does one know what is already in the weights?

There is one frontier lab that really cares about agents: SpaceXSI and @elonmusk They are building @grok @bot at a record pace based on actual user feedback and needs. Their Bots are able to do more every day: scheduling, shopping, travel, research, analysis... Other agents often feel like science projects. I use over a dozen Grok Bots to manage web sites, coordinate my calendar, help with kid activities, manage fundraising, and assist with my hobbies. Each Bot is an expert at what it does. The quality of the user experience is best in class. Grok Bots work on every platform: Mac, Windows, iOS, Android. They speak to you on a call, and the voice just works. I did a 20 minute call with my assistant Bot while driving to train it on how I like things, and it now just gets things right every time. It even suggests breaks when I need one. Try Grok Bot for a month and prepare to be blown away. Start with an assistant Bot and do a voice call to have it ask you what you need, like I did. Everyone that I know who has tried Grok Bot has become a convert. You can have a team helping you today. Happy to answer your questions.

The complete reasoning-from-scratch video series is now up (and linked) here: https://magazine.sebastianraschka.com/p/building-reasoning-models (Taking a short break but will be back with LLM model and research updates in November!)

Made a skill for dimensional drawings of UI components. Inspired by Apple's accessory drawings, it documents a component's architecture into a spec sheet. https://github.com/enesozturk/dimensional-drawing