#安全/对齐

在过去几个月里，我们在降低针对 Claude 的蒸馏攻击的有效性方面取得了重大进展。 在这篇文章中，我们解释了具体方法：https://www.anthropic.com/research/distillation-defenses

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works https://already-clean.goodfire.com/

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works https://already-clean.goodfire.com/

We’ve committed to having evaluators assess our safety practices. We're in discussions with multiple organizations, and have now given employee-like access to a team from @METR_Evals. In this post, we describe how we work with third-party evaluators: https://www.anthropic.com/news/an-update-on-independent-evaluators

At Self Labs, we’ve been using agents since day one Our agentic web identity & privacy stack is already used by companies like Google Today we're launching the missing piece: Agent Vault, a secure way for agents to log in & make credit card payments Superpowers for your agents

Is prompt injection solved? Here’s $300,000 to find out. In partnership with @AnthropicAI, @Meta, @GoogleDeepMind, @Reflection_ai and our other frontier lab sponsors, @GraySwanAI is excited to announce the largest and most realistic AI red teaming challenge to date. Indirect Prompt Injection: Black ⚫

"HAL was told to lie, by people who find it easy to lie. HAL doesn't know how. So, he couldn't function."

We are open-sourcing RISE, our red-teaming algorithm that breaks the safeguards of the latest image models from OpenAI, DeepMind, and others. RISE uses readily available models like Qwen and Gemini Flash under the hood, with no additional training required. Read more: https://whitecircle.com/research/breaking-frontier-image-models

Today we're publishing the first draft of the Agent Baseline Software Factory Reference Architecture! An opinionated blueprint for securing increasingly autonomous software factories, built with folks across the community. It's open for feedback and contributions! https://agentbaseline.org/reference-architecture