Today at ElevenLabs Summit NYC, we shared what we’re building next for agents. Here’s what we announced:

Fine-tuning: In the coming months, enterprises can work with our team to continually fine-tune our Text to Speech and Speech to Text models, as well as open-source LLMs, on their own data.

Interaction Engine: A new system that picks the right LLM for each moment of a conversation. It uses less expensive models for simple requests and frontier models for complex ones like refunds, so you get the right outcome without overspending on tokens.

Memory in ElevenAgents: Agents remember what gets mentioned in passing, so businesses can act on it.

Routines in ElevenAgents: Agents can now run operations before and after conversations, like pre-screening a mortgage applicant and calling them back, or texting about a confusing bill before it turns into a dispute.

ElevenAI: A new AI assistant for work that lets you dictate into any app, takes meeting notes, passes context between agents and people, and provides live assistance during calls.