NVIDIA research papers are on fire recently!

Here is another interesting paper where they give full-duplex speech models tool calls.

(bookmark it)

Commercial duplex voice models complete 31 to 51 percent of grounded customer-service tasks under clean conditions.

Text agents like GPT-5 reach 85 percent on the same tasks in text mode.

Most of what a voice agent loses, it loses in the speech pipeline.

The fix routes the decision out of the speech model.

The duplex frontend learns to emit a delegation token, forwards streaming transcripts to a text backend LLM for the tool call, and receives the result through a lightweight prefill-and-repeat mechanism before streaming TTS speaks it.

Tool-call recall runs 92.0 to 97.2 percent with 81.2 percent accuracy at rejecting irrelevant calls. Turn-taking rate, streaming ASR word error rate and spoken-language intelligence all stay at the no-tool-call baseline.

Paper: https://arxiv.org/abs/2609.19334

Chat with Paper: https://academy.dair.ai/papers/a-frontend-backend-architecture-for-tool-calls-in-full-duplex-speech-models-2609.19334