Google 实测用轻量分类器在 Gemini 模型和思考档位间路由以降低每任务成本

NOV. 3, 2026

Per-token API prices keep falling, yet engineering bills for autonomous coding agents keep climbing. Price per million tokens isn't the metric that actually governs production economics anymore. The dominating metric is now cost per successful task: total spend across every loop iteration, tool call, retry and internal thinking pass divided by the number of tasks that pass verification.

When an agent calls a high-thinking reasoning configuration for every micro-decision in its harness, you're paying deep-reasoning prices for questions a lightweight configuration already answers accurately. To prevent this unnecessary overhead, consider adding a lightweight routing layer that can intercept routine decisions before they ever reach an expensive reasoning model.

That is the premise behind wiring TypeSafe's Jev classifier in front of Google Antigravity's model calls. Rather than generating text token by token, Jev—a specialized structured-output classifier designed for high-throughput decision pipelines—evaluates structured classification primitives in a single forward pass: a Choice from a list, a numeric Score or a Noul, which returns a 0 to 1 probability for a binary statement.

For full disclosure, Google has no partnership with TypeSafe, and this article is not an endorsement of TypeSafe or its products. Jev is used here as one example of a lightweight classifier, and the same routing pattern can be built with other classifiers, including Google-native ones. The Getting started with discriminative models codelab, for example, runs the same kind of classifier on DiffusionGemma, Google DeepMind's open-weights model, in your own Google Cloud project.

We ran a 50-question, auto-graded, coding-agent benchmark against live Gemini and Jev APIs to test whether a near-free classifier can route each task to the cheapest model that still gets it right. Here's what we learned:

1. Regex and token-length harness routers collapse on unseen agent tasks

The naive way to ask Jev to route a coding task is a single broad Choice question: simple or complex. When we tested that first on 16 coding-agent tasks, Jev routed nearly everything to simple, even tasks requiring an LRU cache trace or modular exponentiation. TypeSafe's documentation warns against this exact anti-pattern. If a question weighs multiple factors or requires a meta-judgment, decompose it into an atomic check.

When we replaced the broad Choice question with one atomic Noul question, it fixed the separation immediately. The question was: does answering this require performing more than one sequential calculation, simulation step or logical deduction, rather than a single direct lookup, format check or one-step classification?

def call_jev(question_text):
    body = {
        "state": question_text,
        "model": "jev-latest",
        "questions": {
            "multi_step": {
                "type": "noul",
                "instructions": (
                    "Answering this requires performing more than one "
                    "sequential calculation, simulation step, or logical "
                    "deduction, rather than a single direct lookup, format "
                    "check, or one-step classification."
                ),
            }
        },
    }
    # Send the request above to the classifier's API.
    # response.answers["multi_step"].noul is the probability used to route.

Python

Copied

On those initial 16 tasks, Jev routed 13 of 16 in agreement with our hand-labeled difficulty on the first run and 14 of 16 on the second pass. Many engineering teams ask whether they even need a classifier API when they can write a zero-cost regex and prompt-length heuristic in Python, matching words like sequence, modulo and race condition or escalating any prompt over 35 words.

On the 16 development questions used to write those exact rules, the local heuristic scored 15 of 16, or 93.8%. To test whether that heuristic survived contact with realistic coding-agent traffic, we froze its code without changing a character and added 34 unseen held-out tasks across seven coding-agent buckets: fact extraction, schema validation, short classification, single-hop code checks, state simulation traces, math or logic chains and subtle language-semantics bugs.

On the 34 held-out questions, the frozen heuristic dropped 23.2 percentage points to 70.6%. Realistic simple tasks such as extracting a port from a Kubernetes YAML manifest, a file path from a unified git diff header or a line number from a Python traceback exceeded 35 words and triggered false escalations. Conversely, concise multi-step code traces such as mutable default arguments across calls, Python dictionary key hashing on {1: "a", True: "b", 1.0: "c"} and C short-circuit evaluation lacked the heuristic's hardcoded puzzle words and were falsely routed to the simple path. Jev classified all 34 unseen tasks with 100.0% agreement, 34 of 34, reaching 48 of 50, or 96.0%, across the full suite.

2. MaxOutputTokens is a ceiling, not a thinking-budget dial

A common mistake when building a fast-path router is assuming that setting a low maxOutputTokens cap on a reasoning model will make it spend less on easy tasks. Our 50-question data shows why that assumption fails.

When we compared gemini-3.8-flash with maxOutputTokens=300 against gemini-3.8-flash with maxOutputTokens=4096 while leaving thinkingConfig at the API default, both configurations spent almost identical thinking tokens on the 27 tasks Jev labeled SIMPLE:

Note: Figures reflect single-run experimental measurements on a sample test suite; actual latencies, token counts, and billed costs will vary based on prompt length, context caching, and production load.

Capping the model's output at 300 tokens didn't save any money on easy tasks. The cap is only a ceiling, and easy answers never came close to it. On the easy tasks, the capped version actually cost slightly more than the uncapped one. All the cap did was break 7 of the 25 hard tasks. On six of them, the model ran out of room while still reasoning and never gave an answer. On the seventh, it squeezed its reasoning to fit, finished on time and got the answer wrong.

So a Jev router that sends easy tasks to the capped model and hard ones to normal Flash got all 50 right. It still cost slightly more than skipping the router and using normal Flash for everything, and it was slower. If your cheap option doesn't actually make the model think less on easy tasks, no router can save you money. Not even a perfect one that costs nothing.

3. Benchmark Jev against the strongest setup without a router

The obvious way to spend less on thinking is to ask the model to think less. So we ran a second round of tests with two lighter options: Flash-Lite, which doesn't think at all by default, and Gemini 3.8 Flash with its thinking level set to low. We added those results to the same results file, matched question by question.

A note on timing. Most of the router's time per task is spent in Flash-Lite and Flash calls, which we measured in the same run and time window as the plain Flash baseline, so the speed comparison between them is fair. Jev's own decision is a small share of the total and comes from the first round. Speed comparisons against default Flash or Pro mix the two rounds, so read those as approximate.

Note: Figures reflect single-run experimental measurements on a sample test suite; actual latencies, token counts, and billed costs will vary based on prompt length, context caching, and production load.

The 7-bucket breakdown shows where gemini-3.5-flash-lite and the heuristic router missed tasks and how Router Jev: Flash-Lite to 3.8 LOW recovered 100% accuracy:

The strongest setup without a router was Gemini 3.8 Flash on low thinking. It solved all 50 tasks. That's the number a router had to beat.

The Jev router beat it. It sends every task to Flash-Lite first and escalates only what Jev flags as complex. It also solved all 50 tasks, and it can help reduce both cost and response time compared with running low thinking on everything. On prompts this short the cost difference per task is small, so for an interactive agent the shorter wait is the bigger benefit.

Why it works. On its own, Flash-Lite answered every simple task in half a second but missed 6 of the 25 hard ones. Jev flagged all 6 as complex with high confidence and escalated them. That brought its cost close to what a perfect router would achieve.

The keyword heuristic was cheaper, but only because it got things wrong. It sent 5 hard tasks to Flash-Lite, and 2 of them failed, for 48 of 50.

Escalating to a heavier setting didn't pay here. Routing hard tasks to Flash on default thinking was cheaper than running default thinking everywhere. It still cost more than running low thinking on everything, which already solved all 50. A heavier escalation target only makes sense when the low setting starts failing.

Limits: one run per setup, 50 prompts with hand-assigned difficulty and simple automatic grading.

4. Price per token doesn't tell the whole story

Two things in our results only show up once you look past it.

Routing and caching pull in opposite directions. Our test prompts were short, standalone tasks, and on those the Jev router helped reduce both cost and response time. Real coding agents often look different. They send the same big block of context on every turn, such as a map of the repository and the tool definitions. Agent Platform Context Caching stores that block so you pay 75% to 90% less to resend it. The catch is that a cache belongs to one model. If your agent switches between Flash-Lite and Flash from turn to turn, it either keeps two caches or loses the discount. The practical split: route short, self-contained tasks like log parsing or schema checks, and keep long, context-heavy turns on one model.

Match the model to the workload. Gemini 3.1 Pro is built for deep, multi-step reasoning, and it brings that same depth to every request, including short lookups that don't need it. For a workload made mostly of quick checks and extractions, a Flash configuration is the better fit, and Pro is best saved for the tasks that need its reasoning depth. Model configurations can also apply different safety settings to security-related prompts. In our run, the Pro configuration declined two short code-security review prompts that Flash answered. If your agent performs security reviews, test those prompts on the configuration you plan to use.

Cost per successful task needs the right denominator

Other ways to build the router. Jev is one way to get a fast classifier, but not the only one. The Getting started with discriminative models codelab runs the same Choice, Score and Noul questions on DiffusionGemma, Google DeepMind's open-weights model, hosted on a Compute Engine GPU VM or on Cloud Run in your own Google Cloud project. An open-source server exposes the same API in front of it, so the routing code in this article only needs a different endpoint. Our results come from Jev, so measure accuracy and latency on your own tasks before switching, and keep in mind that a self-hosted model is billed by the GPU-hour rather than by the token.

Taken together, the results give three practical rules.

Start with Flash on low thinking before you build a router. It solved all 50 tasks and cost considerably less than the default setting. Lowering the output token limit is not a substitute, because it doesn't reduce how much the model thinks on easy tasks. Set the thinking level directly, with thinkingLevel: "LOW".

Use Jev to route short, standalone tasks. Send simple tasks to Flash-Lite and hard ones to Flash on low thinking. Ask Jev one simple yes-or-no question per factor. In our test, this helped reduce cost and response time compared with running Flash on low thinking for everything, and came close to what a perfect router would achieve. Keep long, context-heavy turns on a single model so caching keeps working. For high-volume warehouse workloads, Jeff Nelson's BigQuery and Cloud Run cascade pattern applies the same idea.

Measure what your agent actually spends. Send your agent's traces and thinking-token counts to Cloud Logging and BigQuery, so you can see which tasks drive your spend and catch truncation or retries that inflate it. The benchmark code will be published after open-source review.

In summary, the most cost-effective way to run an agent is to match each task to the model that fits it. Start with a fast model on a light setting. You'll often find it already gets the job done. Then add a small, almost free classifier like Jev to send only the hard tasks to a stronger model, and ask simple yes-or-no questions rather than one broad one. Judge every option by what it costs per task done right, not per token.

Previous

Next