Latent Space 实测 Naive-N0.5-Flash：2,100 tokens/s 宣称下的编码表现如何

Editor’s note: With this post, we are launching a new series focused on trying new models and products with unbiased reviews. Reach out with suggestions to tips@latent.space.

Naive-N0.5-Flash is a new open-weights model built for AI coding tasks and tuned to generate as many tokens per second as possible. It was developed by a Chinese company called NaiveAI, with the aim of speeding up engineering and research workflows so that teams can experiment, iterate and solve complex problems faster.

The model is an evolution of Xiaomi’s MiMo-V2.5, which NaiveAI has trained further for long-context reasoning and paired with its own NaiveRT inference engine. Naive-N0.5-Flash arrives with a bold pitch: a model built largely by AI and trained to do AI research itself, with recursive self-improvement as the long-term goal.

To see what changed, it helps to know where it started. MiMo-V2.5 is a 310B-parameter sparse Mixture-of-Experts model with 15B active parameters, trained on 48T tokens. Its language backbone inherits the hybrid design of MiMo-V2-Flash, which mixes sliding-window attention (SWA) with global self-attention.

NaiveAI's recipe starts from those weights. Through continued, multi-stage training, the team adapted the checkpoint to DeepSeek Sparse Attention layers. According to the NaiveAI technical blog, this took roughly 3.25T additional tokens on top of MiMo's original 48T. About 3T of those additional tokens were trained at a fixed 1M context length and with a constant learning rate.

Benchmarking NaiveAI vs. DeepSeek and Z.ai

We benchmarked the ultrafast mode of Naive-N0.5-Flash against DeepSeek V4.1 Flash 1017 and GLM 5.3 Flash, and then gave all three the same app-building prompts.

The trade-off, we discovered, is that Naive-N0.5-Flash takes longer to reason through harder problems, yet it still delivers answers faster overall. However, that comes at a slightly higher cost per question — although that may be an acceptable price for applications where speed comes first.

DeepSeek V4.1 Flash 1017 and GLM 5.3 Flash are two of the most popular models in OpenRouter. Both belong to the Flash model class and demonstrate excellent capabilities in AI-assisted coding.

As of this writing, performance statistics for NaiveAI models are not yet available on Artificial Analysis. Therefore, we report the Intelligence Index scores of the models used in our comparison:

Model Intelligence Index

• DeepSeek V4.1 Flash 1017 (Max): 45

• GLM-5.3-Flash: 43

• Naive-N0.5-Flash: not yet available

Published benchmark results offer some insight into the new model's capabilities. They place its coding ability between the two comparison models, and ahead of both on NL2Repo-Bench. 

The table below shows benchmark scores reported in the official model cards and technical blog posts of the respective models. Sources: GLM-5.3-Flash – Technical Blog, DeepSeek V4.1 Flash 1017 – Model Card, Naive AI – Research. 

Our own evaluations below look at the differences between these models in more detail. 

Speed and verbosity experiments

Speed is the headline feature.¹ NaiveAI promises faster research iteration than comparable models, which matters to developers and AI researchers who need to test ideas quickly.²

We evaluated each model on 10 randomly sampled, text-only questions at each of three difficulty levels: 10 easy questions from MATH-500, 10 medium questions from GPQA-Diamond and 10 hard questions from HLE-Diamond. That makes 30 questions per model. All experiments used the same maximum completion-token limit of 256K.

For each request, we recorded response speed (tokens per second, TPS), the total number of completion tokens, and end-to-end response time. The code for the experiments is at https://github.com/gbeltramo/llm-verbosity.

The tables below report the mean and median across all 30 successful requests for each configuration. Throughput figures come from different sources: OpenRouter for DeepSeek V4.1 Flash 1017 and GLM 5.3 Flash, and server-side measurements for Naive-N0.5-Flash. They should be compared with that in mind.

Key observations

Naive-N0.5-Flash reports exceptionally high throughput, approaching or passing 1,000 tokens per second in both thinking configurations. Because its figures are server-side and the other models' figures come from OpenRouter, we do not claim an exact speed multiple over DeepSeek V4.1 Flash 1017 or GLM 5.3 Flash.

Thinking has a large effect on verbosity. On the hard HLE questions, Naive-N0.5-Flash with thinking on averaged about 108,500 completion tokens, roughly nine times the GLM 5.3 Flash (high) average and 11 times the DeepSeek V4.1 Flash 1017 (high) average.

The medians tell a different story from the means. Across all 30 questions, the median completion for Naive-N0.5-Flash with thinking on was about 3,200 tokens, far below its average of more than 51,000. A small number of requests produced exceptionally long reasoning traces and pulled the average up. The same pattern appears, less sharply, in the other models.

The tables also include end-to-end response time, which is the fairer basis for judging speed. The median question took 3.1 seconds with Naive-N0.5-Flash (thinking), against 5.3 seconds for GLM 5.3 Flash (high) and 10.1 seconds for DeepSeek V4.1 Flash 1017 (high). The means are closer: 45.3 seconds for Naive-N0.5-Flash, 45.0 for DeepSeek V4.1 Flash 1017, and 86.9 for GLM 5.3 Flash. On the hard questions, Naive-N0.5-Flash (thinking) was slower on average than DeepSeek V4.1 Flash 1017 (high), at 89.8 seconds against 79.7.

Two caveats apply. The sample is small, at 10 questions per difficulty level, and we did not grade answer accuracy. Short, fast replies are only useful if they are correct.

App experiments

We also asked the models to build the same web apps from the same initial prompt. We tested six combinations of model and thinking effort, with the thinking effort in parentheses:

• Naive-N0.5-Flash (no thinking)

• Naive-N0.5-Flash (thinking)

• DeepSeek V4.1 Flash 1017 (low)

• DeepSeek V4.1 Flash 1017 (high)

• GLM 5.3 Flash (low)

• GLM 5.3 Flash (high)

The gallery of apps is available online. In summary, the three prompts asked the six model variants to do the following:

1. Create an interactive 3D view of the Earth and the Solar System.

2. Grow a 3D tree from a seeded random generator, with branches that taper and fork, thousands of leaves, and bark and leaf textures.

3. Do a browser port of the book Ray Tracing in One Weekend.

Below we’ve embedded results for the thinking variants of each model.

Naive-N0.5-Flash (thinking)

DeepSeek V4.1 Flash 1017 (high)

GLM 5.3 Flash (high)

Solar system

The first app is an interactive 3D view of the Earth and the Solar System. Viewers can orbit the camera around the scene and watch the planets circle the sun, with controls to speed up or slow down their motion. Every surface is drawn from generated textures, with no image files.

Naive-N0.5-Flash (thinking) produced the best result of the three. Its sun shows sunspots, and the dark side of its Earth glows with city lights at night.

DeepSeek V4.1 Flash 1017 (high) came close behind. Its version lacks an asteroid belt, and its Earth has less textural detail.

GLM 5.3 Flash (high) stumbled on lighting, which makes the planets hard to tell apart. Its speed controls are also too sensitive, so a small nudge sends the planets racing or crawling.

Tree modeling

The second app grows a 3D tree from a seeded random generator, with branches that taper and fork, thousands of leaves, and bark and leaf textures. Wind makes the branches sway and the leaves flutter, and the shadows on the grassy ground move with them. The tree stands under a sky gradient and light fog. A control panel adjusts the seed, branching depth, leaf count, wind strength and direction, and sun position. There’s also a switch to autumn colors and a frame-rate counter sits in the corner.

DeepSeek V4.1 Flash 1017 (high) and Naive-N0.5-Flash (thinking) both produced good results. The surprise was Naive-N0.5-Flash with thinking switched off, which did not produce a usable app at all.

The good results still had small glitches in the wind animation. In the DeepSeek V4.1 Flash 1017 (high) app, thin branches can detach from their parents in strong wind and appear to float. The effect is subtle, but it is there.

The problem is easier to see in the GLM 5.3 Flash (high) app. When the wind intensity is turned up, the whole tree starts to deform and wobble like a jelly sculpture in an earthquake.

Ray Tracing in One Weekend

The third app is a browser port of the book Ray Tracing in One Weekend. It renders the book's cover scene: a vast gray ground, a scatter of small matte, metallic and glass spheres, and three large spheres in the center. One is glass, one a brown matte and one a polished metal. Depth of field blurs the foreground and background. The image builds up progressively on screen, with a progress bar, elapsed time and a rays-per-second estimate. A menu offers the book's earlier milestone scenes. The user can pick a quality preset and download the result as a PNG.

In this more coding-focused scenario, all three models with thinking on produced a working app.

DeepSeek V4.1 Flash 1017 (high) wins because its ray tracer is the most efficient of the three. It also uses the full width of the screen to display the rendered image.

The Naive-N0.5-Flash (thinking) app is correct, but its JavaScript is less efficient.

GLM 5.3 Flash (high) is again the weakest. Its speed is similar to the Naive-N0.5-Flash app, but its interface is more basic.

Pricing

Naive positions N0.5-Flash as a capable, low-cost model. The target prices are $0.10 per million input tokens, $0.40 per million output tokens, and $0.01 per million cache-read tokens.

That puts it in the $0.20 to $0.60 per million output tokens band occupied by several capable Flash models, including the two we compared it against.

The final bill will also depend on how many tokens the model uses. That in turn depends on the thinking budget requested and the difficulty of the questions.

Takeaways

In ultrafast mode, Naive-N0.5-Flash can peak at around 2,000 tokens per second and sustain nearly 1,000 during thinking-heavy tasks.

In our tests, its median response time was several times shorter than those of GLM 5.3 Flash and DeepSeek V4.1 Flash 1017 at their high thinking settings. Mean times were similar to DeepSeek V4.1 Flash 1017.

On complex tasks, it spent far more thinking tokens than the other two models. On easy and medium questions, it consistently replied faster.

In the apps, Naive-N0.5-Flash was much more consistent than GLM 5.3 Flash (high). With thinking on, it also holds its own against DeepSeek V4.1 Flash 1017. It built the most detailed solar system app, tied with DeepSeek V4.1 Flash 1017 on the tree model, and came a close second on the efficiency of the ray tracer.

For developers who value fast iteration over token economy, that speed is the main reason to try Naive-N0.5-Flash.

1

  The speed of Naive-N0.5-Flash depends on an efficient speculative-decoding drafter and a custom inference engine. The reported decoding speeds were measured on a single eight-GPU Blackwell node running speculative decoding in NaiveRT. The drafter was Naive-N0.5-Flash-FP8-Draft, and the setup is tuned for single-stream decode latency rather than batch throughput.

Realized speed depends on how many draft tokens the target model accepts per round. That varies with the checkpoint pair, the prompt, context length, sampling settings, and the content being generated. The reported numbers show this clearly:

- Acceptance rate: 70.6% with thinking disabled, but only 43.4% with thinking enabled.
- Decode speed: about 984 tokens/s in the thinking phase and 1,816 tokens/s in the answer phase.
- Peak rate: roughly 2,100 tokens/s over a single second. Treat this as an upper bound, not a typical rate.

The engine matters as much as the drafter. The developers report that a full speculative round (draft, verify, sampling and commit) takes 3.4 ms in NaiveRT versus 12.3 ms in SGLang, a 72.4% reduction. The thinking-disabled campaign measured about 3.45 ms per round. For the full measurements, see NaiveRT: Performance.

2

  Implications for reinforcement learning: Generating autoregressive rollouts is the main bottleneck in large-scale reinforcement learning. Anything that cuts decode cost, such as the sparse attention (DSA) adopted here, makes rollouts cheaper. Adapting other models to DSA could make that loop considerably more efficient and speed up frontier research.

This fits Naive AI's larger promise: a self-improving AI that can optimize its own inference CUDA kernels and adopt the most efficient architecture for large-scale reinforcement learning training.