The M5 Ultra Mac Studio.

For the past few days, I’ve been testing the (currently) top-of-the-line M5 Ultra Mac Studio with 256 GB of RAM.

I’ll cut to the chase: the M5 Ultra Mac Studio is a dream machine for local AI agents. This computer makes it possible to run personal assistants powered by local models with great performance and no additional cloud costs. If you’ve been skeptical of testing OpenClaw or Hermes Agent with local models because they’d never be even remotely near the intelligence and speed of cloud ones, this Mac will change your mind about that.

Since last Thursday, I’ve been comparing this Mac Studio to its predecessor, the M3 Ultra with 512 GB of RAM, as well as my own desktop gaming PC with an RTX 5090 inside. For its size, price, thermal performance – not to mention Apple’s approach to unified memory – the M5 Ultra Mac Studio has fundamentally changed how I think about models running locally and what they can enable now. A 5090, of course, still has an edge over the M5 Ultra thanks to its higher memory bandwidth. But considering the sheer size of my PC build, as well as its heat and noise, I would prefer an M5 Ultra Mac Studio any day. It also happens to be a Mac, with an operating system that looks nice and doesn’t suck, plus a vibrant app ecosystem. (Windows fans, I’m sorry, but Microsoft software will never get my sympathy.)

As I’ll explore in this article, running the latest Qwen3.8-Flash-Next model on the M5 Ultra Mac Studio has been so nice and fast, I’ve made it my default in both Open Minis for iOS and Hermes Agent. That’s right: the personal assistants I use the most – more than Siri AI, in fact – are now entirely powered by a model running locally on a Mac Studio. Furthermore, thanks to the M5 Ultra’s faster GPU and higher memory bandwidth, these agents start responding more quickly, stay fast at larger context windows, and can run long, multi-turn loops without slowing to a crawl as the session grows. Because of this, I’ve also been using local models in the Codex app on my Mac – either as main threads or subagents orchestrated by GPT-6 Astra – and I’ve had a great experience doing so.

Local subagents running in Codex on the M5 Ultra Mac Studio.

I should note upfront that I’m not an AI developer by trade: I do not train or fine-tune models. I’m a tinkerer at heart, and I’ve been playing around with local AI models for over a year at this point. This summer, I went all-in on local AI usage for a big project I was working on, which I will explain in the following section.

My goal with this article is to provide you with a mix of two things: numbers and visualizations based on the (many) tests I’ve run over the course of four days, and an explanation of my practical use cases for local AI applied to my workflow and how I get things done for MacStories.

Let’s dive in.

Why Local AI?

Let’s address the elephant in the room first: why bother with local AI at all when cloud frontier models are better and often faster?

It’s a fair question. You need expensive hardware to run these models, and by the time you’ve repaid your investment, you could have used the most expensive Anthropic subscription for several years, still saved money, and got better performance in return.

Different people will have different answers to this question. Some might say they use local models because of privacy: they’d rather rely on local intelligence for sensitive data and documents than upload anything to an external cloud. Others might argue that it’s simply cool – and I do not disagree. For some, it’s a work-related task: if you’re an AI developer, it makes sense to have a great local setup for training your own adapters or fine-tuning models.

For me, the journey into local AI has been characterized by a mix of the “cool, why not?” factor of it all as well as considerations about privacy and costs.

As I will share later this week with Club MacStories members, my research and writing setup for the iOS and iPadOS 27 review this summer has been powered and made possible by local AI. Back in June, I created an internal app, called Desk, to organize hundreds of notes, sessions, PDF documents, and clipped webpages related to iOS and iPadOS 27, as well as chapters of the review. By the end of the process, the project consisted of 310 documents. In Desk, a team of agents – all based on DeepSeek V4 Flash, plus olmOCR for PDFs – ran 24/7, for 99 days, to perform the following tasks:

• Transcribe my favorite WWDC sessions (using summarize plus LLM processing)
• Extract features of iOS and iPadOS 27 from clipped webpages, sessions, PDF guides, and my own notes
• Cross-reference features across different sources, and keep track of which features belonged to which chapter of the review
• Extract features and bugs from screenshots I uploaded
• Work with the Notion API to organize everything across multiple databases

One of the views of Desk, the app powered by the Notion API and local AI I used for my iOS 27 review.

The local AI agent runs in my custom Desk app.

When I started working with this setup in early June, I quickly realized that relying on the OpenAI or Anthropic APIs for this kind of always-on, persistent background task would be…cost-prohibitive, to say the least. So I pivoted to local AI, and the result is the iOS and iPadOS 27 review you can read on MacStories. It was all written by me, the old-fashioned human way. But the entire research stack, deep-linking between notes, and keeping track of new features and betas were all performed by my agents, running locally on the Mac Studio, for a total cost of $0.

If you don’t think that’s neat, or a powerful concept to explore, then this article probably isn’t for you – and I understand. Dealing with these models is fiddly, and it’s not something I would ever recommend to someone who (rightfully) just wants to pay $20 to use Claude Cowork. This kind of setup is, by definition, the bleeding edge of AI workflows at the moment.

If you fall on the other end of the spectrum, though, and if you think this kind of stuff is neat…let me tell you: the M5 Ultra Mac Studio is a massive leap in performance for local models powered by MLX, and I have a few examples to prove it.

A Leap for Prompt Processing and Generation

As you may have seen from the announcement and my initial coverage, the M5 Ultra Mac Studio looks identical to the M3 Ultra model it replaces, but it comes with an all-new Apple silicon architecture that uses UltraFusion to connect two dual-die M5 Max chips to form a quad-die architecture, which is a first for the Apple ecosystem. As far as local AI workloads are concerned, there are two areas we have to pay attention to (and which I have been following since my coverage of the M5 iPad Pro for local AI last year): GPU and memory bandwidth.

The M5 Ultra has a next-gen GPU with 80 cores, each with a Neural Accelerator that grants it up to 4.5× the peak GPU compute for AI compared to the M3 Ultra. As for memory, Apple’s unified memory architecture still tops out at 512 GB as before (although that model will come out in late October), but its bandwidth has jumped from 819 GB/s to 1.2 TB/s, or 50% higher than the M3 Ultra.

With these numbers in mind, I started testing the M5 Ultra against the M3 Ultra with 512 GB of RAM and my RTX 5090. I’ll share more details on testing below, but the short version is this: with the M5 Ultra, you spend considerably less time waiting for a model to read your prompt and begin generating a response; and when it does start answering, text appears much faster than it used to on the M3 Ultra. These two improvements alone make the machine viable for modern agentic loops that require fast iteration with a model and, as a result, larger context windows.

In my day-to-day experience with agents running on the M5 Ultra, these improvements to token prefill (or how quickly a prompt can be processed) and token generation are the changes I noticed immediately. When comparing a model running on the M3 Ultra and M5 Ultra side by side with Open Minis on iOS, the M5 Ultra was ~70% faster on average than the M3 Ultra at generating a response. As we’ll see later, having a model such as Qwen3.8-Flash-Next clear 100 tokens/second on short prompts and still write at 60 to 85 with 64K to 256K of context behind it is no joke, and it enables the kind of agentic back-and-forth between you and the model that feels great to use, particularly when tool calls are involved.

Using a local model as my default in Open Minis for iOS. Pictured above: a long-running project, subagents, and local image generation powered by Qwen-Image-2.1, also running on the M5 Ultra.

However, I was more impressed with the performance gains in token prefill. When you use agentic assistants such as Hermes or Codex, a model receives a whole block of instructions that include things like the system prompt, user personalization and session memories, skill and MCP descriptions, and more. Some agents are better than others at trimming the instructions they send, but, generally, whenever you use a modern agent, you’re not starting with an empty context window. Because of this, I’ve never been able to consistently use local models with this new wave of agents: they would work, but I’d stare at an empty screen and a loading indicator for a while before the model would start generating a response. And on every turn of the loop, performance would get worse (because of the larger context of the session), and I’d wait some more time.

In my tests, prompt processing is up 150% on average from the M3 Ultra – a ~2.5× improvement from my previous setup. This change alone makes local models solid choices in apps like Open Minis and Hermes Agent. When I ask Flash-Next on the M5 Ultra to get my tasks for the week with RemCTL, I don’t have to wait around for the agent to process my prompt and Open Minis’: in just a few seconds, it gets to work by reasoning, performing tool calls, and so forth. And when I’m working on a large project, such as the voxel Colosseum demo below, the model is able to process multi-turn loops quickly, dispatch and coordinate subagents, and do it all at 60 to 85 tokens per second as the thread grows longer.

This interactive Colosseum demo was entirely created by Flash-Next running on the M5 Ultra Mac Studio, managed via Open Minis and its harness on iOS.

I’m a big believer in assistants that can agentically perform tasks in addition to answering questions, but in order to feel nice to use, they have to be fast. Over the past few months, I’ve tested several “boutique” cloud providers with Open Minis: Inco, which serves Kimi K3 at over 300 TPS; Cerebras, with Qwen3.8-27B at a whopping 1,800 TPS; and the likes of Fireworks and Baseten, each breaking the 150 TPS barrier. All of those providers feel extremely good to use in Open Minis and Hermes, but they are expensive (I burned through $20 of Inco credits in literally 10 minutes last week), and, of course, all my data is going…somewhere when I use them. When I fire up Open Minis with Flash-Next and the collection of Apple CLIs I’m creating, everything stays local, inside a computer I can see and reboot whenever I want.

Cadu, an upcoming iOS client for Hermes Agent, running live voice mode with Qwen3-TTS and Qwen3.8-Flash-Next as the underlying chat model, with the M5 Ultra as the server.

Most importantly: a model like Flash-Next can be “small” enough to run at higher quantizations on a 256 GB M5 Ultra (I can run 5-bit entirely in RAM; 6- and 8-bit can offload their n-gram tables to SSD with this new architecture) but also intelligent enough to sustain long threads and multiple agentic tool calls.

Text generation at different quants.

For my taste, 5-bit quantization hits the sweet spot on this version of the Ultra with a balance of intelligence, performance, and memory consumption. But I already know that, if I ever get to test a 512 GB M5 Ultra, I’d be really interested to measure performance of the 8-bit quant without SSD offloading.

I have not spent much time tinkering with offloading coding tasks for my various projects to a local model, but I’ve done a few interesting experiments. With this kind of performance, and especially given the ability to stack up to three concurrent Flash-Next sessions with subagents in oMLX with 256 GB of RAM (more later), I can now realistically consider handing off simpler coding tasks to a local model and have frontier cloud ones review their work. For instance, I was able to set up Qwen3.8-Flash-Next in Codex, which lets me use a local model with the Codex harness. This means that I can let a main GPT model orchestrate local subagents, have Flash-Next coordinate its own subagents, or even just use the model from my phone with Codex Remote on iOS.

Using a local model on the M5 Ultra from Codex Remote.

I do not usually rely on image generation, but for the sake of this review: the M5 Ultra chip is an official Apple asset; the wallpaper behind it was generated by Qwen-Image-2.1 locally on the M5 Ultra in 180 seconds, with peak RAM usage of 78 GB.

I’m curious to read more on this topic from actual developers who are getting an M5 Ultra soon. With open-weights models now outperforming on consumer hardware what was considered “frontier” ~10 months ago, and with performance on an M5 Ultra now making agentic coding feasible, I think we’re going to see some fascinating experiments from the MLX community very soon.

M5 Ultra vs. RTX 5090

As you’ll see from the visualizations later in this article, NVIDIA’s RTX 5090 is still faster than Apple’s M5 Ultra despite its “meager” 32 GB of VRAM, for two different reasons.

Prompt processing speeds are dictated by compute: the model reads the whole prompt in one giant matrix multiplication, which is exactly the job NVIDIA’s Tensor Cores were built for. Apple’s new Neural Accelerators (one in each of the M5 Ultra’s 80 GPU cores) narrow the gap, but can’t close it. On a 6,000-token prompt, the M5 Ultra read at ~1,700 tok/s; the 5090 delivered a staggering ~3,000 with the Qwen model I tested in LM Studio. Token generation, on the other hand, is bandwidth: the model writes one token at a time and pulls the entire model back out of memory for each one, so the 5090’s 1.79 TB/s against the M5 Ultra’s 1.2 TB/s gives it a steady ~25% lead at every prompt size. What the 5090 doesn’t have is memory: at 256K, the 5090 only finishes with an 8-bit attention cache. 32 GB of VRAM only goes so far.

There are, however, two problems with this comparison. First, while the 5090 does still edge out the M5 Ultra with smaller models, its lack of a unified memory pool means that I’m limited to the 32 GB of VRAM in the GPU if I want to run a model at blazing-fast speeds. The moment I want to run anything exceeding 32 GB (such as the aforementioned higher Flash-Next quants), the 5090 must offload model layers over PCIe to (much slower) system RAM, and that’s no way to live.

I tested a different Qwen model for the comparisons between Mac and PC.

Second, my gaming PC is massive compared to a Mac Studio that fits on my desk – and I have a compact build with a Lian-Li A3 case. Not to mention how loud and hot it gets when I’m running local models at high context windows: when I walked into my office after some benchmarks had run, it was uncomfortably warmer compared to the rest of my apartment. By contrast, the “diminutive” Mac Studio on my desk was warm to the touch, but it was also appreciably quieter than my 5090, the fans were not spinning as fast or loudly, and, most important, it allowed me to run larger models such as GLM-5.3-Flash locally with decent performance thanks to Apple silicon’s unified memory. In my day-to-day use, when I was running Flash-Next oQ4e all the time, I could never hear the fan of the Studio on my desk unless I placed my ear directly on top of the computer.

Judging by the progress Apple has made in recent years, I wouldn’t be surprised to see an M7 Ultra that outperforms the memory bandwidth of a 5090 in the near future. But that’s a story for another time.

A Note on Testing

Lastly, before we jump into raw numbers and charts: how did I test everything?

Automated tests were conducted with a testing harness I built with GPT-6 Astra, which coordinated multiple instances of Codex across my M3 Ultra and M5 Ultra Mac Studio, as well as my PC with the Codex app for Windows and Computer Use. On macOS, I chose oMLX (version 0.7.0.dev2) as the local backend for MLX models, and ran Qwen3.8-Flash-Next-oQ4e-mtp, GLM-5.3-Flash-MLX-mixed-4_8bit, and Qwen3.8-27B-oQ4e-mtp on macOS Golden Gate 27.0 for the majority of my tests. On Windows, I used LM Studio and Qwen3.8-27B-GGUF with CUDA 12 runtime and with all 66 layers offloaded to the GPU for the full-GPU tests, plus separate tests splitting the model between GPU and system RAM.

Alongside separate experiments with Open Minis’ native subagents, I used a custom testing harness to measure concurrent requests and workflows involving a lead model and multiple helpers, with oMLX serving the Mac models and LM Studio serving the Windows model.

Numbers were collected by Astra over the course of four days, and later visualized by Claude Fable 5.1 and Opus 5 using Anthropic’s upcoming Projects feature, which I was able to test early when working on this story. The interactive visualization was built with pure HTML and CSS based on MacStories’ style, and it includes comments and annotations by yours truly.

Claude’s upcoming Projects feature.

My goal with the following interactive widgets was not only to help you understand the numbers more clearly, but also to visualize what the stats mean in practice. I’m quite happy with the widgets that approximate what different tokens per second feel like, since that’s a metric that’s often tricky to visualize. I hope these animated charts will be more useful than regular “static” ones you’ve probably seen elsewhere (which are also included below).

Visualizing the M5 Ultra

Flash-Next and GLM-5.3 on Two Mac Studios

Let’s start with the comparison I care about the most: the M5 Ultra against the M3 Ultra. In these tests, I used the same models, prompts, and oMLX build. The only difference: the M5 Ultra Apple sent me has “only” 256 GB of RAM.

媒体内容 · 前往原文查看

Reading the promptWriting the answerSame prompt again, cache warm

M3 Ultra512 GBreads 1,163 tok/swrites 47 tok/s3.8 s +2.1 s

Same prompt again, 4,096 tokens reused · 0.52 s

First token 3.62 s

M5 Ultra256 GBreads 2,733 tok/swrites 54 tok/s1.7 s

Same prompt again, 4,096 tokens reused · 0.39 s

First token 1.54 s

0 s5 s

M3 Ultra512 GBreads 1,143 tok/swrites 37 tok/s13.9 s +8.3 s

Same prompt again, cache warm · 3.3 s

First token 13.9 s

M5 Ultra256 GBreads 2,887 tok/swrites 52 tok/s5.6 s

Same prompt again, cache warm · 0.88 s

First token 5.58 s

0 s15 s

M3 Ultra512 GBreads 1,114 tok/swrites 39 tok/s59.7 s +35.2 s

Same prompt again, cache warm · 4.5 s

First token 59.0 s

M5 Ultra256 GBreads 2,732 tok/swrites 73 tok/s24.4 s

Same prompt again, cache warm · 2.1 s

First token 24.2 s

0 s80 s

M3 Ultra512 GBreads 458 tok/swrites 21 tok/s8.4 s +4.9 s

Same prompt again, cache warm · 3.8 s

Thinks from 7.66 s · first visible token 8.03 s

M5 Ultra256 GBreads 1,217 tok/swrites 31 tok/s3.5 s

Same prompt again, cache warm · 1.6 s

Thinks from 2.98 s · first visible token 3.25 s

0 s10 s

M3 Ultra512 GBreads 428 tok/swrites 21 tok/s32.9 s +19.9 s

Same prompt again, cache warm · 4.6 s

Thinks from 32.1 s · first visible token 32.6 s

M5 Ultra256 GBreads 1,107 tok/swrites 32 tok/s13.0 s

Same prompt again, cache warm · 2.2 s

Thinks from 12.5 s · first visible token 12.8 s

0 s40 s

M3 Ultra512 GBreads 448 tok/swrites 21 tok/s140 s +77.5 s

Same prompt again, cache warm · 10.2 s

Thinks from 137.6 s · first visible token 138.8 s

M5 Ultra256 GBreads 1,016 tok/swrites 28 tok/s62.5 s

Same prompt again, cache warm · 11.0 s

Thinks from 60.7 s · first visible token 61.6 s

0 s150 s

Test details

Qwen3.8 Flash-Next oQ4e, exact tested revision: 4-bit default with selected 5-, 6- and 8-bit weights, MLX format, multi-token prediction depth 3, thinking off. GLM-5.3-Flash mixed 4/8-bit, exact tested revision: 4-bit routed experts, 8-bit attention and dense weights, native low reasoning effort, 4,096-token output allowance; its generation rate includes reasoning tokens.

Both Studios: macOS 27.0 (26A428), oMLX 0.7.0.dev2, same model bytes, saved settings, runtime and request bodies. Model loading and warm-up are outside the timings. Reading is time to the first visible token measured by the client; writing runs to the end of the stream. Fresh requests reused no cached tokens. GLM’s 64K run changed one setting on both Macs, a 64 GB in-memory prompt-cache budget instead of 4 GB, and ran with no other model loaded.

媒体内容 · 前往原文查看

M3M3 Ultra512 GB

70tok/s

Reads a 16K prompt1,143 tok/s

Code98 tok/s

First token, 16K prompt13.9 s

M5M5 Ultra256 GB

108tok/s+54%

Reads a 16K prompt2,887 tok/s

Code143 tok/s

First token, 16K prompt5.6 s

M3M3 Ultra512 GB

26tok/s

Reads a 16K prompt428 tok/s

Code26 tok/s

First visible token, 16K prompt32.6 s

M5M5 Ultra256 GB

41tok/s+58%

Reads a 16K prompt1,107 tok/s

Code41 tok/s

First visible token, 16K prompt12.8 s

Test details

Flash-Next, exact revision · GLM-5.3-Flash, exact revision. Same runtime, settings and prompts on both Macs.

媒体内容 · 前往原文查看

Reading the promptWriting the answerSame prompt again, cache warm

64K65,235 tokens

M3 Ultra512 GB59.7 s +35.2 s

Same prompt again, cache warm · 4.5 s

Reads 1,114 tok/s · 9/9 answers correct

M5 Ultra256 GB24.4 s

Same prompt again, cache warm · 2.1 s

Reads 2,732 tok/s · 9/9 answers correct

128K130,781 tokens

M3 Ultra512 GB121 s +70.6 s

Same prompt again, cache warm · 5.4 s

Reads 1,097 tok/s · 9/9 answers correct

M5 Ultra256 GB50.0 s

Same prompt again, cache warm · 3.3 s

Reads 2,654 tok/s · 9/9 answers correct

256K261,856 tokens

M3 Ultra512 GB246 s +143 s

Same prompt again, cache warm · 7.1 s

4 min 5 s · Reads 1,070 tok/s · 9/9 answers correct

M5 Ultra256 GB104 s

Same prompt again, cache warm · 5.8 s

Reads 2,544 tok/s · 9/9 answers correct

0 s300 s

64K61,434 tokens

M3 Ultra512 GB140 s +77.5 s

Same prompt again, cache warm · 10.2 s

2 min 19 s · Reads 448 tok/s · 3/3 answers correct

M5 Ultra256 GB62.5 s

Same prompt again, cache warm · 11.0 s

Reads 1,016 tok/s · 3/3 answers correct

128K126,964 tokens

M3 Ultra512 GB318 s +188 s

Same prompt again, cache warm · 30.3 s

5 min 16 s · Reads 403 tok/s · 9/9 answers correct

M5 Ultra256 GB130 s

2 min 8 s · Reads 997 tok/s · 9/9 answers correct · the same prompt again ran out of memory

256K258,025 tokens

M3 Ultra512 GB664 s

11 min 2 s · Reads 391 tok/s · 9/9 answers correct · the same prompt again ran out of memory

M5 Ultra256 GBNo answer

The same prompt again: also out of memory.

0 s700 s

媒体内容 · 前往原文查看

M3 Ultra · 512 GBM5 Ultra · 256 GB

4K3,578 tokens · first token 4.7 s vs 2.5 s

M3 Ultra59.0 tok/s

M5 Ultra90.7 tok/s

16K15,867 tokens · first token 15.0 s vs 6.2 s

M3 Ultra53.5 tok/s

M5 Ultra87.8 tok/s

64K65,005 tokens · first token 58.9 s vs 23.7 s

M3 Ultra45.3 tok/s

M5 Ultra83.8 tok/s

128K130,522 tokens · first token 121.0 s vs 49.2 s

M3 Ultra37.0 tok/s

M5 Ultra60.6 tok/s

256K261,597 tokens · first token 244.9 s vs 101.5 s

M3 Ultra38.6 tok/s

M5 Ultra74.7 tok/s

Test details

Every request generated exactly 512 tokens: the cap is intentional, so this measures writing speed, not whether the essay was finished. Reading speeds were 861–1,112 tok/s on the M3 Ultra and 2,057–2,771 on the M5 Ultra. Unique prompt prefixes prevented cache reuse between sizes. These are different requests from the short-prompt 70 and 108 tok/s above and are not mixed with them.

Flash-Next, exact revision, thinking off, MTP depth 3, temperature 0, seed 42.

媒体内容 · 前往原文查看

M3 Ultra296 tokens · 65 tok/s · done in 4.8 s

Reading · first token at 0.47 s

Solid State Drives (SSD) and Random Access Memory (RAM) serve distinct but complementary roles in computing systems, primarily differing in volatility, speed, and capacity. RAM is volatile memory, meaning it loses all stored data when power is disconnected. It offers extremely high read and write speeds, allowing the CPU to access active data and instructions almost instantaneously.

The answer goes on; the numbers above are for all of it.

M5 Ultra288 tokens · 103 tok/s · done in 3.0 s

Reading · first token at 0.35 s

Solid State Drives (SSD) and Random Access Memory (RAM) serve distinct roles in computing, primarily differing in volatility, speed, and capacity. RAM is volatile memory, meaning it loses all stored data when power is disconnected. It offers extremely high bandwidth and low latency, allowing the CPU to access data almost instantaneously. This makes RAM ideal for holding active processes and frequently accessed data.

The answer goes on; the numbers above are for all of it.

M3 Ultra291 tokens · 26 tok/s · done in 11.4 s

Thinking, then reading · first visible token at 1.03 s

SSD storage and RAM serve fundamentally different roles in a computer, despite both holding data. An SSD (Solid State Drive) is non-volatile storage: it retains data permanently, even when the power is off. It holds your operating system, applications, and files long-term. RAM (Random Access Memory), by contrast, is volatile memory—it clears completely when the machine shuts down.

The answer goes on; the numbers above are for all of it.

M5 Ultra271 tokens · 42 tok/s · done in 6.8 s

Thinking, then reading · first visible token at 0.57 s

SSD storage and RAM serve fundamentally different roles in a computer, despite both holding data. An SSD (solid-state drive) is persistent storage: it retains files, applications, and the operating system even when the power is off. RAM (random access memory) is volatile working memory: it only holds data while the system is running, and everything is erased at shutdown.
Speed is the key distinction.

The answer goes on; the numbers above are for all of it.

Test details

The text is the model’s actual first answer; each streamed chunk fades in at the time the client received it. GLM’s low-effort reasoning happens before the first visible token and is not shown. Reduced motion shows the whole answer at once.

Flash-Next at Four Quantizations

While I focused on the 4-bit oQ4e build of Flash-Next for the majority of tests in this review, I also put it against the 5-, 6- and 8-bit builds on both Mac Studios. All four ran in a separate session, with three runs each, so the 4-bit numbers here differ a little from the figures above. More bits means a bigger (and more precise) model. The M3 Ultra Mac Studio with 512 GB of RAM holds all four in memory. The M5 Ultra has 256 GB: oQ4e and oQ5e fit, and oQ6e and oQ8e only run with their embedding tables offloaded to SSD, which is how they appear in every figure below.

媒体内容 · 前往原文查看

M3 Ultra · 512 GB, in RAMM5 Ultra · 256 GB, in RAMM5 Ultra, embedding tables on SSD

oQ4enominally 4-bit

M3 Ultrain RAM160 GB

M5 Ultrain RAM155 GB

oQ5enominally 5-bit

M3 Ultrain RAM179 GB

M5 Ultrain RAM179 GB

oQ6enominally 6-bit

M3 Ultrain RAM198 GB

M5 Ultratables on SSD156 GB

Tried in RAM first on the M5 Ultra: the load passed oMLX’s own estimate, then macOS ended the server for memory pressure at 176.6 GB, before any request. The bar is the run with the embedding tables on SSD.

oQ8enominally 8-bit

M3 Ultrain RAM240 GB

M5 Ultratables on SSD187 GB

Tried in RAM first on the M5 Ultra: oMLX refused to load it, projecting 244.2 GB during loading against the 200.4 GB it allows. The bar is the run with the embedding tables on SSD.

0256 GB

Test details

SSD offload is oMLX’s qwen4_ple_ssd_offload setting: the large n-gram embedding tables stay on SSD and entries are read on demand; the rest of the model runs as before, not on the CPU.

Memory is the process’s physical footprint, sampled every two seconds: the whole oMLX process with one model loaded, caches and runtime included, not the weights alone, and a sampled peak can miss a brief higher one. Peaks while loading, sampled the same way: oQ4e 132 GB on the M3 Ultra, oQ4e 134 GB on the M5 Ultra, oQ5e 123 GB on the M3 Ultra, oQ5e 153 GB on the M5 Ultra, oQ6e 174 GB on the M3 Ultra, oQ6e 101 GB on the M5 Ultra, oQ8e 229 GB on the M3 Ultra, oQ8e 130 GB on the M5 Ultra. The M5 Ultra’s oQ5e row comes from a later three-run pass with new prompt identifiers, because the original sampler started late; its timings are not used anywhere on this page.

媒体内容 · 前往原文查看

M3 UltraM5 UltraEmbedding tables on SSD

M3 Ultra512 GB

oQ4ein RAM77.3 tok/s

oQ5ein RAM71.0 tok/s

oQ6ein RAM71.9 tok/s

oQ8ein RAM63.5 tok/s

M5 Ultra256 GB

oQ4ein RAM · retest111.6 tok/s

oQ5ein RAM100.0 tok/s

oQ6etables on SSD95.2 tok/s

oQ8etables on SSD86.8 tok/s

M3 Ultra512 GB

oQ4ein RAM103.9 tok/s

oQ5ein RAM97.3 tok/s

oQ6ein RAM96.6 tok/s

oQ8ein RAM89.1 tok/s

M5 Ultra256 GB

oQ4ein RAM · retest142.9 tok/s

oQ5ein RAM132.6 tok/s

oQ6etables on SSD122.8 tok/s

oQ8etables on SSD119.0 tok/s

Test details

Native oMLX rates, medians of three runs per build, except the M5 Ultra’s oQ4e bars: a later retest of the same prompts, nine prose runs and three code runs. Thinking off, temperature 0, multi-token prediction depth 3, the same runtime on both Macs; the requests were the same apart from the model name.

The prose prompt asked for 180–220 words. Runs that kept an answer inside that range: the M5 Ultra’s oQ5e, 2 of 3; every other run overshot on all three. All 24 code answers per Mac passed their assertions and extra edge cases.

The four builds, at the revisions tested: oQ4e · oQ5e · oQ6e · oQ8e.

媒体内容 · 前往原文查看

Reading the prompt, weights in RAMReading, embedding tables on SSD

M3 Ultra512 GB

oQ4ein RAMreads 1,132 tok/s232 s

oQ5ein RAMreads 1,109 tok/s237 s

oQ6ein RAMreads 1,106 tok/s237 s

oQ8ein RAMreads 1,088 tok/s241 s

M5 Ultra256 GB

oQ4ein RAMreads 2,574 tok/s102 s

oQ5ein RAMreads 2,207 tok/s119 s

oQ6etables on SSDreads 2,394 tok/s110 s

oQ8etables on SSDreads 2,252 tok/s117 s

0 s250 s

Test details

Medians of three runs per build; every fresh request reused zero cached tokens, and every answer passed its retrieval check. The same prompt sent again reused the cache on both Macs for every build, with first tokens between 5.7 and 8.8 seconds; those runs are not drawn here. Reading is the time to the first token measured by the client, as on every other figure; 256 tokens of output were reserved, and thinking was off.

Charts

I’ve also put together some classic line charts, drawn from the numbers measured in these tests. Click any one of them to open it.

媒体内容 · 前往原文查看

Prompt Processing Speed

Flash-Next

M3 UltraM5 UltraOpen

Prompt processing hardly slows as the prompt grows. The M5 Ultra holds between 2,057 and 2,771 tokens a second from 4K to 256K; the M3 Ultra between 861 and 1,112. That flat line is why a long prompt costs time in proportion to its length.

Generation Speed

Flash-Next

M3 UltraM5 UltraOpen

Generation slows as the context fills: the M5 Ultra from 91 to 75 tokens a second, the M3 Ultra from 59 to 39. Both machines pick back up at 256K. These are single runs per size, not medians.

Time to First Token

Flash-Next

M3 UltraM5 UltraOpen

A straight line: double the prompt, double the wait. At 256K the M3 Ultra takes 245 s before it says anything and the M5 Ultra 102 s.

Total Request Time

Flash-Next

M3 UltraM5 UltraOpen

First byte to last, with a 512-token answer every time. At 256K the answer itself is 13.3 s of a 258 s request on the M3 Ultra and 6.8 s of 108 s on the M5 Ultra. Almost all of it is reading.

Generation Speed, Mac vs. PC

Qwen3.8 27B

M3 UltraM5 UltraRTX 5090 PCOpen

The matched test: the same prompts on all three machines, MLX on the Macs and GGUF Q4_K_M in LM Studio on the PC. The 5090 leads at every size it finished, and the M5 Ultra sits closer to it than to the M3 Ultra. The PC line is its default 16-bit attention cache; the bars in the section above use the 8-bit cache that also fits 256K. There is no 256K point for the PC here: that run was stopped after 10 min 1 s without an answer.

Combined Throughput

Flash-Next · 6.5K prompts

M3 UltraM5 UltraOpen

Running requests together costs the M3 Ultra nothing in total throughput and the M5 Ultra gains: 66 tokens a second on its own, 81 across three. Each request is slower, but the Mac gets more work done.

Test details

Charts 1 to 4: Flash-Next oQ4e on both Studios, one run per size from a cold cache, 512 output tokens every time. Prompts of 3,578 to 261,597 tokens for the budgets shown. Flash-Next, exact revision.

Chart 5 is the matched Qwen3.8 27B run from the M5 Ultra vs. RTX 5090 section: single runs, thinking and multi-token prediction off. Chart 6 is the concurrency test from the subagents section, about 6.5K input tokens per request with a 600-token cap, one session per Mac.

The x axis is the context budget, evenly spaced rather than to scale, which is how these charts are normally drawn. Every y axis starts at zero.

媒体内容 · 前往原文查看

Prose Generation Speed

Flash-Next · 180–220-word answer

M3 UltraM5 UltraEmbedding tables on SSDOpen

The M5 Ultra line peaks at oQ4e, 112 tok/s in a later retest of the same prompts, against 100 for oQ5e. On the M3 Ultra the line runs from 77 at oQ4e down to 63 at oQ8e. Open dots ran with their embedding tables on SSD.

Code Generation Speed

Flash-Next · a Python function

M3 UltraM5 UltraEmbedding tables on SSDOpen

Code comes out faster than prose on every build. The M5 Ultra’s oQ4e leads at 143 tok/s, a later retest of the same prompts. oQ8e is the slowest on both Macs: 89 on the M3 Ultra, 119 on the M5 Ultra from SSD.

Prompt Processing Speed at 256K

Flash-Next · 262K-token prompt

M3 UltraM5 UltraEmbedding tables on SSDOpen

Reading a quarter-million tokens: the M5 Ultra between 2,207 and 2,574 tok/s, the M3 Ultra between 1,088 and 1,132. oQ4e reads fastest on both Macs.

Time to First Token at 256K

Flash-Next · 262K-token prompt, cold cache

M3 UltraM5 UltraEmbedding tables on SSDOpen

The wait before the first token, the same runs as the 256K race above. The M5 Ultra’s slowest build, 119 s, is well under the M3 Ultra’s fastest, 232 s.

Peak Memory Footprint

oMLX process while answering

M3 UltraM5 UltraEmbedding tables on SSDOpen

The M3 Ultra line climbs from 160 to 240 GB with everything in RAM. On the M5 Ultra the two open dots are the SSD runs: oQ6e at 156 GB, under oQ5e’s 179 in RAM, and oQ8e at 187. Nothing here is a weights-only size; it is the whole process, caches included.

Test details

Every point is a median of three runs, all from one session; an open dot is a run with the embedding tables on SSD. Memory is the sampled peak physical footprint of the oMLX process. The subagent chart uses service time for the 16K jobs, widths 1 and 4.

The x axis is the quantization, in order of nominal bit depth; these releases differ in bit allocation and grouping, not only in bits. Every y axis starts at zero. The four builds, at the revisions tested: oQ4e · oQ5e · oQ6e · oQ8e.

M5 Ultra vs. RTX 5090

The following tests are based on my gaming PC build: an RTX 5090 with 32 GB of VRAM, 96 GB of system RAM, and the same Qwen3.8 27B running in LM Studio on Windows. For these tests, I used a different model than the one from my Mac comparisons above.

媒体内容 · 前往原文查看

Reading the promptWriting the answer

RTX 5090 PC32 GB VRAMreads 3,031 tok/swrites 59 tok/s5.5 s

First token 2.0 s

M5 Ultra256 GBreads 1,701 tok/swrites 48 tok/s8.4 s +2.9 s

First token 4.0 s

M3 Ultra512 GBreads 414 tok/swrites 31 tok/s22.3 s +16.8 s

First token 15.4 s

0 s25 s

RTX 5090 PC32 GB VRAMwrites 51 tok/s30.8 s

First token 26.4 s · LM Studio reported no reading rate at this size

M5 Ultra256 GBreads 1,501 tok/swrites 39 tok/s48.4 s +17.6 s

First token 42.8 s · 2,048 tokens already cached

M3 Ultra512 GBreads 339 tok/swrites 24 tok/s198 s +167 s

First token 187.7 s

0 s250 s

Test details

PC: RTX 5090 (32 GB VRAM), Ryzen 9 9950X3D, 96 GB of system RAM, Windows 11 Pro, LM Studio 0.4.24, CUDA 12 runtime 2.41.0, NVIDIA driver 616.92, all 66 layers on the GPU. Qwen3.8-27B Q4_K_M, exact revision. Macs: oMLX 0.7.0.dev2, Qwen3.8-27B oQ4e, exact revision. 2,048-token output allowance.

Times are to the first token and to the end of the stream, from the client. The M5 Ultra reused 2,048 cached tokens on the 64K request; LM Studio did not report a native reading rate at 64K, so none is shown.

媒体内容 · 前往原文查看

5090RTX 5090 PC32 GB VRAM + 96 GB RAM

59tok/s

Reads3,031 tok/s

First token2.0 s

Whole answer5.5 s

M5M5 Ultra256 GB unified

48tok/s

Reads1,701 tok/s

First token4.0 s

Whole answer8.4 s

M3M3 Ultra512 GB unified

31tok/s

Reads414 tok/s

First token15.4 s

Whole answer22.3 s

Test details

Same 6,091-token request as the race. Reading is the runtime’s prompt-processing rate; writing is its generation rate; first token is measured by the client. MLX oQ4e and GGUF Q4_K_M are different quantizations of the same 27B model.

媒体内容 · 前往原文查看

M3 Ultra · 512 GBM5 Ultra · 256 GBRTX 5090 PC · 32 GB VRAM

64K63,445 tokens · tok/s

M3 Ultra23.5

M5 Ultra38.9

RTX 5090 PC49.6

128K128,983 tokens · tok/s

M3 Ultra20.0

M5 Ultra32.4

RTX 5090 PC40.4

256K260,037 tokens · tok/s

M3 Ultra15.0

M5 Ultra24.3

RTX 5090 PC30.0

Test details

Single runs, thinking and MTP off, temperature 0, 2,048-token output allowance, same frozen prompts as the race. Each cache setting is its own condition and they are never averaged: with the default 16-bit cache the PC did 51.1 tok/s at 64K, 44.4 at 128K, and was stopped at 256K after ten minutes with 3% of the prompt read. The borrowed-RAM runs keep 50 of 66 layers on the GPU. The M3 and M5 reused some cached input at 128K and 256K (28,672–63,488 tokens), which affects reading time, not writing speed.

The 5090 reported 66/66 layers on CUDA and a CUDA attention cache for the GPU runs; runtime logs still map about 682 MiB of model data on the CPU.

媒体内容 · 前往原文查看

RTX 5090 PC205 tokens · 59 tok/s · done in 5.5 s

Reading · first token at 2.04 s

ORCHID-7391-COBALTTo verify that this recovery code was copied correctly, you should perform a multi-step validation process to ensure integrity and accuracy. First, conduct a character-by-character comparison between your recorded string and the source text, paying close attention to case sensitivity, as “Orchid” or “cobalt” would be incorrect. Second, check for common errors involving visually similar characters; specifically, confirm that the middle segment is the number7391, not letters like B or S, and ensure there are no accidental spaces inserted around the hyphens.

The answer goes on; the numbers above are for all of it.

M5 Ultra221 tokens · 48 tok/s · done in 8.4 s

Reading · first token at 4.02 s

ORCHID-7391-COBALT
To verify that the recovery code was copied correctly, you should perform a character-by-character comparison against the source text. First, confirm the structure consists of three distinct segments separated by hyphens. The first segment must be the six-letter word “ORCHID,” ensuring the capitalization and spelling are exact. The second segment is the four-digit number “7391”; verify that the digits are in the correct order and that no characters have been substituted with visually similar symbols, such as the letter “O” for the zero or the letter “I” for the one.

The answer goes on; the numbers above are for all of it.

M3 Ultra221 tokens · 31 tok/s · done in 22.3 s

Reading · first token at 15.37 s

ORCHID-7391-COBALT
To verify that the recovery code was copied correctly, you should perform a character-by-character comparison against the source text. First, confirm the structure consists of three distinct segments separated by hyphens. The first segment must be the six-letter word “ORCHID,” ensuring the capitalization and spelling are exact. The second segment is the four-digit number “7391”; verify that the digits are in the correct order and that no characters have been substituted with visually similar symbols, such as the letter “O” for the zero or the letter “I” for the one.

The answer goes on; the numbers above are for all of it.

Test details

Same request as the race and the cards. The chunk times are the client’s arrival times; LM Studio and oMLX stream at different granularities.

Subagents and Concurrency

This is the part I was most curious about. Here’s what happens when three requests arrive at once and when a lead model hands work to helpers.

媒体内容 · 前往原文查看

Reading the promptWriting the answer

1 request at once

M3 Ultra512 GB15.6 s +6.6 s

First visible tokens at 6.9 s · 38.4 tok/s combined output

M5 Ultra256 GB9.1 s

First visible tokens at 3.4 s · 66.2 tok/s combined output

2 requests at once

M3 Ultra512 GB31.5 s +14.3 s

First visible tokens at 10.7–18.8 s · 38.1 tok/s combined output

M5 Ultra256 GB17.2 s

First visible tokens at 5.6–8.7 s · 69.6 tok/s combined output

3 requests at once

M3 Ultra512 GB45.1 s +23.0 s

First visible tokens at 9.2–31.9 s · 39.9 tok/s combined output

M5 Ultra256 GB22.1 s

First visible tokens at 5.4–20.6 s · 81.5 tok/s combined output

0 s50 s

Test details

Reading starts at the earliest first visible token among the group; writing runs until the last response ends. Combined output divides every completion token, reasoning included, by the whole group’s time; it is not the sum of native rates. Flash-Next oQ4e, MTP depth 3, thinking on at low effort, 6,523–6,559 input tokens per request, every request capped at 600 tokens on purpose. Native usage reported zero cached input on both Macs. The M5 Ultra also served three simultaneous HTTPS requests through the production iOS bridge: 1,800 tokens in 22.9 s.

媒体内容 · 前往原文查看

Reading the promptWriting the answer

Helpers one after another44.8 s

Lead planswrites 63 tok/s6.7 s

Helper Alphawrites 63 tok/s15.4 s

Helper Bravowrites 64 tok/s24.6 s

Helper Charliewrites 63 tok/s34.4 s

Lead combineswrites 63 tok/s44.8 s

Helpers in parallel32.1 s

Lead planswrites 63 tok/s6.7 s

Helper Alphawrites 44 tok/s19.8 s

Helper Bravowrites 44 tok/s20.6 s

Helper Charliewrites 45 tok/s21.0 s

Lead combineswrites 63 tok/s32.1 s

0 s50 s

Test details

Qwen3.8 27B Q4_K_M in LM Studio, 8,192-token context, three prediction slots, thinking and MTP off, 4,096-token output allowance per call, model reloaded before each run. Helper throughput: 61.0 tok/s serial, 117.8 parallel. Peak sampled GPU memory 17,632 / 17,642 MiB; mean GPU utilization 93% / 88%; peak power 567 / 577 W.

The M5 Ultra for Local AI Agents

As should be clear at this point, the performance gains of the M5 Ultra are real, and they show how Apple’s investment in custom silicon and its unified memory architecture is paying dividends for tinkerers and developers.

Despite my tests, I feel like I’ve barely scratched the surface of what’s possible with the M5 Ultra and its 256 GB of RAM. As more developers and open-source maintainers get their hands (and agents) on the M5 Ultra, I’m sure we’ll see more optimizations in quantization to allow even larger models to run with superior performance on this computer. For instance, I didn’t even have time to test DwarfStar – a fascinating project (made in Italy!) that is making it possible to run local frontier models on all kinds of Mac configurations with even less memory; nor did I have time to check out Inco Splash, a new inference engine designed for Apple silicon and specific models. Likewise, I didn’t have time to test Exo, whose RDMA implementation should (in theory) allow me to split and distribute inference across M3 Ultra and M5 Ultra via Thunderbolt 5, all while running an OpenAI-compatible server in front of it to serve an API for local agents.

And, of course, I can’t even begin to imagine what the high-end M5 Ultra with 512 GB of RAM will allow in terms of scaling up models capable of running locally. I hope to be able to test it eventually, too.

At the end of this experiment, I have a simple, tangible result: the M5 Ultra lets me run local agents with incredible performance, with less time spent staring at a blank screen and everything happening on a single, compact, cool, and quiet machine on my desk.

This would have seemed impossible a couple of years ago. But here we are.