Results from Muse Spark Open, Meta’s newly released 816B (42B active) open-weights model, are now live on AA-AgentPerf. In our initial testing, the model reaches up to ~150M tokens/s throughput per MW of power used on 4x NVIDIA GB300 or B300 GPUs

Muse Spark Open scores 39 on the Artificial Analysis Intelligence Index. It is an MoE with 62 layers, of which 46 use 2,048-token sliding-window attention, cutting the KV cache requirement at 1M context by ~75% vs. full attention. Meta published an NVFP4 checkpoint and AMD published an MXFP4 checkpoint, which we use for the NVIDIA and AMD hardware respectively

Initial results:

➤ NVIDIA B300 and GB300 lead, each reaching ~150M tokens/s per MW in throughput-oriented (lower per-user speed) configs. GB300 pulls ahead at higher output speeds (~108M vs ~92M at 100 tok/s), but both have only been tested in aggregated serving configs so far, and we expect GB300's lead over B300 to grow with multi-node disaggregated configs

➤ Serving configs are improving quickly. On the same B300 node, NVIDIA's tuned SGLang configs deliver ~1.6-1.8x the throughput per MW of the launch-week cookbook configs at 50-100 tok/s. The gains come from TP4 deployments and a prefill-decode interval tuned per concurrency, at the cost of longer time to first token

➤ AMD MI355X reaches 70M tokens/s per MW at 65 tok/s, ~0.9x B200 between 65 and 100 tok/s. AMD's Oct 22 recipe update (prefill-decode and scheduler intervals, more tokenizer workers, more KV cache for full-attention layers) lifted its line ~1.7x per MW over the earlier recipe

We are continuing to test Muse Spark Open serving configurations, including multi-node disaggregation, and will be updating the leaderboard each week with the best results and configs found so far. We will also be publishing deeper reviews of the progress made in optimizing kernels, inference engines and configs for Muse Spark Open