Mac Studio M3 Ultra 512 GB
— measured local LLM benchmarks

Results

All MLX rows use the same ~8,000-byte prompt (2,700–2,900 tokens after tokenization) and generate 300 tokens at temperature 0. Prompt processing (PP) and generation are separate measurements — on Apple Silicon they behave very differently and quoting only one of them is how benchmarks lie.

Power column: p90 during generation / maximum over the whole run (load + PP + generation), in watts. † Kimi K2.7 Code ran on llama.cpp (llama-bench, 2,048-token prompt), so its PP figure is not tokenizer-identical to the MLX rows, its load time is a cold-start wall clock including one token, and generation-window power could not be isolated (whole-run p90 was 103.6 W). Rows marked experimental required unmerged draft mlx-lm pull requests pinned to specific commits — not yet in any official release.

What the numbers mean in practice

The practical split is simple. Qwen3.6 35B-A3B and Qwen3-Coder-Next are interactive — 77–95 tok/s feels immediate for chat and coding. The 3 giant models (316–390 GiB on disk) run at 18–25 tok/s: readable-speed generation that no other single consumer machine can attempt at all, because the weights simply do not fit anywhere else. Devstral 2 at 8.9 tok/s is a batch worker — queue it, do not chat with it.

Prompt processing is the number most benchmarks hide. Small MoE models ingested our ~2,800-token prompt at 2,000–2,900 tok/s, while the 400 GiB-class models sit at 145–207 tok/s — so a long repository context takes real seconds before the first output token. If your workload is long-context-heavy, weight PP as heavily as generation speed.

Local Codex agent soak — 29/29 over 32 minutes

We pointed Codex CLI at Qwen3-Coder-Next (4bit MLX) served locally on the same machine and ran one fixed implementation task (29 iterations, 1,909 s total): implement a Python module against an existing test suite and generate a Markdown report. Every iteration finished in a single agent turn with 6/6 tests passing — median 56 s, p90 102 s per iteration. Median machine power during the soak was 31.1 W.

This is a stability check on one constrained, small-repo task — it shows the local-agent loop is solid, not that a local model matches a frontier API model on arbitrary large repositories.

Read this before quoting the table

These are single-run snapshots

Speed varies with quantization, runtime version, prompt, temperature, and OS state. Treat every number as "what one real run produced on this machine", not a guaranteed or average figure.

Speed is not model quality

This page measures throughput only. A faster model is not a better model — pick by output quality first, then check here whether the wait is acceptable.

GLM-5.2 and DeepSeek V4 Flash needed draft mlx-lm PRs

Both failed on the released mlx-lm 0.31.3 and only ran with pinned, unmerged pull requests (#1410 and #1189). They work, but we do not call them officially supported.

The Kimi row is a reference, not an apples-to-apples comparison

Kimi K2.7 Code ran on llama.cpp with a 2,048-token prompt, a different tokenizer and a different load-time definition (cold-start wall clock). Its generation-window power could not be isolated, so it stays blank rather than estimated.

Power numbers are per-window, not "constant draw"

Load and prompt processing spike much higher than steady generation. We publish the p90 during generation and the maximum over the whole run as separate columns — neither is an idle or average figure.

Method & reproducibility

Machine

Mac Studio / Apple M3 Ultra / 32-core CPU / 80-core GPU / 512 GB unified memory / 4 TB SSD

Software

macOS 26.5.1 · MLX 0.32.0 · mlx-lm 0.31.3 · llama.cpp 571d0d5

Protocol

MLX rows: first 8,000 bytes of the llama.cpp README as prompt, temperature 0, seed 42, 300 generated tokens. Kimi: llama-bench -p 2048 -n 300. Power: powermetrics CPU+GPU+ANE at 1 s intervals.

Pinning

Every measured artifact is a specific Hugging Face quantization repo pinned to a specific revision, recorded in the run manifest — so “DeepSeek R1” here means one exact set of weights, not whatever the repo serves today.

Hygiene

One run that overlapped with background disk I/O was discarded as contaminated and re-measured idle. Failed runs (released mlx-lm rejecting GLM-5.2 and DeepSeek V4) were kept in the logs, not replaced with zeros.

Run your own numbers on this exact machine

These benchmarks were measured through the same checkout, access page, and Tailscale connection you get. 7 days from $499 — bring your own models and workload.