Local AI Models on Apple Silicon
Autonomous coding agents on your Mac. Coding vs reasoning models, real-world RAM after macOS overhead, and tok/s by memory bandwidth. Rebuilt in September 2026 around models you can run today — and around numbers measured on an M5 Max rather than quoted.
Coding Models
Qwen 3.6 / 3.8
Alibaba · Apache 2.0 · measured on an M5 Max
Qwen3.6-35B-A3B · 8-bit
Qwen3.8-27B · 8-bit
Qwen3.8-27B · 4-bit
The 35B-A3B is the coding pick, and not by a small margin. Head to head on the same Mac it scored 41/41 on executed code tests against 33/41 and 39/41 for the dense 27B, and finished the suite 5.5–6.5x faster. Take the 27B for vision and long-context work instead.
Gemma 4
Google · vision + tool calling · 256K context
Gemma 4 31B
Gemma 4 26B-A4B
Gemma 4 12B
Roughly 90% faster on Apple Silicon since the July MLX update. In an independent local matrix the 31B topped the hard coding tier at 100% — but at a throughput score of 22 against Qwen3.6’s 144. Accuracy per token, not per minute.
The rest of the field
Worth knowing, not necessarily worth installing
Muse Glimmer-30B
poolside laguna-s-2.1
GLM-4.7-Flash
Glimmer wins seven of ten published benchmarks against Qwen3.6 and is dense, so all 32.3B fire per token — a gap invisible on a table and obvious on your machine. GLM-4.7-Flash scored 0% on hard coding and 46% on tool use locally; benchmark position is not local behaviour.
Reasoning Models — and What Actually Fits
Qwen 3.8
Alibaba · thinking on by default
Qwen3.8-27B
Qwen3.8-Flash-Next 180B/6B
Flash-Next at 4-bit takes essentially a whole 128 GB Mac, with a 32 GB embedding table streaming from SSD. It runs. It also leaves no room for an embedder, a media model or a build.
DeepSeek V4
MIT · 1M context · replaces the R1 distills
V4-Flash 284B/13B
V4 Pro
The R1 distills this deck once recommended are superseded. V4-Flash needs more memory than a 128 GB Mac has at its shipped precision — it is a Mac Studio model, or a rented one.
Small enough to be useful
When the big model is busy or absent
gpt-oss-20b
Phi-4-reasoning-plus
Kimi K3 2.8T/104B
Phi-4 remains the current Microsoft release — there is no Phi-5, whatever a model list tells you. Kimi K3 is listed only to close the question: 1.56 TB of weights is not a Mac decision.
macOS Memory Overhead
Metal wires about 75% of unified memory by default. sysctl iogpu.wired_limit_mb reads 0 until you change it — 0 means the default, not unlimited.
64 GB
Total Unified Memory
~48 GB wireable
128 GB
Total Unified Memory
~96 GB wireable
512 GB
Mac Studio M5 Ultra
~384 GB wireable
Measured, M5 Max 128 GB, September 2026. Qwen3.6-35B-A3B at 8-bit is 37.75 GB of weights and 40.8 GB wired once used.
Load it alongside Qwen3.8-27B at 4-bit, both declared at a 262K context, and the pair holds 53.83 GB of weights — with only 21.1 GB wired.
MLX allocates the KV cache lazily, so on a Mac declaring the full context is nearly free and using it is what costs. That is the opposite of llama.cpp and vLLM, which pre-allocate the window at load.
Raising
iogpu.wired_limit_mb works and resets on reboot — a shelf that only fits because of a command someone typed once is not a shelf.
RAM Footprint — vs What macOS Will Wire
Bars marked measured are weights as LM Studio reports them on an M5 Max, September 2026, each declared at a 262K context.
The rest are published figures at the quantisation named. The row that matters is the seventh: two models, both at full context, in 54 GB —
which is why a 128 GB Mac does not have the memory problem this deck assumed in March. It has a clock problem instead.
Memory Bandwidth & Tokens/Second
Token generation is memory-bandwidth-bound. tok/s ≈ bandwidth ÷ model_size_bytes. Prefill is compute-bound (M5 neural accelerators: 4x faster TTFT).
| Chip | Bandwidth | 14B Q4 | 32B Q4 | 70B Q4 | 70B Q8 |
|---|---|---|---|---|---|
| M4 Pro | 273 GB/s | ~45 t/s | ~22 t/s | won't fit | won't fit |
| M4 Max | 546 GB/s | ~80 t/s | ~40 t/s | ~28 t/s | ~16 t/s |
| M5 Max | 614 GB/s | ~90 t/s | ~48 t/s | ~32 t/s | ~18 t/s |
| M3 Ultra | 800 GB/s | ~110 t/s | ~55 t/s | ~50 t/s | ~28 t/s |
| M5 Ultra | 1,228 GB/s | ~140 t/s | ~80 t/s | ~65 t/s | ~36 t/s |
Approximate MLX tok/s for token generation. llama.cpp: 20-50% slower. MoE models faster (only active params read). Measured on an M5 Max, September 2026, each model alone at a 262K context and one serving slot: Qwen3.6-35B-A3B 8-bit (37.8 GB) decodes at 78.6 tok/s with a 224 ms median time-to-first-token; Qwen3.8-27B at 4-bit (16.1 GB) manages 27.2 tok/s; the same model at 8-bit (29.5 GB) manages 17.5 tok/s. Read those three together and the rule of thumb breaks in the useful direction: the largest file on disk is the fastest model on the machine, because a MoE reads about 3B parameters per token while a dense 27B reads all 27B. Footprint predicts what fits. Active parameters predict what is quick.
Best Models by Mac Tier
Mac Mini
M4 Pro
64 GB · 273 GB/s
~48 GB wireable
~$1,400
vs frontier~70%
- Qwen3.6-35B-A3B (Q4) MoE · 3B active · ~21 GB ~55 t/s — still the right first model
- Qwen3.8-27B (4-bit) Vision + 262K · 16.1 GB ~12 t/s — dense, so it reads everything
- Gemma 4 12B Vision + tool calling · ~8 GB ~40 t/s
Best combo: Qwen3.6-35B-A3B Q4 + Gemma 4 12B (29 GB), leaving room for a long conversation.
Recommended
MacBook Pro
M5 Max (40-core GPU)
128 GB · 614 GB/s
~96 GB wireable
measured: 78.6 tok/s · 224 ms TTFT
~$5,100
vs frontier~85%
- Qwen3.6-35B-A3B (8-bit) 37.75 GB measured · 262K context · reads images 78.6 t/s · 41/41 on executed code tests
- Qwen3-VL-Embedding + Reranker 2B Retrieval over your own documents · ~9 GB warm alongside the main model
- Qwen3.8-27B (8-bit), on call 29.53 GB measured · vision and long context 17.5 t/s — load it for a reason, not by default
Best: the 35B-A3B warm at 8-bit, retrieval warm beside it, the 27B loaded only when a task needs eyes or a very long window. That is 47 GB resident with half the machine still free.
Mac Studio
M5 Ultra — shipping Sept 22, 2026
256 GB · 1,228 GB/s
~192 GB wireable · 512 GB in late Oct
2x the bandwidth of M5 Max
from ~$5,499
vs frontier~90-95%
- Qwen3.8-Flash-Next 180B/6B ~103 GB at 4-bit · cramped on 128 GB, comfortable here fast for its size — 6B active
- DeepSeek V4-Flash 284B/13B ~160 GB FP8 · 1M context · MIT the first model that needs this machine
- Two big models at once Flash-Next + 35B-A3B 8-bit = ~141 GB no unloading between tasks
Best: the 512 GB configuration is the first Mac that holds a 400B-class model in memory at usable speed. Buy it for that, or do not buy it.
Inference Stack — Performance Hierarchy
| Runner | Best For | Speed |
|---|---|---|
| vllm-mlx | Production serving, continuous batching, paged KV cache | Fastest overall · 2x llama.cpp · 6x Ollama (8 concurrent) |
| MLX (mlx-lm) | Single-stream inference, zero-copy unified memory | 20-50% faster than llama.cpp · purpose-built for Metal |
| MLC-LLM | Cross-platform deployment | Slightly behind MLX |
| Ollama | Easiest setup, OpenAI-compatible API | llama.cpp backend · best DX for agent integration |
| llama.cpp | Portable C++ with Metal backend | Good, but Metal is an add-on — not native like MLX |
| llamafile | Single-file distribution (cosmopolitan libc) | Same as llama.cpp · convenience, not speed |
No C++ engine beats MLX on Apple Silicon. MLX was built by Apple specifically for unified memory + Metal; llama.cpp added Metal support retroactively. Measured head to head on the same model and quantisation, MLX wins decode at every context past short and ingests novel mid-range context about twice as fast. One practical note for anyone serving rather than chatting: LM Studio's /api/v0/models reports whether a model is loaded, at what context length, on which engine and at which quantisation. A plain OpenAI-shaped listing reports none of that, and a server will happily answer for a model you did not think was loaded.
Solo Developer Recommendation
M5 Max MacBook Pro
128 GB is still the sweet spot — for a new reason.
128 GB is still the sweet spot — for a new reason.
March’s version of this page recommended a 70 GB reasoning model plus a small triage model, sized so the pair just fit. Measurement retired that plan. On this machine the constraint is not memory — the whole shelf fits with room to spare — it is wall-clock. A sparse model activating 3B parameters per token answered every task in a five-task executed-code suite in 329 seconds; the dense 27B usually recommended for coding scored lower and took 1,804 s at 4-bit and 2,147 s at 8-bit. Buy the memory so you stop thinking about it, then spend your attention on how long a turn takes.
~$5,100
TOTAL STARTUP
ROI in ~17 months vs API
Hardware
MacBook Pro M5 Max
128 GB · 614 GB/s · ~96 GB wireable
Stack
LM Studio + MLX
/api/v0/models verifies what answered
Primary — everything
Qwen3.6-35B-A3B 8-bit
37.75 GB · 78.6 t/s · 41/41 · reads images
Retrieval
Qwen3-VL-Embedding 2B
~9 GB warm · indexes page images too
On call
Qwen3.8-27B 8-bit
29.53 GB · vision, 262K · load for a reason
API Budget
$20–50/mo for hard tasks
frontier calls, with a ceiling that stops
Why not a 64 GB Mac Mini?
~48 GB wireable at 273 GB/s. It runs the 35B-A3B at Q4 perfectly well and makes a fine second inference box. It stops being fun the moment you want a second model warm.
Why not the M5 Ultra Mac Studio?
It ships September 22, 2026 from $5,499, with 512 GB in late October — “not shipping yet” is no longer the answer. Buy it to keep Flash-Next or DeepSeek V4-Flash resident. For one developer running a 3B-active model, the extra bandwidth buys less than it looks.
Why not the biggest model that fits?
Measured here: the dense 27B at 4-bit spent 602 seconds and 63,727 characters of reasoning on a CSV parser and returned nothing at all. A model that cannot stop thinking is worse than a smaller one that answers. Cap output tokens and give every turn a deadline.