Mac Studio M5 Ultra: practical 512GB guide - models that fit and GGUF setup

The Mac Studio with the M5 Ultra chip finally arrives in the 512GB unified-memory configuration at the end of October. On one side, that's the news. On the other, that's the "press release" part. The practical part is different: there is now a desktop that holds whole MoE models of hundreds of billions of parameters, at 4 bit, with a large context — no cloud, no API, no rental. In this guide I'll show you what actually fits, how to set up GGUF step by step, why the KV cache is the real bottleneck, and what it costs compared to APIs, with the numbers on the table. (Mac Studio tech specs)

The 4-bit rule

At 4 bit, each billion of parameters weighs about 0.5GB. But for MoE models another number matters: the active parameters per token, which determine speed, while the total parameters determine memory. Memory must hold everything (weights + context + cache); speed depends only on what gets activated.

Model (quantization)	Weights on disk	Remaining space in 512GB
Qwen3.5-35B-A3B (Q4_K_M)	~20 GB	~490 GB
Gemma 4 26B-A4B (Q4_K_M)	~16 GB	~496 GB
Kolibri 1, 78B (Q4_K_M)	47.5 GB	~465 GB
Ling 3.1 Flash, 560B (Q4_K_M)	~290 GB	~220 GB
Dense 200B (Q4)	~100 GB	~410 GB

Two readings. First: MoE with few active parameters (Kolibri, Ling) leave you hundreds of GB free for long context and RAG — that is why 512GB makes sense. Second: Ling 3.1 Flash is the limit case — it fits, but the context must be managed (we'll see why below). The official Ling and Kolibri checkpoints are FP8: they weigh ~572GB and ~78GB respectively, so they do not fit at 8 bit; the practical path is the GGUF quantized at 4 bit. (Ling 3.1 Flash, Kolibri-1 GGUF)

GGUF setup: three paths, choose by model

A. Ollama — if the model is in the library. For everyday MoE (Qwen3.5-35B-A3B, Gemma), this is the fastest path:

ollama pull qwen3.5-35b-a3b:q4_k_m
ollama run qwen3.5-35b-a3b

What you see when it works: the >>> prompt and the first response. If it doesn't: ollama ps shows the RAM in use; if the model goes to swap, it's too big for the configuration — drop a quantization step or shrink the context.

B. LM Studio — the GUI path. You download the GGUF file from the repo, select it, set the context window, and load. It's the right path if you don't want to touch the terminal and the model is supported.

C. llama.cpp — for the big MoE (Ling, Kolibri). Kolibri 1 needs the kolibri1 patch (the architecture isn't in the trunk yet); Ling 3.1 Flash has its own hybrid architecture. Here the terminal is mandatory:

./build/bin/llama-server -m Ling-3.1-Flash-Q4_K_M.gguf \
  --n-gpu-layers 999 -c 32768 --jinja --temp 0.6 --top-p 0.9

What you see when it works: it opens http://localhost:8080 with the prompt field; you type and get the streaming response. If it doesn't: OOM error = lower -c or move from Q4 to Q3; on a unified Mac, --n-gpu-layers 999 is correct because all memory is shared.

Comparison of the three paths: Ollama if the model is in the library and you want to start in a minute; LM Studio if you want the GUI; llama.cpp if the model is new or hybrid and you want maximum control (and it's the only path for Kolibri today).

KV cache: the real bottleneck

The model weights aren't the only thing living in memory. The KV cache — the memory that accumulates the "history" of every token already processed — grows with context. The native context of Ling and Kolibri reaches 1M tokens, but using it whole isn't realistic: at 1M the KV cache would eat a good part of Ling's ~220GB remaining at Q4. In practice you operate between 32K and 256K of context, where the KV cache weighs a few dozen GB and you have headroom left for RAG and data. The point: RAM matters more than the model alone, because it's the context — not the weights — that decides how much the conversation "remembers".

Cost: Mac vs API, done with the numbers

Mac Studio 512GB: $9,999. Amortization over 36 months: ~$278/month. The electricity line is negligible: a unified Mac draws a few tens of watts at idle and a little more under load — at $0.30/kWh you're in the range of a few tens of euros a year, not the bill.

The API, instead, is pay-per-use. Ling 3.1 Flash on OpenRouter costs $0.075/M input and $0.225/M output. The math:

Load	Token in	Token out	API cost/month
Light use (5 req/day of 10K)	~1.5M	~0.5M	~$0.60
Moderate use (1 req/day of 100K)	~3.1M	~0.3M	~0.50
Heavy use (1M in + 100K out per day)	~31M	~3M	~$6

The honest point: the $9,999 Mac pays back in tokens only at enterprise volume — you'd need on the order of 3,000 to 4,000 million input tokens per month for the API to exceed the $278/month amortization. For consumer or small-structure use, the API remains by far the cheaper path. The Mac's value is therefore not the per-token price: it's that the model is always there, it's yours, and no byte of the data leaves — which, for some workloads, is worth more than the saving. If your volume is high and the data can't leave, buying wins. The hardware base — unified memory, bandwidth, comparison with the DGX Station — is in the article Unified memory: from Strix Halo to Mac Studio; the full cost method per path (API, rental, buying) is in How much a token costs.

When it's the right choice

• The data can't leave the perimeter (clients, medical, legal, finance): the API is excluded by default and the Mac is the only desktop that holds a 560B locally.
• You want the model always on, with no meter and no dependence on a provider who can change pricing.
• The volume is high enough to justify the $9,999: in that case the API bill would already be the problem.

Otherwise, for 90% of cases, a smaller MoE on 24-48GB hardware or the cheap API are the sensible choice. The Mac Studio 512GB isn't "local AI for everyone": it's local AI for the cases where the machine itself is the product.

In summary

The Mac Studio M5 Ultra in 512GB is the first desktop that holds whole 560B MoE at 4 bit with a large context, no cloud. The practical rule: weights at 4 bit (~0.5GB per billion) + headroom for context and KV cache < 512GB. For everyday models, start from Ollama or LM Studio; for Ling and Kolibri you need llama.cpp (Kolibri with its patch). The real bottleneck is the KV cache, not the weights. On cost: the Mac pays back in tokens only at enterprise volume — for everyone else its value is permanent availability, ownership, and data that never leaves.

Article generated with AI