Optimize performance

Pick the right hardware and get the most out of what you already own: CPU vs GPU, Mac, Raspberry Pi, and fast serving.

11 articles in this path

1
1.HardwareIntermediate
Hardware for Local AI: What You Need in 2026
The memory rule (weights + KV cache), real October 2026 prices for GPUs, Apple Silicon and CPU, the 36-month cost math and the choice by budget.
June 10, 2026 · 7 min read
2
2.HardwareIntermediate
GPU or CPU: when the CPU is enough
How prefill and decode work, what matters for speed (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure.
September 21, 2026 · 7 min read
3
3.GuidesIntermediate
Local AI on Mac: the complete Apple Silicon guide
Why Apple Silicon (unified memory, GPU cores) is the simplest machine for local AI: the RAM you need, installing Ollama, recommended models per chip generation, and realistic performance expectations.
September 10, 2026 · 4 min read
4
4.HardwareIntermediate
Local AI on Raspberry Pi
From first install to a model that answers on a Pi 5: real 2026 prices, the tok/s = bandwidth ÷ file formula, the electricity bill and the tricks that matter.
June 10, 2026 · 6 min read
5
5.GuidesAdvanced
vLLM: serving models fast
vLLM is the open-source engine that turns a local model into a concurrent server: PagedAttention, continuous batching, installation and the key concepts for choosing when to use it instead of Ollama.
September 20, 2026 · 7 min read
6
6.FundamentalsIntermediate
KV cache: why your RAM needs more than just the model
What the KV cache is, the formula to calculate it with real numbers from Llama 3.1 8B and Qwen3 8B, and the three levers to manage it: context, quantization and flash attention.
October 1, 2026 · 6 min read
7
7.HardwareIntermediate
A Model Without a GPU: When the CPU Is Actually Enough
In 2026, CPU-only inference is no longer a fallback: when it genuinely works, what speed to expect, and which levers to pull to squeeze the most out of your processor.
October 2, 2026 · 11 min read
8
8.ComparisonsIntermediate
Ollama vs llama.cpp: under the hood
Ollama runs llama.cpp under the hood, but the two are not the same thing: one is the engine, the other the car. What's really underneath, when the difference matters, and how to choose without slogans.
October 7, 2026 · 6 min read
9
9.FundamentalsIntermediate
Speculative decoding and MTP: speed for free
Speculative decoding makes local AI faster without changing output quality: a small model drafts the tokens, the big model verifies them. Here's how the draft-verify loop works, MTP heads in Qwen models, EAGLE on Mistral Medium 3.5, and how to enable it all in three commands.
October 12, 2026 · 6 min read
10
10.ModelsIntermediate
GGUF: The Format That Keeps Models on Your Hardware
GGUF is the single-file format that ships models for local inference: a header with a signature and version, key-value metadata, a tensor index, and weights aligned for fast mmap loading. Quantization types with real sizes (8B from 14.96 GiB at F16 to 4.58 GiB at Q4_K_M), the filename convention, and the rule for picking a size against your VRAM or RAM.
September 30, 2026 · 7 min read
11
11.ModelsIntermediate
Quantization: Get More From the Same Hardware
Quantization reduces the precision of the weights (from FP16 to 4–8 bit) so models fit in less VRAM and generate more tokens per second. The levels, the real file sizes of Qwen2.5 14B, the rule for choosing from Q4_K_M, and the limits (code, reasoning, languages other than English).
September 28, 2026 · 5 min read