Blog
Articles, guides, and insights on local artificial intelligence
Reading paths
Start here
The in-order path for newcomers to local AI: from basic concepts to installation and your first model.
Optimize performance
Pick the right hardware and get the most out of what you already own: CPU vs GPU, Mac, Raspberry Pi, and fast serving.
Data and privacy
What actually goes over the network with local AI: the data, the security, the compliance, and the control that stays in your hands.
Choosing hardware and model
How to pick the right hardware and the model suited to your use: VRAM, quantization, GPU or CPU and response speed.
5 articles
Speculative decoding and MTP: speed for free
Speculative decoding makes local AI faster without changing output quality: a small model drafts the tokens, the big model verifies them. Here's how the draft-verify loop works, MTP heads in Qwen models, EAGLE on Mistral Medium 3.5, and how to enable it all in three commands.
KV cache: why your RAM needs more than just the model
What the KV cache is, the formula to calculate it with real numbers from Llama 3.1 8B and Qwen3 8B, and the three levers to manage it: context, quantization and flash attention.
RAG: ask questions to your own documents
RAG (Retrieval Augmented Generation) is the simplest way to make a local model answer questions about your own documents: embeddings, vector database and retrieval loop.
What is local AI and why you need it
A practical explanation of local AI: running a model on your own hardware instead of the cloud. Differences from cloud AI, the concrete benefits (availability, privacy, predictable cost, full control), honest limits, and a five-minute start with Ollama.
How to Choose Your First Local AI Model
The three criteria for picking your first local model: use case, memory (with the exact formula) and context, with the recommended models for 8, 16 and 24 GB.