Blog
Articles, guides, and insights on local artificial intelligence
Reading paths
Start here
The in-order path for newcomers to local AI: from basic concepts to installation and your first model.
Optimize performance
Pick the right hardware and get the most out of what you already own: CPU vs GPU, Mac, Raspberry Pi, and fast serving.
Data and privacy
What actually goes over the network with local AI: the data, the security, the compliance, and the control that stays in your hands.
Choosing hardware and model
How to pick the right hardware and the model suited to your use: VRAM, quantization, GPU or CPU and response speed.
14 articles
Nemotron-Parse: NVIDIA's OCR family for reading documents locally
Nemotron Parse is NVIDIA's vision encoder-decoder family that reads PDFs locally: formatted markdown, LaTeX formulas, coordinates and a semantic class for every element, in reading order. v1.1 (Nov 2025), v1.2 (Feb 2026) and 2.0 (Jun 2026), all under 1 billion parameters and re-released on October 20; 2.0 reaches 0.72 ANLS on Indic scripts. Locally with vLLM v0.20 through v0.26: your documents stay on your hardware.
MiniCPM-V 4.7: the 35B MoE that runs on a smartphone with 3 GB of RAM
MiniCPM-V 4.7 (35B total, ~3B active, released October 12, 2026 by OpenBMB and ModelBest) is the first open MoE in the series, vision-language: multiple images, video, OCR. 13+ tok/s with 3 GB of RAM on an iPhone 17 Pro Max, 17+ on Android. Official quantizations BF16, FP8, INT4, AWQ, GPTQ, GGUF; engines Ollama, llama.cpp, vLLM, SGLang. The three techniques: Lookahead Expert Scheduling, fine per-level quantization, DSpark speculative sampling.
Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM
inclusionAI's Ling 3.1 Flash: hybrid MoE 560B total / 25B active, native 1M-token context, FP8 checkpoint, MIT license. What it concretely means for 24GB of VRAM: quantizations, parallelism and API costs.
Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally
Aleph Alpha's Kolibri (78.1B/3.46B, Apache 2.0, FP8) is the European open-weight MoE: reasoning, tool calling and context up to 1M tokens. In FP8 it weighs ~78GB of VRAM: it runs on an RTX 5090 32GB + shared RAM or on unified-memory Macs; for practical adoption the kolibri1 architecture in llama.cpp and Ollama/LM Studio builds are still missing.
gpt-oss: OpenAI's open reasoning model
21B total parameters but only 3.6B active, native MXFP4 weights and 128K of context: OpenAI's first open reasoning model fits in 16GB. What that means in practice.
Local Image Models: from SDXL to Qwen-Image
From SDXL's 2.6B parameters to Qwen-Image 2.1's 7B: how diffusion models work, how much VRAM you actually need, and which model to pick for your hardware.
Italian in local models: what to expect
How many local models actually handle Italian well? The vendor numbers, where they fail, and the half-hour method to measure them yourself.
GGUF: The Format That Keeps Models on Your Hardware
GGUF is the single-file format that ships models for local inference: a header with a signature and version, key-value metadata, a tensor index, and weights aligned for fast mmap loading. Quantization types with real sizes (8B from 14.96 GiB at F16 to 4.58 GiB at Q4_K_M), the filename convention, and the rule for picking a size against your VRAM or RAM.
Quantization: Get More From the Same Hardware
Quantization reduces the precision of the weights (from FP16 to 4–8 bit) so models fit in less VRAM and generate more tokens per second. The levels, the real file sizes of Qwen2.5 14B, the rule for choosing from Q4_K_M, and the limits (code, reasoning, languages other than English).
Qwen-Image 2.1: Generate and Edit Images Locally
Qwen-Image 2.1 unifies text-to-image, editing, and RGBA transparency in a 7B open model (32-layer DiT). Local setup with diffusers, native ComfyUI support, VRAM needs (~14GB in bfloat16), and use cases (stickers, product photos, design workflows).
When a Big Model Isn't Needed: Choosing by Size
The method for choosing model size by hardware: the three limits (memory, speed, perceived latency), the VRAM table for Q4 quantization, the hardware classes, and the cases where the small model wins.
How to Read a Model's Spec Sheet: Family, Size, Context
A guide to reading a model's specs: family, total and active size, context window, quantization, benchmarks and license, with a worked example on DeepSeek V4.1 Flash.
Qwen: the open model with the best multilingual support
Qwen is the open family with the best support for Italian and French: generations, sizes, MoE, when to use it, and a quick comparison with Llama.
Llama 3 and Llama 4: Meta's open family — which model runs on your hardware
Llama 3 and 4 in a table: 3B/8B/70B and Scout MoE 109B, RAM required per model, the Community license and the choice by budget.