vLLM releases version 0.29.0: Model Runner V2 by default and support for new models
On-device & edgeRelated articles
Qwen3.8-27B vs Gemma 4 26B A4B: the new frontier at 24GB
A 27.8B dense or a MoE with 3.8B active, a 262K context against 256K and almost the same weight at Q4_K_M: two opposite ways of putting big-model quality on a 24GB consumer card. Which is faster, which goes deeper, and when to pick one over the other.
Decision models in local: JEV-35B and the 3B-active System 1
You don't need a big LLM to decide. JEV-35B (35B, 3B active) in NVFP4 on a 32GB GPU and ej (11.4MB) on CPU show how to do routing, classification and quality control without generating a single token.
Mac Studio M5 Ultra: practical 512GB guide - models that fit and GGUF setup
The 512GB Mac Studio holds a 560B MoE at 4 bit with a large context. This guide covers the 4-bit rule, GGUF setup in three paths, the real bottleneck (the KV cache), and the Mac-vs-API cost comparison with the numbers on the table.
Get the next local-AI briefing
Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.
After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.