← Back to news3 October 2026

Backburner: iPhone offloading for faster local LLMs

On GitHub, an experimental project offloads part of local LLM Inference to a paired iPhone, using the phone's GPU and memory as a secondary accelerator — an idea for anyone wanting more tokens per second without swapping graphics cards.

Read original article

GPU or CPU: when the CPU is enough

Do you really need a GPU for local AI? For many use cases, no: how prefill and decode work, what matters (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure on your own hardware.

What a token costs: five routes, the math, and the break-even point

Frontier API, subscription, inference providers, rented GPU or your own hardware: five routes to tokens, with real monthly costs, the electricity bill, and the calculated break-even point.

KV cache: why your RAM needs more than just the model

What the KV cache is, the formula to calculate it with real numbers from Llama 3.1 8B and Qwen3 8B, and the three levers to manage it: context, quantization and flash attention.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

Newsletter language

I consent to the processing of my data to receive the iaLocale newsletter. Read the Privacy Policy.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

🌱

Bonsai-2 27B

Models

27B reasoning model with ternary weights at 1.72 bits per weight, derived from Qwen3.8-27B with hybrid attention (~75% linear). In GGUF PTQ1_0 it takes 5.95 GB, retains 98.2% of the FP16 intelligence and reaches ~47 tok/s on an Apple M5 Max laptop, with a 262K-token context. Apache 2.0 license. Requires Prism ML's llama.cpp fork (ternary kernels are not in stock llama.cpp).

⚡

Magnitude

Inference

Open-source inference engine (Rust, YC S25) that compiles and tunes its kernels directly on the target hardware: Apple Silicon, NVIDIA, AMD or CPU. On open models it claims up to 2× the speed of llama.cpp. Also aimed at serving agents.

🖥️