Skip to main content
← Back to news

MiniCPM-V 4.7 35B-A3B: the first smooth on-device open-source MoE, 13 tok/s on 3GB of RAM

Model releases
Source:Hugging Face
OpenBMB releases MiniCPM-V 4.7, 35B-A3B variant: the first open-source MoE able to Decode smoothly on a smartphone without a quality compromise — 13+ tokens/second with only 3GB of memory on an iPhone 17 Pro Max. The secret: Lookahead Expert Scheduling on llama.cpp-omni, combined with fine-grained layer-wise Quantization and speculative sampling. Day-0 support for Rockchip, Intel, AMD and Arm; Ollama, llama.cpp, vLLM and SGLang.
Read original article

Related articles

MiniCPM-V 4.7: the 35B MoE that runs on a smartphone with 3 GB of RAM

The first open 35B MoE (3B active) that runs fluidly on a smartphone: 13+ tok/s with 3 GB of RAM on an iPhone 17 Pro Max, OCR, video and a 21.8 GB GGUF Q4_K_M for desktop. The numbers, the three techniques, the limits.

vLLM: serving models fast

vLLM is the open-source engine that turns a local model into a concurrent server: PagedAttention, continuous batching, installation and the concepts (throughput, quantization, KV cache) for choosing when to use it instead of Ollama.

Llama 3 and Llama 4: Meta's open family — which model runs on your hardware

From the 3.1 8B to the 4 Scout MoE with a 10M context: the Llama family in a table, what runs on 16, 48 and 128 GB of RAM, the Community license translated for the Italian business and the choice by budget.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

MiniCPM-V 4.7 35B-A3B (MoE)

Models

The first open-source MoE of OpenBMB's MiniCPM-V series: 35.2B total parameters, ~3B active per token, multi-image and video understanding with early visual-token compression (LLaVA-UHD v4). First open-source MoE with fluid on-device decoding — 13+ tok/s on 3GB of RAM on an iPhone 17 Pro Max — thanks to Lookahead Expert Scheduling (llama.cpp-omni), fine per-level quantization and DSpark speculative sampling. Day-0 support for Rockchip, Houmo, Intel, AMD and Arm; runs on Ollama, llama.cpp, vLLM and SGLang.

🐑

Llama 4 Maverick (MoE)

Models

Il modello MoE grande della famiglia Llama 4 di Meta: 400B di parametri totali con 17B attivi (128 esperti). Contesto 1M di token, nativamente multimodale (testo+immagini), licenza comunitaria Meta. Richiede multi-GPU o macchine con memoria unificata 128GB+ anche quantizzato.

🐑

Llama 4 Scout (MoE)

Models

The compact MoE of the Llama 4 family: 17B active parameters out of 109B total (16 experts). Meta's community license allows free use under 700 million monthly users. One of the best entry points into the Llama 4 family.

🖥️

AMD Ryzen AI Max+ 395 (Strix Halo)

Hardware

La APU AMD con la massima memoria unificata su x86: fino a 128GB di LPDDR5X condivisa tra CPU (16 core Zen 5), GPU integrata (40 CU RDNA 3.5) e NPU. Fino a 96GB allocabili alla GPU: modelli fino a 100B+ a 4 bit senza GPU dedicata, su mini-PC o laptop. Bandwidth 256 GB/s.

Browse all resources