Skip to main content
← Back to news

llama.cpp adds decision models via the /v1/systemone endpoint

Decision models
Source:Hugging Face
The llama.cpp server now supports decision models: send a state (text, JSON or a screenshot) plus typed questions, and the model returns a probability for each option in a single forward pass, without generating text. Three question types (choice, score, noul) and six models from 144M (Julia-1) to 27B (OpenJev, Clef), answering in 3-43 ms on an RTX PRO 6000, Apache 2.0 except OpenJev (CC BY-NC).
Read original article

Related articles

Qwen: the open model with the best multilingual support

Qwen is the open family with the best support for Italian and French: generations, sizes, MoE, when to use it, and a quick comparison with Llama.

Llama 3 and Llama 4: Meta's open family — which model runs on your hardware

From the 3.1 8B to the 4 Scout MoE with a 10M context: the Llama family in a table, what runs on 16, 48 and 128 GB of RAM, the Community license translated for the Italian business and the choice by budget.

Qwen3.8-27B vs Gemma 4 26B A4B: the new frontier at 24GB

A 27.8B dense or a MoE with 3.8B active, a 262K context against 256K and almost the same weight at Q4_K_M: two opposite ways of putting big-model quality on a 24GB consumer card. Which is faster, which goes deeper, and when to pick one over the other.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

🦙

Llama 3.1 405B

Models

Meta's largest open model: closed-frontier quality but datacenter requirements. In practice it needs high-VRAM multi-GPU or the cloud.

🦙

Llama 3.1 70B

Models

Frontier-class open model with 131K-token context. Needs 48-80GB VRAM, a Mac with unified memory, or a multi-GPU setup.

🦙

Llama 3.1 8B

Models

The de-facto local workhorse: solid quality, 131K-token context and a huge GGUF quantization ecosystem. The standard choice for chat and assistants.

🦙

Llama 3.2 1B

Models

Small, fast model with up to 131K-token context. Ideal entry point for chat and summarization on modest hardware, great on phones and mini-PCs.

Browse all resources