Skip to main content
← Back to news

TACO: ternary optimizer that cuts fine-tuning memory by 174x

Research
Source:arXiv
University of Central Florida researchers propose TACO, a ternary column-wise one-sparse optimizer for LLM Fine-tuning: it keeps first-order gradients while shrinking persistent optimizer state from 27.7 GB to 0.16 GB on OPT-13B.
Read original article

Related articles

Running LLM Models Locally: From First Install to First Working Model

From zero to a first answering model: what you actually need, Ollama step by step, how to pick a model for your memory, expected speeds per hardware, and the typical problems fixed.

KV cache: why your RAM needs more than just the model

What the KV cache is, the formula to calculate it with real numbers from Llama 3.1 8B and Qwen3 8B, and the three levers to manage it: context, quantization and flash attention.

Fine-tuning: adapting a model to your data with LoRA and QLoRA

When prompts and RAG aren't enough: you adapt the model's weights to your data. LoRA, QLoRA, dataset preparation, and local tools (LLaMA-Factory, Unsloth, MLX) on a single consumer GPU.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

🌱

Bonsai-2 27B

Models

27B reasoning model with ternary weights at 1.72 bits per weight, derived from Qwen3.8-27B with hybrid attention (~75% linear). In GGUF PTQ1_0 it takes 5.95 GB, retains 98.2% of the FP16 intelligence and reaches ~47 tok/s on an Apple M5 Max laptop, with a 262K-token context. Apache 2.0 license. Requires Prism ML's llama.cpp fork (ternary kernels are not in stock llama.cpp).

⚡

DeepSeek V4.1 Flash

Models

DeepSeek's MoE model with 552B total parameters and 8-16B active per token. 1M-token context, MIT license: free commercial use. Built for online serving, too large for local use on a single PC.

💎

Gemma 3

Models

Google's open model family in 1B, 4B, 12B and 27B sizes. 128K-token context, 140+ supported languages and multimedia input (text and images). An excellent choice for RAG and long documents on consumer hardware.

💎

Gemma 3 4B

Models

La taglia compatta della famiglia Gemma 3 di Google: 4,32B di parametri, contesto di 128K token e supporto nativo a 140+ lingue (incluso l'italiano), con input testo+immagini. Licenza Gemma (uso commerciale con condizioni). In Q4_K_M occupa circa 2,9GB: gira su GPU da 8GB e su CPU moderne.

Browse all resources