Tools, frameworks, and models for local AI
7 resources found for "Open-weights"
An 11.4MB (single-file) System-1 decision model built on an e5-small-v2 text encoder. 128MB resident-memory peak, runs on CPU. Given a state and typed questions, it returns a calibrated probability per option in one pass, without generating tokens. English-only.
An open decision model from Autotrust, 35B parameters (3B active), a fine-tune of Qwen3.5-35B-A3B for text classification and risk scoring. In NVFP4 quantization it occupies about 25.6 GB of VRAM: it runs on a 32 GB GPU with no frontier inference needed.
A 5.6B open-weights image model for text-to-image, single-reference editing and multi-reference composition in one model, with placement boxes in the prompt. Qwen3-VL-4B text encoder. In BF16 (11.1GB), FP8 and NVFP4 (5.3GB). FLUX Kommunity license (non-commercial + qualified-user exception).
An online guide matching open-weights models to real hardware: for every GPU, Mac and mini-PC it shows what runs, how much memory it needs and at how many tokens per second, with guides on quantization and memory. The right tool for the own-hardware path.
Serverless platform for running code and AI models on GPUs: you pay for real seconds of use, it scales to zero when idle, and you can run any open-weights model. One of the routes to inference on rented hardware without managing servers.
A 500-million-parameter foundation model from NVIDIA for tabular data: prediction, imputation and generation on structured datasets, with OpenMDW 1.1 support and ready-to-use checkpoints. An example of local AI on business data, not text generation.
A 35B-parameter MoE model with ~3B active per token from the Qwen family, Apache-2.0. It is the fine-tune base of JEV-35B. Suited to 40GB+ GPUs in BF16 or to 4/8-bit quantizations for smaller hardware.
