Tools, frameworks, and models for local AI
16 resources in Software
NVIDIA's Rust CLI for AI-assisted Linux kernel development, focused on local AI: patch validation, builds, boot and tests, designed for backporting patches to older kernels. It supports local back-ends (Claude, OpenCode, Codex) or OpenAI-compatible servers. Apache 2.0, led by Andrea Righi.
The node-based visual workflow for diffusion models: the standard for SDXL, FLUX, video and audio. Maximum flexibility, from simple to professional pipelines.
One-click SDXL image generation, Midjourney-style: zero parameters to manage. The simplest way to produce quality images.
Open-source tool that estimates local model inference speed across 250+ hardware configurations (GPU, Mac, CPU): measured estimates from community benchmarks or derived from similar setups. Use it to choose hardware before buying.
Terminal-free desktop app for downloading and running GGUF models: GUI, built-in model search and a local server. The simplest way to get started.
An online guide matching open-weights models to real hardware: for every GPU, Mac and mini-PC it shows what runs, how much memory it needs and at how many tokens per second, with guides on quantization and memory. The right tool for the own-hardware path.
Open-source proxy that intercepts and displays the contents of HTTPS requests in plaintext: the tool to verify what actually goes over the network from your local AI setup.
GTK4/libadwaita app for GNOME to chat with local models via Ollama, in Rust under GPLv3: it handles Ollama setup, models, and loading local documents and images.
The runtimes that run local models on your hardware: Ollama for simplicity, llama.cpp for portability, vLLM for production throughput. The engines in the catalog are the child resources.
Full ChatGPT-style web interface for Ollama and llama.cpp: multi-user accounts, organized conversations, document management for RAG, per-model system prompts, tools and plugins. Version 0.11.4 (September 2026) makes file metadata travel with the retrieved chunks and runs hybrid search natively in the database: large knowledge bases are much faster. The simplest way to turn your local AI into an everyday product.
A zero-dependency Python tool that recognizes and summarizes every file with a local LLM (Ollama, LM Studio or MLX-LM), fully offline. Detects type by content, extracts structure (CSV, email, scripts, logs) and reports files carrying instructions to the assistant. Works without a model (--no-llm). MIT license.
A Swift/MLX runtime for Kyutai's streaming speech recognition on Apple Silicon. The reference checkpoint is the STT-1B (English and French) in INT8/FP16, with a causal Mimi encoder (32 audio codes every 80ms). Supports INT4/INT8 exports. CC-BY-4.0 weights license.
The graphical frontends for using local models without accounts or cloud uploads: LM Studio, ComfyUI, Fooocus and Open WebUI. The studio in the catalog are the child resources.
100% local macOS dictation app: on-device transcription with Whisper and a rewrite pass with a local LLM (Qwen) on the M-series GPU. Text lands in the target app; no data touches the cloud.
The classic command-line tool for capturing network packets: the most direct way to see who is talking to the outside from your machine.
The graphical tool for capturing and analyzing network traffic: filters, colors, statistics and real-time visualization of every connection.
