VRAM

Video RAM. The dedicated memory of the graphics card (GPU). It is the critical resource for local inference: a 7B model quantized at 4-bit requires about 5 GB of VRAM, a 13B model about 9 GB. When VRAM is insufficient, the system can use system RAM, but with reduced performance.