VRAM calculator for local LLMs
Pick any of our 42 open-weight LLMs—or enter your hardware budget—to see how much VRAM you need for model weights, conversation KV cache, and runtime overhead, plus which GPUs, Macs, and mini PCs fit it and how fast they answer (our estimate). Every calculation uses measured GGUF download sizes when available, architecture-specific KV cache math from 2,048 to 65,536 tokens, and dated European (€) and US ($) hardware prices.
Or try an example
42 AI models · 21 machines
Save this VRAM check & get alerted when new models fit your selected hardware
Get a weekly 1-line alert when new open-weight GGUFs fit your selected memory ceiling or when used RTX 3090 / 4090 and Mac Studio prices drop >10% — 1-click unsubscribe.
Example calculation: Llama 3.3 70B
Standard quality (Q4), 8,192 tokens, one person. Our estimate.
- 42.52 GB for the model itself (a measured file size).
- 2.68 GB to keep the conversation in mind.
- 0.80 GB for the software around it.
- 46.00 GB in total.
Speeds and memory are our estimate from the hardware specs, not a measured test. Prices are dated and may change. How we calculate
How We Calculate Local LLM VRAM (Weights + KV Cache + Runtime Overhead)
Every memory requirement in this VRAM calculator is built from three additive parts—Model Weights + Conversation Memory (KV Cache) + Runtime Working Space (0.80 GB)—checked against the usable graphics memory of each GPU, Mac, or mini PC. All totals and token-per-second (tok/s) figures are our estimates from public hardware specs and model config.json files, never presented as lab benchmarks.
1. The Total Local LLM VRAM Formula
Total VRAM (GB) = Weights_quant + (KV Cache per user × Concurrent Users) + 0.80 GB runtime overhead
- Model weights (Weights_quant). Measured Hugging Face GGUF / MLX files first: whenever a verified download exists on Hugging Face (for example unsloth, bartowski, or official vendor GGUF trees), our list stores the exact measured decimal GB file size, such as 18.32 GB for Gemma 4 31B Q4_K_M, 16.95 GB for Gemma 4 26B A4B UD-Q4_K_M, 16.46 GB for Qwen3.8-27B UD-Q4_K_M, or 42.52 GB for Llama 3.3 70B Q4_K_M. When a specific quantization file is not measured directly, we estimate weight size from total parameter count (P in billions, including all routed and unrouted experts on mixture-of-experts models), including GGUF container metadata: Q4 (Q4_K_M / UD-Q4_K_M) ≈ P × 0.60 GB per billion parameters; Q5 (Q5_K_M / UD-Q5_K_XL) ≈ P × 0.71 GB; Q8 (Q8_0) ≈ P × 1.07 GB; FP16 / BF16 ≈ P × 2.00 GB.
- Conversation memory / GGUF KV cache. For a standard dense grouped-query attention (GQA) Transformer keeping key and value tensors in FP16 (2 bytes per element), conversation memory for T context tokens is: KV Cache_FP16 (bytes) = 2 (K+V) × n_layers × n_kv_heads × d_head × 2 bytes × T_tokens.
- Hybrid sliding-window and full attention (Gemma 4, Spark-X2.6, LFM3, Nemotron Bolt E4B). Sliding-window layers are capped at their local window (W = 256, 512, or 1,024 tokens) while only global layers scale with full context T. Gemma 4 26B A4B has 30 layers (25 layers on a 1,024-token sliding window with 8 KV heads × 256 dim, plus only 5 full-attention layers with 2 KV heads × 512 dim), keeping its 8,192-token KV cache to 0.38 GB (~0.046 GB per 1K tokens). Dense Gemma 4 31B (50 sliding layers + 10 full-attention layers) takes 1.51 GB at 8,192 tokens.
- Hybrid linear attention (Gated DeltaNet) and Mamba-2 SSM (Qwen3.8-27B, Qwen3.6-35B-A3B, Nemotron 3.5 Super VL). Linear-attention and Mamba state-space blocks use fixed-size recurrent state rather than growing token KV caches. Qwen3.8-27B (48 linear-attention + 16 full-attention layers) needs only 0.54 GB of KV cache at 8,192 tokens (~0.066 GB per 1K tokens), and Qwen3.6-35B-A3B (30 linear + 10 full-attention layers) needs only 0.17 GB at 8,192 tokens (~0.021 GB per 1K tokens).
- KV cache quantization (FP16 vs Q8_0 vs Q4_0). Our calculator defaults to conservative FP16 (16-bit) KV cache so a recommended GPU never runs out of memory out of the box. In llama.cpp (--cache-type-k q8_0 --cache-type-v q8_0) or Ollama (OLLAMA_KV_CACHE_TYPE=q8_0), Q8_0 KV cache cuts conversation memory by ~50% (1 byte per element) with near-zero quality loss, and Q4_0 cuts KV cache by ~75% (0.5 bytes per element).
- Runtime compute buffer and working space (+0.80 GB). Every run adds 0.80 GB for CUDA, Metal, Vulkan, or ROCm scratch buffers, activation tensors, and compute graphs.
- Usable hardware memory ceiling. Discrete GPUs (NVIDIA, AMD, Intel): we subtract 0.60 GB per card for the display and driver (8 GB → 7.4 GB usable; 12 GB → 11.4 GB; 16 GB → 15.4 GB; 24 GB → 23.4 GB; 32 GB → 31.4 GB; dual 24 GB → 46.8 GB usable). Apple Silicon unified memory (macOS): by default, macOS grants the GPU about two-thirds (66.7%) of unified memory up to 32 GB (16 GB Mac → 10.7 GB usable; 32 GB Mac → 21.3 GB usable) and three-quarters (75%) above 32 GB (36 GB → 27.0 GB; 64 GB → 48.0 GB; 96 GB → 72.0 GB; 128 GB → 96.0 GB; 256 GB → 192.0 GB usable), unless raised manually with sysctl iogpu.wired_limit_mb. AMD Ryzen AI Max+ 395 (Strix Halo): counted at 48.0 GB usable on 64 GB mini PCs and 96.0 GB usable on 128 GB mini PCs.
Verified VRAM requirements across 10 popular local LLMs (8,192 tokens, our estimate)
| Model | Parameters and architecture | Q4 weights (Q4_K_M / UD-Q4_K_M) | 8K KV cache (FP16 est.) | KV per 1K (FP16 / Q8_0 KV) | Total at Q4 (+0.80 GB overhead) | Total at Q8 / FP16 (8K ctx) | Minimum GPU / Mac tier that fits Q4 |
|---|---|---|---|---|---|---|---|
| LFM3-1.7B | 1.73B dense (6 global + 18 conv/256-win layers) | 1.10 GB (measured GGUF) | 0.11 GB | ~0.013 GB / ~0.007 GB | 2.01 GB | 2.76 GB / 4.37 GB (4.38 GB BF16) | 8 GB VRAM (7.4 GB usable) or any 4 GB+ laptop |
| Spark-X2.6-4B | 4.11B dense (9 full + 27 512-win layers) | 2.62 GB (measured GGUF) | 0.36 GB | ~0.044 GB / ~0.022 GB | 3.78 GB | 5.56 GB / 9.38 GB | 8 GB VRAM (7.4 GB usable) |
| Llama 3.1 8B | 8.03B dense (32 layers, 8 KV heads × 128 dim; same 8B tier as Qwen3 8B). See also Nemotron 3.5 Bolt E4B at 7.25 GB. | 4.82 GB (est.) | 1.07 GB | ~0.131 GB / ~0.065 GB | 6.69 GB | 10.47 GB / 17.93 GB | 8 GB VRAM (7.4 GB usable; Q8 fits 12 GB VRAM) |
| Gemma 4 26B A4B | 25.2B total / 3.8B active MoE (5 full + 25 1K-win layers) | 16.95 GB (measured UD-Q4_K_M) | 0.38 GB | ~0.046 GB / ~0.023 GB | 18.13 GB (14.8 GB at UD-IQ4_XS) | 28.14 GB / 51.58 GB | 24 GB VRAM (23.4 GB usable; UD-IQ4_XS fits 16 GB VRAM) |
| Qwen3.8-27B | 27.78B dense (16 full + 48 linear-attention layers) | 16.46 GB (measured UD-Q4_K_M) | 0.54 GB | ~0.066 GB / ~0.033 GB | 17.80 GB (12.3 GB at UD-IQ3_XXS) | 31.06 GB / 56.90 GB | 24 GB VRAM (23.4 GB usable; Q8 fits 32 GB VRAM) |
| Gemma 4 31B | 31.27B dense (10 full + 50 1K-win layers) | 18.32 GB (measured Q4_K_M) | 1.51 GB | ~0.184 GB / ~0.092 GB | 20.63 GB (14.1 GB at UD-IQ3_XXS) | 35.77 GB / 64.85 GB | 24 GB VRAM (23.4 GB usable; UD-IQ3_XXS fits 16 GB VRAM) |
| Nanbeige4.3-33B | 33.12B dense (52 full layers, 8 KV heads × 128 dim) | 20.79 GB (measured Q4_K_M) | 1.74 GB | ~0.212 GB / ~0.106 GB | 23.33 GB (14.7 GB at IQ2_M) | 37.98 GB / 68.78 GB | 24 GB VRAM (23.4 GB usable; IQ2_M fits 16 GB VRAM) |
| Qwen3.6-35B-A3B | 35.95B total / 3B active MoE (10 full + 30 linear layers) | 22.13 GB (measured UD-Q4_K_M) | 0.17 GB | ~0.021 GB / ~0.010 GB | 23.10 GB (18.7 GB at UD-IQ4_XS) | 39.43 GB / 72.87 GB | 24 GB VRAM (23.4 GB usable; Q8 fits 48 GB VRAM) |
| Llama 3.3 70B | 70.6B dense (80 full layers, 8 KV heads × 128 dim) | 42.52 GB (measured Q4_K_M) | 2.68 GB | ~0.327 GB / ~0.164 GB | 46.00 GB | 79.03 GB / 144.68 GB | 48 GB VRAM (Dual RTX 3090/4090) or 64 GB Mac (48.0 GB usable) |
| NVIDIA Nemotron 3.5 Super VL | 120B total / 12B active (8 attention + 40 Mamba + 40 MoE) | 71.80 GB (est.) | 0.07 GB | ~0.009 GB / ~0.004 GB | 72.67 GB | 129.27 GB / 240.87 GB | 96 GB (72.0 GB usable, tight) or 128 GB Unified Memory (96.0 GB usable) |
Not sure which model answers your prompt best? Before you buy hardware or download a 20 GB+ GGUF file, test 10 open-weight LLMs free side by side in your browser on localllmrig.com (no login, 0 GB local VRAM required). Full machine-readable formulas and datasets live at /open-data, /api/models, and /about.
Common questions
- How do you calculate how much VRAM a local LLM needs?
- We add three numbers: (1) the model weights at your chosen quantization (Q4_K_M at ~0.60 bytes/parameter or the exact measured Hugging Face GGUF/MLX download size when available; Q5 at ~0.71 bytes/param; Q8_0 at ~1.07 bytes/param; FP16 at 2.00 bytes/param), (2) the conversation KV cache (2 × layers × kv_heads × head_dim × 2 bytes × tokens in FP16, adjusted for sliding-window and linear-attention layers, multiplied by concurrent users), and (3) 0.80 GB of runtime working space for CUDA, Metal, or Vulkan compute buffers. We then check that total against usable hardware memory (-0.60 GB per discrete GPU card, or 2/3 up to 32 GB and 3/4 above 32 GB on macOS unified memory).
- How much VRAM do I need to run 8B, 27B–35B, and 70B local LLMs at Q4_K_M?
- At standard 4-bit quality (Q4_K_M / UD-Q4_K_M) with an 8,192-token context window in our estimate: sub-4B models (LFM3-1.7B at 2.01 GB, Spark-X2.6-4B at 3.78 GB) and 8B models (Llama 3.1 8B at 6.69 GB, NVIDIA Nemotron 3.5 Bolt E4B at 7.25 GB) fit an 8 GB GPU (7.4 GB usable). Compact 3-bit/4-bit builds (Gemma 4 26B A4B UD-IQ4_XS at 14.8 GB, Qwen3.8-27B UD-IQ3_XXS at 12.3 GB, gpt-oss-20b Q4 at 13.11 GB) fit a 16 GB GPU (15.4 GB usable). Standard 4-bit 26B–35B models (Qwen3.8-27B at 17.80 GB, Gemma 4 26B A4B at 18.13 GB, Gemma 4 31B at 20.63 GB, Qwen3.6-35B-A3B at 23.10 GB, Nanbeige4.3-33B at 23.33 GB) fit a 24 GB GPU (23.4 GB usable, such as a used RTX 3090 at €1,246 / $1,395). Dense 70B models (Llama 3.3 70B at 46.00 GB) need 48 GB of VRAM (Dual RTX 3090/4090) or a 64 GB+ Mac / Strix Halo mini PC (48.0–96.0 GB usable).
- Are these speeds measured?
- No. We estimate them from how fast each machine's memory is, because that mostly decides how quickly a model answers. Real speed can be a bit higher or lower. We call a speed comfortable from about 8 tokens per second (tok/s): then the text keeps up with your reading. Below that, you wait for it.
- What does this page give me?
- Up to three machines that can run the AI model you picked: whether it fits, how fast it should answer, and what it costs. The prices are European shop prices, without shipping. Euro prices include VAT. Dollar prices are converted from our euro prices without EU VAT, before US sales tax.
- Are the prices a shop quote?
- No. Each price has a date. When we checked a real listing, the card links to it. Otherwise the price is marked estimate (our own figure) or unverified. Check price opens a search at the shop, so you see today's price before you buy.
- What if nothing fits my budget?
- Then we show the closest machines above your budget and say so. If everything you can afford would answer too slowly, we name the cheapest machine that runs the model comfortably, and a model that does run well inside your budget. A smaller model is often the cheaper fix.