localLLMriglocal, on your table

Guide

Best GPU for local LLMs in 2026

“Best” here means the card whose memory can hold the model you actually want. This catalog has no llama-bench table, so a higher spec-sheet bandwidth is not a measured win.

The cards in this catalog

  • GeForce RTX 3060 12GB lists 12 GB and 360 GB/s. bandwidthEstimated is false, but bandwidthSourceUrl is null, so that speed is not a citation. GeForce RTX 3060 12GB: $313 estimate, €280 incl. VAT. Global prices (USD), EUR available. Dated 2026-10-14. Marked used. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop listing was checked. sourceUrl is null.
  • GeForce RTX 4060 Ti 16GB lists 16 GB and 288 GB/s. bandwidthEstimated is false, but bandwidthSourceUrl is null, so that speed is not a citation. GeForce RTX 4060 Ti 16GB: $515 estimate, €460 incl. VAT. Global prices (USD), EUR available. Dated 2026-10-14. Marked new. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop listing was checked. sourceUrl is null.
  • GeForce RTX 3090 24GB lists 24 GB and 936 GB/s. bandwidthEstimated is false, but bandwidthSourceUrl is null, so that speed is not a citation. GeForce RTX 3090 24GB: $806 estimate, €720 incl. VAT. Global prices (USD), EUR available. Dated 2026-10-14. Marked used. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop listing was checked. sourceUrl is null.
  • GeForce RTX 4090 24GB lists 24 GB and 1008 GB/s. bandwidthEstimated is false, but bandwidthSourceUrl is null, so that speed is not a citation. GeForce RTX 4090 24GB: $2,071 estimate, €1,850 incl. VAT. Global prices (USD), EUR available. Dated 2026-10-14. Marked new. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop listing was checked. sourceUrl is null.
  • GeForce RTX 5090 32GB lists 32 GB and 1792 GB/s, with bandwidthSourceUrl https://www.nvidia.com/en-eu/geforce/graphics-cards/50-series/. GeForce RTX 5090 32GB: $3,123 estimate, €2,790 incl. VAT. Global prices (USD), EUR available. Dated 2026-10-14. Marked new. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop listing was checked. sourceUrl is null.
  • Radeon RX 7900 XTX 24GB lists 24 GB and 960 GB/s. bandwidthEstimated is false, but bandwidthSourceUrl is null, so that speed is not a citation. Radeon RX 7900 XTX 24GB: $1,030 estimate, €920 incl. VAT. Global prices (USD), EUR available. Dated 2026-10-14. Marked new. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop listing was checked. sourceUrl is null.

GeForce RTX 5090 32GB is the only NVIDIA card here whose bandwidth field points at a page we stored (https://www.nvidia.com/en-eu/geforce/graphics-cards/50-series/). The summary in the catalog says 32 GB GDDR7 according to that NVIDIA 50-series page. It is still too small for Llama 70B Q4: Llama 3.3 70B Q4 at 8,192 tokens is 43.0 GB in this formula (39.5 GB weights, 2.7 GB KV cache, 0.8 GB overhead), and usable memory on that card is 30.8 GB.

Pick by what fits

Llama 3.1 8B Q4 at 8,192 tokens is 6.4 GB in this formula (4.5 GB weights, 1.1 GB KV cache, 0.8 GB overhead). Budget PC with RTX 3060 12GB has 10.8 GB usable after the reserve in this formula (catalog memory 12 GB, vram). Llama 3.2 3B, Llama 3.1 8B, Mistral 7B v0.3, Gemma 2 9B, Qwen2.5 14B, Phi-4 14B fully fits that PC at Q4.

Qwen2.5 32B Q4 at 8,192 tokens is 21.3 GB in this formula (18.4 GB weights, 2.1 GB KV cache, 0.8 GB overhead). Used PC with RTX 3090 24GB has 22.8 GB usable after the reserve in this formula (catalog memory 24 GB, vram). A 16 GB card does not clear that total. Mid-range PC with RTX 4060 Ti 16GB usable memory is 14.8 GB.

Llama 3.3 70B Q4 at 8,192 tokens is 43.0 GB in this formula (39.5 GB weights, 2.7 GB KV cache, 0.8 GB overhead). No single card in this list has enough usable memory. Dual RTX 3090, 48 GB does: usable 45.6 GB across 48 GB. The catalog calls that the classic 70B Q4 rig. The speed estimate then multiplies combined bandwidth by 0.8 because a layer split does not add bandwidth. That factor is an assumption, not a measurement.

Radeon RX 7900 XTX 24GB is the same 24 GB class as a 3090 or 4090. The tokens/s estimate uses AMD utilisation 0.4 instead of NVIDIA 0.55. That is an assumption about the software stack, not a benchmark we ran.

What this page will not rank

Tokens per second on the calculator are estimates from the methodology page. benchmarkSourceUrl stays null. This guide does not add a measured llama-bench result.

Open the same question in the calculator: Llama 3.1 8B Q4, Qwen2.5 32B Q4, Llama 3.3 70B Q4 under €2,000. Hardware table: all cards and builds. Longer 70B page: best GPU for Llama 70B.

Related: How much VRAM does a 70B model need?, Used RTX 3090 buying guide (EU), Quantization explained for local LLMs.

Open the calculator

Best GPU for local LLMs in 2026 · localLLMrig