localLLMriglocal, on your table

landing page

Best GPU for Llama 70B

Memory first, speed second. Llama 70B at Q4 is a question of about 40 GB, not of the most expensive single card.

Q4 uses 0.56 bytes per parameter here. For a 70B model that is about 40 GB of weights, plus a few GB of KV cache at 8,192 tokens. An RTX 4090 or 3090 has 24 GB. An RTX 5090 has 32 GB. None of those cards hold Llama 70B Q4 in full.

What does work: two 24 GB cards (classically two used 3090s) or a Mac with enough unified memory. 64 GB is tight but workable at Q4. 128 GB leaves room for a heavier quant or a longer context.

Under €2000 the dual 3090 usually wins this dataset on speed. A 64 GB Mac also fits, but bandwidth is lower, so the estimated tokens/s sit much lower. Both prices are estimates without a shop source.

Open in the calculator

Three setups

Llama 3.3 70B · Q4 · 8,192 ctx

Estimated memory

43.0 GBestimate

39.5 GB weights · 2.7 GB KV cache · 0.8 GB overhead

  1. 01 · Fastest

    Dual RTX 3090, 48 GB

    The classic 70B Q4 rig. Two cards, more hassle, enough memory.

    $2,217€1,980

    inside budget · used

    estimate

    tok/s

    18.6

    estimate

    needed

    43.0 GB

    full fit

    usable

    45.6 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • It fits, with little room left. A longer context can push it over.
    • The indicative price is inside your budget.
    • Two GPUs. Expect a heavier PSU and a llama.cpp split. Bandwidth does not fully double.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • Second-hand or a previous generation. The price moves around.
    • Highest estimated speed in this shortlist.

    Parts

    • 2× GeForce RTX 3090 24GB, used
    • Ryzen 9 5900X or similar
    • 128 GB DDR4
    • 1200W PSU, a case that can breathe

  2. 02 · Other platform

    Mac mini M4 Pro 64GB

    Enough memory for 70B Q4, but bandwidth is modest. It fits; it is not fast.

    $2,116€1,890

    inside budget · used

    estimate

    tok/s

    3.1

    estimate

    needed

    43.0 GB

    full fit

    usable

    54.0 GB

    unified

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • The indicative price is inside your budget.
    • Unified memory: weights and KV cache share one pool.
    • Second-hand or a previous generation. The price moves around.
    • One box, little noise, no discrete GPU.
    • A different brand, so you don't see the same PC three times.

    Parts

    • Mac mini M4 Pro, 64 GB unified memory
    • 273 GB/s bandwidth is an estimate; no page was re-fetched

  3. 03 · A step up

    Dual RTX 4090, 48 GB

    Same memory as two 3090s, higher bandwidth, a lot more money.

    $6,146€5,490

    over budget

    estimate

    tok/s

    20.0

    estimate

    needed

    43.0 GB

    full fit

    usable

    45.6 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • It fits, with little room left. A longer context can push it over.
    • Above your budget. Nothing fully fit inside the amount, or this is a faster step above it.
    • Two GPUs. Expect a heavier PSU and a llama.cpp split. Bandwidth does not fully double.
    • High memory bandwidth, so a higher tokens-per-second estimate.

    Parts

    • 2× GeForce RTX 4090 24GB
    • Ryzen 9 7950X or similar
    • 128 GB DDR5
    • 1600W PSU

With CPU offload, and therefore slower, this could also work: PC with Radeon RX 7900 XTX 24GB, ~1.8 tok/s, $2,004€1,790.

Tokens per second are estimated from memory bandwidth, not from a measured benchmark. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop source. Run llama-bench yourself before you buy.

Want a build tailored to you? Custom report

Get new builds when GPU prices move

End of results

Best GPU for Llama 70B · localLLMrig