localLLMriglocal, on your table

model

Qwen2.5 72B

Same class as Llama 70B. Same conclusion: you need a large memory pool.

Qwen2.5 72B needs an estimated 44.2 GB at Q4 and 8,192 tokens. No measured source.

QuantisationEstimated memory
Q444.2 GBestimate
Q553.6 GBestimate
Q881.3 GBestimate
FP16148.9 GBestimate

At 8,192 tokens of context. No measured source.

Open in the calculator

Three setups

Qwen2.5 72B · Q4 · 8,192 ctx

Estimated memory

44.2 GBestimate

40.7 GB weights · 2.7 GB KV cache · 0.8 GB overhead

  1. 01 · Fastest

    Dual RTX 4090, 48 GB

    Same memory as two 3090s, higher bandwidth, a lot more money.

    $6,146€5,490

    estimate

    tok/s

    19.4

    estimate

    needed

    44.2 GB

    full fit

    usable

    45.6 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • It fits, with little room left. A longer context can push it over.
    • Two GPUs. Expect a heavier PSU and a llama.cpp split. Bandwidth does not fully double.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • Highest estimated speed in this shortlist.

    Parts

    • 2× GeForce RTX 4090 24GB
    • Ryzen 9 7950X or similar
    • 128 GB DDR5
    • 1600W PSU

  2. 02 · Best value

    Dual RTX 3090, 48 GB

    The classic 70B Q4 rig. Two cards, more hassle, enough memory.

    $2,217€1,980

    used

    estimate

    tok/s

    18.1

    estimate

    needed

    44.2 GB

    full fit

    usable

    45.6 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • It fits, with little room left. A longer context can push it over.
    • Two GPUs. Expect a heavier PSU and a llama.cpp split. Bandwidth does not fully double.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • Second-hand or a previous generation. The price moves around.
    • Most estimated tokens per euro in this shortlist.

    Parts

    • 2× GeForce RTX 3090 24GB, used
    • Ryzen 9 5900X or similar
    • 128 GB DDR4
    • 1200W PSU, a case that can breathe

  3. 03 · Other platform

    Mac Studio M5 Ultra 96GB

    High bandwidth and enough memory for 70B Q4/Q5. FP16 70B does not fit.

    $7,266€6,490

    estimate

    tok/s

    12.8

    estimate

    needed

    44.2 GB

    full fit

    usable

    84.0 GB

    unified

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • Memory left for a longer context or a heavier quantisation.
    • Unified memory: weights and KV cache share one pool.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • One box, little noise, no discrete GPU.
    • A different brand, so you don't see the same PC three times.

    Parts

    • Mac Studio M5 Ultra, 96 GB, 1.2 TB/s

With CPU offload, and therefore slower, this could also work: PC with RTX 5090 32GB, ~2.9 tok/s, $5,027€4,490.

Tokens per second are estimated from memory bandwidth, not from a measured benchmark. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop source. Run llama-bench yourself before you buy.

Want a build tailored to you? Custom report

Get new builds when GPU prices move

End of results

Common questions

How much memory does Qwen2.5 72B need at Q4?
About 44.2 GB at 8,192 tokens: 40.7 GB weights, 2.7 GB KV cache, 0.8 GB overhead. That is an estimate. sourceUrl stays null.
Does Qwen2.5 72B fit on a single 24 GB GPU?
Not in full. The estimate is 44.2 GB, and a 24 GB card leaves 22.8 GB usable. sourceUrl stays null.
Are the tokens per second measured?
No. Speed follows from bandwidth and an assumed utilisation. benchmarkSourceUrl stays null.