localLLMriglocal, on your table

model

Gemma 2 27B

Same caveat as Gemma 2 9B: sliding window makes the KV estimate rougher.

Gemma 2 27B needs an estimated 18.3 GB at Q4 and 8,192 tokens. No measured source.

QuantisationEstimated memory
Q418.3 GBestimate
Q521.9 GBestimate
Q832.2 GBestimate
FP1657.5 GBestimate

At 8,192 tokens of context. No measured source.

Open in the calculator

Three setups

Gemma 2 27B · Q4 · 8,192 ctx

Estimated memory

18.3 GBestimate

15.2 GB weights · 2.3 GB KV cache · 0.8 GB overhead

  1. 01 · Fastest

    PC with RTX 5090 32GB

    Highest bandwidth in a single card. 32 GB is plenty for 32B, still too tight for 70B Q4.

    $5,027€4,490

    estimate

    tok/s

    49

    estimate

    needed

    18.3 GB

    full fit

    usable

    30.8 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • Highest estimated speed in this shortlist.

    Parts

    • GeForce RTX 5090 32GB
    • Ryzen 7 9700X or similar
    • 96 GB DDR5
    • 1200W PSU, roomy case

  2. 02 · Best value

    Used PC with RTX 3090 24GB

    Best single card here for 32B Q4. 70B Q4 does not fully fit.

    $1,433€1,280

    used

    estimate

    tok/s

    27

    estimate

    needed

    18.3 GB

    full fit

    usable

    22.8 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • Second-hand or a previous generation. The price moves around.
    • Most estimated tokens per euro in this shortlist.

    Parts

    • GeForce RTX 3090 24GB, used
    • Ryzen 7 5700X or similar
    • 64 GB DDR4
    • 850W PSU, decent airflow

  3. 03 · Other platform

    Mac Studio M5 Ultra 96GB

    High bandwidth and enough memory for 70B Q4/Q5. FP16 70B does not fit.

    $7,266€6,490

    estimate

    tok/s

    30

    estimate

    needed

    18.3 GB

    full fit

    usable

    84.0 GB

    unified

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • Memory left for a longer context or a heavier quantisation.
    • Unified memory: weights and KV cache share one pool.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • One box, little noise, no discrete GPU.
    • A different brand, so you don't see the same PC three times.

    Parts

    • Mac Studio M5 Ultra, 96 GB, 1.2 TB/s

With CPU offload, and therefore slower, this could also work: Mid-range PC with RTX 4060 Ti 16GB, ~6.1 tok/s, $1,668€1,490.

Tokens per second are estimated from memory bandwidth, not from a measured benchmark. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop source. Run llama-bench yourself before you buy.

Want a build tailored to you? Custom report

Get new builds when GPU prices move

End of results

Common questions

How much memory does Gemma 2 27B need at Q4?
About 18.3 GB at 8,192 tokens: 15.2 GB weights, 2.3 GB KV cache, 0.8 GB overhead. That is an estimate. sourceUrl stays null.
Does Gemma 2 27B fit on a single 24 GB GPU?
The estimate of 18.3 GB sits under the 22.8 GB we treat as usable on a 24 GB card. It is still an estimate, not a measured file.
Are the tokens per second measured?
No. Speed follows from bandwidth and an assumed utilisation. benchmarkSourceUrl stays null.