localLLMriglocal, on your table

model

Llama 3.2 3B

Small enough for a Mac mini or a modest GPU. A good stack to test your setup.

Llama 3.2 3B needs an estimated 3.5 GB at Q4 and 8,192 tokens. No measured source.

QuantisationEstimated memory
Q43.5 GBestimate
Q54.0 GBestimate
Q85.2 GBestimate
FP168.2 GBestimate

At 8,192 tokens of context. No measured source.

Open in the calculator

Three setups

Llama 3.2 3B · Q4 · 8,192 ctx

Estimated memory

3.5 GBestimate

1.8 GB weights · 0.9 GB KV cache · 0.8 GB overhead

  1. 01 · Fastest

    PC with RTX 5090 32GB

    Highest bandwidth in a single card. 32 GB is plenty for 32B, still too tight for 70B Q4.

    $5,027€4,490

    estimate

    tok/s

    189

    estimate

    needed

    3.5 GB

    full fit

    usable

    30.8 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • Memory left for a longer context or a heavier quantisation.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • Highest estimated speed in this shortlist.

    Parts

    • GeForce RTX 5090 32GB
    • Ryzen 7 9700X or similar
    • 96 GB DDR5
    • 1200W PSU, roomy case

  2. 02 · Best value

    Used PC with RTX 3090 24GB

    Best single card here for 32B Q4. 70B Q4 does not fully fit.

    $1,433€1,280

    used

    estimate

    tok/s

    128

    estimate

    needed

    3.5 GB

    full fit

    usable

    22.8 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • Memory left for a longer context or a heavier quantisation.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • Second-hand or a previous generation. The price moves around.
    • Most estimated tokens per euro in this shortlist.

    Parts

    • GeForce RTX 3090 24GB, used
    • Ryzen 7 5700X or similar
    • 64 GB DDR4
    • 850W PSU, decent airflow

  3. 03 · Other platform

    Mac Studio M5 Ultra 96GB

    High bandwidth and enough memory for 70B Q4/Q5. FP16 70B does not fit.

    $7,266€6,490

    estimate

    tok/s

    138

    estimate

    needed

    3.5 GB

    full fit

    usable

    84.0 GB

    unified

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • Memory left for a longer context or a heavier quantisation.
    • Unified memory: weights and KV cache share one pool.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • One box, little noise, no discrete GPU.
    • A different brand, so you don't see the same PC three times.

    Parts

    • Mac Studio M5 Ultra, 96 GB, 1.2 TB/s

Tokens per second are estimated from memory bandwidth, not from a measured benchmark. Global prices (USD), EUR available. EUR includes VAT. USD is an FX estimate. No shop source. Run llama-bench yourself before you buy.

Want a build tailored to you? Custom report

Get new builds when GPU prices move

End of results

Common questions

How much memory does Llama 3.2 3B need at Q4?
About 3.5 GB at 8,192 tokens: 1.8 GB weights, 0.9 GB KV cache, 0.8 GB overhead. That is an estimate. sourceUrl stays null.
Does Llama 3.2 3B fit on a single 24 GB GPU?
The estimate of 3.5 GB sits under the 22.8 GB we treat as usable on a 24 GB card. It is still an estimate, not a measured file.
Are the tokens per second measured?
No. Speed follows from bandwidth and an assumed utilisation. benchmarkSourceUrl stays null.