localLLMriglocal, on your table

model

NVIDIA Nemotron 3.5 Bolt 74B-A9B

Hugging Face model card, cited as 10/15/2026: 74B total, 9B active, OpenMDW-1.1, context up to 256K. VRAM math uses the 74B total. Catalog Q4, Q5, Q8 and FP16 rates are not the published 2-bit pack. The config's language stack is a hybrid of 40 Mamba, 40 MoE and 8 attention blocks; the KV term uses only those 8 attention blocks and is not NVIDIA's cache.

NVIDIA Nemotron 3.5 Bolt 74B-A9B needs an estimated 45.3 GB at Q4 and 8,192 tokens. No measured source.

QuantisationEstimated memory
Q445.3 GBestimate
Q553.4 GBestimate
Q880.0 GBestimate
FP16148.9 GBestimate

At 8,192 tokens of context. Catalog quant rates are a formula. sourceUrl on that memory stays null.

Parameter count source: huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB. 74B total, 9B active. Memory math uses the total.

community measurement, single run

Published 16 Oct 2026. NVIDIA official 2-bit GGUF, stated file size 22.0 GB. llama.cpp 0.7.0-dev build 11669, commit 620d2225c, 4K context, chat-short, one run per config, runs dated 15 Oct 2026. Headline figures are 87.8, 107.3 and 154.7 tokens per second. The table prints 87.75 for the RTX 3090 and says the quant was not reported in that column. Not a localLLMrig measurement. Calculator estimates keep benchmarkSourceUrl null.

HardwareTokens per second
RTX 309087.8community measurement, single run
RTX 4090107.3community measurement, single run
RTX 5090154.7community measurement, single run

llm-speed, published 2026-10-16

Open in the calculator

Three setups

NVIDIA Nemotron 3.5 Bolt 74B-A9B · Q4 · 8,192 ctx

Estimated memory

45.3 GBestimate

44.4 GB weights · 0.1 GB KV cache · 0.8 GB overhead

  1. 01 · Fastest

    Dual RTX 4090, 48 GB

    Same memory as two 3090s, higher bandwidth, a lot more money.

    $8,206€7,330

    price as of 16 Oct 2026

    128 GB DDR4 (estimate): $2,754€2,460. From the 16 Oct 2026 fact-check.

    rest of system (estimate): $560€500. The total is the GPU price plus the RAM line plus this estimate. It is not a checked system price.

    price unverified

    GPU line is 2× the used median €2,185 (€4,370), labelled used, from pcprice.watch (https://www.pcprice.watch/gpu-buying-guide/rtx-4090-price-used-and-specs). 128 GB DDR4 (estimate) €2,460, from the 16 Oct 2026 fact-check. Rest of system (estimate) €500. Other parts from €500 in that fact-check. Not a shop listing. DDR5 was not separately priced, so this total uses the sourced 128 GB DDR4 line. The total is the GPU price plus the RAM line plus that rest estimate. It is not a checked system price.

    used

    tok/s estimate

    19.4

    estimate

    needed

    45.3 GB

    full fit

    usable

    46.8 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • It fits, with little room left. A longer context can push it over.
    • Two GPUs. Expect a heavier PSU and a llama.cpp split. Bandwidth does not fully double.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • Second-hand or a previous generation. The price moves around.
    • Highest estimated speed in this shortlist.

    Parts

    • 2× GeForce RTX 4090 24GB, used
    • 128 GB DDR4 (estimate): €2,460. from the 16 Oct 2026 fact-check.
    • Rest of system (estimate): €500. Other parts from €500 in that fact-check. Not a shop listing.
    • Ryzen 9 7950X or similar
    • 1600W PSU

  2. 02 · Best value

    Dual RTX 3090, 48 GB

    The classic 70B Q4 rig. Two cards, more hassle, enough memory.

    $6,054€5,408

    price as of 16 Oct 2026

    128 GB DDR4 (estimate): $2,754€2,460. From the 16 Oct 2026 fact-check.

    rest of system (estimate): $560€500. The total is the GPU price plus the RAM line plus this estimate. It is not a checked system price.

    price unverified

    GPU line is 2× the used median €1,224 (€2,448), labelled used, from pcprice.watch (https://www.pcprice.watch/gpu-buying-guide/rtx-3090-price-used-and-specs). 128 GB DDR4 (estimate) €2,460, from the 16 Oct 2026 fact-check. Rest of system (estimate) €500. Other parts from €500 in that fact-check. Not a shop listing. The total is the GPU price plus the RAM line plus that rest estimate. It is not a checked system price.

    used

    tok/s estimate

    18.0

    estimate

    needed

    45.3 GB

    full fit

    usable

    46.8 GB

    VRAM

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • It fits, with little room left. A longer context can push it over.
    • Two GPUs. Expect a heavier PSU and a llama.cpp split. Bandwidth does not fully double.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • Second-hand or a previous generation. The price moves around.
    • Most estimated tokens per euro in this shortlist.

    Parts

    • 2× GeForce RTX 3090 24GB, used
    • 128 GB DDR4 (estimate): €2,460. from the 16 Oct 2026 fact-check.
    • Rest of system (estimate): €500. Other parts from €500 in that fact-check. Not a shop listing.
    • Ryzen 9 5900X or similar
    • 1200W PSU, a case that can breathe

  3. 03 · Other platform

    Mac Studio M5 Ultra 96GB

    High bandwidth and enough memory for 70B Q4/Q5. FP16 70B does not fit.

    $7,444€6,649

    price as of 16 Oct 2026

    Belgian Apple Store

    Belgian Apple Store price for this config, from the 16 Oct 2026 fact-check.

    tok/s estimate

    12.9

    estimate

    needed

    45.3 GB

    full fit

    usable

    84.0 GB

    unified

    Why this one

    • The model fits entirely in VRAM or unified memory. No CPU offload.
    • Memory left for a longer context or a heavier quantisation.
    • Unified memory: weights and KV cache share one pool.
    • High memory bandwidth, so a higher tokens-per-second estimate.
    • One box, little noise, no discrete GPU.
    • A different brand, so you don't see the same PC three times.

    Parts

    • Mac Studio M5 Ultra, 96 GB, 1.2 TB/s

With CPU offload, and therefore slower, this could also work: PC with RTX 5090 32GB, ~2.9 tok/s estimate, $8,283€7,399.

Tokens per second are estimated from memory bandwidth, not from a measured benchmark. EUR prices include VAT. USD is an FX estimate. A price without a checked listing is marked price unverified. Run llama-bench yourself before you buy.

Want a build tailored to you? Custom report

Get new builds when GPU prices move

End of results

Common questions

How much memory does NVIDIA Nemotron 3.5 Bolt 74B-A9B need at Q4?
About 45.3 GB at 8,192 tokens: 44.4 GB weights, 0.1 GB KV cache, 0.8 GB overhead. That is an estimate. sourceUrl stays null.
Does NVIDIA Nemotron 3.5 Bolt 74B-A9B fit on a single 24 GB GPU?
Not in full. The estimate is 45.3 GB, and a 24 GB card leaves 23.4 GB usable. sourceUrl stays null.
Are the tokens per second measured?
No. Speed follows from bandwidth and an assumed utilisation. benchmarkSourceUrl stays null.
Run NVIDIA Nemotron 3.5 Bolt 74B-A9B locally · localLLMrig