estimate, except measured file sizes
Can I run NVIDIA Nemotron 3.5 Bolt 74B-A9B locally?
74B total, 9B active. Parameter count is an estimate.
- Licence:
- not recorded
- Context:
- 262,144 tokens
- Modality:
- not recorded
- Released:
- not recorded
Hugging Face model card, cited as 10/15/2026: 74B total, 9B active, OpenMDW-1.1, context up to 256K. VRAM math uses the 74B total. Catalog Q4, Q5, Q8 and FP16 rates are not the published 2-bit pack. The config's language stack is a hybrid of 40 Mamba, 40 MoE and 8 attention blocks; the KV term uses only those 8 attention blocks and is not NVIDIA's cache.
Parameter source · Architecture source · Model page
Memory per quant
KV is at 8,192 tokens. Overhead is 0.8 GB. Both are estimates. A measured file size is labelled. The last column is the cheapest machine that fits the total fully.
| Quant | File size | KV at 8K | +0.8 overhead | Total | Cheapest full fit |
|---|---|---|---|---|---|
| Q8 | 79.2 GB est. | 0.1 GB est. | 0.8 GB est. | 80.0 GB est. | Mac Studio M4 Max 128GB, €3,290 (estimate) |
| Q4 | 44.4 GB est. | 0.1 GB est. | 0.8 GB est. | 45.3 GB est. | Mac mini M4 Pro 64GB, €1,890 (estimate) |
Cheapest rig that fits
Q4, full: Mac mini M4 Pro 64GB, €1,890 (estimate)
Q4, slow, partly in system RAM: Budget PC with RTX 3060 12GB, slow, partly in system RAM, €1,357 (estimate)
On a specific GPU
- NVIDIA Nemotron 3.5 Bolt 74B-A9B on RTX 3060 12GB
- NVIDIA Nemotron 3.5 Bolt 74B-A9B on RTX 4060 Ti 16GB
- NVIDIA Nemotron 3.5 Bolt 74B-A9B on RTX 3090
- NVIDIA Nemotron 3.5 Bolt 74B-A9B on RTX 4090
- NVIDIA Nemotron 3.5 Bolt 74B-A9B on RTX 5080
- NVIDIA Nemotron 3.5 Bolt 74B-A9B on RTX 5090
- NVIDIA Nemotron 3.5 Bolt 74B-A9B on RX 7900 XTX
- NVIDIA Nemotron 3.5 Bolt 74B-A9B on Dual RTX 3090
- NVIDIA Nemotron 3.5 Bolt 74B-A9B on Dual RTX 4090