estimate, except measured file sizes
Can I run NVIDIA Nemotron 3.5 Super VL locally?
120B total, 12B active. Parameter count is an estimate.
- Licence:
- OpenMDW-1.1 source
- Context:
- 1,048,576 tokens
- Modality:
- text+image+video in
- Released:
- not recorded
Hybrid Mamba-2/Transformer: 88 blocks (40 mamba, 40 moe, 8 attention). The KV formula uses only those 8 attention blocks. It does not model the Mamba SSM state or the vision mmproj (+0.97–1.73 GB). 120B total and 12B active are the card's round billions, so paramsEstimated stays true. Local on Ollama as nemotron-3.5-super (default tag 80–90 GB). Speed is not estimated: the formula streams every weight and understates sparse MoE.
Parameter source · Architecture source · Model page
speed: not estimated for MoE yet
Memory per quant
KV is at 8,192 tokens. Overhead is 0.8 GB. Both are estimates. A measured file size is labelled. The last column is the cheapest machine that fits the total fully.
| Quant | File size | KV at 8K | +0.8 overhead | Total | Cheapest full fit |
|---|---|---|---|---|---|
| Q8_0 | 128.5 GB measured GGUF | 0.1 GB est. | 0.8 GB est. | 129.3 GB est. | Mac Studio M5 Ultra 256GB, €12,500 (estimate) |
| NVFP4 | 76.9 GB measured GGUF | 0.1 GB est. | 0.8 GB est. | 77.8 GB est. | Mac Studio M4 Max 128GB, €3,290 (estimate) |
| Q4_0 | 71.8 GB measured GGUF | 0.1 GB est. | 0.8 GB est. | 72.7 GB est. | Mac Studio M4 Max 128GB, €3,290 (estimate) |
| UD-Q4_K_XL | 86.9 GB measured GGUF | 0.1 GB est. | 0.8 GB est. | 87.8 GB est. | Mac Studio M4 Max 128GB, €3,290 (estimate) |
| UD-Q3_K_XL | 65.8 GB measured GGUF | 0.1 GB est. | 0.8 GB est. | 66.6 GB est. | Mac Studio M4 Max 128GB, €3,290 (estimate) |
| UD-Q2_K_XL | 57.8 GB measured GGUF | 0.1 GB est. | 0.8 GB est. | 58.7 GB est. | Mac Studio M4 Max 128GB, €3,290 (estimate) |
| UD-IQ2_XXS | 51.1 GB measured GGUF | 0.1 GB est. | 0.8 GB est. | 52.0 GB est. | Mac mini M4 Pro 64GB, €1,890 (estimate) |
| UD-IQ1_M | 46.1 GB measured GGUF | 0.1 GB est. | 0.8 GB est. | 47.0 GB est. | Mac mini M4 Pro 64GB, €1,890 (estimate) |
Cheapest rig that fits
Q4, full: Mac Studio M4 Max 128GB, €3,290 (estimate)
Q4, slow, partly in system RAM: Used PC with RTX 3090 24GB, slow, partly in system RAM, €2,324 (estimate)
UD-IQ1_M (smallest file), full: Mac mini M4 Pro 64GB, €1,890 (estimate)
UD-IQ1_M (smallest file), slow, partly in system RAM: Budget PC with RTX 3060 12GB, slow, partly in system RAM, €1,357 (estimate)
On a specific GPU
- NVIDIA Nemotron 3.5 Super VL on RTX 3060 12GB
- NVIDIA Nemotron 3.5 Super VL on RTX 4060 Ti 16GB
- NVIDIA Nemotron 3.5 Super VL on RTX 3090
- NVIDIA Nemotron 3.5 Super VL on RTX 4090
- NVIDIA Nemotron 3.5 Super VL on RTX 5080
- NVIDIA Nemotron 3.5 Super VL on RTX 5090
- NVIDIA Nemotron 3.5 Super VL on RX 7900 XTX
- NVIDIA Nemotron 3.5 Super VL on Dual RTX 3090
- NVIDIA Nemotron 3.5 Super VL on Dual RTX 4090