Guide
Nemotron 3.5 Bolt 74B hardware requirements: does it really fit in 24GB?
24 GB is the size NVIDIA packed the weights into. It is not a promise that a 24 GB GPU still has room for a long context, images, or another application. Memory figures below are estimates unless NVIDIA printed them.
What NVIDIA published
The Hugging Face model card for nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB cites its own release as Hugging Face 10/15/2026. The card says the model is ready for commercial or non-commercial use under OpenMDW-1.1. The same card lists 74B total parameters with 9B active, a Mamba-2–Transformer hybrid LatentMoE with a C-RADIO v4-H vision encoder, IQ2 2-bit quantization with quantization-aware training, context up to 256K tokens, and text, image, and video input. NVIDIA Nemotron 3.5 Bolt 74B model card A separate line on that card, Model Dates, says December 2025–May 2026. That is the training window the card prints, not the 15 October publication date.
The card says experts are quantized to IQ2-XXS at 2.0625 bits per element so the network fits in 24 GB, and that some other layers stay at NVFP4 (Mamba) or Q8_0 (embedding and output). The summary table says the packed model deploys with vLLM on NVIDIA Ampere (A100), Hopper (H100-80GB), or Blackwell. A later Software Integration block on the same card lists vLLM on NVIDIA Blackwell SM120 and SM121, and Linux. Both sentences are on the card.
The card also says individual deployment recipes may use a smaller context window than 256K so the job fits the GPU and the concurrency. The prompt, visual inputs, conversation history, tool results, and the generated response share that window.
Weights NVIDIA lists next to the 24 GB checkpoint: the IQ2 model, a BF16 model, a GGUF build, and a LiLiCorr drafter in both formats. BF16, 24GB GGUF, LiLiCorr, LiLiCorr GGUF.
24 GB is the packed file, not spare room
A mixture-of-experts model still stores every expert. Nine billion active parameters per token do not shrink the file to 9B. The technical report says the listed weights occupy 22.01 GB. Cache and activations add 0.87 GB in NVIDIA's estimate for batch size 1, a context of 131,072 tokens, and a Mamba prefill chunk of 8,192. The drafter they include uses another 0.96 GB. NVIDIA adds that up to 23.83 GB of a 24 GB target, with 0.17 GB left, and says the vision encoder and projector are excluded from that text-only total. NVIDIA technical report, 15 October 2026 Those gigabytes are NVIDIA's estimate. They are not a file this site measured. The Mamba-2 hybrid is why that cache term can stay small compared with a pure transformer.
This site's formula is a different estimate. It subtracts 0.6 GB per discrete GPU and then adds 0.8 GB of runtime overhead. On the catalog's 24 GB cards that leaves 23.4 GB usable. NVIDIA's 22.01 GB weight figure fits inside that usable figure. Adding the formula's overhead to those weights is 22.8 GB, which still fits. NVIDIA's own text-only total of 23.83 GB is larger than that usable figure. A 32 GB card in the same formula has 31.4 GB usable. Neither number is a measured Bolt file. The catalog has no 2-bit rate, so it cannot score Bolt itself.
This site's formula, applied to Llama 3.3 70B (a dense transformer, a different model), puts the KV cache at 42.9 GB at 131,072 tokens. That comparison shows the formula's transformer KV term. It is not Bolt's cache, and it is still an estimate.
Which machines are in range
- A 24 GB GeForce card, such as the catalog RTX 3090 or RTX 4090, is the packed-weight class. Expect a shorter context than 256K, and treat images or video as extra memory the text-only 24 GB budget did not include.
- A 32 GB card, such as the catalog RTX 5090, is the headroom class under the reserve above. A separate page treats reports that the 5090 is discontinued as a leak NVIDIA has not confirmed. RTX 5090 discontinuation leak.
- RTX Spark laptops are the machine NVIDIA built this pack for. The 13 October 2026 blog says they are available on Friday 16 October, and the 15 October blog says Bolt is for RTX Spark laptops arriving that day, and that it also runs on GeForce RTX PCs, DGX Spark, DGX Station, and RTX PRO. NVIDIA blog, 13 October 2026 NVIDIA blog, 15 October 2026 This guide does not add a laptop price. RTX Spark, Mac Studio, and Strix Halo.
- Macs with enough unified memory are a memory-size category, not a confirmed runtime. Ollama's library page says Apple Silicon / Metal is not currently supported. Unsloth's guide lists a macOS desktop app. Those are two different claims.
The useful check is still the prompt, not the card. Try your own prompt against other models in the compare tool, then come back to hardware. There is no Bolt preset in the calculator. The closest catalog question is a different model: Qwen2.5 32B Q4 on the calculator. That result is an estimate for Qwen, not for Bolt.
How to run it
NVIDIA's 15 October blog says Bolt 74B-A9B 24GB is on Hugging Face with support for llama.cpp, Ollama, LM Studio, Unsloth, and vLLM. The model card's quick start is a vLLM serve of nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB, including a max model length of 262144. Copy that command from the card. This page does not retype the flag list.
Ollama v0.40.5, published 14 October 2026, describes Bolt as the 74B (9B active) sibling of Nemotron 3.5 Super and says the llama.cpp update adds Nemotron 3.5 support. Ollama v0.40.5 The release note's copy-paste command is ollama run nemotron-3.5-super, not Bolt. The library page's command is ollama run nemotron-3.5-bolt. Ollama library page That page lists the tags nemotron-3.5-bolt:latest, :74b-a9b, and :74b-a9b-iq2, each at 25 GB and 256K. 25 GB is Ollama's listed size, not a second measurement from this site.
Unsloth's GGUF guide gives the command unsloth run --model unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF:IQ2_XXS and says to start at 4,096 tokens and raise context as memory allows, up to 128K. It calls 22 GB the IQ2_XXS download and 24–32 GB an estimated minimum for short context. Those are Unsloth's figures. NVIDIA's card still says context up to 256K, and its recommended text sampling is temperature 1.0 and top_p 0.95. Unsloth Nemotron 3.5 Bolt guide.
Speed, vendor-reported
NVIDIA's technical report says the model reaches 84 tokens per second on RTX Spark. Table 8 in that report lists 83.99 tokens per second for the 24 GB checkpoint with LiLiCorr, measured by NVIDIA with llama.cpp on Windows 11 ARM on a Surface Laptop Ultra with 128 GB. Those numbers are vendor-reported. They are not a localLLMrig measurement. benchmarkSourceUrl on this site stays null.