NVIDIA Nemotron 3.5 Lightning: VRAM requirements by quant
Key facts
NVIDIA Nemotron 3.5 Lightning needs about 19.8 GB at standard quality (Q4, estimate).
Checked . ggml-org Q4_0 GGUF (official ggml-org repo; not Q4_K_M).
NVIDIA Nemotron 3.5 Lightning is a mid-size AI model you can download and run on your own computer. It is a mixture of experts: all of it has to fit in memory, but it uses only a small part for each word. It needs about 20 GB of memory at standard quality.
Try NVIDIA Nemotron 3.5 Lightning free in your browser, no login, no install
LIVE IN BROWSER · $0 · NO LOGIN
Test NVIDIA Nemotron 3.5 Lightning free in your browser before downloading 19 GB
Open full 8-model playground ↗Click a starter prompt below or type your own question to watch Nemotron 3.5 Lightning stream side by side against Gemma 4 31B right on this page:
Watch: 8 open LLMs vs 1 trick question (24s real-time screen recording)
Unedited recording of the free multi-model playground on localLLMrig.com running Gemma 4 31B, NVIDIA Nemotron 3.5 Lightning, Recursive Flint, and Gemma 4 26B A4B side by side on the classic €1.10 bat-and-ball question — then showing the cheapest Mac mini or GPU that runs the winner locally. Watch on YouTube (@localllmrig) ↗
Memory needed
About 20 GB
At standard quality, with room for a conversation of about 12 pages.
Get a 1-line alert when a faster quant for NVIDIA Nemotron 3.5 Lightning drops — or when GPUs that run it drop >10%
Get a 1-line alert when a new open-weight model fits 24 GB VRAM (19.8 GB at Q4 for NVIDIA Nemotron 3.5 Lightning) or when used RTX 3090 / 4090 & Mac Studio prices drop >10% — weekly, zero spam.
Machines that run NVIDIA Nemotron 3.5 Lightning
Each of these holds the whole model in memory. We do not estimate a speed for this kind of model (a mixture of experts), so these are simply the cheapest.
EU shops · Shown to visitors in Europe.US shops · Shown to visitors outside Europe.
If you buy through these links we may earn a commission. It doesn't change our ranking. How shop links work
Mac mini M6 32GB
32 GB shared memory
$1,416€1,530estimate
Machine details →Mac Studio M4 Max 36GB
36 GB shared memory · second-hand
$2,116€1,890usedunverified
Machine details →Mac mini M4 Pro 64GB
64 GB shared memory · second-hand
$2,116€1,890usedunverified
Machine details →Used PC with RTX 3090 24GB
24 GB graphics memory · second-hand
$2,413€2,346estimateused parts
Machine details →The buttons look up the graphics card, the part of this PC worth comparing.
PC with Radeon RX 7900 XTX 24GB
24 GB graphics memory · second-hand
$2,422€2,354estimateused parts
Machine details →The buttons look up the graphics card, the part of this PC worth comparing.
Where our price is for a used card or Mac, these shops mostly sell new ones, so their price can be higher.
As an Amazon Associate I earn from qualifying purchases.
Als Amazon-Partner verdiene ich an qualifizierten Verkäufen.
Als Amazon-Partner verdien ik aan in aanmerking komende aankopen.
About these prices
Each price is the amount we last wrote down for that machine, with its date on the machine page. "Estimate" is our own figure: for a complete PC we add our estimate for the other parts to the graphics card price, and some Mac prices are estimates, not a checked shop listing. "Used" is a second-hand price. "Unverified" means we have no checked shop listing for that exact amount. The shop buttons open a search for that machine at each shop, so you see today's price there. Prices excl. shipping unless marked. Euro prices incl. VAT where the seller charges it. Dollar prices are converted from our euro prices without the 21% EU VAT, before US sales tax.
Or rent one by the hour: RTX A6000 at Vast.ai holds it, for $0.40 an hour (median of live offers, 21 Oct 2026). Compare GPU cloud prices
Speeds and memory are our estimate from the hardware specs, not a measured test. Prices are dated and may change. How we calculate
Memory at each quality level
A lower quality level makes the model smaller, so it fits on a smaller machine. Standard (Q4) is what most people run.
| Quality | Memory needed |
|---|---|
| Q4 · standard (smallest that keeps quality) | 19.8 GB |
| Q5 · a bit sharper, a bit bigger | 22.2 GB |
| Q8 · near full quality, almost twice Q4 | 33.0 GB |
| FP16 · full quality, over three times Q4 | 60.9 GB |
With a conversation of 8,192 tokens · about 12 pages of text.
Sources and method
At standard quality: 18.9 GB for the model itself (a measured file), 0.1 GB for a conversation of 8,192 tokens, and 0.8 GB for the software.
Size source: huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16. 30B parameters in total, 3B used for each word. The memory counts the total.
NVIDIA's hybrid Mamba and Transformer mixture of experts: 30B total, 3B active, open weights under OpenMDW-1.1. The card says up to 1M tokens; the config and the single-H100 recipe use 262,144, which is the context stored here. The stack is 52 blocks (23 Mamba, 23 mixture-of-experts, 6 attention). Our conversation-memory estimate uses only those 6 attention blocks and does not model the Mamba state.
Licence
Commercial use: yes
Commercial use allowed. Keep the licence and copyright notice when you share the weights.
Licence: OpenMDW-1.1 · checked 24 Oct 2026
Our reading of the licence, not legal advice.
Will NVIDIA Nemotron 3.5 Lightning run on my graphics card?
Common questions
- How much memory does NVIDIA Nemotron 3.5 Lightning need?
- About 19.8 GB at standard quality (Q4) with a conversation of 8,192 tokens: 18.9 GB for the model itself, 0.1 GB for the conversation and 0.8 GB for the software. The model size is a measured file; the conversation and software parts are our estimate.
- Does NVIDIA Nemotron 3.5 Lightning fit on a single 24 GB graphics card?
- Yes, by our estimate. It needs about 19.8 GB, and we count 23.4 GB of a 24 GB card as usable.
- Are the speeds measured?
- We do not show a speed for this model. It is a mixture-of-experts model, and our estimate would not be reliable for that design.
- Can I test NVIDIA Nemotron 3.5 Lightning online for free in my browser before downloading 19 GB?
- Yes. NVIDIA Nemotron 3.5 Lightning (30B total, 3B active, parameter count is an estimate) is one of the 10 free open models at https://localllmrig.com/ and in the inline test box on https://localllmrig.com/can-i-run/nemotron-3.5-lightning. You can compare it side by side against Gemma 4 31B (https://localllmrig.com/can-i-run/gemma-4-31b), Recursive Flint (https://localllmrig.com/can-i-run/flint), Laguna S 2.1 (https://localllmrig.com/can-i-run/laguna-s-2.1) for $0 with no account.
- Can I run NVIDIA Nemotron 3.5 Lightning on 24GB?
- Yes, by our estimate. At Q4_0 (18.9 GB measured GGUF + 0.05 GB conversation memory at 8,192 tokens + 0.80 GB working space = 19.75 GB / 19.8 GB total estimate), NVIDIA Nemotron 3.5 Lightning fits inside the 23.4 GB usable memory of a 24 GB card. The cheapest 24 GB card on our list is GeForce RTX 3090 24GB at €1,246 / $1,395. A 24 GB card also holds UD-Q3_K_XL (21.24 GB measured GGUF, 22.1 GB total estimate) and UD-IQ2_M (19.43 GB measured GGUF, 20.3 GB total estimate).
- Can I run NVIDIA Nemotron 3.5 Lightning on a 16GB or 32GB GPU?
- On a 16 GB GPU (15.4 GB usable), none of the listed GGUF files fit in VRAM, so the model has to offload layers to system RAM. On a 32 GB GPU (31.4 GB usable, GeForce RTX 5090 32GB), UD-Q5_K_XL (30.41 GB measured GGUF, 31.3 GB total estimate), UD-Q4_K_XL (25.51 GB measured GGUF, 26.4 GB total estimate), UD-Q3_K_XL (21.24 GB measured GGUF, 22.1 GB total estimate), UD-IQ2_M (19.43 GB measured GGUF, 20.3 GB total estimate), Q4_0 (18.9 GB measured GGUF, 19.8 GB total estimate) fit in VRAM.
- Can I run NVIDIA Nemotron 3.5 Lightning on a 128GB Mac?
- Yes, by our estimate. NVIDIA Nemotron 3.5 Lightning fits on Mac Studio M4 Max 128GB and Mac Studio M5 Max 128GB (96.0 GB usable of 128 GB, which holds BF16 (63.18 GB measured GGUF, 64.0 GB total estimate)). At Q4_0 (19.8 GB estimate), the cheapest 32 GB Mac that holds it is Mac mini M6 32GB (21.3 GB usable, €1,530 / $1,416, price unverified). Mac mini M4 Pro 64GB (48.0 GB usable, €1,890 / $2,116, price unverified) holds Q8_0 (33.59 GB measured GGUF, 34.4 GB total estimate).
- How fast is NVIDIA Nemotron 3.5 Lightning on an RTX 5090?
- We don't give a speed for this model yet. This model only uses part of itself for each word (3B active of 30B total, a mixture of experts), and our speed estimate does not handle that yet.
- Why is NVIDIA Nemotron 3.5 Lightning's conversation cache estimate so small (0.05 GB at 8,192), and what is its license?
- The catalog counts conversation memory for 6 attention blocks (2 key-value heads, head size 128), about 0.05 GB at 8,192 tokens (0.1 GB rounded). The model notes describe a 52-block hybrid stack (23 Mamba blocks, 23 mixture-of-experts blocks, and 6 attention blocks). Licence: OpenMDW-1.1, with 262,144 tokens stored as the context length. The model notes say the card claims up to 1M tokens; the context stored here is the config figure.
- Can I try NVIDIA Nemotron 3.5 Lightning online for free?
- Yes. NVIDIA Nemotron 3.5 Lightning is on the free chat on the home page. Pick it there, ask a question, and read the answer in your browser. No login and no install.
- Do I need a GPU to test it?
- No. The free chat runs in your browser. It does not download the model and it does not use a graphics card on your computer. A graphics card only matters if you run the model yourself.
- Do I need an account to try NVIDIA Nemotron 3.5 Lightning?
- No. The free chat needs no account and no login.