Guide
Can I run Llama 70B on an RTX 4090?
Not in full. Q4_K_M weights alone are already larger than the usable memory on one 24 GB card. The figures below are this site's formula, not a downloaded file.
The short answer
Llama 3.3 70B Q4 at 8,192 tokens is 45.8 GB in this formula (42.4 GB weights, 2.7 GB KV cache, 0.8 GB overhead). Q4 means Q4_K_M. GeForce RTX 4090 24GB has 23.4 GB usable after the 0.6 GB reserve. Fit on that bare card: not a fit. That is an estimate.
The longer memory page is How much VRAM does a 70B model need?.
Offload
PC with RTX 4090 24GB keeps the same 24 GB GPU and lists 64 GB of system RAM. Fit there: offload only, not a full fit. Offload means the formula can spill the rest into system RAM after leaving 8 GB for the OS. It is not a full fit, and the speed estimate for that spill is still an estimate. The bare card row has no system RAM in the catalog, so it cannot take that path.
Two 24 GB cards
Dual RTX 3090, 48 GB usable memory is 46.8 GB. Fit: a full fit. Dual RTX 4090, 48 GB usable memory is 46.8 GB. Fit: a full fit. Dual RTX 3090 for 70B walks through the 3090 pair. Both results are formula fits, not a measured llama-bench.
A smaller quant
A 2-bit pack would be smaller than Q4_K_M. This catalog has no 2-bit rate for Llama 3.3 70B, so this page does not quote a 2-bit file size. The stored rates are Q4, Q5, Q8 and FP16. What the quant names mean.
Renting
If the card you have cannot hold the model, renting GPU time is the other option. This site does not invent a cloud hourly rate. Buy vs rent divides a catalog purchase by a monthly rent you type.
Open the same model in the calculator: Llama 3.3 70B Q4. Hardware: GeForce RTX 4090 24GB.