Does Llama 70B fit on an RTX 4090?

Not in full at Q4. The estimate sits around 40 GB, and a 4090 has 24 GB. Two 24 GB cards, or a Mac with enough unified memory, can. That is still an estimate, not a measured GGUF.

Are the tokens per second measured?

No. Tokens per second are estimated from memory bandwidth. There is no benchmark source: benchmarkSourceUrl stays null.

Are the prices accurate?

Global prices (USD), EUR available, dated 2026-10-14. EUR includes VAT. USD is an FX estimate. There is no shop source: sourceUrl stays null.

Does the newsletter store my address?

Only when the newsletter is open. No email is sent. While it is closed, nothing is stored.

Local inference, no cloud

Which hardware do you need to run your LLM locally?

Give a budget, a model, or both. You get three concrete setups: enough memory or not, an estimated speed, and a price we label as an estimate. Global prices (USD), EUR available.

11 models · 19 setups

Common questions

Does Llama 70B fit on an RTX 4090?: Not in full at Q4. The estimate sits around 40 GB, and a 4090 has 24 GB. Two 24 GB cards, or a Mac with enough unified memory, can. That is still an estimate, not a measured GGUF.
Are the tokens per second measured?: No. Tokens per second are estimated from memory bandwidth. There is no benchmark source: benchmarkSourceUrl stays null.
Are the prices accurate?: Global prices (USD), EUR available, dated 2026-10-14. EUR includes VAT. USD is an FX estimate. There is no shop source: sourceUrl stays null.
Does the newsletter store my address?: Only when the newsletter is open. No email is sent. While it is closed, nothing is stored.

Your question

Common questions