Hardware
PC with RTX 4090 24GB
Fast for everything up to about 32B. Not a 70B machine, whatever the box costs.
Memory
24 GB VRAMBandwidth
1008 GB/sBandwidth source is not stored.
Price
$3,235€2,890
14 Oct 2026
price unverifiedParts in the catalog
- GeForce RTX 4090 24GB
- Ryzen 7 7700 or similar
- 64 GB DDR5
- 1000W PSU
Which models fit
Estimate at 8,192 tokens, or the model's maximum when that is shorter. full means the formula fits in usable memory. offload means GPU plus system RAM. no means it does not fit. This is not a measured file.
| Model | Q4estimate | Q5estimate | Q8estimate | FP16estimate |
|---|---|---|---|---|
| Llama 3.2 3B | full | full | full | full |
| Llama 3.1 8B | full | full | full | full |
| Mistral 7B v0.3 | full | full | full | full |
| Gemma 2 9B | full | full | full | full |
| Qwen2.5 14B | full | full | full | offload |
| Phi-4 14B | full | full | full | offload |
| Gemma 2 27B | full | full | offload | offload |
| Qwen2.5 32B | full | offload | offload | offload |
| Llama 3.3 70B | offload | offload | no | no |
| Qwen2.5 72B | offload | offload | no | no |
| DeepSeek-R1-Distill 70B | offload | offload | no | no |