Hardware
Used PC with RTX 3090 24GB
Best single card here for 32B Q4. 70B Q4 does not fully fit.
Memory
24 GB VRAMBandwidth
936 GB/sBandwidth source is not stored.
Price
$1,433€1,280
14 Oct 2026
price unverifiedParts in the catalog
- GeForce RTX 3090 24GB, used
- Ryzen 7 5700X or similar
- 64 GB DDR4
- 850W PSU, decent airflow
Which models fit
Estimate at 8,192 tokens, or the model's maximum when that is shorter. full means the formula fits in usable memory. offload means GPU plus system RAM. no means it does not fit. This is not a measured file.
| Model | Q4estimate | Q5estimate | Q8estimate | FP16estimate |
|---|---|---|---|---|
| Llama 3.2 3B | full | full | full | full |
| Llama 3.1 8B | full | full | full | full |
| Mistral 7B v0.3 | full | full | full | full |
| Gemma 2 9B | full | full | full | full |
| Qwen2.5 14B | full | full | full | offload |
| Phi-4 14B | full | full | full | offload |
| Gemma 2 27B | full | full | offload | offload |
| Qwen2.5 32B | full | offload | offload | offload |
| Llama 3.3 70B | offload | offload | no | no |
| Qwen2.5 72B | offload | offload | no | no |
| DeepSeek-R1-Distill 70B | offload | offload | no | no |