estimate, except measured file sizes
Can I run DeepSeek-R1-Distill 70B locally?
70.6B parameters. Parameter count is an estimate.
- Licence:
- not recorded
- Context:
- 131,072 tokens
- Modality:
- not recorded
- Released:
- not recorded
A distill on a Llama 70B shape. Architecture is copied from Llama 3.3 70B and marked as an estimate: no separate config was fetched.
Memory per quant
KV is at 8,192 tokens. Overhead is 0.8 GB. Both are estimates. A measured file size is labelled. The last column is the cheapest machine that fits the total fully.
| Quant | File size | KV at 8K | +0.8 overhead | Total | Cheapest full fit |
|---|---|---|---|---|---|
| Q8 | 75.5 GB est. | 2.7 GB est. | 0.8 GB est. | 79.0 GB est. | Mac Studio M4 Max 128GB, €3,290 (estimate) |
| Q4 | 42.4 GB est. | 2.7 GB est. | 0.8 GB est. | 45.8 GB est. | Mac mini M4 Pro 64GB, €1,890 (estimate) |
Cheapest rig that fits
Q4, full: Mac mini M4 Pro 64GB, €1,890 (estimate)
Q4, slow, partly in system RAM: Budget PC with RTX 3060 12GB, slow, partly in system RAM, €1,357 (estimate)
On a specific GPU
- DeepSeek-R1-Distill 70B on RTX 3060 12GB
- DeepSeek-R1-Distill 70B on RTX 4060 Ti 16GB
- DeepSeek-R1-Distill 70B on RTX 3090
- DeepSeek-R1-Distill 70B on RTX 4090
- DeepSeek-R1-Distill 70B on RTX 5080
- DeepSeek-R1-Distill 70B on RTX 5090
- DeepSeek-R1-Distill 70B on RX 7900 XTX
- DeepSeek-R1-Distill 70B on Dual RTX 3090
- DeepSeek-R1-Distill 70B on Dual RTX 4090