Can I run it? Twelve models, nine GPUs

Q4_K_M fit at 8,192 tokens for 12 models on 9 GPUs (estimate). A tick is fully in VRAM. Offload and no stay labelled. Llama 3.3 70B on an RTX 4090 opens the written guide.

Marks are a formula estimate at Q4_K_M and 8,192 tokens: ✓ fully in VRAM, offload means CPU offload, ✗ does not fit. Llama 3.3 70B on an RTX 4090 stays on the written guide.

Model	RTX 3060 12GB	RTX 4060 Ti 16GB	RTX 3090	RTX 4090	RTX 5080	RTX 5090	RX 7900 XTX	Dual RTX 3090	Dual RTX 4090
Llama 3.2 3B	✓	✓	✓	✓	✓	✓	✓	✓	✓
Llama 3.1 8B	✓	✓	✓	✓	✓	✓	✓	✓	✓
Mistral 7B v0.3	✓	✓	✓	✓	✓	✓	✓	✓	✓
Gemma 2 9B	✓	✓	✓	✓	✓	✓	✓	✓	✓
Qwen2.5 14B	✓	✓	✓	✓	✓	✓	✓	✓	✓
Phi-4 14B	✓	✓	✓	✓	✓	✓	✓	✓	✓
Gemma 2 27B	✗	✗	✓	✓	✗	✓	✓	✓	✓
Qwen2.5 32B	✗	✗	✓	✓	✗	✓	✓	✓	✓
Llama 3.3 70B	✗	✗	✗	✗	✗	✗	✗	✓	✓
Qwen2.5 72B	✗	✗	✗	✗	✗	✗	✗	offload	offload
DeepSeek-R1-Distill 70B	✗	✗	✗	✗	✗	✗	✗	✓	✓
NVIDIA Nemotron 3.5 Bolt 74B-A9B	✗	✗	✗	✗	✗	✗	✗	✓	✓