Can I run Mistral Small 3.2 24B locally?
Mistral Small 3.2 24B needs about 16.5 GB of graphics memory. It runs on 6 of the 9 graphics card setups on this page, such as the RTX 3090.
The cheapest graphics card on our list that runs it (it still needs the rest of a PC): GeForce RTX 3090 24GB, $1,370€1,224, used (median of sold listings).
Fast~35 words/sAnswers appear faster than you can read them.Our price is for a used card. These shops mostly sell new ones, so their price can be higher.
EU shops · In the US? Switch the currency to USD.
US shops · In Europe? Switch the currency to EUR.
The cheapest complete machine that runs it: Mac mini M6 32GB, $1,416€1,530. price unverified Slow~3.9 words/s
Which graphics cards run it
Seven popular graphics cards and two PCs with two cards each, at the standard quality (Q4). Click one for the full answer: memory, speed and price.
Runs well6
The whole model fits in the graphics memory.
Doesn't fit3
Not enough graphics memory: it needs about 16.5 GB. In a PC with enough normal memory it runs slowly: Budget PC with RTX 3060 12GB, $1,305€1,357. price unverified Slow~2.9 words/s
- RTX 3060 12GBhas 12 GB
- RTX 4060 Ti 16GBhas 16 GB
- RTX 5080has 16 GB
Cheapest hardware for each size
From all 36 graphics cards, PCs and Macs on our list. "Runs slowly" means part of the model runs from normal memory: cheaper, but much slower. A smaller download needs less memory and answers a little less well.
- Standard quality (Q4_K_M)GeForce RTX 3090 24GB, $1,370€1,224, used (median of sold listings)
- Standard quality (Q4_K_M), runs slowlyBudget PC with RTX 3060 12GB, $1,305€1,357 price unverified
- Smallest download (IQ2_M)GeForce RTX 3060 12GB, $288€257, used (median of sold listings)
- Smallest download (IQ2_M), runs slowlyNo machine on our list runs this size.
Prices excl. shipping. Euro prices incl. VAT. Dollar prices are converted from our euro prices without EU VAT, before US sales tax.
Prices moved: see Building a local LLM rig in the RAMageddon.
About Mistral Small 3.2 24B
24.01B parameters. Parameter count is an estimate. More parameters usually means a smarter model that needs more memory.
- Licence:
- Apache-2.0 source
- Longest conversation:
- about 197 pages of text (131,072 tokens)
- Input:
- text + image in, text out
Technical notes
Mistral's mid-size open model for chat, documents and image input, under Apache 2.0. At our default quality it needs about 16.5 GB, so it just misses a 16 GB graphics card: a slightly smaller download fits one, and a 24 GB card holds the default.
Memory at each quality level
Each download of Mistral Small 3.2 24B is a different size. The total adds the file, the memory a conversation of about 12 pages of text (8,192 tokens) takes, and 0.8 GB of working space. A measured file size is labelled; the rest is our estimate. The last column is the cheapest machine that holds the total.
Swipe the table sideways to see every column, up to the cheapest machine.
| Quality (file) | File size | Conversation | Working space | Total | Cheapest machine that holds it |
|---|---|---|---|---|---|
| BF16 | 47.15 GBmeasured GGUF | 1.3 GB | 0.8 GB | 49.3 GB | Mac Studio M4 Max 128GB, $3,683€3,290 price unverified |
| Q8_0 | 25.05 GBmeasured GGUF | 1.3 GB | 0.8 GB | 27.2 GB | Mac mini M4 Pro 64GB, $2,116€1,890 price unverified |
| Q6_K | 19.35 GBmeasured GGUF | 1.3 GB | 0.8 GB | 21.5 GB | GeForce RTX 3090 24GB, $1,370€1,224, used (median of sold listings) |
| Q5_K_M | 16.76 GBmeasured GGUF | 1.3 GB | 0.8 GB | 18.9 GB | GeForce RTX 3090 24GB, $1,370€1,224, used (median of sold listings) |
| Q4_K_M | 14.33 GBmeasured GGUF | 1.3 GB | 0.8 GB | 16.5 GB | GeForce RTX 3090 24GB, $1,370€1,224, used (median of sold listings) |
| IQ4_XS | 12.76 GBmeasured GGUF | 1.3 GB | 0.8 GB | 14.9 GB | Radeon RX 9060 XT 16GB, $522€564, new (cheapest in-stock board on GPU PRIX's German board list) |
| Q3_K_M | 11.47 GBmeasured GGUF | 1.3 GB | 0.8 GB | 13.6 GB | Radeon RX 9060 XT 16GB, $522€564, new (cheapest in-stock board on GPU PRIX's German board list) |
| IQ3_XXS | 9.28 GBmeasured GGUF | 1.3 GB | 0.8 GB | 11.4 GB | Radeon RX 9060 XT 16GB, $522€564, new (cheapest in-stock board on GPU PRIX's German board list) |
| IQ2_M | 8.11 GBmeasured GGUF | 1.3 GB | 0.8 GB | 10.3 GB | GeForce RTX 3060 12GB, $288€257, used (median of sold listings) |
Waiting for prices to drop?
Speeds and memory are our estimate from the hardware specs, not a measured test. Prices are dated and may change. How we calculate