Framework Desktop (128GB) Ryzen AI Max+ 395 Strix Halo LLM benchmark (Valérian de Thézan de Gaussan)

Measured speeds on the Framework Desktop

Rig owner: Valérian de Thézan de Gaussan (https://valerian.dtdg.fr/). Rig: Framework Desktop — AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU, 128 GB shared LPDDR5X (~111.56 GiB exposed to Vulkan).

The 28 October 2026 follow-up used Arch Linux, Mesa RADV 26.2.5-arch1.1, and llama.cpp (commit f5222f25c, Vulkan + KHR cooperative matrices, llama-bench -p 512 -n 128 -t 6 -ngl 999 -r 5). The post is https://dtdg.fr/blog/2026/amd-strix-halo-llm-benchmark-follow-up/.

Mistral-Small-4-119B-2603-UD-Q4_K_M (119B total / 6.5B active MoE, 68.70 GiB GGUF): 675.07 ± 2.995 tok/s prefill (pp512) and 32.72 ± 0.020 tok/s decode (tg128).

Mistral-Small-4-119B-2603-UD-Q5_K_M (83.04 GiB GGUF): 611.02 ± 3.467 tok/s prefill and 29.83 ± 0.018 tok/s decode.

Qwen3.8-27B-UD-Q4_K_M (15.33 GiB GGUF): 413.77 ± 3.058 tok/s prefill and 12.82 ± 0.002 tok/s decode. UD-Q6_K (20.47 GiB) decoded at 9.71 tok/s. Q8_0 (27.05 GiB) decoded at 7.50 tok/s with 476.89 tok/s prefill.

gemma-4-31B-it-Q4_K_M (17.07 GiB GGUF): 359.99 ± 1.082 tok/s prefill and 11.27 ± 0.002 tok/s decode. Q6_K (23.47 GiB) decoded at 8.10 tok/s. Q8_0 (30.39 GiB) decoded at 6.46 tok/s with 412.12 tok/s prefill.

GLM-5.3-Flash-UD-IQ1_S (320B total / 18B active MoE, 86.69 GiB GGUF, 83.76 GiB Vulkan model buffers): 196.59 ± 0.786 tok/s prefill and 19.02 ± 0.020 tok/s decode.

What models/quants do you run locally on the Framework Desktop?

If you meant what kind of models do I run on the Framework Desktop, then basically any models where the architecture allows at least 30tk/s in output. Since the desktop has a huge bottleneck in the memory bandwith, not all models, even small ones, are suited to run on it. I Just ran a new benchmark today with newer models. Check it out here: https://dtdg.fr/blog/2026/amd-strix-halo-llm-benchmark-follow-up/

2025 baseline on the same Strix Halo machine

His 2025 baseline measurements are at https://valerian.dtdg.fr/blog/2025/amd-strix-halo-ai-395-llm-benchmark/.

gpt-oss-20b MXFP4 (11.27 GB): 1,216 tok/s prefill and 77 tok/s decode.

gpt-oss-120b MXFP4 (59.02 GB): 500 tok/s prefill and 54 tok/s decode.

Llama-3.2 3B Q4_K_XL (1.91 GB): 1,912 tok/s prefill and 93 tok/s decode (28 tok/s in BF16).

Llama-4-Scout 17B (16E) Q4_K_XL (57.73 GB): 225 tok/s prefill and 20.2 tok/s decode.

Llama-3.3 70B Q4_K_XL (39.73 GB): 78 tok/s prefill and 5.05 tok/s decode.

What are you weighing for your next machine or upgrade, and what is the budget roughly?

Honestly for me local AI is not there yet. If you need frontier model intelligence level, go pay a provider. Especially since there are no big open source competitors to chatgpt or claude at the moment. So if you need the best models, just go there. Now, depending on the workload, a local machine can make sense. Me personnaly, I'm fine with my machine, because it fits my workload, but bear in mind than my workload is simple and not heavy (some summarization, a bit of agentic for my home lab, and mainly synthetic dataset generation). So to answer your question, I'm not investing in 2026 in local AI until we can really host near-frontier models locally.

Where do you look for memory and tokens/s numbers before you buy, and what is missing there?

I didn't because I just needed a new computer and saw that this one was apparently capable of running medium sized LLMs thanks to the 128GB unified RAM, so I bought it, and tested it with LLMs.

What would make you trust a tokens/s figure you didn't measure yourself?

A collaborative platform where people can send their benchmarks. With enough people you can see what hardware can do what.

Anything you wish a 'which hardware for this model' site told you?

'Which hardware for this model' is a good idea. If I had a webpage like this in front of me, I would want to know what are the input AND output token processing capabilities, for each of the different tokenizations of that model, as well as the size on disk and VRAM when loaded.

If your Framework Desktop broke tomorrow, would you buy it again, or rent GPU time instead?

I would rent GPU time. But it's such a nice machine for other things that I would probably buy another, not for the LLM part, but for the silent, cool little box on my desk :)