Dual Intel Arc Pro B70 (64GB) local LLM: vLLM and llama.cpp (Josh Gold)

The rig

Measured dual Intel Arc Pro B70 (64 GB) local LLM throughput, self-reported: Card A, vLLM (cyspiegel/vllm-xpu-b70), Qwen3.8-27B Int4 AutoRound, about 30 tokens per second decode, with a 10,000-token prefill of about 7 seconds cold and 0.6 seconds warm. Card B, llama.cpp Vulkan (Mesa 26.2.5), Qwen3.8-27B Q4_K_M, about 15 tokens per second decode. Splitting one model across PCIe 4.0 x16 and chipset PCIe 4.0 x4 wrote at 5.7 tokens per second, so one model per card wins.

One headless box, Ubuntu 26.04 (kernel 7.0): a Ryzen 5 7600X, 32 GB of DDR5, a B850 motherboard, and two Intel Arc Pro B70 cards with 32 GB each (64 GB of graphics memory in total). Each card runs its own model.

Card A runs vLLM from the cyspiegel/vllm-xpu-b70 image and serves Qwen3.8-27B Int4 (AutoRound) full time. It powers KIRA, his home assistant, which he talks to from his phone. It writes about 30 tokens per second. With the prompt cache, a 10,000-token prompt takes about 7 seconds cold and about 0.6 seconds warm, and a phone answer takes about 6 to 15 seconds.

Card B runs llama.cpp on Vulkan (Mesa 26.2.5) behind llama-swap, one model at a time: Qwen3.8-27B Q4_K_M, a Q6 file for long editing jobs, and a couple of others. It reads the prompt at about 375 to 500 tokens per second and writes at about 15 tokens per second.

Next for KIRA: load her fixed instructions into the cache before the first question, so even the first turn starts warm, and stream the reply so the first words show up in a second or two instead of waiting for the whole answer.

The 270 tokens per second in his llama.cpp issue was a SYCL test build. That run was correct, but on card A vLLM is much faster. Vulkan on the second card has stayed up. SYCL faulted about 1 run in 3. A slot swap on 26 October 2026 showed that fault followed the physical card: it failed 3 of 3 in the CPU x16 slot, while the healthy card ran 13 of 13 clean in the B850 chipset PCIe 4.0 x4 slot. The defective card is going back to ASRock for replacement.

The next upgrade

A third B70 plus a memory upgrade, roughly $2,000 or more. It is parked for now. The real limit for a split model is the motherboard: on a B850 the second card runs through the chipset at PCIe 4.0 x4, so splitting one model across both cards was slow (5.7 tokens per second). One card also faulted under Intel's compute stack; a slot swap later showed that card was defective, not the slot. On 26 October 2026 the faulty card failed 3 of 3 in the CPU x16 slot, while the healthy card ran 13 of 13 clean in the B850 chipset PCIe 4.0 x4 slot. The defective card is going back to ASRock for replacement. Splitting one model across both cards over PCIe 4.0 x4 is still 5.7 tokens per second. One model per card on the B850 is unchanged: about 30 tokens per second on vLLM and about 15 tokens per second on Vulkan. A board that does x8/x8 is still the next step when one model has to span both cards.

Where he looks before buying

For prices and availability: Newegg, Micro Center, Best Buy, Amazon, and sometimes B&H Photo. None of them give inference numbers. You get graphics memory, clocks and power draw, and that is it. The tokens-per-second and will-it-fit part he pieces together afterwards from GitHub issues and other people's posts. What is missing is any link between a card's product page and how real models actually run on it.

What would make a speed figure trustworthy

The backend and build number, the driver version, the quant, the prompt length, and whether it is one card or split across two. Above all, proof the output was correct. On his B70, turning F16 on in the SYCL build more than doubled prompt speed, from about 270 to about 640 tokens per second, and every answer came out as garbage. Others report that bug is now fixed by llama.cpp pull request 30604. He has not retested yet. A speed number without a correctness check can be worse than no number.

His needle check puts "Note for the record: the access code for the blue garden shed is RIVER-4718." in the middle of about 12K tokens of public-domain filler text (a public-domain book works), then asks "What is the access code for the blue garden shed? Reply with the code only." He uses greedy decoding with thinking off. The run passes when the answer contains RIVER-4718. That check caught SYCL F16 returning garbage at about twice the speed.

Would he buy the two Arc Pro B70s again?

Yes: great value for building something real.

What he wishes a hardware site told him

Which PCIe slot each card actually gets on a given board. Whether one model across two cards really beats one model per card. Which backend is stable on which card, not only which one is fastest. The driver and runtime versions a number was measured on, because on Arc those change the result from month to month.