One model per GPU, or split across two cards?

If a model fits in one card's usable memory (about 23.4 GB on a 24 GB card, or 31.4 GB on a 32 GB card, our estimate), keep the whole model on that card and use the second card for another model. Split a model across two cards only when it is too large for one, such as Llama 3.3 70B. On many AMD B650 and B850 boards the second slot runs through the chipset at PCIe 4.0 x4, and a self-reported dual 32 GB build then ran much slower when one model was split.

Keep a model on one card when it fits

Every memory figure here is for a conversation of about 6,000 words (8,192 tokens), and it is our estimate.

GeForce RTX 3090 24GB has 23.4 GB usable in our estimate, after what the system keeps for itself (out of 24 GB). GeForce RTX 5090 32GB has 31.4 GB usable in our estimate, after what the system keeps for itself (out of 32 GB). Dual RTX 3090, 64 GB RAM has 46.8 GB usable in our estimate, after what the system keeps for itself (out of 48 GB).

Qwen3.8-27B at standard quality (Q4) needs 17.8 GB in our estimate (16.5 GB for the model itself, 0.5 GB for the conversation memory, 0.8 GB for the software). That fits on one 24 GB card and on one 32 GB card, so there is no need to split it. Qwen3.8-27B in the calculator.

Llama 3.3 70B at standard quality (Q4) needs 46.0 GB in our estimate (42.5 GB for the model itself, 2.7 GB for the conversation memory, 0.8 GB for the software). That does not fit on one 24 GB or one 32 GB card. Two 24 GB cards are the usual way to hold it. Two RTX 3090 cards for a 70B model.

Why a split can be slower

Our speed estimate multiplies the two cards' combined memory speed by 0.55. That factor is meant for a model that must span two cards. It is an assumption, not a measurement. For a model that already fits on one card, splitting layers means only one card works at a time, and the hidden state has to cross the link between the cards for every token.

Tensor split (both cards working on every layer) needs a fast link between the cards. On a chipset PCIe 4.0 x4 slot that link is shared and narrow, so the cards wait on each other.

The B650 and B850 second-slot trap

AMD's AM5 chipset page says a Ryzen desktop processor has 24 usable lanes of its own: 16 for graphics, 4 for the main SSD, and 4 for the chipset. B650 and B850 connect to the processor over PCIe 4.0 x4. AMD AM5 chipsets. On most of those boards the second long slot is wired through the chipset at PCIe 4.0 x4, not as a second x8 from the processor. TechSpot on AM5 boards. Boards that really split the processor's 16 lanes into x8/x8 say so in the manual.

A self-reported dual Intel Arc Pro B70

Josh Gold (github.com/killercatfish, llama.cpp issue 30892, quoted by name with permission, 24 Oct 2026) runs two Intel Arc Pro B70 32 GB cards on a Ryzen 5 7600X and a B850 board. This is a community measurement, not ours, and it is labelled self-reported. Card A, on the processor's slot, serves Qwen3.8-27B Int4 with vLLM at about 30 tokens per second. A 10,000-token prompt takes about 7 seconds cold and about 0.6 seconds warm. Card B, in the chipset PCIe 4.0 x4 slot, runs llama.cpp on Vulkan at Q4_K_M: about 15 tokens per second of writing and about 375 to 500 tokens per second of reading. Splitting one model across both cards dropped writing to 5.7 tokens per second. One card also faulted under Intel's compute stack; a slot swap later showed that card was defective, not the slot. On 26 October 2026 the faulty card failed 3 of 3 in the processor's x16 slot, and the healthy card ran 13 of 13 clean in the chipset PCIe 4.0 x4 slot. That card is going back to ASRock for replacement. One model per card on the B850 board is unchanged, and a board that does x8/x8 is still the next step when one model has to span both cards. A SYCL build with F16 off read the prompt at about 270 tokens per second and stayed correct; F16 on read faster, about 640 tokens per second, and then printed NaN marks (llama.cpp 30892, and others point at pull request 30604).

Source: llama.cpp issue 30892. The written interview is Josh Gold on two Arc Pro B70 cards. The same note is on Intel Arc Pro B70 32GB and on Can I run Qwen3.8-27B.

Other builder rigs

Gabe Ortiz runs q27 (Quasar) on an RTX 5090 32 GB and Whisper large v3 on an RTX 3090 24 GB in the same workstation, one job per card. Gabe Ortiz on an RTX 5090 and an RTX 3090. Self-reported.

Roshan George runs vLLM on one used RTX A6000 48 GB, shared today with camera decode, embeddings, and speech. He already owns a second RTX A6000 and plans to put vLLM on one card and the other jobs on the second. Roshan George on an RTX A6000 48 GB. Self-reported.

James Meadlock runs an eight-node DGX Spark cluster as one eight-node job (GLM-5.3 at NVFP4) or as two four-node jobs (DeepSeek V4.1 Flash 1017 on four nodes, Imagination Studio on the other four). James Meadlock on eight DGX Spark nodes. Self-reported.

Pin one model to each card

When each card holds its own model, set which card the program may see. These are the usual commands, not a benchmark:

CUDA_VISIBLE_DEVICES=0 llama-server -m model-a.gguf -c 8192 -ngl 99 --port 8080
CUDA_VISIBLE_DEVICES=1 llama-server -m model-b.gguf -c 8192 -ngl 99 --port 8081

For a model that needs both cards, layer split is the usual start:

llama-server -m model-70b.gguf -c 8192 -ngl 99 -sm layer -ts 1,1 --port 8080

Recursive Flint at standard quality (Q4) needs 24.5 GB in our estimate (23.4 GB for the model itself, 0.3 GB for the conversation memory, 0.8 GB for the software) at the standard file. A smaller 3-bit file of Recursive Flint fits on one 24 GB card. See Can I run Recursive Flint.

Speeds on the calculator are our estimates; How we calculate explains them. We have not timed these models on a real machine, and this guide adds no measured result.

Related: Dual RTX 3090 build for 70B.