What did Danylo Chernov's Hetzner Cloud CPU LLM benchmark (llm-cpu-bench) test?

In llm-cpu-bench (github.com/DanielChernov7/llm-cpu-bench, self-reported by Danylo Chernov of NoctHost, nocthost.com), he benchmarked 5 Ollama Q4_K_M models (llama3.2:3b, qwen2.5:3b, llama3.1:8b, qwen2.5:7b, and qwen2.5:14b) across three Hetzner Cloud CPU servers: CX23 (2 shared vCPU, 4 GB RAM), CX33 (4 shared vCPU, 8 GB RAM), and CCX23 (4 dedicated vCPU, 16 GB RAM), measuring generation speed, prompt processing speed, cold load time, and peak RAM.

Why does prompt processing speed (time to first token) matter so much on CPU servers?

As Danylo Chernov reports from his llm-cpu-bench runs (self-reported, community measurement, not ours), most people only quote generation tokens/s, but on an ordinary CPU server a 720-token prompt can take 40 seconds before the first word appears. He also found that a 4-core dedicated AMD EPYC (Hetzner CCX23) was about 2x faster than a 4-core shared Intel Xeon (CX33). For models larger than 14B, his team at NoctHost rents NVIDIA A16 and A40 GPUs by the hour instead of running on CPU.

Danylo Chernov (@DanielChernov7, NoctHost): What Happens When You Run 3B–14B Ollama Models on Ordinary Cloud CPUs

Danylo Chernov (@DanielChernov7) runs NoctHost (https://nocthost.com) and published https://github.com/DanielChernov7/llm-cpu-bench. He timed five Ollama Q4_K_M models (llama3.2:3b, qwen2.5:3b, llama3.1:8b, qwen2.5:7b, and qwen2.5:14b) on Hetzner CX23 (2 shared vCPU, 4 GB), CX33 (4 shared vCPU, 8 GB) and CCX23 (4 dedicated vCPU, 16 GB). There is no accuracy check. A 720-token prompt can take 40 seconds before the first word on CPU, and a 4-core dedicated EPYC was about 2x faster than a 4-core shared Xeon. For anything bigger than 14B his team rents NVIDIA A16 and A40 GPUs by the hour. Self-reported, community measurement, not ours.

Danylo Chernov, quoted by name with written permission. He asked that the repo and nocthost.com be linked. 27 Oct 2026. Self-reported. Not a localLLMrig measurement.

Test Gemma 4 31B, Recursive Flint and Nemotron 3.5 Super VL side by side in your browser, free.

github.com/DanielChernov7/llm-cpu-bench. nocthost.com.

What the benchmark covers

The repo benchmarks 5 Ollama models (llama3.2:3b, qwen2.5:3b, llama3.1:8b, qwen2.5:7b and qwen2.5:14b, all Q4_K_M) on three Hetzner Cloud servers: CX23 (2 shared vCPU, 4 GB), CX33 (4 shared vCPU, 8 GB) and CCX23 (4 dedicated vCPU, 16 GB). It measures generation speed, prompt processing speed, cold load time and peak RAM. There is no accuracy check.

Repo: github.com/DanielChernov7/llm-cpu-bench. NoctHost: nocthost.com.

What models, quantizations, and serving stack do you run today — and where do you draw the line between CPU servers and GPU servers?

We don't run models on a home machine. We run NoctHost (nocthost.com), a cloud hosting service, so we test on rented servers: the three CPU boxes above. For anything bigger than 14B we would move to a rented GPU server.

The benchmark script is at github.com/DanielChernov7/llm-cpu-bench.

New model hardware requirements and price updates by email. We email a link to confirm the address. Nothing is sent until you click it. Privacy notice.

Why do you rent A16 and A40 GPUs by the hour instead of buying local GPU hardware?

We're not buying hardware. For bigger models we rent GPUs by the hour (A16 and A40), because for occasional workloads that is cheaper than owning a card. The whole CPU benchmark cost us a few dollars of server time.

NoctHost is at nocthost.com.

Where do you look for memory and speed numbers today, and what is missing from most LLM benchmarks?

Mostly GitHub issues, Reddit threads and our own runs. What's missing is CPU-only numbers on ordinary cloud servers, and especially prompt processing speed. Everyone quotes generation tokens/s, but on CPU a 720-token prompt can take 40 seconds before the first word appears.

What makes you trust a benchmark figure — or ignore it?

The exact command, the Ollama version, the quantization, the prompt length and the raw output. That's why the repo (github.com/DanielChernov7/llm-cpu-bench) publishes all of it, so anyone can rerun the same script and compare.

What is the #1 thing you wish a hardware-fit site told you before deploying a model on CPU or rented cloud hardware?

Two things. First, shared vs dedicated vCPU: in our test a 4-core dedicated EPYC was about 2x faster than a 4-core shared Xeon, though the EPYC is also a newer generation. Second, the time to first token for a realistic prompt, not just generation speed. That is what decides whether a model feels usable.

On our list: Rent a GPU, Buy or rent?, Mac vs PC for a local LLM, Can I run Qwen3.8-27B?, Can I run Gemma 4 31B?.

The numbers on this page were measured by the owner on that hardware. They are not a localLLMrig benchmark, and they do not change the fit scores in the calculator.

Questions

What did Danylo Chernov's Hetzner Cloud CPU LLM benchmark (llm-cpu-bench) test?: In llm-cpu-bench (github.com/DanielChernov7/llm-cpu-bench, self-reported by Danylo Chernov of NoctHost, nocthost.com), he benchmarked 5 Ollama Q4_K_M models (llama3.2:3b, qwen2.5:3b, llama3.1:8b, qwen2.5:7b, and qwen2.5:14b) across three Hetzner Cloud CPU servers: CX23 (2 shared vCPU, 4 GB RAM), CX33 (4 shared vCPU, 8 GB RAM), and CCX23 (4 dedicated vCPU, 16 GB RAM), measuring generation speed, prompt processing speed, cold load time, and peak RAM.
Why does prompt processing speed (time to first token) matter so much on CPU servers?: As Danylo Chernov reports from his llm-cpu-bench runs (self-reported, community measurement, not ours), most people only quote generation tokens/s, but on an ordinary CPU server a 720-token prompt can take 40 seconds before the first word appears. He also found that a 4-core dedicated AMD EPYC (Hetzner CCX23) was about 2x faster than a 4-core shared Intel Xeon (CX33). For models larger than 14B, his team at NoctHost rents NVIDIA A16 and A40 GPUs by the hour instead of running on CPU.

Find hardware · All builder interviews · Send us your llama-bench result