{"estimatesAreNotMeasurements":true,"priceAsOf":"2026-10-24","contextTokens":8192,"models":[{"id":"llama-3.2-3b","name":"Llama 3.2 3B","family":"Llama","paramsB":3.21,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":28,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/unsloth/Llama-3.2-3B-Instruct/raw/main/config.json","notes":"Small enough for a Mac mini or a modest GPU. A good stack to test your setup.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":1.93,"kvGb":0.94,"overheadGb":0.8,"totalGb":3.67,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":2.28,"kvGb":0.94,"overheadGb":0.8,"totalGb":4.02,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":3.43,"kvGb":0.94,"overheadGb":0.8,"totalGb":5.17,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":6.42,"kvGb":0.94,"overheadGb":0.8,"totalGb":8.16,"estimated":true,"sourceUrl":null}}}},{"id":"llama-3.1-8b","name":"Llama 3.1 8B","family":"Llama","paramsB":8.03,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":32,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/unsloth/Meta-Llama-3.1-8B-Instruct/raw/main/config.json","notes":"The default local model. Q4 fits in 8–12 GB, Q8 wants closer to 12–16 GB.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":4.82,"kvGb":1.07,"overheadGb":0.8,"totalGb":6.69,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":5.7,"kvGb":1.07,"overheadGb":0.8,"totalGb":7.58,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":8.59,"kvGb":1.07,"overheadGb":0.8,"totalGb":10.47,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":16.06,"kvGb":1.07,"overheadGb":0.8,"totalGb":17.93,"estimated":true,"sourceUrl":null}}}},{"id":"mistral-7b-v03","name":"Mistral 7B v0.3","family":"Mistral","paramsB":7.25,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":32,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":32768,"architectureEstimated":true,"architectureSourceUrl":null,"notes":"Architecture is an estimate of the familiar 7B shape. No config was fetched for this dataset.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":4.35,"kvGb":1.07,"overheadGb":0.8,"totalGb":6.22,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":5.15,"kvGb":1.07,"overheadGb":0.8,"totalGb":7.02,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":7.76,"kvGb":1.07,"overheadGb":0.8,"totalGb":9.63,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":14.5,"kvGb":1.07,"overheadGb":0.8,"totalGb":16.37,"estimated":true,"sourceUrl":null}}}},{"id":"gemma-2-9b","name":"Gemma 2 9B","family":"Gemma","paramsB":9.24,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":42,"kvHeads":8,"headDim":256,"fullAttentionKv":null,"maxContext":8192,"architectureEstimated":true,"architectureSourceUrl":null,"notes":"Gemma 2 alternates local and global attention. The conversation memory is an approximation, not a measured file.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":5.54,"kvGb":2.11,"overheadGb":0.8,"totalGb":8.46,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":6.56,"kvGb":2.11,"overheadGb":0.8,"totalGb":9.47,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":9.89,"kvGb":2.11,"overheadGb":0.8,"totalGb":12.8,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":18.48,"kvGb":2.11,"overheadGb":0.8,"totalGb":21.39,"estimated":true,"sourceUrl":null}}}},{"id":"qwen2.5-14b","name":"Qwen2.5 14B","family":"Qwen","paramsB":14.7,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":48,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Qwen/Qwen2.5-14B/raw/main/config.json","notes":"A strong mid-size model. Q4 needs about 10 GB plus context. Sliding window is off in the config.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":8.82,"kvGb":1.61,"overheadGb":0.8,"totalGb":11.23,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":10.44,"kvGb":1.61,"overheadGb":0.8,"totalGb":12.85,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":15.73,"kvGb":1.61,"overheadGb":0.8,"totalGb":18.14,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":29.4,"kvGb":1.61,"overheadGb":0.8,"totalGb":31.81,"estimated":true,"sourceUrl":null}}}},{"id":"phi-4","name":"Phi-4 14B","family":"Phi","paramsB":14.7,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":40,"kvHeads":10,"headDim":128,"fullAttentionKv":null,"maxContext":16384,"architectureEstimated":true,"architectureSourceUrl":null,"notes":"Architecture figures are an estimate. No config was fetched, so the memory figure can be slightly off.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":8.82,"kvGb":1.68,"overheadGb":0.8,"totalGb":11.3,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":10.44,"kvGb":1.68,"overheadGb":0.8,"totalGb":12.91,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":15.73,"kvGb":1.68,"overheadGb":0.8,"totalGb":18.21,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":29.4,"kvGb":1.68,"overheadGb":0.8,"totalGb":31.88,"estimated":true,"sourceUrl":null}}}},{"id":"gemma-2-27b","name":"Gemma 2 27B","family":"Gemma","paramsB":27.2,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":46,"kvHeads":16,"headDim":128,"fullAttentionKv":null,"maxContext":8192,"architectureEstimated":true,"architectureSourceUrl":null,"notes":"Same caveat as Gemma 2 9B: sliding window makes the KV estimate rougher.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":16.32,"kvGb":2.32,"overheadGb":0.8,"totalGb":19.44,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":19.31,"kvGb":2.32,"overheadGb":0.8,"totalGb":22.43,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":29.1,"kvGb":2.32,"overheadGb":0.8,"totalGb":32.22,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":54.4,"kvGb":2.32,"overheadGb":0.8,"totalGb":57.52,"estimated":true,"sourceUrl":null}}}},{"id":"qwen2.5-32b","name":"Qwen2.5 32B","family":"Qwen","paramsB":32.8,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":64,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Qwen/Qwen2.5-32B/raw/main/config.json","notes":"Q4 wants a bit over 20 GB. A 16 GB card drops out; a 24 GB card is tight.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":19.68,"kvGb":2.15,"overheadGb":0.8,"totalGb":22.63,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":23.29,"kvGb":2.15,"overheadGb":0.8,"totalGb":26.24,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":35.1,"kvGb":2.15,"overheadGb":0.8,"totalGb":38.04,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":65.6,"kvGb":2.15,"overheadGb":0.8,"totalGb":68.55,"estimated":true,"sourceUrl":null}}}},{"id":"llama-3.3-70b","name":"Llama 3.3 70B","family":"Llama","paramsB":70.6,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":80,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json","notes":"Standard quality (Q4) is the Q4_K_M download. Our estimate at 0.60 bytes per parameter is about 42 GB of weights. A bartowski Llama-3.3-70B-Instruct Q4_K_M file is 42.52 GB (42,520,398,816 bytes). A single 24 GB or 32 GB card is not enough.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":42.52,"kvGb":2.68,"overheadGb":0.8,"totalGb":46,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":50.13,"kvGb":2.68,"overheadGb":0.8,"totalGb":53.61,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":75.54,"kvGb":2.68,"overheadGb":0.8,"totalGb":79.03,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":141.2,"kvGb":2.68,"overheadGb":0.8,"totalGb":144.68,"estimated":true,"sourceUrl":null}}}},{"id":"qwen2.5-72b","name":"Qwen2.5 72B","family":"Qwen","paramsB":72.7,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":80,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Qwen/Qwen2.5-72B/raw/main/config.json","notes":"Same class as Llama 70B. A bartowski Qwen2.5-72B-Instruct Q4_K_M file is 47.42 GB (47,415,715,488 bytes), larger than 72.7 × 0.60. IQ4_XS from the same repo is 39.71 GB. You need a large memory pool.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":47.42,"kvGb":2.68,"overheadGb":0.8,"totalGb":50.9,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":51.62,"kvGb":2.68,"overheadGb":0.8,"totalGb":55.1,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":77.79,"kvGb":2.68,"overheadGb":0.8,"totalGb":81.27,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":145.4,"kvGb":2.68,"overheadGb":0.8,"totalGb":148.88,"estimated":true,"sourceUrl":null}}}},{"id":"deepseek-r1-distill-70b","name":"DeepSeek-R1-Distill 70B","family":"DeepSeek","paramsB":70.6,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":null,"layers":80,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":true,"architectureSourceUrl":null,"notes":"A distill on a Llama 70B shape. Architecture is copied from Llama 3.3 70B and marked as an estimate: no separate config was fetched.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":42.36,"kvGb":2.68,"overheadGb":0.8,"totalGb":45.84,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":50.13,"kvGb":2.68,"overheadGb":0.8,"totalGb":53.61,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":75.54,"kvGb":2.68,"overheadGb":0.8,"totalGb":79.03,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":141.2,"kvGb":2.68,"overheadGb":0.8,"totalGb":144.68,"estimated":true,"sourceUrl":null}}}},{"id":"deepseek-r1-distill-qwen-14b","name":"DeepSeek R1 Distill Qwen 14B","family":"DeepSeek","paramsB":14.77,"activeParamsB":null,"paramsEstimated":false,"paramsSourceUrl":"https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B","layers":48,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B/raw/main/config.json","notes":"DeepSeek's dense reasoning distill of DeepSeek-R1 onto Qwen2.5-14B, MIT licence. Hugging Face safetensors metadata lists 14,770,033,664 parameters (14.77B). 48 layers, 8 KV heads, head dim 128, context 131,072. The Q4_K_M GGUF is 8.99 GB (unsloth, 24 Oct 2026). At 8,192 tokens the rounded parts are 8.99 GB weights + 1.61 GB conversation memory + 0.80 GB working space = 11.40 GB, which equals the 11.4 GB usable line of a 12 GB card. The unrounded sum is a fraction over 11.4 GB, so the fit table does not mark Q4 as a full fit on 12 GB. IQ4_XS is an 8.12 GB file, about 10.53 GB in total at 8,192 tokens (our estimate), and that fits a 12 GB card. Only the 2-bit IQ2_XS file (4.70 GB, about 7.11 GB in total) fits an 8 GB card (about 7.4 GB usable). Ollama tag: deepseek-r1:14b.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":8.99,"kvGb":1.61,"overheadGb":0.8,"totalGb":11.4,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":10.49,"kvGb":1.61,"overheadGb":0.8,"totalGb":12.9,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":15.8,"kvGb":1.61,"overheadGb":0.8,"totalGb":18.21,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":29.54,"kvGb":1.61,"overheadGb":0.8,"totalGb":31.95,"estimated":true,"sourceUrl":null}}}},{"id":"nemotron-3-5-bolt-74b","name":"NVIDIA Nemotron 3.5 Bolt 74B-A9B","family":"Nemotron","paramsB":74,"activeParamsB":9,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB","layers":8,"kvHeads":2,"headDim":128,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":true,"architectureSourceUrl":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB/raw/main/config.json","notes":"Hugging Face model card, cited as 10/15/2026: 74B total, 9B active, OpenMDW-1.1, context up to 256K. Our memory estimate uses the 74B total. Our Q4, Q5, Q8 and FP16 figures are not the published 2-bit pack. The config's language stack is a hybrid of 40 Mamba, 40 MoE and 8 attention blocks; our conversation-memory estimate uses only those 8 attention blocks and is not NVIDIA's cache.","communityMeasurement":{"label":"community measurement, single run","sourceName":"llm-speed","sourceUrl":"https://llm-speed.com/blog/nemotron-3-5-bolt-74b-24gb-measured","published":"2026-10-16","note":"Published 16 Oct 2026. NVIDIA official 2-bit GGUF, stated file size 22.0 GB. llama.cpp 0.7.0-dev build 11669, commit 620d2225c, 4K context, chat-short, one run per config, runs dated 15 Oct 2026. Headline figures are 87.8, 107.3 and 154.7 tokens per second. The table prints 87.75 for the RTX 3090 and says the quant was not reported in that column. Not a localLLMrig measurement.","rows":[{"hardware":"RTX 3090","tokensPerSec":87.8},{"hardware":"RTX 4090","tokensPerSec":107.3},{"hardware":"RTX 5090","tokensPerSec":154.7}]},"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":44.4,"kvGb":0.07,"overheadGb":0.8,"totalGb":45.27,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":52.54,"kvGb":0.07,"overheadGb":0.8,"totalGb":53.41,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":79.18,"kvGb":0.07,"overheadGb":0.8,"totalGb":80.05,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":148,"kvGb":0.07,"overheadGb":0.8,"totalGb":148.87,"estimated":true,"sourceUrl":null}}}},{"id":"beam-1.0","name":"Reflection Beam 1.0","family":"Beam","paramsB":501.3,"activeParamsB":23.4,"paramsEstimated":false,"paramsSourceUrl":"https://huggingface.co/reflection/Beam-1.0-501B-A23B","layers":52,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":1048576,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/reflection/Beam-1.0-501B-A23B/raw/main/config.json","notes":"Text-only MoE, 256 experts, 8 active. Ollama offers beam:cloud only (no local tag, checked 24 Oct 2026, https://ollama.com/library/beam/tags). Official serving reference: 4× NVIDIA B300. Speed is not estimated: our speed estimate reads every weight for each token and would understate a sparse MoE.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":306.53,"kvGb":0.39,"overheadGb":0.8,"totalGb":307.72,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":355.92,"kvGb":0.39,"overheadGb":0.8,"totalGb":357.12,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":536.39,"kvGb":0.39,"overheadGb":0.8,"totalGb":537.58,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":1002.6,"kvGb":0.39,"overheadGb":0.8,"totalGb":1003.79,"estimated":true,"sourceUrl":null}}}},{"id":"nemotron-3.5-super-vl","name":"NVIDIA Nemotron 3.5 Super VL","family":"Nemotron","paramsB":120,"activeParamsB":12,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16","layers":8,"kvHeads":2,"headDim":128,"fullAttentionKv":null,"maxContext":1048576,"architectureEstimated":true,"architectureSourceUrl":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16/raw/main/config.json","notes":"Hybrid Mamba-2/Transformer: 88 blocks (40 mamba, 40 moe, 8 attention). Our conversation-memory estimate uses only those 8 attention blocks. It does not model the Mamba SSM state or the vision mmproj (+0.97–1.73 GB). 120B total and 12B active are the card's round billions, so our parameter count is an estimate. Local on Ollama as nemotron-3.5-super (default tag 80–90 GB). Speed is not estimated: our speed estimate reads every weight for each token and would understate a sparse MoE.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":71.8,"kvGb":0.07,"overheadGb":0.8,"totalGb":72.67,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":85.2,"kvGb":0.07,"overheadGb":0.8,"totalGb":86.07,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":128.4,"kvGb":0.07,"overheadGb":0.8,"totalGb":129.27,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":240,"kvGb":0.07,"overheadGb":0.8,"totalGb":240.87,"estimated":true,"sourceUrl":null}}}},{"id":"deepseek-v4.1-flash-1017","name":"DeepSeek V4.1 Flash 1017","family":"DeepSeek","paramsB":763.21,"activeParamsB":16,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017","layers":40,"kvHeads":1,"headDim":512,"fullAttentionKv":null,"maxContext":1048576,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/main/config.json","notes":"Mainline llama.cpp does not support DeepSeek V4.1 yet. The mrfakename community GGUFs need open PR https://github.com/ggml-org/llama.cpp/pull/28696 (still open, checked 24 Oct 2026) and are text-only. Unsloth's GGUFs need llama.cpp PR https://github.com/ggml-org/llama.cpp/pull/30817 or Unsloth's prebuilt. llama.cpp PR #30817 was still a draft, checked 24 Oct 2026. A tester reported wrong tokens when llama-server reuses a cached prompt prefix (workaround: cache_prompt false), and maintainers asked to replace the MXFP8 tensors, so the UD-Q3_K_XL/Q4_K_XL/Q8_K_XL files may change (https://github.com/ggml-org/llama.cpp/pull/30817). That support is not mainline, and it is not PR #28696. Ollama lists deepseek-v4.1-flash:cloud and deepseek-v4.1-flash:1017-cloud (both cloud tags, no local tag), checked 24 Oct 2026 (https://ollama.com/library/deepseek-v4.1-flash). Official weights: FP8 with FP4 experts, 510.3 GB on HF. Language backbone stored as 551.9B; the architecture section says \"552B backbone parameters\" (1017 is a minor update of https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash). Weights for this row: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017. Also 196.6B Engram conditional-memory parameters (sparse token lookup), 14.2B MTP head and 0.49B vision encoder. Total logical weights 763,205,315,794 (HF safetensors metadata, stored here as 763.21B). Q5, Q8 and FP16 use that total, not the 551.9B backbone: the Q4 file includes the Engram tables, and a backbone-only Q5 estimate was smaller than that file. Causal encoder-decoder, 40 layers (20 encoder + 20 decoder), 384 routed experts plus 1 shared, 6 routed per token, CSA2 compressed attention. KV is projected from encoder states with compressed and sliding attention (persistent KV about 1/8 of V4-Flash). The config sets compress_ratios and kv source layers 2, 8, 14 and 20, so we do not compute that cache and the conversation-memory figure stays at 0. Unsloth GGUF sizes, shards summed from the Hugging Face tree, decimal GB, checked 24 Oct 2026 (https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF, MIT): UD-IQ1_S 232.98 GB (232,979,608,475 bytes), UD-IQ1_M 250.40, UD-IQ2_XXS 300.48, UD-Q2_K_XL 334.79, UD-IQ3_XXS 394.74, UD-IQ3_S 423.63, UD-Q3_K_XL 485.07, UD-Q4_K_XL 500.71, UD-Q8_K_XL 508.38. The smallest llama.cpp-branch file on our list is UD-IQ1_S 232.98 GB. The mrfakename Q2_K file is 264.52 GB and is not the smallest. Each Unsloth quant includes an Engram lookup-table shard (the last shard): IQ1 110.60 GB (IQ4_NL), IQ2_XXS and Q2_K_XL 135.17 GB (Q5_K), IQ3 161.28 GB (Q6_K), Q3_K_XL, Q4_K_XL and Q8_K_XL 202.76 GB (MXFP8). Without that shard, UD-IQ1_S is 122.38 GB. Unsloth says a 1-bit quant needs 128–145 GB of RAM plus VRAM with the Engram table on SSD via mmap (https://unsloth.ai/docs/models/deepseek-v4.1-flash). That is a vendor claim, not tested by us. A 128 GB unified-memory machine is borderline on that claim. We do not score it as a fit: macOS leaves about 96 GB usable on a 128 GB Mac, and the full UD-IQ1_S file fits none of the 34 machines fully in VRAM. mmproj-F16 is 0.97 GB (971,004,064 bytes) and the DSpark drafter Q8_0 is 7.98 GB (7,976,914,240 bytes). Both are separate files and are not inside the quant totals. Other runtimes (not llama.cpp). Not a fit score. Baekpica VQ-GGUF [self-reported]: experimental low-bit release, routed experts about 1.6 bits per weight, files may be replaced, base file 113,556,639,424 bytes, runtime ds4-dfm-rs, serving speed not measured yet (https://huggingface.co/Baekpica/DeepSeek-V4.1-Flash-1017-VQ-GGUF). dsx 2-bit build v4b [self-reported]: 148 GB, KL 0.373, top-1 74.4%, perplexity 10.68. The card's 2× RTX 3090 plus 128 GB DDR4 figure is 28.9 tok/s decode on build v4, which it says has the same expert layout as v4b. RTX 5090 plus 128 GB DDR5 is 12.0 tok/s decode on build v2 (https://huggingface.co/lxe/DeepSeek-V4.1-Flash-1017-dsx-2bit). Project Maya / Maya-S, MIT [self-reported]: model shards 178.19 GB plus Engram Q6_K shards 161.28 GB, 339.48 GB together (https://huggingface.co/peasantsmith/DeepSeek-V4.1-Flash-1017-Maya-GGUF), runtime https://github.com/mw00/project-maya. [community] One post reports an RTX 3090 plus 64 GB RAM at about 4 tok/s decode. That post states 178.2 GB of Maya-S plus 202.8 GB of Engram, which is not this Q6_K Engram size (https://x.com/ItsmeAjayKV/status/2112515177954598912). DeepSeek's DeepInfer (https://github.com/deepseek-ai/DeepInfer, Apache-2.0) is a datacenter serving stack for NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, not a home-rig runtime.","communityMeasurement":{"label":"community measurement, single run","sourceName":"lxe on X","sourceUrl":"https://x.com/lxe/status/2112321482244436305","published":"2026-10-20","note":"Community claim, not verified by us. One post on 20 Oct 2026 reports Unsloth UD-IQ1_M on 2× RTX 3090 with llama.cpp at 12.3 tokens per second decode and about 90 tokens per second tuned prefill. We did not run that test. The prefill figure is the post's, not a second decode measurement.","rows":[{"hardware":"2× RTX 3090, llama.cpp, UD-IQ1_M decode","tokensPerSec":12.3}]},"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":444.74,"kvGb":0,"overheadGb":0.8,"totalGb":445.54,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":541.88,"kvGb":0,"overheadGb":0.8,"totalGb":542.68,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":816.63,"kvGb":0,"overheadGb":0.8,"totalGb":817.43,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":1526.42,"kvGb":0,"overheadGb":0.8,"totalGb":1527.22,"estimated":true,"sourceUrl":null}}}},{"id":"muse-spark-open","name":"Meta Muse Spark Open","family":"Muse","paramsB":816,"activeParamsB":42,"paramsEstimated":false,"paramsSourceUrl":"https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B","layers":62,"kvHeads":16,"headDim":64,"fullAttentionKv":null,"maxContext":1048576,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B/raw/main/config.json","notes":"MoE, 256 experts, 8 active per token, 1.8B perception encoder plus audio encoder. The card states 816B total / 42B active; Hugging Face metadata lists 822B including the vision and audio encoders. Licence (Muse Model License): products must display \"Built with Muse Spark Open\". Model-as-a-Service offerings by licensees whose group revenue exceeds US$20M in any 12 months need a separate agreement with Meta. Internal use and powering your own non-MaaS product are exempt. Gated download (accept terms on Hugging Face). Model card: \"optimized to run on two Mac Studios with M5 Ultra using MLX\". NVFP4 is for Blackwell GPUs only and needs at least 700GB of VRAM per the card. Vision projector (mmproj) is not a fit row: Q8_0 2.18 GB and BF16 4.10 GB, so add +2.2–4.1 GB for image/audio input. HF blog: 2-bit GGUF (304 GB) ran at 18 tokens per second and oQ3e MLX at ~21 tokens per second generation on a Mac Studio M3 Ultra 512GB (vendor/HF-reported, single test, not ours). Ollama offers muse-spark-open:cloud only (no local tag, checked 24 Oct 2026, https://ollama.com/library/muse-spark-open/tags). Supported in mainline llama.cpp (ggml-org GGUFs), vLLM, SGLang, mlx-lm/oMLX. Speed is not estimated: our speed estimate reads every weight for each token and would understate a sparse MoE.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":470.07,"kvGb":0.92,"overheadGb":0.8,"totalGb":471.79,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":579.36,"kvGb":0.92,"overheadGb":0.8,"totalGb":581.08,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":873.12,"kvGb":0.92,"overheadGb":0.8,"totalGb":874.84,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":1632,"kvGb":0.92,"overheadGb":0.8,"totalGb":1633.72,"estimated":true,"sourceUrl":null}}}},{"id":"kolibri-1","name":"Kolibri-1","family":"Kolibri","paramsB":78.1,"activeParamsB":3.46,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/Aleph-Alpha/Kolibri-1","layers":50,"kvHeads":4,"headDim":128,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Aleph-Alpha/Kolibri-1/raw/main/config.json","notes":"Aleph Alpha GmbH's open MoE for German and English, Apache-2.0. The card lists 78,103,074,560 parameters (headline 78B; 78.1B here is that count to one decimal) and 3,457,573,120 active per token (3.46B). Fewer active parameters means less compute per token than a dense model of this total size. That is not a measured tokens-per-second figure: no independent measurement is stored, and speed is not estimated for MoE. 384 experts per layer, 6 active per token, plus a shared expert. llama.cpp b11709 (17 Oct 2026) describes 384 routed experts plus one shared and adds support: https://github.com/ggml-org/llama.cpp/releases/tag/b11709. 50 layers: 40 sliding-window (window 513) and 10 full attention. 48 query heads, 4 KV heads, head dim 128. Config max_position_embeddings is 262,144. The card also validates 1,048,576 tokens and recommends at most 262,144; we use 262,144. The conversation memory in the table is our FP16 estimate on those 10 full layers and the 513-token window. The card evaluates with an FP8 cache for the conversation memory, which our estimate does not use. bartowski GGUF sizes are decimal GB, shards summed, checked 24 Oct 2026 (https://huggingface.co/bartowski/Kolibri-1-GGUF): IQ2_XXS 22.77, Q2_K 29.92, IQ3_XXS 33.65, Q3_K_M 37.62, IQ4_XS 43.63, Q4_K_S 46.40, Q4_K_M 49.76, Q5_K_M 60.30, Q6_K 69.00, Q8_0 83.14, BF16 156.31. The table rounds those files to one decimal. IQ4_XS at 43.63 GB, with the KV and 0.8 GB overhead estimates, fits the 64 GB-class unified-memory machines we score. That fit is our estimate, not a test. Q4_K_M at 49.76 GB does not fit the 64 GB Macs on the same estimate: macOS lets the GPU use about three quarters of 64 GB by default, 48 GB. The 128 GB Macs hold it. The 64 GB Strix Halo row is scored at 48 GB dedicated graphics, enough for IQ4_XS and not for Q4_K_M. The card's own hardware line is about 78 GB of FP8 weights on data-center GPUs; that is a card claim, not a measurement here. The card's release date is 3 Oct 2026. The Hugging Face repo was created 2 Oct 2026. No official Ollama tag, checked 24 Oct 2026.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":49.76,"kvGb":0.21,"overheadGb":0.8,"totalGb":50.77,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":55.45,"kvGb":0.21,"overheadGb":0.8,"totalGb":56.46,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":83.57,"kvGb":0.21,"overheadGb":0.8,"totalGb":84.58,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":156.2,"kvGb":0.21,"overheadGb":0.8,"totalGb":157.21,"estimated":true,"sourceUrl":null}}}},{"id":"orcacyber-nano-1.0","name":"OrcaCyber Nano 1.0","family":"OrcaCyber","paramsB":27,"activeParamsB":null,"paramsEstimated":false,"paramsSourceUrl":"https://huggingface.co/orcarouter/OrcaCyber-Nano-1.0","layers":64,"kvHeads":4,"headDim":256,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":true,"architectureSourceUrl":"https://huggingface.co/Qwen/Qwen3.5-27B/raw/main/config.json","notes":"OrcaRouter's dense 27B vision-language model for defensive security work (reverse engineering, vulnerability analysis, secure code review, triage). Ships only as GGUF. The card says: \"Loads in stock llama.cpp — no custom kernels\". Text, image and video in, text out. Context 262,144. Hugging Face metadata says license: other, and the download is gated (accept the terms; auto-approval). No licence text is public. That is not Apache, MIT, or a Qwen licence. The card does not name the pretrained base. Do not read the Qwen3.5-shaped stack as a claim that this is a Qwen fine-tune. Architecture on the card: Qwen3_5ForConditionalGeneration, 64 layers, hidden size 5120, 48 linear-attention layers and 16 full-attention layers. KV heads (4) and head dim (256) are not on the card; the config is gated. Those two figures are assumed from the Qwen3.5-27B config, which has the same 64 layers, hidden size 5120 and full_attention_interval 4. KV dims assumed from Qwen3.5-27B architecture; OrcaCyber config is gated. Linear-attention layers are stored with a sliding window of 0 so our estimate gives them no conversation memory. The conversation-memory figure uses only the 16 full-attention layers and is an estimate. Mix-15.7G is 15.70 GB and Mix-13G is 12.99 GB (decimal GB, Hugging Face file listing, checked 24 Oct 2026, https://huggingface.co/orcarouter/OrcaCyber-Nano-1.0). About 4.65 and 3.85 bits per weight, estimated from 27B parameters. Vendor custom mixes, not Q4_K_M. The memory table does not include the vision projector or the drafter: +0.93 GB vision projector (mmproj F16) for image input; +2.06 GB optional DFlash2 drafter. The card says Mix-13G \"fits a 16 GB card with the MTP drafter\" and points Mix-15.7G at a \"24 GB+ card\". That is the vendor's guidance, not this site's fit score. An MTP head is built into both main files. Not in the Ollama library, checked 24 Oct 2026. Vendor-reported, unverified: AIME 2025 96.23% and MATH-500 98.60%. The card says the evaluation harness is being finalized.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":15.7,"kvGb":0.54,"overheadGb":0.8,"totalGb":17.04,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":19.17,"kvGb":0.54,"overheadGb":0.8,"totalGb":20.51,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":28.89,"kvGb":0.54,"overheadGb":0.8,"totalGb":30.23,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":54,"kvGb":0.54,"overheadGb":0.8,"totalGb":55.34,"estimated":true,"sourceUrl":null}}}},{"id":"qwen3.8-27b","name":"Qwen3.8-27B","family":"Qwen","paramsB":27.78,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/Qwen/Qwen3.8-27B","layers":64,"kvHeads":4,"headDim":256,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Qwen/Qwen3.8-27B/raw/main/config.json","notes":"Dense vision-language model, Apache-2.0 (the LICENSE file is Apache 2.0, and the card metadata says apache-2.0). The card's parameter headline is 27B. Hugging Face safetensors metadata lists 27,781,427,952 parameters, all BF16. 27.78B is that count to two decimals, so our parameter count is an estimate. 64 layers in text_config: 48 linear_attention (Gated DeltaNet) and 16 full_attention (Gated Attention). 24 query heads, 4 KV heads, head dim 256. Config max_position_embeddings is 262,144. The card says that window is extensible to 1,000,000 tokens with a RoPE change; we use 262,144. Linear-attention layers are stored with a sliding window of 0 so our estimate gives them no conversation memory. The conversation-memory figure uses only the 16 full-attention layers and is an FP16 estimate. The card's citation is August 2026. Hugging Face lists the repo as created on 5 August 2026. That createdAt is repo metadata, not a dated announcement in the citation. Vision projector and the MTP drafter are separate files, not rows in the memory table. On unsloth/Qwen3.8-27B-GGUF, checked 24 Oct 2026 (https://huggingface.co/unsloth/Qwen3.8-27B-GGUF): mmproj-F16 is 0.93 GB (927,607,488 bytes) and mmproj-BF16 is 0.93 GB (931,146,432 bytes). MTP/mtp-Qwen3.8-27B-Q4_0.gguf is 1.37 GB (1,369,590,656 bytes). Main GGUF sizes are decimal GB from that tree, single files: UD-IQ2_XXS 7.27, UD-IQ3_XXS 10.93, UD-IQ4_XS 14.25, UD-Q4_K_M 16.46, UD-Q4_K_XL 17.56, UD-Q5_K_XL 20.88, UD-Q6_K 21.98, Q8_0 29.05. A separate community speed table for one RTX 3090 is stored on this row. It is not the calculator estimate.","communityMeasurement":{"label":"community measurement, not ours","sourceName":"Qwen3.8-27B on RTX 3090","sourceUrl":"https://alexander-ollman.github.io/qwen3.8-on-rtx3090/results.html","published":"2026-09-18","note":"Canonical single-GPU figures on that results page: mean of five runs, one RTX 3090, the other replica stopped, the ~1K-token prompt column, 512-token greedy completions, reasoning disabled, 250 W power cap. The page says every rung is the same 4-bit checkpoint. llama.cpp out of the box 27.6 tokens per second; llama.cpp tuned (FA + q8 KV + MTP) 42.6; stock vLLM + MTP k=4 85.6; patched vLLM + MTP k=4 167.8; patched vLLM + DFlash2 k=7 212.1. The page dates the single-replica correction 31 August. results.html was last committed on 18 September 2026, which is the published date stored here. Not a localLLMrig measurement.","rows":[{"hardware":"RTX 3090, llama.cpp out of the box","tokensPerSec":27.6},{"hardware":"RTX 3090, llama.cpp tuned (FA + q8 KV + MTP)","tokensPerSec":42.6},{"hardware":"RTX 3090, stock vLLM + MTP k=4","tokensPerSec":85.6},{"hardware":"RTX 3090, patched vLLM + MTP k=4","tokensPerSec":167.8},{"hardware":"RTX 3090, patched vLLM + DFlash2 k=7","tokensPerSec":212.1}]},"extraCommunityMeasurements":[{"label":"community measurement, single post, not run by us","sourceName":"Josh Gold (killercatfish)","sourceUrl":"https://github.com/ggml-org/llama.cpp/issues/30892","published":"2026-10-24","note":"Self-reported by Josh Gold (github.com/killercatfish, 24 Oct 2026, quoted by name with written permission; SYCL prefill bug report in llama.cpp #30892 dated 20 Oct 2026). Headless Ubuntu 26.04 (kernel 7.0), Ryzen 5 7600X, 32 GB DDR5, B850 motherboard, two Intel Arc Pro B70 32 GB cards, one model per card. Card A runs vLLM (cyspiegel/vllm-xpu-b70) with Qwen3.8-27B Int4 (AutoRound) at about 30 tok/s decode; a 10K-token prompt takes about 7 s cold and 0.6 s warm with the prompt cache. Card B, in the B850 chipset PCIe 4.0 x4 slot, runs llama.cpp on Vulkan (Mesa 26.2.5) behind llama-swap with Qwen3.8-27B Q4_K_M at about 15 tok/s decode and 375–500 tok/s prompt reading (Vulkan: 0 faults). One card also faulted under Intel's compute stack; a slot swap later showed that card was defective, not the slot. On 26 Oct 2026 the faulty card failed 3 of 3 in the CPU x16 slot, and the healthy card ran 13 of 13 clean in the B850 chipset PCIe 4.0 x4 slot. The defective card is going back to ASRock for replacement. A single-card llama.cpp SYCL test build (b11755, GGML_SYCL_F16=OFF) measured about 270 tok/s prompt reading verified correct to 14K tokens; GGML_SYCL_F16=ON raised prompt speed to about 640 tok/s but produced !!!! NaN garbage (llama.cpp #30892; others point at PR #30604, which he had not retested). Splitting one model across both B70 cards over the B850 PCIe 4.0 x4 chipset slot dropped decode to 5.7 tok/s. Not a localLLMrig measurement.","rows":[{"hardware":"1× Intel Arc Pro B70 32GB, vLLM (cyspiegel/vllm-xpu-b70), Int4 AutoRound decode [self-reported]","tokensPerSec":30},{"hardware":"1× Intel Arc Pro B70 32GB, llama.cpp Vulkan (Mesa 26.2.5), Q4_K_M decode [self-reported]","tokensPerSec":15},{"hardware":"2× Intel Arc Pro B70 32GB split across B850 PCIe 4.0 x4 chipset slot, decode [self-reported]","tokensPerSec":5.7}]}],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":16.46,"kvGb":0.54,"overheadGb":0.8,"totalGb":17.8,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":19.72,"kvGb":0.54,"overheadGb":0.8,"totalGb":21.06,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":29.72,"kvGb":0.54,"overheadGb":0.8,"totalGb":31.06,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":55.56,"kvGb":0.54,"overheadGb":0.8,"totalGb":56.9,"estimated":true,"sourceUrl":null}}}},{"id":"qwen3.8-flash-next","name":"Qwen3.8-Flash-Next","family":"Qwen","paramsB":180,"activeParamsB":6,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/Qwen/Qwen3.8-Flash-Next","layers":48,"kvHeads":2,"headDim":256,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Qwen/Qwen3.8-Flash-Next/raw/main/config.json","notes":"Mixture-of-experts vision-language model. Hugging Face metadata records license: other and license_name: qwen-community-1.0. The LICENSE file is titled Qwen Community License 1.0. It requires the copyright notice. A commercial product or service with more than 100 million monthly active users, or more than US$20 million in monthly revenue, must display the model name. A Model as a Service or AI Work Assistant business needs a separate licence from Qwen before commercial use. Internal use that does not make the model or its outputs available to a third party is exempt from that separate licence. The card says 125B with 6B activated, plus 51B n-gram embedding and 4B MTP. Those round billions are why our parameter count is an estimate. Hugging Face safetensors metadata lists 179,999,981,459 parameters (BF16 179,999,981,424 and I64 35). Our memory estimate uses that total, rounded to 180B. 6B active is the card's activated figure for the 125B portion, not a second measurement. 512 experts in the config, 10 routed per token. The card adds 1 shared expert. 48 layers: 36 linear_attention and 12 full_attention. Those 12 are Qwen Sparse Attention. The card's budget is 512 blocks or 2,048 tokens, and text_config.indexer_budget is 2048. That cache is outside our estimate, so the Conversation column does not show a computed cache. 2 KV heads, head dim 256, max_position_embeddings 262,144. The card says the window is extensible to 1,000,000 tokens with a RoPE change; we use 262,144. The card's citation is August 2026. Hugging Face lists the repo as created on 24 August 2026. Speed is not estimated: our speed estimate reads every weight for each token and would understate a sparse MoE, so we show no speed figure. Unsloth GGUF sizes, shards summed, decimal GB, checked 24 Oct 2026 (https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF): UD-IQ4_XS 93.68, UD-Q4_K_XL 111.33, Q8_0 188.23. MTP/mtp-Qwen3.8-Flash-Next-Q8_0.gguf is 4.14 GB (4,137,429,120 bytes) and is not included in those three totals. mmproj-F16 is 0.90 GB (904,004,000 bytes), also separate. Strata's README says the graphics card keeps the experts that are used most often and RAM holds all of them (https://github.com/Niko1221/Strata). Releases: https://github.com/Niko1221/Strata/releases. Mainline llama.cpp can keep the most-used MoE experts in a GPU cache while all experts stay in system RAM (--cpu-moe or --n-cpu-moe plus --moe-cache-mib, merged 7 Oct 2026, https://github.com/ggml-org/llama.cpp/pull/29887). Multi-GPU support merged 8 Oct 2026 (https://github.com/ggml-org/llama.cpp/pull/30112). https://github.com/ggml-org/llama.cpp/pull/30637 replaced the MoE LRU cache with a 50/50 SLRU cache and merged on 19 Oct 2026 (release b11738 and later). The author says that is about 1–5% better than LRU. That comparison is self-reported. The cache is still not the same as the GGUF fitting in 24 GB of VRAM. https://github.com/ggml-org/llama.cpp/pull/27861 was still an open draft and https://github.com/ggml-org/llama.cpp/pull/30594 was still open, checked 24 Oct 2026, and both stay proposed.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":111.33,"kvGb":0,"overheadGb":0.8,"totalGb":112.13,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":127.8,"kvGb":0,"overheadGb":0.8,"totalGb":128.6,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":192.6,"kvGb":0,"overheadGb":0.8,"totalGb":193.4,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":360,"kvGb":0,"overheadGb":0.8,"totalGb":360.8,"estimated":true,"sourceUrl":null}}}},{"id":"qwen3.6-35b-a3b","name":"Qwen3.6-35B-A3B","family":"Qwen","paramsB":35.95,"activeParamsB":3,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/Qwen/Qwen3.6-35B-A3B","layers":40,"kvHeads":2,"headDim":256,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Qwen/Qwen3.6-35B-A3B/raw/main/config.json","notes":"Mixture-of-experts vision-language model, Apache-2.0. The card says 35B in total and 3B activated. Hugging Face safetensors metadata lists 35,951,822,704 parameters, all BF16. 35.95B is that count to two decimals, and 3B active is the card's round figure, so our parameter count is an estimate. Our memory estimate uses the total. 256 experts, 8 routed per token in the config. The card adds 1 shared expert. 40 layers: 30 linear_attention (Gated DeltaNet) and 10 full_attention (Gated Attention). 16 query heads, 2 KV heads, head dim 256. Linear-attention layers are stored with a sliding window of 0 so our estimate gives them no conversation memory. The conversation-memory figure uses only the 10 full-attention layers and is an FP16 estimate. Config max_position_embeddings is 262,144. The card says that window is extensible to 1,010,000 tokens; we use 262,144. The card's citation is April 2026. Hugging Face lists the repo as created on 15 April 2026. Speed is not estimated: our speed estimate reads every weight for each token and would understate a sparse MoE, so we show no speed figure. Unsloth GGUF sizes, decimal GB, checked 24 Oct 2026 (https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF): UD-Q3_K_XL 16.85, UD-IQ4_XS 17.73, MXFP4_MOE 21.71, UD-Q4_K_M 22.13, UD-Q4_K_XL 22.36, Q8_0 36.90. Q8_0 is 36,903,140,320 bytes. mmproj-F16 is 0.90 GB (899,283,680 bytes) and is not a row in the memory table.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":22.13,"kvGb":0.17,"overheadGb":0.8,"totalGb":23.1,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":25.52,"kvGb":0.17,"overheadGb":0.8,"totalGb":26.49,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":38.47,"kvGb":0.17,"overheadGb":0.8,"totalGb":39.43,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":71.9,"kvGb":0.17,"overheadGb":0.8,"totalGb":72.87,"estimated":true,"sourceUrl":null}}}},{"id":"muse-glimmer-30b","name":"Muse Glimmer 30B","family":"Muse","paramsB":29.6,"activeParamsB":null,"paramsEstimated":false,"paramsSourceUrl":"https://huggingface.co/meta-models/Muse-Glimmer-30B","layers":52,"kvHeads":2,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/meta-models/Muse-Glimmer-30B/raw/main/config.json","notes":"Meta's agentic model, distilled from Muse Spark, built for tool use on consumer hardware. Dense (no experts in the config). The card says about 29.6B parameters, including the vision encoder, of which about 1.8B is a ViT-G/14 perception encoder. Text and image in, text out. Context 131,072. Licence Apache 2.0. The download is not gated. The card's Model Release Date is August 2026. The Hugging Face repo was created on 9 Aug 2026. Knowledge cutoff 4 Jan 2026 (model card). text_config: 52 layers, hidden size 6656, 32 attention heads, 2 KV heads, head dim 128, vocab 202,048. 13 layers are full attention and 39 are sliding attention with a 2048-token window, so the full-attention share is 13/52. KV in this table is this site's FP16 estimate on those 13 full layers plus the 2048-token window. Official GGUFs need llama.cpp b10353 or newer (PR #26841, merged 10 Aug 2026, https://github.com/ggml-org/llama.cpp/pull/26841). Measured GGUF files, decimal GB, Hugging Face listing checked 24 Oct 2026 (https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF), vision projector and drafter excluded: KQuant-17GB-Q4_K_M 16.76 GB and KQuant-Dynamic-Q4_K_XL 19.65 GB. About 4.5 and 5.3 bits per weight, estimated from those file sizes and 29.6B parameters. BF16 safetensors are 59.55 GB (2 shards) on the base repo; that row is not a GGUF. The memory table does not include the vision projector or the drafter: +1.40 GB mmproj for image input; +1.63 GB optional DFlash drafter. The GGUF card's rough totals are about 17 GB, about 19 GB with vision, and about 20 GB with vision and the drafter for the 17 GB quant, and about 20 / 22 / 23 GB for the dynamic quant. The base card says Q4_K_M targets 24 GB VRAM and Dynamic targets 32 GB VRAM, with degradation 1.0% and 0.2%. That is Meta-reported guidance, not this site's fit score. Ollama: muse-glimmer (about 18 GB) and muse-glimmer:30b-mlx (about 19 GB), checked 24 Oct 2026, https://ollama.com/library/muse-glimmer. Tech report posted 19 Oct 2026: https://x.com/AIatMeta/status/2112265822274302066.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":16.76,"kvGb":0.19,"overheadGb":0.8,"totalGb":17.75,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":21.02,"kvGb":0.19,"overheadGb":0.8,"totalGb":22.01,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":31.67,"kvGb":0.19,"overheadGb":0.8,"totalGb":32.66,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":59.2,"kvGb":0.19,"overheadGb":0.8,"totalGb":60.19,"estimated":true,"sourceUrl":null}}}},{"id":"ornith-1.5-122b","name":"Ornith-1.5-122B","family":"Ornith","paramsB":125.09,"activeParamsB":null,"paramsEstimated":false,"paramsSourceUrl":"https://huggingface.co/ornith-ai/Ornith-1.5-122B","layers":48,"kvHeads":2,"headDim":256,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/ornith-ai/Ornith-1.5-122B/raw/main/config.json","notes":"Ornith's open-weight mixture-of-experts vision-language model, MIT (card frontmatter license: mit; LICENSE at https://huggingface.co/ornith-ai/Ornith-1.5-122B/blob/main/LICENSE). The download is not gated. The card says \"122B MOE\". Hugging Face safetensors metadata lists 125,086,497,008 parameters, all BF16. 125.09B is that count to two decimals, so we use 125.09 as a counted figure, not an estimate. The card does not state how many parameters are active. Community posts that say about 10B active are unverified and are not stored. Config: Qwen3_5MoeForConditionalGeneration, model_type qwen3_5_moe. 48 layers, 12 full_attention and 36 linear_attention (full_attention_interval 4), so the full-attention share is 12/48. Hidden size 3072. 32 attention heads, 2 KV heads, head dim 256, vocab 248,320. 256 experts, 8 routed per token, moe intermediate size 1024, shared expert intermediate size 1024. Linear-attention layers are stored with a sliding window of 0 so our estimate gives them no conversation memory. The conversation-memory figure uses only the 12 full-attention layers and is an FP16 estimate. Config max_position_embeddings is 262,144. The card says YaRN with a scaling factor of 4 grows that window to roughly 1M tokens; we use 262,144. Text, image and video in, text out. The card's example reads a <think> trace from reasoning_content. Public launch 20 Oct 2026: https://x.com/ornith_/status/2112553477457268799. Hugging Face lists the repo as created on 16 Oct 2026. Speed is not estimated: our speed estimate reads every weight for each token and would understate a sparse MoE, so we show no speed figure. A separate community measurement is stored on this row. It is not a fit score and not a quant comparison. Official GGUF sizes, decimal GB, Hugging Face tree checked 24 Oct 2026, mmproj excluded (https://huggingface.co/ornith-ai/Ornith-1.5-122B-GGUF): Q4_K_M 75.84 (75,843,911,648 bytes), Q5_K_M 88.71, Q6_K 102.38, Q8_0 132.56. Unsloth GGUF, shards summed the same day (https://huggingface.co/unsloth/Ornith-1.5-122B-GGUF): UD-IQ1_M 40.36, UD-IQ2_XXS 41.97, UD-Q2_K_XL 44.53, UD-IQ3_XXS 49.52, UD-Q3_K_XL 59.76, UD-Q4_K_XL 79.71, UD-Q5_K_XL 94.61, UD-Q6_K_XL 114.22. The bits column is the quant name's nominal width, not a measured bits-per-weight. Official BF16 GGUF is 249.36 GB (249,362,980,832 bytes) and is not a memory-table row. mmproj-Ornith-1.5-122B-BF16.gguf is 0.91 GB (912,263,200 bytes) and is not in the table. Community mixed quants, not rows in this table, checked 24 Oct 2026: AtomicChat AD-IQ2_XXS-IQ1_M 34.96 GB and AD-IQ3_XXS-IQ2_S 47.64 GB (https://huggingface.co/AtomicChat/Ornith-1.5-122B-GGUF). Ollama, checked 24 Oct 2026 (https://ollama.com/library/ornith-1.5/tags): ornith-1.5:122b is listed as 73GB–77GB, ornith-1.5:122b-nvfp4 as 73GB, and ornith-1.5:122b-q4_K_M as 77GB.","communityMeasurement":{"label":"community measurement, single post, not run by us","sourceName":"@filicroval","sourceUrl":"https://x.com/filicroval/status/2112644056069099593","published":"2026-10-20","note":"Community measurement, single post, not run by us. Official Q6_K on one DGX Spark (the post says 95 GiB; this catalog stores that official Q6_K file as 102.38 GB). llama.cpp with the checkpoint's built-in MTP drafter, 262K context: code 29.2 tok/s, math 30.8 tok/s, 8K prefill 1,092 tok/s. Not a quant comparison and not a fit score.","rows":[{"hardware":"one DGX Spark, official Q6_K, llama.cpp with built-in MTP drafter, 262K context, code","tokensPerSec":29.2},{"hardware":"one DGX Spark, official Q6_K, llama.cpp with built-in MTP drafter, 262K context, math","tokensPerSec":30.8}]},"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":75.84,"kvGb":0.2,"overheadGb":0.8,"totalGb":76.84,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":88.81,"kvGb":0.2,"overheadGb":0.8,"totalGb":89.82,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":133.85,"kvGb":0.2,"overheadGb":0.8,"totalGb":134.85,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":250.18,"kvGb":0.2,"overheadGb":0.8,"totalGb":251.18,"estimated":true,"sourceUrl":null}}}},{"id":"gpt-oss-20b","name":"gpt-oss-20b","family":"gpt-oss","paramsB":21,"activeParamsB":3.6,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/openai/gpt-oss-20b","layers":24,"kvHeads":8,"headDim":64,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/openai/gpt-oss-20b/raw/main/config.json","notes":"OpenAI's smaller open model for chat, code and tool use. It ships in 4-bit, so the official file is about 12 GB and fits a 16 GB graphics card. Even the full-quality file is only 13.8 GB, so our higher quality levels use that file's size. Lower-quality versions we checked keep that 4-bit core and save under 1 GB. Our speed estimate streams the 3.6B active parameters, not all 21B.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":12.11,"kvGb":0.2,"overheadGb":0.8,"totalGb":13.11,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":13.79,"kvGb":0.2,"overheadGb":0.8,"totalGb":14.79,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":13.79,"kvGb":0.2,"overheadGb":0.8,"totalGb":14.79,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":13.79,"kvGb":0.2,"overheadGb":0.8,"totalGb":14.79,"estimated":true,"sourceUrl":null}}}},{"id":"gpt-oss-120b","name":"gpt-oss-120b","family":"gpt-oss","paramsB":117,"activeParamsB":5.1,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/openai/gpt-oss-120b","layers":36,"kvHeads":8,"headDim":64,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json","notes":"OpenAI's larger open model for reasoning, code and tool use. It ships in 4-bit, so the official file is about 63 GB: no single graphics card on our list holds it, but a 96 GB or 128 GB Mac or a 128 GB mini PC does. Even the full-quality file is only 65.4 GB, so our higher quality levels use that file's size. Lower-quality versions we checked keep that 4-bit core and save under 1 GB.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":63.39,"kvGb":0.31,"overheadGb":0.8,"totalGb":64.5,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":65.37,"kvGb":0.31,"overheadGb":0.8,"totalGb":66.48,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":65.37,"kvGb":0.31,"overheadGb":0.8,"totalGb":66.48,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":65.37,"kvGb":0.31,"overheadGb":0.8,"totalGb":66.48,"estimated":true,"sourceUrl":null}}}},{"id":"gemma-3-27b","name":"Gemma 3 27B","family":"Gemma","paramsB":27.43,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/google/gemma-3-27b-it","layers":62,"kvHeads":16,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/unsloth/gemma-3-27b-it/raw/main/config.json","notes":"Google's mid-size open model for writing, translation and long documents, and it reads images too. At our default quality it needs about 18.5 GB, so a 24 GB graphics card holds it.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":16.55,"kvGb":1.11,"overheadGb":0.8,"totalGb":18.46,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":19.48,"kvGb":1.11,"overheadGb":0.8,"totalGb":21.38,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":29.35,"kvGb":1.11,"overheadGb":0.8,"totalGb":31.26,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":54.86,"kvGb":1.11,"overheadGb":0.8,"totalGb":56.77,"estimated":true,"sourceUrl":null}}}},{"id":"gemma-4-31b","name":"Gemma 4 31B","family":"Gemma","paramsB":31.27,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/google/gemma-4-31B-it","layers":60,"kvHeads":16,"headDim":256,"fullAttentionKv":{"kvHeads":4,"headDim":512},"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/google/gemma-4-31B-it/raw/main/config.json","notes":"Google's dense Gemma 4 model under the Apache 2.0 licence, for chat, writing and image input. At our default quality it needs about 20.6 GB, so a 24 GB graphics card holds it.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":18.32,"kvGb":1.51,"overheadGb":0.8,"totalGb":20.63,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":22.2,"kvGb":1.51,"overheadGb":0.8,"totalGb":24.51,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":33.46,"kvGb":1.51,"overheadGb":0.8,"totalGb":35.77,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":62.54,"kvGb":1.51,"overheadGb":0.8,"totalGb":64.85,"estimated":true,"sourceUrl":null}}}},{"id":"qwen3-32b","name":"Qwen3 32B","family":"Qwen","paramsB":32.76,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/Qwen/Qwen3-32B","layers":64,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":40960,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json","notes":"Alibaba's dense 32B model with a thinking mode you can switch on and off, built for code, reasoning and many languages. At our default quality it needs about 22.7 GB, so a 24 GB graphics card holds it, but only just.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":19.76,"kvGb":2.15,"overheadGb":0.8,"totalGb":22.71,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":23.26,"kvGb":2.15,"overheadGb":0.8,"totalGb":26.21,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":35.05,"kvGb":2.15,"overheadGb":0.8,"totalGb":38,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":65.52,"kvGb":2.15,"overheadGb":0.8,"totalGb":68.47,"estimated":true,"sourceUrl":null}}}},{"id":"qwen3-coder-30b-a3b","name":"Qwen3 Coder 30B A3B","family":"Qwen","paramsB":30.53,"activeParamsB":3.3,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct","layers":48,"kvHeads":4,"headDim":128,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct/raw/main/config.json","notes":"Alibaba's coding model for writing and fixing code, also with tools. Only about 3B of its 30B parameters are used per word, so it does far less work per word than a dense 30B model. At our default quality it needs about 20.2 GB, so a 24 GB graphics card holds it.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":18.56,"kvGb":0.81,"overheadGb":0.8,"totalGb":20.17,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":21.68,"kvGb":0.81,"overheadGb":0.8,"totalGb":23.28,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":32.67,"kvGb":0.81,"overheadGb":0.8,"totalGb":34.27,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":61.06,"kvGb":0.81,"overheadGb":0.8,"totalGb":62.67,"estimated":true,"sourceUrl":null}}}},{"id":"mistral-small-3.2-24b","name":"Mistral Small 3.2 24B","family":"Mistral","paramsB":24.01,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506","layers":40,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":131072,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/raw/main/config.json","notes":"Mistral's mid-size open model for chat, documents and image input, under Apache 2.0. At our default quality it needs about 16.5 GB, so it just misses a 16 GB graphics card: a slightly smaller download fits one, and a 24 GB card holds the default.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":14.33,"kvGb":1.34,"overheadGb":0.8,"totalGb":16.47,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":17.05,"kvGb":1.34,"overheadGb":0.8,"totalGb":19.19,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":25.69,"kvGb":1.34,"overheadGb":0.8,"totalGb":27.83,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":48.02,"kvGb":1.34,"overheadGb":0.8,"totalGb":50.16,"estimated":true,"sourceUrl":null}}}},{"id":"llama-4-scout","name":"Llama 4 Scout","family":"Llama","paramsB":108.64,"activeParamsB":17,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct","layers":48,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":10485760,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/unsloth/Llama-4-Scout-17B-16E-Instruct/raw/main/config.json","notes":"Meta's mixture-of-experts model with image input and a very long context window. At our default quality it needs about 68 GB, so no single graphics card on our list holds it, but a 96 GB or 128 GB Mac or a 128 GB mini PC does. Because it reads images, Meta's licence gives no rights to people or companies based in the EU. End users of a product built on it are not affected.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":65.36,"kvGb":1.61,"overheadGb":0.8,"totalGb":67.77,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":77.13,"kvGb":1.61,"overheadGb":0.8,"totalGb":79.55,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":116.24,"kvGb":1.61,"overheadGb":0.8,"totalGb":118.66,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":217.28,"kvGb":1.61,"overheadGb":0.8,"totalGb":219.69,"estimated":true,"sourceUrl":null}}}},{"id":"flint","name":"Recursive Flint","family":"Flint","paramsB":39.79,"activeParamsB":2.5,"paramsEstimated":false,"paramsSourceUrl":"https://huggingface.co/RecursiveSI/flint","layers":49,"kvHeads":4,"headDim":128,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/RecursiveSI/flint/raw/main/config.json","notes":"Recursive's coding model for agents that read code, fix bugs and write tests, under Apache 2.0. Only about 2.5B of its 40B parameters are used per word, so it does far less work per word than a dense 40B model. The 4-bit downloads are about 22 to 26 GB before any conversation memory.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":23.39,"kvGb":0.27,"overheadGb":0.8,"totalGb":24.46,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":28.25,"kvGb":0.27,"overheadGb":0.8,"totalGb":29.32,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":42.58,"kvGb":0.27,"overheadGb":0.8,"totalGb":43.65,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":79.58,"kvGb":0.27,"overheadGb":0.8,"totalGb":80.65,"estimated":true,"sourceUrl":null}}}},{"id":"minimax-m3.1-flash","name":"MiniMax M3.1 Flash","family":"MiniMax","paramsB":428,"activeParamsB":23,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash","layers":60,"kvHeads":4,"headDim":128,"fullAttentionKv":null,"maxContext":1048576,"architectureEstimated":true,"architectureSourceUrl":"https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash/raw/main/config.json","notes":"MiniMax's open-weights multimodal mixture of experts: about 428B total, 23B active, context up to 1,048,576 tokens. Open weights under the MiniMax Community Licence. Non-commercial use is free; commercial use needs a credit and a notice to MiniMax, or written permission above US$20 million yearly revenue. This is not an Apache-2.0 licence. On 26 Oct 2026 Unsloth published dynamic GGUFs (https://huggingface.co/unsloth/MiniMax-M3.1-Flash-GGUF, guide https://unsloth.ai/docs/models/minimax-m3.1-flash). UD-Q2_K_XL is 143.9 GB. Unsloth says that 2-bit file runs in about 150 GB of RAM or unified memory, with 91.5% top-1 token agreement, for a 16K context and one slot (Unsloth, with day-zero weights from MiniMax). A 192 GB, 256 GB, or 512 GB Mac Studio, a dual DGX Spark, or an EPYC server with 256 GB of DDR4 or DDR5 is the class that figure points at. UD-Q4_K_XL is 249.4 GB. Unsloth's planning figure for that 4-bit file is 256 GB or more of unified memory, which also covers an eight-node DGX Spark. Official NVFP4 and MXFP8 weights are 252.42 GB and are not a GGUF. Text-only MLX builds remain listed, the smallest of those at 186.49 GB. Our conversation-memory estimate keeps a full cache for every layer; the sparse indexer cache is not modelled.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":249.4,"kvGb":1.01,"overheadGb":0.8,"totalGb":251.21,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":303.88,"kvGb":1.01,"overheadGb":0.8,"totalGb":305.69,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":457.96,"kvGb":1.01,"overheadGb":0.8,"totalGb":459.77,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":856,"kvGb":1.01,"overheadGb":0.8,"totalGb":857.81,"estimated":true,"sourceUrl":null}}}},{"id":"nemotron-3-5-bolt-e4b","name":"NVIDIA Nemotron 3.5 Bolt E4B","family":"Nemotron","paramsB":8,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB","layers":42,"kvHeads":2,"headDim":256,"fullAttentionKv":{"kvHeads":2,"headDim":512},"maxContext":65536,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16/raw/main/config.json","notes":"NVIDIA's small Nemotron 3.5: 8B parameters. The card calls 4B of that effective, from per-layer embeddings, not a mixture of experts. Open weights under OpenMDW-1.1; the Gemma 4 base is Apache 2.0. The card says text in and text out, and context up to 64K (65,536). The GGUF header and Ollama list 128K. The config, Ollama and the unsloth GGUFs also include an image encoder; the mmproj files (0.56–0.99 GB) are not in these totals. Ollama's :e4b download was 7.91 GB on 23 Oct 2026 (the 6.18 GB Q4_K_M plus a projector and a draft). Without :e4b, Ollama pulls the 74B model. 18 of 42 layers share key and value state, which this estimate does not model, so the conversation figure is conservative.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":6.18,"kvGb":0.27,"overheadGb":0.8,"totalGb":7.25,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":7.31,"kvGb":0.27,"overheadGb":0.8,"totalGb":8.38,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":8.56,"kvGb":0.27,"overheadGb":0.8,"totalGb":9.63,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":16,"kvGb":0.27,"overheadGb":0.8,"totalGb":17.07,"estimated":true,"sourceUrl":null}}}},{"id":"gemma-4-26b-a4b","name":"Gemma 4 26B A4B","family":"Gemma","paramsB":25.2,"activeParamsB":3.8,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/google/gemma-4-26B-A4B-it","layers":30,"kvHeads":8,"headDim":256,"fullAttentionKv":{"kvHeads":2,"headDim":512},"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/google/gemma-4-26B-A4B-it/raw/main/config.json","notes":"Google's Gemma 4 mixture of experts. The model card's table lists 25.2B total and 3.8B active, with image input, under Apache 2.0. The chat's size class rounds this to 26B with 4B active. 5 of 30 layers attend to the whole context; the other 25 use a 1,024-token window. The vision projector (about 1.19 GB in BF16) is not included in these file sizes.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":16.95,"kvGb":0.38,"overheadGb":0.8,"totalGb":18.13,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":17.89,"kvGb":0.38,"overheadGb":0.8,"totalGb":19.07,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":26.96,"kvGb":0.38,"overheadGb":0.8,"totalGb":28.14,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":50.4,"kvGb":0.38,"overheadGb":0.8,"totalGb":51.58,"estimated":true,"sourceUrl":null}}}},{"id":"nemotron-3.5-lightning","name":"NVIDIA Nemotron 3.5 Lightning","family":"Nemotron","paramsB":30,"activeParamsB":3,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16","layers":6,"kvHeads":2,"headDim":128,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":true,"architectureSourceUrl":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/raw/main/config.json","notes":"NVIDIA's hybrid Mamba and Transformer mixture of experts: 30B total, 3B active, open weights under OpenMDW-1.1. The card says up to 1M tokens; the config and the single-H100 recipe use 262,144, which is the context stored here. The stack is 52 blocks (23 Mamba, 23 mixture-of-experts, 6 attention). Our conversation-memory estimate uses only those 6 attention blocks and does not model the Mamba state.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":18.9,"kvGb":0.05,"overheadGb":0.8,"totalGb":19.75,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":21.3,"kvGb":0.05,"overheadGb":0.8,"totalGb":22.15,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":32.1,"kvGb":0.05,"overheadGb":0.8,"totalGb":32.95,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":60,"kvGb":0.05,"overheadGb":0.8,"totalGb":60.85,"estimated":true,"sourceUrl":null}}}},{"id":"laguna-s-2.1","name":"Laguna S 2.1","family":"Laguna","paramsB":118,"activeParamsB":8,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/poolside/Laguna-S-2.1","layers":48,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":1048576,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/poolside/Laguna-S-2.1/raw/main/config.json","notes":"Poolside's coding mixture of experts: 118B total, about 8B active, 256 routed experts with 10 per token plus one shared expert, context 1,048,576. Open weights under OpenMDW-1.1. 12 of 48 layers attend to the whole context; the other 36 use a 512-token window. The DFlash draft (2.23 GB) is not in these file sizes.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":96.03,"kvGb":0.48,"overheadGb":0.8,"totalGb":97.31,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":113.64,"kvGb":0.48,"overheadGb":0.8,"totalGb":114.91,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":126.26,"kvGb":0.48,"overheadGb":0.8,"totalGb":127.54,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":236,"kvGb":0.48,"overheadGb":0.8,"totalGb":237.28,"estimated":true,"sourceUrl":null}}}},{"id":"nanbeige-4-3-33b","name":"Nanbeige4.3-33B","family":"Nanbeige","paramsB":33.12,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/Nanbeige/Nanbeige4.3-33B","layers":52,"kvHeads":8,"headDim":128,"fullAttentionKv":null,"maxContext":524288,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/Nanbeige/Nanbeige4.3-33B/raw/main/config.json","notes":"Dense coder and reasoning model, 33.12B parameters, Apache-2.0 on the model-card metadata (the repo had no LICENSE file on 26 Oct 2026). Config: 52 layers, 8 KV heads, head dim 128, context 524,288. The parameter formula is about 19.9 GB at Q4, 35.4 GB at Q8 and 66.2 GB at FP16. Bartowski's GGUF listing on 26 Oct 2026 measured Q4_K_M at 20.79 GB, Q8_0 at 35.20 GB and the two BF16 shards at 66.24 GB together. IQ2_M on the same listing measured 12.19 GB on 27 Oct 2026 (12,187,167,872 bytes). At 8,192 tokens our estimate adds about 1.74 GB of conversation memory and 0.80 GB of working space, so the Q4_K_M total is about 23.3 GB. That fits a 24 GB GPU (an RTX 3090 or RTX 4090 leaves about 23.4 GB usable). A 32 GB Mac leaves about 21.3 GB for the GPU, so the same Q4 total is over that default share; IQ4_XS at 18.64 GB is the file that lands inside it. The IQ2_M total is about 14.7 GB, which fits a 16 GB card and misses a 12 GB card. Q8_0 at 35.20 GB fits 48 GB of VRAM or 64 GB of unified memory. FP16 at 66.24 GB fits 96 GB or 128 GB of unified memory, or two 48 GB GPUs. The Terminal-Bench 80.9 and DeepSWE 65.5 figures are the author's, and are not yet independently verified.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":20.79,"kvGb":1.74,"overheadGb":0.8,"totalGb":23.33,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":23.52,"kvGb":1.74,"overheadGb":0.8,"totalGb":26.06,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":35.44,"kvGb":1.74,"overheadGb":0.8,"totalGb":37.98,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":66.24,"kvGb":1.74,"overheadGb":0.8,"totalGb":68.78,"estimated":true,"sourceUrl":null}}}},{"id":"kimi-k3.1-pro","name":"Kimi K3.1 Pro","family":"Kimi","paramsB":2800,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://openrouter.ai/moonshotai/kimi-k3.1-pro","layers":0,"kvHeads":0,"headDim":0,"fullAttentionKv":null,"maxContext":1048576,"architectureEstimated":true,"architectureSourceUrl":null,"notes":"Moonshot AI's sparse mixture of experts, stated as 2.8T total parameters, the same total class as Kimi K3. This row has no counted weight file and no config: on 26 Oct 2026 the OpenRouter models API listed no Hugging Face repo. A flat 4 bits per weight is 0.5 bytes per parameter, so 2.8T weights are 1,400 GB (1.4 TB). This site's Q4 estimate uses 0.60 bytes per parameter, 1,680.0 GB of weights plus 0.80 GB of working space (1,680.8 GB). Layer count, KV heads and the active-parameter count are not published, so conversation memory is not counted. Moonshot says the open-weight GGUF and FP8 checkpoints arrive on 4 November 2026. Until then the model is API-live as moonshotai/kimi-k3.1-pro. The default Moonshot endpoint is MXFP4 at $2 per million input tokens and $10 per million output tokens, and a cached input read is $0.20 per million. A file this size is a multi-node cluster. You can test it with a pass, or compare it with open models, at https://localllmrig.com/.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":1680,"kvGb":0,"overheadGb":0.8,"totalGb":1680.8,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":1988,"kvGb":0,"overheadGb":0.8,"totalGb":1988.8,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":2996,"kvGb":0,"overheadGb":0.8,"totalGb":2996.8,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":5600,"kvGb":0,"overheadGb":0.8,"totalGb":5600.8,"estimated":true,"sourceUrl":null}}}},{"id":"spark-x2.6-4b","name":"Spark-X2.6-4B","family":"Spark","paramsB":4.11,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/XHToken/Spark-X2.6-4B/raw/main/model.safetensors.index.json","layers":36,"kvHeads":4,"headDim":256,"fullAttentionKv":null,"maxContext":1048576,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/XHToken/Spark-X2.6-4B/raw/main/config.json","notes":"Dense text model, 4.11B parameters. Hugging Face safetensors metadata lists 4,112,079,360 parameters, all BF16. 4.11B is that count to two decimals, so our parameter count is an estimate. Architecture Spark2_5ForCausalLM (model type spark2_5): 36 layers, 27 sliding-window layers with a 512-token window and 9 full-attention layers, 4 KV heads, head dim 256, max context 1,048,576. At 8,192 tokens our estimate adds 0.36 GB of conversation memory across that 1,048,576-token max context, plus 0.80 GB of working space. Bartowski's Q4_K_M measured 2.62 GB on 27 Oct 2026 (2,621,104,768 bytes), so the total at 8,192 tokens is 3.78 GB. XHToken's official Q4_K_M measured 3.54 GB on the same day (3,543,441,312 bytes), so that file's total at 8,192 tokens is 4.70 GB. The bartowski file is the one this page uses for the standard total. Apache-2.0. Other runtimes (not llama.cpp). Spark2_5ForCausalLM (spark2_5) runs out of the box via GGUF in llama.cpp, Ollama and LM Studio. Stock vLLM and Transformers releases need custom model registration.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":2.62,"kvGb":0.36,"overheadGb":0.8,"totalGb":3.78,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":2.92,"kvGb":0.36,"overheadGb":0.8,"totalGb":4.08,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":4.4,"kvGb":0.36,"overheadGb":0.8,"totalGb":5.56,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":8.22,"kvGb":0.36,"overheadGb":0.8,"totalGb":9.38,"estimated":true,"sourceUrl":null}}}},{"id":"lfm3-1.7b","name":"LFM3-1.7B","family":"LFM","paramsB":1.73,"activeParamsB":null,"paramsEstimated":true,"paramsSourceUrl":"https://huggingface.co/LiquidAI/LFM3-1.7B","layers":24,"kvHeads":4,"headDim":128,"fullAttentionKv":null,"maxContext":262144,"architectureEstimated":false,"architectureSourceUrl":"https://huggingface.co/LiquidAI/LFM3-1.7B/raw/main/config.json","notes":"Dense text model for local agents, 1.73B parameters. Hugging Face safetensors metadata lists 1,728,801,536 parameters, all BF16. 1.73B is that count to two decimals, so our parameter count is an estimate. The model card also states 1.73B. Architecture Lfm3ForCausalLM (model type lfm3): 24 layers, 13 gated short convolutions, 5 sliding-window attention layers with a 256-token window and 6 global attention layers, 4 KV heads, head dim 128, max context 262,144 (the card calls this 256K). Our estimate has only two attention buckets, so the 18 layers that are not global attention, including the 13 convolutions, are counted with that 256-token window. Conversation memory is therefore a little higher than the 6 global and 5 sliding layers alone. At 8,192 tokens that estimate is 0.11 GB of conversation memory plus 0.80 GB of working space. LiquidAI's Q4_K_M measured 1.1 GB on 28 Oct 2026 (1,099,627,808 bytes), so the total at 8,192 tokens is 2.01 GB. Q8_0 measured 1.85 GB (1,846,238,496 bytes) and BF16 measured 3.47 GB (3,466,626,336 bytes). F16 is the same 3,466,626,336 bytes. QAD Q4_0 measured 1.05 GB (1,049,615,680 bytes). Official MLX 4-bit weights measured 0.97 GB (973,083,386 bytes). Official GGUF, MLX and ONNX builds are published. ONNX is not in the fit table. LFM Open License v1.0 allows commercial use below $10 million annual revenue. The card says that at 4-bit precision the model takes about 1 GB and fits a phone or laptop. That is Liquid AI's statement. Our total adds conversation memory and working space to the measured file. The GGUF card says the files are for local inference on CPUs and GPUs, and that llama.cpp needs an LFM3-compatible build.","communityMeasurement":null,"extraCommunityMeasurements":[],"memory":{"contextTokens":8192,"quants":{"Q4":{"weightsGb":1.1,"kvGb":0.11,"overheadGb":0.8,"totalGb":2.01,"estimated":true,"sourceUrl":null},"Q5":{"weightsGb":1.23,"kvGb":0.11,"overheadGb":0.8,"totalGb":2.14,"estimated":true,"sourceUrl":null},"Q8":{"weightsGb":1.85,"kvGb":0.11,"overheadGb":0.8,"totalGb":2.76,"estimated":true,"sourceUrl":null},"FP16":{"weightsGb":3.46,"kvGb":0.11,"overheadGb":0.8,"totalGb":4.37,"estimated":true,"sourceUrl":null}}}}]}