Skip to content

17 Oct 2026

Free LLM Showdown: 6 small open models, 10 real prompts

Real outputs, one run each, no cherry-picking. Run on a CPU-only machine.

PromptGuess which model

Coding

Write a Python function `dedupe_emails(emails: list[str]) -> list[str]` that removes duplicate email addresses case-insensitively, trims whitespace, skips empty strings, and keeps the first occurrence order. Include a short docstring and 3 example asserts.

Model A

```python
def dedupe_emails(emails: list[str]) -> list[str]:
    """
    Removes duplicate email addresses case-insensitively, trims whitespace, 
    skips empty strings, and keeps the first occurrence order.

    Args:
        emails (list[str]): A list of email addresses.

    Returns:
        list[str]: A list of unique email addresses.
    """
    # Create a set to store unique email addresses
    unique_emails = set()
    # Create a list to store the result
    result = []
    
    # Iterate over each email in the input list
    for email in emails:
        # Trim whitespace and convert to lowercase
        trimmed_email = email.strip().lower()
        
        # Check if the email is not empty and not already in the set
        if trimmed_email and trimmed_email not in unique_emails:
            # Add the email to the set and the result list
            unique_emails.add(trimmed_email)
            result.append(email)
    
    return result

# Example asserts
assert dedupe_emails(["test@example.com", "test@example.com", "test2@example.com"]) == ["test@example.com", "test2@example.com"]
assert dedupe_emails(["test@example.com", "test@example.com", "test@example.com"]) == ["test@example.com"]
assert dedupe_emails(["", "test@example.com", "test@example.com"]) == ["test@example.com"]
```

This function works by iterating over each email in the input list, trimming whitespace, converting to lowercase, and checking if it's not empty and not already in the set. If it meets these conditions, it's added to the set and the result list. The function returns the result list, which contains the unique email addresses in the order they first appeared in the input list.

Generated tokens: 366

The check stays hidden until you reveal this card.

Model B

```python
def dedupe_emails(emails: list[str]) -> list[str]:
    """
    Removes duplicate email addresses in a case-insensitive manner, trims whitespace,
    skips empty strings, and preserves the order of first occurrence.
    
    Args:
        emails: A list of email strings (may contain duplicates, whitespace, or empty strings)
    
    Returns:
        A list of unique email addresses with duplicates removed, preserving order of first occurrence
    """
    seen = set()
    result = []
    
    for email in emails:
        # Skip empty strings
        if not email or email.strip() == "":
            continue
        
        # Trim whitespace and convert to lowercase for comparison
        cleaned_email = email.strip().lower()
        
        # Only add if we haven't seen this email (case-insensitively) before
        if cleaned_email not in seen:
            seen.add(cleaned_email)
            result.append(email.strip())  # preserve original (trimmed) form
    
    return result


# Example assertions
assert dedupe_emails(["alice@example.com", "Bob@domain.org", "alice@example.com"]) == ["alice@example.com", "Bob@domain.org"]
assert dedupe_emails(["  user@test.com  ", "", "User@Test.com", "  user@test.com  "]) == ["user@test.com"]
assert dedupe_emails([]) == []
```

Generated tokens: 281

The check stays hidden until you reveal this card.

Model C

```python
def dedupe_emails(emails: list[str]) -> list[str]:
  """Removes duplicate email addresses case-insensitively, trims whitespace,
  skips empty strings, and keeps the first occurrence order.

  Args:
    emails: A list of email addresses.

  Returns:
    A list of unique email addresses.
  """
  seen = set()
  result = []
  for email in emails:
    email = email.strip()
    if email and email.lower() not in seen:
      result.append(email)
      seen.add(email.lower())
  return result

assert dedupe_emails(["test@example.com", "Test@example.com", "test@example.com"]) == ["test@example.com"]
assert dedupe_emails(["test@example.com", "  test@example.com  ", "test@example.com"]) == ["test@example.com"]
assert dedupe_emails(["", "test@example.com", ""]) == ["test@example.com"]
```

Generated tokens: 242

The check stays hidden until you reveal this card.

Model D

```python
def dedupe_emails(emails: list[str]) -> list[str]:
    """
    Removes duplicate email addresses case-insensitively, trims whitespace,
    skips empty strings, and keeps the first occurrence order.

    Parameters:
    emails (list[str]): A list of email addresses.

    Returns:
    list[str]: A list of unique email addresses in the order of their first occurrence.
    """
    seen = set()
    unique_emails = []

    for email in emails:
        trimmed_email = email.strip().lower()
        if trimmed_email and trimmed_email not in seen:
            seen.add(trimmed_email)
            unique_emails.append(email)

    return unique_emails

# Example asserts
assert dedupe_emails(["test@example.com", "Test@example.com", "example@example.com"]) == ["test@example.com", "example@example.com"]
assert dedupe_emails(["  test@example.com  ", "test@example.com", "example@example.com  "]) == ["test@example.com", "example@example.com"]
assert dedupe_emails(["", "  ", "test@example.com", "test@example.com", "example@example.com"]) == ["test@example.com", "example@example.com"]
```

Generated tokens: 249

The check stays hidden until you reveal this card.

Model E

Cut off at 768 tokens

Loop


```python
def dedupe_emails(emails: list[str]) -> list[str]:
    """
    Removes duplicate email addresses case-insensitively, trims whitespace, skips empty strings,
    and keeps the first occurrence order.

    Args:
        emails (list[str]): A list of email addresses.

    Returns:
        list[str]: A list of unique email addresses in the original order, with duplicates removed.
    """
    seen = set()
    result = []
    for email in emails:
        email = email.strip().lower()
        if email and email not in seen:
            seen.add(email)
            result.append(email)
    return [email for email in result if email]

# Example asserts
assert dedupe_emails(["  a@b.com", "c@d.com", "a@b.com", "  e@f.com", "  ", "g@g.com"]) == ["a@b.com", "c@d.com", "e@f.com", "g@g.com"]
assert dedupe_emails(["  ", "  a@b.com", "c@d.com", "a@b.com", "  e@f.com", "  ", "g@g.com"]) == ["a@b.com", "c@d.com", "e@f.com", "g@g.com"]
assert dedupe_emails(["  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "  ", "

Generated tokens: 768

The check stays hidden until you reveal this card.

Model F

Here's a Python function that removes duplicate email addresses case-insensitively, trims whitespace, skips empty strings, and keeps the first occurrence order:

```python
def dedupe_emails(emails: list[str]) -> list[str]:
    """
    Removes duplicate email addresses case-insensitively, trims whitespace, skips empty strings,
    and keeps the first occurrence order.

    Args:
        emails (list[str]): A list of email addresses.

    Returns:
        list[str]: A list of unique email addresses.
    """
    unique_emails = []
    seen = set()

    for email in emails:
        email = email.strip().lower()
        if email and email not in seen:
            unique_emails.append(email)
            seen.add(email)

    return unique_emails
```

Explanation:
1. The function takes a list of email addresses (`emails`) as input and returns a list of unique email addresses.
2. It initializes an empty list called `unique_emails` to store the unique email addresses and an empty set called `seen` to keep track of the email addresses that have been encountered.
3. It iterates over each email address in the `emails` list.
4. For each email address, it trims any leading or trailing whitespace using the `strip()` method and converts it to lowercase using the `lower()` method. This ensures case-insensitivity.
5. It checks if the trimmed and lowercase email address is not empty and not already present in the `seen` set.
6. If the email address is unique (not empty and not in `seen`), it appends it to the `unique_emails` list and adds it to the `seen` set.
7. Finally, it returns the `unique_emails` list containing the unique email addresses.

Example asserts:
```python
assert dedupe_emails(['john@example.com', 'JOhn@example.com', 'jane@example.com']) == ['john@example.com', 'jane@example.com']
assert dedupe_emails(['  john@example.com  ', 'jane@example.com', '  jane@example.com  ', '']) == ['john@example.com', 'jane@example.com']
assert dedupe_emails(['john@example.com', 'jane@example.com', 'jane@example.com', '  jane@example.com  ', '']) == ['john@example.com', 'jane@example.com']
```

These asserts demonstrate the expected behavior of the `dedupe_emails` function:
1. The first assert checks that duplicate email addresses with different cases are removed, and the order of the first occurrence is preserved.
2. The second assert checks that leading and trailing whitespace is trimmed from the email addresses.
3. The third assert checks that empty strings are skipped and only unique email addresses are returned.

You can run these asserts to verify that the function works as expected.

Generated tokens: 575

The check stays hidden until you reveal this card.

Speed and checks

Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.

Measured on a CPU-only cloud VM (8 vCPU Intel Xeon, 15 GB RAM, shared with other workloads). Not representative of GPU speed.

A checked prompt counts as correct when the hand-written note does not start with “wrong” or “mostly”. Empty notes are not scored. Writing, translation and summaries are outside these 4 prompts.

Model	Maker	Params	License	GGUF file	Median gen tok/s (min–max). Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	Prompt tok/s. Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	Truncated	Checked prompts correct (of 4)
Llama 3.2 3B Instruct	Meta	3.21B	Llama 3.2 Community License (+ Acceptable Use Policy)	Llama-3.2-3B-Instruct-Q4_K_M.gguf	17.7 (16.3–19.9)Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	200.3Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	0	1 of 4
Qwen3 4B Instruct 2507	Alibaba Qwen	4B	Apache-2.0	Qwen3-4B-Instruct-2507-Q4_K_M.gguf	14.2 (9.2–15)Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	166.1Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	1	2 of 4
Gemma 3 4B IT	Google	4B	Gemma Terms of Use (+ Prohibited Use Policy)	google_gemma-3-4b-it-Q4_K_M.gguf	14.5 (13.5–15.8)Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	153.9Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	2	2 of 4
Phi-4-mini-instruct	Microsoft	3.8B	MIT	microsoft_Phi-4-mini-instruct-Q4_K_M.gguf	14.2 (13.1–15.9)Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	189.9Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	0	2 of 4
SmolLM3 3B	Hugging Face	3B	Apache-2.0	HuggingFaceTB_SmolLM3-3B-Q4_K_M.gguf	19 (16.3–22.1)Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	223.6Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	1	1 of 4
Granite 4.0 Micro	IBM	3B	Apache-2.0	ibm-granite_granite-4.0-micro-Q4_K_M.gguf	18.4 (16.1–19.8)Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	171.5Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.	0	2 of 4

What hardware runs these?

What speed on a GPU?

The figures above are the CPU run. GPU pages on this site, where a model is in the memory catalog, show formula estimates. Those estimates stay labelled as estimates.

Llama 3.2 3B Instruct is in the memory catalog. Llama 3.2 3B Instruct · VRAM calculator. Memory and tokens per second there are estimates.
Qwen3 4B Instruct 2507 is not in the memory catalog, so this site does not publish a GPU speed for it. Hardware · Models
Gemma 3 4B IT is not in the memory catalog, so this site does not publish a GPU speed for it. Hardware · Models
Phi-4-mini-instruct is not in the memory catalog, so this site does not publish a GPU speed for it. Hardware · Models
SmolLM3 3B is not in the memory catalog, so this site does not publish a GPU speed for it. Hardware · Models
Granite 4.0 Micro is not in the memory catalog, so this site does not publish a GPU speed for it. Hardware · Models

Try it yourself

Test your own prompt. Compare currently shows sample answers until live tests open.

What hardware runs these?

Method

Measured on a CPU-only cloud VM (8 vCPU Xeon, shared). Not representative of GPU speed.

Runtime. llama.cpp b11707 (official prebuilt CPU binary llama-b11707-bin-ubuntu-x64.tar.gz, sha256 88879871f765c3dec22260046e10a76e2ed38de5bdf4a7f68049245a8f4056c8), llama-server OpenAI-compatible /v1/chat/completions, model chat template via --jinja

Hardware. Cloud VM, CPU only: 8 vCPU Intel Xeon (AVX-512, AMX), 15 GB RAM, no GPU. The VM was shared with other workloads, so speeds vary per prompt.

Quant. All models Q4_K_M GGUF

Settings. temperature 0, seed 42, max 768 tokens, context 4096, 8 threads, no system prompt, thinking off (enable_thinking false).

{
  "temperature": 0,
  "seed": 42,
  "max_tokens": 768,
  "context": 4096,
  "threads": 8,
  "system_prompt": null,
  "chat_template_kwargs": {
    "enable_thinking": false
  }
}

Speed. gen_tok_per_s = llama-server timings.predicted_per_second (generated tokens / generation time, excluding prompt processing). Median over the 10 prompts. Measured on a CPU-only cloud VM (8 vCPU Intel Xeon, 15 GB RAM, shared with other workloads). Not representative of GPU speed.

Greedy decoding (temperature 0) on a multi-threaded CPU is not bit-for-bit reproducible. Llama 3.2 3B was run twice; 4 of 10 outputs were identical, the others differed in wording. We publish the second run; the first ran while the VM was under heavier load (slower).

Only the 4 prompts with a single right answer were checked, by hand, against the reference answers. Writing, translation and summaries are not scored: judge them yourself.

Each prompt was run once per model, with no retries and no cherry-picking. Answers that hit the 768-token cap are marked as truncated.

Run date 2026-10-17, timezone Europe/Brussels.

Server command

Llama 3.2 3B Instruct

sha256 6c1a2b41161032677be168d354123594c0e6e67d2b9227c84f296ad037c728ff

/workspace/llm/bin/llama-b11707/llama-server -m /workspace/llm/models/Llama-3.2-3B-Instruct-Q4_K_M.gguf --host 127.0.0.1 --port 8099 -c 4096 -t 8 --jinja --no-webui

Qwen3 4B Instruct 2507

sha256 3605803b982cb64aead44f6c1b2ae36e3acdb41d8e46c8a94c6533bc4c67e597

/workspace/llm/bin/llama-b11707/llama-server -m /workspace/llm/models/Qwen3-4B-Instruct-2507-Q4_K_M.gguf --host 127.0.0.1 --port 8099 -c 4096 -t 8 --jinja --no-webui

Gemma 3 4B IT

sha256 4996030242583a40aa151ff93f49ed787ac8c25e4120c3ae4588b2e2a7d1ae94

/workspace/llm/bin/llama-b11707/llama-server -m /workspace/llm/models/google_gemma-3-4b-it-Q4_K_M.gguf --host 127.0.0.1 --port 8099 -c 4096 -t 8 --jinja --no-webui

Phi-4-mini-instruct

sha256 01999f17c39cc3074afae5e9c539bc82d45f2dd7faa3917c66cbef76fce8c0c2

/workspace/llm/bin/llama-b11707/llama-server -m /workspace/llm/models/microsoft_Phi-4-mini-instruct-Q4_K_M.gguf --host 127.0.0.1 --port 8099 -c 4096 -t 8 --jinja --no-webui

SmolLM3 3B

sha256 519732558d5fa7420ab058e1b776dcfe73da78013c2fe59c7ca43c325ef89132

/workspace/llm/bin/llama-b11707/llama-server -m /workspace/llm/models/HuggingFaceTB_SmolLM3-3B-Q4_K_M.gguf --host 127.0.0.1 --port 8099 -c 4096 -t 8 --jinja --no-webui

Granite 4.0 Micro

sha256 bd9d7b4795b9dc44e3e81aeae93bb5d8e6b891b7e823be5bf9910ed3ac060baf

/workspace/llm/bin/llama-b11707/llama-server -m /workspace/llm/models/ibm-granite_granite-4.0-micro-Q4_K_M.gguf --host 127.0.0.1 --port 8099 -c 4096 -t 8 --jinja --no-webui

Request body

This body is reconstructed from the published settings. The saved run did not store the raw HTTP body. Context and threads are llama-server flags (-c and -t), not body fields. There was no system prompt. The user message is the prompt text.

{
  "temperature": 0,
  "seed": 42,
  "max_tokens": 768,
  "messages": [
    {
      "role": "user",
      "content": "<the prompt for that run>"
    }
  ],
  "chat_template_kwargs": {
    "enable_thinking": false
  }
}

Licenses

Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

Gemma is provided under the Gemma Terms of Use (ai.google.dev/gemma/terms).

Llama 3.2 3B Instruct (Meta): Llama 3.2 Community License (+ Acceptable Use Policy). Allowed; Meta claims no rights in outputs. Llama 3.2 license and AUP apply to use of the model.

Qwen3 4B Instruct 2507 (Alibaba Qwen): Apache-2.0. Allowed (Apache-2.0).

Gemma 3 4B IT (Google): Gemma Terms of Use (+ Prohibited Use Policy). Allowed; Google claims no rights in outputs. Gemma terms apply to use of the model.

Phi-4-mini-instruct (Microsoft): MIT. Allowed (MIT).

SmolLM3 3B (Hugging Face): Apache-2.0. Allowed (Apache-2.0). Run with thinking disabled (enable_thinking=false).

Granite 4.0 Micro (IBM): Apache-2.0. Allowed (Apache-2.0).

Download the full report (JSON/CSV): JSON · CSV

Get the full report + notify me when live tests open

The full report is the JSON and CSV on this page. This form only asks to email you when live tests open. Nothing is charged.

Download the full report (JSON/CSV): JSON · CSV

Free LLM Showdown: 6 Small Open Models Compared on 10 Real Prompts · localLLMrig