Use case

Best local LLM for internal documents

This page does not name a winner. The title is the job. You test the suggested models on your own prompts, then decide.

Routed with the catalog tag for documents and RAG.

Catalog label: documents and RAG. A catalog tag is a routing label, not a benchmark, and this site has no measured score for this job.

Example prompts

Using only this note, answer the question and say if the note is silent. Note: expenses over $500 need a manager. Question: can I book a $420 train fare myself?
List the steps in this policy, in order, and quote the sentence that sets the deadline. Policy: submit the draft on Friday. Review takes five working days.
A colleague asks where the holiday rule lives. From this paragraph, give the section name and one sentence they can forward. Paragraph: Section 4, Leave. Annual leave is requested two weeks ahead.

The compare page can drop one of these in with one click. They are starters, not a test set we scored.

Alphabetical. Order here is not a ranking. The compare suggester reorders them from your languages, context, privacy, and budget, and it says why.

Gemma 3 27BCatalog tags: rag, translation, creative. Routing labels, not benchmarks.
LFM2.5 2.6BCatalog tags: chat, reasoning, rag. Routing labels, not benchmarks.
Llama 3.1 8B InstructCatalog tags: chat, code, rag. Routing labels, not benchmarks.
Llama 3.3 70B InstructCatalog tags: chat, rag, reasoning, code. Routing labels, not benchmarks.
Mistral Small 3.2 24BCatalog tags: chat, translation, rag, code. Routing labels, not benchmarks.
Qwen2.5 72B InstructCatalog tags: translation, rag, code, reasoning. Routing labels, not benchmarks.
Qwen3 32BCatalog tags: code, reasoning, translation, rag. Routing labels, not benchmarks.

The compare also keeps one frontier reference, Claude Sonnet 5.5. It is not open weights, and it is not a hardware recommendation.