Home › Agent Harness › Which model
AI · Choosing
Which AI model for which job
There is no best model. There is a best model for each kind of work, and it changes every few months. So this page gives you a pick per job and a way to test, not a ranking. For data and agent work the rule is short: pay for the strongest model when you build, and use the smallest model that passes your own tests when an agent runs.
Written 20 November 2026. No benchmark scores are quoted, because leaderboards disagree and age within weeks. This page was written with Claude, a model from one of the vendors discussed, which is a reason to trust the method (test it yourself) and not the picks. The last section lists what to verify.
The short answer
A pick for each kind of work
Read down for the job you have. The examples are what we saw on offer in November 2026 and will date.
| Job | What it needs | Class of model | Examples on offer now | Verdict |
|---|---|---|---|---|
| Building: coding, long edits, writing specifications and pages | The best reasoning and the longest reliable work on a task. A person watches. | The strongest frontier model you can pay for | Claude's top tier (Opus or above), OpenAI's flagship, Gemini's top tier | Pay for the frontier here. The cost is small next to your time, and a weaker model costs you rework. |
| An agent in production: tool calls in a loop, structured answers | Reliable tool calling, valid JSON every time, follows instructions, steady cost per task. | A mid-tier model, tested on your scenarios | Claude's mid tier (Sonnet class), OpenAI's mid models, Gemini Flash class, or an open model such as gpt-oss-120b | Pick the smallest model that passes your own scenarios, then keep re-testing. Do not start at the top. |
| Bulk and cheap: classify, extract, route, tidy text | Speed and low cost; the task is narrow and checkable. | A small model | Claude's small tier (Haiku class), Gemini Flash-lite class, small open models | Use the smallest that meets your accuracy bar on a labelled sample. Check the errors, not the average. |
| Judging other models' answers | Better than what it grades, consistent, and ideally from a different family. | The strongest model you can afford, checked against human labels | A frontier model from a different vendor than the one being graded | Calibrate it on cases people have labelled. An uncalibrated judge is an opinion. |
| Reading long documents and specifications | Long context that is actually used, not just accepted. | A long-context model, tested on your documents | Gemini's long-context models, Claude's long-context options | Test recall on your own files: plant a fact in the middle and ask for it. |
| Regulated or private data (health, pharma, finance) | The data terms, the region, and who may see prompts. | A model reached through your own cloud account or hosted by you | Models offered by your data platform or cloud (for example Databricks Foundation Model APIs, Azure, Bedrock, Vertex) or open-weight models you host | Where the model runs decides first, which model second. Read the retention and training terms. |
| Personal and learning projects, no budget | Free, good enough, no card needed. | A free hosted open-weight model | gpt-oss-120b and Llama 3.3 70B on a free data-platform workspace (we ran both) | Start free. Move up only when a test shows the free model fails. |
| Search by meaning (embeddings) | A good embedding model and a stable one. | An embedding model, not a chat model | bge or gte class models on your platform | Changing it means re-embedding everything, so pick once and record why. |
How to choose, in five steps
- 1 · Collect real cases. Twenty to fifty tasks from your own work, with answers you know. Include the ones that go wrong: unknown inputs, a tool that fails, a user who pushes for something the agent must refuse.
- 2 · Write checks that need no model. Is the number equal to the tool's number? Is the JSON valid? Did it refuse? Use a judge model only for what code cannot check.
- 3 · Run two or three candidates. One small, one mid, one frontier. Same cases, same tools, same instructions.
- 4 · Compare on cost per successful task. A cheap model that fails and retries can cost more than a dear one that succeeds once.
- 5 · Pick the smallest that passes, write down why, and re-run on every new release. Models change under the same name, and a new version can be better or worse for your task.
What to measure
| Measure | The question |
|---|---|
| Tool-call reliability | Does it call the right tool with valid arguments, and stop when it should? |
| Valid structure | Does it return the JSON shape you asked for every time, or only usually? |
| Grounding | Does every number in its answer match what a tool returned? Count the exceptions. |
| Refusing correctly | Does it decline what it must (advice, filing, claims of cause) and still help with what it may? |
| Cost per successful task | Not price per token: total cost of runs that finished correctly, including retries. |
| Speed | Time to a finished task, and how it behaves under load. |
| Data terms | Region, retention, training use, and whether your cloud already approves the provider. |
| Availability | Is it offered in your region and on your platform today, and for how long? |
Where the model runs matters as much as which model
- On your data platform. Databricks, Azure, AWS and Google Cloud each offer models next to your data. Offered models differ by region and change; the free tier of a platform offers fewer.
- Through the vendor's own API. Often the newest features arrive here first. Check the data terms for your use.
- Hosted by you. Open-weight models give control and cost you operations.
- Through a gateway. One door for keys, limits and logs. See AI gateways: which one to use.
Check before you rely on this
- Names and tiers change. Vendors rename and replace models often. Read the current model list in your own workspace or console.
- Reports disagree. Public comparisons conflict on which model leads, and many are written by sellers. Treat them as a way to pick candidates, never to pick the winner.
- Platform availability. Which models your data platform offers, in your region, on your plan, must be checked there. On one free workspace we could use gpt-oss-120b and Llama 3.3 70B; paid workspaces list more.
- Sources used for the examples (secondary reports; confirm with the vendor): Best LLM for agentic coding, Best LLM for coding, Databricks supported models.
Where to go next
- When to use an agent SDK or framework.
- Agent harness architecture map.
- AI gateways: which one to use.
- Eval Harness: how to test, in depth.