Skip to content
be-human.ai
← Back to the journal

Harness fundamentals

Model Routing Rules for Superintelligence: Match Each Task to a Model

Set model routing rules for AI agents: send simple tasks to cheaper models, hard ones to stronger models, and write a reason for each rule.

Be Human typographic cover with the title: Model Routing Rules for Superintelligence: Match Each Task to a Model
Conceptual illustration created with AI · Be Human

A model routing rule tells an agent which model to call for a specific task and records why. Simple, reversible work such as classifying a request or extracting a date goes to a smaller model, while work with money, customer promises or legal weight goes to a stronger model with tighter checks. Write each rule with its reason, an owner and a test, and the routing table stays reviewable as models change.

Key takeaways

  • Route each task by the cost of a wrong answer and whether the action can be undone, not by how the request sounds.
  • Every route needs a written reason, a named owner and a test, or it turns into a habit nobody can defend.
  • Start with two or three tiers. Extra tiers add review work faster than they add savings.
  • Send uncertain or conflicting cases to a person, and allow only one bounded fallback to a stronger tier.
  • Re-run the same evaluation set after every model, prompt or tool change before a route goes live.

What a routing rule actually decides

A routing rule is a decision the harness makes before each model call. The harness is the layer around the model that holds the tools, permissions, memory and the checks applied to outputs. Inside that layer, the routing rule matches a task to a capability tier, and the tier maps to a model the team can replace without rewriting the workflow around it.

Running every task on one model is simple, and it is a reasonable starting point for a pilot. The cost appears later in two forms. Simple work is paid for at the rate of hard work, and hard work is sometimes handled by a model too small for it, which shows up as errors a person must catch. A routing table makes both trade-offs visible instead of leaving them hidden in one configuration value.

The rule should name a tier, not a model name buried in code. Providers replace models on their own schedules, and a rule that says 'use the strongest tier for customer replies' survives a change that a rule hardcoded to one model name does not. The reason is the part most teams skip. A rule without a reason cannot be reviewed, because nobody can tell whether an exception is still needed.

Sort each task by what a mistake costs

Two questions sort most tasks. The first asks what a wrong answer costs, including the time a person spends fixing it and any harm to a customer, a supplier or the company's records. The second asks whether the action can be undone before anyone else sees it. A third question concerns the input: whether it arrives in a fixed structure or as open text.

A practical decision guide looks like this:

  • Reversible, low cost of error, structured input: use the smallest tier, with a validator that rejects any output outside the allowed values.
  • Reversible, low cost of error, open-ended input: use a middle tier, and have a person review a sample of outputs on a fixed schedule.
  • Irreversible, customer-facing or financial: use the strongest tier, with an approval step before the action runs.
  • Any task where the agent reports low confidence or receives conflicting inputs: send it to a person, whatever the tier.

The last line matters more than it looks. Escalation is a route too, and it should be the default when an input does not match what the rule was written for. The conditions that halt an agent loop and hand the case over are covered in Stop Conditions for AI Agent Loops, and they belong in the same table as the routes.

A step-by-step plan for your first routing table

Start from the list of tasks, not from a model catalog. A team that begins by comparing models usually finishes with a preference, not a rule. Describing each task in the same terms used in a task contract also makes the routes easier to test later.

  • List every task the agent performs, one line each, with the input it receives and the output it must produce.
  • Mark the cost of a wrong answer and whether the action is reversible for each line.
  • Assign a tier to each task and write one sentence explaining why that tier fits.
  • Name an owner for each route who is allowed to change it and who answers for its results.
  • Attach an evaluation set of real past cases to each route, with the correct outcome written beside every case.
  • Set a token and step budget per task so a failing route cannot loop unnoticed.
  • Launch high-risk routes on the strongest tier, and move a route to a smaller tier only after its test passes on the cases that matter.

Keep the first table short. Two to four routes are enough for most first engagements, and each one should fit on a single printed page with its reason beside it.

Illustrative scenario: routing requests at a repair business

The following is an illustrative scenario, not a client case. A repair business receives service requests by email and through a web form. The agent can read the calendar, draft replies and create draft work orders, but it cannot send messages or confirm visits without approval. Its routing table might read like this:

  • Tag the request type from a closed list of service categories. Smallest tier, output limited to the list, because a wrong tag is cheap to correct and the list is fixed.
  • Extract the address, access notes and equipment description into a fixed structure. Smallest tier with a schema check, because missing fields can be flagged instead of guessed.
  • Draft a reply that proposes available time windows. Middle tier, because the wording goes straight to a customer, and a person approves each reply during the first weeks.
  • Answer questions about prices or warranty coverage. No automatic route, always escalated, because the answer carries commercial weight.
  • Any message that mentions a gas smell, an injury or a second visit already promised. Escalated immediately to a person, with the original text attached.

The cheapest tier handles most of the volume, and the stronger tier appears only where a sentence will reach a customer or commit the business to something. The reason beside each row is what lets the owner change a route later without guessing why it was set that way.

Measure each route before you trust it

A route is a hypothesis until it passes a test. Build an evaluation set from real past cases for each route, with the correct outcome recorded next to each case. Run that set before launch, after every change to the model, the prompt or the tools, and on a schedule afterward. The [OpenAI evals guide](https://platform.openai.com/docs/guides/evals) describes this general pattern of writing tests, running them and comparing results across runs.

Measure cost per completed task, not cost per call. A route that looks cheap per call can become expensive if its output needs a retry or a person must correct it. The [FinOps Foundation's unit economics guidance](https://www.finops.org/framework/capabilities/unit-economics/) frames cost against the business output it produces, which is the right unit for this decision.

Be careful with automated graders. A model that judges another model's output can share the same blind spots, so check a sample of its verdicts by hand before relying on them. The LLM judges article explains where automated grading misleads and how to test the grader itself. Also watch for silent decline. A route can pass on the cases you saved and still degrade on the kinds of requests that arrive later.

Where routing fails and how to contain it

The most common failure is a cheap route that returns a confident, wrong answer in a format that looks correct. A validator catches malformed output, but it does not catch a wrong fact inside a well-formed field, which is why sampled human review remains part of the design for open-ended work.

The second failure is an unbounded fallback. When the smallest tier's output is rejected, the harness may retry once on a middle tier. If that also fails, the case goes to a person. Without a bound, a task can climb through every available model while cost grows and the task still does not finish. The model fallback routing guide covers how to keep fallbacks safe, and the token and step budgets article explains how to alert on a task that is spending too much before anyone reads the log.

The third failure is a routing table nobody reads. Tables that were correct at launch often drift from the work the agent actually does. A short monthly review of the log, looking at which tier handled which task and where people corrected the output, catches drift early.

Who owns the routing table after launch

Every route needs a named owner who can change it and a record of each change. Routing decisions are product decisions. They change cost, speed and the risk of a wrong answer, so they belong with the person who answers for the outcome, not only with whoever configured the first version.

Review the table on a fixed schedule and whenever a provider changes a model you depend on. Keep a log that shows which tier handled each task, so that a complaint can be traced to a specific route in minutes instead of reconstructed from memory.

Be Human designs and operates this layer for companies as part of a managed engagement. The routing table is written alongside the rest of the agent's harness, including its memory, tools and permissions, and it is reviewed through monitoring, recurring evaluations and permission reviews. When a route fails its test, the change ships as a controlled release with a written note, and the previous route stays available until the new one passes.

If you are not yet sure which of your tasks deserves a stronger model, the free Be Human AI diagnostic is a sensible place to find the first task worth handing to an agent.

Frequently asked questions

Can I send everything to the cheapest model and fix the errors later? Not safely for work that reaches customers, money or company records. Errors that a small model makes on an irreversible action are expensive to repair, and they often arrive without an obvious signal. Use the smallest tier only where a wrong answer is cheap to catch and the output can be checked against a fixed structure.

Do I need a different model for every task? No. Most teams need two or three tiers, and an extra tier earns its place only when a clear group of tasks requires it. Every added tier brings more tests and more owners, so add one only when a route has its own written reason and its own evaluation set.

How do I know a cheaper route is good enough? Run it against real past cases with known correct outcomes, and compare it with the current route on the same cases. Read the errors, not only the pass rate, because a few serious misses matter more than many small formatting differences. Keep a person reviewing a sample of its outputs after launch.

What should change when a provider releases a new model? Treat the release as a change to the harness, not as a simple configuration swap. Re-run every evaluation set, compare results route by route, and roll the change out to one route at a time with a way to return to the previous model. Record the decision and its reason in the same routing table.

Your next step

See where an AI agent fits in your company first.

Take the free Be Human AI diagnostic. A few questions about your work show which task to hand to an agent first, what it needs to know and where a person approves.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.

Less busywork. More room for you.

Bring us the work you wish you had time for.

Book a call to try a personalized AI agent free for 7 days ↗Or take the free AI diagnostic →