What is Model routing?

Model routing is a layer of software that decides, for each request, which AI model should answer it. Instead of sending everything to one model, a router looks at the task and picks from a menu of models.

Why it exists

Models differ a lot in price, speed and skill. A small model can answer "what's the capital of France?" in a fraction of a second for a fraction of a cent. A frontier model is overkill for that, but may be the only one that can untangle a hard bug. Using the biggest model for everything is like hiring a surgeon to put on a bandage.

How a router decides

Routers use a few common signals:

• The request itself. A classifier, often a small model, estimates how hard the task is or what kind it is: code, math, summary, chat.
• Rules you set. For example, "anything touching customer data goes to this provider" or "keep cost under this limit."
• The draft's quality. In a cascade, a cheap model answers first and a check decides whether the answer is good enough. If not, the request moves up to a stronger model.
• Availability. If one provider is slow or down, the router sends traffic to another. This is often called a fallback.

Some systems also use a second model to review the first one's answer before you see it, a pattern sometimes called critique.

What it means for you

When a product offers an "auto" mode or lets you choose an effort level instead of a model, it is probably routing. That can cut your bill and speed up easy tasks. The tradeoff is control: you may not know which model produced a given answer, and a router can guess wrong, sending a hard problem to a weak model.

To judge a router, test it on your own work using evals and compare cost and quality against always using one model. A benchmark score from the vendor is not a substitute for your own tasks.

Related ideas

Routing often sits inside an AI gateway, the service that gives you one interface to many providers. It is a close cousin of reasoning effort, which tunes how much one model thinks, while routing chooses between models. The software that runs the routing logic, tools and limits around a model is the agent harness. Inside a single model, mixture of experts does something similar at a smaller scale, sending each token to a few specialist sub-networks.

The idea is simple: match the effort to the job. Done well, you get strong answers where it counts and low costs everywhere else.