Every request pays frontier prices. Most don't need to.
A routing layer in front of your pipeline sends each request to the cheapest model that can handle it — a fine-tuned small model in your infrastructure for the routine share, your existing frontier API for the genuinely hard cases. Nothing gets dumber; the bill gets smaller.
Model routing places a lightweight decision layer in front of your AI pipeline that sends each request to the cheapest model capable of handling it — a fine-tuned small model in your infrastructure for the routine share, your frontier API for the hard cases. When roughly three-quarters of traffic routes local, blended cost falls around 70% (our estimate from published list prices, August 2026) — and frontier capability stays in the loop.
The problem isn't which model you use. It's that you're only using one.
One-model pipelines charge you the same rate for “classify this email” as for “reason through this edge case” — difficulty varies per request, price doesn't. But ripping out the frontier model entirely is the wrong fix: some fraction of your traffic genuinely needs it.
Illustrative math, from public list prices, August 2026. Take contract-page review at roughly $0.30 per document on a frontier API. Route 75–80% of documents to a local 7B model running at a fraction of the per-token cost, escalate the rest, and the blended cost lands near $0.05 per document — about 70% off, with the hard cases still going to the big model. The arithmetic generalizes: blended savings ≈ routine share × small-model discount. Our cost model; The Crit replaces it with your numbers.
| One frontier model | Routed: SLM + escalation | |
|---|---|---|
| Price per request | Frontier rate on every call, easy or hard | Routine share at small-model rates; hard tail at frontier |
| Worked example: document review | $0.30 per document (public list prices, Aug 2026) | ~$0.05 per document blended — our estimate |
| The hard cases | Handled — and priced into every routine call too | Escalate to your frontier API by design — no quality cliff |
| Quality drift | Invisible until the complaints arrive | Escalation rate: a tracked, alertable metric |
For the underlying cost evidence on small-model inference, see the published migrations on our LLM-to-SLM Migration page (Forethought, 66–80% lower inference cost, vendor-reported, on AWS).
Savings scale with your routine share. If less than half your traffic is routine — or your volume is small — a router won't pay for its own build cost. We measure the share first, in The Crit, and give you the number either way.
The triage layer. Yours.
- Traffic analysis: difficulty distribution and your measured routine share
- Hitbox routing service deployed in your infrastructure, with per-request decision logs
- A fine-tuned 3–7B local model for the routine slice — weights are yours
- Escalation policy tuned to your accuracy bar, with a frozen eval set gating it
- Dashboard: cost per request, escalation rate, accuracy by route — the three numbers that matter
What is model routing (LLM routing)?
Model routing places a lightweight decision layer in front of an AI pipeline that sends each request to the cheapest model capable of handling it — typically a small fine-tuned model running locally for routine requests, with escalation to a frontier API for hard ones. It cuts cost without removing frontier capability from the system.
How much does a hybrid SLM/LLM router save?
Around 70% blended, when roughly three-quarters of traffic routes to the small model — the arithmetic is routine share times the small model's per-token discount, which runs 10–100× on published list prices (August 2026). The savings depend entirely on your routine share, which we measure from real traffic before the build.
Does routing to a smaller model reduce output quality?
No — that's the point of the escalation path. The small model only answers requests it handles at or above your accuracy bar, verified on a frozen eval set; anything below the confidence threshold escalates to the frontier model automatically. Escalation rate is monitored in production, so quality drift shows up in a metric, not in complaints.
Cut the bill. Keep the custody.
- Data stays in your VPC
- No new subprocessor
- Escalation under your rules
- Keys and logs are yours
Stop paying reasoning rates for routine work.
Two weeks of traffic analysis tells you your routine share, your escalation rate, and your blended cost — before any build starts. The numbers are yours either way.