Complexity-aware routing backfires in long agent sessions if you ignore the prompt cache: switching models throws away the warm, model-scoped prefix, and the cold re-prefill can cost more than the frontier model would have for that one step. Make cache-rebuild cost a term in the routing decision.
Route a Step Down a Tier Only When the Saving Beats the Cache Rebuild
🔒 Pro tip · Advanced
Unlock this tip — and 140 more
This is one of 141 advanced, fact-checked tactics reserved for Pro. Get the full 163-tip library, a searchable archive, and a new tip every morning. Free for 7 days, then $9/mo.
Prefer to browse? The 22 Beginner tips are free forever.
More in Model Selection
🎚️Model Selection
60-80% on routed traffic
Stop Paying Frontier Prices for Boilerplate Work
Most of your token spend is on tasks a small model handles perfectly. Match the model to the job instead of defaulting to your most expensive option for everything.
🎚️Model Selection
Often cheaper than escalating a tier on reasoning-limited hard tasks (qualitative — depends on the price gap and how many extra reasoning tokens the cheaper model spends)
Buy More Reasoning on the Cheap Model Before You Upgrade the Tier
When a cheap model stumbles on a hard task, the reflex is to jump to the frontier tier. Often the cheaper move is to keep the small model and turn its reasoning effort up — its per-token rate is so low it can brute-reason through the problem and still cost far less.
🎚️Model Selection
40-70% when most queries are easy
Cascade: Try the Cheap Model First, Escalate Only When It Fails
Send every request to a small model first, programmatically check the answer, and only escalate to a frontier model when the cheap one falls short.