Model routing sends each request to the most appropriate model.
Simple requests go to a small, cheap, fast model.
Hard requests escalate to a larger, more capable one.
The routing decision can be rules-based or learned from usage data.
It often cuts cost substantially with little quality loss.
Model routing sends each request to the most appropriate model rather than one model to all.
Most traffic is easy and does not need the most capable option.
Classifying a ticket and answering a nuanced question are not the same job.
Routing the easy majority to a small model cuts cost substantially.
It usually cuts latency at the same time, since small models are faster.
The routing decision can be rules-based, which is where most systems should start.
Route by task type, by input length, or by which feature made the call.
A small classifier can decide, which costs a fast cheap call before the real one.
Confidence-based escalation is another shape.
Try the small model, check the answer, and escalate if it looks weak.
That doubles cost on escalated requests and still wins if escalation is rare.
The risk is a router that misjudges, sending hard questions to the weak model.
Measure quality per route rather than in aggregate, or the failures hide inside the average.
Keep an override so a request can be forced to the large model.
Revisit the routing when models change, since today's small model may beat last year's large one.
One question on this card. Every option explains itself once you pick, right or wrong.
22 concepts in this module, then a graded test with every answer explained.