Routing starts with workload intent
A routing policy should begin with what the request needs to accomplish. Interactive chat, code generation, document extraction, and background classification have different quality, latency, and cost requirements.
Group requests by workload before choosing models or providers. This creates policies that reflect user expectations instead of applying one global ranking to every task.
Use several signals together
No single signal produces a reliable routing decision. Combine the evidence that matters for each workload.
- Capability and context-window compatibility.
- Recent success rate and time to first token.
- Evaluated quality on representative tasks.
- Current price and buyer budget limits.
- Regional, privacy, and provider policy requirements.
Define hard rules before preferences
Separate eligibility from optimization. A route that lacks a required capability or violates a policy should never enter the candidate set, even if it is inexpensive. Once ineligible paths are removed, the router can score the remaining options for cost, speed, and quality.
Review routing as the market changes
Model performance, supply health, and pricing all change. Monitor outcomes continuously and re-evaluate policies with real traffic. A good router makes these changes operational rather than requiring application code to be rewritten for every new model or provider.
Use one DIT key to access supported models through a market of qualified AI providers.
Get your API key
