Documentation

Routing

Automatic model selection based on prompt complexity — send simple tasks to fast models, hard ones to powerful models.

Edit on GitHub

Routing automatically selects the best model for each prompt based on its complexity. Simple tasks go to faster, cheaper models. Complex tasks go to more capable ones. You control the thresholds.

How It Works

When routing is enabled, every prompt goes through a classifier:

  1. Score the prompt — scan for low-signal and high-signal keywords
  2. Factor in length — longer prompts get a complexity boost
  3. Pick a tier — Low, Medium, or High
  4. Select model — ask the provider for the best model at that tier

Enabling Routing

[agent.routing]
enabled = true
low_keywords = ["typo", "rename", "format"]
high_keywords = ["refactor", "architect", "security audit"]

Classification Logic

The classifier uses two keyword lists (built-in defaults + your custom keywords):

Low-signal keywords (simple tasks):

typo, rename, format, lint, simple, quick, docs

High-signal keywords (complex tasks):

architect, design, refactor, security, optimize, performance, debug, analyze

Scoring:

  1. Count matches against low keywords → low score
  2. Count matches against high keywords → high score
  3. Length boost:
    • More than 200 words → high score +2
    • More than 80 words → high score +1
  4. Result:
    • High score > low score → High tier
    • Low score > high score → Low tier
    • Tie → Medium tier

Model Selection

Once a tier is determined, Nyzhi asks the provider for an appropriate model:

select_model_for_prompt(prompt, provider, config)

The provider returns its best model for the tier. If no tier-specific model exists, it falls back to the provider’s default model.

Example

With this config:

[agent.routing]
enabled = true
low_keywords = ["fix typo", "rename variable"]
high_keywords = ["redesign the auth system", "optimize database queries"]
PromptTierWhy
”Fix the typo in README”LowMatches “fix typo"
"Add a new endpoint for /users”MediumNo strong signals either way
”Redesign the auth system to use JWT with refresh tokens and role-based access”HighMatches “redesign the auth system” + long prompt

When to Use Routing

Routing works best when:

  • Your provider has distinct model tiers (fast/cheap vs powerful/expensive)
  • You have a mix of simple and complex tasks
  • You want to reduce costs without sacrificing quality on hard problems

Tips

  • Keep keyword lists short — 5-10 keywords per list is plenty
  • Use specific phrases — “fix typo” is better than “fix”
  • Test your keywords — make sure simple prompts actually route to the low tier
  • Don’t over-engineer — if you mostly do complex work, routing may not save much

Next Steps

Back to overview