Provider Routing — Smart Model Dispatching

Imagine you run a food delivery platform. A customer places an order, and instead of sending it to just one restaurant, you automatically pick the best kitchen from dozens of partners based on price, speed, and taste preferences. That’s exactly what Provider Routing does for Hermes Agent — when you’re using OpenRouter or Nous Portal, it automatically selects the most suitable AI provider (like Anthropic, Google, or AWS Bedrock) to handle your request.

Why Do You Need It?

Different AI providers charge different prices, run at different speeds, and deliver different quality. Some are cheap but slow. Some are fast but pricey. Others support special parameters that the rest ignore. Manually switching between them is a chore. Provider Routing acts as your smart dispatcher — it picks the best provider based on rules you set, so you never have to think about it again.

How to Configure It

Add a provider_routing block to your ~/.hermes/config.yaml file:

provider_routing:
  sort: "price"           # What to sort by
  only: []                # Only use these providers
  ignore: []              # Never use these providers
  order: []               # Manual priority list
  require_parameters: false  # Require support for all parameters
  data_collection: null   # Allow or deny data collection

Note: This feature only works when you’re using OpenRouter or Nous Portal. If you connect directly to Anthropic’s API, it won’t apply.

Key Options Explained

sort: How to Rank Providers

• "price": Cheapest first (great for heavy development)
• "throughput": Most tokens per second (great for long-form text)
• "latency": Fastest response (great for chat interactions)

provider_routing:
  sort: "latency"

only / ignore: Whitelist and Blacklist

• only: Use only the listed providers, ignore all others
• ignore: Never use the listed providers

provider_routing:
  only:
    - "anthropic"
    - "google"

order: Manual Priority

Providers listed first get used first. Unlisted ones serve as fallbacks:

provider_routing:
  order:
    - "anthropic"
    - "amazon-bedrock"

require_parameters: Parameter Integrity

Set to true to only select providers that support all your parameters (like temperature or tools). This prevents silent parameter drops.

data_collection: Privacy Control

Set to "deny" to stop providers from using your data for model training. Set to "allow" to permit it.

Practical Scenarios

Budget Mode: During heavy development, pick the cheapest option:

provider_routing:
  sort: "price"

Chat Mode: Need speed? Choose low latency:

provider_routing:
  sort: "latency"

Long-Form Mode: Writing a novel? Maximize throughput:

provider_routing:
  sort: "throughput"

Summary and Pro Tips

Provider Routing is your “AI dispatcher” — it automatically balances price, speed, and quality so you don’t have to. Pro tip: If you have a compliance requirement that forces you to use a specific provider, lock it in with only or order. For everyday use, just set sort: "price" and let the router do the heavy lifting. And if you’re a Nous Portal subscriber, remember that its built-in routing gives you a 10% discount — saving both money and effort!