Tutorial 13: How to Choose the Right AI Model in Hermes

Welcome back to the Hermes Agent series! Today we’re tackling a question every user asks sooner or later: which AI model should I actually use? With hundreds of options out there, it’s easy to feel overwhelmed. But here’s the good news — Hermes is built to make this decision way simpler than you’d think.

Two Kinds of Model Slots

First, let’s clear up a common point of confusion. Hermes doesn’t just use one model — it uses two categories:

Main model — this is the “brain” of your agent. Every user message, every tool call, every streamed response goes through it. Think of it as the star of the show.
Auxiliary models — these are the behind-the-scenes helpers. Hermes uses them for smaller jobs like context compression, image analysis, web-page summarization, approval scoring, MCP tool routing, session-title generation, and skill search. Each of these 11 tasks has its own slot, and you can override them independently.

The Fastest Path: Nous Portal

If you’re just getting started, here’s a pro tip: Nous Portal gives you access to 300+ models under a single subscription. On a fresh install, just run:

hermes setup --portal

That single command logs you in and sets Nous as your provider. You can inspect what’s wired up with hermes portal info. Bonus: Portal subscribers get 10% off token-billed providers. Not bad, right?

The Models Page

Open your dashboard and click Models in the sidebar. You’ll see two sections:

Model Settings — the top panel where you assign models to slots.
Usage analytics — ranked cards showing every model that ran a session, with token counts, cost, and capability badges.

The main row shows what your agent will use for new sessions. Click Change to open the picker.

Setting the Main Model

The picker dialog has two columns:

Left — your authenticated providers. Only providers you’ve actually set up (API key, OAuth, or custom endpoint) show up here. Missing one? Head to Keys and add the credential.
Right — the curated model list for that provider. These are the agentic models Hermes recommends — not the raw /models dump (which on OpenRouter includes 400+ models, including TTS and image generators you don’t need).

Type in the filter box to narrow things down. Pick a model, hit Switch, and Hermes writes it to ~/.hermes/config.yaml under the model section.

Important: This applies to new sessions only. If you have a chat tab already open, it keeps running whatever model it started with. To hot-swap the current chat, use the /model slash command inside it.

Speed Tiers: Fast and Ultrafast

Speed is a separate setting from reasoning effort. On supported subscription models, Fast requests Priority, and Ultrafast requests the Ultrafast tier — picking one clears the other. Your choice is remembered per model and carried into a new chat.

The profile default lives under Settings → Model → Main model → Speed, stored as agent.service_tier (normal, fast, or ultrafast). Ultrafast only appears for models that offer it, and on a subscription only when your account’s Codex catalog lists it for that model. If you ever want to undo it, Use standard speed clears the legacy setting. One caveat: an accepted request doesn’t guarantee OpenAI actually served the tier you asked for.

The GPT-6.1 Sol-900k Choice

Hermes creates the GPT-6.1 Sol-900k option from an eligible discovered GPT-6.1 Sol model. The -900k suffix is a local context choice — it’s stripped from the model ID sent to OpenAI. Hermes limits its budget to the account catalog’s maximum, and Codex may report a smaller usable context after its own reserve. If a new model doesn’t show up, hit Refresh models. A failed discovery uses short-lived compatibility hints rather than caching them as a successful account catalog.

Answers That Arrive in the `reasoning` Field

Some OpenAI-compatible servers return the whole answer in the reasoning field with an empty content. Hermes does not show reasoning as the reply by default, because most providers put private chain-of-thought there. It retries instead. If your server’s parser really does put the final answer in reasoning, opt that model in with answer_in_reasoning:

providers:
  local-vllm:
    api: http://127.0.0.1:8000/v1
    models:
      my-reasoner:
        answer_in_reasoning: true

To opt in every model on a provider, put the flag in a provider-level capabilities: block (capabilities: {answer_in_reasoning: true}). Hermes re-reads the opt-in on the active route on every turn, in the CLI, TUI and gateway alike, so it survives a /model switch to another model on the same provider. A fallback or any other provider does not inherit it, and OpenRouter routes are never trusted, even with the flag set.

A Word About Switching Mid-Session

Switching models mid-conversation works, but there’s a catch: prompt caches reset. Since caches are keyed to the model serving the request, any mid-conversation change means the next message re-reads the entire conversation at full input-token price instead of the cached (~75–90% discounted) rate. On a long session, that one-time re-read can cost more than the per-token difference between models. So switch when you need to, but prefer doing it early in a conversation.

An explicit /model switch also rewrites the Model: (and Provider:) line inside the system prompt so the agent knows what it is running on. That is a deliberate, one-time change: the first request after the switch misses the cached prefix once, and everything after the system prompt stays byte-identical. Automatic fallbacks rewrite the same line only in memory — the stored session keeps the primary’s labels, so a restored primary replays a byte-identical prompt.

Because of that one-time re-read cost, Hermes asks for explicit confirmation before applying a mid-session switch when the live session already holds a large context (default: 100,000 tokens, measured from the latest provider-billed prompt size). You’ll see this prompt in the CLI and TUI /model command and picker, and for a typed gateway /model in a chat with an active agent. Tune or disable it in config.yaml:

model:
  # Ask before mid-session switches when the session exceeds this many
  # context tokens (the next reply re-reads them uncached). 0 disables.
  switch_context_confirm_tokens: 100000

Re-selecting the model you’re already on never prompts (the cache stays warm), and sessions with no measured context (fresh sessions, non-live surfaces) are exempt.

Unattended Data-Training Tiers

Some models with a -contributor suffix (like muse-spark-1.2-contributor or muse-spark-1.3-contributor) are discounted because the vendor may train on your data. Interactive selection always shows a confirmation prompt. But for unattended workloads (Kanban workers, cron agents), Hermes fails closed because it can’t ask you.

If you’re okay with this for unattended tasks, record a persistent acknowledgement:

hermes config set security.allow_data_training_tiers_noninteractive true

Hermes still prints the full data-policy warning on every startup, so you keep an audit trail. Revoke it anytime with hermes config unset security.allow_data_training_tiers_noninteractive.

Setting Auxiliary Models

Click Show auxiliary to reveal the 11 task slots. Every auxiliary task defaults to auto — meaning Hermes tries your main model for that job too. If that route is unavailable or hits a capacity-style failure, it follows any task-specific auxiliary.<task>.fallback_chain, then the main fallback_providers / fallback_model chain. It never guesses a provider you did not configure: with a main provider selected and no fallback declared, the side task is skipped with a warning rather than billed to another account you happen to be logged into. (Hermes’ built-in discovery chain only runs when no main provider is selected at all.)

Override a specific task when you want a cheaper or faster model for a side-job. For example, you might want a tiny model for session-title generation but keep your big model for vision tasks.

The Bottom Line

Choosing the right model doesn’t have to be stressful. Start with the main model that fits your budget and task complexity, let the auxiliary slots default to auto, and only override when you notice a bottleneck. Hermes handles the rest.

Happy modeling! 🚀