Email — AI Sends and Receives Mail

Email — AI Sends and Receives Mail

Imagine this: you send an email to your AI agent, and it replies back — right in the same thread. No fancy chat app, no bot API, no extra software. Just plain old email. That’s exactly what the Email gateway adapter does for Hermes.

In this post, we’ll walk through how to set it up, how it works under the hood, and how to keep your inbox secure.

Why Email?

Email is universal. Everyone has an address. By plugging Hermes into your mailbox via standard IMAP (for receiving) and SMTP (for sending), you get a conversational AI that works with Gmail, Outlook, Yahoo, Fastmail, or any provider that supports these protocols.

And here’s the best part: there are zero external dependencies. The adapter uses Python’s built-in imaplib, smtplib, and email modules. No extra packages, no third-party services.

Note: This is different from the bundled Himalaya email skill, which lets the agent manage your mailbox through terminal commands. That one requires the external himalaya CLI. This gateway adapter is the simpler, self-contained path.

Before You Start

You’ll need three things:

1. A dedicated email account for your agent (please don’t use your personal inbox)
2. IMAP enabled on that account
3. An app password if you’re using Gmail or any provider with two-factor authentication

For Gmail: enable 2FA, then create an App Password at myaccount.google.com/apppasswords. You’ll get a 16-character password — use that instead of your regular one.

For Outlook/Microsoft 365: enable 2FA, create an app password under “Additional security options,” and use outlook.office365.com for IMAP and smtp.office365.com for SMTP.

For other providers, check their docs for IMAP/SMTP hosts and ports (usually 993 for IMAP SSL, 587 for SMTP STARTTLS).

Proton Mail Bridge / local relays. Proton Mail Bridge (and similar local relays such as a self-hosted MTA) listen on loopback with STARTTLS and a self-signed certificate, so the defaults (implicit TLS on IMAP 993, verified certificates) won’t connect. Override the transport in ~/.hermes/config.yaml:

platforms:
  email:
    enabled: true
    extra:
      imap_host: 127.0.0.1
      imap_security: starttls     # tls (default) | starttls | plain
      imap_tls_verify: false      # Bridge uses a self-signed cert
      smtp_host: 127.0.0.1
      smtp_security: starttls     # default: tls on port 465, starttls otherwise
      smtp_tls_verify: false

and set EMAIL_IMAP_PORT=1143 / EMAIL_SMTP_PORT=1025 alongside your Bridge credentials in ~/.hermes/.env. Unknown *_security values log a warning and fall back to the secure default. Only disable *_tls_verify for loopback hosts — Hermes logs a warning when verification is off for any other host.

Step 1: Configure Hermes

The easiest way is the setup wizard:

hermes gateway setup

Select Email from the menu and follow the prompts. It’ll ask for your address, app password, IMAP/SMTP hosts, and allowed senders.

Prefer manual config? Add this to ~/.hermes/.env:

# Required
EMAIL_ADDRESS=hermes@gmail.com
EMAIL_PASSWORD=abcd efgh ijkl mnop    # App password, not your regular one
EMAIL_IMAP_HOST=imap.gmail.com
EMAIL_SMTP_HOST=smtp.gmail.com

# Required: exact authserv-id on your receiving server's topmost
# Authentication-Results header (Gmail: mx.google.com).
# That server must strip inbound headers claiming this id before adding its own.
EMAIL_AUTHSERV_ID=mx.google.com

# Security (recommended)
EMAIL_ALLOWED_USERS=your@email.com,colleague@work.com

# Optional
EMAIL_IMAP_PORT=993                    # Default: 993
EMAIL_SMTP_PORT=587                    # Default: 587
EMAIL_POLL_INTERVAL=15                 # Seconds between inbox checks
EMAIL_HOME_ADDRESS=your@email.com      # Default delivery target for cron jobs

Step 2: Start the Gateway

hermes gateway              # Run in foreground
hermes gateway install      # Install as a user service
sudo hermes gateway install --system   # Linux: boot-time system service

On startup, the adapter tests both IMAP and SMTP connections, marks all existing inbox messages as “seen” (so it only processes new emails), and starts polling.

How It Works

Receiving messages. The adapter checks for UNSEEN emails every 15 seconds by default. For each new message:

• The subject line is included as context (e.g., [Subject: Deploy to production])
• Replies (subject starting with Re:) skip the subject prefix — the thread context is already there
• Attachments are cached locally: images become available to the vision tool, documents like PDFs and ZIPs for file access
• HTML-only emails get tags stripped for plain text extraction
• Self-messages are filtered out to prevent reply loops
• Automated senders are silently ignored — noreply@, mailer-daemon@, bounce@, and emails with Auto-Submitted, Precedence: bulk, or List-Unsubscribe headers

Sending replies. Replies go out via SMTP with proper threading: In-Reply-To and References headers keep the conversation together, the subject keeps its Re: prefix (no double Re: Re:), and responses are sent as plain UTF-8 text.

File attachments. Want the agent to send a file back? Just include MEDIA:/path/to/file in its response and it’ll be attached automatically.

Skipping attachments. Worried about malware or bandwidth? Add this to config.yaml:

platforms:
  email:
    skip_attachments: true

The email body text is still processed normally — only attachments are ignored.

Sender Authentication: The authserv_id Pin

Hermes only acts on a message when the Authentication-Results header stamped by your receiving server authenticates its From: domain (DMARC, or aligned SPF/DKIM). This matters even with open access (EMAIL_ALLOW_ALL_USERS / GATEWAY_ALLOW_ALL_USERS): open access admits any sender, but each conversation and every reply is keyed on From:, so a forged From: would put the attacker’s text into that address’s session and make Hermes mail it. Pairing codes and declines also need an authenticated From:, so neither is mailed to a forged address.

The authserv_id pin is required while sender authentication is on. Without platforms.email.authserv_id (EMAIL_AUTHSERV_ID), Hermes cannot tell your server’s Authentication-Results header from one the sender wrote (a self-hosted MTA that adds none of its own leaves the sender’s on top), so it trusts none: every message that needs an authenticated From: is dropped, and the gateway log says so once, with the fix.

To find the value, open a message your server delivered (“show original” / raw source) and copy the token before the first ; of the topmost Authentication-Results header, for example mx.google.com for Gmail. If that token seems to change between messages, it is a per-relay hostname rather than an authserv-id: RFC 8601 defines the authserv-id as a stable name for the administrative domain, so configure your MTA to stamp one (for example AuthservID in OpenDKIM / OpenDMARC) and pin that. If you cannot pin a stable id, opt out explicitly with platforms.email.require_authenticated_sender: false (or EMAIL_TRUST_FROM_HEADER=true) and accept that From: is then trusted as written. A matching id is a pin, not proof of provenance: confirm your MTA strips inbound Authentication-Results headers claiming its id before it prepends its own.

If your mail server does not stamp that header, set platforms.email.require_authenticated_sender: false (or EMAIL_TRUST_FROM_HEADER=true) to accept the risk; otherwise mail from an admitted sender with no such header is dropped with a Dropping sender with unauthenticated From warning in the gateway log. The same warning, pointing at platforms.email.authserv_id (EMAIL_AUTHSERV_ID) instead, fires when the topmost header does not name exactly the pinned authserv_id. Mail from an address in EMAIL_ALLOWED_USERS / GATEWAY_ALLOWED_USERS that fails authentication otherwise (broken DKIM, a forwarder that breaks alignment) is dropped with the same warning and its verdict, with no opt-out hint. Other senders’ failing mail, such as a forged From: under open access, is dropped quietly, logged at debug level only.

Access Control: Be Strict by Default

Email is more locked down than chat platforms by design:

1. EMAIL_ALLOWED_USERS set → only emails from those addresses (and from GATEWAY_ALLOWED_USERS or an approved pairing) are processed
2. No allowlist set → unknown senders are ignored silently
3. EMAIL_ALLOW_ALL_USERS=true → any sender is accepted (use with caution!)
4. platforms.email.unauthorized_dm_behavior: pair → unknown senders receive a pairing code
5. platforms.email.unauthorized_dm_behavior: decline → an unknown sender receives one polite refusal, then nothing more for 24 hours

Allowlist entries match whole addresses. A bare entry such as alice (a chat username in GATEWAY_ALLOWED_USERS, say) never admits alice@ at any domain, and mail from such an address is dropped rather than paired or declined.

The golden rule: use a dedicated inbox and always configure EMAIL_ALLOWED_USERS. Email pairing is opt-in because shared inboxes often contain unrelated unread messages — Hermes shouldn’t reply to those contacts by default.

Troubleshooting

Problem	Solution
“IMAP connection failed” at startup	Verify EMAIL_IMAP_HOST and EMAIL_IMAP_PORT. Make sure IMAP is enabled on the account (Gmail: Settings → Forwarding and POP/IMAP)
“SMTP connection failed” at startup	Verify EMAIL_SMTP_HOST and EMAIL_SMTP_PORT. Check that your password is correct (use an App Password for Gmail).
Messages not received	Check EMAIL_ALLOWED_USERS includes the sender’s email. If the gateway log shows authserv-id is not configured, set EMAIL_AUTHSERV_ID to the exact id on the topmost Authentication-Results header your server writes. Check spam folder — some providers flag automated replies.
“Authentication failed”	For Gmail, you must use an App Password, not your regular password. Ensure 2FA is enabled first.
Duplicate replies	Ensure only one gateway instance is running. Check hermes gateway status.
Slow response	The default poll interval is 15 seconds. Reduce with EMAIL_POLL_INTERVAL=5 for faster response (but more IMAP connections).
Replies not threading	The adapter uses In-Reply-To headers. Some email clients (especially web-based) may not thread correctly with automated messages.

Configuration Reference

Variable	Required?	Default	Notes
EMAIL_AUTHSERV_ID	Yes, unless sender authentication is off	—	Exact authserv-id on your receiving server’s topmost Authentication-Results header; without it every message needing an authenticated From: is dropped

Wrapping Up

The Email gateway adapter turns any standard mailbox into a communication channel for your AI agent. No external dependencies, no special clients — just IMAP, SMTP, and a little configuration. Set up a dedicated account, lock down your allowed senders, and let the emails flow.

Happy emailing! 📧