Installation & Reference

• Get Started (Docker)
• Release and Update Methodology
• Storage Topology (5 tiers)
• Post-Restore Steps
• Post-Migration Checklist: Legacy → Docker
• Hermes SEG Email Flow

Get Started (Docker)

Get Started (Docker)

This page is the minimum config needed to get mail flowing on a fresh Hermes SEG Docker install. The install script (scripts/install_hermes_docker.sh) does most of the heavy lifting; this page covers the handful of admin-UI steps that still need a human.

Skip these and Postfix will silently bounce or reject mail. The admin dashboard also surfaces two universal nudges (placeholder hostname, self-signed cert) until those are addressed (see Dashboard nudges at the bottom).

Which steps apply to you?

Hermes supports three deployment topologies. Step 1 (System Identity) and the Optional/DNS sections apply to everyone, and Step 2 (Console FQDN and a real certificate) is required for anything that uses Nextcloud. The middle of this guide then splits into a Relay path and a Mail server path. Follow only the one(s) for your topology:

Topology	What it is	Follow
Relay-only	Hermes filters mail and forwards it to a downstream mail server (MX)	Step 1 → Relay configuration
Mail-server-only	Hermes hosts the mailboxes itself (Dovecot + webmail)	Step 1 → Step 2 → Mail server configuration
Hybrid	Both: some domains relay out, others have local mailboxes	Step 1 → Step 2 → Relay configuration → Mail server configuration

Legacy reference: this page replaces the pre-Docker 16-step page. The Docker install script absorbs ~6 of those steps, so the list below is shorter.

---

What the install script already did

You don't need to redo any of this; install_hermes_docker.sh handled it during the install run:

Component	Result
Containers	All Hermes containers running (docker compose ps)
Bootstrap admin	LDAP user in cn=admins + cn=one_factor, password in INSTALL_SUMMARY
TLS	Self-signed bootstrap cert in System Certificates, bound to Console / SMTP / Webmail roles
Databases	MariaDB schemas (hermes, djigzo, opendmarc, syslog, authelia, nextcloud) created + seeded
Console settings	parameters2.console.host set to the host IP, not your FQDN. See the note below
Postfix identity	myhostname / myorigin set from the install-time mail-hostname prompt
Authelia	LDAP backend wired up; 2FA enrollment available on first login
Mail filtering	Amavis, SpamAssassin and ClamAV running. Network checks and Bayes still need setup. See Antispam Settings

Why the console address is an IP. At install time there is usually no DNS record yet for the FQDN you intend to use, so the installer deliberately points the console, Nginx, Authelia and Nextcloud at the host IP. That way you can log in immediately. Once DNS resolves, save System → Console Settings with your FQDN and Hermes re-renders the whole web stack. Do this before handing the console to anyone else.

So after the install you can log in, but mail won't actually flow until you complete the steps below.

---

Step 1: System Identity (all topologies)

Page: System → Server Setup

The install script sets myhostname from what you typed at the FQDN prompt, but you should double-check it matches your DNS A / MX records. Also set:

• Postmaster address: where bounce messages and admin notifications go
• Admin email: where alerts (license, system events) get delivered
• Time zone: affects log timestamps and report scheduling

Dashboard nudge: an orange callout Placeholder hostname fires (any topology) if myhostname still equals the seed default hermes.domain.tld or console.host equals smtp.domain.tld. Both should never appear on a Docker install (the install script overrides them), but if they do, this is the page to fix.

---

Step 2: Console FQDN and a real certificate

Required for mail server and hybrid. Optional for relay-only.

Pages: System → Console Settings, then System → System Certificates

The installer leaves you with a console reachable at the host IP, secured by a self-signed bootstrap certificate whose common name is localhost. That is deliberate, and it is enough to log into the admin console and enough for mail to flow.

It is not enough for Nextcloud. Until you finish both parts of this step, https://<console-host>/nc/ login fails for every user, including the administrator. Webmail, the Nextcloud Mail client, and file access are all unreachable until it is done.

Why it fails

Nextcloud signs users in through OIDC. That is not a browser-only flow: the Nextcloud container makes its own server-side HTTPS call back to the console address to reach the identity provider. On a fresh install that call is refused, because the bootstrap certificate is self-signed and its common name is localhost, which matches neither the host IP nor the FQDN you intend to use.

Importing the bootstrap certificate into Nextcloud's trust store does not fix this. Trust and name are two separate checks, and the name still does not match. The only fix is a certificate issued for the console's real name.

Part 1: point the console at an FQDN

Create a DNS A record for the name you want (for example mail.example.tld) pointing at this host, and wait for it to resolve. Then open System → Console Settings, enter that FQDN, and save. Hermes re-renders Nginx, Authelia and Nextcloud, and restarts the web stack.

The save is gated on DNS: if the name does not resolve to this host, Hermes keeps the current address rather than locking you out.

Part 2: install a real certificate for that FQDN

Go to System → System Certificates and issue or import a certificate covering the FQDN you just set. The three paths are described under Real TLS Certificate below.

The certificate must cover the console address specifically. A certificate valid for your mail domain but not for the console name leaves Nextcloud login broken in exactly the same way.

Confirm it worked

Open https://<your-fqdn>/nc/ and log in as any mailbox user. A successful sign-in means the OIDC round trip completed, which is the thing that was failing.

If you see "Could not reach the OpenID Connect provider", one of the two parts is incomplete: either the console is still on an IP, or the certificate does not cover the name in use.

Part 3: bind the certificate to SMTP as well

This is a separate setting and it is the one people miss. Setting the Console Certificate does not configure Postfix. They are two independent bindings:

Role	Page	Serves
Console Certificate	System → Console Settings	nginx on 443: /admin, /users, /nc
SMTP TLS certificate	System → SMTP TLS Settings	Postfix on 587 and 465

Go to System → SMTP TLS Settings, select the same certificate you bound to the console, and save.

Skip this and the console looks perfect in a browser while every mail client gets a certificate error, because Postfix is still serving the install-time bootstrap certificate with common name localhost. Thunderbird, Outlook and phones either refuse the connection or train the user to click through a warning every time.

It has to be the certificate covering the console host, because that is the name clients are told to use: autoconfig.cfm reads console.host and hands it out as both the IMAP and the SMTP server, and the SRV records on the Mailbox Domains page point at the same host.

Confirm it took:

docker exec hermes_postfix_dkim postconf -n | grep smtpd_tls_cert_file

It should name your real certificate. If it still says bootstrap_hermes.pem, the setting was not saved. A browser check cannot tell you this, because the browser never connects to Postfix.

Relay-only deployments do not use Nextcloud, so the Nextcloud half of this step is not blocking. The certificate still matters if your users connect any mail client. Do it before handing the console to anyone else either way, so administrators are not training themselves to click through certificate warnings.

---

Relay configuration

For relay-only and hybrid deployments. If Hermes hosts your mailboxes and never forwards to a downstream MX, skip this whole section and follow Mail server configuration below.

A. Relay Domains

Page: Email Relay → Domains

Add at least one domain so Hermes knows what mail to accept on the SMTP port. Without this, every inbound message gets rejected with Relay access denied.

For each domain you'll choose:

Field	What it controls
Domain	The recipient domain (e.g. example.com)
Recipient delivery mode	Where validated mail goes next: relay forwards to a downstream MX (the usual relay-topology choice)
Destination address / port	The downstream MX host + port that accepts the forwarded mail
Policy	Encryption policy applied to outbound mail for this domain (Pro only)

B. Relay Networks

Page: Email Relay → Relay Networks

Add the IP addresses or CIDR blocks of any upstream MTA (your customer's mail server, an application server that sends notification mail, etc.) that should be allowed to relay outbound mail through Hermes.

By default Hermes only trusts 127.0.0.1 and the Docker bridge subnet (172.16.32.0/24). Anything else needs to be added here.

C. Relay Recipients

Page: Email Relay → Relay Recipients

Add the individual recipients (or wildcards) that Hermes should accept mail for. Validated mail is then forwarded to the destination set on the domain row in step A. Without at least one recipient, mail for the domain is rejected as unknown.

---

Mail server configuration

For mail-server-only and hybrid deployments. If Hermes only relays to a downstream MX and hosts no mailboxes, skip this whole section.

A. Mailbox Domains

Page: Email Server → Domains

Add at least one mailbox domain, the domain Hermes will host mailboxes for. This is a different page from Email Relay → Domains: it provisions the local-delivery side (Dovecot, autoconfig/autodiscover, webmail), not relay forwarding.

When you add a mailbox domain Hermes sets up the per-domain mail-client autoconfiguration. For TLS, mailbox domains need a certificate that also covers autoconfig.<domain> and autodiscover.<domain>. See Real TLS Certificate below.

B. Mailboxes

Page: Email Server → Mailboxes

Create the individual mailboxes under your mailbox domain(s). Each mailbox row creates an LDAP user, a Dovecot maildir, and (optionally) a Nextcloud account for webmail/file access. Users log in to webmail via Authelia SSO at https://<console-host>/nc/.

---

Optional but recommended

Relay Host (Outbound Smarthost) (relay / hybrid)

Page: Email Relay → Relay Host

If outbound mail should route through an upstream provider (Gmail, Microsoft 365, SendGrid, etc.) instead of being sent directly to recipient MXes, configure the smarthost here. Authentication credentials are encrypted at rest using the Hermes install's key material.

Pro License Activation (all topologies)

Page: System → Server Setup → License section

Enter your serial number to unlock Pro features (organizational signatures, encrypted mail, ARC sealing, Link Guard, etc.). Validation hits validate.hermesseg.io over HTTPS; the result is cached locally so Pro stays available during brief network outages.

Real TLS Certificate (all topologies)

Page: System → System Certificates

Replace the bootstrap self-signed certificate with a real one before going live. On mail server and hybrid installs this is not optional: Nextcloud login cannot work until the console has an FQDN and a certificate issued for it. See Step 2. Three paths:

Path	Tier	Workflow
Request ACME	Both tiers	Click → enter domain → Let's Encrypt issues automatically, auto-renews
Import Certificate	Both tiers	Paste cert + key + chain from any CA you already have
Generate CSR	Both tiers	Generate signing request → submit to CA → import the result via the Import Certificate path

For mailbox-hosting domains, see the in-app "Choosing the Right Certificate Type" panel on the System Certificates page; mailbox certs need SAN coverage for autoconfig.<domain> and autodiscover.<domain>.

Dashboard nudge: blue informational callout Self-signed cert fires when the only row in system_certificates is the install-generated bootstrap (no real cert has been imported yet). Mail flows on the bootstrap certificate and clients get a TLS warning, but on mail server and hybrid installs the consequence is larger than a warning: Nextcloud login does not work at all until a real certificate covering the console FQDN is in place. See Step 2.

DKIM Signing (all topologies)

Page: Content Checks → DKIM Settings

A fresh install has no DKIM keys, and that is expected. Keys are per-domain, so they cannot exist before your domains do. Until you generate one, KeyTable, SigningTable and the dkim_sign table are all empty and /opt/hermes/dkim/keys holds nothing. OpenDKIM is running correctly the whole time; it simply has nothing to sign with.

The order matters:

1. Add the domain first: Email Relay → Domains, or Email Server → Domains
2. Generate the key: Content Checks → DKIM Settings, select the domain, choose a selector and key size
3. Publish the TXT record the page gives you at your DNS provider
4. Verify: the page re-checks the published record and reports a match

Repeat per domain. Outbound mail for a domain is not signed until step 3 has propagated.

DNS for Mail Flow (all topologies)

Beyond the gateway itself, DNS is what makes mail actually arrive. The install script does not touch DNS; you do this at your registrar. "Your domains" below means relay domains, mailbox domains, or both, whichever you configured above.

Record	Where it points	Why
MX for each domain	The Hermes mail hostname (e.g. mail.example.com)	Inbound mail routing
A for the mail hostname	Hermes public IP	Resolves the MX target
Reverse DNS (PTR) for the IP	The mail hostname	Outbound deliverability; most receivers reject mismatched PTR
SPF for each sending domain	Includes Hermes IP	Authenticates outbound; reduces spam-folder rate
DKIM selector → public key	Generated under Content Checks → DKIM Settings	Cryptographic signing of outbound
DMARC policy	TXT at _dmarc.example.com	Defines what receivers do with SPF/DKIM failures

Review the Admin Account Email (all topologies)

Page: System → System Users (edit the admin user), or your My Profile link (top of the sidebar)

The install created the admin account with a generated email of the form <admin-username>@<your-mail-domain> (e.g. apologise4567@example.com). That address is where Hermes sends admin notifications and password-reset mail, so unless it maps to a real, monitored mailbox, change it to one that does.

Antispam Settings (Pyzor / Razor / Bayes) (all topologies)

Page: Content Checks → Antispam Settings (both the on/off switches and the one-time actions)

Spam filtering works out of the box from SpamAssassin's rule set, RBLs, ClamAV and the malware feeds. Three optional components need a deliberate decision from you.

Collaborative network checks (Pyzor, Razor, DCC)

These check a message's fingerprint against a shared network of reporters. They are effective, and they transmit a digest of every message you scan to a third party. That is a decision for you, not for us.

• From v260807 they ship disabled. Enable them under Antispam Settings if you want them.
• On installs predating v260807 they were enabled but never registered, so they contributed nothing. Enabling them properly is the same procedure.

Once enabled:

Check	What it needs
Pyzor	Nothing. It works as soon as it's enabled (outbound internet required).
Razor	A one-time registration via Content Checks → Antispam Settings → Initialize Razor. Until this runs, Razor returns no result.
DCC	Not included in the published image for licensing reasons. See Antispam Settings if you want to add it.

Bayes

The Bayesian classifier learns what your mail looks like. It ships empty and contributes no score at all until it has learned roughly 200 spam and 200 ham messages. That is SpamAssassin's own safety threshold, not a Hermes limitation.

• Train it with Train as Spam / Train as Ham in Quarantine and Message History. Users can do this from their own portal too.
• Automatic learning is off by default and we recommend leaving it off. Autolearn trains on whatever the rule set already decided, so it reinforces the rules' mistakes as readily as their successes, and on a gateway that sees mostly spam it skews badly.
• Installs predating v260807 carry a Bayes corpus that shipped with Hermes, trained on unrelated mail. The v260807 upgrade clears it once so your gateway learns from your own traffic.

Barracuda Central Registration (all topologies)

Hermes' Postfix postscreen DNSBL list includes b.barracudacentral.org, and Barracuda Central only answers queries from registered IPs. Register your gateway's sending IP (free) at the Barracuda Reputation Block List site so those lookups return results instead of being silently ignored.

CipherMail Console Admin Password (all topologies (encryption))

Page: the CipherMail console at /ciphermail (behind Authelia SSO)

The CipherMail encryption console has its own administrator account, separate from the Hermes/Authelia admin login. It ships with CipherMail's stock default credentials:

Username	Password
admin	admin

Sign in with those and change the password immediately. The account is not managed by Hermes, so nothing else will prompt you and no dashboard nudge fires for it. Authelia SSO gates the /ciphermail path, which means the default is not reachable from the internet, but it remains a stock credential on an admin interface and should not survive your first login.

---

Things that look broken but aren't

A fresh install leaves several things deliberately empty. If you go looking under the hood before configuring anything, these are the ones that reliably cause alarm.

What you see	Why	What to do
No DKIM keys. KeyTable, SigningTable and dkim_sign are empty; /opt/hermes/dkim/keys holds nothing	Keys are per-domain and can't precede your domains	Generate them after adding a domain
MariaDB root has no password. docker exec hermes_db_server mariadb -u root connects with no credentials	root@localhost uses the unix_socket plugin, so it only trusts a process already running as root inside the container. The password in INSTALL_SUMMARY.txt belongs to root@'%', the remote entry, and port 3306 is not published to the host	Nothing. This is the intended design
The migrations table is empty	It records one-time upgrade migrations. A fresh install has never upgraded, so there is nothing to record	Nothing
Bayes reports no data / no effect on scores	It ships empty and stays inert until ~200 spam + 200 ham are learned	Train it
ecprivkey.pem and ecpubkey.pem are empty files	Mailbox encryption is off by default. The placeholder files exist only so Docker doesn't create directories in their place	Enable it under Email Server → Settings, which generates the real keypair. Back the keys up: losing them makes encrypted mail permanently unreadable

Nextcloud login needs the console FQDN and a real certificate first

Before anything below matters, the console must be on an FQDN with a certificate issued for it. On a fresh install neither is true, and /nc/ login fails for everyone with "Could not reach the OpenID Connect provider". This is the single most common reason webmail appears broken on a new deployment. See Step 2.

Trusting the bootstrap certificate inside the Nextcloud container does not help, because its common name is localhost and will never match the console address.

Nextcloud must be enabled when you create the mailbox

Enable Nextcloud on the Add Mailbox form, not afterwards. The setting controls three things at once: /nc access, the Nextcloud account itself, and the Nextcloud Mail profile that authenticates to Dovecot on the user's behalf.

The default comes from the domain's own Nextcloud setting, which is off unless you turned it on when you added the domain, so it is easy to leave off without noticing.

If Nextcloud Mail shows no account, or reports an authentication failure, that is the cause. Do not try to add the account by hand with the user's login password. It will never work, because the login password is not a mail credential. Hermes authenticates IMAP and SMTP only against app passwords, and Nextcloud Mail uses an automatically managed one the user never sees. Thunderbird and phones use app passwords the user generates in the portal under My App Passwords.

Installs predating v260807: enabling Nextcloud after a mailbox was created did not provision the Mail profile. The v260807 upgrade repairs affected mailboxes automatically.

---

Dashboard nudges

The admin dashboard surfaces two universal callout banners under the navbar. These apply regardless of topology:

Color	Priority	Trigger
Orange Placeholder hostname	2	myhostname or console.host still at the seed placeholder (hermes.domain.tld / smtp.domain.tld)
Blue Self-signed cert	3	Only the bootstrap cert exists in System Certificates; no real cert imported yet

Each banner links directly to the page where you'd fix the underlying condition and disappears automatically as soon as the check passes. The topology-specific steps (relay domains, networks, recipients, mailbox domains, mailboxes) intentionally don't have dashboard nudges; they depend on which topology you're using and the guidance above covers them.

---

After you finish these steps

1. Inbound test: send a message from an external account to a recipient on one of your domains. Check Reports → Mail Log to confirm it reached Hermes and was handed off (relay) or delivered to the mailbox (mail server).
2. Outbound test (relay / hybrid): send a message from your customer MTA (the one whose IP you added to Relay Networks) to an external recipient. Confirm DKIM/SPF pass on the receiver side.
3. Webmail test (mail server / hybrid): log in to https://<console-host>/nc/ as one of your new mailbox users (Authelia SSO) and confirm send/receive. If this fails with "Could not reach the OpenID Connect provider", Step 2 is incomplete: the console is still on an IP, or the certificate does not cover the name in use.
4. Open the Admin Console home page and confirm both setup nudges are gone (placeholder hostname + self-signed cert).
5. If you set up Pro features, verify session.edition reads "Pro" in the top-right corner of any admin page.

You're done. Welcome to Hermes SEG.

Release and Update Methodology

Release and Update Methodology

This document is the canonical reference for how Hermes SEG (Docker era) is released, distributed, and upgraded. It covers both sides of the loop: the developer side (how a release is cut) and the admin side (how a running install is updated). Read this if you are:

• An operator deciding how to upgrade a running Hermes install
• A developer adding a schema change, a one-shot migration, or a service config edit and need to know where it goes
• A maintainer cutting a new release tag
• A future Claude session that needs the mental model

Linked work: #218 (release engineering pivot), #221 (system_update_docker.sh orchestrator), #231 (beta scope).

Core concepts

Calendar versioning

Versions are vYYMMDD — a label named for a target/planning date. The digits are not the ship date and not the git tag date; a release is often named early and tagged months later as the work lands.

v260119  →  named for 2026-01-19, actually tagged 2026-05-30   (first Docker release, the baseline)
v260609  →  named for 2026-06-09                                (backup/restore/DR + upgrade-tooling release)
v270315  →  hypothetical: a release named for 2027-03-15

The label does not auto-increment — there is no v260120 just because a day passed. For the actual release timing of any tag, read the git tag date (git log -1 <tag>) or the GitHub Release page — never infer it from the digits.

The version stamp is stored in two system_settings rows:

Parameter	Value	Meaning
version_no	'Docker'	Code train identifier (post-bare-metal era)
build_no	e.g. 'v260723'	Specific release tag the install is currently at. On a fresh install, written by install_hermes_docker.sh from the newest updates/v<YYMMDD>/ directory; on an upgrade, advanced by each release's schema_updates.sql.

Image registry

Container images live at ghcr.io/deeztek/hermes-<service>:<tag>. The registry hostname is configurable via the IMAGE_REGISTRY env var in .env so the legacy GitLab CR (hub.deeztek.com/dedwards/hermes-seg-docker-gl) can still be used during the bootstrap period before ghcr.io is fully populated.

Build/push via Docker/build-all-ghcr.sh + Docker/push-all-ghcr.sh (ghcr) or Docker/build-all.sh + Docker/push-all.sh (legacy hub). When you add a new service container, add it to BOTH the build-all and push-all scripts of the relevant pair (the build_image call and the IMAGES/echo lists) — otherwise the release build silently omits it.

ghcr push auth: pushing to ghcr.io/deeztek needs a token with the write:packages scope; a plain gh auth login does NOT include it (its default scopes are repo/workflow/read:org/gist). Add it with gh auth refresh -h github.com -s write:packages then gh auth token | docker login ghcr.io -u deeztek --password-stdin, or use a classic PAT created with write:packages + read:packages. A brand-new package (e.g. the first push of a new service) lands private by default — set it Public on the GitHub package settings page for a public release.

Distribution repo

Code lives on GitLab (dev). Distribution is via GitHub (deeztek/Hermes-Secure-Email-Gateway) — issues, releases, container packages all there. A two-remote scripts/git_release.sh helper codifies the dev-push / release-push split.

Artifact taxonomy — where does what go?

Hermes ships three categories of changes between releases. Each has a dedicated home in the repo.

+------------------------------------------------------------------+
|  CATEGORY 1: BASELINE                                            |
|  Lives at: config/database/hermes_install.sql                    |
|  Purpose:  Self-contained snapshot of the schema AT v260119.     |
|            Imported once on fresh install. Never modified by     |
|            upgrades.                                             |
+------------------------------------------------------------------+

+------------------------------------------------------------------+
|  CATEGORY 2: PER-RELEASE ARTIFACTS                               |
|  Lives at: updates/v<DATE>/                                      |
|    ├── README.md                  (what this release does)      |
|    ├── sql/schema_updates.sql     (schema deltas)               |
|    ├── cfml/*.cfm                 (one-shot CFML migrations)    |
|    └── scripts/*.sh               (one-shot bash one-shots)     |
|  Purpose:  Everything that changed between the PREVIOUS release |
|            and THIS release. Applied once, in order, by the     |
|            update orchestrator. Discoverable: one directory     |
|            tells you everything that release does.              |
+------------------------------------------------------------------+

+------------------------------------------------------------------+
|  CATEGORY 3: PERSISTENT POST-UPGRADE HOOK                        |
|  Lives at: config/hermes/var/www/html/schedule/post_upgrade.cfm  |
|  Purpose:  Cross-release CFML migrations gated by a `migrations` |
|            table. Each named block runs ONCE EVER per install.   |
|            Used for retroactive backfills and migrations that    |
|            don't cleanly bind to a single release.               |
+------------------------------------------------------------------+

Categories side-by-side

	Baseline	Per-release	Persistent hook
File location	config/database/hermes_install.sql	updates/v<DATE>/	schedule/post_upgrade.cfm
Scope	Whole v260119 schema	One release's deltas	Cross-release backfills
When applied	Once, at fresh install	Once, when upgrading past that version	At every upgrade; per-block gated
Lifecycle	Replaced wholesale at major rebaselines (rare)	Lives forever in repo as history	Blocks accumulate over time; stay as documentation
Idempotency	Required (IF NOT EXISTS / INSERT IGNORE)	Required per artifact	Required per block (migrations table check)
Discoverability	One file, mysqldump-style	One dir per release	One file with named blocks

Choosing the right category

Use this decision tree when adding a schema/data/config change:

Is this a one-off "we noticed something needs fixing across all installs"
that doesn't bind to any specific release?
  YES → Persistent post-upgrade hook (category 3)
  NO  → continue

Is this a schema change, seed data, or config that NEW INSTALLS should have?
  YES → Add to baseline (category 1) — ALSO add to the current in-flight
        release's updates/v<DATE>/ (category 2) so existing installs get it
        on upgrade.
  NO  → continue

Is this a one-shot migration that only matters when transitioning past a
specific release (e.g., re-encrypt rows after a key rotation)?
  YES → Per-release artifact (category 2):
          - Pure SQL → updates/v<DATE>/sql/schema_updates.sql
          - Needs CFML (encryption, file I/O, API calls) → updates/v<DATE>/cfml/<name>.cfm
          - Needs host shell (docker exec, file moves outside containers) → updates/v<DATE>/scripts/<name>.sh

Idempotency rules

Every artifact, in every category, MUST be safely re-runnable. The orchestrator does not track "did I apply this already" at fine granularity — it relies on each artifact to no-op when there's nothing to do.

SQL artifacts

Statement	Idempotency pattern
CREATE TABLE	CREATE TABLE IF NOT EXISTS
ALTER TABLE ADD COLUMN	ADD COLUMN IF NOT EXISTS
ALTER TABLE DROP COLUMN	DROP COLUMN IF EXISTS
ALTER TABLE MODIFY COLUMN	Naturally idempotent (re-applying same type is no-op)
CREATE INDEX	CREATE INDEX IF NOT EXISTS
INSERT	INSERT IGNORE (relies on UNIQUE key) OR INSERT ... SELECT ... WHERE NOT EXISTS
UPDATE	Value-gated WHERE clause that only matches pre-fix state (e.g., WHERE value = 'Fail')
DELETE	Targeted WHERE that becomes empty after first application
CREATE USER / GRANT	Wrap in PREPARE block that checks mysql.user / mysql.db first (see config/database/hermes_install.sql for canonical pattern)

When in doubt, look at how an existing block in hermes_install.sql or a prior schema_updates.sql handled the same pattern. The CLAUDE.md "Schema Updates" section also documents these patterns inline.

CFML artifacts

Each .cfm file in updates/v<DATE>/cfml/ or each block in post_upgrade.cfm must:

1. Check before doing: query something (a row's encryption state, a file's existence, a setting's value) to decide whether to act.
2. Act: do the work.
3. Record completion: either update a row to a state the check in step 1 will reject, OR insert into the migrations table.

The migrations table (created by Phase 5 framework, see below):

CREATE TABLE IF NOT EXISTS `migrations` (
    `id` INT NOT NULL AUTO_INCREMENT,
    `name` VARCHAR(255) NOT NULL,
    `completed_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (`id`),
    UNIQUE KEY `uq_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

Bash artifacts

Each .sh file in updates/v<DATE>/scripts/ must:

1. Precondition check: test whether the operation has already been performed (file exists, container running, value set, etc.).
2. Act: do the work.
3. Exit 0 on no-op: a script invoked when its preconditions say "already done" must exit 0, not error.

The update orchestrator (scripts/system_update_docker.sh)

This script (target: #221) is the single command an admin runs to upgrade a Hermes install. It runs five phases in order; each is independently idempotent.

Phase 1 — Pull new code

cd /opt/hermes-seg-docker-gl
git fetch --tags
git checkout <target-tag>

The target tag is either passed as an argument (./system_update_docker.sh v260601) or auto-resolved by polling GitHub Releases API for releases/latest. If the working tree is dirty, the script refuses to continue.

Phase 2 — Update containers

docker compose pull              # only when HERMES_DOCKER_IMG_VERSION in .env bumped
docker compose up -d --build     # always; --build is REQUIRED (see fail2ban note)

Image registry comes from IMAGE_REGISTRY in .env. Containers whose images didn't change are NOT restarted by compose up -d.

--build is mandatory on both install and update. hermes_fail2ban is the only service built locally from its Dockerfile (it adds mariadb-client + iptables-backend detection on top of the upstream lscr.io/linuxserver/fail2ban image) and is not pushed to a registry. docker compose pull fetches the vanilla upstream fail2ban over the locally built image, so an upgrade run with a plain docker compose up -d (no --build) would silently drop those customizations. --build is a no-op for the image-only services, so it is safe to always include.

Phase 3 — Apply per-release artifacts

The orchestrator globs updates/v*/ directories, sorts chronologically (calendar versioning sorts correctly as a string), filters to versions newer than current build_no, and applies each release in order:

For each release dir D in (sorted release dirs above build_no):
    For each file F in D/sql/*.sql       (alphabetical):
        docker exec -i hermes_db_server mysql -u root hermes < F
    For each file F in D/cfml/*.cfm      (alphabetical):
        curl -sf http://localhost:8888/updates/<dir>/cfml/<name>.cfm
    For each file F in D/scripts/*.sh    (alphabetical):
        bash F
    # Each release's schema_updates.sql advances build_no at its end via
    #   UPDATE system_settings SET value='<DATE>' WHERE parameter='build_no';
    # giving the orchestrator a natural cursor across releases.

If any artifact fails, the orchestrator aborts. Because build_no advances at the END of each release's sql/ step, a failed-mid-release re-run picks up at the failed release and re-applies its full set (idempotency makes that safe).

Why this ordering: SQL changes the schema first so CFML and bash artifacts that depend on new columns/tables work. CFML before bash because CFML migrations often need the schema landing but Lucee's lifecycle is already running. Bash last for host-level fixups.

Phase 4 — Standard finalize steps

- Restart hermes_commandbox so it picks up admin/2/, schedule/, schema changes
- NC version-drift detection (#264): if .env's NCVERSION differs from
  the live `occ status` versionstring → run `occ upgrade` + rehydrate
  Hermes-required NC apps (user_oidc, mail, twofactor_totp,
  twofactor_backupcodes, external) + verify needsDbUpgrade=false
- Re-render *.HERMES config templates (reminder only in MVP; v2 will
  detect modified template files and trigger each regen endpoint)

NC upgrade is fully automated — no manual occ upgrade step needed after a release that bumps NCVERSION. The rehydrate loop covers NC's tendency to auto-disable apps it thinks are incompatible with a new core version. Service restart triggers (other than commandbox) will be detected in v2 by diffing the working tree against the previous tag.

Phase 5 — Persistent post-upgrade hook

Last step: orchestrator calls the persistent hook.

curl -sf http://localhost:8888/schedule/post_upgrade.cfm

post_upgrade.cfm contains N named migration blocks. Each block:

1. Queries migrations table: has this block run before?
2. If yes: log "skipped (already applied)" and move on.
3. If no: do the work, then INSERT INTO migrations (name) VALUES ('<block-name>').

This hook is for migrations that:

• Don't bind cleanly to a release boundary
• Need to be retroactive across all existing installs regardless of when they upgrade
• Replace existing inline-in-CFML migrations (lazy backfills currently scattered across page handlers)

Examples (some hypothetical):

Block name	What it does
encrypt-relay-credentials-v1	Re-encrypts parameters rows whose values were stored as plaintext before the key-rotation policy landed
move-pgp-keys-to-vault-dir	One-time file move from /opt/hermes/keys/ to /opt/hermes/keys/pgp/ after a refactor
backfill-mailbox-domain-id	Populates a domain_id column added in a prior release where the existing data needs computed values

The admin update procedure

For the period BEFORE system_update_docker.sh ships (#221), updates are manual. After #221 lands, the same procedure collapses into one command.

Today (pre-#221)

cd /opt/hermes-seg-docker-gl
git fetch --tags
git checkout <new-tag>                                   # or: git reset --hard origin/main
docker compose pull
docker compose up -d --build                             # --build rebuilds fail2ban (see Phase 2 note)
./scripts/install_hermes_docker.sh --apply-schema        # SQL deltas
# Manual steps below ONLY if release notes call them out:
docker exec hermes_commandbox occ upgrade                # if NCVERSION bumped
# Service-specific restarts per release notes

install_hermes_docker.sh --apply-schema globs updates/v*/sql/schema_updates.sql and applies any with version newer than current build_no. On a v260119-only install, it logs "No per-release schema_updates.sql files found" and exits cleanly.

After #221

cd /opt/hermes-seg-docker-gl
./scripts/system_update_docker.sh           # latest release
./scripts/system_update_docker.sh v260601   # specific tag

The orchestrator runs all 5 phases. Idempotent + re-runnable. Logs every step. Refuses to run if working tree is dirty.

Pre-upgrade: take a hypervisor snapshot

Before running system_update_docker.sh, take a hypervisor / VM snapshot of the entire Hermes host (Proxmox, VMware, KVM, AWS EBS, Azure Disk, GCE, Hyper-V). If the upgrade fails partway through, reverting the snapshot is the only currently-supported rollback path — Docker-aware backup/restore tooling is in development at #219 and #220 and is not yet shipped. See docs/admin/01-system/backup-restore.md for the full interim-backup guidance.

Post-upgrade: hard-refresh the browser (any release that bumps NCVERSION or admin/2 assets)

After the orchestrator finishes, operators must hard-refresh their browser (Ctrl-Shift-R on Linux/Windows, Cmd-Shift-R on macOS) on any open Hermes admin tab or Nextcloud tab. The browser cache often serves the pre-upgrade CSS/JS bundle even though the server is now serving the new one, which presents as broken layouts — most visibly NC's top navbar collapsing to a vertical stack of icons when the NC 30 → 31 bundle swaps under it.

Release notes that bump NCVERSION or land changes under config/hermes/var/www/html/admin/2/ should call this out explicitly. One sentence in the release notes prevents the "Nextcloud top bar broke" support ticket.

The release-cut procedure (developer side)

For maintainers preparing a release:

1. Code complete on main: all PRs for the release have landed on GitLab main.

2. Create the release directory: mkdir -p updates/v<DATE>/{sql,cfml,scripts} (only the subdirs you actually need). Add updates/v<DATE>/README.md documenting what changed.

3. Add SQL deltas: write updates/v<DATE>/sql/schema_updates.sql. ALSO add equivalent rows/columns to config/database/hermes_install.sql so fresh installs after this release ship with them baked in. Last block of schema_updates.sql must be:

   UPDATE system_settings SET value='v<DATE>' WHERE parameter='build_no';
   UPDATE system_settings SET value='Docker'  WHERE parameter='version_no';
   

   While you are in the baseline, bump its own build_no literal to v<DATE> too. That row is only a floor — install_hermes_docker.sh overwrites it in seed_install_specific_values() from the newest updates/v<YYMMDD>/ directory, precisely so a missed bump here can no longer surface as a wrong version on the dashboard. Keeping it in step still matters for anyone importing the baseline by hand.

   Why the derivation exists. schema_updates.sql advances build_no on upgrades only (--apply-schema); a fresh install imports the baseline and stops. From v260612 through v260723 the baseline literal was never re-bumped, so every fresh install of four consecutive releases reported v260612. Fixed 2026-07-31 (#288).

4. Add CFML / bash migrations as needed under updates/v<DATE>/cfml/ and updates/v<DATE>/scripts/.

5. Update .env.template: bump HERMES_DOCKER_IMG_VERSION=v<DATE> and (if NC bumped) NCVERSION=.... NCVERSION is release-managed per #261 — operators never edit it; bumps land here only after the integration check in step 6 passes.

6. (If NCVERSION was bumped in step 5) Run the NC integration check on a Test box:

   # On Test, after a fresh clone or pull that includes the new .env.template
   docker compose pull hermes_nextcloud
   docker compose up -d hermes_nextcloud
   sleep 30                                    # let NC finish first-start init
   ./scripts/test_nc_integration.sh
   

   The script (read-only occ queries + log scan) verifies the Hermes-NC integration surface: container responsive, occ status reports the expected version, no needsDbUpgrade, required apps enabled and not flagged incompatible (user_oidc / mail / twofactor_totp / twofactor_backupcodes / external), trusted_domains populated, theming URL set, user_oidc provider Hermes_SEG registered, and no ERROR/FATAL entries in the last 200 nextcloud.log lines. Exit code 0 if no FAIL.

   If anything fails, fix the integration (or revert the NCVERSION bump and pin to the prior NC) before continuing. Do not publish a release that ships a failing NC integration.

7. Draft the GitHub Release body for v<DATE>: list every change in the release. Per-release notes live on the GitHub Release page (created when the tag is pushed) — the cumulative RELEASE-NOTES.md was retired because release notes belong to a specific tag, not an ever-growing file.

8. Build + push images at the version tag: ./Docker/build-all-ghcr.sh v<DATE> && ./Docker/push-all-ghcr.sh v<DATE>.

   Both scripts take the version as an argument and prompt for it if omitted. They build and push ghcr.io/deeztek/hermes-<service>:v<DATE> for all 12 services. push-all-ghcr.sh then asks whether to promote what it pushed to :latest.

   Answer no to that prompt. Promote :latest only after the release has been tested (step 10). Until then :latest keeps pointing at the previous release, so a fresh install during the test window gets the last known-good stack, and :v<DATE> gives you something immutable to test against and to roll back to.

   Build all 12 even when only one Dockerfile changed. docker-compose.yml resolves every service through a single ${HERMES_DOCKER_IMG_VERSION}, so pinning is all-or-nothing and a partially-tagged release breaks compose pull. A full --no-cache rebuild also picks up base-image security updates across the stack, which is worth doing every release for mail-facing containers. It does widen the test surface: every container is new, not just the one you changed.

   Relationship to the Release images workflow. .github/workflows/release-images.yml fires on the tag push and copies :latest into :v<DATE> for all 12 images. When step 8 has already published :v<DATE>, the workflow finds the tags present and no-ops, because it refuses to overwrite an existing image tag. That is the intended outcome: the workflow is a backstop for a release cut without these scripts, not the primary mechanism. Release image tags are immutable once published.

   Per-release image tags are what make a release reproducible. Before they existed, ghcr.io carried only :latest, a git tag did not correspond to a known set of containers, --image-version had nothing to point at, and there was no image-level rollback (#288).

9. Tag + push: ./scripts/git_release.sh --release v<DATE> (pushes branch + tag to both GitLab and GitHub). This is what triggers the image-tagging workflow.

   Push the tag at this point, not the GitHub Release. The orchestrator resolves a target by git tag, so the tag has to exist before step 10 can test the upgrade, but nothing reaches customers until the Release is published in step 11. The tag can be force-moved freely while it is unpublished.

10. Test the release: run ./scripts/system_update_docker.sh v<DATE> on the Test box and confirm a clean upgrade, then run a fresh install and confirm it comes up. Exercise the actual mail path, not just container health: authenticate to IMAP as a mailbox user and push one message end to end through Amavis. Container-and-database checks alone missed five first-run defects that shipped for months (#292).

11. Promote :latest and publish: re-run ./Docker/push-all-ghcr.sh v<DATE> and answer yes to the promotion prompt (or docker tag/docker push each image by hand), then create the GitHub Release.

    This is the point of no return, and the reason :latest was held back. Everything before it is reversible: the tag can move, and no install pulls a new image until :latest does.

12. Verify: GitHub Release page exists, ghcr.io shows both :v<DATE> and an updated :latest, and the console reports the new version after an upgrade.

Common scenarios

"I'm adding a new feature that needs a new table."

1. Add CREATE TABLE IF NOT EXISTS ... to config/database/hermes_install.sql (baseline gets it for fresh installs).
2. Add the same CREATE TABLE IF NOT EXISTS ... to the in-flight release's updates/v<DATE>/sql/schema_updates.sql (so existing installs get it on upgrade).
3. Add seed rows the table needs in BOTH files following the same pattern.

"I'm adding a new system_settings row with a default value."

Same as above: baseline + per-release. Use auto-assigned IDs (omit the id column); UNIQUE KEY on parameter handles dedup. See [[feedback-no-hardcoded-row-ids-on-new-seeds]] in memory.

"I'm changing a default value an existing install might have customized."

UPDATE ... WHERE value = '<old-default>' only in updates/v<DATE>/sql/schema_updates.sql. The WHERE clause preserves admin customizations. Also update hermes_install.sql baseline to the new default for fresh installs.

"I rotated an encryption key and need to re-encrypt rows."

Per-release updates/v<DATE>/cfml/reencrypt-foo.cfm. Pure SQL can't reach the encryption key. Inside the script, check whether each row is already at the new format before re-encrypting.

"I need to move a config file from /old/path to /new/path on the host."

Per-release updates/v<DATE>/scripts/move-config.sh. Precondition check: if /new/path exists, exit 0.

"I just realized all installs need a one-time backfill but I'm not cutting a release for it."

Add a named block to schedule/post_upgrade.cfm. It runs at the next upgrade (or at every upgrade, gated by migrations table).

"Fresh install at v260119, what runs?"

Just config/database/hermes_install.sql. The baseline is self-contained. updates/v260119/ has no sql/ subdirectory by design. --apply-schema finds nothing to apply and reports current build_no=v260119.

"Upgrading from v260119 to v260601 (hypothetical), what runs?"

In order:

1. git checkout v260601
2. docker compose pull && docker compose up -d --build
3. updates/v260601/sql/schema_updates.sql (advances build_no to v260601 at its end)
4. updates/v260601/cfml/*.cfm (each)
5. updates/v260601/scripts/*.sh (each)
6. Standard finalize (restarts, occ upgrade, template regen)
7. schedule/post_upgrade.cfm (any unapplied blocks)

"Upgrading from v260119 to v260801, skipping v260601 (hypothetical)."

The orchestrator applies BOTH release directories in order:

1. Phase 1-2 once
2. Phase 3 walks v260601/ first (sql → cfml → scripts), then v260801/
3. Phase 4-5 once at end

build_no advances after each release's sql/ completes — v260601 then v260801. Each release's artifacts are idempotent so the in-between cursor advancement is safe.

State of the methodology as of v260119

Piece	Status
Baseline (hermes_install.sql)	Self-contained at v260119 (audit completed 2026-05-26)
Per-release directory (updates/v260119/)	Empty by design — v260119 IS the baseline; first real per-release directory is the NEXT release
install_hermes_docker.sh --init-db	Imports baseline only; no schema_updates.sql call. Stamps build_no from the newest updates/v<YYMMDD>/ dir and renders the Ofelia schedule from ofelia_jobs (both added 2026-07-31, #288).
install_hermes_docker.sh --apply-schema	Globs updates/v*/sql/schema_updates.sql, applies in version order; no-op on v260119-only
scripts/system_update_docker.sh (#221)	MVP shipped 2026-05-26 — phases 1-5 functional; some v2 polish deferred (see "Known MVP limitations" below)
schedule/post_upgrade.cfm framework	Stub shipped 2026-05-26 — framework + helpers + migrations table in place, zero blocks registered (first one lands when first migration is needed)
migrations table	Added to baseline 2026-05-26
GitHub Actions release workflow	NOT YET BUILT — image pushes are manual via Docker/push-all-ghcr.sh
Image registry (ghcr.io)	Empty as of 2026-05-25; bootstrap is a pre-Session B task

Known MVP limitations of system_update_docker.sh

The MVP that shipped 2026-05-26 is deliberately scoped. These limitations are tracked for v2:

Limitation	What happens today	v2 plan
Service-restart detection	Always restarts hermes_commandbox in Phase 4; logs a reminder that admin may need to re-save certain config pages	Diff config files between previous and new tag; only restart containers whose volume-mounted config changed
*.HERMES template re-render	Phase 4 logs a reminder; admin must re-save the corresponding admin page manually	Detect modified template files and invoke each one's regen endpoint via curl into commandbox
occ upgrade	Phase 4 auto-detects NCVERSION drift, runs occ upgrade, rehydrates Hermes-required NC apps, verifies post-upgrade state (#264)	— already MVP-complete —
updates/v<DATE>/cfml/*.cfm artifacts	Phase 3 logs a WARN and skips them	First release that ships a CFML migration must add a host→container mount for updates/ plus a Lucee mapping; the orchestrator already curls the right URL pattern, just needs the mount to exist
Mid-upgrade resume	set -e fail-fast; operator re-runs from scratch and idempotency handles re-application	Track per-phase + per-release-artifact completion; resume from last successful step
GitHub Releases API auth	Anonymous polling; subject to GitHub's unauthenticated rate limit (60 req/hr per IP)	Honor a GITHUB_TOKEN env var if present

Storage Topology (5 tiers)

Storage Topology (5 tiers)

Hermes SEG splits storage into five independent tiers so each can live on the right kind of disk for its workload. Four are operator-chosen at install time; the fifth (Config) is implicit — chosen by where the operator git cloned the repo.

Tier	Default path	Contents	Storage profile
1. Config	install root (implicit)	Repo working tree, config/ subtrees, install script, secrets in config/hermes/opt/hermes/keys/, .env, .hermes_install_config	Fast SSD, modest size — chosen by where the repo lives
2. Data	/mnt/data (DATA_MOUNT)	DBs (MariaDB, Authelia, OpenLDAP), Amavis runtime state, ClamAV signatures, Fangfrisch state, Lucee server home, sieve scripts, all service logs, OpenDMARC, Postfix queue	Fast SSD; sized for DB growth + log retention
3. Archive	/mnt/archive (ARCHIVE_MOUNT) — added in #260	Amavis quarantine archive	Cheap bulk; sized for retention policy × quarantine inflow
4. Vmail	/mnt/vmail (VMAIL_MOUNT)	Dovecot mailboxes	Cheap bulk; sized for users × quota
5. Nextcloud	/mnt/files (FILES_MOUNT)	Nextcloud app + user files + Nextcloud's Redis cache	Cheap bulk; sized for user file storage

Each tier is one host path; the install script lays out the canonical sub-directory structure underneath it.

Why split storage

Tier	Why it gets its own disk
Config	Frequent reads (every container start); small footprint; lives with the install script + version control
Data	High write rate (logs + DBs); benefits from fast SSD; backup hot spot
Archive	Grows unboundedly with retention policy; cold reads (admin browses quarantine occasionally); cheaper bulk storage; backup cadence independent of Data
Vmail	Grows linearly with user count × quota; cheaper bulk storage; separate backup cadence (often less frequent than Data)
Nextcloud	Same growth characteristics as Vmail but different access pattern; often shared across multiple Hermes installs in larger deployments

Smaller deployments can collapse tiers — point Archive, Vmail, and Nextcloud at the same path as Data for a single-disk install. Each tier is its own prompt so the operator picks per workload.

Canonical sub-directory layout

Tier 2 — Data (default /mnt/data/)

Sub-path	Named volume	Service
dbase/	db_data	MariaDB
authelia/db/	authelia_db	Authelia state DB
authelia/logs/	authelia_logs	Authelia logs
authelia/redis/	authelia_redis	Authelia Redis
commandbox/serverhome/	commandbox_serverhome	Lucee server home
dmarc/logs/	dmarc_logs	OpenDMARC logs
dovecot/logs/	dovecot_logs	Dovecot service logs
dovecot/sieve/	dovecot_sieve	Sieve scripts (shared by commandbox + dovecot)
ldap/data/	ldap_data	OpenLDAP data
ldap/logs/	ldap_logs	OpenLDAP logs
mail_filter/data/amavis/	mail_filter_data_amavis	Amavis runtime state (PID files, scan tmp dirs — small, latency-sensitive)
mail_filter/data/clamav/	mail_filter_data_clamav	ClamAV signatures
mail_filter/data/fangfrisch/	mail_filter_data_fangfrisch	Fangfrisch state
mail_filter/logs/	mail_filter_logs	Mail filter logs
nginx/logs/	nginx_logs	Nginx logs
openarc/logs/	openarc_logs	OpenARC logs
postfix_dkim/logs/	postfix_dkim_logs	Postfix logs
postfix_dkim/queue/	postfix_dkim_queue	Postfix mail queue

Tier 3 — Archive (default /mnt/archive/) — added in #260

Sub-path	Named volume	Service
amavis/	amavis_data	Amavis quarantine archive (admin-browsable, grows with retention)

Note: the Amavis runtime state (mail_filter/data/amavis/ named volume mail_filter_data_amavis) stays on the Data tier — it's small, doesn't grow with retention, and benefits from fast SSD latency. Only the quarantine archive moved.

Tier 4 — Vmail (default /mnt/vmail/)

Sub-path	Named volume	Service
dovecot/	dovecot_mail	Dovecot mailboxes

Tier 5 — Nextcloud (default /mnt/files/)

Sub-path	Named volume	Service
app/	nextcloud	NC app + user files
redis/	nextcloud_redis	NC's Redis cache

How it works at install time

1. prompt_mount_points() asks the operator for four paths (Data / Archive / Vmail / Nextcloud) — Config is already chosen by where the repo lives. Defaults /mnt/data, /mnt/archive, /mnt/vmail, /mnt/files. Choices saved to .hermes_install_config at the install root.

2. provision_mount_dirs() creates the entire sub-directory layout under each chosen path with the correct UID/GID for the containers that will write to them. Critical: bind-mounted volumes (type: none, o: bind in docker-compose.yml) require the source directory to pre-exist — Docker refuses to start the container otherwise.

3. generate_compose_override() writes the four mount-point variables (DATA_MOUNT / ARCHIVE_MOUNT / VMAIL_MOUNT / FILES_MOUNT) to .env at the install root. docker-compose.yml references these variables directly in its device: lines (e.g. device: ${ARCHIVE_MOUNT}/amavis) — Docker Compose substitutes at runtime. The legacy docker-compose.override.yml approach was retired in #179; the function name was kept for backwards-compatibility with the --generate-override CLI flag.

4. All four mount points are required. Empty values would resolve to dangerous relative paths during device: substitution (e.g. empty ${ARCHIVE_MOUNT}/amavis → /amavis at the host root). For single-disk installs, point all four prompts at the same path.

Self-locating scripts

install_hermes_docker.sh, rotate_db_credentials.sh, and any other Hermes script needing the install root use a walk-up self-locator pattern that finds docker-compose.yml by walking up from BASH_SOURCE[0]:

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [[ -z "${HERMES_ROOT:-}" ]]; then
    HERMES_ROOT="$SCRIPT_DIR"
    while [[ "$HERMES_ROOT" != "/" ]] && [[ ! -f "$HERMES_ROOT/docker-compose.yml" ]]; do
        HERMES_ROOT="$(dirname "$HERMES_ROOT")"
    done
    if [[ "$HERMES_ROOT" == "/" ]]; then
        echo "ERROR: Could not locate docker-compose.yml in any parent of $SCRIPT_DIR" >&2
        echo "Set HERMES_ROOT environment variable manually and retry." >&2
        exit 1
    fi
fi

This is depth-independent (works at 1 level or 5 levels deep in the tree) and survives the script being relocated. Do not use a hardcoded dirname/.. chain — it depends on the script's exact depth and breaks silently if the script moves.

Reading topology at runtime

.hermes_install_config is the source of truth for which paths the operator chose. Scripts that need this (system_backup.sh, system_restore.sh) source the file via the load_config() helper. Format:

DATA_MOUNT=/mnt/data
ARCHIVE_MOUNT=/mnt/archive
VMAIL_MOUNT=/mnt/vmail
FILES_MOUNT=/mnt/files
ENABLE_NEXTCLOUD=true

The file lives in the Config tier (install root), so it's part of every Config-tier backup automatically.

Post-Restore Steps

Post-Restore Steps

Run after scripts/system_restore.sh completes. The restore replaces databases, LDAP, and the storage tiers, but a few things must be reconciled by hand depending on whether you restored onto the same host or a different host (cross-host disaster recovery). The restore script prints a pointer to this page at the end of its run.

Same-host restore

You restored a backup onto the same machine it came from (e.g. rolling back).

1. Log into the admin console — https://<console-host>/admin/.
2. Verify mail flow — send and receive a test message.
3. Check the log — install-logs/system_restore_*.log for any WARN lines.

Nothing else is usually required: .env, creds/, and host identity already match this machine.

Cross-host restore (restore to new hardware)

You restored a backup taken on host A onto a different host B (DR to new hardware, migration, etc.). The restored data carries host A's identity and credentials, so B serves A's configuration until you reconcile it. Do these in order:

1. Rewire host identity — system_rehost.sh (REQUIRED)

The restored .env, parameters2 rows, nginx vhost, Authelia cookie domain, and Postfix hostname all reference host A. Until you rewire them, the console is unreachable at host B's address.

sudo /opt/hermes-seg/scripts/system_rehost.sh

It auto-detects this host's IP/hostname (pass --to-hostname= / --to-ip= to override) and rewrites identity across .env, the database, and the rendered configs. See system_rehost.sh --help. The restore script prints this reminder automatically when it detects a host-identity mismatch.

Note: system_rehost.sh also reconciles cross-host DB credentials for Nextcloud (its config.php rides in the restored data tier). All other services use install-generated configs that hold this host's credentials.

2. Re-activate the Pro license

Hermes Pro licenses are bound to a hardware-derived UUID. On new hardware the UUID changes, so the license reads INVALID / VIOLATION on the next validation and the console drops to Community Edition.

• Re-activate the license, or contact Hermes support to re-bind your serial to this host.
• Community Edition installs are unaffected.

3. Re-save Content Checks (re-apply the milter chain)

smtpd_milters (the DKIM / DMARC / ARC / SPF milter chain) is not restored — it is dynamic config applied by the Content Checks settings pages, not part of the restored main.cf. After a cross-host restore the database still records which checks are enabled, but the live Postfix config does not reflect them, so outbound mail is not DKIM-signed and inbound is not DMARC/ARC-checked until you re-apply them.

For each enabled Content Checks page, open it in the admin console and click Save (no changes needed) to re-apply its Postfix config:

• Content Checks → DKIM
• Content Checks → DMARC
• Content Checks → SPF
• Content Checks → ARC (if used)

Verify afterward:

docker exec hermes_postfix_dkim postconf smtpd_milters

It should list the milter sockets (e.g. inet:hermes_dmarc:54321, OpenDKIM, OpenARC, body_milter), not be empty.

This re-save step is an interim workaround. Automatic re-application of the milter chain on restore is tracked as a follow-up enhancement (#268).

4. Clear Nextcloud maintenance mode (if needed)

The restore clears it automatically, but if Nextcloud still shows maintenance mode:

docker exec -u www-data hermes_nextcloud php occ maintenance:mode --off

5. Verify

• Admin console login at host B's address.
• Mail flow (send + receive).
• Nextcloud login.
• docker exec hermes_postfix_dkim postconf smtpd_milters is populated (step 3).

Health check

After either path, run the smoke test to confirm container health, per-service DB authentication, and the mail chain:

bash /opt/hermes-seg/scripts/hermes_smoke_test.sh

A WARN on smtpd_milters is empty after a cross-host restore is the symptom that step 3 has not been completed yet.

Post-Migration Checklist: Legacy → Docker

Post-Migration Checklist: Legacy → Docker

Run these steps after scripts/migrate_legacy_to_docker.sh completes. The migration restores the databases and portable data (encryption key, TLS certs, DKIM signing keys), migrates admin users to LDAP, and — at the end of its run — invokes system_rehost.sh to rewire the host identity and re-render the Docker-managed service configs. This checklist covers what remains.

Scope: validated for migrating from legacy build 240815 (relay topology) to the current Docker release. The migration script rejects any other source build.

---

⚠️ Outbound email is PAUSED until you release it

The migration deliberately brings the gateway up with all outbound delivery held. A freshly migrated box has the restored quarantine and its notification schedule live — left alone, it will blast stale quarantine notifications to every real recipient in the restored data (this is not hypothetical; it will happen). So nothing leaves the box: outgoing mail parks in the queue.

Work through this checklist, then release outbound as the very last step — after you've reviewed the queue and deleted anything you don't want sent. Inbound mail and filtering run normally the whole time; only egress is held.

---

What the migration already did for you

• Restored hermes (schema-forwarded to the current baseline), djigzo, opendmarc, Syslog; created authelia and nextcloud.
• Copied portable data: hermes.key (DB-secret encryption key), Let's Encrypt certs, DKIM signing keys.
• Migrated Authelia users into LDAP as one-factor admins.
• Ran system_rehost.sh, which rewired host identity (server_ip, console.host, Postfix myhostname/myorigin, nginx server_name, Authelia cookie domain) and re-rendered postfix, amavis, authelia, nginx, and mailname from the restored database, then restarted those containers.

Settings Postfix reads through MySQL maps — Domains, Relay Recipients, Virtual Recipients, sender/recipient rules — are live directly from the restored database and need no action.

If you skipped host rewiring during the migration, run it now: sudo scripts/system_rehost.sh --force (auto-detects this host's IP/hostname; override with --to-hostname= / --to-ip= if needed).

---

1. Log in and sanity-check

1. Log into the admin console (see step 4 if all admins show as one-factor).
2. Sidebar → Domains and Relay Recipients — confirm the migrated data.
3. Sidebar → System Status — confirm all containers are healthy.

2. Regenerate mail authentication (DKIM, SPF, DMARC)

system_rehost.sh does not cover these. Each regenerates its own service config from the database; their TrustedHosts lists must carry the Docker subnet. The DKIM signing keys are already migrated (opt/hermes/dkim/keys).

All three live under the Content Checks section of the sidebar.

1. Content Checks → DKIM Settings → Save & Apply Settings — regenerates OpenDKIM KeyTable / SigningTable / TrustedHosts referencing the migrated keys. Confirm each domain's published DNS TXT record still matches.
2. Content Checks → SPF Settings → Save & Apply Settings — regenerates policyd-spf.
3. Content Checks → DMARC Settings → Save & Apply Settings — regenerates OpenDMARC.

3. Verify Relay Networks (mynetworks)

1. Sidebar → Relay Networks — confirm the customer's allowed sending networks carried over and the Docker subnet is present; Save & Apply if you change anything.
2. Sidebar → Relay Host — verify the upstream smarthost and credentials; re-enter the relay password if the field is blank.

4. Reconfigure admin two-factor authentication

Every migrated admin is in the one-factor group; legacy TOTP / WebAuthn devices were not carried over (Authelia moved SQLite → MySQL).

1. Verify each admin can log in with their existing (legacy) password.
2. Sidebar → System Users — move each admin who should use 2FA to the two_factor group; have them re-enroll an authenticator on next login.

Important: review every admin account — all are password-only until you do this.

5. Reactivate the Pro license (Pro editions)

The Pro license is bound to the server's hardware UUID, which changed on the new Docker host, so it shows invalid / pending.

1. Email support with the license serial number to request reactivation.
2. Support deactivates the old activation so it can re-bind to the new host.
3. Reactivate, then log out and back in to re-validate.

Important: until reactivated, the system runs as Community Edition.

6. Verify TLS certificates

The Let's Encrypt certificates were copied to the Docker cert path.

1. Sidebar → System Certificates — confirm the console and mail certificates are present and selected (not the bootstrap_hermes.pem placeholder).
2. Confirm HTTPS (console) and SMTP STARTTLS work.

7. Email archive (deferred)

Message History metadata is restored, but message bodies live in the separate archive backup (hermes-archive-<build>-…tar.gz). Restore it when convenient — it needs no gateway downtime.

• Sidebar → Content Checks → Message History — entries appear; bodies are viewable only after the archive is restored.

8. Final end-to-end test

1. Send a test message inbound and outbound.
2. Verify: DKIM = pass, spam scoring, and TLS on both legs.

Outbound is still paused at this point, so an outbound test message parks in the queue rather than delivering. That's expected — you confirm actual outbound delivery in step 9 after releasing.

9. Release outbound delivery

This is the last step. Do it only after everything above checks out.

While migrating, outbound has been held so the box could not mail real recipients (see the notice at the top). Now release it — deliberately, after reviewing what's queued.

1. Sidebar → Mail Queue. At the top, the Outbound Delivery card shows PAUSED.
2. Review the queued mail. Expect a large batch of quarantine-notification messages (from postmaster@…) generated from the restored quarantine. Select and delete those — they are stale and should not go out. Keep any genuine mail you want delivered.
3. Click Resume Outbound Delivery. This clears the hold and flushes the queue; the Outbound Delivery card flips to ACTIVE.
4. Confirm the outbound test from step 8 now delivers, and watch the mail log to be sure only the mail you kept goes out.

The Outbound Delivery control (Mail Queue page) is a permanent pause/resume switch — useful for maintenance windows too. Pausing holds all outgoing mail in the queue without affecting inbound or filtering.

Hermes SEG Email Flow

Hermes SEG Email Flow

Reference diagram for the inbound, outbound, and CipherMail-originated mail paths inside the Docker stack. Includes every listening port, every container, the milter chain at each smtpd service, and where body modifications occur relative to DKIM signing.

---

Container + port map (at a glance)

Container	Service / Daemon	Port(s)	Role
hermes_postfix_dkim	postfix smtpd (:25)	25	Inbound MX
hermes_postfix_dkim	postfix smtpd (submission)	587	Authenticated outbound (STARTTLS)
hermes_postfix_dkim	postfix smtpd (smtps)	465	Authenticated outbound (implicit TLS)
hermes_postfix_dkim	postfix smtpd (:10026)	10026	Re-injection (post-CipherMail)
hermes_postfix_dkim	postfix smtpd (:10027)	10027	CipherMail web-GUI originated mail
hermes_postfix_dkim	OpenDKIM primary (sv mode)	8891	Verify inbound / sign outbound at :25/:587/:465
hermes_postfix_dkim	OpenDKIM sign-only (s mode) — #232	8892	Sign post-CipherMail egress at :10026 only
hermes_mail_filter	amavisd-new (filter)	10021	SpamAssassin + ClamAV + policy
hermes_mail_filter	amavisd-new (pickup / bypass)	10030	BYPASSALLCHECKS lane
hermes_body_milter	Python pymilter (#214/#226/#228/#230)	8893	Disclaimer + signature + banner + CID inline
hermes_dmarc	OpenDMARC	54321	DMARC verify / SPF alignment header
hermes_openarc	OpenARC (#229, flowerysong v1.3.0 built from source)	8893	ARC sealing at :10026 only — RFC 8617 chain preservation across body mods
hermes_ciphermail	CipherMail SMTP	25	Encryption decisions + MIME rebuild
hermes_dovecot	LMTP	24	Local mailbox delivery

Milter listening side: the OpenDKIM/OpenDMARC/body_milter/OpenARC daemons listen on TCP and postfix smtpd connects to them per the smtpd_milters line in effect for each port.

Note: hermes_body_milter and hermes_openarc both listen on internal port 8893, but they are separate containers with their own network namespaces. Postfix reaches each by container name (inet:hermes_body_milter:8893 vs inet:hermes_openarc:8893), so the shared port number causes no conflict.

---

Milter chains by smtpd port

The smtpd_milters chain order is set globally in main.cf (built from the parameters DB table by generate_postfix_configuration.cfm) and overridden per-service in master.cf for :10026 and :10027.

smtpd port	Milter chain (in order)	Why
:25 (inbound)	OpenDKIM :8891 → OpenDMARC :54321 → body_milter :8893	Verify DKIM, verify DMARC, then inject External Banner / disclaimer
:587 :465 (submission)	OpenDKIM :8891 → OpenDMARC :54321 → body_milter :8893	Sign DKIM first (before body mods), then disclaimer/signature inject — wrong order; see #232 outbound fix history
:10026 (re-inject)	OpenDKIM :8892 sign-only → OpenARC :8893 (hermes_openarc)	Re-sign body that CipherMail mutated, then ARC-seal the final form so downstream verifiers trust the cumulative auth chain even after body modification (#229)
:10027 (CipherMail GUI)	OpenDKIM :8891	Sign GUI-originated mail; no body mods on this path

Why OpenARC sits ONLY at :10026 (and NOT in main.cf)

OpenARC's ARC-Message-Signature includes a hash of the message body. If ARC sealed at :25, the body would later be mutated by body_milter (:8893) and CipherMail (MIME rebuild), so the seal's body hash would be invalid by the time downstream verifiers received the message — cv=fail.

:10026 is the only point where the body is in its final form (all body modifications + CipherMail MIME rebuild complete), so it's the only correct hop to apply the ARC seal. Adding ARC to main.cf's default smtpd_milters would cause two problems:

1. Pre-modification sealing at :25 → broken seal at the recipient.
2. Double-sealing: mail going through :25 → amavis → :10026 would be sealed twice by the same gateway (i=1 at :25, i=2 at :10026), producing redundant chain bloat / verification ambiguity.

ARC stays out of main.cf deliberately. Master.cf :10026 override is the single point of truth.

The body_milter ordering is recorded in parameters as order1=3.1 so it sits AFTER OpenDKIM signer (0.5) and OpenDMARC. The retro-fix UPDATE in updates/hermes-260119/sql/schema_updates.sql corrects existing installs that had 0.5 for body_milter (which placed body mods BEFORE signing, the root cause of #232 outbound DKIM failures).

---

Inbound flow

External MTA → local mailbox.

                  ┌─────────────────────────────────────────────────────────────────────┐
                  │ External Internet                                                   │
                  └─────────────────────────────────────────────────────────────────────┘
                                                  │
                                                  │ SMTP / DKIM-signed by sender
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_postfix_dkim  ▸  :25  (postfix smtpd, inbound MX)                    │
            │   ▸ smtpd_milters chain:                                                    │
            │       1. OpenDKIM primary  inet:127.0.0.1:8891   (verify sender's DKIM)    │
            │       2. OpenDMARC         inet:hermes_dmarc:54321 (verify DMARC alignment)│
            │       3. body_milter       inet:hermes_body_milter:8893 (External Banner)  │
            │   ▸ content_filter = amavis:[hermes_mail_filter]:10021                     │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  smtp
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_mail_filter   ▸  :10021  (amavisd-new — main filter lane)           │
            │   ▸ SpamAssassin scoring     (sees body, computes its own DKIM_INVALID     │
            │                               since body_milter already injected banner)   │
            │   ▸ ClamAV virus scan                                                       │
            │   ▸ Policy / quarantine decisions                                           │
            │   ▸ $forward_method = smtp:hermes_ciphermail:25                            │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  smtp
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_ciphermail    ▸  :25     (encryption gateway)                        │
            │   ▸ Encryption-mode decisions                                               │
            │   ▸ MIME rebuild  (always — not byte-stable across this hop)               │
            │   ▸ Re-injects to → smtp:hermes_postfix_dkim:10026                         │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  smtp
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_postfix_dkim  ▸  :10026  (postfix smtpd, re-injection)               │
            │   ▸ smtpd_milters = inet:localhost:8892,inet:hermes_openarc:8893            │
            │       1. OpenDKIM sign-only (s mode)  #232                                  │
            │       2. OpenARC seal                  #229                                 │
            │                                                                             │
            │   For INBOUND mail, the From: domain is NOT in the local KeyTable.         │
            │   → sign-only instance does nothing (no key match → no header added)       │
            │   → critically: it also does NOT verify, so no Authentication-Results      │
            │     "dkim=fail" header gets written against the body-modified message.     │
            │                                                                             │
            │   OpenARC then seals the FINAL body, recording the A-R header that         │
            │   OpenDKIM-primary + OpenDMARC wrote back at :25 (which still says         │
            │   "dkim=pass" against the unmodified original).                            │
            │                                                                             │
            │   ⚠ Chain-integrity caveat (#229): when the inbound message ALREADY        │
            │     carried an upstream ARC-Seal (M365 / Workspace / Mimecast /            │
            │     Proofpoint / Exclaimer) and Hermes modified the body at :25            │
            │     (banner injection), OpenARC at :10026 writes cv=fail at i=2.           │
            │     The upstream i=1 body hash no longer matches. To prevent this for      │
            │     relay-out recipients, body_milter automatically skips banner           │
            │     injection in that narrow case — see "Conditional banner skip" below.  │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  transport_maps lookup
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_dovecot       ▸  :24  (LMTP)                                         │
            │   ▸ Sieve filtering (vacation, file-into; redirect is same-domain only)    │
            │   ▸ Local mailbox delivery → /mnt/data/vmail                                │
            └─────────────────────────────────────────────────────────────────────────────┘

Inbound paths that diverge

• Relay-only domains (in transport_maps) skip Dovecot and smtp:-forward to the next-hop MX listed in /etc/postfix/transport.
• Quarantined / virus / bad-header mail is held by amavis at :10021 (final destinies: D_BOUNCE virus/banned, D_DISCARD spam/bad_header per 50-user config).
• Whitelisted senders bypass via BYPASSALLCHECKS at :10030.

Architectural principle: Hermes is the auth boundary (#229)

Hermes is the authoritative auth / security boundary for every domain it relays for. Inbound auth checks (DKIM, SPF, DMARC, ARC verify, spam, virus) happen at Hermes. Body modifications (External Sender Banner, disclaimer, signature insertion, encryption) also happen at Hermes. Customer downstream mail servers (the relay-target MX) must be configured to trust Hermes implicitly: allowlist Hermes by IP or hostname, accept forwarded mail without re-running DKIM / SPF / DMARC / ARC checks. This is the same deployment model Mimecast, Proofpoint, and Barracuda customers use — the SEG IS the trust boundary.

When the inbound message arrives carrying an upstream ARC-Seal: header (M365, Workspace, Mimecast, Proofpoint, Exclaimer, etc.) and Hermes modifies the body (banner, disclaimer), the upstream chain's body hash is invalidated. OpenARC at :10026 honestly records cv=fail on Hermes's own seal because it can no longer validate the upstream chain against the modified body. The original sender's DKIM-Signature body hash is also invalidated.

This is by design and is not a Hermes problem. A correctly-configured customer downstream MX is allowlisting Hermes and not re-checking auth on forwarded mail; the cv=fail and broken DKIM signal never gates delivery. If a customer's downstream MX is doing redundant auth checks on mail Hermes forwards, that's a misconfiguration on the customer's end — the fix is to allowlist Hermes there, not to silence Hermes here.

For external receivers Hermes does NOT have a trust relationship with (third-party MXes encountered via sieve redirect or alias forwarding, should those leak past the same-domain validation), the cv=fail and broken DKIM signals do reach a non-trusting receiver. That's why sieve redirects from the user portal are validated to require a same-domain target (see inc/sieve_user_rule_actions.cfm) — keeping forwarded mail inside Hermes's auth boundary. Aliases configured by admins are constrained to internal Hermes mailboxes by the existing CFML check in inc/add_mailbox_alias_action.cfm.

Why not lift the chain by stripping the upstream ARC?

cv=fail is honest — Hermes correctly admits the chain we received no longer body-validates against the message we're about to send. The verifier walks the chain backward and recomputes the upstream i=1 body hash against the current body, so stripping our i=2 admission does not repair anything. The only mechanisms that could restore trust for body-modifying gateways are:

1. Receiver-side trust configuration — Microsoft 365's "Trusted ARC Sealers" feature, Gmail's internal trust list, etc. Useful when forwarding to receivers OTHER than the customer's own MX. See the Trusted ARC Sealers — M365 guide for cross-org scenarios.
2. Don't modify the body — defeats the purpose of having body modification features.

Multi-instance OpenARC (separate verify-only + sign-only daemons) does not help: OpenARC v1.3.0's sign-only mode re-validates the chain at sealing time and ignores AAR cached by the verify instance. This was empirically tested on DEV on 2026-05-14; see the closed #229 discussion.

---

Outbound flow

Authenticated local user → external recipient.

            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ Mail Client (Outlook / Thunderbird / iOS Mail / Roundcube / NC Mail)        │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  SMTP submission
                                                  │  (SASL AUTH via Dovecot SASL :9587)
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_postfix_dkim  ▸  :587 (submission)  OR  :465 (smtps)                 │
            │   ▸ smtpd_milters chain (same as :25):                                      │
            │       1. OpenDKIM primary :8891   ◀─── signs DKIM here                     │
            │       2. OpenDMARC :54321         (record verify; outbound mostly noop)     │
            │       3. body_milter :8893        ◀─── injects disclaimer/signature        │
            │   ▸ content_filter = amavis:[hermes_mail_filter]:10021                     │
            │                                                                             │
            │   ⚠ #232 historical bug: with body_milter at order1=0.5 (before OpenDKIM),  │
            │     body_milter ran BEFORE signing, so DKIM signed the pre-modified body.   │
            │     Fixed by raising body_milter to order1=3.1 (after OpenDKIM signer).     │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  smtp
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_mail_filter   ▸  :10021  (amavisd-new)                              │
            │   ▸ MYNETS policy_bank (originating=1, higher spam_kill threshold)         │
            │   ▸ Virus / banned-file scan                                                │
            │   ▸ $forward_method = smtp:hermes_ciphermail:25                            │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  smtp
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_ciphermail    ▸  :25     (S/MIME / PGP encryption)                   │
            │   ▸ Encryption-mode lookup per recipient                                    │
            │   ▸ MIME rebuild (BREAKS the original DKIM bh= hash — receipt below)        │
            │   ▸ Re-injects to → smtp:hermes_postfix_dkim:10026                         │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  smtp
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_postfix_dkim  ▸  :10026  (re-injection)                              │
            │   ▸ smtpd_milters = inet:localhost:8892,inet:hermes_openarc:8893            │
            │       1. OpenDKIM sign-only (s mode)  #232                                  │
            │       2. OpenARC seal                  #229                                 │
            │                                                                             │
            │   For OUTBOUND mail, the From: domain IS in the local KeyTable.            │
            │   → sign-only instance signs the post-CipherMail body.                     │
            │   → fresh DKIM header replaces (or oversigns) the stale one from :587.     │
            │                                                                             │
            │   OpenARC then seals the post-DKIM body, attaching ARC-Seal / ARC-Message- │
            │   Signature / ARC-Authentication-Results headers. For outbound traffic     │
            │   originating from a relay user whose own MTA pre-signed DKIM, the ARC     │
            │   seal lets downstream verifiers trust the chain even though our body     │
            │   modification (disclaimer etc.) invalidated the relay's original DKIM.    │
            │                                                                             │
            │   ⚠ Historical bug: master.cf had `no_milters` in receive_override_options │
            │     at :10026 → suppressed all milters → no re-sign → Gmail rejected.      │
            │     Fixed by removing `no_milters` token (commit 7014285).                  │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  smtp (smtp_milters chain on egress)
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ External MX (recipient gateway)                                             │
            │   → DKIM verify against `From:` domain key (DNS TXT)                        │
            │   → DMARC alignment                                                          │
            │   → Deliver                                                                 │
            └─────────────────────────────────────────────────────────────────────────────┘

---

CipherMail web-GUI originated mail

When admins send mail directly from CipherMail's web GUI (rare), it enters postfix at :10027, bypasses amavis content filtering entirely, and is signed by the primary OpenDKIM (:8891) — not the sign-only instance — because this path doesn't traverse any body-modification hop and the standard sv-mode instance is appropriate.

            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ CipherMail web GUI (admin compose)                                          │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  smtp
                                                  ▼
            ┌─────────────────────────────────────────────────────────────────────────────┐
            │ hermes_postfix_dkim  ▸  :10027                                              │
            │   ▸ smtpd_milters = inet:localhost:8891  (OpenDKIM primary, sv mode)        │
            │   ▸ No content_filter — bypasses amavis                                     │
            └─────────────────────────────────────────────────────────────────────────────┘
                                                  │  smtp egress
                                                  ▼
                                    External MX (recipient gateway)

---

Why two OpenDKIM instances? (#232 architecture decision)

A single OpenDKIM instance in sv mode (verify + sign) cannot satisfy both requirements at :10026:

Requirement	Default sv instance at :8891	Sign-only instance at :8892
Verify inbound at :25	✅ does this	❌ wouldn't (correctly)
Sign outbound at :587/:465	✅ does this	❌ wouldn't (correctly)
Sign outbound re-inject at :10026 (post-CipherMail body rebuild)	✅ would sign	✅ signs
Skip verify on inbound re-inject at :10026 (post-body-milter banner)	❌ verifies → dkim=fail Authentication-Results	✅ never verifies

OpenDKIM has no per-port mode override, and InternalHosts / ExternalIgnoreList control signing-vs-verification only by sender IP — not by smtpd inet service. Per OpenDKIM's own project guidance, running a second daemon with a different config file is the supported way to get differential behavior on a single host.

The sign-only instance ignores inbound mail naturally: its KeyTable and SigningTable only contain entries for local domains, so inbound mail (where the From: domain matches no local key) is a no-op pass-through — it neither signs nor adds Authentication-Results.

---

Receipts (forensic proof captured during #232 diagnosis)

Symptom	Evidence	Root cause
Gmail rejected outbound from alice@domain.tld	bounced .eml had bh=z0Xb... in DKIM header but body hashed to bh=vJCS...	CipherMail MIME rebuild after :587 DKIM sign; :10026 had no_milters so no re-sign
Inbound dkim=fail only when External Banner enabled	Same gmail→alice test: banner-on dkim=fail, banner-off dkim=pass. CipherMail held constant.	body_milter modifying body before the next milter hop saw it; primary OpenDKIM at :10026 re-verified the modified body
Spam score ~+1.3 on banner-injected inbound	DKIM_INVALID=0.1 + NML_ADSP_CUSTOM_MED=0.9 when banner on; DKIM_VALID*=-0.3 when banner off	amavis runs its own SpamAssassin DKIM check on the post-body-milter body. Not fixed by multi-instance OpenDKIM — separate problem
Downstream forwarder DMARC fail	Recipient forwards to gmail; gmail re-verifies original sender DKIM against modified body → fail	Solved by ARC sealing (#229)

---

Where master.cf lives

There is exactly one master.cf, at config/postfix-dkim/etc/postfix/master.cf.

docker-compose.yml bind-mounts ./config/postfix-dkim/etc/postfix onto /etc/postfix, so that file is what Postfix reads. It is not a template and it is not generated: it has no placeholders, no CFML page writes it, and install_hermes_docker.sh states plainly that it is "tracked directly at the render-target path ... nothing to render here". To change it, edit it and restart hermes_postfix_dkim.

Do not add a second copy. A duplicate lived at config/hermes/opt/hermes/conf_files/master.cf from the v260612 root commit until it was deleted on 2026-08-12. Nothing read it, so nothing detected when it drifted, and it drifted badly: commit 9e90bc9a restored the #232 :10026 signing design in the real file and did not touch the copy, leaving it frozen for three months on the pre-#232 configuration that caused the outage documented above (:10026 bound to 127.0.0.1, no_milters, mynetworks=127.0.0.0/8, primary OpenDKIM instead of the sign-only secondary, and no :10027 at all). A master.cf.postscreen variant carried the same defects and was referenced by nothing.

The hazard was not that anything loaded it automatically. It was that the path looked canonical inside the three containers that mount /opt/hermes, so hand-copying it into place would silently reinstate a fixed outage.

If a variant is ever genuinely needed, keep it out of a directory that is mounted into running containers, and give it a name that cannot be mistaken for the live file.

---

Open follow-ups that touch this flow

• #228 External Sender Banner re-enable — blocked on this diagram's :10026 sign-only wiring landing.
• #229 ARC sealing at perimeter — required so downstream forwarders trust Hermes' original-sender verdict.
• amavis SpamAssassin DKIM scoring (separate issue, not yet filed) — options A/B/C documented in the #232 handoff doc.
• Dead-weight config-file audit — Docker/postfix_dkim/config/, Docker/mail_filter/config/, Docker/opendmarc/config/ are shadowed by volume mounts at runtime; ~85 files to consolidate or relocate.
• Install-template drift — resolved 2026-08-12. There is now exactly one master.cf in the repository. See Where master.cf lives above for why a second copy must never be reintroduced.