# Administrator Guide

Hermes SEG Docker administrator documentation. Auto-synced from the repository under docs/admin/.

# System

# Console Firewall

# Console Firewall

_Pro Edition feature._ Maps to **System > Console Firewall** (`view_console_firewall.cfm`, `inc/firewall_action.cfm`, `inc/generate_nginx_configuration.cfm`).

Console Firewall is a **static IP allowlist** for the two admin surfaces of the gateway: the Hermes admin console (`/admin/` and `/admin/2/`) and the Ciphermail web admin (`/ciphermail/`). When enabled, nginx returns `403 Forbidden` to any request for those paths from a source IP not on the list. This is enforced at the nginx layer before Authelia ever sees the request — it's a perimeter filter, not an authentication filter.

## How it differs from IPS

Both pages live under System and both touch nginx and ban traffic, so admins routinely confuse them. The distinction is reactive vs. preventative:

| | **Console Firewall** | **[IPS](https://docs.deeztek.com/books/administrator-guide/page/ips)** |
|---|---|---|
| Model | Static allowlist (default-deny) | Dynamic blocklist (default-allow) |
| Layer | nginx `allow`/`deny` directives | iptables drop rules via fail2ban |
| Scope | `/admin/`, `/admin/2/`, `/ciphermail/` only | All exposed surfaces: SMTP/IMAP, Authelia SSO |
| Trigger | Admin adds an IP to the list | Failed-auth threshold tripped in a log |
| Audience | Internal admins / known office IPs | Anyone on the public internet |
| Storage | `firewall` table + `parameters2.firewall_status` | `intrusion_prevention_jails` + `fail2ban_ips` |
| Apply | Auto: regen nginx + preload restart on every save | Manual: admin clicks Apply Settings after edits |

Both layers stack. A request to `/admin/` from a non-allowlisted IP is rejected by Console Firewall (nginx 403) before fail2ban ever sees an Authelia auth event. A request from an allowlisted IP that then fails login five times still gets the IPS ban from the `authelia` jail.

## What's behind the page

```
Browser request to https://<console>/admin/
        │
        ▼
   hermes_nginx  (sites-enabled/<console>_hermes-ssl.conf)
        │
        ├─►  location /admin/ {
        │       allow 10.0.0.5;       ◄── from `firewall` table where hermesadmin='yes'
        │       allow 192.168.1.0/24;
        │       deny all;
        │       ...auth_request /authelia...
        │       proxy_pass http://hermes_commandbox:8888/admin/;
        │    }
        ▼
   Authelia (if allowed)
        ▼
   hermes_commandbox
```

The firewall is **purely an nginx allow/deny block** rendered into the per-console-host vhost. When `firewall_status = enabled`, the rules are present. When `disabled`, the placeholder is rendered as an empty string and nginx falls back to its default allow-all behavior for that location.

## Database schema

| Table / Column | Role |
|---|---|
| `firewall.ip` | Single IP address (no CIDR — see the validation note below) |
| `firewall.hermesadmin` | `'yes'` / `'no'` — include this IP in the `/admin/` allow list |
| `firewall.ciphermailadmin` | `'yes'` / `'no'` — include this IP in the `/ciphermail/` allow list |
| `firewall.note` | Free-text annotation surfaced in the table |
| `firewall.datetime` | Last-modified timestamp |
| `parameters2` row where `parameter='firewall_status' AND module='firewall'` | Master switch — `enabled` or `disabled` |

The schema (`hermes_install.sql` line 812) defines `ip` as `varchar(50)` but the validator at `inc/validate_ip_address.cfm` is a single-address IPv4 regex — there is no CIDR support and no IPv6 support on this page. A 24-bit range needs 256 rows, one per host. For larger ranges, install an upstream firewall instead.

## The auto-apply flow

Every action handler in `inc/firewall_action.cfm` (`addip`, `editip`, `deleteip`, `setfirewall`) ends the same way:

1. Update the `firewall` table (or `parameters2.firewall_status` for the master switch).
2. Set a numeric `session.m` alert code (1–7 for errors, 33–37 for success).
3. **Always** include `generate_nginx_configuration.cfm` at the bottom of the file — re-render every per-console vhost from `/opt/hermes/templates/hermes-ssl.conf` with current firewall rules baked in.
4. `cflocation` to `/admin/2/preload_restart_nginx.cfm?returnUrl=/admin/2/view_console_firewall.cfm`.

There is **no "Apply Settings" button** on this page. The Save & Apply button on the master-status card and the row-level edit/delete buttons are themselves the apply — every individual change triggers a full nginx regen and a restart. This is the opposite of the [IPS](https://docs.deeztek.com/books/administrator-guide/page/ips) page's batched pending-changes model.

> **Operational consequence.** A burst of edits (adding ten allowed IPs one at a time) triggers ten back-to-back nginx regens, each ending in a restart. The `preload_restart_nginx.cfm` pattern bridges this — the page renders a static "please wait" before the restart fires, then polls until nginx is back, so the admin's own session doesn't `ERR_CONNECTION_REFUSED` mid-redirect. There is no batch-add path; bulk imports are an `INSERT INTO firewall ...` SQL job followed by one manual Save & Apply on the status card.

## Template placeholders

`generate_nginx_configuration.cfm` queries `firewall` twice and renders two placeholder substitutions into the per-vhost rendered file:

| Template token | Substituted with | Used in |
|---|---|---|
| `hermes_fw_hermes` | `allow <ip>;` lines for every `firewall` row where `hermesadmin='yes'`, terminated by `deny all;` | `location /admin/ { ... }` block (template line 157) |
| `hermes_fw_ciphermail` | `allow <ip>;` lines for every `firewall` row where `ciphermailadmin='yes'`, terminated by `deny all;` | `location /ciphermail/ { ... }` block (template line 287) |

When the firewall is disabled, both placeholders are blanked out — the `location` blocks render without any `allow`/`deny` and nginx falls back to its default allow-all. When the firewall is enabled but **no row** has the relevant flag set to `yes`, the recordcount-zero branch in the generator also blanks the placeholder. There is no "deny everyone" mode that locks the page from itself; see the safety checks below.

The `/users/`, `/nc/`, `/main/`, `/plugins/`, and `/web/` locations are **not** firewalled by this page — they have no `hermes_fw_*` placeholder. Mailbox users, Nextcloud users, and Ciphermail end-user portal users hit Authelia directly with no IP allowlist. This is deliberate: those are end-user surfaces, not admin surfaces.

## Safety checks — the four guardrails

Without protection, an admin could trivially lock themselves out of the gateway by deleting their own IP, editing it to something wrong, or enabling the firewall before adding their own IP. `inc/firewall_action.cfm` carries four guard rules (each tied to its own alert code):

| Guard | When it fires | Alert |
|---|---|---|
| Can't delete own IP while firewall enabled | `getip.ip = ClientIP AND firewall_status = enabled` on `deleteip` | `m=3` |
| Can't edit own IP while firewall enabled (unless the new IP is also the client's IP) | Same condition on `editip` with a different new IP | `m=4` |
| Can't enable firewall unless current IP is in the list with `hermesadmin='yes'` | `setfirewall` to `enabled` with no matching `firewall` row for `ClientIP` | `m=5` |
| Duplicate IP rejected on add/edit | Unique-IP check by query | `m=2`, `m=6` |

`ClientIP` is set in `Application.cfc` from the `X-Forwarded-For` header (nginx sets it from `$remote_addr`). When testing behind a load balancer or VPN, what the page considers "your IP" may not match what your laptop reports — verify with the per-row table what nginx is actually seeing before clicking the master enable.

## The recovery path when locked out

If a misconfiguration locks the admin out anyway (forgotten to add the new office IP, master flipped before the row was saved, browser using an unexpected egress IP), the recovery sequence is shell-level on the Docker host:

```
# Disable the firewall directly in the DB
docker exec hermes_db_server mariadb -u root hermes -e \
    "UPDATE parameters2 SET value2='disabled' \
     WHERE parameter='firewall_status' AND module='firewall'"

# Add the new admin IP
docker exec hermes_db_server mariadb -u root hermes -e \
    "INSERT INTO firewall (ip, hermesadmin, ciphermailadmin, note) \
     VALUES ('<your-ip>', 'yes', 'yes', 'Recovery add')"

# Trigger a manual nginx regen by hitting the page from inside the CommandBox container
docker exec hermes_commandbox curl -s http://localhost:8888/admin/2/inc/generate_nginx_configuration.cfm

# Reload nginx
docker exec hermes_nginx nginx -s reload
```

The MariaDB call uses unix-socket auth (root via the container) — no password, by design. Once back in, re-enable the firewall from the UI so the lockout-guard alerts are restored.

A planned Hermes CLI Management Console (`scripts/hermes-cli.sh`) will wrap this recovery into a menu option. Until it ships, the docker-exec sequence above is the supported recovery path.

## Interaction with Console Settings

The console hostname change (`edit_console_settings.cfm`) regenerates the same per-console nginx vhost from the same template — meaning a hostname change automatically picks up the current Console Firewall state. The Firewall rules carry over to the new vhost transparently; the admin does not need to revisit this page after a hostname change.

The reverse is not true: editing the Firewall does not change the hostname. But because `firewall_action.cfm` always calls `generate_nginx_configuration.cfm`, which always renders every active console vhost, a stale-vhost scenario (where an old hostname's vhost still exists alongside the new one) gets both vhosts re-rendered on a Firewall save. This is fine in practice; it's been the established behavior since the AdminLTE 4 refactor (`a348e73f`).

## License gating

The page is wrapped in the standard Pro check:

```cfml
<cfif NOT isDefined("session.edition") OR session.edition NEQ "Pro">
    <cfset proFeatureName = "Admin Console Firewall">
    <cfinclude template="./inc/license_pro_required.cfm">
    <cfabort>
</cfif>
```

Community installs see the gating panel. The `firewall` table and `parameters2.firewall_status` row exist anyway (they're seeded); pre-existing rules continue to render into the nginx vhost as long as `firewall_status='enabled'`. Switching from Pro to Community does **not** auto-disable the firewall — if it was on when the license downgraded, it stays on. To turn it off, an admin needs to either reactivate Pro or use the recovery path above.

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_console_firewall.cfm` | `hermes_commandbox` | Main page + modals |
| `config/hermes/var/www/html/admin/2/inc/firewall_action.cfm` | `hermes_commandbox` | All add/edit/delete/status handlers; auto-applies via the nginx regen include |
| `config/hermes/var/www/html/admin/2/inc/generate_nginx_configuration.cfm` | `hermes_commandbox` | Renders `hermes_fw_hermes` and `hermes_fw_ciphermail` placeholders |
| `config/hermes/var/www/html/admin/2/inc/validate_ip_address.cfm` | `hermes_commandbox` | IPv4 single-address regex (no CIDR, no IPv6 on this page) |
| `config/hermes/var/www/html/admin/2/preload_restart_nginx.cfm` | `hermes_commandbox` | Pre-restart splash + polling rejoin so the admin's session survives the reload |
| `config/hermes/opt/hermes/templates/hermes-ssl.conf` | `hermes_commandbox` | nginx vhost template with the `hermes_fw_*` tokens |
| `config/nginx/etc/nginx/sites-available/<token>_hermes-ssl.conf` | `hermes_nginx` (mounted) | Live rendered vhost — what nginx actually serves |

## Related

- [IPS](https://docs.deeztek.com/books/administrator-guide/page/ips) — the reactive blocklist that complements this preventative allowlist
- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — hostname changes regenerate the same vhost and pick up Firewall state automatically
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) — Authelia runs after Console Firewall passes; both layers stack
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — RemoteAuth admins still hit Console Firewall first; the upstream LDAP bind only matters once the request reaches Authelia

# Authentication Settings

# Authentication Settings

Admin path: **System > Authentication Settings** (`view_authentication_settings.cfm`,
`inc/get_authelia_settings.cfm`, `inc/edit_authelia_settings.cfm`,
`inc/auth_generate_secret.cfm`, `inc/generate_authelia_configuration.cfm`,
`inc/restart_authelia.cfm`).

This page configures **Authelia** — the identity-aware proxy that gates
every Hermes web surface (`/admin`, `/users`, `/nc`). It is global
gateway plumbing: secrets, session timing, login-failure regulation,
SMTP notifier credentials, Duo Push integration, and the OIDC client
that Nextcloud uses for SSO. Per-user MFA enforcement, app passwords,
and the local-vs-remote credential model are documented in the
[Credential Model](https://docs.deeztek.com/books/administrator-guide/page/credential-model) chapter
and on the [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) page; this page is
strictly the gateway configuration.

## Where Authelia sits

```
Browser ──► nginx (hermes_nginx) ──► auth_request /authelia
                                          │
                                          ▼
                              hermes_authelia (port 9091)
                                          │
                                          ▼
                            ┌─────────────┴─────────────┐
                            │                           │
                hermes_ldap (cn=admins,                  hermes_db_server
                cn=mailboxes, cn=relays,                 (MariaDB)
                cn=one_factor, cn=two_factor)            database: authelia
                                                         (TOTP, WebAuthn,
                                                          encryption,
                                                          identity_verification,
                                                          authentication_logs)
```

Every protected request triggers nginx's `auth_request /authelia` which
proxies to `hermes_authelia:9091/api/verify`. Authelia checks its
session cookie (stored in Redis via `hermes_authelia_redis`), and if
needed redirects the user through a one-factor (password) or two-factor
(password + MFA) login flow against the LDAP directory. The nginx
snippets that wire this up are
`config/nginx/etc/nginx/snippets/auth.conf` and `snippets/authelia.conf`.

The Authelia container reads its config from `/config/configuration.yml`
inside the container (host path
`config/authelia/configuration.yml`). The config file is
**regenerated from a template** every time this page is saved — the
template at `/opt/hermes/templates/configuration.yml` (host path
`config/authelia/configuration.HERMES`) is read, the `hermes_*`
placeholders are substituted with values from `parameters2` where
`module = 'authelia'`, the result is written, and the container is
restarted. Direct edits to `configuration.yml` are overwritten on the
next save.

## Configuration storage and persistence

| Setting class | Lives in | Read by |
|---|---|---|
| Toggles, durations, hostnames, log level | `parameters2` table, `module = 'authelia'` | Form load via `get_authelia_settings.cfm`; template substitution at regen |
| High-entropy secrets | Files under `/opt/hermes/keys/` (Docker secret mounts) | Authelia reads via `{{ secret "..." }}` directives in `configuration.yml` |
| Sessions (cookie state) | Redis (`hermes_authelia_redis`) | Authelia at runtime |
| MFA registrations | MariaDB `authelia` database | Authelia at runtime; encrypted at rest with the Storage Encryption Key |
| Identity verification tokens | MariaDB `authelia` database | Reset-password and add-device flows |

Secrets are never round-tripped through the form. Read-only fields on
the page show a masked tail (last 4 chars) of the file contents so the
admin can verify the secret exists and roughly recognise it. The
regenerate button next to each field writes a fresh random value
directly to disk (`auth_generate_secret.cfm`), regenerates
`configuration.yml`, and restarts Authelia.

## Storage backend — MySQL, not SQLite

Authelia stores MFA registrations, identity-verification tokens, and
audit logs in the `authelia` MariaDB database on `hermes_db_server`.
This is intentionally different from Authelia's upstream SQLite default:

- **Survives container recreation.** Docker `down`/`up` cycles wipe
  named volumes if the operator hasn't bind-mounted SQLite's storage
  path. MariaDB lives on the Data tier and is backed up by the
  standard system backup.
- **Tolerates concurrent reads.** SQLite serialises writes; with
  hundreds of mailboxes hitting `/users` and `/nc` simultaneously this
  becomes a contention point.
- **Single backup surface.** The Hermes system backup already includes
  all MariaDB databases. The Authelia DB is included automatically; no
  separate path to remember.

The credential to the `authelia` database is stored as a Docker secret
file at `/opt/hermes/keys/authelia_db_password` and referenced from
the Authelia config via `{{ secret "/keys/authelia_db_password" | msquote }}`.

## Cards on the page

### General Settings

| Field | What it controls | Stored as |
|---|---|---|
| **Password Reset JWT Secret** | Signs the time-limited token in password-reset email links. Rotating invalidates every outstanding reset link. | File `/opt/hermes/keys/authelia_identity_validation_reset_password_jwt_secret_file` |
| **Reset Password Function** | Enable/disable the "Forgot password?" link on the login page. Disable when password is owned by remote AD/LDAP. | `parameters2.authentication_backend.disable_reset_password` |
| **Storage Encryption Key** | AES key Authelia uses to encrypt TOTP secrets and WebAuthn credentials at rest inside the `authelia` database. **Rotating this key invalidates every TOTP and WebAuthn registration** — every MFA-enrolled user must re-enrol on next login. | File `/opt/hermes/keys/authelia_storage_encryption_key_file` |

> **Do not rotate the Storage Encryption Key casually.** The red
> callout on the page exists for a reason. Rotation is correct after
> a confirmed compromise; in every other case it locks every MFA user
> out of their tokens. Duo Push survives because Duo enrollment lives
> in Duo's cloud, not the Authelia DB — see the Duo section below.

### Session Settings

| Field | Default | Notes |
|---|---|---|
| **Session Name** | `hermes_session` | Cookie name. Changing forces every active session to log in again. |
| **Session Secret** | random | Signs the session cookie. Rotating invalidates all sessions immediately. |
| **Session Provider Password (Redis)** | random | Auth between Authelia and `hermes_authelia_redis`. Rotating requires the Redis container to pick up the new secret on Authelia restart. |
| **Session Expiration** | `43200` (12h) | Absolute lifetime from login. NIST SP 800-63B AAL2 ceiling. |
| **Session Inactivity** | `3600` (1h) | Idle timeout. NIST 800-63B recommends 1800s (30 min) for AAL2 / 900s (15 min) for AAL3. |
| **Remember Me Duration** | `43200` (12h) | When ticked at login, replaces Session Expiration **and bypasses Session Inactivity entirely**. Set to `-1` to remove the checkbox from the login form. |

The "Remember Me" interaction is the gotcha. Authelia 4.39 source
(`internal/handlers/handler_authz_authn.go`) confirms that a remembered
session is exempt from inactivity checks — its lifetime is the
Remember Me Duration, full stop. If your compliance posture requires
inactivity enforcement on **every** session, set Remember Me Duration
to `-1`; otherwise users who tick the box are governed only by the
absolute ceiling.

Saving this card also pushes matching values into Nextcloud via `occ
config:system:set` (`session_lifetime`, `session_keepalive`,
`remember_login_cookie_lifetime`). NC sessions are kept in lockstep
with Authelia to prevent stale NC sessions from triggering the OIDC
auto-redirect URL mangle.

### SMTP Notification Settings

The address and subject Authelia uses when sending password-reset,
identity-verification, and new-device-registration emails. Hermes
points Authelia at its own internal Postfix re-injection port
(`hermes_postfix_dkim:10026`) so notification mail goes through the
gateway's outbound pipeline like any other Hermes-originated message.
SMTP host/port are not exposed on this page — they are hard-coded in
the template because there's no real reason to change them on a
self-contained install.

### Login Regulation

Authelia's built-in brute-force throttle.

| Field | Effect |
|---|---|
| **Login Failures Before Ban** | Number of consecutive failures from one source before that source is banned (default 5) |
| **Time Between Failed Logins** | Sliding window over which the failure count is measured, in seconds (default 120) |
| **Banned Time** | How long the ban lasts, in seconds (default 300) |

This is the inner brake. The outer brake is the
[Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) `authelia` jail
(`hermes_fail2ban`) which scans `/remotelogs/authelia/authelia.log`
and applies host-level iptables bans for longer durations. The two
layers are complementary: Authelia regulates per-account in the
application; Fail2ban regulates per-source-IP at the firewall.

### Logging

Authelia log level (`trace`, `debug`, `info`, `warn`, `error`),
format (`json` or `text`), and retention in days. The retention
dropdown applies to the **rotated** Authelia log files
(`config/authelia/log/authelia.log.*`) — the live file is rotated
by the host logrotate config and old rotations are pruned to the
retention window. Default 30 days; legal/compliance reviewers may
need 90 or 180.

### Duo Security

Optional second factor via Duo Push (mobile-app one-tap approval).
Disabled by default. When enabled, fields are required:

| Field | Source |
|---|---|
| **Duo Hostname** | Duo Admin Panel → Applications → Auth API → "API hostname" (`api-XXXXXXXX.duosecurity.com`) |
| **Duo Integration Key** | Same panel, "Integration key" |
| **Duo Secret Key** | Same panel, "Secret key" |
| **Duo Self Enrollment** | If enabled, users who don't yet have a Duo account can self-enrol from the Authelia MFA page |

Integration and Secret keys are stored as Docker secret files at
`/opt/hermes/keys/authelia_duo_api_integration_key_file` and
`authelia_duo_api_secret_key_file`. The form blanks the input on
display and only writes when a non-empty value is submitted (the
masked tail under the box shows the current value's last 4 chars).
This lets the admin save other fields without re-entering Duo
credentials every time.

> **Duo survives storage-key rotation and SQLite-to-MySQL migrations.**
> Duo enrollment lives on Duo's servers, not in Authelia's database;
> Hermes only stores the API credentials. The TOTP and WebAuthn tables
> in the `authelia` MariaDB database are wiped when the storage key
> rotates or the SQLite-to-MySQL migration runs; Duo Push keeps working.

### Webmail OIDC (Nextcloud)

Authelia acts as the OpenID Connect provider for Nextcloud's
`user_oidc` app — this is what makes "Sign in with Hermes" work on
`/nc` and (transparently) auto-login users who already have a valid
Authelia session.

| Field | Role | Stored as |
|---|---|---|
| **OIDC HMAC Secret** | Signs Authelia-issued OIDC tokens | `/opt/hermes/keys/authelia_identity_providers_oidc_hmac_secret_file` |
| **OIDC Client Secret** | Shared secret between Authelia (RP) and Nextcloud (client). Hashed with PBKDF2 inside Authelia. | Plain: `authelia_identity_providers_oidc_clients_client_secret_plain_file`; digest: `authelia_identity_providers_oidc_clients_client_secret_digest_file` |
| **OIDC Key** | RSA 2048 private key (JWKS) Authelia uses to sign ID tokens | `/opt/hermes/keys/authelia_identity_providers_oidc_jwks_file` (generated with `openssl genrsa`) |

The OIDC client is registered as `Hermes_SEG_Webmail`, redirect URI
`https://<console>/nc/apps/oidc_login/oidc`, scopes `openid profile
email groups`. The `groups` scope is what gives Nextcloud the LDAP
group claims it needs to apply NC's own group ACLs.

Rotating the OIDC Client Secret triggers a follow-up `occ
user_oidc:provider Hermes_SEG --clientsecret=...` execution against
the `hermes_nextcloud` container so both sides stay in sync. Rotating
the HMAC Secret or OIDC Key on Authelia's side will invalidate all
in-flight OIDC sessions — users will see a fresh login challenge on
next request.

## What this page does NOT configure

| Setting | Lives on |
|---|---|
| **Console hostname** (Authelia `session.cookies[].domain` + `authelia_url`) | [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — regenerating console settings re-templates Authelia and restarts it |
| **LDAP backend address / bind DN / filters** | Hard-coded in the template to point at `hermes_ldap`. The Hermes LDAP container's structure is provisioned at install time and not exposed as a runtime knob. |
| **Upstream AD / LDAP authentication for specific mailboxes or relay recipients** | [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — Authelia still binds locally; the local LDAP entry has a `seeAlso` overlay pointing at the upstream directory |
| **Per-user MFA enforcement** | The admin's mailbox/relay-recipient detail pages — `recipients.enforce_mfa` is a TINYINT(3) admin-policy flag (see [Credential Model](https://docs.deeztek.com/books/administrator-guide/page/credential-model) and #225 below) |
| **Password reset flow UI** | [Password Resets](https://docs.deeztek.com/books/administrator-guide/page/password-resets) — the reset page itself, CAPTCHA, rate limiting |
| **System users / admins list** | [System Users](https://docs.deeztek.com/books/administrator-guide/page/system-users) — managing accounts in `cn=admins`,`ou=users` |
| **Fail2ban brute-force protection** | [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) — the host-firewall layer in front of Authelia |
| **Nextcloud OIDC auto-redirect toggle** | [Email Server Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) — moved off this page; controls whether `/nc` silently SSOs the already-authenticated user |

## MFA enforcement is decoupled from the `cn=two_factor` LDAP group (#225)

This is the single most-often-confused part of Hermes authentication.

**The LDAP group `cn=two_factor` is a *capability* marker, not an
*enforcement* marker.** Membership in `cn=two_factor` tells Authelia
"this user has at least one MFA method registered and should be
prompted for it." Membership in `cn=one_factor` tells Authelia
"password only." A user moves from `one_factor` to `two_factor` by
**enrolling an MFA method themselves** through the user portal's
Account Settings page — admins do not force-flip the group.

**Admin policy lives in `recipients.enforce_mfa` (and
`system_users.enforce_mfa` for system users) — a TINYINT(3) column,
not a group.** When the admin sets this to 1, the user-portal pages
that depend on it consult
`config/hermes/var/www/html/users/2/inc/check_enforce_mfa_restriction.cfm`
on each request. If the user is in `cn=mailboxes` or `cn=relays`, has
`enforce_mfa = 1`, and is **not** yet in `cn=two_factor`, the page
renders a restricted-access panel pointing them at Account Settings
to enable 2FA. Once they enrol, the group flips and the restriction
clears on the next page load.

### Why this two-layer model

The chicken-and-egg without it: enrolling TOTP or WebAuthn requires
the user to receive an identity-verification email from Authelia.
A brand-new mailbox has no working mail client yet — they need to
get into the portal first to set up an app password, configure their
phone, and read the email. Hard-locking them out of the portal until
they enrol means they can never enrol.

The bootstrap surfaces (Account Settings, My App Passwords, Set Up
Your Devices, Webmail) remain accessible under the restriction; the
rest of the portal does not. Once the user enables 2FA, the
restriction lifts automatically.

### Operational consequence — log out, don't just refresh

Authelia caches LDAP group membership in the session for the
refresh interval (default 5 minutes). When a user enables 2FA,
their LDAP group flips to `cn=two_factor` immediately, but
Authelia's session still says `cn=one_factor` until the cache
expires. Hermes works around this by redirecting through
`/logout` after the enable-2FA flow, which forces a fresh
Authelia session and picks up the new group membership on the
next request. If a user reports "I enabled 2FA but the portal
still says I haven't," the answer is always: log out and back in.

## Save flow

Save & Apply Settings runs `edit_authelia_settings.cfm`, which:

1. Validates every form field (whitelist regex per field, length minimums for secrets).
2. Updates the matching `parameters2` rows with `applied = '2'`.
3. After all field updates succeed, flips every `module = 'authelia'`
   row to `applied = '1'`.
4. Calls `generate_nextcloud_configuration.cfm`, pushes session
   parameters into Nextcloud via `occ config:system:set`, and restarts
   `hermes_nextcloud`.
5. Calls `generate_authelia_configuration.cfm` which re-templates
   `configuration.yml` from `/opt/hermes/templates/configuration.yml`.
6. Calls `restart_authelia.cfm` (which uses the canonical preload
   pattern, not a hard restart, to avoid `ERR_CONNECTION_REFUSED` on
   the redirect back).
7. Sleeps 10 seconds to let Authelia come back up before the redirect
   lands.

If validation fails at any step the form short-circuits via `cflocation
url="#cgi.http_referer#"` with a `session.m` alert code; no partial
state is committed because each cascade step gates on `step = N`.

## Failure semantics

| What breaks | What happens |
|---|---|
| Authelia container down | nginx `auth_request` returns 500; every protected page shows "502 Bad Gateway" or similar. Mail flow is unaffected — Postfix, Dovecot, and Amavis don't depend on Authelia. |
| MariaDB `authelia` database unreachable | Authelia starts but cannot authenticate; same symptom as above. |
| Redis (`hermes_authelia_redis`) down | Authelia starts but cannot store sessions; users are bounced to the login page on every request. |
| Storage Encryption Key file missing | Authelia refuses to start. Check `docker logs hermes_authelia` for the missing-secret error. |
| `configuration.yml` syntax-broken after a bad save | Authelia refuses to start. Restore from the on-disk backup `configuration.BACKUP`, fix in the form, save again. |
| LDAP container down | Authelia starts but every login attempt fails. Same recovery as MariaDB-down — fix LDAP first, no Authelia restart needed. |

The Save & Apply Settings button does not have a pre-save dry-run; if
Authelia refuses to start after a save, the previous `configuration.yml`
is no longer on disk. The `restart_authelia.cfm` step will surface the
container start failure in the admin UI's restart-output area; the
admin should not navigate away until the success banner appears.

## Related documentation

- [Credential Model](https://docs.deeztek.com/books/administrator-guide/page/credential-model) — the four-credential architecture (web login, app passwords, NC internal password, Hermes System app password) that this page's session settings gate
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — when web login is bound against an external AD/LDAP instead of Hermes's internal directory
- [Password Resets](https://docs.deeztek.com/books/administrator-guide/page/password-resets) — the forgot-password page that consumes the JWT Secret on this page
- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — the console hostname change that triggers an Authelia template re-render
- [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) — the Fail2ban `authelia` jail that protects this surface at the firewall
- [System Users](https://docs.deeztek.com/books/administrator-guide/page/system-users) — admin accounts that live in `cn=admins`
- [Email Server Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) — the Nextcloud OIDC auto-redirect toggle that complements the OIDC client configured here

# Backup/Restore

# Backup/Restore

Admin path: **System > Backup/Restore** (`view_system_backup.cfm`).

> **CLI-only by design.** Backup and restore run from the Docker host's shell, not from the admin console. The admin console's *Backup/Restore* page is a read-only info surface (CLI examples + a list of backups detected on disk + a link back to this doc). There are no buttons. Long-running operations + web UIs is a known footgun (page reload kills progress, browser timeouts, race conditions); the CLI is the canonical interface.

## What ships in this release

Two scripts under [`scripts/`](../../../scripts/):

| Script | Purpose |
|---|---|
| [`system_backup.sh`](https://github.com/deeztek/Hermes-Secure-Email-Gateway/blob/main/scripts/system_backup.sh) | **Hot mode by default — zero application downtime.** Uses application-native hot-backup primitives: `mariadb-dump --single-transaction`, `slapcat`, and live tar of mail tiers (Dovecot, Amavis, Postfix all use atomic-rename writes safe for live tar). Toggles `occ maintenance:mode --on` briefly during Nextcloud file tar to pause NC user writes (mail flow unaffected). `--cold` flag stops the full stack for legal-hold / forensic snapshots that need absolute byte-level consistency. |
| [`system_restore.sh`](https://github.com/deeztek/Hermes-Secure-Email-Gateway/blob/main/scripts/system_restore.sh) | **Always cold on the restore side** (we're overwriting tier contents — concurrent reads/writes would corrupt). Verifies the manifest + per-archive SHA256 BEFORE any destructive action, **auto-remaps** tiers to this host's paths (refuses only on a build-version mismatch unless `FORCE_VERSION_MISMATCH=1`), restores DBs via socket auth, restores OpenLDAP via `slapadd`, stream-extracts in-scope tiers directly to their mount paths, reconciles the Nextcloud DB user, restarts the stack, and on a cross-host restore offers to run `system_rehost.sh`. |

## Backup scopes

The `-B` flag chooses what to back up. Pick the scope that matches your need — there's no reason to back up 500 GB of vmail every night if only the DBs and configs are churning.

| Scope | Includes | Typical cadence | Hot-mode duration |
|---|---|---|---|
| `system` | Config tier + Data tier + 6 DB dumps + LDAP slapcat | Nightly | seconds to a few minutes (dominated by `/mnt/data` tar size; DB+LDAP dumps are fast) |
| `archive` | Archive tier (Amavis quarantine) | Weekly or per retention policy | proportional to archive size; mail intake continues uninterrupted |
| `vmail` | Vmail tier (Dovecot mailboxes) | Weekly | proportional to mailbox size; mail flow continues uninterrupted |
| `nextcloud` | Nextcloud tier (NC files) | Weekly | proportional to NC file size; NC web UI shows "under maintenance" during the tar; mail unaffected |
| `all` | Everything above | Periodic full-DR snapshot | sum of all of the above |

## Hot-mode safety per component

Why we don't need downtime:

| Component | Hot-backup technique | Why it's safe |
|---|---|---|
| **MariaDB** | `mariadb-dump --single-transaction --routines --triggers --events --databases <db>` | InnoDB MVCC gives a consistent point-in-time snapshot. No table locks. Stored procedures, triggers, and scheduled events captured. |
| **OpenLDAP** | `slapcat -b dc=hermes,dc=local` inside `hermes_ldap` | Standard hot LDIF export. |
| **Dovecot (vmail)** | tar `/mnt/vmail` live | maildir/sdbox writes are atomic-rename (write to temp filename, atomic `mv` to final name). No torn files. Worst case: messages arriving during the tar window may land after the tar's snapshot — they're durable upstream (postfix queue, sender's MX retries) and captured by the next backup. |
| **Amavis (archive)** | tar `/mnt/archive` live | Amavis quarantine writes are atomic-rename. Same as Dovecot. |
| **Nextcloud (files)** | tar `/mnt/files` live, with `occ maintenance:mode --on` toggled around the tar | NC writes are atomic, but the filesystem ↔ `oc_filecache` DB table can drift if a user uploads mid-tar. Maintenance mode pauses NC user writes — the NC web UI shows "under maintenance" briefly, but mail flow is unaffected. Use `--no-nc-maintenance` to skip the toggle if needed. |
| **Postfix (data tier)** | tar `/mnt/data/postfix` live | Postfix queue files are atomic-rename. |
| **Service logs (data tier)** | tar live | Append-only. A torn last line is cosmetic, not data loss. |
| **MariaDB / LDAP / ClamAV raw files** | **Excluded** from the data tier tar | DB dumps + LDAP slapcat are the authoritative restore sources, so the on-disk InnoDB tablespace files and slapd data files are redundant. ClamAV signatures are regenerable, not worth the backup space. |

Hot mode is the daily backup. **Cold mode (`--cold`) is the escape hatch for use cases where absolute byte-level consistency matters more than uptime** — legal hold, forensic snapshots, regulatory archive. Cold mode does `docker compose stop` for the full duration.

## Backup

### Backup quick start

```bash
sudo /opt/hermes-seg-docker-gl/scripts/system_backup.sh -P /mnt/backups -B system --yes
```

The script creates a **backup directory** at `/mnt/backups/hermes-backup-<scope>-<build_no>-<UTC-timestamp>/` (e.g. `hermes-backup-all-v260609-20260609T183616Z/`). It is written under a `.staging-…` name and **atomic-renamed** into place only on success. **There is no outer tarball** — the per-tier archives sit directly in the directory, so the restore verifies and stream-extracts each one in place without unpacking a wrapper first (no ~2× scratch space). Read `manifest.json` directly to inspect a backup before restoring.

### Output layout

Inside the backup directory (only the archives relevant to the chosen scope are present):

```text
manifest.json                  ← scope, mode (hot/cold), topology, source hostname,
                                  build_no, SHA256 per archive
backup.log                     ← the backup run's own log
databases.tar.gz               ← 6 .sql files; system / all scopes only
ldap.ldif.gz                   ← slapcat output; system / all scopes only
config.tar.gz                  ← Config tier USER-DATA subdirs only (keys, .gnupg,
                                  ssl, templates, sa-bayes, sa-learn, dkim, arc,
                                  conf_files) — NOT .env / secrets / compose / scripts
                                  (those are host-specific and excluded by design);
                                  system / all scopes only
data.tar.gz                    ← Data tier user-data only (excludes mysql/ ldap/
                                  clamav/ — captured by dumps / slapcat / regenerable);
                                  system / all scopes only
archive.tar.gz                 ← Archive tier; archive / all scopes only
vmail.tar.gz                   ← Vmail tier; vmail / all scopes only
nextcloud.tar.gz               ← Nextcloud tier; nextcloud / all scopes only
```

### Backup flags

| Flag | Purpose |
|---|---|
| `-P <path>` | **Required.** Output directory. Must exist and be writable. |
| `-B <scope>` | **Required.** One of: `system`, `archive`, `vmail`, `nextcloud`, `all`. |
| `--cold` | Stop the full stack for the duration of the backup. Use for legal-hold / forensic snapshots. Default is HOT mode (zero application downtime). |
| `--no-nc-maintenance` | Skip the brief `occ maintenance:mode --on` that hot-mode nextcloud / all backups use to pause NC user writes during the file tar. Without it, file uploads happening mid-tar may be missed by the backup. |
| `--yes` (or `-y`) | Skip the interactive confirmation prompt. Use for cron / Ofelia. |
| `--dry-run` (or `-n`) | Print what would happen without changing anything. |
| `--help` (or `-h`) | Show usage. |

### Scheduling

For nightly automated backups, use **host cron** on the Docker host. `system_backup.sh` is a host-level script (it runs `docker compose stop`, reads `.env` from the host, writes to `/mnt/backups` on the host) — host cron is the natural fit. Example `/etc/cron.d/hermes-backup`:

```cron
# m h dom mon dow user  command
0 3 * * *      root  /opt/hermes-seg-docker-gl/scripts/system_backup.sh -P /mnt/backups -B system    --yes >> /var/log/hermes-backup.log 2>&1
0 4 * * 0      root  /opt/hermes-seg-docker-gl/scripts/system_backup.sh -P /mnt/backups -B vmail     --yes >> /var/log/hermes-backup.log 2>&1
0 5 1 * *      root  /opt/hermes-seg-docker-gl/scripts/system_backup.sh -P /mnt/backups -B all       --yes >> /var/log/hermes-backup.log 2>&1
```

A typical cadence:

| Cadence | Scope | Why |
|---|---|---|
| Nightly | `system` | Small + fast. Captures DBs, LDAP, configs, install-root state. Run with hot mode = zero downtime. |
| Weekly | `vmail` (or `archive` or `nextcloud`, rotated) | Larger but slower-changing. |
| Monthly | `all` | Full disaster-recovery snapshot. |

The script's exit code reflects success (0) or failure (non-zero). For built-in email alerting, use the `--notify-email=ADDR` flag (see below). For "Hermes is so dead it can't even tell you" cases, see [External monitoring](#external-monitoring-strongly-recommended).

> **Why host cron and not Ofelia?** Ofelia runs as a container (`hermes_ofelia`). Its job model (`job-exec` into a named container, `job-local` on the Ofelia container itself) doesn't fit `system_backup.sh` cleanly — the script needs host-level `docker compose` access, root, and write access to `/mnt/backups`. Ofelia's image lacks `docker compose` plugin and root host access. Native Ofelia integration is deliberately NOT on the roadmap; the existing **System > Scheduled Tasks** admin page lists Ofelia jobs but does NOT support adding new ones from the UI today.

### Failure / success email alerting

Use `--notify-email=ADDR` to receive an email on backup completion. By default emails on **failure only** (the "noisy on failure, silent on success" pattern most operators want). Add `--notify-on-success` to also email on success — useful for "daily I-am-alive confirmation" use cases.

```bash
# Email on failure only (typical)
sudo /opt/hermes-seg-docker-gl/scripts/system_backup.sh -P /mnt/backups -B system --yes \
  --notify-email=admin@example.com

# Email on both failure AND success
sudo /opt/hermes-seg-docker-gl/scripts/system_backup.sh -P /mnt/backups -B all --yes \
  --notify-email=admin@example.com --notify-on-success
```

Subject lines are bracketed for easy scanning in a mail client:

- Success: `[SUCCESS] Hermes backup on <hostname> (scope=<scope>)`
- Failure: `[FAILURE] Hermes backup on <hostname> (scope=<scope>)`

Failure bodies include the timestamp, scope, mode, reason, log file path, and the last 50 lines of the log. Success bodies include the timestamp, scope, mode, output filename, file size, and run duration.

**How it works**: the script shells out to `docker exec -i hermes_postfix_dkim sendmail -t` and pipes the message into the Postfix container's `sendmail` binary. Postfix queues and delivers it like any other outbound mail from Hermes. No host MTA configuration is needed — Hermes's own Postfix does the work.

**Verify the path before wiring into cron** — `--test-notify` sends one `[TEST] [SUCCESS]` sample and one `[TEST] [FAILURE]` sample to the address you give, then exits without running a backup:

```bash
sudo /opt/hermes-seg-docker-gl/scripts/system_backup.sh --test-notify \
  --notify-email=admin@example.com
```

Both test messages have a `[TEST]` prefix in the subject so any ops-alert filters watching for `[FAILURE]` are not tripped. If both arrive, your notification path is good. If neither arrives, check `hermes_postfix_dkim` is running and look at the log file the script prints for sendmail errors.

**Caveat — needs Hermes to be at least partially healthy**: if the failure cause is "the Postfix container is down" or "the Docker daemon is down", `docker exec` has nothing to talk to and the email won't go out. The script logs the failure-to-notify as a warning and exits with the original non-zero status, but you won't get the email. This is the gap external monitoring fills — see below.

### External monitoring (strongly recommended)

Built-in email alerting covers the "backup ran but something went wrong" case (the 99% case). It does NOT cover "Hermes itself is so broken it can't send any email at all" — Docker daemon crashed, host out of disk, container restart loop, network partition, etc. For that, you need an external monitoring tool that lives off the Hermes host and tells YOU when Hermes goes dark.

**Strongly recommended for every production install.** Common choices:

| Tool | Pattern | Best for |
|---|---|---|
| **[Zabbix](https://www.zabbix.com/)** | Agent on the Hermes host reports up/down, disk, container health, custom metrics | Self-hosted, comprehensive; common in business / mid-size deployments |
| **[Nagios / Icinga](https://www.nagios.org/)** | NRPE plugin or similar | Self-hosted, classic; many existing operator setups already have it |
| **[healthchecks.io](https://healthchecks.io/)** | Cron pings a URL on success; if the ping doesn't arrive on schedule, healthchecks alerts you | Dead simple; free tier; cron-native pattern |
| **[Uptime Kuma](https://github.com/louislam/uptime-kuma)** | Self-hosted ping monitor with web UI | Free, self-hosted alternative to healthchecks.io |
| **PRTG / Datadog / New Relic / etc.** | Commercial monitoring | If you already have one, integrate Hermes alongside your other infrastructure |

The healthchecks.io pattern works nicely alongside cron-based backups:

```cron
# Pings healthchecks.io on success only (curl wraps the backup; ping is the URL of your check)
0 3 * * *  root  /opt/.../system_backup.sh -P /mnt/backups -B system --yes \
                 --notify-email=admin@example.com \
                 && curl -fsS --retry 3 https://hc-ping.com/<your-uuid> >/dev/null
```

If the backup fails, the `--notify-email` sends the failure email (assuming Postfix is up). If the backup succeeds, healthchecks.io gets the ping. If the WHOLE HOST is down (no ping, no email), healthchecks.io alerts you after the scheduled interval. Three-layer coverage with minimal moving parts.

### Off-site copy

`system_backup.sh` writes to the local `-P` path only. Off-site copy is left to your existing tooling — `rclone`, `rsync` to remote storage, `aws s3 cp`, `restic`, whatever you already use. Typical pattern:

```bash
sudo /opt/hermes-seg-docker-gl/scripts/system_backup.sh -P /mnt/backups -B system --yes \
  && rclone sync /mnt/backups remote:hermes-backups/
```

## Restore

### Restore quick start

```bash
sudo /opt/hermes-seg-docker-gl/scripts/system_restore.sh -F /mnt/backups/hermes-backup-system-v260609-20260601T103000Z
```

`-F` takes the backup **directory** (not a tarball).

**The restore replaces the data in the backup's scope and leaves other scopes alone.** Restoring a `system` backup overwrites the install root + Data tier + DBs + LDAP; the Vmail / Archive / Nextcloud tiers are untouched. Restoring a `vmail` backup overwrites only `/mnt/vmail`. The stack is stopped for the duration of the restore (always — even hot-mode backups are restored cold).

### Safety: SHA256 + version gates, topology auto-remap

Two gates fire BEFORE any destructive action, plus automatic topology handling:

1. **Manifest SHA256 verification.** Every archive's SHA256 is checked against `manifest.json` (verified in place — no unpacking). If any byte of the backup is corrupt or tampered with, the restore aborts BEFORE stopping the stack or touching any data.
2. **Hermes build-version match.** The backup's `build_no` (captured at backup time from `system_settings.build_no`) is compared against the current host's `build_no`. If they differ, restore refuses unless `FORCE_VERSION_MISMATCH=1` is set. Schema migrations between Hermes builds make cross-version restore unsafe — restoring an older DB dump onto a newer host leaves the schema in a state the running code does not expect, which breaks silently when something hits a missing or renamed column. **The correct procedure is to install Hermes at the matching build first (`git checkout <build>`), restore, then upgrade forward via `scripts/system_update_docker.sh`.**
3. **Storage-topology auto-remap.** If the backup's recorded mount paths (`/mnt/data`, `/mnt/vmail`, etc.) differ from this host's current mount paths in `.env` — typical when restoring onto different hardware — the restore **automatically retargets each tier to this host's paths** and prints a `REMAP` line per tier. No flag is needed; the old `FORCE_REMAP=1` gate was retired as needless friction for new-hardware DR.

### Disaster-recovery flow (different host)

1. Install Hermes fresh on the new host using [`install_hermes_docker.sh`](https://github.com/deeztek/Hermes-Secure-Email-Gateway/blob/main/scripts/install_hermes_docker.sh). The install root + `.env` need to exist before restore can succeed.
2. Make the backup directory reachable on the new host — **either** mount the backup storage (off-site / NAS share) on the new host (recommended: the restore stream-extracts in place, so there's no need to copy the whole backup), **or** `scp -r` the backup directory across to local disk.
3. Run `system_restore.sh -F /path/to/hermes-backup-<scope>-<build>-<ts>`. Storage-topology differences are **auto-remapped** to this host's paths; a build-version difference still requires `FORCE_VERSION_MISMATCH=1` (better: install the matching build first).
4. When the restore detects a **cross-host** restore (backup hostname ≠ this host), it **offers to run `system_rehost.sh` for you** — accept it to rewire host identity (`.env`, DB rows, all rendered configs, and the Nextcloud DB user).
5. Verify the admin console loads and a test message flows end-to-end.

> **A cross-host restore needs more than the restore itself.** The restored data
> carries the *source* host's identity and credentials, so several things must be
> reconciled by hand — run `system_rehost.sh`, re-activate the Pro license, and
> re-save the Content Checks pages to re-apply the milter chain. Follow the full
> checklist: **[Post-Restore Steps](https://docs.deeztek.com/books/installation-reference/page/post-restore-steps)**.

### Restore flags

| Flag | Purpose |
|---|---|
| `-F <path>` | **Required.** Path to the backup **directory** produced by `system_backup.sh`. |
| `--yes` (or `-y`) | Skip the interactive confirmation prompt (and auto-accept the rehost offer on a cross-host restore). |
| `--dry-run` (or `-n`) | Show what would happen without changing anything. |
| `--only=<scope>` | Restore only one scope out of an `all` backup (e.g. `--only=vmail`). |
| `--help` (or `-h`) | Show usage. |
| `FORCE_VERSION_MISMATCH=1` (env) | Override the build-version refusal. Topology differences auto-remap — no flag needed. |

## When to use hypervisor snapshots instead

The cold-mode escape hatch (`--cold`) covers byte-level-consistency use cases that the cold-mode scripts can satisfy. For two other cases, **hypervisor snapshots** are the right tool, not the Hermes scripts:

1. **Pre-upgrade safety net.** Always take a hypervisor snapshot before running `system_update_docker.sh` — that gives you a working rollback if the upgrade fails mid-flight. The methodology doc codifies this.
2. **Zero-downtime full-host snapshot.** If you want a single consistent point-in-time image of the entire Hermes host (every storage tier, the Docker daemon state, the host OS), a hypervisor snapshot is the only tool that captures all of that atomically.

Per-hypervisor snapshot mechanisms:

| Platform | Mechanism |
|---|---|
| Proxmox VE | Datacenter > Backup, or Snapshot from the VM's right-click menu |
| VMware vSphere / ESXi | VM > Snapshots > Take Snapshot |
| KVM / libvirt | `virsh snapshot-create-as <domain> <name> --disk-only --atomic` |
| AWS EC2 | EBS volume snapshot (or AMI for full image) |
| Azure VMs | Disk snapshot, or Recovery Services Vault |
| Google Compute Engine | Disk snapshot |
| Hyper-V | Checkpoint |

## What you should NOT do

### Do NOT run the legacy bare-metal scripts on a Docker host

The pre-Docker `config/hermes/opt/hermes/scripts/system_backup.sh` and `system_restore.sh` are kept in the repo for reference and for the legacy-to-Docker migration path. **Do not run them on a Docker install.** The legacy `system_restore.sh` does `cd / && tar -xvzf <backup-file>` — extracts the backup tarball relative to the host filesystem root and will overwrite host directories with files from a layout that does not match the Docker host's reality. Hermes services fail to start, host OS may become unbootable.

### Do NOT tar a running storage tier with `tar` directly

If for some reason you reach for `tar` directly instead of `system_backup.sh`, do NOT tar `/mnt/data`, `/mnt/vmail`, `/mnt/files`, or `/mnt/archive` while the stack is running **without using the hot-backup primitives the script uses**. Specifically:

- `/mnt/data` contains MariaDB's tablespace files — tar'ing them while `hermes_db_server` is running produces a backup MariaDB will reject as inconsistent on restore. Use `system_backup.sh` (which excludes `mysql/` from the data tar and captures DBs via `mariadb-dump`) instead.
- Without `slapcat`, raw tar of `/mnt/data/ldap` mid-write captures inconsistent slapd database files.

The Hermes scripts handle all of this correctly. Use them.

### Do NOT trust an untested restore procedure

Whatever backup strategy you adopt, **practice the restore at least once on a non-production system before you rely on it.** Take a backup of your live Hermes host, spin up a second VM, run the restore, verify you can log into the admin console and send a test message. A backup procedure that has never been restored from is not a backup procedure — it is wishful thinking.

## What's coming in Phase B

The Phase A scripts cover the common cases (hot daily system backup, scoped tier backups, cold-mode forensic snapshot, scope-aware restore). The Phase B refactor (post-Link-Guard) will add:

- **Retention pruning** (`--retain-last=N` deletes older backups beyond N)
- **Per-tier `--remap-tiers <old>:<new>`** to override individual tiers (today's default is whole-backup auto-remap to this host's paths)
- **Selective container restart** instead of full `compose down` on the restore side (faster restart, smaller blast radius)
- **Filesystem-snapshot integration** (LVM / ZFS / btrfs detection): if a tier lives on a snapshot-capable filesystem, take a filesystem snapshot and tar the snapshot rather than the live mount, for use cases where "best-effort hot tar" isn't good enough but `--cold` is too disruptive

**Not on the Phase B roadmap** (deliberately dropped):

- **Native Ofelia integration**. Cron is the right tool. Ofelia's job model (`job-exec` into a named container, `job-local` on the Ofelia container) doesn't fit a host-level script cleanly. Forcing it would mean a custom Ofelia image with `docker compose` plugin + Docker socket + root access, plus admin-page UI work to add jobs — all to honor a pattern that doesn't fit. Host cron is the answer.
- **Admin-UI launch button**. Long-running operations + web UIs is a footgun; the admin who runs a backup is already in SSH. The Backup/Restore admin page stays read-only / informational, by design.

Failure / success notification is a separate discussion — see the Scheduling section above. Today the answer is cron's `MAILTO=` / pipe exit code into existing alerting; if operators ask for native built-in notification, it's a small Phase B addition.

Tracking: [#219](https://github.com/deeztek/Hermes-Secure-Email-Gateway/issues/219) for the backup-side enhancements, [#220](https://github.com/deeztek/Hermes-Secure-Email-Gateway/issues/220) for the restore-side.

## Migrating from a legacy bare-metal install

A separate tool exists at [`scripts/migrate_legacy_to_docker.sh`](https://github.com/deeztek/Hermes-Secure-Email-Gateway/blob/main/scripts/migrate_legacy_to_docker.sh) for operators moving from a legacy bare-metal install to the Docker install. It consumes a backup produced by the **legacy** `system_backup.sh` (which is correct in the bare-metal context where it ran) and restores it into the Docker layout via a translation step — NOT the same as running the legacy restore script directly. See the migration section of the [v260119 release notes](https://github.com/deeztek/Hermes-Secure-Email-Gateway/releases/tag/v260119) for current scope.

## Cross-references

- [Storage Topology](https://docs.deeztek.com/books/installation-reference/page/storage-topology-5-tiers) — the five-tier layout the backup operates on
- [Release & Update Methodology](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology) — recommends taking a hypervisor snapshot before running `system_update_docker.sh`
- [scripts/migrate_legacy_to_docker.sh](https://github.com/deeztek/Hermes-Secure-Email-Gateway/blob/main/scripts/migrate_legacy_to_docker.sh) — separate from backup/restore; for one-time bare-metal-to-Docker migration only

# Console Settings

# Console Settings

Admin path: **System > Console Settings** (`view_console_settings.cfm`,
`inc/get_console_settings.cfm`, `inc/edit_console_settings.cfm`,
`inc/generate_auth_nginx_configuration.cfm`,
`inc/generate_nginx_configuration.cfm`,
`inc/generate_authelia_configuration.cfm`,
`inc/generate_nextcloud_configuration.cfm`,
`inc/edit_ciphermail_settings.cfm`, `preload_restart_nginx.cfm`).

This page configures **how the outside world reaches the Hermes web
console** — the FQDN or IP that nginx terminates TLS on, the
certificate it presents, and three HTTPS hardening toggles (HSTS, OCSP
stapling, OCSP stapling verify). It is the single source of truth for
the console hostname; every other component that needs to know "where
do I live" (Authelia session cookie, Nextcloud trusted domain and
theming URL, the User Console link in Nextcloud's top bar, Ciphermail
portal redirect URL, OIDC discovery URI) is regenerated from this page
when the Console Address changes.

Pairs with [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup), which configures the
gateway's mail-side identity (Postfix `myorigin` / `myhostname` and the
host IP). The two pages together define every name Hermes presents to
the world.

## Where the console host fits

```
Browser  ──►  hermes_nginx (443)
                  │ server_name = <Console Address>
                  │ ssl_certificate = <Console Certificate>
                  ▼
              auth_request /authelia
                  │
                  ▼
              hermes_authelia
                  │ session.cookies[].domain = <Console Address>
                  │ authelia_url             = https://<Console Address>/authelia
                  ▼
              hermes_commandbox (admin + user portal)
              hermes_nextcloud  (NC trusted_domain + theming URL +
                                 user_oidc discovery URI +
                                 External Sites "User Console" link)
              hermes_ciphermail (portal URL = Console Address)
```

Every one of those downstream consumers is rewritten from the value
saved on this page. Direct edits to `auth.conf`, `hermes-ssl.conf`,
`configuration.yml`, Nextcloud's `config.php`, the Ciphermail portal
URL, or OIDC discovery are **overwritten on the next save**.

## Configuration storage

Both the Console Address and the four hardening / cert settings live in
the `parameters2` table with `module = 'console'`. The page is wired
strictly against that table — there are no file-backed secrets here,
only DB values.

| Setting | `parameters2.parameter` | Default |
|---|---|---|
| Console Address (IP or FQDN) | `console.host` | `smtp.domain.tld` (seed) |
| Console Certificate (FK into `system_certificates.id`) | `console.certificate` | `29` (seed snakeoil) |
| DH parameters | `console.dhparam` | `enable` |
| HSTS | `console.hsts` | `enable` |
| OCSP Stapling | `console.ssl_stapling` | `enable` |
| OCSP Stapling Verify | `console.ssl_stapling_verify` | `enable` |

> **DH parameters note.** The `console.dhparam` row is still in the
> schema and still set by the form handler when a DH file exists, but
> commit `2dbc2bd3` ("ECDHE-only ciphers, remove DH parameters
> feature") moved the active TLS cipher suite to ECDHE-only — DH is
> no longer offered. The setting is therefore inert; leave it at the
> default.

## Fields on the page

### Console Address (IP or FQDN)

The hostname or IP nginx terminates TLS on for `/admin`, `/users`,
`/nc`, `/portal` (Ciphermail), and every other console-served path.
Accepts:

- **IPv4** — validated against the standard dotted-quad regex
- **IPv6** — validated against the bracketed/colon form
- **FQDN** — validated by the email-trick (`IsValid("email",
  "bob@<host>")`)

`edit_console_settings.cfm` trims whitespace and strips any trailing
zone-file dots (`mail.example.com.` becomes `mail.example.com`) before
saving. That stripping happens at the input boundary so every
downstream consumer — `autoconfig.cfm`, `autodiscover.cfm`, nginx vhost
generation, the NC theming URL, the OIDC discovery URI — sees an
identical canonical string. Outlook for Mac is one of several MUAs that
breaks on the trailing dot, hence the strip.

> **If you set Console Address to an IP** and then the server's IP
> changes, you must update **both** Console Address (this page) **and**
> Host IP Address (Server Setup) — they are stored in separate
> parameters and neither cascades to the other. The page surfaces this
> in a warning callout.

### Console Certificate

Free-text autocomplete that searches `system_certificates` via
`getcertificates.cfm` (an ajax endpoint). Selecting a row populates a
hidden `certificateno_1` field with the certificate's row ID, plus
five read-only display fields (subject, issuer, serial, type, friendly
name). The handler validates the ID exists in `system_certificates`
before saving — an empty or unknown ID falls through to the next step
with `step = 3`, which means the existing `console.certificate` value
is preserved.

The selected cert becomes `nginx`'s `ssl_certificate` /
`ssl_certificate_key` for every console-facing vhost. Certificate
upload, renewal, and Let's Encrypt are managed on
[System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates); this page is the
**binding** of one of those certificates to the console hostname.

### HSTS, OCSP Stapling, OCSP Stapling Verify

Three boolean (`enable` / `disable`) selects. Each is substituted into
`/opt/hermes/templates/hermes-ssl.conf` at regen time:

| Toggle | Effect on the generated `hermes-ssl.conf` |
|---|---|
| HSTS | `add_header Strict-Transport-Security "max-age=31536000; preload"` (enabled) vs. the same line commented out (disabled) |
| OCSP Stapling | `ssl_stapling on;` (enabled) vs. `#ssl_stapling on;` (disabled) |
| OCSP Stapling Verify | `ssl_stapling_verify on;` (enabled) vs. `#ssl_stapling_verify on;` (disabled) |

Defaults are all `enable` and should stay that way for any
publicly-reachable console. Disable only if you have a specific reason
(e.g., HSTS preload conflict during a hostname migration window).

## Save flow — the cascade

Clicking **Save & Apply Settings** posts `action=edit`, which runs
`edit_console_settings.cfm` as a strict 7-step sequence. Each step gates
on the previous step's success (`<cfif step is "N">`) — any
validation failure short-circuits with `cflocation
url="#cgi.http_referer#"` and `session.m` set to the matching alert
code; no partial state lands.

```
step 1  Validate + write console.host
step 2  Validate + write console.certificate
step 3  (DH param — inert)              ──► step 4
step 4  Write console.hsts
step 5  Write console.ssl_stapling
step 6  Write console.ssl_stapling_verify
step 7  Regen + restart cascade  ──┐
                                    │
        generate_auth_nginx_configuration.cfm   (rewrites snippets/auth.conf)
        generate_nginx_configuration.cfm        (rewrites snippets/hermes-ssl.conf)
        generate_authelia_configuration.cfm     (rewrites authelia/configuration.yml)
        generate_nextcloud_configuration.cfm    (rewrites nc/config.php trusted_domains)
        occ user_oidc:provider Hermes_SEG       (discovery URI + end-session URI)
        occ config:app:set external sites       (NC top-bar "User Console" link JSON)
        occ theming:config url                  (NC theming URL)
        edit_ciphermail_settings.cfm            (Ciphermail portal URL)
        restart_authelia.cfm                    (preload-style restart)
        restart_ciphermail.cfm                  (preload-style restart)
        preload_restart_nginx.cfm               (last — see below)
```

`preload_restart_nginx.cfm` is the canonical Hermes pattern for
restarting the proxy from inside a request that is **served by the
proxy**. A plain `docker container restart hermes_nginx` would close
the request's own connection and the browser would see
`ERR_CONNECTION_REFUSED` on the redirect back. The preload page returns
a full HTML response that includes a `fetch()` to a separate
`restart_nginx_post.cfm` endpoint and a poll-loop that waits for nginx
to come back before redirecting to `view_console_settings.cfm`. Always
use this pattern from any handler that ends in an nginx restart.

The Nextcloud `occ` calls in steps 7d–7f are all wrapped in
`<cftry>...<cfcatch type="any"></cfcatch></cftry>` and marked
**non-fatal** in the comments. A Nextcloud container that is down or
slow at the moment of save will leave the NC-side values stale; on the
next save (or a manual `occ` invocation) they will catch up.

> **By design.** The cascade is destructive — there is no dry-run, no
> diff preview, no "stage changes." Saving rewrites all four config
> files and restarts three containers. Plan saves outside business
> hours if the deployment is busy.

## Operational consequence — changing the Console Address mid-flight

A live Console Address change is the single most disruptive operation
on this page. While the cascade runs (typically 30–60 seconds end to
end including container restarts):

| Surface | Behavior during the change |
|---|---|
| The admin page that initiated the change | Held by `preload_restart_nginx.cfm` until nginx returns 200 on `/index.cfm`, then redirected back |
| Other open admin sessions | Will see `502 Bad Gateway` for the nginx restart window; their session cookie is also now scoped to the **old** hostname and they will be re-prompted to log in after they reload at the new address |
| User portal / Nextcloud / Webmail sessions | Same — all session cookies are domain-scoped; users at the old hostname must navigate to the new one and re-authenticate |
| Mail flow (SMTP/IMAP/Submission) | **Unaffected.** Postfix and Dovecot do not depend on the console nginx vhost. |
| Outbound DKIM signing | Unaffected. |
| Webmail OIDC | Discovery URI is rewritten at step 7d but the change only takes effect after Nextcloud picks up the new `occ user_oidc` settings — in practice this is instant because `occ` writes synchronously |

If the new Console Address has no DNS record yet, the change still
saves (Hermes does not DNS-resolve the value) but every external
client request will fail until DNS catches up.

## Bypassing this page — risks

There are three other paths that can change the console hostname or
hostname-derived values **without** going through this cascade. Each
one leaves Hermes in an inconsistent state. Do not use them unless you
are recovering from a broken cascade and you know what you are doing.

1. **Direct edit of `parameters2`** — sets `console.host` but does not
   regenerate `auth.conf`, `hermes-ssl.conf`, `configuration.yml`,
   `config.php`, theming, External Sites, OIDC, or Ciphermail.
2. **Direct edit of `config/nginx/.../snippets/*.conf` or
   `config/authelia/configuration.yml`** — the next save on this page
   overwrites your hand-edits.
3. **A future Hermes CLI Management Console** (`scripts/hermes-cli.sh`)
   is planned but not yet built. It will expose Change Console Host as
   a menu option so admins have a recovery path when a bad Console
   Address change has locked them out of the web UI. Until it ships,
   the only recovery is direct SQL + manual regen-script invocations
   against the `hermes_commandbox` container.

## Per-domain nginx vhosts are NOT regenerated by this page

This page rewrites `snippets/auth.conf` and `snippets/hermes-ssl.conf`
— the global console snippets. **Per-domain vhosts** generated for
mailbox domains, autodiscover, autoconfig, and any other
domain-scoped surface live in separate templates and are rendered on
their own pages (Mailboxes > Domains, mostly).

If you edit one of those per-domain templates by hand and expect
already-generated vhosts to pick it up, they will not. Either re-render
each affected domain from its own UI, or run the appropriate
domain-regen include directly. The same rule applies in reverse — a
console hostname change does **not** rewrite per-domain server blocks
that were generated before the change. Most installs do not need to,
because per-domain vhosts use the domain hostname, not the console
hostname. If a per-domain vhost was unusually wired to the console
hostname (manual customisation), re-render it.

## Failure semantics

| What breaks | What happens |
|---|---|
| Console Address validation fails (invalid IPv4/IPv6/FQDN) | `session.m = 3`, redirect, no DB write |
| Console Certificate ID not found in `system_certificates` | `session.m = 2`, redirect, no DB write |
| Nginx config syntax error after template substitution | `nginx -t` fails inside `restart_nginx_post.cfm`; the previous live config stays loaded (nginx never gets the `reload`), but the on-disk file is the broken one. Recovery: fix the template, re-save. |
| Authelia container fails to start after `configuration.yml` regen | See [Authentication Settings § Failure semantics](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings#failure-semantics). The `restart_authelia.cfm` output is logged but not surfaced in the success banner. |
| Nextcloud `occ` calls error out | Logged silently (cftry wrapping); next save retries. |
| Ciphermail not running | The portal URL stays out of sync; next save catches up after the container is back. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_console_settings.cfm` | `hermes_commandbox` | Page |
| `config/hermes/var/www/html/admin/2/inc/edit_console_settings.cfm` | `hermes_commandbox` | Save handler (7-step cascade) |
| `config/hermes/var/www/html/admin/2/inc/get_console_settings.cfm` | `hermes_commandbox` | Load handler |
| `config/hermes/var/www/html/admin/2/preload_restart_nginx.cfm` | `hermes_commandbox` | Restart-and-redirect overlay |
| `config/hermes/opt/hermes/templates/hermes-ssl.conf` | `hermes_commandbox` | Console nginx server-block template |
| `config/hermes/opt/hermes/templates/auth.conf` | `hermes_commandbox` | Console auth_request snippet template |
| `config/hermes/opt/hermes/templates/configuration.yml` | `hermes_commandbox` | Authelia config template |
| `/etc/nginx/snippets/hermes-ssl.conf` | `hermes_nginx` | Live console TLS / hardening snippet (regen target) |
| `/etc/nginx/snippets/auth.conf` | `hermes_nginx` | Live console auth_request snippet (regen target) |
| `/config/configuration.yml` | `hermes_authelia` | Live Authelia config (regen target) |
| `/var/www/html/config/config.php` | `hermes_nextcloud` | Live NC config — `trusted_domains` updated (regen target) |
| `oc_appconfig` (appid `external`, configkey `sites`) | `hermes_nextcloud` MariaDB | Top-bar User Console link JSON blob |
| `oc_appconfig` (appid `theming`, configkey `url`) | `hermes_nextcloud` MariaDB | NC theming URL |
| `user_oidc` provider `Hermes_SEG` | `hermes_nextcloud` | OIDC discovery + end-session URIs |

Every cross-container call uses `docker exec` per the standard Hermes
pattern. The temp-shell-script convention (`/opt/hermes/tmp/<token>_*.sh`)
is used for the External Sites `occ` call because the JSON value has
quoting/escaping that `cfexecute`'s `arguments` parsing handles
unreliably; writing a small shell script and executing it instead of
passing the JSON inline avoids that whole class of bug.

## Related

- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — the mail-side server identity (Postfix `myorigin` / `myhostname`, Host IP). Companion to this page; the two together define every name Hermes presents.
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — uploading, renewing, and managing the certificates this page selects from
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) — Authelia configuration; this page rewrites its config file as part of every save
- [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) — the mail-side TLS certificate binding, the analogue of "Console Certificate" for SMTP banners
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — if the Console Address is an internal-only FQDN, this page's resolver settings determine whether other Hermes containers can reach it

# DNS Resolver

# DNS Resolver

Admin path: **System > DNS Resolver** (`view_dns_resolver.cfm`,
`inc/dns_resolver_action.cfm`, `inc/generate_unbound_forward_conf.cfm`,
`inc/generate_unbound_local_conf.cfm`).

Hermes ships its own **recursive caching DNS resolver** — a stock
`hermes_unbound` container fronted by an admin UI that lets an operator
toggle recursive vs. forwarding mode, manage upstream forwarders, add
local-zone overrides for split-horizon hostnames, inspect cache
statistics, and run ad-hoc lookups. Every other Hermes container points
its `dns:` at `hermes_unbound` (`${IPV4SUBNET}.117`) rather than the
host's resolver, so RBL/DNSBL lookups, MX resolution, ARC verification,
Postfix recipient validation, and OIDC discovery all flow through this
single resolver.

## Why Hermes runs its own resolver

A mail gateway has DNS requirements that a stock host resolver does not
meet:

| Requirement | Why a shared resolver fails | What Unbound gives Hermes |
|---|---|---|
| RBL/DNSBL queries from a low-volume IP | Public resolvers (Cloudflare, Google, Quad9) issue thousands of queries per second on behalf of many tenants. RBL providers throttle or refuse responses to those shared IPs. | Recursive mode queries the authoritative servers directly from the gateway's own IP — well under any per-source rate limit. |
| Deterministic resolution path for DKIM / DMARC / ARC | A flaky host resolver causes intermittent `TEMPFAIL` on DNS-dependent auth | Unbound's cache survives container restarts of the consumers, and its TTLs are tuned for mail traffic |
| Split-horizon DNS (internal AD hostnames) | The host's `/etc/resolv.conf` typically points at public DNS — internal-only names fail | The Local DNS Overrides table writes `local-data` entries that Unbound returns for any container that asks |
| DNSSEC validation across the stack | Trust depends on every container running its own validator (rarely the case) | Unbound validates once; consumers get verified answers automatically |

The container itself is custom-built (Hermes-published image at
`ghcr.io/deeztek/hermes-unbound`) but the configuration is plain
Unbound — there is no Hermes-specific patching at the daemon level.

## How DNS flows through the stack

```
+-------------------+   +-------------------+   +-------------------+
| hermes_postfix    |   | hermes_mail_filter|   | hermes_ldap       |
| (RBL, MX lookups) |   | (SpamAssassin)    |   | (RemoteAuth bind) |
+---------+---------+   +---------+---------+   +---------+---------+
          |                       |                       |
          | dns: 172.16.32.117    |                       |
          v                       v                       v
+-----------------------------------------------------------------+
|  hermes_unbound  (.117 on hermes_net_ext, port 53/udp + 53/tcp) |
|                                                                  |
|   /etc/unbound/unbound.conf       <-- baseline (read-only mount) |
|   /etc/unbound/conf.d/forward.conf <-- generated from DB         |
|   /etc/unbound/conf.d/local.conf   <-- generated from DB         |
|                                                                  |
|   Forwarding mode? ----yes----> upstream forwarders (1.1.1.1 ...)|
|                  ----no -----> root hints, full recursion        |
+-----------------------------------------------------------------+
                              |
                              v
                   Authoritative DNS / Forwarders
```

Every container declares `dns: ${IPV4SUBNET}.117` in
[`docker-compose.yml`](https://github.com/deeztek/Hermes-Secure-Email-Gateway/blob/main/docker-compose.yml) so its `/etc/resolv.conf`
points at the Unbound container regardless of the host's resolver
configuration. The host itself is unaffected.

## Configuration storage

Forwarding mode and the forwarders/local-records lists live in three
places:

| Setting | Storage | Notes |
|---|---|---|
| Forwarding mode | `parameters2.module = 'unbound', parameter = 'forwarding.enabled'` | `yes` or `no` |
| Upstream forwarders | `dns_forwarders` table | `server`, `port`, `tls`, `enabled`, `sort_order`; seeded with Cloudflare (1.1.1.1 / 1.0.0.1) + Google (8.8.8.8 / 8.8.4.4) |
| Local DNS overrides | `dns_local_records` table | `hostname`, `record_type` (A/AAAA/CNAME/MX/TXT/PTR), `value`, `enabled`, `description`; UNIQUE on `(hostname, record_type)` |

The baseline `unbound.conf` (cache sizes, DNSSEC trust anchor, num-threads,
access-control for the Docker subnets) ships as a read-only mount and is
not editable from this page. To change those, edit
`config/unbound/unbound.conf` directly and restart the container.

## Recursive vs. forwarding mode

The default is **Recursive** and the in-page callout pushes hard against
flipping it. The reasoning is operational, not philosophical:

> **Forwarding through public resolvers will cause RBL/DNSBL lookup
> failures.** When queries are forwarded through Cloudflare / Google /
> Quad9, your blocklist lookups originate from their shared IP
> addresses. RBL providers throttle or block these IPs because thousands
> of other customers are making the same queries from the same
> resolvers. With recursive resolution, queries come from your server's
> own IP, keeping you well under per-source rate limits.

Forwarding is still useful in a few specific cases:

- Egress-restricted networks where outbound port 53 to arbitrary
  authoritative servers is blocked but a known forwarder is allowed
- Compliance requirements forcing all DNS through a logged corporate
  resolver
- DNS-over-TLS to a specific provider (set `tls = yes` and `port = 853`
  on each forwarder)

In any of those cases, configure forwarders that you control or that
have a known per-customer SLA. Public flat-rate resolvers cause RBL
breakage that surfaces days later as inflated spam scores.

## The four cards on the page

### 1. DNS Resolver Status

Shows container state (`running`, `exited`, or `error`) via
`docker inspect --format='{{.State.Status}}|{{.State.StartedAt}}'`,
computes the uptime in days/hours/minutes (the StartedAt timestamp is
UTC; the page converts before diffing — earlier versions had a tz-drift
bug, see commit `644d56b1`), and exposes a **Restart Unbound** button.

> **Restarts are mail-safe.** Restarting `hermes_unbound` typically takes
> 1–3 seconds. During that window, consumer containers fall back to
> retry; Postfix, Amavis, and Dovecot all tolerate a brief DNS outage
> without losing mail. Plan restarts freely; you do not need an outage
> window.

### 2. DNS Forwarding

Two sub-controls. The **DNS Resolution Mode** select (recursive vs.
forwarding) writes `parameters2.unbound.forwarding.enabled` and
regenerates `forward.conf`. The **Upstream Forwarders** table is the
working set used when forwarding is enabled — fields are Server IP, Port
(default `853` for DoT, `53` for plain), TLS (yes/no), and per-row
enable/disable + delete.

The two-step "edit then Apply" model is deliberate: adding, deleting, or
toggling a forwarder marks the change pending (the page banner shifts to
amber) but does **not** restart Unbound. Click **Apply &amp; Restart
Unbound** to regenerate `forward.conf` and bounce the container in one
shot. This lets an admin batch a multi-row change without triggering
multiple restarts.

### 3. Local DNS Overrides

A static-entries table that becomes `local-data` lines in
`/etc/unbound/conf.d/local.conf`. The same two-step
edit-then-Apply model applies.

This is the single most operationally important card on the page. Two
canonical use cases:

| Scenario | What to add |
|---|---|
| LDAP RemoteAuth against an internal AD DC (`dc01.corp.example.com`) that is not publicly resolvable | `dc01.corp.example.com` → `10.0.0.10` (A record). See [LDAP RemoteAuth § DNS resolution prerequisite](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth#dns-resolution-prerequisite). |
| Split-horizon: the Console Address resolves externally but you want internal containers to skip the public lookup | `console.example.com` → `192.168.1.10` (A record) |

The generator groups records by their second-level zone and emits a
single `local-zone: "<zone>." transparent` declaration before the
`local-data` lines — `transparent` means Unbound resolves the configured
hostnames locally but **forwards everything else in the same zone**
upstream as normal. This is the right choice for split-horizon: an
override for `dc01.example.com` does not break public lookups for
`www.example.com` against the same zone.

> **Operational consequence.** A misconfigured override can shadow a
> public hostname. Hermes resolves what you write — if you point
> `mail.example.com` at the wrong internal IP, every container that asks
> for that name will get the wrong answer. Test with the **DNS Lookup
> Test** card (below) before relying on the entry in production.

### 4. DNSSEC, Cache Statistics, DNS Lookup Test

Three read-only utility cards.

| Card | What it shows / does |
|---|---|
| **DNSSEC** | Parses the live `unbound.conf` inside the container; reports Enabled / Disabled based on `auto-trust-anchor-file` / `trust-anchor-file` / `module-config: validator` presence. **Test DNSSEC** runs `drill -D example.com` and dumps the response. DNSSEC is enabled in the shipped baseline; this card is informational. |
| **Cache Statistics** | Runs `unbound-control stats_noreset` and parses `total.num.queries`, `cachehits`, `cachemiss`, `prefetch`, plus RRset/message cache counts and average recursion time. Useful for diagnosing cold-cache latency after a restart. **Flush Cache** clears the entire cache (`unbound-control flush_zone .`) — typically used after a downstream DNS record change that you don't want to wait for the TTL on. |
| **DNS Lookup Test** | Runs `drill @127.0.0.1 <TYPE> <name>` inside the container. Supports A / AAAA / MX / TXT / NS / SOA / PTR. Input is validated to `[a-zA-Z0-9.\-]+` before being passed to the shell. This is the right tool to verify a local override actually took effect. |

## Apply flow

A single Save / Apply click runs roughly this:

```
1. Validate input (IP octets in range, port 1-65535, hostname charset, ...)
2. UPDATE or INSERT INTO parameters2 / dns_forwarders / dns_local_records
3. cfinclude generate_unbound_forward_conf.cfm   (or _local_conf.cfm)
        - Read the table back
        - Render the conf into chr(10)-newline plain text
        - fileWrite("/etc/unbound/conf.d/forward.conf", ..., "utf-8")
4. cfexecute /usr/local/bin/docker container restart hermes_unbound
        (30s timeout; typically returns in 1-3s)
5. cflocation back to view_dns_resolver.cfm with session.m set
```

The generated `conf.d/*.conf` files are written via Lucee `fileWrite`
into the `hermes_commandbox`-side bind-mount of `config/unbound/conf.d/`
— the same directory `hermes_unbound` reads on restart. There is no
`docker cp` step; both containers see the same files because they share
the bind mount (commit `06acd4e1` switched away from the legacy `docker
cp` pattern).

## Cache TTL behavior

The baseline `unbound.conf` sets:

| Knob | Value | Why |
|---|---|---|
| `cache-min-ttl: 300` | 5 minutes | Floor — protects against authoritative servers that publish ultra-short TTLs |
| `cache-max-ttl: 86400` | 24 hours | Ceiling |
| `cache-max-negative-ttl: 900` | 15 minutes | Floor on NXDOMAIN cacheing — important for DNSBL hits, which produce intentional NXDOMAINs |
| `prefetch: yes` | — | Refreshes hot records before TTL expiry so cache misses are rare |
| `qname-minimisation: yes` | — | Privacy + reduces authoritative-server query volume |

After a record change you depend on (e.g., updating an MX record at the
registrar), use **Flush Cache** to skip the wait.

## Failure semantics

| What breaks | What happens |
|---|---|
| `dns_forwarders.server` validation fails (non-IPv4, octet > 255, port out of range) | `session.m = 11`, redirect, no DB write. Error text in the alert. |
| `dns_local_records.hostname` empty or invalid record type | Same — `session.m = 11` with specific error text. |
| `fileWrite` to `conf.d/` fails | `session.m = 10`, error surfaces. The container is **not** restarted; the previous `.conf` stays live. |
| Container restart times out (30s) | `session.m = 10`. The restart was issued but did not complete in band; check `docker ps` and `docker logs hermes_unbound` manually. |
| `unbound-control` not available | Cache Statistics card shows "not available" message; the daemon itself is unaffected. |
| `drill` returns SERVFAIL for a DNSSEC test | Surfaced in the test output pane; usually means the test domain has misconfigured DNSSEC, not that Unbound is broken. |
| Local override shadows a public name | No error — Unbound returns the override. Use the DNS Lookup Test card to verify what consumers will actually see. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_dns_resolver.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/dns_resolver_action.cfm` | `hermes_commandbox` | Save handlers (`save_forwarding`, `add_forwarder`, `add_local_record`, `restart_unbound`, `flush_cache`, etc.) |
| `config/hermes/var/www/html/admin/2/inc/generate_unbound_forward_conf.cfm` | `hermes_commandbox` | Renders `forward.conf` from DB and restarts the container |
| `config/hermes/var/www/html/admin/2/inc/generate_unbound_local_conf.cfm` | `hermes_commandbox` | Renders `local.conf` from DB and restarts the container |
| `config/unbound/unbound.conf` | `hermes_unbound` (read-only mount) | Baseline daemon config — cache sizes, DNSSEC, access-control |
| `config/unbound/conf.d/forward.conf` | `hermes_unbound` (read-write mount, regen target) | Generated forwarders |
| `config/unbound/conf.d/local.conf` | `hermes_unbound` (read-write mount, regen target) | Generated local overrides |
| `dns_forwarders`, `dns_local_records` tables | `hermes_db_server` (`hermes` DB) | Source of truth for the regen |
| `parameters2.unbound.forwarding.enabled` | `hermes_db_server` (`hermes` DB) | Recursive vs. forwarding mode |
| `${IPV4SUBNET}.117` | Docker network `hermes_net_ext` | Fixed Unbound IP that every other container's `dns:` declaration points at |

## Related

- [LDAP RemoteAuth § DNS resolution prerequisite](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth#dns-resolution-prerequisite) — the canonical case for adding a Local DNS Override (internal AD DC hostname)
- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — if the Console Address is an internal-only FQDN, this page's overrides decide whether other containers can reach it
- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — the mail-side hostname; RBL accuracy depends on the resolver's egress IP, which is the host's egress IP regardless of where Unbound is running
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) — the Ofelia jobs (RBL refresh, DMARC report fetch, fangfrisch malware-feed sync) that depend on this resolver
- [Storage Topology](https://docs.deeztek.com/books/installation-reference/page/storage-topology-5-tiers) — `hermes_unbound` is stateless; its mounts live in the Config tier (`config/unbound/`)

# IPS

# IPS

_Pro Edition feature._ Maps to **System > IPS** (`view_intrusion_prevention.cfm`, `inc/intrusion_prevention_generate_config.cfm`, `inc/intrusion_prevention_get_status.cfm`, `inc/intrusion_prevention_manual_ban.cfm`, `inc/intrusion_prevention_manual_unban.cfm`).

IPS (Intrusion Prevention System) is Hermes's brute-force defense layer. It binds two operational pieces together: the **`hermes_fail2ban`** container that scans authentication logs and inserts iptables drop rules, and a Hermes database/UI layer that lets an admin tune jail thresholds, manage a never-ban whitelist, manually ban or unban IPs, and see live ban counts. The page also doubles as a troubleshooting reference (the Info card lists every `docker exec` command an admin would need to chase a ban from the shell).

## Pipeline placement — where IPS sits in the stack

```
Attacker on the public internet
        │
        ▼
   Host network stack  (hermes_fail2ban runs network_mode: host)
        │
        ├─►  iptables DOCKER-USER chain
        │       └─►  f2b-dovecot, f2b-authelia chains  ◄── ban rules inserted here
        │
        ▼
   nginx / Docker bridge
        │
        ▼
   hermes_nginx ──► hermes_commandbox / hermes_authelia / hermes_dovecot
                              │
                              ▼  (auth attempt logged)
                    /remotelogs/<service>/<file>.log
                              ▲
                              │
   hermes_fail2ban  ─tails─►  same logs (bind-mounted into the container)
        │
        ├─►  match filter regex N times within findtime
        ▼
   hermes-iptables-<jail> action
        │
        ├─►  iptables -I f2b-<jail> -s <ip> -j DROP
        └─►  hermes-api-notify.sh BAN <ip> <SOURCE>
                  │
                  ▼
            POST http://<commandbox>:8888/hermes-api/
                  │
                  ▼
            INSERT INTO fail2ban_ips (...)
```

Two facts are worth pinning down before anything else:

| Fact | Consequence |
|---|---|
| `hermes_fail2ban` runs in **host network mode** | iptables rules apply to the Docker host directly, not to a bridge namespace. The DOCKER-USER chain is the entry point because Docker honors it before its own auto-inserted rules. |
| Docker DNS is **unavailable** inside the container | The notify script reads container IPs from `/opt/hermes/tmp/container_ips.env`, regenerated on every page load by `inc/generate_container_ips.cfm`. If that file is stale or missing, ban events still iptables-block correctly but fail to log to the database. |

## The container always runs — Pro gating is behavioral

`hermes_fail2ban` starts on every install regardless of edition. The Pro license check happens in CFML at page load, not at the container level. What changes on Community is:

- The configuration UI is replaced by the standard "Pro feature required" panel.
- Jail toggles in `intrusion_prevention_jails.enabled` and the master `intrusion_prevention_settings.enabled` switch default to disabled on a fresh install.
- The jail.local on disk reflects whatever the seed gave you; nothing rewrites it without an admin clicking through the page.

> **Operational consequence.** Stopping `hermes_fail2ban` to "turn off IPS on Community" is the wrong move. The container is needed for the schema, the include scripts, and the manual-unban API path. Leave it running; disable IPS through the UI when the page becomes accessible, or leave the seeded jails disabled.

## The two seeded jails

| Jail name | Display name | Log scanned | Filter | Action | Default thresholds |
|---|---|---|---|---|---|
| `dovecot` | Mail Server (Dovecot) | `/remotelogs/dovecot/dovecot-info.log` | `dovecot` (upstream Fail2ban filter) | `hermes-iptables-dovecot` | maxretry 5 / findtime 86400 (1 day) / bantime 1800 (30 min) |
| `authelia` | SSO Portal (Authelia) | `/remotelogs/authelia/authelia.log` | `authelia-auth` (Hermes-shipped) | `hermes-iptables-authelia` | maxretry 5 / findtime 86400 / bantime 1800 |

Both rows are seeded into `intrusion_prevention_jails` on install (see [hermes_install.sql](https://github.com/deeztek/Hermes-Secure-Email-Gateway/blob/main/config/database/hermes_install.sql) lines 845-846). Adding a third jail is a schema-row plus filter/action insertion exercise — there is no UI for it. The two-jail set covers the two real attack surfaces in Hermes: SMTP/IMAP login brute force and the web-console SSO login. Postfix's own brute-force protection (smtpd anvil rate limits) is the first line of defense for SMTP submission; this jail catches what gets past anvil.

The dovecot jail covers the `dovecot-info.log` line for failed authentication, not the Postfix auth log. SMTP-AUTH attempts terminate against Dovecot SASL — Postfix proxies SASL through Dovecot — so the dovecot filter sees both IMAP/POP and SMTP-AUTH failures from the same surface.

## Database schema

Three tables in the `hermes` database carry IPS state. A fourth (`fail2ban_ips`) is shared with the manual ban/unban flow and the API notify script.

| Table | Role | Notes |
|---|---|---|
| `intrusion_prevention_settings` | Two key/value rows: `enabled` (master switch), `config_synced` (pending-changes flag) | INSERT IGNORE on install, so an admin's local tuning survives upgrades |
| `intrusion_prevention_jails` | One row per jail with display metadata + maxretry/findtime/bantime/enabled/config_synced | Includes the filter and action names that get baked into `jail.local` |
| `intrusion_prevention_whitelist` | One row per IP/CIDR to ignore — three protected entries (`127.0.0.1/8`, `::1`, `172.16.0.0/12`) cannot be deleted | Whitelist rows render into the `ignoreip` directive of `[DEFAULT]` in `jail.local` |
| `fail2ban_ips` | Live ban ledger — one row per (IP, jail) pair currently or recently banned | Written by `hermes-api-notify.sh` (automatic bans) or the CFML manual-ban handler (admin bans) |

The `config_synced` flag works the same way as on other pages: every write handler flips it to `0` and renders a yellow "Pending Changes" badge; **Apply Settings** runs the regen-and-reload sequence and flips it back to `1`. There is no incremental sync — every Apply rewrites the whole `jail.local` from scratch.

## Apply Settings — the regen sequence

`inc/intrusion_prevention_generate_config.cfm` runs five hard-sequenced steps:

1. **Read** `intrusion_prevention_whitelist` (excluding the three protected IPs to avoid double-listing them in `ignoreip`).
2. **Read** `intrusion_prevention_jails` ordered by `jail_name`.
3. **Render** `jail.local` content into a `<cfsavecontent>` block: `[DEFAULT]` with `ignoreip = 127.0.0.1/8 ::1 172.16.0.0/12 <user-whitelist>`, then a `[<jail_name>]` stanza per row.
4. **Write** the rendered config to `/opt/hermes/tmp/jail.local.tmp` (a shared host path mounted into both containers), then `docker exec hermes_fail2ban cp` it into `/config/fail2ban/jail.local` inside the fail2ban container. The two-step copy is required because the `hermes_commandbox` container can't write directly to fail2ban's `/config` mount.
5. **Reload** with `docker exec hermes_fail2ban fail2ban-client reload`, then flip both `intrusion_prevention_settings.config_synced` and every row's `intrusion_prevention_jails.config_synced` to `1`.

If any step fails, `ipSyncSuccess` stays `false`, the sync flags are **not** flipped, and the page surfaces the error banner from `cfcatch.message`. The next attempt retries from scratch — there is no half-applied state to clean up.

## What happens when IPS is disabled

The master `enabled = 0` toggle does two things synchronously, before the redirect:

1. Walks every enabled jail, runs `fail2ban-client status <jail>` to get the live banned IP list, then `fail2ban-client set <jail> unbanip <ip>` for each one. iptables drop rules are removed immediately.
2. Truncates `fail2ban_ips` so the DB ledger matches the now-empty iptables state.

After that, Apply Settings rewrites `jail.local` with `enabled = false` on every jail and reloads fail2ban — meaning **no new bans will be created**, and any in-flight attacker is immediately ungated. This is the right behavior for an emergency "I locked myself out" scenario, but the price is loss of the entire current ban list. Re-enabling does not restore prior bans.

## The IP Whitelist

Whitelist entries are static CIDR ranges that fail2ban's `ignoreip` directive treats as never-banable. The page accepts:

| Format | Example | Validation |
|---|---|---|
| IPv4 single | `192.168.1.100` | `inc/validate_ip_address.cfm` regex |
| IPv4 CIDR | `10.0.0.0/8` | IPv4 regex + numeric prefix 0–32 |
| IPv6 single | `::1` | `inc/validate_ip_address_ipv6.cfm` regex |
| IPv6 CIDR | `fe80::/10` | IPv6 regex + numeric prefix 0–128 |

The three protected entries (localhost v4, localhost v6, the Docker `172.16.0.0/12` block) are seeded on install and the delete handler refuses to remove them. The `172.16.0.0/12` entry exists because internal container-to-container traffic shows up in dovecot/authelia logs as coming from the Docker bridge — without it, an Authelia auth_request loop or a Dovecot LMTP redelivery could end up self-banning the gateway. The lock icon on those rows in the table reflects this.

## Manual Ban and Manual Unban

The Banned IPs card surfaces every row in `fail2ban_ips`, joined to `intrusion_prevention_jails` so the display picks up the friendly name and the bantime for the countdown column. Two admin actions sit on top of it:

### Manual Ban

`inc/intrusion_prevention_manual_ban.cfm` accepts an IP and a jail (or "ALL" to span every enabled jail). For each target jail:

1. Pre-check `fail2ban_ips` for an existing (IP, jail) row — skip if already banned in that jail.
2. Run `docker exec hermes_fail2ban fail2ban-client set <jail> banip <ip>`. Return value 1 (or "already banned" in the output) is treated as success.
3. Sleep 500 ms so the fail2ban action's `hermes-api-notify.sh` invocation has time to insert the row first.
4. `UPDATE fail2ban_ips SET ban_type='MANUAL', ban_source='ADMIN', note='Manually banned via Intrusion Prevention GUI' WHERE ip=... AND jail=...` — overwriting the AUTOMATIC row the notify script just inserted.

The 500 ms sleep is load-bearing: without it, the notify-script INSERT can race the manual UPDATE and the admin attribution is lost.

### Manual Unban

`inc/intrusion_prevention_manual_unban.cfm` accepts pipe-delimited `<ip>|<jail>` pairs from the checkbox row selection, runs `fail2ban-client set <jail> unbanip <ip>` for each pair, and deletes the matching row from `fail2ban_ips`. Errors from individual unbans don't abort the batch — the script counts successes and reports failures separately.

Manual bans are flagged as **Permanent** in the time-remaining column because they have no `bantime` from a jail — the absence of an automatic expiry is the whole point of a manual ban. The admin must explicitly unban them.

## The countdown timer

The Banned IPs DataTable renders a per-row countdown badge using the `banned_at + bantime` arithmetic done CFML-side, then a `data-unban-timestamp` attribute drives a 1-Hz JavaScript tick that recolors the badge as it counts down (yellow > red > expired). The countdown is purely cosmetic — the actual unban happens inside fail2ban's process based on the same arithmetic. If a row shows "Expired" but is still present, it just hasn't been reaped from `fail2ban_ips` yet; reload the page after a few seconds and it'll be gone.

## Operational truths about iptables backends

Modern Ubuntu hosts ship two iptables binaries: `iptables-legacy` (xtables / kernel `xt_*` modules) and `iptables-nft` (nftables backend with iptables-compatible CLI). The fail2ban container ships both. The page surfaces both command variants in the Info card precisely because the right one depends on which backend the host (and Docker) negotiated at install time:

```
docker exec hermes_fail2ban update-alternatives --display iptables
```

Picking the wrong one isn't catastrophic — it just shows empty chains, which can be confusing during a "why isn't my ban working?" investigation. The `hermes-iptables-*` action templates inside fail2ban use the alternatives-resolved `iptables` binary, so the daemon itself always picks the correct backend.

## License gating

The page is wrapped in the standard Pro check:

```cfml
<cfif NOT isDefined("session.edition") OR session.edition NEQ "Pro">
    <cfset proFeatureName = "Intrusion Prevention">
    <cfinclude template="./inc/license_pro_required.cfm">
    <cfabort>
</cfif>
```

Community installs see the gating panel and cannot reach the UI. The `hermes_fail2ban` container continues to run, its seeded jails default to disabled, and `jail.local` on disk reflects whatever was last applied. There is no behind-the-scenes auto-disable on license-state change — switching from Pro to Community does not flip jails off.

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_intrusion_prevention.cfm` | `hermes_commandbox` | Main page (cards, modals, action handlers) |
| `config/hermes/var/www/html/admin/2/inc/intrusion_prevention_generate_config.cfm` | `hermes_commandbox` | Render `jail.local` + reload fail2ban |
| `config/hermes/var/www/html/admin/2/inc/intrusion_prevention_get_status.cfm` | `hermes_commandbox` | Live `fail2ban-client status` parsing for jail/ban counters |
| `config/hermes/var/www/html/admin/2/inc/intrusion_prevention_manual_ban.cfm` | `hermes_commandbox` | Multi-jail manual ban with API-notify race handling |
| `config/hermes/var/www/html/admin/2/inc/intrusion_prevention_manual_unban.cfm` | `hermes_commandbox` | Batch unban handler |
| `config/hermes/var/www/html/admin/2/inc/generate_container_ips.cfm` | `hermes_commandbox` | Writes `/opt/hermes/tmp/container_ips.env` for the notify script |
| `config/hermes/var/www/html/admin/2/inc/fail2ban_ban_unban.cfm` | `hermes_commandbox` | API endpoint hit by `hermes-api-notify.sh` (token-authed) |
| `config/fail2ban/config/fail2ban/jail.local` | `hermes_fail2ban` (mounted) | Live jail config — rewritten on every Apply |
| `config/fail2ban/scripts/hermes-api-notify.sh` | `hermes_fail2ban` | Posts ban/unban events back to Hermes API |
| `config/fail2ban/scripts/detect-iptables-backend.sh` | `hermes_fail2ban` | One-shot at container start to pick legacy vs nft |
| `/opt/hermes/tmp/jail.local.tmp` | both | Ephemeral rendered config; `docker exec cp`-ed into the fail2ban mount |
| `/opt/hermes/tmp/container_ips.env` | both | DB and Commandbox IPs for the API notify script (host networking has no DNS) |

## Related

- [Admin Console Firewall](https://docs.deeztek.com/books/administrator-guide/page/console-firewall) — the complementary static-allowlist layer; IPS is reactive, Console Firewall is preventative
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) — Authelia's own Login Regulation (per-account brake) — the inner brake that complements this page's per-source-IP brake
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — RemoteAuth-mode users also count against the authelia jail
- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — changing the console host triggers a full nginx regen but does not touch fail2ban

# LDAP RemoteAuth

# LDAP RemoteAuth

_Pro Edition feature._ Maps to **System > LDAP RemoteAuth** (`view_remoteauth.cfm`, `edit_remoteauth_mapping.cfm`).

RemoteAuth lets Hermes authenticate selected users against an **upstream LDAP or Active Directory** server instead of storing their password in Hermes's own OpenLDAP. The page configures the upstream-to-domain mapping, global TLS settings, a one-shot bind test, and the apply-to-LDAP sync. Active Directory, OpenLDAP, 389 Directory Server, and FreeIPA are all supported through the same plumbing.

## What RemoteAuth is — and isn't

| Is | Isn't |
|---|---|
| A pass-through bind: at web login, Hermes binds against the upstream DN with the supplied password and accepts or rejects accordingly | A directory sync. Hermes does not import users, groups, photos, or attributes from upstream. |
| Per-user opt-in, via `auth_type = 'remote'` + `remoteauth_domain` on the recipient/system-user row | A whole-installation toggle. Local-auth and remote-auth users coexist in the same directory and the same UI. |
| Implemented as an **OpenLDAP `remoteauth` overlay** in Hermes's `hermes_ldap` container | A reinvented bind proxy. The heavy lifting is `slapo-remoteauth(5)` against a stub user with a `seeAlso` pointer. |
| The credential path for **web login only** — `/users`, `/nc`, `/admin` (via Authelia → LDAP bind) | The credential path for **IMAP/SMTP/CalDAV/CardDAV**. Those continue to authenticate against Hermes-issued app passwords; see [Credential Model](https://docs.deeztek.com/books/administrator-guide/page/credential-model) for the full picture. |

> **Operational consequence.** A remote-auth user's mail-client / DAV passwords still live in Hermes (`app_passwords` table, hashed). The upstream directory password is never exposed to Dovecot or Nextcloud DAV — only to the web gate. If the customer's IT team rotates the upstream password, the user's app passwords keep working until they are explicitly revoked. This is by design (see [Credential Model § Local-auth users vs. remote-auth users](https://docs.deeztek.com/books/administrator-guide/page/credential-model#local-auth-users-vs-remote-auth-users)).

## How it works under the hood

```
Web login (/admin, /users, /nc)
        │
        ▼
   Authelia
        │  LDAP bind to Hermes OpenLDAP
        ▼
hermes_ldap  (slapd)
        │
        │  user entry has seeAlso=<upstream DN>
        │  user entry has associatedDomain=<mapping key>
        │
        ▼
slapo-remoteauth overlay
        │  matches associatedDomain → upstream server URI
        │  rewrites the bind to the seeAlso DN
        ▼
External AD / LDAP server  (customer's DC)
        │
        ▼  bind result returned up the chain
   Authelia decision: PASS or FAIL
```

The overlay is configured in `cn=config` on the `mdb` database. Hermes's CFML never bind-checks the upstream itself at login time — that is the overlay's job. The CFML only **writes** the overlay configuration when an admin clicks **Apply Settings**.

## OpenLDAP remoteauth is a singleton overlay

This is the single most important constraint to understand when reasoning about why the page works the way it does.

| Constraint | Consequence in the UI |
|---|---|
| `slapo-remoteauth` allows **only one overlay instance** per database | All mappings live inside the same overlay |
| `olcRemoteAuthMapping` is multi-valued **but has no equality matching rule** | You cannot `ldapmodify add` a single mapping to an existing overlay. The entire overlay must be rebuilt. |
| `olcRemoteAuthTLS` is a single string applied **to all mappings inside the overlay** | TLS settings (STARTTLS, certificate verification, CA cert path, retry count) are **global**, not per-mapping |

`inc/ldap_remoteauth_sync_all.cfm` therefore implements **full replacement on every save**: delete the existing overlay, rebuild it from `remoteauth_mappings` + `remoteauth_settings`. There is no incremental update path. The page's pending-changes badge reflects this — every edit marks `ldap_synced = 0` on both tables, and **Apply Settings** flips it back to `1` only after the full rebuild succeeds.

### Multiple upstream servers with different CAs

Because TLS is global, an installation that binds to multiple upstream LDAP servers signed by different CAs must upload a **concatenated CA bundle**:

```
cat dc01-ca.pem dc02-ca.pem dc03-ca.pem > ca-bundle.pem
```

The page accepts the bundle as-is in the **CA Certificate** file picker. OpenLDAP walks the bundle when validating any of the configured upstream servers.

## Database schema

Two tables drive the page. Both are in the `hermes` database.

| Table | Role |
|---|---|
| `remoteauth_settings` | Six rows, key/value: `enabled`, `tls_starttls`, `tls_reqcert`, `ca_cert_file`, `retry_count`, `ldap_synced` |
| `remoteauth_mappings` | One row per upstream-LDAP-to-domain mapping (`domain_name` UNIQUE, `server_address`, `server_port`, `remote_dn_pattern`, `description`, `enabled`, `ldap_synced`) |

Two user-bearing tables carry RemoteAuth references:

| Table | Columns | Role |
|---|---|---|
| `recipients` | `auth_type ENUM('local','remote')`, `remoteauth_domain VARCHAR(255)` | Relay recipients can be RemoteAuth-mode |
| `system_users` | `auth_type ENUM('local','remote')`, `remoteauth_domain VARCHAR(255)` | Console admins / reader users can be RemoteAuth-mode |

The `mailboxes` table does **not** carry `auth_type` yet. RemoteAuth-for-mailboxes is planned but not yet wired (see [Future work](#future-work)).

## DN pattern placeholders

The `remote_dn_pattern` column stores the upstream DN with four substitutable tokens. Substitution happens in `inc/ldap_add_user_remoteauth.cfm` at user-create time, baked into the `seeAlso` attribute on the local stub entry.

| Token | Source | Notes |
|---|---|---|
| `{username}` | Local part of email (`jsmith@company.com` → `jsmith`) — uses `ListFirst(..., "@")`. For console admins where the username has no `@`, the whole string is used. | Matches `sAMAccountName`/`uid` patterns |
| `{firstname}` | `givenName` field on the add form | Required if the DN pattern uses it |
| `{lastname}` | `sn` field on the add form | Required if the DN pattern uses it |
| `{email}` | Full email address as entered | Useful for `mail=` patterns |

Common patterns the in-page help surfaces:

| Directory type | Pattern |
|---|---|
| AD (display name as CN) | `cn={firstname} {lastname},ou=Users,dc=example,dc=com` |
| AD (sAMAccountName as CN) | `cn={username},ou=Users,dc=example,dc=com` |
| OpenLDAP / FreeIPA | `uid={username},ou=People,dc=example,dc=com` |

The pattern must match the upstream's actual naming convention **exactly**. A wrong pattern produces `ldap_bind: Invalid DN syntax` or `Invalid credentials` at login time; use the **Test** button before saving to confirm.

## The local stub entry

For each RemoteAuth user, Hermes creates a normal `inetOrgPerson + domainRelatedObject` entry in `ou=users,dc=hermes,dc=local` with **no `userPassword` attribute** and the two overlay-driving attributes set:

```
dn: cn=jsmith,ou=users,dc=hermes,dc=local
objectClass: inetOrgPerson
objectClass: domainRelatedObject
givenName: John
sn: Smith
displayName: John Smith
mail: jsmith@company.com
uid: jsmith
seeAlso: cn=John Smith,ou=Users,dc=company,dc=com    <-- expanded from {firstname}/{lastname}/etc.
associatedDomain: company                            <-- the mapping key
```

At bind time the overlay reads `associatedDomain`, looks up the matching `olcRemoteAuthMapping`, opens an LDAP connection to that upstream URI, and re-binds as `seeAlso` with the supplied password. The local entry has no password to validate against, so the overlay's decision is the only decision.

## Test Connection button

The Test modal does **not** consult the saved settings end-to-end — it does its own `ldapwhoami` against the mapping's `server_address:server_port`, applying the same DN pattern substitution the overlay would and honoring the global STARTTLS setting. The credentials entered in the modal are used for one bind attempt:

```
docker exec hermes_ldap ldapwhoami -x -H ldap://<server>:<port> \
    -D "<DN expanded from pattern>" -w "<password>"  [-ZZ if STARTTLS]
```

Success is detected by `dn:` or `u:` in the response. Failure surfaces the raw stderr from `ldapwhoami`. The bind credentials are never stored — they live only for the duration of the request, then disappear.

This is intentionally **separate from the overlay flow**: it lets an admin verify the DN pattern and network path before clicking Apply Settings (which would rebuild the overlay and potentially break live logins).

## DNS resolution prerequisite

The `hermes_ldap` container resolves hostnames through Hermes's own Unbound resolver — by default, public recursive DNS. **Internal-only AD/LDAP hostnames** (typical: `dc01.corp.example.com` on a split-horizon zone) will not resolve, and bind attempts fail with `remoteauth_bind operations error`.

Fix before creating a mapping: add a **DNS Local Record** at **System > [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver)** pointing the upstream FQDN to its actual IP. Verify from inside the container:

```
docker exec hermes_ldap getent hosts <ad-hostname>
```

Publicly-resolvable hostnames don't need this step.

## TLS settings reference

| Setting | Values | Notes |
|---|---|---|
| **Use STARTTLS** | `yes` / `no` | Upgrades the connection on the standard `389` port. Mutually exclusive with LDAPS on `636` (use one or the other). |
| **TLS Certificate Requirement** | `never`, `allow`, `try`, `demand` | Maps directly to `TLS_REQCERT` in the libldap conf. `never` is the only mode that does **not** require a CA cert; the others all expect a valid `ca_cert_file` to compare against. |
| **CA Certificate** | PEM file (`.pem`, `.crt`, `.cer`) | Stored at `/opt/hermes/certs/remoteauth/global_remoteauth_ca.pem` (single canonical filename — uploading replaces). For multi-server installs, concatenate all CAs into a bundle. |
| **Retry Count** | `1`–`10` (default `3`) | Number of bind retries before reporting failure |

The CA field hides itself when `tls_reqcert = never` (purely a UX hint — the file still exists on disk if previously uploaded).

## Apply Settings — the sync flow

Every save handler (`add_mapping`, `update_mapping`, `delete_mappings`, `update_tls_settings`, `set_remoteauth_status`) sets `ldap_synced = 0` on the touched rows AND on `remoteauth_settings`. The page banner switches from green **Synced** to amber **Pending Changes**. Nothing has actually changed in LDAP yet.

**Apply Settings** runs `inc/ldap_remoteauth_sync_all.cfm`, which is a hard three-step sequence:

1. **Delete** the existing overlay (`ldap_remoteauth_delete_overlay.cfm`) — succeeds whether or not one exists.
2. If `enabled = 1` and at least one mapping has `enabled = 1`: **fetch the next overlay index** and the MDB database index (`ldap_remoteauth_get_overlay.cfm`), then **create** the new overlay with all enabled mappings baked in (`ldap_remoteauth_add_overlay.cfm`). The LDIF template is `/opt/hermes/templates/ldap_remoteauth_add_overlay.ldif`, populated via `REReplace` against `THE_OVERLAY_INDEX`, `THE_MDB_INDEX`, `THE_DEFAULT_DOMAIN`, `THE_MAPPING_LINES`, `THE_STARTTLS`, `THE_TLS_REQCERT`, `THE_TLS_CACERT`, `THE_RETRY_COUNT`.
3. **Flip `ldap_synced = 1`** on both tables.

If step 1 or 2 fails, the database `ldap_synced` flags are **not** flipped — the page stays amber, and the next attempt will retry from scratch. There is no half-applied state to clean up because the overlay is rebuilt from zero each time.

> **Failure semantics.** While the overlay is being rebuilt (typically subsecond), live remote-auth web logins will fail with `Operations error` until step 2 completes. Plan Apply Settings during low-login windows. Local-auth users are unaffected.

## Deletion validation

A domain mapping cannot be deleted if any user references it. The check runs against **two** tables at delete time:

```sql
SELECT remoteauth_domain, COUNT(*) FROM system_users
 WHERE auth_type = 'remote' AND remoteauth_domain IN (...);

SELECT remoteauth_domain, COUNT(*) FROM recipients
 WHERE auth_type = 'remote' AND remoteauth_domain IN (...);
```

If either returns rows, the delete is rejected with a list of the blocked domains. The admin must either reassign those users to a different mapping or delete the users first.

> **Known gap (#102 and the mailbox/relay TODO).** When RemoteAuth is extended to **mailboxes** (a planned feature), this validation must add a third query against the `mailboxes` table. Both `view_remoteauth.cfm` (bulk delete, line ~330) and `edit_remoteauth_mapping.cfm` (single delete, line ~129) need to be updated together — they implement the check independently.

## Adding RemoteAuth users in bulk — CSV format

`add_internal_recipients.cfm` (Relay Recipients > Add) supports a RemoteAuth dropdown when the page detects an enabled mapping. When the selected mapping's DN pattern uses `{firstname}` or `{lastname}`, the textarea switches to **CSV mode** because email-only input doesn't carry enough data to expand the pattern.

| DN pattern tokens used | Textarea format |
|---|---|
| `{username}` and/or `{email}` only | One email address per line |
| Includes `{firstname}` or `{lastname}` | `First,Last,Email` per line — one recipient per row |

Header rows (`"GivenName","Surname","Mail"`) are auto-detected and skipped. Unknown columns are ignored, so common export formats work as-is:

- **PowerShell**: `Get-ADUser -Filter * -Properties GivenName,Surname,Mail | Select GivenName,Surname,Mail | Export-Csv users.csv -NoTypeInformation`
- **CSVDE** (Windows Server built-in): `csvde -f users.csv -l "givenName,sn,mail"`
- **Excel / manual**: three columns saved as CSV

Each row is inserted with `auth_type = 'remote'` and `remoteauth_domain = <mapping key>`. The local LDAP stub is created via `ldap_add_user_relay_remoteauth.cfm`, which calls the same template/placeholder machinery described above. A welcome email is sent via `send_recipient_welcome_email_remoteauth.cfm` — the message tells the user to sign in with their **organization (AD/LDAP) password**, not a Hermes-issued one.

## Status, enable, disable

The **RemoteAuth Status** dropdown (`enabled = 0/1`) is the master switch. Disabling does **not** delete the overlay's mappings — it just causes the next Apply Settings cycle to skip step 2 entirely, leaving the overlay absent. Re-enabling and re-applying rebuilds it from the same `remoteauth_mappings` rows. This is useful for emergency cutover back to a local-only state without losing the mapping configuration.

The **LDAP Overlay** badge on the page reads the live state from `cn=config` (via `ldapsearch -Y EXTERNAL` against `(objectClass=olcRemoteAuthCfg)`) and reports **Active** or **Not configured**. This is independent of the DB-side `enabled` flag — if the two disagree (e.g., DB says enabled but the badge says Not configured), the next Apply Settings will reconcile.

## License gating

The page is wrapped in the standard Pro-only guard:

```
<cfif NOT isDefined("session.edition") OR session.edition NEQ "Pro">
    <cfinclude template="./inc/license_pro_required.cfm">
    <cfabort>
</cfif>
```

Community-edition installs see the standard "Pro feature required" panel and cannot reach the configuration UI. Pre-existing RemoteAuth-mode users continue to authenticate (the overlay itself is in `cn=config` and not license-checked), but no new mappings can be added or edited until a Pro license is activated.

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_remoteauth.cfm` | `hermes_commandbox` | Main page |
| `config/hermes/var/www/html/admin/2/edit_remoteauth_mapping.cfm` | `hermes_commandbox` | Edit single mapping |
| `config/hermes/var/www/html/admin/2/inc/ldap_remoteauth_sync_all.cfm` | `hermes_commandbox` | Apply Settings orchestrator |
| `config/hermes/var/www/html/admin/2/inc/ldap_remoteauth_add_overlay.cfm` | `hermes_commandbox` | LDIF render + `ldapadd` |
| `config/hermes/var/www/html/admin/2/inc/ldap_remoteauth_delete_overlay.cfm` | `hermes_commandbox` | `ldapdelete` of existing overlay |
| `config/hermes/var/www/html/admin/2/inc/ldap_add_user_remoteauth.cfm` | `hermes_commandbox` | Create local stub entry with `seeAlso`/`associatedDomain` |
| `config/hermes/opt/hermes/templates/ldap_remoteauth_add_overlay.ldif` | `hermes_commandbox` | Overlay LDIF template (placeholder-substituted) |
| `config/hermes/opt/hermes/templates/ldap_adduser_remoteauth.ldif` | `hermes_commandbox` | Stub-user LDIF template |
| `/opt/hermes/certs/remoteauth/global_remoteauth_ca.pem` | `hermes_ldap` (mounted) | CA / CA-bundle for upstream TLS |
| `/opt/hermes/tmp/<token>_remoteauth_add_overlay.ldif` | `hermes_commandbox`, `hermes_ldap` | Ephemeral rendered LDIF; deleted after `ldapadd` |
| `cn=config` (in `hermes_ldap`) | `hermes_ldap` | Live overlay configuration |

Every shell-out uses `docker exec hermes_ldap …` per the standard Hermes Docker pattern.

## Future work

- **#102** — when RemoteAuth is wired to mailboxes (currently relay-recipients and console users only), deletion validation in `view_remoteauth.cfm` and `edit_remoteauth_mapping.cfm` must add a third query against `mailboxes`.
- **Position-2 mapping unique index hardening** — `remoteauth_mappings.domain_name` is `UNIQUE` but the upstream `server_address` is not; an admin can accidentally create two mappings to the same DC under different domain keys. Not a bug, but worth surfacing in a validation hint.
- **Group-based authorization** — current model is "if the upstream bind passes, the user is in." There's no upstream-group filter (e.g., "only members of `cn=hermes-users` may log in"). For installs that need this today, restrict at the upstream side with a dedicated OU.

## Related

- [Credential Model](https://docs.deeztek.com/books/administrator-guide/page/credential-model) — full picture of how RemoteAuth slots into the four-credential architecture (web vs. mail vs. DAV)
- [System Users](https://docs.deeztek.com/books/administrator-guide/page/system-users) — creating console admins/readers with RemoteAuth mode
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — required prerequisite for internal-only AD hostnames

# Mail Queue

# Mail Queue

Admin path: **System > Mail Queue** (`view_mail_queue.cfm`,
`inc/get_mail_queue_settings.cfm`, `inc/mail_queue_get_queue.cfm`,
`inc/mail_queue_action.cfm`, `inc/mail_queue_flush_mailqueue.cfm`,
`inc/mail_queue_set_queue_settings.cfm`, `view_mail_queue_message.cfm`,
`inc/mail_queue_view_message.cfm`).

This page is the operator's window into **Postfix's on-disk queue inside
`hermes_postfix_dkim`** — the messages Postfix has accepted but not yet
finally delivered or bounced. It does two unrelated jobs that share one
page:

1. **Queue Settings** — two Postfix tunables (`bounce_queue_lifetime`
   and `maximal_queue_lifetime`) stored in the `parameters` table and
   pushed into `main.cf` via the generic Postfix config regen path.
2. **Queue Viewer / Actions** — a live read of `mailq` plus per-message
   Hold / Unhold / Re-queue / Delete operations and a queue-wide Flush.

The viewer is read-only against `mailq`; everything that mutates the
queue goes through `postqueue` or `postsuper` inside the container.
Hermes never edits `/var/spool/postfix/*` directly, so admin actions
respect Postfix's own queue locking and are safe to run while mail is
flowing.

## The queue this page shows — and the ones it doesn't

```
  ┌────────────────────────────────────────────────────────────┐
  │ hermes_postfix_dkim   (the queue this page reads)          │
  │   /var/spool/postfix/{maildrop, incoming, active,          │
  │                       deferred, hold, corrupt}             │
  └─────────┬──────────────────────────────────────────────────┘
            │ (content filter loop)
            ▼
  ┌────────────────────────────────────────────────────────────┐
  │ hermes_mail_filter    (Amavis + ClamAV + SpamAssassin)     │
  │   transient per-message work, not a persistent queue       │
  └─────────┬──────────────────────────────────────────────────┘
            │
            ▼
  ┌────────────────────────────────────────────────────────────┐
  │ hermes_dovecot        (LMTP delivery to mailboxes)         │
  │   no Postfix queue here; failures bounce back to the       │
  │   postfix queue above                                      │
  └────────────────────────────────────────────────────────────┘
```

Postfix is the only component that maintains a persistent on-disk
spool. A message you see in this viewer is a message Postfix is still
holding — it has not been handed off to the next hop (LMTP to Dovecot,
remote MX, satellite Amavis), or it was handed off and bounced back
into `deferred`, or an admin moved it into `hold`. Amavis's transient
work is not a "queue" in the Postfix sense and is not visible here; if
the content filter is stuck, messages pile up in `active` on the
gateway side, which this page does surface.

## Queue Settings

Two values, both saved into rows of the `parameters` table keyed by
`parameter = 'bounce_queue_lifetime'` / `'maximal_queue_lifetime'`
(`child = 2` parent rows, with the user-selected value stored in the
`child = 1` row). The dropdowns range 0–90 days.

| Setting | `main.cf` directive | Meaning |
|---|---|---|
| **Bounce Queue Lifetime** | `bounce_queue_lifetime` | How long Postfix retries a bounce message that cannot be delivered to its envelope sender before giving up. `0` means single-delivery attempt only — failing bounces are double-bounced to the postmaster immediately. |
| **Max Queue Lifetime** | `maximal_queue_lifetime` | How long Postfix retries a normal message before generating a permanent failure (bounce) to the sender. `0` means single-delivery attempt only. |

Both values are stored as integers in the dropdown but written into the
DB with the `d` suffix (e.g. `5d`) so they go straight into `main.cf`
unmodified. Hermes regenerates `main.cf` from the `parameters` table on
save and reloads Postfix; there is no incremental edit path. See the
[Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) doc for the broader Postfix regen
pipeline.

> **Why `0` is a real choice.** `bounce_queue_lifetime = 0` is the
> upstream-recommended default for relays — a bounce that cannot be
> delivered is more likely a forged sender than a real recipient
> mailbox, and keeping it in the queue for days wastes attempts on
> joe-job traffic. Leave the seed value unless you have a specific
> reason to change it.

## Queue Viewer — how the table is built

`inc/mail_queue_get_queue.cfm` does the live read in three phases:

1. **Summary probe.** Runs
   `docker exec hermes_postfix_dkim /bin/bash -c '/usr/bin/mailq | /usr/bin/tail -1'`
   to read just the trailing `-- N Kbytes in M Requests.` line and
   parse `M` out as the total queue count. This is cheap — no full
   parse, no full transfer of the queue contents.
2. **Overload gate.** If the total exceeds **500** (`maxQueueLoad`),
   the viewer refuses to load the queue at all. The page renders a red
   callout with the count and shell hints (`postsuper -d ALL`,
   `postsuper -H ALL`) for the admin to recover from the command line.
   This is a self-protection step — parsing tens of thousands of
   `mailq` lines in CFML would hang the page and lock a CommandBox
   worker thread.
3. **Full parse.** If under 500, runs `docker exec hermes_postfix_dkim
   /usr/bin/mailq` and parses the multi-line output in CFML into a
   query object with `QueueID`, `Sender`, `Recipient`,
   `ConnectionStatus`, and `MsgStatus`. The display table is capped at
   **100 rows** (`maxQueueDisplay`); a yellow callout appears if the
   queue has between 101 and 500 entries.

The parser reads the per-entry queue-ID suffix to derive the status
column. Postfix's `mailq` marks active messages with `*` and held
messages with `!` after the queue ID; everything else is treated as
`deferred` (rendered as `N/A` in the badge). This is by design — the
viewer is a snapshot, not a queue-state diff.

| Suffix | `mailq` meaning | Rendered as |
|---|---|---|
| `*` | currently being delivered (in `active`) | green `ACTIVE` badge |
| `!` | admin-held (in `hold`) | yellow `ON-HOLD` badge |
| (none) | waiting for retry (in `deferred`) | grey `N/A` badge |

The `ConnectionStatus` column is whatever Postfix put in parentheses on
the line after the message header (typically the SMTP error from the
last delivery attempt — `Connection refused`, `Greylisted, please
try again`, etc.). For messages that have never been attempted it is
blank.

### View Message (`view_mail_queue_message.cfm`)

Clicking the magnifying glass on a row opens a full dump of the queued
message — headers and body — via `docker exec hermes_postfix_dkim
/usr/sbin/postcat -q <queueid>`. The output is rendered into a plain
textarea with a print button. No edit, no resend; if you need the
message to go out, use Re-queue from the main viewer.

## Per-message actions

All four mutation actions converge on `inc/mail_queue_action.cfm`,
which validates the queue ID against `^[A-Fa-f0-9]+$` (defence against
shell injection) and shells out to `postsuper` with the right flag:

| Action | Postsuper flag | What it does | Typical use |
|---|---|---|---|
| **Hold** | `-h` | Moves the message into `hold/`. Postfix will not touch it again until unheld. | Pause a stuck loop, freeze a message for forensic copy, hold while debugging upstream issues |
| **Unhold** | `-H` | Moves the message back into `deferred/` so retries resume | Recover a held message after the underlying issue is fixed |
| **Re-queue** | `-r` | Re-injects the message through the cleanup daemon, re-applying milter chain (OpenDKIM, OpenDMARC, body milter), header_checks, etc. | Force a fresh content-filter pass — useful after fixing a milter, updating a header_check rule, or changing a relay map |
| **Delete** | `-d` | Removes the message from the queue **permanently**. No undo. | Drop spam, drop a stuck message you don't want re-delivered, drop a confirmed mail loop |

The action handler loops the selected queue IDs and invokes `postsuper`
once per ID via a generated temp script under `/opt/hermes/tmp/` —
`postsuper` writes its result to stderr, and the temp-script pattern
(with `2>&1`) is the only reliable way to capture it from `cfexecute`.
Per-ID success or failure is counted independently; the result alert
shows both the count and the queue IDs in each bucket.

> **Re-queue is not the same as Flush.** Re-queue re-injects through
> the milter / content-filter chain (so a fresh OpenDKIM signature is
> generated, the disclaimer milter runs again, etc.). Flush just nudges
> Postfix to retry delivery on what is already in `deferred`. If a
> message is broken because of a milter failure during the original
> intake, Re-queue can fix it; Flush will not.

## Flush Queue

The Flush button runs
`docker exec hermes_postfix_dkim /usr/sbin/postqueue -f`. This is a
queue-wide "retry now" — it scans the `deferred` queue and moves
eligible messages into `active` for an immediate delivery attempt.
Held messages are not touched.

A success result means `postqueue` exited cleanly, not that delivery
succeeded. If a deferred message's destination is still unreachable, it
goes right back into `deferred` after the attempt. Use the System Logs
page (or `/remotelogs/postfix/mail.log` for live tail) to see the
actual delivery outcomes.

## Overload mode — the bulk-recovery path

When the queue exceeds 500 messages the page deliberately refuses to
render the table. Both shell-hint commands in the callout are full
queue-wide operations that bypass the per-message UI:

```bash
# Delete everything in the queue (no exceptions, no confirmation)
docker exec hermes_postfix_dkim postsuper -d ALL

# Move every held message back to deferred
docker exec hermes_postfix_dkim postsuper -H ALL
```

These are the standard Postfix mass-action commands. There is no
selective `-d` for "delete only spam-bounce" or similar; if you need
granular cleanup of a large queue, filter first with `mailq` and a
custom shell pipeline, then run `postsuper -d` on the resulting list.

> **Why a hard cap and not pagination.** Pagination would require
> parsing the full `mailq` output to know the row count anyway, which
> is the expensive operation we are trying to avoid. The hard cap
> forces the admin into the command line where the right tools live for
> bulk queue work.

## Concurrent safety

Every action goes through `postqueue` or `postsuper`, which acquire
Postfix's own queue locks before touching files. Multiple admins
hitting the page in parallel cannot corrupt the queue — at worst, two
Delete clicks on the same queue ID will have one succeed and the other
return "no such queue file", which is rendered as a failure row in the
result alert. The viewer itself is read-only and the `mailq` snapshot
can race with mutations (a message you tick may have already been
delivered by the time you click the action), which is also fine — the
mutation just no-ops with the same "no such queue file" message.

## Related pages

- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — Postfix `myhostname`, `myorigin`,
  and the `parameters` → `main.cf` regen path that this page's Queue
  Settings hooks into.
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — where delivery outcomes for queued
  messages actually surface (Postfix logs to mail.* → rsyslog →
  `SystemEvents` → this viewer).
- [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) — IP-level bans for
  brute-force SMTP-AUTH that show up in Postfix's connection logs.

# Password Resets

# Password Resets

Admin path: **System > Password Resets** (`view_password_reset_requests.cfm`,
`inc/process_admin_password_reset.cfm`,
`inc/cancel_password_reset_requests.cfm`,
`inc/check_hibp.cfm`).

This is the admin-side **queue** for password-reset requests that
users have submitted from the public **Forgot Password** page
(`/user-auth/forgot_password.cfm`). Most requests resolve themselves
via email or Pushover and never need admin attention — the requests
that land on this page are the ones that **couldn't** be self-served.

The page is also where an admin can **manually reset** any user's
password (mailbox or relay) regardless of how the request arrived — it
is the single tool for forcing a password change.

## Where a request comes from

```
End user opens /user-auth/forgot_password.cfm
        │   (link from the /users portal login page; same page
        │    serves admin and user portals at the public URL)
        ▼
fills in email + CAPTCHA
        │
        ▼
process_password_reset_request.cfm runs:
  1. honeypot check (hidden field "fax_number_ext" must be empty)
  2. CAPTCHA validation (built-in math OR reCAPTCHA OR
     hCaptcha OR Turnstile — configured globally)
  3. 15-minute rate limit: refuse if a pending request for this
     email exists less than 15 minutes old
  4. LDAP lookup: find the user, determine type from group membership
        │
        ▼
route by user type
   ┌──────────────────┬──────────────────┬──────────────────┐
   ▼                  ▼                  ▼                  ▼
 RELAY            MAILBOX            ADMIN              REMOTE-AUTH
 (cn=relays)      (cn=mailboxes)     (cn=admins)        (any group)
   │                  │                  │                  │
   ▼                  ▼                  ▼                  ▼
 email token   secondary email     REFUSED              REFUSED
 to relay      verified?           (admins must         (password is
 user's        ├ YES → email       use peer-admin       upstream;
 external      │       to that     reset path on        Hermes never
 email         │       address     this page)           saw it)
               └ NO  → admin                            shown the same
                       queue (this                      generic "if an
                       page)                            account exists"
                                                        success page for
                                                        security
```

The route the request takes determines whether it ever shows up on this
page:

| Request shape | Lands here? |
|---|---|
| Relay user with valid email | **No** — email is sent automatically with a 15-minute reset link |
| Mailbox user with a verified secondary email | **No** — email is sent automatically to the secondary address |
| Mailbox user with no verified secondary email | **Yes** — admin must reset manually |
| Mailbox user with Pushover enabled | **No** — Pushover notification sent automatically |
| Admin self-service | **Never accepted** — admins must be reset by another admin from this page |
| RemoteAuth user (`auth_type = 'remote'`) | **Never accepted** — Hermes does not own the password (see below) |

> **By design.** Admin self-service password reset is blocked because
> a compromised admin email is an easy lateral-movement vector and the
> blast radius is the whole console. The forgot-password page shows
> the same generic "if an account exists, instructions have been sent"
> message for blocked admins as for blocked RemoteAuth users and for
> unknown emails — bots probing for admin usernames learn nothing.

## RemoteAuth requests are never accepted

For users with `recipients.auth_type = 'remote'` (or, in the future,
`mailboxes.auth_type = 'remote'`), the request flow short-circuits at
step 4 with the same generic success message as for unknown emails.
Hermes does **not** store, hash, or have any way to update the user's
password — it lives in the customer's upstream AD/LDAP.

These users must use their organization's own password-reset workflow
(self-service portal, helpdesk ticket, etc.). See
[LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) and
[Credential Model § Local-auth users vs. remote-auth users](https://docs.deeztek.com/books/administrator-guide/page/credential-model#local-auth-users-vs-remote-auth-users).

## Database schema — `password_reset_requests`

| Column | Purpose |
|---|---|
| `id` | PK |
| `email` | The address the user typed into the form |
| `ldap_username` | The `cn` resolved from LDAP at submission time |
| `user_type` | `relay`, `mailbox`, or `admin` (admin rows shouldn't exist in practice — the flow blocks them at submit) |
| `token` | 64-char random — the secret in the reset link emailed to the user |
| `notification_method` | `email`, `pushover`, or `admin` — how the user was notified |
| `status` | `pending`, `completed`, `expired`, `cancelled` |
| `requested_at` | When the user submitted the form |
| `expires_at` | NOW + 15 min for `email`/`pushover` methods; NULL for `admin` method (no link to expire) |
| `completed_at` | When the admin (or self-service flow) resolved it |
| `completed_by` | The admin username, or the system user that auto-resolved |

### Auto-cleanup runs on every page load

The page does **not** rely on a scheduled job for housekeeping. Two
DELETE queries run at the top of every request:

```sql
-- Cull expired pending requests (the reset link is dead anyway)
DELETE FROM password_reset_requests
 WHERE status = 'pending'
   AND expires_at < NOW();

-- Cull completed requests older than 30 days (audit window)
DELETE FROM password_reset_requests
 WHERE status = 'completed'
   AND completed_at < DATE_SUB(NOW(), INTERVAL 30 DAY);
```

This keeps the table bounded with no admin intervention. The 30-day
audit window is hardcoded — if you need longer retention for
compliance, that's a code change, not a configuration knob.

## The page surface

| Column | Notes |
|---|---|
| (checkbox) | Only renders for `pending` rows |
| Email | The user's submitted address |
| User Type | Badge: relay (info-blue), mailbox (primary), admin (warning) |
| Method | Icon + label: email envelope, Pushover bell, admin shield |
| Requested | Submission timestamp |
| Expires | NULL for admin-method rows; for time-bound rows, shows the timestamp + an "Expired" red badge if past and still pending |
| Status | pending (yellow), completed (green), expired (gray), cancelled (red) |
| Completed By | Admin username + timestamp once resolved |

Two action buttons sit above the table:

- **Reset Password** — opens the reset modal for the single selected
  pending row (alerts if zero or more than one is selected)
- **Cancel Request(s)** — opens a confirmation modal that hard-deletes
  every selected pending row

### Why notify-user is shown only for relay rows

The reset modal shows a **Notify user via email** checkbox **only**
when the selected row is a relay user. Mailbox and admin users have
their primary email == their mailbox address, which won't deliver
because the admin is about to change their login credential to a
mail-protocol component that's part of the same auth chain. Relay
users hold an external email address, so sending them a "your
password was reset" notification to that external address works.

## Admin reset flow

When the admin clicks **Reset Password** and confirms the modal,
`process_admin_password_reset.cfm` runs:

```
1. Form validation: passwords match, length >= 8, request_id present
2. (optional) HIBP check via api.pwnedpasswords.com — k-anonymity
   prefix lookup; reject on match
3. Lookup the row — must still be status='pending'
4. docker exec hermes_ldap slappasswd \
        -o module-load=argon2.la -h {ARGON2} \
        -s <new_password>
        --> returns {ARGON2}$argon2id$...
5. Render /opt/hermes/templates/ldap_modifyuserpassword.ldif
   (THE_USERNAME, THE_OU=users, THE_PASSWORD placeholders)
   to /opt/hermes/tmp/<token>_modifyuserpassword.ldif
6. docker exec hermes_ldap ldapmodify -Y EXTERNAL \
        -H ldapi:///... -f /opt/hermes/tmp/<token>_modifyuserpassword.ldif
7. Delete the temp LDIF
8. If the user has a Nextcloud account (mailboxes.nextcloud_enabled=1):
        docker exec -e OC_PASS=<new> -u www-data hermes_nextcloud \
          php /var/www/html/occ user:resetpassword \
          --password-from-env <email>
   (sync NC's local password column — see Credential Model for why
    NC keeps a local password that no human knows)
9. UPDATE password_reset_requests
        SET status='completed', completed_at=NOW(), completed_by=<admin>
10. UPDATE password_reset_requests SET status='expired'
        WHERE email=<email> AND status='pending' AND id != <this one>
    (clears stale pending duplicates the user may have submitted)
11. If notify_user checked (relay rows only):
        cfmail via hermes_postfix_dkim:10026 — generic "your password
        was reset by an administrator" template with the console URL
```

Two non-obvious bits:

- **Two hashing tools, one outcome.** This page uses `slappasswd` with
  the OpenLDAP argon2 module loaded; [System Users](https://docs.deeztek.com/books/administrator-guide/page/system-users)
  uses the Authelia CLI image. Both produce `{ARGON2}$argon2id$...`
  hashes that the same OpenLDAP overlay validates. They are
  interchangeable; the difference is historical (this page predates
  the Authelia-image hashing pattern). Either is correct.

- **Nextcloud password sync via temp shell script.** Step 8 writes a
  shell script to `/opt/hermes/tmp/` and runs it instead of
  `cfexecute`ing `docker exec` directly. The script wrapper exists
  because Lucee's `cfexecute` mishandles stderr, quoting, and `OC_PASS`
  env-var injection on commands of this shape, and the temp-script
  pattern is the established Hermes workaround.

## Cancel flow

`cancel_password_reset_requests.cfm` performs a hard `DELETE` against
every selected `pending` row. There is no soft-delete — the row is
gone, the user must submit a new request if they still need help. This
is the right shape because the request never carried valuable data;
it's just a "please help me" signal.

The admin username doing the cancel is **not** recorded — only
completions record `completed_by`. If audit trail matters for
cancellations, that's a planned schema extension.

## CAPTCHA — the public side

The forgot-password page picks a CAPTCHA provider from `system_settings`
at runtime. Four providers are supported today:

| `captcha_provider` | What appears on the page |
|---|---|
| `builtin` (default) | Math word-problem ("What is three plus seven?") — no third-party JS, no cookie, no API key required. ~225 unique combinations across addition (1-10), subtraction (1-10, positive result), and small multiplication (1-5). |
| `recaptcha` | Google reCAPTCHA v2 — site key + secret key required |
| `hcaptcha` | hCaptcha — site key + secret key required |
| `turnstile` | Cloudflare Turnstile — site key + secret key required |

All four use the same flow: client-side widget posts a token with the
form, server-side `process_password_reset_request.cfm` validates the
token (for external providers, via HTTPS POST to the provider's
`siteverify` endpoint). Failed validation always redirects back with
reason code `9` ("invalid CAPTCHA"). For external providers, if the
provider's API is unreachable from Hermes, the page treats the request
as invalid — failing closed is the right call on a brute-force
defense surface.

A **honeypot** field (named `fax_number_ext`, hidden via CSS) runs
**before** the CAPTCHA check. Real users never see or fill it; bots
that submit the entire form are silently rejected with the same
generic success page so they can't tell their submission was
discarded.

## Rate limiting — the 15-minute window

`process_password_reset_request.cfm` queries for any `pending` row
with the same email submitted in the last 15 minutes; if one exists,
the new submission is refused with reason `8`. The window is per-email,
not per-IP — a malicious actor enumerating addresses can still hit
many emails in parallel, but cannot spam any single one.

The window is hardcoded; if you need longer cool-down for a
high-noise environment, that's a code change.

## Token security

For email and Pushover methods, the user receives a link of shape:

```
https://<console>/user-auth/reset_password.cfm?token=<64-char-random>
```

- The token is 64 hex chars from `inc/generate_customtrans.cfm` —
  cryptographically strong, single-use.
- It expires after **15 minutes** (`expires_at` column).
- It is **single-use**: when the user successfully completes the
  reset, the row's `status` flips to `completed`, and the
  reset_password.cfm endpoint rejects further use.
- Submitting a new request invalidates any earlier pending request
  for the same email (step 10 of the admin reset above; the
  user-side reset endpoint does the equivalent).

For the `admin` method (the rows that show up on this page), the
token still exists in the row but the **expires_at is NULL** — there
is no email link to expire because no email was sent. The admin
resolves the request when they get to it; the queue serves as the
notification channel.

## What this page does NOT do

| Concern | Lives on |
|---|---|
| Admin's own password change | They sign in to `/admin/`, go to **My Settings** (or have another admin reset it from [System Users](https://docs.deeztek.com/books/administrator-guide/page/system-users)'s edit modal) |
| Configuring CAPTCHA provider + keys | Configured via `system_settings` rows; admin UI for this is planned. Defaults to `builtin` math CAPTCHA. |
| Configuring the rate-limit window | Hardcoded 15 minutes — code change required |
| Configuring the token TTL | Hardcoded 15 minutes — code change required |
| Pushover credentials per-user | Set on the user portal's **Account Settings** page; this page just consumes them |
| The reset email template / branding | Hardcoded in `process_password_reset_request.cfm` and `process_admin_password_reset.cfm`; uses `hermes_logo_new_orange2.png` as a CID attachment |
| 2FA device deletion | [System Users](https://docs.deeztek.com/books/administrator-guide/page/system-users)'s **Delete 2FA Devices** button — runs `authelia storage user totp delete` |

## Failure semantics

| What breaks | What happens |
|---|---|
| `hermes_ldap` down during admin reset | The `slappasswd` and `ldapmodify` calls fail; the admin sees the raw error, the request row stays `pending`, no password change. Retry after LDAP recovers. |
| `hermes_postfix_dkim` down during user-initiated email request | The cfmail throws; `process_password_reset_request.cfm` catches, flips the request row to `status='failed'`, and shows reason `6` ("Unable to send password reset"). |
| HIBP API unreachable | Server-side check silently passes (the JavaScript on the modal already warned the user; defense-in-depth pattern). The reset still completes. |
| Token guessed / brute-forced | Computationally infeasible at 64 hex chars (256 bits of entropy). |
| `hermes_nextcloud` down during admin reset step 8 | LDAP password is already updated; the NC sync step fails silently (caught in a non-fatal cftry). The user can log in to `/users` immediately; webmail and DAV will work as soon as NC is back. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_password_reset_requests.cfm` | `hermes_commandbox` | Page (table + 2 modals + auto-cleanup queries) |
| `config/hermes/var/www/html/admin/2/inc/process_admin_password_reset.cfm` | `hermes_commandbox` | Admin reset handler (LDAP + NC sync + audit + optional notify) |
| `config/hermes/var/www/html/admin/2/inc/cancel_password_reset_requests.cfm` | `hermes_commandbox` | Hard-deletes selected pending rows |
| `config/hermes/var/www/html/user-auth/forgot_password.cfm` | `hermes_commandbox` | Public-facing request entry point (CAPTCHA + honeypot + LDAP lookup) |
| `config/hermes/var/www/html/user-auth/inc/process_password_reset_request.cfm` | `hermes_commandbox` | Rate-limit check + token mint + INSERT + route to email/Pushover/admin |
| `config/hermes/var/www/html/user-auth/inc/ldap_get_user_groups.cfm` | `hermes_commandbox` | Determines user type from LDAP group membership |
| `config/hermes/var/www/html/user-auth/reset_password.cfm` | `hermes_commandbox` | Token-consuming endpoint that actually changes the password (user side) |
| `/opt/hermes/templates/ldap_modifyuserpassword.ldif` | `hermes_commandbox` | LDIF template for the password-replace operation |
| `/opt/hermes/tmp/<token>_modifyuserpassword.ldif` | `hermes_commandbox`, `hermes_ldap` | Ephemeral rendered LDIF; deleted after `ldapmodify` |
| `/opt/hermes/tmp/<token>_nc_pwd_update.sh` | `hermes_commandbox` | Ephemeral shell script for the NC `occ user:resetpassword` step |
| `password_reset_requests` table | `hermes_db_server` (`hermes` DB) | The queue itself |

Every shell-out uses `docker exec hermes_ldap …`, `docker exec hermes_nextcloud …`, or the standard `hermes_postfix_dkim:10026` re-injection port per the canonical Hermes pattern.

## Related documentation

- [System Users](https://docs.deeztek.com/books/administrator-guide/page/system-users) — admin-account CRUD; password changes for admins happen there, not on this page
- [Credential Model](https://docs.deeztek.com/books/administrator-guide/page/credential-model) — why mailbox users carry both a web-login password (reset here) and separate per-device app passwords (reset elsewhere)
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — why remote-auth users cannot be reset through this page
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) — the Authelia JWT secret used for the reset-link signature on the user-side reset endpoint
- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — the console hostname embedded in the reset-link emails
- [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) — Fail2ban `authelia` jail; layered defense against brute-force on the login surface this page protects

# Scheduled Tasks

# Scheduled Tasks

Admin path: **System > Scheduled Tasks** (`view_scheduled_tasks.cfm`,
`inc/ofelia_generate_config.cfm`, `inc/run_scheduled_task_action.cfm`,
`inc/toggle_ofelia_job_action.cfm`, `inc/restart_ofelia.cfm`).

This page is the admin surface over **Ofelia**, Hermes's cron runner.
Ofelia (`mcuadros/ofelia:latest`) sits next to the application
containers, mounts the Docker socket, and on a schedule does `docker
exec <container> <command>` for each configured job. The page lists
every job in the `ofelia_jobs` table, displays its humanized schedule
and last manual-run timestamp, and exposes per-row **Enable/Disable**
and **Run Now** controls.

Hermes does not use the host's crond. Every recurring task — certificate
renewal, the daily update check, quarantine notifications,
mail-queue health checks, DMARC report processing, malware-feed refresh,
log rotation — runs through this single Ofelia container and is
manageable from this page.

## Why Ofelia and not host cron

A traditional host crontab does not fit Hermes's deployment model:

| Requirement | Host cron problem | Ofelia behavior |
|---|---|---|
| Run a command inside `hermes_commandbox` or `hermes_dmarc` on a schedule | Host cron has to `docker exec` from outside; failure modes (missing container, wrong user) surface in syslog, not in the admin UI | Ofelia speaks Docker natively; jobs are `job-exec` blocks against a named container |
| Notify the admin when a job fails | Cron emails the local UNIX user; meaningless inside a container deployment | Ofelia has a built-in SMTP notifier that emails `admin_email` via `hermes_postfix_dkim:10026` (the auto-DKIM-signing re-injection port) when `mail-only-on-error = true` |
| Survive a host reboot the same way every other Hermes service does | Cron units have to be packaged separately | `hermes_ofelia` is just another container in [`docker-compose.yml`](https://github.com/deeztek/Hermes-Secure-Email-Gateway/blob/main/docker-compose.yml); `restart: unless-stopped` covers it |
| Be inspectable and runnable on demand from the web UI | Out-of-band; admin would need shell access | This page reads the same table Ofelia reads and can re-fire any job synchronously |

The trade-off is that `config.ini` is regenerated from the database — so
direct hand-edits to `/etc/ofelia/config.ini` are **overwritten on every
save**. The DB is the source of truth.

## How a scheduled job flows through the stack

```
+-----------------------+
| ofelia_jobs (MariaDB) |    <-- canonical source of truth
+-----------+-----------+
            |
            | Save / Toggle / install --apply-schema
            v
+-----------------------------------------------------+
| inc/ofelia_generate_config.cfm                      |
|   1. SELECT * FROM ofelia_jobs WHERE active = '1'   |
|   2. Render /opt/hermes/tmp/<tok>_ofelia_jobs       |
|   3. dos2unix (CRLF safety)                         |
|   4. Read /opt/hermes/conf_files/ofelia_config.ini  |
|        (template with POSTMASTER_EMAIL,             |
|         ADMIN_EMAIL, OFELIA_JOBS_GO_HERE markers)   |
|   5. REReplace each marker with live values         |
|   6. Move final file to /etc/ofelia/config.ini      |
|   7. cfinclude restart_ofelia.cfm                   |
+-----------------------------+-----------------------+
                              |
                              v
+-----------------------------------------------------+
| hermes_ofelia container                             |
|   reads /etc/ofelia/config.ini on start             |
|   fires `docker exec <container> <command>` on      |
|   each job's schedule, capturing stdout/stderr      |
|   on failure: emails admin_email via 10026          |
+-----------------------------------------------------+
```

## Configuration storage

| Table | Role |
|---|---|
| `ofelia_jobs` | One row per scheduled job |
| `scheduled_job_runs` | Append-only history of **manual** Run Now invocations from this page; Ofelia's own scheduled executions are not recorded here |

`ofelia_jobs` schema (relevant columns):

| Column | Type | Notes |
|---|---|---|
| `job_name` | `varchar(255)` | The **full bracketed header** as Ofelia consumes it, e.g. `[job-exec "hermes-quarantine-notify"]`. The display-friendly name shown in the table is the text between the quotes (the page extracts it with a regex). |
| `schedule` | `varchar(255)` | Ofelia format — either 6-field cron (`sec min hr dom mon dow`), 5-field cron, or `@every <duration>` (e.g. `@every 60s`, `@every 10m`, `@every 1h`) |
| `command` | `varchar(255)` | The shell command Ofelia runs inside the container |
| `container` | `varchar(255)` | Target container — `hermes_commandbox` for most jobs, `hermes_dmarc` for DMARC report processing, `hermes_mail_filter` for fangfrisch |
| `active` | `int(11)` | **`1` = enabled, `2` = disabled.** Disabled jobs stay in the DB but are filtered out of the generated `config.ini`. |
| `no_overlap` | `tinyint(3)` | When `1`, Ofelia emits `no-overlap = true` so a still-running invocation prevents the next tick from firing. Used for short-interval jobs (`@every 60s` cert-queue, quarantine-notify). |
| `type` | `varchar(255)` | Category tag for grouping (`certbot`, `hermes`, `dmarc`, `pushover`, `malware_feeds`, `system`) |

## The seeded job set

A fresh install (`hermes_install.sql`) seeds these jobs. All start
enabled.

| Job | Schedule | Container | What it does |
|---|---|---|---|
| `renew-acme-certificate` | Daily 12:05 | `hermes_commandbox` | Runs certbot renew across all ACME-issued certs; reloads dependent services on success |
| `hermes-message-cleanup` | Daily 01:30 | `hermes_commandbox` | Enforces `msgs` retention policy (Pro: per-policy; Community: global) |
| `hermes-update-check` | Daily 04:30 | `hermes_commandbox` | Polls GitHub Releases; writes the cache file the dashboard reads. See [System Update § Daily update check](https://docs.deeztek.com/books/administrator-guide/page/system-update#daily-update-check). |
| `acme-validate-ip` | Every 30 min | `hermes_commandbox` | Refreshes mailbox-domain SAN cert state when the gateway's public IP changes |
| `hermes-health-check-mailqueue` | Every 15 min | `hermes_commandbox` | Pushover alert when `mailq` count exceeds the threshold |
| `hermes-dmarc-report` | Daily 02:30 | `hermes_dmarc` | Fetches DMARC RUA reports, parses them into the `opendmarc` DB |
| `hermes-authelia-log-rotate` | Daily 02:00 | `hermes_commandbox` | Rotates Authelia's access logs |
| `hermes-quarantine-notify` | Every 60s, `no-overlap` | `hermes_commandbox` | Issues quarantine-release emails to recipients with pending messages |
| `hermes-process-cert-queue` | Every 60s, `no-overlap` | `hermes_commandbox` | Drains the encryption cert lookup queue for outbound S/MIME / PGP recipients |
| `hermes-fangfrisch-refresh` | Every 10 min | `hermes_mail_filter` | Refreshes third-party ClamAV signature feeds (SecuriteInfo, Sanesecurity, etc.) |

New jobs added by later features (signature-map regen for the body
milter, the post-upgrade hook caller, etc.) appear here automatically as
they are seeded into `ofelia_jobs`. The page renders whatever is in the
table — there is no hardcoded job list in the CFML.

## The page columns

The DataTable renders one row per `ofelia_jobs` row.

| Column | What it shows |
|---|---|
| **Name** | The display-friendly name (text between the quotes in `job_name`) |
| **Type** | The `type` category tag |
| **Schedule** | Humanized form — `@every 60s` becomes "Every 60 seconds", `0 30 04 * * *` becomes "Daily at 04:30", `0 0 02 * * *` becomes "Daily at 02:00", and so on. Hover for the raw cron expression (commit `8e954d1d`). Anything the humanizer can't cleanly parse falls through to the raw string. |
| **Container** | Target container (`hermes_commandbox`, `hermes_dmarc`, `hermes_mail_filter`, ...) |
| **Command** | The literal command Ofelia runs |
| **Status** | Bootstrap-switch toggle (Enabled / Disabled), AJAX-driven |
| **Last Run (manual)** | Most recent **Run Now** click from this page; Ofelia's own scheduled fires do not write here |
| **Actions** | The **Run Now** button |

## Enable / Disable toggle

The switch posts to `inc/toggle_ofelia_job_action.cfm` with the
`job_name` and `new_state` (`1` or `2`). The handler:

1. Looks up the row; rejects if not found.
2. `UPDATE ofelia_jobs SET active = ?`.
3. Re-runs `ofelia_generate_config.cfm`, which writes a fresh
   `config.ini` containing only the enabled rows.
4. Restarts `hermes_ofelia` via `restart_ofelia.cfm`.
5. On any failure during step 3 or 4, **rolls the `active` flag back**
   and returns the error in JSON. The UI reverts the switch and
   surfaces the error.

The transactional behavior matters — a half-applied state where the DB
says "disabled" but Ofelia is still running the job is exactly the
confusing situation an admin would not be able to diagnose from this
page.

The JS layer surfaces a confirm prompt before disabling jobs on a
**critical list** (`renew-acme-certificate`, `hermes-update-check`,
`hermes-process-cert-queue`, `hermes-quarantine-notify`). The backend
trusts the request — admins with web access already have the means to
disable everything via direct SQL if they want to. The prompt is a
guard against an accidental click, not an authorization gate.

## Run Now

The button posts to `inc/run_scheduled_task_action.cfm`, which executes
the job's `command` synchronously and returns JSON with status,
duration, exit code, and output (capped at 2048 bytes for the DB
history, full body in the response). The result is displayed in a modal
with a spinner-then-summary view.

Three execution strategies, picked from the command shape:

| Command shape | Strategy |
|---|---|
| `/usr/bin/curl --silent http://localhost:8888/schedule/<name>.cfm` | Routed via `cfhttp` for clean body capture. This is the majority of Hermes jobs — the actual work is implemented as a CFML schedule script and Ofelia is just a trigger. |
| `container != hermes_commandbox` | Proxied via `cfexecute docker exec <container> <command>`. Used for `hermes-dmarc-report` (targets `hermes_dmarc`) and `hermes-fangfrisch-refresh` (targets `hermes_mail_filter`). |
| Anything else inside `hermes_commandbox` | `cfexecute` directly — the page itself runs inside `hermes_commandbox`, so this is equivalent to what Ofelia would do. |

Hard cap on the manual-trigger path is **300 seconds**. Ofelia's own
scheduled runs have no such cap; if a job legitimately needs to run
longer, scheduled execution is fine but Run Now will time out.

Every Run Now invocation appends a row to `scheduled_job_runs` —
including failures, including runs of disabled jobs (the page allows
firing a disabled job on demand without re-enabling it). The Last Run
column reads from this table.

> **By design.** Run Now and the schedule run independently. Firing a
> job manually does **not** reset Ofelia's next-scheduled-fire clock.
> If you Run Now a job that is also scheduled to fire in 30 seconds, it
> will fire again 30 seconds later — for the `no-overlap` jobs, Ofelia
> will skip the scheduled fire if the manual run is still in progress;
> for the others, both runs will happen.

## The config.ini template

`config/hermes/opt/hermes/conf_files/ofelia_config.ini` is a small
placeholder file:

```ini
[global]
smtp-host = hermes_postfix_dkim
smtp-port = 10026
email-to = ADMIN_EMAIL
email-from = POSTMASTER_EMAIL
mail-only-on-error = true

OFELIA_JOBS_GO_HERE
```

`ofelia_generate_config.cfm` does three `REReplace` passes against this
template — `ADMIN_EMAIL` and `POSTMASTER_EMAIL` from `system_settings`,
`OFELIA_JOBS_GO_HERE` from the rendered `[job-exec ...]` blocks — and
writes the result to `/etc/ofelia/config.ini`. The intermediate work
happens under `/opt/hermes/tmp/<customtrans3>_*` with a final atomic
`move` into place, which is also why a partial regen does not leave the
live file half-written.

## When direct edits to config.ini are appropriate

There are exactly two situations where editing `/etc/ofelia/config.ini`
directly makes sense:

1. **Debugging Ofelia itself** — flipping `mail-only-on-error` to
   `false` so every successful run notifies, or adding `verbose = true`
   to the global block to flood `docker logs hermes_ofelia` with detail.
2. **Adding a one-shot job that you don't want in the DB** — e.g., a
   migration script that should run once at the next scheduled time.

In both cases, the change survives until the next save on this page or
the next install-script run. If you need a persistent custom job, add
it to `ofelia_jobs` directly via SQL and the regen will pick it up.

## Failure semantics

| What breaks | What happens |
|---|---|
| `ofelia_jobs` is empty | The page shows a warning callout; Ofelia generates no jobs and idles. Re-run `install_hermes_docker.sh --apply-schema` to re-seed. |
| Toggle handler fails mid-regen | `active` flag rolled back to its previous value; switch reverts in the UI; error surfaced. Live `config.ini` is unchanged. |
| `restart_ofelia.cfm` fails (container missing, Docker socket gone) | Toggle response carries the error message; live `config.ini` is the new one but Ofelia hasn't reread it yet. Manual `docker compose restart hermes_ofelia` recovers. |
| Run Now times out (>300s) | `cfexecute` raises; the JSON response is `success: false` with an exception; `scheduled_job_runs` still gets the failure row. |
| Run Now command exits non-zero | Modal shows the stderr in the output pane; the row still inserts into `scheduled_job_runs` with `exit_code` set to whatever the process returned. |
| Ofelia's own scheduled run fails | Ofelia emails `admin_email` via 10026 (auto-DKIM-signed). **Not** reflected in the Last Run column on this page — that column is manual-only. |
| `dos2unix` not installed inside `hermes_commandbox` | Regen aborts with `error.cfm` traceback. The shipped image has it; only relevant for custom builds. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_scheduled_tasks.cfm` | `hermes_commandbox` | The page (renders the table, hosts the toggle + Run Now JS) |
| `config/hermes/var/www/html/admin/2/inc/run_scheduled_task_action.cfm` | `hermes_commandbox` | Run Now AJAX endpoint |
| `config/hermes/var/www/html/admin/2/inc/toggle_ofelia_job_action.cfm` | `hermes_commandbox` | Enable/Disable AJAX endpoint |
| `config/hermes/var/www/html/admin/2/inc/ofelia_generate_config.cfm` | `hermes_commandbox` | Config regenerator — reads `ofelia_jobs`, writes `config.ini` |
| `config/hermes/var/www/html/admin/2/inc/restart_ofelia.cfm` | `hermes_commandbox` | `docker container restart hermes_ofelia` wrapper |
| `config/hermes/opt/hermes/conf_files/ofelia_config.ini` | `hermes_commandbox` | Template with `ADMIN_EMAIL` / `POSTMASTER_EMAIL` / `OFELIA_JOBS_GO_HERE` markers |
| `config/ofelia/config.ini` | `hermes_ofelia` (live) | Regen target |
| `ofelia_jobs` table | `hermes_db_server` (`hermes` DB) | Canonical job list |
| `scheduled_job_runs` table | `hermes_db_server` (`hermes` DB) | Manual-run history |
| `/var/run/docker.sock` (host mount → `hermes_ofelia`) | host filesystem | How Ofelia issues `docker exec` against other containers |

## Future work

- **Inline schedule editing** — today, schedule + command edits happen
  on feature-specific pages (e.g., the Malware Feeds settings page edits
  `hermes-fangfrisch-refresh`'s schedule). A "create new job" and inline
  edit on this page is planned for a later release.
- **External job triggers via API** — issues #222 (Hermes Internal API)
  and #223 (API tokens) will eventually let external systems POST to
  `/api/scheduled-tasks/<name>/run` with a token, replacing the
  web-UI-only Run Now flow. Not yet built.
- **Surface Ofelia's scheduled-run history** — `scheduled_job_runs`
  records manual runs only because that is what the page writes.
  Ofelia's own per-run history sits in `docker logs hermes_ofelia` and
  is not currently tabled. A future enhancement could parse Ofelia's
  stdout into a similar history table.

## Related

- [System Update](https://docs.deeztek.com/books/administrator-guide/page/system-update) — the `hermes-update-check` job is the daily GitHub Releases poll that drives the dashboard's update-available cell
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — most scheduled jobs depend on outbound DNS resolution flowing through `hermes_unbound`
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — the `renew-acme-certificate` job is what actually keeps Let's Encrypt certs current; the page only registers and binds them
- [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings) — `admin_email` (Ofelia failure notification target) and `postmaster` (sender) are both read from here at config regen
- [System Status](https://docs.deeztek.com/books/administrator-guide/page/system-status) — dashboard cells reflect outputs that several of these jobs produce (mail queue, update status)
- [Storage Topology](https://docs.deeztek.com/books/installation-reference/page/storage-topology-5-tiers) — `hermes_ofelia` is stateless; its config lives in the Config tier (`config/ofelia/`)

# Server Setup

# Server Setup

Admin path: **System > Server Setup** (`view_server_setup.cfm`,
`inc/save_server_identity.cfm`, `inc/generate_postfix_configuration.cfm`,
`inc/generate_nextcloud_configuration.cfm`).

This page configures **how Hermes identifies itself to other mail
servers** — the Postfix `myorigin` domain, the `myhostname` FQDN used
in SMTP banners and HELO/EHLO greetings, and the host IPv4 address
used by Nextcloud's `trusted_domains`. These are foundational, mostly
install-time values; changing them in production has visible downstream
effects on outbound mail acceptance and on email-client configuration.

Pairs with [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings), which configures
the web-side identity (Console Address and certificate). The two pages
together define every name Hermes presents to the world: the mail side
on this page, the web side on Console Settings.

## What this page does NOT configure

| Concern | Lives on |
|---|---|
| The hostname/IP that nginx terminates HTTPS on for `/admin`, `/users`, `/nc` | [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — Console Address |
| The TLS certificate presented to mail clients on `:25`, `:465`, `:587` | [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) — separate cert binding from the console cert |
| The TLS certificate presented to the web console | [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — Console Certificate |
| Per-domain mail routing, accepted-domain lists, relay maps | Email Relay > Domains and Email Server > Domains |
| The Docker subnet (`IPV4SUBNET` in `.env`) | Currently hardcoded in 15+ config files. See [Known limitation](#known-limitation--docker-subnet-is-hardcoded) below. |
| Initial install — admin password, LDAP base, secrets generation | `scripts/install_hermes_docker.sh` (see [Release engineering and updates](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology)) |

## Configuration storage — the `parameters` / `parameters2` split

This page is one of the cleanest examples of the **dual-role
`parameters` table** in Hermes. Two of the three fields live there
(under their Postfix directive names), and the third lives in
`parameters2`.

### `myorigin` and `myhostname` — `parameters` table

In the `parameters` table, the same directive is stored as **two
rows**:

| Row | Role | Linked by |
|---|---|---|
| `child = 2` row | The directive **name** (the Postfix keyword), e.g. `parameter = 'myorigin'` | `parent_name` on the value row points back to this row's `parameter` |
| `child = 1` row | The directive **value** (the actual domain/hostname), e.g. `parameter = 'example.com'`, `parent_name = 'myorigin'` | — |

The page reads from the `child = 1` row (the value) and writes back to
the same `child = 1` row when an admin saves. The `child = 2` row's
`enabled` flag is set to `1` on every save to guarantee the directive
is included when Postfix `main.cf` is regenerated.

```sql
-- The name row (directive)
parameter = 'myorigin', child = '2', enabled = '1', conf_file = 'main.cf', module = 'postfix'

-- The value row (the actual domain)
parameter = '<your-domain>', parent_name = 'myorigin', child = '1',
    module = 'postfix', conf_file = 'main.cf'
```

The same shape applies to `myhostname`. Seeded defaults are
`domain.tld` and `hermes.domain.tld` respectively.

> **Why the split.** The dual-row pattern lets Hermes treat any Postfix
> directive uniformly: the parent (`child = 2`) carries metadata —
> display name, help text, default, enable flag — and one or more value
> rows (`child = 1`) carry the actual configuration. Multi-value
> directives (`mynetworks`, `smtpd_recipient_restrictions`, etc.) just
> have more `child = 1` rows under the same `parent_name`. Single-value
> directives like `myhostname` have exactly one.

### Host IP Address — `parameters2` table

Host IP lives in `parameters2` because it is not a Postfix directive
— it is a free-floating piece of installation state consumed by
Nextcloud's `trusted_domains` config.

```sql
parameter = 'server_ip', value2 = '<ip>', module = 'network'
```

Read by `generate_nextcloud_configuration.cfm` and substituted into
`config.php` as `NEXTCLOUD_TRUSTED_DOMAIN_IP`. The same value is also
used by the install script and any other code that needs the
operator-confirmed host IP without parsing it out of `ip addr`.

## Fields on the page

### Mail Server Domain (Postfix `myorigin`)

The origin domain Postfix appends to unqualified sender addresses on
outbound mail. If a local process submits a message from
`root@localhost`, Postfix rewrites it to `root@<myorigin>` before
sending. For internal-only setups this can stay at the install default;
for any system that sends external mail, set it to the operator's
canonical domain.

Validated by the email-trick: `IsValid("email", "test@<value>")` must
return true. Empty input is rejected with `session.m = 2`; invalid
format with `session.m = 4`.

### Mail Server Hostname (Postfix `myhostname`)

The fully-qualified hostname Hermes announces in its SMTP banner and
HELO/EHLO greeting. This is the value other mail servers see when they
connect to Hermes (and that Hermes presents when it connects to them).
Three downstream consequences:

| Consumer | What goes wrong if this doesn't match DNS |
|---|---|
| Receiving MTAs' reverse-DNS checks (PTR lookup → A lookup → match) | Recipient servers reject outbound mail with `450/550 helo not match` errors |
| TLS certificate Common Name / SAN match on SMTP | Strict STARTTLS verifiers refuse to deliver to Hermes |
| Authoritative SPF / DKIM / DMARC alignment for `mailfrom` | Indirect — bounces may align poorly if MAIL FROM uses an unmatched domain |

> **Do not change this in production without planning.** The page
> wraps the field in a red warning callout for a reason. The page
> warning enumerates the user-visible breakages:
>
> - All external email clients (Thunderbird, Outlook, iOS Mail, etc.)
>   need their IMAP/SMTP server hostname reconfigured
> - CalDAV/CardDAV clients need new server URLs
> - Nextcloud Mail profiles for **remote-auth** mailboxes (auto-discovered
>   via the external FQDN) re-prompt for the user's AD password and
>   auto-update on the next login
> - Nextcloud Mail profiles for **local-auth** users are unaffected —
>   those profiles use internal Docker hostnames (`hermes_postfix_dkim`,
>   `hermes_dovecot`), not the external FQDN
>
> Plan the change for a maintenance window, notify users, and have new
> client setup instructions ready.

Validation: email-trick again (`IsValid("email", "test@<value>")`).
Empty → `session.m = 3`; invalid → `session.m = 5`.

After a successful save, also ensure a matching TLS certificate is
bound for SMTP on [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings). The
hostname change does not automatically rebind the cert; both must
match for STARTTLS handshakes to verify.

### Host IP Address

The operator-confirmed IPv4 address of the Docker host. Used to
populate Nextcloud's `trusted_domains` so NC accepts requests routed
through the IP literally (some autoconfig and CalDAV/CardDAV clients
hit the IP before they have the FQDN).

Validation: `^(\d{1,3}\.){3}\d{1,3}$` — basic IPv4 dotted-quad. Empty
is allowed (skips the regen of that field). Invalid → `session.m = 6`.

**The Host IP and the Console Address are independent.** If the
Console Address on [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) is set to an
**IP** (rather than an FQDN) and the host IP changes, you must update
both pages — neither cascades into the other. If Console Address is an
FQDN, only this page needs the IP update.

## Save flow

Clicking **Save & Apply Settings** posts `action=save_settings`, which
runs `save_server_identity.cfm`:

```
1. Validate all three fields (presence + format)
2. UPDATE parameters2.value2 WHERE parameter = 'server_ip'
3. UPDATE parameters.enabled = '1' WHERE parameter IN ('myorigin','myhostname')
   AND child = '2' AND module = 'postfix'         (re-arm both directives)
4. UPDATE parameters.parameter = <domain>
   WHERE parent_name = 'myorigin'  AND child = '1' AND module = 'postfix'
5. UPDATE parameters.parameter = <hostname>
   WHERE parent_name = 'myhostname' AND child = '1' AND module = 'postfix'
6. INCLUDE generate_postfix_configuration.cfm   (rewrites main.cf + reload)
7. INCLUDE generate_nextcloud_configuration.cfm (rewrites NC config.php)
8. cflocation back to view_server_setup.cfm with session.m = 1 (success)
```

There is no nginx restart in this cascade — only **Postfix** and
**Nextcloud** are touched. That is deliberate: nothing in the
nginx-served path consumes `myorigin`, `myhostname`, or the network
`server_ip` (the nginx vhosts use the **Console** Address, configured
separately). The save flow is therefore much lighter than Console
Settings: typically 5–10 seconds, no overlay spinner, no preload-style
restart.

`generate_postfix_configuration.cfm` re-templates
`config/postfix-dkim/etc/postfix/main.cf` from the live `parameters`
rows (walking every `child = 2` row that has `enabled = 1`, emitting
each as `<keyword> = <value>` with values pulled from the matching
`parent_name`-linked `child = 1` rows), copies the result into the
`hermes_postfix_dkim` container, and runs `postfix reload`. The reload
is a SIGHUP — it does **not** drop in-flight SMTP connections; mail
flow continuity is preserved across the save.

`generate_nextcloud_configuration.cfm` rewrites the entire
`config.php` from its template (`/opt/hermes/templates/config.php`),
substituting the host IP into `trusted_domains` along with all the
other NC settings the regenerator owns. Existing
installation-specific values (`passwordsalt`, `secret`, `instanceid`,
`version`) are read back from the live file first and preserved — the
regenerator never invents new versions of these or NC would think it
needs to re-install.

## Failure semantics

| What breaks | What happens |
|---|---|
| Validation fails on any field | `session.m = 2..6`, `cflocation` back to the page, no DB write |
| `parameters` UPDATE succeeds but `generate_postfix_configuration.cfm` fails to write | DB is ahead of the live config. Next save (or any other Postfix-config save) re-regenerates `main.cf` from the same DB rows and catches up. |
| `postfix reload` fails inside the container | DB and on-disk config are in sync but the running Postfix is still on the old config. Symptom: outbound mail still uses the old `myhostname`. Recovery: `docker exec hermes_postfix_dkim postfix reload` manually, or re-save. |
| `generate_nextcloud_configuration.cfm` fails (e.g., NC container down) | Postfix change is committed; NC is stale. Recovery: bring NC up and re-save, or re-run the regen include directly. |
| Hostname change breaks reverse DNS at the recipient | Hermes accepts the change cleanly; the visible failure is **deferred** — outbound mail starts getting rejected by other MTAs minutes to hours later. Always verify PTR + matching A record **before** changing `myhostname`. |

The save flow has no rollback. The previous `main.cf` lives at
`config/postfix-dkim/etc/postfix/main.cf.HERMES` (the CFML write-time
backup convention) and can be restored manually if a regen produces
broken syntax — but the DB has already advanced.

## Known limitation — Docker subnet is hardcoded

The Docker subnet that Postfix and Amavis trust (`IPV4SUBNET=172.16.32`
in `.env`) is **not** managed on this page. It is currently hardcoded
into 15+ config files spanning Postfix (`mynetworks`, `master.cf`),
Amavis (`@inet_acl`), Dovecot (`login_trusted_networks`), Ciphermail
(`authorizedAddresses`), OpenDKIM/OpenDMARC (`TrustedHosts`), and
several CFML queries.

If you need to change the subnet for IP-conflict reasons, **all 15+
files must be updated coherently** or mail flow will break in
subtle ways (Amavis rejecting messages from Hermes itself, OpenDKIM
not signing outbound, etc.). This is a tracked tech-debt item — when
templating is added, the subnet will move into `system_settings` and
get its own admin page rather than living on this one.

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_server_setup.cfm` | `hermes_commandbox` | Page |
| `config/hermes/var/www/html/admin/2/inc/save_server_identity.cfm` | `hermes_commandbox` | Save handler |
| `config/hermes/var/www/html/admin/2/inc/generate_postfix_configuration.cfm` | `hermes_commandbox` | `main.cf` regen + `postfix reload` |
| `config/hermes/var/www/html/admin/2/inc/generate_nextcloud_configuration.cfm` | `hermes_commandbox` | NC `config.php` regen (trusted_domains) |
| `config/postfix-dkim/etc/postfix/main.cf` | `hermes_postfix_dkim` (mounted) | Live Postfix config — regen target |
| `config/postfix-dkim/etc/postfix/main.cf.HERMES` | `hermes_postfix_dkim` (mounted) | Write-time backup of the previous live config |
| `/var/www/html/config/config.php` inside `hermes_nextcloud` | `hermes_nextcloud` | Live Nextcloud config — regen target |
| `parameters` rows where `module = 'postfix'`, `parent_name IN ('myorigin','myhostname')` | `hermes_db_server` (`hermes` DB) | The directive values |
| `parameters2` row where `parameter = 'server_ip'` | `hermes_db_server` (`hermes` DB) | Host IP |

The Postfix reload uses the standard
`docker exec hermes_postfix_dkim /usr/sbin/postfix reload` pattern.
The Nextcloud regen rewrites the bind-mounted `config.php` directly,
no `occ` calls — NC picks up the change on the next request because
`config.php` is read per-request.

## Related

- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — the web-side identity (Console Address, Console Certificate). Companion to this page.
- [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) — bind a TLS certificate to the Mail Server Hostname so STARTTLS handshakes verify
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — issue / renew the cert that SMTP TLS Settings binds
- [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings) — other globals (timezone, language) not part of server identity
- [Release engineering and updates](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology) — initial install flow that populates these values for the first time

# SMTP TLS Settings

# SMTP TLS Settings

Admin path: **System > SMTP TLS Settings** (`view_smtp_tls_settings.cfm`,
`inc/get_smtp_tls_settings.cfm`, `inc/get_smtp_tls_policies.cfm`,
`inc/edit_smtp_tls_settings.cfm`, `inc/smtp_tls_save_settings.cfm`,
`inc/smtp_tls_add_domain.cfm`, `inc/smtp_tls_edit_domain.cfm`,
`inc/smtp_tls_delete_domain.cfm`, `inc/generate_tls_policy.cfm`,
`inc/generate_postfix_configuration.cfm`).

This page configures **Postfix TLS** end to end: the global
inbound/outbound TLS mode (Disabled / Opportunistic / Mandatory), the
certificate Postfix presents on `:25`/`:587`, and per-destination-domain
TLS policy overrides for outbound delivery.

Pairs with [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates), which owns
the certificate **store**; this page is the **binding** of one of those
certs to the Postfix `smtpd_tls_*` / `smtp_tls_*` directives. Pairs
also with [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup), which owns the SMTP banner
hostname (`myhostname`) — the cert's Common Name or SAN must match that
hostname for strict STARTTLS verifiers to accept the handshake.

## TLS modes

```
+----------------------------------------------------------------+
|                    Postfix smtpd_tls_security_level             |
|                    +  smtp_tls_security_level                   |
+----------------------------------------------------------------+
   |                       |                          |
   |  '' (Disabled)        |  'may' (Opportunistic)   |  'encrypt' (Mandatory)
   v                       v                          v
 no STARTTLS         STARTTLS offered; clear-     STARTTLS required;
 advertised          text fallback if peer        peer must support it
 (cleartext only)    can't negotiate              or delivery fails
```

| Mode (`tlsmode` form value) | Postfix value | Use when |
|---|---|---|
| **Disabled** (`""`) | (directive value cleared) | Cleartext-only environments (test, isolated networks); production Internet exposure not recommended |
| **Opportunistic TLS** (`may`) — **Recommended** | `may` | Standard public-Internet config. STARTTLS is advertised; peers that support it use it, peers that don't fall back to cleartext |
| **Mandatory TLS** (`encrypt`) — **NOT recommended for Internet-facing servers** | `encrypt` | Closed networks where every peer is known to support TLS. On the open Internet this **drops mail** from any sender that can't negotiate STARTTLS, which is a long tail of misconfigured small senders |

The mode applies symmetrically to inbound (`smtpd_*`) and outbound
(`smtp_*`). Both directive rows are written on save.

## Selecting a certificate

The **SMTP TLS Certificate** field is a free-text autocomplete that
searches `system_certificates` via the `getcertificates.cfm` ajax
endpoint (the same endpoint used by Console Settings). Picking a row
populates a hidden `certificateno_1` field with the row ID plus four
read-only display fields (Subject, Issuer, Serial, Type).

The certificate picker is **hidden when TLS mode is Disabled**
(`#tlscertificate` div toggled by `#tlsmode` change handler). Switching
back to Opportunistic or Mandatory slides it back into view.

### The system-cert refusal

If an admin tries to save with the system-managed (bootstrap snakeoil)
cert selected, the handler refuses with **error 3**:

> You cannot select the system-self-signed Certificate for SMTP TLS.

This is intentional. A self-signed cert on `:25` would defeat the
purpose — strict STARTTLS verifiers on the receiving side reject the
handshake, and Hermes would silently lose all outbound mail to those
recipients. The refusal forces the admin to import a real cert
(commercial CA, internal PKI, or Let's Encrypt) before flipping TLS on.

The error message text is dated — the comparison is against
`certificateno_1 = 1` in `edit_smtp_tls_settings.cfm`, which works on
Docker fresh installs (where the bootstrap row is `id = 1`) but does
**not** work on installs where the system cert was assigned a different
ID (notably DEV's `ssl-cert-snakeoil` row at `id = 29`). The
[System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates#the-system-column-and-the-system-badge)
runtime helper resolves this for the deletion guard; the SMTP-TLS save
handler still uses the hardcoded `id = 1` check. Practical impact is
small because in either case the admin should not be selecting the
system row, but if you migrate from a legacy install with a non-`id=1`
system row, the SMTP page won't refuse the snakeoil even though the
System Certificates page will block its deletion.

## How directive values are stored

This page is the canonical example of the dual-row `parameters` table
pattern documented in
[Server Setup § Configuration storage](https://docs.deeztek.com/books/administrator-guide/page/server-setup#configuration-storage--the-parameters--parameters2-split).
Each Postfix directive has two rows:

| Row | `parameter` | `child` | `parent_name` | Role |
|---|---|---|---|---|
| Name row | `smtpd_tls_security_level` | `2` | — | Directive **name** |
| Value row | `may` / `encrypt` / `""` | `1` | `smtpd_tls_security_level` | Directive **value** |

Save handler `edit_smtp_tls_settings.cfm` writes to the value row only:

```sql
UPDATE parameters
   SET parameter = '<tls_mode>'
 WHERE parent_name = 'smtpd_tls_security_level'
   AND child = '1'
   AND enabled = '1';

-- same for smtp_tls_security_level (outbound)
-- and smtpd_tls_cert_file, smtpd_tls_key_file, smtpd_tls_CAfile
--   (paths resolved from system_certificates.file_name + type)
```

The selected cert's on-disk paths are derived from
`system_certificates.type` + `file_name`:

| `type` | `smtpd_tls_cert_file` | `smtpd_tls_key_file` | `smtpd_tls_CAfile` |
|---|---|---|---|
| `Imported` | `/opt/hermes/ssl/<file_name>_hermes.pem` | `/opt/hermes/ssl/<file_name>_hermes.key` | `/opt/hermes/ssl/<file_name>_hermes.chain.pem` |
| `Acme` | `/etc/letsencrypt/live/<file_name>/cert.pem` | `/etc/letsencrypt/live/<file_name>/privkey.pem` | `/etc/letsencrypt/live/<file_name>/chain.pem` |

The same path-derivation logic is implemented globally in
`inc/get_active_cert_paths.cfm` for the console binding; the SMTP save
handler open-codes it here (technical debt — the path arithmetic should
be moved to the helper so there's only one place that knows the layout).

The new directive values land in the `parameters` table, then
`generate_postfix_configuration.cfm` regenerates `main.cf` from the
live rows and runs `postfix reload`. Mode changes therefore take effect
on the next SMTP connection without dropping in-flight sessions
(`postfix reload` is a SIGHUP, not a restart).

## What this page does NOT configure

Hermes' TLS surface is opinionated by design. The page deliberately
omits several knobs that Postfix exposes:

| Concern | Status |
|---|---|
| Cipher suite (`smtpd_tls_ciphers`, `smtpd_tls_mandatory_ciphers`) | Hardcoded in `main.cf` baseline; no UI |
| Protocol versions (`smtpd_tls_protocols`, `smtpd_tls_mandatory_protocols`) | Hardcoded in `main.cf` baseline; no UI |
| DH parameters (`smtpd_tls_dh1024_param_file`) | Same ECDHE-only decision as Console Settings — DH is not offered |
| TLS session cache | Hardcoded defaults |
| EECDH curve | Hardcoded defaults |
| Per-mailbox-domain certs (autoconfig/autodiscover) | Lives on [SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management); this page binds the **single** cert Postfix presents on the public SMTP banner |
| Dovecot IMAP/POP cert | Email Server > Settings (separate `mail.certificate` binding) |
| Console (nginx) cert | [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) |

The cipher / protocol decisions are baked into the Postfix baseline
config because they have global security implications and changing them
needs more than a dropdown — there's no curated "modern / intermediate
/ legacy" preset UI yet, and the right defaults for an SEG track
[Mozilla's modern profile](https://wiki.mozilla.org/Security/Server_Side_TLS)
which doesn't churn often enough to warrant operator-tunable UI.

## TLS Policy Domains — per-destination outbound overrides

Below the global card is the **TLS Policy Domains** table. Each row
forces a stricter-than-global TLS policy for outbound mail to a specific
recipient domain.

| Field | Meaning |
|---|---|
| **Domain** | Recipient domain (`example.com`) or domain-and-subdomains pattern (`.example.com` — leading dot matches all subdomains) |
| **Encryption Mode** | Currently always **Mandatory** (`encrypt`) for manually-added rows. Per-row mode tunables are tracked but not exposed. |
| **Note** | Free-text description shown in the row |

Adding a row generates `/etc/postfix/tls_policy` (via
`generate_tls_policy.cfm`), runs `postmap` to compile it into a hash
map, and reloads Postfix:

```
docker exec hermes_postfix_dkim /usr/sbin/postmap /etc/postfix/tls_policy
```

The Postfix daemon then consults the map for every outbound SMTP
connection — entries matching the destination domain override
`smtp_tls_security_level` for that specific destination.

> **Operational consequence.** Adding a `encrypt` policy for a recipient
> domain whose MX **doesn't actually support STARTTLS** silently breaks
> outbound mail to that domain. Postfix will defer + bounce. Verify the
> recipient MX advertises STARTTLS before adding a Mandatory entry. The
> warning callout on the page itself spells this out.

### Auto-added rows (managed by Domains)

When a domain on Email Server > Domains or Email Relay > Domains is
configured to require SASL authentication, Hermes auto-inserts a TLS
policy row to enforce encryption for that destination. These rows are
marked by `description = 'Auto-added: domain requires authentication'`
and rendered with a special **Managed by Domains** badge:

- The row's **checkbox** is suppressed (cannot be bulk-deleted from
  here)
- The row's **Edit** button is suppressed (must be edited on the
  managing page)
- The Note column links to **view_domains.cfm** so the admin lands on
  the right page

This is the same pattern used elsewhere in Hermes for system-owned
rows that would otherwise look user-editable — surface that the row is
managed somewhere else and link to the managing page.

## Save flows

### Save SMTP TLS Settings (`save_settings`)

```
1. Validate form.tlsmode in ("", "may", "encrypt")
2. UPDATE parameters value rows for smtpd_tls_security_level + smtp_tls_security_level
3. If tlsmode is not "" :
     a. Validate certificateno_1 exists in system_certificates
     b. Refuse if certificateno_1 = 1 (legacy bootstrap-id check)
     c. UPDATE parameters2 smtp.certificate
     d. Derive cert/key/CA paths from type + file_name
     e. UPDATE parameters value rows for smtpd_tls_cert_file / smtpd_tls_key_file / smtpd_tls_CAfile
4. generate_postfix_configuration.cfm  (regenerate main.cf + postfix reload)
5. session.m = 35 ("settings saved successfully. Postfix reloaded.")
6. cflocation back to view_smtp_tls_settings.cfm
```

### Add / Edit / Delete TLS Policy Domain (`add_domain` / `edit_domain` / `delete_domain`)

```
1. Validate domain (email-trick: IsValid("email", "bob@<domain>"))
   - Leading "." accepted; validator prepends "subdomain"
2. INSERT / UPDATE / DELETE in tls_policies
3. generate_tls_policy.cfm   (rewrite /etc/postfix/tls_policy + postmap)
4. generate_postfix_configuration.cfm  (postfix reload)
5. session.m = 37 / 39 / 34 (per action)
6. cflocation back to view_smtp_tls_settings.cfm
```

Both save flows end in a `postfix reload`, which is a SIGHUP — no
in-flight SMTP connections are dropped, and queued mail continues
delivering normally.

## Failure semantics

| What breaks | What happens |
|---|---|
| Mode = Opportunistic/Mandatory + Certificate empty | `m = 1`, "SMTP TLS Certificate cannot be blank when TLS Mode is set to Opportunistic or Mandatory" |
| Certificate ID does not exist in `system_certificates` | `m = 2`, "The SMTP TLS Certificate you entered is not valid" |
| Certificate ID is `1` (legacy bootstrap check) | `m = 3`, "You cannot select the system-self-signed Certificate for SMTP TLS" |
| Domain validation fails on add/edit | `m = 4` |
| Duplicate domain on add | `m = 5` |
| Duplicate domain on edit | `m = 6` |
| Missing required form field | `m = 20` |
| `generate_tls_policy.cfm` fails (cp / mv / postmap) | DB is ahead of the live `tls_policy.db`. Next save re-renders cleanly. The previous live map is preserved as `/etc/postfix/tls_policy.HERMES.BACKUP`. |
| `postfix reload` fails inside the container | DB and on-disk config in sync; running daemon stale. Recovery: `docker exec hermes_postfix_dkim postfix reload` manually. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_smtp_tls_settings.cfm` | `hermes_commandbox` | Page |
| `config/hermes/var/www/html/admin/2/inc/edit_smtp_tls_settings.cfm` | `hermes_commandbox` | Save handler (mode + cert binding) |
| `config/hermes/var/www/html/admin/2/inc/smtp_tls_save_settings.cfm` | `hermes_commandbox` | Action handler wrapper around `edit_smtp_tls_settings.cfm` |
| `config/hermes/var/www/html/admin/2/inc/smtp_tls_add_domain.cfm` | `hermes_commandbox` | TLS Policy add |
| `config/hermes/var/www/html/admin/2/inc/smtp_tls_edit_domain.cfm` | `hermes_commandbox` | TLS Policy edit |
| `config/hermes/var/www/html/admin/2/inc/smtp_tls_delete_domain.cfm` | `hermes_commandbox` | TLS Policy delete |
| `config/hermes/var/www/html/admin/2/inc/generate_tls_policy.cfm` | `hermes_commandbox` | Render `/etc/postfix/tls_policy` + `postmap` |
| `config/hermes/var/www/html/admin/2/inc/generate_postfix_configuration.cfm` | `hermes_commandbox` | `main.cf` regen + `postfix reload` |
| `/etc/postfix/main.cf` | `hermes_postfix_dkim` (mounted) | Live Postfix config — regen target |
| `/etc/postfix/tls_policy` + `tls_policy.db` | `hermes_postfix_dkim` (mounted) | Live TLS-policy map (text + postmap-compiled) |
| `/etc/postfix/tls_policy.HERMES.BACKUP` | `hermes_postfix_dkim` (mounted) | Write-time backup of the previous live map |
| `parameters` rows for `smtpd_tls_*` and `smtp_tls_*` | `hermes_db_server` (`hermes` DB) | Directive values |
| `parameters2.smtp.certificate` | `hermes_db_server` (`hermes` DB) | Active SMTP cert binding (FK into `system_certificates.id`) |
| `tls_policies` table | `hermes_db_server` (`hermes` DB) | Per-destination overrides |

Every shell-out uses `docker exec hermes_postfix_dkim ...` per the
standard Hermes Docker pattern. `postmap` is the one operation that
absolutely **must** run inside the container — the `tls_policy.db`
hash format is libdb-version-sensitive, and running it on the host
produces a file Postfix inside the container can't read.

## Related

- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — the certificate store this page selects from; system-managed certs cannot be bound for SMTP
- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — Mail Server Hostname (`myhostname`); the SMTP cert's Subject CN or SAN should match this for strict STARTTLS verifiers
- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — the console-side analogue of this page (binds a System Certificate to nginx)
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) — Authelia / SASL; per-domain SASL requirements auto-insert `tls_policies` rows here
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — upstream LDAP TLS settings; separate CA store at `/opt/hermes/certs/remoteauth/`, not part of System Certificates
- [SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management) — per-mailbox-domain certs for autodiscover/autoconfig; orthogonal to the single SMTP cert this page binds
- [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) — Fail2ban; not TLS-related but relevant for hardening the SMTP service this page configures
- [Admin Console Firewall](https://docs.deeztek.com/books/administrator-guide/page/console-firewall) — IP allowlist for the console (not SMTP); SMTP is open to the Internet for inbound mail

# System Certificates

# System Certificates

Admin path: **System > System Certificates** (`view_system_certificates.cfm`,
`inc/cert_action.cfm`, `inc/import_certificate.cfm`,
`inc/acme_request_certificate.cfm`, `inc/acme_request_san_certificate.cfm`,
`inc/delete_system_certificate.cfm`, `inc/parse_certificate_details.cfm`,
`inc/get_system_cert_ids.cfm`, `inc/get_active_cert_paths.cfm`).

This is the **canonical certificate store** for Hermes. Every X.509
certificate the gateway presents to the outside world is registered as a
row in `system_certificates` and selected by ID from one of the binding
pages: [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) (web console),
[SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) (Postfix SMTP banner), Email
Server > Settings (Dovecot IMAP/POP/Submission), and
[SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management) (per-mailbox-domain
autodiscover/autoconfig).

The page itself is purely a CRUD store plus a CSR generator and the
Let's Encrypt (ACME) integration. It does **not** bind certs to services
— that happens on the consuming pages, each of which writes to its own
row in `parameters2`.

## Where certificate files live

The store has two ingest paths plus a system-managed placeholder. Each
lays down files in a different directory tree.

| `type` | On-disk pattern | Source |
|---|---|---|
| `Imported` | `/opt/hermes/ssl/<file_name>_hermes.pem` (leaf), `.key`, `.chain.pem`, `.bundle.pem` (leaf + chain concatenated) | **Import Certificate** modal or **Generate CSR** → external CA → import |
| `Acme` | `/etc/letsencrypt/live/<file_name>/{fullchain,cert,privkey,chain}.pem` | **Request ACME Certificate** modal; renewals via Ofelia-scheduled certbot runs |
| `Imported` (system) | `/opt/hermes/ssl/bootstrap_hermes.{bundle.pem,key,...}` (Docker fresh installs); `/etc/ssl/{certs,private}/ssl-cert-snakeoil.{pem,key}` (legacy non-Docker) | Installer (`install_hermes_docker.sh`) or Ubuntu `ssl-cert` package |

The `bootstrap` cert is a self-signed snakeoil that ships with every
fresh Docker install — Hermes needs **something** to bind to before the
admin imports a real cert. It is reserved as a placeholder for newly-added
mailbox domains; consumers that actually need a publicly-trusted cert
(SMTP TLS, the console) refuse to bind to it (see
[SMTP TLS Settings § Selecting a certificate](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings#selecting-a-certificate)).

## The `system` column and the SYSTEM badge

The `system_certificates.system` column (added by issue #252) is a
boolean flag marking install-generated rows. The UI surfaces this two
ways:

| Surface | Behavior when `system = 1` |
|---|---|
| **SYSTEM badge** next to the friendly name | Rendered as a gray pill in the Name column |
| **Delete** button | Disabled with a tooltip ("System-managed certificate — cannot be deleted. Used as a placeholder when binding mailbox domains before a real cert is imported.") |

The delete-protection gate lives in `cert_action.cfm` and re-checks
`system = 1` server-side so a crafted POST cannot bypass the disabled
button.

> **Legacy vs Docker file_name.** Fresh Docker installs have
> `file_name = 'bootstrap'`. Legacy non-Docker installs that survived a
> migration have `file_name = 'ssl-cert-snakeoil'` (from the Ubuntu
> `ssl-cert` package). Both are flagged `system = 1` on installs where
> the column exists. The `inc/get_system_cert_ids.cfm` helper resolves
> the row IDs at runtime — code that needs to know "is this a system
> cert" reads from the helper, never from a hardcoded `id = 1`. This is
> the only correct gating signal; `version_no = 'Docker'` does **not**
> tell you which file_name pattern applies because both DEV (Docker,
> legacy install vintage) and Test (Docker, fresh install) report the
> same version string.

## Cert path resolver — `get_active_cert_paths.cfm`

Most consumers don't want the row ID — they want the actual on-disk
paths to pass to `nginx ssl_certificate`, `openssl cms -sign`, Postfix's
`smtpd_tls_cert_file`, etc. The path layout differs between Imported
(`/opt/hermes/ssl/...`) and ACME (`/etc/letsencrypt/live/.../...`), and
the same logical name maps to different files for different consumers
(`fullchain.pem` for nginx vs `cert.pem` for openssl signer).

`inc/get_active_cert_paths.cfm` is the single place that knows this
layout. It reads the active console certificate from `parameters2`,
joins to `system_certificates`, and writes six caller-visible variables:

| Variable | Purpose |
|---|---|
| `hermesCertType` | `"Imported"`, `"Acme"`, or `"Snakeoil"` |
| `hermesCertIsSnakeoil` | `true` when no real cert is bound (signing callers must skip) |
| `hermesCertNginxPath` | Cert for `nginx ssl_certificate` (bundle for Imported, fullchain for Acme) |
| `hermesCertKeyPath` | Private key |
| `hermesCertSignerPath` | Leaf cert only — for `openssl cms -sign` |
| `hermesCertChainPath` | Intermediates only — for `openssl cms -sign -certfile` |

Any new code that touches certificate files should `cfinclude` this
helper rather than reinventing the path arithmetic. The legacy hardcoded
fallback (`/etc/ssl/certs/ssl-cert-snakeoil.pem`) was removed in #251
because the minimal Docker container doesn't have the `ssl-cert` package
and nginx crashed with `BIO_new_file` errors on the missing file.

## Three ingest paths

### 1. Request ACME Certificate (Pro feature)

The **Request ACME Certificate** button issues a Let's Encrypt cert via
an ephemeral certbot container. Disabled when no Pro license is active.

```
Admin clicks Request -> view_system_certificates.cfm action=requestacme
   -> inc/acme_request_certificate.cfm
       docker run --rm --name hermes_certbot --network host \
         -v <repo>/config/hermes/var/www/html:/var/www/certbot \
         -v <repo>/config/certbot/conf:/etc/letsencrypt \
         -v <repo>/config/certbot/logs:/var/log \
         certbot/certbot:latest \
         certonly --webroot --webroot-path /var/www/certbot \
         --email <admin> --agree-tos --no-eff-email \
         [--dry-run]   # staging mode
         -d <domain>
```

- **Staging** mode adds `--dry-run` and never lands a real cert. Always
  test with Staging first to confirm DNS + ports 80/443 work; Let's
  Encrypt's production rate limits will lock the domain out for a week
  if you burn through them with broken HTTP-01 challenges.
- The webroot is mounted to `/var/www/certbot` so certbot can write the
  challenge file where the live nginx vhost expects it.
- Certs land in `config/certbot/conf/live/<domain>/` (bind-mounted to
  `/etc/letsencrypt/live/<domain>/` in the commandbox container).
- Renewals are driven by Ofelia (Scheduled Tasks). Each renewal runs the
  same ephemeral certbot container with `renew`; if the renewal
  succeeds, dependent services (nginx, Postfix, Dovecot) reload to pick
  up the new files.
- ACME certs cannot be renewed manually from this page — the row exists
  for binding and deletion only; renewals are scheduled and silent.

Per-mailbox-domain ACME SAN certs (autoconfig + autodiscover + custom
prefixes) use a separate code path
(`inc/acme_request_san_certificate.cfm`) wired to
[SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management). Both paths land
rows in the same `system_certificates` table.

### 2. Import Certificate

For certs issued by any CA other than Let's Encrypt (commercial CA,
internal PKI, etc.). The admin pastes three PEM blobs in the **Import
Certificate** modal:

| Field | Contents |
|---|---|
| Certificate (PEM) | Leaf cert between `-----BEGIN CERTIFICATE-----` and `-----END CERTIFICATE-----` |
| Unencrypted Key (PEM) | Private key — must be **unencrypted** (no passphrase). Encrypted keys are rejected because nginx / Postfix cannot prompt for a passphrase at startup. |
| Root & Intermediate CA Certificates (PEM) | Chain — root + intermediates concatenated, leaf-omitted |

On save, Hermes writes four files under `/opt/hermes/ssl/`:

```
<file_name>_hermes.pem            (leaf only)
<file_name>_hermes.key            (private key)
<file_name>_hermes.chain.pem      (CA chain, no leaf)
<file_name>_hermes.bundle.pem     (leaf + chain — for nginx ssl_certificate)
```

`<file_name>` is derived from the friendly name with special characters
sanitized. The row is inserted with `type = 'Imported'` and the
extracted Subject/Issuer/Serial/Fingerprint cached in the table for the
expandable row preview.

### 3. Generate CSR

For admins who want to use their own CA but don't have a key+CSR yet.
The modal collects DN fields (Country, State, Locality, Organization,
Department) plus a **Certificate purpose** radio toggle that drives the
rest of the form:

| Purpose | CN source | SANs |
|---|---|---|
| **Server certificate** (single-name DV, ~$10/yr) | Admin enters Common Name field directly | Admin-entered FQDNs only |
| **Mailbox certificate** (SAN / UCC, $50–$200/yr) | Auto-derived as `<first-prefix>.<mailbox_domain>` matching Pro ACME's first-`-d`-flag behavior | Mandatory: `autoconfig.<domain>`, `autodiscover.<domain>`, plus every prefix from `additional_sans`. Additional admin entries auto-expand bare prefixes against the mailbox domain. |

Smart default: if `mailbox_domains` has any rows, the modal defaults to
**Mailbox**; otherwise it defaults to **Server**. The page-level
"Choosing the Right Certificate Type" card above the table walks the
admin through the cost difference and the "a basic DV cert will not
work for mailboxes" trap.

On submit, Hermes generates a 2048- or 4096-bit RSA key + matching CSR
and bundles them into a `.rar` archive at
`/opt/hermes/tmp/<token>_csr_key.rar`. The CSR-pending state is then
surfaced as a **persistent callout** at the top of the page (added by
#249) — the **Download CSR** button stays visible across page reloads
until the admin clicks **Discard**. Submit the CSR to the chosen CA,
receive the signed cert + chain, then come back and use **Import
Certificate** (steps 1 above) to register it. The private key in the
`.rar` is what you'll need for the import.

> **The CSR private key never leaves Hermes until the admin downloads
> the bundle.** If the admin clicks Discard without downloading,
> the key is gone — there is no recovery. The Discard button warns about
> this; the persistent callout pattern (#249) was introduced because the
> one-shot download button that used to live inside the success alert
> was easy to miss on a page reload.

## Service binding cross-reference

The certificate-store rows are referenced from four service-binding
locations. Each location keeps its **own** copy of the cert ID — there
is no cascading delete, so the deletion guard (next section) walks all
four before allowing a row to be removed.

| Service | Where the binding lives | Set on page |
|---|---|---|
| Console (admin, user portal, NC, Ciphermail) | `parameters2.parameter = 'console.certificate', module = 'console'` | [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) |
| SMTP (Postfix `smtpd_tls_cert_file`) | `parameters2.parameter = 'smtp.certificate', module = 'certificates'` | [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) |
| Webmail (Dovecot IMAP/POP) | `parameters2.parameter = 'mail.certificate', module = 'certificates'` | Email Server > Settings |
| Mailbox SAN (per-domain autodiscover/autoconfig) | `mailbox_domains.mailbox_certificate` (multiple rows possible) | Email Server > Domains, [SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management) |

The page renders four YES/NO columns (Console / SMTP / Webmail /
Mailbox SAN) so an admin can see at a glance which services a given
cert is in use by.

## Deletion guard

`inc/delete_system_certificate.cfm` walks every consumer before allowing
a delete:

```
1. system column flag         -> system-managed, refuse
2. parameters2 console.certificate    -> assigned to Web Service, refuse
3. parameters2 smtp.certificate       -> assigned to SMTP Service, refuse
4. parameters2 mail.certificate       -> assigned to Mail Service, refuse
5. (mailbox_domains.mailbox_certificate check is in cert_action.cfm)
6. -> DELETE FROM system_certificates WHERE id = ?
7.    plus filesystem cleanup:
        Imported: rm /opt/hermes/ssl/<file_name>_hermes.{pem,key,chain.pem,bundle.pem}
        Acme:     docker run --rm certbot/certbot:latest delete --cert-name <file_name>
                  + DELETE FROM mailbox_domains_sans WHERE acme_certificate = ?
```

The guard is **stop-on-first-match** with a specific error message per
case so the admin knows which binding is blocking the delete and where
to go to unbind. There is no "force delete" — the only way past the
guard is to unbind on the consuming page first.

## Certificate downloads (gated)

Each row has an expandable details panel with **Download Certificate**,
**Download Private Key**, and **Download CA Chain** buttons. By default
these are **disabled** for safety (downloading a private key over a web
page is a sensitive operation). To enable, set

```
ALLOW_CERT_DOWNLOAD=yes
```

in `/opt/hermes/config/security.conf` on the host filesystem. The page
reads this file on every load (cached in the local request). When the
toggle is off, the buttons render disabled with a tooltip telling the
admin where to set the flag.

Downloads are streamed via a hidden iframe + `class="no-preloader"`
pattern (standard Hermes binary-download convention) so the page's
spinner overlay doesn't get stuck.

## SAN validation sub-table (Pro feature)

When a row is bound to one or more entries in `mailbox_sans`
(autodiscover/autoconfig/custom subdomains for a mailbox domain), the
expanded details panel includes a **Mailbox SAN Validation** sub-table
showing IP-resolve and DNS-resolve status for each SAN. This is
populated by the
[SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management) validator and is
read-only here — it answers "do all the SANs on this cert actually
resolve to this server?" at a glance.

## Failure semantics

| What breaks | What happens |
|---|---|
| CSR field validation (Country != 2 chars, bad CN chars, etc.) | `session.m` set with the specific error, `cflocation` back to the page, no file/DB writes |
| Mailbox CSR with empty `additional_sans` table | Refused with "No SAN prefixes configured in SAN Management. Cannot generate a mailbox certificate without at least autoconfig + autodiscover." |
| ACME staging dry-run fails (DNS, port 80, rate limit) | Raw certbot stderr surfaced in the error alert; no DB row added |
| ACME production fails | Same as staging — error alert with raw stderr |
| Import with mismatched key + cert | The import script's openssl-modulus check fails; error alert with detail |
| Delete blocked by binding | "The Certificate you are attempting to delete is assigned to the X Service" — admin must unbind first on the consuming page |
| `certbot delete` fails on ACME row | DB row kept, error surfaced; manual cleanup of the `/etc/letsencrypt/live/<name>/` tree may be needed |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_system_certificates.cfm` | `hermes_commandbox` | Page |
| `config/hermes/var/www/html/admin/2/inc/cert_action.cfm` | `hermes_commandbox` | Action router (CSR, import, ACME, delete, discard) |
| `config/hermes/var/www/html/admin/2/inc/acme_request_certificate.cfm` | `hermes_commandbox` | Single-domain ACME via certbot container |
| `config/hermes/var/www/html/admin/2/inc/acme_request_san_certificate.cfm` | `hermes_commandbox` | Multi-SAN ACME (mailbox certs) |
| `config/hermes/var/www/html/admin/2/inc/import_certificate.cfm` | `hermes_commandbox` | PEM paste-in handler |
| `config/hermes/var/www/html/admin/2/inc/delete_system_certificate.cfm` | `hermes_commandbox` | Deletion guard + filesystem cleanup |
| `config/hermes/var/www/html/admin/2/inc/parse_certificate_details.cfm` | `hermes_commandbox` | Single `openssl x509` parse for subject/issuer/SAN/etc. |
| `config/hermes/var/www/html/admin/2/inc/get_system_cert_ids.cfm` | `hermes_commandbox` | Resolver — which rows are system-managed |
| `config/hermes/var/www/html/admin/2/inc/get_active_cert_paths.cfm` | `hermes_commandbox` | Resolver — on-disk paths for the active console cert |
| `/opt/hermes/ssl/` | `hermes_commandbox` (bind-mounted) | Imported cert files |
| `/etc/letsencrypt/live/<domain>/` | `hermes_commandbox` (bind-mounted from `config/certbot/conf/`) | ACME cert files |
| `/opt/hermes/tmp/<token>_csr_key.rar` | `hermes_commandbox` | Pending CSR bundle |
| `/opt/hermes/config/security.conf` | host filesystem | `ALLOW_CERT_DOWNLOAD` toggle |
| `system_certificates` table | `hermes_db_server` (`hermes` DB) | The canonical store |
| `certbot/certbot:latest` image | docker.io | Pulled on demand; ephemeral per request |

Every certbot invocation is `docker run --rm` against the public
`certbot/certbot:latest` image — Hermes never runs certbot directly on
the host. The container shares the host network (`--network host`) so
Let's Encrypt's HTTP-01 challenge can reach port 80 on the public IP.

## Related

- [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) — bind a System Certificate to Postfix SMTP TLS
- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — bind a System Certificate to the web console (nginx) and its hardening toggles
- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — Mail Server Hostname; should match the CN/SAN on the SMTP cert for STARTTLS verification
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) — Authelia; uses the console cert via its nginx-fronted vhost
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — separate CA store at `/opt/hermes/certs/remoteauth/` for upstream LDAP; not a System Certificate
- [SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management) — per-mailbox-domain SAN prefixes that drive mailbox-cert CSR + ACME SAN issuance
- [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) — Fail2ban; not cert-related but documents the same nginx-restart cascade pattern this page avoids by not regenerating any nginx config
- [Admin Console Firewall](https://docs.deeztek.com/books/administrator-guide/page/console-firewall) — IP allowlist for the console; layered above the TLS termination this page's certs drive

# System Logs

# System Logs

Admin path: **System > System Logs** (`view_system_logs.cfm`,
`schedule/message_cleanup.cfm`).

This page is a SQL-backed log viewer over **rsyslog's `SystemEvents`
table** in the `Syslog` database. Every mail-side container in the
stack ships its `mail.*` syslog stream to MariaDB via the `ommysql`
rsyslog output module; this page reads from that table with a date
range, optional facility filter, and a row limit, and renders the
result in a sortable DataTable.

Pairs with [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue): the Mail Queue viewer shows
what Postfix is currently holding; this page shows the historical log
trail — connection negotiation, milter results, content-filter
verdicts, delivery outcomes, bounce generation — that explains *why*
a message did or did not make it through.

## The log pipeline — container `mail.*` to `SystemEvents`

```
  ┌──────────────────────────────┐
  │ hermes_postfix_dkim          │  mail.*
  ├──────────────────────────────┤
  │ hermes_mail_filter (Amavis)  │  mail.*
  ├──────────────────────────────┤
  │ hermes_opendmarc             │  mail.*
  ├──────────────────────────────┤
  │ hermes_openldap (slapd)      │  mail.*  (via slapd.conf rsyslog rule)
  ├──────────────────────────────┤
  │ hermes_openarc (optional)    │  mail.*
  └──────────────┬───────────────┘
                 │ each container runs its own rsyslogd with
                 │   /etc/rsyslog.d/mysql.conf:
                 │     $ModLoad ommysql
                 │     mail.* :ommysql:hermes_db_server,Syslog,USER,PASS
                 ▼
       ┌────────────────────────────┐
       │ hermes_db_server (MariaDB) │
       │   Syslog.SystemEvents      │
       │   Syslog.SystemEventsProperties (unused by viewer)
       └─────────────┬──────────────┘
                     │ SELECT ... ORDER BY ReceivedAt DESC LIMIT ?
                     ▼
       ┌────────────────────────────┐
       │ view_system_logs.cfm       │
       └────────────────────────────┘
```

The MySQL output config that wires each container into the pipeline is
templated at install time. Each service gets a per-container template
under `config/<service>/etc/rsyslog.d/mysql.conf.template` with
`__SYSLOG_USER__` / `__SYSLOG_PASS__` placeholders; the install script
substitutes the generated credentials and bind-mounts the rendered file
into the container at `/etc/rsyslog.d/mysql.conf`. There is no
container-side aggregator — every container talks to MariaDB directly.

> **Operational consequence.** If MariaDB is down or unreachable from a
> container, that container's `mail.*` log entries are buffered by
> rsyslog and then dropped when the buffer fills. Log gaps during a
> database outage are expected and are not a bug in the viewer.

## What lands in `SystemEvents` — and what doesn't

The `mail.*` selector covers everything that uses syslog facility 2
(`mail`) on the source containers. That is:

- All Postfix smtpd / cleanup / qmgr / smtp / lmtp / bounce / pickup
  output (connection logs, milter verdicts, `status=sent|deferred|
  bounced`, queue lifecycle events)
- All Amavis content-filter output (verdict, score, virus name,
  per-policy bank decisions)
- All OpenDMARC verdict lines (`policy=`, `disposition=`)
- All OpenDKIM signing and verifying output
- slapd's syslog output (only because `slapd.conf` is explicitly
  configured to use the `mail` facility — see [LDAP &
  RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth))
- OpenARC output if the optional service is enabled

What is **not** here:

- **nginx access / error logs** — not configured to ship to syslog;
  read them with `docker exec hermes_nginx tail -f /var/log/nginx/...`
  or via [Admin Console Firewall](https://docs.deeztek.com/books/administrator-guide/page/console-firewall) /
  [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) for the security
  view.
- **Authelia auth logs** — written to `/remotelogs/authelia/
  authelia.log` for fail2ban consumption; see
  [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) and
  [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips).
- **Dovecot login / IMAP logs** — written to
  `/remotelogs/dovecot/dovecot-info.log` for fail2ban; the LMTP
  delivery side that Postfix talks to is visible here because Postfix
  logs the LMTP handoff result.
- **CommandBox / Lucee application logs** — Lucee internal logs live
  under the Lucee server home on the data tier, not in `SystemEvents`.
- **Container stdout/stderr** — `docker logs <name>` only.

This page is the operator's one-stop view for *mail-flow* questions.
Auth and HTTP-side concerns have their own log surfaces.

## The `SystemEvents` schema

`config/database/syslog_schema.sql` defines the table (MyISAM,
`latin1_swedish_ci` — rsyslog's canonical schema, kept verbatim for
compatibility). The viewer touches only four columns:

| Column | Type | Used for |
|---|---|---|
| `ReceivedAt` | `datetime` | Date-range filter, sort order, displayed timestamp |
| `Message` | `text` | The log line body |
| `SysLogTag` | `varchar(60)` | Facility filter; rendered as a badge per row. Format is typically `<program>[<pid>]:` (e.g. `postfix/smtpd[12345]:`) |
| `Facility` | `smallint` | Present but not read by the viewer |

Two indexes ship in the baseline schema:

```sql
KEY `idx_systemevents_receivedat` (`ReceivedAt`),
KEY `idx_systemevents_tag_receivedat` (`SysLogTag`, `ReceivedAt`)
```

The composite covers both the bare date-range query and the
facility-filtered date-range query (which uses `SysLogTag LIKE
'<facility>%'` and an `ORDER BY ReceivedAt DESC`). Issue #184, which
tracked the missing indexes, was closed when these were added to
`syslog_schema.sql`; existing installs pick them up via the
`schema_updates.sql` path.

The Facility dropdown is populated by a separate query that pulls
distinct values of `SUBSTRING_INDEX(SysLogTag, '[', 1)` over the
current date range — so the available facilities reflect what actually
logged during the window, not a static enum.

## Fields on the page

### Log Retention

Stored in `parameters2` with `parameter = 'system_log_retention'` and
`module = 'systemlog'`. The dropdown offers 7 / 15 / 30 / 60 / 90 / 120
/ 180 days; the seed value is 30. Saving the form just updates the row
— it does not run the cleanup immediately.

The actual deletion runs in `schedule/message_cleanup.cfm`, scheduled
by Ofelia (the in-stack cron container) once per night. The cleanup
job reads the retention value, computes `today - N days`, and runs:

```sql
DELETE FROM SystemEvents WHERE ReceivedAt < '<cutoff-date>'
```

This is the same cleanup job that prunes the Amavis quarantine on the
data tier — log retention and quarantine retention share the schedule
but have independent thresholds. See [Scheduled
Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) for the full Ofelia job list and how to run
the cleanup on demand.

> **Operational consequence.** Changing the retention value does not
> shrink the table until the next nightly cleanup runs. To force an
> immediate prune after dialing the value down, trigger the cleanup
> job from the Scheduled Tasks page.

### Start Date / Time and End Date / Time

Tempus Dominus datetime pickers with second-level resolution. Defaults
are the last 24 hours (midnight-to-midnight on today rounded back).
Both go into the query as `cf_sql_timestamp` parameters via
`cfqueryparam` — there is no string concatenation in the SQL.

### Facility

A Tom Select multi-select. Empty (no chips) means "all facilities".
Selecting one or more populates a `SysLogTag LIKE '<facility>%'`
clause per chip, OR'd together. The facility list is recomputed every
time the page loads against the current date range — there is no
cached enum.

### Limit

One of `1000 / 1500 / 2500 / 5000 / 10000 / 15000`. The viewer
validates against this exact list and falls back to `1000` if an
out-of-range value is passed. A yellow callout appears when the
selected limit is 10000 or higher.

> **Why the cap and not unlimited.** The DataTable widget needs to
> render every row into the DOM up front (it does not use
> server-side pagination). A 10,000-row table is already heavy in the
> browser; an unbounded fetch on a multi-month-deep `SystemEvents`
> table would lock the page.

## Reading the badges

The Facility badge contains the raw `SysLogTag` value, which Postfix
and friends format as `<program>[<pid>]:`. A few high-frequency tags
worth recognising:

| Tag (prefix match) | Meaning |
|---|---|
| `postfix/smtpd` | Inbound SMTP — connection, EHLO, helo, rcpt, milter results |
| `postfix/cleanup` | Header normalisation, header_checks, milter signing |
| `postfix/qmgr` | Queue manager — message scheduling, expiry |
| `postfix/smtp` | Outbound delivery to remote MX |
| `postfix/lmtp` | Local delivery to Dovecot |
| `postfix/bounce` | Bounce message generation |
| `amavis` | Content-filter verdicts (`Passed CLEAN`, `Blocked SPAM`, virus names) |
| `opendkim` | DKIM signing on outbound, verifying on inbound |
| `opendmarc` | DMARC alignment verdicts |
| `openarc` | ARC seal verdicts (if enabled) |
| `slapd` | LDAP — bind / search / modify operations |

A row's badge is exact-match for sort but prefix-match for the
filter — selecting `postfix/smtpd` in the Facility dropdown matches
`postfix/smtpd[12345]`, `postfix/smtpd[12346]`, and so on.

## Performance notes

With the two baseline indexes the common query shapes are O(log n) on
`ReceivedAt`:

- Last-24-hour, all facilities, limit 1000 — fast on any table size.
- Last-24-hour, one facility, limit 1000 — covered by the composite
  index, also fast.
- Multi-month window, all facilities, limit 10000 — slow on large
  tables; the index narrows the range but 10000 rows of `text` data is
  the bottleneck. Pull a tighter window.
- `SELECT DISTINCT SUBSTRING_INDEX(SysLogTag, '[', 1)` for the
  facility dropdown — fast on a 24-hour window, noticeably slower on
  weeks-deep windows because the index does not help with the
  expression.

If the table has grown into the tens of millions of rows because
retention was left at 180 days on a high-traffic gateway, dial
retention down and let the next nightly cleanup prune, or run the
cleanup job manually from [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks).

## Related pages

- [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) — live view of what Postfix is holding;
  pair with this page to trace a stuck message from queue to log.
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) — the Ofelia job that runs the
  retention cleanup.
- [LDAP & RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — context on why slapd
  appears in `mail.*` (it is configured to use the `mail` facility).
- [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) and
  [Admin Console Firewall](https://docs.deeztek.com/books/administrator-guide/page/console-firewall) — auth-side and
  HTTP-side log surfaces that do not land in `SystemEvents`.
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) — Authelia
  log location and what auth events look like on disk.

# System Notifications

# System Notifications

Admin path: **System > System Notifications** (`view_system_notifications.cfm`,
`inc/ofelia_generate_config.cfm`, `schedule/health_check_mailqueue.cfm`).

This page configures **how the gateway tells the operator that
something needs attention** when no admin is at the console. There are
two delivery channels (Pushover and e-mail) and a per-event toggle
list that decides which scheduled checks fire alerts on which channel.

The page itself is small — one settings card, one toggle list — but
its outputs land in three different places: a row in `system_settings`
(Pushover credentials), `active` flags on rows in `ofelia_jobs` (which
container-side scheduled jobs run), and a regenerated Ofelia config
file (`config.ini` on `hermes_ofelia`).

## What this page is — and isn't

| Is | Isn't |
|---|---|
| The configuration page for **outbound** operator alerts: Pushover push notifications + e-mail to `admin_email` | The **on-screen** dashboard alerts under the navbar (those come from `inc/system_alerts.cfm` and render at every page load — they are not configurable here) |
| A toggle list of which scheduled health checks send Pushover alerts when they fire | A free-form "send me this event" rule builder. The set of supported events is fixed and lives in the `pushover_notifications` table. |
| The owner of the Pushover API token + user/group key for the whole install | A per-user setting. There is one Pushover endpoint per gateway; use a **Pushover Group Key** if you need to fan out to multiple admins. |

> **Dashboard alerts vs. notifications.** The yellow / red callout
> banners that appear under the top navbar (license expiring, mail
> queue backed up, certificate near expiry, etc.) are rendered by
> `inc/system_alerts.cfm` and are not configurable. They fire whenever
> their underlying condition is true, every page load, no matter
> who is logged in. This page is for **emailed / pushed** alerts when
> nobody is looking at the console. Both systems can fire on the same
> underlying event (a mail queue spike will show as a callout AND
> trigger a Pushover push) but they are independent code paths.

## Where the values live

| Setting | Table.column | Default |
|---|---|---|
| Pushover master toggle | `system_settings.pushover_enabled` | `0` |
| Pushover API token (Application Token) | `system_settings.pushover_api_token` | empty |
| Pushover user / group key | `system_settings.pushover_user_key` | empty |
| Per-notification enable flag | `pushover_notifications.enabled` | `2` (disabled) — `1` = enabled |
| Per-notification Ofelia binding | `pushover_notifications.ofelia_job_name` | seeded |
| Ofelia job active flag | `ofelia_jobs.active` | per-job |
| Admin destination address | `system_settings.admin_email` | `someone@otherdomain.tld` |
| Notification `From:` envelope | `system_settings.postmaster` | `postmaster@domain.tld` |

The last two rows live on the [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings)
page, not here. This page **reads** them but does not write them — set
those first, then come back here.

`pushover_notifications` is the canonical registry of every alert that
*can* be sent. Each row pairs a display name + description (shown in
the toggle list) with an Ofelia job name that drives the actual check.
The current seed has one row:

| `name` | `display_name` | `ofelia_job_name` | `category` |
|---|---|---|---|
| `mailqueue_check` | Mail Queue Health Check | `[job-exec "hermes-health-check-mailqueue"]` | `health` |

New notification types are added by inserting a row in this table
(plus the matching row in `ofelia_jobs`) — no code change to the page
itself is needed.

## Pushover Settings card

Sets the per-install Pushover endpoint. Three fields:

| Field | Validation in `save_pushover` |
|---|---|
| Pushover Notifications (Enabled / Disabled) | Must be `0` or `1` |
| API Token (Application Token) | Required when enabled; must match `^[a-zA-Z0-9]{30}$` |
| User / Group Key | Required when enabled; must match `^[a-zA-Z0-9]{30}$` |

Get the values from [pushover.net](https://pushover.net): create an
**Application** to mint the API Token, and either use your own User
Key or create a **Group** to fan out to multiple admins.

After a successful save the form re-displays with a **Send Test
Notification** button that POSTs `action = test_pushover`. The test
sends a real Pushover message at priority `0` (default sound `pushover`)
and surfaces the HTTP status — anything non-200 reports the
`fileContent` as the error detail. Use this to confirm the token + key
pair is good before relying on the channel for real alerts.

## Save flow

```
POST action=save_pushover
   │
   ▼
 Validate pushover_enabled in {0,1}
 If enabled, validate token + key length + alphanumeric pattern
   │
   ▼
 UPDATE system_settings SET value=<x> WHERE parameter IN
   ('pushover_enabled','pushover_api_token','pushover_user_key')
   │
   ▼
 Sync ofelia_jobs.active per the rules below
   │
   ▼
 ofelia_generate_config.cfm  ──►  hermes_ofelia /config/config.ini
                                 (Ofelia re-reads on file change)
   │
   ▼
 cflocation back to view_system_notifications.cfm with session.m=1
```

The Ofelia sync rules are the moving part. The page wants two
conditions to BOTH be true before a notification job actually runs:

1. The **per-notification** toggle in the Available Notifications list
   is on (`pushover_notifications.enabled = 1`)
2. The **master** Pushover toggle is on (`system_settings.pushover_enabled = 1`)

| Master toggle | Per-notification toggle | `ofelia_jobs.active` becomes |
|---|---|---|
| `1` (on) | `1` (on) | `1` (job runs on schedule) |
| `1` (on) | `2` (off) | `2` (job dormant) |
| `0` (off) | any | `2` (all `type='pushover'` jobs dormant) |

So disabling the master Pushover toggle is a safe global kill switch
— every individual notification job stops scheduling. Re-enabling
restores only the per-notification rows that were previously on, not
all of them.

## Toggling a single notification

The Available Notifications card renders one row per
`pushover_notifications` entry, with a clickable toggle pill. Clicking
the pill POSTs `form_action = toggle_notification` with the
notification's row ID. The handler flips
`pushover_notifications.enabled` between `1` and `2`, applies the same
two-condition rule above to `ofelia_jobs.active`, regenerates the
Ofelia config, and redirects back with `session.m = 9`.

The card is **only rendered when the master Pushover toggle is on** —
if Pushover is off there is nothing to toggle per-event, so the list
is hidden.

## Ofelia is the scheduler

Hermes runs all of its recurring checks under [Ofelia](https://github.com/mcuadros/ofelia)
in the `hermes_ofelia` container. The `ofelia_jobs` table holds the
authoritative job definitions; `inc/ofelia_generate_config.cfm`
re-renders `config.ini` from the table and Ofelia hot-reloads. The
notification-side toggles on this page write `ofelia_jobs.active` for
rows where `type = 'pushover'`; other Ofelia jobs (DKIM cron,
certificate renewal, DMARC report processing, etc.) are managed by
their own pages or by [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks).

The seeded mailqueue job runs every 15 minutes:

| Field | Value |
|---|---|
| `job_name` | `[job-exec "hermes-health-check-mailqueue"]` |
| `schedule` | `@every 15m` |
| `command` | `/usr/bin/curl --silent http://localhost:8888/schedule/health_check_mailqueue.cfm` |
| `container` | `hermes_commandbox` |
| `type` | `pushover` |

The CFM target (`schedule/health_check_mailqueue.cfm`) is the real
worker — Ofelia just curls it. The CFM reads
`system_settings.pushover_*`, runs `health_check_mailqueue.sh` to
count the Postfix queue, and on `count > 20` sends both a Pushover
warning AND an e-mail to `admin_email`. The Pushover path is wrapped
in `<cftry>` so a Pushover outage falls through to e-mail — both
channels fire for the same event by design, so the admin gets the
alert even if one channel is broken.

## E-mail delivery path

Notification e-mails are sent via `<cfmail server="hermes_postfix_dkim"
port="10026">`. Port `10026` is Postfix's **post-Amavis re-injection**
listener, which means:

| Property | Behaviour |
|---|---|
| `From:` | `system_settings.postmaster` |
| `To:` | `system_settings.admin_email` |
| Content filtering | **Skipped.** 10026 is post-Amavis — these messages never go through SpamAssassin or ClamAV. |
| DKIM signing | Applied normally (OpenDKIM milter on the post-Amavis path) |
| Transport | Normal SMTP from `hermes_postfix_dkim` to the destination MX |

Skipping content filtering is by design — if Amavis itself is the
thing that's broken, the notification still has to reach the admin.
The trade-off is that a hostile actor with write access to the
gateway could in principle use this same path to inject mail; the
mitigation is that only the gateway's own CFML scheduled jobs target
this port (it is not exposed to the world).

## Adding a new notification type

The page is data-driven — adding a new alert requires no UI change.
Three artefacts need to land together in a schema-update script:

1. A new row in `pushover_notifications` (`name`, `display_name`,
   `description`, `ofelia_job_name`, `category = 'health' | 'security' | ...`)
2. A matching row in `ofelia_jobs` (`type = 'pushover'`,
   pointing at the worker URL)
3. The worker CFM under `config/hermes/var/www/html/schedule/` that
   does the actual check and `cfhttp`-POSTs Pushover + `cfmail`s the
   admin

The Available Notifications card will pick up the new row at the next
page load. The master/per-event toggle rules above apply automatically.

## Pro-vs-Community

System Notifications is a **Community-tier** page. The Pushover
integration, e-mail alerts, and toggle list all work on Community
installs. The Pro license check on the page header (the small comment
block in the include's CFML preamble) is part of the file-fingerprint
manifest — it doesn't gate functionality, only proves the file is
unmodified.

## Failure semantics

| What breaks | What happens |
|---|---|
| Pushover credentials wrong | Save succeeds (no live validation), but Test Notification returns non-200; `session.m = 8` surfaces the API response in the error banner |
| API Token / User Key format wrong (not 30 alphanumeric chars) | Save rejected (`session.m = 4` / `5`); no DB write |
| Master Pushover toggle off | All `type='pushover'` Ofelia jobs flipped to `active = 2`; e-mail path still runs from `health_check_mailqueue.cfm` |
| Ofelia config regen errors | The toggle save still commits to the DB; the `cftry` wrapper around `ofelia_generate_config.cfm` swallows the error. Re-save to retry. |
| `admin_email` empty | `cfmail` will accept an empty `to=` and produce an undeliverable message in the queue; set `admin_email` on [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings) first |
| `pushover.net` unreachable | `health_check_mailqueue.cfm` falls through to e-mail; admin still gets the alert |
| `hermes_postfix_dkim:10026` listener down | E-mail path fails too. The on-screen dashboard alerts (from `inc/system_alerts.cfm`) are the last line of defence — they need no transport. |

## Files and tables touched

| Path / table | Role |
|---|---|
| `system_settings` (rows `pushover_enabled`, `pushover_api_token`, `pushover_user_key`, `admin_email`, `postmaster`) | Channel config + addresses |
| `pushover_notifications` | Registry of every alert type the page can toggle |
| `ofelia_jobs` (`type = 'pushover'` rows) | Per-notification scheduler entries |
| `config/hermes/var/www/html/admin/2/view_system_notifications.cfm` | Page |
| `config/hermes/var/www/html/admin/2/inc/ofelia_generate_config.cfm` | Re-renders `hermes_ofelia /config/config.ini` from `ofelia_jobs` |
| `config/hermes/var/www/html/schedule/health_check_mailqueue.cfm` | The mail queue worker; reads Pushover creds, sends push + e-mail |
| `https://api.pushover.net/1/messages.json` | Outbound HTTPS endpoint for every Pushover send (Test + live alerts) |

## Related

- [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings) — sets `admin_email` and `postmaster` (the addresses this page delivers to / from)
- [System Status](https://docs.deeztek.com/books/administrator-guide/page/system-status) — the dashboard-callout side of the alert system (`inc/system_alerts.cfm`)
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) — admin view of the full `ofelia_jobs` table; the same Ofelia container drives every recurring task on the gateway
- [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) — the page the mail queue alert is asking you to look at when it fires

# System Settings

# System Settings

Admin path: **System > System Settings** (`view_system_settings.cfm`,
`inc/get_system_settings.cfm`, `inc/edit_system_settings.cfm`,
`inc/add_serial_number.cfm`, `inc/update_system_email_addresses.cfm`,
`inc/update_system_timezone.cfm`, `inc/update_system_update_check.cfm`,
`inc/update_telemetry.cfm`, `inc/invalidate_user_sessions.cfm`).

This is the **catch-all configuration page** for the gateway's global
identity. Three cards live here:

1. **General Settings** — postmaster + admin e-mail addresses, server
   timezone, daily update check, telemetry, and the Pro Edition serial
   number.
2. **Bot Protection (CAPTCHA)** — chooses the CAPTCHA provider used on
   public-facing forms (Forgot Password, etc.) and stores the per-provider
   keys.
3. **Session Management** — the "Force Logout All Users" red button.

Pairs with [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) (web-facing host /
TLS cert) and [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) (mail-side host identity)
— those define **where** Hermes lives; this page defines **who** runs
it and which administrative addresses receive its automated traffic.
The [System Notifications](https://docs.deeztek.com/books/administrator-guide/page/system-notifications) page reads
`admin_email` from this page when it sends Pushover or e-mail alerts.

## Configuration storage

Every setting on this page lives in the `system_settings` table
(`parameter` UNIQUE key, `value` VARCHAR(1024)). There are no
`parameters2` rows in scope — that table is reserved for module-scoped
config (`console`, `smtp`, etc.). The `parameter`/`value` shape is
deliberately flat key/value; the seed in `config/database/hermes_install.sql`
sets the defaults at install time and every edit on this page is a
straight `UPDATE … WHERE parameter = '<key>'`.

| Card | `system_settings.parameter` | Default | Notes |
|---|---|---|---|
| General | `postmaster` | `postmaster@domain.tld` | Must be a valid e-mail at a **domain that already exists in `domains`** |
| General | `admin_email` | `someone@otherdomain.tld` | Valid e-mail; no domain check |
| General | `timezone` | `America/New_York` | Validated against the `timezones` table |
| General | `serial` | empty | Pro Edition serial; set via the **Add Serial Number** modal |
| General | `users` | `9999` | Set to `9999` automatically when a serial activates (legacy seat-cap field, no longer enforced) |
| General | `daily_update_check` | `2` (Disable) | `1` = enable, `2` = disable; controls the auto-update poll |
| General | `telemetry` | `1` (Enable) | `1` = enable, `2` = disable; anonymised usage data |
| General | `accepted` | `1` | Legacy AGPL acceptance flag; not surfaced in the UI |
| Release stamp | `version_no` | `Docker` | Sentinel that marks this as a Docker install |
| Release stamp | `build_no` | `v260119` | Current release tag |
| CAPTCHA | `captcha_provider` | `builtin` | One of `builtin`, `recaptcha`, `hcaptcha`, `turnstile` |
| CAPTCHA | `recaptcha_site_key` / `recaptcha_secret_key` | empty | reCAPTCHA v2 |
| CAPTCHA | `hcaptcha_site_key` / `hcaptcha_secret_key` | empty | hCaptcha |
| CAPTCHA | `turnstile_site_key` / `turnstile_secret_key` | empty | Cloudflare Turnstile |

The release-stamp rows (`version_no = 'Docker'`, `build_no = v<YYMMDD>`)
are the canonical signal that this install is a Docker install rather
than a legacy non-Docker one. They are surfaced read-only in the
sidebar footer and in INSTALL_SUMMARY output; the schema-update
orchestrator and several upgrade-path code paths gate on them.

## General Settings — fields

### Postmaster E-mail Address (required)

Where bounce notifications, postmaster-class mail, and several internal
alerts originate from. `edit_system_settings.cfm` enforces three rules
in sequence:

1. Must not be empty (`session.m = 2`)
2. Must validate as a real e-mail string (`session.m = 3`)
3. The **domain part must already exist in the `domains` table**
   (`session.m = 4`)

The third rule is the one that surprises people. A bare
`postmaster@example.com` will not save unless `example.com` is already
a recognised mailbox or relay domain on this gateway. If you are
setting this up on a fresh install, add the domain first
(Mailboxes > Domains or Email Relay > Relay Domains) and come back.

The postmaster address is also the `From:` on every notification e-mail
the gateway sends (see [System Notifications § Email path](https://docs.deeztek.com/books/administrator-guide/page/system-notifications#email-delivery-path)),
so it must be a deliverable address from the gateway's perspective —
which is exactly what the domain-existence check guarantees.

### Admin E-mail Address (required)

The destination address for every automated alert and notification
e-mail. Validates as a normal e-mail string (`session.m = 5` empty,
`session.m = 6` malformed) but has no domain-existence check — it is
deliberately allowed to be an external address (your monitoring inbox,
a shared mailbox at a different provider) so the gateway can still
reach you when its own mail flow is broken.

The [System Notifications](https://docs.deeztek.com/books/administrator-guide/page/system-notifications) page reads this
value at every send.

### TimeZone (required)

Free-text autocomplete backed by `inc/gettimezones.cfm` against the
`timezones` table. The submitted value is checked back against the
table before save (`session.m = 7` empty, `session.m = 8` unknown).
Drives every timestamp that Lucee renders in the UI plus the schedule
times shown on Scheduled Tasks.

> **The Lucee server's own timezone is set elsewhere.** Changing this
> field rewrites the application's display timezone; it does **not**
> change the container's `TZ` env var or the OS clock. If the two
> diverge you will see UI timestamps in one zone and log files in
> another.

### Serial Number (read-only here)

Display-only on the General card. To set or change a serial, use the
**Add Serial Number** button at the top of the page — that opens a
modal that POSTs to `inc/add_serial_number.cfm`.

The activation flow (only triggered when a serial is entered, not on
every page load):

```
Modal POST  serial_number + tos
      │
      ▼
  add_serial_number.cfm
      │  validate non-empty / alphanumeric-only / TOS accepted
      │  generate per-request token (customtrans3)
      │  read host UUID via dmi_decode.cfm
      │  RSA-encrypt "<UUID>@<serial>" with /opt/hermes/ssl/public.pem
      ▼
  POST https://activate.hermesseg.io  (TCP/443, no SSL interception)
      │
      ▼
  Server returns "<hash>@<expires>" on success
                  or  INVALID / ALREADY_ACTIVATED / EXPIRED / REVOKED / ERROR
      │
      ▼
  On success: UPDATE system_settings SET value=<serial> WHERE parameter='serial'
              updateRetentionPolicy("VALID", expires, serial, hash)  (cache the result)
              session.license = "VALID"
```

Every login after this point re-validates against
`https://validate.hermesseg.io` and falls back to the cached
`<hash>@<expires>` if the validation endpoint is unreachable (the
"offline mode" path). The page itself never re-runs validation — that
is the job of `inc/setsession.cfm` at login.

| `session.license` value visible after this page | Meaning |
|---|---|
| `VALID` + `session.edition = "Pro"` | Activation succeeded; Pro features available |
| `EXPIRED` | Cached license past expiry; renew at the vendor portal and re-login |
| `REVOKED` | Vendor revoked the serial; contact support |
| `INVALID` | Serial not recognised; double-check the value |
| `TAMPERED` | Pro template files don't match the signed fingerprint; reinstall the release |
| `PENDING_VALIDATION` | Cached license exists but no signed fingerprint baseline; reach the internet and re-login |
| `N/A` | No serial configured — Community Edition |

The two activation-server error paths (`session.m = 12` / `session.m = 13`)
both render the same root-cause hint: Hermes must reach
`activate.hermesseg.io` over HTTPS **without SSL interception**.
Inline-decrypt proxies will break activation because they re-sign the
RSA-encrypted payload.

> **By design.** Deleting the serial value from `system_settings`
> instantly demotes the install to Community Edition. The next login
> sees `session.license = N/A` and stops attempting remote validation.

### Daily Update Check / Telemetry

Two boolean (1 = enable, 2 = disable) selects. Daily Update Check is
the toggle for the auto-update poll that watches for new releases.
Telemetry is the anonymised usage-data feed; the in-card warning
callout links to the public privacy doc. Defaults are: Telemetry =
enabled, Daily Update Check = disabled.

### Save flow

**Save Settings** posts `action = edit`, which runs
`edit_system_settings.cfm` as a strict 5-step sequence
(postmaster → admin_email → timezone → update_check → telemetry).
Each validation failure short-circuits with `cflocation` back to
`view_system_settings.cfm` and `session.m` set to the matching alert
code — no partial state lands. On the final step, four small update
includes write to `system_settings` one parameter at a time
(`update_system_email_addresses.cfm`, `update_system_timezone.cfm`,
`update_system_update_check.cfm`, `update_telemetry.cfm`).

## Bot Protection (CAPTCHA)

CAPTCHA gates the public-facing forms that an unauthenticated visitor
can hit — primarily the Forgot Password flow on `/user-auth/` and
`/admin-auth/`. The provider is chosen here; the form templates check
the same `system_settings` keys at render and validation time. Four
providers are supported:

| Provider | What it needs |
|---|---|
| **Built-in (math)** | No keys. Renders a "what is 7 + 3?" style challenge. Default; works offline. |
| **Google reCAPTCHA v2** | Site key + secret key. Pick the *"I'm not a robot" Checkbox* flavour at the reCAPTCHA admin. |
| **hCaptcha** | Site key + secret key. Privacy-focused reCAPTCHA alternative. |
| **Cloudflare Turnstile** | Site key + secret key. Usually invisible — no user interaction in the happy path. |

`save_captcha` POSTs validate that the provider is one of the four
allowed values and that the matching pair of keys is non-empty when a
non-builtin provider is selected. All seven values are written on
every save regardless of which provider is active — this lets the
admin switch providers back and forth without re-entering keys.

> **Failure mode.** A misconfigured external provider (bad keys,
> domain mismatch) breaks Forgot Password silently for the end user
> — the form renders, the CAPTCHA widget loads, but the server-side
> `siteverify` call fails and the request is rejected. Test the
> provider end-to-end on `/user-auth/forgot_password.cfm` after every
> change.

## Session Management — Force Logout All Users

The red button at the bottom of the page flushes the **entire Authelia
session store** in one call. Every user (admin, mailbox, relay
recipient — and the operator clicking the button) is redirected to the
login page on their next request. There is no per-user logout on this
page; that happens automatically when a user's password is changed,
their account is deactivated, or their account is deleted, because
Authelia's session cookie is encrypted and only Authelia can
invalidate one. The bulk-flush button is the only way to forcibly log
people out from the admin UI.

Use this when:

- A shared admin credential has been rotated and you want every
  inherited session gone
- You suspect a compromised session token
- You have just changed [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) and
  want every old hostname-scoped cookie cleared at once

The action runs `inc/invalidate_user_sessions.cfm` with
`targetSessionUser = "*"` and surfaces `session.m = 36` on return.

## Edition badge — Pro vs Community

Although this page **stores** the serial number, the Pro / Community
edition badge that appears in the sidebar header and in
[System Status](https://docs.deeztek.com/books/administrator-guide/page/system-status) is rendered from `session.edition` /
`session.license` — both of which are set during login by
`inc/setsession.cfm`. Changing the serial here updates the row in
`system_settings`; the badge updates on the **next** login. Use
**Force Logout All Users** above if you need the change to be visible
to other admins immediately.

## Files and tables touched

| Path / table | Role |
|---|---|
| `system_settings` | Every setting on this page (key/value rows) |
| `domains` | Read at postmaster save to validate the domain part |
| `timezones` | Read at timezone autocomplete and save |
| `config/hermes/var/www/html/admin/2/view_system_settings.cfm` | Page |
| `config/hermes/var/www/html/admin/2/inc/edit_system_settings.cfm` | General-card save handler |
| `config/hermes/var/www/html/admin/2/inc/add_serial_number.cfm` | Serial activation against `activate.hermesseg.io` |
| `config/hermes/var/www/html/admin/2/inc/invalidate_user_sessions.cfm` | Force-logout call into Authelia |
| `config/hermes/var/www/html/admin/2/inc/setsession.cfm` | Reads serial + edition at login; this page's read-only Pro Edition state comes from here |
| `https://activate.hermesseg.io` | One-time serial activation endpoint |
| `https://validate.hermesseg.io` | Per-login Pro Edition re-validation endpoint |

## Related

- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — web console host + TLS cert
- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — mail-side host identity (Postfix `myhostname` / `myorigin`)
- [System Notifications](https://docs.deeztek.com/books/administrator-guide/page/system-notifications) — consumes `admin_email` + `postmaster` from this page; also the home of Pushover settings
- [System Status](https://docs.deeztek.com/books/administrator-guide/page/system-status) — surfaces the same Pro / Community badge plus the dashboard-alert stream
- [System Update](https://docs.deeztek.com/books/administrator-guide/page/system-update) — when Daily Update Check is enabled, it is this page that drives the poll
- [Password Resets](https://docs.deeztek.com/books/administrator-guide/page/password-resets) — the public form that CAPTCHA actually protects

# System Status

# System Status

Admin path: **System > System Status** (`index.cfm` —
the post-login landing page; the sidebar entry points here, not to a
separate `view_system_status.cfm`). Supporting includes:
`inc/system_alerts.cfm`, `inc/check_system_update.cfm`,
`inc/get_system_version_build.cfm`, `inc/get_system_uptime.cfm`,
`inc/get_system_reboot_required.cfm`, `inc/get_system_resources.cfm`,
`inc/get_system_cpu_usage.cfm`, `inc/get_system_memory_usage.cfm`,
`inc/get_system_{root,data,vmail,nextcloud}_filesystem_usage.cfm`,
`api/get_system_resources.cfm`, `api/get_message_stats.cfm`.

System Status is the operator's at-a-glance picture of the running
gateway. It is the **default page after login** — every Authelia
post-login redirect lands here — and the union of every "is anything
broken right now" signal Hermes computes: license state, update
availability, host OS reboot pending, fresh-install onboarding nudges,
container resource usage, and live mail-processing volume.

The page is **graceful-degradation by design**: every widget catches
its own errors, every external call has a fallback, and a single
failed query (a missing setting row, an unreachable container, a
malformed log file) does not blank the dashboard. If you log in to
Hermes and the page renders at all, the page is doing its job.

## Page layout

```
+--------------------------------------------------------------+
| Top navbar    [license / fresh-install / update badges]      |
+--------------------------------------------------------------+
| Alert callouts (priority <= 5)                               |  rendered by top_navbar.cfm
|   * Templates Modified  (priority 1)                         |  from request.systemAlerts
|   * License Revoked     (priority 2)                         |  (populated in
|   * Placeholder hostname (priority 2)                        |   inc/system_alerts.cfm)
|   * Invalid / Pending / Grace-period expired (priority 3)    |
|   * Self-signed cert    (priority 3)                         |
|   * License Expired     (priority 4)                         |
|   * Offline Mode        (priority 5)                         |
+--------------------------------------------------------------+
| Welcome <user>   Last login: <timestamp>                     |
+--------------------------------------------------------------+
| System Info card                                             |
|   Version | Build | Edition | Uptime | Console IP/FQDN      |
|   | License Status | OS Updates | Hermes Update              |
+--------------------------------------------------------------+
| Messages Processed card                                      |
|   Donut chart + counts (Clean/Spam/Virus/Banned/             |
|   Bad Header/Other) over 15m / 1h / 8h / 12h / 24h           |
+--------------------------------------------------------------+
| System Resources card                                        |
|   CPU | Memory | Root FS | Data FS | Vmail FS | Nextcloud FS |
|   (seven progress rings, auto-refresh every 10s)             |
+--------------------------------------------------------------+
```

The two cards that show live data (Messages Processed and System
Resources) poll their own JSON endpoints in the background; the rest
of the page is rendered server-side once per load.

## Self-healing on first load

`index.cfm` is the **bootstrap convergence point** for several
secrets that the rest of the app depends on. If any are missing on
first load (fresh install, after a credential rotation, after a key
file deletion), they are generated in-place before the dashboard
renders:

| Missing artifact | Auto-generated by |
|---|---|
| `/opt/hermes/keys/hermes.key` (AES-256 application key) | `inc/generate_hermes_key.cfm` |
| `encryption_settings.user.serverSecret` (Ciphermail) | `inc/generate_ciphermail_server_secret.cfm` |
| `encryption_settings.user.clientSecret` (Ciphermail) | `inc/generate_ciphermail_client_secret.cfm` |
| `encryption_settings.user.systemMailSecret` (Ciphermail) | `inc/generate_ciphermail_mail_secret.cfm` |
| `/opt/hermes/scripts/container_ips.txt` (Fail2ban) | `inc/generate_container_ips.cfm` |

Each generator is idempotent — it checks "is the file/row empty?"
before writing, so subsequent loads are no-ops. This is why the very
first dashboard render on a fresh install is slightly slower than
subsequent loads.

## System Info card

The columns and what they mean:

| Column | Source | Notes |
|---|---|---|
| **Version** | `system_settings.version_no` | Always `Docker` in the Docker era. Hyphenated legacy values (e.g. `2024-08`) belong to bare-metal installs that have not yet been migrated. |
| **Build** | `system_settings.build_no` | Current release tag (`vYYMMDD`). This is the single value the update orchestrator compares against to decide what to apply. See [System Update](https://docs.deeztek.com/books/administrator-guide/page/system-update). |
| **Edition** | `session.edition` | `Community` or `Pro`. Community shows an `ENTER SERIAL` link to [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings); Pro shows the edition with state suffixes (`Pro (Templates Modified)`, `Pro (Validation Required)`) when the license is in a non-VALID state. |
| **Uptime** | `/opt/hermes/scripts/get_uptime.sh` via `cfexecute` | Host uptime in days, not container uptime. |
| **Console IP or FQDN** | `parameters2.console.host` | The value bound on [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings). |
| **License Status** | `session.license` + `session.licenseexpires` | One of `VALID`, `EXPIRED`, `REVOKED`, `INVALID`, `TAMPERED`, `PENDING_VALIDATION`, `VIOLATION`, `N/A`. Community shows `N/A`. The status text resolves through the same license-evaluation logic documented in `inc/setsession.cfm`. |
| **OS Updates** | `/var/run/reboot-required` (file exists?) | `REBOOT REQUIRED` when the kernel/glibc-class update on the host needs a reboot. Hermes does not reboot the host — the admin does, via SSH. |
| **Hermes Update** | `inc/check_system_update.cfm` (reads `/opt/hermes/updates/check_system_update.txt`) | `UPDATE BUILD vYYMMDD FOUND` (clickable, opens GitHub release notes modal), `LATEST VERSION`, `UPDATE CHECK PENDING`, or `UPDATE CHECK UNAVAILABLE`. The cache file is written by the daily `schedule/check_for_update.cfm` job; see [System Update § Daily update check](https://docs.deeztek.com/books/administrator-guide/page/system-update#daily-update-check). |

The Hermes Update cell never makes a network call. It reads the cache
file written by the Ofelia-scheduled CFML job, so the page renders
fast and works offline.

> **By design.** The dashboard never calls the GitHub Releases API at
> page-render time. All network IO for "is there an update" happens
> in the once-a-day Ofelia job. If the cache file is missing (first
> load after install, before the first scheduled run) you see
> `UPDATE CHECK PENDING`, never a hang.

## Release-notes modal

Clicking `UPDATE BUILD vYYMMDD FOUND` opens a modal that fetches the
release body from the GitHub Releases API:

```
GET https://api.github.com/repos/deeztek/Hermes-Secure-Email-Gateway/releases/tags/<vYYMMDD>
```

The response's `body` field (Markdown) is converted client-side to
HTML and rendered in the modal. If the fetch fails (rate limit,
offline, release deleted) the modal degrades to a "View Release on
GitHub" button. The GitHub release page is canonical; the modal is a
convenience.

The tag passed to the URL is the build number **as-is**. Earlier
revisions of this code prepended `build-` to match the legacy
update-server file-name convention; that prefix was removed during
the [#218](https://github.com/deeztek/Hermes-Secure-Email-Gateway/issues/218)
release-engineering pivot because GitHub release tags do not carry it.

## Alert callouts

`inc/system_alerts.cfm` builds a priority-ordered array of alerts
each page load. The array is sorted ascending by priority (lower
number = more urgent), then split:

| Priority | Surface |
|---|---|
| 1–5 | Full-width callout banner under the navbar, rendered by `top_navbar.cfm` |
| 6+  | Compact badge next to the user/edition pill in the navbar |

Every Hermes page that includes `top_navbar.cfm` participates — the
callouts are not exclusive to System Status — but System Status is
where an admin is most likely to be looking when they appear.

### License-state alerts

| Alert | Priority | Trigger |
|---|---|---|
| Templates Modified | 1 | `session.license = TAMPERED` |
| License Revoked | 2 | `session.license = REVOKED` |
| Invalid License | 3 | `session.license = INVALID` |
| Validation Required | 3 | `session.license = PENDING_VALIDATION` (no offline baseline yet) |
| Grace Period Expired | 3 | `session.license = GRACE_PERIOD_EXPIRED` |
| License Expired | 4 | `session.license = EXPIRED` |
| Offline Mode | 5 | `VALID` + `validationMode = cached`; includes remaining grace-period day count |
| Expires in <N> days | 10 | `VALID` + `licensevaliddays <= 30` (badge only, never a callout) |

### Fresh-install onboarding nudges

Two universal nudges fire when the gateway is still using seed
defaults. Both apply to every install regardless of topology
(relay-only, mail-server-only, hybrid) and they live here precisely
**because** they are topology-agnostic.

| Nudge | Priority | Trigger | Fix link |
|---|---|---|---|
| Placeholder hostname | 2 | `parameters.myhostname = 'hermes.domain.tld'` (Postfix seed) OR `parameters2.console.host = 'smtp.domain.tld'` (console seed) | [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) |
| Self-signed cert | 3 | Every row in `system_certificates` is flagged `system = 1` (only bootstrap cert exists) | [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) |

Earlier iterations of this list included three more topology-specific
nudges (no relay domains, no relay networks, no recipients-or-mailboxes).
They were removed because they fired noisily on installs that legitimately
don't have those things — a relay-only install has zero mailboxes and
that is the correct configuration. Topology-specific onboarding guidance
lives in [`docs/install/get-started-docker.md`](https://docs.deeztek.com/books/installation-reference/page/get-started-docker)
instead, where it is read deliberately rather than nagged about every
page load.

### Other alerts (placeholders)

`system_alerts.cfm` includes guarded blocks for **Reboot Required**
(when `session.rebootRequired = true`) and **Cert Expiring** (when
`session.certExpiringSoon = true`). Neither flag is currently
populated by any code path — they are reserved for future widgets
that compute the values and stash them in the session.

## Messages Processed card

Polls `api/get_message_stats.cfm` on initial load and every 60s. The
period selector reloads with the new window value but does not
otherwise change the polling cadence.

| Bucket | Color |
|---|---|
| Clean | Green (`#28a745`) |
| Spam | Yellow (`#ffc107`) |
| Virus | Red (`#dc3545`) |
| Banned | Gray (`#6c757d`) |
| Bad Header | Dark (`#343a40`) |
| Other | Cyan (`#17a2b8`) |

The endpoint reads from the `msgs` table (Amavis-fed; covered in more
detail under [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs)) filtered to the selected
window. A 10,000-row hard cap is applied to keep page-load fast on
busy installs; when the cap is hit, the total is suffixed with `+`
and a small "Showing most recent 10,000 messages" note appears under
the breakdown.

## System Resources card

Seven progress rings, auto-refreshing every 10s via
`api/get_system_resources.cfm`:

| Ring | Source |
|---|---|
| CPU Utilization % | `/opt/hermes/scripts/get_cpu_usage.sh` |
| Memory Utilization % | `/opt/hermes/scripts/get_memory_usage.sh` |
| Root FileSystem % | `df` on `/` (host root) |
| Data FileSystem % | `df` on the Data tier mount (see [Storage Topology](https://docs.deeztek.com/books/installation-reference/page/storage-topology-5-tiers)) |
| Archive FileSystem % | `df` on the Archive tier mount (#260; Amavis quarantine) |
| Vmail FileSystem % | `df` on the Vmail tier mount |
| Nextcloud FileSystem % | `df` on the Nextcloud tier mount |

Each ring color-codes by threshold (`get_system_*_usage.cfm` returns
a hex color alongside the value). The rings degrade independently —
a missing tier mount renders that ring at 0 rather than failing the
whole card.

Tiers that share a host path (a smaller install where Archive, Vmail,
and Nextcloud are pinned to the same disk as Data) will show the same
percentage on multiple rings. That is the correct behavior; the
underlying `df` reading is the same.

## What is NOT on this page

System Status is intentionally a "snapshot" page, not an
investigation tool. It surfaces alerts and current resource state. It
does not surface:

| Want to see | Go here instead |
|---|---|
| Mail queue contents / deferred messages | [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) |
| Per-message processing history | [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) |
| Detailed cert / SAN status | [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates), [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) |
| Container health (`docker ps` output, restart counts) | Host shell — Hermes does not surface raw Docker state in the web UI |
| Scheduled-job last-run / next-run | [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) |
| Fail2ban bans in effect | [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) |
| Past update history | The git log on the host (`git log --oneline -- updates/`) |

## Failure semantics

| What breaks | What happens |
|---|---|
| `/opt/hermes/updates/check_system_update.txt` does not exist | `hermesupdate = "UPDATE CHECK PENDING"`; cell renders cleanly |
| Ofelia job has been failing for days (cache stale or shows old build) | Page still renders; the **Hermes Update** cell reflects whatever the last successful run wrote |
| GitHub API rate-limited or unreachable when an admin clicks the release-notes link | Modal falls back to a "View Release on GitHub" button |
| `df` on a tier mount fails | That ring renders at 0 with default color; other rings render normally |
| `get_uptime.sh` exits non-zero | Page short-circuits to the error template — uptime is treated as critical because its absence usually means a broken commandbox |
| `system_settings.build_no` / `version_no` row missing | Empty value in the matching cell; license cells will display `N/A` |
| `inc/generate_*` first-load generator fails | Logged; affected feature degrades downstream (Ciphermail mail crypto disabled, etc.) — the dashboard itself still renders |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/index.cfm` | `hermes_commandbox` | Page |
| `config/hermes/var/www/html/admin/2/inc/system_alerts.cfm` | `hermes_commandbox` | Alert array builder (license + nudges + future widgets) |
| `config/hermes/var/www/html/admin/2/inc/check_system_update.cfm` | `hermes_commandbox` | Cache-file reader (Docker path) |
| `config/hermes/var/www/html/admin/2/inc/get_system_*.cfm` | `hermes_commandbox` | Per-widget data fetchers |
| `config/hermes/var/www/html/admin/2/api/get_system_resources.cfm` | `hermes_commandbox` | JSON endpoint for progress-ring auto-refresh |
| `config/hermes/var/www/html/admin/2/api/get_message_stats.cfm` | `hermes_commandbox` | JSON endpoint for message-stats card |
| `/opt/hermes/scripts/get_uptime.sh`, `get_cpu_usage.sh`, `get_memory_usage.sh` | `hermes_commandbox` | Shell helpers invoked via `cfexecute` |
| `/opt/hermes/updates/check_system_update.txt` | `hermes_commandbox` | Cache file written by `schedule/check_for_update.cfm`; read here |
| `/var/run/reboot-required` | host filesystem (mounted into `hermes_commandbox`) | Ubuntu's standard "kernel upgrade pending" sentinel |
| `/opt/hermes/keys/hermes.key` | `hermes_commandbox` | Created on first load if missing |
| `encryption_settings` table | `hermes_db_server` (`hermes` DB) | Ciphermail secrets populated on first load if empty |

## Related

- [System Update](https://docs.deeztek.com/books/administrator-guide/page/system-update) — the page System Status's
  "Hermes Update" cell links to; covers the daily update check and
  the orchestrator that consumes the cache file
- [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings) — where the Pro serial number
  is entered to lift Community to Pro
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — the page the
  "Self-signed cert" nudge links to
- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — the page the "Placeholder
  hostname" nudge links to
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — drill-down for the message-volume
  numbers shown in the Messages Processed card
- [Storage Topology](https://docs.deeztek.com/books/installation-reference/page/storage-topology-5-tiers) — explains
  the five tiers the resource-usage rings reflect
- [Release and Update Methodology](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology)
  — methodology behind the version stamp this page surfaces

# System Update

# System Update

Admin path: **System > System Update**
(`view_system_updates.cfm`). Update infrastructure:
`config/hermes/var/www/html/schedule/check_for_update.cfm` (daily
GitHub Releases poll), `config/hermes/var/www/html/admin/2/inc/check_system_update.cfm`
(dashboard cache-file reader), `scripts/system_update_docker.sh`
(the update orchestrator), `config/ofelia/config.ini` (the cron
schedule that triggers the daily check).

This page tells an admin **whether a new Hermes release is available
and how to apply it**. It is intentionally thin: every detail of how
upgrades actually work — the artifact taxonomy, the orchestrator's
five phases, the idempotency rules, the release-cut procedure — lives
in [Release and Update Methodology](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology),
which is the canonical reference. This page documents the **admin
surface** that sits on top of that methodology.

> **Update is currently CLI-driven.** The page itself displays a
> notice that points at the docs and the [release-and-update
> methodology](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology);
> the actual upgrade is run on the Docker host via SSH using
> [`scripts/system_update_docker.sh`](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology#the-update-orchestrator-scriptssystem_update_dockersh).
> A future revision will move the launch button into the page itself;
> until it does, the CLI is the supported path.

## How an admin knows there is an update

Three independent surfaces converge on the same answer:

```
                           +------------------------------+
                           | GitHub Releases API          |
                           | repos/deeztek/                |
                           |   Hermes-Secure-Email-Gateway |
                           |   /releases/latest            |
                           +------------------------------+
                                        ^
                                        | daily 04:30 UTC
                                        |
              +-------------------------+--------------------------+
              |  schedule/check_for_update.cfm                      |
              |    - polls /releases/latest                         |
              |    - compares tag_name to system_settings.build_no  |
              |    - writes /opt/hermes/updates/check_system_update.txt
              |    - emails admin_email when UPDATEFOUND            |
              +-------------------------+--------------------------+
                                        |
            +---------------------------+---------------------------+
            |                           |                           |
            v                           v                           v
    +----------------+         +-----------------+         +----------------+
    | Dashboard cell |         | System Update   |         | Email to       |
    | (Hermes Update)|         | page            |         | admin_email    |
    | reads cache    |         | (today: docs    |         | (one-shot per  |
    | every load     |         |  notice; v2:    |         |  release-found |
    |                |         |  Run Update btn)|         |  detection)    |
    +----------------+         +-----------------+         +----------------+
```

All three are downstream of one cached value — the
`/opt/hermes/updates/check_system_update.txt` file. The dashboard
does not call GitHub on page load; the email is not sent on page
load; only the once-a-day Ofelia job actually hits the API.

## Daily update check

`config/ofelia/config.ini` schedules a single `job-exec` against the
`hermes_commandbox` container:

```
[job-exec "hermes-update-check"]
schedule =  0 30 04 * * *
container = hermes_commandbox
command = /opt/hermes/schedule/update_check.sh
```

The shell wrapper resolves to a `curl --silent
http://localhost:8888/schedule/check_for_update.cfm` against the
internal Lucee port — no auth dance, no X-Token header, same
convention as `hermes-message-cleanup`, `hermes-quarantine-notify`,
and every other Hermes scheduled job. The CFML target does the
actual work:

1. Read current `build_no` from `system_settings`.
2. `GET https://api.github.com/repos/deeztek/Hermes-Secure-Email-Gateway/releases/latest`
   with a 30s timeout.
3. On HTTP 200, parse `tag_name` and compare to the local build via
   simple string comparison (`vYYMMDD` sorts correctly as a string
   because the format is fixed-width calendar versioning — see
   [Release and Update Methodology § Calendar versioning](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology#calendar-versioning)).
4. Write `/opt/hermes/updates/check_system_update.txt` regardless of
   outcome — the dashboard reader needs **something** to display.
5. On `UPDATEFOUND`, send one notification email to `admin_email`.

### Cache file format

The file is a single `@`-delimited line. The format is preserved from
the pre-#218 legacy update server (`updates.deeztek.com`) for
backward-compat with the dashboard reader; for Docker installs,
several fields are unused.

| Position | Field | Docker meaning |
|---|---|---|
| 1 | status | `SUCCESS` (update available), `NOUPDATE`, or `UPDATE CHECK UNAVAILABLE` |
| 2 | build | The new tag (e.g. `v260601`) on `SUCCESS`, current tag on `NOUPDATE` |
| 3 | released | `yyyy-mm-dd` from `published_at` |
| 4 | filename | _empty_ (was tarball name on legacy server) |
| 5 | release_notes_url | GitHub `html_url` for the release |
| 6 | release_notes_file | _empty_ (was per-release HTML file on legacy server) |
| 7 | mysqlroot | _empty_ (was installer credential on legacy server) |
| 8 | dev | `daily_update_check` value from `system_settings` |

### Email notification

The notification is **once per release** — re-runs of the check
against the same latest tag do not re-send (the job re-detects
`UPDATEFOUND` every day, but the email path is gated on the cached
comparison; if the dashboard cell already reads `UPDATEFOUND`, the
admin is already informed). The email is sent through
`hermes_postfix_dkim` on port 10026 (the post-content-filter
re-injection port that auto-DKIM-signs), so the message is signed
under the gateway's own DKIM key like any other system mail.

The message includes a GitHub link and, when `console.host` is set,
a hint to open the admin console where the dashboard prompt is
waiting.

### Toggling the daily check

The `daily_update_check` row in `system_settings` is wired through
to the cache file (field 8 above), but the Ofelia schedule itself
is the actual on/off switch — to stop the daily check, remove or
comment the `[job-exec "hermes-update-check"]` block in
`config/ofelia/config.ini` and restart `hermes_ofelia`. The
`system_settings` toggle is a legacy UI surface from the
pre-Ofelia era; the modern path is the Ofelia config.

## Status values shown on the dashboard

The dashboard's **Hermes Update** cell (System Info card, last
column) is the operator-visible side of this whole pipeline. See
also [System Status § System Info card](https://docs.deeztek.com/books/administrator-guide/page/system-status#system-info-card).

| Cache status | Cell text | What it means |
|---|---|---|
| `SUCCESS` | `UPDATE BUILD vYYMMDD FOUND` (link → release-notes modal) | New release available. Click for GitHub release notes; act via the orchestrator below. |
| `NOUPDATE` | `LATEST VERSION` | Local `build_no` matches `tag_name` on GitHub. |
| `UPDATE CHECK UNAVAILABLE` | `UPDATE CHECK UNAVAILABLE` | GitHub API call failed (rate limit, offline, DNS). Check `hermes_update_check` log on `hermes_commandbox`. |
| _(cache file missing)_ | `UPDATE CHECK PENDING` | First-ever render before the 04:30 job has run. Wait one cycle or invoke manually (below). |

## Running the update

### Today (CLI)

The page is currently a notice that delegates to the docs. To
actually apply an update, SSH to the Docker host and run the
orchestrator:

```
cd /opt/hermes-seg-docker-gl
./scripts/system_update_docker.sh                 # apply latest
./scripts/system_update_docker.sh v260601         # apply a specific tag
./scripts/system_update_docker.sh --dry-run       # show what would run, change nothing
./scripts/system_update_docker.sh --skip-git      # containers + artifacts only
./scripts/system_update_docker.sh --skip-compose  # git + artifacts only
./scripts/system_update_docker.sh -y              # don't prompt for confirmation
```

The orchestrator walks five phases. For the full breakdown of each
phase — preflight, code pull, container update, per-release artifact
application, finalize, and the persistent post-upgrade hook — see
[Release and Update Methodology § The update orchestrator](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology#the-update-orchestrator-scriptssystem_update_dockersh).
For the categories of artifact the orchestrator applies (baseline vs
per-release vs persistent hook), see [§ Artifact taxonomy](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology#artifact-taxonomy--where-does-what-go).

A condensed version of what the orchestrator does:

| Phase | What it does | Idempotent? |
|---|---|---|
| Preflight | Refuses to run if working tree dirty, `hermes_db_server` down, or target older than current | Trivially |
| 1 — Pull new code | `git fetch --tags` + `git checkout <tag>` | Yes |
| 2 — Update containers | `docker compose pull` + `docker compose up -d` | Yes; only restarts services whose image or config changed |
| 3 — Apply per-release artifacts | Walks `updates/v*/` directories newer than current `build_no`, applies `sql/` → `cfml/` → `scripts/` in order; each release's `schema_updates.sql` advances `build_no` at its end | Yes (every artifact must be idempotent — see methodology doc) |
| 4 — Finalize | Restarts `hermes_commandbox`; logs reminders for `occ upgrade` (if `NCVERSION` bumped) and `*.HERMES` template re-render | Yes |
| 5 — Post-upgrade hook | `curl http://localhost:8888/schedule/post_upgrade.cfm` — runs any persistent migrations gated by the `migrations` table | Yes (per-block gated) |

Output is teed to a timestamped log under `install-logs/`:
`install-logs/hermes_update_YYYYMMDD_HHMMSS.log`. If anything fails,
the orchestrator aborts (`set -e`); inspect the log, fix the
underlying issue, and re-run. Idempotency makes mid-upgrade resume
safe — a failed Phase 3 picks up at the same release on the next
run and re-applies its full artifact set; `IF NOT EXISTS` and
`INSERT IGNORE` guards turn the second pass into a no-op.

### Tomorrow (in-page button)

The page is positioned to grow a "Check Now" button (force-runs the
daily check ahead of schedule) and a "Run Update" button (invokes
the orchestrator via a CFML wrapper). Neither is wired today; the
infrastructure they would call is already in place.

Track this in [#221](https://github.com/deeztek/Hermes-Secure-Email-Gateway/issues/221).

## Forcing a manual check

If you cannot wait for the 04:30 UTC schedule (e.g., a release just
shipped and you want the dashboard to update now), invoke the same
endpoint Ofelia does:

```
docker exec hermes_commandbox curl --silent http://localhost:8888/schedule/check_for_update.cfm
```

The response is the literal string `OK` and the cache file is
rewritten in place. The dashboard picks it up on the next page load.

The same invocation is what Ofelia would have run at 04:30 — there
is no difference between manual and scheduled execution.

## The version stamp

What the orchestrator and the dashboard both compare against is
the **`build_no`** row in `system_settings`:

| Setting | Value | Set by |
|---|---|---|
| `version_no` | `Docker` | Baseline (`hermes_install.sql`) on fresh install; never changes in the Docker era |
| `build_no` | `vYYMMDD` | Baseline at install; advanced by each release's `updates/v<DATE>/sql/schema_updates.sql` at its very end |

A successful Phase 3 ends with `build_no` matching the target tag.
If after an orchestrator run those two disagree, something in Phase
3 silently no-op'd a stamp-advance — inspect the log. See [Release
and Update Methodology § The release-cut procedure](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology#the-release-cut-procedure-developer-side)
for the exact `UPDATE system_settings ...` block every release's
`schema_updates.sql` ends with.

## Skipping releases

The orchestrator handles release-skipping natively. Upgrading from
`v260119` straight to `v260801` (skipping a hypothetical intermediate
`v260601`) walks **both** release directories in order during Phase
3 — `v260601/` first, then `v260801/`. `build_no` advances after each
release's `sql/` step, so the in-between cursor advancement is safe.

> **Operational consequence.** Releases are designed to be applied
> in chronological order; skipping is supported (and tested) but is
> not the optimized path. If you upgrade rarely, expect Phase 3 to
> take proportionally longer the further behind you are.

## Failure semantics

| What breaks | What happens |
|---|---|
| GitHub Releases API unreachable | `UPDATE CHECK UNAVAILABLE` in dashboard cell; cached value is overwritten with the unavailable marker. Logged to `hermes_update_check`. |
| GitHub Releases API rate-limited (HTTP 403 or 429) | Same as unreachable — anonymous polling is subject to GitHub's 60 req/hr per-IP limit. The daily schedule keeps usage trivial; the only way to hit the limit is repeated manual invocations. |
| `/releases/latest` returns 404 (no qualifying release on the repo) | Treated as `NOUPDATE`, not an error — the repo simply hasn't shipped its first qualifying release yet. |
| `published_at` in API response fails `ParseDateTime` | Falls back to the raw ISO string in the cache file — non-fatal. |
| `cfmail` notification fails | Logged to `hermes_update_check`; cache file write proceeds (notification is best-effort). |
| Cache file cannot be written (`/opt/hermes/updates/` not writable) | Logged; the dashboard falls through to `UPDATE CHECK PENDING`. |
| Orchestrator Phase 1 fails (tag not pushed, dirty tree) | Aborts before touching containers or DB. Working tree is unchanged. |
| Orchestrator Phase 2 fails (image pull error, registry unreachable) | Aborts; previous containers keep running with their existing images. Re-run after fixing the registry / network issue. |
| Orchestrator Phase 3 fails on a SQL artifact | Aborts; `build_no` reflects whatever the last successful release's stamp set it to. Re-run picks up at the failed release; idempotency guards re-apply the partial work safely. |
| Orchestrator Phase 5 fails | Logged as a warning, **not** treated as fatal — the orchestrator exits 0. Run `post_upgrade.cfm` manually after fixing the underlying issue: `docker exec hermes_commandbox curl --silent http://localhost:8888/schedule/post_upgrade.cfm` |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_system_updates.cfm` | `hermes_commandbox` | The admin page (notice + future Run Update wiring) |
| `config/hermes/var/www/html/admin/2/inc/check_system_update.cfm` | `hermes_commandbox` | Reads the cache file for the dashboard cell |
| `config/hermes/var/www/html/schedule/check_for_update.cfm` | `hermes_commandbox` | Daily poll target |
| `config/ofelia/config.ini` (`hermes-update-check` job) | `hermes_ofelia` | Schedules the daily poll |
| `scripts/system_update_docker.sh` | host shell | The update orchestrator |
| `scripts/install_hermes_docker.sh --apply-schema` | host shell | Legacy pre-orchestrator schema-apply path; superseded by the orchestrator but still functional for emergency manual use |
| `/opt/hermes/updates/check_system_update.txt` | `hermes_commandbox` | Cache file; format above |
| `install-logs/hermes_update_<timestamp>.log` | host filesystem | Orchestrator output, teed live |
| `system_settings.build_no` / `system_settings.version_no` | `hermes_db_server` (`hermes` DB) | The version stamp the orchestrator and the dashboard both read |
| `migrations` table | `hermes_db_server` (`hermes` DB) | Tracks which Phase 5 migration blocks have run; see [Methodology § Phase 5](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology#phase-5--persistent-post-upgrade-hook) |
| `updates/v*/sql/schema_updates.sql` | repo working tree | Per-release SQL deltas; one of three artifact categories |
| `updates/v*/cfml/*.cfm` | repo working tree | Per-release CFML migrations (encryption / file IO / API calls) |
| `updates/v*/scripts/*.sh` | repo working tree | Per-release host-shell one-shots |
| `config/hermes/var/www/html/schedule/post_upgrade.cfm` | `hermes_commandbox` | The persistent cross-release migration hook |

## Related

- [Release and Update Methodology](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology)
  — **the canonical reference** for everything covered on this page.
  Read it before adding a schema change, a one-shot migration, a
  service config edit, or cutting a release tag.
- [System Status](https://docs.deeztek.com/books/administrator-guide/page/system-status) — the dashboard that surfaces
  the **Hermes Update** cell this page's daily check populates
- [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings) — `admin_email` (target of
  the update-found notification email), `postmaster` (sender), and
  the legacy `daily_update_check` toggle
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) — the admin surface over the
  Ofelia config that schedules the daily check
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — where `hermes_update_check` log
  entries surface for debugging failed polls
- [Storage Topology](https://docs.deeztek.com/books/installation-reference/page/storage-topology-5-tiers) — the four
  storage tiers an upgrade touches (Config tier is where `git
  checkout` runs; Data tier holds `/opt/hermes/updates/`)

# System Users

# System Users

Admin path: **System > System Users** (`view_system_users.cfm`,
`inc/system_user_actions.cfm`, `inc/ldap_add_user.cfm`,
`inc/ldap_add_user_remoteauth.cfm`, `inc/ldap_add_user_groups.cfm`,
`inc/ldap_modify_user.cfm`, `inc/ldap_modify_user_password.cfm`,
`inc/ldap_change_user_access_control.cfm`,
`inc/ldap_delete_user.cfm`, `inc/delete_system_user.cfm`,
`inc/delete_system_user_devices.cfm`, `inc/generate_ldap_password.cfm`,
`inc/check_hibp.cfm`).

This page manages **admin console operators** — the accounts that can
sign in at `/admin/`. Mailbox users (Email Server) and relay recipients
(Email Relay) are not managed here even though they share the same
underlying LDAP tree; they have their own admin pages.

Each row written by this page lands in **two** stores: the `system_users`
table (Hermes DB — UI metadata, auth-type flag, `applied`/`ldap_synced`
status), and an LDAP entry under `ou=users,dc=hermes,dc=local` whose
group memberships in `ou=groups` give the user actual access. Authelia
binds against LDAP for every console login; the DB row exists so the
admin UI has something to display and edit.

## What this page creates — and what it doesn't

| Creates | Doesn't create |
|---|---|
| Console admin accounts (`cn=admins` group membership) | Mailbox accounts (those go through **Email Server > Mailboxes**, populate `mailboxes` + `cn=mailboxes`) |
| LDAP entry + DB row in lockstep | Relay-recipient accounts (those go through **Email Relay > Recipients**, populate `recipients` + `cn=relays`) |
| Local-auth (password lives in Hermes LDAP) **or** RemoteAuth (password lives in upstream AD/LDAP) admins | Authelia-side rows (Authelia is stateless against LDAP — no per-user provisioning needed) |
| `cn=one_factor` or `cn=two_factor` group membership at create time | The MFA enrolment itself — the user still has to enrol TOTP/WebAuthn/Duo from the user portal's Account Settings page once they sign in |

> **Operational consequence.** Every account this page creates is an
> admin. There is no "create a reader-only admin" or "create an
> auditor" path today. Granular role assignment is a planned extension;
> the current model is binary — either you're an admin (full console
> access) or you're not. The `access_control` column gates one-factor
> vs. two-factor at the **login gate**, not at the privilege level.

## How LDAP membership is structured

System users live under the same OU as every other identity in Hermes,
and the user's role is determined by **which groups contain their DN**
in the `member` attribute (see [Credential Model](https://docs.deeztek.com/books/administrator-guide/page/credential-model) for the full architecture).

```
dc=hermes,dc=local
├── ou=groups
│   ├── cn=admins              <-- every System User is added here
│   ├── cn=mailboxes           <-- mailbox users (not this page)
│   ├── cn=relays              <-- relay recipients (not this page)
│   ├── cn=one_factor          <-- access_control = one_factor
│   └── cn=two_factor          <-- access_control = two_factor
└── ou=users
    ├── cn=admin               <-- the install-time built-in admin
    ├── cn=jsmith              <-- example local-auth System User
    └── cn=corp_user           <-- example remote-auth stub entry
```

`inc/ldap_add_user_groups.cfm` adds the new System User's DN to **both**
`cn=admins` and the chosen access-control group in a single LDIF
operation. The LDIF template `/opt/hermes/templates/ldap_addusergroup.ldif`
contains two `changetype: modify` blocks that both reference the same
`THE_USERNAME` placeholder.

## Database schema — `system_users`

| Column | Purpose |
|---|---|
| `id` | PK |
| `username` | LDAP `cn` / `uid`. Immutable after create (the edit modal renders this field read-only). |
| `email` | `mail` LDAP attribute; also where forgotten-password notifications would go (but admin self-service reset is **disabled** for security — see Password Resets below) |
| `first_name`, `last_name` | `givenName`, `sn` |
| `password` | Argon2id hash with the `{ARGON2}` prefix that OpenLDAP's argon2 overlay expects. Empty string for RemoteAuth users (their password is upstream). |
| `access_control` | `one_factor` or `two_factor` — drives Authelia's access-control policy at login |
| `auth_type` | `local` or `remote` — drives the entire create/edit flow |
| `remoteauth_domain` | For `auth_type = 'remote'`, the `domain_name` key into `remoteauth_mappings`. NULL for local-auth. |
| `system` | `1` = install-time built-in admin (delete-protected). `2` = admin-created. |
| `applied` | `1` = current state synced to LDAP. `2` = pending sync (transient during a save). |
| `ldap_synced` | `1` = LDAP entry exists. `0` = DB row exists but LDAP entry doesn't (a half-sync state the edit handler explicitly detects and tries to repair). |
| `pushover_user_key`, `pushover_enabled` | Optional Pushover notifications for admin alerts |

## Local-auth user create flow

```
Admin clicks Create System User
        │
        ▼
form validation: username regex, email format,
first/last name regex, password length 8-64
        │
        ▼ (optional)
HIBP check: SHA-1 prefix sent to api.pwnedpasswords.com
        │   reject if hash suffix matches a known breach
        ▼
generate_ldap_password.cfm
        │   docker run --rm authelia/authelia:VERSION \
        │     authelia crypto hash generate argon2 \
        │     --password <plaintext>
        │   returns: {ARGON2}$argon2id$v=19$m=...$...$...
        ▼
INSERT INTO system_users (..., password='{ARGON2}...')
        │
        ▼
ldap_add_user.cfm  -- builds adduser LDIF from template,
        │             docker exec hermes_ldap ldapadd
        │             writes entry to ou=users with userPassword
        ▼
ldap_add_user_groups.cfm  -- adds DN to cn=admins
        │                    + cn=<one_factor|two_factor>
        ▼
UPDATE system_users SET ldap_synced = 1
        ▼
session.m = 20  ("System User was created successfully")
```

The Authelia hash generator runs as a **one-shot `docker run --rm`**
against the same Authelia image the platform already runs — zero host
dependency, format guaranteed to match what Authelia validates at
login. The hashing happens in `inc/generate_ldap_password.cfm`.

## RemoteAuth user create flow

When the **Authentication Type** dropdown is set to `Remote`, the form
shape changes: the password fields disappear and a **RemoteAuth
Domain** dropdown becomes required (populated from
`remoteauth_mappings` where `enabled = 1`). This option only appears
when (a) the install has a Pro license, (b) `remoteauth_settings.enabled = 1`,
and (c) at least one enabled mapping exists.

```
INSERT INTO system_users (..., password='', auth_type='remote',
                          remoteauth_domain='<key>')
        │
        ▼
ldap_add_user_remoteauth.cfm  -- writes a stub entry with NO password,
        │                        with seeAlso pointing at the upstream
        │                        DN (expanded from the mapping's
        │                        remote_dn_pattern) and associatedDomain
        │                        set to the mapping key
        ▼
ldap_add_user_groups.cfm  -- adds DN to cn=admins
                             + cn=<one_factor|two_factor>
```

At login, Authelia binds locally against the stub. Hermes's
`slapo-remoteauth` overlay sees the `associatedDomain`, finds the
matching upstream URI, and rebinds as the `seeAlso` DN. The local entry
has no `userPassword` to validate against — the upstream bind is the
only decision. See [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) for the
overlay mechanics.

> **Username uniqueness is global.** The `system_users.username` column
> is checked for collision across **both** auth types. If your upstream
> AD already has a user named `dedwards` and Hermes already has a
> local-auth admin named `dedwards`, the second account cannot be
> created with the same username. The form's error message suggests
> `username@domain` or `username.domain` as a workaround.

## Edit flow — what can and cannot change

Two fields are **immutable** after create and rendered read-only in
the edit modal:

| Field | Why immutable |
|---|---|
| **Username** | It's the LDAP RDN (`cn=`). Renaming would require a `modrdn` plus updating every group's `member` attribute that references the old DN. The "delete and recreate" path is simpler and safer. |
| **Authentication Type** | Switching local-to-remote or remote-to-local would change the LDAP entry's objectClass set (loses or gains a password attribute) and break the `seeAlso`/`associatedDomain` overlay reference. Recreate the user instead. |

Everything else is editable: email, first/last name, access-control
policy (one/two factor), and — for local-auth users only — the password
(via the **Set User Password = YES** toggle which reveals the password
fields). The password edit re-runs the same HIBP check and Argon2 hash
flow as create.

The access-control change is non-trivial: switching `one_factor` to
`two_factor` (or vice versa) means removing the DN from the old group
and adding it to the new one. `inc/ldap_change_user_access_control.cfm`
handles both ops in sequence.

### Half-synced repair

If a previous save crashed between the DB INSERT and the LDAP write
(`ldap_synced = 0`, no LDAP entry exists), the edit handler refuses
to save the row in a "NO password change" mode — there's no password
to push into LDAP. Alert code `16` surfaces the explicit instruction:
"set **Set User Password** to YES and enter a new password" so the
sync can complete on the next save attempt. The user's stored
password is **not** re-pushed because the DB column holds an Argon2
hash, not a plaintext.

## Built-in admin protection — the `system` column

The install script seeds a single built-in admin row (the username
chosen at install time) with `system = 1`. The page's UI rules:

- **Delete button** is hidden on the row.
- **Cannot delete self**: the row matching `session.userid` also hides
  its Delete button (a separate check).

Both gates are also enforced server-side in `system_user_actions.cfm`'s
`deleteuser` branch — the SQL lookup explicitly filters
`system <> '1' AND id <> <session.userid>` so a crafted POST cannot
bypass the hidden button.

## Delete flow

Soft-delete is **not** the model — the row is physically removed.

```
1. DB lookup: refuse if system='1' or id=session.userid
2. ldap_delete_user.cfm:
     docker exec hermes_ldap ldapdelete \
       cn=<username>,ou=users,dc=hermes,dc=local
     (this auto-removes the DN from any group's member attribute via
      the OpenLDAP referential-integrity overlay)
3. delete_system_user.cfm:
     DELETE FROM system_users WHERE id = <id>
4. delete_system_user_devices.cfm:
     docker exec hermes_authelia authelia storage user totp delete \
       <username> --config /config/configuration.yml
     docker exec hermes_authelia authelia storage user webauthn delete \
       <username> --config /config/configuration.yml --all
5. session.m = 1  ("System User was deleted successfully")
```

> **Duo Push devices do NOT delete here.** Duo enrolment lives on
> Duo's cloud servers, not Authelia's database. If the deleted user
> was Duo-enrolled, the admin must also remove them from the Duo
> Admin Panel — both the delete and the 2FA-only modals say so
> explicitly. See [Authentication Settings § Duo Security](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings#duo-security).

## Delete 2FA Devices — without deleting the user

The **yellow key** button on each row opens a dedicated **Delete 2FA
Devices** modal that runs only step 4 of the delete flow above. Use
this when:

- A user reports they've lost their phone / hardware key
- A user is stuck in a 2FA loop after a session expiry
- A user needs to re-enrol with a new TOTP app

After running this, the user is back to a one-factor login state for
the next sign-in, then can re-enrol from their Account Settings page.
The page waits 5 seconds before redirecting to give Authelia time to
flush the credential cache before the success banner appears.

> **Note on Authelia config path.** The two `authelia storage` commands
> reference `--config /config/configuration.yml`. That is the
> in-container path, which differs from where you'd expect to find the
> file from the host's perspective. Authelia's working config inside
> the container is `/config/configuration.yml`, NOT `/etc/authelia/`.
> See [Authentication Settings § Storage backend — MySQL, not SQLite](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings#storage-backend-mysql-not-sqlite)
> for why the MariaDB `authelia` database is what actually gets cleaned
> when these commands run.

## have-i-been-pwned (HIBP) check

The **Check Password Against haveibeenpwned.com** toggle (YES/NO,
default YES) sends only the first 5 hex chars of the password's
SHA-1 to `api.pwnedpasswords.com/range/<prefix>` (k-anonymity:
the full hash is never transmitted) and rejects the password if
the remaining 35 hex chars appear in the returned breach list.

If `api.pwnedpasswords.com` is unreachable (no outbound 443, DNS
broken, etc.) the create fails with alert `100` — the admin must
either restore outbound connectivity or disable the check explicitly
on the form. Silently skipping a security check on network failure
would be the wrong default.

## What this page does NOT do

| Concern | Lives on |
|---|---|
| Mailbox creation | [Email Server > Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — separate table, separate LDAP group |
| Relay-recipient creation | [Email Relay > Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) — separate table, separate LDAP group |
| Per-user MFA enforcement (admin-policy flag) | The mailbox / relay-recipient detail pages set `enforce_mfa` for those user classes. System Users use `access_control` instead; if you set it to `two_factor`, Authelia challenges every login. There is no separate "encourage but don't require" middle state for admins — see [Authentication Settings § MFA enforcement is decoupled from the cn=two_factor LDAP group](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings#mfa-enforcement-is-decoupled-from-the-cntwo_factor-ldap-group-225). |
| Password reset queue (admin processes user-initiated requests) | [Password Resets](https://docs.deeztek.com/books/administrator-guide/page/password-resets) |
| Authelia session length, brute-force throttle, Duo / OIDC | [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) |
| Upstream AD/LDAP mapping for RemoteAuth admins | [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — must exist + be enabled before this page's Remote dropdown appears |
| Pushover token (per-admin alert notifications) | Set on the per-admin notification configuration page; the `pushover_user_key` column on `system_users` is populated there, not here |

## Failure semantics

| What breaks | What happens |
|---|---|
| `hermes_ldap` container down | Create + Edit fail at the LDAP step. The DB INSERT has already run, so the row exists with `ldap_synced = 0`. Recovery: restart LDAP, edit the user with **Set User Password = YES** to retry the sync (alert `16` will prompt for this on first reload). |
| `hermes_authelia` container down | Create + Edit + Delete still succeed at the DB + LDAP level; the user can't actually log in until Authelia is back. Delete 2FA Devices fails silently (caught and swallowed in the cftry block) — the next attempt after Authelia recovers will succeed. |
| HIBP API unreachable with HIBP check ON | Create + password-change Edit refuse to save (alert `100`). The admin must either fix outbound connectivity or set HIBP to NO. |
| RemoteAuth domain dropdown empty / RemoteAuth disabled | The Remote option doesn't appear in the dropdown at all. To restore: enable a mapping on [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) and click Apply Settings. |
| Username collision | Alert `13` with the suggested `username@domain` or `username.domain` workaround. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_system_users.cfm` | `hermes_commandbox` | Page (table + 4 modals) |
| `config/hermes/var/www/html/admin/2/inc/system_user_actions.cfm` | `hermes_commandbox` | Action router (create / edit / delete / deletedevices) |
| `config/hermes/var/www/html/admin/2/inc/generate_ldap_password.cfm` | `hermes_commandbox` | `docker run --rm authelia/authelia ... crypto hash generate argon2` |
| `config/hermes/var/www/html/admin/2/inc/ldap_add_user.cfm` | `hermes_commandbox` | LDIF render + `ldapadd` for local-auth entries |
| `config/hermes/var/www/html/admin/2/inc/ldap_add_user_remoteauth.cfm` | `hermes_commandbox` | Stub-entry LDIF render + `ldapadd` for remote-auth entries |
| `config/hermes/var/www/html/admin/2/inc/ldap_add_user_groups.cfm` | `hermes_commandbox` | Adds DN to `cn=admins` + access-control group |
| `config/hermes/var/www/html/admin/2/inc/ldap_change_user_access_control.cfm` | `hermes_commandbox` | Moves DN between `cn=one_factor` and `cn=two_factor` |
| `config/hermes/var/www/html/admin/2/inc/ldap_delete_user.cfm` | `hermes_commandbox` | `ldapdelete` of the user entry |
| `config/hermes/var/www/html/admin/2/inc/delete_system_user_devices.cfm` | `hermes_commandbox` | `authelia storage user totp delete` + `webauthn delete --all` |
| `config/hermes/var/www/html/admin/2/inc/check_hibp.cfm` | `hermes_commandbox` | HTTPS GET to `api.pwnedpasswords.com` |
| `/opt/hermes/templates/ldap_adduser.ldif` | `hermes_commandbox` | Add-user LDIF (placeholder-substituted) |
| `/opt/hermes/templates/ldap_adduser_remoteauth.ldif` | `hermes_commandbox` | Stub-user LDIF |
| `/opt/hermes/templates/ldap_addusergroup.ldif` | `hermes_commandbox` | Two-block LDIF for `cn=admins` + access-control group add |
| `system_users` table | `hermes_db_server` (`hermes` DB) | Admin metadata + LDAP sync state |
| `cn=admins,ou=groups,dc=hermes,dc=local` | `hermes_ldap` | Source of truth for who can sign in at `/admin/` |

## Related documentation

- [Credential Model](https://docs.deeztek.com/books/administrator-guide/page/credential-model) — full four-credential architecture; this page's accounts use only the web-login credential
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — required prerequisite for creating remote-auth System Users; covers mappings, DN patterns, TLS settings
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) — Authelia's session lifetime, login regulation, MFA capability vs. enforcement model
- [Password Resets](https://docs.deeztek.com/books/administrator-guide/page/password-resets) — the admin queue for user-initiated reset requests; the page's note on why admin self-service reset is blocked
- [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings) — `/admin/` hostname, cert, and the IP allowlist that layers above this page's access control

# Email Relay

# Domains

# Domains

Admin path: **Email Relay > Domains** (`view_domains.cfm`,
`inc/domain_add_action.cfm`, `inc/domain_edit_action.cfm`,
`inc/domain_delete_action.cfm`, `inc/deletedomain.cfm`,
`inc/get_domain_json.cfm`, `inc/generate_transports.cfm`,
`inc/generate_relay_domains.cfm`, `inc/generate_sasl_password_transport.cfm`,
`inc/generate_postfix_configuration.cfm`,
`inc/add_domain_djigzo.cfm`, `inc/delete_domain_djigzo.cfm`).

This page manages the list of **inbound relay domains** — the SMTP
domains for which Hermes accepts mail and forwards it to a downstream
mail server (Microsoft 365, Exchange, Google Workspace, on-prem
Postfix/Dovecot, an internal hub MTA, etc.). Each row in the
`domains` table is paired with a `transport` row that tells Postfix
where to forward, a `senders` row that flags the domain as a
recognized sender, and a `recipients` row that gates whether the
domain accepts mail for any address or only addresses on the Relay
Recipients allowlist.

This is the inbound counterpart to [Relay Host](https://docs.deeztek.com/books/administrator-guide/page/relay-host). The
two pages together define the **relay topology** half of Hermes:
inbound domains here, outbound smarthost there.

> **Not to be confused with [Email Server > Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v).**
> That page is for the **mail-server topology** — domains where
> Hermes IS the destination MTA and delivers locally to Dovecot
> mailboxes. It writes to the `mailbox_domains` table, not the
> `domains` table. The two tables and the two admin pages are
> separate by design because Hermes supports three topologies (see
> [Hermes topology overview](#hermes-topology-overview) below) and a
> single deployment can run any combination.

## Hermes topology overview

```
                  +--------------------------------+
                  |   Hermes Secure Email Gateway  |
                  +--------------------------------+
                          |                |
   inbound smtp (25) ─────+                +───── inbound smtp (25)
                          |                |
                  +-------v------+  +------v-------+
                  |   domains    |  | mailbox_     |
                  |  (relay)     |  |  domains     |
                  +-------+------+  +------+-------+
                          |                |
                          v                v
            forward via   |                |   deliver locally via
            Postfix       |                |   Dovecot LMTP
            transport map |                |
                          v                v
                +---------+-+      +-------+---------+
                | downstream|      | /mnt/vmail      |
                | MX (M365, |      | (mailbox files) |
                | Exchange, |      +-----------------+
                | etc.)     |
                +-----------+
```

| Topology | `domains` rows | `mailbox_domains` rows | This page edits |
|---|---|---|---|
| Relay-only | one or more | none | Yes |
| Mail-server-only | none | one or more | No — use [Email Server > Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) |
| Hybrid | one or more (forwarded) | one or more (delivered locally) | Yes, for the relay subset |

`view_domains.cfm` filters its main query with
`WHERE (d.type IS NULL OR d.type = '' OR d.type = 'relay')` so it
only shows relay-mode rows. Add Domain writes `type='relay'`
explicitly so the row is unambiguously routed to this page.

## How a relay domain becomes Postfix config

A single Add Domain submission writes four database rows and
regenerates four Postfix maps:

```
form submit  ──► domain_add_action.cfm
                     |
                     |  INSERT transport (domain, transport, dest, port, mx, auth, ...)
                     |  INSERT senders   (sender = domain, action = OK)
                     |  INSERT recipients(recipient = @domain, status = OK|"")
                     |  INSERT domains   (domain, transport_id, senders_id,
                     |                    recipients_id, type='relay')
                     |
                     |  --- regenerate ---
                     v
            generate_transports.cfm        -> /etc/postfix/transport
                                              + postmap (docker exec)
            generate_relay_domains.cfm     -> /etc/postfix/relay_domains
            sync_sasl_parameters.cfm
            generate_sasl_password_transport.cfm
                                           -> /etc/postfix/sasl_passwd
                                              + postmap (docker exec)
            generate_tls_policy.cfm        -> /etc/postfix/tls_policy
                                              + postmap (docker exec)
            generate_postfix_configuration.cfm
                                           -> /etc/postfix/main.cf
                                              + postfix reload (docker exec)
            add_domain_djigzo.cfm          -> registers domain in Ciphermail
                                              (encryption gateway)
```

The same pipeline runs on edit and delete (with the appropriate
deletes substituted for inserts). The page deliberately does not
expose a "dry-run" — every change to a domain is a config-changing
save, and the cascade always runs to completion.

## Configuration storage

| Table | Role | Notes |
|---|---|---|
| `domains` | One row per relay domain | `type` column gates which admin page edits the row (`relay`, NULL/empty = relay; anything else = managed elsewhere). `id`, `transport_id`, `senders_id`, `recipients_id` are the join keys. |
| `transport` | One row per domain delivery target | `transport` column holds the Postfix-formatted string (`smtp:[host]:port` or `smtp:host:port` for MX-lookup mode, or `discard:Discard Email Silently`). `authentication = YES` toggles per-domain SASL. `authentication_username` / `authentication_password` are AES/Base64 encrypted with `/opt/hermes/keys/hermes.key`. |
| `senders` | One row per domain (sender = domain, action = `OK`) | Used by Postfix `smtpd_sender_restrictions` to recognise the domain as a known sender. |
| `recipients` | One row per domain (recipient = `@domain`, `domain='1'`) | `status = OK` = accept mail for any address (recipient_delivery = ANY). `status = ''` = require an entry in [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) (recipient_delivery = SPECIFIED). The default `spam_policies` policy is attached so Amavis applies SVF filtering. |
| `tls_policies` | Optional, one row per domain | Auto-managed: created with `method=encrypt` when **Enforce TLS** is on and Auth is YES; removed when either is turned off. Manually-added policies (different `description`) are untouched. |
| `dkim_sign` | Optional, one or more rows per domain | DKIM keys live separately; managed under the per-row **DKIM Keys** button (`edit_domain_dkim.cfm`). DKIM badge in the table reports `Active` / `Disabled` / `None` based on `enabled = '1'` counts. |

## Fields on the page

### Add Domain card

| Field | Default | Notes |
|---|---|---|
| **Domain Name** | (empty) | Trimmed, lower-cased, validated by the email-trick. Uniqueness checked against `domains.domain` — duplicates rejected with error 12. Stored as-is on the row. |
| **Delivery Method** | `SMTP (Recommended)` | `smtp` forwards via the destination address; `discard` writes `discard:Discard Email Silently` into the transport row and accepts mail only to drop it. Useful for honeypot or sunset domains. |
| **Recipient Delivery** | `ANY` | `OK` = accept any recipient at the domain. `""` = SPECIFIED — only addresses listed under [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) are accepted; everything else is rejected at SMTP time with `relay_recipient_maps`. |
| **Destination Address** | `smtp.<domain>` (placeholder) | FQDN or IP of the downstream MX/smarthost. Lower-cased. Required when method = `smtp`. |
| **Port** | `25` | Free-text but validated as integer. No range cap on this page (vs. Relay Host's explicit 1–65535) but Postfix will reject out-of-range. |
| **MX Lookup** | `NO` | `NO` writes a bracketed transport `smtp:[host]:port` (Postfix skips MX, connects directly). `YES` writes unbracketed `smtp:host:port` (Postfix resolves MX records). MX mode is **automatically forced off** when Auth = YES, because authenticated submission with MX rotation rarely makes sense. |
| **Auth** | `NO` | When `YES`, the username/password and Enforce TLS fields reveal. |
| **Destination Username / Password** | (empty) | Required when Auth = YES. Encrypted with `/opt/hermes/keys/hermes.key` before write. On Edit, blank password keeps the existing ciphertext. |
| **Enforce TLS** | checked | When Auth = YES, auto-inserts a `tls_policies` row with `method=encrypt` and `description='Auto-added: domain requires authentication'`. Manages itself on subsequent edits — turning either off deletes the auto-added row but leaves manually-added TLS policies alone. |

### Domains table

Sortable, searchable, exportable (copy/CSV/Excel/PDF/print via the
DataTables Buttons extension; `stateSave: true` so column ordering
and page-size choices persist across reloads). Columns:

| Column | Source | Badge logic |
|---|---|---|
| Domain | `domains.domain` | Plain text |
| Delivery | `transport.method` | `Discard` (warning) or `SMTP` (success) |
| Destination | `transport.destination` | Dash for discard rows |
| Port | `transport.port` | Dash for discard |
| MX | `transport.mx` | Dash for discard |
| Recipients | `recipients.status` | `Any` (info) when `OK`, `Specified` (secondary) otherwise |
| Auth | `transport.authentication` | `YES` (warning) or `NO` (secondary) |
| DKIM | aggregated from `dkim_sign` | `Active` when any enabled key, `Disabled` when keys exist but all disabled, `None` when no keys |
| TLS | derived from `tls_policies.domain` join | `YES` (success) when a policy exists for the domain, `NO` (secondary) otherwise |
| Actions | — | Edit (opens modal), DKIM Keys (→ `edit_domain_dkim.cfm`), Delete (opens confirm modal) |

### Edit Domain modal

Opens via `openEditModal(id)` which fetches
`./inc/get_domain_json.cfm` over AJAX, hydrates the form fields,
then reveals the modal body. **Domain Name is read-only on edit** —
changing a domain name across `domains`/`transport`/`senders`/
`recipients`/`dkim_sign`/`tls_policies` is risky enough that the
page enforces add-and-delete instead. Every other field is editable.

Blank password keeps the existing ciphertext (the masked hint
beneath the input shows `Current: abcd*****` when a stored value
exists).

### Delete Domain modal

Confirms the destructive action. The handler (`deletedomain.cfm`)
runs four dependency checks before allowing the delete:

| Check | If it returns rows → |
|---|---|
| Relay Recipients still pointing at the domain (`recipients.recipient LIKE '%domain%' AND domain IS NULL`) | Error 1, abort |
| Virtual Recipients referencing the domain (`virtual_recipients.virtual_address LIKE '%domain%'`) | Error 2, abort |
| Postmaster address using the domain (`system_settings.postmaster LIKE '%domain%'`) | Error 3, abort |
| DKIM keys for the domain (`dkim_sign.domain LIKE '%domain%'`) | Error 4, abort |

If all four pass, the handler deletes from `domains`, `transport`,
`senders`, and `recipients` (the four rows linked at creation),
clears the `tls_policies` row for the domain, removes the Ciphermail
registration, and regenerates all Postfix maps.

> **Operational consequence.** The dependency checks force a
> bottom-up cleanup. To remove a domain you must first delete its
> recipients, its DKIM keys, and reassign the system postmaster.
> This is intentional — Hermes will not silently strand referencing
> rows, and the order also prevents you from losing in-flight mail
> for active recipients.

## Per-domain auth vs. relay host auth

Per-domain authentication on this page is **separate from and
additive to** the global Relay Host SASL on the [Relay Host](https://docs.deeztek.com/books/administrator-guide/page/relay-host)
page. Both pages write into the same `/etc/postfix/sasl_passwd`
file via the shared `generate_sasl_password_transport.cfm`
generator:

```
# /etc/postfix/sasl_passwd  (regenerated on every save on either page)
[smtp.upstream-isp.com]:587  globaluser:globalpass    <-- Relay Host page
[mx.partner-a.com]:25        partner_a_user:secret1   <-- Domains page (per-domain)
[mx.partner-b.com]:25        partner_b_user:secret2   <-- Domains page (per-domain)
```

A domain with per-domain auth will use **its own** credentials when
Postfix forwards to its destination. The global relay host
credentials are used only when a message has no matching per-domain
transport (typical for outbound mail to arbitrary recipients).

> **By design.** The error code 15 (`Cannot enable Destination
> Authentication when Relay Host is enabled`) is reserved in the
> page's alert table but not currently raised by the action
> handlers — historically the two auth modes were considered
> mutually exclusive, but the consolidated SASL generator handles
> both cleanly, so the constraint was relaxed. The alert is kept
> in case a future tightening reintroduces the rule.

## Discard delivery

Setting Delivery Method to `discard` writes `discard:Discard Email
Silently` into the transport. Postfix accepts mail for the domain
(passing SMTP-time checks and the content filter), then drops it on
the floor — no NDR, no bounce, no forwarding attempt. Useful for:

- Sunset domains that should not generate backscatter
- Honeypot domains for spam-trap analysis
- Catching mail to a domain you control while migration is in
  progress and you don't want it bouncing

The destination/port/MX/auth/TLS fields are hidden in the UI when
discard is selected because none of them apply.

## Failure semantics

| What breaks | What happens |
|---|---|
| Domain name empty | `session.m = 10`, redirect, no DB write |
| Domain name fails email-trick validation | `session.m = 11`, redirect, no DB write |
| Domain name already exists in `domains` | `session.m = 12`, redirect, no DB write |
| Delivery method not in `smtp,discard` | `session.m = 20`, redirect, no DB write |
| Destination address blank when method = smtp | `session.m = 13`, redirect, no DB write |
| Port not an integer | `session.m = 14`, redirect, no DB write |
| Auth = YES but username blank | `session.m = 16`, redirect, no DB write |
| Auth = YES but password blank AND no cached cipher | `session.m = 17`, redirect, no DB write |
| Delete blocked by dependency check | One of `session.m = 1..4` per the table above, redirect, no DB write |
| `postmap` of `transport`/`sasl_passwd`/`tls_policy` fails | New map file is on disk but `.db` lags; next mail flow uses stale data until next successful postmap |
| `postfix reload` fails | Live config keeps the previous values; reload error is in container logs |
| `add_domain_djigzo.cfm` errors during Ciphermail registration | Domain row is already in the DB; encryption gateway will not know about the domain until the next manual sync. Re-saving the domain triggers a fresh registration attempt. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_domains.cfm` | `hermes_commandbox` | Page + Add/Edit/Delete modals |
| `config/hermes/var/www/html/admin/2/inc/domain_add_action.cfm` | `hermes_commandbox` | Add handler |
| `config/hermes/var/www/html/admin/2/inc/domain_edit_action.cfm` | `hermes_commandbox` | Edit handler |
| `config/hermes/var/www/html/admin/2/inc/domain_delete_action.cfm` | `hermes_commandbox` | Delete dispatch (thin wrapper) |
| `config/hermes/var/www/html/admin/2/inc/deletedomain.cfm` | `hermes_commandbox` | Delete handler with dependency checks |
| `config/hermes/var/www/html/admin/2/inc/get_domain_json.cfm` | `hermes_commandbox` | AJAX hydrator for the Edit modal |
| `config/hermes/var/www/html/admin/2/inc/generate_transports.cfm` | `hermes_commandbox` | Rewrites `/etc/postfix/transport` + postmap |
| `config/hermes/var/www/html/admin/2/inc/generate_relay_domains.cfm` | `hermes_commandbox` | Rewrites `/etc/postfix/relay_domains` |
| `config/hermes/var/www/html/admin/2/inc/generate_sasl_password_transport.cfm` | `hermes_commandbox` | Shared `sasl_passwd` generator (also used by [Relay Host](https://docs.deeztek.com/books/administrator-guide/page/relay-host)) |
| `config/hermes/var/www/html/admin/2/inc/generate_tls_policy.cfm` | `hermes_commandbox` | Rewrites `/etc/postfix/tls_policy` + postmap |
| `config/hermes/var/www/html/admin/2/inc/generate_postfix_configuration.cfm` | `hermes_commandbox` | Template-to-`main.cf` renderer + `postfix reload` |
| `config/hermes/var/www/html/admin/2/inc/add_domain_djigzo.cfm` / `delete_domain_djigzo.cfm` | `hermes_commandbox` | Ciphermail (djigzo) domain registration |
| `/etc/postfix/transport` + `.db` | `hermes_postfix_dkim` | Per-domain transport map (regen target) |
| `/etc/postfix/relay_domains` | `hermes_postfix_dkim` | List of domains Postfix accepts mail for (regen target) |
| `/etc/postfix/sasl_passwd` + `.db` | `hermes_postfix_dkim` | Consolidated SASL credentials (regen target) |
| `/etc/postfix/tls_policy` + `.db` | `hermes_postfix_dkim` | Per-destination TLS policy (regen target) |
| `/etc/postfix/main.cf` | `hermes_postfix_dkim` | Live Postfix config (re-rendered on every save) |
| `/opt/hermes/keys/hermes.key` | `hermes_commandbox` | Symmetric key for AES/Base64 cred encryption |
| `domains`, `transport`, `senders`, `recipients`, `tls_policies`, `dkim_sign` | `hermes_db_server` | The relay-domain row group |

Every shell-out uses `docker exec hermes_postfix_dkim ...` per the
standard Hermes pattern.

## Related

- [Relay Host](https://docs.deeztek.com/books/administrator-guide/page/relay-host) — outbound smarthost; the page's twin.
  Shares the `sasl_passwd` generator and is part of the same relay
  topology.
- [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) — recipient allowlist used
  when a domain's Recipient Delivery is set to `SPECIFIED`. Required
  reading if you tighten recipient validation for a domain.
- [Virtual Recipients](https://docs.deeztek.com/books/administrator-guide/page/virtual-recipients) — alias and catch-all
  mappings (`alias@dom → real@dom`). Independent of this page but
  domain deletes block when virtual rows reference the domain.
- [Relay Networks](https://docs.deeztek.com/books/administrator-guide/page/relay-networks) — `mynetworks` (which clients
  may relay outbound without authentication). The networks that hold
  the per-domain submission clients live here.
- [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) — manages
  per-destination TLS policies (the Enforce TLS checkbox on this
  page is a shortcut into the same table).
- [Email Server > Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — the
  separate page for mail-server-topology domains, backed by
  `mailbox_domains`. **Do not confuse with this page.**

# Relay Host

# Relay Host

Admin path: **Email Relay > Relay Host** (`view_relay_host.cfm`,
`inc/get_relay_host_settings.cfm`, `inc/edit_relay_host_settings.cfm`,
`inc/generate_sasl_password_transport.cfm`,
`inc/generate_postfix_configuration.cfm`).

This page configures the **single global outbound relay host** that
Postfix uses to deliver mail to the Internet — the smarthost an ISP,
M365, SendGrid, or another upstream MTA supplies when direct delivery
is blocked or undesirable. It controls the host/port pair, the optional
SASL credentials, and the outbound TLS security level. Saving rewrites
the relevant rows in the `parameters` table, regenerates
`/etc/postfix/sasl_passwd`, and re-renders `/etc/postfix/main.cf` from
the template so the new values take effect on the next message.

Pairs with [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) for the inbound half of the relay
topology — Relay Host defines where outbound mail goes; Domains defines
which inbound domains Hermes accepts and where each one is forwarded.

## When you need a relay host

By default, Hermes attempts direct MX delivery for outbound mail. A
relay host is required in any of these scenarios:

| Scenario | Why direct delivery fails |
|---|---|
| Hermes is behind a firewall that blocks outbound TCP/25 | Port 25 to the open Internet is filtered |
| ISP forbids outbound SMTP for residential/business links | Outbound TCP/25 is dropped at the ISP edge |
| Outbound IP has no PTR record or is on a blocklist | Recipients reject; deliverability tanks |
| Compliance requires all outbound mail to traverse a known SMTP gateway (M365 connector, SendGrid, on-prem hub) | Centralized policy/journaling/encryption point |
| Hermes sits on a non-routable internal network | No path to the Internet without a smarthost |

If none of those apply and Hermes has a clean public IP with a PTR
record, leave **Enable Relay Host** off and let Postfix do direct
delivery.

## How the relay host fits in the outbound path

```
local pickup / amavis re-inject (10025)
        |
        v
hermes_postfix_dkim (smtp client)
        |
        |  relayhost          = [smtp.example.com]:587   (from parameters)
        |  smtp_sasl_*        = enable + sasl_passwd map (from parameters + sasl_passwd)
        |  smtp_tls_security  = may | encrypt            (from parameters)
        |
        v
upstream smarthost  ──►  recipient MX
```

Only the upstream-bound TCP connection is affected. Inbound SMTP on
port 25, the content-filter loop (Amavis on 10024/10026), and Dovecot
LMTP delivery are untouched.

## Configuration storage

Relay Host settings are spread across two tables. The host/port and
SASL toggles live in the **`parameters`** table using the dual-row
pattern (`child=2` parent name row, `child=1` value row). The SASL
credentials themselves are encrypted at rest in **`system_settings`**
to keep cleartext out of the directive table.

| Setting | Storage | Notes |
|---|---|---|
| Enable Relay Host | `parameters.enabled` on `parameter='relayhost' AND child=2` | Master switch; disabling clears the child value and pushes `relayhost =` (empty) into `main.cf` |
| Relay Host Address | `parameters.name` on the `relayhost` child row | Bare FQDN/IP for display |
| Relay Host Port | Parsed from `parameters.parameter` (`[host]:port`) | Stored as the Postfix-formatted bracketed `[host]:port` literal |
| Outbound TLS Mode | `parameters.parameter` on `smtp_tls_security_level` child row (`""`, `may`, `encrypt`) | Empty value disables both parent and child; `may` = opportunistic STARTTLS; `encrypt` = mandatory TLS |
| Authentication required | `parameters.enabled` on `smtp_sasl_auth_enable` parent + `parameters.parameter` value `yes`/`no` | Flips the `smtp_sasl_password_maps` parent in lockstep |
| Relay Host Username | `system_settings.value` row `relay_host_username` | AES/Base64 encrypted with `/opt/hermes/keys/hermes.key` |
| Relay Host Password | `system_settings.value` row `relay_host_password` | AES/Base64 encrypted with the same key |

> **By design.** The legacy schema kept the SASL username/password
> in plaintext on the `smtp_sasl_password_maps` child row's `name`
> column. The current code path encrypts both into `system_settings`
> and clears the legacy column on every save. The first read against
> a legacy install runs a one-shot migration in
> `get_relay_host_settings.cfm`: if `system_settings` is empty but the
> old `parameters.name` colon-delimited string is present, the values
> are encrypted forward and the plaintext column is cleared. No admin
> action is required.

## Fields on the page

### Enable Relay Host

Master switch. When off, all the other fields are hidden, the
`relayhost` parent is set `enabled=0`, the child value is wiped, and
the SASL parent/child rows + `system_settings` credentials are cleared
in the same save. Postfix is then re-rendered with `relayhost =` empty
so the next outbound message attempts direct delivery again.

### Relay Host Address

Accepts:

- **IPv4** — validated against a dotted-quad regex with 0–255 octet
  bounds
- **IPv6** — validated against a simplified colon/hex check
- **FQDN** — validated by the email-trick (`IsValid("email",
  "bob@<host>")`)

Trimmed before storage. The address is stored on its own (in
`parameters.name`) and also formatted into the Postfix-required
bracketed literal `[host]:port` (in `parameters.parameter`) so that
Postfix skips MX lookups and connects directly. Brackets are always
emitted for the relay host — round-robin via MX is not part of this
page's model; if you need MX-driven relay distribution, configure DNS
upstream of the brackets.

### Relay Host Port

1–65535. Default `25`. The page's helper text surfaces the three
common values:

| Port | Typical use |
|---|---|
| `25` | Inbound MX / unauthenticated relay |
| `587` | Submission with STARTTLS + SASL (most modern smarthosts) |
| `465` | Submission over implicit TLS (SMTPS) — Postfix needs `wrappermode` adjustments not exposed on this page; prefer `587` when the smarthost supports it |

### Outbound TLS Mode

Maps directly to Postfix's `smtp_tls_security_level` for client
connections (not to be confused with the `smtpd_tls_*` server-side
settings configured under [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings)).

| UI value | `main.cf` value | Behavior |
|---|---|---|
| Disabled - No TLS | parent `enabled=0` (no directive emitted) | Plaintext only; STARTTLS not attempted |
| Opportunistic TLS (Recommended) | `smtp_tls_security_level = may` | STARTTLS used if offered; falls back to plaintext otherwise |
| Mandatory TLS | `smtp_tls_security_level = encrypt` | STARTTLS required; delivery fails if the upstream does not offer it. No certificate verification — use a TLS policy for that. |

Pick **may** for port 587 with STARTTLS, **encrypt** if your smarthost
contract requires confirmed encryption. For verified-peer TLS to a
specific smarthost, layer on a TLS policy via
[SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings).

### Authentication

When toggled on, **Username** and **Password** become required. The
password input is masked-and-replaceable: it is rendered blank with
the first 4 characters of the stored value shown beneath as a hint
(`abcd*****`), and a blank submit keeps the existing encrypted value.
Set a new value to rotate.

The handler reads `/opt/hermes/keys/hermes.key`, encrypts both fields
(`AES` / `Base64`), and writes the ciphertext into `system_settings`.
The decryption path is symmetric — `generate_sasl_password_transport.cfm`
reads, decrypts, and writes the `[host]:port  user:pass` line to
`/etc/postfix/sasl_passwd` before postmapping it.

## Save flow — the cascade

Clicking **Save Settings** posts `action=save`. The handler runs a
strict sequence:

```
1. Validate Enable + (if enabled) host + port + (if auth) user/pass
2. edit_relay_host_settings.cfm
   - update parameters rows (relayhost, smtp_sasl_auth_enable,
     smtp_sasl_password_maps, smtp_tls_security_level)
   - if auth: encrypt creds, write to system_settings,
     clear legacy plaintext on parameters.name
   - if not auth or disabled: clear system_settings credentials,
     disable all SASL parameter rows
   - call generate_sasl_password_transport.cfm
     -> rewrites /etc/postfix/sasl_passwd
     -> docker exec hermes_postfix_dkim postmap /etc/postfix/sasl_passwd
3. generate_postfix_configuration.cfm
   - copies /etc/postfix/main.cf to main.cf.HERMES (write-time backup)
   - copies /opt/hermes/conf_files/main.cf.HERMES template -> main.cf
   - chown root:root via docker exec hermes_postfix_dkim
   - iterates enabled parameters rows, substitutes the directive name
     and value into main.cf
   - docker exec hermes_postfix_dkim postfix reload
4. cflocation back with session.m = 10 (success banner)
```

Validation failures short-circuit with `session.m` set to the matching
error code (1–6) and a redirect — no partial DB writes land.

## `sasl_passwd` generation — consolidated, not per-page

`generate_sasl_password_transport.cfm` is a **shared** generator
called by both this page and the [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) Add/Edit/Delete
handlers. It is the single source of truth for `/etc/postfix/sasl_passwd`
and rebuilds the file from scratch each invocation:

```
# /etc/postfix/sasl_passwd  (regenerated on every save)
[smtp.example.com]:587    relayuser:relaypassword       <-- this page (relay host)
[mx1.partner.com]:25      partneruser:partnerpassword   <-- Domains page (per-domain auth)
[mx2.partner.com]:25      otheruser:otherpassword       <-- Domains page (per-domain auth)
```

The relay host entry is added if **all** of:

- `smtp_sasl_auth_enable` parent is enabled
- Decrypted username AND password from `system_settings` are non-empty
- `relayhost` child value is non-empty

Per-domain entries are added from `transport` rows where
`authentication = 'YES'`. Postfix uses the bracketed `[host]:port`
key on the relay host line to match its own bracketed `relayhost`
directive — that exact-key match is why the brackets matter.

> **Operational consequence.** Disabling the relay host on this page
> wipes the relay-host row from `sasl_passwd` but does **not** touch
> per-domain entries from the Domains page. Conversely, deleting a
> domain with `authentication = YES` removes only that domain's
> entry. The two pages compose cleanly via the shared generator.

## Credential rotation

To rotate the relay host password without changing anything else:

1. Open **Email Relay > Relay Host**.
2. Type the new password into the **Password** field.
3. Click **Save Settings**.

The handler encrypts the new value into `system_settings`,
`generate_sasl_password_transport.cfm` rewrites `sasl_passwd` with the
decrypted new value, `postmap` rebuilds the `.db`, and Postfix picks
up the change on the next outbound connection (no daemon restart
needed — Postfix re-reads hash maps lazily).

Rotating the encryption key itself (`/opt/hermes/keys/hermes.key`) is
handled by `rotate_db_credentials.sh` — see that script for the full
re-encryption sweep across `system_settings` and the `transport`
table.

## Failure semantics

| What breaks | What happens |
|---|---|
| Host fails IPv4/IPv6/FQDN validation | `session.m = 2`, redirect, no DB write |
| Port empty or non-integer or out of range | `session.m = 3` or `4`, redirect, no DB write |
| Auth enabled, username blank | `session.m = 5`, redirect, no DB write |
| Auth enabled, password blank AND `system_settings.value` empty | `session.m = 6`, redirect, no DB write |
| Auth enabled, password blank but cached cipher present | Cached value is decrypted and reused; no error |
| Postfix template substitution fails (`generate_postfix_configuration.cfm`) | The error include surfaces the message; the **previous** `main.cf` has already been overwritten with the template copy at that point — recovery is to restore from `main.cf.HERMES` (the write-time backup the same script creates) and re-save |
| `docker exec hermes_postfix_dkim postfix reload` fails | The next inbound delivery attempt re-reads `main.cf`; no immediate user-facing symptom unless directives changed |
| `docker exec hermes_postfix_dkim postmap` fails | The new `sasl_passwd` is on disk but the `.db` lags; outbound auth uses the stale `.db` until the next successful postmap |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_relay_host.cfm` | `hermes_commandbox` | Page |
| `config/hermes/var/www/html/admin/2/inc/get_relay_host_settings.cfm` | `hermes_commandbox` | Load handler + legacy-cred migration |
| `config/hermes/var/www/html/admin/2/inc/edit_relay_host_settings.cfm` | `hermes_commandbox` | Save handler |
| `config/hermes/var/www/html/admin/2/inc/generate_sasl_password_transport.cfm` | `hermes_commandbox` | Consolidated `sasl_passwd` generator (shared with Domains page) |
| `config/hermes/var/www/html/admin/2/inc/generate_postfix_configuration.cfm` | `hermes_commandbox` | Template-to-`main.cf` renderer + `postfix reload` |
| `/opt/hermes/conf_files/main.cf.HERMES` | `hermes_commandbox` | Postfix template Hermes renders from |
| `/etc/postfix/main.cf` | `hermes_postfix_dkim` (volume-mounted) | Live Postfix config (regen target) |
| `/etc/postfix/main.cf.HERMES` | `hermes_postfix_dkim` (volume-mounted) | Write-time backup created on every regen |
| `/etc/postfix/sasl_passwd` | `hermes_postfix_dkim` (volume-mounted) | Plain-text credentials file (regen target) |
| `/etc/postfix/sasl_passwd.db` | `hermes_postfix_dkim` | postmap-built hash database |
| `/opt/hermes/keys/hermes.key` | `hermes_commandbox` | Symmetric key for AES/Base64 cred encryption |
| `system_settings` rows `relay_host_username`, `relay_host_password` | `hermes_db_server` | Encrypted credential storage |
| `parameters` rows: `relayhost`, `smtp_sasl_auth_enable`, `smtp_sasl_password_maps`, `smtp_tls_security_level` (each as `child=2` parent + `child=1` value) | `hermes_db_server` | Postfix directive driver rows |

Every shell-out uses `docker exec hermes_postfix_dkim ...` per the
standard Hermes pattern; nothing on this page touches the host's own
Postfix (there is none).

## Related

- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) — companion page for inbound relay-mode
  domains. The two pages share `generate_sasl_password_transport.cfm`
  and together define the entire relay topology.
- [Relay Networks](https://docs.deeztek.com/books/administrator-guide/page/relay-networks) — `mynetworks` (which clients
  are allowed to relay outbound without authentication). Independent
  of this page but part of the same outbound story.
- [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) — recipient validation for
  inbound relay-mode domains; complements [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains).
- [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) — outbound
  TLS policy per destination (peer verification, cipher pinning).
  The TLS Mode dropdown on this page sets the *default* level;
  per-destination policies override.
- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — Postfix `myorigin` /
  `myhostname` and host IP. Defines the identity the relay host sees
  in EHLO/MAIL FROM.

# Relay Networks

# Relay Networks

Admin path: **Email Relay > Relay Networks** (`view_relay_networks.cfm`,
`inc/get_relay_networks.cfm`, `inc/generate_postfix_configuration.cfm`).

This page manages the **operator-additive list of trusted IPs and CIDR
networks** that are allowed to relay mail through the gateway without
SMTP authentication. The list is composed into Postfix's `mynetworks`
directive alongside two hardcoded baseline entries (`127.0.0.1` and the
Docker subnet) and propagated to Amavis's `@inet_acl` so the content
filter trusts the same source IPs. Every directive listed in
`mynetworks` matches the `permit_mynetworks` clause at the head of
`smtpd_recipient_restrictions` and bypasses RBL, sender, and recipient
checks — misconfiguring it turns the gateway into an open relay.

This is the **trusted-sender** half of the inbound-control story.
Pairs with [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) (the trusted-target
list) and [Relay Host](https://docs.deeztek.com/books/administrator-guide/page/relay-host) / [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) (the
outbound/forwarding configuration).

## When you add entries to this page

| Scenario | What to add |
|---|---|
| On-prem mail server submits outbound via Hermes | The mail server's LAN IP or `/32` CIDR |
| Multifunction printer with scan-to-email | The printer's IP |
| Backup MTA / monitoring system that sends alerts | The host's IP |
| Branch-office router doing NAT for relay clients | The router's public `/32` |
| Microsoft 365 sending via inbound connector to Hermes | M365 outbound SMTP source ranges (large, vendor-published) |
| Application server with a built-in mailer | The app server's IP |

If the source authenticates via SMTP AUTH (a Relay Recipient with a
password), it does **not** need to be listed here — `permit_sasl_authenticated`
covers it via the credential path.

## What `mynetworks` controls — the open-relay risk

```
inbound SMTP (25/587)
        |
        v
hermes_postfix_dkim  (smtpd_recipient_restrictions)
        |
        |  permit_mynetworks                     <-- bypasses all checks below
        |  permit_sasl_authenticated             <-- bypasses checks for authenticated senders
        |  reject_unauth_destination             <-- rejects everything else
        |  reject_unauth_pipelining
        |  check_sender_access mysql:...
        |  reject_*_hostname / reject_*_sender   <-- RBL + hygiene checks
        |  check_policy_service unix:.../policy-spf
        |
        v
accept -> amavis content filter (10024)
```

Any IP listed in `mynetworks` clears `permit_mynetworks` and skips
**every other restriction** — RBL lookups, sender domain checks, SPF,
recipient domain checks. The same IP also clears Amavis's `@inet_acl`
because the file `/etc/amavis/mynetworks` is regenerated from the
identical list on every Apply.

> **By design.** Listing an IP here gives the host **unrestricted
> relay** through the gateway. Add only IPs you control or fully
> trust. A broad CIDR (anything wider than `/24`) is a red flag.
> A wildcard entry like `0.0.0.0/0` makes Hermes an open relay
> reachable from the public Internet — the page does not block such
> entries but the operational consequence is immediate inclusion on
> blocklists. Audit periodically.

## Hardcoded baseline — what's already trusted

Two entries are seeded into the `parameters` table at install time and
are intentionally hidden from this page's table (excluded by
`AND parameter <> '127.0.0.1' AND parameter <> '172.16.32.0/24'` in
`get_relay_networks.cfm`):

| Entry | Source | Purpose |
|---|---|---|
| `127.0.0.1` | `hermes_install.sql` seed (`parameters.id=357`) | Localhost — Hermes's own internal Postfix submission, Amavis re-injection on `10025`, scheduler cron jobs, etc. |
| `172.16.32.0/24` | `hermes_install.sql` seed (`parameters.id=434`) | Default Docker subnet — covers every other Hermes container (CommandBox, OpenLDAP, Authelia, body milter, etc.) talking to Postfix |

These are mandatory for normal operation and the page deliberately
hides them so they cannot be deleted from the UI. Removing either
breaks intra-container submission immediately.

> **Operational consequence.** The Docker subnet is hardcoded to
> `172.16.32.0/24` in the seed row above and in the `IPV4SUBNET=172.16.32`
> entry in `.env`. Changing the subnet requires editing both
> the seed row and `.env` plus a sweep of other config files that
> reference the same literal (Postfix, Amavis, Dovecot, Ciphermail,
> OpenDKIM/OpenDMARC, CFML queries). A future change will template
> this — for now, leave the subnet at the default unless you have
> a specific routing reason to change it.

## Configuration storage — the dual-row pattern

Relay networks live in the **`parameters`** table using the standard
parent-child layout shared by every Postfix directive Hermes manages:

| Row | `parameter` column | `child` | `parent_name` | Purpose |
|---|---|---|---|---|
| Parent (one per directive) | `mynetworks` | `2` | NULL | The directive itself; carries `enabled` and the original description |
| Child (one per IP/network) | the actual IP or CIDR (e.g. `192.168.50.0/24`) | `1` | `mynetworks` | The value Postfix sees in the comma-separated list |

The page reads the parent ID from the parent row (`get_mynetworks_parent`)
and uses it as the `parent` foreign key on every child row.
`generate_postfix_configuration.cfm` walks all enabled children of the
parent in `order1` order and emits them comma-separated into
`/etc/postfix/main.cf`.

Extra columns on the child row drive the page's UX:

| Column | Values | Used for |
|---|---|---|
| `network_entry` | `0` / `1` | `1` when the entry has a `/` (CIDR); `0` for single IPs. Drives the **Network** / **IP** badge in the table. |
| `note` | free text | Optional admin label (e.g. "Office Printer", "Branch Office VPN"). Plain-text, HTML-encoded on render. |
| `enabled` | `0` / `1` | Always `1` in normal use; rows are deleted rather than disabled. |
| `applied` | `1` / `2` | `1` = currently live in `main.cf`; `2` = staged change, not yet applied. |
| `action` | `NONE` / `insert` / `delete` / `APPLY` | What the next Apply Settings cycle will do with this row. |
| `order1` | integer | Sort order. New rows append at `MAX(order1) + 1` so existing ordering is preserved. |

## Staged-edit model — pending changes don't take effect immediately

Unlike most pages in the admin console (which save directly), Relay
Networks uses a **two-step commit**: edits are staged in the DB with
`applied=2`, then a single **Apply Settings** click flushes everything
to Postfix in one cascade.

```
add / edit / delete  ──► row marked applied=2 + action={insert|delete|APPLY}
                                    │
                                    v
                            Pending Changes banner appears
                                    │
                                    v
                          Apply Settings (action=apply)
                                    │
                                    ├─ DELETE rows with action='delete'
                                    ├─ UPDATE applied=1, action='NONE' for inserts
                                    ├─ UPDATE applied=1, action='NONE' for edits
                                    │
                                    v
                       generate_postfix_configuration.cfm
                                    │
                                    ├─ rewrite /etc/postfix/main.cf from template
                                    ├─ rewrite /etc/amavis/mynetworks
                                    ├─ docker exec hermes_postfix_dkim postfix reload
                                    └─ docker exec hermes_mail_filter /etc/init.d/amavis force-reload
```

This is intentional. A relay-networks change is a security-sensitive
event — staging lets you queue several edits, eyeball the **Pending
Additions** / **Pending Deletions** / **Pending Edits** cards (each
shown only when its respective query returns rows), then commit in a
single reload. **Cancel All Additions** and **Cancel All Deletions**
buttons let you back out a pending change before applying.

## Bulk-add textarea — format and validation

The Add IP/Network card takes a multi-line textarea. Each non-blank
line is parsed independently and either accepted or appended to a
`skipped` summary that surfaces in the success/error alert.

Format per line:

```
<IP or CIDR> [optional note]
```

| Example input line | Result |
|---|---|
| `192.168.1.100 Office Printer` | IP `192.168.1.100`, note `Office Printer` |
| `192.168.1.101` | IP `192.168.1.101`, note `192.168.1.101` (defaults to the address) |
| `10.0.0.0/24 Server Network` | CIDR `10.0.0.0/24`, note `Server Network` |
| `192.168.1.300` | Skipped — fails IPv4 octet range check |
| `10.0.0.0/45` | Skipped — CIDR out of 1–32 range |

Validation rules in `view_relay_networks.cfm`:

| Check | Pattern | Failure |
|---|---|---|
| IPv4 octets | `^(25[0-5]\|2[0-4][0-9]\|[01]?[0-9][0-9]?)\.{3}…` | `Invalid IP address` / `Invalid network address` |
| CIDR mask | Integer 1–32 | `Invalid CIDR mask` |
| Octet normalization | `Int(octet)` on each | `192.168.001.005` becomes `192.168.1.5` so duplicates can't sneak in via leading zeros |
| Duplicate check | `SELECT … WHERE parameter = ? AND parent = mynetworks_parent_id AND child = '1'` | `Already exists` (skipped silently in bulk) |

IPv6 is **not** supported by this page — the validator pattern only
accepts dotted-quad IPv4. If you need IPv6 relay sources, add them
directly to `parameters` with the same column layout and run a manual
Apply through the UI.

## Single-row Edit modal

The Edit pencil opens a Bootstrap modal pre-filled with the row's
current IP/Network and note. Two edit modes:

| Change | Behavior |
|---|---|
| **Note only** changed | Updates the `note` column immediately (no config change) — success banner only, no Apply required |
| **IP/Network** changed | Sets `applied=2, action='APPLY'`; Apply Settings is required to push to Postfix |

The IP duplicate check (`AND id <> form.edit_id`) lets you edit a row
to itself (no-op) but blocks renaming to another row's value.

## Bulk delete

The DataTables checkbox column lets you select multiple rows and stage
them all for deletion in one shot. Submission goes through the same
`bulk_delete` action — each selected row is marked `applied=2, action='delete'`,
the **Pending Deletions** card appears, and Apply Settings purges them.

A confirm dialog (`Are you sure you want to delete N selected entries?`)
fires before the form submits.

## How a saved network reaches Postfix and Amavis

`generate_postfix_configuration.cfm` is the same template-render +
postfix-reload helper shared by [Relay Host](https://docs.deeztek.com/books/administrator-guide/page/relay-host),
[Domains](https://docs.deeztek.com/books/administrator-guide/page/domains), and other Postfix-directive pages. For
`mynetworks` specifically:

```
1. Substitute every enabled parameters child into the main.cf template
   (mynetworks line becomes "mynetworks = 127.0.0.1, 172.16.32.0/24,
   <every IP/CIDR you added>")
2. cffile write /etc/amavis/mynetworks  -- one entry per line
3. docker exec hermes_postfix_dkim postfix reload
4. docker exec hermes_mail_filter /etc/init.d/amavis force-reload
```

Both Postfix and Amavis trust the same list, so a relay source bypassing
SMTP-time checks also bypasses content-filter network checks.

## Failure semantics

| What breaks | What happens |
|---|---|
| Textarea empty | `session.m = 30`, redirect, no DB write |
| All entries fail validation | `session.m = 32`, redirect, summary of skipped entries shown |
| Mixed: some valid, some invalid | `session.m = 31`, success count + skipped count + collapsible error list |
| Edit IP changed but duplicate of another row | `session.m = 23`, redirect with the conflicting value surfaced |
| Bulk delete with no rows checked | `session.m = 16`, redirect |
| Apply Settings runs but `postfix reload` fails | `session.m = 20` still fires (the page treats reload as best-effort); inspect `docker logs hermes_postfix_dkim` for the error. Previous `main.cf` is preserved in `main.cf.HERMES.BACKUP`. |
| Apply Settings runs but `amavis force-reload` fails | `generate_postfix_configuration.cfm` aborts with the error surfaced via `error.cfm`; Postfix has already been reloaded, so SMTP-time trust is updated but Amavis is still on the previous list. Re-run Apply to recover. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_relay_networks.cfm` | `hermes_commandbox` | Page + bulk-add / edit / delete handlers |
| `config/hermes/var/www/html/admin/2/inc/get_relay_networks.cfm` | `hermes_commandbox` | Load queries (active + pending splits) |
| `config/hermes/var/www/html/admin/2/inc/generate_postfix_configuration.cfm` | `hermes_commandbox` | Template-to-`main.cf` renderer + amavis `mynetworks` writer + reload calls |
| `/etc/postfix/main.cf` | `hermes_postfix_dkim` (volume-mounted) | Live Postfix config; the `mynetworks = …` line is rewritten on every Apply |
| `/etc/postfix/main.cf.HERMES.BACKUP` | `hermes_postfix_dkim` | Pre-regen backup |
| `/etc/amavis/mynetworks` | `hermes_mail_filter` (volume-mounted) | One entry per line; `@inet_acl` source |
| `parameters` row `mynetworks` (child=2, id=3) + N children (child=1, parent=3) | `hermes_db_server` | Directive parent + per-entry children |

Every shell-out uses `docker exec hermes_postfix_dkim …` /
`docker exec hermes_mail_filter …` per the standard Hermes pattern.

## Related

- [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) — the recipient-validation
  list. Together they answer "which sources are trusted to relay
  (this page) and which destinations does Hermes accept inbound mail
  for (Relay Recipients)?"
- [Relay Host](https://docs.deeztek.com/books/administrator-guide/page/relay-host) — outbound smarthost. A client trusted
  by this page that sends outbound mail still flows through the relay
  host (if configured) on the way out.
- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) — inbound relay-domain definitions. Domain
  recipient-validation mode (`OK` / `SPECIFIED`) interacts with
  Relay Recipients but is independent of this page.
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — alternative
  trust path. A RemoteAuth-mode Relay Recipient authenticates against
  an upstream AD/LDAP and is admitted via `permit_sasl_authenticated`,
  not `permit_mynetworks` — adding their source IP here is
  unnecessary (and weakens the audit trail).
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)
  — broader picture of how SMTP AUTH, mynetworks, and the
  `smtpd_recipient_restrictions` chain interact.

# Relay Recipients

# Relay Recipients

Admin path: **Email Relay > Relay Recipients**
(`view_internal_recipients.cfm`, `add_internal_recipients.cfm`,
`edit_internal_recipient_backend.cfm`, `inc/delete_internal_recipients.cfm`,
`inc/edit_internal_recipients.cfm`, `inc/edit_internal_recipients_djigzo.cfm`,
`inc/get_int_recipient_json.cfm`, `inc/send_recipient_welcome_email.cfm`,
`inc/send_recipient_welcome_email_remoteauth.cfm`).

> **The page filename is `view_internal_recipients.cfm`, not
> `view_relay_recipients.cfm`.** The original concept was "internal"
> recipients (mail accepted into the gateway and forwarded to an
> internal backend); the UI label was renamed to **Relay Recipients**
> in commit `c547fdd9` but the filename, table column
> `recipients.recipient_type='relay'`, and several handler names still
> carry the legacy `internal_recipients` naming. Treat the two terms
> as synonymous.

This page manages the **per-address recipient roster** for relay-mode
domains — the list of mailboxes Hermes accepts inbound mail for and
forwards downstream, and the list of authenticated senders that can
relay outbound mail through the gateway. Each row in the `recipients`
table is one email address with a stack of per-recipient settings:
SVF policy, quarantine notifications, encryption flags (PDF/S/MIME/PGP),
S/MIME certificate + PGP keyring slots, backend override, auth mode
(local vs RemoteAuth), and 2FA enforcement.

This is the **recipient-validation** half of the relay topology. Pairs
with [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) (the domains those recipients live under),
[Relay Networks](https://docs.deeztek.com/books/administrator-guide/page/relay-networks) (the trusted source IPs), and
[Virtual Recipients](https://docs.deeztek.com/books/administrator-guide/page/virtual-recipients) (alias-only addresses
that forward without a real account).

## Relay Recipient vs Virtual Recipient vs Mailbox

Three different recipient concepts share the email-address namespace
in Hermes — keep them straight:

| Concept | Stored in | Has a local account? | Delivered to |
|---|---|---|---|
| **Relay Recipient** (this page) | `recipients` where `recipient_type='relay'`, `domain IS NULL` | Yes — LDAP entry + optional app passwords | Downstream MX (per `domains` row's `transport`) |
| **Virtual Recipient** | `virtual_recipients` | No — alias only | Rewrites to another address, which then needs a Relay Recipient or external destination |
| **Mailbox** | `mailboxes` (separate `mailbox_domains` topology) | Yes — Dovecot mailbox | Local Dovecot LMTP at `/mnt/vmail` |

A Relay Recipient is the only one of the three that authenticates for
outbound submission (SMTP AUTH on port 587) and for web/portal login
(via Authelia). Virtual Recipients are pure forwarding rules; Mailboxes
are the mail-server-topology equivalent. See [Email Server > Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes)
for the Mailbox flow.

## What a Relay Recipient row carries

```
recipients table  (one row per email address)
├── recipient                 jsmith@company.com
├── recipient_type            'relay'
├── domain                    NULL   (domain rows use domain='1')
├── auth_type                 'local' | 'remote'
├── remoteauth_domain         NULL if local; mapping key if remote
├── enforce_mfa               0 | 1   (admin policy — see #225 Phase 2)
├── policy_id  ─────────────► spam_policies.policy_id (SVF policy)
├── pdf_enabled / smime_enabled / pgp_enabled / digital_sign
├── backend_server / backend_port / backend_tls   (per-recipient override)
└── (cert+keyring slots populated lazily by the queue)
```

Side tables linked at create/edit time:

| Table | What it stores |
|---|---|
| `user_settings` | Per-user portal toggles (`report_enabled`, `train_bayes`, `download_msg`), `ldap_username`, mailbox flags |
| `recipient_certificates` | S/MIME certs issued for the recipient (lazy — populated by `cert_generation_queue`) |
| `recipient_keystores` | PGP keyrings (lazy — same queue) |
| `app_passwords` | Per-application passwords (Argon2-hashed) for IMAP/SMTP/CalDAV/CardDAV/Nextcloud — see [Credential Model](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings) |
| `wblist` | Whitelist/blacklist entries owned by the recipient |
| `cert_generation_queue` | Pending S/MIME and PGP generation jobs |

## Add Recipient(s) — `add_internal_recipients.cfm`

The Add Recipient(s) button navigates to a multi-line input form that
creates many recipients in one submission. Three add modes:

### Local-auth bulk add — one email per line

When **Auth Type** is `Local` (the default), the textarea takes one
email per line. The page generates a random password for each new
recipient, sends a welcome email via `send_recipient_welcome_email.cfm`
that includes a **first-login password-reset link**, and stores the
LDAP entry with a placeholder `userPassword` that will be overwritten
when the user follows the link.

```
jsmith@company.com
jdoe@company.com
bob.smith@company.com
```

### RemoteAuth bulk add — same line format

When **Auth Type** is `Remote` and the selected mapping's DN pattern
only uses `{username}` and/or `{email}`, the textarea is still one
email per line. No password is generated — the recipient authenticates
against the upstream LDAP/AD via the `remoteauth` overlay (see
[LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth)). The welcome email
goes through `send_recipient_welcome_email_remoteauth.cfm` and tells
the user to sign in with their **organization password**, not a
Hermes-issued one.

### RemoteAuth CSV add — `First,Last,Email` per line

When the RemoteAuth mapping's DN pattern uses `{firstname}` or
`{lastname}` (typical for AD `cn=` patterns), the textarea **switches
to CSV mode** because email-only input doesn't carry enough data to
expand the pattern. Header rows (`"GivenName","Surname","Mail"`) are
auto-detected and skipped, and unknown columns are ignored.

| Source | Command / file shape |
|---|---|
| **PowerShell** | `Get-ADUser -Filter * -Properties GivenName,Surname,Mail \| Select GivenName,Surname,Mail \| Export-Csv users.csv -NoTypeInformation` |
| **CSVDE** (Windows Server built-in) | `csvde -f users.csv -l "givenName,sn,mail"` |
| **Excel / manual** | Three columns saved as CSV |

See [LDAP RemoteAuth § Adding RemoteAuth users in bulk](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth#adding-remoteauth-users-in-bulk--csv-format)
for the full CSV format reference.

The Add form also accepts the same per-recipient stack of options as
the Edit Options modal (SVF policy, quarantine notifications, etc.) —
those defaults are written to every new row in one shot.

## The Recipients table

Sortable, searchable, exportable (copy/CSV/Excel/PDF/print via
DataTables Buttons; `stateSave: true`). Columns:

| Column | Source | Notes |
|---|---|---|
| Checkbox | — | Multi-select for the action buttons above the table |
| S/MIME | link to `view_recipient_certificates.cfm?type=1&id=…` | Per-recipient cert manager |
| PGP | link to `view_recipient_keyrings.cfm?type=1&id=…` | Per-recipient keyring manager |
| Recipient | `recipients.recipient` | Email address |
| Auth | `recipients.auth_type` + `remoteauth_domain` | `LOCAL` badge (secondary) or `REMOTE` badge (primary, tooltip shows mapping key) |
| Backend | `recipients.backend_server[:port]` | Per-recipient override or `(domain default)` placeholder |
| 2FA | LDAP `cn=two_factor` + `enforce_mfa` | **Two independent pills** — see [Two-pill 2FA column](#two-pill-2fa-column) below |
| Policy | `policy.policy_name` via join | Assigned SVF policy |
| Quarantine Notifications | `user_settings.report_enabled` | `YES` / `NO` badge |
| Train Bayes | `user_settings.train_bayes` | `YES` / `NO` |
| Download Msgs | `user_settings.download_msg` | `YES` / `NO` |
| PDF / S/MIME / PGP Encrypt | per-row encryption flags | `YES` / `NO` badges |
| Sign All | `recipients.digital_sign` | `YES` / `NO` |
| S/MIME Cert | join against `recipient_certificates` | `YES` (green badge) if a cert exists |
| PGP Keyring | join against `recipient_keystores` | `YES` (green badge) if a keyring exists |

The query filters `WHERE recipients.domain IS NULL AND (recipient_type = 'relay' OR recipient_type IS NULL)`
so only relay-mode rows appear — mailbox-topology rows (with
`recipient_type='mailbox'`) are managed under
[Email Server > Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes).

## Two-pill 2FA column

The 2FA column shows **two orthogonal states** as independent pills,
because admin enforcement and user enrollment are decoupled (#225
Phase 1.5 + Phase 2):

| Pill | Source | Means |
|---|---|---|
| **Enrolled** (success badge) | LDAP `cn=two_factor` group membership | The user has registered a 2FA device (TOTP, security key, or Duo Push) and Authelia challenges them at sign-in |
| **Required** (warning badge) | `recipients.enforce_mfa = 1` | Admin policy demands 2FA. The recipient sees an urgent banner in the user portal directing them to Account Settings until they enroll |

| Enrolled | Required | What it looks like | Means |
|---|---|---|---|
| no | no | em-dash | Default state. No 2FA. |
| yes | no | Enrolled only | Voluntary enrollment. User opted in; admin doesn't enforce. |
| no | yes | Required only | Admin set the policy; user hasn't yet registered a device. |
| yes | yes | Both pills | Required and complied with. |

The single LDAP `ldapsearch` query against `cn=two_factor,ou=groups,dc=hermes,dc=local`
runs once per page render, then each row checks for its DN substring
in the result — avoids N+1 LDAP roundtrips.

## Bulk action buttons

| Button | Action | Selection requirement |
|---|---|---|
| **Create Recipient(s)** | Navigates to `add_internal_recipients.cfm` | — |
| **Edit Options** | Opens the Edit Options modal | At least one row |
| **Edit Encryption** | Opens the Edit Encryption modal | At least one row |
| **Edit Backend** | Navigates to `edit_internal_recipient_backend.cfm?ids=…` | At least one row |
| **Reset 2FA Devices** | Opens the Reset 2FA Devices modal | At least one row |
| **Delete** | Opens the delete-confirm modal | At least one row |

Selecting zero rows and clicking any of the edit/delete buttons
surfaces an alert (`Please select at least one recipient`) instead of
opening the modal.

## Edit Options modal — AJAX pre-fill vs bulk-edit warning

The Edit Options modal handles SVF policy, quarantine notifications,
Train Bayes, Download Messages, and 2FA enforcement
(`enforce_mfa`). It has **two modes**, selected by the JS based on
how many rows are checked:

### Single-select: AJAX pre-fill

When exactly one row is checked, the JS calls
`./inc/get_int_recipient_json.cfm?id=<rid>` over POST and hydrates
every form field with that recipient's current values before opening
the modal. The admin sees the recipient's actual policy, current
notification mode, current `enforce_mfa` state, etc. — submit edits
only what changed.

### Multi-select: bulk-edit warning

When 2+ rows are checked, the modal shows a prominent red
**Bulk edit — N recipients selected** alert at the top:

> The fields below are **not pre-filled from each recipient's current
> settings** — they show the form's default values. Submitting will
> **OVERWRITE every field on every selected recipient** with whatever
> you see now.

The 2FA-specific footnote then warns that leaving the Two-Factor
Authentication dropdown at `Disable` will reset every selected
recipient's `enforce_mfa` to `0` — but **the user is not removed from
`cn=two_factor` automatically** (the LDAP cascade only fires on
0→1 transitions). To strip an existing enrollment, the admin must use
the Reset 2FA Devices modal with the nuclear-option checkbox.

This is intentional — the bulk-edit form has been a foot-gun in the
past (admins thinking "Disable" only changed the one row), so the
warning is unmissable. The recommended pattern: **edit a single
recipient with their current values pre-filled, select only one
row.**

## Edit Encryption modal

Handles `pdf_enabled`, `smime_enabled`, `digital_sign`, `pgp_enabled`,
and the cert/keyring generation parameters (CA, validity, key size,
algorithm, PGP key length). Submit triggers
`edit_internal_recipients_djigzo.cfm` which updates the row and **queues
async S/MIME cert + PGP keyring generation** into `cert_generation_queue`
if the flags flip on and no existing cert/keyring is present.

The page renders a **Background Generation in Progress** info banner
while `cert_generation_queue` has any `pending` or `processing` rows,
and a **Generation Failures** warning with a **Retry Failed Jobs**
button if any rows are in `failed` state. The Retry button updates
matching rows to `status='pending', error_message=NULL, started_at=NULL`
so the next scheduler tick re-attempts them.

## Edit Backend page

Per-recipient override of the downstream backend server / port / TLS
mode. The default is `NULL` on all three columns, which falls back to
the parent domain's `transport` row (set on the [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains)
page). Useful for routing specific recipients to a different MX —
e.g., a single user whose mailbox is on a different server than the
rest of the domain.

The Backend column on the main table shows the override host (and
port via tooltip) or `(domain default)` for the fallback case.

## Reset 2FA Devices modal

Replaces the older "Recipient Access Control" modal as of #225 Phase 2.
The one_factor/two_factor radio is gone — the canonical admin policy is
the **Two-Factor Authentication** select on Edit Options. This modal
is now single-purpose: clear Authelia TOTP/WebAuthn devices for the
selected recipients via `docker exec hermes_authelia authelia storage user totp/webauthn delete`.

Two modes:

| Mode | What it does |
|---|---|
| **Default** | Deletes TOTP + WebAuthn device registrations in Authelia. User stays under 2FA enforcement and re-registers on next sign-in. "User lost their phone" recovery. |
| **Nuclear** (checkbox) | Also moves the user from `cn=two_factor` back to `cn=one_factor`. Admin override of voluntary enrollment, or full account reset. |

> **Does not affect Duo Push.** Duo enrollments live on Duo's cloud
> servers, not in Authelia's database. Use the Duo Admin Console for
> Duo device management.

> **Cascade interaction.** If the per-recipient `enforce_mfa` policy
> in Edit Options is still `Enable`, the nuclear option's removal from
> `cn=two_factor` will be **reversed** on the next save of the Edit
> Options modal (the 0→1 LDAP cascade fires again). To truly
> de-enforce, set `enforce_mfa = Disable` first.

## Delete

The Delete modal confirms the irreversible action. The
`delete_internal_recipients.cfm` handler then runs an unusually-long
cleanup sequence per recipient — the kind of cascade that makes orphan
rows the rule when CFML deletes are skimped:

```
For each selected recipient ID:
1. Look up ldap_username via user_settings join
2. docker exec hermes_authelia authelia storage user totp delete <user>
3. docker exec hermes_authelia authelia storage user webauthn delete <user> --all
4. ldap_delete_user_relay.cfm — remove LDAP stub entry + group memberships
5. Cancel any pending password_reset_requests rows for this email
6. DELETE FROM recipients WHERE id = <rid>
7. DELETE FROM recipients_temp WHERE recipient = <email>
8. DELETE FROM wblist WHERE rid = <rid>
9. DELETE FROM user_settings WHERE email = <email>
10. DELETE FROM mailaddr (and wblist by sid) for the address
11. Delete recipient_certificates + cm_keystore from djigzo
12. (caller continues with the next ID)
```

Steps 2–3 prevent a re-created recipient at the same email from
silently inheriting the prior owner's TOTP/WebAuthn enrollments.
Failures inside `cftry` blocks are non-fatal — the desired end-state
("no devices") is achieved whether or not the user had anything
enrolled in the first place.

> **Known gap (#102).** When a Relay Recipient with `auth_type='remote'`
> is deleted, the deletion of the LDAP stub entry happens, but the
> RemoteAuth domain-mapping deletion validation in
> `view_remoteauth.cfm` / `edit_remoteauth_mapping.cfm` does **not**
> check the `mailboxes` table yet (it only checks `system_users` and
> `recipients`). When RemoteAuth is wired to mailboxes, that
> validation must add a third query. Not a bug today — relay
> recipients are correctly covered — but a forward-looking
> integration point. See [LDAP RemoteAuth § Deletion validation](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth#deletion-validation).

## Local-auth vs RemoteAuth — the credential split

| Aspect | `auth_type = 'local'` | `auth_type = 'remote'` |
|---|---|---|
| Web portal sign-in | Hermes LDAP `userPassword` (user sets via reset link) | Upstream AD/LDAP via overlay; Hermes never sees the password |
| IMAP / SMTP / CalDAV / CardDAV / NC | `app_passwords` row (Argon2-hashed in Hermes DB) | Same — `app_passwords` row in Hermes DB |
| Password rotation on the upstream | N/A | Web sign-in immediately picks up the new password; existing app passwords keep working until explicitly revoked |
| Welcome email | "Click here to set your password" | "Sign in with your organization (AD/LDAP) password" |

App passwords are **always Hermes-issued**, regardless of `auth_type`.
The upstream directory password is exposed only to the web gate via
the LDAP overlay's pass-through bind — never to Dovecot or Nextcloud.
See [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)
for the full four-credential architecture and
[LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) for the upstream
binding details.

## Recipient validation in Postfix

The `recipients` table is queried by Postfix at SMTP time via
`mysql:/etc/postfix/mysql-recipients.cf` (mapped to
`relay_recipient_maps` in `main.cf`). When a [Domain](https://docs.deeztek.com/books/administrator-guide/page/domains) has
Recipient Delivery set to `SPECIFIED`, mail arriving for an address
**not** in this table is rejected with a `550 User unknown` reply.
When Recipient Delivery is `ANY`, the lookup is bypassed for that
domain and any recipient is accepted (catch-all).

This is the operational reason to add Relay Recipients **before**
flipping a domain to SPECIFIED — flipping first will start rejecting
live mail.

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_internal_recipients.cfm` | `hermes_commandbox` | Main page + Edit Options / Edit Encryption / Reset 2FA / Delete modals |
| `config/hermes/var/www/html/admin/2/add_internal_recipients.cfm` | `hermes_commandbox` | Bulk-add page (local + RemoteAuth + CSV modes) |
| `config/hermes/var/www/html/admin/2/edit_internal_recipient_backend.cfm` | `hermes_commandbox` | Per-recipient backend override page |
| `config/hermes/var/www/html/admin/2/inc/get_int_recipient_json.cfm` | `hermes_commandbox` | AJAX hydrator for single-select Edit Options pre-fill |
| `config/hermes/var/www/html/admin/2/inc/edit_internal_recipients.cfm` | `hermes_commandbox` | Edit Options handler (+ LDAP cascade on `enforce_mfa` 0→1) |
| `config/hermes/var/www/html/admin/2/inc/edit_internal_recipients_djigzo.cfm` | `hermes_commandbox` | Edit Encryption handler + cert/keyring queue insertion |
| `config/hermes/var/www/html/admin/2/inc/delete_internal_recipients.cfm` | `hermes_commandbox` | Per-recipient delete cascade |
| `config/hermes/var/www/html/admin/2/inc/send_recipient_welcome_email.cfm` | `hermes_commandbox` | Local-auth welcome email (password-reset link) |
| `config/hermes/var/www/html/admin/2/inc/send_recipient_welcome_email_remoteauth.cfm` | `hermes_commandbox` | RemoteAuth welcome email (org-password sign-in) |
| `config/hermes/var/www/html/admin/2/inc/ldap_add_user_relay.cfm` / `ldap_add_user_relay_remoteauth.cfm` | `hermes_commandbox` | LDAP stub creation for local / remote auth |
| `config/hermes/var/www/html/admin/2/inc/ldap_delete_user_relay.cfm` | `hermes_commandbox` | LDAP stub removal on delete |
| `config/hermes/var/www/html/admin/2/inc/ldap_change_user_access_control.cfm` | `hermes_commandbox` | Group membership swap (one_factor ⇄ two_factor) |
| `recipients`, `user_settings`, `app_passwords`, `recipient_certificates`, `recipient_keystores`, `cert_generation_queue`, `wblist`, `mailaddr`, `password_reset_requests`, `recipients_temp` | `hermes_db_server` | The recipient-row group + lazy-generation queue |
| `cn=<user>,ou=users,dc=hermes,dc=local` | `hermes_ldap` | Per-recipient LDAP entry |
| `cn=relays,ou=groups,dc=hermes,dc=local` | `hermes_ldap` | Relay-recipient group membership |
| Authelia `totp_configurations` + `webauthn_devices` | `hermes_authelia` storage backend | Cleaned on delete + Reset 2FA Devices |
| `/etc/postfix/mysql-recipients.cf` | `hermes_postfix_dkim` | Postfix lookup against `recipients` for `relay_recipient_maps` |

Every shell-out uses `docker exec …` per the standard Hermes pattern.

## Related

- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) — relay-domain definitions. Required parent
  context: a recipient is meaningless without a domain that accepts
  mail for it. Domain Recipient Delivery `SPECIFIED` is what makes
  this page's roster authoritative for inbound acceptance.
- [Relay Networks](https://docs.deeztek.com/books/administrator-guide/page/relay-networks) — trusted source IPs. The
  alternative trust path: a source IP listed there can submit
  outbound without authenticating as a recipient on this page.
- [Virtual Recipients](https://docs.deeztek.com/books/administrator-guide/page/virtual-recipients) — alias-only addresses
  that forward to a Relay Recipient or external destination. A
  Virtual Recipient pointing at a deleted Relay Recipient becomes a
  forwarding hole.
- [Relay Host](https://docs.deeztek.com/books/administrator-guide/page/relay-host) — outbound smarthost. A Relay Recipient
  that SMTP-AUTHs to send outbound mail still flows through the
  relay host (if configured) on the way to the Internet.
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — required
  prerequisite for `auth_type='remote'` recipients. Defines the
  upstream LDAP/AD mappings this page references via `remoteauth_domain`.
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)
  — full four-credential architecture (web vs IMAP/SMTP vs DAV vs
  Nextcloud) that recipient app passwords slot into.
- [Email Server > Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — the
  mail-server-topology equivalent. Don't confuse Relay Recipients
  (forwarded downstream) with Mailboxes (delivered locally to Dovecot).

# Virtual Recipients

# Virtual Recipients

Admin path: **Email Relay > Virtual Recipients** (`view_virtual_recipients.cfm`,
`inc/addvirtualrecipients.cfm`, `inc/editvirtualrecipient.cfm`,
`inc/delete_virtual_recipients.cfm`).

This page manages **forward-only address aliases** on the relay-topology
domains configured under [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains). Each row in the
`virtual_recipients` table maps one inbound address (or a domain-wide
catch-all) to exactly one delivery address. The delivery target can be
internal to Hermes, on another relay domain, on a mailbox domain, or
anywhere on the public Internet — the row is consumed by Postfix's
`virtual_alias_maps` and rewritten at SMTP time, so the forward is
transparent to the original sender.

Virtual recipients have **no SMTP authentication, no IMAP/POP3 access,
and no password**. They are not user accounts. They are rewrite rules.

## Not the same as Mailbox Aliases

The Email Server topology has its own alias page — [Email Server >
Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases), backed by the `mailbox_aliases`
table — and it serves a different need. The add handler enforces the
separation explicitly: trying to add a virtual recipient for a domain
flagged as `mailbox` is rejected with the "use Email Server > Aliases"
hint.

| | Virtual Recipients | Mailbox Aliases |
|---|---|---|
| Table | `virtual_recipients` | `mailbox_aliases` |
| Domain type | Relay domains (`domains.type = 'relay'` or NULL) | Mailbox domains (`mailbox_domains.*`) |
| Delivery target | Anywhere — internal or external | A local Dovecot mailbox |
| Resolved by | Postfix `virtual_alias_maps` (MySQL lookup) | Postfix `virtual_alias_maps` (same query, different table) |
| Auth, IMAP, password | No | No (the resolved mailbox owns those) |
| Typical use | `info@company.com → admin@company.com, info@externalpartner.example` | `support@company.com → user1@company.com` (where `user1@` is a local mailbox) |

The shared `mysql-virtual.cf` lookup is a `UNION` across both tables:

```sql
SELECT maps          FROM virtual_recipients WHERE virtual_address = '%s'
UNION
SELECT delivers_to   FROM mailbox_aliases    WHERE alias_address   = '%s'
```

Postfix doesn't care which table the answer comes from — but the admin
UI separates them so the rule for each topology stays focused.

## Storage and lookup path

```
inbound SMTP (port 25) ──► hermes_postfix_dkim
                                  │
                                  │  smtpd checks: helo, sender, recipient
                                  │  relay_recipient_maps / recipient_canonical_maps
                                  │  virtual_alias_maps  ◄── mysql:/etc/postfix/mysql-virtual.cf
                                  │                          │
                                  │                          ▼
                                  │      ┌────────────────────────────────────┐
                                  │      │ hermes_db_server                    │
                                  │      │  SELECT maps FROM virtual_recipients│
                                  │      │   UNION                             │
                                  │      │  SELECT delivers_to FROM            │
                                  │      │   mailbox_aliases                   │
                                  │      └────────────────────────────────────┘
                                  │
                                  v
                          rewritten recipient(s)
                                  │
                                  ▼
                       content filter (amavis on 10024)
                                  │
                                  ▼
                       outbound or local delivery
```

No file regeneration is required when virtual recipients change. The
MySQL lookup is live — adding a row in the admin UI takes effect on
the next inbound message, with zero Postfix restart or postmap step.
This is the operational reason virtual aliases are stored in MySQL
rather than a hash file.

## The `virtual_recipients` table

| Column | Type | Role |
|---|---|---|
| `id` | INT PK | Surrogate key for the row |
| `virtual_address` | VARCHAR(255) | The address being rewritten. Full email (`info@example.com`) **or** a catch-all token (`@example.com`). |
| `maps` | VARCHAR(255) | Destination address. Single recipient per row in the current schema. |
| `alias_type` | VARCHAR(20) | Defaults to `forward`. Reserved for future per-alias behavior flags; not surfaced in the UI today. |
| `send_as` | TINYINT(3) | Reserved for outbound "send-as" support (allow the destination to send mail as the virtual address). Not wired through Postfix yet. |
| `policy_id` | INT | Reserved for per-alias Amavis policy attachment. Not surfaced today. |
| `system` | INT | Provenance marker — `1` = seeded by the install/system-addresses flow (postmaster/abuse/root), `2` = admin-created via this page. The system rows are managed by `update_system_email_addresses.cfm` and recreated when the admin email or postmaster changes. |

There is no UNIQUE constraint on `virtual_address` because a single
inbound address can fan out to multiple destinations — each destination
gets its own row. The add handler dedupes on the `(virtual_address,
maps)` pair so the same forward isn't inserted twice.

## Two address shapes — specific and catch-all

### Specific aliases

A regular forward of one address to one destination:

```
info@company.com       →   owner@company.com
sales@company.com      →   sales-team@externalcrm.example
legal@company.com      →   external-counsel@lawfirm.example
```

The local-part is rewritten by Postfix before content filtering. The
recipient never sees the original `info@`/`sales@`/`legal@` address
unless the destination mail system surfaces the original envelope.

### Catch-alls

A single row starting with `@` matches every local-part on the domain
that is **not** already a more specific virtual recipient or a mailbox:

```
@company.com           →   admin@company.com
```

With the catch-all row above, mail to `jdoe@company.com`,
`random-string@company.com`, and `does-not-exist@company.com` all
forward to `admin@company.com`. Specific aliases on the same domain
(`info@company.com → owner@company.com`) win over the catch-all because
they match the more specific lookup key first.

Catch-alls are useful for sunset domains, migration phases, or small
domains where one mailbox owner is willing to receive everything. They
are not appropriate for high-volume domains: every spam attempt against
a random local-part lands in the catch-all destination.

### Catch-all visibility in the user portal

A user whose mailbox is the **destination** of a catch-all (e.g.,
`admin@company.com` above) has a special branch in the user portal's
Quarantined Messages, Total Messages, and Message History queries.
`config/hermes/var/www/html/users/2/index.cfm`,
`view_message.cfm`, and `view_message_history.cfm` all consult
`virtual_recipients` for catch-all entries that explicitly map TO the
logged-in user, then widen the query with a `LIKE '%@domain.tld'`
clause so the user sees the messages that were swept up by the
catch-all. Specific aliases do **not** get this treatment yet — a
known parity gap for the rare case where one user owns many specific
aliases and wants the same widened visibility.

## Fields on the page

### Add Virtual Recipients card

| Field | Notes |
|---|---|
| **Virtual Address(es)** | Newline-delimited textarea. Each line is one full email address or a `@domain.com` catch-all. Lowercased, trimmed, deduped against `virtual_recipients` AND `mailbox_aliases` before insert. |
| **Delivers To** | Single destination address for the whole batch. Validated as an email. Autocomplete sourced from `inc/getintrecipients.cfm` (existing relay recipients and mailbox addresses) so you can typeahead-pick a known recipient. |

The handler iterates the textarea line-by-line and accumulates per-line
results. The success banner reports the count and addresses that landed,
and separate error banners surface invalid-format lines, lines whose
domain isn't configured as a relay domain, lines whose domain is a
mailbox domain (with the "use Email Server > Aliases" pointer), and
duplicate lines. **No transaction wraps the batch** — partial success is
the expected behavior.

### Virtual Recipients table

Standard DataTables surface — searchable, sortable, exportable
(copy / CSV / Excel / PDF / print), `stateSave: true` so column order
and page size persist across reloads. Columns:

| Column | Source |
|---|---|
| Checkbox | Bulk-select for delete |
| Recipient | `virtual_recipients.virtual_address` |
| Delivers To | `virtual_recipients.maps` |
| Actions | Edit (opens modal) |

### Edit modal

Inline edit of `virtual_address` and `maps`. Re-runs the same domain
validation, catch-all detection, and dedupe check as Add — including
the rejection of mailbox-domain rows.

### Delete

Checkbox-driven bulk delete from the table card. The handler
(`delete_virtual_recipients.cfm`) just runs `DELETE FROM virtual_recipients
WHERE id = ?` per selected row — there is no dependency check, because
nothing else in the schema points back at a virtual recipient row.

## Content filter bypass — by design, loud

The yellow callout on the page exists for a reason. Postfix rewrites
the recipient **before** the message reaches Amavis content filtering,
but Amavis policy lookups key on the **post-rewrite** recipient. If the
destination address is an external Internet address (Gmail, Outlook.com,
a personal mailbox, etc.), Amavis applies the default outbound policy
to it — which typically means lighter spam/banned-files enforcement than
a domain-scoped inbound policy would.

The net effect: mail aliased through a virtual recipient to an external
address is generally **less aggressively filtered** than the same mail
delivered to a local mailbox or relayed to a known partner domain.
This is fine for legitimate forwards, but admins who use virtual
recipients to bridge a sunset domain to a personal Gmail should expect
Amavis to be permissive about it. Tighten the policy by editing the
destination recipient's `recipients` row directly under
[Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) if the destination is itself a
known Hermes recipient.

## Domain-delete dependency

Deleting a relay domain via [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) is blocked when virtual
recipients reference it. `deletedomain.cfm` runs:

```sql
SELECT * FROM virtual_recipients WHERE virtual_address LIKE '%<domain>%'
```

Any match aborts the domain delete with error code 2 and the admin must
clear the matching rows from this page before the domain can be removed.
The same back-pressure protects against silently stranding a forward
when its destination domain disappears.

## System-managed rows

A few rows in `virtual_recipients` are created and managed by the
**System > Server Setup** flow, not by this page directly:

| Pattern | Created by |
|---|---|
| `postmaster@<every-domain>` → admin email | `inc/update_system_email_addresses.cfm` on every Server Setup save |
| `root@<every-domain>` → admin email | Same |
| `abuse@<every-domain>` → admin email | Same |

These rows are marked `system = '1'` (the install/system flow) versus
admin-created rows which are marked `system = '2'`. Editing or
deleting a system-managed row from this page works mechanically, but
the row will be recreated on the next Server Setup save. Edit the
admin email there if you want a different destination for these
reserved local-parts; do not maintain them by hand here.

## Failure semantics

| What breaks | What happens |
|---|---|
| Virtual address blank in Add | error 1 banner, no DB write |
| Delivers To blank or invalid email in Add | error 2/3 banner, no DB write |
| Edit virtual address fails email or catch-all format | `session.m = 10`, redirect, no DB write |
| Edit Delivers To blank or invalid | `session.m = 11`/`12`, redirect, no DB write |
| Domain not in `domains` table | `session.m = 13` on edit; per-line invalid-domain banner on add — line skipped, others continue |
| Domain is a mailbox domain | Per-line invalid-domain banner with the "use Email Server > Aliases" hint; line skipped |
| Duplicate `(virtual_address, maps)` pair in `virtual_recipients` or `mailbox_aliases` | Per-line duplicate banner on add; `session.m = 14` on edit |
| Delete with no rows selected | `session.m = 1` banner, no DB write |
| MySQL `hermes_db_server` down | Postfix `virtual_alias_maps` lookups fail. By default Postfix defers mail to the affected recipients with a temporary error and retries on the next queue run; legitimate mail is held, not bounced. |

## Bulk import

The current page supports newline-delimited paste into the Add textarea,
which is the practical bulk path: paste hundreds of `alias@domain.com`
lines (all forwarding to one destination) at once, click Add, get a
per-line outcome report. A separate CSV import is not provided because
the table is intentionally one-destination-per-row — fan-out is
expressed by adding the same `virtual_address` multiple times with
different `maps`, which is easier to do in the textarea than in a CSV.

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_virtual_recipients.cfm` | `hermes_commandbox` | Page + Add card + table + modals |
| `config/hermes/var/www/html/admin/2/inc/addvirtualrecipients.cfm` | `hermes_commandbox` | Add handler with per-line validation |
| `config/hermes/var/www/html/admin/2/inc/editvirtualrecipient.cfm` | `hermes_commandbox` | Edit handler |
| `config/hermes/var/www/html/admin/2/inc/delete_virtual_recipients.cfm` | `hermes_commandbox` | Delete handler (per selected id) |
| `config/hermes/var/www/html/admin/2/inc/getintrecipients.cfm` | `hermes_commandbox` | Autocomplete source for the Delivers To field |
| `config/hermes/var/www/html/admin/2/inc/update_system_email_addresses.cfm` | `hermes_commandbox` | Manages the `system = '1'` rows (postmaster/root/abuse) |
| `/etc/postfix/mysql-virtual.cf` | `hermes_postfix_dkim` (volume-mounted) | Postfix MySQL lookup definition for `virtual_alias_maps` |
| `virtual_recipients`, `mailbox_aliases`, `domains` | `hermes_db_server` | The lookup tables and the domain-type gate |

Nothing on this page shells out to Postfix — there is no postmap, no
`postfix reload`, no template regeneration. The MySQL lookup is the
only integration surface.

## Related

- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) — the relay-topology domain list these aliases
  attach to. Domain deletes are blocked when virtual recipients still
  reference the domain.
- [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) — recipient validation for
  domains with Recipient Delivery = SPECIFIED. A specific relay
  recipient and a virtual recipient can coexist for the same address;
  the relay recipient wins for recipient-list validation, the virtual
  recipient still rewrites at delivery.
- [Email Server > Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases) — the mailbox-
  topology equivalent. Aliases for domains where Hermes is the
  destination MTA live there.
- [Email Server > Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes)
  — when several users need to read the same incoming mail (not just
  one user receiving forwards), use a shared mailbox instead of a
  fan-out virtual recipient.
- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — manages the
  `system = '1'` postmaster/root/abuse forwards. Change the admin email
  there to retarget those reserved local-parts.

# Email Server

# Aliases

# Aliases

Admin path: **Email Server > Aliases** (`view_mailbox_aliases.cfm`,
`inc/add_mailbox_alias_action.cfm`, `inc/edit_mailbox_alias_action.cfm`,
`inc/delete_mailbox_alias_action.cfm`, `inc/get_mailbox_alias_json.cfm`).

This page manages **alternate email addresses for local mailboxes** on
the Email Server topology. Each row in the `mailbox_aliases` table maps
one inbound address (e.g., `sales@company.com`) to either an existing
local mailbox or to Postfix's discard transport for silent disposal.
The destination must be local — to an existing Dovecot mailbox on this
server. For forwarding to external addresses or for relay-topology
domains, use [Email Relay > Virtual Recipients](https://docs.deeztek.com/books/administrator-guide/page/virtual-recipients)
instead.

Aliases have **no SMTP authentication, no IMAP/POP3 access, and no
password of their own**. They are rewrite rules consumed by Postfix
before content filtering. The optional **Send-As** flag adds a row to
`sender_login_maps` so the destination mailbox owner can send mail
under the alias address from their existing IMAP/Submission session.

## Not the same as Virtual Recipients

Email Server aliases and Email Relay virtual recipients share the same
underlying Postfix lookup but enforce different topology rules. See
[Virtual Recipients](https://docs.deeztek.com/books/administrator-guide/page/virtual-recipients) for the
full distinction; the short version:

| | Mailbox Aliases (this page) | Virtual Recipients |
|---|---|---|
| Table | `mailbox_aliases` | `virtual_recipients` |
| Domain type | Mailbox domains (`domains.type = 'mailbox'`) | Relay domains (`domains.type = 'relay'` or NULL) |
| Delivery target | A local Dovecot mailbox, or `discard:silently` | Anywhere — internal or external |
| UNIQUE on address | Yes (one delivery per alias) | No (fan-out via multiple rows) |
| Send-As | Optional, surfaced as a toggle | Schema flag, not yet wired through |
| Catch-all (`@domain`) | Not supported | Supported |
| Discard transport | Supported (silent drop) | Not supported |
| Typical use | `support@company.com → alice@company.com` (both local) | `info@company.com → admin@externalpartner.example` |

Both tables feed the same `virtual_alias_maps` lookup via a single
UNION query in `mysql-virtual.cf`:

```sql
SELECT maps        FROM virtual_recipients WHERE virtual_address = '%s'
UNION
SELECT delivers_to FROM mailbox_aliases    WHERE alias_address   = '%s'
```

The add handlers in each page enforce the topology gate: trying to
create a mailbox alias for a relay domain is rejected with error 12,
and the Virtual Recipients add handler rejects mailbox-domain rows
with a pointer back to this page.

## Storage and lookup path

```
inbound SMTP (port 25) ──► hermes_postfix_dkim
                                  │
                                  │  smtpd: helo, sender, recipient checks
                                  │  virtual_alias_maps  ◄── mysql:/etc/postfix/mysql-virtual.cf
                                  │                          │
                                  │                          ▼
                                  │      ┌──────────────────────────────────┐
                                  │      │ hermes_db_server                  │
                                  │      │  UNION across virtual_recipients  │
                                  │      │   and mailbox_aliases             │
                                  │      └──────────────────────────────────┘
                                  │
                                  ▼
                          rewritten recipient
                                  │
                  ┌───────────────┴────────────────┐
                  │                                │
       forward (delivers_to =          discard (delivers_to =
       a local mailbox username)       'discard:silently')
                  │                                │
                  ▼                                ▼
       amavis (10024)                   discard(8) transport
                  │                                │
                  ▼                                ▼
       LMTP → hermes_dovecot         message silently dropped
       Maildir for target mailbox      no bounce, no DSN, no log entry
                                       beyond the queue acceptance
```

The MySQL lookup is live — adding a row in this page takes effect on
the next inbound message, with no Postfix reload, no `postmap`, and
no template regeneration.

## The `mailbox_aliases` table

| Column | Type | Role |
|---|---|---|
| `id` | INT PK | Surrogate key |
| `alias_address` | VARCHAR(255), **UNIQUE** | The address being rewritten. Full email only — no catch-all syntax. The UNIQUE constraint enforces one delivery target per alias address. |
| `delivers_to` | VARCHAR(255) | Destination. For `alias_type = 'forward'` this is the local mailbox username; for `alias_type = 'discard'` this is hardcoded to the literal string `discard:silently`, which Postfix routes through the discard(8) transport. |
| `alias_type` | VARCHAR(20) | `forward` (default) or `discard` |
| `send_as` | TINYINT(3) | `1` if the destination mailbox is allowed to send mail as the alias address. Wired into `sender_login_maps` on insert/update. |
| `domain_id` | INT | FK to `domains.id`; set on insert from the parsed domain part of `alias_address`. Used to filter the page by domain and to enforce the mailbox-topology gate. |
| `created_at` | DATETIME | Audit timestamp |

The UNIQUE key on `alias_address` is the reason fan-out isn't supported
here — one inbound address resolves to exactly one destination. To
deliver one inbound address to several mailboxes, use a
[shared mailbox](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes) (which gives multiple users
access to a single inbox) or, for true fan-out, use the relay topology
with virtual recipients.

## The two alias types

### Forward

Delivers mail to an existing local mailbox. The mailbox must exist in
the `mailboxes` table — the add handler verifies this with `error 16`
on failure. The `Delivers To` dropdown is sourced from the live
mailbox list (`mailbox_type = 'user'`), so you can only pick a real
target.

```
sales@company.com    →   alice@company.com
support@company.com  →   helpdesk@company.com
```

Both addresses must be on a mailbox domain that this server hosts.
Cross-domain forwards are allowed as long as both sides are local
mailbox domains.

### Discard

Silently drops all mail with no bounce, no DSN, and no error returned
to the sender. The handler hardcodes `delivers_to = 'discard:silently'`,
which Postfix interprets as the discard(8) transport with the literal
nexthop `silently`. Useful for addresses like `noreply@` or
`donotreply@` where bounces would invite spam-mining attempts.

```
noreply@company.com      →   discarded
donotreply@company.com   →   discarded
unsubscribe@company.com  →   discarded
```

> **Operational consequence.** Discard is irrecoverable — there is no
> queue entry, no quarantine, no recovery. The message is accepted by
> Postfix and immediately dropped. Use discard for addresses that
> should never receive replies; do not use it as a quiet alternative
> to bouncing mail you actually want to reject (use Postfix recipient
> restrictions for that).

## Fields on the page

### Add Alias modal

| Field | Notes |
|---|---|
| **Alias Address** | Full email. Must validate as an email, must be on a mailbox domain (`domains.type = 'mailbox'`), and must not already exist as a mailbox, an alias, or a virtual recipient. Conflicts produce errors 12 / 13 / 14 / 17 respectively. |
| **Type** | `Forward (deliver to mailbox)` (default) or `Discard (silently drop all mail)`. JS toggles the Delivers To and Send-As fields based on selection. |
| **Delivers To** | Tom Select typeahead populated from `mailboxes WHERE mailbox_type = 'user'`. Required for forward type, ignored for discard. The handler verifies the target mailbox exists at submit time. |
| **Allow Send-As** | `No` (default) or `Yes`. Only applies to forward type. When `Yes`, an `INSERT IGNORE` into `sender_login_maps` allows the destination mailbox owner to send under the alias address from their existing Submission session. |

### Aliases table

DataTables surface — searchable, sortable, paginated, `stateSave: true`.
Columns:

| Column | Source |
|---|---|
| Actions | Edit (opens modal) / Delete (opens confirmation modal) |
| Alias | `mailbox_aliases.alias_address` |
| Domain | `domains.domain` (joined via `domain_id`) |
| Type | Badge — `Forward` (blue) or `Discard` (dark) |
| Delivers To | `mailbox_aliases.delivers_to` for forwards; `Silently dropped` for discards |
| Send-As | Badge — `YES` / `NO` for forwards; em-dash for discards |

A Domain filter dropdown above the table narrows the visible rows to a
single mailbox domain. The dropdown only lists domains that currently
have at least one alias.

### Edit modal

Address is read-only after creation — changing the local-part would
break any send-as mappings that already reference it. Type, Delivers
To, and Send-As are all editable, with the same forward/discard
toggle behavior as the Add modal. The handler diffs the old send-as
state against the new one and adds or removes the
`sender_login_maps` row accordingly so the change to send-as is
reflected without rewriting unrelated maps.

### Delete

Per-row delete with a confirmation modal. The handler removes the
alias row and any `sender_login_maps` entries for the alias address.
Because aliases don't own a Maildir or any on-disk state, deletion is
instant and reversible only by re-creating the alias.

## Send-As — what it actually does

When Send-As is enabled on a forward alias, the handler inserts:

```sql
INSERT IGNORE INTO sender_login_maps (sender, login_user)
VALUES ('sales@company.com', 'alice@company.com');
```

That row participates in Postfix's `smtpd_sender_login_maps` lookup
on the submission port. The effect: when `alice@company.com` authenticates
to Submission (587) and tries to send a message with `From:
sales@company.com`, Postfix accepts the From: because the
`(sender, login_user)` pair exists in the map. Without Send-As,
Postfix's `reject_sender_login_mismatch` would reject the submission
because `alice@` is not the canonical owner of `sales@`.

This makes Send-As a true alternate-identity grant, not just a "vanity
From:". The user typically configures the alias as a secondary
identity in their mail client (Outlook → Account Settings → multiple
email addresses; Apple Mail → Edit Email Addresses; Thunderbird →
Manage Identities) and picks it from the From: dropdown when composing.

The deletion handler removes the matching `sender_login_maps` row
when the alias is deleted; the edit handler removes the old row and
inserts the new one when Send-As is toggled or Delivers To changes.

## Conflict checks at insert time

The add handler runs four duplicate checks before the INSERT:

| Check | Error | What it prevents |
|---|---|---|
| `mailboxes WHERE username = alias_address` | 13 | Alias collides with an actual mailbox. The mailbox itself would always win the lookup, so the alias would be dead weight. |
| `mailbox_aliases WHERE alias_address = alias_address` | 14 | Duplicate alias row (also enforced by the UNIQUE key, but caught earlier with a friendlier message). |
| `virtual_recipients WHERE virtual_address = alias_address` | 17 | Alias collides with a relay-topology virtual recipient. The UNION lookup would return both rows and the resulting fan-out is almost never the intent — the error tells the admin to remove the relay-side row first. |
| `domains WHERE domain = X AND type = 'mailbox'` | 12 | Alias's domain isn't on the mailbox-topology side. Use Virtual Recipients for relay domains. |

All four checks are advisory in the UI sense but enforced server-side
so a forged form post can't bypass them.

## Domain-delete dependency

There is no explicit dependency check on mailbox-domain deletion for
aliases — but mailbox domains are typically not removed unless every
mailbox under them is also being removed, and the alias rows become
orphaned (`domain_id` no longer resolves) rather than actively
harmful. Stale `mailbox_aliases` rows whose `domain_id` no longer
exists are skipped by the page query because of the
`INNER JOIN domains ... AND d.type = 'mailbox'`. Operational best
practice: delete aliases first, then mailboxes, then the domain.

## Failure semantics

| What breaks | What happens |
|---|---|
| Blank alias address in Add | error 10 banner, no DB write |
| Invalid email format | error 11 |
| Domain not in `domains` or not mailbox-type | error 12 |
| Address already exists as a mailbox | error 13 |
| Address already exists as an alias | error 14 |
| Address already exists as a virtual recipient | error 17 |
| Forward type with blank Delivers To | error 15 |
| Delivers To target mailbox doesn't exist | error 16 |
| Edit with missing alias_id | error 20 |
| Edit / delete with stale alias_id | error 21 |
| MySQL `hermes_db_server` down | Postfix `virtual_alias_maps` lookups fail. Default behavior is to defer affected mail with a temporary error and retry — legitimate mail is held, not bounced. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_mailbox_aliases.cfm` | `hermes_commandbox` | Page + table + Add / Edit / Delete modals |
| `config/hermes/var/www/html/admin/2/inc/add_mailbox_alias_action.cfm` | `hermes_commandbox` | Add handler with the four-way conflict check |
| `config/hermes/var/www/html/admin/2/inc/edit_mailbox_alias_action.cfm` | `hermes_commandbox` | Edit handler — toggles `sender_login_maps` on send-as changes |
| `config/hermes/var/www/html/admin/2/inc/delete_mailbox_alias_action.cfm` | `hermes_commandbox` | Delete handler — removes alias row + any send-as map entry |
| `config/hermes/var/www/html/admin/2/inc/get_mailbox_alias_json.cfm` | `hermes_commandbox` | AJAX endpoint that hydrates the Edit modal |
| `/etc/postfix/mysql-virtual.cf` | `hermes_postfix_dkim` (volume-mounted) | The UNION lookup definition shared with `virtual_recipients` |
| `mailbox_aliases`, `sender_login_maps`, `mailboxes`, `domains`, `virtual_recipients` | `hermes_db_server` | Storage and conflict-detection tables |

Nothing on this page shells out to Postfix — no `postmap`, no
`postfix reload`, no template regeneration. The MySQL lookup picks up
new rows on the next inbound message.

## Related

- [Email Relay > Virtual Recipients](https://docs.deeztek.com/books/administrator-guide/page/virtual-recipients)
  — the relay-topology equivalent. Use that page when the destination
  is external (Gmail, partner domain) or when fan-out to multiple
  destinations from one address is needed.
- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — the mailbox-domain list this page filters
  against. An alias's domain must exist there with `type = 'mailbox'`.
- [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — the destination mailbox list. The
  Delivers To dropdown is populated from active user mailboxes.
- [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes) — when several users need
  to read the same incoming mail (rather than one user receiving
  forwards), use a shared mailbox instead of a forward alias.
- [Mailbox Rules](https://docs.deeztek.com/books/administrator-guide/page/mailbox-rules) — Sieve-based filtering that runs
  on the destination mailbox after alias rewrite. Aliases route mail
  to a mailbox; Sieve rules then sort it within that mailbox.
- [Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) — the global Email Server toggles. Aliases
  work regardless of the Mailbox Sharing master switch — they have no
  Dovecot-side configuration to be gated on.
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)
  — Submission-port authentication that the Send-As flag piggybacks
  on. A user must be able to authenticate to Submission as their
  primary address before Send-As lets them switch identities.

# Domains

# Domains

Admin path: **Email Server > Domains** (`view_mailbox_domains.cfm`,
`inc/mailbox_domain_add_action.cfm`, `inc/mailbox_domain_edit_action.cfm`,
`inc/mailbox_domain_delete_action.cfm`, `inc/get_mailbox_domain_json.cfm`,
`inc/sync_mailbox_sans.cfm`, `inc/generate_nginx_configuration.cfm`,
`inc/generate_transports.cfm`, `inc/generate_relay_domains.cfm`,
`inc/generate_postfix_configuration.cfm`, `inc/add_domain_djigzo.cfm`,
`inc/delete_domain_djigzo.cfm`).

This page manages the list of **mail-server domains** — the SMTP
domains for which Hermes is itself the destination MTA, accepting
inbound mail via Postfix and delivering it locally over LMTP to
Dovecot mailboxes on `/mnt/vmail`. Each row pairs a `domains` row
(`type='mailbox'`) with a `mailbox_domains` row (the per-domain SAN
certificate binding) plus a `transport` row hardwired to
`lmtp:[hermes_dovecot]:24`, a `senders` row, and a domain-wide
`recipients` row carrying the default Amavis SVF policy.

This is the **mailbox-topology** counterpart to
[Email Relay > Domains](https://docs.deeztek.com/books/administrator-guide/page/domains). Both pages edit
the same `domains` table but use the `type` column to partition rows:
`type='relay'` belongs to the Relay page and forwards mail downstream;
`type='mailbox'` belongs to this page and delivers mail locally. A
single installation can run any mix of the two topologies — see
[Email Relay > Domains § Hermes topology overview](https://docs.deeztek.com/books/administrator-guide/page/domains#hermes-topology-overview)
for the high-level diagram.

> **Not to be confused with [Email Relay > Domains](https://docs.deeztek.com/books/administrator-guide/page/domains).**
> The Relay page handles domains where Hermes forwards mail to a
> downstream MX (M365, Exchange, Google Workspace, an internal hub).
> This page handles domains where Hermes IS the final destination —
> mailboxes, IMAP/POP3, Submission, ManageSieve, Nextcloud Mail,
> autodiscover/autoconfig, DAV — backed by Dovecot.

## Configuration storage

A single Add Mailbox Domain submission writes (or upserts) **five**
rows across four tables and regenerates Postfix + Nginx + Ciphermail:

| Table | Role |
|---|---|
| `domains` | One row per mailbox domain. `type='mailbox'` partitions it from the Relay page. Mailbox-specific metadata lives here: `default_quota_mb` (default per-mailbox quota in MB), `catchall_mailbox` (optional `postmaster@domain` style address), `nextcloud_enabled` (per-domain default — controls whether new mailboxes get a Nextcloud account), `enforce_mfa` (per-domain default for 2FA), `org_name`/`org_phone`/`org_address`/`org_website`/`org_logo_path` (Pro Organization Information for signature placeholder substitution), `allow_user_signatures` (gates the user-portal personal-signature editor for this domain). |
| `mailbox_domains` | One row per mailbox domain. `mailbox_certificate` foreign-keys into `system_certificates` — the per-domain TLS cert used by Dovecot IMAP/POP3/Submission, the autodiscover/autoconfig vhosts, and the DAV per-domain vhost. |
| `mailbox_sans` | One row per SAN prefix × domain (built from `additional_sans`). Drives per-SAN DNS/IP probe state for the certificate validator. |
| `transport` | Always `lmtp:[hermes_dovecot]:24` — mail-server domains never use SMTP forwarding. |
| `senders` + `recipients` | `senders.sender = domain`, `recipients.recipient = @domain` with `domain='1'` + the default `spam_policies` policy attached so Amavis runs on every inbound message. |

The mailbox-domain row in `domains` deliberately reuses many columns
from the relay path so the Postfix generators (`generate_transports`,
`generate_relay_domains`, `generate_postfix_configuration`) treat both
topologies uniformly — the only thing that differs is the transport
string and the per-mailbox personal info / org info columns.

## How a mailbox domain becomes live config

```
form submit  ──► mailbox_domain_add_action.cfm
                     |
                     |  validate domain + cert mode (Pro gate on 'auto')
                     |  duplicate-check against domains.domain
                     |
                     |  --- write DB ---
                     |  INSERT transport (lmtp:[hermes_dovecot]:24)
                     |  INSERT senders   (sender = domain, action = OK)
                     |  INSERT recipients(recipient = @domain,
                     |                    domain='1', policy_id=default,
                     |                    status='OK')
                     |  INSERT domains   (..., type='mailbox', default_quota_mb,
                     |                    catchall_mailbox, nextcloud_enabled,
                     |                    enforce_mfa, created_at, updated_at)
                     |  UPSERT mailbox_domains (domain, mailbox_certificate)
                     |
                     |  --- regenerate ---
                     v
            sync_mailbox_sans.cfm           -> mailbox_sans (one per prefix)
            generate_transports.cfm         -> /etc/postfix/transport + postmap
            generate_relay_domains.cfm      -> /etc/postfix/relay_domains
            generate_postfix_configuration.cfm
                                            -> /etc/postfix/main.cf
                                               + postfix reload (docker exec)
            generate_nginx_configuration.cfm
                                            -> per-domain Nginx vhosts
                                               (autodiscover, autoconfig, DAV)
            add_domain_djigzo.cfm           -> registers domain in Ciphermail
            occ group:add <domain>          -> Nextcloud group (if NC enabled)
                                               (docker exec hermes_nextcloud)
                     |
                     v
            preload_restart_nginx.cfm?returnUrl=... (Nginx restart, then redirect)
```

Edit follows the same shape minus the inserts (UPDATE on `domains`,
UPSERT on `mailbox_domains`, re-sync SANs, regen Nginx). Delete reverses
the writes after running dependency checks (see Delete below).

## Fields on the page

### Add Mailbox Domain card

| Field | Default | Notes |
|---|---|---|
| **Domain Name** | (empty) | Trimmed, lower-cased, validated by the email-trick. Rejected if the domain already exists in `domains` (as relay or mailbox). The `mailbox_domains` table is allowed to have a pre-existing row (left over from prior ACME work) — it gets UPSERTed in place. |
| **Default Quota (GB)** | `5` | Per-domain default for new mailboxes. Stored in DB as MB (`default_quota_mb`). 0.5 GB minimum, 1024 GB max, 0.5 GB step. The per-mailbox quota is set on [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes); this is the value pre-filled when adding a new mailbox under the domain. |
| **Catch-All Mailbox** | (empty) | Optional. An existing mailbox address that receives mail for any unknown recipient at the domain. Free-text — admin's responsibility to point at a real mailbox. |
| **SAN Certificate — Auto-managed (Let's Encrypt)** | Pro: checked / Community: disabled | _Pro Edition only._ Creates a placeholder Acme row in `system_certificates`; the certificate validator then validates SAN DNS + IP, requests the cert, and auto-renews. Zero maintenance once DNS is in place. |
| **SAN Certificate — Use existing certificate** | Community: checked | Pulls from `system_certificates` where `san='1'` OR the row is a system-flagged placeholder. The dropdown labels system placeholders as `TEMPORARY PLACEHOLDER (replace before production)` and sorts them last so the default is a real SAN cert. |
| **Enable Nextcloud webmail for this domain** | unchecked | Per-domain default for new mailboxes. When checked, creates a Nextcloud group named after the domain (via `occ group:add`) and pre-fills the Nextcloud toggle on the [Add Mailbox](https://docs.deeztek.com/books/administrator-guide/page/mailboxes#add-mailbox) form. Does **not** retroactively enable NC for existing mailboxes. |
| **Require Two-Factor Authentication for this domain** | unchecked | Per-domain default for new mailboxes. Same convention as Nextcloud — defaults only, no cascade to existing rows. |

### Mailbox domains table

Sortable, searchable, exportable. Columns:

| Column | Source | Badge logic |
|---|---|---|
| Domain | `domains.domain` | Plain text |
| Certificate | `system_certificates.friendly_name` via `mailbox_domains.mailbox_certificate` | Link to `view_system_certificates.cfm`; badge `Auto (LE)` for `type='Acme'`, `Imported` otherwise; `Missing` if no binding |
| Cert Status | derived from `mailbox_sans` rows for the domain | `Verified` (all SANs DNS-confirmed) / `Partial` / `Awaiting Cert` / `Pending` / `DNS Failed` / `No SANs` / `No Cert`. Imported certs always show `Imported`. |
| Default Quota | `default_quota_mb` | Rendered in GB |
| Catch-All | `catchall_mailbox` | Em-dash if NULL |
| Nextcloud | `nextcloud_enabled` | `Enabled` (success) / `Disabled` (secondary) |
| 2FA | `enforce_mfa` | `Required` (success) / `Optional` (secondary) |
| DKIM | aggregated from `dkim_sign` | `Active` / `Disabled` / `None` — same logic as the Relay page |
| Actions | — | Edit (opens modal), DNS Records (opens helper modal), DKIM Keys (→ `edit_domain_dkim.cfm`), Delete |

### Edit Mailbox Domain modal

Opens via `openEditModal(id)`, fetches `./inc/get_mailbox_domain_json.cfm`
over AJAX, hydrates every form field. **Domain Name is read-only on
edit** — same convention as the Relay page (renaming a domain across
all the joined tables is risky enough that the page enforces
add-and-delete instead).

The Edit modal carries everything from Add plus three extra sections
that exist only after creation:

| Section | Notes |
|---|---|
| **Organization Information** _(Pro only)_ | `org_name`, `org_phone`, `org_address`, `org_website`. Used by the body milter's signature substitution to fill `{{org.name}}`, `{{org.phone}}`, `{{org.address}}`, `{{org.website}}` placeholders in organizational signatures. See [Organizational Signatures](https://docs.deeztek.com/books/administrator-guide/page/organizational-signatures). All fields optional. Community installs see a Pro upsell badge and the inputs are HTML-disabled — the action handler also skips the UPDATE on Community so a tampered form post can't write data and existing values survive a Pro→Community downgrade. |
| **`org_logo_path`** | Column exists but no UI yet — placeholder for follow-up integration with the inline image pipeline that ships organizational signature logos. |
| **Allow users in this domain to manage their own signatures** | Per-domain toggle (`allow_user_signatures`, both tiers). When on, mailbox users see a Signature page in `/users/2/`. When off, the page is hidden and any user-edited signature rows for the domain are ignored at send time. The body milter respects this on the next signature-map regen. |

The modal explicitly tags `Nextcloud webmail` and `Two-Factor
Authentication` as **defaults for new mailboxes** — toggling them
does **not** flip the corresponding per-mailbox flags on existing
rows. To change an existing mailbox use the per-mailbox Edit Options
dialog on [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes).

### DNS Records modal

Per-domain reference card surfacing every DNS record an operator
needs to publish for the domain to actually receive mail and support
client auto-discovery: MX, autoconfig/autodiscover CNAMEs, the SRV
chain (`_imap`, `_imaps`, `_pop3`, `_pop3s`, `_submission`,
`_submissions`, `_sieve`, `_autodiscover`), CalDAV/CardDAV SRV+TXT
(`_caldavs`, `_carddavs` with `path=/nc/remote.php/dav/`), plus
example SPF and DMARC TXT records. DKIM TXT records are listed
separately under DKIM Keys.

Console host (`parameters2 console.host`) is interpolated into every
record so the values are copy-paste ready.

### Delete Mailbox Domain modal

Confirms the destructive action. The handler runs two dependency
checks before allowing the delete:

| Check | If it returns rows → |
|---|---|
| Mailboxes under this domain (`mailboxes.domain_id = <id>`) | Error 16, abort, link admin to [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) to clear them first |
| Recipients still attached to the domain (excluding the domain-wide `@domain` row) | Error 17, abort |

If both pass, the handler:

1. Captures the bound `mailbox_certificate` id (for orphan-cert
   detection).
2. Deletes `mailbox_domains`, `domains`, `transport`, `senders`,
   `recipients` (the five rows linked at creation).
3. Deletes the domain's `mailbox_sans` rows **directly** (does not
   call `sync_mailbox_sans.cfm` — sync would nuke validated IP/DNS
   state on other domains if it ran during a delete→re-add cycle).
4. Regenerates Postfix + Nginx, deregisters from Ciphermail, runs
   `occ group:delete <domain>` against Nextcloud (non-fatal).
5. If the bound certificate now belongs to no other mailbox domain,
   surfaces an **Orphaned Certificate** flash on the next page render
   pointing the admin to [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates).
   The cert is **not** auto-deleted because Let's Encrypt limits
   duplicate certificate issuance to 5 per week and accidentally
   throwing away a cert you might re-need is a non-recoverable
   mistake.

> **Operational consequence — mailbox data on disk is NOT deleted.**
> The delete handler removes the Dovecot domain wiring (transport,
> recipient acceptance, cert binding) but does **not** touch
> `/mnt/vmail/<domain>/`. If you intend to permanently retire a
> domain, remove the mailbox directories from the host after the
> delete completes.

## Per-domain Nginx vhosts

Each mailbox domain generates per-domain Nginx vhosts for:

- `autodiscover.<domain>` — Outlook / iOS Mail auto-configuration
- `autoconfig.<domain>` — Thunderbird / K-9 Mail auto-configuration
- The DAV chain via the SRV records published by the DNS Records modal

Add and Edit both call `generate_nginx_configuration.cfm` then redirect
through `preload_restart_nginx.cfm` (the canonical restart pattern
that avoids the brief `ERR_CONNECTION_REFUSED` blip in user-driven
flows).

> **Known gotcha — editing the vhost template does NOT update
> already-generated vhosts.** The generator writes per-domain files
> at install time and on subsequent saves. If the underlying template
> (in `/opt/hermes/templates/`) is hand-edited, existing vhost files
> stay stale until each domain is re-saved (or until a separate
> re-render pass is run). Operators changing the template should plan
> for a bulk re-save afterwards.

## Cert SAN binding and the validator

`sync_mailbox_sans.cfm` reads `additional_sans` (the global list of
prefixes — `mail.`, `autodiscover.`, `autoconfig.`, plus any custom
ones) and writes one `mailbox_sans` row per prefix × this domain,
pointing at the selected certificate. Each row carries IP and DNS
probe state.

A separate scheduled task (System > [SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management))
walks `mailbox_sans` every 30 minutes, probes each subdomain for the
expected IP and DNS A/CNAME record, and updates `ip_result_msg` /
`dns_result_msg`. The Cert Status column on the main table summarizes
these results.

For Pro Edition's auto-managed certs the validator then triggers a
Let's Encrypt issuance once every SAN passes both probes. For
imported certs the probes are informational only — the cert is
trusted as-is.

See [SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management) for the full SAN editor.

## Failure semantics

| What breaks | What happens |
|---|---|
| Domain name empty | `session.m = 10`, redirect, no DB write |
| Domain name fails email-trick validation | `session.m = 11`, redirect, no DB write |
| Domain already exists in `domains` (relay or mailbox) | `session.m = 12`, redirect, no DB write |
| Auto-managed selected on Community edition | `session.m = 14`, redirect, no DB write |
| `cert_id` invalid for `Use existing` | `session.m = 13`, redirect, no DB write |
| `default_quota_gb` not a positive number | `session.m = 15`, redirect, no DB write |
| Delete blocked: mailboxes still exist | `session.m = 16`, redirect, abort. Detail count shown in the alert. |
| Delete blocked: recipients still exist | `session.m = 17`, redirect, abort |
| `add_domain_djigzo.cfm` errors during Ciphermail registration | Domain is already in the DB; encryption gateway will not know about the domain until the next re-save. Non-fatal. |
| `occ group:add` fails (NC down, group exists) | Non-fatal `cftry` — mailbox-domain creation still succeeds; admin can re-toggle in Edit to retry |
| Nginx vhost regen fails | Domain is in the DB; per-domain auto-discovery URLs will return errors until the next successful Edit/regen |
| Postfix reload fails | Live config keeps the previous values; reload error is in container logs |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_mailbox_domains.cfm` | `hermes_commandbox` | Page + Add card + Edit/Delete/DNS modals |
| `config/hermes/var/www/html/admin/2/inc/mailbox_domain_add_action.cfm` | `hermes_commandbox` | Add handler |
| `config/hermes/var/www/html/admin/2/inc/mailbox_domain_edit_action.cfm` | `hermes_commandbox` | Edit handler |
| `config/hermes/var/www/html/admin/2/inc/mailbox_domain_delete_action.cfm` | `hermes_commandbox` | Delete handler |
| `config/hermes/var/www/html/admin/2/inc/get_mailbox_domain_json.cfm` | `hermes_commandbox` | AJAX hydrator for the Edit modal |
| `config/hermes/var/www/html/admin/2/inc/sync_mailbox_sans.cfm` | `hermes_commandbox` | Builds `mailbox_sans` rows from `additional_sans` × domain |
| `config/hermes/var/www/html/admin/2/inc/generate_nginx_configuration.cfm` | `hermes_commandbox` | Per-domain vhost generator |
| `config/hermes/var/www/html/admin/2/inc/generate_transports.cfm` / `generate_relay_domains.cfm` / `generate_postfix_configuration.cfm` | `hermes_commandbox` | Shared Postfix regenerators (also used by [Email Relay > Domains](https://docs.deeztek.com/books/administrator-guide/page/domains)) |
| `config/hermes/var/www/html/admin/2/inc/add_domain_djigzo.cfm` / `delete_domain_djigzo.cfm` | `hermes_commandbox` | Ciphermail registration |
| `config/hermes/var/www/html/admin/2/inc/signature_regen_map.cfm` | `hermes_commandbox` | Rebuilds the body milter's `signature_by_sender` map + `sender_data.json` after org info / `allow_user_signatures` edits |
| `config/hermes/var/www/html/admin/2/preload_restart_nginx.cfm` | `hermes_commandbox` | Nginx restart shim used on Add and Edit redirect |
| `/etc/postfix/transport` + `.db`, `/etc/postfix/relay_domains`, `/etc/postfix/main.cf` | `hermes_postfix_dkim` | Postfix maps regenerated on every save |
| Per-domain Nginx vhost files | `hermes_nginx` (mounted) | Generated by `generate_nginx_configuration.cfm` |
| `domains`, `mailbox_domains`, `mailbox_sans`, `transport`, `senders`, `recipients` | `hermes_db_server` | The mailbox-domain row group |
| `system_certificates`, `additional_sans` | `hermes_db_server` | Cert inventory + SAN prefix list |
| `hermes_nextcloud` container | — | `occ group:add` / `group:delete <domain>` for the per-domain NC group |
| `hermes_ciphermail` container | — | Domain registration via CLITool |

Every shell-out uses `docker exec ...` per the standard Hermes pattern.

## Related

- [Email Relay > Domains](https://docs.deeztek.com/books/administrator-guide/page/domains) — the relay
  topology twin. Mailbox and relay domains share the same `domains`
  table but partition on `type`. **Do not confuse with this page.**
- [Email Server > Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — per-mailbox CRUD. A
  mailbox domain is meaningless without mailboxes; add the domain
  here first, then add mailboxes there.
- [Email Server > Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) — global Dovecot
  configuration (TLS profile, compression, encryption at rest, quota
  warning thresholds). The per-domain default quota set here is what
  Email Server > Settings's warning thresholds measure against on a
  per-mailbox basis.
- [Email Server > Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases) — alias addresses that resolve
  to local mailboxes within a mailbox domain.
- [Email Server > Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes) — shared
  mailboxes are per-domain just like regular mailboxes.
- [Email Server > Mailbox Rules](https://docs.deeztek.com/books/administrator-guide/page/mailbox-rules) — per-mailbox
  Sieve rules.
- [Email Server > SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management) — the global
  SAN prefix list (`additional_sans`) that `sync_mailbox_sans.cfm`
  multiplies against every mailbox domain.
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) —
  certificate inventory that the SAN Certificate dropdown draws
  from, including the bootstrap placeholder cert.
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — mailbox users
  can authenticate against an upstream LDAP/AD using the same
  `auth_type='remote'` pattern documented for relay recipients.
- [Organizational Signatures](https://docs.deeztek.com/books/administrator-guide/page/organizational-signatures)
  _(Pro)_ — consumer of the Organization Information fields on the
  Edit modal.

# Mailbox Rules

# Mailbox Rules

Admin path: **Email Server > Mailbox Rules** (`view_sieve_rules.cfm`,
`inc/sieve_rule_actions.cfm`, `inc/sieve_helpers.cfm`,
`inc/generate_sieve_global.cfm`, `inc/get_sieve_rule_json.cfm`).

This page manages **global Sieve rules** — server-side filters that
run on every message delivered to every mailbox **before** any
user-defined Sieve script. Sieve is the IETF mail filtering language
(RFC 5228); Dovecot's `sieve` plugin executes it at LMTP delivery
time, after Amavis content scanning and just before the message
lands in the user's mailbox.

This page is the **admin** side. Mailbox users get a parallel UI in
the user portal (`/users/2/view_sieve_rules.cfm`, `scope='user'`)
where they can manage their own rules. Global rules always run first
and **cannot be overridden** by user rules — they are the right
place for organization-wide policy (compliance archiving, mandatory
quarantine routing, blanket discards of known-noise patterns).

## How Sieve fits the delivery pipeline

```
   inbound SMTP -> Postfix -> Amavis (spam/virus) -> Postfix
                                                       |
                                                       v
                                              Dovecot LMTP (port 24)
                                                       |
                                                       v
                                       sieve_before  =  /srv/sieve/global/before.sieve
                                                       |   (this page)
                                                       v
                                       user .sieve scripts (per-mailbox)
                                                       |
                                                       v
                                              final mailbox delivery
```

`sieve_before` is the Dovecot Pigeonhole convention for scripts that
run **before** the user's personal script. Hermes wires that to
`/srv/sieve/global/before.sieve` (mounted from
`/mnt/data/sieve/global/`). The user-portal page writes per-mailbox
scripts to `/mnt/data/sieve/<user>/` which run after the global
script — and only if the global script does not `discard` or
`reject` the message first.

## Configuration storage

Each rule is split across three tables to support multi-condition /
multi-action rule definitions:

| Table | Role |
|---|---|
| `sieve_rules` | One row per rule. `scope='global'` for admin rules; `scope='user'` (with `username`) for per-mailbox rules. Carries `rule_name`, `rule_order` (top-to-bottom evaluation order), `enabled` (0/1), `is_system` (0/1 — system rules can be toggled but not deleted), `match_type` (`all` = `allof` / AND, `any` = `anyof` / OR). |
| `sieve_rule_conditions` | One row per condition for the rule. `condition_field` (`subject`, `from`, `to`, `cc`, `bcc`, `header`, `size`, `all`), `condition_type` (`contains`, `is`, `matches`, `not_contains`, `over`, `under`), `condition_value`, `condition_order`. Cascade-deletes when the parent rule is removed. |
| `sieve_rule_actions` | One row per action. `action_type` (`fileinto`, `discard`, `keep`, `redirect`, `flag_seen`, `reject`), `action_value`, `action_order`. Cascade-deletes with the parent. |
| `sieve_compile_log` | Append-only log of `sievec` compile errors keyed by `scope`/`username`/`rule_id`. Indexed on `(scope, username)` and `created_at` for the troubleshooting view. |

The save handler wraps the child-row delete + re-insert in a single
`cftransaction` so a mid-write failure doesn't leave a rule with
partial conditions or actions.

## How a rule becomes a compiled Sieve script

```
form submit  ──► sieve_rule_actions.cfm
                     |
                     |  validatePayload()   - field/type/value checks
                     |    - rule_name not blank, <= 255 chars
                     |    - >= 1 condition, >= 1 action
                     |    - "all" condition cannot coexist with others
                     |    - size value matches ^\d+\s*[KMGkmg]?[Bb]?$
                     |    - redirect action requires IsValid("email", v)
                     |    - per-value length caps (500 cond, 255 act)
                     |
                     |  --- write DB ---
                     |  INSERT/UPDATE sieve_rules
                     |  cftransaction:
                     |    DELETE child conds + acts for this rule_id
                     |    INSERT every cond_field_<i> / cond_type_<i> / cond_value_<i>
                     |    INSERT every act_type_<i>  / act_value_<i>
                     |
                     |  --- generate ---
                     v
            generate_sieve_global.cfm
                |
                |  read every enabled scope='global' rule (ordered by rule_order)
                |  build "require [...]" header based on action types used
                |    fileinto -> "fileinto", flag_seen -> "imap4flags",
                |    reject   -> "reject",   vacation  -> "vacation"
                |  for each rule:
                |    "## Rule: <name>"
                |    if (single cond):           if <cond> { <actions> }
                |    if (multi-cond, match all): if allof (<cond>, <cond>) { <actions> }
                |    if (multi-cond, match any): if anyof (<cond>, <cond>) { <actions> }
                |    if (all-messages):          (unconditional actions)
                |
                |  cffile write /mnt/data/sieve/global/before.sieve
                |  docker exec hermes_dovecot chown -R 1000:1000 /srv/sieve/global
                |
                v
            docker exec hermes_dovecot sievec /srv/sieve/global/before.sieve
                |
                |  stderr non-empty? -> request.sieveCompileError set,
                |                       row inserted into sieve_compile_log,
                |                       session.m = 30 ("saved, but compile failed")
                |                       previous .svbin remains active
                |
                |  stderr empty?     -> session.m = 1/2/3/4 per action
                |
                v
            cflocation -> view_sieve_rules.cfm
```

The compile-and-keep-old-binary behavior is by design. A broken rule
saved into the DB does **not** break delivery — Dovecot continues
executing the previous good `.svbin`, and the admin sees the compile
error inline in the next page render. Fix and re-save to clear it.

## The condition vocabulary

| `condition_field` | What it matches | `condition_type` options |
|---|---|---|
| `subject` | The `Subject:` header | `contains`, `is`, `matches`, `not_contains` |
| `from` / `to` / `cc` / `bcc` | The respective address header. **Uses Sieve's `address` test, not `header`** — extracts just the email address, ignoring display name and angle brackets. | `contains`, `is`, `matches`, `not_contains` |
| `header` | Custom header. Value field is `Header-Name: value` — the first colon splits name from value, so header values containing colons (`X-Custom: foo:bar`) are preserved. | `contains`, `is`, `matches`, `not_contains` |
| `size` | Message body size. Value accepts `10`, `10M`, `10 MB`, `10mb` — normalized at save time to `10M`. | `over`, `under` |
| `all` | All messages. Cannot be combined with other conditions in the same rule. | (no type) |

`matches` uses Sieve's glob syntax (`*` and `?`), not full regex. Use
it for filename-style patterns; use `contains` for substring matches.

## The action vocabulary

| `action_type` | Effect | Value required? |
|---|---|---|
| `fileinto` | Deliver into the named IMAP folder. Use `/` for nested folders (`Work/Projects`). Folder must exist — the global generator does **not** emit `:create` (admin rules don't create folders for users; only the user-side generator does). | Yes |
| `discard` | Silently drop the message. No delivery, no bounce, no notification. **Irreversible.** Combine with the `all` condition only with extreme care. |  |
| `keep` | Default delivery to INBOX. Useful when chained with `flag_seen` to deliver-and-mark-read. |  |
| `redirect` | Forward the message to another address. **See the Forwarder-trust warning below.** | Yes — must validate as an email address |
| `flag_seen` | Adds the `\Seen` IMAP flag. Combine with `keep` or `fileinto` to deliver as already-read. |  |
| `reject` | Bounce the message back to the sender with the supplied text. **Leaks that the address exists** — use sparingly. | Yes |

The form refuses to save without at least one condition and one
action; the action handler re-validates server-side regardless.

## The Forwarder-trust warning (#229)

The Action row UI surfaces an explicit warning when `redirect` is
selected, because forwarding from a server-side rule breaks all three
of the receiver's sender-authentication signals:

| Signal | Why it breaks |
|---|---|
| **SPF** | The receiver sees Hermes's IP, not an IP authorized by the original sender's SPF record. This break happens on **any** forward, regardless of body modification. |
| **DKIM** | If Hermes-side modifiers (external-sender banner, disclaimer, encryption) altered the body, the original sender's `DKIM-Signature` body hash no longer matches. |
| **ARC** | If the inbound message had an upstream ARC seal, the same body modification invalidates it. Hermes's own seal honestly records `cv=fail`. |

With all three broken, the receiver applies the original sender's
DMARC policy — `p=quarantine` or `p=reject` for strict domains means
the forward lands in spam or is dropped outright. **Internal
redirects** (to a mailbox Hermes itself hosts) are not affected
because Hermes never re-evaluates its own headers. For external
destinations, the receiver must be configured to trust this gateway
as an authorized forwarder (ARC sealer allow-list, internal-relay
exception, etc.) for the redirect to survive DMARC enforcement.

This applies symmetrically to the Sieve `redirect` action on the
user-portal side.

## Dangerous-combination guards

The save form fires a JavaScript `confirm()` dialog before submitting
two specific combinations:

| Combination | Warning |
|---|---|
| `all` condition + `discard` action | "This rule will SILENTLY DELETE every incoming message that reaches a mailbox. This is irreversible. Are you absolutely sure?" |
| `all` condition + `reject` action | "This rule will REJECT every incoming message and bounce it back to the sender. Are you absolutely sure?" |

The guards exist because the global script runs **before** every
user's personal rules — a misclick here black-holes the entire mail
server for every mailbox. The dialog cancels the submit and explicitly
clears the page preloader (the global form-submit hook in
`html_head.cfm` shows the preloader before this handler can decide to
cancel).

## System rules

Rules with `is_system = 1` are seeded by the installer or by future
migrations. The UI surfaces them with a **System** badge and:

- The Delete button is **suppressed** in favor of the badge
- The Edit button is **suppressed** — system rules are read-only
- The Enable / Disable toggle still works — admins can turn a system
  rule off without deleting it
- The action handler's `delete_rule` branch re-checks `is_system`
  server-side and refuses with error 22 if a crafted POST tries to
  bypass the missing button

Reorder is allowed on system rules, so an admin can move a system
rule above or below a custom rule when the order matters.

## The Bcc caveat

The page calls this out explicitly: the `Bcc:` header is **stripped
by the MTA before delivery** in almost every case (that is the entire
purpose of Bcc). A condition matching the `Bcc` field will therefore
rarely fire on incoming mail. The option exists for completeness and
for the rare deployments where an upstream relay preserves the
header, but rules built around it should not be considered reliable.

## Failure semantics

| What breaks | What happens |
|---|---|
| Rule name blank or > 255 chars | `session.m = 10`, no DB write |
| Zero conditions (or all conditions blank) | `session.m = 11` |
| Zero actions (or all actions blank) | `session.m = 12` |
| `size` value fails the `^\d+\s*[KMGkmg]?[Bb]?$` regex | `session.m = 13` |
| `redirect` action with an invalid email address | `session.m = 14` |
| `fileinto` or `reject` action with empty value | `session.m = 15` |
| Condition value > 500 chars or action value > 255 chars | `session.m = 16` |
| `all` condition combined with any other condition | `session.m = 17` |
| Delete attempted on a system rule | `session.m = 22` |
| `sievec` compile error | `session.m = 30`, warning banner with full stderr, **previous compiled script stays active**, error logged to `sieve_compile_log` |
| `sievec` not reachable (Dovecot container down) | Same path as a compile error — wrapped in `cftry`; `request.sieveCompileError` captures the exception text |
| Transaction rollback during child re-insert | Rule row UPDATE is rolled back too (the wrapping `cftransaction` covers both); page surfaces the underlying exception |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_sieve_rules.cfm` | `hermes_commandbox` | Page + Add/Edit/Delete modals + reorder/toggle forms |
| `config/hermes/var/www/html/admin/2/inc/sieve_rule_actions.cfm` | `hermes_commandbox` | Action handler — validate, write DB, regenerate, compile |
| `config/hermes/var/www/html/admin/2/inc/generate_sieve_global.cfm` | `hermes_commandbox` | Reads `sieve_rules` + children, writes `before.sieve`, runs `sievec` |
| `config/hermes/var/www/html/admin/2/inc/sieve_helpers.cfm` | `hermes_commandbox` | Shared condition/action string builders (used by global + user generators) |
| `config/hermes/var/www/html/admin/2/inc/get_sieve_rule_json.cfm` | `hermes_commandbox` | AJAX hydrator for the Edit modal |
| `/mnt/data/sieve/global/before.sieve` | `hermes_dovecot` (mounted from host) | Live global script — overwritten on every save |
| `/mnt/data/sieve/global/before.svbin` | `hermes_dovecot` (mounted from host) | Compiled binary that Dovecot actually executes |
| `/mnt/data/sieve/<user>/*.sieve` | `hermes_dovecot` (mounted from host) | Per-mailbox user scripts (managed by the user portal, not this page) |
| `sieve_rules`, `sieve_rule_conditions`, `sieve_rule_actions`, `sieve_compile_log` | `hermes_db_server` | The rule definition + compile-error log |

`sievec` is the Pigeonhole compiler. It **must** run inside the
Dovecot container because the resulting `.svbin` format is
plugin-version-sensitive and tied to the `pigeonhole` build Dovecot
loads at runtime. Running it on the host would produce a binary
Dovecot can't load.

## Related

- [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — global rules run against every mailbox
  on every domain. There is no per-mailbox or per-domain scoping at
  the global tier — use conditions on `to`, `from`, or a custom
  header to scope.
- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — `domains.allow_user_signatures` is the
  closest per-domain user-rule toggle Hermes has today. There is no
  separate per-domain toggle for user Sieve rules; the user-portal
  Sieve editor is always available to mailbox users.
- [Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) — Dovecot's `sieve` plugin and the
  `sieve_before` directive are configured globally there. The
  per-rule pieces this page edits sit underneath that global wiring.
- [Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases) — silent-discard aliases are an alternative
  to a Sieve `discard` rule when the goal is to nuke mail to a
  specific address rather than match on content.
- [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes) — global Sieve runs on
  shared-mailbox delivery too. A `fileinto` rule referencing a
  shared mailbox path will work as long as the folder exists.
- [Email Relay > Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients)
  — relay recipients do **not** receive Dovecot LMTP delivery (mail
  is forwarded out via Postfix `smtp_*` instead), so global Sieve
  rules do not run against relay-bound mail. Use Amavis policies or
  the body milter for relay-side filtering instead.

# Mailboxes

# Mailboxes

Admin path: **Email Server > Mailboxes** (`view_mailboxes.cfm`,
`add_mailbox.cfm`, `inc/add_mailbox_action.cfm`, `inc/edit_mailbox_action.cfm`,
`inc/edit_mailbox_encryption_action.cfm`, `inc/edit_mailbox_access_control_action.cfm`,
`inc/delete_mailbox_action.cfm`, `inc/get_mailbox_json.cfm`,
`inc/ldap_add_user_mailbox.cfm`, `inc/ldap_add_user_mailbox_remoteauth.cfm`,
`inc/ldap_add_user_groups_mailbox.cfm`, `inc/ldap_delete_user_mailbox.cfm`,
`inc/nextcloud_provision_user.cfm`, `inc/signature_regen_map.cfm`,
`inc/send_mailbox_welcome_email.cfm`, `inc/send_mailbox_welcome_email_remoteauth.cfm`,
`inc/admin_resend_mobile_setup_action.cfm`, `inc/rotate_nc_password_action.cfm`).

This page manages **individual mailboxes** inside the mail-server
topology — one row per address in the `mailboxes` table, joined to a
`recipients` row that carries the per-recipient policy stack (SVF
policy, encryption flags, S/MIME certs, PGP keyrings, 2FA enforcement,
auth type). A mailbox is the local-delivery counterpart to a Relay
Recipient — same `recipients` row shape, different `recipient_type`
column value (`'mailbox'` vs `'relay'`) and a sibling row in
`mailboxes` that gives Dovecot a userdb entry.

This is the **per-mailbox** half of the mail-server topology. Pairs
with [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) (the domains those mailboxes live under
and inherit defaults from), [Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) (global Dovecot
config and quota warning thresholds), and the per-address feature
pages: [Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases), [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes),
[Mailbox Rules](https://docs.deeztek.com/books/administrator-guide/page/mailbox-rules), and per-mailbox app passwords.

## Mailbox vs Alias vs Shared Mailbox vs Relay Recipient

Four address concepts share the namespace under a mailbox domain;
keep them straight:

| Concept | Stored in | Has Dovecot mailbox? | Local sign-in? |
|---|---|---|---|
| **Mailbox** (this page) | `mailboxes` (`mailbox_type='user'`) + `recipients` (`recipient_type='mailbox'`) | Yes — Dovecot LMTP delivery to `/mnt/vmail/<domain>/<user>/` | Yes — IMAP/POP3/Submission, web portal, Nextcloud |
| **Alias** | `mailbox_aliases` | No — forwards to one or more mailboxes (or silently discards) | No |
| **Shared Mailbox** | `mailboxes` (`mailbox_type='shared'`) + `shared_mailbox_permissions` | Yes — but accessed via Dovecot ACL from owner mailboxes | No direct login — owners reach it from their own session |
| **Relay Recipient** | `recipients` (`recipient_type='relay'`) | No — forwarded to a downstream MX | Yes for web portal / Submission (via app passwords) |

See [Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases) and [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes)
for the alias and shared variants, and
[Email Relay > Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients)
for the relay-topology equivalent.

## What a Mailbox row carries

```
mailboxes table  (Dovecot userdb-driving row)
├── id, domain_id          -> joins to domains where type='mailbox'
├── username               full email (e.g. jsmith@company.com)
├── name                   display name
├── quota                  per-mailbox quota in BYTES (DB stores bytes;
│                          UI shows GB)
├── active                 1/0 — Dovecot rejects auth when 0
├── nextcloud_enabled      per-mailbox Nextcloud flag
├── mailbox_type           'user' | 'shared'
└── first_name, last_name, title, phone, mobile, department
                           (Pro Personal Information for signature
                            substitution)

recipients table  (paired row, recipient_type='mailbox')
├── recipient              same as mailboxes.username
├── policy_id              -> spam_policies (SVF policy)
├── auth_type              'local' | 'remote'
├── remoteauth_domain      NULL if local; mapping key if remote
├── enforce_mfa            0 | 1 (admin policy)
├── pdf_enabled / smime_enabled / pgp_enabled / digital_sign
└── (cert + keyring slots populated lazily by cert_generation_queue)
```

Side tables linked at create-time or lazily:

| Table | Role |
|---|---|
| `user_settings` | `report_enabled` (quarantine notifications), `train_bayes`, `download_msg`, `timezone`, `ldap_username` |
| `maddr` | Amavis address index — required for the user portal session machinery |
| `sender_login_maps` | Postfix `smtpd_sender_login_maps` entry — permits the mailbox owner to send AS their own address from Submission |
| `app_passwords` | Per-mailbox application passwords (Argon2-hashed) for IMAP/SMTP/CalDAV/CardDAV/Nextcloud. The Add flow creates an initial `Hermes System` app password used by the Nextcloud Mail auto-profile. |
| `recipient_certificates`, `recipient_keystores` | S/MIME cert + PGP keyring slots (lazy — populated by the queue) |
| `cert_generation_queue` | Async S/MIME + PGP generation jobs |
| `mailbox_aliases` | If any aliases exist pointing at the mailbox |
| `shared_mailbox_permissions` | If the mailbox is granted access to any shared mailbox |

## Add Mailbox — `add_mailbox.cfm`

Single-mailbox page (not a bulk form). The admin selects a target
domain, fills in the address local-part + display name + quota + auth
mode + per-recipient stack (policy, notifications, encryption flags),
and submits. `add_mailbox_action.cfm` then runs the full creation
pipeline:

```
form submit  ──► add_mailbox_action.cfm
                     |
                     |  validate domain + email + auth mode
                     |  duplicate-check against recipients, mailboxes,
                     |     mailbox_aliases, virtual_recipients
                     |
                     |  --- write DB ---
                     |  INSERT recipients   (recipient_type='mailbox', policy,
                     |                       auth_type, remoteauth_domain,
                     |                       enforce_mfa, encryption flags)
                     |  INSERT maddr        (Amavis address index)
                     |  INSERT user_settings(notifications, train_bayes,
                     |                       download_msg, timezone)
                     |  INSERT mailboxes    (domain_id, username, name,
                     |                       quota, active=1, nextcloud_enabled)
                     |  INSERT sender_login_maps (permits send-as)
                     |
                     |  --- LDAP ---
                     |  auth_type=local  : ldap_add_user_mailbox.cfm
                     |                     (random userPassword, will be reset)
                     |  auth_type=remote : ldap_add_user_mailbox_remoteauth.cfm
                     |                     (no userPassword; seeAlso pointer to
                     |                     upstream DN, associatedDomain set to
                     |                     remoteauth_domain)
                     |  ldap_add_user_groups_mailbox.cfm
                     |    -> cn=mailboxes,ou=groups,dc=hermes,dc=local
                     |    -> cn=one_factor OR cn=two_factor (per enforce_mfa)
                     |  if NC enabled:
                     |    -> cn=nextcloud,ou=groups,dc=hermes,dc=local
                     |
                     |  --- Nextcloud (if NC enabled) ---
                     |  nextcloud_provision_user.cfm
                     |    -> occ user:add with RANDOM internal password
                     |       (not the user's real password — they reach NC
                     |        via OIDC; the internal password is defense-in-depth)
                     |    -> occ user:setting to pre-fill email + display name
                     |    -> create initial Hermes System app password
                     |       (used by the Mail app account profile)
                     |    -> create Nextcloud Mail account profile
                     |       (IMAP+SMTP credentials pre-wired)
                     |
                     |  --- lazy cert / keyring queue ---
                     |  if smime_enabled : INSERT cert_generation_queue (smime)
                     |  if pgp_enabled   : INSERT cert_generation_queue (pgp)
                     |
                     |  --- send welcome ---
                     |  local  : send_mailbox_welcome_email.cfm
                     |           (password-reset link, 30-min expiry)
                     |  remote : send_mailbox_welcome_email_remoteauth.cfm
                     |           (sign-in with organization password)
                     |
                     |  --- signature map ---
                     |  if Pro: signature_regen_map.cfm
                     |    -> rebuild body milter signature_by_sender map
                     |    -> rebuild sender_data.json
                     |
                     v
            cflocation -> view_mailboxes.cfm with session.m = 1
```

Dovecot mailbox directories on `/mnt/vmail/<domain>/<user>/` are NOT
pre-created. Dovecot auto-creates the directory tree on first LMTP
delivery or first IMAP login. The mailbox row alone is enough.

### Password handling

Local-auth mailboxes:

- The admin enters a password on the Add form (12-char minimum, no
  special chars, checked against the HIBP "Have I Been Pwned" k-anon
  range API).
- The same password is stored in three places, each hashed by its
  consuming subsystem: OpenLDAP `userPassword` (Argon2id via
  `slappasswd -o module-load=argon2.la -h {ARGON2}`), `app_passwords`
  initial `Hermes System` row (Argon2id), and the Nextcloud internal
  user password (only on the NC side, set by `occ user:add` — but
  immediately replaced with a random value by
  `nextcloud_provision_user.cfm`, see Phase 1 of #197).
- Argon2id hashing uses the canonical
  `docker run --rm authelia/authelia:<version> authelia crypto hash generate argon2 --password <value>`
  pattern. No host-side `argon2` binary required.

RemoteAuth mailboxes (`auth_type='remote'`):

- No password is captured. The local LDAP entry has no
  `userPassword`; bind goes through the OpenLDAP remoteauth overlay
  to the upstream AD/LDAP per the `remoteauth_domain` mapping (see
  [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth)).
- `app_passwords` still issues Hermes-side credentials for
  IMAP/SMTP/DAV — these remain Hermes-owned regardless of upstream
  password rotation.

## The Mailboxes table

Single DataTable with 21 columns and an optional Domain filter
dropdown above (populated only when ≥1 domain has mailboxes). Per-row
columns:

| Column | Source | Notes |
|---|---|---|
| Actions | — | Dropdown: Edit Options, Edit Encryption, Reset 2FA Devices, Manage App Passwords (→ `view_mailbox_app_passwords.cfm`), Send Mobile Setup Profile, Rotate NC Internal Password (only if NC enabled), Delete |
| S/MIME | link to `view_recipient_certificates.cfm?type=1&id=...` | Per-mailbox cert manager |
| PGP | link to `view_recipient_keyrings.cfm?type=1&id=...` | Per-mailbox keyring manager |
| Email | `mailboxes.username` | Full address |
| Display Name | `mailboxes.name` | |
| Domain | join on `domains.domain` | |
| Quota | `mailboxes.quota / 1024 / 1024 / 1024` | Rendered in GB |
| Auth | `recipients.auth_type` | `LOCAL` badge or `REMOTE` badge (tooltip shows `remoteauth_domain`) |
| 2FA | LDAP `cn=two_factor` + `enforce_mfa` | Two independent pills — see [Two-pill 2FA column](#two-pill-2fa-column) |
| Policy | `spam_policies.policy_name` | |
| Notifications, Train Bayes, Download Msgs | `user_settings.*` | `YES` (success) / `NO` (secondary) |
| PDF / S/MIME / PGP Encrypt, Sign All | `recipients.*` | `YES` / `NO` |
| S/MIME Cert, PGP Keyring | join against `recipient_certificates`, `recipient_keystores` | `YES` (green) if a cert/keyring exists; spinner badge if a job is `pending`/`processing` in `cert_generation_queue` |
| Nextcloud | `mailboxes.nextcloud_enabled` | `YES` / `NO` |
| Status | `mailboxes.active` | `Active` (success) / `Inactive` (danger) — Dovecot rejects auth when `active=0` |

The query filters `WHERE m.mailbox_type = 'user'` so shared mailboxes
do not appear here — they have their own page at
[Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes).

## Two-pill 2FA column

Same two-orthogonal-states model as
[Email Relay > Relay Recipients § Two-pill 2FA column](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients#two-pill-2fa-column).
Admin enforcement (`recipients.enforce_mfa`) and user enrollment
(`cn=two_factor` LDAP membership) are decoupled, so the cell can
show **Enrolled**, **Required**, both, or em-dash.

The page pulls all `cn=two_factor` group members in a single
`ldapsearch` (via `docker exec hermes_ldap ldapsearch -Y EXTERNAL`)
once per render, then each row checks for its DN substring in the
result — avoids an N+1 LDAP roundtrip storm.

## Edit Options modal — AJAX pre-fill

Opens via `loadEditModal(mailboxId)`, hits `inc/get_mailbox_json.cfm`
over AJAX, hydrates every field with the mailbox's current values.
Unlike the Relay Recipients bulk-edit foot-gun, this modal is
**always single-mailbox** — there is no bulk Edit Options on this
page.

Fields:

| Section | Notes |
|---|---|
| Email Address | Read-only |
| Display Name | `mailboxes.name` |
| Personal Information _(collapsible, Pro only)_ | `first_name`, `last_name`, `title`, `phone`, `mobile`, `department`. Used by signature placeholder substitution (`{{user.first_name}}`, `{{user.title}}`, etc.) and by department-based signature resolution. Department field uses a typeahead datalist built from the domain's existing departments via `inc/get_dept_options.cfm`. Community inputs are HTML-disabled and the action handler skips the UPDATE on Community so values survive a Pro→Community downgrade. |
| Mailbox Quota (GB) | Per-mailbox override of the domain default |
| Status | `Active` / `Inactive` |
| SVF Policy | Populated from `spam_policies` where `custom='1' OR default_policy='1'` |
| Quarantine Notifications | `user_settings.report_enabled` |
| Train Bayes Filter | `user_settings.train_bayes` — with prominent warning that improperly-trained Bayes affects ALL recipients |
| Download Messages from User Portal | `user_settings.download_msg` — with malware-risk warning |
| Nextcloud Webmail | `mailboxes.nextcloud_enabled`. **Enabling for an existing user requires a new password** (NC needs the password to provision the Mail app profile) — error 51 if the admin enables NC without setting a password. **Disabling** shows a `Keep Nextcloud account data` checkbox that gates whether the NC user account and data are preserved or permanently deleted. |
| Two-Factor Authentication | `recipients.enforce_mfa`. When enabled, the user's web portal access becomes restricted to Account Settings, My App Passwords, Set Up Your Devices, and Webmail & Apps until they enroll. Email/calendar/contacts keep working throughout — only the web portal is gated. The 0→1 transition triggers an LDAP group move from `cn=one_factor` to `cn=two_factor` so Authelia challenges them on next sign-in. |
| Timezone | `user_settings.timezone` (Java `ZoneId` list). Used for the vacation auto-reply schedule and dashboard timestamps. |
| Authentication Type | Read-only — `local` or `remote` |
| Change Password _(local auth only)_ | Optional. Minimum 12 chars, no special chars, HIBP-checked. Blank keeps the current password. |

## Edit Encryption modal

Per-mailbox encryption flags (`pdf_enabled`, `smime_enabled`,
`digital_sign`, `pgp_enabled`) plus the cert/keyring generation
parameters (CA, validity, key size, algorithm, PGP key length).
Submit queues async cert + keyring generation into
`cert_generation_queue` if a flag flips on and no existing
cert/keyring is present — same lazy-queue pattern as
[Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients#edit-encryption-modal).

## Reset 2FA Devices modal

Single-purpose modal that clears Authelia TOTP and WebAuthn device
registrations via
`docker exec hermes_authelia authelia storage user totp delete`
and `... webauthn delete --all`. Two modes:

| Mode | What it does |
|---|---|
| **Default** | Deletes TOTP + WebAuthn devices. User stays under 2FA enforcement and re-registers on next sign-in. "User lost their phone" recovery. |
| **Nuclear** _(checkbox)_ | Also moves the user from `cn=two_factor` back to `cn=one_factor`. Admin override; if `enforce_mfa` is still 1 the next Edit Options save will reverse the LDAP move. |

> **Does not affect Duo Push.** Duo enrollments live on Duo's cloud
> servers. Use the Duo Admin Console.

## Send Mobile Setup Profile

Per-mailbox action that emails the user a signed iOS / iPadOS
mobileconfig profile pre-wired with IMAP + Submission + CalDAV +
CardDAV + the appropriate account name and email. The link in the
email expires in 30 minutes and works only once.

Handler is `inc/admin_resend_mobile_setup_action.cfm`. The
mobileconfig generator itself is shared with the user-portal Setup
Your Devices wizard.

## Rotate NC Internal Password

Visible only when `mailboxes.nextcloud_enabled = 1`. Generates a new
random local password for the Nextcloud user via
`docker exec hermes_nextcloud occ user:resetpassword` and the
displayed value is **never shown** — it is purely defense-in-depth.

Background: the Nextcloud internal password was historically set to
the user's real password, which silently allowed CalDAV/CardDAV to
accept the org password and defeat the app-password isolation
boundary (closed in #197 Phase 1). The internal password is now
random and unused by anything user-facing — users reach NC via OIDC,
and DAV/IMAP go through app passwords. This admin action lets the
admin re-randomize on demand without touching the user's actual
credentials.

## Delete

Cascading delete that mirrors the create pipeline in reverse, with
the same cleanup discipline as Relay Recipients (the goal is
zero-orphan rows). Per mailbox:

```
For the selected mailbox ID:
1. Read mailboxes row + user_settings (for ldap_username)
2. Remove LDAP from cn=mailboxes  (before delete_internal_recipients
                                    runs ldap_delete_user_relay)
3. (If NC enabled) Remove from cn=nextcloud LDAP group
4. delete_internal_recipients.cfm
     - docker exec hermes_authelia authelia storage user totp delete
     - docker exec hermes_authelia authelia storage user webauthn delete --all
     - LDAP user entry delete
     - cert_generation_queue cancel + recipient_certificates clear
     - recipient_keystores + Ciphermail keystore clear
     - wblist, mailaddr, password_reset_requests cancel
5. DELETE mailboxes WHERE id = <id>
6. DELETE sender_login_maps WHERE login_user = <email>
7. DELETE user_settings (if not already cleared by step 4)
8. Re-sync any shared mailbox vfile ACLs the user was a member of
   (so the deleted user vanishes from sharer lists)
9. DELETE app_passwords WHERE username = <email>
10. (If NC enabled AND admin did NOT check "Keep Nextcloud data")
    docker exec hermes_nextcloud occ user:delete <user>
11. signature_regen_map.cfm (rebuild body milter map without this user)
```

The Nextcloud user/data preservation is opt-in via the `Keep Nextcloud
account data` checkbox surfaced when toggling NC off in Edit Options
— deletion from this page asks the same question.

> **Dovecot mailbox data on disk is NOT deleted.**
> `/mnt/vmail/<domain>/<user>/` survives the delete. If you intend
> to permanently retire the mailbox, remove the directory from the
> host after the delete completes. This matches the per-domain
> behavior on [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v).

## Local-auth vs RemoteAuth — the credential split

Identical model to relay recipients. See
[Email Relay > Relay Recipients § Local-auth vs RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients#local-auth-vs-remoteauth--the-credential-split)
and [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)
for the full four-credential architecture.

For mailboxes specifically: app passwords are always Hermes-issued
regardless of `auth_type`. RemoteAuth mailbox users' upstream
directory password is exposed only to the web gate (via the LDAP
overlay's pass-through bind) — never to Dovecot or the Nextcloud
Mail profile.

> **Known forward-looking gap (#102).** RemoteAuth mapping deletion
> validation in `view_remoteauth.cfm` and `edit_remoteauth_mapping.cfm`
> currently only checks `system_users` and `recipients`. When
> RemoteAuth-for-mailboxes activity grows, the validation must add a
> third query against `mailboxes` so an in-use mapping cannot be
> stranded. See
> [LDAP RemoteAuth § Deletion validation](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth#deletion-validation).

## Failure semantics

| What breaks | What happens |
|---|---|
| Quota not a positive number | `session.m = 15`, redirect, no DB write |
| Missing required form fields | `session.m = 20`, redirect, no DB write |
| Mailbox not found (Edit/Delete) | `session.m = 21`, redirect, no DB write |
| Password under 12 characters | `session.m = 22`, redirect, no DB write |
| Password found in HIBP breach | `session.m = 99`, redirect, no DB write |
| HIBP API unavailable | `session.m = 100`, warning banner, mailbox still rejected (fail-closed) |
| Enabling NC for existing user without setting a password | `session.m = 51`, redirect, no DB write |
| Mobile setup profile email failed but profile staged | `session.m = 83`, warning banner, link still works |
| Duplicate email (against recipients / mailboxes / aliases / virtual_recipients) | redirect to `add_mailbox.cfm` with appropriate alert |
| LDAP add fails after DB inserts succeed | DB row exists; subsequent IMAP/SMTP login fails until the LDAP entry is created (admin can re-save Edit Options or delete and re-add) |
| Nextcloud `occ user:add` fails | Mailbox creation succeeds; NC toggle effectively becomes a no-op until re-toggled |
| `cert_generation_queue` row stuck in `processing` | Surfaces in the Add Recipient / Add Mailbox alert banner via [Pending S/MIME or PGP generation](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients#edit-encryption-modal); retry via the same Retry Failed Jobs button on the Relay page |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_mailboxes.cfm` | `hermes_commandbox` | Main page + Edit Options / Edit Encryption / Reset 2FA / Delete modals |
| `config/hermes/var/www/html/admin/2/add_mailbox.cfm` | `hermes_commandbox` | Add page (single mailbox, full per-recipient stack) |
| `config/hermes/var/www/html/admin/2/inc/add_mailbox_action.cfm` | `hermes_commandbox` | Add handler — orchestrates DB + LDAP + NC + cert queue + welcome email |
| `config/hermes/var/www/html/admin/2/inc/edit_mailbox_action.cfm` | `hermes_commandbox` | Edit Options handler |
| `config/hermes/var/www/html/admin/2/inc/edit_mailbox_encryption_action.cfm` | `hermes_commandbox` | Edit Encryption handler + cert/keyring queue insertion |
| `config/hermes/var/www/html/admin/2/inc/edit_mailbox_access_control_action.cfm` | `hermes_commandbox` | Reset 2FA Devices handler (TOTP + WebAuthn clear + optional nuclear move) |
| `config/hermes/var/www/html/admin/2/inc/delete_mailbox_action.cfm` | `hermes_commandbox` | Delete cascade |
| `config/hermes/var/www/html/admin/2/inc/get_mailbox_json.cfm` | `hermes_commandbox` | AJAX hydrator for Edit Options |
| `config/hermes/var/www/html/admin/2/inc/get_dept_options.cfm` | `hermes_commandbox` | Per-domain department datalist (typeahead) |
| `config/hermes/var/www/html/admin/2/inc/ldap_add_user_mailbox.cfm` / `ldap_add_user_mailbox_remoteauth.cfm` | `hermes_commandbox` | Local / remote LDAP entry creation |
| `config/hermes/var/www/html/admin/2/inc/ldap_add_user_groups_mailbox.cfm` | `hermes_commandbox` | Group assignment: `cn=mailboxes`, `cn=one_factor` / `cn=two_factor`, `cn=nextcloud` |
| `config/hermes/var/www/html/admin/2/inc/ldap_delete_user_mailbox.cfm` | `hermes_commandbox` | LDAP entry removal on delete |
| `config/hermes/var/www/html/admin/2/inc/nextcloud_provision_user.cfm` | `hermes_commandbox` | NC user creation, random internal password, Mail app profile, initial app password |
| `config/hermes/var/www/html/admin/2/inc/rotate_nc_password_action.cfm` | `hermes_commandbox` | On-demand NC internal password rotation |
| `config/hermes/var/www/html/admin/2/inc/admin_resend_mobile_setup_action.cfm` | `hermes_commandbox` | Mobile setup profile generation + email |
| `config/hermes/var/www/html/admin/2/inc/send_mailbox_welcome_email.cfm` / `send_mailbox_welcome_email_remoteauth.cfm` | `hermes_commandbox` | Welcome email (local: reset link; remote: org-password instructions) |
| `config/hermes/var/www/html/admin/2/inc/signature_regen_map.cfm` | `hermes_commandbox` | Body milter `signature_by_sender` map + `sender_data.json` rebuild |
| `mailboxes`, `recipients`, `user_settings`, `maddr`, `sender_login_maps`, `app_passwords`, `recipient_certificates`, `recipient_keystores`, `cert_generation_queue`, `mailbox_aliases`, `shared_mailbox_permissions`, `wblist`, `password_reset_requests` | `hermes_db_server` | The mailbox row group |
| `cn=<user>,ou=users,dc=hermes,dc=local` | `hermes_ldap` | Per-mailbox LDAP entry (with `userPassword` Argon2id hash for local-auth or `seeAlso` for remote) |
| `cn=mailboxes`, `cn=one_factor` / `cn=two_factor`, `cn=nextcloud` in `ou=groups` | `hermes_ldap` | Group memberships set at create-time |
| `/mnt/vmail/<domain>/<user>/` | `hermes_dovecot` (mounted) | Mailbox directory tree — auto-created on first LMTP delivery / IMAP login; NOT removed on delete |
| Authelia `totp_configurations` + `webauthn_devices` | `hermes_authelia` storage backend | Cleared on delete + Reset 2FA Devices |
| `hermes_nextcloud` container | — | `occ user:add` / `user:delete` / `user:resetpassword` / `group:add` (the latter from [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v)) |

Every shell-out uses `docker exec ...` per the standard Hermes pattern.

## Related

- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — mailbox-domain registration. A mailbox is
  meaningless without a domain row of `type='mailbox'`. Domain
  defaults (default quota, Nextcloud enabled, 2FA required) pre-fill
  the Add Mailbox form for new mailboxes; toggling the per-domain
  default does NOT cascade to existing mailboxes.
- [Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) — global Dovecot config: TLS profile,
  compression, encryption at rest, quota warning thresholds. The
  warning thresholds measure against the per-mailbox quota set here.
- [Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases) — alias addresses that resolve to mailboxes
  (with optional silent-discard mode). Add aliases AFTER the target
  mailbox exists.
- [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes) — shared-namespace
  mailboxes with per-user ACLs. Distinct from regular mailboxes —
  they live in the same `mailboxes` table but with
  `mailbox_type='shared'`.
- [Mailbox Rules](https://docs.deeztek.com/books/administrator-guide/page/mailbox-rules) — server-side Sieve rules per
  mailbox. Sieve is always-on at the protocol level via
  [Settings](https://docs.deeztek.com/books/administrator-guide/page/settings).
- [SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management) — SAN prefixes that gate
  client auto-discovery for every mailbox domain.
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)
  — Authelia config, OIDC, the four-credential architecture (web
  vs IMAP/SMTP vs DAV vs Nextcloud) that mailbox app passwords
  slot into.
- [LDAP RemoteAuth](https://docs.deeztek.com/books/administrator-guide/page/ldap-remoteauth) — required
  prerequisite for `auth_type='remote'` mailboxes. The Add form
  surfaces only mappings with `enabled=1`.
- [Password Resets](https://docs.deeztek.com/books/administrator-guide/page/password-resets) — admin-driven
  password reset for local-auth mailboxes (the user-facing flow
  uses the link in the welcome email).
- [System Users](https://docs.deeztek.com/books/administrator-guide/page/system-users) — distinct from
  mailboxes; covers console admins / readers, which use the
  `system_users` table rather than `mailboxes`.
- [Email Relay > Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients)
  — the relay-topology equivalent. Mailbox users are delivered
  locally; relay recipients are forwarded downstream. Don't confuse
  the two.
- [Organizational Signatures](https://docs.deeztek.com/books/administrator-guide/page/organizational-signatures)
  _(Pro)_ — consumer of the Personal Information fields on the
  Edit Options modal (plus the domain's Organization Information
  fields).

# SAN Management

# SAN Management

Admin path: **Email Server > SAN Management** (`view_mailbox_sans.cfm`,
`inc/san_actions.cfm`, `inc/sync_mailbox_sans.cfm`,
`inc/acme_request_san_certificate.cfm`,
`inc/smtp_sni_generate_config.cfm`,
`inc/generate_nginx_configuration.cfm`,
`schedule/acme_validate_ip.cfm`).

This page maintains the **global list of SAN (Subject Alternative
Name) prefixes** that Hermes cross-joins with every mailbox-hosting
domain to produce the actual SANs on each domain's TLS certificate.
The prefix `mail` plus the domain `example.com` produces the SAN
`mail.example.com`; doing it once here lets Hermes mint one
certificate per mailbox domain that covers IMAP/POP/Submission,
autoconfig/autodiscover, ManageSieve, CalDAV/CardDAV, and any
additional client-facing hostnames in a single cert.

Pairs tightly with [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates)
(the certificate store these SANs are stamped into) and
[Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) (the mailbox-domain rows the prefixes are
multiplied against). This page is the **only** input UI for the
mailbox-cert SAN list — both the CSR generator on System Certificates
and the ACME SAN request path read from `additional_sans` to build
the `-d` flag list.

## What the page edits

```
additional_sans                              domains (type='mailbox')
+----+---------------+--------+              +----+----------------+
| id | san           | system |              | id | domain         |
+----+---------------+--------+              +----+----------------+
|  1 | autoconfig    |   1    |              |  9 | example.com    |
|  2 | autodiscover  |   1    |              | 10 | acme.org       |
|  3 | mail          |   2    |              +----+----------------+
|  4 | imap          |   2    |
+----+---------------+--------+
              |                                          |
              +--- sync_mailbox_sans.cfm cross-joins ---+
                                  |
                                  v
              mailbox_sans  (one row per prefix x domain)
              +----+-------------+--------------------------+------+------+------+
              | id | certificate | subdomain                | ip   | dns  | acme |
              +----+-------------+--------------------------+------+------+------+
              | 50 |     12      | autoconfig.example.com   | YES  | YES  |  1   |
              | 51 |     12      | autodiscover.example.com | YES  | YES  |  1   |
              | 52 |     12      | mail.example.com         | YES  | YES  |  1   |
              | 53 |     12      | imap.example.com         | NO   | NO   |  1   |
              | 54 |     12      | autoconfig.acme.org      | YES  | YES  |  1   |
              | ...
```

Two storage rows per change:

| Table | Role |
|---|---|
| `additional_sans` | One row per global prefix. `san` is the subdomain label; `system` is `1` for installer-seeded prefixes (`autoconfig`, `autodiscover`) that cannot be deleted, `2` for admin-added prefixes. There is no `enabled` flag — the row's mere presence means active. |
| `mailbox_sans` | One row per `additional_sans.san` x `domains` (`type='mailbox'`) combination. Carries the cert FK (`certificate`), the full FQDN (`subdomain`), and the per-SAN validation state (`ip` / `dns` = `YES`/`NO`, plus `*_result_datetime`, `*_result_msg`). `acme = 1` for ACME-managed certs, `2` for imported certs. |

The page itself only writes to `additional_sans`. The cross-join into
`mailbox_sans` is performed by `sync_mailbox_sans.cfm`, which is also
called from the Domains page on add/edit (so adding a new mailbox
domain populates its SAN rows immediately).

## How a prefix becomes a live SAN

```
form submit (Add SAN Prefix)  ──► san_actions.cfm
                                      |
                                      |  validate:
                                      |    - prefix not blank
                                      |    - matches ^[a-z][a-z0-9-]{0,62}$
                                      |      (DNS label rules: lowercase, starts
                                      |       with letter, <= 63 chars)
                                      |    - not already in additional_sans
                                      |
                                      |  INSERT additional_sans (san, system=2)
                                      |
                                      v
                          sync_mailbox_sans.cfm
                              |
                              |  for each (prefix x mailbox-domain):
                              |     if FQDN missing in mailbox_sans:
                              |        INSERT (cert from mailbox_domains,
                              |                subdomain=fqdn, ip='NO', dns='NO',
                              |                acme=1|2 per cert type)
                              |     if FQDN exists with wrong cert binding:
                              |        UPDATE certificate + acme
                              |        (PRESERVE ip/dns validation state —
                              |         resetting would break nginx vhost
                              |         generation until the next validator
                              |         pass)
                              |  for each existing mailbox_sans row whose
                              |     subdomain is no longer in the cross-join:
                              |        DELETE
                              |
                              v
                      Validator picks up the new rows on its next pass
                      (schedule/acme_validate_ip.cfm @every 1h)
                              |
                              |  POST encrypted subdomain to
                              |    https://verify.hermesseg.io
                              |    -> returns expected IP for the host
                              |  Compare against the SAN's resolved A record
                              |    -> ip = YES/NO with timestamped result_msg
                              |  Resolve DNS for the SAN's CNAME/A chain
                              |    -> dns = YES/NO with timestamped result_msg
                              |
                              v
                  All SANs on a cert at dns=YES + ip=YES?
                              |
                              v
              acme_request_san_certificate.cfm (Pro)
              docker run --rm certbot/certbot:latest \
                certonly --webroot --cert-name <domain> --expand \
                  -d example.com -d autoconfig.example.com \
                  -d autodiscover.example.com -d mail.example.com ...
                              |
                              v
              smtp_sni_generate_config.cfm   (Postfix SNI map)
              generate_nginx_configuration.cfm (per-SAN nginx vhosts)
```

Delete reverses the same path: removing a prefix from
`additional_sans` calls `sync_mailbox_sans.cfm`, which deletes the
corresponding `mailbox_sans` rows for every mailbox domain. The
certificate itself is **not** re-issued automatically on delete — the
next renewal cycle picks up the smaller SAN set when it runs.

## The two seed prefixes

A fresh install seeds two `system = 1` rows:

| Prefix | Required for |
|---|---|
| `autoconfig` | Thunderbird and K-9 Mail auto-configuration. Clients fetch `https://autoconfig.<domain>/mail/config-v1.1.xml`. |
| `autodiscover` | Outlook and iOS Mail auto-configuration. Clients POST to `https://autodiscover.<domain>/autodiscover/autodiscover.xml`. |

Both rows have **Delete** suppressed and the System badge displayed.
The action handler re-checks `system = 1` server-side and refuses
with error 13 if a crafted POST tries to bypass the missing button.
Removing either prefix would break client auto-discovery globally
across every mailbox domain — they are non-optional.

## Prefix validation rules

The Add form enforces DNS-label syntax both client-side
(`pattern="[a-z][a-z0-9-]*"` + `maxlength="63"`) and server-side
(`REFind("^[a-z][a-z0-9-]{0,62}$", ...)`):

- **Lowercase letters, numbers, and hyphens only.** No uppercase, no
  underscores, no dots. Each prefix is a **single** DNS label;
  multi-label SANs (`internal.mail.example.com`) are not supported
  here.
- **Must start with a letter.** Leading digits and leading hyphens
  are rejected per the DNS label spec.
- **Max 63 characters.** Each DNS label is capped at 63 octets.
- **Lowercased on save.** Submitting `Mail` stores as `mail`.

Suggested prefixes from the placeholder text: `mail`, `imap`, `smtp`,
`pop`, `webmail`. Pick whichever match the client-facing hostnames
you've published in DNS; the prefix only does work if a matching DNS
A/CNAME record exists pointing at this server.

## The Let's Encrypt budget callout

The page surfaces a live calculation of the cert budget per domain:

```
Let's Encrypt SAN limit: Each domain certificate supports a maximum
of 100 SANs. With <N> prefixes configured, each domain's certificate
uses <N + 1> SANs (1 for the domain + N prefixes), leaving room for
up to <99 - N> additional prefixes.
```

The +1 accounts for the bare domain itself, which is always included
on the cert regardless of prefix list (this is hardcoded in the ACME
request path).

Other Let's Encrypt rate limits that don't show on this page but
still apply:

| Limit | Value |
|---|---|
| **SANs per certificate** | 100 |
| **Certificates per registered domain per week** | 50 |
| **Duplicate certificates per week** | 5 |
| **Failed validation attempts per account, per hostname, per hour** | 5 |

A misconfigured DNS record (SAN row stuck at `dns = NO`) does **not**
burn the duplicate-cert budget because the certbot run is gated on
the validator marking every SAN ready first. The validator's failed
DNS probes are free and run on Hermes-side resolvers, not Let's
Encrypt's.

## Validation challenge mechanics

ACME issuance uses **HTTP-01** by default. The certbot container
mounts `<repo>/config/hermes/var/www/html` at `/var/www/certbot` so
the challenge file lands where the live nginx vhost for the domain
already serves `/.well-known/acme-challenge/`. The domain's nginx
vhost (generated by `generate_nginx_configuration.cfm`) is therefore
required to be up and serving HTTP on port 80 of the public IP that
the SAN resolves to.

DNS-01 (TXT-record validation) is **not** wired into this UI. The
underlying certbot container supports it but the request path here
hardcodes `--webroot`. Internal-only / DNS-only SANs (subdomains
that resolve to an internal IP but should still be on the public
cert) need either a manual certbot invocation or a public split-DNS
record pointing at the gateway's WAN address — there is no
DNS-challenge bypass on this page.

The validator's `ip = YES` check is **separate from** the ACME
challenge — it confirms that the SAN's DNS A record points at this
gateway's expected IP (which is what `https://verify.hermesseg.io`
returns when probed). It exists to catch broken DNS before burning a
Let's Encrypt rate-limit slot, not to perform the ACME challenge
itself.

## How SAN status surfaces elsewhere

This page edits the prefix list; the per-SAN validation state and
the per-cert SAN sub-table show up on other pages:

| Where | What it shows |
|---|---|
| [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) **Cert Status** column | Per-domain aggregate: `Verified` (all SANs ip+dns=YES), `Partial`, `Awaiting Cert`, `Pending`, `DNS Failed`, `No SANs`, `No Cert`. Imported certs always render `Imported` regardless of probe state because probes are informational only for those. |
| [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) expanded row **Mailbox SAN Validation** sub-table | Per-cert listing: every SAN bound to the cert, with its `ip_result_msg` / `dns_result_msg` / timestamps. Read-only here. |
| [System Certificates § Generate CSR](https://docs.deeztek.com/books/administrator-guide/page/system-certificates#3-generate-csr) — Mailbox certificate purpose | The CSR generator pre-fills the SAN list from `additional_sans` x the chosen mailbox domain. Refuses to generate a mailbox CSR if `additional_sans` is empty (impossible in practice because the two system prefixes can't be deleted). |
| `smtp_sni_generate_config.cfm` (run from Email Server > Settings) | Reads `mailbox_sans WHERE dns = 'YES'`, builds Postfix's `sni_maps`, runs `postmap -F`. Postfix then serves the per-domain cert on `:25`/`:587` via SNI based on the client's TLS SNI extension. |
| `generate_nginx_configuration.cfm` (run from Domains) | Reads validated `mailbox_sans` rows to write per-SAN nginx `server` blocks (autoconfig, autodiscover, DAV). |

## Failure semantics

| What breaks | What happens |
|---|---|
| Prefix blank | `session.m = 10`, redirect, no DB write |
| Prefix fails DNS-label regex | `session.m = 11`, redirect, no DB write |
| Prefix already in `additional_sans` | `session.m = 12`, redirect, no DB write |
| Delete attempted on a `system = 1` prefix | `session.m = 13`, redirect, no DB write |
| Delete with non-numeric `delete_san_id` | `session.m = 20`, redirect |
| `sync_mailbox_sans.cfm` fails mid-cross-join | Partial `mailbox_sans` state possible; re-saving any mailbox domain or re-adding the same prefix triggers another sync that converges |
| Validator can't reach `verify.hermesseg.io` | `mailbox_sans.ip` stays at the previous value; cert request gated until next successful probe. Validator runs hourly. |
| `acme_request_san_certificate.cfm` fails (DNS, port 80, rate limit) | Postmaster email sent with certbot stderr; SAN rows retain validation state; admin can re-trigger by toggling the cert binding on Domains |
| `smtp_sni_generate_config.cfm` finds zero validated SANs | Deletes `/etc/postfix/sni_maps` and `.db` — Postfix falls back to its default cert on every connection. Non-fatal but clients lose per-domain SNI. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_mailbox_sans.cfm` | `hermes_commandbox` | Page + Add card + Delete modal + LE budget callout |
| `config/hermes/var/www/html/admin/2/inc/san_actions.cfm` | `hermes_commandbox` | Add / Delete handler — validates, writes `additional_sans`, calls sync |
| `config/hermes/var/www/html/admin/2/inc/sync_mailbox_sans.cfm` | `hermes_commandbox` | Cross-joins prefixes x mailbox domains into `mailbox_sans`; idempotent |
| `config/hermes/var/www/html/admin/2/inc/acme_request_san_certificate.cfm` | `hermes_commandbox` | Pro — runs ephemeral certbot container for SAN-bearing certs |
| `config/hermes/var/www/html/admin/2/inc/smtp_sni_generate_config.cfm` | `hermes_commandbox` | Pro — builds Postfix `sni_maps` from validated SANs |
| `config/hermes/var/www/html/admin/2/inc/generate_nginx_configuration.cfm` | `hermes_commandbox` | Per-domain nginx vhost generator (called from Domains; consumes validated SANs) |
| `config/hermes/var/www/html/schedule/acme_validate_ip.cfm` | `hermes_commandbox` (Ofelia) | Pro — hourly validator; probes each SAN's IP via `verify.hermesseg.io` and updates `mailbox_sans.ip` / `dns` |
| `additional_sans` table | `hermes_db_server` (`hermes` DB) | The prefix list this page edits |
| `mailbox_sans` table | `hermes_db_server` (`hermes` DB) | Per-SAN rows with validation state and cert binding |
| `system_certificates` table | `hermes_db_server` (`hermes` DB) | Per-cert metadata referenced via `mailbox_sans.certificate` |
| `/etc/letsencrypt/live/<domain>/` | `hermes_commandbox` (bind-mounted from `config/certbot/conf/`) | Issued SAN certs |
| `/etc/postfix/sni_maps` + `.db` | `hermes_postfix_dkim` (mounted) | Live SNI map — Postfix serves per-domain cert based on this |
| `/etc/postfix/sni/*.pem` | `hermes_postfix_dkim` (mounted) | Combined key + fullchain PEM per cert, referenced from `sni_maps` |
| Per-SAN nginx vhost files | `hermes_nginx` (mounted) | One vhost per validated SAN |
| `certbot/certbot:latest` image | docker.io | Pulled on demand for SAN cert issuance + renewal |
| `verify.hermesseg.io` | external (Pro) | Returns expected IP for a given SAN to gate ACME issuance |

Every certbot invocation is `docker run --rm` against the public
`certbot/certbot:latest` image — same pattern as the single-domain
ACME path on [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates).
The container shares the host network (`--network host`) so the
HTTP-01 challenge can reach port 80 on the public IP.

## Related

- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — the
  certificate store these SANs land on. The Mailbox certificate
  purpose on Generate CSR auto-fills its SAN list from this page;
  Pro's auto-managed ACME path mints SAN certs from the same source.
- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — per-mailbox-domain Cert Status column
  summarizes the per-SAN validation state this page's prefixes drive.
  Adding a domain calls `sync_mailbox_sans.cfm`, so new SANs appear
  immediately under existing prefixes.
- [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — mailbox users hit IMAP/Submission via
  the `imap`/`mail`/`smtp` prefixes configured here. Apple iOS and
  Outlook reach autodiscover via the system prefixes.
- [Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) — Dovecot IMAP/POP TLS is gated on the
  validated mailbox cert; the SNI map for Postfix is generated from
  the same `mailbox_sans` table this page populates.
- [Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases) / [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes) —
  both ride on the same per-domain cert; no separate SAN entries
  needed.
- [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) — binds the
  **single** cert Postfix presents on the public SMTP banner. The
  SNI map this page feeds into is an **additional** layer that
  overrides the banner cert when the client sends a matching SNI
  hostname.
- [Email Relay > Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients)
  — relay recipients use Submission via the same `mail.<domain>`
  hostnames as local mailboxes; the SAN prefixes here cover both
  topologies.

# Settings

# Settings

Admin path: **Email Server > Settings** (`view_email_server_settings.cfm`,
`inc/email_server_settings_action.cfm`,
`inc/generate_dovecot_configuration.cfm`,
`inc/generate_mail_crypt_keys.cfm`).

This page is the **global configuration surface for the Email Server
topology** — the half of Hermes where Hermes is itself the destination
MTA, delivering inbound mail into Dovecot mailboxes on `/mnt/vmail` and
serving IMAP/POP3/Submission/Sieve back to end users. Per-domain
addressing lives on [Email Server > Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v), per-mailbox
quotas and personal info on [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes), and aliases on
[Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases); this page handles everything that applies
across all mailboxes regardless of domain — the Dovecot TLS profile,
mail compression and encryption-at-rest, which protocols are exposed,
quota warning thresholds, connection limits, debug logging, the
Nextcloud login-form mode that gates webmail SSO, and the master
toggle for shared mailboxes and folder sharing.

Most pages save and run a small handful of `docker exec` commands.
This page saves and re-renders the entire Dovecot configuration from
a template; the next inbound LMTP delivery sees the new settings.

## What this page does — and what it doesn't

| This page configures | This page does NOT configure |
|---|---|
| Dovecot TLS certificate, profile, ciphers, min protocol | LDAP authentication backend (hard-coded against `hermes_ldap`) |
| Mail compression (LZ4 / Zstd / Zlib) | Per-mailbox quota size (set on [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes)) |
| Mail encryption at rest (mail_crypt plugin + ECC key pair) | Per-domain delivery / acceptance (handled by [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v)) |
| IMAP and POP3 enable/disable | Submission, Sieve, LMTP enable (always on — required for core operation) |
| Quota warning thresholds (medium / high / critical / trash overage) | Default new-mailbox size (set per-mailbox; see [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes)) |
| Per-service client limit + per-user-per-IP connection cap | Postfix-side recipient validation (handled by Postfix `relay_recipient_maps`) |
| Dovecot debug logging | Authelia session timing, MFA enrollment, SMTP notifier ([Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)) |
| Mailbox sharing master toggle (Shared/ namespace + user folder shares) | Per-user shared mailbox access (handled by [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes)) |
| Nextcloud login form mode (auto-redirect / SSO-only / full form) | Nextcloud OIDC client itself ([Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)) |

## Configuration storage

Almost every setting on this page is keyed into `parameters2` under
`module = 'dovecot'` and read back by both the page and
`generate_dovecot_configuration.cfm` at render time. A handful of
adjacent concerns live in sibling modules:

| Settings group | Storage |
|---|---|
| All Dovecot directives (compression, encryption, protocols, quota, connections, logging, sharing, TLS profile/ciphers) | `parameters2` rows where `module = 'dovecot'`, keyed by dotted names like `mail.compression_algorithm`, `quota.warning_critical`, `ssl.min_protocol` |
| TLS certificate selection | `parameters2` row `module = 'certificates'`, `parameter = 'mail.certificate'`, value = `system_certificates.id` |
| Nextcloud login-form mode | `parameters2` row `module = 'nextcloud'`, `parameter = 'oidc.auto_redirect'`, value = `auto_redirect` / `sso_only` / `full_form` (legacy `true`/`false` strings normalized on read) |
| Mail encryption key pair | Files at `/opt/hermes/keys/ecprivkey.pem` and `/opt/hermes/keys/ecpubkey.pem` on the Docker host |
| Live Dovecot config | `/etc/dovecot/dovecot.conf` (regenerated from `/opt/hermes/templates/dovecot.conf` on every save) |

`parameters2` is keyed by the `module + parameter` pair. The action
handler uses an upsert pattern (`checkDovParam` → UPDATE-or-INSERT) so
fresh installs that haven't yet had the schema seeded with every row
land cleanly on first save.

## How a save propagates

```
form submit  ──► email_server_settings_action.cfm
                       │
                       │  1. validate + sanitize (whitelist enums,
                       │     clamp numeric ranges, normalize booleans)
                       │
                       │  2. Nextcloud login-form mode
                       │     - UPDATE/INSERT parameters2 (oidc.auto_redirect)
                       │     - docker exec hermes_nextcloud occ
                       │         config:app:set user_oidc
                       │         allow_multiple_user_backends = 0|1
                       │     - docker exec hermes_nextcloud occ
                       │         config:system:set/delete hide_login_form
                       │
                       │  3. Dovecot TLS cert
                       │     - verify system_certificates row exists
                       │     - UPDATE/INSERT parameters2 (mail.certificate)
                       │
                       │  4. Mail encryption key generation (if enabled
                       │     AND keys missing OR zero-byte)
                       │     - cfinclude generate_mail_crypt_keys.cfm
                       │     - openssl ecparam + ec via docker exec
                       │     - writes /opt/hermes/keys/ecprivkey.pem
                       │             /opt/hermes/keys/ecpubkey.pem
                       │
                       │  5. Dovecot settings batch upsert
                       │     - loop the dovSettings struct
                       │     - UPDATE-or-INSERT each parameters2 row
                       │
                       │  6. cfinclude generate_dovecot_configuration.cfm
                       │     - reads /opt/hermes/templates/dovecot.conf
                       │     - substitutes placeholders from parameters2
                       │     - writes /etc/dovecot/dovecot.conf
                       │     - docker exec hermes_dovecot dovecot reload
                       │
                       v
            cflocation → session.m = 1 (success) or 10 (per-step errors)
```

Validation lives entirely in the action handler. Each step is wrapped
in its own `cftry` so a failure in (e.g.) the Nextcloud `occ` step
accumulates into `session.saveErrors` but doesn't abort the Dovecot
save. Step 6 — the Dovecot regen — gates on `NOT saveError` so a
broken upstream step doesn't push a half-rendered config file.

## Cards on the page

### Nextcloud Webmail Settings

Single dropdown that controls the Nextcloud login page behavior.
Three modes — chosen because two underlying Nextcloud knobs
(`user_oidc.allow_multiple_user_backends` and the system-wide
`hide_login_form`) compose into three meaningful states:

| Mode | `allow_multiple_user_backends` | `hide_login_form` | User experience |
|---|---|---|---|
| **Auto-redirect to SSO** (default) | `0` | (unset) | Clicking "Login to Webmail" silently bounces through Authelia OIDC and lands the user in Nextcloud already authenticated. True SSO — no Nextcloud login page is ever shown. |
| **SSO button only** | `1` | `true` | The Nextcloud login page is shown but with the username/password fields hidden — only the SSO button is visible. Good when you want users to know SSO is required but don't want to auto-redirect. |
| **Show full form** | `1` | (unset) | Both the username/password form and the SSO button are shown. Use temporarily for local Nextcloud admin maintenance. |

The legacy storage key `oidc.auto_redirect` is reused as the slot for
this three-way value so existing installs don't need a migration. The
read path in `view_email_server_settings.cfm` normalizes legacy
`true`/`false` strings to `auto_redirect` / `full_form`.

## Nextcloud Maintenance Mode card

Below the Webmail Settings card sits a second card that controls the local-admin escape hatch. As of [#262](https://github.com/deeztek/Hermes-Secure-Email-Gateway/issues/262) there is **no permanent bypass URL** &mdash; the operator toggles OIDC on/off from this card when they need to administer Nextcloud as the local admin (separate identity from the Authelia/LDAP users that normally SSO in).

| State | What it means |
| --- | --- |
| `OIDC ENABLED` (green) | Normal operation. Mailbox users SSO into Nextcloud via Authelia. The local NC admin **cannot** log in. |
| `MAINTENANCE MODE` (yellow) | Click "Enter Maintenance Mode" ran `occ app:disable user_oidc`. Mailbox-user SSO is offline. The local NC admin can now log in via Nextcloud's own form at `/nc/`. |

**Maintenance procedure:**

1. Click **Enter Maintenance Mode**. The card status flips to yellow, mailbox-user SSO goes offline, and a success banner appears at the top of the page.
2. Click the **Open Nextcloud** button that appears below the toggle &mdash; it opens `https://<console-host>/nc/` in a new tab (`target="_blank"`) so the Hermes admin tab stays put for step 7.
3. In the Nextcloud tab, log in as the NC local admin. Username is shown on the card; password is also in `/opt/hermes-seg-container-gl/INSTALL_SUMMARY.txt` on the host.
4. On first login Nextcloud prompts for TOTP enrollment via its own UI &mdash; scan the QR code with any TOTP authenticator app.
5. **First login only &mdash; generate backup codes immediately**. Click your avatar (top-right) &rarr; **Personal settings** &rarr; **Security**, scroll to **Two-Factor backup codes**, click **Generate backup codes**. Save the 10 single-use codes somewhere safe (password manager, printed copy in a safe, etc.). These codes are the ONLY recovery path if you lose your TOTP authenticator &mdash; without them, recovery requires shell access. Done once per admin; codes persist across sessions until used.
6. Do your admin work in Nextcloud.
7. Switch back to the Hermes admin tab and click **Exit Maintenance Mode**. SSO is restored for mailbox users.

The button uses `fetch()` to call `inc/edit_nc_oidc_action.cfm` (`occ app:disable user_oidc` or `enable`), bypassing the outer settings form so the toggle doesn't collide with a normal Save submission. `redirect: 'manual'` on the fetch prevents the action handler's `cflocation` from being auto-followed and consuming the `session.m` flash before the page can render it.

Operators who need to use this often can ignore step 2's helper link and just type `/nc/` &mdash; the helper link exists to make first-time use obvious.

**Why the toggle pattern and not a permanent bypass URL:**

Earlier attempts at a permanent local-admin URL (the `/nc-admin-login` path) were architecturally infeasible. The Authelia session created by gating that URL fueled `user_oidc` silent OIDC re-auth on every post-form `/nc/` request, overriding whatever local-admin session the form submission had just established. Removing the Authelia gate didn't help either because `user_oidc` itself force-redirects `/login?direct=1` to OIDC under several conditions. The toggle is the only path that reliably wins against `user_oidc`, and it's what most NC operators in OIDC-fronted deployments use anyway. See #262 for the full diagnostic trace.

**Recovery if the NC local admin loses their TOTP authenticator:**

1. **Preferred &mdash; backup codes** (generated at TOTP enrollment time per step 5 of the maintenance procedure above). At the TOTP prompt during login, click **"Use backup code"** (or **"Try another method"**, wording varies by NC version), paste one of the saved codes. Each code is single-use, so re-generate a new set after recovery via Personal &rarr; Security &rarr; Two-Factor backup codes.

2. **Fallback &mdash; disable enforcement via shell** (only if backup codes are also lost or were never generated):

   ```bash
   docker exec hermes_nextcloud php occ twofactorauth:enforce --off
   # log in, re-enroll TOTP via NC UI, generate fresh backup codes, then:
   docker exec hermes_nextcloud php occ twofactorauth:enforce --on
   ```

   This requires shell access to the Hermes host. If you don't have shell access, the only recovery is restoring `/mnt/data/dbase/` from a backup taken when the admin still had TOTP access, which is a significantly more disruptive operation. Generating backup codes at enrollment time is much cheaper.

### Mailbox Sharing

Single dropdown — Enabled or Disabled. Stored as `sharing.enabled` in
`parameters2`.

| State | Dovecot effect |
|---|---|
| **Enabled** | Shared mailbox support is compiled into the Dovecot config (`acl`, `imap_acl`, `imap_quota` plugins and the `Shared/` namespace). Per-mailbox shares are then managed under [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes). Folder-level user-managed shares work in IMAP clients that support them. |
| **Disabled** | The shared namespace is not declared in the Dovecot config and IMAP clients won't see a `Shared/` folder. Existing per-mailbox ACL entries are preserved in their backing files but are inactive until sharing is re-enabled. |

Toggling this is the master switch. The per-mailbox setup work happens
on [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes).

### TLS / SSL Settings

The cert that Dovecot presents on every IMAPS / POP3S / submission
connection. Driven by:

| Field | Notes |
|---|---|
| **Mail Server Certificate** | Autocomplete against `system_certificates` (via `inc/getcertificates.cfm`). Selecting a row populates the four read-only fields below and writes the cert `id` into `parameters2`. Manage certificates on [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates). |
| **TLS Security Profile** | `Modern` (TLS 1.3 only) / `Intermediate` (TLS 1.2+, recommended) / `Legacy` (TLS 1.2+, broad compatibility) / `Custom`. Presets follow [Mozilla Server Side TLS](https://ssl-config.mozilla.org/) guidance. |
| **Minimum TLS Version** | Auto-set by profile (read-only) when a preset is selected; editable in Custom mode. |
| **SSL Cipher List** | Auto-set by profile (read-only) when a preset is selected; editable in Custom mode. The page's JS form-submit hook re-enables disabled fields before submit so their values are POSTed. The action handler's `cfswitch` then re-derives the canonical preset values defensively so the saved values always match the named profile. |

`Intermediate` is the default and the only profile that ships with a
non-empty cipher list. `Modern` deliberately leaves the cipher field
empty because OpenSSL picks TLS 1.3 ciphers automatically.

### Mail Storage — Compression

| Field | Notes |
|---|---|
| **Mail Compression** | Enabled / Disabled. When Disabled, the algorithm and level fields are JS-disabled. |
| **Algorithm** | `LZ4` (fastest, good compression) / `Zstandard` (balanced) / `Zlib/Deflate` (best ratio, slowest). LZ4 is the default. |
| **Compression Level** | Numeric. Hidden for LZ4 (no level knob). 1–22 for Zstandard (default 3), 1–9 for Zlib (default 6). The handler enforces the Zlib ceiling — Zlib with level > 9 is clamped to 6. |

Compression is mailbox-format aware: only newly delivered or saved
messages are compressed, existing messages remain readable, and Dovecot
auto-detects the format per message on read. Changing or disabling
compression never breaks existing mail; mailboxes safely contain a mix
of uncompressed, LZ4, and Zstandard messages.

### Mail Storage — Encryption at Rest

Dovecot's `mail_crypt` plugin with an EC-curve key pair stored on the
Docker host. **This is irreversible-ish — back up the keys.**

| Field | Behavior |
|---|---|
| **Encryption at Rest** | Disabled (default) / Enabled. Saving with Enabled and no key pair triggers `generate_mail_crypt_keys.cfm`, which runs `openssl ecparam` + `openssl ec` via `docker exec hermes_dovecot` to write `/opt/hermes/keys/ecprivkey.pem` and `ecpubkey.pem`. |
| **Elliptic Curve** | `prime256v1` / `secp384r1` / `secp521r1`. Selectable only when no keys exist yet — once keys are generated the field is rendered as a read-only display because changing curves with mismatched keys would render existing encrypted mail unreadable. |
| **Algorithm** | Always `AES-256-GCM`. Not configurable. |
| **Key Status** | Badge: `Keys Present` (green), `Keys Empty` (red — files exist but zero-byte from a failed previous attempt; delete from the host to regenerate), or `No Keys` (gray — auto-generated on enable). |

> **Operational consequence.** Only newly delivered mail is encrypted.
> Disabling encryption later does not affect existing encrypted
> messages — they remain readable as long as the keys are present.
> If the keys are lost there is no recovery mechanism; encrypted mail
> becomes permanently unreadable. The two PEM files belong in every
> system backup. The system-backup script collects `/opt/hermes/keys/`
> automatically, but operators running off-Hermes backup tooling must
> include this directory explicitly.

### Protocols & Connections — Protocols

Per-protocol enable/disable for the end-user-facing services.
**Submission, Sieve, and LMTP are always enabled** — Submission for
authenticated outbound and vacation responder, Sieve for mail filter
rules, LMTP for Postfix-to-Dovecot delivery — and surface in the UI
as read-only `Always Enabled` fields.

| Protocol | Ports | Knob |
|---|---|---|
| IMAP | 993 / 143 | `protocol.imap` — Enabled / Disabled |
| POP3 | 995 / 110 | `protocol.pop3` — Enabled / Disabled |
| Submission | 587 | Always on |
| Sieve / LMTP | 4190 / 24 | Always on |

Disabling IMAP or POP3 takes effect on the next Dovecot reload — the
service is dropped from `protocols = ...` in `dovecot.conf` and the
listener stops.

### Protocols & Connections — Connection Limits

| Field | Default | Notes |
|---|---|---|
| **Login Service Client Limit** | `1000` | Max concurrent connections per login service (IMAP, POP3, Submission, ManageSieve). Clamped 100–10000. Increase for installs with many simultaneous users. |
| **Max Connections per User per IP** | `20` | Per-user-per-source-IP cap. Stops a runaway client from consuming the global pool. Clamped 1–1000. Bump for users with many devices / many open folders. |

### Quota Settings — Warning Thresholds

When a mailbox crosses these usage thresholds, Dovecot's quota-warn
hook sends an email notification. A "back under quota" notice is
always sent when usage drops below 100% — that one is not configurable.
**Per-mailbox quota sizes are set per-mailbox** on
[Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes); this card only controls the warning bands.

| Field | Default | Range |
|---|---|---|
| Critical Warning | `99` % | 1–100. Triggers the "Mailbox Full" notification. |
| High Warning | `95` % | 1–100. Triggers the "Nearly Full" notification. |
| Medium Warning | `80` % | 1–100. Triggers the first warning notification. |
| Trash Quota Overage | `110` % | 100–200. The Trash folder is allowed this percentage of the user's quota so users can still delete messages when they're at 100%. Default leaves 10% headroom in Trash. |

### Logging

| Field | Notes |
|---|---|
| **Debug Logging** | Disabled (production, default) / Enabled (troubleshooting). When Enabled, Dovecot's `mail_debug = yes` and `auth_debug = yes` are emitted. Output lands in `/logs/dovecot-debug.log` inside the container. Significant log volume — leave off in production. |

## Failure semantics

| What breaks | What happens |
|---|---|
| Nextcloud `occ` step fails (container down, OIDC app not installed) | Per-error message appended to `session.saveErrors`, banner shown at top of page, **other steps still run** |
| TLS cert id doesn't match a `system_certificates` row | `parameters2 mail.certificate` is not updated; Dovecot keeps using whatever cert was previously selected |
| `generate_mail_crypt_keys.cfm` fails | Per-error message appended; encryption may be enabled in DB but keys missing — admin sees the Keys Empty badge on the next page load, must clear the partial files and retry |
| Dovecot config regen fails (template missing, substitution error) | `session.m = 10`, error banner with the cfcatch message; **the previous `dovecot.conf` is still on disk** because the template renderer writes to a temp path and atomically moves only on success |
| `dovecot reload` fails | The new config is on disk but the running Dovecot is still on the old config. Recovery is `docker exec hermes_dovecot dovecot reload` from the host or a container restart. |
| Encryption keys deleted from host while encryption is enabled | New incoming mail cannot be encrypted; Dovecot logs the failure and the LMTP delivery is deferred. Existing encrypted mail remains unreadable until the keys are restored from backup. |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_email_server_settings.cfm` | `hermes_commandbox` | Page + cards |
| `config/hermes/var/www/html/admin/2/inc/email_server_settings_action.cfm` | `hermes_commandbox` | Save handler |
| `config/hermes/var/www/html/admin/2/inc/generate_dovecot_configuration.cfm` | `hermes_commandbox` | Template-to-`dovecot.conf` renderer + `dovecot reload` |
| `config/hermes/var/www/html/admin/2/inc/generate_mail_crypt_keys.cfm` | `hermes_commandbox` | EC key pair generator |
| `config/hermes/var/www/html/admin/2/inc/getcertificates.cfm` | `hermes_commandbox` | Autocomplete for the Mail Server Certificate field |
| `/opt/hermes/templates/dovecot.conf` | `hermes_commandbox` | Dovecot template |
| `/etc/dovecot/dovecot.conf` | `hermes_dovecot` (volume-mounted) | Live Dovecot config (regen target) |
| `/opt/hermes/keys/ecprivkey.pem`, `ecpubkey.pem` | `hermes_dovecot` (volume-mounted) | mail_crypt key pair |
| `parameters2` rows where `module IN ('dovecot','certificates','nextcloud')` | `hermes_db_server` | Settings storage |
| `system_certificates` | `hermes_db_server` | TLS certificate lookup |
| `hermes_nextcloud` container | — | `occ config:app:set user_oidc allow_multiple_user_backends`, `occ config:system:set/delete hide_login_form` |

Every shell-out uses `docker exec hermes_dovecot ...` or
`docker exec hermes_nextcloud ...` per the standard Hermes pattern.

## Related

- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — per-domain configuration for the mailbox
  topology. Add a domain there first; this page's settings then apply
  to every mailbox on every domain.
- [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — per-mailbox quota size, personal info,
  encryption opt-in. The quota size set per-mailbox is what the
  warning thresholds on this page measure against.
- [Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases) — alias addresses that resolve to local
  mailboxes. The Email Server alternative to
  [Email Relay > Virtual Recipients](https://docs.deeztek.com/books/administrator-guide/page/virtual-recipients).
- [Shared Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/shared-mailboxes) — per-mailbox shared-access
  configuration. The master switch on this page must be on for any
  shared mailbox to function.
- [Mailbox Rules](https://docs.deeztek.com/books/administrator-guide/page/mailbox-rules) — server-side Sieve rules per
  mailbox; Sieve is always-on at the protocol level via this page.
- [SAN Management](https://docs.deeztek.com/books/administrator-guide/page/san-management) — Subject Alternative Names on
  the Dovecot TLS certificate. The cert selected on this page is the
  one SAN Management edits.
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) —
  managing the certificate inventory that the Mail Server Certificate
  autocomplete draws from.
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)
  — Authelia, the OIDC client, and the Nextcloud-side session-lifetime
  knobs that complement the login-form mode dropdown on this page.

# Shared Mailboxes

# Shared Mailboxes

Admin path: **Email Server > Shared Mailboxes** (`view_shared_mailboxes.cfm`,
`inc/shared_mailbox_actions.cfm`, `inc/sync_shared_mailbox_acl_file.cfm`,
`inc/sync_user_folder_acl_file.cfm`, `inc/get_shared_mailbox_permissions_json.cfm`).

This page manages **mailboxes that several users can read from and
write to** — typically role addresses like `info@`, `support@`, or
`sales@`. A shared mailbox is a real Dovecot mailbox in its own
Maildir, but it has no login of its own; users access it through
their own credentials and the rights granted on this page. The
**master switch** for the entire shared-mailbox feature lives on
[Email Server > Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) (Mailbox Sharing card) — when
that switch is off, the rows on this page are preserved but inactive,
and the Add / Manage Permissions / Rebuild buttons are disabled.

Per-member rights are stored in the `shared_mailbox_permissions`
table and projected to Dovecot's on-disk `dovecot-acl` files via the
vfile driver, which is the only per-mailbox ACL driver shipped with
Dovecot 2.4 (the SQL rights driver was a non-upstream Hermes carry
that was removed in the 2.4 rewrite).

## How a shared mailbox is wired

A shared mailbox is more than just an ACL — six tables and a Maildir
are stitched together on creation:

| Component | Storage | Role |
|---|---|---|
| Mailbox row | `mailboxes` with `mailbox_type = 'shared'` | Gives Dovecot a userdb entry so the mailbox has a quota, a Maildir, and a sender identity |
| Shared mailbox row | `shared_mailboxes` | UI metadata: address, display name, auto-subscribe flag, owning domain |
| Per-member rights | `shared_mailbox_permissions` | Authoritative permission matrix per (shared mailbox, user mailbox) pair |
| On-disk ACL | `/srv/mail/<domain>/<local>/dovecot-acl` | Dovecot vfile driver enforcement file — projected from `shared_mailbox_permissions` |
| Shared namespace visibility | `dovecot_acl_shared` (`acl_sharing_map`) | Tells Dovecot's `Shared/` namespace which users should see this mailbox in their folder list |
| Recipient policy | `recipients` (Amavis SVF policy + `recipient_type = 'shared'`) | Allows mail addressed to the shared address to pass the Amavis recipient gate |
| Sender identity | `sender_login_maps` | Lets the shared address be used as a From: by itself (anchor row) and by each member with Send-As granted |
| Maildir | `/srv/mail/<domain>/<local>/` | The actual on-disk message store. Bootstrapped via `doveadm mailbox create -u <addr> INBOX` so members see it immediately rather than waiting for first delivery |

The add handler creates all of these in a single `cftry` block. If
any step fails the catch sets `session.m = 30` and the operation
fails-loud rather than leaving a partial mailbox.

## Permission model — seven flags, projected to IMAP ACL letters

The UI surfaces seven permission flags. Six are IMAP ACL rights
enforced by Dovecot; one (Send-As) is a Postfix sender-identity grant.

| UI flag | DB column | Dovecot vfile rights | IMAP ACL meaning |
|---|---|---|---|
| Read | `can_read` | `lrs` | `lookup` (see mailbox), `read` (read messages), `write-seen` (set/clear \Seen flag) |
| Write | `can_write` | `wt` | `write` (set/clear flags except \Seen and \Deleted), `write-deleted` (set/clear \Deleted) |
| Delete | `can_delete` | `e` | `expunge` (permanently remove messages) |
| Insert | `can_insert` | `i` | `insert` (append/copy messages into mailbox) |
| Post | `can_post` | `p` | `post` (submit messages via the post address — rarely used) |
| Admin | `can_admin` | `a` | `admin` (modify the ACL itself from an IMAP client) |
| Send-As | `send_as` | — | Inserts `(sender = shared, login_user = member)` into `sender_login_maps` so the member can use the shared address as From: |

The vfile letters are concatenated into a single token per user
(e.g., `lrswtie` for read+write+delete+insert). Dovecot 2.4's vfile
parser reads each character as a separate right, so the full-word
form (`lookup read write-seen ...`) does NOT work — the parser would
treat `o` in `lookup` as an unknown right. The
`sync_shared_mailbox_acl_file.cfm` include knows this and emits the
single-letter form.

The `dovecot_acl` SQL table is still written by the action handlers
for legacy/audit reasons, but Dovecot 2.4 no longer reads it.
`sync_shared_mailbox_acl_file.cfm` writes the on-disk file every time
permissions change, and the **Rebuild ACL Files** button on the page
regenerates every file from scratch — used after upgrading to a new
Dovecot release or when an admin reports a member can't see a
mailbox they should have rights on.

## How a save propagates

```
Add Shared Mailbox  ──► shared_mailbox_actions.cfm (add_shared_mailbox)
                              │
                              │  1. Feature guard (Mailbox Sharing = enabled)
                              │  2. Validate prefix + domain + display name + quota
                              │  3. Four-way conflict check
                              │     (recipients, mailboxes, mailbox_aliases,
                              │      virtual_recipients)
                              │  4. INSERT into recipients (Amavis SVF policy)
                              │     + maddr (Amavis address tracking)
                              │  5. INSERT into mailboxes (mailbox_type='shared')
                              │  6. INSERT into shared_mailboxes
                              │  7. INSERT into sender_login_maps (anchor row)
                              │  8. docker exec hermes_dovecot doveadm mailbox
                              │     create -u <addr> INBOX  (bootstrap Maildir)
                              │  9. For each initial member:
                              │     - INSERT shared_mailbox_permissions
                              │     - INSERT dovecot_acl (legacy)
                              │     - INSERT dovecot_acl_shared (namespace)
                              │     - INSERT sender_login_maps if Send-As
                              │ 10. cfinclude sync_shared_mailbox_acl_file.cfm
                              │     → writes /srv/mail/<dom>/<local>/dovecot-acl
                              │       via temp shell script + docker exec -i
                              │       (heredoc pattern; vmail:vmail 0660)
                              v
                  cflocation → session.m = 1
```

Add / Edit / Remove permission flows follow the same shape but only
touch the rows for one member, then re-call
`sync_shared_mailbox_acl_file.cfm` to rebuild that mailbox's
`dovecot-acl` file in place. The sync include uses the **temp shell
script + heredoc + `docker exec -i`** pattern (it has to — Lucee
`cfexecute` argument quoting can't reliably ship multiline content
with embedded special characters through `docker exec`).

## Cards and modals on the page

### Add Shared Mailbox modal

| Field | Notes |
|---|---|
| **Domain** | Dropdown of mailbox-type domains (`domains.type = 'mailbox'`). The Address Prefix suffix updates live to show the full address. |
| **Address Prefix** | Local-part of the email. Validated against `^[a-z0-9._-]+$` — only lowercase letters, digits, dots, hyphens, underscores. |
| **Display Name** | Free-form text shown as the mailbox's `name` and in the table. Required. |
| **Quota (GB)** | Mailbox quota. Accepts decimals (e.g., `0.5`). Stored as bytes via `Round(quota_gb * 1024^3)`. |
| **Auto-Subscribe** | When `Yes` (default), the shared mailbox appears automatically in each member's IMAP folder list. When `No`, members have to manually subscribe to `Shared/<address>` in their client. |
| **Initial Members** | Checkbox list of user mailboxes in the selected domain (filtered live as the Domain dropdown changes). Optional — you can grant access later. |
| **Default Permissions** | Seven checkboxes applied uniformly to every selected initial member. Defaults are Read + Write + Insert checked. |

The address-prefix suffix and the member-list filter both run client-side
when the Domain dropdown changes. Cross-domain members are excluded
from the picker even before form submit; the server-side handler
re-enforces the same-domain rule with error 26 if a forged post tries
to bypass it.

### Shared Mailboxes table

DataTables surface — searchable, sortable, paginated, `stateSave: true`.

| Column | Source |
|---|---|
| Actions | Manage Permissions (opens modal) / Delete (opens confirmation modal) |
| Address | `shared_mailboxes.address` |
| Display Name | `shared_mailboxes.display_name` |
| Domain | `domains.domain` |
| Members | Count of `shared_mailbox_permissions` rows for this shared mailbox |
| Quota | `mailboxes.quota` divided into GB (1-decimal for whole GB, 2-decimal otherwise) |
| Auto-Subscribe | YES / NO badge |
| Status | `Active` (sharing on + mailbox active) / `Inactive` (sharing on + mailbox disabled) / `Inactive (Sharing Off)` (master switch off) |

A Domain filter dropdown narrows the visible rows to one domain.

### Manage Permissions modal

Opens via the per-row action button. Two sections:

1. **Current Members** — table of every `shared_mailbox_permissions`
   row for this shared mailbox, with per-right YES/NO badges and
   Edit / Remove buttons per row. Loaded via AJAX from
   `get_shared_mailbox_permissions_json.cfm`.
2. **Add Member** — Tom Select user picker (filtered to the same
   domain as the shared mailbox) + the seven permission checkboxes
   + an Add button.

The Edit Member sub-modal opens on top of the Manage Permissions
modal, lets you toggle the seven flags for an existing member, and
re-syncs the on-disk ACL file on save. Changes take effect
immediately; the member does not need to reconnect their mail client.

### Rebuild ACL Files modal

A maintenance action that walks **both** admin-managed shared
mailboxes AND user-managed folder shares and regenerates every
`dovecot-acl` file from the current state of the database.

> **When to use Rebuild ACL Files.**
> - After upgrading to a new Dovecot 2.4 release — backfills the
>   vfile files for any shared mailboxes created before the upgrade.
> - When a member reports they cannot see or access a shared
>   mailbox or shared folder they should have rights on (recovery /
>   drift heal).
> - After manually editing `shared_mailbox_permissions` or
>   `user_folder_shares` in the database.
>
> Safe to run anytime — it rebuilds files from the database and
> never modifies the permission rows themselves. Per-mailbox failures
> are non-fatal; the operation continues to the next.

The success banner reports a count of shared mailboxes rebuilt and a
separate count of user folder shares rebuilt, so the admin can
confirm the operation covered everything they expected.

### Delete Shared Mailbox modal

A confirmation modal that lists exactly what will be removed:

- All member permissions and ACL entries
- Sender login maps (send-as permissions)
- Dovecot shared folder subscriptions
- Amavis policy entry

With an optional **Also delete all email messages from the server**
checkbox (default checked) that, when set, runs
`docker exec hermes_dovecot rm -rf /srv/mail/<domain>/<local>` to
remove the Maildir. The DB rows are deleted regardless of that
checkbox; only the on-disk messages are conditional. Maildir deletion
is wrapped in a non-fatal `cftry` — failure leaves the messages on
disk for an admin to clean up later, but the DB state is correct.

## User-initiated folder shares — same engine, different page

Individual users can share folders from their own mailbox with other
users via the User Portal (`/users/2/`), and those shares land in
`user_folder_shares` rather than `shared_mailbox_permissions`. They
are projected to `dovecot-acl` files by `sync_user_folder_acl_file.cfm`
using the same vfile driver. The **Rebuild ACL Files** button on
this page rebuilds both types of share in one pass, so admins don't
have to think about the distinction when troubleshooting.

The two share types are otherwise independent:

| | Admin-managed shared mailbox | User-initiated folder share |
|---|---|---|
| Surface | This page | User Portal > Folder Sharing |
| Storage | `shared_mailboxes` + `shared_mailbox_permissions` | `user_folder_shares` |
| Underlying mailbox | A dedicated `mailboxes` row with `mailbox_type='shared'` | The owner's existing mailbox + a named folder path |
| Visibility namespace | `Shared/<address>/INBOX` | `Shared/<owner>/<folder_path>` |
| ACL file path | `/srv/mail/<dom>/<local>/dovecot-acl` | `/srv/mail/<owner-dom>/<owner-local>/<folder>/dovecot-acl` |
| Cleanup on member removal | This page's Remove Permission | Owner removes the share from User Portal |

## Cross-domain members — not supported, enforced server-side

A shared mailbox on `company.com` can only be shared with users whose
mailboxes are also on `company.com`. The same-domain rule is enforced
in three places:

1. **Add Shared Mailbox modal** — the Initial Members list is
   filtered client-side to the selected domain.
2. **Manage Permissions modal** — the Tom Select picker is repopulated
   on open to only show users in the shared mailbox's domain.
3. **`add_permission` action handler** — compares `getUserMailbox.domain_id`
   against `getShared.domain_id` and returns error 26 on mismatch,
   so a forged form post can't bypass the UI filter.

The Dovecot shared namespace itself does not enforce this — the
`acl_sharing_map` query keys on username, not domain — so the rule
is a UX contract, not a Dovecot constraint. If you need a single
inbox readable across multiple domains, the workable pattern is one
shared mailbox per domain with a [virtual recipient](https://docs.deeztek.com/books/administrator-guide/page/virtual-recipients)
fan-out feeding both.

## Nextcloud Mail caches the folder tree per account

Nextcloud Mail (the NC webmail app) caches each connected account's
IMAP folder tree the first time the account is added and refreshes it
lazily. **A user who is newly granted access to a shared mailbox via
this page will NOT see it in Nextcloud Mail until they remove and
re-add their NC mail account.** Standalone IMAP clients (Thunderbird,
Outlook, Apple Mail) refresh the folder tree on the next IDLE cycle or
manual sync, so they don't have this gotcha.

This is upstream NC Mail behavior, not a Hermes setting. The
workaround is documented for end-users in the User Portal
documentation; for admins, the remediation is to tell the affected
user to re-add their NC mail account once the share is in place.

## Feature-disabled behavior

When the Mailbox Sharing master switch on
[Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) is **off**:

- The Add / Rebuild / Manage Permissions buttons render disabled with
  a tooltip pointing back to Settings.
- An amber banner at the top of the page explains the state and
  links to Settings.
- Existing shared mailboxes appear in the table with status badge
  `Inactive (Sharing Off)` so the admin can see what would resume
  when the switch is flipped back on.
- The Delete button still works — admins can clean up rows while the
  feature is off.
- The `add_shared_mailbox`, `add_permission`, `edit_permission`, and
  `sync_all_acl_files` action handlers all check the master switch at
  entry and return error 31 if it's off, so a stale tab can't
  silently bypass the guard.

Dovecot itself does not declare the `Shared/` namespace when the
master switch is off, so IMAP clients won't see shared folders even
if the on-disk ACL files exist. Existing ACL files are preserved and
re-activate as soon as the switch is flipped back on.

## Failure semantics

| What breaks | What happens |
|---|---|
| Master switch off + Add / Edit / Sync attempted | error 31, no DB write |
| Blank address prefix | error 10 |
| Address prefix has invalid characters | error 11 |
| Domain missing or not mailbox-type | error 12 |
| Address collides with mailbox / alias / virtual recipient / existing shared mailbox | error 13 |
| Quota not numeric or `<= 0` | error 14 |
| Blank display name | error 15 |
| Stale shared_mailbox_id (deleted between page load and submit) | error 21 |
| Invalid user_mailbox_id | error 22 |
| User already has permissions on this shared mailbox | error 23 |
| Stale permission_id (Edit / Remove) | error 24 |
| Add / Edit Permission with all seven flags off | error 25 |
| Cross-domain member attempt | error 26 |
| Any database operation throws inside the cftry | error 30, no rows committed |
| `doveadm mailbox create` fails | non-fatal — Maildir bootstraps via LMTP on first delivery instead |
| `sync_shared_mailbox_acl_file.cfm` fails | non-fatal — DB is the source of truth; the next permission change retries the sync, or admin can use Rebuild ACL Files |
| Maildir `rm -rf` on delete fails | non-fatal — DB rows are removed regardless; admin can manually clean up `/srv/mail/<domain>/<local>` |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/view_shared_mailboxes.cfm` | `hermes_commandbox` | Page + table + Add / Manage / Delete / Rebuild modals |
| `config/hermes/var/www/html/admin/2/inc/shared_mailbox_actions.cfm` | `hermes_commandbox` | Dispatcher for all six actions (add / delete / add_permission / edit_permission / remove_permission / sync_all_acl_files) |
| `config/hermes/var/www/html/admin/2/inc/sync_shared_mailbox_acl_file.cfm` | `hermes_commandbox` | Rebuilds one `dovecot-acl` file from `shared_mailbox_permissions` |
| `config/hermes/var/www/html/admin/2/inc/sync_user_folder_acl_file.cfm` | `hermes_commandbox` | Same engine for user-initiated folder shares |
| `config/hermes/var/www/html/admin/2/inc/get_shared_mailbox_permissions_json.cfm` | `hermes_commandbox` | AJAX endpoint for the Manage Permissions table |
| `/srv/mail/<domain>/<local>/dovecot-acl` | `hermes_dovecot` (vmail:vmail 0660) | Per-mailbox vfile ACL file — Dovecot 2.4's enforcement source |
| `/srv/mail/<domain>/<local>/` | `hermes_dovecot` | The Maildir itself |
| `/opt/hermes/tmp/<token>_sync_shared_acl.sh` | `hermes_commandbox` | Throwaway shell script used to ship the ACL payload through `docker exec -i` via heredoc |
| `shared_mailboxes`, `shared_mailbox_permissions`, `user_folder_shares`, `mailboxes`, `recipients`, `maddr`, `sender_login_maps`, `dovecot_acl`, `dovecot_acl_shared`, `parameters2` | `hermes_db_server` | Storage |
| `hermes_dovecot` container | — | `doveadm mailbox create` (bootstrap), `rm -rf` (delete), and the in-container `mkdir / cat / chown / chmod` invoked by the sync helper |

## Related

- [Settings](https://docs.deeztek.com/books/administrator-guide/page/settings) — the Mailbox Sharing master switch. Must
  be on for shared mailboxes to actually function at the IMAP layer.
  Also the Dovecot TLS profile and connection limits that all
  shared-mailbox access goes through.
- [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — the user mailbox list. Members granted
  permission on this page must already exist there.
- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — the mailbox domain list. A shared mailbox
  is anchored to exactly one domain; cross-domain sharing is not
  supported.
- [Aliases](https://docs.deeztek.com/books/administrator-guide/page/aliases) — if you want one inbound address to deliver
  into one mailbox (rather than be visible to several users), an
  alias is the lighter-weight option. Aliases have no ACL surface at
  all.
- [Email Relay > Virtual Recipients](https://docs.deeztek.com/books/administrator-guide/page/virtual-recipients)
  — the relay-side fan-out pattern. Sometimes a virtual recipient
  feeding two shared mailboxes (one per domain) is the right tool
  when a single role address needs to be visible to users on more
  than one mailbox domain.
- [Mailbox Rules](https://docs.deeztek.com/books/administrator-guide/page/mailbox-rules) — Sieve rules can be configured
  on shared mailboxes the same way as on user mailboxes; the
  authentication path is the granting user, not the shared address.
- [Authentication Settings](https://docs.deeztek.com/books/administrator-guide/page/authentication-settings)
  — Submission-port auth that the Send-As flag piggybacks on, plus
  the LDAP backend that Dovecot looks up members against.

# Content Checks

# Antispam Settings

# Antispam Settings

Admin path: **Content Checks > Antispam Settings**
(`view_antispam_maintenance.cfm`, `inc/get_spam_settings.cfm`,
`inc/spam_settings_save.cfm`, `inc/update_amavis_config_files.cfm`,
`inc/update_spamassassin_config_files.cfm`, `inc/restart_amavis.cfm`,
`inc/restart_spamassassin.cfm`, `inc/antispam_init_pyzor.cfm`,
`inc/antispam_init_razor.cfm`, `inc/antispam_clear_bayes.cfm`).

This page configures the SpamAssassin engine that Amavis calls inside
`hermes_mail_filter` for every message that clears the SMTP-time
perimeter, plus the Amavis-level handling policies that decide what
happens to a message once it has been scored or otherwise classified.
Per-rule weight adjustments live on [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides);
this page is engine settings and quarantine destiny only.

## Where SpamAssassin sits in the flow

```
                  +-----------------------------------+
   inbound msg -->| Perimeter Checks pass             |
                  +---------------+-------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  Postfix smtpd_proxy_filter       |
                  |    -> hermes_mail_filter:10024    |
                  +---------------+-------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  Amavis (hermes_mail_filter)      |
                  |   - ClamAV virus scan             |
                  |   - SpamAssassin scoring          |
                  |       DCC / Razor / Pyzor net DBs |
                  |       Bayes statistical engine    |
                  |       custom rules + scores       |
                  |   - banned-file checks            |
                  |   - final_*_destiny -> quarantine/DSN/discard
                  +---------------+-------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  Re-inject -> hermes_postfix_dkim:10026
                  +-----------------------------------+
```

A virus verdict from ClamAV always pre-empts the spam score; the
`final_virus_destiny` setting on this page decides what Amavis does
with that already-classified virus. The `final_spam_destiny`,
`final_banned_destiny`, and `final_bad_header_destiny` settings work
the same way for the other three Amavis verdict categories.

## Container and tool placement

| Component | Detail |
| --- | --- |
| Container | `hermes_mail_filter` (IPv4 `.105`) |
| Engine | SpamAssassin (`spamd` / `Mail::SpamAssassin` Perl modules called from Amavis) |
| Amavis config | `/etc/amavis/conf.d/50-user` (rendered from `/opt/hermes/conf_files/50-user.HERMES` on every save) |
| SpamAssassin config | `/etc/spamassassin/local.cf` (rendered from `/opt/hermes/conf_files/local.cf.HERMES` on every save) |
| Bayes DB | Lives in the SpamAssassin user dir inside `hermes_mail_filter` (`sa-learn --dump magic` reports the actual path) |
| Network plugin state | `/etc/razor/identity` (Razor), Pyzor's per-user config dir, DCC's local socket — all inside `hermes_mail_filter` |
| Reload mechanism | `spamassassin --lint` + `docker container restart hermes_mail_filter` on every save |

The container exposes **no host ports** — Amavis is reached only by
Postfix internally at `hermes_mail_filter:10024` and re-injects to
`hermes_postfix_dkim:10026`.

## Spam Detection Plugins card

Three boolean toggles enable third-party network-aware spam DBs.
Storage: `spam_settings.value` for parameters `use_dcc`, `use_razor2`,
`use_pyzor` (each row keyed by `parameter`, value `0` or `1`).

| Plugin | What it does | Maintenance action |
| --- | --- | --- |
| DCC (Distributed Checksum Clearinghouse) | Fuzzy-checksum bulk-mail detection; matches a message against a network of receivers' checksum counters | None — `cdcc` runs as part of the SpamAssassin call chain |
| Razor2 (Vipul's Razor v2) | Collaborative spam catalog; checksum + signature lookup against the Razor network | **Initialize Razor** (see Maintenance) before first use |
| Pyzor | Collaborative digest-based spam detection | **Initialize Pyzor** before first use |

Each toggle substitutes into `local.cf` via the placeholders `USE-DCC`,
`USE-PYZOR`, `USE-RAZOR2` -> `use_dcc 0|1`, `use_pyzor 0|1`,
`use_razor2 0|1`.

> **Operational consequence — network DB connectivity.** All three
> plugins make outbound queries (DCC over UDP, Razor and Pyzor over
> TCP) at scan time. If outbound to the public Internet is blocked
> from `hermes_mail_filter`, the plugins quietly time out per
> message and add measurable per-scan latency. Disable plugins the
> gateway cannot actually reach.

## Subject Tagging card

Single field, `sa_spam_subject_tag` in `spam_settings`. Substitutes
into `50-user` via the `sa-spam-subject-tag` placeholder, which sets
Amavis's `$sa_spam_subject_tag`. Default `[SUSPECTED SPAM]`. Required
(empty value rejected with error 2). Only applied when
`sa_spam_modifies_subj = 1` (a fixed value in `spam_settings`, not
exposed in the UI).

## Message Handling Policies card

Four radio pairs, one per Amavis verdict category. Each row stores
`D_DISCARD` or `D_BOUNCE` in `spam_settings.value` and substitutes
into `50-user` via `final-<category>-destiny`. Amavis acts on the
value as follows:

| Setting | DB row | "Quarantine Only" (`D_DISCARD`) | "Quarantine & Send DSN" (`D_BOUNCE`) |
| --- | --- | --- | --- |
| Virus Messages | `final_virus_destiny` | Message goes to quarantine; no DSN | Message goes to quarantine; DSN sent to envelope sender |
| Banned File Messages | `final_banned_destiny` | Same as above for banned-file matches | DSN sent |
| Spam Messages | `final_spam_destiny` | Quarantined silently | DSN sent |
| Bad-Header Messages | `final_bad_header_destiny` | Quarantined silently | DSN sent |

The labels are deliberately conservative — `D_DISCARD` does **not**
delete the message, it routes it to Amavis's quarantine where Message
History can review and release it. Defaults: virus + banned send DSN;
spam + bad-header quarantine silently.

> **Operational consequence — Send DSN on spam.** Setting
> `final_spam_destiny = D_BOUNCE` means Hermes will deliver a
> non-delivery report to the envelope sender of every quarantined
> spam. Because the envelope sender is almost always forged on spam,
> the DSN will either bounce, contribute to backscatter against
> innocent third parties, or land in a victim's spam folder. The
> safe default for spam is `D_DISCARD`; reserve DSN for virus and
> banned-file (where the sender is more likely to be legitimate).

## Bayes Database card

SpamAssassin's per-installation statistical learning engine. Three
controls, stored in `spam_settings`:

| Field | DB row | Substitution placeholder | Effect |
| --- | --- | --- | --- |
| Enable Bayes Database | `use_bayes` | `USE-BAYES` -> `use_bayes` followed by `0` or `1` | Master switch; when off, Bayes rules contribute no score |
| Enable Auto-Learning | `bayes_auto_learn` | `BAYES-AUTO-LEARN` -> `bayes_auto_learn` followed by `0` or `1` | When on, SpamAssassin trains the Bayes DB automatically based on the message's final score relative to the thresholds below |
| Spam Threshold | `bayes_auto_learn_threshold_spam` | `BAYESAUTOLEARN-SPAM` -> `bayes_auto_learn_threshold_spam <value>` | Final score above which auto-learn treats the message as spam. Must be numeric and in the range `0.01 .. 999` |
| Non-Spam Threshold | `bayes_auto_learn_threshold_nonspam` | `BAYESAUTOLEARN-HAM` -> `bayes_auto_learn_threshold_nonspam <value>` | Final score below which auto-learn treats the message as ham. Must be numeric and in the range `-999 .. -0.01` |

The thresholds are SpamAssassin's `bayes_auto_learn_threshold_spam`
and `bayes_auto_learn_threshold_nonspam` directives. JavaScript on
the page collapses the thresholds when Bayes or auto-learning is
disabled.

> **Operational consequence — Bayes poisoning.** Auto-learning
> trusts the final score (which already includes Bayes's own
> contribution) to decide whether to train. A bad spam wave that
> sneaks past the score threshold can train Bayes to think more spam
> is ham, which lowers detection on the next batch. If detection
> quality regresses noticeably after enabling auto-learning, use the
> Clear Bayes Database action and re-train manually or via a
> known-good corpus before re-enabling.

## Save flow

```
1. View page submits action="save_settings" (all four cards in one POST)
2. spam_settings_save.cfm validates:
     - sa_spam_subject_tag non-empty (error 2)
     - if bayes_auto_learn=1:
         spam threshold numeric (error 5), > 0 and <= 999 (error 4),
                 non-empty (error 3)
         non-spam threshold numeric (error 10), < 0 and >= -999 (error 8),
                 non-empty (error 7)
3. On valid input, UPDATEs 13 rows in spam_settings (sa_spam_subject_tag,
   four final_*_destiny, use_bayes, bayes_auto_learn, both thresholds,
   use_dcc, use_razor2, use_pyzor)
4. cfinclude update_amavis_config_files.cfm:
     - Reads /opt/hermes/conf_files/50-user.HERMES
     - Substitutes SERVER-NAME, SERVER-DOMAIN, sa-spam-subject-tag,
       final-{virus,banned,spam,bad-header}-destiny,
       enable-dkim-{verification,signing},
       HERMES-USERNAME, HERMES-PASSWORD,
       FILE-RULES-GO-HERE (from file_rule_components table),
       DKIM-KEYS-GO-HERE (from dkim_sign table)
     - Backs up /etc/amavis/conf.d/50-user -> 50-user.HERMES.BACKUP
     - Moves rendered file into place
5. cfinclude update_spamassassin_config_files.cfm:
     - Reads /opt/hermes/conf_files/local.cf.HERMES
     - Substitutes USE-DCC, USE-PYZOR, USE-RAZOR2, USE-BAYES,
       BAYES-AUTO-LEARN, BAYESAUTOLEARN-SPAM, BAYESAUTOLEARN-HAM
     - Appends per-rule score lines (from spam_settings where spamfilter=1)
     - Appends custom message rules (from message_rules table)
     - Backs up /etc/spamassassin/local.cf -> local.cf.HERMES.BACKUP
     - Moves rendered file into place
6. cfinclude restart_amavis.cfm -> restart_mail_filter.cfm:
     - docker container restart hermes_mail_filter
7. cfinclude restart_spamassassin.cfm:
     - docker exec hermes_mail_filter /usr/bin/spamassassin --lint
     - docker container restart hermes_mail_filter
8. session.m = 1 -> green "Anti-spam settings have been saved and applied" alert
9. cflocation back to view_antispam_maintenance.cfm
```

The same container is restarted twice (once for Amavis, once for
SpamAssassin) because the restart includes are intentionally
independent helpers used elsewhere; both calls resolve to the same
`docker container restart hermes_mail_filter`. Outbound mail queues
briefly during the restart cycle (typically a few seconds); Postfix
will retry.

## Maintenance card group

Three buttons, each running a single `docker exec` against
`hermes_mail_filter` and surfacing stdout/stderr to the operator.

### Initialize Pyzor

Action handler: `antispam_init_pyzor.cfm`

```
docker exec hermes_mail_filter /usr/bin/pyzor ping
```

Pings the Pyzor servers; success is detected by the literal string
`200` in the output. The command both verifies connectivity and writes
the per-user Pyzor config the first time it runs. Required before
`use_pyzor = 1` returns meaningful results.

### Initialize Razor

Action handler: `antispam_init_razor.cfm`

```
docker exec hermes_mail_filter /bin/bash -c \
  'rm -f /etc/razor/identity && razor-admin -create && razor-admin -register'
```

Deletes the existing Razor identity, creates a fresh config, and
registers the gateway with the Razor network. Success is detected by
`Register successful` or `created` in the output. Re-run if Razor
queries start failing (typically after the identity is rotated or the
network rejects the existing identity).

### Clear Bayes Database

Action handler: `antispam_clear_bayes.cfm`

```
docker exec hermes_mail_filter /usr/bin/sa-learn --clear
```

Wipes the learned spam/ham corpus. SpamAssassin will need to re-learn
from scratch before Bayes rules contribute meaningful scores again.
Use only when the database is known-poisoned or when migrating between
servers without preserving training. The button is gated behind a
JavaScript `confirm()` and renders inside a yellow warning card.

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Empty `sa_spam_subject_tag` | session.m=2, red alert, no save |
| Bayes spam threshold empty | session.m=3 |
| Bayes spam threshold not numeric | session.m=5 |
| Bayes spam threshold <= 0 or > 999 | session.m=4 |
| Bayes non-spam threshold empty | session.m=7 |
| Bayes non-spam threshold not numeric | session.m=10 |
| Bayes non-spam threshold >= 0 or < -999 | session.m=8 |
| Any cfcatch during the save -> apply chain | session.m=9, red alert with `session.saveError` showing `cfcatch.message` |
| `spamassassin --lint` failure during restart | `error.cfm` cfabort with the lint failure message; the rendered `local.cf` is already in place but Amavis is not restarted further |
| Pyzor ping output without `200` | session.m=12, red alert; full output shown in a `<pre>` for diagnosis |
| Razor init output without `Register successful` or `created` | session.m=14, similar surfacing |
| Bayes clear `cfcatch` | session.m=16 with the catch message |

`spamassassin --lint` is the canonical pre-restart sanity check —
when a custom rule (added via Score Overrides or message rules) has
invalid syntax, the lint catches it before the container restart
finishes and prevents Amavis from starting against a broken config.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_antispam_maintenance.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/spam_settings_save.cfm` | `hermes_commandbox` | Validation + UPDATE + apply chain |
| `config/hermes/var/www/html/admin/2/inc/get_spam_settings.cfm` | `hermes_commandbox` | Loads current `spam_settings` rows |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Renders `50-user` from template + DB |
| `config/hermes/var/www/html/admin/2/inc/update_spamassassin_config_files.cfm` | `hermes_commandbox` | Renders `local.cf` from template + DB |
| `config/hermes/var/www/html/admin/2/inc/restart_amavis.cfm` / `restart_spamassassin.cfm` / `restart_mail_filter.cfm` | `hermes_commandbox` | `docker container restart hermes_mail_filter` |
| `config/hermes/var/www/html/admin/2/inc/antispam_init_pyzor.cfm` / `antispam_init_razor.cfm` / `antispam_clear_bayes.cfm` | `hermes_commandbox` | Maintenance docker-exec helpers |
| `config/hermes/opt/hermes/conf_files/50-user.HERMES` | template (read) -> `hermes_mail_filter` (live `/etc/amavis/conf.d/50-user`) | Amavis directives template |
| `config/hermes/opt/hermes/conf_files/local.cf.HERMES` | template (read) -> `hermes_mail_filter` (live `/etc/spamassassin/local.cf`) | SpamAssassin directives template |
| `/etc/amavis/conf.d/50-user.HERMES.BACKUP` | `hermes_mail_filter` | Pre-write backup, refreshed each save |
| `/etc/spamassassin/local.cf.HERMES.BACKUP` | `hermes_mail_filter` | Pre-write backup, refreshed each save |
| `spam_settings` table | `hermes_db_server` (`hermes` DB) | Source of truth for every UI value on this page; also holds per-rule scores (`spamfilter=1` rows) for Score Overrides |
| `message_rules` table | `hermes_db_server` | Custom header/body/full message rules; rendered into `local.cf` |
| `file_rule_components` / `files` tables | `hermes_db_server` | Banned-file rules; rendered into `50-user` |
| `dkim_sign` table | `hermes_db_server` | Per-domain DKIM keys; rendered into `50-user` for outbound signing |

## Related

- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) -- the ClamAV engine
  that runs in the same Amavis pass and whose virus verdict always
  pre-empts the spam score
- [Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds) -- third-party ClamAV signature
  feeds; orthogonal to spam scoring but consumed in the same scan
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) -- per-rule SpamAssassin
  weight adjustments (the `spamfilter=1` rows in `spam_settings`);
  this page sets the engine knobs, that page sets the rule weights
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) -- custom header / body / full
  message regex rules that ride into `local.cf` on every save here
- [SVF Policies](https://docs.deeztek.com/books/administrator-guide/page/svf-policies) -- per-sender and per-recipient
  spam-handling overrides that apply before the engine-wide
  `final_*_destiny` settings on this page
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) -- the SMTP-time gate;
  every check on this page runs only after a connection clears the
  perimeter
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) -- seals over the body Amavis
  passed, so a high spam score (and any quarantine action) naturally
  pre-empts the seal
- [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) -- a DMARC-fail verdict can
  promote a message to a higher spam score via SpamAssassin's DMARC
  rule weights (tunable on Score Overrides)
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) -- `sa-update`
  for the SpamAssassin rule set runs on its own Ofelia schedule; the
  Bayes DB is per-installation and not updated by `sa-update`
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) -- full pipeline diagram

# Antivirus Settings

# Antivirus Settings

Admin path: **Content Checks > Antivirus Settings**
(`view_antivirus_settings.cfm`, `inc/get_antivirus_settings.cfm`,
`inc/antivirus_set_settings.cfm`,
`inc/antivirus_add_whitelists.cfm`,
`inc/antivirus_delete_entry.cfm`,
`inc/generate_antivirus_configuration.cfm`,
`inc/restart_clamav.cfm`).

This page configures the ClamAV antivirus engine that runs inside
`hermes_mail_filter` and is called by Amavis on every message that
clears the SMTP-time perimeter. Two cards: the main settings card
(sixteen toggles that map to `clamd.conf` directives) and a Pro-only
AV Signature Whitelist for suppressing known-bad-signature false
positives. Refreshing third-party signature *feeds* (Sanesecurity,
SecuriteInfo, MalwarePatrol, etc.) is configured separately on
[Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds); this page configures the engine
itself.

## Where antivirus sits in the flow

```
                  +-----------------------------------+
   inbound msg -->| Perimeter Checks pass             |
                  +---------------+-------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  Postfix smtpd_proxy_filter       |
                  |    -> hermes_mail_filter:10024    |
                  +---------------+-------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  Amavis (hermes_mail_filter)      |
                  |   - SpamAssassin scoring          |
                  |   - ClamAV antivirus  <---- this page configures this engine
                  |   - banned-file checks            |
                  +---------------+-------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  Re-inject -> hermes_postfix_dkim:10026
                  +-----------------------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  OpenDKIM sign, ARC seal, deliver |
                  +-----------------------------------+
```

Amavis calls ClamAV over the local socket; the verdict
determines whether Amavis quarantines, blocks, or passes the
message. Amavis's own action policy (the `final_*_destiny`
settings — quarantine vs DSN vs discard) lives in
[Antispam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) and the per-domain
policy table, not on this page. This page is **engine knobs only**.

## Container and socket placement

| Component | Detail |
| --- | --- |
| Container | `hermes_mail_filter` (IPv4 `.105`) |
| Engine | `clamd` daemon, Unix socket inside the container |
| Daemon config | `/etc/clamav/clamd.conf` (volume-mounted from `./config/mail_filter/etc/clamav/clamd.conf`) |
| Signature dir | `/var/lib/clamav/` (Docker named volume `mail_filter_data_clamav`) |
| Signature whitelist | `/var/lib/clamav/local.ign2` (regenerated from `parameters2 WHERE module='clamav-bypass'` on every save) |
| Third-party feeds | `/etc/fangfrisch/fangfrisch.conf` + `/var/lib/fangfrisch/signatures/` (see [Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds)) |
| Base signature refresh | `freshclam` (official ClamAV CVD updates, default 1h) |
| Feed refresh | `fangfrisch refresh` on a 10-minute Ofelia job (`hermes-fangfrisch-refresh`) |

The container exposes **no host ports** — Amavis is reached only by
Postfix internally at `hermes_mail_filter:10024` and re-injects to
`hermes_postfix_dkim:10026`.

## ClamAV Antivirus Settings card

Sixteen toggles, each rendered from the `avSettings` array in
`view_antivirus_settings.cfm` with an inline hint and a "Recommended"
label on the safer default. Every toggle writes
`parameters2.value2 = 'true' | 'false'` for `module = 'clamav'`; on
save, `generate_antivirus_configuration.cfm` selects every active row
and emits one `<directive> <value>` line per toggle into a temp file,
substitutes the temp file into the `HERMES_ANTIVIRUS_SETTINGS_GO_HERE`
placeholder of `clamd.conf.HERMES`, backs up the live config to
`clamd.conf.HERMES`, and moves the rendered file into place.

| UI Toggle | `clamd.conf` directive | Recommended | Notes |
| --- | --- | --- | --- |
| Scan Email Attachments | `ScanMail` | Enabled | Master switch for inbound attachment scanning |
| Scan Archives | `ScanArchive` | Enabled | Recurse into ZIP, RAR, 7z, etc. Without this, only the archive wrapper is scanned |
| Mark Encrypted Archives as Viruses | `ArchiveBlockEncrypted` | Disabled | Aggressive; commonly false-positives on legitimate password-protected files |
| Scan Portable Executables | `ScanPE` | Enabled | Windows PE format; required for decompression of UPX / FSG / Petite packers |
| Scan OLE2 Files | `ScanOLE2` | Enabled | MS Office `.doc/.xls/.ppt` and `.msi` |
| Block OLE2 VBA Macros | `OLE2BlockMacros` | Disabled | Blocks ALL macro-enabled documents regardless of intent (detected as `Heuristics.OLE2.ContainsMacros`); useful in strict environments, breaks legitimate macros otherwise |
| Scan PDF Files | `ScanPDF` | Enabled | PDF embedded JS, exploit detection |
| Scan HTML/JavaScript Content | `ScanHTML` | Enabled | HTML normalization + JavaScript/ScriptEncoder decryption; phishing + script-exploit detection |
| Algorithmic Detection | `AlgorithmicDetection` | Enabled | Engine-level heuristics for complex malware and graphic-file exploits |
| Scan ELF Files | `ScanELF` | Enabled | Linux/Unix executable format |
| Phishing Signature Detection | `PhishingSignatures` | Enabled | ClamAV's phishing signature DB |
| Scan Email URLs for Phishing | `PhishingScanURLs` | Enabled | URL extraction + phishing URL DB lookup |
| Block SSL Mismatches in URLs | `PhishingAlwaysBlockSSLMismatch` | Disabled | False-positives on CDN and redirect URLs |
| Block Cloaked URLs | `PhishingAlwaysBlockCloak` | Disabled | False-positives on URL shorteners and marketing-tracker links |
| Detect Potentially Unwanted Applications | `DetectPUA` | Enabled | Adware, dialers, non-malicious-but-unwanted software |
| Heuristic Scan Precedence | `HeuristicScanPrecedence` | Enabled | When on, heuristic hits stop the scan immediately (saves CPU). When off, scanning continues so a signature-based hit can override a heuristic match |

> **Operational consequence — disabling `ScanMail`.** This effectively
> turns off antivirus for inbound mail. Amavis will still consult
> ClamAV for ban-pattern decisions but the engine will skip the
> attachment scan. Leave on except for very short-term diagnostics.
>
> **Operational consequence — `OLE2BlockMacros` = true.** Every
> macro-enabled Office document is blocked as `Heuristics.OLE2.ContainsMacros`,
> including documents from your own users. Most organizations get
> better results with macro-blocking enforced at the endpoint
> (Microsoft 365 Protected View, Group Policy) rather than at the
> gateway. Turn on only after warning users and ensuring you have a
> release workflow.

## AV Signature Whitelist card (Pro)

When ClamAV produces a false positive on a known-safe file, the
admin enters the exact ClamAV signature name (e.g.
`Heuristics.OLE2.ContainsMacros`) and Hermes appends it to
`/var/lib/clamav/local.ign2`. ClamAV reads `local.ign2` at engine
start and suppresses any detection whose signature name matches a
line in the file.

Storage:
`parameters2 WHERE module = 'clamav-bypass'` (one row per signature
name, `parameter` column holds the signature string). On every save
and on every delete, `generate_antivirus_configuration.cfm` rewrites
the whole `local.ign2` from the table, runs `dos2unix` to scrub line
endings, backs up the current file to `local.ign2.HERMES`, and moves
the new file into place. ClamAV is then restarted via
`restart_clamav.cfm` to pick up the change.

### How to find a signature name

The in-card info box gives admins the lookup steps:

1. From **Message History**, find the blocked message (Type column
   shows `Virus` or `Banned`)
2. Grep the mail-filter log for the message ID:
   `docker logs hermes_mail_filter 2>&1 | grep <mail_id>`
3. The log line shows the signature in parentheses, e.g.
   `Blocked INFECTED (Heuristics.OLE2.ContainsMacros)`
4. Or scan a file directly:
   `docker exec hermes_mail_filter clamscan /path/to/file`

> **Operational consequence — whitelisting is by signature name, not
> by file hash.** If you whitelist `Heuristics.OLE2.ContainsMacros`,
> you have effectively turned off macro detection globally. Prefer
> narrow signature names (specific malware family) over heuristic
> families when possible.

## Signature refresh

Two independent refresh loops keep the engine current:

| Source | Mechanism | Cadence | Database |
| --- | --- | --- | --- |
| Official ClamAV (`main.cvd`, `daily.cvd`, `bytecode.cvd`) | `freshclam` daemon inside `hermes_mail_filter` | Default 1h (configurable in `/etc/clamav/freshclam.conf`) | `/var/lib/clamav/` |
| Third-party feeds (Sanesecurity, SecuriteInfo, MalwarePatrol, etc.) | `fangfrisch refresh` via Ofelia job `hermes-fangfrisch-refresh` | Every 10 minutes (only feeds whose own publish cycle has elapsed actually re-download) | `/var/lib/fangfrisch/signatures/` then linked into `/var/lib/clamav/` by `setup-clamav-sigs` |

`fangfrisch` is the small Python tool that handles auth, cadence
control, and integrity verification for third-party feeds; the feed
list and per-feed enable/disable lives on
[Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds). Enabling premium feeds
(SecuriteInfo paid, MalwarePatrol paid) requires Pro licensing —
the feed list itself is gated on the same page.

## Resource footprint

Loading the full signature database into RAM costs roughly 1.5–2 GB
of memory. If `hermes_mail_filter` is under-provisioned (e.g. shared
host with 4 GB total), `clamd` will fail to start, mail will queue
behind Amavis, and the only sign in the UI is a quiet rise in
deferred queue depth. Plan for at least 4 GB dedicated to the
`hermes_mail_filter` container on systems with all third-party
feeds enabled.

The default ClamAV file-size cap is 25 MB (`MaxFileSize 25M` in
`clamd.conf`). Messages larger than this are passed without scan and
flagged with a `Heuristics.Limits.Exceeded` indicator. Raising the
cap requires editing `clamd.conf.HERMES` directly; the UI does not
expose it because raising it disproportionately increases RAM and
CPU per scan.

## Save flow

```
1. View page submits action="AV Settings" (sixteen booleans),
                       action="Add AV Whitelist" (textarea),
                       action="Delete Entry" (id list)
2. view_antivirus_settings.cfm validates every avFields entry exists and is true|false
   (any failure -> error.cfm + cfabort)
3. antivirus_set_settings.cfm UPDATEs parameters2.value2 for each toggle
   (16 UPDATEs, module='clamav')
4. generate_antivirus_configuration.cfm:
     a. SELECT active='1' rows from parameters2 module='clamav' -> temp avsettings file
     b. dos2unix the temp file
     c. Substitute into clamd.conf.HERMES placeholder HERMES_ANTIVIRUS_SETTINGS_GO_HERE
     d. Back up /etc/clamav/clamd.conf -> clamd.conf.HERMES, move new file into place
     e. Rebuild /var/lib/clamav/local.ign2 from parameters2 module='clamav-bypass'
     f. dos2unix, back up local.ign2 -> local.ign2.HERMES, move new file into place
     g. cfinclude restart_clamav.cfm (docker container restart hermes_mail_filter ClamAV process)
5. session.m = 9 -> green "Antivirus Settings were saved successfully" alert
```

`generate_antivirus_configuration.cfm` also runs on whitelist
add/delete — every change to either card triggers the same full
regen + ClamAV restart cycle. The page does not return until the
restart has completed (timeout per `cfexecute`).

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Toggle form missing a required boolean field | `m = "Antivirus Settings: form.<f> does not exist"`, `error.cfm`, cfabort |
| Toggle value not in `true,false` | `m = "Antivirus Settings: form.<f> is not true or false"`, `error.cfm`, cfabort |
| Delete clicked with no selection | session.m = 11 |
| Add Whitelist with empty textarea | session.m = 13 |
| `dos2unix` failure on the temp avsettings or local.ign2 file | `error.cfm` + cfabort with the failing path in the message |
| `cp /etc/clamav/clamd.conf -> .HERMES` failure | `error.cfm` + cfabort |
| `mv <tmp>_clamd.conf -> /etc/clamav/clamd.conf` failure | `error.cfm` + cfabort |
| `restart_clamav.cfm` failure | Surfaces as cfcatch from the docker restart step |

The save is **not** transactional across the steps — if the SQL
updates succeed but the ClamAV restart fails, the DB state has
already advanced. The next save will re-render and re-apply
because every save regenerates the entire file from the current row
state (no incremental writes).

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_antivirus_settings.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/antivirus_*.cfm` | `hermes_commandbox` | Validate / save / regenerate / restart |
| `config/hermes/var/www/html/admin/2/inc/get_antivirus_settings.cfm` | `hermes_commandbox` | Loads current `parameters2 module='clamav'` values |
| `config/hermes/opt/hermes/conf_files/clamd.conf.HERMES` | `hermes_commandbox` (read) -> `hermes_mail_filter` (live `/etc/clamav/clamd.conf`) | Canonical template with `HERMES_ANTIVIRUS_SETTINGS_GO_HERE` placeholder |
| `config/mail_filter/etc/clamav/clamd.conf` | `hermes_mail_filter` (live config, bind-mounted) | Read by `clamd` at start |
| `/var/lib/clamav/local.ign2` | `hermes_mail_filter` (Docker named volume `mail_filter_data_clamav`) | Signature whitelist; rewritten on every save |
| `/var/lib/clamav/*.cvd`, `*.cld`, `*.ndb`, etc. | `hermes_mail_filter` | Signature databases (official + third-party) |
| `parameters2` table, `module='clamav'` | `hermes_db_server` (`hermes` DB) | Source of truth for the sixteen toggles |
| `parameters2` table, `module='clamav-bypass'` | `hermes_db_server` (`hermes` DB) | Source of truth for the AV Signature Whitelist |
| `malware_databases` table | `hermes_db_server` (`hermes` DB) | Third-party feed list (configured on [Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds)) |
| `ofelia_jobs` row `hermes-fangfrisch-refresh` | `hermes_db_server` | 10-minute feed refresh scheduler |
| `hermes_mail_filter` container | — | `clamd`, `freshclam`, `fangfrisch`, Amavis, SpamAssassin |

## Related

- [Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds) — the third-party signature feed
  configuration (Sanesecurity, SecuriteInfo, MalwarePatrol, etc.)
  that Fangfrisch refreshes every 10 minutes
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — every check on this page
  runs only after a connection clears the SMTP-time perimeter
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — runs in the same
  Amavis pass; a virus verdict overrides any spam score
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — per-rule weight changes
  for SpamAssassin
- [Email Policies > Disclaimers](https://docs.deeztek.com/books/administrator-guide/page/disclaimers) —
  body modification that runs after Amavis re-injection; never
  conflicts with ClamAV because it happens post-scan
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — seals over the body Amavis
  passed, so a virus verdict naturally pre-empts everything
  downstream
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — URL phishing
  lookups (`PhishingScanURLs`) and signature-feed downloads
  (Fangfrisch) all resolve through `hermes_unbound`
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) — full pipeline diagram

# ARC Settings

# ARC Settings

Admin path: **Content Checks > ARC Settings** (`view_arc_settings.cfm`)

## What ARC does

ARC (Authenticated Received Chain, [RFC 8617](https://datatracker.ietf.org/doc/html/rfc8617))
preserves authentication results across forwarding gateways. Each
gateway that handles a message can add a sealed record of the
authentication state it observed, so a downstream verifier can trust
the cumulative chain even when an intermediate gateway modifies the
message body (adding disclaimers, banners, forwarding annotations,
etc.) — body modification would otherwise invalidate the original
sender's DKIM signature and lose DMARC alignment.

Hermes participates in ARC at two roles:

1. **As an originating sealer** for mail submitted by authenticated
   Hermes users to external recipients — Hermes is the first hop in
   the chain (`i=1; cv=none`).
2. **As a forwarding sealer** for inbound mail being relayed to a
   downstream MX (relay-mode domains) — Hermes adds a seal at `i=N+1`
   referencing the upstream chain.

## Container and milter placement

| Component | Detail |
| --- | --- |
| Container | `hermes_openarc` (separate service, IPv4 `.114`) |
| Listen | `inet:8893` |
| Source | [flowerysong/OpenARC](https://github.com/flowerysong/openarc) v1.3.0, built from release tarball |
| Milter chain | `master.cf` `:10026` only (post-amavis re-injection, after OpenDKIM signer at `:8892`) |
| NOT in | `main.cf` default `smtpd_milters` — sealing at `:25` over the pre-modification body would produce an invalid seal once body_milter and CipherMail change the bytes |

## Modes

| Mode | Effect |
| --- | --- |
| `s` (sign only) | Adds Hermes's seal but does not validate upstream chains |
| `v` (verify only) | Records inbound chain validity in `Authentication-Results` headers; does not add a seal |
| `sv` (sign + verify) | The gateway default; validates upstream then seals over the final body |

The ARC Settings page slider auto-syncs Mode between `sv` (enabled) and
`v` (disabled). The master `arc_signing_enabled` flag controls whether
the daemon adds anything at all — when disabled, OpenARC operates in
pass-through mode (every peer in `PeerList`, no headers added).

## Single signing identity per gateway

Unlike DKIM (which uses per-sender-domain keys), ARC uses a single
signing identity per gateway — Gmail seals everything with
`d=google.com`, Microsoft 365 with `d=outlook.com`, and Hermes with
whatever domain you generate the key for. Pick a domain you control
(typically your own organization's primary domain). The selector
follows the same DNS publication pattern as DKIM:
`<selector>._domainkey.<domain>` with value
`v=DKIM1; k=rsa; p=<public-key>`.

## Hermes is the auth boundary — what `cv=fail` means and doesn't mean

Hermes is the authoritative auth / security boundary for every domain
it serves. Inbound DKIM, SPF, DMARC, ARC verify, spam, virus checks
all happen at Hermes. Body modifications (External Sender Banner,
disclaimer, signature insertion, encryption) also happen at Hermes.
Customer downstream mail servers are expected to be configured to
trust Hermes implicitly: allowlist Hermes by IP / hostname, accept
forwarded mail without re-running upstream auth checks. This matches
how Mimecast, Proofpoint, and Barracuda customers deploy those
products — the SEG IS the trust boundary.

When Hermes modifies a message body (banner, disclaimer, etc.), any
cryptographic signature whose body hash was computed over the original
bytes will no longer body-validate against the current bytes. This
affects:

1. The original sender's `DKIM-Signature` body hash
2. The upstream `ARC-Message-Signature` body hash for each prior `i=`

Hermes's own outbound seal at `i=N+1` is mathematically valid (it is
computed over the modified body), but the `cv=` field on that seal
must honestly report whether the upstream chain passed when Hermes
received the message AND remains body-valid in the message it is about
to send. Once Hermes modifies the body, the upstream `bh=` no longer
matches the current body, so `cv=fail` is the correct (and only
defensible) value.

**This is by design.** A correctly-configured customer downstream MX
allowlists Hermes and does not re-check auth on Hermes-forwarded mail;
the `cv=fail` and broken DKIM signals never gate delivery. If a
customer reports forwarded mail being rejected by their downstream
MX due to ARC / DKIM / DMARC failure, the fix is to allowlist Hermes
on their MX, not to silence Hermes.

Removing Hermes's seal does not help: the verifier walks the chain
back to `i=1` and recomputes each prior body hash against the current
body independently of our seal. Stripping the entire upstream chain
would require Hermes to rewrite the `From:` header (mailing-list
style) to maintain DMARC alignment with a domain Hermes controls —
this is a significant UX cost that all major SEG vendors (Mimecast,
Proofpoint, Barracuda) have chosen not to pay.

### Default Hermes behavior

| Scenario | Behavior |
| --- | --- |
| Inbound mail with NO upstream ARC chain → any local recipient | Banner injects; Hermes seals at `i=1; cv=none`; chain is clean |
| Inbound mail with upstream ARC → local mailbox recipient | Banner injects; Hermes seals at `i=N+1; cv=fail`; message ends at Hermes (no downstream chain to protect — `cv=fail` is just bytes in the user's inbox) |
| Inbound mail with upstream ARC → relay-mode recipient | Banner injects; Hermes seals at `i=N+1; cv=fail`; downstream MX (which should be allowlisting Hermes) accepts and delivers regardless |
| Outbound from local Hermes user → external | Hermes is the first sealer; `i=1; cv=none`; clean chain to downstream |

There is no toggle, no conditional skip, no per-domain override.
Hermes always behaves the same way and reports the chain state
honestly. Customer-side trust configuration is the responsibility
of the customer's MX administrator.

### When a Trusted ARC Sealer configuration helps

Trusted ARC Sealer configuration on the customer side is useful in
**cross-org scenarios** that aren't direct relay-to-customer-MX —
for example, when a Hermes-served domain is part of a chain that
forwards through other gateways, or when Hermes is forwarding to a
third-party tenant the customer doesn't control. See the
[Trusted ARC Sealers — M365 guide](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) for
the M365 PowerShell configuration. For the standard
Hermes-as-relay-MX-to-customer-mail-server case, IP allowlisting on
the customer's MX is simpler and sufficient.

## When to ask receivers to trust Hermes as a sealer

For customers running strict downstream verifiers (Microsoft 365
tenants that DMARC-enforce, Gmail Workspace receivers that escalate
on `arc=fail`, etc.), the chain-integrity limitation can cause
relay-out delivery issues even on benign inbound that happens to come
through an upstream sealer. The standard industry remedy is for the
receiver to add Hermes to its **Trusted ARC Sealers** list.

For Microsoft 365 customers, follow the
[Trusted ARC Sealers — M365 guide](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) which
covers the PowerShell command, identity requirements, and verification
steps.

## Key management workflow

1. Click **Add ARC Key** in the *Gateway ARC Signing Identity* card
2. Enter the signing domain (must validate as `bob@<domain>`) and
   selector (DNS-safe label, e.g. `arc1`)
3. Choose key size (RSA 1024 or 2048)
4. Hermes generates the key pair in `/opt/hermes/arc/keys/`
5. Copy the public key TXT record and publish at
   `<selector>._domainkey.<domain>` in your authoritative DNS
6. Verify DNS propagation, then click the slider to enable signing

Without an active key, Mode is forced to `v` (verify only) regardless
of the saved Mode setting.

## Troubleshooting

| Symptom | Likely cause |
| --- | --- |
| Gmail "Show original" shows `arc=fail (signature failed)` on outbound from a local Hermes user | DNS for selector not published, propagated incorrectly, or wrong key |
| Downstream MX rejects forwarded mail from M365 sender with `arc=fail` | Expected when upstream ARC + body modification meet on relay-out; either ensure the conditional banner skip is active (`/etc/hermes/body_milter/relay_domains` is populated) or ask the receiver to configure Hermes as a Trusted ARC Sealer |
| OpenARC fails to start with `key data is not secure` | The signing key file ownership is not `openarc:openarc` or permissions are too loose; check the entrypoint chown step |
| ARC headers absent from outbound entirely | `arc_signing_enabled = 0` (master off), or no enabled key exists for the configured `arc_mode` |

## Related

- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) — full pipeline diagram including ARC placement
- [DKIM Settings](https://docs.deeztek.com/books/administrator-guide/page/dkim-settings) — outbound signing (separate from ARC)
- [Trusted ARC Sealers — M365](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) — receiver-side trust configuration

# BCC Maps

# BCC Maps

Admin path: **Content Checks > BCC Maps**
(`view_bcc_maps.cfm`, `inc/add_bcc_map_action.cfm`,
`inc/edit_bcc_map_action.cfm`,
`inc/delete_bcc_map_action.cfm`,
`inc/get_bcc_map_json.cfm`,
`inc/get_mailbox_bcc_count.cfm`).

This page manages **silent message copies** at the SMTP envelope
layer. Each entry maps an envelope address (sender **or** recipient,
chosen per row) to a BCC target; when mail matching the address flows
through Postfix, an additional copy is generated and routed to the
target. The original delivery is unaffected; neither the original
sender nor the original recipient sees any indication that a copy was
made.

BCC Maps is the sibling envelope-level rule table to
[Global Sender Rules](https://docs.deeztek.com/books/administrator-guide/page/global-sender-rules). Where Global Sender
Rules decide whether a message is allowed in or blocked, BCC Maps
decides whether an additional copy is created — both work on the
envelope, before the message body is parsed.

## How Postfix BCC works

Postfix has two distinct directives for envelope-level BCC injection:

| Directive | Lookup key | Adds BCC when... | Typical use |
| --- | --- | --- | --- |
| `sender_bcc_maps` | Envelope **sender** (`MAIL FROM`) | The matched address is the one sending the message | Journaling outbound mail from an executive, monitoring a compromised account |
| `recipient_bcc_maps` | Envelope **recipient** (`RCPT TO`) | The matched address is the one receiving the message | Compliance journaling of mail to a regulated mailbox, legal-hold copies |

The two maps are queried independently on every message — a single
delivery can hit both if both a sender BCC and a recipient BCC match.
The BCC happens once Postfix has accepted the message; the original
envelope is preserved and the additional copy is queued separately.

Hermes wires both directives to MySQL-backed lookup tables in
`/etc/postfix/main.cf`:

```
sender_bcc_maps    = mysql:/etc/postfix/mysql-sender-bcc-maps.cf
recipient_bcc_maps = mysql:/etc/postfix/mysql-recipient-bcc-maps.cf
```

Each `.cf` file holds a SQL query that selects `bcc_to` from
`bcc_maps` where the address column matches and the row is enabled.

```sql
-- mysql-sender-bcc-maps.cf
SELECT bcc_to FROM bcc_maps
WHERE address='%s' AND bcc_type='sender' AND enabled=1

-- mysql-recipient-bcc-maps.cf
SELECT bcc_to FROM bcc_maps
WHERE address='%s' AND bcc_type='recipient' AND enabled=1
```

> **No reload required.** Unlike hashed `check_sender_access` lookups
> (used by [Global Sender Rules](https://docs.deeztek.com/books/administrator-guide/page/global-sender-rules)), MySQL
> lookups are evaluated live against the database on every message —
> there is no `postmap` step, no `postfix reload`. Adding, editing,
> disabling, or deleting a row takes effect on the **next inbound
> message**. The UI surfaces this implicitly: the success alerts say
> "entry created/updated/deleted" without the
> "Postfix reloaded and Amavis restarted" suffix that other envelope
> pages append.

## The page

A single info callout, an Add button that opens a modal, and one
DataTable.

### Add BCC Map modal

| Field | Stored as | Notes |
| --- | --- | --- |
| Address | `bcc_maps.address` | The envelope address to watch. Full email (`user@domain.tld`) or `@domain.tld` for domain-wide. Lower-cased on save |
| Type | `bcc_maps.bcc_type` | `sender` (outbound mail from this address) or `recipient` (inbound mail to this address) |
| BCC To | `bcc_maps.bcc_to` | The address that receives the silent copy. Single email only; not a pattern. Lower-cased on save |
| Description | `bcc_maps.description` | Free-text label (e.g. "Legal compliance — exec journaling"); nullable |

The handler validates Address against `IsValid("email", ...)` for full
addresses and against a `@domain` pattern check for domain-wide rows.
BCC To must be a valid email address — domain patterns are not
accepted here, only a concrete delivery target. The
`(address, bcc_type)` pair is `UNIQUE` in the schema, so attempting to
add a second row with the same address and type returns alert
`m = 14` and rejects the insert.

### BCC Maps (DataTable)

| Column | Source |
| --- | --- |
| Actions | Edit (modal, AJAX load via `get_bcc_map_json.cfm`), Delete (confirm modal) |
| Address | `bcc_maps.address` |
| Type | `bcc_maps.bcc_type` -> Sender badge (primary) or Recipient badge (info) |
| BCC To | `bcc_maps.bcc_to` |
| Status | `bcc_maps.enabled` -> Enabled badge (green) or Disabled badge (grey) |
| Description | `bcc_maps.description` (em-dash if empty) |

### Edit constraints

The Edit modal makes **Address** and **Type** read-only — they are
the natural key of the row (`UNIQUE (address, bcc_type)`) and changing
them would semantically be a different rule. To re-target a watched
address, delete the row and add a new one. Only **BCC To**, **Status**
(enabled / disabled), and **Description** can be changed in place.

The Status toggle is the right tool for pausing surveillance briefly
without losing the row — e.g. a compliance journaling rule that
should be off during a planned mail-flow test.

## The `bcc_maps` table

| Column | Purpose |
| --- | --- |
| `id` | Auto-increment primary key |
| `address` | The watched envelope address (full email or `@domain.tld`) |
| `bcc_to` | The silent-copy target address |
| `bcc_type` | `sender` or `recipient` |
| `enabled` | `1` = active, `0` = paused (row preserved, no BCC generated) |
| `description` | Optional free-text label |
| `created_at` | Auto-populated timestamp on insert |
| `UNIQUE KEY` | `(address, bcc_type)` — same address can have one sender BCC AND one recipient BCC, but not two of either |

## BCC mail still goes through content filtering

Important behavior to understand: the BCC copy that Postfix generates
is a real message in its own right, with the BCC target as its
recipient. That copy traverses the same pipeline as any other inbound
delivery — it goes through Amavis, SpamAssassin, ClamAV, the
[Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) for the BCC
target, and any per-recipient quarantine policy.

The consequences:

- **A clean original can produce a quarantined BCC.** If the BCC
  target's spam threshold is stricter than the original recipient's,
  or if a recipient rule rejects the BCC sender, the silent copy can
  be quarantined or dropped while the original delivers normally.
- **A clean original can produce a bounced BCC.** If the BCC target
  is on an external server, that server's SPF / DMARC / receiver
  policy will be evaluated against the original sender's domain
  (which almost certainly does not authorise Hermes's IP). The
  external server may reject the BCC even though the original sender
  has nothing to do with the relay.
- **The BCC failure is silent to the original sender.** Postfix
  generated the BCC after accepting the original message; the
  original sender's SMTP transaction has already closed successfully.
  Any bounce of the BCC goes to the BCC target's `MAIL FROM`
  (typically the original sender, depending on `bounce_size_limit`)
  or to a double-bounce mailbox, but never causes the original
  delivery to fail.

The page's info callout flags the SPF case explicitly. For a
journaling / compliance use case where loss of a copy is
unacceptable, the BCC target should be a **local mailbox** on the
same Hermes instance — the message stays inside the gateway, the
external-receiver policy issue does not arise, and any spam-tier
issue is visible to the local mailbox owner.

## Privacy and compliance

BCC Maps is a surveillance feature. The original sender and the
original recipient are never notified that a copy was made; that is
the point.

Operationally that means:

- **Auditability.** Each row carries a `created_at` timestamp; the
  `description` column is intended for the policy reference that
  justifies the watch (regulatory citation, ticket number, legal-hold
  matter ID). Filling it in is strongly recommended for any rule
  that is not strictly self-explanatory.
- **GDPR / employee-monitoring regimes.** In jurisdictions that
  require explicit employee notification of mail surveillance
  (EU member states, several US states for employee monitoring of
  personal communication), the existence of these rules must be
  disclosed in the employee privacy notice. Hermes does not generate
  that notice — the operator is responsible for the legal compliance
  wrapping around any active row.
- **Access control.** The page is only available to authenticated
  admins under `/admin/2/`. There is no end-user surface for BCC
  maps; mailbox owners cannot see whether their address is watched.

## Cascading delete on mailbox removal

When a mailbox is deleted from
[Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes),
`inc/delete_mailbox_action.cfm` (step 4b) issues:

```sql
DELETE FROM bcc_maps
 WHERE address = :deleted_mailbox
    OR bcc_to  = :deleted_mailbox
```

That is — every BCC rule referencing the deleted mailbox is removed,
whether the mailbox was the watched address or the BCC target.
Because the live MySQL lookup re-reads on every message, the change
takes effect immediately; no postmap or reload runs.

The same delete handler calls the AJAX endpoint
`inc/get_mailbox_bcc_count.cfm` from the confirmation modal **before**
the deletion fires, so the admin sees the number of BCC rows that
will be cascaded ("This mailbox is watched by 2 BCC rules and is the
target of 1 BCC rule") and can cancel.

Domain-pattern rows (`@domain.tld`) are **not** cascaded by mailbox
deletion — they reference a domain, not a specific mailbox, and
remain in place until the whole domain is removed or the row is
deleted manually.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` / `2` / `3` | Add / Edit / Delete success |
| `m = 10` | Address field blank on Add |
| `m = 11` | Address fails email-or-`@domain` syntax check |
| `m = 12` | BCC To blank on Add or Edit |
| `m = 13` | BCC To is not a valid email address |
| `m = 14` | An entry with the same `(address, bcc_type)` already exists |
| `m = 20` | Missing required form field on Edit / Delete (no `bcc_id`) |
| `m = 21` | Edit / Delete target row no longer exists |

There is no `session.m = 4` "Apply Failed" path because there is
nothing to apply — the next message Postfix processes will read the
new row from MySQL directly.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_bcc_maps.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/add_bcc_map_action.cfm` | `hermes_commandbox` | Validate + INSERT |
| `config/hermes/var/www/html/admin/2/inc/edit_bcc_map_action.cfm` | `hermes_commandbox` | Validate + UPDATE (only `bcc_to`, `enabled`, `description`) |
| `config/hermes/var/www/html/admin/2/inc/delete_bcc_map_action.cfm` | `hermes_commandbox` | DELETE single row |
| `config/hermes/var/www/html/admin/2/inc/get_bcc_map_json.cfm` | `hermes_commandbox` | AJAX endpoint for the Edit modal |
| `config/hermes/var/www/html/admin/2/inc/get_mailbox_bcc_count.cfm` | `hermes_commandbox` | AJAX endpoint for the mailbox-delete confirmation modal |
| `config/postfix-dkim/etc/postfix/mysql-sender-bcc-maps.cf` | `hermes_postfix_dkim` | MySQL lookup definition for `sender_bcc_maps` |
| `config/postfix-dkim/etc/postfix/mysql-recipient-bcc-maps.cf` | `hermes_postfix_dkim` | MySQL lookup definition for `recipient_bcc_maps` |
| `bcc_maps` table | `hermes_db_server` (`hermes` DB) | Source of truth |
| `hermes_postfix_dkim` container | — | Reads MySQL lookups live on every message |

## Related

- [Global Sender Rules](https://docs.deeztek.com/books/administrator-guide/page/global-sender-rules) — sibling
  envelope-level rule table; allow/block decisions rather than copy
  generation
- [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) — the
  per-pair table that the BCC copy will also pass through on its way
  to the BCC target
- [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — deleting a mailbox
  cascades the cleanup of any BCC rows referencing it; the
  confirmation modal surfaces the count before the deletion
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — sibling Content Checks
  page; envelope-time rejects that fire before any BCC is generated
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) /
  [Anti-Virus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — the content-filter
  tier that BCC copies traverse alongside the original message
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) — both the original and the
  BCC copy appear as separate entries in the message log
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — Postfix's
  `mail.log` records BCC generation as standard delivery lines, one
  per copy
- [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) — a deferred BCC
  (external target rejecting on SPF, for example) sits in the queue
  here for inspection

# DKIM Settings

# DKIM Settings

Admin path: **Content Checks > DKIM Settings**
(`view_dkim_settings.cfm`, `inc/get_dkim_settings.cfm`,
`inc/dkim_save_settings.cfm`, `inc/dkim_set_settings.cfm`,
`inc/dkim_generate_config_file.cfm`,
`inc/dkim_generate_keytable.cfm`,
`inc/dkim_generate_signingtable.cfm`,
`inc/dkim_generate_hosts.cfm`,
`inc/dkim_generate_domains.cfm`,
`inc/restart_opendkim.cfm`,
`inc/generate_postfix_configuration.cfm`).

This page controls **inbound DKIM verification** and the **OpenDKIM
runtime configuration** that also drives outbound signing.
DKIM ([RFC 6376](https://datatracker.ietf.org/doc/html/rfc6376)) lets
a sending domain attach a cryptographic signature
(`DKIM-Signature: v=1; a=rsa-sha256; d=example.com; s=mail1; ...`)
covering selected headers and a hash of the message body; receivers
fetch the public key at `<selector>._domainkey.<domain>` in DNS and
verify the signature. Unlike SPF, DKIM survives most forwarding —
the signature stays attached to the message and verifies wherever the
body and signed headers remain unchanged.

Per-domain key generation (selector, RSA 1024 / 2048, DNS TXT record
to publish) is managed elsewhere — on the
[Email Server Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) page via
`edit_domain_dkim.cfm`, which writes rows into the `dkim_sign` table.
This Settings page configures the OpenDKIM daemon's runtime behavior
and maintains the verification-side bypass lists.

## Two OpenDKIM instances, one config page

To avoid the body-modification trap that breaks any signer running
after a body-modifying milter, Hermes (issue #232) runs **two
separate OpenDKIM instances** inside `hermes_postfix_dkim`:

| Instance | Config | Socket | Mode | Role |
| --- | --- | --- | --- | --- |
| Primary | `/etc/opendkim.conf` | `inet:8891@0.0.0.0` | `sv` (sign + verify) | Verifies inbound DKIM at `smtpd :25`; signs outbound at `:587` / `:465` (submission ports — pre-Amavis, pre-CipherMail) |
| Sign-only | `/etc/opendkim-sign.conf` | `inet:8892@127.0.0.1` | `s` (sign only) | Signs at the `:10026` re-injection port **after** Amavis, CipherMail, and the body milter have finished modifying the body. Never adds an `Authentication-Results` header |

Both instances share the same key tables
(`/opt/hermes/dkim/KeyTable`,
`/opt/hermes/dkim/SigningTable`) and the same trusted-hosts /
exempt-domains lists; the page below feeds both. The reason for two
instances: a single sv-mode OpenDKIM on `:10026` would verify the
post-modification body of inbound mail flowing through the re-inject
port and emit a spurious `dkim=fail` Authentication-Results header.
Sign-only mode at `:10026` produces the final outbound signature over
the byte sequence the receiver will actually see.

## Where DKIM sits in the flow

```
+--------------------------+
| Remote SMTP peer         |
+-----------+--------------+
            |
            v
+-----------+--------------------------------+
| smtpd :25 (hermes_postfix_dkim)             |
|   smtpd_milters = inet:127.0.0.1:8891, ...  |
|     primary OpenDKIM (sv) verifies inbound  |
|     DKIM-Signature, adds                    |
|     Authentication-Results: dkim=pass/...   |
|     (consumed downstream by OpenDMARC)      |
+-----------+--------------------------------+
            |
            v
        Amavis :10024 (content scoring, CipherMail)
            |
            v (reinject)
+-----------+--------------------------------+
| smtpd :10026 (post-content, post-body-mod)  |
|   smtpd_milters = inet:127.0.0.1:8891       |
|     sign-only OpenDKIM at :8892 actually    |
|     signs the final outbound body           |
|     (KeyTable selects per-domain key by     |
|      "*@<domain>" SigningTable match)       |
+-----------+--------------------------------+
            |
            v
        OpenARC seal (if enabled)
            |
            v
        Outbound to receiver
```

The actual signing decision happens against the `SigningTable`:

```
# /opt/hermes/dkim/SigningTable
*@example.com       mail1._domainkey.example.com
*@partner.org       k2024._domainkey.partner.org
```

…joined to the `KeyTable`:

```
# /opt/hermes/dkim/KeyTable
mail1._domainkey.example.com  example.com:mail1:/opt/hermes/dkim/keys/mail1_example.com.dkim.private
k2024._domainkey.partner.org  partner.org:k2024:/opt/hermes/dkim/keys/k2024_partner.org.dkim.private
```

Both files are regenerated from the `dkim_sign` table on every
key add / enable / disable / delete on the per-domain page.

## The two cards on the page

### 1. DKIM Settings (master toggle + OpenDKIM runtime controls)

**DKIM Enabled** flips the child row in `parameters` whose
`parameter` matches `inet:%:8891` under the `smtpd_milters` parent
(and the same under `non_smtpd_milters`). Disabling DKIM here also
disables DMARC, mirroring the SPF-disable behavior — DMARC needs at
least one of the two to align against. The in-page callout warns
about this dependency.

When enabled, nine controls are written to `parameters2` rows in the
`dkim` module, then substituted into the OpenDKIM template at
`/opt/hermes/conf_files/opendkim.conf.HERMES`:

| Control | OpenDKIM directive | Effect |
| --- | --- | --- |
| Body Canonicalization | `Canonicalization` (body half) | `relaxed` (recommended) ignores trailing whitespace and end-of-line changes; `simple` requires byte-exact body. Most relays touch line endings, so `relaxed` is the only practical choice unless you fully control every downstream hop |
| Headers Canonicalization | `Canonicalization` (header half) | `relaxed` lowercases header names and folds whitespace; `simple` requires headers unchanged. Same reasoning — `relaxed` survives normal relay reformatting |
| Default Message Action | `On-Default` | Catch-all for verification outcomes not covered by the more specific actions below. `accept` is the recommended default |
| Bad Signature Action | `On-BadSignature` | Signature present, present-and-valid in syntax, but verification fails (body or signed-header bytes changed). `accept` (recommended) lets DMARC + spam scoring make the call |
| DNS Error Action | `On-DNSError` | The selector's `_domainkey` TXT record is unreachable or returned SERVFAIL. `accept` (recommended) — DNS instability is the sender's problem, not yours; do not block real mail on transient resolver failures |
| Internal Error Action | `On-InternalError` | OpenDKIM ran out of resources or hit an unexpected runtime error. `accept` (recommended) prevents silent mail loss when the verifier itself fails |
| No Signature Action | `On-NoSignature` | Message arrived unsigned. Many legitimate senders still don't sign — DMARC enforcement is the correct gate for "must be signed", not this knob. `accept` (recommended) |
| Security Concern Action | `On-Security` | Signature references a weak algorithm or unusually short key. `accept` (recommended) — score downstream rather than reject at the milter |
| Signature Algorithm | `SignatureAlgorithm` | `rsa-sha256` (current standard, recommended) or the deprecated `rsa-sha1`. Many receivers reject `rsa-sha1` outright; do not change unless you know why |

Each "Action" option set is: `accept`, `discard`, `reject`,
`tempfail`, `quarantine`. The save handler validates that submitted
values are members of this set before writing.

> **Operational consequence — accept everywhere is intentional.** The
> recommended baseline accepts on every error and every failure
> condition because **DKIM at the milter is not a delivery gate**.
> The verification result is meant to be consumed by DMARC and by
> spam scoring, not to drop mail. Setting any of these to `reject`
> means a single sender DNS hiccup or a single intermediate relay
> rewriting a header can cause real mail to bounce. Leave them at
> `accept` and let DMARC enforcement (which considers the
> sender-published policy) make the discard decision.

### 2. Whitelisted Domains and Trusted Hosts

Two row-per-entry lists that together drive three OpenDKIM directives:

| Entry type | OpenDKIM directive(s) | File on disk | Table |
| --- | --- | --- | --- |
| Whitelisted Domain | `ExemptDomains` | `/opt/hermes/dkim/ExemptDomains` | `dkim_bypass` (`entry`, `note`) |
| Trusted Host | `InternalHosts` + `ExternalIgnoreList` | `/opt/hermes/dkim/TrustedHosts` | `dkim_trusted_hosts` (`host`, `note`) |

**Whitelisted Domain** exempts the listed sender domain from inbound
DKIM verification entirely — OpenDKIM logs the bypass and does not
fetch the selector record. Use for known-broken signers whose mail
you still need to receive (some legacy mailing-list infrastructure,
specific government endpoints with unmaintained selectors).

**Trusted Host** is dual-purpose. The same entries are written to
both `InternalHosts` (mail from these hosts is considered locally
originated and will be DKIM-signed on the way out) and
`ExternalIgnoreList` (mail from these hosts skips inbound DKIM
verification). Accepts IP addresses, CIDR ranges, hostnames, and
bare domain names. The Docker subnet (`172.16.32.0/24` by default)
is pre-populated so the post-Amavis re-inject from `127.0.0.1` and
the inter-container hops are correctly treated as internal.

The DataTable supports add (textarea — one entry per line,
deduplicated), inline edit, single delete, and bulk delete; the row
checkboxes carry an `id|type` composite value so the bulk handler can
route each delete to the right table.

## What this page does NOT control

- **Per-domain DKIM key generation, selector choice, key size, key
  rotation, and the DNS TXT record to publish.** Those live on the
  Email Server [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) page via
  `edit_domain_dkim.cfm` — one selector / key per domain, stored in
  the `dkim_sign` table, written under
  `/opt/hermes/dkim/keys/<selector>_<domain>.dkim.{private,txt}`.
- **The KeyTable and SigningTable content.** These are regenerated
  from `dkim_sign` rows on every key change; do not edit them by
  hand.
- **ARC sealing.** The post-modification chain seal is a separate
  daemon — see [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings).
- **Outbound signing for sub-domains of a signed parent.** OpenDKIM's
  `*@<domain>` SigningTable match does not implicitly cover
  `*@sub.<domain>`. If you sign for `example.com` and need
  `mail.example.com` signed too, generate a separate key for it.

## Per-domain key rotation pattern

A working selector-rotation looks like this (operator-side, not a
single button on the page):

```
1. On edit_domain_dkim.cfm, generate a new key with a new selector
   (e.g. existing "mail1" -> new "mail2"). Mark NEW key disabled.
2. Publish the new key's TXT record at
   mail2._domainkey.example.com in authoritative DNS. The old
   mail1._domainkey.example.com record STAYS published.
3. Verify DNS propagation globally.
4. Enable the new key (disables the old one in dkim_sign atomically).
   KeyTable + SigningTable regenerate; OpenDKIM reloads.
5. Outbound mail now signs with mail2; mail signed with mail1 while
   in flight still verifies because the mail1 TXT record is still
   live.
6. Wait through the typical re-delivery window (24-72 hours).
7. Delete the old mail1 row in dkim_sign; remove the
   mail1._domainkey.example.com TXT record.
```

Selectors are arbitrary DNS labels — `mail1`, `2026q1`, `hermes`,
etc. — and there is no DKIM-defined upper bound on how many you
publish concurrently.

## Save flow

```
1. Validate form fields exist (when enabling DKIM)
   - Missing or out-of-set values -> session.m = 20, redirect, no DB write
2. cfinclude dkim_set_settings.cfm
     a. UPDATE parameters child rows for the smtpd_milters / non_smtpd_milters
        :8891 entries (on or off)
     b. UPDATE parameters2 rows for the nine OpenDKIM runtime directives
     c. cfinclude dkim_generate_config_file.cfm — read
        /opt/hermes/conf_files/opendkim.conf.HERMES, REReplace the
        Canonicalization / On-* / SignatureAlgorithm placeholders, write
        /etc/opendkim.conf
     d. cfinclude dkim_generate_hosts.cfm — regenerate
        /opt/hermes/dkim/TrustedHosts from dkim_trusted_hosts
     e. cfinclude dkim_generate_domains.cfm — regenerate
        /opt/hermes/dkim/ExemptDomains from dkim_bypass
     f. cfinclude dkim_generate_keytable.cfm + dkim_generate_signingtable.cfm
        — rebuild from dkim_sign
     g. cfinclude restart_opendkim.cfm — docker exec inside
        hermes_postfix_dkim to restart BOTH opendkim instances
3. cfinclude generate_postfix_configuration.cfm — regenerate main.cf
   (smtpd_milters list reflects DKIM on/off) and reload Postfix
4. If DKIM was DISABLED: also flip off OpenDMARC milter rows, clear
   FailureReports, deactivate the DMARC report Ofelia job, regenerate
   opendmarc.conf, restart OpenDMARC
5. session.m = 9 -> green "DKIM settings saved" alert on redirect
```

Add / Edit / Delete on the second card calls
`dkim_generate_hosts.cfm` or `dkim_generate_domains.cfm` (whichever
applies) plus `restart_opendkim.cfm` inline — Postfix is not
reloaded since the milter chain itself did not change.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_dkim_settings.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/get_dkim_settings.cfm` | `hermes_commandbox` | Loads current parameters / parameters2 / bypass / trusted-host values |
| `config/hermes/var/www/html/admin/2/inc/dkim_save_settings.cfm` | `hermes_commandbox` | Validates form, calls set + generate + restart chain; disables DMARC if DKIM off |
| `config/hermes/var/www/html/admin/2/inc/dkim_set_settings.cfm` | `hermes_commandbox` | UPDATEs the parameters / parameters2 rows, regenerates all four config files, restarts OpenDKIM |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_config_file.cfm` | `hermes_commandbox` | Renders `/etc/opendkim.conf` from the template + DB |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_keytable.cfm` | `hermes_commandbox` | Rebuilds `/opt/hermes/dkim/KeyTable` from `dkim_sign` |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_signingtable.cfm` | `hermes_commandbox` | Rebuilds `/opt/hermes/dkim/SigningTable` from `dkim_sign` |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_hosts.cfm` | `hermes_commandbox` | Rebuilds `/opt/hermes/dkim/TrustedHosts` from `dkim_trusted_hosts` |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_domains.cfm` | `hermes_commandbox` | Rebuilds `/opt/hermes/dkim/ExemptDomains` from `dkim_bypass` |
| `config/hermes/opt/hermes/conf_files/opendkim.conf.HERMES` | `hermes_commandbox` (read) → `hermes_postfix_dkim` (live `/etc/opendkim.conf`) | Template with `HEADER-CANONICALIZATION`, `BODY-CANONICALIZATION`, `DEFAULT-ACTION`, etc. placeholders |
| `config/postfix-dkim/etc/opendkim-sign.conf` | `hermes_postfix_dkim` | Static config for the sign-only instance at `:8892` (no placeholders — relaxed/relaxed + rsa-sha256 are fixed for the re-injection signer) |
| `parameters` table (`inet:%:8891` rows under `smtpd_milters` and `non_smtpd_milters`) | `hermes_db_server` (`hermes` DB) | DKIM milter on/off |
| `parameters2` table (rows where `module='dkim'`) | `hermes_db_server` (`hermes` DB) | The nine OpenDKIM runtime settings |
| `dkim_sign`, `dkim_bypass`, `dkim_trusted_hosts` tables | `hermes_db_server` (`hermes` DB) | Per-domain keys, exempt-domain list, trusted-host list |
| `hermes_postfix_dkim` container | — | Runs both OpenDKIM instances and hosts the live config + key files |
| `hermes_unbound` container | — | Resolves every `<selector>._domainkey.<domain>` lookup |

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Missing form fields when enabling DKIM | `session.m = 20`, redirect, no DB write |
| Out-of-set value submitted for an Action / Canonicalization / Algorithm field | `session.m = 20`, redirect, no DB write |
| Empty entry on Add | `session.m = 13`, redirect, no DB write |
| Invalid syntax on Add / Edit | `session.m = 17`, redirect, no DB write |
| Duplicate entry on Add | `session.m = 14`, redirect, no DB write |
| `dkim_generate_config_file.cfm` write fails | Surfaces as `cfcatch` from the inline include — save aborts |
| `restart_opendkim.cfm` fails | Same path — Postfix is reloaded anyway in step 3, but DKIM service is left in the prior runtime state |
| `KeyTable` / `SigningTable` missing because no `dkim_sign` rows exist yet | OpenDKIM starts but signs nothing — outbound mail goes out unsigned |

## Related

- [SPF Settings](https://docs.deeztek.com/books/administrator-guide/page/spf-settings) — the second authentication
  service whose result is consumed by DMARC; paired conceptually
  with DKIM as a "DNS-based outbound sender authentication"
  mechanism. SPF checks at envelope `MAIL FROM` time; DKIM checks
  header signatures after `DATA`. DKIM survives forwarding;
  SPF generally doesn't
- [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) — the policy layer that
  consumes DKIM (and SPF) results; disabling DKIM here automatically
  disables DMARC
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — the post-modification chain
  seal, which runs after the sign-only OpenDKIM at `:8892` so the
  ARC record covers the final outbound body
- [Trusted ARC Sealers (M365)](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) — for
  M365 customers whose downstream verifiers escalate when a
  Hermes-forwarded message's original DKIM signature breaks against
  the body-modified bytes
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — the SPF / DKIM / DMARC
  status card on Perimeter Checks links here for the per-service
  toggle
- [Domains (Email Server)](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — where
  per-domain DKIM keys are generated, selectors chosen, and DNS TXT
  records exposed for publication
- [Domains (Email Relay)](https://docs.deeztek.com/books/administrator-guide/page/domains) — relay-mode
  domains can also sign outbound; same per-domain key UX
- [Email Policies > Disclaimers](https://docs.deeztek.com/books/administrator-guide/page/disclaimers) —
  documents the body milter that modifies outbound bodies before the
  sign-only OpenDKIM at `:8892` produces the final signature; the
  two-instance OpenDKIM design exists precisely because of this body
  modification
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — every
  `<selector>._domainkey.<domain>` lookup flows through
  `hermes_unbound`; resolver mode directly affects DKIM verification
  reliability
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — TLS
  on outbound delivery is independent of DKIM, but receivers that
  enforce strict transport security may surface DKIM failures more
  prominently in failure reports

# DMARC Settings

# DMARC Settings

Admin path: **Content Checks > DMARC Settings**
(`view_dmarc_settings.cfm`, `inc/get_dmarc_settings.cfm`,
`inc/dmarc_save_settings.cfm`, `inc/dmarc_set_settings.cfm`,
`inc/dmarc_generate_config_file.cfm`,
`inc/dmarc_generate_reports_script.cfm`,
`inc/restart_opendmarc.cfm`).

This page controls Hermes's OpenDMARC milter — both whether DMARC is
evaluated on inbound mail and, when enabled, what happens to
verdicts and whether daily aggregate reports are generated for the
domains that publish a DMARC record. DMARC ([RFC 7489](https://datatracker.ietf.org/doc/html/rfc7489))
is the policy layer that sits on top of SPF and DKIM; a sender
publishes a `_dmarc.<domain>` TXT record telling receivers what to do
when neither SPF nor DKIM aligns with the From: header domain. Hermes
is the receiver that does the work.

## How DMARC fits the auth stack

```
                  +--------------------+
   inbound msg -->| SPF check          |  passes/fails on envelope-from IP
                  +---------+----------+
                            |
                            v
                  +--------------------+
                  | DKIM verify        |  passes/fails on each signature
                  +---------+----------+
                            |
                            v
                  +--------------------+
                  | OpenDMARC          |  reads SPF + DKIM AR headers,
                  |   :54321 milter    |  fetches _dmarc.<from-domain>
                  +---------+----------+  evaluates alignment + policy
                            |
                            v
                  +--------------------+
                  | RejectFailures?    |
                  | -> reject / accept |
                  +--------------------+
```

A message **aligns** when its From: header domain matches the
SPF-pass envelope-from domain OR the DKIM-pass `d=` domain.
Relaxed alignment (the default) accepts org-domain match
(`example.com` aligns with `mail.example.com`); strict alignment
requires exact match. OpenDMARC reads the alignment results that
SPF and DKIM have already written into the `Authentication-Results`
header — both checks must therefore be active before DMARC is useful.
The UI enforces this: enabling DMARC with SPF or DKIM disabled
returns error 1.

## Container and milter placement

| Component | Detail |
| --- | --- |
| Container | `hermes_dmarc` (separate service, IPv4 `.111`) |
| Listen | `inet:54321@[0.0.0.0]` (`Socket` directive in `opendmarc.conf`) |
| Source | OpenDMARC daemon (Trusted Domain Project), packaged in the `hermes-dmarc` image |
| Milter chain | Postfix `smtpd_milters` AND `non_smtpd_milters` parents, child row `inet:<container>:54321` — toggle flips `enabled` on that row |
| DMARC report DB | `opendmarc` database on `hermes_db_server`, credentials in `system_settings` rows `mysql_username_opendmarc` / `mysql_password_opendmarc` |
| History file | `/etc/opendmarc/opendmarc.dat` inside `hermes_dmarc` (volume-mounted from `./config/opendmarc/etc/opendmarc/`) |

The container exposes **no host ports** — Postfix reaches OpenDMARC
internally at `inet:hermes_dmarc:54321`. The whitelist file path
referenced by `DomainWhitelistFile` resolves to
`/etc/opendmarc/whitelist.domains`, written by
`inc/dmarc_generate_domains.cfm` from the `dmarc_domains` table on
every save.

## DMARC Settings card

Six controls drive `opendmarc.conf` directly via placeholder
substitution into `/opt/hermes/conf_files/opendmarc.conf.HERMES`.

| UI Control | `opendmarc.conf` directive | What it does |
| --- | --- | --- |
| DMARC Enabled (YES/NO) | Milter chain toggle | Enables the `inet:%:54321` child row under `smtpd_milters` and `non_smtpd_milters`; OpenDMARC stops being consulted entirely when disabled |
| Reject Failures | `RejectFailures` (true/false) | When true, messages failing DMARC evaluation are rejected (or temp-failed if evaluation could not complete). When false, the message is accepted and only an `Authentication-Results` header records the verdict |
| Hold Quarantine Policy Messages | `HoldQuarantinedMessages` (true/false) | When true, messages from domains publishing `p=quarantine` that fail DMARC are routed to the Postfix `hold` queue for manual release/delete. When false (recommended), quarantine-policy messages are delivered with an `Authentication-Results` annotation and downstream scoring handles them |
| Generate Daily Failure Reports | `FailureReports` (true/false) | When true, OpenDMARC writes failure records to the history file and the daily Ofelia job converts them to RFC 6591 aggregate reports |
| Failure Reports From E-mail | `--report-email` flag on `opendmarc-reports` | RFC 6591 envelope From: for the outgoing report — must be a valid email address (validated by `IsValid("email", ...)`) |
| Failure Reports Reporting Organization | `--report-org` flag | Identifies your gateway as the report source — alphanumeric only (validation regex: `[^A-Za-z0-9]`) |

OpenDMARC's `FailureReports` triggers reports only for domains that
publish `p=quarantine` or `p=reject` (it never auto-reports for
`p=none` unless `FailureReportsOnNone` is also set — Hermes does not
expose that directive).

The "Reject Failures" UI hint and the OpenDMARC docs use the same
language: messages that fail are **rejected** when policy is `reject`,
**delivered with header** when policy is `none`, and **either held or
flagged** when policy is `quarantine` (depending on
`HoldQuarantinedMessages`).

> **Operational consequence — RejectFailures = true.** When this is on,
> OpenDMARC will respond `550 5.7.0` to messages from domains
> publishing `p=reject` that fail evaluation, and Postfix will refuse
> the message in-band. This catches forged messages but also catches
> legitimate forwarded mail from senders whose original SPF / DKIM
> chain breaks at an upstream forwarder. If you start seeing
> legitimate forward-from-mailing-list mail bounce, the fix is to add
> the originating domain to the Whitelisted Domains card below — not
> to disable Reject Failures globally.

## Whitelisted Domains card

Rows from the `dmarc_domains` table (`id`, `domain`, `note`,
`type`) write to `/etc/opendmarc/whitelist.domains`. OpenDMARC reads
that file via `DomainWhitelistFile` and bypasses DMARC evaluation
entirely for any matching From: domain — no alignment check, no
policy enforcement, no failure report. Use for trusted senders with
known broken DMARC, partner domains that forward through aggregators
that strip headers, or legacy mailing lists.

Only domain names are accepted; IP addresses are rejected by the add
handler. Domains are validated by the same regex used elsewhere in
Hermes (e.g. error 17: "The entry is not a valid domain"). Bulk
add is supported one-per-line in the textarea.

## DMARC report generation (daily aggregate / RUA)

When **Generate Daily Failure Reports** is enabled, `dmarc_set_settings.cfm`
calls `dmarc_generate_reports_script.cfm` which renders
`/opt/hermes/scripts/dmarc_report_script.sh` with credentials and
identifiers substituted into placeholders (`DATABASE-SERVER`,
`DATABASE-USER`, `DATABASE-PASSWORD`, `REPORTING-EMAIL`,
`REPORTING-ORGANIZATION`, `POSTMASTER-EMAIL`) and writes the result
to `/opt/hermes/schedule/dmarc_report_script.sh` (chmod +x).

An [Ofelia](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) job named
`hermes-dmarc-report` runs the script daily at 02:30:

```
[job-exec "hermes-dmarc-report"]
schedule:  0 30 02 * * *
container: hermes_dmarc
command:   /opt/hermes/schedule/dmarc_report_script.sh
```

The script does three things in sequence:

1. **`opendmarc-import`** — drains `/etc/opendmarc/opendmarc.dat`
   (the per-message verdict log OpenDMARC writes) into the
   `opendmarc` MariaDB database
2. **`opendmarc-reports`** — generates RFC 6591 aggregate XML reports
   for the prior 24h interval and emails one report per sender
   domain to the `rua=` address that domain published in DNS
3. **`opendmarc-expire`** — drops records older than the retention
   window from the database

The script also emits a Net::SMTP success/failure notification to the
`postmaster` address (from `system_settings`). The Perl one-liner
passes the postmaster address through an environment variable rather
than direct string interpolation — Perl's default array sigil `@`
treats `@domain.tld` as an array dereference and silently loses the
domain part. Passing via `$ENV{POSTMASTER_ARG}` avoids the trap (the
fix landed as issue #215). The notification is also skipped entirely
when `postmaster` is not a valid email address (e.g. bare local-part
like `postmaster`) — this prevents queue pollution with
undeliverable bounces.

SMTP delivery uses `hermes_postfix_dkim:10026` (the post-amavis
re-injection port) — using `:25` would re-process the report through
the inbound pipeline and could re-trigger DMARC evaluation on the
report itself.

When **Generate Daily Failure Reports** is disabled (or DMARC itself
is disabled), the save handler:

- Deletes `/opt/hermes/schedule/dmarc_report_script.sh`
- Sets `ofelia_jobs.active = '2'` on the `hermes-dmarc-report` job
  and regenerates `/etc/ofelia/config.ini` via
  `ofelia_generate_config.cfm`

## Forensic (RUF) reports

Forensic (per-failure) reports are intentionally **not** generated by
Hermes. They are privacy-noisy (they include redacted copies of
failing messages), receivers rarely publish a `ruf=` address, and the
modern operational consensus is that aggregate (RUA) reports give
operators the visibility they need without the per-message exhaust.
The `FailureReportsBcc` / `FailureReportsSentBy` / `CopyFailuresTo`
directives in `opendmarc.conf.HERMES` are left commented and not
exposed in the UI.

## ARC interaction

Hermes also runs an [ARC](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) sealer
(`hermes_openarc`) on the same authentication stack. When Hermes
modifies a message body (External Sender Banner, disclaimer
injection, signature injection, S/MIME or PGP rewrap), the
original sender's DKIM body hash no longer matches the current
body — DMARC alignment is lost on the modified copy. ARC preserves
the pre-modification verdict in a sealed chain so downstream
receivers configured to trust Hermes can still rescue DMARC
alignment. See [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) and the
[Trusted ARC Sealers — M365 guide](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) for
the receiver-side configuration. Hermes is the authoritative auth
boundary for every domain it serves; customer downstream MX
allowlisting is the standard remedy when ARC trust is not in play.

## Save flow

```
1. View page submits action=save_settings or add_domain / edit_domain / delete_domain
2. dmarc_save_settings.cfm validates:
     - SPF + DKIM both enabled (error 1 if not)
     - rejectfailures / holdquarantinedmessages / failurereports are true|false (error 20)
     - if failurereports=true: report_email present + valid (errors 2, 3)
                               report_org present + alphanumeric (errors 4, 5)
3. dmarc_set_settings.cfm UPDATEs:
     - parameters.enabled on the inet:%:54321 child row (smtpd + non_smtpd)
     - parameters2.value2 on FailureReports / RejectFailures / HoldQuarantinedMessages
       (module = 'dmarc')
     - parameters2.value2 on report_email / report_org (when reports enabled)
4. dmarc_generate_config_file.cfm:
     - Copies opendmarc.conf.HERMES to /opt/hermes/tmp/<trans>_opendmarc.conf
     - Substitutes FAILURE-REPORTS, REJECT-FAILURES, HOLD-QUARANTINE-MESSAGES placeholders
     - Backs up /etc/opendmarc/opendmarc.conf -> opendmarc.HERMES
     - Moves the rendered file into place
5. dmarc_generate_reports_script.cfm (if reports enabled):
     - Renders dmarc_report_script.sh, chmod +x
     - Enables ofelia_jobs row for hermes-dmarc-report, regenerates Ofelia config
   (else: deletes the script, disables the Ofelia row)
6. restart_opendmarc.cfm: docker container restart hermes_dmarc
7. generate_postfix_configuration.cfm: postconf -e the milter list, postfix reload
8. session.m = 9 -> green "DMARC settings saved successfully. Postfix reloaded." alert
```

## Failure semantics

| Failure | Behavior |
| --- | --- |
| SPF or DKIM not enabled when DMARC=YES | session.m = 1, redirect, no DB write |
| `report_email` empty | session.m = 2 |
| `report_email` invalid | session.m = 3 |
| `report_org` empty | session.m = 4 |
| `report_org` contains non-alphanumeric | session.m = 5 |
| Missing required form fields | session.m = 20 |
| Delete Domains clicked with nothing selected | session.m = 11 |
| Add Domain with empty Domain field | session.m = 13 |
| Add Domain with invalid format | session.m = 17 |
| Add Domain with duplicate | session.m = 14 (single) or `_exists` alert (bulk) |

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_dmarc_settings.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/dmarc_*.cfm` | `hermes_commandbox` | Validate / save / generate / restart |
| `config/hermes/opt/hermes/conf_files/opendmarc.conf.HERMES` | `hermes_commandbox` (read) -> `hermes_dmarc` (live `/etc/opendmarc/opendmarc.conf`) | Canonical template |
| `config/hermes/opt/hermes/scripts/dmarc_report_script.sh` | `hermes_commandbox` (read) -> rendered into `/opt/hermes/schedule/` (executed in `hermes_dmarc`) | Daily aggregate report script |
| `/etc/opendmarc/whitelist.domains` | `hermes_dmarc` | Generated from `dmarc_domains` table on every save |
| `/etc/opendmarc/opendmarc.dat` | `hermes_dmarc` | Per-message verdict history; drained nightly by `opendmarc-import` |
| `opendmarc` MariaDB DB | `hermes_db_server` | Holds imported verdicts that `opendmarc-reports` reads |
| `parameters` / `parameters2` tables (`module='dmarc'`) | `hermes_db_server` (`hermes` DB) | Source of truth for every directive |
| `system_settings` rows `mysql_username_opendmarc` / `mysql_password_opendmarc` | `hermes_db_server` | DB creds for the report script (managed via `update_opendmarc_db_creds.cfm`) |
| `ofelia_jobs` row `hermes-dmarc-report` | `hermes_db_server` | Daily report scheduler entry |

## Related

- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — the SMTP-time card whose
  Email Authentication badge shows DMARC's wired-up status and the
  "Requires both SPF and DKIM" callout
- [SPF Settings](https://docs.deeztek.com/books/administrator-guide/page/spf-settings) — the alignment input for the
  envelope From: side
- [DKIM Settings](https://docs.deeztek.com/books/administrator-guide/page/dkim-settings) — the alignment input for the
  signature `d=` side
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — preserves the DMARC verdict
  across body-modifying forwarding hops
- [Trusted ARC Sealers — M365](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) —
  receiver-side configuration to trust Hermes's ARC seal
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — runs after DMARC and
  can promote a DMARC-fail message to higher spam score
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — per-rule weight changes
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — every `_dmarc` TXT
  lookup goes through `hermes_unbound`; resolver mode (recursive vs.
  forwarding) directly affects DMARC accuracy and report timing
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) — full pipeline diagram
  with milter placement

# File Expressions

# File Expressions

Admin path: **Content Checks > File Expressions**
(`view_file_expressions.cfm`,
`inc/get_file_expressions.cfm`,
`inc/update_amavis_config_files.cfm`).

This page maintains the catalogue of **regex patterns** that Amavis
can match against attachment filenames. Where
[File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) is a one-extension-per-row list
(`.exe`, `.docm`, `.iso`), File Expressions is the free-form regex
sibling — any Perl-compatible pattern that should fire on the
attachment name: double-extension traps (`^.+\.(exe|scr)\.[a-z0-9]+$`),
disguised-archive patterns (`^invoice.*\.pdf\.zip$`), or any
project-specific filename signature an extension list can't express.
The page itself does not block anything — it only registers patterns.
The block / allow decision is taken by a [File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules)
that bundles expressions (and extensions, file types, MIME types)
into a named ruleset, which is then bound to recipient traffic via
an SVF policy on [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings).

The expression catalogue is **entirely operator-driven** — Hermes
ships no system-managed expressions. The shipped High-Risk catch-all
("Double Extensions in File Name") and the Windows Class ID block
live on the [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) page as
`type = 'FILE-HIGH'` rows. Everything on the File Expressions page
is something the operator added.

## Where File Expressions sits

```
                       +---------------------------------------+
   File Expressions    |  files table                          |
   (this page)  -----> |   id, file ("\.exe$"),                |
                       |   description ("Executable files"),   |
                       |   type ("CUSTOM-EXPRESSION"),         |
                       |   system ("NO"),                      |
                       |   allow ("[qr'\.exe$'i => 0]"),       |
                       |   ban   ("[qr'\.exe$'i => 1]")        |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  File Rules                           |
                       |   bundle expressions + extensions     |
                       |   into named rulesets with per-item   |
                       |   allow / ban / priority              |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  Anti-Spam Settings (SVF Policies)    |
                       |   bind a File Rule to recipient(s)    |
                       |   via policy.banned_rulenames         |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  Amavis 50-user.HERMES                |
                       |   @banned_filename_re emitted per     |
                       |   rule on every save chain            |
                       +---------------------------------------+
```

The rendered `@banned_filename_re` block is enforced at
content-filter time inside `hermes_mail_filter`. A matched expression
triggers Amavis's `final_banned_destiny` action (`D_BOUNCE`,
`D_DISCARD`, or `D_PASS` — set globally on
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings)).

## How the pattern is wrapped

The textarea takes a raw Perl regex. On save the handler wraps it
into Amavis's `qr//` syntax with the `i` (case-insensitive) modifier
and stores both the allow and ban form on the row:

```
[qr'\.exe$'i => 0]      (allow form, stored in files.allow)
[qr'\.exe$'i => 1]      (ban form,  stored in files.ban)
```

Whether the allow or ban form gets rendered into Amavis's
`@banned_filename_re` is decided at File Rule time, not here. The
File Expressions page does not have an allow/ban toggle — both forms
are stored so the same expression can serve allow-rules and
ban-rules without re-typing.

There is no case-sensitive variant on this page. Every File
Expression is stored with the `i` modifier. Operators who need
strict case have to drop down to the File Rule's per-component
selection or use a regex character class on the pattern itself
(`\.[Ee][Xx][Ee]$`).

## The page

A page guide callout, an Expression Helper card (build / pick /
test), an Add Expressions card with a bulk textarea, and a single
DataTable listing every custom expression. The DataTable is
flat — system vs. custom does not apply because the catalogue is
all-custom by design.

### Expression Helper card

A three-section utility, collapsed by default, that exists so
operators don't need to know regex to add common patterns.

| Section | Purpose |
| --- | --- |
| Build an Expression | Pick a match mode (Ends with / Starts with / Contains / Exact), enter plain text, click **Build**. The helper regex-escapes the input, wraps it with the appropriate anchors (`^…`, `…$`, `^…$`), and shows the generated pattern with a plain-English explanation |
| Quick Select Common Patterns | A dropdown of pre-built patterns (`\.exe$`, `\.bat$`, `^invoice`, `\.(exe\|bat\|cmd\|scr\|pif)$`, etc.) — click **Use** to drop the pattern into the Add form |
| Test a Pattern | A pattern + filename pair with a **Test** button — runs `new RegExp(pattern, 'i').test(filename)` in the browser and reports Match / No match / Invalid regex. Lets the operator sanity-check before saving |

The Build helper escapes `. * + ? ^ $ { } ( ) | [ ] \` in the user
input before wrapping, so a builder entry of `invoice.pdf` becomes
`invoice\.pdf$`, not `invoice.pdf$`.

### Add File Expressions card

| Field | Stored as | Notes |
| --- | --- | --- |
| File Expressions | `files.file` (the regex) + `files.description` | One per line; format is `regex_pattern description` where the first space separates pattern from label. A pattern with no space becomes its own description (useful for self-documenting patterns like `\.docm$`) |

The handler line-splits the textarea on LF or CRLF, strips
whitespace, and inserts each non-blank entry. Per entry it
checks one thing: that no row already exists in `files` with the
same `file` value under `type = 'CUSTOM-EXPRESSION'`. Duplicates
are skipped and surfaced in the partial-success alert
("Duplicate: \\.exe\$"); the rest still insert.

There is **no regex-validity check on save** — the regex is stored
as-typed and any syntax error is exposed at Amavis reload time, not
in the alert. Use the Test a Pattern section of the helper before
saving to catch malformed patterns first.

### File Expressions DataTable

| Column | Source |
| --- | --- |
| (checkbox) | Selection for bulk Delete Selected |
| Regex Pattern | `files.file` (rendered inside a `<code>` block) |
| Description | `files.description` |
| Actions | Per-row Delete button (single-row confirm) |

The DataTable shows only `type = 'CUSTOM-EXPRESSION'` rows. No
edit-in-place — to change a pattern the operator deletes it and
re-adds.

## Foreign-key guard on delete

A custom expression cannot be deleted while it is referenced by any
[File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules). The single-row Delete handler runs:

```sql
SELECT COUNT(*) AS cnt FROM file_rule_components
WHERE file_id = :id
```

If `cnt > 0`, the delete is refused with alert `m = 40` and the
DataTable shows the offending rule name(s) ("This expression is
referenced by the following File Rule(s): **Block-Disguised-Exe**").
The operator's path is to open File Rules, remove the expression
from the rule, then come back here and delete it.

Bulk Delete applies the same guard per-id and accumulates partial
results — alert `m = 41` reports "N deleted, M blocked" with the
blocked rows' pattern and rule names attached, so the operator knows
exactly what to unwire first.

## Save and apply flow

```
1. View page submits action="add_entries" | "delete" | "bulk_delete"
2. For each valid entry:
     a. Generate ban  string: "[qr'<pattern>'i => 1]"
     b. Generate allow string: "[qr'<pattern>'i => 0]"
     c. INSERT INTO files (file, description, type, system, allow, ban)
        with type='CUSTOM-EXPRESSION' and system='NO'
3. If at least one row was added or deleted:
     a. update_amavis_config_files.cfm:
          - Read /opt/hermes/conf_files/50-user.HERMES (template)
          - Substitute the SERVER/destiny/DKIM/MySQL-credential
            placeholders from spam_settings and creds files
          - Render every File Rule's components into an
            @banned_filename_re block (per-rule, in priority order,
            using the allow/ban regex stored on each files row -
            including the CUSTOM-EXPRESSION rows this page creates)
          - Back up /etc/amavis/conf.d/50-user -> 50-user.HERMES,
            move rendered file into place
     b. docker exec hermes_mail_filter /etc/init.d/amavis force-reload
        (30-second timeout)
4. session.m = 1 (add) | 2 (single/bulk delete) | 30 (empty submit)
   | 40 (FK refused) | 41 (bulk partial)
```

Amavis is reloaded with `force-reload` rather than restarted — the
daemon re-reads `50-user` without dropping connections, and mail in
flight is not interrupted. The reload step is wrapped in
`cftry`/`cfcatch` and the catch block is intentionally silent: if
the reload itself fails the DB rows are already in place, and the
next save (or a manual `force-reload`) will re-render. The page
does not roll back on reload failure.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` | Add Expressions completed (with `entries_added` / `entries_skipped` / `entry_errors` set on session for the per-row breakdown) |
| `m = 2` | Single Delete succeeded; Amavis reloaded |
| `m = 30` | Add submitted with an empty textarea |
| `m = 31` | Pattern field empty (legacy edit path, no longer reachable from the current UI) |
| `m = 32` | Duplicate pattern (legacy edit path) |
| `m = 40` | Single Delete refused — the expression is wired into at least one File Rule (rule names surfaced in the alert) |
| `m = 41` | Bulk Delete partial — `deleted_count` rows removed, `blocked_count` rows refused (the per-row pattern + rule-name list is HTML-rendered into the alert body) |

The per-row error list is HTML-rendered into alert `m = 1` so the
operator sees every duplicate at once. No row is silently dropped
without an explanation.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_file_expressions.cfm` | `hermes_commandbox` | The page (add + delete + bulk delete + Expression Helper + Amavis reload) |
| `config/hermes/var/www/html/admin/2/inc/get_file_expressions.cfm` | `hermes_commandbox` | Loads `type = 'CUSTOM-EXPRESSION'` rows into the DataTable |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Renders `50-user` from template + File Rules (called on every change here too — expression edits affect rendered `@banned_filename_re` blocks) |
| `config/hermes/opt/hermes/conf_files/50-user.HERMES` | `hermes_commandbox` (read) -> `hermes_mail_filter` (live `/etc/amavis/conf.d/50-user`) | Canonical Amavis template; receives the rendered `@banned_filename_re` blocks |
| `/etc/amavis/conf.d/50-user` | `hermes_mail_filter` | Live Amavis config; reloaded with `force-reload` on every save |
| `files` table, `type = 'CUSTOM-EXPRESSION'` | `hermes_db_server` (`hermes` DB) | Source of truth for the expression catalogue |
| `file_rule_components` table | `hermes_db_server` (`hermes` DB) | Cross-reference checked by the delete guard |
| `hermes_mail_filter` container | — | Hosts Amavis; receives `force-reload` (not restart) on every change |

## Operational consequences

- **No regex validation at save.** A malformed regex inserts cleanly
  and only surfaces at Amavis reload time. The reload itself does
  not roll back the DB. If reload starts failing immediately after
  an Add, the most recent expression is the suspect — open it,
  paste it into the Test a Pattern helper, and look for unescaped
  metacharacters or unbalanced groups. The pattern with `\.exe$`
  works; a typo of `\.exe$.` (trailing dot) parses but matches
  nothing.
- **Case is always insensitive.** Every expression renders with the
  `i` modifier. There is no per-expression case toggle. Operators
  who need strict case have to encode it in the pattern itself.
- **Order does not matter on this page.** Expressions are stored
  flat. The evaluation order that Amavis sees is decided by the
  File Rule that bundles them — each component's `priority` column
  on `file_rule_components`. Changing the description here will
  not reorder anything.
- **Custom-Expression rows are visible to File Rules under
  "Custom Expressions".** When the operator opens the Add/Edit
  modal on [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules), every row this page creates
  shows up in the **Custom Expressions** card alongside the system
  catalogue. That is the only place the bundling happens.

## Related

- [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) — sibling page for plain
  extension entries (`.exe`, `.docm`); the simpler half of the same
  `files` table, distinguished by `type IN ('EXT', 'EXT-HIGH')`
- [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules) — bundles extensions and expressions
  into named, prioritised rulesets; the consumer of every row this
  page creates
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) — content-level SpamAssassin
  rules (header / body / regex) — the body / header equivalent of
  what File Expressions does for attachment names
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — defines
  `final_banned_destiny` (what Amavis does with a banned-expression
  match) and binds File Rules to recipients via SVF Policies
- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — ClamAV runs in the
  same Amavis pass; a virus verdict on the same attachment
  overrides the banned-expression result
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — sibling Amavis tuning
  page; both write into Amavis configuration but expression matches
  are categorical (matched -> banned) where SA rules are weighted
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — note that banned-expression
  rejections are a body-side filter result, not an authentication
  result — they fire after ARC chain evaluation
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) — a banned-expression
  rejection appears with Type `Banned` and the matched expression
  surfaced in the detail view
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — Amavis logs the
  matched regex as `Blocked BANNED (\.exe$,…)` on the
  `amavis[...]:` line

# File Extensions

# File Extensions

Admin path: **Content Checks > File Extensions**
(`view_file_extensions.cfm`,
`inc/get_file_extensions.cfm`,
`inc/update_amavis_config_files.cfm`).

This page maintains the catalogue of **attachment file extensions**
that Amavis can match on. Each entry is a single extension such as
`.exe`, `.docm`, or `.iso` paired with a description and a sensitivity
flag (Standard vs. High Risk). The page itself does not block anything
— it only registers extension candidates. The block / allow decision
is taken by a [File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules) that bundles extensions into
a named ruleset, which is then applied to recipients via an SVF
policy on [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings). File Extensions
is the building-block page; File Rules and SVF Policies are where the
ruleset is composed and bound to traffic.

The extension catalogue ships with a system-managed list of common
high-risk types (`.exe`, `.scr`, `.pif`, `.com`, `.bat`, `.vbs`,
`.js`, `.jar`, `.ps1`, and dozens more) that cannot be deleted from
the UI. Operators add custom extensions on top — typically Office
macro-enabled types in environments that don't allow macros, archive
formats they want to surface separately, or new attack-surface file
types as they appear in the wild.

## Where File Extensions sits

```
                       +---------------------------------------+
   File Extensions     |  files table                          |
   (this page)  -----> |   id, file ("exe"), description,      |
                       |   type ("EXT" | "EXT-HIGH"),          |
                       |   system ("YES"/"NO"),                |
                       |   allow ("[qr'.\.(exe)$'i => 0]"),    |
                       |   ban   ("[qr'.\.(exe)$'i => 1]")     |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  File Rules                           |
                       |   bundle extensions into named        |
                       |   rulesets with per-extension         |
                       |   allow / ban / priority              |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  Anti-Spam Settings (SVF Policies)    |
                       |   bind a File Rule to recipient(s)    |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  Amavis 50-user.HERMES                |
                       |   @banned_filename_re emitted per     |
                       |   rule on every save chain            |
                       +---------------------------------------+
```

Amavis enforces the resulting `@banned_filename_re` regex sets at
content-filter time inside `hermes_mail_filter`. A matched extension
triggers Amavis's `final_banned_destiny` action (`D_BOUNCE`,
`D_DISCARD`, or `D_PASS` — set globally on
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings)).

## What "matched" means in Amavis

The stored allow / ban snippets are case-insensitive regexes anchored
to the end of the filename:

```
[qr'.\.(exe)$'i => 1]      (ban; case-insensitive)
[qr'.\.(exe)$'x  => 0]     (allow; case-sensitive)
```

This means:

- `invoice.exe` matches `.exe`
- `Invoice.EXE` matches `.exe` (because the `i` modifier is set by
  default on Add)
- `invoice.pdf.exe` matches `.exe` (the *trailing* extension is the
  one Amavis tests)
- `invoice.exe.pdf` does **not** match `.exe` — it matches `.pdf`,
  and the trailing-extension rule is the only one that fires

The double-extension confusion case (`invoice.pdf.exe`) is the
historic reason this list exists. Amavis sees the real trailing
extension; the user sees only the displayed-name prefix and a
familiar icon.

## The page

A page guide callout, an Add Extensions card with a bulk textarea,
a Custom File Extensions DataTable (editable / deletable), and a
Read-Only System File Extensions DataTable (the shipped list).

### Add File Extensions card

| Field | Stored as | Notes |
| --- | --- | --- |
| File Extensions | `files.file` + `files.description` | One per line; format `.ext description`. The leading dot is stripped on save (so the row stores `exe`, not `.exe`); the description is auto-prefixed with `(.ext)` so the DataTable shows `(.docm) Microsoft Word Macro-Enabled Document` regardless of how the operator typed it |
| Extension Type | `files.type` | `EXT` (Standard) or `EXT-HIGH` (High Risk). Purely a classification tag for the UI badges — Amavis treats both the same |
| Case Sensitivity | drives which template is rendered into `files.allow` / `files.ban` | Insensitive (default, recommended) uses `_insense` templates with the `i` regex modifier; sensitive uses `_sense` templates with `x` only — for environments where you want `.EXE` to differ from `.exe` |

The handler line-splits the textarea on either LF or CRLF, strips
whitespace, validates each entry, and inserts the valid ones. Per
entry it checks:

- The extension starts with `.`
- The extension matches `^[.][a-zA-Z0-9\-\.\_]+$` (alphanumeric, dash,
  period, underscore — nothing else)
- The description is non-blank (required)
- No row with the same `file` already exists in the `EXT` /
  `EXT-HIGH` type space (a `.docm` cannot exist as both Standard
  and High Risk)

Each rejected line is collected into a per-row error list that
surfaces in the partial-success alert; the valid entries still
insert. The `(.ext)` prefix on the description is auto-prepended so
the catalogue stays self-describing regardless of how the operator
typed the row.

### Custom File Extensions DataTable

| Column | Source |
| --- | --- |
| (checkbox) | Selection for bulk Delete Selected |
| Extension | `.<files.file>` (the leading dot is displayed in the UI even though it isn't stored) |
| Description | `files.description` |
| Actions | Per-row Delete button (single-row confirm) |

The DataTable shows only rows with `system = 'NO'` and excludes
`type = 'CUSTOM-EXPRESSION'` rows (those belong to
[File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions), which uses the same `files`
table with a different `type` discriminator).

### System File Extensions DataTable (read-only)

The shipped catalogue — every row from `files` where `system = 'YES'`
and `type IN ('EXT', 'EXT-HIGH')`. These rows are filtered out of
every DELETE path on this page (`AND system = 'NO'` is part of every
DELETE query). The UI gives them no checkbox and no Delete button;
attempting a forged POST that targets a system row surfaces alert
`m = 11` and is rejected.

Standard rows get an "Info" badge, High Risk rows get a "Danger"
badge. The badge is cosmetic — Amavis treats both the same as
banned-extension candidates once they're wired into a File Rule.

## Foreign-key guard on delete

A custom extension cannot be deleted while it is referenced by any
[File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules). The single-row Delete handler runs:

```sql
SELECT COUNT(*) AS cnt FROM file_rule_components
WHERE file_id = :id
```

If `cnt > 0`, the delete is refused with alert `m = 10` and the
DataTable shows the offending rule name(s) ("This file extension is
used in the following File Rule(s): **HighRisk-block**"). The
operator's path is to open File Rules, remove the extension from
the rule, then come back here and delete it.

Bulk Delete applies the same guard per-id and accumulates partial
results — the success alert reports "N deleted, M skipped" with the
skipped rows' rule names attached so the operator knows exactly
what to unwire first.

## Save and apply flow

```
1. View page submits action="add_entries" | "delete" | "bulk_delete"
2. For each valid entry:
     a. Read the case-sensitive/insensitive allow + ban templates
        from /opt/hermes/scripts/file_allow_{sense|insense} and
        file_deny_{sense|insense}
     b. Substitute THE-EXTENSION placeholder with the (dot-stripped)
        extension name
     c. INSERT INTO files (file, description, type, system, allow, ban)
3. If at least one row was added or deleted:
     a. update_amavis_config_files.cfm:
          - Read /opt/hermes/conf_files/50-user.HERMES (template)
          - Substitute SERVER-NAME, SERVER-DOMAIN, sa-spam-subject-tag,
            final-virus-destiny, final-banned-destiny, final-spam-destiny,
            final-bad-header-destiny, enable-dkim-verification,
            enable-dkim-signing placeholders from spam_settings
          - Render every File Rule's components into an
            @banned_filename_re block (per-rule, in priority order,
            using the allow/ban regex stored on each files row)
          - Substitute HERMES-USERNAME / HERMES-PASSWORD from
            /opt/hermes/creds/ for the Amavis MySQL lookup
          - Back up /etc/amavis/conf.d/50-user -> 50-user.HERMES,
            move rendered file into place
     b. docker exec hermes_mail_filter /etc/init.d/amavis force-reload
        (30-second timeout)
4. session.m = 1 (add) | 2 (single delete) | 12 (bulk delete)
```

Amavis is reloaded with `force-reload` rather than restarted —
the daemon re-reads `50-user` without dropping connections, and
mail in flight is not interrupted. The full container restart that
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) and
[Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) trigger is not needed here
because no SpamAssassin state is being touched.

The reload step is wrapped in `cftry`/`cfcatch` with **comment "Log
but don't block — extensions were added"** — if the reload itself
fails, the DB rows are already in place and the next save (or
manual `force-reload`) will re-render. The page does not roll back
on reload failure.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` | Add Extensions completed (with `entries_added` / `entries_skipped` / `entry_errors` set on session for the per-row breakdown alert) |
| `m = 2` | Single Delete succeeded; Amavis reloaded |
| `m = 10` | Single Delete refused — the extension is wired into at least one File Rule (rule names surfaced in the alert) |
| `m = 11` | Attempt to delete a system row (`system = 'YES'`) — refused at the DB query |
| `m = 12` | Bulk Delete completed (with `bulk_deleted` / `bulk_skipped` / `bulk_errors` set on session) |
| `m = 30` | Add submitted with an empty textarea |

The per-row error list is HTML-rendered into the alert body so the
operator sees every rejection at once ("Must start with dot: foo",
"Invalid characters: .x@y", "Description required: .docm",
"Duplicate: .exe"). No row is silently dropped without an
explanation in the alert.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_file_extensions.cfm` | `hermes_commandbox` | The page (validation + bulk add + DataTables + Amavis reload) |
| `config/hermes/var/www/html/admin/2/inc/get_file_extensions.cfm` | `hermes_commandbox` | Loads custom + system rows for the two DataTables |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Renders `50-user` from template + File Rules (called on every change) |
| `config/hermes/opt/hermes/scripts/file_allow_insense` / `file_allow_sense` | `hermes_commandbox` | Allow-regex templates with `THE-EXTENSION` placeholder |
| `config/hermes/opt/hermes/scripts/file_deny_insense` / `file_deny_sense` | `hermes_commandbox` | Ban-regex templates with `THE-EXTENSION` placeholder |
| `config/hermes/opt/hermes/conf_files/50-user.HERMES` | `hermes_commandbox` (read) -> `hermes_mail_filter` (live `/etc/amavis/conf.d/50-user`) | Canonical Amavis template; receives the rendered `@banned_filename_re` blocks |
| `/etc/amavis/conf.d/50-user` | `hermes_mail_filter` | Live Amavis config; reloaded with `force-reload` on every save |
| `files` table, `type IN ('EXT','EXT-HIGH')` | `hermes_db_server` (`hermes` DB) | Source of truth for the catalogue (system + custom) |
| `file_rule_components` table | `hermes_db_server` (`hermes` DB) | Cross-reference checked by the delete guard |
| `hermes_mail_filter` container | — | Hosts Amavis; receives `force-reload` (not restart) on every change |

## Related

- [File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions) — sibling page for full
  regex patterns against any filename (not just extension); rows
  live in the same `files` table under `type = 'CUSTOM-EXPRESSION'`
- [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules) — bundles extensions and expressions
  into named, prioritised rulesets; the consumer of every row this
  page creates
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) — content-level SpamAssassin
  rules (header / body / regex) — the body / header equivalent of
  what File Extensions does for attachment names
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — defines
  `final_banned_destiny` (what Amavis does with a banned-extension
  match) and binds File Rules to recipients via SVF Policies
- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — ClamAV runs in the
  same Amavis pass; a virus verdict on the same attachment overrides
  the banned-extension result
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — sibling Amavis tuning
  page; both write into Amavis configuration but extension blocks
  are categorical (matched -> banned) where SA rules are weighted
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — none of this matters
  for connections that never make it past the SMTP-time perimeter
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) — a banned-extension
  rejection appears with Type `Banned` and the matched extension
  surfaced in the detail view
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — Amavis logs the
  matched regex as `Blocked BANNED (.exe,.bat,...)` on the
  `amavis[...]:` line

# File Rules

# File Rules

Admin path: **Content Checks > File Rules**
(`view_file_rules.cfm`,
`inc/get_file_rules.cfm`,
`inc/update_amavis_config_files.cfm`).

This page is the **bundling layer** that turns the raw catalogues
on [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) and
[File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions) into named, prioritised
rulesets that Amavis can actually enforce. A File Rule is a named
group of file-type components (extensions, file types, MIME types,
high-risk variants of each, and custom regex expressions) plus a
default action (**Ban** or **Allow**) that the operator binds to
recipient traffic via an SVF Policy under
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings). Without a File Rule
wrapping them, no row on the catalogue pages does anything to mail.

Hermes ships one system rule, **SYSTEM_DEFAULT**, populated with a
broad ban list (executables, scripts, Windows-class-IDs,
double-extension trap, archive formats, dangerous MIME types). It
is read-only — it can be copied, but not edited or deleted. Every
custom rule the operator creates lives alongside it in the same
DataTable, marked **No** in the System Rule column.

## Where File Rules sits

```
          File Extensions          File Expressions
                |                          |
                v                          v
     +-----------------------+    +-----------------------+
     | files table           |    | files table           |
     | type IN ('EXT',       |    | type =                |
     |          'EXT-HIGH',  |    |   'CUSTOM-EXPRESSION' |
     |          'FILE',      |    +----------+------------+
     |          'FILE-HIGH', |               |
     |          'MIME',      |               |
     |          'MIME-HIGH', |               |
     |          'OTHER')     |               |
     +-----------+-----------+               |
                 |                           |
                 +-------------+-------------+
                               |
                               v
                +-----------------------------+
                |  File Rules (this page)     |
                |                             |
                |  file_rule_components:      |
                |   rule_id, rule_name,       |
                |   file_id (FK -> files.id), |
                |   description, type ('ban'  |
                |   or 'allow'), priority,    |
                |   system (1=shipped,        |
                |           2=custom)         |
                |                             |
                |  file_rules (legacy index): |
                |   rule_id, rule_name,       |
                |   system                    |
                +--------------+--------------+
                               |
                               v
                +-----------------------------+
                |  Anti-Spam Settings         |
                |   SVF Policy row            |
                |   policy.banned_rulenames   |
                |   = '<rule_name>'           |
                +--------------+--------------+
                               |
                               v
                +-----------------------------+
                |  Amavis 50-user.HERMES      |
                |   per-rule @banned_         |
                |   filename_re block, with   |
                |   the rule's components in  |
                |   priority order            |
                +-----------------------------+
```

A File Rule that is created but **not** bound to an SVF Policy is
inert. The rule renders into Amavis's config (`50-user` carries
every defined rule), but no recipient policy points at it, so
nothing in `@banned_filename_re` fires for traffic.

## The two backing tables

| Table | Role |
| --- | --- |
| `file_rule_components` | The real source of truth. One row per (rule, file-type) pair. Carries `rule_id`, `rule_name`, `file_id` (FK -> `files.id`), `description`, `type` (`ban` or `allow`), `priority`, `system` (1 = shipped, 2 = custom) |
| `file_rules` | A legacy index table holding only `rule_id`, `rule_name`, `system`. Hermes ships a single row in it (`SYSTEM_DEFAULT`, `system=1`) — the page's CRUD operations write to `file_rule_components` directly and the Delete handler also clears `file_rules` for the matching `rule_id`. New rules are NOT inserted into `file_rules`; rule existence is determined entirely by `DISTINCT rule_id` on `file_rule_components` |

The `system` value is the system / custom discriminator and is the
guard for every modify path:

- `system = 1` -> shipped (SYSTEM_DEFAULT only). Read-only —
  attempting to edit or delete returns `m = 24`. The Copy button
  still works.
- `system = 2` -> operator-added. Editable and deletable, subject
  to the policy-binding guard.

The action column is named `type` (not `action`) on
`file_rule_components` and is per-component: a single rule can mix
**ban** and **allow** components, although the page's UI surfaces
"Default Action" as a single radio button and assigns the same
value to every component on save. Mixing `ban` and `allow` on the
same rule is possible only by direct SQL.

## The page

A page guide callout, a single DataTable listing every rule
(system and custom together), and three modals: Create Custom File
Rule (`Add`), Edit File Rule, and Copy File Rule.

### File Rules DataTable

| Column | Source |
| --- | --- |
| Rule Name | `file_rule_components.rule_name` (distinct) |
| Type | Rendered from the first component's `type` — `<span class="badge bg-danger">Ban</span>` or `<span class="badge bg-success">Allow</span>` |
| File Types | Every component's `description` as a list of `bg-secondary` badges, each suffixed with `(ban)` or `(allow)` |
| System Rule | `Yes` (info badge, `system=1`) or `No` (warning badge, `system=2`) |
| Actions | Copy (always present) + Edit + Delete (only when `system=2`) |

Default sort is **System Rule asc, Rule Name asc**, so the shipped
rule sinks below the custom ones once any exist (custom = `system=2`
sorts above shipped = `system=1`? No — `2` > `1`, but the column
order asc is intentional: shipped first, then custom alphabetised).
The DataTable carries `stateSave: true`, so the operator's sort /
search / page-size choices persist across page loads.

### Create Custom File Rule modal (Add)

| Field | Stored as | Notes |
| --- | --- | --- |
| Rule Name | `file_rule_components.rule_name` | Regex-validated against `[^_a-zA-Z0-9-]` — letters, numbers, dashes, underscores only. No spaces, no punctuation. Max length 50. Duplicates across both system and custom rules are rejected (`m = 22`) |
| Default Action | `file_rule_components.type` on every inserted component | Radio: `ban` (default) or `allow` |
| File Type checkboxes | One INSERT per checked box into `file_rule_components` | Eight grouped cards: High Risk Extensions, High Risk File Types, High Risk MIME Types, File Extensions, File Types, MIME Types, Other Types, Custom Expressions. Each card has a "select-all" master checkbox and a scrollable list of every `files` row of that `type`. At least one file type must be selected (`m = 23`) |

The handler computes the next `rule_id` as `MAX(rule_id) + 1`
(scoped across `file_rule_components`, not `file_rules`), assigns
`priority` sequentially as components are inserted (1, 2, 3, …
in submission order), and marks each row `system = 2`.

### Edit File Rule modal

Opens preloaded with the current rule's name, default action, and
checkbox selections — the JavaScript reads a `ruleComponents` map
written into the page at render time and ticks the matching
checkboxes across all eight category cards.

Save is destructive-then-rebuild: the handler **DELETEs** every
`file_rule_components` row for the rule_id, then re-INSERTs from
the new form selection. The same name / action / file-types
validation as Add applies, plus:

- System rules (`system=1`) are refused with `m = 24`. The button is
  not even rendered for system rows, but the action handler still
  guards against forged POSTs.
- If the rule name changed, the handler also UPDATEs
  `policy.banned_rulenames` so any SVF Policy binding survives the
  rename. The cascade is name-keyed, not id-keyed — the policy
  table stores the name string, not the rule_id.

### Copy File Rule modal

The only path to derive a new rule from SYSTEM_DEFAULT. Asks for a
new name (same `[a-zA-Z0-9_-]+` validation, same duplicate check,
same 50-char max), then INSERTs a fresh set of
`file_rule_components` rows under a new `rule_id` with all the
source rule's `file_id`, `description`, `type`, and `priority`
values preserved. The copy is always `system = 2` regardless of
the source's flag — so a copy of SYSTEM_DEFAULT becomes a fully
editable custom rule.

The default new-name in the modal is `<source>_copy`, so the
operator can hit Copy on SYSTEM_DEFAULT and immediately get
`SYSTEM_DEFAULT_copy` ready to edit.

## Policy-binding guard on delete

A custom rule cannot be deleted while any SVF Policy points at it.
The Delete handler runs:

```sql
SELECT policy_name FROM policy
WHERE banned_rulenames = '<rule_name>'
```

If any row comes back, the delete is refused with alert `m = 25`
and the policy name(s) are surfaced in the alert ("You cannot
delete a file rule that is assigned to SVF Policy:
**Default,Inbound-Strict**. Remove the assignment first under
Content Checks > SVF Policies.").

This is the symmetric counterpart to the FK guard on
[File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) and
[File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions) — those pages refuse to
delete a row that is bundled into a rule; this page refuses to
delete a rule that is bundled into a policy.

## Save and apply flow

```
1. View page submits action="add_rule" | "edit_rule" | "delete_rule"
                          | "copy_rule"
2. Validate name (non-empty, regex-clean, non-duplicate, non-system
   on edit/delete), validate file_ids (non-empty)
3. For Add / Edit / Copy:
     a. Determine rule_id (next MAX+1 for Add/Copy, form value for Edit)
     b. (Edit only) UPDATE policy.banned_rulenames if rule_name changed
     c. (Edit only) DELETE existing file_rule_components for rule_id
     d. INSERT one file_rule_components row per checked file_id, with
        priority assigned sequentially (1..N) and system='2'
   For Delete:
     a. DELETE FROM file_rules WHERE rule_id = :id
     b. DELETE FROM file_rule_components WHERE rule_id = :id
4. update_amavis_config_files.cfm:
     - Read /opt/hermes/conf_files/50-user.HERMES (template)
     - Substitute SERVER/destiny/DKIM/MySQL-credential placeholders
     - Loop every DISTINCT rule_id in file_rule_components
       and emit a per-rule @banned_filename_re block in
       priority order, using each component's allow or ban
       regex from files.allow / files.ban
     - Back up /etc/amavis/conf.d/50-user -> 50-user.HERMES,
       move rendered file into place
5. docker exec hermes_mail_filter /etc/init.d/amavis force-reload
   (60-second timeout - longer than the catalogue pages because
    every rule re-renders)
6. session.m = 1 (add) | 2 (edit) | 3 (delete) | 4 (copy)
              | 10 (reload error) | 20-25 (validation refusals)
```

Amavis is reloaded with `force-reload` rather than restarted. If
the reload itself fails, the rule rows are already committed —
alert `m = 10` ("Configuration Error") fires but the DB is not
rolled back. The next successful save (or a manual `force-reload`)
will re-render.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` | Rule created. The alert also nudges the operator to assign the rule to a policy under SVF Policies — without that binding the rule is inert |
| `m = 2` | Rule updated; Amavis reloaded |
| `m = 3` | Rule deleted; Amavis reloaded |
| `m = 4` | Rule copied. Same nudge as `m = 1` — the copy is inert until bound to an SVF Policy |
| `m = 10` | Amavis reload error — the DB write succeeded but `force-reload` returned non-zero. Open Anti-Spam Settings and save once to re-trigger the render + reload, or restart `hermes_mail_filter` manually |
| `m = 20` | Rule name field empty |
| `m = 21` | Rule name contains characters outside `[a-zA-Z0-9_-]` (spaces, dots, slashes, etc.) |
| `m = 22` | Duplicate rule name — checked against both system and custom rules |
| `m = 23` | No file types selected — at least one checkbox across the eight category cards is required |
| `m = 24` | Attempted to edit or delete a system rule (`system=1`) — refused. The operator's path is to Copy first, then edit the copy |
| `m = 25` | Delete refused — the rule is bound to one or more SVF Policies (policy names surfaced in the alert) |

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_file_rules.cfm` | `hermes_commandbox` | The page (CRUD + Copy + DataTable + three modals + Amavis reload) |
| `config/hermes/var/www/html/admin/2/inc/get_file_rules.cfm` | `hermes_commandbox` | Loads the rule list + every `files` row grouped by `type` for the modal cards (`get_files_ext_high`, `get_files_file_high`, …, `get_files_custom_expr`) |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Renders `50-user` from template + every File Rule's components |
| `config/hermes/opt/hermes/conf_files/50-user.HERMES` | `hermes_commandbox` (read) -> `hermes_mail_filter` (live `/etc/amavis/conf.d/50-user`) | Canonical Amavis template; receives the per-rule `@banned_filename_re` blocks |
| `/etc/amavis/conf.d/50-user` | `hermes_mail_filter` | Live Amavis config; reloaded with `force-reload` on every save |
| `file_rule_components` table | `hermes_db_server` (`hermes` DB) | The real rule store — one row per (rule, file-type) pair |
| `file_rules` table | `hermes_db_server` (`hermes` DB) | Legacy index — only SYSTEM_DEFAULT lives here; custom rules are NOT mirrored. Cleared on delete for the matching `rule_id` |
| `files` table | `hermes_db_server` (`hermes` DB) | Source of the file-type checkboxes; FK target of `file_rule_components.file_id` |
| `policy` table, `banned_rulenames` column | `hermes_db_server` (`hermes` DB) | Where SVF Policies record their rule binding; renamed in step with rule renames, checked by the delete guard |
| `hermes_mail_filter` container | — | Hosts Amavis; receives `force-reload` (not restart) on every change |

## Operational consequences

- **A rule with no policy binding is inert.** Creating a rule does
  not block anything by itself — Amavis renders the rule into
  `50-user` but no recipient policy points at it. The "Please
  assign the rule to a policy under Content Checks > SVF Policies"
  nudge in `m = 1` and `m = 4` is the operational reminder. Until
  the binding is in place the rule exists for the operator's
  benefit only.
- **Edit is destructive-then-rebuild.** Saving an edit DELETEs and
  re-INSERTs every component for the rule. Priorities are
  reassigned 1..N in checkbox-submission order, which is the page
  render order, not the order the operator originally added them.
  An edit that only adds one new file type will reshuffle the
  priority numbers of every existing component on that rule.
  Functionally invisible (`@banned_filename_re` evaluation is
  any-match), but visible if anyone reads the table directly.
- **Renames cascade through `policy.banned_rulenames`.** The page
  joins on name, not id — when the rule name changes, the policy
  row is updated in the same transaction. If a policy binding
  exists, the operator does not need to re-open the SVF Policy
  page after a rename.
- **Copy is the only path off SYSTEM_DEFAULT.** The shipped rule is
  hard-locked (`m = 24` on any edit / delete attempt). Operators
  who want to tighten the defaults (add `.iso`, remove `.rtf`,
  swap MIME types) make a copy, edit the copy, and bind the copy
  to the policy in place of SYSTEM_DEFAULT.
- **The Type badge shows the first component's action only.** A
  hand-mixed rule (`ban` and `allow` components on the same rule)
  will display whichever was inserted at priority 1. The DataTable
  does not flag mixed rules — the File Types column shows each
  component's `(ban)` / `(allow)` suffix, which is the only place
  the mix is surfaced. The UI itself only writes uniform rules.
- **Amavis reload timeout is 60s here, vs 30s on the catalogues.**
  Re-rendering every rule's `@banned_filename_re` block can take
  longer than re-rendering a single allow/ban regex for an added
  extension. If the reload times out, the page shows `m = 10` and
  the rule write still succeeded.

## Related

- [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) — the plain-extension half
  of the file-type catalogue; rows here become checkboxes in this
  page's modals under "High Risk Extensions" and "File Extensions"
- [File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions) — the regex half of the
  file-type catalogue; rows there become checkboxes under "Custom
  Expressions"
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) — the body / header equivalent
  of File Rules; binds SpamAssassin rules to scope rather than
  Amavis filename patterns
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — where File Rules
  are bound to recipient traffic via SVF Policies
  (`policy.banned_rulenames`) and where `final_banned_destiny`
  (the action on a match) is set
- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — ClamAV runs in the
  same Amavis pass; a virus verdict on the same attachment
  overrides the banned-rule result
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — sibling Amavis tuning
  page; both write into the same `50-user` regeneration chain but
  rule matches are categorical where SA score overrides are
  weighted
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — note that banned-rule
  rejections are a body-side filter result, not an authentication
  result — they fire after ARC chain evaluation
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) — a banned-rule rejection
  appears with Type `Banned` and the matched rule + component
  surfaced in the detail view
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — Amavis logs the
  rule name and matched component as
  `Blocked BANNED ('<rule_name>' matched)` on the `amavis[...]:`
  line

# Global Sender Rules

# Global Sender Rules

Admin path: **Content Checks > Global Sender Rules**
(`view_global_sender_block_allow.cfm`,
`inc/get_global_sender_block_allow.cfm`,
`inc/global_sender_add_entries.cfm`,
`inc/global_sender_edit_entry.cfm`,
`inc/global_sender_delete_entry.cfm`,
`inc/global_sender_write_and_reload.cfm`).

This page manages **system-wide envelope-sender rules** that apply
regardless of recipient. Every entry on this page is a single sender
pattern (full address, exact domain, or domain + subdomains) paired
with an action — **Block** or **Allow**. The rules are evaluated by
Postfix at `MAIL FROM` time, before the message body is read; an
Allow match additionally bypasses Amavis content filtering for that
sender.

Global Sender Rules are the system-wide counterpart to
[Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules). A Global rule
matches **all recipients** in the system; a Sender/Recipient rule
requires both a sender and a recipient to match. A Global entry
takes precedence over any Sender/Recipient entry for the same sender.

## Where Global Sender Rules sit in the flow

```
+-------------------+
| Remote SMTP peer  |
+---------+---------+
          |
          v
+-----------------------------------------------+
|  postscreen :25 (perimeter / RBL scoring)     |
+---------+-------------------------------------+
          |
          v
+-----------------------------------------------+
|  smtpd :25                                    |
|   smtpd_sender_restrictions =                 |
|     check_sender_access                       |
|       hash:/etc/postfix/amavis_senderbypass   |
|                                               |
|   match -> REJECT (block)                     |
|   match -> FILTER amavis:[127.0.0.1]:10030    |
|             (allow -> route past content      |
|              filtering)                       |
|   no match -> fall through to recipient rules |
+---------+-------------------------------------+
          |
          v
+-----------------------------------------------+
|  Amavis (white.lst / black.lst consulted      |
|  again at content-filter tier)                |
+-----------------------------------------------+
```

The same rule set is written to **two places** on each save: the
Postfix `check_sender_access` table (`/etc/postfix/amavis_senderbypass`,
`postmap`ed into a Berkeley DB) and the Amavis whitelist/blacklist
files (`/etc/amavis/white.lst`, `/etc/amavis/black.lst`). Block
entries surface at the Postfix tier — the connection is rejected at
`MAIL FROM` and Amavis is never invoked. Allow entries route past
Amavis content scoring via the `FILTER` transport hint, and are also
written to Amavis's own whitelist as a safety net for any mail path
that does reach Amavis (locally-injected, alias-rewritten, etc.).

## Pattern formats

The page accepts three pattern formats. The save handler validates
each line and auto-prepends `@` to bare domains so the stored row is
always in one of the three canonical forms:

| Format | Example | Matches |
| --- | --- | --- |
| Full email | `user@example.com` | A single envelope sender |
| Exact domain (`@`) | `@example.com` | Every sender on `example.com` only — subdomains do not match |
| Domain + subdomains (`.`) | `.example.com` | `example.com` **and** every subdomain (`sub.example.com`, `mail.sub.example.com`, ...) |

Bare-domain input (`example.com`) is treated as a typo for
`@example.com` and rewritten on insert. Email-syntax validation runs
on the host portion of every pattern; entries that fail validation are
collected into a "Invalid Entries" alert and the rest of the batch is
still processed.

## The page

A single warning callout, a multi-line Add form, and one DataTable.

### Add Sender Entries

A textarea (one entry per line) plus a Block/Allow radio. The form
processes the entire batch in one round-trip:

- Each line is trimmed, classified (`@domain`, `.domain`, full email,
  or bare domain), and validated.
- Valid lines are checked against `amavis_sender_bypass` for an
  exact-string duplicate; duplicates are collected separately.
- Surviving lines are inserted with `type = block` or `type = allow`.
  For Allow entries, the row's `transport` column is set to
  `FILTER amavis:[127.0.0.1]:10030` — this is the Postfix transport
  hint that bypasses content filtering when a sender match fires.
- If any entries were added, the page calls the write-and-reload
  include before redirecting.

The redirected page surfaces three separate inline alerts (green
success, red invalid, red duplicate) so a mixed batch reports clearly
on what happened to every line.

A small inline JS check flips a warning banner under the textarea when
the operator types a domain (no `@`) — the consequence of allow-listing
or block-listing an entire domain is significant enough to warrant the
extra nudge.

### Global Sender Entries (DataTable)

Searchable, sortable, paginated, with bulk-delete checkboxes and
per-row Edit / Delete buttons.

| Column | Source |
| --- | --- |
| Sender | `amavis_sender_bypass.sender` |
| Format | Derived from the leading character — `@` -> Domain badge, `.` -> Domain + Subdomains badge, otherwise Email badge |
| Action | `amavis_sender_bypass.type` -> Allow (green) or Block (red) |
| Actions | Edit (modal), Delete (confirm) |

Bulk delete posts a comma-separated list of row IDs from the wrapping
form. Single Edit and Delete use separate hidden forms so they don't
collide with the bulk submit handler.

## Save flow

Every Add, Edit, and Delete runs the full regeneration path inline:

```
1. Validate input + INSERT / UPDATE / DELETE on amavis_sender_bypass
2. cfinclude global_sender_write_and_reload.cfm:
     a. SELECT all type='allow' rows (with transport column)
     b. SELECT all type='block' rows
     c. Write /etc/postfix/amavis_senderbypass    (allow rows + transport)
     d. Write /etc/amavis/white.lst               (allow rows, one per line)
     e. Write /etc/amavis/black.lst               (block rows, one per line)
     f. docker exec hermes_postfix_dkim postmap /etc/postfix/amavis_senderbypass
     g. docker exec hermes_postfix_dkim chown root:root <file + .db>
     h. docker exec hermes_postfix_dkim postfix reload
     i. docker exec hermes_mail_filter /etc/init.d/amavis force-reload
3. session.m = 1 / 2 / 5 (Added / Deleted / Updated)
   On failure -> session.m = 4 ("Apply Failed")
```

The Postfix `postmap` step is what makes Block entries actually take
effect — `check_sender_access` reads the hashed `.db` file, not the
plain-text source. Skipping the postmap (e.g. by editing the source
file out-of-band) is a common cause of "I added a block but mail is
still getting through".

> **Why both Postfix and Amavis get the list.** The Postfix tier
> handles the common case — Block rejects before DATA, Allow routes
> past Amavis via the `FILTER` transport. The Amavis-side
> `white.lst` / `black.lst` files are a defence in depth: any mail
> path that **does** reach Amavis (locally-injected mail, mail that
> was alias-rewritten after the sender check, mail from
> `permit_mynetworks` sources that skipped sender restrictions)
> still gets the same allow/block treatment at the content-filter
> tier. The two layers are kept in sync by the single save flow.

## The `amavis_sender_bypass` table

| Column | Purpose |
| --- | --- |
| `id` | Auto-increment primary key |
| `sender` | The pattern (`user@example.com`, `@example.com`, or `.example.com`) |
| `transport` | For Allow rows: `FILTER amavis:[127.0.0.1]:10030`. Empty for Block rows |
| `action` | Always `NONE` for active rows; reserved for future scheduled-action use |
| `type` | `allow` or `block` |
| `applied` | `1` once the row is live; future use for deferred apply |

The duplicate check on insert is an exact string match on `sender`,
so `@example.com` and `.example.com` are treated as separate (and
both can legitimately coexist — they match different sets of
addresses).

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Empty textarea | `session.m = 30`, redirect, no DB write |
| Invalid email/domain on a line | Line skipped, accumulated into the Invalid Entries alert; other valid lines still processed |
| Exact-string duplicate on a line | Line skipped, accumulated into the Duplicate Entries alert; other valid lines still processed |
| `cffile` / `postmap` / reload failure | `session.m = 4` ("Apply Failed"); inserted rows remain in the DB and will be re-applied on the next successful save |
| Postfix container down | Reload fails -> `session.m = 4`; mail flow continues with the previously-loaded Berkeley DB until the container is back |

The save is **not** transactional across the DB + file-write +
reload steps. If the DB insert succeeds but the postmap or reload
fails, the next Add/Edit/Delete will regenerate from the full DB
state and reapply.

## Operational guidance

- **Allow entries bypass every content filter** — Spam, Virus,
  Banned File, custom Amavis rules — for the matched sender, for
  every recipient in the system. The shipped warning callout on the
  page is not boilerplate; use Allow sparingly and prefer
  [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) for narrower
  exceptions.
- **Block is cheaper than content filtering.** A Block entry rejects
  the SMTP transaction at `MAIL FROM`. The body is never read, no
  spam score is computed, no virus scan runs. For known-phishing
  sender domains this is the right tier to act at.
- **Domain + subdomain (`.example.com`) carries a wide blast radius**
  — a Block entry on `.example.com` will reject mail from
  `support@example.com`, `noreply@news.example.com`, and every other
  subdomain. The textarea's live warning banner exists for exactly
  this case.
- **Order of precedence.** Global Sender Rules beat
  Sender/Recipient Rules. A Block on `@example.com` here will
  reject mail from that sender even if a per-recipient Allow exists
  on the Sender/Recipient Rules page for the same sender.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_global_sender_block_allow.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/get_global_sender_block_allow.cfm` | `hermes_commandbox` | Loads the active row set for the DataTable |
| `config/hermes/var/www/html/admin/2/inc/global_sender_add_entries.cfm` | `hermes_commandbox` | Batch validation + INSERT loop |
| `config/hermes/var/www/html/admin/2/inc/global_sender_edit_entry.cfm` | `hermes_commandbox` | Single-row UPDATE + regen |
| `config/hermes/var/www/html/admin/2/inc/global_sender_delete_entry.cfm` | `hermes_commandbox` | Single or bulk DELETE + regen |
| `config/hermes/var/www/html/admin/2/inc/global_sender_write_and_reload.cfm` | `hermes_commandbox` | Writes the three files, runs postmap, reloads Postfix and Amavis |
| `amavis_sender_bypass` table | `hermes_db_server` (`hermes` DB) | Source of truth |
| `/etc/postfix/amavis_senderbypass` (+ `.db`) | `hermes_postfix_dkim` | Postfix `check_sender_access` lookup |
| `/etc/amavis/white.lst`, `/etc/amavis/black.lst` | `hermes_mail_filter` | Amavis sender whitelist / blacklist |
| `hermes_postfix_dkim` container | — | Runs `postmap` + `postfix reload` |
| `hermes_mail_filter` container | — | Runs `amavis force-reload` |

## Related

- [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) — per-pair
  variant; narrower scope, lower precedence
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — the upstream
  `smtpd_*_restrictions` toggles a connection is evaluated against
  before sender-access lookup
- [Network Block/Allow](https://docs.deeztek.com/books/administrator-guide/page/network-blockallow) — the IP-level
  `postscreen_access.cidr` table consulted **before** any sender
  evaluation; an entry there can short-circuit a peer regardless of
  envelope sender
- [RBL Configuration](https://docs.deeztek.com/books/administrator-guide/page/rbl-configuration) — third-party DNSBL
  scoring at the postscreen tier; runs before sender access lookup
- [BCC Maps](https://docs.deeztek.com/books/administrator-guide/page/bcc-maps) — sibling envelope-level rule table; the
  other half of the envelope-rule pair
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — the content-filter
  tier that Allow entries route around
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — `mail.log` is where
  block rejections and Amavis bypass decisions surface for audit
- [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) — visible flow-of-mail
  diagnostics if a rule change has an unexpected effect

# Malware Feeds

# Malware Feeds

Admin path: **Content Checks > Malware Feeds**
(`view_malware_feeds.cfm`, `inc/get_malware_feeds_settings.cfm`,
`inc/malware_feeds_save_global.cfm`, `inc/malware_feeds_add_feed.cfm`,
`inc/malware_feeds_edit_feed.cfm`, `inc/malware_feeds_delete_feed.cfm`,
`inc/malware_feeds_toggle_feed.cfm`, `inc/malware_feeds_save_urls.cfm`,
`inc/generate_malware_feeds_configuration.cfm`).

This page manages the third-party ClamAV signature feeds that supplement
the stock `freshclam` definitions on [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings).
The feed manager is [Fangfrisch](https://github.com/rseichter/fangfrisch),
a small Python tool that handles per-feed authentication, cadence control,
integrity verification, and post-download deployment. Hermes ships ten
built-in feed definitions (free and commercial), exposes a custom-feed
form for additional sources, and a per-feed URL editor for signature
file selection. Refresh runs as an [Ofelia](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks)
job inside `hermes_mail_filter`.

This page replaced an earlier `view_antivirus_signature_feeds.cfm` page
(orphan cleanup tracked as issue #257); any sidebar bookmark or
external link pointing at the old page should be updated.

## How feeds reach ClamAV

```
                  +-------------------------------------------+
                  |  hermes-fangfrisch-refresh (Ofelia job)   |
                  |    inside hermes_mail_filter              |
                  |    schedule: @every <refresh_interval>    |
                  +----------------+--------------------------+
                                   |
                                   v
                  +-------------------------------------------+
                  |  /usr/bin/fangfrisch refresh              |
                  |    reads /etc/fangfrisch/fangfrisch.conf  |
                  |    iterates enabled feeds                 |
                  |    skips feeds whose own interval has not |
                  |    elapsed                                |
                  +----------------+--------------------------+
                                   |
                                   v
                  +-------------------------------------------+
                  |  Per-feed download                        |
                  |    auth via API key / customer_id /       |
                  |    serial_key when required               |
                  |    integrity check (sha256, md5, off)     |
                  |    -> /var/lib/fangfrisch/signatures/     |
                  +----------------+--------------------------+
                                   |
                                   v
                  +-------------------------------------------+
                  |  on_update_exec=/usr/local/bin/setup-     |
                  |  clamav-sigs (post-update hook)           |
                  |    validates each file with `clamscan`    |
                  |    copies valid files to /var/lib/clamav/ |
                  |    signals clamd to reload                |
                  +-------------------------------------------+
```

The page emits `/etc/fangfrisch/fangfrisch.conf` (an INI file) on every
save. Fangfrisch itself is invoked on a fixed Ofelia schedule; the
schedule is regenerated from `ofelia_jobs.schedule` and reflects the
Global Settings > Refresh Interval picker.

## Container and tool placement

| Component | Detail |
| --- | --- |
| Container | `hermes_mail_filter` (IPv4 `.105`, same container as ClamAV, Amavis, SpamAssassin) |
| Feed manager | `fangfrisch` (Python, third-party ClamAV signature aggregator) |
| INI config | `/etc/fangfrisch/fangfrisch.conf` (bind-mounted, owned `root:clamav`, mode `640`) |
| State DB | `sqlite:////var/lib/fangfrisch/db.sqlite` (per-feed last-refresh, integrity hashes) |
| Download dir | `/var/lib/fangfrisch/signatures/` (raw downloaded files) |
| Deploy dir | `/var/lib/clamav/` (validated files, ClamAV signature store) |
| Post-update hook | `/usr/local/bin/setup-clamav-sigs` (validates with `clamscan`, copies to deploy dir, signals reload) |
| Scheduler | Ofelia job `hermes-fangfrisch-refresh` row in `ofelia_jobs` |
| Default cadence | `@every 10m` (Fangfrisch then honors per-feed `interval =` to decide what to actually re-fetch) |

## Global Settings card

Four controls write to `parameters2 WHERE module = 'malware_feeds'`. The
first three substitute into the `[DEFAULT]` section of `fangfrisch.conf`
on every save; the fourth updates the Ofelia row that schedules the
refresh job.

| Field | Storage | INI / scheduler effect | Notes |
| --- | --- | --- | --- |
| Log Level | `parameters2.value2` (`log_level`) | `[DEFAULT] log_level = ...` | `debug,info,warning,error,fatal`; logs go to `docker logs hermes_mail_filter` |
| Default Max Size | `parameters2.value2` (`max_size`) | `[DEFAULT] max_size = ...` | Per-file cap. Regex anchors a number followed by `KB`, `MB`, `M`, or `B` (e.g. `5MB`, `10M`, `250KB`). Inherited by feeds that don't set their own |
| Update Timeout (sec) | `parameters2.value2` (`on_update_timeout`) | `[DEFAULT] on_update_timeout = ...` | Bounded 1-300. Caps how long `setup-clamav-sigs` is allowed to run |
| Refresh Interval | `parameters2.value2` (`refresh_interval`) AND `ofelia_jobs.schedule` | Ofelia `@every <interval>` | Allowed values: `5m,10m,15m,30m,1h,2h,4h`. Fangfrisch's own per-feed `interval =` still gates whether each feed actually re-downloads on a given run |

The post-update hook path is hard-coded to
`/usr/local/bin/setup-clamav-sigs` and shown read-only beneath the form
as `[DEFAULT] on_update_exec`. The hook lives inside the
`hermes_mail_filter` image and validates each downloaded file with
`clamscan` before copying it to `/var/lib/clamav/`; a file that fails
validation is left in the Fangfrisch download dir and not deployed.

## Malware Feeds card

Rows from `malware_feeds_config` populate a DataTable; per-row form
posts toggle, edit, manage URLs, and (custom feeds only) delete. The
schema:

| Column | Role |
| --- | --- |
| `id` | Surrogate key |
| `section_name` | INI section header, `[<section_name>]`. Lowercase alphanumeric + underscore (`^[a-z0-9_]+$`). Cannot change after creation. Unique. |
| `display_name` | Card label, free text |
| `enabled` | `tinyint(3)`, 0/1. Sliders here flip this. `enabled = yes/no` line in INI |
| `is_builtin` | `tinyint(3)`, 0/1. Built-in rows cannot be deleted (the Delete action button is suppressed in the UI and the delete handler refuses) |
| `prefix` | `${prefix}` interpolation source for URL entries. Optional |
| `interval_value` | Per-feed cadence (e.g. `1h`, `4h`, `1d`); blank = inherit `@every <refresh_interval>` |
| `max_size` | Per-feed cap; blank = inherit Global Default Max Size |
| `integrity_check` | `sha256`, `md5`, `disabled`, or NULL (default `sha256`) |
| `api_key_1_name` / `api_key_1_value` | Optional auth key (e.g. `customer_id`, `receipt`). Value stored AES-encrypted with key `/opt/hermes/keys/hermes.key` |
| `api_key_2_name` / `api_key_2_value` | Second auth key (e.g. MalwarePatrol's `product`). Same encryption |
| `description` | Free text |
| `sort_order` | Display order; custom-add inserts at 100 |

### Built-in feed catalog (factory rows)

| Feed | Type | Default state | Auth | Notes |
| --- | --- | --- | --- | --- |
| SaneSecurity | Free | Enabled | None | Broad zero-day coverage; mirror `https://ftp.swin.edu.au/sanesecurity/` |
| URLhaus | Free | Enabled | None | Malicious URL signatures from abuse.ch |
| MalwarePatrol | Commercial | Enabled | `receipt`, `product` IDs | Configure both keys via Edit; subscription IDs are documented in the in-card help |
| MalwareExpert | Commercial | Enabled | `serial_key` | URL template embeds the serial in the path |
| SecuriteInfo | Commercial | Enabled | `customer_id` | Free tier available; paid tier unlocks extra URLs |
| TwinWave | Free | Enabled | None | Public GitHub-hosted signatures |
| ClamPunch | Free | Enabled | None | Heuristic family signatures |
| RFXN | Free | Enabled | None | R-fx Networks Linux Malware Detect signatures |
| InterServer | Free | Enabled | None | Hash + URL signatures |
| Ditekshen | Free | Enabled | None | YARA/ClamAV detection rules |

A commercial feed is "enabled" only in the sense that its row is
marked `enabled = 1`; without API keys the feed is configured but
will not actually fetch (the in-card help describes the per-vendor
key requirements and the table icon shows a yellow warning triangle
on commercial rows missing keys).

### Add Custom Feed modal

Free-form add for any feed source not in the built-in catalog.
Validation:

| Field | Rule |
| --- | --- |
| Section Name | `^[a-z0-9_]+$`, required, must not already exist |
| Display Name | Required |
| URL Prefix | Optional, becomes the `prefix =` line and the substitution source for `${prefix}` in URL entries |
| Update Interval | Optional, number followed by `m` (minutes), `h` (hours), or `d` (days). Examples: `10m`, `1h`, `1d` |
| Max Size | Optional, number followed by `KB`, `MB`, `M`, or `B`. Examples: `5MB`, `250KB` |
| Integrity Check | Dropdown: default (sha256), sha256, md5, disabled |
| Description | Optional free text |

A new custom feed is inserted with `enabled = 0` and `is_builtin = 0`;
the admin then opens the URL manager to register at least one URL
before turning the row on.

## Manage URLs modal (per-feed)

Rows from `malware_feed_urls` keyed by `feed_id`. Each URL becomes a
line in the corresponding `[<section_name>]` block of `fangfrisch.conf`:

```
url_<url_key> = <url_value>
filename_<url_key> = <filename_override>   ## only when filename_override set
```

When a URL is toggled off, the `url_` prefix is replaced with `!url_` to
inactivate the line without losing the configuration. `${prefix}` in the
URL value is expanded against the feed's `prefix =` at fetch time.

| Field | Rule |
| --- | --- |
| Name (`url_key`) | `^[a-z0-9_.]+$`, must be unique within the feed (`UNIQUE KEY uq_feed_url(feed_id, url_key)`) |
| Download URL (`url_value`) | Full URL, or `${prefix}<path>` shorthand when the feed has a prefix |
| Save As (`filename_override`) | Optional. Renames the downloaded file locally; useful when the source filename is too generic |
| Toggle | Per-URL on/off. Disabled URLs are skipped without being deleted |

Built-in feeds may have URLs that Fangfrisch maintains internally — the
in-modal note explains that an empty URL table for a built-in feed
means it is using its packaged defaults, not that it is broken.

## Save flow

```
1. View page submits action= save_global | add_feed | edit_feed
                          | delete_feed | toggle_feed | url_action
2. malware_feeds_*.cfm validates and UPDATEs/INSERTs/DELETEs the row(s)
3. generate_malware_feeds_configuration.cfm runs on EVERY action:
     a. SELECT module='malware_feeds' rows from parameters2 -> globalSettings
     b. SELECT malware_feeds_config -> all feed rows
     c. SELECT malware_feed_urls -> all URLs grouped by feed_id
     d. Build [DEFAULT] section + one [<section_name>] block per feed
     e. Decrypt api_key_*_value with AES + /opt/hermes/keys/hermes.key
        (key emitted as `<api_key_*_name> = <plain>`)
     f. Write temp file -> /opt/hermes/tmp/<trans>_fangfrisch.conf
     g. dos2unix (tolerated if missing)
     h. cffile write -> /etc/fangfrisch/fangfrisch.conf
     i. docker exec hermes_mail_filter chown root:clamav + chmod 640
        (tolerated if container is down)
     j. cfinclude ofelia_generate_config.cfm
        (rewrites /etc/ofelia/config.ini if any schedule changed)
4. cflocation back to view_malware_feeds.cfm
5. session.m + session.alerttype + session.alertmsg drives the alert banner
```

Every UI action -- including a single-row enable/disable toggle -- runs
the full INI regen, ownership fix, and Ofelia config regen. There is no
incremental write path; the INI is always rendered from the current
database state. This means manual edits to `/etc/fangfrisch/fangfrisch.conf`
are lost on the next save -- store all configuration in the database.

## API key encryption

The `api_key_1_value` and `api_key_2_value` columns store AES-Base64
ciphertext using the key in `/opt/hermes/keys/hermes.key`. The edit
modal shows a masked preview (20 asterisks + last 4 chars of the
plaintext) for visual confirmation without exposing the full key.
Decryption happens only in `generate_malware_feeds_configuration.cfm`
at the moment the INI is rendered; a decryption failure replaces the
key line with a commented `## <name> = [decryption error]` marker
rather than aborting the save.

The encryption key file is mounted into `hermes_commandbox` only;
neither `hermes_mail_filter` nor any other service reads it. This
keeps the plaintext key out of the running config on disk for as
short a window as possible (write -> chmod 640 root:clamav -> next
Fangfrisch run reads -> file remains until next save replaces it).

## Manual refresh

The Ofelia job runs on schedule, but the same command can be invoked
manually from a host shell:

```
docker exec hermes_mail_filter fangfrisch --conf /etc/fangfrisch/fangfrisch.conf refresh
```

Fangfrisch is conservative — it will still skip feeds whose own per-feed
`interval =` window has not elapsed. To force a re-download of a single
feed regardless of cadence, the Fangfrisch state DB can be cleared for
that feed:

```
docker exec hermes_mail_filter sqlite3 /var/lib/fangfrisch/db.sqlite \
  "DELETE FROM refreshlog WHERE source = '<section_name>';"
```

Then re-run the refresh. The post-update hook re-validates with
`clamscan` and deploys to `/var/lib/clamav/`. To inspect downloaded
files:

```
docker exec hermes_mail_filter ls -la /var/lib/fangfrisch/signatures/
```

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Global save with non-allowlisted log_level / max_size / timeout / interval | session.m=`malware_feeds_error`, alerttype=danger, alertmsg explains; no DB write |
| Add Custom Feed with duplicate section name | session.m=error, alertmsg names the conflict; INSERT not attempted |
| Toggle/edit on non-existent `feed_id` | session.m=error "Feed not found"; no UPDATE |
| Delete attempted on a built-in feed | UI suppresses the button; handler refuses the row |
| API key decryption error at INI regen | INI line replaced with `## <name> = [decryption error]`; save still completes; Fangfrisch will treat the auth as missing on the next run |
| Container down during `chown`/`chmod` | `cftry` swallows the exec failure; INI is still written to the bind mount and the chown is applied next save when the container is back up |
| dos2unix binary missing | Tolerated via `cftry`; INI is written without the line-ending normalization step |

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_malware_feeds.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/malware_feeds_*.cfm` | `hermes_commandbox` | Validate / save / regen per action |
| `config/hermes/var/www/html/admin/2/inc/generate_malware_feeds_configuration.cfm` | `hermes_commandbox` | Renders the INI from the DB; runs on every action |
| `/etc/fangfrisch/fangfrisch.conf` | `hermes_mail_filter` (bind-mounted, root:clamav, 640) | Live Fangfrisch config |
| `/var/lib/fangfrisch/db.sqlite` | `hermes_mail_filter` | Per-feed last-refresh state |
| `/var/lib/fangfrisch/signatures/` | `hermes_mail_filter` | Raw downloads (pre-validation) |
| `/var/lib/clamav/` | `hermes_mail_filter` (Docker named volume `mail_filter_data_clamav`) | Validated signature store; ClamAV reads from here |
| `/usr/local/bin/setup-clamav-sigs` | `hermes_mail_filter` (image-baked) | Post-update validation + deploy hook |
| `/opt/hermes/keys/hermes.key` | `hermes_commandbox` only | AES key for `api_key_*_value` columns |
| `malware_feeds_config` table | `hermes_db_server` (`hermes` DB) | Per-feed row state |
| `malware_feed_urls` table | `hermes_db_server` | Per-feed URL list (FK cascade delete on feed delete) |
| `parameters2` rows `module='malware_feeds'` | `hermes_db_server` | Global Settings card |
| `ofelia_jobs` row `hermes-fangfrisch-refresh` | `hermes_db_server` | Schedule (auto-updated when Refresh Interval changes) |

## Related

- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) -- the ClamAV engine that
  consumes the signatures Fangfrisch downloads; engine toggles
  (ScanMail, ScanArchive, etc.) and the per-engine signature whitelist
  live on that page
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) -- the Ofelia
  admin page; the `hermes-fangfrisch-refresh` job row is editable
  there (manual Run Now, enable/disable)
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) -- per-rule SpamAssassin weight
  changes; not related to ClamAV but the closest neighbor for the
  "tune a built-in rule" pattern
- [Antispam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) -- SpamAssassin runs in the
  same Amavis pass; a ClamAV virus verdict from a Fangfrisch-supplied
  signature always pre-empts the spam score
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) -- every Fangfrisch HTTP
  download resolves through `hermes_unbound`; outbound HTTPS to the
  feed providers must be reachable
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) -- full pipeline diagram
  showing where ClamAV (and therefore feed-derived signatures) fits

# Message History

# Message History

Admin path: **Content Checks > Message History**
(`view_message_history.cfm`,
`view_message.cfm`,
`inc/messages_release_message.cfm`,
`inc/messages_block_sender.cfm`,
`inc/messages_allow_sender.cfm`,
`inc/messages_train_ham.cfm`,
`inc/messages_train_spam.cfm`,
`inc/messages_forget_bayes.cfm`,
`inc/messages_sa_learn_sync.cfm`).

This is the **operator inspection surface** for everything that has
flowed through the content filter. Every message Amavis processes lands
as one row in `msgs` (per-message metadata) plus one row per recipient
in `msgrcpt` (per-recipient disposition). This page is the joined view
over those two tables, with a date range filter, a content-type filter,
a delivery-status filter, and per-row actions to release from
quarantine, train Bayes, or block/allow the sender.

Pairs with [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) and
[Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue). System Logs shows the raw
syslog stream (connection negotiation, milter results, queue lifecycle).
Mail Queue shows what Postfix is currently holding. **Message History
shows what the content filter saw, what verdict it produced, and what
landed where** -- and lets the admin act on those rows.

The same `msgs` table feeds the **Messages Processed** donut on
[System Status](https://docs.deeztek.com/books/administrator-guide/page/system-status); the per-user
self-service version of this view lives at `/users/2/view_message_history.cfm`
and is scoped to the logged-in recipient only.

## How a message gets into `msgs` and `msgrcpt`

```
        SMTP in                postfix              amavisd-new
   ──────────────────►  hermes_postfix_dkim ────► hermes_mail_filter
                            (port 25)              (port 10024)
                                                        │
                                                        │  scan: ClamAV,
                                                        │  SpamAssassin,
                                                        │  banned-files
                                                        ▼
                                            ┌─────────────────────┐
                                            │ amavis SQL backend  │
                                            │ datasource: hermes  │
                                            ├─────────────────────┤
                                            │  msgs    (1 row /   │
                                            │           message)  │
                                            │  msgrcpt (1 row /   │
                                            │           recipient)│
                                            │  maddr   (sender +  │
                                            │           rcpt addr │
                                            │           dedup)    │
                                            └──────────┬──────────┘
                                                       │
                                  ┌────────────────────┼────────────────────┐
                                  │                    │                    │
                                  ▼                    ▼                    ▼
                          ds=P (Pass)         ds=D (Discard)        ds=B (Bounce)
                          delivered to        quarantined +         rejected with
                          downstream MTA      no further delivery   DSN to sender
                                              (quar_loc set on
                                               msgs row)
```

The `ds` ("disposition") column on `msgrcpt` is the per-recipient
verdict. The `content` column on `msgs` is the per-message **why** --
virus, spam, banned attachment, bad header, oversized, clean, etc.
Together they answer "did this message get through, and if not, what
blocked it?"

## What's in the search form

The Search Messages card at the top of the page is the filter set; all
fields are submitted as URL params so any search is bookmarkable and
back-button safe.

| Field | URL param | Effect |
| --- | --- | --- |
| Start Date/Time | `startdate` | Lower bound on `msgs.time_iso`. Defaults to **24 hours ago**. Validated as a date by `isValid("date", ...)`; invalid values short-circuit to the error template |
| End Date/Time | `enddate` | Upper bound on `msgs.time_iso`. Defaults to **now**. Same validation as startdate |
| Search Results Limit | `limit` | `LIMIT` clause on the join query. One of `1000`, `1500`, `2500`, `5000`, `10000`, `15000` -- the dropdown is the allowlist, anything else aborts. **Defaults to 1000.** The form text warns: setting limit to 10000+ significantly increases page load time |
| Type | `content_filter` | Multi-select against `msgs.content` -- the per-message content type (see table below). Empty = all types. Tom Select widget with remove and clear buttons |
| Action | `action_filter` | Multi-select against `msgrcpt.ds`. Empty = all actions. Three options: `P` Delivered, `D` Blocked (Discarded), `B` Blocked (Bounced) |

The date pickers are Tempus Dominus widgets bound to the start/end
inputs at page load; they emit `yyyy-MM-dd HH:mm:ss` into the form
fields so the validation regex matches whether the admin types the date
or picks it.

## The `msgs.content` codes -- "what was this?"

These are the values rendered by the Type column and the values used by
the Type multi-select. They come from the `msg_content_type` table
(seeded at install time):

| Code | Description | Meaning |
| --- | --- | --- |
| `V` | Virus | ClamAV (or another configured scanner) hit a signature |
| `B` | Banned | A [File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules) regex matched an attachment name, MIME type, or archive member |
| `U` | Unchecked | Amavis received the message but didn't scan (bypass policy, scanner failure, oversized, etc.) |
| `S` | Spam Quarantined | SpamAssassin score reached `spam_kill_level` per the recipient's [SVF Policy](https://docs.deeztek.com/books/administrator-guide/page/svf-policies) |
| `M` | Bad-Mime | MIME structure invalid in a way that broke the parser |
| `H` | Bad-Header | Header malformed per RFC; subject to per-policy `bad_header_lover` |
| `O` | Oversized | Message exceeded the configured size limit |
| `T` | Mta Error | Downstream MTA rejected the release / delivery attempt |
| `C` | Clean | Scanned, no findings, delivered |
| `Y` | Spam Tagged | Score reached `spam_tag2_level` (tagged with header) but stayed below `spam_kill_level` (delivered) |
| `s` | Spam Tagged (OLD) | Legacy lowercase variant; preserved for back-compat with older `msgs` rows |

The score column shown on the table is `msgs.spam_level` -- the raw
SpamAssassin score from the scan, **not** the per-policy threshold. A
row tagged `S` with score `7.2` means the recipient's SVF policy has a
`spam_kill_level` of `7.2` or lower.

## The `msgrcpt.ds` codes -- "where did it go?"

`ds` is one character per recipient row:

| `ds` | Column header | Meaning |
| --- | --- | --- |
| `P` | Delivered | Pass -- handed to the downstream MTA (Postfix re-injection on port 10025 for relay topology, LMTP to Dovecot for mailbox topology) |
| `D` | Blocked | Discard -- not delivered, quarantined on disk under `/mnt/data/amavis/<quar_loc>` |
| `B` | Blocked | Bounce -- rejected at SMTP time with DSN to sender |
| anything else | N/A | Unexpected disposition; usually means amavis was killed mid-handoff or the row is partial |

Per-recipient is the key: a single message with three recipients can
have one `P`, one `D`, and one `B` row in `msgrcpt`. The table renders
each `msgrcpt` row separately even though they share a `mail_id`.

## The results table

The DataTable below the search card is sortable, paginated (50 / 75 /
100 / All rows per page), and exportable (Copy, CSV, Excel, PDF, Print
buttons rendered by the DataTables Buttons extension). Default sort is
Date/Time descending.

| Column | Source | Notes |
| --- | --- | --- |
| Checkbox | `msgs.mail_id` | Selects the row for the Message Actions modal. **Select All** in the header checks every checkbox on the current page |
| View | -- | Magnifier button; opens `view_message.cfm?mid=<mail_id>` with the same `startdate` / `enddate` / `limit` so the back link round-trips correctly |
| Archived | `msgs.archive` | `Y` if the quarantine file has been moved to the long-term archive mount, `N` if it's still in the live amavis quarantine. Drives where `view_message.cfm` reads the EML from |
| Date/Time | `msgs.time_iso` | Indexed (`idx_msgs_time_iso`); this is the column the date range filters on. Rendered `yyyy-mm-dd HH:mm:ss` |
| Sender IP | `msgs.client_addr` | The client IP that handed the message to Postfix. For inbound that's the upstream MTA; for outbound it's the relay submitter |
| Return-Path | `maddr.email` via `msgs.sid` | The envelope sender (`MAIL FROM`); resolved via the `maddr` address-dedup table |
| From | `msgs.from_addr` | The header `From:` -- which is what users see and what DMARC aligns to |
| To | `maddr.email` via `msgrcpt.rid` | The envelope recipient. **Per-recipient** -- one table row per `msgrcpt` row |
| Subject | `msgs.subject` | Decoded subject header |
| Score | `msgs.spam_level` | Numeric score from SpamAssassin; formatted with 2 decimal places |
| Type | `msg_content_type.description` | Translated from `msgs.content` -- see the code table above |
| Action | derived from `msgrcpt.ds` | `Delivered` / `Blocked` / `Blocked` / `N/A` |

If the date range returns zero rows, the table is replaced by an info
alert ("No messages were found for the selected date range").

## The View action -- `view_message.cfm`

Clicking the magnifier opens the per-message detail page. What that
page can show is gated by two install-time toggles in
`/opt/hermes/config/security.conf`:

| Toggle | Default | Effect |
| --- | --- | --- |
| `ALLOW_MESSAGE_CONTENT=yes` | off | Show the decoded message body (HTML + text). When off, only headers are rendered |
| `ALLOW_ATTACHMENT_DOWNLOAD=yes` | off | Render the attachment list with a download button per attachment. When off, attachments are silently not listed |

Both default off because viewing a quarantined message body is a
privileged operation -- it's the difference between "the admin can see
a message was rejected" and "the admin can read a user's mail." Sites
that need release-decision support enable `ALLOW_MESSAGE_CONTENT`;
sites that need forensic attachment extraction enable
`ALLOW_ATTACHMENT_DOWNLOAD`. The fast path reads only the raw MIME
headers via a buffered Java reader so the headers page loads cheaply
even on huge quarantine files; full-body parsing only happens when the
toggle is on.

The EML is read from one of two paths depending on `msgs.archive`:

- `archive='N'` -> `/mnt/data/amavis/<quar_loc>` (live amavis quarantine)
- `archive='Y'` -> `/mnt/hermesemail_archive/mnt/data/amavis/<quar_loc>` (long-term archive)

If the file no longer exists on disk, the page aborts to the error
template instead of returning a partial render.

## Message Actions -- the bulk-action modal

Above the results table, the **Message Actions** button opens a modal
that applies one of six actions to every row whose checkbox is ticked.
The action runs in a CFML loop over the comma-delimited `mail_id` list;
each iteration includes the matching action template per-message.

| Action | Include | What it does |
| --- | --- | --- |
| Block Sender | `inc/messages_block_sender.cfm` | Adds the envelope sender to the Amavis WB-list as `B` for the recipient of that message. Honors virtual-recipient validation -- bulk attempts against unknown recipients land in `failureinvalidrecipient_email` |
| Allow Sender | `inc/messages_allow_sender.cfm` | Same as Block Sender but writes `W` (whitelist). The recipient's future mail from that sender bypasses spam scoring |
| Release Message(s) to Recipient | `inc/messages_release_message.cfm` | Calls `docker exec hermes_mail_filter /usr/sbin/amavisd-release <quar_loc> <secret_id> <recipient>`. Re-injects the message from the quarantine file into Postfix for delivery. Success detected by parsing `250 2.0.0` out of the amavisd-release stdout |
| Train Message(s) as Spam | `inc/messages_train_spam.cfm` | Runs `sa-learn --spam` against the quarantine EML so Bayes learns that pattern as spam |
| Train Message(s) as Ham (NOT Spam) | `inc/messages_train_ham.cfm` | Runs `sa-learn --ham` so Bayes learns that pattern as legitimate. Use this on the false positives released from quarantine |
| Remove Message(s) Previous Training | `inc/messages_forget_bayes.cfm` | Runs `sa-learn --forget` to undo a prior `--spam` or `--ham` call against the same message |

After any of the three Bayes actions, the page calls
`inc/messages_sa_learn_sync.cfm` (which `docker exec`s `sa-learn --sync`
to flush the in-memory token store to the Bayes database) and then
runs `/opt/hermes/scripts/bayes_chown_amavis.sh` so the freshly written
Bayes files stay owned by the amavis UID inside the content-filter
container. **Don't skip the sync** -- without it, scoring decisions
based on the new training only land after amavis's next periodic
auto-sync, which is up to an hour out.

The release-message path is the most operationally important: it
requires the quarantine file still exists on disk (the message hasn't
been pruned by the cleanup job), `amavisd-release` exits with a `250`,
and the downstream MTA accepts the re-injection. Any of those failing
puts the row in `failurereleasemessage_email` and surfaces a red alert.

> **By design.** Releasing a message does **not** automatically train
> it as ham. If a quarantined spam is actually legitimate, run
> **Release Message** *and* **Train as Ham** as separate bulk actions
> so Bayes learns the false positive.

## Status alerts -- the `m` flow

The page uses a `session.m` integer to pipe action-outcome alerts
between the action-handler block (top of file) and the alert renderers
(also top of file, after parameter setup). The handler sets
`session.m = <code>` and `cflocation`s back to the same URL with the
filter params preserved; the alert renderer reads `session.m`, emits
the matching alert, and clears the variable.

| `m` | Triggering action | Alert |
| --- | --- | --- |
| `1` | Submit clicked with no rows ticked | "You must first select message(s) before clicking the Message Actions button" |
| `3` | Block Sender | success / warning |
| `4` | Allow Sender | success / warning |
| `5` | Release Message(s) | success / warning |
| `6` | Train Ham | success / warning |
| `7` | Train Spam | success / warning |
| `8` | Forget (remove training) | success / warning |

The "warning" path fires when *some* rows in the bulk action failed --
the page lists both the successful and the failed subjects so the admin
can re-target the failures.

## Retention -- the message lifecycle

This page is **not the retention surface**; it is the read/action
surface against rows that the retention pipeline maintains. Two
scheduled jobs (registered as Ofelia jobs against `hermes_commandbox`)
own the message lifecycle:

| Schedule | Endpoint | Job |
| --- | --- | --- |
| `0 30 01 * * *` (01:30 daily) | `schedule/message_cleanup.cfm` | Prunes `msgs` + `msgrcpt` rows past the configured retention window and deletes the matching quarantine files from `/mnt/data/amavis/` |
| `@every 60s` | `schedule/quarantine_notify.cfm` | Reads the `idx_msgrcpt_notify` index, sends recipient-facing quarantine notifications for new `ds=D` rows that haven't been notified yet, and flips `notification_sent=1` |

Both are managed from [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks);
retention thresholds and per-content-type quarantine targets are
configured on [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings). The cleanup
job is the reason a `Release Message` action can fail with "quarantine
file does not exist" -- if you wait past the retention window, the EML
is gone and only the `msgs` row remains as a record.

## Performance notes

The base join (`msgs INNER JOIN msgrcpt ON msgs.mail_id =
msgrcpt.mail_id`) is hit on every page load with a `WHERE
msgs.time_iso BETWEEN ?` range. `idx_msgs_time_iso` is the index that
makes the date range cheap; without it the query degrades to a full
table scan and pages with `limit=15000` would time out on a busy
gateway. The per-row sub-queries (`getfromaddr`, `gettoaddr`, `gettype`)
fire **once per result row** because they were originally written with
N+1 semantics; on `limit=15000` that's 60K+ extra queries plus 15K
DataTable rows being rendered into the DOM. The "10000+ significantly
increases page load time" warning on the form is calibrated against
that reality.

Don't widen the date range and crank the limit at the same time when
debugging a specific incident. Narrow the window first, then widen the
limit only if you have to.

## Related pages

- [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) -- live queue (what Postfix
  is currently holding) vs. this page's historical record
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) -- raw `mail.*` syslog
  stream; use when this page shows a row but you need the
  connection / milter / delivery trace behind it
- [System Status](https://docs.deeztek.com/books/administrator-guide/page/system-status) -- the dashboard
  donut that aggregates the same `msgs` rows
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) -- the cleanup +
  notify jobs that maintain the data this page reads
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) -- spam thresholds, Bayes
  configuration, and quarantine retention windows
- [Anti-Virus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) -- ClamAV configuration
  that drives `content='V'` verdicts
- [SVF Policies](https://docs.deeztek.com/books/administrator-guide/page/svf-policies) -- per-recipient `spam_kill_level`
  that decides whether a scored message lands here as `S` (quarantined)
  or `Y` (delivered with header)
- [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules) -- attachment regexes that drive
  `content='B'` verdicts
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) and [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings)
  -- upstream authentication signals that contribute to spam scoring
  and so influence which messages land here as `S` vs `Y`

# Message Rules

# Message Rules

Admin path: **Content Checks > Message Rules**
(`view_message_rules.cfm`,
`inc/get_message_rules.cfm`,
`inc/apply_message_rules.cfm`,
`inc/update_spamassassin_config_files.cfm`,
`inc/restart_mail_filter.cfm`).

This page maintains a catalogue of **custom SpamAssassin rules**
that score against a regex match in a specific part of the message
(header, body, raw body, full message, or URI). Every rule a row on
this page produces is appended verbatim to SpamAssassin's `local.cf`
as a `<type> <name> <regex>` line, paired with a `score <name>
<value>` line, and (optionally) a `describe <name> <text>` line.
SpamAssassin then runs the rule on every message that reaches the
SpamAssassin pass inside Amavis. The cutoff that turns a final score
into a `tag` / `quarantine` action is set globally on
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings); this page only writes
the rules themselves.

Message Rules is the body/header equivalent of what
[File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) does for attachment names.
Both ride into `local.cf` / `50-user` on save, both are validated
with `spamassassin --lint` before the mail filter restarts, but
File Extensions matches the trailing extension of an attachment
filename while Message Rules matches arbitrary regex against text
inside the message.

## Where Message Rules sits

```
                       +---------------------------------------+
   Message Rules       |  message_rules table                  |
   (this page)  -----> |   id, rule_name, rule_type, header,   |
                       |   regex, score, rule_desc, applied    |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  update_spamassassin_config_files.cfm |
                       |   renders every row as                |
                       |     <type> <name> <regex>             |
                       |     score   <name> <value>            |
                       |     describe <name> <desc>            |
                       |   substituted at ##CUSTOM-MESSAGE-RULES|
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  apply_message_rules.cfm              |
                       |   spamassassin --lint                 |
                       |   restart_mail_filter.cfm             |
                       |     (docker container restart         |
                       |      hermes_mail_filter)              |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  /etc/spamassassin/local.cf in        |
                       |   hermes_mail_filter; rules contribute|
                       |   to every message's total score      |
                       +---------------------------------------+
```

A row added here only affects the SpamAssassin pass — it does not
reject at SMTP-time, it does not modify headers directly, and it
does not bypass content filtering for any recipient. It just adds
or subtracts from the final score, and whether that final score
crosses a quarantine threshold is decided by the recipient's
[SVF Policy](https://docs.deeztek.com/books/administrator-guide/page/svf-policies).

## Rule types

| Type | What it matches | Cost |
| --- | --- | --- |
| `header` | A specific message header (`Subject`, `From`, `Return-Path`, ...) or any header when `ALL` is set | Very cheap; runs against parsed header values |
| `body` | The decoded plain-text body | Cheap |
| `rawbody` | The raw/HTML body before SpamAssassin decodes it (good for catching CSS tricks, hidden text, encoded payloads) | Cheap |
| `full` | The entire raw message including all MIME parts and headers | Most expensive; use sparingly |
| `uri` | URIs extracted from the message body | Cheap; ideal for catching suspicious link patterns |

The Page Guide on the page calls out `full` as resource-intensive
because SpamAssassin runs the regex against the whole raw blob; a
greedy or expensive regex in a `full` rule can noticeably slow
every scan. Prefer `body`, `rawbody`, or `uri` where they cover
the case.

## Score semantics

The `score` value behaves identically to a
[Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) weight, except this page
creates the rule from scratch instead of overriding a shipped rule:

| Score | Effect |
| --- | --- |
| Positive (`5`, `20`, etc.) | Adds to the spam score on match. Higher values push the message toward `tag` / `quarantine` |
| `0` | Rule still runs but contributes nothing — useful for keeping the rule in place during a tuning pass without firing it |
| Negative (`-3`, `-10`) | Subtracts from the score on match — effectively whitelists messages matching the pattern |

The validation accepts any numeric value in the range `-999 .. 999`
(per the input's `step="0.01"` and `min`/`max` attributes). The
SVF policy assigned to the recipient determines what `total score`
threshold triggers tag / quarantine — see [SVF Policies](https://docs.deeztek.com/books/administrator-guide/page/svf-policies).

## The page

A Page Guide callout, a collapsible **Regex Helper** card (three
tools: rule builder, common-pattern picker, regex tester — all
client-side JavaScript that just populates the Add form), an Add
Message Rule card, and an Existing Message Rules DataTable.

### Add Message Rule card

| Field | Stored as | Notes |
| --- | --- | --- |
| Rule Name | `message_rules.rule_name` | Required. Letters, numbers, dashes, underscores only — no spaces. Must be unique. SpamAssassin uses this as the rule identifier in logs (`X-Spam-Status` header reports rule names that fired) |
| Rule Type | `message_rules.rule_type` | Required. One of `header`, `body`, `rawbody`, `full`, `uri` |
| Header | `message_rules.header` | Required when Rule Type is `header`. Letters, numbers, dashes, underscores only. Datalist suggests common headers (`Subject`, `From`, `Return-Path`, ...) plus the special `ALL` token to match any header. For non-header rules the field is force-cleared on save |
| Regex Pattern | `message_rules.regex` | Required. A SpamAssassin-format regex like `/keyword/i`. For `header` rules a `~ ` prefix is auto-added on save (this is the SpamAssassin `=~` operator notation `header_name =~ /pattern/`) and stripped on display so the operator sees only the regex |
| Score | `message_rules.score` | Required, numeric, `-999 .. 999` |
| Description | `message_rules.rule_desc` | Optional. Surfaced into the rendered `local.cf` as a `describe` line, which feeds the rule into SpamAssassin's "why was this scored" explanations |

The handler validates each field in order and returns to the page
with `session.m_rules = <code>` for the first failure (form values
are preserved through `session.form_*` so the operator doesn't
re-type). Successful insert sets `applied = 2` (pending) before
the apply chain runs, then bulk-updates all rows to `applied = 1`
once `spamassassin --lint` and the restart succeed.

### Regex Helper card

Pure client-side, no server roundtrip:

| Tool | What it does |
| --- | --- |
| Build a Rule | Pick "match in body / header / raw / URIs," choose Contains / Exact / Starts / Ends / Any-of, type the text, click Build. JavaScript escapes regex metacharacters and assembles a `/pattern/i` string |
| Quick Select Common Patterns | A `<select>` of pre-built rules for typical spam patterns ("Subject contains lottery winner", "URI: URL shortener", "HTML: hidden text"). Picking one populates the Add form below |
| Test a Pattern | Paste a `/regex/flags` and a sample string; the helper runs JavaScript's `RegExp` against it and reports Match / No match / Invalid regex |

This is operator convenience — none of it touches the database or
SpamAssassin. The pattern that lands in `message_rules.regex` is
exactly what the operator submits, even if it came from one of
these helpers.

### Existing Message Rules DataTable

| Column | Source |
| --- | --- |
| (checkbox) | Selection for bulk Delete Selected |
| Rule Name | `message_rules.rule_name` |
| Type | `message_rules.rule_type` rendered as a coloured badge per type |
| Header | `message_rules.header` for `header` rules; `N/A` otherwise |
| Regex | `message_rules.regex` (with the auto-prefixed `~ ` stripped for `header` rules so the display matches what the operator typed) |
| Score | `message_rules.score` |
| Description | `message_rules.rule_desc` |
| Actions | Per-row Edit and Delete buttons |

Edit reuses the same validation as Add. Rule Name is shown read-only
in the modal — to rename, delete and re-add (renaming would orphan
any `X-Spam-Status` historical correlation anyway).

## Save and apply flow

```
1. View page submits action="add_rule" | "edit_rule" |
   "delete_rule" | "bulk_delete"
2. Action handler validates input, INSERT/UPDATE/DELETE on
   message_rules (applied flag set to '2' = pending on
   add/edit; no applied flag manipulation on delete)
3. cfinclude apply_message_rules.cfm:
     a. cfinclude update_spamassassin_config_files.cfm:
          - Read /opt/hermes/conf_files/local.cf.HERMES (template)
          - Substitute USE-DCC, USE-PYZOR, USE-RAZOR2, USE-BAYES,
            BAYES-AUTO-LEARN, BAYESAUTOLEARN-SPAM, BAYESAUTOLEARN-HAM
            from spam_settings (the Anti-Spam Settings rows)
          - Append per-rule score overrides
            (#CUSTOM-TESTS placeholder, from spam_settings
            rows where spamfilter=1)
          - Append every message_rules row as
            "<type> <name> <regex>"+"score <name> <value>"
            [+"describe <name> <desc>" if non-blank]
            (#CUSTOM-MESSAGE-RULES placeholder)
          - Back up /etc/spamassassin/local.cf ->
            local.cf.HERMES.BACKUP, move rendered file into place
     b. Write a temp shell script wrapping
          docker exec hermes_mail_filter \
            /usr/bin/spamassassin --lint 2>/dev/null
          exit 0
        (stderr redirected to /dev/null and trailing `exit 0` —
        Lucee otherwise throws on stderr warnings; the lint return
        code is captured into lintOutput)
     c. cfinclude restart_mail_filter.cfm:
          docker container restart hermes_mail_filter
     d. UPDATE message_rules SET applied = '1'
        (mark every row as live)
4. session.m_rules = 1|2|3 -> green alert
5. cflocation back to view_message_rules.cfm
```

A few things worth knowing about this chain:

- **Lint failures do not stop the restart.** The lint output is
  captured into `lintOutput` but the include does not branch on
  it; the next step (`restart_mail_filter.cfm`) runs unconditionally.
  An invalid regex in a new rule will surface in the restarted
  container's logs (Amavis will refuse to load SpamAssassin, or
  SpamAssassin will skip the broken rule depending on the failure
  mode), not in the green alert on the page.
- **`applied = '1'` is set for every row, not just the new one.**
  The flag tracks "has every rule been pushed to the running
  SpamAssassin?" rather than "is this specific rule live?" — after
  the restart, every row is by definition live.
- **The full container is restarted** (`docker container restart
  hermes_mail_filter`), not `force-reload`d. This is the same
  restart that [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) does;
  outbound mail queues briefly during the restart (typically a
  few seconds) and Postfix retries.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m_rules = 1` | Add Rule succeeded; SpamAssassin validated and reloaded |
| `m_rules = 2` | Delete (single or bulk) succeeded; SpamAssassin validated and reloaded |
| `m_rules = 3` | Edit Rule succeeded; SpamAssassin validated and reloaded |
| `m_rules = 10` | Rule Name is empty |
| `m_rules = 11` | Rule Name contains characters other than letters, numbers, dashes, underscores |
| `m_rules = 12` | A rule with that name already exists |
| `m_rules = 13` | Header field is empty for a `header` rule |
| `m_rules = 14` | Header field contains invalid characters |
| `m_rules = 15` | Regex/Pattern is empty |
| `m_rules = 16` | Score is empty |
| `m_rules = 17` | Score is not numeric |
| `m_rules = 18` | Rule Type is not one of `header`, `body`, `rawbody`, `full`, `uri` |

The validation order is sequential — the first failure wins and
the rest of the validation does not run. Form values are preserved
into the next page render via `session.form_*` so the operator
sees their submission intact when the error renders.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_message_rules.cfm` | `hermes_commandbox` | The page (validation + Add / Edit / Delete / Bulk Delete + Regex Helper) |
| `config/hermes/var/www/html/admin/2/inc/get_message_rules.cfm` | `hermes_commandbox` | Loads the full rules list and a count of `applied = 2` (pending) rows |
| `config/hermes/var/www/html/admin/2/inc/apply_message_rules.cfm` | `hermes_commandbox` | Orchestrates the render + lint + restart + mark-applied chain |
| `config/hermes/var/www/html/admin/2/inc/update_spamassassin_config_files.cfm` | `hermes_commandbox` | Renders `local.cf` from template, appends every `message_rules` row and every `spamfilter=1` `spam_settings` row |
| `config/hermes/var/www/html/admin/2/inc/restart_mail_filter.cfm` | `hermes_commandbox` | `docker container restart hermes_mail_filter` |
| `config/hermes/opt/hermes/conf_files/local.cf.HERMES` | template (read) -> `hermes_mail_filter` (live `/etc/spamassassin/local.cf`) | Receives the rendered rules at the `#CUSTOM-MESSAGE-RULES` placeholder |
| `/etc/spamassassin/local.cf.HERMES.BACKUP` | `hermes_mail_filter` | Pre-write backup of the prior live `local.cf`, refreshed each save |
| `message_rules` table | `hermes_db_server` (`hermes` DB) | Source of truth for every rule on this page |
| `hermes_mail_filter` container | -- | Hosts SpamAssassin under Amavis; full container restart on every save |

## Related

- [SVF Policies](https://docs.deeztek.com/books/administrator-guide/page/svf-policies) -- the per-recipient policy that
  decides what threshold a rule's score has to push the running
  total above before Amavis tags or quarantines. A rule scored at
  `5` does nothing on a recipient whose SVF policy has
  `spam_kill_level = 12`
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) -- overrides for the
  weights of SpamAssassin's shipped rules; this page **creates**
  rules, that page **reweights** existing ones. Both ride into
  `local.cf` on the same save chain
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) -- engine-wide
  toggles (Bayes, DCC, Razor, Pyzor) and the global
  `final_*_destiny` quarantine actions. Subject tagging
  (`sa_spam_subject_tag`) is also set here
- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) -- the ClamAV pass
  runs before SpamAssassin in the same Amavis call; a virus verdict
  pre-empts any spam score this page contributes
- [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) -- the attachment-name
  equivalent of this page; both write to `hermes_mail_filter` on
  save and share the lint-then-restart pattern (File Extensions
  uses `force-reload` instead of full restart because no
  SpamAssassin state is touched)
- [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules) -- bundles extensions into named
  rulesets that bind to SVF policies; orthogonal to message rules
  but lives in the same Amavis pass
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) -- nothing on this page
  matters for messages rejected at SMTP-time; rules here only see
  the traffic that already passed the perimeter
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) -- a quarantined message
  surfaces the matched rule names in the SpamAssassin verdict
  details, which is the canonical way to see whether a custom
  rule fired
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) -- the `amavis[...]`
  line for a scored message reports `tests=` followed by every
  rule that fired with its weight, including custom rules from
  this page

# Network Block/Allow

# Network Block/Allow

Admin path: **Content Checks > Network Block/Allow**
(`view_network_block_allow.cfm`, `inc/get_network_block_allow.cfm`,
`inc/network_add_entries.cfm`, `inc/network_edit_entry.cfm`,
`inc/network_delete_entry.cfm`,
`inc/generate_postscreen_access.cfm`).

This page manages the **operator-curated CIDR list** that Postfix's
`postscreen` daemon consults at TCP-accept time, before any DNSBL
scoring or SMTP handshake. Each entry pairs a single IP or CIDR with
an action — `permit` (allow / RBL bypass) or `reject` (block) — and
the list is written verbatim to `/etc/postfix/postscreen_access.cidr`
on every save. The directive that wires it in lives in `main.cf`:

```
postscreen_access_list = permit_mynetworks, cidr:/etc/postfix/postscreen_access.cidr
```

This is the **third-party-list override** for the perimeter — the
place an admin overrides a misfiring [RBL](https://docs.deeztek.com/books/administrator-guide/page/rbl-configuration) hit
without disabling the RBL itself, and the place a known-bad source is
dropped before it can even attempt SMTP.

## Where this list sits in the flow

```
+-------------------------+
|  Inbound TCP connect    |
+-----------+-------------+
            |
            v
+-------------------------------------------------+
|  postscreen :25 (hermes_postfix_dkim)           |
|                                                 |
|  1. postscreen_access_list                      |
|     permit_mynetworks                           |
|     cidr:/etc/postfix/postscreen_access.cidr    |
|     -> permit  -> hand off to smtpd, skip all   |
|                    scoring (RBL, greet, etc.)   |
|     -> reject  -> 550, connection closed        |
|     -> no hit  -> fall through                  |
|                                                 |
|  2. postscreen_dnsbl_sites (RBL scoring)        |
|     -> threshold met -> 550                     |
|                                                 |
|  3. pipelining / non-SMTP / bare-newline        |
|     (if enabled on Perimeter Checks)            |
|                                                 |
+-----------+-------------------------------------+
            | passes -> hand to smtpd
            v
+-------------------------------------------------+
|  smtpd :25  (smtpd_*_restrictions)              |
+-------------------------------------------------+
```

The position of `cidr:/etc/postfix/postscreen_access.cidr` matters:
because it sits **before** `postscreen_dnsbl_sites` in
`postscreen_access_list`, a `permit` entry here causes postscreen to
short-circuit and skip every DNSBL lookup for that source. A
`reject` entry closes the connection with no further checks at all.

## Distinction from Relay Networks

This page is easy to confuse with
[Relay Networks](https://docs.deeztek.com/books/administrator-guide/page/relay-networks) — both store IPs
and CIDRs against Postfix. They are not the same:

| Page | Postfix destination | What an entry does |
| --- | --- | --- |
| **Network Block/Allow** (this page) | `cidr:/etc/postfix/postscreen_access.cidr`, consulted by `postscreen_access_list` | `permit` = skip RBL scoring for this IP. `reject` = 550 at TCP accept. **No trust granted** — the source still passes through `smtpd_recipient_restrictions` and content scanning |
| **Relay Networks** | `mynetworks` directive in `main.cf`, also Amavis `@inet_acl` | Sets `permit_mynetworks` — sender is **fully trusted**: bypasses RBL, SPF, sender/recipient checks, *and* is allowed to relay outbound to any destination |

A wrong entry on Relay Networks creates an open relay. A wrong entry
here at worst lets a few extra messages through the perimeter into
content scanning, where Amavis + SpamAssassin + ClamAV still apply.
The two pages serve different jobs — gate the source vs. trust the
source — and the postfix directives they write to are distinct.

## When to add a `permit` entry

| Scenario | Why allow here instead of Relay Networks |
| --- | --- |
| Trusted partner whose IP is listed in an RBL | You want their mail through, but you do **not** want to grant them open relay; the RBL bypass is enough |
| Shared-hosting sender whose IP also hosts a spammer | Same as above — bypass RBL scoring, let content checks still apply |
| Microsoft 365 outbound ranges | EOP IPs are already in the shipped seed list as `permit` (151 rows on a fresh install). They are inbound mail sources — they don't need relay trust |
| Internal monitoring sender whose IP randomly appears in CBL | RBL false positives caught by IP age or shared CGN |

## When to add a `reject` entry

| Scenario | Why reject here instead of waiting for content scoring |
| --- | --- |
| Persistent spam source that consistently slips past RBLs | Cheapest possible reject — no DATA accepted, no Amavis cycles |
| Compromised CIDR block that the operator wants closed off entirely | One CIDR row handles a whole /24, /16, or /8 |
| Manual ban after a Fail2ban-or-equivalent decision is escalated to permanent | A `reject` here outlasts any IP-table or jail-based ban |

## The two cards on the page

### 1. Add IP/Network

A textarea for bulk entry — one per line, `IP_or_Network [Note]`. The
note is everything after the first space on each line; the IP/CIDR is
everything before it. If a line has no space, the entry is its own
note.

Validation runs per line:

- Plain IP: must match a strict IPv4 dotted-quad regex
  (`^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}…$`).
- CIDR: split on `/`, validate the network half against the same
  regex, then validate the prefix is an integer in `1..32`.
- Both forms are normalized through `normalizeIP()` — strips leading
  zeros from each octet (`010.001.001.001/8` becomes `10.1.1.1/8`).
- Duplicates against `postscreen_access.sender` are skipped with a
  warning; processing continues for the rest of the batch.

The single **Action** radio applies to the whole textarea — every
line in one submit gets the same `permit` or `reject`. To mix
actions, submit twice.

On submit: rows are `INSERT`-ed into `postscreen_access` with
`applied=1, action2='NONE'`, then `generate_postscreen_access.cfm` is
included to write the new CIDR file and reload Postfix in the same
request. The green "Entries Added" alert summarizes `added`,
`skipped`, and any per-line errors.

### 2. Network Entries (DataTable)

Searchable, sortable, paginated; bulk-delete checkboxes,
per-row Edit / Delete buttons.

| Column | Source |
| --- | --- |
| IP/Network | `postscreen_access.sender` |
| Note | `postscreen_access.note` (free text from the second half of each Add line) |
| Action | `postscreen_access.action` rendered as a green "Allow" or red "Block" badge |
| Actions | Edit (modal), Delete (confirm) |

The Edit modal lets the operator change the IP, the action (Allow /
Block), or the note in one form post.

## Save flow

```
Add / Edit / Delete
    |
    v
INSERT / UPDATE / DELETE on postscreen_access (datasource: hermes)
    |
    v
cfinclude generate_postscreen_access.cfm
    1. SELECT all enabled rows ORDER BY sender ASC
    2. Write /etc/postfix/postscreen_access.cidr
         <sender>\t<action>\n   per line
    3. docker exec hermes_postfix_dkim /usr/sbin/postfix reload  (30s timeout)
    |
    v
session.m = 1 / 2 / 5 (Added / Deleted / Updated)
On failure -> session.m = 4 ("Configuration Error")
```

The file is written via a direct `cffile action="write"` from the
CommandBox container — possible because `/etc/postfix/` is a
host-bind-mounted volume shared between `hermes_commandbox` and
`hermes_postfix_dkim`. The reload then runs inside the postfix
container via `docker exec`. No `postmap` is required for
`cidr:` tables — Postfix reads them as text at load time.

## The `postscreen_access` table

| Column | Type | Role |
| --- | --- | --- |
| `id` | `int AUTO_INCREMENT` | Primary key (used as form `delete_id` / `edit_id`) |
| `sender` | `varchar(255)` | The IP or CIDR string (the column is named `sender` for historical reasons — it is **not** an envelope sender) |
| `action` | `varchar(255)` | `permit` or `reject` |
| `action2` | `varchar(255)` | Always `NONE` — legacy two-phase apply column kept for compatibility |
| `applied` | `int` | `1` once the row is live in the generated `.cidr` file |
| `note` | `varchar(255)` | Free-text label shown in the table |

Engine is MyISAM (matches other operator-curated tables in the
schema); collation `latin1_swedish_ci`. The shipped seed includes a
large block of Microsoft 365 / Exchange Online Protection ranges as
`permit` so EOP-fronted senders are never RBL-scored on a fresh
install.

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Empty textarea on Add | `session.m = 30`, redirect, no DB write |
| Invalid IP or CIDR on a line | Line skipped, `entries_skipped` incremented, error appended; other lines still process |
| Duplicate against existing `sender` | Same as invalid — skipped with a `Duplicate:` error line |
| `cffile` cannot write `/etc/postfix/postscreen_access.cidr` | `cfcatch` -> `session.m = 4` ("Configuration Error") |
| `postfix reload` fails inside the container | Same `session.m = 4` path |

If the SQL inserts succeed but the file write or reload fails, the
database state has advanced but the live CIDR file lags. The next
successful save (or any Edit / Delete) re-renders the file from the
current table contents, so the page does not strand split-brain state
permanently.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_network_block_allow.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/get_network_block_allow.cfm` | `hermes_commandbox` | Loads active rows |
| `config/hermes/var/www/html/admin/2/inc/network_add_entries.cfm` | `hermes_commandbox` | Per-line validate, INSERT, regen + reload |
| `config/hermes/var/www/html/admin/2/inc/network_edit_entry.cfm` | `hermes_commandbox` | UPDATE, regen + reload |
| `config/hermes/var/www/html/admin/2/inc/network_delete_entry.cfm` | `hermes_commandbox` | DELETE single or bulk, regen + reload |
| `config/hermes/var/www/html/admin/2/inc/generate_postscreen_access.cfm` | `hermes_commandbox` | Rewrites `/etc/postfix/postscreen_access.cidr` and reloads Postfix |
| `postscreen_access` table | `hermes_db_server` (`hermes` DB) | Source of truth |
| `/etc/postfix/postscreen_access.cidr` (volume mount) | `hermes_postfix_dkim` | Live CIDR file consumed by postscreen |
| `hermes_postfix_dkim` container | — | Where `postfix reload` runs |

## Related

- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — postscreen toggles and the
  DNSBL threshold; this page's `permit` / `reject` short-circuits the
  scoring that page configures
- [RBL Configuration](https://docs.deeztek.com/books/administrator-guide/page/rbl-configuration) — the DNSBL list that a
  `permit` entry on this page **skips entirely**; the canonical
  RBL-false-positive override
- [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) — envelope-level
  block/allow applied later in the pipeline (Amavis), not at TCP
  accept
- [Global Sender Rules](https://docs.deeztek.com/books/administrator-guide/page/global-sender-rules) — envelope-sender
  block/allow that applies to every recipient on the system
- [Relay Networks](https://docs.deeztek.com/books/administrator-guide/page/relay-networks) — the
  **trust** list (`mynetworks` / `permit_mynetworks`); explicitly
  different from this page's gate-only semantics
- [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) — the
  authenticated path that supersedes IP-based trust for senders that
  can authenticate
- [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) — the
  Fail2ban-equivalent layer that maintains short-lived IP bans; this
  page is where bans get promoted to permanent
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — postscreen `permit` /
  `reject` decisions surface in the postfix log under
  `postscreen[...]`

# Perimeter Checks

# Perimeter Checks

Admin path: **Content Checks > Perimeter Checks**
(`view_perimeter_checks.cfm`, `inc/get_perimeter_checks.cfm`,
`inc/perimeter_save_settings.cfm`,
`inc/generate_postfix_configuration.cfm`).

This page collects every **SMTP-time** check Hermes can apply before the
message body is even read. Each control here writes a row (or toggles
`enabled`) in the `parameters` table; on save, the
`generate_postfix_configuration.cfm` include rebuilds `main.cf` from
those rows via `postconf -e` and runs `postfix reload` inside
`hermes_postfix_dkim`. There is no message-content inspection on this
page — content scoring lives in [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings)
and [Anti-Virus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings), and runs only after
the perimeter checks accept the connection.

## Where perimeter checks sit in the flow

```
+-------------------+
| Remote SMTP peer  |
+---------+---------+
          |
          v
+-----------------------------------------------+
|  postscreen :25 (hermes_postfix_dkim)         |
|   - postscreen_access.cidr  (whitelist/block) |
|   - DNSBL scoring -> postscreen_dnsbl_sites   |
|   - pipelining / non-SMTP / bare-newline      |
+---------+-------------------------------------+
          | passes -> hand off
          v
+-----------------------------------------------+
|  smtpd :25                                    |
|   - smtpd_helo_required                       |
|   - smtpd_client_restrictions                 |
|   - smtpd_helo_restrictions                   |
|   - smtpd_sender_restrictions                 |
|   - smtpd_recipient_restrictions              |
|     (permit_mynetworks, permit_sasl_auth,     |
|      reject_unauth_destination,               |
|      reject_invalid_hostname, ...,            |
|      reject_rbl_client / DNSBL,               |
|      check_policy_service for SPF)            |
|   - message_size_limit                        |
+---------+-------------------------------------+
          | passes -> DATA accepted
          v
+-----------------------------------------------+
|  Amavis / SpamAssassin / ClamAV (content)     |
+-----------------------------------------------+
```

Perimeter Checks owns the postscreen knobs and the `smtpd_*_restrictions`
toggles. RBL list membership is split out to its own page —
[RBL Configuration](https://docs.deeztek.com/books/administrator-guide/page/rbl-configuration) — because the list is
row-per-entry data, not a fixed set of switches.

## The four cards on the page

### 1. Postscreen Settings

`postscreen` is Postfix's pre-queue connection filter — it sits in
front of `smtpd` on port 25 and runs cheap protocol checks before any
SMTP state machine is built. Three switches:

| Switch | `parameters` row | Postfix directive | What it catches |
| --- | --- | --- | --- |
| Pipelining Detection | `postscreen_pipelining_enable` | `postscreen_pipelining_enable = yes/no` | Clients that send `EHLO` + `MAIL FROM` + `RCPT TO` in one TCP write before the server has finished its greeting — classic spambot shortcut |
| Non-SMTP Command Detection | `postscreen_non_smtp_command_enable` | same | Clients that send something other than the SMTP verbs (typically HTTP `GET` from a misdirected scanner, or shellcode) |
| Bare Newline Detection | `postscreen_bare_newline_enable` | same | Clients that terminate lines with a bare `\n` instead of `\r\n` — RFC 5321 violation, very common in homebrew bot SMTP libraries |

> **Operational consequence.** Enabling any of these activates
> **greylisting-style deferral** for unknown clients. Mail from a
> well-behaved peer is delayed by one retry on first contact; mail from
> a peer that retries incorrectly (or not at all) is lost. The in-page
> callout warns about this explicitly. Leave these off until you have
> a reason to turn them on.

### 2. Message Limits

A single control: **Maximum Message Size (MB)**. The page displays the
value in megabytes; on save it is multiplied by `1024*1024` and the
integer byte count is written to the child row under the
`message_size_limit` parent. Postfix enforces this at `DATA`-accept
time and rejects with `552 5.3.4` if the message exceeds the limit.

Validation rejects zero, negative, and non-numeric input
(`session.m = 3`).

### 3. SMTP Restrictions

The bulk of the page. The HELO toggle and seven recipient-side
rejects each map to a child row under one of two parent parameters:

| Toggle | Parent | Postfix directive | Rejects when... |
| --- | --- | --- | --- |
| Require HELO/EHLO | `smtpd_helo_required` | `smtpd_helo_required = yes` | Client tries to send `MAIL FROM` without first issuing `HELO` or `EHLO` |
| Reject Unauthorized Destination | `smtpd_recipient_restrictions` | `reject_unauth_destination` | Recipient domain is not a relay or hosted domain (open-relay protection — **leave on**) |
| Reject Unauthorized Pipelining | `smtpd_recipient_restrictions` | `reject_unauth_pipelining` | Client pipelines commands without `EHLO` advertising support |
| Reject Invalid Hostname | `smtpd_recipient_restrictions` | `reject_invalid_hostname` | HELO/EHLO name is syntactically invalid (e.g. no dot) |
| Reject Non-FQDN Sender | `smtpd_recipient_restrictions` | `reject_non_fqdn_sender` | `MAIL FROM:` address has no fully-qualified domain |
| Reject Unknown Sender Domain | `smtpd_recipient_restrictions` | `reject_unknown_sender_domain` | Sender domain has neither MX nor A record in DNS |
| Reject Non-FQDN Recipient | `smtpd_recipient_restrictions` | `reject_non_fqdn_recipient` | `RCPT TO:` address has no fully-qualified domain |
| Reject Unknown Recipient Domain | `smtpd_recipient_restrictions` | `reject_unknown_recipient_domain` | Recipient domain has neither MX nor A record in DNS |

The **DNSBL Threshold** field in the same card writes
`postscreen_dnsbl_threshold` — the combined score that any single
connecting IP must reach across all enabled DNSBL zones before
postscreen rejects it. The shipped baseline is `3`. Per-zone weights
are configured on [RBL Configuration](https://docs.deeztek.com/books/administrator-guide/page/rbl-configuration); the
threshold here is what those weights add up against. Validation
requires an integer (`session.m = 2`).

> **Order matters in Postfix.** The save routine does not let an admin
> reorder restrictions — the `order1` column in `parameters` is seeded
> at install time so that `permit_mynetworks` and
> `permit_sasl_authenticated` come first, then the
> `reject_unauth_destination` open-relay guard, then sender / recipient
> validation, then policy services. This is the canonical order; the
> UI only toggles which entries are active, not where they sit in the
> list.

### 4. Email Authentication (read-only status)

Three badges (SPF, DKIM, DMARC) showing whether each authentication
service is wired into `smtpd_milters` / `smtpd_recipient_restrictions`,
each with a small "Configure..." link to its dedicated page. This
card is informational — toggling SPF/DKIM/DMARC on or off happens on:

- [SPF Settings](https://docs.deeztek.com/books/administrator-guide/page/spf-settings) — child row under `smtpd_recipient_restrictions`
- [DKIM Settings](https://docs.deeztek.com/books/administrator-guide/page/dkim-settings) — milter at `inet:%:8891` in `smtpd_milters`
- [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) — milter at `inet:%:54321` in `smtpd_milters`

The DMARC row carries an additional note: DMARC requires SPF **and**
DKIM to both be active. If either is disabled, the card surfaces
"Requires both SPF and DKIM" inline.

## Save flow

A single **Save & Apply Settings** click runs:

```
1. Validate dnsbl_threshold (integer) and message_size_limit (positive float)
   - Fail -> session.m = 2 or 3, cflocation back, no DB write
2. UPDATE parameters child rows for all toggles + values (applied = 2)
3. cfinclude generate_postfix_configuration.cfm
     a. Copy /opt/hermes/conf_files/main.cf.HERMES -> /etc/postfix/main.cf
     b. SELECT all enabled parents (child=2), join children (child=1)
     c. Write /opt/hermes/tmp/<trans>_postconf.sh with one
        `postconf -e "<directive> = <values>"` line per parent
     d. Append `postfix reload`
     e. docker exec hermes_postfix_dkim /bin/bash <script>
     f. UPDATE parameters SET applied=1, action='NONE' WHERE applied=2
4. session.m = 1 -> green "Settings Saved" alert on redirect
   On failure -> session.m = 4 with cfcatch detail surfaced in the alert
```

The reload is in-band — the page does not return until Postfix has
reloaded (timeout: 240s).

## The `parameters` dual-row pattern (perimeter-specific)

Every Postfix directive in Hermes is stored as **two-or-more linked rows**
in the `parameters` table:

| `child` | Role | What the `parameter` column holds |
| --- | --- | --- |
| `2` | Parent (directive name) | The Postfix directive name (e.g. `smtpd_recipient_restrictions`) |
| `1` | Child (directive value) | One value the directive should emit (e.g. `reject_unauth_destination`, or `yes`) |

Rows are linked by `parent_name` (child's `parent_name` matches
parent's `parameter`) or by numeric `parent` (child's `parent` matches
parent's `id`). The `order1` column sequences children inside a
parent so the generated `postconf -e` line emits values in a
predictable order.

For perimeter checks, that means:

- `smtpd_helo_required` has **one** child row whose `parameter` is
  literally the string `yes` or `no` (toggle flips `enabled` on that
  one row).
- `smtpd_recipient_restrictions` has **many** child rows — one per
  restriction value. The toggle for each restriction flips `enabled`
  on its child row; the generator emits only `enabled=1` children.
- `message_size_limit` has one child row whose `parameter` is the
  literal byte-count string (e.g. `78643200`); the save handler
  rewrites that string on every save.

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Invalid `dnsbl_threshold` | `session.m = 2`, redirect, no DB write |
| Invalid `message_size_limit` | `session.m = 3`, redirect, no DB write |
| `generate_postfix_configuration.cfm` throws | `session.m = 4`; `session.postfix_error` is set to `cfcatch.message & cfcatch.detail` and surfaced under a small "Detail:" line in the red alert |
| `postfix reload` fails inside the container | Surfaces as a `cfcatch` from the `cfexecute` of the temp script — same `session.m = 4` path |
| `main.cf.HERMES` template missing in `/opt/hermes/conf_files/` | `cfcatch` on the template copy step — same path |

The save is **not** transactional across the steps — if the SQL
updates succeed but the reload fails, the DB state advances to
`applied=2` and the next save attempt will pick those rows up and
re-apply. The page does not strand partial state.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_perimeter_checks.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/get_perimeter_checks.cfm` | `hermes_commandbox` | Loads parent IDs + current child values |
| `config/hermes/var/www/html/admin/2/inc/perimeter_save_settings.cfm` | `hermes_commandbox` | Validates form, updates `parameters`, calls the generator |
| `config/hermes/var/www/html/admin/2/inc/generate_postfix_configuration.cfm` | `hermes_commandbox` | Writes a temp `postconf -e` shell script, executes inside the postfix container, reloads Postfix |
| `config/hermes/opt/hermes/conf_files/main.cf.HERMES` | `hermes_commandbox` (read) → `hermes_postfix_dkim` (live `/etc/postfix/main.cf`) | Canonical template copied on every regen |
| `parameters` table | `hermes_db_server` (`hermes` DB) | Source of truth for every restriction and toggle |
| `hermes_postfix_dkim` container | — | Where `postconf -e` + `postfix reload` execute |

## Related

- [RBL Configuration](https://docs.deeztek.com/books/administrator-guide/page/rbl-configuration) — the DNSBL list whose
  combined score is compared against the **DNSBL Threshold** on this
  page
- [Network Block/Allow](https://docs.deeztek.com/books/administrator-guide/page/network-blockallow) — the
  `postscreen_access` CIDR table consulted by postscreen before the
  DNSBL checks
- [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) — per-address
  override of perimeter-level rejects
- [SPF Settings](https://docs.deeztek.com/books/administrator-guide/page/spf-settings), [DKIM Settings](https://docs.deeztek.com/books/administrator-guide/page/dkim-settings),
  [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) — the three authentication
  services whose status appears in card 4
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — content-time scoring
  that runs after a connection clears the perimeter
- [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) — the
  cipher/protocol choices applied at the same `smtpd :25` listener
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — every
  `reject_unknown_*_domain`, `reject_invalid_hostname`, and DNSBL
  query goes through `hermes_unbound`; resolver mode (recursive vs.
  forwarding) directly affects perimeter accuracy
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) — full pipeline diagram

# RBL Configuration

# RBL Configuration

Admin path: **Content Checks > RBL Configuration**
(`view_rbl_configuration.cfm`, `inc/get_rbl_configuration.cfm`,
`inc/rbl_add_entry.cfm`, `inc/rbl_edit_entry.cfm`,
`inc/rbl_delete_entry.cfm`, `inc/rbl_test_entry.cfm`,
`inc/generate_postfix_configuration.cfm`).

This page manages the **DNSBL** (block) and **DNSWL** (allow) lists
that Postfix's `postscreen` daemon consults before a connection is
even handed off to `smtpd`. Each enabled entry contributes a
**weighted score** for the connecting IP; when the running total
crosses the threshold set on
[Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks#3-smtp-restrictions), postscreen
rejects the connection with `550 5.7.1`. Allow-list entries subtract
from that score and can rescue a sender that one or two block lists
flag.

The list is row-per-entry data — add, edit, delete, and live-test
operations all happen on this page. The numerical threshold those
weights are compared against is a single integer on the Perimeter
Checks page (`postscreen_dnsbl_threshold`, default `3`).

## How postscreen scoring works

```
Inbound TCP -> postscreen :25
                  |
                  v
        For each enabled DNSBL site:
          dig <reversed-client-ip>.<rbl-zone>
          if A record returned (and matches optional =127.x.x.x filter):
            add (or subtract) the entry's weight
                  |
                  v
        Sum >= postscreen_dnsbl_threshold ?
          yes -> reject 550 5.7.1
          no  -> pass to smtpd for the rest of the perimeter checks
```

The decision is made against **a single connecting IP** in a single
postscreen session. Postscreen does this in parallel across every
enabled zone and waits up to a few seconds for responses.

### Block vs. Allow

| Type | Stored weight | DNS contribution | Typical use |
| --- | --- | --- | --- |
| Block List (DNSBL) | Positive integer (`+1` … `+8` typical) | Adds to the score on hit | `zen.spamhaus.org`, `bl.spamcop.net`, `b.barracudacentral.org` |
| Allow List (DNSWL) | Negative integer (`-2` … `-8` typical) | Subtracts from the score on hit | `list.dnswl.org`, `wl.mailspike.net`, `hostkarma.junkemailfilter.com=127.0.0.1` |

The UI presents two radio buttons (Block List / Allow List) and a
positive weight; the save handler signs the weight automatically
(positive for block, negative for allow) and stores both the signed
integer in the `weight` column and a string representation in the
`parameter` column (`<host>*<weight>` for block,
`<host>*-<abs(weight)>` for allow).

### Return-code filtering

Many DNSBL providers publish **different return codes** for different
sub-lists inside a single zone. Spamhaus ZEN is the canonical
example: `127.0.0.2` for SBL, `127.0.0.3` for the CSS sub-list,
`127.0.0.4`-`7` for XBL, `127.0.0.10`-`11` for PBL. Postfix lets you
match a subset of those codes with the
`<hostname>=127.x.x.x` syntax (and `=127.0.0.[N..M]` /
`=127.0.0.[N;M;O]` for ranges and unions). This lets an admin assign
a different weight to each sub-list:

```
zen.spamhaus.org=127.0.0.2        weight 3   (SBL — moderate confidence)
zen.spamhaus.org=127.0.0.3        weight 4   (CSS)
zen.spamhaus.org=127.0.0.[4..7]   weight 6   (XBL — exploit list)
zen.spamhaus.org=127.0.0.[10;11]  weight 8   (PBL — policy list)
```

The shipped baseline includes exactly this kind of staged Spamhaus
configuration plus per-code weights for several other providers; see
the **RBL Entries** table after a fresh install.

## The two cards on the page

### 1. Add RBL Entry

Four inputs: hostname (with optional `=127.x.x.x` filter), type
(Block / Allow), positive weight, and submit. The hostname is
validated by stripping any `=...` suffix and running the bare host
through `IsValid("email", "test@" & hostPart)` — a permissive
syntactic check that accepts valid DNS labels and rejects empty
strings, whitespace, and obvious garbage.

Duplicates are blocked via a `LIKE '%<host>%'` lookup on the
`parameters` table before insert; the page surfaces a "Duplicate
Entry" warning if a row already contains the hostname (including
existing entries with different `=127.x.x.x` filters — be aware that
the substring check will treat `zen.spamhaus.org=127.0.0.2` and
`zen.spamhaus.org=127.0.0.3` as duplicates of each other, so add
sub-list variants by editing the existing row's filter rather than
inserting a second).

On success: `INSERT` into `parameters` under the
`postscreen_dnsbl_sites` parent, immediately call
`generate_postfix_configuration.cfm`, redirect with `session.m = 1`
(green "Entry Added" alert). The full RBL list takes effect on the
next inbound connection.

### 2. RBL Entries (DataTable)

Searchable, sortable, paginated table with bulk-delete checkboxes,
per-row Test / Edit / Delete buttons, and a **Test All** action.

| Column | Source |
| --- | --- |
| Hostname | `parameter` column with the trailing `*<weight>` stripped for display |
| Type | Derived from sign of `weight` — positive = Block, negative = Allow |
| Weight | `Abs(weight)` |
| Status | Live AJAX result of the per-row DNS test (see below); starts as "Not Tested" |
| Actions | Test (vial icon), Edit, Delete |

The DataTable is wrapped in a `<form>` whose submit target is the
**bulk delete** handler; per-row Delete and Edit use separate hidden
forms outside the DataTable so they don't collide with the bulk form.

## The live RBL test

The vial-icon button on each row triggers
`view_rbl_configuration.cfm?action=test_entry&id=<id>` — an
AJAX-only branch that runs **before** any HTML output and returns
JSON. The handler performs a **two-stage DNS probe** from inside
the same container Postfix uses for its real DNSBL queries:

| Stage | Query | Pass criterion |
| --- | --- | --- |
| 1. Test-data lookup | `dig +short A 2.0.0.127.<zone>` (the IP `127.0.0.2` reversed, prefixed onto the zone — the universal DNSBL "test record") | Response starts with `12` (i.e. a `127.x.x.x` answer) → zone is actively publishing data |
| 2. SOA fallback | `dig +short SOA <zone>` | Non-empty response → zone infrastructure exists even if the test record was not returned |

Both `dig` invocations run via
`docker exec hermes_postfix_dkim dig +short +time=3 +tries=1 ...`
inside a `cfthread` with a 10-second join timeout. This matters for
two reasons:

1. **Same resolver as Postfix.** The CommandBox JVM's DNS resolver
   cannot reliably reach DNSBL zones; querying from the postfix
   container guarantees the test sees what the live mail flow sees.
2. **Same source IP as Postfix.** Many DNSBL providers throttle or
   refuse responses to public-resolver IPs (Cloudflare, Google,
   Quad9). The test must originate from the same egress IP as the
   real queries to give a meaningful result. This is the central
   reason Hermes ships its own [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver);
   if that resolver is flipped to forwarding mode through a public
   provider, both the live tests and real DNSBL traffic will degrade.

Result encoding:

| JSON `status` | Badge | Meaning |
| --- | --- | --- |
| `ok` (stage 1 hit) | Green "Zone Active" with the returned IP in the tooltip | Zone is publishing test data and reachable |
| `ok` (stage 2 hit) | Green "Zone Active" with "Zone active (SOA)" tooltip | Zone infrastructure exists; test record not returned (common — many providers block data-center IPs from test queries) |
| `error` | Red "Error" | No DNS response, NXDOMAIN, or NS delegation only with no SOA |
| `timeout` | Red "Unreachable" | The 10-second thread join expired |

> **Green only confirms zone infrastructure — not that the list is
> actively publishing data.** Many DNSBL providers (Barracuda is the
> common example) block data-center IP ranges from running live data
> queries. A stage-2-only green from such a provider is the **expected**
> healthy result, not a problem — the live mail-flow queries are
> coming from the same blocked IP, so they will also miss, and the
> provider in that case isn't actually contributing to scoring.

### Why dead RBLs are dangerous in both directions

The in-page callout flags this explicitly:

- A **dead Block List** that starts returning wildcard `127.0.0.2`
  matches for every IP will inflate the postscreen score for every
  connection — potentially blocking all inbound mail. Spamhaus's
  domain seizure in 2013 and the SORBS hand-off in 2024 are both
  examples of zones that briefly entered this state.
- A **dead Allow List** that starts wildcard-matching will subtract
  from every score, letting spam through that would otherwise be
  blocked. DNSWL has had brief outages with similar effects.

The live tests catch zones that are flat-out unreachable; they cannot
catch zones that are actively publishing wrong answers. The
operational mitigation is to keep the weight on any single entry
small enough that one misbehaving zone cannot single-handedly cross
the threshold — the shipped weights are set with this in mind
(per-zone weights of `2`-`8` against a threshold of `3` means at
least two corroborating hits are required for a block).

## Edit and delete

The Edit modal preserves the same Block / Allow toggle + positive
weight UX as Add; on save it rewrites both the `parameter` string
and the signed `weight` integer. Single-row delete uses a confirm
prompt + hidden `<form>` POST; bulk delete posts a comma-separated
list of `parameters.id` values from the wrapping DataTable form.
All three (add, edit, delete) call
`generate_postfix_configuration.cfm` inline and reload Postfix in the
same request.

## Save flow

```
1. (Add / Edit / Delete) Validate input, INSERT / UPDATE / DELETE
   on the `parameters` table under postscreen_dnsbl_sites parent
2. cfinclude generate_postfix_configuration.cfm
     - SELECT all enabled children of every enabled parent,
       including the full ordered list of postscreen_dnsbl_sites
     - Render a temp postconf -e script + `postfix reload`
     - docker exec hermes_postfix_dkim /bin/bash <script>
     - UPDATE parameters SET applied=1 WHERE applied=2
3. session.m = 1 / 2 / 5 (Added / Deleted / Updated)
   On failure -> session.m = 4
```

## The `parameters` rows for DNSBL sites

| Column | Value (block-list example) | Value (allow-list example) |
| --- | --- | --- |
| `parameter` | `zen.spamhaus.org=127.0.0.[4..7]*6` | `list.dnswl.org=127.0.[0..255].3*-8` |
| `parent_name` | `postscreen_dnsbl_sites` | `postscreen_dnsbl_sites` |
| `weight` | `6` (positive integer) | `-8` (negative integer) |
| `child` | `1` (it's a child of the directive parent row) | `1` |
| `order1` | Sequence within the directive (auto-incremented on Add) | Same |
| `enabled` | `1` to include in the live `postscreen_dnsbl_sites` value | `1` |
| `applied` | `1` once Postfix has been reloaded against this row, `2` while pending | Same |

The generator joins the children into a single comma-separated value
for the `postscreen_dnsbl_sites` directive — the live Postfix
configuration ends up as one long line of `<zone>=<filter>*<weight>`
tokens.

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Empty hostname on Add | `session.m = 10`, redirect, no DB write |
| Invalid hostname syntax (Add or Edit) | `session.m = 11`, redirect, no DB write |
| Duplicate hostname (Add) | `session.m = 12`, redirect, no DB write |
| `generate_postfix_configuration.cfm` throws | `session.m = 4`, red "Configuration Error" alert |
| `dig` inside `hermes_postfix_dkim` times out (test only) | JSON `{"status":"timeout"}` → red "Unreachable" badge; live mail flow is unaffected |
| `hermes_postfix_dkim` not running (test only) | JSON `{"status":"error"}` → red "Error" badge |

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_rbl_configuration.cfm` | `hermes_commandbox` | The page (with the early `action=test_entry` AJAX intercept) |
| `config/hermes/var/www/html/admin/2/inc/get_rbl_configuration.cfm` | `hermes_commandbox` | Loads the `postscreen_dnsbl_sites` parent ID + all active children |
| `config/hermes/var/www/html/admin/2/inc/rbl_add_entry.cfm` | `hermes_commandbox` | Validate, INSERT, regen + reload |
| `config/hermes/var/www/html/admin/2/inc/rbl_edit_entry.cfm` | `hermes_commandbox` | Validate, UPDATE, regen + reload |
| `config/hermes/var/www/html/admin/2/inc/rbl_delete_entry.cfm` | `hermes_commandbox` | DELETE (single or bulk), regen + reload |
| `config/hermes/var/www/html/admin/2/inc/rbl_test_entry.cfm` | `hermes_commandbox` | Two-stage DNS probe via `docker exec` into the postfix container |
| `config/hermes/var/www/html/admin/2/inc/generate_postfix_configuration.cfm` | `hermes_commandbox` | Rebuilds `main.cf` from `parameters` and reloads Postfix |
| `parameters` table (rows under parent `postscreen_dnsbl_sites`) | `hermes_db_server` (`hermes` DB) | Source of truth |
| `hermes_postfix_dkim` container | — | Runs `dig` for the live tests and `postscreen` for the real DNSBL traffic |
| `hermes_unbound` container | — | The recursive resolver every `dig` (test) and every `postscreen` (live) query flows through |

## Future work

A scheduled RBL health checker that runs the per-entry test on a
timer and emails the admin when a zone goes dark — including
auto-disable of consistently-failing entries — is planned (tracked
on the GitHub issue tracker). Until that ships, the **Test All**
button on this page is the manual equivalent; it triggers every
per-row test in parallel and refreshes the Status column in place.

## Related

- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — postscreen knobs and the
  **DNSBL Threshold** the weights here are compared against
- [Network Block/Allow](https://docs.deeztek.com/books/administrator-guide/page/network-blockallow) — the
  `postscreen_access.cidr` table that runs **before** any DNSBL
  lookup; an entry there can short-circuit an IP and skip RBL
  scoring entirely
- [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) — per-address
  override applied later in the pipeline
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — message-content
  scoring that runs after a connection clears the perimeter
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — `hermes_unbound`
  serves every DNSBL query; recursive vs. forwarding mode is the
  single biggest knob that affects whether DNSBL lookups succeed at
  all
- [Relay Networks](https://docs.deeztek.com/books/administrator-guide/page/relay-networks) — local
  trusted networks where `permit_mynetworks` rescues a connection
  before postscreen scoring applies
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — content-time chain validation;
  unrelated to perimeter scoring but a sibling Content Checks page

# Score Overrides

# Score Overrides

Admin path: **Content Checks > Score Overrides**
(`view_score_overrides.cfm`,
`inc/update_spamassassin_config_files.cfm`,
`inc/update_amavis_config_files.cfm`,
`inc/restart_spamassassin.cfm`,
`inc/restart_amavis.cfm`).

This page tunes the **per-rule scores** that SpamAssassin contributes
to each message's total. SpamAssassin ships with thousands of named
rules; each rule that matches a message adds (or subtracts) a
default score, and the message is tagged or quarantined when the
running total crosses the global threshold configured on
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings). Score Overrides is where
the operator says "this rule should weigh more / less / not at all
for our mail." The threshold itself is **not** changed here.

Every entry written on this page lands in SpamAssassin's `local.cf`
as a `score <RULE_NAME> <value>` line. SpamAssassin reads `local.cf`
on daemon start, and the override takes precedence over the
shipped default the rule was defined with.

## Where Score Overrides sits

```
                       +---------------------------------------+
   inbound msg ------->|  Amavis content-filter pass           |
                       |   - ClamAV (virus verdict pre-empts)  |
                       |   - SpamAssassin SCORING              |
                       |        rule_A 0.3                     |
                       |        rule_B 1.2                     |
                       |        rule_C 4.0     <-- per-rule    |
                       |        ...                weights set |
                       |        SUM = N                here    |
                       +---------------------------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  Anti-Spam Settings thresholds        |
                       |   sa_tag_level                        |
                       |   sa_tag2_level    <-- cutoff points  |
                       |   sa_kill_level         set there     |
                       +---------------------------------------+
```

Score Overrides tunes the **contributions**; Anti-Spam Settings
tunes the **cutoffs**. A message reaches `quarantine` because the
sum of contributions crosses the cutoff — moving either side of
that equation changes behavior, and they are independent knobs.

## What an override actually changes

| Override value | Effect on the rule | Use it when |
| --- | --- | --- |
| Positive (e.g. `3.5`) | Adds more to the spam score on match | A rule catches a genuine pattern your senders see often but the default score is too low to flag |
| `0` | Rule still runs but contributes nothing | A rule produces too many false positives in your mail mix and you want to neuter it without ripping it out of the database |
| Negative (e.g. `-2.0`) | **Subtracts** from the spam score on match | The rule indicates legitimacy in your environment (e.g. a trusted-relay heuristic) and you want it to act as a bonus |

Setting a score to `0` is the safe equivalent of "disable this rule"
— SpamAssassin still evaluates it (so the test name still appears
in `X-Spam-Status` and you can confirm it fired), but the message
total is unaffected. Removing the override does not delete the
underlying SpamAssassin rule; it only stops Hermes's `local.cf`
from overriding the shipped default.

## The page

A collapsible scoring helper (the same text the operator gets in the
in-page guide), a hard-locked "DKIM and SPF rules are not evaluated"
warning, an Add Override modal, a DataTable of current overrides, and
an Edit / Delete modal pair.

### Add Override modal

| Field | Stored as | Notes |
| --- | --- | --- |
| Test Name | `spam_settings.parameter` | The SpamAssassin rule name, uppercase with underscores (e.g. `BAYES_99`, `HTML_MESSAGE`, `FREEMAIL_FROM`) |
| Score | `spam_settings.value` | Numeric, validated `-999 <= value <= 999`. Set to `0` to neuter the rule |
| Description | `spam_settings.description` | Free-text label that surfaces in the DataTable; optional |

Add validates: Test Name non-blank, Score numeric and in range, the
`(parameter)` natural key not already present, and the rule name
not in the SPF / DKIM / ADSP plugin family (see warning below). On
success: INSERT row with `spamfilter='1'`, `active='1'`,
`applied='1'`; then immediately regenerate `local.cf` and
reload the engine — same chain Save uses.

### Score Overrides DataTable

| Column | Source |
| --- | --- |
| (checkbox) | Selection for bulk Delete Selected |
| Test Name | `spam_settings.parameter` |
| Score | `spam_settings.value` |
| Description | `spam_settings.description` |
| Edit | Per-row pencil button -> Edit modal |

System-managed rows (`system_managed = 1`) get a lock icon instead
of a checkbox, a "System-managed" badge next to the test name, and
a disabled Edit button. They are filtered out of any DELETE
generated by the page even if a forged POST targets them
(`AND system_managed = 0` is part of the delete query). The lock
exists for rules that encode a Hermes architectural decision — for
example, the per-rule scores Hermes maintains for the trusted-relay
Return Path lookups.

### Edit Modal

Test Name is read-only — changing it is semantically a different
rule and would orphan the override. Only Score and Description are
editable. Save runs the same regen + reload chain as Add.

## DKIM / SPF / ADSP overrides are silently meaningless

The page mounts a warning callout flagging that **any override
targeting a DKIM, SPF, or ADSP rule has no effect** in Hermes, and
the Add handler rejects them with alert `m = 13`. The rule families
covered:

- `DKIM_*` (e.g. `DKIM_INVALID`, `DKIM_VALID`, `DKIM_ADSP_ALL`)
- `SPF_*` (e.g. `SPF_PASS`, `SPF_FAIL`, `SPF_HELO_SOFTFAIL`)
- Any rule whose name contains `ADSP`

The SpamAssassin DKIM and SPF plugins are intentionally not loaded
in Hermes's `init.pre` — the authoritative DKIM verdict is the
`Authentication-Results:` header that OpenDKIM writes at `:25`, and
the authoritative SPF verdict is the `Received-SPF:` header that
`postfix-policyd-spf-python` writes at envelope time. SpamAssassin's
in-content re-check would otherwise produce false-positive failures
against Hermes-modified bodies (External Sender Banner, disclaimer,
signature insertion) and could pick up the wrong upstream IP from
the Received chain in multi-hop scenarios (federal mail, M365 GOV
cloud, etc.). Letting an operator write an override for a rule that
literally cannot fire would silently mislead them, so the guard
runs at the Add handler.

The block is case-insensitive (`UCase` + `Left` / `FindNoCase`) so
mixed-case rule names cannot sidestep it.

## Save and apply flow

```
1. View page submits action="add" | "edit" | "delete"
2. view_score_overrides.cfm validates the row (per-action rules above)
3. INSERT / UPDATE / DELETE on spam_settings (spamfilter='1'),
   guarded by system_managed=0 on UPDATE and DELETE
4. update_spamassassin_config_files.cfm:
     a. Read /opt/hermes/conf_files/local.cf.HERMES (template)
     b. Substitute USE-BAYES, USE-DCC, USE-PYZOR, USE-RAZOR2, and
        bayes_auto_learn placeholders from their own spam_settings rows
     c. SELECT every spamfilter='1' active='1' row -> tmp/_sa_tests file:
          score <parameter> <value>
          (one line per row)
     d. Substitute the #CUSTOM-TESTS placeholder in local.cf with the
        rendered score list
     e. Render Message Rules into the #CUSTOM-MESSAGE-RULES placeholder
     f. Back up /etc/spamassassin/local.cf -> local.cf.HERMES.BACKUP,
        move the rendered file into place
     g. UPDATE spam_settings SET applied='1' WHERE applied='2'
5. update_amavis_config_files.cfm:
     - Regenerate Amavis 50-user from template (subject tags, destinies,
       DKIM-verification toggle, file rules) so a SA setting change that
       also affects Amavis takes effect in the same write
6. restart_spamassassin.cfm:
     - docker exec hermes_mail_filter /usr/bin/spamassassin --lint
       (validation; abort on failure)
     - Then docker container restart hermes_mail_filter
7. restart_amavis.cfm: same docker container restart hermes_mail_filter
   (idempotent; the engine is back from step 6)
8. session.m = 1 / 7 / 8 -> success alert with "regenerated" wording
```

The restart in step 6 is a full container restart — `hermes_mail_filter`
runs SpamAssassin, ClamAV, Amavis, and Fangfrisch, all of which
re-initialize together. Inbound mail held in Postfix's queue during
the restart is retried on the next queue run; no message is lost.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` | Add succeeded and SpamAssassin reloaded |
| `m = 2` | Test Name blank |
| `m = 3` | Test Name already exists |
| `m = 4` | Score out of `-999..999` range |
| `m = 5` | Score blank |
| `m = 6` | Score not numeric |
| `m = 7` | Edit succeeded and SpamAssassin reloaded |
| `m = 8` | Delete succeeded and SpamAssassin reloaded |
| `m = 10` | Delete clicked with no rows selected |
| `m = 11` | The Apply chain (regen + restart) threw — DB write may already have happened |
| `m = 12` | Attempt to edit or delete a `system_managed = 1` row (forged POST defense; the UI hides the action) |
| `m = 13` | Add of a DKIM / SPF / ADSP family rule — rejected because the underlying plugin is disabled |

`m = 11` is the partial-failure case: the DB row has already been
inserted / updated / deleted but `local.cf` regen or the lint /
restart step failed. The page does not roll back the DB write —
the next successful save will re-render `local.cf` from the current
table state, so the system is self-healing on the next click.

## Finding rule names

The page guide gives the lookup steps that work for any received
message:

1. From [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history), open any message
   and view headers; the `X-Spam-Status:` header lists every rule
   that fired and its score
2. SpamAssassin rule names are uppercase with underscores
   (e.g. `BAYES_99`, `HTML_MESSAGE`, `FREEMAIL_FROM`,
   `RDNS_NONE`, `URIBL_BLOCKED`)
3. To see the default score and description for a rule:
   `docker exec hermes_mail_filter spamassassin --debug rules 2>&1 | grep -i <RULE_NAME>`

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_score_overrides.cfm` | `hermes_commandbox` | The page (validation + alerts + DataTable) |
| `config/hermes/var/www/html/admin/2/inc/update_spamassassin_config_files.cfm` | `hermes_commandbox` | Renders `local.cf` from template + score rows + message rules |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Re-renders Amavis `50-user` (called in the same chain to keep SA-related Amavis flags in sync) |
| `config/hermes/var/www/html/admin/2/inc/restart_spamassassin.cfm` | `hermes_commandbox` | Lints the new `local.cf` then restarts `hermes_mail_filter` |
| `config/hermes/var/www/html/admin/2/inc/restart_amavis.cfm` | `hermes_commandbox` | Calls `restart_mail_filter.cfm` |
| `config/hermes/opt/hermes/conf_files/local.cf.HERMES` | `hermes_commandbox` (read) -> `hermes_mail_filter` (live `/etc/spamassassin/local.cf`) | Canonical template with `##CUSTOM-TESTS` and `##CUSTOM-MESSAGE-RULES` placeholders |
| `/etc/spamassassin/local.cf` | `hermes_mail_filter` | Live file SpamAssassin reads at engine start |
| `/etc/spamassassin/local.cf.HERMES.BACKUP` | `hermes_mail_filter` | Pre-write backup taken every save |
| `spam_settings` table, `spamfilter = '1'` | `hermes_db_server` (`hermes` DB) | Source of truth for every override (and for the Bayes / DCC / Razor / Pyzor / threshold values used by Anti-Spam Settings) |
| `hermes_mail_filter` container | — | Hosts SpamAssassin, ClamAV, Amavis, Fangfrisch — restarted as a unit on every save |

## Related

- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — sets the GLOBAL spam
  thresholds (`sa_tag_level`, `sa_tag2_level`, `sa_kill_level`); the
  cutoffs that the per-rule contributions tuned here are summed
  against
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) — custom SpamAssassin rules
  (header / body / regex) written into the same `local.cf` via the
  `##CUSTOM-MESSAGE-RULES` placeholder during the same regen cycle
- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — ClamAV runs in the
  same Amavis pass; a virus verdict pre-empts any spam-score result
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — SMTP-time rejects that
  fire before SpamAssassin ever sees the message
- [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) / [File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions)
  / [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules) — Amavis attachment filtering that
  runs alongside SpamAssassin scoring in the same pass
- [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) /
  [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — every rule in the DKIM / SPF
  family is the authoritative verifier whose verdict the warning
  callout refers back to
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) — Bayes
  auto-learn and signature refresh cadence are scheduled here, not
  on the Score Overrides page
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — every rule fire and
  its score appears in `mail.log` under the `amavis[...]:` lines,
  prefixed `tests=...`

# Sender/Recipient Rules

# Sender/Recipient Rules

Admin path: **Content Checks > Sender/Recipient Rules**
(`view_sender_recipient_block_allow.cfm`,
`inc/get_sender_recipient_block_allow.cfm`, `inc/sender_add_entry.cfm`,
`inc/sender_edit_entry.cfm`, `inc/sender_delete_entry.cfm`).

This page manages **per-recipient envelope-sender filters** — pairs of
(sender, recipient) that Amavis honors when it scores an inbound
message. Each row says "when this sender writes to this recipient,
apply this rule" — `ALLOW` (skip spam scoring) or `BLOCK`
(quarantine / reject). The rules live in Amavis's native `wblist`
table and are read live on every message, so saves take effect on the
next inbound delivery with no service reload.

This is the **envelope-level** half of the inbound-control story.
Pairs with [Network Block/Allow](https://docs.deeztek.com/books/administrator-guide/page/network-blockallow), which is the
IP-level half evaluated much earlier in the SMTP pipeline.

## Where this list sits in the flow

```
+---------------------------+
|  Inbound TCP / SMTP       |
+-------------+-------------+
              |
              v
+-------------------------------------------------+
|  postscreen / smtpd  (postfix perimeter checks) |
|  - Network Block/Allow  (CIDR)                  |
|  - RBL / DNSBL                                  |
|  - SPF / sender hostname / recipient domain     |
+-------------+-----------------------------------+
              | DATA accepted
              v
+-------------------------------------------------+
|  amavis :10024  (hermes_mail_filter)            |
|                                                 |
|  Per-recipient lookup:                          |
|  $sql_select_white_black_list                   |
|    SELECT wb FROM wblist, mailaddr, recipients  |
|    WHERE recipients.id = wblist.rid             |
|      AND mailaddr.id   = wblist.sid             |
|      AND mailaddr.email IN (%k)                 |
|                                                 |
|  -> wb = 'W'  -> SKIP spam scoring              |
|                 (viruses + banned files +       |
|                  bad headers STILL apply)       |
|  -> wb = 'B'  -> mark as spam / quarantine      |
|  -> no row    -> normal scoring path            |
+-------------------------------------------------+
```

The lookup is keyed on the envelope-sender address (`mailaddr.email`)
**after** Amavis has already accepted the message from Postfix and
started its scoring pass. That is the central operational fact:
this page does not stop mail at SMTP time — it only changes how
Amavis treats it once received.

## Distinction from sibling pages

Three pages share overlapping vocabulary; they apply at three
different points in the pipeline.

| Page | Layer | Match key | Effect |
| --- | --- | --- | --- |
| [Network Block/Allow](https://docs.deeztek.com/books/administrator-guide/page/network-blockallow) | `postscreen` (TCP / pre-SMTP) | Source IP / CIDR | 550 or RBL bypass; **no content-layer effect** |
| [Global Sender Rules](https://docs.deeztek.com/books/administrator-guide/page/global-sender-rules) | Amavis (per-message) | Envelope sender only | Allow / block from this sender to **every recipient** on the system |
| **Sender/Recipient Rules** (this page) | Amavis (per-message) | Envelope sender **and** specific recipient | Allow / block from this sender to **one recipient** (or one recipient-domain) |

Order of precedence within Amavis: a Global Sender Rules entry takes
precedence over a per-recipient entry on this page — the in-page
callout on Global Sender Rules states this explicitly. Use this page
when the policy needs to be scoped to a specific person or mailbox;
use Global Sender Rules only when the policy must apply to everyone.

## `ALLOW` does not bypass virus, banned files, or bad headers

The in-page callout makes this explicit:

> Allow entries only bypass Spam checks. Emails with Viruses, Banned
> Files, and Bad Headers will still be blocked.

That is a property of Amavis itself — `wb='W'` in the `wblist` table
short-circuits the SpamAssassin score path but does not exempt the
message from virus scanning (ClamAV), banned-file extension rules
(`@banned_filename_re`), or RFC-violation header checks. The
operational consequence is that an `ALLOW` here is much narrower than
the `permit` action on Network Block/Allow — there, RBL is skipped
and the message enters Amavis on the same path as any other; here,
only the spam-score gate is removed.

## Sender match formats

The sender field accepts three formats, all distinguished by the
position of `@`:

| What you type | Stored as | Matches |
| --- | --- | --- |
| `user@example.com` | `user@example.com` | A single full envelope-sender address |
| `example.com` | `@example.com` | Any envelope sender on `example.com` (the bare domain — exact match, **no** subdomains) |
| `.example.com` | `@.example.com` | `example.com` **and** any subdomain (`mail.example.com`, `sub.sub.example.com`, …) |

The page accepts the bare domain form for convenience and rewrites it
with the leading `@` before the `mailaddr` lookup. The leading-dot
form is preserved as-is and stored as `@.example.com` — Amavis itself
interprets the dot as the wildcard.

## Recipient match formats

The recipient field is constrained to recipients already known to the
system. It autocompletes from the `recipients` table via a `<datalist>`
populated on page render. Two forms work:

| What you type | What the lookup does | Effect |
| --- | --- | --- |
| `user@example.com` | Matches a single row in `recipients` | One `wblist` row inserted (one rid) |
| `@example.com` | Matches a domain-level row in `recipients` (where `domain='1'`); the handler then enumerates every individual recipient under that domain | One `wblist` row per recipient in the domain — the rule fans out |

If the typed recipient does not exist anywhere in `recipients`, the
save fails with `session.m = 34` ("specified recipient was not found
in the system"). The page does not create recipients on the fly —
add the recipient on [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients)
or as a Mailbox first.

## Same-domain sender / recipient is rejected

A guard rejects entries where the sender domain and recipient domain
are the same (`session.m = 35`). Inbound mail from `user@example.com`
to `boss@example.com` is normally outbound or internal, not the
inbound-filtering case this page is designed for, and an `ALLOW`
across that boundary would be a routine misconfiguration.

## The two cards on the page

### 1. Add Sender/Recipient Entry

Four inputs across one form: **Sender Email or Domain**,
**Recipient** (autocomplete from `recipients`), **Action**
(BLOCK / ALLOW radios), and submit. Validation order on submit:

1. Sender non-empty (`session.m = 30` on fail).
2. Recipient non-empty (`session.m = 31`).
3. Action is BLOCK or ALLOW (`session.m = 32`).
4. Sender is a syntactically valid email *or* a syntactically valid
   domain — checked by `IsValid("email", ...)` against a stub address
   (`session.m = 33`).
5. Recipient resolves to a row in `recipients` (`session.m = 34`).
6. Sender domain != recipient domain (`session.m = 35`).
7. Sender+recipient pair is not already in `wblist` (`session.m = 36`,
   "already exists or already staged for addition").

On success, the handler:

1. Resolves or creates the `mailaddr` row for the sender (one row per
   distinct address — `mailaddr` is shared with the rest of the
   Amavis stack).
2. Inserts the `wblist` row(s):
   - Specific recipient: one row.
   - Domain-wide recipient: one row per individual recipient in that
     domain (the rule fans out at insert time, not at lookup time).
3. Sets `wb = 'W'` (ALLOW) or `wb = 'B'` (BLOCK).

There is no Postfix or Amavis reload — Amavis reads `wblist` live on
every message via its SQL backend.

### 2. Sender/Recipient Entries (DataTable)

Searchable, sortable, paginated; bulk-delete checkboxes; per-row
Edit / Delete buttons.

| Column | Source |
| --- | --- |
| Sender | `mailaddr.email` joined via `wblist.sid` |
| Recipient | `recipients.recipient` joined via `wblist.rid` |
| Type | `wblist.wb` rendered as green "Allow" or red "Block" badge |
| Actions | Edit (modal), Delete (confirm) |

Each row's checkbox value is a composite `rid:sid` (the `wblist`
table's natural primary key — no surrogate `id` column). The bulk
delete handler splits each entry on `:` and deletes the matching
`wblist` row directly.

The **Edit** modal keeps the recipient read-only (with the inline
note "Recipient cannot be changed. Delete and re-add if needed") —
changing the recipient would change `rid`, which is the row's
identity. The sender and the BLOCK/ALLOW type are editable; the save
handler deletes the original row and inserts a new one, using the
sender email strings to find the old row (no integer ID is needed
from the form).

## Save flow

```
Add / Edit / Delete
    |
    v
INSERT / UPDATE / DELETE on wblist (and mailaddr for new senders)
  All queries datasource = "hermes"
    |
    v
(Delete only) Garbage-collect orphaned mailaddr rows:
  DELETE FROM mailaddr WHERE id NOT IN (SELECT DISTINCT sid FROM wblist)
    |
    v
session.m = 1 / 2 / 5  (Added / Deleted / Updated)
On validation failure -> session.m = 30..36
```

No file write, no `postmap`, no service reload. Amavis picks the new
rules up on the next message.

## Tables involved

| Table | Role | Engine |
| --- | --- | --- |
| `wblist` | (`rid`, `sid`, `wb`) composite-key per-pair rule | MyISAM, utf8mb3 |
| `mailaddr` | Distinct envelope-sender addresses; unique key on `email` | MyISAM, utf8mb3 |
| `recipients` | Resolved at lookup time to find `rid`; populated from the rest of the system (Mailboxes, Relay Recipients, domain-level entries) | MyISAM |

`wblist` and `mailaddr` are Amavis's own native tables — Hermes
pre-creates them in `hermes_install.sql` because Amavis would
otherwise lazily create them on its first SQL-backend write, after
the CFML pages that reference them have already started to render.

The composite key `(rid, sid)` is enforced at the database layer, so
the page's duplicate guard (`session.m = 36`) and the database itself
will both refuse a true duplicate. `mailaddr` carries a `UNIQUE KEY`
on `email`, so concurrent sender adds cannot create duplicate rows
even mid-race.

## Relationship to user-portal sender filters

End users in the recipients table see and manage their **own** subset
of `wblist` rules from the user portal (`/users/2/`) — the "Allow
this sender" and "Block this sender" buttons on a quarantined
message, plus the explicit Sender Filters page, both write rows into
the same `wblist` table with the user's own recipient `id` as `rid`.

This admin page sees those user-trained rules in the same table —
they are not flagged separately in the UI. Operators editing or
deleting from this page can affect user-trained rules; that is by
design (this page is the operator's view of the entire `wblist`
table).

## Failure semantics

| Failure | `session.m` | Behavior |
| --- | --- | --- |
| Empty sender | `30` | Redirect, no DB write |
| Empty recipient | `31` | Redirect, no DB write |
| Invalid action (neither BLOCK nor ALLOW) | `32` | Redirect, no DB write |
| Sender not a valid email or domain | `33` | Redirect, no DB write |
| Recipient not found in `recipients` | `34` | Redirect, no DB write |
| Same sender and recipient domain | `35` | Redirect, no DB write |
| Pair already in `wblist` | `36` | Redirect, no DB write |

There is no equivalent of `session.m = 4` ("Configuration Error") on
this page — there is no Postfix / Amavis regen step that could fail.
A SQL error would surface as an uncaught `cfcatch` and the standard
500-error page, not a friendly alert.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_sender_recipient_block_allow.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/get_sender_recipient_block_allow.cfm` | `hermes_commandbox` | Joins `wblist` + `mailaddr` + `recipients` for the table |
| `config/hermes/var/www/html/admin/2/inc/sender_add_entry.cfm` | `hermes_commandbox` | Validate, resolve/insert `mailaddr`, INSERT `wblist` (fans out for domain recipients) |
| `config/hermes/var/www/html/admin/2/inc/sender_edit_entry.cfm` | `hermes_commandbox` | DELETE original row by email-join, INSERT new row, garbage-collect orphan `mailaddr` |
| `config/hermes/var/www/html/admin/2/inc/sender_delete_entry.cfm` | `hermes_commandbox` | DELETE single or bulk by `rid+sid`, garbage-collect orphan `mailaddr` |
| `wblist`, `mailaddr`, `recipients` tables | `hermes_db_server` (`hermes` DB) | Source of truth |
| `hermes_mail_filter` container (Amavis) | — | Consumes the rules live via `$sql_select_white_black_list` on every inbound message |

## Related

- [Network Block/Allow](https://docs.deeztek.com/books/administrator-guide/page/network-blockallow) — IP-level
  (`postscreen`) sibling; runs before any SMTP handshake, much
  earlier than this page in the pipeline
- [Global Sender Rules](https://docs.deeztek.com/books/administrator-guide/page/global-sender-rules) — envelope-sender
  block/allow with no recipient scope; takes precedence over this
  page's per-pair rules
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — the scoring path that
  an `ALLOW` here short-circuits
- [Anti-Virus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — runs even when this
  page sets `ALLOW`; the "bypass spam only" caveat exists because
  virus scanning is non-bypassable
- [BCC Maps](https://docs.deeztek.com/books/administrator-guide/page/bcc-maps) — sibling Content Checks page; takes the
  same per-recipient routing approach for a different purpose
  (silent copies vs. block/allow)
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — the SMTP-time checks that
  run **before** Amavis ever sees the message
- [Relay Recipients](https://docs.deeztek.com/books/administrator-guide/page/relay-recipients) — the
  recipient list this page's autocomplete draws from; an entry here
  presupposes a row there (or in Mailboxes)
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) — where the effect of
  ALLOW / BLOCK decisions on this page shows up after delivery
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — Amavis logs each
  `wblist` lookup result; the wb value (`W` / `B`) is visible in the
  per-message scoring trace

# SPF Settings

# SPF Settings

Admin path: **Content Checks > SPF Settings**
(`view_spf_settings.cfm`, `inc/get_spf_settings.cfm`,
`inc/spf_save_settings.cfm`, `inc/spf_generate_config_file.cfm`,
`inc/spf_add_whitelist.cfm`, `inc/spf_edit_whitelist.cfm`,
`inc/spf_delete_whitelist.cfm`,
`inc/generate_postfix_configuration.cfm`).

This page controls **inbound** SPF policy enforcement. SPF
([RFC 7208](https://datatracker.ietf.org/doc/html/rfc7208)) lets the
owner of a domain publish, in DNS, the list of IP addresses authorized
to send mail using that domain in the envelope `MAIL FROM` (and
optionally the SMTP `HELO`). When Postfix accepts a connection,
Hermes consults the published record for the connecting client and
decides whether to accept, defer, or reject the message based on the
result.

Hermes is responsible only for the **verification** side. Publishing
your own organization's SPF record (the `v=spf1 ...` TXT record at
your sending domain) is a one-time DNS operation done at your
authoritative DNS host — it is not managed from this page.

## Where SPF sits in the flow

```
+----------------------+
| Remote SMTP peer     |
+----------+-----------+
           |
           v
+----------+--------------------------------+
| smtpd :25 (hermes_postfix_dkim)            |
|   smtpd_recipient_restrictions = ...,      |
|     check_policy_service unix:private/     |
|       policy-spf                           |
|       |                                    |
|       v                                    |
|   Postfix spawns policyd-spf (python)      |
|   from master.cf "policy-spf unix" entry   |
|   - reads /etc/postfix-policyd-spf-python/ |
|       policyd-spf.conf                     |
|   - queries DNS for the sender's SPF TXT   |
|   - returns Pass / Fail / Softfail /       |
|     Neutral / None / TempError / PermError |
|   - returns Postfix action verb            |
|     (DUNNO / REJECT / DEFER_IF_REJECT)     |
+----------+--------------------------------+
           |
           v
+----------+--------------------------------+
| OpenDKIM milter :8891 (DKIM verify)        |
| OpenDMARC milter :54321 (DMARC eval)       |
+----------+--------------------------------+
           |
           v
   Amavis / SpamAssassin / ClamAV
```

The policy daemon is a **Postfix policy delegate** — a separate
process that Postfix spawns from `master.cf`:

```
policy-spf  unix  -  n  n  -  -  spawn
            user=nobody argv=/usr/bin/policyd-spf
```

`smtpd_recipient_restrictions` invokes it via
`check_policy_service unix:private/policy-spf`. The daemon's
configuration file at
`/etc/postfix-policyd-spf-python/policyd-spf.conf` is what this admin
page writes; the entire file is regenerated on every save from the
template at `/opt/hermes/templates/policyd-spf.conf.HERMES`.

## SPF result classes and their typical meaning

| Result | Meaning | Default Hermes behavior |
| --- | --- | --- |
| `Pass` | Connecting IP is in the published `v=spf1` record | Accept |
| `Fail` | Sender has published `-all`; this IP is explicitly disallowed | Reject |
| `SoftFail` | Sender has published `~all`; this IP is not authorized but the owner is in monitoring mode | **Reject** (Hermes recommended) — see Operational consequence below |
| `Neutral` | Sender published `?all`; owner expresses no opinion | Accept (treated as `None`) |
| `None` | No SPF record exists for the sender | Accept |
| `TempError` | DNS timeout / SERVFAIL during the lookup | Accept (treat as no record) — operator can switch to defer |
| `PermError` | SPF record is malformed or exceeds the 10-DNS-lookup limit | Accept (treat as no record) — operator can switch to reject |

SPF is **checked twice per message** by the daemon: once against the
SMTP `HELO` identity (before `MAIL FROM`), and once against the
envelope sender domain after `MAIL FROM`. Each check has its own
rejection policy on this page.

## The two cards on the page

### 1. SPF Settings (master toggle + policy daemon controls)

The master **SPF Enabled** dropdown flips a single child row in the
`parameters` table — the row whose `parameter` value is
`check_policy_service unix:private/policy-spf` under the
`smtpd_recipient_restrictions` parent. When SPF is disabled the page
also forces DMARC off (DMARC requires both an SPF and a DKIM result;
without SPF the DMARC milter has nothing to align against). The
in-page callout warns about this dependency.

When SPF is enabled, the policy section exposes six controls, each
written to a `parameters2` row in the `dkim`/`spf` module rows:

| Control | `policyd-spf.conf` directive | Effect |
| --- | --- | --- |
| Logging Level | `debugLevel` | `0`–`4` verbosity; `-1` disables logging. Higher levels log every DNS lookup and the full SMTP envelope data — useful for diagnosing federal / M365 GOV / Proofpoint Government chain issues |
| Test Mode | `TestOnly` | `1` adds the SPF result to message headers but **never** rejects, regardless of the rejection policies below. Use to evaluate impact before enforcing |
| HELO Check Rejection Policy | `HELO_reject` | What to do with the SPF result for the SMTP `HELO`/`EHLO` identity. Options: `Fail`, `SPF_Not_Pass` (Reject All), `Softfail` (Recommended), `Null` (reject HELO of null-sender bounces only), `False` (header only), `No_Check` |
| Mail From Check Rejection Policy | `Mail_From_reject` | Same option set, but applied to the envelope `MAIL FROM` domain |
| Permanent Error Policy | `PermError_reject` | `True` rejects when the published SPF record is broken; `False` (recommended) treats it as no record |
| Temporary Error Policy | `TempError_Defer` | `True` issues a 4xx defer on DNS timeout; `False` (recommended) accepts and continues |

> **Operational consequence — single point of SPF truth.** The Hermes
> baseline disables SpamAssassin's redundant SPF re-check. SA's
> in-process SPF scoring runs after Amavis has reinjected the message
> over a local hop, so SA sees an IP path that does not include the
> original sender — on government/M365 GOV/Proofpoint Government mail
> the wrong IP gets scored, producing false-positive `SPF_SOFTFAIL`
> hits. The policy daemon on this page is the **single authoritative
> SPF verifier**; it sees the real connecting client IP. To preserve
> the spam-coverage SA's `SPF_SOFTFAIL` rule provided, set both HELO
> and Mail From Check Rejection Policy to **Reject SoftFail**. This
> is the in-page recommendation and the shipped baseline.

### 2. SPF Whitelist Entries

Per-row bypass list written to four `Whitelist` directives in
`policyd-spf.conf`:

| Entry type | `policyd-spf.conf` directive | What it matches | Typical use |
| --- | --- | --- | --- |
| IP / Network Address | `Whitelist` | The connecting client IP (single address or CIDR) | Trusted secondary MX, known forwarders, partner relays |
| HELO/EHLO Host Name | `HELO_Whitelist` | The hostname announced in `HELO`/`EHLO`. Daemon DNS-checks the connecting IP against an A/AAAA for that name to prevent forgery | Mailing-list providers that consistently HELO with their own domain |
| Domain Name | `Domain_Whitelist` | The envelope `MAIL FROM` domain | Senders with broken `~all` records whose mail you still need to receive |
| PTR Domain | `Domain_Whitelist_PTR` | The reverse-DNS (PTR) domain of the connecting IP | Hosts whose forward DNS is unstable but whose reverse DNS is well-controlled |

Entries are stored in the `spf_bypass` table
(`entry`, `entry_type`, `entry_note`). The save handler joins all
enabled rows of each type with commas and substitutes them into the
template at `IP-NETWORK-WHITELIST`, `HELO-WHITELIST`,
`DOMAIN-WHITELIST`, `PTR-WHITELIST` placeholders.

A whitelist hit completely **skips** SPF evaluation for that
connection — the daemon returns `Pass` without consulting DNS. Use
IP-based whitelisting when possible; HELO / Domain / PTR entries
incur extra DNS lookups per message.

The DataTable supports add (textarea — one entry per line, validated
and deduplicated), inline edit modal, single delete, and bulk delete
via checkbox selection.

## What this page does NOT control

- **Per-sender allow/block.** Address-level rules live on
  [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) and apply later
  in the pipeline.
- **The SPF record for your own sending domain.** That is a DNS TXT
  record you publish at your authoritative DNS host. A correct
  outbound SPF for a Hermes-served sending domain typically looks
  like `v=spf1 mx ip4:<hermes-egress-ip> include:<isp-relay> ~all` —
  see [Domains (Email Relay)](https://docs.deeztek.com/books/administrator-guide/page/domains) and
  [Domains (Email Server)](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) for the
  egress IP your record needs to authorize.
- **Network-level allow.** Trusted SMTP source ranges (`mynetworks`)
  short-circuit before any policy check via
  [Relay Networks](https://docs.deeztek.com/books/administrator-guide/page/relay-networks).

## Save flow

```
1. Validate form fields exist when SPF is being enabled
   - Missing fields -> session.m = 20, redirect, no DB write
2. UPDATE parameters child row for SPF on/off
3. UPDATE parameters2 rows for the six policy daemon directives
4. cfinclude spf_generate_config_file.cfm
     a. Read /opt/hermes/templates/policyd-spf.conf.HERMES
     b. REReplace placeholders (DEBUG-LEVEL, TEST-ONLY, HELO-REJECT,
        MAIL-FROM-REJECT, PERMERROR-REJECT, TEMPERROR-REJECT)
     c. SELECT all enabled spf_bypass rows by entry_type, comma-join,
        substitute *-WHITELIST placeholders
     d. Backup current /etc/postfix-policyd-spf-python/policyd-spf.conf
        as policyd-spf.conf.HERMES
     e. Move generated tmp file into place
5. cfinclude generate_postfix_configuration.cfm
     - Regenerates main.cf so smtpd_recipient_restrictions reflects
       SPF on/off
     - Reloads Postfix inside hermes_postfix_dkim
6. If SPF was DISABLED: also disable the OpenDMARC milter rows,
   clear FailureReports, deactivate the DMARC report Ofelia job,
   regenerate opendmarc.conf, restart OpenDMARC
7. session.m = 9 -> green "SPF settings saved" alert on redirect
```

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_spf_settings.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/get_spf_settings.cfm` | `hermes_commandbox` | Loads current parameters / parameters2 / spf_bypass values |
| `config/hermes/var/www/html/admin/2/inc/spf_save_settings.cfm` | `hermes_commandbox` | Validates form, updates rows, calls config + Postfix regen; disables DMARC if SPF off |
| `config/hermes/var/www/html/admin/2/inc/spf_generate_config_file.cfm` | `hermes_commandbox` | Renders `policyd-spf.conf` from the template + DB |
| `config/hermes/opt/hermes/templates/policyd-spf.conf.HERMES` | `hermes_commandbox` (read) → `hermes_postfix_dkim` (live `/etc/postfix-policyd-spf-python/policyd-spf.conf`) | Canonical template with `DEBUG-LEVEL`, `TEST-ONLY`, etc. placeholders |
| `parameters` table (`check_policy_service unix:private/policy-spf` row) | `hermes_db_server` (`hermes` DB) | SPF on/off |
| `parameters2` table (rows where `module='spf'`) | `hermes_db_server` (`hermes` DB) | The six daemon settings |
| `spf_bypass` table | `hermes_db_server` (`hermes` DB) | Whitelist entries |
| `hermes_postfix_dkim` container | — | Runs `smtpd`, spawns `policyd-spf`, hosts the live `policyd-spf.conf` |
| `hermes_unbound` container | — | Resolves every SPF DNS query the daemon makes |

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Missing form fields when enabling SPF | `session.m = 20`, redirect, no DB write |
| `spf_generate_config_file.cfm` throws (template missing, write fails, etc.) | Surfaces as a `cfcatch` from the inline include — the save aborts |
| Empty whitelist entry on Add | `session.m = 13`, redirect, no DB write |
| Whitelist entry fails IP / hostname syntax check | `session.m = 17`, redirect, no DB write |
| Duplicate whitelist entry | `session.m = 14`, redirect, no DB write |
| `postfix reload` fails inside the container | Standard `generate_postfix_configuration.cfm` failure path |

## Related

- [DKIM Settings](https://docs.deeztek.com/books/administrator-guide/page/dkim-settings) — the second authentication
  service whose result is consumed by DMARC; paired conceptually with
  SPF as a "DNS-based outbound sender authentication" mechanism
- [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) — the policy layer that
  consumes SPF and DKIM results; disabling SPF here automatically
  disables DMARC
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — chain-of-custody for
  authentication results across forwarders; participates only after
  SPF / DKIM / DMARC have produced their verdicts
- [Trusted ARC Sealers (M365)](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) — for
  M365 customers whose downstream verifiers escalate when SPF fails
  on forwarded mail
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — the rest of the
  `smtpd_recipient_restrictions` chain; the SPF / DKIM / DMARC status
  badges on its fourth card link back to the dedicated pages
- [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) — per-address
  bypass applied after the SPF verdict
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — every SPF lookup
  flows through `hermes_unbound`; resolver mode (recursive vs.
  forwarding through a public provider) directly affects SPF
  reliability and the 10-DNS-lookup limit timing
- [Domains (Email Relay)](https://docs.deeztek.com/books/administrator-guide/page/domains),
  [Domains (Email Server)](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — where the
  egress IP that your authoritative SPF record needs to authorize is
  documented

# SVF Policies

# SVF Policies

Admin path: **Content Checks > SVF Policies**
(`view_svf_policies.cfm`,
`inc/get_svf_policies.cfm`,
`inc/update_amavis_config_files.cfm`,
`inc/restart_amavis.cfm`).

This page manages the **SVF (Spam / Virus / File) policies** that
Amavis applies on a per-recipient basis. Each policy bundles four
groups of decisions -- spam scoring thresholds, a banned-file
ruleset name, four "accept" toggles (deliver instead of quarantine
on virus / spam / banned-file / bad-header), four "bypass" toggles
(skip the corresponding scan entirely), and three recipient
notification toggles. When a message arrives, Amavis looks up the
recipient in the `recipients` table, joins to the `policy` table on
`policy_id`, and uses that policy's row to drive every per-message
decision -- including which [File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules) to enforce
for attachments.

SVF policies are how the gateway expresses "marketing tolerates
more spam than legal does," "abuse@ has to receive raw spam
samples," or "this VIP mailbox skips banned-file checks because
they trade `.iso` images legitimately." The global engine settings
on [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) and the per-rule
weights on [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) decide **how a
message is scored**; the SVF policy assigned to the recipient
decides **what happens to that score**.

## Where SVF Policies sits

```
   incoming msg for                 +--------------------------+
   bob@example.com                  |  Amavis content-filter   |
   ----------------------+--------> |  pass (hermes_mail_filter)|
                         |          |   - ClamAV scan          |
                         |          |   - SpamAssassin scoring |
                         |          |     produces total score |
                         |          |   - banned-file regex set|
                         |          +------------+-------------+
                         |                       |
                         v                       v
   +---------------------+----------+   +-----------------------+
   | $sql_select_policy lookup      |   | resolved per-message: |
   | (in 50-user.HERMES):           |   |   spam_tag2_level     |
   |   SELECT *, recipients.id      |   |   spam_kill_level     |
   |   FROM recipients, policy      +-->|   virus_lover         |
   |   WHERE recipients.policy_id   |   |   spam_lover          |
   |       = policy.id              |   |   banned_files_lover  |
   |   AND recipients.recipient     |   |   bad_header_lover    |
   |       IN (%k)                  |   |   bypass_*_checks     |
   +--------------------------------+   |   banned_rulenames    |
                                        |   warn*recip          |
                                        +-----------+-----------+
                                                    |
                                                    v
                                        +-----------------------+
                                        |  per-recipient verdict|
                                        |  -> deliver / tag /   |
                                        |     quarantine /      |
                                        |     bypass / notify   |
                                        +-----------------------+
```

The recipient lookup is the policy resolver. Every recipient in the
`recipients` table has a `policy_id` pointing at a row in the
`policy` table; the `spam_policies` table is a thin index that
adds `system` / `custom` / `default_policy` flags on top. A
recipient with no matching row falls back to the **default policy**
(`spam_policies.default_policy = '1'`) -- the page enforces that
exactly one default exists at all times.

## What's actually in a policy

The `policy` table is the Amavis-shaped row; only the columns the
UI exposes are documented here (`policy` has additional NULL
columns inherited from Amavis's reference schema that this page
doesn't touch).

| Field | DB column | Effect |
| --- | --- | --- |
| Policy Name | `policy.policy_name` + `spam_policies.policy_name` | Display name; visible in the recipient dropdown on Relay Recipients and Mailbox Recipients. Up to 32 chars; letters, numbers, spaces, underscores, hyphens, `@`, and periods only |
| Spam Tag Score | `policy.spam_tag2_level` | The Amavis `$spam_tag2_level` -- the score at which the spam header is added to the message (e.g. `X-Spam-Status: Yes`). Below this the message is delivered without a spam header. Range `-999 .. 999` |
| Spam Quarantine Score | `policy.spam_kill_level` | The Amavis `$spam_kill_level` -- the score at which the message is quarantined (or bounced, depending on `final_spam_destiny` on Anti-Spam Settings). Below this but above tag, the message is delivered with a spam header. Range `-999 .. 999` |
| File Rule | `policy.banned_rulenames` | The name of a [File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules) (from `file_rule_components.rule_name`) -- Amavis maps this to the `@banned_filename_re` ruleset emitted into `50-user` and applies that ruleset's allow / ban regex to every attachment for this policy's recipients |
| Accept Viruses | `policy.virus_lover` (`Y` / `N`) | When `Y`, virus-flagged messages are **delivered** (with a notation) instead of quarantined. Almost always `N`; exists for forensic mailboxes |
| Accept Spam | `policy.spam_lover` | When `Y`, spam-flagged messages are delivered instead of quarantined. Useful for abuse / postmaster mailboxes that need to see the raw spam |
| Accept Banned Files | `policy.banned_files_lover` | When `Y`, messages with banned attachments are delivered instead of quarantined |
| Accept Bad Headers | `policy.bad_header_lover` | When `Y`, messages with malformed headers (per RFC) are delivered instead of quarantined |
| Bypass Virus Checks | `policy.bypass_virus_checks` | When `Y`, **skip ClamAV entirely** for this policy's recipients. No scan happens; no virus score contributes |
| Bypass Spam Checks | `policy.bypass_spam_checks` | When `Y`, skip SpamAssassin entirely. No score; no rule contributions; no Bayes update |
| Bypass Banned Checks | `policy.bypass_banned_checks` | When `Y`, skip banned-extension matching. Attachments are not screened against any [File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules) |
| Bypass Header Checks | `policy.bypass_header_checks` | When `Y`, skip bad-header detection. Malformed-header messages pass through |
| Notify on Banned File | `policy.warnbannedrecip` | When `Y`, the recipient receives an Amavis notification when a banned-file message is quarantined for them |
| Notify on Virus | `policy.warnvirusrecip` | Same, for virus quarantines |
| Notify on Bad Header | `policy.warnbadhrecip` | Same, for bad-header quarantines |

`policy.spam_modifies_subj` is fixed to `Y` on add (the
checkbox-equivalent isn't on the UI), which lets the subject tag
configured on [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) prepend
to messages between tag and quarantine scores.

> **Operational consequence -- Accept vs Bypass.** "Accept" still
> runs the check; the message is just delivered when it fires.
> "Bypass" doesn't run the check at all. Use Bypass when the
> recipient must not pay the scan cost (e.g. high-volume
> automated relay) and Accept when the recipient must see the
> message but also wants the verdict header for downstream
> filtering (e.g. a SIEM mailbox or a mailbox that runs its own
> filtering on the spam header).

> **Operational consequence -- Bypass disables the verdict
> entirely.** Bypass Virus Checks means the message is never
> scanned by ClamAV; a virus reaching that recipient is not
> caught downstream by anything else in Hermes. Combine Bypass
> with a recipient-specific compensating control (e.g. quarantine
> at the destination mail server) or use Accept instead.

## System vs custom vs default policies

Three orthogonal flags on `spam_policies`:

| Flag | Stored as | Effect |
| --- | --- | --- |
| `system` | `spam_policies.system = '1'` | Ships with the install. Cannot be deleted from the UI. Five system policies are seeded: `No Antispam & No Antivirus`, `Antispam & Antivirus`, `Antispam Only`, `Antivirus Only`, `Default` |
| `custom` | `spam_policies.custom = '1'` | Created by an operator on this page (or via Copy of a system policy). Can be renamed, edited, deleted (unless default or assigned -- see below) |
| `default_policy` | `spam_policies.default_policy = '1'` | The policy applied to any recipient whose `recipients.policy_id` does not resolve. **Exactly one** row in `spam_policies` has this flag; the edit handler toggles it atomically by setting every row to `2` then the target row to `1` |

The DataTable badges each row Yes/No for System and Default so the
operator sees the flags at a glance. System rows lose their delete
checkbox; the default row's "Default Policy" select is read-only
in the edit modal with a hint to "set another policy as the
default instead."

## The page

A Page Guide callout, a collapsible **Add SVF Policy** card, and a
DataTable of every existing policy (system + custom merged) with
per-row Edit, Copy, and Delete actions.

### Add SVF Policy card

A single form covering all four sections (basic + Accept + Bypass +
Notifications). On submit:

1. Validates `policy_name` non-blank, character-safe, and not a
   duplicate
2. Validates `spam_tag2_level` and `spam_kill_level` as floats in
   `-999 .. 999`
3. Validates `banned_rulenames` (File Rule) non-blank
4. INSERTs into `policy` (with `spam_tag_level` hardcoded to
   `-999` and `spam_modifies_subj = 'Y'`)
5. INSERTs into `spam_policies` with `custom = '1'`, `system = '2'`,
   `default_policy = '2'` and `policy_id = <new policy.id>`
6. Runs the Amavis apply chain (see Save and apply flow below)

The Copy action duplicates an existing policy under the name
`Copy of <original>` (with a date-time suffix if that name is
already taken). Useful for branching a system policy into a
custom variant without re-keying every toggle.

### SVF Policies DataTable

| Column | Source |
| --- | --- |
| (checkbox) | Selection for bulk Delete Selected. Disabled with a hover tooltip on system rows |
| Policy Name | `spam_policies.policy_name` |
| System | Yes/No badge driven by `spam_policies.system` |
| Default | Yes/No badge driven by `spam_policies.default_policy` |
| Spam Tag | `policy.spam_tag2_level` |
| Spam Quarantine | `policy.spam_kill_level` |
| File Rule | `policy.banned_rulenames` |
| Actions | Edit, Copy, Delete (Delete hidden on system rows) |

Edit reuses the same validation as Add. Renaming a policy
propagates the new name into `spam_policies.policy_name` in the
same UPDATE.

## Deletion guards

A custom policy can only be deleted when all three guards pass:

| Guard | Source | Alert |
| --- | --- | --- |
| Not a system policy | `spam_policies.system <> '1'` | `m = 10` -- "System policies cannot be deleted" |
| Not the default policy | `spam_policies.default_policy <> '1'` | `m = 11` -- "The default policy cannot be deleted. Set another policy as the default first" |
| Not assigned to any recipient | `recipients.policy_id <> :id` | `m = 12` -- "This policy is assigned to the following recipient(s): <list>. Assign them to a different policy first" |

Single delete reports the specific failure; bulk delete silently
skips guarded rows and reports a per-batch count via `m = 13`
("No policies were deleted") if zero deletes succeeded. The list
of blocking recipients is surfaced in the single-delete failure
alert so the operator can see exactly which entries need to be
reassigned on Relay Recipients or Mailbox Recipients first.

## Save and apply flow

```
1. View page submits action="add_policy" | "edit_policy" |
   "copy_policy" | "delete_policy" | "bulk_delete"
2. Action handler validates input, runs deletion guards,
   INSERTs / UPDATEs / DELETEs on the policy + spam_policies tables
3. cfinclude update_amavis_config_files.cfm:
     - Read /opt/hermes/conf_files/50-user.HERMES
     - Substitute SERVER-NAME, SERVER-DOMAIN, sa-spam-subject-tag,
       final-{virus,banned,spam,bad-header}-destiny,
       enable-dkim-{verification,signing},
       HERMES-USERNAME, HERMES-PASSWORD,
       FILE-RULES-GO-HERE (from file_rule_components table),
       DKIM-KEYS-GO-HERE (from dkim_sign table)
     - Back up /etc/amavis/conf.d/50-user -> 50-user.HERMES.BACKUP
     - Move rendered file into place
4. cfinclude restart_amavis.cfm:
     docker container restart hermes_mail_filter
5. session.m = 1|2|3|5 -> green alert ("Policy Added" / "Updated"
   / "Deleted" / "Copied")
6. cflocation back to view_svf_policies.cfm
```

A few important things about this chain:

- **The policy table is not substituted into `50-user`.** Amavis
  reads it live at scan time via the `$sql_select_policy` SQL
  lookup defined in `50-user.HERMES`. The save-and-apply chain
  still re-renders `50-user` to refresh the static placeholders
  (file rules, DKIM keys, destinies) -- but the SVF policy itself
  is picked up by the next message after the UPDATE commits, no
  Amavis restart strictly required for the policy change alone.
  The restart is there to make the operation atomic with any
  other config that might have drifted, and to surface a clear
  green alert.
- **Copy does not restart Amavis.** It only INSERTs and sets
  `m = 5`; the new policy doesn't affect Amavis until it's
  assigned to a recipient (and `recipients` changes don't go
  through this page).
- **The whole chain is wrapped in `cftry`/`cfcatch`.** If the
  update or restart fails, the policy rows are already committed
  but the operator sees `m = 40` ("Policy was saved but Amavis
  configuration update or reload failed") instead of the green
  alert. A subsequent successful save on any page that triggers
  the same chain re-renders correctly.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` | Add Policy succeeded; Amavis updated and reloaded |
| `m = 2` | Edit Policy succeeded; Amavis updated and reloaded |
| `m = 3` | Delete Policy (single or bulk with at least one success) succeeded |
| `m = 5` | Copy Policy succeeded (no Amavis restart -- new copy is unassigned) |
| `m = 10` | Single delete refused: system policy |
| `m = 11` | Single delete refused: default policy |
| `m = 12` | Single delete refused: policy assigned to recipient(s) -- recipient list surfaced |
| `m = 13` | Bulk delete completed with zero successes (every row was protected) |
| `m = 30` | Policy name empty |
| `m = 31` | Policy name has invalid characters |
| `m = 32` | Policy name duplicates an existing policy |
| `m = 33` | Spam Tag Score empty or non-numeric |
| `m = 34` | Spam Tag Score outside `-999 .. 999` |
| `m = 35` | Spam Quarantine Score empty or non-numeric |
| `m = 36` | Spam Quarantine Score outside `-999 .. 999` |
| `m = 37` | File Rule not selected |
| `m = 38` | Copy: source policy not found |
| `m = 40` | Save succeeded but Amavis apply chain threw |

## Recipient assignment

SVF policies are bound to recipients on the **Email Relay >
Recipients** page (`view_internal_recipients.cfm`) and the
**Email Server > Mailboxes** page (`view_mailboxes.cfm`). Each
page exposes a Policy dropdown populated from `spam_policies`.
Assigning a policy writes the matching `policy.id` into
`recipients.policy_id`, and Amavis picks it up on the next message
to that recipient.

A recipient row with `policy_id` pointing at a row that no longer
exists falls through to the default policy at scan time -- this is
the same fall-through as a recipient with no row in the
`recipients` table at all. The deletion guard on this page (which
refuses delete while any recipient still references the policy) is
the front-line defence against accidentally creating that
fall-through.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_svf_policies.cfm` | `hermes_commandbox` | The page (validation + Add / Edit / Copy / Delete / Bulk Delete) |
| `config/hermes/var/www/html/admin/2/inc/get_svf_policies.cfm` | `hermes_commandbox` | Loads system, custom, and combined policy lists plus the file-rule dropdown |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Renders `50-user` from template + DB (file rules, DKIM keys, destinies) |
| `config/hermes/var/www/html/admin/2/inc/restart_amavis.cfm` | `hermes_commandbox` | `docker container restart hermes_mail_filter` |
| `config/hermes/opt/hermes/conf_files/50-user.HERMES` | template (read) -> `hermes_mail_filter` (live `/etc/amavis/conf.d/50-user`) | Holds `$sql_select_policy` which Amavis uses to resolve a recipient to a policy row at scan time |
| `/etc/amavis/conf.d/50-user.HERMES.BACKUP` | `hermes_mail_filter` | Pre-write backup of the prior live `50-user`, refreshed each save |
| `policy` table | `hermes_db_server` (`hermes` DB) | Amavis-shape policy row -- the source of truth for every per-recipient verdict |
| `spam_policies` table | `hermes_db_server` | Thin index over `policy` with `system` / `custom` / `default_policy` flags |
| `recipients` table | `hermes_db_server` | `recipients.policy_id` is the foreign key Amavis joins on at scan time; the assignment is managed by Relay Recipients and Mailboxes pages |
| `file_rule_components` table | `hermes_db_server` | Source of the File Rule dropdown -- `policy.banned_rulenames` stores the chosen rule name |
| `hermes_mail_filter` container | -- | Hosts Amavis; restarted on add / edit / delete; reads `policy` directly per-message at scan time |

## Related

- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) -- engine-wide
  toggles and the `final_*_destiny` quarantine actions. The SVF
  policy decides whether a message clears the tag/quarantine
  threshold; Anti-Spam Settings decides what Amavis does with a
  quarantine verdict (DSN or silent)
- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) -- ClamAV runs in
  the same Amavis pass that consults the SVF policy. Bypass Virus
  Checks on a policy turns off ClamAV for that recipient entirely
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) -- per-rule SpamAssassin
  weights; tunes the contributions that add up to the final score
  the SVF policy thresholds compare against
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) -- custom SpamAssassin rules
  whose scores contribute to the same final score
- [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules) -- bundles
  [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) and
  [File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions) into the named ruleset
  selected by `policy.banned_rulenames` on this page
- [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) -- the catalogue that
  feeds File Rules; an extension is only enforced when its
  ruleset is bound to a policy here
- [Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds) -- ClamAV signature feeds; a
  policy with Bypass Virus Checks `Y` bypasses every signature
  the feeds ship
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) -- rejection at SMTP-time
  pre-empts every SVF policy decision; the policy only applies to
  mail that clears the perimeter
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) -- a quarantined message
  records the recipient and the verdict; cross-referencing the
  recipient to its policy here explains why that verdict fired
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) -- the full pipeline
  showing where SVF policy lookup happens

# Trusted ARC Sealers — Microsoft 365

# Trusted ARC Sealers — Microsoft 365

## When this guide applies

The standard Hermes-as-relay-MX deployment expects the customer's
downstream mail server (the relay target) to **allowlist Hermes by IP
or hostname** and accept Hermes-forwarded mail without re-running
upstream auth checks. That's how Mimecast, Proofpoint, Barracuda
customers deploy those products; Hermes works the same way. In that
deployment model, you do NOT need a Trusted ARC Sealer configuration
because the receiver doesn't run its own auth checks against
Hermes-forwarded mail in the first place.

**This guide applies when:**

- The customer's downstream MX is M365, AND
- For policy reasons the M365 admin **cannot** simply allowlist
  Hermes (some compliance frameworks require all inbound mail to be
  re-checked, even from trusted gateways), AND
- Hermes-forwarded mail is being quarantined or rejected by M365 due
  to broken upstream `ARC-Message-Signature` body hash (`arc=fail` /
  `cv=fail`) or broken original-sender DKIM (`dkim=fail`) caused by
  Hermes body modification (External Sender Banner, disclaimer, etc.)

In that specific scenario, M365's Trusted ARC Sealers feature lets
the M365 admin tell their tenant "accept Hermes's seal as authoritative
even when the math fails" — which is the receiver-side equivalent of
IP allowlisting for the auth check.

The same scenario is also relevant for **cross-org forwarding cases**
where a Hermes-served message later hops through another Hermes-untrusting
gateway before final delivery (e.g. customer A's Hermes forwards to
customer B's M365 tenant, customer B's tenant doesn't allowlist
customer A's Hermes IP).

## Background: why this comes up

When Hermes modifies a message body — banner injection, disclaimer
injection, S/MIME or PGP rewrap — the modification invalidates any
cryptographic signature whose body hash was computed over the original
bytes. This affects both the original sender's `DKIM-Signature` and
any prior `ARC-Message-Signature` from upstream sealers (M365,
Workspace, Mimecast, Proofpoint, Exclaimer, etc.). Hermes's own ARC
seal at the post-content-filter re-injection point is mathematically
valid (it's computed over the modified body) but honestly records
`cv=fail` on the chain it can no longer body-validate.

A correctly-configured downstream MX allowlists Hermes and ignores
these signals; this guide is for the cases where allowlisting isn't
an option.

## What this fixes (and what it doesn't)

| Symptom | Trusted ARC Sealer helps? |
| --- | --- |
| M365 receiver quarantines forwarded mail with `arc=fail` from Hermes | Yes — M365 will accept Hermes's seal as authoritative |
| M365 receiver delivers but flags forwarded mail as spam due to DMARC fail-on-forward | Yes — DMARC alignment is rescued via the trusted seal |
| Non-M365 downstream MX (Gmail Workspace, on-prem Exchange, third-party SEG) rejects | No — those have their own trust mechanism (Gmail uses an internal list; on-prem typically has none) |
| Outbound mail from Hermes users to external recipients fails DKIM | No — that's a DKIM key/DNS issue, not an ARC trust issue |

## Identity requirements

To add Hermes to the M365 Trusted ARC Sealers list, the receiving M365
tenant administrator needs to know **the ARC signing domain Hermes
uses** — the `d=` value in Hermes's `ARC-Seal:` header. Find this in
the Hermes admin UI under **Content Checks > ARC Settings**: it's the
domain on the active row in the *Gateway ARC Signing Identity* card.

The domain must also have a valid public key published in DNS at
`<selector>._domainkey.<domain>` (this is what M365 fetches to verify
the seal signature before deciding whether to trust the seal). If
DNS isn't right, the math fails before the trust check even runs.

## Configuration steps (M365 admin)

Run in Exchange Online PowerShell connected to the tenant:

```powershell
# Connect (if not already)
Connect-ExchangeOnline

# Inspect existing trusted sealers
Get-ArcConfig

# Add Hermes's signing domain to the trusted list
Set-ArcConfig -Identity Default `
  -ArcTrustedSealers "your-hermes-signing-domain.example.com"
```

If multiple gateways need to be trusted, comma-separate the list:

```powershell
Set-ArcConfig -Identity Default `
  -ArcTrustedSealers "hermes.example.com","mimecast.example.com"
```

To remove a sealer, set the property to a comma-separated list that
omits the entry.

## Verification

After configuration:

1. Send a test message from an ARC-sealing upstream system through
   Hermes (relay-mode domain) to a mailbox on the configured M365
   tenant.
2. Open the message in Outlook on the Web → ellipsis menu → **View →
   View message source**.
3. Look for the `Authentication-Results` header chain that M365 added:
   - `arc=pass` with the `oar=` field referencing Hermes's signing
     domain confirms the trust list took effect.
   - `arc=fail` with a note about `original-authres` indicates the
     trust list did NOT match (most likely cause: domain mismatch or
     DNS not published).

## Troubleshooting

| Problem | Check |
| --- | --- |
| `Get-ArcConfig` returns `ArcTrustedSealers` as empty after Set | Confirm you're connected to the right tenant; verify with `Get-OrganizationConfig \| Select Identity` |
| Test mail still shows `arc=fail` in M365 | Wait up to 60 min for the trust config to propagate; recheck DNS for the Hermes selector |
| Hermes's seal shows `cv=pass` but M365 still rejects | Not an ARC issue — check Connection Filter / Anti-spam policies on the M365 side |

## Related

- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — Hermes-side ARC configuration
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) — full pipeline with ARC placement
- Microsoft official docs: [Trusted ARC Sealers in Exchange Online](https://learn.microsoft.com/en-us/defender-office-365/email-authentication-arc-configure)

# Encryption

# Encryption Settings

# Encryption Settings

Admin path: **Encryption > Encryption Settings** (`view_encryption_settings.cfm`,
`inc/edit_encryption_settings.sh`).

This is the **global Ciphermail policy page** — a thin CFML wrapper over
a fixed set of CipherMail "global" properties that govern subject-based
encryption triggering, the PDF reply-sender identity, and three
internal shared secrets used by the Secure Email Portal back-channel.
Per-recipient policy lives on
[External Recipients](https://docs.deeztek.com/books/administrator-guide/page/external-recipients); CA / S/MIME issuance
lives on [Internal CA](https://docs.deeztek.com/books/administrator-guide/page/internal-ca). This page is the small set of
gateway-wide toggles that affect every encrypted send.

> **Important: not a full encryption-mode picker.** The page does NOT
> pick "always encrypt vs opportunistic vs off" at the system level —
> CipherMail does that per-recipient via the user's `user.encryptMode`
> property (set when the admin creates the recipient on
> [External Recipients](https://docs.deeztek.com/books/administrator-guide/page/external-recipients)). The only system-wide
> opt-in/opt-out exposed here is the **Subject Trigger** mechanism:
> whether `[encrypt]` (or whatever keyword is configured) in a message
> subject promotes that one message to an encryption attempt.

## What the page persists

Every setting on the page is stored twice: once in the Hermes
`encryption_settings` table (so the UI can re-render the current state
on next load) and once in CipherMail's own global property store via
the `CLITool --set-property ... --global` invocation. The two are kept
in sync by re-running the full apply script on every save.

| Field | `encryption_settings.property` | CipherMail property | Notes |
|---|---|---|---|
| Trigger Encryption by Subject (Enabled / Disabled) | `user.subjectTriggerEnabled` | `user.subjectTriggerEnabled` | `true` / `false` string |
| Subject Trigger Keyword | `user.subjectTrigger` | `user.subjectTrigger` | Free text, e.g. `[encrypt]` |
| Remove Trigger After Encryption (Yes / No) | `user.subjectTriggerRemovePattern` | `user.subjectTriggerRemovePattern` | When `true`, the keyword is stripped before the recipient sees the message |
| PDF Reply Sender Email | `user.pdf.replySender` | `user.pdf.replySender` | Email validated as `IsValid("email", ...)` before save |
| Portal URL (read-only, derived) | `user.portal.baseURL` | `user.portal.baseURL` | Built at save time as `https://<console.host>/web/portal` — NOT directly editable on this page; change Console Host on System Settings |
| Server Secret Keyword | `user.serverSecret` | `user.serverSecret` (encrypted) | 64-char auto-generated, masked in UI |
| Client Secret Keyword | `user.clientSecret` | `user.clientSecret` (encrypted) | 64-char auto-generated, masked in UI |
| Mail Secret Keyword | `user.systemMailSecret` | `user.systemMailSecret` (encrypted) | 64-char auto-generated, masked in UI |

Additionally, the script always sets `user.otpEnabled = true --global`
on every save — a fixed override that ensures CipherMail's one-time
password feature is on globally regardless of any prior state.

## Subject Trigger: how it actually works

When **Trigger Encryption by Subject** is enabled, CipherMail inspects
each outbound message's `Subject:` header during processing:

```
+------------------+      +-------------------+      +-----------------+
| Outbound message |----->|  CipherMail       |----->|  Encryption     |
| Subject:         |      |  subject-trigger  | yes  |  policy for     |
| "[encrypt] Q4"   |      |  match?           |----->|  this recipient |
+------------------+      +-------------------+      +-----------------+
                                  | no
                                  v
                          +-----------------+
                          |  Recipient's    |
                          |  user.encryptMode|
                          |  decides        |
                          +-----------------+
```

| Setting combination | Behavior |
|---|---|
| Trigger ENABLED + Keyword present + Recipient `user.encryptMode = allow` | Message encrypted using whichever protocol the recipient has enabled (S/MIME / PGP / PDF). If none, CipherMail falls back to its protocol-selection rules. |
| Trigger ENABLED + Keyword present + Recipient `user.encryptMode = mandatory` | Already always-encrypted; the keyword is redundant. If **Remove Trigger** is on, the keyword is still stripped from the visible subject. |
| Trigger ENABLED + Keyword NOT present + Recipient `user.encryptMode = allow` | Message sent plaintext (the recipient is configured "by subject" and the sender did not opt in). |
| Trigger ENABLED + Keyword NOT present + Recipient `user.encryptMode = mandatory` | Encrypted regardless (recipient policy overrides). |
| Trigger DISABLED | Subject line is never inspected; recipient `user.encryptMode` is the sole authority. Senders cannot opt-in per message. |

Recipient `user.encryptMode` is set when the admin picks a mode (e.g.
"PDF Mandatory" vs "PDF By Subject") on **Encryption > External
Recipients > Create**. See
[External Recipients — Encryption modes](https://docs.deeztek.com/books/administrator-guide/page/external-recipients#encryption-modes).

## PDF Reply Sender

When a recipient receives a PDF-encrypted message and clicks the reply
link in the encrypted PDF, the response comes back to Hermes via the
Secure Email Portal. The **PDF Reply Sender Email** is the `From:`
address CipherMail uses when delivering that reply back to the original
internal sender (and on system notifications about PDF reply
activity). Operators typically set this to a monitored address like
`postmaster@yourdomain.tld` or a dedicated `secure-reply@...` mailbox.

The field is validated: empty or non-email values trigger alerts `m=3`
and `m=2` respectively and abort the save.

## The three secret keywords

CipherMail uses three independent shared secrets to authenticate the
back-channel between the encryption engine and the Secure Email Portal
(`/web/portal/`). They are stored AES-encrypted in
`encryption_settings.value` (using `/opt/hermes/keys/hermes.key` as the
key) and pushed into CipherMail with the `--encrypt` flag so CipherMail
encrypts them again with its own key.

| Secret | Used by | Generated by |
|---|---|---|
| **Server Secret** (`user.serverSecret`) | CipherMail server-side validation of portal session tokens | Click the sync icon on the field; never user-entered |
| **Client Secret** (`user.clientSecret`) | Portal client-side validation handshake | Click the sync icon |
| **Mail Secret** (`user.systemMailSecret`) | Signing of system-generated email notifications (password delivery, portal invitations, etc.) | Click the sync icon |

The UI masks the values to `********************<last 4 chars>` — full
plaintext is never re-displayed after generation. To replace a secret,
click the sync (`fa-sync-alt`) button on its row; a confirmation modal
fires; on confirm Hermes:

1. Generates 64 lowercase hex-ish characters by concatenating 8 rounds
   of the standard `customtrans3` token generator and truncating.
2. AES-encrypts that with `/opt/hermes/keys/hermes.key` and UPDATEs
   `encryption_settings.value` for the corresponding property.
3. Runs the full `edit_encryption_settings.sh` apply script (see below)
   to push **all three** secrets — plus the subject-trigger / PDF
   reply / portal URL settings — into CipherMail in one shot.

Rotating any one secret therefore re-applies the other two as a
side-effect; in practice the values are stable across rotations
because the script reads each from its already-decrypted form before
writing.

**Operational consequence:** rotating a secret invalidates any
in-flight portal sessions for that secret's role. Recipients with an
active portal session may need to log in again; system notifications
in transit may fail signature verification and be re-queued.

## The apply pipeline

Both **Save Settings** and **Generate Secret** funnel through the same
temp-script pattern documented across the Hermes admin:

```
+--------------------+      +-----------------------------+      +-------------------+
| CFML page UPDATEs  |----->| Read /opt/hermes/scripts/   |----->| REReplace 9       |
| encryption_settings|      | edit_encryption_settings.sh |      | placeholders      |
+--------------------+      +-----------------------------+      +-------------------+
                                                                          |
                                                                          v
                                                                +---------------------+
                                                                | Write to            |
                                                                | /opt/hermes/tmp/    |
                                                                | <token>_edit_...sh  |
                                                                +---------------------+
                                                                          |
                                                                          v
                                                                +---------------------+
                                                                | chmod +x and execute|
                                                                | (240s timeout) then |
                                                                | delete the temp file|
                                                                +---------------------+
                                                                          |
                                                                          v
                                                                +---------------------+
                                                                | 9 sequential        |
                                                                | docker exec         |
                                                                | hermes_ciphermail   |
                                                                | CLITool --global    |
                                                                +---------------------+
```

Placeholders substituted in the template:

| Placeholder | Replaced with |
|---|---|
| `PDFREPLY-SENDER` | `user.pdf.replySender` value |
| `PORTAL-URL` | Derived `https://<console.host>/web/portal` |
| `SUBJECT-TRIGGER` | `user.subjectTrigger` value |
| `SUBJECT-ENABLE` | `true` / `false` |
| `TRIGGER-REMOVE` | `true` / `false` |
| `SERVER-SECRET` | Decrypted server secret (pushed with `--encrypt` so CipherMail re-encrypts) |
| `CLIENT-SECRET` | Decrypted client secret |
| `MAIL-SECRET` | Decrypted mail secret |

On a CLITool execution failure the page sets `session.m_enc = 11` and
surfaces "Settings saved to database but failed to apply to
Ciphermail. Please check the logs." — the DB write succeeds first, so
the UI state matches what the operator entered even when the
CipherMail-side push fails. Re-save (with no edits) re-runs the apply
script.

## What's NOT on this page

Several things an operator might reasonably expect from a global
"Encryption Settings" page that live elsewhere:

| Expectation | Where it actually lives |
|---|---|
| Per-recipient "always encrypt vs by subject vs never" | [External Recipients](https://docs.deeztek.com/books/administrator-guide/page/external-recipients) (`user.encryptMode` per CipherMail user) |
| Default cipher / algorithm selection (AES-128 vs AES-256, RSA key sizes) | CipherMail Advanced Settings (`/ciphermail/`, external link in sidebar) |
| Per-mailbox sign / encrypt action defaults | Email Server > Mailboxes (per-mailbox encryption action editor, `inc/edit_mailbox_encryption_action.cfm`) |
| TLS opportunistic vs DANE policy on outbound delivery | Email Relay > Relay Hosts and TLS Settings; this page is about message-content encryption only |
| Subject keyword for DLP-driven (content-based) encryption triggers | Not implemented in Hermes; CipherMail Advanced Settings can express custom DLP rules |
| Portal URL customization | Derived automatically from System > Console Settings (`parameters2.console.host`); editing console host updates this on next save |
| S/MIME signing of every outbound (gateway sign-and-forward) | CipherMail Advanced Settings; not surfaced here |
| Password complexity rules for the auto-generated portal / PDF passwords | Hardcoded in the modal JS on
[External Recipients](https://docs.deeztek.com/books/administrator-guide/page/external-recipients) (16-char mixed alphanumeric) |

## Body-modification interaction

The CipherMail encryption / signing pass runs **after** the
`hermes_body_milter` disclaimer / signature / banner pipeline. That
means PDF, S/MIME, and PGP envelopes always wrap the final body the
recipient sees — including any appended disclaimer (see
[Disclaimers — Behavior with S/MIME, PGP, and DKIM-signed mail](https://docs.deeztek.com/books/administrator-guide/page/disclaimers#behavior-with-smime-pgp-and-dkim-signed-mail)).
The same milter-ordering rationale applies to ARC inbound sealing (see
[ARC Settings — Container and milter placement](https://docs.deeztek.com/books/administrator-guide/page/arc-settings#container-and-milter-placement)):
the cryptographic envelope is the last thing applied so it always
matches what the recipient downloads.

## Container and database touch-points

| Component | Container / path | Role |
|---|---|---|
| Page | `config/hermes/var/www/html/admin/2/view_encryption_settings.cfm` (`hermes_commandbox`) | CRUD UI + apply orchestration |
| Template script | `/opt/hermes/scripts/edit_encryption_settings.sh` (`hermes_commandbox` bind mount) | 9-line shell with 9 placeholders |
| Temp scripts | `/opt/hermes/tmp/<token>_edit_encryption_settings.sh` | Substituted copy, executed once, deleted |
| Settings store (Hermes side) | `encryption_settings` in `hermes` DB (`hermes_db_server`) | One row per property; secrets stored AES-encrypted in `value` |
| Settings store (CipherMail side) | `cm_properties` in `djigzo` DB (`hermes_db_server`) — set indirectly via CLITool `--global` | CipherMail's authoritative global property store |
| Encryption engine | `hermes_ciphermail` (Java; CipherMail Community 5.x branded `djigzo`) | Performs S/MIME / PGP / PDF encryption at send time |
| Encryption key | `/opt/hermes/keys/hermes.key` (`hermes_commandbox` bind mount) | AES key used for CFML-side `encrypt()` / `decrypt()` of the three secrets |
| Console host source | `parameters2.console.host` in `hermes` DB | Drives the auto-derived `user.portal.baseURL` |

## Related

- [External Recipients](https://docs.deeztek.com/books/administrator-guide/page/external-recipients) — per-recipient encryption modes; the page where `user.encryptMode = mandatory` vs `allow` is actually chosen
- [Internal CA](https://docs.deeztek.com/books/administrator-guide/page/internal-ca) — where the S/MIME root CAs that mint per-recipient certs live; cross-referenced by recipient PDF / S/MIME / PGP rows on External Recipients
- [PGP Key Servers](https://docs.deeztek.com/books/administrator-guide/page/pgp-key-servers) — outbound key publishing list (note: publish-only, not lookup)
- [Disclaimers](https://docs.deeztek.com/books/administrator-guide/page/disclaimers) — body-mod ordering against the CipherMail encryption pass
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — same milter-ordering pattern applied to inbound chain sealing
- [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) — cross-references the body-mod pipeline that also feeds DKIM signing
- **Advanced Settings** (sidebar link to `/ciphermail/`) — CipherMail's own admin UI; everything not surfaced on this page (per-protocol cipher selection, custom DLP, gateway-wide always-sign) lives there

# External Recipients

# External Recipients

Admin path: **Encryption > External Recipients**
(`view_ext_rec_encryption.cfm`, `view_create_ext_recipient.cfm`,
`view_ext_smime_certificates.cfm`, `view_ext_pgp_keyrings.cfm`,
`view_ext_add_smime_cert.cfm`, `view_ext_add_pgp_keyring.cfm`,
`inc/create_ext_recipient.cfm`, `inc/delete_ext_recipient.cfm`,
`inc/reset_pdf_password.cfm`, `inc/reset_portal_password.cfm`).

This is the **per-counterparty encryption policy and key store** for
external (non-managed) email addresses. Each row binds a single
external email to one of three protocols (PDF / S/MIME / PGP) and to
one of two trigger modes (Mandatory / By Subject). It is the page
where the policy referenced by
[Encryption Settings](https://docs.deeztek.com/books/administrator-guide/page/encryption-settings) actually takes effect —
the global page chooses the **mechanism** (subject trigger keyword,
shared secrets, PDF reply sender); this page chooses the **policy**
for every external recipient the gateway encrypts to.

The DataTable is the master view across **both** Hermes-side metadata
(`external_recipients` in the `hermes` DB) and CipherMail's own user
table (`cm_users` in the `djigzo` DB), joined on email address. Rows
are tagged **Admin-Configured** (explicitly created on this page,
with a matching `external_recipients` row) or **Auto-Discovered**
(materialized by CipherMail during message processing, no
`external_recipients` row).

## Schema: two tables, one view

```
+--------------------------+         +--------------------------+
|  hermes.external_recipients         |  djigzo.cm_users         |
|  (admin metadata)        |         |  (CipherMail user store) |
+--------------------------+         +--------------------------+
| email                    |  ----   | cm_email                 |
| encryption_mode          |         | cm_id  -->  cm_properties|
| pdf, smime, pgp (flags)  |         |               (per-user  |
| pdf_mode                 |         |                policy)   |
| pdf_password (AES-enc.)  |         +--------------------------+
+--------------------------+
            |
            v
   Page renders Admin badge
            |
+--------------------------+
| If NO matching row,      |
| recipient is "Auto" with |
| inferred policy from     |
| cm_certificates_email /  |
| cm_keyring_email         |
+--------------------------+
```

The page never N+1's against CipherMail — three batch queries build
struct lookups (`adminLookup`, `smimeLookup`, `pgpLookup`) and the row
loop reads from those instead of per-row queries. That matters at any
scale beyond a few hundred recipients.

`external_recipients` columns:

| Column | Purpose |
|---|---|
| `id` | PK |
| `email` | External email address (joined to `cm_users.cm_email`) |
| `encryption_mode` | `pdf_mandatory` / `pdf_by_subject` / `smime_mandatory` / `smime_by_subject` / `pgp_mandatory` / `pgp_by_subject` |
| `pdf` / `smime` / `pgp` | Flag (1 / NULL) indicating which protocol is the active one for this recipient |
| `pdf_mode` | For PDF only: `static` / `random` / `backtosender` |
| `pdf_password` | AES-encrypted (with `/opt/hermes/keys/hermes.key`) copy of the static PDF password — for admin re-display only; CipherMail holds its own copy |
| `smime_mode` / `pgp_mode` | Reserved for parity; populated identically to `encryption_mode` for the matching protocol |

## Encryption modes

The 6 encryption modes map cleanly onto two axes (protocol × trigger):

| Mode | CipherMail `user.encryptMode` | CipherMail `user.pdf.encryptionAllowed` | CipherMail `user.sMIMEEnabled` | CipherMail `user.pgp.enabled` |
|---|---|---|---|---|
| `pdf_mandatory` | `mandatory` | `true` | `false` | `false` |
| `pdf_by_subject` | `allow` | `true` | `false` | `false` |
| `smime_mandatory` | `mandatory` | `false` | `true` | `false` |
| `smime_by_subject` | `allow` | `false` | `true` | `false` |
| `pgp_mandatory` | `mandatory` | `false` | `false` | `true` |
| `pgp_by_subject` | `allow` | `false` | `false` | `true` |

"By Subject" requires **Encryption Settings > Trigger Encryption by
Subject = Enabled** plus the configured keyword (default `[encrypt]`)
in the message subject. See
[Encryption Settings — Subject Trigger](https://docs.deeztek.com/books/administrator-guide/page/encryption-settings#subject-trigger-how-it-actually-works)
for the decision tree.

## PDF mode: three sub-policies

PDF encryption is the lowest-friction protocol (recipient needs only a
PDF reader and a password — no certs, no keys, no portal account
required up front), so it ships with three independent
password-distribution sub-modes:

| `pdf_mode` | How the password reaches the recipient | When to use |
|---|---|---|
| `random` | CipherMail auto-generates a one-time password per message and pushes it through the Secure Email Portal (`https://<console>/web/portal`); recipient self-registers on first use | Default. Best for ad-hoc / first-time external recipients |
| `static` | Admin sets a fixed password once (minimum 12 chars); recipient must already know it via out-of-band channel | Long-term partners who have agreed on a shared secret |
| `backtosender` | CipherMail generates a per-message password and emails it back to the **original internal sender** for them to relay to the recipient | Compliance scenarios where the sender must explicitly hand the password to the recipient (auditable trail) |

For `backtosender`, two extra fields are configurable per recipient:

| Field | Range | Purpose |
|---|---|---|
| Password Age (minutes) | 15-240 | How long the random password is valid |
| Password Length | 16-bit / 20-bit | Bit-strength of the generated random password |

## Bulk vs single create

The **Create External Recipient** page (`view_create_ext_recipient.cfm`)
exposes a Single / Bulk toggle:

| Mode | Protocol options | Use case |
|---|---|---|
| **Single** | PDF, S/MIME, PGP (all three modes available) | One-off precise configuration including S/MIME / PGP recipients that need a cert/key uploaded afterward |
| **Bulk** | **PDF only** (Mandatory or By Subject) | Mass-onboard a list of external addresses, one per line; the UI auto-hides S/MIME and PGP because those protocols need per-recipient cert/key material that has no bulk equivalent |

The bulk path validates and skips per-row (invalid format / internal
domain / already-exists rows are reported but do not abort the batch);
session variables `bulk_created`, `bulk_skipped`, `bulk_failed` feed a
partial-success alert on return.

**Both paths refuse internal domains.** The check is a `COUNT(*) FROM
domains WHERE domain = <recipient-domain>` — if Hermes is the
authoritative MX for that domain, the recipient is a local mailbox or
relay recipient, not an external recipient, and per-mailbox encryption
policy belongs on Email Server > Mailboxes instead.

## Auto-Discovered recipients

When CipherMail processes mail to an address it has never seen, it
materializes a `cm_users` row with the global defaults. These
recipients show up here with **Source = Auto** and no
`external_recipients` row backing them. They:

- Use the global Subject Trigger policy (from
  [Encryption Settings](https://docs.deeztek.com/books/administrator-guide/page/encryption-settings))
- Have no per-recipient password mode (PDF random is the CipherMail
  default)
- Display only the cert / keyring counts CipherMail actually holds
- Cannot be edited from this page (no Admin badge, no action buttons
  for cert / PGP / password reset) — managing them means **either**
  promoting them to Admin-Configured by creating an explicit row,
  **or** dropping into CipherMail's own admin UI at `/ciphermail/`

The Source dropdown defaults to **Admin-Configured** on page load —
operators most often want to see what they explicitly configured, not
the long tail of mail CipherMail has touched.

## Per-row actions

The action column varies by what the recipient is configured for:

| Action | Icon | Visible when | What it does |
|---|---|---|---|
| **S/MIME Certificates** | `fa-certificate` (green) | Admin row, `smime = 1` | Links to `view_ext_smime_certificates.cfm?email=...` for cert add / delete / send |
| **PGP Keyrings** | `fa-key` (blue) | Admin row, `pgp = 1` | Links to `view_ext_pgp_keyrings.cfm?email=...` for keyring add / delete / publish |
| **Reset PDF Password** | `fa-file-pdf` (yellow) | Admin row, `pdf = 1` AND `pdf_mode = static` | Opens modal; auto-generates a 16-char mixed-case-alphanumeric password client-side via `generatePassword(16)`; submits to `inc/reset_pdf_password.cfm` |
| **Reset Portal Password** | `fa-lock` (grey) | Admin row, `pdf = 1` AND `pdf_mode = random` | Opens modal; same 16-char generator; submits to `inc/reset_portal_password.cfm` (two-step: encode via `--encode-password`, then set `user.portal.password`) |
| **Delete Recipient** | `fa-trash-alt` (red) | Every row | Confirms, then submits to `delete_recipient` handler |

The Cert Expiry column derives from a batch join of
`cm_certificates_email + cm_certificates`, picking the **earliest**
`cm_not_after` across all certs for that recipient. Color coding:
red bold (already expired), yellow bold (within 30 days), grey muted
(more than 30 days).

## Delete cascade

Deleting an external recipient is a multi-table operation handled by
`inc/delete_ext_recipient.cfm`:

```
+---------------------------+
| For each row in           |
| recipient_certificates    |
| where user_id = recipient |
+---------------------------+
            |
            v
+---------------------------+      +---------------------------+
| inc/delete_smime_         |----->| Removes from              |
| certificate.cfm           |      | cm_certificates_email,    |
|                           |      | CipherMail user store,    |
|                           |      | on-disk PFX               |
+---------------------------+      +---------------------------+
            |
            v
+---------------------------+
| For each master keyring   |
| in recipient_keystores    |
+---------------------------+
            |
            v
+---------------------------+
| inc/delete_pgp_keyring.   |
| cfm                       |
+---------------------------+
            |
            v
+---------------------------+
| DELETE FROM               |
| external_recipients       |
| WHERE id = ...            |
+---------------------------+
            |
            v
+----------------------------------------+
| docker exec hermes_ciphermail CLITool  |
| --delete-user <email>                  |
| (cascades all cm_properties, cm_users) |
+----------------------------------------+
```

On success the page surfaces a callout reminding the operator that any
**Sender Checks Bypass** mapping tied to this recipient must be
re-created — that relationship is not auto-cascaded.

## Password reset specifics

**PDF static password reset** (`inc/reset_pdf_password.cfm`):

1. Writes a one-liner `CLITool --set-property user.password --value
   <newpass> --encrypt --email <recipient>` to
   `/opt/hermes/tmp/<token>_reset_pdf_password.sh`.
2. `chmod +x`, executes (240s timeout), deletes.
3. AES-encrypts the new password with `/opt/hermes/keys/hermes.key`
   and UPDATEs `external_recipients.pdf_password` so the admin
   re-display path still works.

**Portal password reset** (`inc/reset_portal_password.cfm`) is
**two-step** because CipherMail's portal password is stored as an
encoded value, not the raw string:

1. **Step 1 — encode:** runs `CLITool --encode-password <newpass>`,
   captures stdout to `/opt/hermes/tmp/<token>_portal_password`, reads
   that file back into CFML, deletes the temp file.
2. **Step 2 — set:** runs `CLITool --set-property user.portal.password
   --encrypt --email <recipient> --value <encoded>` to push the
   encoded value into CipherMail.

Both modals auto-generate a 16-character mixed-case-alphanumeric
password client-side and pre-populate the hidden confirm field; the
operator can regenerate or type-in their own. Min length 12 is
enforced server-side; the regenerator produces 16.

The modal text explicitly notes that **unencrypted voice calls and
texts are NOT considered secure** for relaying the password to the
recipient — operators are expected to use Signal, an in-person
exchange, or a separately-encrypted channel.

## CipherMail integration: every action is docker exec

Every CipherMail-side mutation on this page uses the same pattern
documented across the Hermes admin:

```
+----------------------+      +----------------------+      +-------------------+
| CFML builds shell    |----->| Write to             |----->| chmod +x          |
| string with N        |      | /opt/hermes/tmp/     |      |                   |
| docker exec CLITool  |      | <token>_<purpose>.sh |      |                   |
| lines                |      |                      |      |                   |
+----------------------+      +----------------------+      +-------------------+
                                                                       |
                                                                       v
                                                              +--------------------+
                                                              | cfexecute (240s),  |
                                                              | then delete the    |
                                                              | temp file          |
                                                              +--------------------+
                                                                       |
                                                                       v
                                                       +-------------------------------+
                                                       | docker exec hermes_ciphermail |
                                                       | /usr/bin/java -cp '/.../lib/*'|
                                                       | mitm.application.djigzo.tools |
                                                       | .CLITool <args>               |
                                                       +-------------------------------+
```

The Hermes app container (`hermes_commandbox`) holds no JVM and no
CipherMail libraries; everything reaches into `hermes_ciphermail` over
the docker socket via `CLITool`. The temp-script pattern (write +
chmod + execute + delete) survives the Lucee `cfexecute` quirks around
stderr and quoting that would otherwise make a direct inline
invocation unreliable.

## What's NOT on this page

| Expectation | Where it actually lives |
|---|---|
| Per-recipient cipher / algorithm selection (AES-128 vs AES-256, RSA / EC) | CipherMail Advanced Settings (`/ciphermail/`); per-recipient overrides live in `cm_properties` directly |
| Auto-lookup of recipient PGP keys from a keyserver at send time | Not implemented; see [PGP Key Servers](https://docs.deeztek.com/books/administrator-guide/page/pgp-key-servers) — that page is publish-only. Keys must be uploaded manually on the PGP Keyrings sub-page |
| Auto-lookup of recipient S/MIME certs via LDAP / public directory | Not implemented; certs must be uploaded manually on the S/MIME Certificates sub-page, OR minted from an [Internal CA](https://docs.deeztek.com/books/administrator-guide/page/internal-ca) row and sent to the recipient |
| Per-recipient subject-trigger keyword override | Not implemented; the keyword is global (one row in `encryption_settings`) |
| Recipient-side enrollment / self-service for their own keys | The Secure Email Portal handles recipient password registration for PDF-random mode; there is no self-service cert / PGP upload UI |
| Bulk import from CSV with mixed protocols | Bulk path is PDF-only by design (S/MIME / PGP need per-recipient material that doesn't bulk-import cleanly) |
| Sender-side "force encrypt for this thread" UI | Senders use the subject trigger; there is no per-mailbox sender UI |

## Container and database touch-points

| Component | Container / path | Role |
|---|---|---|
| Page | `config/hermes/var/www/html/admin/2/view_ext_rec_encryption.cfm` (`hermes_commandbox`) | List, filter, password resets, delete |
| Create page | `view_create_ext_recipient.cfm` + sub-pages for cert / keyring management | Single + bulk insertion |
| Action includes | `inc/create_ext_recipient.cfm`, `inc/delete_ext_recipient.cfm`, `inc/reset_pdf_password.cfm`, `inc/reset_portal_password.cfm` | One-liner CLITool dispatchers via temp script |
| Admin metadata | `external_recipients` in `hermes` DB (`hermes_db_server`) | Per-recipient policy choices + AES-encrypted static PDF password copy |
| CipherMail user store | `cm_users`, `cm_properties` in `djigzo` DB | Authoritative per-recipient state |
| CipherMail cert / key index | `cm_certificates_email`, `cm_certificates`, `cm_keyring_email` in `djigzo` DB | Joined batch into `smimeLookup` / `pgpLookup` for column rendering |
| Encryption engine | `hermes_ciphermail` (Java; CipherMail Community 5.x branded `djigzo`) | Actual S/MIME / PGP / PDF encryption + portal back-channel |
| AES key | `/opt/hermes/keys/hermes.key` (`hermes_commandbox` bind mount) | Encrypts `pdf_password` for re-display |
| Secure Email Portal | `https://<console.host>/web/portal/` (served by `hermes_ciphermail`) | Recipient-facing landing page for PDF random + portal account flows |

## Related

- [Encryption Settings](https://docs.deeztek.com/books/administrator-guide/page/encryption-settings) — global Subject Trigger, PDF reply sender, three shared secrets; the policy mechanism this page applies per recipient
- [Internal CA](https://docs.deeztek.com/books/administrator-guide/page/internal-ca) — where the private CAs that can mint S/MIME certs for these recipients live (operator-issued S/MIME chain delivered out-of-band)
- [PGP Key Servers](https://docs.deeztek.com/books/administrator-guide/page/pgp-key-servers) — the publish list for the Publish action on the PGP Keyrings sub-page (note: publish-only, not lookup)
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — distinct TLS cert store; not related to message-content S/MIME
- [Disclaimers](https://docs.deeztek.com/books/administrator-guide/page/disclaimers) — body-mod ordering vs the CipherMail encryption pass (disclaimer is appended before encryption wraps the message)
- [Organizational Signatures](https://docs.deeztek.com/books/administrator-guide/page/organizational-signatures) — same milter ordering applies to signature injection
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — same milter-ordering pattern applied to inbound chain sealing
- **Advanced Settings** (sidebar link to `/ciphermail/`) — CipherMail's own admin UI for everything not surfaced here (per-recipient cipher tuning, custom DLP, direct `cm_properties` editing)

# Internal CA

# Internal CA

Admin path: **Encryption > Internal CA** (`view_internal_ca.cfm`,
`inc/download_ca_file.cfm`, `inc/create_certificate.cfm`,
`inc/send_smime_certificate.cfm`, `inc/delete_smime_certificate.cfm`).

This is the **gateway's built-in Certificate Authority** for issuing
S/MIME certificates to local users and relay recipients. Each CA row
here corresponds to a private CA cert + key on disk under
`/opt/hermes/CA/<directory>/root_ca/` and a matching `roots`-store
entry in the CipherMail (`djigzo`) trust list. Per-recipient S/MIME
certs minted from a CA on this page are stored in
`recipient_certificates` and listed on Email Server > Relay Recipients
(and Email Server > Mailboxes when S/MIME is enabled on a mailbox).

This page is **distinct from** [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates):

| | System Certificates | Internal CA |
|---|---|---|
| What it stores | Operator-uploaded TLS leaf certs (nginx, Postfix, Dovecot) | Private CAs that *mint* S/MIME end-user certs |
| Trust direction | Hermes presents these to clients | Hermes issues certs that *recipients* present |
| Backing store | `system_certificates` table + `/opt/hermes/ssl/` or `/etc/letsencrypt/` | `ca_settings` table + `/opt/hermes/CA/<dir>/` + CipherMail `cm_certificates` (`roots` store) + `cm_ctl` trust list |
| Typical lifetime | 90 d (ACME) or 1-3 yr (commercial) | 5 yr root (recommended), extendable in place |
| Lifecycle owner | nginx / Postfix / Dovecot via TLS handshake | CipherMail S/MIME signer / encryptor for outbound; per-recipient cert issuance for inbound encrypt |

## The two ingest paths

The page exposes two collapsing cards (**Create Internal CA**,
**Import External CA**) plus a DataTable of existing CAs. Both paths
land a row in `ca_settings` and register the cert in CipherMail's
`cm_certificates` table as a root (`cm_store_name = 'roots'`) plus an
entry in `cm_ctl` (Certificate Trust List) flagged `whitelisted`.

### 1. Create Internal CA

Operator fills the DN fields, picks a key size (2048 / 4096) and a
validity (1-5 years; 5 years recommended). Hermes:

1. Validates inputs (regex-restricted character set per field, 2-char
   ISO country code, uniqueness against `ca_settings.ca_commonname`).
2. Materializes a per-CA on-disk skeleton at
   `/opt/hermes/CA/<sanitized-cn>/root_ca/` with the standard OpenSSL
   layout (`certs/`, `crl/`, `newcerts/`, `private/`, `requests/`,
   `PFX/`, `serial`, `index.txt`, `crlnumber`).
3. Materializes an `openssl.cnf` from `/opt/hermes/templates/rootca_openssl.cnf`
   with the directory placeholder substituted.
4. Snapshots `cm_certificates` into `cm_certificates_tmp`, runs the
   OpenSSL root-CA generation script as a one-shot temp script
   (`/opt/hermes/scripts/<token>_create_ca.sh`), then diffs to find
   the new cert.
5. Marks the new CipherMail row `cm_store_name = 'roots'`, inserts a
   `cm_ctl` row with status `whitelisted` and `allowExpired = false`,
   and back-fills `ca_settings.ca_djigzo_id` + `ca_djigzo_subject`.

### 2. Import External CA

For organizations that already have a private CA (commercial issuer,
internal PKI, prior Hermes install). Operator uploads the **CA cert**
(PEM) and the **CA private key** (PEM, unencrypted). Hermes:

1. Lands the files at `/opt/hermes/CA/<sanitized-cn>/root_ca/certs/cacert.pem`
   and `.../private/cakey.pem`.
2. Runs an OpenSSL validation script that checks:
   - Cert parses as X.509 (`openssl x509 -modulus`)
   - Key parses as RSA (`openssl rsa -modulus`)
   - Cert and key moduli match (private key matches public key)
   - Cert has `CA:TRUE` basic constraint
3. On any check failure the upload directory is removed and the
   operator gets a specific error alert (m=48 / 49 / 50 / 51).
4. Generates `openssl.cnf` from the template + `cachain.pem` = copy of
   the cert (needed for later PFX export of per-user certs).
5. Pipes the cert into CipherMail via
   `docker exec -i hermes_ciphermail /usr/bin/java -cp '/usr/share/djigzo/lib/*' mitm.application.djigzo.tools.CertStore --import-certificates`
   and back-fills the `ca_djigzo_id` exactly as the Create path does.

The Import path is the only way to migrate a CA that already has
issued certs in the wild — re-creating a CA from scratch with the same
DN does NOT reproduce the original key material, so previously issued
certs would not chain to it.

## Default CA flag (`default2`)

Exactly one row in `ca_settings` has `default2 = '1'`; all others have
`'2'`. The default CA is the one Hermes mints from when an admin
clicks **Create Certificate** for a recipient on Email Server > Relay
Recipients (or the mailbox equivalent) without explicitly choosing a
CA. The page enforces single-default by:

- Setting all rows to `'2'` before flipping the new row to `'1'`
- Forcing the first-ever CA to default regardless of the checkbox
- Disabling the **Make Default** checkbox on the Create card when no
  default exists (forced default is implicit)

The DataTable Default column renders a green **YES** badge for the
default row and a one-click **set default** button for the others.

## CA lifecycle workflow

```
+----------------+      +----------------+      +----------------+
|  Admin creates |----->|  CipherMail    |----->|  Recipient     |
|  Internal CA   |      |  trusts root   |      |  cert minted   |
+----------------+      +----------------+      +----------------+
                                                        |
                                                        v
                                                +----------------+
                                                | Outbound mail  |
                                                | signed by      |
                                                | recipient cert |
                                                +----------------+
```

| Stage | Where the data lives | Trigger |
|---|---|---|
| CA root created | `ca_settings` + `/opt/hermes/CA/<dir>/` + `cm_certificates` (`roots`) + `cm_ctl` (`whitelisted`) | Create / Import buttons on this page |
| Per-recipient cert minted | `recipient_certificates` (or `external_recipient_certificates`) + CipherMail user store | **Create Certificate** button on a recipient page; uses `default2 = '1'` CA unless overridden |
| Cert self-introduction | Bundled into the first signed outbound message the recipient sends | Automatic on first S/MIME-signed send |
| Cert revocation | `delete_smime_certificate.cfm` removes the row + CipherMail entry; CRL is maintained by CipherMail's own scheduled job | **Delete** button on the recipient cert row |
| CA renewal | Re-sign the existing cert + key with `openssl x509 -days <N>` and re-import into CipherMail; `ca_settings.expires` updated | **Renew** button (sync icon) on the CA row |
| CA deletion | Refused if any `recipient_certificates.ca_id` row references it; otherwise removes DB row + CipherMail `cm_certificates` / `cm_ctl` + on-disk tree | **Delete** button (only enabled when zero issued certs) |

## CA Renewal: 5-year extension in place

Clicking the **Renew** (sync) icon does NOT generate a new key pair —
it re-signs the existing CA cert against its own key with an extended
`notAfter`. The math:

```
new_expires = current_expires + 5 years
days_param  = max(1825, days_from_now_to_new_expires)
openssl x509 -in cacert.pem -days <days_param> -out cacert.pem.new -signkey cakey.pem
mv cacert.pem.new cacert.pem
cp cacert.pem cachain.pem
cat cacert.pem | docker exec -i hermes_ciphermail \
  /usr/bin/java -cp '/usr/share/djigzo/lib/*' \
  mitm.application.djigzo.tools.CertStore --import-certificates
```

Because the key stays the same, every previously issued recipient cert
**still chains to a valid CA cert** — there is no need to re-mint
recipient certs after a CA renewal. This is the operator-friendly path:
recipients on the outside who already trust the CA root continue to
trust it transparently.

The old CipherMail row is deleted and the renewed cert re-imported so
the `cm_certificates`/`cm_ctl` rows reflect the new validity window
(otherwise CipherMail would keep enforcing the old expiry).

## Trust distribution to external recipients

A Hermes-issued S/MIME cert is signed by a private CA that **no
operating system or mail client trusts by default**. External
recipients see Hermes-signed mail as "signed by an unknown CA" until
they explicitly install the Internal CA root in their trust store.

Two practical paths:

| Path | Effort | Reach |
|---|---|---|
| **Operator distributes the CA root** out-of-band (download from this page, email or publish on a portal, recipient installs in Outlook / macOS Keychain / iOS Profile / Thunderbird) | Manual per recipient | Small fixed counterparty set (B2B, partner orgs) |
| **Issue recipient certs from a publicly-rooted CA** (commercial S/MIME issuer signs your CA, or you buy per-user S/MIME certs from a public issuer) | One-time cross-sign or per-user cost | Every MUA on the planet trusts the chain |

For most Hermes deployments the Internal CA is the right answer
(per-user public S/MIME costs $20-$80/yr/user); for high-volume
B2C senders the publicly-rooted route is sometimes worth the cost.

Hermes does not generate a CRL distribution URL on this page; CipherMail
maintains the revocation list internally and applies it when verifying
inbound S/MIME from local recipients. External recipients have no
automatic way to consume the CRL — revocation is effectively local-only
unless the operator publishes the CRL manually.

## CA file downloads (gated)

Each row's action column exposes a **Download Certificate** and
**Download Private Key** button. These are **disabled by default** —
downloading a CA private key off a web console is a high-risk
operation. To enable, set

```
ALLOW_CA_DOWNLOAD=yes
```

in `/opt/hermes/config/security.conf` on the host filesystem. This is
the same toggle pattern used by [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates#certificate-downloads-gated)
(`ALLOW_CERT_DOWNLOAD`) — read on every page load, surfaced as a
disabled-button + tooltip when off. When enabled, downloads stream
via a hidden iframe (`<iframe id="caDownloadFrame">`) so the page
preloader doesn't get stuck.

## Body-modification interaction with S/MIME

CipherMail-side S/MIME signing happens **after** the `hermes_body_milter`
disclaimer / signature / banner insertion (see
[Disclaimers — Behavior with S/MIME, PGP, and DKIM-signed mail](https://docs.deeztek.com/books/administrator-guide/page/disclaimers#behavior-with-smime-pgp-and-dkim-signed-mail)).
That means outbound mail signed by an Internal-CA-minted recipient cert
covers the **final body** the recipient sees — including any
disclaimer or banner Hermes appended. The body milter passes
already-S/MIME-signed mail through untouched, so end-to-end MUA-signed
mail (Outlook + per-user S/MIME) is never re-signed or invalidated.

This is the same ordering rationale that drives ARC sealing placement
(see [ARC Settings — Container and milter placement](https://docs.deeztek.com/books/administrator-guide/page/arc-settings#container-and-milter-placement)):
the cryptographic envelope is the last thing applied so it always
matches the bytes the recipient sees.

## Container and database touch-points

| Component | Container / path | Role |
|---|---|---|
| Page | `config/hermes/var/www/html/admin/2/view_internal_ca.cfm` (`hermes_commandbox`) | CRUD + DataTable + action router |
| CA tree | `/opt/hermes/CA/<sanitized-cn>/root_ca/` (`hermes_commandbox` bind mount) | OpenSSL working tree per CA |
| Templates | `/opt/hermes/templates/rootca_openssl.cnf` + `/opt/hermes/scripts/create_ca.sh` | Placeholder-substituted at create time |
| Trust store | `cm_certificates` + `cm_ctl` + `cm_ctl_cm_name_values` in `djigzo` DB (`hermes_db_server`) | CipherMail's view of the root CA |
| Engine | `hermes_ciphermail` (Java; CipherMail Community 5.x branded `djigzo`) | Signing / encryption / decryption engine; reached via `docker exec -i hermes_ciphermail /usr/bin/java -cp '/usr/share/djigzo/lib/*' mitm.application.djigzo.tools.CertStore` |
| Recipient certs | `recipient_certificates` + `external_recipient_certificates` in `hermes` DB | One-row-per-user, joined to a CA via `ca_id` |
| Security toggle | `/opt/hermes/config/security.conf` on host | `ALLOW_CA_DOWNLOAD=yes` to expose cert/key download buttons |

Every CipherMail interaction is **temp-script + docker exec** rather
than direct invocation — the `hermes_commandbox` container has no JVM
of its own; the CipherMail Java tooling lives in `hermes_ciphermail`
and is reached over the docker socket.

## Related

- [Encryption Settings](https://docs.deeztek.com/books/administrator-guide/page/encryption-settings) — outbound encryption policy (force / opportunistic / off); decides whether Hermes signs at all and whether it falls back to plaintext when no recipient key is available
- [External Recipients](https://docs.deeztek.com/books/administrator-guide/page/external-recipients) — per-counterparty key store; `external_recipient_certificates` rows pair with Internal-CA-issued or externally-issued S/MIME chains
- [PGP Key Servers](https://docs.deeztek.com/books/administrator-guide/page/pgp-key-servers) — sibling page for the PGP side of recipient key distribution
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — distinct TLS cert store for nginx / Postfix / Dovecot
- [Disclaimers](https://docs.deeztek.com/books/administrator-guide/page/disclaimers) — body-modification ordering vs the S/MIME signer
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — same milter-ordering pattern applied to inbound chain sealing
- **Advanced Settings** (sidebar link to `/ciphermail/`) — CipherMail's own admin UI, exposed for deep operations (CRL publishing, per-user policy tuning) not surfaced in the Hermes admin

# PGP Key Servers

# PGP Key Servers

Admin path: **Encryption > PGP Key Servers** (`view_pgp_key_servers.cfm`,
`inc/publish_pgp_keyring.cfm`).

This page maintains the **HKP keyserver publish list** — the set of
public OpenPGP keyservers Hermes will push (`gpg --send-keys`) recipient
public keys to when an admin clicks **Publish** on a keyring row in
**Encryption > External Recipients**. Each row is a hostname only
(no scheme, no port, no path) stored in the `pgp_keyservers` table.

> **Important: publish, not lookup.** Despite the page name, the
> keyserver list is currently **outbound-only**. Hermes does NOT
> auto-query these servers to fetch a recipient's PGP key at send time
> — recipient keys must be imported manually (paste-in or file upload)
> on **Encryption > External Recipients > PGP Keyrings**. The
> keyservers configured here are used solely by the **Publish** action
> in `inc/publish_pgp_keyring.cfm`, which pushes a key the operator
> already holds (typically the local CipherMail server's public key
> or a recipient's key that was imported and now needs broader
> distribution).

## What the page does

The page is a thin CRUD over a 3-column table:

| `pgp_keyservers` column | Purpose |
|---|---|
| `id` | PK |
| `keyserver` | Hostname only, e.g. `keys.openpgp.org` |
| `note` | Free-text label, e.g. "Primary keyserver" |

Three actions:

| Action | Form value | Effect |
|---|---|---|
| Add | `action=add` | Validates hostname via `IsValid("email", "bob@" & ks)` (rejects URLs and `host:port`), checks for duplicate `keyserver`, INSERTs the row |
| Single delete | `action=delete` with `delete_id` | DELETE one row by id |
| Bulk delete | `action=bulk_delete` with `selected_ids` (CSV) | DELETE every selected id in a loop |

The existing-servers card is a DataTable with select-all + per-row
checkboxes + a **Delete Selected** button. There is no per-row enable
flag, no protocol/port column, no priority ordering — every row in
the table is offered as a publish target in the modal on the keyring
page, indexed by `id`.

## What "publish" actually runs

When the operator clicks **Publish** on a keyring row at
**External Recipients > PGP Keyrings**, the `publish_pgp_keyring.cfm`
include does the following for each selected keyserver:

```
/usr/bin/gpg --homedir /opt/hermes/.gnupg/ \
             --keyserver <hostname-from-pgp_keyservers> \
             --send-keys <recipient-PGP-key-id>
```

The temp script is written to `/opt/hermes/tmp/<token>_publish_pgp_key.sh`,
chmod'd, executed, and deleted. The standard Hermes temp-script
pattern. The keyserver hostname is substituted via `REReplace` of the
`THE-KEY-SERVER` placeholder in `/opt/hermes/scripts/publish_pgp_key.sh`.

GPG itself picks the protocol — `gpg` defaults to `hkps://` (HKP over
TLS on tcp/443) for a bare hostname when the local `dirmngr` is
configured for it; otherwise it falls back to `hkp://` (tcp/11371).
Hermes does not pass an explicit scheme.

Failure modes the include recognizes (sets `session.m` and redirects):

| GPG stderr fragment | Meaning | `session.m` |
|---|---|---|
| `Server indicated a failure` | Keyserver rejected the upload (rate limit, policy, malformed key) | 22 |
| `No name` | Local GPG keyring has no user-id matching the requested key id | 23 |
| `Not found` | Local GPG keyring does not hold the requested key id | 24 |
| `Not a key ID` | The key id parameter was malformed | 25 |

A successful publish returns no recognized fragment and falls through
to the success branch.

## Recommended seed list

The default install seeds one row:

| Hostname | Note |
|---|---|
| `keyserver.ubuntu.com` | Ubuntu SKS OpenPGP Public Key Server |

Practical 2026 replacements / additions the operator should consider:

| Hostname | Network | Caveats |
|---|---|---|
| `keys.openpgp.org` | Identity-verified standalone (Hagrid) | Strips third-party signatures (no web-of-trust); requires email verification before a key becomes searchable by email address; **does not distribute revocation certificates the way SKS did** |
| `keyserver.ubuntu.com` | SKS-style federated | Was the last reliable SKS-network bridge; survives but is no longer broadly federated |
| `pgp.mit.edu` | Legacy SKS | Largely defunct in 2026 — uploads may not propagate; leave off unless legacy compatibility is required |
| `<your-org-keyserver>` | Internal HKP daemon (e.g. Hagrid) | Useful if the operator runs an authoritative keyserver for their own domain — same publish path |

The page does NOT validate keyserver reachability at add time; an
unreachable host simply produces a publish failure when the operator
clicks Publish later.

## What is NOT on this page

Several things an operator might reasonably expect from a "PGP Key
Servers" page that are intentionally elsewhere or absent:

| Expectation | Where it actually lives |
|---|---|
| Per-server enable/disable toggle | Not implemented — every row is a publish target |
| Search-order priority | Not applicable — publish iterates the explicit selection from the modal, not the full list |
| Inbound recipient-key auto-lookup at send time (`gpg --search-keys` / `recv-keys`) | Not implemented anywhere in Hermes; recipient keys must be imported manually on **External Recipients > PGP Keyrings** |
| Automatic refresh of imported keys (re-fetch + merge updates) | Not implemented; operators must re-import a key if a recipient rotates |
| DANE `OPENPGPKEY` DNS lookup | Not currently surfaced in the Hermes admin or CipherMail engine config |
| WKD (Web Key Directory) discovery at `https://<domain>/.well-known/openpgpkey/...` | Not currently surfaced in the Hermes admin or CipherMail engine config |
| HKP port override | Not on this page; GPG picks the port |
| Encryption policy decisions ("fail closed vs send plaintext if no key") | [Encryption Settings](https://docs.deeztek.com/books/administrator-guide/page/encryption-settings), not here |

The page is deliberately scoped to one job: **a list of HKP endpoints
the publish flow can push to.**

## When the operator should populate this list

Two practical scenarios:

1. **The organization wants its own gateway PGP key to be publicly
   discoverable.** Add the operator's preferred public keyserver(s),
   then publish the local CipherMail key from
   **External Recipients > PGP Keyrings**. External counterparties
   running `gpg --recv-keys` against the same keyserver can then pull
   it for encrypting mail back to Hermes-served users.
2. **A specific recipient has asked for their key (which the operator
   already holds locally) to be pushed somewhere centralized.** Less
   common — usually recipients self-publish — but the workflow
   supports it.

If the deployment never publishes keys outward (typical Community
deployments that use S/MIME exclusively, or PGP deployments that
exchange keys out-of-band via attachment), this page can remain empty
with no functional impact.

## Container and database touch-points

| Component | Location | Role |
|---|---|---|
| Page | `config/hermes/var/www/html/admin/2/view_pgp_key_servers.cfm` (`hermes_commandbox`) | CRUD UI |
| Publish include | `config/hermes/var/www/html/admin/2/inc/publish_pgp_keyring.cfm` (`hermes_commandbox`) | Builds + runs the temp `gpg --send-keys` script |
| Template script | `/opt/hermes/scripts/publish_pgp_key.sh` | Single line: `/usr/bin/gpg --homedir /opt/hermes/.gnupg/ --keyserver THE-KEY-SERVER --send-keys THE_KEY_ID 2>&1` |
| GPG home | `/opt/hermes/.gnupg/` (bind-mounted into `hermes_commandbox`) | Local GPG keyring holding the keys eligible for publish |
| Storage | `pgp_keyservers` in `hermes` DB (`hermes_db_server`) | The list itself |
| Engine | `hermes_ciphermail` (separate from publish — handles actual signing/encryption at send time) | NOT touched by this page; this page only manages the GPG outbound-publish list |

The publish flow runs **gpg on `hermes_commandbox`** (which has the
`/opt/hermes/.gnupg/` keyring bind-mounted) — not inside
`hermes_ciphermail`. CipherMail keeps its own per-recipient PGP store
in the `djigzo` DB for actual encryption/decryption operations.

## Related

- [External Recipients](https://docs.deeztek.com/books/administrator-guide/page/external-recipients) — per-counterparty key store; the **Publish** action that consumes this list lives on the keyring sub-page there
- [Encryption Settings](https://docs.deeztek.com/books/administrator-guide/page/encryption-settings) — outbound encryption policy that decides whether absence of a recipient PGP key blocks the message or falls through to plaintext
- [Internal CA](https://docs.deeztek.com/books/administrator-guide/page/internal-ca) — sibling page for the S/MIME side of recipient key issuance and trust
- **Advanced Settings** (sidebar link to `/ciphermail/`) — CipherMail's own admin UI for the deep PGP keyring operations the Hermes admin does not surface

# Authentication

# Credential Model

# Credential Model

This page describes how Hermes authenticates users across all of its surfaces — the web admin console, the user portal, Nextcloud (mail / calendar / contacts), and direct mail-protocol clients (IMAP, SMTP, CalDAV, CardDAV). The model is **uniform**: it works the same way whether a user is authenticated locally (against Hermes's built-in LDAP) or remotely (against an external Active Directory or LDAP server).

Understanding this model is a prerequisite for everything else in the Authentication chapter — app password management, MFA, OIDC SSO, and the iOS device setup wizard all build on it.

## The credentials a user has

Every Hermes user — local or remote — has up to **four distinct credentials**, each with a single, well-defined purpose. None of them is a "master password."

| Credential | Where it's stored | What it logs you into | How a user obtains it |
|---|---|---|---|
| **Web login password** | LDAP (local users) or external AD/LDAP (remote users) | `/users` portal · `/nc` (Nextcloud web UI) · `/admin` (admin console, admins only) | **Local auth:** set by admin at mailbox creation · changed by user in the portal · forgotten-password reset flow available. **Remote auth:** set and managed entirely in your external AD/LDAP — Hermes never sees or stores it. |
| **NC internal password** | Nextcloud's `oc_users` table | Nothing the user ever needs. It exists purely as a defense-in-depth backstop. | The user **never sees it**. It is set to a random value at mailbox creation and is never disclosed to anyone. |
| **"Hermes System" app password** | `app_passwords` table with `is_system = 1` | Used internally by the Nextcloud Mail app to authenticate IMAP/SMTP against Dovecot. Nothing the user ever needs to type. | Generated automatically at mailbox creation. Hidden from the user portal so users can't accidentally revoke it and break webmail. Admin can revoke + regenerate it from the per-mailbox admin page if needed. |
| **User app passwords** | `app_passwords` table with `is_system = 0` (read by Dovecot) AND `oc_authtoken` table (read by Nextcloud DAV) — same plaintext, both stores | IMAP, SMTP, CalDAV, CardDAV — i.e. mail/calendar/contact apps on devices | User generates them in the portal under **My App Passwords**. Each one is shown **once** and labelled per-device ("iPhone", "Thunderbird"). On create, the credential is registered with both Hermes and Nextcloud atomically; on revoke, both sides are removed. |

The two non-obvious parts here are the **NC internal password** (a back-channel-closing trick — see [§ Why a random NC internal password](#why-a-random-nc-internal-password)) and the **"Hermes System" app password** (admin-managed plumbing that lets webmail work without the user ever seeing a credential — see [§ The "Hermes System" app password](#the-hermes-system-app-password)).

## High-level flow diagram

```
                        ┌───────────────────────────────┐
                        │         WEB SURFACES          │
                        │   /users · /nc · /admin       │
                        └────────────────┬──────────────┘
                                         │
                                         ▼
                                    ┌─────────┐
                                    │ Authelia│  ── optional MFA challenge
                                    └────┬────┘
                                         │ LDAP bind
                                         ▼
                       ┌──────────────────────────────────┐
                       │  LDAP (local users)              │
                       │      OR                          │
                       │  External AD / LDAP (remote)     │
                       └──────────────────────────────────┘

                        ┌───────────────────────────────┐
                        │       MAIL PROTOCOLS          │
                        │  IMAP 993 · SMTP 465          │
                        └────────────────┬──────────────┘
                                         │
                                         ▼
                                 ┌──────────────┐
                                 │   Dovecot    │
                                 │  passdb lua  │
                                 │  (multi-row  │
                                 │   capable)   │
                                 └──────┬───────┘
                                        │ SELECT password
                                        │ WHERE revoked_at IS NULL
                                        ▼
                                ┌─────────────────┐
                                │  app_passwords  │
                                │  (Hermes DB)    │
                                └─────────────────┘

                        ┌───────────────────────────────┐
                        │        DAV PROTOCOLS          │
                        │  CalDAV · CardDAV (port 443)  │
                        └────────────────┬──────────────┘
                                         │
                                         ▼
                                ┌─────────────────┐
                                │   Nextcloud     │
                                │  oc_authtoken   │
                                │  (NC DB)        │
                                └─────────────────┘
```

Three surfaces, three back-ends. **No credential is shared across surfaces.** The web login password never reaches Dovecot. App passwords never reach Authelia. The NC internal password is never accepted by anything a user holds.

## Local-auth users vs. remote-auth users

The model applies identically to both. The only thing that changes is **what backs the LDAP bind** for the web login password.

### Local-auth user

```
Web login password lives in:   Hermes's built-in OpenLDAP
                               (cn=<user>,ou=users,dc=hermes,dc=local)

Set / changed by:              Admin (at create time) → user (in portal)

Reset path:                    Forgot password → email link → reset

App passwords:                 Generated and revoked by the user in /users
                               (no external dependency)

NC internal password:          Random at create. Admin can rotate via the
                               "Rotate NC Internal Password" action in the
                               mailbox detail view.
```

### Remote-auth user

```
Web login password lives in:   The customer's external AD or LDAP server.
                               Hermes's Authelia binds against it.
                               Hermes never stores or hashes it.

Set / changed by:              The customer's IT team in their own directory.
                               Hermes has no control surface for it.

Reset path:                    Customer's existing AD/LDAP reset workflow.
                               Hermes does not handle it.

App passwords:                 Generated and revoked by the user in /users —
                               same UI, same table, same lifecycle as local.
                               These ARE stored in Hermes; they have to be,
                               because IMAP/SMTP/DAV cannot speak the
                               protocols a corporate AD typically uses.

NC internal password:          Random at create. Same admin rotation path.
```

> **Key takeaway:** the operational surface a user touches — web login, app password mgmt, mail/calendar/contacts setup — looks **identical** between local and remote. The only difference is which directory their **login password** lives in.

## Why three credentials? Why not just one?

The single-credential approach (using the login password everywhere) has three problems:

1. **You can't revoke a single device.** A user loses their phone — to lock that phone out, you have to change the password on every device they own and re-enter it everywhere. With per-device app passwords, you revoke just the one row.

2. **You can't enforce MFA on devices.** IMAP, SMTP, CalDAV, and CardDAV cannot prompt for a TOTP code or a Duo Push. They authenticate with one round trip and one secret. So if MFA matters at all, it can only live at the web gate (Authelia). A separate device credential lets you keep MFA on the web while devices use a non-MFA bearer token.

3. **You can't safely embed a login password on a device.** The login password is the user's keys to the kingdom — email, AD, often other corporate apps. Every device that holds it is a leak risk. App passwords are scoped (mail/DAV only), revocable, and have no other privilege.

The three-credential model makes each problem disappear:

- **Revoke a device:** revoke its app password row. Done.
- **MFA at the gate:** Authelia challenges on web login. Devices use bearer-token-style app passwords and are never prompted.
- **No login password on devices:** devices receive a 30-character random string with mail/DAV scope only. If it leaks, it leaks one mail account, not the whole identity.

## The "Hermes System" app password

When a mailbox is created, Hermes mints **one** app password automatically and labels it `Hermes System`. It is stored in `app_passwords` with `is_system = 1`. It is used in exactly one place: as the IMAP credential that the **Nextcloud Mail webmail app** uses to read mail from Dovecot on the user's behalf.

### Why it exists

Without it, the user couldn't read mail through the `/nc` Mail app on day 1, because:

1. The user has no app password yet (they generate their own from the user portal).
2. Their login password no longer works for IMAP (it never reaches Dovecot under the new model).
3. NC Mail needs *some* credential to authenticate IMAP on the server side — there's no SSO from NC Mail down into Dovecot in this stack.

The Hermes System app password is that credential. The user never sees it, never types it, and never knows it exists.

### Where it lives — two stores, two roles

The Hermes System credential is held in two databases at the same time, each playing a different role:

| Location | Form | Used by | Role |
|---|---|---|---|
| `hermes.app_passwords` (`is_system = 1`) | ARGON2ID **hash** | Dovecot's Lua passdb | **Validation store.** Dovecot password-verifies incoming IMAP/SMTP attempts against this hash. This is "the credential" — its source of truth. |
| `nextcloud.oc_mail_accounts` | NC-encrypted **plaintext** | NC Mail (the webmail app inside Nextcloud) | **Operational copy.** NC Mail decrypts this on each poll and presents `email + plaintext` to Dovecot. It does *not* authenticate against `oc_authtoken` — `oc_mail_accounts` is a separate NC Mail table for stored mail-server credentials. |

```
       hermes.app_passwords (hashed, is_system=1)  ◄── validation
                  │
                  │  Dovecot's lua passdb reads here
                  ▼
        ┌──────────────────────┐
        │   hermes_dovecot     │ accepts IMAP/SMTP if hash matches
        └──────────────────────┘
                  ▲
                  │  IMAP/SMTP login attempt
                  │  (email + plaintext)
                  │
        ┌──────────────────────┐
        │   NC Mail (in NC)    │
        └──────────────────────┘
                  ▲
                  │  decrypts on each poll, sends to Dovecot
                  │
       nextcloud.oc_mail_accounts (encrypted plaintext)  ◄── stash
```

Two important consequences:

- **Deleting the `app_passwords` row instantly disables NC Mail.** NC Mail's stored plaintext becomes garbage that never validates. Dovecot rejects every poll attempt. Webmail starts erroring.
- **Hermes System is *not* an `oc_authtoken` row.** That's a different NC table used for NC's own session/DAV authentication. NC Mail authenticates *to Dovecot* (an external IMAP server from NC's perspective), not *to NC*, so it has no business in `oc_authtoken`. Only user-generated app passwords (Phase 1b dual-write) live in `oc_authtoken`.

### Why it's hidden from the user portal

If the user could see this row in **My App Passwords**, they might revoke it — at which point NC Mail breaks, webmail starts erroring, and the user has no obvious way to know what happened. The `is_system = 1` flag filters it out of the user's list. The admin sees it on the per-mailbox app-password page, marked with a "System" badge.

### Lifecycle

- **Created:** at mailbox create, by `add_mailbox_action.cfm` (step 4h).
- **Used:** every time NC Mail polls IMAP for that user.
- **Rotated:** if the admin uses the "Regenerate Hermes System" action (planned), or if the admin manually revokes + creates a new one and updates the NC Mail account config. Phase 1c polish.
- **Deleted:** when the mailbox is deleted (`delete_mailbox_action.cfm` removes all `app_passwords` rows for that user).

### How the Nextcloud `oc_authtoken` mirror works (Phase 1b)

User-generated app passwords are stored in **both** Hermes and Nextcloud at create time, with the same plaintext:

```
User clicks "Create App Password" labeled "iPhone"
        │
        ├─ occ user:auth-tokens:add → NC mints a fresh plaintext token
        │  (NC chooses the value; we don't pick it)
        │
        ├─ Token row gets renamed in oc_authtoken to "iPhone"
        │
        ├─ The new oc_authtoken.id is captured and stored in
        │  app_passwords.nc_token_id
        │
        ├─ The plaintext from NC is hashed via doveadm pw -s ARGON2ID
        │  and stored in app_passwords.password
        │
        └─ The plaintext is shown to the user once (one-shot callout)
                │
                ▼
        Same plaintext now authenticates:
          • IMAP/SMTP via Dovecot (lua passdb, app_passwords)
          • CalDAV/CardDAV via Nextcloud (oc_authtoken)
```

On revoke, the flow runs in the opposite direction:

```
User clicks Revoke
        │
        ├─ Look up the row's nc_token_id
        │
        ├─ occ user:auth-tokens:delete → removes the NC oc_authtoken row
        │  (DAV stops authenticating immediately)
        │
        └─ UPDATE app_passwords SET revoked_at = NOW()
           (Dovecot stops authenticating on next IMAP/SMTP attempt)
```

The "Hermes System" admin-managed app password (`is_system = 1`) is **not** mirrored to `oc_authtoken`. It exists purely as NC Mail's IMAP credential to Dovecot. Keeping it out of NC's auth store means it cannot be used for DAV access — defensive separation between admin plumbing and user-facing credentials.

## Why a random NC internal password

This is the part that is non-obvious. Walk through it carefully.

When a Nextcloud user logs in via OIDC SSO, NC internally provisions a row in `oc_users` for them. That row has a `password` column. NC needs *something* there because some NC subsystems (notably the DAV endpoints, before app-password enforcement) will accept a password against `oc_users.password` as a valid auth.

The natural temptation is to set this `oc_users.password` to either (a) the user's login password or (b) some predictable derivative of it. **Both are wrong**, for the same reason: it creates a silent back-channel.

Picture the failure mode:

```
1. User authenticates to /nc via OIDC. Web is fine.
2. User configures their phone's CalDAV with the login password.
3. NC's DAV endpoint, finding a matching oc_users.password, accepts it.
4. From that moment on, the login password is now embedded on a device.
5. User loses the phone. Org password leaks. Worse, neither admin nor user
   realises DAV ever silently "worked" — they assumed only OIDC was in play.
```

Setting `oc_users.password` to a **random value that no one knows** removes the back-channel:

```
1. User authenticates to /nc via OIDC. Web is still fine.
2. User configures their phone's CalDAV with the login password.
3. NC's DAV endpoint compares the supplied password against
   oc_users.password. No match (the stored value is random).
   Auth fails.
4. User is forced to either (a) generate a NC app password through the
   normal app-password flow, or (b) realise they need to use a different
   credential. Either way, the login password is NOT on the device.
```

The random value is generated at mailbox creation and is **never disclosed**. There is no UI to view it. The admin can **rotate** it (regenerate to a fresh random value) via the **Rotate NC Internal Password** action on the mailbox detail page. Rotation is a defense-in-depth move — it costs the operator one click and protects against the unlikely event of a NC password-hash store leak.

## Where each credential is checked

| User action | Surface | Credential checked | Backend |
|---|---|---|---|
| Open `/users` in a browser | Web | Web login password (+ MFA) | Authelia → LDAP |
| Open `/nc` in a browser | Web | Web login password (+ MFA) | Authelia → LDAP |
| Open `/admin` in a browser (admins) | Web | Web login password (+ MFA) | Authelia → LDAP |
| Mail.app fetches from IMAP 993 | Mail | User app password | Dovecot `passdb lua` → `app_passwords` (any non-revoked row) |
| Mail.app sends via SMTP 465 | Mail | User app password | Dovecot SASL → `passdb lua` → `app_passwords` |
| Nextcloud Mail webmail fetches IMAP | Mail (server-side) | "Hermes System" app password | Dovecot `passdb lua` → `app_passwords` (`is_system = 1` row, set up at provisioning) |
| Calendar.app sync via CalDAV 443 | DAV | User app password | Nextcloud → `oc_authtoken` (mirror of `app_passwords` row) |
| Contacts.app sync via CardDAV 443 | DAV | User app password | Nextcloud → `oc_authtoken` (mirror of `app_passwords` row) |
| Anything tries `oc_users.password` directly | (varies) | NC internal password | Nextcloud → `oc_users` (random — won't match anything a user holds) |

## Multi-active app passwords

A user can have **many** active app passwords at once. Each device gets its own row, with its own hash and its own label.

### How the iteration works

Stock Dovecot's `passdb sql` driver looks at the **first returned row only** — it does not try multiple hashes against the supplied password. To support per-device app passwords, Hermes uses a Lua-backed passdb script (`/etc/dovecot/auth_app_passwords.lua`) instead. The script:

1. Connects to MariaDB on each authentication attempt.
2. Selects all `app_passwords` rows for the user where `revoked_at IS NULL`.
3. Iterates the rows, calling Dovecot's `password_verify()` against each hash.
4. Returns success on the first match; failure if none match.

A successful match also updates `last_used_at` on the matching row (rate-limited to once per hour per row, to avoid hammering the DB on chatty IMAP IDLE clients).

### Why this matters

Multi-row support is what makes device swaps zero-downtime:

```
1. User creates "iPhone (new)" app password, enters into new phone.
2. New phone works (matches its own row).
3. Old phone keeps working in parallel (matches "iPhone (old)" row).
4. User revokes "iPhone (old)" — old phone immediately stops working.
5. No window during which either device is locked out.
```

A revocation is **instant**: setting `revoked_at` excludes the row from the next Lua lookup. There is no cache to wait on.

### What this costs

- One extra MySQL connection per authentication attempt (per-call open/close — safe under Dovecot's `use_worker = yes` mode).
- One extra `UPDATE` per matched authentication, throttled to once per hour per row.
- A new Docker image dependency (`dovecot-lua` + `lua-sql-mysql`) on the `hermes_dovecot` container.

These are small. The win — true per-device revocation — is large.

## What this model deliberately does NOT do

- **It does not support typing the login password on a phone for IMAP/SMTP.** That was never the goal. It is by design.
- **It does not let users set their own NC internal password.** That column is internal plumbing, not a user credential. There is no UI for it on either side.
- **It does not store anything that lets us recover a lost app password.** App passwords are shown once at creation. If a user loses one, they revoke it and create a new one. There is no "show me my app password again" path.

## What happens when an admin creates a new mailbox

The flow runs through `admin/2/inc/add_mailbox_action.cfm`. Most steps are identical for local-auth and remote-auth mailboxes; the one place they branch is step 2.

1. A row is inserted into `mailboxes` (Dovecot userdb — quota, active flag, etc.).

2. An LDAP entry is created in Hermes's directory:
   - **Local auth**: with the login password the admin entered in the form. This is the user's only login password going forward — Hermes is the source of truth for it.
   - **Remote auth**: as a stub entry with `seeAlso` pointing at the user's account in your external AD/LDAP. The user's login password lives in that external directory; Hermes never stores or hashes it.

3. The Nextcloud user account is provisioned with a **random** local password (`add_mailbox_action.cfm` step 4c). This password is never disclosed and never used by anyone — it exists only to close the back-channel risk where the user's login password could otherwise be silently accepted by NC's DAV endpoint. Same behavior for both auth types.

4. The "Hermes System" app password is minted (`is_system = 1`, label `Hermes System`) — step 4h. Same for both auth types.

5. The NC Mail account is provisioned with the Hermes System app password — step 4f. This is what makes webmail work on day 1 without the user having ever set up an app password themselves. Same for both auth types.

6. The welcome email is sent. **No credential is included in the email.** The email tells the user to sign in to the user portal with their login password (received out-of-band from the admin) and generate per-device app passwords from My App Passwords. The local-auth and remote-auth variants of the welcome email differ only in tone — local-auth says "your login password" while remote-auth specifies "your organization (AD/LDAP) password."

## Related documentation

- [My App Passwords (user)](https://docs.deeztek.com/books/user-guide/page/my-app-passwords) — what end users see and do
- [Email Server > Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — the per-mailbox detail page is where an admin revokes a user's app passwords or rotates the per-mailbox Nextcloud internal password

# Email Policies

# Disclaimers

# Disclaimers

_Pro Edition feature._ Maps to **Email Policies > Disclaimers** (`view_disclaimers.cfm`, `edit_disclaimer.cfm`, `disclaimer_delete.cfm`).

Hermes appends a configurable disclaimer to outbound mail at the gateway, with two scopes:

| Scope | Sender match | Use case |
|---|---|---|
| **Domain** | All senders in `@example.com` | Default org-wide compliance/legal language |
| **Relay Recipient** | Specific full address (e.g. `vendor@example.com`) | Per-relay-user override for tenants with extra regulatory language |

Most-specific match wins: a relay-recipient match is used before the domain default.

## Pipeline placement

Disclaimers are applied at SMTP receive time by the **`hermes_body_milter`** container, which Postfix consults as a milter alongside OpenDKIM and OpenDMARC.

```
External MTA / MUA submission
        │
        ▼
Postfix smtpd
   ├─ smtpd_milters chain (in order):
   │   1. OpenDKIM            (signs/verifies)
   │   2. OpenDMARC           (DMARC policy)
   │   3. hermes_body_milter  (THIS — disclaimers, signatures, banners)
   ▼
content_filter → Amavis    (unmodified path; sees the body milter's output)
   ▼
Ciphermail              (server-side S/MIME or PGP, if configured)
   ▼
Postfix :10026          (OpenDKIM signs the final composed body here)
   ▼
external
```

Body modification happens at smtpd time, **before** content_filter routes to Amavis. By the time Amavis sees the message, the disclaimer is already baked in. Amavis processes a normal-looking message; no internal-state coupling, no temp-file races.

OpenDKIM's outbound signing fires at the `:10026` re-injection — after both the body milter and Ciphermail. Hermes' own DKIM therefore always covers whatever the recipient ultimately receives. Ciphermail's server-side crypto also covers the disclaimer because Ciphermail runs after the milter.

## Behavior with S/MIME, PGP, and DKIM-signed mail

The behavior depends on **who** signed/encrypted the message and **when** in the pipeline.

### Server-side: signed/encrypted by Ciphermail — disclaimer is applied

Ciphermail runs **after** the body milter. Mail arrives at the milter as plaintext, the disclaimer is appended, then Ciphermail signs or encrypts the modified body. The recipient sees a valid signature **and** the disclaimer. No conflict.

### Client-side: signed/encrypted by the user's MUA — disclaimer is skipped

Mail signed in Outlook (S/MIME) or Thunderbird+Enigmail (PGP) arrives at the gateway with the cryptographic envelope already sealed. Modifying the body would either invalidate the signature or mangle the ciphertext.

The body milter detects the following patterns in the headers (or first 32 KB of the body) and exits unchanged when any matches:

| Pattern matched | Meaning |
|---|---|
| `Content-Type: multipart/signed; protocol="application/pkcs7-signature"` | S/MIME detached signature |
| `Content-Type: application/pkcs7-mime` | S/MIME opaque-signed or enveloped |
| `Content-Type: multipart/signed; protocol="application/pgp-signature"` | PGP/MIME detached signature |
| `Content-Type: multipart/encrypted; protocol="application/pgp-encrypted"` | PGP/MIME encrypted |
| `-----BEGIN PGP SIGNED MESSAGE-----` in body | PGP inline-signed |
| `-----BEGIN PGP MESSAGE-----` in body | PGP inline-encrypted |

When any of those match, the body is left untouched, the signature stays valid, the user's legal-text expectations are preserved (their MUA template is already in the body), and the gateway gets out of the way.

> **Operational consequence.** A site whose users sign client-side will not get gateway disclaimers on those specific signed messages — by design. If org-wide legal text on **all** outbound is mandatory, the only safe pattern is server-side signing in Ciphermail with the disclaimer applied first.

### DKIM: Hermes-signed mail is fine; upstream-signed mail is skipped

OpenDKIM signs at the Postfix `:10026` re-injection step — **after** the body milter. So Hermes' own DKIM signature always covers the recipient's view of the message (with disclaimer baked in). No conflict.

The risk is mail that arrives at Hermes **already DKIM-signed by an upstream MTA** — typically a relay user whose own mail server signs before forwarding through us. Modifying that body would invalidate the upstream signature at the recipient.

The body milter treats a pre-existing `DKIM-Signature:` header the same way as a sealed S/MIME or PGP envelope and skips the disclaimer. Since Hermes' own DKIM signs at `:10026` (downstream of this milter), any DKIM-Signature header present at the milter's point in the pipeline came from somewhere upstream of Hermes.

## Reply-chain handling — no dedup, by design

The milter does **not** detect or skip messages that already carry a previous disclaimer in their quoted history. Every outbound message gets a fresh disclaimer applied — including replies inside a long thread.

This matches industry norm: commercial server-side disclaimer / signature platforms (Exclaimer, Crossware, CodeTwo, Microsoft 365 transport rules) all stamp every outbound without dedup. The reasoning:

- **Compliance.** Many regulatory regimes (HIPAA email confidentiality, GDPR data-controller notices, financial-services disclosure) treat each transmission as requiring its own disclaimer. Stamping only the first message in a thread arguably leaves later replies non-compliant.
- **Self-contained messages.** If a recipient forwards a reply (with quoted history) to a third party, the disclaimer is preserved per-message in the forwarded text.
- **Predictable behavior.** Operators don't have to explain "sometimes the disclaimer shows, sometimes it doesn't."
- **Cosmetic concern is weak.** Modern MUAs (Gmail, Outlook, Apple Mail) collapse quoted history by default, so stacked disclaimers in long threads are rarely visible to readers.

Earlier iterations of #214 included a sentinel-marker dedup mechanism (`[HD]` / `<!-- HERMES_DISCLAIMER_V1 -->`). That was removed during DEV testing in favor of the industry-norm pattern.

## Position: append vs prepend

The schema and UI both expose `position = append | prepend`, but **v1 honors append only.** Prepend is tracked as a v2 enhancement.

## Failure semantics

The body milter is **graceful-degradation** by design. Postfix's `milter_default_action = accept` means:

- Milter container down or unreachable → mail flows **unmodified** (missed disclaimer, but no delivery outage)
- Map file unreadable → no entries match → all mail flows unmodified
- Modifier raises an exception → caught and logged → mail flows unmodified
- altermime / parse errors → caught and logged → mail flows unmodified

In every failure case, mail keeps flowing. Worst case is a missed disclaimer, never lost mail. Compare the legacy "modify in amavis hook" approach (#214 Phase 3 v1, retired) which silently dropped messages when the in-place body modification desynced amavis's internal state.

## Files generated on save/delete

The CFML include `inc/disclaimer_write_and_reload.cfm` runs after every save or delete and rewrites the entire on-disk state from the `disclaimers` table:

```
/etc/hermes/body_milter/disclaimers/disclaimer_by_sender   sender → option map
/etc/hermes/body_milter/disclaimers/files/<option>/
    body.txt          plain-text disclaimer
    body.html         html disclaimer (may have <img src="cid:..."> refs)
    images/
        1.png         per-disclaimer inline images (#230)
        2.jpg
        ...
```

Where `<option>` is `domain_<safe>` or `relay_<safe>` (non-alphanumeric chars in the source key are replaced with `_`).

Each disclaimer gets its own subdirectory. The files directory is wiped (per-option subdirectories deleted recursively, but the parent `files/` directory and its `.gitkeep` are preserved) and rewritten on every save. There is no incremental update — this guarantees deleted rows and renamed scope keys never leave stale files (or stale image binaries) behind.

**No reload step needed.** The body milter mtime-watches each map file on every message and reloads when it changes. The CFML `cffile` write to the map file is enough to make the change take effect on the next message processed by the milter.

## Inline images (#230)

Admins can paste or upload images directly into the Quill editor when authoring a disclaimer. Supported formats: **PNG, JPEG, GIF**. SVG and WebP are explicitly rejected (security and recipient-compatibility reasons). Limits enforced at save time:

- **5 images max** per disclaimer
- **200 KB per image** (after base64 decode)
- **1 MB total** across all images in a single disclaimer

If any limit is exceeded, the save is rejected with a specific error explaining what failed. Admins can reduce image count or size and re-save.

**How it works:**

1. Quill embeds pasted/uploaded images as base64 inline `<img src="data:image/...;base64,...">` in the HTML body. The base64 representation is what's stored in the `disclaimers.body_html` column.
2. At save time, the regenerator parses `body_html` for `data:` URLs, decodes each base64 blob, writes the binary as `<option>/images/<N>.<ext>`, and rewrites the HTML in `<option>/body.html` to use `<img src="cid:disclaimer_<option>_img_<N>">` references.
3. At message-send time, the body milter reads `body.html`, walks `<img src="cid:...">` references, and attaches each referenced image as an `image/<format>` MIME part with `Content-ID: <disclaimer_<option>_img_<N>>` and `Content-Disposition: inline`.
4. The milter wraps the message as `multipart/related` so the recipient MUA resolves cid references against the inline parts.

**MIME structure transformation** (representative example):

```
Original outbound:
  multipart/alternative
    text/plain
    text/html (no images)

After milter (with disclaimer including 1 image):
  multipart/related
    multipart/alternative
      text/plain  (with text disclaimer appended; images omitted from text)
      text/html   (with html disclaimer + <img src="cid:...">)
    image/png
      Content-ID: <disclaimer_..._img_1>
      Content-Disposition: inline
```

This structure renders inline in all major MUAs (Gmail, Outlook, Apple Mail, Thunderbird, mobile clients).

**The plain-text version of the disclaimer omits images entirely** — base64 inline images don't translate to text, and recipients viewing the message in plain-text mode see the disclaimer text without any image markers.

**Hermes' own DKIM signature covers the modified body** (including the multipart/related wrap and image parts), because OpenDKIM signs at the postfix `:10026` re-injection step — downstream of the body milter. The signature validates against what the recipient receives.

## Auto-derive of plain-text part

The Quill editor on `edit_disclaimer.cfm` drives `body_html`. By default the plain-text part shipped to recipients with a non-HTML MUA is auto-derived from the HTML on save: `<br>`, `</p>`, `</li>` become newlines, all other tags are stripped, runs of 3+ newlines collapse to 2.

Admins who need character-perfect plain text different from the auto-strip (e.g. for regulated industries) can toggle **Edit plain-text version separately** to expose a second editor. When set, `body_text` is shipped verbatim instead of derived.

## Disabled rows

Rows with `enabled = 0` are skipped entirely on regen — no files written, no map entry. The milter never matches that scope until the row is re-enabled.

## Internal-only mail

v1 does **not** suppress disclaimers for internal-only mail (sender + all recipients in `@local_domains`). Domain disclaimers will be applied to internal mail in the same domain. If this is a problem for your install, file a feature request to add an internal-only bypass.

## Why a separate milter and not an amavis hook

Earlier #214 iterations attempted to dispatch the disclaimer from inside an amavisd-new `Custom.pm` `before_send` hook, calling altermime via `system()` on the temp file amavis was managing. amavisd-new 2.13 caused two problems: the legacy `@disclaimer_options_bysender_maps` dispatch path was removed (variables still parse but no code reads them), and the `before_send` hook documentation says "may modify mail" but in practice in-place body modification desynchronizes amavis's internal MIME state and silently loses mail.

The body milter approach moves the body-modification step out of amavis entirely. amavis's role is unchanged from before #214 ever existed; the milter sits in postfix's smtpd_milters chain alongside OpenDKIM and OpenDMARC, the same architectural pattern Hermes already uses for body-touching policy enforcement. amavis is fully decoupled from the disclaimer feature, which means amavis upgrades and the disclaimer feature evolve independently.

This same milter container is intended to host:

- **#226 User Signatures** (per-mailbox personal text from LDAP attributes or user-portal editor)
- **#228 External Sender Banner** (warning banner on inbound external mail)
- **Future Link Guard** (URL rewriting through a click-through endpoint)

Each is a `Modifier` subclass in `/usr/local/bin/hermes-body-milter` registered in the `MODIFIERS` list. The dispatcher is unchanged.

# External Banner

# External Banner

Maps to **Email Policies > External Banner** (`view_external_banners.cfm`, `edit_external_banner.cfm`, `external_banner_delete.cfm`). Available on both **Community** and **Pro** editions &mdash; phishing protection is a baseline security feature, not a Pro upsell.

Hermes prepends (or optionally appends) a warning banner to **inbound** mail from external senders destined for a local recipient. The banner is injected into the message body itself, so every MUA &mdash; webmail, Outlook, Apple Mail, mobile clients &mdash; renders it without relying on transport rules or recipient-side configuration. Tracked as #228.

## Scope

| Scope | Recipient match | Use case |
|---|---|---|
| **System default** | All recipient domains (no override) | Single banner used everywhere; recommended starting point |
| **Per-recipient-domain** | Specific local mailbox domain (e.g. `legal.example.com`) | Different copy or compliance language for one domain |

Resolution at message time, in the body milter's `ExternalBannerModifier`:

1. Look up the first **local recipient**'s domain in `/etc/hermes/body_milter/banners/banner_by_recipient_domain`.
2. If a matching row exists, use it.
3. Otherwise fall back to the `_default` system-wide entry.
4. Otherwise no banner is applied.

Only the first local recipient is consulted &mdash; mixed-domain envelopes get the banner of the first local recipient encountered. This keeps the modification deterministic regardless of envelope ordering.

The `recipient_domain` field is **locked after creation**. Delete and re-create the row to change scope.

## What counts as "external"

The body milter uses Postfix's `/etc/postfix/relay_domains` file as the source of truth for "local". A message is considered **inbound from an external sender** when:

- The `MAIL FROM` sender domain is **not** in `relay_domains`, AND
- At least one `RCPT TO` recipient domain **is** in `relay_domains`.

Internal-to-internal mail (sender + all recipients local) is classified as `direction = internal` and the banner is **not** applied. There is no separate allowlist of "trusted partner" external senders today &mdash; every external sender to a local recipient triggers the banner if one is configured for that recipient's domain.

## Pipeline placement

The banner is injected at SMTP receive time by the **`hermes_body_milter`** container, the same container that emits outbound disclaimers ([disclaimers.md](https://docs.deeztek.com/books/administrator-guide/page/disclaimers)) and organizational signatures ([organizational-signatures.md](https://docs.deeztek.com/books/administrator-guide/page/organizational-signatures)). The milter listens on `inet:hermes_body_milter:8893` and Postfix consults it as part of `smtpd_milters`.

```
Inbound external MTA
        |
        v
Postfix smtpd
   +- smtpd_milters chain (in order):
   |    1. OpenDKIM            (verifies upstream DKIM signature)
   |    2. OpenDMARC           (DMARC policy + ARC verification)
   |    3. hermes_body_milter  (THIS -- banner prepended here)
   |       --> Authentication-Results header has already been written
   |           by OpenDKIM/OpenDMARC BEFORE the banner touches the body
   v
content_filter --> Amavis    (sees the banner-prepended body)
   v
Ciphermail              (server-side S/MIME or PGP, if configured)
   v
Postfix :10026          (multi-instance OpenDKIM re-signs the final body)
   v
Local delivery (Dovecot LMTP)
```

Key ordering points:

- **OpenDKIM verifies first.** The upstream sender's DKIM verdict is captured in `Authentication-Results:` headers **before** the banner is injected. The header is preserved on the message; the banner does not retroactively change what OpenDKIM saw at smtpd time.
- **Amavis sees the modified body.** Spam scoring runs against the banner-prepended message. This is intentional &mdash; the banner content is short and stable and does not skew SpamAssassin scores in practice.
- **Hermes' downstream re-sign covers the modified body.** The multi-instance OpenDKIM at `:10026` (#232) signs after Ciphermail rebuild, so the final outgoing-to-Dovecot body is covered by Hermes' own signature.

## Behavior with signed and encrypted mail

The modifier inherits the same skip rules as [Disclaimers](https://docs.deeztek.com/books/administrator-guide/page/disclaimers#behavior-with-smime-pgp-and-dkim-signed-mail) for sealed envelopes:

| Pattern matched | Meaning | Banner action |
|---|---|---|
| `Content-Type: multipart/signed; protocol="application/pkcs7-signature"` | S/MIME detached | **Skip** |
| `Content-Type: application/pkcs7-mime` | S/MIME opaque/enveloped | **Skip** |
| `Content-Type: multipart/signed; protocol="application/pgp-signature"` | PGP/MIME detached | **Skip** |
| `Content-Type: multipart/encrypted; protocol="application/pgp-encrypted"` | PGP/MIME encrypted | **Skip** |
| `-----BEGIN PGP SIGNED MESSAGE-----` in body | PGP inline-signed | **Skip** |
| `-----BEGIN PGP MESSAGE-----` in body | PGP inline-encrypted | **Skip** |
| Pre-existing `DKIM-Signature:` header on inbound mail | Upstream DKIM signed | **Modify anyway** (see below) |

The corresponding flags on `ExternalBannerModifier` are `skip_on_signed = True`, `skip_on_pgp_inline = True`, `skip_on_dkim = False`.

### Why the banner does NOT skip on upstream DKIM

About 95% of inbound mail today carries a `DKIM-Signature:` header. If the banner skipped on DKIM, the feature would be effectively inert &mdash; the warning would only land on the unsigned minority that needs it least.

Hermes already records the upstream DKIM verdict in `Authentication-Results:` before modifying the body. Recipients overwhelmingly read mail through Dovecot/IMAP and the recipient MUA does not re-verify upstream DKIM. The banner is therefore safe in the common case.

The narrower edge case &mdash; a recipient who forwards Hermes-banner'd mail to a downstream MX that **does** re-verify upstream DKIM &mdash; is addressed by [ARC sealing](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) (#229). Hermes' ARC seal at `:10026` records `cv=fail` for the upstream chain (because we modified the body), but the seal itself is mathematically valid and the downstream MX can trust Hermes' ARC verdict if Hermes is on its allowlist. See [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) for the full discussion of the `cv=fail`-by-design pattern.

> **Operational consequence.** Banner injection breaks the original sender's DKIM body hash and any upstream ARC body hash. **This is by design.** Hermes is the authoritative auth boundary for the domains it relays; customer downstream MX servers must allowlist Hermes and accept its delivered mail without re-running DKIM/SPF/DMARC/ARC. A downstream MX that re-verifies upstream auth on mail Hermes forwards is misconfigured &mdash; cross-ref [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings), [DKIM Settings](https://docs.deeztek.com/books/administrator-guide/page/dkim-settings), and [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings).

## Position: prepend vs append

| Position | Behavior | Recommendation |
|---|---|---|
| **Top (`prepend`)** | Banner becomes the first child of the message body (above any quoted history) | **Industry standard** &mdash; users see the warning before reading any content |
| **Bottom (`append`)** | Banner is appended after the user-visible body | Available for sites that prefer it; rarely used |

Both positions are implemented end to end (unlike Disclaimers, where only `append` is honored in v1). HTML prepend is done with BeautifulSoup: the banner fragment is inserted as the first child of `<body>` when present, otherwise prepended to the root.

## Templates

Banners use a **server-side template gallery**, not a free-form WYSIWYG editor. Quill 2.x's HTML normalization strips inline styles that Gmail and Outlook need (the same problem hit on Organizational Signatures #226 Phase 2 and on this feature), so admins pick a template and fill in form fields; the server renders pixel-perfect HTML at save time.

Bundled templates (each `inc/external_banner_templates/<key>.cfm`):

| Template key | Display name | When to pick it |
|---|---|---|
| `warning_yellow` | **Warning Yellow** | Default. Yellow background with orange accent. Matches Microsoft 365 / Mimecast banner style most users recognize |
| `critical_red` | **Critical Red** | Red background, white text. Phishing-prone industries or post-incident periods where alert level needs to be raised |
| `subtle_info` | **Subtle Info** | Light gray with blue accent. Less alarming for high-volume inbound (support/sales) where alert fatigue is a concern |
| `plain_text` | **Plain Text** | Bold prefix + text, no background or border. Maximum cross-MUA compatibility, including text-only clients |

All four templates expose the same field set:

| Field | Type | Default | Notes |
|---|---|---|---|
| `prefix` | text | `[EXTERNAL]` | Short tag rendered bold at the start. Plain ASCII recommended for Outlook |
| `headline` | text | "This message originated from outside your organization." | First line, regular weight |
| `body` | text | "Do not click links or open attachments unless you recognize the sender..." | Second line, smaller text |
| `show_learn_more` | checkbox | `false` | Reveals the next two fields |
| `learn_more_url` | url | _empty_ | Optional link to internal phishing-awareness training or wiki |
| `learn_more_label` | text | "Learn more about phishing" | Visible label for the learn-more link |

All templates emit **table-based HTML with `bgcolor=` attributes** so Outlook (which strips inline CSS but honors deprecated HTML attributes) renders the banner correctly. Inline styles are belt-and-suspenders for Gmail, Apple Mail, and mobile clients.

The edit page renders a live preview in an iframe via `inc/render_external_banner_preview.cfm` so the admin sees exactly what `save_external_banner_action.cfm` will store.

## Files generated on save/delete

`inc/external_banner_write_and_reload.cfm` runs after every save or delete and rewrites the entire on-disk state from the `external_banners` table:

```
/etc/hermes/body_milter/banners/banner_by_recipient_domain
    <recipient_domain>\t<option>
    _default\t<option>          special key, system-wide fallback

/etc/hermes/body_milter/banners/files/<option>/
    body.txt          plain-text banner (auto-derived at save)
    body.html         pre-rendered html banner
    position          "prepend" or "append" sidecar file
    images/           per-banner inline images (#230 cid pattern)
        1.png
        2.jpg
        ...
```

Where `<option>` is:

- `banner_default` for the system-wide row (NULL `recipient_domain`)
- `banner_<sanitized_recipient_domain>` for per-domain overrides (non-alphanumeric characters replaced with `_`)

The `files/` subdirectory is wiped on every regen (per-banner subdirs deleted recursively; the `.gitkeep` is preserved). This guarantees deleted rows and renamed scopes never leave stale files behind.

**No reload step needed.** The body milter mtime-stats each map file on every message and reloads automatically when its mtime changes. The CFML `cffile` write to the map file is enough to make the change take effect on the next message.

## Plain-text part

The HTML body stored in `external_banners.body_html` is rendered server-side from the chosen template. The plain-text counterpart in `body_text` is auto-derived at save time:

- `<br>` becomes a newline
- `</p>`, `</li>`, `</tr>`, `</td>`, `</div>` become newlines
- All remaining tags are stripped
- Runs of 3+ newlines collapse to 2

The plain-text version is shipped to recipients viewing the message as `text/plain`. Inline images are omitted from the plain-text part &mdash; data URLs don't translate to text and recipients in text mode see the banner copy without image markers.

## Inline images (#230)

The banner modifier inherits the `#230` cid inline-image pattern from Disclaimers. If a template's HTML contains `<img src="cid:banner_<option>_img_<N>">` references, the body milter:

1. Loads matching `images/<N>.<ext>` files from the option directory.
2. Attaches each as an `image/<format>` MIME part with `Content-ID: <banner_..._img_N>` and `Content-Disposition: inline`.
3. Wraps the message as `multipart/related` so MUAs resolve cid references against the inline parts.

The cid prefix is `banner_` so banner images cannot collide with `disclaimer_` or `signature_` cids inside the same composed message (the three modifiers can all add images to the same outbound; namespacing keeps them separate).

The bundled templates do not currently use inline images &mdash; banners are pure text. The infrastructure is present for future template additions (logo, warning icon, etc.).

## Failure semantics

The body milter is **graceful-degradation** by design. Postfix's `milter_default_action = accept` means:

- Milter container down or unreachable -> mail flows **unmodified** (missed banner, no delivery outage)
- Map file unreadable -> no entries match -> all mail flows unmodified
- Per-option files missing -> log + skip the modify -> mail flows unmodified
- MIME parse exception -> caught and logged -> mail flows unmodified
- Modifier raises any other exception -> caught and logged -> mail flows unmodified

In every failure case, mail keeps flowing. Worst case is a missed banner, never lost mail. Compare the legacy "modify in amavis hook" approach (#214 Phase 3 v1, retired) which silently dropped messages when the in-place body modification desynced amavis's internal state.

## Disabled rows

Rows with `enabled = 0` are skipped entirely during regen &mdash; no files written, no map entry. The milter never matches that scope until the row is re-enabled. Useful for staging copy changes before going live (build the new row disabled, preview it on `edit_external_banner.cfm`, flip the switch when ready).

## Schema

```sql
CREATE TABLE IF NOT EXISTS external_banners (
  id               int(11)         NOT NULL AUTO_INCREMENT,
  recipient_domain varchar(255)    DEFAULT NULL,                -- NULL = system default
  template_key     varchar(64)     NOT NULL DEFAULT 'warning_yellow',
  fields_json      longtext        DEFAULT NULL,                 -- form values for re-edit
  body_text        longtext        DEFAULT NULL,                 -- auto-derived plain text
  body_html        longtext        NOT NULL,                     -- pre-rendered html
  position         enum('prepend','append') NOT NULL DEFAULT 'prepend',
  enabled          tinyint(3)      NOT NULL DEFAULT 1,
  updated_at       timestamp       NULL DEFAULT current_timestamp() ON UPDATE current_timestamp(),
  PRIMARY KEY (id),
  UNIQUE KEY uk_recipient_domain (recipient_domain)
);
```

The `UNIQUE KEY` on `recipient_domain` ensures only one row per recipient domain (and at most one system-default row where `recipient_domain IS NULL`). The `fields_json` blob stores the original form values so reopening the editor restores exactly what the admin typed; `body_html` is the rendered output the milter actually ships.

## Verifying it works

The banner appears in the message body, so the easiest verification is to send an inbound message from an external account to a local mailbox and view the result in any MUA (webmail, Outlook, Apple Mail). Beyond that:

- **Body milter logs** &mdash; the modifier logs `external_banner applied: option=<name> position=<prepend|append> plain=<n> html=<n>` per modified message. Surface with `docker logs hermes_body_milter` or via [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs).
- **`Authentication-Results:` header** is preserved from upstream and visible in the recipient's "view source"; this confirms OpenDKIM ran **before** the banner.
- **`ARC-Seal: ... cv=fail`** in the outgoing message confirms the body was modified after the upstream chain &mdash; expected behavior, cross-ref [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings).

## Related

- [Disclaimers](https://docs.deeztek.com/books/administrator-guide/page/disclaimers) &mdash; the outbound counterpart; same `hermes_body_milter` container, parallel design (sender-keyed instead of recipient-keyed)
- [Organizational Signatures](https://docs.deeztek.com/books/administrator-guide/page/organizational-signatures) &mdash; second outbound modifier in the same container, with per-recipient resolution
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) &mdash; full explanation of `cv=fail` after body modification and the Hermes-as-auth-boundary model
- [DKIM Settings](https://docs.deeztek.com/books/administrator-guide/page/dkim-settings), [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) &mdash; upstream-verdict context preserved in `Authentication-Results`
- [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) &mdash; local mailbox-hosting domains drive the per-domain dropdown on `edit_external_banner.cfm`
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) &mdash; surface the body-milter log stream for troubleshooting

# Link Guard

# Link Guard

_Pro Edition feature._ Maps to **Email Policies > Link Guard** (`view_linkguard.cfm`, `inc/linkguard_write_and_reload.cfm`).

Link Guard provides **time-of-click protection** for inbound mail. Links in delivered messages are rewritten to point at a Hermes redirect endpoint; when a recipient clicks, Hermes evaluates the destination's reputation **at that moment** and decides — instantly — whether to allow, warn, or block. This closes the gap that delivery-time scanning misses: a link that is clean when the message arrives but is weaponized hours or days later.

Protection travels with the link. It works in the inbox, after a forward to a colleague, or when the message is opened days later on a phone — because the verdict is computed on click, not on delivery.

## Components

Link Guard spans three containers plus the admin console:

| Component | Role |
|---|---|
| **`hermes_body_milter`** | Rewrites inbound links at SMTP receive time (`LinkGuardModifier`) and restores original links on outbound replies/forwards (`LinkGuardRestoreModifier`). |
| **`hermes_linkguard`** | The verdict + redirect engine. Serves the public `/lg/` click endpoint and a console-only management API. Holds the operational SQLite store (verdict cache, feeds, click log). |
| **`hermes_nginx`** | Reverse-proxies `/lg/` from the public console host to the Link Guard container's public port. |
| **Admin console** | `view_linkguard.cfm` page; on save, `inc/linkguard_write_and_reload.cfm` pushes settings, scope, URL rules, and HMAC keys to the engine and reloads the milter maps. |

The Link Guard container exposes a **hardened two-port split**:

| Port | Surface | Exposure |
|---|---|---|
| **8894** (public) | `GET /lg/?t=<token>`, `POST /lg/proceed`, `GET /healthz` | nginx-proxied; reachable by recipients clicking links |
| **8895** (mgmt) | `POST /api/config`, `POST /api/keys`, `POST /api/feed-refresh`, `GET /api/stats` | console-only; never exposed publicly |

The public surface can never push config or read keys; the management surface is never reachable from the internet.

## Pipeline placement

Link Guard's two body-modification steps run in the **`hermes_body_milter`** container, the same milter Postfix consults for disclaimers, signatures, and banners.

```
INBOUND (rewrite)                          OUTBOUND (restore)
External MTA ──► Postfix smtpd             User reply/forward ──► Postfix smtpd
        │                                          │
        ▼                                          ▼
   smtpd_milters chain:                       smtpd_milters chain:
     1. OpenDKIM                                1. OpenDKIM
     2. OpenDMARC                               2. OpenDMARC
     3. hermes_body_milter                      3. hermes_body_milter
          └─ LinkGuardModifier:                      └─ LinkGuardRestoreModifier:
             rewrite links ──► /lg/ token              unwrap /lg/ tokens ──► original URLs
        │                                          │
        ▼                                          ▼
   Amavis ──► Ciphermail ──► deliver           Amavis ──► deliver to external
```

Rewriting happens at smtpd time, before content filtering. Hermes' own DKIM signs at the Postfix `:10026` re-injection (downstream of the milter), so the signature always covers the rewritten body the recipient receives. Inbound mail that arrives already DKIM-signed, S/MIME-signed, or PGP-sealed is **skipped** — the same envelope-detection logic the disclaimer feature uses — so Link Guard never breaks an existing signature.

## The click flow

```
Recipient clicks rewritten link
        │
        ▼
GET https://<console-host>/lg/?t=<token>   (nginx ──► linkguard :8894)
        │
        ├─ token invalid / expired ──► block page
        ▼
   resolve original URL from token
        │
        ▼
   verdict pipeline (see below) ──► {clean | suspicious | malicious}
        │
        ▼
   admin action for that tier:
     clean      ──► 302 redirect to the real URL   (default)
     suspicious ──► warning interstitial (resolved host shown; user may proceed)
     malicious  ──► block page (hard block, or block_override allowing proceed)
```

Every click is logged (recipient domain, URL hash, resolved host, verdict, source, action taken, client IP) for the reporting dashboard.

## Verdict pipeline

`verdict.resolve(url, recipient_domain)` evaluates layers in **precedence order** and returns the first match:

| # | Layer | Result | Notes |
|---|---|---|---|
| 1 | **Admin blocklist** | malicious | Operator-curated, console-managed |
| 2 | **Admin allowlist** | clean | Operator-curated; trumps feeds and heuristics |
| 3 | **Verdict cache** | cached result | Avoids re-running heuristics / re-hitting external APIs |
| 4 | **Open-redirect extraction** | escalate | Embedded target in an open-redirect param / nested URL is resolved and inherits its verdict (no fetch) |
| 5 | **Local feeds** | malicious | URLhaus / OpenPhish, stored in SQLite; only ever escalate |
| 6 | **Heuristics** | suspicious | Lookalike/punycode, IP-literal host, `@` in authority, known shorteners, excessive subdomains, **abused cloud-storage/redirector hosts** |
| 7 | **GSB / VirusTotal** | malicious | Optional, admin-supplied keys; string-reputation lookups, cached. Also escalates a warn-tier (step 6) link to a **hard block** when flagged |
| 8 | **Guarded redirect-follow** | escalate | **Optional** (admin toggle): follow the `30x` chain under SSRF guards, verdict the final destination; also runs proactively on a step-6 warn so a malicious destination becomes a hard block |
| 9 | **Default** | clean | Nothing flagged it |

**Steps 1–7 never fetch the target URL** — every reputation check sends the URL only as a *string* (Google Safe Browsing, VirusTotal). The **only** layer that makes an outbound request is the optional **guarded redirect-follow** (step 8), and only when an admin enables it; every hop is SSRF-fenced (see [Redirect detection](#redirect-detection) below). Local feeds only ever escalate a link to malicious; they never auto-allow.

URL shorteners are flagged **suspicious** (warn), not blocked — a shortener hides its real destination, which is exactly what time-of-click protection exists to surface. The warning interstitial shows the resolved host so the user can make an informed choice. The shortener set is a curated list of **generic, anyone-shortens-anything** services (referenced from the maintained [`PeterDaveHello/url-shorteners`](https://github.com/PeterDaveHello/url-shorteners) list); **branded** corporate shorteners (`aka.ms`, `amzn.to`, `a.co`, `adobe.ly`, …) are deliberately **excluded** so legitimate branded short links don't warn-fatigue users. When `follow_redirects` is enabled, the real destination of *any* shortener is resolved regardless. VirusTotal requires **≥2 vendors** flagging a URL before it counts as malicious, to cut false positives.

## Verdict tiers and actions

Each verdict tier maps to an admin-configurable action:

| Tier | Setting | Default action | Behavior |
|---|---|---|---|
| clean | `action_clean` | `redirect` | 302 straight to the destination |
| suspicious | `action_suspicious` | `warn` | Interstitial; user may proceed |
| malicious | `action_malicious` | `block` | Block page |

Available actions: `redirect` / `allow` (pass through), `warn` (interstitial with proceed), `block` (hard block, no proceed), `block_override` (block page that allows an explicit proceed).

**A hard `block` can never be bypassed.** `POST /lg/proceed` re-resolves and re-authorizes the verdict server-side — only a `warn` tier, or a `block_override` tier with the override flag, is allowed to continue. A user cannot escape a hard block by replaying the proceed request.

## Redirect detection

Attackers increasingly chain through **reputable hosts** — `storage.googleapis.com`, `firebasestorage.googleapis.com`, `*.web.app`, Azure `*.blob.core.windows.net`, Cloudflare `*.r2.dev` / `*.pages.dev`, and classic open redirectors like `google.com/url?q=…` — because the host reputation is clean, so a string-only check passes the link through. Link Guard adds three layers to catch this "living off trusted services" pattern.

### Open-redirect extraction (always on, no fetch)

The engine scans a link's query and fragment (raw, once-, and twice-URL-decoded) for an **embedded** `http(s)://` target on a different host than the redirector — the real destination of `…/url?q=https://evil.example`. That embedded target is re-run through the verdict pipeline (string-only); if it is suspicious or malicious, the original link inherits the verdict (`source = redirect`, detail `open-redirect -> <host>`). A benign embedded URL is a no-op. No outbound request is made.

### Abused-host heuristic (always on, no fetch)

Cloud-storage and app-hosting hosts commonly abused to host or bounce to phishing are flagged **suspicious** (warn) in the heuristics layer, so the user gets the interstitial and the resolved host instead of a silent redirect. Gated by `flag_cloud_storage` (default on). The match is suffix-based (`host == suffix` or `host` ends with `.suffix`).

The list is an **operator-managed table** (`linkguard_abused_hosts`), **shipped pre-seeded** with a curated baseline — there is no authoritative machine-readable feed of "abused hosting platforms" (the actual-bad *URLs* are what the URLhaus / OpenPhish feeds cover), so the seed is hand-curated from public abuse research (Trustwave SpiderLabs, Proofpoint, Netskope, Phishing.Database). It covers object storage (`storage.googleapis.com`, `firebasestorage.googleapis.com`, `firebaseapp.com`, `web.app`, `appspot.com`, `blob.core.windows.net`, `s3.amazonaws.com`), edge/static-site hosting (`r2.dev`, `pages.dev`, `workers.dev`, `github.io`, `netlify.app`, `vercel.app`, `herokuapp.com`, `onrender.com`, `surge.sh`), free site builders (`weebly.com`, `wixsite.com`, `000webhostapp.com`), and tunneling services (`trycloudflare.com`, `ngrok.io`, `ngrok-free.app`).

Manage it from the **Abused / redirector hosts** card on the Link Guard page:

- **Add** any host (covers its subdomains) — react to a new abuse pattern immediately, no rebuild.
- **Remove** a baseline entry you don't want (select → Delete). The seed is **one-time** (only when the table is empty), so your add/deletes survive upgrades.
- Or **suppress** without deleting by adding the host/path to the admin **URL allowlist** — the allowlist wins over the heuristic (e.g. a company that legitimately serves files from `storage.googleapis.com/<your-bucket>`).

Add/delete in this card **applies immediately** — the handler regenerates the engine's `config.json` on each change (same as the URL-rules and domains cards), so there's no separate Save for table edits. The `flag_cloud_storage` master switch and the other settings live in the settings card and take effect on its **Save & reload settings** button.

### Guarded redirect-follow (optional, the one fetch layer)

When `follow_redirects` is enabled, the engine — at click time, after the string layers — follows the HTTP `30x` redirect chain and verdicts the **real destination**, catching a trusted-host link that issues a server-side redirect to phishing (the `storage.googleapis.com` → phishing case). This is the **only** layer that makes an outbound request, so every hop is fenced (`_safe_to_fetch`):

- **http/https only**, and only a **standard web port** (80/443).
- Each hop's host must resolve to **public IPs only** — any answer in a loopback / RFC1918 / link-local / reserved / multicast range aborts the follow. This stops the follower being used as an SSRF pivot into the internal Docker network.
- **HEAD only**, no body downloaded; bounded by `follow_max_hops` (default 5), a short per-hop timeout, and a loop guard.
- No cookies/credentials sent; neutral User-Agent; `Referrer-Policy: no-referrer`.

Each followed hop is verdicted string-only (a follow never recurses into another follow). If the chain reaches a suspicious/malicious destination, the link inherits that verdict (`source = redirect`, detail `redirect chain -> <final host>`). A follow failure (timeout, guard stop, HEAD not allowed) fails **closed for the follow** — the link keeps whatever verdict the string layers produced; a click is never blocked by a follow error.

**Proactive on warn-tier links.** A shortener or abused-host link is flagged *suspicious* by the heuristics layer (step 6), which is *before* the follow layer in precedence. When `follow_redirects` is on, those warn-tier links are **also followed at that point** — proactively, during the verdict, not on "Continue":

- destination resolves **malicious** → the link is escalated to a **hard block** (the user never gets a *Continue* into a known-bad site);
- destination resolves **clean or suspicious**, or the chain can't be followed → the link **stays a warn**, now annotated with the resolved final host (`<reason> (resolves to <final host>)`).

So with redirect-following enabled, a `storage.googleapis.com` / shortener link that actually bounces to phishing becomes a **block**, while one that resolves somewhere benign stays an informative **warn**. The verdict (block or warn) is cached, so the follow runs at most once per link per cache-TTL.

> **SSRF posture.** Steps 1–7 never fetch the target. Enabling `follow_redirects` is a deliberate trade: it resolves redirect chains a string check cannot, at the cost of one guarded outbound request per uncached click (latency) and a controlled egress surface. The residual DNS-rebinding window (resolve-then-connect) is accepted for this release. Leave it **off** to preserve the zero-fetch guarantee; the two no-fetch layers above still run.

**Out of scope for this release:** following **JavaScript** or `<meta>` **refresh** redirects, which require fetching and parsing the page body — tracked as a later enhancement.

## Tokens — stateful v2 (default) with stateless v1 fallback

The rewritten link carries a token in the `t` query parameter. Two formats exist:

**v2 — stateful (default).** The token is just `2.<128-bit opaque id>`. The milter writes the mapping `id → {original_url, recipient_domain, expiry}` to a shared SQLite store (`url_map.db`) on the `linkguard_data` volume; the Link Guard container reads it. Because the token itself is tiny, **there is no link-length limit** — every link is protected regardless of how long the original URL is. This closes the v1 over-length fail-open gap (see below).

**v1 — stateless (fallback + in-flight).** The token is a self-contained HMAC signature: `1.<recipient_domain>.<url>.<expiry>.<signature>`. The milter mints a v1 token if the shared store is unavailable (e.g. off-box deployment, or transient DB contention), so **mail flow never depends on the store**. v1 tokens already in delivered mailboxes continue to verify until they age out via the token TTL.

The milter's mint/verify logic is a **byte-for-byte mirror** of the container's `lg_token.py`, so the container verifies exactly what the milter mints. The `url_map.db` store uses a rollback journal (not WAL), so the container can read it cross-container without a `-shm` file.

> **Why v2 exists.** Under v1, a URL longer than the inline cap was left *unprotected* (the original link was passed through unrewritten). An attacker could pad a URL past the cap to dodge Link Guard entirely. v2's short opaque id removes the length dependency, so nothing is ever skipped. The `max_inline_url` setting is now a **fallback-only** bound for the v1 path.

## Outbound link restoration

`restore_outbound` (default **on**) unwraps Link Guard tokens back to the original URLs on **outbound** mail — when a recipient replies to or forwards a protected message, the quoted history shows the real links again, not `/lg/?t=...` redirects. This keeps conversations readable and prevents Hermes redirect URLs from leaking to external parties. (Microsoft 365 was verified not to strip the tokens on manual replies, so restoration is the correct default.)

## HMAC key rotation

The signing key for v1 tokens is rotatable from the console. Rotation keeps a **current + previous** overlap: newly minted tokens use the current key, while tokens signed with the previous key still verify until they age out. The teardown on a Pro license lapse blanks only the dispatch maps and **never** the keys, so in-flight links keep resolving and a renew resumes minting with the same key.

## Settings reference

Settings live in the `parameters2` table under `module = 'linkguard'` (not `system_settings`). On save they are pushed to the engine via `POST /api/config`.

| Setting | Default | Meaning |
|---|---|---|
| `enabled` | `0` | Master on/off for Link Guard |
| `redirect_base_url` | (console host) | Public base URL for `/lg/` links |
| `action_clean` | `redirect` | Action for clean verdicts |
| `action_suspicious` | `warn` | Action for suspicious verdicts |
| `action_malicious` | `block` | Action for malicious verdicts |
| `restore_outbound` | `1` | Unwrap tokens on outbound replies/forwards |
| `token_ttl_days` | `14` | How long a rewritten link stays valid |
| `max_inline_url` | `4000` | Fallback-only length bound for v1 stateless tokens |
| `rate_limit_per_min` | `120` | Per-client-IP rate limit on `/lg/` |
| `flag_cloud_storage` | `1` | Flag abused cloud-storage/redirector hosts as suspicious (warn) |
| `follow_redirects` | `0` | Follow `30x` redirect chains at click time (guarded outbound fetch) |
| `follow_max_hops` | `5` | Max hops to follow when `follow_redirects` is on |
| `cache_ttl_clean_hours` | `24` | Verdict cache lifetime — clean |
| `cache_ttl_suspicious_hours` | `6` | Verdict cache lifetime — suspicious |
| `cache_ttl_malicious_hours` | `168` | Verdict cache lifetime — malicious |
| `feed_urlhaus_enabled` | `1` | Pull the URLhaus blocklist feed |
| `feed_openphish_enabled` | `1` | Pull the OpenPhish blocklist feed |
| `feed_refresh_minutes` | `60` | Feed refresh interval |
| `gsb_enabled` / `gsb_api_key` | `0` / — | Google Safe Browsing lookups (optional key) |
| `vt_enabled` / `vt_api_key` | `0` / — | VirusTotal lookups (optional key) |
| `clicks_retention_days` | `90` | Click-log retention for reporting |

Two additional console-managed lists drive the verdict pipeline:

- **Protected recipient domains** (`linkguard_domains`) — which recipient domains have their inbound links rewritten. A `_default` catch-all entry protects all domains.
- **URL allow / block rules** (`linkguard_url_rules`) — operator allow/block patterns that take precedence over feeds and heuristics (layers 1–2 above).
- **Abused / redirector hosts** (`linkguard_abused_hosts`) — the seeded, operator-managed warn list for the abused-host heuristic (layer 6); see [Redirect detection](#redirect-detection).

## Reputation feeds and optional API lookups

- **URLhaus** and **OpenPhish** are pulled on the `feed_refresh_minutes` interval into the container's SQLite store and matched as exact URL-hash lookups (a phishing URL on a shared host blocks only that URL, not the whole host).
- **Google Safe Browsing** and **VirusTotal** are off by default; enable each and supply an API key to add a string-reputation layer. Results are cached per the cache-TTL settings to limit API calls.

### Setting up VirusTotal and Google Safe Browsing

Both are **optional** — Link Guard works without them (admin lists + feeds + heuristics + redirect-following). They add a malicious-verdict layer that checks the URL **as a string** against the provider (the target is never fetched). Each is enabled the moment you save its key (no Save & Reload), and a malicious result will **escalate even a warn-tier link to a hard block**.

**Where to enter keys:** Link Guard page → **Reputation sources** → the provider's **Edit key** button. Entering a key auto-enables the provider; clearing it disables and wipes the stored key.

**VirusTotal (recommended):**

1. Create a free account at [virustotal.com](https://www.virustotal.com/) and sign in.
2. Open your profile menu → **API key**, and copy it.
3. Paste it into Link Guard → VirusTotal → Edit key.
4. **Quota:** the **free Public API** is rate-limited (≈ **4 lookups/minute, 500/day, ~15.5k/month**) and is for non-commercial use; high-volume or commercial sites need a paid **Premium API** key. The verdict cache (and the per-tier cache-TTL settings) is what keeps you under quota — which is why **Clear verdict cache** warns before re-querying. A verdict counts as malicious only when **≥ 2 vendors** flag the URL.

**Google Safe Browsing:**

1. In the [Google Cloud Console](https://console.cloud.google.com/), create (or pick) a project.
2. **APIs & Services → Library →** search **"Safe Browsing API" → Enable**.
3. **APIs & Services → Credentials → Create credentials → API key**, and copy it.
4. Paste it into Link Guard → Google Safe Browsing → Edit key.
5. ⚠️ **Commercial-use caveat:** the Safe Browsing **Lookup API v4** that Link Guard uses is **free but designated non-commercial only by Google, and is deprecated** in favour of the paid **Web Risk API** for commercial use. Review Google's [usage terms](https://developers.google.com/safe-browsing/v4/usage-limits) for your deployment. If you only want one reputation provider, **VirusTotal is the simpler choice**; Web Risk support may be added later.

**Privacy:** with either provider enabled, the clicked URL string is sent to that provider's API for the lookup. Nothing else leaves Link Guard.

## Branded interstitials

The warning and block pages are served by the container (`templates.py`) and carry Hermes SEG branding — an inline logo, a "Hermes SEG Link Guard" header, and a footer link to hermesseg.io — rather than a generic browser error. The warning page shows the **resolved host** so a user can judge a shortened or suspicious link before proceeding.

## Reporting and diagnostics

The admin page includes:

- **Check a URL** — enter any URL to see the live verdict, which pipeline layer decided it, and the resolved host. This is side-effect-free (`verdict.resolve(cache_write=False)`) so it does not pollute the cache.
- **Recent activity** — a table of recent clicks (Time, Recipient domain, Host, Verdict, Decided by, **Detail**, Action). The **Host** column is the link's original host; the **Detail** column carries the verdict detail per click — e.g. `redirect chain -> evil.example`, `Safe Browsing: SOCIAL_ENGINEERING`, `open-redirect -> …` — so a follow-decided click shows its **final/destination host** right in the table (host-only, never the full URL). The `redirect-follow` container log below additionally captures follows that came back **clean** (which still produce a click row, now with the chain in Detail).
- **Redirect-follow log** — when `follow_redirects` is on, every actual follow is logged (hosts only, never full URLs):
  ```bash
  docker logs hermes_linkguard | grep "redirect-follow:"
  # redirect-follow: bit.ly -> evil.example (1 hop) verdict=malicious
  ```
  This shows *what* got followed, the hop chain, the hop count, and the resulting verdict — including follows that came back clean (which leave no row in Recent activity).
- **Troubleshooting commands** — a collapsible card of `docker exec` one-liners for inspecting the scope map, store, and feeds.
- **Clear verdict cache** — a button at the **top of the page** that flushes the **entire** verdict cache (every source, including GSB / VirusTotal and feed-derived results) via the mgmt endpoint `POST /api/cache-clear`, forcing a complete re-evaluation on next click. Its confirm modal warns that this **re-queries Google Safe Browsing / VirusTotal**, consuming additional lookups against your API quota. You rarely need it: a config edit already auto-purges the *config-dependent* cache (see below).

### Verdict cache invalidation

The engine caches each URL's verdict (per-source TTL — clean 24h, suspicious 6h, malicious 168h) to avoid re-running heuristics / re-hitting APIs on every click. Two things keep it from going stale against admin edits:

- **Automatic, scoped:** when `config.json` reloads after any console change (remove an abused host, add/remove a URL rule, toggle follow), `config.py` drops the **config-dependent** cached verdicts (`admin` / `heuristic` / `redirect` / `none`) so the edit takes effect on the **next click** — no TTL wait, no button. Feed (`local`) and external (`gsb`/`vt`) caches are kept (they don't depend on console config). This is why removing a host stops the warning immediately.
- **Manual, full:** the **Clear verdict cache** button above flushes *everything*, for the rare "force a complete re-check / I think an external result is stale" case (it re-queries GSB/VirusTotal, so it's confirm-gated).

## Deployment — in-stack or separate host

Link Guard runs **inside the Hermes SEG stack** (the default; `hermes_linkguard` service on the compose network) or on a **separate host** for isolation and scale. The nginx `/lg/` location lives in the **vhost template**; it is delivered by regenerating the per-domain vhosts from the template (via the headless `schedule/regen_nginx_config.cfm`), not by hand-editing a generated vhost.

When Link Guard runs off-box, the milter cannot reach the shared `url_map.db`, so it mints **v1 stateless** tokens — the same fallback path described above. Mail flow is never affected by the container's location or availability.

## Failure semantics

Link Guard is **graceful-degradation** by design, consistent with the rest of the body milter:

- **Link Guard container down / unreachable** → the milter falls back to v1 stateless tokens (rewrite still happens); already-delivered links cannot be resolved until the container returns, but **mail keeps flowing** (`milter_default_action = accept`).
- **Shared store unavailable** → milter mints v1 tokens; mail flow never depends on the store.
- **Scope map empty** (e.g. after a Pro license-lapse teardown) → no inbound links are rewritten, but mail flows unmodified. Re-enabling / re-saving Link Guard repopulates the map.
- **External API (GSB/VT) error or timeout** → that layer is skipped; the verdict falls through to the next layer (worst case `clean`).

In every failure case the worst outcome is a missed rewrite or a fall-through verdict — never lost mail.

## Files and data locations

| Path | Container | Contents |
|---|---|---|
| `/etc/hermes/body_milter/linkguard/linkguard_by_recipient_domain` | body_milter | Scope map: protected recipient domains (`_default` = all) |
| `/var/lib/linkguard/url_map.db` | body_milter (writer) / linkguard (reader) | v2 token id → original URL store, on the shared `linkguard_data` volume |
| `/opt/linkguard/app/` | linkguard | Engine code (server, verdict, feeds, store, token, templates) |
| Operational SQLite store | linkguard | Verdict cache, feed entries, click log |

The scope map is mtime-watched by the milter and reloaded on the next message when it changes — no explicit milter reload step is needed after a console save.

## Security properties (summary)

- **SSRF-fenced** — steps 1–7 never fetch the target; the optional redirect-follow (step 8) is the only fetch, and every hop is restricted to http/https + standard ports + hosts that resolve to public IPs only.
- **Hard blocks are unbypassable** — `/lg/proceed` re-authorizes server-side.
- **Hardened port split** — public click surface cannot push config or read keys.
- **Rate-limited** public surface (`rate_limit_per_min`, per client IP).
- **Signature-safe** — inbound S/MIME, PGP, and upstream-DKIM-signed mail is skipped, never re-bodied.
- **Mail-flow-safe** — the container being down, off-box, or torn down on a license lapse never blocks delivery.

# Organizational Signatures

# Organizational Signatures

_Pro Edition feature._ Maps to **Email Policies > Org Signatures** (`view_org_signatures.cfm`, `edit_org_signature.cfm`, `org_signature_delete.cfm`).

Hermes attaches a centrally-managed signature to outbound mail at the gateway. Admins design the signature once per domain (and optionally per department); every user on that domain gets a personalized version of it on every outbound message — no per-user setup required.

## Two signature types, one pipeline

Hermes ships two distinct signature concepts that run through the same body milter and the same resolver:

| Type | Tier | Owner | Storage | Per-domain control |
|---|---|---|---|---|
| **Personal Signature** | Community + Pro | The user (in `/users/2/view_signature.cfm`) | `user_signatures` table, one row per user | Toggled via `domains.allow_user_signatures` |
| **Organizational Signature** | Pro only | The admin (in `Email Policies > Org Signatures`) | `org_signatures` table, one row per `(domain_id, department_label)` | One default per domain + optional per-department variants |

The milter never decides which one to apply at message time. The CFML resolver picks a winner per mailbox at admin-action time and writes a precomputed `sender → option` map; the milter just looks up the option and applies whatever it finds.

## Department names — single source of truth

Departments are defined once on the **mailbox edit form** (Email Server > Mailboxes > Edit Options > Personal Information > Department), as free-text values typed by the admin. There is no separate "Departments" table; a department exists as soon as one mailbox is in it.

The Org Sig form's Department field is a **strict dropdown sourced from the distinct `mailboxes.department` values for the selected domain.** This means:

- You cannot create an Org Sig for a dept that has no mailboxes — the dept won't appear in the dropdown.
- The dept name on both sides is guaranteed to match exactly. No typo-class drift.
- Workflow: assign at least one mailbox to the new dept first, then come back and create the Org Sig targeting it.
- Changing the domain in the Org Sig form repopulates the dropdown with that domain's depts via JavaScript (no AJAX round-trip; the per-domain map is dumped into a JS const at page load).

The mailbox edit form's Department field is a free-text input with a **`<datalist>` typeahead** showing the same per-domain dept list. Admins can pick an existing dept (auto-completes) or type a brand-new dept name (which then appears in the dropdown next time).

If you edit an existing Org Sig whose `department_label` no longer matches any current mailbox (the dept was renamed elsewhere, or all mailboxes in it were reassigned), the orphan value is preserved in the dropdown with a `(no mailboxes)` suffix so you can still see and edit/delete the row instead of silently losing the value.

The resolver at send time does a **case-insensitive trimmed match** against `mailboxes.department`, so casing or whitespace drift across edits is forgiving even in the rare cases the dropdown is bypassed (e.g. direct SQL changes).

## Resolution order

For every enabled mailbox, the resolver walks this priority chain top-down and stops at the first match:

```
1. Personal Signature
   └─ if domains.allow_user_signatures = 1
      AND user_signatures has an enabled, non-empty row for this mailbox
   ─> wins. option = user_<sanitized_email>

2. Department Organizational Signature
   └─ else if mailboxes.department is non-empty
      AND org_signatures has enabled = 1 row matching
          (domain_id, department_label)
   ─> wins. option = org_<row_id>

3. Domain Default Organizational Signature
   └─ else if org_signatures has enabled = 1 row matching
          (domain_id, department_label IS NULL)
   ─> wins. option = org_<row_id>

4. None
   └─ no map entry; the milter applies no signature to this sender's mail.
```

The chain is **per-mailbox**, not per-message. The resolver runs at admin-action time (see [Triggers](#triggers)), serializes the winning option for every mailbox into one map file, and the milter consults that map at send time. There is no per-message DB query and no per-message resolution logic in the milter.

## Pipeline placement

Same chain as Disclaimers (#214) — see [disclaimers.md](https://docs.deeztek.com/books/administrator-guide/page/disclaimers) for the full Postfix / OpenDKIM / Ciphermail picture. The summarized order:

```
External MTA / MUA submission
        │
        ▼
Postfix smtpd
   ├─ smtpd_milters chain (in order):
   │   1. OpenDKIM            (signs/verifies)
   │   2. OpenDMARC           (DMARC policy)
   │   3. hermes_body_milter  (THIS — signatures, then disclaimers)
   ▼
Amavis  →  Ciphermail  →  Postfix :10026 (DKIM sign) → external
```

Inside the body milter, **SignatureModifier runs before DisclaimerModifier**, so the layered output on the outbound message is:

```
[user body]
[signature]      ← Personal or Organizational, picked by resolver
[disclaimer]     ← if a disclaimer is configured for this sender
```

OpenDKIM signs at the `:10026` re-injection — after both modifiers — so Hermes' own DKIM signature always covers the recipient's view of the message (with signature and disclaimer baked in).

## Templates

Phase 2A ships **six bundled templates** under `/admin/2/inc/org_signature_templates/`:

| Template key | Layout |
|---|---|
| `modern_card` | Logo left, accent bar, contact stack right |
| `two_column_pro` | Left contact, right org block + CTA button |
| `with_social_bar` | Vertical contact + horizontal social-icon row |
| `banner_with_logo` | Full-width banner with logo top, contact below |
| `promo_footer` | Contact + bottom promotional image with link |
| `compact_text` | Minimal text-only, no images, no styling |

Each template is a `.cfm` file that declares its **field schema** (text / email / url / color / checkbox / image fields with optional `showIf` gating) and renders pixel-perfect HTML when the renderer is invoked. Admins fill in the form on `edit_org_signature.cfm`; the gallery thumbnail + live sandboxed-iframe preview show the result before save.

All the auto-filled fields — Name, Title, Phone, Mobile, Email (`{{user.*}}` from the mailbox row) plus Website and Address (`{{org.*}}` from the domain row) — are **collapsed under an "Override auto-filled fields" toggle** by default. The admin doesn't see or edit them in the common case; the placeholders flow through to the rendered HTML unchanged and the milter fills in the recipient's data at send time. Toggling the override on exposes the fields for the rare cases that need literal text instead of substitution (shared mailboxes without personal info, seasonal URL overrides, etc.).

The genuinely admin-supplied fields stay always visible — Logo, accent color, show/hide toggles for each line, CTA text and URL, social URLs, disclaimer text, and any template-specific extras (banner height, promo image, etc.). These are the admin's actual editing surface.

Net workflow: pick a template, upload a logo, set the accent color, save. Done. Every mailbox on the domain gets a fully personalized signature on its next outbound message without any per-user form input.

Templates are version-controlled in the repo, not in the database. To add a new template, drop a new `.cfm` file in `org_signature_templates/`, add its key to the registry in `inc/org_signature_template_loader.cfm`, and produce a 240×140 thumbnail PNG. No schema migration needed.

## Placeholder substitution at send time

The signature HTML stored in `org_signatures.rendered_html` (and on disk in `body.html`) keeps `{{namespace.field}}` tokens **literal**. The body milter substitutes them per recipient at message-send time against the sender's row in `sender_data.json`.

Available placeholders (Phase 2B):

| Token | Source column |
|---|---|
| `{{user.first_name}}` | `mailboxes.first_name` |
| `{{user.last_name}}` | `mailboxes.last_name` |
| `{{user.title}}` | `mailboxes.title` |
| `{{user.phone}}` | `mailboxes.phone` |
| `{{user.mobile}}` | `mailboxes.mobile` |
| `{{user.department}}` | `mailboxes.department` |
| `{{user.email}}` | `mailboxes.username` |
| `{{org.name}}` | `domains.org_name` |
| `{{org.phone}}` | `domains.org_phone` |
| `{{org.address}}` | `domains.org_address` |
| `{{org.website}}` | `domains.org_website` |
| `{{org.logo_url}}` | `domains.org_logo_path` (raw URL — not cid: extracted) |
| `{{dept.name}}` | `mailboxes.department` |

Tokens whose corresponding field is empty resolve to **empty string**, not literal `{{...}}`. So if `mailboxes.title` is blank for a given user, the `{{user.title}}` token disappears cleanly from delivered mail. Unknown namespaces (anything outside `user`, `org`, `dept`) are also substituted to empty.

The substitution is a single regex pass on the body html and body text inside `SignatureModifier.modify()` — well under a millisecond per message. The map and `sender_data.json` both live in process memory, refreshed only when their file mtime changes.

**No DB connection from the milter, ever.** All resolution and substitution data is precomputed by CFML and dropped on disk; the milter consumes the file artifacts.

## Triggers — when the resolver re-runs

Both `signature_by_sender` and `sender_data.json` are rewritten in full by `inc/signature_regen_map.cfm` on every event that could affect a winner or a substitution value:

| Event | Why it matters |
|---|---|
| Admin saves an Org Sig | New / edited row may win for senders that previously had no match or a lower-tier match |
| Admin deletes an Org Sig | Losers fall back to the next tier (or none) |
| Admin edits a domain (`allow_user_signatures` or `org_*` columns) | Per-domain toggle flips the Personal-vs-Org winner; org_* values feed `{{org.*}}` substitution |
| Admin edits a mailbox (Pro fields: first_name, title, dept, etc.) | `{{user.*}}` and `{{dept.name}}` substitution data changes; a department change can flip Default → Department winner |
| Admin adds a mailbox | New sender enters resolution and may pick up a domain-default Org Sig |
| Admin deletes a mailbox | Sender must drop from the map |
| User saves their Personal Signature | May now win over the previously-resolved Org Sig (or vice versa if disabling) |

Each trigger runs the same shared resolver. **Full rebuild every time, not incremental.** With low-thousands of mailboxes the rebuild is well under a second, and the simplicity rules out drift bugs ("did we forget to update X for sender Y" can't happen).

The body milter mtime-watches both files and reloads in process memory on the next message after the file changes. **No SIGHUP, no IPC, no container restart.**

## Files generated on save

The CFML resolver writes:

```
/etc/hermes/body_milter/signatures/signature_by_sender   sender → option map
/etc/hermes/body_milter/signatures/sender_data.json       sender → {{token}} dict
/etc/hermes/body_milter/signatures/files/<option>/
    body.txt         plain-text signature (auto-derived from html)
    body.html        html signature with cid: refs (placeholders intact)
    images/
        1.png        per-option inline images (#230 pattern)
        2.jpg
        ...
```

Where `<option>` is `user_<sanitized_email>` (Personal Sig) or `org_<row_id>` (Org Sig). `<sanitized_email>` is `bob.smith@example.com` → `bob_smith_at_example_com` (`@` → `_at_`, non-alphanumerics → `_`).

`signature_by_sender` example:

```
alice@example.com   org_42
bob@example.com     user_bob_at_example_com
carol@example.com   org_43
```

`sender_data.json` example (post-Lucee uppercasing — the milter normalizes to lowercase on load):

```json
{
  "alice@example.com": {
    "USER.FIRST_NAME": "Alice",
    "USER.TITLE": "Sales Manager",
    "ORG.NAME": "Acme",
    "ORG.PHONE": "555-0100",
    "DEPT.NAME": "Sales"
  }
}
```

The `files/<option>/` dir is wiped before re-render on every save of that row, so deleted images and renamed scope keys never leave stale binaries behind.

## Inline images (cid: extraction)

Same pattern as Disclaimers (#230) — see [disclaimers.md](https://docs.deeztek.com/books/administrator-guide/page/disclaimers#inline-images-230) for the MIME / multipart-related details.

For Org Signatures, the cid: namespace is `signature_org_<row_id>_img_<N>` (Personal Signatures use `signature_user_<sanitized_email>_img_<N>`). Both share the milter regex `cid:(signature_[\w.-]+_img_\d+)`, so cid: refs from either signature type are queued for inline attachment alongside any cid: refs from a domain disclaimer on the same message — no namespace collisions.

Per-template image fields use the **same data: → cid: pipeline** as user-pasted Personal Sig images. At admin-save time:

1. Admin uploads the file in the form (or pastes a URL — both are handled).
2. Browser converts the file to a `data:image/...;base64,...` URI via `FileReader`, capped at 1 MB per image.
3. CFML renders the template; the resulting HTML carries the data: URI inline.
4. `inc/org_signature_write_files.cfm` extracts each `data:` URI, decodes the base64 into a binary file under `images/`, and rewrites the html to reference `<img src="cid:signature_org_<id>_img_<N>">`.
5. At message-send time the milter walks the cid: refs, attaches each image as an `image/<format>` MIME part with `Content-ID` and `Content-Disposition: inline`, and wraps the message as `multipart/related`.

`{{org.logo_url}}` is **not** cid: extracted — it's a raw URL substituted into the html as-is. Use it for hosted-elsewhere logos (your CDN, your website). Use the per-template **image** field for cid:-attached inline logos when you want them to render even in MUAs that block external images.

## Behavior with S/MIME, PGP, DKIM-signed mail

Identical to Disclaimers — same skip rules in the same Modifier base class. Pre-signed envelopes, PGP inline, and pre-existing DKIM-Signature headers all cause the body milter to leave the message untouched.

See [disclaimers.md "Behavior with S/MIME, PGP, and DKIM-signed mail"](https://docs.deeztek.com/books/administrator-guide/page/disclaimers#behavior-with-smime-pgp-and-dkim-signed-mail) for the table of patterns and the operational consequences.

## Reply-chain handling

No dedup — every outbound gets a fresh signature, including replies inside a long thread. Same industry-norm pattern Disclaimers uses; same rationale (compliance, self-contained messages, predictability). See [disclaimers.md "Reply-chain handling"](https://docs.deeztek.com/books/administrator-guide/page/disclaimers#reply-chain-handling--no-dedup-by-design).

## Failure semantics

Same graceful-degradation contract as Disclaimers (`milter_default_action = accept`). If the milter container is down, if the map file is unreadable, if the modifier raises an exception, if substitution blows up — **mail flows unmodified**. Worst case is a missed signature; mail never gets dropped.

See [disclaimers.md "Failure semantics"](https://docs.deeztek.com/books/administrator-guide/page/disclaimers#failure-semantics).

## Disabled rows

`org_signatures.enabled = 0` causes the resolver to skip the row entirely:

- The on-disk `org_<id>/` dir is wiped clean
- No mailboxes resolve to that option
- Mailboxes that previously resolved to that option fall back to the next tier (department → default → none)

Re-enabling rebuilds the on-disk files and re-points the affected mailboxes' map entries on the next regen.

## Interaction with `domains.allow_user_signatures`

This per-domain toggle is the single switch that controls whether Personal Signatures can win over Organizational Signatures.

| `allow_user_signatures` | Personal Sig present? | Result |
|---|---|---|
| `0` | yes | Personal Sig **ignored**; resolver falls to Department / Default Org Sig |
| `0` | no | Resolver falls to Department / Default Org Sig |
| `1` | yes | Personal Sig **wins** (top of resolution chain) |
| `1` | no | Resolver falls to Department / Default Org Sig |

Toggle this off when you need to **lock everyone into Organizational Signatures** for branding/compliance — useful when a marketing or legal team wants centrally-controlled output and doesn't want individual users overriding it.

A previously-saved Personal Signature is **not deleted** when the toggle goes off — it just stops being resolved. Toggling back on re-activates it on the next regen.

## Pro license behavior

The Org Signatures admin page is gated by `session.edition EQ "Pro"`:

- **Pro license valid:** Full UI access; admins can create / edit / delete / enable / disable Org Signatures and toggle `allow_user_signatures`.
- **Pro license missing or expired:** Sidebar entry shows a `PRO` badge; the list and edit pages reject the load with an upsell flash. The save action handler ALSO rejects on the server side (defense in depth — UI-level gating alone isn't a security control).
- **Existing Org Signatures on a downgrade:** Rows persist in the database. The resolver still runs and the milter continues applying them at send time. Personal Signatures continue working as well. The downgrade is a **UI restriction**, not a runtime feature kill.

This is the same "feature stays on, admin UI locks" pattern as Disclaimers and other Pro features. If a customer wants the Pro feature actually disabled at runtime on downgrade, the path is to delete the rows or set them all to `enabled = 0`.

## Why a separate milter and not an amavis hook

Same reasoning as Disclaimers (#214 Phase 3). amavisd-new 2.13's `before_send` hook silently desynchronizes amavis's internal MIME state during in-place body modification, which can drop mail. The body milter approach moves body modification out of amavis entirely; amavis is fully decoupled.

See [disclaimers.md "Why a separate milter and not an amavis hook"](https://docs.deeztek.com/books/administrator-guide/page/disclaimers#why-a-separate-milter-and-not-an-amavis-hook).