# Content Checks # Antispam Settings # Antispam Settings Admin path: **Content Checks > Antispam Settings** (`view_antispam_maintenance.cfm`, `inc/get_spam_settings.cfm`, `inc/spam_settings_save.cfm`, `inc/update_amavis_config_files.cfm`, `inc/update_spamassassin_config_files.cfm`, `inc/restart_amavis.cfm`, `inc/restart_spamassassin.cfm`, `inc/antispam_init_pyzor.cfm`, `inc/antispam_init_razor.cfm`, `inc/antispam_clear_bayes.cfm`). This page configures the SpamAssassin engine that Amavis calls inside `hermes_mail_filter` for every message that clears the SMTP-time perimeter, plus the Amavis-level handling policies that decide what happens to a message once it has been scored or otherwise classified. Per-rule weight adjustments live on [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides); this page is engine settings and quarantine destiny only. ## Where SpamAssassin sits in the flow ``` +-----------------------------------+ inbound msg -->| Perimeter Checks pass | +---------------+-------------------+ | v +-----------------------------------+ | Postfix smtpd_proxy_filter | | -> hermes_mail_filter:10024 | +---------------+-------------------+ | v +-----------------------------------+ | Amavis (hermes_mail_filter) | | - ClamAV virus scan | | - SpamAssassin scoring | | DCC / Razor / Pyzor net DBs | | Bayes statistical engine | | custom rules + scores | | - banned-file checks | | - final_*_destiny -> quarantine/DSN/discard +---------------+-------------------+ | v +-----------------------------------+ | Re-inject -> hermes_postfix_dkim:10026 +-----------------------------------+ ``` A virus verdict from ClamAV always pre-empts the spam score; the `final_virus_destiny` setting on this page decides what Amavis does with that already-classified virus. The `final_spam_destiny`, `final_banned_destiny`, and `final_bad_header_destiny` settings work the same way for the other three Amavis verdict categories. ## Container and tool placement | Component | Detail | | --- | --- | | Container | `hermes_mail_filter` (IPv4 `.105`) | | Engine | SpamAssassin (`spamd` / `Mail::SpamAssassin` Perl modules called from Amavis) | | Amavis config | `/etc/amavis/conf.d/50-user` (rendered from `/opt/hermes/conf_files/50-user.HERMES` on every save) | | SpamAssassin config | `/etc/spamassassin/local.cf` (rendered from `/opt/hermes/conf_files/local.cf.HERMES` on every save) | | Bayes DB | Lives in the SpamAssassin user dir inside `hermes_mail_filter` (`sa-learn --dump magic` reports the actual path) | | Network plugin state | `/etc/razor/identity` (Razor), Pyzor's per-user config dir, DCC's local socket — all inside `hermes_mail_filter` | | Reload mechanism | `spamassassin --lint` + `docker container restart hermes_mail_filter` on every save | The container exposes **no host ports** — Amavis is reached only by Postfix internally at `hermes_mail_filter:10024` and re-injects to `hermes_postfix_dkim:10026`. ## Spam Detection Plugins card Three boolean toggles enable third-party network-aware spam DBs. Storage: `spam_settings.value` for parameters `use_dcc`, `use_razor2`, `use_pyzor` (each row keyed by `parameter`, value `0` or `1`). | Plugin | What it does | Maintenance action | | --- | --- | --- | | DCC (Distributed Checksum Clearinghouse) | Fuzzy-checksum bulk-mail detection; matches a message against a network of receivers' checksum counters | None — `cdcc` runs as part of the SpamAssassin call chain | | Razor2 (Vipul's Razor v2) | Collaborative spam catalog; checksum + signature lookup against the Razor network | **Initialize Razor** (see Maintenance) before first use | | Pyzor | Collaborative digest-based spam detection | **Initialize Pyzor** before first use | Each toggle substitutes into `local.cf` via the placeholders `USE-DCC`, `USE-PYZOR`, `USE-RAZOR2` -> `use_dcc 0|1`, `use_pyzor 0|1`, `use_razor2 0|1`. > **Operational consequence — network DB connectivity.** All three > plugins make outbound queries (DCC over UDP, Razor and Pyzor over > TCP) at scan time. If outbound to the public Internet is blocked > from `hermes_mail_filter`, the plugins quietly time out per > message and add measurable per-scan latency. Disable plugins the > gateway cannot actually reach. ## Subject Tagging card Single field, `sa_spam_subject_tag` in `spam_settings`. Substitutes into `50-user` via the `sa-spam-subject-tag` placeholder, which sets Amavis's `$sa_spam_subject_tag`. Default `[SUSPECTED SPAM]`. Required (empty value rejected with error 2). Only applied when `sa_spam_modifies_subj = 1` (a fixed value in `spam_settings`, not exposed in the UI). ## Message Handling Policies card Four radio pairs, one per Amavis verdict category. Each row stores `D_DISCARD` or `D_BOUNCE` in `spam_settings.value` and substitutes into `50-user` via `final--destiny`. Amavis acts on the value as follows: | Setting | DB row | "Quarantine Only" (`D_DISCARD`) | "Quarantine & Send DSN" (`D_BOUNCE`) | | --- | --- | --- | --- | | Virus Messages | `final_virus_destiny` | Message goes to quarantine; no DSN | Message goes to quarantine; DSN sent to envelope sender | | Banned File Messages | `final_banned_destiny` | Same as above for banned-file matches | DSN sent | | Spam Messages | `final_spam_destiny` | Quarantined silently | DSN sent | | Bad-Header Messages | `final_bad_header_destiny` | Quarantined silently | DSN sent | The labels are deliberately conservative — `D_DISCARD` does **not** delete the message, it routes it to Amavis's quarantine where Message History can review and release it. Defaults: virus + banned send DSN; spam + bad-header quarantine silently. > **Operational consequence — Send DSN on spam.** Setting > `final_spam_destiny = D_BOUNCE` means Hermes will deliver a > non-delivery report to the envelope sender of every quarantined > spam. Because the envelope sender is almost always forged on spam, > the DSN will either bounce, contribute to backscatter against > innocent third parties, or land in a victim's spam folder. The > safe default for spam is `D_DISCARD`; reserve DSN for virus and > banned-file (where the sender is more likely to be legitimate). ## Bayes Database card SpamAssassin's per-installation statistical learning engine. Three controls, stored in `spam_settings`: | Field | DB row | Substitution placeholder | Effect | | --- | --- | --- | --- | | Enable Bayes Database | `use_bayes` | `USE-BAYES` -> `use_bayes` followed by `0` or `1` | Master switch; when off, Bayes rules contribute no score | | Enable Auto-Learning | `bayes_auto_learn` | `BAYES-AUTO-LEARN` -> `bayes_auto_learn` followed by `0` or `1` | When on, SpamAssassin trains the Bayes DB automatically based on the message's final score relative to the thresholds below | | Spam Threshold | `bayes_auto_learn_threshold_spam` | `BAYESAUTOLEARN-SPAM` -> `bayes_auto_learn_threshold_spam ` | Final score above which auto-learn treats the message as spam. Must be numeric and in the range `0.01 .. 999` | | Non-Spam Threshold | `bayes_auto_learn_threshold_nonspam` | `BAYESAUTOLEARN-HAM` -> `bayes_auto_learn_threshold_nonspam ` | Final score below which auto-learn treats the message as ham. Must be numeric and in the range `-999 .. -0.01` | The thresholds are SpamAssassin's `bayes_auto_learn_threshold_spam` and `bayes_auto_learn_threshold_nonspam` directives. JavaScript on the page collapses the thresholds when Bayes or auto-learning is disabled. > **Operational consequence — Bayes poisoning.** Auto-learning > trusts the final score (which already includes Bayes's own > contribution) to decide whether to train. A bad spam wave that > sneaks past the score threshold can train Bayes to think more spam > is ham, which lowers detection on the next batch. If detection > quality regresses noticeably after enabling auto-learning, use the > Clear Bayes Database action and re-train manually or via a > known-good corpus before re-enabling. ## Save flow ``` 1. View page submits action="save_settings" (all four cards in one POST) 2. spam_settings_save.cfm validates: - sa_spam_subject_tag non-empty (error 2) - if bayes_auto_learn=1: spam threshold numeric (error 5), > 0 and <= 999 (error 4), non-empty (error 3) non-spam threshold numeric (error 10), < 0 and >= -999 (error 8), non-empty (error 7) 3. On valid input, UPDATEs 13 rows in spam_settings (sa_spam_subject_tag, four final_*_destiny, use_bayes, bayes_auto_learn, both thresholds, use_dcc, use_razor2, use_pyzor) 4. cfinclude update_amavis_config_files.cfm: - Reads /opt/hermes/conf_files/50-user.HERMES - Substitutes SERVER-NAME, SERVER-DOMAIN, sa-spam-subject-tag, final-{virus,banned,spam,bad-header}-destiny, enable-dkim-{verification,signing}, HERMES-USERNAME, HERMES-PASSWORD, FILE-RULES-GO-HERE (from file_rule_components table), DKIM-KEYS-GO-HERE (from dkim_sign table) - Backs up /etc/amavis/conf.d/50-user -> 50-user.HERMES.BACKUP - Moves rendered file into place 5. cfinclude update_spamassassin_config_files.cfm: - Reads /opt/hermes/conf_files/local.cf.HERMES - Substitutes USE-DCC, USE-PYZOR, USE-RAZOR2, USE-BAYES, BAYES-AUTO-LEARN, BAYESAUTOLEARN-SPAM, BAYESAUTOLEARN-HAM - Appends per-rule score lines (from spam_settings where spamfilter=1) - Appends custom message rules (from message_rules table) - Backs up /etc/spamassassin/local.cf -> local.cf.HERMES.BACKUP - Moves rendered file into place 6. cfinclude restart_amavis.cfm -> restart_mail_filter.cfm: - docker container restart hermes_mail_filter 7. cfinclude restart_spamassassin.cfm: - docker exec hermes_mail_filter /usr/bin/spamassassin --lint - docker container restart hermes_mail_filter 8. session.m = 1 -> green "Anti-spam settings have been saved and applied" alert 9. cflocation back to view_antispam_maintenance.cfm ``` The same container is restarted twice (once for Amavis, once for SpamAssassin) because the restart includes are intentionally independent helpers used elsewhere; both calls resolve to the same `docker container restart hermes_mail_filter`. Outbound mail queues briefly during the restart cycle (typically a few seconds); Postfix will retry. ## Maintenance card group Three buttons, each running a single `docker exec` against `hermes_mail_filter` and surfacing stdout/stderr to the operator. ### Initialize Pyzor Action handler: `antispam_init_pyzor.cfm` ``` docker exec hermes_mail_filter /usr/bin/pyzor ping ``` Pings the Pyzor servers; success is detected by the literal string `200` in the output. The command both verifies connectivity and writes the per-user Pyzor config the first time it runs. Required before `use_pyzor = 1` returns meaningful results. ### Initialize Razor Action handler: `antispam_init_razor.cfm` ``` docker exec hermes_mail_filter /bin/bash -c \ 'rm -f /etc/razor/identity && razor-admin -create && razor-admin -register' ``` Deletes the existing Razor identity, creates a fresh config, and registers the gateway with the Razor network. Success is detected by `Register successful` or `created` in the output. Re-run if Razor queries start failing (typically after the identity is rotated or the network rejects the existing identity). ### Clear Bayes Database Action handler: `antispam_clear_bayes.cfm` ``` docker exec hermes_mail_filter /usr/bin/sa-learn --clear ``` Wipes the learned spam/ham corpus. SpamAssassin will need to re-learn from scratch before Bayes rules contribute meaningful scores again. Use only when the database is known-poisoned or when migrating between servers without preserving training. The button is gated behind a JavaScript `confirm()` and renders inside a yellow warning card. ## Failure semantics | Failure | Behavior | | --- | --- | | Empty `sa_spam_subject_tag` | session.m=2, red alert, no save | | Bayes spam threshold empty | session.m=3 | | Bayes spam threshold not numeric | session.m=5 | | Bayes spam threshold <= 0 or > 999 | session.m=4 | | Bayes non-spam threshold empty | session.m=7 | | Bayes non-spam threshold not numeric | session.m=10 | | Bayes non-spam threshold >= 0 or < -999 | session.m=8 | | Any cfcatch during the save -> apply chain | session.m=9, red alert with `session.saveError` showing `cfcatch.message` | | `spamassassin --lint` failure during restart | `error.cfm` cfabort with the lint failure message; the rendered `local.cf` is already in place but Amavis is not restarted further | | Pyzor ping output without `200` | session.m=12, red alert; full output shown in a `
` for diagnosis |
| Razor init output without `Register successful` or `created` | session.m=14, similar surfacing |
| Bayes clear `cfcatch` | session.m=16 with the catch message |

`spamassassin --lint` is the canonical pre-restart sanity check —
when a custom rule (added via Score Overrides or message rules) has
invalid syntax, the lint catches it before the container restart
finishes and prevents Amavis from starting against a broken config.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_antispam_maintenance.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/spam_settings_save.cfm` | `hermes_commandbox` | Validation + UPDATE + apply chain |
| `config/hermes/var/www/html/admin/2/inc/get_spam_settings.cfm` | `hermes_commandbox` | Loads current `spam_settings` rows |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Renders `50-user` from template + DB |
| `config/hermes/var/www/html/admin/2/inc/update_spamassassin_config_files.cfm` | `hermes_commandbox` | Renders `local.cf` from template + DB |
| `config/hermes/var/www/html/admin/2/inc/restart_amavis.cfm` / `restart_spamassassin.cfm` / `restart_mail_filter.cfm` | `hermes_commandbox` | `docker container restart hermes_mail_filter` |
| `config/hermes/var/www/html/admin/2/inc/antispam_init_pyzor.cfm` / `antispam_init_razor.cfm` / `antispam_clear_bayes.cfm` | `hermes_commandbox` | Maintenance docker-exec helpers |
| `config/hermes/opt/hermes/conf_files/50-user.HERMES` | template (read) -> `hermes_mail_filter` (live `/etc/amavis/conf.d/50-user`) | Amavis directives template |
| `config/hermes/opt/hermes/conf_files/local.cf.HERMES` | template (read) -> `hermes_mail_filter` (live `/etc/spamassassin/local.cf`) | SpamAssassin directives template |
| `/etc/amavis/conf.d/50-user.HERMES.BACKUP` | `hermes_mail_filter` | Pre-write backup, refreshed each save |
| `/etc/spamassassin/local.cf.HERMES.BACKUP` | `hermes_mail_filter` | Pre-write backup, refreshed each save |
| `spam_settings` table | `hermes_db_server` (`hermes` DB) | Source of truth for every UI value on this page; also holds per-rule scores (`spamfilter=1` rows) for Score Overrides |
| `message_rules` table | `hermes_db_server` | Custom header/body/full message rules; rendered into `local.cf` |
| `file_rule_components` / `files` tables | `hermes_db_server` | Banned-file rules; rendered into `50-user` |
| `dkim_sign` table | `hermes_db_server` | Per-domain DKIM keys; rendered into `50-user` for outbound signing |

## Related

- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) -- the ClamAV engine
  that runs in the same Amavis pass and whose virus verdict always
  pre-empts the spam score
- [Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds) -- third-party ClamAV signature
  feeds; orthogonal to spam scoring but consumed in the same scan
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) -- per-rule SpamAssassin
  weight adjustments (the `spamfilter=1` rows in `spam_settings`);
  this page sets the engine knobs, that page sets the rule weights
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) -- custom header / body / full
  message regex rules that ride into `local.cf` on every save here
- [SVF Policies](https://docs.deeztek.com/books/administrator-guide/page/svf-policies) -- per-sender and per-recipient
  spam-handling overrides that apply before the engine-wide
  `final_*_destiny` settings on this page
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) -- the SMTP-time gate;
  every check on this page runs only after a connection clears the
  perimeter
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) -- seals over the body Amavis
  passed, so a high spam score (and any quarantine action) naturally
  pre-empts the seal
- [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) -- a DMARC-fail verdict can
  promote a message to a higher spam score via SpamAssassin's DMARC
  rule weights (tunable on Score Overrides)
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) -- `sa-update`
  for the SpamAssassin rule set runs on its own Ofelia schedule; the
  Bayes DB is per-installation and not updated by `sa-update`
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) -- full pipeline diagram

# Antivirus Settings

# Antivirus Settings

Admin path: **Content Checks > Antivirus Settings**
(`view_antivirus_settings.cfm`, `inc/get_antivirus_settings.cfm`,
`inc/antivirus_set_settings.cfm`,
`inc/antivirus_add_whitelists.cfm`,
`inc/antivirus_delete_entry.cfm`,
`inc/generate_antivirus_configuration.cfm`,
`inc/restart_clamav.cfm`).

This page configures the ClamAV antivirus engine that runs inside
`hermes_mail_filter` and is called by Amavis on every message that
clears the SMTP-time perimeter. Two cards: the main settings card
(sixteen toggles that map to `clamd.conf` directives) and a Pro-only
AV Signature Whitelist for suppressing known-bad-signature false
positives. Refreshing third-party signature *feeds* (Sanesecurity,
SecuriteInfo, MalwarePatrol, etc.) is configured separately on
[Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds); this page configures the engine
itself.

## Where antivirus sits in the flow

```
                  +-----------------------------------+
   inbound msg -->| Perimeter Checks pass             |
                  +---------------+-------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  Postfix smtpd_proxy_filter       |
                  |    -> hermes_mail_filter:10024    |
                  +---------------+-------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  Amavis (hermes_mail_filter)      |
                  |   - SpamAssassin scoring          |
                  |   - ClamAV antivirus  <---- this page configures this engine
                  |   - banned-file checks            |
                  +---------------+-------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  Re-inject -> hermes_postfix_dkim:10026
                  +-----------------------------------+
                                  |
                                  v
                  +-----------------------------------+
                  |  OpenDKIM sign, ARC seal, deliver |
                  +-----------------------------------+
```

Amavis calls ClamAV over the local socket; the verdict
determines whether Amavis quarantines, blocks, or passes the
message. Amavis's own action policy (the `final_*_destiny`
settings — quarantine vs DSN vs discard) lives in
[Antispam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) and the per-domain
policy table, not on this page. This page is **engine knobs only**.

## Container and socket placement

| Component | Detail |
| --- | --- |
| Container | `hermes_mail_filter` (IPv4 `.105`) |
| Engine | `clamd` daemon, Unix socket inside the container |
| Daemon config | `/etc/clamav/clamd.conf` (volume-mounted from `./config/mail_filter/etc/clamav/clamd.conf`) |
| Signature dir | `/var/lib/clamav/` (Docker named volume `mail_filter_data_clamav`) |
| Signature whitelist | `/var/lib/clamav/local.ign2` (regenerated from `parameters2 WHERE module='clamav-bypass'` on every save) |
| Third-party feeds | `/etc/fangfrisch/fangfrisch.conf` + `/var/lib/fangfrisch/signatures/` (see [Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds)) |
| Base signature refresh | `freshclam` (official ClamAV CVD updates, default 1h) |
| Feed refresh | `fangfrisch refresh` on a 10-minute Ofelia job (`hermes-fangfrisch-refresh`) |

The container exposes **no host ports** — Amavis is reached only by
Postfix internally at `hermes_mail_filter:10024` and re-injects to
`hermes_postfix_dkim:10026`.

## ClamAV Antivirus Settings card

Sixteen toggles, each rendered from the `avSettings` array in
`view_antivirus_settings.cfm` with an inline hint and a "Recommended"
label on the safer default. Every toggle writes
`parameters2.value2 = 'true' | 'false'` for `module = 'clamav'`; on
save, `generate_antivirus_configuration.cfm` selects every active row
and emits one ` ` line per toggle into a temp file,
substitutes the temp file into the `HERMES_ANTIVIRUS_SETTINGS_GO_HERE`
placeholder of `clamd.conf.HERMES`, backs up the live config to
`clamd.conf.HERMES`, and moves the rendered file into place.

| UI Toggle | `clamd.conf` directive | Recommended | Notes |
| --- | --- | --- | --- |
| Scan Email Attachments | `ScanMail` | Enabled | Master switch for inbound attachment scanning |
| Scan Archives | `ScanArchive` | Enabled | Recurse into ZIP, RAR, 7z, etc. Without this, only the archive wrapper is scanned |
| Mark Encrypted Archives as Viruses | `ArchiveBlockEncrypted` | Disabled | Aggressive; commonly false-positives on legitimate password-protected files |
| Scan Portable Executables | `ScanPE` | Enabled | Windows PE format; required for decompression of UPX / FSG / Petite packers |
| Scan OLE2 Files | `ScanOLE2` | Enabled | MS Office `.doc/.xls/.ppt` and `.msi` |
| Block OLE2 VBA Macros | `OLE2BlockMacros` | Disabled | Blocks ALL macro-enabled documents regardless of intent (detected as `Heuristics.OLE2.ContainsMacros`); useful in strict environments, breaks legitimate macros otherwise |
| Scan PDF Files | `ScanPDF` | Enabled | PDF embedded JS, exploit detection |
| Scan HTML/JavaScript Content | `ScanHTML` | Enabled | HTML normalization + JavaScript/ScriptEncoder decryption; phishing + script-exploit detection |
| Algorithmic Detection | `AlgorithmicDetection` | Enabled | Engine-level heuristics for complex malware and graphic-file exploits |
| Scan ELF Files | `ScanELF` | Enabled | Linux/Unix executable format |
| Phishing Signature Detection | `PhishingSignatures` | Enabled | ClamAV's phishing signature DB |
| Scan Email URLs for Phishing | `PhishingScanURLs` | Enabled | URL extraction + phishing URL DB lookup |
| Block SSL Mismatches in URLs | `PhishingAlwaysBlockSSLMismatch` | Disabled | False-positives on CDN and redirect URLs |
| Block Cloaked URLs | `PhishingAlwaysBlockCloak` | Disabled | False-positives on URL shorteners and marketing-tracker links |
| Detect Potentially Unwanted Applications | `DetectPUA` | Enabled | Adware, dialers, non-malicious-but-unwanted software |
| Heuristic Scan Precedence | `HeuristicScanPrecedence` | Enabled | When on, heuristic hits stop the scan immediately (saves CPU). When off, scanning continues so a signature-based hit can override a heuristic match |

> **Operational consequence — disabling `ScanMail`.** This effectively
> turns off antivirus for inbound mail. Amavis will still consult
> ClamAV for ban-pattern decisions but the engine will skip the
> attachment scan. Leave on except for very short-term diagnostics.
>
> **Operational consequence — `OLE2BlockMacros` = true.** Every
> macro-enabled Office document is blocked as `Heuristics.OLE2.ContainsMacros`,
> including documents from your own users. Most organizations get
> better results with macro-blocking enforced at the endpoint
> (Microsoft 365 Protected View, Group Policy) rather than at the
> gateway. Turn on only after warning users and ensuring you have a
> release workflow.

## AV Signature Whitelist card (Pro)

When ClamAV produces a false positive on a known-safe file, the
admin enters the exact ClamAV signature name (e.g.
`Heuristics.OLE2.ContainsMacros`) and Hermes appends it to
`/var/lib/clamav/local.ign2`. ClamAV reads `local.ign2` at engine
start and suppresses any detection whose signature name matches a
line in the file.

Storage:
`parameters2 WHERE module = 'clamav-bypass'` (one row per signature
name, `parameter` column holds the signature string). On every save
and on every delete, `generate_antivirus_configuration.cfm` rewrites
the whole `local.ign2` from the table, runs `dos2unix` to scrub line
endings, backs up the current file to `local.ign2.HERMES`, and moves
the new file into place. ClamAV is then restarted via
`restart_clamav.cfm` to pick up the change.

### How to find a signature name

The in-card info box gives admins the lookup steps:

1. From **Message History**, find the blocked message (Type column
   shows `Virus` or `Banned`)
2. Grep the mail-filter log for the message ID:
   `docker logs hermes_mail_filter 2>&1 | grep `
3. The log line shows the signature in parentheses, e.g.
   `Blocked INFECTED (Heuristics.OLE2.ContainsMacros)`
4. Or scan a file directly:
   `docker exec hermes_mail_filter clamscan /path/to/file`

> **Operational consequence — whitelisting is by signature name, not
> by file hash.** If you whitelist `Heuristics.OLE2.ContainsMacros`,
> you have effectively turned off macro detection globally. Prefer
> narrow signature names (specific malware family) over heuristic
> families when possible.

## Signature refresh

Two independent refresh loops keep the engine current:

| Source | Mechanism | Cadence | Database |
| --- | --- | --- | --- |
| Official ClamAV (`main.cvd`, `daily.cvd`, `bytecode.cvd`) | `freshclam` daemon inside `hermes_mail_filter` | Default 1h (configurable in `/etc/clamav/freshclam.conf`) | `/var/lib/clamav/` |
| Third-party feeds (Sanesecurity, SecuriteInfo, MalwarePatrol, etc.) | `fangfrisch refresh` via Ofelia job `hermes-fangfrisch-refresh` | Every 10 minutes (only feeds whose own publish cycle has elapsed actually re-download) | `/var/lib/fangfrisch/signatures/` then linked into `/var/lib/clamav/` by `setup-clamav-sigs` |

`fangfrisch` is the small Python tool that handles auth, cadence
control, and integrity verification for third-party feeds; the feed
list and per-feed enable/disable lives on
[Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds). Enabling premium feeds
(SecuriteInfo paid, MalwarePatrol paid) requires Pro licensing —
the feed list itself is gated on the same page.

## Resource footprint

Loading the full signature database into RAM costs roughly 1.5–2 GB
of memory. If `hermes_mail_filter` is under-provisioned (e.g. shared
host with 4 GB total), `clamd` will fail to start, mail will queue
behind Amavis, and the only sign in the UI is a quiet rise in
deferred queue depth. Plan for at least 4 GB dedicated to the
`hermes_mail_filter` container on systems with all third-party
feeds enabled.

The default ClamAV file-size cap is 25 MB (`MaxFileSize 25M` in
`clamd.conf`). Messages larger than this are passed without scan and
flagged with a `Heuristics.Limits.Exceeded` indicator. Raising the
cap requires editing `clamd.conf.HERMES` directly; the UI does not
expose it because raising it disproportionately increases RAM and
CPU per scan.

## Save flow

```
1. View page submits action="AV Settings" (sixteen booleans),
                       action="Add AV Whitelist" (textarea),
                       action="Delete Entry" (id list)
2. view_antivirus_settings.cfm validates every avFields entry exists and is true|false
   (any failure -> error.cfm + cfabort)
3. antivirus_set_settings.cfm UPDATEs parameters2.value2 for each toggle
   (16 UPDATEs, module='clamav')
4. generate_antivirus_configuration.cfm:
     a. SELECT active='1' rows from parameters2 module='clamav' -> temp avsettings file
     b. dos2unix the temp file
     c. Substitute into clamd.conf.HERMES placeholder HERMES_ANTIVIRUS_SETTINGS_GO_HERE
     d. Back up /etc/clamav/clamd.conf -> clamd.conf.HERMES, move new file into place
     e. Rebuild /var/lib/clamav/local.ign2 from parameters2 module='clamav-bypass'
     f. dos2unix, back up local.ign2 -> local.ign2.HERMES, move new file into place
     g. cfinclude restart_clamav.cfm (docker container restart hermes_mail_filter ClamAV process)
5. session.m = 9 -> green "Antivirus Settings were saved successfully" alert
```

`generate_antivirus_configuration.cfm` also runs on whitelist
add/delete — every change to either card triggers the same full
regen + ClamAV restart cycle. The page does not return until the
restart has completed (timeout per `cfexecute`).

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Toggle form missing a required boolean field | `m = "Antivirus Settings: form. does not exist"`, `error.cfm`, cfabort |
| Toggle value not in `true,false` | `m = "Antivirus Settings: form. is not true or false"`, `error.cfm`, cfabort |
| Delete clicked with no selection | session.m = 11 |
| Add Whitelist with empty textarea | session.m = 13 |
| `dos2unix` failure on the temp avsettings or local.ign2 file | `error.cfm` + cfabort with the failing path in the message |
| `cp /etc/clamav/clamd.conf -> .HERMES` failure | `error.cfm` + cfabort |
| `mv _clamd.conf -> /etc/clamav/clamd.conf` failure | `error.cfm` + cfabort |
| `restart_clamav.cfm` failure | Surfaces as cfcatch from the docker restart step |

The save is **not** transactional across the steps — if the SQL
updates succeed but the ClamAV restart fails, the DB state has
already advanced. The next save will re-render and re-apply
because every save regenerates the entire file from the current row
state (no incremental writes).

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_antivirus_settings.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/antivirus_*.cfm` | `hermes_commandbox` | Validate / save / regenerate / restart |
| `config/hermes/var/www/html/admin/2/inc/get_antivirus_settings.cfm` | `hermes_commandbox` | Loads current `parameters2 module='clamav'` values |
| `config/hermes/opt/hermes/conf_files/clamd.conf.HERMES` | `hermes_commandbox` (read) -> `hermes_mail_filter` (live `/etc/clamav/clamd.conf`) | Canonical template with `HERMES_ANTIVIRUS_SETTINGS_GO_HERE` placeholder |
| `config/mail_filter/etc/clamav/clamd.conf` | `hermes_mail_filter` (live config, bind-mounted) | Read by `clamd` at start |
| `/var/lib/clamav/local.ign2` | `hermes_mail_filter` (Docker named volume `mail_filter_data_clamav`) | Signature whitelist; rewritten on every save |
| `/var/lib/clamav/*.cvd`, `*.cld`, `*.ndb`, etc. | `hermes_mail_filter` | Signature databases (official + third-party) |
| `parameters2` table, `module='clamav'` | `hermes_db_server` (`hermes` DB) | Source of truth for the sixteen toggles |
| `parameters2` table, `module='clamav-bypass'` | `hermes_db_server` (`hermes` DB) | Source of truth for the AV Signature Whitelist |
| `malware_databases` table | `hermes_db_server` (`hermes` DB) | Third-party feed list (configured on [Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds)) |
| `ofelia_jobs` row `hermes-fangfrisch-refresh` | `hermes_db_server` | 10-minute feed refresh scheduler |
| `hermes_mail_filter` container | — | `clamd`, `freshclam`, `fangfrisch`, Amavis, SpamAssassin |

## Related

- [Malware Feeds](https://docs.deeztek.com/books/administrator-guide/page/malware-feeds) — the third-party signature feed
  configuration (Sanesecurity, SecuriteInfo, MalwarePatrol, etc.)
  that Fangfrisch refreshes every 10 minutes
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — every check on this page
  runs only after a connection clears the SMTP-time perimeter
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — runs in the same
  Amavis pass; a virus verdict overrides any spam score
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — per-rule weight changes
  for SpamAssassin
- [Email Policies > Disclaimers](https://docs.deeztek.com/books/administrator-guide/page/disclaimers) —
  body modification that runs after Amavis re-injection; never
  conflicts with ClamAV because it happens post-scan
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — seals over the body Amavis
  passed, so a virus verdict naturally pre-empts everything
  downstream
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — URL phishing
  lookups (`PhishingScanURLs`) and signature-feed downloads
  (Fangfrisch) all resolve through `hermes_unbound`
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) — full pipeline diagram

# ARC Settings

# ARC Settings

Admin path: **Content Checks > ARC Settings** (`view_arc_settings.cfm`)

## What ARC does

ARC (Authenticated Received Chain, [RFC 8617](https://datatracker.ietf.org/doc/html/rfc8617))
preserves authentication results across forwarding gateways. Each
gateway that handles a message can add a sealed record of the
authentication state it observed, so a downstream verifier can trust
the cumulative chain even when an intermediate gateway modifies the
message body (adding disclaimers, banners, forwarding annotations,
etc.) — body modification would otherwise invalidate the original
sender's DKIM signature and lose DMARC alignment.

Hermes participates in ARC at two roles:

1. **As an originating sealer** for mail submitted by authenticated
   Hermes users to external recipients — Hermes is the first hop in
   the chain (`i=1; cv=none`).
2. **As a forwarding sealer** for inbound mail being relayed to a
   downstream MX (relay-mode domains) — Hermes adds a seal at `i=N+1`
   referencing the upstream chain.

## Container and milter placement

| Component | Detail |
| --- | --- |
| Container | `hermes_openarc` (separate service, IPv4 `.114`) |
| Listen | `inet:8893` |
| Source | [flowerysong/OpenARC](https://github.com/flowerysong/openarc) v1.3.0, built from release tarball |
| Milter chain | `master.cf` `:10026` only (post-amavis re-injection, after OpenDKIM signer at `:8892`) |
| NOT in | `main.cf` default `smtpd_milters` — sealing at `:25` over the pre-modification body would produce an invalid seal once body_milter and CipherMail change the bytes |

## Modes

| Mode | Effect |
| --- | --- |
| `s` (sign only) | Adds Hermes's seal but does not validate upstream chains |
| `v` (verify only) | Records inbound chain validity in `Authentication-Results` headers; does not add a seal |
| `sv` (sign + verify) | The gateway default; validates upstream then seals over the final body |

The ARC Settings page slider auto-syncs Mode between `sv` (enabled) and
`v` (disabled). The master `arc_signing_enabled` flag controls whether
the daemon adds anything at all — when disabled, OpenARC operates in
pass-through mode (every peer in `PeerList`, no headers added).

## Single signing identity per gateway

Unlike DKIM (which uses per-sender-domain keys), ARC uses a single
signing identity per gateway — Gmail seals everything with
`d=google.com`, Microsoft 365 with `d=outlook.com`, and Hermes with
whatever domain you generate the key for. Pick a domain you control
(typically your own organization's primary domain). The selector
follows the same DNS publication pattern as DKIM:
`._domainkey.` with value
`v=DKIM1; k=rsa; p=`.

## Hermes is the auth boundary — what `cv=fail` means and doesn't mean

Hermes is the authoritative auth / security boundary for every domain
it serves. Inbound DKIM, SPF, DMARC, ARC verify, spam, virus checks
all happen at Hermes. Body modifications (External Sender Banner,
disclaimer, signature insertion, encryption) also happen at Hermes.
Customer downstream mail servers are expected to be configured to
trust Hermes implicitly: allowlist Hermes by IP / hostname, accept
forwarded mail without re-running upstream auth checks. This matches
how Mimecast, Proofpoint, and Barracuda customers deploy those
products — the SEG IS the trust boundary.

When Hermes modifies a message body (banner, disclaimer, etc.), any
cryptographic signature whose body hash was computed over the original
bytes will no longer body-validate against the current bytes. This
affects:

1. The original sender's `DKIM-Signature` body hash
2. The upstream `ARC-Message-Signature` body hash for each prior `i=`

Hermes's own outbound seal at `i=N+1` is mathematically valid (it is
computed over the modified body), but the `cv=` field on that seal
must honestly report whether the upstream chain passed when Hermes
received the message AND remains body-valid in the message it is about
to send. Once Hermes modifies the body, the upstream `bh=` no longer
matches the current body, so `cv=fail` is the correct (and only
defensible) value.

**This is by design.** A correctly-configured customer downstream MX
allowlists Hermes and does not re-check auth on Hermes-forwarded mail;
the `cv=fail` and broken DKIM signals never gate delivery. If a
customer reports forwarded mail being rejected by their downstream
MX due to ARC / DKIM / DMARC failure, the fix is to allowlist Hermes
on their MX, not to silence Hermes.

Removing Hermes's seal does not help: the verifier walks the chain
back to `i=1` and recomputes each prior body hash against the current
body independently of our seal. Stripping the entire upstream chain
would require Hermes to rewrite the `From:` header (mailing-list
style) to maintain DMARC alignment with a domain Hermes controls —
this is a significant UX cost that all major SEG vendors (Mimecast,
Proofpoint, Barracuda) have chosen not to pay.

### Default Hermes behavior

| Scenario | Behavior |
| --- | --- |
| Inbound mail with NO upstream ARC chain → any local recipient | Banner injects; Hermes seals at `i=1; cv=none`; chain is clean |
| Inbound mail with upstream ARC → local mailbox recipient | Banner injects; Hermes seals at `i=N+1; cv=fail`; message ends at Hermes (no downstream chain to protect — `cv=fail` is just bytes in the user's inbox) |
| Inbound mail with upstream ARC → relay-mode recipient | Banner injects; Hermes seals at `i=N+1; cv=fail`; downstream MX (which should be allowlisting Hermes) accepts and delivers regardless |
| Outbound from local Hermes user → external | Hermes is the first sealer; `i=1; cv=none`; clean chain to downstream |

There is no toggle, no conditional skip, no per-domain override.
Hermes always behaves the same way and reports the chain state
honestly. Customer-side trust configuration is the responsibility
of the customer's MX administrator.

### When a Trusted ARC Sealer configuration helps

Trusted ARC Sealer configuration on the customer side is useful in
**cross-org scenarios** that aren't direct relay-to-customer-MX —
for example, when a Hermes-served domain is part of a chain that
forwards through other gateways, or when Hermes is forwarding to a
third-party tenant the customer doesn't control. See the
[Trusted ARC Sealers — M365 guide](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) for
the M365 PowerShell configuration. For the standard
Hermes-as-relay-MX-to-customer-mail-server case, IP allowlisting on
the customer's MX is simpler and sufficient.

## When to ask receivers to trust Hermes as a sealer

For customers running strict downstream verifiers (Microsoft 365
tenants that DMARC-enforce, Gmail Workspace receivers that escalate
on `arc=fail`, etc.), the chain-integrity limitation can cause
relay-out delivery issues even on benign inbound that happens to come
through an upstream sealer. The standard industry remedy is for the
receiver to add Hermes to its **Trusted ARC Sealers** list.

For Microsoft 365 customers, follow the
[Trusted ARC Sealers — M365 guide](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) which
covers the PowerShell command, identity requirements, and verification
steps.

## Key management workflow

1. Click **Add ARC Key** in the *Gateway ARC Signing Identity* card
2. Enter the signing domain (must validate as `bob@`) and
   selector (DNS-safe label, e.g. `arc1`)
3. Choose key size (RSA 1024 or 2048)
4. Hermes generates the key pair in `/opt/hermes/arc/keys/`
5. Copy the public key TXT record and publish at
   `._domainkey.` in your authoritative DNS
6. Verify DNS propagation, then click the slider to enable signing

Without an active key, Mode is forced to `v` (verify only) regardless
of the saved Mode setting.

## Troubleshooting

| Symptom | Likely cause |
| --- | --- |
| Gmail "Show original" shows `arc=fail (signature failed)` on outbound from a local Hermes user | DNS for selector not published, propagated incorrectly, or wrong key |
| Downstream MX rejects forwarded mail from M365 sender with `arc=fail` | Expected when upstream ARC + body modification meet on relay-out; either ensure the conditional banner skip is active (`/etc/hermes/body_milter/relay_domains` is populated) or ask the receiver to configure Hermes as a Trusted ARC Sealer |
| OpenARC fails to start with `key data is not secure` | The signing key file ownership is not `openarc:openarc` or permissions are too loose; check the entrypoint chown step |
| ARC headers absent from outbound entirely | `arc_signing_enabled = 0` (master off), or no enabled key exists for the configured `arc_mode` |

## Related

- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) — full pipeline diagram including ARC placement
- [DKIM Settings](https://docs.deeztek.com/books/administrator-guide/page/dkim-settings) — outbound signing (separate from ARC)
- [Trusted ARC Sealers — M365](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) — receiver-side trust configuration

# BCC Maps

# BCC Maps

Admin path: **Content Checks > BCC Maps**
(`view_bcc_maps.cfm`, `inc/add_bcc_map_action.cfm`,
`inc/edit_bcc_map_action.cfm`,
`inc/delete_bcc_map_action.cfm`,
`inc/get_bcc_map_json.cfm`,
`inc/get_mailbox_bcc_count.cfm`).

This page manages **silent message copies** at the SMTP envelope
layer. Each entry maps an envelope address (sender **or** recipient,
chosen per row) to a BCC target; when mail matching the address flows
through Postfix, an additional copy is generated and routed to the
target. The original delivery is unaffected; neither the original
sender nor the original recipient sees any indication that a copy was
made.

BCC Maps is the sibling envelope-level rule table to
[Global Sender Rules](https://docs.deeztek.com/books/administrator-guide/page/global-sender-rules). Where Global Sender
Rules decide whether a message is allowed in or blocked, BCC Maps
decides whether an additional copy is created — both work on the
envelope, before the message body is parsed.

## How Postfix BCC works

Postfix has two distinct directives for envelope-level BCC injection:

| Directive | Lookup key | Adds BCC when... | Typical use |
| --- | --- | --- | --- |
| `sender_bcc_maps` | Envelope **sender** (`MAIL FROM`) | The matched address is the one sending the message | Journaling outbound mail from an executive, monitoring a compromised account |
| `recipient_bcc_maps` | Envelope **recipient** (`RCPT TO`) | The matched address is the one receiving the message | Compliance journaling of mail to a regulated mailbox, legal-hold copies |

The two maps are queried independently on every message — a single
delivery can hit both if both a sender BCC and a recipient BCC match.
The BCC happens once Postfix has accepted the message; the original
envelope is preserved and the additional copy is queued separately.

Hermes wires both directives to MySQL-backed lookup tables in
`/etc/postfix/main.cf`:

```
sender_bcc_maps    = mysql:/etc/postfix/mysql-sender-bcc-maps.cf
recipient_bcc_maps = mysql:/etc/postfix/mysql-recipient-bcc-maps.cf
```

Each `.cf` file holds a SQL query that selects `bcc_to` from
`bcc_maps` where the address column matches and the row is enabled.

```sql
-- mysql-sender-bcc-maps.cf
SELECT bcc_to FROM bcc_maps
WHERE address='%s' AND bcc_type='sender' AND enabled=1

-- mysql-recipient-bcc-maps.cf
SELECT bcc_to FROM bcc_maps
WHERE address='%s' AND bcc_type='recipient' AND enabled=1
```

> **No reload required.** Unlike hashed `check_sender_access` lookups
> (used by [Global Sender Rules](https://docs.deeztek.com/books/administrator-guide/page/global-sender-rules)), MySQL
> lookups are evaluated live against the database on every message —
> there is no `postmap` step, no `postfix reload`. Adding, editing,
> disabling, or deleting a row takes effect on the **next inbound
> message**. The UI surfaces this implicitly: the success alerts say
> "entry created/updated/deleted" without the
> "Postfix reloaded and Amavis restarted" suffix that other envelope
> pages append.

## The page

A single info callout, an Add button that opens a modal, and one
DataTable.

### Add BCC Map modal

| Field | Stored as | Notes |
| --- | --- | --- |
| Address | `bcc_maps.address` | The envelope address to watch. Full email (`user@domain.tld`) or `@domain.tld` for domain-wide. Lower-cased on save |
| Type | `bcc_maps.bcc_type` | `sender` (outbound mail from this address) or `recipient` (inbound mail to this address) |
| BCC To | `bcc_maps.bcc_to` | The address that receives the silent copy. Single email only; not a pattern. Lower-cased on save |
| Description | `bcc_maps.description` | Free-text label (e.g. "Legal compliance — exec journaling"); nullable |

The handler validates Address against `IsValid("email", ...)` for full
addresses and against a `@domain` pattern check for domain-wide rows.
BCC To must be a valid email address — domain patterns are not
accepted here, only a concrete delivery target. The
`(address, bcc_type)` pair is `UNIQUE` in the schema, so attempting to
add a second row with the same address and type returns alert
`m = 14` and rejects the insert.

### BCC Maps (DataTable)

| Column | Source |
| --- | --- |
| Actions | Edit (modal, AJAX load via `get_bcc_map_json.cfm`), Delete (confirm modal) |
| Address | `bcc_maps.address` |
| Type | `bcc_maps.bcc_type` -> Sender badge (primary) or Recipient badge (info) |
| BCC To | `bcc_maps.bcc_to` |
| Status | `bcc_maps.enabled` -> Enabled badge (green) or Disabled badge (grey) |
| Description | `bcc_maps.description` (em-dash if empty) |

### Edit constraints

The Edit modal makes **Address** and **Type** read-only — they are
the natural key of the row (`UNIQUE (address, bcc_type)`) and changing
them would semantically be a different rule. To re-target a watched
address, delete the row and add a new one. Only **BCC To**, **Status**
(enabled / disabled), and **Description** can be changed in place.

The Status toggle is the right tool for pausing surveillance briefly
without losing the row — e.g. a compliance journaling rule that
should be off during a planned mail-flow test.

## The `bcc_maps` table

| Column | Purpose |
| --- | --- |
| `id` | Auto-increment primary key |
| `address` | The watched envelope address (full email or `@domain.tld`) |
| `bcc_to` | The silent-copy target address |
| `bcc_type` | `sender` or `recipient` |
| `enabled` | `1` = active, `0` = paused (row preserved, no BCC generated) |
| `description` | Optional free-text label |
| `created_at` | Auto-populated timestamp on insert |
| `UNIQUE KEY` | `(address, bcc_type)` — same address can have one sender BCC AND one recipient BCC, but not two of either |

## BCC mail still goes through content filtering

Important behavior to understand: the BCC copy that Postfix generates
is a real message in its own right, with the BCC target as its
recipient. That copy traverses the same pipeline as any other inbound
delivery — it goes through Amavis, SpamAssassin, ClamAV, the
[Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) for the BCC
target, and any per-recipient quarantine policy.

The consequences:

- **A clean original can produce a quarantined BCC.** If the BCC
  target's spam threshold is stricter than the original recipient's,
  or if a recipient rule rejects the BCC sender, the silent copy can
  be quarantined or dropped while the original delivers normally.
- **A clean original can produce a bounced BCC.** If the BCC target
  is on an external server, that server's SPF / DMARC / receiver
  policy will be evaluated against the original sender's domain
  (which almost certainly does not authorise Hermes's IP). The
  external server may reject the BCC even though the original sender
  has nothing to do with the relay.
- **The BCC failure is silent to the original sender.** Postfix
  generated the BCC after accepting the original message; the
  original sender's SMTP transaction has already closed successfully.
  Any bounce of the BCC goes to the BCC target's `MAIL FROM`
  (typically the original sender, depending on `bounce_size_limit`)
  or to a double-bounce mailbox, but never causes the original
  delivery to fail.

The page's info callout flags the SPF case explicitly. For a
journaling / compliance use case where loss of a copy is
unacceptable, the BCC target should be a **local mailbox** on the
same Hermes instance — the message stays inside the gateway, the
external-receiver policy issue does not arise, and any spam-tier
issue is visible to the local mailbox owner.

## Privacy and compliance

BCC Maps is a surveillance feature. The original sender and the
original recipient are never notified that a copy was made; that is
the point.

Operationally that means:

- **Auditability.** Each row carries a `created_at` timestamp; the
  `description` column is intended for the policy reference that
  justifies the watch (regulatory citation, ticket number, legal-hold
  matter ID). Filling it in is strongly recommended for any rule
  that is not strictly self-explanatory.
- **GDPR / employee-monitoring regimes.** In jurisdictions that
  require explicit employee notification of mail surveillance
  (EU member states, several US states for employee monitoring of
  personal communication), the existence of these rules must be
  disclosed in the employee privacy notice. Hermes does not generate
  that notice — the operator is responsible for the legal compliance
  wrapping around any active row.
- **Access control.** The page is only available to authenticated
  admins under `/admin/2/`. There is no end-user surface for BCC
  maps; mailbox owners cannot see whether their address is watched.

## Cascading delete on mailbox removal

When a mailbox is deleted from
[Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes),
`inc/delete_mailbox_action.cfm` (step 4b) issues:

```sql
DELETE FROM bcc_maps
 WHERE address = :deleted_mailbox
    OR bcc_to  = :deleted_mailbox
```

That is — every BCC rule referencing the deleted mailbox is removed,
whether the mailbox was the watched address or the BCC target.
Because the live MySQL lookup re-reads on every message, the change
takes effect immediately; no postmap or reload runs.

The same delete handler calls the AJAX endpoint
`inc/get_mailbox_bcc_count.cfm` from the confirmation modal **before**
the deletion fires, so the admin sees the number of BCC rows that
will be cascaded ("This mailbox is watched by 2 BCC rules and is the
target of 1 BCC rule") and can cancel.

Domain-pattern rows (`@domain.tld`) are **not** cascaded by mailbox
deletion — they reference a domain, not a specific mailbox, and
remain in place until the whole domain is removed or the row is
deleted manually.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` / `2` / `3` | Add / Edit / Delete success |
| `m = 10` | Address field blank on Add |
| `m = 11` | Address fails email-or-`@domain` syntax check |
| `m = 12` | BCC To blank on Add or Edit |
| `m = 13` | BCC To is not a valid email address |
| `m = 14` | An entry with the same `(address, bcc_type)` already exists |
| `m = 20` | Missing required form field on Edit / Delete (no `bcc_id`) |
| `m = 21` | Edit / Delete target row no longer exists |

There is no `session.m = 4` "Apply Failed" path because there is
nothing to apply — the next message Postfix processes will read the
new row from MySQL directly.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_bcc_maps.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/add_bcc_map_action.cfm` | `hermes_commandbox` | Validate + INSERT |
| `config/hermes/var/www/html/admin/2/inc/edit_bcc_map_action.cfm` | `hermes_commandbox` | Validate + UPDATE (only `bcc_to`, `enabled`, `description`) |
| `config/hermes/var/www/html/admin/2/inc/delete_bcc_map_action.cfm` | `hermes_commandbox` | DELETE single row |
| `config/hermes/var/www/html/admin/2/inc/get_bcc_map_json.cfm` | `hermes_commandbox` | AJAX endpoint for the Edit modal |
| `config/hermes/var/www/html/admin/2/inc/get_mailbox_bcc_count.cfm` | `hermes_commandbox` | AJAX endpoint for the mailbox-delete confirmation modal |
| `config/postfix-dkim/etc/postfix/mysql-sender-bcc-maps.cf` | `hermes_postfix_dkim` | MySQL lookup definition for `sender_bcc_maps` |
| `config/postfix-dkim/etc/postfix/mysql-recipient-bcc-maps.cf` | `hermes_postfix_dkim` | MySQL lookup definition for `recipient_bcc_maps` |
| `bcc_maps` table | `hermes_db_server` (`hermes` DB) | Source of truth |
| `hermes_postfix_dkim` container | — | Reads MySQL lookups live on every message |

## Related

- [Global Sender Rules](https://docs.deeztek.com/books/administrator-guide/page/global-sender-rules) — sibling
  envelope-level rule table; allow/block decisions rather than copy
  generation
- [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) — the
  per-pair table that the BCC copy will also pass through on its way
  to the BCC target
- [Mailboxes](https://docs.deeztek.com/books/administrator-guide/page/mailboxes) — deleting a mailbox
  cascades the cleanup of any BCC rows referencing it; the
  confirmation modal surfaces the count before the deletion
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — sibling Content Checks
  page; envelope-time rejects that fire before any BCC is generated
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) /
  [Anti-Virus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — the content-filter
  tier that BCC copies traverse alongside the original message
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) — both the original and the
  BCC copy appear as separate entries in the message log
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — Postfix's
  `mail.log` records BCC generation as standard delivery lines, one
  per copy
- [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) — a deferred BCC
  (external target rejecting on SPF, for example) sits in the queue
  here for inspection

# DKIM Settings

# DKIM Settings

Admin path: **Content Checks > DKIM Settings**
(`view_dkim_settings.cfm`, `inc/get_dkim_settings.cfm`,
`inc/dkim_save_settings.cfm`, `inc/dkim_set_settings.cfm`,
`inc/dkim_generate_config_file.cfm`,
`inc/dkim_generate_keytable.cfm`,
`inc/dkim_generate_signingtable.cfm`,
`inc/dkim_generate_hosts.cfm`,
`inc/dkim_generate_domains.cfm`,
`inc/restart_opendkim.cfm`,
`inc/generate_postfix_configuration.cfm`).

This page controls **inbound DKIM verification** and the **OpenDKIM
runtime configuration** that also drives outbound signing.
DKIM ([RFC 6376](https://datatracker.ietf.org/doc/html/rfc6376)) lets
a sending domain attach a cryptographic signature
(`DKIM-Signature: v=1; a=rsa-sha256; d=example.com; s=mail1; ...`)
covering selected headers and a hash of the message body; receivers
fetch the public key at `._domainkey.` in DNS and
verify the signature. Unlike SPF, DKIM survives most forwarding —
the signature stays attached to the message and verifies wherever the
body and signed headers remain unchanged.

Per-domain key generation (selector, RSA 1024 / 2048, DNS TXT record
to publish) is managed elsewhere — on the
[Email Server Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) page via
`edit_domain_dkim.cfm`, which writes rows into the `dkim_sign` table.
This Settings page configures the OpenDKIM daemon's runtime behavior
and maintains the verification-side bypass lists.

## Two OpenDKIM instances, one config page

To avoid the body-modification trap that breaks any signer running
after a body-modifying milter, Hermes (issue #232) runs **two
separate OpenDKIM instances** inside `hermes_postfix_dkim`:

| Instance | Config | Socket | Mode | Role |
| --- | --- | --- | --- | --- |
| Primary | `/etc/opendkim.conf` | `inet:8891@0.0.0.0` | `sv` (sign + verify) | Verifies inbound DKIM at `smtpd :25`; signs outbound at `:587` / `:465` (submission ports — pre-Amavis, pre-CipherMail) |
| Sign-only | `/etc/opendkim-sign.conf` | `inet:8892@127.0.0.1` | `s` (sign only) | Signs at the `:10026` re-injection port **after** Amavis, CipherMail, and the body milter have finished modifying the body. Never adds an `Authentication-Results` header |

Both instances share the same key tables
(`/opt/hermes/dkim/KeyTable`,
`/opt/hermes/dkim/SigningTable`) and the same trusted-hosts /
exempt-domains lists; the page below feeds both. The reason for two
instances: a single sv-mode OpenDKIM on `:10026` would verify the
post-modification body of inbound mail flowing through the re-inject
port and emit a spurious `dkim=fail` Authentication-Results header.
Sign-only mode at `:10026` produces the final outbound signature over
the byte sequence the receiver will actually see.

## Where DKIM sits in the flow

```
+--------------------------+
| Remote SMTP peer         |
+-----------+--------------+
            |
            v
+-----------+--------------------------------+
| smtpd :25 (hermes_postfix_dkim)             |
|   smtpd_milters = inet:127.0.0.1:8891, ...  |
|     primary OpenDKIM (sv) verifies inbound  |
|     DKIM-Signature, adds                    |
|     Authentication-Results: dkim=pass/...   |
|     (consumed downstream by OpenDMARC)      |
+-----------+--------------------------------+
            |
            v
        Amavis :10024 (content scoring, CipherMail)
            |
            v (reinject)
+-----------+--------------------------------+
| smtpd :10026 (post-content, post-body-mod)  |
|   smtpd_milters = inet:127.0.0.1:8891       |
|     sign-only OpenDKIM at :8892 actually    |
|     signs the final outbound body           |
|     (KeyTable selects per-domain key by     |
|      "*@" SigningTable match)       |
+-----------+--------------------------------+
            |
            v
        OpenARC seal (if enabled)
            |
            v
        Outbound to receiver
```

The actual signing decision happens against the `SigningTable`:

```
# /opt/hermes/dkim/SigningTable
*@example.com       mail1._domainkey.example.com
*@partner.org       k2024._domainkey.partner.org
```

…joined to the `KeyTable`:

```
# /opt/hermes/dkim/KeyTable
mail1._domainkey.example.com  example.com:mail1:/opt/hermes/dkim/keys/mail1_example.com.dkim.private
k2024._domainkey.partner.org  partner.org:k2024:/opt/hermes/dkim/keys/k2024_partner.org.dkim.private
```

Both files are regenerated from the `dkim_sign` table on every
key add / enable / disable / delete on the per-domain page.

## The two cards on the page

### 1. DKIM Settings (master toggle + OpenDKIM runtime controls)

**DKIM Enabled** flips the child row in `parameters` whose
`parameter` matches `inet:%:8891` under the `smtpd_milters` parent
(and the same under `non_smtpd_milters`). Disabling DKIM here also
disables DMARC, mirroring the SPF-disable behavior — DMARC needs at
least one of the two to align against. The in-page callout warns
about this dependency.

When enabled, nine controls are written to `parameters2` rows in the
`dkim` module, then substituted into the OpenDKIM template at
`/opt/hermes/conf_files/opendkim.conf.HERMES`:

| Control | OpenDKIM directive | Effect |
| --- | --- | --- |
| Body Canonicalization | `Canonicalization` (body half) | `relaxed` (recommended) ignores trailing whitespace and end-of-line changes; `simple` requires byte-exact body. Most relays touch line endings, so `relaxed` is the only practical choice unless you fully control every downstream hop |
| Headers Canonicalization | `Canonicalization` (header half) | `relaxed` lowercases header names and folds whitespace; `simple` requires headers unchanged. Same reasoning — `relaxed` survives normal relay reformatting |
| Default Message Action | `On-Default` | Catch-all for verification outcomes not covered by the more specific actions below. `accept` is the recommended default |
| Bad Signature Action | `On-BadSignature` | Signature present, present-and-valid in syntax, but verification fails (body or signed-header bytes changed). `accept` (recommended) lets DMARC + spam scoring make the call |
| DNS Error Action | `On-DNSError` | The selector's `_domainkey` TXT record is unreachable or returned SERVFAIL. `accept` (recommended) — DNS instability is the sender's problem, not yours; do not block real mail on transient resolver failures |
| Internal Error Action | `On-InternalError` | OpenDKIM ran out of resources or hit an unexpected runtime error. `accept` (recommended) prevents silent mail loss when the verifier itself fails |
| No Signature Action | `On-NoSignature` | Message arrived unsigned. Many legitimate senders still don't sign — DMARC enforcement is the correct gate for "must be signed", not this knob. `accept` (recommended) |
| Security Concern Action | `On-Security` | Signature references a weak algorithm or unusually short key. `accept` (recommended) — score downstream rather than reject at the milter |
| Signature Algorithm | `SignatureAlgorithm` | `rsa-sha256` (current standard, recommended) or the deprecated `rsa-sha1`. Many receivers reject `rsa-sha1` outright; do not change unless you know why |

Each "Action" option set is: `accept`, `discard`, `reject`,
`tempfail`, `quarantine`. The save handler validates that submitted
values are members of this set before writing.

> **Operational consequence — accept everywhere is intentional.** The
> recommended baseline accepts on every error and every failure
> condition because **DKIM at the milter is not a delivery gate**.
> The verification result is meant to be consumed by DMARC and by
> spam scoring, not to drop mail. Setting any of these to `reject`
> means a single sender DNS hiccup or a single intermediate relay
> rewriting a header can cause real mail to bounce. Leave them at
> `accept` and let DMARC enforcement (which considers the
> sender-published policy) make the discard decision.

### 2. Whitelisted Domains and Trusted Hosts

Two row-per-entry lists that together drive three OpenDKIM directives:

| Entry type | OpenDKIM directive(s) | File on disk | Table |
| --- | --- | --- | --- |
| Whitelisted Domain | `ExemptDomains` | `/opt/hermes/dkim/ExemptDomains` | `dkim_bypass` (`entry`, `note`) |
| Trusted Host | `InternalHosts` + `ExternalIgnoreList` | `/opt/hermes/dkim/TrustedHosts` | `dkim_trusted_hosts` (`host`, `note`) |

**Whitelisted Domain** exempts the listed sender domain from inbound
DKIM verification entirely — OpenDKIM logs the bypass and does not
fetch the selector record. Use for known-broken signers whose mail
you still need to receive (some legacy mailing-list infrastructure,
specific government endpoints with unmaintained selectors).

**Trusted Host** is dual-purpose. The same entries are written to
both `InternalHosts` (mail from these hosts is considered locally
originated and will be DKIM-signed on the way out) and
`ExternalIgnoreList` (mail from these hosts skips inbound DKIM
verification). Accepts IP addresses, CIDR ranges, hostnames, and
bare domain names. The Docker subnet (`172.16.32.0/24` by default)
is pre-populated so the post-Amavis re-inject from `127.0.0.1` and
the inter-container hops are correctly treated as internal.

The DataTable supports add (textarea — one entry per line,
deduplicated), inline edit, single delete, and bulk delete; the row
checkboxes carry an `id|type` composite value so the bulk handler can
route each delete to the right table.

## What this page does NOT control

- **Per-domain DKIM key generation, selector choice, key size, key
  rotation, and the DNS TXT record to publish.** Those live on the
  Email Server [Domains](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) page via
  `edit_domain_dkim.cfm` — one selector / key per domain, stored in
  the `dkim_sign` table, written under
  `/opt/hermes/dkim/keys/_.dkim.{private,txt}`.
- **The KeyTable and SigningTable content.** These are regenerated
  from `dkim_sign` rows on every key change; do not edit them by
  hand.
- **ARC sealing.** The post-modification chain seal is a separate
  daemon — see [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings).
- **Outbound signing for sub-domains of a signed parent.** OpenDKIM's
  `*@` SigningTable match does not implicitly cover
  `*@sub.`. If you sign for `example.com` and need
  `mail.example.com` signed too, generate a separate key for it.

## Per-domain key rotation pattern

A working selector-rotation looks like this (operator-side, not a
single button on the page):

```
1. On edit_domain_dkim.cfm, generate a new key with a new selector
   (e.g. existing "mail1" -> new "mail2"). Mark NEW key disabled.
2. Publish the new key's TXT record at
   mail2._domainkey.example.com in authoritative DNS. The old
   mail1._domainkey.example.com record STAYS published.
3. Verify DNS propagation globally.
4. Enable the new key (disables the old one in dkim_sign atomically).
   KeyTable + SigningTable regenerate; OpenDKIM reloads.
5. Outbound mail now signs with mail2; mail signed with mail1 while
   in flight still verifies because the mail1 TXT record is still
   live.
6. Wait through the typical re-delivery window (24-72 hours).
7. Delete the old mail1 row in dkim_sign; remove the
   mail1._domainkey.example.com TXT record.
```

Selectors are arbitrary DNS labels — `mail1`, `2026q1`, `hermes`,
etc. — and there is no DKIM-defined upper bound on how many you
publish concurrently.

## Save flow

```
1. Validate form fields exist (when enabling DKIM)
   - Missing or out-of-set values -> session.m = 20, redirect, no DB write
2. cfinclude dkim_set_settings.cfm
     a. UPDATE parameters child rows for the smtpd_milters / non_smtpd_milters
        :8891 entries (on or off)
     b. UPDATE parameters2 rows for the nine OpenDKIM runtime directives
     c. cfinclude dkim_generate_config_file.cfm — read
        /opt/hermes/conf_files/opendkim.conf.HERMES, REReplace the
        Canonicalization / On-* / SignatureAlgorithm placeholders, write
        /etc/opendkim.conf
     d. cfinclude dkim_generate_hosts.cfm — regenerate
        /opt/hermes/dkim/TrustedHosts from dkim_trusted_hosts
     e. cfinclude dkim_generate_domains.cfm — regenerate
        /opt/hermes/dkim/ExemptDomains from dkim_bypass
     f. cfinclude dkim_generate_keytable.cfm + dkim_generate_signingtable.cfm
        — rebuild from dkim_sign
     g. cfinclude restart_opendkim.cfm — docker exec inside
        hermes_postfix_dkim to restart BOTH opendkim instances
3. cfinclude generate_postfix_configuration.cfm — regenerate main.cf
   (smtpd_milters list reflects DKIM on/off) and reload Postfix
4. If DKIM was DISABLED: also flip off OpenDMARC milter rows, clear
   FailureReports, deactivate the DMARC report Ofelia job, regenerate
   opendmarc.conf, restart OpenDMARC
5. session.m = 9 -> green "DKIM settings saved" alert on redirect
```

Add / Edit / Delete on the second card calls
`dkim_generate_hosts.cfm` or `dkim_generate_domains.cfm` (whichever
applies) plus `restart_opendkim.cfm` inline — Postfix is not
reloaded since the milter chain itself did not change.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_dkim_settings.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/get_dkim_settings.cfm` | `hermes_commandbox` | Loads current parameters / parameters2 / bypass / trusted-host values |
| `config/hermes/var/www/html/admin/2/inc/dkim_save_settings.cfm` | `hermes_commandbox` | Validates form, calls set + generate + restart chain; disables DMARC if DKIM off |
| `config/hermes/var/www/html/admin/2/inc/dkim_set_settings.cfm` | `hermes_commandbox` | UPDATEs the parameters / parameters2 rows, regenerates all four config files, restarts OpenDKIM |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_config_file.cfm` | `hermes_commandbox` | Renders `/etc/opendkim.conf` from the template + DB |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_keytable.cfm` | `hermes_commandbox` | Rebuilds `/opt/hermes/dkim/KeyTable` from `dkim_sign` |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_signingtable.cfm` | `hermes_commandbox` | Rebuilds `/opt/hermes/dkim/SigningTable` from `dkim_sign` |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_hosts.cfm` | `hermes_commandbox` | Rebuilds `/opt/hermes/dkim/TrustedHosts` from `dkim_trusted_hosts` |
| `config/hermes/var/www/html/admin/2/inc/dkim_generate_domains.cfm` | `hermes_commandbox` | Rebuilds `/opt/hermes/dkim/ExemptDomains` from `dkim_bypass` |
| `config/hermes/opt/hermes/conf_files/opendkim.conf.HERMES` | `hermes_commandbox` (read) → `hermes_postfix_dkim` (live `/etc/opendkim.conf`) | Template with `HEADER-CANONICALIZATION`, `BODY-CANONICALIZATION`, `DEFAULT-ACTION`, etc. placeholders |
| `config/postfix-dkim/etc/opendkim-sign.conf` | `hermes_postfix_dkim` | Static config for the sign-only instance at `:8892` (no placeholders — relaxed/relaxed + rsa-sha256 are fixed for the re-injection signer) |
| `parameters` table (`inet:%:8891` rows under `smtpd_milters` and `non_smtpd_milters`) | `hermes_db_server` (`hermes` DB) | DKIM milter on/off |
| `parameters2` table (rows where `module='dkim'`) | `hermes_db_server` (`hermes` DB) | The nine OpenDKIM runtime settings |
| `dkim_sign`, `dkim_bypass`, `dkim_trusted_hosts` tables | `hermes_db_server` (`hermes` DB) | Per-domain keys, exempt-domain list, trusted-host list |
| `hermes_postfix_dkim` container | — | Runs both OpenDKIM instances and hosts the live config + key files |
| `hermes_unbound` container | — | Resolves every `._domainkey.` lookup |

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Missing form fields when enabling DKIM | `session.m = 20`, redirect, no DB write |
| Out-of-set value submitted for an Action / Canonicalization / Algorithm field | `session.m = 20`, redirect, no DB write |
| Empty entry on Add | `session.m = 13`, redirect, no DB write |
| Invalid syntax on Add / Edit | `session.m = 17`, redirect, no DB write |
| Duplicate entry on Add | `session.m = 14`, redirect, no DB write |
| `dkim_generate_config_file.cfm` write fails | Surfaces as `cfcatch` from the inline include — save aborts |
| `restart_opendkim.cfm` fails | Same path — Postfix is reloaded anyway in step 3, but DKIM service is left in the prior runtime state |
| `KeyTable` / `SigningTable` missing because no `dkim_sign` rows exist yet | OpenDKIM starts but signs nothing — outbound mail goes out unsigned |

## Related

- [SPF Settings](https://docs.deeztek.com/books/administrator-guide/page/spf-settings) — the second authentication
  service whose result is consumed by DMARC; paired conceptually
  with DKIM as a "DNS-based outbound sender authentication"
  mechanism. SPF checks at envelope `MAIL FROM` time; DKIM checks
  header signatures after `DATA`. DKIM survives forwarding;
  SPF generally doesn't
- [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings) — the policy layer that
  consumes DKIM (and SPF) results; disabling DKIM here automatically
  disables DMARC
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — the post-modification chain
  seal, which runs after the sign-only OpenDKIM at `:8892` so the
  ARC record covers the final outbound body
- [Trusted ARC Sealers (M365)](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) — for
  M365 customers whose downstream verifiers escalate when a
  Hermes-forwarded message's original DKIM signature breaks against
  the body-modified bytes
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — the SPF / DKIM / DMARC
  status card on Perimeter Checks links here for the per-service
  toggle
- [Domains (Email Server)](https://docs.deeztek.com/books/administrator-guide/page/domains-i8v) — where
  per-domain DKIM keys are generated, selectors chosen, and DNS TXT
  records exposed for publication
- [Domains (Email Relay)](https://docs.deeztek.com/books/administrator-guide/page/domains) — relay-mode
  domains can also sign outbound; same per-domain key UX
- [Email Policies > Disclaimers](https://docs.deeztek.com/books/administrator-guide/page/disclaimers) —
  documents the body milter that modifies outbound bodies before the
  sign-only OpenDKIM at `:8892` produces the final signature; the
  two-instance OpenDKIM design exists precisely because of this body
  modification
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — every
  `._domainkey.` lookup flows through
  `hermes_unbound`; resolver mode directly affects DKIM verification
  reliability
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — TLS
  on outbound delivery is independent of DKIM, but receivers that
  enforce strict transport security may surface DKIM failures more
  prominently in failure reports

# DMARC Settings

# DMARC Settings

Admin path: **Content Checks > DMARC Settings**
(`view_dmarc_settings.cfm`, `inc/get_dmarc_settings.cfm`,
`inc/dmarc_save_settings.cfm`, `inc/dmarc_set_settings.cfm`,
`inc/dmarc_generate_config_file.cfm`,
`inc/dmarc_generate_reports_script.cfm`,
`inc/restart_opendmarc.cfm`).

This page controls Hermes's OpenDMARC milter — both whether DMARC is
evaluated on inbound mail and, when enabled, what happens to
verdicts and whether daily aggregate reports are generated for the
domains that publish a DMARC record. DMARC ([RFC 7489](https://datatracker.ietf.org/doc/html/rfc7489))
is the policy layer that sits on top of SPF and DKIM; a sender
publishes a `_dmarc.` TXT record telling receivers what to do
when neither SPF nor DKIM aligns with the From: header domain. Hermes
is the receiver that does the work.

## How DMARC fits the auth stack

```
                  +--------------------+
   inbound msg -->| SPF check          |  passes/fails on envelope-from IP
                  +---------+----------+
                            |
                            v
                  +--------------------+
                  | DKIM verify        |  passes/fails on each signature
                  +---------+----------+
                            |
                            v
                  +--------------------+
                  | OpenDMARC          |  reads SPF + DKIM AR headers,
                  |   :54321 milter    |  fetches _dmarc.
                  +---------+----------+  evaluates alignment + policy
                            |
                            v
                  +--------------------+
                  | RejectFailures?    |
                  | -> reject / accept |
                  +--------------------+
```

A message **aligns** when its From: header domain matches the
SPF-pass envelope-from domain OR the DKIM-pass `d=` domain.
Relaxed alignment (the default) accepts org-domain match
(`example.com` aligns with `mail.example.com`); strict alignment
requires exact match. OpenDMARC reads the alignment results that
SPF and DKIM have already written into the `Authentication-Results`
header — both checks must therefore be active before DMARC is useful.
The UI enforces this: enabling DMARC with SPF or DKIM disabled
returns error 1.

## Container and milter placement

| Component | Detail |
| --- | --- |
| Container | `hermes_dmarc` (separate service, IPv4 `.111`) |
| Listen | `inet:54321@[0.0.0.0]` (`Socket` directive in `opendmarc.conf`) |
| Source | OpenDMARC daemon (Trusted Domain Project), packaged in the `hermes-dmarc` image |
| Milter chain | Postfix `smtpd_milters` AND `non_smtpd_milters` parents, child row `inet::54321` — toggle flips `enabled` on that row |
| DMARC report DB | `opendmarc` database on `hermes_db_server`, credentials in `system_settings` rows `mysql_username_opendmarc` / `mysql_password_opendmarc` |
| History file | `/etc/opendmarc/opendmarc.dat` inside `hermes_dmarc` (volume-mounted from `./config/opendmarc/etc/opendmarc/`) |

The container exposes **no host ports** — Postfix reaches OpenDMARC
internally at `inet:hermes_dmarc:54321`. The whitelist file path
referenced by `DomainWhitelistFile` resolves to
`/etc/opendmarc/whitelist.domains`, written by
`inc/dmarc_generate_domains.cfm` from the `dmarc_domains` table on
every save.

## DMARC Settings card

Six controls drive `opendmarc.conf` directly via placeholder
substitution into `/opt/hermes/conf_files/opendmarc.conf.HERMES`.

| UI Control | `opendmarc.conf` directive | What it does |
| --- | --- | --- |
| DMARC Enabled (YES/NO) | Milter chain toggle | Enables the `inet:%:54321` child row under `smtpd_milters` and `non_smtpd_milters`; OpenDMARC stops being consulted entirely when disabled |
| Reject Failures | `RejectFailures` (true/false) | When true, messages failing DMARC evaluation are rejected (or temp-failed if evaluation could not complete). When false, the message is accepted and only an `Authentication-Results` header records the verdict |
| Hold Quarantine Policy Messages | `HoldQuarantinedMessages` (true/false) | When true, messages from domains publishing `p=quarantine` that fail DMARC are routed to the Postfix `hold` queue for manual release/delete. When false (recommended), quarantine-policy messages are delivered with an `Authentication-Results` annotation and downstream scoring handles them |
| Generate Daily Failure Reports | `FailureReports` (true/false) | When true, OpenDMARC writes failure records to the history file and the daily Ofelia job converts them to RFC 6591 aggregate reports |
| Failure Reports From E-mail | `--report-email` flag on `opendmarc-reports` | RFC 6591 envelope From: for the outgoing report — must be a valid email address (validated by `IsValid("email", ...)`) |
| Failure Reports Reporting Organization | `--report-org` flag | Identifies your gateway as the report source — alphanumeric only (validation regex: `[^A-Za-z0-9]`) |

OpenDMARC's `FailureReports` triggers reports only for domains that
publish `p=quarantine` or `p=reject` (it never auto-reports for
`p=none` unless `FailureReportsOnNone` is also set — Hermes does not
expose that directive).

The "Reject Failures" UI hint and the OpenDMARC docs use the same
language: messages that fail are **rejected** when policy is `reject`,
**delivered with header** when policy is `none`, and **either held or
flagged** when policy is `quarantine` (depending on
`HoldQuarantinedMessages`).

> **Operational consequence — RejectFailures = true.** When this is on,
> OpenDMARC will respond `550 5.7.0` to messages from domains
> publishing `p=reject` that fail evaluation, and Postfix will refuse
> the message in-band. This catches forged messages but also catches
> legitimate forwarded mail from senders whose original SPF / DKIM
> chain breaks at an upstream forwarder. If you start seeing
> legitimate forward-from-mailing-list mail bounce, the fix is to add
> the originating domain to the Whitelisted Domains card below — not
> to disable Reject Failures globally.

## Whitelisted Domains card

Rows from the `dmarc_domains` table (`id`, `domain`, `note`,
`type`) write to `/etc/opendmarc/whitelist.domains`. OpenDMARC reads
that file via `DomainWhitelistFile` and bypasses DMARC evaluation
entirely for any matching From: domain — no alignment check, no
policy enforcement, no failure report. Use for trusted senders with
known broken DMARC, partner domains that forward through aggregators
that strip headers, or legacy mailing lists.

Only domain names are accepted; IP addresses are rejected by the add
handler. Domains are validated by the same regex used elsewhere in
Hermes (e.g. error 17: "The entry is not a valid domain"). Bulk
add is supported one-per-line in the textarea.

## DMARC report generation (daily aggregate / RUA)

When **Generate Daily Failure Reports** is enabled, `dmarc_set_settings.cfm`
calls `dmarc_generate_reports_script.cfm` which renders
`/opt/hermes/scripts/dmarc_report_script.sh` with credentials and
identifiers substituted into placeholders (`DATABASE-SERVER`,
`DATABASE-USER`, `DATABASE-PASSWORD`, `REPORTING-EMAIL`,
`REPORTING-ORGANIZATION`, `POSTMASTER-EMAIL`) and writes the result
to `/opt/hermes/schedule/dmarc_report_script.sh` (chmod +x).

An [Ofelia](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) job named
`hermes-dmarc-report` runs the script daily at 02:30:

```
[job-exec "hermes-dmarc-report"]
schedule:  0 30 02 * * *
container: hermes_dmarc
command:   /opt/hermes/schedule/dmarc_report_script.sh
```

The script does three things in sequence:

1. **`opendmarc-import`** — drains `/etc/opendmarc/opendmarc.dat`
   (the per-message verdict log OpenDMARC writes) into the
   `opendmarc` MariaDB database
2. **`opendmarc-reports`** — generates RFC 6591 aggregate XML reports
   for the prior 24h interval and emails one report per sender
   domain to the `rua=` address that domain published in DNS
3. **`opendmarc-expire`** — drops records older than the retention
   window from the database

The script also emits a Net::SMTP success/failure notification to the
`postmaster` address (from `system_settings`). The Perl one-liner
passes the postmaster address through an environment variable rather
than direct string interpolation — Perl's default array sigil `@`
treats `@deeztek.net` as an array dereference and silently loses the
domain part. Passing via `$ENV{POSTMASTER_ARG}` avoids the trap (the
fix landed as issue #215). The notification is also skipped entirely
when `postmaster` is not a valid email address (e.g. bare local-part
like `postmaster`) — this prevents queue pollution with
undeliverable bounces.

SMTP delivery uses `hermes_postfix_dkim:10026` (the post-amavis
re-injection port) — using `:25` would re-process the report through
the inbound pipeline and could re-trigger DMARC evaluation on the
report itself.

When **Generate Daily Failure Reports** is disabled (or DMARC itself
is disabled), the save handler:

- Deletes `/opt/hermes/schedule/dmarc_report_script.sh`
- Sets `ofelia_jobs.active = '2'` on the `hermes-dmarc-report` job
  and regenerates `/etc/ofelia/config.ini` via
  `ofelia_generate_config.cfm`

## Forensic (RUF) reports

Forensic (per-failure) reports are intentionally **not** generated by
Hermes. They are privacy-noisy (they include redacted copies of
failing messages), receivers rarely publish a `ruf=` address, and the
modern operational consensus is that aggregate (RUA) reports give
operators the visibility they need without the per-message exhaust.
The `FailureReportsBcc` / `FailureReportsSentBy` / `CopyFailuresTo`
directives in `opendmarc.conf.HERMES` are left commented and not
exposed in the UI.

## ARC interaction

Hermes also runs an [ARC](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) sealer
(`hermes_openarc`) on the same authentication stack. When Hermes
modifies a message body (External Sender Banner, disclaimer
injection, signature injection, S/MIME or PGP rewrap), the
original sender's DKIM body hash no longer matches the current
body — DMARC alignment is lost on the modified copy. ARC preserves
the pre-modification verdict in a sealed chain so downstream
receivers configured to trust Hermes can still rescue DMARC
alignment. See [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) and the
[Trusted ARC Sealers — M365 guide](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) for
the receiver-side configuration. Hermes is the authoritative auth
boundary for every domain it serves; customer downstream MX
allowlisting is the standard remedy when ARC trust is not in play.

## Save flow

```
1. View page submits action=save_settings or add_domain / edit_domain / delete_domain
2. dmarc_save_settings.cfm validates:
     - SPF + DKIM both enabled (error 1 if not)
     - rejectfailures / holdquarantinedmessages / failurereports are true|false (error 20)
     - if failurereports=true: report_email present + valid (errors 2, 3)
                               report_org present + alphanumeric (errors 4, 5)
3. dmarc_set_settings.cfm UPDATEs:
     - parameters.enabled on the inet:%:54321 child row (smtpd + non_smtpd)
     - parameters2.value2 on FailureReports / RejectFailures / HoldQuarantinedMessages
       (module = 'dmarc')
     - parameters2.value2 on report_email / report_org (when reports enabled)
4. dmarc_generate_config_file.cfm:
     - Copies opendmarc.conf.HERMES to /opt/hermes/tmp/_opendmarc.conf
     - Substitutes FAILURE-REPORTS, REJECT-FAILURES, HOLD-QUARANTINE-MESSAGES placeholders
     - Backs up /etc/opendmarc/opendmarc.conf -> opendmarc.HERMES
     - Moves the rendered file into place
5. dmarc_generate_reports_script.cfm (if reports enabled):
     - Renders dmarc_report_script.sh, chmod +x
     - Enables ofelia_jobs row for hermes-dmarc-report, regenerates Ofelia config
   (else: deletes the script, disables the Ofelia row)
6. restart_opendmarc.cfm: docker container restart hermes_dmarc
7. generate_postfix_configuration.cfm: postconf -e the milter list, postfix reload
8. session.m = 9 -> green "DMARC settings saved successfully. Postfix reloaded." alert
```

## Failure semantics

| Failure | Behavior |
| --- | --- |
| SPF or DKIM not enabled when DMARC=YES | session.m = 1, redirect, no DB write |
| `report_email` empty | session.m = 2 |
| `report_email` invalid | session.m = 3 |
| `report_org` empty | session.m = 4 |
| `report_org` contains non-alphanumeric | session.m = 5 |
| Missing required form fields | session.m = 20 |
| Delete Domains clicked with nothing selected | session.m = 11 |
| Add Domain with empty Domain field | session.m = 13 |
| Add Domain with invalid format | session.m = 17 |
| Add Domain with duplicate | session.m = 14 (single) or `_exists` alert (bulk) |

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_dmarc_settings.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/dmarc_*.cfm` | `hermes_commandbox` | Validate / save / generate / restart |
| `config/hermes/opt/hermes/conf_files/opendmarc.conf.HERMES` | `hermes_commandbox` (read) -> `hermes_dmarc` (live `/etc/opendmarc/opendmarc.conf`) | Canonical template |
| `config/hermes/opt/hermes/scripts/dmarc_report_script.sh` | `hermes_commandbox` (read) -> rendered into `/opt/hermes/schedule/` (executed in `hermes_dmarc`) | Daily aggregate report script |
| `/etc/opendmarc/whitelist.domains` | `hermes_dmarc` | Generated from `dmarc_domains` table on every save |
| `/etc/opendmarc/opendmarc.dat` | `hermes_dmarc` | Per-message verdict history; drained nightly by `opendmarc-import` |
| `opendmarc` MariaDB DB | `hermes_db_server` | Holds imported verdicts that `opendmarc-reports` reads |
| `parameters` / `parameters2` tables (`module='dmarc'`) | `hermes_db_server` (`hermes` DB) | Source of truth for every directive |
| `system_settings` rows `mysql_username_opendmarc` / `mysql_password_opendmarc` | `hermes_db_server` | DB creds for the report script (managed via `update_opendmarc_db_creds.cfm`) |
| `ofelia_jobs` row `hermes-dmarc-report` | `hermes_db_server` | Daily report scheduler entry |

## Related

- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — the SMTP-time card whose
  Email Authentication badge shows DMARC's wired-up status and the
  "Requires both SPF and DKIM" callout
- [SPF Settings](https://docs.deeztek.com/books/administrator-guide/page/spf-settings) — the alignment input for the
  envelope From: side
- [DKIM Settings](https://docs.deeztek.com/books/administrator-guide/page/dkim-settings) — the alignment input for the
  signature `d=` side
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — preserves the DMARC verdict
  across body-modifying forwarding hops
- [Trusted ARC Sealers — M365](https://docs.deeztek.com/books/administrator-guide/page/trusted-arc-sealers-microsoft-365) —
  receiver-side configuration to trust Hermes's ARC seal
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — runs after DMARC and
  can promote a DMARC-fail message to higher spam score
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — per-rule weight changes
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) — every `_dmarc` TXT
  lookup goes through `hermes_unbound`; resolver mode (recursive vs.
  forwarding) directly affects DMARC accuracy and report timing
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) — full pipeline diagram
  with milter placement

# File Expressions

# File Expressions

Admin path: **Content Checks > File Expressions**
(`view_file_expressions.cfm`,
`inc/get_file_expressions.cfm`,
`inc/update_amavis_config_files.cfm`).

This page maintains the catalogue of **regex patterns** that Amavis
can match against attachment filenames. Where
[File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) is a one-extension-per-row list
(`.exe`, `.docm`, `.iso`), File Expressions is the free-form regex
sibling — any Perl-compatible pattern that should fire on the
attachment name: double-extension traps (`^.+\.(exe|scr)\.[a-z0-9]+$`),
disguised-archive patterns (`^invoice.*\.pdf\.zip$`), or any
project-specific filename signature an extension list can't express.
The page itself does not block anything — it only registers patterns.
The block / allow decision is taken by a [File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules)
that bundles expressions (and extensions, file types, MIME types)
into a named ruleset, which is then bound to recipient traffic via
an SVF policy on [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings).

The expression catalogue is **entirely operator-driven** — Hermes
ships no system-managed expressions. The shipped High-Risk catch-all
("Double Extensions in File Name") and the Windows Class ID block
live on the [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) page as
`type = 'FILE-HIGH'` rows. Everything on the File Expressions page
is something the operator added.

## Where File Expressions sits

```
                       +---------------------------------------+
   File Expressions    |  files table                          |
   (this page)  -----> |   id, file ("\.exe$"),                |
                       |   description ("Executable files"),   |
                       |   type ("CUSTOM-EXPRESSION"),         |
                       |   system ("NO"),                      |
                       |   allow ("[qr'\.exe$'i => 0]"),       |
                       |   ban   ("[qr'\.exe$'i => 1]")        |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  File Rules                           |
                       |   bundle expressions + extensions     |
                       |   into named rulesets with per-item   |
                       |   allow / ban / priority              |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  Anti-Spam Settings (SVF Policies)    |
                       |   bind a File Rule to recipient(s)    |
                       |   via policy.banned_rulenames         |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  Amavis 50-user.HERMES                |
                       |   @banned_filename_re emitted per     |
                       |   rule on every save chain            |
                       +---------------------------------------+
```

The rendered `@banned_filename_re` block is enforced at
content-filter time inside `hermes_mail_filter`. A matched expression
triggers Amavis's `final_banned_destiny` action (`D_BOUNCE`,
`D_DISCARD`, or `D_PASS` — set globally on
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings)).

## How the pattern is wrapped

The textarea takes a raw Perl regex. On save the handler wraps it
into Amavis's `qr//` syntax with the `i` (case-insensitive) modifier
and stores both the allow and ban form on the row:

```
[qr'\.exe$'i => 0]      (allow form, stored in files.allow)
[qr'\.exe$'i => 1]      (ban form,  stored in files.ban)
```

Whether the allow or ban form gets rendered into Amavis's
`@banned_filename_re` is decided at File Rule time, not here. The
File Expressions page does not have an allow/ban toggle — both forms
are stored so the same expression can serve allow-rules and
ban-rules without re-typing.

There is no case-sensitive variant on this page. Every File
Expression is stored with the `i` modifier. Operators who need
strict case have to drop down to the File Rule's per-component
selection or use a regex character class on the pattern itself
(`\.[Ee][Xx][Ee]$`).

## The page

A page guide callout, an Expression Helper card (build / pick /
test), an Add Expressions card with a bulk textarea, and a single
DataTable listing every custom expression. The DataTable is
flat — system vs. custom does not apply because the catalogue is
all-custom by design.

### Expression Helper card

A three-section utility, collapsed by default, that exists so
operators don't need to know regex to add common patterns.

| Section | Purpose |
| --- | --- |
| Build an Expression | Pick a match mode (Ends with / Starts with / Contains / Exact), enter plain text, click **Build**. The helper regex-escapes the input, wraps it with the appropriate anchors (`^…`, `…$`, `^…$`), and shows the generated pattern with a plain-English explanation |
| Quick Select Common Patterns | A dropdown of pre-built patterns (`\.exe$`, `\.bat$`, `^invoice`, `\.(exe\|bat\|cmd\|scr\|pif)$`, etc.) — click **Use** to drop the pattern into the Add form |
| Test a Pattern | A pattern + filename pair with a **Test** button — runs `new RegExp(pattern, 'i').test(filename)` in the browser and reports Match / No match / Invalid regex. Lets the operator sanity-check before saving |

The Build helper escapes `. * + ? ^ $ { } ( ) | [ ] \` in the user
input before wrapping, so a builder entry of `invoice.pdf` becomes
`invoice\.pdf$`, not `invoice.pdf$`.

### Add File Expressions card

| Field | Stored as | Notes |
| --- | --- | --- |
| File Expressions | `files.file` (the regex) + `files.description` | One per line; format is `regex_pattern description` where the first space separates pattern from label. A pattern with no space becomes its own description (useful for self-documenting patterns like `\.docm$`) |

The handler line-splits the textarea on LF or CRLF, strips
whitespace, and inserts each non-blank entry. Per entry it
checks one thing: that no row already exists in `files` with the
same `file` value under `type = 'CUSTOM-EXPRESSION'`. Duplicates
are skipped and surfaced in the partial-success alert
("Duplicate: \\.exe\$"); the rest still insert.

There is **no regex-validity check on save** — the regex is stored
as-typed and any syntax error is exposed at Amavis reload time, not
in the alert. Use the Test a Pattern section of the helper before
saving to catch malformed patterns first.

### File Expressions DataTable

| Column | Source |
| --- | --- |
| (checkbox) | Selection for bulk Delete Selected |
| Regex Pattern | `files.file` (rendered inside a `` block) |
| Description | `files.description` |
| Actions | Per-row Delete button (single-row confirm) |

The DataTable shows only `type = 'CUSTOM-EXPRESSION'` rows. No
edit-in-place — to change a pattern the operator deletes it and
re-adds.

## Foreign-key guard on delete

A custom expression cannot be deleted while it is referenced by any
[File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules). The single-row Delete handler runs:

```sql
SELECT COUNT(*) AS cnt FROM file_rule_components
WHERE file_id = :id
```

If `cnt > 0`, the delete is refused with alert `m = 40` and the
DataTable shows the offending rule name(s) ("This expression is
referenced by the following File Rule(s): **Block-Disguised-Exe**").
The operator's path is to open File Rules, remove the expression
from the rule, then come back here and delete it.

Bulk Delete applies the same guard per-id and accumulates partial
results — alert `m = 41` reports "N deleted, M blocked" with the
blocked rows' pattern and rule names attached, so the operator knows
exactly what to unwire first.

## Save and apply flow

```
1. View page submits action="add_entries" | "delete" | "bulk_delete"
2. For each valid entry:
     a. Generate ban  string: "[qr''i => 1]"
     b. Generate allow string: "[qr''i => 0]"
     c. INSERT INTO files (file, description, type, system, allow, ban)
        with type='CUSTOM-EXPRESSION' and system='NO'
3. If at least one row was added or deleted:
     a. update_amavis_config_files.cfm:
          - Read /opt/hermes/conf_files/50-user.HERMES (template)
          - Substitute the SERVER/destiny/DKIM/MySQL-credential
            placeholders from spam_settings and creds files
          - Render every File Rule's components into an
            @banned_filename_re block (per-rule, in priority order,
            using the allow/ban regex stored on each files row -
            including the CUSTOM-EXPRESSION rows this page creates)
          - Back up /etc/amavis/conf.d/50-user -> 50-user.HERMES,
            move rendered file into place
     b. docker exec hermes_mail_filter /etc/init.d/amavis force-reload
        (30-second timeout)
4. session.m = 1 (add) | 2 (single/bulk delete) | 30 (empty submit)
   | 40 (FK refused) | 41 (bulk partial)
```

Amavis is reloaded with `force-reload` rather than restarted — the
daemon re-reads `50-user` without dropping connections, and mail in
flight is not interrupted. The reload step is wrapped in
`cftry`/`cfcatch` and the catch block is intentionally silent: if
the reload itself fails the DB rows are already in place, and the
next save (or a manual `force-reload`) will re-render. The page
does not roll back on reload failure.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` | Add Expressions completed (with `entries_added` / `entries_skipped` / `entry_errors` set on session for the per-row breakdown) |
| `m = 2` | Single Delete succeeded; Amavis reloaded |
| `m = 30` | Add submitted with an empty textarea |
| `m = 31` | Pattern field empty (legacy edit path, no longer reachable from the current UI) |
| `m = 32` | Duplicate pattern (legacy edit path) |
| `m = 40` | Single Delete refused — the expression is wired into at least one File Rule (rule names surfaced in the alert) |
| `m = 41` | Bulk Delete partial — `deleted_count` rows removed, `blocked_count` rows refused (the per-row pattern + rule-name list is HTML-rendered into the alert body) |

The per-row error list is HTML-rendered into alert `m = 1` so the
operator sees every duplicate at once. No row is silently dropped
without an explanation.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_file_expressions.cfm` | `hermes_commandbox` | The page (add + delete + bulk delete + Expression Helper + Amavis reload) |
| `config/hermes/var/www/html/admin/2/inc/get_file_expressions.cfm` | `hermes_commandbox` | Loads `type = 'CUSTOM-EXPRESSION'` rows into the DataTable |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Renders `50-user` from template + File Rules (called on every change here too — expression edits affect rendered `@banned_filename_re` blocks) |
| `config/hermes/opt/hermes/conf_files/50-user.HERMES` | `hermes_commandbox` (read) -> `hermes_mail_filter` (live `/etc/amavis/conf.d/50-user`) | Canonical Amavis template; receives the rendered `@banned_filename_re` blocks |
| `/etc/amavis/conf.d/50-user` | `hermes_mail_filter` | Live Amavis config; reloaded with `force-reload` on every save |
| `files` table, `type = 'CUSTOM-EXPRESSION'` | `hermes_db_server` (`hermes` DB) | Source of truth for the expression catalogue |
| `file_rule_components` table | `hermes_db_server` (`hermes` DB) | Cross-reference checked by the delete guard |
| `hermes_mail_filter` container | — | Hosts Amavis; receives `force-reload` (not restart) on every change |

## Operational consequences

- **No regex validation at save.** A malformed regex inserts cleanly
  and only surfaces at Amavis reload time. The reload itself does
  not roll back the DB. If reload starts failing immediately after
  an Add, the most recent expression is the suspect — open it,
  paste it into the Test a Pattern helper, and look for unescaped
  metacharacters or unbalanced groups. The pattern with `\.exe$`
  works; a typo of `\.exe$.` (trailing dot) parses but matches
  nothing.
- **Case is always insensitive.** Every expression renders with the
  `i` modifier. There is no per-expression case toggle. Operators
  who need strict case have to encode it in the pattern itself.
- **Order does not matter on this page.** Expressions are stored
  flat. The evaluation order that Amavis sees is decided by the
  File Rule that bundles them — each component's `priority` column
  on `file_rule_components`. Changing the description here will
  not reorder anything.
- **Custom-Expression rows are visible to File Rules under
  "Custom Expressions".** When the operator opens the Add/Edit
  modal on [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules), every row this page creates
  shows up in the **Custom Expressions** card alongside the system
  catalogue. That is the only place the bundling happens.

## Related

- [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) — sibling page for plain
  extension entries (`.exe`, `.docm`); the simpler half of the same
  `files` table, distinguished by `type IN ('EXT', 'EXT-HIGH')`
- [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules) — bundles extensions and expressions
  into named, prioritised rulesets; the consumer of every row this
  page creates
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) — content-level SpamAssassin
  rules (header / body / regex) — the body / header equivalent of
  what File Expressions does for attachment names
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — defines
  `final_banned_destiny` (what Amavis does with a banned-expression
  match) and binds File Rules to recipients via SVF Policies
- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — ClamAV runs in the
  same Amavis pass; a virus verdict on the same attachment
  overrides the banned-expression result
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — sibling Amavis tuning
  page; both write into Amavis configuration but expression matches
  are categorical (matched -> banned) where SA rules are weighted
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — note that banned-expression
  rejections are a body-side filter result, not an authentication
  result — they fire after ARC chain evaluation
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) — a banned-expression
  rejection appears with Type `Banned` and the matched expression
  surfaced in the detail view
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — Amavis logs the
  matched regex as `Blocked BANNED (\.exe$,…)` on the
  `amavis[...]:` line

# File Extensions

# File Extensions

Admin path: **Content Checks > File Extensions**
(`view_file_extensions.cfm`,
`inc/get_file_extensions.cfm`,
`inc/update_amavis_config_files.cfm`).

This page maintains the catalogue of **attachment file extensions**
that Amavis can match on. Each entry is a single extension such as
`.exe`, `.docm`, or `.iso` paired with a description and a sensitivity
flag (Standard vs. High Risk). The page itself does not block anything
— it only registers extension candidates. The block / allow decision
is taken by a [File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules) that bundles extensions into
a named ruleset, which is then applied to recipients via an SVF
policy on [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings). File Extensions
is the building-block page; File Rules and SVF Policies are where the
ruleset is composed and bound to traffic.

The extension catalogue ships with a system-managed list of common
high-risk types (`.exe`, `.scr`, `.pif`, `.com`, `.bat`, `.vbs`,
`.js`, `.jar`, `.ps1`, and dozens more) that cannot be deleted from
the UI. Operators add custom extensions on top — typically Office
macro-enabled types in environments that don't allow macros, archive
formats they want to surface separately, or new attack-surface file
types as they appear in the wild.

## Where File Extensions sits

```
                       +---------------------------------------+
   File Extensions     |  files table                          |
   (this page)  -----> |   id, file ("exe"), description,      |
                       |   type ("EXT" | "EXT-HIGH"),          |
                       |   system ("YES"/"NO"),                |
                       |   allow ("[qr'.\.(exe)$'i => 0]"),    |
                       |   ban   ("[qr'.\.(exe)$'i => 1]")     |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  File Rules                           |
                       |   bundle extensions into named        |
                       |   rulesets with per-extension         |
                       |   allow / ban / priority              |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  Anti-Spam Settings (SVF Policies)    |
                       |   bind a File Rule to recipient(s)    |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  Amavis 50-user.HERMES                |
                       |   @banned_filename_re emitted per     |
                       |   rule on every save chain            |
                       +---------------------------------------+
```

Amavis enforces the resulting `@banned_filename_re` regex sets at
content-filter time inside `hermes_mail_filter`. A matched extension
triggers Amavis's `final_banned_destiny` action (`D_BOUNCE`,
`D_DISCARD`, or `D_PASS` — set globally on
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings)).

## What "matched" means in Amavis

The stored allow / ban snippets are case-insensitive regexes anchored
to the end of the filename:

```
[qr'.\.(exe)$'i => 1]      (ban; case-insensitive)
[qr'.\.(exe)$'x  => 0]     (allow; case-sensitive)
```

This means:

- `invoice.exe` matches `.exe`
- `Invoice.EXE` matches `.exe` (because the `i` modifier is set by
  default on Add)
- `invoice.pdf.exe` matches `.exe` (the *trailing* extension is the
  one Amavis tests)
- `invoice.exe.pdf` does **not** match `.exe` — it matches `.pdf`,
  and the trailing-extension rule is the only one that fires

The double-extension confusion case (`invoice.pdf.exe`) is the
historic reason this list exists. Amavis sees the real trailing
extension; the user sees only the displayed-name prefix and a
familiar icon.

## The page

A page guide callout, an Add Extensions card with a bulk textarea,
a Custom File Extensions DataTable (editable / deletable), and a
Read-Only System File Extensions DataTable (the shipped list).

### Add File Extensions card

| Field | Stored as | Notes |
| --- | --- | --- |
| File Extensions | `files.file` + `files.description` | One per line; format `.ext description`. The leading dot is stripped on save (so the row stores `exe`, not `.exe`); the description is auto-prefixed with `(.ext)` so the DataTable shows `(.docm) Microsoft Word Macro-Enabled Document` regardless of how the operator typed it |
| Extension Type | `files.type` | `EXT` (Standard) or `EXT-HIGH` (High Risk). Purely a classification tag for the UI badges — Amavis treats both the same |
| Case Sensitivity | drives which template is rendered into `files.allow` / `files.ban` | Insensitive (default, recommended) uses `_insense` templates with the `i` regex modifier; sensitive uses `_sense` templates with `x` only — for environments where you want `.EXE` to differ from `.exe` |

The handler line-splits the textarea on either LF or CRLF, strips
whitespace, validates each entry, and inserts the valid ones. Per
entry it checks:

- The extension starts with `.`
- The extension matches `^[.][a-zA-Z0-9\-\.\_]+$` (alphanumeric, dash,
  period, underscore — nothing else)
- The description is non-blank (required)
- No row with the same `file` already exists in the `EXT` /
  `EXT-HIGH` type space (a `.docm` cannot exist as both Standard
  and High Risk)

Each rejected line is collected into a per-row error list that
surfaces in the partial-success alert; the valid entries still
insert. The `(.ext)` prefix on the description is auto-prepended so
the catalogue stays self-describing regardless of how the operator
typed the row.

### Custom File Extensions DataTable

| Column | Source |
| --- | --- |
| (checkbox) | Selection for bulk Delete Selected |
| Extension | `.` (the leading dot is displayed in the UI even though it isn't stored) |
| Description | `files.description` |
| Actions | Per-row Delete button (single-row confirm) |

The DataTable shows only rows with `system = 'NO'` and excludes
`type = 'CUSTOM-EXPRESSION'` rows (those belong to
[File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions), which uses the same `files`
table with a different `type` discriminator).

### System File Extensions DataTable (read-only)

The shipped catalogue — every row from `files` where `system = 'YES'`
and `type IN ('EXT', 'EXT-HIGH')`. These rows are filtered out of
every DELETE path on this page (`AND system = 'NO'` is part of every
DELETE query). The UI gives them no checkbox and no Delete button;
attempting a forged POST that targets a system row surfaces alert
`m = 11` and is rejected.

Standard rows get an "Info" badge, High Risk rows get a "Danger"
badge. The badge is cosmetic — Amavis treats both the same as
banned-extension candidates once they're wired into a File Rule.

## Foreign-key guard on delete

A custom extension cannot be deleted while it is referenced by any
[File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules). The single-row Delete handler runs:

```sql
SELECT COUNT(*) AS cnt FROM file_rule_components
WHERE file_id = :id
```

If `cnt > 0`, the delete is refused with alert `m = 10` and the
DataTable shows the offending rule name(s) ("This file extension is
used in the following File Rule(s): **HighRisk-block**"). The
operator's path is to open File Rules, remove the extension from
the rule, then come back here and delete it.

Bulk Delete applies the same guard per-id and accumulates partial
results — the success alert reports "N deleted, M skipped" with the
skipped rows' rule names attached so the operator knows exactly
what to unwire first.

## Save and apply flow

```
1. View page submits action="add_entries" | "delete" | "bulk_delete"
2. For each valid entry:
     a. Read the case-sensitive/insensitive allow + ban templates
        from /opt/hermes/scripts/file_allow_{sense|insense} and
        file_deny_{sense|insense}
     b. Substitute THE-EXTENSION placeholder with the (dot-stripped)
        extension name
     c. INSERT INTO files (file, description, type, system, allow, ban)
3. If at least one row was added or deleted:
     a. update_amavis_config_files.cfm:
          - Read /opt/hermes/conf_files/50-user.HERMES (template)
          - Substitute SERVER-NAME, SERVER-DOMAIN, sa-spam-subject-tag,
            final-virus-destiny, final-banned-destiny, final-spam-destiny,
            final-bad-header-destiny, enable-dkim-verification,
            enable-dkim-signing placeholders from spam_settings
          - Render every File Rule's components into an
            @banned_filename_re block (per-rule, in priority order,
            using the allow/ban regex stored on each files row)
          - Substitute HERMES-USERNAME / HERMES-PASSWORD from
            /opt/hermes/creds/ for the Amavis MySQL lookup
          - Back up /etc/amavis/conf.d/50-user -> 50-user.HERMES,
            move rendered file into place
     b. docker exec hermes_mail_filter /etc/init.d/amavis force-reload
        (30-second timeout)
4. session.m = 1 (add) | 2 (single delete) | 12 (bulk delete)
```

Amavis is reloaded with `force-reload` rather than restarted —
the daemon re-reads `50-user` without dropping connections, and
mail in flight is not interrupted. The full container restart that
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) and
[Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) trigger is not needed here
because no SpamAssassin state is being touched.

The reload step is wrapped in `cftry`/`cfcatch` with **comment "Log
but don't block — extensions were added"** — if the reload itself
fails, the DB rows are already in place and the next save (or
manual `force-reload`) will re-render. The page does not roll back
on reload failure.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` | Add Extensions completed (with `entries_added` / `entries_skipped` / `entry_errors` set on session for the per-row breakdown alert) |
| `m = 2` | Single Delete succeeded; Amavis reloaded |
| `m = 10` | Single Delete refused — the extension is wired into at least one File Rule (rule names surfaced in the alert) |
| `m = 11` | Attempt to delete a system row (`system = 'YES'`) — refused at the DB query |
| `m = 12` | Bulk Delete completed (with `bulk_deleted` / `bulk_skipped` / `bulk_errors` set on session) |
| `m = 30` | Add submitted with an empty textarea |

The per-row error list is HTML-rendered into the alert body so the
operator sees every rejection at once ("Must start with dot: foo",
"Invalid characters: .x@y", "Description required: .docm",
"Duplicate: .exe"). No row is silently dropped without an
explanation in the alert.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_file_extensions.cfm` | `hermes_commandbox` | The page (validation + bulk add + DataTables + Amavis reload) |
| `config/hermes/var/www/html/admin/2/inc/get_file_extensions.cfm` | `hermes_commandbox` | Loads custom + system rows for the two DataTables |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Renders `50-user` from template + File Rules (called on every change) |
| `config/hermes/opt/hermes/scripts/file_allow_insense` / `file_allow_sense` | `hermes_commandbox` | Allow-regex templates with `THE-EXTENSION` placeholder |
| `config/hermes/opt/hermes/scripts/file_deny_insense` / `file_deny_sense` | `hermes_commandbox` | Ban-regex templates with `THE-EXTENSION` placeholder |
| `config/hermes/opt/hermes/conf_files/50-user.HERMES` | `hermes_commandbox` (read) -> `hermes_mail_filter` (live `/etc/amavis/conf.d/50-user`) | Canonical Amavis template; receives the rendered `@banned_filename_re` blocks |
| `/etc/amavis/conf.d/50-user` | `hermes_mail_filter` | Live Amavis config; reloaded with `force-reload` on every save |
| `files` table, `type IN ('EXT','EXT-HIGH')` | `hermes_db_server` (`hermes` DB) | Source of truth for the catalogue (system + custom) |
| `file_rule_components` table | `hermes_db_server` (`hermes` DB) | Cross-reference checked by the delete guard |
| `hermes_mail_filter` container | — | Hosts Amavis; receives `force-reload` (not restart) on every change |

## Related

- [File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions) — sibling page for full
  regex patterns against any filename (not just extension); rows
  live in the same `files` table under `type = 'CUSTOM-EXPRESSION'`
- [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules) — bundles extensions and expressions
  into named, prioritised rulesets; the consumer of every row this
  page creates
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) — content-level SpamAssassin
  rules (header / body / regex) — the body / header equivalent of
  what File Extensions does for attachment names
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — defines
  `final_banned_destiny` (what Amavis does with a banned-extension
  match) and binds File Rules to recipients via SVF Policies
- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — ClamAV runs in the
  same Amavis pass; a virus verdict on the same attachment overrides
  the banned-extension result
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — sibling Amavis tuning
  page; both write into Amavis configuration but extension blocks
  are categorical (matched -> banned) where SA rules are weighted
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — none of this matters
  for connections that never make it past the SMTP-time perimeter
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) — a banned-extension
  rejection appears with Type `Banned` and the matched extension
  surfaced in the detail view
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — Amavis logs the
  matched regex as `Blocked BANNED (.exe,.bat,...)` on the
  `amavis[...]:` line

# File Rules

# File Rules

Admin path: **Content Checks > File Rules**
(`view_file_rules.cfm`,
`inc/get_file_rules.cfm`,
`inc/update_amavis_config_files.cfm`).

This page is the **bundling layer** that turns the raw catalogues
on [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) and
[File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions) into named, prioritised
rulesets that Amavis can actually enforce. A File Rule is a named
group of file-type components (extensions, file types, MIME types,
high-risk variants of each, and custom regex expressions) plus a
default action (**Ban** or **Allow**) that the operator binds to
recipient traffic via an SVF Policy under
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings). Without a File Rule
wrapping them, no row on the catalogue pages does anything to mail.

Hermes ships one system rule, **SYSTEM_DEFAULT**, populated with a
broad ban list (executables, scripts, Windows-class-IDs,
double-extension trap, archive formats, dangerous MIME types). It
is read-only — it can be copied, but not edited or deleted. Every
custom rule the operator creates lives alongside it in the same
DataTable, marked **No** in the System Rule column.

## Where File Rules sits

```
          File Extensions          File Expressions
                |                          |
                v                          v
     +-----------------------+    +-----------------------+
     | files table           |    | files table           |
     | type IN ('EXT',       |    | type =                |
     |          'EXT-HIGH',  |    |   'CUSTOM-EXPRESSION' |
     |          'FILE',      |    +----------+------------+
     |          'FILE-HIGH', |               |
     |          'MIME',      |               |
     |          'MIME-HIGH', |               |
     |          'OTHER')     |               |
     +-----------+-----------+               |
                 |                           |
                 +-------------+-------------+
                               |
                               v
                +-----------------------------+
                |  File Rules (this page)     |
                |                             |
                |  file_rule_components:      |
                |   rule_id, rule_name,       |
                |   file_id (FK -> files.id), |
                |   description, type ('ban'  |
                |   or 'allow'), priority,    |
                |   system (1=shipped,        |
                |           2=custom)         |
                |                             |
                |  file_rules (legacy index): |
                |   rule_id, rule_name,       |
                |   system                    |
                +--------------+--------------+
                               |
                               v
                +-----------------------------+
                |  Anti-Spam Settings         |
                |   SVF Policy row            |
                |   policy.banned_rulenames   |
                |   = ''           |
                +--------------+--------------+
                               |
                               v
                +-----------------------------+
                |  Amavis 50-user.HERMES      |
                |   per-rule @banned_         |
                |   filename_re block, with   |
                |   the rule's components in  |
                |   priority order            |
                +-----------------------------+
```

A File Rule that is created but **not** bound to an SVF Policy is
inert. The rule renders into Amavis's config (`50-user` carries
every defined rule), but no recipient policy points at it, so
nothing in `@banned_filename_re` fires for traffic.

## The two backing tables

| Table | Role |
| --- | --- |
| `file_rule_components` | The real source of truth. One row per (rule, file-type) pair. Carries `rule_id`, `rule_name`, `file_id` (FK -> `files.id`), `description`, `type` (`ban` or `allow`), `priority`, `system` (1 = shipped, 2 = custom) |
| `file_rules` | A legacy index table holding only `rule_id`, `rule_name`, `system`. Hermes ships a single row in it (`SYSTEM_DEFAULT`, `system=1`) — the page's CRUD operations write to `file_rule_components` directly and the Delete handler also clears `file_rules` for the matching `rule_id`. New rules are NOT inserted into `file_rules`; rule existence is determined entirely by `DISTINCT rule_id` on `file_rule_components` |

The `system` value is the system / custom discriminator and is the
guard for every modify path:

- `system = 1` -> shipped (SYSTEM_DEFAULT only). Read-only —
  attempting to edit or delete returns `m = 24`. The Copy button
  still works.
- `system = 2` -> operator-added. Editable and deletable, subject
  to the policy-binding guard.

The action column is named `type` (not `action`) on
`file_rule_components` and is per-component: a single rule can mix
**ban** and **allow** components, although the page's UI surfaces
"Default Action" as a single radio button and assigns the same
value to every component on save. Mixing `ban` and `allow` on the
same rule is possible only by direct SQL.

## The page

A page guide callout, a single DataTable listing every rule
(system and custom together), and three modals: Create Custom File
Rule (`Add`), Edit File Rule, and Copy File Rule.

### File Rules DataTable

| Column | Source |
| --- | --- |
| Rule Name | `file_rule_components.rule_name` (distinct) |
| Type | Rendered from the first component's `type` — `Ban` or `Allow` |
| File Types | Every component's `description` as a list of `bg-secondary` badges, each suffixed with `(ban)` or `(allow)` |
| System Rule | `Yes` (info badge, `system=1`) or `No` (warning badge, `system=2`) |
| Actions | Copy (always present) + Edit + Delete (only when `system=2`) |

Default sort is **System Rule asc, Rule Name asc**, so the shipped
rule sinks below the custom ones once any exist (custom = `system=2`
sorts above shipped = `system=1`? No — `2` > `1`, but the column
order asc is intentional: shipped first, then custom alphabetised).
The DataTable carries `stateSave: true`, so the operator's sort /
search / page-size choices persist across page loads.

### Create Custom File Rule modal (Add)

| Field | Stored as | Notes |
| --- | --- | --- |
| Rule Name | `file_rule_components.rule_name` | Regex-validated against `[^_a-zA-Z0-9-]` — letters, numbers, dashes, underscores only. No spaces, no punctuation. Max length 50. Duplicates across both system and custom rules are rejected (`m = 22`) |
| Default Action | `file_rule_components.type` on every inserted component | Radio: `ban` (default) or `allow` |
| File Type checkboxes | One INSERT per checked box into `file_rule_components` | Eight grouped cards: High Risk Extensions, High Risk File Types, High Risk MIME Types, File Extensions, File Types, MIME Types, Other Types, Custom Expressions. Each card has a "select-all" master checkbox and a scrollable list of every `files` row of that `type`. At least one file type must be selected (`m = 23`) |

The handler computes the next `rule_id` as `MAX(rule_id) + 1`
(scoped across `file_rule_components`, not `file_rules`), assigns
`priority` sequentially as components are inserted (1, 2, 3, …
in submission order), and marks each row `system = 2`.

### Edit File Rule modal

Opens preloaded with the current rule's name, default action, and
checkbox selections — the JavaScript reads a `ruleComponents` map
written into the page at render time and ticks the matching
checkboxes across all eight category cards.

Save is destructive-then-rebuild: the handler **DELETEs** every
`file_rule_components` row for the rule_id, then re-INSERTs from
the new form selection. The same name / action / file-types
validation as Add applies, plus:

- System rules (`system=1`) are refused with `m = 24`. The button is
  not even rendered for system rows, but the action handler still
  guards against forged POSTs.
- If the rule name changed, the handler also UPDATEs
  `policy.banned_rulenames` so any SVF Policy binding survives the
  rename. The cascade is name-keyed, not id-keyed — the policy
  table stores the name string, not the rule_id.

### Copy File Rule modal

The only path to derive a new rule from SYSTEM_DEFAULT. Asks for a
new name (same `[a-zA-Z0-9_-]+` validation, same duplicate check,
same 50-char max), then INSERTs a fresh set of
`file_rule_components` rows under a new `rule_id` with all the
source rule's `file_id`, `description`, `type`, and `priority`
values preserved. The copy is always `system = 2` regardless of
the source's flag — so a copy of SYSTEM_DEFAULT becomes a fully
editable custom rule.

The default new-name in the modal is `_copy`, so the
operator can hit Copy on SYSTEM_DEFAULT and immediately get
`SYSTEM_DEFAULT_copy` ready to edit.

## Policy-binding guard on delete

A custom rule cannot be deleted while any SVF Policy points at it.
The Delete handler runs:

```sql
SELECT policy_name FROM policy
WHERE banned_rulenames = ''
```

If any row comes back, the delete is refused with alert `m = 25`
and the policy name(s) are surfaced in the alert ("You cannot
delete a file rule that is assigned to SVF Policy:
**Default,Inbound-Strict**. Remove the assignment first under
Content Checks > SVF Policies.").

This is the symmetric counterpart to the FK guard on
[File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) and
[File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions) — those pages refuse to
delete a row that is bundled into a rule; this page refuses to
delete a rule that is bundled into a policy.

## Save and apply flow

```
1. View page submits action="add_rule" | "edit_rule" | "delete_rule"
                          | "copy_rule"
2. Validate name (non-empty, regex-clean, non-duplicate, non-system
   on edit/delete), validate file_ids (non-empty)
3. For Add / Edit / Copy:
     a. Determine rule_id (next MAX+1 for Add/Copy, form value for Edit)
     b. (Edit only) UPDATE policy.banned_rulenames if rule_name changed
     c. (Edit only) DELETE existing file_rule_components for rule_id
     d. INSERT one file_rule_components row per checked file_id, with
        priority assigned sequentially (1..N) and system='2'
   For Delete:
     a. DELETE FROM file_rules WHERE rule_id = :id
     b. DELETE FROM file_rule_components WHERE rule_id = :id
4. update_amavis_config_files.cfm:
     - Read /opt/hermes/conf_files/50-user.HERMES (template)
     - Substitute SERVER/destiny/DKIM/MySQL-credential placeholders
     - Loop every DISTINCT rule_id in file_rule_components
       and emit a per-rule @banned_filename_re block in
       priority order, using each component's allow or ban
       regex from files.allow / files.ban
     - Back up /etc/amavis/conf.d/50-user -> 50-user.HERMES,
       move rendered file into place
5. docker exec hermes_mail_filter /etc/init.d/amavis force-reload
   (60-second timeout - longer than the catalogue pages because
    every rule re-renders)
6. session.m = 1 (add) | 2 (edit) | 3 (delete) | 4 (copy)
              | 10 (reload error) | 20-25 (validation refusals)
```

Amavis is reloaded with `force-reload` rather than restarted. If
the reload itself fails, the rule rows are already committed —
alert `m = 10` ("Configuration Error") fires but the DB is not
rolled back. The next successful save (or a manual `force-reload`)
will re-render.

## Failure semantics

| Alert | Trigger |
| --- | --- |
| `m = 1` | Rule created. The alert also nudges the operator to assign the rule to a policy under SVF Policies — without that binding the rule is inert |
| `m = 2` | Rule updated; Amavis reloaded |
| `m = 3` | Rule deleted; Amavis reloaded |
| `m = 4` | Rule copied. Same nudge as `m = 1` — the copy is inert until bound to an SVF Policy |
| `m = 10` | Amavis reload error — the DB write succeeded but `force-reload` returned non-zero. Open Anti-Spam Settings and save once to re-trigger the render + reload, or restart `hermes_mail_filter` manually |
| `m = 20` | Rule name field empty |
| `m = 21` | Rule name contains characters outside `[a-zA-Z0-9_-]` (spaces, dots, slashes, etc.) |
| `m = 22` | Duplicate rule name — checked against both system and custom rules |
| `m = 23` | No file types selected — at least one checkbox across the eight category cards is required |
| `m = 24` | Attempted to edit or delete a system rule (`system=1`) — refused. The operator's path is to Copy first, then edit the copy |
| `m = 25` | Delete refused — the rule is bound to one or more SVF Policies (policy names surfaced in the alert) |

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_file_rules.cfm` | `hermes_commandbox` | The page (CRUD + Copy + DataTable + three modals + Amavis reload) |
| `config/hermes/var/www/html/admin/2/inc/get_file_rules.cfm` | `hermes_commandbox` | Loads the rule list + every `files` row grouped by `type` for the modal cards (`get_files_ext_high`, `get_files_file_high`, …, `get_files_custom_expr`) |
| `config/hermes/var/www/html/admin/2/inc/update_amavis_config_files.cfm` | `hermes_commandbox` | Renders `50-user` from template + every File Rule's components |
| `config/hermes/opt/hermes/conf_files/50-user.HERMES` | `hermes_commandbox` (read) -> `hermes_mail_filter` (live `/etc/amavis/conf.d/50-user`) | Canonical Amavis template; receives the per-rule `@banned_filename_re` blocks |
| `/etc/amavis/conf.d/50-user` | `hermes_mail_filter` | Live Amavis config; reloaded with `force-reload` on every save |
| `file_rule_components` table | `hermes_db_server` (`hermes` DB) | The real rule store — one row per (rule, file-type) pair |
| `file_rules` table | `hermes_db_server` (`hermes` DB) | Legacy index — only SYSTEM_DEFAULT lives here; custom rules are NOT mirrored. Cleared on delete for the matching `rule_id` |
| `files` table | `hermes_db_server` (`hermes` DB) | Source of the file-type checkboxes; FK target of `file_rule_components.file_id` |
| `policy` table, `banned_rulenames` column | `hermes_db_server` (`hermes` DB) | Where SVF Policies record their rule binding; renamed in step with rule renames, checked by the delete guard |
| `hermes_mail_filter` container | — | Hosts Amavis; receives `force-reload` (not restart) on every change |

## Operational consequences

- **A rule with no policy binding is inert.** Creating a rule does
  not block anything by itself — Amavis renders the rule into
  `50-user` but no recipient policy points at it. The "Please
  assign the rule to a policy under Content Checks > SVF Policies"
  nudge in `m = 1` and `m = 4` is the operational reminder. Until
  the binding is in place the rule exists for the operator's
  benefit only.
- **Edit is destructive-then-rebuild.** Saving an edit DELETEs and
  re-INSERTs every component for the rule. Priorities are
  reassigned 1..N in checkbox-submission order, which is the page
  render order, not the order the operator originally added them.
  An edit that only adds one new file type will reshuffle the
  priority numbers of every existing component on that rule.
  Functionally invisible (`@banned_filename_re` evaluation is
  any-match), but visible if anyone reads the table directly.
- **Renames cascade through `policy.banned_rulenames`.** The page
  joins on name, not id — when the rule name changes, the policy
  row is updated in the same transaction. If a policy binding
  exists, the operator does not need to re-open the SVF Policy
  page after a rename.
- **Copy is the only path off SYSTEM_DEFAULT.** The shipped rule is
  hard-locked (`m = 24` on any edit / delete attempt). Operators
  who want to tighten the defaults (add `.iso`, remove `.rtf`,
  swap MIME types) make a copy, edit the copy, and bind the copy
  to the policy in place of SYSTEM_DEFAULT.
- **The Type badge shows the first component's action only.** A
  hand-mixed rule (`ban` and `allow` components on the same rule)
  will display whichever was inserted at priority 1. The DataTable
  does not flag mixed rules — the File Types column shows each
  component's `(ban)` / `(allow)` suffix, which is the only place
  the mix is surfaced. The UI itself only writes uniform rules.
- **Amavis reload timeout is 60s here, vs 30s on the catalogues.**
  Re-rendering every rule's `@banned_filename_re` block can take
  longer than re-rendering a single allow/ban regex for an added
  extension. If the reload times out, the page shows `m = 10` and
  the rule write still succeeded.

## Related

- [File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) — the plain-extension half
  of the file-type catalogue; rows here become checkboxes in this
  page's modals under "High Risk Extensions" and "File Extensions"
- [File Expressions](https://docs.deeztek.com/books/administrator-guide/page/file-expressions) — the regex half of the
  file-type catalogue; rows there become checkboxes under "Custom
  Expressions"
- [Message Rules](https://docs.deeztek.com/books/administrator-guide/page/message-rules) — the body / header equivalent
  of File Rules; binds SpamAssassin rules to scope rather than
  Amavis filename patterns
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — where File Rules
  are bound to recipient traffic via SVF Policies
  (`policy.banned_rulenames`) and where `final_banned_destiny`
  (the action on a match) is set
- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) — ClamAV runs in the
  same Amavis pass; a virus verdict on the same attachment
  overrides the banned-rule result
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) — sibling Amavis tuning
  page; both write into the same `50-user` regeneration chain but
  rule matches are categorical where SA score overrides are
  weighted
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) — note that banned-rule
  rejections are a body-side filter result, not an authentication
  result — they fire after ARC chain evaluation
- [Message History](https://docs.deeztek.com/books/administrator-guide/page/message-history) — a banned-rule rejection
  appears with Type `Banned` and the matched rule + component
  surfaced in the detail view
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — Amavis logs the
  rule name and matched component as
  `Blocked BANNED ('' matched)` on the `amavis[...]:`
  line

# Global Sender Rules

# Global Sender Rules

Admin path: **Content Checks > Global Sender Rules**
(`view_global_sender_block_allow.cfm`,
`inc/get_global_sender_block_allow.cfm`,
`inc/global_sender_add_entries.cfm`,
`inc/global_sender_edit_entry.cfm`,
`inc/global_sender_delete_entry.cfm`,
`inc/global_sender_write_and_reload.cfm`).

This page manages **system-wide envelope-sender rules** that apply
regardless of recipient. Every entry on this page is a single sender
pattern (full address, exact domain, or domain + subdomains) paired
with an action — **Block** or **Allow**. The rules are evaluated by
Postfix at `MAIL FROM` time, before the message body is read; an
Allow match additionally bypasses Amavis content filtering for that
sender.

Global Sender Rules are the system-wide counterpart to
[Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules). A Global rule
matches **all recipients** in the system; a Sender/Recipient rule
requires both a sender and a recipient to match. A Global entry
takes precedence over any Sender/Recipient entry for the same sender.

## Where Global Sender Rules sit in the flow

```
+-------------------+
| Remote SMTP peer  |
+---------+---------+
          |
          v
+-----------------------------------------------+
|  postscreen :25 (perimeter / RBL scoring)     |
+---------+-------------------------------------+
          |
          v
+-----------------------------------------------+
|  smtpd :25                                    |
|   smtpd_sender_restrictions =                 |
|     check_sender_access                       |
|       hash:/etc/postfix/amavis_senderbypass   |
|                                               |
|   match -> REJECT (block)                     |
|   match -> FILTER amavis:[127.0.0.1]:10030    |
|             (allow -> route past content      |
|              filtering)                       |
|   no match -> fall through to recipient rules |
+---------+-------------------------------------+
          |
          v
+-----------------------------------------------+
|  Amavis (white.lst / black.lst consulted      |
|  again at content-filter tier)                |
+-----------------------------------------------+
```

The same rule set is written to **two places** on each save: the
Postfix `check_sender_access` table (`/etc/postfix/amavis_senderbypass`,
`postmap`ed into a Berkeley DB) and the Amavis whitelist/blacklist
files (`/etc/amavis/white.lst`, `/etc/amavis/black.lst`). Block
entries surface at the Postfix tier — the connection is rejected at
`MAIL FROM` and Amavis is never invoked. Allow entries route past
Amavis content scoring via the `FILTER` transport hint, and are also
written to Amavis's own whitelist as a safety net for any mail path
that does reach Amavis (locally-injected, alias-rewritten, etc.).

## Pattern formats

The page accepts three pattern formats. The save handler validates
each line and auto-prepends `@` to bare domains so the stored row is
always in one of the three canonical forms:

| Format | Example | Matches |
| --- | --- | --- |
| Full email | `user@example.com` | A single envelope sender |
| Exact domain (`@`) | `@example.com` | Every sender on `example.com` only — subdomains do not match |
| Domain + subdomains (`.`) | `.example.com` | `example.com` **and** every subdomain (`sub.example.com`, `mail.sub.example.com`, ...) |

Bare-domain input (`example.com`) is treated as a typo for
`@example.com` and rewritten on insert. Email-syntax validation runs
on the host portion of every pattern; entries that fail validation are
collected into a "Invalid Entries" alert and the rest of the batch is
still processed.

## The page

A single warning callout, a multi-line Add form, and one DataTable.

### Add Sender Entries

A textarea (one entry per line) plus a Block/Allow radio. The form
processes the entire batch in one round-trip:

- Each line is trimmed, classified (`@domain`, `.domain`, full email,
  or bare domain), and validated.
- Valid lines are checked against `amavis_sender_bypass` for an
  exact-string duplicate; duplicates are collected separately.
- Surviving lines are inserted with `type = block` or `type = allow`.
  For Allow entries, the row's `transport` column is set to
  `FILTER amavis:[127.0.0.1]:10030` — this is the Postfix transport
  hint that bypasses content filtering when a sender match fires.
- If any entries were added, the page calls the write-and-reload
  include before redirecting.

The redirected page surfaces three separate inline alerts (green
success, red invalid, red duplicate) so a mixed batch reports clearly
on what happened to every line.

A small inline JS check flips a warning banner under the textarea when
the operator types a domain (no `@`) — the consequence of allow-listing
or block-listing an entire domain is significant enough to warrant the
extra nudge.

### Global Sender Entries (DataTable)

Searchable, sortable, paginated, with bulk-delete checkboxes and
per-row Edit / Delete buttons.

| Column | Source |
| --- | --- |
| Sender | `amavis_sender_bypass.sender` |
| Format | Derived from the leading character — `@` -> Domain badge, `.` -> Domain + Subdomains badge, otherwise Email badge |
| Action | `amavis_sender_bypass.type` -> Allow (green) or Block (red) |
| Actions | Edit (modal), Delete (confirm) |

Bulk delete posts a comma-separated list of row IDs from the wrapping
form. Single Edit and Delete use separate hidden forms so they don't
collide with the bulk submit handler.

## Save flow

Every Add, Edit, and Delete runs the full regeneration path inline:

```
1. Validate input + INSERT / UPDATE / DELETE on amavis_sender_bypass
2. cfinclude global_sender_write_and_reload.cfm:
     a. SELECT all type='allow' rows (with transport column)
     b. SELECT all type='block' rows
     c. Write /etc/postfix/amavis_senderbypass    (allow rows + transport)
     d. Write /etc/amavis/white.lst               (allow rows, one per line)
     e. Write /etc/amavis/black.lst               (block rows, one per line)
     f. docker exec hermes_postfix_dkim postmap /etc/postfix/amavis_senderbypass
     g. docker exec hermes_postfix_dkim chown root:root 
     h. docker exec hermes_postfix_dkim postfix reload
     i. docker exec hermes_mail_filter /etc/init.d/amavis force-reload
3. session.m = 1 / 2 / 5 (Added / Deleted / Updated)
   On failure -> session.m = 4 ("Apply Failed")
```

The Postfix `postmap` step is what makes Block entries actually take
effect — `check_sender_access` reads the hashed `.db` file, not the
plain-text source. Skipping the postmap (e.g. by editing the source
file out-of-band) is a common cause of "I added a block but mail is
still getting through".

> **Why both Postfix and Amavis get the list.** The Postfix tier
> handles the common case — Block rejects before DATA, Allow routes
> past Amavis via the `FILTER` transport. The Amavis-side
> `white.lst` / `black.lst` files are a defence in depth: any mail
> path that **does** reach Amavis (locally-injected mail, mail that
> was alias-rewritten after the sender check, mail from
> `permit_mynetworks` sources that skipped sender restrictions)
> still gets the same allow/block treatment at the content-filter
> tier. The two layers are kept in sync by the single save flow.

## The `amavis_sender_bypass` table

| Column | Purpose |
| --- | --- |
| `id` | Auto-increment primary key |
| `sender` | The pattern (`user@example.com`, `@example.com`, or `.example.com`) |
| `transport` | For Allow rows: `FILTER amavis:[127.0.0.1]:10030`. Empty for Block rows |
| `action` | Always `NONE` for active rows; reserved for future scheduled-action use |
| `type` | `allow` or `block` |
| `applied` | `1` once the row is live; future use for deferred apply |

The duplicate check on insert is an exact string match on `sender`,
so `@example.com` and `.example.com` are treated as separate (and
both can legitimately coexist — they match different sets of
addresses).

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Empty textarea | `session.m = 30`, redirect, no DB write |
| Invalid email/domain on a line | Line skipped, accumulated into the Invalid Entries alert; other valid lines still processed |
| Exact-string duplicate on a line | Line skipped, accumulated into the Duplicate Entries alert; other valid lines still processed |
| `cffile` / `postmap` / reload failure | `session.m = 4` ("Apply Failed"); inserted rows remain in the DB and will be re-applied on the next successful save |
| Postfix container down | Reload fails -> `session.m = 4`; mail flow continues with the previously-loaded Berkeley DB until the container is back |

The save is **not** transactional across the DB + file-write +
reload steps. If the DB insert succeeds but the postmap or reload
fails, the next Add/Edit/Delete will regenerate from the full DB
state and reapply.

## Operational guidance

- **Allow entries bypass every content filter** — Spam, Virus,
  Banned File, custom Amavis rules — for the matched sender, for
  every recipient in the system. The shipped warning callout on the
  page is not boilerplate; use Allow sparingly and prefer
  [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) for narrower
  exceptions.
- **Block is cheaper than content filtering.** A Block entry rejects
  the SMTP transaction at `MAIL FROM`. The body is never read, no
  spam score is computed, no virus scan runs. For known-phishing
  sender domains this is the right tier to act at.
- **Domain + subdomain (`.example.com`) carries a wide blast radius**
  — a Block entry on `.example.com` will reject mail from
  `support@example.com`, `noreply@news.example.com`, and every other
  subdomain. The textarea's live warning banner exists for exactly
  this case.
- **Order of precedence.** Global Sender Rules beat
  Sender/Recipient Rules. A Block on `@example.com` here will
  reject mail from that sender even if a per-recipient Allow exists
  on the Sender/Recipient Rules page for the same sender.

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_global_sender_block_allow.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/get_global_sender_block_allow.cfm` | `hermes_commandbox` | Loads the active row set for the DataTable |
| `config/hermes/var/www/html/admin/2/inc/global_sender_add_entries.cfm` | `hermes_commandbox` | Batch validation + INSERT loop |
| `config/hermes/var/www/html/admin/2/inc/global_sender_edit_entry.cfm` | `hermes_commandbox` | Single-row UPDATE + regen |
| `config/hermes/var/www/html/admin/2/inc/global_sender_delete_entry.cfm` | `hermes_commandbox` | Single or bulk DELETE + regen |
| `config/hermes/var/www/html/admin/2/inc/global_sender_write_and_reload.cfm` | `hermes_commandbox` | Writes the three files, runs postmap, reloads Postfix and Amavis |
| `amavis_sender_bypass` table | `hermes_db_server` (`hermes` DB) | Source of truth |
| `/etc/postfix/amavis_senderbypass` (+ `.db`) | `hermes_postfix_dkim` | Postfix `check_sender_access` lookup |
| `/etc/amavis/white.lst`, `/etc/amavis/black.lst` | `hermes_mail_filter` | Amavis sender whitelist / blacklist |
| `hermes_postfix_dkim` container | — | Runs `postmap` + `postfix reload` |
| `hermes_mail_filter` container | — | Runs `amavis force-reload` |

## Related

- [Sender/Recipient Rules](https://docs.deeztek.com/books/administrator-guide/page/senderrecipient-rules) — per-pair
  variant; narrower scope, lower precedence
- [Perimeter Checks](https://docs.deeztek.com/books/administrator-guide/page/perimeter-checks) — the upstream
  `smtpd_*_restrictions` toggles a connection is evaluated against
  before sender-access lookup
- [Network Block/Allow](https://docs.deeztek.com/books/administrator-guide/page/network-blockallow) — the IP-level
  `postscreen_access.cidr` table consulted **before** any sender
  evaluation; an entry there can short-circuit a peer regardless of
  envelope sender
- [RBL Configuration](https://docs.deeztek.com/books/administrator-guide/page/rbl-configuration) — third-party DNSBL
  scoring at the postscreen tier; runs before sender access lookup
- [BCC Maps](https://docs.deeztek.com/books/administrator-guide/page/bcc-maps) — sibling envelope-level rule table; the
  other half of the envelope-rule pair
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) — the content-filter
  tier that Allow entries route around
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — `mail.log` is where
  block rejections and Amavis bypass decisions surface for audit
- [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) — visible flow-of-mail
  diagnostics if a rule change has an unexpected effect

# Malware Feeds

# Malware Feeds

Admin path: **Content Checks > Malware Feeds**
(`view_malware_feeds.cfm`, `inc/get_malware_feeds_settings.cfm`,
`inc/malware_feeds_save_global.cfm`, `inc/malware_feeds_add_feed.cfm`,
`inc/malware_feeds_edit_feed.cfm`, `inc/malware_feeds_delete_feed.cfm`,
`inc/malware_feeds_toggle_feed.cfm`, `inc/malware_feeds_save_urls.cfm`,
`inc/generate_malware_feeds_configuration.cfm`).

This page manages the third-party ClamAV signature feeds that supplement
the stock `freshclam` definitions on [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings).
The feed manager is [Fangfrisch](https://github.com/rseichter/fangfrisch),
a small Python tool that handles per-feed authentication, cadence control,
integrity verification, and post-download deployment. Hermes ships ten
built-in feed definitions (free and commercial), exposes a custom-feed
form for additional sources, and a per-feed URL editor for signature
file selection. Refresh runs as an [Ofelia](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks)
job inside `hermes_mail_filter`.

This page replaced an earlier `view_antivirus_signature_feeds.cfm` page
(orphan cleanup tracked as issue #257); any sidebar bookmark or
external link pointing at the old page should be updated.

## How feeds reach ClamAV

```
                  +-------------------------------------------+
                  |  hermes-fangfrisch-refresh (Ofelia job)   |
                  |    inside hermes_mail_filter              |
                  |    schedule: @every     |
                  +----------------+--------------------------+
                                   |
                                   v
                  +-------------------------------------------+
                  |  /usr/bin/fangfrisch refresh              |
                  |    reads /etc/fangfrisch/fangfrisch.conf  |
                  |    iterates enabled feeds                 |
                  |    skips feeds whose own interval has not |
                  |    elapsed                                |
                  +----------------+--------------------------+
                                   |
                                   v
                  +-------------------------------------------+
                  |  Per-feed download                        |
                  |    auth via API key / customer_id /       |
                  |    serial_key when required               |
                  |    integrity check (sha256, md5, off)     |
                  |    -> /var/lib/fangfrisch/signatures/     |
                  +----------------+--------------------------+
                                   |
                                   v
                  +-------------------------------------------+
                  |  on_update_exec=/usr/local/bin/setup-     |
                  |  clamav-sigs (post-update hook)           |
                  |    validates each file with `clamscan`    |
                  |    copies valid files to /var/lib/clamav/ |
                  |    signals clamd to reload                |
                  +-------------------------------------------+
```

The page emits `/etc/fangfrisch/fangfrisch.conf` (an INI file) on every
save. Fangfrisch itself is invoked on a fixed Ofelia schedule; the
schedule is regenerated from `ofelia_jobs.schedule` and reflects the
Global Settings > Refresh Interval picker.

## Container and tool placement

| Component | Detail |
| --- | --- |
| Container | `hermes_mail_filter` (IPv4 `.105`, same container as ClamAV, Amavis, SpamAssassin) |
| Feed manager | `fangfrisch` (Python, third-party ClamAV signature aggregator) |
| INI config | `/etc/fangfrisch/fangfrisch.conf` (bind-mounted, owned `root:clamav`, mode `640`) |
| State DB | `sqlite:////var/lib/fangfrisch/db.sqlite` (per-feed last-refresh, integrity hashes) |
| Download dir | `/var/lib/fangfrisch/signatures/` (raw downloaded files) |
| Deploy dir | `/var/lib/clamav/` (validated files, ClamAV signature store) |
| Post-update hook | `/usr/local/bin/setup-clamav-sigs` (validates with `clamscan`, copies to deploy dir, signals reload) |
| Scheduler | Ofelia job `hermes-fangfrisch-refresh` row in `ofelia_jobs` |
| Default cadence | `@every 10m` (Fangfrisch then honors per-feed `interval =` to decide what to actually re-fetch) |

## Global Settings card

Four controls write to `parameters2 WHERE module = 'malware_feeds'`. The
first three substitute into the `[DEFAULT]` section of `fangfrisch.conf`
on every save; the fourth updates the Ofelia row that schedules the
refresh job.

| Field | Storage | INI / scheduler effect | Notes |
| --- | --- | --- | --- |
| Log Level | `parameters2.value2` (`log_level`) | `[DEFAULT] log_level = ...` | `debug,info,warning,error,fatal`; logs go to `docker logs hermes_mail_filter` |
| Default Max Size | `parameters2.value2` (`max_size`) | `[DEFAULT] max_size = ...` | Per-file cap. Regex anchors a number followed by `KB`, `MB`, `M`, or `B` (e.g. `5MB`, `10M`, `250KB`). Inherited by feeds that don't set their own |
| Update Timeout (sec) | `parameters2.value2` (`on_update_timeout`) | `[DEFAULT] on_update_timeout = ...` | Bounded 1-300. Caps how long `setup-clamav-sigs` is allowed to run |
| Refresh Interval | `parameters2.value2` (`refresh_interval`) AND `ofelia_jobs.schedule` | Ofelia `@every ` | Allowed values: `5m,10m,15m,30m,1h,2h,4h`. Fangfrisch's own per-feed `interval =` still gates whether each feed actually re-downloads on a given run |

The post-update hook path is hard-coded to
`/usr/local/bin/setup-clamav-sigs` and shown read-only beneath the form
as `[DEFAULT] on_update_exec`. The hook lives inside the
`hermes_mail_filter` image and validates each downloaded file with
`clamscan` before copying it to `/var/lib/clamav/`; a file that fails
validation is left in the Fangfrisch download dir and not deployed.

## Malware Feeds card

Rows from `malware_feeds_config` populate a DataTable; per-row form
posts toggle, edit, manage URLs, and (custom feeds only) delete. The
schema:

| Column | Role |
| --- | --- |
| `id` | Surrogate key |
| `section_name` | INI section header, `[]`. Lowercase alphanumeric + underscore (`^[a-z0-9_]+$`). Cannot change after creation. Unique. |
| `display_name` | Card label, free text |
| `enabled` | `tinyint(3)`, 0/1. Sliders here flip this. `enabled = yes/no` line in INI |
| `is_builtin` | `tinyint(3)`, 0/1. Built-in rows cannot be deleted (the Delete action button is suppressed in the UI and the delete handler refuses) |
| `prefix` | `${prefix}` interpolation source for URL entries. Optional |
| `interval_value` | Per-feed cadence (e.g. `1h`, `4h`, `1d`); blank = inherit `@every ` |
| `max_size` | Per-feed cap; blank = inherit Global Default Max Size |
| `integrity_check` | `sha256`, `md5`, `disabled`, or NULL (default `sha256`) |
| `api_key_1_name` / `api_key_1_value` | Optional auth key (e.g. `customer_id`, `receipt`). Value stored AES-encrypted with key `/opt/hermes/keys/hermes.key` |
| `api_key_2_name` / `api_key_2_value` | Second auth key (e.g. MalwarePatrol's `product`). Same encryption |
| `description` | Free text |
| `sort_order` | Display order; custom-add inserts at 100 |

### Built-in feed catalog (factory rows)

| Feed | Type | Default state | Auth | Notes |
| --- | --- | --- | --- | --- |
| SaneSecurity | Free | Enabled | None | Broad zero-day coverage; mirror `https://ftp.swin.edu.au/sanesecurity/` |
| URLhaus | Free | Enabled | None | Malicious URL signatures from abuse.ch |
| MalwarePatrol | Commercial | Enabled | `receipt`, `product` IDs | Configure both keys via Edit; subscription IDs are documented in the in-card help |
| MalwareExpert | Commercial | Enabled | `serial_key` | URL template embeds the serial in the path |
| SecuriteInfo | Commercial | Enabled | `customer_id` | Free tier available; paid tier unlocks extra URLs |
| TwinWave | Free | Enabled | None | Public GitHub-hosted signatures |
| ClamPunch | Free | Enabled | None | Heuristic family signatures |
| RFXN | Free | Enabled | None | R-fx Networks Linux Malware Detect signatures |
| InterServer | Free | Enabled | None | Hash + URL signatures |
| Ditekshen | Free | Enabled | None | YARA/ClamAV detection rules |

A commercial feed is "enabled" only in the sense that its row is
marked `enabled = 1`; without API keys the feed is configured but
will not actually fetch (the in-card help describes the per-vendor
key requirements and the table icon shows a yellow warning triangle
on commercial rows missing keys).

### Add Custom Feed modal

Free-form add for any feed source not in the built-in catalog.
Validation:

| Field | Rule |
| --- | --- |
| Section Name | `^[a-z0-9_]+$`, required, must not already exist |
| Display Name | Required |
| URL Prefix | Optional, becomes the `prefix =` line and the substitution source for `${prefix}` in URL entries |
| Update Interval | Optional, number followed by `m` (minutes), `h` (hours), or `d` (days). Examples: `10m`, `1h`, `1d` |
| Max Size | Optional, number followed by `KB`, `MB`, `M`, or `B`. Examples: `5MB`, `250KB` |
| Integrity Check | Dropdown: default (sha256), sha256, md5, disabled |
| Description | Optional free text |

A new custom feed is inserted with `enabled = 0` and `is_builtin = 0`;
the admin then opens the URL manager to register at least one URL
before turning the row on.

## Manage URLs modal (per-feed)

Rows from `malware_feed_urls` keyed by `feed_id`. Each URL becomes a
line in the corresponding `[]` block of `fangfrisch.conf`:

```
url_ = 
filename_ =    ## only when filename_override set
```

When a URL is toggled off, the `url_` prefix is replaced with `!url_` to
inactivate the line without losing the configuration. `${prefix}` in the
URL value is expanded against the feed's `prefix =` at fetch time.

| Field | Rule |
| --- | --- |
| Name (`url_key`) | `^[a-z0-9_.]+$`, must be unique within the feed (`UNIQUE KEY uq_feed_url(feed_id, url_key)`) |
| Download URL (`url_value`) | Full URL, or `${prefix}` shorthand when the feed has a prefix |
| Save As (`filename_override`) | Optional. Renames the downloaded file locally; useful when the source filename is too generic |
| Toggle | Per-URL on/off. Disabled URLs are skipped without being deleted |

Built-in feeds may have URLs that Fangfrisch maintains internally — the
in-modal note explains that an empty URL table for a built-in feed
means it is using its packaged defaults, not that it is broken.

## Save flow

```
1. View page submits action= save_global | add_feed | edit_feed
                          | delete_feed | toggle_feed | url_action
2. malware_feeds_*.cfm validates and UPDATEs/INSERTs/DELETEs the row(s)
3. generate_malware_feeds_configuration.cfm runs on EVERY action:
     a. SELECT module='malware_feeds' rows from parameters2 -> globalSettings
     b. SELECT malware_feeds_config -> all feed rows
     c. SELECT malware_feed_urls -> all URLs grouped by feed_id
     d. Build [DEFAULT] section + one [] block per feed
     e. Decrypt api_key_*_value with AES + /opt/hermes/keys/hermes.key
        (key emitted as ` = `)
     f. Write temp file -> /opt/hermes/tmp/_fangfrisch.conf
     g. dos2unix (tolerated if missing)
     h. cffile write -> /etc/fangfrisch/fangfrisch.conf
     i. docker exec hermes_mail_filter chown root:clamav + chmod 640
        (tolerated if container is down)
     j. cfinclude ofelia_generate_config.cfm
        (rewrites /etc/ofelia/config.ini if any schedule changed)
4. cflocation back to view_malware_feeds.cfm
5. session.m + session.alerttype + session.alertmsg drives the alert banner
```

Every UI action -- including a single-row enable/disable toggle -- runs
the full INI regen, ownership fix, and Ofelia config regen. There is no
incremental write path; the INI is always rendered from the current
database state. This means manual edits to `/etc/fangfrisch/fangfrisch.conf`
are lost on the next save -- store all configuration in the database.

## API key encryption

The `api_key_1_value` and `api_key_2_value` columns store AES-Base64
ciphertext using the key in `/opt/hermes/keys/hermes.key`. The edit
modal shows a masked preview (20 asterisks + last 4 chars of the
plaintext) for visual confirmation without exposing the full key.
Decryption happens only in `generate_malware_feeds_configuration.cfm`
at the moment the INI is rendered; a decryption failure replaces the
key line with a commented `##  = [decryption error]` marker
rather than aborting the save.

The encryption key file is mounted into `hermes_commandbox` only;
neither `hermes_mail_filter` nor any other service reads it. This
keeps the plaintext key out of the running config on disk for as
short a window as possible (write -> chmod 640 root:clamav -> next
Fangfrisch run reads -> file remains until next save replaces it).

## Manual refresh

The Ofelia job runs on schedule, but the same command can be invoked
manually from a host shell:

```
docker exec hermes_mail_filter fangfrisch --conf /etc/fangfrisch/fangfrisch.conf refresh
```

Fangfrisch is conservative — it will still skip feeds whose own per-feed
`interval =` window has not elapsed. To force a re-download of a single
feed regardless of cadence, the Fangfrisch state DB can be cleared for
that feed:

```
docker exec hermes_mail_filter sqlite3 /var/lib/fangfrisch/db.sqlite \
  "DELETE FROM refreshlog WHERE source = '';"
```

Then re-run the refresh. The post-update hook re-validates with
`clamscan` and deploys to `/var/lib/clamav/`. To inspect downloaded
files:

```
docker exec hermes_mail_filter ls -la /var/lib/fangfrisch/signatures/
```

## Failure semantics

| Failure | Behavior |
| --- | --- |
| Global save with non-allowlisted log_level / max_size / timeout / interval | session.m=`malware_feeds_error`, alerttype=danger, alertmsg explains; no DB write |
| Add Custom Feed with duplicate section name | session.m=error, alertmsg names the conflict; INSERT not attempted |
| Toggle/edit on non-existent `feed_id` | session.m=error "Feed not found"; no UPDATE |
| Delete attempted on a built-in feed | UI suppresses the button; handler refuses the row |
| API key decryption error at INI regen | INI line replaced with `##  = [decryption error]`; save still completes; Fangfrisch will treat the auth as missing on the next run |
| Container down during `chown`/`chmod` | `cftry` swallows the exec failure; INI is still written to the bind mount and the chown is applied next save when the container is back up |
| dos2unix binary missing | Tolerated via `cftry`; INI is written without the line-ending normalization step |

## Files and containers touched

| Path | Owner | Role |
| --- | --- | --- |
| `config/hermes/var/www/html/admin/2/view_malware_feeds.cfm` | `hermes_commandbox` | The page |
| `config/hermes/var/www/html/admin/2/inc/malware_feeds_*.cfm` | `hermes_commandbox` | Validate / save / regen per action |
| `config/hermes/var/www/html/admin/2/inc/generate_malware_feeds_configuration.cfm` | `hermes_commandbox` | Renders the INI from the DB; runs on every action |
| `/etc/fangfrisch/fangfrisch.conf` | `hermes_mail_filter` (bind-mounted, root:clamav, 640) | Live Fangfrisch config |
| `/var/lib/fangfrisch/db.sqlite` | `hermes_mail_filter` | Per-feed last-refresh state |
| `/var/lib/fangfrisch/signatures/` | `hermes_mail_filter` | Raw downloads (pre-validation) |
| `/var/lib/clamav/` | `hermes_mail_filter` (Docker named volume `mail_filter_data_clamav`) | Validated signature store; ClamAV reads from here |
| `/usr/local/bin/setup-clamav-sigs` | `hermes_mail_filter` (image-baked) | Post-update validation + deploy hook |
| `/opt/hermes/keys/hermes.key` | `hermes_commandbox` only | AES key for `api_key_*_value` columns |
| `malware_feeds_config` table | `hermes_db_server` (`hermes` DB) | Per-feed row state |
| `malware_feed_urls` table | `hermes_db_server` | Per-feed URL list (FK cascade delete on feed delete) |
| `parameters2` rows `module='malware_feeds'` | `hermes_db_server` | Global Settings card |
| `ofelia_jobs` row `hermes-fangfrisch-refresh` | `hermes_db_server` | Schedule (auto-updated when Refresh Interval changes) |

## Related

- [Antivirus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) -- the ClamAV engine that
  consumes the signatures Fangfrisch downloads; engine toggles
  (ScanMail, ScanArchive, etc.) and the per-engine signature whitelist
  live on that page
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) -- the Ofelia
  admin page; the `hermes-fangfrisch-refresh` job row is editable
  there (manual Run Now, enable/disable)
- [Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) -- per-rule SpamAssassin weight
  changes; not related to ClamAV but the closest neighbor for the
  "tune a built-in rule" pattern
- [Antispam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) -- SpamAssassin runs in the
  same Amavis pass; a ClamAV virus verdict from a Fangfrisch-supplied
  signature always pre-empts the spam score
- [DNS Resolver](https://docs.deeztek.com/books/administrator-guide/page/dns-resolver) -- every Fangfrisch HTTP
  download resolves through `hermes_unbound`; outbound HTTPS to the
  feed providers must be reachable
- [Email flow](https://docs.deeztek.com/books/installation-reference/page/hermes-seg-email-flow) -- full pipeline diagram
  showing where ClamAV (and therefore feed-derived signatures) fits

# Message History

# Message History

Admin path: **Content Checks > Message History**
(`view_message_history.cfm`,
`view_message.cfm`,
`inc/messages_release_message.cfm`,
`inc/messages_block_sender.cfm`,
`inc/messages_allow_sender.cfm`,
`inc/messages_train_ham.cfm`,
`inc/messages_train_spam.cfm`,
`inc/messages_forget_bayes.cfm`,
`inc/messages_sa_learn_sync.cfm`).

This is the **operator inspection surface** for everything that has
flowed through the content filter. Every message Amavis processes lands
as one row in `msgs` (per-message metadata) plus one row per recipient
in `msgrcpt` (per-recipient disposition). This page is the joined view
over those two tables, with a date range filter, a content-type filter,
a delivery-status filter, and per-row actions to release from
quarantine, train Bayes, or block/allow the sender.

Pairs with [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) and
[Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue). System Logs shows the raw
syslog stream (connection negotiation, milter results, queue lifecycle).
Mail Queue shows what Postfix is currently holding. **Message History
shows what the content filter saw, what verdict it produced, and what
landed where** -- and lets the admin act on those rows.

The same `msgs` table feeds the **Messages Processed** donut on
[System Status](https://docs.deeztek.com/books/administrator-guide/page/system-status); the per-user
self-service version of this view lives at `/users/2/view_message_history.cfm`
and is scoped to the logged-in recipient only.

## How a message gets into `msgs` and `msgrcpt`

```
        SMTP in                postfix              amavisd-new
   ──────────────────►  hermes_postfix_dkim ────► hermes_mail_filter
                            (port 25)              (port 10024)
                                                        │
                                                        │  scan: ClamAV,
                                                        │  SpamAssassin,
                                                        │  banned-files
                                                        ▼
                                            ┌─────────────────────┐
                                            │ amavis SQL backend  │
                                            │ datasource: hermes  │
                                            ├─────────────────────┤
                                            │  msgs    (1 row /   │
                                            │           message)  │
                                            │  msgrcpt (1 row /   │
                                            │           recipient)│
                                            │  maddr   (sender +  │
                                            │           rcpt addr │
                                            │           dedup)    │
                                            └──────────┬──────────┘
                                                       │
                                  ┌────────────────────┼────────────────────┐
                                  │                    │                    │
                                  ▼                    ▼                    ▼
                          ds=P (Pass)         ds=D (Discard)        ds=B (Bounce)
                          delivered to        quarantined +         rejected with
                          downstream MTA      no further delivery   DSN to sender
                                              (quar_loc set on
                                               msgs row)
```

The `ds` ("disposition") column on `msgrcpt` is the per-recipient
verdict. The `content` column on `msgs` is the per-message **why** --
virus, spam, banned attachment, bad header, oversized, clean, etc.
Together they answer "did this message get through, and if not, what
blocked it?"

## What's in the search form

The Search Messages card at the top of the page is the filter set; all
fields are submitted as URL params so any search is bookmarkable and
back-button safe.

| Field | URL param | Effect |
| --- | --- | --- |
| Start Date/Time | `startdate` | Lower bound on `msgs.time_iso`. Defaults to **24 hours ago**. Validated as a date by `isValid("date", ...)`; invalid values short-circuit to the error template |
| End Date/Time | `enddate` | Upper bound on `msgs.time_iso`. Defaults to **now**. Same validation as startdate |
| Search Results Limit | `limit` | `LIMIT` clause on the join query. One of `1000`, `1500`, `2500`, `5000`, `10000`, `15000` -- the dropdown is the allowlist, anything else aborts. **Defaults to 1000.** The form text warns: setting limit to 10000+ significantly increases page load time |
| Type | `content_filter` | Multi-select against `msgs.content` -- the per-message content type (see table below). Empty = all types. Tom Select widget with remove and clear buttons |
| Action | `action_filter` | Multi-select against `msgrcpt.ds`. Empty = all actions. Three options: `P` Delivered, `D` Blocked (Discarded), `B` Blocked (Bounced) |

The date pickers are Tempus Dominus widgets bound to the start/end
inputs at page load; they emit `yyyy-MM-dd HH:mm:ss` into the form
fields so the validation regex matches whether the admin types the date
or picks it.

## The `msgs.content` codes -- "what was this?"

These are the values rendered by the Type column and the values used by
the Type multi-select. They come from the `msg_content_type` table
(seeded at install time):

| Code | Description | Meaning |
| --- | --- | --- |
| `V` | Virus | ClamAV (or another configured scanner) hit a signature |
| `B` | Banned | A [File Rule](https://docs.deeztek.com/books/administrator-guide/page/file-rules) regex matched an attachment name, MIME type, or archive member |
| `U` | Unchecked | Amavis received the message but didn't scan (bypass policy, scanner failure, oversized, etc.) |
| `S` | Spam Quarantined | SpamAssassin score reached `spam_kill_level` per the recipient's [SVF Policy](https://docs.deeztek.com/books/administrator-guide/page/svf-policies) |
| `M` | Bad-Mime | MIME structure invalid in a way that broke the parser |
| `H` | Bad-Header | Header malformed per RFC; subject to per-policy `bad_header_lover` |
| `O` | Oversized | Message exceeded the configured size limit |
| `T` | Mta Error | Downstream MTA rejected the release / delivery attempt |
| `C` | Clean | Scanned, no findings, delivered |
| `Y` | Spam Tagged | Score reached `spam_tag2_level` (tagged with header) but stayed below `spam_kill_level` (delivered) |
| `s` | Spam Tagged (OLD) | Legacy lowercase variant; preserved for back-compat with older `msgs` rows |

The score column shown on the table is `msgs.spam_level` -- the raw
SpamAssassin score from the scan, **not** the per-policy threshold. A
row tagged `S` with score `7.2` means the recipient's SVF policy has a
`spam_kill_level` of `7.2` or lower.

## The `msgrcpt.ds` codes -- "where did it go?"

`ds` is one character per recipient row:

| `ds` | Column header | Meaning |
| --- | --- | --- |
| `P` | Delivered | Pass -- handed to the downstream MTA (Postfix re-injection on port 10025 for relay topology, LMTP to Dovecot for mailbox topology) |
| `D` | Blocked | Discard -- not delivered, quarantined on disk under `/mnt/data/amavis/` |
| `B` | Blocked | Bounce -- rejected at SMTP time with DSN to sender |
| anything else | N/A | Unexpected disposition; usually means amavis was killed mid-handoff or the row is partial |

Per-recipient is the key: a single message with three recipients can
have one `P`, one `D`, and one `B` row in `msgrcpt`. The table renders
each `msgrcpt` row separately even though they share a `mail_id`.

## The results table

The DataTable below the search card is sortable, paginated (50 / 75 /
100 / All rows per page), and exportable (Copy, CSV, Excel, PDF, Print
buttons rendered by the DataTables Buttons extension). Default sort is
Date/Time descending.

| Column | Source | Notes |
| --- | --- | --- |
| Checkbox | `msgs.mail_id` | Selects the row for the Message Actions modal. **Select All** in the header checks every checkbox on the current page |
| View | -- | Magnifier button; opens `view_message.cfm?mid=` with the same `startdate` / `enddate` / `limit` so the back link round-trips correctly |
| Archived | `msgs.archive` | `Y` if the quarantine file has been moved to the long-term archive mount, `N` if it's still in the live amavis quarantine. Drives where `view_message.cfm` reads the EML from |
| Date/Time | `msgs.time_iso` | Indexed (`idx_msgs_time_iso`); this is the column the date range filters on. Rendered `yyyy-mm-dd HH:mm:ss` |
| Sender IP | `msgs.client_addr` | The client IP that handed the message to Postfix. For inbound that's the upstream MTA; for outbound it's the relay submitter |
| Return-Path | `maddr.email` via `msgs.sid` | The envelope sender (`MAIL FROM`); resolved via the `maddr` address-dedup table |
| From | `msgs.from_addr` | The header `From:` -- which is what users see and what DMARC aligns to |
| To | `maddr.email` via `msgrcpt.rid` | The envelope recipient. **Per-recipient** -- one table row per `msgrcpt` row |
| Subject | `msgs.subject` | Decoded subject header |
| Score | `msgs.spam_level` | Numeric score from SpamAssassin; formatted with 2 decimal places |
| Type | `msg_content_type.description` | Translated from `msgs.content` -- see the code table above |
| Action | derived from `msgrcpt.ds` | `Delivered` / `Blocked` / `Blocked` / `N/A` |

If the date range returns zero rows, the table is replaced by an info
alert ("No messages were found for the selected date range").

## The View action -- `view_message.cfm`

Clicking the magnifier opens the per-message detail page. What that
page can show is gated by two install-time toggles in
`/opt/hermes/config/security.conf`:

| Toggle | Default | Effect |
| --- | --- | --- |
| `ALLOW_MESSAGE_CONTENT=yes` | off | Show the decoded message body (HTML + text). When off, only headers are rendered |
| `ALLOW_ATTACHMENT_DOWNLOAD=yes` | off | Render the attachment list with a download button per attachment. When off, attachments are silently not listed |

Both default off because viewing a quarantined message body is a
privileged operation -- it's the difference between "the admin can see
a message was rejected" and "the admin can read a user's mail." Sites
that need release-decision support enable `ALLOW_MESSAGE_CONTENT`;
sites that need forensic attachment extraction enable
`ALLOW_ATTACHMENT_DOWNLOAD`. The fast path reads only the raw MIME
headers via a buffered Java reader so the headers page loads cheaply
even on huge quarantine files; full-body parsing only happens when the
toggle is on.

The EML is read from one of two paths depending on `msgs.archive`:

- `archive='N'` -> `/mnt/data/amavis/` (live amavis quarantine)
- `archive='Y'` -> `/mnt/hermesemail_archive/mnt/data/amavis/` (long-term archive)

If the file no longer exists on disk, the page aborts to the error
template instead of returning a partial render.

## Message Actions -- the bulk-action modal

Above the results table, the **Message Actions** button opens a modal
that applies one of six actions to every row whose checkbox is ticked.
The action runs in a CFML loop over the comma-delimited `mail_id` list;
each iteration includes the matching action template per-message.

| Action | Include | What it does |
| --- | --- | --- |
| Block Sender | `inc/messages_block_sender.cfm` | Adds the envelope sender to the Amavis WB-list as `B` for the recipient of that message. Honors virtual-recipient validation -- bulk attempts against unknown recipients land in `failureinvalidrecipient_email` |
| Allow Sender | `inc/messages_allow_sender.cfm` | Same as Block Sender but writes `W` (whitelist). The recipient's future mail from that sender bypasses spam scoring |
| Release Message(s) to Recipient | `inc/messages_release_message.cfm` | Calls `docker exec hermes_mail_filter /usr/sbin/amavisd-release   `. Re-injects the message from the quarantine file into Postfix for delivery. Success detected by parsing `250 2.0.0` out of the amavisd-release stdout |
| Train Message(s) as Spam | `inc/messages_train_spam.cfm` | Runs `sa-learn --spam` against the quarantine EML so Bayes learns that pattern as spam |
| Train Message(s) as Ham (NOT Spam) | `inc/messages_train_ham.cfm` | Runs `sa-learn --ham` so Bayes learns that pattern as legitimate. Use this on the false positives released from quarantine |
| Remove Message(s) Previous Training | `inc/messages_forget_bayes.cfm` | Runs `sa-learn --forget` to undo a prior `--spam` or `--ham` call against the same message |

After any of the three Bayes actions, the page calls
`inc/messages_sa_learn_sync.cfm` (which `docker exec`s `sa-learn --sync`
to flush the in-memory token store to the Bayes database) and then
runs `/opt/hermes/scripts/bayes_chown_amavis.sh` so the freshly written
Bayes files stay owned by the amavis UID inside the content-filter
container. **Don't skip the sync** -- without it, scoring decisions
based on the new training only land after amavis's next periodic
auto-sync, which is up to an hour out.

The release-message path is the most operationally important: it
requires the quarantine file still exists on disk (the message hasn't
been pruned by the cleanup job), `amavisd-release` exits with a `250`,
and the downstream MTA accepts the re-injection. Any of those failing
puts the row in `failurereleasemessage_email` and surfaces a red alert.

> **By design.** Releasing a message does **not** automatically train
> it as ham. If a quarantined spam is actually legitimate, run
> **Release Message** *and* **Train as Ham** as separate bulk actions
> so Bayes learns the false positive.

## Status alerts -- the `m` flow

The page uses a `session.m` integer to pipe action-outcome alerts
between the action-handler block (top of file) and the alert renderers
(also top of file, after parameter setup). The handler sets
`session.m = ` and `cflocation`s back to the same URL with the
filter params preserved; the alert renderer reads `session.m`, emits
the matching alert, and clears the variable.

| `m` | Triggering action | Alert |
| --- | --- | --- |
| `1` | Submit clicked with no rows ticked | "You must first select message(s) before clicking the Message Actions button" |
| `3` | Block Sender | success / warning |
| `4` | Allow Sender | success / warning |
| `5` | Release Message(s) | success / warning |
| `6` | Train Ham | success / warning |
| `7` | Train Spam | success / warning |
| `8` | Forget (remove training) | success / warning |

The "warning" path fires when *some* rows in the bulk action failed --
the page lists both the successful and the failed subjects so the admin
can re-target the failures.

## Retention -- the message lifecycle

This page is **not the retention surface**; it is the read/action
surface against rows that the retention pipeline maintains. Two
scheduled jobs (registered as Ofelia jobs against `hermes_commandbox`)
own the message lifecycle:

| Schedule | Endpoint | Job |
| --- | --- | --- |
| `0 30 01 * * *` (01:30 daily) | `schedule/message_cleanup.cfm` | Prunes `msgs` + `msgrcpt` rows past the configured retention window and deletes the matching quarantine files from `/mnt/data/amavis/` |
| `@every 60s` | `schedule/quarantine_notify.cfm` | Reads the `idx_msgrcpt_notify` index, sends recipient-facing quarantine notifications for new `ds=D` rows that haven't been notified yet, and flips `notification_sent=1` |

Both are managed from [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks);
retention thresholds and per-content-type quarantine targets are
configured on [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings). The cleanup
job is the reason a `Release Message` action can fail with "quarantine
file does not exist" -- if you wait past the retention window, the EML
is gone and only the `msgs` row remains as a record.

## Performance notes

The base join (`msgs INNER JOIN msgrcpt ON msgs.mail_id =
msgrcpt.mail_id`) is hit on every page load with a `WHERE
msgs.time_iso BETWEEN ?` range. `idx_msgs_time_iso` is the index that
makes the date range cheap; without it the query degrades to a full
table scan and pages with `limit=15000` would time out on a busy
gateway. The per-row sub-queries (`getfromaddr`, `gettoaddr`, `gettype`)
fire **once per result row** because they were originally written with
N+1 semantics; on `limit=15000` that's 60K+ extra queries plus 15K
DataTable rows being rendered into the DOM. The "10000+ significantly
increases page load time" warning on the form is calibrated against
that reality.

Don't widen the date range and crank the limit at the same time when
debugging a specific incident. Narrow the window first, then widen the
limit only if you have to.

## Related pages

- [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) -- live queue (what Postfix
  is currently holding) vs. this page's historical record
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) -- raw `mail.*` syslog
  stream; use when this page shows a row but you need the
  connection / milter / delivery trace behind it
- [System Status](https://docs.deeztek.com/books/administrator-guide/page/system-status) -- the dashboard
  donut that aggregates the same `msgs` rows
- [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) -- the cleanup +
  notify jobs that maintain the data this page reads
- [Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings) -- spam thresholds, Bayes
  configuration, and quarantine retention windows
- [Anti-Virus Settings](https://docs.deeztek.com/books/administrator-guide/page/antivirus-settings) -- ClamAV configuration
  that drives `content='V'` verdicts
- [SVF Policies](https://docs.deeztek.com/books/administrator-guide/page/svf-policies) -- per-recipient `spam_kill_level`
  that decides whether a scored message lands here as `S` (quarantined)
  or `Y` (delivered with header)
- [File Rules](https://docs.deeztek.com/books/administrator-guide/page/file-rules) -- attachment regexes that drive
  `content='B'` verdicts
- [ARC Settings](https://docs.deeztek.com/books/administrator-guide/page/arc-settings) and [DMARC Settings](https://docs.deeztek.com/books/administrator-guide/page/dmarc-settings)
  -- upstream authentication signals that contribute to spam scoring
  and so influence which messages land here as `S` vs `Y`

# Message Rules

# Message Rules

Admin path: **Content Checks > Message Rules**
(`view_message_rules.cfm`,
`inc/get_message_rules.cfm`,
`inc/apply_message_rules.cfm`,
`inc/update_spamassassin_config_files.cfm`,
`inc/restart_mail_filter.cfm`).

This page maintains a catalogue of **custom SpamAssassin rules**
that score against a regex match in a specific part of the message
(header, body, raw body, full message, or URI). Every rule a row on
this page produces is appended verbatim to SpamAssassin's `local.cf`
as a `  ` line, paired with a `score 
` line, and (optionally) a `describe  ` line.
SpamAssassin then runs the rule on every message that reaches the
SpamAssassin pass inside Amavis. The cutoff that turns a final score
into a `tag` / `quarantine` action is set globally on
[Anti-Spam Settings](https://docs.deeztek.com/books/administrator-guide/page/antispam-settings); this page only writes
the rules themselves.

Message Rules is the body/header equivalent of what
[File Extensions](https://docs.deeztek.com/books/administrator-guide/page/file-extensions) does for attachment names.
Both ride into `local.cf` / `50-user` on save, both are validated
with `spamassassin --lint` before the mail filter restarts, but
File Extensions matches the trailing extension of an attachment
filename while Message Rules matches arbitrary regex against text
inside the message.

## Where Message Rules sits

```
                       +---------------------------------------+
   Message Rules       |  message_rules table                  |
   (this page)  -----> |   id, rule_name, rule_type, header,   |
                       |   regex, score, rule_desc, applied    |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  update_spamassassin_config_files.cfm |
                       |   renders every row as                |
                       |                    |
                       |     score                |
                       |     describe              |
                       |   substituted at ##CUSTOM-MESSAGE-RULES|
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  apply_message_rules.cfm              |
                       |   spamassassin --lint                 |
                       |   restart_mail_filter.cfm             |
                       |     (docker container restart         |
                       |      hermes_mail_filter)              |
                       +---------------+-----------------------+
                                       |
                                       v
                       +---------------------------------------+
                       |  /etc/spamassassin/local.cf in        |
                       |   hermes_mail_filter; rules contribute|
                       |   to every message's total score      |
                       +---------------------------------------+
```

A row added here only affects the SpamAssassin pass — it does not
reject at SMTP-time, it does not modify headers directly, and it
does not bypass content filtering for any recipient. It just adds
or subtracts from the final score, and whether that final score
crosses a quarantine threshold is decided by the recipient's
[SVF Policy](https://docs.deeztek.com/books/administrator-guide/page/svf-policies).

## Rule types

| Type | What it matches | Cost |
| --- | --- | --- |
| `header` | A specific message header (`Subject`, `From`, `Return-Path`, ...) or any header when `ALL` is set | Very cheap; runs against parsed header values |
| `body` | The decoded plain-text body | Cheap |
| `rawbody` | The raw/HTML body before SpamAssassin decodes it (good for catching CSS tricks, hidden text, encoded payloads) | Cheap |
| `full` | The entire raw message including all MIME parts and headers | Most expensive; use sparingly |
| `uri` | URIs extracted from the message body | Cheap; ideal for catching suspicious link patterns |

The Page Guide on the page calls out `full` as resource-intensive
because SpamAssassin runs the regex against the whole raw blob; a
greedy or expensive regex in a `full` rule can noticeably slow
every scan. Prefer `body`, `rawbody`, or `uri` where they cover
the case.

## Score semantics

The `score` value behaves identically to a
[Score Overrides](https://docs.deeztek.com/books/administrator-guide/page/score-overrides) weight, except this page
creates the rule from scratch instead of overriding a shipped rule:

| Score | Effect |
| --- | --- |
| Positive (`5`, `20`, etc.) | Adds to the spam score on match. Higher values push the message toward `tag` / `quarantine` |
| `0` | Rule still runs but contributes nothing — useful for keeping the rule in place during a tuning pass without firing it |
| Negative (`-3`, `-10`) | Subtracts from the score on match — effectively whitelists messages matching the pattern |

The validation accepts any numeric value in the range `-999 .. 999`
(per the input's `step="0.01"` and `min`/`max` attributes). The
SVF policy assigned to the recipient determines what `total score`
threshold triggers tag / quarantine — see [SVF Policies](https://docs.deeztek.com/books/administrator-guide/page/svf-policies).

## The page

A Page Guide callout, a collapsible **Regex Helper** card (three
tools: rule builder, common-pattern picker, regex tester — all
client-side JavaScript that just populates the Add form), an Add
Message Rule card, and an Existing Message Rules DataTable.

### Add Message Rule card

| Field | Stored as | Notes |
| --- | --- | --- |
| Rule Name | `message_rules.rule_name` | Required. Letters, numbers, dashes, underscores only — no spaces. Must be unique. SpamAssassin uses this as the rule identifier in logs (`X-Spam-Status` header reports rule names that fired) |
| Rule Type | `message_rules.rule_type` | Required. One of `header`, `body`, `rawbody`, `full`, `uri` |
| Header | `message_rules.header` | Required when Rule Type is `header`. Letters, numbers, dashes, underscores only. Datalist suggests common headers (`Subject`, `From`, `Return-Path`, ...) plus the special `ALL` token to match any header. For non-header rules the field is force-cleared on save |
| Regex Pattern | `message_rules.regex` | Required. A SpamAssassin-format regex like `/keyword/i`. For `header` rules a `~ ` prefix is auto-added on save (this is the SpamAssassin `=~` operator notation `header_name =~ /pattern/`) and stripped on display so the operator sees only the regex |
| Score | `message_rules.score` | Required, numeric, `-999 .. 999` |
| Description | `message_rules.rule_desc` | Optional. Surfaced into the rendered `local.cf` as a `describe` line, which feeds the rule into SpamAssassin's "why was this scored" explanations |

The handler validates each field in order and returns to the page
with `session.m_rules = ` for the first failure (form values
are preserved through `session.form_*` so the operator doesn't
re-type). Successful insert sets `applied = 2` (pending) before
the apply chain runs, then bulk-updates all rows to `applied = 1`
once `spamassassin --lint` and the restart succeed.

### Regex Helper card

Pure client-side, no server roundtrip:

| Tool | What it does |
| --- | --- |
| Build a Rule | Pick "match in body / header / raw / URIs," choose Contains / Exact / Starts / Ends / Any-of, type the text, click Build. JavaScript escapes regex metacharacters and assembles a `/pattern/i` string |
| Quick Select Common Patterns | A `