# System Status

# System Status

Admin path: **System > System Status** (`index.cfm` —
the post-login landing page; the sidebar entry points here, not to a
separate `view_system_status.cfm`). Supporting includes:
`inc/system_alerts.cfm`, `inc/check_system_update.cfm`,
`inc/get_system_version_build.cfm`, `inc/get_system_uptime.cfm`,
`inc/get_system_reboot_required.cfm`, `inc/get_system_resources.cfm`,
`inc/get_system_cpu_usage.cfm`, `inc/get_system_memory_usage.cfm`,
`inc/get_system_{root,data,vmail,nextcloud}_filesystem_usage.cfm`,
`api/get_system_resources.cfm`, `api/get_message_stats.cfm`.

System Status is the operator's at-a-glance picture of the running
gateway. It is the **default page after login** — every Authelia
post-login redirect lands here — and the union of every "is anything
broken right now" signal Hermes computes: license state, update
availability, host OS reboot pending, fresh-install onboarding nudges,
container resource usage, and live mail-processing volume.

The page is **graceful-degradation by design**: every widget catches
its own errors, every external call has a fallback, and a single
failed query (a missing setting row, an unreachable container, a
malformed log file) does not blank the dashboard. If you log in to
Hermes and the page renders at all, the page is doing its job.

## Page layout

```
+--------------------------------------------------------------+
| Top navbar    [license / fresh-install / update badges]      |
+--------------------------------------------------------------+
| Alert callouts (priority <= 5)                               |  rendered by top_navbar.cfm
|   * Templates Modified  (priority 1)                         |  from request.systemAlerts
|   * License Revoked     (priority 2)                         |  (populated in
|   * Placeholder hostname (priority 2)                        |   inc/system_alerts.cfm)
|   * Invalid / Pending / Grace-period expired (priority 3)    |
|   * Self-signed cert    (priority 3)                         |
|   * License Expired     (priority 4)                         |
|   * Offline Mode        (priority 5)                         |
+--------------------------------------------------------------+
| Welcome <user>   Last login: <timestamp>                     |
+--------------------------------------------------------------+
| System Info card                                             |
|   Version | Build | Edition | Uptime | Console IP/FQDN      |
|   | License Status | OS Updates | Hermes Update              |
+--------------------------------------------------------------+
| Messages Processed card                                      |
|   Donut chart + counts (Clean/Spam/Virus/Banned/             |
|   Bad Header/Other) over 15m / 1h / 8h / 12h / 24h           |
+--------------------------------------------------------------+
| System Resources card                                        |
|   CPU | Memory | Root FS | Data FS | Vmail FS | Nextcloud FS |
|   (seven progress rings, auto-refresh every 10s)             |
+--------------------------------------------------------------+
```

The two cards that show live data (Messages Processed and System
Resources) poll their own JSON endpoints in the background; the rest
of the page is rendered server-side once per load.

## Self-healing on first load

`index.cfm` is the **bootstrap convergence point** for several
secrets that the rest of the app depends on. If any are missing on
first load (fresh install, after a credential rotation, after a key
file deletion), they are generated in-place before the dashboard
renders:

| Missing artifact | Auto-generated by |
|---|---|
| `/opt/hermes/keys/hermes.key` (AES-256 application key) | `inc/generate_hermes_key.cfm` |
| `encryption_settings.user.serverSecret` (Ciphermail) | `inc/generate_ciphermail_server_secret.cfm` |
| `encryption_settings.user.clientSecret` (Ciphermail) | `inc/generate_ciphermail_client_secret.cfm` |
| `encryption_settings.user.systemMailSecret` (Ciphermail) | `inc/generate_ciphermail_mail_secret.cfm` |
| `/opt/hermes/scripts/container_ips.txt` (Fail2ban) | `inc/generate_container_ips.cfm` |

Each generator is idempotent — it checks "is the file/row empty?"
before writing, so subsequent loads are no-ops. This is why the very
first dashboard render on a fresh install is slightly slower than
subsequent loads.

## System Info card

The columns and what they mean:

| Column | Source | Notes |
|---|---|---|
| **Version** | `system_settings.version_no` | Always `Docker` in the Docker era. Hyphenated legacy values (e.g. `2024-08`) belong to bare-metal installs that have not yet been migrated. |
| **Build** | `system_settings.build_no` | Current release tag (`vYYMMDD`). This is the single value the update orchestrator compares against to decide what to apply. See [System Update](https://docs.deeztek.com/books/administrator-guide/page/system-update). |
| **Edition** | `session.edition` | `Community` or `Pro`. Community shows an `ENTER SERIAL` link to [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings); Pro shows the edition with state suffixes (`Pro (Templates Modified)`, `Pro (Validation Required)`) when the license is in a non-VALID state. |
| **Uptime** | `/opt/hermes/scripts/get_uptime.sh` via `cfexecute` | Host uptime in days, not container uptime. |
| **Console IP or FQDN** | `parameters2.console.host` | The value bound on [Console Settings](https://docs.deeztek.com/books/administrator-guide/page/console-settings). |
| **License Status** | `session.license` + `session.licenseexpires` | One of `VALID`, `EXPIRED`, `REVOKED`, `INVALID`, `TAMPERED`, `PENDING_VALIDATION`, `VIOLATION`, `N/A`. Community shows `N/A`. The status text resolves through the same license-evaluation logic documented in `inc/setsession.cfm`. |
| **OS Updates** | `/var/run/reboot-required` (file exists?) | `REBOOT REQUIRED` when the kernel/glibc-class update on the host needs a reboot. Hermes does not reboot the host — the admin does, via SSH. |
| **Hermes Update** | `inc/check_system_update.cfm` (reads `/opt/hermes/updates/check_system_update.txt`) | `UPDATE BUILD vYYMMDD FOUND` (clickable, opens GitHub release notes modal), `LATEST VERSION`, `UPDATE CHECK PENDING`, or `UPDATE CHECK UNAVAILABLE`. The cache file is written by the daily `schedule/check_for_update.cfm` job; see [System Update § Daily update check](https://docs.deeztek.com/books/administrator-guide/page/system-update#daily-update-check). |

The Hermes Update cell never makes a network call. It reads the cache
file written by the Ofelia-scheduled CFML job, so the page renders
fast and works offline.

> **By design.** The dashboard never calls the GitHub Releases API at
> page-render time. All network IO for "is there an update" happens
> in the once-a-day Ofelia job. If the cache file is missing (first
> load after install, before the first scheduled run) you see
> `UPDATE CHECK PENDING`, never a hang.

## Release-notes modal

Clicking `UPDATE BUILD vYYMMDD FOUND` opens a modal that fetches the
release body from the GitHub Releases API:

```
GET https://api.github.com/repos/deeztek/Hermes-Secure-Email-Gateway/releases/tags/<vYYMMDD>
```

The response's `body` field (Markdown) is converted client-side to
HTML and rendered in the modal. If the fetch fails (rate limit,
offline, release deleted) the modal degrades to a "View Release on
GitHub" button. The GitHub release page is canonical; the modal is a
convenience.

The tag passed to the URL is the build number **as-is**. Earlier
revisions of this code prepended `build-` to match the legacy
update-server file-name convention; that prefix was removed during
the [#218](https://github.com/deeztek/Hermes-Secure-Email-Gateway/issues/218)
release-engineering pivot because GitHub release tags do not carry it.

## Alert callouts

`inc/system_alerts.cfm` builds a priority-ordered array of alerts
each page load. The array is sorted ascending by priority (lower
number = more urgent), then split:

| Priority | Surface |
|---|---|
| 1–5 | Full-width callout banner under the navbar, rendered by `top_navbar.cfm` |
| 6+  | Compact badge next to the user/edition pill in the navbar |

Every Hermes page that includes `top_navbar.cfm` participates — the
callouts are not exclusive to System Status — but System Status is
where an admin is most likely to be looking when they appear.

### License-state alerts

| Alert | Priority | Trigger |
|---|---|---|
| Templates Modified | 1 | `session.license = TAMPERED` |
| License Revoked | 2 | `session.license = REVOKED` |
| Invalid License | 3 | `session.license = INVALID` |
| Validation Required | 3 | `session.license = PENDING_VALIDATION` (no offline baseline yet) |
| Grace Period Expired | 3 | `session.license = GRACE_PERIOD_EXPIRED` |
| License Expired | 4 | `session.license = EXPIRED` |
| Offline Mode | 5 | `VALID` + `validationMode = cached`; includes remaining grace-period day count |
| Expires in <N> days | 10 | `VALID` + `licensevaliddays <= 30` (badge only, never a callout) |

### Fresh-install onboarding nudges

Two universal nudges fire when the gateway is still using seed
defaults. Both apply to every install regardless of topology
(relay-only, mail-server-only, hybrid) and they live here precisely
**because** they are topology-agnostic.

| Nudge | Priority | Trigger | Fix link |
|---|---|---|---|
| Placeholder hostname | 2 | `parameters.myhostname = 'hermes.domain.tld'` (Postfix seed) OR `parameters2.console.host = 'smtp.domain.tld'` (console seed) | [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) |
| Self-signed cert | 3 | Every row in `system_certificates` is flagged `system = 1` (only bootstrap cert exists) | [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) |

Earlier iterations of this list included three more topology-specific
nudges (no relay domains, no relay networks, no recipients-or-mailboxes).
They were removed because they fired noisily on installs that legitimately
don't have those things — a relay-only install has zero mailboxes and
that is the correct configuration. Topology-specific onboarding guidance
lives in [`docs/install/get-started-docker.md`](https://docs.deeztek.com/books/installation-reference/page/get-started-docker)
instead, where it is read deliberately rather than nagged about every
page load.

### Other alerts (placeholders)

`system_alerts.cfm` includes guarded blocks for **Reboot Required**
(when `session.rebootRequired = true`) and **Cert Expiring** (when
`session.certExpiringSoon = true`). Neither flag is currently
populated by any code path — they are reserved for future widgets
that compute the values and stash them in the session.

## Messages Processed card

Polls `api/get_message_stats.cfm` on initial load and every 60s. The
period selector reloads with the new window value but does not
otherwise change the polling cadence.

| Bucket | Color |
|---|---|
| Clean | Green (`#28a745`) |
| Spam | Yellow (`#ffc107`) |
| Virus | Red (`#dc3545`) |
| Banned | Gray (`#6c757d`) |
| Bad Header | Dark (`#343a40`) |
| Other | Cyan (`#17a2b8`) |

The endpoint reads from the `msgs` table (Amavis-fed; covered in more
detail under [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs)) filtered to the selected
window. A 10,000-row hard cap is applied to keep page-load fast on
busy installs; when the cap is hit, the total is suffixed with `+`
and a small "Showing most recent 10,000 messages" note appears under
the breakdown.

## System Resources card

Seven progress rings, auto-refreshing every 10s via
`api/get_system_resources.cfm`:

| Ring | Source |
|---|---|
| CPU Utilization % | `/opt/hermes/scripts/get_cpu_usage.sh` |
| Memory Utilization % | `/opt/hermes/scripts/get_memory_usage.sh` |
| Root FileSystem % | `df` on `/` (host root) |
| Data FileSystem % | `df` on the Data tier mount (see [Storage Topology](https://docs.deeztek.com/books/installation-reference/page/storage-topology-5-tiers)) |
| Archive FileSystem % | `df` on the Archive tier mount (#260; Amavis quarantine) |
| Vmail FileSystem % | `df` on the Vmail tier mount |
| Nextcloud FileSystem % | `df` on the Nextcloud tier mount |

Each ring color-codes by threshold (`get_system_*_usage.cfm` returns
a hex color alongside the value). The rings degrade independently —
a missing tier mount renders that ring at 0 rather than failing the
whole card.

Tiers that share a host path (a smaller install where Archive, Vmail,
and Nextcloud are pinned to the same disk as Data) will show the same
percentage on multiple rings. That is the correct behavior; the
underlying `df` reading is the same.

## What is NOT on this page

System Status is intentionally a "snapshot" page, not an
investigation tool. It surfaces alerts and current resource state. It
does not surface:

| Want to see | Go here instead |
|---|---|
| Mail queue contents / deferred messages | [Mail Queue](https://docs.deeztek.com/books/administrator-guide/page/mail-queue) |
| Per-message processing history | [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) |
| Detailed cert / SAN status | [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates), [SMTP TLS Settings](https://docs.deeztek.com/books/administrator-guide/page/smtp-tls-settings) |
| Container health (`docker ps` output, restart counts) | Host shell — Hermes does not surface raw Docker state in the web UI |
| Scheduled-job last-run / next-run | [Scheduled Tasks](https://docs.deeztek.com/books/administrator-guide/page/scheduled-tasks) |
| Fail2ban bans in effect | [Intrusion Prevention](https://docs.deeztek.com/books/administrator-guide/page/ips) |
| Past update history | The git log on the host (`git log --oneline -- updates/`) |

## Failure semantics

| What breaks | What happens |
|---|---|
| `/opt/hermes/updates/check_system_update.txt` does not exist | `hermesupdate = "UPDATE CHECK PENDING"`; cell renders cleanly |
| Ofelia job has been failing for days (cache stale or shows old build) | Page still renders; the **Hermes Update** cell reflects whatever the last successful run wrote |
| GitHub API rate-limited or unreachable when an admin clicks the release-notes link | Modal falls back to a "View Release on GitHub" button |
| `df` on a tier mount fails | That ring renders at 0 with default color; other rings render normally |
| `get_uptime.sh` exits non-zero | Page short-circuits to the error template — uptime is treated as critical because its absence usually means a broken commandbox |
| `system_settings.build_no` / `version_no` row missing | Empty value in the matching cell; license cells will display `N/A` |
| `inc/generate_*` first-load generator fails | Logged; affected feature degrades downstream (Ciphermail mail crypto disabled, etc.) — the dashboard itself still renders |

## Files and containers touched

| Path | Owner | Role |
|---|---|---|
| `config/hermes/var/www/html/admin/2/index.cfm` | `hermes_commandbox` | Page |
| `config/hermes/var/www/html/admin/2/inc/system_alerts.cfm` | `hermes_commandbox` | Alert array builder (license + nudges + future widgets) |
| `config/hermes/var/www/html/admin/2/inc/check_system_update.cfm` | `hermes_commandbox` | Cache-file reader (Docker path) |
| `config/hermes/var/www/html/admin/2/inc/get_system_*.cfm` | `hermes_commandbox` | Per-widget data fetchers |
| `config/hermes/var/www/html/admin/2/api/get_system_resources.cfm` | `hermes_commandbox` | JSON endpoint for progress-ring auto-refresh |
| `config/hermes/var/www/html/admin/2/api/get_message_stats.cfm` | `hermes_commandbox` | JSON endpoint for message-stats card |
| `/opt/hermes/scripts/get_uptime.sh`, `get_cpu_usage.sh`, `get_memory_usage.sh` | `hermes_commandbox` | Shell helpers invoked via `cfexecute` |
| `/opt/hermes/updates/check_system_update.txt` | `hermes_commandbox` | Cache file written by `schedule/check_for_update.cfm`; read here |
| `/var/run/reboot-required` | host filesystem (mounted into `hermes_commandbox`) | Ubuntu's standard "kernel upgrade pending" sentinel |
| `/opt/hermes/keys/hermes.key` | `hermes_commandbox` | Created on first load if missing |
| `encryption_settings` table | `hermes_db_server` (`hermes` DB) | Ciphermail secrets populated on first load if empty |

## Related

- [System Update](https://docs.deeztek.com/books/administrator-guide/page/system-update) — the page System Status's
  "Hermes Update" cell links to; covers the daily update check and
  the orchestrator that consumes the cache file
- [System Settings](https://docs.deeztek.com/books/administrator-guide/page/system-settings) — where the Pro serial number
  is entered to lift Community to Pro
- [System Certificates](https://docs.deeztek.com/books/administrator-guide/page/system-certificates) — the page the
  "Self-signed cert" nudge links to
- [Server Setup](https://docs.deeztek.com/books/administrator-guide/page/server-setup) — the page the "Placeholder
  hostname" nudge links to
- [System Logs](https://docs.deeztek.com/books/administrator-guide/page/system-logs) — drill-down for the message-volume
  numbers shown in the Messages Processed card
- [Storage Topology](https://docs.deeztek.com/books/installation-reference/page/storage-topology-5-tiers) — explains
  the five tiers the resource-usage rings reflect
- [Release and Update Methodology](https://docs.deeztek.com/books/installation-reference/page/release-and-update-methodology)
  — methodology behind the version stamp this page surfaces