Oikos Week 4: Console v0, approval hardening, docs pass, backlog

Oikos Console v0 (oikos/console/) — read-mostly, server-rendered FastAPI
+ Jinja2 web UI, no SPA build chain. Signals landing page, service grid
+ detail, node/blast-radius view, live Mermaid relationship graph, drift
findings, approvals queue (approve/deny, destructive confirmation-phrase
enforced), daily/weekly reports. Tested end-to-end via the preview tools
against live production data, including a real click-through of the
approve/deny flow.

Found and fixed two bugs during that testing:
- Severity-dot CSS classes didn't match the actual severity strings
  (dot-warn/dot-crit vs "warning"/"critical") — warning-severity signals
  rendered with no visible indicator at all.
- The console's sys.path setup pointed at its own webhook checkout
  (/opt/oikos-console) rather than /opt/homelab-context, which would have
  made its oikos.* imports resolve to a SEPARATE copy of oikos/signal.py
  etc. than the scheduler and CLI use — silently forking signal/approval
  data into two locations in production. Fixed to match mcp/server.py's
  CONTEXT_DIR pattern. Also added _commit_push() so the console's writes
  (approval replies, signal ack/resolve) don't sit uncommitted against
  the 5-min-synced clone.

Split oikos/gen_topology_lib.py out of oikos/gen-topology.py (hyphenated
filenames aren't importable) so the console's /graph route can render
live without shelling out.

oikos/console/deploy/ — third webhook on dtoro/Homelab-Docs (port 9831),
matching the homelab-mcp/secrets-issuance precedent. README documents the
Caddy route and Gitea webhook registration this repo can't do for itself,
and that Authentik step-up on /approvals needs a live instance to
configure.

Approval hardening: grants are now single-use (oikos/approve.py
check_grant marks the request "executed" atomically, so a second call
for the same id fails even within the TTL) — verified with a test. Per-
agent age-key-signed requests, as originally planned, turned out not to
be buildable as stated: age is encryption-only, no signing primitive.
Documented the real alternative (SSH-key signing) and moved it to the
60/90-day backlog pending an inventory schema gap (no SSH pubkeys
recorded today).

Docs pass: added the Oikos command surface to operations/commands.md,
new MCP tools to AGENTS.md. Found two more stale references while at
it — commands.md and AGENTS.md both still pointed DNS at the destroyed
LXC 124/dnsmasq instead of Technitium on dns (107), and a claudio-monitor
reference deprecated since 2026-06-04 — fixed both.

60/90-day backlog written into OIKOS.md, derived from gaps actually
observed this month, not guesswork.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-06 00:03:51 +02:00
parent 2084a1583e
commit 205d8a1a43
25 changed files with 1116 additions and 117 deletions

View File

@@ -24,8 +24,9 @@ one pass through **Observe → Orient → Decide → Act**:
3. **Decide** — the classifier scores **risk class × blast radius ×
confidence** and routes:
- **auto-act**: within autonomy policy, high confidence, contained radius
- **escalate**: operator approval via Matrix (✅/❌ reaction; destructive
actions additionally need a typed confirmation phrase)
- **escalate**: operator approval via Matrix (✅/❌ reaction) or the
Oikos Console's `/approvals` page (destructive actions additionally
need a typed confirmation phrase either way)
- **queue**: informational — console + reports
The classifier can only *lower* autonomy relative to policy, never raise
it. When in doubt, escalate.
@@ -80,7 +81,8 @@ the `archaeology:` section. Each transition is a runbook checklist;
deprecation completes only when inbound edges reach zero.
Generated views: [infrastructure/topology.md](infrastructure/topology.md)
(Mermaid, regenerated from inventory).
(Mermaid, regenerated from inventory) and the live, clickable version at
`oikos.hubris.network/graph` once the Console is deployed.
## Conventions carried forward
@@ -112,10 +114,31 @@ Generated views: [infrastructure/topology.md](infrastructure/topology.md)
temperature isn't probed at all yet (no confirmed sensor path on
hubris/strong). DNS-vs-inventory and generic tracked-config-cleanliness
drift checks are also deferred (see `oikos/drift.py` docstring).
- **Week 4**: Oikos Console (oikos.hubris.network, behind Authentik with
step-up re-auth on approvals), per-agent age-key-signed approval
requests (upgrading from Week 3's shared-HMAC), docs pass, 60/90-day
backlog.
- **Week 4**: Oikos Console v0 shipped — signals landing page, service
grid + detail, node/blast-radius view, live Mermaid graph, drift view,
approvals queue (approve/deny, destructive confirmation-phrase
enforced), daily/weekly reports. Server-rendered FastAPI + Jinja2, no
SPA build chain, tested end-to-end against live production data (see
`oikos/console/`). Deploys as a third webhook on `dtoro/Homelab-Docs`
(`/opt/oikos-console`, port :9831) — see
[oikos/console/deploy/README.md](oikos/console/deploy/README.md) for
the Caddy route and Gitea webhook registration this repo can't do for
itself. Approval grants are now single-use (a second `check_grant` call
for the same request fails even within the TTL) and already exact-bound
to request id + entity + action.
**Not shipped as originally planned:** per-agent *age-key-signed*
request authentication — age has no signing primitive (it's an
encryption-only keypair format), so "age-key-signed" wasn't
buildable as stated. The real alternative (SSH-key signing via
`ssh-keygen -Y sign`/`-Y verify`, using each host's already-provisioned
SSH key) is real and buildable, but needs SSH public keys recorded in
inventory first — not there today. Moved to the 60/90-day backlog.
Authentik step-up re-auth on the approve/deny route is documented but
needs a live Authentik instance to configure — also backlog.
Docs pass done (this file, AGENTS.md, operations/commands.md); found
and fixed two more stale references while at it (DNS section still
pointed at destroyed LXC 124/dnsmasq instead of Technitium on 107, and
a `claudio-monitor` reference that's been deprecated since 2026-06-04).
### Real drift found while building Week 3 (unresolved, needs operator action)
@@ -138,3 +161,59 @@ each is a `config_mutation`/`destructive`-class decision:
- Three `lifecycle-pve-id-reuse` info findings (100, 106, 107 each shared
between an active host and an archaeology entry) — expected/benign ID
reuse after destroy, no action needed.
## 60/90-day backlog
Derived from gaps observed while building the 30-day roadmap, not
guesswork. Roughly ordered by what unblocks the most:
- **SSH-key-signed approval requests.** Replaces the design note in
Week 4: age keys can't sign (encryption-only format), so per-agent
request authentication needs `ssh-keygen -Y sign`/`-Y verify` against
each host's existing SSH key. Blocked on a schema gap: inventory
doesn't record SSH public keys today, only ports/users. First step is
populating that field on enrollment, then wiring `oikos/approve.py` to
require and verify a signature over the request payload.
- **Authentik step-up re-auth** on the Console's `/approvals` POST route
— needs a live Authentik `PromptStage`/reauth flow scoped to that path;
not configurable without a running instance to test against.
- **Prometheus provisioning** (see
[plans/2026-07-05-oikos-prometheus-lxc.md](plans/2026-07-05-oikos-prometheus-lxc.md))
— unblocks trend signals (disk-full prediction, temp creep) and real
sparklines in the Console; investigate the undocumented `pve_id 131`
on hubris first.
- **CPU/NVMe temperature probing** in the scheduler — needs a confirmed
sensor path on hubris and strong (lm-sensors vs vendor tool) before a
real check can be written; guessing one risks a probe that silently
never fires.
- **DNS-vs-inventory drift check** — compare Technitium zone records
against `services.*.url`/`public_host`; not implemented (`oikos/drift.py`
has no Technitium API wiring yet).
- **Generic tracked-config-cleanliness drift check** — today only caddy's
`/etc/caddy` git-checkout path is hardcoded in `oikos/drift.py`; every
other service with a `config_repo` needs its local checkout path
recorded (a `mutation_path`-style field, same gap Week 1's service
contract flagged but didn't backfill) before this generalizes.
- **Per-service policy overrides** (`oikos/policy.yaml`
`service_overrides`) — schema is ready (caddy/dns already use it);
populate more as specific services turn out to need non-default risk
classes.
- **Incident timeline generator** — stitch ledger + signal history into
a single narrative for `investigations/` entries instead of writing
them by hand.
- **Secret access audit** — who-can-decrypt-what report from
`.sops.yaml` + inventory `age_pubkey`s, extending what
`oikos/drift.py`'s SOPS check already partially does.
- **Restore drills** — exercise `backs-up-to` (once populated) by
actually restoring from a backup target on a schedule, not just
checking freshness.
- **Multi-agent delegation model** — more than one agent acting
concurrently; needs the ledger's `agent` field to carry real identity
(age pubkey, not just hostname) consistently, which it mostly does
already but hasn't been stress-tested with concurrent writers.
- **Grafana** — only if the Console's own Prometheus-backed sparklines
turn out to be insufficient once Prometheus ships.
- **"Generalize later" extraction** — the original decision was personal-
first, generalize-later (see Week 1). Once patterns stabilize, extract
a config-driven Oikos core with no `hubris.network`/`hubris`/`strong`
hardcoding, so it's installable on a different homelab.