dtoro 8850b85325 plans: remediate all HIGH audit items + architect/developer review
Addresses 15 original HIGH audit findings + 18 new findings from
systems architect + senior Go developer review (572 lines added).

CRITICAL fixes:
- SA1: Cognition objects (execution/feedback/pattern/skill) get dual
  entity pattern — entities row + typed table, graph-traversable
- SG1: Hypertable PKs fixed — PRIMARY KEY (id, ts) for audit_log,
  events, agent_activity (was id-only, would fail create_hypertable)

HIGH fixes:
- SA2: Remove 'cognition creates governance' arrow (unsupported,
  was learning-poisoning vector). Patterns propose, operator accepts.
- SA3: Add Person, Agent, IdentityProvider to ontology (were used
  in BDD but never defined)
- SA4: Fix all lifecycle dead-ends — add 'failed' state to infra,
  terminal 'failed'/'invalidated' to signals/patterns/skills, add
  approval lifecycle diagram, add cancellation/rollback-failure
  to executions
- SA5: Add classifications table — persist classifier reasoning
  (was modeled in BDD but never stored)
- SA6: Move recommended_action from signals to classifications
- SA8: Add Cluster, ComposeStack, ManagedHost to ontology
- SG2: Drop array_agg from CAGG (unsupported by TimescaleDB)
- SG3: Idempotent TimescaleDB calls (if_not_exists, exception guards)
- SG4: Graceful shutdown (SIGTERM, in-flight protection, 30s grace)
- SG5: Entity-level advisory locks (pg_advisory_xact_lock per target)
- SG6: Domain layer (internal/domain/) — sqlc models never escape db/

Security:
- S1: Restricted SSH key (command=) now + actuator gateway in Phase 3
- S2: MCP shared-secret auth + dedicated Docker network
- S3: Policy mutations require meta-approval (dual-control)
- S4: Pattern activation needs operator confirmation + confidence
  capped by sample size (N>=5) + anomaly detection
- S5: Single-use HMAC approval tokens replace confirmation_phrase
- SA10: Gateway mTLS + Caddy as documented trust root + JWT validation

Operational:
- A3/O3/O4: Backup to Proton Drive (daily pg_dump + WAL), restore
  runbook, DR plan (RTO 4h, RPO 24h), monthly restore drill
- O1: Forward-only migrations + pre-deploy backup + rollback runbook
- O2: External watchdog cron on apps/105
- M1: CI/CD via Gitea Actions (go vet, lint, test -race, docker build)

Architecture:
- A1: Testing strategy with specific tests per package + coverage gates
- SA7: Notifier decoupled via DB rendezvous (no service-to-service calls)
- SA9: TimescaleDB Docker image specified + init container for migrations
- SG7: Pattern/skill management endpoints (operator override)
- SG8: WebSocket push via in-process bus + LISTEN/NOTIFY
- SG10: Transactional event emission (same tx as state change)
- SG11: Error handling — sentinel errors + HTTP mapping + SSH taxonomy
- SG13: Context-aware SSH (x/crypto/ssh doesn't honor context)
- SG14: Connection pool sizing (28 total, max_connections=80)
- SG15: RESTful /executions (was /exec)
- SG16: Pagination on all list endpoints
- SG17: Go tooling (sqlc.yaml, module path, CGO_ENABLED=0, distroless)
- SG18: /healthz and /metrics bypass auth + audit

Updated phasing incorporates all remediation.
2026-07-06 23:35:50 +02:00
2026-07-06 00:46:27 +02:00
2026-07-06 21:08:13 +02:00

Homelab OS

Living documentation for the hubris Proxmox homelab + Oikos operating system.

For agents running on enrolled clients: start with AGENTS.md, then OIKOS.md.


For Agents — Navigation & Entry Points

You are running on a client enrolled in the hubris homelab

  1. First: Read AGENTS.md once. It explains who you are, the topology, available tools, conventions, and how to act.
  2. Before any mutation: Read OIKOS.md. It defines the operating model, risk classes, approval flow, and the ontology you'll consult.
  3. For specific workflows: Load the matching skill from .agents/skills/<name>/SKILL.md (e.g., service-health-check).
  4. When in doubt: Use MCP tools (search_docs, get_page, explain, get_changelog) — they're cheaper and more reliable than grepping.

Key References for Agents

  • What am I?/opt/homelab-context/hosts/<hostname>.yaml (read on first run)
  • Live topologyinventory.yaml + hosts/*.yaml (canonical, always wins)
  • Risk & approvaloikos/policy.yaml (enforced, not advisory)
  • Runbooks & workflows.agents/skills/ (risk class + verification checklist included)
  • State of OikosOIKOS.md build status (scheduled probes, drift detectors, signals, approval engine)

When to Use MCP vs Files vs Shell

Task Use Tool
Resolve hostname → address MCP get_host(name) or list_services()
Search wiki by content MCP search_docs(query)
Read a wiki page MCP or file get_page(path) or cat knowledge/wiki/.../...md
Get changelog entries MCP get_changelog(page, since?)
Understand a service MCP explain(service) — compact context card, cheaper than search+read
Blast-radius query MCP get_relations(entity) (ontology walk)
List available secrets MCP list_my_secrets() (scoped to your age key)
Browse or grep File Raw grep when MCP unreachable, or exploratory browsing

When MCP is unreachable: fall back to grepping the clone at /opt/homelab-context/. The local files are the same; MCP is just an index.


Understanding the Operating Model

Before you act, classify your action against oikos/policy.yaml.

The Oikos OODA Loop + Decision Tree

flowchart TD
    Observe["**Observe**<br/>probes, drift detectors, agent signals"]
    Orient["**Orient**<br/>ontology, context, state, entity relations"]
    Decide{"**Decide**<br/>classify against oikos/policy.yaml"}
    Auto["Auto-act<br/>(unattended)"]
    Escalate["Escalate<br/>homelab approval request"]
    Act["**Act**<br/>homelab CLI, runbooks, skills"]
    Verify["**Verify**<br/>checklist from SKILL.md"]
    Ledger["**Ledger**<br/>mutation record: who/what/risk"]
    Document["**Document**<br/>wiki update, same-session rule"]

    Observe --> Orient --> Decide
    Decide -->|read_only, reversible_low| Auto
    Decide -->|config_mutation, destructive| Escalate
    Auto --> Act
    Escalate -->|approval granted| Act
    Act --> Verify --> Ledger --> Document
    Document -.loop.-> Observe

Risk Classes (enforced, not advisory)

From oikos/policy.yaml:

  • read_only — status, logs, docs, inventory queries. Unattended. MCP tools are all read_only.
  • reversible_low — restart, cache clear, sync pull. Unattended + ledger entry.
  • config_mutation — tracked-config edits (commit+push, never local), deploys, upgrades, DNS/ingress changes. Operator approval required.
  • destructive — destroy, format, wipe, rotate, revoke. Approval + typed confirmation phrase.

Decision Flow

  1. Decide: Use homelab decide <action> <entity> to classify (risk class × blast radius × confidence).
  2. Escalate if needed: homelab approval request (Matrix-delivered to operator; see operations/commands.md).
  3. Execute: Use homelab CLI (not ad-hoc SSH) — it enforces policy, logs mutations, and verifies outcomes.
  4. Document: Update wiki in the same session (per AGENTS.md §5 and the same-session rule).

The Ontology Graph

Everything that can break, be changed, or hold data has an entity in inventory.yaml + oikos/ontology.yaml. Blast-radius questions ("what breaks if strong goes down?") are graph walks via homelab node <name> relations, not doc archaeology.

See: OIKOS.md (full operating model, OODA loop, primitives, lifecycle gates, build status).


Finding & Understanding Information

The narrative documentation is organized in layers:

Layer What it is Where Immutable? How agents use it
Sources Raw evidence: incidents, external refs, live state knowledge/sources/investigations/ Yes Read to understand root causes; do not rewrite
Wiki Synthesized current-state: one page per node & per system knowledge/wiki/{containers,hosts,vms,infrastructure}/ No This is the reference layer — if wiki disagrees with live state, update it in the same session
Index Pure listings — every page in scope with one-line summary index.md / folder README.md No Navigation aid; keep it current when wiki restructures
Log Append-only doc-maintenance record (restructures, ingests, lints) knowledge/log.md Yes (append-only) Read to understand past doc changes; never edit directly

Changelog ≠ Log: Each wiki page ends with a ## Changelog (infrastructure changes to that node, machine-parsed). That's not the Log; the Log records doc operations only.

See: llm-wiki.md (full rules, page structure, immutability contract).


Map & Quick Navigation

Agent Entry Points (Start Here)

Topology & Infrastructure

Node counts, IPs, and service lists change often — treat inventory.yaml and the index pages below as the source of truth, not this README.

Knowledge & References


Conventions

All pages follow:

  • File naming. Foundational docs (entry-points, agent instruction, references) are ALL-CAPS (AGENTS.md, OIKOS.md, GLOSSARY.md); containers use <id>-<name>.md; infrastructure pages use lowercase-with-dashes; plans and incidents use YYYY-MM-DD-slug.md; skills are <name>/SKILL.md. See page-templates.md for the full rules.
  • Voice & vocabulary. Concise, technical, sysadmin-to-sysadmin. No marketing prose, no puffers (seamless, robust, leverage, etc.). Full rules in writing-style.md.
  • Cross-linking is mandatory. If a page references a node or system, link to it. Treat orphans as a bug.
  • Live state wins. When something here disagrees with pct config / docker inspect / running state, fix the wiki and add a changelog entry in the same session.
  • Tracked configs. Pages for configs living in git repos (Caddy, Gitea, Artifacto, mule-image) must note the repo. Edits go through commit+push, never local changes. See auto-deploy.
  • No secrets. This is a private repo, but still: reference secret paths, never secret values.

For agents: Read caveman.md (terse communication standard). Use templates at page-templates.md when creating pages.


Updating the Wiki

When You Change Infrastructure

  1. Update the relevant page (config snapshot, ports, mounts, IP address).
  2. Add a ### YYYY-MM-DD — title entry to the page's ## Changelog section (reverse chronological order).
  3. If the change touches a cross-cutting system (DNS, Caddy, Authentik, mesh), update that page too and link from the changelog.
  4. If it's an incident, add a record to knowledge/sources/investigations/.

When You Restructure the Wiki

  1. Update the relevant index.md / README.md in that section.
  2. Add a single-line entry to knowledge/log.md: ## [YYYY-MM-DD] <operation> | <summary> (e.g., ## [2026-07-06] restructure | split infrastructure/dns into dns.md + dns-advanced.md).

The Same-Session Update Rule

Any meaningful state change made in this session requires a wiki update before the session closes. A change that touches a container page must also update:

  • The containers/index.md table (IPs, host, mounts, status)
  • The root README.md table (if affected)
  • The Caddy page site list (if affects *.hubris.network routing)
  • The DNS / ingress infrastructure pages (if affects routing)
  • The hosts/hubris.md or hosts/strong.md page (if container count changes)
  • The inventory.yaml host entry (source of truth for hosts/*.yaml generation)
  • The knowledge/wiki/infrastructure/topology.md (regenerate if needed)

Not updating all linked places is a bug. See page-templates.md — same-session update rule.


More Information

  • For Hermes agentsHERMES.md (persona, source-of-truth hierarchy, token efficiency)
  • For manual workflows.agents/operations/ (commands cheatsheet, agent enrollment, Hermes guide)
  • For skills/runbooks.agents/skills/ (load the matching SKILL.md before acting; includes risk class + verification)
  • MCP toolsAGENTS.md §3 (available tools, when to use MCP vs files)
  • Page templates & voice.agents/shared/ (page-templates.md, writing-style.md, caveman.md, llm-wiki.md)
  • Machine-readable substrateinventory.yaml, oikos/policy.yaml, oikos/ontology.yaml (not part of the wiki; see llm-wiki.md)
Description
Agentic OS for running a Homelab
Readme 37 MiB
Languages
Go 53.1%
Svelte 25.7%
TypeScript 14%
Shell 3.8%
Python 1.7%
Other 1.5%