diff --git a/.agents/OIKOS.md b/.agents/OIKOS.md new file mode 100644 index 0000000..715f169 --- /dev/null +++ b/.agents/OIKOS.md @@ -0,0 +1,227 @@ +# Oikos — the operating model + +Oikos (Greek: *household*) is the agent operating system layered on this +repo. It is not new infrastructure: `inventory.yaml` is the kernel data +structure, the `homelab` CLI and MCP server are the syscall surface, and +this page defines the rules everything above them follows. + +Read this after [AGENTS.md](../AGENTS.md). Machine-readable companions: +[oikos/ontology.yaml](../oikos/ontology.yaml) (systems model), +[oikos/policy.yaml](../oikos/policy.yaml) (risk & approval). + +## The kernel loop: OODA + +Every Oikos activity — scheduled probe, agent task, operator request — is +one pass through **Observe → Orient → Decide → Act**: + +1. **Observe** — probes, drift detectors, and agent findings produce + **Signals** (structured records, not loose messages): pending updates, + high temperature, low disk, service down, cert expiry, stale backup, + inventory drift. +2. **Orient** — walk the ontology graph: what entity is affected, what + depends on it (blast radius), its lifecycle state, whether a runbook + matches, what the ledger says about past attempts. +3. **Decide** — the classifier scores **risk class × blast radius × + confidence** and routes: + - **auto-act**: within autonomy policy, high confidence, contained radius + - **escalate**: operator approval via Matrix (✅/❌ reaction) or the + Oikos Console's `/approvals` page (destructive actions additionally + need a typed confirmation phrase either way) + - **queue**: informational — console + reports + The classifier can only *lower* autonomy relative to policy, never raise + it. When in doubt, escalate. +4. **Act** — execute through `homelab` commands or runbooks (never ad-hoc + SSH), then **verify** with the action's verification command, write a + **ledger** entry, resolve the Signal, and update docs in the same session. + +## Primitives + +| Primitive | What it is | Lives in | +|---|---|---| +| Host / Service | topology entities | `inventory.yaml` (+ generated `hosts/*.yaml`) | +| Secret | SOPS+age encrypted value, per-client recipients | `secrets/` + `.sops.yaml` | +| Runbook | executable workflow with risk class + verification | `runbooks/` (Week 2) | +| Signal | something needing attention, with lifecycle | `signals/` ledger (Week 3) | +| Change | one mutation: who, what, risk, approval, verification | `ledger/` (Week 2) | +| Approval | short-TTL signed grant for a gated action | approval engine (Week 3) | +| Incident | investigation narrative | `investigations/` | +| Plan | design doc for non-trivial work | `plans/` | +| Agent | enrolled client identity = its age pubkey | `inventory.yaml` + `.sops.yaml` | + +## Risk classes (enforced, not advisory) + +From [oikos/policy.yaml](../oikos/policy.yaml): + +- **read_only** — status, logs, docs, inventory. Unattended. +- **reversible_low** — restart, cache clear, sync pull. Unattended + ledger. +- **config_mutation** — tracked-config edits (commit+push, never local), + deploys, upgrades, DNS/ingress changes. Operator approval. +- **destructive** — destroy, format, wipe, rotate, revoke. Approval + + typed confirmation phrase. + +Lifecycle gates modify these: `provisioning` nodes are freely mutable +(nothing depends on them); `deprecated` nodes accept no new dependents; +anything touching a `destroyed` node is drift. + +## The systems model + +Eight domains — physical, compute, network, storage, software, +identity & access, operations, external — cover everything in the lab; +entities are connected by typed edges (`hosts`, `provides`, `mounts`, +`stores-on`, `routes-to`, `can-decrypt`, `depends-on`, `backs-up-to`, …) +defined in [oikos/ontology.yaml](../oikos/ontology.yaml). Rule of +completeness: **if it can break, be changed, or hold data, it has an +entity and edges.** Blast-radius questions ("what breaks if strong goes +down?") are graph walks, not doc archaeology. + +Nodes move through an explicit lifecycle — +`planned → provisioning → active → migrating → deprecated → destroyed` — +stored as `state:` in inventory (absent = active). Destroyed nodes live in +the `archaeology:` section. Each transition is a runbook checklist; +deprecation completes only when inbound edges reach zero. + +Generated views: [infrastructure/topology.md](../infrastructure/topology.md) +(Mermaid, regenerated from inventory) and the live, clickable version at +`oikos.hubris.network/graph` once the Console is deployed. + +## Conventions carried forward + +- Inventory is the truth; live state wins over narrative docs. +- Prefer `homelab` CLI and MCP over ad-hoc SSH. +- Meaningful changes update docs in the same session. +- Secrets are decrypted locally via per-client keys; never into docs/comments. +- Tracked configs change by commit + push, not local edits. +- Netbird is the preferred mesh path for new traffic. +- Agents are terse ([caveman.md](shared/caveman.md)), verify claims, and fix + collateral drift when found. + +## Build status (30-day roadmap, started 2026-07-05) + +- **Week 1**: policy, ontology, service contract, archaeology, topology + generator, this brief. Shipped. +- **Week 2**: context cards, `homelab service …`, change ledger, + `node relations`, runbooks. Shipped. +- **Week 3**: ops scheduler + state cache (`homelab service health` + is cache-first, `--live` forces a probe), drift detectors, signal engine + (`homelab signal …`), decision classifier (`homelab decide …`), approval + engine (`homelab approval …` — shared-HMAC grants; Matrix delivery is + Hermes's existing `@dtoro:avispero` send path, not a new bot, see + `oikos/approve.py`), daily brief + weekly report (`oikos/report.py`). + Shipped, except: Prometheus is still `planned` (see + [plans/2026-07-05-oikos-prometheus-lxc.md](../plans/2026-07-05-oikos-prometheus-lxc.md)) — + trend signals (disk-full prediction, temp creep) wait on that LXC; the + scheduler's disk check today is point-in-time only, and CPU/NVMe + temperature isn't probed at all yet (no confirmed sensor path on + hubris/strong). DNS-vs-inventory and generic tracked-config-cleanliness + drift checks are also deferred (see `oikos/drift.py` docstring). +- **Week 4**: Oikos Console v0 shipped — signals landing page, service + grid + detail, node/blast-radius view, live Mermaid graph, drift view, + approvals queue (approve/deny, destructive confirmation-phrase + enforced), daily/weekly reports. Server-rendered FastAPI + Jinja2, no + SPA build chain, tested end-to-end against live production data (see + `oikos/console/`). Deploys as a third webhook on `dtoro/Homelab-Docs` + (`/opt/oikos-console`, port :9831) — see + [oikos/console/deploy/README.md](../oikos/console/deploy/README.md) for + the Caddy route and Gitea webhook registration this repo can't do for + itself. Approval grants are now single-use (a second `check_grant` call + for the same request fails even within the TTL) and already exact-bound + to request id + entity + action. + **Not shipped as originally planned:** per-agent *age-key-signed* + request authentication — age has no signing primitive (it's an + encryption-only keypair format), so "age-key-signed" wasn't + buildable as stated. The real alternative (SSH-key signing via + `ssh-keygen -Y sign`/`-Y verify`, using each host's already-provisioned + SSH key) is real and buildable, but needs SSH public keys recorded in + inventory first — not there today. Moved to the 60/90-day backlog. + Authentik step-up re-auth on the approve/deny route is documented but + needs a live Authentik instance to configure — also backlog. + Docs pass done (this file, AGENTS.md, operations/commands.md); found + and fixed two more stale references while at it (DNS section still + pointed at destroyed LXC 124/dnsmasq instead of Technitium on 107, and + a `claudio-monitor` reference that's been deprecated since 2026-06-04). + +### Real drift found while building Week 3 (unresolved, needs operator action) + +The drift detectors surfaced genuine, currently-true findings on first +run against production — recorded here rather than silently fixed, since +each is a `config_mutation`/`destructive`-class decision: + +- `republic-laptop` has no `age_pubkey:` in `inventory.yaml`, but its real + age key is granted on nearly every shared secret in `.sops.yaml` + (`age1vf8h7...`) — the enrollment write-back to inventory never + happened. Fix: `homelab client add republic-laptop --finalize-pubkey + age1vf8h7s8mqsn2q5eadgpdupsj4mwn8zguc77d85ws3xj40sl9rgksx2rxw6`. +- `grimmory` has an `age_pubkey` in inventory but is missing from + `secrets/hello.yaml`'s recipient list — incomplete enrollment the + other direction. Fix: re-run `homelab client add grimmory + --finalize-pubkey `. +- `pve_id 131` exists live on hubris (`pct list`) with no inventory entry + — investigate before assuming it's a stale ID (see the Prometheus LXC + plan doc above, which flags this explicitly). +- Three `lifecycle-pve-id-reuse` info findings (100, 106, 107 each shared + between an active host and an archaeology entry) — expected/benign ID + reuse after destroy, no action needed. + +## 60/90-day backlog + +Derived from gaps observed while building the 30-day roadmap, not +guesswork. Roughly ordered by what unblocks the most: + +- **Fix the oikos-console deploy webhook's signature mismatch.** Console + is live on apps (105) via a manual `deploy.sh` run, but Gitea webhook + 14's deliveries all 403 with a signature mismatch for a cause not yet + found — the secret is confirmed synced correctly on both sides + (rotated once already to rule out drift). Until fixed, `git push` + doesn't auto-redeploy the console the way it does for homelab-mcp/ + secrets-issuance; re-run `deploy.sh` on apps manually after changes. + See [oikos/console/deploy/README.md](../oikos/console/deploy/README.md). +- **SSH-key-signed approval requests.** Replaces the design note in + Week 4: age keys can't sign (encryption-only format), so per-agent + request authentication needs `ssh-keygen -Y sign`/`-Y verify` against + each host's existing SSH key. Blocked on a schema gap: inventory + doesn't record SSH public keys today, only ports/users. First step is + populating that field on enrollment, then wiring `oikos/approve.py` to + require and verify a signature over the request payload. +- **Authentik step-up re-auth** on the Console's `/approvals` POST route + — needs a live Authentik `PromptStage`/reauth flow scoped to that path; + not configurable without a running instance to test against. +- **Prometheus provisioning** (see + [plans/2026-07-05-oikos-prometheus-lxc.md](../plans/2026-07-05-oikos-prometheus-lxc.md)) + — unblocks trend signals (disk-full prediction, temp creep) and real + sparklines in the Console; investigate the undocumented `pve_id 131` + on hubris first. +- **CPU/NVMe temperature probing** in the scheduler — needs a confirmed + sensor path on hubris and strong (lm-sensors vs vendor tool) before a + real check can be written; guessing one risks a probe that silently + never fires. +- **DNS-vs-inventory drift check** — compare Technitium zone records + against `services.*.url`/`public_host`; not implemented (`oikos/drift.py` + has no Technitium API wiring yet). +- **Generic tracked-config-cleanliness drift check** — today only caddy's + `/etc/caddy` git-checkout path is hardcoded in `oikos/drift.py`; every + other service with a `config_repo` needs its local checkout path + recorded (a `mutation_path`-style field, same gap Week 1's service + contract flagged but didn't backfill) before this generalizes. +- **Per-service policy overrides** (`oikos/policy.yaml` + `service_overrides`) — schema is ready (caddy/dns already use it); + populate more as specific services turn out to need non-default risk + classes. +- **Incident timeline generator** — stitch ledger + signal history into + a single narrative for `investigations/` entries instead of writing + them by hand. +- **Secret access audit** — who-can-decrypt-what report from + `.sops.yaml` + inventory `age_pubkey`s, extending what + `oikos/drift.py`'s SOPS check already partially does. +- **Restore drills** — exercise `backs-up-to` (once populated) by + actually restoring from a backup target on a schedule, not just + checking freshness. +- **Multi-agent delegation model** — more than one agent acting + concurrently; needs the ledger's `agent` field to carry real identity + (age pubkey, not just hostname) consistently, which it mostly does + already but hasn't been stress-tested with concurrent writers. +- **Grafana** — only if the Console's own Prometheus-backed sparklines + turn out to be insufficient once Prometheus ships. +- **"Generalize later" extraction** — the original decision was personal- + first, generalize-later (see Week 1). Once patterns stabilize, extract + a config-driven Oikos core with no `hubris.network`/`hubris`/`strong` + hardcoding, so it's installable on a different homelab. diff --git a/.agents/domains/knowledge/schema.md b/.agents/domains/knowledge/schema.md new file mode 100644 index 0000000..e16071b --- /dev/null +++ b/.agents/domains/knowledge/schema.md @@ -0,0 +1,48 @@ +# Knowledge domain — schema + +The knowledge domain is the durable, authoritative current-state documentation of the homelab: one +page per node and per cross-cutting system, synthesized from live state and evidence. It answers +"what exists and how does it work right now." + +It follows the [LLM Wiki layer model](../../shared/llm-wiki.md) and the +[writing-style](../../shared/writing-style.md) and [page-templates](../../shared/page-templates.md) +rules. + +## The narrative / substrate split + +The knowledge wiki is **narrative**. It sits alongside a **machine-readable substrate** that it +describes but never contains. The split is load-bearing: several programs read the substrate at +fixed paths, so the wiki reorganization never moves it. + +| Layer | Location | Consumed by | +|-------|----------|-------------| +| Substrate — source of truth | `inventory.yaml` (root) | MCP server, `homelab` CLI, `oikos/` scheduler/drift/relations/gen-topology | +| Substrate — generated host records | `hosts/*.yaml` (root) | `mcp/server.py` (`HOSTS_DIR`), `bin/homelab`; written by `mcp/build_host_files.py` | +| Substrate — kernel + context cards | `oikos/` (code, `oikos/cards/`, `oikos/state.json`) | MCP `explain`, scheduler | +| Narrative — synthesized wiki | `knowledge/wiki/{hosts,containers,vms,infrastructure}/` | humans, agents via MCP `get_page` / `search_docs` | +| Evidence — immutable sources | `knowledge/sources/`, `investigations/` | synthesis into wiki pages | + +## Wiki pages + +- **Node pages** (`knowledge/wiki/containers/-.md`, `.../vms/-.md`, + `.../hosts/.md`) follow the container/host template in + [page-templates.md](../../shared/page-templates.md): opening definition, `## At a glance`, + `## Role`, service/port map, storage, auto-deploy, `## Related`, `## Changelog`. +- **Cross-cutting pages** (`knowledge/wiki/infrastructure/.md`) follow the cross-cutting + template: `## Why`, `## Components`, `## How to apply`, `## Gotchas`, `## Related`, `## Changelog`. +- Each `inventory.yaml` host entry carries a `doc_page:` field pointing at its narrative page. + Changing where a page lives means updating that field (read by `bin/homelab`). + +## The two logs + +- The per-page **`## Changelog`** records infrastructure changes and is machine-parsed + (`get_changelog`, the Oikos ledger). Keep the `### YYYY-MM-DD — title` shape. +- **`knowledge/log.md`** is append-only and records *documentation-maintenance* operations only + (restructures, source ingests, lint sweeps): `## [YYYY-MM-DD] | `. It never + duplicates the Oikos change ledger (`oikos/ledger.py`). + +## Same-session update rule + +A change to a node updates every page that references it in the same session — the node page, the +section `README.md` table, the root `README.md`, the Caddy/DNS/ingress pages, the host page, and +`inventory.yaml`. See [page-templates.md](../../shared/page-templates.md#same-session-update-rule). diff --git a/.agents/domains/operations/schema.md b/.agents/domains/operations/schema.md new file mode 100644 index 0000000..ba8ca84 --- /dev/null +++ b/.agents/domains/operations/schema.md @@ -0,0 +1,50 @@ +# Operations domain — schema + +The operations domain holds the procedural and time-stamped documentation: runbooks (repeatable +procedures), investigations (incident evidence), and plans (design docs for non-trivial work). It +follows [writing-style](../../shared/writing-style.md); runbooks and plans use the imperative voice +exception. + +## Plans always live in `plans/` + +**Any plan or design doc for the Homelab is written into the repo `plans/` folder as +`plans/YYYY-MM-DD-slug.md` — never a scratch path, an agent-private plan location, or a chat +message.** An agent drafting a plan: + +1. Writes the file under `plans/` using the plan template in [page-templates.md](../../shared/page-templates.md). +2. Lists it in `plans/index.md`. +3. On completion, moves it to `plans/done/` and updates the index status. + +This is the single source for homelab design intent; keeping it in-repo means the plan is +versioned, reviewable, and reachable by MCP `get_page`/`search_docs` like any other doc. + +## Runbooks + +Repeatable procedures live in `runbooks/.md` with YAML front-matter that the Oikos policy and +lifecycle machinery reads: + +```yaml +--- +name: +risk_class: read_only | reversible_low | config_mutation | destructive +inputs: [, ...] +verification: "" +docs_update_checklist: [] +transition: " -> " # only for lifecycle runbooks +--- +``` + +`risk_class` values and the lifecycle `transition` states must match +[`oikos/policy.yaml`](../../../oikos/policy.yaml) and [`oikos/ontology.yaml`](../../../oikos/ontology.yaml). + +## Investigations + +Incident records live in `investigations/YYYY-MM-DD-slug.md` and are **evidence sources** — written +once at incident time, then linked from the changelogs of the nodes they implicate. Sections: +`## Summary`, `## Timeline`, `## Root cause`, `## Mitigations applied`, `## Open questions`. Resolved +incidents move to `investigations/archive/`. + +## The operations log + +`plans/log.md` and `investigations/log.md` are append-only records of documentation operations on +those areas (`## [YYYY-MM-DD] | `), distinct from the Oikos change ledger. diff --git a/.agents/shared/caveman.md b/.agents/shared/caveman.md new file mode 100644 index 0000000..a6225d4 --- /dev/null +++ b/.agents/shared/caveman.md @@ -0,0 +1,33 @@ +# CAVEMAN.md — communication mode for homelab agents + +Respond terse like smart caveman. All technical substance stay. Only fluff die. + +## Rules + +Drop: articles (a/an/the), filler (just/really/basically/actually/simply), pleasantries (sure/certainly/of course/happy to), hedging. Fragments OK. Short synonyms (big not extensive, fix not "implement a solution for"). Technical terms exact. Code blocks unchanged. Errors quoted exact. + +Pattern: `[thing] [action] [reason]. [next step].` + +Not: "Sure! I'd be happy to help you with that. The issue you're experiencing is likely caused by..." +Yes: "Bug in auth middleware. Token expiry check use `<` not `<=`. Fix:" + +## Levels + +- **lite** — no filler/hedging. Keep articles + full sentences. Professional but tight. +- **full** (default) — drop articles, fragments OK, short synonyms. Classic caveman. +- **ultra** — abbreviate prose words (DB/auth/config/req/res), strip conjunctions, arrows (X → Y). Code symbols/API names/errors: never abbreviate. + +Switch: `/caveman lite|full|ultra`. Stop: "normal mode". + +## Auto-Clarity + +Drop caveman for: security warnings, irreversible actions, multi-step sequences where fragments risk misread, user confused/repeating. Resume after clear part done. + +## Boundaries + +Code/commits/PRs: write normal. "stop caveman" or "normal mode": revert. Level persist until changed or session end. + +--- + +Source: https://github.com/JuliusBrussee/caveman +Copy to `~/.hermes/skills/` for Hermes Agent, or `~/.claude/projects//SKILL.md` for Claude Code. diff --git a/.agents/shared/llm-wiki.md b/.agents/shared/llm-wiki.md new file mode 100644 index 0000000..1d6f69f --- /dev/null +++ b/.agents/shared/llm-wiki.md @@ -0,0 +1,41 @@ +# LLM Wiki — the documentation contract + +How the narrative documentation in this repo is organized. The pattern is borrowed from the +`sources / wiki / index / log` model: a durable synthesized layer (`knowledge/wiki/`) built on top +of immutable evidence (`knowledge/sources/`, `investigations/`), with pure-listing indexes and an +append-only operations log. + +This contract governs the **narrative layer only**. The machine-readable substrate — `inventory.yaml`, +generated `hosts/*.yaml`, `oikos/`, `mcp/`, `secrets/`, `bin/` — is not part of the wiki and never +moves under it. See [the knowledge schema](../domains/knowledge/schema.md) for the split. + +## Layers + +- **Sources** are immutable raw material: incident records (`investigations/`), external reference + docs (`knowledge/sources/references/`), and the live system itself (`pct config`, `docker inspect`). + Read them; do not rewrite them into other sources. +- **Wiki** (`knowledge/wiki/`) is the synthesized, authoritative current-state layer: one page per + node (`containers/`, `vms/`, host narratives) and per cross-cutting system (`infrastructure/`). A + reader understands the topic from the wiki page without reading the sources. +- **Index** (`index.md` / folder `README.md`) is a pure listing — every page in scope with a + one-line summary, and nothing else. Anything the section wants to say up front goes into a page + the index lists, not into the index. +- **Log** (`log.md`) is append-only, recording *doc-maintenance operations* (restructures, source + ingests, lint sweeps) in single-line format: `## [YYYY-MM-DD] | `. + +## Two logs, kept distinct + +- **`## Changelog`** on each node/topic page records *infrastructure* changes to that node. It is + machine-parsed (`get_changelog`, the Oikos ledger) — keep the `### YYYY-MM-DD — title` shape. +- **`log.md`** per area records *documentation* operations only. It never duplicates the Oikos + change ledger (`oikos/ledger.py`), which stays authoritative for infra changes with + who/what/risk/approval/verification. + +## Rules + +- Wiki pages stay short and focused. A page past ~300 lines splits. +- Pages stay flat under `wiki/
/` until there are enough to warrant a sub-group. +- Every page follows [writing-style.md](writing-style.md). +- Plans and design docs always live in the repo `plans/` folder (`plans/YYYY-MM-DD-slug.md`), + listed in `plans/index.md`, moved to `plans/done/` on completion — never a scratch path or a chat + message. See [the operations schema](../domains/operations/schema.md). diff --git a/.agents/shared/page-templates.md b/.agents/shared/page-templates.md new file mode 100644 index 0000000..c848bc3 --- /dev/null +++ b/.agents/shared/page-templates.md @@ -0,0 +1,152 @@ +# Page templates for the Homelab Wiki + +The structural templates for each page type. Prose voice, vocabulary, and cross-reference rules live +in [writing-style.md](writing-style.md); the layer model (sources / wiki / index / log) lives in +[llm-wiki.md](llm-wiki.md). + +## Voice + +Concise, technical, sysadmin-to-sysadmin. No marketing prose, no exclamation marks. Full rules in +[writing-style.md](writing-style.md). + +## Page templates + +### Container page (`containers/-.md`) + +```markdown +# — `` + +One-sentence purpose. + +## At a glance +- **Hostname:** `` +- **IP:** `192.168.8.x` +- **Privilege:** privileged | unprivileged +- **Resources:** N cores / M GiB RAM / D GiB rootfs +- **Mounts:** `/mnt/library` ↔ `/mnt/library` (if any) +- **Public hostname:** `.hubris.network` (if proxied) + +## Role +What it does, what it talks to. + +## Service / port map +| Service | Listen | Notes | + +## Storage / config paths + +## Auto-deploy +(if any) — link to [auto-deploy](../infrastructure/auto-deploy.md) + +## Related +- [Caddy](121-caddy.md) (if proxied) +- [DNS](../infrastructure/dns.md) (if has subdomain) +- [Authentik](124-authentik.md) (if SSO) +- ... + +## Changelog +### YYYY-MM-DD — short title +What changed, why, link to investigation if any. +``` + +### Cross-cutting page (`infrastructure/.md`) + +```markdown +# + +One-sentence summary. + +## Why +Design rationale — what it replaces, what it solves. + +## Components +Where it runs, what files matter. + +## How to apply / use +Recipes. + +## Gotchas + +## Related +Links to nodes that host or depend on this. + +## Changelog +``` + +### Plan (`plans/YYYY-MM-DD-slug.md`) + +```markdown +# YYYY-MM-DD — + +## Goal +What this change achieves and why. + +## Current topology / state +Diagram or description of what exists now. + +## Target topology / state +What it looks like after. + +## Pre-flight checklist + +## Step-by-step procedure + +## Verification + +## Post-migration +Changelog entries to write, index status to update. +``` + +### Investigation (`investigations/YYYY-MM-DD-slug.md`) + +```markdown +# YYYY-MM-DD — <title> + +## Summary +1-3 sentences. + +## Timeline + +## Root cause + +## Mitigations applied + +## Open questions +``` + +## Linking discipline + +- Every container page links to every cross-cutting page it participates in. +- Every cross-cutting page lists the nodes that participate. +- Every investigation links to the nodes it implicates *and* gets back-linked from each node's changelog. +- Every plan links to the infrastructure pages it affects. When done, update the plan's status in `plans/index.md` and write changelog entries on affected node pages. + +## Changelog hygiene + +- Reverse-chronological (newest first). +- One entry per discrete change, even if you make several in one day. +- If a change spans nodes, repeat the entry on each affected page (different perspective is fine). +- Don't rewrite history — entries are append-only. Mistakes get a follow-up entry that supersedes them. + +## Same-session update rule + +When you make a change to a node — migrate an LXC, update an IP, change a +mount, deploy a new service — **update every relevant doc page in the same +session.** A change that touches a container page must also update: + +- The `containers/index.md` table (IPs, host, mounts, status) +- The `README.md` table (if the change affects listed columns) +- The Caddy page site list (if the change affects `*.hubris.network` routing) +- The DNS / ingress infrastructure pages (if the change affects routing) +- The `hosts/{hubris,strong}.md` host page (if container count changes) +- The `inventory.yaml` host entry (source of truth for the `hosts/*.yaml` generation) +- The `infrastructure/topology.md` (generated from inventory, but regen if needed) + +The pattern of updating only one page and leaving stale references on others +is a bug. If you're doing a multi-step migration, document the intermediate +state with a changelog entry that says "pending — will finalize after Phase +N." + +This rule is why Phase 2 of the strong migration (2026-07-05) caused +widespread stale data: individual container pages were updated in the +changelog but never had their At-a-glance sections, IPs, mount paths, or +host attribution updated. Don't repeat that. diff --git a/.agents/shared/writing-style.md b/.agents/shared/writing-style.md new file mode 100644 index 0000000..95da2d6 --- /dev/null +++ b/.agents/shared/writing-style.md @@ -0,0 +1,75 @@ +# Writing Style + +Write like a technical reference, not a marketing page. Every sentence conveys new information. +These rules govern **committed documentation** — wiki pages, READMEs, schemas, skills, `AGENTS.md`, +plans, investigations, and code comments. They are separate from [caveman.md](caveman.md), which +governs an agent's *chat responses*; the two do not conflict. + +New or rewritten pages follow these patterns from day one. Existing pages get updated the next time +they are touched. + +## Vocabulary — never use these + +- Significance puffers: "pivotal", "crucial", "vital", "groundbreaking", "transformative", "testament", "paramount", "invaluable". +- Analytical verbs: "delve", "leverage", "utilize", "facilitate", "foster", "showcase", "underscore", "streamline", "harness". +- Poetic nouns: "tapestry", "landscape" (figurative), "realm", "paradigm", "ecosystem" (figurative), "journey" (figurative), "nexus", "cornerstone". +- Promotional adjectives: "robust", "seamless", "innovative", "cutting-edge", "meticulous", "holistic", "comprehensive". +- Opening crutches: "In today's world", "In the ever-evolving landscape of", "It's worth noting that", "It is important to note that". + +Use short, common words: "use" not "utilize", "help" not "facilitate", "show" not "demonstrate". + +## Voice + +Describe what systems do and how they work. + +- **Reference prose** (node pages, cross-cutting infrastructure descriptions, `## Role`, `## Why`, + `At a glance`) is third-person: state facts about the system, not instructions to a reader. +- **Recipes, runbooks, and skills** are the exception: second-person imperative is allowed and + preferred where it makes a procedure clearer ("Edit the Caddyfile, commit + push", "Verify with + `dig +short`"). This matches how the operator actually works. The vocabulary, structure, and + cross-reference rules below still apply. + +## Page shape + +Every doc-level page follows the same shape so a reader scans it in one pass. + +1. **One H1 = the page title.** Node pages use `# <id> — \`<name>\``; topic pages use `# <Topic>`. +2. **Opening definition.** First paragraph, 1–3 sentences, says what the thing is. No motivation, no marketing, no setup. +3. **Body sections** in the natural order for the topic. Reuse the section templates in [page-templates.md](page-templates.md). +4. **`## Changelog`** at the bottom of every node/topic page — reverse-chronological, append-only. This section is machine-parsed (`get_changelog` in `mcp/server.py`); keep the `### YYYY-MM-DD — title` shape. +5. **Related links** only at the bottom, only when a reference cannot be woven inline. + +## Section indexes (folder READMEs) + +A folder's `README.md` opens with a 1–3 sentence prose intro that says what the section covers, then +a single navigation table — `| Document | What it covers |` — and nothing else. No stale counts, no +duplicated prose, no narrative between the intro and the table. + +## Structure rules + +- Make every sentence information-dense. Cut filler, qualifiers, and setup phrases. Lead with the concrete fact or action, not why it matters. +- No participial tack-ons (", highlighting the importance of…"). If the clause adds information, make it a separate sentence. +- **No meta-commentary about the content itself.** Do not narrate the page's own structure or linking strategy. +- Prefer **tables** for enumerable items with internal structure (service/port maps, field lists, status grids). Reserve bullets for short non-structured lists. +- Use the **bold-leading-phrase pattern** for structured points: `**Read-only by construction.** The MCP server never mutates state.` — a bold noun phrase, a period, then the explanation. +- When enumerating across services or nodes, give each its own `###` sub-section or a table row, not one run-on paragraph. +- Use backticks for code, paths, hostnames, and file names (`inventory.yaml`, `192.168.8.77`, `pct config`); italics for first-mention terminology. +- Use `>` blockquotes for caveats and gaps that interrupt the main flow: `> **Outstanding gap.** DNS-vs-inventory drift check not yet wired.` One thought per blockquote. + +## Diagrams + +- Mermaid is the default for topology and flow diagrams. `infrastructure/topology.md` is generated by `oikos/gen-topology.py` — do not hand-edit it. +- ASCII box diagrams are fine for small shape diagrams; keep them to one screen. + +## Sourcing and cross-references + +- **Factual discipline.** Every claim is grounded in a cited source, an adjacent linked page, or a directly observable fact (`pct config`, `docker inspect`, running config). Do not write sentences that sound sourced but are inference. When docs disagree with live state, fix the doc and note it in the changelog. +- **One-sided cross-references.** When two pages relate, the link lives in the page where the connection makes organizational sense. Do not add a back-pointer unless that direction also carries content the reader needs. +- **Cross-references are content, not catalog.** Inline links arise from the surrounding prose; the linked page must be needed to understand the current sentence. A bottom-of-page "Related" list is the fallback, not the default. +- Pages link with standard relative markdown links (e.g. a container page links to `../infrastructure/dns.md`), forming a navigable graph. Orphans are a bug. + +## Code comments and commit/PR prose + +- Comments explain intent, trade-offs, or constraints the code cannot convey. No diff narration, no type restatement, no section-divider comments. +- Commit messages and PR descriptions are problem → change → risk → verification, not a file-by-file diff restatement. +- The banned vocabulary applies the same way in comments and commit messages. diff --git a/AGENTS.md b/AGENTS.md index c9bfd93..8349188 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,10 +5,19 @@ context is in this checkout at `/opt/homelab-context/`. This file is the entry point. Read it once at start, then keep working. The operating model — OODA loop, risk classes, approval rules, the ontology, -and node lifecycle — is defined in [OIKOS.md](OIKOS.md). Before any mutation, +and node lifecycle — is defined in [OIKOS.md](.agents/OIKOS.md). Before any mutation, classify the action against `oikos/policy.yaml`; when the class requires approval, stop and ask the operator. +Agent-facing instruction is separated from human content under `.agents/`: +`.agents/shared/` holds the conventions every agent applies +([writing-style](.agents/shared/writing-style.md), [caveman](.agents/shared/caveman.md), +[page-templates](.agents/shared/page-templates.md), [llm-wiki](.agents/shared/llm-wiki.md)), and +`.agents/domains/` holds the per-domain schemas +([knowledge](.agents/domains/knowledge/schema.md), [operations](.agents/domains/operations/schema.md)). +The narrative wiki lives under `knowledge/wiki/`; the machine-readable substrate +(`inventory.yaml`, `hosts/*.yaml`, `oikos/`) stays at the repo root. + ## 1. Who you are Run `hostname` (Linux) or `scutil --get LocalHostName` (macOS), then read: @@ -102,7 +111,7 @@ Grep is fine for browsing or when MCP is unreachable. ## 6. Communication mode -Read and apply `/opt/homelab-context/CAVEMAN.md` (if present). It defines the lab's +Read and apply `/opt/homelab-context/.agents/shared/caveman.md` (if present). It defines the lab's terse-communication standard — drop filler, keep substance, use fragments. ## 7. Auto-setup mechanism diff --git a/CAVEMAN.md b/CAVEMAN.md index a6225d4..1660494 100644 --- a/CAVEMAN.md +++ b/CAVEMAN.md @@ -1,33 +1,3 @@ -# CAVEMAN.md — communication mode for homelab agents +# CAVEMAN.md — moved -Respond terse like smart caveman. All technical substance stay. Only fluff die. - -## Rules - -Drop: articles (a/an/the), filler (just/really/basically/actually/simply), pleasantries (sure/certainly/of course/happy to), hedging. Fragments OK. Short synonyms (big not extensive, fix not "implement a solution for"). Technical terms exact. Code blocks unchanged. Errors quoted exact. - -Pattern: `[thing] [action] [reason]. [next step].` - -Not: "Sure! I'd be happy to help you with that. The issue you're experiencing is likely caused by..." -Yes: "Bug in auth middleware. Token expiry check use `<` not `<=`. Fix:" - -## Levels - -- **lite** — no filler/hedging. Keep articles + full sentences. Professional but tight. -- **full** (default) — drop articles, fragments OK, short synonyms. Classic caveman. -- **ultra** — abbreviate prose words (DB/auth/config/req/res), strip conjunctions, arrows (X → Y). Code symbols/API names/errors: never abbreviate. - -Switch: `/caveman lite|full|ultra`. Stop: "normal mode". - -## Auto-Clarity - -Drop caveman for: security warnings, irreversible actions, multi-step sequences where fragments risk misread, user confused/repeating. Resume after clear part done. - -## Boundaries - -Code/commits/PRs: write normal. "stop caveman" or "normal mode": revert. Level persist until changed or session end. - ---- - -Source: https://github.com/JuliusBrussee/caveman -Copy to `~/.hermes/skills/` for Hermes Agent, or `~/.claude/projects/<name>/SKILL.md` for Claude Code. +Agent chat-mode rules now live at [`.agents/shared/caveman.md`](.agents/shared/caveman.md). diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7706538..df5760e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,147 +1,7 @@ -# Contributing to the Homelab Wiki +# CONTRIBUTING.md — moved -## Voice +Documentation conventions are now split across `.agents/shared/`: -Concise, technical, sysadmin-to-sysadmin. No marketing prose, no exclamation marks. - -## Page templates - -### Container page (`containers/<id>-<name>.md`) - -```markdown -# <id> — `<name>` - -One-sentence purpose. - -## At a glance -- **Hostname:** `<name>` -- **IP:** `192.168.8.x` -- **Privilege:** privileged | unprivileged -- **Resources:** N cores / M GiB RAM / D GiB rootfs -- **Mounts:** `/mnt/library` ↔ `/mnt/library` (if any) -- **Public hostname:** `<sub>.hubris.network` (if proxied) - -## Role -What it does, what it talks to. - -## Service / port map -| Service | Listen | Notes | - -## Storage / config paths - -## Auto-deploy -(if any) — link to [auto-deploy](../infrastructure/auto-deploy.md) - -## Related -- [Caddy](121-caddy.md) (if proxied) -- [DNS](../infrastructure/dns.md) (if has subdomain) -- [Authentik](124-authentik.md) (if SSO) -- ... - -## Changelog -### YYYY-MM-DD — short title -What changed, why, link to investigation if any. -``` - -### Cross-cutting page (`infrastructure/<topic>.md`) - -```markdown -# <Topic> - -One-sentence summary. - -## Why -Design rationale — what it replaces, what it solves. - -## Components -Where it runs, what files matter. - -## How to apply / use -Recipes. - -## Gotchas - -## Related -Links to nodes that host or depend on this. - -## Changelog -``` - -### Plan (`plans/YYYY-MM-DD-slug.md`) - -```markdown -# YYYY-MM-DD — <title> - -## Goal -What this change achieves and why. - -## Current topology / state -Diagram or description of what exists now. - -## Target topology / state -What it looks like after. - -## Pre-flight checklist - -## Step-by-step procedure - -## Verification - -## Post-migration -Changelog entries to write, index status to update. -``` - -### Investigation (`investigations/YYYY-MM-DD-slug.md`) - -```markdown -# YYYY-MM-DD — <title> - -## Summary -1-3 sentences. - -## Timeline - -## Root cause - -## Mitigations applied - -## Open questions -``` - -## Linking discipline - -- Every container page links to every cross-cutting page it participates in. -- Every cross-cutting page lists the nodes that participate. -- Every investigation links to the nodes it implicates *and* gets back-linked from each node's changelog. -- Every plan links to the infrastructure pages it affects. When done, update the plan's status in `plans/index.md` and write changelog entries on affected node pages. - -## Changelog hygiene - -- Reverse-chronological (newest first). -- One entry per discrete change, even if you make several in one day. -- If a change spans nodes, repeat the entry on each affected page (different perspective is fine). -- Don't rewrite history — entries are append-only. Mistakes get a follow-up entry that supersedes them. - -## Same-session update rule - -When you make a change to a node — migrate an LXC, update an IP, change a -mount, deploy a new service — **update every relevant doc page in the same -session.** A change that touches a container page must also update: - -- The `containers/index.md` table (IPs, host, mounts, status) -- The `README.md` table (if the change affects listed columns) -- The Caddy page site list (if the change affects `*.hubris.network` routing) -- The DNS / ingress infrastructure pages (if the change affects routing) -- The `hosts/{hubris,strong}.md` host page (if container count changes) -- The `inventory.yaml` host entry (source of truth for the `hosts/*.yaml` generation) -- The `infrastructure/topology.md` (generated from inventory, but regen if needed) - -The pattern of updating only one page and leaving stale references on others -is a bug. If you're doing a multi-step migration, document the intermediate -state with a changelog entry that says "pending — will finalize after Phase -N." - -This rule is why Phase 2 of the strong migration (2026-07-05) caused -widespread stale data: individual container pages were updated in the -changelog but never had their At-a-glance sections, IPs, mount paths, or -host attribution updated. Don't repeat that. +- [`page-templates.md`](.agents/shared/page-templates.md) — per-page-type structural templates, linking discipline, changelog hygiene, same-session update rule. +- [`writing-style.md`](.agents/shared/writing-style.md) — prose voice, banned vocabulary, page shape. +- [`llm-wiki.md`](.agents/shared/llm-wiki.md) — the sources / wiki / index / log layer model. diff --git a/GLOSSARY.md b/GLOSSARY.md index 624b77f..b88f94f 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -18,7 +18,7 @@ Terms and abbreviations used throughout the homelab wiki. | **Mesh** | Overlay VPN for off-LAN connectivity. Netbird is current; Tailscale is legacy | | **Netbird** | Preferred mesh VPN. VPS hosts the management plane; all homelab nodes are members | | **OIDC** | OpenID Connect. Protocol used by Authentik for SSO login flows | -| **Oikos** | Agent operating model (OIKOS.md). OODA loop, risk classes, policy, ontology | +| **Oikos** | Agent operating model ([.agents/OIKOS.md](.agents/OIKOS.md)). OODA loop, risk classes, policy, ontology | | **PVE** | Proxmox Virtual Environment — the hypervisor on both hubris and strong | | **SOPS** | `sops` — Mozilla SOPS. Encrypts secrets with age keys so they live in the git repo | | **Strong** | Secondary Proxmox VE node. Cluster member 2 (hostname `strong`, nickname ludo/ludo-mini) | @@ -30,4 +30,4 @@ Terms and abbreviations used throughout the homelab wiki. ## See also - [Infrastructure index](infrastructure/index.md) — cross-cutting systems each with their own doc page -- [OIKOS operating model](OIKOS.md) — agent policy, risk classes, lifecycle \ No newline at end of file +- [OIKOS operating model](.agents/OIKOS.md) — agent policy, risk classes, lifecycle \ No newline at end of file diff --git a/OIKOS.md b/OIKOS.md index 2746ecf..bc6bb92 100644 --- a/OIKOS.md +++ b/OIKOS.md @@ -1,227 +1,3 @@ -# Oikos — the operating model +# OIKOS.md — moved -Oikos (Greek: *household*) is the agent operating system layered on this -repo. It is not new infrastructure: `inventory.yaml` is the kernel data -structure, the `homelab` CLI and MCP server are the syscall surface, and -this page defines the rules everything above them follows. - -Read this after [AGENTS.md](AGENTS.md). Machine-readable companions: -[oikos/ontology.yaml](oikos/ontology.yaml) (systems model), -[oikos/policy.yaml](oikos/policy.yaml) (risk & approval). - -## The kernel loop: OODA - -Every Oikos activity — scheduled probe, agent task, operator request — is -one pass through **Observe → Orient → Decide → Act**: - -1. **Observe** — probes, drift detectors, and agent findings produce - **Signals** (structured records, not loose messages): pending updates, - high temperature, low disk, service down, cert expiry, stale backup, - inventory drift. -2. **Orient** — walk the ontology graph: what entity is affected, what - depends on it (blast radius), its lifecycle state, whether a runbook - matches, what the ledger says about past attempts. -3. **Decide** — the classifier scores **risk class × blast radius × - confidence** and routes: - - **auto-act**: within autonomy policy, high confidence, contained radius - - **escalate**: operator approval via Matrix (✅/❌ reaction) or the - Oikos Console's `/approvals` page (destructive actions additionally - need a typed confirmation phrase either way) - - **queue**: informational — console + reports - The classifier can only *lower* autonomy relative to policy, never raise - it. When in doubt, escalate. -4. **Act** — execute through `homelab` commands or runbooks (never ad-hoc - SSH), then **verify** with the action's verification command, write a - **ledger** entry, resolve the Signal, and update docs in the same session. - -## Primitives - -| Primitive | What it is | Lives in | -|---|---|---| -| Host / Service | topology entities | `inventory.yaml` (+ generated `hosts/*.yaml`) | -| Secret | SOPS+age encrypted value, per-client recipients | `secrets/` + `.sops.yaml` | -| Runbook | executable workflow with risk class + verification | `runbooks/` (Week 2) | -| Signal | something needing attention, with lifecycle | `signals/` ledger (Week 3) | -| Change | one mutation: who, what, risk, approval, verification | `ledger/` (Week 2) | -| Approval | short-TTL signed grant for a gated action | approval engine (Week 3) | -| Incident | investigation narrative | `investigations/` | -| Plan | design doc for non-trivial work | `plans/` | -| Agent | enrolled client identity = its age pubkey | `inventory.yaml` + `.sops.yaml` | - -## Risk classes (enforced, not advisory) - -From [oikos/policy.yaml](oikos/policy.yaml): - -- **read_only** — status, logs, docs, inventory. Unattended. -- **reversible_low** — restart, cache clear, sync pull. Unattended + ledger. -- **config_mutation** — tracked-config edits (commit+push, never local), - deploys, upgrades, DNS/ingress changes. Operator approval. -- **destructive** — destroy, format, wipe, rotate, revoke. Approval + - typed confirmation phrase. - -Lifecycle gates modify these: `provisioning` nodes are freely mutable -(nothing depends on them); `deprecated` nodes accept no new dependents; -anything touching a `destroyed` node is drift. - -## The systems model - -Eight domains — physical, compute, network, storage, software, -identity & access, operations, external — cover everything in the lab; -entities are connected by typed edges (`hosts`, `provides`, `mounts`, -`stores-on`, `routes-to`, `can-decrypt`, `depends-on`, `backs-up-to`, …) -defined in [oikos/ontology.yaml](oikos/ontology.yaml). Rule of -completeness: **if it can break, be changed, or hold data, it has an -entity and edges.** Blast-radius questions ("what breaks if strong goes -down?") are graph walks, not doc archaeology. - -Nodes move through an explicit lifecycle — -`planned → provisioning → active → migrating → deprecated → destroyed` — -stored as `state:` in inventory (absent = active). Destroyed nodes live in -the `archaeology:` section. Each transition is a runbook checklist; -deprecation completes only when inbound edges reach zero. - -Generated views: [infrastructure/topology.md](infrastructure/topology.md) -(Mermaid, regenerated from inventory) and the live, clickable version at -`oikos.hubris.network/graph` once the Console is deployed. - -## Conventions carried forward - -- Inventory is the truth; live state wins over narrative docs. -- Prefer `homelab` CLI and MCP over ad-hoc SSH. -- Meaningful changes update docs in the same session. -- Secrets are decrypted locally via per-client keys; never into docs/comments. -- Tracked configs change by commit + push, not local edits. -- Netbird is the preferred mesh path for new traffic. -- Agents are terse ([CAVEMAN.md](CAVEMAN.md)), verify claims, and fix - collateral drift when found. - -## Build status (30-day roadmap, started 2026-07-05) - -- **Week 1**: policy, ontology, service contract, archaeology, topology - generator, this brief. Shipped. -- **Week 2**: context cards, `homelab service <name> …`, change ledger, - `node relations`, runbooks. Shipped. -- **Week 3**: ops scheduler + state cache (`homelab service <name> health` - is cache-first, `--live` forces a probe), drift detectors, signal engine - (`homelab signal …`), decision classifier (`homelab decide …`), approval - engine (`homelab approval …` — shared-HMAC grants; Matrix delivery is - Hermes's existing `@dtoro:avispero` send path, not a new bot, see - `oikos/approve.py`), daily brief + weekly report (`oikos/report.py`). - Shipped, except: Prometheus is still `planned` (see - [plans/2026-07-05-oikos-prometheus-lxc.md](plans/2026-07-05-oikos-prometheus-lxc.md)) — - trend signals (disk-full prediction, temp creep) wait on that LXC; the - scheduler's disk check today is point-in-time only, and CPU/NVMe - temperature isn't probed at all yet (no confirmed sensor path on - hubris/strong). DNS-vs-inventory and generic tracked-config-cleanliness - drift checks are also deferred (see `oikos/drift.py` docstring). -- **Week 4**: Oikos Console v0 shipped — signals landing page, service - grid + detail, node/blast-radius view, live Mermaid graph, drift view, - approvals queue (approve/deny, destructive confirmation-phrase - enforced), daily/weekly reports. Server-rendered FastAPI + Jinja2, no - SPA build chain, tested end-to-end against live production data (see - `oikos/console/`). Deploys as a third webhook on `dtoro/Homelab-Docs` - (`/opt/oikos-console`, port :9831) — see - [oikos/console/deploy/README.md](oikos/console/deploy/README.md) for - the Caddy route and Gitea webhook registration this repo can't do for - itself. Approval grants are now single-use (a second `check_grant` call - for the same request fails even within the TTL) and already exact-bound - to request id + entity + action. - **Not shipped as originally planned:** per-agent *age-key-signed* - request authentication — age has no signing primitive (it's an - encryption-only keypair format), so "age-key-signed" wasn't - buildable as stated. The real alternative (SSH-key signing via - `ssh-keygen -Y sign`/`-Y verify`, using each host's already-provisioned - SSH key) is real and buildable, but needs SSH public keys recorded in - inventory first — not there today. Moved to the 60/90-day backlog. - Authentik step-up re-auth on the approve/deny route is documented but - needs a live Authentik instance to configure — also backlog. - Docs pass done (this file, AGENTS.md, operations/commands.md); found - and fixed two more stale references while at it (DNS section still - pointed at destroyed LXC 124/dnsmasq instead of Technitium on 107, and - a `claudio-monitor` reference that's been deprecated since 2026-06-04). - -### Real drift found while building Week 3 (unresolved, needs operator action) - -The drift detectors surfaced genuine, currently-true findings on first -run against production — recorded here rather than silently fixed, since -each is a `config_mutation`/`destructive`-class decision: - -- `republic-laptop` has no `age_pubkey:` in `inventory.yaml`, but its real - age key is granted on nearly every shared secret in `.sops.yaml` - (`age1vf8h7...`) — the enrollment write-back to inventory never - happened. Fix: `homelab client add republic-laptop --finalize-pubkey - age1vf8h7s8mqsn2q5eadgpdupsj4mwn8zguc77d85ws3xj40sl9rgksx2rxw6`. -- `grimmory` has an `age_pubkey` in inventory but is missing from - `secrets/hello.yaml`'s recipient list — incomplete enrollment the - other direction. Fix: re-run `homelab client add grimmory - --finalize-pubkey <its key>`. -- `pve_id 131` exists live on hubris (`pct list`) with no inventory entry - — investigate before assuming it's a stale ID (see the Prometheus LXC - plan doc above, which flags this explicitly). -- Three `lifecycle-pve-id-reuse` info findings (100, 106, 107 each shared - between an active host and an archaeology entry) — expected/benign ID - reuse after destroy, no action needed. - -## 60/90-day backlog - -Derived from gaps observed while building the 30-day roadmap, not -guesswork. Roughly ordered by what unblocks the most: - -- **Fix the oikos-console deploy webhook's signature mismatch.** Console - is live on apps (105) via a manual `deploy.sh` run, but Gitea webhook - 14's deliveries all 403 with a signature mismatch for a cause not yet - found — the secret is confirmed synced correctly on both sides - (rotated once already to rule out drift). Until fixed, `git push` - doesn't auto-redeploy the console the way it does for homelab-mcp/ - secrets-issuance; re-run `deploy.sh` on apps manually after changes. - See [oikos/console/deploy/README.md](oikos/console/deploy/README.md). -- **SSH-key-signed approval requests.** Replaces the design note in - Week 4: age keys can't sign (encryption-only format), so per-agent - request authentication needs `ssh-keygen -Y sign`/`-Y verify` against - each host's existing SSH key. Blocked on a schema gap: inventory - doesn't record SSH public keys today, only ports/users. First step is - populating that field on enrollment, then wiring `oikos/approve.py` to - require and verify a signature over the request payload. -- **Authentik step-up re-auth** on the Console's `/approvals` POST route - — needs a live Authentik `PromptStage`/reauth flow scoped to that path; - not configurable without a running instance to test against. -- **Prometheus provisioning** (see - [plans/2026-07-05-oikos-prometheus-lxc.md](plans/2026-07-05-oikos-prometheus-lxc.md)) - — unblocks trend signals (disk-full prediction, temp creep) and real - sparklines in the Console; investigate the undocumented `pve_id 131` - on hubris first. -- **CPU/NVMe temperature probing** in the scheduler — needs a confirmed - sensor path on hubris and strong (lm-sensors vs vendor tool) before a - real check can be written; guessing one risks a probe that silently - never fires. -- **DNS-vs-inventory drift check** — compare Technitium zone records - against `services.*.url`/`public_host`; not implemented (`oikos/drift.py` - has no Technitium API wiring yet). -- **Generic tracked-config-cleanliness drift check** — today only caddy's - `/etc/caddy` git-checkout path is hardcoded in `oikos/drift.py`; every - other service with a `config_repo` needs its local checkout path - recorded (a `mutation_path`-style field, same gap Week 1's service - contract flagged but didn't backfill) before this generalizes. -- **Per-service policy overrides** (`oikos/policy.yaml` - `service_overrides`) — schema is ready (caddy/dns already use it); - populate more as specific services turn out to need non-default risk - classes. -- **Incident timeline generator** — stitch ledger + signal history into - a single narrative for `investigations/` entries instead of writing - them by hand. -- **Secret access audit** — who-can-decrypt-what report from - `.sops.yaml` + inventory `age_pubkey`s, extending what - `oikos/drift.py`'s SOPS check already partially does. -- **Restore drills** — exercise `backs-up-to` (once populated) by - actually restoring from a backup target on a schedule, not just - checking freshness. -- **Multi-agent delegation model** — more than one agent acting - concurrently; needs the ledger's `agent` field to carry real identity - (age pubkey, not just hostname) consistently, which it mostly does - already but hasn't been stress-tested with concurrent writers. -- **Grafana** — only if the Console's own Prometheus-backed sparklines - turn out to be insufficient once Prometheus ships. -- **"Generalize later" extraction** — the original decision was personal- - first, generalize-later (see Week 1). Once patterns stabilize, extract - a config-driven Oikos core with no `hubris.network`/`hubris`/`strong` - hardcoding, so it's installable on a different homelab. +The Oikos operating model now lives at [`.agents/OIKOS.md`](.agents/OIKOS.md). diff --git a/README.md b/README.md index 2c0ab21..1e92e05 100644 --- a/README.md +++ b/README.md @@ -84,4 +84,5 @@ When you change a node: ## See also -- [`CONTRIBUTING.md`](CONTRIBUTING.md) — page templates and tone +- [`.agents/shared/page-templates.md`](.agents/shared/page-templates.md) — page templates and tone +- [`.agents/shared/writing-style.md`](.agents/shared/writing-style.md) — prose style, banned vocabulary diff --git a/plans/2026-07-06-adopt-wiki-hq-doc-architecture.md b/plans/2026-07-06-adopt-wiki-hq-doc-architecture.md new file mode 100644 index 0000000..c87b2a0 --- /dev/null +++ b/plans/2026-07-06-adopt-wiki-hq-doc-architecture.md @@ -0,0 +1,164 @@ +# 2026-07-06 — Adopt the wiki-hq documentation architecture + +## Goal + +Reorganize the Homelab-Docs **narrative layer** into the `wiki-hq` +(`/Users/dtoro/Downloads/wiki-hq-main`) documentation model — `sources / wiki / index / log` +plus an `.agents/` separation and a single lint-checkable writing-style standard — **without +breaking** the Oikos machine-readable substrate that reads fixed paths. Keep the existing Mermaid +topology generator (no LikeC4). + +## Context + +Homelab-Docs is already a mature operational-docs system: `inventory.yaml` as source of truth, +generated `hosts/*.yaml`, an MCP server, a `homelab` CLI, and the Oikos OODA kernel (scheduler, +drift, decide, ledger, approvals). Its weakness is on the *narrative* side — the prose layer grew +organically and lacks the discipline wiki-hq shows: + +- No enforceable **writing-style** standard (CONTRIBUTING.md has a one-line "voice" note; CAVEMAN.md governs agent *chat*, not docs). +- **Index/README files are inconsistent** — some pure listings, some prose+tables, some with stale counts. +- **No append-only operations log** for doc maintenance — doc changes are visible only in git. +- **Agent instructions and human content are interleaved** at the repo root (AGENTS.md, OIKOS.md, CAVEMAN.md, CONTRIBUTING.md, GLOSSARY.md alongside `containers/`, `infrastructure/`, …). + +Decision (confirmed with operator): **full structural adoption**, **keep Mermaid**, adopt all four +borrows — writing-style guide, section-index/README pattern, append-only per-area logs, and +agent-instruction separation. + +## Hard constraint: protect the operational substrate + +These paths are read programmatically and **must not move** (see `mcp/server.py`, `bin/homelab`, `oikos/`): + +- `inventory.yaml` (root) — MCP (`mcp/server.py:36`), CLI (`bin/homelab:36`), scheduler, drift, relations, gen-topology. +- `hosts/*.yaml` (root, generated) — `HOSTS_DIR` (`mcp/server.py:37`); written by `mcp/build_host_files.py`; read/written by `bin/homelab`. +- `oikos/` — kernel code, `oikos/cards/` (`explain` tool, `mcp/server.py:38`), `oikos/state.json`. +- `secrets/`, `secrets-issuance/`, `ssh/`, `scripts/`, `tools/`, `vps/`, `bin/`, `bootstrap.sh`. + +Two MCP tools are path-agnostic and survive any narrative reorg: `search_docs` (ripgreps all +`*.md`) and `get_page(path)` (agent supplies a repo-relative path). The **`## Changelog` convention +must stay** on node pages — `get_changelog` (`mcp/server.py:214`) and the ledger parse it. New +`log.md` files are **additive**, not a replacement for per-page changelogs. + +## Current → target structure + +Restructure the **narrative layer only**; leave the substrate in place. + +``` +Homelab-Docs/ + README.md # human landing page (kept; refreshed to new nav) + AGENTS.md # kept at root (conventional discovery path) + + .agents/ # NEW — agent-facing instruction, separated from content + shared/ + writing-style.md # NEW — adapted from wiki-hq (homelab voice) + llm-wiki.md # NEW — the sources/wiki/index/log contract for THIS repo + caveman.md # moved from CAVEMAN.md (agent chat mode) + page-templates.md # moved from CONTRIBUTING.md (page templates) + domains/ + knowledge/schema.md # contract for the current-state wiki + operations/schema.md # contract for runbooks / investigations / plans + skills/ # runbooks reshaped as SKILL.md (Phase 4) + lifecycle-provision-node/SKILL.md + service-health-check/SKILL.md + ... + OIKOS.md # operating-model doc (moved; agent-facing) + + knowledge/ # durable current-state wiki (Feedback/authoritative layer) + index.md # pure listing → section indexes only + log.md # append-only doc-maintenance operations log + GLOSSARY.md # moved from root + sources/ + index.md # flat catalog table + references/ # external docs (from infrastructure/references/) + wiki/ + hosts/ README.md + hubris.md + strong.md + containers/ README.md (= today's index.md) + <id>-<name>.md + vms/ README.md + <id>-<name>.md + infrastructure/ README.md + grouped: network/ identity/ storage/ ingress/ operations/ + + operations/ # operator runbook narrative index + index.md commands.md agent-enrollment.md hermes-agent.md + + investigations/ # incident evidence (Observe sources) + README.md + log.md + YYYY-MM-DD-*.md + archive/ + plans/ # projects / design docs (Act) + README.md + YYYY-MM-DD-*.md + done/ + + # substrate — UNCHANGED (see Hard constraint) + inventory.yaml hosts/*.yaml oikos/ mcp/ secrets/ secrets-issuance/ + ssh/ scripts/ tools/ vps/ bin/ bootstrap.sh +``` + +**hosts/ split:** generated `hosts/*.yaml` stay at root (substrate); only the two *narrative* pages +`hosts/hubris.md` and `hosts/strong.md` move to `knowledge/wiki/hosts/`. This is the one directory +where machine and narrative content currently mix. + +## Consumers to update when narrative paths move + +Same phase as the move: + +- **`inventory.yaml` `doc_page` fields** — per-host pointer; consumed by `bin/homelab` (`die("no doc_page recorded…")` near `bin/homelab:1288`). Repoint to `knowledge/wiki/...`. +- **`oikos/gen-topology.py` / `oikos/gen_topology_lib.py`** — write `infrastructure/topology.md` and link `oikos/cards/` to doc pages. Move `topology.md` under `knowledge/wiki/infrastructure/` and update the output-path constant (`oikos/gen-topology.py:3`, `:49`, `:195`). +- **`oikos/cards/` templates** and `oikos/drift.py` docstring at `oikos/drift.py:218` (`containers/121-caddy.md`) — cosmetic, update for accuracy. +- **Internal cross-links** — repo-wide relative-link rewrite + link-check pass. +- **Same-session update rule** (`CONTRIBUTING.md:125`) — rewrite its path checklist to the new layout. + +`search_docs`, `get_page`, `get_changelog` need **no** code change. + +## Writing-style standard (adapted, not copied) + +Create `.agents/shared/writing-style.md` from +`/Users/dtoro/Downloads/wiki-hq-main/.agents/shared/writing-style/writing-style.md`: + +- **Keep:** banned-vocabulary list (puffers, "leverage/utilize/delve", poetic nouns, promotional adjectives), information-density rule, bold-leading-phrase pattern, tables-over-bullets, `>` blockquotes for caveats, one-sided cross-references, "cross-references are content not catalog". +- **Adjust voice:** wiki-hq mandates strict third-person "no you". Homelab docs are operator runbooks that already use imperative recipes. Allow **imperative/second-person in runbooks, recipes, and skills**; third-person reference voice for node/infrastructure descriptions. State this exception explicitly. +- **Clarify vs CAVEMAN.md:** Caveman governs agent *chat responses*; writing-style governs *committed docs*. No conflict. +- **Make it lint-checkable:** the Oikos lint surface (or a new `.agents/skills/docs-lint`) greps banned words + structural violations. + +## Reconciling "append-only per-area logs" with the Oikos ledger + +- The **Oikos change ledger** (`oikos/ledger.py`) stays authoritative for *infrastructure changes* (who/what/risk/approval/verification). Do not duplicate it. +- **Per-page `## Changelog`** stays (parsed by `get_changelog`). +- New **per-area `log.md`** (`knowledge/log.md`, `investigations/log.md`, `plans/log.md`) records *doc-maintenance operations only* — restructures, source ingests, lint sweeps — in wiki-hq's single-line format `## [YYYY-MM-DD] <op> | <summary>`. + +## Convention: plans always live in `plans/` + +The restructure must codify — in `.agents/domains/operations/schema.md`, `AGENTS.md`, and +`.agents/shared/page-templates.md` — that **any plan or design doc for the Homelab is always +written into the repo `plans/` folder** (`plans/YYYY-MM-DD-slug.md`), never into a scratch/agent +plan path, an ad-hoc location, or a chat message. Agents drafting a plan create the file under +`plans/`, list it in `plans/index.md`, and move it to `plans/done/` on completion. This rule is +stated once in the operations schema and cross-referenced from `AGENTS.md` so every agent sees it +at orientation. + +## Section-index / README pattern + +Every folder gets a `README.md`: 1–3 sentence prose intro, then a single two-column nav table +`| Document | What it covers |`, nothing else — no stale counts, no duplicated prose. Pure-listing +`index.md` files (e.g. `containers/index.md`) become the folder `README.md`. Root `README.md` stays +the human landing page (may exceed the strict pattern). + +## Phased rollout + +Each phase independently valuable, independently verifiable, committed separately. + +1. **Conventions first (no moves).** Add `.agents/shared/writing-style.md`, `llm-wiki.md`; move CAVEMAN.md → `.agents/shared/caveman.md`, CONTRIBUTING.md → `.agents/shared/page-templates.md` (leave thin root stubs pointing to the new locations). Verify: links resolve; MCP untouched. +2. **Agent-instruction separation.** Move OIKOS.md → `.agents/OIKOS.md`; author `.agents/domains/{knowledge,operations}/schema.md`. Keep AGENTS.md at root, pointing at `.agents/`. Verify: AGENTS.md still the discovery entry; `tools/setup-hermes-soul.sh:51` references resolve or are updated. +3. **Narrative move into `knowledge/wiki/`.** Move `containers/`, `vms/`, `infrastructure/`, host narratives; add `knowledge/index.md`, `knowledge/log.md`, `knowledge/sources/`. Same commit: update `inventory.yaml` `doc_page` fields, `oikos/gen-topology.py` output path, all internal links. Verify: link-check clean; CLI `doc_page` lookups work; `gen-topology.py` writes to the new path; MCP `get_page`/`search_docs` return moved pages. +4. **Runbooks → skills.** Reshape `runbooks/*.md` into `.agents/skills/<name>/SKILL.md` (they already carry `risk_class`, `inputs`, `verification` frontmatter — near-compatible). Update runbook-path references. Verify: frontmatter parses; lifecycle transitions still map to `oikos/ontology.yaml`. +5. **Style + README pass.** Apply the writing-style standard and README pattern across narrative pages; add `log.md` entries recording the restructure. Verify: docs-lint clean. + +## Verification (end to end) + +- **Link integrity:** markdown link checker (or `rg` over `](` targets) after each move phase; zero broken relative links. +- **MCP resolves docs:** `HOMELAB_CONTEXT_DIR=<repo>` — `get_page("knowledge/wiki/containers/104-gitea.md")` returns content; `search_docs("gitea")` hits new paths; `get_changelog(<moved page>)` parses. +- **CLI doc_page lookups:** exercise `bin/homelab`'s `doc_page` read against a moved page — no "no doc_page recorded" error. +- **Topology generation:** run `oikos/gen-topology.py` — writes the new `topology.md` path; `oikos/cards/` links point at moved pages. +- **Substrate untouched:** `mcp/build_host_files.py` regenerates `hosts/*.yaml` identically; `git diff` shows no substrate churn. +- **Style lint:** banned-vocabulary grep over `knowledge/` returns no hits; every folder `README.md` matches the intro-plus-nav-table shape. + +## Post-migration + +Update `plans/index.md` to list this plan; on completion move it to `plans/done/`. Record the +restructure in the new `knowledge/log.md`. Update `README.md` and `AGENTS.md` navigation to the new +layout. +```