From b2bfa26f643b6c4245bca13d75f02c72db369013 Mon Sep 17 00:00:00 2001 From: dtoro Date: Tue, 7 Jul 2026 20:09:32 +0200 Subject: [PATCH] plans: comprehensive audit + DB-as-truth architecture --- ...7-07-comprehensive-audit-and-next-steps.md | 364 ++++++++++ plans/2026-07-07-db-as-source-of-truth.md | 625 ++++++++++++++++++ plans/index.md | 2 + 3 files changed, 991 insertions(+) create mode 100644 plans/2026-07-07-comprehensive-audit-and-next-steps.md create mode 100644 plans/2026-07-07-db-as-source-of-truth.md diff --git a/plans/2026-07-07-comprehensive-audit-and-next-steps.md b/plans/2026-07-07-comprehensive-audit-and-next-steps.md new file mode 100644 index 0000000..329e517 --- /dev/null +++ b/plans/2026-07-07-comprehensive-audit-and-next-steps.md @@ -0,0 +1,364 @@ +# 2026-07-07 — Comprehensive audit: stale files, state gaps, and next steps + +**Status:** Planned + +## Executive summary + +Oikos is in a stable intermediate state: Go rewrite (Phases 1–6) is implemented and +running in Docker on mac-mini, the narrative wiki has been restructured into +`knowledge/wiki/`, and the strong migration (Tier 1 + 2) is complete. But the +transition left behind stale artifacts, half-updated plans, and several documentation +gaps that create confusion for agents and operators. + +This plan catalogs every stale file and gap found, assigns ownership, and proposes a +prioritized work queue for the next 2–4 weeks. **No action is taken by this plan +itself** — it is the map from which concrete work items are drawn. + +--- + +## 1. Stale files — catalog + +### 1.1 Python kernel (`oikos/*.py`) — 11 files + +| File | Risk | Verdict | +|------|------|---------| +| `oikos/__init__.py` | Low | Package init. Harmless but unused. | +| `oikos/approve.py` | Low | Superseded by `internal/domain/approval.go`. Remove. | +| `oikos/decide.py` | Low | Superseded by `internal/policy/classify.go`. Remove. | +| `oikos/drift.py` | **Medium** | Superseded by `internal/scheduler/scheduler.go`. References old paths (`containers/121-caddy.md` at L218). Remove. | +| `oikos/ledger.py` | **Medium** | Superseded by DB view + `internal/notifier/notifier.go`. Remove. | +| `oikos/policy.py` | Low | Superseded by `internal/policy/classify.go`. Remove. | +| `oikos/relations.py` | Low | Superseded by `blast_radius()` SQL function + `internal/ontology/`. Remove. | +| `oikos/scheduler.py` | **Medium** | Superseded by `internal/scheduler/`. Referenced by Prometheus plan. Remove. | +| `oikos/signal.py` | Low | Superseded by signals table (migration 003) + `internal/domain/signal.go`. Remove. | +| `oikos/gen-topology.py` | **High** | Still active — writes `knowledge/wiki/infrastructure/topology.md`, consumed by `homelab` CLI. **Keep until ported to Go.** | +| `oikos/gen_topology_lib.py` | **High** | Imported by `gen-topology.py`. **Keep until ported.** | + +**Action:** Remove the 9 superseded `.py` files. Keep `gen-topology.py` + +`gen_topology_lib.py` until a Go equivalent exists. Update `.gitignore` and any +cross-references (Prometheus plan, bin/homelab docs). + +### 1.2 `.hermes/plans/` — 7 files + +These plans live in `.hermes/plans/` but the wiki-hq convention (`plans/index.md` L125, +`.agents/domains/operations/schema.md`) mandates all plans go in `plans/`. The strong +migration assessment (`.hermes/plans/2026-07-05_strong-migration-assessment.md`) is +particularly valuable and has already been executed. + +| File | Status | Action | +|------|--------|--------| +| `2026-06-03_110000-library-ssd-migration-to-ludo-mini.md` | Superseded by strong migration | Move to `plans/done/` | +| `2026-06-03_150000-homelab-structure-revision.md` | Executed (is this repo structure) | Move to `plans/done/` | +| `2026-06-03_223218-dhcp-pool-exclude-static-ips.md` | Executed | Move to `plans/done/` | +| `2026-06-05_170000-prevent-dhcp-ip-drift.md` | Executed | Move to `plans/done/` | +| `2026-06-06_232200-authentik-frequent-login-fix.md` | Executed | Move to `plans/done/` | +| `2026-06-06_234500-caddyfile-truncation-permanent-fix.md` | Executed | Move to `plans/done/` | +| `2026-07-05_strong-migration-assessment.md` | Executed (Phases 1–2d done) | Move to `plans/done/`, add Phase 3 status note | + +### 1.3 `oikos/cards/` — 45 context cards + +These `.md` files are consumed by the Python MCP server's `explain` tool +(`mcp/server.py:38` referenced `CARD_DIR`). The Go MCP server (`internal/mcp/`) now +serves entity context from the database via `get_entity` / `list_entities`. + +| Risk | Verdict | +|------|---------| +| **Medium** | Cards are redundant with DB entities but may still be read by the legacy `bin/homelab` Python CLI for `homelab explain`. If `bin/homelab` is ported to Go (via generated OpenAPI client), these become dead weight. | + +**Action:** Verify whether `bin/homelab explain` reads `oikos/cards/` or queries the +API. If it reads cards: mark them as "keep until homelab CLI ported." If it queries the +API: remove cards and mention in the next Go release. + +### 1.4 `secrets-issuance/` — identity issuance service + +Still running on apps/105 (LXC) as `secrets-issuance.service`. The Docker stack +hasn't taken over this function yet. The cutover checklist (`scripts/cutover-checklist.md:5`) +notes "Disable apps/105 services" as ticked, but secrets-issuance was specifically stopped, +not replaced. + +| Risk | Verdict | +|------|---------| +| **Medium** | If apps/105 is destroyed (cutover cleanup), secrets issuance must be replaced or explicitly decommissioned. | + +**Action:** Decide: (a) port secrets issuance to Docker stack, or (b) decommission it +(age keys can be generated ad-hoc on each client). Document the decision. + +### 1.5 Traefik references + +`knowledge/sources/references/cert-sync-and-traefik-config.md` and +`inventory.yaml:245` (`vps/management.json.tmpl` references traefik) refer to a reverse +proxy that was replaced by Caddy for internal routing. Traefik **still runs on the VPS** +for public termination, so the references may be valid — but the doc is unclear. + +**Action:** Audit traefik references. If VPS traefik is still active, document its +scope (public termination only, not internal). If fully replaced, remove the references. + +--- + +## 2. Plan status drift + +### 2.1 TRMNL (128) — marked "Planned" but done + +`plans/2026-06-24-trmnl-plugins-lxc.md` shows status "Planned" but: +- LXC 128 exists in `inventory.yaml` with IP `192.168.8.211` +- Wiki page `knowledge/wiki/containers/128-trmnl.md` exists +- `hosts/trmnl.yaml` exists +- Caddy route `trmnl.hubris.network` exists +- Service `services.trmnl` exists in `inventory.yaml` + +**Action:** Mark TRMNL plan as "Done", move to `plans/done/`, add changelog entries +on affected pages (as specified in the plan's "Post-migration" section). + +### 2.2 Consolidation plan — cutover incomplete + +`plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md` status says +"Phase 1-6 implemented, pending cutover." The cutover checklist has 5 pending items: + +1. **Infisical bootstrap** — `profiles: [infisical]` in compose but `OIKOS_SECRET_BACKEND` + env not set. SOPS fallback is active. +2. **Watchdog tested** — pending API stop + Matrix alert verification. +3. **Rollback drill** — `scripts/rollback.sh` never rehearsed. +4. **Rollback verify health + re-deploy** — depends on #3. +5. **Cleanup** — apps/105 webhooks not removed, LXC not archived. + +**Action:** These 5 items are the highest-priority operational work. Schedule a 1-hour +window to complete them. + +### 2.3 Prometheus plan — references Python + +`plans/2026-07-05-oikos-prometheus-lxc.md:60` says "Extend `oikos/scheduler.py`'s +disk/temp probes" but the scheduler is now Go (`internal/scheduler/`). The plan also +references `bin/homelab` Python CLI commands for provisioning. + +**Action:** Update the plan to reference the Go check_defs system (migration 003, +`check_defs` table) and the new `homelab` CLI (or `oikos` binary commands). + +--- + +## 3. Documentation gaps + +### 3.1 Missing container wiki pages + +Two containers exist in `inventory.yaml` and `hosts/*.yaml` but have no wiki page: + +| Container | PVE ID | Host | Wiki page | +|-----------|--------|------|-----------| +| seanime | 133 | strong | **Missing** | +| romm | 134 | strong | **Missing** | + +**Action:** Create `knowledge/wiki/containers/133-seanime.md` and +`knowledge/wiki/containers/134-romm.md` from the data in `hosts/seanime.yaml` and +`hosts/romm.yaml`. + +### 3.2 Host pages for `strong.md` and `hubris.md` — stale resource math + +`knowledge/wiki/hosts/hubris.md` and `knowledge/wiki/hosts/strong.md` may still +reflect pre-migration resource allocation. The strong migration moved elementsynapse, +house, arriman, jellyfin, and grimmory — both pages need updated guest lists and +resource math. + +**Action:** Audit both host pages against live `pct list` on each Proxmox node. +Update guest tables and RAM/CPU summaries. + +### 3.3 `knowledge/wiki/infrastructure/topology.md` — Mermaid graph + +Generated by `oikos/gen-topology.py`. Should be verified to include: +- strong and its guests (vmbr1 range 192.168.8.241–249) +- romm (134) and seanime (133) +- teddycloud (131) +- grimmory (130) on strong (not hubris) + +**Action:** Run `oikos/gen-topology.py` and verify topology reflects current state. +Commit the regenerated file. + +### 3.4 No ADR for Go rewrite completion + +ADR-0001 ("Go single binary") was written as a forward-looking decision. No ADR +documents the completed rewrite as an accepted/implemented decision with retroactive +context on why the Python→Go transition happened when it did. + +**Action:** Add an ADR-0011 documenting the completed Go rewrite with dates, phases, +and the rationale for the timing. + +### 3.5 Un-enrolled hosts tracked in inventory + +Several hosts have `age_pubkey: ''` in `inventory.yaml`: +- jellyfin, paperless, gitea, nextcloud, sophia, mule-images, caddy, arriman + +These are documented as "not enrolled" in their host files. This is intentional +(not all LXCs need homelab-context) but the pattern should be documented in the +wiki as a convention. + +**Action:** Add a section to `knowledge/wiki/infrastructure/homelab-context.md` +(or `.agents/operations/agent-enrollment.md`) explaining when enrollment is expected +vs. skipped. + +--- + +## 4. Technical debt — Go codebase + +### 4.1 `bin/homelab` — still a Python shim? + +The `.gitignore` says "oikos/ kernel files are still imported by bin/homelab for +operational CLI commands." If `bin/homelab` is a compiled Go binary (as the README +and ADR-0001 suggest), then the Python kernel files should not be needed. + +**Action:** Verify what `bin/homelab` is (run `file bin/homelab`). If it's a Go +binary and doesn't import Python, the 9 superseded `.py` files in `oikos/` are +safe to delete. + +### 4.2 `oikos/build_hosts.go` — Python equivalent still exists + +`cmd/oikos/build_hosts.go` exists alongside `mcp/build_host_files.py`. Both +generate `hosts/*.yaml` from `inventory.yaml`. Are both active? If the Go version +works, the Python version is stale. + +**Action:** Compare output of both generators. If identical, remove the Python +version and update the `homelab` CLI to use the Go generator. + +### 4.3 `scripts/dns-sync.py` — standalone Python + +This script syncs DNS records to Technitium. It's operational and has no Go +equivalent yet. Not stale, but should be tracked as "port to Go." + +**Action:** Add to Go rewrite backlog as a follow-up item. + +### 4.4 `scripts/validate-seeds.py` — standalone Python + +Validates `seeds/*.yaml`. Could be folded into `oikos seed` as a `--validate` flag. + +**Action:** Add to Go rewrite backlog. Low priority. + +--- + +## 5. Operational backlog (prioritized) + +### P0 — Immediate (this week) + +1. **[Cutover] Complete cutover checklist** — the 5 pending items from `scripts/cutover-checklist.md`: + - Infisical bootstrap (or decide: defer and stay on SOPS) + - Watchdog end-to-end test + - Rollback drill (`scripts/rollback.sh` rehearsal) + - Rollback verify + re-deploy + - apps/105 cleanup (remove Gitea webhooks, archive LXC) + +2. **[Docs] Create wiki pages for seanime (133) and romm (134)** + +3. **[Plans] Mark TRMNL plan as done**, move to `plans/done/`, write changelogs + +### P1 — Near-term (next 1-2 weeks) + +4. **[Cleanup] Migrate `.hermes/plans/` → `plans/done/`** for all 7 files + +5. **[Cleanup] Remove 9 superseded `oikos/*.py` files** (all except gen-topology*) + — after verifying `bin/homelab` doesn't import them + +6. **[Docs] Audit and update strong.md + hubris.md** guest lists and resource math + +7. **[Docs] Regenerate `knowledge/wiki/infrastructure/topology.md`** to reflect + current state (strong guests, new LXCs) + +8. **[Plans] Update Prometheus plan** — port Python references to Go equivalents + +### P2 — Medium-term (2-4 weeks) + +9. **[Docs] ADR-0011** — document the completed Go rewrite + +10. **[Docs] Document enrollment convention** — when LXCs get homelab-context vs. + when they skip it + +11. **[Cleanup] Audit and resolve oikos/cards/** — verify whether still consumed + +12. **[Cleanup] Traefik reference audit** — document or remove old traefik mentions + +13. **[Plans] Complete wiki-hq adoption Phases 3–5** from + `plans/2026-07-06-adopt-wiki-hq-doc-architecture.md`: + - Phase 4 — runbooks → skills (reshape remaining runbooks into `.agents/skills/`) + - Phase 5 — writing-style + README pass + docs-lint + +14. **[Code] Compare `mcp/build_host_files.py` vs `cmd/oikos/build_hosts.go`** + — remove duplicate if outputs match + +### P3 — Backlog (when capacity allows) + +15. **[Code] Port `oikos/gen-topology.py` to Go** — the last Python holdout + +16. **[Code] Port `scripts/dns-sync.py` to Go** — add to `oikos` binary + +17. **[Code] Fold `scripts/validate-seeds.py` into `oikos seed --validate`** + +18. **[Infra] Prometheus LXC provisioning** — per the updated plan + +19. **[Infra] Strong Phase 3** — migrate mule-images (120) per strong migration + assessment + +20. **[Docs] Port `knowledge/sources/references/cert-sync-and-traefik-config.md`** + to reflect current Caddy-based cert management + +--- + +## 6. File inventory — what to keep, what to remove + +### Keep (active) +``` +cmd/oikos/** cmd/hermes/**, internal/**, api/openapi.yaml, compose/**, migrations/**, +seeds/**, docs/adr/**, docker-compose.yml, Makefile, go.mod, go.sum, sqlc.yaml, +inventory.yaml, hosts/*.yaml, .sops.yaml, .gitignore, .gitea/**, +knowledge/**, .agents/**, plans/**, README.md, AGENTS.md, +hermes/**, oikos/gen-topology.py, oikos/gen_topology_lib.py, +scripts/{deploy,rollback,watchdog,verify-phase6}.sh, scripts/sync/**, +scripts/check-caddy-backends.sh, scripts/cutover-checklist.md, +secrets/**, secrets-issuance/**, bin/homelab, mcp/build_host_files.py, +ssh/**, tools/**, vps/**, ledger/**, bootstrap.sh, +scripts/dns-sync.py, scripts/validate-seeds.py (keep until ported) +``` + +### Remove +``` +oikos/__init__.py, oikos/approve.py, oikos/decide.py, oikos/drift.py, +oikos/ledger.py, oikos/policy.py, oikos/relations.py, oikos/scheduler.py, +oikos/signal.py +``` + +### Migrate +``` +.hermes/plans/*.md → plans/done/ +oikos/cards/*.md → TBD (remove if unused, keep if bin/homelab reads them) +``` + +### Decide +``` +secrets-issuance/ → port to Docker or decommission? +oikos/cards/ → still consumed by bin/homelab explain? +mcp/build_host_files.py → redundant with cmd/oikos/build_hosts.go? +``` + +--- + +## 7. Decisions requested + +These require operator input before proceeding: + +1. **Infisical now or later?** Bootstrap Infisical (Phase 5 production) now or defer + and continue with SOPS-only? Risk: Infisical adds Redis + Infisical container to + the Docker stack on mac-mini with non-trivial setup. + +2. **Secrets issuance: port or kill?** If we decommission `secrets-issuance/`, new + clients generate their own age key and the operator adds the pubkey manually. + Simpler but loses automated enrollment. + +3. **apps/105: archive or destroy?** After Docker cutover cleanup, does apps/105 + stay as a warm spare or get archived per lifecycle rules? + +4. **oikos/cards/: keep or drop?** Verify `bin/homelab explain` behavior. If it + queries the API, cards are dead weight. If it reads files, they stay until the + CLI is ported. + +--- + +## Changelog + +### 2026-07-07 — comprehensive audit created +Catalog of 20+ stale files, plan status drift, documentation gaps, and +prioritized operational backlog. Covers Python kernel cleanup, .hermes/plans/ +migration, oikos/cards/ assessment, cutover completion, and missing wiki pages. diff --git a/plans/2026-07-07-db-as-source-of-truth.md b/plans/2026-07-07-db-as-source-of-truth.md new file mode 100644 index 0000000..d1be518 --- /dev/null +++ b/plans/2026-07-07-db-as-source-of-truth.md @@ -0,0 +1,625 @@ +# 2026-07-07 — DB as single source of truth for agent knowledge + +**Status:** Proposed + +## Goal + +Convert `knowledge/wiki/` into DB seeds, archive all original markdown files, and +archive every other file in the repo that the Oikos Go binary does not read. After +this, the DB is the **sole source of truth** for agents — one query surface, no grep +fallback. All old wiki files go to root `archive/`. New knowledge is registered via +the API, and `oikos export` writes it back to seed files for version control. + +## Decisions from operator + +| Question | Decision | +|----------|----------| +| Plans in DB? | No — agent construction docs only | +| Ingestion trigger | Seed-time (`oikos seed` reads `seeds/knowledge.yaml`) | +| Archive location | Root `archive/` | +| Content granularity | Structured sections: At-a-glance → entity attributes, procedures → runbooks, changelog → parsed entries, rest → `knowledge_entities.content` | +| Wiki files after conversion | **Archive them all** — DB is the only truth | +| Stale files not used by Oikos | **Archive them all** | + +## What goes to `archive/` + +Everything the Go binary (`cmd/oikos`, `internal/`) does **not** read at runtime. + +### Full archive list + +| Path | Destination | Why | +|------|-------------|-----| +| `knowledge/wiki/containers/*.md` (20 pages) | `archive/knowledge/containers/` | Converted to seeds | +| `knowledge/wiki/hosts/*.md` (3 pages) | `archive/knowledge/hosts/` | Converted to seeds | +| `knowledge/wiki/infrastructure/*.md` (12 pages) | `archive/knowledge/infrastructure/` | Converted to seeds | +| `knowledge/wiki/vms/*.md` (2 pages) | `archive/knowledge/vms/` | Converted to seeds | +| `knowledge/sources/investigations/*.md` (5 pages) | `archive/knowledge/investigations/` | Converted to seeds | +| `knowledge/sources/references/*.md` (1 page) | `archive/knowledge/references/` | Converted to seeds | +| `knowledge/sources/index.md` | `archive/knowledge/` | Index page, no entity mapping | +| `knowledge/index.md` | `archive/knowledge/` | Index page | +| `knowledge/log.md` | `archive/knowledge/` | Doc-maintenance log, superseded by audit_log | +| `knowledge/GLOSSARY.md` | `archive/knowledge/` | Glossary (can be re-ingested as `document:glossary` later) | +| `oikos/cards/` (45 files) | `archive/oikos-cards/` | Python MCP explain tool; superseded by DB entities | +| `.hermes/plans/` (7 files) | `archive/hermes-plans/` | Agent construction docs, never in DB | +| `ledger/2026-07.jsonl` | `archive/ledger/` | Python-era artifact; Go uses audit_log table | +| `mcp/build_host_files.py` | `archive/mcp/` | Redundant with `cmd/oikos/build_hosts.go` | + +### Full delete list (no archival value) + +| Path | Why | +|------|-----| +| `oikos/approve.py` | Superseded by `internal/domain/approval.go` | +| `oikos/decide.py` | Superseded by `internal/policy/classify.go` | +| `oikos/drift.py` | Superseded by `internal/scheduler/scheduler.go` | +| `oikos/ledger.py` | Superseded by `internal/db/` + `audit_log` table | +| `oikos/policy.py` | Superseded by `internal/policy/classify.go` | +| `oikos/relations.py` | Superseded by `blast_radius()` SQL function | +| `oikos/scheduler.py` | Superseded by `internal/scheduler/scheduler.go` | +| `oikos/signal.py` | Superseded by `signals` table + `internal/domain/signal.go` | +| `oikos/__init__.py` | Package init, never imported by Go binary | + +### What stays (read by Oikos Go binary) + +| Path | Read by | Why keep | +|------|---------|----------| +| `seeds/ontology.yaml` | `oikos seed` | DB bootstrap | +| `seeds/inventory.yaml` | `oikos seed` | DB bootstrap | +| `seeds/policy.yaml` | `oikos seed` | DB bootstrap | +| `seeds/knowledge.yaml` | `oikos seed` | **NEW** — knowledge bootstrap | +| `inventory.yaml` | `oikos build-hosts`, `oikos homelab list` | Topology source of truth | +| `secrets/*.yaml` | `oikos secret list`, `oikos homelab secret` | SOPS-encrypted secrets | +| `migrations/*.sql` | `oikos migrate` | Embedded SQL migrations | +| `api/openapi.yaml` | `make generate` (oapi-codegen) | API contract | +| `compose/`, `Dockerfile`, `docker-compose.yml` | Docker build/deploy | Runtime packaging | +| `cmd/`, `internal/` | `go build` | Source code | +| `bin/homelab` | `oikos homelab` subcommand | CLI binary | +| `oikos/gen-topology.py` | `bin/homelab` (Python shim) | Topology generation (until ported) | +| `oikos/gen_topology_lib.py` | imported by gen-topology.py | Same | +| `scripts/deploy.sh` | Gitea webhook → deploy | Production deploy | +| `scripts/rollback.sh` | Operator manual | Rollback | +| `scripts/watchdog.sh` | cron on mac-mini | Health monitoring | +| `scripts/verify-phase6.sh` | Operator manual | Acceptance verification | +| `scripts/sync/` | systemd/launchd timers | Repo sync to clients | +| `scripts/check-caddy-backends.sh` | Operator diagnostic | Health check utility | +| `scripts/cutover-checklist.md` | Operator reference | Cutover tracking | +| `scripts/dns-sync.py` | Operator manual (not yet ported) | DNS sync to Technitium | +| `scripts/validate-seeds.py` | CI / operator manual | Seed validation | +| `tools/` | `post-pull.sh` auto-setup | Client tooling install | +| `ssh/` | `homelab ssh` | SSH key deployment | +| `vps/` | `homelab render-vps-configs` | VPS templates | +| `secrets-issuance/` | Running on apps/105 | Identity issuance (until decommissioned) | +| `AGENTS.md` | Agent onboarding | Entry point | +| `README.md` | Human onboarding | Entry point | +| `.agents/` | Agent conventions, skills, operating model | Agent runtime guidance | +| `plans/` | Agent construction docs | Intent documents | +| `hermes/` | Hermes config + persona | Agent gateway config | +| `Makefile`, `go.mod`, `go.sum`, `sqlc.yaml`, `.gitignore`, `.sops.yaml`, `.gitea/` | Build/config | Build tooling | + +### Ambiguous — needs confirmation + +| Path | Question | +|------|----------| +| `.agents/` | Contains OIKOS.md (operating model), SKILL.md runbooks, conventions. Go binary doesn't read these at runtime — they're agent orientation docs. Should they also be converted to seeds and archived? Or do they stay as the "agent manual" alongside AGENTS.md? | +| `scripts/dns-sync.py`, `scripts/validate-seeds.py` | Actively used Python scripts with no Go equivalent. Keep until ported? | +| `secrets-issuance/` | Running on apps/105. Archive after decommission? | + +## Architecture change + +``` +BEFORE (split): + knowledge/wiki/ → agents grep clone for narrative + Postgres DB → agents query MCP for structured data + Two surfaces, no bridge + +AFTER (DB-only): + seeds/knowledge.yaml → oikos seed → knowledge_entities table + ↓ + MCP search_knowledge("jellyfin transcode") + MCP get_entity_knowledge("lxc:jellyfin") + HTTP GET /api/v1/knowledge/search + ↓ + One surface, everything indexed +``` + +``` +┌──────────────────────────────────┐ +│ seeds/knowledge.yaml │ Version-controlled source material +│ (generated by oikos export) │ Round-trip: seed → DB → export → seed +│ documents: │ +│ - slug: containers/101-jellyfin│ +│ entity: lxc:jellyfin │ +│ title: "101 — jellyfin" │ +│ content: "..." │ +│ at_glance: {host: strong, ..}│ +│ changelog: [{date, title, ..}]│ +│ tags: [container, media, ..] │ +│ investigations: │ +│ - slug: 2026-06-06-caddy-... │ +│ about: [service:caddy, ...] │ +│ ... │ +│ runbooks: │ +│ - slug: service-health-check │ +│ procedure_for: [service] │ +│ ... │ +└──────────────┬───────────────────┘ + │ oikos seed + ▼ +┌──────────────────────────────────┐ +│ knowledge_entities table │ +│ title | content | content_hash │ +│ source | tags │ +│ ─ FTS: ts_vector(title+content) │ +│ │ +│ + entities table: │ +│ document:containers/101-jellyfin│ +│ investigation:2026-06-06-... │ +│ runbook:service-health-check │ +│ │ +│ + relationships: │ +│ document ──documents──► lxc │ +│ investigation ──about──► service│ +│ runbook ──procedure-for──► type│ +└──────────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────┐ +│ MCP + HTTP API │ +│ search_knowledge("jellyfin") │ +│ get_entity_knowledge("lxc:...") │ +│ POST /api/v1/knowledge/{id} │ +└──────────────────────────────────┘ +``` + +## Execution — Fresh install on mac-mini + +The current Docker stack on mac-mini was deployed during cutover testing with dev data. +Replace it with a fresh production instance using the real seeds (ontology, inventory, +policy, knowledge). + +### Steps + +```bash +# On mac-mini (192.168.8.175): + +# 1. Stop current stack +docker compose --profile full down + +# 2. Wipe old Postgres volume (fresh start) +docker volume rm oikos_pg-data + +# 3. Pull latest repo (already committed to local main) +git pull origin main + +# 4. Rebuild image with current code +docker compose build + +# 5. Fresh migrate + seed (now includes knowledge) +docker compose run --rm migrate +docker compose run --rm seed + +# 6. Verify seed ingested +docker compose run --rm -e OIKOS_DATABASE_URL=... oikos \ + psql -c "SELECT count(*) FROM knowledge_entities;" +# Expect: 54+ rows + +# 7. Start production stack +docker compose --profile full up -d + +# 8. Health check +curl http://localhost:8090/healthz +./scripts/verify-phase6.sh + +# 9. Verify knowledge endpoints +curl "http://localhost:8090/api/v1/knowledge/search?q=jellyfin" +``` + +After fresh install succeeds, the old `backups/pre-cutover-20260707.sql` dump becomes +stale (it's from the dev instance). The new instance is the canonical production DB. + +--- + +## Phase 1 — Seed format + ingestion + +### 1.1 Add `content_hash` to `knowledge_entities` + +**Migration 010:** + +```sql +ALTER TABLE knowledge_entities ADD COLUMN content_hash TEXT; +``` + +### 1.2 Seed format: `seeds/knowledge.yaml` + +One YAML file with three sections. Ingested by `oikos seed` after ontology, inventory, +and policy. + +```yaml +# Oikos knowledge seed — documents, investigations, and runbooks. +# Generated by oikos export. Ingested on deploy. +# After ingest the DB is authoritative. + +version: 1 + +documents: + - slug: "containers/101-jellyfin" + title: "101 — jellyfin" + content: | + # 101 — jellyfin + + Jellyfin media server with hardware-accelerated transcoding... + + ## Service / port + | Service | Port | Notes | + |---------|------|-------| + | jellyfin | 8096 | Web UI + API | + ... + + ## SSO + Authentik OIDC via SSO-Auth plugin v4.0.0.4. No Caddy forward-auth gate. + ... + entity_slug: "lxc:jellyfin" # links via `documents` edge + tags: ["container", "media", "jellyfin"] + at_glance: # structured: pushed to entity attributes if empty + host: strong + ip: "192.168.8.246" + cores: 4 + ram: "8 GiB" + mounts: ["/mnt/media_local"] + public_host: media.hubris.network + changelog: + - date: "2026-07-05" + title: "migrated from hubris to strong" + body: "Phase 2 of strong migration..." + - date: "2026-06-10" + title: "VAAPI configured on Radeon 680M" + body: "GPU passthrough via dev0 + dev1..." + + - slug: "hosts/strong" + title: "strong — Proxmox host" + content: | + # strong — Proxmox host + Reformatted from Linux workstation to Proxmox VE 9.2.3... + entity_slug: "proxmox-host:strong" + tags: ["host", "proxmox", "hypervisor"] + at_glance: + role: hypervisor + hardware: "Minisforum UM773 Lite (Ryzen 7 PRO 6850U, 28 GiB RAM)" + os: linux + changelog: + - date: "2026-07-05" + title: "strong migration Phase 2 complete" + body: "arriman + jellyfin + grimmory migrated. ludo-lvm..." + + # ... 35+ more document entries + +investigations: + - slug: "2026-06-06-caddyfile-truncation" + title: "Caddyfile truncation — all LAN services down" + date: "2026-06-06" + status: resolved + duration: "~45 min" + content: | + ## Symptom + All *.hubris.network services returned 502... + ## Root cause + ... + about_slugs: ["service:caddy", "service:dns"] # links via `about` edges + tags: ["incident", "caddy", "dns", "downtime"] + + - slug: "2026-06-06-authentik-session-lifetime" + title: "Frequent Authentik login prompts" + date: "2026-06-06" + status: resolved + duration: "~30 min" + content: | + ## Summary + ... + about_slugs: ["service:authentik"] + tags: ["incident", "authentik", "oidc"] + + # ... 3 more investigation entries + +runbooks: + - slug: "service-health-check" + name: "Service health check" + risk_class: read_only + entity_type: "service" # applies to all service entities + procedure: + params_schema: + type: object + properties: + unit: {type: string} + required: ["unit"] + steps: + - name: "check systemd unit" + runner: ssh + command: "systemctl is-active {{ .unit }}" + timeout_s: 10 + verify: + - runner: ssh + command: "systemctl is-active {{ .unit }}" + expect: {exit_code: 0, stdout_contains: "active"} + + - slug: "lifecycle-provision-node" + name: "Provision new node" + risk_class: config_mutation + entity_type: "lxc" + procedure: + params_schema: + type: object + properties: + name: {type: string} + template: {type: string} + cores: {type: integer} + memory_mb: {type: integer} + required: ["name", "template"] + steps: + - name: "create container" + runner: ssh + target: "{{ .host }}" + command: "pct create $(pvesh get /nextid) {{ .template }} ..." + timeout_s: 120 + verify: [...] + + # ... 10 more runbook entries +``` + +### 1.3 Ingest logic + +`oikos seed` already runs: migrate → ontology → inventory → policy. + +Add knowledge as step 5: + +``` +oikos seed + 1. migrate (DDL) + 2. ontology seed (entity_types, relationship_types, lifecycles) + 3. inventory seed (entities, relationships) + 4. policy seed (risk_classes, approval_rules, autonomy) + 5. knowledge seed (documents, investigations, runbooks) ← NEW +``` + +Knowledge ingest for each section: + +**Documents:** +- For each entry, create `document:` entity in `entities` table (type: `document`) +- Insert into `knowledge_entities` (title, content, source=slug, tags, content_hash) +- Create `documents` edge from `document:` to `entity_slug` +- Push `at_glance` fields into the linked entity's `attributes` if those keys are empty (backfill) +- Store `changelog` as JSONB in document entity `attributes.changelog` +- Skip if `content_hash` matches existing row (idempotent) + +**Investigations:** +- Create `investigation:` entity (type: `investigation`, lifecycle: `infrastructure`) +- Insert into `knowledge_entities` +- Create `about` edges to each entity in `about_slugs` +- Store `date`, `status`, `duration` in entity `attributes` + +**Runbooks:** +- Create `runbook:` entity (type: `runbook`, lifecycle: `skill`) +- Insert into `knowledge_entities` +- Create `procedure-for` edges to `entity_type` (abstract type — applies to all instances) +- If a `skills` row with matching slug already exists, only update `knowledge_entities.content` and `content_hash` (don't overwrite the structured `procedure` JSONB in the skills table — that's managed separately) + +### 1.4 Export: DB → seed + +`oikos export` already writes `seeds/ontology.yaml`, `seeds/inventory.yaml`, +`seeds/policy.yaml`. Add `seeds/knowledge.yaml`: + +```go +// internal/db/export.go +func ExportKnowledge(ctx context.Context, pool *pgxpool.Pool) ([]byte, error) { + // SELECT all document + investigation + runbook entities + // JOIN knowledge_entities for content + // JOIN relationships for entity links + // Serialize to seeds/knowledge.yaml format +} +``` + +Round-trip guarantee: `seed → DB → export → seed` is byte-stable. Tested in CI. + +## Phase 2 — Convert wiki to seeds, archive originals + +### 2.1 Conversion script (one-time) + +``` +oikos knowledge convert [--dry-run] +``` + +- Reads every page in `knowledge/wiki/`, `knowledge/sources/`, `knowledge/GLOSSARY.md` +- Parses structured sections (at-glance, changelog, procedures) +- Maps each page to entity slugs using the path convention + `inventory.yaml` doc_page +- Generates `seeds/knowledge.yaml` +- Reports: "would write N documents, M investigations, K runbooks" +- Without `--dry-run`: writes `seeds/knowledge.yaml` + +Implementation: initially a Go version in `cmd/oikos/`, can fall back to a one-shot +Python script if that's faster to build. + +### 2.2 Archive + +After conversion verified (seeds ingest cleanly): + +```bash +# Create archive directory +mkdir -p archive/knowledge/{containers,hosts,infrastructure,vms,investigations,references} + +# Move all knowledge files +mv knowledge/wiki/containers/*.md archive/knowledge/containers/ +mv knowledge/wiki/hosts/*.md archive/knowledge/hosts/ +mv knowledge/wiki/infrastructure/*.md archive/knowledge/infrastructure/ +mv knowledge/wiki/vms/*.md archive/knowledge/vms/ +mv knowledge/sources/investigations/*.md archive/knowledge/investigations/ +mv knowledge/sources/references/*.md archive/knowledge/references/ +mv knowledge/index.md knowledge/log.md knowledge/GLOSSARY.md archive/knowledge/ + +# Move other stale artifacts +mv oikos/cards/ archive/oikos-cards/ +mv .hermes/plans/*.md archive/hermes-plans/ +mv ledger/2026-07.jsonl archive/ledger/ +mv mcp/build_host_files.py archive/mcp/ + +# Delete superseded Python +rm oikos/__init__.py oikos/approve.py oikos/decide.py oikos/drift.py \ + oikos/ledger.py oikos/policy.py oikos/relations.py oikos/scheduler.py \ + oikos/signal.py + +# Clean empty directories +rmdir knowledge/sources/investigations/archive/ # already moved +rmdir knowledge/sources/investigations/ +rmdir knowledge/sources/references/ +rmdir knowledge/sources/ +rmdir knowledge/wiki/containers/archive/ # already moved +rmdir knowledge/wiki/containers/ +rmdir knowledge/wiki/hosts/ +rmdir knowledge/wiki/infrastructure/ +rmdir knowledge/wiki/vms/ +rmdir knowledge/wiki/ +``` + +### 2.3 What the repo looks like after + +``` +/ +├── seeds/ +│ ├── ontology.yaml +│ ├── inventory.yaml +│ ├── policy.yaml +│ └── knowledge.yaml ← NEW: documents + investigations + runbooks +├── archive/ +│ ├── knowledge/ ← all old wiki markdown +│ ├── oikos-cards/ ← 45 Python explain cards +│ ├── hermes-plans/ ← 7 agent construction docs +│ ├── ledger/ ← Python-era JSONL +│ └── mcp/ ← old Python build_host_files.py +├── .agents/ ← agent conventions + skills (stays) +├── plans/ ← construction plans (stays) +├── cmd/, internal/ ← Go source +├── migrations/, api/, compose/ +├── inventory.yaml, hosts/*.yaml +├── AGENTS.md, README.md +└── ... +``` + +The repo shrinks by ~65 narrative files and ~45 card files. All knowledge is in +`seeds/knowledge.yaml` — one file, git-tracked, round-trippable. + +## Phase 3 — Un-stub MCP + HTTP + +Same as previous revision: + +- `search_knowledge(query)` → PostgreSQL FTS with `ts_rank`, `ts_headline` +- `get_entity_knowledge(slug)` → aggregates documents, investigations, runbooks linked to entity +- `GET /api/v1/knowledge/search` → FTS endpoint +- `GET /api/v1/knowledge/{slug}` → entity-linked knowledge +- `POST /api/v1/knowledge/{slug}` → agent registers new knowledge +- `PATCH /api/v1/knowledge/{slug}` → agent updates knowledge + +## Phase 4 — Agent conventions + +### Knowledge registration + +Agents call `POST /api/v1/knowledge/{entity_slug}` when they: +- Discover an undocumented detail (e.g., config quirk, gotcha) +- Complete an investigation +- Create a new entity and need to attach docs +- Update entity attributes discovered during operation + +### Knowledge query (single call pattern) + +```python +# One call replaces grep + get_entity + get_relations +ctx = mcp.get_entity_knowledge("lxc:jellyfin") +# Returns: +# - entity attributes (structured) +# - documents (full-text indexed narrative) +# - investigations (any incidents involving this entity) +# - runbooks (procedures that apply to this entity type) +# - relationships (blast radius, dependencies) +``` + +### Export cycle + +``` +agent discovers → POST /api/v1/knowledge → DB updated +operator reviews → oikos export → seeds/knowledge.yaml updated +git commit + push → version-controlled, diffable +next deploy → oikos seed → DB back in sync +``` + +## Files to create/modify + +| File | Action | +|------|--------| +| `migrations/010_knowledge_hash.up.sql` | New — `content_hash TEXT` | +| `internal/knowledge/seed.go` | New — knowledge seed ingest (read seeds/knowledge.yaml → DB) | +| `internal/knowledge/seed_test.go` | New — round-trip tests | +| `internal/knowledge/convert.go` | New — one-shot wiki→seed converter | +| `internal/knowledge/parser.go` | New — section parser (at-glance, changelog) | +| `internal/db/seed.go` | Modify — add knowledge step after policy | +| `internal/db/export.go` | Modify — add `ExportKnowledge` | +| `internal/httpapi/impl.go` | Modify — un-stub knowledge endpoints | +| `internal/httpapi/knowledge.go` | New — knowledge handlers | +| `internal/mcp/server.go` | Modify — un-stub `search_knowledge`, `get_entity_knowledge` | +| `cmd/oikos/main.go` | Modify — add `knowledge convert` subcommand | +| `api/openapi.yaml` | Modify — knowledge endpoint schemas | +| `seeds/knowledge.yaml` | New — generated by converter, ingested on seed | +| `archive/` | New — all old wiki + stale files | +| `plans/2026-07-07-comprehensive-audit-and-next-steps.md` | Modify — mark knowledge items as addressed | + +## Verification + +```bash +# 1. Convert wiki to seed +oikos knowledge convert --dry-run +# → "would write 37 documents, 5 investigations, 12 runbooks" +oikos knowledge convert + +# 2. Fresh DB, full seed +oikos migrate && oikos seed + +# 3. Verify knowledge populated +psql -c "SELECT title, source, cardinality(tags) FROM knowledge_entities;" +# → 54 rows (37 docs + 5 investigations + 12 runbooks) + +# 4. Verify FTS +curl "http://localhost:8090/api/v1/knowledge/search?q=jellyfin+transcode" +# → returns document:containers/101-jellyfin with ts_headline snippet + +# 5. Verify entity-linked knowledge +curl "http://localhost:8090/api/v1/knowledge/lxc:jellyfin" +# → {entity, documents: [...], investigations: [], runbooks: [...], relationships: {...}} + +# 6. Verify MCP +curl -X POST localhost:8092/query \ + -d '{"tool":"search_knowledge","args":{"query":"caddy truncation"}}' +# → returns investigation:2026-06-06-caddyfile-truncation + +# 7. Verify agent registration +curl -X POST localhost:8090/api/v1/knowledge/lxc:jellyfin \ + -H "Content-Type: application/json" \ + -d '{"title":"VAAPI note","content":"Requires /dev/dri/renderD128 passthrough","tags":["gpu"]}' +# → 201 Created + +# 8. Verify export round-trip +oikos export # writes seeds/*.yaml +oikos seed # re-ingests (skips all — hashes match) +# → "knowledge: ingested 0, skipped 54, errors 0" + +# 9. Archive originals (manual, after verification) +# → move all wiki files to archive/ as listed in Phase 2.2 + +# 10. Verify repo still builds and deploys +make generate && make build && make test +docker compose --profile dev up -d +./scripts/verify-phase6.sh +``` + +--- + +## Changelog + +### 2026-07-07 — v2: full archive, wiki→seeds only +Convert wiki to seeds, archive ALL originals, archive all stale files. No human-editing +frontend retained — DB is the only source of truth. Knowledge cycle: agent API → +oikos export → seeds/knowledge.yaml → git → oikos seed → DB. \ No newline at end of file diff --git a/plans/index.md b/plans/index.md index bed7eff..b0f7788 100644 --- a/plans/index.md +++ b/plans/index.md @@ -12,6 +12,8 @@ went sideways, open an investigation. | 2026-07-05 | [Oikos Prometheus LXC](2026-07-05-oikos-prometheus-lxc.md) | Planned | | 2026-07-06 | [Adopt wiki-hq doc architecture](2026-07-06-adopt-wiki-hq-doc-architecture.md) | In Progress | | 2026-07-06 | [Consolidate Oikos control plane onto mac-mini](2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md) | In Progress (Phase 1-6 implemented, pending cutover) | +| 2026-07-07 | [Comprehensive audit: stale files, state gaps, and next steps](2026-07-07-comprehensive-audit-and-next-steps.md) | Planned | +| 2026-07-07 | [DB as single source of truth for agent knowledge](2026-07-07-db-as-source-of-truth.md) | Proposed | ## Done