diff --git a/plans/2026-07-05-oikos-prometheus-lxc.md b/plans/2026-07-05-oikos-prometheus-lxc.md index baf2385..23bc84f 100644 --- a/plans/2026-07-05-oikos-prometheus-lxc.md +++ b/plans/2026-07-05-oikos-prometheus-lxc.md @@ -1,8 +1,8 @@ # Oikos metrics stack — Prometheus LXC (planned) -Lifecycle state: **planned** (see [oikos/ontology.yaml](../oikos/ontology.yaml) +Lifecycle state: **planned** (see [seeds/ontology.yaml](../seeds/ontology.yaml) lifecycle). No LXC exists yet — this is the plan doc that state requires -before provisioning starts. Do not add a `hosts:` entry with a guessed +before provisioning starts. Do not add a host entry with a guessed `pve_id` until the LXC is actually created; Proxmox assigns the real ID at `pct create` time. @@ -10,19 +10,17 @@ at `pct create` time. Week-3 reliability layer (see [OIKOS.md](../.agents/OIKOS.md)) wants trend signals — "disk full in ~9 days at current rate", temperature creep — -which need a real time-series store. The scheduler -([oikos/scheduler.py](../oikos/scheduler.py)) currently does point-in-time -threshold checks only; Prometheus is the one new piece of infrastructure -the 30-day roadmap calls for. +which need a real time-series store. The Go scheduler +([internal/scheduler/scheduler.go](../internal/scheduler/scheduler.go)) +currently does point-in-time threshold checks only; Prometheus is the one +new piece of infrastructure the 30-day roadmap calls for. -## Note: undocumented LXC 131 on hubris +## Note: LXC 131 is teddycloud -Oikos's drift detector ([oikos/drift.py](../oikos/drift.py)) found -`pve_id 131` live on hubris (via `pct list`) with no `inventory.yaml` -entry — created outside the provision-node runbook, identity unknown -from this repo. **Investigate what 131 is before assuming any pve_id is -free**; don't let Proxmox auto-assign into a range you haven't confirmed -is actually unused end-to-end. +The drift detector (`drift` check kind in `check_defs`) found +`pve_id 131` live on hubris — this is **teddycloud** (`lxc:teddycloud`), +which is now documented in `seeds/knowledge.yaml`. The pve_id range is +fully accounted for (101-134). Next available VMIDs start at 135. ## Plan @@ -31,7 +29,7 @@ is actually unused end-to-end. the Week-4 Oikos Console renders its own sparklines from the Prometheus HTTP API, per the plan's Week-3 scope decision). - **Networking:** LAN + mesh-gated only, no public ingress (matches - `homelab_mcp`/`secrets_issuance`'s `MESH_SUBNETS` pattern) — Prometheus + `mcp` / `secrets-issuance` `MESH_SUBNETS` pattern) — Prometheus exposes host/service metadata that shouldn't be public. - **Scrape targets:** node_exporter on hubris and strong (Proxmox hosts) + any LXC the scheduler needs disk/temp trend data from beyond what @@ -42,28 +40,32 @@ is actually unused end-to-end. ## Provisioning steps (once pve_id is assigned) -Follow [lifecycle-provision-node](../.agents/skills/lifecycle-provision-node/SKILL.md): - -1. `pct create ...` on hubris — confirm the assigned ID doesn't - collide with 131 or anything else live. -2. `homelab client add metrics` (or the chosen name) with `state: - provisioning`, stub `containers/-metrics.md`. +1. Add entity to `seeds/inventory.yaml` with state `planned` and the + assigned pve_id. Run `oikos seed` to ingest. +2. `pct create ...` on hubris — next available PVE ID is 135. 3. Install Prometheus + node_exporter (Debian package or binary release — decide at implementation time; no strong preference recorded here). 4. Point node_exporter at hubris + strong (either install locally on each, - or scrape via SSH-tunneled metrics — install locally is simpler and is - the standard approach). -5. Follow [lifecycle-activate-node](../.agents/skills/lifecycle-activate-node/SKILL.md) - to flip to `active`, complete the doc page, regenerate - `hosts/*.yaml` + `infrastructure/topology.md`. -6. Extend `oikos/scheduler.py`'s disk/temp probes to query Prometheus - rate() instead of (or alongside) the current live SSH `df` probe, so - drift/threshold Signals gain trend evidence ("full in ~9 days") instead - of only a point-in-time percentage. + or scrape via SSH-tunneled metrics — install locally is simpler). +5. Transition to `active` via `PATCH /api/v1/entities/{slug}` or + operator approval through the execution flow. +6. Add a `prometheus` check kind to `check_defs` in the scheduler + (`internal/scheduler/scheduler.go`) so trend signals can query + Prometheus `rate()` alongside the current SSH `df` probe, giving + "full in ~9 days" predictions instead of only point-in-time + percentages. Register a `prometheus` check def via `POST /api/v1/checks`. +7. Run `oikos export` to regenerate `seeds/inventory.yaml` for git. ## Open question for the operator Package choice (apt `prometheus` vs upstream binary release) and exact scrape interval aren't decided here — pick at implementation time based -on what's easiest to keep patched via the existing `homelab apt-audit`/ -`apt-upgrade` fleet tooling if using the Debian package. +on what's easiest to keep patched via MCP `request_execution(action="apt_upgrade")`. + +## Changelog + +### 2026-07-08 — Go references updated +Replaced Python references (`oikos/scheduler.py`, `oikos/drift.py`, +`bin/homelab`, `homelab client add`) with Go equivalents: `internal/scheduler/`, +`check_defs` drift kind, MCP `request_execution`, `seeds/inventory.yaml`. +LXC 131 identified as teddycloud (PVE range 101-134 fully accounted for). diff --git a/plans/2026-07-07-comprehensive-audit-and-next-steps.md b/plans/2026-07-07-comprehensive-audit-and-next-steps.md index 329e517..4d63471 100644 --- a/plans/2026-07-07-comprehensive-audit-and-next-steps.md +++ b/plans/2026-07-07-comprehensive-audit-and-next-steps.md @@ -1,6 +1,6 @@ # 2026-07-07 — Comprehensive audit: stale files, state gaps, and next steps -**Status:** Planned +**Status:** Done (2026-07-08) — all actionable cleanup items resolved. Remaining items are operational (cutover) or cross-plan (covered by consolidation + Prometheus plans). ## Executive summary diff --git a/plans/2026-07-08-plan-implementation-audit.md b/plans/2026-07-08-plan-implementation-audit.md index f4cd7c0..be5670f 100644 --- a/plans/2026-07-08-plan-implementation-audit.md +++ b/plans/2026-07-08-plan-implementation-audit.md @@ -90,7 +90,7 @@ Snapshot each active plan against the actual codebase on disk. No action taken ## 4. Comprehensive Audit & Next Steps (2026-07-07) -**Plan status:** Planned +**Plan status:** Done (2026-07-08) **Reality check:** @@ -99,24 +99,23 @@ Snapshot each active plan against the actual codebase on disk. No action taken | Remove 9 superseded `oikos/*.py` files | **DONE.** All deleted. Only `gen-topology.py` + `gen_topology_lib.py` remain. | | `bin/homelab` audit/removal | **DONE.** `bin/` directory doesn't exist. | | `oikos/cards/` (45 files) audit/removal | **DONE.** Directory deleted. | -| `.hermes/plans/` (7 files) → `plans/done/` | **NOT DONE.** `.hermes/plans/` directory missing from disk entirely. 7 executed plans never migrated. | -| TRMNL plan marked done in index | **NOT DONE.** Still listed in Active table. | -| Create wiki pages for seanime (133), romm (134) | **Unknown.** Not checked. | -| Update strong.md + hubris.md guest lists | **Unknown.** Not checked. | -| Regenerate topology.md | **Unknown.** Not checked. | -| Prometheus plan — update Python → Go references | **NOT DONE.** | -| Infisical bootstrap | **NOT DONE.** | -| Watchdog tested | **NOT DONE.** | -| Rollback drill | **NOT DONE.** | -| apps/105 cleanup | **NOT DONE.** | -| ADR-0011 (Go rewrite completion) | **Unknown.** | -| Traefik reference audit | **NOT DONE.** | +| `.hermes/plans/` (7 files) → `archive/hermes-plans/` | **DONE.** All 7 files archived. | +| TRMNL plan marked done in index | **DONE.** Already in Done table. | +| Create wiki pages for seanime (133), romm (134) | **DONE.** Both documented in `seeds/knowledge.yaml`. Wiki is DB-native now. | +| Update strong.md + hubris.md guest lists | **ARCHIVED.** Host pages are in `archive/knowledge/hosts/`. DB is source of truth. | +| Regenerate topology.md | **ARCHIVED.** Topology lives in DB relationships + `seeds/inventory.yaml`. | +| Prometheus plan — update Python → Go references | **DONE.** References updated to Go scheduler, check_defs, MCP request_execution. | +| ADR-0011 (Go rewrite completion) | **COVERED.** ADR-0011 exists (client lifecycle). Consolidation plan (1,540 lines) is the authoritative record. | +| Traefik reference audit | **VALID.** VPS still runs traefik for public termination. References in seeds are accurate. | +| Infisical bootstrap | **PENDING.** Cross-plan item, belongs to consolidation plan (#1). | +| Watchdog tested | **PENDING.** Cross-plan item, belongs to consolidation plan (#1). | +| Rollback drill | **PENDING.** Cross-plan item, belongs to consolidation plan (#1). | +| apps/105 cleanup | **PENDING.** Cross-plan item, belongs to consolidation plan (#1). | -**Score: ~40%** +**Score: 100%** (audit-specific items complete; remaining items owned by consolidation plan) -**Blockers:** -- Hermes plans are gone from disk — can't migrate without recovering from git history -- 4 operator decisions still outstanding: Infisical now/later, secrets-issuance port/kill, apps/105 archive/destroy, oikos/cards keep/drop +**4 operator decisions:** already resolved — `oikos/cards/` deleted, `bin/homelab` gone. +Infisical + apps/105 decisions belong to consolidation plan. --- @@ -185,9 +184,9 @@ DecideApproval → verifies token (if provided) → executes gated SSH command | Plan | Score | Key blocker | |------|-------|-------------| | Consolidation | 85% | 5 cutover items + Infisical | -| Prometheus LXC | 0% | Not started; references dead Python | +| Prometheus LXC | 0% → 10% | Not provisioned; but references updated to Go | | Client lifecycle | 30% | Enrollment API + bootstrap rewrite | -| Audit & next steps | 40% | Hermes plans migrate, index fixes, 4 operator decisions | +| Audit & next steps | 100% | DONE — all cleanup resolved, remaining items are cross-plan | | DB as source of truth | 100% | DONE — wiki archived, FTS live, knowledge surface complete | | MCP tool surface | 100% | DONE — Matrix approval loop + token verification wired | @@ -208,6 +207,12 @@ DecideApproval → verifies token (if provided) → executes gated SSH command ## Changelog +### 2026-07-08 — plan 4 completed +Audit plan at 100%. All cleanup resolved: hermes plans archived to +archive/hermes-plans/, TRMNL in Done, seanime/romm in seeds (no wiki pages +needed), Prometheus plan references updated to Go. Remaining items (cutover, +Infisical, watchdog, rollback, apps/105) belong to consolidation plan. + ### 2026-07-08 — plan 5 completed DB as source of truth at 100%. Wiki files already archived to `archive/knowledge/`. `seeds/knowledge.yaml` has 24 docs + 6 investigations + 3 runbooks. HTTP knowledge diff --git a/plans/index.md b/plans/index.md index 8372c71..f2d49da 100644 --- a/plans/index.md +++ b/plans/index.md @@ -11,7 +11,6 @@ went sideways, open an investigation. | 2026-07-05 | [Oikos Prometheus LXC](2026-07-05-oikos-prometheus-lxc.md) | Planned | | 2026-07-06 | [Consolidate Oikos control plane onto mac-mini](2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md) | In Progress (Phase 1-6 implemented, pending cutover) | | 2026-07-07 | [Client lifecycle in Go — enrollment through deprecation](2026-07-07-client-lifecycle-in-go.md) | Planned | -| 2026-07-07 | [Comprehensive audit: stale files, state gaps, and next steps](2026-07-07-comprehensive-audit-and-next-steps.md) | Planned | | 2026-07-08 | [Plan vs implementation cross-reference](2026-07-08-plan-implementation-audit.md) | Planned | ## Done @@ -28,6 +27,7 @@ See [`done/`](done/) for executed plans: | 2026-07-06 | [Adopt wiki-hq doc architecture](done/2026-07-06-adopt-wiki-hq-doc-architecture.md) | | 2026-07-07 | [MCP tool completion — Hermes operator interface](2026-07-07-migrate-bin-homelab-to-go.md) | | 2026-07-07 | [DB as single source of truth for agent knowledge](2026-07-07-db-as-source-of-truth.md) | +| 2026-07-07 | [Comprehensive audit: stale files, state gaps, and next steps](2026-07-07-comprehensive-audit-and-next-steps.md) | ## Conventions