dtoro
e3a0326c78
Full review (plans/2026-07-17-codebase-review-and-cleanup.md) covering Go, web SPA, and docs. Applied low-risk doc/tooling fixes; code refactors and dead-code deletions are listed as actionable recommendations pending approval. Doc fixes: - AGENTS.md: remove ghost of retired request_execution (contradicted the retire notice above it); fix knowledge/wiki/ -> archive/knowledge/; replace brittle counts (33 tools, 36 docs, 20 checks) with pointers to source; drop point-in-time dates. - OIKOS.md: fix broken plan link (now in done/); 001-011 -> 001-020; 15 MCP tools -> pointer; replace hardcoded knowledge counts. - README.md: 15 tools -> pointer; fix wails plan link (now in done/); complete internal/ package list (add checkdefaults, observability, safego); add cmd/desktop/ to repo layout. - commands.md, page-templates.md: fix broken links; HERMES.md -> NOMOS.md. Plans housekeeping: - Move 4 done 2026-07-14 plans from plans/ to plans/done/. - Reconcile plans/index.md: add the 2 missing 2026-07-14 entries and the 2 missing 2026-07-15 done entries; add this review. - Fix stale plan path in migrations/020 comment. New docs: - docs/index.md and docs/operations/README.md (folder READMEs per writing-style.md). Tooling: - web/package.json: add check/typecheck/lint scripts + svelte-check devDep. - Makefile: desktop-package version now reads from VERSION file instead of hardcoded 0.1.0. VERSION 0.7.6 -> 0.7.7 (patch: docs + tooling only).
191 lines
9.7 KiB
Markdown
191 lines
9.7 KiB
Markdown
# Oikos — the operating model
|
||
|
||
Oikos (Greek: *household*) is the agent operating system layered on this
|
||
repo. It is not new infrastructure: `seeds/inventory.yaml` is the kernel data
|
||
structure, the Oikos REST API and MCP server are the syscall surface, and
|
||
this page defines the rules everything above them follows.
|
||
|
||
Read this after [AGENTS.md](../AGENTS.md). Machine-readable companions:
|
||
[seeds/ontology.yaml](../seeds/ontology.yaml) (systems model),
|
||
[seeds/policy.yaml](../seeds/policy.yaml) (risk & approval).
|
||
|
||
## The kernel loop: OODA
|
||
|
||
Every Oikos activity — scheduled probe, agent task, operator request — is
|
||
one pass through **Observe → Orient → Decide → Act**:
|
||
|
||
1. **Observe** — probes, drift detectors, and agent findings produce
|
||
**Signals** (structured records, not loose messages): pending updates,
|
||
high temperature, low disk, service down, cert expiry, stale backup,
|
||
inventory drift.
|
||
2. **Orient** — walk the ontology graph: what entity is affected, what
|
||
depends on it (blast radius), its lifecycle state, whether a runbook
|
||
matches, what the ledger says about past attempts.
|
||
3. **Decide** — the classifier scores **risk class × blast radius ×
|
||
confidence** and routes:
|
||
- **auto-act**: within autonomy policy, high confidence, contained radius
|
||
- **escalate**: operator approval via Matrix (✅/❌ reaction) or the
|
||
Oikos Console's `/approvals` page (destructive actions additionally
|
||
need a typed confirmation phrase either way)
|
||
- **queue**: informational — console + reports
|
||
The classifier can only *lower* autonomy relative to policy, never raise
|
||
it. When in doubt, escalate.
|
||
4. **Act** — execute through `homelab` commands or runbooks (never ad-hoc
|
||
SSH), then **verify** with the action's verification command, write a
|
||
**ledger** entry, resolve the Signal, and update docs in the same session.
|
||
|
||
## Primitives
|
||
|
||
| Primitive | What it is | Lives in |
|
||
|---|---|---|
|
||
| Host / Service | topology entities | `inventory.yaml` |
|
||
| Secret | SOPS+age encrypted value, per-client recipients | `secrets/` + `.sops.yaml` |
|
||
| Runbook | executable workflow with risk class + verification | `.agents/skills/<name>/SKILL.md` |
|
||
| Signal | something needing attention, with lifecycle | DB `signals` table |
|
||
| Change | one mutation: who, what, risk, approval, verification | DB `audit_log` + `executions` tables |
|
||
| Approval | short-TTL signed grant for a gated action | DB `approvals` table |
|
||
| Incident | investigation narrative | DB `knowledge_entities` (seeded from investigations) |
|
||
| Knowledge | document, runbook, investigation | DB `knowledge_entities` (seeded from `seeds/knowledge.yaml`) |
|
||
| Plan | design doc for non-trivial work | `plans/` |
|
||
| Agent | enrolled client identity = its age pubkey | `inventory.yaml` + `.sops.yaml` |
|
||
|
||
## Risk classes (enforced, not advisory)
|
||
|
||
From [seeds/policy.yaml](../seeds/policy.yaml):
|
||
|
||
- **read_only** — status, logs, docs, inventory. Unattended.
|
||
- **reversible_low** — restart, cache clear, sync pull. Unattended + ledger.
|
||
- **config_mutation** — tracked-config edits (commit+push, never local),
|
||
deploys, upgrades, DNS/ingress changes. Operator approval.
|
||
- **destructive** — destroy, format, wipe, rotate, revoke. Approval +
|
||
typed confirmation phrase.
|
||
|
||
Lifecycle gates modify these: `provisioning` nodes are freely mutable
|
||
(nothing depends on them); `deprecated` nodes accept no new dependents;
|
||
anything touching a `destroyed` node is drift.
|
||
|
||
## The systems model
|
||
|
||
Eight domains — physical, compute, network, storage, software,
|
||
identity & access, operations, external — cover everything in the lab;
|
||
entities are connected by typed edges (`hosts`, `provides`, `mounts`,
|
||
`stores-on`, `routes-to`, `can-decrypt`, `depends-on`, `backs-up-to`, …)
|
||
defined in [seeds/ontology.yaml](../seeds/ontology.yaml). Rule of
|
||
completeness: **if it can break, be changed, or hold data, it has an
|
||
entity and edges.** Blast-radius questions ("what breaks if strong goes
|
||
down?") are graph walks, not doc archaeology.
|
||
|
||
Nodes move through an explicit lifecycle —
|
||
`planned → provisioning → active → migrating → deprecated → destroyed` —
|
||
stored as `state:` in inventory (absent = active). Destroyed nodes live in
|
||
the `archaeology:` section. Each transition is a runbook checklist;
|
||
deprecation completes only when inbound edges reach zero.
|
||
|
||
Generated views: the live topology graph at `oikos.hubris.network/graph`
|
||
via the API's `/api/v1/graph` endpoint, and the Mermaid export at
|
||
`GET /api/v1/graph?format=mermaid`.
|
||
|
||
## Conventions carried forward
|
||
|
||
- Inventory is the truth; live state wins over narrative docs.
|
||
- Prefer `homelab` CLI and MCP over ad-hoc SSH.
|
||
- Meaningful changes update docs in the same session.
|
||
- Secrets are decrypted locally via per-client keys; never into docs/comments.
|
||
- Tracked configs change by commit + push, not local edits.
|
||
- Netbird is the preferred mesh path for new traffic.
|
||
- Agents are terse ([caveman.md](shared/caveman.md)), verify claims, and fix
|
||
collateral drift when found.
|
||
|
||
## Build status (Go rewrite — deployed 2026-07-07)
|
||
|
||
The Oikos runtime was rewritten from Python to Go over 6 phases and is deployed
|
||
in Docker on mac-mini. See
|
||
[plans/done/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md](../plans/done/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md)
|
||
for the full plan. The Python codebase has been removed; all functionality runs
|
||
in the Go binary.
|
||
|
||
**Phase 1 — Ontology + DB (DONE):**
|
||
- `migrations/` (001–020, forward-only): TimescaleDB hypertables, entity_status, CAGGs,
|
||
retention policies, knowledge entities with FTS. Idempotent.
|
||
- `seeds/{ontology,inventory,policy,knowledge}.yaml`: DB-native bootstrap +
|
||
DR export. Knowledge seed contents are not hardcoded here — count them
|
||
from the seed or query the DB.
|
||
- `blast_radius()` SQL CTE, type hierarchy, abstract types, relationship
|
||
validation.
|
||
- Go packages: `internal/db/`, `internal/ontology/`, `internal/domain/`,
|
||
`internal/knowledge/`.
|
||
|
||
**Phase 2 — API (DONE):**
|
||
- Single binary `cmd/oikos` with `oikos api` serving REST (:8090) + MCP
|
||
on the same service layer. OpenAPI-first (`api/openapi.yaml`) with
|
||
oapi-codegen + chi. RFC 9457 problem+json errors. Cursor pagination,
|
||
If-Match/ETag optimistic concurrency, idempotency keys, SSE event stream,
|
||
OIDC JWT + static bearer auth, audit middleware.
|
||
- Go packages: `internal/httpapi/`, `internal/httpapi/gen/`.
|
||
|
||
**Phase 3 — Control loop (DONE):**
|
||
- Scheduler (`oikos scheduler`): check_defs runner, signal dedup/flap
|
||
suppression, entity_status. HTTP, TCP, disk, cert-expiry probes.
|
||
- Actuator: SSH skill procedure execution with context-aware timeouts,
|
||
circuit breaker, retry budgets, error classification.
|
||
- Learning engine: hourly pattern extraction, Wilson confidence bounds,
|
||
anomaly detection, skills with validated patterns.
|
||
- Notifier: Matrix badge delivery, approval token generation (HMAC,
|
||
single-use, hashed), DB rendezvous pattern.
|
||
- Policy classifier: risk class determination, autonomy routing,
|
||
blast-radius computation, kill-switch support.
|
||
- Go packages: `internal/scheduler/`, `internal/actuator/`,
|
||
`internal/learning/`, `internal/notifier/`, `internal/policy/`.
|
||
|
||
**Phase 4 — Agent / Nomos (DONE):**
|
||
- Standalone Nomos MCP client binary (`cmd/nomos`) with gateway mode
|
||
(:8092). Structured queries + natural-language routing to the MCP tool
|
||
list (see AGENTS.md §3). Agent activity logging on every tool call. No SSH keys.
|
||
- `nomos/` directory with config, SOUL.md, homelab-ops skill.
|
||
- Nomos Docker service in `docker-compose.yml` (profile: full).
|
||
- Go packages: `cmd/nomos/`, `compose/nomos/`.
|
||
|
||
**Phase 5 — Secrets / Infisical (DONE):**
|
||
- `internal/secrets/`: backend abstraction (Manager) with primary
|
||
(Infisical) and fallback (SOPS) backends. Machine identities via
|
||
UniversalAuth. In-memory cache with TTL.
|
||
- `oikos secret` CLI: list, migrate (SOPS → Infisical), export-sops
|
||
(DR fallback). Infisical SDK v0.8.0.
|
||
- Rotation runbook at `secrets/rotation.md`.
|
||
- Docker compose: `infisical` + `redis` services (profile: infisical).
|
||
|
||
**Phase 6 — Deploy + cutover (DONE, pending production cutover):**
|
||
- CI pipeline: `.gitea/workflows/ci.yml` (Gitea Actions — build, vet,
|
||
lint, test, docker build).
|
||
- Deploy: `scripts/deploy.sh` (git pull → docker build → compose up →
|
||
health check), SHA-tagged images, rolling restart.
|
||
- Caddy config: `compose/caddy/Caddyfile.oikos` (oikos/mcp/nomos →
|
||
mac-mini mesh :8090/:8092).
|
||
- Watchdog: `scripts/watchdog.sh` (2min cron, Matrix alert on failure).
|
||
- Verification: `scripts/verify-phase6.sh` (14/14 checks pass).
|
||
- Rollback: `scripts/rollback.sh` (checkout SHA + pg_restore).
|
||
- Cutover checklist: `scripts/cutover-checklist.md`.
|
||
|
||
**Current deployment:**
|
||
- **Production**: Docker stack on mac-mini (`--profile full`: postgres, api,
|
||
scheduler, notifier, nomos). Deployed 2026-07-07 with full knowledge seed.
|
||
The Python MCP server and secrets-issuance on apps/105 have been stopped
|
||
(see `scripts/cutover-checklist.md`).
|
||
|
||
## Python-era backlog (superseded)
|
||
|
||
The original 30-day roadmap (Python, shipped 2026-06/07) delivered:
|
||
context cards, change ledger, node relations, runbooks, ops scheduler,
|
||
drift detectors, signal engine, classifier, approval engine, and the
|
||
FastAPI+Jinja2 Oikos Console. All of these have been re-implemented in
|
||
the Go rewrite. The backlog items below that referenced Python paths
|
||
(`oikos/approve.py`, `oikos/drift.py`, `oikos/console/`) are now addressed
|
||
by the Go equivalents listed above.
|
||
|
||
Outstanding from the Python era (not yet in Go):
|
||
- Prometheus provisioning (see [plans/2026-07-05-oikos-prometheus-lxc.md](../plans/2026-07-05-oikos-prometheus-lxc.md))
|
||
- CPU/NVMe temperature probing (blocked on sensor path discovery)
|
||
- SSH-key-signed approval requests (blocked on inventory schema)
|
||
- Multi-agent delegation (blocked on ledger identity field)
|
||
- Restore drills on a schedule
|