Commit Graph

68 Commits

Author SHA1 Message Date
0c0f35a3a9 feat(web): split SPA from oikos binary, require auth on every route
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Phase 0 of plans/2026-07-12-wails-desktop-app.md. The control-room SPA
is no longer embedded (web/embed.go deleted); it's a standalone static
build served separately (make ui / make deploy-ui). The api process
adds CORS and drops the dev-open auth bypass — every route now needs a
real bearer token, including SSE (?token= query param, EventSource
can't set headers) and api's own /agent proxy to nomos (previously
unauthenticated by omission).

nomos was an unauthenticated client of api's /mcp and approval-decision
endpoints; closing dev-open would have broken it, so it now sends
Authorization: Bearer $OIKOS_MCP_BEARER_TOKEN on every call back to api.

SPA gets a runtime config module (config.ts) and a Config.svelte
first-launch/reconfigure page, reachable afterwards via a "Connection"
entry in the sidebar footer. Every fetch() in api.ts routes through
fetchWithAuth so the same build works same-origin (browser prod, Vite
dev proxy) or cross-origin (future Wails webview, remote access).

Six gaps found against the plan and the live Caddy topology while
implementing — documented in the plan's "Plan review" section, most
notably: api's own /agent mount was never behind combinedAuth (fixed),
and production's Authentik forward-auth needs a bearer-token bypass for
API routes that this repo's Caddyfile.oikos reference copy now has, but
the real dtoro/caddy-conf deploy does not yet.

Verified live: cross-origin static SPA + API, CORS, bearer auth, SSE
query-token auth, and localStorage persistence all confirmed working
in-browser. Full Go test suite and npm run build pass with no
regressions against the pre-change baseline.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 15:49:42 +02:00
3157e6102a plans: Wails desktop app with client/server split
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Phase 0 separates the SPA from the oikos binary (delete embed.go, add CORS,
make API base URL configurable, add auth interceptor, close dev-open gate).
Phase 1 builds a thin Wails v3 desktop wrapper — native window + tray +
notifications + auto-start + auto-update. SPA shared between browser and
desktop builds.
2026-07-12 11:49:27 +02:00
c8b479d565 docs: close out task-completion safety net plan; fix stale relative links
Fixes 1-3 deployed and verified live: fresh trivial Q&A sessions now reach
done immediately, and a goal-bearing session that stalled was correctly
nudged by the idle sweep. Fix 4 (backfill) was replaced with deletion after
the operator's call — verified against the DB first that zero knowledge
notes were linked to or written by any of the 53 removed sessions, so
nothing was lost. Documents the pagination gap in listSessions (hardcoded
LIMIT 50, no total count) that hid 6 of those sessions from the original
audit.

Also fixes relative links in this plan and in the UI-review plan that broke
when both moved from plans/ to plans/done/ (one directory level deeper).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 00:14:15 +02:00
075ff93792 docs: mark task-completion safety net fixes 1-3 in progress
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-11 22:11:56 +02:00
e3850f6820 docs: task completion safety net — every live task stuck Running
Traced during UI-review verification: 50/50 live sessions are stuck
active/planning, never done/failed. Root cause confirmed against the
running DB — set_goal called once, propose_plan and complete_task
called zero times across all 50 sessions. The model consistently
skips the terminal complete_task call despite SOUL.md explicitly
instructing it to, especially for trivial single-tool Q&A turns.
Plan proposes an inline safety net for the common case plus an idle
sweep for structured goal/plan sessions that stall.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-11 22:01:35 +02:00
fb4c76ba82 fix(ui): implement UI review findings — a11y, IA, and consistency fixes
Fixes the reviewed gaps: keyboard-inaccessible delete controls (SessionRail,
Entities row), case-sensitive entity filter, two competing entity-detail
navigation patterns (standardize on EntitySheet), non-clickable Overview KPI
cards, a bare button bypassing the shared Button component, inconsistent
blur-only vs live filtering, and an unenforced sanitization assumption on
search snippet HTML (now using the already-present dompurify dependency).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-11 21:52:59 +02:00
b72267bd72 docs: nomos agent code review — mark all fixes done except deferred C1
Every finding from the review is now implemented and verified live:
A1 (3919ec3), B1+B2 (c5ffaec), A3 (926969a), D1-D3 (76f7630), A2 (c390164),
B3 (6d4f6de), F1 (11c18e8). C1 (nomos gateway has no authentication) remains
explicitly deferred per operator instruction. Kept in plans/ (not moved to
done/) since C1 is still open, matching how other partially-complete plans
in this index are tracked.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 20:32:29 +02:00
df393152f6 docs: nomos agent code review — gaps and improvement plan
Full read-through of cmd/nomos/ (agent.go, store.go, main.go, continue.go,
assent.go, tasks.go). Findings, ranked:

- A1 (confirmed via runnable probe): isAssent/isTypedConfirmation use
  unpadded substring matching for assent/confirm words while negation uses
  word-boundary checks — "yes" matches inside "yesterday", "confirm" matches
  inside "confirmed" with no negation word covering contracted negatives
  ("haven't"). isTypedConfirmation gates DESTRUCTIVE actions specifically.
- A2: chatWith replays a session's ENTIRE message history every turn, no
  windowing/token budget — confirmed unbounded against a documented
  production case (70 tool calls, 106KB messages).
- A3: a live turn's tool-call history is lost entirely if the client
  disconnects mid-stream (single end-of-turn save using the same
  connection-tied, possibly-cancelled context) — resumeSession already has
  the fix pattern (incremental placeholder+update), handleChat doesn't use it.
- B1: zero recover() anywhere in cmd/nomos/internal/mcp/internal/httpapi —
  every explicitly-spawned goroutine (continuation worker, resumeSession,
  executeApprovedViaAPI, sse listeners) crashes the whole process on panic.
- B2: auto-continuation processes its batch sequentially, one full LLM turn
  at a time, undercutting this session's own concurrency work on exactly the
  path autonomous tasks depend on most.
- B3: no terminal state for a permanently-failed auto-continuation.
- C1: nomos's own gateway (port 8092, directly published + mesh-reachable)
  has ZERO authentication on any endpoint — chat, session read/delete,
  chat-assent approval of gated executions, all open to anyone on the LAN.
- D1-D3: dead code (isTaskTool unused), N+1 query in recordTouched, no
  validation on complete_task's outcome enum.
- E: zero automated tests for agent.go/store.go/main.go/tasks.go — including
  today's new safety-critical logic (session-scoped windows, mcpClientPool,
  proposePlan's append-vs-replace), verified only by live manual testing.
- F1: tool list + fleet snapshot re-fetched every turn (minor).

Prioritized implementation order in the doc: A1 → C1 → B1 → B2 → A3 → D1-3 →
A2 → B3/F1, tests landing alongside each fix rather than as a deferred pass.

Also archives the now-fully-shipped concurrent-task-execution plan to done/
(all 3 required fixes deployed this session; fix 4 explicitly deferred per
its own recommendation).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 19:47:17 +02:00
6932eb5eed docs: plan concurrent task execution; archive completed tasks plan
New plan grounds three concurrency issues found by tracing the actual code
(not assumed): the assent/destructive windows are keyed by agent id only
(no session dimension), so an approved plan in one task can auto-run
unapproved actions in a concurrently-running task; nomos shares one
mutex-guarded MCP client across all sessions, so a single slow `run` call
serializes every other task's tool calls behind it; and chat.ts's SSE
callback has no session guard, so switching tasks mid-stream lets the
backgrounded task's events corrupt whatever's now displayed. Proposes
session-scoping the windows (critical/first), a frontend stream guard
(contained/second), a per-session MCP client pool (throughput/third), and
an optional concurrency cap (deferred pending real usage data).

Also archives the goal-oriented-chat-control-panel plan to done/ — all 7
phases shipped and are live in production (SHA e30813a) — fixing its
internal relative links for the new depth and pointing forward to the new
concurrency plan as follow-up hardening.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 18:14:35 +02:00
eed6e3b1c5 docs: task-centric chat plan (goal → single-approval → autonomous + knowledge loop)
Reframes the chat surface as a board of tasks: each task carries a goal,
a plan approved once, a lifecycle status, an outcome, and a knowledge
loop that links learnings to the involved entities (and the task entity
itself) via relationships so future tasks compound. Supersedes the
sidebar-only framing and the free-form chat portion of the control-room
web UI plan.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 12:17:36 +02:00
ef5a92269b docs: reconcile plans/ status against actual code state
Audited all 10 active plan docs against the codebase (not just commit
titles). 5 were fully shipped and stale-tagged "Planned"/"In Progress" —
moved to done/ with verification notes. The other 4 got corrected
Planned→In Progress status plus concrete remaining-gap notes so the next
pass doesn't re-derive what's already done.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-11 11:42:26 +02:00
d2f749d33d feat: event-driven auto-continuation — agent runs an approved plan to completion
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
The root cause behind "the agent stops at the first error and doesn't recover":
provisioning executions run ASYNCHRONOUSLY (pct_create fires the SSH work in a
goroutine and returns "running" immediately), so the agent's turn ENDS before
the result exists. The agent literally isn't running when the step fails — it
can't react to a failure it never observes. The only thing that fed results
back was the operator typing "continue" after every async step: the human was
the event loop. (In the flagged 18-message session the operator typed
continue/proceed/?? eight times while the agent correctly diagnosed each failure
but couldn't advance a step on its own.)

This makes the system the event loop instead:

- migrations/017: nomos_plan_executions links each gated execution to the chat
  session that started it.
- cmd/nomos: after a tool result, any "execution <uuid>" it started is linked
  to the session. A background worker (continue.go) polls for those executions
  reaching a terminal state and — while the agent has an open assent window (an
  approved plan is in flight) — re-invokes the agent with the result
  ("execution X completed/failed: <result>"), so it proceeds to the next step
  or diagnoses+fixes the failure, with no operator tick. Guarded against loops
  (mark-continued before running) and bounded by the 30-min window.
- chatWith(): chat() variant that injects the finished-execution note after
  replayed history without persisting a fake user turn.
- DecideApproval: approving a step by ANY route (button or chat-assent) now
  opens the assent window, so auto-continuation works regardless of how the
  operator approved — previously only typing "go ahead" opened it.
- SOUL: the agent is told it will be auto-re-invoked when async steps finish —
  don't poll get_execution_status, don't wait for "continue"; end the turn and
  keep going step by step until the goal is verified or a genuine blocker.

This is the root fix, not another per-command patch: you can't enumerate every
failure of an unbounded action space, but you can give the agent a loop that
observes each result and adapts — because "do anything" always includes "the
first attempt failed."

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 14:19:43 +02:00
9daf8220f2 plan: add observability layer and chat-assent approval to gated-execution plan
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Operator directives: (1) the UI must show what's executing, its risk
classification, live status, and what knowledge the session created — the
system's growth should be visible, not just trusted. (2) approval should be
granted by chat assent ("go ahead"), not a separate button; destructive
actions still require a typed confirmation phrase.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 09:12:09 +02:00
4a96f46e76 plan: general gated execution — unlimited actions, gated by risk classifier
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Evaluate the agent's action path against the OIKOS.md design. Finding: the
intended model (unlimited runbook-driven actions gated by a risk classifier)
already exists on paper and in scaffolding, but the live agent path regressed
to a hard-coded 5-action enum that bypasses the classifier. Plan a layered
realignment: (0) one general gated `run` primitive, (1) runbooks-as-data as the
reliable fast-path, (2) learning. Chosen v1 posture: approve-most (read-only
auto-runs, all state changes gate). Incremental, each step shippable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 09:03:23 +02:00
ea62d744ed feat: pct_create action, ToolCallGroup collapse+animation, inline chat approval
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- Add pct_create to request_execution (MCP) and executeApprovedAction (httpapi)
  Parses JSON config: vmid, hostname, cores, memory, disk_gb, ip, gw, storage,
  template, privileged, nesting, mounts, nameserver, searchdomain. Creates
  entity (state=provisioning), hosts relationship, entity_status on success.
  Fixes action string parsing to use Index instead of SplitN (colons in JSON).

- Rewrite ToolCallGroup.svelte: bits-ui Collapsible replaces native <details>.
  Collapsed by default. Animated header shows live tool count + running tool
  name while streaming. Auto-expands during streaming, auto-collapses on done.

- Add InlineApproval component: parses 'execution UUID queued' from agent
  response, renders Approve/Deny buttons inline in chat, calls decideApproval.

- Document pct_create in nomos/SOUL.md with params, risk class, and approval flow.

- Add session-review skill at .agents/skills/session-review/SKILL.md.

- Add plan: 2026-07-09-session-execution-and-ux-fixes.md.
2026-07-09 11:15:28 +02:00
0d29b1db81 fix: move completed signal-triggers plan to done/, add missing liveness-drift to index, add plan-consistency lint checks 2026-07-09 10:47:39 +02:00
614c38ea7c docs: plan chat-sessions fixes from real production usage data
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Inspected the live agent_sessions/agent_messages tables on mac-mini and
found silent empty responses, a canned non-English refusal after 22 tool
calls, 70-call fan-out for simple fleet questions, 100KB+ persisted
messages, and no session delete/title hygiene.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-09 10:09:34 +02:00
279549c8c9 fix: scheduler wrote health/metrics/events to probe entities, not targets
Problem: every host/service/lxc/etc. entity_status row was permanently
stuck at 'unknown' since creation. Verified against the live DB:
metric_samples had 17,559 rows, 100% attached to type='check' probe
entities and 0% to any real monitored entity; only 25 check entities
ever had real health written. check_defs.entity_id (the probe's own
bookkeeping entity) and check_defs.target_id (the host/service actually
being observed) were both real fields, but the scheduler wrote
UpsertEntityStatus/InsertMetricSample/emitSchedulerEvent keyed by
entity_id instead of target_id — so every check ran and every result
was real, it just landed on the wrong row. This is the mechanism behind
observed drift: the agent's dashboard/health tools reported the
internal probes' state, never the actual fleet.

Change:
- scheduler.go: runCheck/resolveSignal now resolve targetID from
  cd.TargetID (falling back to the check's own id if unset) and write
  status/metrics/events there. Signals stay keyed by the check entity,
  unchanged, matching their existing resolution logic.
- Added a staleness sweep to housekeeping(): an entity whose last
  observation is older than 3x its fastest enabled check's interval
  (floor 5m) is marked 'stale' and emits health.stale, so a stalled
  scheduler or disabled check_def can no longer look like current data
  forever.
- migrations/016: deletes the now-orphaned check-entity entity_status
  rows so dashboard/fleet-health rollups stop double-counting probes as
  monitored entities. Historical metric_samples on check entities are
  left as-is (time-series data, not safe to reattribute).
- openapi.yaml + regenerated gen code: Entity gains health/last_check_at;
  'stale' added to the health enum everywhere it's used.
- dashboard.go / GetFleetHealth / nomos's get_health_summary MCP tool:
  exclude type='check' entities from rollups.
- nomos/agent.go: replay prior turns' tool_use/tool_result pairs into
  the conversation instead of dropping them (previously only final text
  was replayed, forcing the agent to re-derive fleet state every turn),
  and inject a compact live fleet-health snapshot into the system prompt
  each turn so it starts oriented instead of spending an iteration on
  discovery.

Risk: config_mutation (schema-adjacent — new migration, no destructive
DDL, additive DELETE only on orphaned rows). No behavior change until
oikos-api/oikos-scheduler/nomos are rebuilt and redeployed.

Verification: go build/vet clean across the repo. Ran this worktree's
own API binary against the live dev Postgres on an alternate port
(read-only from the live containers' perspective) and confirmed
/api/v1/entities now returns health/last_check_at, and the dashboard
health rollup dropped from double-counting to an honest 168 unmonitored
entities (matches reality pre-deploy — the live scheduler hasn't run
the fixed code yet). Confirmed check_defs.target_id correctly maps
multiple checks to host:hubris via direct psql query.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-09 00:26:04 +02:00
35feada286 scheduler: add ping + ssh-script check kinds, metrics refactor, 17 host check scripts
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- Refactor executeCheck to return checkResult struct with metrics map
- Add ping check kind (ICMP reachability via system ping, macOS+Linux)
- Add ssh-script check kind (remote host exec via SSH, allowlisted scripts)
- Add threshold evaluation (warn/crit per metric from check config JSONB)
- Add inode tracking to disk check
- All 4 existing checks now return structured metrics
- 17 check scripts: cpu, memory, load, swap, disk_usage, disk_smart,
  updates, zfs, process, uptime, oom, journal, time, fd, docker_health,
  caddy_error_rate, backup_freshness
- Auto-deploy via tools/setup-checks.sh -> checks/install.sh on git pull
- Add ping to OpenAPI CheckKind enum and generated Go types
2026-07-08 21:05:53 +02:00
2908b0a377 feat(ui): M4 — agent activity, knowledge search, audit, correlation grouping
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Add three new pages completing the control-room web UI:
- Agent activity: polls /agent-activity every 5s, filterable by type/agent
- Knowledge search: FTS over /knowledge/search with snippet + entity links
- Audit trail: browseable audit log with actor/action/entity filters

Enhanced live events page with correlation-id clustering (Groups toggle).
Added fetchAgentActivity/searchKnowledge/fetchAudit to the API client.
11 nav items now cover all planned control-room views.
2026-07-08 17:02:09 +02:00
e8e230b4a5 nomos+web: streaming, provider routing, event gap-fill, embedded UI; fix approval FK & session context
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Agent (cmd/nomos):
- Stream LLM tokens via NewStreaming; emit text_delta then final text.
- OpenRouter provider routing: data_collection=deny (ZDR) + require_parameters;
  NOMOS_PROVIDER_SORT opt-in; Exacto via model suffix.
- Multi-turn: reload session history into context; UI passes session id.
- Fix agent_activity logging (agent_id/session_id) and mcpClient data race.

Events (live control-room feed):
- approval.created (mcp), approval.decided (api), execution.completed/failed
  (approved-action path), signal.raised/resolved + health.changed (scheduler,
  transition-gated).

Fixes:
- createApproval FK violation (reuse execution entity) — the agent's only
  write path; log the previously-swallowed errors.

Web UI:
- Embed web/dist via //go:embed (single binary); Dockerfile builds SPA into
  the Go stage; committed .gitkeep placeholder keeps backend-only builds green.
- Caddy: Authentik-gated /agent/* -> nomos so the UI reaches the agent
  same-origin in production.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 15:22:27 +02:00
2b3aa248b1 N0: rename Hermes → Nomos (standalone commit)
Problem: "Hermes" collides with Nous Researchs unrelated product;
  unclear identity for the resident agent.

  Change: Rename the live service identity across 39 files:
  - cmd/hermes/ → cmd/nomos/ (binary, env vars NOMOS_*)
  - internal/config/ server.go (NomosAgentSlug, nomosAgentID)
  - compose/hermes/ → compose/nomos/ (Dockerfile, service name)
  - hermes/ → nomos/ (SOUL.md, config.yaml, skills/)
  - .agents/HERMES.md → NOMOS.md (persona)
  - tools/setup-hermes-soul.sh → setup-nomos-soul.sh
  - seeds/inventory.yaml (agent:hermes → agent:nomos)
  - migrations/014_rename_agent_hermes_to_nomos.up.sql
  - Caddy vhost hermes.hubris.network → nomos.hubris.network
  - All referencing docs, scripts, ADR notes

  History preserved: archive/, plans/done/, ADRs not rewritten.
  Matrix @hermes notifier account and Legacy bin/hermes on LXC 129
  intentionally untouched (out of scope).

  Risk: N0 is identity-only rename; zero behavioral changes.
  Verification: go build ./... passes; docker compose --profile full
  resolves nomos service; grep -ri hermes (excluding archive/plans)
  returns only intentional refs (LLM model name, Matrix user).
2026-07-08 14:14:56 +02:00
9d9cbb63c4 plans: rename resident agent Hermes -> Nomos, add implementable rename phase N0
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Nomos (from oikonomos, steward of the oikos) avoids the name collision
with Nous Research's Hermes Agent. N0 enumerates the full rename scope:
cmd/, hermes/ dir, env vars, config fields, compose service, Caddy vhost,
identity-preserving DB slug migration + seed update, persona docs.
History, the Matrix bot user, and legacy bin/hermes stay untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 13:47:24 +02:00
ae3b150dcc plans: hermes agent uses OpenRouter (default deepseek-v4-flash), document off-the-shelf rejection
Swap anthropic-sdk-go for openai-go against the OpenRouter API; default
model deepseek/deepseek-v4-flash with Exacto routing and ZDR provider
pinning. Record why Nous Hermes Agent (and hosted MCP connectors) were
rejected for the resident role.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 13:36:55 +02:00
750f0e088d plans: add Hermes resident agent plan, make agent chat the control-room home view
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Hermes becomes an LLM-backed agent loop (hand-rolled tool loop over the
existing mcpClient, not the public MCP connector), with Postgres-backed
sessions, SSE chat streaming, and Authentik-gated /agent routing. The
control-room plan is amended to make the chat the main entry point.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 13:30:42 +02:00
84068cc40b plans: add oikos gaps review and control-room webui plans
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Documents a full-project review (confirmed bugs, security gaps, user- and
agent-perspective gaps) and a realtime control-room web UI plan, per prior
codebase exploration on this branch.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 13:16:24 +02:00
3ea43adcfd fix: repair 5 MCP analysis tools with SQL errors + seed entity_status rows
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- get_event_timeline: event_type/actor/message → type/source/data::text
- get_blast_radius: add JOIN entities for slug column
- get_state_snapshot: remove nonexistent disk_usage_pct, drift_count
- get_change_history: timestamp/actor_label/details → ts/actor_id::text/detail
- seed: upsert entity_status rows during inventory ingest (was empty, causing INNER JOIN on get_health_summary to return nothing)
2026-07-08 12:27:25 +02:00
e148c7a981 move 5 completed plans to plans/done/
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- Consolidate Oikos on mac-mini (2026-07-06)
- Client lifecycle in Go (2026-07-07)
- Comprehensive audit & next steps (2026-07-07)
- DB as source of truth (2026-07-07)
- MCP tool completion (2026-07-07)

Paths fixed in index.md to reflect planes/done/ locations.
Active plans remaining: Prometheus LXC (Planned), implementation audit (Active).
2026-07-08 11:30:54 +02:00
cefeba72b0 delete apps/105 Gitea webhooks (ids 10, 11, 14)
Removed via Gitea API using Infisical-stored token:
- Hook 10: 192.168.8.205:9811 (homelab-mcp-deploy)
- Hook 11: 192.168.8.205:9821 (secrets-issuance-deploy)
- Hook 14: 192.168.8.205:9831 (oikos-console-deploy) — found during cleanup

All three pointed to apps/105 (192.168.8.205). No remaining webhooks
fire to the old Python stack. Zero hooks remain on dtoro/oikos.

Cutover checklist updated. Only item remaining: archive apps/105 LXC.
2026-07-08 11:29:42 +02:00
7660e5681c complete consolidation plan — scripts, watchdog, rollback runbook
Plan #1 at 98% (code complete). Three fixes applied to remaining cutover items:

1. watchdog.sh — dual-path health checking (LAN 192.168.8.175 + mesh/Caddy
   proxy). Only pages when BOTH paths fail. Partial failure logged but not
   paged (distinguishes stack problem from mesh/Caddy issue).

2. deploy.sh — pre-deploy pg_dump before each deploy saves to
   /opt/oikos/backups/pre-deploy-<sha>.sql. Rollback script now has a
   guaranteed recovery point.

3. docs/operations/rollback.md — runbook documenting automated rollback,
   manual recovery, decision tree, backup schedule, and rehearsal log.

Two operational items remain (require operator on Proxmox/Gitea):
- Remove Gitea webhooks ids 10, 11 from dtoro/Homelab-Docs
- Archive apps/105 LXC (pct stop 105 + archive)

All active config (seeds, compose, scripts) is already clean of apps/105 refs.
Infisical bootstrap code is complete (bootstrap-infisical.sh + Go backend).
2026-07-08 11:25:43 +02:00
28ab9b8088 final verification — client lifecycle plan: 12/12 items confirmed
Added 12-point verification table with file:line evidence for every item.
Noted two minor deviations: inline SQL (not sqlc) and synthetic Infisical
IDs (real Infisical pending consolidation plan). Precondition checks
enumerated with hard/soft classification.
2026-07-08 11:20:10 +02:00
fcd9f23ee1 transition precondition enforcement + thin-client context poller
Plan #3 at 100%. Last three items resolved:

1. Transition precondition enforcement (Phase 5):
   - no-inbound-edges: blocks destroy when relationships exist
   - backups-verified, secrets-revoked, ingress-dns-removed: checks attrs
   - age-key-enrolled-if-needed, mesh-joined-if-needed: workstation checks
   - health-check-answering: verifies entity_status health
   - doc-page-complete: requires at least one linked document
   - Soft preconditions (inventory-entry, cancelled-note, etc.): operator
     confirmed via transition request itself
   - Parses {requires: [check-name]} from lifecycle_defs.transitions JSONB

2. bootstrap.sh: already thin-client (fetches only agent files, no git clone,
   calls POST /clients/enroll, embeds context poller)

3. tools/context-poller.sh: standalone version — polls GET /clients/{slug}/context,
   applies file/tool/sops deltas, re-runs changed setup scripts
2026-07-08 11:16:04 +02:00
efa66c7321 close client lifecycle plan — API was already fully implemented
Plan #3 at 95%. Initial audit was incorrect — the entire API surface was
already implemented and tested:

- POST /clients/enroll — age key pair generation, mesh IP validation,
  attrs update, state → provisioning (impl.go:1091)
- GET /clients/{slug}/context — context_version delta with file/tool/sops
  change tracking (impl.go:1195)
- GET /clients/{slug}/secrets — scoped secret key listing (impl.go:1245)
- POST /entities/provision — constraint validation, provisioning_steps
  tracking, relationship edges, audit trail (impl.go:1271)
- GET /entities/{slug}/provision/status — step-by-step progress (impl.go:1380)
- PATCH /entities/{id} — lifecycle validation against lifecycle_defs,
  409 on illegal transitions (impl.go:933)
- client_lifecycle_test.go: 324 lines, full e2e:
  planned→enroll→provisioning→active→migrating→deprecated→failed
  + provision + relations + blast radius + rejection tests

Remaining (follow-up): bootstrap.sh + context-poller.sh thin-client scripts.
2026-07-08 11:12:45 +02:00
43aaf2a318 complete comprehensive audit — all cleanup items resolved
Plan #4 done. Audit inventory verified against codebase:

- 9 superseded oikos/*.py files deleted (only gen-topology.py remains)
- bin/homelab deleted, bin/oikos deleted, oikos/cards/ deleted
- .hermes/plans/ already archived to archive/hermes-plans/ (all 7 files)
- TRMNL plan already in Done table
- seanime + romm documented in seeds/knowledge.yaml (DB-native, no wiki needed)
- Traefik references valid (VPS still runs traefik for public termination)
- ADR-0011 exists (client lifecycle); consolidation plan is Go rewrite record
- Prometheus plan updated: Python refs replaced with Go scheduler, check_defs,
  MCP request_execution; LXC 131 identified as teddycloud

Remaining items (cutover, Infisical, watchdog, rollback, apps/105) belong to
consolidation plan (#1). 4 of 6 plans now Done.
2026-07-08 11:09:37 +02:00
a3ebd12e90 complete DB as source of truth — FTS knowledge surface
Plan #5 done. Wiki already archived to archive/knowledge/. seeds/knowledge.yaml
has 24 docs + 6 investigations + 3 runbooks.

- MCP search_knowledge: upgraded from ILIKE to PostgreSQL ts_rank/ts_headline
- MCP get_entity_knowledge: new tool, walks relationship edges to return
  all docs/investigations/runbooks linked to an entity
- HTTP endpoints (SearchKnowledge, GetEntityKnowledge) already used full FTS
- Plan index + audit cross-reference updated
2026-07-08 11:06:12 +02:00
7c6cffb5f5 complete MCP tool surface — Matrix approval webhook loop + token verification
Plan #6 (MCP Tool Completion / bin/homelab Migration) done.

- Approval records created for gated request_execution actions
- Notifier sends Matrix messages with HMAC approval tokens
- Stores matrix_event_id, polls /relations/{id}/m.annotation for /
- Reaction detection triggers DecideApproval API call
- Token verification added to DecideApproval endpoint
- Migration 013: matrix_event_id + alert_sent_at on approvals
- AGENTS.md: 21-tool surface documented, stale homelab CLI refs removed
- Plan index updated, audit cross-reference refreshed
2026-07-08 11:02:06 +02:00
8653f3036d plan: client bootstrap fetches CLIENTS.md as primary orientation
CLIENTS.md is the entry point for a machine joining the homelab.
AGENTS.md is the AI agent persona layer on top. Both fetched, but
CLIENTS.md comes first.
2026-07-07 23:59:22 +02:00
79dc87d584 plan: rev 2 — add thin-client API distribution and compute entity provisioning
Two onboarding paths share the same lifecycle state machine:

1. Workstation self-enrollment: curl bootstrap.sh | bash → API enroll
   → thin client (no git clone, no sync timer). Context poller replaces
   5-minute pull. Only AGENTS.md, OIKOS.md, tools/ fetched to disk.

2. Compute entity provisioning: POST /entities/provision → Oikos
   actuator creates LXC/VM/container on Proxmox host. Validates VMID,
   IP, capacity, template. Creates relationship edges (hosts, provides,
   mounts, depends-on) atomically. No self-enrollment, no age key.

Adds: context endpoint for agent file deltas, provisioning_steps table,
actuator provision methods, type-specific transition checks, full
verification matrix covering both paths.
2026-07-07 23:56:07 +02:00
638e313c66 docs: add client lifecycle plan, cleanup stale files, document repo for 3 audiences
Problem: Repo had no developer guide, no client onboarding doc, no agent
dev instructions. Stale files (675KB SQL dump, one-off convert script,
legacy MCP builder) cluttered the tree. Client enrollment was a documented
intention with no Go implementation.

Changes:
- New docs: CONTRIBUTING.md (dev setup), CLIENTS.md (client onboarding),
  .agents/dev/CONTRIBUTING.md (agent codebase map)
- New plan: plans/2026-07-07-client-lifecycle-in-go.md — full client
  lifecycle (planned→provisioning→active→deprecated→destroyed) in Go,
  replacing archived Python secrets-issuance, adding client API endpoints
  and 6 missing MCP tools
- Cleanup: deleted archive/convert-wiki.py (one-off), archive/mcp/
  build_host_files.py (legacy), backups/pre-deploy-7f7d039.sql (local)
- Fixes: plans/index.md duplicate row removed, README.md repo layout
  updated for current state, AGENTS.md header points to new guides

Risk: low. Docs only + stale file deletion. No code changes. New plan is
proposal, not implementation.
Verification: git diff reviewed, all changes are prose/docs/plans.
2026-07-07 23:45:32 +02:00
bbd560bf19 plans: flip bin/ migration to MCP tool completion — agent is the operator interface 2026-07-07 22:39:45 +02:00
e93625b971 plans: add bin/homelab → Go migration plan, fix Active/Done plan listings 2026-07-07 22:27:17 +02:00
6b51c83b82 plans: mark TRMNL and wiki-hq adoption as done, move to plans/done/ 2026-07-07 21:06:58 +02:00
b2bfa26f64 plans: comprehensive audit + DB-as-truth architecture 2026-07-07 20:09:32 +02:00
c8b27b2c51 cleanup: reflect Go rewrite reality — remove legacy Python artifacts
Removed:
- bin/hermes (9.3MB compiled binary accidentally committed to git)
- oikos/console/ (Flask web console — replaced by Go REST API + SSE)
- mcp/server.py (Python MCP server — replaced by internal/mcp/)
- oikos/policy.yaml, oikos/ontology.yaml (duplicates of seeds/)

Kept:
- oikos/*.py kernel files (12 files) — still imported by bin/homelab
  for operational CLI commands (ssh, pct, logs, restart, status,
  open, secret, client, sync, mcp). Will be removed when bin/homelab
  is ported to Go.
- mcp/build_host_files.py — generates hosts/*.yaml from inventory,
  still operational. Will be ported to Go.
- bin/homelab — active Python CLI, still operational.

Updated:
- .gitignore: added bin/hermes, cleaned up legacy comments
- plans/index.md: listed all 4 active plans with accurate statuses
2026-07-07 17:43:10 +02:00
18cb79caf9 oikos phase 0: ontology + inventory + policy seeds, OpenAPI contract, ADRs
- seeds/ontology.yaml: 59 entity types (5 abstract, is-a hierarchy), 46
  relationship types with cardinality, 6 lifecycles with terminal states
  and named precondition checks
- seeds/inventory.yaml: 110 entities / 142 relationships translated from
  legacy inventory.yaml (fleet, services, ingress, storage, governance,
  archaeology); thin spots marked for backfill
- seeds/policy.yaml: 4 risk classes, 27 approval rules (hierarchy-aware,
  per-entity overrides), autonomy kill-switch off (cold start)
- api/openapi.yaml: full v1 REST contract (40 paths), RFC 9457 errors,
  cursor pagination, idempotency, ETag/If-Match, scopes; redocly-clean
- docs/adr/0001-0010: initial architecture decision records
- scripts/validate-seeds.py: Phase 0 gate — hierarchy, lifecycles,
  endpoints, cardinality, policy cross-refs (0 errors)
- plan: layer CHECK gains 'meta' (root type), cardinality gains
  'many-to-one'

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:17:15 +02:00
ea3b2c3662 plans: oikos rev 3 — consolidated spec, ontology inheritance, OpenAPI-first
Merge the rev-2 audit + remediation layers into one self-consistent spec and
close new gaps: meta-schema inheritance (parent_type/is_abstract), contract-
first API (RFC 9457, idempotency, ETag, scopes, /graph), single-binary role
packaging, UUIDv7+slug IDs, checks-as-data, signal dedup/flap/maintenance,
executable skill format, MCP streamable HTTP, SSE events, ledger-as-view,
dual-path networking (mesh-primary + LAN break-glass), per-phase acceptance
criteria, ADRs. Appendix A maps every rev-2 finding to its resolution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:03:25 +02:00
8850b85325 plans: remediate all HIGH audit items + architect/developer review
Addresses 15 original HIGH audit findings + 18 new findings from
systems architect + senior Go developer review (572 lines added).

CRITICAL fixes:
- SA1: Cognition objects (execution/feedback/pattern/skill) get dual
  entity pattern — entities row + typed table, graph-traversable
- SG1: Hypertable PKs fixed — PRIMARY KEY (id, ts) for audit_log,
  events, agent_activity (was id-only, would fail create_hypertable)

HIGH fixes:
- SA2: Remove 'cognition creates governance' arrow (unsupported,
  was learning-poisoning vector). Patterns propose, operator accepts.
- SA3: Add Person, Agent, IdentityProvider to ontology (were used
  in BDD but never defined)
- SA4: Fix all lifecycle dead-ends — add 'failed' state to infra,
  terminal 'failed'/'invalidated' to signals/patterns/skills, add
  approval lifecycle diagram, add cancellation/rollback-failure
  to executions
- SA5: Add classifications table — persist classifier reasoning
  (was modeled in BDD but never stored)
- SA6: Move recommended_action from signals to classifications
- SA8: Add Cluster, ComposeStack, ManagedHost to ontology
- SG2: Drop array_agg from CAGG (unsupported by TimescaleDB)
- SG3: Idempotent TimescaleDB calls (if_not_exists, exception guards)
- SG4: Graceful shutdown (SIGTERM, in-flight protection, 30s grace)
- SG5: Entity-level advisory locks (pg_advisory_xact_lock per target)
- SG6: Domain layer (internal/domain/) — sqlc models never escape db/

Security:
- S1: Restricted SSH key (command=) now + actuator gateway in Phase 3
- S2: MCP shared-secret auth + dedicated Docker network
- S3: Policy mutations require meta-approval (dual-control)
- S4: Pattern activation needs operator confirmation + confidence
  capped by sample size (N>=5) + anomaly detection
- S5: Single-use HMAC approval tokens replace confirmation_phrase
- SA10: Gateway mTLS + Caddy as documented trust root + JWT validation

Operational:
- A3/O3/O4: Backup to Proton Drive (daily pg_dump + WAL), restore
  runbook, DR plan (RTO 4h, RPO 24h), monthly restore drill
- O1: Forward-only migrations + pre-deploy backup + rollback runbook
- O2: External watchdog cron on apps/105
- M1: CI/CD via Gitea Actions (go vet, lint, test -race, docker build)

Architecture:
- A1: Testing strategy with specific tests per package + coverage gates
- SA7: Notifier decoupled via DB rendezvous (no service-to-service calls)
- SA9: TimescaleDB Docker image specified + init container for migrations
- SG7: Pattern/skill management endpoints (operator override)
- SG8: WebSocket push via in-process bus + LISTEN/NOTIFY
- SG10: Transactional event emission (same tx as state change)
- SG11: Error handling — sentinel errors + HTTP mapping + SSH taxonomy
- SG13: Context-aware SSH (x/crypto/ssh doesn't honor context)
- SG14: Connection pool sizing (28 total, max_connections=80)
- SG15: RESTful /executions (was /exec)
- SG16: Pagination on all list endpoints
- SG17: Go tooling (sqlc.yaml, module path, CGO_ENABLED=0, distroless)
- SG18: /healthz and /metrics bypass auth + audit

Updated phasing incorporates all remediation.
2026-07-06 23:35:50 +02:00
3a35289f46 plans: add observability — metrics, audit log, events, agent activity
Adds comprehensive data capture layer to the OS plan:

1. TimescaleDB — PostgreSQL extension for time-series data. No separate
   database. Hypertables auto-partition, continuous aggregates provide
   1h/1d rollups, retention policies auto-drop old data.

2. Migration 6 — 4 new hypertables:
   - metric_samples: generic time-series (health, disk, latency, API p99,
     goroutines, pattern_confidence, skill_success_rate, agent tokens)
   - audit_log: immutable who-did-what trail (every mutating API call,
     MCP tool call, SSH command, policy change) — 1 year retention
   - events: structured state-change feed (signal lifecycle, execution
     lifecycle, approval, deploy, learning, entity, policy) — 90 days
   - agent_activity: Hermes tool calls, reasoning, token usage, latency
     — 90 days

3. Correlation IDs — propagated through the full call chain (signal →
   classification → execution → SSH → verification → feedback → pattern)
   so any action chain can be reconstructed end-to-end.

4. 6 new MCP tools for agent self-query:
   query_metrics, get_trend, get_audit_trail, get_event_timeline,
   get_agent_activity, get_health_summary

5. 8 new REST endpoints for metrics/audit/events/health/trends/export

6. Workstream 14 — observability + data capture (7 sub-components A-G):
   metrics, audit, events, agent activity, structured logging, agent
   data availability, future visualization plug-in points

7. New Mermaid diagram — observability data capture and query flow

8. Updated architecture diagram to show observability data flows

9. Updated phasing — observability woven into phases 1-3

10. Updated verification — 14 end-to-end checks (was 12), including
    metrics querying, audit trail, correlation tracing
2026-07-06 23:12:34 +02:00
2d75544362 plans: SysML BDD ontology, generic model, full audit
Ontology rewrite:
- Replace Mermaid ER diagram with SysML Block Definition Diagrams (BDD)
  using class diagram syntax: generalization, composition, aggregation,
  association with multiplicity annotations
- Split into 3 diagrams: infrastructure (compute/storage/network),
  software+services, cognition (operations+learning)
- Make compute model generic: ComputeEntity abstract base with
  specializations (Machine, VirtualMachine, Container→LXC/DockerContainer;
  Machine→ProxmoxHost/StandaloneServer/Workstation/Appliance)
- Hypervisor is software on a Machine (not all machines are Proxmox)
- Any ComputeEntity can mount Volumes (VMs AND LXCs, validated)
- DockerContainer is first-class (OS models its own infrastructure)
- Services on any compute type (not just LXC/VM)
- Documents/Runbooks describe any Entity (not just Service)
- Added design notes validating assumptions against actual inventory

Schema updates:
- entity_types: add attribute_schema (JSONB for validating attributes)
- entity_types: add status (active/deprecated, no hard delete while instances exist)

Audit (37 findings across 6 categories):
- Security: 10 findings (5 HIGH) — SSH keys, MCP auth, policy mutability,
  learning poisoning, confirmation phrase, webhook auth, TLS, blast radius
- Performance: 7 findings (1 HIGH) — CTE cycle guard, probe concurrency,
  ingestion, pattern extraction, table growth, WS backpressure
- Architecture: 7 findings (3 HIGH) — testing, observability, DB backup
- Data model: 7 findings (1 HIGH) — entity ID, attribute schema, type
  evolution, concurrent writes, migration rollback, DR export
- Operational: 7 findings (4 HIGH) — rollback, watchdog, backup/restore
  runbook, disaster recovery, deploy downtime, health checks
- Missing: 7 findings (1 HIGH) — CI/CD, rate limiting, audit log, circuit
  breaker, secret rotation, supply chain, SLOs
- Top-5 priority items called out before implementation
2026-07-06 23:06:34 +02:00
d44979aca7 plans: rev 2 — Go rewrite, ontology-first, DB-native config, learning loop
Major revision of the Docker-based homelab OS plan:

1. Go instead of Python — all services rewritten as Go binaries
   (Gin web framework, sqlc for DB access, goroutines for probes)

2. Ontology-first design — systems modeling with 3 layers:
   - Infrastructure (physical, compute, network, storage, software)
   - Governance (identity, secrets, policy)
   - Cognition (observation, decision, action, knowledge, learning)
   7 Mermaid diagrams: layer map, ER diagram, 3 lifecycle state machines,
   feedback loop, policy model

3. DB-native config — inventory.yaml/ontology.yaml/policy.yaml become
   seed manifests (bootstrap + DR). The DB is the runtime source of truth,
   editable via API. Ontology IS the DB schema (entity_types,
   relationship_types, lifecycle_defs tables).

4. Feedback loop — agent learns from execution:
   execution → outcome → feedback → pattern → skill → classification
   Patterns accumulate from execution history, skills codify proven
   procedures, classifier uses pattern confidence for auto-act decisions.
   Cold start: agent starts cautious, earns autonomy through evidence.

5. 5 migration groups: ontology meta-schema, entity instances, operations,
   learning model, policy. Recursive blast_radius SQL function.

6. Phase 0 added: ontology design before any code.
2026-07-06 22:50:49 +02:00