Commit Graph

29 Commits

Author SHA1 Message Date
d2950dd09d refactor: sqlc vs raw SQL — hybrid approach (R3)
Deleted 8 genuinely unused sqlc queries (no inline equivalent):
- UpsertCurrentRelationship, ListEntitiesCapped, ListEntityStatus,
  UpdateSignalState, InsertClassification, InsertFeedback, InsertSkill,
  UpsertCurrentRelationship — all had zero call sites.

Migrated 9 inline raw SQL sites to use sqlc queries:
- GetOntology (impl.go): ListEntityTypes, ListRelationshipTypes,
  ListLifecycleDefs — replaces 3 raw pool.Query blocks with typed sqlcgen
  calls, eliminating manual row scanning.
- EndRelationship (phase3.go): EndCurrentRelationship — replaces tx.Exec
  with sqlcgen.New(tx).EndCurrentRelationship.
- checkPrecondition (impl.go): GetEntityStatus — replaces tx.QueryRow +
  manual Scan with sqlcgen.New(tx).GetEntityStatus.
- GetEntityRelations (impl.go): ListEntityRelations — replaces raw pool.Query
  + scanRelationships helper (now deleted).
- GetGraph (impl.go): ListGraphEdges — replaces raw pool.Query +
  scanRelationships.
- resolveEntityID (impl.go): GetEntityBySlug/GetEntityByID — replaces
  raw pool.QueryRow + Scan.
- createApproval (mcp/server.go): InsertApproval — replaces raw pool.Exec
  with sqlcgen.InsertApproval.

Deleted scanRelationships helper (was only used by the two migrated
graph queries above).

Regenerated sqlcgen — also picks up stale model updates (AgentSession,
SessionPlanStep, SessionQuestion, etc. from recent migrations).

Documented the carve-out in .agents/dev/CONTRIBUTING.md §SQL conventions:
sqlc is the default; raw pool.Query/Exec is reserved for LISTEN/NOTIFY,
dynamic WHERE builders, blast_radius(), and COPY.

go vet, build, httpapi/mcp/db tests all pass. -383/+170 lines.
2026-07-17 22:24:23 +02:00
c5ffaec85b fix(agent): panic recovery on every background goroutine (B1+B2)
Fixes B1 and B2 of plans/2026-07-11-nomos-agent-code-review.md together,
since the right granularity for B1 in the auto-continuation worker turned
out to require B2's restructuring anyway (see below).

B1: grep -rn "recover()" cmd/nomos/ internal/mcp/ internal/httpapi/ returned
nothing before this — every explicitly-spawned goroutine (continuation
worker, resumed chat turns, async execution dispatch, the SSE listener, two
duplicate sshExec implementations' output-collector goroutines) crashed the
whole process on an unhandled panic, not just that one goroutine. More
consequential post-concurrency: more simultaneous unattended background work
means more surface area for one bad input to end every running task.

New internal/safego package: Go(label, fn) launches fn in a goroutine with a
recover-and-log wrapper. Applied at every bare `go` spawn site across the
three packages. Two sites needed bespoke handling instead of the generic
helper because their callers block on a channel and a silent recover would
just make them hang until timeout: sshExec's output-collector goroutine (two
near-identical copies, internal/mcp/server.go and internal/httpapi/phase3.go)
and httpapi's ListenAndServe goroutine — both now recover AND send a
synthetic error result so the waiting select unblocks immediately instead of
waiting out the full timeout.

httpapi's sseListener got extra treatment: its per-notification handling was
extracted into handleNotification with its own recover, so a panic decoding
ONE malformed pg_notify payload can't kill the listener goroutine for every
connected SSE client — the outer goroutine spawn only needs to guard the
connection setup/reconnect code around it.

B2: cmd/nomos/continue.go's processContinuations used to run every pending
continuation SEQUENTIALLY in a plain for loop, in the SAME goroutine as the
ticker — meaning (a) task B's continuation waited for task A's full (up to
10-minute) resumed turn to finish first, undercutting this session's earlier
concurrency work on exactly the path autonomous tasks depend on most, and
(b) an unrecovered panic anywhere in that call chain didn't just crash the
process (B1) — even WITH B1's recovery wrapped only at the top-level worker
spawn, the panic would still unwind the ENTIRE ticker-loop goroutine,
silently ending auto-continuation for every task until nomos restarted.
Fixed by spawning each pending item via safego.Go individually: real
parallelism, and a bad item can now only ever take down its own goroutine.

Added internal/safego/safego_test.go: TestGo_RecoversPanic is the concrete
proof — a deliberate panic inside Go() that would otherwise crash the whole
test binary; reaching the assertion after it IS the evidence recovery works.

Verified live against the rebuilt containers: full chat turn round-tripped
correctly (hostname lookup, 2 iterations, normal completion) — no regression
from threading safego.Go through the tool-dispatch/continuation paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 20:05:19 +02:00
be3ce761d4 feat(tasks): phase 4 — structured plan steps (set_goal/propose_plan/update_plan_step)
Gives a task a legible, live-advancing plan via three more nomos-local tools:

- set_goal(goal): records the task goal, status → planning, emits goal.set.
- propose_plan(steps[]): persists ordered steps (clean replace for v1 — a
  revision starts a new list), status → executing, emits plan.proposed with
  the persisted steps (id+seq) so the panel can address them.
- update_plan_step(seq, status, execution_id?): advances a step, stamping
  started_at/finished_at, emits plan.step.started/finished. Anchors the event
  to the step's target entity when it has one.

Belt-and-suspenders: when an execution linked to a step reaches a terminal
state, the api auto-closes the step (closePlanStepForExecution in
emitExecutionEvent) and emits plan.step.finished — so the board stays honest
even if the agent forgets to close a step it started.

Verified end-to-end: a goal-driven task fired goal.set → plan.proposed →
2× step.started/finished → task.status on the SSE stream; both steps persisted
done with start/finish timestamps; status progressed planning→executing→done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 12:50:27 +02:00
2e922f6421 feat: decompose pct_create into atomic create + agent-driven install; add scoped destructive window
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Closes the two remaining open points from the auto-continuation work.

1. Atomic pct_create (observability, the bigger of the two):
   pct_create used to bundle create + apt install + post_install script into
   one black-box multi-minute SSH call — the agent got back a single opaque
   success/fail with no way to see (or fix) which step actually broke.
   Removed the whole post-create provisioning block (and the now-dead
   provisionScript/sanitizePkgs helpers + their tests). pct_create is now
   create + start + register ONLY — fast, and its result is fed back to the
   agent via auto-continuation almost immediately. The agent installs
   packages and runs setup as its OWN sequence of `run` calls against the new
   lxc:<hostname>, observing each command's real output and able to diagnose
   and retry exactly the step that failed — the same recovery loop already
   proven for the general case, now applied to installs too, instead of
   requiring a separate black-box mechanism.
   - services/post_install removed from the pct_create params struct and
     from the MCP tool schema/SOUL.md docs.
   - SOUL.md: explains the new flow, moves the Docker CLI gotcha and DNS
     troubleshooting guidance to be steps the agent runs itself.

2. Scoped destructive window (targeted autonomy for recovery):
   Verified live in the previous session that a destructive recovery (a
   failed destroy needing stop-then-destroy on the same container) required
   TWO separate typed confirmations for what was clearly one recovery
   action. Added a narrow, TARGET-scoped 15-minute grant
   (destructive_window.agent:<id>.target:<slug> in autonomy_settings,
   shared key format across cmd/nomos and internal/mcp) that opens only
   after an EXPLICIT typed confirmation (never loose assent) or an explicit
   button-approval of a destructive step, and only ever covers further
   destructive commands against that SAME target. A different target always
   needs its own fresh confirmation — this narrows risk instead of loosening
   it globally, unlike broadening the general assent window to cover
   destructive actions would have.
   - cmd/nomos/store.go: openDestructiveWindow/destructiveWindowActive/
     executionTarget.
   - cmd/nomos/agent.go: opens the window when a typed confirmation grants a
     destructive chat-assent execution.
   - internal/mcp/server.go: `run` tool checks the window before gating a
     destructive command; auto-runs if active.
   - internal/httpapi/phase3.go: DecideApproval opens the same window when a
     destructive execution is approved via the button/API, for parity with
     the chat-assent path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 14:45:09 +02:00
d2f749d33d feat: event-driven auto-continuation — agent runs an approved plan to completion
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
The root cause behind "the agent stops at the first error and doesn't recover":
provisioning executions run ASYNCHRONOUSLY (pct_create fires the SSH work in a
goroutine and returns "running" immediately), so the agent's turn ENDS before
the result exists. The agent literally isn't running when the step fails — it
can't react to a failure it never observes. The only thing that fed results
back was the operator typing "continue" after every async step: the human was
the event loop. (In the flagged 18-message session the operator typed
continue/proceed/?? eight times while the agent correctly diagnosed each failure
but couldn't advance a step on its own.)

This makes the system the event loop instead:

- migrations/017: nomos_plan_executions links each gated execution to the chat
  session that started it.
- cmd/nomos: after a tool result, any "execution <uuid>" it started is linked
  to the session. A background worker (continue.go) polls for those executions
  reaching a terminal state and — while the agent has an open assent window (an
  approved plan is in flight) — re-invokes the agent with the result
  ("execution X completed/failed: <result>"), so it proceeds to the next step
  or diagnoses+fixes the failure, with no operator tick. Guarded against loops
  (mark-continued before running) and bounded by the 30-min window.
- chatWith(): chat() variant that injects the finished-execution note after
  replayed history without persisting a fake user turn.
- DecideApproval: approving a step by ANY route (button or chat-assent) now
  opens the assent window, so auto-continuation works regardless of how the
  operator approved — previously only typing "go ahead" opened it.
- SOUL: the agent is told it will be auto-re-invoked when async steps finish —
  don't poll get_execution_status, don't wait for "continue"; end the turn and
  keep going step by step until the goal is verified or a genuine blocker.

This is the root fix, not another per-command patch: you can't enumerate every
failure of an unbounded action space, but you can give the agent a loop that
observes each result and adapts — because "do anything" always includes "the
first attempt failed."

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 14:19:43 +02:00
7a7ce2b89b fix: gateway pre-flight check could never actually fail
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Verified live that after deploying the "fixed" bridge-bound pre-flight, it
still let a known-bad vmbr0+192.168.8.2 config straight through to a full
pct_create with no error. Root cause: the check used
`strings.Contains(pingOut, "REACHABLE")` against markers "REACHABLE" /
"UNREACHABLE" — but "UNREACHABLE" contains "REACHABLE" as a substring, so the
containment check was true for BOTH outcomes. The pre-flight was structurally
incapable of ever failing, regardless of the actual ping result.

Fixed with distinct, non-overlapping markers (PREFLIGHT_OK/PREFLIGHT_FAIL)
and exact-match comparison, pulled into a small gatewayPreflightPassed()
helper with a unit test asserting the exact historical bug case
("UNREACHABLE" must be false) so this bug class can't silently recur.

Re-verified live end-to-end: manually re-tested the exact ping command
(confirmed UNREACHABLE via vmbr0), and this was caught only by actually
running the check against production, not by reading the code.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 12:02:42 +02:00
3f3de18b23 fix: pre-flight gateway ping must bind to the specific bridge, not the host default route
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Verified live that the pre-flight check added in the previous commit had a
real gap: a plain `ping <gateway>` from the Proxmox host succeeds via the
HOST's own routing table (which can have routes to a subnet through paths
the host alone knows about), even when the CONTAINER — attached via a plain
bridge with only a naive on-link default route — can never actually ARP that
gateway. Confirmed by creating a real test container on vmbr0 with
gw=192.168.8.2: the host-wide ping had said "reachable," but pinging from
inside the container showed 100% packet loss. Fixed by binding the pre-flight
ping to the specific requested bridge (`ping -I <bridge>`), which correctly
rejects vmbr0 for that gateway instead of false-positiving via the host's
broader routing table.

Also confirmed live: vmbr1 does exist and is up on strong (contrary to the
possibly-stale host doc), matching what romm/seanime's docs already said.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 11:55:53 +02:00
82b0ad2298 fix: pct_create fast gateway pre-flight + bridge param (real root cause of TypeType's DNS failures)
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Investigated why the operator couldn't get past "no DNS/connectivity" across
multiple retries even after Nomos correctly diagnosed and fixed the gateway
(192.168.8.1 -> 192.168.8.2). It still failed. Root cause, confirmed from
strong's own documented network topology: on `strong`, vmbr0 physically
bridges only to 192.168.178.0/24 — the 192.168.8.0/24 service network is
reached via a Fritz!Box static route, not a local bridge. A container
attached to vmbr0 can never reach a 192.168.8.x gateway no matter which
address in that range is picked; ARP for it just gets silently dropped
(matching the earlier hang symptom). The gateway was never the problem — the
bridge was. 192.168.8.0/24 is also segmented into /28 blocks each with their
own gateway (192.168.8.2 is only the .0-.15 block's gateway), so even a
correct bridge with a copy-pasted gateway from a different block would still
fail.

No amount of retrying with a different gateway guess could have fixed this —
the missing fact (which bridge reaches which subnet, and the per-/28 gateway)
isn't inferable from the subnet alone.

- pct_create gets a `bridge` param (was hardcoded to vmbr0) so a correct
  bridge can actually be requested once known.
- Fast pre-flight: for any static IP, ping the gateway from the target HOST
  before creating anything. Was: a bad config took a multi-minute hang (or,
  after last commit's timeout fix, ~2min) before failing. Now: ~2 seconds,
  with a message that explicitly says not to guess a different gateway in
  the same subnet — find a real neighbor's config or use DHCP.
- SOUL.md: DHCP is now framed as the default, not a fallback; static IP
  requires finding an existing LXC on the same host in the same /28 and
  copying its bridge+gateway verbatim — inventing one is explicitly called
  out as the failure mode that caused this exact incident.
- MCP tool schema: pct_create's params description now documents `bridge`
  and the neighbor-copy rule directly in what the model reads at call time,
  not just in SOUL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 11:48:06 +02:00
8950bada44 fix: sshExec had no timeout — a hung remote command blocked forever
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Root cause of "running for 10+ minutes without stopping": a real production
execution (TypeType pct_create) was found genuinely stuck 17+ minutes into a
single blocking SSH call. The container's post_install script was looping on
`getent hosts deb.debian.org`, waiting on a network that could never come up
— the operator's static IP config used gw:192.168.8.1, but the actual gateway
on that subnet is 192.168.8.2, so every network call hung instead of failing
fast (packets dropped, not rejected).

Two compounding bugs made this unrecoverable without manual intervention:

1. sshExec (both internal/httpapi/phase3.go and internal/mcp/server.go) had
   NO execution timeout — `session.CombinedOutput()` blocks until the remote
   command exits, with no deadline. A hung remote process blocks the Go
   goroutine forever; the execution can never leave 'running', and the
   operator has no way to make it stop. Fixed: both now race the SSH call
   against a 10-minute hard timeout, closing the session/client and
   returning a clear "timed out after 10m0s" error if exceeded. (The
   mcp/server.go copy also still had the original "swallowed non-zero exit"
   bug from before that fix was applied to httpapi's copy only — fixed here
   too.)

2. provisionScript's DNS-wait loop assumed `getent hosts` fails fast on no
   connectivity — it doesn't; a black-holed network can make each call hang
   far past the resolver's nominal timeout, so the documented "~90s" budget
   was never real. Wrapped every attempt in `timeout 3` so the wall-clock
   budget is now actually enforced (~2min worst case), and the failure
   message now suggests checking the net0 gateway.

Also fixes the matching UI-side gap (operator's literal question: "is there
a way to get more details? it has been running for 10+ minutes without
stopping"):

- InlineApproval's track() polling loop had its own ~6min ceiling and simply
  STOPPED polling after that — silently going stale before the backend (now
  correctly capped at 10min) could ever resolve. Raised to a 14min ceiling
  with margin, and added a distinct 'stalled' state if that's ever exceeded
  (explicitly says something's wrong, rather than freezing silently).
- The running-card now shows live elapsed time (ticking, from the
  execution's created_at), the actual command being run, and the execution
  ID — previously just a static "this can take a minute" with zero
  information. Also added command display to the destructive pending-
  approval card for full transparency before confirming.

Verified live end-to-end in a real browser (dev server proxying to
production): queued a real command via chat, approved via the button,
watched the elapsed-time counter tick in real time, and saw it transition to
a completed card with real output once the command finished.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 10:45:47 +02:00
d08a985ea9 fix: execution slug collision under back-to-back requests
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Live testing hit `entities_slug_key` violations: exec slugs used an 8-char
prefix of a UUIDv7, whose leading bytes encode a millisecond timestamp — two
executions created seconds apart can share a prefix. Use the full UUID
(guaranteed unique) for the exec entity's slug/name in request_execution, the
new `run` tool, and the REST RequestExecution handler — all three had the
same pattern.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 09:37:08 +02:00
9539759db6 fix: NULL-scan bug in LXC target resolution for entities without a host attribute
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Found live: `run` against lxc:caddy failed with "missing pve_id" even though
pve_id=121 was present — caddy is an inventory-seeded LXC with no `host`
attribute at all (only pct_create-provisioned LXCs set one). The combined
query scanned attributes->>'host' (SQL NULL) into a plain Go string, which
errors the whole Scan — including the pve_id column that scanned fine.
COALESCE the host column to '' so a missing host attribute degrades to the
documented default instead of failing the whole resolution.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 09:32:45 +02:00
d52968876a feat: general gated run primitive + chat-assent approval (Layer 0)
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Implements the first slice of plans/2026-07-10-general-gated-execution.md:
Nomos gets one general execution tool instead of only a fixed action enum,
gated by an automatic risk classifier, and approval can be granted by the
operator just replying in chat instead of clicking a button.

- internal/policy/command.go: ClassifyCommand(cmd, declaredRisk) — rule-based
  read-only allowlist + destructive denylist, default-escalate to
  config_mutation for anything else. Classification can only ESCALATE the
  caller's declared risk, never de-escalate it (destructive always wins even
  if declared read_only). Compound commands (&&, ;, |, $()) never qualify for
  the read-only fast path. Full test corpus.
- internal/mcp/server.go: new `run` MCP tool — target (host:/lxc:), command,
  purpose, optional declared_risk. Read-only commands execute immediately;
  everything else queues an approval exactly like pct_create today, executed
  via httpapi's existing executeApprovedAction. Also fixes a real latent bug:
  pct_exec resolved an LXC's host attribute without the "host:" prefix, so it
  could never find the Proxmox host — new resolveExecTarget/resolveRunTarget
  helpers (mcp + httpapi) fix this for both the new `run` action and existing
  actions that route through the same execution path.
- internal/httpapi/phase3.go: "run" case in executeApprovedAction; fixes two
  bugs found while wiring this up — (1) DecideApproval hardcoded risk_class to
  'config_mutation' on every approve, silently corrupting the audit ledger for
  every other risk class; (2) denying/revoking an approval never updated the
  linked execution's status, so it stayed 'pending_approval' forever instead
  of reflecting the decision.
- cmd/nomos/assent.go: deterministic (not LLM-judged) chat-assent detection.
  Scoped to the immediately-preceding assistant turn's pending approvals only
  — an old "yes" can't retroactively approve something new. Destructive-risk
  actions are excluded from loose assent. Approves via the same HTTP decision
  endpoint the UI button calls, so both paths share one audit trail.
- web/.../InlineApproval.svelte: self-healing poll — a pending approval card
  now picks up being decided via ANY path (chat assent, Ops page, Matrix), not
  just its own button. Previously the banner stayed stuck showing
  Approve/Deny even after the action had already run elsewhere.
- nomos/SOUL.md: `run` is now the general capability ("no fixed menu, only a
  risk gate"); documents chat-assent behavior and the destructive exception.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 09:28:00 +02:00
5f888e6386 fix: set C.UTF-8 locale in provisioning script
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Fresh Debian LXCs have no locale configured, spamming "apt-listchanges:
Can't set locale" / perl warnings across every install and breaking some
packages' post-install scripts. Pin LANG/LC_ALL=C.UTF-8 (and hoist
DEBIAN_FRONTEND) at the top of the in-container bootstrap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 01:21:23 +02:00
f248508919 feat: robust provisioning (DNS self-heal) + live execution feedback in chat
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Production session provisioned the container but the service never installed:
apt failed with "Temporary failure resolving deb.debian.org" — a static-IP LXC
whose assigned nameserver couldn't resolve. The operator also got zero feedback:
the approval banner just sat there with no running/complete/failed status.

Backend robustness (provisionScript):
- Wait for real DNS/connectivity inside the container before apt, and self-heal
  /etc/resolv.conf to a public resolver (1.1.1.1/8.8.8.8) if the assigned one
  is dead. `set -e` after the gate so apt/post_install failures surface.
- apt-get update/install with Acquire::Retries=3.

Frontend feedback (InlineApproval):
- After approve, poll GET /executions/{id} and show live phase: submitting →
  provisioning… → provisioned successfully / execution failed (with the error).
- add getExecution() to api.ts.

Agent guidance (SOUL.md):
- omit vmid (auto-assigned), prefer dhcp, docker-compose-plugin is not in Debian
  (use docker.io + get.docker.com), end post_install with a health check.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 01:15:21 +02:00
a1f666f68a fix: build execution result JSON via json.Marshal (was stuck at 'approved')
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
The final "UPDATE executions SET result=$::jsonb" built its payload with
fmt.Sprintf and only escaped newlines. apt/pct output contains quotes,
backslashes and control chars, so the payload was invalid JSON, the jsonb
cast failed, and the (unchecked) UPDATE was silently discarded — the
execution stayed 'approved' with a NULL result even though the LXC was fully
provisioned (verified live: vmid auto-assigned, container running, service
installed, post_install ran).

- executeApprovedAction: marshal result via json.Marshal; log UPDATE errors
- add jsonErr() helper; route all pct_create failure-path results through it
- mcp/server.go: add jsonOut() for restart/systemctl/pct_exec inline results
- regression test for JSON validity on quote/backslash/control-char output

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 00:58:43 +02:00
ac86302f52 fix: pct_create make vmid optional, fix dhcp+gw, longer boot settle
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Follow-ups found while verifying the approve→provision path end to end:
- vmid is now optional: the early required-field check rejected vmid:0
  before the cluster VMID guard could auto-assign a free id. Only hostname
  is required now; 0 (or a collision) resolves to `pvesh get /cluster/nextid`.
- net0: use ip=dhcp with no gateway when no static IP is given (Proxmox
  rejects gw alongside dhcp); only attach gw for a static CIDR.
- bump post-create settle to 10s so a DHCP lease is up before apt runs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 00:50:54 +02:00
8ed2b88495 fix: pct_create false-success, VMID collision, and stuck approval banner
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Real production failure when the operator clicked Approve in chat: nothing
provisioned, banner never cleared, execution marked completed.

Three root causes:
- sshExec swallowed non-zero exits when the command produced output, so a
  `pct create` that printed "CT 132 already exists" and failed was reported
  as success and a bogus lxc entity was registered. Now any non-zero exit
  returns an error (with output) so the execution is correctly marked failed.
- The LLM reused VMID 132 (belongs to lxc:rclone; VMIDs are cluster-wide).
  pct_create now checks in-use VMIDs via `pvesh get /cluster/resources` and
  falls back to `pvesh get /cluster/nextid` when the requested id is taken.
- InlineApproval.svelte reset its state on every prop change (done was also
  compared against the wrong string), so the banner never cleared and each
  click re-POSTed /decision. Rewritten to track outcome per executionId,
  clear on success, and block resubmits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 00:44:51 +02:00
b37f85ae08 fix: make Nomos actually provision LXCs from chat (pct_create + web fetch)
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Root cause of "asks permission but never acts": the approved pct_create
execution failed to parse because the LLM emitted `"privileged":0` /
`"nesting":1` (numbers) into strict `bool` fields, so the container was
never created. Compounded by a hardcoded template name (debian-13.0-1)
that no longer exists on the host, and no way for the agent to read the web.

- flexBool: accept 0/1, "true", bool for privileged/nesting (the exact prod failure)
- pct_create template pre-flight: list host cache, validate/auto-pick newest debian
- pct_create services[] + post_install: one approval provisions a working service
- new http_get MCP tool (sanitized, size-capped, SSRF-guarded) — agent can read repos/sites
- request_execution description: target=host, full JSON schema + example
- SOUL.md: agent CAN fetch the web; prefer one-step provisioning
- default model deepseek-v4-flash -> v4-pro; maxIterations 15 -> 25
- unit tests for flexBool, template resolve, pkg sanitize, HTML sanitize + SSRF block

Verified live on host:strong with a throwaway VMID 999: template auto-resolved,
container created + booted, services installed, post_install ran, then destroyed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 00:28:56 +02:00
ea62d744ed feat: pct_create action, ToolCallGroup collapse+animation, inline chat approval
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- Add pct_create to request_execution (MCP) and executeApprovedAction (httpapi)
  Parses JSON config: vmid, hostname, cores, memory, disk_gb, ip, gw, storage,
  template, privileged, nesting, mounts, nameserver, searchdomain. Creates
  entity (state=provisioning), hosts relationship, entity_status on success.
  Fixes action string parsing to use Index instead of SplitN (colons in JSON).

- Rewrite ToolCallGroup.svelte: bits-ui Collapsible replaces native <details>.
  Collapsed by default. Animated header shows live tool count + running tool
  name while streaming. Auto-expands during streaming, auto-collapses on done.

- Add InlineApproval component: parses 'execution UUID queued' from agent
  response, renders Approve/Deny buttons inline in chat, calls decideApproval.

- Document pct_create in nomos/SOUL.md with params, risk class, and approval flow.

- Add session-review skill at .agents/skills/session-review/SKILL.md.

- Add plan: 2026-07-09-session-execution-and-ux-fixes.md.
2026-07-09 11:15:28 +02:00
279549c8c9 fix: scheduler wrote health/metrics/events to probe entities, not targets
Problem: every host/service/lxc/etc. entity_status row was permanently
stuck at 'unknown' since creation. Verified against the live DB:
metric_samples had 17,559 rows, 100% attached to type='check' probe
entities and 0% to any real monitored entity; only 25 check entities
ever had real health written. check_defs.entity_id (the probe's own
bookkeeping entity) and check_defs.target_id (the host/service actually
being observed) were both real fields, but the scheduler wrote
UpsertEntityStatus/InsertMetricSample/emitSchedulerEvent keyed by
entity_id instead of target_id — so every check ran and every result
was real, it just landed on the wrong row. This is the mechanism behind
observed drift: the agent's dashboard/health tools reported the
internal probes' state, never the actual fleet.

Change:
- scheduler.go: runCheck/resolveSignal now resolve targetID from
  cd.TargetID (falling back to the check's own id if unset) and write
  status/metrics/events there. Signals stay keyed by the check entity,
  unchanged, matching their existing resolution logic.
- Added a staleness sweep to housekeeping(): an entity whose last
  observation is older than 3x its fastest enabled check's interval
  (floor 5m) is marked 'stale' and emits health.stale, so a stalled
  scheduler or disabled check_def can no longer look like current data
  forever.
- migrations/016: deletes the now-orphaned check-entity entity_status
  rows so dashboard/fleet-health rollups stop double-counting probes as
  monitored entities. Historical metric_samples on check entities are
  left as-is (time-series data, not safe to reattribute).
- openapi.yaml + regenerated gen code: Entity gains health/last_check_at;
  'stale' added to the health enum everywhere it's used.
- dashboard.go / GetFleetHealth / nomos's get_health_summary MCP tool:
  exclude type='check' entities from rollups.
- nomos/agent.go: replay prior turns' tool_use/tool_result pairs into
  the conversation instead of dropping them (previously only final text
  was replayed, forcing the agent to re-derive fleet state every turn),
  and inject a compact live fleet-health snapshot into the system prompt
  each turn so it starts oriented instead of spending an iteration on
  discovery.

Risk: config_mutation (schema-adjacent — new migration, no destructive
DDL, additive DELETE only on orphaned rows). No behavior change until
oikos-api/oikos-scheduler/nomos are rebuilt and redeployed.

Verification: go build/vet clean across the repo. Ran this worktree's
own API binary against the live dev Postgres on an alternate port
(read-only from the live containers' perspective) and confirmed
/api/v1/entities now returns health/last_check_at, and the dashboard
health rollup dropped from double-counting to an honest 168 unmonitored
entities (matches reality pre-deploy — the live scheduler hasn't run
the fixed code yet). Confirmed check_defs.target_id correctly maps
multiple checks to host:hubris via direct psql query.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-09 00:26:04 +02:00
d45f2326b6 fix: CreateCheck uses type "check" not "check_def"
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
FK violation — entity_types table has name "check" but
phase3.go:320 was inserting type "check_def" causing:
entities_type_fkey (SQLSTATE 23503)
2026-07-08 21:21:44 +02:00
2908b0a377 feat(ui): M4 — agent activity, knowledge search, audit, correlation grouping
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Add three new pages completing the control-room web UI:
- Agent activity: polls /agent-activity every 5s, filterable by type/agent
- Knowledge search: FTS over /knowledge/search with snippet + entity links
- Audit trail: browseable audit log with actor/action/entity filters

Enhanced live events page with correlation-id clustering (Groups toggle).
Added fetchAgentActivity/searchKnowledge/fetchAudit to the API client.
11 nav items now cover all planned control-room views.
2026-07-08 17:02:09 +02:00
e8e230b4a5 nomos+web: streaming, provider routing, event gap-fill, embedded UI; fix approval FK & session context
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Agent (cmd/nomos):
- Stream LLM tokens via NewStreaming; emit text_delta then final text.
- OpenRouter provider routing: data_collection=deny (ZDR) + require_parameters;
  NOMOS_PROVIDER_SORT opt-in; Exacto via model suffix.
- Multi-turn: reload session history into context; UI passes session id.
- Fix agent_activity logging (agent_id/session_id) and mcpClient data race.

Events (live control-room feed):
- approval.created (mcp), approval.decided (api), execution.completed/failed
  (approved-action path), signal.raised/resolved + health.changed (scheduler,
  transition-gated).

Fixes:
- createApproval FK violation (reuse execution entity) — the agent's only
  write path; log the previously-swallowed errors.

Web UI:
- Embed web/dist via //go:embed (single binary); Dockerfile builds SPA into
  the Go stage; committed .gitkeep placeholder keeps backend-only builds green.
- Caddy: Authentik-gated /agent/* -> nomos so the UI reaches the agent
  same-origin in production.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 15:22:27 +02:00
7c6cffb5f5 complete MCP tool surface — Matrix approval webhook loop + token verification
Plan #6 (MCP Tool Completion / bin/homelab Migration) done.

- Approval records created for gated request_execution actions
- Notifier sends Matrix messages with HMAC approval tokens
- Stores matrix_event_id, polls /relations/{id}/m.annotation for /
- Reaction detection triggers DecideApproval API call
- Token verification added to DecideApproval endpoint
- Migration 013: matrix_event_id + alert_sent_at on approvals
- AGENTS.md: 21-tool surface documented, stale homelab CLI refs removed
- Plan index updated, audit cross-reference refreshed
2026-07-08 11:02:06 +02:00
6b75f7302d db as source of truth: wiki→seeds, archive old artifacts, knowledge ingestion
- Migrations 010 (content_hash) + 011 (search tsvector column)
- new: internal/knowledge/seed.go — knowledge seed ingest engine
- new: internal/httpapi/knowledge.go — SearchKnowledge + GetEntityKnowledge
- wire knowledge ingest into oikos seed pipeline
- convert all 36 wiki docs + 6 investigations + 12 runbooks → seeds/knowledge.yaml
- archive: knowledge/wiki/→archive/, oikos/cards/→archive/, .hermes/plans/→archive/
- delete: 9 superseded Python kernel files, ledger/, mcp/build_host_files.py
- remove empty knowledge/ directory tree
2026-07-07 20:22:30 +02:00
74a6b6bb18 phase 4: fix execution FK violation — create entity row before insert
- phase3.go: RequestExecution now calls InsertEntity before InsertExecution
  (executions.entity_id references entities.id via FK constraint).
- mcp/server.go: request_execution MCP tool same fix — inserts entities row
  with slug 'exec:<target>:<id8>' before executions insert.
- docker-compose.yml: fix seed OIKOS_SEEDS_DIR from /app/seeds to /seeds
  (distroless image COPY destination).
2026-07-07 16:45:28 +02:00
3823a82417 phase 4: hermes agent — MCP tools, activity logging, 'all' role, wiring fixes
- mcp/server.go: 7 new tools (get_signal_history, get_patterns, get_skills,
  request_execution, get_trend, get_event_timeline, get_agent_activity),
  agent_activity logging middleware on every tool call.
- phase3.go: QueryAgentActivity REST handler implemented (was stub).
  Fixed scan count mismatch in ListSkills/PatchSkill/ListSkillVersions
  (13 cols → 12 targets). Fixed AgentActivity cursor pagination
  (lexicographic → integer comparison). Fixed s.Slug → s.Name in log.
- cmd/oikos/main.go: 'all' role now runs api + scheduler + notifier in
  one process. Replaced nil SchedulerRunner/NotifierRunner with direct
  scheduler.RunnerForMain() / notifier.RunnerForMain() imports.
  Added runWithPool helper for standalone scheduler/notifier roles.
- internal/config/config.go: added HermesAgentID env var.
- internal/httpapi/server.go: pass HermesAgentID to MCP handler.
- docker-compose.yml: added scheduler and notifier services (dev profile).
- hermes/: config.yaml, SOUL.md, skills/homelab-ops/SKILL.md.
- Cleaned up: scheduler/init.go dead code, mcp/server.go pgx import guard.
2026-07-07 16:07:08 +02:00
aa197190cd phase 3 review: fix broken error classification, stub checks, wasted uuid, token idempotency, dead code
- actuator/ssh.go: custom errorsAs chain broken — all SSH errors classified as SSHErrorOther.
  Replaced with standard errors.As + errors.Is.
- scheduler/scheduler.go: all four check functions were stubs returning healthy.
  Implemented real HTTP GET, TCP dial, unix.Statfs disk, and TLS cert expiry checks.
- learning/learning.go: uuid.NewV7() called unconditionally before ON CONFLICT upsert.
  Now looks up existing pattern first, reuses entity_id.
- notifier/notifier.go: removed dead var_, fixed token regeneration every 15s.
  Now skips if token_hash already set.
- phase3.go: removed dead GetPattern+dummy args call in PatchPattern.
- classify.go: removed unused var_ guard.
2026-07-07 15:27:31 +02:00
095a3967c4 phase 3: control loop — scheduler, actuator, learning, notifier, policy, API endpoints
Implemented the full OODA control loop:

Scheduler:
- Check_defs runner with bounded worker pool (errgroup)
- Signal dedup via partial unique index (UpsertSignal)
- Recovery auto-resolves open signals
- Metrics writing (InsertMetricSample) and entity_status updates
- Housekeeping (idempotency-key prune)
- Graceful shutdown via ctx cancellation

Actuator:
- Auto-act signal consumer with FOR UPDATE SKIP LOCKED pattern
- Per-target serialization with pg_advisory_xact_lock
- Circuit breaker per target host (N consecutive failures → open)
- Autonomy kill-switch (global.auto_act, never_auto_act.<slug>)
- Execution record lifecycle (proposed → running → completed)

Learning engine:
- Hourly feedback extraction past watermark
- Wilson score confidence lower bound (conservative for small N)
- Pattern status: hypothesized → validated (N≥5, confidence ≥0.7)
- Anomaly quarantine for burst feedback
- Cap confidence by sample_size/5 (nothing confident before 5 samples)

Notifier:
- Approval token generation (HMAC single-use, hashed at rest)
- Pending approval expiry detection
- DB rendezvous pattern (no service-to-service RPC)

Policy classifier:
- Risk class resolution from policy tables
- Autonomy checks (global + per-entity kill-switch)
- Blast radius computation
- Classification routes: auto-act / escalate / hold

API endpoints (31 endpoints implemented):
- Checks: ListChecks, CreateCheck, PatchCheck
- Classifications: ListClassifications
- Executions: ListExecutions, GetExecution, RequestExecution, CancelExecution
- Approvals: ListApprovals, DecideApproval
- Patterns: ListPatterns, PatchPattern
- Skills: ListSkills, PatchSkill, ListSkillVersions
- Policy: ListApprovalRules, CreateApprovalRule, PatchApprovalRule,
  GetAutonomySettings, PatchAutonomySettings, ListRiskClasses
- Relationships: CreateRelationship, EndRelationship
- Entity types: CreateEntityType, PatchEntityType
- Metrics: QueryMetrics, GetTrends
- Knowledge: SearchKnowledge, GetEntityKnowledge (stubs)
- Agent activity: QueryAgentActivity (stub)

Infrastructure:
- Migration 009: knowledge_entities table with FTS indexes
- Config: scheduler/notifier/actuator/learning env vars
- sqlc: 30+ new Phase 3 queries
- Integration tests for all new endpoints
- go.sum updated with golang.org/x/sync
2026-07-07 15:19:25 +02:00