Gives a task a legible, live-advancing plan via three more nomos-local tools:
- set_goal(goal): records the task goal, status → planning, emits goal.set.
- propose_plan(steps[]): persists ordered steps (clean replace for v1 — a
revision starts a new list), status → executing, emits plan.proposed with
the persisted steps (id+seq) so the panel can address them.
- update_plan_step(seq, status, execution_id?): advances a step, stamping
started_at/finished_at, emits plan.step.started/finished. Anchors the event
to the step's target entity when it has one.
Belt-and-suspenders: when an execution linked to a step reaches a terminal
state, the api auto-closes the step (closePlanStepForExecution in
emitExecutionEvent) and emits plan.step.finished — so the board stays honest
even if the agent forgets to close a step it started.
Verified end-to-end: a goal-driven task fired goal.set → plan.proposed →
2× step.started/finished → task.status on the SSE stream; both steps persisted
done with start/finish timestamps; status progressed planning→executing→done.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the compounding knowledge loop the task model is built around:
- complete_task(outcome, summary): a nomos-LOCAL, session-scoped tool (the
shared MCP server has no session id). Introduces the local-tool mechanism —
buildTools appends task tools, the agent loop routes them to handleTaskTool
instead of the MCP client. Sets the task's terminal status/outcome/summary,
mirrors it onto the task entity, and emits task.status.
- Knowledge → task linkage: after a successful upsert_knowledge in a task,
nomos links the note to the task entity (documents) and emits
knowledge.recorded, so the task's outcome view shows what it learned. The
note's about-link to the involved entity (written by upsert_knowledge) is the
retrieval path future tasks use.
- SOUL: every chat is a task loop — retrieve prior knowledge FIRST
(get_entity_knowledge on the target), plan, execute, record learnings, then
complete_task. Scales down for trivial read-only tasks.
- deleteSession now cleans up the task entity, its relationships, and its
task-scoped events (was orphaning them); the knowledge doc itself and its
about-links survive, as knowledge should outlive the task.
Verified end-to-end: a task recorded a note and completed; task.status +
knowledge.recorded hit the SSE stream; status=done/outcome=success persisted;
the note linked to both lxc:caddy (retrieval) and the task; a future
get_entity_knowledge(lxc:caddy) surfaces it; delete cleaned edges+events (0/0/0)
while the knowledge survived.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
As the agent runs a task, record which entities each tool call references:
write an idempotent task —involves→ entity relationship and publish one
entity.touched event per entity (correlation_id = session, data {slug,tool}).
Extracted from tool ARGS only — never results — so a bulk fleet query can't
drag every entity into the task graph; bulk/no-slug tools stay silent.
Emitted from the nomos agent loop rather than the shared MCP wrapper, which
has no session id. The involves edges make a task's graph neighborhood its
involved-entity set (queryable via get_relations) — the substrate for the
knowledge loop; the events are the live pulse the context panel consumes in
phase 6.
Verified end-to-end on the local stack: a chat referencing lxc:caddy/lxc:gitea
produced entity.touched on the browser SSE stream with slug+tool+correlation,
and exactly one involves edge per entity despite repeated touches.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migration 018 adds goal/status/outcome/summary/entity_id to agent_sessions
and creates session_plan_steps + session_questions. Registers a 'task'
entity type and an 'involves' (task→entity) relationship in the ontology
so each session anchors its knowledge and involved-entity edges on the
existing relationships graph.
nomos createSession now mints a task:<session-id> entity (type task) and
links it via agent_sessions.entity_id — best-effort so chat never blocks on
it. listSessions/GET /sessions surface the new task fields.
No behaviour change yet; this is the data foundation for the task board and
live context panel. Verified end-to-end against the local stack: migration
applied, ontology ingested (60 types/47 rels), a new session mints a linked
task entity and the API returns status/goal/entity_id.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reframes the chat surface as a board of tasks: each task carries a goal,
a plan approved once, a lifecycle status, an outcome, and a knowledge
loop that links learnings to the involved entities (and the task entity
itself) via relationships so future tasks compound. Supersedes the
sidebar-only framing and the free-form chat portion of the control-room
web UI plan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audited all 10 active plan docs against the codebase (not just commit
titles). 5 were fully shipped and stale-tagged "Planned"/"In Progress" —
moved to done/ with verification notes. The other 4 got corrected
Planned→In Progress status plus concrete remaining-gap notes so the next
pass doesn't re-derive what's already done.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The plan's "learning view" (runbook success-rate trends, promoted
skills, capability timeline) assumes the patterns/skills/feedback
pipeline is populated. It isn't: all three tables are empty in
production and nothing in the codebase ever writes to feedback, so
building the UI against them today would ship a permanently-empty
page. Scoped instead around data that's real and growing —
executions — while still wiring up /patterns and /skills so the page
needs no rework once that pipeline exists.
New /api/v1/learning/timeline (per-verb first-success date + success
rate, parsed via the existing splitAction helper) and
/api/v1/learning/trend (30-day daily success/fail counts), both
read-only queries against executions. Patterns and skills sections
call the existing (untouched) ListPatterns/ListSkills endpoints and
render an explanatory empty state instead of nothing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found live: a chat request to restart caddy (the reverse proxy for the
whole fleet) executed instantly over SSH with zero approval. Root
cause was in request_execution's legacy handler — restart, pct_exec,
and systemctl (outside enable/disable) executed immediately with a
hardcoded risk_class='reversible_low' that was never actually checked
against anything, bypassing the classifier entirely. Only the `run`
tool's commands were ever gated.
Extracted the run tool's classify -> execute-or-queue logic into a
shared classifyAndGate() and route restart/pct_exec/systemctl through
it too, so every mutating path — regardless of which tool the model
reaches for — gets the same read-only/config-mutation/destructive
classification and approval gate. systemctl restart is already covered
by an existing classifier test (config_mutation), so no new test
needed; the gap was that request_execution never called the
classifier at all.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Pending-approval cards showed target and risk but not what else the
action would affect — the operator approved config_mutation/destructive
commands blind to downstream impact, even though the graph-walk
(blast_radius() SQL, GetBlastRadius endpoint) already existed and was
just never wired into the approval path.
Fetch it once per pending approval and render "Affects N downstream: …"
on both the normal and destructive approval cards, reusing the existing
fetchBlastRadius() API client function which was already written but
unused anywhere.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ops "Executions" tab showed raw target UUIDs, alphabetical (not
recency) order, and a stale status vocabulary from an earlier schema
iteration — never actually usable as a live "what's happening" view.
Replaced with a new recency-ordered /api/v1/activity/recent endpoint
and matching table (human-readable action summaries, risk/status
badges, duration, inline error preview).
Also added /api/v1/activity/session/{id} + a collapsible SessionDigest
panel in the chat rail, answering "what did this session actually do"
(executions by status, entities touched, knowledge written) — the
missing piece for proactive outcome reporting to be visible in the UI,
not just in the chat transcript.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verified live immediately after deploying: the endpoint returned 200 with
correct-looking stats (total=56, agent_authored=2) but items=[] always,
regardless of limit/source. Root cause: pgx v5 can't scan a timestamptz
column directly into a Go string — Scan() errored on every single row, and
that error was silently swallowed by a bare `continue`, so every row was
dropped with no trace in the logs. Fixed by casting updated_at::text in the
SQL (matching how every other handler in this codebase already returns
timestamps) and logging scan failures instead of swallowing them, so this
class of bug can't hide silently again.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First slice of the observability/learning UI (the "see the system come alive
and learn" ask). The Knowledge page was search-only — blank until you typed —
so the knowledge Nomos now writes via upsert_knowledge was invisible unless
you knew to search for it. Now the page LEADS with what the system knows and
is learning:
- internal/httpapi/knowledge.go: GET /api/v1/knowledge/recent — recency-ordered
knowledge + a stats header (total, agent-authored, learned-this-week,
by-kind). Custom route (not OpenAPI-generated), same auth as the rest.
- web Knowledge page rewrite: stat cards up top (Total / Written by Nomos /
Learned this week / runbooks-investigations), then a "Recently learned" feed
with agent-authored notes highlighted and badged "learned by Nomos", tags,
and relative timestamps. A toggle filters to Nomos-only. Search still works,
now as a mode you enter/clear rather than the whole page.
This turns "the system is getting smarter" from a claim into something you
watch fill up: every gotcha the agent records shows here within seconds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
From the last (successful) TypeType deploy session, two gaps the operator hit:
1. Knowledge write-back — the missing half of the loop.
The agent could read the knowledge base (search_knowledge/get_entity_knowledge)
but had no way to WRITE it, so everything it learned (the Dragonfly memlock
rlimit gotcha, the NAT-hairpin DNS issue, etc.) lived only in an ephemeral
chat message and was lost — the system could never actually "get better."
This is the `upsert_knowledge` MCP tool the 2026-07-08 gaps plan called for.
- internal/mcp/server.go: upsert_knowledge(title, content, about?, tags?,
kind?) writes a document/investigation/runbook entity + knowledge_entities
row (search column is generated), upserts by slug so re-titling updates in
place, and optionally links it to the entity it's about so
get_entity_knowledge surfaces it there.
- SOUL.md: capture non-obvious findings/deploys/gotchas as part of finishing
work, not only when asked "what did we learn".
2. "I had to ask for status multiple times."
The clearest cause: a long working turn (64 tool calls) that exhausted the
iteration cap ended with a bare "max iterations reached without final
answer" — a dead end that forced the operator to ask what happened.
- cmd/nomos/agent.go: on exhaustion, make one final no-tools LLM call
(finalSummary) asking for a status report — what was accomplished, current
state, what remains — so the turn always ends with a real outcome.
- maxIterations 25 -> 40 (the decomposed per-step pct_create flow legitimately
needs more steps).
- SOUL.md: always end a turn with a clear outcome; never end silently or on a
bare tool call — the operator can't see the tools working and reads silence
as "nothing happened".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Operator: "I'd like to be able to see in the chat what the agent is actually
running, right now I just wait while nothing happens." Two compounding gaps:
1. The auto-continuation worker (cmd/nomos/continue.go) had zero live push —
its result only appeared on a manual page reload, so approving a plan and
watching the chat looked completely dead even while the agent was actively
working.
2. Even with polling, continueSession only persisted ONE message at the very
end of a continuation — a continuation that runs several tool calls before
concluding would still show total silence for however long that took.
Fixed both:
- web/src/lib/stores/chat.ts: polls the current session's messages every 3s
between turns (never while a live stream owns the message list) and merges
in anything new. Started after a live turn ends and when a session loads;
stopped on new-chat/session-switch.
- cmd/nomos/store.go: insertMessageReturningID/updateMessage — lets a message
be created as a placeholder and updated in place.
- cmd/nomos/continue.go: continueSession now inserts a placeholder the
instant it starts (renders as the existing "thinking" dots — immediate
feedback that something is happening) and updates that SAME row after
EVERY tool call, not just at the end. A poll within ~3s of any tool call
landing shows it — individual `run` commands appear as the agent issues
them, not just the final rolled-up summary.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous commit fixed a context-cancellation bug in the auto-approve path
and appeared to fix things, but re-testing end-to-end after deploy showed the
execution STILL never completed — just via a different symptom
("no pending execution found for approval" in the logs). Dug further and
found the real, deeper bug underneath: this whole mechanism has never
actually worked.
autoApprove() directly flipped BOTH approvals.status and executions.status to
'approved' via raw SQL, then called executeApprovedViaAPI to POST to the
decision endpoint. But DecideApproval's own logic specifically looks for the
execution still at status='pending_approval' to find and dispatch the real
SSH work (executeApprovedAction) — autoApprove's premature flip meant that
lookup always found zero rows. DecideApproval's UpdateApprovalStatus call
also silently no-ops the same way (sqlc :exec doesn't surface "0 rows
affected" as an error). Every assent-window auto-approved pct_create/
apt_upgrade has been sitting at 'approved' forever with the real work never
triggered — indistinguishable from "still running" until you check.
Fix: remove autoApprove() entirely. Call executeApprovedViaAPI directly
against the untouched pending_approval row from createApproval — identical
to the manual Approve-button path, just without the human click. DecideApproval
is now the single place that transitions status and dispatches, for both the
manual and auto-approved paths, closing the class of bug where two code paths
raced to do the same state transition.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified live testing the new atomic pct_create: an assent-window
auto-approved pct_create appeared to "run" (logged "auto-approved... running
now") but the execution stayed stuck at 'approved' forever. Root cause:
`go executeApprovedViaAPI(ctx, ...)` passed the MCP tool-call's own context —
which is cancelled the instant the triggering /chat request's HTTP response
completes, i.e. on every normal turn. The spawned goroutine's POST to the
approval-decision endpoint died with "context canceled" before it could even
start the real work, and nothing surfaced this to the operator or the agent —
the execution just sat at 'approved' with no error, indistinguishable from
"still running."
This is exactly the context-lifetime bug class httpapi's own approval
goroutine (executeApprovedAction) already avoided by using
context.Background() — it had just been missed in these two call sites
(apt_upgrade and pct_create auto-approve). Fixed both to use
context.Background(), matching the correct pattern already in place
elsewhere. Audited for other goroutines spawned with a request-scoped ctx —
none found; the sshExec internal goroutines are synchronous/waited-on via
select and correctly scoped to the call.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the two remaining open points from the auto-continuation work.
1. Atomic pct_create (observability, the bigger of the two):
pct_create used to bundle create + apt install + post_install script into
one black-box multi-minute SSH call — the agent got back a single opaque
success/fail with no way to see (or fix) which step actually broke.
Removed the whole post-create provisioning block (and the now-dead
provisionScript/sanitizePkgs helpers + their tests). pct_create is now
create + start + register ONLY — fast, and its result is fed back to the
agent via auto-continuation almost immediately. The agent installs
packages and runs setup as its OWN sequence of `run` calls against the new
lxc:<hostname>, observing each command's real output and able to diagnose
and retry exactly the step that failed — the same recovery loop already
proven for the general case, now applied to installs too, instead of
requiring a separate black-box mechanism.
- services/post_install removed from the pct_create params struct and
from the MCP tool schema/SOUL.md docs.
- SOUL.md: explains the new flow, moves the Docker CLI gotcha and DNS
troubleshooting guidance to be steps the agent runs itself.
2. Scoped destructive window (targeted autonomy for recovery):
Verified live in the previous session that a destructive recovery (a
failed destroy needing stop-then-destroy on the same container) required
TWO separate typed confirmations for what was clearly one recovery
action. Added a narrow, TARGET-scoped 15-minute grant
(destructive_window.agent:<id>.target:<slug> in autonomy_settings,
shared key format across cmd/nomos and internal/mcp) that opens only
after an EXPLICIT typed confirmation (never loose assent) or an explicit
button-approval of a destructive step, and only ever covers further
destructive commands against that SAME target. A different target always
needs its own fresh confirmation — this narrows risk instead of loosening
it globally, unlike broadening the general assent window to cover
destructive actions would have.
- cmd/nomos/store.go: openDestructiveWindow/destructiveWindowActive/
executionTarget.
- cmd/nomos/agent.go: opens the window when a typed confirmation grants a
destructive chat-assent execution.
- internal/mcp/server.go: `run` tool checks the window before gating a
destructive command; auto-runs if active.
- internal/httpapi/phase3.go: DecideApproval opens the same window when a
destructive execution is approved via the button/API, for parity with
the chat-assent path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified live: the new auto-continuation worker (previous commit) worked end
to end for the happy path (provision -> auto-verify -> report success, zero
operator ticks). But testing a failure-recovery case (a destroy that failed
because the container was still running) surfaced a real bug: continueSession's
emit closure only captured "text" events, so when chatWith ended the turn on
an "error" event (LLM returned an empty/refusal response, internal retry also
empty), the worker persisted a completely blank, uninformative "auto" message
— no sign anything had gone wrong, undermining observability of the very
mechanism just built.
- Capture "error" events and, if the turn produced no text/tool_calls at all,
persist an explanatory placeholder instead of blank.
- Add one outer retry of the whole chatWith call when the first attempt
produces nothing — the principle behind this whole feature ("don't give up
on the first error") should apply to the continuation mechanism itself, not
just the homelab commands it's continuing.
Also verified live: recovery-from-failure works via the normal chat path once
prompted, and cleaned up the test container.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The root cause behind "the agent stops at the first error and doesn't recover":
provisioning executions run ASYNCHRONOUSLY (pct_create fires the SSH work in a
goroutine and returns "running" immediately), so the agent's turn ENDS before
the result exists. The agent literally isn't running when the step fails — it
can't react to a failure it never observes. The only thing that fed results
back was the operator typing "continue" after every async step: the human was
the event loop. (In the flagged 18-message session the operator typed
continue/proceed/?? eight times while the agent correctly diagnosed each failure
but couldn't advance a step on its own.)
This makes the system the event loop instead:
- migrations/017: nomos_plan_executions links each gated execution to the chat
session that started it.
- cmd/nomos: after a tool result, any "execution <uuid>" it started is linked
to the session. A background worker (continue.go) polls for those executions
reaching a terminal state and — while the agent has an open assent window (an
approved plan is in flight) — re-invokes the agent with the result
("execution X completed/failed: <result>"), so it proceeds to the next step
or diagnoses+fixes the failure, with no operator tick. Guarded against loops
(mark-continued before running) and bounded by the 30-min window.
- chatWith(): chat() variant that injects the finished-execution note after
replayed history without persisting a fake user turn.
- DecideApproval: approving a step by ANY route (button or chat-assent) now
opens the assent window, so auto-continuation works regardless of how the
operator approved — previously only typing "go ahead" opened it.
- SOUL: the agent is told it will be auto-re-invoked when async steps finish —
don't poll get_execution_status, don't wait for "continue"; end the turn and
keep going step by step until the goal is verified or a genuine blocker.
This is the root fix, not another per-command patch: you can't enumerate every
failure of an unbounded action space, but you can give the agent a loop that
observes each result and adapts — because "do anything" always includes "the
first attempt failed."
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three fixes for the session where the agent proposed a plan, waited
for 'proceed', then re-queued instead of being auto-approved:
1. Chat-assent fallback: when the operator says 'proceed' but the
preceding turn had NO pending approvals (agent proposed plan in text
without calling request_execution), inject a system note telling the
agent to execute the plan now. Opens the assent window so subsequent
config_mutation commands auto-run.
2. SOUL.md: instruct agent to ALWAYS call request_execution/run when
proposing a plan, not wait for 'proceed' first. This ensures a
pending approval exists for chat-assent to grant.
3. SOUL.md: stronger Docker instructions — Debian 13's docker.io package
installs the daemon but NOT the docker CLI binary. Must use
get.docker.com in post_install. Added 'handling errors' section:
diagnose, try alternatives, continue — don't stop after one failure.
When the operator has approved a plan via chat assent (assent window
active), pct_create and apt_upgrade now auto-approve and execute
instead of queuing for a separate approval round. The auto-approve
path updates the approval+execution status in the DB, then calls the
HTTP API's decision endpoint to trigger executeApprovedAction — same
code path as a manual Approve button, consistent audit trail.
Agent stopped after every approval step, forcing operator to type
'continue' 7× per deploy session. Root causes and fixes:
1. Compound read-only commands (e.g. 'systemctl status; journalctl')
defaulted to config_mutation — now splits on ;/&&/||/| and classifies
as read_only if all segments are inspection verbs. Added grep, wc,
sort, uniq, cut, tr, dpkg -l, apt list, docker stats to allowlist.
2. curl|sh was classified destructive, forcing typed confirmation for
legitimate installs (get.docker.com). Demoted to config_mutation —
loose assent grants it, no typed phrase needed.
3. SOUL.md said 'STOP after queuing' — replaced with 'continue working
on non-blocked steps'. Added assent window section instructing agent
to carry out the full plan after approval.
4. Assent window: when operator approves a plan via chat assent, a
30-minute window opens where config_mutation commands auto-run
without re-approval. Agent writes expiry to autonomy_settings; MCP
run tool checks it before gating. Destructive never auto-runs.
5. System note after approval now says 'CONTINUE executing the full
plan — do not stop and wait for continue.'
Verified live that after deploying the "fixed" bridge-bound pre-flight, it
still let a known-bad vmbr0+192.168.8.2 config straight through to a full
pct_create with no error. Root cause: the check used
`strings.Contains(pingOut, "REACHABLE")` against markers "REACHABLE" /
"UNREACHABLE" — but "UNREACHABLE" contains "REACHABLE" as a substring, so the
containment check was true for BOTH outcomes. The pre-flight was structurally
incapable of ever failing, regardless of the actual ping result.
Fixed with distinct, non-overlapping markers (PREFLIGHT_OK/PREFLIGHT_FAIL)
and exact-match comparison, pulled into a small gatewayPreflightPassed()
helper with a unit test asserting the exact historical bug case
("UNREACHABLE" must be false) so this bug class can't silently recur.
Re-verified live end-to-end: manually re-tested the exact ping command
(confirmed UNREACHABLE via vmbr0), and this was caught only by actually
running the check against production, not by reading the code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified live that the pre-flight check added in the previous commit had a
real gap: a plain `ping <gateway>` from the Proxmox host succeeds via the
HOST's own routing table (which can have routes to a subnet through paths
the host alone knows about), even when the CONTAINER — attached via a plain
bridge with only a naive on-link default route — can never actually ARP that
gateway. Confirmed by creating a real test container on vmbr0 with
gw=192.168.8.2: the host-wide ping had said "reachable," but pinging from
inside the container showed 100% packet loss. Fixed by binding the pre-flight
ping to the specific requested bridge (`ping -I <bridge>`), which correctly
rejects vmbr0 for that gateway instead of false-positiving via the host's
broader routing table.
Also confirmed live: vmbr1 does exist and is up on strong (contrary to the
possibly-stale host doc), matching what romm/seanime's docs already said.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Investigated why the operator couldn't get past "no DNS/connectivity" across
multiple retries even after Nomos correctly diagnosed and fixed the gateway
(192.168.8.1 -> 192.168.8.2). It still failed. Root cause, confirmed from
strong's own documented network topology: on `strong`, vmbr0 physically
bridges only to 192.168.178.0/24 — the 192.168.8.0/24 service network is
reached via a Fritz!Box static route, not a local bridge. A container
attached to vmbr0 can never reach a 192.168.8.x gateway no matter which
address in that range is picked; ARP for it just gets silently dropped
(matching the earlier hang symptom). The gateway was never the problem — the
bridge was. 192.168.8.0/24 is also segmented into /28 blocks each with their
own gateway (192.168.8.2 is only the .0-.15 block's gateway), so even a
correct bridge with a copy-pasted gateway from a different block would still
fail.
No amount of retrying with a different gateway guess could have fixed this —
the missing fact (which bridge reaches which subnet, and the per-/28 gateway)
isn't inferable from the subnet alone.
- pct_create gets a `bridge` param (was hardcoded to vmbr0) so a correct
bridge can actually be requested once known.
- Fast pre-flight: for any static IP, ping the gateway from the target HOST
before creating anything. Was: a bad config took a multi-minute hang (or,
after last commit's timeout fix, ~2min) before failing. Now: ~2 seconds,
with a message that explicitly says not to guess a different gateway in
the same subnet — find a real neighbor's config or use DHCP.
- SOUL.md: DHCP is now framed as the default, not a fallback; static IP
requires finding an existing LXC on the same host in the same /28 and
copying its bridge+gateway verbatim — inventing one is explicitly called
out as the failure mode that caused this exact incident.
- MCP tool schema: pct_create's params description now documents `bridge`
and the neighbor-copy rule directly in what the model reads at call time,
not just in SOUL.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "running for 10+ minutes without stopping": a real production
execution (TypeType pct_create) was found genuinely stuck 17+ minutes into a
single blocking SSH call. The container's post_install script was looping on
`getent hosts deb.debian.org`, waiting on a network that could never come up
— the operator's static IP config used gw:192.168.8.1, but the actual gateway
on that subnet is 192.168.8.2, so every network call hung instead of failing
fast (packets dropped, not rejected).
Two compounding bugs made this unrecoverable without manual intervention:
1. sshExec (both internal/httpapi/phase3.go and internal/mcp/server.go) had
NO execution timeout — `session.CombinedOutput()` blocks until the remote
command exits, with no deadline. A hung remote process blocks the Go
goroutine forever; the execution can never leave 'running', and the
operator has no way to make it stop. Fixed: both now race the SSH call
against a 10-minute hard timeout, closing the session/client and
returning a clear "timed out after 10m0s" error if exceeded. (The
mcp/server.go copy also still had the original "swallowed non-zero exit"
bug from before that fix was applied to httpapi's copy only — fixed here
too.)
2. provisionScript's DNS-wait loop assumed `getent hosts` fails fast on no
connectivity — it doesn't; a black-holed network can make each call hang
far past the resolver's nominal timeout, so the documented "~90s" budget
was never real. Wrapped every attempt in `timeout 3` so the wall-clock
budget is now actually enforced (~2min worst case), and the failure
message now suggests checking the net0 gateway.
Also fixes the matching UI-side gap (operator's literal question: "is there
a way to get more details? it has been running for 10+ minutes without
stopping"):
- InlineApproval's track() polling loop had its own ~6min ceiling and simply
STOPPED polling after that — silently going stale before the backend (now
correctly capped at 10min) could ever resolve. Raised to a 14min ceiling
with margin, and added a distinct 'stalled' state if that's ever exceeded
(explicitly says something's wrong, rather than freezing silently).
- The running-card now shows live elapsed time (ticking, from the
execution's created_at), the actual command being run, and the execution
ID — previously just a static "this can take a minute" with zero
information. Also added command display to the destructive pending-
approval card for full transparency before confirming.
Verified live end-to-end in a real browser (dev server proxying to
production): queued a real command via chat, approved via the button,
watched the elapsed-time counter tick in real time, and saw it transition to
a completed card with real output once the command finished.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "chat gave me no further feedback — had to go to Ops": two
compounding bugs, found by reading the actual production session transcript.
1. InlineApproval.svelte — all of last session's live-status/self-heal work —
was never imported or rendered anywhere. Chat.svelte had its own separate,
much dumber approval bar (no status tracking, no destructive handling, just
silently disappears after clicking) that WAS the one users actually saw.
Deleted the dead bar and its state; InlineApproval now renders per-message.
2. chat.ts's extractApprovals hardcoded `tool.name === 'request_execution'`,
so any approval raised by the newer `run` tool was invisible — no card, no
feedback, nothing to self-heal, forcing the operator to the Ops page with
zero acknowledgement in the conversation. This was the actual proximate
cause of last night's destroy-135 session. Fixed to match on response
shape, not tool name, so it doesn't silently break again for the next new
gated tool.
3. Nomos was telling operators "type something like 'I confirm destroy 135'"
for destructive actions (SOUL.md) but no backend path ever consumed that
phrase — chat-assent explicitly (and correctly) excludes destructive from
loose assent, but I never built the alternative. Added
isTypedConfirmation() (cmd/nomos/assent.go): stricter than loose assent,
requires an explicit "confirm" statement, only applies to destructive-
flagged pending approvals.
4. InlineApproval's completed-state hardcoded "Provisioned successfully" —
wrong/confusing for a destroy or arbitrary `run` command. Now says
"Completed on <target>" and shows the actual command output, verified live
against the real destroy-135 execution.
Verified live in a real browser against the production API/DB (dev server
proxying to :8090): the historical stuck session now retroactively renders
both executions as resolved with correct wording and real output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Live testing hit `entities_slug_key` violations: exec slugs used an 8-char
prefix of a UUIDv7, whose leading bytes encode a millisecond timestamp — two
executions created seconds apart can share a prefix. Use the full UUID
(guaranteed unique) for the exec entity's slug/name in request_execution, the
new `run` tool, and the REST RequestExecution handler — all three had the
same pattern.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Found live: `run` against lxc:caddy failed with "missing pve_id" even though
pve_id=121 was present — caddy is an inventory-seeded LXC with no `host`
attribute at all (only pct_create-provisioned LXCs set one). The combined
query scanned attributes->>'host' (SQL NULL) into a plain Go string, which
errors the whole Scan — including the pve_id column that scanned fine.
COALESCE the host column to '' so a missing host attribute degrades to the
documented default instead of failing the whole resolution.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the first slice of plans/2026-07-10-general-gated-execution.md:
Nomos gets one general execution tool instead of only a fixed action enum,
gated by an automatic risk classifier, and approval can be granted by the
operator just replying in chat instead of clicking a button.
- internal/policy/command.go: ClassifyCommand(cmd, declaredRisk) — rule-based
read-only allowlist + destructive denylist, default-escalate to
config_mutation for anything else. Classification can only ESCALATE the
caller's declared risk, never de-escalate it (destructive always wins even
if declared read_only). Compound commands (&&, ;, |, $()) never qualify for
the read-only fast path. Full test corpus.
- internal/mcp/server.go: new `run` MCP tool — target (host:/lxc:), command,
purpose, optional declared_risk. Read-only commands execute immediately;
everything else queues an approval exactly like pct_create today, executed
via httpapi's existing executeApprovedAction. Also fixes a real latent bug:
pct_exec resolved an LXC's host attribute without the "host:" prefix, so it
could never find the Proxmox host — new resolveExecTarget/resolveRunTarget
helpers (mcp + httpapi) fix this for both the new `run` action and existing
actions that route through the same execution path.
- internal/httpapi/phase3.go: "run" case in executeApprovedAction; fixes two
bugs found while wiring this up — (1) DecideApproval hardcoded risk_class to
'config_mutation' on every approve, silently corrupting the audit ledger for
every other risk class; (2) denying/revoking an approval never updated the
linked execution's status, so it stayed 'pending_approval' forever instead
of reflecting the decision.
- cmd/nomos/assent.go: deterministic (not LLM-judged) chat-assent detection.
Scoped to the immediately-preceding assistant turn's pending approvals only
— an old "yes" can't retroactively approve something new. Destructive-risk
actions are excluded from loose assent. Approves via the same HTTP decision
endpoint the UI button calls, so both paths share one audit trail.
- web/.../InlineApproval.svelte: self-healing poll — a pending approval card
now picks up being decided via ANY path (chat assent, Ops page, Matrix), not
just its own button. Previously the banner stayed stuck showing
Approve/Deny even after the action had already run elsewhere.
- nomos/SOUL.md: `run` is now the general capability ("no fixed menu, only a
risk gate"); documents chat-assent behavior and the destructive exception.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Operator directives: (1) the UI must show what's executing, its risk
classification, live status, and what knowledge the session created — the
system's growth should be visible, not just trusted. (2) approval should be
granted by chat assent ("go ahead"), not a separate button; destructive
actions still require a typed confirmation phrase.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Evaluate the agent's action path against the OIKOS.md design. Finding: the
intended model (unlimited runbook-driven actions gated by a risk classifier)
already exists on paper and in scaffolding, but the live agent path regressed
to a hard-coded 5-action enum that bypasses the classifier. Plan a layered
realignment: (0) one general gated `run` primitive, (1) runbooks-as-data as the
reliable fast-path, (2) learning. Chosen v1 posture: approve-most (read-only
auto-runs, all state changes gate). Incremental, each step shippable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fresh Debian LXCs have no locale configured, spamming "apt-listchanges:
Can't set locale" / perl warnings across every install and breaking some
packages' post-install scripts. Pin LANG/LC_ALL=C.UTF-8 (and hoist
DEBIAN_FRONTEND) at the top of the in-container bootstrap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Production session provisioned the container but the service never installed:
apt failed with "Temporary failure resolving deb.debian.org" — a static-IP LXC
whose assigned nameserver couldn't resolve. The operator also got zero feedback:
the approval banner just sat there with no running/complete/failed status.
Backend robustness (provisionScript):
- Wait for real DNS/connectivity inside the container before apt, and self-heal
/etc/resolv.conf to a public resolver (1.1.1.1/8.8.8.8) if the assigned one
is dead. `set -e` after the gate so apt/post_install failures surface.
- apt-get update/install with Acquire::Retries=3.
Frontend feedback (InlineApproval):
- After approve, poll GET /executions/{id} and show live phase: submitting →
provisioning… → provisioned successfully / execution failed (with the error).
- add getExecution() to api.ts.
Agent guidance (SOUL.md):
- omit vmid (auto-assigned), prefer dhcp, docker-compose-plugin is not in Debian
(use docker.io + get.docker.com), end post_install with a health check.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The final "UPDATE executions SET result=$::jsonb" built its payload with
fmt.Sprintf and only escaped newlines. apt/pct output contains quotes,
backslashes and control chars, so the payload was invalid JSON, the jsonb
cast failed, and the (unchecked) UPDATE was silently discarded — the
execution stayed 'approved' with a NULL result even though the LXC was fully
provisioned (verified live: vmid auto-assigned, container running, service
installed, post_install ran).
- executeApprovedAction: marshal result via json.Marshal; log UPDATE errors
- add jsonErr() helper; route all pct_create failure-path results through it
- mcp/server.go: add jsonOut() for restart/systemctl/pct_exec inline results
- regression test for JSON validity on quote/backslash/control-char output
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-ups found while verifying the approve→provision path end to end:
- vmid is now optional: the early required-field check rejected vmid:0
before the cluster VMID guard could auto-assign a free id. Only hostname
is required now; 0 (or a collision) resolves to `pvesh get /cluster/nextid`.
- net0: use ip=dhcp with no gateway when no static IP is given (Proxmox
rejects gw alongside dhcp); only attach gw for a static CIDR.
- bump post-create settle to 10s so a DHCP lease is up before apt runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Real production failure when the operator clicked Approve in chat: nothing
provisioned, banner never cleared, execution marked completed.
Three root causes:
- sshExec swallowed non-zero exits when the command produced output, so a
`pct create` that printed "CT 132 already exists" and failed was reported
as success and a bogus lxc entity was registered. Now any non-zero exit
returns an error (with output) so the execution is correctly marked failed.
- The LLM reused VMID 132 (belongs to lxc:rclone; VMIDs are cluster-wide).
pct_create now checks in-use VMIDs via `pvesh get /cluster/resources` and
falls back to `pvesh get /cluster/nextid` when the requested id is taken.
- InlineApproval.svelte reset its state on every prop change (done was also
compared against the wrong string), so the banner never cleared and each
click re-POSTed /decision. Rewritten to track outcome per executionId,
clear on success, and block resubmits.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "asks permission but never acts": the approved pct_create
execution failed to parse because the LLM emitted `"privileged":0` /
`"nesting":1` (numbers) into strict `bool` fields, so the container was
never created. Compounded by a hardcoded template name (debian-13.0-1)
that no longer exists on the host, and no way for the agent to read the web.
- flexBool: accept 0/1, "true", bool for privileged/nesting (the exact prod failure)
- pct_create template pre-flight: list host cache, validate/auto-pick newest debian
- pct_create services[] + post_install: one approval provisions a working service
- new http_get MCP tool (sanitized, size-capped, SSRF-guarded) — agent can read repos/sites
- request_execution description: target=host, full JSON schema + example
- SOUL.md: agent CAN fetch the web; prefer one-step provisioning
- default model deepseek-v4-flash -> v4-pro; maxIterations 15 -> 25
- unit tests for flexBool, template resolve, pkg sanitize, HTML sanitize + SSRF block
Verified live on host:strong with a throwaway VMID 999: template auto-resolved,
container created + booted, services installed, post_install ran, then destroyed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Execution entity name now includes UUID suffix: 'pct_create on host:strong (abc12345)'
so the (type,name) UNIQUE constraint doesn't block subsequent executions for the
same target+action. Dedup now uses JOIN + LIKE prefix match to find only
pending_approval executions.
- Move persistent approval bar from top of messages area to just above the
chat input box (bottom-fixed position, above the textarea form).
- Add dedup in request_execution: check entities(type,name) uniqueness before
creating duplicate executions. Returns 'already queued' message to the LLM,
preventing tool-calling loops.
- Fix createApproval JSON payload: use json.Marshal instead of fmt.Sprintf
to escape params (could contain unescaped double quotes from JSON config).
- Add ON CONFLICT DO NOTHING to entity/execution inserts for dedup race safety.
- Persistent approval bar at top of Chat.svelte: aggregates pendingApprovals
from all messages, fixed position (won't scroll away). Approve/deny/approve-all.
- Update SOUL.md: agent must STOP after queuing a gated action.
- Fix ToolCallGroup reactivity: wasActive = (active).
- Replace text-based regex parsing in InlineApproval with structured
pendingApprovals extracted from request_execution tool results. The tool
result text is deterministic (not LLM-generated), making UUID extraction
reliable regardless of how the LLM rephrases the response.
- Fix ToolCallGroup reactivity: wasActive = active captured initial
value. Now uses (active) so re-runs on prop changes.
- Extract approvals in both live streaming (done event) and history loading
for consistent behavior on resumed sessions.
- Empty/refusal responses retried once, then surfaced as errors instead of silent blanks
- Chinese refusal boilerplate detected via denylist + non-ASCII heuristic
- Bulk-tool preference added to SOUL.md (list_lxcs over per-entity get_lxc_state)
- Tool results truncated to 4KB on persist; get_state_snapshot filters null-state entities
- Session delete (DELETE /sessions/{id} + confirm-on-second-click UI)
- Session titles auto-generated from assistant answer instead of raw user message
Inspected the live agent_sessions/agent_messages tables on mac-mini and
found silent empty responses, a canned non-English refusal after 22 tool
calls, 70-call fan-out for simple fleet questions, 100KB+ persisted
messages, and no session delete/title hygiene.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two chat UX changes (share Chat.svelte, committed together).
Tool-call grouping (ToolCallGroup.svelte):
- Replaced the one-<details>-per-tool-call list with a single
collapsible group per assistant turn, headed by tool count + a
name preview and a status icon (spinning wrench in progress, check
done, X on error).
- The group auto-collapses the instant its turn finishes streaming, so
a completed round shows as one compact pill; historical/loaded turns
start collapsed. The auto-collapse fires once at the streaming→done
transition, leaving manual toggles alone afterward.
Session graph rail (SessionGraph.svelte) — replaces the old
ContextRail (fleet health / pending approvals / live events), which is
deleted:
- A force-directed graph that starts empty (animated constellation
empty state) and grows as the conversation references entities.
Slugs are extracted from message text and tool *arguments* only —
never bulk result rows, so a single get_health_summary doesn't dump
all 168 entities — then validated against the backend via fetchGraph
(cached) with check/execution probe entities excluded. Nodes are
colored by health; edges appear once both endpoints are present.
- Clicking a node highlights it and its neighbors and opens an inline
detail panel below: slug/type/state, health + freshness, top
attributes, in-graph relations (clickable to hop), and a Full detail
button opening the entity sheet.
- The rail is resizable via a drag handle (260–620px, persisted to
localStorage). The header's global fleet-health dots are unchanged;
only the right-rail content was replaced.
Risk: reversible_low (UI-only). The slug extractor is scoped to
focused mentions by design; edges may be slightly incomplete since
only root-fetched entities contribute edges, which is acceptable for a
session overview.
Verification: verified in the browser preview — loading a real session
built a 4-node graph (hubris/caddy/netbird-vps/strong) with the
hubris→caddy relationship edge; clicking hubris showed
"proxmox-host · active · healthy · checked 40s ago" with attributes and
relations; dragging the handle resized 320→440px and persisted; a
34-tool historical turn renders as one collapsed pill that expands on
click. tsc clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three UI issues reported after the neutral-gray redesign:
- favicon.svg was still filled #58a6ff (the pre-redesign accent blue);
changed to white to match the sidebar logo mark.
- Sidebar nav items all showed a filled background even when inactive.
Root cause: sidebar-menu-button.svelte (and -sub-button) rendered
`data-active="false"` as a literal attribute, but Tailwind's bare
`data-active:` variant matches attribute *presence*, not value — so
data-active:bg-sidebar-accent applied to every item regardless of
state. Fixed by emitting the attribute only when active
(`isActive || undefined`), a latent bug in the vendored shadcn
component that read as intentional until flagged.
- Tailwind's preflight resets <button> to cursor: default, so no button
in the app showed a pointer. Added one base rule restoring
cursor: pointer for buttons, [role=button], links, summary, and
select (respecting :disabled / aria-disabled) rather than annotating
each call site — covers new interactive elements automatically.
Risk: reversible_low (UI-only).
Verification: verified in the browser preview that inactive sidebar
items are transparent (only the current page shows a background),
nav buttons report cursor: pointer via computed styles, and the
favicon renders white in the tab.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Problem: two issues surfaced after the dashboard-01 shell change
(cbfd09c). (1) Sidebar.Inset previously had an explicit h-svh that
hard-capped the app's height at the viewport; adding variant="inset"
put a margin on that same fixed-height box, pushing it taller than the
viewport with nothing left in the chain to cap it (Sidebar.Provider's
own wrapper only sets min-h-svh — a floor, not a ceiling). Result: the
whole page scrolled as one long document instead of each page's own
content scrolling internally with the header pinned — confirmed via
computed styles, e.g. Entities.svelte's table wrapper measured
scrollHeight 6531px against a 900px viewport, all of it spilling past
body instead of scrolling in its own rounded-border container.
(2) The color palette was GitHub-dark-inspired (blue-tinted grays:
#0d1117 bg, #58a6ff primary/accent) rather than the neutral grays the
shadcn-svelte dashboard-01 reference actually uses.
Change:
- App.svelte: moved the height cap up to Sidebar.Provider itself
(class="h-svh") instead of Sidebar.Inset, since the cap needs to sit
above wherever the inset variant's margin gets applied, not on the
same box as the margin.
- app.css: replaced the core tokens (background/foreground/card/
popover/primary/secondary/muted/accent/border/input/ring/sidebar-*)
with shadcn's canonical dark-theme OKLCH values (0-chroma neutral
grays), pulled directly from huntabyte/shadcn-svelte's own
docs/src/app.css rather than approximated. --success/--warning
deliberately kept as real, distinguishable colors — they signal
actual health state, and desaturating them to match the neutral
chrome would reintroduce the "can't tell what's actually happening"
problem this whole project started from (see 279549c). --accent-blue
now aliases --sidebar-primary (still a real blue) instead of
--primary, so the couple of spots wanting an interactive "pop" still
have one while buttons/links/focus rings ride the neutral --primary.
Risk: reversible_low (UI-only).
Verification: npx tsc --noEmit clean (excluding pre-existing unrelated
.svelte type-resolution warnings). go build/vet clean (backend
untouched, sanity check only). Manually verified in the browser
preview at 1400px: document.body.scrollHeight now exactly matches
window.innerHeight on both Overview and the 193-row Entities table
(previously 6531px vs 900px); scrolled the Entities table wrapper to
row ~60 and confirmed the header/filter bar/column headers stay
pinned while only the table body scrolls; confirmed neutral gray
rendering across Overview's stat cards, the event-rate chart, and
Chat's tool-call list.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Problem: requested visual alignment with the shadcn-svelte dashboard-01
reference block (shadcn-svelte.com/blocks#dashboard-01) — the app's
sidebar/header shell and Overview stat cards looked plain by comparison.
Change: pulled the actual reference source (app-sidebar.svelte,
nav-main.svelte, site-header.svelte, section-cards.svelte from
huntabyte/shadcn-svelte) rather than approximating from screenshots.
- App.svelte: Sidebar.Root now uses variant="inset" (the floating,
rounded, shadowed content panel — already fully built into the
existing Sidebar.Inset component via peer-data selectors, just never
enabled). Brand mark is now a proper Sidebar.MenuButton matching the
reference's padding/hover treatment; "New chat" uses the reference's
primary-colored button styling. Header matches the reference exactly:
h-(--header-height) (48px, was 44px), vertical separator after the
sidebar trigger, right-aligned actions group.
- Overview.svelte: stat cards rebuilt to match section-cards.svelte —
gradient background, Card.Action badge, Card.Footer with a bold line
+ muted context line, tabular-nums, responsive @container grid
(1/2/4 columns). Deliberately did NOT copy the reference's fake
trend-percentage badges (Oikos doesn't track historical trends, and
this project's whole thrust has been eliminating dishonest UI state —
see 279549c). Badges instead reflect real current-state signals
(healthy/degraded/down, clear/needs-review) computed from the actual
dashboard summary.
- EntityDetailContent.svelte + Entities/Signals/Ops/Events/Agent/
Audit/Knowledge pages: normalized root padding to p-4 md:p-6 (was a
flat p-6) to match the reference's responsive py-4 md:py-6 convention.
Risk: reversible_low (UI-only, no data or behavior changes).
Verification: npx tsc --noEmit clean (excluding pre-existing unrelated
.svelte type-resolution warnings, same as prior commits). go build/vet
clean (backend untouched, sanity check only). Manually verified in the
browser preview at 1400px: inset sidebar's margin/rounded-corner/shadow
classes confirmed applied via computed styles; Overview cards render
with real live numbers from the now-fixed dashboard summary endpoint;
Signals/Ops pages confirmed visually consistent with the new spacing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
