Full review (plans/2026-07-17-codebase-review-and-cleanup.md) covering Go, web SPA, and docs. Applied low-risk doc/tooling fixes; code refactors and dead-code deletions are listed as actionable recommendations pending approval. Doc fixes: - AGENTS.md: remove ghost of retired request_execution (contradicted the retire notice above it); fix knowledge/wiki/ -> archive/knowledge/; replace brittle counts (33 tools, 36 docs, 20 checks) with pointers to source; drop point-in-time dates. - OIKOS.md: fix broken plan link (now in done/); 001-011 -> 001-020; 15 MCP tools -> pointer; replace hardcoded knowledge counts. - README.md: 15 tools -> pointer; fix wails plan link (now in done/); complete internal/ package list (add checkdefaults, observability, safego); add cmd/desktop/ to repo layout. - commands.md, page-templates.md: fix broken links; HERMES.md -> NOMOS.md. Plans housekeeping: - Move 4 done 2026-07-14 plans from plans/ to plans/done/. - Reconcile plans/index.md: add the 2 missing 2026-07-14 entries and the 2 missing 2026-07-15 done entries; add this review. - Fix stale plan path in migrations/020 comment. New docs: - docs/index.md and docs/operations/README.md (folder READMEs per writing-style.md). Tooling: - web/package.json: add check/typecheck/lint scripts + svelte-check devDep. - Makefile: desktop-package version now reads from VERSION file instead of hardcoded 0.1.0. VERSION 0.7.6 -> 0.7.7 (patch: docs + tooling only).
55 KiB
2026-07-14 — Session reliability & UX audit: "empty responses, disconnects, and silent failures"
Status: Planned — 2026-07-14. Audit from a Nomos production session that
exhibited multiple reliability gaps. Findings are grounded in code paths and
empirical session data from the live DB (00a344e4 + 42424b92).
Session under audit
| Session | Goal | Messages | Outcome | Top tools |
|---|---|---|---|---|
00a344e4 (most recent) |
Upgrade 4 low-risk LXCs | 22 (11 turns) | success | run (76), request_execution (86), update_plan_step (42), get_execution_status (24) |
42424b92 (prior failure) |
Fleet health check | 3 messages | failure | — empty response |
Findings — Part A: UX reliability (the session-breaking class)
1. Empty/unusable responses — "Nomos returned an empty or unusable response — please retry."
Where: cmd/nomos/agent.go:362-371 (isRefusalOrEmpty) + cmd/nomos/continue.go:207-259 (resumeSession retry).
What happened: The LLM (deepseek/deepseek-v4-pro over OpenRouter) returned
a blank or non-English response. maxLLMRetries = 1 (agent.go:33) gives exactly
one retry inside chatWith. resumeSession adds a second outer retry
(continue.go:216). After both fail, the auto-continuation marks the task as
failure with "Auto-resume failed after retrying: Nomos returned an empty or unusable response — please retry.".
Root cause: No state-specific recovery. The user sees "please retry" but
(a) the task is marked terminal (failure), so retrying in the same session
won't work — the agent won't auto-resume a completed task; (b) there's no
suggestion about how to retry (new chat message? resend the same prompt?);
(c) the error surface in the UI is a plain text error store update that
disappears on the next send.
Severity: Blocker — this is the session-ending failure class.
2. Client loses connection and does not recover
Where: cmd/nomos/main.go:153-272 (SSE handleChat) + web/src/lib/stores/chat.ts:218-331 (sendMessage).
What happened: SSE relies on a persistent HTTP connection. When it drops
(network blip, server restart, 10-min timeout), the frontend's stream error
callback sets error but:
- No auto-reconnect. The user has to logout/log in to get a fresh SSE stream.
- No fallback to polling during a stream outage. The polling loop
(
startPolling, chat.ts:152) deliberately bails when$streamingis true. - The error message is not persistent — it clears on next send.
Root cause: SSE streams have no reconnect logic. The connection check is binary: streaming or not. There's no "connection lost but task still running" state.
Severity: Blocker — forces logout/login cycle for the user.
3. Agent stops mid-plan — needs "status" nudge to wake up
Where: cmd/nomos/continue.go:100-155 (runContinuationWorker + processContinuations).
What happened: The auto-continuation worker polls every 4 seconds for
finished executions and checks assentWindowActive (continue.go:145). If the
assent window expired or was never opened, the execution is marked continued
without actually resuming — the agent silently drops the ball. The idle sweep
(continue.go:49-65) only fires every 5 minutes, and its first pass only
nudges the agent with a system note. If that nudge gets lost (empty response,
or the model ignores it), the next sweep auto-closes the task.
Root causes:
assentWindowActivecheck is too aggressive — if the window expired while an execution was running, the result is silently discarded.- No user-visible indication that the agent is "waiting for something."
- The idle sweep's 5-minute interval + 2-pass nudge-then-close is too slow for a "the agent just stopped talking" situation.
Severity: Blocker — operator has to manually ask "status?" to resume work.
4. No feedback when background work is running
Where: web/src/lib/stores/chat.ts:149-170 (polling) + web/src/lib/components/ToolCallGroup.svelte (tool display).
What happened: When the agent queues an execution (e.g. apt upgrade -y),
the tool result says "execution queued — requires approval." After
approval, the execution runs in the background. The chat shows:
- ToolCallGroup: the
runtool call with its result (static, no updates). - InlineApproval: the approval card polls execution status and shows it.
But: when the auto-continuation worker picks up the finished execution and resumes the agent, the agent's next turn (polling for its messages) is invisible unless the user happens to be watching the session at that exact moment. There's no "agent is working on step 4 of 6" indicator.
Root causes:
- Polling only updates the message list — there's no activity indicator.
get_execution_statusresults are rendered as generic JSON in ToolCallGroup, not as custom execution-status cards.- No "last activity" timestamp in the chat header.
- SessionDigest is computed at session fetch time, not live-updated.
Severity: Friction — the agent IS working, the user just can't tell.
5. Approvals sometimes not shown/asked in chat
Where: web/src/lib/stores/chat.ts:33-52 (extractApprovals) + web/src/lib/components/InlineApproval.svelte.
What happened: extractApprovals scans tool results for "requires approval" +
execution UUID. But it only runs on the SSE done event (chat.ts:296-302).
If the stream disconnects before done (finding #2), approvals are never
extracted in the live view — they'd appear on reload but not in real time.
Additional gap: the extraction uses t.args?.action ?? t.args?.purpose ?? t.name
(chat.ts:43). For run tools, t.name is "run" — not informative. The
purpose text is in t.args.purpose but it's a full sentence, not a short label.
Root causes:
- Approvals only extracted on
done, not on eachtool_result. - No fallback extraction on message reload/poll.
- The card label doesn't clearly show what's being approved.
Severity: Friction — approvals ARE in the system (Ops page shows them), but the chat doesn't always surface them.
6. No custom renderer for get_execution_status — 24 calls rendered as raw JSON
Where: web/src/lib/components/ToolCallGroup.svelte (generic tool display).
What happened: The agent called get_execution_status 24 times in the
session. Each result is a JSON blob with execution ID, status, target, output,
error. All 24 are rendered as generic collapsed JSON in ToolCallGroup — the
user has to expand each one manually and parse the JSON to understand what's
happening.
Fix needed: Custom renderer via the tool-renderers system (web/src/lib/tool-renderers/)
that shows: execution ID (truncated), status badge (running/completed/failed),
target slug, elapsed time, collapsible output. Optionally auto-poll while
the execution is running.
Severity: Friction — 24 calls in one session makes the chat cluttered and hard to follow.
7. Plan changed mid-flight, stale steps left behind
Where: cmd/nomos/tasks.go:182-205 (propose_plan) + cmd/nomos/store.go:367-423 (plan storage).
What happened: The agent called propose_plan 6 times in the session.
Each call appends new steps (store.go:418-423: appended=true). But:
- Five
propose_plancalls mean five generations of plan steps. - Old steps that were pending when a new plan was proposed remain in the list
with their old status (some
running, some never started). update_plan_steponly updates byseqnumber — if the new plan reuses the sameseqvalues, old steps get overwritten. If it adds newseqvalues, old steps remain orphaned.
Root causes:
- No auto-cleanup of pending steps when a new plan is proposed.
- The plan panel shows ALL steps across all plan generations.
- No "plan revision" tracking — just a flat append.
Fix needed: When propose_plan is called again, auto-set any still-pending
steps (status ≠ done/failed/skipped/blocked) from the previous generation to
replaced. Track a generation column on plan steps so the frontend can
collapse or dim old generations.
Severity: Friction — the operator's plan view shows progress that doesn't reflect reality, eroding trust.
8. 4 orphaned approvals after session completed
Where: cmd/nomos/store.go:480-509 (completeTask) — no approval cleanup.
What happened: When the agent calls complete_task, it updates
agent_sessions.status and outcome. But it does NOT:
- Cancel any
pending_approvalexecutions linked to this session. - Close any
approvedbut unfinished executions. - Clean up the
autonomy_settingsassent window rows.
Root causes:
completeTaskis a simple status update with no cascading cleanup.- No lifecycle link between
agent_sessions.idandexecutions.session_idfor approval cleanup. - The executions table has
correlation_idbutcompleteTaskdoesn't use it to find and cancel open executions.
Fix needed: completeTask should auto-cancel any executions in
pending_approval or approved state for this session. Also clean up the
assent window key from autonomy_settings.
Severity: Blocker — leaves the operator with a stale approval count, confusion about what's pending, and no automatic cleanup path.
Findings — Part B: Agent knowledge loop (what the agent learned vs. what it wrote back)
9. ZERO update_entity_attributes or create_relationship calls in the entire session
Where: Session 00a344e4 — 22 messages, 11 turns, 348 total tool calls.
What happened: The agent discovered massive drift between the DB and reality but wrote none of it back to entity attributes or relationships:
| Discovery | Tool used | Written back? |
|---|---|---|
13 LXCs are state: destroyed |
get_entity × 14 |
No |
| 5 LXCs reachable via pct exec (SSH auth fails) | run × 5 + get_entity |
No |
teddycloud runs on host:hubris not host:strong |
get_relations |
No |
| rclone was false-negative (resolver glitch), actually alive | get_entity |
No |
romm has broken pct config (nodes/h error) |
run |
No |
| Updated upgrade counts for 4 LXCs | run (apt-get upgrade) |
No |
Every future fleet audit will re-discover these same facts from scratch,
issuing the same 44 wasted tool calls (14 request_execution against
destroyed LXCs + 14 get_entity to confirm they're destroyed).
Root cause: The plan's final step descriptions were too vague:
- Plan 1 step 4: "Record findings" → agent called
upsert_knowledgeonly. - Plan 2 step 4: "Write back knowledge" → agent called
upsert_knowledgeonly. - Plan 3 step 5: "Write back results" → agent called
upsert_knowledgeonly.
The SOUL.md (lines 86-103) is explicit about the three-step writeback:
update_entity_attributes → create_relationship → upsert_knowledge →
complete_task. But the plan step descriptions the agent wrote for itself
said "Record findings" — too ambiguous. The agent satisfied this with
narrative knowledge alone and skipped structured entity updates entirely.
Severity: Blocker — the knowledge loop is broken. The graph keeps drifting further from reality each session because nothing is written back.
10. list_lxcs returns all LXCs including destroyed ones — no state filter
Where: internal/mcp/server.go:664-675
What happened: The list_lxcs MCP tool queries WHERE e.type = 'lxc' with
no state filter. The result included 13 destroyed LXCs mixed with 19 live ones.
The agent then:
- Issued 14
request_executioncalls against no-IP LXCs (all failed because the LXCs are destroyed → 14 wasted tool calls). - Issued 14
get_entitycalls to discover they'restate: destroyed. - Had to manually classify which are live and which are destroyed.
If list_lxcs excluded destroyed entities (or accepted a state filter),
these 28 tool calls would have been eliminated from the first turn alone —
reducing turn 1 from 92 tool calls to 64 (and making the plan simpler).
Root cause: list_lxcs is a fixed SQL query. The list_entities API
already supports state filtering (entities.sql.go:118:
$1::text IS NULL OR e.state = $1), but list_lxcs hardcodes its query
without exposing the parameter.
Severity: Friction — wastes ~30% of tool calls per fleet-wide audit.
11. Knowledge recorded was good quality but linked too broadly
Where: Session 00a344e4 — 4 knowledge entries via upsert_knowledge.
What was recorded:
| Entry | Kind | Linked to | Quality |
|---|---|---|---|
| "Fleet-wide apt audit — 2026-07-10" | investigation | cluster:homelab |
Good — full table + caveats |
| "Fleet-wide apt audit — complete — 2026-07-10" | investigation | cluster:homelab |
Duplicate of v1 — should have updated v1 instead |
| "Low-risk tier upgrade results — 2026-07-10" | investigation | cluster:homelab |
Good — per-LXC breakdown |
| "How to run fleet apt upgrades — lessons" | document | agent:nomos |
Good — actionable runbook |
What should have been different:
- The two audit investigations are near-duplicates — v2 supersedes v1 but v1
wasn't marked as superseded.
upsert_knowledgewith the same title should update the existing entry. - All entries linked to
cluster:homelab— none to individual LXCs. The upgraded LXCs (lxc:nfs-export,lxc:gitea,lxc:dns,lxc:auth-outpost) should each have the upgrade result linked viaabout. Futureget_entity_knowledge("lxc:nfs-export")returns nothing for this session's work. - The 13 destroyed LXCs aren't linked to any knowledge entry documenting
why they were destroyed. A future agent looking at
lxc:arr-yunohostwon't find the knowledge entry saying "migrated to arriman, 2026-04-28."
Root cause: The agent used about: cluster:homelab for everything.
The upsert_knowledge tool accepts a single about slug, so the agent
cannot link one entry to multiple entities. Multi-entity linking would
require either multiple upsert_knowledge calls (one per LXC) or extending
the tool to accept an about array.
Severity: Friction — knowledge exists but is hard to discover per-entity.
12. Plan step descriptions are too vague — agent interprets them weakly
Where: The plan steps the agent proposed for itself across 3 plans.
What happened: The agent wrote plan steps like:
- "Record findings" / "Update knowledge base if anything notable"
- "Write back knowledge" / "Record deprecated list and SSH strategy"
- "Write back results" / "Record what was upgraded and any issues."
Every time, the agent satisfied these with upsert_knowledge alone and
skipped update_entity_attributes + create_relationship. The SOUL.md
instructions are explicit about the three-step writeback, but the plan
step descriptions the agent generated for itself didn't reinforce this.
Root cause: The agent proposes its own plan steps. The tool description
for propose_plan (tasks.go:36-70) says the last step should include
update_entity_attributes / create_relationship / upsert_knowledge. But the
agent still wrote vague step descriptions. The instruction is present but
not being followed.
Severity: Blocker — this is the direct cause of finding #9.
13. Plan step completion order was wrong — step 5 marked done before step 4
Where: Turn 11 (the "lets stop here" final turn) in session 00a344e4.
What happened: The agent called update_plan_step in this order:
seq=5, status=done— "Write back results"seq=4, status=done— "Upgrade auth-outpost"seq=1, status=done— "Upgrade nfs-export"seq=2, status=done— "Upgrade gitea"seq=3, status=done— "Upgrade dns"
The plan panel would have shown step 5 completing before step 4, then steps 1-3 completing in reverse order. This is the "plan changed mid-air and the agent forgot to keep its progress up to date" issue — the agent rushed to close all remaining steps in the final turn, in no particular order, without verifying each one was actually done.
Root cause: No ordering validation on update_plan_step. The agent can
mark any step as done in any order. At minimum, steps should complete in
seq order. The frontend should also handle out-of-order completions
gracefully (don't reorder the list just because completions arrived out of
sequence).
Severity: Friction — the progress view looks wrong, eroding trust.
14. The 1st turn had 92 tool calls — 30% were wasted on destroyed LXCs
Where: Turn 1 of session 00a344e4.
Breakdown:
| Category | Count | Tool calls |
|---|---|---|
| Plan/task management | 12 | set_goal × 2, propose_plan × 2, update_plan_step × 8 |
| Research | 2 | list_lxcs × 1, search_knowledge × 1 |
| Legitimate audits | 50 | request_execution against reachable LXCs |
| Wasted — destroyed LXCs | 28 | request_execution × 14 (failed), get_entity × 14 (confirm destroyed) |
92 tool calls in one turn. The agent was efficient (it batched them), but 28
of them were entirely avoidable if list_lxcs had filtered out destroyed
entities or if the DB had up-to-date entity attributes from a prior session.
Root cause: Combination of #9 (no entity attributes written back) + #10 (no state filter on list_lxcs). Each problem compounds the other.
Severity: Friction — costs latency, OpenRouter credits, and model context window. A single fleet audit shouldn't need 92 tool calls.
Findings — Part C: The "wins" (things that worked well)
Despite the issues above, the session had several things working correctly that should be preserved:
- Chat assent →
runupgrade path works. Once the agent learned to useruninstead ofrequest_execution, upgrades auto-ran under the assent window without re-approval. This is the correct pattern. pct execfallback discovered autonomously. The agent realized 5 LXCs fail SSH butpct execfrom their Proxmox host works. No operator input needed — the agent investigated and found the alternative.- The agent self-corrected
request_execution → run. Whenrequest_executioncalls got stuck atpending_approvaldespite chat assent, the agent diagnosed the problem ("assent window only coversruncommands") and switched tools. Good resilience. - Knowledge content quality was high. All 4 knowledge entries had structured tables, relevant caveats, and actionable instructions. Content-wise, the knowledge loop is producing good output — just not linking it to the right entities.
- The agent documented a process lesson as a reusable document ("How to run fleet apt upgrades"). This is exactly the kind of knowledge that prevents future sessions from repeating mistakes. The format (runbook with examples + verification commands) is correct.
- Multi-step plan execution worked end-to-end. 3 plans, 21
update_plan_stepcalls, all steps eventually completed. The plan mechanism itself is solid — the issues are in step descriptions and writeback completeness.
Improvement plan — with concrete fix descriptions
Phase 1: Crash recovery (addresses #1, #2, #3 — the "session dies" class)
1.1 — SSE auto-reconnect + fallback to polling
Files: web/src/lib/stores/chat.ts, web/src/lib/api.ts
What: When the SSE stream drops (network blip, server restart), the frontend should automatically reconnect instead of requiring a logout/login cycle.
How:
- Add a
reconnectCountstate tosendMessage()in chat.ts. When the stream errors or completes without adoneevent (chat.ts:316-330), set adisconnected = trueflag on the current session instead of callingstreaming.set(false). - When
disconnectedis true, start a backoff reconnect loop: wait 1s, 2s, 4s (capped at 8s), then re-post to/chatwith the samesession_idand an empty message string +resume: trueflag. The backend'shandleChatroutes this intoresumeSessionwith a system note like"[System: the stream reconnected — continue from where you left off.]". On reconnect success, stop the loop and cleardisconnected. - In
streamChat()(api.ts:105-156), thecatchandfinallyblocks call the same callbacks but need to distinguish "aborted by user" (AbortError) from "connection dropped": don't callonDone()on network errors — let the newonDisconnectcallback handle it instead. Add a third callback param:onDisconnect: (reason: string) => void. - Maximum 3 reconnect attempts. After exhausting retries, fall back to
polling: set
streamingto false, cleardisconnected, and callstartPolling(sessionId)(the existing poll loop in chat.ts:152-170). The task continues server-side — the poller will catch whatever happened during the outage.
1.2 — Connection-lost banner with retry button
Files: web/src/pages/Chat.svelte
What: A persistent banner above the input area that shows when the SSE connection is lost, with a "Reconnect" button and a countdown timer for the next auto-retry.
How:
- Add a
connectionStatestore tochat.ts:'connected' | 'disconnected' | 'reconnecting'. Expose it via aconnectionexport. - In
Chat.svelte, add a banner between the message area and the input bar (around line 170, replacing the current{#if $error}block):{#if $connection === 'disconnected'} <div class="connection-banner">Agent connection lost. <Button size="sm" onclick={reconnect}>Reconnect</Button></div> {:else if $connection === 'reconnecting'} <div class="connection-banner muted">Reconnecting in {countdown}s… <Button size="sm" variant="ghost" onclick={cancelReconnect}>Cancel</Button></div> {/if} reconnect()triggers an immediate reconnect attempt (reset the backoff timer, callsendMessage("", { resume: true })).- The current
$errorbanner (Chat.svelte:164-170) becomes the non-connection error path — displayed for LLM errors, tool errors, etc. This is a sibling banner, not a replacement.
1.3 — On stream drop, immediately poll for messages
Files: web/src/lib/stores/chat.ts
What: When the SSE stream drops, don't wait for the user to log out/in. Start the poller immediately so the chat shows whatever the agent did server-side during the outage.
How:
- In
sendMessage()'s error/complete callbacks (chat.ts:316-330), after settingdisconnected(from 1.1), callstartPolling(sessionId)immediately instead of waiting for the reconnect loop or manual reload. startPollingalready bails when$streamingis true (chat.ts:156). After the stream drops,streamingstays true because ofdisconnected. Change the gate: allow polling whendisconnectedis true even ifstreamingis true. The polled messages are from the persisted DB, so they won't conflict with the dead SSE stream.- When reconnection succeeds (the SSE stream is live again), stop polling to avoid double-rendering.
1.4 — Empty-response: retry 3 times instead of 2
Files: cmd/nomos/agent.go:33
What: maxLLMRetries = 1 means the inner loop (agent.go:331-375) retries
once, and resumeSession (continue.go:216) retries the whole call once — 2
total chances. DeepSeek sometimes needs 3.
How:
- Change
const maxLLMRetries = 1toconst maxLLMRetries = 2atagent.go:33. This gives 3 total attempts in the inner loop. - In
resumeSession(continue.go:216), changefor attempt := 0; attempt < 2tofor attempt := 0; attempt < 3for 3 outer-loop attempts. - On each retry inside
resumeSession, inject a stronger system note:"[System: your previous response was empty or invalid — the operator is waiting. Produce a real response this time.]"instead of just re-running the same prompt.
1.5 — Empty-response: don't auto-complete the task on failure
Files: cmd/nomos/continue.go:252-258
What: When resumeSession exhausts retries, it calls completeTask with
outcome='failure' (continue.go:256). This marks the session as terminal,
so the next user message in chat can't resume it — the user has to know to
start a new task.
How:
- Remove the
completeTaskcall at continue.go:256. Instead, persist a"resume_failed"system note as a regular assistant message in the transcript so the user sees what happened:resumeFailedNote := fmt.Sprintf("[System: auto-resume failed after retrying: %s. The task is paused — send another message to try again.]", errText) - Leave
agent_sessions.statusat'executing'(or whatever it was before the failed resume). The user's next chat message will re-enterhandleChat, which replays the full history and picks up from the last tool calls. - Only auto-close if the task was already in a genuinely terminal state
(check
statusbefore deciding).
1.6 — Persistent, dismissible error card in chat
Files: web/src/lib/stores/chat.ts (new store: chatErrors), web/src/pages/Chat.svelte
What: The current $error store (chat.ts:72) is a single string that
disappears on the next sendMessage(). For empty-response errors, the user
needs a card that stays visible until dismissed, explaining what went wrong
and suggesting a recovery action.
How:
- Add
chatErrorsas a writable store of error objects:interface ChatError { id: string; message: string; dismissible: boolean; action?: string } - In
sendMessage()'s error handler, push aChatErrorinstead of settingerror.set(err). Theactionfield suggests recovery (e.g. "type 'status' to check what happened" or "click Retry to resend"). - In
Chat.svelte, renderchatErrorsas dismissible cards above the input bar (replacing or alongside the current$errorbanner):{#each $chatErrors as err (err.id)} <div class="error-card"> <span>{err.message}</span> {#if err.action}<Button size="xs" variant="outline" onclick={() => dismissError(err.id)}>{err.action}</Button>{/if} <button onclick={() => dismissError(err.id)}>×</button> </div> {/each} - Clear
chatErrorsonnewChat()but NOT onsendMessage()— errors survive across messages until explicitly dismissed.
Phase 2: Visibility (addresses #4, #5, #6 — "what is the agent doing?")
2.1 — Custom renderer for get_execution_status
Files: New: web/src/lib/tool-renderers/ExecutionStatus.svelte, web/src/lib/tool-renderers/index.ts
What: get_execution_status results are large JSON blobs rendered in
ToolCallGroup's generic <pre> blocks. A custom renderer shows execution
state inline with a status badge, live polling, and collapsible output.
How:
-
Create
web/src/lib/tool-renderers/ExecutionStatus.svelte:- Props:
tool: ToolCallResult - Extract from
tool.result:execution_id,status,target,action,result,error,duration_ms - Render a compact card:
[spinner/check/X] execution 019f5f.. | apt_upgrade on lxc:nfs-export | completed (12.3s) ▶ output: 0 upgraded, 0 newly installed... - If status is
runningorapproved, auto-pollgetExecution(id)every 3 seconds (viaapi.ts) and update the card in place. Stop when terminal. - Show elapsed wall time (live counter while running).
- Collapsible output section (default collapsed for completed, expanded for failed with error text in red).
- Props:
-
Register in
tool-renderers.ts:import ExecutionStatus from './tool-renderers/ExecutionStatus.svelte' registerToolRenderer({ match: (t) => t.name === 'get_execution_status' && t.type === 'tool_result', component: ExecutionStatus }) -
In
ToolCallGroup.svelte, the tool will now be matched bygetToolRenderer()and rendered inline byChat.svelte'sgetInlineTools— no changes needed to the existing ToolCallGroup unless the card should also appear inside the group (it should — add the custom card to ToolCallGroup's body too, or simply ensuregetInlineToolsclaims it and ToolCallGroup'sunmatchedprop excludes it).
2.2 — Extract approvals on tool_result events, not just done
Files: web/src/lib/stores/chat.ts:243-277
What: extractApprovals() only runs on the SSE done event. If the stream
drops before done, the approvals never appear in chat (they exist on the Ops
page but the chat shows nothing).
How:
- In the
tool_resultbranch ofsendMessage()(chat.ts:258-277), after updating the tool inactiveToolsand the message'stoolsarray, runextractApprovalson the fulltoolsarray of the current message and setmsg.pendingApprovalsimmediately:// inside tool_result handler, after updating the tool: messages.update((ms) => { const last = ms[ms.length - 1] if (last && last.role === 'assistant') { last.tools = last.tools.map((t) => t.id === ev.data.id ? updated : t) last.pendingApprovals = extractApprovals(last.tools) // <-- add this } return [...ms] }) - Keep the
done-event extraction as a final sanity pass (it catches any edge case wheretool_resultarrived before the tool was registered inactiveTools).
2.3 — Approval card label: use purpose text
Files: web/src/lib/stores/chat.ts:43
What: extractApprovals() picks t.args?.action ?? t.args?.purpose ?? t.name.
For run tools, t.name is "run" — not informative. The purpose text is
a full sentence like "Upgrade nfs-export (21 packages)". Better to truncate it.
How:
- Change the label logic in
extractApprovals:action: t.args?.purpose?.slice(0, 60) ?? t.args?.action ?? t.name ?? 'unknown',purposeis always present and meaningful forruncalls; falling back toaction(forrequest_execution) and thenname(last resort). Truncate to 60 chars to fit the card.
2.4 — "Agent is working" indicator in chat header
Files: web/src/pages/Chat.svelte
What: The chat page has no indicator that the agent is doing autonomous work (executions running, auto-continuation polling, plan steps advancing). The user stares at a static chat and wonders if anything is happening.
How:
- Add a
sessionActivitystore inchat.ts:interface SessionActivity { lastMessageAt: Date | null executionsRunning: number planStep: string | null // "3/6 — Upgrade nfs-export" } - Update
sessionActivityfrom:- Poll fetches: when a new message arrives via
startPolling, setlastMessageAt. - SSE events:
tool_resultandtext_deltaevents resetlastMessageAt. - API calls to fetch active executions for the session (new minimal endpoint or derived from the session digest).
- Poll fetches: when a new message arrives via
- In
Chat.svelte, render a thin header bar above the messages area:{#if $currentSession} <div class="activity-bar"> {#if $activity.executionsRunning > 0} <LoaderCircleIcon class="animate-spin" /> {$activity.executionsRunning} running {/if} {#if $activity.planStep} <span>{$activity.planStep}</span> {/if} <span class="ml-auto text-muted-foreground">Last active: {timeago($activity.lastMessageAt)}</span> </div> {/if}
2.5 — SessionDigest: poll live when session is active
Files: web/src/lib/components/SessionDigest.svelte
What: SessionDigest fetches data once when the session loads. For an active session with background work, it should refresh periodically.
How:
- In
SessionDigest.svelte, add a$effectthat watches$currentSessionand the session's status:$effect(() => { if (!sessionId || status === 'done' || status === 'failed') return const timer = setInterval(() => fetchDigest(sessionId), 10000) return () => clearInterval(timer) }) - Only re-render changed parts — Svelte's reactivity handles this since
digestis a reactive declaration. The fetch updates the store; the template re-renders only the changed values (e.g.executions.runninggoes from 4→3).
2.6 — Live execution count in the task header
Files: web/src/pages/Chat.svelte, web/src/lib/api.ts
What: The task context panel shows plan progress, but the chat itself has no execution counter. "3 executions running" in the chat header tells the user at a glance that work is happening.
How:
- Add a
fetchSessionExecutions(sessionId)toapi.tsthat callsGET /api/v1/executions?session_id={id}&status=running,approved. (This endpoint may need adding — or use the existing executions endpoint with a newsession_idfilter.) - Expose as a derived store in
chat.ts; poll it alongside the message poller. - Integrate with the activity bar from 2.4 (they share the same data).
Phase 3: Cleanup (addresses #7, #8 — "leftover state")
3.1 — complete_task: auto-cancel pending executions
Files: cmd/nomos/store.go:480-509
What: When the agent calls complete_task, any executions in
pending_approval or approved state for this session stay that way forever.
The approvals list on the Ops page shows stale items, and the operator
has to manually cancel them.
How:
- In
completeTask, BEFORE updatingagent_sessions, run two cleanup queries:-- 1. Cancel pending-approval executions linked to this session UPDATE executions SET status = 'cancelled', result = '{"message": "task completed — auto-cancelled"}'::jsonb WHERE entity_id IN ( SELECT execution_id FROM nomos_plan_executions WHERE session_id = $1 ) AND status IN ('pending_approval', 'approved', 'queued'); -- 2. Mark them as continued so the worker won't try to auto-continue them UPDATE nomos_plan_executions SET continued_at = now() WHERE session_id = $1 AND continued_at IS NULL; - For each cancelled execution, emit an
execution.cancelledevent so the live approvals list refreshes (same pattern asobservability.Eventcalls elsewhere in store.go). - The sqlcgen observability event from
completeTaskalready firestask.status— addcancelled_count: Nto the event data so the frontend can show "task completed (3 pending approvals auto-cancelled)."
3.2 — complete_task: delete assent window + destructive window
Files: cmd/nomos/store.go:480-509
What: The autonomy_settings table holds the assent window key
(nomos:assent_window:<agent>:<session>) and destructive window key
(nomos:destructive:<agent>:<target>:<session>). On task completion,
these are stale and should be cleaned up.
How:
- Add a cleanup query to
completeTask:WhereDELETE FROM autonomy_settings WHERE key = $1 OR key LIKE $2$1= the session's assent window key and$2= the destructive window pattern for this session. - Compute the keys:
assentWindowKey(a.agentID, sessionID)andnomos:destructive:<agent>:*:<session>pattern. - This is a single DELETE before the return — no transaction needed, fire and forget (failure is logged, not blocking).
3.3 — propose_plan: auto-replace pending steps with replaced status
Files: cmd/nomos/store.go:367-423
What: When the agent calls propose_plan again mid-flight, new steps are
appended after the current max seq (store.go:379-389). Pending steps from the
old plan (status = 'pending') remain in the list forever — they were never
started and never will be.
How:
- In
proposePlan, after theDELETE FROM session_plan_steps WHERE session_id = $1for the fresh-start path (store.go:386), add ELSE logic for the append path:-- Before appending new steps, mark any still-pending steps as 'replaced' UPDATE session_plan_steps SET status = 'replaced', finished_at = now() WHERE session_id = $1 AND status = 'pending'; - The
replacedstatus is already a valid terminal state — the plan panel should treat it the same asskipped(dimmed, no progress contribution). - Add
'replaced'to the stamp switch inupdatePlanStep(store.go:436-441) so no one can accidentally un-replace a step.
3.4 — Plan steps: generation column + frontend collapse
Files: cmd/nomos/store.go (schema migration + proposePlan), web/src/lib/components/PlanPanel.svelte
What: With 3.3, old steps get marked replaced, but the flat list still
shows every step ever created. The operator sees a confusing mix of old and
new plan steps. A generation column lets the frontend group and collapse.
How:
- Migration:
ALTER TABLE session_plan_steps ADD COLUMN generation INTEGER NOT NULL DEFAULT 1; - In
proposePlan, resolve the new generation number:If this is a fresh-start (no steps started, delete + re-insert), reset to 1. If appending, use next generation number (typically 2, 3, ...).SELECT COALESCE(MAX(generation), 0) + 1 FROM session_plan_steps WHERE session_id = $1 - Each new step inserted gets this
generationvalue. - In the
planStepstruct (store.go:567), addGeneration int \json:"generation"``. - In
getPlanSteps, includegenerationin the SELECT. - In
PlanPanel.svelte(or wherever plan steps are rendered), group bygeneration. Show the current generation (highest number) expanded; collapse older ones with a label like "Plan v1 (replaced)" and a count of steps.
3.5 — Sessions list: show open-approval count per session
Files: cmd/nomos/store.go (new query), cmd/nomos/main.go (API response),
web/src/lib/components/SessionRail.svelte
What: The sessions list shows title, status, time. Adding "2 approvals pending" tells the operator at a glance which sessions have open actions.
How:
- Add a
pendingApprovalCountmethod on store:func (s *store) pendingApprovalCounts(ctx context.Context, sessionIDs []string) map[string]int { // SELECT l.session_id, COUNT(*) FROM nomos_plan_executions l // JOIN executions e ON e.entity_id = l.execution_id // WHERE l.session_id = ANY($1) AND e.status = 'pending_approval' // GROUP BY l.session_id } - In
handleSessionsList(main.go:276), call this for the returned sessions and addpending_approvalsto each session JSON object. - In
SessionRail.svelte, show a yellow badge next to sessions withpending_approvals > 0. Clicking navigates to that session AND opens the approvals section.
Phase 4: Continuation hardening (addresses #3 — "agent stops")
4.1 — When assent window is missing, inject a visible note
Files: cmd/nomos/continue.go:145-151
What: When assentWindowActive returns false, the execution is marked
continued without resuming. The user sees nothing. Instead, persist a
visible note in the transcript explaining WHY the agent didn't continue.
How:
- In
processContinuations(continue.go:145-151), beforemarkContinued, insert a system note as an assistant message in the transcript:note := fmt.Sprintf("[System: execution %s finished, but the assent window for this session is not active (may have expired). The agent will not auto-continue. Reply 'continue' or re-approve the plan to resume.]", p.ExecID) a.store.saveMessage(context.Background(), p.SessionID, "assistant", jsonNote) - This makes the auto-continue failure visible to the operator in the chat history — they can see WHY the agent stopped and what to do about it.
4.2 — Re-open expired assent window for running executions
Files: cmd/nomos/continue.go:145-151
What: The plan was approved. The execution ran. The window expired during execution. Penalizing timing is wrong — the agent should still get the result and continue.
How:
- In
processContinuations, whenassentWindowActiveis false, don't just mark it — check if the session's goal is set and the task status is'executing'(meaning a plan was approved and is being worked):if !a.store.assentWindowActive(ctx, a.agentID, p.SessionID) { sesh, _ := a.store.getSession(ctx, p.SessionID) if sesh.Goal != "" && sesh.Status == "executing" { // Plan was approved, work is in progress — re-open the window a.openAssentWindow(ctx, p.SessionID) // THEN continue (don't markContinued — execute the normal path) } else { // No plan, no goal — genuinely shouldn't continue (4.1's note) a.store.markContinued(ctx, p.ExecID) continue } } - The
a.store.markContinued(...)line at continue.go:152 must move inside theif !activeblock. Currently it fires unconditionally before thesafego.Go— that's a bug: it marks executions as continued even when the continuation IS dispatched, which is safe but unnecessary. With this fix, it becomes important thatmarkContinuedONLY fires when we're NOT continuing.
4.3 — Reduce idle sweep interval to 2 min (first pass)
Files: cmd/nomos/continue.go:55
What: The idle sweep fires every 5 minutes. A stopped agent feels dead long before 5 minutes pass.
How:
- Change
time.NewTicker(5 * time.Minute)totime.NewTicker(2 * time.Minute)at continue.go:55. - Keep the two-pass logic (nudge → auto-close) but at 2-minute spacing instead of 5-minute. Net: an agent that stops responding due to a stuck continuation is nudged at 2 min and auto-closed at 4 min instead of 5 and 10.
4.4 — "Continue" button in chat
Files: web/src/lib/stores/chat.ts, web/src/pages/Chat.svelte,
cmd/nomos/main.go
What: When the agent seems stuck, the operator should be able to click "Continue" instead of typing "continue" or "status?".
How:
- Add a
resumeSessionAPI call toapi.ts:export function resumeSession(sessionId: string): Promise<boolean> { return fetchWithAuth(`${BASE}/sessions/${sessionId}/resume`, { method: 'POST' }).then(r => r.ok) } - Add a handler in
main.go:case "/resume": note := "[System: the operator wants you to continue. Pick up where you left off — execute the next step of the plan, diagnose and fix any failures, or report progress if everything is done.]" safego.Go("nomos:resume-session", func() { a.resumeSession(context.Background(), sessionID, note) }) w.WriteHeader(202) - In
Chat.svelte, add a "Continue" button in the activity bar from 2.4. Show it when:- The session is active (
status === 'executing'or'awaiting_input'). - The most recent message is from the agent (not the user).
- No stream is active.
- More than 30 seconds have passed since the last message (agent might
be stuck).
On click, calls
resumeSession($currentSession), starts polling, and shows a brief "Agent resumed" toast.
- The session is active (
- Debounce: disable the button for 30 seconds after clicking to prevent spam.
Phase 5: Knowledge loop (addresses #9, #10, #11, #12, #13, #14 — "the graph keeps drifting")
5.1 — list_lxcs: add state filter parameter
Files: internal/mcp/server.go:664-675
What: list_lxcs returns all 33 LXCs including 13 destroyed ones. The SQL
for list_entities already supports state filtering — port the same pattern.
How:
- Add an optional
stateparameter to the tool'sInputSchema:InputSchema: objSchema( prop{"state", "string", "Optional: filter by entity state (active, destroyed, …)"}, ), - Change the SQL to use the parameter:
WHERE e.type = 'lxc' AND ($1::text IS NULL OR e.state = $1) ORDER BY (e.attributes->>'pve_id')::int - Default (no filter) returns all LXCs — backward compatible. When
stateis passed, only matching entities are returned. - Update SOUL.md to recommend:
list_lxcs(state="active")for fleet audits,list_lxcs(state="destroyed")for cleanup lists. This alone eliminates ~30% of tool calls from the 92-call first turn.
5.2 — upsert_knowledge: accept about as an array of entity slugs
Files: internal/mcp/server.go (the upsert_knowledge handler)
What: upsert_knowledge accepts a single about slug (e.g. cluster:homelab).
The agent can't link one knowledge entry to multiple entities without calling
the tool once per entity. The upgrade results session should have linked to
each upgraded LXC individually.
How:
- Add
aboutas an array in the tool schema (accept both single string and array for backward compat):InputSchema: objSchema( prop{"title", "string", "…"}, prop{"content", "string", "…"}, prop{"kind", "string", "…"}, prop{"tags", "string", "…"}, prop{"about", "array", "Entity slugs this knowledge is about (e.g. ['lxc:nfs-export', 'lxc:gitea'])"}, ), - In the handler, normalize single string → single-element array. Create
documentedrelationships for each slug:knowledge_entity → about → entity. - This lets the agent record: "Low-risk tier upgrades" about
[lxc:nfs-export, lxc:gitea, lxc:dns, lxc:auth-outpost]in one call.
5.3 — SOUL.md: make the writeback instruction unmissable
Files: nomos/SOUL.md (lines 86-103)
What: The instructions exist but the agent skipped them. The plan step descriptions the agent wrote for itself were too vague.
How:
- Add a bold, standalone section near the top of SOUL.md (before the
"Every chat is a task" section):
## ⚠️ AFTER EVERY TASK: WRITE BACK OR LOSE IT What you discovered but didn't write back is **lost** — the next session starts from scratch. Before calling complete_task, you MUST: 1. `update_entity_attributes` — ANY concrete fact (IP, version, host, port, state) for ANY entity you learned about. Every LXC you queried, every target you ran against. 2. `create_relationship` — ANY edge you discovered (hosts, depends-on, provides). Every "X runs on Y" fact. 3. `upsert_knowledge` — narrative: what you did, what broke, the fix. Link to ALL affected entities via `about`. **The plan's LAST step must list these by name.** Not "record findings" — "1. update_entity_attributes for each audited LXC, 2. create_relationship for any discovered host/container edges, 3. upsert_knowledge." Future you depends on this. - Update the
propose_plantool description (tasks.go:42-45) to add stronger language: "Your LAST step MUST have a detail line that literally says '1. update_entity_attributes for X, 2. create_relationship for Y, 3. upsert_knowledge about Z' so you don't skip entity writeback." - This is a cheap, high-impact change — it doesn't require code, just clearer instructions.
5.4 — propose_plan: validate that the last step mentions entity writeback tools
Files: cmd/nomos/tasks.go:196-205 (propose_plan handler)
What: The agent can propose a plan whose last step is "Record findings" and no one checks whether it includes entity attribute updates. Add a validation nudge.
How:
- In
propose_plan, after persisting steps, check if the final step'stitleordetailcontains"update_entity_attributes"or"create_relationship":lastStep := steps[len(steps)-1] hasWriteback := strings.Contains(lastStep.Title+lastStep.Detail, "update_entity_attributes") || strings.Contains(lastStep.Title+lastStep.Detail, "create_relationship") if !hasWriteback { return fmt.Sprintf("Plan set: %d step(s). ⚠️ The final step doesn't mention update_entity_attributes or create_relationship. Without those, any facts you discovered about entities will be lost. Consider adding them to the last step.", len(persisted)), true } - This doesn't enforce — it's a nudge in the tool result. The agent sees it
and can self-correct in the same turn (call
update_plan_stepto fix the last step's detail, or callpropose_planagain with a corrected plan).
5.5 — complete_task: validate that entity attributes were written back
Files: cmd/nomos/store.go:480-509
What: If the agent calls complete_task without having called
update_entity_attributes or create_relationship in this session, the
completion message should warn about it.
How:
- Add a check in
completeTask: queryaudit_logfor this session's use ofupdate_entity_attributesandcreate_relationshipwithin the last N minutes (the session's active window):SELECT EXISTS( SELECT 1 FROM audit_log WHERE details->>'session_id' = $1 AND action IN ('update_entity_attributes', 'create_relationship') AND created_at > now() - interval '60 minutes' ) AS has_writeback - If no writeback was done, return
"Task marked success: … ⚠️ No entity attributes or relationships were updated in this session. Call update_entity_attributes and create_relationship before completing to keep the graph current.". - The existing
complete_taskreturn value (tasks.go:264) includes this text, which becomes the assistant's visible message — the operator and the agent both see the warning.
5.6 — Plan step ordering: enforce seq-order on completion
Files: cmd/nomos/store.go:431-473 (updatePlanStep)
What: The agent marked step 5 as done before step 4 (finding #13). The frontend sees steps completing in nonsensical order.
How:
- In
updatePlanStep, when status isdone/failed/skipped/blocked, check that all lower seq numbers are also in a terminal state:If no rows are affected (because a prior step is still pending), return an error:UPDATE session_plan_steps SET status = $3, … WHERE session_id = $1 AND seq = $2 AND NOT EXISTS ( SELECT 1 FROM session_plan_steps ps WHERE ps.session_id = $1 AND ps.seq < $2 AND ps.status = 'pending' ) RETURNING id, target_slug"Cannot complete step %d — step %d is still pending." - This prevents the out-of-order completion. The agent must mark steps in
order. If the design genuinely allows out-of-order (parallel steps), add
a
depends_on_seqcolumn — but for now, strict ordering is simpler and correct for the agent's sequential plan style.
5.7 — Audit log: persist tool results (not just args) for analysis
Files: cmd/nomos/agent.go:446-461 (logActivity call)
What: The audit log records tool, args, result, success per tool
call. To audit a session like this one programmatically, we need to know
which entities were touched, what their state was before/after, and whether
entity attributes were updated.
How:
- Add a
session_idcolumn to theaudit_logentries written bylogActivity(agent.go:447 and 461). Currently the log has no session linkage — you can't query "what did session X do?" - Add structured fields:
entity_slugs(array of slugs found in args) andtool_name(already present asaction). This lets queries like:which 5.5 needs.SELECT COUNT(*) FROM audit_log WHERE session_id = $1 AND action = 'update_entity_attributes' - This is a low-priority improvement that makes 5.5's check fast (a log
query vs. a full text scan of
result_json).
Database migration (updated — covers 3.4, 3.5, and 5.7)
-- Add generation tracking to plan steps
ALTER TABLE session_plan_steps ADD COLUMN IF NOT EXISTS generation INTEGER NOT NULL DEFAULT 1;
-- Add index for pending-approval lookup by session
CREATE INDEX IF NOT EXISTS idx_nomos_plan_executions_session
ON nomos_plan_executions (session_id) WHERE continued_at IS NULL;
-- Add session_id to audit_log for per-session analysis
ALTER TABLE audit_log ADD COLUMN IF NOT EXISTS session_id UUID;
CREATE INDEX IF NOT EXISTS idx_audit_log_session ON audit_log (session_id);
Implementation order
- Phase 1 first — the session-ending failures (#1 empty responses, #2 disconnects, #3 agent stops) are stop-ship issues that ruin the user experience.
- Phase 5 second — the knowledge loop (#9, #10, #11, #12, #13, #14) is the most impactful set of improvements per token spent. Making
list_lxcsfilterable (5.1) and updating SOUL.md (5.3) are 30-minute changes that eliminate 30% of wasted tool calls and close the writeback gap. Theaboutarray (5.2) and completion validation (5.5) make the knowledge graph actually learn from sessions. - Phase 2 third — visibility improvements. Custom renderers and activity indicators are polish compared to the knowledge loop actually working.
- Phase 3 fourth — cleanup of leftover state. The
complete_taskcancellation (3.1, 3.2) directly addresses the 4 orphaned approvals. - Phase 4 last — continuation hardening builds on the other phases.
What this plan does NOT address
- C1 (unauthenticated nomos gateway) — already in
plans/2026-07-11-nomos-agent-code-review.md. request_executionenum retirement — already inplans/2026-07-10-general-gated-execution.mdLayer 1.- Token-aware history windowing — already in the code review plan (A2), deferred.
- Auto-act revival — already in the general-gated-execution plan, deferred.
- Full audit of the
list_lxcsoutput format — the__renderer: lxc_listannotation works for the frontend but the JSON structure is still flat (no distinction between active/destroyed entities). A richer shape (grouped by state) is future work. - Deduplication of
upsert_knowledgeentries with the same title — the tool currently creates a new entry each time. Same-title updates should upsert instead of duplicate (future work).
Verification
- 1.1-1.3: Kill the nomos process mid-turn, confirm the frontend shows a reconnect banner and resumes polling within 3 seconds.
- 1.4-1.6: Send a prompt known to produce empty responses, confirm retries fire 3x and the error card stays visible until dismissed.
- 2.1: Open a session with
get_execution_statuscalls, confirm each renders as a status card with live polling, not raw JSON. - 2.2: Queue a run that requires approval, confirm the approval card appears immediately (not just on
done). - 3.1-3.2: Complete a task that has pending approvals, confirm they're auto-cancelled and the assent window is cleaned up.
- 3.3-3.4: Call
propose_plantwice in one session, confirm old pending steps are markedreplacedand the panel shows only the current plan. - 4.1-4.2: Run an execution that finishes after the assent window expires, confirm the agent still picks it up and resumes.
- 5.1: Call
list_lxcs(state="active")— confirm only active LXCs are returned, not destroyed ones. Re-run the apt audit scenario — confirm the first turn has ~64 tool calls instead of 92. - 5.2: Call
upsert_knowledgewithabout: ["lxc:nfs-export", "lxc:gitea"]— confirm knowledge is linked to both entities. Queryget_entity_knowledge("lxc:nfs-export")— confirm the entry appears. - 5.3: Send a new fleet audit request. Confirm the agent's proposed plan has a last step that explicitly lists
update_entity_attributes,create_relationship, andupsert_knowledgeby name. - 5.4: Propose a plan whose last step says "Record findings" without mentioning entity writeback. Confirm the tool result warns about it.
- 5.5: Complete a task that never called
update_entity_attributes. Confirm the completion message warns "No entity attributes or relationships were updated." - 5.6: Call
update_plan_step(seq=5, status="done")while step 4 ispending. Confirm the call returns an error: "Cannot complete step 5 — step 4 is still pending."