876f181068703e0f9575937ae42b6cfb560a44f4
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| e3fa6736c0 |
feat(agent): plan-first gate, iterative follow-ups, reasoning persistence
P1 plan-first: run handler refuses without propose_plan (structural gate,
not SOUL.md prose). Plan window decoupled from set_goal — config_mutation
auto-run only on operator approval (assent window). Closes the approval-free
config_mutation hole confirmed in session d0d562e0.
P2 iteration: reopenSession flips terminal→executing, marks prior plan steps
replaced, clears outcome. proposePlan excludes replaced from in-flight check,
bumps generation. A follow-up on a completed session starts a new sub-task
with a fresh plan — no more errPlanInFlight dead end.
P3 reasoning: accumulate per-iteration text into the persisted row instead
of overwriting with the last text event. Reload shows intermediate thinking,
not just the final summary.
P4 read-only allowlist: add find, tree, locate, systemctl list-timers/
list-unit-files/show, timedatectl, hostnamectl, systemd-analyze, rclone
ls/lsl/md5sum/check/cryptcheck. Fixes the find misclassification from
d0d562e0.
P5 eval harness: new assertion kinds (proposes_plan, plan_before_run,
plan_generations), multi-turn followups, fetch /sessions/{id}/plan. Four
manifests under evals/.
P6 SOUL.md: strip degenerate-case carve-out, add ITERATE step, update
set_goal guidance.
VERSION 0.6.0 → 0.7.0
|
|||
| dd3076a23a |
feat(agent): close all post-fix remainders + golden eval harness (F.1-F.2, C.1-C.2, B.4-B.6, E.1-E.2)
Ships the 9 remaining post-fix items and a golden-conversation eval harness that validates them against the live agent. All 4 evals pass. SOUL.md (F.1, C.2, E.1): - Consolidated three overlapping task-flow sections (MANDATORY TASK FLOW, 'Every chat is a task', 'AFTER EVERY TASK: WRITE BACK') into one. ~50 lines shorter. The operator's 'be more crisp' feedback. - Added anti-patterns: don't re-execute on UI/sidebar complaints (C.2); don't re-run fleet-wide audits when same-day knowledge exists (E.1). - Updated approval vocabulary in step 4 to match tasks.go (approved/yes/ go/proceed/continue/ok/go ahead). Tool-result strings (F.2): - set_goal: tightened to 'Goal set. NEXT: pre-plan (read-only tools only). Then propose_plan. Do not call run.' - update_plan_step: added '(Advance with update_plan_step + run; do not re-propose.)' C.1 — completeTask rejects re-completion of a terminal session: - Returns errTaskAlreadyComplete when status is already done/failed. - The tool result directs: 'Task is already complete. Do not call complete_task again. If the operator pointed out a UI/sidebar inconsistency, fix it with update_plan_step...' B.4 — Surface real model error text: - chatWith's error event now includes finish_reason + refusal text: 'Nomos returned an empty or unusable response (finish_reason=length). Retry or rephrase.' instead of generic 'empty response'. - The resume-failed note already carried errText (B.3), which now has the real context. B.5 — Back off between resume retries (4s, 8s): - resumeSession now sleeps before attempts 1 and 2 (exponential backoff). A transient provider issue gets time to clear instead of 3 identical calls in 3 seconds. B.6 — Don't persist the empty placeholder as a visible bubble: - If a chat turn ends with no text and no tool calls (model empty-response'd and all retries failed), delete the placeholder row instead of persisting an empty bubble. The error was already streamed via done+error=true. E.2 — list_lxcs last-audited hint: - The list_lxcs result now includes last_audited_at — the most recent knowledge entry (tagged audit/update, or titled audit/update) linked via an 'about' edge. The agent can see 'nextcloud — last audited today' and skip re-running it. Tool-call doubling bug fix (found by the eval harness): - main.go + continue.go: the tool_use and tool_result events were both appending separate entries to the persisted tool_calls array, doubling every tool call in the transcript. Confirmed pre-existing (d9cdcee1, v0.3.x era). Fixed: tool_use creates the entry, tool_result merges the result into the same entry (matched by id). One entry per tool call. Golden eval harness (cmd/nomos/eval/): - A standalone Go program that loads YAML manifests of golden conversations + assertions, sends prompts to the chat endpoint, drains the SSE stream (keeping the agent's context alive), and scores structural assertions against the persisted transcript. - 4 golden conversations covering: trivial read-only (degenerate case), plan + proceed (the original duplication bug), UI complaint (no re-exec), fleet audit (knowledge preferred over re-execution). - Structural assertions only (tool-call sequences, plan steps, writeback, completion) — text quality is model-dependent and not scored. - Run: go run ./cmd/nomos/eval -gateway http://localhost:8092 -manifest cmd/nomos/eval/evals/*.yaml (~$0.10/run in OpenRouter credits). Eval results (4/4 passed): trivial_readonly: 2 tool calls, no plan, no run plan_advances_on_proceed: 13 tool calls, propose_plan x1, writes back ui_complaint_no_rerun: 12 tool calls, propose_plan x1, writes back knowledge_preferred_over_rerun: 7 tool calls, search_knowledge x1, 0 run Version 0.5.2 -> 0.5.3 (minor: eval harness + structural hardening). |
|||
| 337d577f00 |
fix(agent): refuse plan re-proposal + emit done on error (close divergence chain)
Operator-reported bug: on 'proceed with the rest' the agent re-proposed the
plan, duplicating it in the sidebar. Root cause was a three-bug chain, not
one bug:
1. Trigger — model empty-response on 'proceed' (approval vocabulary didn't
list 'proceed', so the agent wasn't sure it was approved and no-op'd).
2. Amplifier — chatWith emitted 'error' without 'done' on empty response
(agent.go:370). The frontend's onComplete saw !receivedDone and
misclassified the model failure as a network disconnect, calling
handleDisconnect -> resumeSession.
3. Divergence — the reconnect note was generic ('report your state'), so
the agent re-proposed + re-executed instead of advancing the plan.
Fixes (shipped, e2e-validated against the live agent on oikos-nomos-1):
- A.2: proposePlan refuses re-proposal once a step has started (returns
errPlanInFlight). Drops the append-mode safety net (commit
|
|||
| 60effcb2fe |
session reliability: reconnect, knowledge loop, retire request_execution
Phase 1 — crash recovery: SSE auto-reconnect + backoff, polling gate during disconnect, connection banner with retry button, empty-response retry 3x, non-terminal resume on empty response, persistent error cards. Phase 2/4 — visibility + continuation: custom ExecutionStatus renderer, approvals extracted on every tool_result (not just done), activity bar with status/goal, SessionDigest live polling, Continue button. Phase 3 — cleanup: complete_task auto-cancels orphaned approvals, deletes assent/destructive window keys, propose_plan marks pending steps as replaced, plan step seq-order enforcement. Phase 5 — knowledge loop: list_lxcs state filter (active/destroyed), SOUL.md unmissable writeback section, propose_plan validation nudge, complete_task writeback check, upsert_knowledge about array support, plan generation grouping in frontend, session approval count badge. Retire request_execution — all mutations now route through run. Updated SOUL.md, AGENTS.md, CLIENTS.md, skills, and agent system notes. Migration 020: plan step generation column, audit_log session_id index, nomos_plan_executions pending-approval index. |
|||
| 3b9c75fa3f |
fix(agent): task completion safety net — stop tasks sticking at Running
Implements fixes 1-3 of plans/2026-07-11-task-completion-safety-net.md. Confirmed live that 50/50 production sessions never reached a terminal status because the model almost never calls complete_task, even for trivial single-tool Q&A turns SOUL.md explicitly calls out as needing it. - Inline safety net (agent.go): a session that never called set_goal never framed itself as a structured task, so its first plain-text turn-end IS the task ending — auto-complete it there instead of leaving status stuck at its creation default forever. - Idle sweep (continue.go, new completion_nudges column): goal-bearing sessions that stall get one nudge, then auto-close with outcome=partial if the nudge goes unanswered, mirroring the pattern resumeSession already uses for a different stuck-session failure mode. Fix 4 (backfill of the 50 already-stuck live sessions) is deliberately separate — deferred until this is deployed and verified live, per the plan's implementation order. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
|||
| 6d4f6de676 |
fix(agent): mark a task failed when its resume permanently gives up (B3)
Fix B3 of plans/2026-07-11-nomos-agent-code-review.md. resumeSession's retry loop (used by both auto-continuation and panel-answered questions) already retried once on a transient LLM failure, but if BOTH attempts came back empty/erroring, the code just logged and returned — the task was left at whatever status it already had (typically 'executing' or 'awaiting_input') with no outcome, no operator-visible signal beyond an inert error line buried in the transcript, and no way to tell a genuinely stuck task apart from one quietly still working. On permanent failure, now calls store.completeTask(outcome='failure', a summary built from the error) so the task board reflects reality instead of showing a task that looks perpetually in-progress. Uses context.Background() for that write, matching resumeSession's own persistence pattern, since the context that led to the failure may itself be in a bad state. This doesn't prevent the operator from continuing to work the task via a fresh chat message afterward — it only replaces silent hanging with a real status. A full live induction of a permanent LLM outage would require breaking the model/API-key config for the whole nomos container — too invasive for this fix's priority. Verified instead that the new branch stays correctly dormant on the happy path: ran a real ask_operator → panel-answer → resume cycle end-to-end and confirmed the task landed at status='executing' with no outcome set, proving the failure-handling code doesn't false-positive on a normal successful resume. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
|||
| c5ffaec85b |
fix(agent): panic recovery on every background goroutine (B1+B2)
Fixes B1 and B2 of plans/2026-07-11-nomos-agent-code-review.md together, since the right granularity for B1 in the auto-continuation worker turned out to require B2's restructuring anyway (see below). B1: grep -rn "recover()" cmd/nomos/ internal/mcp/ internal/httpapi/ returned nothing before this — every explicitly-spawned goroutine (continuation worker, resumed chat turns, async execution dispatch, the SSE listener, two duplicate sshExec implementations' output-collector goroutines) crashed the whole process on an unhandled panic, not just that one goroutine. More consequential post-concurrency: more simultaneous unattended background work means more surface area for one bad input to end every running task. New internal/safego package: Go(label, fn) launches fn in a goroutine with a recover-and-log wrapper. Applied at every bare `go` spawn site across the three packages. Two sites needed bespoke handling instead of the generic helper because their callers block on a channel and a silent recover would just make them hang until timeout: sshExec's output-collector goroutine (two near-identical copies, internal/mcp/server.go and internal/httpapi/phase3.go) and httpapi's ListenAndServe goroutine — both now recover AND send a synthetic error result so the waiting select unblocks immediately instead of waiting out the full timeout. httpapi's sseListener got extra treatment: its per-notification handling was extracted into handleNotification with its own recover, so a panic decoding ONE malformed pg_notify payload can't kill the listener goroutine for every connected SSE client — the outer goroutine spawn only needs to guard the connection setup/reconnect code around it. B2: cmd/nomos/continue.go's processContinuations used to run every pending continuation SEQUENTIALLY in a plain for loop, in the SAME goroutine as the ticker — meaning (a) task B's continuation waited for task A's full (up to 10-minute) resumed turn to finish first, undercutting this session's earlier concurrency work on exactly the path autonomous tasks depend on most, and (b) an unrecovered panic anywhere in that call chain didn't just crash the process (B1) — even WITH B1's recovery wrapped only at the top-level worker spawn, the panic would still unwind the ENTIRE ticker-loop goroutine, silently ending auto-continuation for every task until nomos restarted. Fixed by spawning each pending item via safego.Go individually: real parallelism, and a bad item can now only ever take down its own goroutine. Added internal/safego/safego_test.go: TestGo_RecoversPanic is the concrete proof — a deliberate panic inside Go() that would otherwise crash the whole test binary; reaching the assertion after it IS the evidence recovery works. Verified live against the rebuilt containers: full chat turn round-tripped correctly (hostname lookup, 2 iterations, normal completion) — no regression from threading safego.Go through the tool-dispatch/continuation paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
|||
| 9ef1ba3702 |
fix(concurrency): scope assent/destructive windows to session, not just agent
Fix 1 of plans/2026-07-11-concurrent-task-execution.md — the safety-critical
one. The assent window (and destructive window) were keyed purely by agent
id ("assent_window.agent:<uuid>"). With one agent:nomos entity serving every
concurrent task, this meant approving Task A's plan opened a window that ANY
concurrently-running task's config-mutation/destructive actions could also
ride, auto-executing without their own approval.
- store.go / agent.go: assentWindowActive/openAssentWindow and
destructiveWindowActive/openDestructiveWindow/destructiveWindowKey all gain
a sessionID parameter; keys become
"assent_window.agent:<id>.session:<sessionID>" and
"destructive_window.agent:<id>.target:<slug>.session:<sessionID>". Missing
session id fails closed (no window) rather than falling back to the old
agent-wide key.
- continue.go: the auto-continuation worker's window check moved from once-
per-batch to once-per-pending-item, scoped to that item's own session —
it was previously checking ONE agent-wide window for a batch that can span
multiple tasks.
- agent.go tool-dispatch: injects `_session_id` into a COPY of the wire args
sent to the MCP server (never into the args used for the emitted/logged/
persisted tool call, and never part of any tool's declared InputSchema —
invisible to the model) so the gating checks on the OTHER side of the
process boundary know which task is asking.
- internal/mcp/server.go: assentWindowActive/destructiveWindowActive/
classifyAndGate gain the same sessionID parameter, read from
args["_session_id"] at the three call sites (request_execution's
apt_upgrade/pct_create branches, and the shared classifyAndGate used by
restart/pct_exec/systemctl/run).
Verified against the live stack with the exact scenario from the plan: opened
an assent window for session A only, then called `run` with an identical
config-mutation command for session A (window open) and session B (same
agent, no window). A auto-ran (execution status completed); B correctly
queued for approval (pending_approval) instead of bleeding through — proven
at both the MCP response text and the executions table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|||
| 014e5c74e0 |
feat(tasks): phase 5 — ask_operator (structured question, pause, resume)
The last backend piece: when the agent hits a decision only the operator can
make, it surfaces a structured question instead of guessing or stalling.
- ask_operator(prompt, why?, options?, context_entities?): nomos-local tool
that records a session_questions row, moves the task to awaiting_input, emits
question.raised, and ENDS the turn (the agent loop returns after it, so the
agent can't barrel past its own question). The prompt becomes the assistant's
visible message so the question also shows inline in the transcript.
- Two resume paths, both close the question + emit question.answered + return
the task to executing:
- Panel: POST /sessions/{id}/questions/{qid}/answer → resumes the agent in the
background with the answer injected (reusing the continuation machinery,
refactored continueSession → resumeSession). Returns 202; the reply lands via
message polling.
- Chat reply: the next chat message on a task with an open question IS the
answer — auto-closed in handleChat; the turn itself is the resume.
Verified end-to-end: forcing a decision paused the task at awaiting_input with
the structured question (prompt/why/options/entities); a panel answer resumed
the agent (it acknowledged host:strong and continued); a plain chat reply
auto-closed a second question. Cleanup + tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|||
| 233b5e4519 |
feat: live visibility into what the agent is running (no more silent waiting)
Operator: "I'd like to be able to see in the chat what the agent is actually running, right now I just wait while nothing happens." Two compounding gaps: 1. The auto-continuation worker (cmd/nomos/continue.go) had zero live push — its result only appeared on a manual page reload, so approving a plan and watching the chat looked completely dead even while the agent was actively working. 2. Even with polling, continueSession only persisted ONE message at the very end of a continuation — a continuation that runs several tool calls before concluding would still show total silence for however long that took. Fixed both: - web/src/lib/stores/chat.ts: polls the current session's messages every 3s between turns (never while a live stream owns the message list) and merges in anything new. Started after a live turn ends and when a session loads; stopped on new-chat/session-switch. - cmd/nomos/store.go: insertMessageReturningID/updateMessage — lets a message be created as a placeholder and updated in place. - cmd/nomos/continue.go: continueSession now inserts a placeholder the instant it starts (renders as the existing "thinking" dots — immediate feedback that something is happening) and updates that SAME row after EVERY tool call, not just at the end. A poll within ~3s of any tool call landing shows it — individual `run` commands appear as the agent issues them, not just the final rolled-up summary. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
|||
| 6f9998fa29 |
fix: auto-continuation silently dropped LLM errors + added outer retry
Verified live: the new auto-continuation worker (previous commit) worked end
to end for the happy path (provision -> auto-verify -> report success, zero
operator ticks). But testing a failure-recovery case (a destroy that failed
because the container was still running) surfaced a real bug: continueSession's
emit closure only captured "text" events, so when chatWith ended the turn on
an "error" event (LLM returned an empty/refusal response, internal retry also
empty), the worker persisted a completely blank, uninformative "auto" message
— no sign anything had gone wrong, undermining observability of the very
mechanism just built.
- Capture "error" events and, if the turn produced no text/tool_calls at all,
persist an explanatory placeholder instead of blank.
- Add one outer retry of the whole chatWith call when the first attempt
produces nothing — the principle behind this whole feature ("don't give up
on the first error") should apply to the continuation mechanism itself, not
just the homelab commands it's continuing.
Also verified live: recovery-from-failure works via the normal chat path once
prompted, and cleaned up the test container.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|||
| d2f749d33d |
feat: event-driven auto-continuation — agent runs an approved plan to completion
The root cause behind "the agent stops at the first error and doesn't recover":
provisioning executions run ASYNCHRONOUSLY (pct_create fires the SSH work in a
goroutine and returns "running" immediately), so the agent's turn ENDS before
the result exists. The agent literally isn't running when the step fails — it
can't react to a failure it never observes. The only thing that fed results
back was the operator typing "continue" after every async step: the human was
the event loop. (In the flagged 18-message session the operator typed
continue/proceed/?? eight times while the agent correctly diagnosed each failure
but couldn't advance a step on its own.)
This makes the system the event loop instead:
- migrations/017: nomos_plan_executions links each gated execution to the chat
session that started it.
- cmd/nomos: after a tool result, any "execution <uuid>" it started is linked
to the session. A background worker (continue.go) polls for those executions
reaching a terminal state and — while the agent has an open assent window (an
approved plan is in flight) — re-invokes the agent with the result
("execution X completed/failed: <result>"), so it proceeds to the next step
or diagnoses+fixes the failure, with no operator tick. Guarded against loops
(mark-continued before running) and bounded by the 30-min window.
- chatWith(): chat() variant that injects the finished-execution note after
replayed history without persisting a fake user turn.
- DecideApproval: approving a step by ANY route (button or chat-assent) now
opens the assent window, so auto-continuation works regardless of how the
operator approved — previously only typing "go ahead" opened it.
- SOUL: the agent is told it will be auto-re-invoked when async steps finish —
don't poll get_execution_status, don't wait for "continue"; end the turn and
keep going step by step until the goal is verified or a genuine blocker.
This is the root fix, not another per-command patch: you can't enumerate every
failure of an unbounded action space, but you can give the agent a loop that
observes each result and adapts — because "do anything" always includes "the
first attempt failed."
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
