The auto-complete fired when the agent hit the P5 approval gate — it
queued a config_mutation run for approval, the P5 gate blocked further
runs, the turn ended, and auto-complete closed the session as 'partial'.
The operator's approval would then land on a dead task.
Fix: hasPendingApprovals check — if the session has any executions in
pending_approval state, skip auto-complete. The session stays in
'executing' until the operator approves (or denies).
VERSION 0.7.5 → 0.7.6
The auto-complete safety net required hadEntityWriteback to be true,
which meant sessions where the agent did the work but forgot to call
update_entity_attributes stayed stuck in 'executing' forever.
Relax: auto-complete fires if the agent did discovery (ran run),
regardless of writeback. If writeback happened → success; if not →
partial (honest: work was done but knowledge graph not updated).
VERSION 0.7.3 → 0.7.4
The agent often skips update_plan_step bookkeeping (leaving steps
pending/running) but still does the work + writeback. The strict
allPlanStepsTerminal check missed these cases.
Add path (b): if the agent did discovery (ran `run`) AND wrote back
(update_entity_attributes/create_relationship), auto-complete. D.1 already
enforces writeback before completion — if writeback happened, the work
is done.
The #1 remaining model reliability gap: the agent does the work (proposes
plan, executes all steps, writes back) but forgets to call complete_task,
leaving the session stuck in 'executing'. The eval showed 3/8 failures
with this pattern.
Fix: autoCompleteIfPlanDone — a structural safety net that fires at both
chat exit paths (normal completion + maxIterations). If the session has a
goal, the agent didn't call complete_task, and ALL plan steps are in a
terminal state (done/failed/replaced/skipped/blocked), auto-complete with
the agent's final text as the summary. Mirrors autoCompleteTrivialTask
but for structured tasks where the work is provably done.
Also: bump maxLLMRetries from 2 to 3 (complex multi-turn flows benefit
from one more retry on empty responses).
SOUL.md step 4: all-read-only plans skip the approval wait and execute
immediately. Only config_mutation/destructive steps need operator approval.
set_goal + propose_plan return text updated to match.
Fixes 3/4 eval failures where the agent proposed a plan then waited
for approval on a read-only task.
P1 plan-first: run handler refuses without propose_plan (structural gate,
not SOUL.md prose). Plan window decoupled from set_goal — config_mutation
auto-run only on operator approval (assent window). Closes the approval-free
config_mutation hole confirmed in session d0d562e0.
P2 iteration: reopenSession flips terminal→executing, marks prior plan steps
replaced, clears outcome. proposePlan excludes replaced from in-flight check,
bumps generation. A follow-up on a completed session starts a new sub-task
with a fresh plan — no more errPlanInFlight dead end.
P3 reasoning: accumulate per-iteration text into the persisted row instead
of overwriting with the last text event. Reload shows intermediate thinking,
not just the final summary.
P4 read-only allowlist: add find, tree, locate, systemctl list-timers/
list-unit-files/show, timedatectl, hostnamectl, systemd-analyze, rclone
ls/lsl/md5sum/check/cryptcheck. Fixes the find misclassification from
d0d562e0.
P5 eval harness: new assertion kinds (proposes_plan, plan_before_run,
plan_generations), multi-turn followups, fetch /sessions/{id}/plan. Four
manifests under evals/.
P6 SOUL.md: strip degenerate-case carve-out, add ITERATE step, update
set_goal guidance.
VERSION 0.6.0 → 0.7.0
Ships the 9 remaining post-fix items and a golden-conversation eval harness
that validates them against the live agent. All 4 evals pass.
SOUL.md (F.1, C.2, E.1):
- Consolidated three overlapping task-flow sections (MANDATORY TASK FLOW,
'Every chat is a task', 'AFTER EVERY TASK: WRITE BACK') into one. ~50
lines shorter. The operator's 'be more crisp' feedback.
- Added anti-patterns: don't re-execute on UI/sidebar complaints (C.2);
don't re-run fleet-wide audits when same-day knowledge exists (E.1).
- Updated approval vocabulary in step 4 to match tasks.go (approved/yes/
go/proceed/continue/ok/go ahead).
Tool-result strings (F.2):
- set_goal: tightened to 'Goal set. NEXT: pre-plan (read-only tools only).
Then propose_plan. Do not call run.'
- update_plan_step: added '(Advance with update_plan_step + run; do not
re-propose.)'
C.1 — completeTask rejects re-completion of a terminal session:
- Returns errTaskAlreadyComplete when status is already done/failed.
- The tool result directs: 'Task is already complete. Do not call
complete_task again. If the operator pointed out a UI/sidebar
inconsistency, fix it with update_plan_step...'
B.4 — Surface real model error text:
- chatWith's error event now includes finish_reason + refusal text:
'Nomos returned an empty or unusable response (finish_reason=length).
Retry or rephrase.' instead of generic 'empty response'.
- The resume-failed note already carried errText (B.3), which now has
the real context.
B.5 — Back off between resume retries (4s, 8s):
- resumeSession now sleeps before attempts 1 and 2 (exponential backoff).
A transient provider issue gets time to clear instead of 3 identical
calls in 3 seconds.
B.6 — Don't persist the empty placeholder as a visible bubble:
- If a chat turn ends with no text and no tool calls (model empty-response'd
and all retries failed), delete the placeholder row instead of persisting
an empty bubble. The error was already streamed via done+error=true.
E.2 — list_lxcs last-audited hint:
- The list_lxcs result now includes last_audited_at — the most recent
knowledge entry (tagged audit/update, or titled audit/update) linked
via an 'about' edge. The agent can see 'nextcloud — last audited today'
and skip re-running it.
Tool-call doubling bug fix (found by the eval harness):
- main.go + continue.go: the tool_use and tool_result events were both
appending separate entries to the persisted tool_calls array, doubling
every tool call in the transcript. Confirmed pre-existing (d9cdcee1,
v0.3.x era). Fixed: tool_use creates the entry, tool_result merges the
result into the same entry (matched by id). One entry per tool call.
Golden eval harness (cmd/nomos/eval/):
- A standalone Go program that loads YAML manifests of golden conversations
+ assertions, sends prompts to the chat endpoint, drains the SSE stream
(keeping the agent's context alive), and scores structural assertions
against the persisted transcript.
- 4 golden conversations covering: trivial read-only (degenerate case),
plan + proceed (the original duplication bug), UI complaint (no re-exec),
fleet audit (knowledge preferred over re-execution).
- Structural assertions only (tool-call sequences, plan steps, writeback,
completion) — text quality is model-dependent and not scored.
- Run: go run ./cmd/nomos/eval -gateway http://localhost:8092 -manifest
cmd/nomos/eval/evals/*.yaml (~$0.10/run in OpenRouter credits).
Eval results (4/4 passed):
trivial_readonly: 2 tool calls, no plan, no run
plan_advances_on_proceed: 13 tool calls, propose_plan x1, writes back
ui_complaint_no_rerun: 12 tool calls, propose_plan x1, writes back
knowledge_preferred_over_rerun: 7 tool calls, search_knowledge x1, 0 run
Version 0.5.2 -> 0.5.3 (minor: eval harness + structural hardening).
D.1 — complete_task structural gate:
- hadDiscovery(ctx, session) reports whether the session ran `run` successfully
against a live target (NOT get_entity/list_lxcs — those are DB lookups, not
new facts). A trivial Q&A that only calls get_entity is a degenerate case
and must NOT be blocked.
- complete_task with outcome=success is REFUSED when hadDiscovery && !
hadEntityWriteback. The refusal fires BEFORE completeTask runs, so the
session stays in 'executing' state and the agent must call
update_entity_attributes/create_relationship then retry complete_task.
An explicit failure/partial is allowed through (the agent is acknowledging
it didn't finish — no reason to force writeback).
- Replaces the prior advisory warning (5.5) which the agent consistently
ignored. The agent saw the warning and ended the task anyway; this gate
makes the writeback a hard prerequisite for success.
D.2 — propose_plan auto-append writeback step:
- When the agent proposes a plan whose steps don't mention
update_entity_attributes or create_relationship, D.2 appends a final
'Write back: update_entity_attributes + create_relationship +
upsert_knowledge' step before persisting. The result string tells the
agent it was appended.
- With the seq-order enforcement (5.6) and D.1's complete_task gate, the
agent must complete the writeback step (and actually call the tools) to
finish. Neither relies on the agent reading SOUL.md.
- Removed the old advisory writeback nudge from propose_plan's result
string — D.2 makes it structural.
- Updated the propose_plan tool description to state both gates crisply.
Verification:
- TestHadDiscoveryAndWriteback: hadDiscovery true only after a successful
`run`; false after failed run, get_entity, or no calls. hadEntityWriteback
true only after update_entity_attributes/create_relationship.
- e2e against the live agent (oikos-nomos-1, v0.5.1):
- D.2: agent proposed 3 steps (no writeback); D.2 auto-appended step 4
'Write back: update_entity_attributes + ...'. Result string said
'(appended a writeback step — your plan didn't include one; step 4)'.
- D.1: agent ran `run` (uptime on lxc:gitea), called complete_task, was
REFUSED ('Refused: this session ran run against live targets (discovery)
but did not call update_entity_attributes...'). Agent self-corrected:
called update_entity_attributes, retried complete_task, succeeded.
Knowledge loop closed end-to-end.
Version 0.5.0 -> 0.5.1 (patch: structural enforcement of existing intent).
Operator-reported bug: on 'proceed with the rest' the agent re-proposed the
plan, duplicating it in the sidebar. Root cause was a three-bug chain, not
one bug:
1. Trigger — model empty-response on 'proceed' (approval vocabulary didn't
list 'proceed', so the agent wasn't sure it was approved and no-op'd).
2. Amplifier — chatWith emitted 'error' without 'done' on empty response
(agent.go:370). The frontend's onComplete saw !receivedDone and
misclassified the model failure as a network disconnect, calling
handleDisconnect -> resumeSession.
3. Divergence — the reconnect note was generic ('report your state'), so
the agent re-proposed + re-executed instead of advancing the plan.
Fixes (shipped, e2e-validated against the live agent on oikos-nomos-1):
- A.2: proposePlan refuses re-proposal once a step has started (returns
errPlanInFlight). Drops the append-mode safety net (commit 5384499) that
was the direct source of the sidebar duplication. The agent must advance
with update_plan_step + run; the tool result directs it.
- A.1: proposePlan sets the 'generation' column on INSERT (migration 020
added the column + frontend grouping, but the INSERT never wired it).
- A.3: propose_plan tool description restated as a crisp contract (ONCE,
STOP and wait, REFUSES once a step started, advance with update_plan_step).
- F.3: approval vocabulary expanded to approved/yes/go/proceed/continue/ok/
go ahead; propose_plan result string tightened to an imperative.
- B.1: chatWith emits 'done' after 'error' on every terminal path via a new
emitError helper. The frontend now treats model errors as ended (not
disconnected), so no auto-reconnect -> resumeSession fires.
- B.2: reconnect/resume note carries the operator's last message + an
explicit 'advance the plan, do NOT call propose_plan again' directive when
a plan is in flight. Wired into all 4 resume entry points (reconnect,
/resume, idle-sweep, question-answer) via enrichResumeNote.
- B.3: resumeSession escalates the recovery note across its 3 attempts (final
retry: 'pick the lowest-pending step, mark it running, call run — do that
now') instead of 3 identical notes -> 3 identical empties.
Verification: TestProposePlan_RefuseInFlight replaces TestProposePlan_
AppendVsReplace. e2e conversations against the rebuilt container:
conv2 ('proceed with the rest') -> 0 propose_plan calls, plan stayed at
3 steps (was 6+ before), update_plan_step x5 + run x2 + complete_task.
conv3 (full plan, 'go ahead') -> apt-get update on lxc:dns auto-ran under
the plan window, update_entity_attributes writeback, clean complete_task.
nomos logs show zero reconnect/resume entries for the plan-proposing
sessions (the three-bug chain is closed).
Remaining (not in this commit): D.1 refuse complete_task without writeback
(next blocker), C.1/C.2, F.1/F.2 SOUL.md consolidation, B.4-B.6, E.1/E.2.
See plans/2026-07-14-post-fix-session-remainders.md.
Also: re-audit 2026-07-10-general-gated-execution.md — request_execution enum
retirement (60effcb) closes item 9; only auto-act revival (item 10) remains.
Version 0.4.1 -> 0.5.0 (minor: new structural behavior, not a bugfix).
SOUL.md: mandatory 6-step task flow at TOP of file, unmissable.
Agent MUST: set_goal → pre-plan (research only) → propose_plan → STOP
and wait for approval → execute (auto-run under plan window).
Backend:
- set_goal now opens plan window immediately (config_mutation auto-runs)
- set_goal result tells agent to do pre-plan + propose_plan, not run
- propose_plan result tells agent to STOP and wait for approval
- plan window value unified to 'active' (set_goal + propose_plan)
This prevents 23 individual approval popups — one plan approval instead.
Implements fixes 1-3 of plans/2026-07-11-task-completion-safety-net.md.
Confirmed live that 50/50 production sessions never reached a terminal
status because the model almost never calls complete_task, even for
trivial single-tool Q&A turns SOUL.md explicitly calls out as needing it.
- Inline safety net (agent.go): a session that never called set_goal never
framed itself as a structured task, so its first plain-text turn-end IS
the task ending — auto-complete it there instead of leaving status stuck
at its creation default forever.
- Idle sweep (continue.go, new completion_nudges column): goal-bearing
sessions that stall get one nudge, then auto-close with outcome=partial
if the nudge goes unanswered, mirroring the pattern resumeSession already
uses for a different stuck-session failure mode.
Fix 4 (backfill of the 50 already-stuck live sessions) is deliberately
separate — deferred until this is deployed and verified live, per the
plan's implementation order.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes D1-D3 of plans/2026-07-11-nomos-agent-code-review.md:
- D1: deleted isTaskTool — defined, never called (dispatch already checks
handleTaskTool's own `handled` return value).
- D2: recordTouched issued one SELECT per entity slug found in a tool call's
args; batched into one `WHERE slug = ANY($1)` query. Verified live: a turn
naming three separate entities recorded involves edges for all three via
the single batched lookup.
- D3: complete_task's outcome had a declared enum (success|failure|partial)
in its tool schema but nothing validated it — an out-of-enum value (model
typo or a weaker model not respecting the schema) silently persisted as-is,
with only "failure" special-cased (anything else became status='done'
regardless of what the value actually said). Now validated in
handleTaskTool: empty defaults to "success" (unchanged), a recognized value
passes through, anything else defaults to "partial" (safer than silently
treating an unrecognized value as success) with a warning logged. Verified
live: instructed the agent to call complete_task with outcome="unclear" —
persisted as outcome='partial', not the literal invalid string.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the gap that made the knowledge loop optional/implicit: every
non-trivial task now has an EXPLICIT first plan step (research) and last
plan step (write back), not just background behavior the model might skip.
New MCP tools (the agent had no way to do these before — only REST endpoints
existed, unexposed to it):
- update_entity_attributes(slug, attributes): shallow-merge new/changed facts
into an entity (an IP, a version, a discovered port) so a future task
doesn't have to rediscover them from scratch. No approval required — this
updates the knowledge graph, not live infra.
- create_relationship(source, target, type): record a discovered edge
(depends-on, hosts, provides, ...). Idempotent, FK-validated against the
ontology's relationship_types, no approval required.
SOUL.md: restructured the task loop so step 1 is explicitly "gather
knowledge, not just status" (get_entity_knowledge, search_knowledge,
get_relations, get_blast_radius, http_get) and the last step before
complete_task is explicitly "write back" (update_entity_attributes,
create_relationship, upsert_knowledge) — both called out as real plan
entries the operator should see in propose_plan, not silent side-work. This
is what prevents the graph drifting from reality and is the concrete
mechanism behind "tasks compound."
propose_plan's tool description reinforces the same first-step/last-step
convention at the call site.
Verified against the live stack: both tools registered and callable via MCP;
update_entity_attributes merged an attribute correctly; create_relationship
rejected an invalid type (FK violation, clear error) and succeeded with a
valid type+direction, confirmed idempotent (2 calls, 1 row).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause: proposePlan unconditionally deleted and replaced the whole
session_plan_steps list on every call. The model isn't strictly held to
"call propose_plan once with the full list" — nothing stopped it (and
production evidence + live testing showed it happening) from calling
propose_plan once per step as it worked. Each such call wiped every
already-completed step, so the operator only ever saw the model's latest
single step ("1/1") instead of the real, growing plan.
Fix, two layers:
- store.go: proposePlan now only does a destructive replace when no step
has left 'pending' yet (a genuine pre-execution revision). Once any step
has started, a new call APPENDS after the current max seq instead of
wiping — so the panel accumulates the full history regardless of how the
model chooses to call the tool. plan.proposed now carries `appended` so
the frontend knows whether to replace or append.
- workspace.ts: plan.proposed handler respects `appended` (update vs set).
- tasks.go / SOUL.md: strengthened the propose_plan description and task-
loop guidance to call it ONCE with the complete step list end-to-end,
using update_plan_step (not re-calling propose_plan) to advance — fixing
the root behavioral cause, with the store-side append as a safety net
that holds even if the model still calls it incrementally.
Verified: forced the exact incremental-call pattern (propose_plan with 1
step, mark it running, propose_plan again with 1 more step) — the second
call appended at seq 2 instead of erasing seq 1, and its plan.proposed
event carried appended=true.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The last backend piece: when the agent hits a decision only the operator can
make, it surfaces a structured question instead of guessing or stalling.
- ask_operator(prompt, why?, options?, context_entities?): nomos-local tool
that records a session_questions row, moves the task to awaiting_input, emits
question.raised, and ENDS the turn (the agent loop returns after it, so the
agent can't barrel past its own question). The prompt becomes the assistant's
visible message so the question also shows inline in the transcript.
- Two resume paths, both close the question + emit question.answered + return
the task to executing:
- Panel: POST /sessions/{id}/questions/{qid}/answer → resumes the agent in the
background with the answer injected (reusing the continuation machinery,
refactored continueSession → resumeSession). Returns 202; the reply lands via
message polling.
- Chat reply: the next chat message on a task with an open question IS the
answer — auto-closed in handleChat; the turn itself is the resume.
Verified end-to-end: forcing a decision paused the task at awaiting_input with
the structured question (prompt/why/options/entities); a panel answer resumed
the agent (it acknowledged host:strong and continued); a plain chat reply
auto-closed a second question. Cleanup + tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gives a task a legible, live-advancing plan via three more nomos-local tools:
- set_goal(goal): records the task goal, status → planning, emits goal.set.
- propose_plan(steps[]): persists ordered steps (clean replace for v1 — a
revision starts a new list), status → executing, emits plan.proposed with
the persisted steps (id+seq) so the panel can address them.
- update_plan_step(seq, status, execution_id?): advances a step, stamping
started_at/finished_at, emits plan.step.started/finished. Anchors the event
to the step's target entity when it has one.
Belt-and-suspenders: when an execution linked to a step reaches a terminal
state, the api auto-closes the step (closePlanStepForExecution in
emitExecutionEvent) and emits plan.step.finished — so the board stays honest
even if the agent forgets to close a step it started.
Verified end-to-end: a goal-driven task fired goal.set → plan.proposed →
2× step.started/finished → task.status on the SSE stream; both steps persisted
done with start/finish timestamps; status progressed planning→executing→done.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the compounding knowledge loop the task model is built around:
- complete_task(outcome, summary): a nomos-LOCAL, session-scoped tool (the
shared MCP server has no session id). Introduces the local-tool mechanism —
buildTools appends task tools, the agent loop routes them to handleTaskTool
instead of the MCP client. Sets the task's terminal status/outcome/summary,
mirrors it onto the task entity, and emits task.status.
- Knowledge → task linkage: after a successful upsert_knowledge in a task,
nomos links the note to the task entity (documents) and emits
knowledge.recorded, so the task's outcome view shows what it learned. The
note's about-link to the involved entity (written by upsert_knowledge) is the
retrieval path future tasks use.
- SOUL: every chat is a task loop — retrieve prior knowledge FIRST
(get_entity_knowledge on the target), plan, execute, record learnings, then
complete_task. Scales down for trivial read-only tasks.
- deleteSession now cleans up the task entity, its relationships, and its
task-scoped events (was orphaning them); the knowledge doc itself and its
about-links survive, as knowledge should outlive the task.
Verified end-to-end: a task recorded a note and completed; task.status +
knowledge.recorded hit the SSE stream; status=done/outcome=success persisted;
the note linked to both lxc:caddy (retrieval) and the task; a future
get_entity_knowledge(lxc:caddy) surfaces it; delete cleaned edges+events (0/0/0)
while the knowledge survived.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
