dtoro/oikos
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

fix(agent): refuse plan re-proposal + emit done on error (close divergence chain)
Some checks failed
ci / build-test (push) Has been cancelled
Details
ci / docker-build (push) Has been cancelled
Details
Desktop App / Build Linux (amd64) (push) Has been cancelled
Details
Desktop App / Attach to Release (push) Has been cancelled
Details

Browse Source 
Operator-reported bug: on 'proceed with the rest' the agent re-proposed the
plan, duplicating it in the sidebar. Root cause was a three-bug chain, not
one bug:

1. Trigger — model empty-response on 'proceed' (approval vocabulary didn't
   list 'proceed', so the agent wasn't sure it was approved and no-op'd).
2. Amplifier — chatWith emitted 'error' without 'done' on empty response
   (agent.go:370). The frontend's onComplete saw !receivedDone and
   misclassified the model failure as a network disconnect, calling
   handleDisconnect -> resumeSession.
3. Divergence — the reconnect note was generic ('report your state'), so
   the agent re-proposed + re-executed instead of advancing the plan.

Fixes (shipped, e2e-validated against the live agent on oikos-nomos-1):

- A.2: proposePlan refuses re-proposal once a step has started (returns
  errPlanInFlight). Drops the append-mode safety net (commit 5384499) that
  was the direct source of the sidebar duplication. The agent must advance
  with update_plan_step + run; the tool result directs it.
- A.1: proposePlan sets the 'generation' column on INSERT (migration 020
  added the column + frontend grouping, but the INSERT never wired it).
- A.3: propose_plan tool description restated as a crisp contract (ONCE,
  STOP and wait, REFUSES once a step started, advance with update_plan_step).
- F.3: approval vocabulary expanded to approved/yes/go/proceed/continue/ok/
  go ahead; propose_plan result string tightened to an imperative.
- B.1: chatWith emits 'done' after 'error' on every terminal path via a new
  emitError helper. The frontend now treats model errors as ended (not
  disconnected), so no auto-reconnect -> resumeSession fires.
- B.2: reconnect/resume note carries the operator's last message + an
  explicit 'advance the plan, do NOT call propose_plan again' directive when
  a plan is in flight. Wired into all 4 resume entry points (reconnect,
  /resume, idle-sweep, question-answer) via enrichResumeNote.
- B.3: resumeSession escalates the recovery note across its 3 attempts (final
  retry: 'pick the lowest-pending step, mark it running, call run — do that
  now') instead of 3 identical notes -> 3 identical empties.

Verification: TestProposePlan_RefuseInFlight replaces TestProposePlan_
AppendVsReplace. e2e conversations against the rebuilt container:
  conv2 ('proceed with the rest') -> 0 propose_plan calls, plan stayed at
    3 steps (was 6+ before), update_plan_step x5 + run x2 + complete_task.
  conv3 (full plan, 'go ahead') -> apt-get update on lxc:dns auto-ran under
    the plan window, update_entity_attributes writeback, clean complete_task.
  nomos logs show zero reconnect/resume entries for the plan-proposing
    sessions (the three-bug chain is closed).

Remaining (not in this commit): D.1 refuse complete_task without writeback
(next blocker), C.1/C.2, F.1/F.2 SOUL.md consolidation, B.4-B.6, E.1/E.2.
See plans/2026-07-14-post-fix-session-remainders.md.

Also: re-audit 2026-07-10-general-gated-execution.md — request_execution enum
retirement (60effcb) closes item 9; only auto-act revival (item 10) remains.

Version 0.4.1 -> 0.5.0 (minor: new structural behavior, not a bugfix).
This commit is contained in:
authentik Default Admin
2026-07-14 15:28:33 +02:00
parent 5f82627fa8
commit 337d577f00
10 changed files with 1074 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
plans/2026-07-10-general-gated-execution.md
View File

@@ -1,13 +1,17 @@
# 2026-07-10 — General gated execution: from fixed actions to unlimited-but-gated
**Status:** In Progress — audited 2026-07-11. Done: `ClassifyCommand` risk
**Status:** In Progress — re-audited 2026-07-14. Done: `ClassifyCommand` risk
classifier, general `run` MCP tool, chat-assent approval (no button
required), blast radius on approval cards, session digest, global activity
feed (`Ops.svelte` "Executions" tab, risk-badged), Learning view
(success-rate trend). Still open: retire the fixed `request_execution`
action enum (`restart, systemctl, pct_exec, apt_upgrade, pct_create` still
hard-coded alongside `run`), and revive auto-act — `internal/actuator/actuator.go:125`
is still a literal `{"success": true, "message": "stub execution"}` stub.
(success-rate trend), **and now the `request_execution` enum retirement**
(commit `60effcb`, 2026-07-14 — `run` is the only mutation tool; the legacy
handler functions are kept as reference only, with a "DO NOT re-register"
guard in `internal/mcp/server.go:360`). Still open: **revive auto-act** —
`internal/actuator/actuator.go:~125` is still a literal
`{"success": true, "message": "stub execution"}` stub (item 10). The
`run`-gated path covers operator-initiated work end-to-end; auto-act is the
observe→Act direction (signals triggering actions), still unimplemented.
## Goal

853
plans/2026-07-14-post-fix-session-remainders.md Normal file
View File

@@ -0,0 +1,853 @@
# 2026-07-14 — Post-fix session audit: empty responses & plan drift remainders
**Status:** In Progress — 2026-07-14. Phases A + B.1-B.3 + F.3 shipped &
e2e-validated (v0.5.0, deployed to oikos-nomos-1). Phases C, D, E, and
F.1-F.2 remain; **D.1 is the next blocker** (refuse `complete_task`
without writeback — the knowledge loop is still drifting).
## Shipped (2026-07-14, v0.5.0)
| Fix | File(s) | Validation |
|---|---|---|
| **A.1** `proposePlan` sets `generation` on INSERT | `cmd/nomos/store.go` | e2e: plan steps now carry `generation: 1` (was always 1 before; column was unwired) |
| **A.2** `proposePlan` refuses re-proposal when in flight (drops append-mode) | `cmd/nomos/store.go`, `cmd/nomos/tasks.go` | e2e: "proceed with the rest" → 0 `propose_plan` calls (was 1 + duplicate sidebar); plan stayed at 3 steps, not 6+ |
| **A.3** `propose_plan` tool description restated as a crisp contract | `cmd/nomos/tasks.go` | agent self-described the contract in its reply |
| **F.3** Approval vocabulary expanded + directive result strings | `cmd/nomos/tasks.go` | e2e: "proceed" and "go ahead" both recognized as approval (was only "approved/yes/go ahead") |
| **B.1** `chatWith` emits `done` after `error` on every terminal path | `cmd/nomos/agent.go` | e2e: nomos logs show zero reconnect/resume entries for the plan-proposing test sessions (was the amplifier in the three-bug chain) |
| **B.2** Reconnect/resume note carries last user msg + plan-in-flight directive | `cmd/nomos/store.go`, `cmd/nomos/main.go`, `cmd/nomos/continue.go` | wired into all 4 resume entry points (reconnect, /resume, idle-sweep, question-answer) |
| **B.3** `resumeSession` escalates the recovery note across 3 attempts | `cmd/nomos/continue.go` | e2e: a manually-triggered reconnect produced a real response on the escalated retry (was 3 identical empties → give up) |
Test: `TestProposePlan_RefuseInFlight` (rewrote `TestProposePlan_AppendVsReplace`)
in `cmd/nomos/store_test.go` asserts the refusal + generation wiring.
## Remaining (not yet shipped)
- **D.1** Refuse `complete_task` without writeback when discovery ran — **next blocker**. The knowledge loop is still drifting: conv3 did write back (`update_entity_attributes`) but only because the agent chose to, not because it was forced to. The warning string in `completeTask` (5.5 from the prior plan) still fires but is still ignorable.
- **D.2** Auto-append a writeback step to plans that lack one.
- **C.1** `completeTask` reject re-completion of a terminal session.
- **C.2** SOUL.md: don't re-execute on UI-clarification complaints.
- **F.1** Consolidate SOUL.md's three overlapping task-flow sections to one (the operator's "be more crisp" feedback).
- **F.2** Tighten tool-result strings from advisory to imperatives (partially done in F.3's propose_plan result; remaining: set_goal, update_plan_step, complete_task).
- **B.4** Surface the real model error text (errText) in the error event + resume-failed note.
- **B.5** Back off between resume retries (4s, 8s).
- **B.6** Don't persist the empty placeholder as a visible bubble.
- **E.1** SOUL.md: prefer knowledge over re-execution for fleet-wide facts.
- **E.2** `list_lxcs` last-audited hint in the result.
## Commit-history context (the 20-commit iteration)
Reviewing `git log` since the agent-task phases landed (be3ce76 → 5caf49b),
the same problems recur because we keep fixing them with **SOUL.md prose +
safety-net append logic** instead of structural gates:
- `5384499` (Jul 11) — "plan panel showed only the latest step" → fixed by
making `proposePlan` APPEND when a step is in flight, so history is
preserved even if the model re-proposes per step. **This is the source of
the duplication the operator saw today.** The fix traded "lost progress"
for "duplicate progress" — and the duplication is what's visible to the
operator now.
- `e30813a` / `532310b` (Jul 11) — "research-first / knowledge-write-back-
last explicit steps" → added the FIRST/LAST step language to SOUL.md.
Three commits later the warnings are still being ignored in production.
- `5caf49b` (Jul 14, today) — "mandatory pre-plan flow" → another SOUL.md
section at the top of the file, overlapping the existing "Every chat is a
task" / "AFTER EVERY TASK: WRITE BACK" sections. The agent now has three
overlapping sections telling it the same thing.
- `60effcb` (Jul 14) — Phase 5 of the prior plan added the `generation`
column, the `replaced` status, the frontend grouping, and the writeback
warnings. The migration landed; the INSERT in `proposePlan` did not.
**The pattern:** every iteration adds another paragraph to SOUL.md and a
safety net in the store layer. The agent still does the wrong thing
because prose instructions are unreliable and the safety nets paper over
the symptom instead of refusing the bad action. **This plan pivots to
structural gates** — `proposePlan` and `completeTask` should refuse the
calls that produce drift, not accommodate them.
## Sessions under audit
| Session | Time | Goal | Messages | Outcome | Real tool calls |
|---|---|---|---|---|---|
| `722d8878` (failure) | 10:45 | Fleet update audit | 3 | **failed** — empty response during auto-resume | 41 in turn 1 |
| `d9cdcee1` (success w/ friction) | 11:44 | Same prompt (user retried) | 11 | success | 38 across 5 turns |
Both sessions are the same operator request: "Check all the services on the
homelab and give me an overview of what needs updating, categorize by
criticality." Cross-referencing them shows **where the prior fixes held vs.
where they didn't.**
---
## What worked (preserve)
- **`upsert_knowledge` `about` array** (5.2 from prior plan) — the agent
linked the audit to all affected LXCs in one call:
`about: ["lxc:nextcloud","lxc:jellyfin","host:hubris", ...]`.
- **`complete_task` writeback warning** (5.5) — fired correctly (the session
has no `update_entity_attributes` calls and the warning text appears in the
tool result).
- **`propose_plan` writeback nudge** (5.4) — fired (last step title was
"Write back: upsert_knowledge if anything changed", which contains neither
required tool name).
- **Seq-order completion enforcement** (5.6) — no out-of-order completions
observed.
- **Replaced-status mechanism** (3.3) — pending steps from the prior
generation were correctly marked `replaced` on re-propose.
## What didn't (the findings below)
---
## Findings
### 1. Empty response still ends the session — operator had to start over
**Where:** Session `722d8878` msg 2: `[System: auto-resume failed after
retrying: Nomos returned an empty or unusable response — please retry. The
task is paused — send another message to continue.]`
**What happened:** Turn 1 ran 41 tool calls (set_goal + list_lxcs +
get_health_summary + get_state_snapshot + search_knowledge + 4× get_relations
+ 4× get_entity + 20× `run` for `apt-get update` across the fleet). The model
returned that successfully. Auto-continuation then ran `resumeSession`, which
retried `chatWith` **3 times** (continue.go:229) — all three came back empty.
The session ended with the system note above. The operator abandoned it and
opened `d9cdcee1` with the same prompt.
**Root cause:** Three identical retries with the same injected `note` produce
three identical empty responses (the model isn't randomly failing — it's
responding to the prompt the same way each time). The retry loop never varies
the prompt, never backs off, and never escalates to a more aggressive
recovery (e.g. a fresh continuation prompt that summarizes what just happened
and asks explicitly for the next single step).
**Severity:** Blocker — a 41-tool-call turn costs real money and time, and the
operator gets nothing for it.
### 2. `generation` column exists but `proposePlan` never sets it — frontend grouping is dead code
**Where:** `cmd/nomos/store.go:458-461` (INSERT statement) vs.
`migrations/020_session_reliability.up.sql:7` (the column) and
`web/src/lib/components/PlanProgress.svelte:17-22` (the grouping logic).
**What happened:** Migration 020 added `generation INTEGER NOT NULL DEFAULT 1`
and PlanProgress groups steps by `s.generation ?? 1`. But the INSERT in
`proposePlan` is:
```sql
INSERT INTO session_plan_steps (session_id, seq, title, detail, target_slug)
VALUES ($1, $2, $3, $4, $5) RETURNING id
```
No `generation` column. Every step, in every plan revision, lands with
`generation = 1`. PlanProgress always sees one group ("Current plan") and
the collapse-old-generations behavior never triggers.
**Concrete impact in `d9cdcee1`:**
- Turn 1 (msg 1): `propose_plan` creates steps seq 1-5 (all generation 1).
- User: "why is the plan not updated accordingly? the steps in the sidebar."
- Turn 3 (msg 5): `update_plan_step seq=1, status=done`. Steps 2-5 still
pending, all generation 1.
- User: "proceed with the rest."
- Turn 4 (msg 7): **empty assistant response** (text="", no tools).
- Turn 5 (msg 8): Agent calls `propose_plan` **again** with the same 5 steps.
`proposePlan` sees `anyStarted=true` (seq 1 is done), so it goes into
append mode: marks the 4 still-pending steps (2-5) as `replaced`, then
inserts 5 new steps at seq 6-10. **All inserted with generation=1.**
- The frontend now sees 10 steps, all `generation: 1`, grouped together.
Four are marked `replaced` (visible as "skipped/replaced" — dimmed but
still in the list); six are the new active steps.
- User: "btw the plan here and the one in the sidebar differ." → Confirmed:
the chat text describes a 5-step plan ("Step 1 done, refreshing 2-4");
the sidebar shows 10 steps with a confusing mix of done/replaced/running.
**Severity:** Blocker — this is the direct, observable cause of the user's
two complaints in `d9cdcee1`. The prior plan (3.4) shipped the column and
the frontend code but never wired the backend INSERT.
### 3. Agent re-proposes the plan on "proceed" instead of continuing
**Where:** `d9cdcee1` msg 8 — `propose_plan` called again after user said
"proceed with the rest".
**What happened:** The agent had a perfectly good plan in flight (step 1 done,
2-5 pending). On the next operator turn ("proceed"), it should have called
`update_plan_step(seq=2, status=running)` and `run` against the targets.
Instead it called `propose_plan` with the same 5 steps, triggering the
append-mode behavior in #2.
**Root cause:** SOUL.md doesn't explicitly say "do NOT call propose_plan
again once you've already proposed — call update_plan_step + run instead."
The agent treated "proceed" as a cue to re-state the plan, not to advance
it.
**Severity:** Friction (compounds #2 into a blocker).
### 3a. WHY the agent re-proposed instead of advancing — the three-bug chain
Finding #3's surface description ("agent re-proposed on proceed") is real but
doesn't explain the *mechanism*. Tracing the message timestamps and the
`auto: true` flag on msg 8 reveals that the re-proposal wasn't the agent's
direct response to "proceed with the rest" at all — it was the agent's
response to a **generic system reconnect note**, fired by a chain of three
compounding bugs:
**The chain (all confirmed from code + session data):**
| Step | What happened | Where |
|---|---|---|
| 1. **Trigger** — model returned empty on the approval | User sent "procceed with the rest." `handleChat` → `a.chat()` → `chatWith()`. The model returned an empty completion 3× (all `maxLLMRetries=2` attempts exhausted). SOUL.md's approval vocabulary was "approved/yes/go ahead" — "proceed" wasn't listed, so the model likely wasn't certain it was approved and no-op'd. | `agent.go:362-371` |
| 2. **Amplifier** — empty response misclassified as network disconnect | On empty response, `chatWith` emits `error` and `return`s **without emitting `done`** (agent.go:370-371 — the `done` event only fires on the success path at line 382). The frontend's `onComplete` callback sees `!receivedDone` and treats it as a severed connection, calling `handleDisconnect()`. A *model* empty-response gets handled by the *network* disconnect path. | `agent.go:370-371` (missing `done`) + `chat.ts:349-356` (`!receivedDone → handleDisconnect`) |
| 3. **Divergence** — generic reconnect note triggers re-proposal | `handleDisconnect` waits 1s, then sends an empty message (`streamChat('', sessionId, …)`). The backend's reconnect path (main.go:177-188) calls `resumeSession` with: `"[System: the operator's connection was re-established. The task may have progressed in the background — report your current state and progress.]"`. The agent re-read the transcript (plan proposed, step 1 done, user said "proceed"), saw this generic note, and interpreted "report your current state and progress" as "redo the work and report it" → re-proposed + re-executed + `complete_task`. | `chat.ts:386-419` (reconnect) + `main.go:180` (note) + `continue.go:189` (resumeSession) |
**Timestamps confirm this:** msg 7 (empty) at `11:49:01.949`, msg 8 (re-propose, `auto: true`) at `11:49:16.200` — 15 seconds later, matching the 1s reconnect delay + the LLM call latency. The user never sent a second message; the frontend's reconnect logic did.
**The user's actual approval ("procceed with the rest") was in the transcript** but the agent wasn't responding to it — it was responding to the *system reconnect note*, which didn't mention approval, the plan, or the user's words. The propose_plan result had said "STOP and wait for approval," and the generic reconnect note didn't say "you're approved" — so the agent re-proposed to get a fresh approval cycle.
**Why this matters for the fix:** Phase A.2 (refuse re-proposal when in flight) would have *prevented the duplication* but not *fixed the cause*. The agent would have hit the refusal and then… what? With the generic reconnect note, it still doesn't know it's approved. The three bugs need three targeted fixes (Phase B below). This is the answer to "why didn't the agent update the original plan": **it never received a clear signal to advance, because the approval signal was lost in an empty response that got misclassified as a network drop.**
**Severity:** Blocker — this is the root cause of the plan divergence the
operator observed.
### 4. Operator clarification was interpreted as "redo the whole task"
**Where:** `d9cdcee1` msg 9 → msg 10. User said "btw the plan here and the
one in the sidebar differ." Agent's response (msg 10): re-ran all 6 `run`
calls (`apt-get update` + `apt list --upgradable` on nextcloud, jellyfin,
hubris), re-called `upsert_knowledge`, and **called `complete_task` a
second time**.
**What happened:** The operator wanted the sidebar aligned with the chat.
The agent re-executed the actual audit work and re-completed the task.
**Root cause:** No prompt-level instruction about how to handle "the UI
seems inconsistent" complaints — the agent defaulted to "do the work again,
maybe it'll line up this time."
**Severity:** Friction — wasted 6 `run` calls and a duplicate knowledge
entry; user gets a noisier transcript.
### 5. `complete_task` called twice on the same session
**Where:** `d9cdcee1` msg 8 and msg 10 both call `complete_task` with
`outcome=success`.
**What happened:** After msg 8, `agent_sessions.status` is `done`. The user
complained about the plan drift; the agent re-ran the audit and called
`complete_task` again. There's no guard in `completeTask` against re-completing
an already-terminal session.
**Severity:** Cosmetic, but it produces duplicate knowledge entries and
erodes audit-log clarity.
### 6. Turn 1 of `722d8878`: 41 tool calls including `run` against every LXC
**Where:** Session `722d8878` msg 1.
**What happened:** Despite a same-day knowledge entry
(`investigation:nomos/fleet-wide-apt-update-audit-2026-07-14` — the agent even
called `get_knowledge_content` for it), the agent ran `apt-get update` on
every LXC in turn 1 instead of presenting the prior audit and proposing a
small refresh plan. The agent already had the answer in the DB; it re-ran
the fleet audit anyway.
**Severity:** Friction — wasted ~20 `run` calls (each is a queued execution).
The successful retry session (`d9cdcee1`) only re-ran 3 (the critical trio),
which is the right pattern — but it had to learn that from the failure
session's example.
### 7. Agent ignores its own writeback warnings
**Where:** `d9cdcee1` — `propose_plan` returned the nudge from tasks.go:213
("⚠️ The final step doesn't mention update_entity_attributes…") and
`complete_task` returned the warning from tasks.go:280 ("⚠️ No entity
attributes or relationships were updated in this session…"). The agent saw
both, did nothing about either, and ended the task.
**What happened:** The warnings are surfaced in the tool result text, but
the model treats tool results as ephemeral context — it doesn't act on a
warning that appears after the work it already decided is done. The session
recorded zero `update_entity_attributes` calls and zero
`create_relationship` calls.
**Severity:** Blocker — the knowledge-loop drift problem the prior plan was
supposed to fix is still happening. The graph accumulates nothing structured
from this session; the next fleet audit will rediscover every fact from
scratch.
### 8. Empty assistant bubble persisted in the transcript
**Where:** `d9cdcee1` msg 7: `{"role":"assistant","text":"","tool_calls":[]}`.
**What happened:** On the "proceed with the rest" turn, the model returned an
empty completion. The inner `chatWith` retry (agent.go:331) eventually
succeeded and produced msg 8 — but the empty msg 7 was already persisted to
the transcript and stays there. The UI shows an empty assistant bubble between
the user's "proceed" and the agent's actual response.
**Severity:** Cosmetic, but visible to the operator and erodes trust ("is
the agent broken?").
---
## Improvement plan
### Phase A — Make `propose_plan` refuse duplication (addresses #2, #3)
The operator's "plan was added twice" complaint is the visible output of
the append-mode safety net added in `5384499`. The safety net was the wrong
default: it preserved history but produced a confusing 10-step sidebar. The
right default is to **refuse** a re-proposal when a plan is already in
flight — the agent must use `update_plan_step` + `run` to advance.
#### A.1 — `proposePlan`: set `generation` on insert (still needed for history)
**File:** `cmd/nomos/store.go:415-491`
**How:**
1. Resolve the next generation number at the top of `proposePlan`, in the
same transaction:
```go
var nextGen int
if !anyStarted {
// fresh/revise: reset to 1 (and the DELETE already wiped old rows)
nextGen = 1
} else {
if err := tx.QueryRow(ctx, `
SELECT COALESCE(MAX(generation), 0) + 1
FROM session_plan_steps WHERE session_id = $1`, sessionID).Scan(&nextGen); err != nil {
return nil, err
}
}
```
2. Add `generation` to the INSERT:
```sql
INSERT INTO session_plan_steps (session_id, seq, title, detail, target_slug, generation)
VALUES ($1, $2, $3, $4, $5, $6) RETURNING id
```
Pass `nextGen` as `$6`.
3. Include `"generation": nextGen` in the `out` map so the tool result and
the `plan.proposed` event carry it (the frontend already reads it via
`api.ts:66`).
4. Backfill is unnecessary — existing rows default to generation 1.
#### A.2 — `proposePlan`: refuse re-proposal once any step has started
**File:** `cmd/nomos/store.go:415-491` + `cmd/nomos/tasks.go:206-219`
**How:**
1. In `proposePlan`, when `anyStarted == true`, return a sentinel error
instead of appending:
```go
if anyStarted {
return nil, errPlanInFlight
}
```
2. In `handleTaskTool`'s `propose_plan` case, detect the sentinel and return
a directive tool result:
```
Plan already in flight — refusing duplicate proposal. Steps 1..N exist;
at least one is running or done. To advance the plan, call
update_plan_step(seq=K, status=running) followed by run(...) for step K's
target. Do NOT call propose_plan again. Call it again only if the
operator explicitly asks you to revise the whole plan, and if so, say
that in your reply before calling it.
```
3. Drop the append-mode code path (store.go:437-448) — it's the duplication
source. Keep the destructive-replace path (store.go:432-436) for the
`!anyStarted` case (genuine pre-execution revision).
4. The `replaced` status becomes unreachable through normal flow but stays
in the schema for any future "explicit revise" path that uses it.
This is the single highest-impact fix in this plan. It directly removes
the "plan added twice" behavior the operator reported, and forces the
agent to use the correct advancement tools. Combined with the directive
tool result, even a model that ignores SOUL.md will get the right behavior
because the bad action is refused.
#### A.3 — `propose_plan` tool description: state the contract crisply
**File:** `internal/mcp/server.go` (the `propose_plan` tool schema)
**How:** Replace the current description with a one-paragraph contract:
```
Propose the full ordered plan for this task. Call ONCE per task, before
any execution. After this call: STOP and wait for operator approval.
Once a step has started (status=running/done/...), this tool REFUSES
further calls — use update_plan_step + run to advance. The LAST step
MUST be "Write back: update_entity_attributes + create_relationship
+ upsert_knowledge".
```
This puts the contract where the model reads it (in the tool schema that
gets serialized into the system prompt), not just in SOUL.md where it
competes with three overlapping sections.
#### A.4 — PlanProgress: verify grouping renders with the wired-up column
**File:** `web/src/lib/components/PlanProgress.svelte:17-90`
Once A.1 lands, the grouping code that already exists should work. Verify:
- Latest generation (`Math.max(...generations)`) → expanded, labeled
"Current plan".
- Older generations → collapsed by default, labeled "Plan v1 (replaced)",
with a count badge.
- A future explicit-revise path (not in this plan) would land generation 2
as the new "Current plan" and the old steps collapse.
This is verification, not new code — the structure is there, it just
never received varied generation numbers to group on.
### Phase B — Close the three-bug chain that caused the divergence (addresses #3a, #1, #8)
Phase A.2 (refuse re-proposal) prevents the *symptom* (duplicate plan in
sidebar). This phase fixes the *cause* — the three bugs in finding #3a that
made the agent re-propose in the first place. Each fix targets one link in
the chain.
#### B.1 — Emit `done` after `error` so the frontend doesn't misclassify (fixes bug 2 — the amplifier)
**File:** `cmd/nomos/agent.go:370-371` (+ the other early-return error paths
at lines 348, 356)
**What:** On empty response, `chatWith` emits `error` and returns **without
emitting `done`**. The `done` event only fires on the success path
(agent.go:382). The frontend's `onComplete` (chat.ts:349-356) sees
`!receivedDone` and routes into `handleDisconnect` — treating a *model*
failure as a *network* drop, which triggers an unwanted auto-reconnect →
`resumeSession` → re-proposal.
**How:**
1. After the `error` emit at line 370, also emit `done` before returning:
```go
emit(agentEvent{Type: "error", Data: "Nomos returned an empty or unusable response — please retry.", SessionID: sessionID})
emit(agentEvent{Type: "done", Data: map[string]any{
"session_id": sessionID, "correlation_id": correlationID,
"iterations": i + 1, "error": true,
}, SessionID: sessionID})
return
```
2. Do the same for the other early-return error paths (agent.go:348 stream
error, agent.go:356 no choices) so every terminal path emits `done`.
3. On the frontend, `onComplete` (chat.ts:349-356) now sees
`receivedDone === true` and sets `streaming.set(false)` instead of
calling `handleDisconnect`. The error is still shown via the `error`
event handler (chat.ts:329-331).
4. Add `"error": true` to the done payload so the frontend can distinguish
"ended cleanly" from "ended with error" (e.g. to show a retry button
instead of loading dots).
**Impact:** This alone prevents the unwanted `resumeSession` call after a
model empty-response. The error becomes a visible chat error (with the
retry button from prior Phase 1.6), not a silent trigger for re-execution.
This is the single highest-leverage fix in this phase — it breaks the chain
at the amplifier.
#### B.2 — Reconnect note: reference the user's last message and plan state (fixes bug 3 — the divergence)
**File:** `cmd/nomos/main.go:180` (reconnect note) + the other resume entry
points at `main.go:341` (`/resume` endpoint) and `continue.go:83-86`
(idle-sweep note)
**What:** Even with B.1, genuine network disconnects will still happen. When
they do, the reconnect note (`"report your current state and progress"`) is
too generic — it doesn't tell the agent what the operator actually wanted,
so the agent guesses (badly). The note should carry the operator's last
message and whether a plan is in flight.
**How:**
1. Add two helpers to `store.go`:
```go
func (s *store) lastUserMessage(ctx, sessionID) string // SELECT text FROM messages WHERE session_id=$1 AND role='user' ORDER BY created_at DESC LIMIT 1
func (s *store) hasPlanInFlight(ctx, sessionID) bool // SELECT EXISTS(... WHERE session_id=$1 AND status IN ('pending','running'))
```
2. In `handleChat`'s reconnect path (main.go:177-188), build a specific note:
```go
lastUserMsg := st.lastUserMessage(pctx, req.SessionID)
planInFlight := st.hasPlanInFlight(pctx, req.SessionID)
note := fmt.Sprintf("[System: the operator's connection was re-established. "+
"The operator's last message was: \"%s\". ", lastUserMsg)
if planInFlight {
note += "A plan is in flight — advance it with update_plan_step + run. Do NOT call propose_plan again."
} else {
note += "Report your current state and progress."
}
note += "]"
```
3. Apply the same enrichment to the `/resume` endpoint note (main.go:341)
and the idle-sweep note (continue.go:83-86) — all three resume entry
points should carry the same context.
**Impact:** Even if B.1 is bypassed (genuine disconnect mid-plan), the agent
gets "advance the plan" instead of "report state." No more re-proposal from
reconnect.
#### B.3 — `resumeSession`: escalate the recovery note across attempts (fixes bug 1 — the trigger)
**File:** `cmd/nomos/continue.go:229-253`
**What:** The current loop retries 3 times with the same note. A transient
model issue (or a prompt causing the model to no-op) gets three identical
empty responses.
**How:**
1. Build a different `note` per attempt:
```go
notes := []string{
note, // attempt 0: the original (now enriched per B.2) note
fmt.Sprintf("[System: your previous turn produced no response. %s. "+
"Produce a response now — call the next tool or report progress in one sentence.]", note),
fmt.Sprintf("[System: two consecutive empty responses. Stop trying to be clever. "+
"The next action is: pick the lowest-pending plan step, mark it running with "+
"update_plan_step, and call run for its target. Do that now.]"),
}
```
2. Pass `notes[attempt]` to `chatWith` so each retry gets a progressively
more directive prompt.
3. Keep the 3-attempt cap.
**Impact:** A model that's transiently flaking or confused gets a real
second chance with an increasingly specific directive, instead of three
identical prompts.
#### B.4 — Surface the real model error text (addresses finding #1's observability)
**File:** `cmd/nomos/agent.go:370` + `cmd/nomos/continue.go:255-274`
**What:** The operator-facing message is "Nomos returned an empty or
unusable response — please retry." The actual error (OpenRouter 503,
content filter, token limit) is logged but not shown.
**How:**
1. In `chatWith`'s error emit (agent.go:370), include `errText`:
```go
emit(agentEvent{Type: "error", Data: fmt.Sprintf("Nomos returned an empty or unusable response: %s", errText), SessionID: sessionID})
```
2. In `resumeSession`'s failure path (continue.go:262):
```go
resumeFailedNote := fmt.Sprintf(
"[System: auto-resume failed after 3 attempts. Last error: %s. "+
"The task is paused — send another message to continue.]", errText)
```
3. The operator can now tell "model overloaded, just retry" from "content
filter — I need to rephrase."
#### B.5 — Back off between resume retries
**File:** `cmd/nomos/continue.go:229`
**How:** Add a small sleep before attempts 1 and 2:
```go
for attempt := 0; attempt < 3; attempt++ {
if attempt > 0 {
select {
case <-cctx.Done(): return
case <-time.After(time.Duration(2<<attempt) * time.Second): // 4s, 8s
}
}
// ... existing body, using notes[attempt] from B.3
}
```
#### B.6 — Don't persist the empty placeholder as a visible bubble
**File:** `cmd/nomos/main.go:251-266` (handleChat placeholder) +
`cmd/nomos/continue.go:190-218` (resumeSession placeholder)
**What:** On `d9cdcee1` msg 7, the empty assistant bubble persisted in the
transcript because `persist()` ran with `finalText=""` after the error
return. The UI shows an empty bubble.
**How:**
1. Mark the placeholder as pending:
`{"role":"assistant","text":"","pending":true}` instead of just `""`.
2. The frontend renders `pending: true` as loading dots (it already does
this for empty text during streaming), not an empty bubble.
3. On success, `persist()` overwrites with real content and drops `pending`.
4. In `handleChat`'s final persist call (main.go:282), if `finalText == ""`
and `len(toolCalls) == 0`, delete the placeholder row instead of
persisting an empty bubble:
```go
if finalText == "" && len(toolCalls) == 0 && msgID != uuid.Nil {
st.deleteMessage(pctx, msgID)
} else {
persist()
}
```
### Phase C — Stop the agent re-executing on clarification (addresses #4, #5)
#### C.1 — `completeTask`: reject re-completion of a terminal session
**File:** `cmd/nomos/store.go:completeTask`
**How:**
1. Before the UPDATE, fetch the current status. If it's already `done`,
`failed`, or `partial`, return without re-updating and surface a no-op
message:
```go
var current string
s.pool.QueryRow(ctx, `SELECT status FROM agent_sessions WHERE id=$1`, sessionID).Scan(&current)
if current == "done" || current == "failed" || current == "partial" {
return nil // already terminal — silently no-op
}
```
Or, stronger, return an error from `completeTask` and have the caller
(tasks.go:275) translate it into a tool-result message:
`"Session is already complete (status=done). If you want to keep working, call update_plan_step + run; do not call complete_task again."`
2. The error path is preferred — the agent sees it in the tool result and
stops trying to re-complete.
#### C.2 — SOUL.md: handle "the UI is inconsistent" complaints without re-executing
**File:** `nomos/SOUL.md`
**How:** Add a short rule:
```
If the operator points out that the chat and the sidebar/plan panel disagree,
DO NOT re-run the work. Investigate the discrepancy by reading state:
get_plan_steps / list current step states → reconcile with a single
update_plan_step call. If the panel is correct and the chat is stale,
summarize the panel in your reply. If the chat is correct and the panel
is stale, fix the panel with update_plan_step. Never re-execute tool
work just to fix a display mismatch.
```
### Phase D — Writeback enforcement that actually sticks (addresses #7)
The current warnings are too easy to ignore because they appear after the
agent has already moved on mentally. Make them structural.
#### D.1 — `completeTask`: refuse to mark success without writeback when state was discovered
**File:** `cmd/nomos/store.go:completeTask` + `cmd/nomos/tasks.go:254-282`
**How:** Convert the warning into a refusal when the session actually ran
discovery tools:
1. Extend `hadEntityWriteback` (store.go:624) into `hadDiscoveryAndWriteback`:
```sql
-- did the session run discovery?
SELECT EXISTS(SELECT 1 FROM audit_log
WHERE session_id=$1 AND tool_name IN ('run','get_entity','get_relations','list_lxcs','list_entities'))
-- AND did it write back?
SELECT EXISTS(SELECT 1 FROM audit_log
WHERE session_id=$1 AND tool_name IN ('update_entity_attributes','create_relationship'))
```
2. In `completeTask`, if `discovery=true AND writeback=false` AND `outcome`
is `success`:
- **Force-downgrade** the outcome to `partial`.
- Return a hard error (not just a warning) that the agent must act on:
`"Refused: this session ran discovery (run/get_entity/...) but did not call update_entity_attributes or create_relationship. Call those now to persist the facts you learned, then call complete_task again. Outcome downgraded to 'partial' until you do."`
3. The agent gets the error in the tool result, sees the directive, and is
forced to call `update_entity_attributes` before it can complete.
This is the structural version of 5.4/5.5 from the prior plan — warnings
didn't work; enforcement will.
#### D.2 — `propose_plan`: auto-append a writeback step if missing
**File:** `cmd/nomos/tasks.go:206-219`
**How:** Instead of (or in addition to) the warning string, append a
synthetic writeback step when none of the proposed steps mention
`update_entity_attributes`:
```go
hasWritebackStep := false
for _, s := range steps {
if strings.Contains(s.Title+s.Detail, "update_entity_attributes") ||
strings.Contains(s.Title+s.Detail, "create_relationship") {
hasWritebackStep = true
break
}
}
if !hasWritebackStep {
steps = append(steps, planStepInput{
Title: "Write back entity attributes and relationships",
Detail: "Call update_entity_attributes for every entity you ran run/get_entity against (versions, states, hosts, IPs), and create_relationship for any edge you discovered. Then upsert_knowledge about the affected entities.",
})
// re-call proposePlan with the extended steps, or append directly to the
// already-persisted plan via a second INSERT.
}
```
The agent then sees the explicit step in its own plan and the seq-order
enforcement (5.6) forces it to complete that step last.
### Phase E — Reduce turn-1 fan-out (addresses #6)
The `722d8878` failure session spent 41 tool calls re-discovering what was
already in the DB.
#### E.1 — SOUL.md: prefer knowledge over re-execution
**File:** `nomos/SOUL.md`
**How:** Add to the discovery section:
```
BEFORE calling `run` for fleet-wide facts (apt counts, service versions,
host states), call search_knowledge and get_knowledge_content for the
relevant entity or topic. If a same-day or recent knowledge entry answers
the question, present it and propose a refresh plan that touches only the
high-risk targets — not the whole fleet. Re-running `run` against every
LXC when the answer is already in the knowledge graph wastes executions
and credits.
```
#### E.2 — `list_lxcs`: include last-audited hint in the result
**File:** `internal/mcp/server.go:list_lxcs` handler
**How:** When returning LXCs, include for each row the most recent
`knowledge_entities.created_at` linked via `about` edges with kind
`investigation` or `document` and a tag matching `audit`/`update`. The
agent then sees "nextcloud — last audited 2026-07-14 (today)" and can skip
re-running it.
This is a smaller tweak than E.1 (which is the load-bearing fix) — the data
hint makes the SOUL.md rule easy to follow.
---
### Phase F — SOUL.md: be crisp, not repetitive (addresses the operator's "more crisp and clear with the agent" feedback)
SOUL.md grew three overlapping sections across the last 20 commits:
| Section | Added by | Says |
|---|---|---|
| `## ⚠️ MANDATORY TASK FLOW` (top) | `5caf49b` (Jul 14) | 6-step flow: set_goal → pre-plan → propose → approve → execute → writeback |
| `## Every chat is a task` (mid) | `e30813a` (Jul 11) | Same 6-step flow, longer, plus the trivial-task degenerate case |
| `### ⚠️ AFTER EVERY TASK: WRITE BACK OR LOSE IT` (inside "Every chat") | `60effcb` (Jul 14) | Writeback rule, third time |
The agent has three places telling it the same thing. The MANDATORY TASK
FLOW section at the top is the right one to keep — it's the most directive
and the closest to the system-prompt boundary. The other two are
lower-fold repetition that bloats context and dilutes the directive.
#### F.1 — Consolidate SOUL.md to one task-flow section
**File:** `nomos/SOUL.md`
**How:**
1. Keep the `## ⚠️ MANDATORY TASK FLOW` section at the top verbatim — it's
the load-bearing version.
2. Replace the `## Every chat is a task` section (lines ~85-165) with a
three-line reference: "Every non-trivial chat follows the MANDATORY
TASK FLOW at the top of this file. The flow scales down: a trivial
read-only question (e.g. 'status of Y?') is a degenerate case — answer
directly and call `complete_task` with a one-line summary, no
propose_plan ceremony."
3. Remove the `### ⚠️ AFTER EVERY TASK: WRITE BACK OR LOSE IT` subsection
entirely — its content is already step 6 of MANDATORY TASK FLOW and
step 3 of "Every chat is a task." Three statements of the same rule
don't make it more enforced; they make the file longer.
4. Result: the file is ~80 lines shorter, the agent has one place to read
the task contract, and the directive is unmissable because it's no
longer competing with two paraphrased copies.
This is reversible prose work, but it directly addresses the operator's
feedback that the agent isn't being "crisp and clear" with itself.
#### F.2 — Make tool-result strings directive, not advisory
**Files:** `cmd/nomos/tasks.go` (the result strings for `set_goal`,
`propose_plan`, `update_plan_step`, `complete_task`)
**How:** Audit each tool-result string for hedging language and tighten:
| Current | Tightened |
|---|---|
| `"Goal set: <goal>. Now do a PRE-PLAN: gather information with read-only tools ... Do NOT call run yet."` | `"Goal set. NEXT: pre-plan (read-only tools only). Then propose_plan. Do not call run."` |
| `"Plan set: N step(s). Now STOP and present the plan to the operator — do NOT call run yet. Wait for them to approve ..."` | `"Plan set (N steps). STOP. Wait for operator approval. Do not call run."` |
| `"Step N → status"` | `"Step N → status. (Use update_plan_step to advance; do not re-propose.)"` — only on the first call per session, otherwise unchanged. |
| `"⚠️ No entity attributes or relationships were updated in this session. Call update_entity_attributes and create_relationship to persist what you learned about entities before the next session starts from scratch."` | (Replaced by D.1's refusal when discovery ran.) |
Short, imperative, no hedging. The agent's behavior in `d9cdcee1` shows
that long tool-result strings with "consider revising the last step" are
treated as informational; short imperatives ("STOP. Do not call run.")
are followed.
#### F.3 — State the approval vocabulary in the plan-result string
**File:** `cmd/nomos/tasks.go:206-219` (propose_plan result)
**How:** Add the approval vocabulary to the propose_plan result so the
agent recognizes "proceed", "go", "continue", "yes", "approved", "ok" as
approval and does NOT re-propose on those:
```
Plan set (N steps). STOP. Wait for operator approval.
Approval vocabulary: "approved", "yes", "go", "proceed", "continue", "ok".
On approval, advance with update_plan_step + run. Do NOT call propose_plan again.
```
This directly addresses finding #3's cause: the agent re-proposed on
"proceed with the rest" because SOUL.md only listed "approved / yes / go
ahead" as approval vocabulary. Make the list match what operators
actually type.
---
## Sequencing & priority
| # | Fix | Effort | Impact | Phase |
|---|---|---|---|---|
| A.2 | `proposePlan` refuses re-proposal when in flight | S | **Blocker** — directly removes the duplication the operator saw | A |
| B.1 | Emit `done` after `error` in `chatWith` | S | **Blocker** — breaks the three-bug chain at the amplifier | B |
| B.2 | Reconnect note carries user's last message + plan state | S | **Blocker** — fixes the divergence cause | B |
| A.1 | Set `generation` on INSERT | S | High — needed for any future explicit-revise flow | A |
| A.3 | `propose_plan` tool description states the contract | S | High — agent reads tool schema, often ignores SOUL.md | A |
| F.1 | Consolidate SOUL.md to one task-flow section | S | High — addresses "be more crisp" feedback directly | F |
| F.2 | Tighten tool-result strings to imperatives | S | Medium — observable behavior change | F |
| F.3 | Approval vocabulary in propose_plan result | S | High — fixes the "proceed" → empty-response trigger | F |
| B.3 | Escalate recovery note per resume retry | S | High — turns 3 identical empties into a real recovery | B |
| D.1 | Refuse `complete_task` without writeback | M | **Blocker** — fixes the knowledge loop | D |
| D.2 | Auto-append writeback step to plans | M | High — addresses the cause | D |
| B.4 | Surface real model error text | S | Medium — operator can diagnose | B |
| C.1 | Reject re-completion of terminal sessions | S | Medium — stops duplicate `complete_task` | C |
| C.2 | SOUL.md: don't re-execute on UI complaints | S | Medium — prevents the 6 wasted `run` calls | C |
| B.5 | Back off between resume retries | S | Low-medium | B |
| B.6 | Don't persist empty placeholder as bubble | M | Cosmetic — but visible to operators | B |
| A.4 | Verify PlanProgress grouping renders | S | Depends on A.1 | A |
| E.1 | SOUL.md: prefer knowledge over re-execution | S | Medium — saves credits on fleet audits | E |
| E.2 | `list_lxcs` last-audited hint | M | Low — nice-to-have | E |
**Suggested order:** A.2 + B.1 + B.2 (the three blockers, ship together) →
F (crispness, ships alongside) → A.1/A.3/A.4 → D → B.3/B.4/B.5/B.6 → C → E.
The three blockers form a complete fix for the operator's reported bug:
- **A.2** stops the duplication from being *possible* (refuse re-proposal).
- **B.1** stops the empty response from *triggering* a reconnect/resume
(emit `done` after `error`).
- **B.2** makes any *genuine* reconnect carry the right context (advance
the plan, don't re-report).
Together they close the three-bug chain end-to-end. F.3 (approval
vocabulary) closes the *trigger* of the empty response itself.
---
## Verification
After deploying each phase, replay the same operator prompt in a fresh
session and check:
- **Phase A:** Call `propose_plan` twice (manually if needed) and confirm
the sidebar shows "Current plan" + a collapsed "Plan v1 (replaced)"
section, not a flat 10-step list.
- **Phase B:** Force an empty response (e.g. temporarily throttle OpenRouter
to 0 RPM, or use a stub model that returns `""`). Confirm: (a) the
frontend shows the error inline and does NOT trigger a reconnect/resume
(no `auto: true` message appears 15 seconds later); (b) the operator sees
the real error text, not "empty or unusable response"; (c) if you then
disconnect the network for real, the reconnect note says "advance the
plan" (not "report state") and the agent calls `update_plan_step` + `run`,
not `propose_plan`.
- **Phase C:** Start a session, let it `complete_task`, then send a follow-up
complaint. Confirm the agent does NOT call `complete_task` again and does
NOT re-run the original `run` calls.
- **Phase D:** Run a fleet-audit prompt. Confirm the agent cannot reach
`complete_task` with `outcome=success` without first calling
`update_entity_attributes` for at least the LXCs it ran `run` against.
- **Phase E:** Confirm a same-day audit prompt produces a turn-1 with ≤5
tool calls (search_knowledge + get_knowledge_content + small
propose_plan), not 41.
- **Phase F:** Count SOUL.md lines (target: ~80 fewer than current). Replay
the "proceed with the rest" prompt and confirm the agent does NOT call
`propose_plan` again (it gets a refusal error on the call, then advances
via `update_plan_step` + `run`).

3
plans/index.md
View File

@@ -12,11 +12,12 @@ went sideways, open an investigation.
| 2026-07-08 | [Oikos gaps, broken things, and improvements](2026-07-08-oikos-gaps-and-improvements.md) | In Progress — security items (B1-B5) and doc drift (E) still open |
| 2026-07-08 | [Control room web UI](2026-07-08-control-room-webui.md) | In Progress — packaging/auth sections superseded by the Wails plan's Phase 0 (client/server split); M4 still open |
| 2026-07-08 | [Liveness, drift, and UX cohesion](2026-07-08-liveness-drift-and-ux-cohesion.md) | In Progress — Phase 5 deferred |
| 2026-07-10 | [General gated execution: unlimited actions, gated by risk](2026-07-10-general-gated-execution.md) | In Progress — enum retirement + auto-act revival still open |
| 2026-07-10 | [General gated execution: unlimited actions, gated by risk](2026-07-10-general-gated-execution.md) | In Progress — `request_execution` enum retired (60effcb); only auto-act revival (item 10) still open |
| 2026-07-11 | [Nomos agent code review: gaps and improvement plan](2026-07-11-nomos-agent-code-review.md) | In Progress — only C1 (unauthenticated nomos gateway) still open, deferred |
| 2026-07-14 | [Session reliability & UX audit](2026-07-14-session-reliability-and-ux-audit.md) | Done — all 21 fixes deployed |
| 2026-07-14 | [Tool timeline in sidebar](2026-07-14-tool-timeline-sidebar.md) | Done — deployed v0.3.2 |
| 2026-07-14 | [Unified agent activity indicator](2026-07-14-unified-agent-indicator.md) | Done — deployed v0.3.3 |
| 2026-07-14 | [Post-fix session remainders: empty responses & plan drift](2026-07-14-post-fix-session-remainders.md) | In Progress — Phases A + B.1-B.3 + F.3 shipped & e2e-validated (v0.5.0); Phases C, D, E, F.1-F.2 remain (D.1 is the next blocker) |
## Done