oikos/plans/done/2026-08-03-nomos-chat-working-visibility.md

2026-08-03 — Nomos chat: working-visibility, message queue, generation-aware timeline

Status: Implemented (F1–F4) in v0.17.0. See Resolution at the end.

Context (grounded in last-session logs + DB, not just code)

Operator report: "On the chat window I can't tell the agent is working; it's making tool calls but no feedback. Typing returns 'Nomos is still finishing a previous step…'. Activity not up to date. Several plans at once, some don't execute."

Verified against runtime state:

• Last session 23da10db ran ONE live turn for 6m33s (21 iterations, 19:45:36→19:52:03, correlation 679566cb). At 19:48:00 the operator typed status; at 19:48:05 the turn gate deferred it (turn already active, deferring operator message). The operator could type at all only because the client had already lost the stream (streaming=false) while the server kept running — i.e. the client showed an idle window over a working turn. It ended awaiting_input.
• Turn runtimes are long: sessions in the DB run 15-27 min (e.g. 44df8802 24:24, 4319b9f8 27:05). handleChat (main.go:170) has no SSE keepalive; inter-iteration gaps reach 20-40s, so a proxy/browser idle close mid-turn resets streaming while the turn continues on context.Background() (pctx).
• Re-proposing is real: 44df8802 has generation 1 (5 steps, all replaced) → generation 2 (25 steps, done), with 2 propose_plan + 52 update_plan_step calls persisted. The activity timeline renders every one of those across both generations.

Root causes

• G1 — "working" == streaming. Every working-indication in the chat window (AgentTrace running status, indicator headline, stream cursor, panel spinner, disabled={streaming} input) is gated on the live SSE flag. A background turn (resumeSession/continuation worker) has no stream; a desynced long live turn has a dead stream. In both cases streaming=false while the server is actively working. The session status (planning/executing/awaiting_input) is the reliable "server is running a turn" signal and is already live-refreshed (workspace.ts taskFor, STATUS_AFFECTING), but the chat UI never uses it.
• G2 — busy-turn message is rejected, not queued. main.go:292-302: the turn gate waits 5s then emits the "still finishing a previous step" error and returns. The user message is persisted (main.go:270) but is inert — the user must manually re-send.
• G3 — activity is poll/event laggy. Tool-level activity derives from messages, refreshed only by the 3s poller; plan steps are events-only (workspace.ts hydrateSession) with no poll, so a missed plan.proposed event leaves the panel stuck on a stale generation.
• G4 — timeline is generation-unaware. activity.ts computeActivityLog walks all messages' tool calls, so a re-proposed task renders N "Proposed plan" entries and attributes tools to steps via currentStepSeq inferred from update_plan_step calls across every generation — tools land under the wrong (current-gen) step or under steps that were replaced. This is the "several plans / some steps never run" view.

Fixes (ordered)

F1 — Status-driven working signal (fixes G1)

Add a derived store taskWorking(sessionId) = $streaming OR status ∈ {planning, executing} (explicitly not awaiting_input — that is paused for input), plus a global currentWorking for the main view backed by currentTask. Use it wherever streaming currently drives "is it working":

• ChatThread.svelte: traceStatus last-message = working ? 'running' : …; indicatorLabel and the AgentTrace status/label props.
• TaskContextPanel.svelte:137 spinner and UnifiedTimeline streaming prop → working.
• Keep a separate streaming for the literal "live text deltas are arriving" cursor; working is the superset for indicators/input.
• Input stays enabled while working (the user must be able to interject); the send path queues when busy (F2). Show a muted "Nomos is working…" hint in the composer when working && !streaming.

F2 — Queue operator messages; auto-run when free (fixes G2)

• Server: in-memory per-session FIFO on the agent struct (mirrors turnGate), {message, reply} entries. handleChat: when the gate is busy, enqueue instead of rejecting, and emit a queued SSE event (replaces today's error at main.go:294-302). Persist the user message as today (already done pre-acquire).
• Drain: arm a per-session drainer that, on gate release, acquires again and runs the next queued message as a normal turn (same persist/emit path as handleChat). Strictly one-at-a-time under the gate — this cannot stack turns (the hazard v0.15.0 F1 removed); background resumeSession keeps its non-blocking skip and never touches the queue.
• If the session is terminal (done/failed) or awaiting_input when a queued message runs, reopenSession/answer handling applies as for any follow-up.
• Frontend: on the queued event show an inline "Queued — will run when the current step finishes" chip on that user bubble; clear it when the turn's real events begin. Drop the humanized "still finishing" error for the busy case.

F3 — SSE keepalive on handleChat (prevents the G1 desync at the source)

Wrap a.chat(...) in a goroutine + select with a 10-15s ticker that writes an SSE comment (:keepalive\n\n) and flushes, so 20-40s inter-iteration gaps no longer trip proxy/browser idle timeouts. Stop the ticker when a.chat returns. (EventSource ignores comment lines by spec — safe.)

F4 — Generation-aware timeline + self-healing plan panel (fixes G3/G4)

• activity.ts computeActivityLog: find the last propose_plan in the message stream; ignore propose_plan/update_plan_step calls before it for both rendering and currentStepSeq inference. Render at most one "Proposed plan" entry (the current generation). Steps continue to come from $steps (already current-gen via fetchPlan MAX(generation)). Optionally emit a single "Plan revised" entry when >1 generation exists.
• Plan-panel resilience: on any STATUS_AFFECTING event (and on reconnect), re-fetch the plan (fetchPlan) in addition to the live plan.proposed handler, so a missed event self-heals instead of leaving a stale generation.

Validation

• go test ./cmd/nomos/: extend turngate_test.go/new messagequeue_test.go — queued message runs strictly after release; FIFO order preserved across 3 queued sends; a background resumeSession busy-skip does not consume or starve the queue; queued message runs even if session went awaiting_input.
• Web vitest: activity.test.ts — add a 2-generation fixture (2× propose_plan, interleaved update_plan_step) asserting exactly one "Proposed plan" and correct step attribution to gen-2 steps; chat/store test — working is true from status==='executing' even with streaming=false; queued event renders the queued chip and clears on first tool_use.
• Manual: (a) start a long task, reload the window mid-turn → the working indicator stays on (status-driven); (b) send a message mid-turn → "Queued" → runs after the turn; (c) open 44df8802-style 2-gen session → timeline shows one plan, no ghost proposals.

Risks

• F2 must not reintroduce concurrent turns. The queue drains one-at-a-time under the gate; background resume remains non-blocking and queue-agnostic. Existing turngate_test.go concurrency assertion (max in-flight = 1) must stay green.
• Status-driven working could stick on if a terminal event is missed. Mitigated by the existing terminal task.status → clearTurnState recovery plus a loadSessions refresh on reconnect (F4).
• Keepalive comments must stay SSE comments (: prefix) so they aren't parsed as events.

Out of scope / follow-ups

• Model efficiency: the 8+ pure-exploration iterations (repeated list_entities/get_relations) that inflate turn length to 15-27 min — prompt/iteration-budget tuning, separate effort.
• F8 from the prior plan (oldest-first timeline toggle; per-tool tool.* events for background turns). F1's status-driven working makes background work visible without live per-tool deltas, so this remains lower priority.

Open implementation note

Host the per-session message queue on the agent struct (in-memory map[string] []queuedMsg + per-session drainer goroutine), mirroring turnGate. No DB table needed — messages are already persisted by handleChat before enqueue; the queue only schedules when a turn runs, not whether the message is stored.

---

Resolution (2026-08-03)

Implemented F1–F4 in v0.15.1 → v0.17.0 (the intermediate 0.16.0 was the cyberspace-aesthetic commit, landed via auto-pull during this work).

Item	What shipped	Where
F1	Status-driven working signal (taskWorking(sessionId) / currentWorking) = live stream OR session status ∈ {planning, executing}. Drives the chat trace running state, the "thinking" headline, the activity spinner, and the timeline streaming prop — so a background/long/desynced turn still looks alive (the "can't tell it's working" symptom). The composer stays enabled during background work so the operator can interject.	web/src/lib/stores/workspace.ts (isWorking, taskWorking, currentWorking), ChatThread.svelte (working prop, traceStatus, indicator), TaskContextPanel.svelte, SessionChatWindow.svelte, NewTaskChat.svelte.
F2	Operator messages sent during an in-flight turn are now QUEUED and auto-run when the gate frees, replacing the "still finishing a previous step… send it again" rejection. Per-session in-memory FIFO drained strictly one-at-a-time under the turn gate (no concurrent-turn reintroduction). A queued SSE event tells the client, which drops the optimistic bubble and shows a "Queued — will run when it finishes the current step" hint (derived from working + last-message shape, so it survives the poller).	cmd/nomos/messagequeue.go (+messagequeue_test.go), agent.go (queue field), main.go (runChatTurn, drainQueued, handleChat queue path), continue.go (resumeSession drains on release), web/src/lib/types.ts (ChatQueuedEvent), chat.ts (queued handling in sendSessionMessage/startTask).
F3	SSE keepalive: a 12s :keepalive comment ticker during handleChat so 20-40s inter-iteration gaps no longer trip a proxy/browser idle timeout (the desync root cause). All SSE writes (events + keepalive) serialized through one mutex — http.ResponseWriter is not concurrency-safe.	cmd/nomos/main.go (writeMu/writeEvent, keepalive goroutine).
F4	Generation-aware activity timeline: only the LAST propose_plan renders as "Proposed plan"; superseded ones collapse to a single "Earlier plan revised" marker, and step-attribution only follows the current generation's update_plan_step calls. Plus plan-panel self-heal: the plan is refetched (debounced) on any task-lifecycle event so a missed plan.proposed no longer freezes the panel on a stale generation.	web/src/lib/stores/activity.ts (computeActivityLog), workspace.ts (schedulePlanRefetch).

Verification:

• go vet ./cmd/nomos/ clean; go test ./cmd/nomos/ green, incl. new messagequeue_test.go (FIFO, requeueFront, per-session isolation, concurrency, drainQueued no-op-on-empty, drainQueued requeues-when-busy). Existing turngate_test.go/continue_test.go still green (single-flight guarantee intact).
• Web vitest 72/72 green (added 2 F4 generation-awareness tests to activity.test.ts: one "Proposed plan" + revised marker + current-gen-only step attribution; plan-less Q&A attributes nothing).
• vite build succeeds. tsc --noEmit shows only the pre-existing baseline errors (ui/*, oidc.ts, windows.ts, workspace.ts:123/201/221) noted in v0.15.0 — no new errors from this change. ESLint: no new errors (the one new svelte/valid-compile on chatWorking got the same disable its siblings have).

Follow-ups (not in this pass):

• Model efficiency: the long (15-27 min) exploration-heavy turns that made the desync so painful — prompt / iteration-budget tuning, separate effort.
• F8 from the prior plan (oldest-first timeline toggle; per-tool tool.* events for background turns). F1's status-driven working makes background work visible without live per-tool deltas, so this stays lower priority.