oikos/plans/done/2026-08-03-nomos-chat-working-visibility.md

# 2026-08-03 — Nomos chat: working-visibility, message queue, generation-aware timeline

**Status:** Implemented (F1–F4) in v0.17.0. See
[Resolution](#resolution-2026-08-03) at the end.

## Context (grounded in last-session logs + DB, not just code)

Operator report: *"On the chat window I can't tell the agent is working; it's
making tool calls but no feedback. Typing returns 'Nomos is still finishing a
previous step…'. Activity not up to date. Several plans at once, some don't
execute."*

Verified against runtime state:

- **Last session `23da10db`** ran ONE live turn for **6m33s** (21 iterations,
  19:45:36→19:52:03, correlation `679566cb`). At 19:48:00 the operator typed
  `status`; at **19:48:05 the turn gate deferred it** (`turn already active,
  deferring operator message`). The operator could type at all only because the
  client had already lost the stream (`streaming=false`) while the server kept
  running — i.e. the client showed an *idle* window over a *working* turn. It
  ended `awaiting_input`.
- **Turn runtimes are long**: sessions in the DB run 15-27 min
  (e.g. `44df8802` 24:24, `4319b9f8` 27:05). `handleChat` (main.go:170) has **no
  SSE keepalive**; inter-iteration gaps reach 20-40s, so a proxy/browser idle
  close mid-turn resets `streaming` while the turn continues on
  `context.Background()` (pctx).
- **Re-proposing is real**: `44df8802` has **generation 1 (5 steps, all
  `replaced`) → generation 2 (25 steps, done)**, with **2 `propose_plan` + 52
  `update_plan_step`** calls persisted. The activity timeline renders every one
  of those across both generations.

## Root causes

- **G1 — "working" == `streaming`.** Every working-indication in the chat window
  (AgentTrace running status, indicator headline, stream cursor, panel spinner,
  `disabled={streaming}` input) is gated on the live SSE flag. A background turn
  (`resumeSession`/continuation worker) has no stream; a desynced long live turn
  has a dead stream. In both cases `streaming=false` while the server is actively
  working. The **session `status`** (`planning`/`executing`/`awaiting_input`) is
  the reliable "server is running a turn" signal and is already live-refreshed
  (`workspace.ts` `taskFor`, `STATUS_AFFECTING`), but the chat UI never uses it.
- **G2 — busy-turn message is rejected, not queued.** main.go:292-302: the turn
  gate waits 5s then emits the "still finishing a previous step" error and
  returns. The user message *is* persisted (main.go:270) but is **inert** — the
  user must manually re-send.
- **G3 — activity is poll/event laggy.** Tool-level activity derives from
  `messages`, refreshed only by the 3s poller; plan steps are **events-only**
  (`workspace.ts` `hydrateSession`) with no poll, so a missed `plan.proposed`
  event leaves the panel stuck on a stale generation.
- **G4 — timeline is generation-unaware.** `activity.ts` `computeActivityLog`
  walks **all** messages' tool calls, so a re-proposed task renders N
  "Proposed plan" entries and attributes tools to steps via `currentStepSeq`
  inferred from `update_plan_step` calls across **every** generation — tools land
  under the wrong (current-gen) step or under steps that were `replaced`. This is
  the "several plans / some steps never run" view.

## Fixes (ordered)

### F1 — Status-driven `working` signal (fixes G1)
Add a derived store `taskWorking(sessionId)` = `$streaming OR status ∈
{planning, executing}` (explicitly **not** `awaiting_input` — that is paused for
input), plus a global `currentWorking` for the main view backed by `currentTask`.
Use it wherever `streaming` currently drives "is it working":
- `ChatThread.svelte`: `traceStatus` last-message = `working ? 'running' : …`;
  `indicatorLabel` and the AgentTrace `status`/`label` props.
- `TaskContextPanel.svelte:137` spinner and `UnifiedTimeline` `streaming` prop →
  `working`.
- Keep a separate `streaming` for the literal "live text deltas are arriving"
  cursor; `working` is the superset for indicators/input.
- Input stays **enabled** while `working` (the user must be able to interject);
  the send path queues when busy (F2). Show a muted "Nomos is working…" hint in
  the composer when `working && !streaming`.

### F2 — Queue operator messages; auto-run when free (fixes G2)
- Server: in-memory per-session FIFO on the `agent` struct (mirrors `turnGate`),
  `{message, reply}` entries. `handleChat`: when the gate is busy, **enqueue**
  instead of rejecting, and emit a `queued` SSE event (replaces today's error at
  main.go:294-302). Persist the user message as today (already done pre-acquire).
- Drain: arm a per-session drainer that, on gate release, acquires again and runs
  the next queued message as a normal turn (same persist/emit path as
  `handleChat`). Strictly one-at-a-time under the gate — this cannot stack turns
  (the hazard v0.15.0 F1 removed); background `resumeSession` keeps its
  non-blocking skip and never touches the queue.
- If the session is terminal (`done`/`failed`) or `awaiting_input` when a queued
  message runs, `reopenSession`/answer handling applies as for any follow-up.
- Frontend: on the `queued` event show an inline "Queued — will run when the
  current step finishes" chip on that user bubble; clear it when the turn's real
  events begin. Drop the humanized "still finishing" error for the busy case.

### F3 — SSE keepalive on `handleChat` (prevents the G1 desync at the source)
Wrap `a.chat(...)` in a goroutine + `select` with a **10-15s ticker** that writes
an SSE comment (`:keepalive\n\n`) and flushes, so 20-40s inter-iteration gaps no
longer trip proxy/browser idle timeouts. Stop the ticker when `a.chat` returns.
(EventSource ignores comment lines by spec — safe.)

### F4 — Generation-aware timeline + self-healing plan panel (fixes G3/G4)
- `activity.ts` `computeActivityLog`: find the **last** `propose_plan` in the
  message stream; ignore `propose_plan`/`update_plan_step` calls **before** it
  for both rendering and `currentStepSeq` inference. Render at most one
  "Proposed plan" entry (the current generation). Steps continue to come from
  `$steps` (already current-gen via `fetchPlan` MAX(generation)). Optionally emit
  a single "Plan revised" entry when >1 generation exists.
- Plan-panel resilience: on any `STATUS_AFFECTING` event (and on reconnect),
  re-fetch the plan (`fetchPlan`) in addition to the live `plan.proposed` handler,
  so a missed event self-heals instead of leaving a stale generation.

## Validation

- `go test ./cmd/nomos/`: extend `turngate_test.go`/new `messagequeue_test.go` —
  queued message runs strictly after release; FIFO order preserved across 3
  queued sends; a background `resumeSession` busy-skip does **not** consume or
  starve the queue; queued message runs even if session went `awaiting_input`.
- Web `vitest`: `activity.test.ts` — add a 2-generation fixture (2× propose_plan,
  interleaved update_plan_step) asserting exactly one "Proposed plan" and correct
  step attribution to gen-2 steps; `chat`/store test — `working` is true from
  `status==='executing'` even with `streaming=false`; `queued` event renders the
  queued chip and clears on first tool_use.
- Manual: (a) start a long task, **reload the window mid-turn** → the working
  indicator stays on (status-driven); (b) send a message mid-turn → "Queued" →
  runs after the turn; (c) open `44df8802`-style 2-gen session → timeline shows
  one plan, no ghost proposals.

## Risks

- **F2 must not reintroduce concurrent turns.** The queue drains one-at-a-time
  under the gate; background resume remains non-blocking and queue-agnostic.
  Existing `turngate_test.go` concurrency assertion (max in-flight = 1) must stay
  green.
- **Status-driven `working` could stick on** if a terminal event is missed.
  Mitigated by the existing terminal `task.status` → `clearTurnState` recovery
  plus a `loadSessions` refresh on reconnect (F4).
- **Keepalive comments** must stay SSE comments (`:` prefix) so they aren't
  parsed as events.

## Out of scope / follow-ups

- Model efficiency: the 8+ pure-exploration iterations (repeated
  `list_entities`/`get_relations`) that inflate turn length to 15-27 min —
  prompt/iteration-budget tuning, separate effort.
- F8 from the prior plan (oldest-first timeline toggle; per-tool `tool.*` events
  for background turns). F1's status-driven `working` makes background work
  visible without live per-tool deltas, so this remains lower priority.

## Open implementation note

Host the per-session message queue on the `agent` struct (in-memory `map[string]
[]queuedMsg` + per-session drainer goroutine), mirroring `turnGate`. No DB table
needed — messages are already persisted by `handleChat` before enqueue; the queue
only schedules *when* a turn runs, not *whether* the message is stored.

---

## Resolution (2026-08-03)

Implemented F1–F4 in v0.15.1 → v0.17.0 (the intermediate 0.16.0 was the
cyberspace-aesthetic commit, landed via auto-pull during this work).

| Item | What shipped | Where |
|---|---|---|
| **F1** | Status-driven `working` signal (`taskWorking(sessionId)` / `currentWorking`) = live stream OR session status ∈ {planning, executing}. Drives the chat trace running state, the "thinking" headline, the activity spinner, and the timeline `streaming` prop — so a background/long/desynced turn still looks alive (the "can't tell it's working" symptom). The composer stays enabled during background work so the operator can interject. | `web/src/lib/stores/workspace.ts` (`isWorking`, `taskWorking`, `currentWorking`), `ChatThread.svelte` (`working` prop, `traceStatus`, indicator), `TaskContextPanel.svelte`, `SessionChatWindow.svelte`, `NewTaskChat.svelte`. |
| **F2** | Operator messages sent during an in-flight turn are now QUEUED and auto-run when the gate frees, replacing the "still finishing a previous step… send it again" rejection. Per-session in-memory FIFO drained strictly one-at-a-time under the turn gate (no concurrent-turn reintroduction). A `queued` SSE event tells the client, which drops the optimistic bubble and shows a "Queued — will run when it finishes the current step" hint (derived from `working` + last-message shape, so it survives the poller). | `cmd/nomos/messagequeue.go` (+`messagequeue_test.go`), `agent.go` (queue field), `main.go` (`runChatTurn`, `drainQueued`, handleChat queue path), `continue.go` (resumeSession drains on release), `web/src/lib/types.ts` (`ChatQueuedEvent`), `chat.ts` (`queued` handling in sendSessionMessage/startTask). |
| **F3** | SSE keepalive: a 12s `:keepalive` comment ticker during `handleChat` so 20-40s inter-iteration gaps no longer trip a proxy/browser idle timeout (the desync root cause). All SSE writes (events + keepalive) serialized through one mutex — `http.ResponseWriter` is not concurrency-safe. | `cmd/nomos/main.go` (`writeMu`/`writeEvent`, keepalive goroutine). |
| **F4** | Generation-aware activity timeline: only the LAST `propose_plan` renders as "Proposed plan"; superseded ones collapse to a single "Earlier plan revised" marker, and step-attribution only follows the current generation's `update_plan_step` calls. Plus plan-panel self-heal: the plan is refetched (debounced) on any task-lifecycle event so a missed `plan.proposed` no longer freezes the panel on a stale generation. | `web/src/lib/stores/activity.ts` (`computeActivityLog`), `workspace.ts` (`schedulePlanRefetch`). |

**Verification:**
- `go vet ./cmd/nomos/` clean; `go test ./cmd/nomos/` green, incl. new
  `messagequeue_test.go` (FIFO, requeueFront, per-session isolation, concurrency,
  drainQueued no-op-on-empty, drainQueued requeues-when-busy). Existing
  `turngate_test.go`/`continue_test.go` still green (single-flight guarantee
  intact).
- Web `vitest` 72/72 green (added 2 F4 generation-awareness tests to
  `activity.test.ts`: one "Proposed plan" + revised marker + current-gen-only
  step attribution; plan-less Q&A attributes nothing).
- `vite build` succeeds. `tsc --noEmit` shows only the pre-existing baseline
  errors (`ui/*`, `oidc.ts`, `windows.ts`, `workspace.ts:123/201/221`) noted in
  v0.15.0 — no new errors from this change. ESLint: no new errors (the one new
  `svelte/valid-compile` on `chatWorking` got the same disable its siblings have).

**Follow-ups (not in this pass):**
- Model efficiency: the long (15-27 min) exploration-heavy turns that made the
  desync so painful — prompt / iteration-budget tuning, separate effort.
- F8 from the prior plan (oldest-first timeline toggle; per-tool `tool.*` events
  for background turns). F1's status-driven `working` makes background work
  visible without live per-tool deltas, so this stays lower priority.