oikos/plans/2026-07-09-chat-sessions-improvements.md

# 2026-07-09 — Chat sessions: reliability, cost, and session-management fixes

**Status:** Planned

## Goal

Fix concrete problems found by inspecting the *actual* production Nomos chat
data (`agent_sessions`/`agent_messages` on the `oikos` Postgres, 24 sessions /
52 messages as of 2026-07-09), not a code-review of the UI in the abstract.
Findings below are backed by real rows, not hypotheticals.

## How this was investigated

Queried `oikos-postgres-1` directly (`docker exec oikos-postgres-1 psql -U oikos
-d oikos`) since this runs on mac-mini, the same host as the production
containers. Pulled session list, per-message content sizes, tool-call
breakdowns, and cross-checked against `cmd/nomos/agent.go` to explain what was
observed.

---

## Findings

### 1. ~17% of turns come back completely empty, silently

4 of 24 sessions ("hi" ×3, "what services are healthy?") have an assistant
message with `text=""` and zero tool calls — the model returned a blank
completion. `cmd/nomos/agent.go:175-183` treats `len(msg.ToolCalls) == 0` as a
normal final answer and emits `text: ""` + `done`. No `error` event fires, so
[chat.ts](../web/src/lib/stores/chat.ts) never sets `$error`, and
[Chat.svelte](../web/src/pages/Chat.svelte) renders a permanently-blank
assistant bubble (the "…" typing dots only show while `$streaming` is true;
once `done` fires they vanish, leaving nothing). The user has no idea the turn
failed and no obvious way to retry — they have to notice the silence and
retype.

**Fix:** in `agent.chat()`, if `msg.Content == "" && len(msg.ToolCalls) == 0`,
treat it as a retryable failure: log it, retry once against the provider
before giving up, and if still empty, emit a real `error` event instead of an
empty `text`/`done` pair. On the frontend, surface a "Nomos didn't respond —
retry?" affordance on empty assistant messages rather than a silent blank
bubble.

### 2. A tool-heavy turn came back as a canned non-English refusal

The session "what are the termals of hubris?" ran 22 tool calls and then
returned, verbatim: `关于这个问题，我没有相关信息，您可以尝试问我其它问题，我会尽力为您解答~`
("I don't have relevant information on this, try asking me something else").
This is after successfully gathering data via tools — the model discarded its
own tool results and emitted a boilerplate deflection in the wrong language.
`NOMOS_MODEL` is currently a DeepSeek flash-tier model on OpenRouter, which is
consistent with this kind of degraded-tier fallback text leaking through.

**Fix:** add a response-quality guard in `agent.chat()` — if the final text
doesn't match the conversation's language/looks like a canned refusal (simple
heuristic: non-ASCII-majority reply to an ASCII-majority conversation, or
matches a small denylist of known refusal boilerplate), treat it like the
empty-response case (retry, then surface an error rather than showing it to
the operator as a real answer). Separately, reconsider whether the flash-tier
model is worth the latency/cost tradeoff given it's producing failures like
this in a small sample — worth an eval pass against a couple of alternative
OpenRouter models on the same 24 real prompts before deciding.

### 3. Simple fleet questions fan out into dozens of individual tool calls

"Are any of the proxmox hosts saturated?" (2 hosts) triggered **70 tool
calls** in one turn, 21 of them individual `get_lxc_state` calls — one per LXC
container — instead of using the already-available `list_lxcs()` bulk tool.
Similar pattern in "What should be updated with high priority?" (68 calls) and
"What needs updating?" (54 calls). Each `get_lxc_state` is a live `pct status`
SSH round-trip to the Proxmox host, so this is 21 sequential SSH round trips
to answer a question `list_lxcs()` already answers in one call. This is the
direct cause of both slow responses and the huge persisted payloads in
finding 4.

**Fix:** two angles, not mutually exclusive:
- **Prompt-level**: tighten the Nomos system prompt (`nomos/SOUL.md`) to
  explicitly prefer bulk tools (`list_lxcs`, `get_state_snapshot`,
  `query_metrics`) over per-entity tools when the question is fleet-wide, and
  only fall back to `get_lxc_state`/`tail_log` for a specific named entity.
- **Tool-level**: `get_lxc_state` already exists per-slug; consider whether
  `list_lxcs()`'s summary is actually sufficient for "saturated" (CPU/mem %
  per container) — if it's missing that field, that's *why* the model loops
  per-container, and the real fix is enriching `list_lxcs()` rather than
  prompting around the gap.

### 4. Tool results are persisted raw and unbounded, inflating messages to 100KB+

Message content sizes in `agent_messages.content` (JSONB) range up to
**106KB** for a single assistant turn. Even a plain "hi" greeting produced a
44KB message, because `get_state_snapshot()`'s full result — every entity in
the DB, including ~15 `document:containers/*` rows that are all
`state: <nil>, health: unknown` and contribute nothing — gets embedded
verbatim in the `tool_calls[].result` field and stored as-is
(`cmd/nomos/store.go:68-76` just JSON-inserts whatever the tool returned).
This bloats the DB, and every time a session is opened via
[loadSessionMessages](../web/src/lib/stores/chat.ts#L56) or the
[SessionRail](../web/src/lib/components/SessionRail.svelte)/
[Sessions](../web/src/pages/Sessions.svelte) page loads history, the browser
downloads and parses all of it just to render a collapsed tool-call summary.

**Fix:**
- Filter `get_state_snapshot()`'s result server-side (in the MCP tool, not
  the agent) to drop entities with no meaningful state/health signal, or add
  a `type` filter param the agent can pass.
- In `store.saveMessage`, cap persisted tool-result size (e.g. truncate to a
  few KB with a `"...truncated, N bytes"` marker) — the full result already
  served its purpose informing that turn's answer; historical replay
  (`agent.chat()`'s history-replay loop at `agent.go:122-137`) doesn't need
  the full blob, just enough for the model to know what it already checked.

### 5. No session hygiene: duplicate/typo'd titles, no delete/archive

Session titles are the raw, unprocessed first user message
(`store.createSession`), with no dedup, normalization, or cleanup. Real
production titles include **6 sessions titled "hi"**, **2 titled "say ok"**,
and typos preserved verbatim ("what are the **termals** of hubris?", "whats
the **termans** of strong", "Are any of the **proxomox** hosts saturated?").
Neither [Sessions.svelte](../web/src/pages/Sessions.svelte) nor
[SessionRail.svelte](../web/src/lib/components/SessionRail.svelte) nor the
`store`/API layer (`cmd/nomos/store.go`, `web/src/lib/api.ts`) has any delete
or archive path — grepped the whole stack, confirmed absent. Throwaway test
sessions accumulate forever with no way to clean them from the UI.

**Fix:**
- Add `DELETE /sessions/{id}` to the nomos gateway + a matching store method
  and wire a delete affordance into `SessionRail`/`Sessions` (hover trash
  icon, confirm on click).
- Generate titles from the assistant's actual answer once the turn completes
  (or a cheap follow-up summarization call) instead of the raw first message,
  so distinct "hi" sessions become distinguishable by what was actually
  discussed.

---

## Implementation order

1. **Empty-response + refusal-leak guard** (`cmd/nomos/agent.go`) — highest
   user-visible impact, smallest change, no schema/API changes.
2. **Bulk-tool prompting fix** (`nomos/SOUL.md`) — cheap, directly cuts
   latency and tool-call volume; re-run the same 24 real prompts against the
   updated prompt to confirm `get_lxc_state` fan-out drops.
3. **Tool-result truncation on persist** (`cmd/nomos/store.go`) — bounds
   future DB growth; pair with a one-off cleanup pass on the 52 existing rows
   if the table needs to be shrunk immediately.
4. **`get_state_snapshot` filtering** — coordinate with whichever MCP tool
   file defines it; verify with `jsonb_pretty` on a fresh "hi" session that
   payload drops well below the current ~44KB.
5. **Session delete + title generation** — UI + gateway change, lowest risk,
   can ship independently of 1-4.

## Verification

- Re-run the same 24 real user prompts (recorded in this plan's investigation)
  against the patched agent; confirm zero empty/refusal-leak responses and
  `get_lxc_state`-style fan-out drops to O(hosts) not O(containers).
- `docker exec oikos-postgres-1 psql -U oikos -d oikos -c "SELECT max(length(content::text)) FROM agent_messages;"`
  before/after — expect the ceiling to move from ~106KB to low single-digit KB.
- Manually delete a test session via the new UI affordance, confirm it's gone
  from both `SessionRail` and the `agent_sessions` table.