oikos/plans/done/2026-07-09-chat-sessions-improvements.md

2026-07-09 — Chat sessions: reliability, cost, and session-management fixes

Status: Done — 2026-07-11. All 5 findings fixed on main (49c37fe fix: chat session reliability, cost, and hygiene): empty/refusal retry guard in agent.go, bulk-tool guidance in SOUL.md, tool-result truncation in store.go, get_state_snapshot filtering, and session delete + generated titles.

Goal

Fix concrete problems found by inspecting the actual production Nomos chat data (agent_sessions/agent_messages on the oikos Postgres, 24 sessions / 52 messages as of 2026-07-09), not a code-review of the UI in the abstract. Findings below are backed by real rows, not hypotheticals.

How this was investigated

Queried oikos-postgres-1 directly (docker exec oikos-postgres-1 psql -U oikos -d oikos) since this runs on mac-mini, the same host as the production containers. Pulled session list, per-message content sizes, tool-call breakdowns, and cross-checked against cmd/nomos/agent.go to explain what was observed.

---

Findings

1. ~17% of turns come back completely empty, silently

4 of 24 sessions ("hi" ×3, "what services are healthy?") have an assistant message with text="" and zero tool calls — the model returned a blank completion. cmd/nomos/agent.go:175-183 treats len(msg.ToolCalls) == 0 as a normal final answer and emits text: "" + done. No error event fires, so chat.ts never sets $error, and Chat.svelte renders a permanently-blank assistant bubble (the "…" typing dots only show while $streaming is true; once done fires they vanish, leaving nothing). The user has no idea the turn failed and no obvious way to retry — they have to notice the silence and retype.

Fix: in agent.chat(), if msg.Content == "" && len(msg.ToolCalls) == 0, treat it as a retryable failure: log it, retry once against the provider before giving up, and if still empty, emit a real error event instead of an empty text/done pair. On the frontend, surface a "Nomos didn't respond — retry?" affordance on empty assistant messages rather than a silent blank bubble.

2. A tool-heavy turn came back as a canned non-English refusal

The session "what are the termals of hubris?" ran 22 tool calls and then returned, verbatim: 关于这个问题，我没有相关信息，您可以尝试问我其它问题，我会尽力为您解答~ ("I don't have relevant information on this, try asking me something else"). This is after successfully gathering data via tools — the model discarded its own tool results and emitted a boilerplate deflection in the wrong language. NOMOS_MODEL is currently a DeepSeek flash-tier model on OpenRouter, which is consistent with this kind of degraded-tier fallback text leaking through.

Fix: add a response-quality guard in agent.chat() — if the final text doesn't match the conversation's language/looks like a canned refusal (simple heuristic: non-ASCII-majority reply to an ASCII-majority conversation, or matches a small denylist of known refusal boilerplate), treat it like the empty-response case (retry, then surface an error rather than showing it to the operator as a real answer). Separately, reconsider whether the flash-tier model is worth the latency/cost tradeoff given it's producing failures like this in a small sample — worth an eval pass against a couple of alternative OpenRouter models on the same 24 real prompts before deciding.

3. Simple fleet questions fan out into dozens of individual tool calls

"Are any of the proxmox hosts saturated?" (2 hosts) triggered 70 tool calls in one turn, 21 of them individual get_lxc_state calls — one per LXC container — instead of using the already-available list_lxcs() bulk tool. Similar pattern in "What should be updated with high priority?" (68 calls) and "What needs updating?" (54 calls). Each get_lxc_state is a live pct status SSH round-trip to the Proxmox host, so this is 21 sequential SSH round trips to answer a question list_lxcs() already answers in one call. This is the direct cause of both slow responses and the huge persisted payloads in finding 4.

Fix: two angles, not mutually exclusive:

• Prompt-level: tighten the Nomos system prompt (nomos/SOUL.md) to explicitly prefer bulk tools (list_lxcs, get_state_snapshot, query_metrics) over per-entity tools when the question is fleet-wide, and only fall back to get_lxc_state/tail_log for a specific named entity.
• Tool-level: get_lxc_state already exists per-slug; consider whether list_lxcs()'s summary is actually sufficient for "saturated" (CPU/mem % per container) — if it's missing that field, that's why the model loops per-container, and the real fix is enriching list_lxcs() rather than prompting around the gap.

4. Tool results are persisted raw and unbounded, inflating messages to 100KB+

Message content sizes in agent_messages.content (JSONB) range up to 106KB for a single assistant turn. Even a plain "hi" greeting produced a 44KB message, because get_state_snapshot()'s full result — every entity in the DB, including ~15 document:containers/* rows that are all state: <nil>, health: unknown and contribute nothing — gets embedded verbatim in the tool_calls[].result field and stored as-is (cmd/nomos/store.go:68-76 just JSON-inserts whatever the tool returned). This bloats the DB, and every time a session is opened via loadSessionMessages or the SessionRail/ Sessions page loads history, the browser downloads and parses all of it just to render a collapsed tool-call summary.

Fix:

• Filter get_state_snapshot()'s result server-side (in the MCP tool, not the agent) to drop entities with no meaningful state/health signal, or add a type filter param the agent can pass.
• In store.saveMessage, cap persisted tool-result size (e.g. truncate to a few KB with a "...truncated, N bytes" marker) — the full result already served its purpose informing that turn's answer; historical replay (agent.chat()'s history-replay loop at agent.go:122-137) doesn't need the full blob, just enough for the model to know what it already checked.

5. No session hygiene: duplicate/typo'd titles, no delete/archive

Session titles are the raw, unprocessed first user message (store.createSession), with no dedup, normalization, or cleanup. Real production titles include 6 sessions titled "hi", 2 titled "say ok", and typos preserved verbatim ("what are the termals of hubris?", "whats the termans of strong", "Are any of the proxomox hosts saturated?"). Neither Sessions.svelte nor SessionRail.svelte nor the store/API layer (cmd/nomos/store.go, web/src/lib/api.ts) has any delete or archive path — grepped the whole stack, confirmed absent. Throwaway test sessions accumulate forever with no way to clean them from the UI.

Fix:

• Add DELETE /sessions/{id} to the nomos gateway + a matching store method and wire a delete affordance into SessionRail/Sessions (hover trash icon, confirm on click).
• Generate titles from the assistant's actual answer once the turn completes (or a cheap follow-up summarization call) instead of the raw first message, so distinct "hi" sessions become distinguishable by what was actually discussed.

---

Implementation order

1. Empty-response + refusal-leak guard (cmd/nomos/agent.go) — highest user-visible impact, smallest change, no schema/API changes.
2. Bulk-tool prompting fix (nomos/SOUL.md) — cheap, directly cuts latency and tool-call volume; re-run the same 24 real prompts against the updated prompt to confirm get_lxc_state fan-out drops.
3. Tool-result truncation on persist (cmd/nomos/store.go) — bounds future DB growth; pair with a one-off cleanup pass on the 52 existing rows if the table needs to be shrunk immediately.
4. get_state_snapshot filtering — coordinate with whichever MCP tool file defines it; verify with jsonb_pretty on a fresh "hi" session that payload drops well below the current ~44KB.
5. Session delete + title generation — UI + gateway change, lowest risk, can ship independently of 1-4.

Verification

• Re-run the same 24 real user prompts (recorded in this plan's investigation) against the patched agent; confirm zero empty/refusal-leak responses and get_lxc_state-style fan-out drops to O(hosts) not O(containers).
• docker exec oikos-postgres-1 psql -U oikos -d oikos -c "SELECT max(length(content::text)) FROM agent_messages;" before/after — expect the ceiling to move from ~106KB to low single-digit KB.
• Manually delete a test session via the new UI affordance, confirm it's gone from both SessionRail and the agent_sessions table.