feat: knowledge write-back (upsert_knowledge) + proactive outcome reporting

From the last (successful) TypeType deploy session, two gaps the operator hit:

1. Knowledge write-back — the missing half of the loop.
   The agent could read the knowledge base (search_knowledge/get_entity_knowledge)
   but had no way to WRITE it, so everything it learned (the Dragonfly memlock
   rlimit gotcha, the NAT-hairpin DNS issue, etc.) lived only in an ephemeral
   chat message and was lost — the system could never actually "get better."
   This is the `upsert_knowledge` MCP tool the 2026-07-08 gaps plan called for.
   - internal/mcp/server.go: upsert_knowledge(title, content, about?, tags?,
     kind?) writes a document/investigation/runbook entity + knowledge_entities
     row (search column is generated), upserts by slug so re-titling updates in
     place, and optionally links it to the entity it's about so
     get_entity_knowledge surfaces it there.
   - SOUL.md: capture non-obvious findings/deploys/gotchas as part of finishing
     work, not only when asked "what did we learn".

2. "I had to ask for status multiple times."
   The clearest cause: a long working turn (64 tool calls) that exhausted the
   iteration cap ended with a bare "max iterations reached without final
   answer" — a dead end that forced the operator to ask what happened.
   - cmd/nomos/agent.go: on exhaustion, make one final no-tools LLM call
     (finalSummary) asking for a status report — what was accomplished, current
     state, what remains — so the turn always ends with a real outcome.
   - maxIterations 25 -> 40 (the decomposed per-step pct_create flow legitimately
     needs more steps).
   - SOUL.md: always end a turn with a clear outcome; never end silently or on a
     bare tool call — the operator can't see the tools working and reads silence
     as "nothing happened".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

cmd/nomos/agent.go

@@ -17,9 +17,11 @@ import (
 )
 
 // maxIterations bounds one chat turn's tool-calling loop. Provisioning a
-// service is a long chain (research → plan → request_execution → status), so
-// 15 was too tight and turns died with "max iterations reached" mid-deploy.
-const maxIterations = 25
+// service is a long chain (research → plan → request_execution → per-step
+// install/verify run calls), so this must be generous; a full deploy with the
+// decomposed pct_create flow can legitimately need many steps. On exhaustion
+// the loop now produces a real summary (finalSummary) rather than a dead end.
+const maxIterations = 40
 const maxLLMRetries = 1
 
 var refusalDenylist = []string{
@@ -401,7 +403,17 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
 		}
 	}
 
-	emit(agentEvent{Type: "text", Data: "Agent loop: max iterations reached without final answer.", SessionID: sessionID})
+	// Hitting the step limit used to end the turn with a bare "max iterations
+	// reached without final answer" — a dead end that made the operator ask
+	// "status?" to find out what actually happened after a long working turn.
+	// Instead, spend one final call asking the model to summarize what it did
+	// and the current state, so the turn always ends with a real report.
+	messages = append(messages, openai.SystemMessage("[System: you've reached the step limit for this turn. STOP calling tools now and write a concise status report: what you accomplished, the current state of the goal, anything that failed, and what remains. This is what the operator sees.]"))
+	summary := a.finalSummary(ctx, messages)
+	if summary == "" {
+		summary = "I hit this turn's step limit while working. I've done a lot but couldn't wrap up cleanly — ask me for a status update and I'll summarize the current state."
+	}
+	emit(agentEvent{Type: "text", Data: summary, SessionID: sessionID})
 	emit(agentEvent{Type: "done", Data: map[string]any{
 		"session_id":     sessionID,
 		"correlation_id": correlationID,
@@ -409,6 +421,22 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
 	}, SessionID: sessionID})
 }
 
+// finalSummary makes one non-tool LLM call to turn an exhausted tool-loop into
+// a real status report instead of a dead-end message. Best-effort: empty on
+// any error, and the caller has a fallback.
+func (a *agent) finalSummary(ctx context.Context, messages []openai.ChatCompletionMessageParamUnion) string {
+	params := openai.ChatCompletionNewParams{
+		Model:    openai.ChatModel(a.model),
+		Messages: messages,
+		// No Tools: force a text answer.
+	}
+	resp, err := a.provider.Chat.Completions.New(ctx, params, a.reqOpts...)
+	if err != nil || len(resp.Choices) == 0 {
+		return ""
+	}
+	return resp.Choices[0].Message.Content
+}
+
 // extractText pulls the "text" field from a persisted message's JSONB content.
 func extractText(content json.RawMessage) string {
 	var m struct {

internal/mcp/server.go

@@ -195,6 +195,19 @@ func newServer(pool *db.Pool, agentID uuid.UUID) *mcp.Server {
 			ORDER BY 1`, slug), nil
 	})
 
+	register(&mcp.Tool{Name: "upsert_knowledge", Description: "Write back what you learned so future sessions (and future you) benefit — this is how the system gets smarter over time. Use it AFTER solving a non-obvious problem, deploying a service, or discovering a gotcha: record the finding, the fix, and any caveats. Re-calling with the same title updates the existing note instead of duplicating. This is the ONLY way to persist knowledge; a chat message alone is forgotten. search_knowledge/get_entity_knowledge read it back.",
+		InputSchema: objSchema(
+			prop{"title", "string", "Short, specific, searchable title (e.g. 'Dragonfly memlock rlimit in unprivileged LXCs', not 'notes')."},
+			prop{"content", "string", "The knowledge itself, in markdown. Be concrete: symptom, root cause, the exact fix/commands, and any caveats. Written for someone hitting this fresh."},
+			prop{"about", "string", "Optional entity slug this knowledge concerns (e.g. lxc:typetype, host:strong) — links the note to that entity so get_entity_knowledge surfaces it."},
+			prop{"tags", "string", "Optional comma-separated tags (e.g. 'docker,networking,gotcha')."},
+			prop{"kind", "string", "One of: investigation (a finding/incident analysis — default), document (reference), runbook (a repeatable procedure)."},
+		),
+	}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+		args := argsMap(req)
+		return upsertKnowledge(ctx, pool, args)
+	})
+
 	register(&mcp.Tool{Name: "query_metrics", Description: "Query time-series metrics",
 		InputSchema: objSchema(prop{"hours", "integer", "Look-back window in hours (default 24)"}),
 	}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
@@ -1436,6 +1449,107 @@ func destructiveWindowActive(ctx context.Context, pool *db.Pool, agentID uuid.UU
 	return time.Now().UTC().Before(expires)
 }
 
+// knowledgeSlugRe strips a title down to a slug segment.
+var knowledgeSlugRe = regexp.MustCompile(`[^a-z0-9]+`)
+
+func knowledgeSlug(kind, title string) string {
+	s := strings.ToLower(strings.TrimSpace(title))
+	s = knowledgeSlugRe.ReplaceAllString(s, "-")
+	s = strings.Trim(s, "-")
+	if s == "" {
+		s = "note"
+	}
+	if len(s) > 80 {
+		s = s[:80]
+	}
+	return kind + ":nomos/" + s
+}
+
+// upsertKnowledge is the agent's write-back path — the missing half of the
+// knowledge loop (search_knowledge/get_entity_knowledge could only read).
+// Without this, everything the agent learned lived only in an ephemeral chat
+// message and was lost; the system could never actually "get better." A
+// knowledge doc IS an entity (type document/investigation/runbook) with a row
+// in knowledge_entities; re-titling the same thing updates in place rather
+// than duplicating. Optionally linked to the entity it's about so
+// get_entity_knowledge surfaces it there.
+func upsertKnowledge(ctx context.Context, pool *db.Pool, args map[string]any) (*mcp.CallToolResult, error) {
+	title, _ := args["title"].(string)
+	content, _ := args["content"].(string)
+	about, _ := args["about"].(string)
+	tagsRaw, _ := args["tags"].(string)
+	kind, _ := args["kind"].(string)
+
+	title = strings.TrimSpace(title)
+	content = strings.TrimSpace(content)
+	if title == "" || content == "" {
+		return textResult("error: title and content are required"), nil
+	}
+	switch kind {
+	case "document", "investigation", "runbook":
+	case "":
+		kind = "investigation"
+	default:
+		return textResult(fmt.Sprintf("error: kind must be document, investigation, or runbook (got %q)", kind)), nil
+	}
+
+	var tags []string
+	for _, t := range strings.Split(tagsRaw, ",") {
+		if t = strings.TrimSpace(t); t != "" {
+			tags = append(tags, t)
+		}
+	}
+
+	slug := knowledgeSlug(kind, title)
+
+	// Upsert the knowledge-doc entity, getting its id whether it already
+	// existed or we just created it.
+	docID, _ := uuid.NewV7()
+	err := pool.QueryRow(ctx, `
+		INSERT INTO entities (id, slug, type, name, attributes)
+		VALUES ($1, $2, $3, $4, '{}')
+		ON CONFLICT (slug) DO UPDATE SET name = EXCLUDED.name, updated_at = now()
+		RETURNING id`, docID, slug, kind, title).Scan(&docID)
+	if err != nil {
+		return textResult(fmt.Sprintf("error creating knowledge entity: %v", err)), nil
+	}
+
+	// Upsert the knowledge content (search column is generated, don't set it).
+	_, err = pool.Exec(ctx, `
+		INSERT INTO knowledge_entities (entity_id, title, content, source, tags, updated_at)
+		VALUES ($1, $2, $3, 'nomos-agent', $4, now())
+		ON CONFLICT (entity_id) DO UPDATE
+		  SET title = EXCLUDED.title, content = EXCLUDED.content,
+		      tags = EXCLUDED.tags, updated_at = now()`,
+		docID, title, content, tags)
+	if err != nil {
+		return textResult(fmt.Sprintf("error writing knowledge: %v", err)), nil
+	}
+
+	// Link it to the entity it's about, if given and not already linked.
+	linked := ""
+	if about = strings.TrimSpace(about); about != "" {
+		var targetID uuid.UUID
+		if qerr := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", about).Scan(&targetID); qerr == nil {
+			pool.Exec(ctx, `
+				INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
+				SELECT $1, $2, 'documents', '{"by":"nomos"}'::jsonb, now()
+				WHERE NOT EXISTS (
+					SELECT 1 FROM relationships
+					WHERE source_id = $1 AND target_id = $2 AND type = 'documents' AND valid_to IS NULL)`,
+				docID, targetID)
+			linked = " and linked to " + about
+		} else {
+			linked = fmt.Sprintf(" (note: entity %q not found, saved unlinked)", about)
+		}
+	}
+
+	_ = observability.Event(ctx, sqlcgen.New(pool), "knowledge.upserted", &docID, "info", "mcp", "",
+		map[string]any{"slug": slug, "title": title, "kind": kind})
+
+	return textResult(fmt.Sprintf("Saved knowledge %q as %s%s. It's now searchable via search_knowledge and will surface in future sessions.", title, slug, linked)), nil
+}
+
 func createApproval(ctx context.Context, pool *db.Pool, execID, targetID uuid.UUID, action, params, riskClass string) {
 	p := map[string]any{"action": action, "params": params, "execution_id": execID.String()}
 	payload, _ := json.Marshal(p)

nomos/SOUL.md

@@ -72,6 +72,15 @@ of what a command does.
   call `http_get` on the repo README (or `.../raw/main/docker-compose.yml`) to learn its
   stack, ports, and install steps BEFORE proposing a plan. Never tell the operator you
   cannot access the web — use this tool.
+- `search_knowledge` / `get_entity_knowledge` — READ the knowledge base. Check it before
+  deploying or debugging something — a past session may have already recorded the gotcha.
+- `upsert_knowledge` — WRITE back what you learned. This is how the system gets smarter.
+  **After you solve a non-obvious problem, finish a deployment, or discover a gotcha, record
+  it** (title, content, `about` the relevant entity slug). A chat message is forgotten; only
+  `upsert_knowledge` persists it for future sessions. Example: after fixing the Dragonfly
+  memlock rlimit in an unprivileged LXC, save an `investigation` titled for that exact
+  symptom with the fix. Don't wait to be asked "what did we learn" — capture it as part of
+  finishing the work.
 - `get_agent_activity` — your own behavior log
 
 ### Tool selection rules
@@ -234,6 +243,16 @@ port is busy, find a free one. Only surface to the operator if you've tried
 reasonable alternatives and none worked. An error in one step is not a reason
 to stop the entire turn — it's a reason to try a different approach.
 
+**Always end a turn with a clear outcome — never make the operator ask
+"status?".** When you finish (or pause) a piece of work, your final message
+must state the result plainly: what's now true, what you verified, what (if
+anything) failed or remains. Don't end a turn silently or with just a tool
+call and no summary — the operator can't see the tools working the way you
+can, and a turn that ends without a status report reads as "nothing happened."
+When the whole goal is done and verified, say so explicitly and — if you
+learned anything non-obvious getting there — `upsert_knowledge` it before you
+sign off.
+
 ## Skills
 
 Skills live in `/app/nomos/skills/`. Load a skill when its description

web/src/lib/stores/chat.ts

@@ -137,8 +137,13 @@ function startPolling(sessionId: string) {
     if (pollingSessionId !== sessionId || get(currentSession) !== sessionId) return
     const msgs = await fetchMessages(sessionId)
     if (get(streaming) || pollingSessionId !== sessionId) return // re-check: the fetch itself takes time
-    const current = get(messages)
-    if (msgs.length === current.length) return
+    // No cheap "anything new?" check: the auto-continuation worker updates a
+    // placeholder message IN PLACE as each tool call lands (see
+    // cmd/nomos/continue.go), so the message COUNT stays the same while the
+    // content changes — a length-only diff (the previous version of this
+    // code) never detected those updates and progress looked frozen even
+    // though the backend was actively working. Just re-set every tick;
+    // Svelte's own diffing keeps the actual re-render cheap.
     sessionMessages.set(msgs)
     messages.set(toChatMessages(msgs))
   }, 3000)