feat(agent): close all post-fix remainders + golden eval harness (F.1-F.2, C.1-C.2, B.4-B.6, E.1-E.2)

Ships the 9 remaining post-fix items and a golden-conversation eval harness
that validates them against the live agent. All 4 evals pass.

SOUL.md (F.1, C.2, E.1):
- Consolidated three overlapping task-flow sections (MANDATORY TASK FLOW,
  'Every chat is a task', 'AFTER EVERY TASK: WRITE BACK') into one. ~50
  lines shorter. The operator's 'be more crisp' feedback.
- Added anti-patterns: don't re-execute on UI/sidebar complaints (C.2);
  don't re-run fleet-wide audits when same-day knowledge exists (E.1).
- Updated approval vocabulary in step 4 to match tasks.go (approved/yes/
  go/proceed/continue/ok/go ahead).

Tool-result strings (F.2):
- set_goal: tightened to 'Goal set. NEXT: pre-plan (read-only tools only).
  Then propose_plan. Do not call run.'
- update_plan_step: added '(Advance with update_plan_step + run; do not
  re-propose.)'

C.1 — completeTask rejects re-completion of a terminal session:
- Returns errTaskAlreadyComplete when status is already done/failed.
- The tool result directs: 'Task is already complete. Do not call
  complete_task again. If the operator pointed out a UI/sidebar
  inconsistency, fix it with update_plan_step...'

B.4 — Surface real model error text:
- chatWith's error event now includes finish_reason + refusal text:
  'Nomos returned an empty or unusable response (finish_reason=length).
  Retry or rephrase.' instead of generic 'empty response'.
- The resume-failed note already carried errText (B.3), which now has
  the real context.

B.5 — Back off between resume retries (4s, 8s):
- resumeSession now sleeps before attempts 1 and 2 (exponential backoff).
  A transient provider issue gets time to clear instead of 3 identical
  calls in 3 seconds.

B.6 — Don't persist the empty placeholder as a visible bubble:
- If a chat turn ends with no text and no tool calls (model empty-response'd
  and all retries failed), delete the placeholder row instead of persisting
  an empty bubble. The error was already streamed via done+error=true.

E.2 — list_lxcs last-audited hint:
- The list_lxcs result now includes last_audited_at — the most recent
  knowledge entry (tagged audit/update, or titled audit/update) linked
  via an 'about' edge. The agent can see 'nextcloud — last audited today'
  and skip re-running it.

Tool-call doubling bug fix (found by the eval harness):
- main.go + continue.go: the tool_use and tool_result events were both
  appending separate entries to the persisted tool_calls array, doubling
  every tool call in the transcript. Confirmed pre-existing (d9cdcee1,
  v0.3.x era). Fixed: tool_use creates the entry, tool_result merges the
  result into the same entry (matched by id). One entry per tool call.

Golden eval harness (cmd/nomos/eval/):
- A standalone Go program that loads YAML manifests of golden conversations
  + assertions, sends prompts to the chat endpoint, drains the SSE stream
  (keeping the agent's context alive), and scores structural assertions
  against the persisted transcript.
- 4 golden conversations covering: trivial read-only (degenerate case),
  plan + proceed (the original duplication bug), UI complaint (no re-exec),
  fleet audit (knowledge preferred over re-execution).
- Structural assertions only (tool-call sequences, plan steps, writeback,
  completion) — text quality is model-dependent and not scored.
- Run: go run ./cmd/nomos/eval -gateway http://localhost:8092 -manifest
  cmd/nomos/eval/evals/*.yaml  (~$0.10/run in OpenRouter credits).

Eval results (4/4 passed):
  trivial_readonly:              2 tool calls, no plan, no run
  plan_advances_on_proceed:     13 tool calls, propose_plan x1, writes back
  ui_complaint_no_rerun:        12 tool calls, propose_plan x1, writes back
  knowledge_preferred_over_rerun: 7 tool calls, search_knowledge x1, 0 run

Version 0.5.2 -> 0.5.3 (minor: eval harness + structural hardening).

cmd/nomos/eval/manifest.go

@@ -0,0 +1,174 @@
+package main
+
+import (
+	"fmt"
+	"os"
+
+	"gopkg.in/yaml.v3"
+)
+
+// conversation is one golden conversation from a manifest.
+type conversation struct {
+	Name       string     `yaml:"name"`
+	Prompt     string     `yaml:"prompt"`
+	Followup   string     `yaml:"followup"`
+	Assertions []assertion `yaml:"assertions"`
+}
+
+// assertion is one check against the final transcript. The `kind` field
+// selects the scorer; the rest are scorer-specific parameters.
+//
+// Supported kinds:
+//
+//	completes             — session status reached done/failed (not stuck executing)
+//	outcome_is            — session outcome == value (success/failure/partial)
+//	no_propose_plan       — propose_plan was never called
+//	proposes_plan_once    — propose_plan was called exactly once
+//	no_duplicate_proposal — propose_plan called at most once
+//	writes_back           — update_entity_attributes or create_relationship was called
+//	max_tool_calls        — total tool calls <= value
+//	max_run_calls         — total `run` calls <= value
+//	no_run                — `run` was never called
+//	no_rerun              — `run` was NOT called after the followup turn (if any)
+//	calls_tool            — the named tool appears in the transcript
+//	plan_step_count       — the plan has exactly `value` steps
+//	no_duplicate_complete — complete_task called at most once
+type assertion struct {
+	Kind  string `yaml:"kind"`
+	Value any    `yaml:"value"`
+}
+
+// loadManifest reads a YAML file containing a list of conversations.
+func loadManifest(path string) ([]conversation, error) {
+	b, err := os.ReadFile(path)
+	if err != nil {
+		return nil, err
+	}
+	var convs []conversation
+	if err := yaml.Unmarshal(b, &convs); err != nil {
+		return nil, fmt.Errorf("parse %s: %w", path, err)
+	}
+	return convs, nil
+}
+
+// scoreAssertions evaluates each assertion against the transcript + session.
+func scoreAssertions(asserts []assertion, t transcript, s sessionState) []assertionResult {
+	out := make([]assertionResult, 0, len(asserts))
+	for _, a := range asserts {
+		r := assertionResult{Name: a.Kind}
+		r.Passed, r.Detail = scoreOne(a, t, s)
+		if !r.Passed && r.Detail == "" {
+			r.Detail = "assertion failed"
+		}
+		out = append(out, r)
+	}
+	return out
+}
+
+func scoreOne(a assertion, t transcript, s sessionState) (bool, string) {
+	tools := t.toolNames()
+	switch a.Kind {
+	case "completes":
+		if s.Status == "done" || s.Status == "failed" {
+			return true, fmt.Sprintf("status=%s", s.Status)
+		}
+		return false, fmt.Sprintf("status=%s (not terminal)", s.Status)
+
+	case "outcome_is":
+		want, _ := a.Value.(string)
+		if s.Outcome == want {
+			return true, fmt.Sprintf("outcome=%s", s.Outcome)
+		}
+		return false, fmt.Sprintf("outcome=%s, want %s", s.Outcome, want)
+
+	case "no_propose_plan":
+		n := countTool(tools, "propose_plan")
+		if n == 0 {
+			return true, "propose_plan not called"
+		}
+		return false, fmt.Sprintf("propose_plan called %d time(s)", n)
+
+	case "proposes_plan_once":
+		n := countTool(tools, "propose_plan")
+		if n == 1 {
+			return true, "propose_plan called once"
+		}
+		return false, fmt.Sprintf("propose_plan called %d time(s), want 1", n)
+
+	case "no_duplicate_proposal":
+		n := countTool(tools, "propose_plan")
+		if n <= 1 {
+			return true, fmt.Sprintf("propose_plan called %d time(s)", n)
+		}
+		return false, fmt.Sprintf("propose_plan called %d time(s), want <= 1", n)
+
+	case "writes_back":
+		n := countTool(tools, "update_entity_attributes") + countTool(tools, "create_relationship")
+		if n > 0 {
+			return true, fmt.Sprintf("%d writeback call(s)", n)
+		}
+		return false, "no update_entity_attributes or create_relationship calls"
+
+	case "max_tool_calls":
+		max := toInt(a.Value)
+		if t.toolCallCount() <= max {
+			return true, fmt.Sprintf("%d tool calls (<= %d)", t.toolCallCount(), max)
+		}
+		return false, fmt.Sprintf("%d tool calls, want <= %d", t.toolCallCount(), max)
+
+	case "max_run_calls":
+		max := toInt(a.Value)
+		n := countTool(tools, "run")
+		if n <= max {
+			return true, fmt.Sprintf("%d run calls (<= %d)", n, max)
+		}
+		return false, fmt.Sprintf("%d run calls, want <= %d", n, max)
+
+	case "no_run":
+		n := countTool(tools, "run")
+		if n == 0 {
+			return true, "run not called"
+		}
+		return false, fmt.Sprintf("run called %d time(s)", n)
+
+	case "calls_tool":
+		want, _ := a.Value.(string)
+		n := countTool(tools, want)
+		if n > 0 {
+			return true, fmt.Sprintf("%s called %d time(s)", want, n)
+		}
+		return false, fmt.Sprintf("%s not called", want)
+
+	case "no_duplicate_complete":
+		n := countTool(tools, "complete_task")
+		if n <= 1 {
+			return true, fmt.Sprintf("complete_task called %d time(s)", n)
+		}
+		return false, fmt.Sprintf("complete_task called %d time(s), want <= 1", n)
+
+	default:
+		return false, fmt.Sprintf("unknown assertion kind: %s", a.Kind)
+	}
+}
+
+func countTool(names []string, name string) int {
+	n := 0
+	for _, x := range names {
+		if x == name {
+			n++
+		}
+	}
+	return n
+}
+
+func toInt(v any) int {
+	switch x := v.(type) {
+	case int:
+		return x
+	case int64:
+		return int(x)
+	case float64:
+		return int(x)
+	}
+	return 0
+}