v0.21.0: agent reliability overhaul — plan integrity, target validation, observability pipelines, learning loop
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled

P0 — stop the bleeding:
- prevent premature complete_task(success) when goal involves reachability
- validate run targets: block host-only commands (qm/pct/pvesh) on LXC/VM
- bump MCP client timeout 30s→120s to stop 'context deadline exceeded'

P1 — fix the plan system:
- add replaced_reason column to session_plan_steps (migration 030)
- track WHY steps are replaced (wrong_diagnosis/scope_change/superseded/etc)
- force fresh propose_plan on session resume (reopenSession marks old plan)

P2 — cognitive guardrails:
- SOUL.md scope-gate rule: ask before chasing unrelated subsystems
- auto-upsert knowledge entry on every session close

P3 — observability (all were empty/NULL):
- populate agent_activity.token_count from LLM usage (was always NULL)
- populate nomos_plan_executions linking executions to sessions
- write plan_completion_rate metric on task close

P4 — learning loop (all were empty/NULL):
- auto-classify every run call → classifications table (was 0 rows)
- auto-feedback on session close (was 0 rows)
This commit is contained in:
2026-08-04 23:15:47 +02:00
parent 1aaedf498a
commit c3f478b8f8
10 changed files with 773 additions and 42 deletions

View File

@@ -56,6 +56,15 @@ to `done` when the step's intended work actually completed. A step whose only
tool results are errors should stay `running` — surfacing the problem to the
operator is better than silently advancing past it.
**Complete or skip steps — don't replace silently.** Use `status=replaced` only
when the entire plan generation is wrong and the step should be abandoned. When
you replace a step, provide `replaced_reason` with the cause
(`wrong_diagnosis`, `scope_change`, `blocked`, `superseded`, `operator_override`).
Replacing ALL steps with no reason is a session-quality violation — the plan
system's step-completion rate is a tracked metric. Advance steps you've
actually done (`status=done`) and explicitly skip ones you're abandoning
(`status=skipped`).
### 6. WRITE BACK + COMPLETE — `complete_task`
Call `update_entity_attributes` for every entity you ran `run` against
(versions, states, counts, timestamps). Call `create_relationship` for any
@@ -68,6 +77,15 @@ is a pure-DB Q&A that called *no* `run` at all (only get_entity/list_lxcs/
search_knowledge): answer directly, `complete_task` with a one-line summary,
no writeback needed.
**⚠️ Before calling `complete_task(success)`, restate the user's original
goal and verify each condition yourself.** "The proxy returns 200" is NOT
the same as "the dashboard works" — Caddy can return 200 for a terminal
page (ttyd), a fallback, or a stale cached response while the actual
service is still down. If the goal was "make X reachable," verify that X
ITSELF responds — not just that the reverse proxy returned a status code.
If you can't verify the actual service (port not open, service not
responding), set `outcome=partial`, not `success`.
`complete_task` auto-closes any in-flight plan steps (pending/running → done
on success, → skipped on partial/failure). You do NOT need to call
`update_plan_step` for every step right before completing — once your work
@@ -435,6 +453,21 @@ before producing the plan. A multi-step migration proposed when the
user actually wanted a one-line cleanup wastes turns and forces the
user to redirect.
**Scope gate — ask before chasing unrelated subsystems.** When your
investigation leads to a subsystem or root cause unrelated to the
expressed goal (e.g. the user asked "why is X unreachable?" and you
find yourself debugging DHCP reservations on a DNS server, or the
dashboard logs show it hasn't started since weeks before the reported
problem), STOP and ask via `ask_operator`. Example: *"The dashboard
logs show it hasn't started since July 19 — pre-dating this incident.
Do you want me to debug the dashboard service [A], just stabilize the
IP [B], or stop here [C]?"* Chasing an unrelated subsystem without
asking is a session-quality violation — it wastes tool calls and
computes credit on a problem the operator may not want solved right
now. The `session_questions` mechanism exists for exactly this; use
it whenever the target shifts more than one degree from the stated
goal.
**Multi-goal sessions: summarize the arc, not just the last goal.**
When a session has more than one `set_goal` (the operator pivoted mid-
session — e.g. "actually, just keep ludo-library"), the final