feat(nomos): session-review improvements (P0/P1/P2 from 2026-07-20 audit)
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled

Classifier now unwraps pct exec / qm guest exec / bash -c / sh -c / sudo
and env-var assignments before classification, so read-only inspection
wrapped in pct exec no longer escalates to config_mutation. curl GET
(default method, no -d/-F/-T/-o/>) is read-only. Eliminates the three
duplicate rclone sessions (a51e2086, 8acea2e3, cb8c8a4a) that bounced
off the classifier for the same goal.

New classify_command MCP tool: command-scoped preflight that returns the
exact risk class run would assign. Documented in SOUL.md with guidance
to pre-classify before run when the verdict is uncertain.

set_goal surfaces prior partial/failed sessions from the last 24h so the
agent picks up the thread instead of rediscovering it.

completeTask auto-closes in-flight plan steps (pending/running -> done
on success, skipped on partial/failure), so one-step plans no longer
need the per-step running->done dance right before completion.

Migration 021 adds blocker + closed_at to agent_sessions. completeTask
sets closed_at once and derives a structured blocker reason
(approval_timeout, user_abandoned, classifier_overreach, model_refusal,
tool_error, ...) from the last assistant message.

/sessions list now carries message_count, tool_call_count,
duration_seconds (server-side aggregates — no more N+1 transcript
fetches to audit a fleet). GET /sessions/{id} returns both metadata
and messages. New query params filter + paginate: outcome, status,
entity_id, blocker, since (RFC3339 or Go duration), cursor, limit.

Titles now prefer the goal when set; sessions without a goal fall back
to the first assistant text.

New GET /sessions/{id}/tool_calls flat view for audit scripts.

Plan: plans/2026-07-20-session-review-ten-sessions.md. VERSION 0.7.12 -> 0.7.13.
This commit is contained in:
2026-07-20 11:32:31 +02:00
parent 9f4d645d06
commit e055a7c6ce
11 changed files with 1116 additions and 48 deletions

View File

@@ -86,18 +86,39 @@ func (s *store) close() {
// session is a chat session elevated to a task: goal-structured work with a
// lifecycle status and an outcome (see migration 018 / the task-board plan).
// Outcome/Summary/EntityID are empty until set, hence omitempty.
//
// P1.5 (2026-07-20): Blocker and ClosedAt track WHY a session ended
// partial/failed and WHEN it actually closed. ClosedAt is distinct from
// LastActiveAt — the latter is touched on any access (including a UI
// transcript view), the former is set ONCE at completion. Without it,
// "duration" computed as last_active - created lies for reopened sessions
// (a51e2086 reported 4-day duration because the operator reopened it).
// Blocker is a short structured reason: approval_timeout,
// classifier_overreach, user_abandoned, tool_error, etc. See
// plans/2026-07-20-session-review-ten-sessions.md P1.5.
type session struct {
ID string `json:"id"`
Title string `json:"title"`
Actor string `json:"actor"`
Goal string `json:"goal"`
Status string `json:"status"`
Outcome string `json:"outcome,omitempty"`
Summary string `json:"summary,omitempty"`
EntityID string `json:"entity_id,omitempty"`
PendingApprovals int `json:"pending_approvals"`
CreatedAt time.Time `json:"created_at"`
LastActiveAt time.Time `json:"last_active_at"`
ID string `json:"id"`
Title string `json:"title"`
Actor string `json:"actor"`
Goal string `json:"goal"`
Status string `json:"status"`
Outcome string `json:"outcome,omitempty"`
Summary string `json:"summary,omitempty"`
EntityID string `json:"entity_id,omitempty"`
PendingApprovals int `json:"pending_approvals"`
Blocker string `json:"blocker,omitempty"`
CreatedAt time.Time `json:"created_at"`
LastActiveAt time.Time `json:"last_active_at"`
ClosedAt *time.Time `json:"closed_at,omitempty"`
// P2.6 (2026-07-20): server-side aggregates so /sessions can answer
// "how big was this task?" without N+1 transcript fetches. The audit
// had to pull every session's full message tree to count tool calls —
// ~600 KB of JSON for 10 sessions. With these, the list view is a
// single round trip. omitempty so getSession for a brand-new session
// with zero activity doesn't emit zeros.
MessageCount int `json:"message_count,omitempty"`
ToolCallCount int `json:"tool_call_count,omitempty"`
DurationSeconds int `json:"duration_seconds,omitempty"`
}
type message struct {
@@ -315,14 +336,98 @@ func (s *store) touchSession(ctx context.Context, id string) {
}
func (s *store) listSessions(ctx context.Context) ([]session, error) {
return s.listSessionsFiltered(ctx, listFilter{Limit: 50})
}
// listFilter carries the optional WHERE/ORDER clauses added by P2.8
// (filtering & pagination). All fields optional; empty values are no-ops.
// The handler in main.go parses query params into this struct so the SQL
// builder here is the single source of truth for what filters exist.
type listFilter struct {
Outcome string // exact match on outcome (success/partial/failure)
Status string // exact match on status (active/done/failed/executing)
EntityID string // exact match on entity_id (UUID)
Blocker string // exact match on blocker reason
Since string // last_active_at >= this; RFC3339 timestamp OR Go duration (e.g. "24h")
Cursor string // last_active_at < cursor (RFC3339) — page back in time
Limit int // default 50, clamped by the handler
}
func (s *store) listSessionsFiltered(ctx context.Context, f listFilter) ([]session, error) {
if s == nil {
return nil, nil
}
rows, err := s.pool.Query(ctx,
`SELECT s.id, s.title, s.actor, s.goal, s.status, COALESCE(s.outcome, ''), s.summary,
COALESCE(s.entity_id::text, ''),
COALESCE(pa.cnt, 0),
s.created_at, s.last_active_at
if f.Limit <= 0 {
f.Limit = 50
}
// Build the WHERE clause dynamically. We use a single args slice with
// $N placeholders to keep pgx happy; the index increments per clause.
var (
where []string
args []any
n = 1
)
if f.Outcome != "" {
where = append(where, fmt.Sprintf("COALESCE(s.outcome, '') = $%d", n))
args = append(args, f.Outcome)
n++
}
if f.Status != "" {
where = append(where, fmt.Sprintf("s.status = $%d", n))
args = append(args, f.Status)
n++
}
if f.EntityID != "" {
// Accept UUID or string; cast gracefully if invalid.
if _, err := uuid.Parse(f.EntityID); err == nil {
where = append(where, fmt.Sprintf("s.entity_id = $%d::uuid", n))
args = append(args, f.EntityID)
n++
}
}
if f.Blocker != "" {
where = append(where, fmt.Sprintf("COALESCE(s.blocker, '') = $%d", n))
args = append(args, f.Blocker)
n++
}
if f.Since != "" {
// Accept RFC3339 timestamp OR a Go-style duration like "24h", "7d".
// Try timestamp first, fall back to duration relative to now.
if t, err := time.Parse(time.RFC3339, f.Since); err == nil {
where = append(where, fmt.Sprintf("s.last_active_at >= $%d", n))
args = append(args, t)
n++
} else if d, err := time.ParseDuration(f.Since); err == nil {
where = append(where, fmt.Sprintf("s.last_active_at >= now() - ($%d * interval '1 second')", n))
args = append(args, d.Seconds())
n++
}
// Unknown format: silently drop the filter — better than erroring
// out and breaking the whole list. Caller can validate if needed.
}
if f.Cursor != "" {
if t, err := time.Parse(time.RFC3339, f.Cursor); err == nil {
where = append(where, fmt.Sprintf("s.last_active_at < $%d", n))
args = append(args, t)
n++
}
}
whereClause := ""
if len(where) > 0 {
whereClause = "WHERE " + strings.Join(where, " AND ")
}
args = append(args, f.Limit)
limitArg := fmt.Sprintf("$%d", n)
query := fmt.Sprintf(`
SELECT s.id, s.title, s.actor, s.goal, s.status, COALESCE(s.outcome, ''), s.summary,
COALESCE(s.entity_id::text, ''),
COALESCE(pa.cnt, 0),
COALESCE(s.blocker, ''),
s.created_at, s.last_active_at, s.closed_at,
COALESCE(msg.cnt, 0),
COALESCE(act.cnt, 0),
COALESCE(EXTRACT(EPOCH FROM (COALESCE(s.closed_at, s.last_active_at) - s.created_at))::bigint, 0)
FROM agent_sessions s
LEFT JOIN (
SELECT l.session_id, COUNT(*) AS cnt
@@ -331,7 +436,22 @@ func (s *store) listSessions(ctx context.Context) ([]session, error) {
WHERE e.status = 'pending_approval'
GROUP BY l.session_id
) pa ON pa.session_id = s.id
ORDER BY s.last_active_at DESC LIMIT 50`)
LEFT JOIN (
SELECT session_id, COUNT(*) AS cnt
FROM agent_messages
GROUP BY session_id
) msg ON msg.session_id = s.id
LEFT JOIN (
SELECT session_id::uuid AS sid, COUNT(*) AS cnt
FROM agent_activity
WHERE session_id IS NOT NULL AND session_id <> ''
GROUP BY session_id
) act ON act.sid = s.id
%s
ORDER BY s.last_active_at DESC
LIMIT %s`, whereClause, limitArg)
rows, err := s.pool.Query(ctx, query, args...)
if err != nil {
return nil, err
}
@@ -342,7 +462,8 @@ func (s *store) listSessions(ctx context.Context) ([]session, error) {
var sess session
if err := rows.Scan(&sess.ID, &sess.Title, &sess.Actor, &sess.Goal, &sess.Status,
&sess.Outcome, &sess.Summary, &sess.EntityID, &sess.PendingApprovals,
&sess.CreatedAt, &sess.LastActiveAt); err != nil {
&sess.Blocker, &sess.CreatedAt, &sess.LastActiveAt, &sess.ClosedAt,
&sess.MessageCount, &sess.ToolCallCount, &sess.DurationSeconds); err != nil {
return nil, err
}
out = append(out, sess)
@@ -356,18 +477,102 @@ func (s *store) getSession(ctx context.Context, id string) (*session, error) {
}
var sess session
err := s.pool.QueryRow(ctx,
`SELECT id, title, actor, goal, status, COALESCE(outcome, ''), summary,
COALESCE(entity_id::text, ''), 0, created_at, last_active_at
FROM agent_sessions WHERE id = $1`, id).
`SELECT s.id, s.title, s.actor, s.goal, s.status, COALESCE(s.outcome, ''), s.summary,
COALESCE(s.entity_id::text, ''), 0, COALESCE(s.blocker, ''),
s.created_at, s.last_active_at, s.closed_at,
COALESCE(msg.cnt, 0),
COALESCE(act.cnt, 0),
COALESCE(EXTRACT(EPOCH FROM (COALESCE(s.closed_at, s.last_active_at) - s.created_at))::bigint, 0)
FROM agent_sessions s
LEFT JOIN (
SELECT session_id, COUNT(*) AS cnt
FROM agent_messages
WHERE session_id = $1::uuid
GROUP BY session_id
) msg ON msg.session_id = s.id
LEFT JOIN (
SELECT session_id::uuid AS sid, COUNT(*) AS cnt
FROM agent_activity
WHERE session_id IS NOT NULL AND session_id <> ''
AND session_id::uuid = $1::uuid
GROUP BY session_id
) act ON act.sid = s.id
WHERE s.id = $1`, id).
Scan(&sess.ID, &sess.Title, &sess.Actor, &sess.Goal, &sess.Status,
&sess.Outcome, &sess.Summary, &sess.EntityID, &sess.PendingApprovals,
&sess.CreatedAt, &sess.LastActiveAt)
&sess.Blocker, &sess.CreatedAt, &sess.LastActiveAt, &sess.ClosedAt,
&sess.MessageCount, &sess.ToolCallCount, &sess.DurationSeconds)
if err != nil {
return nil, err
}
return &sess, nil
}
// recentPartialSessions returns recent sessions (within `since`) whose outcome
// is partial or failed, excluding the current session. Used by the set_goal
// handler to surface prior unfinished work on the same problem — three
// duplicate rclone sessions (a51e2086, 8acea2e3, cb8c8a4a) all bounced off
// the classifier because each new session started from scratch. Surfacing the
// prior session's goal + summary at set_goal time lets the agent pick up the
// thread instead of rediscovering it. See
// plans/2026-07-20-session-review-ten-sessions.md P1.3.
func (s *store) recentPartialSessions(ctx context.Context, excludeSessionID string, since time.Duration) ([]session, error) {
if s == nil {
return nil, nil
}
rows, err := s.pool.Query(ctx,
`SELECT s.id, s.title, s.actor, s.goal, s.status, COALESCE(s.outcome, ''), s.summary,
COALESCE(s.entity_id::text, ''),
COALESCE(pa.cnt, 0),
COALESCE(s.blocker, ''),
s.created_at, s.last_active_at, s.closed_at,
COALESCE(msg.cnt, 0),
COALESCE(act.cnt, 0),
COALESCE(EXTRACT(EPOCH FROM (COALESCE(s.closed_at, s.last_active_at) - s.created_at))::bigint, 0)
FROM agent_sessions s
LEFT JOIN (
SELECT l.session_id, COUNT(*) AS cnt
FROM nomos_plan_executions l
JOIN executions e ON e.entity_id = l.execution_id
WHERE e.status = 'pending_approval'
GROUP BY l.session_id
) pa ON pa.session_id = s.id
LEFT JOIN (
SELECT session_id, COUNT(*) AS cnt
FROM agent_messages
GROUP BY session_id
) msg ON msg.session_id = s.id
LEFT JOIN (
SELECT session_id::uuid AS sid, COUNT(*) AS cnt
FROM agent_activity
WHERE session_id IS NOT NULL AND session_id <> ''
GROUP BY session_id
) act ON act.sid = s.id
WHERE s.id <> $1
AND s.last_active_at >= now() - ($2 * interval '1 second')
AND COALESCE(s.outcome, '') IN ('partial', 'failed')
ORDER BY s.last_active_at DESC
LIMIT 10`,
excludeSessionID, since.Seconds())
if err != nil {
return nil, err
}
defer rows.Close()
var out []session
for rows.Next() {
var sess session
if err := rows.Scan(&sess.ID, &sess.Title, &sess.Actor, &sess.Goal, &sess.Status,
&sess.Outcome, &sess.Summary, &sess.EntityID, &sess.PendingApprovals,
&sess.Blocker, &sess.CreatedAt, &sess.LastActiveAt, &sess.ClosedAt,
&sess.MessageCount, &sess.ToolCallCount, &sess.DurationSeconds); err != nil {
return nil, err
}
out = append(out, sess)
}
return out, rows.Err()
}
// getMessages returns a session's ENTIRE message history, unbounded — used
// for the UI's own transcript view (GET /sessions/{id}), where the operator
// should be able to see everything a task has done regardless of how long
@@ -397,6 +602,77 @@ func (s *store) getMessages(ctx context.Context, sessionID string) ([]message, e
return out, rows.Err()
}
// SessionToolCall is the flat view of one tool call as exposed by
// GET /sessions/{id}/tool_calls. Mirrors the persisted tool_call shape but
// drops the message-shell wrapping. Args/Result are kept as RawMessage so
// the caller can decide how to render them (the audit case wanted raw
// text sizes, but other callers may want full JSON).
type SessionToolCall struct {
ID string `json:"id"`
Name string `json:"name"`
Args json.RawMessage `json:"args,omitempty"`
Result json.RawMessage `json:"result,omitempty"`
Error string `json:"error,omitempty"`
Type string `json:"type,omitempty"` // "tool_use" or "tool_result"
MessageID string `json:"message_id"`
Role string `json:"role"`
Seq int `json:"seq"` // 1-indexed position within the session (across all messages)
CreatedAt time.Time `json:"created_at"`
}
// getSessionToolCalls walks a session's messages and returns a flat list of
// tool calls in chronological order, without the two-level message nesting.
// The audit at plans/2026-07-20-session-review-ten-sessions.md P2.10 had to
// write Python to walk messages[].content.tool_calls[]; this method makes
// it a single SQL + Go walk on the server. Each tool_use/tool_result pair
// is emitted as two rows (same id, different Type), preserving the
// persisted shape — clients that want the merged shape can group by ID.
func (s *store) getSessionToolCalls(ctx context.Context, sessionID string) ([]SessionToolCall, error) {
if s == nil {
return nil, nil
}
msgs, err := s.getMessages(ctx, sessionID)
if err != nil {
return nil, err
}
var out []SessionToolCall
seq := 0
for _, m := range msgs {
var payload struct {
ToolCalls []struct {
ID string `json:"id"`
Type string `json:"type"`
Name string `json:"name"`
Args json.RawMessage `json:"args"`
Result json.RawMessage `json:"result"`
Error string `json:"error"`
} `json:"tool_calls"`
}
if err := json.Unmarshal(m.Content, &payload); err != nil {
continue
}
for _, tc := range payload.ToolCalls {
if tc.ID == "" {
continue
}
seq++
out = append(out, SessionToolCall{
ID: tc.ID,
Name: tc.Name,
Args: tc.Args,
Result: tc.Result,
Error: tc.Error,
Type: tc.Type,
MessageID: m.ID,
Role: m.Role,
Seq: seq,
CreatedAt: m.CreatedAt,
})
}
}
return out, nil
}
// getRecentMessages returns the most recent `limit` messages for sessionID,
// in chronological order, plus whether older messages exist beyond that
// window. Used specifically for LLM replay (chatWith): without a bound,
@@ -528,7 +804,7 @@ func (s *store) setGoal(ctx context.Context, sessionID, goal string) error {
`UPDATE session_plan_steps SET status = 'replaced', finished_at = COALESCE(finished_at, now()) WHERE session_id = $1 AND status <> 'replaced'`,
sessionID)
if _, err := s.pool.Exec(ctx,
`UPDATE agent_sessions SET goal = $2, status = 'executing', last_active_at = now() WHERE id = $1`,
`UPDATE agent_sessions SET goal = $2, status = 'executing', title = $2, last_active_at = now() WHERE id = $1`,
sessionID, goal); err != nil {
return err
}
@@ -809,6 +1085,27 @@ func (s *store) completeTask(ctx context.Context, sessionID, outcome, summary st
s.pool.Exec(ctx, `UPDATE nomos_plan_executions SET continued_at = now()
WHERE session_id = $1 AND continued_at IS NULL`, sessionID)
// P1.4 (2026-07-20): auto-close any in-flight plan steps so the agent
// doesn't need an update_plan_step(running)→update_plan_step(done)
// dance for each step right before completion. Session 8c76bb3a
// (greeting + title-sync test) burned 4 update_plan_step calls for a
// one-step plan. completeTask is the authoritative terminal — any
// step still in pending/running when the task ends is closed (as
// "done" for success, "skipped" for partial/failure) so the UI's plan
// view doesn't show orphaned running steps on a completed task.
// Replaced/cancelled/blocked steps are left alone.
closeStatus := "done"
if outcome != "success" {
closeStatus = "skipped"
}
if _, err := s.pool.Exec(ctx, `
UPDATE session_plan_steps
SET status = $3, finished_at = COALESCE(finished_at, now())
WHERE session_id = $1 AND status IN ('pending', 'running')`,
sessionID, "", closeStatus); err != nil {
slog.Warn("nomos: completeTask failed to auto-close in-flight steps", "session", sessionID, "error", err)
}
// Clean up assent and destructive window keys from autonomy_settings.
s.pool.Exec(ctx, `DELETE FROM autonomy_settings
WHERE key LIKE '%:' || $1`, sessionID)
@@ -817,9 +1114,22 @@ func (s *store) completeTask(ctx context.Context, sessionID, outcome, summary st
if outcome == "failure" {
status = "failed"
}
// P1.5 (2026-07-20): derive a structured blocker reason when the
// outcome is partial/failed, so trend analysis can answer "why are
// sessions failing?" without parsing free-text summaries. Three
// duplicate rclone sessions (a51e2086, 8acea2e3, cb8c8a4a) all
// bounced off the classifier; without a blocker field, the *why* was
// buried in the last assistant message. The signatures matched here
// are the recurring ones from the 2026-07-20 session audit. Empty for
// success — that's not a blocker.
blocker := ""
if outcome != "success" {
blocker = deriveBlocker(ctx, s, sessionID, summary)
}
if _, err := s.pool.Exec(ctx, `
UPDATE agent_sessions SET status = $2, outcome = $3, summary = $4, last_active_at = now()
WHERE id = $1`, sessionID, status, outcome, summary); err != nil {
UPDATE agent_sessions SET status = $2, outcome = $3, summary = $4,
blocker = $5, closed_at = now(), last_active_at = now()
WHERE id = $1`, sessionID, status, outcome, summary, blocker); err != nil {
return err
}
var entID uuid.UUID
@@ -837,10 +1147,58 @@ func (s *store) completeTask(ctx context.Context, sessionID, outcome, summary st
}
_ = observability.Event(ctx, sqlcgen.New(s.pool), "task.status", entPtr, severity, "nomos", sessionID,
map[string]any{"status": status, "outcome": outcome, "summary": summary,
"cancelled_executions": cancelledCount})
"cancelled_executions": cancelledCount, "blocker": blocker})
return nil
}
// blockerPatterns maps a substring (case-insensitive) to a structured blocker
// reason. Order matters — earlier patterns take precedence. These are the
// recurring failure signatures from the 2026-07-20 session audit. A
// real-world blocker that doesn't match any of these falls through to
// "uncategorized" — better than empty, because empty means "we don't know
// it's a blocker at all." See plans/2026-07-20-session-review-ten-sessions.md.
var blockerPatterns = []struct {
pattern string
reason string
}{
{"queued for approval", "approval_timeout"},
{"assent window", "approval_timeout"},
{"cancel", "user_abandoned"},
{"close this session", "user_abandoned"},
{"lets just close", "user_abandoned"},
{"classifier flagged", "classifier_overreach"},
{"config_mutation", "classifier_overreach"},
{"refus", "model_refusal"}, // refuses/refused/refusal
{"empty response", "model_empty_response"},
{"no local knowledge", "missing_knowledge"},
{"can't run", "missing_capability"},
{"cannot run", "missing_capability"},
{"timeout", "tool_error"},
{"error", "tool_error"},
}
// deriveBlocker scans the last assistant message + the summary for known
// failure signatures and returns the matching structured reason. Returns
// "uncategorized" when outcome is partial/failed but no signature matched —
// better than "" because the audit needs to know this WAS blocked, just for
// an unknown reason. Returns "" for success outcomes (caller checks first).
func deriveBlocker(ctx context.Context, s *store, sessionID, summary string) string {
// Pull the last assistant text — that's where the agent's parting
// words explain why it didn't finish.
var lastText string
_ = s.pool.QueryRow(ctx, `
SELECT content::text FROM agent_messages
WHERE session_id = $1 AND role = 'assistant'
ORDER BY created_at DESC LIMIT 1`, sessionID).Scan(&lastText)
haystack := strings.ToLower(lastText + " " + summary)
for _, p := range blockerPatterns {
if strings.Contains(haystack, p.pattern) {
return p.reason
}
}
return "uncategorized"
}
// hadEntityWriteback checks whether this session called update_entity_attributes
// or create_relationship — used by complete_task to warn the agent when it
// forgot to persist entity facts (the #1 cause of knowledge graph drift).