98e19bb14a139e6b9d144b73b2ce74015e86caf7
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 1dca2cfd7a |
feat(observability): restore monitoring coverage, make gaps visible, stream executions
Monitoring coverage was 3 of 89 active entities. Three bugs, each hidden by discarded errors in checkdefaults: - writeCheck generated a fresh uuid, inserted the check entity ON CONFLICT (slug) DO NOTHING, then wrote a check_defs row referencing it. On any re-seed the slug already existed, the entity insert no-oped, and the FK violated — aborting the ingest transaction and surfacing as an unrelated failure several entities later. Re-seeding has been broken since; prod's coverage was frozen at its first successful seed. This is what TestSeedIngestIdempotentAndNoDuplicateEdges had been reporting. - shortSlug truncated to the last 8 chars, so all 21 ingress routes collapsed to ".network" and overwrote each other; service:jellyfin collided with lxc:jellyfin. - The ssh-script checker never read the `args` config checkdefaults wrote, so process_check.sh always ran without its unit name and returned "unknown". Coverage is now 75/89. Monitoring is declared per entity type in seeds/ontology.yaml and resolved through the is-a hierarchy, so a type can say it warrants nothing (site, lan, mesh, cluster) and never be reported as a gap. coverageSweep raises an `unmonitored` signal only where a type declares monitoring it lacks — 8 real gaps, no false positives. Also: - entity_types.attribute_schema was never ingested: the seed loader read "attribute_schema" but the YAML says "attributes", so all 60 types stored JSON null. - ListExecutions ignored its declared target/action/correlation_id filters and paginated on a non-unique target slug, dropping and repeating rows. - started_at was captured but only written at terminal state, so a running execution reported NULL for its whole life. The three MCP auto-run copies wrote no timing at all; they are now one autoRun helper. - SSH output was buffered to completion and discarded entirely on timeout. Both sshExec copies now stream through a shared execlog sink into execution_logs, and keep partial output when a command is cancelled. - executions.correlation_id was a random per-execution uuid that correlated nothing; it is now the chat session id, which is what lets the chat tail live output. - reversible_low had no auto-run branch despite policy declaring it unattended. Since computeCommandRisk never returns it, the class only arises when an agent declares it over a read_only command — so gating it penalised candor without adding safety. - backup-target gains a backup-freshness checker (portable find -mmin, since the first target is on macOS), resolving its host by walking backs-up-to backwards. The pre-deploy pg_dump is now a tracked backup target. UI: an Executions section on entity detail with live output tailing, and streamed output under a running `run` call in the chat timeline. Migrations 022-024. Ops.svelte and context.ts exclude execution.output from their refetch triggers, which would otherwise fire once a second per command. Co-Authored-By: Claude <noreply@anthropic.com> |
|||
| e055a7c6ce |
feat(nomos): session-review improvements (P0/P1/P2 from 2026-07-20 audit)
Classifier now unwraps pct exec / qm guest exec / bash -c / sh -c / sudo
and env-var assignments before classification, so read-only inspection
wrapped in pct exec no longer escalates to config_mutation. curl GET
(default method, no -d/-F/-T/-o/>) is read-only. Eliminates the three
duplicate rclone sessions (a51e2086, 8acea2e3, cb8c8a4a) that bounced
off the classifier for the same goal.
New classify_command MCP tool: command-scoped preflight that returns the
exact risk class run would assign. Documented in SOUL.md with guidance
to pre-classify before run when the verdict is uncertain.
set_goal surfaces prior partial/failed sessions from the last 24h so the
agent picks up the thread instead of rediscovering it.
completeTask auto-closes in-flight plan steps (pending/running -> done
on success, skipped on partial/failure), so one-step plans no longer
need the per-step running->done dance right before completion.
Migration 021 adds blocker + closed_at to agent_sessions. completeTask
sets closed_at once and derives a structured blocker reason
(approval_timeout, user_abandoned, classifier_overreach, model_refusal,
tool_error, ...) from the last assistant message.
/sessions list now carries message_count, tool_call_count,
duration_seconds (server-side aggregates — no more N+1 transcript
fetches to audit a fleet). GET /sessions/{id} returns both metadata
and messages. New query params filter + paginate: outcome, status,
entity_id, blocker, since (RFC3339 or Go duration), cursor, limit.
Titles now prefer the goal when set; sessions without a goal fall back
to the first assistant text.
New GET /sessions/{id}/tool_calls flat view for audit scripts.
Plan: plans/2026-07-20-session-review-ten-sessions.md. VERSION 0.7.12 -> 0.7.13.
|
|||
| c96c795126 |
test: add unit tests for 6 previously-untested packages (R7)
Added pure unit tests for all packages that had 0% coverage. Where pure logic was entangled with DB calls, extracted testable helpers first. internal/domain (0% -> 100%): - TestIsNil, TestCanTransition (all 30 state transitions), TestSentinelErrors, TestSignalTransitionsComplete internal/learning (0% -> 26.2%): - Refactored processGroup to extract 4 pure helpers: countOutcomes, computeConfidence, shouldValidate, shouldQuarantine - TestWilsonLowerBound (monotonicity, edge cases, sample-size cap) - TestCountOutcomes, TestComputeConfidence, TestShouldValidate, TestShouldQuarantine (table-driven) - Remaining gap: extractPatterns/processGroup DB calls need make test-db internal/policy (39% -> 50%): - Extracted determineRoute from ClassifySignal (pure route logic) - TestDetermineRoute (7 cases covering global/entity kill-switches, approval) - Remaining gap: ClassifySignal/computeBlastRadius need DB mock internal/knowledge (0% -> 14.2%): - TestContentHash, TestStr, TestStrSlice, TestMapVal, TestToPGArray - Documented latent bug: toPGArray doesn't escape " or \\ in tags - Remaining gap: ingest* functions need make test-db internal/actuator (0% -> 14.7%): - TestSSHErrorClassString, TestClassifySSHError (11 cases incl. net.Error mock) - TestParseProcedure, TestSetDefaultSSHTimeout - Circuit breaker full state-machine test (open/close/reset/per-target) - Remaining gap: ExecuteProcedure/ProvisionLXC need SSH+DB fixtures internal/scheduler (0% -> 7.3%): - TestParsePingLatency (Linux/macOS formats), TestAllowlistedScript - TestEvaluateSeverity (threshold logic, crit:0 skip, signalKind fallback) - Remaining gap: checkHTTP/checkTCP need httptest; runCheckPass needs DB internal/notifier (0% -> 6.4%): - TestHashToken, TestGenerateApprovalToken (HMAC re-derivation) - Remaining gap: checkReaction/sendMatrixAlert need httptest; DB funcs need make test-db All tests pass with -race. domain hits its 60% gate at 100%. The remaining packages need integration tests (make test-db) and/or httptest-based tests to reach their coverage gates — tracked as follow-up. |
|||
| 7ef8446825 |
fix(agent+ui): whatsapp session audit — approvals, stuck indicator, stale execs
P1: add docker compose (logs|ps|top|config|images|port|cp) to read-only allowlist. docker compose logs was classified as config_mutation, causing individual approval cards for read-only inspection commands. P2: remove approval entries from activityLog. They were always status=running and never transitioned to done (the derived store builds from tool-call text, not execution status), causing AgentIndicator to latch onto a stale 'Approval: ...' entry and never clear — even after the session completed. P3: remove InlineApproval from Chat.svelte. The green 'Completed in 1s on lxc:...' boxes were noise in the chat stream. Approval UX belongs in the Operations page (already has it via Ops.svelte), not inline in the chat. P4: stale execution cleanup. Startup sweep (mark >1hr non-terminal as cancelled) + 5-min periodic sweep (mark >10min non-terminal as cancelled). 98 orphaned executions accumulated from eval testing (39 running from apt_upgrade:audit timeouts, 19 pending_approval, 3 approved). P5: refuse second config_mutation run when an approval is already pending for the session. Without this, the agent queues N individual approvals before the operator can respond — confirmed in session 20757eb9 (two approval cards for what should have been one plan-level approval). VERSION 0.7.0 → 0.7.1 |
|||
| e3fa6736c0 |
feat(agent): plan-first gate, iterative follow-ups, reasoning persistence
P1 plan-first: run handler refuses without propose_plan (structural gate,
not SOUL.md prose). Plan window decoupled from set_goal — config_mutation
auto-run only on operator approval (assent window). Closes the approval-free
config_mutation hole confirmed in session d0d562e0.
P2 iteration: reopenSession flips terminal→executing, marks prior plan steps
replaced, clears outcome. proposePlan excludes replaced from in-flight check,
bumps generation. A follow-up on a completed session starts a new sub-task
with a fresh plan — no more errPlanInFlight dead end.
P3 reasoning: accumulate per-iteration text into the persisted row instead
of overwriting with the last text event. Reload shows intermediate thinking,
not just the final summary.
P4 read-only allowlist: add find, tree, locate, systemctl list-timers/
list-unit-files/show, timedatectl, hostnamectl, systemd-analyze, rclone
ls/lsl/md5sum/check/cryptcheck. Fixes the find misclassification from
d0d562e0.
P5 eval harness: new assertion kinds (proposes_plan, plan_before_run,
plan_generations), multi-turn followups, fetch /sessions/{id}/plan. Four
manifests under evals/.
P6 SOUL.md: strip degenerate-case carve-out, add ITERATE step, update
set_goal guidance.
VERSION 0.6.0 → 0.7.0
|
|||
| 657e1a8be1 |
feat: assent window + compound read-only classification + continue-after-approval
Agent stopped after every approval step, forcing operator to type 'continue' 7× per deploy session. Root causes and fixes: 1. Compound read-only commands (e.g. 'systemctl status; journalctl') defaulted to config_mutation — now splits on ;/&&/||/| and classifies as read_only if all segments are inspection verbs. Added grep, wc, sort, uniq, cut, tr, dpkg -l, apt list, docker stats to allowlist. 2. curl|sh was classified destructive, forcing typed confirmation for legitimate installs (get.docker.com). Demoted to config_mutation — loose assent grants it, no typed phrase needed. 3. SOUL.md said 'STOP after queuing' — replaced with 'continue working on non-blocked steps'. Added assent window section instructing agent to carry out the full plan after approval. 4. Assent window: when operator approves a plan via chat assent, a 30-minute window opens where config_mutation commands auto-run without re-approval. Agent writes expiry to autonomy_settings; MCP run tool checks it before gating. Destructive never auto-runs. 5. System note after approval now says 'CONTINUE executing the full plan — do not stop and wait for continue.' |
|||
| d52968876a |
feat: general gated run primitive + chat-assent approval (Layer 0)
Implements the first slice of plans/2026-07-10-general-gated-execution.md:
Nomos gets one general execution tool instead of only a fixed action enum,
gated by an automatic risk classifier, and approval can be granted by the
operator just replying in chat instead of clicking a button.
- internal/policy/command.go: ClassifyCommand(cmd, declaredRisk) — rule-based
read-only allowlist + destructive denylist, default-escalate to
config_mutation for anything else. Classification can only ESCALATE the
caller's declared risk, never de-escalate it (destructive always wins even
if declared read_only). Compound commands (&&, ;, |, $()) never qualify for
the read-only fast path. Full test corpus.
- internal/mcp/server.go: new `run` MCP tool — target (host:/lxc:), command,
purpose, optional declared_risk. Read-only commands execute immediately;
everything else queues an approval exactly like pct_create today, executed
via httpapi's existing executeApprovedAction. Also fixes a real latent bug:
pct_exec resolved an LXC's host attribute without the "host:" prefix, so it
could never find the Proxmox host — new resolveExecTarget/resolveRunTarget
helpers (mcp + httpapi) fix this for both the new `run` action and existing
actions that route through the same execution path.
- internal/httpapi/phase3.go: "run" case in executeApprovedAction; fixes two
bugs found while wiring this up — (1) DecideApproval hardcoded risk_class to
'config_mutation' on every approve, silently corrupting the audit ledger for
every other risk class; (2) denying/revoking an approval never updated the
linked execution's status, so it stayed 'pending_approval' forever instead
of reflecting the decision.
- cmd/nomos/assent.go: deterministic (not LLM-judged) chat-assent detection.
Scoped to the immediately-preceding assistant turn's pending approvals only
— an old "yes" can't retroactively approve something new. Destructive-risk
actions are excluded from loose assent. Approves via the same HTTP decision
endpoint the UI button calls, so both paths share one audit trail.
- web/.../InlineApproval.svelte: self-healing poll — a pending approval card
now picks up being decided via ANY path (chat assent, Ops page, Matrix), not
just its own button. Previously the banner stayed stuck showing
Approve/Deny even after the action had already run elsewhere.
- nomos/SOUL.md: `run` is now the general capability ("no fixed menu, only a
risk gate"); documents chat-assent behavior and the destructive exception.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|||
| aa197190cd |
phase 3 review: fix broken error classification, stub checks, wasted uuid, token idempotency, dead code
- actuator/ssh.go: custom errorsAs chain broken — all SSH errors classified as SSHErrorOther. Replaced with standard errors.As + errors.Is. - scheduler/scheduler.go: all four check functions were stubs returning healthy. Implemented real HTTP GET, TCP dial, unix.Statfs disk, and TLS cert expiry checks. - learning/learning.go: uuid.NewV7() called unconditionally before ON CONFLICT upsert. Now looks up existing pattern first, reuses entity_id. - notifier/notifier.go: removed dead var_, fixed token regeneration every 15s. Now skips if token_hash already set. - phase3.go: removed dead GetPattern+dummy args call in PatchPattern. - classify.go: removed unused var_ guard. |
|||
| 095a3967c4 |
phase 3: control loop — scheduler, actuator, learning, notifier, policy, API endpoints
Implemented the full OODA control loop: Scheduler: - Check_defs runner with bounded worker pool (errgroup) - Signal dedup via partial unique index (UpsertSignal) - Recovery auto-resolves open signals - Metrics writing (InsertMetricSample) and entity_status updates - Housekeeping (idempotency-key prune) - Graceful shutdown via ctx cancellation Actuator: - Auto-act signal consumer with FOR UPDATE SKIP LOCKED pattern - Per-target serialization with pg_advisory_xact_lock - Circuit breaker per target host (N consecutive failures → open) - Autonomy kill-switch (global.auto_act, never_auto_act.<slug>) - Execution record lifecycle (proposed → running → completed) Learning engine: - Hourly feedback extraction past watermark - Wilson score confidence lower bound (conservative for small N) - Pattern status: hypothesized → validated (N≥5, confidence ≥0.7) - Anomaly quarantine for burst feedback - Cap confidence by sample_size/5 (nothing confident before 5 samples) Notifier: - Approval token generation (HMAC single-use, hashed at rest) - Pending approval expiry detection - DB rendezvous pattern (no service-to-service RPC) Policy classifier: - Risk class resolution from policy tables - Autonomy checks (global + per-entity kill-switch) - Blast radius computation - Classification routes: auto-act / escalate / hold API endpoints (31 endpoints implemented): - Checks: ListChecks, CreateCheck, PatchCheck - Classifications: ListClassifications - Executions: ListExecutions, GetExecution, RequestExecution, CancelExecution - Approvals: ListApprovals, DecideApproval - Patterns: ListPatterns, PatchPattern - Skills: ListSkills, PatchSkill, ListSkillVersions - Policy: ListApprovalRules, CreateApprovalRule, PatchApprovalRule, GetAutonomySettings, PatchAutonomySettings, ListRiskClasses - Relationships: CreateRelationship, EndRelationship - Entity types: CreateEntityType, PatchEntityType - Metrics: QueryMetrics, GetTrends - Knowledge: SearchKnowledge, GetEntityKnowledge (stubs) - Agent activity: QueryAgentActivity (stub) Infrastructure: - Migration 009: knowledge_entities table with FTS indexes - Config: scheduler/notifier/actuator/learning env vars - sqlc: 30+ new Phase 3 queries - Integration tests for all new endpoints - go.sum updated with golang.org/x/sync |
