Files
oikos/plans/2026-07-10-general-gated-execution.md
dtoro 4a96f46e76
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
plan: general gated execution — unlimited actions, gated by risk classifier
Evaluate the agent's action path against the OIKOS.md design. Finding: the
intended model (unlimited runbook-driven actions gated by a risk classifier)
already exists on paper and in scaffolding, but the live agent path regressed
to a hard-coded 5-action enum that bypasses the classifier. Plan a layered
realignment: (0) one general gated `run` primitive, (1) runbooks-as-data as the
reliable fast-path, (2) learning. Chosen v1 posture: approve-most (read-only
auto-runs, all state changes gate). Incremental, each step shippable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 09:03:23 +02:00

10 KiB
Raw Blame History

2026-07-10 — General gated execution: from fixed actions to unlimited-but-gated

Status: Planned

Goal

Make Nomos able to do anything needed to maintain the homelab — provision LXCs, deploy services, debug, restart, fix configs, investigate — without that capability being a fixed enum of hand-coded actions. The action space is unlimited; the gate on it is a risk classifier + operator approval, not a whitelist of tricks. Knowledge (the graph + runbooks) supplies the how; the agent's reasoning supplies the what; the classifier supplies the may I.

Operator directive (2026-07-10): "The number of actions the agent should be able to do is unlimited. We need logic to gate destructive actions, but we should not limit what the agent can do."

Chosen autonomy posture for v1: approve-most (cautious) — only genuinely read-only commands auto-run; anything that changes state requires operator approval. We can relax later once the classifier and ledger have earned trust.

This is a realignment, not a new idea

.agents/OIKOS.md already specifies this exact model:

The classifier scores risk class × blast radius × confidence and routes: auto-act / escalate / queue. The classifier can only lower autonomy relative to policy, never raise it. When in doubt, escalate. Act — execute through homelab commands or runbooks (never ad-hoc SSH).

So the target architecture is the documented architecture. The problem is the implementation diverged from it on the agent's action path.

Gap analysis (grounded in code)

Designed (OIKOS.md) Actually implemented today
Classifier routes every action by risk × blast × confidence internal/policy/classify.go ClassifySignal() only classifies Signals (the Observe pipeline), by entity+action-type. It is not called by the agent's mutation path.
Agent acts through unlimited runbooks Agent acts through request_execution with a hard-coded enum: restart, systemctl, pct_exec, apt_upgrade, pct_create (internal/mcp/server.go), each bespoke Go, gated by per-action ifs, not the classifier. New capability = new Go + redeploy.
Runbooks/skills are executable data skills table + .agents/skills/* + knowledge_entities exist and are readable (get_skills), but nothing executes a runbook. The knowledge is inert w.r.t. action.
Auto-act loop consumes classified signals and acts internal/actuator/actuator.go:124 is a literal "stub execution" — it marks work done without doing it.
"Never ad-hoc SSH" pct_exec is ad-hoc SSH (arbitrary shell in a container) and auto-runs with no approval or classification.

Net: the elegant model exists as scaffolding (classifier, policy schema, risk classes, blast-radius graph walks, skills-as-data, ledger, and the approval+feedback plumbing hardened in the 2026-07-09/10 sessions), but the live agent→action path is a bag of tricks that bypasses all of it. Everything added in the recent LXC-deploy work (pct_create + DNS/VMID/template logic) made the bag bigger — reliable, but on the wrong axis.

Bones that already exist and get reused: internal/policy (classifier + computeBlastRadius/blast_radius() SQL), risk_classes/action_risk tables, seeds/policy.yaml, executions/approvals/audit_log, the MCP SSH machinery, and the inline approval + execution-status feedback loop (chat polls GET /executions/{id}).

Target architecture — three layers

Layer 0 — one general gated primitive (the foundation)

Collapse the fixed enum into essentially one tool:

run(target, command, purpose, [declared_risk])
  • target — any host or LXC slug; resolves to SSH (host) or pct exec (LXC).
  • command — arbitrary shell.
  • purpose — the agent's stated intent (shown to the operator, feeds classify).
  • declared_risk — optional agent self-assessment.

Every call flows through:

  1. Classify the command → read_only | reversible_low | config_mutation | destructive. Rule-based:
    • read-only allowlist (e.g. leading verb in cat, ls, stat, journalctl, systemctl status|is-active, pct config|status, df, free, uptime, ip, ss, docker ps|logs, git status|log) → read_only;
    • destructive denylist (rm -rf, dd, mkfs, wipefs, pct destroy, qm destroy, shutdown, reboot, > /dev/, :(){ :|:& };:, secret exfiltration, piping remote scripts to a root shell) → destructive;
    • anything writing state / installing / editing configs → config_mutation;
    • default → escalate (config_mutation) when unsure. The classifier may only make declared_risk stricter, never looser (mirrors "can only lower autonomy, never raise").
  2. Route (approve-most posture):
    • read_only → auto-run + ledger, no approval.
    • reversible_low / config_mutationoperator approval (v1 gates all state changes; a later posture can auto-run reversible_low).
    • destructive → approval + typed confirmation phrase.
  3. Execute (existing SSH/pct exec), verify (optional check command), ledger (executions + audit_log), stream feedback to chat (reuse the GET /executions/{id} polling + InlineApproval phases already built).

Layer 0 alone delivers "the agent can attempt anything; state changes are gated."

Layer 1 — runbooks as executable data (reliability without rigidity)

The hard-won procedures become runbooks in the knowledge DB, retrieved and executed step-by-step via Layer 0 — not frozen Go:

  • pct_create + its DNS-self-heal / VMID-collision / template-resolution / locale logic becomes the canonical "provision LXC" runbook (parametric steps the agent fills in and runs through run). The reliability survives as documented, reusable steps rather than a compiled handler.
  • New capability = new runbook (data), no redeploy.
  • Keep a small set of mechanical helpers where a shell step is genuinely fiddly (e.g. "pick a free cluster VMID"), exposed as callable sub-tools — but the flow is agent-driven, not enum-driven.

This is the crucial both/and: the general primitive is the unlimited escape hatch; curated runbooks are the reliable fast-path so the agent doesn't re-derive DNS/VMID/docker every time (the exact thing that failed repeatedly in the 2026-07-09 sessions).

Layer 2 — learning closes the loop

Successful ad-hoc run sequences get promoted into runbooks/patterns (the learning engine + skills table already exist for this); the failure ledger informs retries. The system grows more capable as data, not as code.

Safety model (the whole point of the gate)

  • Default-escalate. Nothing is forbidden; risky things need the operator's "yes." Unknown/unparseable risk → approval.
  • Hard denylist for catastrophic patterns → always typed confirmation, even if the agent declared them safe.
  • Blast radius at approval time — graph walk (blast_radius() exists): "this restarts caddy → 8 downstream services."
  • Preview / dry-run where the command supports it.
  • Kill-switch (global.auto_act, per-target never_auto_act.*) already exists; extend to a global "require approval for everything" flip.
  • Full audit ledger — every command, its classification, decision, actor, output. Non-negotiable.
  • Scope guards — resolve target to a real entity first; refuse commands against destroyed/unknown targets; cap output size (already done).

Honest risks / tradeoffs

  • Trades a small vetted surface (5 actions) for arbitrary root across the fleet, LLM-driven, gated only by classifier + approval. Classifying arbitrary shell perfectly is impossible; default-escalate + hard denylist + always-on audit is the mitigation, not perfect classification.
  • Approve-most means more operator clicks initially. Acceptable while trust is built; the posture is a config knob, not a rewrite.
  • Runbook-as-data can drift from reality like any doc; the ledger + verify step
    • learning loop are the correction mechanism.

Migration path (incremental, each step shippable)

  1. Command classifier — extend internal/policy with ClassifyCommand(cmd, declaredRisk) → riskClass (allowlist/denylist/default- escalate + can-only-escalate rule). Unit-tested against a corpus of safe / mutating / catastrophic commands.
  2. run tool — new MCP tool routing classify → gate → execute → the existing feedback path. Ship alongside the current tools (no removal yet).
  3. Approval context — surface risk class + blast radius + purpose on the approval (chat InlineApproval + Ops page); typed-confirmation for destructive.
  4. Runbook execution — a "provision LXC" runbook (ports the current pct_create logic) executed via run; validate parity with today's handler.
  5. Retire the enum — convert remaining hard-coded actions to runbooks; make request_execution a thin deprecated alias or remove it.
  6. Revive auto-act — replace the actuator stub, reusing the same classifier for the Observe→Act direction (signals), still approve-most.

Verification

  • Classifier corpus test: read-only commands auto-pass; a set of known catastrophic commands always route to destructive+confirmation; ambiguous commands escalate. No command auto-runs that mutates state.
  • End-to-end: operator asks Nomos a novel task not in the old enum (e.g. "tail caddy's error log and restart it if it's flapping"); Nomos composes run calls; read-only steps auto-run, the restart gates for approval; chat shows live status; ledger records each command + classification.
  • Parity: "provision an LXC with a service" via the runbook path matches the reliability proven for the pct_create handler (free VMID, DNS, install, verify), then destroy.

Open questions for the operator

  • Reversible-low posture: keep gating restarts/syncs in v1 (chosen), or auto-run them once the classifier is trusted?
  • Confirmation phrase: per-action typed phrase for destructive, or a global one?
  • Runbook authorship: operator-authored only, or may Nomos propose new runbooks (subject to approval) from successful ad-hoc sequences?
  • Blast-radius threshold: should a large blast radius force approval even for otherwise-reversible actions?