Files
oikos/plans/2026-07-10-general-gated-execution.md
dtoro ef5a92269b docs: reconcile plans/ status against actual code state
Audited all 10 active plan docs against the codebase (not just commit
titles). 5 were fully shipped and stale-tagged "Planned"/"In Progress" —
moved to done/ with verification notes. The other 4 got corrected
Planned→In Progress status plus concrete remaining-gap notes so the next
pass doesn't re-derive what's already done.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-11 11:42:26 +02:00

284 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 2026-07-10 — General gated execution: from fixed actions to unlimited-but-gated
**Status:** In Progress — audited 2026-07-11. Done: `ClassifyCommand` risk
classifier, general `run` MCP tool, chat-assent approval (no button
required), blast radius on approval cards, session digest, global activity
feed (`Ops.svelte` "Executions" tab, risk-badged), Learning view
(success-rate trend). Still open: retire the fixed `request_execution`
action enum (`restart, systemctl, pct_exec, apt_upgrade, pct_create` still
hard-coded alongside `run`), and revive auto-act — `internal/actuator/actuator.go:125`
is still a literal `{"success": true, "message": "stub execution"}` stub.
## Goal
Make Nomos able to do **anything** needed to maintain the homelab — provision
LXCs, deploy services, debug, restart, fix configs, investigate — without that
capability being a fixed enum of hand-coded actions. The action space is
unlimited; the *gate* on it is a risk classifier + operator approval, not a
whitelist of tricks. Knowledge (the graph + runbooks) supplies the *how*; the
agent's reasoning supplies the *what*; the classifier supplies the *may I*.
Operator directive (2026-07-10): **"The number of actions the agent should be
able to do is unlimited. We need logic to gate destructive actions, but we
should not limit what the agent can do."**
Chosen autonomy posture for v1: **approve-most (cautious)** — only genuinely
read-only commands auto-run; anything that changes state requires operator
approval. We can relax later once the classifier and ledger have earned trust.
Operator directive #2 (2026-07-10): **"I want to see the system come alive and
learn and get better."** Observability is a first-class deliverable, not a
side-effect. As a user I must be able to see, in real time: what is being
executed, on what, and why; how it was classified and routed; what the outcome
was; and — crucially — **what knowledge the session created** (new runbooks,
patterns, resolved signals, ledger entries) so the system's growth is visible.
Operator directive #3 (2026-07-10): **approval is granted by chat assent, not a
button.** When Nomos proposes a plan/action and the operator replies "go ahead"
/ "yes" / "do it" in the chat, that assent *is* the approval. No separate
Approve button for the normal case. (Destructive actions still require an
explicit typed confirmation phrase — see Safety.)
## This is a realignment, not a new idea
[.agents/OIKOS.md](../.agents/OIKOS.md) already specifies this exact model:
> The classifier scores **risk class × blast radius × confidence** and routes:
> auto-act / escalate / queue. The classifier can only *lower* autonomy relative
> to policy, never raise it. When in doubt, escalate.
> Act — execute through `homelab` commands or **runbooks** (never ad-hoc SSH).
So the target architecture is the *documented* architecture. The problem is the
implementation diverged from it on the agent's action path.
## Gap analysis (grounded in code)
| Designed (OIKOS.md) | Actually implemented today |
|---|---|
| Classifier routes every action by risk × blast × confidence | [`internal/policy/classify.go`](../internal/policy/classify.go) `ClassifySignal()` only classifies **Signals** (the Observe pipeline), by entity+action-type. It is **not** called by the agent's mutation path. |
| Agent acts through unlimited **runbooks** | Agent acts through `request_execution` with a **hard-coded enum**: `restart, systemctl, pct_exec, apt_upgrade, pct_create` ([`internal/mcp/server.go`](../internal/mcp/server.go)), each bespoke Go, gated by per-action `if`s, not the classifier. New capability = new Go + redeploy. |
| Runbooks/skills are executable data | `skills` table + `.agents/skills/*` + `knowledge_entities` exist and are *readable* (`get_skills`), but **nothing executes a runbook**. The knowledge is inert w.r.t. action. |
| Auto-act loop consumes classified signals and acts | [`internal/actuator/actuator.go:124`](../internal/actuator/actuator.go) is a literal `"stub execution"` — it marks work done without doing it. |
| "Never ad-hoc SSH" | `pct_exec` **is** ad-hoc SSH (arbitrary shell in a container) and **auto-runs with no approval or classification**. |
Net: the elegant model exists as scaffolding (classifier, policy schema, risk
classes, blast-radius graph walks, skills-as-data, ledger, and the
approval+feedback plumbing hardened in the 2026-07-09/10 sessions), but the live
agent→action path is a bag of tricks that bypasses all of it. Everything added
in the recent LXC-deploy work (`pct_create` + DNS/VMID/template logic) made the
bag *bigger* — reliable, but on the wrong axis.
**Bones that already exist and get reused:** `internal/policy` (classifier +
`computeBlastRadius`/`blast_radius()` SQL), `risk_classes`/`action_risk` tables,
`seeds/policy.yaml`, `executions`/`approvals`/`audit_log`, the MCP SSH machinery,
and the inline approval + execution-status feedback loop (chat polls
`GET /executions/{id}`).
## Target architecture — three layers
### Layer 0 — one general gated primitive (the foundation)
Collapse the fixed enum into essentially one tool:
```
run(target, command, purpose, [declared_risk])
```
- `target` — any host or LXC slug; resolves to SSH (host) or `pct exec` (LXC).
- `command` — arbitrary shell.
- `purpose` — the agent's stated intent (shown to the operator, feeds classify).
- `declared_risk` — optional agent self-assessment.
Every call flows through:
1. **Classify** the command → `read_only | reversible_low | config_mutation |
destructive`. Rule-based:
- read-only **allowlist** (e.g. leading verb in `cat, ls, stat, journalctl,
systemctl status|is-active, pct config|status, df, free, uptime, ip, ss,
docker ps|logs, git status|log`) → `read_only`;
- destructive **denylist** (`rm -rf`, `dd`, `mkfs`, `wipefs`, `pct destroy`,
`qm destroy`, `shutdown`, `reboot`, `> /dev/`, `:(){ :|:& };:`, secret
exfiltration, piping remote scripts to a root shell) → `destructive`;
- anything writing state / installing / editing configs → `config_mutation`;
- **default → escalate** (`config_mutation`) when unsure.
The classifier may only make `declared_risk` **stricter**, never looser
(mirrors "can only lower autonomy, never raise").
2. **Route** (approve-most posture):
- `read_only` → auto-run + ledger, no approval.
- `reversible_low` / `config_mutation` → **operator approval via chat assent**
(v1 gates all state changes; a later posture can auto-run `reversible_low`).
- `destructive` → approval **+ typed confirmation phrase**.
3. **Execute** (existing SSH/`pct exec`), **verify** (optional check command),
**ledger** (`executions` + `audit_log`), **stream feedback to chat** (reuse
the `GET /executions/{id}` polling + `InlineApproval` phases already built).
### Approval by chat assent (replaces the Approve button)
The operator is already authenticated in the chat session, so their words are
the authorization — a separate button is redundant friction. Flow:
- Nomos proposes an action/plan; the gated `run` calls sit in `pending_approval`
(created in the same turn, tied to that turn's `correlation_id`).
- The operator's next message is checked for **assent** ("go ahead", "yes",
"do it", "proceed", "ship it") scoped to *that* proposal. On assent, the
pending approvals from that turn are granted and execute.
- Mechanism: Nomos detects assent and calls an `approve_pending(correlation_id)`
action; the backend flips the linked approvals → the existing
`executeApprovedAction` path runs. The **grant is recorded with the exact
operator message** that constituted assent (audit).
- Guards: assent only applies to approvals from the immediately-preceding turn
(no stale "yes" approving something old); ambiguous replies ("maybe",
"later", a follow-up question) do **not** grant — Nomos re-confirms;
**destructive** actions ignore loose assent and still require the typed
confirmation phrase.
- The inline UI still *shows* the pending action and its classification (so the
operator sees what they're assenting to) and reflects the grant — but the
primary path is "say yes," with the button demoted to an optional affordance.
Layer 0 alone delivers "the agent can attempt anything; state changes are gated."
### Layer 1 — runbooks as executable data (reliability without rigidity)
The hard-won procedures become **runbooks in the knowledge DB**, retrieved and
executed step-by-step via Layer 0 — not frozen Go:
- `pct_create` + its DNS-self-heal / VMID-collision / template-resolution /
locale logic becomes the canonical **"provision LXC" runbook** (parametric
steps the agent fills in and runs through `run`). The reliability survives as
documented, reusable steps rather than a compiled handler.
- New capability = **new runbook (data)**, no redeploy.
- Keep a *small* set of mechanical helpers where a shell step is genuinely
fiddly (e.g. "pick a free cluster VMID"), exposed as callable sub-tools — but
the flow is agent-driven, not enum-driven.
This is the crucial **both/and**: the general primitive is the unlimited escape
hatch; curated runbooks are the reliable fast-path so the agent doesn't
re-derive DNS/VMID/docker every time (the exact thing that failed repeatedly in
the 2026-07-09 sessions).
### Layer 2 — learning closes the loop
Successful ad-hoc `run` sequences get promoted into runbooks/patterns (the
`learning` engine + `skills` table already exist for this); the failure ledger
informs retries. The system grows more capable **as data**, not as code.
## Layer 3 — Observability: watch the system come alive
The user must *see* the OODA loop working, not just trust it. Four surfaces,
built on data the loop already produces (`executions`, `audit_log`, `signals`,
`skills`, `knowledge_entities`) — the job is to make it visible, live, and
legible, not to invent new telemetry.
**1. Live action feed (in the chat turn).** Every `run` renders a card as it
happens: `target` · `purpose` · **risk badge** (green read-only / amber
config / red destructive) · status (queued → running → ok/failed) · collapsible
output. Streams in real time (SSE, extend the existing execution-status feed).
The operator watches Nomos *work*, step by step, with the reasoning (`purpose`)
and the classifier's verdict on every step.
**2. "What this session did" digest.** At the end of a task/turn, a summary
card: N commands (X auto / Y assented / Z denied), entities changed (linked),
signals resolved, and **knowledge created** — new/updated runbooks, patterns
promoted, notes written — each linked to its record. This is the "what did the
agent actually change and learn" answer in one glance.
**3. The learning view — "the system is getting better."** A dedicated page:
runbooks and their **success-rate trend**, newly promoted skills, pattern
confidence (Wilson bounds already computed by the learning engine), recent
auto-acts that succeeded unattended, and a **capability timeline** ("2026-07-11:
learned to deploy Compose stacks; success 4/4"). Growth made tangible.
**4. Global activity/ledger stream.** A live feed of every action across the
fleet — command, target, classification, decision (auto / assented-by-whom),
outcome — the audit log rendered as a heartbeat. Filterable by entity, risk,
outcome.
These reuse existing tables; the work is API endpoints + SSE fan-out + Svelte
views, plus writing knowledge-creation events into the ledger so the digest has
something to show.
## Safety model (the whole point of the gate)
- **Default-escalate.** Nothing is *forbidden*; risky things need the operator's
"yes." Unknown/unparseable risk → approval.
- **Hard denylist** for catastrophic patterns → always typed confirmation, even
if the agent declared them safe.
- **Blast radius at approval time** — graph walk (`blast_radius()` exists):
"this restarts caddy → 8 downstream services."
- **Preview / dry-run** where the command supports it.
- **Kill-switch** (`global.auto_act`, per-target `never_auto_act.*`) already
exists; extend to a global "require approval for everything" flip.
- **Full audit ledger** — every command, its classification, decision, actor,
output. Non-negotiable.
- **Scope guards** — resolve `target` to a real entity first; refuse commands
against `destroyed`/unknown targets; cap output size (already done).
## Honest risks / tradeoffs
- Trades a small vetted surface (5 actions) for arbitrary root across the fleet,
LLM-driven, gated only by classifier + approval. Classifying arbitrary shell
perfectly is impossible; **default-escalate + hard denylist + always-on audit**
is the mitigation, not perfect classification.
- Approve-most means more operator clicks initially. Acceptable while trust is
built; the posture is a config knob, not a rewrite.
- Runbook-as-data can drift from reality like any doc; the ledger + verify step
+ learning loop are the correction mechanism.
## Migration path (incremental, each step shippable)
1. **Command classifier** — extend `internal/policy` with
`ClassifyCommand(cmd, declaredRisk) → riskClass` (allowlist/denylist/default-
escalate + can-only-escalate rule). Unit-tested against a corpus of safe /
mutating / catastrophic commands.
2. **`run` tool** — new MCP tool routing classify → gate → execute → the
existing feedback path. Ship alongside the current tools (no removal yet).
3. **Live action feed (UI)** — render each `run` as a streaming card in chat:
purpose, target, risk badge, status, output. This is the first "come alive"
win and validates the SSE fan-out.
4. **Chat-assent approval** — assent detection scoped to the last turn's
`correlation_id` → `approve_pending`; grant records the operator's message;
destructive still needs the typed phrase. Demote the Approve button.
5. **Approval context** — surface risk class + blast radius + purpose inline so
the operator sees what they're assenting to.
6. **Session digest + activity stream (UI)** — "what this session did / created"
card and the global ledger feed; write knowledge-creation events to the
ledger so there's something to show.
7. **Runbook execution** — a "provision LXC" runbook (ports the current
`pct_create` logic) executed via `run`; validate parity with today's handler.
8. **Learning view (UI)** — runbook success-rate trends, promoted skills,
capability timeline.
9. **Retire the enum** — convert remaining hard-coded actions to runbooks; make
`request_execution` a thin deprecated alias or remove it.
10. **Revive auto-act** — replace the actuator stub, reusing the *same*
classifier for the Observe→Act direction (signals), still approve-most.
## Verification
- Classifier corpus test: read-only commands auto-pass; a set of known
catastrophic commands always route to destructive+confirmation; ambiguous
commands escalate. No command auto-runs that mutates state.
- End-to-end: operator asks Nomos a novel task **not** in the old enum (e.g.
"tail caddy's error log and restart it if it's flapping"); Nomos composes
`run` calls; read-only steps auto-run and **stream as live cards**; the restart
gates; the operator types "go ahead" and the restart executes (no button);
ledger records each command + classification + the assent message.
- Observability: the session ends with a digest listing what ran, what changed,
and any knowledge created; the learning view shows the run's contribution.
- Parity: "provision an LXC with a service" via the runbook path matches the
reliability proven for the `pct_create` handler (free VMID, DNS, install,
verify), then destroy.
## Open questions for the operator
- **Reversible-low posture:** keep gating restarts/syncs in v1 (chosen), or
auto-run them once the classifier is trusted?
- **Confirmation phrase:** per-action typed phrase for destructive, or a global
one? (Assent covers non-destructive; destructive keeps the typed phrase.)
- **Assent detection:** rule/keyword match, or let the model judge assent (with
a re-confirm on ambiguity)? How strict — does "yeah do the restart but not the
upgrade" partially grant?
- **Runbook authorship:** operator-authored only, or may Nomos propose new
runbooks (subject to approval) from successful ad-hoc sequences?
- **Blast-radius threshold:** should a large blast radius force approval even for
otherwise-reversible actions?