Files
oikos/plans/2026-08-15-session-audit-arr-improvements.md
dtoro 809c16f6fd
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
docs: add session audit for arr-improvements (2026-08-15)
2026-08-15 18:08:24 +02:00

194 lines
7.9 KiB
Markdown

# Oikos Improvement Plan — Session Audit 2026-08-15
## Session context
A 4-hour session to fix the arr-media automation pipeline on the `arriman` LXC.
The agent:
- Used the **terminal tool** (raw SSH) exclusively — no `run` calls
- Used **Infisical** via `set_secret` (worked well)
- Did **not** consult the knowledge graph before touching entities
- Did **not** register ephemeral entities (nzbhydra2) in the graph
- Did **not** log mutations (Caddy changes, docker-compose edits, DNS records)
- Did **not** write knowledge entries for discoveries made
## Root cause analysis
The agent's default behavior is to use whatever tool provides the fastest path.
Raw SSH (`terminal`) is faster than `mcp__oikos__run` because:
1. **`run` gates behind approval** — even read-only commands on LXCs touching `/etc/` or `/opt/` escalate from `read_only` to `config_mutation` (transport-aware escalation, `server.go:770`). The agent didn't know this and assumed `run` would be slower/blocked.
2. **`run` requires a plan-first gate** — every `run` in a Nomos session checks for a plan before executing (`server.go:795`). The agent had no plan registered.
3. **No docker-exec native support** — common *arr operations (`docker exec <container> curl ...`) don't map cleanly to Oikos targets. The agent would have needed to do `run(target="lxc:arriman", command="docker exec prowlarr curl ...")`, which is verbose and classifies as config_mutation (touching container internals).
4. **No "batch run" or pre-authorization** — the agent made ~200+ SSH calls. With `run` each one would individually classify, deduplicate, create execution rows, and potentially queue for approval. Raw SSH just ran.
5. **Graph drift is invisible at session end** — there's no automatic `audit_knowledge_graph` trigger or summary that says "you changed X, Y, Z — update the graph."
---
## Improvement plan
### P0: Docker-exec execution primitive
**Problem:** The most common operation in *arr management is `docker exec <name> <cmd>`. Currently requires `run(target="lxc:arriman", command="docker exec prowlarr curl ...")`, which is verbose and classifies as config_mutation.
**Proposed:** Add a `docker_exec` tool:
```
mcp__oikos__docker_exec(lxc_slug, container, command, risk)
```
- `lxc_slug`: the LXC entity (e.g. `lxc:arriman`)
- `container`: the Docker container name (e.g. `prowlarr`, `sonarr`)
- `command`: what to run inside the container
- `risk`: self-declared risk (default read_only for curl/status)
Implementation:
- Resolves the LXC slug to SSH host/user via existing `resolveHost`
- Wraps the command as: `docker exec <container> sh -c '<escaped_command>'`
- Classifies with the same `policy.ClassifyCommand` but with a **lower default** (read_only for curl queries, `docker exec <name> cat/config/status`)
- Records a `docker_exec` execution type so the audit trail distinguishes from raw `run`
**Files to change:**
- `internal/mcp/ops_tools.go` — add `dockerexec_tool` registration
- `internal/policy/classify.go` — add docker-exec-specific command patterns
**Estimate:** 1 day
---
### P1: Session-scoped trust (assent window expansion)
**Problem:** Agents make 200+ SSH calls in a session. The approval model gates each one individually, forcing agents to bypass Oikos.
**Proposed:** Expand the existing "assent window" pattern (`server.go:839`) from plan-level to session-level:
1. Agent calls `propose_plan` (already exists) — operator approves
2. All subsequent `run` calls within the same session auto-approve for the scope declared in the plan
3. Commands outside the plan's scope still require new approval
**Implementation:**
- Track the plan's declared scope (entity slugs + risk classes) in `nomos_plan_executions`
- `classifyAndGate` checks: is `sessionID` in an approved plan whose scope covers this target + risk class?
- If yes → auto-execute, record audit entry referencing the plan approval
**Estimate:** 2 days
---
### P2: Post-session drift summary
**Problem:** After 200+ tool calls, the knowledge graph is stale. There's no signal that the agent should update it.
**Proposed:** A new MCP tool `upsert_session_summary`:
```
mcp__oikos__upsert_session_summary(session_id, summary, entities_touched[], mutations[], discoveries[])
```
Called at session end by the agent. Auto-creates:
- Knowledge entries for each discovery
- Entity attribute updates for changed facts
- Relationship records for new edges discovered
- Audit trail entries linking everything to the session
**Implementation:**
- Wraps existing `upsert_knowledge`, `update_entity_attributes`, `create_relationship`
- Batch upsert in a single transaction
- Returns a report: "Created 3 knowledge entries, updated 5 entity attributes, added 2 relationships"
**Estimate:** 1.5 days
---
### P3: Docker-container entity type + drift detection
**Problem:** NZBHydra2 was deployed, used, and removed — the graph never knew it existed. Docker containers on the arriman LXC (prowlarr, sonarr, radarr, lidarr, sabnzbd, etc.) have no entity representation.
**Proposed:** Add a `container` entity type to the ontology:
```yaml
- name: container
label: "Docker Container"
parent: service
attributes_schema:
image: string
port: integer
network_mode: string
```
And extend `discover_infra_drift` to detect Docker-container-to-entity drift:
1. Query `docker ps --format '{{.ID}} {{.Names}} {{.Image}}'` on LXCs with role `docker-host`
2. Compare against `container:*` entities in the DB
3. Report: containers with no entity (ghost) and entities with no running container (missing)
**Files to change:**
- `seeds/ontology.yaml` — add `container` type
- `internal/mcp/discover.go` — add docker drift detection
- `internal/mcp/entity_tools.go` — auto-derive container checks from `monitoring: ["http", "docker"]`
**Estimate:** 2 days
---
### P4: Knowledge-graph-first agent instruction
**Problem:** The session never consulted Oikos for entity topology. The agent's SOUL.md (HERMES.md) doesn't instruct it to do so.
**Proposed:** Add a mandatory step to the homelab agent persona (`/opt/homelab-context/HERMES.md`):
```
## Before any terminal/SSH command
1. Check Oikos knowledge graph: `list_entities(type="lxc")` + `get_entity(slug="...")`
2. If the entity exists → use `run` not raw SSH
3. If it doesn't exist → create it with `create_entity`
4. After changes → update attributes with `update_entity_attributes`
```
**Estimate:** 0.5 days (documentation-only)
---
### P5: DNS management tools
**Problem:** Session had to write custom Python to interact with Technitium API (token redaction, shell escaping). This is common enough to warrant native tools.
**Proposed:** Add DNS management tools:
```
mcp__oikos__add_dns_record(zone, domain, type, value, ttl)
mcp__oikos__delete_dns_record(zone, domain, type, value)
mcp__oikos__list_dns_records(zone, filter)
```
Implementation wraps the Technitium HTTP API behind the MCP layer, handling token authentication automatically via Infisical secrets.
**Estimate:** 1 day
---
## Priority matrix
| # | Item | Effort | Impact | Risk |
|---|---|---|---|---|
| P0 | Docker-exec tool | 1d | High — single biggest friction point | Low |
| P1 | Session-scoped trust | 2d | High — removes approval barrier | Medium (auth model change) |
| P2 | Post-session drift summary | 1.5d | Medium — closes the feedback loop | Low |
| P3 | Container entity + drift | 2d | Medium — fills a blind spot | Low |
| P4 | Agent instruction update | 0.5d | High — behavioral change | None |
| P5 | DNS management tools | 1d | Low — but saves time every session | Low |
## Recommended execution order
1. **P0 + P4** (1.5 days) — Remove the biggest friction point + update instructions
2. **P1** (2 days) — Make Oikos the preferred execution pathway
3. **P2** (1.5 days) — Ensure knowledge doesn't leak out of sessions
4. **P3** (2 days) — Fill the Docker blind spot
5. **P5** (1 day) — DNS nicety
Total: ~8 days of work for a complete session-to-graph feedback loop.