Files
oikos/plans/2026-08-16-dsh-as-agent-replace-nomos.md
dtoro eca81ae9af feat: dsh-gate — auto-run non-destructive mutations when no session ID
PolicyService.route now auto-runs config_mutation and reversible_low when
sessionID is empty (dsh sidecar path), routing as 'dsh-gate'. Destructive
still queues. This eliminates the two-layer consent issue: dsh pre-execute
is the sole consent gate, oikos no longer queues dsh-originated mutations.

Also updates the dsh plan with Phase 1-3 completion status and the
architectural decision that _session_id injection is permanently deferred.

v0.38.0
2026-08-16 17:03:27 +02:00

264 lines
21 KiB
Markdown

# dsh-as-agent: replace nomos with DeepSeek Harness
**Date:** 2026-08-16
**Status:** Active
**Scope:** `cmd/nomos/` → dsh sidecar; oikos stays as Go backend behind MCP
## 1. Summary
Replace the nomos agent (`cmd/nomos/`, ~5,500 LOC) with [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (dsh), a TypeScript/Cordis agent harness where everything is a plugin — model adapters, tool registry, agent loop, session log, Web UI. dsh runs as a Node.js sidecar alongside the oikos API, connecting via the existing MCP interface (67+ tools). Custom dsh plugins bridge oikos's Postgres-backed session model, policy engine, and approval gating.
oikos-web (Svelte 5 SPA) is replaced by dsh's built-in Web UI. The Go backend remains untouched — dsh is purely an agent/UI replacement.
## 2. Session persistence: Postgres vs dsh SQLite
dsh ships with `@deepseek-ai/dsh-session` backed by JSONL or SQLite (event-sourced log). oikos uses flat Postgres tables (`agent_sessions`, `agent_messages`, `agent_activity`, `session_plan_steps`, etc.) with ~2,200 lines of domain logic in `internal/nomos/session/store.go`.
| | dsh SQLite (default) | oikos Postgres |
|---|---|---|
| **Model** | Event-sourced: every event appended, `deriveMessages()` projects model history | Flat: typed tables, aggregate columns (message_count, tool_call_count), precomputed views |
| **Tightness with oikos data** | dsh owns session data in isolation; oikos backend cannot JOIN across sessions→entities | Session data lives in oikos Postgres alongside entities, executions, knowledge, events — one FK graph |
| **Cross-cutting queries** | dsh would need its own API for "all sessions touching host X" | `SELECT ... FROM agent_messages JOIN entities ...` works directly — no bridge |
| **Auto-upsert knowledge** | dsh would replicate nomos's `autoUpsertKnowledge` logic | Direct `INSERT INTO knowledge_entities` in the same DB — atomic, no network hop |
| **Agent activity/audit** | dsh would replicate `agent_activity` table writes | Already exists: `agent_activity` with entity FK, tool_name, success, duration, tokens |
| **Plan + execution linking** | dsh would replicate `nomos_plan_executions` join table | Already exists: `executions``session_plan_steps``agent_sessions` |
| **dsh ecosystem compatibility** | Full — dsh's event-sourced model, built-in compaction, fork, replay, persistence seams all work out of the box | Partial — must write a custom persistence plugin implementing dsh's `session-persistence` seam against Postgres |
| **Session events (chunks, boundaries)** | dsh stores raw `assistant/chunk` and `turn/start`/`turn/end` events for faithful replay | oikos stores only the rolled-up `assistant`/`tool` messages — loses per-chunk granularity |
| **Migration cost** | None — dsh owns its storage | Medium — must write the Postgres persistence plugin (~1 week) |
| **DB schema churn** | None | Adds migration files for dsh's event-sourced log format alongside existing flat tables |
**Verdict:** Postgres is the right choice. The tight coupling with entities, knowledge, executions, and the event bus (`observability.Event` via PG `NOTIFY`) is too valuable to sever — it's what makes oikos oikos rather than a generic agent host. The cost is writing a custom `dsh-session-persistence-postgres` plugin that maps dsh's event-sourced `SessionEvent` log onto Postgres rows, while preserving enough granularity for dsh's `deriveMessages()` to reconstruct model history faithfully. The oikos flat tables (`agent_sessions`, `agent_messages`) become projections/views over the event log, maintained for backward compat with the REST API.
## 3. Architecture
```
┌─────────────────────────────────────────┐ MCP (JSON-RPC over HTTP)
│ dsh (Node.js) │ ◄────────────────────────────┐
│ │ │
│ ┌───────────────────────────────────┐ │ │
│ │ dsh-base bundle │ │ │
│ │ - dsh-agent-loop (turn/step) │ │ │
│ │ - dsh-llm-deepseek (model) │ │ │
│ │ - dsh-tools (tool pipeline) │ │ │
│ │ - dsh-session (event log) │ │ │
│ │ - dsh-interaction (approvals) │ │ │
│ │ - dsh-web-app (built-in UI) │ │ │
│ └───────────────────────────────────┘ │ │
│ │ │
│ ┌───────────────────────────────────┐ │ │
│ │ oikos dsh plugins │ │ │
│ │ │ │ │
│ │ @oikos/dsh-mcp-tools │──┤ tools/list + tools/call │
│ │ → discovers 67+ tools via MCP │ │ to oikos MCP server │
│ │ → ctx.tools.register() each │ │ │
│ │ │ │ │
│ │ @oikos/dsh-policy │──┤ classify_command / │
│ │ → tools/pre-execute listener │ │ preflight MCP tools │
│ │ → calls oikos classification │ │ │
│ │ → returns allow/deny/ask │ │ │
│ │ │ │ │
│ │ @oikos/dsh-session-pg │──┤ INSERT/UPDATE/SELECT │
│ │ → implements session- │ │ on oikos Postgres │
│ │ persistence seam vs Postgres │ │ │
│ │ │ │ │
│ │ @oikos/dsh-task-tools │ │ (same Postgres pool) │
│ │ → set_goal, propose_plan, etc. │──┤ │
│ │ → writes oikos tables directly │ │ │
│ │ → emits oikos observability │ │ │
│ │ Events for SSE fan-out │ │ │
│ │ │ │ │
│ │ @oikos/dsh-experiences │ │ │
│ │ → homelab-specific UI nodes │ │ │
│ │ and workflows │ │ │
│ └───────────────────────────────────┘ │ │
│ │ │
│ dsh Web UI (replaces oikos-web) │ │
│ → Serves at :3080 (dsh default) │ │
│ → oikos pages migrated as dsh │ │
│ ConversationNodes + custom views │ │
└─────────────────────────────────────────┘ │
┌───────────────────────────────────────────────────────────────────────┤
│ oikos (Go service — unchanged) │
│ │
│ ┌──────────────────────┐ ┌──────────────────────────┐ │
│ │ MCP server: 67+ tools │ │ REST API (chi) │ │
│ │ - Entity, Ops, │ │ - /api/v1/entities │ │
│ │ Knowledge, Analysis │ │ - /api/v1/executions │ │
│ └──────────────────────┘ │ - /api/v1/knowledge │ │
│ │ - health, metrics, etc. │ │
│ ┌──────────────────────┐ └──────────────────────────┘ │
│ │ Policy engine │ │
│ │ (internal/policy) │ ┌──────────────────────────┐ │
│ └──────────────────────┘ │ Postgres │ │
│ │ - entities, relationships │ │
│ ┌──────────────────────┐ │ - executions, signals │ │
│ │ Scheduler + probes │ │ - agent_sessions, messages│ │
│ └──────────────────────┘ │ - knowledge_entities │ │
│ │ - events (SSE NOTIFY) │ │
│ ┌──────────────────────┐ │ - agent_activity │ │
│ │ Secrets (Infisical) │ └──────────────────────────┘ │
│ └──────────────────────┘ │
└──────────────────────────────────────────────────────────────────────┘
```
## 4. Phases
### Phase 1: Scaffold + MCP tool bridge ~~(2 weeks)~~ DONE
**Status:** Complete. 8/8 golden evals passing.
**What was built:**
- `@deepseek-ai/dsh-mcp-client` discovery of all 67+ oikos MCP tools
- `@deepseek-ai/dsh-llm-deepseek` model adapter via OpenRouter
- Golden eval suite at `packages/oikos/evals/src/golden.test.ts`
- Bundle at `packages/oikos/bundle/cordis.patch.yml`
- Plugin packages inside deepseek-harness workspace at `packages/oikos/`
**Known limitation:** `_session_id` cannot be injected into MCP tool call args — dsh deep-freezes args before pre-execute hooks fire, tool definitions have no interceptor mechanism, and MCP protocol has no per-call metadata. This is accepted as a permanent architectural constraint (see Phase 3 note below).
### Phase 2: Postgres session persistence ~~(1.5 weeks)~~ DONE
**Status:** Complete. Hybrid approach — dsh SQLite owns the event-sourced log; thin Postgres mirror for cross-cutting queries.
**What was built:**
- `@deepseek-ai/dsh-oikos-session-summary` plugin (`packages/oikos/session-summary/`)
- Mirrors `session/created``agent_sessions` INSERT, `session/disposed` → UPDATE status/outcome/closed_at
- Tracks `session/title`, `turn/start`, `turn/end`, message counts for status transitions
- UUID v5 deterministic mapping from dsh `session-<n>` IDs to oikos UUID `agent_sessions.id`
**What was deferred:**
- Full `session-persistence` seam replacement (dsh SQLite → Postgres) — hybrid mirror is sufficient for oikos cross-cutting queries
- `@oikos/dsh-task-tools` (set_goal, propose_plan, etc.) — nomos task model not yet ported
### Phase 3: Policy bridge ~~(1 week)~~ DONE
**Status:** Complete. Consent window end-to-end flow working.
**What was built:**
- `@deepseek-ai/dsh-oikos-mcp-scope` plugin (`packages/oikos/mcp-scope/`)
- `tools/pre-execute` waterfall: read-only → allow, mutation → check consent window → ask if no window
- `tools/post-execute` auto-approve: detects oikos "requires approval" response → calls `decide_approval` HTTP API → replaces result text
- Consent window: `approval/decided` event with `allowed-once` opens 30-min assent window in Postgres `autonomy_settings`
- Auto-expiry cleanup (1-hour interval) for consent/destructive windows
- Go-side fixes: `decide_approval` token mismatch, `X-Oikos-Session-Id` header fallback, LIKE-based assent query
**Architecture note — two-layer consent:**
Because `_session_id` cannot be passed to oikos MCP, the consent flow operates in two layers:
1. **dsh layer** (mcp-scope pre-execute): checks Postgres assent window by session UUID → allows without asking operator again
2. **oikos layer** (Go classifyAndGate): always sees empty `_session_id` → queues execution → post-execute auto-approve detects the queued result and calls the oikos `decide_approval` HTTP API
This is the permanent solution — not a workaround. The dsh layer provides the operator consent UX; the post-execute layer bridges the gap to oikos's execution pipeline.
### Phase 4: UI migration (3-4 weeks) — IN PROGRESS
**Goal:** dsh Web UI replaces oikos-web.
**Prerequisites:**
- [x] Phase 1-3 complete
- [x] dsh running at http://127.0.0.1:3080 with all oikos MCP tools
- [x] Consent/approval flow working end-to-end
- [ ] Commit dsh-harness plugin changes (packages/oikos/ untracked)
1. **dsh Web UI basics**
- dsh ships its own Web UI: session list, chat window with tool cards, assistant chunks, turn/step boundaries
- No changes needed for basic agent chat — it works out of the box
2. **Custom ConversationNodes for oikos pages**
- **Entity Graph page** — reimplement sigma.js graph as a dsh Web Client plugin
- ConversationNode listens for tool/call events, renders entity graph
- Health/Type color mode toggle, filter presets (All, Problems, Infra)
- Port `EntityGraph.svelte`'s logic to a dsh conversation node
- **Operations page** — execution list, approval management
- Use dsh's existing `interaction` UI for approvals
- Custom node for execution history + systemctl status
- **Knowledge page** — wiki browser, search, quick-open
- dsh already has `search_knowledge` tool; add a Knowledge conversation node
- Port `WikiTree`, `WikiReader`, `WikiOverview` from oikos-web
- **Signals page** — signal list, ack/mute/resolve
- Custom node reading from oikos REST API (via dsh `agent.inject` or API call)
- **Overview/Dashboard** — fleet summary, health counts
- dsh `get_health_summary` already exists; render as dashboard cards
- **Config page** — API token, server URL, theme settings
- dsh has `settings` and `credentials` seams; hook into them
- **Desktop shell / mascot** — app launcher, dock, taskbar, Cluck mascot
- dsh has no desktop paradigm — either skip the shell or implement as a ConversationNode
- Mascot can be ported as a persistent UI element
3. **Route mapping**
| oikos-web page | dsh equivalent |
|---|---|
| Overview.svelte | Custom dashboard ConversationNode |
| EntityGraph.svelte | Custom entity-graph ConversationNode |
| Ops.svelte | Custom operations ConversationNode |
| Signals.svelte | Custom signals ConversationNode |
| Knowledge.svelte / KnowledgeBase.svelte | Custom knowledge ConversationNode |
| Config.svelte | dsh settings/credentials |
| Chat session | Built-in dsh chat window |
| Learning.svelte | Custom learning ConversationNode |
| AppStore.svelte | Custom app-store ConversationNode |
4. **CSS theme migration**
- oikos uses dark terminal aesthetic (cyberspace theme, amber/green, dithered images)
- dsh has its own light/dark theme — customize via CSS overrides in the profile
- Port the GlyphIndicator, MascotLayer, and other visual signatures
**Check:** All major oikos-web pages have a functional equivalent in dsh UI. Entity graph renders with force layout and health coloring.
### Phase 5: Experiences as plugins (ongoing)
With the bridge complete, "experiences" are standard dsh plugins registered in the profile:
| Plugin | What it does |
|---|---|
| `@oikos/dsh-incident-response` | Guided workflow: detect signal → classify → `run` remediation → verify → document with `upsert_knowledge`. Uses dsh `plan-mode` for structured steps. |
| `@oikos/dsh-infra-deploy` | Provision LXCs with blast radius visualization. Pre-flight check via `get_blast_radius`, then step-by-step `run` with approval gates. |
| `@oikos/dsh-knowledge-autosync` | Background `ctx.jobs` that periodically audits knowledge gaps (orphan docs, stale entities) and suggests upserts. |
| `@oikos/dsh-session-review` | Port the `session-review` skill from `.agents/skills/` to a dsh tool: given a session ID, analyze transcripts, compare objective to outcome, propose fixes. |
| `@oikos/dsh-fleet-dashboard` | Real-time fleet health with drill-down. Uses oikos SSE event stream + dsh Web Client custom rendering. |
Each plugin:
- Registers tools on `ctx.tools` (model-visible capabilities)
- Registers ConversationNodes on the Web Client (UI components)
- Listens on `agent/*` or `session/event` for reactive behavior
- Is independently versioned and hot-loadable via Cordis
## 5. Deleted code
On completion of Phases 1-3, the following oikos code is decommissioned:
- `cmd/nomos/` — entire directory (~5,500 LOC): agent.go, server.go, mcp.go, store.go (the old flat store), assent.go, continue.go, tasks.go, turngate.go, messagequeue.go, retrycap.go, plus tests
- `nomos/` — SOUL.md, config.yaml, skills/
- `internal/nomos/session/` — moved to dsh plugin, but the domain types and some logic may be extracted into a shared `oikos-dsh` npm package
- `internal/httpapi/` chat-related endpoints — replaced by dsh's own agent session endpoints
- `compose/web/` — web service in docker-compose (served oikos-web SPA)
- `desktop/` — Wails desktop wrapper (dsh Web UI is a PWA, no native wrapper needed)
The following oikos code stays:
- `internal/httpapi/` — REST API for entities, executions, knowledge, signals, health
- `internal/mcp/` — the 67+ MCP tools (now serving dsh instead of nomos)
- `internal/policy/` — risk classification engine
- `internal/scheduler/` — health checks, metrics, probes
- `internal/secrets/` — Infisical/SOPS integration
- `internal/nomos/assent/`, `internal/nomos/session/` domain types (may be extracted to shared package)
## 6. Migration path
The cutover is a rolling deployment:
1. **Deploy dsh alongside nomos** — both agent runtimes run in parallel during development. `compose/dsh/` joins the docker-compose stack.
2. **Port the UI incrementally** — dsh UI and oikos-web coexist on different ports: dsh on `:3080`, oikos-web on `:3000`. The Caddy reverse proxy routes `/chat/*` and `/` to dsh during testing.
3. **Switch the default route** — once dsh passes all golden evals and the UI covers the main pages, Caddy routes all traffic to dsh. oikos-web becomes available at `/legacy` during the transition.
4. **Cleanup** — remove `cmd/nomos/`, `compose/web/`, `oikos-web` repo (or archive).
## 7. Risks
| Risk | Mitigation |
|---|---|
| **dsh breaking changes** — dev preview, no semver | Pin a specific git commit + `pnpm-lock.yaml`. Pin in VERSION file. Stretch: fork the core packages we depend on. |
| **Session persistence bridge lag** — dsh expects event-sourced model, oikos has flat tables | Accept dual-write during migration. The `session_event_log` table feeds dsh's `deriveMessages()`; legacy `agent_messages` stays for REST API backward compat until all consumers migrate. |
| **UI migration scope** — entity graph, desktop shell, mascot are non-trivial ports | Start with chat + operations (90% of daily use). Entity graph and mascot come after. The old oikos-web stays readable during transition. |
| **Golden eval regressions** — subtle behavioral differences between nomos and dsh agent loops | Run evals in CI on every dsh change. Nomos stays deployed until evals pass at parity. |
| **Team TS inexperience** — you said TS is OK, but ramp-up for Go developers | Start with small plugins (MCP bridge is ~200 LOC). The dsh extension cookbook is well-documented. |
| **Performance** — every tool call crosses TS → HTTP → Go | Same architecture as nomos (which also crossed HTTP). Latency is the same. The MCP server is fast (no serialization overhead beyond JSON). |
| **oikos-web features not supported by dsh UI** — desktop shell, window management, mascot | Assess during Phase 4. If the desktop paradigm is essential, implement it as a dsh conversation node (which can render any HTML/CSS) rather than maintaining two UIs. |