PolicyService.route now auto-runs config_mutation and reversible_low when sessionID is empty (dsh sidecar path), routing as 'dsh-gate'. Destructive still queues. This eliminates the two-layer consent issue: dsh pre-execute is the sole consent gate, oikos no longer queues dsh-originated mutations. Also updates the dsh plan with Phase 1-3 completion status and the architectural decision that _session_id injection is permanently deferred. v0.38.0
264 lines
21 KiB
Markdown
264 lines
21 KiB
Markdown
# dsh-as-agent: replace nomos with DeepSeek Harness
|
|
|
|
**Date:** 2026-08-16
|
|
**Status:** Active
|
|
**Scope:** `cmd/nomos/` → dsh sidecar; oikos stays as Go backend behind MCP
|
|
|
|
## 1. Summary
|
|
|
|
Replace the nomos agent (`cmd/nomos/`, ~5,500 LOC) with [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (dsh), a TypeScript/Cordis agent harness where everything is a plugin — model adapters, tool registry, agent loop, session log, Web UI. dsh runs as a Node.js sidecar alongside the oikos API, connecting via the existing MCP interface (67+ tools). Custom dsh plugins bridge oikos's Postgres-backed session model, policy engine, and approval gating.
|
|
|
|
oikos-web (Svelte 5 SPA) is replaced by dsh's built-in Web UI. The Go backend remains untouched — dsh is purely an agent/UI replacement.
|
|
|
|
## 2. Session persistence: Postgres vs dsh SQLite
|
|
|
|
dsh ships with `@deepseek-ai/dsh-session` backed by JSONL or SQLite (event-sourced log). oikos uses flat Postgres tables (`agent_sessions`, `agent_messages`, `agent_activity`, `session_plan_steps`, etc.) with ~2,200 lines of domain logic in `internal/nomos/session/store.go`.
|
|
|
|
| | dsh SQLite (default) | oikos Postgres |
|
|
|---|---|---|
|
|
| **Model** | Event-sourced: every event appended, `deriveMessages()` projects model history | Flat: typed tables, aggregate columns (message_count, tool_call_count), precomputed views |
|
|
| **Tightness with oikos data** | dsh owns session data in isolation; oikos backend cannot JOIN across sessions→entities | Session data lives in oikos Postgres alongside entities, executions, knowledge, events — one FK graph |
|
|
| **Cross-cutting queries** | dsh would need its own API for "all sessions touching host X" | `SELECT ... FROM agent_messages JOIN entities ...` works directly — no bridge |
|
|
| **Auto-upsert knowledge** | dsh would replicate nomos's `autoUpsertKnowledge` logic | Direct `INSERT INTO knowledge_entities` in the same DB — atomic, no network hop |
|
|
| **Agent activity/audit** | dsh would replicate `agent_activity` table writes | Already exists: `agent_activity` with entity FK, tool_name, success, duration, tokens |
|
|
| **Plan + execution linking** | dsh would replicate `nomos_plan_executions` join table | Already exists: `executions` ↔ `session_plan_steps` ↔ `agent_sessions` |
|
|
| **dsh ecosystem compatibility** | Full — dsh's event-sourced model, built-in compaction, fork, replay, persistence seams all work out of the box | Partial — must write a custom persistence plugin implementing dsh's `session-persistence` seam against Postgres |
|
|
| **Session events (chunks, boundaries)** | dsh stores raw `assistant/chunk` and `turn/start`/`turn/end` events for faithful replay | oikos stores only the rolled-up `assistant`/`tool` messages — loses per-chunk granularity |
|
|
| **Migration cost** | None — dsh owns its storage | Medium — must write the Postgres persistence plugin (~1 week) |
|
|
| **DB schema churn** | None | Adds migration files for dsh's event-sourced log format alongside existing flat tables |
|
|
|
|
**Verdict:** Postgres is the right choice. The tight coupling with entities, knowledge, executions, and the event bus (`observability.Event` via PG `NOTIFY`) is too valuable to sever — it's what makes oikos oikos rather than a generic agent host. The cost is writing a custom `dsh-session-persistence-postgres` plugin that maps dsh's event-sourced `SessionEvent` log onto Postgres rows, while preserving enough granularity for dsh's `deriveMessages()` to reconstruct model history faithfully. The oikos flat tables (`agent_sessions`, `agent_messages`) become projections/views over the event log, maintained for backward compat with the REST API.
|
|
|
|
## 3. Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────┐ MCP (JSON-RPC over HTTP)
|
|
│ dsh (Node.js) │ ◄────────────────────────────┐
|
|
│ │ │
|
|
│ ┌───────────────────────────────────┐ │ │
|
|
│ │ dsh-base bundle │ │ │
|
|
│ │ - dsh-agent-loop (turn/step) │ │ │
|
|
│ │ - dsh-llm-deepseek (model) │ │ │
|
|
│ │ - dsh-tools (tool pipeline) │ │ │
|
|
│ │ - dsh-session (event log) │ │ │
|
|
│ │ - dsh-interaction (approvals) │ │ │
|
|
│ │ - dsh-web-app (built-in UI) │ │ │
|
|
│ └───────────────────────────────────┘ │ │
|
|
│ │ │
|
|
│ ┌───────────────────────────────────┐ │ │
|
|
│ │ oikos dsh plugins │ │ │
|
|
│ │ │ │ │
|
|
│ │ @oikos/dsh-mcp-tools │──┤ tools/list + tools/call │
|
|
│ │ → discovers 67+ tools via MCP │ │ to oikos MCP server │
|
|
│ │ → ctx.tools.register() each │ │ │
|
|
│ │ │ │ │
|
|
│ │ @oikos/dsh-policy │──┤ classify_command / │
|
|
│ │ → tools/pre-execute listener │ │ preflight MCP tools │
|
|
│ │ → calls oikos classification │ │ │
|
|
│ │ → returns allow/deny/ask │ │ │
|
|
│ │ │ │ │
|
|
│ │ @oikos/dsh-session-pg │──┤ INSERT/UPDATE/SELECT │
|
|
│ │ → implements session- │ │ on oikos Postgres │
|
|
│ │ persistence seam vs Postgres │ │ │
|
|
│ │ │ │ │
|
|
│ │ @oikos/dsh-task-tools │ │ (same Postgres pool) │
|
|
│ │ → set_goal, propose_plan, etc. │──┤ │
|
|
│ │ → writes oikos tables directly │ │ │
|
|
│ │ → emits oikos observability │ │ │
|
|
│ │ Events for SSE fan-out │ │ │
|
|
│ │ │ │ │
|
|
│ │ @oikos/dsh-experiences │ │ │
|
|
│ │ → homelab-specific UI nodes │ │ │
|
|
│ │ and workflows │ │ │
|
|
│ └───────────────────────────────────┘ │ │
|
|
│ │ │
|
|
│ dsh Web UI (replaces oikos-web) │ │
|
|
│ → Serves at :3080 (dsh default) │ │
|
|
│ → oikos pages migrated as dsh │ │
|
|
│ ConversationNodes + custom views │ │
|
|
└─────────────────────────────────────────┘ │
|
|
│
|
|
┌───────────────────────────────────────────────────────────────────────┤
|
|
│ oikos (Go service — unchanged) │
|
|
│ │
|
|
│ ┌──────────────────────┐ ┌──────────────────────────┐ │
|
|
│ │ MCP server: 67+ tools │ │ REST API (chi) │ │
|
|
│ │ - Entity, Ops, │ │ - /api/v1/entities │ │
|
|
│ │ Knowledge, Analysis │ │ - /api/v1/executions │ │
|
|
│ └──────────────────────┘ │ - /api/v1/knowledge │ │
|
|
│ │ - health, metrics, etc. │ │
|
|
│ ┌──────────────────────┐ └──────────────────────────┘ │
|
|
│ │ Policy engine │ │
|
|
│ │ (internal/policy) │ ┌──────────────────────────┐ │
|
|
│ └──────────────────────┘ │ Postgres │ │
|
|
│ │ - entities, relationships │ │
|
|
│ ┌──────────────────────┐ │ - executions, signals │ │
|
|
│ │ Scheduler + probes │ │ - agent_sessions, messages│ │
|
|
│ └──────────────────────┘ │ - knowledge_entities │ │
|
|
│ │ - events (SSE NOTIFY) │ │
|
|
│ ┌──────────────────────┐ │ - agent_activity │ │
|
|
│ │ Secrets (Infisical) │ └──────────────────────────┘ │
|
|
│ └──────────────────────┘ │
|
|
└──────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## 4. Phases
|
|
|
|
### Phase 1: Scaffold + MCP tool bridge ~~(2 weeks)~~ DONE
|
|
|
|
**Status:** Complete. 8/8 golden evals passing.
|
|
|
|
**What was built:**
|
|
- `@deepseek-ai/dsh-mcp-client` discovery of all 67+ oikos MCP tools
|
|
- `@deepseek-ai/dsh-llm-deepseek` model adapter via OpenRouter
|
|
- Golden eval suite at `packages/oikos/evals/src/golden.test.ts`
|
|
- Bundle at `packages/oikos/bundle/cordis.patch.yml`
|
|
- Plugin packages inside deepseek-harness workspace at `packages/oikos/`
|
|
|
|
**Known limitation:** `_session_id` cannot be injected into MCP tool call args — dsh deep-freezes args before pre-execute hooks fire, tool definitions have no interceptor mechanism, and MCP protocol has no per-call metadata. This is accepted as a permanent architectural constraint (see Phase 3 note below).
|
|
|
|
### Phase 2: Postgres session persistence ~~(1.5 weeks)~~ DONE
|
|
|
|
**Status:** Complete. Hybrid approach — dsh SQLite owns the event-sourced log; thin Postgres mirror for cross-cutting queries.
|
|
|
|
**What was built:**
|
|
- `@deepseek-ai/dsh-oikos-session-summary` plugin (`packages/oikos/session-summary/`)
|
|
- Mirrors `session/created` → `agent_sessions` INSERT, `session/disposed` → UPDATE status/outcome/closed_at
|
|
- Tracks `session/title`, `turn/start`, `turn/end`, message counts for status transitions
|
|
- UUID v5 deterministic mapping from dsh `session-<n>` IDs to oikos UUID `agent_sessions.id`
|
|
|
|
**What was deferred:**
|
|
- Full `session-persistence` seam replacement (dsh SQLite → Postgres) — hybrid mirror is sufficient for oikos cross-cutting queries
|
|
- `@oikos/dsh-task-tools` (set_goal, propose_plan, etc.) — nomos task model not yet ported
|
|
|
|
### Phase 3: Policy bridge ~~(1 week)~~ DONE
|
|
|
|
**Status:** Complete. Consent window end-to-end flow working.
|
|
|
|
**What was built:**
|
|
- `@deepseek-ai/dsh-oikos-mcp-scope` plugin (`packages/oikos/mcp-scope/`)
|
|
- `tools/pre-execute` waterfall: read-only → allow, mutation → check consent window → ask if no window
|
|
- `tools/post-execute` auto-approve: detects oikos "requires approval" response → calls `decide_approval` HTTP API → replaces result text
|
|
- Consent window: `approval/decided` event with `allowed-once` opens 30-min assent window in Postgres `autonomy_settings`
|
|
- Auto-expiry cleanup (1-hour interval) for consent/destructive windows
|
|
- Go-side fixes: `decide_approval` token mismatch, `X-Oikos-Session-Id` header fallback, LIKE-based assent query
|
|
|
|
**Architecture note — two-layer consent:**
|
|
Because `_session_id` cannot be passed to oikos MCP, the consent flow operates in two layers:
|
|
1. **dsh layer** (mcp-scope pre-execute): checks Postgres assent window by session UUID → allows without asking operator again
|
|
2. **oikos layer** (Go classifyAndGate): always sees empty `_session_id` → queues execution → post-execute auto-approve detects the queued result and calls the oikos `decide_approval` HTTP API
|
|
|
|
This is the permanent solution — not a workaround. The dsh layer provides the operator consent UX; the post-execute layer bridges the gap to oikos's execution pipeline.
|
|
|
|
### Phase 4: UI migration (3-4 weeks) — IN PROGRESS
|
|
|
|
**Goal:** dsh Web UI replaces oikos-web.
|
|
|
|
**Prerequisites:**
|
|
- [x] Phase 1-3 complete
|
|
- [x] dsh running at http://127.0.0.1:3080 with all oikos MCP tools
|
|
- [x] Consent/approval flow working end-to-end
|
|
- [ ] Commit dsh-harness plugin changes (packages/oikos/ untracked)
|
|
|
|
1. **dsh Web UI basics**
|
|
- dsh ships its own Web UI: session list, chat window with tool cards, assistant chunks, turn/step boundaries
|
|
- No changes needed for basic agent chat — it works out of the box
|
|
|
|
2. **Custom ConversationNodes for oikos pages**
|
|
- **Entity Graph page** — reimplement sigma.js graph as a dsh Web Client plugin
|
|
- ConversationNode listens for tool/call events, renders entity graph
|
|
- Health/Type color mode toggle, filter presets (All, Problems, Infra)
|
|
- Port `EntityGraph.svelte`'s logic to a dsh conversation node
|
|
- **Operations page** — execution list, approval management
|
|
- Use dsh's existing `interaction` UI for approvals
|
|
- Custom node for execution history + systemctl status
|
|
- **Knowledge page** — wiki browser, search, quick-open
|
|
- dsh already has `search_knowledge` tool; add a Knowledge conversation node
|
|
- Port `WikiTree`, `WikiReader`, `WikiOverview` from oikos-web
|
|
- **Signals page** — signal list, ack/mute/resolve
|
|
- Custom node reading from oikos REST API (via dsh `agent.inject` or API call)
|
|
- **Overview/Dashboard** — fleet summary, health counts
|
|
- dsh `get_health_summary` already exists; render as dashboard cards
|
|
- **Config page** — API token, server URL, theme settings
|
|
- dsh has `settings` and `credentials` seams; hook into them
|
|
- **Desktop shell / mascot** — app launcher, dock, taskbar, Cluck mascot
|
|
- dsh has no desktop paradigm — either skip the shell or implement as a ConversationNode
|
|
- Mascot can be ported as a persistent UI element
|
|
|
|
3. **Route mapping**
|
|
| oikos-web page | dsh equivalent |
|
|
|---|---|
|
|
| Overview.svelte | Custom dashboard ConversationNode |
|
|
| EntityGraph.svelte | Custom entity-graph ConversationNode |
|
|
| Ops.svelte | Custom operations ConversationNode |
|
|
| Signals.svelte | Custom signals ConversationNode |
|
|
| Knowledge.svelte / KnowledgeBase.svelte | Custom knowledge ConversationNode |
|
|
| Config.svelte | dsh settings/credentials |
|
|
| Chat session | Built-in dsh chat window |
|
|
| Learning.svelte | Custom learning ConversationNode |
|
|
| AppStore.svelte | Custom app-store ConversationNode |
|
|
|
|
4. **CSS theme migration**
|
|
- oikos uses dark terminal aesthetic (cyberspace theme, amber/green, dithered images)
|
|
- dsh has its own light/dark theme — customize via CSS overrides in the profile
|
|
- Port the GlyphIndicator, MascotLayer, and other visual signatures
|
|
|
|
**Check:** All major oikos-web pages have a functional equivalent in dsh UI. Entity graph renders with force layout and health coloring.
|
|
|
|
### Phase 5: Experiences as plugins (ongoing)
|
|
|
|
With the bridge complete, "experiences" are standard dsh plugins registered in the profile:
|
|
|
|
| Plugin | What it does |
|
|
|---|---|
|
|
| `@oikos/dsh-incident-response` | Guided workflow: detect signal → classify → `run` remediation → verify → document with `upsert_knowledge`. Uses dsh `plan-mode` for structured steps. |
|
|
| `@oikos/dsh-infra-deploy` | Provision LXCs with blast radius visualization. Pre-flight check via `get_blast_radius`, then step-by-step `run` with approval gates. |
|
|
| `@oikos/dsh-knowledge-autosync` | Background `ctx.jobs` that periodically audits knowledge gaps (orphan docs, stale entities) and suggests upserts. |
|
|
| `@oikos/dsh-session-review` | Port the `session-review` skill from `.agents/skills/` to a dsh tool: given a session ID, analyze transcripts, compare objective to outcome, propose fixes. |
|
|
| `@oikos/dsh-fleet-dashboard` | Real-time fleet health with drill-down. Uses oikos SSE event stream + dsh Web Client custom rendering. |
|
|
|
|
Each plugin:
|
|
- Registers tools on `ctx.tools` (model-visible capabilities)
|
|
- Registers ConversationNodes on the Web Client (UI components)
|
|
- Listens on `agent/*` or `session/event` for reactive behavior
|
|
- Is independently versioned and hot-loadable via Cordis
|
|
|
|
## 5. Deleted code
|
|
|
|
On completion of Phases 1-3, the following oikos code is decommissioned:
|
|
|
|
- `cmd/nomos/` — entire directory (~5,500 LOC): agent.go, server.go, mcp.go, store.go (the old flat store), assent.go, continue.go, tasks.go, turngate.go, messagequeue.go, retrycap.go, plus tests
|
|
- `nomos/` — SOUL.md, config.yaml, skills/
|
|
- `internal/nomos/session/` — moved to dsh plugin, but the domain types and some logic may be extracted into a shared `oikos-dsh` npm package
|
|
- `internal/httpapi/` chat-related endpoints — replaced by dsh's own agent session endpoints
|
|
- `compose/web/` — web service in docker-compose (served oikos-web SPA)
|
|
- `desktop/` — Wails desktop wrapper (dsh Web UI is a PWA, no native wrapper needed)
|
|
|
|
The following oikos code stays:
|
|
- `internal/httpapi/` — REST API for entities, executions, knowledge, signals, health
|
|
- `internal/mcp/` — the 67+ MCP tools (now serving dsh instead of nomos)
|
|
- `internal/policy/` — risk classification engine
|
|
- `internal/scheduler/` — health checks, metrics, probes
|
|
- `internal/secrets/` — Infisical/SOPS integration
|
|
- `internal/nomos/assent/`, `internal/nomos/session/` domain types (may be extracted to shared package)
|
|
|
|
## 6. Migration path
|
|
|
|
The cutover is a rolling deployment:
|
|
|
|
1. **Deploy dsh alongside nomos** — both agent runtimes run in parallel during development. `compose/dsh/` joins the docker-compose stack.
|
|
2. **Port the UI incrementally** — dsh UI and oikos-web coexist on different ports: dsh on `:3080`, oikos-web on `:3000`. The Caddy reverse proxy routes `/chat/*` and `/` to dsh during testing.
|
|
3. **Switch the default route** — once dsh passes all golden evals and the UI covers the main pages, Caddy routes all traffic to dsh. oikos-web becomes available at `/legacy` during the transition.
|
|
4. **Cleanup** — remove `cmd/nomos/`, `compose/web/`, `oikos-web` repo (or archive).
|
|
|
|
## 7. Risks
|
|
|
|
| Risk | Mitigation |
|
|
|---|---|
|
|
| **dsh breaking changes** — dev preview, no semver | Pin a specific git commit + `pnpm-lock.yaml`. Pin in VERSION file. Stretch: fork the core packages we depend on. |
|
|
| **Session persistence bridge lag** — dsh expects event-sourced model, oikos has flat tables | Accept dual-write during migration. The `session_event_log` table feeds dsh's `deriveMessages()`; legacy `agent_messages` stays for REST API backward compat until all consumers migrate. |
|
|
| **UI migration scope** — entity graph, desktop shell, mascot are non-trivial ports | Start with chat + operations (90% of daily use). Entity graph and mascot come after. The old oikos-web stays readable during transition. |
|
|
| **Golden eval regressions** — subtle behavioral differences between nomos and dsh agent loops | Run evals in CI on every dsh change. Nomos stays deployed until evals pass at parity. |
|
|
| **Team TS inexperience** — you said TS is OK, but ramp-up for Go developers | Start with small plugins (MCP bridge is ~200 LOC). The dsh extension cookbook is well-documented. |
|
|
| **Performance** — every tool call crosses TS → HTTP → Go | Same architecture as nomos (which also crossed HTTP). Latency is the same. The MCP server is fast (no serialization overhead beyond JSON). |
|
|
| **oikos-web features not supported by dsh UI** — desktop shell, window management, mascot | Assess during Phase 4. If the desktop paradigm is essential, implement it as a dsh conversation node (which can render any HTML/CSS) rather than maintaining two UIs. | |