Files
oikos/docs/mbse/README.md
dtoro 55781984c7
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
docs(mbse): add MBSE system model, framework, component and ontology views
Four cross-linked documents under docs/mbse/, structured after Jon Holt's
Systems Engineering Demystified (2nd ed.): Framework = Ontology + Viewpoints,
producing a Model made of Views.

- framework.md — the Ontology (SE meta-concepts + Oikos's domain ontology)
  and an 11-entry Viewpoint catalog (two repeating: Component, Ontology).
- README.md — the Model's 9 concern-based Views (mission, requirements,
  functional/physical architecture, interfaces, behavior, V&V, risk, roadmap).
- components.md — 8 per-component Views going one layer deeper into each
  running part of the system's own internal structure.
- ontology.md — 4 Views on the domain ontology itself: entity type
  hierarchy (split into 9 digestible per-domain diagrams), full relationship
  catalog, lifecycle state machines with their requires: gates, and concrete
  population.

Grounded in direct verification against source (grep/read), not just
existing docs — every finding is graded verified vs. per-research-pass.
Surfaced several real, previously undocumented findings along the way:
the policy kill-switch (global.auto_act/never_auto_act) is checked only by
dead code and an unstarted actuator package, so it doesn't gate the live
run path; internal/actuator and internal/learning are compiled but never
started by any process; the relationship catalog grew from 34 to 47 types
since ADR-0014; and task has no registered lifecycle_defs entry despite
having a documented, code-enforced state machine.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 21:00:14 +02:00

1611 lines
95 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Oikos — System Model
A single Model-Based Systems Engineering (MBSE) view of Oikos, structured
after *Systems Engineering Demystified* (2nd ed., Jon Holt): one underlying
system — the Oikos entity graph, its OODA control loop, and the services
that implement it — represented here through a sequence of **views**, each
a projection of that same model for a different concern (mission,
structure, behavior, requirements, risk, ...). They are kept in one
document deliberately: a change that matters (a new gating rule, a retired
tool) is rarely relevant to only one view, and putting them back-to-back
makes that cross-view impact visible on read, not something you have to
jump files to notice.
This document **is the Model** in Holt's sense — Ontology + Viewpoints
already applied, producing Views. The template that generated it — the
Ontology, the Viewpoint definitions, and why each View below earns its
place — lives separately in **[framework.md](framework.md)**. Read that
first if you're deciding whether to add a View, extending the Ontology, or
just want to understand why the documentation is shaped the way it is
rather than what it currently says.
The nine Views below cut across the system by *concern* (requirements,
behavior, risk...). Two companion cuts exist alongside it: for a
per-component cut — one View per running part of the system, going one
layer deeper into its own internal structure — see
**[components.md](components.md)**; for the domain-specific language every
View here is written in — the full entity/relationship/lifecycle
definitions, not the condensed sketch in framework.md §3 — see
**[ontology.md](ontology.md)**.
Findings below are graded by confidence. **Verified** means checked directly
against source during this documentation pass (`grep`/`Read`, cross-checked
in more than one place). **Per research pass** means gathered by parallel
research agents feeding this document and not independently re-confirmed
line by line — high-confidence, not certain. This distinction matters most
in the [Risk & Safety](#8-risk--safety) and [Roadmap & Traceability](#9-roadmap--traceability)
views, so it's called out again there.
**Read this if:** you want to understand what Oikos *is* and *why* it's
built the way it is — as opposed to the [ADRs](../adr/) (why one specific
decision was made) or [plans/](../../plans/) (what's being built right now,
task by task). This is a synthesis across both, cross-checked directly
against the running code as of **2026-07-16**.
## Views in this model
| # | View | Concern it addresses |
|---|---|---|
| [1](#1-mission-stakeholders--system-context) | Mission, Stakeholders & System Context | Why does this exist, who is it for, what's inside vs. outside the boundary? |
| [2](#2-requirements) | Requirements | What must the system do, traced to source and implementation status? |
| [3](#3-functional-architecture) | Functional Architecture | What functions does the system perform, and which component actually performs each one? |
| [4](#4-physical-architecture) | Physical Architecture | What physically runs, where, on what ports, talking to what? |
| [5](#5-interfaces-icd) | Interfaces (ICD) | Every MCP tool, REST group, SSE event, external protocol |
| [6](#6-behavior) | Behavior | How a task, a signal, an execution, an approval moves through time |
| [7](#7-verification--validation) | Verification & Validation | How do we know it works, and what isn't checked yet? |
| [8](#8-risk--safety) | Risk & Safety | What actually stops this system from doing something dangerous? |
| [9](#9-roadmap--traceability) | Roadmap & Traceability | What's code-real vs. dead code vs. schema-only, and where is this headed? |
If you only read one section: [§8 Risk & Safety](#8-risk--safety) — it
contains the single most consequential finding from this documentation pass
(the policy kill-switch doesn't reach the live execution path) and is the
shortest path to understanding what "gated" actually means in this system
today, versus what the schema implies.
**What this model corrects.** [ADR-0014](../adr/0014-entity-model.md)
already distinguished "code-real" from "schema-only" for the entity model
and OODA loop as of 2026-07-08 — excellent discipline, extended here to
what changed since: the `general-gated-execution` rebuild (the `run`
primitive, command classifier, assent/destructive windows) and the
goal-oriented Task model (plan steps, auto-continuation, the knowledge
loop). It also corrects one specific claim: ADR-0014 and `OIKOS.md`
describe Phase 3 (control loop — scheduler, actuator, learning, classifier,
notifier) as fully "DONE." This pass found that **the actuator and learning
packages are not started by any running process**, and that the classifier
described in the schema (`ClassifySignal`) is dead code — the actually-live
classifier (`ClassifyCommand`) was added later by a different plan and
lives in a different function. Full evidence in
[§9.2](#92-code-real--dead-code--schema-only-matrix).
---
## 1. Mission, Stakeholders & System Context
*Concern: why does Oikos exist, who does it serve, what's inside vs.
outside the boundary? This is the Concept-of-Operations / Mission-Analysis
view — stakeholders → needs → mission → boundary → context.*
### 1.1 Mission statement
> **Oikos** (Greek: *household*) is an agentic operating system for the
> **hubris** homelab. It observes the fleet, orients findings against a
> typed ontology graph, decides what to do under an enforced risk policy,
> acts through a single gated execution primitive, and learns from the
> outcome — so that the homelab is administered with the same rigor as a
> managed production environment, without requiring the operator to be
> present for every routine decision.
The name is deliberate: Oikos is not "a monitoring tool" or "a chatbot for
servers." It is the **household's operating system** — the substrate every
other agent (human or AI) reads state from and requests action through.
### 1.2 Stakeholders
| Stakeholder | Role | What they need from Oikos |
|---|---|---|
| **Operator** (dtoro) | Owner, sole human decision-maker | Situational awareness without babysitting; a single approval gate for risky actions, not a flood of pings; confidence that destructive actions cannot happen silently |
| **Nomos** (AI agent) | Primary autonomous actor, MCP client | A safe, complete, well-typed action surface (`run` + read tools) it cannot bypass; enough context (ontology, knowledge, history) to act correctly without re-discovering facts every session |
| **Enrolled clients** (hosts, LXCs, VMs, workstations) | Managed infrastructure, also *sources* of signals | Correct classification of their state (health, lifecycle); truthful, current inventory; checks/agents deployed via the auto-setup mechanism |
| **Future agents / delegates** | Not yet built (§9) | A ledger and identity model that supports multi-agent delegation |
| **Control-room web UI** | The operator's window into the system | Real-time visibility (SSE) into fleet health, tasks, approvals, activity — without being the system of record itself |
| **The system itself (future sessions of Nomos/Claude)** | Re-orients from persisted state, not memory | DB-as-source-of-truth, knowledge that compounds task over task |
### 1.3 Mission drivers
These recur through every other view below and explain design choices that
would otherwise look arbitrary:
1. **The homelab is unattended most of the time.** A human cannot review
every restart or disk check. The system must be able to act — but only
within a bound it cannot exceed on its own (§8).
2. **Autonomy must be legible, not implicit.** Every mutation traces to a
risk class, an approval (or an explicit auto-act rule), and a ledger
entry. "Why did the agent do that?" must always be answerable from the
DB, not from a chat transcript that might scroll away.
3. **The agent should get smarter without getting less safe.** Learning
(patterns, knowledge) is explicitly **proposal-only** — see
[ADR-0006](../adr/0006-learning-proposal-only.md) — it can suggest, never
self-authorize.
4. **One approval, not one-per-step.** The operator's stated pain point
(see [plans/done/2026-07-11-goal-oriented-chat-control-panel.md](../../plans/done/2026-07-11-goal-oriented-chat-control-panel.md))
was re-approving every step of a multi-step task. The system evolved
from "approve every command" to "approve a plan once, then the system
is the event loop, not the human" (auto-continuation, §6.5).
5. **General gated execution over a fixed action enum.** Early design used
named RPCs per action type (`restart_service`, `apt_upgrade`, …). The
system moved to **one primitive (`run`) that is classified by what it
actually does**, not by what it's called — see
[plans/2026-07-10-general-gated-execution.md](../../plans/2026-07-10-general-gated-execution.md)
and the north-star note in memory: unlimited actions, classifier-gated,
not a whitelist of verbs. Full status in §9.
### 1.4 System boundary
```mermaid
flowchart TB
subgraph boundary["Oikos system boundary"]
API["oikos api\n(REST + MCP, :8090)"]
SCHED["oikos scheduler\n(observe loop)"]
NOTIFY["oikos notifier\n(approvals, Matrix)"]
NOMOS["nomos serve\n(MCP gateway, :8092)"]
DB[("PostgreSQL/TimescaleDB")]
WEB["control-room SPA\n(Svelte 5)"]
end
OPERATOR["Operator (human)"]
LLM["LLM provider\n(OpenRouter — Nomos's model)"]
MATRIX["Matrix homeserver\n(approval alerts)"]
INFISICAL["Infisical\n(secrets, primary)"]
SOPS["SOPS+age\n(secrets, DR fallback)"]
FLEET["Managed fleet\nhosts, LXCs, VMs\n(SSH targets)"]
GITEA["Gitea\n(git remote, CI, deploy webhook)"]
OPERATOR <-->|chat, approvals, console| WEB
OPERATOR -->|✅/❌ reactions| MATRIX
WEB -->|bearer token, REST + SSE| API
NOMOS -->|MCP tool calls| API
NOMOS <-->|completions| LLM
API --> DB
SCHED --> DB
SCHED -->|SSH probes| FLEET
NOTIFY --> DB
NOTIFY -->|alerts| MATRIX
API -->|secrets read| INFISICAL
API -.fallback.-> SOPS
API -->|SSH exec, run tool| FLEET
GITEA -->|push webhook| API
GITEA -->|pull| FLEET
style boundary fill:#e8f5e9,stroke:#2e7d32
```
**Inside the boundary:** the `oikos` binary's four roles (api, scheduler,
notifier — plus `all`), the `nomos` binary, the Postgres/TimescaleDB
instance, the control-room SPA, and the webhook receiver (`cmd/webhook`).
All are versioned in this repo and deployed together on the mac-mini.
**Outside the boundary** (external systems Oikos depends on but does not
own):
| External system | Purpose | Coupling |
|---|---|---|
| LLM provider (OpenRouter, model configurable) | Nomos's reasoning engine | Nomos calls out per turn; no LLM = no natural-language agent, but MCP tools + REST API still work headless |
| Matrix homeserver | Approval alerts, ✅/❌ reactions | Notifier pushes; loss of Matrix degrades to console-only approval |
| Infisical (+ SOPS/age fallback) | Secret storage | `internal/secrets` abstracts both; see [ADR-0010](../adr/0010-infisical-with-sops-fallback.md) |
| Gitea | Git remote, CI (`.gitea/workflows/ci.yml`), push-to-deploy webhook | Deploy trigger only; not a runtime dependency |
| The fleet itself (hosts/LXCs/VMs) | What's being managed | SSH is the only actuation channel — "never ad-hoc SSH" outside the actuator, see [OIKOS.md](../../.agents/OIKOS.md) |
### 1.5 Operational concept — the OODA loop
Every unit of work in Oikos — a scheduled probe, an operator's chat
message, a signal reacting to drift — is one pass through **Observe →
Orient → Decide → Act (→ Learn)**. This is the system's one behavioral
pattern, applied at every scale from "check disk usage" to "operator asks
for a fleet-wide update audit."
```mermaid
flowchart LR
O1["OBSERVE\nprobes, drift, agent findings\n→ Signals"] --> O2["ORIENT\nontology graph walk,\nblast radius, history"]
O2 --> O3["DECIDE\nrisk class × blast radius\n× confidence → route"]
O3 --> O4["ACT\nrun primitive → SSH exec\n→ verify → ledger"]
O4 --> O5["LEARN\nfeedback → pattern → skill\n(proposal-only)"]
O5 -.-> O1
```
The full functional decomposition of each phase is §3; the concrete state
machines and sequence traces are §6.
**The unit of autonomous work: the Task.** As of the 2026-07-11 redesign,
the operator-facing unit of the OODA loop is not a raw chat session but a
**Task**: a goal, a proposed plan, a single approval gate, autonomous
execution across every step, and a knowledge deposit at the end (see
[plans/done/2026-07-11-goal-oriented-chat-control-panel.md](../../plans/done/2026-07-11-goal-oriented-chat-control-panel.md)).
This is *how* the OODA loop is packaged for a human to supervise without
narrating every step — detailed in §3.4 and §6.56.6.
---
## 2. Requirements
*Concern: what must the system do, traced to source and implementation
status? Stakeholder needs from §1.2 are decomposed here into verifiable
requirements.*
Legend: ✅ implemented and live · ⚠ implemented but not wired into the
running system · ❌ schema/policy only, no code path · 🔜 planned.
### 2.1 Observe
| ID | Requirement | Source | Status |
|---|---|---|---|
| REQ-OBS-1 | The system shall continuously probe the fleet on a fixed interval without operator intervention. | [OIKOS.md](../../.agents/OIKOS.md) §OODA | ✅ `scheduler.go` — 30s default tick |
| REQ-OBS-2 | Probes shall support at least: HTTP reachability, TCP reachability, disk usage, TLS certificate expiry, ICMP ping, and arbitrary SSH-delivered health scripts. | [ADR-0013](../adr/0013-signal-triggers.md) | ✅ 6 check kinds, `scheduler.go:executeCheck` |
| REQ-OBS-3 | An unhealthy probe result shall raise a deduplicated Signal; a healthy result shall resolve any open Signal for that check. | [ADR-0014](../adr/0014-entity-model.md) §5 | ✅ `UpsertSignal`/`resolveSignal`, unique partial index on open signals |
| REQ-OBS-4 | Entities whose checks have gone silent beyond a multiple of their expected interval shall be marked `stale`, not silently reported healthy. | — | ✅ `staleSweep`, 3× fastest interval, 5 min floor |
| REQ-OBS-5 | New entities shall receive a sane default check set at creation time, without manual check authoring. | — | ✅ `internal/checkdefaults` |
### 2.2 Orient
| ID | Requirement | Source | Status |
|---|---|---|---|
| REQ-ORI-1 | All infrastructure shall be represented as typed entities connected by typed, validated relationships ("rule of completeness": if it can break, change, or hold data, it has an entity). | [OIKOS.md](../../.agents/OIKOS.md), [seeds/ontology.yaml](../../seeds/ontology.yaml) | ✅ 60 entity types, 47 relationship types |
| REQ-ORI-2 | The system shall answer "what breaks if X goes down" as a graph query, not a document search. | README §Architecture | ✅ `blast_radius()` recursive CTE |
| REQ-ORI-3 | Narrative knowledge (docs, runbooks, investigations) shall be queryable by full-text search and linkable to the entities they concern. | AGENTS.md §5 | ✅ `knowledge_entities` + FTS (migration 011) |
| REQ-ORI-4 | An agent orienting on a task shall be able to retrieve prior successful/failed attempts on the same entity before acting. | [goal-oriented-chat-control-panel](../../plans/done/2026-07-11-goal-oriented-chat-control-panel.md) | ✅ `get_entity_knowledge`, outcome-tagged |
### 2.3 Decide
| ID | Requirement | Source | Status |
|---|---|---|---|
| REQ-DEC-1 | Every state-changing action shall be classified into exactly one risk class (`read_only`, `reversible_low`, `config_mutation`, `destructive`) before it is allowed to run. | [seeds/policy.yaml](../../seeds/policy.yaml), [general-gated-execution](../../plans/2026-07-10-general-gated-execution.md) | ✅ `policy.ClassifyCommand` inside `classifyAndGate`, `internal/mcp/server.go` |
| REQ-DEC-2 | A classifier shall be able to raise the required risk class above what the caller declares, but never lower it. | plan `general-gated-execution` | ✅ `riskRank` max() in `command.go` |
| REQ-DEC-3 | `config_mutation` actions shall require operator approval — either a console/Matrix decision or in-chat assent to an already-approved plan. | seeds/policy.yaml, plan `general-gated-execution` | ✅ approval row + assent window |
| REQ-DEC-4 | `destructive` actions shall require a typed confirmation phrase distinct from ordinary assent, scoped narrowly (target + session), even inside an approved plan. | plan `general-gated-execution` | ✅ destructive window, `internal/mcp/server.go:1526` |
| REQ-DEC-5 | A global kill-switch and per-entity kill-switches shall be able to force escalation regardless of the computed risk class. | seeds/policy.yaml `autonomy_settings` | ❌ **Not enforced on the live path.** Only checked by the dead `ClassifySignal` and the unwired `actuator` package — see [§8.1](#81-the-kill-switch-gap-verified-most-important-finding-in-this-model). |
| REQ-DEC-6 | Lifecycle state shall modify approval requirements (`provisioning` nodes cheaper to mutate; `deprecated`/`destroyed` nodes refuse new dependents/actions). | [OIKOS.md](../../.agents/OIKOS.md) | ✅ policy.yaml lifecycle overrides (ingested; enforcement scope not independently re-verified this pass) |
### 2.4 Act
| ID | Requirement | Source | Status |
|---|---|---|---|
| REQ-ACT-1 | The system shall expose exactly one execution primitive for mutating the fleet (`run`), classified by what the command actually does, not by a fixed named-action enum. | plan `general-gated-execution` | ✅ `request_execution` retired 2026-07-14; `run` is sole mutation tool |
| REQ-ACT-2 | Execution shall be auditable: every mutation records who/what/when/risk class/result. | [ADR-0007](../adr/0007-threat-model.md) | ✅ `audit_log` hypertable, `executions` rows |
| REQ-ACT-3 | Long-running or asynchronous actions shall be pollable for status without blocking the caller. | AGENTS.md §3 | ✅ `get_execution_status` |
| REQ-ACT-4 | A signal that crosses a risk threshold shall be able to trigger autonomous action without an operator initiating a chat turn. | [OIKOS.md](../../.agents/OIKOS.md) §OODA "auto-act" | ❌ **Not implemented.** `internal/actuator` is a stub and is not started by any binary — see §9. Only operator/agent-initiated `run` calls execute today. |
| REQ-ACT-5 | An operator shall approve a multi-step plan once, and the system — not the human — shall drive subsequent steps to completion. | [goal-oriented-chat-control-panel](../../plans/done/2026-07-11-goal-oriented-chat-control-panel.md) | ✅ assent window + auto-continuation worker (`nomos_plan_executions`) |
### 2.5 Learn
| ID | Requirement | Source | Status |
|---|---|---|---|
| REQ-LRN-1 | Execution outcomes shall be aggregated into confidence-scored patterns per (entity type, action). | [ADR-0006](../adr/0006-learning-proposal-only.md) | ⚠ Algorithm implemented (Wilson lower bound, evidence≥5 ∧ confidence≥0.7 → `validated`) but `learning.Run` is never started — dormant |
| REQ-LRN-2 | Learning shall be structurally incapable of expanding its own autonomy — it may propose, never self-authorize. | ADR-0006 | ⚠ Consistent by design (no governance-table writes in `learning.go`) but currently moot since the loop never runs |
| REQ-LRN-3 | A burst of same-key feedback in a single aggregation pass shall be quarantined rather than trusted. | ADR-0006 | ⚠ Implemented (`total > 10` per group ⇒ quarantine) but dormant, same as LRN-1 |
| REQ-LRN-4 | On task completion, the agent shall deposit a knowledge note linked to the entities involved and to the task itself, tagged with outcome. | goal-oriented-chat-control-panel | ✅ `upsert_knowledge` + `about`/`involves` relationships; SOUL-nudged, partially enforced server-side |
### 2.6 Non-functional requirements
| ID | Requirement | Rationale | Status |
|---|---|---|---|
| REQ-NFR-1 (Security) | Every API/MCP route except health check and enrollment shall require a real bearer credential — no dev-open bypass. | [ADR-0015](../adr/0015-api-bearer-auth-client-server-split.md) closed a production hole where `docker-compose.yml` ran with auth disabled | ✅ `combinedAuth`, both static-token and OIDC JWT paths |
| REQ-NFR-2 (Security) | The Nomos agent gateway (`:8092`) shall require authentication before accepting chat/session/approval-assent requests. | Anyone on the LAN/mesh can currently chat as the agent and grant approvals via assent | ❌ Open gap (C1), deferred by explicit operator decision — see §8 |
| REQ-NFR-3 (Security) | Secrets shall never be hardcoded; they shall be resolved from a managed secret store with a documented fallback. | ADR-0010 | ✅ Infisical primary, SOPS+age DR fallback |
| REQ-NFR-4 (Security) | Destructive shell patterns (`rm -rf`, `dd of=`, `mkfs`, `DROP TABLE`, credential-file reads, fork bombs, …) shall be denylisted independent of what the caller declares. | plan `general-gated-execution` | ✅ `destructivePatterns` regex list, `command.go` |
| REQ-NFR-5 (Reliability) | A failed deploy shall be automatically health-checked and the operator shall have a scripted, tested path to roll back to the previous known-good state. | [docs/operations/rollback.md](../operations/rollback.md) | ✅ `scripts/deploy.sh` health poll + `scripts/rollback.sh` (manual trigger, not automatic) |
| REQ-NFR-6 (Reliability) | Database schema changes shall be forward-only and idempotent so that redeploys never require manual intervention. | [ADR-0008](../adr/0008-forward-only-migrations.md) | ✅ 20 migrations, `IF NOT EXISTS` throughout |
| REQ-NFR-7 (Observability) | Operators shall see fleet state, pending approvals, and agent activity in real time without polling. | plan `control-room-webui` | ✅ SSE via Postgres `LISTEN/NOTIFY` fan-out |
| REQ-NFR-8 (Observability) | Every mutation and every agent tool call shall be traceable to the session/task that caused it. | plan `session-reliability-and-ux-audit` | ✅ `audit_log.session_id` (migration 020), `agent_activity` correlation |
| REQ-NFR-9 (Data) | Time-series telemetry shall be retained with automatic rollup/expiry rather than growing unbounded. | ADR-0002 | ✅ TimescaleDB continuous aggregates + retention policies |
| REQ-NFR-10 (Versioning) | Every change to `main` shall bump a semver-ish version visible in the UI, so operators can correlate behavior with a build. | AGENTS.md §9 | ✅ `VERSION` file, enforced by convention not CI gate |
### 2.7 Explicitly out of scope
- **Perfect classification of arbitrary shell.** The general `run`
primitive accepts free-form commands; the plan authoring the design
([general-gated-execution](../../plans/2026-07-10-general-gated-execution.md))
states outright that perfect static classification of shell is impossible
and commits instead to default-escalate + a hard denylist + always-on
audit as the mitigation, not a guarantee of catching every dangerous
command shape.
- **Per-client bearer tokens.** All enrolled clients currently share one
`OIKOS_MCP_BEARER_TOKEN`; per-client token issuance is acknowledged as
future work (AGENTS.md §4).
- **Multi-agent delegation.** Blocked on a ledger identity field
([OIKOS.md](../../.agents/OIKOS.md) backlog).
### 2.8 Requirements not yet written down anywhere formal
Two observations from this pass that aren't captured as a requirement in
any plan/ADR but should be, given what the code does today:
- There is no requirement anywhere stating **"the kill-switch shall apply
to operator/agent-initiated `run` calls, not just autonomous auto-act."**
The policy schema implies it should (`seeds/policy.yaml` calls
`never_auto_act.service:caddy` a "hard block"), but REQ-DEC-5's status
shows the code doesn't honor that today for the primary execution path.
Either a formal requirement + fix, or an ADR narrowing the kill-switch's
documented scope to match reality, would close this.
- There is no requirement governing **circuit-breaking of repeated failed
executions against the same target** on the live `run` path. A real
circuit breaker exists (`internal/actuator/actuator.go:156-202`) but
guards only the unused auto-act loop.
---
## 3. Functional Architecture
*Concern: what functions does the system perform, and which component
actually performs each one? Functional architecture is drawn before
physical architecture (§4) deliberately, so a function's allocation can be
called out when it diverges from where the schema/ontology implies it
lives — which happens more than once below.*
### 3.1 Top-level function decomposition
```mermaid
flowchart TB
F0["F0 Operate the homelab autonomously\n(the mission function)"]
F0 --> F1["F1 Observe"]
F0 --> F2["F2 Orient"]
F0 --> F3["F3 Decide"]
F0 --> F4["F4 Act"]
F0 --> F5["F5 Learn"]
F0 --> F6["F6 Communicate with the operator"]
F1 --> F1a["F1.1 Run scheduled probes"]
F1 --> F1b["F1.2 Deduplicate & lifecycle-manage signals"]
F1 --> F1c["F1.3 Detect staleness"]
F2 --> F2a["F2.1 Maintain the ontology graph"]
F2 --> F2b["F2.2 Compute blast radius"]
F2 --> F2c["F2.3 Retrieve prior knowledge for a target"]
F2 --> F2d["F2.4 Track what a task has touched"]
F3 --> F3a["F3.1 Classify a command's risk"]
F3 --> F3b["F3.2 Gate on plan-first / dedup / anti-flood"]
F3 --> F3c["F3.3 Grant/consume approval or assent"]
F3 --> F3d["F3.4 Enforce kill-switches"]
F4 --> F4a["F4.1 Execute over SSH"]
F4 --> F4b["F4.2 Verify & record outcome"]
F4 --> F4c["F4.3 Auto-continue a plan across steps"]
F5 --> F5a["F5.1 Extract patterns from feedback"]
F5 --> F5b["F5.2 Quarantine anomalous bursts"]
F5 --> F5c["F5.3 Deposit task-outcome knowledge"]
F6 --> F6a["F6.1 Render live fleet/task state"]
F6 --> F6b["F6.2 Deliver approval prompts (Matrix)"]
F6 --> F6c["F6.3 Stream real-time events (SSE)"]
```
### 3.2 Function allocation matrix
The "expected owner" column is what the schema/ontology/ADRs imply; the
"actual owner" column is what this pass verified in the running system
(grep + direct read, corroborated by four independent research passes —
see §9). Where they differ, the function is marked **misallocated /
orphaned**.
| Function | Expected owner | Actual owner (live) | Note |
|---|---|---|---|
| F1.1 Run scheduled probes | `internal/scheduler` | `internal/scheduler/scheduler.go` | ✅ matches |
| F1.2 Signal dedup/lifecycle | `internal/scheduler` + `internal/domain.Signal` state machine | `scheduler.go` raw SQL (`UpsertSignal`, `resolveSignal`) | ⚠ `resolveSignal` updates `raised→resolved` directly without calling `Signal.CanTransition` — the one enforced state machine in the domain layer is bypassed by its own primary caller |
| F2.2 Compute blast radius | policy/ontology layer | Postgres `blast_radius()` SQL function, called from both `internal/policy/classify.go` (dead) and `internal/mcp/server.go` (live, `get_blast_radius` tool) | ✅ the SQL function itself is shared and correct; only the *classifier* caller is dead |
| **F3.1 Classify a command's risk** | `internal/policy` (one classifier) | **Two independent classifiers exist.** `policy.ClassifySignal` (signal-oriented, DB-rule driven) is **dead code — zero callers**. `policy.ClassifyCommand` (regex allow/deny-list on the literal shell command) is the one actually invoked, from `internal/mcp/server.go:classifyAndGate`. | 🔴 misallocated — the documented/schema-implied classifier is not the one running |
| F3.2 Plan-first/dedup/anti-flood gates | — (not modeled in ontology at all) | `internal/mcp/server.go:classifyAndGate` (P1, dedup, P5) | New function, added by the `general-gated-execution` plan; not yet reflected in `seeds/ontology.yaml` |
| F3.3 Approval / assent | `internal/notifier` (token issuance) + operator decision | `internal/notifier/notifier.go` (Matrix leg) **and** a second, independent inline reimplementation of token verification in `internal/httpapi/phase3.go:DecideApproval` | ⚠ `notifier.VerifyApprovalToken` is dead code; the live verification path duplicates its logic rather than calling it |
| **F3.4 Enforce kill-switches** | `internal/policy` + `autonomy_settings` (`global.auto_act`, `never_auto_act.*`) | Checked only in `internal/policy/classify.go` (dead) and `internal/actuator/actuator.go` (never started) | 🔴 **not enforced anywhere in the live request path.** See [§8.1](#81-the-kill-switch-gap-verified-most-important-finding-in-this-model) |
| **F4.1 Execute over SSH** | `internal/actuator` (`ExecuteProcedure`, with SSH error classification, circuit breaker, retry budget) | `internal/httpapi/phase3.go:sshExec` and an equivalent in `internal/mcp/server.go` — a simpler, independently-written single-command executor | 🔴 `internal/actuator/ssh.go`'s well-built procedure runner has **zero callers**; the live executor is a separate, less defended implementation (no circuit breaker) |
| F4.2 Verify & record outcome | `internal/actuator` + `executions` state machine | `internal/httpapi/phase3.go` (`executeApprovedAction`) using ad-hoc string statuses, not the `domain.Execution` constants | ⚠ works, but the domain-layer state machine is descriptive only — no `CanTransition`-equivalent guards it |
| F4.3 Auto-continue a plan | `cmd/nomos` continuation worker | `cmd/nomos/continue.go` polling `nomos_plan_executions` | ✅ matches, and is the mechanism behind REQ-ACT-5 ("the system is the event loop, not the human") |
| F5.1F5.2 Learning | `internal/learning` | `internal/learning/learning.go` — algorithm is real (Wilson bound, quarantine) | 🔴 **never runs.** Not started by `cmd/oikos/main.go` or any `docker-compose.yml` service. Feedback accumulates; nothing processes it |
| F5.3 Knowledge deposit | Nomos SOUL + `upsert_knowledge` | `cmd/nomos` (SOUL-nudged) + `internal/mcp/server.go:upsert_knowledge` | ✅ matches, partially enforced (see §9 — a 348-tool-call audited session made zero `update_entity_attributes`/`create_relationship` calls despite discovered drift) |
| F6.3 SSE stream | `internal/httpapi/sse.go` | same | ✅ Postgres `LISTEN/NOTIFY` → in-memory ring buffer → per-connection fan-out |
**Reading this table honestly:** F1 (Observe) and F6 (Communicate) are
solidly allocated to where you'd expect. F3 (Decide) and F4 (Act) — the two
functions that actually gate and perform mutation — are where the
implementation diverged furthest from the schema/ontology's implied design,
because they were rebuilt under the `general-gated-execution` plan
*alongside*, not *instead of*, the original `internal/policy` /
`internal/actuator` packages. The old packages were never deleted, so they
read as "the" implementation to anyone navigating by package name alone.
### 3.3 Functional flow — the `run` gate (F3 in detail)
This is the most safety-relevant function in the system and the one worth
seeing end to end. It supersedes the classifier flow drawn in
[ADR-0014](../adr/0014-entity-model.md) §3, which predates the command
classifier.
```mermaid
flowchart TD
START(["run(target, command, purpose, declared_risk)\ncalled by Nomos, session-scoped"]) --> P1{"P1: does this session\nhave a plan?"}
P1 -->|no, and _session_id set| REJECT1["reject — propose_plan first"]
P1 -->|"yes, or no session (direct call)"| DEDUP{"identical pending-approval\ncommand already queued\nfor this target?"}
DEDUP -->|yes| REJECT2["reject — dedup"]
DEDUP -->|no| FLOOD{"P5: another config_mutation\nalready pending for\nthis session?"}
FLOOD -->|yes| REJECT3["reject — anti-flood"]
FLOOD -->|no| CLASSIFY["ClassifyCommand(command, declared_risk)\ndenylist → destructive\nallowlist-only-segments → read_only\notherwise → config_mutation\nfinal = max(computed, declared)"]
CLASSIFY -->|read_only| EXEC["execute immediately over SSH"]
CLASSIFY -->|config_mutation| ASSENT{"assent window\nactive for this\nagent+session?"}
ASSENT -->|yes| EXEC
ASSENT -->|no| QUEUE1["queue pending_approval\n+ approvals row\n+ approval.created SSE event"]
CLASSIFY -->|destructive| DWIN{"destructive window\nactive (typed 'I confirm'\nfor this target+session)?"}
DWIN -->|yes| EXEC
DWIN -->|no| QUEUE2["queue pending_approval\nmarked DESTRUCTIVE"]
EXEC --> RECORD["insert execution entity +\nexecutions row + targets/involves\nrelationships"]
QUEUE1 --> WAIT["operator decides via\nconsole or Matrix ✅/❌"]
QUEUE2 --> WAIT
WAIT -->|approved| EXEC
WAIT -->|denied/expired| END(["terminal, no execution"])
RECORD --> POLL["get_execution_status\npollable"]
```
**What is conspicuously absent from this diagram**, verified directly
against `internal/mcp/server.go`: a check against `autonomy_settings`
(`global.auto_act`, `never_auto_act.<slug>`). Compare against
[ADR-0014](../adr/0014-entity-model.md) §3's OODA diagram, which shows the
kill-switch gating the *orient→decide* transition — that gate exists in
code (`internal/policy/classify.go`), but on a function (`ClassifySignal`)
nothing calls. See [§8.1](#81-the-kill-switch-gap-verified-most-important-finding-in-this-model)
for the safety implication.
### 3.4 Functional flow — the Task lifecycle (F3/F4/F5 packaged for a human)
The Task is the unit the operator actually supervises (§1.5). It packages
one pass of Decide→Act→Learn behind a single approval:
```mermaid
stateDiagram-v2
[*] --> planning: goal set via set_goal
planning --> awaiting_approval: propose_plan
awaiting_approval --> executing: operator approves,\nopens assent window
executing --> executing: update_plan_step per step,\npending to running to done
executing --> awaiting_input: ask_operator,\nreal decision needed
awaiting_input --> executing: operator answers
executing --> done: complete_task, outcome success or partial
executing --> failed: complete_task, outcome failure
done --> [*]
failed --> [*]
```
Approving the plan once authorizes every `read_only` and `config_mutation`
step already named in it — this is F3.3 folded into a single grant rather
than one grant per `run` call. `destructive` steps still require the typed
confirmation phrase regardless of plan approval (F3.4's *intended* design,
not the missing kill-switch check above — these are different mechanisms:
the destructive-window requirement is enforced in `classifyAndGate`; the
policy-level kill-switch is not).
**Structural gates, not prose reminders.** An earlier version of this
lifecycle relied on the model's system prompt (SOUL.md) to avoid
re-proposing a plan or double-completing a task. A 2026-07-14 incident chain
(empty LLM response → misclassified as disconnect → generic reconnect note
→ agent re-proposed the plan) led to a deliberate architecture shift: SOUL
prose was demoted from primary safeguard to instruction, and
`propose_plan`/`complete_task` themselves now refuse invalid calls
server-side (`plans/done/2026-07-14-post-fix-session-remainders.md`). This
is a general pattern worth carrying forward: **anywhere the model's
correctness depends on a system-prompt instruction rather than a tool-level
guard, treat it as a latent bug, not a solved problem.**
---
## 4. Physical Architecture
*Concern: what physically runs, where, on what ports, talking to what?
Where §3 asked "what does the system do and who actually does it," this
asks "what actually runs." This is the Block Definition Diagram /
deployment-diagram layer, expressed as containers, processes, and ports —
there is no SysML tool in this repo's toolchain, so Mermaid stands in.*
### 4.1 Component block diagram
```mermaid
flowchart TB
subgraph binary["cmd/oikos — single Go binary, role-selected by CLI arg"]
API["api role\nREST :8090 + MCP\ninternal/httpapi, internal/mcp"]
SCHED["scheduler role\ninternal/scheduler"]
NOTIFY["notifier role\ninternal/notifier"]
ORPHAN1["internal/actuator\n(package exists, NO role\nstarts it — dead weight)"]
ORPHAN2["internal/learning\n(package exists, NO role\nstarts it — dead weight)"]
end
NOMOSBIN["cmd/nomos — separate binary\nserve :8092\nagent gateway + task tools"]
WEBHOOK["cmd/webhook\n:9797\nGitea deploy receiver"]
DESKTOP["cmd/desktop\nWails-wrapped SPA\n(distributed as .app, not a server role)"]
WEB["web/ — Svelte 5 SPA\nstatic, served by Caddy container\nnot embedded in cmd/oikos"]
DB[("postgres 16\n+ TimescaleDB 2.17.2\n:5432")]
binary --- DB
NOMOSBIN -->|MCP over HTTP| API
WEBHOOK -->|"exec scripts/deploy.sh"| binary
WEB -->|REST + SSE, bearer/OIDC| API
WEB -.optional desktop shell.- DESKTOP
style ORPHAN1 fill:#ffebee,stroke:#c62828,stroke-dasharray: 5 5
style ORPHAN2 fill:#ffebee,stroke:#c62828,stroke-dasharray: 5 5
```
The dashed red boxes are drawn *inside* the binary because they compile
into it, but no `cmd/oikos` role wires their `Run()` loop — see §3.2 and §9
for the full evidence trail. Physically they are dead code shipped in the
binary, not a running process.
### 4.2 Deployment topology (mac-mini, Docker Compose)
```mermaid
flowchart TB
subgraph host["mac-mini — Docker host"]
subgraph profile_dev["profile: dev"]
PG[("postgres:5432")]
MIG["migrate\n(one-shot, exits)"]
SEED["seed\n(one-shot, exits)"]
APID["api :8090"]
SCHEDD["scheduler"]
NOTIFYD["notifier"]
end
subgraph profile_full["profile: full (adds)"]
NOMOSD["nomos :8092"]
WEBD["web :8091\n(Caddy static)"]
end
subgraph profile_infisical["profile: infisical (adds)"]
REDIS["redis :6379"]
INFISICALD["infisical :8080"]
end
CADDY["Caddy (external repo\ndtoro/caddy-conf, LXC 121)\nnot in this compose file"]
end
MIG --> PG
SEED -->|after migrate succeeds| PG
APID --> PG
SCHEDD --> PG
NOTIFYD --> PG
NOMOSD -->|MCP calls, bearer token| APID
WEBD -->|REST + SSE| APID
APID -.secrets.-> INFISICALD
CADDY -->|"oikos.hubris.network → :8091 (browser)\n/api,/mcp,/agent → :8090 direct"| APID
CADDY -->|"mcp.hubris.network → :8090"| APID
CADDY -->|"nomos.hubris.network → :8092\n(NO auth of its own)"| NOMOSD
CADDY --> WEBD
```
| Service | Port | Profile(s) | Notes |
|---|---|---|---|
| `postgres` | 5432 | all | TimescaleDB 2.17.2 / PG16, healthcheck-gated startup |
| `migrate` | — | all | one-shot, `compose/oikos/Dockerfile … migrate`, exits after applying `migrations/` |
| `seed` | — | all | one-shot, runs only after `migrate` succeeds; ingests `seeds/*.yaml` |
| `api` | 8090 | dev, full | REST + MCP; **every route requires a real bearer token**, no dev-open bypass ([ADR-0015](../adr/0015-api-bearer-auth-client-server-split.md)); mounts an SSH key read-only for the actuation path |
| `scheduler` | — | dev, full | observe loop, `NET_RAW` capability for ping checks, mounts the same SSH key |
| `notifier` | — | dev, full | Matrix alerts + approval token issuance, needs `OIKOS_APPROVAL_HMAC_SECRET` |
| `nomos` | 8092 | full only | agent gateway, calls back into `api` over MCP with the shared bearer token; talks to OpenRouter for completions |
| `web` | 8091 (host) → 80 (container) | dev, full | Svelte 5 SPA, multi-stage build (`node:22-alpine``caddy:2-alpine` static serve); **standalone**, deployed and versioned independently of the Go binary |
| `redis`, `infisical` | 6379, 8080 | infisical, full | secrets backend; SOPS+age is the DR fallback if this stack is down |
**Reference vs. live Caddy config.** `compose/caddy/Caddyfile.oikos` in this
repo is a *reference copy* — the config actually enforced in production
lives in a separate `dtoro/caddy-conf` repo deployed to LXC 121 and is kept
in sync manually. As of the last verified check
(`plans/done/2026-07-12-wails-desktop-app.md` completion note),
`oikos.hubris.network` had **no live site block yet** in the real Caddy
config — the routing table above is the *designed* topology, not a
confirmed-live one for that specific hostname. `mcp.hubris.network` and
`nomos.hubris.network` route directly to their respective ports with no
browser-session layer in front — `nomos.hubris.network` in particular has
**no auth of its own** at the Caddy layer or the Nomos process layer
(§8).
### 4.3 Physical/network trust zones
Per [ADR-0007](../adr/0007-threat-model.md), the system's trust boundary is
expressed as Docker networks rather than host firewalling:
| Zone | Contains | Trust level |
|---|---|---|
| `net-front` | Caddy, web SPA | Internet/mesh-facing, TLS terminated here |
| `net-data` | postgres, redis | No direct external exposure; only backend services reach it |
| `net-ops` | api, scheduler, notifier, nomos | Holds the SSH key and secret-store credentials; the actual actuation surface |
The agent (Nomos) itself holds **no SSH key**`SOUL.md` states this
explicitly, and it's structurally true: the key lives only inside the `api`
container's environment (`OIKOS_SSH_KEY_PATH`, default
`/etc/oikos/ssh_key`), read by `internal/mcp/server.go`'s `initSSH()`. Nomos
can only ever reach a host by asking `api` to run something through the
gated `run` tool — it cannot open its own SSH session even if compromised,
short of getting `api` to do it for it.
### 4.4 External systems and their coupling
| External system | Reached from | Failure mode if unavailable |
|---|---|---|
| OpenRouter (LLM provider) | `cmd/nomos` only | Natural-language chat stops; MCP tools + REST API remain fully usable headless |
| Matrix homeserver | `internal/notifier` | Approvals fall back to console-only decision; no ✅/❌ reaction path |
| Infisical | `internal/secrets` (used from `api`) | Falls back to SOPS+age files on disk ([ADR-0010](../adr/0010-infisical-with-sops-fallback.md)) |
| Gitea | `cmd/webhook`, CI (`.gitea/workflows/`), the deploy poller | Push-to-deploy webhook stops firing; the 2-minute `launchd` poller (`scripts/deploy/network.hubris.oikos-deploy-poller.plist`) is the redundant fallback path |
| The fleet itself | `internal/scheduler` (probes), `api`'s `sshExec` (actuation) | No live health data; no new mutations possible, but read paths served from last-known DB state still work |
### 4.5 Build & release artifacts
| Artifact | Built by | Distributed as |
|---|---|---|
| `oikos` binary | `.gitea/workflows/ci.yml` build-test job (vet/lint/vulncheck/build/test), `compose/oikos/Dockerfile` for the runtime image | Docker image, rebuilt on the deploy host from source (no registry push) |
| `nomos` binary | same CI job, `compose/nomos/Dockerfile` | Docker image, same local-rebuild model |
| `web` SPA | `compose/web/Dockerfile` (multi-stage) | Docker image serving static assets via Caddy, **or** bundled into the desktop app below |
| Desktop app (`cmd/desktop`) | `.gitea/workflows/desktop.yml`, CGO-enabled Wails build, Linux amd64 currently | Tarball attached to a Gitea release on tag push; installed manually to `/Applications` on the operator's Mac as of 2026-07-14 |
Note the asymmetry: server-side services are deployed via
**push-to-main → local rebuild on mac-mini** (§4.2), while the desktop app
is deployed via **explicit version tags → release artifact → manual
install**. These are two different release cadences for the same frontend
codebase (`web/`) — worth knowing before assuming "deployed" means the same
thing for both.
### 4.6 Physical architecture vs. functional allocation — the gap in one sentence
The functional architecture (§3) shows F3 (Decide) and F4 (Act) allocated
to `internal/mcp` and `internal/httpapi`; this section's block diagram
shows *why* that's surprising to someone reading package names cold —
`internal/actuator` and `internal/policy`'s signal classifier look, from
directory structure alone, like they should be the answer. They compile,
they're well-tested in isolation in places, and they are not wired into
anything `cmd/oikos` starts. Anyone extending the execution or
classification logic should extend `internal/mcp/server.go` and
`internal/policy/command.go` — not `internal/actuator` or
`internal/policy/classify.go` — until/unless a deliberate decision is made
to consolidate (tracked as an open question in §9.4).
---
## 5. Interfaces (ICD)
*Concern: every interface a component in §4 exposes to another component
or to the operator, with enough detail to integrate against without
reading the source.*
### 5.1 Authentication (applies to every interface below)
`internal/httpapi/server.go`'s `combinedAuth` middleware wraps every route
except `GET /healthz`, `POST /api/v1/clients/enroll`,
`GET /api/v1/auth/oidc-config`, `POST /api/v1/auth/oidc-token`, and
`GET /oidc-callback`. It tries, in order:
1. **OIDC JWT** — if `OIDCIssuer`/`OIDCClientID` are configured, validates
against a cached JWKS fetched from the issuer's
`.well-known/openid-configuration`, checking issuer + audience, RS/ES
algorithms only.
2. **Static bearer token**`OIKOS_API_TOKEN` (operator/SPA) or
`OIKOS_MCP_BEARER_TOKEN` (agent), compared in constant time.
There is deliberately **no dev-open bypass** — this was a real production
gap closed by [ADR-0015](../adr/0015-api-bearer-auth-client-server-split.md).
The one interface in this document not covered by this middleware is the
Nomos gateway itself (§5.4) — see §8.
The SSE endpoint accepts the token as a query parameter
(`?token=…`, `allowQueryToken=true`) because browser `EventSource` cannot
set custom headers.
### 5.2 MCP tool catalog (`internal/mcp/server.go`) — 33 tools
This is the agent-facing action surface — every tool an LLM-driven agent
(Nomos or any other MCP client) can call. Every call is wrapped in
`withActivityLogging`, so every row below produces an `agent_activity`
audit entry regardless of outcome.
**Context — observe & orient** (read-only, no approval):
| Tool | Purpose |
|---|---|
| `get_entity(slug)` | Fetch one entity by slug/UUID |
| `list_entities(type, limit, cursor)` | Cursor-paginated entity listing |
| `get_relations(entity)` | Relationships for an entity |
| `get_blast_radius(entity)` | Downstream-impact graph walk (`blast_radius()` SQL fn) |
| `search_knowledge(query)` | Full-text search over documents/investigations/runbooks |
| `get_entity_knowledge(entity_slug)` | Every knowledge item linked to one entity, one call, outcome-tagged |
| `get_knowledge_content(id)` | Full markdown body of one knowledge item |
| `get_patterns(status, entity_type, action)` | Learned action patterns (dormant source — §9) |
| `get_skills(status)` | Automation skill catalog |
| `http_get(url)` | Public-web fetch, SSRF-guarded (`isPrivateHost`), ~16KB cap, HTTP/HTTPS only |
**Management — live state** (read, but some do live SSH reads):
| Tool | Purpose | Live SSH? |
|---|---|---|
| `get_service_status(service_slug)` | `systemctl is-active` on target host | yes |
| `tail_log(service_slug, lines=200)` | `journalctl` tail | yes |
| `list_lxcs()` | All LXC containers, ID/host/IP/health | no (DB) |
| `get_lxc_state(lxc_slug)` | `pct status` from the Proxmox host | yes |
| `ping_service(service_slug)` | HTTP reachability from `entity_status` | no (DB) |
| `list_my_secrets(caller_pubkey)` | Secrets reachable by this client's age pubkey | no (DB) — see §8 re: scoping gap |
**Oikos — decisions:**
| Tool | Purpose |
|---|---|
| `explain(service_slug)` | Compact context card: type, state, health, relations |
| `preflight(service_slug, action)` | **Static** risk-class lookup by action string — not the real command classifier, see caution below |
| `whoami(hostname)` | Entity record, peers, health for a client |
| `get_change_history(entity_slug, limit=20)` | Recent audit-log entries for one entity |
| `get_state_snapshot()` | Fleet health, disk, drift count |
> **Caution:** `preflight` looks up a risk class from `approval_rules` by
> `(entity_type, action)` — the same table the *dead* `ClassifySignal`
> reads. It does **not** run `ClassifyCommand`'s regex classifier and does
> **not** reflect what will actually happen when `run` is called with a
> specific shell command. Treat its output as advisory, not authoritative.
**Operations — observe & act:**
| Tool | Purpose |
|---|---|
| `get_health_summary()` | Fleet health counts (healthy/degraded/down/unknown) |
| `get_signal_history(entity_slug, state, limit)` | Open + recent signals |
| `get_audit_trail(entity_id)` | Audit log filter + browse |
| `get_agent_activity(limit)` | Agent self-inspection of its own activity log |
| `query_metrics(hours=24)` | Time-bucketed metric averages (TimescaleDB) |
| `get_trend(entity_id, days=7)` | Metric slope over time |
| `get_event_timeline(severity, entity_slug, limit)` | Recent events |
**Knowledge — write** (no approval required; updates the graph, not live
infra):
| Tool | Purpose |
|---|---|
| `upsert_knowledge(title, content)` | Record a learning; the only way anything persists past a session |
| `update_entity_attributes(slug, attributes)` | Shallow-merge a discovered fact into an entity |
| `create_relationship(source, target, type)` | Idempotent edge insert between two entities |
**Execution — mutating the live infrastructure:**
| Tool | Purpose |
|---|---|
| `run(target, command, purpose, declared_risk)` | **The sole mutation primitive.** Classified, gated, audited — full flow in §3.3 |
| `get_execution_status(execution_id)` | Poll an execution by UUID or slug prefix |
**Retired:** `request_execution` (the old fixed action-type enum:
`restart`/`systemctl`/`pct_exec`/`apt_upgrade`/`pct_create`) was retired
2026-07-14 (commit `60effcb`). `internal/mcp/server.go:360` carries an
explicit comment: "legacy fixed enum … DO NOT re-register this tool" — the
old handlers remain as dead reference code, not a live interface.
**Task tools — Nomos-local, not on the shared MCP server.**
`cmd/nomos/tasks.go` defines five additional tools handled **in-process by
the gateway**, not forwarded to `api`'s MCP server (because the shared
server has no session concept): `set_goal(goal)`,
`propose_plan(steps:[{title,detail?,target_slug?}])`,
`update_plan_step(seq,status,execution_id?)`,
`ask_operator(prompt,options?,context_entities?,why?)`,
`complete_task(outcome,summary)`. These are what an agent connecting
directly to `api`'s `/mcp` endpoint (bypassing Nomos) would **not** have
access to.
### 5.3 REST API (`api/openapi.yaml`, contract-first — [ADR-0004](../adr/0004-openapi-first.md))
Top-level resource groups (`GET /api/v1/...` unless noted):
| Group | Examples |
|---|---|
| Entities & graph | `entities`, `relationships`, `graph` (+ `?format=mermaid`), `blast-radius` |
| Client lifecycle | `clients/enroll` (unauthenticated, IP-gated), client secrets, provisioning |
| Ontology | entity types, relationship types, lifecycle defs |
| Signals | list, ack, resolve, mute |
| Checks | check_defs CRUD |
| Executions | list, get, **cancel** |
| Classifications | list (writes are dead-path, see §9) |
| Approvals | list, **decision** (approve/deny) |
| Patterns / Skills | list, patch (status/quarantine), skill versions |
| Policy | risk-classes, approval-rules, autonomy (read + write — autonomy write is how an operator would toggle the kill-switch, see §8 for why that toggle doesn't reach the live `run` path today) |
| Knowledge | search, `/knowledge/recent`, `/knowledge/content/{id}` (non-OpenAPI, added directly on the chi router) |
| Metrics | `query`, `trend` |
| Audit | `audit` |
| Events | `events`, `events/stream` (SSE, §5.5) |
| Agent activity | `activity/recent`, `activity/session/{id}` (non-OpenAPI) |
| Learning | `learning/timeline`, `learning/trend` (non-OpenAPI) |
| Health / Dashboard | `healthz` (unauthenticated), `dashboard/summary` |
| Export | full DB state → YAML (regenerates `seeds/*.yaml` for DR) |
Cross-cutting conventions ([ADR-0004](../adr/0004-openapi-first.md),
[ADR-0009](../adr/0009-sse-over-websocket.md)): RFC 9457 `problem+json`
errors, cursor pagination, `If-Match`/ETag optimistic concurrency,
idempotency keys on mutating POSTs.
### 5.4 Nomos HTTP interface (`cmd/nomos`, port 8092)
| Route | Purpose | Auth |
|---|---|---|
| `GET /healthz` | Liveness | none |
| `POST /query` | Structured tool call (`{"tool":..., "args":...}`) — bypasses the LLM entirely; or `{"query":...}`, which returns a pointer to `/chat` rather than answering | **none** |
| `POST /chat` (SSE) | The real agentic loop — natural language in, streamed tool calls + tokens out | **none** |
| `GET/POST /sessions`, `/sessions/{id}` | Session (Task) CRUD | **none** |
This entire interface is unauthenticated today — see §8 for the risk
statement; it is a known, tracked, and explicitly deferred gap (`C1` in
`plans/2026-07-11-nomos-agent-code-review.md`), not an oversight this
document is newly discovering.
Nomos's own outbound interface to `api` is a hand-rolled Streamable-HTTP MCP
JSON-RPC client (`cmd/nomos/main.go:583-810`): `initialize` → session ID →
`tools/call`/`tools/list`, pooled **per chat session** (not globally shared)
specifically so one task's long-running `run` call (up to a 10-minute
timeout) doesn't serialize every other task behind one mutex.
### 5.5 Server-Sent Events (`internal/httpapi/sse.go`)
Delivery mechanism: Postgres `LISTEN/NOTIFY`, not polling. A dedicated
pooled connection issues `LISTEN oikos_events`; on notify, the full row is
fetched and fanned out to all live subscribers over non-blocking channel
sends (slow subscribers drop frames, recovered via `Last-Event-ID` replay
against a 10,000-entry in-memory ring buffer, falling back to a full DB
query if the buffer doesn't cover the gap).
**Deliberate protocol choice:** the SSE `event:` field is *never* set — all
events arrive on the generic `onmessage` handler with `type` embedded in the
JSON payload, because a named `event:` field only fires
`addEventListener(type)` listeners and the frontend listens uniformly via
`onmessage`. There is no fixed enum of event types in `sse.go` itself; types
are whatever gets written to the `events` table. Task-relevant types
introduced by the goal-oriented redesign:
| Type | Payload |
|---|---|
| `task.status` | `{ status, outcome?, summary? }` |
| `goal.set` | `{ goal }` |
| `plan.proposed` | `{ steps:[{seq,title,detail,target_slug}] }` |
| `plan.step.started` / `plan.step.finished` | `{ step_id, seq, status, execution_id? }` |
| `question.raised` / `question.answered` | `{ question_id, prompt?, context?, answer? }` |
| `entity.touched` | `{ slug, tool }` — emitted for every tool call via `withActivityLogging`, zero agent-side effort |
| `knowledge.recorded` | `{ title, about, outcome }` |
| `approval.created` | emitted from `classifyAndGate` when a `run` call queues for approval |
| `health.changed` | entity health transition |
### 5.6 Web SPA → API interface
`web/src/lib/config.ts`'s `fetchWithAuth` is the single wrapper every API
call in the frontend goes through: resolves an OIDC token first (refreshing
if expired), falls back to the static bearer token in `localStorage`
(`oikos_token`), injects `Authorization: Bearer …`. A 401 on an
OIDC-authenticated request triggers one forced `oidcLogout()` before
falling back. Config (API URL + token) is resolved **per request**, not
memoized at import time, so the same bundle works same-origin (production,
Vite dev proxy) or cross-origin (the Wails desktop webview).
### 5.7 External interfaces
| Interface | Protocol | Direction | Notes |
|---|---|---|---|
| OpenRouter | HTTPS, OpenAI-compatible chat completions | Nomos → OpenRouter | `data_collection: deny` pinned; model configurable via `NOMOS_MODEL`, default `deepseek/deepseek-v4-pro` |
| Matrix homeserver | Matrix client-server API (`PUT /rooms/.../send/m.room.message`, `GET /relations/.../m.annotation`) | notifier ↔ Matrix | ✅/❌ (and 👍/👎, "approve"/"deny") reactions turned into a normal `POST /api/v1/approvals/{id}/decision` call — same code path as a console click |
| Infisical | Infisical SDK v0.8.0, UniversalAuth machine identity | `api` → Infisical | see [ADR-0010](../adr/0010-infisical-with-sops-fallback.md) |
| Gitea webhook | `POST /deploy`, `X-Hub-Signature-256` HMAC-SHA256 | Gitea → `cmd/webhook` (:9797) | verified with `hmac.Equal`; triggers `scripts/deploy.sh` asynchronously, responds `202` immediately |
| SSH | standard SSH protocol, key-based | `api` (scheduler + actuation) → fleet hosts/LXCs | **host key verification is disabled** (`InsecureIgnoreHostKey`) — flagged in §8 |
---
## 6. Behavior
*Concern: how does a task, a signal, an execution, an approval move
through time? This does not repeat [ADR-0014](../adr/0014-entity-model.md)'s
entity-type hierarchy, full relationship catalog, or ERD — those are
unchanged and still authoritative. What follows is the behavioral layer:
state machines and sequence/interaction diagrams, refreshed against what
this pass verified in the running code as of 2026-07-16, plus two
sequences ADR-0014 doesn't cover.*
### 6.1 Signal state machine — the one domain type with an enforced guard
```mermaid
stateDiagram-v2
[*] --> raised
raised --> acknowledged
raised --> muted: mute_until set
raised --> resolved: condition cleared
acknowledged --> acting: classification exists
acknowledged --> muted
acknowledged --> resolved
acting --> resolved: verification passed
acting --> raised: retry budget
acting --> failed
failed --> acknowledged: operator retry
muted --> raised: mute_until expired
resolved --> [*]
```
`internal/domain/signal.go` is the **only** domain type with a real guard:
`ValidSignalTransitions` (a map of allowed next-states) backs a
`(*Signal) CanTransition(to string) bool` method. Every other domain type
with a documented lifecycle (`Execution`, `Approval`, `Pattern`, `Skill`) has
the states defined as constants but **no code enforces the arrows** — the
state machine is descriptive, not guarded, and call sites are trusted to
only make legal moves.
**Even the one guarded type is bypassed by its main caller.** The
scheduler's `resolveSignal` (`scheduler.go:206-225`) runs
`UPDATE signals SET state='resolved' WHERE state='raised'` directly — it
never calls `Signal.CanTransition`. In this specific case the transition
(`raised → resolved`) is one the map does allow, so no invalid state is
reached today, but the guard providing zero actual protection is worth
knowing before relying on it as a safety net for a future code change that
adds an illegal transition elsewhere.
**Dedup mechanism** (unchanged from ADR-0014): a partial unique index —
`UNIQUE (target_entity_id, kind) WHERE state NOT IN ('resolved','failed')`
enforces at most one open signal per (entity, kind) at the database level,
independent of the Go-level state machine.
### 6.2 Execution state machine — schema-defined, convention-enforced
```mermaid
stateDiagram-v2
[*] --> proposed
proposed --> approved
proposed --> auto_approved
proposed --> denied
approved --> executing
approved --> expired
auto_approved --> executing
executing --> verified
executing --> failed
executing --> timed_out
executing --> cancelled
timed_out --> verifying
verifying --> verified
verifying --> failed
failed --> rolled_back
failed --> rollback_failed
verified --> [*]
denied --> [*]
expired --> [*]
cancelled --> [*]
rolled_back --> [*]
rollback_failed --> [*]
```
Corrected against a direct read of `seeds/ontology.yaml`'s registered
`execution` lifecycle (see [ontology.md §11c](ontology.md#11c-lifecycle-state-machines)
for the full state machine with its named `requires:` gates) — the
earlier version of this diagram omitted `verifying` as a state distinct
from `executing`, and the `timed_out → verifying` recovery path ("check if
the command completed anyway").
`domain.Execution`'s constants (`proposed/approved/executing/verified/
failed/timed-out/rolled-back/cancelled/expired`) are never actually written
by the live path — `internal/httpapi/phase3.go`'s `executeApprovedAction`
writes ad-hoc SQL string statuses (`'running'`, `'completed'`, `'failed'`,
`'pending_approval'`, `'cancelled'`) that overlap but don't map 1:1 onto the
domain constants (`'completed'`/`'running'` vs. `verified`/`executing`).
Treat the diagram above as the *intended* shape, and the actual column
values as whatever `phase3.go` and `mcp/server.go`'s call sites happen to
write — there's no single source of truth enforcing agreement between them.
### 6.3 Approval state machine
```mermaid
stateDiagram-v2
[*] --> pending
pending --> approved
pending --> denied
approved --> revoked
approved --> expired
denied --> [*]
revoked --> [*]
expired --> [*]
```
Unchanged from ADR-0014 and still accurate — `internal/httpapi/phase3.go:
DecideApproval` is the single write path for `pending → approved/denied`,
and the expiry sweep in `internal/notifier/notifier.go:90-93` handles
`pending → expired` inline during its poll pass.
### 6.4 Sequence — the `run` primitive, end to end
Supersedes ADR-0014 §3's OODA sequence, which predates the command
classifier and the `request_execution` retirement. This is the live path as
of 2026-07-16, for a `config_mutation` command with no pre-existing assent
window (the "operator must approve" case — the assent-window
already-granted case is the same minus the queue/wait steps):
```mermaid
sequenceDiagram
participant Op as Operator
participant N as Nomos agent
participant API as api on 8090
participant DB as Postgres
participant Mx as Matrix
participant H as Target host
Op->>N: chat: "restart caddy"
N->>API: MCP run(target=lxc:caddy, command="systemctl restart caddy", purpose=...)
API->>API: classifyAndGate: P1 plan-first check
API->>API: dedup check, P5 anti-flood check
API->>API: policy.ClassifyCommand → config_mutation\n(scope override: service:caddy always escalates)
API->>DB: no assent window active → INSERT executions (pending_approval)\nINSERT approvals (pending)
API-->>N: "queued for approval, execution_id=..."
N-->>Op: "queued — needs your approval"
par Matrix path
API->>DB: (notifier poll) SELECT pending approvals
DB-->>API: approval row, no alert_sent_at yet
API->>Mx: PUT room message: risk_class=config_mutation, ✅/❌
and Console path
Op->>API: GET /api/v1/approvals (control-room UI)
end
Op->>Mx: reacts ✅
Mx-->>API: (notifier poll) GET relations → match ✅
API->>API: POST /api/v1/approvals/{id}/decision (self-call, same code as console)
API->>DB: UPDATE approvals SET status=approved
API->>H: SSH exec systemctl restart caddy
H-->>API: exit 0
API->>DB: UPDATE executions SET status=completed
API->>DB: INSERT audit_log
API-->>N: (auto-continuation worker polls nomos_plan_executions) execution result
N-->>Op: "caddy restarted, verified healthy"
```
Note the **dual path to the same decision endpoint**: a Matrix ✅ reaction
and a console click both terminate in the identical
`POST /api/v1/approvals/{id}/decision` call — the notifier's
`callDecideApproval` (`notifier.go:204-228`) makes an HTTP call to itself
rather than reimplementing the decision logic, so there is exactly one
place that decides what "approved" does.
### 6.5 Sequence — plan auto-continuation ("the system is the event loop")
This is the mechanism behind REQ-ACT-5: after a plan is approved once, the
operator is not required to say "continue" after every asynchronous step.
```mermaid
sequenceDiagram
participant N as Nomos agent loop
participant W as continuation worker
participant DB as Postgres
participant API as api
N->>API: run(...) for plan step 2
API-->>N: execution_id=E2, status=pending (async)
N->>DB: INSERT nomos_plan_executions(execution_id=E2, session_id=S, continued_at=NULL)
N-->>N: turn ends (nothing left to say yet)
loop worker poll
W->>DB: SELECT nomos_plan_executions WHERE continued_at IS NULL
DB-->>W: E2 still pending
end
API->>DB: (async, elsewhere) executions.status → completed for E2
W->>DB: SELECT nomos_plan_executions WHERE continued_at IS NULL
DB-->>W: E2 now terminal
W->>N: re-invoke agent loop for session S with E2's result
N->>DB: UPDATE nomos_plan_executions SET continued_at=now()
N->>API: run(...) for plan step 3 (no operator input needed)
```
The `nomos_plan_executions` table (migration 017) deliberately has **no
hard foreign key** on `execution_id` — the header comment explains this is
to avoid a race between the API creating the execution entity and Nomos
recording the link breaking the insert. This is a documented, intentional
looseness, not an oversight.
### 6.6 Sequence — a Task from goal to knowledge deposit
```mermaid
sequenceDiagram
participant Op as Operator
participant N as Nomos
participant DB as Postgres
Op->>N: "install jellyfin on strong"
N->>DB: get_entity_knowledge(lxc:strong) — prior learnings first
N->>N: set_goal("install jellyfin on strong")
N->>DB: session.status = planning
N->>N: propose_plan([{seq:1,...}, {seq:2,...}, ...])
N->>DB: session.status = awaiting_approval
Op->>N: approves plan (opens assent window)
N->>DB: session.status = executing
loop each step
N->>N: run(...) or ask_operator(...) if a real decision is needed
N->>DB: update_plan_step(seq, status)
end
N->>DB: upsert_knowledge("jellyfin install on strong: gotcha X")
N->>DB: link knowledge —about→ lxc:strong, —outcome_of→ task:S\n(tagged success/failure)
N->>N: complete_task(outcome=success, summary=...)
N->>DB: session.status = done
```
This is the mechanism behind REQ-LRN-4 — the compounding knowledge loop is
real and structurally wired (unlike the pattern-learning pipeline in
§3.2/§9, which is algorithmically real but never executes). Its weak
point, per an audited session referenced in §9, is compliance: the loop
depends on the agent actually calling
`upsert_knowledge`/`update_entity_attributes` at the end of a task, which is
SOUL-nudged rather than structurally enforced the way `propose_plan`'s
gating is.
### 6.7 Infrastructure lifecycle
The 8-state `planned → provisioning → active → migrating → deprecated →
destroyed` machine (with `failed` as a recovery/write-off branch) and its
precondition table are unchanged from
[ADR-0014](../adr/0014-entity-model.md) §5 and not reproduced here.
---
## 7. Verification & Validation
*Concern: how do we know it works, and what isn't checked yet? MBSE
distinguishes **verification** ("did we build the system right" — does it
meet its specified requirements) from **validation** ("did we build the
right system" — does it do what the operator actually needs).*
### 7.1 Verification — automated
**CI pipeline** (`.gitea/workflows/ci.yml`), triggered on push to `main`
and on PRs. One job, `build-test`:
1. Spin up a TimescaleDB service container.
2. `go vet ./...`
3. `golangci-lint`**advisory** (`continue-on-error: true`)
4. `govulncheck`**advisory**
5. Generated-code drift check (`make generate-check` — catches stale
`oapi-codegen`/`sqlc` output committed out of sync with its source)
6. `go build ./...`
7. `go test -race -covermode=atomic ./...` with a coverage report
A second job, `docker-build`, verifies `compose/oikos/Dockerfile` builds (no
registry push — see §4.5 for why deploy doesn't need one).
**Coverage is measured, not gated.** The report is informational; an
80%-on-`policy`/`learning`-packages enforcement gate is noted as deferred
("Phase 3") rather than active. This means REQ-DEC-1/REQ-DEC-2 (classifier
correctness) currently rely on `internal/policy`'s own test file plus the
eval suite (below) for confidence, not a CI-enforced coverage floor.
A separate workflow, `desktop.yml`, builds and (on tag push) releases the
Wails desktop binary — see §4.5.
**Health checks as continuous verification.** The scheduler's 6 probe
kinds (`http`, `tcp`, `disk`, `cert-expiry`, `ping`, `ssh-script`) are,
functionally, a continuously-running verification suite against the
*deployed fleet* rather than the codebase — every 30s, every enrolled
entity is re-verified against its expected health, independent of whether
any code changed. The 18 scripts under `checks/` (`cpu_check.sh`,
`disk_usage_check.sh`, `docker_health_check.sh`, `zfs_check.sh`, …) are the
concrete verification procedures for the `ssh-script` kind, auto-deployed to
every enrolled client via `tools/setup-checks.sh` (AGENTS.md §8).
**Agent evals** (`evals/*.yaml`, `cmd/nomos/eval`) — golden-conversation
evals for the Nomos agent loop, **structural** assertions only (tool-call
sequences, plan steps, writeback, completion); text quality is explicitly
out of scope since it's model-dependent. Each run costs real OpenRouter
credits (~$0.010.05/conversation), so these are not part of CI — they're
an operator-invoked regression suite:
```
go run ./cmd/nomos/eval -gateway http://localhost:8092 -manifest evals/*.yaml
```
| Eval | What it guards against |
|---|---|
| `golden.yaml: trivial_readonly` | Ceremony creep — a one-line question shouldn't trigger a plan or a `run` call |
| `golden.yaml: plan_advances_on_proceed` | The original operator-reported bug: re-proposing a plan instead of advancing it on "proceed" |
| `golden.yaml: ui_complaint_no_rerun` | Approval-vocabulary ("go ahead") correctly recognized as assent, not misfired re-execution |
| `golden.yaml: knowledge_preferred_over_rerun` | REQ-ORI-4/REQ-LRN-4 in practice — the agent must call `search_knowledge` before re-probing 20 LXCs it already has a same-day audit for |
| `iteration-followup.yaml`, `iteration-readonly.yaml`, `plan-always-readonly.yaml`, `no-plan-no-run.yaml` | Additional plan/iteration-boundary regressions |
This is the closest thing in the repo to **validation** of REQ-ACT-5 ("one
approval → autonomous execution") and REQ-DEC-1-adjacent agent behavior —
it tests the *agent's* use of the gated primitives, not the primitives
themselves.
**Database migration safety.** [ADR-0008](../adr/0008-forward-only-migrations.md):
all 20 migrations are forward-only and idempotent (`IF NOT EXISTS`,
`ON CONFLICT DO NOTHING` throughout), verified implicitly by every
`migrate` one-shot container run on every deploy (§7.2) — a migration
that isn't idempotent fails loudly on the next deploy rather than silently
corrupting state.
### 7.2 Verification — deployment-time
```mermaid
flowchart LR
PUSH["push to main\n(webhook or 2-min poller)"] --> DUMP["pg_dump\npre-deploy-SHA.sql\n(best-effort, warns not fails)"]
DUMP --> PULL["git pull origin main"]
PULL --> SANITY["sanity-check last commit\nhas a message"]
SANITY --> BUILD["docker compose --profile full build"]
BUILD --> UP["docker compose --profile full up -d\n--remove-orphans (rolling restart)"]
UP --> POLL["poll /healthz\nup to 60s (default)"]
POLL -->|healthy| DONE(["deploy complete"])
POLL -->|never healthy| FAIL(["exit non-zero\nNO automatic rollback"])
```
**There is no automatic rollback on health-check failure.** `scripts/deploy.sh`
exits non-zero if `/healthz` never returns healthy within the retry budget,
but reverting to the previous SHA + restoring the pre-deploy dump is a
**manual, operator-invoked** step via `scripts/rollback.sh <previous-sha>`
(documented in [docs/operations/rollback.md](../operations/rollback.md)).
This is a deliberate scope boundary, not a gap — REQ-NFR-5 requires the
*path* to exist and be tested, not that it fire automatically.
`scripts/watchdog.sh` (separate 5-minute cron, outside the Docker stack —
"watching the watcher") probes `/healthz` on two paths (LAN direct IP and
NetBird mesh IP) and only pages via Matrix if **both** fail for 3
consecutive runs — a single-path failure is logged but not alerted,
treating it as a probable transient NetBird/Caddy issue rather than a real
outage.
### 7.3 Verification — what's not covered
- **No test coverage identified for `internal/actuator` or
`internal/learning`'s integration into a running process** — because
there is no such integration to test (§3.2). Any unit tests on these
packages verify the algorithms in isolation, not that the system behaves
as the ADRs claim end-to-end.
- **No automated check that `policy.ClassifyCommand`'s denylist actually
blocks every pattern the plan (`general-gated-execution`) claims it
blocks** — the regex list in `internal/policy/command.go` is the
authority; there's no eval or test enumerated in this pass that walks the
full pattern list and asserts each one classifies as `destructive`. This
would be a cheap, high-value addition (§9.4).
- **No automated check that `autonomy_settings` kill-switches are honored**
— appropriate, since (per §8) they currently aren't, on the live path. A
test asserting they *are* honored would currently fail correctly, which
is itself informative.
- **CI coverage gate is informational, not enforced** (§7.1).
### 7.4 Validation — does the system do what the operator needs
The clearest validation evidence in the repo is the trail of
operator-reported problems → shipped fixes in `plans/done/` and the active
`plans/` UX/reliability audits — e.g. the original "approve every step" pain
point directly drove the Task/plan-approval redesign (§1.3 item 4), and a
specific audited session (348 tool calls, zero
`update_entity_attributes`/`create_relationship` calls despite discovered
drift) directly drove scrutiny of the knowledge-loop's compliance (§6.6).
This is validation by **operator-in-the-loop iteration**, not a formal
acceptance-test suite — consistent with a single-operator homelab system,
but worth naming explicitly as the validation methodology rather than
implying a UAT-style process that doesn't exist here.
---
## 8. Risk & Safety
*Concern: what actually stops this system from doing something dangerous,
and where does that break down? Oikos exists to act autonomously on real
infrastructure with no human in the loop for most decisions — this section
is the honest account of what keeps that safe today, versus what the
schema/ADRs describe as the design intent. Findings marked **verified**
were checked directly against source during this pass (grep + read);
findings marked **per research pass** were not independently re-confirmed
line by line — high-confidence, not certain.*
### 8.1 The kill-switch gap (verified, most important finding in this model)
`seeds/policy.yaml` defines a global autonomy toggle
(`global.auto_act: "off"` at cold start) and per-entity hard blocks
(`never_auto_act.service:caddy/dns/authentik`,
`never_auto_act.host:hubris/strong` — described in the policy file itself as
"hard blocks"). These are read from the `autonomy_settings` table by exactly
two code paths:
- `internal/policy/classify.go` (`ClassifySignal`) — **dead code, zero
callers anywhere in the codebase.**
- `internal/actuator/actuator.go`**never started.** No `cmd/oikos` role
and no `docker-compose.yml` service runs `actuator.Run()`.
The live, actually-executing gate for every mutation —
`internal/mcp/server.go`'s `classifyAndGate`, invoked by the `run` MCP
tool — reads `autonomy_settings` too, but **only** for its own
session-scoped `assent_window.*` / `destructive_window.*` keys (per-task
approval grants), never for `global.auto_act` or `never_auto_act.*`.
**Practical consequence:** an operator who sets `never_auto_act.service:
caddy = true`, believing it hard-blocks any action against `service:caddy`,
gets no such protection against an agent-initiated `run` call today. The
only things actually standing between an operator's chat message and a
command running against `service:caddy` are:
1. the plan-first gate (a plan must exist),
2. `ClassifyCommand`'s regex classification of the *literal command text*,
3. the approval/assent-window requirement for anything above `read_only`,
4. the `scope_entity` override in `seeds/policy.yaml` that escalates
`caddy`/`dns`/`authentik` restarts to `config_mutation` regardless of the
generic per-action rule.
Layer 4 means an ordinary `systemctl restart caddy` still requires operator
approval today — so the practical exposure is narrower than "the kill
switch does nothing" — but it is exposure the policy schema's own language
("hard block", "never") does not accurately describe, and it is exposure
that widens for any action *not* covered by a `scope_entity` override.
**This should be treated as a real defect, not a documentation nuance**
either wire `global.auto_act`/`never_auto_act.*` into `classifyAndGate`, or
rewrite `seeds/policy.yaml`'s language and the operator-facing UI to state
plainly that these settings currently gate only the (unused) autonomous
auto-act pipeline.
### 8.2 Defense in depth that *is* live on the `run` path
Layered, in the order a `run` call actually passes through them
(`internal/mcp/server.go:classifyAndGate`, per research pass, corroborated
by direct grep of `command.go`'s pattern list):
1. **Plan-first gate (P1).** A session-scoped call to `run` is refused
unless the session already has a proposed plan — closes the original gap
where `pct_exec` ran ad-hoc SSH with zero classification (the incident
that motivated the whole `general-gated-execution` redesign). Does
**not** apply to direct/scripted MCP calls with no `_session_id`.
2. **Dedup.** An identical pending-approval command already queued for the
same target is refused rather than double-queued.
3. **Anti-flood (P5).** A second `config_mutation` approval cannot be queued
while one is already pending for the session — bounds how much an
operator can be asked to approve at once.
4. **Command classification** (`internal/policy/command.go`, verified by
direct read): a **denylist** of destructive shell patterns
(`rm -rf`/`rm -fr`, `dd of=`, `mkfs`, `wipefs`, `shred`,
`pct destroy`/`qm destroy`, `zpool destroy`, `lvremove`/`vgremove`/
`pvremove`, `DROP TABLE`/`TRUNCATE TABLE`, redirects to raw block devices,
`shutdown`/`reboot`/`halt`/`poweroff`, fork bombs, `chmod -R 000`,
`iptables -F`/`ufw disable`, and reads of credential files
`id_rsa`/`id_ed25519`/`*.pem`/`shadow`/`*.age`) forces `destructive`
regardless of what the caller declares; an **allowlist** of read-only
lead verbs (`cat`, `ls`, `ps`, `docker ps/inspect/logs`,
`systemctl status`, `git status/log/diff`, `curl -I`, …) permits
`read_only` only if *every* segment of a compound command matches it;
everything else defaults to `config_mutation` ("when in doubt,
escalate").
- Documented deliberate omission: `curl|sh` was removed from the
denylist because it's a common legitimate install pattern
(get.docker.com); it falls to `config_mutation` gating instead of an
outright block.
5. **Assent / destructive windows.** `config_mutation` auto-runs only inside
an assent window opened by the operator approving the containing plan;
`destructive` auto-runs only inside a narrower, target-and-session-scoped
window opened by a **typed confirmation phrase** distinct from ordinary
assent — and this requirement holds even for steps inside an
already-approved plan.
6. **Risk can only go up, never down.** `riskRank(computed, declared)` takes
the max — a model that under-declares risk on a call cannot talk its way
past the classifier's own assessment.
7. **Audit trail.** Every `run` call, whatever the outcome, produces an
`execution` entity, an `executions` row, and (via `withActivityLogging`)
an `agent_activity` row — nothing routes around the ledger.
This is a genuinely defensible design *for the threat model it targets*
(an LLM agent making mistakes or being subtly misled by ambiguous
instructions). It is explicitly **not** designed to catch perfectly
laundered malicious shell — the plan authoring this design says so outright:
perfect static classification of arbitrary shell is acknowledged as
impossible, and default-escalate + denylist + always-on audit is the stated
mitigation, not a claim of completeness.
### 8.3 Threat model & trust zones ([ADR-0007](../adr/0007-threat-model.md))
- Trust boundaries are Docker networks (`net-front`/`net-data`/`net-ops`
§4.3), not host-level firewalling.
- The agent (Nomos) holds no SSH key — verified structurally: the key lives
only in the `api` container's environment, read by
`internal/mcp/server.go`'s `initSSH()`. Nomos can only reach a host by
asking `api` to run something through the gated `run` tool.
- Caddy is a trust root for browser traffic but the API independently
validates OIDC JWTs — defense in depth, not sole reliance on the edge
proxy.
- Documented residual risks in the ADR: a LAN break-glass path carries
plaintext, and there's a shared-Postgres single point of failure across
every service.
- **Stale terminology note:** ADR-0007 refers to the agent as "Hermes"
throughout ("Hermes holds no SSH keys", "the Hermes gateway remains
mesh-only") — this predates the Hermes→Nomos rename
(migration `014_rename_agent_hermes_to_nomos.up.sql`). The architectural
claims are still accurate; the name is not. Flagged here rather than
silently "corrected," since the ADR itself should not be edited after
acceptance (repo convention, see [docs/adr/README.md](../adr/README.md)).
### 8.4 Known open security gaps
These come from `plans/2026-07-08-oikos-gaps-and-improvements.md` and
`plans/2026-07-11-nomos-agent-code-review.md` (per research pass — not
independently re-verified in this pass beyond what's cross-checked
elsewhere in this document). Status reflects each plan's own
last-updated header.
| ID | Gap | Status |
|---|---|---|
| **C1** | Nomos's own HTTP gateway (`:8092`) has **zero authentication**`/chat`, `/query`, `/sessions` are open to anyone on the LAN/mesh, including the ability to grant chat-assent approvals with no credential check at all. | **Open, explicitly deferred** by operator instruction ("leave auth out for this round of fixes") — a scoping decision, not an oversight, but still a live gap. See §5.4. |
| B1 | Unauthenticated enrollment can leak a newly-enrolled node's age private key. | Open |
| B2 | Fake/placeholder Infisical credentials returned to enrollees. | Open |
| B3 | Nomos's `mesh_only: true` config exists but is never enforced. | Open |
| B4 | SSH host key verification disabled (`InsecureIgnoreHostKey`) for actuation SSH. | Open |
| B5 | `list_my_secrets` enumerates all node pubkeys without scoping to the calling client. | Open |
| A2 | Matrix notifier can flood: `alert_sent_at` guard is written *after* send, so a failed UPDATE re-sends on retry; no approval dedup; unbounded reaction-polling API calls. | Open |
Prioritized fix order per the gaps plan: A1 (fixed) → A2 → B1/B2/B3 →
D (mostly resolved) → remaining items.
### 8.5 What safety machinery exists but doesn't run
Cross-reference to §3.2 for the full allocation table; the safety-relevant
subset:
- **Circuit breaker** (`internal/actuator/actuator.go:156-202`) — threshold
+ cooldown logic to stop hammering a target after repeated failures.
Fully implemented, guards only the unused auto-act loop. **The live `run`
path has no equivalent** — nothing currently stops an agent from retrying
a failing command against the same target in a tight loop beyond the
dedup/anti-flood gates in §8.2, which guard *queueing*, not *repeated
execution* of already-approved or auto-run commands.
- **SSH error classification & context-aware timeouts**
(`internal/actuator/ssh.go`) — network/auth/timeout/remote error
classification, per-step timeout defaulting. Zero callers; the live SSH
executors (`internal/httpapi/phase3.go:sshExec` and its `mcp/server.go`
counterpart) are simpler, independently-written implementations with a
flat 10-minute hard cap and less granular error classification.
- **Learning's anomaly quarantine** (§3.2, F5.2) — a same-pass burst of
>10 identical-key feedback events gets quarantined rather than trusted.
Algorithmically real, never executes (learning loop not started).
### 8.6 Safety properties that *are* structurally guaranteed
Worth stating plainly, not just gaps:
- **Learning cannot self-authorize.** [ADR-0006](../adr/0006-learning-proposal-only.md):
the learning engine's DB role has no grants on governance/autonomy
tables — a structural (role-permission), not merely logical, guarantee.
Currently moot in practice since the engine doesn't run (§8.5), but the
guarantee would hold the moment it's wired back in, without further code
changes, per the ADR's design.
- **Destructive actions require an explicit typed phrase, always**,
independent of plan approval — verified in §8.2 item 5, this is not
bypassable by pre-authorizing a plan that merely *lists* a destructive
step.
- **The agent cannot hold an SSH key even if compromised** (§4.3) — the
blast radius of a fully-compromised Nomos process is bounded by what the
gated `run` tool will classify and allow, not by what SSH access the
process itself has (it has none).
- **Every mutation is audited whether it succeeds, fails, or is denied** —
there is no silent-failure path in `classifyAndGate` that skips the
ledger.
### 8.7 Recommended reading order if triaging this for real
1. §8.1 (kill-switch gap) — highest-severity, cheapest to reason about, not
yet fixed.
2. §8.4 table — known, triaged, prioritized elsewhere; don't re-discover
these, just track them.
3. §8.5 — lower urgency than §8.1/§8.4 (the missing circuit breaker matters
only once the auto-act path or a much higher-volume agent workload
exists), but worth deciding *deliberately* whether to port the existing
`actuator/ssh.go` machinery into the live path or delete it, rather than
letting it silently rot as an attractive nuisance for the next person who
assumes it's load-bearing.
---
## 9. Roadmap & Traceability
*Concern: what's code-real vs. dead-code vs. schema-only, and where is
this headed? Two things live here: the authoritative status matrix every
other section points back to, and the north-star direction. This
supersedes the equivalent table in [ADR-0014](../adr/0014-entity-model.md)
§6, which predates the `general-gated-execution` rebuild and the discovery
that `actuator`/`learning` are unwired.*
### 9.1 The north star: general gated execution
Direction: **"general gated execution (unlimited actions, classifier-gated),
not a fixed action enum."** `plans/2026-07-10-general-gated-execution.md`
is the design document and describes four layers:
| Layer | Description | Status |
|---|---|---|
| **Layer 0** | The general gated `run` primitive itself | ✅ Done — `ClassifyCommand`, `run` MCP tool, chat-assent, `request_execution` retired (commit `60effcb`, 2026-07-14) |
| **Layer 1** | Runbooks as *executable data* in the knowledge DB, not prose | ⚠ Mostly unimplemented — "the knowledge is inert w.r.t. action." `skill` entities carry a JSON `procedure` field; no execution engine reads or runs it |
| **Layer 2** | Learning closes the loop — successful ad-hoc sequences get promoted into runbooks | ⚠ Partial — the pattern-extraction algorithm exists (`internal/learning/learning.go`) and is faithful to [ADR-0006](../adr/0006-learning-proposal-only.md)'s design, but is never started by any process (§9.2) |
| **Layer 3** | Observability — live action feed, session digest, learning view, global ledger | ✅ Largely built (SSE event types, Activity timeline, Learning page) |
**The one item the plan's own header still marks open:** "revive auto-act"
— the Observe→Act autonomous direction (a signal triggering action without
an operator initiating a chat turn) is unimplemented in practice, because
`internal/actuator.Run()` is a literal
`{"success": true, "message": "stub execution"}` stub *and* is never
started. Only operator/agent-initiated `run` calls execute end-to-end
today. Reaching the north star requires either completing this stub and
wiring it into a `cmd/oikos` role, or making an explicit decision that
auto-act is out of scope and updating the plan/policy language accordingly
(§8.1 — this decision also has to resolve the kill-switch question, since
`actuator.go` is where that check currently lives).
### 9.2 Code-real / dead-code / schema-only matrix
Legend: ✅ live in production · ⚠ implemented, not wired in · 🔴 dead code
(zero callers) · ❌ schema/ontology only, no Go implementation.
| Component | Status | Evidence |
|---|---|---|
| Entity CRUD, relationships, ontology validation | ✅ | `internal/httpapi/impl.go`, unchanged from ADR-0014 |
| Lifecycle transitions + preconditions | ✅ | `impl.go`, unchanged from ADR-0014 |
| Scheduler observe loop (6 probe kinds, dedup, entity_status, stale sweep) | ✅ | `internal/scheduler/scheduler.go` |
| **`policy.ClassifySignal`** (signal → risk class, DB-rule driven) | 🔴 | Zero callers anywhere in the codebase — verified |
| **`policy.ClassifyCommand`** (command text → risk class, regex driven) | ✅ | Called from `internal/mcp/server.go:classifyAndGate` — verified, the actually-live classifier |
| `run` MCP tool + gating (plan-first, dedup, anti-flood, assent/destructive windows) | ✅ | `internal/mcp/server.go` |
| `request_execution` (old fixed-enum mutation tool) | 🔴 | Retired 2026-07-14, explicit "DO NOT re-register" guard left in place |
| **`internal/actuator.Run`** (auto-act loop, circuit breaker, kill-switch check) | 🔴 | Not started by any `cmd/oikos` role or compose service; its own executor is a hardcoded stub result |
| **`internal/actuator.ExecuteProcedure`** (SSH procedure runner, error classification) | 🔴 | Zero callers; live SSH execution is a separate, simpler implementation in `httpapi/phase3.go` / `mcp/server.go` |
| Live SSH execution (`sshExec` in `phase3.go` and `mcp/server.go`) | ✅ | Actual executor for approved/auto-run commands — duplicated across two files, no shared implementation |
| Approvals (create, decide, expire) | ✅ | `internal/httpapi/phase3.go` |
| `notifier.VerifyApprovalToken` | 🔴 | Zero callers; `phase3.go:DecideApproval` reimplements the same check inline |
| Notifier (Matrix alerts, reaction polling, token issuance) | ✅ | `internal/notifier/notifier.go` |
| **`internal/learning.Run`** (pattern extraction, Wilson bound, quarantine) | ⚠ | Algorithm faithful to ADR-0006; never started by any process — feedback accumulates, nothing processes it |
| Patterns/Skills list + patch endpoints | ✅ | `internal/httpapi/phase3.go` (read/admin surface exists independent of whether learning produces new rows) |
| `domain.Execution` state machine (constants) | ❌ (descriptive only) | No `CanTransition`-equivalent guard; live code writes ad-hoc string statuses that don't map 1:1 |
| `domain.Signal` state machine | ✅ (guarded) but bypassed | `CanTransition` exists and is real; `scheduler.go:resolveSignal` bypasses it with raw SQL (transition happens to be legal today) |
| `autonomy_settings` kill-switch (`global.auto_act`, `never_auto_act.*`) | 🔴 on the live path | Only read by `ClassifySignal` (dead) and `actuator` (unstarted) — §8.1 |
| `autonomy_settings` assent/destructive windows | ✅ | Different keys, same table, read/written live by `classifyAndGate` |
| Task/plan-step/question schema (migrations 018020) | ✅ | Fully wired: `set_goal`/`propose_plan`/`update_plan_step`/`ask_operator`/`complete_task`, SSE event types, `TaskContextPanel.svelte` |
| Plan auto-continuation worker | ✅ | `cmd/nomos/continue.go`, polls `nomos_plan_executions` |
| Knowledge write loop (`upsert_knowledge`, `about`/`involves` relationships) | ✅ (mechanism) / ⚠ (compliance) | Mechanism fully wired; a 348-tool-call audited session made zero `update_entity_attributes`/`create_relationship` calls despite discovered drift — compliance is SOUL-nudged, not enforced |
| SSE event stream (Postgres `LISTEN/NOTIFY` fan-out) | ✅ | `internal/httpapi/sse.go` |
| Blast radius (`blast_radius()` SQL fn) | ✅ | Called from both the dead classifier and the live `get_blast_radius` MCP tool — the SQL itself is shared and correct |
| Bearer/OIDC auth on `api` | ✅ | `combinedAuth`, no dev-open bypass (ADR-0015) |
| Auth on Nomos gateway (`:8092`) | 🔴 | None — open, deferred gap (C1) |
### 9.3 Documentation/code divergences found during this pass
| Divergence | Where | Recommendation |
|---|---|---|
| ADR-0007 calls the agent "Hermes" throughout | `docs/adr/0007-threat-model.md` | Leave as-is (ADRs aren't edited post-acceptance per repo convention) but don't cite it without noting the rename |
| `nomos/config.yaml` says `max_iterations: 15`; the Go constant actually enforced is `40` (`cmd/nomos/agent.go:24`) | config vs. code | Either wire the config value through or delete it so it stops implying control it doesn't have |
| Dual `agent_activity` logging — both `cmd/nomos/store.go`'s `logActivity` and `internal/mcp/server.go`'s `withActivityLogging` log the same forwarded tool call | per research pass, not independently re-verified this pass | Confirm whether this is intentional dual-audit-trail (nomos-side + api-side) or accidental duplication before relying on `agent_activity` row counts for anything quantitative |
| `plans/2026-07-14-activity-gaps.md` and `plans/2026-07-14-activity-timeline.md` still say `Status: Planned` in their own headers but are **not listed** in `plans/index.md`'s active table at all (unlike four sibling 07-14 plans, which are listed and marked Done) | plans/ | Likely superseded/absorbed by `post-fix-session-remainders.md` and the tool-timeline/unified-indicator work — worth an explicit note in `plans/index.md` either way so the ambiguity doesn't recur |
| `internal/ontology`/`seeds/ontology.yaml` mention relationship types `powers`/`monitors` and entity type `certificate` with no concrete instances or, in the `powers`/`monitors` case, no `relationship_types` row | ADR-0014 §6 (still accurate, unchanged) | Low priority — cosmetic ontology completeness |
### 9.4 Suggested next steps (informational — not a commitment, not a plan)
These are documentation-pass observations, offered for the operator to
triage into an actual plan if desired — not a proposal for unilateral
action:
1. **Decide the fate of `internal/actuator` and `internal/learning`.**
Either finish wiring them in (which would also require resolving the
kill-switch gap, since `actuator.go` is where that check currently
lives) or delete them and update ADR-0014/OIKOS.md's "Phase 3 — DONE"
claim, which currently overstates what's actually running.
2. **Fix the kill-switch gap** (§8.1) — either wire
`autonomy_settings.global.auto_act`/`never_auto_act.*` into
`classifyAndGate`, or rewrite the policy schema's "hard block" language
to describe what it actually gates today.
3. **Consolidate the two SSH executors** (`internal/actuator/ssh.go` vs.
`httpapi/phase3.go`'s `sshExec`) — the unused one has better error
classification and timeout handling; the live one is simpler. Picking
one and deleting the other removes a maintenance trap.
4. **Add a test enumerating `command.go`'s destructive-pattern list**
against a fixture of known-dangerous commands, closing the verification
gap named in §7.3.
5. **Resolve C1** (Nomos gateway auth) when the operator is ready to pick it
back up — currently a deliberate, tracked deferral, not a forgotten item.
### 9.5 Traceability summary
Every requirement in §2 carries a Status column that points into §9.2's
matrix by implication (✅/⚠/❌ map directly). Every function in §3.2's
allocation table cites the same evidence. §9.2 is the place to update first
when code changes — every other section's status markers should be treated
as **derived from**, not independent of, that matrix.
## Keeping this model current
This is a snapshot, not a live view. When you change:
- **What a component does** → update §3.2's allocation table and, if it
changes what's live vs. dormant, §9.2's matrix.
- **A gating/approval rule** → update §8 and re-check whether §8.1's
kill-switch finding is still accurate.
- **An MCP tool, REST route, or SSE event type** → update §5.
- **A state machine or the sequence of a core flow** → update §6.
If a change resolves one of the divergences or gaps named in §9.39.4,
remove it from that list rather than leaving it to rot as a stale finding —
the whole value of this model depends on it staying honest about current
state, the same discipline ADR-0014 modeled first.