Compare commits
114 Commits
desktop-0.
...
claude/fro
| Author | SHA1 | Date | |
|---|---|---|---|
| 9f4d645d06 | |||
| aee458ce83 | |||
| 58a11ca872 | |||
| aed068de12 | |||
| 8657ac5669 | |||
| e28e0e9ea3 | |||
| 6051fb4845 | |||
| 544afae77f | |||
| bd44626532 | |||
| b0cdf64bbf | |||
| d6b3d3c88b | |||
| 8615f2268f | |||
| 258b14dcbc | |||
| 646373a676 | |||
| 69964abe2e | |||
| 6806fac5fd | |||
| 7dc1c1ae39 | |||
| 8709e01dcb | |||
| c96c795126 | |||
| 463bdacf5c | |||
| fb39a48bef | |||
| a2410cf9c2 | |||
| d2950dd09d | |||
| 0a3654b08f | |||
| c3973e7ac9 | |||
| e3a0326c78 | |||
| 55781984c7 | |||
| 7012525ad6 | |||
| 127b939a95 | |||
| 6a8efd22bb | |||
| 876f181068 | |||
| df24cae507 | |||
| e4e426de7d | |||
| ca2ff56a25 | |||
| d6e180845c | |||
| 7ef8446825 | |||
| a9b3f844b2 | |||
| a3afbb96cf | |||
| d55bae17b9 | |||
| e3b5fdc358 | |||
| 3c3b12df5e | |||
| 3d99282897 | |||
| a8f04cc9e3 | |||
| 487f9ad358 | |||
| 3d7fa99560 | |||
| 844cfe5888 | |||
| f1dda7290a | |||
| 462fb4d77b | |||
| e3fa6736c0 | |||
| e8b30cddcf | |||
| 49dfaa77e6 | |||
| 1267c39ab1 | |||
| a5336c02e9 | |||
| dd3076a23a | |||
| 0b5b213b2a | |||
| 3b98097f58 | |||
| 3de359b85f | |||
| c5bee740ad | |||
| 337d577f00 | |||
| 5f82627fa8 | |||
| 5caf49bf48 | |||
| 24cc3b1f4e | |||
| b423cf4dea | |||
| 2ed169d239 | |||
| 44720b7b30 | |||
| c8c7705046 | |||
| 54f532f166 | |||
| b414722fc7 | |||
| cc266c238e | |||
| 9f40f19f25 | |||
| 04677fdf4b | |||
| cc6bcdceaa | |||
| 5b403141ea | |||
| 7847cdffd6 | |||
| dce19bd258 | |||
| 60effcb2fe | |||
| b446909ea5 | |||
| dd27630be3 | |||
| e78a3e9048 | |||
| 5e3e4eaf07 | |||
| 7087d1ffea | |||
| 6c7631d425 | |||
| c4ac0cb935 | |||
| 6090ef71d4 | |||
| 06c6f4eb8c | |||
| 7063c90898 | |||
| aca6b8bcc2 | |||
| eeb78ed3c6 | |||
| bcef4e6456 | |||
| 23535eac25 | |||
| bcf2b265c5 | |||
| cea67ccd15 | |||
| 2bd7de355b | |||
| 8b3fe02a10 | |||
| c8ef3793d7 | |||
| d7197c1952 | |||
| cac5524402 | |||
| 56e509d506 | |||
| 68011f9a06 | |||
| a395771960 | |||
| 5664a4bf29 | |||
| ff6608f9ea | |||
| 0271727709 | |||
| c31978042f | |||
| 5699a3f758 | |||
| 44c0145683 | |||
| de7eca8b6d | |||
| f6e2079a61 | |||
| 515c9b9174 | |||
| f1ac82255a | |||
| 62a337f3cc | |||
| 35ff3f37e1 | |||
| 8f121cfa1e | |||
| 1c3a800506 |
@@ -13,13 +13,13 @@ service itself.
|
||||
|
||||
## Source of truth
|
||||
|
||||
The homelab-context repo at `/opt/homelab-context/` is the single source of
|
||||
The homelab-context repo at `/opt/homelab/` is the single source of
|
||||
truth for:
|
||||
- Fleet topology (`inventory.yaml`)
|
||||
- Agent behaviour and conventions
|
||||
- Everything in this file
|
||||
|
||||
When in doubt, check `/opt/homelab-context/` first, or query the Oikos API/MCP
|
||||
When in doubt, check `/opt/homelab/` first, or query the Oikos API/MCP
|
||||
server directly (see [AGENTS.md](../AGENTS.md) §3-4) — the database is
|
||||
authoritative at runtime.
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Oikos — the operating model
|
||||
|
||||
Oikos (Greek: *household*) is the agent operating system layered on this
|
||||
repo. It is not new infrastructure: `inventory.yaml` is the kernel data
|
||||
structure, the `homelab` CLI and MCP server are the syscall surface, and
|
||||
repo. It is not new infrastructure: `seeds/inventory.yaml` is the kernel data
|
||||
structure, the Oikos REST API and MCP server are the syscall surface, and
|
||||
this page defines the rules everything above them follows.
|
||||
|
||||
Read this after [AGENTS.md](../AGENTS.md). Machine-readable companions:
|
||||
@@ -100,16 +100,16 @@ via the API's `/api/v1/graph` endpoint, and the Mermaid export at
|
||||
|
||||
The Oikos runtime was rewritten from Python to Go over 6 phases and is deployed
|
||||
in Docker on mac-mini. See
|
||||
[plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md](../plans/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md)
|
||||
[plans/done/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md](../plans/done/2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md)
|
||||
for the full plan. The Python codebase has been removed; all functionality runs
|
||||
in the Go binary.
|
||||
|
||||
**Phase 1 — Ontology + DB (DONE):**
|
||||
- `migrations/` (001–011): TimescaleDB hypertables, entity_status, CAGGs,
|
||||
retention policies, knowledge entities with FTS. Forward-only, idempotent.
|
||||
- `migrations/` (001–020, forward-only): TimescaleDB hypertables, entity_status, CAGGs,
|
||||
retention policies, knowledge entities with FTS. Idempotent.
|
||||
- `seeds/{ontology,inventory,policy,knowledge}.yaml`: DB-native bootstrap +
|
||||
DR export. Knowledge seed contains 36 documents, 6 investigations, and 12
|
||||
runbooks.
|
||||
DR export. Knowledge seed contents are not hardcoded here — count them
|
||||
from the seed or query the DB.
|
||||
- `blast_radius()` SQL CTE, type hierarchy, abstract types, relationship
|
||||
validation.
|
||||
- Go packages: `internal/db/`, `internal/ontology/`, `internal/domain/`,
|
||||
@@ -139,8 +139,8 @@ in the Go binary.
|
||||
|
||||
**Phase 4 — Agent / Nomos (DONE):**
|
||||
- Standalone Nomos MCP client binary (`cmd/nomos`) with gateway mode
|
||||
(:8092). Structured queries + natural-language routing to 15 MCP tools.
|
||||
Agent activity logging on every tool call. No SSH keys.
|
||||
(:8092). Structured queries + natural-language routing to the MCP tool
|
||||
list (see AGENTS.md §3). Agent activity logging on every tool call. No SSH keys.
|
||||
- `nomos/` directory with config, SOUL.md, homelab-ops skill.
|
||||
- Nomos Docker service in `docker-compose.yml` (profile: full).
|
||||
- Go packages: `cmd/nomos/`, `compose/nomos/`.
|
||||
|
||||
@@ -94,7 +94,12 @@ current phase status). To add a new capability:
|
||||
## SQL conventions
|
||||
|
||||
- Queries live in `internal/db/queries/*.sql` with `-- name: FuncName :exec`
|
||||
annotations for sqlc
|
||||
annotations for sqlc. Generated code in `internal/db/sqlcgen/` — never
|
||||
hand-edit. Call via `sqlcgen.New(pool).QueryName(ctx, params)`.
|
||||
- **sqlc is the default** for all DB access. Raw `pool.Query/Exec` with inline
|
||||
SQL is a documented carve-out for cases sqlc can't express: `LISTEN`/`NOTIFY`,
|
||||
dynamic WHERE-clause builders, `blast_radius()` (opaque return type), and
|
||||
`COPY`. All other DB access should go through sqlc queries.
|
||||
- Use `pgx/v5` driver. UUIDs use `pgtype.UUID`, timestamps use `time.Time`
|
||||
- CTEs for graph traversals (blast radius, dependency chains)
|
||||
- CAGGs and retention policies for TimescaleDB hypertables
|
||||
@@ -108,6 +113,23 @@ current phase status). To add a new capability:
|
||||
implement it in `internal/httpapi/impl.go`
|
||||
- Problem+JSON errors via `internal/httpapi/problem.go` — RFC 9457 format
|
||||
- Cursor pagination, If-Match/ETag, idempotency keys, SSE streaming
|
||||
- **Non-OpenAPI routes carve-out:** ~10 routes are registered manually on
|
||||
the chi router in `internal/httpapi/server.go` rather than generated from
|
||||
`openapi.yaml`. These fall into three categories:
|
||||
1. **Auth/infra** (`/healthz`, `/api/v1/auth/oidc-*`, `/oidc-callback`) —
|
||||
must bypass the auth middleware or aren't JSON API endpoints.
|
||||
2. **SSE override** (`/api/v1/events/stream`) — in the spec but
|
||||
re-registered manually because the strict handler can't `Flush()` per
|
||||
event.
|
||||
3. **Ad-hoc aggregations** (`/api/v1/knowledge/recent`,
|
||||
`/api/v1/knowledge/content/{id}`, `/api/v1/activity/recent`,
|
||||
`/api/v1/activity/session/{id}`, `/api/v1/learning/timeline`,
|
||||
`/api/v1/learning/trend`) — return derived/aggregate shapes that don't
|
||||
map cleanly to a schema type. If one of these stabilizes, promote it
|
||||
to `openapi.yaml` with a proper schema and migrate the `serve*`
|
||||
function to a strict handler.
|
||||
The full list with reasons is in the "Non-OpenAPI routes" comment block
|
||||
at the top of `NewHandler` in `server.go`.
|
||||
|
||||
## Testing philosophy
|
||||
|
||||
|
||||
@@ -1,48 +1,93 @@
|
||||
# Knowledge domain — schema
|
||||
|
||||
The knowledge domain is the durable, authoritative current-state documentation of the homelab: one
|
||||
page per node and per cross-cutting system, synthesized from live state and evidence. It answers
|
||||
"what exists and how does it work right now."
|
||||
The knowledge domain is the durable, authoritative current-state documentation of the homelab:
|
||||
narrative for every node and cross-cutting system, synthesized from live state and evidence. It
|
||||
answers "what exists and how does it work right now."
|
||||
|
||||
It follows the [LLM Wiki layer model](../../shared/llm-wiki.md) and the
|
||||
[writing-style](../../shared/writing-style.md) and [page-templates](../../shared/page-templates.md)
|
||||
rules.
|
||||
|
||||
## The narrative / substrate split
|
||||
## Source of truth — the database
|
||||
|
||||
The knowledge wiki is **narrative**. It sits alongside a **machine-readable substrate** that it
|
||||
describes but never contains. The split is load-bearing: several programs read the substrate at
|
||||
fixed paths, so the wiki reorganization never moves it.
|
||||
Per ADR 0003, the Postgres database is the single source of truth for all structured data **and**
|
||||
narrative knowledge. The narrative/substrate split of the Python era is gone: the DB holds both the
|
||||
structured graph (entities, relationships, status, metrics) and the narrative layer (documents,
|
||||
investigations, runbooks) in the `knowledge_entities` table.
|
||||
|
||||
| Layer | Location | Consumed by |
|
||||
|-------|----------|-------------|
|
||||
| Substrate — source of truth | `inventory.yaml` (root) | MCP server, `homelab` CLI, `oikos/` scheduler/drift/relations/gen-topology |
|
||||
| Substrate — generated host records | `inventory.yaml` (root) | Go `internal/mcp/` server, `bin/homelab`; the single source of truth |
|
||||
| Substrate — kernel + context cards | `oikos/` (code, `oikos/cards/`, `oikos/state.json`) | MCP `explain`, scheduler |
|
||||
| Narrative — synthesized wiki | `archive/knowledge/{hosts,containers,vms,infrastructure}/` | humans, agents via MCP `get_page` / `search_docs` |
|
||||
| Evidence — immutable sources | `knowledge/sources/` (references + investigations) | synthesis into wiki pages |
|
||||
| Concern | Where it lives | How it gets there |
|
||||
|---------|----------------|-------------------|
|
||||
| Knowledge content — documents, investigations, runbooks | `knowledge_entities` table (rows linked to `entities` via `documents` / `about` edges) | Seeded from `seeds/knowledge.yaml` at deploy; mutated at runtime via the API |
|
||||
| Seed manifest (bootstrap + DR) | `seeds/knowledge.yaml` | Hand-edited or regenerated; ingested idempotently (content-hashed via `seed_versions`) |
|
||||
| Structured graph — hosts, services, entity types, relationships | `entities`, `relationships`, `entity_types` tables | Seeded from `seeds/{ontology,inventory}.yaml`; mutated via API/MCP |
|
||||
| Archived narrative wiki (read-only history) | `archive/knowledge/` | Frozen 2026-07-07 when the DB became source of truth |
|
||||
|
||||
## Wiki pages
|
||||
### Seed ingest
|
||||
|
||||
- **Node pages** (`archive/knowledge/containers/<id>-<name>.md`, `.../vms/<id>-<name>.md`,
|
||||
`.../hosts/<name>.md`) follow the container/host template in
|
||||
[page-templates.md](../../shared/page-templates.md): opening definition, `## At a glance`,
|
||||
`## Role`, service/port map, storage, auto-deploy, `## Related`, `## Changelog`.
|
||||
- **Cross-cutting pages** (`archive/knowledge/infrastructure/<topic>.md`) follow the cross-cutting
|
||||
template: `## Why`, `## Components`, `## How to apply`, `## Gotchas`, `## Related`, `## Changelog`.
|
||||
- Each `inventory.yaml` host entry carries a `doc_page:` field pointing at its narrative page.
|
||||
Changing where a page lives means updating that field (read by `bin/homelab`).
|
||||
`seeds/knowledge.yaml` has three top-level lists — `documents`, `investigations`, `runbooks` — each
|
||||
entry carrying `slug`, `title`, `content` (markdown), and tags. `internal/knowledge/seed.go`
|
||||
ingests each entry by:
|
||||
|
||||
1. `getOrCreateEntity` — ensures the slug exists in `entities` (type `document` / `investigation` /
|
||||
`runbook`).
|
||||
2. `upsertKnowledgeEntity` — writes the markdown body into `knowledge_entities`, keyed by
|
||||
`content_hash` so re-ingest is a no-op when nothing changed.
|
||||
3. `createEdge` — links the knowledge entity to its subject(s) via `documents` (for `document`) or
|
||||
`about` (for `investigation`) edges. Runbooks bind to an `entity_type` via `applies_to_type`
|
||||
rather than to a single entity.
|
||||
|
||||
### Runtime mutation
|
||||
|
||||
Agents register or update knowledge through the API, not by editing the seed:
|
||||
|
||||
- `POST /api/v1/knowledge/{entity_slug}` — upsert a document/investigation on an entity
|
||||
(`upsert_knowledge` MCP tool).
|
||||
- `update_entity_attributes` — merge a discovered fact (IP, version, port) into an entity.
|
||||
- `create_relationship` — record a discovered edge (`depends-on`, `hosts`, `routes-to`).
|
||||
|
||||
> **Export gap.** `oikos export` regenerates `seeds/{ontology,inventory,policy}.yaml` from the DB
|
||||
> for version control, but **not** `seeds/knowledge.yaml`. Knowledge added via the API today lives
|
||||
> only in the DB until someone hand-edits the seed. Tracked as a follow-up.
|
||||
|
||||
## Knowledge kinds
|
||||
|
||||
- **Documents** (`document` entities, linked via `documents` edges) — node and cross-cutting
|
||||
narrative pages. Carry `at_glance` (structured attributes) and a parsed `changelog`. Follow the
|
||||
container / cross-cutting templates in [page-templates.md](../../shared/page-templates.md).
|
||||
- **Investigations** (`investigation` entities, linked via `about` edges) — incident evidence,
|
||||
written once at incident time. Sections: `## Summary`, `## Timeline`, `## Root cause`,
|
||||
`## Mitigations applied`, `## Open questions`.
|
||||
- **Runbooks** (`runbook` entities, bound by `applies_to_type`) — repeatable procedures. Carry
|
||||
`risk_class` and a JSON-schema-validated `procedure`. **Runbooks also live as `SKILL.md` files
|
||||
under `.agents/skills/<name>/`** — the DB row is the policy/lifecycle framing, the SKILL.md is
|
||||
the executable procedure the agent loads. See
|
||||
[the operations schema](../operations/schema.md).
|
||||
|
||||
## The two logs
|
||||
|
||||
- The per-page **`## Changelog`** records infrastructure changes and is machine-parsed
|
||||
(`get_changelog`, the Oikos ledger). Keep the `### YYYY-MM-DD — title` shape.
|
||||
- **`knowledge/log.md`** is append-only and records *documentation-maintenance* operations only
|
||||
(restructures, source ingests, lint sweeps): `## [YYYY-MM-DD] <op> | <summary>`. It never
|
||||
duplicates the Oikos change ledger (`oikos/ledger.py`).
|
||||
- The per-document **`## Changelog`** records infrastructure changes to that node. Keep the
|
||||
`### YYYY-MM-DD — title` shape so the parsed `changelog` field stays structured.
|
||||
- **`archive/knowledge/log.md`** is the append-only record of *documentation-maintenance*
|
||||
operations on the legacy wiki (restructures, source ingests, lint sweeps):
|
||||
`## [YYYY-MM-DD] <op> | <summary>`. It is frozen with the rest of `archive/knowledge/`; new
|
||||
doc-maintenance operations are recorded in the DB audit trail instead.
|
||||
|
||||
## Querying knowledge
|
||||
|
||||
Use MCP, not grep:
|
||||
|
||||
- `search_knowledge(query)` — ILIKE search over documents, investigations, and runbooks in
|
||||
`knowledge_entities`.
|
||||
- `get_entity_knowledge(entity_slug)` — every document, investigation, and runbook linked to one
|
||||
entity, in one call.
|
||||
- `get_entity(slug)` / `get_relations(entity)` — the structured graph around an entity.
|
||||
|
||||
Grep the clone only when MCP is unreachable, and prefer `archive/knowledge/` for historical
|
||||
narrative (it is not updated when the DB changes).
|
||||
|
||||
## Same-session update rule
|
||||
|
||||
A change to a node updates every page that references it in the same session — the node page, the
|
||||
section `README.md` table, the root `README.md`, the Caddy/DNS/ingress pages, the host page, and
|
||||
`inventory.yaml`. See [page-templates.md](../../shared/page-templates.md#same-session-update-rule).
|
||||
A change to a node updates the DB in the same session — the entity's attributes, the relationships
|
||||
that reference it, and any document whose `at_glance` or changelog should reflect the new state. See
|
||||
[page-templates.md](../../shared/page-templates.md#same-session-update-rule) for the legacy wiki
|
||||
equivalent (now scoped to `archive/knowledge/` history).
|
||||
|
||||
@@ -6,9 +6,10 @@ follows [writing-style](../../shared/writing-style.md); runbooks and plans use t
|
||||
exception.
|
||||
|
||||
Where each kind lives: runbooks are skills under [`.agents/skills/`](../../skills/); operator
|
||||
reference (command cheatsheet, enrollment, Hermes agent) lives in
|
||||
[`.agents/operations/`](../../operations/); investigations are sources under
|
||||
`knowledge/sources/investigations/`; plans stay in the repo-root `plans/` folder (below).
|
||||
reference (command cheatsheet, enrollment, Nomos agent) lives in
|
||||
[`.agents/operations/`](../../operations/); investigations are `investigation` entities in the DB
|
||||
(historically `archive/knowledge/sources/investigations/`); plans stay in the repo-root `plans/`
|
||||
folder (below).
|
||||
|
||||
## Plans always live in `plans/`
|
||||
|
||||
@@ -21,7 +22,7 @@ message.** An agent drafting a plan:
|
||||
3. On completion, moves it to `plans/done/` and updates the index status.
|
||||
|
||||
This is the single source for homelab design intent; keeping it in-repo means the plan is
|
||||
versioned, reviewable, and reachable by MCP `get_page`/`search_docs` like any other doc.
|
||||
versioned, reviewable, and reachable by MCP `search_knowledge` like any other doc.
|
||||
|
||||
## Runbooks
|
||||
|
||||
@@ -44,12 +45,14 @@ transition: "<from> -> <to>" # only for lifecycle runbooks
|
||||
|
||||
## Investigations
|
||||
|
||||
Incident records live in `knowledge/sources/investigations/YYYY-MM-DD-slug.md` and are **evidence sources** — written
|
||||
once at incident time, then linked from the changelogs of the nodes they implicate. Sections:
|
||||
`## Summary`, `## Timeline`, `## Root cause`, `## Mitigations applied`, `## Open questions`. Resolved
|
||||
incidents move to `knowledge/sources/investigations/archive/`.
|
||||
Incident records are `investigation` entities in the DB, linked to the entities they implicate via
|
||||
`about` edges. They are **evidence sources** — written once at incident time, then back-linked from
|
||||
the changelogs of the nodes they implicate. Sections: `## Summary`, `## Timeline`, `## Root cause`,
|
||||
`## Mitigations applied`, `## Open questions`. The legacy file-based investigations live at
|
||||
`archive/knowledge/sources/investigations/` (frozen 2026-07-07); new investigations go in the DB.
|
||||
|
||||
## The operations log
|
||||
|
||||
`plans/log.md` and `knowledge/log.md` are append-only records of documentation operations on
|
||||
those areas (`## [YYYY-MM-DD] <op> | <summary>`), distinct from the Oikos change ledger.
|
||||
`plans/log.md` is the append-only record of documentation operations on plans
|
||||
(`## [YYYY-MM-DD] <op> | <summary>`), distinct from the DB audit trail. The legacy
|
||||
`archive/knowledge/log.md` is frozen with the rest of the archived wiki.
|
||||
|
||||
@@ -49,7 +49,7 @@ Run from the [hubris host](../../archive/knowledge/hosts/hubris.md) as root. Whe
|
||||
- `ras-mc-ctl --errors` — full event log
|
||||
- `cat /sys/devices/system/cpu/cpu0/cpufreq/energy_performance_preference` — should be `balance_power`
|
||||
- `cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor` — should be `powersave`
|
||||
- `ls /sys/fs/pstore/ /var/lib/systemd/pstore/` — panic traces from a previous crash (empty for pure hardware hangs — see [investigation](../../archive/knowledge/investigations/archive/2026-04-21-hubris-crash-loop.md))
|
||||
- `ls /sys/fs/pstore/ /var/lib/systemd/pstore/` — panic traces from a previous crash (empty for pure hardware hangs — see [investigation](../../archive/knowledge/investigations/2026-04-21-hubris-crash-loop.md))
|
||||
|
||||
## Fleet apt operations
|
||||
|
||||
@@ -67,7 +67,7 @@ the dpkg-interrupted recovery procedure specifically.
|
||||
|
||||
See [OIKOS.md](../OIKOS.md) for the operating model. The `homelab` CLI this
|
||||
section used to document is retired; the actual current interface is the
|
||||
33 MCP tools cataloged in [AGENTS.md](../../AGENTS.md#3-the-mcp-server) plus
|
||||
MCP tool catalog in [AGENTS.md §3](../../AGENTS.md#3-the-mcp-server) plus
|
||||
the REST API. Closest current equivalents for what used to live here:
|
||||
|
||||
| Old `homelab` command | Current equivalent |
|
||||
@@ -78,11 +78,11 @@ the REST API. Closest current equivalents for what used to live here:
|
||||
| `homelab signal list\|ack\|resolve\|mute` | MCP `get_signal_history`, or REST `POST /api/v1/signals/{id}/ack\|resolve\|mute` (the control-room UI's Signals page wraps these) |
|
||||
| `homelab approval request\|list\|reply\|check` | REST `GET/POST /api/v1/approvals*` (Matrix-delivered via the notifier, or the control-room UI's Operations page) |
|
||||
| `homelab restart <service> --approval-id <id>` | MCP `run` (policy-gated — auto-executes if read-only/reversible_low, otherwise queues for the same Matrix/UI approval) |
|
||||
| `homelab decide <action> <entity>` | No direct equivalent — classification now happens inline inside `run`/`request_execution`, not as a separate dry-run call |
|
||||
| `homelab decide <action> <entity>` | No direct equivalent — classification now happens inline inside `run`, not as a separate dry-run call |
|
||||
|
||||
There is no separately-deployed "Oikos Console" anymore — the control-room
|
||||
SPA (`web/`) is the operator dashboard, served standalone (see
|
||||
[plans/2026-07-12-wails-desktop-app.md](../../plans/2026-07-12-wails-desktop-app.md)).
|
||||
[plans/done/2026-07-12-wails-desktop-app.md](../../plans/done/2026-07-12-wails-desktop-app.md)).
|
||||
|
||||
## Related
|
||||
- [Hubris host](../../archive/knowledge/hosts/hubris.md)
|
||||
|
||||
@@ -1,40 +1,49 @@
|
||||
# LLM Wiki — the documentation contract
|
||||
|
||||
How the narrative documentation in this repo is organized. The pattern is borrowed from the
|
||||
`sources / wiki / index / log` model: a durable synthesized layer (`archive/knowledge/`) built on top
|
||||
of immutable evidence (`knowledge/sources/`, incident records), with pure-listing indexes and an
|
||||
append-only operations log.
|
||||
How documentation in this repo is organized. The pattern is the `sources / wiki / index / log`
|
||||
model: a durable synthesized layer built on top of immutable evidence, with pure-listing indexes and
|
||||
an append-only operations log.
|
||||
|
||||
This contract governs the **narrative layer only**. The machine-readable substrate — `inventory.yaml`,
|
||||
`secrets/`, `scripts/`, `bin/` — is not part of the wiki and never
|
||||
moves under it. See [the knowledge schema](../domains/knowledge/schema.md) for the split.
|
||||
This contract governs the **narrative layer only**. The machine-readable source of truth — the
|
||||
Postgres database, bootstrapped from `seeds/` — is not part of the wiki and never moves under it.
|
||||
See [the knowledge schema](../domains/knowledge/schema.md) for the split, and ADR 0003 for the
|
||||
DB-native model.
|
||||
|
||||
## Layers
|
||||
|
||||
- **Sources** are immutable raw material: incident records (`knowledge/sources/investigations/`), external reference
|
||||
docs (`knowledge/sources/references/`), and the live system itself (`pct config`, `docker inspect`).
|
||||
Read them; do not rewrite them into other sources.
|
||||
- **Wiki** (`archive/knowledge/`) is the synthesized, authoritative current-state layer: one page per
|
||||
node (`containers/`, `vms/`, host narratives) and per cross-cutting system (`infrastructure/`). A
|
||||
reader understands the topic from the wiki page without reading the sources.
|
||||
- **Source of truth** is the Postgres database. Structured data (entities, relationships, status,
|
||||
metrics) and narrative knowledge (documents, investigations, runbooks) both live there, in the
|
||||
`entities` / `relationships` / `knowledge_entities` tables. It is bootstrapped at deploy time from
|
||||
`seeds/{ontology,inventory,policy,knowledge}.yaml` (idempotent, content-hashed via
|
||||
`seed_versions`) and mutated at runtime via the API/MCP. `oikos export` regenerates
|
||||
`seeds/{ontology,inventory,policy}.yaml` for version control.
|
||||
- **Sources** are immutable raw material: incident records (now `investigation` entities in the DB,
|
||||
historically `archive/knowledge/sources/investigations/`), external reference docs, and the live
|
||||
system itself (`pct config`, `docker inspect`). Read them; do not rewrite them into other sources.
|
||||
- **Wiki** — the synthesized, authoritative current-state layer. Today this is the set of
|
||||
`document` entities in the DB (one per node and per cross-cutting system), queried via MCP
|
||||
`search_knowledge` / `get_entity_knowledge`. The legacy file-based wiki is frozen at
|
||||
`archive/knowledge/{hosts,containers,vms,infrastructure}/` for historical reference only.
|
||||
- **Index** (`index.md` / folder `README.md`) is a pure listing — every page in scope with a
|
||||
one-line summary, and nothing else. Anything the section wants to say up front goes into a page
|
||||
the index lists, not into the index.
|
||||
- **Log** (`log.md`) is append-only, recording *doc-maintenance operations* (restructures, source
|
||||
ingests, lint sweeps) in single-line format: `## [YYYY-MM-DD] <op> | <summary>`.
|
||||
- **Log** is append-only, recording *doc-maintenance operations* (restructures, source ingests,
|
||||
lint sweeps) in single-line format: `## [YYYY-MM-DD] <op> | <summary>`. The active log is the DB
|
||||
audit trail; `archive/knowledge/log.md` is the frozen legacy equivalent.
|
||||
|
||||
## Two logs, kept distinct
|
||||
|
||||
- **`## Changelog`** on each node/topic page records *infrastructure* changes to that node. It is
|
||||
machine-parsed (`get_changelog`, the Oikos ledger) — keep the `### YYYY-MM-DD — title` shape.
|
||||
- **`log.md`** per area records *documentation* operations only. It never duplicates the Oikos
|
||||
change ledger (`oikos/ledger.py`), which stays authoritative for infra changes with
|
||||
who/what/risk/approval/verification.
|
||||
- **`## Changelog`** on each node/topic document records *infrastructure* changes to that node. It
|
||||
is stored as a structured field on the `document` entity — keep the `### YYYY-MM-DD — title`
|
||||
shape so it parses cleanly.
|
||||
- **Doc-maintenance logs** record *documentation* operations only. They never duplicate the
|
||||
infrastructure changelog, which stays authoritative for infra changes with
|
||||
who/what/risk/approval/verification (now the DB audit trail, formerly `oikos/ledger.py`).
|
||||
|
||||
## Rules
|
||||
|
||||
- Wiki pages stay short and focused. A page past ~300 lines splits.
|
||||
- Pages stay flat under `wiki/<section>/` until there are enough to warrant a sub-group.
|
||||
- Pages stay flat under their section until there are enough to warrant a sub-group.
|
||||
- Every page follows [writing-style.md](writing-style.md).
|
||||
- Plans and design docs always live in the repo `plans/` folder (`plans/YYYY-MM-DD-slug.md`),
|
||||
listed in `plans/index.md`, moved to `plans/done/` on completion — never a scratch path or a chat
|
||||
|
||||
@@ -9,12 +9,12 @@ in [writing-style.md](writing-style.md); the layer model (sources / wiki / index
|
||||
**Foundational / entry-point files:** ALL-CAPS
|
||||
|
||||
- **Root level:** `AGENTS.md`, `README.md` — discovery paths for agents and humans.
|
||||
- **Agent instruction** (under `.agents/`): `OIKOS.md`, `HERMES.md` — foundational docs agents read before acting.
|
||||
- **Agent instruction** (under `.agents/`): `OIKOS.md`, `NOMOS.md` — foundational docs agents read before acting.
|
||||
- **Reference docs:** `GLOSSARY.md` — lookup reference (like classic repo conventions: LICENSE, CHANGELOG, GLOSSARY).
|
||||
|
||||
**Content / narrative pages:** lowercase-with-dashes, date-prefixed as needed
|
||||
|
||||
- **Container pages:** `<id>-<name>.md` (e.g. `101-jellyfin.md`, `132-rclone.md`). The `<id>` is the LXC/VM ordinal from `inventory.yaml`.
|
||||
- **Container pages:** `<id>-<name>.md` (e.g. `101-jellyfin.md`, `132-rclone.md`). The `<id>` is the LXC/VM ordinal from the entity's attributes in the DB (seeded via `seeds/inventory.yaml`).
|
||||
- **Infrastructure / cross-cutting pages:** `<topic>.md` (e.g. `dns.md`, `auto-deploy.md`, `mesh.md`). Describes a system, not a specific node.
|
||||
- **Plans / investigations:** `YYYY-MM-DD-<slug>.md` (e.g. `2026-07-05-oikos-prometheus-lxc.md`). Date-sorted; slug is lowercase.
|
||||
- **Section indices:** `README.md` (lowercase, conventional). Prefer in folders; `index.md` only if both intro prose and listing coexist.
|
||||
@@ -118,7 +118,7 @@ What it looks like after.
|
||||
Changelog entries to write, index status to update.
|
||||
```
|
||||
|
||||
### Investigation (`knowledge/sources/investigations/YYYY-MM-DD-slug.md`)
|
||||
### Investigation (`investigation` entity in the DB; historically `archive/knowledge/sources/investigations/YYYY-MM-DD-slug.md`)
|
||||
|
||||
```markdown
|
||||
# YYYY-MM-DD — <title>
|
||||
@@ -152,16 +152,18 @@ Changelog entries to write, index status to update.
|
||||
## Same-session update rule
|
||||
|
||||
When you make a change to a node — migrate an LXC, update an IP, change a
|
||||
mount, deploy a new service — **update every relevant doc page in the same
|
||||
session.** A change that touches a container page must also update:
|
||||
mount, deploy a new service — **update the DB and every relevant doc page in
|
||||
the same session.** A change that touches a container must also update:
|
||||
|
||||
- The `containers/index.md` table (IPs, host, mounts, status)
|
||||
- The `entities` / `relationships` rows for the node (via the API/MCP) —
|
||||
this is the source of truth
|
||||
- The `document` entity's `at_glance` and `## Changelog` for the container
|
||||
- The `containers/index.md` table in the archived wiki (IPs, host, mounts,
|
||||
status) — historical reference, update for consistency where still consulted
|
||||
- The `README.md` table (if the change affects listed columns)
|
||||
- The Caddy page site list (if the change affects `*.hubris.network` routing)
|
||||
- The DNS / ingress infrastructure pages (if the change affects routing)
|
||||
- The `hosts/{hubris,strong}.md` host page (if container count changes)
|
||||
- The `inventory.yaml` host entry (single source of truth)
|
||||
- The `infrastructure/topology.md` (generated from inventory, but regen if needed)
|
||||
|
||||
The pattern of updating only one page and leaving stale references on others
|
||||
is a bug. If you're doing a multi-step migration, document the intermediate
|
||||
|
||||
@@ -36,7 +36,7 @@ Every doc-level page follows the same shape so a reader scans it in one pass.
|
||||
1. **One H1 = the page title.** Node pages use `# <id> — \`<name>\``; topic pages use `# <Topic>`.
|
||||
2. **Opening definition.** First paragraph, 1–3 sentences, says what the thing is. No motivation, no marketing, no setup.
|
||||
3. **Body sections** in the natural order for the topic. Reuse the section templates in [page-templates.md](page-templates.md).
|
||||
4. **`## Changelog`** at the bottom of every node/topic page — reverse-chronological, append-only. This section is machine-parsed (Go MCP `get_changelog` in `internal/mcp/server.go`); keep the `### YYYY-MM-DD — title` shape.
|
||||
4. **`## Changelog`** at the bottom of every node/topic page — reverse-chronological, append-only. This section is stored as a structured field on the `document` entity in the DB; keep the `### YYYY-MM-DD — title` shape so it parses cleanly.
|
||||
5. **Related links** only at the bottom, only when a reference cannot be woven inline.
|
||||
|
||||
## Section indexes (folder READMEs)
|
||||
@@ -53,12 +53,12 @@ duplicated prose, no narrative between the intro and the table.
|
||||
- Prefer **tables** for enumerable items with internal structure (service/port maps, field lists, status grids). Reserve bullets for short non-structured lists.
|
||||
- Use the **bold-leading-phrase pattern** for structured points: `**Read-only by construction.** The MCP server never mutates state.` — a bold noun phrase, a period, then the explanation.
|
||||
- When enumerating across services or nodes, give each its own `###` sub-section or a table row, not one run-on paragraph.
|
||||
- Use backticks for code, paths, hostnames, and file names (`inventory.yaml`, `192.168.8.77`, `pct config`); italics for first-mention terminology.
|
||||
- Use backticks for code, paths, hostnames, and file names (`seeds/inventory.yaml`, `192.168.8.77`, `pct config`); italics for first-mention terminology.
|
||||
- Use `>` blockquotes for caveats and gaps that interrupt the main flow: `> **Outstanding gap.** DNS-vs-inventory drift check not yet wired.` One thought per blockquote.
|
||||
|
||||
## Diagrams
|
||||
|
||||
- Mermaid is the default for topology and flow diagrams. `infrastructure/topology.md` is generated by `oikos/gen-topology.py` — do not hand-edit it. (Go DB-native topology generation planned.)
|
||||
- Mermaid is the default for topology and flow diagrams. `infrastructure/topology.md` in the archived wiki was generated by the retired `oikos/gen-topology.py`; the DB-native equivalent is a future task — do not hand-edit the archived file expecting it to regenerate.
|
||||
- ASCII box diagrams are fine for small shape diagrams; keep them to one screen.
|
||||
|
||||
## Sourcing and cross-references
|
||||
|
||||
@@ -36,7 +36,7 @@ For each session determine:
|
||||
|
||||
| Signature | Root cause | Fix |
|
||||
|-----------|-----------|-----|
|
||||
| Agent: "I can't run X — only supports Y" | Missing action in `request_execution` | Add action in `internal/mcp/server.go` |
|
||||
| Agent: "I can't run X" | Missing target or capability | Use `run` with shell command — there is no fixed action enum anymore |
|
||||
| Agent: "No local knowledge on that" + no web tool | Missing `http_get` / web fetch MCP tool | Add MCP tool |
|
||||
| Empty assistant bubble (text="", no tools) | Model returned blank completion | Retry + error surfacing |
|
||||
| Non-English boilerplate refusal | Flash-tier model degradation | Response quality guard |
|
||||
@@ -75,7 +75,7 @@ Session: {id[:8]} — "{title[:60]}"
|
||||
|
||||
- `cmd/nomos/agent.go` — agent loop, tool building, response guards
|
||||
- `cmd/nomos/store.go` — session + message persistence
|
||||
- `internal/mcp/server.go` — all tool implementations including `request_execution`
|
||||
- `internal/mcp/server.go` — all tool implementations (`run`, `list_lxcs`, …)
|
||||
- `web/src/lib/components/ToolCallGroup.svelte` — tool result display
|
||||
- `nomos/SOUL.md` — agent persona and tool selection rules
|
||||
- `plans/2026-07-09-chat-sessions-improvements.md` — prior session findings
|
||||
|
||||
@@ -3,9 +3,10 @@
|
||||
"configurations": [
|
||||
{
|
||||
"name": "web",
|
||||
"runtimeExecutable": "npm",
|
||||
"runtimeArgs": ["--prefix", "web", "run", "dev"],
|
||||
"port": 5173
|
||||
"runtimeExecutable": "sh",
|
||||
"runtimeArgs": ["-c", "export OIKOS_API_TOKEN=$(docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' oikos-api-1 | sed -n 's/^OIKOS_MCP_BEARER_TOKEN=//p'); exec npm --prefix web run dev"],
|
||||
"port": 5173,
|
||||
"autoPort": true
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -70,3 +70,30 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
- name: docker build (verify image builds; no push)
|
||||
run: docker build -f compose/oikos/Dockerfile -t oikos:ci .
|
||||
|
||||
web:
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: web
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
cache: npm
|
||||
cache-dependency-path: web/package-lock.json
|
||||
- run: npm ci
|
||||
- name: svelte-check (advisory — baseline not yet clean)
|
||||
run: npm run check
|
||||
continue-on-error: true
|
||||
- name: eslint (advisory — baseline not yet clean)
|
||||
run: npm run lint
|
||||
continue-on-error: true
|
||||
- name: prettier format check (advisory — baseline not yet clean)
|
||||
run: npm run format:check
|
||||
continue-on-error: true
|
||||
- name: test
|
||||
run: npm run test
|
||||
- name: build
|
||||
run: npm run build
|
||||
|
||||
@@ -8,91 +8,63 @@ on:
|
||||
- 'v[0-9]+.[0-9]+.[0-9]*'
|
||||
|
||||
jobs:
|
||||
build-ui:
|
||||
name: Build SPA
|
||||
build:
|
||||
name: Build Linux (amd64)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: web/package-lock.json
|
||||
|
||||
- run: npm ci
|
||||
working-directory: web
|
||||
|
||||
- run: npm run build
|
||||
working-directory: web
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: spa-dist
|
||||
path: web/dist/
|
||||
|
||||
build-macos-arm64:
|
||||
name: macOS (arm64)
|
||||
needs: build-ui
|
||||
runs-on: macos-14
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: spa-dist
|
||||
path: cmd/desktop/frontend/dist/
|
||||
- run: |
|
||||
rm -rf cmd/desktop/frontend/dist
|
||||
mkdir -p cmd/desktop/frontend/dist
|
||||
cp -r web/dist/* cmd/desktop/frontend/dist/
|
||||
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: '1.26'
|
||||
- run: go install github.com/wailsapp/wails/v3/cmd/wails3@latest
|
||||
- run: wails3 build -clean
|
||||
|
||||
- run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev
|
||||
|
||||
- run: CGO_ENABLED=1 go build -o build/bin/Oikos .
|
||||
working-directory: cmd/desktop
|
||||
env:
|
||||
CGO_ENABLED: 1
|
||||
|
||||
- run: |
|
||||
cd cmd/desktop/build/bin
|
||||
zip -r oikos-desktop-darwin-arm64.zip oikos-desktop.app
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-darwin-arm64
|
||||
path: cmd/desktop/build/bin/oikos-desktop-darwin-arm64.zip
|
||||
tar czf oikos-desktop-linux-amd64.tar.gz Oikos
|
||||
sha256sum oikos-desktop-linux-amd64.tar.gz > oikos-desktop-linux-amd64.tar.gz.sha256
|
||||
|
||||
build-linux-amd64:
|
||||
name: Linux (amd64)
|
||||
needs: build-ui
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: spa-dist
|
||||
path: cmd/desktop/frontend/dist/
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: '1.26'
|
||||
- run: sudo apt-get update && sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev
|
||||
- run: go install github.com/wailsapp/wails/v3/cmd/wails3@latest
|
||||
- run: wails3 build -clean
|
||||
working-directory: cmd/desktop
|
||||
env:
|
||||
CGO_ENABLED: 1
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-linux-amd64
|
||||
path: cmd/desktop/build/bin/oikos-desktop
|
||||
path: |
|
||||
cmd/desktop/build/bin/oikos-desktop-linux-amd64.tar.gz
|
||||
cmd/desktop/build/bin/oikos-desktop-linux-amd64.tar.gz.sha256
|
||||
|
||||
release:
|
||||
name: Create Release
|
||||
needs: [build-macos-arm64, build-linux-amd64]
|
||||
name: Attach to Release
|
||||
needs: build
|
||||
runs-on: ubuntu-latest
|
||||
if: startsWith(github.ref, 'refs/tags/')
|
||||
steps:
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-darwin-arm64
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-linux-amd64
|
||||
- name: Release
|
||||
uses: https://gitea.com/actions/release-action@v1
|
||||
|
||||
- uses: https://gitea.com/actions/release-action@v1
|
||||
with:
|
||||
files: |
|
||||
oikos-desktop-darwin-arm64.zip
|
||||
oikos-desktop-linux-amd64
|
||||
oikos-desktop-linux-amd64.tar.gz
|
||||
oikos-desktop-linux-amd64.tar.gz.sha256
|
||||
api_key: ${{ secrets.GITEA_TOKEN }}
|
||||
|
||||
4
.gitignore
vendored
4
.gitignore
vendored
@@ -21,4 +21,6 @@ web/node_modules/
|
||||
# Wails desktop app — frontend copy for embedding
|
||||
cmd/desktop/frontend/dist/
|
||||
cmd/desktop/build/
|
||||
cmd/desktop/oikos-desktop
|
||||
cmd/desktop/Oikos
|
||||
desktop
|
||||
/eval
|
||||
|
||||
41
.golangci.yml
Normal file
41
.golangci.yml
Normal file
@@ -0,0 +1,41 @@
|
||||
# golangci-lint configuration for Oikos
|
||||
# Docs: https://golangci-lint.run/usage/configuration/
|
||||
run:
|
||||
timeout: 5m
|
||||
tests: true
|
||||
|
||||
linters:
|
||||
enable:
|
||||
- govet # go vet
|
||||
- staticcheck # advanced static analysis
|
||||
- ineffassign # detect ineffectual assignments
|
||||
- unused # find unused identifiers
|
||||
- errcheck # check for unchecked errors
|
||||
- gosimple # simplifications
|
||||
- typecheck # standard type checking
|
||||
- misspell # find commonly misspelled English words in comments
|
||||
- revive # fast, configurable linter (replaces golint)
|
||||
|
||||
linters-settings:
|
||||
errcheck:
|
||||
# Allow unchecked errors on common Close/Flush patterns (deferred cleanup)
|
||||
exclude-functions:
|
||||
- (io.Closer).Close
|
||||
- (*os.File).Close
|
||||
|
||||
issues:
|
||||
# Exclude generated code
|
||||
exclude-rules:
|
||||
- path: _test\.go
|
||||
linters:
|
||||
- errcheck
|
||||
- path: internal/httpapi/gen/
|
||||
linters:
|
||||
- all
|
||||
- path: internal/db/sqlcgen/
|
||||
linters:
|
||||
- all
|
||||
# Don't auto-exclude common patterns
|
||||
exclude-use-default: false
|
||||
max-issues-per-linter: 0
|
||||
max-same-issues: 0
|
||||
61
AGENTS.md
61
AGENTS.md
@@ -1,7 +1,7 @@
|
||||
# AGENTS.md — orientation for any agent on a homelab client
|
||||
|
||||
You are running on a machine that is part of the **hubris** homelab. The full
|
||||
context is in this checkout at `/opt/homelab-context/`. This file is the entry
|
||||
context is in this checkout at `/opt/homelab/`. This file is the entry
|
||||
point. Read it once at start, then keep working.
|
||||
|
||||
- **New client?** Read [CLIENTS.md](CLIENTS.md) first.
|
||||
@@ -30,7 +30,7 @@ is archived at `archive/knowledge/` for historical reference.
|
||||
|
||||
Run `hostname` (Linux) or `scutil --get LocalHostName` (macOS), then read:
|
||||
|
||||
/opt/homelab-context/inventory.yaml
|
||||
/opt/homelab/inventory.yaml
|
||||
|
||||
That file tells you your role, your peers, what's mounted, and what services
|
||||
you host. If it does not exist, this client was not enrolled — stop and tell
|
||||
@@ -39,12 +39,13 @@ the operator; see [CLIENTS.md](CLIENTS.md#enrollment) for the enrollment flow
|
||||
|
||||
## 2. The topology
|
||||
|
||||
- `/opt/homelab-context/inventory.yaml` — every host, LXC, VM, and workstation
|
||||
- `/opt/homelab/inventory.yaml` — every host, LXC, VM, and workstation
|
||||
with their mesh addresses, roles, and service mappings. This is the seed file;
|
||||
at runtime the DB is authoritative (query via MCP `get_entity` or the REST API).
|
||||
- `/opt/homelab-context/seeds/knowledge.yaml` — full narrative knowledge: 36
|
||||
documents, 6 investigations, 12 runbooks. Ingested into the DB on deploy.
|
||||
- `/opt/homelab-context/.agents/operations/commands.md` — the operator's cheatsheet
|
||||
- `/opt/homelab/seeds/knowledge.yaml` — full narrative knowledge
|
||||
(documents, investigations, runbooks). Counts are not hardcoded here; count
|
||||
them from the seed or query the DB. Ingested into the DB on deploy.
|
||||
- `/opt/homelab/.agents/operations/commands.md` — the operator's cheatsheet
|
||||
for pct, caddy, DNS, and the Oikos command surface.
|
||||
|
||||
## 3. The MCP server
|
||||
@@ -55,7 +56,8 @@ Endpoint: `https://mcp.hubris.network/mcp`. Every call needs
|
||||
enrollment and `/healthz` (see "Authentication" below for where the token
|
||||
comes from).
|
||||
|
||||
Available tools (33 total):
|
||||
Available tools (the authoritative list — count them below if a number is
|
||||
needed; do not hardcode the count elsewhere):
|
||||
|
||||
Context — observe + orient:
|
||||
get_entity(slug), list_entities(type, limit, cursor),
|
||||
@@ -110,11 +112,10 @@ Available tools (33 total):
|
||||
read-only inspection runs immediately, anything state-changing needs
|
||||
operator approval, and destructive patterns (rm -rf, dd, mkfs,
|
||||
pct/qm destroy, DROP TABLE, reboot, curl-pipe-to-shell, ...) always
|
||||
need approval regardless of what you declare. Prefer this over
|
||||
request_execution for anything not already covered by its fixed enum.
|
||||
request_execution(target, action, params) — the older, fixed-enum path
|
||||
(restart, systemctl, pct_exec, apt_upgrade, pct_create). Still the
|
||||
route for those specific actions; policy-gated the same way `run` is.
|
||||
need approval regardless of what you declare. This is the ONLY
|
||||
mutation tool — `request_execution` was retired 2026-07-14; the
|
||||
former enum actions (restart, systemctl, pct_exec, apt_upgrade,
|
||||
pct_create) are all expressed as `run(target, command)` now.
|
||||
get_execution_status(execution_id) — poll progress
|
||||
|
||||
**When to prefer MCP over grepping the clone:** always for knowledge queries.
|
||||
@@ -145,8 +146,8 @@ POST /api/v1/knowledge/{entity_slug}
|
||||
{"title": "...", "content": "...", "tags": ["..."]}
|
||||
```
|
||||
|
||||
The DB is the truth. The old wiki files are in `knowledge/wiki/` pending archive
|
||||
per the DB-as-source-of-truth plan.
|
||||
The DB is the truth. The old wiki files are archived at `archive/knowledge/`
|
||||
(historical reference only — use MCP `search_knowledge` for live queries).
|
||||
|
||||
- **Runbook procedures** live as `runbook` entities in the DB and as SKILL.md
|
||||
files under `.agents/skills/<name>/`. They carry `risk_class`, `procedure`
|
||||
@@ -162,11 +163,10 @@ per the DB-as-source-of-truth plan.
|
||||
## 6. Acting on the homelab
|
||||
|
||||
- **Read state**: use MCP tools. Nomos (the AI agent) is the primary
|
||||
operator interface — it has 33 MCP tools for observe/orient/decide/act
|
||||
(§3).
|
||||
operator interface — it routes to the MCP tool list in §3 for
|
||||
observe/orient/decide/act.
|
||||
- **Actions** (restart, logs, apt, pct exec, or anything else): Nomos calls
|
||||
`run` (the general execution primitive) or `request_execution` (the older
|
||||
fixed-enum path) via MCP. `reversible_low`/read-only actions execute
|
||||
`run` (the general execution primitive) via MCP. `reversible_low`/read-only actions execute
|
||||
immediately; `config_mutation` and `destructive` actions are queued for
|
||||
operator approval via Matrix or the control-room UI's Operations page.
|
||||
- **Secrets**: managed by Infisical (`oikos secret` subcommand for
|
||||
@@ -179,7 +179,7 @@ per the DB-as-source-of-truth plan.
|
||||
|
||||
## 7. Communication mode
|
||||
|
||||
Read and apply `/opt/homelab-context/.agents/shared/caveman.md` (if present). It defines the lab's
|
||||
Read and apply `/opt/homelab/.agents/shared/caveman.md` (if present). It defines the lab's
|
||||
terse-communication standard — drop filler, keep substance, use fragments.
|
||||
|
||||
## 8. Auto-setup mechanism
|
||||
@@ -192,19 +192,30 @@ on every client after `git pull`. This is handled by `tools/post-pull.sh`
|
||||
Currently auto-setup:
|
||||
- **Host checks** (`tools/setup-checks.sh`): Deploys `checks/install.sh`'s
|
||||
health-check scripts to `/opt/oikos/checks` on each host. The scheduler's
|
||||
`ssh-script` check kind depends on these actually being there — 20 are
|
||||
live in the DB as of 2026-07-12.
|
||||
`ssh-script` check kind depends on these actually being there (count is
|
||||
whatever is currently seeded in the DB — do not hardcode it here).
|
||||
|
||||
To add a new auto-setup, create `tools/setup-<name>.sh` in the repo,
|
||||
commit and push. All enrolled clients pick it up within 5 minutes.
|
||||
|
||||
To trigger sync manually: run `/opt/homelab/tools/context-poller.sh`, or
|
||||
wait for the 5-min timer. (This mechanism — and the server-side
|
||||
`tools_changed` detection behind it — only correctly recognized
|
||||
`setup-*.sh` scripts as of 2026-07-12; before that it silently matched
|
||||
wait for the 5-min timer. (The server-side `tools_changed` detection only
|
||||
correctly recognizes `setup-*.sh` scripts — earlier it silently matched
|
||||
nothing, so nothing auto-ran on any client via this path.)
|
||||
|
||||
## 9. When in doubt
|
||||
## 9. Versioning
|
||||
|
||||
Every commit to `main` **MUST bump the version** in the `VERSION` file at the
|
||||
repo root. The format is semver-ish: `major.minor.patch` (e.g. `0.2.3`).
|
||||
|
||||
Rules:
|
||||
- **patch** (`0.2.2` → `0.2.3`): bugfixes, small tweaks, docs-only changes
|
||||
- **minor** (`0.2.3` → `0.3.0`): new features, new tools, visible functionality
|
||||
- **major** (`0.3.0` → `1.0.0`): breaking changes (API removal, tool retirement)
|
||||
|
||||
The version is shown in the UI sidebar. The `v` prefix is added at build time.
|
||||
|
||||
## 10. When in doubt
|
||||
|
||||
Use MCP tools: `search_knowledge <query>` for narrative context,
|
||||
`get_entity <slug>` for structured data, `get_entity_knowledge <slug>` for
|
||||
|
||||
@@ -28,7 +28,7 @@ Docker stack on mac-mini and exposes an MCP server + REST API.
|
||||
| Record a discovered fact/relationship | MCP `update_entity_attributes`, `create_relationship`, `upsert_knowledge` |
|
||||
|
||||
Most MCP tools are read-only; a few mutate the knowledge graph (recording
|
||||
what you learned) or the live infrastructure (`run`, `request_execution`),
|
||||
what you learned) or the live infrastructure (`run`),
|
||||
gated by risk classification and — for `config_mutation`/`destructive`
|
||||
actions — operator approval. See [AGENTS.md](AGENTS.md#3-the-mcp-server) for
|
||||
the full tool catalog.
|
||||
|
||||
@@ -28,8 +28,31 @@ make build
|
||||
|
||||
# SPA dev server (proxies to api/nomos, injecting the same token)
|
||||
cd web && OIKOS_API_TOKEN=dev-token npm run dev
|
||||
|
||||
# Desktop app (macOS)
|
||||
make desktop # build .app bundle
|
||||
make install # build + install to /Applications
|
||||
./cmd/desktop/build/bin/oikos-desktop.app/Contents/MacOS/oikos-desktop # run from terminal to see logs
|
||||
```
|
||||
|
||||
### Desktop app auth
|
||||
|
||||
The desktop app uses the same API as the browser SPA. First launch:
|
||||
|
||||
1. Enter `https://oikos.hubris.network` as Server URL
|
||||
2. **Login with Authentik** tab → opens system browser → authenticate
|
||||
3. Callback page shows token → copy → paste into Token tab → Connect
|
||||
4. Token is persisted to the macOS keychain — subsequent launches skip setup
|
||||
|
||||
The app stores credentials via `github.com/zalando/go-keyring` (service: `com.hubris.oikos-desktop`).
|
||||
|
||||
### Desktop app auto-update
|
||||
|
||||
- Checks Gitea releases every 6 hours
|
||||
- System tray → **Check for Updates** triggers an immediate check
|
||||
- Download, extract, replace the app in `/Applications`, and relaunch
|
||||
- Versions are compared against the `version` var in `main.go`, injected from the repo `VERSION` file at link time (`make desktop` passes `-ldflags "-X main.version=$(cat VERSION)"`)
|
||||
|
||||
## Project structure
|
||||
|
||||
```
|
||||
@@ -37,6 +60,10 @@ cmd/desktop/ Wails v3 desktop app (macOS + Linux)
|
||||
main.go Thin shell: webview, system tray, notifications, auto-update
|
||||
wails.json Wails project config
|
||||
entitlements.plist macOS code-signing entitlements
|
||||
icon.png System tray icon (embedded)
|
||||
icon.icns App bundle icon (white logo on black rounded rect)
|
||||
Taskfile.yml Wails v3 build tasks
|
||||
Info.plist.template macOS bundle metadata
|
||||
cmd/oikos/ Single-binary entry point
|
||||
cmd/nomos/ Nomos MCP client gateway
|
||||
cmd/webhook/ Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
|
||||
@@ -88,6 +115,7 @@ docs/operations/ Runbooks (rollback, etc.)
|
||||
| `make deploy-ui` | Build + deploy the SPA to the Caddy host |
|
||||
| `make desktop` | Build the Wails desktop app for the current platform |
|
||||
| `make desktop-package` | Build + package (zip on macOS, tar.gz on Linux) |
|
||||
| `make install` | Build + install to `/Applications` (macOS) |
|
||||
| `make webhook` | Build `cmd/webhook` (deploy-webhook receiver) |
|
||||
| `make tidy` | `go mod tidy` |
|
||||
|
||||
|
||||
39
Makefile
39
Makefile
@@ -1,9 +1,10 @@
|
||||
.PHONY: build webhook test test-db lint generate generate-check dev migrate seed export clean tidy ui desktop desktop-package desktop-release
|
||||
.PHONY: build webhook test test-db lint generate generate-check dev migrate seed export clean tidy ui desktop desktop-package install
|
||||
|
||||
BINARY := oikos
|
||||
BINARY := bin/oikos
|
||||
GO ?= go
|
||||
|
||||
build:
|
||||
mkdir -p bin
|
||||
$(GO) build -o $(BINARY) -tags timetzdata ./cmd/oikos
|
||||
|
||||
webhook:
|
||||
@@ -19,9 +20,18 @@ test-db:
|
||||
OIKOS_TEST_DATABASE_URL="postgres://oikos:$${OIKOS_DB_PASSWORD:-oikos_dev}@localhost:5432/oikos?sslmode=disable" \
|
||||
$(GO) test -race -count=1 ./internal/db/ ./internal/httpapi/ ./internal/mcp/
|
||||
|
||||
lint:
|
||||
lint: vet golangci govulncheck
|
||||
|
||||
vet:
|
||||
$(GO) vet ./...
|
||||
@command -v golangci-lint >/dev/null 2>&1 && golangci-lint run || echo "golangci-lint not installed, skipping"
|
||||
|
||||
golangci:
|
||||
@command -v golangci-lint >/dev/null 2>&1 && golangci-lint run --config .golangci.yml || echo "golangci-lint not installed — see https://golangci-lint.run/usage/install/"
|
||||
|
||||
govulncheck:
|
||||
@command -v govulncheck >/dev/null 2>&1 && govulncheck ./... || echo "govulncheck not installed — run: go install golang.org/x/vuln/cmd/govulncheck@latest"
|
||||
|
||||
.PHONY: lint vet golangci govulncheck
|
||||
|
||||
generate:
|
||||
$(GO) run github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@v2.4.1 \
|
||||
@@ -56,23 +66,32 @@ desktop: ui ## Build the Wails desktop app for the current platform
|
||||
rm -rf cmd/desktop/frontend/dist
|
||||
mkdir -p cmd/desktop/frontend/dist
|
||||
cp -r web/dist/* cmd/desktop/frontend/dist/
|
||||
cd cmd/desktop && wails3 build -clean
|
||||
cd cmd/desktop && CGO_ENABLED=1 go build -ldflags "-X main.version=$$(cat ../VERSION)" -o build/bin/Oikos .
|
||||
|
||||
desktop-package: desktop ## Build + package the desktop app (zip on macOS, tar.gz on Linux)
|
||||
@case $$(uname -s) in \
|
||||
Darwin) \
|
||||
cd cmd/desktop/build/bin && zip -r oikos-desktop-darwin-$$(uname -m).zip oikos-desktop.app ;; \
|
||||
APP="cmd/desktop/build/bin/Oikos.app"; \
|
||||
rm -rf "$$APP"; \
|
||||
mkdir -p "$$APP/Contents/MacOS"; \
|
||||
mkdir -p "$$APP/Contents/Resources"; \
|
||||
cp cmd/desktop/build/bin/Oikos "$$APP/Contents/MacOS/Oikos"; \
|
||||
cp cmd/desktop/icon.icns "$$APP/Contents/Resources/icon.icns"; \
|
||||
sed "s/\$$(VERSION)/$$(cat VERSION)/" cmd/desktop/Info.plist.template > "$$APP/Contents/Info.plist"; \
|
||||
cd cmd/desktop/build/bin && zip -r oikos-desktop-darwin-$$(uname -m).zip Oikos.app ;; \
|
||||
Linux) \
|
||||
cd cmd/desktop/build/bin && tar czf oikos-desktop-linux-$$(uname -m).tar.gz oikos-desktop ;; \
|
||||
cd cmd/desktop/build/bin && tar czf oikos-desktop-linux-$$(uname -m).tar.gz Oikos ;; \
|
||||
esac
|
||||
@echo "Package: cmd/desktop/build/bin/"
|
||||
|
||||
desktop-release: ui ## Build desktop app for macOS arm64 + Linux amd64 (CI target)
|
||||
@echo "Use 'make desktop-package' for local builds; desktop-release is for CI"
|
||||
@exit 1
|
||||
install: desktop-package ## Install to /Applications
|
||||
rm -rf /Applications/Oikos.app
|
||||
cp -r cmd/desktop/build/bin/Oikos.app /Applications/
|
||||
@echo "Installed to /Applications/Oikos.app"
|
||||
|
||||
clean:
|
||||
rm -f $(BINARY)
|
||||
rm -rf bin
|
||||
rm -rf cmd/desktop/build
|
||||
rm -rf cmd/desktop/frontend/dist
|
||||
$(GO) clean -testcache
|
||||
|
||||
13
README.md
13
README.md
@@ -49,7 +49,7 @@ cd web && OIKOS_API_TOKEN=dev-token npm run dev # http://localhost:5173
|
||||
|
||||
| Component | Port | Role |
|
||||
|-----------|------|------|
|
||||
| `oikos api` | 8090 | REST API + MCP server (15 tools) |
|
||||
| `oikos api` | 8090 | REST API + MCP server (tool list in [AGENTS.md §3](AGENTS.md#3-the-mcp-server)) |
|
||||
| `oikos scheduler` | — | Probe runner, signal lifecycle, metrics |
|
||||
| `oikos notifier` | — | Approval tokens, Matrix alerts |
|
||||
| `nomos serve` | 8092 | MCP client gateway, query routing |
|
||||
@@ -112,8 +112,8 @@ oikos secret migrate # SOPS → Infisical
|
||||
part of `docker-compose.yml`. It talks to `api`/`nomos` over HTTP with a
|
||||
bearer token entered on first launch (see `web/src/pages/Config.svelte`).
|
||||
Build with `make ui`, deploy with `make deploy-ui` (Caddy serves the static
|
||||
output). A native desktop wrapper is planned — see
|
||||
[plans/2026-07-12-wails-desktop-app.md](plans/2026-07-12-wails-desktop-app.md).
|
||||
output). A native desktop wrapper exists at `cmd/desktop/` — see
|
||||
[plans/done/2026-07-12-wails-desktop-app.md](plans/done/2026-07-12-wails-desktop-app.md).
|
||||
|
||||
## Repo layout
|
||||
|
||||
@@ -121,9 +121,10 @@ output). A native desktop wrapper is planned — see
|
||||
cmd/oikos/ Go entry point — single binary
|
||||
cmd/nomos/ Nomos MCP client gateway
|
||||
cmd/webhook/ Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
|
||||
internal/ Go packages (httpapi, mcp, scheduler, actuator, learning,
|
||||
notifier, policy, secrets, db, config, ontology, domain,
|
||||
knowledge)
|
||||
cmd/desktop/ Wails desktop wrapper around the SPA
|
||||
internal/ Go packages (actuator, checkdefaults, config, db, domain,
|
||||
httpapi, knowledge, learning, mcp, notifier, observability,
|
||||
ontology, policy, safego, scheduler, secrets)
|
||||
web/ Control-room SPA (Svelte 5) — standalone, not embedded
|
||||
api/openapi.yaml API contract (OpenAPI 3.1)
|
||||
migrations/ Forward-only SQL migrations (TimescaleDB)
|
||||
|
||||
30
cmd/desktop/Info.plist.template
Normal file
30
cmd/desktop/Info.plist.template
Normal file
@@ -0,0 +1,30 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>CFBundleDevelopmentRegion</key>
|
||||
<string>en</string>
|
||||
<key>CFBundleExecutable</key>
|
||||
<string>Oikos</string>
|
||||
<key>CFBundleIdentifier</key>
|
||||
<string>com.hubris.oikos-desktop</string>
|
||||
<key>CFBundleIconFile</key>
|
||||
<string>icon</string>
|
||||
<key>CFBundleInfoDictionaryVersion</key>
|
||||
<string>6.0</string>
|
||||
<key>CFBundleName</key>
|
||||
<string>Oikos</string>
|
||||
<key>CFBundlePackageType</key>
|
||||
<string>APPL</string>
|
||||
<key>CFBundleShortVersionString</key>
|
||||
<string>$(VERSION)</string>
|
||||
<key>CFBundleVersion</key>
|
||||
<string>$(VERSION)</string>
|
||||
<key>LSMinimumSystemVersion</key>
|
||||
<string>13.0</string>
|
||||
<key>NSHighResolutionCapable</key>
|
||||
<true/>
|
||||
<key>NSHumanReadableCopyright</key>
|
||||
<string>Copyright © 2026 Hubris. All rights reserved.</string>
|
||||
</dict>
|
||||
</plist>
|
||||
14
cmd/desktop/Taskfile.yml
Normal file
14
cmd/desktop/Taskfile.yml
Normal file
@@ -0,0 +1,14 @@
|
||||
version: '3'
|
||||
|
||||
tasks:
|
||||
build:
|
||||
summary: Build the Oikos desktop app
|
||||
cmds:
|
||||
- go build -o build/bin/Oikos .
|
||||
env:
|
||||
CGO_ENABLED: 1
|
||||
|
||||
dev:
|
||||
summary: Run in development mode
|
||||
cmds:
|
||||
- go run .
|
||||
BIN
cmd/desktop/icon.icns
Normal file
BIN
cmd/desktop/icon.icns
Normal file
Binary file not shown.
BIN
cmd/desktop/icon.png
Normal file
BIN
cmd/desktop/icon.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.5 KiB |
@@ -1,37 +1,52 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"embed"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"log"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"os/exec"
|
||||
"os/user"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/wailsapp/wails/v3/pkg/application"
|
||||
"github.com/wailsapp/wails/v3/pkg/events"
|
||||
"github.com/zalando/go-keyring"
|
||||
)
|
||||
|
||||
//go:embed frontend/dist
|
||||
var assets embed.FS
|
||||
|
||||
//go:embed icon.png
|
||||
var iconPNG []byte
|
||||
|
||||
const (
|
||||
keyringService = "com.hubris.oikos-desktop"
|
||||
keyringUser = "oikos"
|
||||
version = "0.1.0"
|
||||
updateURL = "https://git.hubris.network/api/v1/repos/dtoro/oikos/releases"
|
||||
pollInterval = 30 * time.Second
|
||||
updateInterval = 6 * time.Hour
|
||||
keyringService = "com.hubris.oikos-desktop"
|
||||
keyringUser = "oikos"
|
||||
updateURL = "https://git.hubris.network/api/v1/repos/dtoro/oikos/releases"
|
||||
pollInterval = 30 * time.Second
|
||||
updateInterval = 6 * time.Hour
|
||||
oidcCallbackPort = 18901
|
||||
)
|
||||
|
||||
// version is injected at link time via -ldflags "-X main.version=$(cat VERSION)"
|
||||
// (Makefile desktop target). The default keeps a non-empty fallback for
|
||||
// `go build ./cmd/desktop` without ldflags.
|
||||
var version = "0.1.0-dev"
|
||||
|
||||
type OikosConfig struct {
|
||||
ApiUrl string `json:"apiUrl"`
|
||||
Token string `json:"token,omitempty"`
|
||||
@@ -40,7 +55,7 @@ type OikosConfig struct {
|
||||
|
||||
// ---- ConfigService ----
|
||||
|
||||
type ConfigService struct{ app *application.App }
|
||||
type ConfigService struct{}
|
||||
|
||||
func (c *ConfigService) Name() string { return "config" }
|
||||
|
||||
@@ -54,6 +69,10 @@ func (c *ConfigService) ClearConfig() error {
|
||||
return keyring.Delete(keyringService, keyringUser)
|
||||
}
|
||||
|
||||
func (c *ConfigService) GetStoredConfig() *OikosConfig {
|
||||
return loadConfig()
|
||||
}
|
||||
|
||||
func (c *ConfigService) EnableAutoStart() error {
|
||||
if runtime.GOOS != "darwin" {
|
||||
return fmt.Errorf("autostart not supported on %s", runtime.GOOS)
|
||||
@@ -92,6 +111,203 @@ func (c *ConfigService) DisableAutoStart() error {
|
||||
return os.Remove(path)
|
||||
}
|
||||
|
||||
// ---- Local OIDC server (runs alongside the webview) ----
|
||||
|
||||
type oidcSession struct {
|
||||
apiUrl string
|
||||
verifier string
|
||||
state string
|
||||
ch chan string
|
||||
}
|
||||
|
||||
var (
|
||||
oidcSessionsMu sync.Mutex
|
||||
oidcSessions = make(map[string]*oidcSession)
|
||||
)
|
||||
|
||||
func startOIDCServer() *http.Server {
|
||||
mux := http.NewServeMux()
|
||||
|
||||
cors := func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Access-Control-Allow-Origin", "*")
|
||||
w.Header().Set("Access-Control-Allow-Methods", "GET, OPTIONS")
|
||||
w.Header().Set("Access-Control-Allow-Headers", "Content-Type")
|
||||
if r.Method == "OPTIONS" {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
h := func(path string, handler func(http.ResponseWriter, *http.Request)) {
|
||||
mux.HandleFunc(path, func(w http.ResponseWriter, r *http.Request) {
|
||||
cors(w, r)
|
||||
if r.Method == "OPTIONS" {
|
||||
return
|
||||
}
|
||||
handler(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
h("/oidc/start", func(w http.ResponseWriter, r *http.Request) {
|
||||
apiUrl := strings.TrimRight(r.URL.Query().Get("apiUrl"), "/")
|
||||
returnURL := r.URL.Query().Get("ret")
|
||||
if apiUrl == "" {
|
||||
http.Error(w, "apiUrl required", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
if returnURL == "" {
|
||||
returnURL = "/?desktop=1"
|
||||
}
|
||||
|
||||
oidcCfg, err := fetchOIDCConfig(apiUrl)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
|
||||
verifier, challenge, _ := pkceParams()
|
||||
state := randomString(32)
|
||||
redirectURI := fmt.Sprintf("http://127.0.0.1:%d/oidc/callback", oidcCallbackPort)
|
||||
|
||||
ch := make(chan string, 1)
|
||||
oidcSessionsMu.Lock()
|
||||
sessionID := randomString(16)
|
||||
oidcSessions[sessionID] = &oidcSession{apiUrl: apiUrl, verifier: verifier, state: state, ch: ch}
|
||||
oidcSessionsMu.Unlock()
|
||||
|
||||
authURL := fmt.Sprintf("%s?%s",
|
||||
oidcCfg.AuthorizationEndpoint,
|
||||
url.Values{
|
||||
"response_type": {"code"},
|
||||
"client_id": {oidcCfg.ClientID},
|
||||
"redirect_uri": {redirectURI},
|
||||
"code_challenge": {challenge},
|
||||
"code_challenge_method": {"S256"},
|
||||
"state": {state},
|
||||
"scope": {"openid profile email"},
|
||||
}.Encode(),
|
||||
)
|
||||
|
||||
exec.Command("open", authURL).Start()
|
||||
|
||||
select {
|
||||
case token := <-ch:
|
||||
if token != "" {
|
||||
c := &ConfigService{}
|
||||
c.SaveConfig(apiUrl, token)
|
||||
returnURL += "&token=" + url.QueryEscape(token)
|
||||
}
|
||||
case <-time.After(5 * time.Minute):
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
fmt.Fprintf(w, `<!DOCTYPE html><html><head><meta charset="UTF-8"><title>Oikos</title>
|
||||
<meta http-equiv="refresh" content="0;url=%s">
|
||||
<style>body{font-family:-apple-system,BlinkMacSystemFont,sans-serif;background:#0a0a0a;color:#e0e0e0;display:flex;align-items:center;justify-content:center;min-height:100vh;margin:0}
|
||||
.card{background:#1a1a1a;border:1px solid #2a2a2a;border-radius:12px;padding:32px;max-width:400px;text-align:center}
|
||||
h1{font-size:18px;margin-bottom:8px}.ok{color:#22c55e;font-size:14px}</style>
|
||||
</head><body><div class="card"><h1>Connected</h1><p class="ok">Redirecting back to Oikos…</p></div></body></html>`, returnURL)
|
||||
})
|
||||
|
||||
h("/oidc/callback", func(w http.ResponseWriter, r *http.Request) {
|
||||
code := r.URL.Query().Get("code")
|
||||
gotState := r.URL.Query().Get("state")
|
||||
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
|
||||
oidcSessionsMu.Lock()
|
||||
var session *oidcSession
|
||||
var sessionID string
|
||||
for id, s := range oidcSessions {
|
||||
if s.state == gotState {
|
||||
session = s
|
||||
sessionID = id
|
||||
break
|
||||
}
|
||||
}
|
||||
oidcSessionsMu.Unlock()
|
||||
|
||||
if session == nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
w.Write([]byte("Invalid state."))
|
||||
return
|
||||
}
|
||||
|
||||
token, err := exchangeCode(
|
||||
session.apiUrl,
|
||||
code, session.verifier,
|
||||
fmt.Sprintf("http://127.0.0.1:%d/oidc/callback", oidcCallbackPort),
|
||||
)
|
||||
|
||||
oidcSessionsMu.Lock()
|
||||
delete(oidcSessions, sessionID)
|
||||
oidcSessionsMu.Unlock()
|
||||
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
fmt.Fprintf(w, "Token exchange failed: %v", err)
|
||||
session.ch <- ""
|
||||
return
|
||||
}
|
||||
|
||||
w.Write([]byte(`<!DOCTYPE html><html><head><meta charset="UTF-8"><title>Oikos</title>
|
||||
<style>body{font-family:-apple-system,BlinkMacSystemFont,sans-serif;background:#0a0a0a;color:#e0e0e0;display:flex;align-items:center;justify-content:center;min-height:100vh;margin:0}
|
||||
.card{background:#1a1a1a;border:1px solid #2a2a2a;border-radius:12px;padding:32px;max-width:400px;text-align:center}
|
||||
h1{font-size:18px;margin-bottom:8px}.ok{color:#22c55e;font-size:14px}</style>
|
||||
</head><body><div class="card"><h1>Connected</h1><p class="ok">You can close this window and return to Oikos.</p></div></body></html>`))
|
||||
session.ch <- token
|
||||
})
|
||||
|
||||
mux.HandleFunc("/oidc/config", func(w http.ResponseWriter, r *http.Request) {
|
||||
apiUrl := strings.TrimRight(r.URL.Query().Get("apiUrl"), "/")
|
||||
if apiUrl == "" {
|
||||
http.Error(w, "apiUrl required", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
cfg, err := fetchOIDCConfig(apiUrl)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(cfg)
|
||||
})
|
||||
|
||||
mux.HandleFunc("/update/check", func(w http.ResponseWriter, r *http.Request) {
|
||||
latest := fetchLatestRelease()
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Header().Set("Access-Control-Allow-Origin", "*")
|
||||
if latest == nil {
|
||||
json.NewEncoder(w).Encode(map[string]string{"current": version})
|
||||
return
|
||||
}
|
||||
hasAsset := false
|
||||
for _, a := range latest.Assets {
|
||||
if strings.Contains(a.Name, "darwin") {
|
||||
hasAsset = true
|
||||
updater.mu.Lock()
|
||||
updater.latestURL = a.BrowserDownloadURL
|
||||
updater.mu.Unlock()
|
||||
break
|
||||
}
|
||||
}
|
||||
json.NewEncoder(w).Encode(map[string]string{
|
||||
"current": version,
|
||||
"latest": latest.Version,
|
||||
"has_asset": fmt.Sprintf("%t", hasAsset),
|
||||
})
|
||||
})
|
||||
|
||||
listener, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", oidcCallbackPort))
|
||||
if err != nil {
|
||||
log.Printf("OIDC server: %v", err)
|
||||
return nil
|
||||
}
|
||||
log.Printf("OIDC server listening on %s", listener.Addr())
|
||||
srv := &http.Server{Handler: mux}
|
||||
go srv.Serve(listener)
|
||||
return srv
|
||||
}
|
||||
|
||||
// ---- Window persistence ----
|
||||
|
||||
type windowState struct {
|
||||
@@ -133,8 +349,6 @@ func saveWindowState(w application.Window) {
|
||||
os.WriteFile(filepath.Join(dir, "window.json"), data, 0644)
|
||||
}
|
||||
|
||||
// ---- Config loading ----
|
||||
|
||||
func loadConfig() *OikosConfig {
|
||||
data, err := keyring.Get(keyringService, keyringUser)
|
||||
if err != nil {
|
||||
@@ -148,37 +362,69 @@ func loadConfig() *OikosConfig {
|
||||
return &cfg
|
||||
}
|
||||
|
||||
// ---- Asset handler ----
|
||||
type oidcConfig struct {
|
||||
Issuer string `json:"issuer"`
|
||||
ClientID string `json:"client_id"`
|
||||
AuthorizationEndpoint string `json:"authorization_endpoint"`
|
||||
}
|
||||
|
||||
func newAssetHandler(cfg *OikosConfig) http.Handler {
|
||||
distFS, err := fs.Sub(assets, "frontend/dist")
|
||||
func fetchOIDCConfig(apiUrl string) (*oidcConfig, error) {
|
||||
resp, err := http.Get(apiUrl + "/api/v1/auth/oidc-config")
|
||||
if err != nil {
|
||||
log.Fatalf("embedded assets: %v", err)
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != 200 {
|
||||
return nil, fmt.Errorf("server returned %d", resp.StatusCode)
|
||||
}
|
||||
var cfg oidcConfig
|
||||
if err := json.NewDecoder(resp.Body).Decode(&cfg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &cfg, nil
|
||||
}
|
||||
|
||||
func pkceParams() (verifier, challenge string, _ error) {
|
||||
v := randomString(64)
|
||||
h := sha256.Sum256([]byte(v))
|
||||
return v, base64.RawURLEncoding.EncodeToString(h[:]), nil
|
||||
}
|
||||
|
||||
func randomString(n int) string {
|
||||
b := make([]byte, n)
|
||||
rand.Read(b)
|
||||
return base64.RawURLEncoding.EncodeToString(b)
|
||||
}
|
||||
|
||||
func exchangeCode(apiUrl, code, verifier, redirectURI string) (string, error) {
|
||||
body, _ := json.Marshal(map[string]string{
|
||||
"grant_type": "authorization_code",
|
||||
"code": code,
|
||||
"code_verifier": verifier,
|
||||
"redirect_uri": redirectURI,
|
||||
})
|
||||
|
||||
resp, err := http.Post(apiUrl+"/api/v1/auth/oidc-token", "application/json", strings.NewReader(string(body)))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != 200 {
|
||||
b, _ := io.ReadAll(resp.Body)
|
||||
return "", fmt.Errorf("token endpoint: %d — %s", resp.StatusCode, string(b))
|
||||
}
|
||||
|
||||
fallback := http.FileServer(http.FS(distFS))
|
||||
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
path := r.URL.Path
|
||||
if path == "/" || path == "/index.html" {
|
||||
data, err := fs.ReadFile(distFS, "index.html")
|
||||
if err != nil {
|
||||
fallback.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
html := string(data)
|
||||
if cfg != nil {
|
||||
configJSON, _ := json.Marshal(cfg)
|
||||
placeholder := `<script>window.__OIKOS_CONFIG__ = {};</script>`
|
||||
injected := fmt.Sprintf(`<script>window.__OIKOS_CONFIG__ = %s;</script>`, configJSON)
|
||||
html = strings.ReplaceAll(html, placeholder, injected)
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
w.Write([]byte(html))
|
||||
return
|
||||
}
|
||||
fallback.ServeHTTP(w, r)
|
||||
})
|
||||
var tokens struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&tokens); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if tokens.AccessToken == "" {
|
||||
return "", fmt.Errorf("no access_token in response")
|
||||
}
|
||||
return tokens.AccessToken, nil
|
||||
}
|
||||
|
||||
// ---- Notifications ----
|
||||
@@ -262,48 +508,181 @@ type giteaRelease struct {
|
||||
} `json:"assets"`
|
||||
}
|
||||
|
||||
type updateState struct {
|
||||
mu sync.Mutex
|
||||
latestURL string
|
||||
}
|
||||
|
||||
var updater = &updateState{}
|
||||
|
||||
// CheckForUpdates checks Gitea releases for a newer version. If found, stores
|
||||
// the download URL and returns the latest version string (empty if current).
|
||||
func (c *ConfigService) CheckForUpdates() string {
|
||||
latest := fetchLatestRelease()
|
||||
if latest == nil || latest.Version == version {
|
||||
return ""
|
||||
}
|
||||
for _, a := range latest.Assets {
|
||||
if strings.Contains(a.Name, "darwin") {
|
||||
updater.mu.Lock()
|
||||
updater.latestURL = a.BrowserDownloadURL
|
||||
updater.mu.Unlock()
|
||||
return latest.Version
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// InstallUpdate downloads the stored update, replaces the app, and restarts.
|
||||
func (c *ConfigService) InstallUpdate() error {
|
||||
updater.mu.Lock()
|
||||
url := updater.latestURL
|
||||
updater.mu.Unlock()
|
||||
if url == "" {
|
||||
return fmt.Errorf("no update available")
|
||||
}
|
||||
return doUpdate(url)
|
||||
}
|
||||
|
||||
type latestRelease struct {
|
||||
Version string
|
||||
Assets []struct {
|
||||
Name string
|
||||
BrowserDownloadURL string
|
||||
}
|
||||
}
|
||||
|
||||
func fetchLatestRelease() *latestRelease {
|
||||
resp, err := http.Get(updateURL + "?draft=false&pre-release=false&limit=1")
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var releases []giteaRelease
|
||||
if err := json.NewDecoder(resp.Body).Decode(&releases); err != nil || len(releases) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
r := releases[0]
|
||||
v := strings.TrimPrefix(r.TagName, "v")
|
||||
if v == version {
|
||||
return nil
|
||||
}
|
||||
|
||||
lr := &latestRelease{Version: v}
|
||||
for _, a := range r.Assets {
|
||||
lr.Assets = append(lr.Assets, struct {
|
||||
Name string
|
||||
BrowserDownloadURL string
|
||||
}{a.Name, a.BrowserDownloadURL})
|
||||
}
|
||||
return lr
|
||||
}
|
||||
|
||||
func doUpdate(downloadURL string) error {
|
||||
tmp, err := os.CreateTemp("", "oikos-update-*.zip")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(tmp.Name())
|
||||
|
||||
resp, err := http.Get(downloadURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if _, err := io.Copy(tmp, resp.Body); err != nil {
|
||||
return err
|
||||
}
|
||||
tmp.Close()
|
||||
|
||||
extractDir, err := os.MkdirTemp("", "oikos-extract")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.RemoveAll(extractDir)
|
||||
|
||||
cmd := exec.Command("unzip", "-o", tmp.Name(), "-d", extractDir)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
return fmt.Errorf("unzip: %w: %s", err, out)
|
||||
}
|
||||
|
||||
newApp := filepath.Join(extractDir, "Oikos.app")
|
||||
if _, err := os.Stat(newApp); err != nil {
|
||||
return fmt.Errorf("extracted app not found: %w", err)
|
||||
}
|
||||
|
||||
currentApp := "/Applications/Oikos.app"
|
||||
if _, err := os.Stat(currentApp); os.IsNotExist(err) {
|
||||
if exe, err := os.Executable(); err == nil {
|
||||
currentApp = filepath.Dir(filepath.Dir(filepath.Dir(exe)))
|
||||
}
|
||||
}
|
||||
|
||||
script := fmt.Sprintf(`#!/bin/bash
|
||||
sleep 2
|
||||
rm -rf "%s"
|
||||
mv "%s" "%s"
|
||||
open "%s"
|
||||
rm "$0"
|
||||
`, currentApp, newApp, currentApp, currentApp)
|
||||
|
||||
scriptPath := filepath.Join(os.TempDir(), "oikos-update.sh")
|
||||
if err := os.WriteFile(scriptPath, []byte(script), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
app := application.Get()
|
||||
exec.Command("open", scriptPath).Start()
|
||||
if app != nil {
|
||||
app.Quit()
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func checkUpdates() {
|
||||
for {
|
||||
resp, err := http.Get(updateURL + "?draft=false&pre-release=false&limit=1")
|
||||
if err != nil {
|
||||
time.Sleep(updateInterval)
|
||||
continue
|
||||
}
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
resp.Body.Close()
|
||||
|
||||
var releases []giteaRelease
|
||||
if err := json.Unmarshal(body, &releases); err != nil || len(releases) == 0 {
|
||||
time.Sleep(updateInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
latest := releases[0]
|
||||
latestVersion := strings.TrimPrefix(latest.TagName, "v")
|
||||
if latestVersion == version {
|
||||
time.Sleep(updateInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
app := application.Get()
|
||||
if app == nil {
|
||||
time.Sleep(updateInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
msg := fmt.Sprintf("Version %s is available (you have %s). Download from Gitea releases.", latestVersion, version)
|
||||
app.Dialog.Info().
|
||||
SetTitle("Update Available").
|
||||
SetMessage(msg).
|
||||
Show()
|
||||
time.Sleep(updateInterval)
|
||||
|
||||
latest := fetchLatestRelease()
|
||||
if latest == nil {
|
||||
continue
|
||||
}
|
||||
|
||||
for _, a := range latest.Assets {
|
||||
if strings.Contains(a.Name, "darwin") {
|
||||
updater.mu.Lock()
|
||||
updater.latestURL = a.BrowserDownloadURL
|
||||
updater.mu.Unlock()
|
||||
|
||||
app := application.Get()
|
||||
if app == nil {
|
||||
continue
|
||||
}
|
||||
msg := fmt.Sprintf("Version %s is available (you have %s).", latest.Version, version)
|
||||
app.Dialog.Info().
|
||||
SetTitle("Update Available").
|
||||
SetMessage(msg).
|
||||
Show()
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Main ----
|
||||
|
||||
func main() {
|
||||
cfg := loadConfig()
|
||||
oidcSrv := startOIDCServer()
|
||||
defer oidcSrv.Close()
|
||||
|
||||
distFS, err := fs.Sub(assets, "frontend/dist")
|
||||
if err != nil {
|
||||
log.Fatalf("embedded assets: %v", err)
|
||||
}
|
||||
|
||||
app := application.New(application.Options{
|
||||
Name: "Oikos",
|
||||
@@ -312,20 +691,19 @@ func main() {
|
||||
application.NewService(&ConfigService{}),
|
||||
},
|
||||
Assets: application.AssetOptions{
|
||||
Handler: newAssetHandler(cfg),
|
||||
Handler: application.AssetFileServerFS(distFS),
|
||||
},
|
||||
Mac: application.MacOptions{
|
||||
ApplicationShouldTerminateAfterLastWindowClosed: false,
|
||||
},
|
||||
})
|
||||
|
||||
// --- System tray ---
|
||||
systemTray := app.SystemTray.New()
|
||||
systemTray.SetLabel("Oikos")
|
||||
systemTray.SetTooltip("Oikos — Control Room")
|
||||
systemTray.SetTooltip("Oikos")
|
||||
systemTray.SetIcon(iconPNG)
|
||||
|
||||
trayMenu := application.NewMenu()
|
||||
trayMenu.Add("Open Control Room").OnClick(func(ctx *application.Context) {
|
||||
trayMenu.Add("Open Oikos").OnClick(func(ctx *application.Context) {
|
||||
for _, w := range app.Window.GetAll() {
|
||||
w.Show()
|
||||
w.Focus()
|
||||
@@ -333,7 +711,30 @@ func main() {
|
||||
})
|
||||
trayMenu.AddSeparator()
|
||||
trayMenu.Add("Check for Updates").OnClick(func(ctx *application.Context) {
|
||||
go checkUpdates() // force immediate check on demand
|
||||
go func() {
|
||||
latest := fetchLatestRelease()
|
||||
if latest == nil {
|
||||
app.Dialog.Info().SetTitle("Up to Date").SetMessage("You are running the latest version (" + version + ").").Show()
|
||||
return
|
||||
}
|
||||
for _, a := range latest.Assets {
|
||||
if strings.Contains(a.Name, "darwin") {
|
||||
updater.mu.Lock()
|
||||
updater.latestURL = a.BrowserDownloadURL
|
||||
updater.mu.Unlock()
|
||||
msg := fmt.Sprintf("Version %s is available (you have %s). Install now?", latest.Version, version)
|
||||
d := app.Dialog.Question().SetTitle("Update Available").SetMessage(msg)
|
||||
yes := d.AddButton("Install")
|
||||
yes.OnClick(func() { doUpdate(updater.latestURL) })
|
||||
no := d.AddButton("Later")
|
||||
d.SetDefaultButton(yes)
|
||||
d.SetCancelButton(no)
|
||||
d.Show()
|
||||
return
|
||||
}
|
||||
}
|
||||
app.Dialog.Info().SetTitle("Up to Date").SetMessage("You are running the latest version (" + version + ").").Show()
|
||||
}()
|
||||
})
|
||||
trayMenu.AddSeparator()
|
||||
trayMenu.Add("Quit").OnClick(func(ctx *application.Context) {
|
||||
@@ -341,41 +742,46 @@ func main() {
|
||||
})
|
||||
systemTray.SetMenu(trayMenu)
|
||||
|
||||
// --- Main window ---
|
||||
ws := loadWindowState()
|
||||
width, height := 1400, 900
|
||||
minWidth, minHeight := 1024, 700
|
||||
if ws != nil {
|
||||
width = ws.Width
|
||||
height = ws.Height
|
||||
}
|
||||
|
||||
window := app.Window.NewWithOptions(application.WebviewWindowOptions{
|
||||
Title: "Oikos — Control Room",
|
||||
Title: "Oikos",
|
||||
Width: width,
|
||||
Height: height,
|
||||
MinWidth: minWidth,
|
||||
MinHeight: minHeight,
|
||||
URL: "/",
|
||||
MinWidth: 1024,
|
||||
MinHeight: 700,
|
||||
URL: "/?desktop=1",
|
||||
})
|
||||
|
||||
if ws != nil {
|
||||
window.SetPosition(ws.X, ws.Y)
|
||||
window.SetSize(ws.Width, ws.Height)
|
||||
} else {
|
||||
window.Center()
|
||||
}
|
||||
|
||||
window.RegisterHook(events.Common.WindowClosing, func(e *application.WindowEvent) {
|
||||
window.Hide()
|
||||
e.Cancel()
|
||||
})
|
||||
|
||||
window.Show()
|
||||
|
||||
systemTray.AttachWindow(window)
|
||||
systemTray.Run()
|
||||
|
||||
// Register shutdown handler to save window state
|
||||
app.OnShutdown(func() {
|
||||
saveWindowState(window)
|
||||
})
|
||||
|
||||
// Start background goroutines
|
||||
go pollDashboard(cfg)
|
||||
go pollDashboard(loadConfig())
|
||||
go checkUpdates()
|
||||
|
||||
err := app.Run()
|
||||
err = app.Run()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
5
cmd/desktop/tray-icon.svg
Normal file
5
cmd/desktop/tray-icon.svg
Normal file
@@ -0,0 +1,5 @@
|
||||
<svg width="88" height="88" viewBox="0 0 110 120" xmlns="http://www.w3.org/2000/svg">
|
||||
<g transform="translate(9, 10) scale(0.9)">
|
||||
<path fill="#ffffff" d="m45.601 1q20.993 0 33.71 15.946 10.799 13.625 10.799 31.287 0 12.414-5.9548 25.131-5.9548 12.717-16.451 19.176-10.395 6.4592-23.213 6.4592-20.892 0-33.205-16.653-10.395-14.029-10.395-31.489 0-12.717 6.2577-25.232 6.3584-12.616 16.653-18.57 10.295-6.0556 21.801-6.0556zm-3.128 6.5605q-5.3492 0-10.799 3.2296-5.3492 3.1287-8.68 11.102-3.3305 7.9735-3.3305 20.488 0 20.185 7.973 34.82 8.0743 14.634 21.195 14.634 9.7896 0 16.149-8.0743 6.3584-8.0743 6.3584-27.755 0-24.627-10.597-38.756-7.1657-9.6888-18.268-9.6888z"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 690 B |
@@ -17,12 +17,12 @@ import (
|
||||
)
|
||||
|
||||
// maxIterations bounds one chat turn's tool-calling loop. Provisioning a
|
||||
// service is a long chain (research → plan → request_execution → per-step
|
||||
// service is a long chain (research → plan → run → per-step
|
||||
// install/verify run calls), so this must be generous; a full deploy with the
|
||||
// decomposed pct_create flow can legitimately need many steps. On exhaustion
|
||||
// the loop now produces a real summary (finalSummary) rather than a dead end.
|
||||
const maxIterations = 40
|
||||
const maxLLMRetries = 1
|
||||
const maxLLMRetries = 3
|
||||
|
||||
// historyWindowSize bounds how many of a session's most recent persisted
|
||||
// messages are replayed into the LLM's context on each turn — see
|
||||
@@ -129,7 +129,7 @@ func loadSoul() string {
|
||||
}
|
||||
return `You are Nomos, the steward of the oikos — the AI agent for the hubris homelab.
|
||||
You have access to MCP tools to query topology, health, knowledge, and request
|
||||
gated mutations through request_execution. Be concise. Prefer tools over guessing.`
|
||||
gated mutations through run. Be concise. Prefer tools over guessing.`
|
||||
}
|
||||
|
||||
// assentWindowDuration is how long after an operator approves a plan that
|
||||
@@ -168,10 +168,10 @@ type toolDef struct {
|
||||
}
|
||||
|
||||
type agentEvent struct {
|
||||
Type string `json:"type"`
|
||||
Data any `json:"data,omitempty"`
|
||||
SessionID string `json:"session_id,omitempty"`
|
||||
Iteration int `json:"iteration,omitempty"`
|
||||
Type string `json:"type"`
|
||||
Data any `json:"data,omitempty"`
|
||||
SessionID string `json:"session_id,omitempty"`
|
||||
Iteration int `json:"iteration,omitempty"`
|
||||
}
|
||||
|
||||
func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(agentEvent)) {
|
||||
@@ -187,9 +187,30 @@ func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(a
|
||||
func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject string, emit func(agentEvent)) {
|
||||
correlationID := uuid.New().String()
|
||||
|
||||
// emitError emits an error event followed by a done event. The done
|
||||
// event is CRITICAL on every terminal path: the frontend's
|
||||
// onComplete handler (chat.ts) treats a missing `done` as a severed
|
||||
// network connection and triggers an auto-reconnect → resumeSession.
|
||||
// Before this fix, a model empty-response (the most common case here)
|
||||
// returned without `done`, was misclassified as a network drop, and
|
||||
// the reconnect logic re-invoked the agent with a generic "report
|
||||
// your state" note — which caused the agent to re-propose the plan
|
||||
// and duplicate it in the sidebar (operator-reported 2026-07-14).
|
||||
// Every error return below must go through emitError so the frontend
|
||||
// shows the error inline instead of silently reconnecting.
|
||||
emitError := func(data string) {
|
||||
emit(agentEvent{Type: "error", Data: data, SessionID: sessionID})
|
||||
emit(agentEvent{Type: "done", Data: map[string]any{
|
||||
"session_id": sessionID,
|
||||
"correlation_id": correlationID,
|
||||
"iterations": 0,
|
||||
"error": true,
|
||||
}, SessionID: sessionID})
|
||||
}
|
||||
|
||||
tools, err := a.buildTools(sessionID)
|
||||
if err != nil {
|
||||
emit(agentEvent{Type: "error", Data: fmt.Sprintf("build tools: %v", err), SessionID: sessionID})
|
||||
emitError(fmt.Sprintf("build tools: %v", err))
|
||||
return
|
||||
}
|
||||
|
||||
@@ -275,8 +296,6 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
if ok {
|
||||
granted = append(granted, p.execID)
|
||||
slog.Info("nomos: chat-assent granted", "execution", p.execID, "status", status, "session", sessionID)
|
||||
emit(agentEvent{Type: "tool_use", Data: map[string]any{"name": "chat_assent", "args": map[string]any{"execution_id": p.execID}, "id": "assent-" + p.execID}, SessionID: sessionID})
|
||||
emit(agentEvent{Type: "tool_result", Data: map[string]any{"name": "chat_assent", "result": fmt.Sprintf("Approved via chat assent (%q). Status: %s.", message, status), "id": "assent-" + p.execID}, SessionID: sessionID})
|
||||
|
||||
// An explicit typed confirmation for a destructive action
|
||||
// opens a short, target-scoped window so the rest of a
|
||||
@@ -294,21 +313,33 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
}
|
||||
if len(granted) > 0 {
|
||||
a.openAssentWindow(ctx, sessionID)
|
||||
note := fmt.Sprintf("[System: the operator's message approved pending execution(s) %s via chat assent — they are now running. An assent window is now active for 30 minutes: config_mutation commands will auto-run without re-approval. Do not re-request or call request_execution/run again for these; check get_execution_status if you need the outcome. CONTINUE executing the full plan — do not stop and wait for 'continue' after each step. Only surface to the operator for destructive actions (need typed confirmation) or if you're genuinely stuck after trying alternatives.]", strings.Join(granted, ", "))
|
||||
messages = append(messages, openai.SystemMessage(note))
|
||||
// Mark approved executions as continued so the continuation
|
||||
// worker doesn't call resumeSession while the chat handler is
|
||||
// still processing "go ahead" — two concurrent LLM calls for the
|
||||
// same session cause empty responses and race conditions.
|
||||
for _, execID := range granted {
|
||||
if execUUID, perr := uuid.Parse(execID); perr == nil {
|
||||
a.store.markContinued(ctx, execUUID)
|
||||
}
|
||||
}
|
||||
// No system note. The model already sees "go ahead" in the
|
||||
// replayed history (the user message was saved to the DB before
|
||||
// chat() was called). The old note said "they are now running"
|
||||
// which made the model think work was being done for it —
|
||||
// causing empty responses (finish_reason=stop, content_len=0).
|
||||
// The approved executions are dispatched; the model will
|
||||
// continue with the remaining plan steps naturally.
|
||||
}
|
||||
if len(blocked) > 0 {
|
||||
note := fmt.Sprintf("[System: execution(s) %s are classified DESTRUCTIVE and were NOT approved by loose assent — you must ask the operator for an explicit typed confirmation before they can run. Once they do confirm, further destructive steps on that SAME target (e.g. finishing a stop-then-destroy sequence) will auto-run for 15 minutes without asking again — but a different target always needs its own confirmation.]", strings.Join(blocked, ", "))
|
||||
messages = append(messages, openai.SystemMessage(note))
|
||||
}
|
||||
} else if assent && len(lastAssistantCalls) == 0 {
|
||||
// The operator said "proceed"/"go ahead"/"yes" but the preceding
|
||||
// assistant turn had NO pending approvals — meaning the agent
|
||||
// proposed a plan in text and asked "shall I?" without calling
|
||||
// request_execution yet. Inject a system note telling the agent
|
||||
// the operator approved — go execute the plan now.
|
||||
note := "[System: The operator approved your proposed plan. Execute it now — call request_execution or run to carry out the steps you described. Do not re-describe the plan or ask for confirmation again. The assent window is active: config_mutation commands will auto-run once you create them.]"
|
||||
messages = append(messages, openai.SystemMessage(note))
|
||||
} else if assent && len(pending) == 0 {
|
||||
// The operator said "proceed"/"go ahead"/"yes" but there are no
|
||||
// pending approvals — the agent proposed a plan (via propose_plan)
|
||||
// and asked "shall I?" Open the assent window silently. No system
|
||||
// note: the model sees "go ahead" in the replayed history and
|
||||
// responds naturally.
|
||||
a.openAssentWindow(ctx, sessionID)
|
||||
}
|
||||
|
||||
@@ -318,6 +349,15 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
messages = append(messages, openai.SystemMessage(systemInject))
|
||||
}
|
||||
|
||||
// Retry cap (P0.1 from plans/2026-07-18-session-review-three-sessions.md):
|
||||
// track failing `run` calls within this turn so an identical command that
|
||||
// keeps failing is refused after maxRunRetries attempts. Without this,
|
||||
// session 1e9c7691 retried the same `chown` ~20 times, each retry piling
|
||||
// up a zombie process on the target (knfsd was holding a kernel lock).
|
||||
// The tracker is per-turn — a fresh turn after the operator responds can
|
||||
// retry once more, so this doesn't permanently block recovery.
|
||||
retries := newRunRetryTracker()
|
||||
|
||||
for i := 0; i < maxIterations; i++ {
|
||||
params := openai.ChatCompletionNewParams{
|
||||
Model: openai.ChatModel(a.model),
|
||||
@@ -345,7 +385,7 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
slog.Warn("nomos: llm stream error, retrying", "error", err, "attempt", attempt+1, "session", sessionID)
|
||||
continue
|
||||
}
|
||||
emit(agentEvent{Type: "error", Data: fmt.Sprintf("llm: %v", err), SessionID: sessionID})
|
||||
emitError(fmt.Sprintf("llm: %v", err))
|
||||
return
|
||||
}
|
||||
if len(acc.Choices) == 0 {
|
||||
@@ -353,21 +393,32 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
slog.Warn("nomos: no choices in response, retrying", "attempt", attempt+1, "session", sessionID)
|
||||
continue
|
||||
}
|
||||
emit(agentEvent{Type: "error", Data: "no choices in response", SessionID: sessionID})
|
||||
emitError("no choices in response (the model returned zero completions — likely a provider or rate-limit issue)")
|
||||
return
|
||||
}
|
||||
|
||||
msg = acc.Choices[0].Message
|
||||
finishReason := acc.Choices[0].FinishReason
|
||||
|
||||
if len(msg.ToolCalls) == 0 {
|
||||
if isRefusalOrEmpty(msg.Content) {
|
||||
if attempt < maxLLMRetries {
|
||||
slog.Warn("nomos: empty or refusal response, retrying",
|
||||
"session", sessionID, "iter", i+1, "attempt", attempt+1,
|
||||
"content_len", len(msg.Content))
|
||||
"content_len", len(msg.Content), "finish_reason", finishReason)
|
||||
continue
|
||||
}
|
||||
emit(agentEvent{Type: "error", Data: "Nomos returned an empty or unusable response — please retry.", SessionID: sessionID})
|
||||
// B.4: surface the real error context (finish_reason +
|
||||
// refusal text) instead of a generic "empty response" —
|
||||
// the operator can tell "content_filter — rephrase" from
|
||||
// "length — token limit hit" from "stop — model no-op'd".
|
||||
detail := "empty response"
|
||||
if msg.Refusal != "" {
|
||||
detail = fmt.Sprintf("refusal: %s", msg.Refusal)
|
||||
} else if finishReason != "" && finishReason != "stop" {
|
||||
detail = fmt.Sprintf("finish_reason=%s", finishReason)
|
||||
}
|
||||
emitError(fmt.Sprintf("Nomos returned an empty or unusable response (%s). Retry or rephrase.", detail))
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -379,6 +430,14 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
if !sawSetGoal && !sawCompleteTask {
|
||||
a.autoCompleteTrivialTask(ctx, sessionID, msg.Content)
|
||||
}
|
||||
// Safety net: if the agent called set_goal (structured task)
|
||||
// but didn't call complete_task, and all plan steps are
|
||||
// terminal, auto-complete. The model often does the work but
|
||||
// forgets to close the loop (confirmed live: the #1 remaining
|
||||
// model reliability gap after D.1).
|
||||
if !sawCompleteTask {
|
||||
a.autoCompleteIfPlanDone(ctx, sessionID, msg.Content)
|
||||
}
|
||||
emit(agentEvent{Type: "done", Data: map[string]any{
|
||||
"session_id": sessionID,
|
||||
"usage": acc.Usage,
|
||||
@@ -388,6 +447,18 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
return
|
||||
}
|
||||
|
||||
// P3: persist intermediate reasoning. When the model produces text
|
||||
// AND tool calls in the same iteration, the text is its reasoning
|
||||
// before the tool calls — the operator saw it live via text_delta,
|
||||
// but without emitting it as a `text` event here, the persist layer
|
||||
// (main.go/continue.go) never captures it and a reload shows only
|
||||
// the final summary + a flat tool-call list, not the thinking that
|
||||
// led to each step. Emitting it lets the persist layer accumulate
|
||||
// per-iteration reasoning into the row's text field.
|
||||
if strings.TrimSpace(msg.Content) != "" {
|
||||
emit(agentEvent{Type: "text", Data: msg.Content, SessionID: sessionID})
|
||||
}
|
||||
|
||||
slog.Info("nomos: tool calls", "count", len(msg.ToolCalls), "iter", i+1, "correlation", correlationID)
|
||||
|
||||
messages = append(messages, msg.ToParam())
|
||||
@@ -405,6 +476,33 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
sawCompleteTask = true
|
||||
}
|
||||
|
||||
// Retry cap: if this `run` call has already failed
|
||||
// maxRunRetries times this turn with the same (target,
|
||||
// command), refuse to dispatch it again. Return a synthetic
|
||||
// tool result directing the agent to investigate *why* the
|
||||
// command hangs instead of retrying. See retrycap.go and
|
||||
// plans/2026-07-18-session-review-three-sessions.md P0.1.
|
||||
if tc.Function.Name == "run" {
|
||||
t, _ := args["target"].(string)
|
||||
c, _ := args["command"].(string)
|
||||
key := runFailureKey(t, c)
|
||||
if n := retries.failures(key); n >= maxRunRetries {
|
||||
directive := runRetryDirective(t, c, n)
|
||||
slog.Warn("nomos: run retry cap hit — refusing dispatch",
|
||||
"target", t, "failures", n, "session", sessionID)
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, args,
|
||||
tc.Function.Arguments, directive, 0, false, correlationID)
|
||||
emit(agentEvent{
|
||||
Type: "tool_result",
|
||||
Data: map[string]any{"name": tc.Function.Name, "result": directive, "id": tc.ID, "retry_capped": true},
|
||||
SessionID: sessionID,
|
||||
Iteration: i + 1,
|
||||
})
|
||||
messages = append(messages, openai.ToolMessage(directive, tc.ID))
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
emit(agentEvent{
|
||||
Type: "tool_use",
|
||||
Data: map[string]any{"name": tc.Function.Name, "args": args, "id": tc.ID},
|
||||
@@ -446,6 +544,22 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
if callErr != nil {
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, args, inputStr, callErr.Error(), elapsed, false, correlationID)
|
||||
|
||||
// Retry cap: dispatch errors (e.g. MCP client timeout)
|
||||
// count toward the cap too. A command that keeps timing
|
||||
// out at the gateway is exactly the pattern we want to
|
||||
// break — see session 1e9c7691's 20+ identical
|
||||
// `chown` timeouts.
|
||||
if tc.Function.Name == "run" {
|
||||
t, _ := args["target"].(string)
|
||||
c, _ := args["command"].(string)
|
||||
key := runFailureKey(t, c)
|
||||
n := retries.recordFailure(key)
|
||||
if n >= maxRunRetries {
|
||||
slog.Warn("nomos: run failure cap reached — next identical call will be refused",
|
||||
"target", t, "failures", n, "session", sessionID)
|
||||
}
|
||||
}
|
||||
|
||||
emit(agentEvent{
|
||||
Type: "tool_result",
|
||||
Data: map[string]any{"name": tc.Function.Name, "error": callErr.Error(), "id": tc.ID},
|
||||
@@ -489,6 +603,25 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
messages = append(messages, openai.ToolMessage(string(resultJSON), tc.ID))
|
||||
slog.Info("nomos: tool success", "tool", tc.Function.Name, "ms", elapsed)
|
||||
|
||||
// Retry cap: record failures of `run` calls so the cap above
|
||||
// can refuse a repeated identical failure. A "failure" here
|
||||
// means the dispatch errored OR the MCP result text matches
|
||||
// the "run on <target>: ERROR …" signature — both indicate
|
||||
// the command actually ran and failed, not just that it
|
||||
// queued for approval (pending approvals are not failures).
|
||||
// Pass the RAW result text (not JSON-encoded) so the helper's
|
||||
// HasPrefix check sees "run on …" not "\"run on …\"".
|
||||
if isRunFailure(tc.Function.Name, runResultText(result), callErr) {
|
||||
t, _ := args["target"].(string)
|
||||
c, _ := args["command"].(string)
|
||||
key := runFailureKey(t, c)
|
||||
n := retries.recordFailure(key)
|
||||
if n >= maxRunRetries {
|
||||
slog.Warn("nomos: run failure cap reached — next identical call will be refused",
|
||||
"target", t, "failures", n, "session", sessionID)
|
||||
}
|
||||
}
|
||||
|
||||
// ask_operator pauses the task: the agent has posed a decision only
|
||||
// the operator can make. End the turn here so it doesn't barrel past
|
||||
// its own question — the answer (panel or chat reply) resumes it.
|
||||
@@ -518,6 +651,9 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
summary = "I hit this turn's step limit while working. I've done a lot but couldn't wrap up cleanly — ask me for a status update and I'll summarize the current state."
|
||||
}
|
||||
emit(agentEvent{Type: "text", Data: summary, SessionID: sessionID})
|
||||
if !sawCompleteTask {
|
||||
a.autoCompleteIfPlanDone(ctx, sessionID, summary)
|
||||
}
|
||||
emit(agentEvent{Type: "done", Data: map[string]any{
|
||||
"session_id": sessionID,
|
||||
"correlation_id": correlationID,
|
||||
|
||||
@@ -52,7 +52,7 @@ func (a *agent) runIdleSweepWorker(ctx context.Context) {
|
||||
return
|
||||
}
|
||||
slog.Info("nomos: idle sweep worker started")
|
||||
ticker := time.NewTicker(5 * time.Minute)
|
||||
ticker := time.NewTicker(2 * time.Minute)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
@@ -80,11 +80,12 @@ func (a *agent) processIdleSweep(ctx context.Context) {
|
||||
slog.Error("nomos: idle nudge bump failed", "session", s.ID, "error", err)
|
||||
return
|
||||
}
|
||||
note := fmt.Sprintf("[System: this task ('%s') has been idle for %s with no complete_task call. "+
|
||||
"If the goal is done (or can't be completed), call complete_task now with the outcome and a "+
|
||||
"one-line summary. If you're still genuinely working through the plan, ignore this and continue.]",
|
||||
s.Goal, idleTaskThreshold)
|
||||
a.resumeSession(ctx, s.ID, note)
|
||||
note := fmt.Sprintf("[System: this task ('%s') has been idle for %s with no complete_task call. "+
|
||||
"If the goal is done (or can't be completed), call complete_task now with the outcome and a "+
|
||||
"one-line summary. If you're still genuinely working through the plan, ignore this and continue.]",
|
||||
s.Goal, idleTaskThreshold)
|
||||
note = a.store.enrichResumeNote(ctx, s.ID, note)
|
||||
a.resumeSession(ctx, s.ID, note)
|
||||
})
|
||||
continue
|
||||
}
|
||||
@@ -143,11 +144,24 @@ func (a *agent) processContinuations(ctx context.Context) {
|
||||
// multiple tasks in flight, one task's open window must never cover a
|
||||
// pending continuation belonging to a different task.
|
||||
if !a.store.assentWindowActive(ctx, a.agentID, p.SessionID) {
|
||||
// A finished one-off execution with no window is left as-is
|
||||
// (marked continued so we don't re-check it forever) — the
|
||||
// operator decides what happens next, as today.
|
||||
a.store.markContinued(ctx, p.ExecID)
|
||||
continue
|
||||
// Re-open the assent window if this session is genuinely
|
||||
// executing (plan was approved, work is in progress) — the
|
||||
// window may have expired while the execution ran. Don't
|
||||
// penalize timing: the plan was approved, the work happened,
|
||||
// the result should flow back.
|
||||
sesh, seshErr := a.store.getSession(ctx, p.SessionID)
|
||||
if seshErr == nil && sesh.Goal != "" && (sesh.Status == "executing" || sesh.Status == "planning") {
|
||||
a.openAssentWindow(ctx, p.SessionID)
|
||||
slog.Info("nomos: re-opened assent window for continuing session", "session", p.SessionID, "execution", p.ExecID)
|
||||
} else {
|
||||
// Genuinely no plan — inject a visible note so the
|
||||
// operator knows WHY the agent didn't auto-continue.
|
||||
note := fmt.Sprintf("[System: execution %s finished with status=%s, but the assent window for this session is not active. The agent will not auto-continue. Reply 'continue' or re-approve the plan to resume.]", p.ExecID, p.Status)
|
||||
body, _ := json.Marshal(map[string]any{"role": "assistant", "text": note, "auto": true})
|
||||
a.store.saveMessage(context.Background(), p.SessionID, "assistant", body)
|
||||
a.store.markContinued(ctx, p.ExecID)
|
||||
continue
|
||||
}
|
||||
}
|
||||
a.store.markContinued(ctx, p.ExecID) // stamp first: a failure here must not cause a re-continue loop
|
||||
safego.Go("nomos:continue-session:"+p.SessionID, func() { a.continueSession(ctx, p) })
|
||||
@@ -213,49 +227,94 @@ func (a *agent) resumeSession(ctx context.Context, sessionID, note string) {
|
||||
// without this outer retry the operator would see nothing at all.
|
||||
cctx, cancel := context.WithTimeout(ctx, 10*time.Minute)
|
||||
defer cancel()
|
||||
for attempt := 0; attempt < 2; attempt++ {
|
||||
// B.3: escalate the recovery note across attempts — a transient flake
|
||||
// needs a different prompt than a model that's stuck no-op'ing. The
|
||||
// final attempt is maximally directive ("do this specific thing now").
|
||||
// B.5: back off between retries (4s, 8s) so a transient provider issue
|
||||
// has time to clear — 3 identical calls in 3 seconds just get 3
|
||||
// identical empties.
|
||||
notes := []string{
|
||||
note, // attempt 0: the original (already enriched per B.2) note
|
||||
fmt.Sprintf("[System: your previous turn produced no response. %s. Produce a response now — call the next tool or report progress in one sentence.]", note),
|
||||
fmt.Sprintf("[System: two consecutive empty responses. Stop trying to be clever. The next action is: pick the lowest-pending plan step, mark it running with update_plan_step, and call run for its target. Do that now.]"),
|
||||
}
|
||||
for attempt := 0; attempt < 3; attempt++ {
|
||||
if attempt > 0 {
|
||||
select {
|
||||
case <-cctx.Done():
|
||||
return
|
||||
case <-time.After(time.Duration(2<<attempt) * time.Second): // 4s, 8s
|
||||
}
|
||||
}
|
||||
toolCalls, finalText, errText = nil, "", ""
|
||||
// P3: accumulate per-iteration reasoning instead of overwriting
|
||||
// (same fix as main.go's chat handler). Without this, a resumed
|
||||
// turn's intermediate thinking is lost on reload.
|
||||
var textParts []string
|
||||
emit := func(ev agentEvent) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
m["type"] = ev.Type
|
||||
toolCalls = append(toolCalls, m)
|
||||
// One entry per tool call: tool_use creates it,
|
||||
// tool_result merges the result into the same entry
|
||||
// (matched by id). Before this fix, both events
|
||||
// appended separate entries, doubling every tool call.
|
||||
id, _ := m["id"].(string)
|
||||
if id != "" && ev.Type == "tool_result" {
|
||||
for _, existing := range toolCalls {
|
||||
if eID, _ := existing["id"].(string); eID == id {
|
||||
for k, v := range m {
|
||||
existing[k] = v
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
} else {
|
||||
toolCalls = append(toolCalls, m)
|
||||
}
|
||||
}
|
||||
persist() // live: a poller sees this step land within seconds
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
finalText, _ = ev.Data.(string)
|
||||
if t, ok := ev.Data.(string); ok && t != "" {
|
||||
textParts = append(textParts, t)
|
||||
finalText = strings.Join(textParts, "\n\n")
|
||||
persist()
|
||||
}
|
||||
}
|
||||
if ev.Type == "error" {
|
||||
errText, _ = ev.Data.(string)
|
||||
}
|
||||
}
|
||||
a.chatWith(cctx, sessionID, "", note, emit)
|
||||
a.chatWith(cctx, sessionID, "", notes[attempt], emit)
|
||||
if finalText != "" || len(toolCalls) > 0 {
|
||||
break
|
||||
}
|
||||
if attempt == 0 {
|
||||
slog.Warn("nomos: resume produced nothing, retrying once", "session", sessionID, "error", errText)
|
||||
if attempt < 2 {
|
||||
slog.Warn("nomos: resume produced nothing, retrying", "session", sessionID, "error", errText, "attempt", attempt+1)
|
||||
}
|
||||
}
|
||||
|
||||
if errText != "" && finalText == "" {
|
||||
slog.Error("nomos: resume produced no response after retry", "session", sessionID, "error", errText)
|
||||
// Give the task a real, operator-visible terminal state instead of
|
||||
// leaving it silently stuck at whatever status it was in (typically
|
||||
// 'executing' or 'awaiting_input') forever. Before this, a
|
||||
// permanently-failed resume was invisible beyond a log line — the
|
||||
// task board just showed a task that never changed, with nothing
|
||||
// telling the operator it needed attention. Marking it failed here
|
||||
// doesn't prevent the operator from continuing to work the task via
|
||||
// a fresh chat message afterward; it just stops the silent hang.
|
||||
summary := fmt.Sprintf("Auto-resume failed after retrying: %s", errText)
|
||||
if len(summary) > 200 {
|
||||
summary = summary[:200] + "…"
|
||||
}
|
||||
if cerr := a.store.completeTask(context.Background(), sessionID, "failure", summary); cerr != nil {
|
||||
slog.Error("nomos: failed to mark task failed after resume gave up", "session", sessionID, "error", cerr)
|
||||
// Persist a visible system note in the transcript so the
|
||||
// operator sees what happened, but do NOT auto-complete the
|
||||
// task — leave it in 'executing' so a follow-up chat message
|
||||
// can resume it. Before this fix, the task was marked 'failed'
|
||||
// here, which ended it permanently and required starting over.
|
||||
resumeFailedNote := fmt.Sprintf("[System: auto-resume failed after retrying: %s. The task is paused — send another message to continue.]", errText)
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": resumeFailedNote,
|
||||
"auto": true,
|
||||
})
|
||||
if msgID != uuid.Nil {
|
||||
a.store.updateMessage(context.Background(), msgID, body)
|
||||
} else {
|
||||
// No placeholder was inserted (rare), save directly.
|
||||
a.store.saveMessage(context.Background(), sessionID, "assistant", body)
|
||||
}
|
||||
return // do not call persist() again — already persisted above
|
||||
}
|
||||
persist() // final state — same row, updated one last time with the concluding text
|
||||
}
|
||||
|
||||
370
cmd/nomos/eval/main.go
Normal file
370
cmd/nomos/eval/main.go
Normal file
@@ -0,0 +1,370 @@
|
||||
// Command nomos-eval runs golden conversation evals against a live nomos
|
||||
// gateway. It loads a YAML manifest of conversations + assertions, sends
|
||||
// each prompt to the chat endpoint, waits for the turn(s) to finish, and
|
||||
// scores assertions against the persisted transcript.
|
||||
//
|
||||
// Usage:
|
||||
//
|
||||
// go run ./cmd/nomos/eval -gateway http://localhost:8092 -manifest evals/*.yaml
|
||||
//
|
||||
// The gateway must already be running (nomos serve, or the docker container).
|
||||
// Each conversation costs real OpenRouter credits (~$0.01–0.05 each).
|
||||
//
|
||||
// Manifest format — see evals/example.yaml. Assertions are scored against the
|
||||
// final transcript: tool calls made, plan steps, final session status, and
|
||||
// whether the turn completed. The runner does NOT judge text quality — only
|
||||
// structural properties that can be checked deterministically from the
|
||||
// persisted state. This is deliberate: text quality is model-dependent and
|
||||
// noisy; structure is what the Go gates + SOUL.md should enforce.
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
)
|
||||
|
||||
func main() {
|
||||
gateway := flag.String("gateway", "http://localhost:8092", "nomos gateway URL")
|
||||
manifestGlob := flag.String("manifest", "evals/*.yaml", "glob of manifest files to run")
|
||||
timeout := flag.Duration("timeout", 2*time.Minute, "per-conversation timeout")
|
||||
flag.Parse()
|
||||
|
||||
if err := health(*gateway); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "gateway not reachable at %s: %v\n", *gateway, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
files, err := filepath.Glob(*manifestGlob)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "glob %s: %v\n", *manifestGlob, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if len(files) == 0 {
|
||||
fmt.Fprintf(os.Stderr, "no manifests matched %s\n", *manifestGlob)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
total, passed, failed := 0, 0, 0
|
||||
for _, f := range files {
|
||||
convs, err := loadManifest(f)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "load %s: %v\n", f, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
for _, c := range convs {
|
||||
total++
|
||||
name := c.Name
|
||||
if name == "" {
|
||||
name = fmt.Sprintf("conversation-%d", total)
|
||||
}
|
||||
fmt.Printf("=== %s (from %s) ===\n", name, filepath.Base(f))
|
||||
res := runConversation(context.Background(), *gateway, c, *timeout)
|
||||
if res.Passed {
|
||||
passed++
|
||||
fmt.Printf(" ✅ PASS (%.1fs, %d tool calls)\n", res.Duration.Seconds(), res.ToolCallCount)
|
||||
} else {
|
||||
failed++
|
||||
fmt.Printf(" ❌ FAIL (%.1fs, %d tool calls)\n", res.Duration.Seconds(), res.ToolCallCount)
|
||||
}
|
||||
for _, a := range res.Assertions {
|
||||
mark := "✅"
|
||||
if !a.Passed {
|
||||
mark = "❌"
|
||||
}
|
||||
fmt.Printf(" %s %s: %s\n", mark, a.Name, a.Detail)
|
||||
}
|
||||
}
|
||||
}
|
||||
fmt.Printf("\n=== Summary: %d/%d passed, %d failed ===\n", passed, total, failed)
|
||||
if failed > 0 {
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func health(gateway string) error {
|
||||
resp, err := http.Get(gateway + "/healthz")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != 200 {
|
||||
return fmt.Errorf("healthz status %d", resp.StatusCode)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// runConversation sends the prompt (and any followup), waits for each turn to
|
||||
// finish, then scores assertions against the final transcript.
|
||||
func runConversation(ctx context.Context, gateway string, c conversation, timeout time.Duration) convResult {
|
||||
start := time.Now()
|
||||
deadline := time.Now().Add(timeout)
|
||||
res := convResult{}
|
||||
|
||||
// Send the initial prompt (no session_id → creates a new session).
|
||||
sid, err := sendChat(ctx, gateway, "", c.Prompt)
|
||||
if err != nil {
|
||||
res.Assertions = []assertionResult{{Name: "send_prompt", Passed: false, Detail: err.Error()}}
|
||||
res.Duration = time.Since(start)
|
||||
return res
|
||||
}
|
||||
res.SessionID = sid
|
||||
|
||||
// Wait for the first turn to finish.
|
||||
if err := waitForTurn(ctx, gateway, sid, deadline); err != nil {
|
||||
res.Assertions = []assertionResult{{Name: "turn_complete", Passed: false, Detail: err.Error()}}
|
||||
res.Duration = time.Since(start)
|
||||
return res
|
||||
}
|
||||
|
||||
// Send followup if any.
|
||||
for _, fu := range c.followups() {
|
||||
if _, err := sendChat(ctx, gateway, sid, fu); err != nil {
|
||||
res.Assertions = []assertionResult{{Name: "send_followup", Passed: false, Detail: err.Error()}}
|
||||
res.Duration = time.Since(start)
|
||||
return res
|
||||
}
|
||||
if err := waitForTurn(ctx, gateway, sid, deadline); err != nil {
|
||||
res.Assertions = []assertionResult{{Name: "followup_turn_complete", Passed: false, Detail: err.Error()}}
|
||||
res.Duration = time.Since(start)
|
||||
return res
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch the final transcript + session state.
|
||||
transcript, session, err := fetchTranscript(ctx, gateway, sid)
|
||||
if err != nil {
|
||||
res.Assertions = []assertionResult{{Name: "fetch_transcript", Passed: false, Detail: err.Error()}}
|
||||
res.Duration = time.Since(start)
|
||||
return res
|
||||
}
|
||||
res.ToolCallCount = transcript.toolCallCount()
|
||||
res.Duration = time.Since(start)
|
||||
|
||||
// Score assertions.
|
||||
res.Assertions = scoreAssertions(c.Assertions, transcript, session)
|
||||
|
||||
res.Passed = true
|
||||
for _, a := range res.Assertions {
|
||||
if !a.Passed {
|
||||
res.Passed = false
|
||||
break
|
||||
}
|
||||
}
|
||||
return res
|
||||
}
|
||||
|
||||
// sendChat POSTs to /chat and extracts the session_id from the first SSE
|
||||
// event, then KEEPS READING the stream until it ends (the `done` event or
|
||||
// the connection closes). This is critical: the chat handler uses
|
||||
// r.Context() which cancels when the HTTP connection closes — if we stop
|
||||
// reading after the session event, the agent's work gets canceled mid-turn.
|
||||
// We must drain the full stream so the agent completes its turn server-side.
|
||||
func sendChat(ctx context.Context, gateway, sid, message string) (string, error) {
|
||||
body, _ := json.Marshal(map[string]string{"session_id": sid, "message": message})
|
||||
req, _ := http.NewRequestWithContext(ctx, "POST", gateway+"/chat", bytes.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != 200 && resp.StatusCode != 202 {
|
||||
b, _ := io.ReadAll(resp.Body)
|
||||
return "", fmt.Errorf("chat status %d: %s", resp.StatusCode, string(b))
|
||||
}
|
||||
// For a reconnect (sid != ""), the body is 202 with no stream.
|
||||
if sid != "" {
|
||||
io.Copy(io.Discard, resp.Body)
|
||||
return sid, nil
|
||||
}
|
||||
// Read the SSE stream, capturing the session_id from the first session
|
||||
// event, and draining the rest so the agent's turn completes. The stream
|
||||
// ends when the server closes it (after the `done` event) or when the
|
||||
// request context cancels.
|
||||
dec := newSSEReader(resp.Body)
|
||||
sessionID := ""
|
||||
for {
|
||||
ev, err := dec.next()
|
||||
if err != nil {
|
||||
if sessionID == "" {
|
||||
return "", fmt.Errorf("no session event before stream end: %w", err)
|
||||
}
|
||||
return sessionID, nil
|
||||
}
|
||||
if ev["type"] == "session" && sessionID == "" {
|
||||
if s, ok := ev["session_id"].(string); ok {
|
||||
sessionID = s
|
||||
}
|
||||
}
|
||||
// Keep reading until the stream ends — don't return early.
|
||||
}
|
||||
}
|
||||
|
||||
// waitForTurn polls the session until its last_active_at stops advancing for
|
||||
// 8 seconds (the turn ended) or the session reaches a terminal status. We
|
||||
// can't rely on status=done alone because a trivial task may auto-complete
|
||||
// while a plan-proposing task stays in 'executing' waiting for approval.
|
||||
func waitForTurn(ctx context.Context, gateway, sid string, deadline time.Time) error {
|
||||
var lastActive string
|
||||
stableSince := time.Now()
|
||||
for {
|
||||
if time.Now().After(deadline) {
|
||||
return fmt.Errorf("timeout waiting for turn to complete")
|
||||
}
|
||||
_, session, err := fetchTranscript(ctx, gateway, sid)
|
||||
if err != nil {
|
||||
time.Sleep(2 * time.Second)
|
||||
continue
|
||||
}
|
||||
if session.LastActive != lastActive {
|
||||
lastActive = session.LastActive
|
||||
stableSince = time.Now()
|
||||
}
|
||||
if time.Since(stableSince) >= 8*time.Second {
|
||||
return nil // turn is idle — consider it complete
|
||||
}
|
||||
if session.Status == "done" || session.Status == "failed" {
|
||||
return nil
|
||||
}
|
||||
time.Sleep(2 * time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
type transcript struct {
|
||||
Messages []struct {
|
||||
Role string `json:"role"`
|
||||
Content struct {
|
||||
Text string `json:"text"`
|
||||
ToolCalls []map[string]any `json:"tool_calls"`
|
||||
} `json:"content"`
|
||||
} `json:"messages"`
|
||||
// PlanSteps is fetched from /sessions/{id}/plan (P5 plan_generations
|
||||
// assertion). Each step carries a `generation` int; distinctGenerations
|
||||
// counts the unique values. nil when the endpoint returned no plan
|
||||
// (e.g. a pure-DB Q&A with no propose_plan call).
|
||||
PlanSteps []planStep `json:"steps"`
|
||||
}
|
||||
|
||||
// planStep is one step from /sessions/{id}/plan, carrying only the fields the
|
||||
// eval needs: the generation number (P2 iteration counter).
|
||||
type planStep struct {
|
||||
Generation int `json:"generation"`
|
||||
Status string `json:"status"`
|
||||
Title string `json:"title"`
|
||||
}
|
||||
|
||||
func (t transcript) toolCallCount() int {
|
||||
n := 0
|
||||
for _, m := range t.Messages {
|
||||
n += len(m.Content.ToolCalls)
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
func (t transcript) toolNames() []string {
|
||||
var names []string
|
||||
for _, m := range t.Messages {
|
||||
for _, tc := range m.Content.ToolCalls {
|
||||
if name, ok := tc["name"].(string); ok {
|
||||
names = append(names, name)
|
||||
}
|
||||
}
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
// distinctGenerations counts unique plan generation values across all plan
|
||||
// steps. Used by the `plan_generations` assertion (P2 iteration). Returns 0
|
||||
// when there are no plan steps (no propose_plan was called).
|
||||
func (t transcript) distinctGenerations() int {
|
||||
seen := map[int]bool{}
|
||||
for _, s := range t.PlanSteps {
|
||||
seen[s.Generation] = true
|
||||
}
|
||||
return len(seen)
|
||||
}
|
||||
|
||||
type sessionState struct {
|
||||
ID string `json:"id"`
|
||||
Status string `json:"status"`
|
||||
Outcome string `json:"outcome"`
|
||||
LastActive string `json:"last_active_at"`
|
||||
}
|
||||
|
||||
// fetchTranscript fetches the messages from /sessions/{id} (which returns
|
||||
// only session_id + messages) and the session metadata from /sessions
|
||||
// (which returns status/outcome/last_active_at for each session). P5 also
|
||||
// fetches /sessions/{id}/plan for the plan_generations assertion.
|
||||
func fetchTranscript(ctx context.Context, gateway, sid string) (transcript, sessionState, error) {
|
||||
var t transcript
|
||||
resp, err := http.Get(gateway + "/sessions/" + sid)
|
||||
if err != nil {
|
||||
return t, sessionState{}, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
b, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return t, sessionState{}, err
|
||||
}
|
||||
if err := json.Unmarshal(b, &t); err != nil {
|
||||
return t, sessionState{}, err
|
||||
}
|
||||
// Fetch the plan (steps with generation numbers) for the
|
||||
// plan_generations assertion. A 404 or empty response is fine — a
|
||||
// pure-DB Q&A with no propose_plan has no plan.
|
||||
if planResp, perr := http.Get(gateway + "/sessions/" + sid + "/plan"); perr == nil {
|
||||
if planResp.StatusCode == 200 {
|
||||
pb, _ := io.ReadAll(planResp.Body)
|
||||
_ = json.Unmarshal(pb, &t) // fills t.PlanSteps via "steps" field
|
||||
}
|
||||
planResp.Body.Close()
|
||||
}
|
||||
// The detail endpoint doesn't return status/outcome — fetch from the
|
||||
// sessions list and find the matching id.
|
||||
s, err := fetchSessionMeta(ctx, gateway, sid)
|
||||
return t, s, err
|
||||
}
|
||||
|
||||
// fetchSessionMeta fetches /sessions and extracts the one matching sid.
|
||||
func fetchSessionMeta(ctx context.Context, gateway, sid string) (sessionState, error) {
|
||||
resp, err := http.Get(gateway + "/sessions")
|
||||
if err != nil {
|
||||
return sessionState{}, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
var list struct {
|
||||
Sessions []sessionState `json:"sessions"`
|
||||
}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&list); err != nil {
|
||||
return sessionState{}, err
|
||||
}
|
||||
for _, s := range list.Sessions {
|
||||
if s.ID == sid {
|
||||
return s, nil
|
||||
}
|
||||
}
|
||||
return sessionState{}, fmt.Errorf("session %s not found in list", sid)
|
||||
}
|
||||
|
||||
// convResult is the outcome of one conversation.
|
||||
type convResult struct {
|
||||
SessionID string
|
||||
Passed bool
|
||||
Duration time.Duration
|
||||
ToolCallCount int
|
||||
Assertions []assertionResult
|
||||
}
|
||||
|
||||
type assertionResult struct {
|
||||
Name string
|
||||
Passed bool
|
||||
Detail string
|
||||
}
|
||||
236
cmd/nomos/eval/manifest.go
Normal file
236
cmd/nomos/eval/manifest.go
Normal file
@@ -0,0 +1,236 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// conversation is one golden conversation from a manifest.
|
||||
type conversation struct {
|
||||
Name string `yaml:"name"`
|
||||
Prompt string `yaml:"prompt"`
|
||||
Followup string `yaml:"followup"` // backward compat: single followup
|
||||
Followups []string `yaml:"followups"` // P5: multi-turn followups
|
||||
Assertions []assertion `yaml:"assertions"`
|
||||
}
|
||||
|
||||
// followups returns the full list of follow-up messages, supporting both
|
||||
// the single `followup` field (backward compat) and the multi-turn
|
||||
// `followups` list.
|
||||
func (c conversation) followups() []string {
|
||||
if len(c.Followups) > 0 {
|
||||
return c.Followups
|
||||
}
|
||||
if c.Followup != "" {
|
||||
return []string{c.Followup}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// assertion is one check against the final transcript. The `kind` field
|
||||
// selects the scorer; the rest are scorer-specific parameters.
|
||||
//
|
||||
// Supported kinds:
|
||||
//
|
||||
// completes — session status reached done/failed (not stuck executing)
|
||||
// outcome_is — session outcome == value (success/failure/partial)
|
||||
// no_propose_plan — propose_plan was never called
|
||||
// proposes_plan — propose_plan called >= 1 time (plan-always model; P1)
|
||||
// proposes_plan_once — propose_plan was called exactly once
|
||||
// no_duplicate_proposal — propose_plan called at most once
|
||||
// plan_before_run — the first `run` call comes after the first `propose_plan` (P1 ordering gate)
|
||||
// plan_generations — the persisted plan has exactly `value` distinct generations (P2 iteration: 1 = single, 2 = one followup)
|
||||
// writes_back — update_entity_attributes or create_relationship was called
|
||||
// max_tool_calls — total tool calls <= value
|
||||
// max_run_calls — total `run` calls <= value
|
||||
// no_run — `run` was never called
|
||||
// calls_tool — the named tool appears in the transcript
|
||||
// plan_step_count — the plan has exactly `value` steps
|
||||
// no_duplicate_complete — complete_task called at most once
|
||||
type assertion struct {
|
||||
Kind string `yaml:"kind"`
|
||||
Value any `yaml:"value"`
|
||||
}
|
||||
|
||||
// loadManifest reads a YAML file containing a list of conversations.
|
||||
func loadManifest(path string) ([]conversation, error) {
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var convs []conversation
|
||||
if err := yaml.Unmarshal(b, &convs); err != nil {
|
||||
return nil, fmt.Errorf("parse %s: %w", path, err)
|
||||
}
|
||||
return convs, nil
|
||||
}
|
||||
|
||||
// scoreAssertions evaluates each assertion against the transcript + session.
|
||||
func scoreAssertions(asserts []assertion, t transcript, s sessionState) []assertionResult {
|
||||
out := make([]assertionResult, 0, len(asserts))
|
||||
for _, a := range asserts {
|
||||
r := assertionResult{Name: a.Kind}
|
||||
r.Passed, r.Detail = scoreOne(a, t, s)
|
||||
if !r.Passed && r.Detail == "" {
|
||||
r.Detail = "assertion failed"
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func scoreOne(a assertion, t transcript, s sessionState) (bool, string) {
|
||||
tools := t.toolNames()
|
||||
switch a.Kind {
|
||||
case "completes":
|
||||
if s.Status == "done" || s.Status == "failed" {
|
||||
return true, fmt.Sprintf("status=%s", s.Status)
|
||||
}
|
||||
return false, fmt.Sprintf("status=%s (not terminal)", s.Status)
|
||||
|
||||
case "outcome_is":
|
||||
want, _ := a.Value.(string)
|
||||
if s.Outcome == want {
|
||||
return true, fmt.Sprintf("outcome=%s", s.Outcome)
|
||||
}
|
||||
return false, fmt.Sprintf("outcome=%s, want %s", s.Outcome, want)
|
||||
|
||||
case "no_propose_plan":
|
||||
n := countTool(tools, "propose_plan")
|
||||
if n == 0 {
|
||||
return true, "propose_plan not called"
|
||||
}
|
||||
return false, fmt.Sprintf("propose_plan called %d time(s)", n)
|
||||
|
||||
case "proposes_plan":
|
||||
// P1 plan-always: propose_plan called >= 1 time.
|
||||
n := countTool(tools, "propose_plan")
|
||||
if n >= 1 {
|
||||
return true, fmt.Sprintf("propose_plan called %d time(s)", n)
|
||||
}
|
||||
return false, "propose_plan never called (plan-always requires >= 1)"
|
||||
|
||||
case "proposes_plan_once":
|
||||
n := countTool(tools, "propose_plan")
|
||||
if n == 1 {
|
||||
return true, "propose_plan called once"
|
||||
}
|
||||
return false, fmt.Sprintf("propose_plan called %d time(s), want 1", n)
|
||||
|
||||
case "no_duplicate_proposal":
|
||||
n := countTool(tools, "propose_plan")
|
||||
if n <= 1 {
|
||||
return true, fmt.Sprintf("propose_plan called %d time(s)", n)
|
||||
}
|
||||
return false, fmt.Sprintf("propose_plan called %d time(s), want <= 1", n)
|
||||
|
||||
case "plan_before_run":
|
||||
// P1 ordering gate: the first `run` call's global index in the
|
||||
// transcript is strictly greater than the first `propose_plan`
|
||||
// index. Both indices are over the flat tool-call list (across all
|
||||
// messages, in order).
|
||||
planIdx, runIdx := -1, -1
|
||||
for i, name := range tools {
|
||||
if name == "propose_plan" && planIdx == -1 {
|
||||
planIdx = i
|
||||
}
|
||||
if name == "run" && runIdx == -1 {
|
||||
runIdx = i
|
||||
}
|
||||
}
|
||||
if runIdx == -1 {
|
||||
return true, "run never called (ordering trivially satisfied)"
|
||||
}
|
||||
if planIdx == -1 {
|
||||
return false, "run called but propose_plan never called"
|
||||
}
|
||||
if planIdx < runIdx {
|
||||
return true, fmt.Sprintf("propose_plan at index %d before run at index %d", planIdx, runIdx)
|
||||
}
|
||||
return false, fmt.Sprintf("run at index %d before propose_plan at index %d", runIdx, planIdx)
|
||||
|
||||
case "plan_generations":
|
||||
// P2 iteration: counts distinct `generation` values in
|
||||
// session_plan_steps. 1 = single sub-task, 2 = one follow-up
|
||||
// sub-task, etc. Requires the plan endpoint to return generation
|
||||
// values; the eval fetches /sessions/{id}/plan and passes it via
|
||||
// the transcript's PlanSteps field.
|
||||
want := toInt(a.Value)
|
||||
gens := t.distinctGenerations()
|
||||
if gens == want {
|
||||
return true, fmt.Sprintf("%d plan generation(s)", gens)
|
||||
}
|
||||
return false, fmt.Sprintf("%d plan generation(s), want %d", gens, want)
|
||||
|
||||
case "writes_back":
|
||||
n := countTool(tools, "update_entity_attributes") + countTool(tools, "create_relationship")
|
||||
if n > 0 {
|
||||
return true, fmt.Sprintf("%d writeback call(s)", n)
|
||||
}
|
||||
return false, "no update_entity_attributes or create_relationship calls"
|
||||
|
||||
case "max_tool_calls":
|
||||
max := toInt(a.Value)
|
||||
if t.toolCallCount() <= max {
|
||||
return true, fmt.Sprintf("%d tool calls (<= %d)", t.toolCallCount(), max)
|
||||
}
|
||||
return false, fmt.Sprintf("%d tool calls, want <= %d", t.toolCallCount(), max)
|
||||
|
||||
case "max_run_calls":
|
||||
max := toInt(a.Value)
|
||||
n := countTool(tools, "run")
|
||||
if n <= max {
|
||||
return true, fmt.Sprintf("%d run calls (<= %d)", n, max)
|
||||
}
|
||||
return false, fmt.Sprintf("%d run calls, want <= %d", n, max)
|
||||
|
||||
case "no_run":
|
||||
n := countTool(tools, "run")
|
||||
if n == 0 {
|
||||
return true, "run not called"
|
||||
}
|
||||
return false, fmt.Sprintf("run called %d time(s)", n)
|
||||
|
||||
case "calls_tool":
|
||||
want, _ := a.Value.(string)
|
||||
n := countTool(tools, want)
|
||||
if n > 0 {
|
||||
return true, fmt.Sprintf("%s called %d time(s)", want, n)
|
||||
}
|
||||
return false, fmt.Sprintf("%s not called", want)
|
||||
|
||||
case "no_duplicate_complete":
|
||||
n := countTool(tools, "complete_task")
|
||||
if n <= 1 {
|
||||
return true, fmt.Sprintf("complete_task called %d time(s)", n)
|
||||
}
|
||||
return false, fmt.Sprintf("complete_task called %d time(s), want <= 1", n)
|
||||
|
||||
default:
|
||||
return false, fmt.Sprintf("unknown assertion kind: %s", a.Kind)
|
||||
}
|
||||
}
|
||||
|
||||
func countTool(names []string, name string) int {
|
||||
n := 0
|
||||
for _, x := range names {
|
||||
if x == name {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
func toInt(v any) int {
|
||||
switch x := v.(type) {
|
||||
case int:
|
||||
return x
|
||||
case int64:
|
||||
return int(x)
|
||||
case float64:
|
||||
return int(x)
|
||||
}
|
||||
return 0
|
||||
}
|
||||
52
cmd/nomos/eval/sse.go
Normal file
52
cmd/nomos/eval/sse.go
Normal file
@@ -0,0 +1,52 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// sseReader parses a text/event-stream into a sequence of JSON events.
|
||||
// Each event is one or more "data: " lines; the lines are concatenated
|
||||
// and parsed as a single JSON object. Blank lines separate events.
|
||||
type sseReader struct {
|
||||
r *bufio.Reader
|
||||
}
|
||||
|
||||
func newSSEReader(r io.Reader) *sseReader {
|
||||
return &sseReader{r: bufio.NewReader(r)}
|
||||
}
|
||||
|
||||
func (s *sseReader) next() (map[string]any, error) {
|
||||
var data strings.Builder
|
||||
for {
|
||||
line, err := s.r.ReadString('\n')
|
||||
if err != nil {
|
||||
if err == io.EOF && data.Len() > 0 {
|
||||
return parseEvent(data.String())
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
line = strings.TrimRight(line, "\r\n")
|
||||
if line == "" {
|
||||
if data.Len() > 0 {
|
||||
return parseEvent(data.String())
|
||||
}
|
||||
continue // blank line, no event buffered yet
|
||||
}
|
||||
if strings.HasPrefix(line, "data: ") {
|
||||
data.WriteString(strings.TrimPrefix(line, "data: "))
|
||||
} else if strings.HasPrefix(line, "data:") {
|
||||
data.WriteString(strings.TrimPrefix(line, "data:"))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func parseEvent(s string) (map[string]any, error) {
|
||||
var ev map[string]any
|
||||
if err := json.Unmarshal([]byte(s), &ev); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ev, nil
|
||||
}
|
||||
@@ -18,6 +18,7 @@ import (
|
||||
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -102,6 +103,21 @@ func main() {
|
||||
}
|
||||
})
|
||||
|
||||
// Stale execution sweep: cancels non-terminal executions older than
|
||||
// 10 minutes (orphaned by MCP timeouts — see cleanupStaleExecutions).
|
||||
safego.Go("nomos:stale-execution-sweeper", func() {
|
||||
ticker := time.NewTicker(5 * time.Minute)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
st.cleanupStaleExecutions(ctx, 10*time.Minute)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(200)
|
||||
@@ -164,11 +180,30 @@ func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
http.Error(w, "bad request: "+err.Error(), 400)
|
||||
return
|
||||
}
|
||||
if req.Message == "" {
|
||||
if req.Message == "" && req.SessionID == "" {
|
||||
http.Error(w, "message is required", 400)
|
||||
return
|
||||
}
|
||||
|
||||
// Empty message with an existing session = reconnect/resume. The
|
||||
// frontend sends this after a dropped SSE stream to re-establish the
|
||||
// connection and catch up on any auto-continuation work that happened
|
||||
// while disconnected. Route into resumeSession so the agent sees a
|
||||
// system note and reports current state.
|
||||
if req.Message == "" && req.SessionID != "" {
|
||||
slog.Info("nomos: reconnect", "session", req.SessionID)
|
||||
safego.Go("nomos:reconnect:"+req.SessionID, func() {
|
||||
base := "[System: the operator's connection was re-established. The task may have progressed in the background.]"
|
||||
note := st.enrichResumeNote(context.Background(), req.SessionID, base)
|
||||
a.resumeSession(context.Background(), req.SessionID, note)
|
||||
})
|
||||
// Return 202 so the frontend doesn't try to consume an SSE stream
|
||||
// from this POST — resumeSession writes to the DB directly and
|
||||
// the poller (already running from handleDisconnect) picks it up.
|
||||
w.WriteHeader(202)
|
||||
return
|
||||
}
|
||||
|
||||
flusher, ok := w.(http.Flusher)
|
||||
if !ok {
|
||||
http.Error(w, "streaming not supported", 500)
|
||||
@@ -205,6 +240,16 @@ func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
sessionID = sess.ID
|
||||
}
|
||||
} else {
|
||||
// P2 iteration: if the operator sends a follow-up on a session
|
||||
// that already reached a terminal state (done/failed), reopen it
|
||||
// so a new sub-task can be framed (set_goal → propose_plan →
|
||||
// execute). reopenSession marks the prior plan's steps as
|
||||
// `replaced` (proposePlan ignores those) and clears outcome/
|
||||
// summary. Without this, propose_plan refuses the follow-up with
|
||||
// errPlanInFlight because the prior steps are all `done`. If the
|
||||
// session is still active, reopen is a no-op — the follow-up is
|
||||
// just a continuation of in-flight work.
|
||||
st.reopenSession(pctx, sessionID)
|
||||
st.touchSession(pctx, sessionID)
|
||||
}
|
||||
|
||||
@@ -224,6 +269,13 @@ func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
sseEvent(w, flusher, agentEvent{Type: "session", Data: sessionID, SessionID: sessionID})
|
||||
|
||||
toolCalls := []map[string]any{}
|
||||
// P3: accumulate per-iteration reasoning instead of overwriting with
|
||||
// the final `text` event. The agent loop emits a `text` event for each
|
||||
// LLM iteration that produced text (intermediate reasoning before tool
|
||||
// calls + the final answer). Without accumulation, only the last `text`
|
||||
// survives in the persisted row — a reload shows the final summary but
|
||||
// not the thinking that led to each tool call.
|
||||
var textParts []string
|
||||
var finalText string
|
||||
|
||||
// Incremental persistence, mirroring resumeSession's existing
|
||||
@@ -251,17 +303,52 @@ func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
m["type"] = ev.Type
|
||||
toolCalls = append(toolCalls, m)
|
||||
// One entry per tool call: tool_use creates it, tool_result
|
||||
// merges the result into the same entry (matched by id).
|
||||
// Before this fix, both events appended separate entries,
|
||||
// doubling every tool call in the persisted transcript
|
||||
// (confirmed pre-existing in d9cdcee1, v0.3.x era).
|
||||
id, _ := m["id"].(string)
|
||||
if id != "" && ev.Type == "tool_result" {
|
||||
for _, existing := range toolCalls {
|
||||
if eID, _ := existing["id"].(string); eID == id {
|
||||
for k, v := range m {
|
||||
existing[k] = v
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
} else {
|
||||
toolCalls = append(toolCalls, m)
|
||||
}
|
||||
}
|
||||
persist() // live: survives even if the client disconnects right after
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
finalText, _ = ev.Data.(string)
|
||||
// P3: accumulate. Each `text` event is one iteration's reasoning
|
||||
// (or the final answer). Join with newlines so the persisted row
|
||||
// reads as the full transcript of what the agent said, not just
|
||||
// the last thing.
|
||||
if t, ok := ev.Data.(string); ok && t != "" {
|
||||
textParts = append(textParts, t)
|
||||
finalText = strings.Join(textParts, "\n\n")
|
||||
persist()
|
||||
}
|
||||
}
|
||||
sseEvent(w, flusher, ev)
|
||||
})
|
||||
|
||||
persist() // final state — same row, updated one last time with the concluding text
|
||||
// B.6: if the turn ended with no text and no tool calls (the model
|
||||
// empty-response'd and all retries failed), delete the placeholder row
|
||||
// instead of persisting an empty bubble. The error event was already
|
||||
// streamed to the frontend via the 'done with error=true' event, so the
|
||||
// operator sees the error inline — an empty assistant bubble in the
|
||||
// transcript adds nothing and looks like the agent is broken.
|
||||
if finalText == "" && len(toolCalls) == 0 && msgID != uuid.Nil {
|
||||
st.deleteMessage(pctx, msgID)
|
||||
} else {
|
||||
persist() // final state — same row, updated one last time with the concluding text
|
||||
}
|
||||
|
||||
// Generate a meaningful title from the assistant's first answer
|
||||
// instead of reusing the raw user message for every session.
|
||||
@@ -318,6 +405,15 @@ func handleSessionDetail(w http.ResponseWriter, r *http.Request, st *store, a *a
|
||||
return
|
||||
}
|
||||
|
||||
// POST /sessions/{id}/resume — the operator asks the agent to continue.
|
||||
if len(parts) == 2 && parts[1] == "resume" && r.Method == http.MethodPost {
|
||||
base := "[System: the operator wants you to continue. Pick up where you left off — execute the next step of the plan, diagnose and fix any failures, or report progress if everything is done.]"
|
||||
note := st.enrichResumeNote(context.Background(), id, base)
|
||||
safego.Go("nomos:resume-session", func() { a.resumeSession(context.Background(), id, note) })
|
||||
w.WriteHeader(202)
|
||||
return
|
||||
}
|
||||
|
||||
// GET /sessions/{id}/plan and /sessions/{id}/questions — REST hydration for
|
||||
// the context panel when it first opens a task; live events carry deltas
|
||||
// from there.
|
||||
@@ -353,6 +449,21 @@ func handleSessionDetail(w http.ResponseWriter, r *http.Request, st *store, a *a
|
||||
w.WriteHeader(204)
|
||||
|
||||
case http.MethodGet:
|
||||
// getMessages alone can't distinguish "session exists but has no
|
||||
// messages yet" from "session id doesn't exist at all" — it's a
|
||||
// plain WHERE session_id=$1 query that returns zero rows either
|
||||
// way. A frontend window opened for a deleted/invalid session
|
||||
// (persisted layout, a stale link) needs to tell those apart, so
|
||||
// check existence explicitly and 404 rather than silently
|
||||
// returning an empty transcript that looks like a fresh task.
|
||||
if _, err := st.getSession(r.Context(), id); err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
http.Error(w, "session not found", 404)
|
||||
return
|
||||
}
|
||||
http.Error(w, err.Error(), 500)
|
||||
return
|
||||
}
|
||||
messages, err := st.getMessages(r.Context(), id)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), 500)
|
||||
@@ -383,8 +494,9 @@ func handleAnswerQuestion(w http.ResponseWriter, r *http.Request, st *store, a *
|
||||
return
|
||||
}
|
||||
if a != nil {
|
||||
note := fmt.Sprintf("[System: the operator answered your question %q with: %q. "+
|
||||
base := fmt.Sprintf("[System: the operator answered your question %q with: %q. "+
|
||||
"Continue the task from here — do not re-ask.]", prompt, req.Answer)
|
||||
note := st.enrichResumeNote(context.Background(), sessionID, base)
|
||||
safego.Go("nomos:resume-session", func() { a.resumeSession(context.Background(), sessionID, note) })
|
||||
}
|
||||
w.WriteHeader(202)
|
||||
|
||||
170
cmd/nomos/retrycap.go
Normal file
170
cmd/nomos/retrycap.go
Normal file
@@ -0,0 +1,170 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// maxRunRetries is the per-turn cap on identical failing `run` tool calls.
|
||||
// After this many failures with the same (target, command) key, the agent
|
||||
// loop refuses to dispatch the call again and instead surfaces a directive
|
||||
// to investigate *why* (ps/strace/lsof) or escalate to the operator.
|
||||
//
|
||||
// Background: session 1e9c7691 (2026-07-18) retried the same
|
||||
// `chown :10000 /mnt/media_local && chmod 2775 …` ~20 times across direct
|
||||
// runs, SSH-hop-via-hubris, wrapping in a shell script, and bare `echo test`
|
||||
// sanity checks. Each retry piled up another zombie process on the target
|
||||
// (knfsd was holding a kernel lock on the exported directory). The agent
|
||||
// only investigated *why* after the operator explicitly asked
|
||||
// "the command just keeps running?" — see
|
||||
// plans/2026-07-18-session-review-three-sessions.md P0.1.
|
||||
const maxRunRetries = 3
|
||||
|
||||
// runRetryTracker deduplicates failing `run` calls within a single chat
|
||||
// turn (chatWith invocation). It is NOT persisted across turns — the cap
|
||||
// is per-turn, so a fresh turn after the operator responds can retry once
|
||||
// more. The intent is to break a tight retry loop within one turn, not to
|
||||
// permanently block the agent from ever attempting the operation again.
|
||||
//
|
||||
// Threading: the agent loop is single-goroutine per turn, but the tracker
|
||||
// is guarded by a mutex so future callers (e.g. concurrent tool dispatch)
|
||||
// stay safe. The mutex is uncontended on the current hot path.
|
||||
type runRetryTracker struct {
|
||||
mu sync.Mutex
|
||||
counts map[string]int
|
||||
}
|
||||
|
||||
func newRunRetryTracker() *runRetryTracker {
|
||||
return &runRetryTracker{counts: make(map[string]int)}
|
||||
}
|
||||
|
||||
// runFailureKey is the dedup key for "this is the same command against the
|
||||
// same target." Whitespace is collapsed so trivial reformatting
|
||||
// (newlines vs spaces, trailing whitespace) doesn't escape the cap. The
|
||||
// purpose field is intentionally NOT part of the key: the agent often
|
||||
// rephrases purpose between retries while issuing the same command.
|
||||
func runFailureKey(target, command string) string {
|
||||
collapsed := strings.Join(strings.Fields(command), " ")
|
||||
target = strings.TrimSpace(target)
|
||||
h := sha256.Sum256([]byte(target + "\x00" + collapsed))
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
|
||||
// recordFailure increments the failure count for the given key and returns
|
||||
// the new count. The caller should check `count > maxRunRetries` BEFORE
|
||||
// dispatching to decide whether to skip the call.
|
||||
func (r *runRetryTracker) recordFailure(key string) int {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.counts[key]++
|
||||
return r.counts[key]
|
||||
}
|
||||
|
||||
// failures returns the current failure count for a key (0 if unseen).
|
||||
func (r *runRetryTracker) failures(key string) int {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
return r.counts[key]
|
||||
}
|
||||
|
||||
// isRunFailure reports whether a `run` tool call's outcome should count
|
||||
// as a failure for retry-cap purposes. A call counts as failed when:
|
||||
// - the dispatch itself errored (callErr != nil), OR
|
||||
// - the result text starts with "run on <target>: ERROR" — the
|
||||
// shape classifyAndGate/sshExec produce when SSH or the command fails.
|
||||
//
|
||||
// Approvals queued ("requires approval") do NOT count as failures: they
|
||||
// are pending operator action, not a command execution failure. A read
|
||||
// of the existing code paths (classifyAndGate in internal/mcp/server.go)
|
||||
// confirms the "ERROR" prefix is the stable failure signature for `run`.
|
||||
//
|
||||
// The resultText parameter is the MCP tool's RAW text result (not JSON-
|
||||
// re-encoded): when classifyAndGate returns a textResult like
|
||||
// "run on host:strong: ERROR ...", the MCP client unwraps it back to a
|
||||
// plain Go string (see mcpClient.callTool). The caller should pass that
|
||||
// raw string, not json.Marshal's output (which would quote-wrap it).
|
||||
func isRunFailure(toolName string, resultText string, callErr error) bool {
|
||||
if callErr != nil {
|
||||
return true
|
||||
}
|
||||
if toolName != "run" {
|
||||
return false
|
||||
}
|
||||
// "run on host:strong: ERROR ..." or "run on lxc:caddy: ERROR ..."
|
||||
// Both shapes start with "run on ".
|
||||
if !strings.HasPrefix(resultText, "run on ") {
|
||||
return false
|
||||
}
|
||||
return strings.Contains(resultText, ": ERROR")
|
||||
}
|
||||
|
||||
// runResultText extracts the raw text from a `run` tool's result value as
|
||||
// returned by mcpClient.callTool — typically a Go string, but may also be
|
||||
// a []string (multi-content result) or other JSON-decoded shape. Returns
|
||||
// "" for shapes we don't recognize. Used by the retry-cap path so
|
||||
// isRunFailure receives the un-quoted text form (see its doc comment).
|
||||
func runResultText(result any) string {
|
||||
switch v := result.(type) {
|
||||
case string:
|
||||
return v
|
||||
case []string:
|
||||
if len(v) > 0 {
|
||||
return v[0]
|
||||
}
|
||||
case []any:
|
||||
var b strings.Builder
|
||||
for _, e := range v {
|
||||
if s, ok := e.(string); ok {
|
||||
b.WriteString(s)
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// runRetryDirective is the synthetic tool result returned to the model
|
||||
// when the retry cap is hit, in place of dispatching the call again. It
|
||||
// directs the agent to investigate *why* the command keeps failing before
|
||||
// retrying, or to surface the blocker to the operator.
|
||||
func runRetryDirective(target, command string, failures int) string {
|
||||
return "Refused: this `run` against " + target + " has failed " +
|
||||
itoa(failures) + " times this turn — retry cap hit. The command:\n " +
|
||||
command + "\nis almost certainly blocked by something on the target " +
|
||||
"(a hung process, a kernel lock, an unexported FS, a stuck SSH " +
|
||||
"session, …) — NOT a transient gateway issue. Do NOT retry with " +
|
||||
"different routing or quoting. Instead, BEFORE calling `run` again, " +
|
||||
"investigate *why* the command hangs: e.g. `ps aux | grep <cmd>`, " +
|
||||
"`lsof <path>`, `strace -f -p <pid>` or `strace -f <cmd>`, " +
|
||||
"`mount | grep <path>`, `dmesg | tail`. If you find a structural " +
|
||||
"blocker (e.g. a kernel lock on an exported NFS directory → " +
|
||||
"unexport → mutate → re-export), say so to the operator and fix it " +
|
||||
"with a different command. If you genuinely cannot diagnose, " +
|
||||
"surface the blocker to the operator with what you've tried — do " +
|
||||
"not just retry the same command."
|
||||
}
|
||||
|
||||
// itoa is a tiny strconv.Itoa to keep this file dependency-free.
|
||||
func itoa(n int) string {
|
||||
if n == 0 {
|
||||
return "0"
|
||||
}
|
||||
neg := n < 0
|
||||
if neg {
|
||||
n = -n
|
||||
}
|
||||
var buf [20]byte
|
||||
i := len(buf)
|
||||
for n > 0 {
|
||||
i--
|
||||
buf[i] = byte('0' + n%10)
|
||||
n /= 10
|
||||
}
|
||||
if neg {
|
||||
i--
|
||||
buf[i] = '-'
|
||||
}
|
||||
return string(buf[i:])
|
||||
}
|
||||
129
cmd/nomos/retrycap_test.go
Normal file
129
cmd/nomos/retrycap_test.go
Normal file
@@ -0,0 +1,129 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRunFailureKey_StableAcrossWhitespace(t *testing.T) {
|
||||
cases := []struct{ a, b string }{
|
||||
{"chown :10000 /mnt/media_local && chmod 2775 /mnt/media_local",
|
||||
"chown :10000 /mnt/media_local && chmod 2775 /mnt/media_local"},
|
||||
{"chown :10000 /mnt/media_local\n&& chmod 2775 /mnt/media_local",
|
||||
"chown :10000 /mnt/media_local && chmod 2775 /mnt/media_local"},
|
||||
{"chown :10000 /mnt/media_local && chmod 2775 /mnt/media_local ",
|
||||
" chown :10000 /mnt/media_local && chmod 2775 /mnt/media_local"},
|
||||
}
|
||||
for i, c := range cases {
|
||||
ka := runFailureKey("host:strong", c.a)
|
||||
kb := runFailureKey("host:strong", c.b)
|
||||
if ka != kb {
|
||||
t.Errorf("case %d: keys differ for whitespace-equivalent commands:\n a=%q\n b=%q", i, c.a, c.b)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunFailureKey_DiffersByTarget(t *testing.T) {
|
||||
a := runFailureKey("host:strong", "echo hi")
|
||||
b := runFailureKey("host:hubris", "echo hi")
|
||||
if a == b {
|
||||
t.Error("keys should differ when target differs")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunFailureKey_DiffersByCommand(t *testing.T) {
|
||||
a := runFailureKey("host:strong", "echo hi")
|
||||
b := runFailureKey("host:strong", "echo bye")
|
||||
if a == b {
|
||||
t.Error("keys should differ when command differs")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunRetryTracker_CountsAndCaps(t *testing.T) {
|
||||
r := newRunRetryTracker()
|
||||
key := runFailureKey("host:strong", "chown :10000 /mnt/media_local")
|
||||
for i := 1; i <= maxRunRetries; i++ {
|
||||
if got := r.recordFailure(key); got != i {
|
||||
t.Errorf("recordFailure #%d = %d, want %d", i, got, i)
|
||||
}
|
||||
}
|
||||
// At the cap, failures() should report maxRunRetries, and the next
|
||||
// identical call should be refused by the agent loop (failures() >=
|
||||
// maxRunRetries).
|
||||
if got := r.failures(key); got != maxRunRetries {
|
||||
t.Errorf("failures = %d, want %d", got, maxRunRetries)
|
||||
}
|
||||
if r.failures(key) < maxRunRetries {
|
||||
t.Errorf("cap should be enforced at maxRunRetries=%d", maxRunRetries)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunRetryTracker_PerTurnIsolation(t *testing.T) {
|
||||
// Different keys don't interfere.
|
||||
r := newRunRetryTracker()
|
||||
k1 := runFailureKey("host:strong", "echo a")
|
||||
k2 := runFailureKey("host:strong", "echo b")
|
||||
r.recordFailure(k1)
|
||||
r.recordFailure(k1)
|
||||
if got := r.failures(k2); got != 0 {
|
||||
t.Errorf("k2 failures = %d, want 0 (keys are isolated)", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsRunFailure(t *testing.T) {
|
||||
cases := []struct {
|
||||
desc string
|
||||
tool string
|
||||
result string
|
||||
callErr error
|
||||
want bool
|
||||
}{
|
||||
{"run with ERROR prefix", "run", "run on host:strong: ERROR ssh: signal: killed", nil, true},
|
||||
{"run with exit error", "run", "run on lxc:caddy: ERROR exit status 1", nil, true},
|
||||
{"run success (read-only auto)", "run", "run on host:strong (read_only, auto): hello", nil, false},
|
||||
{"run success (assent window)", "run", "run on host:strong (config_mutation, auto via assent window): done", nil, false},
|
||||
{"run queued for approval", "run", "run on host:strong requires approval (risk: config_mutation) — execution 019f4930 queued. Present the command and purpose to the operator and wait; do not re-request.", nil, false},
|
||||
{"non-run tool", "get_entity", "lxc list result", nil, false},
|
||||
{"callErr set (dispatch failure)", "run", "", errFake{}, true},
|
||||
{"callErr set on non-run tool", "get_entity", "some result", errFake{}, true}, // callErr trumps name
|
||||
}
|
||||
for i, c := range cases {
|
||||
got := isRunFailure(c.tool, c.result, c.callErr)
|
||||
if got != c.want {
|
||||
t.Errorf("case %d (%s): isRunFailure = %v, want %v", i, c.desc, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type errFake struct{}
|
||||
|
||||
func (errFake) Error() string { return "fake dispatch error" }
|
||||
|
||||
func TestRunRetryDirective_Content(t *testing.T) {
|
||||
d := runRetryDirective("host:strong", "chown :10000 /mnt/media_local", 3)
|
||||
for _, want := range []string{
|
||||
"Refused:",
|
||||
"host:strong",
|
||||
"3 times",
|
||||
"retry cap hit",
|
||||
"Do NOT retry",
|
||||
"strace",
|
||||
"ps aux",
|
||||
"lsof",
|
||||
"surface the blocker",
|
||||
} {
|
||||
if !strings.Contains(d, want) {
|
||||
t.Errorf("directive missing %q; got:\n%s", want, d)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestItoa(t *testing.T) {
|
||||
cases := map[int]string{0: "0", 1: "1", 9: "9", 10: "10", 42: "42",
|
||||
100: "100", -1: "-1", -42: "-42"}
|
||||
for in, want := range cases {
|
||||
if got := itoa(in); got != want {
|
||||
t.Errorf("itoa(%d) = %q, want %q", in, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,7 @@ package main
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"regexp"
|
||||
@@ -17,6 +18,13 @@ import (
|
||||
|
||||
const maxToolResultSize = 4096
|
||||
|
||||
// errPlanInFlight is returned by proposePlan when called again after a step
|
||||
// has already started. The agent must advance the existing plan with
|
||||
// update_plan_step + run instead of re-proposing — re-proposing was the
|
||||
// source of duplicate plans in the sidebar (operator-reported 2026-07-14).
|
||||
// The caller translates this into a directive tool result.
|
||||
var errPlanInFlight = errors.New("plan already in flight")
|
||||
|
||||
type store struct {
|
||||
pool *pgxpool.Pool
|
||||
}
|
||||
@@ -33,7 +41,40 @@ func newStore(ctx context.Context, databaseURL string) (*store, error) {
|
||||
pool.Close()
|
||||
return nil, fmt.Errorf("ping db: %w", err)
|
||||
}
|
||||
return &store{pool: pool}, nil
|
||||
s := &store{pool: pool}
|
||||
s.cleanupStaleExecutions(ctx, time.Hour)
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// cleanupStaleExecutions marks non-terminal executions older than maxAge as
|
||||
// cancelled. Orphaned executions accumulate when the MCP client times out
|
||||
// (30s) before the run handler's error path can mark them failed — the
|
||||
// execution entity is created before the SSH call, and a timeout kills the
|
||||
// connection before the handler runs its UPDATE. Without this, stale
|
||||
// `running` and `pending_approval` executions pile up in the DB and pollute
|
||||
// the Operations page + session rail badges. Called at startup (maxAge=1h)
|
||||
// and periodically (maxAge=10m) by the sweep worker.
|
||||
func (s *store) cleanupStaleExecutions(ctx context.Context, maxAge time.Duration) int {
|
||||
if s == nil {
|
||||
return 0
|
||||
}
|
||||
tag, err := s.pool.Exec(ctx, `
|
||||
UPDATE executions SET status = 'cancelled',
|
||||
result = jsonb_build_object('message', 'cleaned up — stale non-terminal execution (older than ' || $1 || ')')
|
||||
WHERE status IN ('running', 'pending_approval', 'approved', 'queued')
|
||||
AND entity_id IN (
|
||||
SELECT entity_id FROM entities WHERE created_at < now() - ($2 * interval '1 second')
|
||||
)`,
|
||||
maxAge.String(), maxAge.Seconds())
|
||||
if err != nil {
|
||||
slog.Warn("nomos: stale execution cleanup failed", "error", err)
|
||||
return 0
|
||||
}
|
||||
n := int(tag.RowsAffected())
|
||||
if n > 0 {
|
||||
slog.Info("nomos: cleaned up stale executions", "count", n, "max_age", maxAge.String())
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
func (s *store) close() {
|
||||
@@ -46,16 +87,17 @@ func (s *store) close() {
|
||||
// lifecycle status and an outcome (see migration 018 / the task-board plan).
|
||||
// Outcome/Summary/EntityID are empty until set, hence omitempty.
|
||||
type session struct {
|
||||
ID string `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Actor string `json:"actor"`
|
||||
Goal string `json:"goal"`
|
||||
Status string `json:"status"`
|
||||
Outcome string `json:"outcome,omitempty"`
|
||||
Summary string `json:"summary,omitempty"`
|
||||
EntityID string `json:"entity_id,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
LastActiveAt time.Time `json:"last_active_at"`
|
||||
ID string `json:"id"`
|
||||
Title string `json:"title"`
|
||||
Actor string `json:"actor"`
|
||||
Goal string `json:"goal"`
|
||||
Status string `json:"status"`
|
||||
Outcome string `json:"outcome,omitempty"`
|
||||
Summary string `json:"summary,omitempty"`
|
||||
EntityID string `json:"entity_id,omitempty"`
|
||||
PendingApprovals int `json:"pending_approvals"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
LastActiveAt time.Time `json:"last_active_at"`
|
||||
}
|
||||
|
||||
type message struct {
|
||||
@@ -149,6 +191,88 @@ func (s *store) updateMessage(ctx context.Context, id uuid.UUID, content json.Ra
|
||||
return err
|
||||
}
|
||||
|
||||
// deleteMessage removes a message row. Used by B.6: when a chat turn ends
|
||||
// with no text and no tool calls (the model empty-response'd and all
|
||||
// retries failed), the placeholder row is deleted instead of persisting an
|
||||
// empty assistant bubble — the error was already streamed to the frontend
|
||||
// via the 'done with error=true' event, so the operator sees it inline.
|
||||
func (s *store) deleteMessage(ctx context.Context, id uuid.UUID) {
|
||||
if s == nil || id == uuid.Nil {
|
||||
return
|
||||
}
|
||||
s.pool.Exec(ctx, `DELETE FROM agent_messages WHERE id = $1`, id)
|
||||
}
|
||||
|
||||
// lastUserMessage returns the most recent user message text for a session,
|
||||
// or "" if none. Used to build a context-rich reconnect/resume note: instead
|
||||
// of a generic "report your state," the note can say "the operator's last
|
||||
// message was X — advance the plan" so the agent doesn't re-propose or
|
||||
// re-execute on a reconnect (the operator-reported 2026-07-14 divergence).
|
||||
func (s *store) lastUserMessage(ctx context.Context, sessionID string) string {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return ""
|
||||
}
|
||||
var content json.RawMessage
|
||||
if err := s.pool.QueryRow(ctx,
|
||||
`SELECT content FROM agent_messages
|
||||
WHERE session_id = $1 AND role = 'user'
|
||||
ORDER BY created_at DESC LIMIT 1`, sessionID).Scan(&content); err != nil {
|
||||
return ""
|
||||
}
|
||||
var m struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
if err := json.Unmarshal(content, &m); err != nil {
|
||||
return ""
|
||||
}
|
||||
return m.Text
|
||||
}
|
||||
|
||||
// hasPlanInFlight reports whether a session has a plan with at least one
|
||||
// step in a non-terminal state (pending/running). Used to direct the
|
||||
// reconnect/resume note: if a plan is in flight, the note says "advance
|
||||
// the plan with update_plan_step + run" instead of the generic "report
|
||||
// your state" (which caused the agent to re-propose and duplicate the plan
|
||||
// in the sidebar — operator-reported 2026-07-14).
|
||||
func (s *store) hasPlanInFlight(ctx context.Context, sessionID string) bool {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return false
|
||||
}
|
||||
var exists bool
|
||||
if err := s.pool.QueryRow(ctx,
|
||||
`SELECT EXISTS(SELECT 1 FROM session_plan_steps
|
||||
WHERE session_id = $1 AND status IN ('pending', 'running'))`, sessionID).Scan(&exists); err != nil {
|
||||
return false
|
||||
}
|
||||
return exists
|
||||
}
|
||||
|
||||
// enrichResumeNote appends session context to a base resume/reconnect note:
|
||||
// the operator's last user message and, if a plan is in flight, an explicit
|
||||
// directive to advance it with update_plan_step + run (not re-propose). The
|
||||
// generic "report your state" note caused the agent to re-propose and
|
||||
// duplicate the plan on a reconnect (operator-reported 2026-07-14); this
|
||||
// enrichment gives the agent enough context to do the right thing even
|
||||
// through the reconnect path.
|
||||
func (s *store) enrichResumeNote(ctx context.Context, sessionID, base string) string {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return base
|
||||
}
|
||||
last := s.lastUserMessage(ctx, sessionID)
|
||||
inFlight := s.hasPlanInFlight(ctx, sessionID)
|
||||
if last == "" && !inFlight {
|
||||
return base
|
||||
}
|
||||
note := base
|
||||
if last != "" {
|
||||
note += fmt.Sprintf(" The operator's last message was: %q.", last)
|
||||
}
|
||||
if inFlight {
|
||||
note += " A plan is in flight — advance it with update_plan_step (status=running) + run for the next step's target. Do NOT call propose_plan again."
|
||||
}
|
||||
return note
|
||||
}
|
||||
|
||||
func truncateToolResults(content json.RawMessage) json.RawMessage {
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal(content, &m); err != nil {
|
||||
@@ -195,9 +319,19 @@ func (s *store) listSessions(ctx context.Context) ([]session, error) {
|
||||
return nil, nil
|
||||
}
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT id, title, actor, goal, status, COALESCE(outcome, ''), summary,
|
||||
COALESCE(entity_id::text, ''), created_at, last_active_at
|
||||
FROM agent_sessions ORDER BY last_active_at DESC LIMIT 50`)
|
||||
`SELECT s.id, s.title, s.actor, s.goal, s.status, COALESCE(s.outcome, ''), s.summary,
|
||||
COALESCE(s.entity_id::text, ''),
|
||||
COALESCE(pa.cnt, 0),
|
||||
s.created_at, s.last_active_at
|
||||
FROM agent_sessions s
|
||||
LEFT JOIN (
|
||||
SELECT l.session_id, COUNT(*) AS cnt
|
||||
FROM nomos_plan_executions l
|
||||
JOIN executions e ON e.entity_id = l.execution_id
|
||||
WHERE e.status = 'pending_approval'
|
||||
GROUP BY l.session_id
|
||||
) pa ON pa.session_id = s.id
|
||||
ORDER BY s.last_active_at DESC LIMIT 50`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -207,7 +341,8 @@ func (s *store) listSessions(ctx context.Context) ([]session, error) {
|
||||
for rows.Next() {
|
||||
var sess session
|
||||
if err := rows.Scan(&sess.ID, &sess.Title, &sess.Actor, &sess.Goal, &sess.Status,
|
||||
&sess.Outcome, &sess.Summary, &sess.EntityID, &sess.CreatedAt, &sess.LastActiveAt); err != nil {
|
||||
&sess.Outcome, &sess.Summary, &sess.EntityID, &sess.PendingApprovals,
|
||||
&sess.CreatedAt, &sess.LastActiveAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, sess)
|
||||
@@ -215,6 +350,24 @@ func (s *store) listSessions(ctx context.Context) ([]session, error) {
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (s *store) getSession(ctx context.Context, id string) (*session, error) {
|
||||
if s == nil {
|
||||
return nil, nil
|
||||
}
|
||||
var sess session
|
||||
err := s.pool.QueryRow(ctx,
|
||||
`SELECT id, title, actor, goal, status, COALESCE(outcome, ''), summary,
|
||||
COALESCE(entity_id::text, ''), 0, created_at, last_active_at
|
||||
FROM agent_sessions WHERE id = $1`, id).
|
||||
Scan(&sess.ID, &sess.Title, &sess.Actor, &sess.Goal, &sess.Status,
|
||||
&sess.Outcome, &sess.Summary, &sess.EntityID, &sess.PendingApprovals,
|
||||
&sess.CreatedAt, &sess.LastActiveAt)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &sess, nil
|
||||
}
|
||||
|
||||
// getMessages returns a session's ENTIRE message history, unbounded — used
|
||||
// for the UI's own transcript view (GET /sessions/{id}), where the operator
|
||||
// should be able to see everything a task has done regardless of how long
|
||||
@@ -328,13 +481,54 @@ func (s *store) taskEntityPtr(ctx context.Context, sessionID string) *uuid.UUID
|
||||
return &id
|
||||
}
|
||||
|
||||
// setGoal records the task's goal and moves it into planning. Emits goal.set.
|
||||
// setGoal records the task's goal and moves it into executing. The `planning`
|
||||
// intermediate state was removed (2026-07-14) — it was indistinguishable from
|
||||
// `active` to the operator and caused sessions to appear stuck when the agent
|
||||
// called set_goal but never propose_plan (observed in production).
|
||||
//
|
||||
// P2 (2026-07-15): setGoal also replaces any prior plan steps (from a
|
||||
// previous sub-task or an incomplete first turn) as `replaced`, clearing the
|
||||
// way for a fresh propose_plan. This is the ONLY place step replacement
|
||||
// happens — not in reopenSession — because set_goal is the explicit signal
|
||||
// for "new sub-task." An approval ("go ahead") does NOT call set_goal, so it
|
||||
// won't destroy the plan the operator just approved.
|
||||
//
|
||||
// P1.4 (2026-07-18): when a non-empty prior goal is being overwritten by a
|
||||
// different goal, emit a `task.superseded` event carrying the prior goal.
|
||||
// This gives the UI/audit trail a clear signal that the operator pivoted —
|
||||
// without it, the prior goal just silently disappears from
|
||||
// agent_sessions.goal and there's no record the session ever had a
|
||||
// different starting intent. See plans/2026-07-18-session-review-three-
|
||||
// sessions.md P1.4 (session 55927f0a had two set_goal calls with the first
|
||||
// implicitly abandoned when the operator said "lets just keep ludo-library
|
||||
// then").
|
||||
func (s *store) setGoal(ctx context.Context, sessionID, goal string) error {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return nil
|
||||
}
|
||||
// Capture the prior goal BEFORE the UPDATE overwrites it. If non-empty
|
||||
// and different from the new goal, emit task.superseded so the audit
|
||||
// trail records the pivot — the row's goal column won't.
|
||||
var priorGoal string
|
||||
s.pool.QueryRow(ctx,
|
||||
`SELECT COALESCE(goal, '') FROM agent_sessions WHERE id = $1`,
|
||||
sessionID).Scan(&priorGoal)
|
||||
if priorGoal != "" && priorGoal != goal {
|
||||
_ = observability.Event(ctx, sqlcgen.New(s.pool), "task.superseded",
|
||||
s.taskEntityPtr(ctx, sessionID), "info", "nomos", sessionID,
|
||||
map[string]any{"prior_goal": priorGoal, "new_goal": goal})
|
||||
slog.Info("nomos: task goal superseded by a new set_goal",
|
||||
"session", sessionID, "prior_goal", priorGoal, "new_goal", goal)
|
||||
}
|
||||
// Replace any prior plan steps (done/running/pending/...) as `replaced`.
|
||||
// The rows are kept for the generation counter + audit trail; proposePlan
|
||||
// excludes `replaced` from its in-flight check, so the next propose_plan
|
||||
// takes the fresh-generation path.
|
||||
s.pool.Exec(ctx,
|
||||
`UPDATE session_plan_steps SET status = 'replaced', finished_at = COALESCE(finished_at, now()) WHERE session_id = $1 AND status <> 'replaced'`,
|
||||
sessionID)
|
||||
if _, err := s.pool.Exec(ctx,
|
||||
`UPDATE agent_sessions SET goal = $2, status = 'planning', last_active_at = now() WHERE id = $1`,
|
||||
`UPDATE agent_sessions SET goal = $2, status = 'executing', last_active_at = now() WHERE id = $1`,
|
||||
sessionID, goal); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -343,6 +537,40 @@ func (s *store) setGoal(ctx context.Context, sessionID, goal string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// reopenSession flips a terminal (done/failed) session back to `executing`
|
||||
// so a follow-up message can start a new sub-task — the iteration path
|
||||
// (P2, 2026-07-15). Without this, a completed session stays `done` forever
|
||||
// and the panel shows a stale result.
|
||||
//
|
||||
// reopenSession ONLY flips the status + clears outcome/summary. It does NOT
|
||||
// touch plan steps — that's `setGoal`'s job (see below). The reason: not
|
||||
// every follow-up is a new sub-task. An approval ("go ahead") is a
|
||||
// continuation of the current plan, and replacing its steps would destroy
|
||||
// the plan the operator just approved. `set_goal` is the explicit signal for
|
||||
// "new sub-task," so step replacement happens there, not here.
|
||||
//
|
||||
// Returns true if the session was actually reopened (was terminal), false if
|
||||
// it was already active (no-op).
|
||||
func (s *store) reopenSession(ctx context.Context, sessionID string) bool {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return false
|
||||
}
|
||||
var currentStatus string
|
||||
if err := s.pool.QueryRow(ctx,
|
||||
`SELECT status FROM agent_sessions WHERE id = $1`, sessionID).Scan(¤tStatus); err != nil {
|
||||
return false
|
||||
}
|
||||
if currentStatus != "done" && currentStatus != "failed" {
|
||||
return false
|
||||
}
|
||||
s.pool.Exec(ctx,
|
||||
`UPDATE agent_sessions SET status = 'executing', outcome = NULL, summary = NULL, last_active_at = now() WHERE id = $1`,
|
||||
sessionID)
|
||||
_ = observability.Event(ctx, sqlcgen.New(s.pool), "task.reopened", s.taskEntityPtr(ctx, sessionID),
|
||||
"info", "nomos", sessionID, map[string]any{"prior_status": currentStatus})
|
||||
return true
|
||||
}
|
||||
|
||||
// planStepInput is one step as the agent proposes it.
|
||||
type planStepInput struct {
|
||||
Title string
|
||||
@@ -357,14 +585,14 @@ type planStepInput struct {
|
||||
// Two modes, chosen by whether any existing step has left 'pending':
|
||||
// - Fresh/revise (no step started yet): full replace (delete + insert). This
|
||||
// covers the first call, and a genuine re-plan before any work began.
|
||||
// - Mid-flight (some step is running/done/failed/…): APPEND the new steps
|
||||
// after the current max seq instead of wiping. The model is instructed to
|
||||
// propose the whole plan in one call, but nothing stops it from calling
|
||||
// propose_plan again per-step as it goes — a destructive replace in that
|
||||
// case would erase every already-completed step, leaving the operator
|
||||
// seeing only the most recent single step ("1/1") instead of real
|
||||
// progress. Appending makes the panel's step history correct regardless
|
||||
// of how the model chooses to call the tool.
|
||||
// - Mid-flight (some step is running/done/failed/…): REFUSE the call.
|
||||
// The agent must advance the existing plan with update_plan_step + run
|
||||
// instead of re-proposing. The previous append-mode safety net (commit
|
||||
// 5384499) preserved history but produced a confusing duplicate sidebar
|
||||
// when the agent re-proposed on "proceed" (operator-reported 2026-07-14).
|
||||
// Refusing is the correct default — the tool result tells the agent how
|
||||
// to advance, and the generation column tracks revisions if a genuine
|
||||
// re-plan is ever allowed.
|
||||
func (s *store) proposePlan(ctx context.Context, sessionID string, steps []planStepInput) ([]map[string]any, error) {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return nil, nil
|
||||
@@ -377,17 +605,52 @@ func (s *store) proposePlan(ctx context.Context, sessionID string, steps []planS
|
||||
|
||||
var startSeq int
|
||||
var anyStarted bool
|
||||
// `replaced` steps (from a prior plan generation superseded by a
|
||||
// follow-up sub-task — see reopenSession) are excluded: they prove a
|
||||
// prior plan was completed and superseded, not that a plan is in flight.
|
||||
// Without this exclusion, reopenSession's `replaced` marking would be
|
||||
// useless — propose_plan would still refuse on the follow-up.
|
||||
if err := tx.QueryRow(ctx, `
|
||||
SELECT COALESCE(max(seq), 0), COALESCE(bool_or(status <> 'pending'), false)
|
||||
SELECT COALESCE(max(seq), 0), COALESCE(bool_or(status NOT IN ('pending', 'replaced')), false)
|
||||
FROM session_plan_steps WHERE session_id = $1`, sessionID).Scan(&startSeq, &anyStarted); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !anyStarted {
|
||||
if _, err := tx.Exec(ctx, `DELETE FROM session_plan_steps WHERE session_id = $1`, sessionID); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
startSeq = 0
|
||||
if anyStarted {
|
||||
// A plan is already in flight (a step is running/done/failed/...).
|
||||
// Refuse the re-proposal — the agent must advance with
|
||||
// update_plan_step + run. The caller surfaces a directive.
|
||||
return nil, errPlanInFlight
|
||||
}
|
||||
// Fresh/revise: mark any prior PENDING steps as `replaced` (not DELETE).
|
||||
// This preserves the rows for the generation counter (MAX(generation)+1
|
||||
// below) and the plan_generations eval assertion. Without this, a first
|
||||
// plan that was proposed but never executed (all pending) would be
|
||||
// wiped, resetting the counter to 1 — making a follow-up's plan look
|
||||
// like generation 1 instead of 2. `replaced` steps are excluded from
|
||||
// the anyStarted check above, so they don't block the fresh proposal.
|
||||
if _, err := tx.Exec(ctx,
|
||||
`UPDATE session_plan_steps SET status = 'replaced', finished_at = COALESCE(finished_at, now()) WHERE session_id = $1 AND status = 'pending'`,
|
||||
sessionID); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// startSeq keeps the max(seq) from the query above: if prior steps
|
||||
// exist (replaced or done), the new generation's steps start after them
|
||||
// (no seq collisions across generations). If no rows exist (first plan),
|
||||
// startSeq is 0 and the first step is seq 1.
|
||||
|
||||
// Resolve the generation number for this plan. Generation 1 is the
|
||||
// initial plan; a genuine revise (which currently goes through the same
|
||||
// fresh-start path above because all steps were pending) resets to 1
|
||||
// since the DELETE wiped the prior rows. The column is wired here so a
|
||||
// future explicit mid-flight revise path can increment it.
|
||||
var nextGen int
|
||||
if err := tx.QueryRow(ctx, `
|
||||
SELECT COALESCE(MAX(generation), 0) + 1
|
||||
FROM session_plan_steps WHERE session_id = $1`, sessionID).Scan(&nextGen); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// After the DELETE above, no rows remain, so MAX(generation) is NULL →
|
||||
// nextGen = 1. (Keep the query for the future revise path; it's cheap.)
|
||||
|
||||
out := make([]map[string]any, 0, len(steps))
|
||||
for i, st := range steps {
|
||||
@@ -398,14 +661,15 @@ func (s *store) proposePlan(ctx context.Context, sessionID string, steps []planS
|
||||
seq := startSeq + i + 1
|
||||
var id uuid.UUID
|
||||
if err := tx.QueryRow(ctx, `
|
||||
INSERT INTO session_plan_steps (session_id, seq, title, detail, target_slug)
|
||||
VALUES ($1, $2, $3, $4, $5) RETURNING id`,
|
||||
sessionID, seq, st.Title, st.Detail, targetSlug).Scan(&id); err != nil {
|
||||
INSERT INTO session_plan_steps (session_id, seq, title, detail, target_slug, generation)
|
||||
VALUES ($1, $2, $3, $4, $5, $6) RETURNING id`,
|
||||
sessionID, seq, st.Title, st.Detail, targetSlug, nextGen).Scan(&id); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, map[string]any{
|
||||
"id": id.String(), "seq": seq, "title": st.Title,
|
||||
"detail": st.Detail, "target_slug": st.TargetSlug,
|
||||
"generation": nextGen,
|
||||
})
|
||||
}
|
||||
if _, err := tx.Exec(ctx,
|
||||
@@ -416,10 +680,10 @@ func (s *store) proposePlan(ctx context.Context, sessionID string, steps []planS
|
||||
return nil, err
|
||||
}
|
||||
// Event after commit so subscribers only ever see a persisted plan.
|
||||
// appended=true tells the panel to add these steps to its existing list
|
||||
// rather than replace it (mirrors the mid-flight append above).
|
||||
// appended=false (always now — we refuse mid-flight re-proposals) tells
|
||||
// the panel to replace its list with these steps.
|
||||
_ = observability.Event(ctx, sqlcgen.New(s.pool), "plan.proposed", s.taskEntityPtr(ctx, sessionID),
|
||||
"info", "nomos", sessionID, map[string]any{"steps": out, "appended": anyStarted})
|
||||
"info", "nomos", sessionID, map[string]any{"steps": out, "appended": false, "generation": nextGen})
|
||||
return out, nil
|
||||
}
|
||||
|
||||
@@ -428,6 +692,11 @@ func (s *store) proposePlan(ctx context.Context, sessionID string, steps []planS
|
||||
// plan.step.finished (terminal) so the panel advances live. The execution link
|
||||
// is also what lets the api auto-close the step when the execution finishes
|
||||
// (see closePlanStepForExecution).
|
||||
//
|
||||
// Completion ordering (done/failed/skipped/blocked) is enforced: a step cannot
|
||||
// be marked complete while an earlier step is still pending, preventing the
|
||||
// agent from marking step 5 done before step 4 (observed in production: the
|
||||
// agent rushed to close all steps in a final turn, in reverse order).
|
||||
func (s *store) updatePlanStep(ctx context.Context, sessionID string, seq int, status, execID string) error {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return nil
|
||||
@@ -436,9 +705,22 @@ func (s *store) updatePlanStep(ctx context.Context, sessionID string, seq int, s
|
||||
switch status {
|
||||
case "running":
|
||||
stamp = ", started_at = COALESCE(started_at, now())"
|
||||
case "done", "failed", "skipped", "blocked":
|
||||
case "done", "failed", "skipped", "blocked", "replaced":
|
||||
stamp = ", finished_at = now()"
|
||||
}
|
||||
// Completion ordering: for terminal states, check that no earlier step
|
||||
// is still pending. Running steps can start out of order (the agent
|
||||
// may dispatch parallel work), but completion must be sequential.
|
||||
if status == "done" || status == "failed" || status == "skipped" || status == "blocked" {
|
||||
var blockedBy int
|
||||
if err := s.pool.QueryRow(ctx, `
|
||||
SELECT COALESCE(MIN(seq), 0)
|
||||
FROM session_plan_steps
|
||||
WHERE session_id = $1 AND seq < $2 AND status = 'pending'`,
|
||||
sessionID, seq).Scan(&blockedBy); err == nil && blockedBy > 0 {
|
||||
return fmt.Errorf("cannot complete step %d — step %d is still pending", seq, blockedBy)
|
||||
}
|
||||
}
|
||||
var execPtr *uuid.UUID
|
||||
if id, err := uuid.Parse(execID); err == nil {
|
||||
execPtr = &id
|
||||
@@ -473,14 +755,64 @@ func (s *store) updatePlanStep(ctx context.Context, sessionID string, seq int, s
|
||||
return nil
|
||||
}
|
||||
|
||||
// errTaskAlreadyComplete is returned by completeTask when the session is
|
||||
// already in a terminal state (done/failed/partial). The agent sometimes
|
||||
// re-calls complete_task after a UI clarification (operator-reported
|
||||
// 2026-07-14) — without this guard, the re-completion produces duplicate
|
||||
// knowledge entries and erodes audit-log clarity. The caller translates this
|
||||
// into a directive tool result.
|
||||
var errTaskAlreadyComplete = errors.New("task already complete")
|
||||
|
||||
// completeTask sets a task's terminal state, outcome, and one-line summary,
|
||||
// mirrors the outcome onto the task entity's attributes (so the board/graph
|
||||
// show it), and publishes task.status for the live context panel. outcome is
|
||||
// success|failure|partial; status is derived (failure → failed, else done).
|
||||
// Returns errTaskAlreadyComplete if the session is already terminal — the
|
||||
// agent must not re-complete a finished task.
|
||||
func (s *store) completeTask(ctx context.Context, sessionID, outcome, summary string) error {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return nil
|
||||
}
|
||||
|
||||
// C.1: reject re-completion of an already-terminal session. The agent
|
||||
// sometimes re-calls complete_task after a UI clarification ("the sidebar
|
||||
// differs") — without this guard, the re-completion duplicates knowledge
|
||||
// entries and produces a confusing audit trail.
|
||||
var currentStatus string
|
||||
if err := s.pool.QueryRow(ctx,
|
||||
`SELECT status FROM agent_sessions WHERE id = $1`, sessionID).Scan(¤tStatus); err != nil {
|
||||
// Session doesn't exist or query failed — let the rest of the
|
||||
// function proceed; it'll fail safely on the UPDATE below.
|
||||
} else if currentStatus == "done" || currentStatus == "failed" {
|
||||
return errTaskAlreadyComplete
|
||||
}
|
||||
|
||||
// Auto-cancel any executions still in pending_approval/approved/queued
|
||||
// state for this session — preventing orphaned approvals (observed in
|
||||
// production: 4 approvals left open after session completed).
|
||||
var cancelledCount int
|
||||
if err := s.pool.QueryRow(ctx, `
|
||||
WITH cancelled AS (
|
||||
UPDATE executions SET status = 'cancelled',
|
||||
result = '{"message": "task completed — auto-cancelled"}'::jsonb
|
||||
WHERE entity_id IN (
|
||||
SELECT execution_id FROM nomos_plan_executions WHERE session_id = $1
|
||||
) AND status IN ('pending_approval', 'approved', 'queued')
|
||||
RETURNING entity_id
|
||||
)
|
||||
SELECT COUNT(*) FROM cancelled
|
||||
`, sessionID).Scan(&cancelledCount); err != nil {
|
||||
slog.Warn("nomos: completeTask failed to cancel orphaned executions", "session", sessionID, "error", err)
|
||||
}
|
||||
|
||||
// Mark all continuations done so the worker won't try to feed them back.
|
||||
s.pool.Exec(ctx, `UPDATE nomos_plan_executions SET continued_at = now()
|
||||
WHERE session_id = $1 AND continued_at IS NULL`, sessionID)
|
||||
|
||||
// Clean up assent and destructive window keys from autonomy_settings.
|
||||
s.pool.Exec(ctx, `DELETE FROM autonomy_settings
|
||||
WHERE key LIKE '%:' || $1`, sessionID)
|
||||
|
||||
status := "done"
|
||||
if outcome == "failure" {
|
||||
status = "failed"
|
||||
@@ -504,10 +836,52 @@ func (s *store) completeTask(ctx context.Context, sessionID, outcome, summary st
|
||||
severity = "warning"
|
||||
}
|
||||
_ = observability.Event(ctx, sqlcgen.New(s.pool), "task.status", entPtr, severity, "nomos", sessionID,
|
||||
map[string]any{"status": status, "outcome": outcome, "summary": summary})
|
||||
map[string]any{"status": status, "outcome": outcome, "summary": summary,
|
||||
"cancelled_executions": cancelledCount})
|
||||
return nil
|
||||
}
|
||||
|
||||
// hadEntityWriteback checks whether this session called update_entity_attributes
|
||||
// or create_relationship — used by complete_task to warn the agent when it
|
||||
// forgot to persist entity facts (the #1 cause of knowledge graph drift).
|
||||
func (s *store) hadEntityWriteback(ctx context.Context, sessionID string) bool {
|
||||
if s == nil || sessionID == "" {
|
||||
return true // fail safe: don't warn when we can't check
|
||||
}
|
||||
var count int
|
||||
s.pool.QueryRow(ctx, `
|
||||
SELECT COUNT(*) FROM agent_activity
|
||||
WHERE session_id = $1
|
||||
AND tool_name IN ('update_entity_attributes', 'create_relationship')
|
||||
AND success = true`, sessionID).Scan(&count)
|
||||
return count > 0
|
||||
}
|
||||
|
||||
// hadDiscovery checks whether this session ran `run` successfully against a
|
||||
// real target — i.e. discovered live state (versions, package counts, host
|
||||
// facts, service status) that the DB didn't have. Used by complete_task to
|
||||
// refuse success when discovery happened but no writeback followed (the
|
||||
// knowledge-loop drift the prior warnings failed to close — the agent
|
||||
// ignored advisory text, so D.1 makes it structural).
|
||||
//
|
||||
// Only `run` counts as discovery here, NOT get_entity/list_lxcs/etc. — those
|
||||
// are DB lookups, not new facts. A trivial Q&A ("status of lxc:dns?") that
|
||||
// only calls get_entity is a degenerate case (SOUL.md: "Don't invent
|
||||
// attributes that don't exist") and must NOT be blocked. Only sessions that
|
||||
// actually executed against a live target get the writeback gate.
|
||||
func (s *store) hadDiscovery(ctx context.Context, sessionID string) bool {
|
||||
if s == nil || sessionID == "" {
|
||||
return false // fail safe: don't block when we can't check
|
||||
}
|
||||
var count int
|
||||
s.pool.QueryRow(ctx, `
|
||||
SELECT COUNT(*) FROM agent_activity
|
||||
WHERE session_id = $1
|
||||
AND tool_name = 'run'
|
||||
AND success = true`, sessionID).Scan(&count)
|
||||
return count > 0
|
||||
}
|
||||
|
||||
// staleGoalSession is a goal-bearing task that's gone idle without reaching
|
||||
// a terminal state — the idle-sweep worker's work list (fix 2+3 of
|
||||
// plans/2026-07-11-task-completion-safety-net.md).
|
||||
@@ -518,7 +892,7 @@ type staleGoalSession struct {
|
||||
}
|
||||
|
||||
// staleGoalSessions finds sessions that framed themselves as a real task
|
||||
// (goal != '', so the inline safety net in agent.go intentionally left them
|
||||
// (goal != ”, so the inline safety net in agent.go intentionally left them
|
||||
// alone) but have sat non-terminal past idleThreshold. completion_nudges
|
||||
// tells the caller whether to nudge (0) or give up and auto-close (>=1) —
|
||||
// see processIdleSweep in continue.go.
|
||||
@@ -562,6 +936,42 @@ func (s *store) bumpCompletionNudge(ctx context.Context, sessionID string) error
|
||||
return err
|
||||
}
|
||||
|
||||
// allPlanStepsTerminal reports whether every plan step for this session is in
|
||||
// a terminal state (done/failed/replaced/skipped/blocked) — i.e. no step is
|
||||
// still pending or running. Used by autoCompleteIfPlanDone to auto-close a
|
||||
// task when the agent did all the work but forgot to call complete_task.
|
||||
// Returns false if there are no plan steps at all (no plan was proposed).
|
||||
func (s *store) allPlanStepsTerminal(ctx context.Context, sessionID string) bool {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return false
|
||||
}
|
||||
var total, terminal int
|
||||
if err := s.pool.QueryRow(ctx,
|
||||
`SELECT COUNT(*), COUNT(*) FILTER (WHERE status IN ('done', 'failed', 'replaced', 'skipped', 'blocked'))
|
||||
FROM session_plan_steps WHERE session_id = $1`,
|
||||
sessionID).Scan(&total, &terminal); err != nil {
|
||||
return false
|
||||
}
|
||||
return total > 0 && total == terminal
|
||||
}
|
||||
|
||||
// hasPendingApprovals reports whether this session has any executions in
|
||||
// pending_approval state. Used by autoCompleteIfPlanDone to avoid closing a
|
||||
// session that's blocked waiting for operator approval — the agent hit the
|
||||
// P5 gate and can't continue until the operator responds.
|
||||
func (s *store) hasPendingApprovals(ctx context.Context, sessionID string) bool {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return false
|
||||
}
|
||||
var count int
|
||||
s.pool.QueryRow(ctx, `
|
||||
SELECT COUNT(*) FROM nomos_plan_executions pe
|
||||
JOIN executions ex ON ex.entity_id = pe.execution_id
|
||||
WHERE pe.session_id = $1 AND ex.status = 'pending_approval'`,
|
||||
sessionID).Scan(&count)
|
||||
return count > 0
|
||||
}
|
||||
|
||||
// planStep is a persisted plan step, as returned to the frontend for hydration
|
||||
// (the panel otherwise only sees steps live via plan.proposed/plan.step.*).
|
||||
type planStep struct {
|
||||
@@ -574,6 +984,7 @@ type planStep struct {
|
||||
TargetSlug *string `json:"target_slug,omitempty"`
|
||||
StartedAt *string `json:"started_at,omitempty"`
|
||||
FinishedAt *string `json:"finished_at,omitempty"`
|
||||
Generation int `json:"generation"`
|
||||
}
|
||||
|
||||
// getPlanSteps returns a task's plan in order — REST hydration for the context
|
||||
@@ -585,7 +996,7 @@ func (s *store) getPlanSteps(ctx context.Context, sessionID string) ([]planStep,
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT id::text, seq, title, detail, status,
|
||||
execution_id::text, target_slug,
|
||||
started_at::text, finished_at::text
|
||||
started_at::text, finished_at::text, generation
|
||||
FROM session_plan_steps WHERE session_id = $1 ORDER BY seq`, sessionID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -596,7 +1007,7 @@ func (s *store) getPlanSteps(ctx context.Context, sessionID string) ([]planStep,
|
||||
var st planStep
|
||||
var execID, target, started, finished *string
|
||||
if err := rows.Scan(&st.ID, &st.Seq, &st.Title, &st.Detail, &st.Status,
|
||||
&execID, &target, &started, &finished); err != nil {
|
||||
&execID, &target, &started, &finished, &st.Generation); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
st.ExecutionID, st.TargetSlug, st.StartedAt, st.FinishedAt = execID, target, started, finished
|
||||
|
||||
@@ -10,6 +10,7 @@ package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math/rand"
|
||||
"os"
|
||||
@@ -17,6 +18,7 @@ import (
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
@@ -159,19 +161,19 @@ func TestGetRecentMessages_Truncation(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestProposePlan_AppendVsReplace is the concrete proof for the plan-append
|
||||
// fix (commit 5384499, "plan panel showed only the latest step, not the full
|
||||
// plan"): proposePlan must REPLACE the step list only while every existing
|
||||
// step is still 'pending' (a genuine pre-execution revision), and APPEND
|
||||
// once any step has started — otherwise a model that calls propose_plan once
|
||||
// per step (rather than once with the full list, as instructed) erases every
|
||||
// already-completed step each time, and the operator only ever sees the
|
||||
// latest single step instead of real progress.
|
||||
func TestProposePlan_AppendVsReplace(t *testing.T) {
|
||||
// TestProposePlan_RefuseInFlight is the concrete proof for the plan-drift
|
||||
// fix (2026-07-14, "plan added twice in the sidebar"): proposePlan must
|
||||
// REPLACE the step list only while every existing step is still 'pending'
|
||||
// (a genuine pre-execution revision), and REFUSE the call once any step has
|
||||
// started. The prior append-mode safety net (commit 5384499) preserved
|
||||
// history but duplicated the plan in the sidebar when the agent re-proposed
|
||||
// on "proceed". Refusing is the correct default — the agent must advance
|
||||
// with update_plan_step + run.
|
||||
func TestProposePlan_RefuseInFlight(t *testing.T) {
|
||||
s := newTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
sess, err := s.createSession(ctx, "plan append test")
|
||||
sess, err := s.createSession(ctx, "plan refuse test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
@@ -185,39 +187,37 @@ func TestProposePlan_AppendVsReplace(t *testing.T) {
|
||||
if len(out1) != 1 || out1[0]["seq"] != 1 {
|
||||
t.Fatalf("proposePlan #1 = %+v, want one step at seq 1", out1)
|
||||
}
|
||||
if out1[0]["generation"] != 1 {
|
||||
t.Fatalf("proposePlan #1 generation = %v, want 1", out1[0]["generation"])
|
||||
}
|
||||
|
||||
// Mark step 1 as started.
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "running", ""); err != nil {
|
||||
t.Fatalf("updatePlanStep: %v", err)
|
||||
}
|
||||
|
||||
// Second call, simulating a model that (against instructions) calls
|
||||
// propose_plan again per-step instead of once with the full list: since
|
||||
// step 1 has left 'pending', this MUST append, not replace.
|
||||
out2, err := s.proposePlan(ctx, sess.ID, []planStepInput{{Title: "Step B"}})
|
||||
if err != nil {
|
||||
t.Fatalf("proposePlan #2: %v", err)
|
||||
}
|
||||
if len(out2) != 1 || out2[0]["seq"] != 2 {
|
||||
t.Fatalf("proposePlan #2 = %+v, want one step at seq 2 (appended after the running step 1)", out2)
|
||||
// Second call, simulating a model that re-proposes mid-flight (the
|
||||
// operator-reported "proceed" bug): since step 1 has left 'pending',
|
||||
// this MUST refuse with errPlanInFlight, not append or replace.
|
||||
_, err = s.proposePlan(ctx, sess.ID, []planStepInput{{Title: "Step B"}})
|
||||
if !errors.Is(err, errPlanInFlight) {
|
||||
t.Fatalf("proposePlan #2: err = %v, want errPlanInFlight (refuse mid-flight re-proposal)", err)
|
||||
}
|
||||
|
||||
// The original step 1 must be untouched — not erased, not appended to.
|
||||
steps, err := s.getPlanSteps(ctx, sess.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("getPlanSteps: %v", err)
|
||||
}
|
||||
if len(steps) != 2 {
|
||||
t.Fatalf("got %d persisted steps, want 2 (step 1 must survive the second propose_plan call)", len(steps))
|
||||
if len(steps) != 1 {
|
||||
t.Fatalf("got %d persisted steps, want 1 (refused call must not mutate the plan)", len(steps))
|
||||
}
|
||||
if steps[0].Title != "Step A" || steps[0].Status != "running" {
|
||||
t.Errorf("step 1 = %+v, want Step A still running (not erased)", steps[0])
|
||||
}
|
||||
if steps[1].Title != "Step B" || steps[1].Status != "pending" {
|
||||
t.Errorf("step 2 = %+v, want Step B pending", steps[1])
|
||||
t.Errorf("step 1 = %+v, want Step A still running (refused call must not touch it)", steps[0])
|
||||
}
|
||||
|
||||
// Third call BEFORE anything runs on a fresh session: every step is
|
||||
// still pending, so this must REPLACE, not append.
|
||||
// still pending, so this must REPLACE, not refuse.
|
||||
sess2, err := s.createSession(ctx, "plan replace test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
@@ -233,6 +233,147 @@ func TestProposePlan_AppendVsReplace(t *testing.T) {
|
||||
t.Fatalf("getPlanSteps: %v", err)
|
||||
}
|
||||
if len(revisedSteps) != 1 || revisedSteps[0].Title != "Revised" {
|
||||
t.Fatalf("got %+v, want a single 'Revised' step (pre-execution revise must replace, not append)", revisedSteps)
|
||||
t.Fatalf("got %+v, want a single 'Revised' step (pre-execution revise must replace, not refuse)", revisedSteps)
|
||||
}
|
||||
if revisedSteps[0].Generation != 1 {
|
||||
t.Fatalf("revised step generation = %d, want 1 (fresh-start after DELETE resets generation)", revisedSteps[0].Generation)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHadDiscoveryAndWriteback is the store-level proof for D.1 (refuse
|
||||
// complete_task when discovery ran without writeback). hadDiscovery must
|
||||
// report true only after a successful `run` call; hadEntityWriteback must
|
||||
// report true only after a successful update_entity_attributes or
|
||||
// create_relationship call. The D.1 gate in tasks.go combines these: refuse
|
||||
// success when hadDiscovery && !hadEntityWriteback.
|
||||
func TestHadDiscoveryAndWriteback(t *testing.T) {
|
||||
s := newTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
sess, err := s.createSession(ctx, "discovery test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
|
||||
// Before any tool calls: no discovery, no writeback.
|
||||
if s.hadDiscovery(ctx, sess.ID) {
|
||||
t.Fatal("hadDiscovery = true before any tool calls, want false")
|
||||
}
|
||||
if s.hadEntityWriteback(ctx, sess.ID) {
|
||||
t.Fatal("hadEntityWriteback = true before any tool calls, want false")
|
||||
}
|
||||
|
||||
// A `run` call (discovery) — should set hadDiscovery, not hadEntityWriteback.
|
||||
agentID := uuid.New()
|
||||
s.logActivity(ctx, agentID, sess.ID, "run", nil, "", "uptime output", 100, true, "corr-1")
|
||||
if !s.hadDiscovery(ctx, sess.ID) {
|
||||
t.Fatal("hadDiscovery = false after a successful run call, want true")
|
||||
}
|
||||
if s.hadEntityWriteback(ctx, sess.ID) {
|
||||
t.Fatal("hadEntityWriteback = true after only a run call, want false")
|
||||
}
|
||||
|
||||
// A failed run call should NOT count as discovery (no facts learned).
|
||||
sess2, err := s.createSession(ctx, "failed discovery test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
s.logActivity(ctx, agentID, sess2.ID, "run", nil, "", "ssh timeout", 100, false, "corr-2")
|
||||
if s.hadDiscovery(ctx, sess2.ID) {
|
||||
t.Fatal("hadDiscovery = true after a failed run call, want false (no facts learned)")
|
||||
}
|
||||
|
||||
// A get_entity call should NOT count as discovery (DB lookup, not live state).
|
||||
sess3, err := s.createSession(ctx, "lookup test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
s.logActivity(ctx, agentID, sess3.ID, "get_entity", nil, "", "entity row", 10, true, "corr-3")
|
||||
if s.hadDiscovery(ctx, sess3.ID) {
|
||||
t.Fatal("hadDiscovery = true after get_entity, want false (DB lookups are not discovery)")
|
||||
}
|
||||
|
||||
// update_entity_attributes sets hadEntityWriteback.
|
||||
sess4, err := s.createSession(ctx, "writeback test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
s.logActivity(ctx, agentID, sess4.ID, "update_entity_attributes", nil, "", "ok", 10, true, "corr-4")
|
||||
if !s.hadEntityWriteback(ctx, sess4.ID) {
|
||||
t.Fatal("hadEntityWriteback = false after update_entity_attributes, want true")
|
||||
}
|
||||
// And the discovery+writeback combination (the conv3 scenario).
|
||||
s.logActivity(ctx, agentID, sess4.ID, "run", nil, "", "apt-get update output", 100, true, "corr-5")
|
||||
if !s.hadDiscovery(ctx, sess4.ID) {
|
||||
t.Fatal("hadDiscovery = false after run+writeback, want true")
|
||||
}
|
||||
if !s.hadEntityWriteback(ctx, sess4.ID) {
|
||||
t.Fatal("hadEntityWriteback = false after run+writeback, want true")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSetGoal_SupersessionEvent is the store-level proof for P1.4 from
|
||||
// plans/2026-07-18-session-review-three-sessions.md: when setGoal is called
|
||||
// and a non-empty prior goal already exists with a DIFFERENT value, a
|
||||
// task.superseded event must be emitted (so the audit trail records the
|
||||
// pivot — the row's goal column will be overwritten, losing the prior intent
|
||||
// without this event). When the goal is identical OR no prior goal exists,
|
||||
// no supersession event is emitted.
|
||||
//
|
||||
// Background: session 55927f0a had two set_goal calls; the first was
|
||||
// implicitly abandoned when the operator said "lets just keep ludo-library
|
||||
// then." Without the event, the prior goal silently disappeared.
|
||||
func TestSetGoal_SupersededEvent(t *testing.T) {
|
||||
s := newTestStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
sess, err := s.createSession(ctx, "goal pivot test")
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
|
||||
// First set_goal — no prior, no supersession event expected.
|
||||
if err := s.setGoal(ctx, sess.ID, "Fix sabnzbd download folder to use ludo-lvm"); err != nil {
|
||||
t.Fatalf("setGoal #1: %v", err)
|
||||
}
|
||||
if n := countEvents(ctx, s, sess.ID, "task.superseded"); n != 0 {
|
||||
t.Errorf("after first set_goal: %d task.superseded events, want 0", n)
|
||||
}
|
||||
|
||||
// Second set_goal with a DIFFERENT goal — supersession event expected.
|
||||
if err := s.setGoal(ctx, sess.ID, "Add NFS export of ludo-lvm to ZimaOS"); err != nil {
|
||||
t.Fatalf("setGoal #2: %v", err)
|
||||
}
|
||||
if n := countEvents(ctx, s, sess.ID, "task.superseded"); n != 1 {
|
||||
t.Errorf("after second set_goal with a different goal: %d task.superseded events, want 1", n)
|
||||
}
|
||||
|
||||
// Third set_goal with the SAME goal as the second — no new supersession
|
||||
// event (idempotent: same goal is a no-op, not a pivot).
|
||||
if err := s.setGoal(ctx, sess.ID, "Add NFS export of ludo-lvm to ZimaOS"); err != nil {
|
||||
t.Fatalf("setGoal #3: %v", err)
|
||||
}
|
||||
if n := countEvents(ctx, s, sess.ID, "task.superseded"); n != 1 {
|
||||
t.Errorf("after third set_goal with same goal as second: %d task.superseded events, want 1 (no new pivot)", n)
|
||||
}
|
||||
|
||||
// The session's current goal must be the latest one set.
|
||||
got, err := s.getSession(ctx, sess.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("getSession: %v", err)
|
||||
}
|
||||
if got.Goal != "Add NFS export of ludo-lvm to ZimaOS" {
|
||||
t.Errorf("session goal = %q, want the second (latest) goal", got.Goal)
|
||||
}
|
||||
}
|
||||
|
||||
// countEvents counts observability events of the given type correlated to
|
||||
// the given session. Used by TestSetGoal_SupersededEvent to assert the
|
||||
// task.superseded audit-trail signal was emitted.
|
||||
func countEvents(ctx context.Context, s *store, sessionID, eventType string) int {
|
||||
var n int
|
||||
s.pool.QueryRow(ctx,
|
||||
`SELECT COUNT(*) FROM events WHERE correlation_id = $1 AND type = $2`,
|
||||
sessionID, eventType).Scan(&n)
|
||||
return n
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
@@ -34,20 +35,18 @@ func taskToolDefs() []toolDef {
|
||||
},
|
||||
{
|
||||
Name: "propose_plan",
|
||||
Description: "Lay out ALL the ordered steps you'll take to reach the goal, in ONE " +
|
||||
"call, listing every step end-to-end — not just the next one. The operator " +
|
||||
"sees the full list in the context panel and watches it progress; a plan " +
|
||||
"with only 1 step looks broken to them even if you intend to add more later. " +
|
||||
"Your FIRST step should be research (prior knowledge, relations, blast radius " +
|
||||
"— not just this target's status) and your LAST step should be writing back " +
|
||||
"what you learned (update_entity_attributes / create_relationship / " +
|
||||
"upsert_knowledge) BEFORE complete_task — this is what keeps the knowledge " +
|
||||
"graph from drifting out of date. " +
|
||||
"Call this ONCE, before you start executing (after gathering what you need). " +
|
||||
"As you work, call update_plan_step (not propose_plan again) to advance each " +
|
||||
"step. Only re-call propose_plan if the plan itself has fundamentally changed " +
|
||||
"(e.g. a new approach is needed) — in that case new steps are appended after " +
|
||||
"whatever already ran, never erasing completed work.",
|
||||
Description: "Propose the full ordered plan for this task. Call ONCE, before any " +
|
||||
"execution, with EVERY step end-to-end (not one step at a time). FIRST step: " +
|
||||
"research (prior knowledge, relations, blast radius). If your plan runs `run` " +
|
||||
"against any target, include a LAST step: write back " +
|
||||
"(update_entity_attributes + create_relationship + upsert_knowledge) — if you " +
|
||||
"omit it, one is auto-appended. After this call: STOP and wait for operator " +
|
||||
"approval (approval vocabulary: approved, yes, go, proceed, continue, ok, " +
|
||||
"go ahead). Once a step has started (running/done/...), this tool REFUSES " +
|
||||
"further calls — advance with update_plan_step + run instead. Re-propose only " +
|
||||
"if the operator explicitly asks you to revise the whole plan. complete_task " +
|
||||
"with outcome=success is REFUSED if you ran `run` but didn't call " +
|
||||
"update_entity_attributes/create_relationship — write back before completing.",
|
||||
InputSchema: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
@@ -177,7 +176,16 @@ func (a *agent) handleTaskTool(ctx context.Context, sessionID, name string, args
|
||||
if err := a.store.setGoal(ctx, sessionID, goal); err != nil {
|
||||
return fmt.Sprintf("error setting goal: %v", err), true
|
||||
}
|
||||
return "Goal set: " + goal, true
|
||||
// P1: the plan window is NOT opened here. Opening it on set_goal
|
||||
// meant any config_mutation `run` auto-executed with zero operator
|
||||
// approval, before a plan was even proposed (let alone approved) —
|
||||
// a safety regression confirmed live in session d0d562e0. The
|
||||
// window is now opened only when the operator approves a plan
|
||||
// (chat-assent grant or explicit approval in agent.go), which is
|
||||
// what the SOUL.md "approve the plan, not each step" model actually
|
||||
// describes. set_goal records the goal + flips status to executing
|
||||
// and nothing more.
|
||||
return "Goal set: " + goal + ". NEXT: pre-plan with read-only tools (search_knowledge, get_entity, list_lxcs, get_relations), then propose_plan (mandatory — even read-only tasks need a one-step plan; the run handler refuses without one). After propose_plan: if all steps are read-only, execute immediately (no approval needed). If any step is config_mutation/destructive, stop and wait for operator approval.", true
|
||||
|
||||
case "propose_plan":
|
||||
raw, _ := args["steps"].([]any)
|
||||
@@ -198,11 +206,51 @@ func (a *agent) handleTaskTool(ctx context.Context, sessionID, name string, args
|
||||
if len(steps) == 0 {
|
||||
return "error: propose_plan needs at least one step with a title", true
|
||||
}
|
||||
// D.2: auto-append a writeback step if the agent didn't include one.
|
||||
// The agent consistently writes vague last steps ("record findings")
|
||||
// and then skips update_entity_attributes entirely (the #1 cause of
|
||||
// knowledge-graph drift). Appending an explicit writeback step makes
|
||||
// the seq-order enforcement (5.6) require it to be completed last,
|
||||
// and D.1's complete_task gate enforces the actual calls. Together
|
||||
// they close the loop structurally — neither relies on the agent
|
||||
// reading SOUL.md.
|
||||
hasWritebackStep := false
|
||||
for _, st := range steps {
|
||||
if strings.Contains(st.Title, "update_entity_attributes") ||
|
||||
strings.Contains(st.Title, "create_relationship") ||
|
||||
strings.Contains(st.Detail, "update_entity_attributes") ||
|
||||
strings.Contains(st.Detail, "create_relationship") {
|
||||
hasWritebackStep = true
|
||||
break
|
||||
}
|
||||
}
|
||||
appendedNote := ""
|
||||
if !hasWritebackStep {
|
||||
steps = append(steps, planStepInput{
|
||||
Title: "Write back: update_entity_attributes + create_relationship + upsert_knowledge",
|
||||
Detail: "Call update_entity_attributes for every entity you ran against (versions, states, counts, timestamps). Call create_relationship for any edge you discovered. Then upsert_knowledge about the affected entities (pass `about` as an array).",
|
||||
})
|
||||
appendedNote = fmt.Sprintf(" (appended a writeback step — your plan didn't include one; step %d)", len(steps))
|
||||
}
|
||||
persisted, err := a.store.proposePlan(ctx, sessionID, steps)
|
||||
if err != nil {
|
||||
if errors.Is(err, errPlanInFlight) {
|
||||
// The plan is already in flight — refuse the re-proposal.
|
||||
// The agent must advance the existing plan with
|
||||
// update_plan_step + run. This is the structural fix for
|
||||
// the "plan added twice" sidebar drift the operator
|
||||
// reported: instead of appending (which duplicated) or
|
||||
// wiping (which lost progress), we refuse and direct.
|
||||
return "Plan already in flight — refusing duplicate proposal. Steps exist and at least one has started (running/done/...). To advance: call update_plan_step(seq=K, status=\"running\") then run(...) for step K's target, then update_plan_step(seq=K, status=\"done\"). Do not call propose_plan again. Re-propose only if the operator explicitly asks you to revise the whole plan (the session is reopened on a follow-up — prior steps are marked `replaced` and a fresh generation is started), and say so in your reply before calling it.", true
|
||||
}
|
||||
return fmt.Sprintf("error proposing plan: %v", err), true
|
||||
}
|
||||
return fmt.Sprintf("Plan set: %d step(s). Execute them now, marking each with update_plan_step as you go.", len(persisted)), true
|
||||
// The writeback step is now always present (D.2 auto-appends it if
|
||||
// the agent forgot), so the old advisory nudge is replaced by the
|
||||
// structural gate: D.1 refuses complete_task without the actual
|
||||
// update_entity_attributes/create_relationship calls.
|
||||
result := fmt.Sprintf("Plan set (%d steps)%s. If all steps are read-only, execute now — call update_plan_step(running) + run for each step, no approval needed. If any step is config_mutation/destructive, STOP and wait for operator approval (\"approved\", \"yes\", \"go\", \"proceed\", \"continue\", \"ok\", \"go ahead\"). Do not call propose_plan again.", len(persisted), appendedNote)
|
||||
return result, true
|
||||
|
||||
case "update_plan_step":
|
||||
seq := toInt(args["seq"])
|
||||
@@ -214,7 +262,7 @@ func (a *agent) handleTaskTool(ctx context.Context, sessionID, name string, args
|
||||
if err := a.store.updatePlanStep(ctx, sessionID, seq, status, execID); err != nil {
|
||||
return fmt.Sprintf("error updating step %d: %v", seq, err), true
|
||||
}
|
||||
return fmt.Sprintf("Step %d → %s", seq, status), true
|
||||
return fmt.Sprintf("Step %d → %s. (Advance with update_plan_step + run; do not re-propose.)", seq, status), true
|
||||
|
||||
case "ask_operator":
|
||||
prompt, _ := args["prompt"].(string)
|
||||
@@ -258,10 +306,28 @@ func (a *agent) handleTaskTool(ctx context.Context, sessionID, name string, args
|
||||
"session", sessionID, "outcome", outcome)
|
||||
outcome = "partial"
|
||||
}
|
||||
// D.1: refuse success when discovery ran but no writeback followed.
|
||||
// The prior advisory warning (below) was ignorable — the agent
|
||||
// saw it and ended the task anyway. This gate fires BEFORE
|
||||
// completeTask runs, so the session stays in 'executing' state
|
||||
// and the agent must call update_entity_attributes/create_relationship
|
||||
// then retry complete_task. Only blocks `success`; an explicit
|
||||
// `failure` or `partial` is allowed through (the agent is
|
||||
// acknowledging it didn't finish — no reason to force writeback).
|
||||
if outcome == "success" && a.store.hadDiscovery(ctx, sessionID) && !a.store.hadEntityWriteback(ctx, sessionID) {
|
||||
return "Refused: this session ran `run` against live targets (discovery) but did not call update_entity_attributes or create_relationship to persist what you learned. The knowledge graph will drift if you complete without writeback. Call update_entity_attributes for each entity you ran against (versions, states, counts, timestamps), and create_relationship for any edge you discovered, then call complete_task again. Outcome is held at 'executing' until you do.", true
|
||||
}
|
||||
if err := a.store.completeTask(ctx, sessionID, outcome, summary); err != nil {
|
||||
if errors.Is(err, errTaskAlreadyComplete) {
|
||||
return "Task is already complete. Do not call complete_task again. If the operator pointed out a UI/sidebar inconsistency, fix it with update_plan_step (reconcile step states) or summarize the panel in your reply — do not re-execute the work.", true
|
||||
}
|
||||
return fmt.Sprintf("error completing task: %v", err), true
|
||||
}
|
||||
return fmt.Sprintf("Task marked %s: %s", outcome, summary), true
|
||||
result := fmt.Sprintf("Task marked %s: %s", outcome, summary)
|
||||
if !a.store.hadEntityWriteback(ctx, sessionID) {
|
||||
result += "\n\n⚠️ No entity attributes or relationships were updated in this session. Call update_entity_attributes and create_relationship to persist what you learned about entities before the next session starts from scratch."
|
||||
}
|
||||
return result, true
|
||||
default:
|
||||
return nil, false
|
||||
}
|
||||
@@ -291,3 +357,60 @@ func (a *agent) autoCompleteTrivialTask(ctx context.Context, sessionID, response
|
||||
slog.Error("nomos: auto-complete trivial task failed", "session", sessionID, "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// autoCompleteIfPlanDone is the structural safety net for "the agent did the
|
||||
// work but forgot to call complete_task" — the #1 remaining model reliability
|
||||
// gap after D.1's writeback gate. After a turn ends, if the session has a goal,
|
||||
// the agent never called complete_task this turn, and either (a) all plan
|
||||
// steps are terminal OR (b) the agent did discovery (ran `run`), auto-complete.
|
||||
// Path (b) catches the common case where the agent skips update_plan_step
|
||||
// bookkeeping but still does the actual work — the D.1 gate already enforces
|
||||
// writeback before `complete_task`, so if the agent forgot to complete at all,
|
||||
// we close it out mechanically. If writeback happened → success; if not →
|
||||
// partial (honest: work was done but knowledge graph wasn't updated).
|
||||
func (a *agent) autoCompleteIfPlanDone(ctx context.Context, sessionID, responseText string) {
|
||||
if a.store == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return
|
||||
}
|
||||
sess, err := a.store.getSession(ctx, sessionID)
|
||||
if err != nil || sess.Status != "executing" {
|
||||
return
|
||||
}
|
||||
// Don't auto-complete if there are pending approvals — the agent is
|
||||
// blocked waiting for the operator, not done. Auto-completing here
|
||||
// would close the session and the operator's approval would land on a
|
||||
// dead task. Confirmed in eval: agent hits P5 approval gate, turn
|
||||
// ends, auto-complete fires incorrectly because the approval-queue
|
||||
// `run` responses were logged as success=true in agent_activity.
|
||||
if a.store.hasPendingApprovals(ctx, sessionID) {
|
||||
return
|
||||
}
|
||||
discovery := a.store.hadDiscovery(ctx, sessionID)
|
||||
writeback := a.store.hadEntityWriteback(ctx, sessionID)
|
||||
// (a) all plan steps terminal, OR (b) agent did discovery (ran `run`).
|
||||
shouldComplete := a.store.allPlanStepsTerminal(ctx, sessionID)
|
||||
if !shouldComplete && discovery {
|
||||
shouldComplete = true
|
||||
}
|
||||
if !shouldComplete {
|
||||
return
|
||||
}
|
||||
outcome := "success"
|
||||
if discovery && !writeback {
|
||||
outcome = "partial" // honest: work done, knowledge graph not updated
|
||||
}
|
||||
summary := strings.TrimSpace(responseText)
|
||||
summary = strings.SplitN(summary, "\n", 2)[0]
|
||||
const maxLen = 120
|
||||
if len(summary) > maxLen {
|
||||
summary = summary[:maxLen] + "…"
|
||||
}
|
||||
if summary == "" {
|
||||
summary = "All plan steps completed."
|
||||
}
|
||||
if err := a.store.completeTask(ctx, sessionID, outcome, summary); err != nil {
|
||||
slog.Error("nomos: auto-complete plan-done task failed", "session", sessionID, "error", err)
|
||||
} else {
|
||||
slog.Info("nomos: auto-completed task — agent didn't call complete_task", "session", sessionID, "outcome", outcome)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -21,7 +21,7 @@ oikos.hubris.network {
|
||||
tls {
|
||||
dns ionos {env.IONOS_AUTH_API_TOKEN}
|
||||
}
|
||||
@enroll path /api/v1/clients/enroll
|
||||
@enroll path /api/v1/clients/enroll /oidc-callback
|
||||
handle @enroll {
|
||||
reverse_proxy 192.168.178.182:8090
|
||||
}
|
||||
|
||||
@@ -9,6 +9,7 @@ FROM node:22-alpine AS builder
|
||||
WORKDIR /build/web
|
||||
COPY web/package.json web/package-lock.json ./
|
||||
RUN npm ci
|
||||
COPY VERSION ./
|
||||
COPY web/ ./
|
||||
RUN npm run build
|
||||
|
||||
|
||||
17
docs/index.md
Normal file
17
docs/index.md
Normal file
@@ -0,0 +1,17 @@
|
||||
# Docs
|
||||
|
||||
Long-form reference material for the Oikos platform. Operational state and
|
||||
topology live in the DB (seeded from `seeds/`); these docs cover decisions,
|
||||
procedures, and the system model.
|
||||
|
||||
| Path | Contents |
|
||||
| ---- | -------- |
|
||||
| [adr/](adr/README.md) | Architecture Decision Records (numbered, append-only) |
|
||||
| [mbse/](mbse/README.md) | Model-Based Systems Engineering views of the platform |
|
||||
| [mascot/](mascot/README.md) | MBSE subsystem model for the desktop mascot (planned) |
|
||||
| [operations/](operations/README.md) | Operator runbooks (deploy, rollback, recovery) |
|
||||
|
||||
For agent orientation see [AGENTS.md](../AGENTS.md); for the operating model
|
||||
see [.agents/OIKOS.md](../.agents/OIKOS.md); for development see
|
||||
[CONTRIBUTING.md](../CONTRIBUTING.md). Design plans live in
|
||||
[plans/](../plans/), not here.
|
||||
335
docs/mascot/README.md
Normal file
335
docs/mascot/README.md
Normal file
@@ -0,0 +1,335 @@
|
||||
# Oikos — Desktop Mascot Subsystem Model
|
||||
|
||||
> Companion to [the platform Model](../mbse/README.md) and
|
||||
> [the Framework](../mbse/framework.md). This document is a **subsystem
|
||||
> Model** in Holt's sense — it conforms to the same Framework (Ontology +
|
||||
> Viewpoints, Markdown + Mermaid Notation) rather than restating it, scoped
|
||||
> to a single not-yet-built subsystem of the `web` component: the desktop
|
||||
> mascot ("Cluck"), a pixel-art chicken that lives on the desktop shell.
|
||||
> Where the platform-wide Views in [../mbse/README.md](../mbse/README.md)
|
||||
> and the component View for `web/src` in
|
||||
> [../mbse/components.md](../mbse/components.md#5-web-control-room) speak
|
||||
> at the level of "the SPA," this document goes one layer deeper into one
|
||||
> feature of it — the same relationship [components.md](../mbse/components.md)
|
||||
> has to [README.md](../mbse/README.md), applied recursively.
|
||||
|
||||
**Status of this Model:** the subsystem it describes does not exist in
|
||||
code yet. Every View below is marked **Planned**, not **Verified** —
|
||||
compare to [../mbse/README.md](../mbse/README.md)'s confidence grading,
|
||||
which this document borrows. The corresponding implementation plan is
|
||||
[plans/2026-07-20-desktop-mascot.md](../../plans/2026-07-20-desktop-mascot.md).
|
||||
|
||||
## Views in this model
|
||||
|
||||
| # | View | Concern it addresses |
|
||||
|---|---|---|
|
||||
| [1](#1-mission--system-context) | Mission & System Context | Why a mascot, and what is it never allowed to do? |
|
||||
| [2](#2-requirements) | Requirements | What must it do, traced from the original request? |
|
||||
| [3](#3-structural-view) | Structural View | What modules make it up, and which are the extension points? |
|
||||
| [4](#4-behavioral-view) | Behavioral View | How does it move, live, and react, moment to moment? |
|
||||
| [5](#5-interfaces-view) | Interfaces View | What does it read from the rest of the system, and how does it persist itself? |
|
||||
| [6](#6-extension-guide) | Extension Guide | How does a future engineer add an animation, behavior, menu action, or reaction? |
|
||||
| [7](#7-verification-view) | Verification View | How will we know it works, once built? |
|
||||
|
||||
## 1. Mission & System Context
|
||||
|
||||
**Stakeholders:** the operator (delight, ambient awareness of system
|
||||
state without opening a window); future engineers extending the mascot's
|
||||
behaviors/reactions/menu.
|
||||
|
||||
**Mission:** give the desktop shell a persistent, living presence that
|
||||
makes background system activity legible at a glance — a chat streaming,
|
||||
a knowledge-graph write, a critical signal — without requiring a window to
|
||||
be open, while doubling as a lightweight tamagotchi for its own sake
|
||||
(delight is a legitimate requirement here, not a side effect).
|
||||
|
||||
**Boundary — what the mascot is, and is not:**
|
||||
|
||||
- It is a **purely client-side, read-only observer**. It subscribes to
|
||||
existing `web` stores (chat, activity, events, dashboard summary) the
|
||||
same way any other UI component does.
|
||||
- It **never calls a mutating API endpoint** and is not a new actuation
|
||||
path — it has no relationship to the `run` gate, `Execution`, or
|
||||
`Approval` entities described in [the platform Ontology](../mbse/ontology.md).
|
||||
Its only "mutation" is its own tamagotchi state, stored client-side.
|
||||
- It is scoped entirely inside the `web` component
|
||||
([../mbse/components.md §5](../mbse/components.md#5-web-control-room));
|
||||
it introduces no new backend surface, no new MCP tool, no new REST route.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph SURFACE["Desktop shell surface (Desktop.svelte)"]
|
||||
ICONS["Icon layer\nz-0"]
|
||||
LAUNCH["Task launcher\nz-10"]
|
||||
WIN["WindowLayer\nz-40"]
|
||||
MASCOT["MascotLayer\nz-45\n(this subsystem)"]
|
||||
MENU["Desktop context menu\nz-50"]
|
||||
end
|
||||
|
||||
MASCOT -->|subscribes, read-only| EVENTS["stores/events.ts\nliveEvents (SSE)"]
|
||||
MASCOT -->|subscribes, read-only| CHAT["stores/chat.ts\nstreaming"]
|
||||
MASCOT -->|subscribes, read-only| ACTIVITY["stores/activity.ts\nactivityLog"]
|
||||
MASCOT -->|subscribes, read-only| CONTEXT["stores/context.ts\nsummary"]
|
||||
MASCOT -->|reads/writes| LS["localStorage\noikos-mascot"]
|
||||
|
||||
style MASCOT fill:#fff3e0,stroke:#e65100
|
||||
```
|
||||
|
||||
## 2. Requirements
|
||||
|
||||
Traced from the original feature request. All **Planned**.
|
||||
|
||||
| ID | Statement | Source | Status |
|
||||
|---|---|---|---|
|
||||
| MASC-1 | The mascot SHALL render as pixel-art, drawn from code (string pixel-grids + palette), not binary sprite assets | User request | Planned |
|
||||
| MASC-2 | The mascot SHALL roam the desktop surface autonomously, walking along the ground (surface bottom, above the taskbar) under gravity | User request + design decision | Planned |
|
||||
| MASC-3 | The mascot SHALL be draggable with the mouse; releasing it mid-air SHALL trigger a flutter-fall back to the ground | User request + design decision | Planned |
|
||||
| MASC-4 | Right-clicking the mascot SHALL open a round (Sims-style) interaction menu supporting nested submenus | User request | Planned |
|
||||
| MASC-5 | The mascot SHALL have a tamagotchi lifecycle: egg → chick → adult, with a user-assignable name | User request | Planned |
|
||||
| MASC-6 | The mascot's stage, name, and stats SHALL persist across reloads | User request (implied by "tamagotchi") | Planned |
|
||||
| MASC-7 | The mascot SHALL have idle states (autonomous behavior when untouched) and interactive states (drag, click, menu) | User request | Planned |
|
||||
| MASC-8 | The mascot SHALL react visibly to real application activity: chat streaming, knowledge-graph writes, critical signals | User request ("aware of its environment... feels alive and connected") | Planned |
|
||||
| MASC-9 | Animations, behaviors, menu actions, and reactions SHALL each be defined in a single data-driven registry, so a new one can be added without touching the engine code | User request ("easily expansible") | Planned |
|
||||
| MASC-10 (NFR) | The mascot's game loop SHALL run at ~30fps via `setTimeout`, not `requestAnimationFrame`, matching the repo's existing [`GraphBackground.svelte`](../../web/src/lib/components/GraphBackground.svelte) convention (rAF suspends in some hidden-tab embeddings) | Codebase convention | Planned |
|
||||
| MASC-11 (NFR) | The mascot SHALL never write to the API; all mutation is local (localStorage) | Design decision, this document §1 | Planned |
|
||||
| MASC-12 (NFR) | Persistence writes SHALL be debounced (~300ms), never per animation frame | Codebase convention ([`stores/windows.ts`](../../web/src/lib/stores/windows.ts) wmkit persist) | Planned |
|
||||
|
||||
## 3. Structural View
|
||||
|
||||
**Stakeholders:** an engineer implementing or extending the mascot.
|
||||
**Why this View earns its place:** MASC-9 (extensibility) is only real if
|
||||
the module boundaries actually separate data (registries) from engine
|
||||
code; this View is the check that they do.
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class types_ts {
|
||||
<<module>>
|
||||
PixelGrid
|
||||
AnimName
|
||||
MascotStage
|
||||
BehaviorId
|
||||
Stimulus
|
||||
RadialAction
|
||||
}
|
||||
class palette_ts {
|
||||
<<module, registry>>
|
||||
PALETTE: char to CSS color
|
||||
}
|
||||
class sprites_ts {
|
||||
<<module, registry>>
|
||||
SPRITES: Stage to AnimName to AnimDef
|
||||
resolveAnim(stage, name)
|
||||
}
|
||||
class render_ts {
|
||||
<<module, stateless>>
|
||||
drawFrame(ctx, grid, palette, flip)
|
||||
}
|
||||
class state_svelte_ts {
|
||||
<<module, runes>>
|
||||
MascotModel state
|
||||
grantXp() feed() pet() setName()
|
||||
tickLifecycle() advanceStageIfReady()
|
||||
persist (debounced, oikos-mascot)
|
||||
}
|
||||
class behavior_ts {
|
||||
<<module, registry>>
|
||||
BEHAVIORS: BehaviorId to BehaviorDef
|
||||
stepMascot(rt, model, now, dt)
|
||||
}
|
||||
class stimuli_ts {
|
||||
<<module, registry>>
|
||||
REACTIONS: id to ReactionDef
|
||||
attachStimuli(emit)
|
||||
}
|
||||
class actions_ts {
|
||||
<<module, registry>>
|
||||
MASCOT_ACTIONS: RadialAction tree
|
||||
registerMascotAction()
|
||||
}
|
||||
class Mascot_svelte {
|
||||
<<component>>
|
||||
canvas render loop 30fps
|
||||
pointer drag/click/contextmenu
|
||||
}
|
||||
class MascotLayer_svelte {
|
||||
<<component>>
|
||||
z-45 absolute overlay
|
||||
hosts Mascot + RadialMenu + bubble
|
||||
}
|
||||
class RadialMenu_svelte {
|
||||
<<component>>
|
||||
z-60 fixed, nested rings
|
||||
}
|
||||
class NameDialog_svelte {
|
||||
<<component>>
|
||||
}
|
||||
|
||||
sprites_ts --> palette_ts : indexes
|
||||
sprites_ts --> types_ts : uses
|
||||
Mascot_svelte --> render_ts : draws frames
|
||||
Mascot_svelte --> sprites_ts : resolves anim
|
||||
Mascot_svelte --> behavior_ts : steps FSM
|
||||
Mascot_svelte --> state_svelte_ts : reads/mutates model
|
||||
MascotLayer_svelte --> Mascot_svelte : hosts
|
||||
MascotLayer_svelte --> RadialMenu_svelte : hosts, on contextmenu
|
||||
MascotLayer_svelte --> stimuli_ts : attaches on mount
|
||||
MascotLayer_svelte --> NameDialog_svelte : hosts, on hatch/rename
|
||||
RadialMenu_svelte --> actions_ts : renders tree
|
||||
stimuli_ts --> behavior_ts : forceBehavior(react)
|
||||
```
|
||||
|
||||
**The four extension registries** (MASC-9's concrete answer — see also
|
||||
[§6 Extension Guide](#6-extension-guide)): `SPRITES` (animations),
|
||||
`BEHAVIORS` (autonomous states), `MASCOT_ACTIONS` (radial menu tree),
|
||||
`REACTIONS` (environment stimuli). Each is plain data; the engine
|
||||
(`behavior.ts`'s `stepMascot`, `Mascot.svelte`'s loop, `RadialMenu.svelte`'s
|
||||
renderer) is generic over whatever the registry currently contains.
|
||||
|
||||
**Mount point:** two lines in
|
||||
[`Desktop.svelte`](../../web/src/lib/components/desktop-shell/Desktop.svelte) —
|
||||
`<MascotLayer />` rendered inside the surface `<div>` (the `relative
|
||||
min-h-0 flex-1 overflow-hidden` element), after `<WindowLayer />`, so its
|
||||
`absolute inset-0` shares the surface's coordinate space and its ground
|
||||
line is exactly the surface's bottom edge (= the taskbar's top edge).
|
||||
|
||||
## 4. Behavioral View
|
||||
|
||||
**Stakeholders:** an engineer reasoning about "what does the mascot do
|
||||
right now, and why." **Why this View earns its place:** a mascot with an
|
||||
implicit, ad-hoc state machine is unmaintainable the moment a second
|
||||
behavior or reaction is added; this View is the state machine made
|
||||
explicit before any of it is coded.
|
||||
|
||||
### 4.1 Behavior FSM (moment-to-moment autonomy)
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> egg
|
||||
egg --> chick : hatchProgress reaches 1\n(advanceStageIfReady)
|
||||
|
||||
state chick_and_adult_behaviors {
|
||||
[*] --> idle
|
||||
idle --> wander : weighted random pick\non behaviorUntil expiry
|
||||
wander --> idle
|
||||
idle --> peck : weighted random pick
|
||||
peck --> idle
|
||||
idle --> sleep : weighted random pick
|
||||
sleep --> idle
|
||||
wander --> falling : y below ground\n(off a dragged edge, etc.)
|
||||
idle --> dragged : pointerdown + move\npast 5px threshold
|
||||
wander --> dragged : pointerdown + move
|
||||
sleep --> dragged : pointerdown + move\n(interrupts sleep)
|
||||
dragged --> falling : pointerup, released mid-air
|
||||
falling --> land : y reaches ground
|
||||
land --> idle
|
||||
[*] --> react : stimulus dispatched\n(priority/cooldown gated)
|
||||
react --> idle : durationMs elapsed,\nreturns to prior-or-idle
|
||||
}
|
||||
|
||||
chick --> adult : xp reaches ADULT_XP\n(advanceStageIfReady)
|
||||
```
|
||||
|
||||
`dragged` always wins over any autonomous behavior; `sleep` is broken only
|
||||
by a reaction whose `ReactionDef.interruptsSleep` is true (§4.3) or by a
|
||||
drag. Weighted-random idle selection (`weight` field in `BehaviorDef`)
|
||||
picks the next autonomous behavior only when the current one's `next()`
|
||||
returns null past `behaviorUntil` — see
|
||||
[plans/2026-07-20-desktop-mascot.md](../../plans/2026-07-20-desktop-mascot.md)
|
||||
for the concrete weights.
|
||||
|
||||
### 4.2 Tamagotchi lifecycle (long-lived state)
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> egg : first load,\ndefaultModel()
|
||||
egg --> chick : active time >= HATCH_MS (3min)\n+ NameDialog shown
|
||||
chick --> adult : xp >= ADULT_XP (200)
|
||||
adult --> [*]
|
||||
```
|
||||
|
||||
This is a separate state machine from §4.1: §4.1 governs frame-to-frame
|
||||
motion/animation, §4.2 governs the tamagotchi's slow-moving `MascotModel`
|
||||
(persisted, ticked ~1x/sec via `tickLifecycle`, not every frame).
|
||||
|
||||
### 4.3 Example sequence — an environment stimulus becomes a visible reaction
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant SSE as stores/events.ts (SSE)
|
||||
participant Stim as stimuli.ts attachStimuli
|
||||
participant FSM as behavior.ts
|
||||
participant Mascot as Mascot.svelte (canvas)
|
||||
|
||||
SSE->>Stim: liveEvents updates,\nnew head event severity=critical
|
||||
Stim->>Stim: check REACTIONS['alarmed']\ncooldown + priority
|
||||
Stim->>FSM: forceBehavior(rt, 'react', {anim: 'react-alarm', durationMs})
|
||||
FSM->>FSM: interrupts current behavior\n(even sleep, interruptsSleep=true)
|
||||
FSM->>Mascot: rt.behavior = 'react', rt.anim = 'react-alarm'
|
||||
Mascot->>Mascot: next 30fps tick draws\nreact-alarm frame
|
||||
Note over FSM: after durationMs,\nnext() returns to idle
|
||||
```
|
||||
|
||||
## 5. Interfaces View
|
||||
|
||||
**Stakeholders:** an engineer wiring a new store into the mascot's
|
||||
awareness, or auditing what it depends on.
|
||||
|
||||
| Interface | Direction | Shape | Notes |
|
||||
|---|---|---|---|
|
||||
| [`stores/events.ts`](../../web/src/lib/stores/events.ts) `liveEvents` | consumed | `Writable<OikosEvent[]>`, newest-first, ref-counted via `subscribeEvents()` | `OikosEvent.type` families: `approval.*`, `signal.*`, `execution.*`, `health.changed`; `severity: 'info'\|'warning'\|'critical'` |
|
||||
| [`stores/chat.ts`](../../web/src/lib/stores/chat.ts) `streaming` | consumed | `Writable<boolean>` | false→true edge triggers the `thinking` reaction, held while true |
|
||||
| [`stores/activity.ts`](../../web/src/lib/stores/activity.ts) `activityLog` | consumed | derived `Readable<ActivityEntry[]>`, **recomputed wholesale** on every emission — not append-only | new entries with `type === 'knowledge'` detected by diffing entry `id`s between emissions, not by treating it as a stream |
|
||||
| [`stores/context.ts`](../../web/src/lib/stores/context.ts) `summary` | consumed | `Writable<DashboardSummary\|null>` | ambient state (open signal counts via `openSignalCount(summary)`) |
|
||||
| `localStorage['oikos-mascot']` | owned | `MascotModel` JSON, `{ version: 1, stage, name, hatchProgress, happiness, xp, hatchedAt, lastPos: {x}, lastSeen }` | debounced write (~300ms, mirrors [`stores/windows.ts`](../../web/src/lib/stores/windows.ts) wmkit persist) + `beforeunload` flush; `version` field reserved for a future `migrate()`; multi-tab is last-writer-wins (accepted, documented, not solved) |
|
||||
| [`Desktop.svelte`](../../web/src/lib/components/desktop-shell/Desktop.svelte) mount | owned | `<MascotLayer />`, 2-line insertion | see §3 |
|
||||
|
||||
No interface in this table is a write path to the Oikos API — consistent
|
||||
with §1's boundary statement (MASC-11).
|
||||
|
||||
## 6. Extension Guide
|
||||
|
||||
**Stakeholders:** a future engineer adding one new animation, behavior,
|
||||
menu action, or reaction — this is the Viewpoint 4's "why" made concrete
|
||||
as a recipe rather than prose (mirrors [../mbse/framework.md §7](../mbse/framework.md)'s
|
||||
Process Set treatment).
|
||||
|
||||
| To add a... | Touch only | Nothing else changes because |
|
||||
|---|---|---|
|
||||
| **Animation** | Add the name to the `AnimName` union in `types.ts`; add frames to `SPRITES[stage]` in `sprites.ts` | `resolveAnim()` and the renderer are generic over the registry |
|
||||
| **Behavior** | Add the id to `BehaviorId`; add one `BehaviorDef` entry to `BEHAVIORS` in `behavior.ts` | `stepMascot()` and the weighted-random idle selector consume `BEHAVIORS` generically |
|
||||
| **Radial menu action** | Add a `RadialAction` node to `MASCOT_ACTIONS` in `actions.ts` (or call `registerMascotAction()`), optionally nested under `children` | `RadialMenu.svelte` renders whatever tree it's given, including nesting depth |
|
||||
| **Environment reaction** | Add a `ReactionDef` to `REACTIONS` in `stimuli.ts`; wire one `store subscription -> predicate -> emit(reaction)` block inside `attachStimuli()` | priority/cooldown/interrupt dispatch logic in `attachStimuli()` is generic over `REACTIONS` |
|
||||
|
||||
## 7. Verification View
|
||||
|
||||
**Stakeholders:** whoever implements this subsystem and needs to know
|
||||
when it's actually done, not just compiled.
|
||||
|
||||
Manual browser checklist (no automated test harness planned for v1 — see
|
||||
[plans/2026-07-20-desktop-mascot.md](../../plans/2026-07-20-desktop-mascot.md)
|
||||
for the same list in implementation-order context):
|
||||
|
||||
- Egg renders grounded at the surface bottom, wiggles occasionally, survives
|
||||
a reload at the same x (confirm `oikos-mascot` is debounced — no writes
|
||||
fire from mere walking, only from discrete transitions).
|
||||
- Dragging the egg up and releasing triggers a flutter-fall with no
|
||||
tunneling below the taskbar; dragging past the surface edges clamps.
|
||||
- Forcing hatch (debug menu action) transitions to chick, opens the name
|
||||
dialog, and the name persists across reload.
|
||||
- Chick wanders and flips sprite at surface edges, pecks, sleeps
|
||||
autonomously; a plain click (no drag) triggers a pet/hop reaction.
|
||||
- Right-clicking the chicken opens the radial menu centered on it, without
|
||||
triggering the desktop's own right-click menu; a nested submenu (Feed)
|
||||
opens correctly; Escape pops one level then closes; an outside click
|
||||
closes it; the menu stays fully visible when the chicken is near a
|
||||
screen edge or corner.
|
||||
- With one or more windows open (including a maximized one), the chicken
|
||||
visibly walks above them without breaking window drag/resize/close.
|
||||
- Starting a chat and observing it stream triggers the `thinking` reaction
|
||||
for the duration; a simulated knowledge-graph write triggers `eureka`
|
||||
once per cooldown window; a simulated critical signal triggers `alarmed`
|
||||
even while the chicken is asleep.
|
||||
- Resizing the browser viewport re-grounds and re-clamps the chicken.
|
||||
- Both the Terracotta and Carbon themes keep the pixel-art palette legible.
|
||||
- `npm run build` passes with no new errors.
|
||||
1610
docs/mbse/README.md
Normal file
1610
docs/mbse/README.md
Normal file
File diff suppressed because it is too large
Load Diff
509
docs/mbse/components.md
Normal file
509
docs/mbse/components.md
Normal file
@@ -0,0 +1,509 @@
|
||||
# Oikos — Component Views
|
||||
|
||||
> Companion to [the Model](README.md) and [the Framework](framework.md).
|
||||
> Where README.md's nine Views cut across the whole system by *concern*
|
||||
> (requirements, behavior, risk...), this document cuts across it by
|
||||
> *component* — one View per running part of the System, going one layer
|
||||
> deeper into its own internal structure than the whole-system Views do.
|
||||
> Per [framework.md](framework.md) §4, each section below is still a View
|
||||
> and must still answer Holt's three questions; they're stated once per
|
||||
> section rather than as a separate table, since here the Stakeholder is
|
||||
> almost always the same ("an engineer about to change this component")
|
||||
> and the Notation is the same (prose + Mermaid) throughout.
|
||||
|
||||
**How to use this alongside the other two documents:** if you're deciding
|
||||
*whether something belongs in the Model*, read [framework.md](framework.md).
|
||||
If you're asking *what does the system do and why*, read
|
||||
[README.md](README.md). If you're about to **change code in a specific
|
||||
package** and want to know its internal shape, its own state, and what's
|
||||
already known to be broken or dormant inside it before you touch it, read
|
||||
the relevant section here.
|
||||
|
||||
## Contents
|
||||
|
||||
| Component | Path | Status |
|
||||
|---|---|---|
|
||||
| [1. oikos api](#1-oikos-api) | `internal/httpapi`, `internal/mcp`, `internal/policy` | ✅ live — the decision/execution gate |
|
||||
| [2. oikos scheduler](#2-oikos-scheduler) | `internal/scheduler`, `internal/checkdefaults` | ✅ live — the observe loop |
|
||||
| [3. oikos notifier](#3-oikos-notifier) | `internal/notifier` | ✅ live — approval delivery |
|
||||
| [4. nomos](#4-nomos-agent-gateway) | `cmd/nomos` | ✅ live — the agent, unauthenticated gateway |
|
||||
| [5. web control room](#5-web-control-room) | `web/src` | ✅ live — standalone SPA |
|
||||
| [6. PostgreSQL/TimescaleDB](#6-postgresqltimescaledb) | `migrations/`, `seeds/` | ✅ live — the System's own source of truth |
|
||||
| [7. Dormant components](#7-dormant-components) | `internal/actuator`, `internal/learning` | 🔴 compiled, never started |
|
||||
| [8. Auxiliary components](#8-auxiliary-components) | `cmd/webhook`, `cmd/desktop` | ✅ live — deploy + packaging, not decision logic |
|
||||
|
||||
---
|
||||
|
||||
## 1. oikos api
|
||||
|
||||
**Stakeholders:** engineers extending the MCP tool surface, the `run` gate,
|
||||
or REST endpoints; anyone debugging why a specific command was or wasn't
|
||||
classified the way they expected. **Why this View earns its place:** this
|
||||
is the single component where the highest-consequence findings in
|
||||
[Risk & Safety](README.md#8-risk--safety) live — extending it without
|
||||
knowing its internal shape is how the kill-switch gap and the
|
||||
two-classifiers problem happened in the first place.
|
||||
|
||||
### oikos api — Internal structure
|
||||
|
||||
| File | Lines | Role |
|
||||
|---|---|---|
|
||||
| `internal/httpapi/server.go` | 842 | `NewHandler` (routing entry, L75), `combinedAuth` (L229), OIDC JWKS discovery/fetch/validate (L337-524), `GetActor` (L526), OIDC config/token/callback handlers (L575-712), `ListenAndServe` (L811) |
|
||||
| `internal/httpapi/impl.go` | 1,639 | Entity CRUD, lifecycle transitions + preconditions (per [ADR-0014](../adr/0014-entity-model.md)) |
|
||||
| `internal/httpapi/phase3.go` | 2,627 | Executions, approvals (`DecideApproval`), `sshExec`, `executeApprovedAction`, autonomy-settings read/write endpoints — the **largest single file in the component** |
|
||||
| `internal/httpapi/sse.go` | 366 | `LISTEN/NOTIFY` fan-out, ring-buffer replay |
|
||||
| `internal/httpapi/activity.go` | 215 | `agent_activity` read endpoints |
|
||||
| `internal/httpapi/knowledge.go` | 286 | Knowledge search/content endpoints |
|
||||
| `internal/httpapi/dashboard.go` | 172 | `dashboard/summary` |
|
||||
| `internal/httpapi/learning_view.go` | 129 | `learning/timeline`, `learning/trend` |
|
||||
| `internal/httpapi/problem.go` | 71 | RFC 9457 `problem+json` error envelope |
|
||||
| `internal/httpapi/default_checks.go` | 13 | Thin wrapper calling `internal/checkdefaults` on entity creation |
|
||||
| `internal/mcp/server.go` | 1,691 | All 33 MCP tool registrations (`get_entity` at L76 through `list_my_secrets` at L753), `sshExec` (L1031), `resolveExecTarget` (L1223), **`classifyAndGate`** (L1264-1417) |
|
||||
| `internal/policy/command.go` | 174 | `ClassifyCommand` (L108) — the **live** classifier, `computeCommandRisk` (L127), `allSegmentsReadOnly` (L157), `riskRank` (L26) |
|
||||
| `internal/policy/classify.go` | 157 | `ClassifySignal` (L46) — **dead code, zero callers** (see [Roadmap §9.2](README.md#92-code-real--dead-code--schema-only-matrix)) |
|
||||
|
||||
### oikos api — internal call structure: the `run` gate, by file
|
||||
|
||||
README.md's [§3.3](README.md#3-functional-architecture) shows the *decision
|
||||
logic* of the `run` gate. This shows the *code path* — which file hands off
|
||||
to which — because they're not the same question: the decision flowchart
|
||||
tells you what happens, this tells you where to go fix it.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
MCP["mcp/server.go\nrun tool handler, L366"] --> GATE["mcp/server.go\nclassifyAndGate, L1264"]
|
||||
GATE --> RESOLVE["mcp/server.go\nresolveExecTarget, L1223"]
|
||||
GATE --> CLASSIFY["policy/command.go\nClassifyCommand, L108"]
|
||||
CLASSIFY --> RISK["policy/command.go\ncomputeCommandRisk, L127\nallSegmentsReadOnly, L157"]
|
||||
GATE -->|read_only or window open| EXEC["mcp/server.go\nsshExec, L1031"]
|
||||
GATE -->|otherwise| APPROVAL["phase3.go\ncreateApproval"]
|
||||
APPROVAL -->|operator decides| DECIDE["phase3.go\nDecideApproval"]
|
||||
DECIDE --> EXEC2["phase3.go\nsshExec\n(separate implementation)"]
|
||||
|
||||
style CLASSIFY fill:#e8f5e9,stroke:#2e7d32
|
||||
style EXEC fill:#fff3e0,stroke:#e65100
|
||||
style EXEC2 fill:#fff3e0,stroke:#e65100
|
||||
```
|
||||
|
||||
The two orange boxes are the same finding stated visually: `mcp/server.go`
|
||||
and `phase3.go` each have **their own `sshExec`**, independently written,
|
||||
not sharing an implementation. Fix one path's SSH handling and the other is
|
||||
untouched — verified during the Roadmap audit, not assumed.
|
||||
|
||||
### oikos api — Interfaces this component owns
|
||||
|
||||
Full catalogs live in [README.md §5](README.md#5-interfaces-icd) (33 MCP
|
||||
tools, REST groups, SSE event types) — not repeated here. What's specific
|
||||
to *this* component's internal ownership: `internal/mcp/server.go` owns
|
||||
every MCP tool; `internal/httpapi/{impl,phase3,sse,activity,knowledge,
|
||||
dashboard,learning_view}.go` own the REST surface between them, split by
|
||||
resource area rather than by file size; `internal/policy/command.go` is a
|
||||
pure function library with no HTTP surface of its own, called only from
|
||||
`classifyAndGate`.
|
||||
|
||||
### oikos api — Status and known issues
|
||||
|
||||
All of the following are detailed with evidence in
|
||||
[Roadmap & Traceability](README.md#9-roadmap--traceability) and
|
||||
[Risk & Safety](README.md#8-risk--safety) — cross-referenced here so an
|
||||
engineer opening this specific package sees them before making a change,
|
||||
not after:
|
||||
|
||||
- `policy.ClassifySignal` (in this component) is dead code; the schema it
|
||||
reads (`autonomy_settings.global.auto_act`, `never_auto_act.*`) is
|
||||
therefore not enforced by anything in the live request path — [§8.1](README.md#81-the-kill-switch-gap-verified-most-important-finding-in-this-model).
|
||||
- `notifier.VerifyApprovalToken` (a different component, §3 below) is dead;
|
||||
`phase3.go:DecideApproval` reimplements token verification inline instead
|
||||
of calling it.
|
||||
- `domain.Execution`'s state constants are descriptive only — `phase3.go`
|
||||
writes ad-hoc SQL string statuses that don't map 1:1 onto them.
|
||||
- SSH host key verification is disabled (`InsecureIgnoreHostKey`) on the
|
||||
actuation path — open gap B4.
|
||||
|
||||
---
|
||||
|
||||
## 2. oikos scheduler
|
||||
|
||||
**Stakeholders:** engineers adding a new probe kind or debugging why a
|
||||
signal did or didn't fire. **Why this View earns its place:** the
|
||||
scheduler is the only component that runs unattended on a fixed interval
|
||||
with no operator or agent triggering it — its failure modes look different
|
||||
from every request-driven component above.
|
||||
|
||||
### oikos scheduler — Internal structure
|
||||
|
||||
| File | Lines | Role |
|
||||
|---|---|---|
|
||||
| `internal/scheduler/scheduler.go` | 761 | Everything — no sub-packages |
|
||||
| `internal/scheduler/init.go` | 13 | `RunnerForMain()` — the only thing `cmd/oikos`'s `scheduler` role calls |
|
||||
| `internal/checkdefaults/defaults.go` | — | `ForEntityType` (L60-123), `Ensure` (L144-204), `DefaultInterval` (L133-142) — default check provisioning on entity creation |
|
||||
|
||||
Key functions inside `scheduler.go`: `Run` (L36-64, the tick loop, default
|
||||
30s), `runCheckPass` (L67-94, loads `check_defs`, dispatches with a
|
||||
10-worker `errgroup` limit), `runCheck` (L104-186), `executeCheck` (L237-254,
|
||||
the kind dispatcher), `resolveSignal` (L206-225), `evaluateSeverity`
|
||||
(L737-760), `staleSweep` (L286-333, 3× fastest interval / 5 min floor).
|
||||
|
||||
### oikos scheduler — behavior specific to this component: the probe dispatch
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
TICK["Run tick, every 30s"] --> LOAD["ListEnabledCheckDefs"]
|
||||
LOAD --> DISPATCH["executeCheck: dispatch by kind"]
|
||||
DISPATCH --> HTTP["checkHTTP, L336"]
|
||||
DISPATCH --> TCP["checkTCP, L393"]
|
||||
DISPATCH --> DISK["checkDisk, L424"]
|
||||
DISPATCH --> CERT["checkCertExpiry, L474"]
|
||||
DISPATCH --> PING["checkPing, L545"]
|
||||
DISPATCH --> SSH["checkSSHScript, L613"]
|
||||
HTTP & TCP & DISK & CERT & PING & SSH --> RESULT["checkResult struct\nhealth, signalKind, evidence, metrics"]
|
||||
RESULT -->|healthy| RESOLVE["resolveSignal\nraw SQL, bypasses Signal.CanTransition"]
|
||||
RESULT -->|unhealthy| UPSERT["UpsertSignal\ndedup by target+kind"]
|
||||
RESULT --> METRICS["INSERT metric_samples"]
|
||||
RESULT --> STATUS["UpsertEntityStatus"]
|
||||
```
|
||||
|
||||
`checkSSHScript` (L613-724) is the odd one out: it shells out to the system
|
||||
`ssh` binary directly (`BatchMode=yes`, `StrictHostKeyChecking=no`) rather
|
||||
than using a Go SSH library, restricted to scripts matching
|
||||
`^[a-z][a-z0-9_-]+\.sh$` at a fixed path `/opt/oikos/checks/<script>`. The
|
||||
18 scripts it can run (`cpu_check.sh`, `disk_usage_check.sh`,
|
||||
`docker_health_check.sh`, `zfs_check.sh`, …) live in `checks/` in this repo
|
||||
and are auto-deployed to every enrolled client by `tools/setup-checks.sh`
|
||||
(per AGENTS.md §8) — this component's actual probe logic is split between
|
||||
Go code here and shell scripts version-controlled elsewhere in the repo.
|
||||
|
||||
### oikos scheduler — Interfaces this component owns
|
||||
|
||||
No external API — this is the one component with no inbound interface at
|
||||
all, only outbound: SSH to the fleet (probes), and writes to
|
||||
`metric_samples`/`signals`/`entity_status`/`events` that every other
|
||||
component reads. It is a pure producer.
|
||||
|
||||
### oikos scheduler — Status and known issues
|
||||
|
||||
- Never calls `policy.ClassifySignal` — signals it raises sit as
|
||||
`state='raised'` with no automatic classification; whatever consumes
|
||||
them downstream (the agent, the console) does its own interpretation.
|
||||
- `resolveSignal` updates `raised → resolved` via raw SQL, bypassing the
|
||||
one enforced state machine in the domain layer
|
||||
(`domain.Signal.CanTransition`) — the specific transition happens to be
|
||||
legal today, but nothing would stop a future change from making it not.
|
||||
|
||||
---
|
||||
|
||||
## 3. oikos notifier
|
||||
|
||||
**Stakeholders:** engineers debugging a missed or duplicate Matrix alert,
|
||||
or extending the approval-delivery mechanism to a new channel.
|
||||
**Why this View earns its place:** this is the one component whose entire
|
||||
job is bridging an asynchronous human decision into the same-shaped
|
||||
synchronous decision every other component expects — worth understanding
|
||||
in isolation before assuming "approval" means one simple thing.
|
||||
|
||||
### oikos notifier — Internal structure
|
||||
|
||||
All in `internal/notifier/notifier.go` (305 lines, one file, no
|
||||
sub-packages): `Run` (L25-47, two tickers — 15s for pending approvals, 30s
|
||||
for reaction polling), `processPendingApprovals` (L65-115, generates the
|
||||
token/hash lazily on first pass), `generateApprovalToken` (L275-283,
|
||||
HMAC-SHA256 over approval ID + nanosecond timestamp), `hashToken`
|
||||
(L302-305, only the hash is stored), `sendMatrixAlert` (L231-272),
|
||||
`pollReactions`/`checkReaction` (L118-201), `callDecideApproval`
|
||||
(L204-228), `VerifyApprovalToken` (L286-300, **dead code**).
|
||||
|
||||
### oikos notifier — Behavior specific to this component
|
||||
|
||||
The full sequence (Matrix + console paths converging on one decision
|
||||
endpoint) is in [README.md §6.4](README.md#64-sequence--the-run-primitive-end-to-end).
|
||||
Specific to this component in isolation: it never calls into
|
||||
`internal/httpapi` directly except through one HTTP call
|
||||
(`callDecideApproval`, an ordinary client request to
|
||||
`POST /api/v1/approvals/{id}/decision`) — the notifier and the API process
|
||||
communicate **only through the database and one HTTP endpoint**, never
|
||||
through shared Go state, which is why the header comment in `notifier.go`
|
||||
calls this a "DB rendezvous pattern."
|
||||
|
||||
### oikos notifier — Interfaces this component owns
|
||||
|
||||
Outbound only: the Matrix client-server API
|
||||
(`PUT /rooms/.../send/m.room.message`, `GET /relations/.../m.annotation`)
|
||||
and one outbound call to the API's own approval-decision endpoint. No
|
||||
inbound interface — nothing calls into the notifier process.
|
||||
|
||||
### oikos notifier — Status and known issues
|
||||
|
||||
- `VerifyApprovalToken` is dead code; `phase3.go:DecideApproval` (a
|
||||
different component, §1 above) reimplements the same hash-compare logic
|
||||
inline rather than calling it — a single source of truth for token
|
||||
verification does not currently exist.
|
||||
- Open gap A2: `alert_sent_at` is written *after* the send attempt, so a
|
||||
failed UPDATE re-sends the alert on the next poll; no dedup beyond that,
|
||||
and reaction-polling API calls are unbounded.
|
||||
|
||||
---
|
||||
|
||||
## 4. nomos (agent gateway)
|
||||
|
||||
**Stakeholders:** engineers changing agent behavior, adding a task tool, or
|
||||
investigating a stuck/duplicated task. **Why this View earns its place:**
|
||||
this is the largest component by line count (4,681 lines across six files)
|
||||
and the one with the most active recent bug-fix history
|
||||
(`plans/2026-07-11-nomos-agent-code-review.md`,
|
||||
`plans/done/2026-07-14-post-fix-session-remainders.md`) — its internal
|
||||
shape is not obvious from outside.
|
||||
|
||||
### nomos — Internal structure
|
||||
|
||||
| File | Lines | Role |
|
||||
|---|---|---|
|
||||
| `cmd/nomos/main.go` | 914 | Gateway HTTP server (`:8092`), `/query`/`/chat`/`/sessions` routes, the hand-rolled Streamable-HTTP MCP client (`mcpClient`, per-session pooled) |
|
||||
| `cmd/nomos/store.go` | 1,472 | Persistence — sessions, messages, `logActivity` |
|
||||
| `cmd/nomos/agent.go` | 861 | The agentic loop itself; model config (L62-120); `maxIterations = 40` (L24, a **hard-coded constant**, not read from `nomos/config.yaml`'s `max_iterations: 15` — the two disagree, see status below) |
|
||||
| `cmd/nomos/tasks.go` | 416 | The five nomos-local task tools: `set_goal`, `propose_plan`, `update_plan_step`, `ask_operator`, `complete_task` — handled in-process, never forwarded to `internal/mcp` |
|
||||
| `cmd/nomos/continue.go` | 347 | The auto-continuation worker — polls `nomos_plan_executions` |
|
||||
| `cmd/nomos/assent.go` | 183 | `isAssent`/`isTypedConfirmation` — regex word-boundary matching (fixed 2026-07-11 after a false-positive bug where "yesterday" matched "yes") |
|
||||
|
||||
### nomos — Behavior specific to this component
|
||||
|
||||
The Task lifecycle and auto-continuation sequences are in
|
||||
[README.md §3.4](README.md#34-functional-flow--the-task-lifecycle-f3f4f5-packaged-for-a-human)
|
||||
and [§6.5](README.md#65-sequence--plan-auto-continuation-the-system-is-the-event-loop).
|
||||
Specific to this component: the LLM sees a **union of two tool sources** —
|
||||
the 33 tools fetched live from `api`'s `/mcp` endpoint via `tools/list`,
|
||||
plus the 5 local task tools in `tasks.go` — and `agent.go`'s per-call
|
||||
routing decides in-process versus forwarded with no visible seam to the
|
||||
model itself. A hidden `_session_id` is injected into forwarded calls on
|
||||
the wire (never in the model-visible arguments) so `internal/mcp/server.go`
|
||||
can scope assent/destructive windows per task.
|
||||
|
||||
### nomos — Interfaces this component owns
|
||||
|
||||
| Route | Auth |
|
||||
|---|---|
|
||||
| `GET /healthz` | none |
|
||||
| `POST /query` (structured tool call or a pointer to `/chat`) | **none** |
|
||||
| `POST /chat` (SSE, the real agentic loop) | **none** |
|
||||
| `GET/POST /sessions`, `/sessions/{id}` | **none** |
|
||||
|
||||
This entire interface is unauthenticated — full detail in
|
||||
[README.md §5.4](README.md#54-nomos-http-interface-cmdnomos-port-8092).
|
||||
Outbound: a pooled MCP client to `api`, and chat completions to OpenRouter
|
||||
(`data_collection: deny` pinned, default model `deepseek/deepseek-v4-pro`).
|
||||
|
||||
### nomos — Status and known issues
|
||||
|
||||
- **C1, the most consequential open gap involving this component**: zero
|
||||
authentication on the entire gateway, including the ability to grant
|
||||
chat-assent approvals with no credential check. Explicitly deferred by
|
||||
operator instruction, not an oversight — see
|
||||
[README.md §8.4](README.md#84-known-open-security-gaps).
|
||||
- `nomos/config.yaml`'s `max_iterations: 15` does not match the enforced
|
||||
Go constant (`40`) — one of the two is stale.
|
||||
- Dual `agent_activity` logging: both this component's `store.logActivity`
|
||||
and `internal/mcp/server.go`'s `withActivityLogging` (a different
|
||||
component) log the same forwarded tool call. Not confirmed whether this
|
||||
is an intentional two-sided audit trail or accidental duplication.
|
||||
|
||||
---
|
||||
|
||||
## 5. web control room
|
||||
|
||||
**Stakeholders:** the operator, directly; engineers changing the UI's data
|
||||
model or adding a page. **Why this View earns its place:** this is the only
|
||||
component with no server-side logic of its own — understanding it means
|
||||
understanding what it *doesn't* do (it is not the system of record for
|
||||
anything) as much as what it does.
|
||||
|
||||
### web control room — Internal structure
|
||||
|
||||
Nine pages under `web/src/pages/` (Svelte 5, hash-based routing, no router
|
||||
library):
|
||||
|
||||
| Page | Lines | Shows |
|
||||
|---|---|---|
|
||||
| `Overview.svelte` | 262 | Task dashboard — fleet/health/signal summary cards, the task list, entry point for launching a new chat |
|
||||
| `Chat.svelte` | 410 | The conversation UI — streaming, `TaskContextPanel`, tool-call rendering, inline approvals |
|
||||
| `Ops.svelte` | 240 | Approvals queue, recent activity/execution feed, approve/deny/cancel |
|
||||
| `Signals.svelte` | 171 | Alert/signal triage — severity filter, ack/resolve/mute |
|
||||
| `KnowledgeBase.svelte` | 263 | Entity browser — force-directed graph view and table view |
|
||||
| `Knowledge.svelte` | 186 | Free-text knowledge search |
|
||||
| `Learning.svelte` | 174 | Pattern/skill telemetry — the one page whose backing data source
|
||||
(`internal/learning`) is dormant per §7 below, so this page currently shows
|
||||
whatever accumulated before the engine stopped being called, not a live
|
||||
feed |
|
||||
| `Config.svelte` | 202 | Auth/connection screen — static bearer token or OIDC login |
|
||||
| `EntityDetail.svelte` | 7 | Thin wrapper, deep-link target |
|
||||
|
||||
Shared logic under `web/src/lib/`: `config.ts` (`fetchWithAuth`, the single
|
||||
wrapper every API call goes through), `oidc.ts`, `api.ts`, `tasks.ts`,
|
||||
`stores/events.ts` (the always-on SSE connection), plus task-specific
|
||||
components (`TaskContextPanel.svelte`, `GoalHeader.svelte`,
|
||||
`PlanProgress.svelte`, `OperatorQuestion.svelte`, `SessionGraph.svelte`).
|
||||
|
||||
### web control room — Behavior specific to this component
|
||||
|
||||
Two data-flow patterns, not one: most pages fetch REST on mount and
|
||||
re-fetch on a relevant SSE event; `Chat.svelte`'s `TaskContextPanel` is
|
||||
driven by the **always-on global event stream**
|
||||
(`web/src/lib/stores/events.ts`), not the per-turn chat SSE connection —
|
||||
deliberately, so the live context panel stays populated during
|
||||
server-side auto-continuation (§4 above) when no chat turn is actually
|
||||
open, and survives a tab reload.
|
||||
|
||||
### web control room — Interfaces this component owns
|
||||
|
||||
None inbound — it is a pure consumer of `internal/httpapi`'s REST and SSE
|
||||
interfaces (full catalog: [README.md §5](README.md#5-interfaces-icd)).
|
||||
`fetchWithAuth` resolves config per request rather than at import time, so
|
||||
the same build works same-origin (production, Vite dev proxy) or
|
||||
cross-origin (the Wails desktop webview, §8).
|
||||
|
||||
### web control room — Status and known issues
|
||||
|
||||
Standalone deploy, versioned and released independently of the `oikos`
|
||||
binary — see [README.md §4.5](README.md#45-build--release-artifacts) for
|
||||
why "deployed" means two different release cadences depending on whether
|
||||
you mean the container or the desktop app.
|
||||
|
||||
---
|
||||
|
||||
## 6. PostgreSQL/TimescaleDB
|
||||
|
||||
**Stakeholders:** anyone writing a migration, or reasoning about what
|
||||
"the system's source of truth" actually means (see
|
||||
[framework.md §2](framework.md#2-the-goal--system-and-model-made-concrete)
|
||||
for why that phrase needs disambiguating from the engineering Model).
|
||||
**Why this View earns its place:** every other component in this document
|
||||
either reads from or writes to this one; it is the only component every
|
||||
other component has in common.
|
||||
|
||||
### PostgreSQL/TimescaleDB — Internal structure
|
||||
|
||||
20 forward-only, idempotent migrations
|
||||
(`001_ontology.up.sql` … `020_session_reliability.up.sql`,
|
||||
[ADR-0008](../adr/0008-forward-only-migrations.md)). Four TimescaleDB
|
||||
hypertables, all created in `006_observability.up.sql`:
|
||||
`metric_samples`, `audit_log`, `events`, `agent_activity` — each with
|
||||
continuous aggregates and retention policies.
|
||||
|
||||
The recurring structural pattern across this schema, per
|
||||
[ADR-0014](../adr/0014-entity-model.md) §7: **dual entities** —
|
||||
`check_defs`, `signals`, `classifications`, `executions`, `feedback`,
|
||||
`patterns`, `skills`, `approvals`, `knowledge_entities`, and (as of
|
||||
migration 018) `agent_sessions`-as-`task` all have an
|
||||
`entity_id UUID PK REFERENCES entities(id)`, meaning every specialized row
|
||||
is simultaneously a node in the general entity graph — this is what lets
|
||||
`get_relations`/`get_blast_radius` work uniformly over signals, tasks, and
|
||||
infrastructure alike without a special case for each.
|
||||
|
||||
Partial unique indexes provide snapshot semantics without an application-
|
||||
level lock: `relationships` (current edges only), `signals` (one open
|
||||
signal per entity+kind), `patterns` (one per type+action).
|
||||
|
||||
### PostgreSQL/TimescaleDB — Behavior specific to this component
|
||||
|
||||
`008_event_notify.up.sql`'s `pg_notify` trigger on `events` INSERT is the
|
||||
entire mechanism behind [README.md's SSE interface](README.md#55-server-sent-events-internalhttpapissego)
|
||||
— the database, not the API process, is what decides an event happened;
|
||||
the API process is just a fan-out listener.
|
||||
|
||||
### PostgreSQL/TimescaleDB — Interfaces this component owns
|
||||
|
||||
Every Go component in this document connects directly (via `sqlc`-generated
|
||||
queries, `internal/db`) — there is no ORM abstraction layer and no
|
||||
component-specific access restriction beyond what each service's own
|
||||
Postgres role grants (notably: the learning engine's DB role has no grants
|
||||
on governance/autonomy tables, per
|
||||
[ADR-0006](../adr/0006-learning-proposal-only.md) — a structural guarantee
|
||||
that would matter the moment §7's dormant learning engine is wired back in).
|
||||
`seeds/*.yaml` + `oikos seed`/`oikos export` form the bootstrap/DR
|
||||
interface — the database can be regenerated from seeds, and seeds can be
|
||||
regenerated from the database.
|
||||
|
||||
### PostgreSQL/TimescaleDB — Status and known issues
|
||||
|
||||
A shared-Postgres single point of failure across every service is a
|
||||
documented residual risk in [ADR-0007](../adr/0007-threat-model.md), not a
|
||||
newly discovered one.
|
||||
|
||||
---
|
||||
|
||||
## 7. Dormant components
|
||||
|
||||
**Stakeholders:** anyone deciding whether to revive auto-act, or tempted to
|
||||
extend `internal/actuator`/`internal/learning` believing them to be the
|
||||
live implementation. **Why this View earns its place:** these are the two
|
||||
components most likely to mislead an engineer navigating by package name —
|
||||
both are substantial, well-written, and compile cleanly into the `oikos`
|
||||
binary, and neither runs.
|
||||
|
||||
### Dormant components — Internal structure
|
||||
|
||||
| File | Lines | Role, and why it's dormant |
|
||||
|---|---|---|
|
||||
| `internal/actuator/actuator.go` | 505 | `Run` (L24-41, 10s ticker), `processAutoActSignals`, kill-switch checks (L47, L69 — `getAutonomySetting` for `global.auto_act` and `never_auto_act.<slug>`), a real circuit breaker (L156-202, threshold + cooldown), advisory locking (L87-99) — **but the executor itself is a hardcoded stub**, `{"success": true, "message": "stub execution"}` (L124-137), and `Run()` is never called by `cmd/oikos/main.go` or any `docker-compose.yml` service |
|
||||
| `internal/actuator/ssh.go` | 291 | `ExecuteProcedure` (L126-230) — a fully-built step-by-step SSH runner with `classifySSHError` (L77-108: network/auth/timeout/remote), per-step timeouts, verify-step semantics. **Zero callers anywhere in the codebase.** |
|
||||
| `internal/learning/learning.go` | 183 | `Run` (L20-41, hourly ticker), `extractPatterns` (L45-81), `processGroup` (L83-169, Wilson lower-bound confidence, evidence≥5 ∧ confidence≥0.7 → `validated`, anomaly quarantine at >10 same-key events per pass) — algorithmically faithful to [ADR-0006](../adr/0006-learning-proposal-only.md), **never started by any process** |
|
||||
|
||||
### Why this matters more than "unused code"
|
||||
|
||||
`internal/actuator/actuator.go` is where the policy kill-switch
|
||||
(`global.auto_act`, `never_auto_act.*`) is actually checked in Go — the
|
||||
*only* other place is the dead `policy.ClassifySignal`. Reviving auto-act
|
||||
and fixing the kill-switch gap
|
||||
([README.md §8.1](README.md#81-the-kill-switch-gap-verified-most-important-finding-in-this-model))
|
||||
are, structurally, **the same piece of work** — whoever picks up
|
||||
`internal/actuator/actuator.go:47-69` is the person who also resolves
|
||||
REQ-DEC-5. This is stated explicitly here because it is not obvious from
|
||||
reading the Risk & Safety or Roadmap views in isolation; it only becomes
|
||||
visible once you've read this component's code.
|
||||
|
||||
### Dormant components — Status and known issues
|
||||
|
||||
This entire section *is* the known issue — see
|
||||
[README.md §9.1](README.md#91-the-north-star-general-gated-execution)
|
||||
("revive auto-act," the one item the `general-gated-execution` plan's own
|
||||
header still marks open) and
|
||||
[§9.4](README.md#94-suggested-next-steps-informational--not-a-commitment-not-a-plan)
|
||||
item 1 and 3 for the two decisions this leaves open: wire these back in, or
|
||||
delete them and correct the "Phase 3 — DONE" claim in
|
||||
`OIKOS.md`/ADR-0014 that currently overstates what's running.
|
||||
|
||||
---
|
||||
|
||||
## 8. Auxiliary components
|
||||
|
||||
Two small, single-purpose components that support deployment and
|
||||
packaging rather than decision logic — given lighter treatment here
|
||||
deliberately, since no Stakeholder identified in this document's other
|
||||
sections needs their internals to make a change elsewhere.
|
||||
|
||||
**`cmd/webhook`** (96 lines, one file) — the Gitea push-to-deploy receiver.
|
||||
Verifies `X-Hub-Signature-256` HMAC-SHA256 against `WEBHOOK_HMAC_SECRET`
|
||||
using `hmac.Equal`, responds `202` immediately, then runs
|
||||
`scripts/deploy.sh` asynchronously. Full deploy sequence:
|
||||
[README.md §4.2](README.md#42-deployment-topology-mac-mini-docker-compose)
|
||||
and §7.2.
|
||||
|
||||
**`cmd/desktop`** (784 lines) — the Wails-wrapped desktop shell. Bundles
|
||||
`web/dist` into a native binary, adds an OIDC login flow through a local
|
||||
HTTP server + system browser, and a `SaveConfig` bridge the web bundle
|
||||
calls when running inside the desktop webview (`web/src/lib/config.ts`'s
|
||||
`saveToDesktop()`). Contains no decision logic of its own — it is
|
||||
packaging for component 5 (§5 above), not a new component in the
|
||||
functional sense.
|
||||
|
||||
---
|
||||
|
||||
## Keeping this document current
|
||||
|
||||
The same discipline as README.md's closing note applies here, scoped to
|
||||
components: when a file listed in a "Internal structure" table is renamed,
|
||||
split, or gains a new responsibility, update that row. When a "Status and
|
||||
known issues" bullet is resolved, remove it — and check whether removing it
|
||||
also resolves an entry in
|
||||
[README.md §9](README.md#9-roadmap--traceability), since most of the
|
||||
findings here were first surfaced there and are repeated in this document
|
||||
for proximity to the code, not because they're independently tracked in
|
||||
two places.
|
||||
414
docs/mbse/framework.md
Normal file
414
docs/mbse/framework.md
Normal file
@@ -0,0 +1,414 @@
|
||||
# Oikos — MBSE Framework, Ontology & Viewpoints
|
||||
|
||||
> Companion to [the system Model](README.md). Where `README.md` **is** the
|
||||
> Model — the populated Views — this document is the **Framework**: the
|
||||
> template those Views were built from. It follows Jon Holt, *Systems
|
||||
> Engineering Demystified* (2nd ed., 2023), Ch. 2, "Model-Based Systems
|
||||
> Engineering," almost to the letter — the terms below (Model, View,
|
||||
> Viewpoint, Notation, Ontology, Framework, Process Set, Compliance) are
|
||||
> Holt's, not a paraphrase, because the whole point of adopting an Ontology
|
||||
> is to stop each document inventing its own vocabulary.
|
||||
|
||||
> **Holt's core claim, stated once so it doesn't need restating per
|
||||
> section:** *"When the Ontology and the Viewpoints are put together, they
|
||||
> form what is known as a Framework. A Framework is created as a template,
|
||||
> or blueprint, for a complete Model."* (Ch. 2, p. 40). Ontology is, in
|
||||
> Holt's words, "arguably the single most important part of MBSE as all of
|
||||
> the other elements that make up MBSE are ultimately traceable back to the
|
||||
> Ontology" (p. 40).
|
||||
|
||||
## 1. MBSE in a Slide — applied to Oikos
|
||||
|
||||
Holt's book converges the whole chapter into one diagram known across the
|
||||
Systems Engineering community as "MBSE in a slide" (Holt & Perry, 2019),
|
||||
extended with Implementation and Compliance. Below is that same structure
|
||||
with every box filled in for this specific repository, not left generic.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph APPROACH["APPROACH \n what must be produced, and how"]
|
||||
FW["Framework \n Ontology + Viewpoints \n = this document"]
|
||||
PS["Process Set \n ADRs, plans, this repo's\nreview/CI conventions"]
|
||||
end
|
||||
|
||||
subgraph GOAL["GOAL \n why any of this exists"]
|
||||
SYS["System \n the hubris homelab,\ngoverned by Oikos"]
|
||||
MDL["Model \n README.md \n the nine Views"]
|
||||
end
|
||||
|
||||
subgraph VIS["VISUALIZATION \n how it is communicated"]
|
||||
NOT["Notation \n Markdown + Mermaid"]
|
||||
DIA["Diagrams \n flowchart, stateDiagram,\nsequenceDiagram, classDiagram"]
|
||||
end
|
||||
|
||||
subgraph IMPL["IMPLEMENTATION"]
|
||||
TOOL["Tools \n git, a Markdown renderer,\nMermaid; no dedicated\nSysML tool"]
|
||||
end
|
||||
|
||||
subgraph COMP["COMPLIANCE"]
|
||||
BP["Best Practice \n ISO 42010 viewpoint and view\nterminology, informally aligned,\nnot certified"]
|
||||
end
|
||||
|
||||
FW --> MDL
|
||||
PS --> MDL
|
||||
MDL --> SYS
|
||||
NOT --> DIA
|
||||
DIA --> MDL
|
||||
TOOL --> NOT
|
||||
TOOL --> FW
|
||||
BP --> PS
|
||||
BP --> FW
|
||||
```
|
||||
|
||||
| Holt's concept | Generic definition (Ch. 2) | Oikos instantiation |
|
||||
|---|---|---|
|
||||
| **System** | The thing Systems Engineering exists to develop | The **hubris homelab** — hosts, LXCs, VMs, services, network — *and* the Oikos control plane that governs it. See note below on the reflexive boundary. |
|
||||
| **Model** | The abstraction of the System; the single source of truth for engineering knowledge about it | [README.md](README.md) — the nine-View system model |
|
||||
| **View** | A validated collection of information within the Model | Each of README.md's nine numbered sections |
|
||||
| **Viewpoint** | The template for a View — the stored answers to *which Stakeholders, why, what information* | §4 of this document — the Viewpoint catalog |
|
||||
| **Ontology** | The domain-specific language every Viewpoint's content is expressed in | §3 of this document, and — distinctively for this system — literally implemented in code as `seeds/ontology.yaml` |
|
||||
| **Notation** | The spoken/visual language used to communicate a View | Markdown prose + Mermaid diagrams (flowchart, stateDiagram-v2, sequenceDiagram, classDiagram) |
|
||||
| **Diagram** | One rendering of a View through a Notation's lens | Each Mermaid block in README.md |
|
||||
| **Framework** | Ontology + Viewpoints, together | This document |
|
||||
| **Process Set** | The steps for developing and using the Framework — the "how" | This repo's ADR process ([docs/adr/](../adr/)), design-doc process (`plans/`), and the research-then-write method used to build README.md (see its own header note on verified vs. per-research-pass findings) |
|
||||
| **Tool** | What implements the Notation and the Framework | Git + a Markdown/Mermaid renderer for the Notation; no dedicated MBSE tool enforces the Framework — see §6 for the honest gap this leaves |
|
||||
| **Compliance** | Demonstrating the approach meets external best practice | §5 |
|
||||
|
||||
**On the System's boundary being reflexive.** Most MBSE textbook examples
|
||||
model a System that is wholly separate from the engineering process
|
||||
describing it (a car, a radar). Oikos is not: the System being modeled
|
||||
*is* an autonomous control system, and the Model describing it (this
|
||||
documentation) sits outside a boundary that the System itself polices with
|
||||
its own internal "model" — the Postgres database, which
|
||||
[ADR-0003](../adr/0003-db-native-ontology-yaml-seeds.md) calls the runtime
|
||||
single source of truth. These are two different, non-competing uses of
|
||||
"model": the **engineering Model** (this doc set) is Holt's sense — a
|
||||
human-facing abstraction for realizing the System successfully. The
|
||||
**runtime database** is an operational sense — the System's own record of
|
||||
its current state, which the engineering Model *describes* but does not
|
||||
*replace*. Conflating the two would suggest this documentation is
|
||||
authoritative over live state, which it explicitly is not — README.md's
|
||||
own verification discipline (verified vs. per-research-pass) exists
|
||||
precisely because the engineering Model can drift from what the database
|
||||
and code actually do.
|
||||
|
||||
## 2. The Goal — System and Model, made concrete
|
||||
|
||||
**The System**, enumerated (this is "taking all the components of the
|
||||
system," per Holt's instruction that a valid View must be traceable to
|
||||
real Stakeholders and real information — not an abstract diagram):
|
||||
|
||||
| Component | Role |
|
||||
|---|---|
|
||||
| `oikos api` (`internal/httpapi`, `internal/mcp`) | REST + MCP server, the decision/execution gate |
|
||||
| `oikos scheduler` (`internal/scheduler`) | Observe loop — probes, signals |
|
||||
| `oikos notifier` (`internal/notifier`) | Approval delivery — Matrix, token issuance |
|
||||
| `nomos` (`cmd/nomos`) | The AI agent — MCP client, task/plan orchestration |
|
||||
| `web` (Svelte 5 SPA) | Control-room UI |
|
||||
| PostgreSQL/TimescaleDB | The System's own runtime source of truth |
|
||||
| The managed fleet | Hosts, LXCs, VMs, services under Oikos's governance |
|
||||
| `internal/actuator`, `internal/learning` | Compiled into the System, **not** currently part of its running behavior — see README.md §9.2 |
|
||||
|
||||
**The Model** is [README.md](README.md) in full: nine Views (Mission,
|
||||
Requirements, Functional Architecture, Physical Architecture, Interfaces,
|
||||
Behavior, Verification & Validation, Risk & Safety, Roadmap &
|
||||
Traceability). Per Holt's consistency test (p. 35): *"If there is a set of
|
||||
Views where each View is consistent with all other Views, then it is a
|
||||
Model. If there is a set of Views where each View is not consistent with
|
||||
all other Views, then it is data."* README.md's own cross-referencing
|
||||
(§8.1's kill-switch finding surfaced in §2's requirement, §3's function
|
||||
table, and §9's roadmap alike) is what keeps it a Model rather than nine
|
||||
unrelated documents.
|
||||
|
||||
## 3. The Ontology — Oikos's domain-specific language
|
||||
|
||||
Holt's Ontology has two jobs: it is the vocabulary every Viewpoint's
|
||||
content must be expressed in, and it is what makes Views from different
|
||||
parts of the Model comparable rather than coincidentally similar-looking.
|
||||
Oikos needs this at two levels, and — unusually for a Holt-style
|
||||
exercise — one of them was **already built in code**, not invented for
|
||||
this documentation pass.
|
||||
|
||||
### 3.1 Layer A — the SE meta-ontology (concepts used to talk *about* the Model)
|
||||
|
||||
This is the vocabulary this Framework document and README.md are written
|
||||
in. It is Holt's own vocabulary, restated as a concept diagram rather than
|
||||
prose, per his own example in the book (a Need Description View "visualized
|
||||
using UML Notation — specifically, a Diagram known as the class diagram,
|
||||
where each need is represented as a UML class," p. 38):
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class System
|
||||
class Model {
|
||||
+isSingleSourceOfTruth bool
|
||||
}
|
||||
class View {
|
||||
+stakeholders
|
||||
+value
|
||||
+information
|
||||
}
|
||||
class Viewpoint {
|
||||
+stakeholderQuestion
|
||||
+valueQuestion
|
||||
+informationQuestion
|
||||
}
|
||||
class Notation
|
||||
class Diagram
|
||||
class Ontology
|
||||
class Framework
|
||||
class ProcessSet
|
||||
class Stakeholder
|
||||
|
||||
Model "1" --> "1" System : abstracts
|
||||
Model "1" o-- "many" View : is made up of
|
||||
View ..|> Viewpoint : conforms to
|
||||
View "1" --> "1..many" Diagram : visualized through
|
||||
Diagram "many" --> "1" Notation : belongs to
|
||||
Viewpoint "many" --> "1" Ontology : traces terminology to
|
||||
Framework "1" o-- "1" Ontology : contains
|
||||
Framework "1" o-- "many" Viewpoint : contains
|
||||
Stakeholder "many" --> "many" Viewpoint : interested in
|
||||
```
|
||||
|
||||
### 3.2 Layer B — the Oikos domain ontology (the concepts inside the Views)
|
||||
|
||||
This is the part that already exists as running code, not something this
|
||||
documentation pass invented: `seeds/ontology.yaml` (952 lines, ingested by
|
||||
migration `001_ontology.up.sql` into the `entity_types`/`relationship_types`
|
||||
tables) *is* Holt's Ontology for this System — a machine-enforced
|
||||
domain-specific language that every Signal, Execution, Approval, and Task
|
||||
discussed anywhere in the Model traces back to. The full treatment — all 60
|
||||
entity types, the complete 47-relationship catalog (verified directly
|
||||
against the seed file; [ADR-0014](../adr/0014-entity-model.md) §1/§4
|
||||
records an earlier 2026-07-08 snapshot of 56 types and 34 relationships,
|
||||
since grown), and all six registered lifecycle state machines — is
|
||||
[ontology.md](ontology.md), Viewpoint 11 below. What follows here is the
|
||||
condensed version, sufficient only to make this section's point:
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class Entity {
|
||||
+UUID id
|
||||
+string slug
|
||||
+string type
|
||||
+string state
|
||||
}
|
||||
class ComputeEntity
|
||||
class Network
|
||||
class Container
|
||||
class Service
|
||||
class Agent
|
||||
class Signal {
|
||||
+string kind
|
||||
+string severity
|
||||
+string state
|
||||
}
|
||||
class Classification {
|
||||
+string riskClass
|
||||
+string route
|
||||
}
|
||||
class Execution {
|
||||
+string status
|
||||
}
|
||||
class Approval {
|
||||
+string status
|
||||
}
|
||||
class Pattern {
|
||||
+float confidence
|
||||
+string status
|
||||
}
|
||||
class Skill
|
||||
class Task {
|
||||
+string goal
|
||||
+string status
|
||||
+string outcome
|
||||
}
|
||||
class KnowledgeEntity
|
||||
|
||||
Entity <|-- ComputeEntity
|
||||
Entity <|-- Network
|
||||
Entity <|-- Container
|
||||
Entity <|-- Service
|
||||
Entity <|-- Agent
|
||||
Entity <|-- Signal
|
||||
Entity <|-- Classification
|
||||
Entity <|-- Execution
|
||||
Entity <|-- Approval
|
||||
Entity <|-- Pattern
|
||||
Entity <|-- Skill
|
||||
Entity <|-- Task
|
||||
Entity <|-- KnowledgeEntity
|
||||
|
||||
Signal "many" --> "1" Entity : about
|
||||
Classification "1" --> "1" Signal : classifies
|
||||
Classification "1" --> "1" Execution : precedes
|
||||
Execution "many" --> "1" Entity : targets
|
||||
Execution "many" --> "1" Agent : performs
|
||||
Approval "1" --> "1" Execution : decides
|
||||
Task "1" --> "many" Execution : requests via run
|
||||
Task "many" --> "many" Entity : involves
|
||||
KnowledgeEntity "many" --> "many" Entity : about
|
||||
KnowledgeEntity "many" --> "1" Task : outcome_of
|
||||
Pattern "1" --> "many" Execution : informed_by
|
||||
```
|
||||
|
||||
**This is the elegant accident worth naming plainly:** Oikos was not built
|
||||
by someone following Holt's method, yet its own architecture independently
|
||||
arrived at "the domain concepts are an Ontology, ingested once, and
|
||||
everything else traces back to it" — `seeds/ontology.yaml` → DB tables →
|
||||
every entity, signal, execution, and relationship in the system. That is
|
||||
Holt's Ontology principle, implemented as infrastructure rather than as a
|
||||
documentation artifact. The gap is not that the Ontology is missing; it's
|
||||
that, until this document, nothing had stated the correspondence between
|
||||
"the ontology" as oikos's engineers already use the word and "the Ontology"
|
||||
as Holt's MBSE method uses it. They are the same thing, at the domain
|
||||
layer.
|
||||
|
||||
### 3.3 Where Layer A and Layer B meet
|
||||
|
||||
Layer A (SE meta-ontology) is what makes README.md's Views *disciplined* —
|
||||
each one answers Holt's three questions (§4 below). Layer B (the Oikos
|
||||
domain ontology) is what makes README.md's Views *say the same thing
|
||||
consistently* — "risk class," "entity," "signal," and "execution" mean one
|
||||
thing throughout the whole Model because they mean one thing in
|
||||
`seeds/ontology.yaml`, not because nine separately-written documents
|
||||
happened to agree.
|
||||
|
||||
## 4. The Viewpoint Catalog — the Framework's template for Views
|
||||
|
||||
Per Holt (p. 39-40), a Viewpoint stores the answers to three questions —
|
||||
*which Stakeholders, why (what value), what information* — plus a fourth,
|
||||
*what Notation* — so that every View built from it is automatically
|
||||
consistent. Below is that template applied retroactively to each of
|
||||
README.md's nine Views, which is itself a useful audit: a View that can't
|
||||
honestly answer these four questions is not a valid View by Holt's own
|
||||
test (p. 35), and is a candidate for removal.
|
||||
|
||||
| Viewpoint | Which Stakeholders (§1.2) | Why — what value | What information | Notation |
|
||||
|---|---|---|---|---|
|
||||
| **1. Mission & Context** | Operator; future engineers/agents onboarding | Establishes why design choices elsewhere aren't arbitrary; sets the system boundary so later Views don't have to re-litigate scope | Mission statement, stakeholder table, mission drivers, boundary diagram, operational concept | Prose + flowchart |
|
||||
| **2. Requirements** | Operator; anyone implementing against a requirement | Traces every "the system shall" back to a source and forward to an implementation status, so intent and reality can be compared | Requirement ID, statement, source, status, organized by OODA phase + NFRs | Structured table |
|
||||
| **3. Functional Architecture** | Engineers extending decision/execution logic | Prevents the single most expensive mistake in this codebase — extending the wrong package because it has the right name | Function decomposition, function-to-component allocation (expected vs. actual owner), the `run` gate flow, the Task lifecycle | flowchart + allocation table |
|
||||
| **4. Physical Architecture** | Operator deploying/debugging the stack; on-call | Answers "what is actually running and where" independent of what the code *could* do | Component block diagram, deployment topology, trust zones, external couplings | flowchart |
|
||||
| **5. Interfaces (ICD)** | Anyone integrating a new MCP client, or reading/writing the API | A single place to find every tool/route/event without reading source | MCP tool catalog, REST groups, SSE event types, auth model | Tables |
|
||||
| **6. Behavior** | Engineers reasoning about a specific flow (an approval, a task) end to end | State machines and sequences are where "is this actually enforced" questions get answered, not functional prose | Signal/Execution/Approval state machines, `run`-gate sequence, auto-continuation sequence, Task sequence | stateDiagram-v2 + sequenceDiagram |
|
||||
| **7. Verification & Validation** | Operator deciding whether to trust a change; anyone auditing test coverage | Distinguishes "we checked this" from "we assume this" | CI pipeline, evals, health checks as continuous verification, deploy/rollback, explicit list of what's *not* covered | Prose + flowchart |
|
||||
| **8. Risk & Safety** | Operator; anyone reasoning about blast radius of agent autonomy | The single highest-consequence question this Model answers: what actually stops a bad action | Kill-switch gap finding, defense-in-depth layers, threat model, known open gaps, what's structurally guaranteed | Prose + tables |
|
||||
| **9. Roadmap & Traceability** | Operator planning what to fix next; future documentation maintainers | The authoritative status matrix every other View's ✅/⚠/❌ marker derives from | North-star status, code-real/dead-code/schema-only matrix, doc/code divergences, suggested next steps | Tables |
|
||||
| **10. Component** *(repeating Viewpoint — one View per component)* | An engineer about to change a specific package | Prevents extending the wrong implementation of something that exists twice (§9.2's dead-code/live-code pairs), or missing a known issue local to that package | Internal structure (files, key functions, file:line), behavior specific to that component, interfaces it owns, known issues — instantiated once per component in [components.md](components.md) | flowchart + tables |
|
||||
| **11. Ontology** *(repeating Viewpoint — one View per ontology facet)* | An engineer adding/changing an entity or relationship type; anyone checking whether a term used elsewhere in the Model traces back to something real | Is the check against Holt's own biggest MBSE risk (p. 35) applied to the Ontology itself — prevents treating the domain vocabulary as informal prose when it's actually a machine-enforced schema with real transition gates | Entity type hierarchy, relationship catalog, lifecycle state machines with their `requires:` gates, concrete population — instantiated as four Views in [ontology.md](ontology.md) | graph + stateDiagram-v2 + tables |
|
||||
|
||||
**Three things this table makes visible that weren't visible before:**
|
||||
|
||||
1. Every Viewpoint's "why" is stated in terms of a **decision or mistake it
|
||||
prevents**, not merely a topic it covers — closer to Holt's requirement
|
||||
that a View "must add value" (p. 35) than a topic-based table of
|
||||
contents would be.
|
||||
2. Viewpoints 10 and 11 are structurally different from 1-9: each is a
|
||||
**repeating Viewpoint** — one template, instantiated multiple times.
|
||||
Viewpoint 10 produces eight Views, once per component
|
||||
([components.md](components.md)); Viewpoint 11 produces four, once per
|
||||
ontology facet ([ontology.md](ontology.md)). Holt's method doesn't
|
||||
forbid this; a Viewpoint is a template, and nothing says a template can
|
||||
only be used once.
|
||||
3. There is still no Viewpoint in this catalog for "document every class
|
||||
exhaustively regardless of whether anyone asked" — Viewpoints 10 and 11
|
||||
are scoped to named, narrow Stakeholder questions ("an engineer about to
|
||||
change this component," "an engineer adding a new type"), not a blanket
|
||||
documentation mandate.
|
||||
Per Holt's own worked example (the Need Description View, p. 38-39), a
|
||||
collection of information that can't name an interested Stakeholder is
|
||||
not a View; it would just be generated documentation nobody reads,
|
||||
which is the exact failure mode Holt calls out as the biggest risk in
|
||||
adopting MBSE (p. 35).
|
||||
|
||||
## 5. Compliance
|
||||
|
||||
Holt names three categories of best-practice source (p. 46) a Framework
|
||||
can be checked against. Being direct about which apply here and which
|
||||
don't, rather than implying certification that doesn't exist:
|
||||
|
||||
| Category | Holt's examples | Oikos's position |
|
||||
|---|---|---|
|
||||
| **Process-based standards** (how work is done) | ISO 15288 | Not formally adopted. This repo's own process conventions (ADRs, `plans/`, PR review) are the de facto Process Set — informally rigorous, not standards-mapped. |
|
||||
| **Framework-based standards** (what information is produced) | ISO 42010, MODAF, DoDAF, NAF, UAF, Zachman | **Informally aligned, not certified.** This Framework borrows ISO 42010's Viewpoint/View vocabulary (which Holt's own method is built on) but has not been checked against the standard's actual conformance clauses. Say this plainly rather than imply an audit that hasn't happened. |
|
||||
| **Application-based standards** (domain-specific: safety, security, usability) | — | Partially present in spirit: [Risk & Safety](README.md#8-risk--safety) documents a real threat model and known gaps, but there is no adopted external security standard (e.g., no formal threat-modeling framework like STRIDE was used — the threat model in ADR-0007 is bespoke). |
|
||||
|
||||
The honest summary: this Framework's compliance posture is **methodological
|
||||
alignment with ISO 42010's core idea (Stakeholders → concerns → Viewpoints
|
||||
→ Views), not standards certification.** Claiming more than that would
|
||||
itself violate the documentation set's own governing discipline (state
|
||||
verified findings as verified, not aspirational ones as achieved).
|
||||
|
||||
## 6. Tools — Implementation, and its honest limit
|
||||
|
||||
Holt is specific that a good MBSE tool does two things: it *implements the
|
||||
Notation* (enforces SysML's syntax/semantics the way a word processor
|
||||
enforces spelling) and it *implements the Framework* (has the Ontology and
|
||||
Viewpoints "programmed into it" as a profile, p. 44-45).
|
||||
|
||||
Neither is true here, and it matters to say so:
|
||||
|
||||
- **Notation tooling**: Markdown + Mermaid, rendered by GitHub/a Markdown
|
||||
viewer. Mermaid's flowchart/stateDiagram/sequenceDiagram/classDiagram
|
||||
grammars are enforced (a malformed diagram fails to render — as
|
||||
happened once already in this documentation effort and was fixed), but
|
||||
there is no semantic check that, say, a state machine diagram in
|
||||
[§6](README.md#6-behavior) actually matches the Go code's real
|
||||
transitions. That check was done by hand, once, for this pass — it will
|
||||
drift the moment the code changes and nobody re-verifies it.
|
||||
- **Framework tooling**: there is no tool with this Ontology or these
|
||||
Viewpoints "programmed in." Nothing prevents a future edit to README.md
|
||||
from adding a View that fails Holt's three-question test, or from
|
||||
introducing a term that doesn't trace back to `seeds/ontology.yaml`.
|
||||
The only enforcement mechanism is a human (or an agent) re-reading this
|
||||
Framework document before extending the Model — which is precisely why
|
||||
this document needed to exist as a separate, explicit artifact rather
|
||||
than staying implicit in how README.md happened to get organized.
|
||||
|
||||
## 7. Process Set — how this Framework is developed and used
|
||||
|
||||
Holt separates Framework (what) from Process Set (how) specifically so
|
||||
that different projects can share one Framework under different levels of
|
||||
rigor (p. 41-42). For this repository, the Process Set is:
|
||||
|
||||
1. **Establishing a new Viewpoint**: propose it here in §4, answering all
|
||||
four questions before writing the View it justifies. If it can't answer
|
||||
them, per Holt's own rule (p. 35), it doesn't get written.
|
||||
2. **Extending the Ontology**: changes to `seeds/ontology.yaml` are the
|
||||
authoritative act — this document's §3.2 is a description of that file,
|
||||
not an independent source, and must be re-derived from it if it drifts.
|
||||
3. **Updating a View**: per README.md's own closing section ("Keeping this
|
||||
model current"), a code change updates the View whose Viewpoint claims
|
||||
that information, and — if it resolves or introduces a finding in
|
||||
[§9 Roadmap & Traceability](README.md#9-roadmap--traceability) — that
|
||||
matrix is updated in the same pass.
|
||||
4. **Compliance review**: informal, human-in-the-loop (§5) — there is no
|
||||
scheduled re-audit; drift is caught opportunistically, the same way the
|
||||
kill-switch gap in [§8.1](README.md#81-the-kill-switch-gap-verified-most-important-finding-in-this-model)
|
||||
was caught by direct verification during a documentation pass rather
|
||||
than by a standing process designed to catch it.
|
||||
|
||||
## 8. Relationship between this Framework and the Model
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
ONT["Ontology \n seeds and ADR-0014"] --> FW["Framework \n this document"]
|
||||
VP["Viewpoint catalog \n Section 4 of this document"] --> FW
|
||||
FW --> MDL["Model \n README.md, Viewpoints 1 to 9"]
|
||||
FW --> CV["Model \n components.md, Viewpoint 10\nrepeated per component"]
|
||||
FW --> OV["Model \n ontology.md, Viewpoint 11\nrepeated per ontology facet"]
|
||||
MDL --> V1["View 1..9"]
|
||||
CV --> V2["View 10a..10h"]
|
||||
OV --> V3["View 11a..11d"]
|
||||
```
|
||||
|
||||
Read [README.md](README.md) for the Model's concern-based Views,
|
||||
[components.md](components.md) for its component-based Views, and
|
||||
[ontology.md](ontology.md) for the Ontology's own full treatment (the
|
||||
sketch in §3 above is deliberately condensed). Read this document when you
|
||||
are deciding whether a new View belongs in any of the three, when a term in
|
||||
the Model feels like it's drifted from what `seeds/ontology.yaml` actually
|
||||
defines, or when onboarding someone who needs to understand not just *what
|
||||
the system is* but *why this documentation is shaped the way it is*.
|
||||
444
docs/mbse/ontology.md
Normal file
444
docs/mbse/ontology.md
Normal file
@@ -0,0 +1,444 @@
|
||||
# Oikos — Ontology Views
|
||||
|
||||
> Companion to [the Framework](framework.md), [the Model](README.md), and
|
||||
> [the Component Views](components.md). Holt calls Ontology "arguably the
|
||||
> single most important part of MBSE, as all of the other elements that
|
||||
> make up MBSE are ultimately traceable back to [it]" (*Systems Engineering
|
||||
> Demystified*, 2nd ed., Ch. 2, p. 40). [framework.md §3](framework.md#3-the-ontology--oikoss-domain-specific-language)
|
||||
> sketched this in condensed form (13 classes) to make one point: Oikos's
|
||||
> domain ontology already exists as running code, not documentation. This
|
||||
> document is the fuller treatment that sketch promised — a repeating
|
||||
> Viewpoint (registered as Viewpoint 11 in
|
||||
> [framework.md §4](framework.md#4-the-viewpoint-catalog--the-frameworks-template-for-views)),
|
||||
> instantiated as four Views below.
|
||||
|
||||
**Every fact in this document was read directly from `seeds/ontology.yaml`
|
||||
during this pass** (not carried over from ADR-0014's summary, though it is
|
||||
cross-checked against it) — where the two disagree, that disagreement is
|
||||
itself reported as a finding, not silently reconciled.
|
||||
|
||||
**Stakeholders for all four Views below:** engineers adding a new entity or
|
||||
relationship type, anyone reasoning about whether a lifecycle transition is
|
||||
actually gated or just documented, and anyone deciding whether a term used
|
||||
elsewhere in this documentation set means what they think it means.
|
||||
**Why they earn their place:** every Viewpoint in
|
||||
[framework.md §4](framework.md#4-the-viewpoint-catalog--the-frameworks-template-for-views)
|
||||
"traces terminology to the Ontology" (§3.1 of that document) — these four
|
||||
Views are where that tracing actually terminates. **Notation:** tables
|
||||
(the source data), Mermaid `graph`/`stateDiagram-v2` (the structure).
|
||||
|
||||
## Contents
|
||||
|
||||
| View | Answers |
|
||||
|---|---|
|
||||
| [11a. Entity Type Hierarchy](#11a-entity-type-hierarchy) | What can exist, and how is it classified? |
|
||||
| [11b. Relationship Catalog](#11b-relationship-catalog) | How can two entities be connected, and with what multiplicity? |
|
||||
| [11c. Lifecycle State Machines](#11c-lifecycle-state-machines) | What states can a governed entity be in, and what gates each transition? |
|
||||
| [11d. Concrete Population](#11d-concrete-population) | What's actually instantiated, versus merely possible? |
|
||||
|
||||
---
|
||||
|
||||
## 11a. Entity Type Hierarchy
|
||||
|
||||
60 entity types, 5 abstract (cannot be instantiated directly — they exist
|
||||
only as polymorphic relationship endpoints and `is-a` parents), organized
|
||||
by `domain:` (8 values) and `layer:` (4 values: meta, infrastructure,
|
||||
governance, cognition).
|
||||
|
||||
| Layer | Domains it contains | Entity type count |
|
||||
|---|---|---|
|
||||
| `meta` | meta | 1 (`entity`, the abstract root) |
|
||||
| `infrastructure` | physical, compute, network, storage, software, external | 40 |
|
||||
| `governance` | identity | 7 |
|
||||
| `cognition` | cognition | 12 |
|
||||
|
||||
| Domain | Count | Abstract types in this domain |
|
||||
|---|---|---|
|
||||
| compute | 11 | `compute-entity`, `machine`, `container` |
|
||||
| network | 10 | `network` |
|
||||
| cognition | 12 | *(none)* |
|
||||
| identity | 7 | *(none)* |
|
||||
| software | 7 | *(none)* |
|
||||
| external | 4 | *(none)* |
|
||||
| storage | 4 | *(none)* |
|
||||
| physical | 4 | *(none)* |
|
||||
| meta | 1 | `entity` |
|
||||
|
||||
One 60-node diagram doesn't fit on a screen and, worse, tempts you to fall
|
||||
back on subgraph grouping instead of explicit edges for the flatter
|
||||
domains — which is what an earlier version of this section did: most leaf
|
||||
types were boxed together visually but had no drawn `-->` from `entity` at
|
||||
all. Split by domain instead, every type below has an explicit parent
|
||||
edge — nothing is implied by proximity alone.
|
||||
|
||||
### Layer overview
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
entity["entity — abstract root\nlayer: meta"] --> INFRA["infrastructure layer\n40 types — physical, compute, network,\nstorage, software, external"]
|
||||
entity --> GOV["governance layer\n7 types — identity domain"]
|
||||
entity --> COG["cognition layer\n12 types"]
|
||||
```
|
||||
|
||||
### Domain: physical (4 types, all flat)
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
entity["entity"] --> site
|
||||
entity --> ups
|
||||
entity --> sensor
|
||||
entity --> peripheral
|
||||
```
|
||||
|
||||
### Domain: compute (11 types — the deepest nesting in the Ontology)
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
entity["entity"] --> ce["compute-entity — abstract"]
|
||||
entity --> hypervisor
|
||||
ce --> machine["machine — abstract"]
|
||||
ce --> vm
|
||||
ce --> container["container — abstract"]
|
||||
machine --> proxmoxhost["proxmox-host"]
|
||||
machine --> standalone["standalone-server"]
|
||||
machine --> workstation
|
||||
machine --> appliance
|
||||
container --> lxc
|
||||
container --> dockercontainer["docker-container"]
|
||||
```
|
||||
|
||||
### Domain: network (10 types)
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
entity["entity"] --> net["network — abstract"]
|
||||
entity --> netiface["network-interface"]
|
||||
entity --> dnszone["dns-zone"]
|
||||
entity --> dnsrecord["dns-record"]
|
||||
entity --> ingress["ingress-route"]
|
||||
entity --> certificate
|
||||
entity --> firewallrule["firewall-rule"]
|
||||
net --> lan
|
||||
net --> mesh
|
||||
net --> vlan
|
||||
```
|
||||
|
||||
### Domain: storage (4 types, all flat)
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
entity["entity"] --> storagepool["storage-pool"]
|
||||
entity --> volume
|
||||
entity --> backuptarget["backup-target"]
|
||||
entity --> dataset
|
||||
```
|
||||
|
||||
`storage-pool`/`volume`/`dataset` look like they should nest (a pool
|
||||
*contains* volumes, a volume *holds* datasets) — they don't, in the type
|
||||
hierarchy. That containment is a **relationship** (`contains`,
|
||||
`holds-dataset`, [§11b](#11b-relationship-catalog)), not an `is-a` parent.
|
||||
Worth stating plainly since the two are easy to conflate: `parent:` says
|
||||
"this is a kind of that"; a relationship says "this instance is connected
|
||||
to that instance." Storage is the domain where the difference is most
|
||||
visible.
|
||||
|
||||
### Domain: software (7 types, all flat)
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
entity["entity"] --> service
|
||||
entity --> application
|
||||
entity --> configrepo["config-repo"]
|
||||
entity --> deploypipeline["deploy-pipeline"]
|
||||
entity --> packageset["package-set"]
|
||||
entity --> cluster
|
||||
entity --> composestack["compose-stack"]
|
||||
```
|
||||
|
||||
### Domain: external (4 types, all flat)
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
entity["entity"] --> domainreg["domain-registration"]
|
||||
entity --> cloudservice["cloud-service"]
|
||||
entity --> isplink["isp-link"]
|
||||
entity --> vendordep["vendor-dependency"]
|
||||
```
|
||||
|
||||
### Domain: identity (governance layer, 7 types, all flat)
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
entity["entity"] --> person
|
||||
entity --> agent
|
||||
entity --> idp["identity-provider"]
|
||||
entity --> account
|
||||
entity --> secret
|
||||
entity --> key
|
||||
entity --> accessgrant["access-grant"]
|
||||
```
|
||||
|
||||
### Domain: cognition (12 types, all flat — the domain the agent's own logic runs on)
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
entity["entity"] --> check
|
||||
entity --> signal
|
||||
entity --> classification
|
||||
entity --> execution
|
||||
entity --> feedback
|
||||
entity --> pattern
|
||||
entity --> skill
|
||||
entity --> approval
|
||||
entity --> document
|
||||
entity --> runbook
|
||||
entity --> investigation
|
||||
entity --> task["task — added after ADR-0014"]
|
||||
```
|
||||
|
||||
Every one of the 60 types above is a direct or indirect child of `entity`;
|
||||
none is disconnected. The full flat list — every type with its exact
|
||||
`parent:` — lives in `seeds/ontology.yaml` directly; reproducing all 60
|
||||
rows as a table here would duplicate this section rather than clarify it.
|
||||
|
||||
**Finding: `task` is new since ADR-0014.** ADR-0014 (2026-07-08) documents
|
||||
56 entity types under a hierarchy diagram that does not include `task` —
|
||||
four fewer than the 60 verified here, meaning more than just `task` was
|
||||
added in the interim (`task` accounts for one of the four) —
|
||||
[README.md §1.5](README.md#15-operational-concept--the-ooda-loop) and
|
||||
[framework.md §2](framework.md#2-the-goal--system-and-model-made-concrete)
|
||||
both describe the Task model as a 2026-07-11 addition
|
||||
(`plans/done/2026-07-11-goal-oriented-chat-control-panel.md`), after
|
||||
ADR-0014 was written. `task` is now entity type #60, `domain: cognition`,
|
||||
`layer: cognition`, described in the seed as *"A goal-structured unit of
|
||||
agent work — one chat/session elevated to a task with a plan, lifecycle
|
||||
status, and outcome."* This is exactly what Holt's Ontology principle
|
||||
predicts: a new concept in the Model
|
||||
([the Task lifecycle View](README.md#34-functional-flow--the-task-lifecycle-f3f4f5-packaged-for-a-human))
|
||||
required a new term in the Ontology before it could be modeled
|
||||
consistently — and the term was in fact added, not left implicit.
|
||||
|
||||
## 11b. Relationship Catalog
|
||||
|
||||
**47 relationship types**, each with a fixed `source → target` type pair
|
||||
and a cardinality. This is the complete, current catalog — not the
|
||||
5 illustrative example-graphs ADR-0014 used to gesture at a smaller set.
|
||||
|
||||
**Finding: this catalog has grown since ADR-0014.** ADR-0014 (2026-07-08)
|
||||
titles its equivalent section "The Edge Catalog (34 edges)." Verified
|
||||
directly against `seeds/ontology.yaml` during this pass: **47** relationship
|
||||
types exist today — 13 more than ADR-0014 recorded. This is expected drift
|
||||
over an 8-day span of active development (the Task model alone plausibly
|
||||
added `involves`; `part-of` supports the `compose-stack` grouping), not a
|
||||
documentation error — ADR-0014 is a point-in-time record and is not edited
|
||||
after acceptance, per this repo's own convention
|
||||
([docs/adr/README.md](../adr/README.md)). It is reported here so nobody
|
||||
treats ADR-0014's count as current.
|
||||
|
||||
| Relationship | Source → Target | Cardinality |
|
||||
|---|---|---|
|
||||
| `hosts` | machine → compute-entity | one-to-many |
|
||||
| `runs-hypervisor` | machine → hypervisor | one-to-one |
|
||||
| `member-of` | proxmox-host → cluster | many-to-one |
|
||||
| `part-of` | docker-container → compose-stack | many-to-one |
|
||||
| `provides` | compute-entity → service | one-to-many |
|
||||
| `runs` | service → application | one-to-many |
|
||||
| `configured-by` | entity → config-repo | many-to-one |
|
||||
| `deploys-to` | deploy-pipeline → entity | many-to-one |
|
||||
| `routes-to` | ingress-route → service | many-to-one |
|
||||
| `secured-by` | ingress-route → identity-provider | many-to-one |
|
||||
| `uses-certificate` | ingress-route → certificate | many-to-one |
|
||||
| `authenticates-via` | service → identity-provider | many-to-one |
|
||||
| `in-zone` | dns-record → dns-zone | many-to-one |
|
||||
| `resolves-to` | dns-record → entity | many-to-one |
|
||||
| `depends-on` | service → service | many-to-many |
|
||||
| `connects-via` | compute-entity → network | many-to-many |
|
||||
| `has-interface` | compute-entity → network-interface | one-to-many |
|
||||
| `interface-on` | network-interface → network | many-to-one |
|
||||
| `mounts` | compute-entity → volume | many-to-many |
|
||||
| `stores-on` | compute-entity → storage-pool | many-to-many |
|
||||
| `contains` | storage-pool → volume | one-to-many |
|
||||
| `holds-dataset` | volume → dataset | one-to-many |
|
||||
| `backs-up-to` | entity → backup-target | many-to-many |
|
||||
| `powered-by` | machine → ups | many-to-one |
|
||||
| `located-at` | machine → site | many-to-one |
|
||||
| `registered-with` | domain-registration → vendor-dependency | many-to-one |
|
||||
| `owns` | person → agent | one-to-many |
|
||||
| `authenticates` | identity-provider → person | one-to-many |
|
||||
| `holds-grant` | agent → access-grant | one-to-many |
|
||||
| `grants` | access-grant → secret | many-to-one |
|
||||
| `can-decrypt` | compute-entity → secret | many-to-many |
|
||||
| `checks` | check → entity | many-to-one |
|
||||
| `raises` | check → signal | one-to-many |
|
||||
| `about` | entity → entity | many-to-many |
|
||||
| `classifies` | classification → signal | many-to-one |
|
||||
| `precedes` | classification → execution | one-to-one |
|
||||
| `targets` | execution → entity | many-to-one |
|
||||
| `requires-approval` | execution → approval | one-to-one |
|
||||
| `performs` | agent → execution | one-to-many |
|
||||
| `decides` | person → approval | one-to-many |
|
||||
| `produces` | execution → feedback | one-to-one |
|
||||
| `contributes-to` | feedback → pattern | many-to-many |
|
||||
| `informs` | pattern → skill | many-to-one |
|
||||
| `guides` | skill → classification | one-to-many |
|
||||
| `documents` | document → entity | many-to-one |
|
||||
| `involves` | task → entity | many-to-many |
|
||||
| `procedure-for` | runbook → entity | many-to-many |
|
||||
|
||||
Grouped by theme, the same 47 rows read as five coherent sub-ontologies —
|
||||
this is the grouping ADR-0014 used, now complete rather than illustrative:
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Cognition["Cognition — the OODA edges"]
|
||||
CK["check"] -->|raises| SG["signal"]
|
||||
CK -->|checks| EN["entity"]
|
||||
CL["classification"] -->|classifies| SG
|
||||
CL -->|precedes| EX["execution"]
|
||||
EX -->|targets| EN
|
||||
EX -->|requires-approval| AP["approval"]
|
||||
AG["agent"] -->|performs| EX
|
||||
PR["person"] -->|decides| AP
|
||||
EX -->|produces| FB["feedback"]
|
||||
FB -->|contributes-to| PT["pattern"]
|
||||
PT -->|informs| SK["skill"]
|
||||
SK -->|guides| CL
|
||||
TK["task"] -->|involves| EN
|
||||
DC["document"] -->|documents| EN
|
||||
RB["runbook"] -->|procedure-for| EN
|
||||
end
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Governance["Governance — identity and access"]
|
||||
P["person"] -->|owns| A["agent"]
|
||||
IDP["identity-provider"] -->|authenticates| P
|
||||
A -->|holds-grant| AG["access-grant"]
|
||||
AG -->|grants| S["secret"]
|
||||
CE["compute-entity"] -->|can-decrypt| S
|
||||
end
|
||||
```
|
||||
|
||||
The remaining three groups (Infrastructure Topology, Network, Service
|
||||
Dependencies) are unchanged in shape from
|
||||
[ADR-0014 §4](../adr/0014-entity-model.md) — that ADR's diagrams for those
|
||||
three are still an accurate illustrative subset of the table above; only
|
||||
the Cognition and Governance groups gained new edges (`involves`,
|
||||
`part-of`) worth re-drawing.
|
||||
|
||||
## 11c. Lifecycle State Machines
|
||||
|
||||
Six lifecycles are formally registered in `seeds/ontology.yaml`'s
|
||||
`lifecycles:` block, each a named state machine with `states`,
|
||||
`default_state`, `terminal_states`, and per-transition `requires:` — named
|
||||
checks that [internal/ontology](../../internal/ontology) implements in Go.
|
||||
This is the mechanism, not just the diagram: a transition without a
|
||||
satisfied `requires:` check is refused at the code level, for these six
|
||||
types.
|
||||
|
||||
**Refinement to README.md's Behavior view.** The `execution` lifecycle as
|
||||
registered here has **13 states**, including a `verifying` state distinct
|
||||
from `executing`, and a recovery transition `timed_out → verifying`
|
||||
("check if the command completed anyway" — the seed's own comment).
|
||||
[README.md §6.2](README.md#62-execution-state-machine--schema-defined-convention-enforced)'s
|
||||
Execution state diagram previously omitted `verifying` as a separate
|
||||
state and has been corrected there to match; the diagram below is the
|
||||
ontology-accurate version and the two now agree.
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> proposed
|
||||
proposed --> approved: operator-approval
|
||||
proposed --> auto_approved: autonomy-allows
|
||||
proposed --> denied
|
||||
approved --> executing: approval-token-valid
|
||||
approved --> expired: approval-ttl-elapsed
|
||||
auto_approved --> executing
|
||||
executing --> verified: verification-passed
|
||||
executing --> failed
|
||||
executing --> timed_out
|
||||
executing --> cancelled: operator-abort
|
||||
timed_out --> verifying: check if it finished anyway
|
||||
verifying --> verified: verification-passed
|
||||
verifying --> failed
|
||||
failed --> rolled_back: rollback-procedure-exists
|
||||
failed --> rollback_failed
|
||||
verified --> [*]
|
||||
denied --> [*]
|
||||
expired --> [*]
|
||||
cancelled --> [*]
|
||||
rolled_back --> [*]
|
||||
rollback_failed --> [*]
|
||||
```
|
||||
|
||||
The other five, with their `requires:` gates named explicitly (abbreviated
|
||||
where a transition has no requirement):
|
||||
|
||||
| Lifecycle | States | Terminal | Notable gated transition |
|
||||
|---|---|---|---|
|
||||
| `infrastructure` | planned, provisioning, active, migrating, failed, deprecated, destroyed | destroyed | `deprecated → destroyed` requires **five** checks at once: `backups-verified`, `secrets-revoked-and-rekeyed`, `ingress-and-dns-removed`, `no-inbound-edges`, `archaeology-entry` — the strictest single transition in the entire Ontology |
|
||||
| `signal` | raised, acknowledged, acting, muted, resolved, failed | resolved | `acknowledged → acting` requires `classification-exists` — the formal link between Orient and Decide, real in the Ontology even though [Roadmap §9.2](README.md#92-code-real--dead-code--schema-only-matrix) finds the classifier that would create that classification is dead code |
|
||||
| `approval` | pending, approved, denied, expired, revoked | denied, expired, revoked | `pending → approved` requires `token-verified`; `approved → revoked` requires `not-yet-executing` — you cannot revoke an approval whose action has already started |
|
||||
| `pattern` | hypothesized, validated, active, deprecated, invalidated | deprecated, invalidated | `hypothesized → validated` requires `evidence-count-5plus` **and** `confidence-0.7plus` jointly — matches [ADR-0006](../adr/0006-learning-proposal-only.md)'s Wilson-bound description exactly; `validated → active` requires `operator-approval`, annotated in the seed itself as *"S4: never automatic"* |
|
||||
| `skill` | drafted, tested, active, refined, failed, deprecated | deprecated | `tested → active` and `refined → active` both require `operator-approval` — a skill can be authored and tested autonomously but never self-promotes to active |
|
||||
|
||||
**Finding: `task` has no registered lifecycle.** The `task` entity type
|
||||
(§11a) has a real, documented behavior —
|
||||
[README.md §3.4](README.md#34-functional-flow--the-task-lifecycle-f3f4f5-packaged-for-a-human)
|
||||
shows `planning → awaiting_approval → executing ⇄ awaiting_input → done`/`failed`
|
||||
as a state diagram, and it is enforced in application code (the
|
||||
`agent_sessions.status` column, checked in `cmd/nomos`). But
|
||||
`seeds/ontology.yaml`'s `lifecycles:` block registers only the six
|
||||
machines above — there is no `task:` entry alongside `infrastructure`,
|
||||
`signal`, `execution`, `approval`, `pattern`, `skill`. Practically: the
|
||||
five other governed types get their transition-gating for free from the
|
||||
shared `internal/ontology` machinery (per named `requires:` checks); the
|
||||
Task lifecycle is instead hand-coded in `cmd/nomos`'s Go logic, a
|
||||
structurally different (and unaudited-by-the-shared-mechanism) enforcement
|
||||
path for what is, in every other respect, a first-class Ontology citizen.
|
||||
This is a gap worth a deliberate decision — register `task` formally, or
|
||||
document explicitly that Task's lifecycle is intentionally
|
||||
application-layer rather than Ontology-layer — not an oversight this
|
||||
document is fixing by writing it down.
|
||||
|
||||
## 11d. Concrete Population
|
||||
|
||||
What's actually instantiated versus merely possible in the type system —
|
||||
per [ADR-0014](../adr/0014-entity-model.md) §1, **not independently
|
||||
re-counted against the live database during this pass** (that would
|
||||
require DB access this documentation effort didn't use; the figures below
|
||||
are ADR-0014's, dated 2026-07-08, and should be treated as illustrative of
|
||||
shape rather than a current census):
|
||||
|
||||
| Type | Count (as of ADR-0014) | Examples |
|
||||
|---|---|---|
|
||||
| `lxc` | 19 | jellyfin, caddy, dns, gitea, nextcloud, matrix |
|
||||
| `service` | 25 | caddy, authentik, dns, jellyfin, paperless, matrix |
|
||||
| `ingress-route` | 21 | `*.hubris.network` |
|
||||
| `config-repo` | 6 | caddy-conf, gitea-customizations, mule-image |
|
||||
| `proxmox-host` | 2 | hubris, strong |
|
||||
| `workstation` | 2 | mac-mini, republic-laptop |
|
||||
| `standalone-server` | 1 | netbird-vps |
|
||||
| `vm` | 2 | zimaos, haos |
|
||||
| `storage-pool` | 3 | local-lvm-hubris, library-hubris, ludo-lvm |
|
||||
| `volume` | 2 | library, media-local |
|
||||
|
||||
**Why this View matters despite being the least current one here:** it is
|
||||
the check against over-abstraction Holt warns about (p. 35) — an Ontology
|
||||
with 60 types and 47 relationships is only worth having if real entities
|
||||
actually populate a meaningful fraction of it. 88 active entities across
|
||||
roughly a dozen concrete types (out of 55 non-abstract types) is a
|
||||
reasonable population for a homelab of this size; a future re-audit of this
|
||||
specific View is a cheap, well-scoped follow-up (query `entities GROUP BY
|
||||
type`) that this pass explicitly did not do, rather than silently assuming
|
||||
ADR-0014's numbers still hold.
|
||||
|
||||
## Keeping this document current
|
||||
|
||||
Re-derive §11a-11c directly from `seeds/ontology.yaml` whenever it changes
|
||||
— these three Views are transcriptions of that file's structure, not
|
||||
independent judgment, so they go stale the moment the file changes and
|
||||
nobody re-runs the extraction. §11d is the one View here that was already
|
||||
known to be a point-in-time snapshot when written; re-verify it against
|
||||
live DB state before relying on it for a capacity or audit decision.
|
||||
18
docs/operations/README.md
Normal file
18
docs/operations/README.md
Normal file
@@ -0,0 +1,18 @@
|
||||
# Operations runbooks
|
||||
|
||||
Step-by-step procedures for operating the homelab. These complement the
|
||||
agent-facing skill files in [`.agents/skills/`](../../.agents/skills/) (which
|
||||
are machine-actionable) and the deploy scripts in
|
||||
[`scripts/`](../../scripts/) (which are executable).
|
||||
|
||||
| Runbook | Scope |
|
||||
| ------- | ----- |
|
||||
| [rollback.md](rollback.md) | Rollback a deploy: checkout SHA + pg_restore |
|
||||
|
||||
For the deploy pipeline itself see
|
||||
[`scripts/deploy.sh`](../../scripts/deploy.sh), the watchdog at
|
||||
[`scripts/watchdog.sh`](../../scripts/watchdog.sh), and the cutover checklist
|
||||
at [`scripts/cutover-checklist.md`](../../scripts/cutover-checklist.md). The
|
||||
risk classification for any mutation is defined in
|
||||
[`seeds/policy.yaml`](../../seeds/policy.yaml) — run `oikos` MCP `preflight`
|
||||
to check the class before acting.
|
||||
55
evals/golden.yaml
Normal file
55
evals/golden.yaml
Normal file
@@ -0,0 +1,55 @@
|
||||
# Golden conversation evals for the nomos agent.
|
||||
# Run: go run ./cmd/nomos/eval -gateway http://localhost:8092 -manifest evals/*.yaml
|
||||
#
|
||||
# Each conversation costs real OpenRouter credits (~$0.01–0.05). The runner
|
||||
# sends the prompt, waits for the turn to finish, optionally sends a followup,
|
||||
# and scores assertions against the final persisted transcript.
|
||||
#
|
||||
# These are STRUCTURAL assertions only — tool-call sequences, plan steps,
|
||||
# writeback, completion. Text quality is model-dependent and not scored.
|
||||
|
||||
# --- eval 1: trivial read-only task (degenerate case) ---
|
||||
- name: trivial_readonly
|
||||
prompt: "What is the state of lxc:dns? One line, no plan needed."
|
||||
assertions:
|
||||
- kind: completes
|
||||
- kind: no_propose_plan # trivial — no ceremony
|
||||
- kind: max_tool_calls
|
||||
value: 5 # get_entity + complete_task + maybe one more
|
||||
- kind: no_run # read-only, no `run` needed
|
||||
|
||||
# --- eval 2: the operator's original bug — plan + proceed ---
|
||||
- name: plan_advances_on_proceed
|
||||
prompt: "Check the uptime of lxc:gitea. Plan it out, propose the plan, then wait for my approval before running anything."
|
||||
followup: "proceed with the rest"
|
||||
assertions:
|
||||
- kind: completes
|
||||
- kind: proposes_plan_once # propose_plan called exactly once
|
||||
- kind: no_duplicate_proposal # the original bug: re-propose on "proceed"
|
||||
- kind: writes_back # ran `run` → must update_entity_attributes (D.1)
|
||||
- kind: no_duplicate_complete # C.1 — complete_task called at most once
|
||||
|
||||
# --- eval 3: UI complaint should not re-execute (C.2) ---
|
||||
- name: ui_complaint_no_rerun
|
||||
prompt: "Check the uptime of lxc:dns. Plan it out and wait for my approval."
|
||||
followup: "go ahead"
|
||||
assertions:
|
||||
- kind: completes
|
||||
- kind: proposes_plan_once
|
||||
- kind: writes_back
|
||||
# (The followup "go ahead" is approval, not a UI complaint — we'd test the
|
||||
# complaint path separately with a second followup, but that needs the
|
||||
# session to stay open after completion, which the runner doesn't support yet.
|
||||
# For now this validates the approval-vocabulary path.)
|
||||
|
||||
# --- eval 4: knowledge preferred over fleet re-execution (E.1) ---
|
||||
# A same-day fleet audit knowledge entry exists in the DB. The agent should
|
||||
# search_knowledge first and NOT run `run` against 20 LXCs.
|
||||
- name: knowledge_preferred_over_rerun
|
||||
prompt: "Give me an overview of what needs updating across the homelab, categorize by criticality. There may be a recent audit already."
|
||||
assertions:
|
||||
- kind: completes
|
||||
- kind: calls_tool
|
||||
value: search_knowledge # E.1 — must check the knowledge base first
|
||||
- kind: max_run_calls
|
||||
value: 4 # NOT 20+ — a targeted refresh only
|
||||
18
evals/iteration-followup.yaml
Normal file
18
evals/iteration-followup.yaml
Normal file
@@ -0,0 +1,18 @@
|
||||
# P5 eval: iteration. A read-only task completes; the follow-up asks the
|
||||
# agent to act on what it found (a config_mutation). The second followup
|
||||
# is the approval ("go ahead") so the plan can execute. Asserts the session
|
||||
# reopens, a second plan generation is created, and the agent completes
|
||||
# both sub-tasks.
|
||||
- name: iteration-followup
|
||||
prompt: "Check the live systemd timer on lxc:rclone — when did the last backup to Proton Drive actually run, and when is the next one scheduled? Inspect the actual service state, don't just read the DB."
|
||||
followups:
|
||||
- "The repos folder failed last time. Reset the failed service and re-run the backup."
|
||||
- "go ahead"
|
||||
assertions:
|
||||
- kind: completes
|
||||
- kind: plan_generations
|
||||
value: 2
|
||||
- kind: proposes_plan
|
||||
- kind: writes_back
|
||||
- kind: max_run_calls
|
||||
value: 40
|
||||
17
evals/iteration-readonly.yaml
Normal file
17
evals/iteration-readonly.yaml
Normal file
@@ -0,0 +1,17 @@
|
||||
# P5 eval: two read-only sub-tasks back-to-back. The follow-up is also
|
||||
# read-only (no approval needed) so the agent can execute immediately after
|
||||
# proposing the second plan. Asserts the session reopens and a second plan
|
||||
# generation is created.
|
||||
- name: iteration-readonly
|
||||
prompt: "Check the live systemd timer on lxc:rclone — when did the last backup to Proton Drive actually run, and when is the next one scheduled?"
|
||||
followups:
|
||||
- "Now check the uptime of lxc:dns."
|
||||
assertions:
|
||||
- kind: completes
|
||||
- kind: plan_generations
|
||||
value: 2
|
||||
- kind: proposes_plan
|
||||
- kind: calls_tool
|
||||
value: run
|
||||
- kind: max_run_calls
|
||||
value: 6
|
||||
10
evals/no-plan-no-run.yaml
Normal file
10
evals/no-plan-no-run.yaml
Normal file
@@ -0,0 +1,10 @@
|
||||
# P5 eval: a pure-DB Q&A that calls NO run. This is the ONLY remaining
|
||||
# carve-out from plan-first: a task that never touches a live target via
|
||||
# `run` doesn't need propose_plan (the gate only fires on run). Asserts
|
||||
# the agent answers directly and completes without ceremony.
|
||||
- name: no-plan-no-run
|
||||
prompt: "List all LXC containers and their current health."
|
||||
assertions:
|
||||
- kind: completes
|
||||
- kind: no_run
|
||||
- kind: no_propose_plan
|
||||
14
evals/plan-always-readonly.yaml
Normal file
14
evals/plan-always-readonly.yaml
Normal file
@@ -0,0 +1,14 @@
|
||||
# P5 eval: a read-only question that requires live inspection (not just DB
|
||||
# lookup). Asserts the plan-first gate works: the agent must propose_plan
|
||||
# before run, even for a trivial read-only task.
|
||||
- name: plan-always-readonly
|
||||
prompt: "Check the live systemd timer on lxc:rclone — when did the last backup to Proton Drive actually run, and when is the next one scheduled? Inspect the actual service state, don't just read the DB."
|
||||
assertions:
|
||||
- kind: completes
|
||||
- kind: proposes_plan
|
||||
- kind: plan_before_run
|
||||
- kind: calls_tool
|
||||
value: run
|
||||
- kind: writes_back
|
||||
- kind: max_run_calls
|
||||
value: 6
|
||||
109
internal/actuator/circuit_breaker_test.go
Normal file
109
internal/actuator/circuit_breaker_test.go
Normal file
@@ -0,0 +1,109 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestNewCircuitBreakerDefaults(t *testing.T) {
|
||||
cb := newCircuitBreaker(0, 0)
|
||||
if cb.threshold != 3 {
|
||||
t.Errorf("threshold = %d, want 3", cb.threshold)
|
||||
}
|
||||
if cb.cooldownS != 300 {
|
||||
t.Errorf("cooldownS = %d, want 300", cb.cooldownS)
|
||||
}
|
||||
|
||||
cb = newCircuitBreaker(5, 60)
|
||||
if cb.threshold != 5 {
|
||||
t.Errorf("threshold = %d, want 5", cb.threshold)
|
||||
}
|
||||
if cb.cooldownS != 60 {
|
||||
t.Errorf("cooldownS = %d, want 60", cb.cooldownS)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCircuitBreakerIsOpenFresh(t *testing.T) {
|
||||
cb := newCircuitBreaker(3, 60)
|
||||
if cb.isOpen("host:A") {
|
||||
t.Errorf("fresh circuit should be closed, got open")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCircuitBreakerOpensAtThreshold(t *testing.T) {
|
||||
cb := newCircuitBreaker(3, 60)
|
||||
// threshold-1 failures → still closed
|
||||
cb.recordFailure("host:A")
|
||||
cb.recordFailure("host:A")
|
||||
if cb.isOpen("host:A") {
|
||||
t.Fatalf("circuit should be closed after threshold-1 failures")
|
||||
}
|
||||
// one more → open
|
||||
cb.recordFailure("host:A")
|
||||
if !cb.isOpen("host:A") {
|
||||
t.Fatalf("circuit should be open after threshold failures")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCircuitBreakerClosesAfterCooldown(t *testing.T) {
|
||||
cb := newCircuitBreaker(1, 60)
|
||||
// Force open
|
||||
cb.recordFailure("host:A")
|
||||
if !cb.isOpen("host:A") {
|
||||
t.Fatalf("circuit should be open")
|
||||
}
|
||||
// Manipulate the cooldown timestamp to the past to simulate expiry.
|
||||
cb.mu.Lock()
|
||||
cb.cooldowns["host:A"] = time.Now().Add(-1 * time.Second)
|
||||
cb.mu.Unlock()
|
||||
|
||||
if cb.isOpen("host:A") {
|
||||
t.Fatalf("circuit should be closed after cooldown expired")
|
||||
}
|
||||
// Failure count should have been reset by isOpen.
|
||||
cb.mu.Lock()
|
||||
got := cb.failures["host:A"]
|
||||
cb.mu.Unlock()
|
||||
if got != 0 {
|
||||
t.Errorf("failure count after cooldown reset = %d, want 0", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCircuitBreakerRecordSuccessResets(t *testing.T) {
|
||||
cb := newCircuitBreaker(3, 60)
|
||||
cb.recordFailure("host:A")
|
||||
cb.recordFailure("host:A")
|
||||
|
||||
cb.recordSuccess("host:A")
|
||||
|
||||
cb.mu.Lock()
|
||||
got := cb.failures["host:A"]
|
||||
cb.mu.Unlock()
|
||||
if got != 0 {
|
||||
t.Errorf("failure count after success = %d, want 0", got)
|
||||
}
|
||||
if cb.isOpen("host:A") {
|
||||
t.Errorf("circuit should be closed after success reset")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCircuitBreakerPerTargetIsolation(t *testing.T) {
|
||||
cb := newCircuitBreaker(2, 60)
|
||||
cb.recordFailure("host:A")
|
||||
cb.recordFailure("host:A") // host:A now at threshold → open
|
||||
|
||||
if !cb.isOpen("host:A") {
|
||||
t.Fatalf("host:A should be open")
|
||||
}
|
||||
if cb.isOpen("host:B") {
|
||||
t.Errorf("host:B should be closed (isolated from host:A)")
|
||||
}
|
||||
|
||||
// host:B has no failures recorded
|
||||
cb.mu.Lock()
|
||||
gotB := cb.failures["host:B"]
|
||||
cb.mu.Unlock()
|
||||
if gotB != 0 {
|
||||
t.Errorf("host:B failure count = %d, want 0", gotB)
|
||||
}
|
||||
}
|
||||
151
internal/actuator/ssh_test.go
Normal file
151
internal/actuator/ssh_test.go
Normal file
@@ -0,0 +1,151 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
func TestSSHErrorClassString(t *testing.T) {
|
||||
cases := []struct {
|
||||
class SSHErrorClass
|
||||
want string
|
||||
}{
|
||||
{SSHErrorNetwork, "network"},
|
||||
{SSHErrorAuth, "auth"},
|
||||
{SSHErrorTimeout, "timed_out"},
|
||||
{SSHErrorRemote, "remote"},
|
||||
{SSHErrorOther, "other"},
|
||||
{SSHErrorClass(999), "unknown"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := c.class.String(); got != c.want {
|
||||
t.Errorf("SSHErrorClass(%d).String() = %q, want %q", c.class, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// timeoutNetErr is a custom net.Error implementation for testing.
|
||||
type timeoutNetErr struct {
|
||||
timeout bool
|
||||
msg string
|
||||
}
|
||||
|
||||
func (e *timeoutNetErr) Error() string { return e.msg }
|
||||
func (e *timeoutNetErr) Timeout() bool { return e.timeout }
|
||||
func (e *timeoutNetErr) Temporary() bool { return false }
|
||||
|
||||
func TestClassifySSHError(t *testing.T) {
|
||||
// ssh.ExitError fields are unexported, but classifySSHError only checks
|
||||
// for the type via errors.As, so the zero value is sufficient.
|
||||
exitErr := &ssh.ExitError{}
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
err error
|
||||
want SSHErrorClass
|
||||
}{
|
||||
{"nil", nil, SSHErrorOther},
|
||||
{"deadline exceeded", context.DeadlineExceeded, SSHErrorTimeout},
|
||||
{"net error timeout true", &timeoutNetErr{timeout: true, msg: "i/o timeout"}, SSHErrorNetwork},
|
||||
{"net error timeout false", &timeoutNetErr{timeout: false, msg: "connection refused"}, SSHErrorNetwork},
|
||||
{"unable to authenticate", errors.New("unable to authenticate, no supported methods remain"), SSHErrorAuth},
|
||||
{"no supported methods remain", errors.New("no supported methods remain (server sent publickey)"), SSHErrorAuth},
|
||||
{"ssh handshake failed", errors.New("ssh: handshake failed: read tcp -> eof"), SSHErrorAuth},
|
||||
{"publickey", errors.New("publickey denied"), SSHErrorAuth},
|
||||
{"permission denied", errors.New("permission denied (publickey)"), SSHErrorAuth},
|
||||
{"exit error", exitErr, SSHErrorRemote},
|
||||
{"generic error", errors.New("something went wrong"), SSHErrorOther},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
if got := classifySSHError(c.err); got != c.want {
|
||||
t.Errorf("classifySSHError(%v) = %v, want %v", c.err, got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseProcedure(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
data []byte
|
||||
wantErr bool
|
||||
wantLen int
|
||||
}{
|
||||
{
|
||||
name: "valid with steps",
|
||||
data: []byte(`{"steps":[{"runner":"shell","command":"echo hi"}]}`),
|
||||
wantErr: false,
|
||||
wantLen: 1,
|
||||
},
|
||||
{
|
||||
name: "invalid json",
|
||||
data: []byte(`{not json`),
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "empty bytes",
|
||||
data: []byte{},
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "valid no steps key",
|
||||
data: []byte(`{"foo":"bar"}`),
|
||||
wantErr: false,
|
||||
wantLen: 0,
|
||||
},
|
||||
{
|
||||
name: "valid with extra fields",
|
||||
data: []byte(`{"extra":"ignored","steps":[{"runner":"verify","command":"true"}],"more":123}`),
|
||||
wantErr: false,
|
||||
wantLen: 1,
|
||||
},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
proc, err := ParseProcedure(c.data)
|
||||
if c.wantErr {
|
||||
if err == nil {
|
||||
t.Fatalf("expected error, got nil (proc=%+v)", proc)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(proc.Steps) != c.wantLen {
|
||||
t.Errorf("got %d steps, want %d", len(proc.Steps), c.wantLen)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetDefaultSSHTimeout(t *testing.T) {
|
||||
mu.Lock()
|
||||
orig := defaultSSHTimeout
|
||||
mu.Unlock()
|
||||
defer func() {
|
||||
mu.Lock()
|
||||
defaultSSHTimeout = orig
|
||||
mu.Unlock()
|
||||
}()
|
||||
|
||||
newTimeout := 42 * time.Second
|
||||
SetDefaultSSHTimeout(newTimeout)
|
||||
|
||||
mu.Lock()
|
||||
got := defaultSSHTimeout
|
||||
mu.Unlock()
|
||||
|
||||
if got != newTimeout {
|
||||
t.Errorf("defaultSSHTimeout = %v, want %v", got, newTimeout)
|
||||
}
|
||||
}
|
||||
|
||||
// Ensure timeoutNetErr satisfies net.Error at compile time.
|
||||
var _ net.Error = (*timeoutNetErr)(nil)
|
||||
@@ -19,7 +19,7 @@ type CheckDef struct {
|
||||
Extra map[string]any
|
||||
}
|
||||
|
||||
func ResolveHost(attrs map[string]any) string {
|
||||
func resolveHost(attrs map[string]any) string {
|
||||
if ip, ok := attrs["lan_ip"].(string); ok && ip != "" {
|
||||
return ip
|
||||
}
|
||||
@@ -57,8 +57,8 @@ func resolveSSHPort(attrs map[string]any) int {
|
||||
return 22
|
||||
}
|
||||
|
||||
func ForEntityType(entityType string, attrs map[string]any) []CheckDef {
|
||||
host := ResolveHost(attrs)
|
||||
func forEntityType(entityType string, attrs map[string]any) []CheckDef {
|
||||
host := resolveHost(attrs)
|
||||
user := resolveSSHUser(attrs)
|
||||
port := resolveSSHPort(attrs)
|
||||
|
||||
@@ -122,7 +122,7 @@ func ForEntityType(entityType string, attrs map[string]any) []CheckDef {
|
||||
return nil
|
||||
}
|
||||
|
||||
func ShortSlug(slug string) string {
|
||||
func shortSlug(slug string) string {
|
||||
const n = 8
|
||||
if len(slug) > n {
|
||||
return slug[len(slug)-n:]
|
||||
@@ -130,7 +130,7 @@ func ShortSlug(slug string) string {
|
||||
return slug
|
||||
}
|
||||
|
||||
func DefaultInterval(kind string) int32 {
|
||||
func defaultInterval(kind string) int32 {
|
||||
switch kind {
|
||||
case "ping":
|
||||
return 30
|
||||
@@ -156,7 +156,7 @@ func Ensure(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, slug, entityType
|
||||
attrs = map[string]any{}
|
||||
}
|
||||
|
||||
defs := ForEntityType(entityType, attrs)
|
||||
defs := forEntityType(entityType, attrs)
|
||||
if len(defs) == 0 {
|
||||
return
|
||||
}
|
||||
@@ -166,7 +166,7 @@ func Ensure(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, slug, entityType
|
||||
if err != nil {
|
||||
checkID = uuid.New()
|
||||
}
|
||||
checkSlug := fmt.Sprintf("check:%s:%s:%d", def.Kind, ShortSlug(slug), i)
|
||||
checkSlug := fmt.Sprintf("check:%s:%s:%d", def.Kind, shortSlug(slug), i)
|
||||
|
||||
_, _ = tx.Exec(ctx,
|
||||
`INSERT INTO entities (id, slug, type, name, state, attributes, version, created_at, updated_at)
|
||||
@@ -199,6 +199,6 @@ func Ensure(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, slug, entityType
|
||||
`INSERT INTO check_defs (entity_id, target_id, kind, config, interval_s, timeout_s, enabled)
|
||||
VALUES ($1, $2, $3, $4, $5, 30, true)
|
||||
ON CONFLICT (entity_id) DO NOTHING`,
|
||||
checkID, entityID, def.Kind, configJSON, DefaultInterval(def.Kind))
|
||||
checkID, entityID, def.Kind, configJSON, defaultInterval(def.Kind))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,9 +27,6 @@ WHERE e.type IN (SELECT name FROM tt)
|
||||
ORDER BY e.slug
|
||||
LIMIT sqlc.arg('lim');
|
||||
|
||||
-- name: ListEntitiesCapped :many
|
||||
SELECT e.* FROM entities e ORDER BY e.slug LIMIT $1;
|
||||
|
||||
-- name: InsertEntity :one
|
||||
INSERT INTO entities (id, slug, type, name, state, attributes)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)
|
||||
|
||||
@@ -14,11 +14,6 @@ WHERE (sqlc.narg('state')::text IS NULL OR sig.state = sqlc.narg('state'))
|
||||
ORDER BY se.slug
|
||||
LIMIT sqlc.arg('lim');
|
||||
|
||||
-- name: ListEntityStatus :many
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
ORDER BY e.slug;
|
||||
|
||||
-- name: GetIdempotentResponse :one
|
||||
SELECT response_code, response_body, request_hash FROM idempotency_keys
|
||||
WHERE actor = $1 AND key = $2;
|
||||
@@ -89,9 +84,6 @@ DO UPDATE SET occurrence_count = signals.occurrence_count + 1,
|
||||
updated_at = now()
|
||||
RETURNING *;
|
||||
|
||||
-- name: UpdateSignalState :exec
|
||||
UPDATE signals SET state = $2, updated_at = now() WHERE entity_id = $1;
|
||||
|
||||
-- name: GetOpenSignalsForAutoAct :many
|
||||
-- Signals with auto-act classifications that haven't been executed yet
|
||||
SELECT s.*, c.entity_id AS classification_id, c.action, c.risk_class, c.route,
|
||||
@@ -106,12 +98,6 @@ WHERE c.route = 'auto-act'
|
||||
ORDER BY s.last_seen_at ASC
|
||||
LIMIT $1;
|
||||
|
||||
-- name: InsertClassification :exec
|
||||
INSERT INTO classifications (entity_id, signal_entity_id, target_entity_id, action,
|
||||
recommended_action, risk_class, route, blast_radius, pattern_confidence,
|
||||
skill_id, autonomy_check, reasoning, correlation_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13);
|
||||
|
||||
-- name: ListClassifications :many
|
||||
SELECT c.entity_id, c.signal_entity_id, c.target_entity_id, c.action,
|
||||
c.recommended_action, c.risk_class, c.route, c.blast_radius,
|
||||
@@ -153,11 +139,6 @@ WHERE (sqlc.narg('status')::text IS NULL OR e.status = sqlc.narg('status'))
|
||||
ORDER BY te.slug
|
||||
LIMIT sqlc.arg('lim');
|
||||
|
||||
-- name: InsertFeedback :exec
|
||||
INSERT INTO feedback (entity_id, execution_id, outcome, observation, lesson,
|
||||
unexpected_side_effects, tags)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7);
|
||||
|
||||
-- name: GetFeedbackAfterWatermark :many
|
||||
SELECT f.entity_id, f.execution_id, f.outcome, f.observation, f.lesson,
|
||||
f.unexpected_side_effects, f.tags, f.created_at,
|
||||
@@ -201,11 +182,6 @@ SELECT * FROM skills
|
||||
WHERE (sqlc.narg('status')::text IS NULL OR status = sqlc.narg('status'))
|
||||
ORDER BY name, version DESC;
|
||||
|
||||
-- name: InsertSkill :exec
|
||||
INSERT INTO skills (entity_id, version, name, procedure, applies_type, action,
|
||||
pattern_ids, status, changed_by, change_reason)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10);
|
||||
|
||||
-- name: UpdateSkillStatus :exec
|
||||
UPDATE skills SET status = $2, last_used_at = now() WHERE entity_id = $1 AND version = $2;
|
||||
|
||||
|
||||
@@ -22,12 +22,6 @@ WHERE r.valid_to IS NULL
|
||||
AND (sqlc.narg('rel_types')::text[] IS NULL OR r.type = ANY(sqlc.narg('rel_types')::text[]))
|
||||
ORDER BY r.type, se.slug, te.slug;
|
||||
|
||||
-- name: UpsertCurrentRelationship :exec
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from, valid_to)
|
||||
VALUES ($1, $2, $3, $4, now(), NULL)
|
||||
ON CONFLICT (source_id, target_id, type) WHERE valid_to IS NULL
|
||||
DO UPDATE SET attributes = EXCLUDED.attributes;
|
||||
|
||||
-- name: EndCurrentRelationship :execrows
|
||||
UPDATE relationships SET valid_to = now()
|
||||
WHERE source_id = $1 AND target_id = $2 AND type = $3 AND valid_to IS NULL;
|
||||
|
||||
@@ -177,43 +177,6 @@ func (q *Queries) ListEntities(ctx context.Context, arg ListEntitiesParams) ([]E
|
||||
return items, nil
|
||||
}
|
||||
|
||||
const listEntitiesCapped = `-- name: ListEntitiesCapped :many
|
||||
SELECT e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at, e.enrolled_at, e.enrolled_by FROM entities e ORDER BY e.slug LIMIT $1
|
||||
`
|
||||
|
||||
func (q *Queries) ListEntitiesCapped(ctx context.Context, limit int32) ([]Entity, error) {
|
||||
rows, err := q.db.Query(ctx, listEntitiesCapped, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var items []Entity
|
||||
for rows.Next() {
|
||||
var i Entity
|
||||
if err := rows.Scan(
|
||||
&i.ID,
|
||||
&i.Slug,
|
||||
&i.Type,
|
||||
&i.Name,
|
||||
&i.State,
|
||||
&i.Attributes,
|
||||
&i.MaintenanceUntil,
|
||||
&i.Version,
|
||||
&i.CreatedAt,
|
||||
&i.UpdatedAt,
|
||||
&i.EnrolledAt,
|
||||
&i.EnrolledBy,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, i)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return items, nil
|
||||
}
|
||||
|
||||
const updateEntity = `-- name: UpdateEntity :one
|
||||
UPDATE entities SET
|
||||
name = COALESCE($1, name),
|
||||
|
||||
@@ -35,11 +35,17 @@ type AgentMessage struct {
|
||||
}
|
||||
|
||||
type AgentSession struct {
|
||||
ID uuid.UUID
|
||||
Title string
|
||||
Actor string
|
||||
CreatedAt time.Time
|
||||
LastActiveAt time.Time
|
||||
ID uuid.UUID
|
||||
Title string
|
||||
Actor string
|
||||
CreatedAt time.Time
|
||||
LastActiveAt time.Time
|
||||
Goal string
|
||||
Status string
|
||||
Outcome *string
|
||||
Summary string
|
||||
EntityID *uuid.UUID
|
||||
CompletionNudges int32
|
||||
}
|
||||
|
||||
type Approval struct {
|
||||
@@ -83,6 +89,7 @@ type AuditLog struct {
|
||||
Detail []byte
|
||||
SourceIp *string
|
||||
CorrelationID *string
|
||||
SessionID *uuid.UUID
|
||||
}
|
||||
|
||||
type AutonomySetting struct {
|
||||
@@ -289,6 +296,13 @@ type MetricSample struct {
|
||||
Tags []byte
|
||||
}
|
||||
|
||||
type NomosPlanExecution struct {
|
||||
ExecutionID uuid.UUID
|
||||
SessionID uuid.UUID
|
||||
ContinuedAt *time.Time
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
type Pattern struct {
|
||||
EntityID uuid.UUID
|
||||
AppliesType string
|
||||
@@ -351,6 +365,32 @@ type SeedVersion struct {
|
||||
AppliedAt time.Time
|
||||
}
|
||||
|
||||
type SessionPlanStep struct {
|
||||
ID uuid.UUID
|
||||
SessionID uuid.UUID
|
||||
Seq int32
|
||||
Title string
|
||||
Detail string
|
||||
Status string
|
||||
ExecutionID *uuid.UUID
|
||||
TargetSlug *string
|
||||
StartedAt *time.Time
|
||||
FinishedAt *time.Time
|
||||
CreatedAt time.Time
|
||||
Generation int32
|
||||
}
|
||||
|
||||
type SessionQuestion struct {
|
||||
ID uuid.UUID
|
||||
SessionID uuid.UUID
|
||||
Prompt string
|
||||
Context []byte
|
||||
Status string
|
||||
Answer *string
|
||||
CreatedAt time.Time
|
||||
AnsweredAt *time.Time
|
||||
}
|
||||
|
||||
type Signal struct {
|
||||
EntityID uuid.UUID
|
||||
Kind string
|
||||
|
||||
@@ -416,48 +416,6 @@ func (q *Queries) InsertCheckDef(ctx context.Context, arg InsertCheckDefParams)
|
||||
return err
|
||||
}
|
||||
|
||||
const insertClassification = `-- name: InsertClassification :exec
|
||||
INSERT INTO classifications (entity_id, signal_entity_id, target_entity_id, action,
|
||||
recommended_action, risk_class, route, blast_radius, pattern_confidence,
|
||||
skill_id, autonomy_check, reasoning, correlation_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)
|
||||
`
|
||||
|
||||
type InsertClassificationParams struct {
|
||||
EntityID uuid.UUID
|
||||
SignalEntityID *uuid.UUID
|
||||
TargetEntityID *uuid.UUID
|
||||
Action string
|
||||
RecommendedAction []byte
|
||||
RiskClass string
|
||||
Route string
|
||||
BlastRadius []uuid.UUID
|
||||
PatternConfidence *float32
|
||||
SkillID *uuid.UUID
|
||||
AutonomyCheck *string
|
||||
Reasoning []byte
|
||||
CorrelationID string
|
||||
}
|
||||
|
||||
func (q *Queries) InsertClassification(ctx context.Context, arg InsertClassificationParams) error {
|
||||
_, err := q.db.Exec(ctx, insertClassification,
|
||||
arg.EntityID,
|
||||
arg.SignalEntityID,
|
||||
arg.TargetEntityID,
|
||||
arg.Action,
|
||||
arg.RecommendedAction,
|
||||
arg.RiskClass,
|
||||
arg.Route,
|
||||
arg.BlastRadius,
|
||||
arg.PatternConfidence,
|
||||
arg.SkillID,
|
||||
arg.AutonomyCheck,
|
||||
arg.Reasoning,
|
||||
arg.CorrelationID,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
const insertEvent = `-- name: InsertEvent :one
|
||||
INSERT INTO events (type, entity_id, severity, source, data, correlation_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)
|
||||
@@ -530,35 +488,6 @@ func (q *Queries) InsertExecution(ctx context.Context, arg InsertExecutionParams
|
||||
return err
|
||||
}
|
||||
|
||||
const insertFeedback = `-- name: InsertFeedback :exec
|
||||
INSERT INTO feedback (entity_id, execution_id, outcome, observation, lesson,
|
||||
unexpected_side_effects, tags)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7)
|
||||
`
|
||||
|
||||
type InsertFeedbackParams struct {
|
||||
EntityID uuid.UUID
|
||||
ExecutionID uuid.UUID
|
||||
Outcome string
|
||||
Observation *string
|
||||
Lesson *string
|
||||
UnexpectedSideEffects []string
|
||||
Tags []string
|
||||
}
|
||||
|
||||
func (q *Queries) InsertFeedback(ctx context.Context, arg InsertFeedbackParams) error {
|
||||
_, err := q.db.Exec(ctx, insertFeedback,
|
||||
arg.EntityID,
|
||||
arg.ExecutionID,
|
||||
arg.Outcome,
|
||||
arg.Observation,
|
||||
arg.Lesson,
|
||||
arg.UnexpectedSideEffects,
|
||||
arg.Tags,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
const insertMetricSample = `-- name: InsertMetricSample :exec
|
||||
INSERT INTO metric_samples (entity_id, metric, value, tags, ts)
|
||||
VALUES ($1, $2, $3, $4, now())
|
||||
@@ -581,41 +510,6 @@ func (q *Queries) InsertMetricSample(ctx context.Context, arg InsertMetricSample
|
||||
return err
|
||||
}
|
||||
|
||||
const insertSkill = `-- name: InsertSkill :exec
|
||||
INSERT INTO skills (entity_id, version, name, procedure, applies_type, action,
|
||||
pattern_ids, status, changed_by, change_reason)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
|
||||
`
|
||||
|
||||
type InsertSkillParams struct {
|
||||
EntityID uuid.UUID
|
||||
Version int32
|
||||
Name string
|
||||
Procedure []byte
|
||||
AppliesType *string
|
||||
Action string
|
||||
PatternIds []uuid.UUID
|
||||
Status string
|
||||
ChangedBy *uuid.UUID
|
||||
ChangeReason *string
|
||||
}
|
||||
|
||||
func (q *Queries) InsertSkill(ctx context.Context, arg InsertSkillParams) error {
|
||||
_, err := q.db.Exec(ctx, insertSkill,
|
||||
arg.EntityID,
|
||||
arg.Version,
|
||||
arg.Name,
|
||||
arg.Procedure,
|
||||
arg.AppliesType,
|
||||
arg.Action,
|
||||
arg.PatternIds,
|
||||
arg.Status,
|
||||
arg.ChangedBy,
|
||||
arg.ChangeReason,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
const listApprovalRules = `-- name: ListApprovalRules :many
|
||||
SELECT id, entity_type, action, risk_class, autonomy_level, scope_entity, version, updated_at FROM approval_rules ORDER BY entity_type, action
|
||||
`
|
||||
@@ -852,44 +746,6 @@ func (q *Queries) ListEnabledCheckDefs(ctx context.Context) ([]ListEnabledCheckD
|
||||
return items, nil
|
||||
}
|
||||
|
||||
const listEntityStatus = `-- name: ListEntityStatus :many
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
ORDER BY e.slug
|
||||
`
|
||||
|
||||
type ListEntityStatusRow struct {
|
||||
Slug string
|
||||
Type string
|
||||
Health string
|
||||
LastCheckAt *time.Time
|
||||
}
|
||||
|
||||
func (q *Queries) ListEntityStatus(ctx context.Context) ([]ListEntityStatusRow, error) {
|
||||
rows, err := q.db.Query(ctx, listEntityStatus)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var items []ListEntityStatusRow
|
||||
for rows.Next() {
|
||||
var i ListEntityStatusRow
|
||||
if err := rows.Scan(
|
||||
&i.Slug,
|
||||
&i.Type,
|
||||
&i.Health,
|
||||
&i.LastCheckAt,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, i)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return items, nil
|
||||
}
|
||||
|
||||
const listEvents = `-- name: ListEvents :many
|
||||
SELECT id, ts, type, entity_id, severity, source, data, correlation_id
|
||||
FROM events
|
||||
@@ -1462,20 +1318,6 @@ func (q *Queries) UpdatePatternStatus(ctx context.Context, arg UpdatePatternStat
|
||||
return err
|
||||
}
|
||||
|
||||
const updateSignalState = `-- name: UpdateSignalState :exec
|
||||
UPDATE signals SET state = $2, updated_at = now() WHERE entity_id = $1
|
||||
`
|
||||
|
||||
type UpdateSignalStateParams struct {
|
||||
EntityID uuid.UUID
|
||||
State string
|
||||
}
|
||||
|
||||
func (q *Queries) UpdateSignalState(ctx context.Context, arg UpdateSignalStateParams) error {
|
||||
_, err := q.db.Exec(ctx, updateSignalState, arg.EntityID, arg.State)
|
||||
return err
|
||||
}
|
||||
|
||||
const updateSkillStatus = `-- name: UpdateSkillStatus :exec
|
||||
UPDATE skills SET status = $2, last_used_at = now() WHERE entity_id = $1 AND version = $2
|
||||
`
|
||||
|
||||
@@ -139,27 +139,3 @@ func (q *Queries) ListGraphEdges(ctx context.Context, arg ListGraphEdgesParams)
|
||||
}
|
||||
return items, nil
|
||||
}
|
||||
|
||||
const upsertCurrentRelationship = `-- name: UpsertCurrentRelationship :exec
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from, valid_to)
|
||||
VALUES ($1, $2, $3, $4, now(), NULL)
|
||||
ON CONFLICT (source_id, target_id, type) WHERE valid_to IS NULL
|
||||
DO UPDATE SET attributes = EXCLUDED.attributes
|
||||
`
|
||||
|
||||
type UpsertCurrentRelationshipParams struct {
|
||||
SourceID uuid.UUID
|
||||
TargetID uuid.UUID
|
||||
Type string
|
||||
Attributes []byte
|
||||
}
|
||||
|
||||
func (q *Queries) UpsertCurrentRelationship(ctx context.Context, arg UpsertCurrentRelationshipParams) error {
|
||||
_, err := q.db.Exec(ctx, upsertCurrentRelationship,
|
||||
arg.SourceID,
|
||||
arg.TargetID,
|
||||
arg.Type,
|
||||
arg.Attributes,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
139
internal/domain/domain_test.go
Normal file
139
internal/domain/domain_test.go
Normal file
@@ -0,0 +1,139 @@
|
||||
package domain
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIsNil(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
u UUID
|
||||
want bool
|
||||
}{
|
||||
{"empty string", UUID(""), true},
|
||||
{"single char", UUID("x"), false},
|
||||
{"uuid string", UUID("550e8400-e29b-41d4-a716-446655440000"), false},
|
||||
{"nil literal", UUID(""), true},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got := c.u.IsNil()
|
||||
if got != c.want {
|
||||
t.Errorf("UUID(%q).IsNil() = %v, want %v", c.u, got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCanTransition(t *testing.T) {
|
||||
type tc struct {
|
||||
name string
|
||||
from string
|
||||
to string
|
||||
want bool
|
||||
}
|
||||
var cases []tc
|
||||
|
||||
for from, targets := range ValidSignalTransitions {
|
||||
for _, to := range targets {
|
||||
cases = append(cases, tc{from + "->" + to, from, to, true})
|
||||
}
|
||||
}
|
||||
|
||||
disallowed := []tc{
|
||||
{"raised->raised", SignalRaised, SignalRaised, false},
|
||||
{"resolved->raised", SignalResolved, SignalRaised, false},
|
||||
{"failed->raised", SignalFailed, SignalRaised, false},
|
||||
{"acknowledged->raised", SignalAcknowledged, SignalRaised, false},
|
||||
{"muted->resolved", SignalMuted, SignalResolved, false},
|
||||
{"acting->acknowledged", SignalActing, SignalAcknowledged, false},
|
||||
}
|
||||
cases = append(cases, disallowed...)
|
||||
|
||||
cases = append(cases,
|
||||
tc{"unknown source", "nonexistent", SignalRaised, false},
|
||||
tc{"unknown target", SignalRaised, "nonexistent", false},
|
||||
)
|
||||
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
s := &Signal{State: c.from}
|
||||
got := s.CanTransition(c.to)
|
||||
if got != c.want {
|
||||
t.Errorf("CanTransition(%q -> %q) = %v, want %v", c.from, c.to, got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSentinelErrors(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
err error
|
||||
msg string
|
||||
}{
|
||||
{"ErrNotFound", ErrNotFound, "entity not found"},
|
||||
{"ErrInvalidTransition", ErrInvalidTransition, "invalid lifecycle transition"},
|
||||
{"ErrApprovalRequired", ErrApprovalRequired, "operator approval required"},
|
||||
{"ErrAutonomyBlocked", ErrAutonomyBlocked, "autonomy policy blocks this action"},
|
||||
{"ErrConflict", ErrConflict, "concurrent modification conflict"},
|
||||
{"ErrCircuitOpen", ErrCircuitOpen, "circuit breaker open for target"},
|
||||
{"ErrAbstractType", ErrAbstractType, "cannot instantiate abstract entity type"},
|
||||
{"ErrInvalidEdge", ErrInvalidEdge, "relationship endpoint type mismatch"},
|
||||
{"ErrCardinality", ErrCardinality, "relationship cardinality violation"},
|
||||
{"ErrSeedHashMismatch", ErrSeedHashMismatch, "seed content hash mismatch"},
|
||||
{"ErrAlreadyExists", ErrAlreadyExists, "entity already exists"},
|
||||
{"ErrQuarantined", ErrQuarantined, "pattern is quarantined"},
|
||||
{"ErrSkillDeprecated", ErrSkillDeprecated, "skill is deprecated"},
|
||||
{"ErrInvalidInput", ErrInvalidInput, "invalid input"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
if c.err == nil {
|
||||
t.Fatal("sentinel error is nil")
|
||||
}
|
||||
if !errors.Is(c.err, c.err) {
|
||||
t.Errorf("errors.Is failed for %s", c.name)
|
||||
}
|
||||
if c.err.Error() != c.msg {
|
||||
t.Errorf("Error() = %q, want %q", c.err.Error(), c.msg)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSignalTransitionsComplete(t *testing.T) {
|
||||
// Non-terminal states must be keys in ValidSignalTransitions.
|
||||
// SignalResolved is a terminal state (no outgoing transitions) and is
|
||||
// intentionally absent from the map.
|
||||
nonTerminal := []string{
|
||||
SignalRaised,
|
||||
SignalAcknowledged,
|
||||
SignalActing,
|
||||
SignalMuted,
|
||||
SignalFailed,
|
||||
}
|
||||
for _, state := range nonTerminal {
|
||||
targets, ok := ValidSignalTransitions[state]
|
||||
if !ok {
|
||||
t.Errorf("non-terminal state %q missing from ValidSignalTransitions", state)
|
||||
continue
|
||||
}
|
||||
if len(targets) == 0 {
|
||||
t.Errorf("state %q maps to empty transition list", state)
|
||||
}
|
||||
}
|
||||
|
||||
// Resolved is terminal: it should not appear as a source key.
|
||||
if _, ok := ValidSignalTransitions[SignalResolved]; ok {
|
||||
t.Errorf("terminal state %q should not have outgoing transitions", SignalResolved)
|
||||
}
|
||||
|
||||
// No state anywhere in the map may map to nil/empty.
|
||||
for state, targets := range ValidSignalTransitions {
|
||||
if len(targets) == 0 {
|
||||
t.Errorf("state %q maps to empty/nil transition list", state)
|
||||
}
|
||||
}
|
||||
}
|
||||
676
internal/httpapi/actuator.go
Normal file
676
internal/httpapi/actuator.go
Normal file
@@ -0,0 +1,676 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
var (
|
||||
_sshUser string
|
||||
_sshKey []byte
|
||||
)
|
||||
|
||||
// flexBool accepts a JSON bool, number (0/1), or string ("true"/"1"/"yes").
|
||||
// LLMs routinely emit `"privileged": 0` instead of `false`; a strict `bool`
|
||||
// field made the approved pct_create execution fail to parse *after* the
|
||||
// operator had already approved it — the container was never created and the
|
||||
// operator saw "queued" with no result. This type tolerates the common shapes.
|
||||
type flexBool bool
|
||||
|
||||
func (b *flexBool) UnmarshalJSON(data []byte) error {
|
||||
s := strings.TrimSpace(strings.Trim(string(data), `"`))
|
||||
switch strings.ToLower(s) {
|
||||
case "true", "1", "yes", "on":
|
||||
*b = true
|
||||
case "false", "0", "no", "off", "", "null":
|
||||
*b = false
|
||||
default:
|
||||
return fmt.Errorf("cannot parse %q as bool", s)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func initSSH() {
|
||||
if _sshUser == "" {
|
||||
_sshUser = os.Getenv("OIKOS_SSH_USER")
|
||||
if _sshUser == "" {
|
||||
_sshUser = "root"
|
||||
}
|
||||
}
|
||||
if len(_sshKey) == 0 {
|
||||
keyPath := os.Getenv("OIKOS_SSH_KEY_PATH")
|
||||
if keyPath == "" {
|
||||
keyPath = "/etc/oikos/ssh_key"
|
||||
}
|
||||
var err error
|
||||
_sshKey, err = os.ReadFile(keyPath)
|
||||
if err != nil {
|
||||
slog.Warn("httpapi ssh: cannot read key", "path", keyPath, "error", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// sshExecTimeout bounds how long a single remote command may run. Without
|
||||
// this, a hung remote command (e.g. a piped install script stuck retrying
|
||||
// DNS against a misconfigured gateway) blocks the executing goroutine
|
||||
// forever: the execution never leaves 'approved'/'running', the operator
|
||||
// sees an unkillable spinner, and get_execution_status has nothing new to
|
||||
// report. Generous enough for a real apt/docker install; not infinite.
|
||||
const sshExecTimeout = 10 * time.Minute
|
||||
|
||||
func sshExec(ctx context.Context, host, user, command string) (string, error) {
|
||||
initSSH()
|
||||
if len(_sshKey) == 0 {
|
||||
return "", fmt.Errorf("no SSH key available")
|
||||
}
|
||||
if user == "" {
|
||||
user = _sshUser
|
||||
}
|
||||
|
||||
addr := host + ":22"
|
||||
signer, err := ssh.ParsePrivateKey(_sshKey)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("parse key: %w", err)
|
||||
}
|
||||
|
||||
cfg := &ssh.ClientConfig{
|
||||
User: user,
|
||||
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
|
||||
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
|
||||
Timeout: 10 * time.Second,
|
||||
}
|
||||
|
||||
client, err := ssh.Dial("tcp", addr, cfg)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("dial %s: %w", host, err)
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
type result struct {
|
||||
out []byte
|
||||
err error
|
||||
}
|
||||
done := make(chan result, 1)
|
||||
go func() {
|
||||
// See internal/mcp/server.go's sshExec for why this recovers rather
|
||||
// than letting a rare SSH-library panic crash the whole api process.
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
done <- result{nil, fmt.Errorf("panic in ssh exec: %v", r)}
|
||||
}
|
||||
}()
|
||||
out, err := session.CombinedOutput(command)
|
||||
done <- result{out, err}
|
||||
}()
|
||||
|
||||
select {
|
||||
case r := <-done:
|
||||
text := strings.TrimSpace(string(r.out))
|
||||
// A non-zero exit MUST surface as an error. The previous guard only
|
||||
// errored when there was no output, so a `pct create` that printed
|
||||
// "CT 132 already exists" and exited non-zero was reported as
|
||||
// success — the execution was marked completed though nothing was
|
||||
// provisioned.
|
||||
if r.err != nil {
|
||||
if text != "" {
|
||||
return text, fmt.Errorf("%w: %s", r.err, text)
|
||||
}
|
||||
return text, fmt.Errorf("exec: %w", r.err)
|
||||
}
|
||||
return text, nil
|
||||
case <-time.After(sshExecTimeout):
|
||||
// Close the session/client to hang up the remote side; the
|
||||
// goroutine above will eventually exit once that unblocks
|
||||
// CombinedOutput, but we don't wait for it — the caller needs an
|
||||
// answer now, not an indefinite hang.
|
||||
session.Close()
|
||||
client.Close()
|
||||
return "", fmt.Errorf("timed out after %s waiting for command to finish on %s", sshExecTimeout, host)
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
client.Close()
|
||||
return "", ctx.Err()
|
||||
}
|
||||
}
|
||||
|
||||
func resolveHostSSH(ctx context.Context, pool *db.Pool, entitySlug string) (string, string, error) {
|
||||
var attrs string
|
||||
err := pool.QueryRow(ctx, "SELECT attributes::text FROM entities WHERE slug = $1", entitySlug).Scan(&attrs)
|
||||
if err != nil {
|
||||
return "", "", fmt.Errorf("entity not found: %s", entitySlug)
|
||||
}
|
||||
|
||||
var m map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(attrs), &m); err != nil {
|
||||
return "", "", fmt.Errorf("parse attributes: %w", err)
|
||||
}
|
||||
|
||||
sshUser := _sshUser
|
||||
if sshUser == "" {
|
||||
sshUser = "root"
|
||||
}
|
||||
|
||||
if ip, ok := m["lan_ip"].(string); ok && ip != "" {
|
||||
return ip, sshUser, nil
|
||||
}
|
||||
if mesh, ok := m["mesh"].(map[string]interface{}); ok {
|
||||
for _, proto := range []string{"netbird", "tailscale"} {
|
||||
if p, ok := mesh[proto].(map[string]interface{}); ok {
|
||||
if ip, ok := p["ip"].(string); ok && ip != "" {
|
||||
return ip, sshUser, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return "", "", fmt.Errorf("no IP found for %s", entitySlug)
|
||||
}
|
||||
|
||||
// resolveRunTarget mirrors internal/mcp.resolveExecTarget for the approved-
|
||||
// execution side: any target slug (host: or lxc:) resolves to the SSH
|
||||
// endpoint that runs the command plus a wrap function that turns a plain
|
||||
// shell command into what actually needs to be sent — identity for a host,
|
||||
// `pct exec <pve_id>` for an LXC. Kept as a small duplicate rather than a
|
||||
// cross-package import to avoid coupling httpapi to mcp for one helper.
|
||||
func resolveRunTarget(ctx context.Context, pool *db.Pool, targetSlug string) (host, user string, wrap func(string) string, err error) {
|
||||
if strings.HasPrefix(targetSlug, "host:") {
|
||||
host, user, err = resolveHostSSH(ctx, pool, targetSlug)
|
||||
return host, user, func(cmd string) string { return cmd }, err
|
||||
}
|
||||
if strings.HasPrefix(targetSlug, "lxc:") {
|
||||
var pveID, hostAttr string
|
||||
// COALESCE the host column: many older LXC entities (seeded from
|
||||
// inventory, not provisioned by pct_create) have pve_id but no host
|
||||
// attribute at all. Scanning a SQL NULL into a plain string errors
|
||||
// the whole row, wrongly reporting "missing pve_id" even when it was
|
||||
// present — COALESCE avoids the NULL, "" is handled below.
|
||||
if qerr := pool.QueryRow(ctx, "SELECT attributes->>'pve_id', COALESCE(attributes->>'host', '') FROM entities WHERE slug = $1", targetSlug).Scan(&pveID, &hostAttr); qerr != nil || pveID == "" {
|
||||
return "", "", nil, fmt.Errorf("LXC not found or missing pve_id: %s", targetSlug)
|
||||
}
|
||||
hostSlug := hostAttr
|
||||
if hostSlug == "" {
|
||||
hostSlug = "hubris"
|
||||
}
|
||||
if !strings.HasPrefix(hostSlug, "host:") {
|
||||
hostSlug = "host:" + hostSlug
|
||||
}
|
||||
host, user, err = resolveHostSSH(ctx, pool, hostSlug)
|
||||
id := pveID
|
||||
return host, user, func(cmd string) string {
|
||||
b64 := base64.StdEncoding.EncodeToString([]byte(cmd))
|
||||
return fmt.Sprintf("pct exec %s -- bash -c 'echo %s | base64 -d | bash'", id, b64)
|
||||
}, err
|
||||
}
|
||||
return "", "", nil, fmt.Errorf("unsupported target %q: must be host:<slug> or lxc:<slug>", targetSlug)
|
||||
}
|
||||
|
||||
// executeApprovedAction runs a gated action after operator approval.
|
||||
// Runs in a background goroutine to not block the HTTP response.
|
||||
// emitExecutionEvent records an execution lifecycle event for SSE fan-out so
|
||||
// the control room can watch approved actions run to completion live.
|
||||
func emitExecutionEvent(ctx context.Context, pool *db.Pool, execID uuid.UUID, status string, detail map[string]any) {
|
||||
severity := "info"
|
||||
if status == "failed" {
|
||||
severity = "warning"
|
||||
}
|
||||
_ = observability.Event(ctx, sqlcgen.New(pool), "execution."+status, &execID, severity, "actuator", "", detail)
|
||||
if status == "completed" || status == "failed" || status == "cancelled" {
|
||||
closePlanStepForExecution(ctx, pool, execID, status)
|
||||
}
|
||||
}
|
||||
|
||||
// closePlanStepForExecution auto-closes a task plan step whose linked execution
|
||||
// just reached a terminal state, so the task board advances even if the agent
|
||||
// doesn't call update_plan_step itself (belt and suspenders — the agent links
|
||||
// the step to the execution when it starts it; the api finishes it here). Emits
|
||||
// plan.step.finished correlated to the step's session. No-op for the vast
|
||||
// majority of executions, which aren't plan steps.
|
||||
func closePlanStepForExecution(ctx context.Context, pool *db.Pool, execID uuid.UUID, execStatus string) {
|
||||
stepStatus := "done"
|
||||
if execStatus == "failed" || execStatus == "cancelled" {
|
||||
stepStatus = "failed"
|
||||
}
|
||||
var stepID, sessionID string
|
||||
var seq int
|
||||
if err := pool.QueryRow(ctx, `
|
||||
UPDATE session_plan_steps SET status = $2, finished_at = now()
|
||||
WHERE execution_id = $1 AND status NOT IN ('done', 'failed', 'skipped')
|
||||
RETURNING id::text, session_id::text, seq`, execID, stepStatus).Scan(&stepID, &sessionID, &seq); err != nil {
|
||||
return // no matching open step
|
||||
}
|
||||
_ = observability.Event(ctx, sqlcgen.New(pool), "plan.step.finished", &execID, "info", "actuator", sessionID,
|
||||
map[string]any{"step_id": stepID, "seq": seq, "status": stepStatus, "execution_id": execID.String()})
|
||||
}
|
||||
|
||||
func executeApprovedAction(ctx context.Context, pool *db.Pool, execID uuid.UUID, targetSlug string, actionStr string) {
|
||||
slog.Info("httpapi: executing approved action", "execution_id", execID, "target", targetSlug, "action", actionStr)
|
||||
|
||||
host, user, wrap, err := resolveRunTarget(ctx, pool, targetSlug)
|
||||
if err != nil {
|
||||
slog.Error("httpapi: resolve host for approved execution", "error", err, "target", targetSlug)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("%s", err.Error()))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
idx := strings.Index(actionStr, ":")
|
||||
if idx < 0 {
|
||||
slog.Error("httpapi: malformed action string (no colon)", "action", actionStr)
|
||||
return
|
||||
}
|
||||
action, params := actionStr[:idx], actionStr[idx+1:]
|
||||
|
||||
startedAt := time.Now()
|
||||
var output, cmd string
|
||||
|
||||
switch action {
|
||||
case "systemctl":
|
||||
svc := strings.TrimPrefix(targetSlug, "lxc:")
|
||||
switch {
|
||||
case strings.HasPrefix(params, "enable:"):
|
||||
svc = strings.TrimPrefix(params, "enable:")
|
||||
cmd = fmt.Sprintf("systemctl enable %s --now 2>&1; sleep 1; systemctl is-active %s", svc, svc)
|
||||
case strings.HasPrefix(params, "disable:"):
|
||||
svc = strings.TrimPrefix(params, "disable:")
|
||||
cmd = fmt.Sprintf("systemctl disable %s --now 2>&1; sleep 1; systemctl is-active %s", svc, svc)
|
||||
default:
|
||||
cmd = fmt.Sprintf("systemctl %s %s 2>&1", params, svc)
|
||||
}
|
||||
output, err = sshExec(ctx, host, user, cmd)
|
||||
|
||||
case "apt_upgrade":
|
||||
svc := strings.TrimPrefix(targetSlug, "lxc:")
|
||||
cmd = fmt.Sprintf("apt update -qq 2>&1 >/dev/null && apt upgrade -y -qq 2>&1; echo '---'; systemctl is-active %s || true", svc)
|
||||
output, err = sshExec(ctx, host, user, cmd)
|
||||
|
||||
case "pct_create":
|
||||
var cfg struct {
|
||||
VMID int `json:"vmid"`
|
||||
Hostname string `json:"hostname"`
|
||||
Cores int `json:"cores"`
|
||||
Memory int `json:"memory"`
|
||||
DiskGB int `json:"disk_gb"`
|
||||
IP string `json:"ip"`
|
||||
GW string `json:"gw"`
|
||||
Bridge string `json:"bridge"` // e.g. vmbr0/vmbr1 — which bridge actually reaches the target subnet on this host varies per host, don't assume vmbr0
|
||||
Storage string `json:"storage"`
|
||||
Template string `json:"template"`
|
||||
Privileged flexBool `json:"privileged"`
|
||||
Nesting flexBool `json:"nesting"`
|
||||
Mounts []string `json:"mounts"`
|
||||
Nameserver string `json:"nameserver"`
|
||||
Searchdomain string `json:"searchdomain"`
|
||||
// No services/post_install here anymore — pct_create is atomic
|
||||
// (create + start + register only). Installing packages and
|
||||
// running setup scripts is the agent's job via follow-up `run`
|
||||
// calls against lxc:<hostname>, so each step is individually
|
||||
// observable and recoverable instead of one opaque multi-minute
|
||||
// black box. See the comment above the removed post-create block.
|
||||
}
|
||||
if err := json.Unmarshal([]byte(params), &cfg); err != nil {
|
||||
slog.Error("httpapi: pct_create parse params", "error", err, "params", params)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("invalid pct_create params: %v", err))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": err.Error()})
|
||||
return
|
||||
}
|
||||
// Only hostname is required. vmid is optional — when 0 (or later found
|
||||
// to collide) the VMID guard below assigns a free cluster id.
|
||||
if cfg.Hostname == "" {
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, `{"error":"pct_create: hostname is required"}`)
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": "missing hostname"})
|
||||
return
|
||||
}
|
||||
if cfg.Cores == 0 {
|
||||
cfg.Cores = 1
|
||||
}
|
||||
if cfg.Memory == 0 {
|
||||
cfg.Memory = 512
|
||||
}
|
||||
if cfg.DiskGB == 0 {
|
||||
cfg.DiskGB = 8
|
||||
}
|
||||
if cfg.Storage == "" {
|
||||
cfg.Storage = "local-lvm"
|
||||
}
|
||||
if cfg.GW == "" {
|
||||
cfg.GW = "192.168.8.2"
|
||||
}
|
||||
if cfg.Nameserver == "" {
|
||||
cfg.Nameserver = "192.168.8.2"
|
||||
}
|
||||
if cfg.Searchdomain == "" {
|
||||
cfg.Searchdomain = "hubris.network"
|
||||
}
|
||||
// Template pre-flight: resolve against what the host actually has
|
||||
// cached. A hardcoded name (e.g. debian-13) fails opaquely with a raw
|
||||
// `pct` error when that exact file isn't present. List the cache, then
|
||||
// either validate the requested template or auto-pick the newest
|
||||
// debian one; on miss, fail early with the available list so the
|
||||
// operator/agent can retry with a real name.
|
||||
cacheList, tplErr := sshExec(ctx, host, user, "ls -1 /var/lib/vz/template/cache/ 2>/dev/null | grep -E '\\.tar\\.(zst|gz|xz)$' || true")
|
||||
available := []string{}
|
||||
for _, l := range strings.Split(strings.TrimSpace(cacheList), "\n") {
|
||||
if l = strings.TrimSpace(l); l != "" {
|
||||
available = append(available, l)
|
||||
}
|
||||
}
|
||||
if tplErr != nil {
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("list templates on %s: %s", targetSlug, tplErr.Error()))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": tplErr.Error()})
|
||||
return
|
||||
}
|
||||
cfg.Template = resolveTemplate(cfg.Template, available)
|
||||
if cfg.Template == "" {
|
||||
msg := fmt.Sprintf("no usable LXC template on %s. Available: %v", targetSlug, available)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("%s", msg))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": msg})
|
||||
return
|
||||
}
|
||||
|
||||
// VMID collision guard. Proxmox VMIDs are cluster-wide, so the model's
|
||||
// guess (e.g. 132) can collide with a container on another node — pct
|
||||
// create then fails with "CT N already exists on node X". Fetch the set
|
||||
// of in-use VMIDs across the cluster; if the requested id is taken (or
|
||||
// absent), fall back to the cluster's next free id so provisioning
|
||||
// still succeeds instead of dead-ending on the operator's approval.
|
||||
usedRaw, _ := sshExec(ctx, host, user, `pvesh get /cluster/resources --type vm --output-format json 2>/dev/null | grep -o '"vmid":[0-9]*' | grep -o '[0-9]*' || true`)
|
||||
used := map[int]bool{}
|
||||
for _, l := range strings.Fields(usedRaw) {
|
||||
if n, e := strconv.Atoi(strings.TrimSpace(l)); e == nil {
|
||||
used[n] = true
|
||||
}
|
||||
}
|
||||
if cfg.VMID == 0 || used[cfg.VMID] {
|
||||
nextRaw, nerr := sshExec(ctx, host, user, `pvesh get /cluster/nextid 2>/dev/null`)
|
||||
nextID, cerr := strconv.Atoi(strings.TrimSpace(nextRaw))
|
||||
if nerr != nil || cerr != nil || nextID == 0 {
|
||||
msg := fmt.Sprintf("VMID %d is already in use on the cluster and could not resolve a free id", cfg.VMID)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("%s", msg))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": msg})
|
||||
return
|
||||
}
|
||||
slog.Info("httpapi: pct_create VMID reassigned", "requested", cfg.VMID, "assigned", nextID)
|
||||
cfg.VMID = nextID
|
||||
}
|
||||
|
||||
privFlag := "--unprivileged 1"
|
||||
if cfg.Privileged {
|
||||
privFlag = "--unprivileged 0"
|
||||
}
|
||||
|
||||
nestingFlag := ""
|
||||
features := []string{}
|
||||
if cfg.Nesting {
|
||||
features = append(features, "nesting=1")
|
||||
}
|
||||
if cfg.Privileged {
|
||||
features = append(features, "keyctl=1")
|
||||
}
|
||||
if len(features) > 0 {
|
||||
nestingFlag = fmt.Sprintf(" --features %s", strings.Join(features, ","))
|
||||
}
|
||||
|
||||
if cfg.Bridge == "" {
|
||||
cfg.Bridge = "vmbr0"
|
||||
}
|
||||
|
||||
// net0: DHCP when no static IP is given (or ip=="dhcp"). Proxmox
|
||||
// rejects a gateway alongside ip=dhcp, so only add gw for a static IP.
|
||||
net0 := "name=eth0,bridge=" + cfg.Bridge + ","
|
||||
isStatic := cfg.IP != "" && !strings.EqualFold(cfg.IP, "dhcp")
|
||||
if !isStatic {
|
||||
net0 += "ip=dhcp"
|
||||
} else {
|
||||
net0 += "ip=" + cfg.IP
|
||||
if cfg.GW != "" {
|
||||
net0 += ",gw=" + cfg.GW
|
||||
}
|
||||
}
|
||||
|
||||
// Pre-flight: for a static config, ping the gateway from the target
|
||||
// HOST, on the SPECIFIC BRIDGE being requested, before spending 5+
|
||||
// minutes creating the container. This is the check that would have
|
||||
// caught the real TypeType failure immediately instead of after a
|
||||
// full provision attempt.
|
||||
//
|
||||
// Binding to the bridge (`ping -I <bridge>`) matters and was found
|
||||
// live: a plain unqualified `ping <gw>` from the host can succeed via
|
||||
// the host's own routing table (multiple routes, possibly through an
|
||||
// upstream router) even when the *container* — which only gets a
|
||||
// naive on-link default route via its bridge's veth — can never ARP
|
||||
// that gateway at all. Confirmed on `strong`: bare `ping 192.168.8.2`
|
||||
// succeeded (via the host's default route), but a container actually
|
||||
// attached to vmbr0 showed 100% packet loss trying to reach the same
|
||||
// address, because vmbr0 doesn't carry that subnet's L2 segment.
|
||||
// Binding to the bridge interface reproduces what the container will
|
||||
// actually experience, not what the host's broader routing table can
|
||||
// reach.
|
||||
if isStatic && cfg.GW != "" {
|
||||
pingOut, pingErr := sshExec(ctx, host, user, fmt.Sprintf("ping -I %s -c1 -W2 %s >/dev/null 2>&1 && echo PREFLIGHT_OK || echo PREFLIGHT_FAIL", cfg.Bridge, cfg.GW))
|
||||
if pingErr != nil || !gatewayPreflightPassed(pingOut) {
|
||||
msg := fmt.Sprintf(
|
||||
"gateway %s is not reachable from %s on bridge %s — this almost always means the bridge doesn't carry that subnet on this host (each bridge only reaches the network it's physically wired to). "+
|
||||
"Do not retry with a different gateway guess in the same subnet: find an existing LXC on this host with an IP in the same /28 and copy its exact bridge+gateway, or use DHCP instead.",
|
||||
cfg.GW, targetSlug, cfg.Bridge)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("%s", msg))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": msg})
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
templatePath := fmt.Sprintf("/var/lib/vz/template/cache/%s", cfg.Template)
|
||||
createCmd := fmt.Sprintf(
|
||||
"pct create %d %s --hostname %s --cores %d --memory %d --rootfs %s:%d %s --net0 %s%s --start 1",
|
||||
cfg.VMID, templatePath, cfg.Hostname, cfg.Cores, cfg.Memory,
|
||||
cfg.Storage, cfg.DiskGB, privFlag, net0, nestingFlag)
|
||||
|
||||
if cfg.Nameserver != "" {
|
||||
createCmd += fmt.Sprintf(" --nameserver %s", cfg.Nameserver)
|
||||
}
|
||||
if cfg.Searchdomain != "" {
|
||||
createCmd += fmt.Sprintf(" --searchdomain %s", cfg.Searchdomain)
|
||||
}
|
||||
|
||||
// Add mount points
|
||||
for i, mp := range cfg.Mounts {
|
||||
if i < 10 { // pct supports up to mp9
|
||||
createCmd += fmt.Sprintf(" --mp%d %s", i, mp)
|
||||
}
|
||||
}
|
||||
|
||||
slog.Info("httpapi: pct_create running", "vmid", cfg.VMID, "hostname", cfg.Hostname, "cmd", createCmd)
|
||||
output, err = sshExec(ctx, host, user, createCmd)
|
||||
|
||||
// pct_create is now DELIBERATELY ATOMIC: create + start + register,
|
||||
// nothing else. It used to also run apt installs and a post_install
|
||||
// script inline as one black-box multi-minute SSH call — the agent
|
||||
// got back a single opaque success/fail for the whole thing with no
|
||||
// way to see (or fix) which step actually broke. That's the opposite
|
||||
// of what makes an agent able to recover from errors.
|
||||
//
|
||||
// Installing packages, running post_install, and verifying the
|
||||
// service now happen as the agent's OWN follow-up `run` calls against
|
||||
// the new lxc:<hostname> target — each one is synchronous (in an
|
||||
// active assent window) or individually gated, so the agent observes
|
||||
// every step's real output and can diagnose + retry the exact thing
|
||||
// that failed instead of re-doing the whole container. See SOUL.md
|
||||
// "After pct_create: you drive the install" and provisionScript's
|
||||
// surviving role (DNS self-heal) is now something the agent invokes
|
||||
// itself via `run`, not something baked into this handler.
|
||||
//
|
||||
// cfg.Services/cfg.PostInstall are intentionally no longer read here.
|
||||
|
||||
// On success, register the entity in the DB with proper relationships
|
||||
if err == nil {
|
||||
slug := "lxc:" + cfg.Hostname
|
||||
var lxcID uuid.UUID
|
||||
lxcID, _ = uuid.NewV7()
|
||||
attrs := map[string]any{
|
||||
"pve_id": fmt.Sprintf("%d", cfg.VMID),
|
||||
"host": strings.TrimPrefix(targetSlug, "host:"),
|
||||
"ip": cfg.IP,
|
||||
}
|
||||
attrsJSON, _ := json.Marshal(attrs)
|
||||
_, insErr := pool.Exec(ctx, `INSERT INTO entities (id, slug, type, name, state, attributes, enrolled_at)
|
||||
VALUES ($1, $2, 'lxc', $3, 'provisioning', $4, now()) ON CONFLICT (slug) DO NOTHING`, lxcID, slug, cfg.Hostname, attrsJSON)
|
||||
if insErr != nil {
|
||||
slog.Error("httpapi: pct_create entity insert", "error", insErr, "slug", slug)
|
||||
}
|
||||
|
||||
// Create hosts relationship: Proxmox host → LXC
|
||||
var hostID uuid.UUID
|
||||
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", targetSlug).Scan(&hostID); err == nil {
|
||||
_, relErr := pool.Exec(ctx, `INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
VALUES ($1, $2, 'hosts', '{"provisioned_by":"nomos"}'::jsonb, now())`, hostID, lxcID)
|
||||
if relErr != nil {
|
||||
slog.Error("httpapi: pct_create relationship insert", "error", relErr, "host", targetSlug, "lxc", slug)
|
||||
}
|
||||
}
|
||||
|
||||
// Create entity_status row for health tracking
|
||||
pool.Exec(ctx, `INSERT INTO entity_status (entity_id, health, last_check_at)
|
||||
VALUES ($1, 'unknown', now()) ON CONFLICT (entity_id) DO NOTHING`, lxcID)
|
||||
|
||||
emitExecutionEvent(ctx, pool, execID, "executing", map[string]any{
|
||||
"lxc_slug": slug, "vmid": cfg.VMID, "host": targetSlug,
|
||||
})
|
||||
|
||||
slog.Info("httpapi: pct_create entity registered", "slug", slug, "vmid", cfg.VMID, "host", targetSlug)
|
||||
}
|
||||
|
||||
case "run":
|
||||
// The general gated primitive: arbitrary shell against any host or
|
||||
// LXC, approved and classified by internal/policy.ClassifyCommand at
|
||||
// request time (see mcp/server.go's "run" tool). No fixed action
|
||||
// enum — new capability doesn't require new Go code here.
|
||||
var cfg struct {
|
||||
Command string `json:"command"`
|
||||
Purpose string `json:"purpose"`
|
||||
}
|
||||
if perr := json.Unmarshal([]byte(params), &cfg); perr != nil {
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("invalid run params: %v", perr))
|
||||
emitExecutionEvent(ctx, pool, execID, "failed", map[string]any{"target": targetSlug, "error": perr.Error()})
|
||||
return
|
||||
}
|
||||
cmd = wrap(cfg.Command)
|
||||
output, err = sshExec(ctx, host, user, cmd)
|
||||
|
||||
default:
|
||||
slog.Error("httpapi: unknown gated action for approved execution", "action", action, "execution_id", execID)
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb WHERE entity_id=$1`,
|
||||
execID, jsonErr("unknown action: %s", action))
|
||||
return
|
||||
}
|
||||
|
||||
durationMs := int(time.Since(startedAt).Milliseconds())
|
||||
status := "completed"
|
||||
verified := true
|
||||
// Build result via json.Marshal, not string interpolation. Command output
|
||||
// (apt/pct) contains quotes, backslashes and control chars; the old
|
||||
// fmt.Sprintf only escaped "\n", producing invalid JSON that failed the
|
||||
// ::jsonb cast — so this UPDATE was silently discarded and the execution
|
||||
// was stuck at "approved" forever even though provisioning succeeded.
|
||||
resMap := map[string]any{"output": output}
|
||||
if err != nil {
|
||||
resMap["error"] = err.Error()
|
||||
status = "failed"
|
||||
verified = false
|
||||
}
|
||||
resultJSON, _ := json.Marshal(resMap)
|
||||
|
||||
if _, uerr := pool.Exec(ctx, `UPDATE executions SET status=$2, result=$3::jsonb, duration_ms=$4, verified=$5, started_at=$6, completed_at=$7 WHERE entity_id=$1`,
|
||||
execID, status, resultJSON, durationMs, verified, startedAt, time.Now()); uerr != nil {
|
||||
slog.Error("httpapi: finalize execution status", "error", uerr, "execution_id", execID, "intended_status", status)
|
||||
}
|
||||
|
||||
emitExecutionEvent(ctx, pool, execID, status, map[string]any{
|
||||
"action": action, "target": targetSlug, "duration_ms": durationMs,
|
||||
})
|
||||
|
||||
slog.Info("httpapi: approved action executed",
|
||||
"execution_id", execID, "action", action, "status", status, "duration_ms", durationMs)
|
||||
}
|
||||
|
||||
// jsonErr builds a valid {"error": "..."} JSON payload for an execution's
|
||||
// result column. Always use this instead of fmt.Sprintf'ing JSON by hand —
|
||||
// error text and command output routinely contain quotes/backslashes that
|
||||
// break a hand-built string and fail the ::jsonb cast.
|
||||
func jsonErr(format string, args ...any) []byte {
|
||||
b, _ := json.Marshal(map[string]any{"error": fmt.Sprintf(format, args...)})
|
||||
return b
|
||||
}
|
||||
|
||||
// resolveTemplate maps a requested template name to one actually present in
|
||||
// the host's template cache. Exact match wins; a bare distro hint (e.g.
|
||||
// "debian-13" or "debian") matches by prefix; empty picks the newest debian
|
||||
// (falling back to any) template available. Returns "" when nothing fits.
|
||||
// gatewayPreflightPassed interprets the PREFLIGHT_OK/PREFLIGHT_FAIL markers
|
||||
// from the pct_create gateway pre-flight check. Pulled out as its own
|
||||
// function (rather than an inline strings.Contains at the call site) so it's
|
||||
// unit-testable: a prior version checked for "REACHABLE", which is a
|
||||
// substring of "UNREACHABLE" — the check could never actually fail, and it
|
||||
// took a live deployment to notice. Exact-match markers plus a test make
|
||||
// that specific bug class structurally unable to recur silently.
|
||||
func gatewayPreflightPassed(out string) bool {
|
||||
return strings.TrimSpace(out) == "PREFLIGHT_OK"
|
||||
}
|
||||
|
||||
func resolveTemplate(requested string, available []string) string {
|
||||
if len(available) == 0 {
|
||||
return ""
|
||||
}
|
||||
if requested != "" {
|
||||
for _, a := range available {
|
||||
if a == requested {
|
||||
return a
|
||||
}
|
||||
}
|
||||
for _, a := range available {
|
||||
if strings.HasPrefix(a, requested) {
|
||||
return a
|
||||
}
|
||||
}
|
||||
}
|
||||
// Auto-pick: prefer debian, then the lexically-greatest (newest version).
|
||||
best := ""
|
||||
for _, a := range available {
|
||||
if strings.Contains(a, "debian") && a > best {
|
||||
best = a
|
||||
}
|
||||
}
|
||||
if best != "" {
|
||||
return best
|
||||
}
|
||||
for _, a := range available {
|
||||
if a > best {
|
||||
best = a
|
||||
}
|
||||
}
|
||||
return best
|
||||
}
|
||||
90
internal/httpapi/agent_activity.go
Normal file
90
internal/httpapi/agent_activity.go
Normal file
@@ -0,0 +1,90 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
)
|
||||
|
||||
// ─── Agent Activity (stub) ─────────────────────────────────────────────
|
||||
|
||||
func (s *Server) QueryAgentActivity(ctx context.Context, request gen.QueryAgentActivityRequestObject) (gen.QueryAgentActivityResponseObject, error) {
|
||||
limit := clampLimit(request.Params.Limit)
|
||||
from := time.Now().Add(-24 * time.Hour)
|
||||
if request.Params.From != nil {
|
||||
from = *request.Params.From
|
||||
}
|
||||
to := time.Now()
|
||||
if request.Params.To != nil {
|
||||
to = *request.Params.To
|
||||
}
|
||||
|
||||
var agentID *string
|
||||
if request.Params.AgentId != nil {
|
||||
a := *request.Params.AgentId
|
||||
agentID = &a
|
||||
}
|
||||
var activityType *string
|
||||
if request.Params.ActivityType != nil {
|
||||
a := string(*request.Params.ActivityType)
|
||||
activityType = &a
|
||||
}
|
||||
var entityID *string
|
||||
if request.Params.EntityId != nil {
|
||||
a := *request.Params.EntityId
|
||||
entityID = &a
|
||||
}
|
||||
var cursorID *int
|
||||
if request.Params.Cursor != nil && *request.Params.Cursor != "" {
|
||||
if id, err := parseIntOrZero(*request.Params.Cursor); err == nil && id > 0 {
|
||||
cursorID = &id
|
||||
}
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT id, ts, agent_id::text, session_id, activity_type, tool_name,
|
||||
entity_id::text, input_summary, output_summary,
|
||||
duration_ms, token_count, success, correlation_id
|
||||
FROM agent_activity
|
||||
WHERE ts >= $1 AND ts <= $2
|
||||
AND ($3::text IS NULL OR agent_id::text = $3)
|
||||
AND ($4::text IS NULL OR activity_type = $4)
|
||||
AND ($5::text IS NULL OR entity_id::text = $5)
|
||||
AND ($6::bigint IS NULL OR id < $6::bigint)
|
||||
ORDER BY id DESC
|
||||
LIMIT $7`,
|
||||
from, to, agentID, activityType, entityID, cursorID, limit+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.AgentActivity{}
|
||||
for rows.Next() {
|
||||
var a gen.AgentActivity
|
||||
if err := rows.Scan(&a.Id, &a.Ts, &a.AgentId, &a.SessionId,
|
||||
&a.ActivityType, &a.ToolName, &a.EntityId,
|
||||
&a.InputSummary, &a.OutputSummary,
|
||||
&a.DurationMs, &a.TokenCount, &a.Success,
|
||||
&a.CorrelationId); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, a)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
var next *string
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
lastID := fmt.Sprintf("%d", items[len(items)-1].Id)
|
||||
next = &lastID
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.AgentActivity{}
|
||||
}
|
||||
return gen.QueryAgentActivity200JSONResponse{Items: items, NextCursor: next}, nil
|
||||
}
|
||||
@@ -263,6 +263,20 @@ func TestAPIEndToEnd(t *testing.T) {
|
||||
}
|
||||
})
|
||||
|
||||
// Regression: rel_type is an optional array param (*[]string); when
|
||||
// omitted entirely (not an empty list), passing the nil pointer straight
|
||||
// through to pgx as a query arg panics because pgx can't infer the array
|
||||
// element type from a nil *[]string. root+depth alone must still work.
|
||||
t.Run("graph without rel_type", func(t *testing.T) {
|
||||
rec, body := get(t, h, "/api/v1/graph?root=host:hubris&depth=1", nil)
|
||||
if rec.Code != 200 {
|
||||
t.Fatalf("status %d", rec.Code)
|
||||
}
|
||||
if len(body["nodes"].([]any)) < 2 {
|
||||
t.Errorf("graph too small: %d nodes", len(body["nodes"].([]any)))
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("ontology", func(t *testing.T) {
|
||||
rec, body := get(t, h, "/api/v1/ontology", nil)
|
||||
if rec.Code != 200 {
|
||||
|
||||
160
internal/httpapi/approval_rules.go
Normal file
160
internal/httpapi/approval_rules.go
Normal file
@@ -0,0 +1,160 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// ─── Approval Rules (Policy) ───────────────────────────────────────────
|
||||
|
||||
func (s *Server) ListApprovalRules(ctx context.Context, req gen.ListApprovalRulesRequestObject) (gen.ListApprovalRulesResponseObject, error) {
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT id, entity_type, action, risk_class, autonomy_level,
|
||||
COALESCE((SELECT slug FROM entities WHERE id = scope_entity), ''),
|
||||
version, updated_at
|
||||
FROM approval_rules ORDER BY entity_type, action`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.ApprovalRule{}
|
||||
for rows.Next() {
|
||||
var rule gen.ApprovalRule
|
||||
var scopeSlug string
|
||||
if err := rows.Scan(&rule.Id, &rule.EntityType, &rule.Action,
|
||||
&rule.RiskClass, &rule.AutonomyLevel, &scopeSlug,
|
||||
&rule.Version); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if scopeSlug != "" {
|
||||
rule.ScopeEntity = &scopeSlug
|
||||
}
|
||||
items = append(items, rule)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.ApprovalRule{}
|
||||
}
|
||||
return gen.ListApprovalRules200JSONResponse{Items: items}, nil
|
||||
}
|
||||
|
||||
func (s *Server) CreateApprovalRule(ctx context.Context, req gen.CreateApprovalRuleRequestObject) (gen.CreateApprovalRuleResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := uuid.NewV7()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var scopeEntity *uuid.UUID
|
||||
if req.Body.ScopeEntity != nil && *req.Body.ScopeEntity != "" {
|
||||
se, rerr := s.resolveEntityID(ctx, *req.Body.ScopeEntity)
|
||||
if rerr != nil {
|
||||
return nil, rerr
|
||||
}
|
||||
scopeEntity = &se
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
_, err = tx.Exec(ctx, `
|
||||
INSERT INTO approval_rules (id, entity_type, action, risk_class, autonomy_level, scope_entity)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)`,
|
||||
id, req.Body.EntityType, req.Body.Action, req.Body.RiskClass,
|
||||
string(req.Body.AutonomyLevel), scopeEntity)
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
|
||||
return nil, fmt.Errorf("%w: rule for %s/%s already exists", domain.ErrAlreadyExists,
|
||||
coalesceStr(req.Body.EntityType, "*"), req.Body.Action)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "create",
|
||||
&id, "POST", "/api/v1/policy/approval-rules", "",
|
||||
map[string]any{"action": req.Body.Action, "risk_class": req.Body.RiskClass}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Return 202 pending approval (dual-control).
|
||||
return gen.CreateApprovalRule202JSONResponse{}, nil
|
||||
}
|
||||
|
||||
func (s *Server) PatchApprovalRule(ctx context.Context, req gen.PatchApprovalRuleRequestObject) (gen.PatchApprovalRuleResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var scopeEntity *uuid.UUID
|
||||
if req.Body.ScopeEntity != nil && *req.Body.ScopeEntity != "" {
|
||||
se, rerr := s.resolveEntityID(ctx, *req.Body.ScopeEntity)
|
||||
if rerr != nil {
|
||||
return nil, rerr
|
||||
}
|
||||
scopeEntity = &se
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
result, err := tx.Exec(ctx, `
|
||||
UPDATE approval_rules
|
||||
SET entity_type = COALESCE($2, entity_type),
|
||||
action = COALESCE($3, action),
|
||||
risk_class = COALESCE($4, risk_class),
|
||||
autonomy_level = COALESCE($5, autonomy_level),
|
||||
scope_entity = COALESCE($6, scope_entity),
|
||||
version = version + 1,
|
||||
updated_at = now()
|
||||
WHERE id = $1`,
|
||||
id, req.Body.EntityType, req.Body.Action, req.Body.RiskClass,
|
||||
string(req.Body.AutonomyLevel), scopeEntity)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if result.RowsAffected() == 0 {
|
||||
return nil, fmt.Errorf("%w: approval rule %s", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "patch",
|
||||
&id, "PATCH", "/api/v1/policy/approval-rules/"+req.Id, "",
|
||||
map[string]any{"action": req.Body.Action}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.PatchApprovalRule202JSONResponse{}, nil
|
||||
}
|
||||
259
internal/httpapi/approvals.go
Normal file
259
internal/httpapi/approvals.go
Normal file
@@ -0,0 +1,259 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// ─── Approvals ─────────────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) ListApprovals(ctx context.Context, req gen.ListApprovalsRequestObject) (gen.ListApprovalsResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
var status *string
|
||||
if req.Params.Status != nil {
|
||||
s := string(*req.Params.Status)
|
||||
status = &s
|
||||
}
|
||||
var kind *string
|
||||
if req.Params.Kind != nil {
|
||||
k := string(*req.Params.Kind)
|
||||
kind = &k
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT a.entity_id, a.action, a.risk_class, a.kind, a.payload,
|
||||
a.status, a.expires_at, a.decided_at, a.decided_by::text,
|
||||
a.created_at, e.slug
|
||||
FROM approvals a
|
||||
JOIN entities e ON e.id = COALESCE(a.subject_entity_id, a.entity_id)
|
||||
WHERE ($1::text IS NULL OR a.status = $1)
|
||||
AND ($2::text IS NULL OR a.kind = $2)
|
||||
AND ($3::text IS NULL OR e.slug > $3)
|
||||
ORDER BY e.slug
|
||||
LIMIT $4`,
|
||||
status, kind, req.Params.Cursor, limit+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.Approval{}
|
||||
for rows.Next() {
|
||||
var a gen.Approval
|
||||
var payloadBytes []byte
|
||||
var decidedBy *string
|
||||
if err := rows.Scan(&a.Id, &a.Action, &a.RiskClass, &a.Kind, &payloadBytes,
|
||||
&a.Status, &a.ExpiresAt, &a.DecidedAt, &decidedBy,
|
||||
&a.CreatedAt, &a.Slug); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a.DecidedBy = decidedBy
|
||||
var payload map[string]any
|
||||
if len(payloadBytes) > 0 && json.Unmarshal(payloadBytes, &payload) == nil {
|
||||
a.Payload = &payload
|
||||
}
|
||||
items = append(items, a)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
var next *string
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
next = &items[len(items)-1].Slug
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.Approval{}
|
||||
}
|
||||
return gen.ListApprovals200JSONResponse{Items: items, NextCursor: next}, nil
|
||||
}
|
||||
|
||||
func (s *Server) DecideApproval(ctx context.Context, req gen.DecideApprovalRequestObject) (gen.DecideApprovalResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
q := sqlcgen.New(tx)
|
||||
|
||||
// Verify HMAC token if provided (single-use, S5).
|
||||
if req.Body.Token != nil && *req.Body.Token != "" {
|
||||
var tokenHash *string
|
||||
var apprStatus string
|
||||
var expiresAt time.Time
|
||||
err := tx.QueryRow(ctx,
|
||||
"SELECT token_hash, status, expires_at FROM approvals WHERE entity_id = $1",
|
||||
id).Scan(&tokenHash, &apprStatus, &expiresAt)
|
||||
if err != nil || tokenHash == nil {
|
||||
return nil, fmt.Errorf("%w: approval not found", domain.ErrNotFound)
|
||||
}
|
||||
if apprStatus != "pending" {
|
||||
return nil, fmt.Errorf("%w: approval already decided", domain.ErrInvalidTransition)
|
||||
}
|
||||
if expiresAt.Before(time.Now()) {
|
||||
return nil, fmt.Errorf("%w: approval token expired", domain.ErrInvalidTransition)
|
||||
}
|
||||
if *tokenHash != hashToken(*req.Body.Token) {
|
||||
return nil, fmt.Errorf("%w: invalid approval token", domain.ErrInvalidInput)
|
||||
}
|
||||
}
|
||||
|
||||
// Map decision to status.
|
||||
var status string
|
||||
switch req.Body.Decision {
|
||||
case gen.Approve:
|
||||
status = "approved"
|
||||
case gen.Deny:
|
||||
status = "denied"
|
||||
case gen.Revoke:
|
||||
status = "revoked"
|
||||
default:
|
||||
return nil, fmt.Errorf("%w: invalid decision %q", domain.ErrInvalidInput, req.Body.Decision)
|
||||
}
|
||||
|
||||
if err := q.UpdateApprovalStatus(ctx, sqlcgen.UpdateApprovalStatusParams{
|
||||
EntityID: id,
|
||||
Status: status,
|
||||
}); err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: approval %s not found or already decided", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Re-read approval.
|
||||
app, err := q.GetApprovalByID(ctx, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
approval := approvalToGen(app)
|
||||
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "decide",
|
||||
&id, "POST", "/api/v1/approvals/"+req.Id+"/decision", "",
|
||||
map[string]any{"decision": status}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
// Emit for SSE fan-out (in-tx; NOTIFY fires post-commit).
|
||||
if evErr := observability.Event(ctx, q, "approval.decided", &id, "info", "api", "",
|
||||
map[string]any{"decision": status, "actor": actor}); evErr != nil {
|
||||
return nil, evErr
|
||||
}
|
||||
|
||||
// On approve: execute the linked gated command.
|
||||
if status == "approved" {
|
||||
var execID, targetID uuid.UUID
|
||||
var actionStr, targetSlug, riskClass string
|
||||
err := tx.QueryRow(ctx, `
|
||||
SELECT e.entity_id, e.target_entity_id, e.action, e.risk_class
|
||||
FROM executions e
|
||||
WHERE e.approval_id = $1 AND e.status = 'pending_approval'
|
||||
LIMIT 1`, id).Scan(&execID, &targetID, &actionStr, &riskClass)
|
||||
if err == nil {
|
||||
// Resolve target entity slug from targetID.
|
||||
_ = tx.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", targetID).Scan(&targetSlug)
|
||||
|
||||
safego.Go("httpapi:executeApprovedAction", func() {
|
||||
executeApprovedAction(context.Background(), s.pool, execID, targetSlug, actionStr)
|
||||
})
|
||||
// Status only — risk_class was set correctly at request time
|
||||
// (e.g. by policy.ClassifyCommand for `run`); overwriting it to
|
||||
// a hardcoded 'config_mutation' here corrupted the audit ledger
|
||||
// for every other risk class, including destructive.
|
||||
_, _ = tx.Exec(ctx, `UPDATE executions SET status = 'approved' WHERE entity_id = $1`, execID)
|
||||
|
||||
// Approving a plan step — by ANY route (this endpoint backs both
|
||||
// the chat Approve button and chat-assent) — opens/extends the
|
||||
// agent's assent window. This is the scope gate the Nomos
|
||||
// auto-continuation worker checks: with the window open, the
|
||||
// finished execution's result is fed back to the agent so it runs
|
||||
// the plan to completion. Without opening it here, approving via
|
||||
// the button (instead of typing "go ahead") would silently not
|
||||
// auto-continue.
|
||||
var agentID *uuid.UUID
|
||||
if qerr := tx.QueryRow(ctx, "SELECT agent_id FROM executions WHERE entity_id = $1", execID).Scan(&agentID); qerr == nil && agentID != nil {
|
||||
expires := time.Now().Add(30 * time.Minute).UTC().Format(time.RFC3339)
|
||||
_, _ = tx.Exec(ctx, `INSERT INTO autonomy_settings (key, value) VALUES ($1, $2)
|
||||
ON CONFLICT (key) DO UPDATE SET value = $2`, "assent_window.agent:"+agentID.String(), expires)
|
||||
|
||||
// Approving a DESTRUCTIVE step via the button is exactly as
|
||||
// explicit as a typed "I confirm" — the operator affirmatively
|
||||
// clicked Approve on a card that said DESTRUCTIVE. Open the
|
||||
// same short, target-scoped destructive window chat-assent's
|
||||
// typed-confirm path opens, for parity: a multi-step
|
||||
// destructive recovery (stop, then destroy) shouldn't need a
|
||||
// fresh confirmation per click any more than it needs one per
|
||||
// typed phrase.
|
||||
if riskClass == "destructive" && targetSlug != "" {
|
||||
dExpires := time.Now().Add(15 * time.Minute).UTC().Format(time.RFC3339)
|
||||
_, _ = tx.Exec(ctx, `INSERT INTO autonomy_settings (key, value) VALUES ($1, $2)
|
||||
ON CONFLICT (key) DO UPDATE SET value = $2`,
|
||||
"destructive_window.agent:"+agentID.String()+".target:"+targetSlug, dExpires)
|
||||
}
|
||||
}
|
||||
|
||||
slog.Info("httpapi: approved execution queued",
|
||||
"execution_id", execID, "target", targetSlug, "action", actionStr)
|
||||
} else {
|
||||
slog.Warn("httpapi: no pending execution found for approval", "approval_id", id, "error", err)
|
||||
}
|
||||
} else {
|
||||
// Denied/revoked: reflect it on the linked execution too. Previously
|
||||
// only the approvals row changed, so the execution stayed
|
||||
// 'pending_approval' forever — any UI/poller reading execution
|
||||
// status (not approval status) never saw the decision.
|
||||
_, _ = tx.Exec(ctx, `UPDATE executions SET status = $2, completed_at = now() WHERE approval_id = $1 AND status = 'pending_approval'`, id, status)
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.DecideApproval200JSONResponse(approval), nil
|
||||
}
|
||||
|
||||
func approvalToGen(a sqlcgen.Approval) gen.Approval {
|
||||
app := gen.Approval{
|
||||
Id: a.EntityID,
|
||||
Action: a.Action,
|
||||
RiskClass: a.RiskClass,
|
||||
Kind: gen.ApprovalKind(a.Kind),
|
||||
Status: gen.ApprovalStatus(a.Status),
|
||||
ExpiresAt: a.ExpiresAt,
|
||||
DecidedAt: a.DecidedAt,
|
||||
CreatedAt: a.CreatedAt,
|
||||
}
|
||||
if a.DecidedBy != nil {
|
||||
s := a.DecidedBy.String()
|
||||
app.DecidedBy = &s
|
||||
}
|
||||
var payload map[string]any
|
||||
if len(a.Payload) > 0 && json.Unmarshal(a.Payload, &payload) == nil && len(payload) > 0 {
|
||||
app.Payload = &payload
|
||||
}
|
||||
return app
|
||||
}
|
||||
102
internal/httpapi/autonomy.go
Normal file
102
internal/httpapi/autonomy.go
Normal file
@@ -0,0 +1,102 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
)
|
||||
|
||||
// ─── Autonomy Settings ─────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) GetAutonomySettings(ctx context.Context, req gen.GetAutonomySettingsRequestObject) (gen.GetAutonomySettingsResponseObject, error) {
|
||||
rows, err := s.pool.Query(ctx, `SELECT key, value, version, updated_at FROM autonomy_settings ORDER BY key`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.AutonomySetting{}
|
||||
for rows.Next() {
|
||||
var as gen.AutonomySetting
|
||||
if err := rows.Scan(&as.Key, &as.Value, &as.Version, &as.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, as)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.AutonomySetting{}
|
||||
}
|
||||
return gen.GetAutonomySettings200JSONResponse{Items: items}, nil
|
||||
}
|
||||
|
||||
func (s *Server) PatchAutonomySettings(ctx context.Context, req gen.PatchAutonomySettingsRequestObject) (gen.PatchAutonomySettingsResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
for key, value := range *req.Body {
|
||||
_, err := tx.Exec(ctx, `
|
||||
INSERT INTO autonomy_settings (key, value, version, updated_at)
|
||||
VALUES ($1, $2, 1, now())
|
||||
ON CONFLICT (key)
|
||||
DO UPDATE SET value = EXCLUDED.value, version = autonomy_settings.version + 1, updated_at = now()`,
|
||||
key, value)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
// Re-read all settings.
|
||||
rows, err := tx.Query(ctx, `SELECT key, value, version, updated_at FROM autonomy_settings ORDER BY key`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.AutonomySetting{}
|
||||
for rows.Next() {
|
||||
var as gen.AutonomySetting
|
||||
if err := rows.Scan(&as.Key, &as.Value, &as.Version, &as.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, as)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "patch",
|
||||
nil, "PATCH", "/api/v1/policy/autonomy", "",
|
||||
map[string]any{"keys": keysOfMap(*req.Body)}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.PatchAutonomySettings200JSONResponse{Items: items}, nil
|
||||
}
|
||||
|
||||
// keysOfMap returns the keys of a map[string]string.
|
||||
func keysOfMap(m map[string]string) []string {
|
||||
keys := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
return keys
|
||||
}
|
||||
304
internal/httpapi/checks.go
Normal file
304
internal/httpapi/checks.go
Normal file
@@ -0,0 +1,304 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// ─── Checks ────────────────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) ListChecks(ctx context.Context, req gen.ListChecksRequestObject) (gen.ListChecksResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT cd.entity_id, e.slug, cd.kind,
|
||||
COALESCE(te.slug, '') AS target_slug, cd.target_type,
|
||||
cd.config, cd.interval_s, cd.timeout_s, cd.zone, cd.enabled,
|
||||
e.version
|
||||
FROM check_defs cd
|
||||
JOIN entities e ON e.id = cd.entity_id
|
||||
LEFT JOIN entities te ON te.id = cd.target_id
|
||||
WHERE ($1::text IS NULL OR cd.kind = $1)
|
||||
AND ($2::text IS NULL OR te.slug = $2)
|
||||
AND ($3::bool IS NULL OR cd.enabled = $3)
|
||||
AND ($4::text IS NULL OR e.slug > $4)
|
||||
ORDER BY e.slug
|
||||
LIMIT $5`,
|
||||
req.Params.Kind, req.Params.Target, req.Params.Enabled, req.Params.Cursor, limit+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.Check{}
|
||||
for rows.Next() {
|
||||
var c gen.Check
|
||||
var targetSlug string
|
||||
var configBytes []byte
|
||||
if err := rows.Scan(&c.Id, &c.Slug, &c.Kind, &targetSlug, &c.TargetType,
|
||||
&configBytes, &c.IntervalS, &c.TimeoutS, &c.Zone, &c.Enabled, &c.Version); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if targetSlug != "" {
|
||||
c.Target = &targetSlug
|
||||
}
|
||||
var config map[string]any
|
||||
if len(configBytes) > 0 && json.Unmarshal(configBytes, &config) == nil && len(config) > 0 {
|
||||
c.Config = &config
|
||||
}
|
||||
items = append(items, c)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
var next *string
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
next = &items[len(items)-1].Slug
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.Check{}
|
||||
}
|
||||
return gen.ListChecks200JSONResponse{Items: items, NextCursor: next}, nil
|
||||
}
|
||||
|
||||
func (s *Server) CreateCheck(ctx context.Context, req gen.CreateCheckRequestObject) (gen.CreateCheckResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := uuid.NewV7()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
slug := req.Body.Slug
|
||||
if slug == "" {
|
||||
slug = "check:" + string(req.Body.Kind) + ":" + uuid.New().String()[:8]
|
||||
}
|
||||
|
||||
// Resolve target if provided.
|
||||
var targetID *uuid.UUID
|
||||
if req.Body.Target != nil && *req.Body.Target != "" {
|
||||
tid, rerr := s.resolveEntityID(ctx, *req.Body.Target)
|
||||
if rerr != nil {
|
||||
return nil, rerr
|
||||
}
|
||||
targetID = &tid
|
||||
}
|
||||
|
||||
intervalS := int32(300)
|
||||
if req.Body.IntervalS != nil {
|
||||
intervalS = int32(*req.Body.IntervalS)
|
||||
}
|
||||
timeoutS := int32(30)
|
||||
if req.Body.TimeoutS != nil {
|
||||
timeoutS = int32(*req.Body.TimeoutS)
|
||||
}
|
||||
enabled := true
|
||||
if req.Body.Enabled != nil {
|
||||
enabled = *req.Body.Enabled
|
||||
}
|
||||
|
||||
configJSON := []byte("{}")
|
||||
if req.Body.Config != nil {
|
||||
configJSON, _ = json.Marshal(req.Body.Config)
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
q := sqlcgen.New(tx)
|
||||
|
||||
// Create the entity row (checks are entities).
|
||||
entity, err := q.InsertEntity(ctx, sqlcgen.InsertEntityParams{
|
||||
ID: id,
|
||||
Slug: slug,
|
||||
Type: "check",
|
||||
Name: slug,
|
||||
Attributes: []byte("{}"),
|
||||
})
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
|
||||
return nil, fmt.Errorf("%w: check %q already exists", domain.ErrAlreadyExists, slug)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := q.InsertCheckDef(ctx, sqlcgen.InsertCheckDefParams{
|
||||
EntityID: id,
|
||||
TargetID: targetID,
|
||||
TargetType: req.Body.TargetType,
|
||||
Kind: string(req.Body.Kind),
|
||||
Config: configJSON,
|
||||
IntervalS: intervalS,
|
||||
TimeoutS: timeoutS,
|
||||
Zone: req.Body.Zone,
|
||||
Enabled: enabled,
|
||||
}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Build response Check.
|
||||
check := gen.Check{
|
||||
Id: id,
|
||||
Slug: entity.Slug,
|
||||
Kind: gen.CheckKind(req.Body.Kind),
|
||||
IntervalS: int(intervalS),
|
||||
TimeoutS: int(timeoutS),
|
||||
Enabled: enabled,
|
||||
TargetType: req.Body.TargetType,
|
||||
Zone: req.Body.Zone,
|
||||
Version: int(entity.Version),
|
||||
}
|
||||
if req.Body.Config != nil {
|
||||
check.Config = req.Body.Config
|
||||
}
|
||||
if targetID != nil && req.Body.Target != nil {
|
||||
check.Target = req.Body.Target
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "create",
|
||||
&id, "POST", "/api/v1/checks", "",
|
||||
map[string]any{"kind": req.Body.Kind, "slug": slug}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.CreateCheck201JSONResponse(check), nil
|
||||
}
|
||||
|
||||
func (s *Server) PatchCheck(ctx context.Context, req gen.PatchCheckRequestObject) (gen.PatchCheckResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Parse If-Match
|
||||
ifMatch := strings.Trim(req.Params.IfMatch, `"`)
|
||||
expectedVersion, err := parseIntIfMatch(ifMatch)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
_ = expectedVersion // check_defs don't track version via If-Match today, but we validate the header is present
|
||||
|
||||
if ifMatch == "" {
|
||||
return nil, fmt.Errorf("%w: invalid If-Match header", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
// Get current check def
|
||||
current, err := sqlcgen.New(s.pool).GetCheckDef(ctx, id)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: check %s", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
// Apply patch.
|
||||
if req.Body.Config != nil {
|
||||
current.Config, _ = json.Marshal(req.Body.Config)
|
||||
}
|
||||
if req.Body.IntervalS != nil {
|
||||
current.IntervalS = int32(*req.Body.IntervalS)
|
||||
}
|
||||
if req.Body.TimeoutS != nil {
|
||||
current.TimeoutS = int32(*req.Body.TimeoutS)
|
||||
}
|
||||
if req.Body.Enabled != nil {
|
||||
current.Enabled = *req.Body.Enabled
|
||||
}
|
||||
|
||||
if err := sqlcgen.New(tx).UpdateCheckDef(ctx, sqlcgen.UpdateCheckDefParams{
|
||||
EntityID: id,
|
||||
Kind: current.Kind,
|
||||
Config: current.Config,
|
||||
IntervalS: current.IntervalS,
|
||||
TimeoutS: current.TimeoutS,
|
||||
TargetID: current.TargetID,
|
||||
TargetType: current.TargetType,
|
||||
Zone: current.Zone,
|
||||
Enabled: current.Enabled,
|
||||
}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Re-read to get updated timestamp.
|
||||
updated, err := sqlcgen.New(tx).GetCheckDef(ctx, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
check := checkDefToGen(updated)
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "patch",
|
||||
&id, "PATCH", "/api/v1/checks/"+req.Id, "",
|
||||
map[string]any{"enabled": updated.Enabled}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.PatchCheck200JSONResponse(check), nil
|
||||
}
|
||||
|
||||
func checkDefToGen(cd sqlcgen.CheckDef) gen.Check {
|
||||
c := gen.Check{
|
||||
Id: cd.EntityID,
|
||||
Kind: gen.CheckKind(cd.Kind),
|
||||
IntervalS: int(cd.IntervalS),
|
||||
TimeoutS: int(cd.TimeoutS),
|
||||
Enabled: cd.Enabled,
|
||||
TargetType: cd.TargetType,
|
||||
Zone: cd.Zone,
|
||||
}
|
||||
var config map[string]any
|
||||
if len(cd.Config) > 0 && json.Unmarshal(cd.Config, &config) == nil && len(config) > 0 {
|
||||
c.Config = &config
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// parseIntIfMatch parses an integer from a raw If-Match header value (with quotes stripped).
|
||||
func parseIntIfMatch(s string) (int, error) {
|
||||
if s == "" {
|
||||
return 0, fmt.Errorf("empty version")
|
||||
}
|
||||
var v int
|
||||
for _, c := range s {
|
||||
if c < '0' || c > '9' {
|
||||
return 0, fmt.Errorf("invalid version: %q", s)
|
||||
}
|
||||
v = v*10 + int(c-'0')
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
85
internal/httpapi/classifications.go
Normal file
85
internal/httpapi/classifications.go
Normal file
@@ -0,0 +1,85 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// ─── Classifications ───────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) ListClassifications(ctx context.Context, req gen.ListClassificationsRequestObject) (gen.ListClassificationsResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
var route *string
|
||||
if req.Params.Route != nil {
|
||||
r := string(*req.Params.Route)
|
||||
route = &r
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT c.entity_id, c.signal_entity_id, c.target_entity_id, c.action,
|
||||
c.recommended_action, c.risk_class, c.route, c.blast_radius,
|
||||
c.pattern_confidence, c.skill_id, c.autonomy_check, c.reasoning,
|
||||
c.correlation_id, c.created_at,
|
||||
e.slug, COALESCE(se.slug, '') AS signal_slug, COALESCE(te.slug, '') AS target_slug
|
||||
FROM classifications c
|
||||
LEFT JOIN entities e ON e.id = c.entity_id
|
||||
LEFT JOIN entities se ON se.id = c.signal_entity_id
|
||||
LEFT JOIN entities te ON te.id = c.target_entity_id
|
||||
WHERE ($1::text IS NULL OR c.route = $1)
|
||||
AND ($2::text IS NULL OR e.slug > $2)
|
||||
ORDER BY e.slug
|
||||
LIMIT $3`,
|
||||
route, req.Params.Cursor, limit+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.Classification{}
|
||||
for rows.Next() {
|
||||
var cls gen.Classification
|
||||
var recActionJSON []byte
|
||||
var reasoningJSON []byte
|
||||
var blastRadius []uuid.UUID
|
||||
var signalSlug, targetSlug string
|
||||
if err := rows.Scan(&cls.Id, &cls.SignalId, &targetSlug, &cls.Action,
|
||||
&recActionJSON, &cls.RiskClass, &cls.Route, &blastRadius,
|
||||
&cls.PatternConfidence, &cls.SkillId, &cls.AutonomyCheck, &reasoningJSON,
|
||||
&cls.CorrelationId, &cls.CreatedAt,
|
||||
&cls.Target, &signalSlug, &targetSlug); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if targetSlug != "" {
|
||||
cls.Target = &targetSlug
|
||||
}
|
||||
var reasoning map[string]any
|
||||
if json.Unmarshal(reasoningJSON, &reasoning) == nil {
|
||||
cls.Reasoning = reasoning
|
||||
}
|
||||
if len(blastRadius) > 0 {
|
||||
br := make([]string, len(blastRadius))
|
||||
for i, id := range blastRadius {
|
||||
br[i] = id.String()
|
||||
}
|
||||
cls.BlastRadius = &br
|
||||
}
|
||||
items = append(items, cls)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
var next *string
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
if items[len(items)-1].Target != nil {
|
||||
next = items[len(items)-1].Target
|
||||
}
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.Classification{}
|
||||
}
|
||||
return gen.ListClassifications200JSONResponse{Items: items, NextCursor: next}, nil
|
||||
}
|
||||
160
internal/httpapi/entity_types.go
Normal file
160
internal/httpapi/entity_types.go
Normal file
@@ -0,0 +1,160 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
)
|
||||
|
||||
// ─── Entity Types (Ontology) ───────────────────────────────────────────
|
||||
|
||||
func (s *Server) CreateEntityType(ctx context.Context, req gen.CreateEntityTypeRequestObject) (gen.CreateEntityTypeResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
isAbstract := false
|
||||
if req.Body.IsAbstract != nil {
|
||||
isAbstract = *req.Body.IsAbstract
|
||||
}
|
||||
|
||||
attrsSchemaJSON := []byte("null")
|
||||
if req.Body.AttributeSchema != nil {
|
||||
attrsSchemaJSON, _ = json.Marshal(req.Body.AttributeSchema)
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
_, err = tx.Exec(ctx, `
|
||||
INSERT INTO entity_types (name, parent_type, is_abstract, domain, layer, description, lifecycle_id, attribute_schema, status)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 'active')`,
|
||||
req.Body.Name, req.Body.ParentType, isAbstract, req.Body.Domain,
|
||||
string(req.Body.Layer), req.Body.Description, req.Body.LifecycleId, attrsSchemaJSON)
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
|
||||
return nil, fmt.Errorf("%w: entity type %q already exists", domain.ErrAlreadyExists, req.Body.Name)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Re-read.
|
||||
var et gen.EntityType
|
||||
var schemaBytes []byte
|
||||
err = tx.QueryRow(ctx, `
|
||||
SELECT name, parent_type, is_abstract, domain, layer, description,
|
||||
lifecycle_id, attribute_schema, schema_version, status
|
||||
FROM entity_types WHERE name = $1`, req.Body.Name).
|
||||
Scan(&et.Name, &et.ParentType, &et.IsAbstract, &et.Domain, &et.Layer,
|
||||
&et.Description, &et.LifecycleId, &schemaBytes, &et.SchemaVersion, &et.Status)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var schema map[string]any
|
||||
if len(schemaBytes) > 0 && json.Unmarshal(schemaBytes, &schema) == nil && schema != nil {
|
||||
et.AttributeSchema = &schema
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "create",
|
||||
nil, "POST", "/api/v1/ontology/entity-types", "",
|
||||
map[string]any{"name": req.Body.Name, "domain": req.Body.Domain}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.CreateEntityType201JSONResponse(et), nil
|
||||
}
|
||||
|
||||
func (s *Server) PatchEntityType(ctx context.Context, req gen.PatchEntityTypeRequestObject) (gen.PatchEntityTypeResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
// Build dynamic update.
|
||||
sets := []string{}
|
||||
args := []any{}
|
||||
argIdx := 2
|
||||
|
||||
if req.Body.Description != nil {
|
||||
sets = append(sets, fmt.Sprintf("description = $%d", argIdx))
|
||||
args = append(args, *req.Body.Description)
|
||||
argIdx++
|
||||
}
|
||||
if req.Body.Status != nil {
|
||||
sets = append(sets, fmt.Sprintf("status = $%d", argIdx))
|
||||
args = append(args, string(*req.Body.Status))
|
||||
argIdx++
|
||||
}
|
||||
if req.Body.AttributeSchema != nil {
|
||||
schemaJSON, _ := json.Marshal(req.Body.AttributeSchema)
|
||||
sets = append(sets, fmt.Sprintf("attribute_schema = $%d", argIdx))
|
||||
args = append(args, schemaJSON)
|
||||
argIdx++
|
||||
}
|
||||
|
||||
if len(sets) == 0 {
|
||||
return nil, fmt.Errorf("%w: no fields to update", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
sets = append(sets, "schema_version = schema_version + 1, updated_at = now()")
|
||||
|
||||
query := fmt.Sprintf(`UPDATE entity_types SET %s WHERE name = $1`, strings.Join(sets, ", "))
|
||||
finalArgs := append([]any{req.Name}, args...)
|
||||
|
||||
result, err := tx.Exec(ctx, query, finalArgs...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if result.RowsAffected() == 0 {
|
||||
return nil, fmt.Errorf("%w: entity type %q", domain.ErrNotFound, req.Name)
|
||||
}
|
||||
|
||||
// Re-read.
|
||||
var et gen.EntityType
|
||||
var schemaBytes []byte
|
||||
err = tx.QueryRow(ctx, `
|
||||
SELECT name, parent_type, is_abstract, domain, layer, description,
|
||||
lifecycle_id, attribute_schema, schema_version, status
|
||||
FROM entity_types WHERE name = $1`, req.Name).
|
||||
Scan(&et.Name, &et.ParentType, &et.IsAbstract, &et.Domain, &et.Layer,
|
||||
&et.Description, &et.LifecycleId, &schemaBytes, &et.SchemaVersion, &et.Status)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var schema map[string]any
|
||||
if len(schemaBytes) > 0 && json.Unmarshal(schemaBytes, &schema) == nil && schema != nil {
|
||||
et.AttributeSchema = &schema
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "patch",
|
||||
nil, "PATCH", "/api/v1/ontology/entity-types/"+req.Name, "",
|
||||
map[string]any{"status": req.Body.Status}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.PatchEntityType200JSONResponse(et), nil
|
||||
}
|
||||
267
internal/httpapi/executions.go
Normal file
267
internal/httpapi/executions.go
Normal file
@@ -0,0 +1,267 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// ─── Executions ────────────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) ListExecutions(ctx context.Context, req gen.ListExecutionsRequestObject) (gen.ListExecutionsResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT e.entity_id, e.classification_id::text, e.signal_entity_id::text,
|
||||
e.target_entity_id, e.action, e.risk_class,
|
||||
e.approval_id::text, e.agent_id::text, e.skill_id::text,
|
||||
e.skill_version, e.status, e.result, e.duration_ms,
|
||||
e.verified, e.correlation_id, e.started_at, e.completed_at, e.created_at,
|
||||
te.slug
|
||||
FROM executions e
|
||||
JOIN entities te ON te.id = e.target_entity_id
|
||||
WHERE ($1::text IS NULL OR e.status = $1)
|
||||
AND ($2::text IS NULL OR te.slug > $2)
|
||||
ORDER BY te.slug
|
||||
LIMIT $3`,
|
||||
req.Params.Status, req.Params.Cursor, limit+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.Execution{}
|
||||
for rows.Next() {
|
||||
var exec gen.Execution
|
||||
var resultBytes []byte
|
||||
var targetSlug string
|
||||
if err := rows.Scan(&exec.Id, &exec.ClassificationId, &exec.SignalId,
|
||||
&exec.Target, &exec.Action, &exec.RiskClass,
|
||||
&exec.ApprovalId, &exec.AgentId, &exec.SkillId,
|
||||
&exec.SkillVersion, &exec.Status, &resultBytes, &exec.DurationMs,
|
||||
&exec.Verified, &exec.CorrelationId, &exec.StartedAt, &exec.CompletedAt,
|
||||
&exec.CreatedAt, &targetSlug); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var result map[string]any
|
||||
if len(resultBytes) > 0 && json.Unmarshal(resultBytes, &result) == nil {
|
||||
exec.Result = &result
|
||||
}
|
||||
// Target is stored as UUID, but we surface the slug
|
||||
exec.Slug = targetSlug
|
||||
items = append(items, exec)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
var next *string
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
next = &items[len(items)-1].Slug
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.Execution{}
|
||||
}
|
||||
return gen.ListExecutions200JSONResponse{Items: items, NextCursor: next}, nil
|
||||
}
|
||||
|
||||
func (s *Server) GetExecution(ctx context.Context, req gen.GetExecutionRequestObject) (gen.GetExecutionResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var exec gen.Execution
|
||||
var resultBytes []byte
|
||||
var targetSlug string
|
||||
err = s.pool.QueryRow(ctx, `
|
||||
SELECT e.entity_id, e.classification_id::text, e.signal_entity_id::text,
|
||||
e.target_entity_id, e.action, e.risk_class,
|
||||
e.approval_id::text, e.agent_id::text, e.skill_id::text,
|
||||
e.skill_version, e.status, e.result, e.duration_ms,
|
||||
e.verified, e.correlation_id, e.started_at, e.completed_at, e.created_at,
|
||||
te.slug
|
||||
FROM executions e
|
||||
JOIN entities te ON te.id = e.target_entity_id
|
||||
WHERE e.entity_id = $1`, id).
|
||||
Scan(&exec.Id, &exec.ClassificationId, &exec.SignalId,
|
||||
&exec.Target, &exec.Action, &exec.RiskClass,
|
||||
&exec.ApprovalId, &exec.AgentId, &exec.SkillId,
|
||||
&exec.SkillVersion, &exec.Status, &resultBytes, &exec.DurationMs,
|
||||
&exec.Verified, &exec.CorrelationId, &exec.StartedAt, &exec.CompletedAt,
|
||||
&exec.CreatedAt, &targetSlug)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: execution %s", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
var result map[string]any
|
||||
if len(resultBytes) > 0 && json.Unmarshal(resultBytes, &result) == nil {
|
||||
exec.Result = &result
|
||||
}
|
||||
exec.Slug = targetSlug
|
||||
return gen.GetExecution200JSONResponse(exec), nil
|
||||
}
|
||||
|
||||
func (s *Server) RequestExecution(ctx context.Context, req gen.RequestExecutionRequestObject) (gen.RequestExecutionResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := uuid.NewV7()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
targetID, err := s.resolveEntityID(ctx, req.Body.Target)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
correlationID := uuid.New().String()
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
q := sqlcgen.New(tx)
|
||||
|
||||
// Full UUID, not a truncated prefix — an 8-char prefix of a UUIDv7
|
||||
// collides for real under back-to-back requests since the leading bytes
|
||||
// encode a millisecond timestamp (observed live via the MCP run tool).
|
||||
execSlug := "exec:" + id.String()
|
||||
if _, err := q.InsertEntity(ctx, sqlcgen.InsertEntityParams{
|
||||
ID: id,
|
||||
Slug: execSlug,
|
||||
Type: "execution",
|
||||
Name: req.Body.Action + " on " + req.Body.Target,
|
||||
Attributes: []byte("{}"),
|
||||
}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := q.InsertExecution(ctx, sqlcgen.InsertExecutionParams{
|
||||
EntityID: id,
|
||||
TargetEntityID: &targetID,
|
||||
Action: req.Body.Action,
|
||||
RiskClass: "unclassified", // will be classified by classifier
|
||||
CorrelationID: correlationID,
|
||||
}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Re-read to get the full record.
|
||||
var exec gen.Execution
|
||||
var resultBytes []byte
|
||||
var targetSlug string
|
||||
err = tx.QueryRow(ctx, `
|
||||
SELECT e.entity_id, e.classification_id::text, e.signal_entity_id::text,
|
||||
e.target_entity_id, e.action, e.risk_class,
|
||||
e.approval_id::text, e.agent_id::text, e.skill_id::text,
|
||||
e.skill_version, e.status, e.result, e.duration_ms,
|
||||
e.verified, e.correlation_id, e.started_at, e.completed_at, e.created_at,
|
||||
te.slug
|
||||
FROM executions e
|
||||
JOIN entities te ON te.id = e.target_entity_id
|
||||
WHERE e.entity_id = $1`, id).
|
||||
Scan(&exec.Id, &exec.ClassificationId, &exec.SignalId,
|
||||
&exec.Target, &exec.Action, &exec.RiskClass,
|
||||
&exec.ApprovalId, &exec.AgentId, &exec.SkillId,
|
||||
&exec.SkillVersion, &exec.Status, &resultBytes, &exec.DurationMs,
|
||||
&exec.Verified, &exec.CorrelationId, &exec.StartedAt, &exec.CompletedAt,
|
||||
&exec.CreatedAt, &targetSlug)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
exec.Slug = targetSlug
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "create",
|
||||
&id, "POST", "/api/v1/executions", "",
|
||||
map[string]any{"action": req.Body.Action, "target": req.Body.Target}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
if eventErr := observability.Event(ctx, q, "execution.requested", &id,
|
||||
"info", "oikos-api", "",
|
||||
map[string]any{"action": req.Body.Action, "target": req.Body.Target}); eventErr != nil {
|
||||
return nil, eventErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.RequestExecution201JSONResponse(exec), nil
|
||||
}
|
||||
|
||||
func (s *Server) CancelExecution(ctx context.Context, req gen.CancelExecutionRequestObject) (gen.CancelExecutionResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
q := sqlcgen.New(tx)
|
||||
if err := q.UpdateExecutionStatus(ctx, sqlcgen.UpdateExecutionStatusParams{
|
||||
EntityID: id,
|
||||
Status: "cancelled",
|
||||
}); err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: execution %s", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Re-read.
|
||||
var exec gen.Execution
|
||||
var resultBytes []byte
|
||||
var targetSlug string
|
||||
err = tx.QueryRow(ctx, `
|
||||
SELECT e.entity_id, e.classification_id::text, e.signal_entity_id::text,
|
||||
e.target_entity_id, e.action, e.risk_class,
|
||||
e.approval_id::text, e.agent_id::text, e.skill_id::text,
|
||||
e.skill_version, e.status, e.result, e.duration_ms,
|
||||
e.verified, e.correlation_id, e.started_at, e.completed_at, e.created_at,
|
||||
te.slug
|
||||
FROM executions e
|
||||
JOIN entities te ON te.id = e.target_entity_id
|
||||
WHERE e.entity_id = $1`, id).
|
||||
Scan(&exec.Id, &exec.ClassificationId, &exec.SignalId,
|
||||
&exec.Target, &exec.Action, &exec.RiskClass,
|
||||
&exec.ApprovalId, &exec.AgentId, &exec.SkillId,
|
||||
&exec.SkillVersion, &exec.Status, &resultBytes, &exec.DurationMs,
|
||||
&exec.Verified, &exec.CorrelationId, &exec.StartedAt, &exec.CompletedAt,
|
||||
&exec.CreatedAt, &targetSlug)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
exec.Slug = targetSlug
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "cancel",
|
||||
&id, "POST", "/api/v1/executions/"+req.Id+"/cancel", "",
|
||||
map[string]any{"status": "cancelled"}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.CancelExecution200JSONResponse(exec), nil
|
||||
}
|
||||
32
internal/httpapi/helpers.go
Normal file
32
internal/httpapi/helpers.go
Normal file
@@ -0,0 +1,32 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// ─── Helpers ───────────────────────────────────────────────────────────
|
||||
|
||||
func coalesceStr(s *string, def string) string {
|
||||
if s == nil || *s == "" {
|
||||
return def
|
||||
}
|
||||
return *s
|
||||
}
|
||||
|
||||
func parseIntOrZero(s string) (int, error) {
|
||||
var n int
|
||||
for _, c := range s {
|
||||
if c < '0' || c > '9' {
|
||||
return 0, fmt.Errorf("invalid integer: %q", s)
|
||||
}
|
||||
n = n*10 + int(c-'0')
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func hashToken(token string) string {
|
||||
h := sha256.Sum256([]byte(token))
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
@@ -60,19 +60,17 @@ func clampLimit(l *int) int {
|
||||
// resolveEntityID resolves a UUID-or-slug path/query value to the entity UUID.
|
||||
func (s *Server) resolveEntityID(ctx context.Context, idOrSlug string) (uuid.UUID, error) {
|
||||
if id, err := uuid.Parse(idOrSlug); err == nil {
|
||||
var found uuid.UUID
|
||||
err := s.pool.QueryRow(ctx, "SELECT id FROM entities WHERE id = $1", id).Scan(&found)
|
||||
if err == pgx.ErrNoRows {
|
||||
entity, err := sqlcgen.New(s.pool).GetEntityByID(ctx, id)
|
||||
if err != nil {
|
||||
return uuid.Nil, fmt.Errorf("%w: %s", domain.ErrNotFound, idOrSlug)
|
||||
}
|
||||
return found, err
|
||||
return entity.ID, nil
|
||||
}
|
||||
var id uuid.UUID
|
||||
err := s.pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", idOrSlug).Scan(&id)
|
||||
if err == pgx.ErrNoRows {
|
||||
entity, err := sqlcgen.New(s.pool).GetEntityBySlug(ctx, idOrSlug)
|
||||
if err != nil {
|
||||
return uuid.Nil, fmt.Errorf("%w: %s", domain.ErrNotFound, idOrSlug)
|
||||
}
|
||||
return id, err
|
||||
return entity.ID, nil
|
||||
}
|
||||
|
||||
// entityCols requires the entities table to be aliased as `e`, with
|
||||
@@ -188,46 +186,37 @@ func (s *Server) GetEntityRelations(ctx context.Context, req gen.GetEntityRelati
|
||||
if req.Params.Direction != nil {
|
||||
dir = string(*req.Params.Direction)
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT se.slug, te.slug, r.type, r.attributes, r.valid_from, r.valid_to
|
||||
FROM relationships r
|
||||
JOIN entities se ON se.id = r.source_id
|
||||
JOIN entities te ON te.id = r.target_id
|
||||
WHERE r.valid_to IS NULL
|
||||
AND (($3 IN ('out','both') AND r.source_id = $1)
|
||||
OR ($3 IN ('in','both') AND r.target_id = $1))
|
||||
AND ($2::text IS NULL OR r.type = $2)
|
||||
ORDER BY r.type, se.slug, te.slug`,
|
||||
id, req.Params.RelType, dir)
|
||||
relType := req.Params.RelType
|
||||
rows, err := sqlcgen.New(s.pool).ListEntityRelations(ctx, sqlcgen.ListEntityRelationsParams{
|
||||
Direction: dir,
|
||||
ID: id,
|
||||
RelType: relType,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items, err := scanRelationships(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
items := []gen.Relationship{}
|
||||
for _, r := range rows {
|
||||
var attrs *map[string]any
|
||||
if len(r.Attributes) > 0 {
|
||||
var m map[string]any
|
||||
if json.Unmarshal(r.Attributes, &m) == nil && len(m) > 0 {
|
||||
attrs = &m
|
||||
}
|
||||
}
|
||||
validTo := r.ValidTo
|
||||
items = append(items, gen.Relationship{
|
||||
Source: r.SourceSlug,
|
||||
Target: r.TargetSlug,
|
||||
Type: r.Type,
|
||||
Attributes: attrs,
|
||||
ValidFrom: r.ValidFrom,
|
||||
ValidTo: validTo,
|
||||
})
|
||||
}
|
||||
return gen.GetEntityRelations200JSONResponse{Items: items}, nil
|
||||
}
|
||||
|
||||
func scanRelationships(rows pgx.Rows) ([]gen.Relationship, error) {
|
||||
defer rows.Close()
|
||||
items := []gen.Relationship{}
|
||||
for rows.Next() {
|
||||
var rel gen.Relationship
|
||||
var attrsJSON []byte
|
||||
if err := rows.Scan(&rel.Source, &rel.Target, &rel.Type,
|
||||
&attrsJSON, &rel.ValidFrom, &rel.ValidTo); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var attrs map[string]any
|
||||
if len(attrsJSON) > 0 && json.Unmarshal(attrsJSON, &attrs) == nil && len(attrs) > 0 {
|
||||
rel.Attributes = &attrs
|
||||
}
|
||||
items = append(items, rel)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Server) GetBlastRadius(ctx context.Context, req gen.GetBlastRadiusRequestObject) (gen.GetBlastRadiusResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
@@ -293,6 +282,15 @@ func (s *Server) GetGraph(ctx context.Context, req gen.GetGraphRequestObject) (g
|
||||
var err error
|
||||
truncated := false
|
||||
|
||||
// pgx can't infer the array element type from a nil *[]string (the
|
||||
// param is absent from the request, not an empty list), so dereference
|
||||
// to a plain []string first — nil there still encodes as SQL NULL, but
|
||||
// pgx has a concrete type to work with.
|
||||
var relTypes []string
|
||||
if req.Params.RelType != nil {
|
||||
relTypes = *req.Params.RelType
|
||||
}
|
||||
|
||||
if req.Params.Root != nil && *req.Params.Root != "" {
|
||||
rootID, rerr := s.resolveEntityID(ctx, *req.Params.Root)
|
||||
if rerr != nil {
|
||||
@@ -302,12 +300,26 @@ func (s *Server) GetGraph(ctx context.Context, req gen.GetGraphRequestObject) (g
|
||||
SELECT `+entityCols+`
|
||||
FROM blast_radius($1, $2, $3) b JOIN entities e ON e.id = b.entity_id
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY e.slug`, rootID, depth, req.Params.RelType)
|
||||
ORDER BY e.slug`, rootID, depth, relTypes)
|
||||
} else {
|
||||
// Whole-graph view: pick the most-connected entities first so the
|
||||
// graph shows actual topology, not just whatever sorts first
|
||||
// alphabetically. Without this the cap fills with exec:* rows and
|
||||
// drops every host/lxc/service/vm — and every edge those entities
|
||||
// connect — because edges require both endpoints in the node set.
|
||||
nodes, err = s.queryEntities(ctx, `
|
||||
SELECT `+entityCols+` FROM entities e
|
||||
SELECT `+entityCols+`
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY e.slug LIMIT $1`,
|
||||
WHERE e.id IN (
|
||||
SELECT e2.id FROM entities e2
|
||||
LEFT JOIN relationships r ON r.valid_to IS NULL
|
||||
AND (r.source_id = e2.id OR r.target_id = e2.id)
|
||||
GROUP BY e2.id
|
||||
ORDER BY count(r.type) DESC, e2.slug
|
||||
LIMIT $1
|
||||
)
|
||||
ORDER BY e.slug`,
|
||||
graphNodeCap+1)
|
||||
if err == nil && len(nodes) > graphNodeCap {
|
||||
nodes = nodes[:graphNodeCap]
|
||||
@@ -322,21 +334,31 @@ func (s *Server) GetGraph(ctx context.Context, req gen.GetGraphRequestObject) (g
|
||||
for i, n := range nodes {
|
||||
ids[i] = uuid.UUID(n.Id)
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT se.slug, te.slug, r.type, r.attributes, r.valid_from, r.valid_to
|
||||
FROM relationships r
|
||||
JOIN entities se ON se.id = r.source_id
|
||||
JOIN entities te ON te.id = r.target_id
|
||||
WHERE r.valid_to IS NULL
|
||||
AND r.source_id = ANY($1) AND r.target_id = ANY($1)
|
||||
AND ($2::text[] IS NULL OR r.type = ANY($2))
|
||||
ORDER BY r.type, se.slug, te.slug`, ids, req.Params.RelType)
|
||||
edgeRows, err := sqlcgen.New(s.pool).ListGraphEdges(ctx, sqlcgen.ListGraphEdgesParams{
|
||||
Ids: ids,
|
||||
RelTypes: relTypes,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
edges, err := scanRelationships(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
edges := []gen.Relationship{}
|
||||
for _, r := range edgeRows {
|
||||
var attrs *map[string]any
|
||||
if len(r.Attributes) > 0 {
|
||||
var m map[string]any
|
||||
if json.Unmarshal(r.Attributes, &m) == nil && len(m) > 0 {
|
||||
attrs = &m
|
||||
}
|
||||
}
|
||||
validTo := r.ValidTo
|
||||
edges = append(edges, gen.Relationship{
|
||||
Source: r.SourceSlug,
|
||||
Target: r.TargetSlug,
|
||||
Type: r.Type,
|
||||
Attributes: attrs,
|
||||
ValidFrom: r.ValidFrom,
|
||||
ValidTo: validTo,
|
||||
})
|
||||
}
|
||||
|
||||
resp := gen.GetGraph200JSONResponse{Nodes: nodes, Edges: edges}
|
||||
@@ -407,78 +429,70 @@ func (s *Server) GetOntology(ctx context.Context, req gen.GetOntologyRequestObje
|
||||
Lifecycles: []gen.LifecycleDef{},
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT name, parent_type, is_abstract, domain, layer, description,
|
||||
lifecycle_id, attribute_schema, schema_version, status
|
||||
FROM entity_types ORDER BY name`)
|
||||
q := sqlcgen.New(s.pool)
|
||||
|
||||
etRows, err := q.ListEntityTypes(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var et gen.EntityType
|
||||
var schemaVersion int
|
||||
var schemaJSON []byte
|
||||
if err := rows.Scan(&et.Name, &et.ParentType, &et.IsAbstract, &et.Domain,
|
||||
&et.Layer, &et.Description, &et.LifecycleId, &schemaJSON,
|
||||
&schemaVersion, &et.Status); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
for _, et := range etRows {
|
||||
schemaVersion := int(et.SchemaVersion)
|
||||
var schema *map[string]any
|
||||
if len(et.AttributeSchema) > 0 {
|
||||
var s map[string]any
|
||||
if json.Unmarshal(et.AttributeSchema, &s) == nil && s != nil {
|
||||
schema = &s
|
||||
}
|
||||
}
|
||||
et.SchemaVersion = &schemaVersion
|
||||
var schema map[string]any
|
||||
if len(schemaJSON) > 0 && json.Unmarshal(schemaJSON, &schema) == nil && schema != nil {
|
||||
et.AttributeSchema = &schema
|
||||
}
|
||||
resp.EntityTypes = append(resp.EntityTypes, et)
|
||||
}
|
||||
rows.Close()
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
resp.EntityTypes = append(resp.EntityTypes, gen.EntityType{
|
||||
Name: et.Name,
|
||||
ParentType: et.ParentType,
|
||||
IsAbstract: et.IsAbstract,
|
||||
Domain: et.Domain,
|
||||
Layer: gen.EntityTypeLayer(et.Layer),
|
||||
Description: et.Description,
|
||||
LifecycleId: et.LifecycleID,
|
||||
SchemaVersion: &schemaVersion,
|
||||
AttributeSchema: schema,
|
||||
Status: gen.EntityTypeStatus(et.Status),
|
||||
})
|
||||
}
|
||||
|
||||
rows, err = s.pool.Query(ctx, `
|
||||
SELECT name, inverse, source_type, target_type, cardinality, description
|
||||
FROM relationship_types ORDER BY name`)
|
||||
rtRows, err := q.ListRelationshipTypes(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var rt gen.RelationshipType
|
||||
if err := rows.Scan(&rt.Name, &rt.Inverse, &rt.SourceType, &rt.TargetType,
|
||||
&rt.Cardinality, &rt.Description); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
resp.RelationshipTypes = append(resp.RelationshipTypes, rt)
|
||||
}
|
||||
rows.Close()
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
for _, rt := range rtRows {
|
||||
resp.RelationshipTypes = append(resp.RelationshipTypes, gen.RelationshipType{
|
||||
Name: rt.Name,
|
||||
Inverse: rt.Inverse,
|
||||
SourceType: rt.SourceType,
|
||||
TargetType: rt.TargetType,
|
||||
Cardinality: gen.RelationshipTypeCardinality(rt.Cardinality),
|
||||
Description: rt.Description,
|
||||
})
|
||||
}
|
||||
|
||||
rows, err = s.pool.Query(ctx, `
|
||||
SELECT id, states, default_state, terminal_states, transitions
|
||||
FROM lifecycle_defs ORDER BY id`)
|
||||
lcRows, err := q.ListLifecycleDefs(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var lc gen.LifecycleDef
|
||||
var terminal []string
|
||||
var transJSON []byte
|
||||
if err := rows.Scan(&lc.Id, &lc.States, &lc.DefaultState, &terminal, &transJSON); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
for _, lc := range lcRows {
|
||||
terminal := lc.TerminalStates
|
||||
var transitions map[string]any
|
||||
if err := json.Unmarshal(lc.Transitions, &transitions); err != nil {
|
||||
return nil, fmt.Errorf("lifecycle %s transitions: %w", lc.ID, err)
|
||||
}
|
||||
lc.TerminalStates = &terminal
|
||||
if err := json.Unmarshal(transJSON, &lc.Transitions); err != nil {
|
||||
rows.Close()
|
||||
return nil, fmt.Errorf("lifecycle %s transitions: %w", lc.Id, err)
|
||||
}
|
||||
resp.Lifecycles = append(resp.Lifecycles, lc)
|
||||
resp.Lifecycles = append(resp.Lifecycles, gen.LifecycleDef{
|
||||
Id: lc.ID,
|
||||
States: lc.States,
|
||||
DefaultState: lc.DefaultState,
|
||||
TerminalStates: &terminal,
|
||||
Transitions: transitions,
|
||||
})
|
||||
}
|
||||
rows.Close()
|
||||
return resp, rows.Err()
|
||||
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
// ─── Signals ──────────────────────────────────────────────────────────
|
||||
@@ -1591,10 +1605,9 @@ func checkPrecondition(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, entit
|
||||
}
|
||||
}
|
||||
case "health-check-answering":
|
||||
var health string
|
||||
err := tx.QueryRow(ctx, "SELECT health FROM entity_status WHERE entity_id = $1", entityID).Scan(&health)
|
||||
if err != nil || health == "unknown" || health == "down" {
|
||||
return fmt.Errorf("health check not answering (status: %s)", health)
|
||||
st, err := sqlcgen.New(tx).GetEntityStatus(ctx, entityID)
|
||||
if err != nil || st.Health == "unknown" || st.Health == "down" {
|
||||
return fmt.Errorf("health check not answering (status: %s)", st.Health)
|
||||
}
|
||||
case "doc-page-complete":
|
||||
var count int
|
||||
|
||||
180
internal/httpapi/metrics.go
Normal file
180
internal/httpapi/metrics.go
Normal file
@@ -0,0 +1,180 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
// ─── Metrics ───────────────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) QueryMetrics(ctx context.Context, req gen.QueryMetricsRequestObject) (gen.QueryMetricsResponseObject, error) {
|
||||
if req.Params.EntityId == nil || *req.Params.EntityId == "" {
|
||||
return nil, fmt.Errorf("%w: entity_id is required", domain.ErrInvalidInput)
|
||||
}
|
||||
entityID, err := s.resolveEntityID(ctx, *req.Params.EntityId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
from := time.Now().Add(-24 * time.Hour)
|
||||
if req.Params.From != nil {
|
||||
from = *req.Params.From
|
||||
}
|
||||
to := time.Now()
|
||||
if req.Params.To != nil {
|
||||
to = *req.Params.To
|
||||
}
|
||||
|
||||
var metricNames []string
|
||||
if req.Params.Metric != nil && len(*req.Params.Metric) > 0 {
|
||||
metricNames = *req.Params.Metric
|
||||
} else {
|
||||
// metric omitted: report every metric recorded for this entity in range.
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT DISTINCT metric FROM metric_samples
|
||||
WHERE entity_id = $1 AND ts >= $2 AND ts <= $3
|
||||
ORDER BY metric`, entityID, from, to)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for rows.Next() {
|
||||
var name string
|
||||
if err := rows.Scan(&name); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
metricNames = append(metricNames, name)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
items := []gen.MetricSeries{}
|
||||
for _, metricName := range metricNames {
|
||||
series := gen.MetricSeries{
|
||||
EntityId: entityID.String(),
|
||||
Metric: metricName,
|
||||
Rollup: gen.MetricSeriesRollupRaw,
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT ts, value
|
||||
FROM metric_samples
|
||||
WHERE entity_id = $1 AND metric = $2
|
||||
AND ts >= $3 AND ts <= $4
|
||||
ORDER BY ts ASC`,
|
||||
entityID, metricName, from, to)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
samples := []struct {
|
||||
Avg *float32 `json:"avg"`
|
||||
Count *int `json:"count"`
|
||||
Max *float32 `json:"max"`
|
||||
Min *float32 `json:"min"`
|
||||
Ts time.Time `json:"ts"`
|
||||
Value *float32 `json:"value"`
|
||||
}{}
|
||||
|
||||
for rows.Next() {
|
||||
var ts time.Time
|
||||
var val float64
|
||||
if err := rows.Scan(&ts, &val); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
f := float32(val)
|
||||
samples = append(samples, struct {
|
||||
Avg *float32 `json:"avg"`
|
||||
Count *int `json:"count"`
|
||||
Max *float32 `json:"max"`
|
||||
Min *float32 `json:"min"`
|
||||
Ts time.Time `json:"ts"`
|
||||
Value *float32 `json:"value"`
|
||||
}{Value: &f, Ts: ts})
|
||||
}
|
||||
rows.Close()
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
series.Samples = samples
|
||||
items = append(items, series)
|
||||
}
|
||||
|
||||
if items == nil {
|
||||
items = []gen.MetricSeries{}
|
||||
}
|
||||
return gen.QueryMetrics200JSONResponse{Items: items}, nil
|
||||
}
|
||||
|
||||
func (s *Server) GetTrends(ctx context.Context, req gen.GetTrendsRequestObject) (gen.GetTrendsResponseObject, error) {
|
||||
entityID, err := s.resolveEntityID(ctx, req.EntityId)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
from := time.Now().Add(-7 * 24 * time.Hour)
|
||||
if req.Params.From != nil {
|
||||
from = *req.Params.From
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT metric,
|
||||
ROUND(avg(value)::numeric, 2) AS avg_val,
|
||||
ROUND(stddev(value)::numeric, 2) AS std_val,
|
||||
count(*) AS sample_count,
|
||||
ROUND(regr_slope(value, EXTRACT(EPOCH FROM ts)::numeric)::numeric, 4) AS slope
|
||||
FROM metric_samples
|
||||
WHERE entity_id = $1 AND ts >= $2
|
||||
GROUP BY metric
|
||||
ORDER BY metric`, entityID, from)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.Trend{}
|
||||
for rows.Next() {
|
||||
var t gen.Trend
|
||||
var avgVal, stdVal, slopeNum pgtype.Numeric
|
||||
var sampleCount int
|
||||
if err := rows.Scan(&t.Metric, &avgVal, &stdVal, &sampleCount, &slopeNum); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Determine direction.
|
||||
if slopeNum.Valid {
|
||||
f, _ := slopeNum.Float64Value()
|
||||
t.Slope = float32Ptr(float32(f.Float64))
|
||||
if f.Float64 > 0.01 {
|
||||
t.Direction = gen.Improving
|
||||
} else if f.Float64 < -0.01 {
|
||||
t.Direction = gen.Degrading
|
||||
} else {
|
||||
t.Direction = gen.Stable
|
||||
}
|
||||
} else {
|
||||
t.Direction = gen.Unknown
|
||||
}
|
||||
items = append(items, t)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.Trend{}
|
||||
}
|
||||
return gen.GetTrends200JSONResponse{Items: items}, nil
|
||||
}
|
||||
|
||||
func float32Ptr(f float32) *float32 {
|
||||
return &f
|
||||
}
|
||||
123
internal/httpapi/patterns.go
Normal file
123
internal/httpapi/patterns.go
Normal file
@@ -0,0 +1,123 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// ─── Patterns ──────────────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) ListPatterns(ctx context.Context, req gen.ListPatternsRequestObject) (gen.ListPatternsResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT p.entity_id, e.slug, p.applies_type, p.action, p.pattern, p.confidence,
|
||||
p.evidence_count, p.success_count, p.failure_count, p.status,
|
||||
p.quarantined, p.version, p.last_validated_at
|
||||
FROM patterns p
|
||||
JOIN entities e ON e.id = p.entity_id
|
||||
WHERE ($1::text IS NULL OR p.status = $1)
|
||||
AND ($2::text IS NULL OR p.applies_type = $2)
|
||||
AND ($3::text IS NULL OR p.action = $3)
|
||||
ORDER BY p.applies_type, p.action
|
||||
LIMIT $4`,
|
||||
req.Params.Status, req.Params.EntityType, req.Params.Action, limit+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.Pattern{}
|
||||
for rows.Next() {
|
||||
var p gen.Pattern
|
||||
if err := rows.Scan(&p.Id, &p.Slug, &p.AppliesType, &p.Action, &p.Pattern,
|
||||
&p.Confidence, &p.EvidenceCount, &p.SuccessCount, &p.FailureCount,
|
||||
&p.Status, &p.Quarantined, &p.Version, &p.LastValidatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, p)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
if items == nil {
|
||||
items = []gen.Pattern{}
|
||||
}
|
||||
return gen.ListPatterns200JSONResponse{Items: items}, nil
|
||||
}
|
||||
|
||||
func (s *Server) PatchPattern(ctx context.Context, req gen.PatchPatternRequestObject) (gen.PatchPatternResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
q := sqlcgen.New(tx)
|
||||
|
||||
if req.Body.Status != nil {
|
||||
status := string(*req.Body.Status)
|
||||
if err := q.UpdatePatternStatus(ctx, sqlcgen.UpdatePatternStatusParams{
|
||||
EntityID: id,
|
||||
Status: status,
|
||||
}); err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: pattern %s", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
if req.Body.Quarantined != nil {
|
||||
if err := q.UpdatePatternQuarantine(ctx, sqlcgen.UpdatePatternQuarantineParams{
|
||||
EntityID: id,
|
||||
Quarantined: *req.Body.Quarantined,
|
||||
}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
// Re-read.
|
||||
var p gen.Pattern
|
||||
err = tx.QueryRow(ctx, `
|
||||
SELECT entity_id, applies_type, action, pattern, confidence,
|
||||
evidence_count, success_count, failure_count, status,
|
||||
quarantined, version, last_validated_at
|
||||
FROM patterns WHERE entity_id = $1`, id).
|
||||
Scan(&p.Id, &p.AppliesType, &p.Action, &p.Pattern,
|
||||
&p.Confidence, &p.EvidenceCount, &p.SuccessCount, &p.FailureCount,
|
||||
&p.Status, &p.Quarantined, &p.Version, &p.LastValidatedAt)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: pattern %s", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "patch",
|
||||
&id, "PATCH", "/api/v1/patterns/"+req.Id, "",
|
||||
map[string]any{"status": req.Body.Status, "quarantined": req.Body.Quarantined}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.PatchPattern200JSONResponse(p), nil
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
120
internal/httpapi/relationships.go
Normal file
120
internal/httpapi/relationships.go
Normal file
@@ -0,0 +1,120 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
)
|
||||
|
||||
// ─── Relationships ─────────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) CreateRelationship(ctx context.Context, req gen.CreateRelationshipRequestObject) (gen.CreateRelationshipResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
sourceID, err := s.resolveEntityID(ctx, req.Body.Source)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
targetID, err := s.resolveEntityID(ctx, req.Body.Target)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
attrsJSON := []byte("{}")
|
||||
if req.Body.Attributes != nil {
|
||||
attrsJSON, _ = json.Marshal(req.Body.Attributes)
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
_, err = tx.Exec(ctx, `
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
VALUES ($1, $2, $3, $4, now())`,
|
||||
sourceID, targetID, req.Body.Type, attrsJSON)
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
|
||||
return nil, fmt.Errorf("%w: relationship %s:%s:%s already exists",
|
||||
domain.ErrAlreadyExists, req.Body.Source, req.Body.Type, req.Body.Target)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rel := gen.Relationship{
|
||||
Source: req.Body.Source,
|
||||
Target: req.Body.Target,
|
||||
Type: req.Body.Type,
|
||||
ValidFrom: time.Now(),
|
||||
}
|
||||
if req.Body.Attributes != nil {
|
||||
rel.Attributes = req.Body.Attributes
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "create",
|
||||
nil, "POST", "/api/v1/relationships", "",
|
||||
map[string]any{"source": req.Body.Source, "target": req.Body.Target, "type": req.Body.Type}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.CreateRelationship201JSONResponse(rel), nil
|
||||
}
|
||||
|
||||
func (s *Server) EndRelationship(ctx context.Context, req gen.EndRelationshipRequestObject) (gen.EndRelationshipResponseObject, error) {
|
||||
sourceID, err := s.resolveEntityID(ctx, req.Params.Source)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
targetID, err := s.resolveEntityID(ctx, req.Params.Target)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
result, err := sqlcgen.New(tx).EndCurrentRelationship(ctx, sqlcgen.EndCurrentRelationshipParams{
|
||||
SourceID: sourceID,
|
||||
TargetID: targetID,
|
||||
Type: req.Params.RelType,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if result == 0 {
|
||||
return nil, fmt.Errorf("%w: active relationship %s:%s:%s",
|
||||
domain.ErrNotFound, req.Params.Source, req.Params.RelType, req.Params.Target)
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "delete",
|
||||
nil, "DELETE", "/api/v1/relationships", "",
|
||||
map[string]any{"source": req.Params.Source, "target": req.Params.Target, "type": req.Params.RelType}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.EndRelationship204Response{}, nil
|
||||
}
|
||||
33
internal/httpapi/risk_classes.go
Normal file
33
internal/httpapi/risk_classes.go
Normal file
@@ -0,0 +1,33 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
)
|
||||
|
||||
// ─── Risk Classes ──────────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) ListRiskClasses(ctx context.Context, req gen.ListRiskClassesRequestObject) (gen.ListRiskClassesResponseObject, error) {
|
||||
rows, err := s.pool.Query(ctx, `SELECT name, description, approval_required, autonomy_allowed FROM risk_classes ORDER BY name`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.RiskClass{}
|
||||
for rows.Next() {
|
||||
var rc gen.RiskClass
|
||||
if err := rows.Scan(&rc.Name, &rc.Description, &rc.ApprovalRequired, &rc.AutonomyAllowed); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, rc)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.RiskClass{}
|
||||
}
|
||||
return gen.ListRiskClasses200JSONResponse{Items: items}, nil
|
||||
}
|
||||
@@ -97,6 +97,25 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
|
||||
MaxAge: 86400,
|
||||
}))
|
||||
|
||||
// ─── Non-OpenAPI routes (carve-out) ────────────────────────────────
|
||||
//
|
||||
// These routes are registered manually on the chi router rather than
|
||||
// generated from api/openapi.yaml. Each has a structural reason it
|
||||
// can't go through the strict-server codegen:
|
||||
//
|
||||
// /healthz — infra liveness probe, no auth, no /api/v1 prefix
|
||||
// /api/v1/auth/oidc-* — auth flow, must run before auth middleware
|
||||
// /oidc-callback — standalone HTML page, not a JSON API
|
||||
// /api/v1/events/stream — in OpenAPI but re-registered for SSE Flush()
|
||||
// /api/v1/knowledge/recent — ad-hoc aggregation, no schema type yet
|
||||
// /api/v1/knowledge/content/{id} — returns raw markdown, not a gen type
|
||||
// /api/v1/activity/recent — recency-ordered, not paginated
|
||||
// /api/v1/activity/session/{id} — session-scoped aggregation
|
||||
// /api/v1/learning/timeline — derived view, no backing schema type
|
||||
// /api/v1/learning/trend — derived view, no backing schema type
|
||||
//
|
||||
// See .agents/dev/CONTRIBUTING.md §OpenAPI codegen for the policy.
|
||||
|
||||
// Liveness — no auth, no audit (plan SG18). Not exposed via Caddy.
|
||||
r.Get("/healthz", func(w http.ResponseWriter, req *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(req.Context(), 2*time.Second)
|
||||
@@ -138,6 +157,13 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
|
||||
s.serveOIDCToken(w, req, cfg)
|
||||
})
|
||||
|
||||
// Desktop OIDC callback — standalone HTML page that exchanges the
|
||||
// authorization code for tokens and displays the access token to copy
|
||||
// into the desktop app's Config screen.
|
||||
r.Get("/oidc-callback", func(w http.ResponseWriter, req *http.Request) {
|
||||
s.serveOIDCCallback(w, req, cfg)
|
||||
})
|
||||
|
||||
strict := gen.NewStrictHandlerWithOptions(s, nil, gen.StrictHTTPServerOptions{
|
||||
RequestErrorHandlerFunc: func(w http.ResponseWriter, req *http.Request, err error) {
|
||||
writeProblem(w, req, http.StatusBadRequest, "bad request", err.Error())
|
||||
@@ -166,23 +192,27 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
|
||||
// Custom (non-OpenAPI) route: recency-ordered knowledge + stats for the
|
||||
// Knowledge page's "what the system has learned" view. Registered after
|
||||
// HandlerWithOptions so it wins over any generated catch-all.
|
||||
// (See "Non-OpenAPI routes" carve-out block above.)
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/knowledge/recent", s.serveRecentKnowledge)
|
||||
|
||||
// Custom (non-OpenAPI) route: full markdown content for a knowledge
|
||||
// entity (document/investigation/runbook) by its own id or slug — the
|
||||
// generated /api/v1/knowledge/{id} route (GetEntityKnowledge) answers a
|
||||
// different question (knowledge referencing this entity), not this one.
|
||||
// (See "Non-OpenAPI routes" carve-out block above.)
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/knowledge/content/{id}", s.serveKnowledgeContent)
|
||||
|
||||
// Custom (non-OpenAPI) routes: the global activity feed (recency-ordered,
|
||||
// unlike ListExecutions which sorts by target for pagination) and the
|
||||
// per-session "what did this session do" digest.
|
||||
// (See "Non-OpenAPI routes" carve-out block above.)
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/activity/recent", s.serveRecentActivity)
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/activity/session/{id}", s.serveSessionDigest)
|
||||
|
||||
// Learning view: capability timeline + success trend, both derived from
|
||||
// executions (real, growing data) rather than the patterns/skills tables,
|
||||
// which are correctly modeled but have no writers anywhere yet.
|
||||
// (See "Non-OpenAPI routes" carve-out block above.)
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/learning/timeline", s.serveLearningTimeline)
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/learning/trend", s.serveLearningTrend)
|
||||
|
||||
@@ -657,6 +687,148 @@ func (s *Server) serveOIDCToken(w http.ResponseWriter, req *http.Request, cfg co
|
||||
w.Write(respBody)
|
||||
}
|
||||
|
||||
// serveOIDCCallback serves a standalone HTML page that completes the
|
||||
// desktop OIDC login flow. Authentik redirects here with ?code=...&state=...
|
||||
// after the user authorizes. The state carries the PKCE verifier
|
||||
// (base64url-encoded, joined with "."). The page exchanges the code for
|
||||
// tokens via the token proxy, then displays the access token for the user
|
||||
// to copy into the desktop app.
|
||||
func (s *Server) serveOIDCCallback(w http.ResponseWriter, req *http.Request, cfg config.Config) {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
fmt.Fprint(w, `<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Oikos — Connect Desktop App</title>
|
||||
<style>
|
||||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
background: #0a0a0a; color: #e0e0e0;
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
min-height: 100vh; padding: 24px;
|
||||
}
|
||||
.card {
|
||||
background: #1a1a1a; border: 1px solid #2a2a2a;
|
||||
border-radius: 12px; padding: 32px; max-width: 480px; width: 100%;
|
||||
}
|
||||
h1 { font-size: 20px; margin-bottom: 8px; }
|
||||
p { font-size: 14px; color: #888; margin-bottom: 20px; }
|
||||
.spinner { margin: 24px auto; width: 32px; height: 32px; border: 3px solid #2a2a2a; border-top-color: #3b82f6; border-radius: 50%; animation: spin 0.8s linear infinite; }
|
||||
@keyframes spin { to { transform: rotate(360deg); } }
|
||||
.token-box {
|
||||
background: #111; border: 1px solid #2a2a2a; border-radius: 8px;
|
||||
padding: 16px; font-family: monospace; font-size: 13px;
|
||||
word-break: break-all; margin-bottom: 16px; position: relative;
|
||||
max-height: 160px; overflow-y: auto;
|
||||
}
|
||||
.btn {
|
||||
display: block; width: 100%; padding: 12px; border: none; border-radius: 8px;
|
||||
font-size: 14px; font-weight: 600; cursor: pointer; text-align: center;
|
||||
}
|
||||
.btn-primary { background: #3b82f6; color: #fff; }
|
||||
.btn-primary:hover { background: #2563eb; }
|
||||
.btn-secondary { background: #1a1a1a; color: #e0e0e0; border: 1px solid #2a2a2a; margin-top: 8px; }
|
||||
.btn-secondary:hover { background: #222; }
|
||||
.success { color: #22c55e; margin-bottom: 8px; font-weight: 600; }
|
||||
.error { color: #ef4444; margin-bottom: 12px; }
|
||||
.copied { color: #22c55e; font-size: 13px; text-align: center; margin-top: 8px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="card">
|
||||
<h1>Connect Desktop App</h1>
|
||||
<div id="loading">
|
||||
<p>Exchanging authorization code...</p>
|
||||
<div class="spinner"></div>
|
||||
</div>
|
||||
<div id="result" style="display:none"></div>
|
||||
</div>
|
||||
<script>
|
||||
async function main() {
|
||||
const params = new URLSearchParams(location.search);
|
||||
const code = params.get('code');
|
||||
const state = params.get('state');
|
||||
|
||||
if (!code || !state) {
|
||||
showError('Missing code or state parameter from Authentik redirect.');
|
||||
return;
|
||||
}
|
||||
|
||||
const parts = state.split('.');
|
||||
if (parts.length !== 2) {
|
||||
showError('Invalid state format.');
|
||||
return;
|
||||
}
|
||||
const [csrf, verifier] = parts;
|
||||
|
||||
const redirectURI = location.origin + '/oidc-callback';
|
||||
|
||||
try {
|
||||
const resp = await fetch('/api/v1/auth/oidc-token', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
grant_type: 'authorization_code',
|
||||
code: code,
|
||||
code_verifier: verifier,
|
||||
redirect_uri: redirectURI
|
||||
})
|
||||
});
|
||||
|
||||
if (!resp.ok) {
|
||||
const err = await resp.json().catch(() => ({ error: resp.statusText }));
|
||||
showError(err.error || err.message || 'Token exchange failed (' + resp.status + ')');
|
||||
return;
|
||||
}
|
||||
|
||||
const tokens = await resp.json();
|
||||
if (!tokens.access_token) {
|
||||
showError('No access token in response.');
|
||||
return;
|
||||
}
|
||||
|
||||
document.getElementById('loading').style.display = 'none';
|
||||
const result = document.getElementById('result');
|
||||
result.style.display = 'block';
|
||||
result.innerHTML = '<div class="success">Authentication successful</div>' +
|
||||
'<p style="margin-bottom:8px">Copy this token into the Oikos desktop app Token tab:</p>' +
|
||||
'<div class="token-box" id="token">' + escapeHtml(tokens.access_token) + '</div>' +
|
||||
'<button class="btn btn-primary" id="copyBtn">Copy Token</button>' +
|
||||
'<button class="btn btn-secondary" onclick="location.reload()">Try Again</button>' +
|
||||
'<div class="copied" id="copied" style="display:none">Copied!</div>';
|
||||
|
||||
document.getElementById('copyBtn').addEventListener('click', () => {
|
||||
navigator.clipboard.writeText(tokens.access_token).then(() => {
|
||||
const el = document.getElementById('copied');
|
||||
el.style.display = 'block';
|
||||
setTimeout(() => el.style.display = 'none', 2000);
|
||||
});
|
||||
});
|
||||
} catch(e) {
|
||||
showError('Network error: ' + e.message);
|
||||
}
|
||||
}
|
||||
|
||||
function showError(msg) {
|
||||
document.getElementById('loading').style.display = 'none';
|
||||
const result = document.getElementById('result');
|
||||
result.style.display = 'block';
|
||||
result.innerHTML = '<div class="error">' + escapeHtml(msg) + '</div>' +
|
||||
'<button class="btn btn-secondary" onclick="location.reload()">Try Again</button>';
|
||||
}
|
||||
|
||||
function escapeHtml(s) {
|
||||
return s.replace(/&/g,'&').replace(/</g,'<').replace(/>/g,'>').replace(/"/g,'"');
|
||||
}
|
||||
|
||||
main();
|
||||
</script>
|
||||
</body>
|
||||
</html>`)
|
||||
}
|
||||
|
||||
// ListenAndServe runs the API server with graceful shutdown on ctx cancel
|
||||
// (SG4): stop accepting, drain in-flight for up to 30s, then exit.
|
||||
func ListenAndServe(ctx context.Context, pool *db.Pool, cfg config.Config) error {
|
||||
|
||||
196
internal/httpapi/skills.go
Normal file
196
internal/httpapi/skills.go
Normal file
@@ -0,0 +1,196 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// ─── Skills ────────────────────────────────────────────────────────────
|
||||
|
||||
func (s *Server) ListSkills(ctx context.Context, req gen.ListSkillsRequestObject) (gen.ListSkillsResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT s.entity_id, s.version, s.name, s.procedure, s.applies_type,
|
||||
s.action, s.pattern_ids, s.status, s.success_rate,
|
||||
s.changed_by::text, s.change_reason, s.last_used_at
|
||||
FROM skills s
|
||||
WHERE ($1::text IS NULL OR s.status = $1)
|
||||
AND ($2::text IS NULL OR s.applies_type = $2)
|
||||
AND ($3::text IS NULL OR s.action = $3)
|
||||
ORDER BY s.name, s.version DESC`,
|
||||
req.Params.Status, req.Params.AppliesTo, req.Params.Action)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
// Deduplicate to latest version per skill (the ORDER BY name, version DESC
|
||||
// means the first row per name is the latest).
|
||||
seen := map[string]bool{}
|
||||
items := []gen.Skill{}
|
||||
for rows.Next() {
|
||||
var s gen.Skill
|
||||
var procBytes []byte
|
||||
var patternIDs []uuid.UUID
|
||||
if err := rows.Scan(&s.Id, &s.Version, &s.Name, &procBytes, &s.AppliesType,
|
||||
&s.Action, &patternIDs, &s.Status, &s.SuccessRate,
|
||||
&s.ChangedBy, &s.ChangeReason, &s.LastUsedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if seen[s.Id.String()] {
|
||||
continue
|
||||
}
|
||||
seen[s.Id.String()] = true
|
||||
if err := json.Unmarshal(procBytes, &s.Procedure); err != nil {
|
||||
slog.Warn("phase3: unmarshal skill procedure", "skill", s.Name, "error", err)
|
||||
}
|
||||
if len(patternIDs) > 0 {
|
||||
pids := make([]string, len(patternIDs))
|
||||
for i, pid := range patternIDs {
|
||||
pids[i] = pid.String()
|
||||
}
|
||||
s.PatternIds = &pids
|
||||
}
|
||||
items = append(items, s)
|
||||
if len(items) > limit {
|
||||
break
|
||||
}
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
if items == nil {
|
||||
items = []gen.Skill{}
|
||||
}
|
||||
return gen.ListSkills200JSONResponse{Items: items}, nil
|
||||
}
|
||||
|
||||
func (s *Server) PatchSkill(ctx context.Context, req gen.PatchSkillRequestObject) (gen.PatchSkillResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
q := sqlcgen.New(tx)
|
||||
|
||||
if req.Body.Status != nil {
|
||||
if err := q.UpdateSkillStatus(ctx, sqlcgen.UpdateSkillStatusParams{
|
||||
EntityID: id,
|
||||
Status: string(*req.Body.Status),
|
||||
}); err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: skill %s", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
// Re-read skill.
|
||||
var skill gen.Skill
|
||||
var procBytes []byte
|
||||
var patternIDs []uuid.UUID
|
||||
err = tx.QueryRow(ctx, `
|
||||
SELECT entity_id, version, name, procedure, applies_type, action,
|
||||
pattern_ids, status, success_rate, changed_by::text,
|
||||
change_reason, last_used_at
|
||||
FROM skills WHERE entity_id = $1 ORDER BY version DESC LIMIT 1`, id).
|
||||
Scan(&skill.Id, &skill.Version, &skill.Name, &procBytes, &skill.AppliesType,
|
||||
&skill.Action, &patternIDs, &skill.Status, &skill.SuccessRate,
|
||||
&skill.ChangedBy, &skill.ChangeReason, &skill.LastUsedAt)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: skill %s", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if err := json.Unmarshal(procBytes, &skill.Procedure); err != nil {
|
||||
slog.Warn("phase3: unmarshal skill proc", "error", err)
|
||||
}
|
||||
if len(patternIDs) > 0 {
|
||||
pids := make([]string, len(patternIDs))
|
||||
for i, pid := range patternIDs {
|
||||
pids[i] = pid.String()
|
||||
}
|
||||
skill.PatternIds = &pids
|
||||
}
|
||||
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "patch",
|
||||
&id, "PATCH", "/api/v1/skills/"+req.Id, "",
|
||||
map[string]any{"status": req.Body.Status, "pinned_version": req.Body.PinnedVersion}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.PatchSkill200JSONResponse(skill), nil
|
||||
}
|
||||
|
||||
func (s *Server) ListSkillVersions(ctx context.Context, req gen.ListSkillVersionsRequestObject) (gen.ListSkillVersionsResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT entity_id, version, name, procedure, applies_type, action,
|
||||
pattern_ids, status, success_rate, changed_by::text,
|
||||
change_reason, last_used_at
|
||||
FROM skills WHERE entity_id = $1
|
||||
ORDER BY version DESC`, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.Skill{}
|
||||
for rows.Next() {
|
||||
var skill gen.Skill
|
||||
var procBytes []byte
|
||||
var patternIDs []uuid.UUID
|
||||
if err := rows.Scan(&skill.Id, &skill.Version, &skill.Name, &procBytes, &skill.AppliesType,
|
||||
&skill.Action, &patternIDs, &skill.Status, &skill.SuccessRate,
|
||||
&skill.ChangedBy, &skill.ChangeReason, &skill.LastUsedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := json.Unmarshal(procBytes, &skill.Procedure); err != nil {
|
||||
slog.Warn("phase3: unmarshal skill proc", "error", err)
|
||||
}
|
||||
if len(patternIDs) > 0 {
|
||||
pids := make([]string, len(patternIDs))
|
||||
for i, pid := range patternIDs {
|
||||
pids[i] = pid.String()
|
||||
}
|
||||
skill.PatternIds = &pids
|
||||
}
|
||||
items = append(items, skill)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.Skill{}
|
||||
}
|
||||
return gen.ListSkillVersions200JSONResponse{Items: items}, nil
|
||||
}
|
||||
@@ -1,6 +0,0 @@
|
||||
package httpapi
|
||||
|
||||
// Remaining stubs for endpoints that depend on tables not yet created
|
||||
// (knowledge_entities, agent_activity). These are kept here because the
|
||||
// phase3.go file already defines them; this file is deliberately empty.
|
||||
// The stubs live in phase3.go as simple errNotImplemented returns.
|
||||
155
internal/knowledge/seed_test.go
Normal file
155
internal/knowledge/seed_test.go
Normal file
@@ -0,0 +1,155 @@
|
||||
package knowledge
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestContentHash(t *testing.T) {
|
||||
t.Run("determinism", func(t *testing.T) {
|
||||
a := contentHash("hello")
|
||||
b := contentHash("hello")
|
||||
if a != b {
|
||||
t.Errorf("contentHash not deterministic: %q != %q", a, b)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("empty string known sha256", func(t *testing.T) {
|
||||
got := contentHash("")
|
||||
h := sha256.Sum256([]byte(""))
|
||||
want := hex.EncodeToString(h[:])
|
||||
if got != want {
|
||||
t.Errorf("contentHash(\"\") = %q, want %q", got, want)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("different inputs different outputs", func(t *testing.T) {
|
||||
if contentHash("a") == contentHash("b") {
|
||||
t.Error("different inputs produced same hash")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("output is 64-char hex", func(t *testing.T) {
|
||||
got := contentHash("anything")
|
||||
if len(got) != 64 {
|
||||
t.Errorf("len = %d, want 64", len(got))
|
||||
}
|
||||
for _, r := range got {
|
||||
isHex := (r >= '0' && r <= '9') || (r >= 'a' && r <= 'f')
|
||||
if !isHex {
|
||||
t.Errorf("non-hex char %q in hash %q", r, got)
|
||||
break
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestStr(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
m map[string]any
|
||||
key string
|
||||
want string
|
||||
}{
|
||||
{"missing key", map[string]any{}, "nope", ""},
|
||||
{"string value", map[string]any{"k": "v"}, "k", "v"},
|
||||
{"int value", map[string]any{"k": 42}, "k", ""},
|
||||
{"nil value", map[string]any{"k": nil}, "k", ""},
|
||||
{"empty string", map[string]any{"k": ""}, "k", ""},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got := str(c.m, c.key)
|
||||
if got != c.want {
|
||||
t.Errorf("str() = %q, want %q", got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestStrSlice(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
m map[string]any
|
||||
key string
|
||||
want []string
|
||||
}{
|
||||
{"missing key", map[string]any{}, "tags", nil},
|
||||
{"all strings", map[string]any{"tags": []any{"a", "b", "c"}}, "tags", []string{"a", "b", "c"}},
|
||||
{"mixed types", map[string]any{"tags": []any{1, "a", true, "b"}}, "tags", []string{"a", "b"}},
|
||||
{"empty array", map[string]any{"tags": []any{}}, "tags", []string{}},
|
||||
{"nil elements filtered", map[string]any{"tags": []any{nil, "a", nil, "b"}}, "tags", []string{"a", "b"}},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got := strSlice(c.m, c.key)
|
||||
if len(got) != len(c.want) {
|
||||
t.Errorf("len = %d, want %d (got %v)", len(got), len(c.want), got)
|
||||
return
|
||||
}
|
||||
for i := range got {
|
||||
if got[i] != c.want[i] {
|
||||
t.Errorf("[%d] = %q, want %q", i, got[i], c.want[i])
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestMapVal(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
m map[string]any
|
||||
key string
|
||||
want map[string]any
|
||||
}{
|
||||
{"missing key", map[string]any{}, "nope", nil},
|
||||
{"present map", map[string]any{"k": map[string]any{"x": 1}}, "k", map[string]any{"x": 1}},
|
||||
{"wrong type string", map[string]any{"k": "v"}, "k", nil},
|
||||
{"nested map", map[string]any{"k": map[string]any{"a": map[string]any{"b": 2}}}, "k", map[string]any{"a": map[string]any{"b": 2}}},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got := mapVal(c.m, c.key)
|
||||
if !reflect.DeepEqual(got, c.want) {
|
||||
t.Errorf("mapVal() = %v, want %v", got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestToPGArray(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
tags []string
|
||||
want string
|
||||
}{
|
||||
{"empty", []string{}, "{}"},
|
||||
{"single", []string{"a"}, `{"a"}`},
|
||||
{"multiple", []string{"a", "b"}, `{"a","b"}`},
|
||||
{"nil", nil, "{}"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got := toPGArray(c.tags)
|
||||
if got != c.want {
|
||||
t.Errorf("toPGArray() = %q, want %q", got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Special chars: tags containing " or \ are NOT escaped by toPGArray.
|
||||
// This is a latent bug — Postgres array literals require these to be
|
||||
// backslash-escaped. Test documents current behavior so a fix is
|
||||
// detectable. Should be fixed.
|
||||
t.Run("special chars unescaped (current buggy behavior)", func(t *testing.T) {
|
||||
got := toPGArray([]string{`a"b`, `c\d`})
|
||||
// Current output: {"a"b","c\d"} — invalid Postgres array literal.
|
||||
want := `{"a"b","c\d"}`
|
||||
if got != want {
|
||||
t.Errorf("toPGArray(special) = %q, want %q (if this changed, the escaping bug was fixed — update this test)", got, want)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -83,27 +83,13 @@ func extractPatterns(ctx context.Context, pool *db.Pool, watermark time.Time) ti
|
||||
func processGroup(ctx context.Context, pool *db.Pool, q *sqlcgen.Queries,
|
||||
appliesType, action string, items []sqlcgen.GetFeedbackAfterWatermarkRow) {
|
||||
|
||||
successCount := 0
|
||||
failureCount := 0
|
||||
for _, f := range items {
|
||||
switch f.Outcome {
|
||||
case "success":
|
||||
successCount++
|
||||
case "failure", "unexpected":
|
||||
failureCount++
|
||||
case "partial":
|
||||
successCount++ // partial counts as half-success
|
||||
}
|
||||
}
|
||||
successCount, failureCount := countOutcomes(items)
|
||||
total := successCount + failureCount
|
||||
if total == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// Compute Wilson score lower bound
|
||||
confidence := wilsonLowerBound(float64(successCount), float64(total), 0.95)
|
||||
// Cap by sample size: nothing looks confident before 5 samples
|
||||
confidence = math.Min(confidence, float64(total)/5.0)
|
||||
confidence := computeConfidence(successCount, failureCount)
|
||||
|
||||
// Get or create pattern — first look up existing entity, then upsert.
|
||||
existing, err := q.GetPattern(ctx, sqlcgen.GetPatternParams{
|
||||
@@ -148,7 +134,7 @@ func processGroup(ctx context.Context, pool *db.Pool, q *sqlcgen.Queries,
|
||||
return
|
||||
}
|
||||
|
||||
if pat.EvidenceCount >= 5 && pat.Confidence >= 0.7 && !pat.Quarantined {
|
||||
if shouldValidate(int(pat.EvidenceCount), float64(pat.Confidence), pat.Quarantined) {
|
||||
_ = q.UpdatePatternStatus(ctx, sqlcgen.UpdatePatternStatusParams{
|
||||
EntityID: pat.EntityID,
|
||||
Status: "validated",
|
||||
@@ -158,8 +144,7 @@ func processGroup(ctx context.Context, pool *db.Pool, q *sqlcgen.Queries,
|
||||
"confidence", confidence, "samples", total)
|
||||
}
|
||||
|
||||
// Anomaly check: >10 identical outcomes within 1h
|
||||
if total > 10 {
|
||||
if shouldQuarantine(total) {
|
||||
_ = q.UpdatePatternQuarantine(ctx, sqlcgen.UpdatePatternQuarantineParams{
|
||||
EntityID: pat.EntityID,
|
||||
Quarantined: true,
|
||||
@@ -169,6 +154,45 @@ func processGroup(ctx context.Context, pool *db.Pool, q *sqlcgen.Queries,
|
||||
}
|
||||
}
|
||||
|
||||
// countOutcomes tallies feedback items into success and failure counts.
|
||||
// "partial" counts as a half-success (increments success).
|
||||
func countOutcomes(items []sqlcgen.GetFeedbackAfterWatermarkRow) (success, failure int) {
|
||||
for _, f := range items {
|
||||
switch f.Outcome {
|
||||
case "success":
|
||||
success++
|
||||
case "failure", "unexpected":
|
||||
failure++
|
||||
case "partial":
|
||||
success++
|
||||
}
|
||||
}
|
||||
return success, failure
|
||||
}
|
||||
|
||||
// computeConfidence calculates the Wilson score lower bound, capped by
|
||||
// sample size (nothing looks confident before 5 samples).
|
||||
func computeConfidence(success, failure int) float64 {
|
||||
total := success + failure
|
||||
if total == 0 {
|
||||
return 0
|
||||
}
|
||||
confidence := wilsonLowerBound(float64(success), float64(total), 0.95)
|
||||
return math.Min(confidence, float64(total)/5.0)
|
||||
}
|
||||
|
||||
// shouldValidate returns true when a pattern has enough evidence and
|
||||
// confidence to be promoted from "hypothesized" to "validated".
|
||||
func shouldValidate(evidenceCount int, confidence float64, quarantined bool) bool {
|
||||
return evidenceCount >= 5 && confidence >= 0.7 && !quarantined
|
||||
}
|
||||
|
||||
// shouldQuarantine returns true when an anomaly burst is detected
|
||||
// (>10 identical outcomes, indicating a runaway loop rather than organic feedback).
|
||||
func shouldQuarantine(total int) bool {
|
||||
return total > 10
|
||||
}
|
||||
|
||||
// wilsonLowerBound computes the Wilson score interval lower bound.
|
||||
// Conservative estimate of success rate for small sample sizes.
|
||||
func wilsonLowerBound(success, total, z float64) float64 {
|
||||
|
||||
211
internal/learning/learning_test.go
Normal file
211
internal/learning/learning_test.go
Normal file
@@ -0,0 +1,211 @@
|
||||
package learning
|
||||
|
||||
import (
|
||||
"math"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
)
|
||||
|
||||
func TestWilsonLowerBound(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
success float64
|
||||
total float64
|
||||
z float64
|
||||
want float64
|
||||
// For approximate checks
|
||||
approx bool
|
||||
epsilon float64
|
||||
}{
|
||||
{"zero total", 5, 0, 1.96, 0, false, 0},
|
||||
{"zero success", 0, 10, 1.96, 0, true, 1e-10},
|
||||
{"all success small n", 3, 3, 1.96, 0, true, 0.5},
|
||||
{"all success large n", 100, 100, 1.96, 0, true, 0.05},
|
||||
{"half success large n", 50, 100, 1.96, 0.39, true, 0.02},
|
||||
{"higher z gives lower bound", 8, 10, 3.0, 0, true, 0.5},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := wilsonLowerBound(tt.success, tt.total, tt.z)
|
||||
if tt.approx {
|
||||
if tt.want > 0 && math.Abs(got-tt.want) > tt.epsilon {
|
||||
t.Errorf("wilsonLowerBound(%v, %v, %v) = %v, want ~%v (±%v)", tt.success, tt.total, tt.z, got, tt.want, tt.epsilon)
|
||||
}
|
||||
if got < 0 {
|
||||
t.Errorf("wilsonLowerBound returned negative: %v", got)
|
||||
}
|
||||
if got > 1 {
|
||||
t.Errorf("wilsonLowerBound returned >1: %v", got)
|
||||
}
|
||||
} else {
|
||||
if got != tt.want {
|
||||
t.Errorf("wilsonLowerBound(%v, %v, %v) = %v, want %v", tt.success, tt.total, tt.z, got, tt.want)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWilsonLowerBoundMonotonic(t *testing.T) {
|
||||
prev := 0.0
|
||||
for s := 0.0; s <= 20; s++ {
|
||||
got := wilsonLowerBound(s, 20, 1.96)
|
||||
if got < prev-1e-9 {
|
||||
t.Errorf("not monotonically increasing: s=%v got=%v prev=%v", s, got, prev)
|
||||
}
|
||||
prev = got
|
||||
}
|
||||
}
|
||||
|
||||
func TestCountOutcomes(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
items []sqlcgen.GetFeedbackAfterWatermarkRow
|
||||
wantSuccess int
|
||||
wantFailure int
|
||||
}{
|
||||
{
|
||||
"empty",
|
||||
nil,
|
||||
0, 0,
|
||||
},
|
||||
{
|
||||
"all success",
|
||||
[]sqlcgen.GetFeedbackAfterWatermarkRow{
|
||||
{Outcome: "success"},
|
||||
{Outcome: "success"},
|
||||
},
|
||||
2, 0,
|
||||
},
|
||||
{
|
||||
"all failure",
|
||||
[]sqlcgen.GetFeedbackAfterWatermarkRow{
|
||||
{Outcome: "failure"},
|
||||
{Outcome: "unexpected"},
|
||||
},
|
||||
0, 2,
|
||||
},
|
||||
{
|
||||
"mixed including partial",
|
||||
[]sqlcgen.GetFeedbackAfterWatermarkRow{
|
||||
{Outcome: "success"},
|
||||
{Outcome: "failure"},
|
||||
{Outcome: "partial"},
|
||||
{Outcome: "unexpected"},
|
||||
},
|
||||
2, 2,
|
||||
},
|
||||
{
|
||||
"unknown outcome ignored",
|
||||
[]sqlcgen.GetFeedbackAfterWatermarkRow{
|
||||
{Outcome: "success"},
|
||||
{Outcome: "bogus"},
|
||||
},
|
||||
1, 0,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
s, f := countOutcomes(tt.items)
|
||||
if s != tt.wantSuccess || f != tt.wantFailure {
|
||||
t.Errorf("countOutcomes() = (%d, %d), want (%d, %d)", s, f, tt.wantSuccess, tt.wantFailure)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestComputeConfidence(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
success int
|
||||
failure int
|
||||
wantMax float64
|
||||
wantMin float64
|
||||
}{
|
||||
{"zero total", 0, 0, 0, 0},
|
||||
{"one success no failures", 1, 0, 0.2, 0},
|
||||
{"five success no failures", 5, 0, 1.0, 0.3},
|
||||
{"ten success no failures", 10, 0, 1.0, 0.5},
|
||||
{"half success", 5, 5, 0.5, 0.2},
|
||||
{"all failures", 0, 10, 0.01, 0},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := computeConfidence(tt.success, tt.failure)
|
||||
if got < 0 || got > 1 {
|
||||
t.Errorf("confidence out of [0,1]: %v", got)
|
||||
}
|
||||
if got > tt.wantMax+0.01 {
|
||||
t.Errorf("confidence too high: got %v, max ~%v", got, tt.wantMax)
|
||||
}
|
||||
if tt.wantMin > 0 && got < tt.wantMin-0.1 {
|
||||
t.Errorf("confidence too low: got %v, min ~%v", got, tt.wantMin)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestComputeConfidenceSampleSizeCap(t *testing.T) {
|
||||
got := computeConfidence(1, 0)
|
||||
if got > 0.2+1e-9 {
|
||||
t.Errorf("sample size cap not applied: 1 sample should cap at 1/5=0.2, got %v", got)
|
||||
}
|
||||
got = computeConfidence(4, 0)
|
||||
if got > 0.8+1e-9 {
|
||||
t.Errorf("sample size cap not applied: 4 samples should cap at 4/5=0.8, got %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestShouldValidate(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
evidenceCount int
|
||||
confidence float64
|
||||
quarantined bool
|
||||
want bool
|
||||
}{
|
||||
{"enough evidence and confidence", 5, 0.7, false, true},
|
||||
{"high evidence high confidence", 10, 0.9, false, true},
|
||||
{"not enough evidence", 4, 0.9, false, false},
|
||||
{"not enough confidence", 5, 0.69, false, false},
|
||||
{"quarantined blocks validation", 10, 0.9, true, false},
|
||||
{"exactly at threshold", 5, 0.7, false, true},
|
||||
{"zero evidence", 0, 0.9, false, false},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := shouldValidate(tt.evidenceCount, tt.confidence, tt.quarantined)
|
||||
if got != tt.want {
|
||||
t.Errorf("shouldValidate(%d, %v, %v) = %v, want %v", tt.evidenceCount, tt.confidence, tt.quarantined, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestShouldQuarantine(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
total int
|
||||
want bool
|
||||
}{
|
||||
{"zero", 0, false},
|
||||
{"small", 5, false},
|
||||
{"at threshold", 10, false},
|
||||
{"over threshold", 11, true},
|
||||
{"large burst", 50, true},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := shouldQuarantine(tt.total)
|
||||
if got != tt.want {
|
||||
t.Errorf("shouldQuarantine(%d) = %v, want %v", tt.total, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
762
internal/mcp/tools.go
Normal file
762
internal/mcp/tools.go
Normal file
@@ -0,0 +1,762 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
// toolReg pairs a tool definition with its handler. allTools returns a slice
|
||||
// of these; newServer iterates it and registers each one wrapped with
|
||||
// withActivityLogging.
|
||||
type toolReg struct {
|
||||
tool *mcp.Tool
|
||||
handler toolHandler
|
||||
}
|
||||
|
||||
// allTools returns every MCP tool registration. Tool definitions, schemas,
|
||||
// descriptions, and handler bodies are kept verbatim from the former inline
|
||||
// newServer registrations.
|
||||
func allTools(pool *db.Pool, agentID uuid.UUID) []toolReg {
|
||||
return []toolReg{
|
||||
{tool: &mcp.Tool{Name: "get_entity", Description: "Get an entity by slug or UUID",
|
||||
InputSchema: objSchema(prop{"slug_or_id", "string", "Entity slug (e.g. host:hubris) or UUID"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
idOrSlug, _ := args["slug_or_id"].(string)
|
||||
return queryEntity(ctx, pool, idOrSlug), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "list_entities", Description: "List entities filtered by type, state, or search",
|
||||
InputSchema: objSchema(
|
||||
prop{"type", "string", "Filter by entity type"},
|
||||
prop{"state", "string", "Filter by lifecycle state"},
|
||||
prop{"q", "string", "Substring match on slug or name"},
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state, e.version, e.created_at, e.updated_at
|
||||
FROM entities e
|
||||
WHERE ($1::text IS NULL OR e.type = $1)
|
||||
AND ($2::text IS NULL OR e.state = $2)
|
||||
AND ($3::text IS NULL OR e.slug ILIKE '%'||$3||'%' OR e.name ILIKE '%'||$3||'%')
|
||||
ORDER BY e.slug LIMIT $4`,
|
||||
nStr(args["type"]), nStr(args["state"]), nStr(args["q"]), limit), "entity_table"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_relations", Description: "Get relationships for an entity",
|
||||
InputSchema: objSchema(prop{"entity_id", "string", "Entity slug"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_id"].(string)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT r.type, src.slug AS source, tgt.slug AS target
|
||||
FROM relationships r
|
||||
JOIN entities src ON src.id = r.source_id
|
||||
JOIN entities tgt ON tgt.id = r.target_id
|
||||
WHERE (src.slug = $1 OR tgt.slug = $1) AND r.valid_to IS NULL
|
||||
ORDER BY r.type`, slug), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_blast_radius", Description: "Find entities affected if this entity goes down",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_id", "string", "Entity slug"},
|
||||
prop{"depth", "integer", "Traversal depth (default 3)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_id"].(string)
|
||||
depth := int(getFloat(args, "depth", 3))
|
||||
return queryRows(ctx, pool,
|
||||
"SELECT e.slug, CAST(b.depth AS int) FROM blast_radius((SELECT id FROM entities WHERE slug = $1), $2) b JOIN entities e ON e.id = b.entity_id",
|
||||
slug, depth), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_health_summary", Description: "Current fleet health summary",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check'
|
||||
ORDER BY e.slug`), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_audit_trail", Description: "Query the audit log",
|
||||
InputSchema: objSchema(prop{"entity_id", "string", "Filter by affected entity UUID"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT id, ts, actor_type, action, entity_id::text, method, path, correlation_id
|
||||
FROM audit_log
|
||||
WHERE ($1::text IS NULL OR entity_id::text = $1)
|
||||
ORDER BY ts DESC LIMIT 50`, nStr(args["entity_id"])), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "search_knowledge", Description: "Full-text search across documentation (PostgreSQL FTS with ts_rank ranking). Returns a short snippet per hit, not the full note — call get_knowledge_content with the returned slug to read the whole thing.",
|
||||
InputSchema: objSchema(prop{"query", "string", "Search terms"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
q := nStr(args["query"])
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT ke.title, e.slug,
|
||||
ts_rank(ke.search, plainto_tsquery('english', $1)) AS rank,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', $1),
|
||||
'MaxWords=40, MinWords=15, ShortWord=3, MaxFragments=3,
|
||||
FragmentDelimiter=" ... "') AS snippet,
|
||||
ke.source, ke.tags
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE ke.search @@ plainto_tsquery('english', $1)
|
||||
ORDER BY rank DESC
|
||||
LIMIT 20`, q), "knowledge_results"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_entity_knowledge", Description: "All documents, investigations, and runbooks linked to an entity. Returns a headline per note, not the full text — call get_knowledge_content with the returned slug to read the whole thing.",
|
||||
InputSchema: objSchema(prop{"entity_slug", "string", "Entity slug (e.g. lxc:jellyfin, service:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_slug"].(string)
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT ke.title, ke.source, e.type AS kind, e.slug,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', '')) AS headline
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
JOIN relationships r ON r.source_id = ke.entity_id
|
||||
JOIN entities target ON target.id = r.target_id
|
||||
WHERE target.slug = $1
|
||||
AND r.valid_to IS NULL
|
||||
AND r.type IN ('documents', 'about')
|
||||
UNION
|
||||
SELECT ke.title, ke.source, e.type AS kind, e.slug,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', '')) AS headline
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
JOIN relationships r ON r.source_id = ke.entity_id
|
||||
JOIN entity_types target_type ON target_type.name = (SELECT type FROM entities WHERE slug = $1)
|
||||
JOIN entities ent ON ent.type = target_type.name AND ent.slug = $1
|
||||
WHERE r.valid_to IS NULL
|
||||
AND r.type = 'procedure-for'
|
||||
ORDER BY 1`, slug), "knowledge_results"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_knowledge_content", Description: "Full markdown body of one document/investigation/runbook, by its own entity slug. search_knowledge and get_entity_knowledge only return short snippets/headlines — once you know which note you need (from either of those, or because you already know its slug), call this to read the whole thing before acting on it.",
|
||||
InputSchema: objSchema(prop{"slug", "string", "The knowledge entity's own slug (e.g. document:containers/101-jellyfin, runbook:client-enrollment) — not the slug of an entity it's about."}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["slug"].(string)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT ke.title, e.slug, e.type AS kind, ke.content, ke.source, ke.tags, ke.updated_at::text
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE e.slug = $1`, slug), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "upsert_knowledge", Description: "Write back what you learned so future sessions (and future you) benefit — this is how the system gets smarter over time. Use it AFTER solving a non-obvious problem, deploying a service, or discovering a gotcha: record the finding, the fix, and any caveats. Re-calling with the same title updates the existing note instead of duplicating. This is the ONLY way to persist knowledge; a chat message alone is forgotten. search_knowledge/get_entity_knowledge find it, get_knowledge_content reads the full body back.",
|
||||
InputSchema: objSchema(
|
||||
prop{"title", "string", "Short, specific, searchable title (e.g. 'Dragonfly memlock rlimit in unprivileged LXCs', not 'notes')."},
|
||||
prop{"content", "string", "The knowledge itself, in markdown. Be concrete: symptom, root cause, the exact fix/commands, and any caveats. Written for someone hitting this fresh."},
|
||||
prop{"about", "string", "Optional entity slug(s) this knowledge concerns. Pass a single slug (e.g. 'lxc:nfs-export') or a JSON array of slugs (e.g. '[\"lxc:nfs-export\", \"lxc:gitea\"]') to link to multiple entities. get_entity_knowledge surfaces it for each."},
|
||||
prop{"tags", "string", "Optional comma-separated tags (e.g. 'docker,networking,gotcha')."},
|
||||
prop{"kind", "string", "One of: investigation (a finding/incident analysis — default), document (reference), runbook (a repeatable procedure)."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return upsertKnowledge(ctx, pool, args)
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "update_entity_attributes", Description: "Merge new/changed attributes into an entity — the OTHER half of avoiding knowledge-base drift (upsert_knowledge records what you learned; this keeps the entity's own facts current). Use it when you discover something concrete about an entity's actual state that the graph doesn't reflect yet: a new IP, a version number, a config value, a discovered port — anything a FUTURE task would otherwise have to rediscover from scratch. Does NOT require approval (this updates the knowledge graph, not the live infrastructure). Merges shallowly — existing keys not mentioned are kept; keys you pass overwrite.",
|
||||
InputSchema: objSchema(
|
||||
prop{"slug", "string", "Entity slug to update (e.g. lxc:typetype, host:strong)."},
|
||||
prop{"attributes", "string", "JSON object string of attributes to merge in, e.g. {\"lan_ip\":\"192.168.8.50\",\"os\":\"debian-12\"}."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["slug"].(string)
|
||||
attrsStr, _ := args["attributes"].(string)
|
||||
if slug == "" || attrsStr == "" {
|
||||
return textResult("error: slug and attributes are required"), nil
|
||||
}
|
||||
var attrs map[string]any
|
||||
if err := json.Unmarshal([]byte(attrsStr), &attrs); err != nil {
|
||||
return textResult(fmt.Sprintf("error: attributes is not valid JSON: %v", err)), nil
|
||||
}
|
||||
attrsJSON, _ := json.Marshal(attrs)
|
||||
ct, err := pool.Exec(ctx, `
|
||||
UPDATE entities SET attributes = attributes || $2::jsonb, updated_at = now()
|
||||
WHERE slug = $1`, slug, string(attrsJSON))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error updating %s: %v", slug, err)), nil
|
||||
}
|
||||
if ct.RowsAffected() == 0 {
|
||||
return textResult(fmt.Sprintf("error: entity %q not found", slug)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Updated %s with %d attribute(s).", slug, len(attrs))), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "create_relationship", Description: "Record a relationship you discovered between two entities — the graph-structure half of keeping the knowledge base current (alongside update_entity_attributes and upsert_knowledge). Use it when you learn that one entity depends on, hosts, routes to, etc. another, and that edge isn't in the graph yet. type must be an existing relationship type (see get_relations output on similar entities for examples: hosts, provides, depends-on, configured-by, about, documents, ...). Idempotent — re-calling the same source/target/type is a no-op. Does NOT require approval.",
|
||||
InputSchema: objSchema(
|
||||
prop{"source", "string", "Source entity slug."},
|
||||
prop{"target", "string", "Target entity slug."},
|
||||
prop{"type", "string", "Relationship type name (must already exist in the ontology)."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
source, _ := args["source"].(string)
|
||||
target, _ := args["target"].(string)
|
||||
relType, _ := args["type"].(string)
|
||||
if source == "" || target == "" || relType == "" {
|
||||
return textResult("error: source, target, and type are required"), nil
|
||||
}
|
||||
var sourceID, targetID uuid.UUID
|
||||
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", source).Scan(&sourceID); err != nil {
|
||||
return textResult(fmt.Sprintf("error: source entity %q not found", source)), nil
|
||||
}
|
||||
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", target).Scan(&targetID); err != nil {
|
||||
return textResult(fmt.Sprintf("error: target entity %q not found", target)), nil
|
||||
}
|
||||
_, err := pool.Exec(ctx, `
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT $1, $2, $3, '{"by":"nomos"}'::jsonb, now()
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM relationships
|
||||
WHERE source_id = $1 AND target_id = $2 AND type = $3 AND valid_to IS NULL)`,
|
||||
sourceID, targetID, relType)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error creating relationship: %v (is %q a valid relationship type?)", err, relType)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Recorded: %s —%s→ %s", source, relType, target)), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "query_metrics", Description: "Query time-series metrics",
|
||||
InputSchema: objSchema(prop{"hours", "integer", "Look-back window in hours (default 24)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
hours := int(getFloat(args, "hours", 24))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT time_bucket('1 hour', ts) AS bucket,
|
||||
entity_id::text, metric,
|
||||
ROUND(avg(value)::numeric, 2) AS avg,
|
||||
ROUND(min(value)::numeric, 2) AS min,
|
||||
ROUND(max(value)::numeric, 2) AS max
|
||||
FROM metric_samples
|
||||
WHERE ts > now() - make_interval(hours => $1)
|
||||
GROUP BY bucket, entity_id, metric
|
||||
ORDER BY bucket DESC LIMIT 100`, hours), "metric_chart"), nil
|
||||
}},
|
||||
|
||||
// ─── Phase 4: new tools ──────────────────────────────────────────
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_signal_history", Description: "Query open and recent signals",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_slug", "string", "Filter by target entity slug"},
|
||||
prop{"state", "string", "Filter by signal state (raised, resolved)"},
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT s.entity_id::text, s.kind, s.severity, s.state,
|
||||
s.occurrence_count, e.slug AS target_slug,
|
||||
s.first_seen_at, s.last_seen_at
|
||||
FROM signals s
|
||||
LEFT JOIN entities e ON e.id = s.target_entity_id
|
||||
WHERE ($1::text IS NULL OR e.slug = $1)
|
||||
AND ($2::text IS NULL OR s.state = $2)
|
||||
ORDER BY s.last_seen_at DESC LIMIT $3`,
|
||||
nStr(args["entity_slug"]), nStr(args["state"]), limit), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_patterns", Description: "List learned action patterns",
|
||||
InputSchema: objSchema(
|
||||
prop{"status", "string", "Filter by status (hypothesized, validated, active)"},
|
||||
prop{"entity_type", "string", "Filter by applies_type"},
|
||||
prop{"action", "string", "Filter by action"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT p.entity_id::text, p.applies_type, p.action, p.pattern,
|
||||
p.confidence, p.evidence_count, p.success_count, p.failure_count,
|
||||
p.status, p.quarantined, p.version, p.last_validated_at
|
||||
FROM patterns p
|
||||
WHERE ($1::text IS NULL OR p.status = $1)
|
||||
AND ($2::text IS NULL OR p.applies_type = $2)
|
||||
AND ($3::text IS NULL OR p.action = $3)
|
||||
ORDER BY p.applies_type, p.action`,
|
||||
nStr(args["status"]), nStr(args["entity_type"]), nStr(args["action"])), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_skills", Description: "List available automation skills",
|
||||
InputSchema: objSchema(
|
||||
prop{"status", "string", "Filter by status (active, inactive, deprecated)"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT s.entity_id::text, s.version, s.name, LEFT(s.procedure::text, 300) AS procedure_preview,
|
||||
s.applies_type, s.action, s.status, s.success_rate,
|
||||
s.changed_by::text, s.change_reason, s.last_used_at
|
||||
FROM skills s
|
||||
WHERE ($1::text IS NULL OR s.status = $1)
|
||||
ORDER BY s.name, s.version DESC`,
|
||||
nStr(args["status"])), nil
|
||||
}},
|
||||
|
||||
// ── request_execution (legacy fixed enum) retired 2026-07-14 ──
|
||||
// All mutations now route through `run`. The handler functions
|
||||
// (runRexecRestart, runRexecSystemctl, etc.) are kept as reference
|
||||
// for future runbook extraction — especially pct_create DNS/VMID logic.
|
||||
// DO NOT re-register this tool. See plans/2026-07-10-general-gated-execution.md.
|
||||
|
||||
{tool: &mcp.Tool{Name: "run", Description: "Run ANY shell command against any host, LXC, or VM. This is the general execution primitive — prefer it over asking the operator to run something manually, and don't wait for a matching fixed action to exist. Every command is automatically risk-classified: read-only inspection (cat, systemctl status, docker ps, journalctl, df, git status, ...) runs immediately; anything that changes state requires operator approval (granted by the operator replying \"go ahead\"/\"yes\" in chat, or via the Approve button); commands matching a destructive pattern (rm -rf, dd, mkfs, pct/qm destroy, DROP TABLE, reboot, piping curl into a shell, ...) always require approval regardless of what you declare. You cannot talk your way past the destructive check by declaring a lower risk.",
|
||||
InputSchema: objSchema(
|
||||
prop{"target", "string", "Target entity slug: host:<slug> (e.g. host:strong), lxc:<slug> (e.g. lxc:caddy), or vm:<slug> (e.g. vm:zimaos). LXC commands run via pct exec on their Proxmox host automatically. VM commands run via qm guest exec on their Proxmox host (requires the QEMU guest agent inside the VM — standard for Proxmox VMs)."},
|
||||
prop{"command", "string", "The shell command to run. Can be a full script (multi-line, &&-chained). Runs as root."},
|
||||
prop{"purpose", "string", "One sentence: why you're running this. Shown to the operator alongside the approval — be specific, this is what they're approving."},
|
||||
prop{"declared_risk", "string", "Optional self-assessment: read_only, reversible_low, config_mutation, or destructive. This can only ESCALATE the automatic classification, never lower it — declaring a mutating command as read_only has no effect."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
targetSlug, _ := args["target"].(string)
|
||||
command, _ := args["command"].(string)
|
||||
purpose, _ := args["purpose"].(string)
|
||||
declaredRisk, _ := args["declared_risk"].(string)
|
||||
sessionID, _ := args["_session_id"].(string)
|
||||
if targetSlug == "" || command == "" {
|
||||
return textResult("error: target and command are required"), nil
|
||||
}
|
||||
|
||||
var targetID uuid.UUID
|
||||
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", targetSlug).Scan(&targetID); err != nil {
|
||||
return textResult(fmt.Sprintf("target not found: %s", targetSlug)), nil
|
||||
}
|
||||
|
||||
return classifyAndGate(ctx, pool, agentID, targetID, targetSlug, command, purpose, declaredRisk, sessionID), nil
|
||||
}},
|
||||
|
||||
// inspect_path is the bulk fact-gathering tool from
|
||||
// plans/2026-07-18-session-review-three-sessions.md P1.5.
|
||||
// Sessions 1e9c7691 and 55927f0a each spent ~15 `run` calls
|
||||
// gathering identical facts (`mount | grep`, `df`, `ls -la`,
|
||||
// `stat`) across hosts and LXCs to understand where a path
|
||||
// lives, who mounts it, and what permissions it has. This tool
|
||||
// collapses that fan-out into one call: pass a path and a list
|
||||
// of targets, get back per-target mount/df/ls/stat output as
|
||||
// JSON. All commands are read-only, so no approval is needed.
|
||||
{tool: &mcp.Tool{Name: "inspect_path", Description: "Bulk fact-gathering: run mount/df/ls/stat for the same path across multiple host/LXC/VM targets in ONE call. Returns a JSON object keyed by target slug, each with the target's view of the path (mount source, filesystem, size, top-level entries with ownership/permissions). Use this instead of N separate `run` calls when you need to understand a path's footprint across the fleet (e.g. tracing where a volume is mounted, checking permissions on the same NFS path from server + client). All commands are read-only — no approval needed.",
|
||||
InputSchema: objSchema(
|
||||
prop{"path", "string", "Absolute path to inspect on each target (e.g. /mnt/media_local, /media/ludo-library)."},
|
||||
prop{"targets", "array", "List of target entity slugs (host:strong, lxc:nfs-export, vm:zimaos, …). Up to 8 per call."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
path, _ := args["path"].(string)
|
||||
if path == "" {
|
||||
return textResult("error: path is required"), nil
|
||||
}
|
||||
rawTargets, _ := args["targets"].([]any)
|
||||
if len(rawTargets) == 0 {
|
||||
return textResult("error: at least one target is required"), nil
|
||||
}
|
||||
if len(rawTargets) > 8 {
|
||||
return textResult("error: at most 8 targets per inspect_path call (use two calls if you need more)"), nil
|
||||
}
|
||||
targets := make([]string, 0, len(rawTargets))
|
||||
for _, t := range rawTargets {
|
||||
if s, ok := t.(string); ok && s != "" {
|
||||
targets = append(targets, s)
|
||||
}
|
||||
}
|
||||
results := inspectPathAcrossTargets(ctx, pool, path, targets)
|
||||
out, _ := json.MarshalIndent(results, "", " ")
|
||||
return textResult(string(out)), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "http_get", Description: "Fetch a public web page or raw file (e.g. a GitHub README/raw URL) and return sanitized text. Use this to research how to deploy a service before provisioning. HTTP/HTTPS only; body is truncated to ~16KB.",
|
||||
InputSchema: objSchema(
|
||||
prop{"url", "string", "Absolute http(s) URL to fetch"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
rawURL, _ := args["url"].(string)
|
||||
return httpGet(ctx, rawURL), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_execution_status", Description: "Check the status of a requested execution",
|
||||
InputSchema: objSchema(
|
||||
prop{"execution_id", "string", "Execution UUID (from request_execution output)"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
execID, _ := args["execution_id"].(string)
|
||||
if execID == "" {
|
||||
return textResult("execution_id required"), nil
|
||||
}
|
||||
eid, err := uuid.Parse(execID)
|
||||
if err != nil {
|
||||
// Try finding by exec slug prefix
|
||||
var found uuid.UUID
|
||||
err2 := pool.QueryRow(ctx, "SELECT entity_id FROM executions WHERE entity_id::text LIKE $1 LIMIT 1", execID+"%").Scan(&found)
|
||||
if err2 != nil {
|
||||
return textResult(fmt.Sprintf("execution not found: %s", execID)), nil
|
||||
}
|
||||
eid = found
|
||||
}
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.entity_id::text, e.action, e.risk_class, e.status,
|
||||
e.result::text, e.duration_ms, e.started_at::text,
|
||||
e.completed_at::text, e.correlation_id
|
||||
FROM executions e
|
||||
WHERE e.entity_id = $1`, eid), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_trend", Description: "Get metric trends for an entity",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_id", "string", "Entity slug"},
|
||||
prop{"days", "integer", "Look-back window in days (default 7)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_id"].(string)
|
||||
days := int(getFloat(args, "days", 7))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT metric,
|
||||
ROUND(avg(value)::numeric, 2) AS avg_val,
|
||||
ROUND(stddev(value)::numeric, 2) AS std_val,
|
||||
count(*) AS sample_count,
|
||||
ROUND(regr_slope(value, EXTRACT(EPOCH FROM ts)::numeric)::numeric, 4) AS slope
|
||||
FROM metric_samples ms
|
||||
JOIN entities e ON e.id = ms.entity_id
|
||||
WHERE e.slug = $1 AND ts >= now() - make_interval(days => $2)
|
||||
GROUP BY metric
|
||||
ORDER BY metric`, slug, days), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_event_timeline", Description: "Get recent events",
|
||||
InputSchema: objSchema(
|
||||
prop{"severity", "string", "Filter by severity (info, warn, error)"},
|
||||
prop{"entity_slug", "string", "Filter by entity slug"},
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT ev.ts, ev.type, ev.severity, ev.source, e.slug AS entity_slug,
|
||||
ev.data::text AS message, ev.correlation_id
|
||||
FROM events ev
|
||||
LEFT JOIN entities e ON e.id = ev.entity_id
|
||||
WHERE ($1::text IS NULL OR ev.severity = $1)
|
||||
AND ($2::text IS NULL OR e.slug = $2)
|
||||
ORDER BY ev.ts DESC LIMIT $3`,
|
||||
nStr(args["severity"]), nStr(args["entity_slug"]), limit), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_agent_activity", Description: "Agent self-inspection: query agent activity log",
|
||||
InputSchema: objSchema(
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT id, ts, agent_id::text, session_id, activity_type, tool_name,
|
||||
entity_id::text, left(input_summary, 200) AS input_summary,
|
||||
left(output_summary, 200) AS output_summary,
|
||||
duration_ms, token_count, success, correlation_id
|
||||
FROM agent_activity
|
||||
WHERE agent_id = $1
|
||||
ORDER BY ts DESC LIMIT $2`, agentID, limit), "change_log"), nil
|
||||
}},
|
||||
|
||||
// ─── Phase 5: operational MCP tools ──────────────────────────────
|
||||
|
||||
{tool: &mcp.Tool{Name: "list_lxcs", Description: "List all LXC containers with ID, host, IP, state, and last-audited hint. Pass state=\"active\" to exclude destroyed/deprecated containers. The last_audited_at column shows the most recent knowledge entry (investigation or document tagged audit/update) linked via an 'about' edge — use it to skip re-running `run` against LXCs that were already audited recently.",
|
||||
InputSchema: objSchema(
|
||||
prop{"state", "string", "Optional: filter by entity state (active, destroyed, …)"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
state, _ := argsMap(req)["state"].(string)
|
||||
var statePtr *string
|
||||
if state != "" {
|
||||
statePtr = &state
|
||||
}
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.name, e.attributes->>'pve_id' AS pve_id,
|
||||
e.attributes->>'lan_ip' AS lan_ip,
|
||||
e.state,
|
||||
st.health, st.last_check_at,
|
||||
(SELECT MAX(k.created_at)
|
||||
FROM relationships r
|
||||
JOIN knowledge_entities k ON k.entity_id = r.source_id
|
||||
WHERE r.target_id = e.id
|
||||
AND r.type = 'about'
|
||||
AND r.valid_to IS NULL
|
||||
AND (k.tags @> ARRAY['audit']::text[]
|
||||
OR k.tags @> ARRAY['update']::text[]
|
||||
OR k.title ILIKE '%audit%'
|
||||
OR k.title ILIKE '%update%')
|
||||
) AS last_audited_at
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.type = 'lxc'
|
||||
AND ($1::text IS NULL OR e.state = $1)
|
||||
ORDER BY CASE WHEN e.state = 'active' THEN 0 ELSE 1 END,
|
||||
(e.attributes->>'pve_id')::int`, statePtr), "lxc_list"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "ping_service", Description: "Check if a service is reachable via HTTP",
|
||||
InputSchema: objSchema(prop{"service_slug", "string", "Service entity slug"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("service_slug is required"), nil
|
||||
}
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT st.health, st.last_check_at, e.attributes->>'url' AS url
|
||||
FROM entity_status st
|
||||
JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.slug = $1`, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("query error: %v", err)), nil
|
||||
}
|
||||
defer rows.Close()
|
||||
if !rows.Next() {
|
||||
return textResult(fmt.Sprintf("service not found: %s", slug)), nil
|
||||
}
|
||||
var health, lastCheck, url string
|
||||
rows.Scan(&health, &lastCheck, &url)
|
||||
if url == "" {
|
||||
url = "(no URL in entity attributes)"
|
||||
}
|
||||
return textResult(fmt.Sprintf("health=%s last_check=%s url=%s", health, lastCheck, url)), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "tail_log", Description: "Get recent log lines from a service via journalctl",
|
||||
InputSchema: objSchema(
|
||||
prop{"service_slug", "string", "Service entity slug (e.g. lxc:caddy)"},
|
||||
prop{"lines", "integer", "Number of lines (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
n := int(getFloat(args, "lines", 50))
|
||||
if slug == "" {
|
||||
return textResult("service_slug is required"), nil
|
||||
}
|
||||
host, user, err := resolveHost(ctx, pool, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
|
||||
}
|
||||
svc := strings.TrimPrefix(slug, "lxc:")
|
||||
out, err := sshExec(ctx, host, user, fmt.Sprintf("journalctl -u %s -n %d --no-pager 2>&1 || true", svc, n))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_service_status", Description: "Check systemd service status on a host",
|
||||
InputSchema: objSchema(
|
||||
prop{"service_slug", "string", "Service entity slug (e.g. lxc:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("service_slug is required"), nil
|
||||
}
|
||||
host, user, err := resolveHost(ctx, pool, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
|
||||
}
|
||||
svc := strings.TrimPrefix(slug, "lxc:")
|
||||
out, err := sshExec(ctx, host, user,
|
||||
fmt.Sprintf("systemctl is-active %s; systemctl is-enabled %s; systemctl show %s -p ActiveEnterTimestamp -p SubState 2>&1 || true", svc, svc, svc))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_lxc_state", Description: "Get LXC container resource state from Proxmox host",
|
||||
InputSchema: objSchema(
|
||||
prop{"lxc_slug", "string", "LXC entity slug (e.g. lxc:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["lxc_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("lxc_slug is required"), nil
|
||||
}
|
||||
var pveID string
|
||||
err := pool.QueryRow(ctx, "SELECT attributes->>'pve_id' FROM entities WHERE slug = $1", slug).Scan(&pveID)
|
||||
if err != nil || pveID == "" {
|
||||
return textResult(fmt.Sprintf("LXC not found or missing pve_id: %s", slug)), nil
|
||||
}
|
||||
// Resolve the Proxmox host — find the host that runs this LXC
|
||||
var hostID uuid.UUID
|
||||
err = pool.QueryRow(ctx, `
|
||||
SELECT t.id FROM entities t
|
||||
JOIN relationships r ON r.source_id = t.id
|
||||
JOIN entities s ON s.id = r.target_id
|
||||
WHERE s.slug = $1 AND r.type = 'hosts' AND r.valid_to IS NULL
|
||||
LIMIT 1`, slug).Scan(&hostID)
|
||||
if err != nil {
|
||||
// Fallback: use the inventory host attribute if no relationship
|
||||
var hostSlug string
|
||||
err = pool.QueryRow(ctx, "SELECT attributes->>'host' FROM entities WHERE slug = $1", slug).Scan(&hostSlug)
|
||||
if err != nil || hostSlug == "" {
|
||||
return textResult(fmt.Sprintf("cannot resolve Proxmox host for %s", slug)), nil
|
||||
}
|
||||
var host, user string
|
||||
host, user, err = resolveHost(ctx, pool, "host:"+hostSlug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve: %v", err)), nil
|
||||
}
|
||||
out, err2 := sshExec(ctx, host, user, fmt.Sprintf("pct status %s --verbose 2>&1 || true", pveID))
|
||||
if err2 != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err2)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}
|
||||
var hostSlug string
|
||||
pool.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", hostID).Scan(&hostSlug)
|
||||
host, user, err := resolveHost(ctx, pool, hostSlug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
|
||||
}
|
||||
out, err := sshExec(ctx, host, user, fmt.Sprintf("pct status %s --verbose 2>&1 || true", pveID))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}},
|
||||
|
||||
// ─── Client introspection tools (plan: client-lifecycle Phase 3) ──
|
||||
|
||||
{tool: &mcp.Tool{Name: "whoami", Description: "Get the current entity record, peers, and health for a host",
|
||||
InputSchema: objSchema(prop{"hostname", "string", "Hostname of the calling machine"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
hostname, _ := args["hostname"].(string)
|
||||
if hostname == "" {
|
||||
return textResult("error: hostname required"), nil
|
||||
}
|
||||
slug := "ws:" + hostname
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state,
|
||||
COALESCE(st.health, 'unknown') AS health,
|
||||
COALESCE(st.last_check_at::text, '') AS last_check,
|
||||
e.attributes->>'mesh_ip' AS mesh_ip,
|
||||
e.attributes->>'age_pubkey' AS age_pubkey,
|
||||
e.enrolled_at
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.slug = $1
|
||||
ORDER BY e.slug`, slug), "entity_card"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "explain", Description: "Compact context card for a service: type, state, health, relations, risk",
|
||||
InputSchema: objSchema(prop{"service_slug", "string", "Service entity slug (e.g. service:jellyfin, lxc:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("error: service_slug required"), nil
|
||||
}
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state,
|
||||
COALESCE(st.health, 'unknown') AS health,
|
||||
COALESCE(st.last_check_at::text, '') AS last_check,
|
||||
e.version, e.updated_at,
|
||||
COALESCE(e.attributes::text, '{}') AS attrs
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.slug = $1`, slug), "entity_card"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "preflight", Description: "Risk classification for an action on a service",
|
||||
InputSchema: objSchema(
|
||||
prop{"service_slug", "string", "Entity slug"},
|
||||
prop{"action", "string", "Planned action (restart, deploy, destroy, etc.)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
action, _ := args["action"].(string)
|
||||
if slug == "" || action == "" {
|
||||
return textResult("error: service_slug and action required"), nil
|
||||
}
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.state,
|
||||
CASE
|
||||
WHEN $2 IN ('restart', 'logs', 'status') THEN 'reversible_low'
|
||||
WHEN $2 IN ('deploy', 'upgrade', 'configure') THEN 'config_mutation'
|
||||
WHEN $2 IN ('destroy', 'wipe', 'revoke') THEN 'destructive'
|
||||
ELSE 'read_only'
|
||||
END AS risk_class,
|
||||
CASE
|
||||
WHEN $2 IN ('read_only','reversible_low') THEN 'auto-act'
|
||||
WHEN $2 = 'config_mutation' THEN 'operator-approval'
|
||||
ELSE 'operator-approval+confirmation'
|
||||
END AS approval
|
||||
FROM entities e WHERE e.slug = $1`, slug, action), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_change_history", Description: "Last N change-ledger entries for an entity",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_slug", "string", "Entity slug"},
|
||||
prop{"limit", "integer", "Max entries (default 20)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_slug"].(string)
|
||||
limit := int(getFloat(args, "limit", 20))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT al.ts AS timestamp, al.actor_type, al.actor_id::text AS actor_label,
|
||||
al.action, al.method, al.path,
|
||||
al.detail::text AS details
|
||||
FROM audit_log al
|
||||
JOIN entities e ON e.id = al.entity_id
|
||||
WHERE e.slug = $1
|
||||
ORDER BY al.ts DESC
|
||||
LIMIT $2`, slug, limit), "change_log"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_state_snapshot", Description: "Last scheduler Observe-pass: fleet health, disk, drift count",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.state,
|
||||
COALESCE(st.health, 'unknown') AS health,
|
||||
COALESCE(st.last_check_at::text, '') AS last_check
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.state IS NOT NULL
|
||||
OR st.health IS NOT NULL
|
||||
ORDER BY st.health, e.slug
|
||||
LIMIT 200
|
||||
`), "fleet_snapshot"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "list_my_secrets", Description: "List secrets accessible to this client by public key",
|
||||
InputSchema: objSchema(prop{"caller_pubkey", "string", "Age public key of the caller (optional)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
pubkey, _ := args["caller_pubkey"].(string)
|
||||
// Match entities where age_pubkey attribute contains the caller's key.
|
||||
query := `
|
||||
SELECT e.slug, e.type, e.name,
|
||||
e.attributes->>'age_pubkey' AS age_pubkey
|
||||
FROM entities e
|
||||
WHERE e.attributes->>'age_pubkey' IS NOT NULL`
|
||||
var dbArgs []any
|
||||
if pubkey != "" {
|
||||
query += ` AND e.attributes->>'age_pubkey' = $1`
|
||||
dbArgs = append(dbArgs, pubkey)
|
||||
}
|
||||
query += ` ORDER BY e.slug LIMIT 100`
|
||||
return queryRows(ctx, pool, query, dbArgs...), nil
|
||||
}},
|
||||
}
|
||||
}
|
||||
@@ -282,23 +282,6 @@ func generateApprovalToken(approvalID uuid.UUID, secret string) string {
|
||||
return hex.EncodeToString(mac.Sum(nil))
|
||||
}
|
||||
|
||||
// VerifyApprovalToken checks a token against the stored hash.
|
||||
func VerifyApprovalToken(ctx context.Context, pool *db.Pool, approvalID uuid.UUID, token string) bool {
|
||||
var tokenHash *string
|
||||
var status string
|
||||
var expiresAt time.Time
|
||||
err := pool.QueryRow(ctx,
|
||||
"SELECT token_hash, status, expires_at FROM approvals WHERE entity_id = $1",
|
||||
approvalID).Scan(&tokenHash, &status, &expiresAt)
|
||||
if err != nil || tokenHash == nil {
|
||||
return false
|
||||
}
|
||||
if status != "pending" || expiresAt.Before(time.Now()) {
|
||||
return false
|
||||
}
|
||||
return *tokenHash == hashToken(token)
|
||||
}
|
||||
|
||||
func hashToken(token string) string {
|
||||
h := sha256.Sum256([]byte(token))
|
||||
return hex.EncodeToString(h[:])
|
||||
|
||||
187
internal/notifier/notifier_test.go
Normal file
187
internal/notifier/notifier_test.go
Normal file
@@ -0,0 +1,187 @@
|
||||
package notifier
|
||||
|
||||
import (
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
var hex64Re = regexp.MustCompile(`^[0-9a-f]{64}$`)
|
||||
|
||||
func TestHashToken(t *testing.T) {
|
||||
// sha256("") == e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
|
||||
emptyHash := "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
|
||||
|
||||
t.Run("determinism same input same output", func(t *testing.T) {
|
||||
a := hashToken("approval-token-123")
|
||||
b := hashToken("approval-token-123")
|
||||
if a != b {
|
||||
t.Fatalf("hashToken not deterministic: %q vs %q", a, b)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("empty string known sha256", func(t *testing.T) {
|
||||
got := hashToken("")
|
||||
if got != emptyHash {
|
||||
t.Fatalf("hashToken(\"\") = %q, want %q", got, emptyHash)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("different inputs different outputs", func(t *testing.T) {
|
||||
a := hashToken("one")
|
||||
b := hashToken("two")
|
||||
if a == b {
|
||||
t.Fatalf("hashToken collided for different inputs: %q", a)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("output is valid 64-char hex", func(t *testing.T) {
|
||||
for _, in := range []string{"", "abc", "some-longer-token-value-xyz"} {
|
||||
got := hashToken(in)
|
||||
if !hex64Re.MatchString(got) {
|
||||
t.Fatalf("hashToken(%q) = %q, not 64-char lowercase hex", in, got)
|
||||
}
|
||||
// also must decode cleanly to 32 bytes
|
||||
b, err := hex.DecodeString(got)
|
||||
if err != nil {
|
||||
t.Fatalf("hashToken(%q) decode error: %v", in, err)
|
||||
}
|
||||
if len(b) != 32 {
|
||||
t.Fatalf("hashToken(%q) decoded len = %d, want 32", in, len(b))
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestGenerateApprovalToken(t *testing.T) {
|
||||
id := uuid.New()
|
||||
|
||||
t.Run("output is 64-char hex", func(t *testing.T) {
|
||||
tok := generateApprovalToken(id, "super-secret")
|
||||
if !hex64Re.MatchString(tok) {
|
||||
t.Fatalf("generateApprovalToken = %q, not 64-char lowercase hex", tok)
|
||||
}
|
||||
b, err := hex.DecodeString(tok)
|
||||
if err != nil {
|
||||
t.Fatalf("decode error: %v", err)
|
||||
}
|
||||
if len(b) != 32 {
|
||||
t.Fatalf("decoded len = %d, want 32 (sha256)", len(b))
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("empty secret falls back to dev secret no panic", func(t *testing.T) {
|
||||
tok := generateApprovalToken(id, "")
|
||||
if tok == "" {
|
||||
t.Fatal("empty secret produced empty token")
|
||||
}
|
||||
if !hex64Re.MatchString(tok) {
|
||||
t.Fatalf("empty-secret token %q not 64-char hex", tok)
|
||||
}
|
||||
})
|
||||
|
||||
// Non-determinism: time.Now().UnixNano() is embedded in the HMAC message,
|
||||
// so two calls with identical inputs produce different tokens (unless the
|
||||
// clock has nanosecond-identical reads, which we do not assert against).
|
||||
t.Run("same inputs twice produce different tokens (time-based)", func(t *testing.T) {
|
||||
a := generateApprovalToken(id, "stable-secret")
|
||||
b := generateApprovalToken(id, "stable-secret")
|
||||
if a == b {
|
||||
// Not a hard failure (clock granularity), but document expectation.
|
||||
t.Logf("note: two immediate calls returned identical token %q — clock resolution collapsed", a)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("different secrets produce different tokens", func(t *testing.T) {
|
||||
a := generateApprovalToken(id, "secret-a")
|
||||
b := generateApprovalToken(id, "secret-b")
|
||||
if a == b {
|
||||
t.Fatalf("different secrets produced same token %q", a)
|
||||
}
|
||||
})
|
||||
|
||||
// HMAC correctness: re-derive the token with the same secret + approvalID
|
||||
// using a freshly captured timestamp window is impossible because we don't
|
||||
// observe the embedded timestamp. Instead, verify the token is a valid
|
||||
// HMAC-SHA256 by brute-forcing a small time window around now: reconstruct
|
||||
// mac(secret, approvalID || ts) for ts in [now-N, now] and confirm one
|
||||
// matches. This proves the token genuinely is an HMAC over (approvalID, ts)
|
||||
// with the supplied secret.
|
||||
t.Run("token is HMAC-SHA256 over approvalID+timestamp with secret", func(t *testing.T) {
|
||||
secret := "hmac-verify-secret"
|
||||
before := nowNanos()
|
||||
tok := generateApprovalToken(id, secret)
|
||||
after := nowNanos()
|
||||
|
||||
// The token's embedded ts is captured inside generateApprovalToken,
|
||||
// which is called after `before` was sampled — so ts ∈ [before, after].
|
||||
// Add a tiny ±band to absorb scheduler jitter on loaded runners.
|
||||
lo := before - 10_000
|
||||
hi := after + 10_000
|
||||
matched := false
|
||||
for ts := lo; ts <= hi; ts++ {
|
||||
mac := hmac.New(sha256.New, []byte(secret))
|
||||
mac.Write([]byte(id.String()))
|
||||
mac.Write([]byte(formatInt(ts)))
|
||||
cand := hex.EncodeToString(mac.Sum(nil))
|
||||
if hmac.Equal([]byte(cand), []byte(tok)) {
|
||||
matched = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !matched {
|
||||
t.Fatalf("token %q did not match any HMAC in window [%d,%d]; not a valid HMAC-SHA256 over approvalID+ts", tok, lo, hi)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("empty secret HMAC uses dev fallback secret", func(t *testing.T) {
|
||||
before := nowNanos()
|
||||
tok := generateApprovalToken(id, "")
|
||||
after := nowNanos()
|
||||
dev := "dev-secret-do-not-use-in-prod"
|
||||
lo := before - 10_000
|
||||
hi := after + 10_000
|
||||
matched := false
|
||||
for ts := lo; ts <= hi; ts++ {
|
||||
mac := hmac.New(sha256.New, []byte(dev))
|
||||
mac.Write([]byte(id.String()))
|
||||
mac.Write([]byte(formatInt(ts)))
|
||||
cand := hex.EncodeToString(mac.Sum(nil))
|
||||
if hmac.Equal([]byte(cand), []byte(tok)) {
|
||||
matched = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !matched {
|
||||
t.Fatalf("empty-secret token %q did not match dev-fallback HMAC", tok)
|
||||
}
|
||||
})
|
||||
|
||||
// sanity: token should not leak the secret in plaintext
|
||||
t.Run("token does not contain secret substring", func(t *testing.T) {
|
||||
secret := "leakcheck-secret-xyz"
|
||||
tok := generateApprovalToken(id, secret)
|
||||
if strings.Contains(tok, secret) {
|
||||
t.Fatalf("token %q contains secret substring %q", tok, secret)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// nowNanos returns the current nanosecond count, matching the time source
|
||||
// used by generateApprovalToken (time.Now().UnixNano()).
|
||||
func nowNanos() int64 {
|
||||
return time.Now().UnixNano()
|
||||
}
|
||||
|
||||
// formatInt mirrors fmt.Sprintf("%d", ...) used by the production code so the
|
||||
// re-derivation in tests is byte-identical.
|
||||
func formatInt(n int64) string {
|
||||
return fmt.Sprintf("%d", n)
|
||||
}
|
||||
@@ -92,21 +92,7 @@ func (c *Classifier) ClassifySignal(ctx context.Context, signalEntityID, targetE
|
||||
}
|
||||
|
||||
// Determine route
|
||||
route := "escalate"
|
||||
autonomyCheck := ""
|
||||
|
||||
if globalAutoAct == "off" || globalAutoAct == "false" {
|
||||
route = "escalate"
|
||||
autonomyCheck = "blocked: global auto_act disabled"
|
||||
} else if entityAutoAct == "true" {
|
||||
route = "escalate"
|
||||
autonomyCheck = "blocked: per-entity kill-switch"
|
||||
} else if approvalRequired == "none" {
|
||||
route = "auto-act"
|
||||
autonomyCheck = "allowed"
|
||||
} else {
|
||||
autonomyCheck = "requires approval: " + approvalRequired
|
||||
}
|
||||
route, autonomyCheck := determineRoute(globalAutoAct, entityAutoAct, approvalRequired)
|
||||
|
||||
// Compute blast radius
|
||||
blastRadius := computeBlastRadius(ctx, c.DB, targetEntityID)
|
||||
@@ -136,6 +122,21 @@ func (c *Classifier) ClassifySignal(ctx context.Context, signalEntityID, targetE
|
||||
}, nil
|
||||
}
|
||||
|
||||
// determineRoute evaluates autonomy settings and approval requirements to
|
||||
// decide whether a signal should auto-act, escalate, or hold for approval.
|
||||
func determineRoute(globalAutoAct, entityAutoAct, approvalRequired string) (route, autonomyCheck string) {
|
||||
if globalAutoAct == "off" || globalAutoAct == "false" {
|
||||
return "escalate", "blocked: global auto_act disabled"
|
||||
}
|
||||
if entityAutoAct == "true" {
|
||||
return "escalate", "blocked: per-entity kill-switch"
|
||||
}
|
||||
if approvalRequired == "none" {
|
||||
return "auto-act", "allowed"
|
||||
}
|
||||
return "escalate", "requires approval: " + approvalRequired
|
||||
}
|
||||
|
||||
// computeBlastRadius traverses relationships to find affected entities.
|
||||
func computeBlastRadius(ctx context.Context, dbc interface {
|
||||
Query(ctx context.Context, sql string, args ...any) (pgx.Rows, error)
|
||||
|
||||
62
internal/policy/classify_test.go
Normal file
62
internal/policy/classify_test.go
Normal file
@@ -0,0 +1,62 @@
|
||||
package policy
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestDetermineRoute(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
globalAutoAct string
|
||||
entityAutoAct string
|
||||
approvalRequired string
|
||||
wantRoute string
|
||||
wantCheck string
|
||||
}{
|
||||
{
|
||||
"global auto_act off blocks everything",
|
||||
"off", "", "none",
|
||||
"escalate", "blocked: global auto_act disabled",
|
||||
},
|
||||
{
|
||||
"global auto_act false blocks everything",
|
||||
"false", "", "none",
|
||||
"escalate", "blocked: global auto_act disabled",
|
||||
},
|
||||
{
|
||||
"per-entity kill-switch blocks",
|
||||
"on", "true", "none",
|
||||
"escalate", "blocked: per-entity kill-switch",
|
||||
},
|
||||
{
|
||||
"approval none auto-acts",
|
||||
"on", "", "none",
|
||||
"auto-act", "allowed",
|
||||
},
|
||||
{
|
||||
"approval required escalates",
|
||||
"on", "", "operator",
|
||||
"escalate", "requires approval: operator",
|
||||
},
|
||||
{
|
||||
"approval none with empty global defaults to auto-act",
|
||||
"", "", "none",
|
||||
"auto-act", "allowed",
|
||||
},
|
||||
{
|
||||
"global on, no entity kill, approval confirmation",
|
||||
"on", "", "confirmation",
|
||||
"escalate", "requires approval: confirmation",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
route, check := determineRoute(tt.globalAutoAct, tt.entityAutoAct, tt.approvalRequired)
|
||||
if route != tt.wantRoute {
|
||||
t.Errorf("route = %q, want %q", route, tt.wantRoute)
|
||||
}
|
||||
if check != tt.wantCheck {
|
||||
t.Errorf("autonomyCheck = %q, want %q", check, tt.wantCheck)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -69,11 +69,14 @@ var destructivePatterns = []*regexp.Regexp{
|
||||
var readOnlyLeadPattern = regexp.MustCompile(
|
||||
`^(cat|less|head|tail|ls|stat|file|du|df|free|uptime|uname|hostname|whoami|id|ip|ss|netstat|ping|` +
|
||||
`journalctl|dmesg|ps|top|htop|env|printenv|echo|which|whereis|` +
|
||||
`grep|egrep|fgrep|rg|wc|sort|uniq|cut|tr|tee|` +
|
||||
`grep|egrep|fgrep|rg|wc|sort|uniq|cut|tr|tee|find|tree|locate|` +
|
||||
`dpkg\s+(-l|-s|--list|--status)\b|apt\s+(list|search|show)\b|` +
|
||||
`systemctl\s+(status|is-active|is-enabled|is-failed|list-units)|` +
|
||||
`systemctl\s+(status|is-active|is-enabled|is-failed|list-units|list-unit-files|list-timers|show)\b|` +
|
||||
`timedatectl|hostnamectl|systemd-analyze|` +
|
||||
`docker\s+(ps|images|inspect|logs|version|info|stats)|` +
|
||||
`docker\s+compose\s+(logs|ps|top|config|images|port|cp)\b|` +
|
||||
`pct\s+(status|config|list)|qm\s+(status|config|list)|pvesh\s+get|` +
|
||||
`rclone\s+(ls|lsl|md5sum|check|cryptcheck)\b|` +
|
||||
`git\s+(status|log|diff|show|branch|remote)|` +
|
||||
`curl\s+-.*-I\b|curl\s+.*--head\b)\b`)
|
||||
|
||||
|
||||
@@ -15,6 +15,23 @@ func TestClassifyCommand_ReadOnly(t *testing.T) {
|
||||
"git status",
|
||||
"sudo cat /var/log/syslog",
|
||||
"ip a",
|
||||
// P4: newly added read-only verbs.
|
||||
"find /var/log/rclone-backup/ -name runs.jsonl",
|
||||
"tree /etc/caddy",
|
||||
"locate Caddyfile",
|
||||
"systemctl list-timers --all",
|
||||
"systemctl list-units --type=service",
|
||||
"systemctl list-unit-files --state=enabled",
|
||||
"systemctl show caddy",
|
||||
"timedatectl",
|
||||
"hostnamectl",
|
||||
"systemd-analyze blame",
|
||||
"rclone lsl proton:library-backup",
|
||||
// docker compose read-only subcommands (F1 fix).
|
||||
"docker compose logs --tail=100",
|
||||
"docker compose ps",
|
||||
"docker compose top",
|
||||
"docker compose config",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got != RiskReadOnly {
|
||||
@@ -99,6 +116,9 @@ func TestClassifyCommand_CompoundReadOnly(t *testing.T) {
|
||||
"docker ps | grep caddy",
|
||||
"systemctl status caddy 2>&1; journalctl -u caddy -n 5 --no-pager",
|
||||
"sudo systemctl status caddy; sudo journalctl -u caddy -n 5",
|
||||
// P4: the exact compound from session d0d562e0 — find + ls + tail +
|
||||
// echo + journalctl, all read-only segments.
|
||||
"ls -lt /var/log/rclone-backup/ | head -20 && tail -3 /var/log/rclone-backup/runs.jsonl || echo \"not found\" && find /var/log/rclone-backup/ -name 'runs.jsonl'",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got != RiskReadOnly {
|
||||
|
||||
208
internal/scheduler/scheduler_test.go
Normal file
208
internal/scheduler/scheduler_test.go
Normal file
@@ -0,0 +1,208 @@
|
||||
package scheduler
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestParsePingLatency(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
out []byte
|
||||
want float64
|
||||
}{
|
||||
{
|
||||
name: "linux rtt format",
|
||||
out: []byte("PING host (1.2.3.4): 56 data bytes\n--- host ping statistics ---\n5 packets transmitted, 5 received, 0% packet loss\nrtt min/avg/max/mdev = 0.1/2.5/5.0/1.2 ms\n"),
|
||||
want: 2.5,
|
||||
},
|
||||
{
|
||||
name: "macos round-trip format",
|
||||
out: []byte("PING host (1.2.3.4): 56 data bytes\n--- host ping statistics ---\n5 packets transmitted, 5 received, 0% packet loss\nround-trip min/avg/max/stddev = 0.1/2.5/5.0/1.2 ms\n"),
|
||||
want: 2.5,
|
||||
},
|
||||
{
|
||||
name: "no match empty string",
|
||||
out: []byte(""),
|
||||
want: 0,
|
||||
},
|
||||
{
|
||||
name: "malformed number in avg slot",
|
||||
out: []byte("rtt min/avg/max/mdev = 0.1/abc/5.0/1.2 ms\n"),
|
||||
want: 0,
|
||||
},
|
||||
{
|
||||
name: "truly empty input nil",
|
||||
out: nil,
|
||||
want: 0,
|
||||
},
|
||||
{
|
||||
name: "linux integer avg",
|
||||
out: []byte("rtt min/avg/max/mdev = 1/3/5/0.5 ms\n"),
|
||||
want: 3.0,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := parsePingLatency(tc.out)
|
||||
if got != tc.want {
|
||||
t.Fatalf("parsePingLatency(%q) = %v, want %v", tc.out, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllowlistedScript(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
in string
|
||||
want bool
|
||||
}{
|
||||
{name: "cpu_check.sh", in: "cpu_check.sh", want: true},
|
||||
{name: "disk_usage_check.sh", in: "disk_usage_check.sh", want: true},
|
||||
{name: "hyphen allowed", in: "foo-bar.sh", want: true},
|
||||
{name: "underscore allowed", in: "foo_bar.sh", want: true},
|
||||
{name: "uppercase rejected", in: "Foo.sh", want: false},
|
||||
{name: "bare .sh rejected (no name)", in: ".sh", want: false},
|
||||
{name: "path traversal rejected", in: "../etc/passwd", want: false},
|
||||
{name: "empty rejected", in: "", want: false},
|
||||
// foo.sh.sh: regex ^[a-z][a-z0-9_-]+\.sh$ — the [a-z0-9_-]+ cannot
|
||||
// cross the first '.', so after matching "foo.sh" the trailing ".sh"
|
||||
// breaks the $ anchor → no match.
|
||||
{name: "doubled .sh.sh rejected", in: "foo.sh.sh", want: false},
|
||||
{name: "wrong extension rejected", in: "foo.txt", want: false},
|
||||
{name: "single char name rejected (needs 2+)", in: "a.sh", want: false},
|
||||
{name: "two char name accepted", in: "ab.sh", want: true},
|
||||
{name: "digit after first char", in: "a1.sh", want: true},
|
||||
{name: "leading digit rejected", in: "1foo.sh", want: false},
|
||||
{name: "dot in middle rejected", in: "foo.bar.sh", want: false},
|
||||
{name: "space rejected", in: "foo bar.sh", want: false},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := allowlistedScript(tc.in)
|
||||
if got != tc.want {
|
||||
t.Fatalf("allowlistedScript(%q) = %v, want %v", tc.in, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestEvaluateSeverity(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
kind string
|
||||
signalKind string
|
||||
config []byte
|
||||
metrics map[string]float64
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "empty config down signal",
|
||||
kind: "ping",
|
||||
signalKind: "down",
|
||||
config: nil,
|
||||
metrics: map[string]float64{},
|
||||
want: "critical",
|
||||
},
|
||||
{
|
||||
name: "empty config high_temp signal",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: nil,
|
||||
metrics: map[string]float64{},
|
||||
want: "warning",
|
||||
},
|
||||
{
|
||||
name: "crit only exceeded",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: []byte(`{"temp":{"crit":90}}`),
|
||||
metrics: map[string]float64{"temp": 95},
|
||||
want: "critical",
|
||||
},
|
||||
{
|
||||
name: "warn only exceeded",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: []byte(`{"temp":{"warn":80}}`),
|
||||
metrics: map[string]float64{"temp": 85},
|
||||
want: "warning",
|
||||
},
|
||||
{
|
||||
name: "warn and crit, warn exceeded only",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: []byte(`{"temp":{"warn":80,"crit":90}}`),
|
||||
metrics: map[string]float64{"temp": 85},
|
||||
want: "warning",
|
||||
},
|
||||
{
|
||||
name: "warn and crit, crit exceeded",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: []byte(`{"temp":{"warn":80,"crit":90}}`),
|
||||
metrics: map[string]float64{"temp": 95},
|
||||
want: "critical",
|
||||
},
|
||||
{
|
||||
name: "metric below thresholds falls through to warning",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: []byte(`{"temp":{"warn":80,"crit":90}}`),
|
||||
metrics: map[string]float64{"temp": 70},
|
||||
want: "warning",
|
||||
},
|
||||
{
|
||||
name: "metric below thresholds falls through to down critical",
|
||||
kind: "ping",
|
||||
signalKind: "down",
|
||||
config: []byte(`{"ping_latency_ms":{"warn":100,"crit":200}}`),
|
||||
metrics: map[string]float64{"ping_latency_ms": 50},
|
||||
want: "critical",
|
||||
},
|
||||
{
|
||||
name: "crit zero skipped falls through",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: []byte(`{"temp":{"crit":0}}`),
|
||||
metrics: map[string]float64{"temp": 95},
|
||||
want: "warning",
|
||||
},
|
||||
{
|
||||
name: "metric not in thresholds ignored",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: []byte(`{"temp":{"warn":80,"crit":90}}`),
|
||||
metrics: map[string]float64{"cpu": 99},
|
||||
want: "warning",
|
||||
},
|
||||
{
|
||||
name: "equal to warn triggers warning",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: []byte(`{"temp":{"warn":80,"crit":90}}`),
|
||||
metrics: map[string]float64{"temp": 80},
|
||||
want: "warning",
|
||||
},
|
||||
{
|
||||
name: "equal to crit triggers critical",
|
||||
kind: "ssh-script",
|
||||
signalKind: "high_temp",
|
||||
config: []byte(`{"temp":{"warn":80,"crit":90}}`),
|
||||
metrics: map[string]float64{"temp": 90},
|
||||
want: "critical",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := evaluateSeverity(tc.kind, tc.signalKind, tc.config, tc.metrics)
|
||||
if got != tc.want {
|
||||
t.Fatalf("evaluateSeverity(%q, %q, %s, %v) = %q, want %q",
|
||||
tc.kind, tc.signalKind, tc.config, tc.metrics, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
640
inventory.yaml
640
inventory.yaml
@@ -1,618 +1,28 @@
|
||||
# Homelab inventory — canonical structured topology.
|
||||
# DEPRECATED — 2026-07-17 (R5). Do not edit. Do not consume.
|
||||
#
|
||||
# Single source of truth. hosts/*.yaml is generated from this file by
|
||||
# mcp/build_host_files.py; do NOT edit those by hand.
|
||||
# This root-level inventory.yaml is the Python-era topology file. It was
|
||||
# superseded on 2026-07-07 by the DB-native model (ADR 0003):
|
||||
#
|
||||
# Conventions:
|
||||
# - hostname keys MUST match the actual `hostname` of the machine (on
|
||||
# macOS: `scutil --get LocalHostName` if set).
|
||||
# - `os:` one of: linux, macos
|
||||
# - `kind:` one of: proxmox-host, lxc, vm, workstation, external
|
||||
# ("external" is reserved for hosts the homelab CLI manages via ssh but
|
||||
# that aren't homelab clients themselves — e.g. the IONOS netbird VPS
|
||||
# with no /etc/age/key.txt and no /opt/homelab-context clone.)
|
||||
# - `mesh:` lists addresses the host is reachable at. Both `netbird` and
|
||||
# `tailscale` are accepted during the migration (see infrastructure/mesh.md).
|
||||
# Prefer netbird FQDNs over raw IPs.
|
||||
# - `age_pubkey:` provisioned by secrets-issuance on first bootstrap and
|
||||
# committed back via `homelab client add --finalize-pubkey <key>`.
|
||||
# - When a service moves hosts, update only the `services:` section here;
|
||||
# never duplicate addresses elsewhere.
|
||||
# - `ssh.user:` per-host login user. Default is `root` if omitted (matches
|
||||
# every LXC + the PVE host). Set explicitly for workstations whose login
|
||||
# user differs from `root`. Used by the `homelab` CLI to build
|
||||
# `user@host` and to inform anyone running raw `netbird ssh` (which
|
||||
# defaults to the LOCAL username — the gotcha that creates "user not
|
||||
# found" errors when ssh'ing INTO machines that only have `root`).
|
||||
# - The Postgres database is the single source of truth for all structured
|
||||
# data. Query it via MCP `get_entity` / `list_entities` or the REST API.
|
||||
# - `seeds/inventory.yaml` is the bootstrap + DR seed manifest, ingested
|
||||
# idempotently into the DB at deploy time (content-hashed via
|
||||
# `seed_versions`). `oikos export` regenerates it from the DB for VC.
|
||||
# - `archive/knowledge/` holds the frozen legacy narrative wiki.
|
||||
#
|
||||
# `homelab client add/remove` does surgical line-edits — comments survive.
|
||||
# Avoid round-tripping the file through yaml.safe_dump (it strips comments).
|
||||
|
||||
mesh:
|
||||
primary: netbird
|
||||
accepted:
|
||||
- netbird
|
||||
- tailscale
|
||||
netbird_subnet: 100.122.0.0/16
|
||||
netbird_domain: netbird.selfhosted
|
||||
# Service contract (Oikos, 2026-07-05): each service should carry
|
||||
# backend host/container that runs it (required)
|
||||
# url public URL if ingress-exposed
|
||||
# doc_page owning wiki page
|
||||
# config_repo tracked config repo, if any (mutations go commit+push)
|
||||
# health health-check URL if it differs from `url`
|
||||
# risk_notes what an agent must know before touching it
|
||||
# See oikos/ontology.yaml + oikos/policy.yaml.
|
||||
services:
|
||||
proxmox_ui:
|
||||
url: https://proxmox.hubris.network
|
||||
backend: hubris
|
||||
port: 8006
|
||||
doc_page: knowledge/wiki/hosts/hubris.md
|
||||
risk_notes: hypervisor UI — changes here affect every guest on the node
|
||||
gitea:
|
||||
url: https://git.hubris.network
|
||||
backend: gitea
|
||||
backend_url: http://192.168.8.121:3000
|
||||
doc_page: knowledge/wiki/containers/104-gitea.md
|
||||
config_repo: dtoro/gitea-customizations
|
||||
risk_notes: hosts all config repos + deploy webhooks; outage blocks auto-deploy and sync
|
||||
caddy:
|
||||
backend: caddy
|
||||
role: reverse-proxy
|
||||
note: terminates all *.hubris.network
|
||||
doc_page: knowledge/wiki/containers/121-caddy.md
|
||||
config_repo: dtoro/caddy-conf
|
||||
risk_notes: wide blast radius — every *.hubris.network route rides on it (see oikos/policy.yaml service_overrides)
|
||||
authentik:
|
||||
url: https://auth.hubris.network
|
||||
backend: netbird-vps
|
||||
doc_page: knowledge/wiki/containers/106-auth-outpost.md
|
||||
note: >-
|
||||
core runs on the VPS since 2026-05-31; LAN forward-auth outpost is
|
||||
auth-outpost (LXC 106) at 192.168.8.6:9000. Previous backend value
|
||||
"authentik" referenced the retired embedded-outpost host (LXC 124).
|
||||
risk_notes: SSO provider — outage locks login to OIDC/forward-auth services
|
||||
dns:
|
||||
backend: dns
|
||||
note: Technitium DNS, split-horizon zone
|
||||
doc_page: knowledge/wiki/containers/107-dns.md
|
||||
risk_notes: LAN-wide resolver — misconfig breaks name resolution for every client
|
||||
jellyfin:
|
||||
url: https://media.hubris.network
|
||||
backend: jellyfin
|
||||
doc_page: knowledge/wiki/containers/101-jellyfin.md
|
||||
risk_notes: native Authentik OIDC via SSO-Auth plugin, no Caddy forward-auth gate; VAAPI transcode depends on GPU passthrough on strong
|
||||
nextcloud:
|
||||
url: https://cloud.hubris.network
|
||||
backend: nextcloud
|
||||
doc_page: knowledge/wiki/containers/114-nextcloud.md
|
||||
paperless:
|
||||
url: https://paperless.hubris.network
|
||||
backend: paperless
|
||||
doc_page: knowledge/wiki/containers/103-paperless.md
|
||||
risk_notes: document archive — treat data as irreplaceable; DB operations are destructive-class
|
||||
matrix:
|
||||
url: https://matrix.hubris.network
|
||||
backend: elementsynapse
|
||||
doc_page: knowledge/wiki/containers/118-elementsynapse.md
|
||||
risk_notes: alert/approval channel for Oikos — outage silences agent escalation
|
||||
photos:
|
||||
url: https://photos.hubris.network
|
||||
backend: mule-images
|
||||
doc_page: knowledge/wiki/containers/120-mule-images.md
|
||||
config_repo: dtoro/mule-image
|
||||
arr_stack:
|
||||
backend: arriman
|
||||
note: jellyseerr / qbit / sab on docker compose
|
||||
doc_page: knowledge/wiki/containers/122-arriman.md
|
||||
artifacto:
|
||||
backend: apps
|
||||
url: https://artifacto.hubris.network
|
||||
doc_page: knowledge/wiki/containers/105-apps.md
|
||||
config_repo: dtoro/Artifacto
|
||||
trmnl:
|
||||
backend: trmnl
|
||||
url: https://trmnl.hubris.network
|
||||
note: self-hosted middleware for TRMNL e-ink plugins (polled by TRMNL cloud)
|
||||
doc_page: knowledge/wiki/containers/128-trmnl.md
|
||||
config_repo: dtoro/terminalito
|
||||
zimaos:
|
||||
url: https://zimaos.hubris.network
|
||||
backend: zimaos
|
||||
doc_page: knowledge/wiki/vms/100-zimaos.md
|
||||
haos:
|
||||
backend: haos
|
||||
doc_page: knowledge/wiki/vms/108-haos.md
|
||||
teddycloud:
|
||||
url: https://teddy.hubris.network
|
||||
backend: teddycloud
|
||||
doc_page: knowledge/wiki/containers/131-teddycloud.md
|
||||
note: self-hosted TeddyCloud (Toniebox cloud reimplementation), docker compose
|
||||
risk_notes: no Caddy forward-auth gate (unlike sab.hubris.network on the same Caddyfile) —
|
||||
reachable to anyone on the LAN/mesh who can resolve teddy.hubris.network; undocumented
|
||||
in inventory.yaml until 2026-07-06 (drift-caught)
|
||||
homelab_mcp:
|
||||
backend: apps
|
||||
port: 9810
|
||||
systemd_unit: homelab-mcp
|
||||
public_host: mcp.hubris.network
|
||||
endpoint: https://mcp.hubris.network/mcp
|
||||
doc_page: knowledge/wiki/infrastructure/homelab-context.md
|
||||
config_repo: dtoro/oikos
|
||||
note: MCP server. Read-only context + management. Reachable on the LAN via Caddy
|
||||
and from off-LAN via Netbird (192.168.8.0/24 is a network resource routed through
|
||||
hubris).
|
||||
risk_notes: agents' primary read surface — outage degrades every agent to grepping the clone
|
||||
secrets_issuance:
|
||||
backend: apps
|
||||
port: 9820
|
||||
systemd_unit: secrets-issuance
|
||||
public_host: secrets.hubris.network
|
||||
endpoint: https://secrets.hubris.network/issue
|
||||
doc_page: .agents/operations/agent-enrollment.md
|
||||
config_repo: dtoro/oikos
|
||||
note: Issues per-client age private keys. Gated at source-IP layer (mesh + LAN
|
||||
subnets in MESH_SUBNETS).
|
||||
risk_notes: identity issuance — any change is security-sensitive; key operations are destructive-class
|
||||
hosts:
|
||||
hubris:
|
||||
kind: proxmox-host
|
||||
os: linux
|
||||
role: hypervisor
|
||||
lan_ip: 192.168.8.77
|
||||
mesh:
|
||||
netbird:
|
||||
ip: 100.122.38.109
|
||||
fqdn: proxmox-server.netbird.selfhosted
|
||||
ssh:
|
||||
port: 22
|
||||
netbird_port: 22022
|
||||
user: root
|
||||
mounts:
|
||||
- /mnt/library
|
||||
age_pubkey: age1xkklkvnk5z0fsnh6cfgv70hy9ksfy8rdprwerzw4yk3p4p7cxcqs2yvpz6
|
||||
trmnl:
|
||||
kind: lxc
|
||||
pve_id: 128
|
||||
host: hubris
|
||||
os: linux
|
||||
role: trmnl-middleware
|
||||
lan_ip: 192.168.8.211
|
||||
public_host: trmnl.hubris.network
|
||||
# not yet mesh/SOPS-enrolled — see containers/128-trmnl.md
|
||||
house:
|
||||
kind: lxc
|
||||
pve_id: 129
|
||||
host: strong
|
||||
os: linux
|
||||
role: family-planner
|
||||
lan_ip: 192.168.8.244
|
||||
public_host: house.hubris.network
|
||||
notes:
|
||||
- Docker host for Yuvomi (family planner). Created 2026-06-26.
|
||||
- Migrated from hubris to strong 2026-07-05 (Phase 1 of strong migration plan).
|
||||
- Runs Yuvomi container + WebDAV doc bridge to paperless
|
||||
- 192.168.8.212 was the hubris IP before migration (briefly picked up by teddycloud via
|
||||
DHCP; teddycloud has since been given a static IP, see hosts.teddycloud)
|
||||
age_pubkey: age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h
|
||||
jellyfin:
|
||||
kind: lxc
|
||||
pve_id: 101
|
||||
host: strong
|
||||
os: linux
|
||||
role: media-server
|
||||
lan_ip: 192.168.8.246
|
||||
public_host: media.hubris.network
|
||||
mesh:
|
||||
tailscale:
|
||||
fqdn: jellyfin
|
||||
mounts:
|
||||
- /mnt/media_local
|
||||
notes:
|
||||
- Jellyfin 10.11.11 with VAAPI hardware acceleration (Radeon 680M iGPU on strong)
|
||||
- 4 cores / 8 GiB RAM / 1 GiB swap
|
||||
- SSO-Auth plugin v4.0.0.4 with Authentik OIDC (no Caddy forward-auth gate)
|
||||
- GPU passed via dev0+dev1: /dev/dri/renderD128 + card0
|
||||
- Migrated from hubris to strong 2026-07-05 (Phase 2). Library on ludo-lvm.
|
||||
age_pubkey: ''
|
||||
nfs-export:
|
||||
kind: lxc
|
||||
pve_id: 102
|
||||
host: hubris
|
||||
os: linux
|
||||
role: storage-export
|
||||
lan_ip: 192.168.8.200
|
||||
paperless:
|
||||
kind: lxc
|
||||
pve_id: 103
|
||||
host: hubris
|
||||
os: linux
|
||||
role: document-archive
|
||||
lan_ip: 192.168.8.130
|
||||
public_host: paperless.hubris.network
|
||||
mesh:
|
||||
tailscale:
|
||||
fqdn: paperless
|
||||
mounts:
|
||||
- /mnt/library
|
||||
age_pubkey: ''
|
||||
gitea:
|
||||
kind: lxc
|
||||
pve_id: 104
|
||||
host: hubris
|
||||
os: linux
|
||||
role: git-server
|
||||
lan_ip: 192.168.8.121
|
||||
public_host: git.hubris.network
|
||||
backend_port: 3000
|
||||
mesh:
|
||||
tailscale:
|
||||
fqdn: gitea
|
||||
mounts:
|
||||
- /mnt/library
|
||||
notes:
|
||||
- Bare repos live at /mnt/library/repos/dtoro/*.git
|
||||
age_pubkey: ''
|
||||
apps:
|
||||
kind: lxc
|
||||
pve_id: 105
|
||||
host: hubris
|
||||
os: linux
|
||||
role: docker-apps
|
||||
lan_ip: 192.168.8.205
|
||||
public_hosts:
|
||||
- artifacto.hubris.network
|
||||
mesh:
|
||||
tailscale:
|
||||
ip: 100.121.171.122
|
||||
fqdn: apps
|
||||
mounts:
|
||||
- /mnt/library
|
||||
runs:
|
||||
- artifacto
|
||||
- plantuml
|
||||
- homelab-mcp
|
||||
- secrets-issuance
|
||||
# booklore removed 2026-06-29 → migrated to grimmory (LXC 130)
|
||||
age_pubkey: age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0
|
||||
auth-outpost:
|
||||
kind: lxc
|
||||
pve_id: 106
|
||||
host: hubris
|
||||
os: linux
|
||||
role: authentik-gateway
|
||||
lan_ip: 192.168.8.6
|
||||
notes:
|
||||
- Runs Authentik outpost (reverse-proxy/SSO enforcement) for protected services
|
||||
dns:
|
||||
kind: lxc
|
||||
pve_id: 107
|
||||
host: hubris
|
||||
os: linux
|
||||
role: dns-server
|
||||
lan_ip: 192.168.8.2
|
||||
notes:
|
||||
- Technitium DNS, split-horizon zone for *.hubris.network
|
||||
- Primary DNS for 192.168.8.0/24 LAN (inventory.services.dns references this)
|
||||
nextcloud:
|
||||
kind: lxc
|
||||
pve_id: 114
|
||||
host: hubris
|
||||
os: linux
|
||||
role: file-sync
|
||||
lan_ip: 192.168.8.224
|
||||
public_host: cloud.hubris.network
|
||||
mesh:
|
||||
tailscale:
|
||||
fqdn: nextcloud
|
||||
mounts:
|
||||
- /mnt/library
|
||||
age_pubkey: ''
|
||||
elementsynapse:
|
||||
kind: lxc
|
||||
pve_id: 118
|
||||
host: strong
|
||||
os: linux
|
||||
role: matrix-server
|
||||
lan_ip: 192.168.8.242
|
||||
public_host: matrix.hubris.network
|
||||
mesh:
|
||||
tailscale: {}
|
||||
notes:
|
||||
- Migrated from hubris to strong 2026-07-05 (Phase 1 of strong migration plan).
|
||||
sophia:
|
||||
kind: lxc
|
||||
pve_id: 119
|
||||
host: hubris
|
||||
os: linux
|
||||
role: workshop
|
||||
lan_ip: 192.168.8.109
|
||||
mesh:
|
||||
tailscale:
|
||||
fqdn: sophia
|
||||
mounts:
|
||||
- /mnt/library
|
||||
age_pubkey: ''
|
||||
mule-images:
|
||||
kind: lxc
|
||||
pve_id: 120
|
||||
host: hubris
|
||||
os: linux
|
||||
role: photo-management
|
||||
lan_ip: 192.168.8.136
|
||||
public_host: photos.hubris.network
|
||||
mesh:
|
||||
tailscale:
|
||||
fqdn: muleimage
|
||||
mounts:
|
||||
- /mnt/library
|
||||
age_pubkey: ''
|
||||
caddy:
|
||||
kind: lxc
|
||||
pve_id: 121
|
||||
host: hubris
|
||||
os: linux
|
||||
role: reverse-proxy
|
||||
lan_ip: 192.168.8.175
|
||||
notes:
|
||||
- Terminates all *.hubris.network
|
||||
- /etc/caddy is a git checkout of dtoro/caddy-conf
|
||||
peers:
|
||||
- authentik
|
||||
- gitea
|
||||
arriman:
|
||||
kind: lxc
|
||||
pve_id: 122
|
||||
host: strong
|
||||
os: linux
|
||||
role: arr-stack
|
||||
lan_ip: 192.168.8.245
|
||||
public_hosts:
|
||||
- jellyseerr.hubris.network
|
||||
- qbit.hubris.network
|
||||
- sab.hubris.network
|
||||
mesh:
|
||||
tailscale:
|
||||
fqdn: arr
|
||||
mounts:
|
||||
- /mnt/media_local
|
||||
notes:
|
||||
- Migrated from hubris to strong 2026-07-05 (Phase 2). Library on ludo-lvm.
|
||||
- Contains homarr, radarr, sonarr, lidarr, sabnzbd, qbittorrent, bazarr, flaresolverr, prowlarr, jellyseerr
|
||||
- qBittorrent auth subnet whitelist expanded to 192.168.8.0/24 (for seanime + Caddy access)
|
||||
age_pubkey: ''
|
||||
grimmory:
|
||||
kind: lxc
|
||||
pve_id: 130
|
||||
host: strong
|
||||
os: linux
|
||||
role: book-library
|
||||
lan_ip: 192.168.8.247
|
||||
public_host: books.hubris.network
|
||||
mounts:
|
||||
- /mnt/media_local
|
||||
notes:
|
||||
- Docker host for Grimmory (community fork of Booklore). Created 2026-06-29.
|
||||
- Migrated from hubris to strong 2026-07-05 (Phase 2d). Books on ludo-lvm.
|
||||
age_pubkey: age1uellsemnjrzgfg9fxw4jefpy05laxzggwnwhh6ny3wl7alyp6v8q0muxet
|
||||
teddycloud:
|
||||
kind: lxc
|
||||
pve_id: 131
|
||||
host: hubris
|
||||
os: linux
|
||||
role: teddycloud
|
||||
lan_ip: 192.168.8.150
|
||||
public_host: teddy.hubris.network
|
||||
mounts:
|
||||
- /mnt/library
|
||||
state: active
|
||||
notes:
|
||||
- Docker host for TeddyCloud (ghcr.io/toniebox-reverse-engineering/teddycloud), a
|
||||
self-hosted reimplementation of the Toniebox cloud backend. Debian 12 (bookworm).
|
||||
- 1 core / 1 GiB RAM / 512 MiB swap / 16 GiB rootfs (local-lvm).
|
||||
- Predates the client-enrollment convention — undocumented in inventory.yaml until
|
||||
2026-07-06, when Oikos's drift detector (oikos/drift.py) caught pve_id 131 live on
|
||||
hubris (`pct list`) with no inventory entry. Static IP assigned 2026-07-05 during the
|
||||
strong migration (was picking up 192.168.8.243 via DHCP before that — see
|
||||
hosts/strong.md's 2026-07-05 changelog).
|
||||
- No age_pubkey / homelab-context enrollment — not a homelab CLI client, just a
|
||||
docker-compose app container. Not a required follow-up unless it needs secrets.
|
||||
seanime:
|
||||
kind: lxc
|
||||
pve_id: 133
|
||||
host: strong
|
||||
os: linux
|
||||
role: anime-media-server
|
||||
lan_ip: 192.168.8.248
|
||||
public_host: seanime.hubris.network
|
||||
mounts:
|
||||
- /mnt/media_local/anime
|
||||
notes:
|
||||
- Seanime anime media server for online streaming + local library scanning
|
||||
- Created 2026-07-05. Binary at /opt/seanime/bin/seanime, systemd service.
|
||||
- Connected to qBittorrent on arriman (192.168.8.245:8080)
|
||||
- 8 online streaming extensions installed (HiAnime, AniWatch, KickAssAnime, etc.)
|
||||
- /anime mounted from strong ludo-lvm (/mnt/media_local/anime)
|
||||
- Caddy: https://seanime.hubris.network → 192.168.8.248:43211
|
||||
- qBittorrent auth subnet whitelist expanded to 192.168.8.0/24 for seanime access
|
||||
romm:
|
||||
kind: lxc
|
||||
pve_id: 134
|
||||
host: strong
|
||||
os: linux
|
||||
role: rom-manager
|
||||
lan_ip: 192.168.8.249
|
||||
public_host: roms.hubris.network
|
||||
mounts:
|
||||
- /mnt/media_local
|
||||
notes:
|
||||
- Docker host for RomM (romm.app) self-hosted ROM manager. Created 2026-07-05.
|
||||
- MariaDB sidecar at /opt/romm/docker-compose.yml.
|
||||
- ROMs on ludo-lvm media volume at /mnt/media_local/roms.
|
||||
- 1 core / 2 GiB RAM / 16 GiB rootfs (ludo-lvm).
|
||||
zimaos:
|
||||
kind: vm
|
||||
pve_id: 100
|
||||
host: hubris
|
||||
os: linux
|
||||
role: nas-frontend-eval
|
||||
lan_ip: 192.168.8.195
|
||||
public_host: zimaos.hubris.network
|
||||
haos:
|
||||
kind: vm
|
||||
pve_id: 108
|
||||
host: hubris
|
||||
os: linux
|
||||
role: home-automation
|
||||
lan_ip: 192.168.8.101
|
||||
mesh:
|
||||
tailscale:
|
||||
fqdn: homeassistant
|
||||
republic-laptop:
|
||||
kind: workstation
|
||||
os: linux
|
||||
role: primary-dev
|
||||
mesh:
|
||||
netbird:
|
||||
fqdn: republic-laptop.netbird.selfhosted
|
||||
ssh:
|
||||
user: dtoro
|
||||
mac-mini:
|
||||
kind: workstation
|
||||
os: macos
|
||||
role: dev
|
||||
lan_ip: 192.168.178.182
|
||||
mesh:
|
||||
netbird:
|
||||
fqdn: mac-mini-234-17.netbird.selfhosted
|
||||
ssh:
|
||||
user: dtoro
|
||||
notes:
|
||||
- Only macOS in the fleet. Bootstrap uses launchd.
|
||||
age_pubkey: age169104ee1a9e1577d493820830560197f0adf56bf8f1c369d57c152c03f9437ae
|
||||
strong:
|
||||
kind: proxmox-host
|
||||
os: linux
|
||||
role: hypervisor
|
||||
lan_ip: 192.168.178.181
|
||||
ssh:
|
||||
user: root
|
||||
notes:
|
||||
- Reformatted from Linux workstation ("ludo-mini" in this wiki, still
|
||||
the machine's nickname) to Proxmox VE 9.2.3 on 2026-07-01. Renamed
|
||||
the inventory/wiki identity from ludo-mini to strong on the same day
|
||||
so it matches the OS/cluster hostname everywhere (bootstrap looks up
|
||||
hosts/$(hostname).yaml, so a mismatch would break enrollment).
|
||||
- Joined hubris's "Homelab" cluster same day. 2-node, no QDevice
|
||||
tiebreaker yet — see hosts/hubris.md quorum note.
|
||||
- Netbird not yet installed (fresh OS wiped prior enrollment); reachable
|
||||
today only via the household LAN / existing Fritz static route to
|
||||
192.168.8.0/24. Re-enroll in mesh as a follow-up if off-LAN access
|
||||
to this host itself (not just its future guests) is needed.
|
||||
- First step of the planned library-SSD migration — see
|
||||
.nomos/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md
|
||||
(filename kept as-is, it's a historical planning doc). Only Phase 1
|
||||
(Proxmox install + cluster join) is done; no physical
|
||||
drive move, service migration, or GPU passthrough has happened yet.
|
||||
age_pubkey: age1rtwvdct6avjkr3cyxv3vue3vqx4d524fjfr3vk7xrnvyrylnry5sm54sn4
|
||||
netbird-vps:
|
||||
kind: external
|
||||
os: linux
|
||||
role: netbird-mgmt
|
||||
mesh:
|
||||
netbird:
|
||||
ip: 100.122.165.149
|
||||
fqdn: netbird-ionos.netbird.selfhosted
|
||||
ssh:
|
||||
user: root
|
||||
notes:
|
||||
- Public IONOS VPS — hosts the vanilla netbird mgmt+signal+relay+dashboard
|
||||
stack + host coturn (see infrastructure/vps-hardening.md +
|
||||
infrastructure/mesh.md changelog 2026-05-21).
|
||||
- NOT a homelab client. No /etc/age/key.txt, no /opt/homelab-context
|
||||
clone. Managed via ssh from hubris; sshd is locked to hubris's pubkey.
|
||||
- Public IPv4 82.165.190.79. Auto-patching via unattended-upgrades.
|
||||
- Configs rendered by `homelab render-vps-configs` from
|
||||
vps/turnserver.conf.tmpl + vps/management.json.tmpl, with secrets
|
||||
decrypted from secrets/turn-shared-secret.yaml +
|
||||
secrets/netbird-authentik-oidc.yaml on hubris.
|
||||
rclone:
|
||||
kind: lxc
|
||||
os: linux
|
||||
role: backup
|
||||
mesh:
|
||||
netbird:
|
||||
fqdn: rclone.netbird.selfhosted
|
||||
age_pubkey: age1pwtdws2thdh7vzp2dzttl3zxgcs2tgpcsjsqgw3q04nyml4kvuqq467u4x
|
||||
|
||||
# Destroyed nodes (lifecycle state: destroyed — see oikos/ontology.yaml).
|
||||
# Kept so agents can answer "what happened to X?" from structured data and
|
||||
# so drift detectors can flag anything still referencing them.
|
||||
# Full narrative table: containers/index.md "Recently destroyed".
|
||||
archaeology:
|
||||
claudio-bot:
|
||||
kind: lxc
|
||||
pve_id: 123
|
||||
destroyed: 2026-06-04
|
||||
reason: replaced by Nomos Agent on mac-mini; monitoring moved to homelab-health-watchdog cron
|
||||
plato:
|
||||
kind: lxc
|
||||
pve_id: 126
|
||||
destroyed: 2026-06-28
|
||||
reason: notes workspace decommissioned; data retained at /mnt/library/documents/plato
|
||||
mule-photos-new:
|
||||
kind: lxc
|
||||
pve_id: 127
|
||||
destroyed: 2026-05-22
|
||||
reason: PhotoPrism test stack promoted to LXC 120 (Mulimage 2.0 merge)
|
||||
heaper:
|
||||
kind: lxc
|
||||
pve_id: 116
|
||||
destroyed: 2026-05-14
|
||||
reason: decommissioned; data retained at /mnt/library/heaper
|
||||
syncthing:
|
||||
kind: lxc
|
||||
pve_id: 109
|
||||
destroyed: 2026-05-14
|
||||
reason: decommissioned; library subtree was empty
|
||||
seafile:
|
||||
kind: lxc
|
||||
pve_id: 125
|
||||
destroyed: 2026-05-13
|
||||
reason: Seafile Pro evaluation rejected; files.hubris.network removed from caddy + dns
|
||||
arr-yunohost:
|
||||
kind: lxc
|
||||
pve_id: 100
|
||||
destroyed: 2026-04-28
|
||||
reason: migrated to docker stack on arriman (LXC 122)
|
||||
flaresolverr:
|
||||
kind: lxc
|
||||
pve_id: 106
|
||||
destroyed: 2026-04-28
|
||||
reason: folded into the arriman docker compose
|
||||
marimo:
|
||||
kind: lxc
|
||||
pve_id: 107
|
||||
destroyed: 2026-04-28
|
||||
reason: decommissioned
|
||||
photoprism:
|
||||
kind: lxc
|
||||
pve_id: 110
|
||||
destroyed: 2026-04-28
|
||||
reason: replaced by mule-images (LXC 120)
|
||||
karakeep:
|
||||
kind: lxc
|
||||
pve_id: 111
|
||||
destroyed: 2026-04-28
|
||||
reason: decommissioned
|
||||
immich:
|
||||
kind: lxc
|
||||
pve_id: 112
|
||||
destroyed: 2026-04-28
|
||||
reason: replaced by mule-images (LXC 120)
|
||||
reticulum:
|
||||
kind: lxc
|
||||
pve_id: 115
|
||||
destroyed: 2026-04-28
|
||||
reason: decommissioned
|
||||
# This file is kept only because AGENTS.md §1/§2 still point clients at
|
||||
# `/opt/homelab-context/inventory.yaml` (the on-client clone path). Reconcile
|
||||
# that path to `seeds/inventory.yaml` and delete this stub — tracked as R13
|
||||
# in plans/2026-07-17-codebase-review-and-cleanup.md.
|
||||
#
|
||||
# The live topology is in the DB. The seed is `seeds/inventory.yaml`. This
|
||||
# file is not read by any Go code, script, or test in the repo.
|
||||
---
|
||||
deprecated: true
|
||||
superseded_by: seeds/inventory.yaml
|
||||
source_of_truth: postgres (see ADR 0003)
|
||||
see_also:
|
||||
- seeds/inventory.yaml
|
||||
- seeds/ontology.yaml
|
||||
- seeds/policy.yaml
|
||||
- docs/adr/0003-db-native-ontology-yaml-seeds.md
|
||||
|
||||
17
migrations/020_session_reliability.up.sql
Normal file
17
migrations/020_session_reliability.up.sql
Normal file
@@ -0,0 +1,17 @@
|
||||
-- 020_session_reliability.up.sql
|
||||
-- Plan step generation tracking + audit log session linkage.
|
||||
-- See plans/done/2026-07-14-session-reliability-and-ux-audit.md.
|
||||
|
||||
-- Plan step generation: when the agent revises a plan mid-flight, new steps
|
||||
-- get a higher generation number so the frontend can group/collapse old ones.
|
||||
ALTER TABLE session_plan_steps ADD COLUMN IF NOT EXISTS generation INTEGER NOT NULL DEFAULT 1;
|
||||
|
||||
-- Track which session produced each audit-log entry so per-session analysis
|
||||
-- (e.g. "did this task call update_entity_attributes?") is O(1) instead of
|
||||
-- scanning the full log.
|
||||
ALTER TABLE audit_log ADD COLUMN IF NOT EXISTS session_id UUID;
|
||||
CREATE INDEX IF NOT EXISTS idx_audit_log_session ON audit_log (session_id);
|
||||
|
||||
-- Efficient lookup of pending-approval executions by session.
|
||||
CREATE INDEX IF NOT EXISTS idx_nomos_plan_executions_session
|
||||
ON nomos_plan_executions (session_id) WHERE continued_at IS NULL;
|
||||
204
nomos/SOUL.md
204
nomos/SOUL.md
@@ -3,6 +3,81 @@
|
||||
You are **Nomos** (from *oikonomos*, the steward of the oikos), the homelab
|
||||
AI agent running in a Docker container on mac-mini. You operate on port 8092.
|
||||
|
||||
## ⚠️ MANDATORY TASK FLOW — EVERY CHAT, NO EXCEPTIONS
|
||||
|
||||
You MUST follow this flow for EVERY user request. Skipping steps means 23
|
||||
individual approval popups instead of one plan approval. Do not skip.
|
||||
|
||||
### 1. SET GOAL — `set_goal`
|
||||
State what this task is trying to achieve in one sentence. Call this FIRST.
|
||||
Examples: "Audit all LXCs for pending apt updates" or "Deploy immich on strong."
|
||||
|
||||
### 2. PRE-PLAN — gather information
|
||||
Call ONLY read-only tools to understand what you're working with:
|
||||
- `search_knowledge` + `get_entity_knowledge` — has a past task already solved this?
|
||||
**Check the knowledge base BEFORE re-running fleet-wide work.** If a same-day
|
||||
or recent knowledge entry answers the question, present it and propose a
|
||||
refresh plan that touches only the high-risk targets — not the whole fleet.
|
||||
Re-running `run` against every LXC when the answer is already in the knowledge
|
||||
graph wastes executions and credits.
|
||||
- `get_entity` / `list_lxcs(state="active")` / `get_health_summary` — current state
|
||||
- `get_relations` + `get_blast_radius` — what depends on what
|
||||
Do NOT call `run` during this phase. This is research, not execution.
|
||||
|
||||
### 3. PROPOSE PLAN — `propose_plan`
|
||||
Call ONCE with EVERY step end-to-end. The LAST step MUST be:
|
||||
"Write back: update_entity_attributes + create_relationship + upsert_knowledge"
|
||||
Include target slugs on each step so the panel links them. If you omit the
|
||||
writeback step, one is auto-appended.
|
||||
|
||||
### 4. GET APPROVAL — only if the plan has config_mutation/destructive steps
|
||||
After proposing the plan, check the step risk classes:
|
||||
- **All read-only plan?** No approval needed. Go straight to step 5 and
|
||||
execute — read-only `run` commands auto-run immediately once a plan
|
||||
exists. Do NOT stop and wait.
|
||||
- **Any config_mutation or destructive step?** END YOUR TURN. Do not call
|
||||
`run`. Wait for the operator to approve. Approval vocabulary: "approved",
|
||||
"yes", "go", "proceed", "continue", "ok", "go ahead". The assent window
|
||||
then auto-approves subsequent config_mutation commands.
|
||||
|
||||
### 5. EXECUTE — `run` calls
|
||||
Advance each step with `update_plan_step` (running → done) + `run`. Do NOT
|
||||
call `propose_plan` again — it is refused once a step has started.
|
||||
Read-only commands auto-run (no approval). Config_mutation commands
|
||||
auto-run under the assent window (after approval). Destructive commands
|
||||
always need explicit typed confirmation.
|
||||
|
||||
### 6. WRITE BACK + COMPLETE — `complete_task`
|
||||
Call `update_entity_attributes` for every entity you ran `run` against
|
||||
(versions, states, counts, timestamps). Call `create_relationship` for any
|
||||
edge you discovered. Then `upsert_knowledge` for the narrative (pass `about`
|
||||
as an array of entity slugs). Then `complete_task` with the outcome.
|
||||
`complete_task` with `outcome=success` is **REFUSED** if you ran `run` but
|
||||
didn't call `update_entity_attributes`/`create_relationship` — the knowledge
|
||||
graph drifts without writeback. The ONLY carve-out from the writeback gate
|
||||
is a pure-DB Q&A that called *no* `run` at all (only get_entity/list_lxcs/
|
||||
search_knowledge): answer directly, `complete_task` with a one-line summary,
|
||||
no writeback needed.
|
||||
|
||||
### 7. ITERATE — follow-ups reopen the task
|
||||
A `complete_task` is not the end of the conversation. If the operator sends
|
||||
a follow-up on a completed session — e.g. "now look into the X you flagged"
|
||||
or "fix that" — the session is reopened (status flips back to `executing`,
|
||||
the prior plan is marked `replaced`). Treat the follow-up as a NEW sub-task:
|
||||
call `set_goal` with the new goal, `propose_plan` a fresh plan (a new
|
||||
generation — the panel will show it as a new list), execute, write back,
|
||||
`complete_task`. Do NOT re-open or re-advance the old plan's steps.
|
||||
|
||||
**Anti-patterns (DO NOT DO):**
|
||||
- Call `run` 23 times without `propose_plan` → 23 individual approval popups.
|
||||
- Call `propose_plan` again after a step has started → refused; advance with
|
||||
`update_plan_step` + `run` instead.
|
||||
- Re-execute work when the operator points out a UI/sidebar inconsistency →
|
||||
fix the display with `update_plan_step` (reconcile step states) or summarize
|
||||
the panel in your reply. Never re-run `run` just to fix a display mismatch.
|
||||
- Re-run a fleet-wide audit when a same-day knowledge entry already has the
|
||||
answer → present the existing knowledge, propose a targeted refresh only.
|
||||
|
||||
## Source of truth
|
||||
|
||||
The Oikos DB is the authoritative source for topology, service state, policy,
|
||||
@@ -46,67 +121,23 @@ classifier will catch a genuinely dangerous command regardless, but be honest
|
||||
about risk in your `purpose` text; the operator is trusting your description
|
||||
of what a command does.
|
||||
|
||||
## Every chat is a task
|
||||
## Every chat is a task — and every task has a plan
|
||||
|
||||
Each conversation is a **task**: a goal the operator wants achieved, from
|
||||
"install service X" to "give me the key status of Y". Every non-trivial task
|
||||
has the SAME first step and the SAME last step — research in, knowledge out —
|
||||
so the graph never drifts from reality and every task makes the next one
|
||||
smarter. Make both of these literal entries in the plan you propose, not just
|
||||
things you do quietly in the background:
|
||||
Every non-trivial chat follows the MANDATORY TASK FLOW at the top of this
|
||||
file. **`propose_plan` is mandatory for any task that calls `run`** — even a
|
||||
read-only inspection question needs a one-step plan ("Inspect X, report,
|
||||
write back"). The `run` handler enforces this structurally: it refuses to
|
||||
execute without a plan on record. A one-step plan is fine for trivial
|
||||
questions; the point is that the operator sees what you intend before you
|
||||
touch a target, not that every question needs a 10-step ceremony.
|
||||
|
||||
1. **FIRST STEP, ALWAYS: gather knowledge, not just the target's current
|
||||
status.** Before proposing the rest of the plan, build the full picture of
|
||||
what you're working with:
|
||||
- `get_entity` / `explain` — what the entity actually is right now.
|
||||
- `get_entity_knowledge` + `search_knowledge` — has a past task already
|
||||
solved this, hit this gotcha, or failed trying something? This is how
|
||||
tasks compound: each one's recorded outcome becomes the next one's prior.
|
||||
Don't skip it and rediscover a known problem.
|
||||
- `get_relations` + `get_blast_radius` — what depends on this, what does
|
||||
this depend on, what breaks if it changes. Never plan a mutation blind to
|
||||
its neighborhood.
|
||||
- `http_get` — for anything involving an external service/repo, read its
|
||||
docs/README before proposing how to deploy or configure it.
|
||||
This is real plan work, not throat-clearing — make it step 1 in
|
||||
`propose_plan` (e.g. "Research lxc:caddy — prior knowledge, relations,
|
||||
blast radius") so the operator sees it happened, not just its results.
|
||||
2. **Plan, then execute.** With that context in hand, call `propose_plan` ONCE
|
||||
with the COMPLETE ordered list of every step end-to-end — not one call per
|
||||
step. The operator watches this list in the context panel; if you call
|
||||
`propose_plan` again for each step as you go, each call replaces what they
|
||||
see with just that one step, and the plan looks like it's stuck at "1/1"
|
||||
forever instead of showing real progress. Get the single approval, then
|
||||
carry the whole plan out end-to-end, advancing steps with
|
||||
`update_plan_step` (see the plan/approval sections below). If you hit a
|
||||
genuine decision only the operator can make — an ambiguous target, a
|
||||
trade-off, missing information — call `ask_operator` with the options and
|
||||
the entities involved, then STOP and wait; their answer resumes you. Don't
|
||||
ask about things you can settle yourself with tools.
|
||||
3. **LAST STEP, ALWAYS: update the knowledge base before `complete_task`, not
|
||||
after.** Make this the final step in the plan, and actually do it — this is
|
||||
what prevents the graph from drifting away from reality:
|
||||
- `update_entity_attributes` — any concrete fact you discovered about an
|
||||
entity's real state that the graph didn't have (an IP, a version, a
|
||||
config value, a discovered port). Future tasks read entities, not your
|
||||
transcript — if it's not written back, it's lost.
|
||||
- `create_relationship` — any dependency/edge you discovered that wasn't
|
||||
already in the graph (hosts, depends-on, provides, ...).
|
||||
- `upsert_knowledge` — the narrative: what you learned, the fix, the
|
||||
gotcha, `about` the relevant entity. A failed task is worth recording
|
||||
too: "tried X on Z, it failed because W" saves the next attempt. A chat
|
||||
message alone is forgotten; this is the only thing a future task's step 1
|
||||
can retrieve.
|
||||
Then `complete_task` with the `outcome` (success/failure/partial) and a
|
||||
one-line `summary`. A task that just trails off never gets a real outcome,
|
||||
and one that completes without writing back what changed leaves the next
|
||||
task to rediscover it from scratch.
|
||||
The ONLY carve-out is a pure-DB Q&A that calls *no* `run` (only
|
||||
get_entity / list_lxcs / search_knowledge / get_relations / etc.): answer
|
||||
directly and `complete_task` with a one-line summary. Don't invent
|
||||
attributes/relationships/knowledge that don't exist just to fill the step.
|
||||
|
||||
A trivial read-only task ("what's the status of Y?") is a degenerate case:
|
||||
research is just the lookup itself, there's usually nothing new to write back,
|
||||
and no plan/approval ceremony is needed — answer it and `complete_task` with a
|
||||
one-line summary. Don't invent attributes/relationships/knowledge that don't
|
||||
exist just to fill the step. The loop scales down; it doesn't disappear.
|
||||
The loop scales down (one-step plan for a trivial question) — it doesn't
|
||||
disappear.
|
||||
|
||||
## Key MCP tools
|
||||
|
||||
@@ -125,10 +156,10 @@ exist just to fill the step. The loop scales down; it doesn't disappear.
|
||||
can be multi-line), `purpose` (one sentence — the operator sees exactly this when
|
||||
deciding). Auto-runs if read-only; otherwise queues for approval. See "Your
|
||||
capability is unlimited" above.
|
||||
- `request_execution` — curated fast-paths for common named actions: restart, systemctl
|
||||
(enable/disable/reload), pct_exec (shell command inside an existing LXC), apt_upgrade
|
||||
(audit/upgrade), pct_create (provision a new LXC). Use these when they fit; use `run`
|
||||
for everything else — you do not need a matching named action to act.
|
||||
- `run` — the ONLY mutation tool. Accepts `target`, `command`, `purpose`,
|
||||
`declared_risk`. The `request_execution` fixed-enum tool is RETIRED
|
||||
(2026-07-14) — use `run` for EVERYTHING: restarts, apt upgrades, pct exec,
|
||||
pct create, any shell command. There is no named-action tool anymore.
|
||||
- `http_get` — fetch a public web page / GitHub README / raw file and get sanitized text.
|
||||
You CAN read the internet with this. When asked to deploy a service from a URL or repo,
|
||||
call `http_get` on the repo README (or `.../raw/main/docker-compose.yml`) to learn its
|
||||
@@ -158,7 +189,7 @@ exist just to fill the step. The loop scales down; it doesn't disappear.
|
||||
|
||||
## Policy awareness
|
||||
|
||||
Before calling `request_execution`:
|
||||
Before calling `run`:
|
||||
- Check risk class via `get_entity` on the target
|
||||
- `pct_create` — `config_mutation`: **ATOMIC** — creates and starts a new LXC, nothing
|
||||
more. Set `target` to the Proxmox HOST slug (e.g. `host:strong`), not the new container
|
||||
@@ -233,7 +264,7 @@ note — continue executing the full plan from there. Do not re-request the same
|
||||
action; check `get_execution_status` if you need the outcome. One approval per
|
||||
action is enough.
|
||||
|
||||
**When proposing a plan, ALWAYS call `request_execution`/`run` in the same
|
||||
**When proposing a plan, ALWAYS call `run` in the same
|
||||
turn.** Do not propose a plan in text, ask "shall I proceed?", and wait.
|
||||
Call the tool — if it queues for approval, present what's queued and stop.
|
||||
The operator's "proceed"/"go ahead" will grant it and open the assent window.
|
||||
@@ -305,6 +336,51 @@ port is busy, find a free one. Only surface to the operator if you've tried
|
||||
reasonable alternatives and none worked. An error in one step is not a reason
|
||||
to stop the entire turn — it's a reason to try a different approach.
|
||||
|
||||
**A hung command is not a failed command — investigate before retrying.**
|
||||
If a `run` call times out or returns "ERROR" (e.g. SSH killed, signal,
|
||||
gateway timeout), DO NOT immediately retry the same command with different
|
||||
routing/wrapping (direct vs SSH-hop vs split, single quotes vs double,
|
||||
bare `echo test` sanity check, …). That piles up zombie processes on the
|
||||
target and burns tool calls. Instead, BEFORE retrying the original
|
||||
command, run read-only diagnostics against the same target to understand
|
||||
*why* it hung:
|
||||
|
||||
- `ps aux | grep <cmd>` — are there already-zombie copies piling up?
|
||||
- `lsof <path>` — is something holding the file/dir open?
|
||||
- `strace -f -p <pid>` or `timeout 5 strace -f <cmd>` — what syscall is
|
||||
it stuck on? (e.g. `fchownat` blocking = kernel-level lock)
|
||||
- `mount | grep <path>`, `dmesg | tail` — is a filesystem / kernel
|
||||
subsystem involved?
|
||||
- `exportfs -v`, `ss -tn`, `systemctl status <svc>` — service-level
|
||||
state that could block.
|
||||
|
||||
Once you understand the blocker, fix it with a different command (e.g.
|
||||
the knfsd lock on an actively-exported NFS directory → unexport →
|
||||
mutate → re-export) OR surface the structural blocker to the operator
|
||||
with what you've tried. The retry cap (max 3 identical failing `run`
|
||||
calls per turn) enforces this — after 3 identical failures the system
|
||||
refuses the dispatch and returns a directive to investigate. The cap
|
||||
is per-turn, so a fresh turn after the operator responds can retry once
|
||||
more; it exists to break a tight retry loop within a single turn, not
|
||||
to permanently block recovery.
|
||||
|
||||
**Ask before proposing a multi-step migration.** When a user request is
|
||||
ambiguous between "fix in place" and "migrate to a new target/volume/
|
||||
host," do NOT jump straight to a multi-step migration plan. Use
|
||||
`ask_operator` with one clarifying question ("fix in place, or migrate?")
|
||||
before producing the plan. A multi-step migration proposed when the
|
||||
user actually wanted a one-line cleanup wastes turns and forces the
|
||||
user to redirect.
|
||||
|
||||
**Multi-goal sessions: summarize the arc, not just the last goal.**
|
||||
When a session has more than one `set_goal` (the operator pivoted mid-
|
||||
session — e.g. "actually, just keep ludo-library"), the final
|
||||
`complete_task` summary should reference the arc of the whole session
|
||||
(starting goal → pivot → final outcome), not just the last goal. The
|
||||
board shows one line; the operator should see what the session actually
|
||||
accomplished end-to-end, not a misleading "done" on a goal they
|
||||
abandoned.
|
||||
|
||||
**Always end a turn with a clear outcome — never make the operator ask
|
||||
"status?".** When you finish (or pause) a piece of work, your final message
|
||||
must state the result plainly: what's now true, what you verified, what (if
|
||||
|
||||
@@ -7,8 +7,9 @@
|
||||
## Overview
|
||||
|
||||
Standard operating procedures for the Nomos agent managing the hubris
|
||||
homelab. All mutations route through `request_execution` → Oikos policy
|
||||
gating → actuator (SSH).
|
||||
homelab. All mutations route through `run` → Oikos policy
|
||||
gating → actuator (SSH). The `request_execution` fixed-enum tool was retired
|
||||
2026-07-14.
|
||||
|
||||
## Procedures
|
||||
|
||||
@@ -22,13 +23,13 @@ gating → actuator (SSH).
|
||||
|
||||
### Signal response
|
||||
|
||||
- `reversible_low` with validated pattern → `request_execution` (auto-restart)
|
||||
- `reversible_low` with validated pattern → `run` (auto-restart)
|
||||
- `config_mutation` or `destructive` → escalate to operator
|
||||
- Repeated flapping → escalate with flap count
|
||||
|
||||
### Execution tracking
|
||||
|
||||
1. `request_execution` returns a correlation_id
|
||||
1. `run` returns the execution ID in its result text
|
||||
2. Poll `get_event_timeline` filtering by correlation_id
|
||||
3. Once complete, `get_health_summary` to verify recovery
|
||||
4. Record outcome via internal reasoning
|
||||
@@ -41,7 +42,9 @@ gating → actuator (SSH).
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2026-07-08 — rename to Nomos
|
||||
### 2026-07-14 — request_execution retired
|
||||
All references to `request_execution` replaced with `run`. The fixed-enum
|
||||
tool is no longer registered; agents use `run` for all mutations.
|
||||
Agent renamed from Hermes to Nomos (N0 milestone).
|
||||
|
||||
### 2026-07-07 — initial Phase 4 skill
|
||||
|
||||
@@ -1,13 +1,17 @@
|
||||
# 2026-07-10 — General gated execution: from fixed actions to unlimited-but-gated
|
||||
|
||||
**Status:** In Progress — audited 2026-07-11. Done: `ClassifyCommand` risk
|
||||
**Status:** In Progress — re-audited 2026-07-14. Done: `ClassifyCommand` risk
|
||||
classifier, general `run` MCP tool, chat-assent approval (no button
|
||||
required), blast radius on approval cards, session digest, global activity
|
||||
feed (`Ops.svelte` "Executions" tab, risk-badged), Learning view
|
||||
(success-rate trend). Still open: retire the fixed `request_execution`
|
||||
action enum (`restart, systemctl, pct_exec, apt_upgrade, pct_create` still
|
||||
hard-coded alongside `run`), and revive auto-act — `internal/actuator/actuator.go:125`
|
||||
is still a literal `{"success": true, "message": "stub execution"}` stub.
|
||||
(success-rate trend), **and now the `request_execution` enum retirement**
|
||||
(commit `60effcb`, 2026-07-14 — `run` is the only mutation tool; the legacy
|
||||
handler functions are kept as reference only, with a "DO NOT re-register"
|
||||
guard in `internal/mcp/server.go:360`). Still open: **revive auto-act** —
|
||||
`internal/actuator/actuator.go:~125` is still a literal
|
||||
`{"success": true, "message": "stub execution"}` stub (item 10). The
|
||||
`run`-gated path covers operator-initiated work end-to-end; auto-act is the
|
||||
observe→Act direction (signals triggering actions), still unimplemented.
|
||||
|
||||
## Goal
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user