Compare commits
52 Commits
0dd8c28815
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| cd44501fa9 | |||
| 814e020986 | |||
| cb1b6cecc5 | |||
| 92b503e2c2 | |||
| b02f94bfd4 | |||
| 973a6bd92a | |||
| 65f415f9db | |||
| 9e3783734e | |||
| d4f5084a6d | |||
| 23c8144436 | |||
| 64f7d54011 | |||
| d4d99a7473 | |||
| e074f04bdf | |||
| ec119566fd | |||
| 809c16f6fd | |||
|
|
e7cc57a929 | ||
|
|
9ec05e2e3f | ||
| 8fbe39cf2a | |||
| 7160eee1e1 | |||
| 30ecdc16c2 | |||
| d79b0862bd | |||
| 53823595de | |||
| 7ecf720166 | |||
| febc153b7f | |||
| 7d6a3320d4 | |||
| 60bc9d555d | |||
| ebe1b95acf | |||
| 5e10437fe3 | |||
| 7236c46e5c | |||
| 75c0848a6f | |||
| 712b66422b | |||
| a30c024ef8 | |||
| 137a2afb8d | |||
| c8b1ec5af2 | |||
| 8ff382a50d | |||
| fa79c1ea25 | |||
| ef762794e7 | |||
| 0b6c546aae | |||
| cecfd8b0e4 | |||
| 653ea3a116 | |||
| 89a94c24c9 | |||
| 7a34d8c8a0 | |||
| c9d506b0f8 | |||
| e3449b24c1 | |||
| 38c472a118 | |||
| 1d0197da69 | |||
| 0920c4cb6d | |||
| 2254a07baf | |||
| 1b9c761274 | |||
| a126cfa710 | |||
| 86fa57b5cd | |||
| 8e97d589af |
@@ -30,9 +30,9 @@ one pass through **Observe → Orient → Decide → Act**:
|
||||
- **queue**: informational — console + reports
|
||||
The classifier can only *lower* autonomy relative to policy, never raise
|
||||
it. When in doubt, escalate.
|
||||
4. **Act** — execute through `homelab` commands or runbooks (never ad-hoc
|
||||
SSH), then **verify** with the action's verification command, write a
|
||||
**ledger** entry, resolve the Signal, and update docs in the same session.
|
||||
4. **Act** — execute through MCP `run` or runbooks (never ad-hoc
|
||||
SSH), then **verify** with the action's verification command, write a
|
||||
**ledger** entry, resolve the Signal, and update docs in the same session.
|
||||
|
||||
## Primitives
|
||||
|
||||
@@ -88,7 +88,7 @@ via the API's `/api/v1/graph` endpoint, and the Mermaid export at
|
||||
## Conventions carried forward
|
||||
|
||||
- Inventory is the truth; live state wins over narrative docs.
|
||||
- Prefer `homelab` CLI and MCP over ad-hoc SSH.
|
||||
- Prefer MCP tools over ad-hoc SSH.
|
||||
- Meaningful changes update docs in the same session.
|
||||
- Secrets are decrypted locally via per-client keys; never into docs/comments.
|
||||
- Tracked configs change by commit + push, not local edits.
|
||||
@@ -141,7 +141,7 @@ in the Go binary.
|
||||
- Standalone Nomos MCP client binary (`cmd/nomos`) with gateway mode
|
||||
(:8092). Structured queries + natural-language routing to the MCP tool
|
||||
list (see AGENTS.md §3). Agent activity logging on every tool call. No SSH keys.
|
||||
- `nomos/` directory with config, SOUL.md, homelab-ops skill.
|
||||
- `nomos/` directory with config, SOUL.md, `homelab-ops` skill at `nomos/skills/homelab-ops/`.
|
||||
- Nomos Docker service in `docker-compose.yml` (profile: full).
|
||||
- Go packages: `cmd/nomos/`, `compose/nomos/`.
|
||||
|
||||
|
||||
@@ -14,9 +14,9 @@ cmd/webhook/main.go Gitea deploy-webhook receiver (push-to-deploy on mac-
|
||||
internal/httpapi/ REST + MCP server. Chi router. OpenAPI-generated types from
|
||||
internal/httpapi/gen/api.gen.go. Strict server in impl.go.
|
||||
internal/mcp/ MCP tool implementations (get_entity, search_knowledge, etc.)
|
||||
internal/db/ Connection pool (pool.go), seed ingestion (seed.go), DB→YAML
|
||||
internal/adapters/postgres/ Connection pool (pool.go), seed ingestion (seed.go), DB→YAML
|
||||
export (export.go), type hierarchy (typetree.go)
|
||||
internal/db/queries/ SQL query files → sqlc generates internal/db/sqlcgen/
|
||||
internal/adapters/postgres/queries/ SQL query files → sqlc generates sqlcgen/ (same dir)
|
||||
internal/scheduler/ Observe loop: probes, signals, check_defs
|
||||
internal/actuator/ SSH execution with circuit breaker + retry
|
||||
internal/learning/ Pattern extraction, anomaly detection
|
||||
@@ -27,8 +27,6 @@ internal/domain/ Core types: entities, approvals, executions, signals
|
||||
internal/ontology/ Type hierarchy validation, relationship checks
|
||||
internal/knowledge/ Knowledge YAML seed ingestion
|
||||
internal/config/ Config loading from env vars
|
||||
web/ Control-room SPA (Svelte 5) — standalone static build, not
|
||||
embedded in the oikos binary (plans/2026-07-12-wails-desktop-app.md)
|
||||
api/openapi.yaml REST API contract. Source of truth for endpoints.
|
||||
api/codegen.yaml oapi-codegen config → generates internal/httpapi/gen/
|
||||
migrations/ Forward-only SQL. Format: NNN_name.up.sql. No down migrations.
|
||||
@@ -93,8 +91,8 @@ current phase status). To add a new capability:
|
||||
|
||||
## SQL conventions
|
||||
|
||||
- Queries live in `internal/db/queries/*.sql` with `-- name: FuncName :exec`
|
||||
annotations for sqlc. Generated code in `internal/db/sqlcgen/` — never
|
||||
- Queries live in `internal/adapters/postgres/queries/*.sql` with `-- name: FuncName :exec`
|
||||
annotations for sqlc. Generated code in `internal/adapters/postgres/sqlcgen/` — never
|
||||
hand-edit. Call via `sqlcgen.New(pool).QueryName(ctx, params)`.
|
||||
- **sqlc is the default** for all DB access. Raw `pool.Query/Exec` with inline
|
||||
SQL is a documented carve-out for cases sqlc can't express: `LISTEN`/`NOTIFY`,
|
||||
|
||||
@@ -81,8 +81,10 @@ the REST API. Closest current equivalents for what used to live here:
|
||||
| `homelab decide <action> <entity>` | No direct equivalent — classification now happens inline inside `run`, not as a separate dry-run call |
|
||||
|
||||
There is no separately-deployed "Oikos Console" anymore — the control-room
|
||||
SPA (`web/`) is the operator dashboard, served standalone (see
|
||||
[plans/done/2026-07-12-wails-desktop-app.md](../../plans/done/2026-07-12-wails-desktop-app.md)).
|
||||
SPA is the operator dashboard. It lives in its own repo
|
||||
(`dtoro/oikos-web`, local checkout `~/Projects/oikos-web`) with its own
|
||||
deploy pipeline, publishing the same host port 8091 as before (Phase 1 of
|
||||
[plans/2026-08-15-hexagonal-architecture.md](../../plans/2026-08-15-hexagonal-architecture.md)).
|
||||
|
||||
## Related
|
||||
- [Hubris host](../../archive/knowledge/hosts/hubris.md)
|
||||
|
||||
@@ -20,6 +20,6 @@ Run from the repo root:
|
||||
Exit code is non-zero when any violation is found, so it can gate a commit. The banned-vocabulary
|
||||
list mirrors `writing-style.md`; update both together if the standard changes.
|
||||
|
||||
> **Known baseline.** `archive/knowledge/archive/knowledge/containers/101-jellyfin.md` links into a sibling repo
|
||||
> **Known baseline.** `archive/knowledge/containers/101-jellyfin.md` links into a sibling repo
|
||||
> (`devops/homelab-authentik-admin`) that this checkout does not contain — expected, not a bug.
|
||||
> Any other broken link is a real regression; investigate before dismissing it as baseline noise.
|
||||
|
||||
@@ -36,8 +36,9 @@ ledger entry.
|
||||
6. Update the entity's `state` to `destroyed` in `seeds/inventory.yaml`
|
||||
(or move it to an `archaeology:`-style section if the schema still has
|
||||
one) — `pve_id`, `destroyed` date, `reason` — then `oikos seed` to
|
||||
ingest. Add a row to `containers/index.md` "Recently destroyed" table
|
||||
(kept for human-readable browsing alongside the structured data).
|
||||
ingest. Add a row to the legacy `archive/knowledge/containers/index.md`
|
||||
"Recently destroyed" table (kept for human-readable browsing
|
||||
alongside the structured data in the DB).
|
||||
7. No manual ledger step — mutations through the API are recorded
|
||||
automatically in the `audit_log` table (MCP `get_audit_trail`,
|
||||
`get_change_history`). The old `oikos/ledger.py append` was retired
|
||||
|
||||
@@ -27,9 +27,10 @@ chosen, doc page stub.
|
||||
will self-enroll as a client afterward (see
|
||||
[CLIENTS.md](../../../CLIENTS.md#enrollment)), the entity must exist in
|
||||
`planned`/`provisioning` state before `bootstrap.sh` runs there.
|
||||
3. Stub the doc page (`containers/<pve_id>-<name>.md` or
|
||||
`vms/<pve_id>-<name>.md`) — even a one-line "provisioning, see plan X"
|
||||
is enough to satisfy the transition requirement.
|
||||
3. Stub a document entity via MCP `upsert_knowledge` with
|
||||
`kind: document` and about set to the new entity slug — even a
|
||||
one-line "provisioning, see plan X" is enough to satisfy the
|
||||
transition requirement.
|
||||
4. Reserve the IP in DNS/DHCP notes if it's a fixed LAN address.
|
||||
|
||||
Next: [lifecycle-activate-node.md](../lifecycle-activate-node/SKILL.md).
|
||||
|
||||
@@ -77,6 +77,8 @@ Session: {id[:8]} — "{title[:60]}"
|
||||
- `cmd/nomos/store.go` — session + message persistence
|
||||
- `internal/mcp/server.go` — all tool implementations (`run`, `list_lxcs`, …)
|
||||
- `web/src/lib/components/ToolCallGroup.svelte` — tool result display
|
||||
(in the `dtoro/oikos-web` repo, `~/Projects/oikos-web`, since the
|
||||
Phase 1 client extraction)
|
||||
- `nomos/SOUL.md` — agent persona and tool selection rules
|
||||
- `plans/2026-07-09-chat-sessions-improvements.md` — prior session findings
|
||||
- `plans/2026-07-09-session-execution-and-ux-fixes.md` — latest plan
|
||||
- `plans/done/2026-07-09-chat-sessions-improvements.md` — prior session findings
|
||||
- `plans/done/2026-07-09-session-execution-and-ux-fixes.md` — latest plan
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
{
|
||||
"version": "0.0.1",
|
||||
"configurations": [
|
||||
{
|
||||
"name": "web",
|
||||
"runtimeExecutable": "sh",
|
||||
"runtimeArgs": ["-c", "export OIKOS_API_TOKEN=$(docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' oikos-api-1 | sed -n 's/^OIKOS_MCP_BEARER_TOKEN=//p'); exec npm --prefix web run dev"],
|
||||
"port": 5173,
|
||||
"autoPort": true
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -70,30 +70,3 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
- name: docker build (verify image builds; no push)
|
||||
run: docker build -f compose/oikos/Dockerfile -t oikos:ci .
|
||||
|
||||
web:
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: web
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
cache: npm
|
||||
cache-dependency-path: web/package-lock.json
|
||||
- run: npm ci
|
||||
- name: svelte-check (advisory — baseline not yet clean)
|
||||
run: npm run check
|
||||
continue-on-error: true
|
||||
- name: eslint (advisory — baseline not yet clean)
|
||||
run: npm run lint
|
||||
continue-on-error: true
|
||||
- name: prettier format check (advisory — baseline not yet clean)
|
||||
run: npm run format:check
|
||||
continue-on-error: true
|
||||
- name: test
|
||||
run: npm run test
|
||||
- name: build
|
||||
run: npm run build
|
||||
|
||||
@@ -1,70 +0,0 @@
|
||||
name: Desktop App
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
tags:
|
||||
- 'desktop-*'
|
||||
- 'v[0-9]+.[0-9]+.[0-9]*'
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Build Linux (amd64)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
|
||||
- run: npm ci
|
||||
working-directory: web
|
||||
|
||||
- run: npm run build
|
||||
working-directory: web
|
||||
|
||||
- run: |
|
||||
rm -rf cmd/desktop/frontend/dist
|
||||
mkdir -p cmd/desktop/frontend/dist
|
||||
cp -r web/dist/* cmd/desktop/frontend/dist/
|
||||
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: '1.26'
|
||||
|
||||
- run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev
|
||||
|
||||
- run: CGO_ENABLED=1 go build -o build/bin/Oikos .
|
||||
working-directory: cmd/desktop
|
||||
|
||||
- run: |
|
||||
cd cmd/desktop/build/bin
|
||||
tar czf oikos-desktop-linux-amd64.tar.gz Oikos
|
||||
sha256sum oikos-desktop-linux-amd64.tar.gz > oikos-desktop-linux-amd64.tar.gz.sha256
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-linux-amd64
|
||||
path: |
|
||||
cmd/desktop/build/bin/oikos-desktop-linux-amd64.tar.gz
|
||||
cmd/desktop/build/bin/oikos-desktop-linux-amd64.tar.gz.sha256
|
||||
|
||||
release:
|
||||
name: Attach to Release
|
||||
needs: build
|
||||
runs-on: ubuntu-latest
|
||||
if: startsWith(github.ref, 'refs/tags/')
|
||||
steps:
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: oikos-desktop-linux-amd64
|
||||
|
||||
- uses: https://gitea.com/actions/release-action@v1
|
||||
with:
|
||||
files: |
|
||||
oikos-desktop-linux-amd64.tar.gz
|
||||
oikos-desktop-linux-amd64.tar.gz.sha256
|
||||
api_key: ${{ secrets.GITEA_TOKEN }}
|
||||
17
.gitignore
vendored
17
.gitignore
vendored
@@ -12,19 +12,8 @@ backups/
|
||||
.env
|
||||
.infisical-credentials
|
||||
|
||||
# Web UI (Svelte 5) — build artifacts. The SPA is a standalone static build,
|
||||
# deployed separately from the oikos binary (plans/2026-07-12-wails-desktop-app.md
|
||||
# 0.1), so the output dir is just a build artifact.
|
||||
web/dist/
|
||||
web/node_modules/
|
||||
|
||||
# Wails desktop app — frontend copy for embedding
|
||||
cmd/desktop/frontend/dist/
|
||||
cmd/desktop/build/
|
||||
cmd/desktop/Oikos
|
||||
desktop
|
||||
/eval
|
||||
|
||||
# Local tooling artifacts (Playwright MCP session logs, stray screenshots)
|
||||
# Local tooling artifacts (agent worktrees, Playwright MCP logs, screenshots)
|
||||
.claude/
|
||||
.playwright-mcp/
|
||||
config-screen.png
|
||||
|
||||
|
||||
114
.golangci.yml
114
.golangci.yml
@@ -1,41 +1,89 @@
|
||||
# golangci-lint configuration for Oikos
|
||||
# golangci-lint v2 configuration for Oikos
|
||||
# Docs: https://golangci-lint.run/usage/configuration/
|
||||
# Default-enabled linters (errcheck, govet, ineffassign, staticcheck, unused)
|
||||
# are not listed below. gosimple/typecheck were absorbed into staticcheck in v2.
|
||||
version: "2"
|
||||
run:
|
||||
timeout: 5m
|
||||
tests: true
|
||||
|
||||
linters:
|
||||
enable:
|
||||
- govet # go vet
|
||||
- staticcheck # advanced static analysis
|
||||
- ineffassign # detect ineffectual assignments
|
||||
- unused # find unused identifiers
|
||||
- errcheck # check for unchecked errors
|
||||
- gosimple # simplifications
|
||||
- typecheck # standard type checking
|
||||
- misspell # find commonly misspelled English words in comments
|
||||
- revive # fast, configurable linter (replaces golint)
|
||||
|
||||
linters-settings:
|
||||
errcheck:
|
||||
# Allow unchecked errors on common Close/Flush patterns (deferred cleanup)
|
||||
exclude-functions:
|
||||
- (io.Closer).Close
|
||||
- (*os.File).Close
|
||||
|
||||
- depguard
|
||||
- misspell
|
||||
- revive
|
||||
settings:
|
||||
depguard:
|
||||
# ADR 0016 dependency rules. Rules only constrain files that exist:
|
||||
# internal/core is live since Phase 0 (domain moved); internal/nomos
|
||||
# and its bans activate in Phase 8; full audit at Phase 9.
|
||||
rules:
|
||||
core-no-agent-tech:
|
||||
files:
|
||||
- "**/internal/core/**"
|
||||
deny:
|
||||
- pkg: github.com/dtoro/oikos/internal/nomos
|
||||
desc: core never links agent-client packages (ADR 0016 §3.1 rule 3)
|
||||
- pkg: github.com/dtoro/oikos/internal/nomos/**
|
||||
desc: core never links agent-client packages (ADR 0016 §3.1 rule 3)
|
||||
- pkg: github.com/openai/openai-go
|
||||
desc: core never links the LLM SDK — nomos is an external client
|
||||
- pkg: github.com/openai/openai-go/**
|
||||
desc: core never links the LLM SDK — nomos is an external client
|
||||
- pkg: github.com/modelcontextprotocol/go-sdk
|
||||
desc: core never links MCP packages — mcpserver is a driving adapter
|
||||
- pkg: github.com/modelcontextprotocol/go-sdk/**
|
||||
desc: core never links MCP packages — mcpserver is a driving adapter
|
||||
core-purity:
|
||||
files:
|
||||
- "**/internal/core/**"
|
||||
deny:
|
||||
- pkg: github.com/dtoro/oikos/internal/adapters
|
||||
desc: core must not import adapters — depend on core/ports instead
|
||||
- pkg: github.com/dtoro/oikos/internal/adapters/**
|
||||
desc: core must not import adapters — depend on core/ports instead
|
||||
- pkg: github.com/dtoro/oikos/cmd
|
||||
desc: core must not import composition roots
|
||||
- pkg: github.com/dtoro/oikos/cmd/**
|
||||
desc: core must not import composition roots
|
||||
nomos-isolation:
|
||||
files:
|
||||
- "**/internal/nomos/**"
|
||||
deny:
|
||||
- pkg: github.com/dtoro/oikos/internal/core
|
||||
desc: nomos must not import core — consume oikos via MCP/REST
|
||||
- pkg: github.com/dtoro/oikos/internal/core/**
|
||||
desc: nomos must not import core — consume oikos via MCP/REST
|
||||
- pkg: github.com/dtoro/oikos/internal/adapters
|
||||
desc: nomos must not import adapters — consume oikos via MCP/REST
|
||||
- pkg: github.com/dtoro/oikos/internal/adapters/**
|
||||
desc: nomos must not import adapters — consume oikos via MCP/REST
|
||||
errcheck:
|
||||
# Allow unchecked errors on common Close/Flush patterns (deferred cleanup)
|
||||
exclude-functions:
|
||||
- (io.Closer).Close
|
||||
- (*os.File).Close
|
||||
exclusions:
|
||||
generated: lax
|
||||
rules:
|
||||
- linters:
|
||||
- errcheck
|
||||
path: _test\.go
|
||||
- linters:
|
||||
- all
|
||||
path: internal/httpapi/gen/
|
||||
- linters:
|
||||
- all
|
||||
path: internal/adapters/postgres/sqlcgen/
|
||||
paths:
|
||||
- third_party$
|
||||
- builtin$
|
||||
- examples$
|
||||
issues:
|
||||
# Exclude generated code
|
||||
exclude-rules:
|
||||
- path: _test\.go
|
||||
linters:
|
||||
- errcheck
|
||||
- path: internal/httpapi/gen/
|
||||
linters:
|
||||
- all
|
||||
- path: internal/db/sqlcgen/
|
||||
linters:
|
||||
- all
|
||||
# Don't auto-exclude common patterns
|
||||
exclude-use-default: false
|
||||
max-issues-per-linter: 0
|
||||
max-same-issues: 0
|
||||
formatters:
|
||||
exclusions:
|
||||
generated: lax
|
||||
paths:
|
||||
- third_party$
|
||||
- builtin$
|
||||
- examples$
|
||||
|
||||
136
AGENTS.md
136
AGENTS.md
@@ -56,67 +56,88 @@ Endpoint: `https://mcp.hubris.network/mcp`. Every call needs
|
||||
enrollment and `/healthz` (see "Authentication" below for where the token
|
||||
comes from).
|
||||
|
||||
Available tools (the authoritative list — count them below if a number is
|
||||
needed; do not hardcode the count elsewhere):
|
||||
Available tools (67 total — the authoritative list; do not hardcode the count
|
||||
elsewhere; regenerate from `internal/mcp/` when tools change):
|
||||
|
||||
Context — observe + orient:
|
||||
get_entity(slug), list_entities(type, limit, cursor),
|
||||
get_relations(entity), get_blast_radius(entity),
|
||||
search_knowledge(query) — ILIKE search over documents, investigations,
|
||||
runbooks in the knowledge_entities table
|
||||
get_entity_knowledge(entity_slug) — every document, investigation, and
|
||||
runbook linked to one entity, in one call
|
||||
get_patterns(status, entity_type, action) — learned action patterns
|
||||
get_skills(status) — available automation skills
|
||||
http_get(url) — fetch a public page/raw file (e.g. researching how to
|
||||
deploy something before provisioning it); HTTP/HTTPS only, ~16KB cap
|
||||
💡 Slug param alias: all entity-lookup tools now accept `slug` in addition to
|
||||
their declared param name (e.g. `get_entity(slug="lxc:seanime")` works).
|
||||
|
||||
Management — live state:
|
||||
get_service_status(service_slug) — systemctl is-active on target host
|
||||
tail_log(service_slug, lines=200) — journalctl
|
||||
list_lxcs() — all LXC containers with ID, host, IP, health
|
||||
get_lxc_state(lxc_slug) — pct status from Proxmox host
|
||||
ping_service(service_slug) — HTTP reachability from entity_status
|
||||
list_my_secrets(caller_pubkey) — secrets accessible to this client by
|
||||
age public key
|
||||
|
||||
Oikos — decisions:
|
||||
Entity Tools — knowledge graph, discovery, and lifecycle:
|
||||
ping — lightweight connectivity check
|
||||
get_entity(slug_or_id) — get an entity by slug or UUID
|
||||
list_entities(type, state, q, limit) — entities filtered by type, state, or search
|
||||
get_relations(entity_id, types) — list inbound/outbound edges for one entity
|
||||
get_blast_radius(entity_id, depth=3) — entities affected if this one goes down
|
||||
create_entity(type, name, slug, attributes, state) — create a new entity in the graph
|
||||
update_entity_attributes(slug, attributes) — merge discovered facts into an entity
|
||||
set_entity_state(slug, state) — transition entity to a new lifecycle state
|
||||
create_relationship(source, target, type) — record a discovered edge
|
||||
end_relationship(source, target, type) — soft-delete an active edge
|
||||
whoami(hostname) — entity record, peers, and health for a host
|
||||
explain(service_slug) — compact context card (type, state, health, relations)
|
||||
preflight(service_slug, action) — risk class + approval requirement
|
||||
whoami(hostname) — entity record, peers, health for a client
|
||||
get_change_history(entity_slug, limit=20) — last audit-log entries per entity
|
||||
get_state_snapshot() — fleet health, disk, drift count
|
||||
get_state_snapshot() — last scheduler Observe-pass: fleet health, disk, drift
|
||||
audit_knowledge_graph() — read-only drift report over the graph and checks
|
||||
discover_infra_drift() — running guests vs DB: missing/ghost entities
|
||||
find_entities_by(key, value, limit=25) — search entities by attribute values
|
||||
|
||||
Operations — observe + act:
|
||||
get_health_summary() — fleet health counts (healthy/degraded/down/unknown)
|
||||
get_signal_history(entity_slug, state, limit) — open + recent signals
|
||||
get_audit_trail(entity_id) — audit log filter + browse
|
||||
get_agent_activity(limit) — agent self-inspection
|
||||
query_metrics(hours=24) — time-series metric bucketed averages
|
||||
get_trend(entity_id, days=7) — metric slope over time
|
||||
get_event_timeline(severity, entity_slug, limit) — recent events
|
||||
Ops Tools — live state, signals, checks, and execution:
|
||||
run(target, command, purpose, declared_risk) — general execution primitive; read-only auto-acts, mutations queue for approval, destructive always needs explicit confirmation
|
||||
inspect_path(path, targets) — bulk mount/df/ls/stat across multiple hosts/LXCs
|
||||
get_execution_status(execution_id) — poll execution progress
|
||||
tail_log(service_slug, lines=50) — journalctl for a service
|
||||
get_service_status(service_slug) — systemctl is-active/is-enabled
|
||||
get_lxc_state(lxc_slug) — pct status from Proxmox host
|
||||
ping_service(service_slug) — HTTP reachability + scheduler health state
|
||||
list_lxcs(state) — all LXC containers with ID, host, IP, last-audited hint
|
||||
restart_service(target, service) — restart a systemd service (config_mutation, requires approval)
|
||||
docker_exec(lxc_slug, container, command, purpose) — run a command inside a Docker container on an LXC; handles escaping, resolves target from entity graph; read-only auto-runs, mutations require approval
|
||||
push_file(target, source_path, dest_path, backup=true) — push a file into an LXC from the Proxmox host (config_mutation, requires approval)
|
||||
ack_signal(signal_id) — acknowledge an open signal
|
||||
resolve_signal(signal_id, resolution) — resolve a signal with optional note
|
||||
mute_signal(signal_id, duration_s=3600) — temporarily mute a signal
|
||||
cancel_execution(execution_id, reason) — cancel a queued/running execution
|
||||
update_check(check_id, enabled) — enable or disable a health check
|
||||
list_checks(entity_slug, enabled) — list health checks with verdict, probe kind
|
||||
list_executions(entity_slug, status, limit=25) — cursor-paginated execution history
|
||||
list_entity_sessions(entity_slug) — active Nomos sessions linked to an entity
|
||||
get_dashboard_summary() — fleet overview: counts, health, signals, approvals
|
||||
list_approvals(status, entity_slug, limit) — list pending/recent approvals; filter by status (pending, approved, denied) or entity
|
||||
decide_approval(approval_id, decision) — approve or deny a pending execution; calls the same API endpoint as the Approve button in the UI
|
||||
get_secret(key, path, environment) — retrieve a secret from the Infisical vault
|
||||
list_secrets(path_prefix) — list secret keys in the Infisical vault
|
||||
set_secret(key, value, path, environment) — store/update a secret (requires approval)
|
||||
|
||||
Knowledge — keep the graph current (none require approval; this updates
|
||||
the knowledge graph, not live infrastructure):
|
||||
upsert_knowledge(title, content) — record what you learned after solving
|
||||
a non-obvious problem; the only way anything persists past a session
|
||||
update_entity_attributes(slug, attributes) — merge a discovered fact
|
||||
(IP, version, port, ...) into an entity so a future task doesn't
|
||||
rediscover it from scratch
|
||||
create_relationship(source, target, type) — record a discovered edge
|
||||
(depends-on, hosts, routes-to, ...) between two entities
|
||||
Knowledge Tools — search, read, and maintain the knowledge base:
|
||||
search_knowledge(query) — full-text search across docs (snippets, not full body)
|
||||
get_entity_knowledge(entity_slug) — all docs/investigations/runbooks linked to a slug
|
||||
get_knowledge_content(slug) — full markdown body of one knowledge entry
|
||||
upsert_knowledge(title, content, about, tags, kind) — write what you learned
|
||||
upsert_session_summary(session_id, summary, entities_touched, discoveries) — batch-write session findings into the graph; creates knowledge entries, links entities, records a session-audit entry
|
||||
delete_knowledge(knowledge_slug) — soft-delete a knowledge entry
|
||||
restore_knowledge(knowledge_slug) — restore a soft-deleted entry
|
||||
merge_knowledge(target_slug, source_slugs) — fold entries into a target
|
||||
rename_knowledge_tag(from, to) — bulk-rename tags across all entries
|
||||
get_knowledge_revisions(knowledge_slug) — version history for a knowledge entry
|
||||
get_knowledge_duplicates(threshold=0.6) — near-duplicate detection via trigram similarity
|
||||
get_knowledge_orphans(stale_days=90) — unlinked, untagged, or stale entries
|
||||
list_knowledge_tags() — all tags with usage counts and casing variants
|
||||
list_my_secrets(caller_pubkey) — secrets accessible to a client by age public key
|
||||
|
||||
Execution — mutating the live infrastructure:
|
||||
run(target, command) — the general execution primitive. Run any shell
|
||||
command against a host or LXC; every command is auto-classified —
|
||||
read-only inspection runs immediately, anything state-changing needs
|
||||
operator approval, and destructive patterns (rm -rf, dd, mkfs,
|
||||
pct/qm destroy, DROP TABLE, reboot, curl-pipe-to-shell, ...) always
|
||||
need approval regardless of what you declare. This is the ONLY
|
||||
mutation tool — `request_execution` was retired 2026-07-14; the
|
||||
former enum actions (restart, systemctl, pct_exec, apt_upgrade,
|
||||
pct_create) are all expressed as `run(target, command)` now.
|
||||
get_execution_status(execution_id) — poll progress
|
||||
Analysis Tools — fleet health, metrics, and introspection:
|
||||
get_health_summary(health) — fleet health per entity, optionally filtered
|
||||
get_audit_trail(entity_id) — query the audit log
|
||||
query_metrics(hours=24) — time-series with bucketed avg/min/max
|
||||
get_signal_history(entity_slug, state, limit=50) — open and recent signals
|
||||
get_patterns(status, entity_type, action) — learned action patterns
|
||||
get_skills(status) — available automation skills
|
||||
get_trend(entity_id, days=7) — metric slope, variance, and averages
|
||||
get_event_timeline(severity, entity_slug, limit=50) — recent events
|
||||
get_agent_activity(limit=50) — agent self-inspection log
|
||||
classify_command(command, declared_risk) — pre-flight risk classification before `run`
|
||||
get_ontology() — entity types, relationship types, and lifecycle definitions
|
||||
http_get(url) — fetch a public web page/raw file; ~16KB cap
|
||||
|
||||
**When to prefer MCP over grepping the clone:** always for knowledge queries.
|
||||
`search_knowledge("jellyfin hardware acceleration")` returns ranked results from
|
||||
@@ -133,7 +154,9 @@ API/MCP bearer token — there is one shared
|
||||
secret (`OIKOS_MCP_BEARER_TOKEN`, validated in `internal/httpapi/server.go`'s
|
||||
`combinedAuth`); get it from the operator until per-client token issuance
|
||||
exists. The SPA has its own flow instead: a first-launch Config screen that
|
||||
stores a token in `localStorage` (see `web/src/pages/Config.svelte`).
|
||||
stores a token in `localStorage` (the SPA lives in the `dtoro/oikos-web`
|
||||
repo — `web/src/pages/Config.svelte` there — since the Phase 1 client
|
||||
extraction).
|
||||
|
||||
## 5. Knowledge conventions
|
||||
|
||||
@@ -168,13 +191,14 @@ The DB is the truth. The old wiki files are archived at `archive/knowledge/`
|
||||
- **Actions** (restart, logs, apt, pct exec, or anything else): Nomos calls
|
||||
`run` (the general execution primitive) via MCP. `reversible_low`/read-only actions execute
|
||||
immediately; `config_mutation` and `destructive` actions are queued for
|
||||
operator approval via Matrix or the control-room UI's Operations page.
|
||||
operator approval via the App button in the control-room UI or via
|
||||
`list_approvals`/`decide_approval` MCP tools from any agent.
|
||||
- **Secrets**: managed by Infisical (`oikos secret` subcommand for
|
||||
migration). Never hardcode secrets — use env vars from `.env`.
|
||||
- **Mutations** (restart, edit configs, etc.): classified against
|
||||
`seeds/policy.yaml`. `reversible_low` actions auto-execute;
|
||||
`config_mutation`/`destructive` actions require approval — granted by
|
||||
the operator via Matrix reply or the control-room UI, not a CLI flag.
|
||||
the operator via the App button or `decide_approval` MCP tool call.
|
||||
See OIKOS.md.
|
||||
|
||||
## 7. Communication mode
|
||||
|
||||
@@ -23,7 +23,7 @@ Docker stack on mac-mini and exposes an MCP server + REST API.
|
||||
| Change history | MCP `get_change_history` |
|
||||
| State snapshot (health, disk, drift) | MCP `get_state_snapshot` |
|
||||
| Secrets (Infisical) | REST API + `oikos secret` CLI |
|
||||
| Approval tokens | Matrix via notifier |
|
||||
| Approve/deny pending executions | MCP `list_approvals`, `decide_approval` (chat-native, works on any platform) |
|
||||
| Run a command on a host/LXC (policy-gated) | MCP `run` |
|
||||
| Record a discovered fact/relationship | MCP `update_entity_attributes`, `create_relationship`, `upsert_knowledge` |
|
||||
|
||||
|
||||
@@ -9,8 +9,9 @@ repo, see [.agents/dev/CONTRIBUTING.md](.agents/dev/CONTRIBUTING.md).
|
||||
- **Go 1.26+** (see `go.mod` for pinned version)
|
||||
- **PostgreSQL with TimescaleDB** — the compose stack includes `timescale/timescaledb:2.17.2-pg16`
|
||||
- **Docker** for the full dev stack
|
||||
- **Node 22+** for `web/` (the control-room SPA — standalone, not part of the
|
||||
compose stack or the `oikos` binary)
|
||||
- The control-room SPA and desktop app live in their own repo —
|
||||
[dtoro/oikos-web](https://git.hubris.network/dtoro/oikos-web) (Node 22+
|
||||
there); this repo is backend-only since the hexagonal refactor Phase 1
|
||||
|
||||
```bash
|
||||
# Start dependencies (Postgres + Redis). api/nomos require a shared bearer
|
||||
@@ -26,62 +27,52 @@ make test-db
|
||||
# Build the binary
|
||||
make build
|
||||
|
||||
# SPA dev server (proxies to api/nomos, injecting the same token)
|
||||
cd web && OIKOS_API_TOKEN=dev-token npm run dev
|
||||
# SPA dev server (own repo — dtoro/oikos-web)
|
||||
cd ~/Projects/oikos-web/web && OIKOS_API_TOKEN=dev-token npm run dev
|
||||
|
||||
# Desktop app (macOS)
|
||||
make desktop # build .app bundle
|
||||
make install # build + install to /Applications
|
||||
./cmd/desktop/build/bin/oikos-desktop.app/Contents/MacOS/oikos-desktop # run from terminal to see logs
|
||||
# Desktop app (macOS — also in the oikos-web repo)
|
||||
cd ~/Projects/oikos-web && make install
|
||||
```
|
||||
|
||||
### Desktop app auth
|
||||
### Desktop app
|
||||
|
||||
The desktop app uses the same API as the browser SPA. First launch:
|
||||
|
||||
1. Enter `https://oikos.hubris.network` as Server URL
|
||||
2. **Login with Authentik** tab → opens system browser → authenticate
|
||||
3. Callback page shows token → copy → paste into Token tab → Connect
|
||||
4. Token is persisted to the macOS keychain — subsequent launches skip setup
|
||||
|
||||
The app stores credentials via `github.com/zalando/go-keyring` (service: `com.hubris.oikos-desktop`).
|
||||
|
||||
### Desktop app auto-update
|
||||
|
||||
- Checks Gitea releases every 6 hours
|
||||
- System tray → **Check for Updates** triggers an immediate check
|
||||
- Download, extract, replace the app in `/Applications`, and relaunch
|
||||
- Versions are compared against the `version` var in `main.go`, injected from the repo `VERSION` file at link time (`make desktop` passes `-ldflags "-X main.version=$(cat VERSION)"`)
|
||||
The desktop app's auth (Authentik login → keychain-persisted token) and
|
||||
auto-update (Gitea releases every 6 hours) moved with it to
|
||||
[dtoro/oikos-web](https://git.hubris.network/dtoro/oikos-web) — see that
|
||||
repo's README. Auto-update now tracks oikos-web releases; builds installed
|
||||
before the split need one manual reinstall.
|
||||
|
||||
## Project structure
|
||||
|
||||
```
|
||||
cmd/desktop/ Wails v3 desktop app (macOS + Linux)
|
||||
main.go Thin shell: webview, system tray, notifications, auto-update
|
||||
wails.json Wails project config
|
||||
entitlements.plist macOS code-signing entitlements
|
||||
icon.png System tray icon (embedded)
|
||||
icon.icns App bundle icon (white logo on black rounded rect)
|
||||
Taskfile.yml Wails v3 build tasks
|
||||
Info.plist.template macOS bundle metadata
|
||||
cmd/oikos/ Single-binary entry point
|
||||
cmd/nomos/ Nomos MCP client gateway
|
||||
cmd/webhook/ Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
|
||||
internal/ All Go packages
|
||||
httpapi/ REST + MCP server (OpenAPI-generated)
|
||||
mcp/ MCP tool implementations
|
||||
db/ Connection pool, migrations, seeds, sqlc queries
|
||||
scheduler/ Observe loop, probes, signals
|
||||
actuator/ SSH execution
|
||||
learning/ Pattern recognition, anomaly detection
|
||||
notifier/ Matrix notifications, approval tokens
|
||||
policy/ Risk classifier
|
||||
secrets/ Infisical + SOPS backend
|
||||
domain/ Core types: entities, approvals, signals, patterns
|
||||
ontology/ Type hierarchy, relationship validation
|
||||
core/ Hexagon core ([ADR 0016](docs/adr/0016-hexagonal-ports-adapters.md)):
|
||||
domain/ (pure model, stdlib only), ports/ (driven-port
|
||||
interfaces), app/ (use-case services — EntityService,
|
||||
RelationshipService, SignalService, ObservationService,
|
||||
KnowledgeService, LearningService, etc.) — populated by
|
||||
the phased refactor; core may not import adapters,
|
||||
enforced by depguard (verified Phase 9 audit)
|
||||
adapters/ Ports' implementations: postgres/ (pool, migrations, seeds,
|
||||
sqlcgen, repos — EntityRepo, RelRepo, OntologyRepo,
|
||||
EntityReader, SignalRepo, MetricsRepo, etc.),
|
||||
ssh/ (CommandExecutor via actuator),
|
||||
remote/ (TargetResolver via internal/remote),
|
||||
probes/ (Checker implementations per probe kind:
|
||||
HTTP, TCP, ping, DNS, SSH)
|
||||
httpapi/ REST server (OpenAPI-generated) — driving adapter
|
||||
mcp/ MCP tool implementations — driving adapter
|
||||
scheduler/ Observe loop, coverage sweep — driving adapter (moves
|
||||
to ObservationService/SignalService in Phase 5)
|
||||
execworker/ Execution worker — driving adapter
|
||||
actuator/ SSH execution — consumed by adapters/ssh
|
||||
remote/ Target resolution — consumed by adapters/remote
|
||||
audit/ Audit report helpers
|
||||
observability/ Event/Audit recorder helpers
|
||||
knowledge/ Knowledge YAML seed ingestion
|
||||
web/ Control-room SPA (Svelte 5) — standalone, not embedded
|
||||
in the oikos binary; see plans/2026-07-12-wails-desktop-app.md
|
||||
api/openapi.yaml API contract — the source of truth for endpoints
|
||||
migrations/ Forward-only SQL migrations (TimescaleDB)
|
||||
seeds/ Bootstrap YAML: ontology, inventory, policy, knowledge
|
||||
@@ -111,11 +102,6 @@ docs/operations/ Runbooks (rollback, etc.)
|
||||
| `make export` | Export DB state to YAML seeds |
|
||||
| `make dev` | Start compose dev stack |
|
||||
| `make clean` | Remove binary + test cache |
|
||||
| `make ui` | Build the SPA (`web/dist/`) |
|
||||
| `make deploy-ui` | Build + deploy the SPA to the Caddy host |
|
||||
| `make desktop` | Build the Wails desktop app for the current platform |
|
||||
| `make desktop-package` | Build + package (zip on macOS, tar.gz on Linux) |
|
||||
| `make install` | Build + install to `/Applications` (macOS) |
|
||||
| `make webhook` | Build `cmd/webhook` (deploy-webhook receiver) |
|
||||
| `make tidy` | `go mod tidy` |
|
||||
|
||||
@@ -135,8 +121,8 @@ Never hand-edit `internal/httpapi/gen/api.gen.go`.
|
||||
|
||||
### Database access is sqlc-first
|
||||
|
||||
SQL queries live in `internal/db/queries/*.sql`. Go code is generated with
|
||||
`sqlc` into `internal/db/sqlcgen/`. Config in `sqlc.yaml`.
|
||||
SQL queries live in `internal/adapters/postgres/queries/*.sql`. Go code is generated with
|
||||
`sqlc` into `internal/adapters/postgres/sqlcgen/`. Config in `sqlc.yaml`.
|
||||
|
||||
- Queries target pgx/v5 with UUID + timestamptz overrides
|
||||
- Never hand-edit generated sqlc code
|
||||
|
||||
42
Makefile
42
Makefile
@@ -1,4 +1,4 @@
|
||||
.PHONY: build webhook test test-db lint generate generate-check dev migrate seed export clean tidy ui desktop desktop-package install
|
||||
.PHONY: build webhook test test-db lint generate generate-check dev migrate seed export clean tidy
|
||||
|
||||
BINARY := bin/oikos
|
||||
GO ?= go
|
||||
@@ -18,7 +18,7 @@ test-db:
|
||||
docker compose up -d postgres
|
||||
@sleep 3
|
||||
OIKOS_TEST_DATABASE_URL="postgres://oikos:$${OIKOS_DB_PASSWORD:-oikos_dev}@localhost:5432/oikos?sslmode=disable" \
|
||||
$(GO) test -race -count=1 ./internal/db/ ./internal/httpapi/ ./internal/mcp/
|
||||
$(GO) test -race -count=1 ./internal/adapters/postgres/ ./internal/httpapi/ ./internal/mcp/
|
||||
|
||||
lint: vet golangci govulncheck
|
||||
|
||||
@@ -40,7 +40,7 @@ generate:
|
||||
|
||||
# CI drift guard: regenerate and fail if the committed output changed.
|
||||
generate-check: generate
|
||||
@git diff --exit-code -- internal/httpapi/gen internal/db/sqlcgen \
|
||||
@git diff --exit-code -- internal/httpapi/gen internal/adapters/postgres/sqlcgen \
|
||||
|| (echo "generated code is stale — run 'make generate' and commit" && exit 1)
|
||||
|
||||
migrate:
|
||||
@@ -55,45 +55,9 @@ export:
|
||||
dev:
|
||||
docker compose --profile dev up -d
|
||||
|
||||
# Local sanity-check build of the SPA. Not embedded in the oikos binary
|
||||
# (plans/2026-07-12-wails-desktop-app.md 0.1) — deploys as its own
|
||||
# container (compose/web/Dockerfile) via `docker compose --profile full
|
||||
# up -d web`, same push-to-main pipeline as everything else.
|
||||
ui:
|
||||
cd web && npm run build
|
||||
|
||||
desktop: ui ## Build the Wails desktop app for the current platform
|
||||
rm -rf cmd/desktop/frontend/dist
|
||||
mkdir -p cmd/desktop/frontend/dist
|
||||
cp -r web/dist/* cmd/desktop/frontend/dist/
|
||||
cd cmd/desktop && CGO_ENABLED=1 go build -ldflags "-X main.version=$$(cat ../VERSION)" -o build/bin/Oikos .
|
||||
|
||||
desktop-package: desktop ## Build + package the desktop app (zip on macOS, tar.gz on Linux)
|
||||
@case $$(uname -s) in \
|
||||
Darwin) \
|
||||
APP="cmd/desktop/build/bin/Oikos.app"; \
|
||||
rm -rf "$$APP"; \
|
||||
mkdir -p "$$APP/Contents/MacOS"; \
|
||||
mkdir -p "$$APP/Contents/Resources"; \
|
||||
cp cmd/desktop/build/bin/Oikos "$$APP/Contents/MacOS/Oikos"; \
|
||||
cp cmd/desktop/icon.icns "$$APP/Contents/Resources/icon.icns"; \
|
||||
sed "s/\$$(VERSION)/$$(cat VERSION)/" cmd/desktop/Info.plist.template > "$$APP/Contents/Info.plist"; \
|
||||
cd cmd/desktop/build/bin && zip -r oikos-desktop-darwin-$$(uname -m).zip Oikos.app ;; \
|
||||
Linux) \
|
||||
cd cmd/desktop/build/bin && tar czf oikos-desktop-linux-$$(uname -m).tar.gz Oikos ;; \
|
||||
esac
|
||||
@echo "Package: cmd/desktop/build/bin/"
|
||||
|
||||
install: desktop-package ## Install to /Applications
|
||||
rm -rf /Applications/Oikos.app
|
||||
cp -r cmd/desktop/build/bin/Oikos.app /Applications/
|
||||
@echo "Installed to /Applications/Oikos.app"
|
||||
|
||||
clean:
|
||||
rm -f $(BINARY)
|
||||
rm -rf bin
|
||||
rm -rf cmd/desktop/build
|
||||
rm -rf cmd/desktop/frontend/dist
|
||||
$(GO) clean -testcache
|
||||
|
||||
tidy:
|
||||
|
||||
34
README.md
34
README.md
@@ -13,7 +13,7 @@ learns from outcomes, and escalates when uncertain.
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# Dev stack (postgres + api + scheduler + notifier). The api/nomos
|
||||
# Dev stack (postgres + api + scheduler). The api/nomos
|
||||
# services need a shared token — every route requires a real bearer
|
||||
# credential, there's no dev-open bypass.
|
||||
OIKOS_MCP_BEARER_TOKEN=dev-token docker compose --profile dev up -d
|
||||
@@ -29,8 +29,8 @@ OIKOS_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disa
|
||||
OIKOS_API_TOKEN=dev-token \
|
||||
go run ./cmd/oikos all
|
||||
|
||||
# Control-room SPA (separate from the Go binary — see web/)
|
||||
cd web && OIKOS_API_TOKEN=dev-token npm run dev # http://localhost:5173
|
||||
# Control-room SPA + desktop app: own repo — dtoro/oikos-web
|
||||
# (~/Projects/oikos-web; cd web && OIKOS_API_TOKEN=dev-token npm run dev)
|
||||
```
|
||||
|
||||
## Architecture
|
||||
@@ -42,8 +42,8 @@ cd web && OIKOS_API_TOKEN=dev-token npm run dev # http://localhost:5173
|
||||
Workstation ─── │ nomos (8092) ──MCP── api (8090) │
|
||||
(mesh) │ MCP gateway REST + MCP │
|
||||
│ │
|
||||
│ scheduler ── notifier ── postgres │
|
||||
│ (observe) (Matrix) (Timescale)│
|
||||
│ scheduler ─── postgres │
|
||||
│ (observe) (Timescale) │
|
||||
└──────────────────────────────────┘
|
||||
```
|
||||
|
||||
@@ -51,7 +51,6 @@ cd web && OIKOS_API_TOKEN=dev-token npm run dev # http://localhost:5173
|
||||
|-----------|------|------|
|
||||
| `oikos api` | 8090 | REST API + MCP server (tool list in [AGENTS.md §3](AGENTS.md#3-the-mcp-server)) |
|
||||
| `oikos scheduler` | — | Probe runner, signal lifecycle, metrics |
|
||||
| `oikos notifier` | — | Approval tokens, Matrix alerts |
|
||||
| `nomos serve` | 8092 | MCP client gateway, query routing |
|
||||
|
||||
## Phases
|
||||
@@ -60,7 +59,7 @@ cd web && OIKOS_API_TOKEN=dev-token npm run dev # http://localhost:5173
|
||||
|-------|--------|-------------|
|
||||
| 1 — Ontology + DB | ✅ | TimescaleDB, migrations, seeds, blast_radius |
|
||||
| 2 — API | ✅ | OpenAPI-first REST + MCP, auth, SSE, audit |
|
||||
| 3 — Control loop | ✅ | Scheduler, actuator, learning, classifier, notifier |
|
||||
| 3 — Control loop | ✅ | Scheduler, actuator, learning, classifier |
|
||||
| 4 — Nomos agent | ✅ | Standalone MCP client gateway, agent activity |
|
||||
| 5 — Secrets | ✅ | Infisical backend + SOPS fallback, rotation runbooks |
|
||||
| 6 — Deploy | ✅ | CI pipeline, cutover checklist, watchdog, rollback |
|
||||
@@ -100,7 +99,6 @@ oikos seed # ingest ontology/inventory/policy seeds
|
||||
oikos export # export DB state to YAML
|
||||
oikos api # serve REST + MCP
|
||||
oikos scheduler # run observe loop
|
||||
oikos notifier # run notification loop
|
||||
oikos all # all roles in one process
|
||||
oikos secret list # enumerate SOPS secrets
|
||||
oikos secret migrate # SOPS → Infisical
|
||||
@@ -108,12 +106,14 @@ oikos secret migrate # SOPS → Infisical
|
||||
|
||||
### Web UI
|
||||
|
||||
`web/` is a standalone Svelte 5 SPA — not embedded in the `oikos` binary, not
|
||||
part of `docker-compose.yml`. It talks to `api`/`nomos` over HTTP with a
|
||||
bearer token entered on first launch (see `web/src/pages/Config.svelte`).
|
||||
Build with `make ui`, deploy with `make deploy-ui` (Caddy serves the static
|
||||
output). A native desktop wrapper exists at `cmd/desktop/` — see
|
||||
[plans/done/2026-07-12-wails-desktop-app.md](plans/done/2026-07-12-wails-desktop-app.md).
|
||||
The control-room SPA and the Wails desktop wrapper live in their own repo,
|
||||
[dtoro/oikos-web](https://git.hubris.network/dtoro/oikos-web) (local
|
||||
checkout `~/Projects/oikos-web`) — extracted in Phase 1 of
|
||||
[plans/2026-08-15-hexagonal-architecture.md](plans/2026-08-15-hexagonal-architecture.md).
|
||||
The SPA talks to `api`/`nomos` over HTTP with a bearer token entered on
|
||||
first launch. It deploys as its own compose project publishing `8091:80`;
|
||||
the outer Caddy (LXC 121) targets that published port, so serving and auth
|
||||
are unchanged from the pre-split stack.
|
||||
|
||||
## Repo layout
|
||||
|
||||
@@ -121,11 +121,9 @@ output). A native desktop wrapper exists at `cmd/desktop/` — see
|
||||
cmd/oikos/ Go entry point — single binary
|
||||
cmd/nomos/ Nomos MCP client gateway
|
||||
cmd/webhook/ Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
|
||||
cmd/desktop/ Wails desktop wrapper around the SPA
|
||||
internal/ Go packages (actuator, checkdefaults, config, db, domain,
|
||||
httpapi, knowledge, learning, mcp, notifier, observability,
|
||||
internal/ Go packages (actuator, checkdefaults, config, core, db,
|
||||
domain, httpapi, knowledge, learning, mcp, observability,
|
||||
ontology, policy, safego, scheduler, secrets)
|
||||
web/ Control-room SPA (Svelte 5) — standalone, not embedded
|
||||
api/openapi.yaml API contract (OpenAPI 3.1)
|
||||
migrations/ Forward-only SQL migrations (TimescaleDB)
|
||||
seeds/ Bootstrap YAML (ontology, inventory, policy, knowledge)
|
||||
|
||||
@@ -3185,8 +3185,6 @@ components:
|
||||
required:
|
||||
- age_public_key
|
||||
- age_private_key
|
||||
- infisical_client_id
|
||||
- infisical_client_secret
|
||||
properties:
|
||||
age_public_key:
|
||||
type: string
|
||||
@@ -3200,9 +3198,6 @@ components:
|
||||
infisical_client_secret:
|
||||
type: string
|
||||
description: Infisical UniversalAuth client secret
|
||||
machine_identity_token:
|
||||
type: string
|
||||
description: Infisical machine identity access token
|
||||
ClientContext:
|
||||
type: object
|
||||
required:
|
||||
|
||||
12
checks/pvecm_quorum_check.sh
Normal file
12
checks/pvecm_quorum_check.sh
Normal file
@@ -0,0 +1,12 @@
|
||||
#!/usr/bin/env bash
|
||||
# pvecm_quorum_check.sh — Proxmox cluster quorum status.
|
||||
# Runs on a PVE host. Fails if the node is not quorate.
|
||||
set -euo pipefail
|
||||
|
||||
# pvecm status exit code is non-zero on non-quorate nodes
|
||||
# (e.g. "Quorate: No — Activity blocked")
|
||||
if pvecm status 2>/dev/null | grep -q 'Quorate.*Yes'; then
|
||||
echo '{"health":"healthy","metrics":{"quorate":1}}'
|
||||
else
|
||||
echo '{"health":"unhealthy","metrics":{"quorate":0}}'
|
||||
fi
|
||||
@@ -1,30 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>CFBundleDevelopmentRegion</key>
|
||||
<string>en</string>
|
||||
<key>CFBundleExecutable</key>
|
||||
<string>Oikos</string>
|
||||
<key>CFBundleIdentifier</key>
|
||||
<string>com.hubris.oikos-desktop</string>
|
||||
<key>CFBundleIconFile</key>
|
||||
<string>icon</string>
|
||||
<key>CFBundleInfoDictionaryVersion</key>
|
||||
<string>6.0</string>
|
||||
<key>CFBundleName</key>
|
||||
<string>Oikos</string>
|
||||
<key>CFBundlePackageType</key>
|
||||
<string>APPL</string>
|
||||
<key>CFBundleShortVersionString</key>
|
||||
<string>$(VERSION)</string>
|
||||
<key>CFBundleVersion</key>
|
||||
<string>$(VERSION)</string>
|
||||
<key>LSMinimumSystemVersion</key>
|
||||
<string>13.0</string>
|
||||
<key>NSHighResolutionCapable</key>
|
||||
<true/>
|
||||
<key>NSHumanReadableCopyright</key>
|
||||
<string>Copyright © 2026 Hubris. All rights reserved.</string>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -1,14 +0,0 @@
|
||||
version: '3'
|
||||
|
||||
tasks:
|
||||
build:
|
||||
summary: Build the Oikos desktop app
|
||||
cmds:
|
||||
- go build -o build/bin/Oikos .
|
||||
env:
|
||||
CGO_ENABLED: 1
|
||||
|
||||
dev:
|
||||
summary: Run in development mode
|
||||
cmds:
|
||||
- go run .
|
||||
@@ -1,26 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>com.apple.security.app-sandbox</key>
|
||||
<false/>
|
||||
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
|
||||
<true/>
|
||||
<key>com.apple.security.cs.disable-library-validation</key>
|
||||
<true/>
|
||||
<key>com.apple.security.device.audio-input</key>
|
||||
<false/>
|
||||
<key>com.apple.security.device.camera</key>
|
||||
<false/>
|
||||
<key>com.apple.security.files.user-selected.read-write</key>
|
||||
<true/>
|
||||
<key>com.apple.security.network.client</key>
|
||||
<true/>
|
||||
<key>com.apple.security.network.server</key>
|
||||
<false/>
|
||||
<key>keychain-access-groups</key>
|
||||
<array>
|
||||
<string>$(AppIdentifierPrefix)com.hubris.oikos-desktop</string>
|
||||
</array>
|
||||
</dict>
|
||||
</plist>
|
||||
Binary file not shown.
Binary file not shown.
|
Before Width: | Height: | Size: 1.5 KiB |
@@ -1,788 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"embed"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"log"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"os/exec"
|
||||
"os/user"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/wailsapp/wails/v3/pkg/application"
|
||||
"github.com/wailsapp/wails/v3/pkg/events"
|
||||
"github.com/zalando/go-keyring"
|
||||
)
|
||||
|
||||
//go:embed frontend/dist
|
||||
var assets embed.FS
|
||||
|
||||
//go:embed icon.png
|
||||
var iconPNG []byte
|
||||
|
||||
const (
|
||||
keyringService = "com.hubris.oikos-desktop"
|
||||
keyringUser = "oikos"
|
||||
updateURL = "https://git.hubris.network/api/v1/repos/dtoro/oikos/releases"
|
||||
pollInterval = 30 * time.Second
|
||||
updateInterval = 6 * time.Hour
|
||||
oidcCallbackPort = 18901
|
||||
)
|
||||
|
||||
// version is injected at link time via -ldflags "-X main.version=$(cat VERSION)"
|
||||
// (Makefile desktop target). The default keeps a non-empty fallback for
|
||||
// `go build ./cmd/desktop` without ldflags.
|
||||
var version = "0.1.0-dev"
|
||||
|
||||
type OikosConfig struct {
|
||||
ApiUrl string `json:"apiUrl"`
|
||||
Token string `json:"token,omitempty"`
|
||||
IsDesktop bool `json:"isDesktop"`
|
||||
}
|
||||
|
||||
// ---- ConfigService ----
|
||||
|
||||
type ConfigService struct{}
|
||||
|
||||
func (c *ConfigService) Name() string { return "config" }
|
||||
|
||||
func (c *ConfigService) SaveConfig(apiUrl, token string) error {
|
||||
cfg := OikosConfig{ApiUrl: apiUrl, Token: token, IsDesktop: true}
|
||||
data, _ := json.Marshal(cfg)
|
||||
return keyring.Set(keyringService, keyringUser, string(data))
|
||||
}
|
||||
|
||||
func (c *ConfigService) ClearConfig() error {
|
||||
return keyring.Delete(keyringService, keyringUser)
|
||||
}
|
||||
|
||||
func (c *ConfigService) GetStoredConfig() *OikosConfig {
|
||||
return loadConfig()
|
||||
}
|
||||
|
||||
func (c *ConfigService) EnableAutoStart() error {
|
||||
if runtime.GOOS != "darwin" {
|
||||
return fmt.Errorf("autostart not supported on %s", runtime.GOOS)
|
||||
}
|
||||
usr, _ := user.Current()
|
||||
dir := filepath.Join(usr.HomeDir, "Library", "LaunchAgents")
|
||||
os.MkdirAll(dir, 0755)
|
||||
|
||||
exe, _ := os.Executable()
|
||||
plist := fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>com.hubris.oikos-desktop</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>%s</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
<key>KeepAlive</key>
|
||||
<false/>
|
||||
</dict>
|
||||
</plist>`, exe)
|
||||
|
||||
return os.WriteFile(filepath.Join(dir, "com.hubris.oikos-desktop.plist"), []byte(plist), 0644)
|
||||
}
|
||||
|
||||
func (c *ConfigService) DisableAutoStart() error {
|
||||
if runtime.GOOS != "darwin" {
|
||||
return fmt.Errorf("autostart not supported on %s", runtime.GOOS)
|
||||
}
|
||||
usr, _ := user.Current()
|
||||
path := filepath.Join(usr.HomeDir, "Library", "LaunchAgents", "com.hubris.oikos-desktop.plist")
|
||||
return os.Remove(path)
|
||||
}
|
||||
|
||||
// ---- Local OIDC server (runs alongside the webview) ----
|
||||
|
||||
type oidcSession struct {
|
||||
apiUrl string
|
||||
verifier string
|
||||
state string
|
||||
ch chan string
|
||||
}
|
||||
|
||||
var (
|
||||
oidcSessionsMu sync.Mutex
|
||||
oidcSessions = make(map[string]*oidcSession)
|
||||
)
|
||||
|
||||
func startOIDCServer() *http.Server {
|
||||
mux := http.NewServeMux()
|
||||
|
||||
cors := func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Access-Control-Allow-Origin", "*")
|
||||
w.Header().Set("Access-Control-Allow-Methods", "GET, OPTIONS")
|
||||
w.Header().Set("Access-Control-Allow-Headers", "Content-Type")
|
||||
if r.Method == "OPTIONS" {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
h := func(path string, handler func(http.ResponseWriter, *http.Request)) {
|
||||
mux.HandleFunc(path, func(w http.ResponseWriter, r *http.Request) {
|
||||
cors(w, r)
|
||||
if r.Method == "OPTIONS" {
|
||||
return
|
||||
}
|
||||
handler(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
h("/oidc/start", func(w http.ResponseWriter, r *http.Request) {
|
||||
apiUrl := strings.TrimRight(r.URL.Query().Get("apiUrl"), "/")
|
||||
returnURL := r.URL.Query().Get("ret")
|
||||
if apiUrl == "" {
|
||||
http.Error(w, "apiUrl required", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
if returnURL == "" {
|
||||
returnURL = "/?desktop=1"
|
||||
}
|
||||
|
||||
oidcCfg, err := fetchOIDCConfig(apiUrl)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
|
||||
verifier, challenge, _ := pkceParams()
|
||||
state := randomString(32)
|
||||
redirectURI := fmt.Sprintf("http://127.0.0.1:%d/oidc/callback", oidcCallbackPort)
|
||||
|
||||
ch := make(chan string, 1)
|
||||
oidcSessionsMu.Lock()
|
||||
sessionID := randomString(16)
|
||||
oidcSessions[sessionID] = &oidcSession{apiUrl: apiUrl, verifier: verifier, state: state, ch: ch}
|
||||
oidcSessionsMu.Unlock()
|
||||
|
||||
authURL := fmt.Sprintf("%s?%s",
|
||||
oidcCfg.AuthorizationEndpoint,
|
||||
url.Values{
|
||||
"response_type": {"code"},
|
||||
"client_id": {oidcCfg.ClientID},
|
||||
"redirect_uri": {redirectURI},
|
||||
"code_challenge": {challenge},
|
||||
"code_challenge_method": {"S256"},
|
||||
"state": {state},
|
||||
"scope": {"openid profile email"},
|
||||
}.Encode(),
|
||||
)
|
||||
|
||||
exec.Command("open", authURL).Start()
|
||||
|
||||
select {
|
||||
case token := <-ch:
|
||||
if token != "" {
|
||||
c := &ConfigService{}
|
||||
c.SaveConfig(apiUrl, token)
|
||||
returnURL += "&token=" + url.QueryEscape(token)
|
||||
}
|
||||
case <-time.After(5 * time.Minute):
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
fmt.Fprintf(w, `<!DOCTYPE html><html><head><meta charset="UTF-8"><title>Oikos</title>
|
||||
<meta http-equiv="refresh" content="0;url=%s">
|
||||
<style>body{font-family:-apple-system,BlinkMacSystemFont,sans-serif;background:#0a0a0a;color:#e0e0e0;display:flex;align-items:center;justify-content:center;min-height:100vh;margin:0}
|
||||
.card{background:#1a1a1a;border:1px solid #2a2a2a;border-radius:12px;padding:32px;max-width:400px;text-align:center}
|
||||
h1{font-size:18px;margin-bottom:8px}.ok{color:#22c55e;font-size:14px}</style>
|
||||
</head><body><div class="card"><h1>Connected</h1><p class="ok">Redirecting back to Oikos…</p></div></body></html>`, returnURL)
|
||||
})
|
||||
|
||||
h("/oidc/callback", func(w http.ResponseWriter, r *http.Request) {
|
||||
code := r.URL.Query().Get("code")
|
||||
gotState := r.URL.Query().Get("state")
|
||||
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
|
||||
oidcSessionsMu.Lock()
|
||||
var session *oidcSession
|
||||
var sessionID string
|
||||
for id, s := range oidcSessions {
|
||||
if s.state == gotState {
|
||||
session = s
|
||||
sessionID = id
|
||||
break
|
||||
}
|
||||
}
|
||||
oidcSessionsMu.Unlock()
|
||||
|
||||
if session == nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
w.Write([]byte("Invalid state."))
|
||||
return
|
||||
}
|
||||
|
||||
token, err := exchangeCode(
|
||||
session.apiUrl,
|
||||
code, session.verifier,
|
||||
fmt.Sprintf("http://127.0.0.1:%d/oidc/callback", oidcCallbackPort),
|
||||
)
|
||||
|
||||
oidcSessionsMu.Lock()
|
||||
delete(oidcSessions, sessionID)
|
||||
oidcSessionsMu.Unlock()
|
||||
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
fmt.Fprintf(w, "Token exchange failed: %v", err)
|
||||
session.ch <- ""
|
||||
return
|
||||
}
|
||||
|
||||
w.Write([]byte(`<!DOCTYPE html><html><head><meta charset="UTF-8"><title>Oikos</title>
|
||||
<style>body{font-family:-apple-system,BlinkMacSystemFont,sans-serif;background:#0a0a0a;color:#e0e0e0;display:flex;align-items:center;justify-content:center;min-height:100vh;margin:0}
|
||||
.card{background:#1a1a1a;border:1px solid #2a2a2a;border-radius:12px;padding:32px;max-width:400px;text-align:center}
|
||||
h1{font-size:18px;margin-bottom:8px}.ok{color:#22c55e;font-size:14px}</style>
|
||||
</head><body><div class="card"><h1>Connected</h1><p class="ok">You can close this window and return to Oikos.</p></div></body></html>`))
|
||||
session.ch <- token
|
||||
})
|
||||
|
||||
mux.HandleFunc("/oidc/config", func(w http.ResponseWriter, r *http.Request) {
|
||||
apiUrl := strings.TrimRight(r.URL.Query().Get("apiUrl"), "/")
|
||||
if apiUrl == "" {
|
||||
http.Error(w, "apiUrl required", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
cfg, err := fetchOIDCConfig(apiUrl)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(cfg)
|
||||
})
|
||||
|
||||
mux.HandleFunc("/update/check", func(w http.ResponseWriter, r *http.Request) {
|
||||
latest := fetchLatestRelease()
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Header().Set("Access-Control-Allow-Origin", "*")
|
||||
if latest == nil {
|
||||
json.NewEncoder(w).Encode(map[string]string{"current": version})
|
||||
return
|
||||
}
|
||||
hasAsset := false
|
||||
for _, a := range latest.Assets {
|
||||
if strings.Contains(a.Name, "darwin") {
|
||||
hasAsset = true
|
||||
updater.mu.Lock()
|
||||
updater.latestURL = a.BrowserDownloadURL
|
||||
updater.mu.Unlock()
|
||||
break
|
||||
}
|
||||
}
|
||||
json.NewEncoder(w).Encode(map[string]string{
|
||||
"current": version,
|
||||
"latest": latest.Version,
|
||||
"has_asset": fmt.Sprintf("%t", hasAsset),
|
||||
})
|
||||
})
|
||||
|
||||
listener, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", oidcCallbackPort))
|
||||
if err != nil {
|
||||
log.Printf("OIDC server: %v", err)
|
||||
return nil
|
||||
}
|
||||
log.Printf("OIDC server listening on %s", listener.Addr())
|
||||
srv := &http.Server{Handler: mux}
|
||||
go srv.Serve(listener)
|
||||
return srv
|
||||
}
|
||||
|
||||
// ---- Window persistence ----
|
||||
|
||||
type windowState struct {
|
||||
X int `json:"x"`
|
||||
Y int `json:"y"`
|
||||
Width int `json:"width"`
|
||||
Height int `json:"height"`
|
||||
}
|
||||
|
||||
func windowStatePath() string {
|
||||
usr, _ := user.Current()
|
||||
return filepath.Join(usr.HomeDir, ".config", "oikos", "window.json")
|
||||
}
|
||||
|
||||
func loadWindowState() *windowState {
|
||||
data, err := os.ReadFile(windowStatePath())
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var ws windowState
|
||||
if err := json.Unmarshal(data, &ws); err != nil {
|
||||
return nil
|
||||
}
|
||||
if ws.Width < 200 || ws.Height < 200 {
|
||||
return nil
|
||||
}
|
||||
return &ws
|
||||
}
|
||||
|
||||
func saveWindowState(w application.Window) {
|
||||
x, y := w.Position()
|
||||
width, height := w.Size()
|
||||
ws := windowState{X: x, Y: y, Width: width, Height: height}
|
||||
data, _ := json.Marshal(ws)
|
||||
|
||||
usr, _ := user.Current()
|
||||
dir := filepath.Join(usr.HomeDir, ".config", "oikos")
|
||||
os.MkdirAll(dir, 0755)
|
||||
os.WriteFile(filepath.Join(dir, "window.json"), data, 0644)
|
||||
}
|
||||
|
||||
func loadConfig() *OikosConfig {
|
||||
data, err := keyring.Get(keyringService, keyringUser)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var cfg OikosConfig
|
||||
if err := json.Unmarshal([]byte(data), &cfg); err != nil {
|
||||
return nil
|
||||
}
|
||||
cfg.IsDesktop = true
|
||||
return &cfg
|
||||
}
|
||||
|
||||
type oidcConfig struct {
|
||||
Issuer string `json:"issuer"`
|
||||
ClientID string `json:"client_id"`
|
||||
AuthorizationEndpoint string `json:"authorization_endpoint"`
|
||||
}
|
||||
|
||||
func fetchOIDCConfig(apiUrl string) (*oidcConfig, error) {
|
||||
resp, err := http.Get(apiUrl + "/api/v1/auth/oidc-config")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != 200 {
|
||||
return nil, fmt.Errorf("server returned %d", resp.StatusCode)
|
||||
}
|
||||
var cfg oidcConfig
|
||||
if err := json.NewDecoder(resp.Body).Decode(&cfg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &cfg, nil
|
||||
}
|
||||
|
||||
func pkceParams() (verifier, challenge string, _ error) {
|
||||
v := randomString(64)
|
||||
h := sha256.Sum256([]byte(v))
|
||||
return v, base64.RawURLEncoding.EncodeToString(h[:]), nil
|
||||
}
|
||||
|
||||
func randomString(n int) string {
|
||||
b := make([]byte, n)
|
||||
rand.Read(b)
|
||||
return base64.RawURLEncoding.EncodeToString(b)
|
||||
}
|
||||
|
||||
func exchangeCode(apiUrl, code, verifier, redirectURI string) (string, error) {
|
||||
body, _ := json.Marshal(map[string]string{
|
||||
"grant_type": "authorization_code",
|
||||
"code": code,
|
||||
"code_verifier": verifier,
|
||||
"redirect_uri": redirectURI,
|
||||
})
|
||||
|
||||
resp, err := http.Post(apiUrl+"/api/v1/auth/oidc-token", "application/json", strings.NewReader(string(body)))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != 200 {
|
||||
b, _ := io.ReadAll(resp.Body)
|
||||
return "", fmt.Errorf("token endpoint: %d — %s", resp.StatusCode, string(b))
|
||||
}
|
||||
|
||||
var tokens struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&tokens); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if tokens.AccessToken == "" {
|
||||
return "", fmt.Errorf("no access_token in response")
|
||||
}
|
||||
return tokens.AccessToken, nil
|
||||
}
|
||||
|
||||
// ---- Notifications ----
|
||||
|
||||
type dashboardSummary struct {
|
||||
ApprovalsPending int `json:"approvals_pending"`
|
||||
Signals struct {
|
||||
Critical int `json:"critical"`
|
||||
} `json:"signals_by_severity"`
|
||||
}
|
||||
|
||||
func (d *dashboardSummary) alertCount() int {
|
||||
return d.ApprovalsPending + d.Signals.Critical
|
||||
}
|
||||
|
||||
func notify(title, subtitle string) {
|
||||
if runtime.GOOS != "darwin" {
|
||||
return
|
||||
}
|
||||
script := fmt.Sprintf(
|
||||
`display notification "%s" with title "%s" sound name "default"`,
|
||||
strings.ReplaceAll(subtitle, `"`, `\"`),
|
||||
strings.ReplaceAll(title, `"`, `\"`),
|
||||
)
|
||||
exec.Command("osascript", "-e", script).Run()
|
||||
}
|
||||
|
||||
func pollDashboard(cfg *OikosConfig) {
|
||||
if cfg == nil || cfg.ApiUrl == "" || cfg.Token == "" {
|
||||
return
|
||||
}
|
||||
|
||||
var lastCount int
|
||||
first := true
|
||||
|
||||
for {
|
||||
req, err := http.NewRequest("GET", cfg.ApiUrl+"/api/v1/dashboard/summary", nil)
|
||||
if err != nil {
|
||||
time.Sleep(pollInterval)
|
||||
continue
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+cfg.Token)
|
||||
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
time.Sleep(pollInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
resp.Body.Close()
|
||||
|
||||
var summary dashboardSummary
|
||||
if err := json.Unmarshal(body, &summary); err != nil {
|
||||
time.Sleep(pollInterval)
|
||||
continue
|
||||
}
|
||||
|
||||
if first {
|
||||
lastCount = summary.alertCount()
|
||||
first = false
|
||||
} else {
|
||||
current := summary.alertCount()
|
||||
if current > lastCount {
|
||||
notify("Oikos", fmt.Sprintf("%d pending approval(s), %d critical signal(s)", summary.ApprovalsPending, summary.Signals.Critical))
|
||||
}
|
||||
lastCount = current
|
||||
}
|
||||
|
||||
time.Sleep(pollInterval)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Auto-update ----
|
||||
|
||||
type giteaRelease struct {
|
||||
TagName string `json:"tag_name"`
|
||||
Assets []struct {
|
||||
Name string `json:"name"`
|
||||
BrowserDownloadURL string `json:"browser_download_url"`
|
||||
} `json:"assets"`
|
||||
}
|
||||
|
||||
type updateState struct {
|
||||
mu sync.Mutex
|
||||
latestURL string
|
||||
}
|
||||
|
||||
var updater = &updateState{}
|
||||
|
||||
// CheckForUpdates checks Gitea releases for a newer version. If found, stores
|
||||
// the download URL and returns the latest version string (empty if current).
|
||||
func (c *ConfigService) CheckForUpdates() string {
|
||||
latest := fetchLatestRelease()
|
||||
if latest == nil || latest.Version == version {
|
||||
return ""
|
||||
}
|
||||
for _, a := range latest.Assets {
|
||||
if strings.Contains(a.Name, "darwin") {
|
||||
updater.mu.Lock()
|
||||
updater.latestURL = a.BrowserDownloadURL
|
||||
updater.mu.Unlock()
|
||||
return latest.Version
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// InstallUpdate downloads the stored update, replaces the app, and restarts.
|
||||
func (c *ConfigService) InstallUpdate() error {
|
||||
updater.mu.Lock()
|
||||
url := updater.latestURL
|
||||
updater.mu.Unlock()
|
||||
if url == "" {
|
||||
return fmt.Errorf("no update available")
|
||||
}
|
||||
return doUpdate(url)
|
||||
}
|
||||
|
||||
type latestRelease struct {
|
||||
Version string
|
||||
Assets []struct {
|
||||
Name string
|
||||
BrowserDownloadURL string
|
||||
}
|
||||
}
|
||||
|
||||
func fetchLatestRelease() *latestRelease {
|
||||
resp, err := http.Get(updateURL + "?draft=false&pre-release=false&limit=1")
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var releases []giteaRelease
|
||||
if err := json.NewDecoder(resp.Body).Decode(&releases); err != nil || len(releases) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
r := releases[0]
|
||||
v := strings.TrimPrefix(r.TagName, "v")
|
||||
if v == version {
|
||||
return nil
|
||||
}
|
||||
|
||||
lr := &latestRelease{Version: v}
|
||||
for _, a := range r.Assets {
|
||||
lr.Assets = append(lr.Assets, struct {
|
||||
Name string
|
||||
BrowserDownloadURL string
|
||||
}{a.Name, a.BrowserDownloadURL})
|
||||
}
|
||||
return lr
|
||||
}
|
||||
|
||||
func doUpdate(downloadURL string) error {
|
||||
tmp, err := os.CreateTemp("", "oikos-update-*.zip")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(tmp.Name())
|
||||
|
||||
resp, err := http.Get(downloadURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if _, err := io.Copy(tmp, resp.Body); err != nil {
|
||||
return err
|
||||
}
|
||||
tmp.Close()
|
||||
|
||||
extractDir, err := os.MkdirTemp("", "oikos-extract")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.RemoveAll(extractDir)
|
||||
|
||||
cmd := exec.Command("unzip", "-o", tmp.Name(), "-d", extractDir)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
return fmt.Errorf("unzip: %w: %s", err, out)
|
||||
}
|
||||
|
||||
newApp := filepath.Join(extractDir, "Oikos.app")
|
||||
if _, err := os.Stat(newApp); err != nil {
|
||||
return fmt.Errorf("extracted app not found: %w", err)
|
||||
}
|
||||
|
||||
currentApp := "/Applications/Oikos.app"
|
||||
if _, err := os.Stat(currentApp); os.IsNotExist(err) {
|
||||
if exe, err := os.Executable(); err == nil {
|
||||
currentApp = filepath.Dir(filepath.Dir(filepath.Dir(exe)))
|
||||
}
|
||||
}
|
||||
|
||||
script := fmt.Sprintf(`#!/bin/bash
|
||||
sleep 2
|
||||
rm -rf "%s"
|
||||
mv "%s" "%s"
|
||||
open "%s"
|
||||
rm "$0"
|
||||
`, currentApp, newApp, currentApp, currentApp)
|
||||
|
||||
scriptPath := filepath.Join(os.TempDir(), "oikos-update.sh")
|
||||
if err := os.WriteFile(scriptPath, []byte(script), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
app := application.Get()
|
||||
exec.Command("open", scriptPath).Start()
|
||||
if app != nil {
|
||||
app.Quit()
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func checkUpdates() {
|
||||
for {
|
||||
time.Sleep(updateInterval)
|
||||
|
||||
latest := fetchLatestRelease()
|
||||
if latest == nil {
|
||||
continue
|
||||
}
|
||||
|
||||
for _, a := range latest.Assets {
|
||||
if strings.Contains(a.Name, "darwin") {
|
||||
updater.mu.Lock()
|
||||
updater.latestURL = a.BrowserDownloadURL
|
||||
updater.mu.Unlock()
|
||||
|
||||
app := application.Get()
|
||||
if app == nil {
|
||||
continue
|
||||
}
|
||||
msg := fmt.Sprintf("Version %s is available (you have %s).", latest.Version, version)
|
||||
app.Dialog.Info().
|
||||
SetTitle("Update Available").
|
||||
SetMessage(msg).
|
||||
Show()
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Main ----
|
||||
|
||||
func main() {
|
||||
oidcSrv := startOIDCServer()
|
||||
defer oidcSrv.Close()
|
||||
|
||||
distFS, err := fs.Sub(assets, "frontend/dist")
|
||||
if err != nil {
|
||||
log.Fatalf("embedded assets: %v", err)
|
||||
}
|
||||
|
||||
app := application.New(application.Options{
|
||||
Name: "Oikos",
|
||||
Description: "Homelab Control Room",
|
||||
Services: []application.Service{
|
||||
application.NewService(&ConfigService{}),
|
||||
},
|
||||
Assets: application.AssetOptions{
|
||||
Handler: application.AssetFileServerFS(distFS),
|
||||
},
|
||||
Mac: application.MacOptions{
|
||||
ApplicationShouldTerminateAfterLastWindowClosed: false,
|
||||
},
|
||||
})
|
||||
|
||||
systemTray := app.SystemTray.New()
|
||||
systemTray.SetTooltip("Oikos")
|
||||
systemTray.SetIcon(iconPNG)
|
||||
|
||||
trayMenu := application.NewMenu()
|
||||
trayMenu.Add("Open Oikos").OnClick(func(ctx *application.Context) {
|
||||
for _, w := range app.Window.GetAll() {
|
||||
w.Show()
|
||||
w.Focus()
|
||||
}
|
||||
})
|
||||
trayMenu.AddSeparator()
|
||||
trayMenu.Add("Check for Updates").OnClick(func(ctx *application.Context) {
|
||||
go func() {
|
||||
latest := fetchLatestRelease()
|
||||
if latest == nil {
|
||||
app.Dialog.Info().SetTitle("Up to Date").SetMessage("You are running the latest version (" + version + ").").Show()
|
||||
return
|
||||
}
|
||||
for _, a := range latest.Assets {
|
||||
if strings.Contains(a.Name, "darwin") {
|
||||
updater.mu.Lock()
|
||||
updater.latestURL = a.BrowserDownloadURL
|
||||
updater.mu.Unlock()
|
||||
msg := fmt.Sprintf("Version %s is available (you have %s). Install now?", latest.Version, version)
|
||||
d := app.Dialog.Question().SetTitle("Update Available").SetMessage(msg)
|
||||
yes := d.AddButton("Install")
|
||||
yes.OnClick(func() { doUpdate(updater.latestURL) })
|
||||
no := d.AddButton("Later")
|
||||
d.SetDefaultButton(yes)
|
||||
d.SetCancelButton(no)
|
||||
d.Show()
|
||||
return
|
||||
}
|
||||
}
|
||||
app.Dialog.Info().SetTitle("Up to Date").SetMessage("You are running the latest version (" + version + ").").Show()
|
||||
}()
|
||||
})
|
||||
trayMenu.AddSeparator()
|
||||
trayMenu.Add("Quit").OnClick(func(ctx *application.Context) {
|
||||
app.Quit()
|
||||
})
|
||||
systemTray.SetMenu(trayMenu)
|
||||
|
||||
ws := loadWindowState()
|
||||
width, height := 1400, 900
|
||||
if ws != nil {
|
||||
width = ws.Width
|
||||
height = ws.Height
|
||||
}
|
||||
|
||||
window := app.Window.NewWithOptions(application.WebviewWindowOptions{
|
||||
Title: "Oikos",
|
||||
Width: width,
|
||||
Height: height,
|
||||
MinWidth: 1024,
|
||||
MinHeight: 700,
|
||||
URL: "/?desktop=1",
|
||||
})
|
||||
|
||||
if ws != nil {
|
||||
window.SetPosition(ws.X, ws.Y)
|
||||
} else {
|
||||
window.Center()
|
||||
}
|
||||
|
||||
window.RegisterHook(events.Common.WindowClosing, func(e *application.WindowEvent) {
|
||||
window.Hide()
|
||||
e.Cancel()
|
||||
})
|
||||
|
||||
window.Show()
|
||||
|
||||
systemTray.AttachWindow(window)
|
||||
systemTray.Run()
|
||||
|
||||
app.OnShutdown(func() {
|
||||
saveWindowState(window)
|
||||
})
|
||||
|
||||
go pollDashboard(loadConfig())
|
||||
go checkUpdates()
|
||||
|
||||
err = app.Run()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -1,5 +0,0 @@
|
||||
<svg width="88" height="88" viewBox="0 0 110 120" xmlns="http://www.w3.org/2000/svg">
|
||||
<g transform="translate(9, 10) scale(0.9)">
|
||||
<path fill="#ffffff" d="m45.601 1q20.993 0 33.71 15.946 10.799 13.625 10.799 31.287 0 12.414-5.9548 25.131-5.9548 12.717-16.451 19.176-10.395 6.4592-23.213 6.4592-20.892 0-33.205-16.653-10.395-14.029-10.395-31.489 0-12.717 6.2577-25.232 6.3584-12.616 16.653-18.57 10.295-6.0556 21.801-6.0556zm-3.128 6.5605q-5.3492 0-10.799 3.2296-5.3492 3.1287-8.68 11.102-3.3305 7.9735-3.3305 20.488 0 20.185 7.973 34.82 8.0743 14.634 21.195 14.634 9.7896 0 16.149-8.0743 6.3584-8.0743 6.3584-27.755 0-24.627-10.597-38.756-7.1657-9.6888-18.268-9.6888z"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 690 B |
@@ -1,9 +0,0 @@
|
||||
{
|
||||
"name": "oikos",
|
||||
"outputfilename": "oikos-desktop",
|
||||
"frontend:dir": "frontend",
|
||||
"author": {
|
||||
"name": "Hubris",
|
||||
"email": "d.toro.v@pm.me"
|
||||
}
|
||||
}
|
||||
@@ -66,9 +66,9 @@ type agent struct {
|
||||
queue *messageQueue
|
||||
}
|
||||
|
||||
func newAgent(ctx context.Context, clients *mcpClientPool, st *store, agentSlug string) (*agent, error) {
|
||||
func newAgent(ctx context.Context, clients *mcpClientPool, st *store, agentSlug string, openrouterAPIKey string) (*agent, error) {
|
||||
system := loadSoul()
|
||||
apiKey := os.Getenv("OPENROUTER_API_KEY")
|
||||
apiKey := openrouterAPIKey
|
||||
model := os.Getenv("NOMOS_MODEL")
|
||||
if model == "" {
|
||||
// v4-pro over v4-flash: the flash tier over-narrates, occasionally
|
||||
|
||||
348
cmd/nomos/mcp.go
Normal file
348
cmd/nomos/mcp.go
Normal file
@@ -0,0 +1,348 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ─── MCP Streamable HTTP client ────────────────────────────────────────
|
||||
|
||||
type mcpClient struct {
|
||||
baseURL string
|
||||
token string // OIKOS_MCP_BEARER_TOKEN — api's combinedAuth requires it on every request (no dev-open bypass)
|
||||
sessionID string
|
||||
http *http.Client
|
||||
nextID int
|
||||
mu sync.Mutex // one client serializes its own MCP calls (the pool gives each session its own client, so this never blocks another session)
|
||||
// toolsCache holds the last tools/list result. The tool list is static
|
||||
// for the lifetime of one MCP connection — it only changes when the api
|
||||
// process (re)registers tools, i.e. on a restart, which this client
|
||||
// already detects and reacts to via reconnectLocked. Without this,
|
||||
// buildTools (called at the start of EVERY chat turn, including every
|
||||
// auto-continuation resume) paid a full tools/list round-trip every
|
||||
// single time for a list that's almost always identical to the last one.
|
||||
// Guarded separately from mu (not reused) so a cache check never
|
||||
// contends with an in-flight doRequest call for a different method.
|
||||
toolsMu sync.Mutex
|
||||
toolsCache []toolDef
|
||||
}
|
||||
|
||||
func newMCPClient(baseURL, token string) (*mcpClient, error) {
|
||||
c := &mcpClient{
|
||||
baseURL: baseURL,
|
||||
token: token,
|
||||
http: &http.Client{Timeout: 120 * time.Second},
|
||||
}
|
||||
|
||||
resp, err := c.doRequest("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initialize: %w", err)
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return nil, fmt.Errorf("no session ID in initialize response")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
|
||||
c.doRequest("notifications/initialized", map[string]any{})
|
||||
|
||||
slog.Info("nomos: mcp connected", "session", c.sessionID[:16]+"...")
|
||||
return c, nil
|
||||
}
|
||||
|
||||
type mcpJSONRPCResponse struct {
|
||||
sessionID string
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error json.RawMessage `json:"error"`
|
||||
}
|
||||
|
||||
// errStaleSession signals that the MCP server rejected our session id (e.g.
|
||||
// after an api/MCP restart), so the client should re-initialize and retry.
|
||||
var errStaleSession = fmt.Errorf("mcp session stale")
|
||||
|
||||
// doRequest serializes MCP calls and transparently re-initializes the session
|
||||
// if the server has forgotten it (common after an api redeploy), retrying the
|
||||
// original call once. Without this, an api restart permanently breaks nomos
|
||||
// until it is itself restarted.
|
||||
func (c *mcpClient) doRequest(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
resp, err := c.send(method, params)
|
||||
if err != nil && method != "initialize" && errors.Is(err, errStaleSession) {
|
||||
slog.Warn("nomos: mcp session stale, reconnecting")
|
||||
if rerr := c.reconnectLocked(); rerr != nil {
|
||||
return nil, fmt.Errorf("mcp reconnect: %w (original: %v)", rerr, err)
|
||||
}
|
||||
return c.send(method, params)
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
// reconnectLocked re-initializes the MCP session. The caller must hold c.mu.
|
||||
func (c *mcpClient) reconnectLocked() error {
|
||||
c.sessionID = ""
|
||||
// A reconnect means the api process was restarted (or forgot us) — its
|
||||
// tool registration may have changed, so the cached list is no longer
|
||||
// trustworthy.
|
||||
c.toolsMu.Lock()
|
||||
c.toolsCache = nil
|
||||
c.toolsMu.Unlock()
|
||||
resp, err := c.send("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return fmt.Errorf("no session ID on re-initialize")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
_, _ = c.send("notifications/initialized", map[string]any{})
|
||||
slog.Info("nomos: mcp reconnected", "session", c.sessionID[:16]+"...")
|
||||
return nil
|
||||
}
|
||||
|
||||
// send performs one MCP round-trip. It does not lock; callers hold c.mu.
|
||||
func (c *mcpClient) send(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.nextID++
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"jsonrpc": "2.0",
|
||||
"method": method,
|
||||
"params": params,
|
||||
"id": c.nextID,
|
||||
})
|
||||
|
||||
req, err := http.NewRequest(http.MethodPost, c.baseURL, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json, text/event-stream")
|
||||
if c.sessionID != "" {
|
||||
req.Header.Set("Mcp-Session-Id", c.sessionID)
|
||||
}
|
||||
if c.token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
}
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// A rejected/unknown session comes back as 4xx (commonly 400/404).
|
||||
if resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusBadRequest {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
result := &mcpJSONRPCResponse{}
|
||||
result.sessionID = resp.Header.Get("Mcp-Session-Id")
|
||||
|
||||
scanner := bufio.NewScanner(resp.Body)
|
||||
scanner.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
|
||||
gotData := false
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if strings.HasPrefix(line, "data: ") {
|
||||
gotData = true
|
||||
data := line[6:]
|
||||
if err := json.Unmarshal([]byte(data), result); err != nil {
|
||||
return nil, fmt.Errorf("parse response: %w", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if result.Error != nil {
|
||||
return nil, fmt.Errorf("rpc error: %s", string(result.Error))
|
||||
}
|
||||
|
||||
// Empty body with no result and a session set: the server likely dropped
|
||||
// our session. Notifications legitimately return no data, so exempt them.
|
||||
if !gotData && result.Result == nil && method != "notifications/initialized" {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
if result.sessionID != "" {
|
||||
c.sessionID = result.sessionID
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) callTool(name string, args map[string]any) (any, error) {
|
||||
resp, err := c.doRequest("tools/call", map[string]any{
|
||||
"name": name,
|
||||
"arguments": args,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var toolResult struct {
|
||||
Content []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &toolResult); err != nil {
|
||||
return string(resp.Result), nil
|
||||
}
|
||||
|
||||
var texts []string
|
||||
for _, c := range toolResult.Content {
|
||||
if c.Type == "text" {
|
||||
var parsed any
|
||||
if json.Unmarshal([]byte(c.Text), &parsed) == nil {
|
||||
return parsed, nil
|
||||
}
|
||||
texts = append(texts, c.Text)
|
||||
}
|
||||
}
|
||||
if len(texts) == 1 {
|
||||
return texts[0], nil
|
||||
}
|
||||
return texts, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) listTools() ([]string, error) {
|
||||
resp, err := c.doRequest("tools/list", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tr struct {
|
||||
Tools []struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
} `json:"tools"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &tr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var names []string
|
||||
for _, t := range tr.Tools {
|
||||
names = append(names, t.Name)
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) close() {
|
||||
}
|
||||
|
||||
// ─── Per-session MCP client pool ────────────────────────────────────────
|
||||
//
|
||||
// A single shared mcpClient serializes EVERY tool call across EVERY
|
||||
// concurrently-running task through one mutex (see mcpClient.mu) — `run`
|
||||
// executes its SSH command synchronously inside that lock and is capped at
|
||||
// up to 10 minutes, so one task mid-`run` stalled every other task's tool
|
||||
// calls, even trivial reads, behind it. The MCP *server* has no per-
|
||||
// connection state to protect (newServer in internal/mcp/server.go returns
|
||||
// one shared *mcp.Server instance whose tool handlers close only over the DB
|
||||
// pool, which is already safe for concurrent use) — the mutex existed purely
|
||||
// because the *client* reused one stateful transport session, not because
|
||||
// the server needed it. Giving each task's own session its own client
|
||||
// removes the cross-task serialization entirely: a task's own tool calls
|
||||
// stay sequential (which they already are — the agent loop calls tools one
|
||||
// at a time within a turn), but no longer block anyone else's.
|
||||
type mcpClientPool struct {
|
||||
baseURL string
|
||||
token string
|
||||
mu sync.Mutex
|
||||
clients map[string]*pooledMCPClient
|
||||
}
|
||||
|
||||
type pooledMCPClient struct {
|
||||
client *mcpClient
|
||||
lastUsed time.Time
|
||||
}
|
||||
|
||||
func newMCPClientPool(baseURL, token string) *mcpClientPool {
|
||||
return &mcpClientPool{baseURL: baseURL, token: token, clients: make(map[string]*pooledMCPClient)}
|
||||
}
|
||||
|
||||
// get returns the client for sessionID, creating and initializing one (a
|
||||
// real MCP handshake) on first use. Session ids that don't identify a real
|
||||
// persisted conversation ("" / "ephemeral", the no-DB-store path; "query",
|
||||
// the structured /query endpoint) still get exactly one dedicated,
|
||||
// reused client each via the same map — just keyed on a fixed string instead
|
||||
// of a real session id — so that traffic doesn't pay a fresh handshake per
|
||||
// request while still never sharing a connection with an actual task.
|
||||
func (p *mcpClientPool) get(sessionID string) (*mcpClient, error) {
|
||||
key := sessionID
|
||||
if key == "" {
|
||||
key = "ephemeral"
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
if pc, ok := p.clients[key]; ok {
|
||||
pc.lastUsed = time.Now()
|
||||
p.mu.Unlock()
|
||||
return pc.client, nil
|
||||
}
|
||||
p.mu.Unlock()
|
||||
|
||||
// Initialize outside the lock — it's a network round-trip, and holding
|
||||
// the pool mutex for it would serialize unrelated sessions' first calls
|
||||
// behind each other, undermining the whole point of this pool.
|
||||
c, err := newMCPClient(p.baseURL, p.token)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
// Another goroutine may have created one for the same key while we were
|
||||
// initializing (two of this session's tool calls racing on a cold
|
||||
// start); keep whichever won, close out the loser's connection (a no-op
|
||||
// today, but future-proof if mcpClient.close ever does real teardown).
|
||||
if existing, ok := p.clients[key]; ok {
|
||||
p.mu.Unlock()
|
||||
c.close()
|
||||
return existing.client, nil
|
||||
}
|
||||
p.clients[key] = &pooledMCPClient{client: c, lastUsed: time.Now()}
|
||||
p.mu.Unlock()
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// mcpClientIdleTimeout is how long an idle session's MCP client is kept
|
||||
// before eviction — long enough to outlive a single slow `run` (capped at 10
|
||||
// minutes server-side) plus normal think-time between a task's tool calls,
|
||||
// short enough not to accumulate one abandoned connection per finished task
|
||||
// forever.
|
||||
const mcpClientIdleTimeout = 20 * time.Minute
|
||||
|
||||
// sweep evicts clients idle past mcpClientIdleTimeout. Call on a ticker.
|
||||
func (p *mcpClientPool) sweep() {
|
||||
cutoff := time.Now().Add(-mcpClientIdleTimeout)
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
if pc.lastUsed.Before(cutoff) {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *mcpClientPool) closeAll() {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
@@ -1,11 +1,8 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
@@ -18,7 +15,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
"github.com/google/uuid"
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
@@ -27,13 +24,14 @@ func main() {
|
||||
fmt.Fprintln(os.Stderr, "usage: nomos serve")
|
||||
os.Exit(1)
|
||||
}
|
||||
if os.Args[1] == "healthcheck" {
|
||||
runHealthcheck()
|
||||
return
|
||||
}
|
||||
mcpURL := os.Getenv("NOMOS_MCP_URL")
|
||||
if mcpURL == "" {
|
||||
mcpURL = "http://localhost:8090/mcp"
|
||||
}
|
||||
// api's combinedAuth requires a bearer token on every request (no
|
||||
// dev-open bypass — plans/2026-07-12-wails-desktop-app.md 0.4); this is
|
||||
// the same shared secret api validates against (OIKOS_MCP_BEARER_TOKEN).
|
||||
mcpToken := os.Getenv("OIKOS_MCP_BEARER_TOKEN")
|
||||
|
||||
agentSlug := os.Getenv("NOMOS_AGENT_SLUG")
|
||||
@@ -46,6 +44,32 @@ func main() {
|
||||
databaseURL = os.Getenv("OIKOS_DATABASE_URL")
|
||||
}
|
||||
|
||||
sec := secrets.NewManagerFromConfig(
|
||||
os.Getenv("OIKOS_INFISICAL_SITE_URL"),
|
||||
os.Getenv("OIKOS_INFISICAL_CLIENT_ID"),
|
||||
os.Getenv("OIKOS_INFISICAL_CLIENT_SECRET"),
|
||||
os.Getenv("OIKOS_INFISICAL_PROJECT_ID"),
|
||||
os.Getenv("OIKOS_INFISICAL_ENV"),
|
||||
os.Getenv("OIKOS_SECRETS_DIR"),
|
||||
)
|
||||
var openrouterAPIKey string
|
||||
var secretsResolved int
|
||||
if sec != nil {
|
||||
resCtx, resCancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
if v := secrets.ResolveSecret(resCtx, sec, "mcp_bearer-token", ""); v != "" {
|
||||
mcpToken = v
|
||||
secretsResolved++
|
||||
}
|
||||
openrouterAPIKey = secrets.ResolveSecret(resCtx, sec, "openrouter_api-key", os.Getenv("OPENROUTER_API_KEY"))
|
||||
if openrouterAPIKey != "" && openrouterAPIKey != os.Getenv("OPENROUTER_API_KEY") {
|
||||
secretsResolved++
|
||||
}
|
||||
resCancel()
|
||||
if secretsResolved > 0 {
|
||||
slog.Info("nomos: secrets resolved from Infisical", "count", secretsResolved)
|
||||
}
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "serve":
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
|
||||
@@ -75,7 +99,7 @@ func main() {
|
||||
defer st.close()
|
||||
}
|
||||
|
||||
nAgent, err := newAgent(ctx, clientPool, st, agentSlug)
|
||||
nAgent, err := newAgent(ctx, clientPool, st, agentSlug, openrouterAPIKey)
|
||||
if err != nil {
|
||||
slog.Error("nomos: agent init", "error", err)
|
||||
os.Exit(1)
|
||||
@@ -161,153 +185,32 @@ func main() {
|
||||
}
|
||||
}
|
||||
|
||||
func runHealthcheck() {
|
||||
addr := os.Getenv("NOMOS_LISTEN")
|
||||
if addr == "" {
|
||||
addr = ":8092"
|
||||
}
|
||||
host := addr
|
||||
if strings.HasPrefix(host, ":") {
|
||||
host = "127.0.0.1" + host
|
||||
}
|
||||
client := &http.Client{Timeout: 3 * time.Second}
|
||||
resp, err := client.Get("http://" + host + "/healthz")
|
||||
if err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func sseEvent(w http.ResponseWriter, flusher http.Flusher, event agentEvent) {
|
||||
data, _ := json.Marshal(event)
|
||||
fmt.Fprintf(w, "data: %s\n\n", data)
|
||||
flusher.Flush()
|
||||
}
|
||||
|
||||
// runChatTurn is the shared core of an operator-initiated turn: insert an
|
||||
// assistant placeholder, run a.chat with incremental persistence (so whatever
|
||||
// happened before an abort is never lost), finalize the row, and derive a
|
||||
// title. It is agnostic to the transport: `sink` receives every agent event
|
||||
// for delivery (SSE for a live handleChat, a no-op for a queued turn that has
|
||||
// no client attached — the frontend learns about those via the poller + the
|
||||
// status-driven "working" signal). The caller MUST already hold the session's
|
||||
// turn-gate permit.
|
||||
func (a *agent) runChatTurn(pctx, ctx context.Context, sessionID, message string, sink func(agentEvent)) {
|
||||
toolCalls := []map[string]any{}
|
||||
// P3: accumulate per-iteration reasoning instead of overwriting with the
|
||||
// final `text` event (see the original inline comment in handleChat).
|
||||
var textParts []string
|
||||
var thinkingParts []string
|
||||
var finalText string
|
||||
var finalThinking string
|
||||
|
||||
placeholder, _ := json.Marshal(map[string]any{"role": "assistant", "text": ""})
|
||||
msgID, err := a.store.insertMessageReturningID(pctx, sessionID, "assistant", placeholder)
|
||||
if err != nil {
|
||||
slog.Error("nomos: chat placeholder insert failed", "session", sessionID, "error", err)
|
||||
}
|
||||
persist := func() {
|
||||
if msgID == uuid.Nil {
|
||||
return
|
||||
}
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": finalText,
|
||||
"thinking": finalThinking,
|
||||
"tool_calls": toolCalls,
|
||||
})
|
||||
a.store.updateMessage(pctx, msgID, body)
|
||||
}
|
||||
|
||||
a.chat(ctx, sessionID, message, func(ev agentEvent) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
m["type"] = ev.Type
|
||||
// One entry per tool call: tool_use creates it, tool_result
|
||||
// merges the result into the same entry (matched by id).
|
||||
id, _ := m["id"].(string)
|
||||
if id != "" && ev.Type == "tool_result" {
|
||||
for _, existing := range toolCalls {
|
||||
if eID, _ := existing["id"].(string); eID == id {
|
||||
for k, v := range m {
|
||||
existing[k] = v
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
} else {
|
||||
toolCalls = append(toolCalls, m)
|
||||
}
|
||||
}
|
||||
persist() // live: survives even if the client disconnects right after
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
if t, ok := ev.Data.(string); ok && t != "" {
|
||||
if ev.IsThinking {
|
||||
thinkingParts = append(thinkingParts, t)
|
||||
finalThinking = strings.Join(thinkingParts, "\n\n")
|
||||
} else {
|
||||
textParts = append(textParts, t)
|
||||
finalText = strings.Join(textParts, "\n\n")
|
||||
}
|
||||
persist()
|
||||
}
|
||||
}
|
||||
sink(ev)
|
||||
})
|
||||
|
||||
// B.6: if the turn ended with no text and no tool calls (the model
|
||||
// empty-response'd and all retries failed), delete the placeholder row
|
||||
// instead of persisting an empty bubble.
|
||||
if finalText == "" && len(toolCalls) == 0 && msgID != uuid.Nil {
|
||||
a.store.deleteMessage(pctx, msgID)
|
||||
} else {
|
||||
persist() // final state — same row, updated one last time
|
||||
}
|
||||
|
||||
// Title: prefer the goal once set; else the first assistant answer.
|
||||
if finalText != "" && sessionID != "ephemeral" {
|
||||
var goalTitle string
|
||||
if sess, gerr := a.store.getSession(pctx, sessionID); gerr == nil && sess.Goal != "" {
|
||||
goalTitle = truncate(sess.Goal, 120)
|
||||
}
|
||||
title := goalTitle
|
||||
if title == "" {
|
||||
title = truncate(finalText, 80)
|
||||
}
|
||||
if title != "" {
|
||||
a.store.updateSessionTitle(pctx, sessionID, title)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// drainAcquireWait is how long drainQueued blocks for a busy gate before
|
||||
// re-queuing and deferring to the holder's own release-drain. A package var so
|
||||
// tests can shorten it; in production it just needs to outlast the brief
|
||||
// release→drain handoff window.
|
||||
var drainAcquireWait = 5 * time.Second
|
||||
|
||||
// drainQueued runs every queued operator message for a session as its own turn,
|
||||
// one at a time, under the turn gate. Called (in a goroutine) whenever a turn
|
||||
// releases the gate — from handleChat (live) and resumeSession (background) —
|
||||
// so a message queued while the agent was busy is acted on as soon as it's
|
||||
// free, without the operator re-sending. See messagequeue.go (plan 2026-08-03
|
||||
// F2).
|
||||
//
|
||||
// Each queued turn is persisted incrementally and has no SSE client (the
|
||||
// browser detached after receiving the `queued` event); the frontend sees the
|
||||
// result via the 3s poller and the status-driven "working" indicator.
|
||||
func (a *agent) drainQueued(ctx context.Context, sessionID string) {
|
||||
for {
|
||||
msg, ok := a.queue.dequeue(sessionID)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
// Block briefly for the gate. If a live turn grabbed it first, put the
|
||||
// message back — that turn's release will drain it again. Never stack.
|
||||
if !a.gate.acquire(sessionID, drainAcquireWait) {
|
||||
a.queue.requeueFront(sessionID, msg)
|
||||
return
|
||||
}
|
||||
slog.Info("nomos: running queued operator message", "session", sessionID)
|
||||
pctx := context.Background()
|
||||
// Run the turn inside a per-iteration closure so the gate release is
|
||||
// deferred to the end of THIS turn (and runs even if runChatTurn
|
||||
// panics — safego recovers the panic at the goroutine boundary, so a
|
||||
// non-deferred release would be skipped and the session's permit held
|
||||
// forever, deadlocking all future turns). A bare `defer release` in
|
||||
// the loop would be wrong too: Go defers run at function exit, not
|
||||
// iteration exit, so the gate would stay held across iterations.
|
||||
func() {
|
||||
defer a.gate.release(sessionID)
|
||||
a.runChatTurn(pctx, ctx, sessionID, msg, func(agentEvent) {})
|
||||
}()
|
||||
}
|
||||
}
|
||||
|
||||
func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "method not allowed", 405)
|
||||
@@ -790,338 +693,3 @@ func truncate(s string, n int) string {
|
||||
}
|
||||
return s[:n] + "..."
|
||||
}
|
||||
|
||||
// ─── MCP Streamable HTTP client ────────────────────────────────────────
|
||||
|
||||
type mcpClient struct {
|
||||
baseURL string
|
||||
token string // OIKOS_MCP_BEARER_TOKEN — api's combinedAuth requires it on every request (no dev-open bypass)
|
||||
sessionID string
|
||||
http *http.Client
|
||||
nextID int
|
||||
mu sync.Mutex // this client is one stateful MCP session; serialize ITS OWN calls
|
||||
|
||||
// toolsCache holds the last tools/list result. The tool list is static
|
||||
// for the lifetime of one MCP connection — it only changes when the api
|
||||
// process (re)registers tools, i.e. on a restart, which this client
|
||||
// already detects and reacts to via reconnectLocked. Without this,
|
||||
// buildTools (called at the start of EVERY chat turn, including every
|
||||
// auto-continuation resume) paid a full tools/list round-trip every
|
||||
// single time for a list that's almost always identical to the last one.
|
||||
// Guarded separately from mu (not reused) so a cache check never
|
||||
// contends with an in-flight doRequest call for a different method.
|
||||
toolsMu sync.Mutex
|
||||
toolsCache []toolDef
|
||||
}
|
||||
|
||||
func newMCPClient(baseURL, token string) (*mcpClient, error) {
|
||||
c := &mcpClient{
|
||||
baseURL: baseURL,
|
||||
token: token,
|
||||
http: &http.Client{Timeout: 120 * time.Second},
|
||||
}
|
||||
|
||||
resp, err := c.doRequest("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initialize: %w", err)
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return nil, fmt.Errorf("no session ID in initialize response")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
|
||||
c.doRequest("notifications/initialized", map[string]any{})
|
||||
|
||||
slog.Info("nomos: mcp connected", "session", c.sessionID[:16]+"...")
|
||||
return c, nil
|
||||
}
|
||||
|
||||
type mcpJSONRPCResponse struct {
|
||||
sessionID string
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error json.RawMessage `json:"error"`
|
||||
}
|
||||
|
||||
// errStaleSession signals that the MCP server rejected our session id (e.g.
|
||||
// after an api/MCP restart), so the client should re-initialize and retry.
|
||||
var errStaleSession = fmt.Errorf("mcp session stale")
|
||||
|
||||
// doRequest serializes MCP calls and transparently re-initializes the session
|
||||
// if the server has forgotten it (common after an api redeploy), retrying the
|
||||
// original call once. Without this, an api restart permanently breaks nomos
|
||||
// until it is itself restarted.
|
||||
func (c *mcpClient) doRequest(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
resp, err := c.send(method, params)
|
||||
if err != nil && method != "initialize" && errors.Is(err, errStaleSession) {
|
||||
slog.Warn("nomos: mcp session stale, reconnecting")
|
||||
if rerr := c.reconnectLocked(); rerr != nil {
|
||||
return nil, fmt.Errorf("mcp reconnect: %w (original: %v)", rerr, err)
|
||||
}
|
||||
return c.send(method, params)
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
// reconnectLocked re-initializes the MCP session. The caller must hold c.mu.
|
||||
func (c *mcpClient) reconnectLocked() error {
|
||||
c.sessionID = ""
|
||||
// A reconnect means the api process was restarted (or forgot us) — its
|
||||
// tool registration may have changed, so the cached list is no longer
|
||||
// trustworthy.
|
||||
c.toolsMu.Lock()
|
||||
c.toolsCache = nil
|
||||
c.toolsMu.Unlock()
|
||||
resp, err := c.send("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return fmt.Errorf("no session ID on re-initialize")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
_, _ = c.send("notifications/initialized", map[string]any{})
|
||||
slog.Info("nomos: mcp reconnected", "session", c.sessionID[:16]+"...")
|
||||
return nil
|
||||
}
|
||||
|
||||
// send performs one MCP round-trip. It does not lock; callers hold c.mu.
|
||||
func (c *mcpClient) send(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.nextID++
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"jsonrpc": "2.0",
|
||||
"method": method,
|
||||
"params": params,
|
||||
"id": c.nextID,
|
||||
})
|
||||
|
||||
req, err := http.NewRequest(http.MethodPost, c.baseURL, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json, text/event-stream")
|
||||
if c.sessionID != "" {
|
||||
req.Header.Set("Mcp-Session-Id", c.sessionID)
|
||||
}
|
||||
if c.token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
}
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// A rejected/unknown session comes back as 4xx (commonly 400/404).
|
||||
if resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusBadRequest {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
result := &mcpJSONRPCResponse{}
|
||||
result.sessionID = resp.Header.Get("Mcp-Session-Id")
|
||||
|
||||
scanner := bufio.NewScanner(resp.Body)
|
||||
scanner.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
|
||||
gotData := false
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if strings.HasPrefix(line, "data: ") {
|
||||
gotData = true
|
||||
data := line[6:]
|
||||
if err := json.Unmarshal([]byte(data), result); err != nil {
|
||||
return nil, fmt.Errorf("parse response: %w", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if result.Error != nil {
|
||||
return nil, fmt.Errorf("rpc error: %s", string(result.Error))
|
||||
}
|
||||
|
||||
// Empty body with no result and a session set: the server likely dropped
|
||||
// our session. Notifications legitimately return no data, so exempt them.
|
||||
if !gotData && result.Result == nil && method != "notifications/initialized" {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
if result.sessionID != "" {
|
||||
c.sessionID = result.sessionID
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) callTool(name string, args map[string]any) (any, error) {
|
||||
resp, err := c.doRequest("tools/call", map[string]any{
|
||||
"name": name,
|
||||
"arguments": args,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var toolResult struct {
|
||||
Content []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &toolResult); err != nil {
|
||||
return string(resp.Result), nil
|
||||
}
|
||||
|
||||
var texts []string
|
||||
for _, c := range toolResult.Content {
|
||||
if c.Type == "text" {
|
||||
var parsed any
|
||||
if json.Unmarshal([]byte(c.Text), &parsed) == nil {
|
||||
return parsed, nil
|
||||
}
|
||||
texts = append(texts, c.Text)
|
||||
}
|
||||
}
|
||||
if len(texts) == 1 {
|
||||
return texts[0], nil
|
||||
}
|
||||
return texts, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) listTools() ([]string, error) {
|
||||
resp, err := c.doRequest("tools/list", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tr struct {
|
||||
Tools []struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
} `json:"tools"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &tr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var names []string
|
||||
for _, t := range tr.Tools {
|
||||
names = append(names, t.Name)
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) close() {
|
||||
}
|
||||
|
||||
// ─── Per-session MCP client pool ────────────────────────────────────────
|
||||
//
|
||||
// A single shared mcpClient serializes EVERY tool call across EVERY
|
||||
// concurrently-running task through one mutex (see mcpClient.mu) — `run`
|
||||
// executes its SSH command synchronously inside that lock and is capped at
|
||||
// up to 10 minutes, so one task mid-`run` stalled every other task's tool
|
||||
// calls, even trivial reads, behind it. The MCP *server* has no per-
|
||||
// connection state to protect (newServer in internal/mcp/server.go returns
|
||||
// one shared *mcp.Server instance whose tool handlers close only over the DB
|
||||
// pool, which is already safe for concurrent use) — the mutex existed purely
|
||||
// because the *client* reused one stateful transport session, not because
|
||||
// the server needed it. Giving each task's own session its own client
|
||||
// removes the cross-task serialization entirely: a task's own tool calls
|
||||
// stay sequential (which they already are — the agent loop calls tools one
|
||||
// at a time within a turn), but no longer block anyone else's.
|
||||
type mcpClientPool struct {
|
||||
baseURL string
|
||||
token string
|
||||
mu sync.Mutex
|
||||
clients map[string]*pooledMCPClient
|
||||
}
|
||||
|
||||
type pooledMCPClient struct {
|
||||
client *mcpClient
|
||||
lastUsed time.Time
|
||||
}
|
||||
|
||||
func newMCPClientPool(baseURL, token string) *mcpClientPool {
|
||||
return &mcpClientPool{baseURL: baseURL, token: token, clients: make(map[string]*pooledMCPClient)}
|
||||
}
|
||||
|
||||
// get returns the client for sessionID, creating and initializing one (a
|
||||
// real MCP handshake) on first use. Session ids that don't identify a real
|
||||
// persisted conversation ("" / "ephemeral", the no-DB-store path; "query",
|
||||
// the structured /query endpoint) still get exactly one dedicated,
|
||||
// reused client each via the same map — just keyed on a fixed string instead
|
||||
// of a real session id — so that traffic doesn't pay a fresh handshake per
|
||||
// request while still never sharing a connection with an actual task.
|
||||
func (p *mcpClientPool) get(sessionID string) (*mcpClient, error) {
|
||||
key := sessionID
|
||||
if key == "" {
|
||||
key = "ephemeral"
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
if pc, ok := p.clients[key]; ok {
|
||||
pc.lastUsed = time.Now()
|
||||
p.mu.Unlock()
|
||||
return pc.client, nil
|
||||
}
|
||||
p.mu.Unlock()
|
||||
|
||||
// Initialize outside the lock — it's a network round-trip, and holding
|
||||
// the pool mutex for it would serialize unrelated sessions' first calls
|
||||
// behind each other, undermining the whole point of this pool.
|
||||
c, err := newMCPClient(p.baseURL, p.token)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
// Another goroutine may have created one for the same key while we were
|
||||
// initializing (two of this session's tool calls racing on a cold
|
||||
// start); keep whichever won, close out the loser's connection (a no-op
|
||||
// today, but future-proof if mcpClient.close ever does real teardown).
|
||||
if existing, ok := p.clients[key]; ok {
|
||||
p.mu.Unlock()
|
||||
c.close()
|
||||
return existing.client, nil
|
||||
}
|
||||
p.clients[key] = &pooledMCPClient{client: c, lastUsed: time.Now()}
|
||||
p.mu.Unlock()
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// mcpClientIdleTimeout is how long an idle session's MCP client is kept
|
||||
// before eviction — long enough to outlive a single slow `run` (capped at 10
|
||||
// minutes server-side) plus normal think-time between a task's tool calls,
|
||||
// short enough not to accumulate one abandoned connection per finished task
|
||||
// forever.
|
||||
const mcpClientIdleTimeout = 20 * time.Minute
|
||||
|
||||
// sweep evicts clients idle past mcpClientIdleTimeout. Call on a ticker.
|
||||
func (p *mcpClientPool) sweep() {
|
||||
cutoff := time.Now().Add(-mcpClientIdleTimeout)
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
if pc.lastUsed.Before(cutoff) {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *mcpClientPool) closeAll() {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
@@ -10,7 +10,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
@@ -179,6 +179,15 @@ func (s *store) createTaskEntity(ctx context.Context, sessionID, title string) s
|
||||
`UPDATE agent_sessions SET entity_id = $1 WHERE id = $2`, entityID, sessionID); err != nil {
|
||||
slog.Warn("nomos: could not link task entity", "session", sessionID, "error", err)
|
||||
}
|
||||
// Graph edge: task —involves→ agent:nomos (gives every task at least one
|
||||
// edge from creation, even if no run calls are ever made).
|
||||
s.pool.Exec(ctx, `INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT $1, id, 'involves', '{"by":"nomos"}'::jsonb, now()
|
||||
FROM entities WHERE slug = 'agent:nomos'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM relationships r
|
||||
WHERE r.source_id = $1 AND r.target_id = entities.id AND r.type = 'involves' AND r.valid_to IS NULL)`,
|
||||
entityID)
|
||||
return entityID.String()
|
||||
}
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
152
cmd/nomos/workers.go
Normal file
152
cmd/nomos/workers.go
Normal file
@@ -0,0 +1,152 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// runChatTurn is the shared core of an operator-initiated turn: insert an
|
||||
// assistant placeholder, run a.chat with incremental persistence (so whatever
|
||||
// happened before an abort is never lost), finalize the row, and derive a
|
||||
// title. It is agnostic to the transport: `sink` receives every agent event
|
||||
// for delivery (SSE for a live handleChat, a no-op for a queued turn that has
|
||||
// no client attached — the frontend learns about those via the poller + the
|
||||
// status-driven "working" signal). The caller MUST already hold the session's
|
||||
// turn-gate permit.
|
||||
func (a *agent) runChatTurn(pctx, ctx context.Context, sessionID, message string, sink func(agentEvent)) {
|
||||
toolCalls := []map[string]any{}
|
||||
// P3: accumulate per-iteration reasoning instead of overwriting with the
|
||||
// final `text` event (see the original inline comment in handleChat).
|
||||
var textParts []string
|
||||
var thinkingParts []string
|
||||
var finalText string
|
||||
var finalThinking string
|
||||
|
||||
placeholder, _ := json.Marshal(map[string]any{"role": "assistant", "text": ""})
|
||||
msgID, err := a.store.insertMessageReturningID(pctx, sessionID, "assistant", placeholder)
|
||||
if err != nil {
|
||||
slog.Error("nomos: chat placeholder insert failed", "session", sessionID, "error", err)
|
||||
}
|
||||
persist := func() {
|
||||
if msgID == uuid.Nil {
|
||||
return
|
||||
}
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": finalText,
|
||||
"thinking": finalThinking,
|
||||
"tool_calls": toolCalls,
|
||||
})
|
||||
a.store.updateMessage(pctx, msgID, body)
|
||||
}
|
||||
|
||||
a.chat(ctx, sessionID, message, func(ev agentEvent) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
m["type"] = ev.Type
|
||||
// One entry per tool call: tool_use creates it, tool_result
|
||||
// merges the result into the same entry (matched by id).
|
||||
id, _ := m["id"].(string)
|
||||
if id != "" && ev.Type == "tool_result" {
|
||||
for _, existing := range toolCalls {
|
||||
if eID, _ := existing["id"].(string); eID == id {
|
||||
for k, v := range m {
|
||||
existing[k] = v
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
} else {
|
||||
toolCalls = append(toolCalls, m)
|
||||
}
|
||||
}
|
||||
persist() // live: survives even if the client disconnects right after
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
if t, ok := ev.Data.(string); ok && t != "" {
|
||||
if ev.IsThinking {
|
||||
thinkingParts = append(thinkingParts, t)
|
||||
finalThinking = strings.Join(thinkingParts, "\n\n")
|
||||
} else {
|
||||
textParts = append(textParts, t)
|
||||
finalText = strings.Join(textParts, "\n\n")
|
||||
}
|
||||
persist()
|
||||
}
|
||||
}
|
||||
sink(ev)
|
||||
})
|
||||
|
||||
// B.6: if the turn ended with no text and no tool calls (the model
|
||||
// empty-response'd and all retries failed), delete the placeholder row
|
||||
// instead of persisting an empty bubble.
|
||||
if finalText == "" && len(toolCalls) == 0 && msgID != uuid.Nil {
|
||||
a.store.deleteMessage(pctx, msgID)
|
||||
} else {
|
||||
persist() // final state — same row, updated one last time
|
||||
}
|
||||
|
||||
// Title: prefer the goal once set; else the first assistant answer.
|
||||
if finalText != "" && sessionID != "ephemeral" {
|
||||
var goalTitle string
|
||||
if sess, gerr := a.store.getSession(pctx, sessionID); gerr == nil && sess.Goal != "" {
|
||||
goalTitle = truncate(sess.Goal, 120)
|
||||
}
|
||||
title := goalTitle
|
||||
if title == "" {
|
||||
title = truncate(finalText, 80)
|
||||
}
|
||||
if title != "" {
|
||||
a.store.updateSessionTitle(pctx, sessionID, title)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// drainAcquireWait is how long drainQueued blocks for a busy gate before
|
||||
// re-queuing and deferring to the holder's own release-drain. A package var so
|
||||
// tests can shorten it; in production it just needs to outlast the brief
|
||||
// release→drain handoff window.
|
||||
var drainAcquireWait = 5 * time.Second
|
||||
|
||||
// drainQueued runs every queued operator message for a session as its own turn,
|
||||
// one at a time, under the turn gate. Called (in a goroutine) whenever a turn
|
||||
// releases the gate — from handleChat (live) and resumeSession (background) —
|
||||
// so a message queued while the agent was busy is acted on as soon as it's
|
||||
// free, without the operator re-sending. See messagequeue.go (plan 2026-08-03
|
||||
// F2).
|
||||
//
|
||||
// Each queued turn is persisted incrementally and has no SSE client (the
|
||||
// browser detached after receiving the `queued` event); the frontend sees the
|
||||
// result via the 3s poller and the status-driven "working" indicator.
|
||||
func (a *agent) drainQueued(ctx context.Context, sessionID string) {
|
||||
for {
|
||||
msg, ok := a.queue.dequeue(sessionID)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
// Block briefly for the gate. If a live turn grabbed it first, put the
|
||||
// message back — that turn's release will drain it again. Never stack.
|
||||
if !a.gate.acquire(sessionID, drainAcquireWait) {
|
||||
a.queue.requeueFront(sessionID, msg)
|
||||
return
|
||||
}
|
||||
slog.Info("nomos: running queued operator message", "session", sessionID)
|
||||
pctx := context.Background()
|
||||
// Run the turn inside a per-iteration closure so the gate release is
|
||||
// deferred to the end of THIS turn (and runs even if runChatTurn
|
||||
// panics — safego recovers the panic at the goroutine boundary, so a
|
||||
// non-deferred release would be skipped and the session's permit held
|
||||
// forever, deadlocking all future turns). A bare `defer release` in
|
||||
// the loop would be wrong too: Go defers run at function exit, not
|
||||
// iteration exit, so the gate would stay held across iterations.
|
||||
func() {
|
||||
defer a.gate.release(sessionID)
|
||||
a.runChatTurn(pctx, ctx, sessionID, msg, func(agentEvent) {})
|
||||
}()
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"time"
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
@@ -11,10 +12,11 @@ import (
|
||||
"syscall"
|
||||
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/core/app"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres"
|
||||
"github.com/dtoro/oikos/internal/execworker"
|
||||
"github.com/dtoro/oikos/internal/httpapi"
|
||||
"github.com/dtoro/oikos/internal/knowledge"
|
||||
"github.com/dtoro/oikos/internal/notifier"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/dtoro/oikos/internal/scheduler"
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
@@ -22,7 +24,7 @@ import (
|
||||
)
|
||||
|
||||
var schedulerRunner = scheduler.RunnerForMain()
|
||||
var notifierRunner = notifier.RunnerForMain()
|
||||
var execWorkerRunner = execworker.RunnerForMain()
|
||||
|
||||
func main() {
|
||||
if len(os.Args) < 2 {
|
||||
@@ -37,12 +39,37 @@ func main() {
|
||||
logger := observability.NewLogger(cfg.Debug)
|
||||
slog.SetDefault(logger)
|
||||
|
||||
slog.Info("starting oikos", "role", role, "config", cfg)
|
||||
|
||||
ctx, cancel := signal.NotifyContext(context.Background(),
|
||||
syscall.SIGTERM, syscall.SIGINT)
|
||||
defer cancel()
|
||||
|
||||
// Resolve secrets from Infisical, overlaying env-derived config values.
|
||||
// If Infisical is not configured, env vars are used as-is (no change).
|
||||
sec := secrets.NewManagerFromConfig(
|
||||
cfg.InfisicalSiteURL,
|
||||
cfg.InfisicalClientID,
|
||||
cfg.InfisicalClientSecret,
|
||||
cfg.InfisicalProjectID,
|
||||
cfg.InfisicalEnv,
|
||||
cfg.SecretsDir,
|
||||
)
|
||||
if sec != nil {
|
||||
overlays := secrets.ConfigOverlays(map[string]func(string){
|
||||
"mcp_bearer-token": func(v string) { cfg.MCPBearerToken = v },
|
||||
"api_token": func(v string) { cfg.APIToken = v },
|
||||
"oidc_client-secret": func(v string) { cfg.OIDCClientSecret = v },
|
||||
})
|
||||
n := secrets.OverlayConfig(ctx, sec, overlays)
|
||||
slog.Info("secrets resolved from Infisical", "count", n)
|
||||
|
||||
secrets.VerifyExpectedSecrets(ctx, sec, []string{
|
||||
"approval_hmac-secret", "mcp_bearer-token",
|
||||
"api_token", "openrouter_api-key", "webhook_hmac-secret",
|
||||
})
|
||||
}
|
||||
|
||||
slog.Info("starting oikos", "role", role, "config", cfg)
|
||||
|
||||
switch role {
|
||||
case "migrate":
|
||||
if err := runMigrate(ctx, cfg); err != nil {
|
||||
@@ -66,8 +93,8 @@ func main() {
|
||||
}
|
||||
case "scheduler":
|
||||
runWithPool(ctx, cfg, "scheduler", schedulerRunner)
|
||||
case "notifier":
|
||||
runWithPool(ctx, cfg, "notifier", notifierRunner)
|
||||
case "execution-worker":
|
||||
runWithPool(ctx, cfg, "execution-worker", execWorkerRunner)
|
||||
case "all":
|
||||
pool, err := db.New(ctx, cfg.DatabaseURL)
|
||||
if err != nil {
|
||||
@@ -82,10 +109,17 @@ func main() {
|
||||
}
|
||||
|
||||
go schedulerRunner(ctx, pool, cfg)
|
||||
go notifierRunner(ctx, pool, cfg)
|
||||
go execWorkerRunner(ctx, pool, cfg)
|
||||
|
||||
slog.Info("all: starting api with scheduler + notifier in background")
|
||||
if err := httpapi.ListenAndServe(ctx, pool, cfg); err != nil {
|
||||
// Build composition-root dependencies (ADR 0016).
|
||||
entityRepo := db.NewEntityRepo(pool)
|
||||
onto := db.NewOntologyRepo(pool, time.Minute)
|
||||
readModels := db.NewEntityReader(pool)
|
||||
entities := app.NewEntityService(entityRepo, onto)
|
||||
relService := app.NewRelationshipService(db.NewRelRepo(pool), onto)
|
||||
|
||||
slog.Info("all: starting api with scheduler + execution-worker in background")
|
||||
if err := httpapi.ListenAndServe(ctx, pool, cfg, entities, entityRepo, readModels, relService); err != nil {
|
||||
slog.Error("api failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
@@ -113,9 +147,8 @@ Roles:
|
||||
export Export DB state back to seed YAMLs (DR / version control)
|
||||
api Run the REST + MCP API server
|
||||
scheduler Run the observe loop
|
||||
notifier Run the notification service (Matrix alerts)
|
||||
all Run all roles in one process (dev mode)
|
||||
secret Secret management (Infisical)
|
||||
secret Secret management (Infisical: get, set, list, verify, audit, migrate, export-sops)
|
||||
knowledge Convert wiki to knowledge seed (one-shot)
|
||||
version Print version info
|
||||
|
||||
@@ -264,7 +297,14 @@ func runAPI(ctx context.Context, cfg config.Config) error {
|
||||
return fmt.Errorf("migrations: %w", err)
|
||||
}
|
||||
|
||||
err = httpapi.ListenAndServe(ctx, pool, cfg)
|
||||
// Composition root — build the service dependencies (ADR 0016, plan §3.5).
|
||||
entityRepo := db.NewEntityRepo(pool)
|
||||
onto := db.NewOntologyRepo(pool, time.Minute)
|
||||
readModels := db.NewEntityReader(pool)
|
||||
entities := app.NewEntityService(entityRepo, onto)
|
||||
relService := app.NewRelationshipService(db.NewRelRepo(pool), onto)
|
||||
|
||||
err = httpapi.ListenAndServe(ctx, pool, cfg, entities, entityRepo, readModels, relService)
|
||||
if err == http.ErrServerClosed {
|
||||
return nil
|
||||
}
|
||||
@@ -289,20 +329,45 @@ func runWithPool(ctx context.Context, cfg config.Config, name string, fn func(co
|
||||
|
||||
func runSecret(ctx context.Context, cfg config.Config) {
|
||||
if len(os.Args) < 3 {
|
||||
fmt.Fprintln(os.Stderr, "usage: oikos secret <list|migrate|export-sops>")
|
||||
fmt.Fprintln(os.Stderr, "usage: oikos secret <get|set|list|migrate|export-sops>")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
sub := os.Args[2]
|
||||
secretsDir := cfg.SecretsDir
|
||||
if secretsDir == "" {
|
||||
secretsDir = "archive/secrets-sops-backup"
|
||||
}
|
||||
sopsBackend := secrets.NewSOPSBackend(secretsDir)
|
||||
|
||||
// For get/set/list: use Infisical directly
|
||||
switch sub {
|
||||
case "get":
|
||||
if len(os.Args) < 4 {
|
||||
fmt.Fprintln(os.Stderr, "usage: oikos secret get <key>")
|
||||
os.Exit(1)
|
||||
}
|
||||
key := os.Args[3]
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
val, err := backend.Get(ctx, key)
|
||||
if err != nil {
|
||||
slog.Error("secret get", "key", key, "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Println(val)
|
||||
|
||||
case "set":
|
||||
if len(os.Args) < 5 {
|
||||
fmt.Fprintln(os.Stderr, "usage: oikos secret set <key> <value>")
|
||||
os.Exit(1)
|
||||
}
|
||||
key := os.Args[3]
|
||||
value := os.Args[4]
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
if err := backend.Set(ctx, key, value); err != nil {
|
||||
slog.Error("secret set", "key", key, "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("stored: %s\n", key)
|
||||
|
||||
case "list":
|
||||
keys, err := sopsBackend.List(ctx)
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
keys, err := backend.List(ctx)
|
||||
if err != nil {
|
||||
slog.Error("secret list", "error", err)
|
||||
os.Exit(1)
|
||||
@@ -311,24 +376,134 @@ func runSecret(ctx context.Context, cfg config.Config) {
|
||||
fmt.Println(k)
|
||||
}
|
||||
|
||||
case "migrate":
|
||||
infCfg := secrets.InfisicalConfig{
|
||||
SiteURL: cfg.InfisicalSiteURL,
|
||||
ClientID: cfg.InfisicalClientID,
|
||||
ClientSecret: cfg.InfisicalClientSecret,
|
||||
ProjectID: cfg.InfisicalProjectID,
|
||||
SecretPath: "/",
|
||||
Env: cfg.InfisicalEnv,
|
||||
}
|
||||
if infCfg.Env == "" {
|
||||
infCfg.Env = "dev"
|
||||
}
|
||||
if infCfg.SiteURL == "" {
|
||||
fmt.Fprintln(os.Stderr, "error: OIKOS_INFISICAL_SITE_URL not set")
|
||||
os.Exit(1)
|
||||
}
|
||||
case "verify":
|
||||
runSecretVerify(ctx, cfg)
|
||||
|
||||
infBackend := secrets.NewInfisicalBackend(infCfg)
|
||||
case "audit":
|
||||
runSecretAudit(ctx, cfg)
|
||||
|
||||
case "migrate", "export-sops":
|
||||
runSecretLegacy(ctx, cfg, sub)
|
||||
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "unknown secret command: %s\n", sub)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// expectedSecrets is the set of keys that should exist in Infisical
|
||||
// for a fully-migrated deployment.
|
||||
var expectedSecrets = []string{
|
||||
"mcp_bearer-token",
|
||||
"api_token",
|
||||
"openrouter_api-key",
|
||||
"webhook_hmac-secret",
|
||||
}
|
||||
|
||||
// runSecretVerify checks that all expected secrets are present in Infisical.
|
||||
func runSecretVerify(ctx context.Context, cfg config.Config) {
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
keys, err := backend.List(ctx)
|
||||
if err != nil {
|
||||
slog.Error("verify: list", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
keySet := make(map[string]struct{}, len(keys))
|
||||
for _, k := range keys {
|
||||
keySet[k] = struct{}{}
|
||||
}
|
||||
|
||||
missing := 0
|
||||
for _, exp := range expectedSecrets {
|
||||
if _, ok := keySet[exp]; !ok {
|
||||
fmt.Printf("MISSING: %s\n", exp)
|
||||
missing++
|
||||
} else {
|
||||
fmt.Printf("OK: %s\n", exp)
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Printf("\n%d/%d present, %d missing\n", len(expectedSecrets)-missing, len(expectedSecrets), missing)
|
||||
if missing > 0 {
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// runSecretAudit resolves all expected secrets from Infisical and prints
|
||||
// a diff against current env-derived values. Values are truncated for safety.
|
||||
func runSecretAudit(ctx context.Context, cfg config.Config) {
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
|
||||
envValues := map[string]string{
|
||||
"mcp_bearer-token": cfg.MCPBearerToken,
|
||||
"api_token": cfg.APIToken,
|
||||
"oidc_client-secret": cfg.OIDCClientSecret,
|
||||
}
|
||||
|
||||
fmt.Println("key infisical env status")
|
||||
fmt.Println(strings.Repeat("-", 72))
|
||||
|
||||
for _, key := range expectedSecrets {
|
||||
infVal, infErr := backend.Get(ctx, key)
|
||||
envVal := envValues[key]
|
||||
|
||||
if infErr != nil {
|
||||
fmt.Printf("%-29s ERROR %-10s NOT-IN-INFISICAL\n", key, trunc(envVal, 8))
|
||||
continue
|
||||
}
|
||||
if envVal == "" {
|
||||
fmt.Printf("%-29s %-10s (empty) INFISICAL-ONLY\n", key, trunc(infVal, 8))
|
||||
continue
|
||||
}
|
||||
if infVal == envVal {
|
||||
fmt.Printf("%-29s %-10s %-10s MATCH\n", key, trunc(infVal, 8), trunc(envVal, 8))
|
||||
} else {
|
||||
fmt.Printf("%-29s %-10s %-10s DRIFT\n", key, trunc(infVal, 8), trunc(envVal, 8))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func trunc(s string, n int) string {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
if n > 1 {
|
||||
return s[:n-1] + "…"
|
||||
}
|
||||
return s[:n]
|
||||
}
|
||||
|
||||
// newInfisicalBackendOrFail creates an Infisical backend from config or exits.
|
||||
func newInfisicalBackendOrFail(cfg config.Config) *secrets.InfisicalBackend {
|
||||
if cfg.InfisicalSiteURL == "" {
|
||||
fmt.Fprintln(os.Stderr, "error: OIKOS_INFISICAL_SITE_URL not set")
|
||||
os.Exit(1)
|
||||
}
|
||||
infCfg := secrets.InfisicalConfig{
|
||||
SiteURL: cfg.InfisicalSiteURL,
|
||||
ClientID: cfg.InfisicalClientID,
|
||||
ClientSecret: cfg.InfisicalClientSecret,
|
||||
ProjectID: cfg.InfisicalProjectID,
|
||||
SecretPath: "/",
|
||||
Env: cfg.InfisicalEnv,
|
||||
}
|
||||
if infCfg.Env == "" {
|
||||
infCfg.Env = "dev"
|
||||
}
|
||||
return secrets.NewInfisicalBackend(infCfg)
|
||||
}
|
||||
|
||||
// runSecretLegacy handles SOPS-only commands (migrate, export-sops).
|
||||
func runSecretLegacy(ctx context.Context, cfg config.Config, sub string) {
|
||||
secretsDir := cfg.SecretsDir
|
||||
if secretsDir == "" {
|
||||
secretsDir = "archive/secrets-sops-backup"
|
||||
}
|
||||
sopsBackend := secrets.NewSOPSBackend(secretsDir)
|
||||
|
||||
switch sub {
|
||||
case "migrate":
|
||||
infBackend := newInfisicalBackendOrFail(cfg)
|
||||
keys, err := sopsBackend.List(ctx)
|
||||
if err != nil {
|
||||
slog.Error("migrate: read sops", "error", err)
|
||||
@@ -366,10 +541,6 @@ func runSecret(ctx context.Context, cfg config.Config) {
|
||||
fmt.Printf("%s: <sops-encrypted>\n", k)
|
||||
}
|
||||
fmt.Printf("\n# To restore: sops -d secrets/*.yaml\n")
|
||||
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "unknown secret command: %s\n", sub)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
@@ -12,26 +13,31 @@ import (
|
||||
"os/exec"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
)
|
||||
|
||||
func main() {
|
||||
ctx := context.Background()
|
||||
|
||||
port := os.Getenv("WEBHOOK_LISTEN")
|
||||
if port == "" {
|
||||
port = ":9797"
|
||||
}
|
||||
|
||||
secret := os.Getenv("WEBHOOK_HMAC_SECRET")
|
||||
if secret == "" {
|
||||
fmt.Fprintln(os.Stderr, "WEBHOOK_HMAC_SECRET must be set")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
repoDir := os.Getenv("WEBHOOK_REPO_DIR")
|
||||
if repoDir == "" {
|
||||
repoDir = os.Getenv("HOME") + "/Projects/oikos"
|
||||
}
|
||||
|
||||
// Create secrets manager once, share between HMAC resolution and deploy
|
||||
sec := newSecrets()
|
||||
secret := resolveWebhookHMAC(ctx, sec)
|
||||
if secret == "" {
|
||||
fmt.Fprintln(os.Stderr, "WEBHOOK_HMAC_SECRET must be set (env var or Infisical webhook_hmac-secret)")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/deploy", func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
@@ -66,11 +72,16 @@ func main() {
|
||||
w.Write([]byte(`{"status":"deploy started"}`))
|
||||
|
||||
safego.Go("webhook:deploy", func() {
|
||||
apiToken := ""
|
||||
if sec != nil {
|
||||
apiToken = secrets.ResolveSecret(ctx, sec, "api_token", "")
|
||||
}
|
||||
cmd := exec.Command(repoDir + "/scripts/deploy.sh")
|
||||
cmd.Dir = repoDir
|
||||
cmd.Env = append(os.Environ(),
|
||||
"REPO_DIR="+repoDir,
|
||||
"PROFILE=full",
|
||||
"OIKOS_API_TOKEN="+apiToken,
|
||||
)
|
||||
cmd.Stdout = os.Stdout
|
||||
cmd.Stderr = os.Stderr
|
||||
@@ -94,3 +105,25 @@ func main() {
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// newSecrets creates the Infisical secrets manager from env vars.
|
||||
func newSecrets() *secrets.Manager {
|
||||
return secrets.NewManagerFromConfig(
|
||||
os.Getenv("OIKOS_INFISICAL_SITE_URL"),
|
||||
os.Getenv("OIKOS_INFISICAL_CLIENT_ID"),
|
||||
os.Getenv("OIKOS_INFISICAL_CLIENT_SECRET"),
|
||||
os.Getenv("OIKOS_INFISICAL_PROJECT_ID"),
|
||||
os.Getenv("OIKOS_INFISICAL_ENV"),
|
||||
os.Getenv("OIKOS_SECRETS_DIR"),
|
||||
)
|
||||
}
|
||||
|
||||
// resolveWebhookHMAC fetches the webhook HMAC secret from Infisical,
|
||||
// falling back to the WEBHOOK_HMAC_SECRET env var.
|
||||
func resolveWebhookHMAC(ctx context.Context, sec *secrets.Manager) string {
|
||||
envFallback := os.Getenv("WEBHOOK_HMAC_SECRET")
|
||||
if sec == nil {
|
||||
return envFallback
|
||||
}
|
||||
return secrets.ResolveSecret(ctx, sec, "webhook_hmac-secret", envFallback)
|
||||
}
|
||||
@@ -1,27 +0,0 @@
|
||||
:80 {
|
||||
root * /srv
|
||||
|
||||
# /wails/runtime.js is injected by the Wails desktop wrapper, which serves
|
||||
# the same dist/ from its own asset handler. In a browser it does not
|
||||
# exist, and the SPA fallback below answered it with index.html — so the
|
||||
# browser parsed "<!doctype html>" as JavaScript and threw
|
||||
# "SyntaxError: expected expression, got '<'" on every page load.
|
||||
# Return a real 404 instead: the tag fails quietly, and the desktop app is
|
||||
# unaffected because it never reaches this server.
|
||||
handle /wails/* {
|
||||
error 404
|
||||
}
|
||||
|
||||
# Same reasoning for any other asset: a missing .js/.css/.map answered with
|
||||
# HTML is always a confusing parse error rather than an honest 404. Only
|
||||
# real routes should fall through to the SPA.
|
||||
@asset path_regexp \.(js|mjs|css|map|json|png|jpg|svg|ico|woff2?)$
|
||||
handle @asset {
|
||||
file_server
|
||||
}
|
||||
|
||||
handle {
|
||||
file_server
|
||||
try_files {path} /index.html
|
||||
}
|
||||
}
|
||||
@@ -1,19 +0,0 @@
|
||||
# Dockerfile for the oikos control-room SPA. Built separately from the
|
||||
# oikos binary (compose/oikos/Dockerfile) — see docker-compose.yml's `web`
|
||||
# service. The outer production Caddy (caddy-conf repo, LXC 121) handles
|
||||
# Authentik + splits /api/*, /mcp, /agent/* off to the api service; this
|
||||
# container only serves static files with SPA-fallback routing.
|
||||
|
||||
FROM node:22-alpine AS builder
|
||||
|
||||
WORKDIR /build/web
|
||||
COPY web/package.json web/package-lock.json ./
|
||||
RUN npm ci
|
||||
COPY VERSION ./
|
||||
COPY web/ ./
|
||||
RUN npm run build
|
||||
|
||||
FROM caddy:2-alpine
|
||||
|
||||
COPY --from=builder /build/web/dist /srv
|
||||
COPY compose/web/Caddyfile /etc/caddy/Caddyfile
|
||||
@@ -20,6 +20,8 @@ services:
|
||||
- "5432:5432"
|
||||
volumes:
|
||||
- pg-data:/var/lib/postgresql/data
|
||||
mem_limit: 1g
|
||||
cpus: 2.0
|
||||
healthcheck:
|
||||
test: ["CMD", "pg_isready", "-U", "oikos"]
|
||||
interval: 5s
|
||||
@@ -28,6 +30,7 @@ services:
|
||||
|
||||
# One-shot: run migrations then exit
|
||||
migrate:
|
||||
image: oikos-migrate:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -38,9 +41,12 @@ services:
|
||||
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
|
||||
command: ["migrate"]
|
||||
restart: "no"
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
|
||||
# One-shot: ingest seeds then exit
|
||||
seed:
|
||||
image: oikos-seed:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -52,9 +58,12 @@ services:
|
||||
OIKOS_SEEDS_DIR: /seeds
|
||||
command: ["seed"]
|
||||
restart: "no"
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
|
||||
# API server (Phase 2)
|
||||
api:
|
||||
image: oikos-api:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -76,6 +85,16 @@ services:
|
||||
OIKOS_OIDC_CLIENT_ID: ${OIKOS_OIDC_CLIENT_ID:-otkHBSueHJsYtOHstL6rn5izeGgyOsavp1qA1hod}
|
||||
OIKOS_NOMOS_AGENT_SLUG: ${OIKOS_NOMOS_AGENT_SLUG:-agent:nomos}
|
||||
NOMOS_PROXY_URL: http://nomos:8092
|
||||
# Rate limiting (plan D3). Default off; set OIKOS_API_RATE_LIMIT to a
|
||||
# requests/sec value to throttle runaway agent loops per source IP.
|
||||
OIKOS_API_RATE_LIMIT: ${OIKOS_API_RATE_LIMIT:-}
|
||||
OIKOS_API_RATE_BURST: ${OIKOS_API_RATE_BURST:-}
|
||||
# Infisical secret store (Phase 5)
|
||||
OIKOS_INFISICAL_SITE_URL: ${OIKOS_INFISICAL_SITE_URL:-}
|
||||
OIKOS_INFISICAL_CLIENT_ID: ${OIKOS_INFISICAL_CLIENT_ID:-}
|
||||
OIKOS_INFISICAL_CLIENT_SECRET: ${OIKOS_INFISICAL_CLIENT_SECRET:-}
|
||||
OIKOS_INFISICAL_PROJECT_ID: ${OIKOS_INFISICAL_PROJECT_ID:-}
|
||||
OIKOS_INFISICAL_ENV: ${OIKOS_INFISICAL_ENV:-dev}
|
||||
volumes:
|
||||
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
|
||||
ports:
|
||||
@@ -83,6 +102,8 @@ services:
|
||||
command: ["api"]
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 30s
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
# Exists so nomos can wait for the API to actually answer rather than just
|
||||
# for its container to exist — see nomos's depends_on below. wget is
|
||||
# BusyBox's, already in the alpine runtime image, so this adds no
|
||||
@@ -94,10 +115,15 @@ services:
|
||||
retries: 10
|
||||
# Migrations and seed run before this container, but the first bind can
|
||||
# still take a moment; failures inside the start period don't count.
|
||||
start_period: 10s
|
||||
# The api's NewHandler stalls on TWO unreachable external deps at startup
|
||||
# before binding :8090: Infisical (4x auth retries, ~40s) and OIDC
|
||||
# discovery (auth.hubris.network, ~35s of timeouts). Total ~90-95s, so
|
||||
# the start period must clear it or nomos (depends_on: api-healthy) fails.
|
||||
start_period: 180s
|
||||
|
||||
# Scheduler (Phase 3) — observe loop
|
||||
scheduler:
|
||||
image: oikos-scheduler:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -112,6 +138,9 @@ services:
|
||||
OIKOS_SCHEDULER_INTERVAL: "30s"
|
||||
OIKOS_SSH_KEY_PATH: /etc/oikos/ssh_key
|
||||
OIKOS_SSH_USER: root
|
||||
# Liveness probe (plan D5): exposes a staleness-aware /healthz inside
|
||||
# the container; the scheduler bumps it each check pass.
|
||||
OIKOS_HEALTH_LISTEN: ":8093"
|
||||
volumes:
|
||||
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
|
||||
cap_add:
|
||||
@@ -119,9 +148,18 @@ services:
|
||||
command: ["scheduler"]
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 30s
|
||||
mem_limit: 256m
|
||||
cpus: 1.0
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8093/healthz"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 90s
|
||||
|
||||
# Notifier (Phase 3) — Matrix alerts
|
||||
notifier:
|
||||
# Execution worker (Phase 6) — Postgres-backed job queue
|
||||
execution-worker:
|
||||
image: oikos-execution-worker:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -133,17 +171,26 @@ services:
|
||||
environment:
|
||||
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
|
||||
OIKOS_DEBUG: "true"
|
||||
OIKOS_APPROVAL_HMAC_SECRET: ${OIKOS_APPROVAL_HMAC_SECRET:-dev-secret}
|
||||
OIKOS_MATRIX_HOMESERVER: ${OIKOS_MATRIX_HOMESERVER:-https://matrix.hubris.network}
|
||||
OIKOS_MATRIX_USER: ${OIKOS_MATRIX_USER:-@hermes:hubris.network}
|
||||
OIKOS_MATRIX_TOKEN: ${OIKOS_MATRIX_TOKEN}
|
||||
OIKOS_MATRIX_ROOM: ${OIKOS_MATRIX_ROOM:-!alerts:hubris.network}
|
||||
command: ["notifier"]
|
||||
OIKOS_SSH_KEY_PATH: /etc/oikos/ssh_key
|
||||
OIKOS_SSH_USER: root
|
||||
OIKOS_HEALTH_LISTEN: ":8095"
|
||||
volumes:
|
||||
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
|
||||
command: ["execution-worker"]
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 30s
|
||||
mem_limit: 256m
|
||||
cpus: 1.0
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8095/healthz"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 90s
|
||||
|
||||
# Nomos agent gateway (Phase 4) — mesh-published :8092
|
||||
nomos:
|
||||
image: oikos-nomos:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/nomos/Dockerfile
|
||||
@@ -166,24 +213,32 @@ services:
|
||||
# Must match api's OIKOS_MCP_BEARER_TOKEN above — api's combinedAuth
|
||||
# rejects every request without it now (no dev-open bypass).
|
||||
OIKOS_MCP_BEARER_TOKEN: ${OIKOS_MCP_BEARER_TOKEN:-dev-token}
|
||||
# Infisical secret store (Phase 5) — nomos resolves mcp_bearer-token
|
||||
# and openrouter_api-key from here, overriding the env values above.
|
||||
OIKOS_INFISICAL_SITE_URL: ${OIKOS_INFISICAL_SITE_URL:-}
|
||||
OIKOS_INFISICAL_CLIENT_ID: ${OIKOS_INFISICAL_CLIENT_ID:-}
|
||||
OIKOS_INFISICAL_CLIENT_SECRET: ${OIKOS_INFISICAL_CLIENT_SECRET:-}
|
||||
OIKOS_INFISICAL_PROJECT_ID: ${OIKOS_INFISICAL_PROJECT_ID:-}
|
||||
OIKOS_INFISICAL_ENV: ${OIKOS_INFISICAL_ENV:-dev}
|
||||
ports:
|
||||
- "8092:8092"
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 10s
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
# nomos runs on a distroless image (no shell/wget), so the healthcheck
|
||||
# uses the binary's own `healthcheck` subcommand to self-probe /healthz.
|
||||
healthcheck:
|
||||
test: ["CMD", "/nomos", "healthcheck"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 15s
|
||||
|
||||
# Control-room SPA — static build served behind Caddy. The outer
|
||||
# production Caddy (caddy-conf repo, LXC 121) splits /api/*, /mcp,
|
||||
# /agent/* off to api:8090 and sends everything else here; this
|
||||
# container only serves static files with SPA-fallback routing.
|
||||
web:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/web/Dockerfile
|
||||
restart: unless-stopped
|
||||
profiles: ["dev", "full"]
|
||||
ports:
|
||||
- "8091:80"
|
||||
stop_signal: SIGTERM
|
||||
# The control-room SPA moved to its own repo (dtoro/oikos-web, Phase 1 of
|
||||
# plans/2026-08-15-hexagonal-architecture.md) with its own compose project
|
||||
# publishing the same host port 8091 — the outer Caddy targets the published
|
||||
# port, so nothing here changes for routing.
|
||||
|
||||
# Redis (required by Infisical — Phase 5)
|
||||
redis:
|
||||
@@ -192,6 +247,8 @@ services:
|
||||
profiles: ["infisical", "full"]
|
||||
volumes:
|
||||
- redis-data:/data
|
||||
mem_limit: 128m
|
||||
cpus: 0.5
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 5s
|
||||
@@ -200,7 +257,7 @@ services:
|
||||
|
||||
# Infisical self-hosted (Phase 5 secrets management)
|
||||
infisical:
|
||||
image: infisical/infisical:latest
|
||||
image: infisical/infisical:v0.162.19
|
||||
restart: unless-stopped
|
||||
profiles: ["infisical", "full"]
|
||||
depends_on:
|
||||
@@ -222,6 +279,8 @@ services:
|
||||
REDIS_URL: redis://redis:6379
|
||||
ports:
|
||||
- "8080:8080"
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
|
||||
volumes:
|
||||
pg-data:
|
||||
|
||||
69
docs/adr/0016-hexagonal-ports-adapters.md
Normal file
69
docs/adr/0016-hexagonal-ports-adapters.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# ADR 0016 — Hexagonal (ports & adapters) architecture for the oikos backend
|
||||
|
||||
Status: accepted (2026-08-15) · Plan: plans/2026-08-15-hexagonal-architecture.md
|
||||
|
||||
## Context
|
||||
The oikos backend grew as delivery-layer packages with business logic inside
|
||||
them. `internal/httpapi` (~3500 LOC) and `internal/mcp` (~2100 LOC, 67 tools)
|
||||
each embed raw SQL next to validation, policy, and audit writes — two parallel
|
||||
silos re-implementing the same use-cases. The scheduler is a ~1100-LOC
|
||||
monolith mixing probe dispatch, health aggregation, signal transitions, and
|
||||
metric writes. There is one accidental port/adapter pair
|
||||
(`secrets.Backend`) and one inverted dependency (`internal/db` imports
|
||||
`checkdefaults`). Use-cases have no home: every new consumer (REST handler,
|
||||
MCP tool, scheduler pass) copy-pastes query + policy + audit logic, and
|
||||
behavior drifts between paths that must agree — the run/approve/execute
|
||||
lifecycle exists in three variants.
|
||||
|
||||
A staff-level review of the refactor plan settled the open questions: one
|
||||
hexagon (not per-binary), nomos stays an external agent client over the wire,
|
||||
aggregate-scoped repository methods instead of a UnitOfWork port, reads bypass
|
||||
services via a shared `ReadModels` port, and the client (web SPA + desktop)
|
||||
moves to its own repository before the backend churn starts.
|
||||
|
||||
## Decision
|
||||
Adopt ports & adapters across the backend, delivered in ten shippable phases:
|
||||
|
||||
- `internal/core/domain` (pure model, stdlib only), `internal/core/app`
|
||||
(application services = use-cases), `internal/core/ports` (driven-port
|
||||
interfaces). Core imports nothing from adapters; adapters and `cmd/*`
|
||||
import core.
|
||||
- All I/O behind named ports implemented by adapters under
|
||||
`internal/adapters/`: postgres (repositories), ssh (CommandExecutor),
|
||||
probes/* (one Checker per check kind), remote (TargetResolver), secrets,
|
||||
events (EventPublisher/SSE), plus the driving adapters httpapi, mcpserver,
|
||||
scheduler, execworker, cli.
|
||||
- One repository method = one transaction = one aggregate's atomic boundary
|
||||
(`§3.6` of the plan). Inputs carry audit/event entries and derived checks;
|
||||
no `pgx.Tx` crosses into core; no UnitOfWork port.
|
||||
- Commands flow through `core/app` services; invariant-free reads flow
|
||||
adapter → `ReadModels` → presenter with no service hop (CQRS-lite).
|
||||
- REST and MCP keep their existing wire formats and become thin driving
|
||||
adapters over the same services; presenters stay per-adapter.
|
||||
- nomos never links core: it consumes oikos exclusively through MCP/REST.
|
||||
Its Phase 8 cleanup defines nomos-local ports (`LLMClient`, `HomelabClient`,
|
||||
`SessionStore`) outside `core/ports`.
|
||||
- The Gitea `webhook` receiver and the desktop shell are leaf utilities —
|
||||
documented, not restructured.
|
||||
- Dependency rules are enforced with `depguard` from Phase 0 (rule 3, the
|
||||
`internal/nomos` bans, activates when that package exists in Phase 8).
|
||||
|
||||
## Consequences
|
||||
- No API, MCP-tool, DB-schema, or wire-format changes ride along; behavior
|
||||
parity is guarded by the existing handler contract tests plus a repository
|
||||
conformance suite asserting atomicity and check-then-act per command method.
|
||||
- `internal/db` shrinks to connection/migrations/sqlcgen inside the postgres
|
||||
adapter; the `db → checkdefaults` inversion disappears with `SeedService`.
|
||||
- Each phase merges green (`make lint test generate-check`, `make test-db`
|
||||
where repositories change) with a patch version bump; per-phase abort
|
||||
criteria revert a merge that breaks transaction semantics. Minor bump at
|
||||
Phase 9 completion.
|
||||
- Cost accepted: ~10 phases of import churn, sqlc path updates under
|
||||
`make generate-check`, and a temporary period where old and new package
|
||||
locations coexist (depguard denies core→adapter imports from Phase 0 so
|
||||
the new tree can never grow the old inversions).
|
||||
- Risk trade-off recorded in the plan: a repository method whose tx span is
|
||||
too narrow is a bug class this design makes possible; the conformance
|
||||
suite is the mitigation, and the check-then-act sites enumerated in the
|
||||
plan (approval decide, entity transition, check derivation, execution
|
||||
claim) are its first assertions.
|
||||
@@ -1,5 +1,9 @@
|
||||
# Oikos — Desktop Mascot Subsystem Model
|
||||
|
||||
> **Path note (2026-08-15):** the `web/src/...` paths and relative links
|
||||
> below predate the Phase 1 client extraction — that tree now lives in the
|
||||
> `dtoro/oikos-web` repo. Read them as `web/src/...` under that checkout.
|
||||
|
||||
> Companion to [the platform Model](../mbse/README.md) and
|
||||
> [the Framework](../mbse/framework.md). This document is a **subsystem
|
||||
> Model** in Holt's sense — it conforms to the same Framework (Ontology +
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# Oikos — System Model
|
||||
|
||||
> **Path note (2026-08-15):** `web/`, `cmd/desktop`, and
|
||||
> `compose/web/Dockerfile` references below moved to the `dtoro/oikos-web`
|
||||
> repo in the Phase 1 client extraction
|
||||
> (plans/2026-08-15-hexagonal-architecture.md).
|
||||
|
||||
A single Model-Based Systems Engineering (MBSE) view of Oikos, structured
|
||||
after *Systems Engineering Demystified* (2nd ed., Jon Holt): one underlying
|
||||
system — the Oikos entity graph, its OODA control loop, and the services
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# Oikos — Component Views
|
||||
|
||||
> **Path note (2026-08-15):** `web/src/...` and `cmd/desktop` references
|
||||
> below moved to the `dtoro/oikos-web` repo in the Phase 1 client
|
||||
> extraction (plans/2026-08-15-hexagonal-architecture.md).
|
||||
|
||||
> Companion to [the Model](README.md) and [the Framework](framework.md).
|
||||
> Where README.md's nine Views cut across the whole system by *concern*
|
||||
> (requirements, behavior, risk...), this document cuts across it by
|
||||
|
||||
13
go.mod
13
go.mod
@@ -14,11 +14,10 @@ require (
|
||||
github.com/modelcontextprotocol/go-sdk v1.6.1
|
||||
github.com/oapi-codegen/runtime v1.4.2
|
||||
github.com/openai/openai-go v1.12.0
|
||||
github.com/wailsapp/wails/v3 v3.0.0-alpha2.117
|
||||
github.com/zalando/go-keyring v0.2.8
|
||||
golang.org/x/crypto v0.53.0
|
||||
golang.org/x/sync v0.21.0
|
||||
golang.org/x/sys v0.46.0
|
||||
golang.org/x/time v0.14.0
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
|
||||
@@ -27,7 +26,6 @@ require (
|
||||
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
|
||||
cloud.google.com/go/compute/metadata v0.9.0 // indirect
|
||||
cloud.google.com/go/iam v1.1.11 // indirect
|
||||
github.com/adrg/xdg v0.5.3 // indirect
|
||||
github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
|
||||
github.com/aws/aws-sdk-go-v2 v1.27.2 // indirect
|
||||
github.com/aws/aws-sdk-go-v2/config v1.27.18 // indirect
|
||||
@@ -43,16 +41,13 @@ require (
|
||||
github.com/aws/aws-sdk-go-v2/service/sts v1.28.12 // indirect
|
||||
github.com/aws/smithy-go v1.20.2 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/coder/websocket v1.8.14 // indirect
|
||||
github.com/danieljoos/wincred v1.2.3 // indirect
|
||||
github.com/dlclark/regexp2 v1.11.5 // indirect
|
||||
github.com/felixge/httpsnoop v1.0.4 // indirect
|
||||
github.com/go-logr/logr v1.4.3 // indirect
|
||||
github.com/go-logr/stdr v1.2.2 // indirect
|
||||
github.com/go-ole/go-ole v1.3.0 // indirect
|
||||
github.com/go-openapi/jsonpointer v0.22.5 // indirect
|
||||
github.com/go-openapi/swag/jsonname v0.25.5 // indirect
|
||||
github.com/go-resty/resty/v2 v2.13.1 // indirect
|
||||
github.com/godbus/dbus/v5 v5.2.2 // indirect
|
||||
github.com/gofrs/flock v0.8.1 // indirect
|
||||
github.com/google/s2a-go v0.1.9 // indirect
|
||||
github.com/googleapis/enterprise-certificate-proxy v0.3.11 // indirect
|
||||
@@ -61,9 +56,6 @@ require (
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||
github.com/jackc/puddle/v2 v2.2.2 // indirect
|
||||
github.com/jchv/go-winloader v0.0.0-20250406163304-c1995be93bd1 // indirect
|
||||
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/oasdiff/yaml v0.1.0 // indirect
|
||||
github.com/oasdiff/yaml3 v0.0.13 // indirect
|
||||
github.com/oracle/oci-go-sdk/v65 v65.95.2 // indirect
|
||||
@@ -88,7 +80,6 @@ require (
|
||||
golang.org/x/net v0.55.0 // indirect
|
||||
golang.org/x/oauth2 v0.35.0 // indirect
|
||||
golang.org/x/text v0.38.0 // indirect
|
||||
golang.org/x/time v0.14.0 // indirect
|
||||
google.golang.org/api v0.267.0 // indirect
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409 // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 // indirect
|
||||
|
||||
29
go.sum
29
go.sum
@@ -7,8 +7,6 @@ cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCB
|
||||
cloud.google.com/go/iam v1.1.11 h1:0mQ8UKSfdHLut6pH9FM3bI55KWR46ketn0PuXleDyxw=
|
||||
cloud.google.com/go/iam v1.1.11/go.mod h1:biXoiLWYIKntto2joP+62sd9uW5EpkZmKIvfNcTWlnQ=
|
||||
github.com/RaveNoX/go-jsoncommentstrip v1.0.0/go.mod h1:78ihd09MekBnJnxpICcwzCMzGrKSKYe4AqU6PDYYpjk=
|
||||
github.com/adrg/xdg v0.5.3 h1:xRnxJXne7+oWDatRhR1JLnvuccuIeCoBu2rtuLqQB78=
|
||||
github.com/adrg/xdg v0.5.3/go.mod h1:nlTsY+NNiCBGCK2tpm09vRqfVzrc2fLmXGpBLF0zlTQ=
|
||||
github.com/apapsch/go-jsonmerge/v2 v2.0.0 h1:axGnT1gRIfimI7gJifB699GoE/oq+F2MU7Dml6nw9rQ=
|
||||
github.com/apapsch/go-jsonmerge/v2 v2.0.0/go.mod h1:lvDnEdqiQrp0O42VQGgmlKpxL1AP2+08jFMw88y4klk=
|
||||
github.com/aws/aws-sdk-go-v2 v1.27.2 h1:pLsTXqX93rimAOZG2FIYraDQstZaaGVVN4tNw65v0h8=
|
||||
@@ -42,11 +40,7 @@ github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UF
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 h1:6xNmx7iTtyBRev0+D/Tv1FZd4SCg8axKApyNyRsAt/w=
|
||||
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5/go.mod h1:KdCmV+x/BuvyMxRnYBlmVaq4OLiKW6iRQfvC62cvdkI=
|
||||
github.com/coder/websocket v1.8.14 h1:9L0p0iKiNOibykf283eHkKUHHrpG7f65OE3BhhO7v9g=
|
||||
github.com/coder/websocket v1.8.14/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
|
||||
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
|
||||
github.com/danieljoos/wincred v1.2.3 h1:v7dZC2x32Ut3nEfRH+vhoZGvN72+dQ/snVXo/vMFLdQ=
|
||||
github.com/danieljoos/wincred v1.2.3/go.mod h1:6qqX0WNrS4RzPZ1tnroDzq9kY3fu1KwE7MRLQK4X0bs=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
@@ -65,15 +59,11 @@ github.com/go-chi/chi/v5 v5.3.1 h1:3j4HZLGZQ3JpMCrPJF/Jl3mYJfWLKBfNJ6quurUGCf8=
|
||||
github.com/go-chi/chi/v5 v5.3.1/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
|
||||
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
|
||||
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
|
||||
github.com/go-json-experiment/json v0.0.0-20251027170946-4849db3c2f7e h1:Lf/gRkoycfOBPa42vU2bbgPurFong6zXeFtPoxholzU=
|
||||
github.com/go-json-experiment/json v0.0.0-20251027170946-4849db3c2f7e/go.mod h1:uNVvRXArCGbZ508SxYYTC5v1JWoz2voff5pm25jU1Ok=
|
||||
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
|
||||
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
|
||||
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
|
||||
github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
|
||||
github.com/go-openapi/jsonpointer v0.22.5 h1:8on/0Yp4uTb9f4XvTrM2+1CPrV05QPZXu+rvu2o9jcA=
|
||||
github.com/go-openapi/jsonpointer v0.22.5/go.mod h1:gyUR3sCvGSWchA2sUBJGluYMbe1zazrYWIkWPjjMUY0=
|
||||
github.com/go-openapi/swag/jsonname v0.25.5 h1:8p150i44rv/Drip4vWI3kGi9+4W9TdI3US3uUYSFhSo=
|
||||
@@ -83,8 +73,6 @@ github.com/go-openapi/testify/v2 v2.4.0/go.mod h1:HCPmvFFnheKK2BuwSA0TbbdxJ3I16p
|
||||
github.com/go-resty/resty/v2 v2.13.1 h1:x+LHXBI2nMB1vqndymf26quycC4aggYJ7DECYbiz03g=
|
||||
github.com/go-resty/resty/v2 v2.13.1/go.mod h1:GznXlLxkq6Nh4sU59rPmUw3VtgpO3aS96ORAI6Q7d+0=
|
||||
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
|
||||
github.com/godbus/dbus/v5 v5.2.2 h1:TUR3TgtSVDmjiXOgAAyaZbYmIeP3DPkld3jgKGV8mXQ=
|
||||
github.com/godbus/dbus/v5 v5.2.2/go.mod h1:3AAv2+hPq5rdnr5txxxRwiGjPXamgoIHgz9FPBfOp3c=
|
||||
github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
|
||||
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||
@@ -115,19 +103,11 @@ github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
|
||||
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
|
||||
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
||||
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
||||
github.com/jchv/go-winloader v0.0.0-20250406163304-c1995be93bd1 h1:njuLRcjAuMKr7kI3D85AXWkw6/+v9PwtV6M6o11sWHQ=
|
||||
github.com/jchv/go-winloader v0.0.0-20250406163304-c1995be93bd1/go.mod h1:alcuEEnZsY1WQsagKhZDsoPCRoOijYqhZvPwLG0kzVs=
|
||||
github.com/juju/gnuflag v0.0.0-20171113085948-2ce1bb71843d/go.mod h1:2PavIy+JPciBPrBUjwbNvtwB6RQlve+hkpll6QSNmOE=
|
||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
||||
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
||||
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||
github.com/matryer/is v1.4.1 h1:55ehd8zaGABKLXQUe2awZ99BD/PTc2ls+KV/dXphgEQ=
|
||||
github.com/matryer/is v1.4.1/go.mod h1:8I/i5uYgLzgsgEloJE1U6xx5HkBQpAZvepWuujKwMRU=
|
||||
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
|
||||
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/modelcontextprotocol/go-sdk v1.6.1 h1:0zOSupjKUxPKSocPT1Wtago+mUHU2/uZ4xSOY0FGReU=
|
||||
github.com/modelcontextprotocol/go-sdk v1.6.1/go.mod h1:kzm3kzFL1/+AziGOE0nUs3gvPoNxMCvkxokMkuFapXQ=
|
||||
github.com/oapi-codegen/nullable v1.1.0 h1:eAh8JVc5430VtYVnq00Hrbpag9PFRGWLjxR1/3KntMs=
|
||||
@@ -164,8 +144,6 @@ github.com/spkg/bom v0.0.0-20160624110644-59b7046e48ad/go.mod h1:qLr4V1qq6nMqFKk
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
|
||||
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
@@ -183,16 +161,12 @@ github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
|
||||
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
|
||||
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
|
||||
github.com/wailsapp/wails/v3 v3.0.0-alpha2.117 h1:udyjqPG3AIgkod5QDR/WblCkpV8R86BFPSrsWxSyt5Y=
|
||||
github.com/wailsapp/wails/v3 v3.0.0-alpha2.117/go.mod h1:74WH2FScMsgucZvHHvv7eOefDXCm/CjuIxqhhZgPhKg=
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
|
||||
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
|
||||
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 h1:ilQV1hzziu+LLM3zUTJ0trRztfwgjqKnBWNtSRkbmwM=
|
||||
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78/go.mod h1:aL8wCCfTfSfmXjznFBSZNN13rSJjlIOI1fUNAtF7rmI=
|
||||
github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
|
||||
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
|
||||
github.com/zalando/go-keyring v0.2.8 h1:6sD/Ucpl7jNq10rM2pgqTs0sZ9V3qMrqfIIy5YPccHs=
|
||||
github.com/zalando/go-keyring v0.2.8/go.mod h1:tsMo+VpRq5NGyKfxoBVjCuMrG47yj8cmakZDO5QGii0=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
||||
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 h1:q4XOmH/0opmeuJtPsbFNivyl7bCt7yRBbeEm2sC/XtQ=
|
||||
@@ -242,16 +216,13 @@ golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
|
||||
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200810151505-1b9f1253b3ed/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
|
||||
@@ -14,10 +14,9 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres/sqlcgen"
|
||||
"github.com/google/uuid"
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
// Run starts the actuator loop. Blocks until ctx is cancelled.
|
||||
@@ -403,7 +402,7 @@ func ProvisionVM(ctx context.Context, pool *db.Pool, entityID uuid.UUID, attrs m
|
||||
return nil
|
||||
}
|
||||
|
||||
// sshExecSimple runs a command over SSH with a simple client setup.
|
||||
// sshExecSimple runs a command over SSH using the shared dial/run primitives.
|
||||
// Uses the default SSH key from SSH_KEY_PATH or ~/.ssh/id_rsa.
|
||||
func sshExecSimple(ctx context.Context, host, user, command string) (string, error) {
|
||||
keyPath := os.Getenv("SSH_KEY_PATH")
|
||||
@@ -411,55 +410,19 @@ func sshExecSimple(ctx context.Context, host, user, command string) (string, err
|
||||
keyPath = os.Getenv("HOME") + "/.ssh/id_rsa"
|
||||
}
|
||||
|
||||
keyBytes, err := os.ReadFile(keyPath)
|
||||
signer, err := LoadSigner(keyPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("read ssh key: %w", err)
|
||||
return "", err
|
||||
}
|
||||
|
||||
signer, err := ssh.ParsePrivateKey(keyBytes)
|
||||
client, err := Dial(ctx, DialOptions{Host: host, User: user, Signer: signer})
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("parse ssh key: %w", err)
|
||||
}
|
||||
|
||||
clientCfg := &ssh.ClientConfig{
|
||||
User: user,
|
||||
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
|
||||
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
|
||||
Timeout: 10 * time.Second,
|
||||
}
|
||||
|
||||
client, err := ssh.Dial("tcp", host+":22", clientCfg)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("ssh dial %s: %w", host, err)
|
||||
return "", err
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
type result struct {
|
||||
output string
|
||||
err error
|
||||
}
|
||||
ch := make(chan result, 1)
|
||||
go func() {
|
||||
out, e := session.CombinedOutput(command)
|
||||
ch <- result{output: string(out), err: e}
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
return "", ctx.Err()
|
||||
case res := <-ch:
|
||||
if res.err != nil {
|
||||
return res.output, res.err
|
||||
}
|
||||
return res.output, nil
|
||||
}
|
||||
out, err := RunCombinedOutput(ctx, client, command)
|
||||
return string(out), err
|
||||
}
|
||||
|
||||
// resolveHost resolves a host entity slug to (address, user) for SSH.
|
||||
|
||||
141
internal/actuator/client.go
Normal file
141
internal/actuator/client.go
Normal file
@@ -0,0 +1,141 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"net"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
// defaultDialTimeout bounds an SSH dial when the caller leaves Timeout unset.
|
||||
// 10s matches the previous hardcoded value at every dial site.
|
||||
const defaultDialTimeout = 10 * time.Second
|
||||
|
||||
// LoadSigner reads and parses the private key at keyPath.
|
||||
func LoadSigner(keyPath string) (ssh.Signer, error) {
|
||||
key, err := os.ReadFile(keyPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read ssh key: %w", err)
|
||||
}
|
||||
return LoadSignerFromBytes(key)
|
||||
}
|
||||
|
||||
// LoadSignerFromBytes parses an in-memory private key into an ssh.Signer.
|
||||
func LoadSignerFromBytes(key []byte) (ssh.Signer, error) {
|
||||
signer, err := ssh.ParsePrivateKey(key)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse ssh key: %w", err)
|
||||
}
|
||||
return signer, nil
|
||||
}
|
||||
|
||||
// DialOptions configures an SSH dial.
|
||||
type DialOptions struct {
|
||||
Host string
|
||||
Port int // 0 means 22
|
||||
User string
|
||||
Signer ssh.Signer
|
||||
Timeout time.Duration // dial timeout; <=0 means defaultDialTimeout
|
||||
}
|
||||
|
||||
// Dial opens a crypto/ssh connection through the centralized HostKeyCallback.
|
||||
// The connection itself is bounded by Timeout; ctx is respected by callers
|
||||
// via RunCombinedOutput once the session is running.
|
||||
func Dial(ctx context.Context, opts DialOptions) (*ssh.Client, error) {
|
||||
port := opts.Port
|
||||
if port <= 0 {
|
||||
port = 22
|
||||
}
|
||||
timeout := opts.Timeout
|
||||
if timeout <= 0 {
|
||||
timeout = defaultDialTimeout
|
||||
}
|
||||
cfg := &ssh.ClientConfig{
|
||||
User: opts.User,
|
||||
Auth: []ssh.AuthMethod{ssh.PublicKeys(opts.Signer)},
|
||||
HostKeyCallback: HostKeyCallback(),
|
||||
Timeout: timeout,
|
||||
}
|
||||
addr := net.JoinHostPort(opts.Host, fmt.Sprintf("%d", port))
|
||||
client, err := ssh.Dial("tcp", addr, cfg)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("ssh dial %s:%d: %w", opts.Host, port, err)
|
||||
}
|
||||
return client, nil
|
||||
}
|
||||
|
||||
// RunCombinedOutput runs cmd on an established client and returns its combined
|
||||
// stdout/stderr. Context cancellation closes the session to abort the remote
|
||||
// command instead of blocking until it finishes — the same goroutine+select
|
||||
// pattern the actuator, mcp, and scheduler each reimplemented before.
|
||||
func RunCombinedOutput(ctx context.Context, client *ssh.Client, cmd string) ([]byte, error) {
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
type result struct {
|
||||
out []byte
|
||||
err error
|
||||
}
|
||||
ch := make(chan result, 1)
|
||||
go func() {
|
||||
out, err := session.CombinedOutput(cmd)
|
||||
ch <- result{out: out, err: err}
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
return nil, ctx.Err()
|
||||
case res := <-ch:
|
||||
if res.err != nil {
|
||||
return res.out, fmt.Errorf("command: %w", res.err)
|
||||
}
|
||||
return res.out, nil
|
||||
}
|
||||
}
|
||||
|
||||
// RunOutput runs cmd on an established client and returns stdout only.
|
||||
// Stderr is folded into the returned error so callers that parse stdout
|
||||
// as JSON (e.g. the scheduler's check scripts) don't get interleaved
|
||||
// stderr in the output stream.
|
||||
func RunOutput(ctx context.Context, client *ssh.Client, cmd string) ([]byte, error) {
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
var outBuf, errBuf bytes.Buffer
|
||||
session.Stdout = &outBuf
|
||||
session.Stderr = &errBuf
|
||||
|
||||
type result struct {
|
||||
runErr error
|
||||
}
|
||||
ch := make(chan result, 1)
|
||||
go func() {
|
||||
ch <- result{runErr: session.Run(cmd)}
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
return nil, ctx.Err()
|
||||
case res := <-ch:
|
||||
if res.runErr != nil {
|
||||
if errBuf.Len() > 0 {
|
||||
return outBuf.Bytes(), fmt.Errorf("command: %w\nstderr: %s", res.runErr, strings.TrimSpace(errBuf.String()))
|
||||
}
|
||||
return outBuf.Bytes(), fmt.Errorf("command: %w", res.runErr)
|
||||
}
|
||||
return outBuf.Bytes(), nil
|
||||
}
|
||||
}
|
||||
88
internal/actuator/client_test.go
Normal file
88
internal/actuator/client_test.go
Normal file
@@ -0,0 +1,88 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"encoding/pem"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
func TestLoadSignerRejectsBadInput(t *testing.T) {
|
||||
if _, err := LoadSignerFromBytes([]byte("not a private key")); err == nil {
|
||||
t.Error("LoadSignerFromBytes should reject a non-key input")
|
||||
}
|
||||
if _, err := LoadSigner("/nonexistent/key"); err == nil {
|
||||
t.Error("LoadSigner should fail on a missing file")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadSignerRoundTrip(t *testing.T) {
|
||||
_, priv, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatalf("generate key: %v", err)
|
||||
}
|
||||
block, err := ssh.MarshalPrivateKey(priv, "")
|
||||
if err != nil {
|
||||
t.Fatalf("marshal private key: %v", err)
|
||||
}
|
||||
pemBytes := pem.EncodeToMemory(block)
|
||||
|
||||
signer, err := LoadSignerFromBytes(pemBytes)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadSignerFromBytes on a valid key: %v", err)
|
||||
}
|
||||
if signer == nil {
|
||||
t.Fatal("signer is nil")
|
||||
}
|
||||
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "id_ed25519")
|
||||
if err := os.WriteFile(path, pemBytes, 0o600); err != nil {
|
||||
t.Fatalf("write key file: %v", err)
|
||||
}
|
||||
fromFile, err := LoadSigner(path)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadSigner(%s): %v", path, err)
|
||||
}
|
||||
if !bytes.Equal(fromFile.PublicKey().Marshal(), signer.PublicKey().Marshal()) {
|
||||
t.Error("file and in-memory signers resolved to different public keys")
|
||||
}
|
||||
}
|
||||
|
||||
// Dial needs a real SSH server to run a command, but its option normalization
|
||||
// is verifiable without one: a zero Port must default to 22 (so the dial error
|
||||
// references host:22, not host:0), and a closed port yields a dial error rather
|
||||
// than panicking.
|
||||
func TestDialDefaultsPort(t *testing.T) {
|
||||
_, err := Dial(context.Background(), DialOptions{Host: "127.0.0.1", Signer: mustSigner(t)})
|
||||
if err == nil {
|
||||
t.Fatal("Dial to a closed port should fail")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "127.0.0.1:22") {
|
||||
t.Errorf("Dial error = %q, want it to reference 127.0.0.1:22", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustSigner(t *testing.T) ssh.Signer {
|
||||
t.Helper()
|
||||
_, priv, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatalf("generate key: %v", err)
|
||||
}
|
||||
block, err := ssh.MarshalPrivateKey(priv, "")
|
||||
if err != nil {
|
||||
t.Fatalf("marshal key: %v", err)
|
||||
}
|
||||
s, err := LoadSignerFromBytes(pem.EncodeToMemory(block))
|
||||
if err != nil {
|
||||
t.Fatalf("parse key: %v", err)
|
||||
}
|
||||
return s
|
||||
}
|
||||
129
internal/actuator/hostkeys.go
Normal file
129
internal/actuator/hostkeys.go
Normal file
@@ -0,0 +1,129 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
var (
|
||||
hostKeyMu sync.RWMutex
|
||||
hostKeyCache map[string]ssh.PublicKey
|
||||
hostKeyOnce sync.Once
|
||||
hostKeySrc HostKeySource
|
||||
)
|
||||
|
||||
// HostKeySource provides storage for SSH host public keys.
|
||||
type HostKeySource interface {
|
||||
GetHostKey(ctx context.Context, hostname string) (string, error)
|
||||
SetHostKey(ctx context.Context, hostname string, key string) error
|
||||
}
|
||||
|
||||
// SetHostKeySource sets the host key source. Must be called before
|
||||
// any SSH connections. A nil source enables TOFU-only mode (keys
|
||||
// accepted in memory but not persisted).
|
||||
func SetHostKeySource(src HostKeySource) {
|
||||
hostKeyMu.Lock()
|
||||
defer hostKeyMu.Unlock()
|
||||
hostKeySrc = src
|
||||
}
|
||||
|
||||
// HostKeyCallback returns an ssh.HostKeyCallback that verifies host keys.
|
||||
// Known keys are verified (MITM detection). Unknown keys are accepted
|
||||
// via TOFU and optionally persisted to the source.
|
||||
func HostKeyCallback() ssh.HostKeyCallback {
|
||||
return hostKeyVerify
|
||||
}
|
||||
|
||||
func hostKeyVerify(hostname string, remote net.Addr, key ssh.PublicKey) error {
|
||||
hostKeyOnce.Do(func() {
|
||||
hostKeyCache = make(map[string]ssh.PublicKey)
|
||||
})
|
||||
|
||||
normalized := hostWithoutPort(hostname)
|
||||
|
||||
hostKeyMu.RLock()
|
||||
known, exists := hostKeyCache[normalized]
|
||||
hostKeyMu.RUnlock()
|
||||
|
||||
if exists {
|
||||
if bytes.Equal(key.Marshal(), known.Marshal()) {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("SSH HOST KEY CHANGED for %s (possible MITM)", normalized)
|
||||
}
|
||||
|
||||
hostKeyMu.Lock()
|
||||
hostKeyCache[normalized] = key
|
||||
hostKeyMu.Unlock()
|
||||
|
||||
slog.Info("ssh: accepting new host key (TOFU)", "host", normalized)
|
||||
|
||||
if hostKeySrc != nil {
|
||||
go persistHostKey(normalized, key)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func persistHostKey(hostname string, key ssh.PublicKey) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
keyBase64 := key.Type() + " " + string(key.Marshal())
|
||||
if err := hostKeySrc.SetHostKey(ctx, "ssh/host-keys/"+hostname, keyBase64); err != nil {
|
||||
slog.Warn("ssh: failed to persist host key", "host", hostname, "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// LoadHostKeys pre-loads known host keys from the source into the
|
||||
// in-memory cache. Call at startup to avoid TOFU on first connection.
|
||||
// The source should return key lines in the format "key-type base64-data".
|
||||
func LoadHostKeys(ctx context.Context, hostnames []string, src HostKeySource) {
|
||||
if src == nil {
|
||||
return
|
||||
}
|
||||
SetHostKeySource(src)
|
||||
|
||||
hostKeyMu.Lock()
|
||||
defer hostKeyMu.Unlock()
|
||||
|
||||
if hostKeyCache == nil {
|
||||
hostKeyCache = make(map[string]ssh.PublicKey)
|
||||
}
|
||||
|
||||
loaded := 0
|
||||
for _, hostname := range hostnames {
|
||||
keyData, err := src.GetHostKey(ctx, "ssh/host-keys/"+hostname)
|
||||
if err != nil {
|
||||
slog.Debug("ssh: no stored key for host", "host", hostname, "error", err)
|
||||
continue
|
||||
}
|
||||
pubKey, _, _, _, err := ssh.ParseAuthorizedKey([]byte(keyData))
|
||||
if err != nil {
|
||||
slog.Warn("ssh: invalid stored key for host", "host", hostname, "error", err)
|
||||
continue
|
||||
}
|
||||
hostKeyCache[hostname] = pubKey
|
||||
loaded++
|
||||
}
|
||||
|
||||
if loaded > 0 {
|
||||
slog.Info("ssh: loaded host keys from Infisical", "count", loaded)
|
||||
}
|
||||
}
|
||||
|
||||
func hostWithoutPort(hostname string) string {
|
||||
for i := len(hostname) - 1; i >= 0; i-- {
|
||||
if hostname[i] == ':' {
|
||||
return hostname[:i]
|
||||
}
|
||||
}
|
||||
return hostname
|
||||
}
|
||||
56
internal/actuator/hostkeys_infisical.go
Normal file
56
internal/actuator/hostkeys_infisical.go
Normal file
@@ -0,0 +1,56 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres"
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
)
|
||||
|
||||
// InfisicalHostKeySource implements HostKeySource backed by Infisical.
|
||||
type InfisicalHostKeySource struct {
|
||||
sec secrets.Backend
|
||||
}
|
||||
|
||||
// NewInfisicalHostKeySource creates a HostKeySource that reads/writes
|
||||
// SSH host public keys from Infisical under the `ssh/host-keys/` prefix.
|
||||
func NewInfisicalHostKeySource(sec secrets.Backend) *InfisicalHostKeySource {
|
||||
return &InfisicalHostKeySource{sec: sec}
|
||||
}
|
||||
|
||||
func (s *InfisicalHostKeySource) GetHostKey(ctx context.Context, path string) (string, error) {
|
||||
val, err := s.sec.Get(ctx, path)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return val, nil
|
||||
}
|
||||
|
||||
func (s *InfisicalHostKeySource) SetHostKey(ctx context.Context, path string, key string) error {
|
||||
return s.sec.Set(ctx, path, key)
|
||||
}
|
||||
|
||||
// ResolveSSHHosts queries the DB for active proxmox-host and standalone-server
|
||||
// entities, returning their slugs as SSH host identifiers.
|
||||
func ResolveSSHHosts(ctx context.Context, pool *db.Pool) []string {
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT slug FROM entities
|
||||
WHERE type IN ('proxmox-host', 'standalone-server')
|
||||
AND state = 'active'
|
||||
ORDER BY slug`)
|
||||
if err != nil {
|
||||
slog.Warn("ssh: failed to list hosts", "error", err)
|
||||
return nil
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var hosts []string
|
||||
for rows.Next() {
|
||||
var slug string
|
||||
if rows.Scan(&slug) == nil {
|
||||
hosts = append(hosts, slug)
|
||||
}
|
||||
}
|
||||
return hosts
|
||||
}
|
||||
122
internal/actuator/pool.go
Normal file
122
internal/actuator/pool.go
Normal file
@@ -0,0 +1,122 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
type poolEntry struct {
|
||||
client *ssh.Client
|
||||
createdAt time.Time
|
||||
}
|
||||
|
||||
type DialPool struct {
|
||||
mu sync.RWMutex
|
||||
entries map[string]*poolEntry
|
||||
ttl time.Duration
|
||||
done chan struct{}
|
||||
stopped bool
|
||||
}
|
||||
|
||||
func NewDialPool(ttl time.Duration) *DialPool {
|
||||
p := &DialPool{
|
||||
entries: make(map[string]*poolEntry),
|
||||
ttl: ttl,
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
if ttl > 0 {
|
||||
go p.evictLoop()
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func (p *DialPool) key(opts DialOptions) string {
|
||||
port := opts.Port
|
||||
if port <= 0 {
|
||||
port = 22
|
||||
}
|
||||
return fmt.Sprintf("%s:%d", opts.Host, port)
|
||||
}
|
||||
|
||||
func (p *DialPool) Get(ctx context.Context, opts DialOptions) (*ssh.Client, error) {
|
||||
k := p.key(opts)
|
||||
p.mu.RLock()
|
||||
entry, ok := p.entries[k]
|
||||
p.mu.RUnlock()
|
||||
if ok {
|
||||
// Quick health check: a session can be created without running a
|
||||
// command — if it fails, the connection is dead and we evict it.
|
||||
testSession, err := entry.client.NewSession()
|
||||
if err == nil {
|
||||
testSession.Close()
|
||||
return entry.client, nil
|
||||
}
|
||||
p.mu.Lock()
|
||||
if p.entries[k] == entry {
|
||||
entry.client.Close()
|
||||
delete(p.entries, k)
|
||||
}
|
||||
p.mu.Unlock()
|
||||
// Fall through to dial below
|
||||
}
|
||||
client, err := Dial(ctx, opts)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
p.mu.Lock()
|
||||
if p.stopped {
|
||||
p.mu.Unlock()
|
||||
client.Close()
|
||||
return nil, fmt.Errorf("ssh dial pool: closed")
|
||||
}
|
||||
if existing, ok2 := p.entries[k]; ok2 {
|
||||
p.mu.Unlock()
|
||||
client.Close()
|
||||
return existing.client, nil
|
||||
}
|
||||
p.entries[k] = &poolEntry{client: client, createdAt: time.Now()}
|
||||
p.mu.Unlock()
|
||||
return client, nil
|
||||
}
|
||||
|
||||
func (p *DialPool) Close() {
|
||||
p.mu.Lock()
|
||||
p.stopped = true
|
||||
for k, entry := range p.entries {
|
||||
entry.client.Close()
|
||||
delete(p.entries, k)
|
||||
}
|
||||
p.mu.Unlock()
|
||||
if p.ttl > 0 {
|
||||
close(p.done)
|
||||
}
|
||||
}
|
||||
|
||||
func (p *DialPool) evictLoop() {
|
||||
ticker := time.NewTicker(p.ttl / 2)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-p.done:
|
||||
return
|
||||
case <-ticker.C:
|
||||
p.evict()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *DialPool) evict() {
|
||||
deadline := time.Now().Add(-p.ttl)
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for k, entry := range p.entries {
|
||||
if entry.createdAt.Before(deadline) {
|
||||
entry.client.Close()
|
||||
delete(p.entries, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
@@ -131,37 +130,19 @@ func ExecuteProcedure(
|
||||
start := time.Now()
|
||||
|
||||
// Parse the SSH key
|
||||
key, err := os.ReadFile(cfg.KeyPath)
|
||||
signer, err := LoadSigner(cfg.KeyPath)
|
||||
if err != nil {
|
||||
return SSHResult{
|
||||
Err: fmt.Errorf("read ssh key: %w", err),
|
||||
Err: err,
|
||||
Duration: time.Since(start),
|
||||
Verified: false,
|
||||
}
|
||||
}
|
||||
|
||||
signer, err := ssh.ParsePrivateKey(key)
|
||||
if err != nil {
|
||||
return SSHResult{
|
||||
Err: fmt.Errorf("parse ssh key: %w", err),
|
||||
Duration: time.Since(start),
|
||||
Verified: false,
|
||||
}
|
||||
}
|
||||
|
||||
addr := net.JoinHostPort(cfg.Host, fmt.Sprintf("%d", cfg.Port))
|
||||
if cfg.Port == 0 {
|
||||
addr = net.JoinHostPort(cfg.Host, "22")
|
||||
}
|
||||
|
||||
clientCfg := &ssh.ClientConfig{
|
||||
User: cfg.User,
|
||||
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
|
||||
HostKeyCallback: ssh.InsecureIgnoreHostKey(), // restricted key; host trust via inventory
|
||||
Timeout: cfg.Timeout,
|
||||
}
|
||||
|
||||
client, err := ssh.Dial("tcp", addr, clientCfg)
|
||||
client, err := Dial(ctx, DialOptions{
|
||||
Host: cfg.Host, Port: cfg.Port, User: cfg.User,
|
||||
Signer: signer, Timeout: cfg.Timeout,
|
||||
})
|
||||
if err != nil {
|
||||
class := classifySSHError(err)
|
||||
return SSHResult{
|
||||
@@ -229,38 +210,11 @@ func ExecuteProcedure(
|
||||
}
|
||||
}
|
||||
|
||||
// runSSHCommand executes a single command over an established SSH session.
|
||||
// Uses context-aware goroutines: ctx.Done() closes the session.
|
||||
// runSSHCommand executes a single command over an established SSH session via
|
||||
// the shared RunCombinedOutput primitive (context-aware abort + combined output).
|
||||
func runSSHCommand(ctx context.Context, client *ssh.Client, command string) (string, error) {
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
// Wrap in goroutine so we can abort on ctx.Done()
|
||||
type result struct {
|
||||
output string
|
||||
err error
|
||||
}
|
||||
|
||||
ch := make(chan result, 1)
|
||||
go func() {
|
||||
out, err := session.CombinedOutput(command)
|
||||
ch <- result{output: string(out), err: err}
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
// Close the session to abort the SSH command
|
||||
session.Close()
|
||||
return "", ctx.Err()
|
||||
case res := <-ch:
|
||||
if res.err != nil {
|
||||
return res.output, fmt.Errorf("command: %w", res.err)
|
||||
}
|
||||
return res.output, nil
|
||||
}
|
||||
out, err := RunCombinedOutput(ctx, client, command)
|
||||
return string(out), err
|
||||
}
|
||||
|
||||
// ─── Procedure parsing ────────────────────────────────────────────────────
|
||||
|
||||
86
internal/actuator/stream.go
Normal file
86
internal/actuator/stream.go
Normal file
@@ -0,0 +1,86 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
// RunStreaming runs a command on an established SSH client and forwards output
|
||||
// chunks to sink as they arrive. A nil sink collects output silently. Returns
|
||||
// the full combined output and any command error.
|
||||
func RunStreaming(ctx context.Context, client *ssh.Client, command string, sink func(stream string, chunk []byte), timeout time.Duration) (string, error) {
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
outPipe, err := session.StdoutPipe()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("stdout pipe: %w", err)
|
||||
}
|
||||
errPipe, err := session.StderrPipe()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("stderr pipe: %w", err)
|
||||
}
|
||||
|
||||
type streamResult struct {
|
||||
out string
|
||||
err error
|
||||
}
|
||||
|
||||
resultCh := make(chan streamResult, 1)
|
||||
go func() {
|
||||
var combined []byte
|
||||
done := make(chan struct{}, 2)
|
||||
|
||||
readStream := func(stream string, r io.Reader) {
|
||||
sc := bufio.NewScanner(r)
|
||||
for sc.Scan() {
|
||||
line := sc.Bytes()
|
||||
chunk := make([]byte, len(line))
|
||||
copy(chunk, line)
|
||||
if sink != nil {
|
||||
sink(stream, chunk)
|
||||
}
|
||||
if stream == "stdout" || stream == "" {
|
||||
if len(combined) > 0 {
|
||||
combined = append(combined, '\n')
|
||||
}
|
||||
combined = append(combined, chunk...)
|
||||
}
|
||||
}
|
||||
done <- struct{}{}
|
||||
}
|
||||
|
||||
go readStream("stdout", outPipe)
|
||||
go readStream("stderr", errPipe)
|
||||
|
||||
runErr := session.Run(command)
|
||||
<-done
|
||||
<-done
|
||||
resultCh <- streamResult{out: string(combined), err: runErr}
|
||||
}()
|
||||
|
||||
if timeout > 0 {
|
||||
var cancel context.CancelFunc
|
||||
ctx, cancel = context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
}
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
return "", ctx.Err()
|
||||
case res := <-resultCh:
|
||||
if res.err != nil {
|
||||
return res.out, fmt.Errorf("command: %w", res.err)
|
||||
}
|
||||
return res.out, nil
|
||||
}
|
||||
}
|
||||
5
internal/adapters/doc.go
Normal file
5
internal/adapters/doc.go
Normal file
@@ -0,0 +1,5 @@
|
||||
// Package adapters hosts the ports' implementations (postgres, ssh, probes,
|
||||
// remote, secrets, events) and the driving adapters (httpapi, mcpserver,
|
||||
// scheduler, execworker, cli). Packages move here phase by phase per
|
||||
// plans/2026-08-15-hexagonal-architecture.md.
|
||||
package adapters
|
||||
207
internal/adapters/postgres/checks.go
Normal file
207
internal/adapters/postgres/checks.go
Normal file
@@ -0,0 +1,207 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/app"
|
||||
"github.com/dtoro/oikos/internal/ontology"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// EnsureEntityChecks derives an entity's default check_defs from the
|
||||
// monitoring spec of its type (resolving per-entity `monitoring` overrides).
|
||||
//
|
||||
// This is the single shared hook that keeps the check graph in sync with
|
||||
// entity mutations. Both the HTTP create/patch handlers and the MCP
|
||||
// entity-mutation tools (create_entity, update_entity_attributes) call it so
|
||||
// that flipping an entity's `monitoring` attribute regenerates checks
|
||||
// regardless of which surface made the change — previously only the HTTP
|
||||
// path ran check derivation, so entities mutated via MCP silently produced no
|
||||
// checks (see plans/2026-08-03-session-review-haos-monitoring-capability-gaps.md, A2).
|
||||
func EnsureEntityChecks(ctx context.Context, tx pgx.Tx, id uuid.UUID, slug, entityType, name string, attrs []byte) (app.DeriveResult, error) {
|
||||
tree, err := LoadTypeTree(ctx, tx)
|
||||
if err != nil {
|
||||
return app.DeriveResult{}, err
|
||||
}
|
||||
res, err := EnsureChecks(ctx, tx, tree, app.CheckTarget{
|
||||
ID: id.String(), Slug: slug, Type: entityType, Name: name, Attrs: attrs,
|
||||
})
|
||||
if err != nil {
|
||||
return res, err
|
||||
}
|
||||
app.LogDeriveResult(slug, entityType, res)
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// EnsureChecks writes the derived check_defs for one entity, idempotently.
|
||||
// Derivation is pure core logic (app.Derive); this function owns the
|
||||
// entity_status row, the graph host fallback, and the upserts.
|
||||
func EnsureChecks(ctx context.Context, tx pgx.Tx, tree *ontology.TypeTree, t app.CheckTarget) (app.DeriveResult, error) {
|
||||
var res app.DeriveResult
|
||||
|
||||
if _, err := tx.Exec(ctx,
|
||||
`INSERT INTO entity_status (entity_id, health, updated_at)
|
||||
VALUES ($1, 'unknown', now())
|
||||
ON CONFLICT (entity_id) DO NOTHING`, t.ID); err != nil {
|
||||
return res, fmt.Errorf("entity_status %s: %w", t.Slug, err)
|
||||
}
|
||||
|
||||
defs, dres := app.Derive(tree, t, func() map[string]any {
|
||||
attrs, err := hostViaGraph(ctx, tx, t.ID)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
return attrs
|
||||
})
|
||||
res.Skipped, res.Undeclared = dres.Skipped, dres.Undeclared
|
||||
|
||||
for i, def := range defs {
|
||||
created, err := writeCheck(ctx, tx, t, i, def)
|
||||
if err != nil {
|
||||
return res, fmt.Errorf("check %s/%s: %w", t.Slug, def.Kind, err)
|
||||
}
|
||||
if created {
|
||||
res.Created++
|
||||
}
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// writeCheck upserts one check_def and its backing check entity.
|
||||
//
|
||||
// The entity upsert MUST return the row's id. The previous version generated
|
||||
// a fresh uuid, inserted ON CONFLICT (slug) DO NOTHING, then wrote a
|
||||
// check_defs row referencing that uuid. On any re-seed the slug already
|
||||
// existed, the entity insert became a no-op, and the check_defs insert
|
||||
// violated its foreign key — which aborted the whole ingest transaction and
|
||||
// made every subsequent statement fail with 25P02. Because the errors were
|
||||
// discarded, the only visible symptom was an unrelated failure much later.
|
||||
func writeCheck(ctx context.Context, tx pgx.Tx, t app.CheckTarget, idx int, def app.CheckDef) (bool, error) {
|
||||
// The full target slug, not a truncation of it. shortSlug() took the last
|
||||
// 8 characters, so all 21 ingress routes collapsed to ".network" and
|
||||
// generated one identical check slug — they overwrote each other and 20
|
||||
// of them ended up with no check at all. It also collided service:jellyfin
|
||||
// with lxc:jellyfin. Entity slugs are unique; use them.
|
||||
checkSlug := fmt.Sprintf("check:%s:%s:%d", def.Kind, t.Slug, idx)
|
||||
|
||||
newID, err := uuid.NewV7()
|
||||
if err != nil {
|
||||
newID = uuid.New()
|
||||
}
|
||||
|
||||
var checkID uuid.UUID
|
||||
err = tx.QueryRow(ctx,
|
||||
`INSERT INTO entities (id, slug, type, name, state, attributes, version, created_at, updated_at)
|
||||
VALUES ($1, $2, 'check', $2, 'active', '{}', 1, now(), now())
|
||||
ON CONFLICT (slug) DO UPDATE SET updated_at = now()
|
||||
RETURNING id`,
|
||||
newID, checkSlug).Scan(&checkID)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("upsert check entity %s: %w", checkSlug, err)
|
||||
}
|
||||
|
||||
configJSON, err := json.Marshal(def.Config)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
// Config is derived from the seed, so the seed wins on re-ingest and
|
||||
// attribute changes propagate. `enabled` is deliberately left alone: it
|
||||
// is operational state an operator may have toggled.
|
||||
// last_run_at is seeded to a random point inside the interval so checks
|
||||
// created together do not stay in lockstep. Every check the seed creates
|
||||
// would otherwise come due in the same instant forever: ~165 probes
|
||||
// landing at once each minute rather than spread across it. Deliberately
|
||||
// absent from the DO UPDATE below — a re-seed must not reset the schedule
|
||||
// and re-herd everything.
|
||||
tag, err := tx.Exec(ctx,
|
||||
`INSERT INTO check_defs (entity_id, target_id, target_type, kind, config, interval_s, timeout_s, enabled, last_run_at)
|
||||
VALUES ($1, $2, $6, $3, $4, $5, 30, true,
|
||||
now() - make_interval(secs => random() * $5::int))
|
||||
ON CONFLICT (entity_id) DO UPDATE
|
||||
SET target_id = EXCLUDED.target_id, target_type = EXCLUDED.target_type,
|
||||
kind = EXCLUDED.kind,
|
||||
config = EXCLUDED.config, interval_s = EXCLUDED.interval_s,
|
||||
updated_at = now()`,
|
||||
checkID, t.ID, def.Kind, configJSON, def.IntervalS, t.Type)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("upsert check_def %s: %w", checkSlug, err)
|
||||
}
|
||||
return tag.RowsAffected() > 0, nil
|
||||
}
|
||||
|
||||
// hostViaGraph returns the attributes of the entity that hosts or provides
|
||||
// this one, so a service can inherit its container's address.
|
||||
func hostViaGraph(ctx context.Context, tx pgx.Tx, entityID string) (map[string]any, error) {
|
||||
rows, err := tx.Query(ctx, `
|
||||
SELECT e.attributes
|
||||
FROM relationships r
|
||||
JOIN entities e ON e.id = r.source_id
|
||||
WHERE r.target_id = $1
|
||||
AND r.valid_to IS NULL
|
||||
-- backs-up-to points from the thing being backed up TO the target,
|
||||
-- so walking it backwards finds the machine that writes the backups
|
||||
-- — which is the only place a freshness check can run.
|
||||
AND r.type IN ('provides', 'hosts', 'runs-on', 'backs-up-to')
|
||||
ORDER BY CASE r.type
|
||||
WHEN 'provides' THEN 0 WHEN 'runs-on' THEN 1
|
||||
WHEN 'backs-up-to' THEN 2 ELSE 3 END`,
|
||||
entityID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
for rows.Next() {
|
||||
var raw []byte
|
||||
if err := rows.Scan(&raw); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var attrs map[string]any
|
||||
if json.Unmarshal(raw, &attrs) != nil {
|
||||
continue
|
||||
}
|
||||
if resolveGraphHost(attrs) != "" {
|
||||
return attrs, nil
|
||||
}
|
||||
}
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
// resolveGraphHost mirrors app's address resolution for graph-walk results.
|
||||
// It re-implements the small pure helper rather than exporting internals of
|
||||
// the core package: the shapes it accepts are exactly the seed attribute
|
||||
// shapes hostViaGraph can return.
|
||||
func resolveGraphHost(attrs map[string]any) string {
|
||||
if attrs == nil {
|
||||
return ""
|
||||
}
|
||||
if ip, ok := attrs["lan_ip"].(string); ok && ip != "" {
|
||||
return ip
|
||||
}
|
||||
if ip, ok := attrs["public_ipv4"].(string); ok && ip != "" {
|
||||
return ip
|
||||
}
|
||||
if mesh, ok := attrs["mesh"].(map[string]any); ok {
|
||||
if nb, ok := mesh["netbird"].(map[string]any); ok {
|
||||
if ip, ok := nb["ip"].(string); ok && ip != "" {
|
||||
return ip
|
||||
}
|
||||
if fqdn, ok := nb["fqdn"].(string); ok && fqdn != "" {
|
||||
return fqdn
|
||||
}
|
||||
}
|
||||
}
|
||||
if ip, ok := attrs["mesh_ip"].(string); ok && ip != "" {
|
||||
return ip
|
||||
}
|
||||
for _, key := range []string{"host", "address", "public_host"} {
|
||||
if v, ok := attrs[key].(string); ok && v != "" {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
6
internal/adapters/postgres/doc.go
Normal file
6
internal/adapters/postgres/doc.go
Normal file
@@ -0,0 +1,6 @@
|
||||
// Package db is the postgres adapter: connection pool, migrations, seed
|
||||
// ingest, and sqlc-generated queries. It moved from internal/db in Phase 2
|
||||
// of the hexagonal refactor (ADR 0016); the package identifier stays `db`
|
||||
// until the repository split (Phase 3) renames it alongside the first
|
||||
// ports implementations landing here.
|
||||
package db
|
||||
68
internal/adapters/postgres/entity_cache.go
Normal file
68
internal/adapters/postgres/entity_cache.go
Normal file
@@ -0,0 +1,68 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type entityCacheEntry struct {
|
||||
slug string
|
||||
id string
|
||||
attrs string
|
||||
exp time.Time
|
||||
}
|
||||
|
||||
// EntityCache is a TTL cache mapping entity IDs to slugs and back,
|
||||
// keyed for the hot resolution paths.
|
||||
type EntityCache struct {
|
||||
mu sync.RWMutex
|
||||
m map[string]entityCacheEntry
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
// NewEntityCache builds a cache with the given TTL.
|
||||
func NewEntityCache(ttl time.Duration) *EntityCache {
|
||||
return &EntityCache{
|
||||
m: make(map[string]entityCacheEntry),
|
||||
ttl: ttl,
|
||||
}
|
||||
}
|
||||
|
||||
// GetSlug resolves an entity ID to its slug.
|
||||
func (c *EntityCache) GetSlug(id string) (string, bool) {
|
||||
c.mu.RLock()
|
||||
e, ok := c.m[id]
|
||||
c.mu.RUnlock()
|
||||
if !ok || time.Now().After(e.exp) {
|
||||
return "", false
|
||||
}
|
||||
return e.slug, true
|
||||
}
|
||||
|
||||
// GetID resolves a slug to its entity ID.
|
||||
func (c *EntityCache) GetID(slug string) (string, bool) {
|
||||
c.mu.RLock()
|
||||
e, ok := c.m[slug]
|
||||
c.mu.RUnlock()
|
||||
if !ok || time.Now().After(e.exp) {
|
||||
return "", false
|
||||
}
|
||||
return e.id, true
|
||||
}
|
||||
|
||||
// Set records the slug/id pair and serialized attributes.
|
||||
func (c *EntityCache) Set(slug, id, attrs string) {
|
||||
exp := time.Now().Add(c.ttl)
|
||||
c.mu.Lock()
|
||||
c.m[slug] = entityCacheEntry{slug: slug, id: id, attrs: attrs, exp: exp}
|
||||
c.m[id] = entityCacheEntry{slug: slug, id: id, attrs: attrs, exp: exp}
|
||||
c.mu.Unlock()
|
||||
}
|
||||
|
||||
// Invalidate drops the cached entries for one slug/id pair.
|
||||
func (c *EntityCache) Invalidate(slug, id string) {
|
||||
c.mu.Lock()
|
||||
delete(c.m, slug)
|
||||
delete(c.m, id)
|
||||
c.mu.Unlock()
|
||||
}
|
||||
@@ -18,12 +18,12 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
func seedsDir() string { return "../../seeds" }
|
||||
func seedsDir() string { return "../../../seeds" }
|
||||
|
||||
// newTestPool creates a throwaway database (dropped on cleanup), runs all
|
||||
// migrations, and returns a pool connected to it.
|
||||
102
internal/adapters/postgres/knowledge.go
Normal file
102
internal/adapters/postgres/knowledge.go
Normal file
@@ -0,0 +1,102 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// KnowledgeRepo implements ports.KnowledgeRepository.
|
||||
type KnowledgeRepo struct {
|
||||
pool *Pool
|
||||
}
|
||||
|
||||
var _ ports.KnowledgeRepository = (*KnowledgeRepo)(nil)
|
||||
|
||||
func NewKnowledgeRepo(pool *Pool) *KnowledgeRepo { return &KnowledgeRepo{pool: pool} }
|
||||
|
||||
func (r *KnowledgeRepo) Search(ctx context.Context, query string, limit int) ([]ports.KnowledgeEntry, error) {
|
||||
rows, err := r.pool.Query(ctx, `
|
||||
SELECT k.slug, k.title, k.kind, k.updated_at
|
||||
FROM knowledge_entities k
|
||||
WHERE k.deleted_at IS NULL
|
||||
AND (k.slug ILIKE '%'||$1||'%' OR k.title ILIKE '%'||$1||'%' OR k.content ILIKE '%'||$1||'%')
|
||||
ORDER BY k.updated_at DESC LIMIT $2`, query, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var items []ports.KnowledgeEntry
|
||||
for rows.Next() {
|
||||
entry, err := scanKnowledgeEntry(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, entry)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func (r *KnowledgeRepo) GetContent(ctx context.Context, slug string) (ports.KnowledgeEntry, error) {
|
||||
return scanKnowledgeEntry(r.pool.QueryRow(ctx, `
|
||||
SELECT slug, title, kind, content, updated_at
|
||||
FROM knowledge_entities WHERE slug = $1 AND deleted_at IS NULL`, slug))
|
||||
}
|
||||
|
||||
func (r *KnowledgeRepo) Upsert(ctx context.Context, input ports.KnowledgeUpsertInput) (ports.KnowledgeEntry, error) {
|
||||
id, _ := uuid.NewV7()
|
||||
_, err := r.pool.Exec(ctx, `
|
||||
INSERT INTO knowledge_entities (slug, title, kind, content, updated_at)
|
||||
VALUES ($1, $2, $3, $4, now())
|
||||
ON CONFLICT (slug) DO UPDATE
|
||||
SET title = $2, kind = $3, content = $4, updated_at = now()`,
|
||||
input.Entry.Slug, input.Entry.Title, input.Entry.Kind, input.Entry.Content)
|
||||
if err != nil {
|
||||
return ports.KnowledgeEntry{}, err
|
||||
}
|
||||
_ = id
|
||||
return input.Entry, nil
|
||||
}
|
||||
|
||||
func (r *KnowledgeRepo) Tags(ctx context.Context) (map[string]int, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (r *KnowledgeRepo) SoftDelete(ctx context.Context, slug string) error {
|
||||
_, err := r.pool.Exec(ctx, `UPDATE knowledge_entities SET deleted_at = now() WHERE slug = $1`, slug)
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *KnowledgeRepo) Restore(ctx context.Context, slug string) error {
|
||||
_, err := r.pool.Exec(ctx, `UPDATE knowledge_entities SET deleted_at = NULL WHERE slug = $1`, slug)
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *KnowledgeRepo) Revisions(ctx context.Context, slug string, limit int) ([]ports.KnowledgeEntry, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (r *KnowledgeRepo) Orphans(ctx context.Context, staleDays int) ([]ports.KnowledgeEntry, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (r *KnowledgeRepo) Duplicates(ctx context.Context, threshold float64) ([]ports.KnowledgeEntry, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (r *KnowledgeRepo) Merge(ctx context.Context, targetSlug string, sourceSlugs []string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func scanKnowledgeEntry(row interface{ Scan(dest ...any) error }) (ports.KnowledgeEntry, error) {
|
||||
var slug, title, kind, content string
|
||||
var updatedAt time.Time
|
||||
if err := row.Scan(&slug, &title, &kind, &content, &updatedAt); err != nil {
|
||||
return ports.KnowledgeEntry{}, err
|
||||
}
|
||||
return ports.KnowledgeEntry{
|
||||
Slug: slug, Title: title, Kind: kind, Content: content, UpdatedAt: updatedAt,
|
||||
}, nil
|
||||
}
|
||||
@@ -5,9 +5,8 @@ import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres/sqlcgen"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
@@ -79,35 +78,35 @@ func checkPrecondition(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, entit
|
||||
return fmt.Errorf("%d inbound relationship edges remaining", count)
|
||||
}
|
||||
case "backups-verified", "secrets-revoked", "ingress-dns-removed":
|
||||
var attrs string
|
||||
if err := tx.QueryRow(ctx, "SELECT coalesce(attributes::text,'{}') FROM entities WHERE id = $1", entityID).Scan(&attrs); err != nil {
|
||||
attrs, err := fetchAttrs(ctx, tx, entityID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
want := map[string]string{
|
||||
"backups-verified": "backups_verified",
|
||||
"secrets-revoked": "secrets_revoked",
|
||||
"backups-verified": "backups_verified",
|
||||
"secrets-revoked": "secrets_revoked",
|
||||
"ingress-dns-removed": "ingress_dns_removed",
|
||||
}[check]
|
||||
if !strings.Contains(attrs, want) {
|
||||
if !attrTruthy(attrs, want) {
|
||||
return fmt.Errorf("%s not recorded in entity attributes", want)
|
||||
}
|
||||
case "age-key-enrolled-if-needed":
|
||||
if entityType == "workstation" {
|
||||
var attrs string
|
||||
if err := tx.QueryRow(ctx, "SELECT coalesce(attributes::text,'{}') FROM entities WHERE id = $1", entityID).Scan(&attrs); err != nil {
|
||||
attrs, err := fetchAttrs(ctx, tx, entityID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !strings.Contains(attrs, "age_pubkey") {
|
||||
if !attrTruthy(attrs, "age_pubkey") {
|
||||
return fmt.Errorf("age key not enrolled (no age_pubkey in attributes)")
|
||||
}
|
||||
}
|
||||
case "mesh-joined-if-needed":
|
||||
if entityType == "workstation" {
|
||||
var attrs string
|
||||
if err := tx.QueryRow(ctx, "SELECT coalesce(attributes::text,'{}') FROM entities WHERE id = $1", entityID).Scan(&attrs); err != nil {
|
||||
attrs, err := fetchAttrs(ctx, tx, entityID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !strings.Contains(attrs, "mesh_ip") {
|
||||
if !attrTruthy(attrs, "mesh_ip") {
|
||||
return fmt.Errorf("mesh not joined (no mesh_ip in attributes)")
|
||||
}
|
||||
}
|
||||
@@ -144,3 +143,40 @@ func checkPrecondition(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, entit
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// fetchAttrs loads an entity's JSONB attributes column as a decoded map.
|
||||
// Missing attributes decode to an empty map (every key absent).
|
||||
func fetchAttrs(ctx context.Context, tx pgx.Tx, entityID uuid.UUID) (map[string]any, error) {
|
||||
var raw string
|
||||
if err := tx.QueryRow(ctx, "SELECT coalesce(attributes::text,'{}') FROM entities WHERE id = $1", entityID).Scan(&raw); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var attrs map[string]any
|
||||
if err := json.Unmarshal([]byte(raw), &attrs); err != nil {
|
||||
return nil, fmt.Errorf("decode entity attributes: %w", err)
|
||||
}
|
||||
if attrs == nil {
|
||||
attrs = map[string]any{}
|
||||
}
|
||||
return attrs, nil
|
||||
}
|
||||
|
||||
// attrTruthy reports whether key is present in attrs with a meaningful value.
|
||||
// It replaces substring matching on raw JSONB text: a previous strings.Contains
|
||||
// check treated {"backups_verified": false} as satisfied (the key text was
|
||||
// present) and bypassed the attributes GIN index. Booleans must be true;
|
||||
// strings must be non-empty; nil/absent fail.
|
||||
func attrTruthy(attrs map[string]any, key string) bool {
|
||||
v, ok := attrs[key]
|
||||
if !ok || v == nil {
|
||||
return false
|
||||
}
|
||||
switch t := v.(type) {
|
||||
case bool:
|
||||
return t
|
||||
case string:
|
||||
return t != ""
|
||||
default:
|
||||
return true // numbers, objects, arrays count as present
|
||||
}
|
||||
}
|
||||
55
internal/adapters/postgres/lifecycle_test.go
Normal file
55
internal/adapters/postgres/lifecycle_test.go
Normal file
@@ -0,0 +1,55 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// attrTruthy replaces a previous strings.Contains check over raw JSONB text.
|
||||
// The key regression it guards: a literal attribute like
|
||||
// {"backups_verified": false} must NOT satisfy the "backups-verified"
|
||||
// precondition, even though the key text is present in the column.
|
||||
func TestAttrTruthy(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
attrs map[string]any
|
||||
key string
|
||||
want bool
|
||||
}{
|
||||
{"absent", map[string]any{}, "backups_verified", false},
|
||||
{"nil map", nil, "backups_verified", false},
|
||||
{"explicit nil value", map[string]any{"backups_verified": nil}, "backups_verified", false},
|
||||
{"bool true", map[string]any{"backups_verified": true}, "backups_verified", true},
|
||||
{"bool false is the regression case", map[string]any{"backups_verified": false}, "backups_verified", false},
|
||||
{"nonempty string age pubkey", map[string]any{"age_pubkey": "age1abc"}, "age_pubkey", true},
|
||||
{"empty string is falsy", map[string]any{"mesh_ip": ""}, "mesh_ip", false},
|
||||
{"number counts as present", map[string]any{"port": float64(22)}, "port", true},
|
||||
{"other keys present", map[string]any{"backups_verified": true, "unrelated": "x"}, "backups_verified", true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if got := attrTruthy(tc.attrs, tc.key); got != tc.want {
|
||||
t.Fatalf("attrTruthy(%v, %q) = %v, want %v", tc.attrs, tc.key, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// fetchAttrs decodes the JSONB column text; verify the decode shape that
|
||||
// attrTruthy then evaluates (the DB round-trip itself is covered by make test-db).
|
||||
func TestAttrTruthyAfterDecode(t *testing.T) {
|
||||
raw := `{"backups_verified": true, "mesh_ip": "10.0.0.5", "secrets_revoked": false}`
|
||||
var got map[string]any
|
||||
if err := json.Unmarshal([]byte(raw), &got); err != nil {
|
||||
t.Fatalf("unmarshal: %v", err)
|
||||
}
|
||||
if !attrTruthy(got, "backups_verified") {
|
||||
t.Error("backups_verified should be truthy after decode")
|
||||
}
|
||||
if !attrTruthy(got, "mesh_ip") {
|
||||
t.Error("mesh_ip should be truthy after decode")
|
||||
}
|
||||
if attrTruthy(got, "secrets_revoked") {
|
||||
t.Error("secrets_revoked:false is the regression — must be falsy")
|
||||
}
|
||||
}
|
||||
@@ -56,7 +56,11 @@ func (p *Pool) Migrate(ctx context.Context) error {
|
||||
if _, err := conn.Exec(ctx, "SELECT pg_advisory_lock($1)", migrationLockKey); err != nil {
|
||||
return fmt.Errorf("acquire migration lock: %w", err)
|
||||
}
|
||||
defer conn.Exec(context.WithoutCancel(ctx), "SELECT pg_advisory_unlock($1)", migrationLockKey)
|
||||
defer func() {
|
||||
if _, err := conn.Exec(context.WithoutCancel(ctx), "SELECT pg_advisory_unlock($1)", migrationLockKey); err != nil {
|
||||
slog.Warn("postgres: release migration lock failed", "error", err)
|
||||
}
|
||||
}()
|
||||
|
||||
// Create tracking table if not exists
|
||||
_, err = conn.Exec(ctx, `
|
||||
@@ -156,7 +160,11 @@ func (p *Pool) SeedIngest(ctx context.Context, filename string, content []byte,
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin tx: %w", err)
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
defer func() {
|
||||
if err := tx.Rollback(ctx); err != nil {
|
||||
slog.Debug("postgres: rollback after failed ingest", "error", err)
|
||||
}
|
||||
}()
|
||||
|
||||
if err := ingestFn(ctx, tx, data); err != nil {
|
||||
return fmt.Errorf("ingest %s: %w", filename, err)
|
||||
@@ -190,7 +198,10 @@ func hasSuffix(s, suffix string) bool {
|
||||
}
|
||||
|
||||
// splitSQL splits a SQL string into individual statements.
|
||||
// Handles $$ ... $$ dollar-quoted blocks and -- line comments.
|
||||
// Handles $$ ... $$ dollar-quoted blocks, $tag$ ... $tag$ tagged quotes,
|
||||
// -- line comments, /* ... */ block comments, and '...' string literals
|
||||
// so that semicolons inside any of these constructs are not treated as
|
||||
// statement boundaries.
|
||||
func splitSQL(sql string) []string {
|
||||
var statements []string
|
||||
var current strings.Builder
|
||||
@@ -201,7 +212,6 @@ func splitSQL(sql string) []string {
|
||||
for i < len(sql) {
|
||||
// Handle line comments (-- to end of line)
|
||||
if !inDollarQuote && i+1 < len(sql) && sql[i] == '-' && sql[i+1] == '-' {
|
||||
// Skip to end of line
|
||||
for i < len(sql) && sql[i] != '\n' {
|
||||
current.WriteByte(sql[i])
|
||||
i++
|
||||
@@ -209,6 +219,34 @@ func splitSQL(sql string) []string {
|
||||
continue
|
||||
}
|
||||
|
||||
// Handle block comments (/* ... */)
|
||||
if !inDollarQuote && i+1 < len(sql) && sql[i] == '/' && sql[i+1] == '*' {
|
||||
end := strings.Index(sql[i+2:], "*/")
|
||||
if end >= 0 {
|
||||
current.WriteString(sql[i : i+end+4])
|
||||
i += end + 4
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
// Handle single-quoted string literals ('...')
|
||||
if !inDollarQuote && sql[i] == '\'' {
|
||||
j := i + 1
|
||||
for j < len(sql) {
|
||||
if sql[j] == '\'' {
|
||||
if j+1 < len(sql) && sql[j+1] == '\'' {
|
||||
j += 2 // skip doubled quote ''
|
||||
continue
|
||||
}
|
||||
break
|
||||
}
|
||||
j++
|
||||
}
|
||||
current.WriteString(sql[i : j+1])
|
||||
i = j + 1
|
||||
continue
|
||||
}
|
||||
|
||||
// Check for dollar-quote start/end
|
||||
if !inDollarQuote && sql[i] == '$' {
|
||||
j := i + 1
|
||||
@@ -44,6 +44,36 @@ UPDATE entities SET
|
||||
WHERE id = sqlc.arg('id') AND version = sqlc.arg('version')
|
||||
RETURNING *;
|
||||
|
||||
-- name: MergeEntityAttributes :execrows
|
||||
-- Shallow-merge a JSON patch into an entity's attributes (the
|
||||
-- update_entity_attributes MCP/HTTP surface). Replaces the raw
|
||||
-- `attributes = attributes || $2::jsonb` used in entity_tools.go.
|
||||
UPDATE entities SET
|
||||
attributes = attributes || sqlc.arg('patch')::jsonb,
|
||||
updated_at = now()
|
||||
WHERE slug = sqlc.arg('slug');
|
||||
|
||||
-- name: SetEntityState :execrows
|
||||
-- Set an entity's lifecycle state by id (the set_entity_state surface, run
|
||||
-- after db.ValidateTransition). Replaces the raw
|
||||
-- `UPDATE entities SET state = $2 ... WHERE id = $1`.
|
||||
UPDATE entities SET
|
||||
state = sqlc.arg('state'),
|
||||
updated_at = now()
|
||||
WHERE id = sqlc.arg('id');
|
||||
|
||||
-- blast_radius(): the recursive-CTE traversal function's TABLE return type
|
||||
-- is opaque to sqlc's analyzer — that one query stays hand-written pgx in
|
||||
-- internal/httpapi (see impl.go).
|
||||
-- internal/httpapi (see entities.go GetBlastRadius).
|
||||
--
|
||||
-- Deliberate raw-SQL exceptions (plan E2): the httpapi entity *read* handlers
|
||||
-- (ListEntities/GetEntity/GetGraph/queryEntities) project a fixed
|
||||
-- `entityCols` column set (entities.* + a LEFT JOIN to entity_status for
|
||||
-- health/last_check_at) and scan it positionally into the oapi-generated
|
||||
-- gen.Entity shape. sqlc generates its own row struct per query and cannot
|
||||
-- emit gen.Entity, so migrating those reads would add a per-call field-by-
|
||||
-- field mapping with no compile-time gain and real column-order risk. They
|
||||
-- stay hand-written pgx, like blast_radius and the seed/export bulk paths
|
||||
-- noted in sqlc.yaml. The mutation/relationship surface (MergeEntityAttributes,
|
||||
-- SetEntityState, InsertRelationshipIfAbsent, EndCurrentRelationship) IS
|
||||
-- migrated and is what the entity CRUD tools now call.
|
||||
@@ -25,3 +25,18 @@ ORDER BY r.type, se.slug, te.slug;
|
||||
-- name: EndCurrentRelationship :execrows
|
||||
UPDATE relationships SET valid_to = now()
|
||||
WHERE source_id = $1 AND target_id = $2 AND type = $3 AND valid_to IS NULL;
|
||||
|
||||
-- name: InsertRelationshipIfAbsent :execrows
|
||||
-- Idempotent relationship insert (the create_relationship surface): no-op if
|
||||
-- an active edge of the same source/target/type already exists. Replaces the
|
||||
-- raw INSERT...WHERE NOT EXISTS used in entity_tools.go.
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT sqlc.arg('source_id'), sqlc.arg('target_id'), sqlc.arg('type'),
|
||||
sqlc.arg('attributes')::jsonb, now()
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM relationships
|
||||
WHERE source_id = sqlc.arg('source_id')
|
||||
AND target_id = sqlc.arg('target_id')
|
||||
AND type = sqlc.arg('type')
|
||||
AND valid_to IS NULL
|
||||
);
|
||||
349
internal/adapters/postgres/readmodels.go
Normal file
349
internal/adapters/postgres/readmodels.go
Normal file
@@ -0,0 +1,349 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"encoding/json"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// EntityReader implements ports.ReadModels over the postgres pool.
|
||||
type EntityReader struct {
|
||||
pool *Pool
|
||||
}
|
||||
|
||||
var _ ports.ReadModels = (*EntityReader)(nil)
|
||||
|
||||
// NewEntityReader builds the read-models service.
|
||||
func NewEntityReader(pool *Pool) *EntityReader { return &EntityReader{pool: pool} }
|
||||
|
||||
const entityHealthCols = `e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at, COALESCE(st.health, 'unknown'), st.last_check_at`
|
||||
|
||||
type rowScanner2 interface{ Scan(dest ...any) error }
|
||||
|
||||
func scanWithHealth(row rowScanner2) (ports.EntityWithHealth, error) {
|
||||
var id uuid.UUID
|
||||
var slug, typ, name string
|
||||
var state *string
|
||||
var attrs []byte
|
||||
var maint *time.Time
|
||||
var version int32
|
||||
var createdAt, updatedAt time.Time
|
||||
var health string
|
||||
var lastCheck *time.Time
|
||||
if err := row.Scan(&id, &slug, &typ, &name, &state, &attrs, &maint, &version, &createdAt, &updatedAt, &health, &lastCheck); err != nil {
|
||||
return ports.EntityWithHealth{}, err
|
||||
}
|
||||
e := domain.Entity{
|
||||
ID: domain.UUID(id.String()),
|
||||
Slug: slug,
|
||||
Type: typ,
|
||||
Name: name,
|
||||
Version: int(version),
|
||||
CreatedAt: createdAt,
|
||||
UpdatedAt: updatedAt,
|
||||
MaintenanceUntil: maint,
|
||||
}
|
||||
if state != nil {
|
||||
e.State = *state
|
||||
}
|
||||
if len(attrs) > 0 {
|
||||
var m map[string]any
|
||||
if json.Unmarshal(attrs, &m) == nil {
|
||||
e.Attributes = m
|
||||
}
|
||||
}
|
||||
return ports.EntityWithHealth{Entity: e, Health: health, LastCheckAt: lastCheck}, nil
|
||||
}
|
||||
|
||||
func (r *EntityReader) ListEntities(ctx context.Context, f ports.EntityFilters) ([]ports.EntityWithHealth, string, error) {
|
||||
limit := f.Limit
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
nullable := func(s string) any {
|
||||
if s == "" {
|
||||
return nil
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
rows, err := r.pool.Query(ctx, `
|
||||
WITH RECURSIVE tt AS (
|
||||
SELECT name FROM entity_types WHERE $1::text IS NULL OR name = $1
|
||||
UNION
|
||||
SELECT et.name FROM entity_types et JOIN tt ON et.parent_type = tt.name
|
||||
WHERE $1::text IS NOT NULL
|
||||
)
|
||||
SELECT `+entityHealthCols+`
|
||||
FROM entities e
|
||||
JOIN entity_types et ON et.name = e.type
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.type IN (SELECT name FROM tt)
|
||||
AND ($2::text IS NULL OR e.state = $2)
|
||||
AND ($3::text IS NULL OR et.domain = $3)
|
||||
AND ($4::text IS NULL OR et.layer = $4)
|
||||
AND ($5::text IS NULL OR e.slug ILIKE '%'||$5||'%' OR e.name ILIKE '%'||$5||'%')
|
||||
AND ($6::text IS NULL OR e.slug > $6)
|
||||
ORDER BY e.slug
|
||||
LIMIT $7`,
|
||||
nullable(f.Type), nullable(f.State), nullable(f.Domain), nullable(f.Layer),
|
||||
nullable(f.Q), nullable(f.Cursor), limit+1)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []ports.EntityWithHealth{}
|
||||
for rows.Next() {
|
||||
e, err := scanWithHealth(rows)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
items = append(items, e)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, "", rows.Err()
|
||||
}
|
||||
|
||||
next := ""
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
next = items[len(items)-1].Entity.Slug
|
||||
}
|
||||
return items, next, nil
|
||||
}
|
||||
|
||||
func (r *EntityReader) GetEntity(ctx context.Context, id domain.UUID) (ports.EntityWithHealth, error) {
|
||||
e, err := scanWithHealth(r.pool.QueryRow(ctx,
|
||||
`SELECT `+entityHealthCols+` FROM entities e LEFT JOIN entity_status st ON st.entity_id = e.id WHERE e.id = $1`, mustUUID(id)))
|
||||
if err != nil {
|
||||
return ports.EntityWithHealth{}, mapRowErr(err)
|
||||
}
|
||||
return e, nil
|
||||
}
|
||||
|
||||
func (r *EntityReader) GetEntityBySlug(ctx context.Context, slug string) (ports.EntityWithHealth, error) {
|
||||
e, err := scanWithHealth(r.pool.QueryRow(ctx,
|
||||
`SELECT `+entityHealthCols+` FROM entities e LEFT JOIN entity_status st ON st.entity_id = e.id WHERE e.slug = $1`, slug))
|
||||
if err != nil {
|
||||
return ports.EntityWithHealth{}, mapRowErr(err)
|
||||
}
|
||||
return e, nil
|
||||
}
|
||||
|
||||
func (r *EntityReader) GetEntityRelations(ctx context.Context, entityID domain.UUID, direction, relType string) ([]domain.Relationship, error) {
|
||||
eid := mustUUID(entityID)
|
||||
switch direction {
|
||||
case "outbound":
|
||||
return r.queryRels(ctx, `SELECT source_id, target_id, type, attributes, valid_from, valid_to
|
||||
FROM relationships WHERE valid_to IS NULL AND source_id = $1`, eid, relType)
|
||||
case "inbound":
|
||||
return r.queryRels(ctx, `SELECT source_id, target_id, type, attributes, valid_from, valid_to
|
||||
FROM relationships WHERE valid_to IS NULL AND target_id = $1`, eid, relType)
|
||||
default:
|
||||
return r.queryRels(ctx, `SELECT source_id, target_id, type, attributes, valid_from, valid_to
|
||||
FROM relationships WHERE valid_to IS NULL AND (source_id = $1 OR target_id = $1)`, eid, relType)
|
||||
}
|
||||
}
|
||||
|
||||
func (r *EntityReader) queryRels(ctx context.Context, query string, eid uuid.UUID, relType string) ([]domain.Relationship, error) {
|
||||
var args []any
|
||||
args = append(args, eid)
|
||||
if relType != "" {
|
||||
query += ` AND type = $2`
|
||||
args = append(args, relType)
|
||||
} else {
|
||||
query += ` ORDER BY type`
|
||||
}
|
||||
rows, err := r.pool.Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanEdges(rows)
|
||||
}
|
||||
|
||||
func (r *EntityReader) GetBlastRadius(ctx context.Context, entityID domain.UUID, depth int) ([]ports.EntityWithHealth, error) {
|
||||
rows, err := r.pool.Query(ctx, `
|
||||
SELECT `+entityHealthCols+`, b.depth
|
||||
FROM blast_radius($1, $2) b
|
||||
JOIN entities e ON e.id = b.entity_id
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY b.depth, e.slug`, mustUUID(entityID), depth)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var items []ports.EntityWithHealth
|
||||
for rows.Next() {
|
||||
var d int
|
||||
e, err := scanWithHealth(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
_ = d
|
||||
items = append(items, e)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func (r *EntityReader) GetGraph(ctx context.Context, depth int, root *domain.UUID, relTypes []string, cap int) ([]ports.EntityWithHealth, []domain.Relationship, bool, error) {
|
||||
truncated := false
|
||||
var nodes []ports.EntityWithHealth
|
||||
var err error
|
||||
|
||||
if root != nil {
|
||||
nodes, err = r.blastRadiusNodes(ctx, mustUUID(*root), depth, relTypes)
|
||||
} else {
|
||||
nodes, err = r.topologyNodes(ctx, cap)
|
||||
if err == nil && len(nodes) > cap {
|
||||
nodes = nodes[:cap]
|
||||
truncated = true
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return nil, nil, false, err
|
||||
}
|
||||
|
||||
ids := make([]uuid.UUID, len(nodes))
|
||||
for i, n := range nodes {
|
||||
uid, _ := uuid.Parse(string(n.Entity.ID))
|
||||
ids[i] = uid
|
||||
}
|
||||
edges, err := r.listGraphEdges(ctx, ids, relTypes)
|
||||
if err != nil {
|
||||
return nil, nil, false, err
|
||||
}
|
||||
return nodes, edges, truncated, nil
|
||||
}
|
||||
|
||||
func (r *EntityReader) blastRadiusNodes(ctx context.Context, rootID uuid.UUID, depth int, relTypes []string) ([]ports.EntityWithHealth, error) {
|
||||
query := `
|
||||
SELECT ` + entityHealthCols + `
|
||||
FROM blast_radius($1, $2, $3) b JOIN entities e ON e.id = b.entity_id
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY e.slug`
|
||||
var relTypesArg any
|
||||
if len(relTypes) > 0 {
|
||||
relTypesArg = relTypes
|
||||
}
|
||||
rows, err := r.pool.Query(ctx, query, rootID, depth, relTypesArg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var items []ports.EntityWithHealth
|
||||
for rows.Next() {
|
||||
e, err := scanWithHealth(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, e)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func (r *EntityReader) topologyNodes(ctx context.Context, cap int) ([]ports.EntityWithHealth, error) {
|
||||
rows, err := r.pool.Query(ctx, `
|
||||
SELECT `+entityHealthCols+`
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.type NOT IN ('execution','task')
|
||||
AND e.id IN (
|
||||
SELECT e2.id FROM entities e2
|
||||
LEFT JOIN relationships r ON r.valid_to IS NULL
|
||||
AND (r.source_id = e2.id OR r.target_id = e2.id)
|
||||
WHERE e2.type NOT IN ('execution','task')
|
||||
GROUP BY e2.id
|
||||
ORDER BY count(r.type) DESC, e2.slug
|
||||
LIMIT $1
|
||||
)
|
||||
ORDER BY e.slug`, cap+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var items []ports.EntityWithHealth
|
||||
for rows.Next() {
|
||||
e, err := scanWithHealth(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, e)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func (r *EntityReader) listGraphEdges(ctx context.Context, ids []uuid.UUID, relTypes []string) ([]domain.Relationship, error) {
|
||||
query := `SELECT r.source_id, r.target_id, r.type, r.attributes, r.valid_from, r.valid_to
|
||||
FROM relationships r
|
||||
WHERE r.valid_to IS NULL AND (r.source_id = ANY($1) OR r.target_id = ANY($1))`
|
||||
var args []any
|
||||
args = append(args, ids)
|
||||
if len(relTypes) > 0 {
|
||||
query += ` AND r.type = ANY($2)`
|
||||
args = append(args, relTypes)
|
||||
} else {
|
||||
query += ` ORDER BY r.type`
|
||||
}
|
||||
rows, err := r.pool.Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanEdges(rows)
|
||||
}
|
||||
|
||||
func (r *EntityReader) ListEntityTypes(ctx context.Context) ([]domain.EntityType, error) {
|
||||
rows, err := r.pool.Query(ctx,
|
||||
`SELECT name, COALESCE(parent_type,''), is_abstract, COALESCE(domain,''), COALESCE(layer,''), COALESCE(description,''),
|
||||
COALESCE(lifecycle_id,''), '{}'::jsonb, COALESCE(schema_version,0), COALESCE(status,'')
|
||||
FROM entity_types ORDER BY name`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var items []domain.EntityType
|
||||
for rows.Next() {
|
||||
var t domain.EntityType
|
||||
if err := rows.Scan(&t.Name, &t.ParentType, &t.IsAbstract, &t.Domain, &t.Layer, &t.Description,
|
||||
&t.LifecycleID, &t.AttributeSchema, &t.SchemaVersion, &t.Status); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, t)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func scanEdges(rows pgx.Rows) ([]domain.Relationship, error) {
|
||||
var items []domain.Relationship
|
||||
for rows.Next() {
|
||||
var src, tgt uuid.UUID
|
||||
var rType string
|
||||
var attrs []byte
|
||||
var vf time.Time
|
||||
var vt *time.Time
|
||||
if err := rows.Scan(&src, &tgt, &rType, &attrs, &vf, &vt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rel := domain.Relationship{
|
||||
SourceID: domain.UUID(src.String()), TargetID: domain.UUID(tgt.String()),
|
||||
Type: rType, Attributes: map[string]any{}, ValidFrom: vf, ValidTo: vt,
|
||||
}
|
||||
if len(attrs) > 0 {
|
||||
var m map[string]any
|
||||
if json.Unmarshal(attrs, &m) == nil {
|
||||
rel.Attributes = m
|
||||
}
|
||||
}
|
||||
items = append(items, rel)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
613
internal/adapters/postgres/repositories.go
Normal file
613
internal/adapters/postgres/repositories.go
Normal file
@@ -0,0 +1,613 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/app"
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/dtoro/oikos/internal/ontology"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// EntityRepo implements ports.EntityRepository over the postgres pool.
|
||||
// Command methods run everything in their input inside one transaction
|
||||
// (ADR 0016 §3.6): entity write, derived checks, idempotency record, audit,
|
||||
// event.
|
||||
type EntityRepo struct {
|
||||
pool *Pool
|
||||
}
|
||||
|
||||
var _ ports.EntityRepository = (*EntityRepo)(nil)
|
||||
|
||||
// NewEntityRepo builds the entity repository.
|
||||
func NewEntityRepo(pool *Pool) *EntityRepo { return &EntityRepo{pool: pool} }
|
||||
|
||||
|
||||
// mustUUID converts a domain.UUID (string alias) to uuid.UUID. The domain
|
||||
// layer guarantees UUID-shaped strings; parse failures are programming
|
||||
// errors and panic loudly rather than half-failing a transaction.
|
||||
func mustUUID(id domain.UUID) uuid.UUID {
|
||||
u, err := uuid.Parse(string(id))
|
||||
if err != nil {
|
||||
panic(fmt.Sprintf("invalid entity UUID %q", string(id)))
|
||||
}
|
||||
return u
|
||||
}
|
||||
|
||||
const entityFullCols = `id, slug, type, name, state, attributes, maintenance_until, version, created_at, updated_at`
|
||||
|
||||
type rowScanner interface{ Scan(dest ...any) error }
|
||||
|
||||
func scanDomainEntity(row rowScanner) (domain.Entity, error) {
|
||||
var id uuid.UUID
|
||||
var slug, typ, name string
|
||||
var state *string
|
||||
var attrs []byte
|
||||
var maint *time.Time
|
||||
var version int32
|
||||
var createdAt, updatedAt time.Time
|
||||
if err := row.Scan(&id, &slug, &typ, &name, &state, &attrs, &maint, &version, &createdAt, &updatedAt); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
d := domain.Entity{
|
||||
ID: domain.UUID(id.String()),
|
||||
Slug: slug,
|
||||
Type: typ,
|
||||
Name: name,
|
||||
Version: int(version),
|
||||
CreatedAt: createdAt,
|
||||
UpdatedAt: updatedAt,
|
||||
MaintenanceUntil: maint,
|
||||
}
|
||||
if state != nil {
|
||||
d.State = *state
|
||||
}
|
||||
if len(attrs) > 0 {
|
||||
var m map[string]any
|
||||
if json.Unmarshal(attrs, &m) == nil {
|
||||
d.Attributes = m
|
||||
}
|
||||
}
|
||||
return d, nil
|
||||
}
|
||||
|
||||
func mapRowErr(err error) error {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return domain.ErrNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// Get returns the entity by ID.
|
||||
func (r *EntityRepo) Get(ctx context.Context, id domain.UUID) (domain.Entity, error) {
|
||||
uid, err := uuid.Parse(string(id))
|
||||
if err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
e, err := scanDomainEntity(r.pool.QueryRow(ctx,
|
||||
`SELECT `+entityFullCols+` FROM entities WHERE id = $1`, uid))
|
||||
return e, mapRowErr(err)
|
||||
}
|
||||
|
||||
// BySlug returns the entity by slug.
|
||||
func (r *EntityRepo) BySlug(ctx context.Context, slug string) (domain.Entity, error) {
|
||||
e, err := scanDomainEntity(r.pool.QueryRow(ctx,
|
||||
`SELECT `+entityFullCols+` FROM entities WHERE slug = $1`, slug))
|
||||
return e, mapRowErr(err)
|
||||
}
|
||||
|
||||
// List returns entities filtered by type (including descendant types),
|
||||
// state, domain, layer, and a slug/name substring, keyset-paginated by
|
||||
// slug. The second return is the next cursor ("" when exhausted).
|
||||
func (r *EntityRepo) List(ctx context.Context, f ports.EntityFilters) ([]domain.Entity, string, error) {
|
||||
limit := f.Limit
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
// NULL-able filter args: an absent filter must bind SQL NULL, not "".
|
||||
nullable := func(s string) any {
|
||||
if s == "" {
|
||||
return nil
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// Type filter includes descendants via the parent hierarchy (R3-1).
|
||||
rows, err := r.pool.Query(ctx, `
|
||||
WITH RECURSIVE tt AS (
|
||||
SELECT name FROM entity_types WHERE $1::text IS NULL OR name = $1
|
||||
UNION
|
||||
SELECT et.name FROM entity_types et JOIN tt ON et.parent_type = tt.name
|
||||
WHERE $1::text IS NOT NULL
|
||||
)
|
||||
SELECT `+entityFullCols+`
|
||||
FROM entities e
|
||||
JOIN entity_types et ON et.name = e.type
|
||||
WHERE e.type IN (SELECT name FROM tt)
|
||||
AND ($2::text IS NULL OR e.state = $2)
|
||||
AND ($3::text IS NULL OR et.domain = $3)
|
||||
AND ($4::text IS NULL OR et.layer = $4)
|
||||
AND ($5::text IS NULL OR e.slug ILIKE '%'||$5||'%' OR e.name ILIKE '%'||$5||'%')
|
||||
AND ($6::text IS NULL OR e.slug > $6)
|
||||
ORDER BY e.slug
|
||||
LIMIT $7`,
|
||||
nullable(f.Type), nullable(f.State), nullable(f.Domain), nullable(f.Layer),
|
||||
nullable(f.Q), nullable(f.Cursor), limit+1)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []domain.Entity{}
|
||||
for rows.Next() {
|
||||
e, err := scanDomainEntity(rows)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
items = append(items, e)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, "", rows.Err()
|
||||
}
|
||||
|
||||
next := ""
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
next = items[len(items)-1].Slug
|
||||
}
|
||||
return items, next, nil
|
||||
}
|
||||
|
||||
// Search matches slug/name substrings.
|
||||
func (r *EntityRepo) Search(ctx context.Context, q string, limit int) ([]domain.Entity, error) {
|
||||
items, _, err := r.List(ctx, ports.EntityFilters{Q: q, Limit: limit})
|
||||
return items, err
|
||||
}
|
||||
|
||||
// writeSideEffects writes audit entries and the event inside the open
|
||||
// transaction.
|
||||
func writeSideEffects(ctx context.Context, tx pgx.Tx, entityID domain.UUID, audits []ports.AuditEntry, event *ports.Event) error {
|
||||
q := sqlcgen.New(tx)
|
||||
eid := mustUUID(entityID)
|
||||
for _, a := range audits {
|
||||
if err := observability.Audit(ctx, q, a.ActorType, a.ActorLabel, a.Action, &eid,
|
||||
a.Method, a.Path, a.CorrelationID, nil, a.Details); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if event != nil {
|
||||
if err := observability.Event(ctx, q, event.Type, &eid, event.Severity, event.Source,
|
||||
event.CorrelationID, event.Data); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func appTargetOf(e domain.Entity) app.CheckTarget {
|
||||
attrs, _ := json.Marshal(e.Attributes)
|
||||
return app.CheckTarget{ID: string(e.ID), Slug: e.Slug, Type: e.Type, Name: e.Name, Attrs: attrs}
|
||||
}
|
||||
|
||||
func writeDerivedChecks(ctx context.Context, tx pgx.Tx, e domain.Entity, checks []ports.DerivedCheck) (int, error) {
|
||||
if _, err := tx.Exec(ctx,
|
||||
`INSERT INTO entity_status (entity_id, health, updated_at)
|
||||
VALUES ($1, 'unknown', now())
|
||||
ON CONFLICT (entity_id) DO NOTHING`, mustUUID(e.ID)); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
created := 0
|
||||
t := appTargetOf(e)
|
||||
for i, dc := range checks {
|
||||
ok, err := writeCheck(ctx, tx, t, i, app.CheckDef{Kind: dc.Kind, Config: dc.Config, IntervalS: dc.IntervalS})
|
||||
if err != nil {
|
||||
return created, err
|
||||
}
|
||||
if ok {
|
||||
created++
|
||||
}
|
||||
}
|
||||
return created, nil
|
||||
}
|
||||
|
||||
// rederiveChecks re-runs derivation for an entity inside the open tx, with
|
||||
// the graph host fallback active (a service inherits its container's
|
||||
// address once the hosting edge exists).
|
||||
func rederiveChecks(ctx context.Context, tx pgx.Tx, e domain.Entity, tree *ontology.TypeTree) (int, error) {
|
||||
t := appTargetOf(e)
|
||||
defs, _ := app.Derive(tree, t, func() map[string]any {
|
||||
attrs, err := hostViaGraph(ctx, tx, t.ID)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
return attrs
|
||||
})
|
||||
derived := make([]ports.DerivedCheck, len(defs))
|
||||
for i, d := range defs {
|
||||
derived[i] = ports.DerivedCheck{Kind: d.Kind, Config: d.Config, IntervalS: d.IntervalS}
|
||||
}
|
||||
return writeDerivedChecks(ctx, tx, e, derived)
|
||||
}
|
||||
|
||||
// Create inserts the entity with its derived checks, idempotency record,
|
||||
// audit, and event — one transaction.
|
||||
func (r *EntityRepo) Create(ctx context.Context, in ports.EntityCreateInput) (domain.Entity, error) {
|
||||
tx, err := r.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
e := in.Entity
|
||||
attrsJSON, _ := json.Marshal(e.Attributes)
|
||||
if e.Attributes == nil {
|
||||
attrsJSON = []byte("{}")
|
||||
}
|
||||
var state *string
|
||||
if e.State != "" {
|
||||
state = &e.State
|
||||
}
|
||||
|
||||
created, err := scanDomainEntity(tx.QueryRow(ctx, `
|
||||
INSERT INTO entities (id, slug, type, name, state, attributes)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)
|
||||
RETURNING `+entityFullCols,
|
||||
mustUUID(e.ID), e.Slug, e.Type, e.Name, state, attrsJSON))
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
|
||||
return domain.Entity{}, errors.Join(domain.ErrAlreadyExists, err)
|
||||
}
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
|
||||
if in.Idempotency != nil {
|
||||
var body []byte
|
||||
if in.Idempotency.RenderBody != nil {
|
||||
body = in.Idempotency.RenderBody(created)
|
||||
}
|
||||
code := int32(201)
|
||||
if err := sqlcgen.New(tx).PutIdempotentResponse(ctx, sqlcgen.PutIdempotentResponseParams{
|
||||
Actor: in.Idempotency.Actor, Key: in.Idempotency.Key,
|
||||
RequestHash: in.Idempotency.RequestHash, ResponseCode: &code, ResponseBody: body,
|
||||
}); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
}
|
||||
|
||||
if _, err := writeDerivedChecks(ctx, tx, created, in.DerivedChecks); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
if err := writeSideEffects(ctx, tx, created.ID, in.Audit, in.Event); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
return created, nil
|
||||
}
|
||||
|
||||
// Update applies name/state/attributes/maintenance atomically with an
|
||||
// optimistic-version check, re-deriving default checks when asked. The
|
||||
// declared-transition + precondition validation runs inside the
|
||||
// transaction (check-then-act, plan §3.6).
|
||||
func (r *EntityRepo) Update(ctx context.Context, in ports.EntityUpdateInput) (domain.Entity, error) {
|
||||
tx, err := r.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
current, err := scanDomainEntity(tx.QueryRow(ctx,
|
||||
`SELECT `+entityFullCols+` FROM entities WHERE id = $1`, mustUUID(in.Entity.ID)))
|
||||
if err != nil {
|
||||
return domain.Entity{}, mapRowErr(err)
|
||||
}
|
||||
|
||||
if in.ExpectedVersion > 0 && current.Version != in.ExpectedVersion {
|
||||
return domain.Entity{}, domain.ErrConflict
|
||||
}
|
||||
|
||||
e := in.Entity
|
||||
// Fields the caller left zero keep their current values.
|
||||
if e.Name == "" {
|
||||
e.Name = current.Name
|
||||
}
|
||||
if e.Slug == "" {
|
||||
e.Slug = current.Slug
|
||||
}
|
||||
if e.Type == "" {
|
||||
e.Type = current.Type
|
||||
}
|
||||
|
||||
// Lifecycle validation when state changes (declared transition +
|
||||
// preconditions, in-tx).
|
||||
if e.State != "" && e.State != current.State {
|
||||
if err := ValidateTransition(ctx, tx, mustUUID(current.ID), current.Type, current.State, e.State); err != nil {
|
||||
if errors.Is(err, ErrTransitionInvalid) {
|
||||
return domain.Entity{}, errors.Join(domain.ErrInvalidTransition, err)
|
||||
}
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
} else if e.State == "" {
|
||||
e.State = current.State
|
||||
}
|
||||
|
||||
if e.Attributes == nil {
|
||||
e.Attributes = current.Attributes
|
||||
}
|
||||
setMaintenance := e.MaintenanceUntil != nil
|
||||
maint := e.MaintenanceUntil
|
||||
if !setMaintenance {
|
||||
maint = current.MaintenanceUntil
|
||||
}
|
||||
|
||||
attrsJSON, _ := json.Marshal(e.Attributes)
|
||||
if e.Attributes == nil {
|
||||
attrsJSON = []byte("{}")
|
||||
}
|
||||
var state *string
|
||||
if e.State != "" {
|
||||
state = &e.State
|
||||
}
|
||||
|
||||
updated, err := scanDomainEntity(tx.QueryRow(ctx, `
|
||||
UPDATE entities
|
||||
SET name = $2, state = $3, attributes = $4, maintenance_until = $5, version = version + 1, updated_at = now()
|
||||
WHERE id = $1 AND version = $6
|
||||
RETURNING `+entityFullCols,
|
||||
mustUUID(e.ID), e.Name, state, attrsJSON, maint, current.Version))
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return domain.Entity{}, domain.ErrConflict
|
||||
}
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
|
||||
if in.RederiveChecks {
|
||||
tree, err := LoadTypeTree(ctx, tx)
|
||||
if err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
if _, err := rederiveChecks(ctx, tx, updated, tree); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
}
|
||||
|
||||
if err := writeSideEffects(ctx, tx, updated.ID, in.Audit, in.Event); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
return updated, nil
|
||||
}
|
||||
|
||||
// SetState transitions an entity's lifecycle state. Declared-transition
|
||||
// and precondition validation run in-tx; a stale From is refused.
|
||||
func (r *EntityRepo) SetState(ctx context.Context, in ports.EntityTransitionInput) (domain.Entity, error) {
|
||||
tx, err := r.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
current, err := scanDomainEntity(tx.QueryRow(ctx,
|
||||
`SELECT `+entityFullCols+` FROM entities WHERE slug = $1`, in.Slug))
|
||||
if err != nil {
|
||||
return domain.Entity{}, mapRowErr(err)
|
||||
}
|
||||
if current.State != in.From {
|
||||
return domain.Entity{}, domain.ErrConflict
|
||||
}
|
||||
if err := ValidateTransition(ctx, tx, mustUUID(current.ID), current.Type, in.From, in.To); err != nil {
|
||||
if errors.Is(err, ErrTransitionInvalid) {
|
||||
return domain.Entity{}, errors.Join(domain.ErrInvalidTransition, err)
|
||||
}
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
if _, err := tx.Exec(ctx,
|
||||
`UPDATE entities SET state = $2, version = version + 1, updated_at = now() WHERE id = $1`,
|
||||
mustUUID(current.ID), in.To); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
|
||||
after := current
|
||||
after.State = in.To
|
||||
if err := writeSideEffects(ctx, tx, after.ID, in.Audit, in.Event); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
return after, nil
|
||||
}
|
||||
|
||||
// GetIdempotent returns the cached response for (actor, key) or
|
||||
// domain.ErrNotFound.
|
||||
func (r *EntityRepo) GetIdempotent(ctx context.Context, actor, key string) (ports.IdempotentResponse, error) {
|
||||
cached, err := sqlcgen.New(r.pool).GetIdempotentResponse(ctx, sqlcgen.GetIdempotentResponseParams{Actor: actor, Key: key})
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return ports.IdempotentResponse{}, domain.ErrNotFound
|
||||
}
|
||||
return ports.IdempotentResponse{}, err
|
||||
}
|
||||
code := 0
|
||||
if cached.ResponseCode != nil {
|
||||
code = int(*cached.ResponseCode)
|
||||
}
|
||||
return ports.IdempotentResponse{
|
||||
RequestHash: cached.RequestHash,
|
||||
ResponseCode: code,
|
||||
ResponseBody: cached.ResponseBody,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// RelRepo implements ports.RelationshipRepository over the postgres pool.
|
||||
type RelRepo struct {
|
||||
pool *Pool
|
||||
}
|
||||
|
||||
var _ ports.RelationshipRepository = (*RelRepo)(nil)
|
||||
|
||||
func NewRelRepo(pool *Pool) *RelRepo { return &RelRepo{pool: pool} }
|
||||
|
||||
func (r *RelRepo) Create(ctx context.Context, input ports.RelationshipCreateInput) (domain.Relationship, error) {
|
||||
tx, err := r.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return domain.Relationship{}, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
attrsJSON := []byte("{}")
|
||||
if len(input.Relationship.Attributes) > 0 {
|
||||
attrsJSON, _ = json.Marshal(input.Relationship.Attributes)
|
||||
}
|
||||
|
||||
_, err = tx.Exec(ctx, `
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
VALUES ($1, $2, $3, $4, now())`,
|
||||
mustUUID(input.Relationship.SourceID), mustUUID(input.Relationship.TargetID),
|
||||
input.Relationship.Type, attrsJSON)
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
|
||||
return domain.Relationship{}, errors.Join(domain.ErrAlreadyExists, err)
|
||||
}
|
||||
return domain.Relationship{}, err
|
||||
}
|
||||
|
||||
if err := writeSideEffects(ctx, tx, input.Relationship.SourceID, input.Audit, input.Event); err != nil {
|
||||
return domain.Relationship{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return domain.Relationship{}, err
|
||||
}
|
||||
input.Relationship.ValidFrom = time.Now()
|
||||
return input.Relationship, nil
|
||||
}
|
||||
|
||||
func (r *RelRepo) End(ctx context.Context, source, target domain.UUID, relType string) error {
|
||||
tx, err := r.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
result, err := sqlcgen.New(tx).EndCurrentRelationship(ctx, sqlcgen.EndCurrentRelationshipParams{
|
||||
SourceID: mustUUID(source), TargetID: mustUUID(target), Type: relType,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if result == 0 {
|
||||
return domain.ErrNotFound
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
func (r *RelRepo) ListFor(ctx context.Context, entityID domain.UUID, direction string) ([]domain.Relationship, error) {
|
||||
eid := mustUUID(entityID)
|
||||
switch direction {
|
||||
case "outbound":
|
||||
return queryRelsBySource(r.pool, eid)
|
||||
case "inbound":
|
||||
return queryRelsByTarget(r.pool, eid)
|
||||
default:
|
||||
return queryRelsBoth(r.pool, eid)
|
||||
}
|
||||
}
|
||||
|
||||
func queryRelsBySource(pool *Pool, eid uuid.UUID) ([]domain.Relationship, error) {
|
||||
rows, err := pool.Query(context.Background(),
|
||||
`SELECT source_id, target_id, type, attributes, valid_from, valid_to
|
||||
FROM relationships WHERE valid_to IS NULL AND source_id = $1 ORDER BY type`, eid)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanEdges(rows)
|
||||
}
|
||||
|
||||
func queryRelsByTarget(pool *Pool, eid uuid.UUID) ([]domain.Relationship, error) {
|
||||
rows, err := pool.Query(context.Background(),
|
||||
`SELECT source_id, target_id, type, attributes, valid_from, valid_to
|
||||
FROM relationships WHERE valid_to IS NULL AND target_id = $1 ORDER BY type`, eid)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanEdges(rows)
|
||||
}
|
||||
|
||||
func queryRelsBoth(pool *Pool, eid uuid.UUID) ([]domain.Relationship, error) {
|
||||
rows, err := pool.Query(context.Background(),
|
||||
`SELECT source_id, target_id, type, attributes, valid_from, valid_to
|
||||
FROM relationships WHERE valid_to IS NULL AND (source_id = $1 OR target_id = $1) ORDER BY type`, eid)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanEdges(rows)
|
||||
}
|
||||
|
||||
// OntologyRepo implements ports.OntologyStore with a TTL cache — entity
|
||||
// types change at seed time, not per request, so a short cache trades a
|
||||
// little staleness for avoiding the meta-schema load on every mutation.
|
||||
type OntologyRepo struct {
|
||||
pool *Pool
|
||||
ttl time.Duration
|
||||
|
||||
mu sync.Mutex
|
||||
loaded time.Time
|
||||
tree *ontology.TypeTree
|
||||
}
|
||||
|
||||
var _ ports.OntologyStore = (*OntologyRepo)(nil)
|
||||
|
||||
// NewOntologyRepo builds the ontology store with the given cache TTL
|
||||
// (values <= 0 disable caching).
|
||||
func NewOntologyRepo(pool *Pool, ttl time.Duration) *OntologyRepo {
|
||||
return &OntologyRepo{pool: pool, ttl: ttl}
|
||||
}
|
||||
|
||||
// LoadTypeTree returns the (possibly cached) ontology tree.
|
||||
func (o *OntologyRepo) LoadTypeTree(ctx context.Context) (ports.TypeTree, error) {
|
||||
if o.ttl <= 0 {
|
||||
return o.load(ctx)
|
||||
}
|
||||
o.mu.Lock()
|
||||
defer o.mu.Unlock()
|
||||
if o.tree != nil && time.Since(o.loaded) < o.ttl {
|
||||
return o.tree, nil
|
||||
}
|
||||
tree, err := o.load(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
o.tree = tree
|
||||
o.loaded = time.Now()
|
||||
return tree, nil
|
||||
}
|
||||
|
||||
func (o *OntologyRepo) load(ctx context.Context) (ports.TypeTree, error) {
|
||||
tx, err := o.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
tree, err := LoadTypeTree(ctx, tx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ports.TypeTree(tree), nil
|
||||
}
|
||||
@@ -6,7 +6,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/dtoro/oikos/internal/checkdefaults"
|
||||
"github.com/dtoro/oikos/internal/core/app"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
@@ -105,7 +105,7 @@ func IngestInventorySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*
|
||||
// Entities
|
||||
entities, _ := data["entities"].([]any)
|
||||
entityTypes := make(map[string]string) // slug -> type, for edge validation
|
||||
var pendingChecks []checkdefaults.Target
|
||||
var pendingChecks []app.CheckTarget
|
||||
for _, raw := range entities {
|
||||
eMap, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
@@ -157,8 +157,8 @@ func IngestInventorySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*
|
||||
// Default checks are deferred until after relationships are ingested:
|
||||
// a service has no address of its own and inherits its container's,
|
||||
// which means the hosting edge has to exist first.
|
||||
pendingChecks = append(pendingChecks, checkdefaults.Target{
|
||||
ID: entityID, Slug: slug, Type: typeName, Name: name, Attrs: attrsBytes,
|
||||
pendingChecks = append(pendingChecks, app.CheckTarget{
|
||||
ID: entityID.String(), Slug: slug, Type: typeName, Name: name, Attrs: attrsBytes,
|
||||
})
|
||||
|
||||
r.Entities++
|
||||
@@ -228,11 +228,11 @@ func IngestInventorySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*
|
||||
// transaction while surfacing as an unrelated failure several entities
|
||||
// later.
|
||||
for _, target := range pendingChecks {
|
||||
res, err := checkdefaults.Ensure(ctx, tx, tree, target)
|
||||
res, err := EnsureChecks(ctx, tx, tree, target)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("default checks for %s: %w", target.Slug, err)
|
||||
}
|
||||
checkdefaults.LogResult(target.Slug, target.Type, res)
|
||||
app.LogDeriveResult(target.Slug, target.Type, res)
|
||||
r.Checks += res.Created
|
||||
}
|
||||
|
||||
88
internal/adapters/postgres/signals.go
Normal file
88
internal/adapters/postgres/signals.go
Normal file
@@ -0,0 +1,88 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// MetricsRepo implements ports.MetricsRepository.
|
||||
type MetricsRepo struct {
|
||||
pool *Pool
|
||||
}
|
||||
|
||||
var _ ports.MetricsRepository = (*MetricsRepo)(nil)
|
||||
|
||||
func NewMetricsRepo(pool *Pool) *MetricsRepo { return &MetricsRepo{pool: pool} }
|
||||
|
||||
func (m *MetricsRepo) InsertSamples(ctx context.Context, entityID domain.UUID, samples []ports.MetricSample) error {
|
||||
for _, s := range samples {
|
||||
if err := sqlcgen.New(m.pool).InsertMetricSample(ctx, sqlcgen.InsertMetricSampleParams{
|
||||
EntityID: mustUUID(entityID), Metric: s.Metric, Value: s.Value,
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// SignalRepo implements ports.SignalRepository with inline SQL.
|
||||
type SignalRepo struct {
|
||||
pool *Pool
|
||||
}
|
||||
|
||||
var _ ports.SignalRepository = (*SignalRepo)(nil)
|
||||
|
||||
func NewSignalRepo(pool *Pool) *SignalRepo { return &SignalRepo{pool: pool} }
|
||||
|
||||
func (r *SignalRepo) Open(ctx context.Context) ([]domain.Signal, error) {
|
||||
rows, err := r.pool.Query(ctx,
|
||||
`SELECT entity_id, kind, severity, state FROM signals WHERE state NOT IN ('resolved','failed') ORDER BY severity DESC`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var items []domain.Signal
|
||||
for rows.Next() {
|
||||
var s domain.Signal
|
||||
var id uuid.UUID
|
||||
if err := rows.Scan(&id, &s.Kind, &s.Severity, &s.State); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.EntityID = domain.UUID(id.String())
|
||||
items = append(items, s)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func (r *SignalRepo) History(ctx context.Context, entityID domain.UUID, limit int) ([]domain.Signal, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (r *SignalRepo) UpsertWithTriggers(ctx context.Context, input ports.SignalUpsertInput) error {
|
||||
eid := mustUUID(input.Signal.EntityID)
|
||||
_, err := r.pool.Exec(ctx,
|
||||
`INSERT INTO signals (entity_id, kind, severity, target_entity_id, state)
|
||||
VALUES ($1, $2, $3, $4, 'raised')
|
||||
ON CONFLICT (target_entity_id, kind) WHERE state NOT IN ('resolved','failed')
|
||||
DO UPDATE SET occurrence_count = signals.occurrence_count + 1,
|
||||
last_seen_at = now(), updated_at = now()`,
|
||||
eid, input.Signal.Kind, input.Signal.Severity, eid)
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *SignalRepo) Transition(ctx context.Context, input ports.SignalTransitionInput) (domain.Signal, error) {
|
||||
tag, err := r.pool.Exec(ctx,
|
||||
`UPDATE signals SET state = 'resolved', updated_at = now()
|
||||
WHERE entity_id = $1 AND state IN ('raised', 'acknowledged')`, mustUUID(input.SignalID))
|
||||
if err != nil {
|
||||
return domain.Signal{}, err
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return domain.Signal{}, domain.ErrNotFound
|
||||
}
|
||||
return domain.Signal{}, nil
|
||||
}
|
||||
107
internal/adapters/postgres/splitsql_test.go
Normal file
107
internal/adapters/postgres/splitsql_test.go
Normal file
@@ -0,0 +1,107 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func nonEmpty(stmts []string) []string {
|
||||
var out []string
|
||||
for _, s := range stmts {
|
||||
if strings.TrimSpace(s) != "" {
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestSplitSQLBasic(t *testing.T) {
|
||||
stmts := nonEmpty(splitSQL("CREATE TABLE a (id int); CREATE TABLE b (id int);"))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLDollarQuotedFunction(t *testing.T) {
|
||||
sql := `CREATE FUNCTION f() RETURNS int AS $$
|
||||
SELECT 1; SELECT 2;
|
||||
$$ LANGUAGE sql;
|
||||
CREATE TABLE t (id int);`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
if !strings.Contains(stmts[0], "SELECT 1; SELECT 2;") {
|
||||
t.Errorf("dollar-quoted body was split: %q", stmts[0])
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLTaggedDollarQuote(t *testing.T) {
|
||||
sql := `DO $body$ BEGIN PERFORM 1; END $body$;SELECT 1;`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLSemicolonInComment(t *testing.T) {
|
||||
sql := "-- comment with ; semicolon\nCREATE TABLE t (id int); -- trailing; note\nSELECT 1;"
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLSemicolonInStringLiteral(t *testing.T) {
|
||||
sql := `SELECT 'hello; world'; INSERT INTO t VALUES (1);`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLDollarSignInStringLiteral(t *testing.T) {
|
||||
sql := `SELECT '$100'; SELECT 2;`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLBlockComment(t *testing.T) {
|
||||
sql := `SELECT 1; /* block; with; semicolons */ SELECT 2;`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLBlockCommentWithDollarQuote(t *testing.T) {
|
||||
sql := `/* $$ not a dollar quote */ SELECT 1;`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 1 {
|
||||
t.Fatalf("got %d statements, want 1: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLDoubledQuoteInString(t *testing.T) {
|
||||
sql := `SELECT 'O''Brien'; SELECT 2;`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLEmptyInput(t *testing.T) {
|
||||
stmts := nonEmpty(splitSQL(""))
|
||||
if len(stmts) != 0 {
|
||||
t.Fatalf("got %d statements, want 0", len(stmts))
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLNoSemicolon(t *testing.T) {
|
||||
stmts := nonEmpty(splitSQL("SELECT 1"))
|
||||
if len(stmts) != 1 {
|
||||
t.Fatalf("got %d statements, want 1", len(stmts))
|
||||
}
|
||||
}
|
||||
@@ -177,6 +177,52 @@ func (q *Queries) ListEntities(ctx context.Context, arg ListEntitiesParams) ([]E
|
||||
return items, nil
|
||||
}
|
||||
|
||||
const mergeEntityAttributes = `-- name: MergeEntityAttributes :execrows
|
||||
UPDATE entities SET
|
||||
attributes = attributes || $1::jsonb,
|
||||
updated_at = now()
|
||||
WHERE slug = $2
|
||||
`
|
||||
|
||||
type MergeEntityAttributesParams struct {
|
||||
Patch []byte
|
||||
Slug string
|
||||
}
|
||||
|
||||
// Shallow-merge a JSON patch into an entity's attributes (the
|
||||
// update_entity_attributes MCP/HTTP surface). Replaces the raw
|
||||
// `attributes = attributes || $2::jsonb` used in entity_tools.go.
|
||||
func (q *Queries) MergeEntityAttributes(ctx context.Context, arg MergeEntityAttributesParams) (int64, error) {
|
||||
result, err := q.db.Exec(ctx, mergeEntityAttributes, arg.Patch, arg.Slug)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
|
||||
const setEntityState = `-- name: SetEntityState :execrows
|
||||
UPDATE entities SET
|
||||
state = $1,
|
||||
updated_at = now()
|
||||
WHERE id = $2
|
||||
`
|
||||
|
||||
type SetEntityStateParams struct {
|
||||
State *string
|
||||
ID uuid.UUID
|
||||
}
|
||||
|
||||
// Set an entity's lifecycle state by id (the set_entity_state surface, run
|
||||
// after db.ValidateTransition). Replaces the raw
|
||||
// `UPDATE entities SET state = $2 ... WHERE id = $1`.
|
||||
func (q *Queries) SetEntityState(ctx context.Context, arg SetEntityStateParams) (int64, error) {
|
||||
result, err := q.db.Exec(ctx, setEntityState, arg.State, arg.ID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
|
||||
const updateEntity = `-- name: UpdateEntity :one
|
||||
UPDATE entities SET
|
||||
name = COALESCE($1, name),
|
||||
@@ -31,6 +31,42 @@ func (q *Queries) EndCurrentRelationship(ctx context.Context, arg EndCurrentRela
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
|
||||
const insertRelationshipIfAbsent = `-- name: InsertRelationshipIfAbsent :execrows
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT $1, $2, $3,
|
||||
$4::jsonb, now()
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM relationships
|
||||
WHERE source_id = $1
|
||||
AND target_id = $2
|
||||
AND type = $3
|
||||
AND valid_to IS NULL
|
||||
)
|
||||
`
|
||||
|
||||
type InsertRelationshipIfAbsentParams struct {
|
||||
SourceID uuid.UUID
|
||||
TargetID uuid.UUID
|
||||
Type string
|
||||
Attributes []byte
|
||||
}
|
||||
|
||||
// Idempotent relationship insert (the create_relationship surface): no-op if
|
||||
// an active edge of the same source/target/type already exists. Replaces the
|
||||
// raw INSERT...WHERE NOT EXISTS used in entity_tools.go.
|
||||
func (q *Queries) InsertRelationshipIfAbsent(ctx context.Context, arg InsertRelationshipIfAbsentParams) (int64, error) {
|
||||
result, err := q.db.Exec(ctx, insertRelationshipIfAbsent,
|
||||
arg.SourceID,
|
||||
arg.TargetID,
|
||||
arg.Type,
|
||||
arg.Attributes,
|
||||
)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
|
||||
const listEntityRelations = `-- name: ListEntityRelations :many
|
||||
SELECT se.slug AS source_slug, te.slug AS target_slug, r.type, r.attributes,
|
||||
r.valid_from, r.valid_to
|
||||
171
internal/adapters/probes/network.go
Normal file
171
internal/adapters/probes/network.go
Normal file
@@ -0,0 +1,171 @@
|
||||
// Package probes implements the Checker port for each check kind.
|
||||
// Each file exports a Checker constructor (e.g. NewHTTPChecker) that
|
||||
// returns ports.Checker wrapping the scheduler's probe logic.
|
||||
package probes
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
)
|
||||
|
||||
// httpChecker is the HTTP reachability probe.
|
||||
type httpChecker struct {
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
// NewHTTPChecker builds an HTTP probe with a connection-scoped client.
|
||||
func NewHTTPChecker() ports.Checker {
|
||||
return &httpChecker{
|
||||
client: &http.Client{
|
||||
Timeout: 10 * time.Second,
|
||||
Transport: &http.Transport{
|
||||
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (c *httpChecker) Check(ctx context.Context, def ports.CheckDef, _ ports.Target) ports.CheckResult {
|
||||
cfg := configMap(def)
|
||||
|
||||
url, _ := parseStr(cfg, "url")
|
||||
if url == "" {
|
||||
return ports.CheckResult{State: "unknown", Message: "no url in config"}
|
||||
}
|
||||
maxStatus := int(parseFloat(cfg, "max_status", 500))
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return ports.CheckResult{State: "unknown", Message: err.Error()}
|
||||
}
|
||||
|
||||
resp, err := c.client.Do(req)
|
||||
if err != nil {
|
||||
return ports.CheckResult{State: "unknown", Message: err.Error()}
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
io.Copy(io.Discard, resp.Body)
|
||||
|
||||
code := resp.StatusCode
|
||||
if code > maxStatus {
|
||||
return ports.CheckResult{
|
||||
State: "critical", Value: float64(code),
|
||||
Message: fmt.Sprintf("HTTP %d exceeds max_status %d", code, maxStatus),
|
||||
}
|
||||
}
|
||||
return ports.CheckResult{State: "ok", Value: float64(code)}
|
||||
}
|
||||
|
||||
// tcpChecker checks TCP port reachability.
|
||||
type tcpChecker struct{}
|
||||
|
||||
func NewTCPChecker() ports.Checker { return &tcpChecker{} }
|
||||
|
||||
func (c *tcpChecker) Check(ctx context.Context, def ports.CheckDef, _ ports.Target) ports.CheckResult {
|
||||
cfg := configMap(def)
|
||||
|
||||
host, _ := parseStr(cfg, "host")
|
||||
port := int(parseFloat(cfg, "port", 0))
|
||||
timeout := time.Duration(parseFloat(cfg, "timeout", 5)) * time.Second
|
||||
|
||||
addr := fmt.Sprintf("%s:%d", host, port)
|
||||
d := &net.Dialer{Timeout: timeout}
|
||||
conn, err := d.DialContext(ctx, "tcp", addr)
|
||||
if err != nil {
|
||||
return ports.CheckResult{State: "critical", Message: err.Error()}
|
||||
}
|
||||
conn.Close()
|
||||
return ports.CheckResult{State: "ok", Value: float64(port)}
|
||||
}
|
||||
|
||||
// pingChecker is the ICMP/connectivity probe. Falls back to TCP ping on
|
||||
// systems without raw socket access.
|
||||
type pingChecker struct{}
|
||||
|
||||
func NewPingChecker() ports.Checker { return &pingChecker{} }
|
||||
|
||||
func (c *pingChecker) Check(ctx context.Context, def ports.CheckDef, _ ports.Target) ports.CheckResult {
|
||||
cfg := configMap(def)
|
||||
|
||||
host, _ := parseStr(cfg, "host")
|
||||
if host == "" {
|
||||
return ports.CheckResult{State: "unknown", Message: "no host in config"}
|
||||
}
|
||||
port := int(parseFloat(cfg, "port", 80))
|
||||
timeout := time.Duration(parseFloat(cfg, "timeout", 5)) * time.Second
|
||||
|
||||
d := &net.Dialer{Timeout: timeout}
|
||||
addr := fmt.Sprintf("%s:%d", host, port)
|
||||
conn, err := d.DialContext(ctx, "tcp", addr)
|
||||
if err != nil {
|
||||
return ports.CheckResult{State: "critical", Message: err.Error(), Value: -1}
|
||||
}
|
||||
conn.Close()
|
||||
return ports.CheckResult{State: "ok", Value: 0}
|
||||
}
|
||||
|
||||
// dnsChecker resolves DNS names.
|
||||
type dnsChecker struct{}
|
||||
|
||||
func NewDNSChecker() ports.Checker { return &dnsChecker{} }
|
||||
|
||||
func (c *dnsChecker) Check(ctx context.Context, def ports.CheckDef, _ ports.Target) ports.CheckResult {
|
||||
cfg := configMap(def)
|
||||
|
||||
name, _ := parseStr(cfg, "name")
|
||||
if name == "" {
|
||||
return ports.CheckResult{State: "unknown", Message: "no name in config"}
|
||||
}
|
||||
var r net.Resolver
|
||||
addrs, err := r.LookupHost(ctx, name)
|
||||
if err != nil {
|
||||
return ports.CheckResult{State: "critical", Message: err.Error()}
|
||||
}
|
||||
return ports.CheckResult{State: "ok", Value: float64(len(addrs))}
|
||||
}
|
||||
|
||||
|
||||
// configMap unmarshals a check's JSON config into a map.
|
||||
func configMap(def ports.CheckDef) map[string]any {
|
||||
var m map[string]any
|
||||
if len(def.Config) > 0 {
|
||||
json.Unmarshal(def.Config, &m)
|
||||
}
|
||||
if m == nil {
|
||||
m = map[string]any{}
|
||||
}
|
||||
return m
|
||||
}
|
||||
func parseStr(m map[string]any, key string) (string, bool) {
|
||||
v, ok := m[key]
|
||||
if !ok {
|
||||
return "", false
|
||||
}
|
||||
s, ok := v.(string)
|
||||
return s, ok
|
||||
}
|
||||
|
||||
func parseFloat(m map[string]any, key string, def float64) float64 {
|
||||
v, ok := m[key]
|
||||
if !ok {
|
||||
return def
|
||||
}
|
||||
switch n := v.(type) {
|
||||
case float64:
|
||||
return n
|
||||
case int:
|
||||
return float64(n)
|
||||
case json.Number:
|
||||
f, _ := n.Float64()
|
||||
return f
|
||||
}
|
||||
return def
|
||||
}
|
||||
105
internal/adapters/probes/ssh.go
Normal file
105
internal/adapters/probes/ssh.go
Normal file
@@ -0,0 +1,105 @@
|
||||
package probes
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/actuator"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
// SSHChecker runs ssh-script probes via the actuator. Constructed with a
|
||||
// key source so the caller controls SSH key resolution.
|
||||
type SSHChecker struct {
|
||||
pool *actuator.DialPool
|
||||
signerFn func() (ssh.Signer, error)
|
||||
}
|
||||
|
||||
func NewSSHChecker(pool *actuator.DialPool, signerFn func() (ssh.Signer, error)) *SSHChecker {
|
||||
return &SSHChecker{pool: pool, signerFn: signerFn}
|
||||
}
|
||||
|
||||
func (c *SSHChecker) Check(ctx context.Context, def ports.CheckDef, target ports.Target) ports.CheckResult {
|
||||
cfg := configMap(def)
|
||||
script, _ := parseStr(cfg, "script")
|
||||
if script == "" {
|
||||
return ports.CheckResult{State: "unknown", Message: "no script in config"}
|
||||
}
|
||||
|
||||
signer, err := c.signerFn()
|
||||
if err != nil {
|
||||
return ports.CheckResult{State: "unknown", Message: fmt.Sprintf("signer: %v", err)}
|
||||
}
|
||||
|
||||
client, err := actuator.Dial(ctx, actuator.DialOptions{
|
||||
Host: target.Host,
|
||||
User: target.User,
|
||||
Signer: signer,
|
||||
})
|
||||
if err != nil {
|
||||
return ports.CheckResult{State: "unknown", Message: fmt.Sprintf("dial %s: %v", target.Host, err)}
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
cmd := fmt.Sprintf("/opt/oikos/checks/%s", script)
|
||||
if target.Wrap != nil {
|
||||
cmd = target.Wrap(cmd)
|
||||
}
|
||||
|
||||
output, err := actuator.RunCombinedOutput(ctx, client, cmd)
|
||||
if err != nil {
|
||||
return ports.CheckResult{
|
||||
Value: -1,
|
||||
State: "critical",
|
||||
Message: err.Error(),
|
||||
}
|
||||
}
|
||||
return parseSSHResult(output)
|
||||
}
|
||||
|
||||
func parseSSHResult(output []byte) ports.CheckResult {
|
||||
line := strings.TrimSpace(string(output))
|
||||
var value float64
|
||||
if len(line) > 0 {
|
||||
parts := strings.SplitN(line, " ", 3)
|
||||
switch parts[0] {
|
||||
case "OK":
|
||||
if len(parts) > 1 {
|
||||
fmt.Sscanf(parts[1], "%f", &value)
|
||||
}
|
||||
return ports.CheckResult{State: "ok", Value: value, Message: line}
|
||||
case "WARN":
|
||||
if len(parts) > 1 {
|
||||
fmt.Sscanf(parts[1], "%f", &value)
|
||||
}
|
||||
return ports.CheckResult{State: "warning", Value: value, Message: line}
|
||||
case "CRIT":
|
||||
if len(parts) > 1 {
|
||||
fmt.Sscanf(parts[1], "%f", &value)
|
||||
}
|
||||
return ports.CheckResult{State: "critical", Value: value, Message: line}
|
||||
}
|
||||
}
|
||||
return ports.CheckResult{State: "unknown", Message: "unparseable output"}
|
||||
}
|
||||
|
||||
// Registry maps check kinds to Checker implementations.
|
||||
type Registry map[string]ports.Checker
|
||||
|
||||
func NewRegistry() Registry {
|
||||
return Registry{
|
||||
"http": NewHTTPChecker(),
|
||||
"tcp": NewTCPChecker(),
|
||||
"ping": NewPingChecker(),
|
||||
"dns": NewDNSChecker(),
|
||||
"vm-status": nil,
|
||||
"ssh-script": nil,
|
||||
"backup-freshness": nil,
|
||||
"cert-expiry": nil,
|
||||
}
|
||||
}
|
||||
|
||||
func (r Registry) Get(kind string) ports.Checker { return r[kind] }
|
||||
func (r Registry) Register(kind string, c ports.Checker) { r[kind] = c }
|
||||
63
internal/adapters/remote/resolver.go
Normal file
63
internal/adapters/remote/resolver.go
Normal file
@@ -0,0 +1,63 @@
|
||||
// Package remote implements ports.TargetResolver over internal/remote.
|
||||
// The resolver logic (address preference, guest wrapping, hosting-compute
|
||||
// walks) is unchanged; this adapter maps its results onto the port types.
|
||||
// When the postgres repositories land (Phase 3+), the underlying functions
|
||||
// move into this package on top of ports.EntityRepository.
|
||||
package remote
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
postgres "github.com/dtoro/oikos/internal/adapters/postgres"
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
intremote "github.com/dtoro/oikos/internal/remote"
|
||||
)
|
||||
|
||||
// Resolver resolves execution targets from the entity graph.
|
||||
type Resolver struct {
|
||||
pool *postgres.Pool
|
||||
}
|
||||
|
||||
var _ ports.TargetResolver = (*Resolver)(nil)
|
||||
|
||||
// NewResolver builds a resolver over the postgres pool.
|
||||
func NewResolver(pool *postgres.Pool) *Resolver { return &Resolver{pool: pool} }
|
||||
|
||||
func toPort(t intremote.ExecTarget) ports.Target {
|
||||
return ports.Target{Host: t.Host, User: t.User, Wrap: t.Wrap}
|
||||
}
|
||||
|
||||
// ResolveExecTarget resolves a slug to its execution endpoint.
|
||||
func (r *Resolver) ResolveExecTarget(ctx context.Context, targetSlug string) (ports.Target, error) {
|
||||
t, err := intremote.ResolveExecTarget(ctx, r.pool, targetSlug, intremote.DefaultUser)
|
||||
if err != nil {
|
||||
return ports.Target{}, err
|
||||
}
|
||||
return toPort(t), nil
|
||||
}
|
||||
|
||||
// ResolveForCheck resolves a check's entity (by ID and type) to its endpoint.
|
||||
func (r *Resolver) ResolveForCheck(ctx context.Context, targetID domain.UUID, targetType string) (ports.Target, error) {
|
||||
id, err := uuid.Parse(string(targetID))
|
||||
if err != nil {
|
||||
return ports.Target{}, err
|
||||
}
|
||||
t, err := intremote.ResolveExecTargetForCheck(ctx, r.pool, id, targetType, intremote.DefaultUser)
|
||||
if err != nil {
|
||||
return ports.Target{}, err
|
||||
}
|
||||
return toPort(t), nil
|
||||
}
|
||||
|
||||
// ResolveHost resolves a host slug to address and SSH user.
|
||||
func (r *Resolver) ResolveHost(ctx context.Context, hostSlug, fallbackUser string) (string, string, error) {
|
||||
return intremote.ResolveHost(ctx, r.pool, hostSlug, fallbackUser)
|
||||
}
|
||||
|
||||
// IsGuest reports whether an entity type is reached via pct/qm exec.
|
||||
func (r *Resolver) IsGuest(entityType string) bool {
|
||||
return intremote.IsGuest(entityType)
|
||||
}
|
||||
114
internal/adapters/ssh/executor.go
Normal file
114
internal/adapters/ssh/executor.go
Normal file
@@ -0,0 +1,114 @@
|
||||
// Package ssh implements ports.CommandExecutor over internal/actuator:
|
||||
// the dial pool, host-key handling, and streaming/combined execution.
|
||||
package ssh
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
cryptossh "golang.org/x/crypto/ssh"
|
||||
|
||||
"github.com/dtoro/oikos/internal/actuator"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultExecTimeout = 10 * time.Minute
|
||||
defaultKeyPathEnv = "OIKOS_SSH_KEY_PATH"
|
||||
defaultKeyPath = "/etc/oikos/ssh_key"
|
||||
)
|
||||
|
||||
// SignerSource supplies the SSH signer used for all dials. The secrets
|
||||
// adapter provides one backed by Infisical/SOPS; tests inject a static one.
|
||||
type SignerSource func(ctx context.Context) (cryptossh.Signer, error)
|
||||
|
||||
// FileSignerSource reads an OpenSSH private key from disk once and parses
|
||||
// it (path from env OIKOS_SSH_KEY_PATH, default /etc/oikos/ssh_key — the
|
||||
// same resolution the httpapi path used before the extraction).
|
||||
func FileSignerSource() SignerSource {
|
||||
var (
|
||||
once sync.Once
|
||||
signer cryptossh.Signer
|
||||
err error
|
||||
)
|
||||
return func(context.Context) (cryptossh.Signer, error) {
|
||||
once.Do(func() {
|
||||
path := os.Getenv(defaultKeyPathEnv)
|
||||
if path == "" {
|
||||
path = defaultKeyPath
|
||||
}
|
||||
key, rerr := os.ReadFile(path)
|
||||
if rerr != nil {
|
||||
err = fmt.Errorf("read ssh key %s: %w", path, rerr)
|
||||
return
|
||||
}
|
||||
signer, err = actuator.LoadSignerFromBytes(key)
|
||||
})
|
||||
return signer, err
|
||||
}
|
||||
}
|
||||
|
||||
// Executor runs commands over SSH through a dial pool.
|
||||
type Executor struct {
|
||||
signer SignerSource
|
||||
pool *actuator.DialPool
|
||||
}
|
||||
|
||||
var _ ports.CommandExecutor = (*Executor)(nil)
|
||||
|
||||
// NewExecutor builds an executor. The dial pool reuses connections per
|
||||
// host/user for the given TTL.
|
||||
func NewExecutor(signer SignerSource, poolTTL time.Duration) *Executor {
|
||||
return &Executor{
|
||||
signer: signer,
|
||||
pool: actuator.NewDialPool(poolTTL),
|
||||
}
|
||||
}
|
||||
|
||||
// Close releases pooled connections.
|
||||
func (e *Executor) Close() { e.pool.Close() }
|
||||
|
||||
// Run dials the target (via the pool), wraps the command for transport when
|
||||
// the target needs it (pct/qm guests), executes with streaming output, and
|
||||
// maps the outcome onto ports.ExecResult.
|
||||
func (e *Executor) Run(ctx context.Context, target ports.Target, command string, opts ports.ExecOpts) ports.ExecResult {
|
||||
start := time.Now()
|
||||
|
||||
signer, err := e.signer(ctx)
|
||||
if err != nil {
|
||||
return ports.ExecResult{Err: err, Duration: time.Since(start)}
|
||||
}
|
||||
|
||||
client, err := e.pool.Get(ctx, actuator.DialOptions{
|
||||
Host: target.Host,
|
||||
User: target.User,
|
||||
Signer: signer,
|
||||
})
|
||||
if err != nil {
|
||||
return ports.ExecResult{Err: err, Duration: time.Since(start)}
|
||||
}
|
||||
// Pooled client: do not close here; the pool evicts on TTL.
|
||||
|
||||
if target.Wrap != nil {
|
||||
command = target.Wrap(command)
|
||||
}
|
||||
|
||||
timeout := opts.Timeout
|
||||
if timeout <= 0 {
|
||||
timeout = defaultExecTimeout
|
||||
}
|
||||
|
||||
output, runErr := actuator.RunStreaming(ctx, client, command, opts.Sink, timeout)
|
||||
if runErr != nil {
|
||||
slog.Debug("ssh exec: command failed", "host", target.Host, "error", runErr)
|
||||
}
|
||||
return ports.ExecResult{
|
||||
Output: output,
|
||||
Duration: time.Since(start),
|
||||
Err: runErr,
|
||||
}
|
||||
}
|
||||
@@ -13,7 +13,7 @@ package audit
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres"
|
||||
)
|
||||
|
||||
// Finding is one drift item the operator should look at.
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/adapters/postgres"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
|
||||
@@ -30,6 +30,17 @@ type Config struct {
|
||||
// port than the API); a no-op when the SPA and API share an origin.
|
||||
CORSAllowedOrigin string
|
||||
|
||||
// Rate limiting (plan D3). APIRateLimit is the per-IP requests/sec cap;
|
||||
// APIRateBurst is the token-bucket burst (defaults to 2x the limit when
|
||||
// unset). A limit of 0 disables rate limiting entirely.
|
||||
APIRateLimit int
|
||||
APIRateBurst int
|
||||
|
||||
// Health probe HTTP listener (plan D5). Background-loop roles (scheduler,
|
||||
// execution-worker) expose a staleness-aware /healthz here. Empty disables the
|
||||
// health server (local/non-docker runs).
|
||||
HealthListen string
|
||||
|
||||
// Observability
|
||||
Debug bool // verbose logging, probe payloads, SQL
|
||||
|
||||
@@ -42,12 +53,6 @@ type Config struct {
|
||||
// Scheduler (Phase 3)
|
||||
SchedulerInterval time.Duration // check loop interval (default 30s)
|
||||
|
||||
// Notifier (Phase 3)
|
||||
MatrixHomeserver string // Matrix server URL
|
||||
MatrixUserID string // bot user ID (e.g. @oikos:matrix.hubris.network)
|
||||
MatrixToken string // Matrix access token
|
||||
MatrixRoomID string // alert room ID
|
||||
|
||||
// Actuator (Phase 3)
|
||||
SSHKeyPath string // path to the restricted SSH key
|
||||
SSHUser string // SSH user on targets (default "oikos")
|
||||
@@ -57,9 +62,6 @@ type Config struct {
|
||||
// Learning (Phase 3)
|
||||
LearningInterval time.Duration // pattern extraction interval (default 3600s)
|
||||
|
||||
// Approval HMAC secret (Phase 3)
|
||||
ApprovalHMACSecret string
|
||||
|
||||
// Nomos agent entity ID (Phase 4)
|
||||
NomosAgentID string
|
||||
NomosAgentSlug string
|
||||
@@ -121,6 +123,11 @@ func FromEnv() Config {
|
||||
if v := os.Getenv("OIKOS_CORS_ORIGIN"); v != "" {
|
||||
c.CORSAllowedOrigin = v
|
||||
}
|
||||
c.APIRateLimit = parseInt(os.Getenv("OIKOS_API_RATE_LIMIT"))
|
||||
c.APIRateBurst = parseInt(os.Getenv("OIKOS_API_RATE_BURST"))
|
||||
if v := os.Getenv("OIKOS_HEALTH_LISTEN"); v != "" {
|
||||
c.HealthListen = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_SEEDS_DIR"); v != "" {
|
||||
c.SeedsDir = v
|
||||
}
|
||||
@@ -132,18 +139,6 @@ func FromEnv() Config {
|
||||
c.SchedulerInterval = d
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("OIKOS_MATRIX_HOMESERVER"); v != "" {
|
||||
c.MatrixHomeserver = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_MATRIX_USER"); v != "" {
|
||||
c.MatrixUserID = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_MATRIX_TOKEN"); v != "" {
|
||||
c.MatrixToken = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_MATRIX_ROOM"); v != "" {
|
||||
c.MatrixRoomID = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_SSH_KEY_PATH"); v != "" {
|
||||
c.SSHKeyPath = v
|
||||
}
|
||||
@@ -161,9 +156,6 @@ func FromEnv() Config {
|
||||
c.LearningInterval = d
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("OIKOS_APPROVAL_HMAC_SECRET"); v != "" {
|
||||
c.ApprovalHMACSecret = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_NOMOS_AGENT_ID"); v != "" {
|
||||
c.NomosAgentID = v
|
||||
}
|
||||
|
||||
@@ -1,23 +1,11 @@
|
||||
// Package checkdefaults derives an entity's default check_defs from the
|
||||
// monitoring kinds its type declares in seeds/ontology.yaml.
|
||||
//
|
||||
// The type says WHAT to watch (`service: [http, process]`); this package
|
||||
// works out HOW — which concrete check_defs rows to write, and what host,
|
||||
// script or URL each needs. Deriving config here rather than in YAML keeps
|
||||
// the ontology declarative and keeps address resolution (which has to walk
|
||||
// the graph) in code.
|
||||
package checkdefaults
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/ontology"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// Semantic monitoring kinds, as declared on entity types. These are not
|
||||
@@ -33,6 +21,8 @@ const (
|
||||
KindBackup = "backup-freshness"
|
||||
KindCertExpiry = "cert-expiry"
|
||||
KindVMStatus = "vm-status"
|
||||
KindQuorum = "quorum"
|
||||
KindDNS = "dns"
|
||||
)
|
||||
|
||||
// defaultBackupMaxAge is how long a backup target may go without a new
|
||||
@@ -40,9 +30,9 @@ const (
|
||||
// override per target with `backup_max_age_s` in the entity's attributes.
|
||||
const defaultBackupMaxAge = 86400
|
||||
|
||||
// Target is the entity default checks are being ensured for.
|
||||
type Target struct {
|
||||
ID uuid.UUID
|
||||
// CheckTarget is the entity default checks are being derived for.
|
||||
type CheckTarget struct {
|
||||
ID string
|
||||
Slug string
|
||||
Type string
|
||||
// Name is the entity's name column, not an attribute. The old code read
|
||||
@@ -53,9 +43,16 @@ type Target struct {
|
||||
Attrs []byte
|
||||
}
|
||||
|
||||
// Result reports what Ensure did, so callers can log a type that declared
|
||||
// monitoring but produced nothing instead of failing silently.
|
||||
type Result struct {
|
||||
// CheckDef is one concrete derived check: kind, config payload, interval.
|
||||
type CheckDef struct {
|
||||
Kind string
|
||||
Config map[string]any
|
||||
IntervalS int
|
||||
}
|
||||
|
||||
// DeriveResult reports what Derive produced, so callers can log a type that
|
||||
// declared monitoring but produced nothing instead of failing silently.
|
||||
type DeriveResult struct {
|
||||
Created int
|
||||
// Skipped records kinds that were declared but could not be built, with
|
||||
// the reason. A non-empty Skipped on an active entity is a real gap.
|
||||
@@ -71,35 +68,27 @@ type Skip struct {
|
||||
Reason string
|
||||
}
|
||||
|
||||
type checkDef struct {
|
||||
kind string
|
||||
config map[string]any
|
||||
interval int32
|
||||
}
|
||||
// HostLookup resolves the hosting entity's attributes when the entity
|
||||
// itself carries no address (a service lives on its container; a backup
|
||||
// target on whatever writes to it). It is invoked lazily — only when the
|
||||
// entity's own attributes lack a host — so the pure derivation below stays
|
||||
// separated from the graph read the caller performs.
|
||||
type HostLookup func() map[string]any
|
||||
|
||||
// Ensure writes the default check_defs for one entity, idempotently.
|
||||
//
|
||||
// Returns the number of checks created. An entity whose type declares
|
||||
// monitoring it cannot satisfy comes back with a populated Skipped rather
|
||||
// than an error — a missing address is a modelling gap, not a failure of
|
||||
// this call.
|
||||
func Ensure(ctx context.Context, tx pgx.Tx, tree *ontology.TypeTree, t Target) (Result, error) {
|
||||
var res Result
|
||||
|
||||
if _, err := tx.Exec(ctx,
|
||||
`INSERT INTO entity_status (entity_id, health, updated_at)
|
||||
VALUES ($1, 'unknown', now())
|
||||
ON CONFLICT (entity_id) DO NOTHING`, t.ID); err != nil {
|
||||
return res, fmt.Errorf("entity_status %s: %w", t.Slug, err)
|
||||
}
|
||||
// Derive computes the default checks for one entity from its type's
|
||||
// monitoring spec (with per-entity `monitoring` attribute overrides).
|
||||
// lookup may be nil. It performs no I/O of its own; the caller's lookup
|
||||
// thunk may. The postgres adapter pairs this with writeCheck upserts.
|
||||
func Derive(tree *ontology.TypeTree, t CheckTarget, lookup HostLookup) ([]CheckDef, DeriveResult) {
|
||||
var res DeriveResult
|
||||
|
||||
mon := tree.Monitoring(t.Type)
|
||||
if !mon.Declared {
|
||||
res.Undeclared = true
|
||||
return res, nil
|
||||
return nil, res
|
||||
}
|
||||
if mon.None() {
|
||||
return res, nil
|
||||
return nil, res
|
||||
}
|
||||
|
||||
var attrs map[string]any
|
||||
@@ -118,18 +107,15 @@ func Ensure(ctx context.Context, tx pgx.Tx, tree *ontology.TypeTree, t Target) (
|
||||
if mo, ok := attrs["monitoring"]; ok {
|
||||
mon = resolveMonitoringAttr(mo, mon)
|
||||
if mon.None() {
|
||||
return res, nil
|
||||
return nil, res
|
||||
}
|
||||
}
|
||||
|
||||
// A service has no address of its own — it lives on the container that
|
||||
// provides it. Fall back to the graph before giving up.
|
||||
host := resolveHost(attrs)
|
||||
if host == "" {
|
||||
hostAttrs, err := hostViaGraph(ctx, tx, t.ID)
|
||||
if err != nil {
|
||||
return res, fmt.Errorf("resolve host for %s: %w", t.Slug, err)
|
||||
}
|
||||
if host == "" && lookup != nil {
|
||||
hostAttrs := lookup()
|
||||
host = resolveHost(hostAttrs)
|
||||
if user := resolveSSHUser(hostAttrs); host != "" && user != "root" {
|
||||
attrs["ssh"] = hostAttrs["ssh"]
|
||||
@@ -138,7 +124,7 @@ func Ensure(ctx context.Context, tx pgx.Tx, tree *ontology.TypeTree, t Target) (
|
||||
user := resolveSSHUser(attrs)
|
||||
port := resolveSSHPort(attrs)
|
||||
|
||||
var defs []checkDef
|
||||
var defs []CheckDef
|
||||
for _, kind := range mon.Kinds {
|
||||
built, reason := buildKind(kind, t, attrs, host, user, port)
|
||||
if len(built) == 0 {
|
||||
@@ -147,17 +133,7 @@ func Ensure(ctx context.Context, tx pgx.Tx, tree *ontology.TypeTree, t Target) (
|
||||
}
|
||||
defs = append(defs, built...)
|
||||
}
|
||||
|
||||
for i, def := range defs {
|
||||
created, err := writeCheck(ctx, tx, t, i, def)
|
||||
if err != nil {
|
||||
return res, fmt.Errorf("check %s/%s: %w", t.Slug, def.kind, err)
|
||||
}
|
||||
if created {
|
||||
res.Created++
|
||||
}
|
||||
}
|
||||
return res, nil
|
||||
return defs, res
|
||||
}
|
||||
|
||||
// resolveMonitoringAttr turns an entity's `monitoring` attribute into a
|
||||
@@ -181,10 +157,10 @@ func resolveMonitoringAttr(v any, fallback ontology.MonitoringResolution) ontolo
|
||||
return fallback
|
||||
}
|
||||
|
||||
// buildKind turns one declared semantic kind into concrete check_defs, or
|
||||
// buildKind turns one declared semantic kind into concrete checks, or
|
||||
// returns the reason it could not.
|
||||
func buildKind(kind string, t Target, attrs map[string]any, host, user string, port int) ([]checkDef, string) {
|
||||
ssh := func(script string, args ...string) checkDef {
|
||||
func buildKind(kind string, t CheckTarget, attrs map[string]any, host, user string, port int) ([]CheckDef, string) {
|
||||
ssh := func(script string, args ...string) CheckDef {
|
||||
cfg := map[string]any{"script": script, "host": host}
|
||||
if user != "" && user != "root" {
|
||||
cfg["user"] = user
|
||||
@@ -195,7 +171,7 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
if len(args) > 0 && args[0] != "" {
|
||||
cfg["args"] = args[0]
|
||||
}
|
||||
return checkDef{kind: "ssh-script", config: cfg, interval: 60}
|
||||
return CheckDef{Kind: "ssh-script", Config: cfg, IntervalS: 60}
|
||||
}
|
||||
|
||||
switch kind {
|
||||
@@ -203,13 +179,13 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
if host == "" {
|
||||
return nil, "no address on the entity or its host"
|
||||
}
|
||||
return []checkDef{{kind: "ping", config: map[string]any{"host": host}, interval: 30}}, ""
|
||||
return []CheckDef{{Kind: "ping", Config: map[string]any{"host": host}, IntervalS: 30}}, ""
|
||||
|
||||
case KindResource:
|
||||
if host == "" {
|
||||
return nil, "no address on the entity or its host"
|
||||
}
|
||||
return []checkDef{
|
||||
return []CheckDef{
|
||||
ssh("cpu_check.sh"), ssh("memory_check.sh"),
|
||||
ssh("load_check.sh"), ssh("disk_usage_check.sh"),
|
||||
}, ""
|
||||
@@ -223,14 +199,14 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
// mirror hits per machine per day to answer a question whose answer
|
||||
// changes about once a day.
|
||||
u := ssh("updates_check.sh")
|
||||
u.interval = 86400
|
||||
return []checkDef{u}, ""
|
||||
u.IntervalS = 86400
|
||||
return []CheckDef{u}, ""
|
||||
|
||||
case KindCapacity:
|
||||
if host == "" {
|
||||
return nil, "no address on the entity or its host"
|
||||
}
|
||||
return []checkDef{ssh("disk_usage_check.sh")}, ""
|
||||
return []CheckDef{ssh("disk_usage_check.sh")}, ""
|
||||
|
||||
case KindProcess:
|
||||
if host == "" {
|
||||
@@ -265,7 +241,7 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
}
|
||||
// process_check.sh takes the unit/container name as $1 and reports
|
||||
// "unknown" without it.
|
||||
return []checkDef{ssh("process_check.sh", unit)}, ""
|
||||
return []CheckDef{ssh("process_check.sh", unit)}, ""
|
||||
|
||||
case KindBackup:
|
||||
// A backup target is checked from the machine that writes to it, so it
|
||||
@@ -291,7 +267,7 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
}
|
||||
// Daily. The freshness budget itself is a day, so probing more often
|
||||
// cannot surface anything sooner — it just costs an SSH round trip.
|
||||
return []checkDef{{kind: "backup-freshness", config: cfg, interval: 86400}}, ""
|
||||
return []CheckDef{{Kind: "backup-freshness", Config: cfg, IntervalS: 86400}}, ""
|
||||
|
||||
case KindHTTP:
|
||||
url := httpURL(t, attrs)
|
||||
@@ -300,10 +276,27 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
}
|
||||
// max_status rather than an exact expected_status: most services sit
|
||||
// behind Authentik and answer 302/401, which is a working service.
|
||||
return []checkDef{{
|
||||
kind: "http",
|
||||
config: map[string]any{"url": url, "max_status": 500},
|
||||
interval: 60,
|
||||
return []CheckDef{{
|
||||
Kind: "http",
|
||||
Config: map[string]any{"url": url, "max_status": 500},
|
||||
IntervalS: 60,
|
||||
}}, ""
|
||||
|
||||
case KindDNS:
|
||||
// Resolve the entity's name via DNS to verify the zone is reachable.
|
||||
// Uses the entity name (zone apex) or falls back to the slug.
|
||||
name := t.Name
|
||||
if name == "" {
|
||||
name = strings.TrimPrefix(t.Slug, "zone:")
|
||||
}
|
||||
if name == "" {
|
||||
return nil, "no name to resolve"
|
||||
}
|
||||
return []CheckDef{{
|
||||
Kind: "dns",
|
||||
Config: map[string]any{"name": name},
|
||||
IntervalS: 300, // 5 min — DNS changes are rare; the cost of a miss
|
||||
// is a stale IP, not a service outage.
|
||||
}}, ""
|
||||
|
||||
case KindCertExpiry:
|
||||
@@ -324,10 +317,10 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
if dial != "" {
|
||||
config["dial"] = dial
|
||||
}
|
||||
return []checkDef{{
|
||||
kind: "cert-expiry",
|
||||
config: config,
|
||||
interval: 3600,
|
||||
return []CheckDef{{
|
||||
Kind: "cert-expiry",
|
||||
Config: config,
|
||||
IntervalS: 3600,
|
||||
}}, ""
|
||||
|
||||
case KindVMStatus:
|
||||
@@ -337,18 +330,27 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
if _, ok := attrs["pve_id"]; !ok {
|
||||
return nil, "no pve_id to run qm status"
|
||||
}
|
||||
return []checkDef{{
|
||||
kind: "vm-status",
|
||||
config: map[string]any{},
|
||||
interval: 60,
|
||||
return []CheckDef{{
|
||||
Kind: "vm-status",
|
||||
Config: map[string]any{},
|
||||
IntervalS: 60,
|
||||
}}, ""
|
||||
|
||||
case KindQuorum:
|
||||
// Proxmox cluster quorum via `pvecm status`. Only meaningful on
|
||||
// proxmox-host entities. Runs every 60s — corosync flaps are
|
||||
// transient and the probe is lightweight (local binary, no network).
|
||||
if host == "" {
|
||||
return nil, "no address on the entity or its host"
|
||||
}
|
||||
return []CheckDef{ssh("pvecm_quorum_check.sh")}, ""
|
||||
}
|
||||
|
||||
return nil, "no builder for this kind yet"
|
||||
}
|
||||
|
||||
// certHost works out the hostname to TLS-dial for a certificate's expiry.
|
||||
func certHost(t Target, attrs map[string]any) string {
|
||||
func certHost(t CheckTarget, attrs map[string]any) string {
|
||||
for _, key := range []string{"hostname", "cn", "san"} {
|
||||
if v, ok := attrs[key].(string); ok && v != "" {
|
||||
return v
|
||||
@@ -361,78 +363,15 @@ func certHost(t Target, attrs map[string]any) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// writeCheck upserts one check_def and its backing check entity.
|
||||
//
|
||||
// The entity upsert MUST return the row's id. The previous version generated
|
||||
// a fresh uuid, inserted ON CONFLICT (slug) DO NOTHING, then wrote a
|
||||
// check_defs row referencing that uuid. On any re-seed the slug already
|
||||
// existed, the entity insert became a no-op, and the check_defs insert
|
||||
// violated its foreign key — which aborted the whole ingest transaction and
|
||||
// made every subsequent statement fail with 25P02. Because the errors were
|
||||
// discarded, the only visible symptom was an unrelated failure much later.
|
||||
func writeCheck(ctx context.Context, tx pgx.Tx, t Target, idx int, def checkDef) (bool, error) {
|
||||
// The full target slug, not a truncation of it. shortSlug() took the last
|
||||
// 8 characters, so all 21 ingress routes collapsed to ".network" and
|
||||
// generated one identical check slug — they overwrote each other and 20
|
||||
// of them ended up with no check at all. It also collided service:jellyfin
|
||||
// with lxc:jellyfin. Entity slugs are unique; use them.
|
||||
checkSlug := fmt.Sprintf("check:%s:%s:%d", def.kind, t.Slug, idx)
|
||||
|
||||
newID, err := uuid.NewV7()
|
||||
if err != nil {
|
||||
newID = uuid.New()
|
||||
}
|
||||
|
||||
var checkID uuid.UUID
|
||||
err = tx.QueryRow(ctx,
|
||||
`INSERT INTO entities (id, slug, type, name, state, attributes, version, created_at, updated_at)
|
||||
VALUES ($1, $2, 'check', $2, 'active', '{}', 1, now(), now())
|
||||
ON CONFLICT (slug) DO UPDATE SET updated_at = now()
|
||||
RETURNING id`,
|
||||
newID, checkSlug).Scan(&checkID)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("upsert check entity %s: %w", checkSlug, err)
|
||||
}
|
||||
|
||||
configJSON, err := json.Marshal(def.config)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
// Config is derived from the seed, so the seed wins on re-ingest and
|
||||
// attribute changes propagate. `enabled` is deliberately left alone: it
|
||||
// is operational state an operator may have toggled.
|
||||
// last_run_at is seeded to a random point inside the interval so checks
|
||||
// created together do not stay in lockstep. Every check the seed creates
|
||||
// would otherwise come due in the same instant forever: ~165 probes
|
||||
// landing at once each minute rather than spread across it. Deliberately
|
||||
// absent from the DO UPDATE below — a re-seed must not reset the schedule
|
||||
// and re-herd everything.
|
||||
tag, err := tx.Exec(ctx,
|
||||
`INSERT INTO check_defs (entity_id, target_id, target_type, kind, config, interval_s, timeout_s, enabled, last_run_at)
|
||||
VALUES ($1, $2, $6, $3, $4, $5, 30, true,
|
||||
now() - make_interval(secs => random() * $5::int))
|
||||
ON CONFLICT (entity_id) DO UPDATE
|
||||
SET target_id = EXCLUDED.target_id, target_type = EXCLUDED.target_type,
|
||||
kind = EXCLUDED.kind,
|
||||
config = EXCLUDED.config, interval_s = EXCLUDED.interval_s,
|
||||
updated_at = now()`,
|
||||
checkID, t.ID, def.kind, configJSON, def.interval, t.Type)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("upsert check_def %s: %w", checkSlug, err)
|
||||
}
|
||||
return tag.RowsAffected() > 0, nil
|
||||
}
|
||||
|
||||
// httpURL works out what to GET for an http check.
|
||||
//
|
||||
// Ingress routes carry their hostname as the entity name rather than as an
|
||||
// attribute (`name: media.hubris.network`), and most declare no attributes at
|
||||
// all — so the name is the only thing to go on. Requiring a `url` attribute
|
||||
// attribute (`name: media.hubris.network`), and most declare no attributes
|
||||
// at all — so the name is the only thing to go on. Requiring a `url` attribute
|
||||
// left all 21 of them unmonitored, which is a shame given an ingress check is
|
||||
// the most end-to-end probe available: it exercises Caddy, DNS, TLS and the
|
||||
// upstream in one request.
|
||||
func httpURL(t Target, attrs map[string]any) string {
|
||||
func httpURL(t CheckTarget, attrs map[string]any) string {
|
||||
if url, ok := attrs["url"].(string); ok && url != "" {
|
||||
return url
|
||||
}
|
||||
@@ -446,44 +385,6 @@ func httpURL(t Target, attrs map[string]any) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// hostViaGraph returns the attributes of the entity that hosts or provides
|
||||
// this one, so a service can inherit its container's address.
|
||||
func hostViaGraph(ctx context.Context, tx pgx.Tx, entityID uuid.UUID) (map[string]any, error) {
|
||||
rows, err := tx.Query(ctx, `
|
||||
SELECT e.attributes
|
||||
FROM relationships r
|
||||
JOIN entities e ON e.id = r.source_id
|
||||
WHERE r.target_id = $1
|
||||
AND r.valid_to IS NULL
|
||||
-- backs-up-to points from the thing being backed up TO the target,
|
||||
-- so walking it backwards finds the machine that writes the backups
|
||||
-- — which is the only place a freshness check can run.
|
||||
AND r.type IN ('provides', 'hosts', 'runs-on', 'backs-up-to')
|
||||
ORDER BY CASE r.type
|
||||
WHEN 'provides' THEN 0 WHEN 'runs-on' THEN 1
|
||||
WHEN 'backs-up-to' THEN 2 ELSE 3 END`,
|
||||
entityID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
for rows.Next() {
|
||||
var raw []byte
|
||||
if err := rows.Scan(&raw); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var attrs map[string]any
|
||||
if json.Unmarshal(raw, &attrs) != nil {
|
||||
continue
|
||||
}
|
||||
if resolveHost(attrs) != "" {
|
||||
return attrs, nil
|
||||
}
|
||||
}
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
func resolveHost(attrs map[string]any) string {
|
||||
if attrs == nil {
|
||||
return ""
|
||||
@@ -548,9 +449,9 @@ func resolveSSHPort(attrs map[string]any) int {
|
||||
return 22
|
||||
}
|
||||
|
||||
// LogResult emits the one line that was missing: a type that asked for
|
||||
// LogDeriveResult emits the one line that was missing: a type that asked for
|
||||
// monitoring and did not get it.
|
||||
func LogResult(slug, entityType string, res Result) {
|
||||
func LogDeriveResult(slug, entityType string, res DeriveResult) {
|
||||
switch {
|
||||
case res.Undeclared:
|
||||
slog.Info("checkdefaults: type declares no monitoring",
|
||||
202
internal/core/app/checkdefaults_build_test.go
Normal file
202
internal/core/app/checkdefaults_build_test.go
Normal file
@@ -0,0 +1,202 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/ontology"
|
||||
)
|
||||
|
||||
// Table-driven coverage of every implemented buildKind branch and the ssh()
|
||||
// helper's user/port/args propagation. The previous tests exercised only
|
||||
// ping/process/http/resource; updates, capacity, backup, cert-expiry,
|
||||
// vm-status and dns were unverified.
|
||||
func TestBuildKindAllImplementedKinds(t *testing.T) {
|
||||
host := "10.0.0.5"
|
||||
cases := []struct {
|
||||
name string
|
||||
kind string
|
||||
target CheckTarget
|
||||
attrs map[string]any
|
||||
host string
|
||||
wantSkip bool // true → expect a reason and zero defs
|
||||
wantDefs int
|
||||
wantKind string
|
||||
wantKey string // a config key to assert
|
||||
wantVal any // its expected value
|
||||
wantReason string // substring when skipping
|
||||
wantInterv int // expected interval on the (single) produced def
|
||||
}{
|
||||
{
|
||||
name: "ping with host", kind: KindPing, host: host,
|
||||
wantDefs: 1, wantKind: "ping", wantKey: "host", wantVal: host, wantInterv: 30,
|
||||
},
|
||||
{name: "ping no host skips", kind: KindPing, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "resource expands to four ssh scripts", kind: KindResource, host: host,
|
||||
wantDefs: 4, wantKind: "ssh-script", wantKey: "host", wantVal: host, wantInterv: 60,
|
||||
},
|
||||
{name: "resource no host skips", kind: KindResource, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "updates is daily", kind: KindUpdates, host: host,
|
||||
wantDefs: 1, wantKind: "ssh-script", wantKey: "script", wantVal: "updates_check.sh", wantInterv: 86400,
|
||||
},
|
||||
{name: "updates no host skips", kind: KindUpdates, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "capacity is one disk script", kind: KindCapacity, host: host,
|
||||
wantDefs: 1, wantKind: "ssh-script", wantKey: "script", wantVal: "disk_usage_check.sh", wantInterv: 60,
|
||||
},
|
||||
{name: "capacity no host skips", kind: KindCapacity, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "backup needs path and host", kind: KindBackup, host: host,
|
||||
attrs: map[string]any{"path": "/backups/db"},
|
||||
wantDefs: 1, wantKind: "backup-freshness", wantKey: "path", wantVal: "/backups/db", wantInterv: 86400,
|
||||
},
|
||||
{name: "backup without path skips", kind: KindBackup, host: host, wantSkip: true, wantReason: "no path"},
|
||||
{name: "backup without host skips", kind: KindBackup, attrs: map[string]any{"path": "/x"}, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "backup honors backup_max_age_s override", kind: KindBackup, host: host,
|
||||
attrs: map[string]any{"path": "/x", "backup_max_age_s": float64(3600)},
|
||||
wantDefs: 1, wantKey: "max_age_s", wantVal: 3600,
|
||||
},
|
||||
|
||||
{
|
||||
name: "cert-expiry from hostname attr", kind: KindCertExpiry,
|
||||
attrs: map[string]any{"hostname": "media.hubris.network"},
|
||||
wantDefs: 1, wantKind: "cert-expiry", wantKey: "host", wantVal: "media.hubris.network", wantInterv: 3600,
|
||||
},
|
||||
{
|
||||
name: "cert-expiry from dotted name", kind: KindCertExpiry, target: CheckTarget{Name: "media.hubris.network"},
|
||||
wantDefs: 1, wantKey: "host", wantVal: "media.hubris.network",
|
||||
},
|
||||
{
|
||||
name: "cert-expiry propagates dial attr", kind: KindCertExpiry,
|
||||
attrs: map[string]any{"hostname": "media.hubris.network", "dial": "10.0.0.2"},
|
||||
wantDefs: 1, wantKey: "dial", wantVal: "10.0.0.2",
|
||||
},
|
||||
{name: "cert-expiry without a host name skips", kind: KindCertExpiry, target: CheckTarget{Name: "jellyfin"}, wantSkip: true, wantReason: "no hostname"},
|
||||
|
||||
{
|
||||
name: "vm-status needs pve_id", kind: KindVMStatus, attrs: map[string]any{"pve_id": float64(101)},
|
||||
wantDefs: 1, wantKind: "vm-status", wantInterv: 60,
|
||||
},
|
||||
{name: "vm-status without pve_id skips", kind: KindVMStatus, wantSkip: true, wantReason: "no pve_id"},
|
||||
|
||||
{
|
||||
name: "dns resolves entity name", kind: KindDNS, target: CheckTarget{Name: "hubris.network"},
|
||||
wantDefs: 1, wantKind: "dns", wantKey: "name", wantVal: "hubris.network", wantInterv: 300,
|
||||
},
|
||||
{name: "dns without a name skips", kind: KindDNS, target: CheckTarget{}, wantSkip: true, wantReason: "no name"},
|
||||
|
||||
{
|
||||
name: "quorum runs pvecm script via ssh", kind: KindQuorum, host: host,
|
||||
wantDefs: 1, wantKind: "ssh-script", wantKey: "script", wantVal: "pvecm_quorum_check.sh", wantInterv: 60,
|
||||
},
|
||||
{name: "quorum no host skips", kind: KindQuorum, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{name: "unknown kind skips", kind: "telepathy", host: host, wantSkip: true, wantReason: "no builder"},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
defs, reason := buildKind(c.kind, c.target, c.attrs, c.host, "root", 22)
|
||||
if c.wantSkip {
|
||||
if len(defs) != 0 {
|
||||
t.Fatalf("expected zero defs, got %d", len(defs))
|
||||
}
|
||||
if c.wantReason != "" && !strings.Contains(reason, c.wantReason) {
|
||||
t.Errorf("reason = %q, want substring %q", reason, c.wantReason)
|
||||
}
|
||||
return
|
||||
}
|
||||
if len(defs) != c.wantDefs {
|
||||
t.Fatalf("got %d defs (%s), want %d", len(defs), reason, c.wantDefs)
|
||||
}
|
||||
if reason != "" {
|
||||
t.Errorf("unexpected skip reason: %q", reason)
|
||||
}
|
||||
if c.wantKind != "" {
|
||||
if got := defs[0].Kind; got != c.wantKind {
|
||||
t.Errorf("kind = %q, want %q", got, c.wantKind)
|
||||
}
|
||||
}
|
||||
if c.wantKey != "" {
|
||||
if got := defs[0].Config[c.wantKey]; !reflect.DeepEqual(got, c.wantVal) {
|
||||
t.Errorf("config[%q] = %v (%T), want %v (%T)", c.wantKey, got, got, c.wantVal, c.wantVal)
|
||||
}
|
||||
}
|
||||
if c.wantInterv != 0 && defs[0].IntervalS != c.wantInterv {
|
||||
t.Errorf("interval = %d, want %d", defs[0].IntervalS, c.wantInterv)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// ssh() must add user/port/args only when they differ from the root/22/empty
|
||||
// defaults, so generated configs stay minimal and stable across re-seeds.
|
||||
func TestBuildKindSSHOnlyEmitsNonDefaultUserPortArgs(t *testing.T) {
|
||||
t.Run("default root 22 omits user and port", func(t *testing.T) {
|
||||
defs, _ := buildKind(KindResource, CheckTarget{}, nil, "10.0.0.1", "root", 22)
|
||||
for _, d := range defs {
|
||||
if _, ok := d.Config["user"]; ok {
|
||||
t.Errorf("root should not emit user: %v", d.Config)
|
||||
}
|
||||
if _, ok := d.Config["port"]; ok {
|
||||
t.Errorf("port 22 should not emit port: %v", d.Config)
|
||||
}
|
||||
}
|
||||
})
|
||||
t.Run("non-root user and non-22 port are emitted", func(t *testing.T) {
|
||||
defs, _ := buildKind(KindResource, CheckTarget{}, nil, "10.0.0.1", "oikos", 2222)
|
||||
if defs[0].Config["user"] != "oikos" {
|
||||
t.Errorf("user = %v, want oikos", defs[0].Config["user"])
|
||||
}
|
||||
if defs[0].Config["port"] != 2222 {
|
||||
t.Errorf("port = %v, want 2222", defs[0].Config["port"])
|
||||
}
|
||||
})
|
||||
t.Run("process unit name lands in args", func(t *testing.T) {
|
||||
defs, _ := buildKind(KindProcess, CheckTarget{Name: "jellyfin"}, nil, "10.0.0.1", "root", 22)
|
||||
if defs[0].Config["args"] != "jellyfin" {
|
||||
t.Errorf("args = %v, want jellyfin", defs[0].Config["args"])
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// resolveMonitoringAttr implements the entity-level `monitoring` override
|
||||
// (project decision health_checks.monitoring_override): "none"/"" opts out,
|
||||
// a kind-list replaces the type defaults, anything else falls back.
|
||||
func TestResolveMonitoringAttr(t *testing.T) {
|
||||
fallback := ontology.MonitoringResolution{Declared: true, Kinds: []string{"ping"}, Source: "type"}
|
||||
cases := []struct {
|
||||
name string
|
||||
in any
|
||||
want ontology.MonitoringResolution
|
||||
}{
|
||||
{"none opts out", "none", ontology.MonitoringResolution{Declared: true, Source: "attribute"}},
|
||||
{"empty opts out", "", ontology.MonitoringResolution{Declared: true, Source: "attribute"}},
|
||||
{
|
||||
"kind list overrides",
|
||||
[]any{"http", "process"},
|
||||
ontology.MonitoringResolution{Declared: true, Kinds: []string{"http", "process"}, Source: "attribute"},
|
||||
},
|
||||
{"list drops empty and non-string entries", []any{"http", "", 7, "dns"}, ontology.MonitoringResolution{Declared: true, Kinds: []string{"http", "dns"}, Source: "attribute"}},
|
||||
{"non-string scalar falls back to type default", float64(42), fallback},
|
||||
{"nil falls back", nil, fallback},
|
||||
{"unrecognized string falls back", "weird", fallback},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got := resolveMonitoringAttr(c.in, fallback)
|
||||
if !reflect.DeepEqual(got, c.want) {
|
||||
t.Errorf("resolveMonitoringAttr(%v) = %+v, want %+v", c.in, got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
package checkdefaults
|
||||
package app
|
||||
|
||||
import (
|
||||
"testing"
|
||||
@@ -58,7 +58,7 @@ func TestHTTPURLPrefersAttributeThenName(t *testing.T) {
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
got := httpURL(Target{Name: c.name}, c.attrs)
|
||||
got := httpURL(CheckTarget{Name: c.name}, c.attrs)
|
||||
if got != c.want {
|
||||
t.Errorf("%s: httpURL = %q, want %q", c.desc, got, c.want)
|
||||
}
|
||||
@@ -68,13 +68,13 @@ func TestHTTPURLPrefersAttributeThenName(t *testing.T) {
|
||||
func TestBuildKindReportsWhyItSkipped(t *testing.T) {
|
||||
// A declared kind that cannot be built must explain itself rather than
|
||||
// vanish — that silence is what hid the coverage gap.
|
||||
if defs, reason := buildKind(KindPing, Target{}, nil, "", "root", 22); len(defs) != 0 || reason == "" {
|
||||
if defs, reason := buildKind(KindPing, CheckTarget{}, nil, "", "root", 22); len(defs) != 0 || reason == "" {
|
||||
t.Errorf("ping without a host should skip with a reason, got %d defs / %q", len(defs), reason)
|
||||
}
|
||||
if defs, reason := buildKind(KindProcess, Target{Name: ""}, nil, "10.0.0.1", "root", 22); len(defs) != 0 || reason == "" {
|
||||
if defs, reason := buildKind(KindProcess, CheckTarget{Name: ""}, nil, "10.0.0.1", "root", 22); len(defs) != 0 || reason == "" {
|
||||
t.Errorf("process without a name should skip with a reason, got %d defs / %q", len(defs), reason)
|
||||
}
|
||||
if defs, reason := buildKind("dns", Target{}, nil, "10.0.0.1", "root", 22); len(defs) != 0 || reason == "" {
|
||||
if defs, reason := buildKind("dns", CheckTarget{}, nil, "10.0.0.1", "root", 22); len(defs) != 0 || reason == "" {
|
||||
t.Errorf("an unimplemented kind should skip with a reason, got %d defs / %q", len(defs), reason)
|
||||
}
|
||||
}
|
||||
@@ -82,44 +82,44 @@ func TestBuildKindReportsWhyItSkipped(t *testing.T) {
|
||||
func TestBuildKindProcessPassesTheUnitName(t *testing.T) {
|
||||
// process_check.sh reads $1 and answers "no service name provided"
|
||||
// without it. checkdefaults always wrote args; nothing read them.
|
||||
defs, reason := buildKind(KindProcess, Target{Name: "jellyfin"}, nil, "10.0.0.1", "root", 22)
|
||||
defs, reason := buildKind(KindProcess, CheckTarget{Name: "jellyfin"}, nil, "10.0.0.1", "root", 22)
|
||||
if len(defs) != 1 {
|
||||
t.Fatalf("expected one process check, got %d (%s)", len(defs), reason)
|
||||
}
|
||||
if got := defs[0].config["args"]; got != "jellyfin" {
|
||||
if got := defs[0].Config["args"]; got != "jellyfin" {
|
||||
t.Errorf("process check args = %v, want jellyfin", got)
|
||||
}
|
||||
if got := defs[0].config["script"]; got != "process_check.sh" {
|
||||
if got := defs[0].Config["script"]; got != "process_check.sh" {
|
||||
t.Errorf("process check script = %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildKindHTTPUsesAStatusRangeNotAnExactCode(t *testing.T) {
|
||||
// Most services sit behind Authentik and answer 302/401.
|
||||
defs, _ := buildKind(KindHTTP, Target{Name: "jellyfin"},
|
||||
defs, _ := buildKind(KindHTTP, CheckTarget{Name: "jellyfin"},
|
||||
map[string]any{"url": "https://media.hubris.network"}, "", "root", 22)
|
||||
if len(defs) != 1 {
|
||||
t.Fatalf("expected one http check, got %d", len(defs))
|
||||
}
|
||||
if got := defs[0].config["max_status"]; got != 500 {
|
||||
if got := defs[0].Config["max_status"]; got != 500 {
|
||||
t.Errorf("max_status = %v, want 500", got)
|
||||
}
|
||||
if _, exact := defs[0].config["expected_status"]; exact {
|
||||
if _, exact := defs[0].Config["expected_status"]; exact {
|
||||
t.Error("default http checks must not pin an exact status")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildKindResourceExpandsToFourScripts(t *testing.T) {
|
||||
defs, _ := buildKind(KindResource, Target{}, nil, "10.0.0.1", "root", 22)
|
||||
defs, _ := buildKind(KindResource, CheckTarget{}, nil, "10.0.0.1", "root", 22)
|
||||
if len(defs) != 4 {
|
||||
t.Fatalf("resource should expand to 4 checks, got %d", len(defs))
|
||||
}
|
||||
for _, d := range defs {
|
||||
if d.kind != "ssh-script" {
|
||||
t.Errorf("resource check kind = %q, want ssh-script", d.kind)
|
||||
if d.Kind != "ssh-script" {
|
||||
t.Errorf("resource check kind = %q, want ssh-script", d.Kind)
|
||||
}
|
||||
if d.config["host"] != "10.0.0.1" {
|
||||
t.Errorf("resource check lost its host: %v", d.config)
|
||||
if d.Config["host"] != "10.0.0.1" {
|
||||
t.Errorf("resource check lost its host: %v", d.Config)
|
||||
}
|
||||
}
|
||||
}
|
||||
5
internal/core/app/doc.go
Normal file
5
internal/core/app/doc.go
Normal file
@@ -0,0 +1,5 @@
|
||||
// Package app hosts the application services (use-cases) of the oikos
|
||||
// core. Services take ports as constructor arguments, return domain types
|
||||
// and sentinel errors, and own the command side of the system. See
|
||||
// docs/adr/0016-hexagonal-ports-adapters.md.
|
||||
package app
|
||||
262
internal/core/app/entities.go
Normal file
262
internal/core/app/entities.go
Normal file
@@ -0,0 +1,262 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// EntityService owns the entity aggregate's use-cases: create with derived
|
||||
// checks, update with optimistic versioning and check regeneration,
|
||||
// lifecycle transitions. Validation against the (cached) ontology happens
|
||||
// here, in the pure half; transactional check-then-act invariants
|
||||
// (version, declared transitions, preconditions) run in the repository
|
||||
// (plan §3.6).
|
||||
type EntityService struct {
|
||||
entities ports.EntityRepository
|
||||
onto ports.OntologyStore
|
||||
}
|
||||
|
||||
// NewEntityService wires the service.
|
||||
func NewEntityService(entities ports.EntityRepository, onto ports.OntologyStore) *EntityService {
|
||||
return &EntityService{entities: entities, onto: onto}
|
||||
}
|
||||
|
||||
// CreateEntityCmd is one entity creation. Actor identifies the calling
|
||||
// surface for the audit trail ("operator:<label>", "agent:mcp").
|
||||
type CreateEntityCmd struct {
|
||||
Slug string
|
||||
Type string
|
||||
Name string
|
||||
State string // "" → lifecycle default
|
||||
Attributes map[string]any
|
||||
ActorType string
|
||||
Actor string
|
||||
Method string // audit context, e.g. "POST" / "TOOL"
|
||||
Path string // audit context, e.g. "/api/v1/entities" / "create_entity"
|
||||
// Idempotency, when set, replays-protects the create. The adapter owns
|
||||
// the request hash and the cached-body renderer (its wire shape); the
|
||||
// repository stores the record in the create's transaction.
|
||||
Idempotency *ports.Idempotency
|
||||
}
|
||||
|
||||
// Create validates the type (exists, concrete, state declared), derives
|
||||
// default checks, and commits entity + checks + audit + event in one
|
||||
// transaction.
|
||||
func (s *EntityService) Create(ctx context.Context, cmd CreateEntityCmd) (domain.Entity, DeriveResult, error) {
|
||||
var res DeriveResult
|
||||
|
||||
tree, err := s.onto.LoadTypeTree(ctx)
|
||||
if err != nil {
|
||||
return domain.Entity{}, res, err
|
||||
}
|
||||
|
||||
slug := cmd.Slug
|
||||
if slug == "" {
|
||||
slug = cmd.Type + ":" + cmd.Name
|
||||
}
|
||||
|
||||
// Caller-supplied states are validated against the lifecycle's declared
|
||||
// states — a create bypass of lifecycle guardrails would let an agent
|
||||
// create in a terminal state without satisfying the preconditions that
|
||||
// SetState enforces for the same transition. Both surfaces (REST, MCP)
|
||||
// now share this rule.
|
||||
state := cmd.State
|
||||
if state == "" {
|
||||
state = tree.DefaultState(cmd.Type)
|
||||
}
|
||||
if err := tree.ValidateEntity(cmd.Type, state); err != nil {
|
||||
return domain.Entity{}, res, err
|
||||
}
|
||||
|
||||
id, err := uuid.NewV7()
|
||||
if err != nil {
|
||||
return domain.Entity{}, res, err
|
||||
}
|
||||
e := domain.Entity{
|
||||
ID: domain.UUID(id.String()),
|
||||
Slug: slug,
|
||||
Type: cmd.Type,
|
||||
Name: cmd.Name,
|
||||
State: state,
|
||||
Attributes: cmd.Attributes,
|
||||
}
|
||||
|
||||
// Derivation for a create has no graph fallback available — a new entity
|
||||
// has no edges yet. A type whose address comes from its host (a service)
|
||||
// produces no checks on this pass; the gap is deliberate and visible
|
||||
// (coverage sweep), and the next mutation or ingest fills it once the
|
||||
// hosting edge exists.
|
||||
attrsJSON, _ := json.Marshal(cmd.Attributes)
|
||||
defs, dres := Derive(tree, CheckTarget{
|
||||
ID: string(e.ID), Slug: slug, Type: cmd.Type, Name: cmd.Name, Attrs: attrsJSON,
|
||||
}, nil)
|
||||
// Derive is pure and cannot count writes; every derived def is ensured
|
||||
// by the repository in the create's transaction (a write failure aborts
|
||||
// the whole create), so the ensured count is the derived count.
|
||||
dres.Created = len(defs)
|
||||
res = dres
|
||||
derived := make([]ports.DerivedCheck, len(defs))
|
||||
for i, d := range defs {
|
||||
derived[i] = ports.DerivedCheck{Kind: d.Kind, Config: d.Config, IntervalS: d.IntervalS}
|
||||
}
|
||||
|
||||
input := ports.EntityCreateInput{
|
||||
Entity: e,
|
||||
DerivedChecks: derived,
|
||||
Audit: []ports.AuditEntry{{
|
||||
ActorType: cmd.ActorType, ActorLabel: cmd.Actor, Action: "create",
|
||||
EntityID: e.ID, Method: cmd.Method, Path: cmd.Path,
|
||||
Details: map[string]any{"type": cmd.Type, "slug": slug},
|
||||
}},
|
||||
Event: &ports.Event{
|
||||
Type: "entity.created", Severity: "info", Source: "oikos-api", EntityID: e.ID,
|
||||
Data: map[string]any{"slug": slug, "type": cmd.Type},
|
||||
},
|
||||
}
|
||||
input.Idempotency = cmd.Idempotency
|
||||
|
||||
created, err := s.entities.Create(ctx, input)
|
||||
if err != nil {
|
||||
return domain.Entity{}, res, err
|
||||
}
|
||||
return created, res, nil
|
||||
}
|
||||
|
||||
// UpdateEntityCmd mutates an entity. Attributes are REPLACED when
|
||||
// AttrsReplace is true and shallow-merged otherwise. SetState semantics
|
||||
// ride along: a State change is lifecycle-validated.
|
||||
type UpdateEntityCmd struct {
|
||||
SlugOrID string
|
||||
ExpectedVer int
|
||||
Name string
|
||||
State string
|
||||
Attributes map[string]any
|
||||
AttrsReplace bool
|
||||
Maintenance *time.Time
|
||||
SetMaint bool
|
||||
// RederiveChecks regenerates default checks from the post-update
|
||||
// attributes (a `monitoring` attribute change re-wires probes).
|
||||
RederiveChecks bool
|
||||
ActorType string
|
||||
Actor string
|
||||
Method string
|
||||
Path string
|
||||
}
|
||||
|
||||
// Update loads the current entity, validates the target state against the
|
||||
// ontology, and commits the mutation atomically. The optimistic-version and
|
||||
// transition-precondition checks run inside the repository's transaction.
|
||||
// When RederiveChecks is set the returned DeriveResult summarizes the
|
||||
// derivation the transaction applied (for surface messages).
|
||||
func (s *EntityService) Update(ctx context.Context, cmd UpdateEntityCmd) (domain.Entity, DeriveResult, error) {
|
||||
var dres DeriveResult
|
||||
current, err := s.resolve(ctx, cmd.SlugOrID)
|
||||
if err != nil {
|
||||
return domain.Entity{}, dres, err
|
||||
}
|
||||
|
||||
tree, err := s.onto.LoadTypeTree(ctx)
|
||||
if err != nil {
|
||||
return domain.Entity{}, dres, err
|
||||
}
|
||||
if cmd.State != "" && cmd.State != current.State {
|
||||
if err := tree.ValidateEntity(current.Type, cmd.State); err != nil {
|
||||
return domain.Entity{}, dres, err
|
||||
}
|
||||
}
|
||||
|
||||
e := domain.Entity{
|
||||
ID: current.ID,
|
||||
Slug: current.Slug,
|
||||
Type: current.Type,
|
||||
Name: cmd.Name,
|
||||
State: cmd.State,
|
||||
Version: current.Version,
|
||||
}
|
||||
switch {
|
||||
case cmd.Attributes == nil:
|
||||
e.Attributes = nil // keep current
|
||||
case cmd.AttrsReplace:
|
||||
e.Attributes = cmd.Attributes
|
||||
default:
|
||||
merged := map[string]any{}
|
||||
for k, v := range current.Attributes {
|
||||
merged[k] = v
|
||||
}
|
||||
for k, v := range cmd.Attributes {
|
||||
merged[k] = v
|
||||
}
|
||||
e.Attributes = merged
|
||||
}
|
||||
if cmd.SetMaint {
|
||||
e.MaintenanceUntil = cmd.Maintenance
|
||||
}
|
||||
|
||||
updated, err := s.entities.Update(ctx, ports.EntityUpdateInput{
|
||||
Entity: e,
|
||||
ExpectedVersion: cmd.ExpectedVer,
|
||||
RederiveChecks: cmd.RederiveChecks,
|
||||
Audit: []ports.AuditEntry{{
|
||||
ActorType: cmd.ActorType, ActorLabel: cmd.Actor, Action: "patch",
|
||||
EntityID: current.ID, Method: cmd.Method, Path: cmd.Path,
|
||||
Details: map[string]any{"version": current.Version},
|
||||
}},
|
||||
Event: &ports.Event{
|
||||
Type: "entity.updated", Severity: "info", Source: "oikos-api", EntityID: current.ID,
|
||||
Data: map[string]any{"slug": current.Slug, "type": current.Type},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return domain.Entity{}, dres, err
|
||||
}
|
||||
if cmd.RederiveChecks {
|
||||
attrsJSON, _ := json.Marshal(updated.Attributes)
|
||||
defs, d := Derive(tree, CheckTarget{
|
||||
ID: string(updated.ID), Slug: updated.Slug, Type: updated.Type,
|
||||
Name: updated.Name, Attrs: attrsJSON,
|
||||
}, nil)
|
||||
d.Created = len(defs)
|
||||
dres = d
|
||||
}
|
||||
return updated, dres, nil
|
||||
}
|
||||
|
||||
// SetState transitions an entity's lifecycle. Declared-transition and
|
||||
// precondition enforcement happens in the repository's transaction; a
|
||||
// stale FromState is refused there (check-then-act).
|
||||
func (s *EntityService) SetState(ctx context.Context, slug, fromState, toState, actorType, actor, method, path string) (domain.Entity, error) {
|
||||
current, err := s.entities.BySlug(ctx, slug)
|
||||
if err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
after, err := s.entities.SetState(ctx, ports.EntityTransitionInput{
|
||||
Slug: slug,
|
||||
From: fromState,
|
||||
To: toState,
|
||||
Audit: []ports.AuditEntry{{
|
||||
ActorType: actorType, ActorLabel: actor, Action: "state",
|
||||
EntityID: current.ID, Method: method, Path: path,
|
||||
Details: map[string]any{"from": fromState, "to": toState},
|
||||
}},
|
||||
Event: &ports.Event{
|
||||
Type: "entity.state.changed", Severity: "info", Source: "oikos-api", EntityID: current.ID,
|
||||
Data: map[string]any{"slug": slug, "from": fromState, "to": toState},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return domain.Entity{}, err
|
||||
}
|
||||
return after, nil
|
||||
}
|
||||
|
||||
func (s *EntityService) resolve(ctx context.Context, slugOrID string) (domain.Entity, error) {
|
||||
if _, err := uuid.Parse(slugOrID); err == nil {
|
||||
return s.entities.Get(ctx, domain.UUID(slugOrID))
|
||||
}
|
||||
return s.entities.BySlug(ctx, slugOrID)
|
||||
}
|
||||
181
internal/core/app/entities_test.go
Normal file
181
internal/core/app/entities_test.go
Normal file
@@ -0,0 +1,181 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
"github.com/dtoro/oikos/internal/core/ports/portstest"
|
||||
"github.com/dtoro/oikos/internal/ontology"
|
||||
)
|
||||
|
||||
// fakeOntology is a static OntologyStore over a hand-built tree.
|
||||
type fakeOntology struct{ tree *ontology.TypeTree }
|
||||
|
||||
func (f *fakeOntology) LoadTypeTree(context.Context) (ports.TypeTree, error) {
|
||||
return f.tree, nil
|
||||
}
|
||||
|
||||
func testTree() *ontology.TypeTree {
|
||||
return &ontology.TypeTree{
|
||||
Types: map[string]ontology.TypeInfo{
|
||||
"service": {LifecycleID: "infra"},
|
||||
"host": {LifecycleID: "infra"},
|
||||
"widget": {IsAbstract: true},
|
||||
},
|
||||
Lifecycles: map[string]ontology.LifecycleInfo{
|
||||
"infra": {States: map[string]bool{"planned": true, "active": true, "deprecated": true}, DefaultState: "active"},
|
||||
},
|
||||
RelTypes: map[string]ontology.RelTypeInfo{},
|
||||
}
|
||||
}
|
||||
|
||||
func newSvc(t *testing.T) (*EntityService, *portstest.EntityRepo) {
|
||||
t.Helper()
|
||||
repo := portstest.NewEntityRepo()
|
||||
svc := NewEntityService(repo, &fakeOntology{tree: testTree()})
|
||||
return svc, repo
|
||||
}
|
||||
|
||||
func TestEntityServiceCreate(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
svc, repo := newSvc(t)
|
||||
|
||||
e, res, err := svc.Create(ctx, CreateEntityCmd{
|
||||
Type: "service", Name: "jellyfin", Attributes: map[string]any{"url": "https://media.example"},
|
||||
ActorType: "operator", Actor: "tester", Method: "POST", Path: "/api/v1/entities",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
if e.Slug != "service:jellyfin" {
|
||||
t.Errorf("default slug = %q, want service:jellyfin", e.Slug)
|
||||
}
|
||||
if e.State != "active" {
|
||||
t.Errorf("default state = %q, want active (lifecycle default)", e.State)
|
||||
}
|
||||
if res.Created != 0 {
|
||||
// testTree's service declares no monitoring → nothing derived
|
||||
t.Errorf("derived = %d, want 0 (no monitoring spec in fake tree)", res.Created)
|
||||
}
|
||||
if len(repo.Audits) != 1 || repo.Audits[0].Action != "create" {
|
||||
t.Errorf("audit = %+v, want one create entry", repo.Audits)
|
||||
}
|
||||
if len(repo.Events) != 1 || repo.Events[0].Type != "entity.created" {
|
||||
t.Errorf("event = %+v, want entity.created", repo.Events)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEntityServiceCreateValidation(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
svc, _ := newSvc(t)
|
||||
|
||||
if _, _, err := svc.Create(ctx, CreateEntityCmd{Type: "no-such-type", Name: "x"}); !errors.Is(err, domain.ErrNotFound) {
|
||||
t.Errorf("unknown type: got %v, want ErrNotFound", err)
|
||||
}
|
||||
if _, _, err := svc.Create(ctx, CreateEntityCmd{Type: "widget", Name: "x"}); !errors.Is(err, domain.ErrAbstractType) {
|
||||
t.Errorf("abstract type: got %v, want ErrAbstractType", err)
|
||||
}
|
||||
// The stricter (MCP) rule now governs both surfaces: caller-supplied
|
||||
// states must be declared in the lifecycle.
|
||||
if _, _, err := svc.Create(ctx, CreateEntityCmd{Type: "service", Name: "x", State: "destroyed"}); !errors.Is(err, domain.ErrInvalidTransition) {
|
||||
t.Errorf("undeclared state: got %v, want ErrInvalidTransition", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEntityServiceCreateIdempotency(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
svc, repo := newSvc(t)
|
||||
|
||||
idem := &ports.Idempotency{
|
||||
Actor: "tester", Key: "k1", RequestHash: "abc",
|
||||
RenderBody: func(e domain.Entity) []byte { return []byte(`{"slug":"` + e.Slug + `"}`) },
|
||||
}
|
||||
if _, _, err := svc.Create(ctx, CreateEntityCmd{Type: "host", Name: "h1", Idempotency: idem}); err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
|
||||
cached, err := repo.GetIdempotent(ctx, "tester", "k1")
|
||||
if err != nil {
|
||||
t.Fatalf("get idempotent: %v", err)
|
||||
}
|
||||
if cached.RequestHash != "abc" || cached.ResponseCode != 201 {
|
||||
t.Errorf("cached = %+v, want hash abc / code 201", cached)
|
||||
}
|
||||
if string(cached.ResponseBody) != `{"slug":"host:h1"}` {
|
||||
t.Errorf("cached body = %s, want the rendered wire shape", cached.ResponseBody)
|
||||
}
|
||||
|
||||
if _, err := repo.GetIdempotent(ctx, "tester", "other"); !errors.Is(err, domain.ErrNotFound) {
|
||||
t.Errorf("missing key: got %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEntityServiceUpdate(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
svc, repo := newSvc(t)
|
||||
|
||||
created, _, err := svc.Create(ctx, CreateEntityCmd{Type: "service", Name: "svc", Attributes: map[string]any{"a": 1}})
|
||||
if err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
|
||||
// Merge semantics: existing keys survive.
|
||||
updated, _, err := svc.Update(ctx, UpdateEntityCmd{
|
||||
SlugOrID: "service:svc", ExpectedVer: created.Version,
|
||||
Attributes: map[string]any{"b": 2}, RederiveChecks: true,
|
||||
ActorType: "agent", Actor: "mcp", Method: "TOOL", Path: "update_entity_attributes",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("update: %v", err)
|
||||
}
|
||||
if updated.Attributes["a"] != 1 || updated.Attributes["b"] != 2 {
|
||||
t.Errorf("merged attrs = %v, want a+b", updated.Attributes)
|
||||
}
|
||||
if len(repo.Rederived) != 1 || repo.Rederived[0] != created.ID {
|
||||
t.Errorf("redrive = %v, want one pass for the entity", repo.Rederived)
|
||||
}
|
||||
|
||||
// Optimistic concurrency: stale version refuses.
|
||||
if _, _, err := svc.Update(ctx, UpdateEntityCmd{SlugOrID: "service:svc", ExpectedVer: created.Version}); !errors.Is(err, domain.ErrConflict) {
|
||||
t.Errorf("stale version: got %v, want ErrConflict", err)
|
||||
}
|
||||
|
||||
// Replace semantics: wholesale swap.
|
||||
replaced, _, err := svc.Update(ctx, UpdateEntityCmd{
|
||||
SlugOrID: "service:svc", Attributes: map[string]any{"only": true}, AttrsReplace: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("replace update: %v", err)
|
||||
}
|
||||
if len(replaced.Attributes) != 1 || replaced.Attributes["only"] != true {
|
||||
t.Errorf("replaced attrs = %v, want only", replaced.Attributes)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEntityServiceSetState(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
svc, repo := newSvc(t)
|
||||
|
||||
if _, _, err := svc.Create(ctx, CreateEntityCmd{Type: "host", Name: "old"}); err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
|
||||
after, err := svc.SetState(ctx, "host:old", "active", "deprecated", "operator", "t", "PATCH", "/x")
|
||||
if err != nil {
|
||||
t.Fatalf("setState: %v", err)
|
||||
}
|
||||
if after.State != "deprecated" {
|
||||
t.Errorf("state = %s, want deprecated", after.State)
|
||||
}
|
||||
if len(repo.Audits) != 2 || repo.Audits[1].Action != "state" {
|
||||
t.Errorf("audits = %+v, want a state entry after create", repo.Audits)
|
||||
}
|
||||
|
||||
// Stale From refuses (check-then-act).
|
||||
if _, err := svc.SetState(ctx, "host:old", "active", "deprecated", "operator", "t", "PATCH", "/x"); !errors.Is(err, domain.ErrConflict) {
|
||||
t.Errorf("stale from-state: got %v, want ErrConflict", err)
|
||||
}
|
||||
}
|
||||
91
internal/core/app/knowledge.go
Normal file
91
internal/core/app/knowledge.go
Normal file
@@ -0,0 +1,91 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
)
|
||||
|
||||
// KnowledgeService is the use-case layer for knowledge-base operations.
|
||||
// Shared by REST + MCP knowledge tools (Phase 6 convergence).
|
||||
type KnowledgeService struct {
|
||||
repo ports.KnowledgeRepository
|
||||
}
|
||||
|
||||
func NewKnowledgeService(repo ports.KnowledgeRepository) *KnowledgeService {
|
||||
return &KnowledgeService{repo: repo}
|
||||
}
|
||||
|
||||
type SearchCmd struct {
|
||||
Query string
|
||||
Limit int
|
||||
}
|
||||
|
||||
func (s *KnowledgeService) Search(ctx context.Context, cmd SearchCmd) ([]ports.KnowledgeEntry, error) {
|
||||
return s.repo.Search(ctx, cmd.Query, cmd.Limit)
|
||||
}
|
||||
|
||||
type UpsertKnowledgeCmd struct {
|
||||
Entry KnowledgeEntryCmd
|
||||
ActorType string
|
||||
Actor string
|
||||
}
|
||||
|
||||
type KnowledgeEntryCmd struct {
|
||||
Slug string
|
||||
Title string
|
||||
Kind string
|
||||
Tags []string
|
||||
Content string
|
||||
About []string
|
||||
}
|
||||
|
||||
func (s *KnowledgeService) Upsert(ctx context.Context, cmd UpsertKnowledgeCmd) (ports.KnowledgeEntry, error) {
|
||||
return s.repo.Upsert(ctx, ports.KnowledgeUpsertInput{
|
||||
Entry: ports.KnowledgeEntry{
|
||||
Slug: cmd.Entry.Slug, Title: cmd.Entry.Title,
|
||||
Kind: cmd.Entry.Kind, Tags: cmd.Entry.Tags,
|
||||
Content: cmd.Entry.Content, About: cmd.Entry.About,
|
||||
UpdatedAt: time.Now(),
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func (s *KnowledgeService) GetContent(ctx context.Context, slug string) (ports.KnowledgeEntry, error) {
|
||||
return s.repo.GetContent(ctx, slug)
|
||||
}
|
||||
|
||||
func (s *KnowledgeService) SoftDelete(ctx context.Context, slug string) error {
|
||||
return s.repo.SoftDelete(ctx, slug)
|
||||
}
|
||||
|
||||
func (s *KnowledgeService) Restore(ctx context.Context, slug string) error {
|
||||
return s.repo.Restore(ctx, slug)
|
||||
}
|
||||
|
||||
// LearningService handles patterns, feedback, and skills.
|
||||
type LearningService struct {
|
||||
repo ports.LearningRepository
|
||||
}
|
||||
|
||||
func NewLearningService(repo ports.LearningRepository) *LearningService {
|
||||
return &LearningService{repo: repo}
|
||||
}
|
||||
|
||||
func (s *LearningService) ListPatterns(ctx context.Context) ([]domain.Pattern, error) {
|
||||
return s.repo.ListPatterns(ctx, 100)
|
||||
}
|
||||
|
||||
func (s *LearningService) UpsertPattern(ctx context.Context, pattern domain.Pattern) error {
|
||||
return s.repo.UpsertPattern(ctx, pattern)
|
||||
}
|
||||
|
||||
func (s *LearningService) Validate(ctx context.Context, patternID domain.UUID) error {
|
||||
return s.repo.Validate(ctx, patternID)
|
||||
}
|
||||
|
||||
func (s *LearningService) Quarantine(ctx context.Context, patternID domain.UUID, reason string) error {
|
||||
return s.repo.Quarantine(ctx, patternID, reason)
|
||||
}
|
||||
128
internal/core/app/observation.go
Normal file
128
internal/core/app/observation.go
Normal file
@@ -0,0 +1,128 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
)
|
||||
|
||||
// ObserveConfig controls the observe pass.
|
||||
type ObserveConfig struct {
|
||||
MaxConcurrency int
|
||||
}
|
||||
|
||||
// ObserveResult summarizes one pass.
|
||||
type ObserveResult struct {
|
||||
ChecksRun int
|
||||
HealthChanges int
|
||||
Duration time.Duration
|
||||
}
|
||||
|
||||
// CheckerLookup resolves a check kind to its probe implementation.
|
||||
type CheckerLookup interface {
|
||||
Get(kind string) ports.Checker
|
||||
}
|
||||
|
||||
// ObservationService runs observe passes: load enabled checks, resolve
|
||||
// targets, run probes via the Checker registry, process results through
|
||||
// SignalService, and aggregate health.
|
||||
type ObservationService struct {
|
||||
checks ports.CheckRepository
|
||||
signals *SignalService
|
||||
targets ports.TargetResolver
|
||||
reg CheckerLookup
|
||||
}
|
||||
|
||||
func NewObservationService(
|
||||
checks ports.CheckRepository,
|
||||
signals *SignalService,
|
||||
targets ports.TargetResolver,
|
||||
reg CheckerLookup,
|
||||
) *ObservationService {
|
||||
return &ObservationService{checks: checks, signals: signals, targets: targets, reg: reg}
|
||||
}
|
||||
|
||||
// RunPass loads enabled checks, resolves targets, runs probes with bounded
|
||||
// concurrency, and processes results through SignalService.
|
||||
func (o *ObservationService) RunPass(ctx context.Context, cfg ObserveConfig) (ObserveResult, error) {
|
||||
start := time.Now()
|
||||
var res ObserveResult
|
||||
|
||||
defs, err := o.checks.ListEnabled(ctx)
|
||||
if err != nil {
|
||||
return res, err
|
||||
}
|
||||
|
||||
if cfg.MaxConcurrency <= 0 {
|
||||
cfg.MaxConcurrency = 10
|
||||
}
|
||||
|
||||
type work struct {
|
||||
def ports.CheckDef
|
||||
target ports.Target
|
||||
health string
|
||||
}
|
||||
var jobs []work
|
||||
|
||||
for _, d := range defs {
|
||||
checker := o.reg.Get(d.Kind)
|
||||
if checker == nil {
|
||||
slog.Debug("observation: no checker for kind", "kind", d.Kind, "check", d.ID)
|
||||
continue
|
||||
}
|
||||
target, err := o.targets.ResolveForCheck(ctx, d.EntityID, d.Kind)
|
||||
if err != nil {
|
||||
slog.Warn("observation: resolve target", "check", d.ID, "error", err)
|
||||
continue
|
||||
}
|
||||
jobs = append(jobs, work{def: d, target: target})
|
||||
}
|
||||
|
||||
sem := make(chan struct{}, cfg.MaxConcurrency)
|
||||
type result struct {
|
||||
j work
|
||||
out ports.CheckResult
|
||||
}
|
||||
results := make(chan result, len(jobs))
|
||||
|
||||
for _, j := range jobs {
|
||||
j := j
|
||||
go func() {
|
||||
sem <- struct{}{}
|
||||
defer func() { <-sem }()
|
||||
checker := o.reg.Get(j.def.Kind)
|
||||
out := checker.Check(ctx, j.def, j.target)
|
||||
results <- result{j: j, out: out}
|
||||
}()
|
||||
}
|
||||
|
||||
var healthChanges int
|
||||
for i := 0; i < len(jobs); i++ {
|
||||
r := <-results
|
||||
res.ChecksRun++
|
||||
|
||||
hc, err := o.signals.ProcessCheckResult(ctx, CheckOutcome{
|
||||
EntityID: r.j.def.ID,
|
||||
TargetID: r.j.def.EntityID,
|
||||
Slug: r.j.def.Name,
|
||||
Kind: r.j.def.Kind,
|
||||
Value: r.out.Value,
|
||||
State: r.out.State,
|
||||
Message: r.out.Message,
|
||||
PrevHealth: "",
|
||||
})
|
||||
if err != nil {
|
||||
slog.Warn("observation: signal processing failed", "check", r.j.def.ID, "error", err)
|
||||
continue
|
||||
}
|
||||
if hc != nil {
|
||||
healthChanges++
|
||||
}
|
||||
}
|
||||
res.HealthChanges = healthChanges
|
||||
res.Duration = time.Since(start)
|
||||
|
||||
return res, nil
|
||||
}
|
||||
78
internal/core/app/relationships.go
Normal file
78
internal/core/app/relationships.go
Normal file
@@ -0,0 +1,78 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
)
|
||||
|
||||
// RelationshipService owns the relationship aggregate's use-cases: create
|
||||
// with ontology edge validation, soft-delete, and list reads. The adapter
|
||||
// resolves slug→entity and extracts types before calling the service; the
|
||||
// service validates endpoint types against the (cached) ontology.
|
||||
type RelationshipService struct {
|
||||
rels ports.RelationshipRepository
|
||||
onto ports.OntologyStore
|
||||
}
|
||||
|
||||
// NewRelationshipService wires the service.
|
||||
func NewRelationshipService(rels ports.RelationshipRepository, onto ports.OntologyStore) *RelationshipService {
|
||||
return &RelationshipService{rels: rels, onto: onto}
|
||||
}
|
||||
|
||||
// CreateRelationshipCmd is one edge creation. SourceType and TargetType are
|
||||
// the resolved entity types (used for ontology edge validation).
|
||||
type CreateRelationshipCmd struct {
|
||||
SourceID domain.UUID
|
||||
TargetID domain.UUID
|
||||
SourceType string
|
||||
TargetType string
|
||||
Type string
|
||||
Attributes map[string]any
|
||||
ActorType string
|
||||
Actor string
|
||||
Method string
|
||||
Path string
|
||||
}
|
||||
|
||||
// Create validates the edge against the ontology and creates it in one
|
||||
// transaction.
|
||||
func (s *RelationshipService) Create(ctx context.Context, cmd CreateRelationshipCmd) (domain.Relationship, error) {
|
||||
tree, err := s.onto.LoadTypeTree(ctx)
|
||||
if err != nil {
|
||||
return domain.Relationship{}, err
|
||||
}
|
||||
if err := tree.ValidateEdge(cmd.Type, cmd.SourceType, cmd.TargetType); err != nil {
|
||||
return domain.Relationship{}, fmt.Errorf("invalid edge: %w", err)
|
||||
}
|
||||
|
||||
rel := domain.Relationship{
|
||||
SourceID: cmd.SourceID,
|
||||
TargetID: cmd.TargetID,
|
||||
Type: cmd.Type,
|
||||
Attributes: cmd.Attributes,
|
||||
ValidFrom: time.Now(),
|
||||
}
|
||||
|
||||
created, err := s.rels.Create(ctx, ports.RelationshipCreateInput{
|
||||
Relationship: rel,
|
||||
Audit: []ports.AuditEntry{{
|
||||
ActorType: cmd.ActorType, ActorLabel: cmd.Actor, Action: "create",
|
||||
EntityID: cmd.SourceID, Method: cmd.Method, Path: cmd.Path,
|
||||
Details: map[string]any{"source": cmd.SourceID, "target": cmd.TargetID, "type": cmd.Type},
|
||||
}},
|
||||
})
|
||||
if err != nil {
|
||||
return domain.Relationship{}, err
|
||||
}
|
||||
return created, nil
|
||||
}
|
||||
|
||||
// End terminates an active relationship (soft-delete). Audit is kept at the
|
||||
// adapter level since End is a simple state toggle with no ontology check.
|
||||
func (s *RelationshipService) End(ctx context.Context, source, target domain.UUID, relType string) error {
|
||||
return s.rels.End(ctx, source, target, relType)
|
||||
}
|
||||
123
internal/core/app/signals.go
Normal file
123
internal/core/app/signals.go
Normal file
@@ -0,0 +1,123 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/core/domain"
|
||||
"github.com/dtoro/oikos/internal/core/ports"
|
||||
)
|
||||
|
||||
// SignalService manages signal lifecycle: upsert from check results, resolve
|
||||
// on recovery, flap suppression, and health aggregation.
|
||||
type SignalService struct {
|
||||
signals ports.SignalRepository
|
||||
metrics ports.MetricsRepository
|
||||
}
|
||||
|
||||
func NewSignalService(signals ports.SignalRepository, metrics ports.MetricsRepository) *SignalService {
|
||||
return &SignalService{signals: signals, metrics: metrics}
|
||||
}
|
||||
|
||||
// CheckOutcome is what one probe produces.
|
||||
type CheckOutcome struct {
|
||||
EntityID domain.UUID
|
||||
TargetID domain.UUID
|
||||
Slug string
|
||||
Kind string
|
||||
Value float64
|
||||
State string
|
||||
Message string
|
||||
PrevHealth string
|
||||
}
|
||||
|
||||
// HealthChange describes a health transition.
|
||||
type HealthChange struct {
|
||||
TargetID domain.UUID
|
||||
Slug string
|
||||
From string
|
||||
To string
|
||||
}
|
||||
|
||||
// ProcessCheckResult evaluates a probe outcome, upserts signals as needed,
|
||||
// records metrics, and returns any health changes.
|
||||
func (s *SignalService) ProcessCheckResult(ctx context.Context, oc CheckOutcome) (*HealthChange, error) {
|
||||
_ = s.metrics.InsertSamples(ctx, oc.TargetID, []ports.MetricSample{
|
||||
{Metric: oc.Kind + "_value", Value: oc.Value, Timestamp: time.Now()},
|
||||
})
|
||||
|
||||
switch oc.State {
|
||||
case "ok":
|
||||
_, _ = s.signals.Transition(ctx, ports.SignalTransitionInput{
|
||||
SignalID: oc.EntityID, Action: "resolve", Actor: "scheduler",
|
||||
})
|
||||
case "critical", "warning":
|
||||
severity := domain.SeverityWarning
|
||||
if oc.State == "critical" {
|
||||
severity = domain.SeverityCritical
|
||||
}
|
||||
_ = s.signals.UpsertWithTriggers(ctx, ports.SignalUpsertInput{
|
||||
Signal: domain.Signal{
|
||||
EntityID: oc.TargetID,
|
||||
Kind: oc.Kind,
|
||||
Severity: severity,
|
||||
State: domain.SignalRaised,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
health := aggregateCheckStates(oc.State, oc.PrevHealth)
|
||||
if health == oc.PrevHealth {
|
||||
return nil, nil
|
||||
}
|
||||
return &HealthChange{TargetID: oc.TargetID, Slug: oc.Slug, From: oc.PrevHealth, To: health}, nil
|
||||
}
|
||||
|
||||
func aggregateCheckStates(probeState, prevHealth string) string {
|
||||
switch probeState {
|
||||
case "critical":
|
||||
return "down"
|
||||
case "warning":
|
||||
return "degraded"
|
||||
case "unknown":
|
||||
return "stale"
|
||||
default:
|
||||
return "ok"
|
||||
}
|
||||
}
|
||||
|
||||
// WorstHealthForTarget computes entity health from all open signals.
|
||||
func (s *SignalService) WorstHealthForTarget(ctx context.Context, targetID domain.UUID) (string, error) {
|
||||
signals, err := s.signals.Open(ctx)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(signals) == 0 {
|
||||
return "ok", nil
|
||||
}
|
||||
sort.Slice(signals, func(i, j int) bool {
|
||||
return severityRank(signals[i].Severity) > severityRank(signals[j].Severity)
|
||||
})
|
||||
switch signals[0].Severity {
|
||||
case domain.SeverityCritical:
|
||||
return "down", nil
|
||||
case domain.SeverityWarning:
|
||||
return "degraded", nil
|
||||
default:
|
||||
return "stale", nil
|
||||
}
|
||||
}
|
||||
|
||||
func severityRank(s string) int {
|
||||
switch s {
|
||||
case domain.SeverityCritical:
|
||||
return 3
|
||||
case domain.SeverityWarning:
|
||||
return 2
|
||||
case domain.SeverityInfo:
|
||||
return 1
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
4
internal/core/domain/doc.go
Normal file
4
internal/core/domain/doc.go
Normal file
@@ -0,0 +1,4 @@
|
||||
// Package domain holds the pure model of the oikos core: entities, signals,
|
||||
// executions, approvals, patterns, and their sentinel errors. It imports
|
||||
// only the standard library.
|
||||
package domain
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user