Compare commits
47 Commits
39e9227fdb
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 7160eee1e1 | |||
| 30ecdc16c2 | |||
| d79b0862bd | |||
| 53823595de | |||
| 7ecf720166 | |||
| febc153b7f | |||
| 7d6a3320d4 | |||
| 60bc9d555d | |||
| ebe1b95acf | |||
| 5e10437fe3 | |||
| 7236c46e5c | |||
| 75c0848a6f | |||
| 712b66422b | |||
| a30c024ef8 | |||
| 137a2afb8d | |||
| c8b1ec5af2 | |||
| 8ff382a50d | |||
| fa79c1ea25 | |||
| ef762794e7 | |||
| 0b6c546aae | |||
| cecfd8b0e4 | |||
| 653ea3a116 | |||
| 89a94c24c9 | |||
| 7a34d8c8a0 | |||
| c9d506b0f8 | |||
| e3449b24c1 | |||
| 38c472a118 | |||
| 1d0197da69 | |||
| 0920c4cb6d | |||
| 2254a07baf | |||
| 1b9c761274 | |||
| a126cfa710 | |||
| 86fa57b5cd | |||
| 8e97d589af | |||
| 0dd8c28815 | |||
| 4e294b3630 | |||
| 85f0bb67fa | |||
| c3f478b8f8 | |||
| 1aaedf498a | |||
| 20adb89650 | |||
| 058f1afcdc | |||
| 2b73290994 | |||
| 428f4fe945 | |||
| 195d45a0e9 | |||
| 5b68bdc16c | |||
| 757ef2f34b | |||
| b27e1bf3ec |
@@ -30,9 +30,9 @@ one pass through **Observe → Orient → Decide → Act**:
|
||||
- **queue**: informational — console + reports
|
||||
The classifier can only *lower* autonomy relative to policy, never raise
|
||||
it. When in doubt, escalate.
|
||||
4. **Act** — execute through `homelab` commands or runbooks (never ad-hoc
|
||||
SSH), then **verify** with the action's verification command, write a
|
||||
**ledger** entry, resolve the Signal, and update docs in the same session.
|
||||
4. **Act** — execute through MCP `run` or runbooks (never ad-hoc
|
||||
SSH), then **verify** with the action's verification command, write a
|
||||
**ledger** entry, resolve the Signal, and update docs in the same session.
|
||||
|
||||
## Primitives
|
||||
|
||||
@@ -88,7 +88,7 @@ via the API's `/api/v1/graph` endpoint, and the Mermaid export at
|
||||
## Conventions carried forward
|
||||
|
||||
- Inventory is the truth; live state wins over narrative docs.
|
||||
- Prefer `homelab` CLI and MCP over ad-hoc SSH.
|
||||
- Prefer MCP tools over ad-hoc SSH.
|
||||
- Meaningful changes update docs in the same session.
|
||||
- Secrets are decrypted locally via per-client keys; never into docs/comments.
|
||||
- Tracked configs change by commit + push, not local edits.
|
||||
@@ -141,7 +141,7 @@ in the Go binary.
|
||||
- Standalone Nomos MCP client binary (`cmd/nomos`) with gateway mode
|
||||
(:8092). Structured queries + natural-language routing to the MCP tool
|
||||
list (see AGENTS.md §3). Agent activity logging on every tool call. No SSH keys.
|
||||
- `nomos/` directory with config, SOUL.md, homelab-ops skill.
|
||||
- `nomos/` directory with config, SOUL.md, `homelab-ops` skill at `nomos/skills/homelab-ops/`.
|
||||
- Nomos Docker service in `docker-compose.yml` (profile: full).
|
||||
- Go packages: `cmd/nomos/`, `compose/nomos/`.
|
||||
|
||||
|
||||
@@ -20,6 +20,6 @@ Run from the repo root:
|
||||
Exit code is non-zero when any violation is found, so it can gate a commit. The banned-vocabulary
|
||||
list mirrors `writing-style.md`; update both together if the standard changes.
|
||||
|
||||
> **Known baseline.** `archive/knowledge/archive/knowledge/containers/101-jellyfin.md` links into a sibling repo
|
||||
> **Known baseline.** `archive/knowledge/containers/101-jellyfin.md` links into a sibling repo
|
||||
> (`devops/homelab-authentik-admin`) that this checkout does not contain — expected, not a bug.
|
||||
> Any other broken link is a real regression; investigate before dismissing it as baseline noise.
|
||||
|
||||
@@ -36,8 +36,9 @@ ledger entry.
|
||||
6. Update the entity's `state` to `destroyed` in `seeds/inventory.yaml`
|
||||
(or move it to an `archaeology:`-style section if the schema still has
|
||||
one) — `pve_id`, `destroyed` date, `reason` — then `oikos seed` to
|
||||
ingest. Add a row to `containers/index.md` "Recently destroyed" table
|
||||
(kept for human-readable browsing alongside the structured data).
|
||||
ingest. Add a row to the legacy `archive/knowledge/containers/index.md`
|
||||
"Recently destroyed" table (kept for human-readable browsing
|
||||
alongside the structured data in the DB).
|
||||
7. No manual ledger step — mutations through the API are recorded
|
||||
automatically in the `audit_log` table (MCP `get_audit_trail`,
|
||||
`get_change_history`). The old `oikos/ledger.py append` was retired
|
||||
|
||||
@@ -27,9 +27,10 @@ chosen, doc page stub.
|
||||
will self-enroll as a client afterward (see
|
||||
[CLIENTS.md](../../../CLIENTS.md#enrollment)), the entity must exist in
|
||||
`planned`/`provisioning` state before `bootstrap.sh` runs there.
|
||||
3. Stub the doc page (`containers/<pve_id>-<name>.md` or
|
||||
`vms/<pve_id>-<name>.md`) — even a one-line "provisioning, see plan X"
|
||||
is enough to satisfy the transition requirement.
|
||||
3. Stub a document entity via MCP `upsert_knowledge` with
|
||||
`kind: document` and about set to the new entity slug — even a
|
||||
one-line "provisioning, see plan X" is enough to satisfy the
|
||||
transition requirement.
|
||||
4. Reserve the IP in DNS/DHCP notes if it's a fixed LAN address.
|
||||
|
||||
Next: [lifecycle-activate-node.md](../lifecycle-activate-node/SKILL.md).
|
||||
|
||||
@@ -78,5 +78,5 @@ Session: {id[:8]} — "{title[:60]}"
|
||||
- `internal/mcp/server.go` — all tool implementations (`run`, `list_lxcs`, …)
|
||||
- `web/src/lib/components/ToolCallGroup.svelte` — tool result display
|
||||
- `nomos/SOUL.md` — agent persona and tool selection rules
|
||||
- `plans/2026-07-09-chat-sessions-improvements.md` — prior session findings
|
||||
- `plans/2026-07-09-session-execution-and-ux-fixes.md` — latest plan
|
||||
- `plans/done/2026-07-09-chat-sessions-improvements.md` — prior session findings
|
||||
- `plans/done/2026-07-09-session-execution-and-ux-fixes.md` — latest plan
|
||||
|
||||
4
.gitignore
vendored
4
.gitignore
vendored
@@ -24,3 +24,7 @@ cmd/desktop/build/
|
||||
cmd/desktop/Oikos
|
||||
desktop
|
||||
/eval
|
||||
|
||||
# Local tooling artifacts (Playwright MCP session logs, stray screenshots)
|
||||
.playwright-mcp/
|
||||
config-screen.png
|
||||
|
||||
120
AGENTS.md
120
AGENTS.md
@@ -56,67 +56,79 @@ Endpoint: `https://mcp.hubris.network/mcp`. Every call needs
|
||||
enrollment and `/healthz` (see "Authentication" below for where the token
|
||||
comes from).
|
||||
|
||||
Available tools (the authoritative list — count them below if a number is
|
||||
needed; do not hardcode the count elsewhere):
|
||||
Available tools (63 total — the authoritative list; do not hardcode the count
|
||||
elsewhere; regenerate from `internal/mcp/` when tools change):
|
||||
|
||||
Context — observe + orient:
|
||||
get_entity(slug), list_entities(type, limit, cursor),
|
||||
get_relations(entity), get_blast_radius(entity),
|
||||
search_knowledge(query) — ILIKE search over documents, investigations,
|
||||
runbooks in the knowledge_entities table
|
||||
get_entity_knowledge(entity_slug) — every document, investigation, and
|
||||
runbook linked to one entity, in one call
|
||||
get_patterns(status, entity_type, action) — learned action patterns
|
||||
get_skills(status) — available automation skills
|
||||
http_get(url) — fetch a public page/raw file (e.g. researching how to
|
||||
deploy something before provisioning it); HTTP/HTTPS only, ~16KB cap
|
||||
|
||||
Management — live state:
|
||||
get_service_status(service_slug) — systemctl is-active on target host
|
||||
tail_log(service_slug, lines=200) — journalctl
|
||||
list_lxcs() — all LXC containers with ID, host, IP, health
|
||||
get_lxc_state(lxc_slug) — pct status from Proxmox host
|
||||
ping_service(service_slug) — HTTP reachability from entity_status
|
||||
list_my_secrets(caller_pubkey) — secrets accessible to this client by
|
||||
age public key
|
||||
|
||||
Oikos — decisions:
|
||||
Entity Tools — knowledge graph, discovery, and lifecycle:
|
||||
ping — lightweight connectivity check
|
||||
get_entity(slug_or_id) — get an entity by slug or UUID
|
||||
list_entities(type, state, q, limit) — entities filtered by type, state, or search
|
||||
get_relations(entity_id, types) — list inbound/outbound edges for one entity
|
||||
get_blast_radius(entity_id, depth=3) — entities affected if this one goes down
|
||||
create_entity(type, name, slug, attributes, state) — create a new entity in the graph
|
||||
update_entity_attributes(slug, attributes) — merge discovered facts into an entity
|
||||
set_entity_state(slug, state) — transition entity to a new lifecycle state
|
||||
create_relationship(source, target, type) — record a discovered edge
|
||||
end_relationship(source, target, type) — soft-delete an active edge
|
||||
whoami(hostname) — entity record, peers, and health for a host
|
||||
explain(service_slug) — compact context card (type, state, health, relations)
|
||||
preflight(service_slug, action) — risk class + approval requirement
|
||||
whoami(hostname) — entity record, peers, health for a client
|
||||
get_change_history(entity_slug, limit=20) — last audit-log entries per entity
|
||||
get_state_snapshot() — fleet health, disk, drift count
|
||||
get_state_snapshot() — last scheduler Observe-pass: fleet health, disk, drift
|
||||
audit_knowledge_graph() — read-only drift report over the graph and checks
|
||||
discover_infra_drift() — running guests vs DB: missing/ghost entities
|
||||
find_entities_by(key, value, limit=25) — search entities by attribute values
|
||||
|
||||
Operations — observe + act:
|
||||
get_health_summary() — fleet health counts (healthy/degraded/down/unknown)
|
||||
get_signal_history(entity_slug, state, limit) — open + recent signals
|
||||
get_audit_trail(entity_id) — audit log filter + browse
|
||||
get_agent_activity(limit) — agent self-inspection
|
||||
query_metrics(hours=24) — time-series metric bucketed averages
|
||||
get_trend(entity_id, days=7) — metric slope over time
|
||||
get_event_timeline(severity, entity_slug, limit) — recent events
|
||||
Ops Tools — live state, signals, checks, and execution:
|
||||
run(target, command, purpose, declared_risk) — general execution primitive; read-only auto-acts, mutations queue for approval, destructive always needs explicit confirmation
|
||||
inspect_path(path, targets) — bulk mount/df/ls/stat across multiple hosts/LXCs
|
||||
get_execution_status(execution_id) — poll execution progress
|
||||
tail_log(service_slug, lines=50) — journalctl for a service
|
||||
get_service_status(service_slug) — systemctl is-active/is-enabled
|
||||
get_lxc_state(lxc_slug) — pct status from Proxmox host
|
||||
ping_service(service_slug) — HTTP reachability + scheduler health state
|
||||
list_lxcs(state) — all LXC containers with ID, host, IP, last-audited hint
|
||||
ack_signal(signal_id) — acknowledge an open signal
|
||||
resolve_signal(signal_id, resolution) — resolve a signal with optional note
|
||||
mute_signal(signal_id, duration_s=3600) — temporarily mute a signal
|
||||
cancel_execution(execution_id, reason) — cancel a queued/running execution
|
||||
update_check(check_id, enabled) — enable or disable a health check
|
||||
list_checks(entity_slug, enabled) — list health checks with verdict, probe kind
|
||||
list_executions(entity_slug, status, limit=25) — cursor-paginated execution history
|
||||
list_entity_sessions(entity_slug) — active Nomos sessions linked to an entity
|
||||
get_dashboard_summary() — fleet overview: counts, health, signals, approvals
|
||||
get_secret(key, path, environment) — retrieve a secret from the Infisical vault
|
||||
list_secrets(path_prefix) — list secret keys in the Infisical vault
|
||||
set_secret(key, value, path, environment) — store/update a secret (requires approval)
|
||||
|
||||
Knowledge — keep the graph current (none require approval; this updates
|
||||
the knowledge graph, not live infrastructure):
|
||||
upsert_knowledge(title, content) — record what you learned after solving
|
||||
a non-obvious problem; the only way anything persists past a session
|
||||
update_entity_attributes(slug, attributes) — merge a discovered fact
|
||||
(IP, version, port, ...) into an entity so a future task doesn't
|
||||
rediscover it from scratch
|
||||
create_relationship(source, target, type) — record a discovered edge
|
||||
(depends-on, hosts, routes-to, ...) between two entities
|
||||
Knowledge Tools — search, read, and maintain the knowledge base:
|
||||
search_knowledge(query) — full-text search across docs (snippets, not full body)
|
||||
get_entity_knowledge(entity_slug) — all docs/investigations/runbooks linked to a slug
|
||||
get_knowledge_content(slug) — full markdown body of one knowledge entry
|
||||
upsert_knowledge(title, content, about, tags, kind) — write what you learned
|
||||
delete_knowledge(knowledge_slug) — soft-delete a knowledge entry
|
||||
restore_knowledge(knowledge_slug) — restore a soft-deleted entry
|
||||
merge_knowledge(target_slug, source_slugs) — fold entries into a target
|
||||
rename_knowledge_tag(from, to) — bulk-rename tags across all entries
|
||||
get_knowledge_revisions(knowledge_slug) — version history for a knowledge entry
|
||||
get_knowledge_duplicates(threshold=0.6) — near-duplicate detection via trigram similarity
|
||||
get_knowledge_orphans(stale_days=90) — unlinked, untagged, or stale entries
|
||||
list_knowledge_tags() — all tags with usage counts and casing variants
|
||||
list_my_secrets(caller_pubkey) — secrets accessible to a client by age public key
|
||||
|
||||
Execution — mutating the live infrastructure:
|
||||
run(target, command) — the general execution primitive. Run any shell
|
||||
command against a host or LXC; every command is auto-classified —
|
||||
read-only inspection runs immediately, anything state-changing needs
|
||||
operator approval, and destructive patterns (rm -rf, dd, mkfs,
|
||||
pct/qm destroy, DROP TABLE, reboot, curl-pipe-to-shell, ...) always
|
||||
need approval regardless of what you declare. This is the ONLY
|
||||
mutation tool — `request_execution` was retired 2026-07-14; the
|
||||
former enum actions (restart, systemctl, pct_exec, apt_upgrade,
|
||||
pct_create) are all expressed as `run(target, command)` now.
|
||||
get_execution_status(execution_id) — poll progress
|
||||
Analysis Tools — fleet health, metrics, and introspection:
|
||||
get_health_summary(health) — fleet health per entity, optionally filtered
|
||||
get_audit_trail(entity_id) — query the audit log
|
||||
query_metrics(hours=24) — time-series with bucketed avg/min/max
|
||||
get_signal_history(entity_slug, state, limit=50) — open and recent signals
|
||||
get_patterns(status, entity_type, action) — learned action patterns
|
||||
get_skills(status) — available automation skills
|
||||
get_trend(entity_id, days=7) — metric slope, variance, and averages
|
||||
get_event_timeline(severity, entity_slug, limit=50) — recent events
|
||||
get_agent_activity(limit=50) — agent self-inspection log
|
||||
classify_command(command, declared_risk) — pre-flight risk classification before `run`
|
||||
get_ontology() — entity types, relationship types, and lifecycle definitions
|
||||
http_get(url) — fetch a public web page/raw file; ~16KB cap
|
||||
|
||||
**When to prefer MCP over grepping the clone:** always for knowledge queries.
|
||||
`search_knowledge("jellyfin hardware acceleration")` returns ranked results from
|
||||
|
||||
@@ -3185,8 +3185,6 @@ components:
|
||||
required:
|
||||
- age_public_key
|
||||
- age_private_key
|
||||
- infisical_client_id
|
||||
- infisical_client_secret
|
||||
properties:
|
||||
age_public_key:
|
||||
type: string
|
||||
@@ -3200,9 +3198,6 @@ components:
|
||||
infisical_client_secret:
|
||||
type: string
|
||||
description: Infisical UniversalAuth client secret
|
||||
machine_identity_token:
|
||||
type: string
|
||||
description: Infisical machine identity access token
|
||||
ClientContext:
|
||||
type: object
|
||||
required:
|
||||
|
||||
104
archive/knowledge/infrastructure/oikos-check-lifecycle.md
Normal file
104
archive/knowledge/infrastructure/oikos-check-lifecycle.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# Oikos check lifecycle — how monitoring works
|
||||
|
||||
This runbook covers how Oikos health checks are derived, created, and wired so
|
||||
an agent (Nomos) doesn't reverse-engineer source when asked to add monitoring to
|
||||
an entity — the problem that stranded session `23da10db` (2026-08-03).
|
||||
|
||||
## Concepts
|
||||
|
||||
- **`check_defs`** (scheduler config, table `check_defs`): the row the scheduler
|
||||
reads to know *what* to probe and *when*. One per check instance.
|
||||
- **`check` entity** (type `check`, slug `check:<kind>:<target>:<n>`): the
|
||||
knowledge-graph entity for that check. It carries attributes
|
||||
(`check_type`, `target`, `port`, …) and `checks` edges to the probed target.
|
||||
- **`monitoring` spec** on an entity type (`entity_types.monitoring_spec`): the
|
||||
default list of check kinds (e.g. `[http, process]` for `service`).
|
||||
- Per-entity override: set `monitoring` in the entity's attributes —
|
||||
`"none"` for zero checks, `["http"]` to replace the type defaults.
|
||||
- **`checkdefaults.Ensure`** (`internal/checkdefaults/defaults.go`): the
|
||||
function that reads the monitoring spec, resolves host/port/URL from
|
||||
attributes + relationships, and writes `check_defs` rows. Idempotent.
|
||||
|
||||
## When checks are derived
|
||||
|
||||
`checkdefaults.Ensure` runs in three situations (as of v0.17.1+):
|
||||
|
||||
1. **Seed/deploy ingest** — `internal/db/seed.go:231`. Every entity gets its
|
||||
default checks once on initial ingest.
|
||||
2. **HTTP `POST /api/v1/entities` (create)** — `ensureDefaultChecks` at
|
||||
`internal/httpapi/impl.go:1012`. Creating an entity via the REST API derives
|
||||
its checks in the same transaction.
|
||||
3. **HTTP `PATCH /api/v1/entities` (patch)** — `ensureDefaultChecks` at
|
||||
`internal/httpapi/impl.go:1280`. Changing an entity's attributes (especially
|
||||
`monitoring`) via the REST API regenerates its checks.
|
||||
4. **MCP `create_entity`** — SAME hook. Creating an entity via the MCP tool
|
||||
derives checks. (Added 2026-08-03; previously MCP had no create.)
|
||||
5. **MCP `update_entity_attributes`** — SAME hook. Changing an entity's
|
||||
`monitoring` attribute via MCP now regenerates checks. (Added 2026-08-03;
|
||||
previously MCP updates silently skipped check derivation — the exact bug
|
||||
that stranded the haos session.)
|
||||
|
||||
## Check slug grammar
|
||||
|
||||
```
|
||||
check:<kind>:<target-type>:<target-name>:<n>
|
||||
```
|
||||
|
||||
Examples: `check:http:service:jellyfin:0`, `check:vm-status:vm:haos:0`,
|
||||
`check:cert-expiry:cert:house.hubris.network:0`.
|
||||
|
||||
## Adding monitoring to an entity
|
||||
|
||||
**If the entity already exists:**
|
||||
|
||||
```
|
||||
update_entity_attributes(slug="service:haos", attributes={"monitoring":["http"]})
|
||||
```
|
||||
|
||||
This regenerates checks via `checkdefaults.Ensure`. The result message tells you
|
||||
how many checks were derived and whether any kinds were skipped (and why).
|
||||
|
||||
**If the entity does not exist yet (a new check, ingress, cert, etc.):**
|
||||
|
||||
```
|
||||
create_entity(type="check", name="HAOS http check",
|
||||
slug="check:http:service:haos:0",
|
||||
attributes={"check_type":"http:service","target":"service:haos","port":"8123"})
|
||||
```
|
||||
|
||||
This creates the entity AND derives its `check_defs`. Same for a new `ingress`
|
||||
(`type=ingress`, monitoring `[http]`) or `cert` (`type=cert`,
|
||||
monitoring `[cert-expiry]`).
|
||||
|
||||
**To remove monitoring:** set `monitoring:["none"]` or transition the entity
|
||||
to a terminal lifecycle state (`set_entity_state` → `deprecated`/`destroyed`).
|
||||
|
||||
## Caveats
|
||||
|
||||
- **A service without a `url` attribute AND without a `probe_unit` gets no
|
||||
process check** (the http check covers liveness; the process check would
|
||||
be redundant without an opt-in `probe_unit`). The skip is logged.
|
||||
- **A service whose address comes from a `hosts` edge** may produce no checks on
|
||||
initial create because the edge doesn't exist yet — the next inventory ingest
|
||||
(or a later `update_entity_attributes` after the edge is created) fills it in.
|
||||
- **A `not found` error from `update_entity_attributes`** means the entity
|
||||
doesn't exist — use `create_entity` instead.
|
||||
- **`check_defs` has target columns** (`target_id`, `target_type`). A check
|
||||
entity needs a `checks` relationship (`create_relationship(source=check:…,
|
||||
target=service:…, type="checks")`) so the scheduler can resolve what to
|
||||
probe. `create_entity` derives the check_def; `create_relationship` links
|
||||
the check entity to its target in the graph.
|
||||
|
||||
## Related files
|
||||
|
||||
- `internal/checkdefaults/defaults.go` — `Ensure`, `Target`, `LogResult`
|
||||
- `internal/httpapi/default_checks.go` — `ensureDefaultChecks` (HTTP hook)
|
||||
- `internal/db/checks.go` — `db.EnsureEntityChecks` (shared hook)
|
||||
- `internal/db/seed.go` — seed-time check derivation
|
||||
- `internal/mcp/tools.go` — `create_entity`, `update_entity_attributes`
|
||||
|
||||
## Revision history
|
||||
|
||||
- **2026-08-03:** Created after session `23da10db` stranded for lack of entity-
|
||||
creation tool and unawareness of check-derivation triggers. Covers the MCP
|
||||
create_entity + update_entity_attributes regen paths added same day.
|
||||
12
checks/pvecm_quorum_check.sh
Normal file
12
checks/pvecm_quorum_check.sh
Normal file
@@ -0,0 +1,12 @@
|
||||
#!/usr/bin/env bash
|
||||
# pvecm_quorum_check.sh — Proxmox cluster quorum status.
|
||||
# Runs on a PVE host. Fails if the node is not quorate.
|
||||
set -euo pipefail
|
||||
|
||||
# pvecm status exit code is non-zero on non-quorate nodes
|
||||
# (e.g. "Quorate: No — Activity blocked")
|
||||
if pvecm status 2>/dev/null | grep -q 'Quorate.*Yes'; then
|
||||
echo '{"health":"healthy","metrics":{"quorate":1}}'
|
||||
else
|
||||
echo '{"health":"unhealthy","metrics":{"quorate":0}}'
|
||||
fi
|
||||
@@ -60,11 +60,15 @@ type agent struct {
|
||||
// gate serializes turns per session (at most one in-flight turn per
|
||||
// sessionID). See turngate.go and plan 2026-08-03 F1.
|
||||
gate *turnGate
|
||||
// queue holds operator messages that arrived while a turn was already
|
||||
// running; they are auto-run when the gate frees (plan 2026-08-03 F2).
|
||||
// See messagequeue.go.
|
||||
queue *messageQueue
|
||||
}
|
||||
|
||||
func newAgent(ctx context.Context, clients *mcpClientPool, st *store, agentSlug string) (*agent, error) {
|
||||
func newAgent(ctx context.Context, clients *mcpClientPool, st *store, agentSlug string, openrouterAPIKey string) (*agent, error) {
|
||||
system := loadSoul()
|
||||
apiKey := os.Getenv("OPENROUTER_API_KEY")
|
||||
apiKey := openrouterAPIKey
|
||||
model := os.Getenv("NOMOS_MODEL")
|
||||
if model == "" {
|
||||
// v4-pro over v4-flash: the flash tier over-narrates, occasionally
|
||||
@@ -121,6 +125,7 @@ func newAgent(ctx context.Context, clients *mcpClientPool, st *store, agentSlug
|
||||
apiToken: os.Getenv("OIKOS_MCP_BEARER_TOKEN"),
|
||||
httpClient: &http.Client{Timeout: 15 * time.Second},
|
||||
gate: newTurnGate(),
|
||||
queue: newMessageQueue(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -176,6 +181,11 @@ type agentEvent struct {
|
||||
Data any `json:"data,omitempty"`
|
||||
SessionID string `json:"session_id,omitempty"`
|
||||
Iteration int `json:"iteration,omitempty"`
|
||||
// IsThinking marks text/text_delta events that carry the model's internal
|
||||
// reasoning (text produced before tool calls in the same iteration), as
|
||||
// distinct from the final response text. The frontend renders these as
|
||||
// collapsible thinking blocks separated from the response.
|
||||
IsThinking bool `json:"is_thinking,omitempty"`
|
||||
}
|
||||
|
||||
func (a *agent) chat(ctx context.Context, sessionID, message string, emit func(agentEvent)) {
|
||||
@@ -372,7 +382,12 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
var msg openai.ChatCompletionMessage
|
||||
var acc openai.ChatCompletionAccumulator
|
||||
|
||||
for attempt := 0; attempt <= maxLLMRetries; attempt++ {
|
||||
// Capture token usage from this LLM response for activity logging.
|
||||
// Previously always NULL — every agent_activity row had no token
|
||||
// count. Now each tool call in this iteration gets the same total.
|
||||
totalTokens := 0
|
||||
|
||||
for attempt := 0; attempt <= maxLLMRetries; attempt++ {
|
||||
acc = openai.ChatCompletionAccumulator{}
|
||||
stream := a.provider.Chat.Completions.NewStreaming(ctx, params, a.reqOpts...)
|
||||
for stream.Next() {
|
||||
@@ -404,14 +419,19 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
msg = acc.Choices[0].Message
|
||||
finishReason := acc.Choices[0].FinishReason
|
||||
|
||||
// Capture token usage from this iteration.
|
||||
if acc.Usage.TotalTokens > 0 {
|
||||
totalTokens = int(acc.Usage.TotalTokens)
|
||||
}
|
||||
|
||||
if len(msg.ToolCalls) == 0 {
|
||||
if isRefusalOrEmpty(msg.Content) {
|
||||
if attempt < maxLLMRetries {
|
||||
slog.Warn("nomos: empty or refusal response, retrying",
|
||||
"session", sessionID, "iter", i+1, "attempt", attempt+1,
|
||||
"content_len", len(msg.Content), "finish_reason", finishReason)
|
||||
continue
|
||||
}
|
||||
if attempt < maxLLMRetries {
|
||||
slog.Warn("nomos: empty or refusal response, retrying",
|
||||
"session", sessionID, "iter", i+1, "attempt", attempt+1,
|
||||
"content_len", len(msg.Content), "finish_reason", finishReason)
|
||||
continue
|
||||
}
|
||||
// B.4: surface the real error context (finish_reason +
|
||||
// refusal text) instead of a generic "empty response" —
|
||||
// the operator can tell "content_filter — rephrase" from
|
||||
@@ -460,7 +480,7 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
// led to each step. Emitting it lets the persist layer accumulate
|
||||
// per-iteration reasoning into the row's text field.
|
||||
if strings.TrimSpace(msg.Content) != "" {
|
||||
emit(agentEvent{Type: "text", Data: msg.Content, SessionID: sessionID})
|
||||
emit(agentEvent{Type: "text", Data: msg.Content, SessionID: sessionID, IsThinking: true})
|
||||
}
|
||||
|
||||
slog.Info("nomos: tool calls", "count", len(msg.ToolCalls), "iter", i+1, "correlation", correlationID)
|
||||
@@ -495,7 +515,7 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
slog.Warn("nomos: run retry cap hit — refusing dispatch",
|
||||
"target", t, "failures", n, "session", sessionID)
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, args,
|
||||
tc.Function.Arguments, directive, 0, false, correlationID)
|
||||
tc.Function.Arguments, directive, 0, false, correlationID, totalTokens)
|
||||
emit(agentEvent{
|
||||
Type: "tool_result",
|
||||
Data: map[string]any{"name": tc.Function.Name, "result": directive, "id": tc.ID, "retry_capped": true},
|
||||
@@ -546,7 +566,7 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
inputStr := string(inputJSON)
|
||||
|
||||
if callErr != nil {
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, args, inputStr, callErr.Error(), elapsed, false, correlationID)
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, args, inputStr, callErr.Error(), elapsed, false, correlationID, totalTokens)
|
||||
|
||||
// Retry cap: dispatch errors (e.g. MCP client timeout)
|
||||
// count toward the cap too. A command that keeps timing
|
||||
@@ -576,7 +596,7 @@ func (a *agent) chatWith(ctx context.Context, sessionID, message, systemInject s
|
||||
}
|
||||
|
||||
resultJSON, _ := json.Marshal(result)
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, args, inputStr, string(resultJSON), elapsed, true, correlationID)
|
||||
a.store.logActivity(ctx, a.agentID, sessionID, tc.Function.Name, args, inputStr, string(resultJSON), elapsed, true, correlationID, totalTokens)
|
||||
|
||||
// Link any execution this tool queued/started back to this
|
||||
// session, so the auto-continuation worker can feed its result
|
||||
|
||||
@@ -222,7 +222,13 @@ func (a *agent) resumeSession(ctx context.Context, sessionID, note string) bool
|
||||
slog.Info("nomos: turn already active, skipping background resume", "session", sessionID)
|
||||
return false
|
||||
}
|
||||
defer a.gate.release(sessionID)
|
||||
// Release the gate, then drain any operator message that was queued while
|
||||
// this background turn ran (plan 2026-08-03 F2). Queued messages are run as
|
||||
// real user turns server-side; resumeSession itself never enqueues.
|
||||
defer func() {
|
||||
a.gate.release(sessionID)
|
||||
safego.Go("nomos:drain:"+sessionID, func() { a.drainQueued(context.Background(), sessionID) })
|
||||
}()
|
||||
|
||||
placeholder, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
@@ -236,6 +242,7 @@ func (a *agent) resumeSession(ctx context.Context, sessionID, note string) bool
|
||||
|
||||
var toolCalls []map[string]any
|
||||
var finalText, errText string
|
||||
var finalThinking string
|
||||
|
||||
persist := func() {
|
||||
if msgID == uuid.Nil {
|
||||
@@ -248,6 +255,7 @@ func (a *agent) resumeSession(ctx context.Context, sessionID, note string) bool
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": text,
|
||||
"thinking": finalThinking,
|
||||
"tool_calls": toolCalls,
|
||||
"auto": true, // marks this as an autonomous continuation, not an operator turn
|
||||
})
|
||||
@@ -283,10 +291,14 @@ func (a *agent) resumeSession(ctx context.Context, sessionID, note string) bool
|
||||
}
|
||||
}
|
||||
toolCalls, finalText, errText = nil, "", ""
|
||||
finalThinking = ""
|
||||
// P3: accumulate per-iteration reasoning instead of overwriting
|
||||
// (same fix as main.go's chat handler). Without this, a resumed
|
||||
// turn's intermediate thinking is lost on reload.
|
||||
// (same fix as main.go's chat handler). Without this, a resumed
|
||||
// turn's intermediate thinking is lost on reload.
|
||||
var textParts []string
|
||||
var thinkingParts []string
|
||||
emit := func(ev agentEvent) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
@@ -313,8 +325,13 @@ func (a *agent) resumeSession(ctx context.Context, sessionID, note string) bool
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
if t, ok := ev.Data.(string); ok && t != "" {
|
||||
textParts = append(textParts, t)
|
||||
finalText = strings.Join(textParts, "\n\n")
|
||||
if ev.IsThinking {
|
||||
thinkingParts = append(thinkingParts, t)
|
||||
finalThinking = strings.Join(thinkingParts, "\n\n")
|
||||
} else {
|
||||
textParts = append(textParts, t)
|
||||
finalText = strings.Join(textParts, "\n\n")
|
||||
}
|
||||
persist()
|
||||
}
|
||||
}
|
||||
|
||||
348
cmd/nomos/mcp.go
Normal file
348
cmd/nomos/mcp.go
Normal file
@@ -0,0 +1,348 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ─── MCP Streamable HTTP client ────────────────────────────────────────
|
||||
|
||||
type mcpClient struct {
|
||||
baseURL string
|
||||
token string // OIKOS_MCP_BEARER_TOKEN — api's combinedAuth requires it on every request (no dev-open bypass)
|
||||
sessionID string
|
||||
http *http.Client
|
||||
nextID int
|
||||
mu sync.Mutex // one client serializes its own MCP calls (the pool gives each session its own client, so this never blocks another session)
|
||||
// toolsCache holds the last tools/list result. The tool list is static
|
||||
// for the lifetime of one MCP connection — it only changes when the api
|
||||
// process (re)registers tools, i.e. on a restart, which this client
|
||||
// already detects and reacts to via reconnectLocked. Without this,
|
||||
// buildTools (called at the start of EVERY chat turn, including every
|
||||
// auto-continuation resume) paid a full tools/list round-trip every
|
||||
// single time for a list that's almost always identical to the last one.
|
||||
// Guarded separately from mu (not reused) so a cache check never
|
||||
// contends with an in-flight doRequest call for a different method.
|
||||
toolsMu sync.Mutex
|
||||
toolsCache []toolDef
|
||||
}
|
||||
|
||||
func newMCPClient(baseURL, token string) (*mcpClient, error) {
|
||||
c := &mcpClient{
|
||||
baseURL: baseURL,
|
||||
token: token,
|
||||
http: &http.Client{Timeout: 120 * time.Second},
|
||||
}
|
||||
|
||||
resp, err := c.doRequest("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initialize: %w", err)
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return nil, fmt.Errorf("no session ID in initialize response")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
|
||||
c.doRequest("notifications/initialized", map[string]any{})
|
||||
|
||||
slog.Info("nomos: mcp connected", "session", c.sessionID[:16]+"...")
|
||||
return c, nil
|
||||
}
|
||||
|
||||
type mcpJSONRPCResponse struct {
|
||||
sessionID string
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error json.RawMessage `json:"error"`
|
||||
}
|
||||
|
||||
// errStaleSession signals that the MCP server rejected our session id (e.g.
|
||||
// after an api/MCP restart), so the client should re-initialize and retry.
|
||||
var errStaleSession = fmt.Errorf("mcp session stale")
|
||||
|
||||
// doRequest serializes MCP calls and transparently re-initializes the session
|
||||
// if the server has forgotten it (common after an api redeploy), retrying the
|
||||
// original call once. Without this, an api restart permanently breaks nomos
|
||||
// until it is itself restarted.
|
||||
func (c *mcpClient) doRequest(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
resp, err := c.send(method, params)
|
||||
if err != nil && method != "initialize" && errors.Is(err, errStaleSession) {
|
||||
slog.Warn("nomos: mcp session stale, reconnecting")
|
||||
if rerr := c.reconnectLocked(); rerr != nil {
|
||||
return nil, fmt.Errorf("mcp reconnect: %w (original: %v)", rerr, err)
|
||||
}
|
||||
return c.send(method, params)
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
// reconnectLocked re-initializes the MCP session. The caller must hold c.mu.
|
||||
func (c *mcpClient) reconnectLocked() error {
|
||||
c.sessionID = ""
|
||||
// A reconnect means the api process was restarted (or forgot us) — its
|
||||
// tool registration may have changed, so the cached list is no longer
|
||||
// trustworthy.
|
||||
c.toolsMu.Lock()
|
||||
c.toolsCache = nil
|
||||
c.toolsMu.Unlock()
|
||||
resp, err := c.send("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return fmt.Errorf("no session ID on re-initialize")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
_, _ = c.send("notifications/initialized", map[string]any{})
|
||||
slog.Info("nomos: mcp reconnected", "session", c.sessionID[:16]+"...")
|
||||
return nil
|
||||
}
|
||||
|
||||
// send performs one MCP round-trip. It does not lock; callers hold c.mu.
|
||||
func (c *mcpClient) send(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.nextID++
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"jsonrpc": "2.0",
|
||||
"method": method,
|
||||
"params": params,
|
||||
"id": c.nextID,
|
||||
})
|
||||
|
||||
req, err := http.NewRequest(http.MethodPost, c.baseURL, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json, text/event-stream")
|
||||
if c.sessionID != "" {
|
||||
req.Header.Set("Mcp-Session-Id", c.sessionID)
|
||||
}
|
||||
if c.token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
}
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// A rejected/unknown session comes back as 4xx (commonly 400/404).
|
||||
if resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusBadRequest {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
result := &mcpJSONRPCResponse{}
|
||||
result.sessionID = resp.Header.Get("Mcp-Session-Id")
|
||||
|
||||
scanner := bufio.NewScanner(resp.Body)
|
||||
scanner.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
|
||||
gotData := false
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if strings.HasPrefix(line, "data: ") {
|
||||
gotData = true
|
||||
data := line[6:]
|
||||
if err := json.Unmarshal([]byte(data), result); err != nil {
|
||||
return nil, fmt.Errorf("parse response: %w", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if result.Error != nil {
|
||||
return nil, fmt.Errorf("rpc error: %s", string(result.Error))
|
||||
}
|
||||
|
||||
// Empty body with no result and a session set: the server likely dropped
|
||||
// our session. Notifications legitimately return no data, so exempt them.
|
||||
if !gotData && result.Result == nil && method != "notifications/initialized" {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
if result.sessionID != "" {
|
||||
c.sessionID = result.sessionID
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) callTool(name string, args map[string]any) (any, error) {
|
||||
resp, err := c.doRequest("tools/call", map[string]any{
|
||||
"name": name,
|
||||
"arguments": args,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var toolResult struct {
|
||||
Content []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &toolResult); err != nil {
|
||||
return string(resp.Result), nil
|
||||
}
|
||||
|
||||
var texts []string
|
||||
for _, c := range toolResult.Content {
|
||||
if c.Type == "text" {
|
||||
var parsed any
|
||||
if json.Unmarshal([]byte(c.Text), &parsed) == nil {
|
||||
return parsed, nil
|
||||
}
|
||||
texts = append(texts, c.Text)
|
||||
}
|
||||
}
|
||||
if len(texts) == 1 {
|
||||
return texts[0], nil
|
||||
}
|
||||
return texts, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) listTools() ([]string, error) {
|
||||
resp, err := c.doRequest("tools/list", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tr struct {
|
||||
Tools []struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
} `json:"tools"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &tr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var names []string
|
||||
for _, t := range tr.Tools {
|
||||
names = append(names, t.Name)
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) close() {
|
||||
}
|
||||
|
||||
// ─── Per-session MCP client pool ────────────────────────────────────────
|
||||
//
|
||||
// A single shared mcpClient serializes EVERY tool call across EVERY
|
||||
// concurrently-running task through one mutex (see mcpClient.mu) — `run`
|
||||
// executes its SSH command synchronously inside that lock and is capped at
|
||||
// up to 10 minutes, so one task mid-`run` stalled every other task's tool
|
||||
// calls, even trivial reads, behind it. The MCP *server* has no per-
|
||||
// connection state to protect (newServer in internal/mcp/server.go returns
|
||||
// one shared *mcp.Server instance whose tool handlers close only over the DB
|
||||
// pool, which is already safe for concurrent use) — the mutex existed purely
|
||||
// because the *client* reused one stateful transport session, not because
|
||||
// the server needed it. Giving each task's own session its own client
|
||||
// removes the cross-task serialization entirely: a task's own tool calls
|
||||
// stay sequential (which they already are — the agent loop calls tools one
|
||||
// at a time within a turn), but no longer block anyone else's.
|
||||
type mcpClientPool struct {
|
||||
baseURL string
|
||||
token string
|
||||
mu sync.Mutex
|
||||
clients map[string]*pooledMCPClient
|
||||
}
|
||||
|
||||
type pooledMCPClient struct {
|
||||
client *mcpClient
|
||||
lastUsed time.Time
|
||||
}
|
||||
|
||||
func newMCPClientPool(baseURL, token string) *mcpClientPool {
|
||||
return &mcpClientPool{baseURL: baseURL, token: token, clients: make(map[string]*pooledMCPClient)}
|
||||
}
|
||||
|
||||
// get returns the client for sessionID, creating and initializing one (a
|
||||
// real MCP handshake) on first use. Session ids that don't identify a real
|
||||
// persisted conversation ("" / "ephemeral", the no-DB-store path; "query",
|
||||
// the structured /query endpoint) still get exactly one dedicated,
|
||||
// reused client each via the same map — just keyed on a fixed string instead
|
||||
// of a real session id — so that traffic doesn't pay a fresh handshake per
|
||||
// request while still never sharing a connection with an actual task.
|
||||
func (p *mcpClientPool) get(sessionID string) (*mcpClient, error) {
|
||||
key := sessionID
|
||||
if key == "" {
|
||||
key = "ephemeral"
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
if pc, ok := p.clients[key]; ok {
|
||||
pc.lastUsed = time.Now()
|
||||
p.mu.Unlock()
|
||||
return pc.client, nil
|
||||
}
|
||||
p.mu.Unlock()
|
||||
|
||||
// Initialize outside the lock — it's a network round-trip, and holding
|
||||
// the pool mutex for it would serialize unrelated sessions' first calls
|
||||
// behind each other, undermining the whole point of this pool.
|
||||
c, err := newMCPClient(p.baseURL, p.token)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
// Another goroutine may have created one for the same key while we were
|
||||
// initializing (two of this session's tool calls racing on a cold
|
||||
// start); keep whichever won, close out the loser's connection (a no-op
|
||||
// today, but future-proof if mcpClient.close ever does real teardown).
|
||||
if existing, ok := p.clients[key]; ok {
|
||||
p.mu.Unlock()
|
||||
c.close()
|
||||
return existing.client, nil
|
||||
}
|
||||
p.clients[key] = &pooledMCPClient{client: c, lastUsed: time.Now()}
|
||||
p.mu.Unlock()
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// mcpClientIdleTimeout is how long an idle session's MCP client is kept
|
||||
// before eviction — long enough to outlive a single slow `run` (capped at 10
|
||||
// minutes server-side) plus normal think-time between a task's tool calls,
|
||||
// short enough not to accumulate one abandoned connection per finished task
|
||||
// forever.
|
||||
const mcpClientIdleTimeout = 20 * time.Minute
|
||||
|
||||
// sweep evicts clients idle past mcpClientIdleTimeout. Call on a ticker.
|
||||
func (p *mcpClientPool) sweep() {
|
||||
cutoff := time.Now().Add(-mcpClientIdleTimeout)
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
if pc.lastUsed.Before(cutoff) {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *mcpClientPool) closeAll() {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
82
cmd/nomos/messagequeue.go
Normal file
82
cmd/nomos/messagequeue.go
Normal file
@@ -0,0 +1,82 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"log/slog"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// maxQueuedPerSession caps a session's queue. A held turn plus unbounded
|
||||
// enqueues would grow memory without limit; an operator nudging a long
|
||||
// autonomous turn realistically queues only a handful, so a generous cap is
|
||||
// pure insurance. Overflow drops the newest enqueue and logs (the message is
|
||||
// already persisted in the DB by handleChat before enqueue, so it isn't lost
|
||||
// from the transcript — it just won't auto-run).
|
||||
const maxQueuedPerSession = 20
|
||||
|
||||
// messageQueue holds operator messages that arrived while a turn was already
|
||||
// running for a session. Plan 2026-08-03 (F2): instead of rejecting the
|
||||
// operator's message with "Nomos is still finishing a previous step… send it
|
||||
// again", the message is queued and auto-run when the in-flight turn releases
|
||||
// the session's turn-gate permit.
|
||||
//
|
||||
// The queue only schedules WHEN a turn runs, not WHETHER the message is stored
|
||||
// — handleChat persists the user message before acquiring the gate, so a queued
|
||||
// message is already in the transcript; this just makes sure a turn eventually
|
||||
// acts on it.
|
||||
//
|
||||
// Draining is strictly one-at-a-time under the turn gate (see drainQueued in
|
||||
// main.go), so this cannot stack concurrent turns — the exact hazard the gate
|
||||
// itself exists to prevent. Background resumeSession callers never touch this
|
||||
// queue; they keep their non-blocking skip.
|
||||
type messageQueue struct {
|
||||
mu sync.Mutex
|
||||
queue map[string][]string
|
||||
}
|
||||
|
||||
func newMessageQueue() *messageQueue {
|
||||
return &messageQueue{queue: map[string][]string{}}
|
||||
}
|
||||
|
||||
// enqueue appends a message to the back of the session's FIFO. Returns false
|
||||
// (and logs) if the session is already at maxQueuedPerSession — the caller's
|
||||
// message is already persisted in the DB, so this only skips auto-running it.
|
||||
func (q *messageQueue) enqueue(sessionID, msg string) bool {
|
||||
q.mu.Lock()
|
||||
defer q.mu.Unlock()
|
||||
if len(q.queue[sessionID]) >= maxQueuedPerSession {
|
||||
slog.Warn("nomos: message queue full; dropping auto-run for operator message", "session", sessionID, "cap", maxQueuedPerSession)
|
||||
return false
|
||||
}
|
||||
q.queue[sessionID] = append(q.queue[sessionID], msg)
|
||||
return true
|
||||
}
|
||||
|
||||
// dequeue pops the next message from the front of the session's FIFO. Returns
|
||||
// ok=false when empty.
|
||||
func (q *messageQueue) dequeue(sessionID string) (string, bool) {
|
||||
q.mu.Lock()
|
||||
defer q.mu.Unlock()
|
||||
xs := q.queue[sessionID]
|
||||
if len(xs) == 0 {
|
||||
return "", false
|
||||
}
|
||||
m := xs[0]
|
||||
q.queue[sessionID] = xs[1:]
|
||||
return m, true
|
||||
}
|
||||
|
||||
// requeueFront pushes a message back to the front — used when a drainer popped
|
||||
// a message but lost the race for the gate to a live turn; that turn's own
|
||||
// release will drain it again.
|
||||
func (q *messageQueue) requeueFront(sessionID, msg string) {
|
||||
q.mu.Lock()
|
||||
defer q.mu.Unlock()
|
||||
q.queue[sessionID] = append([]string{msg}, q.queue[sessionID]...)
|
||||
}
|
||||
|
||||
// peek reports the queued depth for a session (test/diagnostic helper).
|
||||
func (q *messageQueue) peek(sessionID string) int {
|
||||
q.mu.Lock()
|
||||
defer q.mu.Unlock()
|
||||
return len(q.queue[sessionID])
|
||||
}
|
||||
142
cmd/nomos/messagequeue_test.go
Normal file
142
cmd/nomos/messagequeue_test.go
Normal file
@@ -0,0 +1,142 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestMessageQueue_FIFO(t *testing.T) {
|
||||
q := newMessageQueue()
|
||||
q.enqueue("s", "first")
|
||||
q.enqueue("s", "second")
|
||||
q.enqueue("s", "third")
|
||||
|
||||
want := []string{"first", "second", "third"}
|
||||
for _, w := range want {
|
||||
got, ok := q.dequeue("s")
|
||||
if !ok || got != w {
|
||||
t.Fatalf("dequeue = %q,%v want %q,true", got, ok, w)
|
||||
}
|
||||
}
|
||||
if _, ok := q.dequeue("s"); ok {
|
||||
t.Fatal("dequeue on drained queue should return ok=false")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMessageQueue_RequeueFront(t *testing.T) {
|
||||
q := newMessageQueue()
|
||||
q.enqueue("s", "a")
|
||||
q.enqueue("s", "b")
|
||||
// Pop "a", then push it back to the front; "a" must come out before "b".
|
||||
a, _ := q.dequeue("s")
|
||||
q.requeueFront("s", a)
|
||||
got, _ := q.dequeue("s")
|
||||
if got != "a" {
|
||||
t.Fatalf("after requeueFront, dequeue = %q want %q", got, "a")
|
||||
}
|
||||
got2, _ := q.dequeue("s")
|
||||
if got2 != "b" {
|
||||
t.Fatalf("next dequeue = %q want %q", got2, "b")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMessageQueue_IsolatedPerSession(t *testing.T) {
|
||||
q := newMessageQueue()
|
||||
q.enqueue("s1", "one")
|
||||
q.enqueue("s2", "two")
|
||||
if got, _ := q.dequeue("s1"); got != "one" {
|
||||
t.Fatalf("s1 = %q want one", got)
|
||||
}
|
||||
if got, _ := q.dequeue("s2"); got != "two" {
|
||||
t.Fatalf("s2 = %q want two", got)
|
||||
}
|
||||
if q.peek("s1") != 0 || q.peek("s2") != 0 {
|
||||
t.Fatal("both sessions should be drained")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMessageQueue_Concurrent(t *testing.T) {
|
||||
q := newMessageQueue()
|
||||
const n = maxQueuedPerSession // stay under the cap so every enqueue lands
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < n; i++ {
|
||||
wg.Add(1)
|
||||
go func(i int) {
|
||||
defer wg.Done()
|
||||
q.enqueue("s", "m")
|
||||
}(i)
|
||||
}
|
||||
wg.Wait()
|
||||
if q.peek("s") != n {
|
||||
t.Fatalf("peek = %d want %d (all enqueues must be counted)", q.peek("s"), n)
|
||||
}
|
||||
seen := 0
|
||||
for {
|
||||
if _, ok := q.dequeue("s"); !ok {
|
||||
break
|
||||
}
|
||||
seen++
|
||||
}
|
||||
if seen != n {
|
||||
t.Fatalf("drained %d want %d", seen, n)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMessageQueue_CapsOverflow(t *testing.T) {
|
||||
q := newMessageQueue()
|
||||
for i := 0; i < maxQueuedPerSession; i++ {
|
||||
if !q.enqueue("s", "m") {
|
||||
t.Fatalf("enqueue #%d within cap should succeed", i)
|
||||
}
|
||||
}
|
||||
if q.enqueue("s", "overflow") {
|
||||
t.Fatal("enqueue past the cap should return false (dropped)")
|
||||
}
|
||||
if got := q.peek("s"); got != maxQueuedPerSession {
|
||||
t.Fatalf("peek = %d want %d (overflow must not append)", got, maxQueuedPerSession)
|
||||
}
|
||||
}
|
||||
|
||||
// drainQueued on an empty queue must be a no-op: it returns immediately and
|
||||
// never touches the gate (so the session stays free for the next turn).
|
||||
func TestDrainQueued_NoOpOnEmpty(t *testing.T) {
|
||||
a := &agent{gate: newTurnGate(), queue: newMessageQueue()}
|
||||
a.drainQueued(context.Background(), "s")
|
||||
if !a.gate.acquire("s", 0) {
|
||||
t.Fatal("gate should be free after a no-op drain (drain must not hold it)")
|
||||
}
|
||||
a.gate.release("s")
|
||||
}
|
||||
|
||||
// With a queued message but the gate held by another turn, drainQueued must
|
||||
// re-queue the message and return WITHOUT running a turn (no store/provider → a
|
||||
// real run would panic). This is the "never stack" property: a busy gate
|
||||
// defers to the holder's own release-drain.
|
||||
func TestDrainQueued_RequeuesWhenBusy(t *testing.T) {
|
||||
prev := drainAcquireWait
|
||||
drainAcquireWait = 10 * time.Millisecond
|
||||
t.Cleanup(func() { drainAcquireWait = prev })
|
||||
|
||||
a := &agent{gate: newTurnGate(), queue: newMessageQueue()}
|
||||
if !a.gate.acquire("s", 0) {
|
||||
t.Fatal("precondition: hold the gate")
|
||||
}
|
||||
a.queue.enqueue("s", "queued-msg")
|
||||
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
a.drainQueued(context.Background(), "s") // must not panic; must requeue
|
||||
close(done)
|
||||
}()
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("drainQueued did not return promptly while the gate was busy")
|
||||
}
|
||||
if got := a.queue.peek("s"); got != 1 {
|
||||
t.Fatalf("message should be re-queued while busy; peek = %d want 1", got)
|
||||
}
|
||||
a.gate.release("s")
|
||||
}
|
||||
@@ -1,11 +1,8 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
@@ -18,7 +15,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
"github.com/google/uuid"
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
@@ -27,13 +24,14 @@ func main() {
|
||||
fmt.Fprintln(os.Stderr, "usage: nomos serve")
|
||||
os.Exit(1)
|
||||
}
|
||||
if os.Args[1] == "healthcheck" {
|
||||
runHealthcheck()
|
||||
return
|
||||
}
|
||||
mcpURL := os.Getenv("NOMOS_MCP_URL")
|
||||
if mcpURL == "" {
|
||||
mcpURL = "http://localhost:8090/mcp"
|
||||
}
|
||||
// api's combinedAuth requires a bearer token on every request (no
|
||||
// dev-open bypass — plans/2026-07-12-wails-desktop-app.md 0.4); this is
|
||||
// the same shared secret api validates against (OIKOS_MCP_BEARER_TOKEN).
|
||||
mcpToken := os.Getenv("OIKOS_MCP_BEARER_TOKEN")
|
||||
|
||||
agentSlug := os.Getenv("NOMOS_AGENT_SLUG")
|
||||
@@ -46,6 +44,32 @@ func main() {
|
||||
databaseURL = os.Getenv("OIKOS_DATABASE_URL")
|
||||
}
|
||||
|
||||
sec := secrets.NewManagerFromConfig(
|
||||
os.Getenv("OIKOS_INFISICAL_SITE_URL"),
|
||||
os.Getenv("OIKOS_INFISICAL_CLIENT_ID"),
|
||||
os.Getenv("OIKOS_INFISICAL_CLIENT_SECRET"),
|
||||
os.Getenv("OIKOS_INFISICAL_PROJECT_ID"),
|
||||
os.Getenv("OIKOS_INFISICAL_ENV"),
|
||||
os.Getenv("OIKOS_SECRETS_DIR"),
|
||||
)
|
||||
var openrouterAPIKey string
|
||||
var secretsResolved int
|
||||
if sec != nil {
|
||||
resCtx, resCancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
if v := secrets.ResolveSecret(resCtx, sec, "mcp_bearer-token", ""); v != "" {
|
||||
mcpToken = v
|
||||
secretsResolved++
|
||||
}
|
||||
openrouterAPIKey = secrets.ResolveSecret(resCtx, sec, "openrouter_api-key", os.Getenv("OPENROUTER_API_KEY"))
|
||||
if openrouterAPIKey != "" && openrouterAPIKey != os.Getenv("OPENROUTER_API_KEY") {
|
||||
secretsResolved++
|
||||
}
|
||||
resCancel()
|
||||
if secretsResolved > 0 {
|
||||
slog.Info("nomos: secrets resolved from Infisical", "count", secretsResolved)
|
||||
}
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "serve":
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
|
||||
@@ -75,7 +99,7 @@ func main() {
|
||||
defer st.close()
|
||||
}
|
||||
|
||||
nAgent, err := newAgent(ctx, clientPool, st, agentSlug)
|
||||
nAgent, err := newAgent(ctx, clientPool, st, agentSlug, openrouterAPIKey)
|
||||
if err != nil {
|
||||
slog.Error("nomos: agent init", "error", err)
|
||||
os.Exit(1)
|
||||
@@ -161,6 +185,26 @@ func main() {
|
||||
}
|
||||
}
|
||||
|
||||
func runHealthcheck() {
|
||||
addr := os.Getenv("NOMOS_LISTEN")
|
||||
if addr == "" {
|
||||
addr = ":8092"
|
||||
}
|
||||
host := addr
|
||||
if strings.HasPrefix(host, ":") {
|
||||
host = "127.0.0.1" + host
|
||||
}
|
||||
client := &http.Client{Timeout: 3 * time.Second}
|
||||
resp, err := client.Get("http://" + host + "/healthz")
|
||||
if err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func sseEvent(w http.ResponseWriter, flusher http.Flusher, event agentEvent) {
|
||||
data, _ := json.Marshal(event)
|
||||
fmt.Fprintf(w, "data: %s\n\n", data)
|
||||
@@ -227,6 +271,17 @@ func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
w.Header().Set("X-Accel-Buffering", "no") // disable proxy buffering
|
||||
w.WriteHeader(200)
|
||||
|
||||
// All writes to w (events + the keepalive comment below) go through one
|
||||
// mutex: http.ResponseWriter is NOT safe for concurrent use, and the
|
||||
// keepalive ticker runs alongside the turn's event sink (plan 2026-08-03
|
||||
// F3). Without this, interleaved writes corrupt the SSE stream.
|
||||
var writeMu sync.Mutex
|
||||
writeEvent := func(ev agentEvent) {
|
||||
writeMu.Lock()
|
||||
defer writeMu.Unlock()
|
||||
sseEvent(w, flusher, ev)
|
||||
}
|
||||
|
||||
ctx := r.Context()
|
||||
sessionID := req.SessionID
|
||||
|
||||
@@ -278,134 +333,65 @@ func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
|
||||
st.answerQuestion(pctx, sessionID, qid, req.Message)
|
||||
}
|
||||
|
||||
sseEvent(w, flusher, agentEvent{Type: "session", Data: sessionID, SessionID: sessionID})
|
||||
writeEvent(agentEvent{Type: "session", Data: sessionID, SessionID: sessionID})
|
||||
|
||||
// F1 (plan 2026-08-03): serialize turns per session. The user message is
|
||||
// already persisted above, so even if we can't run this turn right now it
|
||||
// isn't lost. Wait briefly for a finishing background turn (continuation /
|
||||
// resume) so the common case is seamless; if one is still running after
|
||||
// that, tell the operator to retry rather than spawning a second
|
||||
// concurrent turn (the interleaving this gate exists to prevent). On
|
||||
// success the permit is held until this handler returns (stream + post-
|
||||
// processing done); background resumeSession callers skip while it's held.
|
||||
// F1/F2 (plan 2026-08-03): serialize turns per session. The user message is
|
||||
// already persisted above, so it is never lost. Wait briefly for a finishing
|
||||
// background turn; if one is still running after that, QUEUE this message
|
||||
// (don't reject it) and tell the client so it shows a "queued" state. The
|
||||
// in-flight turn's release drains the queue (drainQueued) and runs it as a
|
||||
// real turn server-side. This never stacks concurrent turns — the gate still
|
||||
// guarantees one in-flight turn per session.
|
||||
const turnWait = 5 * time.Second
|
||||
if !a.gate.acquire(sessionID, turnWait) {
|
||||
slog.Info("nomos: turn already active, deferring operator message", "session", sessionID)
|
||||
sseEvent(w, flusher, agentEvent{
|
||||
Type: "error",
|
||||
Data: "Nomos is still finishing a previous step. Your message was saved — give it a moment to finish, then send it again.",
|
||||
})
|
||||
sseEvent(w, flusher, agentEvent{Type: "done", Data: map[string]any{
|
||||
a.queue.enqueue(sessionID, req.Message)
|
||||
slog.Info("nomos: turn already active, queued operator message", "session", sessionID)
|
||||
writeEvent(agentEvent{Type: "queued", Data: sessionID, SessionID: sessionID})
|
||||
writeEvent(agentEvent{Type: "done", Data: map[string]any{
|
||||
"session_id": sessionID,
|
||||
"error": true,
|
||||
"queued": true,
|
||||
}, SessionID: sessionID})
|
||||
return
|
||||
}
|
||||
defer a.gate.release(sessionID)
|
||||
defer func() {
|
||||
a.gate.release(sessionID)
|
||||
// Run any message that was queued while this turn held the gate. In a
|
||||
// goroutine so the HTTP response finishes without waiting on the next
|
||||
// turn; the queued turn has no SSE client of its own.
|
||||
safego.Go("nomos:drain:"+sessionID, func() { a.drainQueued(context.Background(), sessionID) })
|
||||
}()
|
||||
|
||||
toolCalls := []map[string]any{}
|
||||
// P3: accumulate per-iteration reasoning instead of overwriting with
|
||||
// the final `text` event. The agent loop emits a `text` event for each
|
||||
// LLM iteration that produced text (intermediate reasoning before tool
|
||||
// calls + the final answer). Without accumulation, only the last `text`
|
||||
// survives in the persisted row — a reload shows the final summary but
|
||||
// not the thinking that led to each tool call.
|
||||
var textParts []string
|
||||
var finalText string
|
||||
|
||||
// Incremental persistence, mirroring resumeSession's existing
|
||||
// placeholder+update pattern (continue.go): insert a placeholder now,
|
||||
// update the SAME row after every tool call, so whatever happened before
|
||||
// an abort is never lost — only what hadn't happened yet is.
|
||||
placeholder, _ := json.Marshal(map[string]any{"role": "assistant", "text": ""})
|
||||
msgID, err := st.insertMessageReturningID(pctx, sessionID, "assistant", placeholder)
|
||||
if err != nil {
|
||||
slog.Error("nomos: chat placeholder insert failed", "session", sessionID, "error", err)
|
||||
}
|
||||
persist := func() {
|
||||
if msgID == uuid.Nil {
|
||||
return
|
||||
}
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": finalText,
|
||||
"tool_calls": toolCalls,
|
||||
})
|
||||
st.updateMessage(pctx, msgID, body)
|
||||
}
|
||||
|
||||
a.chat(ctx, sessionID, req.Message, func(ev agentEvent) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
m["type"] = ev.Type
|
||||
// One entry per tool call: tool_use creates it, tool_result
|
||||
// merges the result into the same entry (matched by id).
|
||||
// Before this fix, both events appended separate entries,
|
||||
// doubling every tool call in the persisted transcript
|
||||
// (confirmed pre-existing in d9cdcee1, v0.3.x era).
|
||||
id, _ := m["id"].(string)
|
||||
if id != "" && ev.Type == "tool_result" {
|
||||
for _, existing := range toolCalls {
|
||||
if eID, _ := existing["id"].(string); eID == id {
|
||||
for k, v := range m {
|
||||
existing[k] = v
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
} else {
|
||||
toolCalls = append(toolCalls, m)
|
||||
}
|
||||
}
|
||||
persist() // live: survives even if the client disconnects right after
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
// P3: accumulate. Each `text` event is one iteration's reasoning
|
||||
// (or the final answer). Join with newlines so the persisted row
|
||||
// reads as the full transcript of what the agent said, not just
|
||||
// the last thing.
|
||||
if t, ok := ev.Data.(string); ok && t != "" {
|
||||
textParts = append(textParts, t)
|
||||
finalText = strings.Join(textParts, "\n\n")
|
||||
persist()
|
||||
// F3 (plan 2026-08-03): keep the SSE alive during long turns. A turn can
|
||||
// run for many minutes (provisioning chains, deep research); the model
|
||||
// often takes 20-40s between tool iterations, and with nothing flushed in
|
||||
// that gap a proxy/browser idle timeout silently closes the stream. The
|
||||
// client then sees streaming=false while the server keeps working — the
|
||||
// "I can't tell it's working" desync. An SSE comment line (":keepalive") is
|
||||
// ignored by EventSource but resets idle timers.
|
||||
keepDone := make(chan struct{})
|
||||
go func() {
|
||||
t := time.NewTicker(12 * time.Second)
|
||||
defer t.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-keepDone:
|
||||
return
|
||||
case <-t.C:
|
||||
writeMu.Lock()
|
||||
fmt.Fprintf(w, ":keepalive\n\n")
|
||||
flusher.Flush()
|
||||
writeMu.Unlock()
|
||||
}
|
||||
}
|
||||
sseEvent(w, flusher, ev)
|
||||
}()
|
||||
// Defer the close (not a statement after runChatTurn) so the goroutine
|
||||
// exits even if runChatTurn panics — net/http recovers handler panics, so
|
||||
// a non-deferred close would be skipped and the ticker would keep writing
|
||||
// to a dead ResponseWriter forever.
|
||||
defer close(keepDone)
|
||||
a.runChatTurn(pctx, ctx, sessionID, req.Message, func(ev agentEvent) {
|
||||
writeEvent(ev)
|
||||
})
|
||||
|
||||
// B.6: if the turn ended with no text and no tool calls (the model
|
||||
// empty-response'd and all retries failed), delete the placeholder row
|
||||
// instead of persisting an empty bubble. The error event was already
|
||||
// streamed to the frontend via the 'done with error=true' event, so the
|
||||
// operator sees the error inline — an empty assistant bubble in the
|
||||
// transcript adds nothing and looks like the agent is broken.
|
||||
if finalText == "" && len(toolCalls) == 0 && msgID != uuid.Nil {
|
||||
st.deleteMessage(pctx, msgID)
|
||||
} else {
|
||||
persist() // final state — same row, updated one last time with the concluding text
|
||||
}
|
||||
|
||||
// Generate a meaningful title from the assistant's first answer
|
||||
// instead of reusing the raw user message for every session.
|
||||
// P2.9 (2026-07-20): prefer the goal as the title when one is set —
|
||||
// the first assistant text is often a greeting or narrative that
|
||||
// doesn't describe the task ("Hey! 👋 Nomos here, running on
|
||||
// mac-mini:8092..."). The goal is the operator's actual intent.
|
||||
// Sessions that never call set_goal (pure Q&A) fall back to the
|
||||
// assistant text, which is still better than the raw user message.
|
||||
if finalText != "" && sessionID != "ephemeral" {
|
||||
var goalTitle string
|
||||
if sess, gerr := st.getSession(pctx, sessionID); gerr == nil && sess.Goal != "" {
|
||||
goalTitle = truncate(sess.Goal, 120)
|
||||
}
|
||||
title := goalTitle
|
||||
if title == "" {
|
||||
title = truncate(finalText, 80)
|
||||
}
|
||||
if title != "" {
|
||||
st.updateSessionTitle(pctx, sessionID, title)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func handleSessionsList(w http.ResponseWriter, r *http.Request, st *store) {
|
||||
@@ -706,339 +692,4 @@ func truncate(s string, n int) string {
|
||||
return s
|
||||
}
|
||||
return s[:n] + "..."
|
||||
}
|
||||
|
||||
// ─── MCP Streamable HTTP client ────────────────────────────────────────
|
||||
|
||||
type mcpClient struct {
|
||||
baseURL string
|
||||
token string // OIKOS_MCP_BEARER_TOKEN — api's combinedAuth requires it on every request (no dev-open bypass)
|
||||
sessionID string
|
||||
http *http.Client
|
||||
nextID int
|
||||
mu sync.Mutex // this client is one stateful MCP session; serialize ITS OWN calls
|
||||
|
||||
// toolsCache holds the last tools/list result. The tool list is static
|
||||
// for the lifetime of one MCP connection — it only changes when the api
|
||||
// process (re)registers tools, i.e. on a restart, which this client
|
||||
// already detects and reacts to via reconnectLocked. Without this,
|
||||
// buildTools (called at the start of EVERY chat turn, including every
|
||||
// auto-continuation resume) paid a full tools/list round-trip every
|
||||
// single time for a list that's almost always identical to the last one.
|
||||
// Guarded separately from mu (not reused) so a cache check never
|
||||
// contends with an in-flight doRequest call for a different method.
|
||||
toolsMu sync.Mutex
|
||||
toolsCache []toolDef
|
||||
}
|
||||
|
||||
func newMCPClient(baseURL, token string) (*mcpClient, error) {
|
||||
c := &mcpClient{
|
||||
baseURL: baseURL,
|
||||
token: token,
|
||||
http: &http.Client{Timeout: 30 * time.Second},
|
||||
}
|
||||
|
||||
resp, err := c.doRequest("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initialize: %w", err)
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return nil, fmt.Errorf("no session ID in initialize response")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
|
||||
c.doRequest("notifications/initialized", map[string]any{})
|
||||
|
||||
slog.Info("nomos: mcp connected", "session", c.sessionID[:16]+"...")
|
||||
return c, nil
|
||||
}
|
||||
|
||||
type mcpJSONRPCResponse struct {
|
||||
sessionID string
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error json.RawMessage `json:"error"`
|
||||
}
|
||||
|
||||
// errStaleSession signals that the MCP server rejected our session id (e.g.
|
||||
// after an api/MCP restart), so the client should re-initialize and retry.
|
||||
var errStaleSession = fmt.Errorf("mcp session stale")
|
||||
|
||||
// doRequest serializes MCP calls and transparently re-initializes the session
|
||||
// if the server has forgotten it (common after an api redeploy), retrying the
|
||||
// original call once. Without this, an api restart permanently breaks nomos
|
||||
// until it is itself restarted.
|
||||
func (c *mcpClient) doRequest(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
resp, err := c.send(method, params)
|
||||
if err != nil && method != "initialize" && errors.Is(err, errStaleSession) {
|
||||
slog.Warn("nomos: mcp session stale, reconnecting")
|
||||
if rerr := c.reconnectLocked(); rerr != nil {
|
||||
return nil, fmt.Errorf("mcp reconnect: %w (original: %v)", rerr, err)
|
||||
}
|
||||
return c.send(method, params)
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
// reconnectLocked re-initializes the MCP session. The caller must hold c.mu.
|
||||
func (c *mcpClient) reconnectLocked() error {
|
||||
c.sessionID = ""
|
||||
// A reconnect means the api process was restarted (or forgot us) — its
|
||||
// tool registration may have changed, so the cached list is no longer
|
||||
// trustworthy.
|
||||
c.toolsMu.Lock()
|
||||
c.toolsCache = nil
|
||||
c.toolsMu.Unlock()
|
||||
resp, err := c.send("initialize", map[string]any{
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": map[string]any{},
|
||||
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.sessionID == "" {
|
||||
return fmt.Errorf("no session ID on re-initialize")
|
||||
}
|
||||
c.sessionID = resp.sessionID
|
||||
_, _ = c.send("notifications/initialized", map[string]any{})
|
||||
slog.Info("nomos: mcp reconnected", "session", c.sessionID[:16]+"...")
|
||||
return nil
|
||||
}
|
||||
|
||||
// send performs one MCP round-trip. It does not lock; callers hold c.mu.
|
||||
func (c *mcpClient) send(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
|
||||
c.nextID++
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"jsonrpc": "2.0",
|
||||
"method": method,
|
||||
"params": params,
|
||||
"id": c.nextID,
|
||||
})
|
||||
|
||||
req, err := http.NewRequest(http.MethodPost, c.baseURL, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json, text/event-stream")
|
||||
if c.sessionID != "" {
|
||||
req.Header.Set("Mcp-Session-Id", c.sessionID)
|
||||
}
|
||||
if c.token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
}
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// A rejected/unknown session comes back as 4xx (commonly 400/404).
|
||||
if resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusBadRequest {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
result := &mcpJSONRPCResponse{}
|
||||
result.sessionID = resp.Header.Get("Mcp-Session-Id")
|
||||
|
||||
scanner := bufio.NewScanner(resp.Body)
|
||||
scanner.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
|
||||
gotData := false
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if strings.HasPrefix(line, "data: ") {
|
||||
gotData = true
|
||||
data := line[6:]
|
||||
if err := json.Unmarshal([]byte(data), result); err != nil {
|
||||
return nil, fmt.Errorf("parse response: %w", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if result.Error != nil {
|
||||
return nil, fmt.Errorf("rpc error: %s", string(result.Error))
|
||||
}
|
||||
|
||||
// Empty body with no result and a session set: the server likely dropped
|
||||
// our session. Notifications legitimately return no data, so exempt them.
|
||||
if !gotData && result.Result == nil && method != "notifications/initialized" {
|
||||
return nil, errStaleSession
|
||||
}
|
||||
|
||||
if result.sessionID != "" {
|
||||
c.sessionID = result.sessionID
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) callTool(name string, args map[string]any) (any, error) {
|
||||
resp, err := c.doRequest("tools/call", map[string]any{
|
||||
"name": name,
|
||||
"arguments": args,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var toolResult struct {
|
||||
Content []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &toolResult); err != nil {
|
||||
return string(resp.Result), nil
|
||||
}
|
||||
|
||||
var texts []string
|
||||
for _, c := range toolResult.Content {
|
||||
if c.Type == "text" {
|
||||
var parsed any
|
||||
if json.Unmarshal([]byte(c.Text), &parsed) == nil {
|
||||
return parsed, nil
|
||||
}
|
||||
texts = append(texts, c.Text)
|
||||
}
|
||||
}
|
||||
if len(texts) == 1 {
|
||||
return texts[0], nil
|
||||
}
|
||||
return texts, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) listTools() ([]string, error) {
|
||||
resp, err := c.doRequest("tools/list", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tr struct {
|
||||
Tools []struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
} `json:"tools"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Result, &tr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var names []string
|
||||
for _, t := range tr.Tools {
|
||||
names = append(names, t.Name)
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
func (c *mcpClient) close() {
|
||||
}
|
||||
|
||||
// ─── Per-session MCP client pool ────────────────────────────────────────
|
||||
//
|
||||
// A single shared mcpClient serializes EVERY tool call across EVERY
|
||||
// concurrently-running task through one mutex (see mcpClient.mu) — `run`
|
||||
// executes its SSH command synchronously inside that lock and is capped at
|
||||
// up to 10 minutes, so one task mid-`run` stalled every other task's tool
|
||||
// calls, even trivial reads, behind it. The MCP *server* has no per-
|
||||
// connection state to protect (newServer in internal/mcp/server.go returns
|
||||
// one shared *mcp.Server instance whose tool handlers close only over the DB
|
||||
// pool, which is already safe for concurrent use) — the mutex existed purely
|
||||
// because the *client* reused one stateful transport session, not because
|
||||
// the server needed it. Giving each task's own session its own client
|
||||
// removes the cross-task serialization entirely: a task's own tool calls
|
||||
// stay sequential (which they already are — the agent loop calls tools one
|
||||
// at a time within a turn), but no longer block anyone else's.
|
||||
type mcpClientPool struct {
|
||||
baseURL string
|
||||
token string
|
||||
mu sync.Mutex
|
||||
clients map[string]*pooledMCPClient
|
||||
}
|
||||
|
||||
type pooledMCPClient struct {
|
||||
client *mcpClient
|
||||
lastUsed time.Time
|
||||
}
|
||||
|
||||
func newMCPClientPool(baseURL, token string) *mcpClientPool {
|
||||
return &mcpClientPool{baseURL: baseURL, token: token, clients: make(map[string]*pooledMCPClient)}
|
||||
}
|
||||
|
||||
// get returns the client for sessionID, creating and initializing one (a
|
||||
// real MCP handshake) on first use. Session ids that don't identify a real
|
||||
// persisted conversation ("" / "ephemeral", the no-DB-store path; "query",
|
||||
// the structured /query endpoint) still get exactly one dedicated,
|
||||
// reused client each via the same map — just keyed on a fixed string instead
|
||||
// of a real session id — so that traffic doesn't pay a fresh handshake per
|
||||
// request while still never sharing a connection with an actual task.
|
||||
func (p *mcpClientPool) get(sessionID string) (*mcpClient, error) {
|
||||
key := sessionID
|
||||
if key == "" {
|
||||
key = "ephemeral"
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
if pc, ok := p.clients[key]; ok {
|
||||
pc.lastUsed = time.Now()
|
||||
p.mu.Unlock()
|
||||
return pc.client, nil
|
||||
}
|
||||
p.mu.Unlock()
|
||||
|
||||
// Initialize outside the lock — it's a network round-trip, and holding
|
||||
// the pool mutex for it would serialize unrelated sessions' first calls
|
||||
// behind each other, undermining the whole point of this pool.
|
||||
c, err := newMCPClient(p.baseURL, p.token)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
p.mu.Lock()
|
||||
// Another goroutine may have created one for the same key while we were
|
||||
// initializing (two of this session's tool calls racing on a cold
|
||||
// start); keep whichever won, close out the loser's connection (a no-op
|
||||
// today, but future-proof if mcpClient.close ever does real teardown).
|
||||
if existing, ok := p.clients[key]; ok {
|
||||
p.mu.Unlock()
|
||||
c.close()
|
||||
return existing.client, nil
|
||||
}
|
||||
p.clients[key] = &pooledMCPClient{client: c, lastUsed: time.Now()}
|
||||
p.mu.Unlock()
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// mcpClientIdleTimeout is how long an idle session's MCP client is kept
|
||||
// before eviction — long enough to outlive a single slow `run` (capped at 10
|
||||
// minutes server-side) plus normal think-time between a task's tool calls,
|
||||
// short enough not to accumulate one abandoned connection per finished task
|
||||
// forever.
|
||||
const mcpClientIdleTimeout = 20 * time.Minute
|
||||
|
||||
// sweep evicts clients idle past mcpClientIdleTimeout. Call on a ticker.
|
||||
func (p *mcpClientPool) sweep() {
|
||||
cutoff := time.Now().Add(-mcpClientIdleTimeout)
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
if pc.lastUsed.Before(cutoff) {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *mcpClientPool) closeAll() {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for key, pc := range p.clients {
|
||||
pc.client.close()
|
||||
delete(p.clients, key)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -179,6 +179,15 @@ func (s *store) createTaskEntity(ctx context.Context, sessionID, title string) s
|
||||
`UPDATE agent_sessions SET entity_id = $1 WHERE id = $2`, entityID, sessionID); err != nil {
|
||||
slog.Warn("nomos: could not link task entity", "session", sessionID, "error", err)
|
||||
}
|
||||
// Graph edge: task —involves→ agent:nomos (gives every task at least one
|
||||
// edge from creation, even if no run calls are ever made).
|
||||
s.pool.Exec(ctx, `INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT $1, id, 'involves', '{"by":"nomos"}'::jsonb, now()
|
||||
FROM entities WHERE slug = 'agent:nomos'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM relationships r
|
||||
WHERE r.source_id = $1 AND r.target_id = entities.id AND r.type = 'involves' AND r.valid_to IS NULL)`,
|
||||
entityID)
|
||||
return entityID.String()
|
||||
}
|
||||
|
||||
@@ -807,10 +816,11 @@ func (s *store) setGoal(ctx context.Context, sessionID, goal string) error {
|
||||
// Replace any prior plan steps (done/running/pending/...) as `replaced`.
|
||||
// The rows are kept for the generation counter + audit trail; proposePlan
|
||||
// excludes `replaced` from its in-flight check, so the next propose_plan
|
||||
// takes the fresh-generation path.
|
||||
// takes the fresh-generation path. replaced_reason records the cause
|
||||
// (2026-08-04 plan-step integrity audit).
|
||||
s.pool.Exec(ctx,
|
||||
`UPDATE session_plan_steps SET status = 'replaced', finished_at = COALESCE(finished_at, now()) WHERE session_id = $1 AND status <> 'replaced'`,
|
||||
sessionID)
|
||||
`UPDATE session_plan_steps SET status = 'replaced', replaced_reason = $2, finished_at = COALESCE(finished_at, now()) WHERE session_id = $1 AND status <> 'replaced'`,
|
||||
sessionID, "goal superseded")
|
||||
if _, err := s.pool.Exec(ctx,
|
||||
`UPDATE agent_sessions SET goal = $2, status = 'executing', title = $2, last_active_at = now() WHERE id = $1`,
|
||||
sessionID, goal); err != nil {
|
||||
@@ -850,6 +860,14 @@ func (s *store) reopenSession(ctx context.Context, sessionID string) bool {
|
||||
s.pool.Exec(ctx,
|
||||
`UPDATE agent_sessions SET status = 'executing', outcome = NULL, summary = NULL, last_active_at = now() WHERE id = $1`,
|
||||
sessionID)
|
||||
// Mark the prior plan's steps as replaced so the P1 plan-first gate in
|
||||
// classifyAndGate forces a fresh propose_plan before any run. Without
|
||||
// this, the agent could resume a session and call run against the old
|
||||
// (completed) plan — exactly what caused the ZimaOS continuation to
|
||||
// have 81 ad-hoc tool calls with zero plan structure (2026-08-04).
|
||||
s.pool.Exec(ctx,
|
||||
`UPDATE session_plan_steps SET status = 'replaced', replaced_reason = $2, finished_at = COALESCE(finished_at, now()) WHERE session_id = $1 AND status <> 'replaced'`,
|
||||
sessionID, "session reopened — awaiting new plan")
|
||||
_ = observability.Event(ctx, sqlcgen.New(s.pool), "task.reopened", s.taskEntityPtr(ctx, sessionID),
|
||||
"info", "nomos", sessionID, map[string]any{"prior_status": currentStatus})
|
||||
return true
|
||||
@@ -908,9 +926,11 @@ func (s *store) proposePlan(ctx context.Context, sessionID string, steps []planS
|
||||
// The rows are kept for the generation counter (MAX(generation)+1 below)
|
||||
// and the plan_generations eval assertion. `replaced` steps are excluded
|
||||
// from the anyStarted check above, so they don't block this proposal.
|
||||
// replaced_reason records the cause — required by the plan-step integrity
|
||||
// gate (2026-08-04 session audit).
|
||||
if _, err := tx.Exec(ctx,
|
||||
`UPDATE session_plan_steps SET status = 'replaced', finished_at = COALESCE(finished_at, now()) WHERE session_id = $1 AND status = 'pending'`,
|
||||
sessionID); err != nil {
|
||||
`UPDATE session_plan_steps SET status = 'replaced', replaced_reason = $2, finished_at = COALESCE(finished_at, now()) WHERE session_id = $1 AND status = 'pending'`,
|
||||
sessionID, "superseded by new plan generation"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -972,7 +992,7 @@ func (s *store) proposePlan(ctx context.Context, sessionID string, steps []planS
|
||||
// be marked complete while an earlier step is still pending, preventing the
|
||||
// agent from marking step 5 done before step 4 (observed in production: the
|
||||
// agent rushed to close all steps in a final turn, in reverse order).
|
||||
func (s *store) updatePlanStep(ctx context.Context, sessionID string, seq int, status, execID string) error {
|
||||
func (s *store) updatePlanStep(ctx context.Context, sessionID string, seq int, status, execID, replacedReason string) error {
|
||||
if s == nil || sessionID == "" || sessionID == "ephemeral" {
|
||||
return nil
|
||||
}
|
||||
@@ -1025,16 +1045,30 @@ func (s *store) updatePlanStep(ctx context.Context, sessionID string, seq int, s
|
||||
// status <> 'replaced' is defense-in-depth: MAX(generation) can't hold a
|
||||
// replaced row, but if it ever could, this refuses the write instead of
|
||||
// resurrecting it. No matching row → errPlanStepNotFound (stale/out-of-range seq).
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
UPDATE session_plan_steps
|
||||
SET status = $4, execution_id = COALESCE($5, execution_id)`+stamp+`
|
||||
WHERE session_id = $1 AND generation = $2 AND seq = $3 AND status <> 'replaced'
|
||||
RETURNING id, target_slug`, sessionID, curGen, seq, status, execPtr).Scan(&stepID, &targetSlug)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return errPlanStepNotFound
|
||||
if status == "replaced" && replacedReason != "" {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
UPDATE session_plan_steps
|
||||
SET status = $4, execution_id = COALESCE($5, execution_id), replaced_reason = $6`+stamp+`
|
||||
WHERE session_id = $1 AND generation = $2 AND seq = $3 AND status <> 'replaced'
|
||||
RETURNING id, target_slug`, sessionID, curGen, seq, status, execPtr, replacedReason).Scan(&stepID, &targetSlug)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return errPlanStepNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
UPDATE session_plan_steps
|
||||
SET status = $4, execution_id = COALESCE($5, execution_id)`+stamp+`
|
||||
WHERE session_id = $1 AND generation = $2 AND seq = $3 AND status <> 'replaced'
|
||||
RETURNING id, target_slug`, sessionID, curGen, seq, status, execPtr).Scan(&stepID, &targetSlug)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return errPlanStepNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
return err
|
||||
}
|
||||
// Anchor the event to the step's target entity when it has one, else the task.
|
||||
entPtr := s.taskEntityPtr(ctx, sessionID)
|
||||
@@ -1218,9 +1252,141 @@ func (s *store) completeTask(ctx context.Context, sessionID, outcome, summary st
|
||||
_ = observability.Event(ctx, sqlcgen.New(s.pool), "task.status", entPtr, severity, "nomos", sessionID,
|
||||
map[string]any{"status": status, "outcome": outcome, "summary": summary,
|
||||
"cancelled_executions": cancelledCount, "blocker": blocker})
|
||||
// Auto-persist knowledge so the graph learns from this session regardless
|
||||
// of whether the agent remembered to call upsert_knowledge (2026-08-04
|
||||
// session audit: only 2.4% of sessions called upsert_knowledge manually).
|
||||
if outcome == "success" || outcome == "partial" {
|
||||
autoUpsertKnowledge(ctx, s, sessionID, outcome, summary)
|
||||
}
|
||||
// Plan quality metric: compute step completion rate for the session's
|
||||
// current plan generation. Tracked as a task attribute so the trend
|
||||
// can be monitored over time (2026-08-04 session audit: 38% baseline).
|
||||
writePlanCompletionRate(ctx, s, sessionID)
|
||||
// Auto-feedback: create a feedback entry linking the session's outcome
|
||||
// to its last execution, feeding the pattern-extraction pipeline that
|
||||
// has been empty since launch (2026-08-04 session audit: 0 feedback rows).
|
||||
if outcome == "success" || outcome == "partial" {
|
||||
autoFeedback(ctx, s, sessionID, outcome, summary)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// autoUpsertKnowledge creates a knowledge entry for a completed session,
|
||||
// capturing what was done and linking it to the entities involved. Called
|
||||
// automatically from completeTask so every session leaves a trace, even if
|
||||
// the agent forgot to call upsert_knowledge. Only fired for success/partial
|
||||
// outcomes (failures don't have actionable discoveries).
|
||||
func autoUpsertKnowledge(ctx context.Context, s *store, sessionID, outcome, summary string) {
|
||||
var goal string
|
||||
if err := s.pool.QueryRow(ctx,
|
||||
`SELECT COALESCE(goal, '') FROM agent_sessions WHERE id = $1`,
|
||||
sessionID).Scan(&goal); err != nil || goal == "" {
|
||||
return
|
||||
}
|
||||
title := "Session " + sessionID[:8] + ": " + goal
|
||||
if len(title) > 200 {
|
||||
title = title[:200]
|
||||
}
|
||||
content := "## Outcome\n" + outcome + "\n\n## Summary\n" + summary
|
||||
kind := "investigation"
|
||||
slug := "investigation:nomos/" + sessionID
|
||||
tags := []string{"nomos-session", "auto-generated"}
|
||||
|
||||
// Upsert the knowledge entity.
|
||||
docID, _ := uuid.NewV7()
|
||||
if err := s.pool.QueryRow(ctx, `
|
||||
INSERT INTO entities (id, slug, type, name, attributes)
|
||||
VALUES ($1, $2, $3, $4, '{}')
|
||||
ON CONFLICT (slug) DO UPDATE SET name = EXCLUDED.name, updated_at = now()
|
||||
RETURNING id`, docID, slug, kind, title).Scan(&docID); err != nil {
|
||||
slog.Warn("nomos: autoUpsertKnowledge entity insert", "session", sessionID, "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
// Upsert the knowledge content.
|
||||
if _, err := s.pool.Exec(ctx, `
|
||||
INSERT INTO knowledge_entities (entity_id, title, content, source, tags, updated_at)
|
||||
VALUES ($1, $2, $3, 'nomos-agent', $4, now())
|
||||
ON CONFLICT (entity_id) DO UPDATE
|
||||
SET title = EXCLUDED.title, content = EXCLUDED.content,
|
||||
tags = EXCLUDED.tags, updated_at = now()`,
|
||||
docID, title, content, tags); err != nil {
|
||||
slog.Warn("nomos: autoUpsertKnowledge content insert", "session", sessionID, "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
// Link to the task entity.
|
||||
var taskEntID uuid.UUID
|
||||
if s.pool.QueryRow(ctx, `SELECT entity_id FROM agent_sessions WHERE id = $1`,
|
||||
sessionID).Scan(&taskEntID) == nil && taskEntID != uuid.Nil {
|
||||
s.pool.Exec(ctx, `
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT $1, $2, 'involves', '{"by":"nomos","auto":true}'::jsonb, now()
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM relationships
|
||||
WHERE source_id = $1 AND target_id = $2 AND type = 'involves' AND valid_to IS NULL)`,
|
||||
taskEntID, docID)
|
||||
}
|
||||
|
||||
slog.Info("nomos: auto-upserted knowledge for session",
|
||||
"session", sessionID, "outcome", outcome, "slug", slug)
|
||||
}
|
||||
|
||||
// writePlanCompletionRate computes the step completion rate for the current
|
||||
// plan generation and writes it as a task entity attribute so the trend can
|
||||
// be tracked. Baseline from 2026-08-04 audit: 38% (15/39 steps reached done).
|
||||
func writePlanCompletionRate(ctx context.Context, s *store, sessionID string) {
|
||||
var total, completed int
|
||||
s.pool.QueryRow(ctx, `
|
||||
SELECT COUNT(*), COALESCE(SUM(CASE WHEN status = 'done' THEN 1 ELSE 0 END), 0)
|
||||
FROM session_plan_steps
|
||||
WHERE session_id = $1
|
||||
AND generation = (SELECT MAX(generation) FROM session_plan_steps WHERE session_id = $1)
|
||||
AND status <> 'replaced'`, sessionID).Scan(&total, &completed)
|
||||
if total > 0 {
|
||||
rate := float64(completed) / float64(total)
|
||||
attrs, _ := json.Marshal(map[string]any{"plan_completion_rate": rate, "plan_steps_total": total, "plan_steps_completed": completed})
|
||||
s.pool.Exec(ctx, `
|
||||
UPDATE entities SET attributes = attributes || $2::jsonb, updated_at = now()
|
||||
WHERE id = (SELECT entity_id FROM agent_sessions WHERE id = $1)`,
|
||||
sessionID, string(attrs))
|
||||
slog.Info("nomos: plan completion rate", "session", sessionID, "rate", fmt.Sprintf("%.0f%%", rate*100),
|
||||
"completed", completed, "total", total)
|
||||
}
|
||||
}
|
||||
|
||||
// autoFeedback creates a feedback entry linking the session's outcome to its
|
||||
// last execution, feeding the pattern-extraction pipeline that has been empty
|
||||
// since launch. Only created for success/partial outcomes (failures don't
|
||||
// have a specific execution to tie to).
|
||||
func autoFeedback(ctx context.Context, s *store, sessionID, outcome, summary string) {
|
||||
// Find the last execution linked to this session.
|
||||
var execID uuid.UUID
|
||||
if err := s.pool.QueryRow(ctx, `
|
||||
SELECT pe.execution_id FROM nomos_plan_executions pe
|
||||
WHERE pe.session_id = $1::uuid
|
||||
ORDER BY pe.created_at DESC LIMIT 1`, sessionID).Scan(&execID); err != nil || execID == uuid.Nil {
|
||||
return
|
||||
}
|
||||
fbID, _ := uuid.NewV7()
|
||||
slug := "feedback:" + fbID.String()
|
||||
if _, err := s.pool.Exec(ctx, `
|
||||
INSERT INTO entities (id, slug, type, name, attributes) VALUES ($1, $2, 'feedback', $3, '{}')`,
|
||||
fbID, slug, "feedback for "+sessionID[:8]); err != nil {
|
||||
slog.Warn("nomos: autoFeedback entity insert", "session", sessionID, "error", err)
|
||||
return
|
||||
}
|
||||
_, err := s.pool.Exec(ctx, `
|
||||
INSERT INTO feedback (entity_id, execution_id, outcome, observation, lesson, tags, created_at)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, now())`,
|
||||
fbID, execID, outcome, summary, summary, []string{"nomos-session", "auto-generated", "session:" + sessionID[:8]})
|
||||
if err != nil {
|
||||
slog.Warn("nomos: autoFeedback insert", "session", sessionID, "error", err)
|
||||
return
|
||||
}
|
||||
slog.Info("nomos: auto-feedback created for session", "session", sessionID, "outcome", outcome)
|
||||
}
|
||||
|
||||
// blockerPatterns maps a substring (case-insensitive) to a structured blocker
|
||||
// reason. Order matters — earlier patterns take precedence. These are the
|
||||
// recurring failure signatures from the 2026-07-20 session audit. A
|
||||
@@ -1310,6 +1476,53 @@ func (s *store) hadDiscovery(ctx context.Context, sessionID string) bool {
|
||||
return count > 0
|
||||
}
|
||||
|
||||
// sessionGoal returns the session's goal text, empty string if not found.
|
||||
// Used by complete_task to check whether the goal involved a reachability
|
||||
// verification before marking success.
|
||||
func (s *store) sessionGoal(ctx context.Context, sessionID string) string {
|
||||
if s == nil || sessionID == "" {
|
||||
return ""
|
||||
}
|
||||
var goal string
|
||||
s.pool.QueryRow(ctx,
|
||||
`SELECT COALESCE(goal, '') FROM agent_sessions WHERE id = $1`,
|
||||
sessionID).Scan(&goal)
|
||||
return goal
|
||||
}
|
||||
|
||||
// hadRecentVerification checks whether the session successfully verified
|
||||
// reachability in recent turns — ping_service, or a run with curl/wget that
|
||||
// returned successfully. Used by complete_task as a soft warning when the
|
||||
// goal involved a reachability check but no recent verification occurred.
|
||||
func (s *store) hadRecentVerification(ctx context.Context, sessionID string) bool {
|
||||
if s == nil || sessionID == "" {
|
||||
return true // fail safe: don't warn when we can't check
|
||||
}
|
||||
// Check for ping_service calls in the last 5 activity entries for this session.
|
||||
var pingCount int
|
||||
s.pool.QueryRow(ctx, `
|
||||
SELECT COUNT(*) FROM (
|
||||
SELECT 1 FROM agent_activity
|
||||
WHERE session_id = $1 AND tool_name = 'ping_service' AND success = true
|
||||
ORDER BY ts DESC LIMIT 5
|
||||
) sub`, sessionID).Scan(&pingCount)
|
||||
if pingCount > 0 {
|
||||
return true
|
||||
}
|
||||
// Check for run calls with curl/wget that returned successfully.
|
||||
var curlCount int
|
||||
s.pool.QueryRow(ctx, `
|
||||
SELECT COUNT(*) FROM (
|
||||
SELECT 1 FROM agent_activity
|
||||
WHERE session_id = $1
|
||||
AND tool_name = 'run'
|
||||
AND success = true
|
||||
AND (input_summary LIKE '%curl%' OR input_summary LIKE '%wget%')
|
||||
ORDER BY ts DESC LIMIT 10
|
||||
) sub`, sessionID).Scan(&curlCount)
|
||||
return curlCount > 0
|
||||
}
|
||||
|
||||
// staleGoalSession is a goal-bearing task that's gone idle without reaching
|
||||
// a terminal state — the idle-sweep worker's work list (fix 2+3 of
|
||||
// plans/2026-07-11-task-completion-safety-net.md).
|
||||
@@ -1913,7 +2126,7 @@ func (s *store) resolveArgEntityID(ctx context.Context, args map[string]any) uui
|
||||
// The (nullable) session_id column carries the conversation id. args is the
|
||||
// tool call's own arguments, used to best-effort tag the row with the
|
||||
// entity it acted on (see resolveArgEntityID).
|
||||
func (s *store) logActivity(ctx context.Context, agentID uuid.UUID, sessionID, toolName string, args map[string]any, inputSummary, outputSummary string, durationMs int, success bool, correlationID string) {
|
||||
func (s *store) logActivity(ctx context.Context, agentID uuid.UUID, sessionID, toolName string, args map[string]any, inputSummary, outputSummary string, durationMs int, success bool, correlationID string, tokenCount int) {
|
||||
if s == nil || agentID == uuid.Nil {
|
||||
return
|
||||
}
|
||||
@@ -1925,8 +2138,8 @@ func (s *store) logActivity(ctx context.Context, agentID uuid.UUID, sessionID, t
|
||||
s.pool.Exec(ctx, `
|
||||
INSERT INTO agent_activity
|
||||
(agent_id, session_id, activity_type, tool_name, entity_id, input_summary, output_summary,
|
||||
duration_ms, success, correlation_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)`,
|
||||
duration_ms, success, correlation_id, token_count)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)`,
|
||||
agentID, sessionID, "tool_call", toolName, entityIDArg, inputSummary, outputSummary,
|
||||
durationMs, success, correlationID)
|
||||
durationMs, success, correlationID, tokenCount)
|
||||
}
|
||||
|
||||
@@ -192,7 +192,7 @@ func TestProposePlan_RefuseInFlight(t *testing.T) {
|
||||
}
|
||||
|
||||
// Mark step 1 as started.
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "running", ""); err != nil {
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "running", "", ""); err != nil {
|
||||
t.Fatalf("updatePlanStep: %v", err)
|
||||
}
|
||||
|
||||
@@ -288,10 +288,10 @@ func TestUpdatePlanStep_GenerationRelative(t *testing.T) {
|
||||
|
||||
// The model addresses the new plan with 1-based seq. seq=1 must hit
|
||||
// gen-2 "C", leaving gen-1 "A" (replaced) untouched.
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "running", ""); err != nil {
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "running", "", ""); err != nil {
|
||||
t.Fatalf("updatePlanStep(seq=1, running): %v", err)
|
||||
}
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "done", ""); err != nil {
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "done", "", ""); err != nil {
|
||||
t.Fatalf("updatePlanStep(seq=1, done): %v", err)
|
||||
}
|
||||
|
||||
@@ -319,7 +319,7 @@ func TestUpdatePlanStep_GenerationRelative(t *testing.T) {
|
||||
}
|
||||
|
||||
// Out-of-range seq must be refused (no current-gen step there).
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 99, "running", ""); !errors.Is(err, errPlanStepNotFound) {
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 99, "running", "", ""); !errors.Is(err, errPlanStepNotFound) {
|
||||
t.Fatalf("updatePlanStep(seq=99) err = %v, want errPlanStepNotFound", err)
|
||||
}
|
||||
}
|
||||
@@ -341,7 +341,7 @@ func TestCompleteTask_AutoCloseEmitsEvents(t *testing.T) {
|
||||
t.Fatalf("proposePlan: %v", err)
|
||||
}
|
||||
// A is running, B still pending at completion time.
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "running", ""); err != nil {
|
||||
if err := s.updatePlanStep(ctx, sess.ID, 1, "running", "", ""); err != nil {
|
||||
t.Fatalf("updatePlanStep(1, running): %v", err)
|
||||
}
|
||||
if err := s.completeTask(ctx, sess.ID, "success", "done"); err != nil {
|
||||
@@ -397,7 +397,7 @@ func TestHadDiscoveryAndWriteback(t *testing.T) {
|
||||
|
||||
// A `run` call (discovery) — should set hadDiscovery, not hadEntityWriteback.
|
||||
agentID := uuid.New()
|
||||
s.logActivity(ctx, agentID, sess.ID, "run", nil, "", "uptime output", 100, true, "corr-1")
|
||||
s.logActivity(ctx, agentID, sess.ID, "run", nil, "", "uptime output", 100, true, "corr-1", 0)
|
||||
if !s.hadDiscovery(ctx, sess.ID) {
|
||||
t.Fatal("hadDiscovery = false after a successful run call, want true")
|
||||
}
|
||||
@@ -410,7 +410,7 @@ func TestHadDiscoveryAndWriteback(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
s.logActivity(ctx, agentID, sess2.ID, "run", nil, "", "ssh timeout", 100, false, "corr-2")
|
||||
s.logActivity(ctx, agentID, sess2.ID, "run", nil, "", "ssh timeout", 100, false, "corr-2", 0)
|
||||
if s.hadDiscovery(ctx, sess2.ID) {
|
||||
t.Fatal("hadDiscovery = true after a failed run call, want false (no facts learned)")
|
||||
}
|
||||
@@ -420,7 +420,7 @@ func TestHadDiscoveryAndWriteback(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
s.logActivity(ctx, agentID, sess3.ID, "get_entity", nil, "", "entity row", 10, true, "corr-3")
|
||||
s.logActivity(ctx, agentID, sess3.ID, "get_entity", nil, "", "entity row", 10, true, "corr-3", 0)
|
||||
if s.hadDiscovery(ctx, sess3.ID) {
|
||||
t.Fatal("hadDiscovery = true after get_entity, want false (DB lookups are not discovery)")
|
||||
}
|
||||
@@ -430,12 +430,12 @@ func TestHadDiscoveryAndWriteback(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("createSession: %v", err)
|
||||
}
|
||||
s.logActivity(ctx, agentID, sess4.ID, "update_entity_attributes", nil, "", "ok", 10, true, "corr-4")
|
||||
s.logActivity(ctx, agentID, sess4.ID, "update_entity_attributes", nil, "", "ok", 10, true, "corr-4", 0)
|
||||
if !s.hadEntityWriteback(ctx, sess4.ID) {
|
||||
t.Fatal("hadEntityWriteback = false after update_entity_attributes, want true")
|
||||
}
|
||||
// And the discovery+writeback combination (the conv3 scenario).
|
||||
s.logActivity(ctx, agentID, sess4.ID, "run", nil, "", "apt-get update output", 100, true, "corr-5")
|
||||
s.logActivity(ctx, agentID, sess4.ID, "run", nil, "", "apt-get update output", 100, true, "corr-5", 0)
|
||||
if !s.hadDiscovery(ctx, sess4.ID) {
|
||||
t.Fatal("hadDiscovery = false after run+writeback, want true")
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
@@ -306,7 +307,8 @@ func (a *agent) handleTaskTool(ctx context.Context, sessionID, name string, args
|
||||
if seq <= 0 || status == "" {
|
||||
return "error: update_plan_step needs seq (>=1) and status", true
|
||||
}
|
||||
if err := a.store.updatePlanStep(ctx, sessionID, seq, status, execID); err != nil {
|
||||
reason, _ := args["replaced_reason"].(string)
|
||||
if err := a.store.updatePlanStep(ctx, sessionID, seq, status, execID, reason); err != nil {
|
||||
if errors.Is(err, errPlanStepNotFound) {
|
||||
// The seq doesn't address a step in the CURRENT plan — most
|
||||
// often a stale 1-based number the model carried across a
|
||||
@@ -373,6 +375,18 @@ func (a *agent) handleTaskTool(ctx context.Context, sessionID, name string, args
|
||||
if outcome == "success" && a.store.hadDiscovery(ctx, sessionID) && !a.store.hadEntityWriteback(ctx, sessionID) {
|
||||
return "Refused: this session ran `run` against live targets (discovery) but did not call update_entity_attributes or create_relationship to persist what you learned. The knowledge graph will drift if you complete without writeback. Call update_entity_attributes for each entity you ran against (versions, states, counts, timestamps), and create_relationship for any edge you discovered, then call complete_task again. Outcome is held at 'executing' until you do.", true
|
||||
}
|
||||
// D.2: refuse success when the goal mentions a reachability/uptime
|
||||
// check but no verification was done. The agent can't claim "X is
|
||||
// reachable" based on a shell command alone — the proxy (Caddy) can
|
||||
// return 200 for a terminal page (ttyd) or fallback while the actual
|
||||
// dashboard is still down. Must call ping_service or run a successful
|
||||
// curl before claiming success.
|
||||
if outcome == "success" && a.store.hadDiscovery(ctx, sessionID) {
|
||||
goal := a.store.sessionGoal(ctx, sessionID)
|
||||
if mentionsReachability(goal) && !a.store.hadRecentVerification(ctx, sessionID) {
|
||||
return "Refused: the goal involves a reachability or uptime check (\"make X reachable\", \"get X up\", etc.), but no ping_service call or successful curl/HTTP request against the target was detected. Caddy can return 200 for a terminal or fallback page while the actual service is still down — you must verify the service itself, not just the proxy. Call ping_service(target) or run a curl against the actual service URL, then call complete_task again. Outcome held until verified.", true
|
||||
}
|
||||
}
|
||||
if err := a.store.completeTask(ctx, sessionID, outcome, summary); err != nil {
|
||||
if errors.Is(err, errTaskAlreadyComplete) {
|
||||
return "Task is already complete. Do not call complete_task again. If the operator pointed out a UI/sidebar inconsistency, fix it with update_plan_step (reconcile step states) or summarize the panel in your reply — do not re-execute the work.", true
|
||||
@@ -389,6 +403,28 @@ func (a *agent) handleTaskTool(ctx context.Context, sessionID, name string, args
|
||||
}
|
||||
}
|
||||
|
||||
// reachabilityPatterns matches goal text that involves making something
|
||||
// reachable/accessible/working. Used by complete_task to surface a soft
|
||||
// warning when the session goal was about reachability but no verification
|
||||
// occurred before marking success.
|
||||
var reachabilityPatterns = []*regexp.Regexp{
|
||||
regexp.MustCompile(`(?i)https?://[^\s]+`),
|
||||
regexp.MustCompile(`(?i)\.hubris\.net\w+`),
|
||||
regexp.MustCompile(`(?i)(un)?reachable`),
|
||||
regexp.MustCompile(`(?i)(not?\s+)?(accessible|reachable|responding|resolving)`),
|
||||
regexp.MustCompile(`(?i)diagnose\s+why`),
|
||||
regexp.MustCompile(`(?i)(fix|restore|bring\s+back).*(accessible|reachable|online)`),
|
||||
}
|
||||
|
||||
func mentionsReachability(goal string) bool {
|
||||
for _, p := range reachabilityPatterns {
|
||||
if p.MatchString(goal) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// autoCompleteTrivialTask is the case-1 fix from
|
||||
// plans/2026-07-11-task-completion-safety-net.md: a session that never
|
||||
// called set_goal never framed itself as a structured task, so a turn that
|
||||
|
||||
152
cmd/nomos/workers.go
Normal file
152
cmd/nomos/workers.go
Normal file
@@ -0,0 +1,152 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// runChatTurn is the shared core of an operator-initiated turn: insert an
|
||||
// assistant placeholder, run a.chat with incremental persistence (so whatever
|
||||
// happened before an abort is never lost), finalize the row, and derive a
|
||||
// title. It is agnostic to the transport: `sink` receives every agent event
|
||||
// for delivery (SSE for a live handleChat, a no-op for a queued turn that has
|
||||
// no client attached — the frontend learns about those via the poller + the
|
||||
// status-driven "working" signal). The caller MUST already hold the session's
|
||||
// turn-gate permit.
|
||||
func (a *agent) runChatTurn(pctx, ctx context.Context, sessionID, message string, sink func(agentEvent)) {
|
||||
toolCalls := []map[string]any{}
|
||||
// P3: accumulate per-iteration reasoning instead of overwriting with the
|
||||
// final `text` event (see the original inline comment in handleChat).
|
||||
var textParts []string
|
||||
var thinkingParts []string
|
||||
var finalText string
|
||||
var finalThinking string
|
||||
|
||||
placeholder, _ := json.Marshal(map[string]any{"role": "assistant", "text": ""})
|
||||
msgID, err := a.store.insertMessageReturningID(pctx, sessionID, "assistant", placeholder)
|
||||
if err != nil {
|
||||
slog.Error("nomos: chat placeholder insert failed", "session", sessionID, "error", err)
|
||||
}
|
||||
persist := func() {
|
||||
if msgID == uuid.Nil {
|
||||
return
|
||||
}
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"role": "assistant",
|
||||
"text": finalText,
|
||||
"thinking": finalThinking,
|
||||
"tool_calls": toolCalls,
|
||||
})
|
||||
a.store.updateMessage(pctx, msgID, body)
|
||||
}
|
||||
|
||||
a.chat(ctx, sessionID, message, func(ev agentEvent) {
|
||||
if ev.Type == "tool_use" || ev.Type == "tool_result" {
|
||||
if m, ok := ev.Data.(map[string]any); ok {
|
||||
m["type"] = ev.Type
|
||||
// One entry per tool call: tool_use creates it, tool_result
|
||||
// merges the result into the same entry (matched by id).
|
||||
id, _ := m["id"].(string)
|
||||
if id != "" && ev.Type == "tool_result" {
|
||||
for _, existing := range toolCalls {
|
||||
if eID, _ := existing["id"].(string); eID == id {
|
||||
for k, v := range m {
|
||||
existing[k] = v
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
} else {
|
||||
toolCalls = append(toolCalls, m)
|
||||
}
|
||||
}
|
||||
persist() // live: survives even if the client disconnects right after
|
||||
}
|
||||
if ev.Type == "text" {
|
||||
if t, ok := ev.Data.(string); ok && t != "" {
|
||||
if ev.IsThinking {
|
||||
thinkingParts = append(thinkingParts, t)
|
||||
finalThinking = strings.Join(thinkingParts, "\n\n")
|
||||
} else {
|
||||
textParts = append(textParts, t)
|
||||
finalText = strings.Join(textParts, "\n\n")
|
||||
}
|
||||
persist()
|
||||
}
|
||||
}
|
||||
sink(ev)
|
||||
})
|
||||
|
||||
// B.6: if the turn ended with no text and no tool calls (the model
|
||||
// empty-response'd and all retries failed), delete the placeholder row
|
||||
// instead of persisting an empty bubble.
|
||||
if finalText == "" && len(toolCalls) == 0 && msgID != uuid.Nil {
|
||||
a.store.deleteMessage(pctx, msgID)
|
||||
} else {
|
||||
persist() // final state — same row, updated one last time
|
||||
}
|
||||
|
||||
// Title: prefer the goal once set; else the first assistant answer.
|
||||
if finalText != "" && sessionID != "ephemeral" {
|
||||
var goalTitle string
|
||||
if sess, gerr := a.store.getSession(pctx, sessionID); gerr == nil && sess.Goal != "" {
|
||||
goalTitle = truncate(sess.Goal, 120)
|
||||
}
|
||||
title := goalTitle
|
||||
if title == "" {
|
||||
title = truncate(finalText, 80)
|
||||
}
|
||||
if title != "" {
|
||||
a.store.updateSessionTitle(pctx, sessionID, title)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// drainAcquireWait is how long drainQueued blocks for a busy gate before
|
||||
// re-queuing and deferring to the holder's own release-drain. A package var so
|
||||
// tests can shorten it; in production it just needs to outlast the brief
|
||||
// release→drain handoff window.
|
||||
var drainAcquireWait = 5 * time.Second
|
||||
|
||||
// drainQueued runs every queued operator message for a session as its own turn,
|
||||
// one at a time, under the turn gate. Called (in a goroutine) whenever a turn
|
||||
// releases the gate — from handleChat (live) and resumeSession (background) —
|
||||
// so a message queued while the agent was busy is acted on as soon as it's
|
||||
// free, without the operator re-sending. See messagequeue.go (plan 2026-08-03
|
||||
// F2).
|
||||
//
|
||||
// Each queued turn is persisted incrementally and has no SSE client (the
|
||||
// browser detached after receiving the `queued` event); the frontend sees the
|
||||
// result via the 3s poller and the status-driven "working" indicator.
|
||||
func (a *agent) drainQueued(ctx context.Context, sessionID string) {
|
||||
for {
|
||||
msg, ok := a.queue.dequeue(sessionID)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
// Block briefly for the gate. If a live turn grabbed it first, put the
|
||||
// message back — that turn's release will drain it again. Never stack.
|
||||
if !a.gate.acquire(sessionID, drainAcquireWait) {
|
||||
a.queue.requeueFront(sessionID, msg)
|
||||
return
|
||||
}
|
||||
slog.Info("nomos: running queued operator message", "session", sessionID)
|
||||
pctx := context.Background()
|
||||
// Run the turn inside a per-iteration closure so the gate release is
|
||||
// deferred to the end of THIS turn (and runs even if runChatTurn
|
||||
// panics — safego recovers the panic at the goroutine boundary, so a
|
||||
// non-deferred release would be skipped and the session's permit held
|
||||
// forever, deadlocking all future turns). A bare `defer release` in
|
||||
// the loop would be wrong too: Go defers run at function exit, not
|
||||
// iteration exit, so the gate would stay held across iterations.
|
||||
func() {
|
||||
defer a.gate.release(sessionID)
|
||||
a.runChatTurn(pctx, ctx, sessionID, msg, func(agentEvent) {})
|
||||
}()
|
||||
}
|
||||
}
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/execworker"
|
||||
"github.com/dtoro/oikos/internal/httpapi"
|
||||
"github.com/dtoro/oikos/internal/knowledge"
|
||||
"github.com/dtoro/oikos/internal/notifier"
|
||||
@@ -23,6 +24,7 @@ import (
|
||||
|
||||
var schedulerRunner = scheduler.RunnerForMain()
|
||||
var notifierRunner = notifier.RunnerForMain()
|
||||
var execWorkerRunner = execworker.RunnerForMain()
|
||||
|
||||
func main() {
|
||||
if len(os.Args) < 2 {
|
||||
@@ -37,12 +39,39 @@ func main() {
|
||||
logger := observability.NewLogger(cfg.Debug)
|
||||
slog.SetDefault(logger)
|
||||
|
||||
slog.Info("starting oikos", "role", role, "config", cfg)
|
||||
|
||||
ctx, cancel := signal.NotifyContext(context.Background(),
|
||||
syscall.SIGTERM, syscall.SIGINT)
|
||||
defer cancel()
|
||||
|
||||
// Resolve secrets from Infisical, overlaying env-derived config values.
|
||||
// If Infisical is not configured, env vars are used as-is (no change).
|
||||
sec := secrets.NewManagerFromConfig(
|
||||
cfg.InfisicalSiteURL,
|
||||
cfg.InfisicalClientID,
|
||||
cfg.InfisicalClientSecret,
|
||||
cfg.InfisicalProjectID,
|
||||
cfg.InfisicalEnv,
|
||||
cfg.SecretsDir,
|
||||
)
|
||||
if sec != nil {
|
||||
overlays := secrets.ConfigOverlays(map[string]func(string){
|
||||
"matrix_token": func(v string) { cfg.MatrixToken = v },
|
||||
"approval_hmac-secret": func(v string) { cfg.ApprovalHMACSecret = v },
|
||||
"mcp_bearer-token": func(v string) { cfg.MCPBearerToken = v },
|
||||
"api_token": func(v string) { cfg.APIToken = v },
|
||||
"oidc_client-secret": func(v string) { cfg.OIDCClientSecret = v },
|
||||
})
|
||||
n := secrets.OverlayConfig(ctx, sec, overlays)
|
||||
slog.Info("secrets resolved from Infisical", "count", n)
|
||||
|
||||
secrets.VerifyExpectedSecrets(ctx, sec, []string{
|
||||
"matrix_token", "approval_hmac-secret", "mcp_bearer-token",
|
||||
"api_token", "openrouter_api-key", "webhook_hmac-secret",
|
||||
})
|
||||
}
|
||||
|
||||
slog.Info("starting oikos", "role", role, "config", cfg)
|
||||
|
||||
switch role {
|
||||
case "migrate":
|
||||
if err := runMigrate(ctx, cfg); err != nil {
|
||||
@@ -68,6 +97,8 @@ func main() {
|
||||
runWithPool(ctx, cfg, "scheduler", schedulerRunner)
|
||||
case "notifier":
|
||||
runWithPool(ctx, cfg, "notifier", notifierRunner)
|
||||
case "execution-worker":
|
||||
runWithPool(ctx, cfg, "execution-worker", execWorkerRunner)
|
||||
case "all":
|
||||
pool, err := db.New(ctx, cfg.DatabaseURL)
|
||||
if err != nil {
|
||||
@@ -83,8 +114,9 @@ func main() {
|
||||
|
||||
go schedulerRunner(ctx, pool, cfg)
|
||||
go notifierRunner(ctx, pool, cfg)
|
||||
go execWorkerRunner(ctx, pool, cfg)
|
||||
|
||||
slog.Info("all: starting api with scheduler + notifier in background")
|
||||
slog.Info("all: starting api with scheduler + notifier + execution-worker in background")
|
||||
if err := httpapi.ListenAndServe(ctx, pool, cfg); err != nil {
|
||||
slog.Error("api failed", "error", err)
|
||||
os.Exit(1)
|
||||
@@ -115,7 +147,7 @@ Roles:
|
||||
scheduler Run the observe loop
|
||||
notifier Run the notification service (Matrix alerts)
|
||||
all Run all roles in one process (dev mode)
|
||||
secret Secret management (Infisical)
|
||||
secret Secret management (Infisical: get, set, list, verify, audit, migrate, export-sops)
|
||||
knowledge Convert wiki to knowledge seed (one-shot)
|
||||
version Print version info
|
||||
|
||||
@@ -289,20 +321,45 @@ func runWithPool(ctx context.Context, cfg config.Config, name string, fn func(co
|
||||
|
||||
func runSecret(ctx context.Context, cfg config.Config) {
|
||||
if len(os.Args) < 3 {
|
||||
fmt.Fprintln(os.Stderr, "usage: oikos secret <list|migrate|export-sops>")
|
||||
fmt.Fprintln(os.Stderr, "usage: oikos secret <get|set|list|migrate|export-sops>")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
sub := os.Args[2]
|
||||
secretsDir := cfg.SecretsDir
|
||||
if secretsDir == "" {
|
||||
secretsDir = "archive/secrets-sops-backup"
|
||||
}
|
||||
sopsBackend := secrets.NewSOPSBackend(secretsDir)
|
||||
|
||||
// For get/set/list: use Infisical directly
|
||||
switch sub {
|
||||
case "get":
|
||||
if len(os.Args) < 4 {
|
||||
fmt.Fprintln(os.Stderr, "usage: oikos secret get <key>")
|
||||
os.Exit(1)
|
||||
}
|
||||
key := os.Args[3]
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
val, err := backend.Get(ctx, key)
|
||||
if err != nil {
|
||||
slog.Error("secret get", "key", key, "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Println(val)
|
||||
|
||||
case "set":
|
||||
if len(os.Args) < 5 {
|
||||
fmt.Fprintln(os.Stderr, "usage: oikos secret set <key> <value>")
|
||||
os.Exit(1)
|
||||
}
|
||||
key := os.Args[3]
|
||||
value := os.Args[4]
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
if err := backend.Set(ctx, key, value); err != nil {
|
||||
slog.Error("secret set", "key", key, "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("stored: %s\n", key)
|
||||
|
||||
case "list":
|
||||
keys, err := sopsBackend.List(ctx)
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
keys, err := backend.List(ctx)
|
||||
if err != nil {
|
||||
slog.Error("secret list", "error", err)
|
||||
os.Exit(1)
|
||||
@@ -311,24 +368,138 @@ func runSecret(ctx context.Context, cfg config.Config) {
|
||||
fmt.Println(k)
|
||||
}
|
||||
|
||||
case "migrate":
|
||||
infCfg := secrets.InfisicalConfig{
|
||||
SiteURL: cfg.InfisicalSiteURL,
|
||||
ClientID: cfg.InfisicalClientID,
|
||||
ClientSecret: cfg.InfisicalClientSecret,
|
||||
ProjectID: cfg.InfisicalProjectID,
|
||||
SecretPath: "/",
|
||||
Env: cfg.InfisicalEnv,
|
||||
}
|
||||
if infCfg.Env == "" {
|
||||
infCfg.Env = "dev"
|
||||
}
|
||||
if infCfg.SiteURL == "" {
|
||||
fmt.Fprintln(os.Stderr, "error: OIKOS_INFISICAL_SITE_URL not set")
|
||||
os.Exit(1)
|
||||
}
|
||||
case "verify":
|
||||
runSecretVerify(ctx, cfg)
|
||||
|
||||
infBackend := secrets.NewInfisicalBackend(infCfg)
|
||||
case "audit":
|
||||
runSecretAudit(ctx, cfg)
|
||||
|
||||
case "migrate", "export-sops":
|
||||
runSecretLegacy(ctx, cfg, sub)
|
||||
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "unknown secret command: %s\n", sub)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// expectedSecrets is the set of keys that should exist in Infisical
|
||||
// for a fully-migrated deployment.
|
||||
var expectedSecrets = []string{
|
||||
"matrix_token",
|
||||
"approval_hmac-secret",
|
||||
"mcp_bearer-token",
|
||||
"api_token",
|
||||
"openrouter_api-key",
|
||||
"webhook_hmac-secret",
|
||||
}
|
||||
|
||||
// runSecretVerify checks that all expected secrets are present in Infisical.
|
||||
func runSecretVerify(ctx context.Context, cfg config.Config) {
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
keys, err := backend.List(ctx)
|
||||
if err != nil {
|
||||
slog.Error("verify: list", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
keySet := make(map[string]struct{}, len(keys))
|
||||
for _, k := range keys {
|
||||
keySet[k] = struct{}{}
|
||||
}
|
||||
|
||||
missing := 0
|
||||
for _, exp := range expectedSecrets {
|
||||
if _, ok := keySet[exp]; !ok {
|
||||
fmt.Printf("MISSING: %s\n", exp)
|
||||
missing++
|
||||
} else {
|
||||
fmt.Printf("OK: %s\n", exp)
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Printf("\n%d/%d present, %d missing\n", len(expectedSecrets)-missing, len(expectedSecrets), missing)
|
||||
if missing > 0 {
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// runSecretAudit resolves all expected secrets from Infisical and prints
|
||||
// a diff against current env-derived values. Values are truncated for safety.
|
||||
func runSecretAudit(ctx context.Context, cfg config.Config) {
|
||||
backend := newInfisicalBackendOrFail(cfg)
|
||||
|
||||
envValues := map[string]string{
|
||||
"matrix_token": cfg.MatrixToken,
|
||||
"approval_hmac-secret": cfg.ApprovalHMACSecret,
|
||||
"mcp_bearer-token": cfg.MCPBearerToken,
|
||||
"api_token": cfg.APIToken,
|
||||
"oidc_client-secret": cfg.OIDCClientSecret,
|
||||
}
|
||||
|
||||
fmt.Println("key infisical env status")
|
||||
fmt.Println(strings.Repeat("-", 72))
|
||||
|
||||
for _, key := range expectedSecrets {
|
||||
infVal, infErr := backend.Get(ctx, key)
|
||||
envVal := envValues[key]
|
||||
|
||||
if infErr != nil {
|
||||
fmt.Printf("%-29s ERROR %-10s NOT-IN-INFISICAL\n", key, trunc(envVal, 8))
|
||||
continue
|
||||
}
|
||||
if envVal == "" {
|
||||
fmt.Printf("%-29s %-10s (empty) INFISICAL-ONLY\n", key, trunc(infVal, 8))
|
||||
continue
|
||||
}
|
||||
if infVal == envVal {
|
||||
fmt.Printf("%-29s %-10s %-10s MATCH\n", key, trunc(infVal, 8), trunc(envVal, 8))
|
||||
} else {
|
||||
fmt.Printf("%-29s %-10s %-10s DRIFT\n", key, trunc(infVal, 8), trunc(envVal, 8))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func trunc(s string, n int) string {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
if n > 1 {
|
||||
return s[:n-1] + "…"
|
||||
}
|
||||
return s[:n]
|
||||
}
|
||||
|
||||
// newInfisicalBackendOrFail creates an Infisical backend from config or exits.
|
||||
func newInfisicalBackendOrFail(cfg config.Config) *secrets.InfisicalBackend {
|
||||
if cfg.InfisicalSiteURL == "" {
|
||||
fmt.Fprintln(os.Stderr, "error: OIKOS_INFISICAL_SITE_URL not set")
|
||||
os.Exit(1)
|
||||
}
|
||||
infCfg := secrets.InfisicalConfig{
|
||||
SiteURL: cfg.InfisicalSiteURL,
|
||||
ClientID: cfg.InfisicalClientID,
|
||||
ClientSecret: cfg.InfisicalClientSecret,
|
||||
ProjectID: cfg.InfisicalProjectID,
|
||||
SecretPath: "/",
|
||||
Env: cfg.InfisicalEnv,
|
||||
}
|
||||
if infCfg.Env == "" {
|
||||
infCfg.Env = "dev"
|
||||
}
|
||||
return secrets.NewInfisicalBackend(infCfg)
|
||||
}
|
||||
|
||||
// runSecretLegacy handles SOPS-only commands (migrate, export-sops).
|
||||
func runSecretLegacy(ctx context.Context, cfg config.Config, sub string) {
|
||||
secretsDir := cfg.SecretsDir
|
||||
if secretsDir == "" {
|
||||
secretsDir = "archive/secrets-sops-backup"
|
||||
}
|
||||
sopsBackend := secrets.NewSOPSBackend(secretsDir)
|
||||
|
||||
switch sub {
|
||||
case "migrate":
|
||||
infBackend := newInfisicalBackendOrFail(cfg)
|
||||
keys, err := sopsBackend.List(ctx)
|
||||
if err != nil {
|
||||
slog.Error("migrate: read sops", "error", err)
|
||||
@@ -366,10 +537,6 @@ func runSecret(ctx context.Context, cfg config.Config) {
|
||||
fmt.Printf("%s: <sops-encrypted>\n", k)
|
||||
}
|
||||
fmt.Printf("\n# To restore: sops -d secrets/*.yaml\n")
|
||||
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "unknown secret command: %s\n", sub)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
@@ -12,26 +13,31 @@ import (
|
||||
"os/exec"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
)
|
||||
|
||||
func main() {
|
||||
ctx := context.Background()
|
||||
|
||||
port := os.Getenv("WEBHOOK_LISTEN")
|
||||
if port == "" {
|
||||
port = ":9797"
|
||||
}
|
||||
|
||||
secret := os.Getenv("WEBHOOK_HMAC_SECRET")
|
||||
if secret == "" {
|
||||
fmt.Fprintln(os.Stderr, "WEBHOOK_HMAC_SECRET must be set")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
repoDir := os.Getenv("WEBHOOK_REPO_DIR")
|
||||
if repoDir == "" {
|
||||
repoDir = os.Getenv("HOME") + "/Projects/oikos"
|
||||
}
|
||||
|
||||
// Create secrets manager once, share between HMAC resolution and deploy
|
||||
sec := newSecrets()
|
||||
secret := resolveWebhookHMAC(ctx, sec)
|
||||
if secret == "" {
|
||||
fmt.Fprintln(os.Stderr, "WEBHOOK_HMAC_SECRET must be set (env var or Infisical webhook_hmac-secret)")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/deploy", func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
@@ -66,11 +72,16 @@ func main() {
|
||||
w.Write([]byte(`{"status":"deploy started"}`))
|
||||
|
||||
safego.Go("webhook:deploy", func() {
|
||||
apiToken := ""
|
||||
if sec != nil {
|
||||
apiToken = secrets.ResolveSecret(ctx, sec, "api_token", "")
|
||||
}
|
||||
cmd := exec.Command(repoDir + "/scripts/deploy.sh")
|
||||
cmd.Dir = repoDir
|
||||
cmd.Env = append(os.Environ(),
|
||||
"REPO_DIR="+repoDir,
|
||||
"PROFILE=full",
|
||||
"OIKOS_API_TOKEN="+apiToken,
|
||||
)
|
||||
cmd.Stdout = os.Stdout
|
||||
cmd.Stderr = os.Stderr
|
||||
@@ -94,3 +105,25 @@ func main() {
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// newSecrets creates the Infisical secrets manager from env vars.
|
||||
func newSecrets() *secrets.Manager {
|
||||
return secrets.NewManagerFromConfig(
|
||||
os.Getenv("OIKOS_INFISICAL_SITE_URL"),
|
||||
os.Getenv("OIKOS_INFISICAL_CLIENT_ID"),
|
||||
os.Getenv("OIKOS_INFISICAL_CLIENT_SECRET"),
|
||||
os.Getenv("OIKOS_INFISICAL_PROJECT_ID"),
|
||||
os.Getenv("OIKOS_INFISICAL_ENV"),
|
||||
os.Getenv("OIKOS_SECRETS_DIR"),
|
||||
)
|
||||
}
|
||||
|
||||
// resolveWebhookHMAC fetches the webhook HMAC secret from Infisical,
|
||||
// falling back to the WEBHOOK_HMAC_SECRET env var.
|
||||
func resolveWebhookHMAC(ctx context.Context, sec *secrets.Manager) string {
|
||||
envFallback := os.Getenv("WEBHOOK_HMAC_SECRET")
|
||||
if sec == nil {
|
||||
return envFallback
|
||||
}
|
||||
return secrets.ResolveSecret(ctx, sec, "webhook_hmac-secret", envFallback)
|
||||
}
|
||||
@@ -8,7 +8,8 @@ FROM node:22-alpine AS builder
|
||||
|
||||
WORKDIR /build/web
|
||||
COPY web/package.json web/package-lock.json ./
|
||||
RUN npm ci
|
||||
COPY web/vendor /build/vendor
|
||||
RUN npm install --no-audit --no-fund
|
||||
COPY VERSION ./
|
||||
COPY web/ ./
|
||||
RUN npm run build
|
||||
|
||||
@@ -20,6 +20,8 @@ services:
|
||||
- "5432:5432"
|
||||
volumes:
|
||||
- pg-data:/var/lib/postgresql/data
|
||||
mem_limit: 1g
|
||||
cpus: 2.0
|
||||
healthcheck:
|
||||
test: ["CMD", "pg_isready", "-U", "oikos"]
|
||||
interval: 5s
|
||||
@@ -28,6 +30,7 @@ services:
|
||||
|
||||
# One-shot: run migrations then exit
|
||||
migrate:
|
||||
image: oikos-migrate:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -38,9 +41,12 @@ services:
|
||||
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
|
||||
command: ["migrate"]
|
||||
restart: "no"
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
|
||||
# One-shot: ingest seeds then exit
|
||||
seed:
|
||||
image: oikos-seed:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -52,9 +58,12 @@ services:
|
||||
OIKOS_SEEDS_DIR: /seeds
|
||||
command: ["seed"]
|
||||
restart: "no"
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
|
||||
# API server (Phase 2)
|
||||
api:
|
||||
image: oikos-api:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -76,6 +85,16 @@ services:
|
||||
OIKOS_OIDC_CLIENT_ID: ${OIKOS_OIDC_CLIENT_ID:-otkHBSueHJsYtOHstL6rn5izeGgyOsavp1qA1hod}
|
||||
OIKOS_NOMOS_AGENT_SLUG: ${OIKOS_NOMOS_AGENT_SLUG:-agent:nomos}
|
||||
NOMOS_PROXY_URL: http://nomos:8092
|
||||
# Rate limiting (plan D3). Default off; set OIKOS_API_RATE_LIMIT to a
|
||||
# requests/sec value to throttle runaway agent loops per source IP.
|
||||
OIKOS_API_RATE_LIMIT: ${OIKOS_API_RATE_LIMIT:-}
|
||||
OIKOS_API_RATE_BURST: ${OIKOS_API_RATE_BURST:-}
|
||||
# Infisical secret store (Phase 5)
|
||||
OIKOS_INFISICAL_SITE_URL: ${OIKOS_INFISICAL_SITE_URL:-}
|
||||
OIKOS_INFISICAL_CLIENT_ID: ${OIKOS_INFISICAL_CLIENT_ID:-}
|
||||
OIKOS_INFISICAL_CLIENT_SECRET: ${OIKOS_INFISICAL_CLIENT_SECRET:-}
|
||||
OIKOS_INFISICAL_PROJECT_ID: ${OIKOS_INFISICAL_PROJECT_ID:-}
|
||||
OIKOS_INFISICAL_ENV: ${OIKOS_INFISICAL_ENV:-dev}
|
||||
volumes:
|
||||
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
|
||||
ports:
|
||||
@@ -83,6 +102,8 @@ services:
|
||||
command: ["api"]
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 30s
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
# Exists so nomos can wait for the API to actually answer rather than just
|
||||
# for its container to exist — see nomos's depends_on below. wget is
|
||||
# BusyBox's, already in the alpine runtime image, so this adds no
|
||||
@@ -94,10 +115,15 @@ services:
|
||||
retries: 10
|
||||
# Migrations and seed run before this container, but the first bind can
|
||||
# still take a moment; failures inside the start period don't count.
|
||||
start_period: 10s
|
||||
# The api's NewHandler stalls on TWO unreachable external deps at startup
|
||||
# before binding :8090: Infisical (4x auth retries, ~40s) and OIDC
|
||||
# discovery (auth.hubris.network, ~35s of timeouts). Total ~90-95s, so
|
||||
# the start period must clear it or nomos (depends_on: api-healthy) fails.
|
||||
start_period: 180s
|
||||
|
||||
# Scheduler (Phase 3) — observe loop
|
||||
scheduler:
|
||||
image: oikos-scheduler:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -112,6 +138,9 @@ services:
|
||||
OIKOS_SCHEDULER_INTERVAL: "30s"
|
||||
OIKOS_SSH_KEY_PATH: /etc/oikos/ssh_key
|
||||
OIKOS_SSH_USER: root
|
||||
# Liveness probe (plan D5): exposes a staleness-aware /healthz inside
|
||||
# the container; the scheduler bumps it each check pass.
|
||||
OIKOS_HEALTH_LISTEN: ":8093"
|
||||
volumes:
|
||||
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
|
||||
cap_add:
|
||||
@@ -119,9 +148,18 @@ services:
|
||||
command: ["scheduler"]
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 30s
|
||||
mem_limit: 256m
|
||||
cpus: 1.0
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8093/healthz"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 90s
|
||||
|
||||
# Notifier (Phase 3) — Matrix alerts
|
||||
notifier:
|
||||
image: oikos-notifier:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
@@ -138,12 +176,59 @@ services:
|
||||
OIKOS_MATRIX_USER: ${OIKOS_MATRIX_USER:-@hermes:hubris.network}
|
||||
OIKOS_MATRIX_TOKEN: ${OIKOS_MATRIX_TOKEN}
|
||||
OIKOS_MATRIX_ROOM: ${OIKOS_MATRIX_ROOM:-!alerts:hubris.network}
|
||||
# Liveness probe (plan D5): bumps each approval/reaction tick.
|
||||
OIKOS_HEALTH_LISTEN: ":8094"
|
||||
OIKOS_INFISICAL_SITE_URL: ${OIKOS_INFISICAL_SITE_URL:-}
|
||||
OIKOS_INFISICAL_CLIENT_ID: ${OIKOS_INFISICAL_CLIENT_ID:-}
|
||||
OIKOS_INFISICAL_CLIENT_SECRET: ${OIKOS_INFISICAL_CLIENT_SECRET:-}
|
||||
OIKOS_INFISICAL_PROJECT_ID: ${OIKOS_INFISICAL_PROJECT_ID:-}
|
||||
OIKOS_INFISICAL_ENV: ${OIKOS_INFISICAL_ENV:-dev}
|
||||
command: ["notifier"]
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 30s
|
||||
mem_limit: 128m
|
||||
cpus: 0.5
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8094/healthz"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 120s
|
||||
|
||||
# Execution worker (Phase 6) — Postgres-backed job queue
|
||||
execution-worker:
|
||||
image: oikos-execution-worker:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/oikos/Dockerfile
|
||||
restart: unless-stopped
|
||||
profiles: ["dev", "full"]
|
||||
depends_on:
|
||||
seed:
|
||||
condition: service_completed_successfully
|
||||
environment:
|
||||
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
|
||||
OIKOS_DEBUG: "true"
|
||||
OIKOS_SSH_KEY_PATH: /etc/oikos/ssh_key
|
||||
OIKOS_SSH_USER: root
|
||||
OIKOS_HEALTH_LISTEN: ":8095"
|
||||
volumes:
|
||||
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
|
||||
command: ["execution-worker"]
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 30s
|
||||
mem_limit: 256m
|
||||
cpus: 1.0
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8095/healthz"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 90s
|
||||
|
||||
# Nomos agent gateway (Phase 4) — mesh-published :8092
|
||||
nomos:
|
||||
image: oikos-nomos:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/nomos/Dockerfile
|
||||
@@ -166,16 +251,34 @@ services:
|
||||
# Must match api's OIKOS_MCP_BEARER_TOKEN above — api's combinedAuth
|
||||
# rejects every request without it now (no dev-open bypass).
|
||||
OIKOS_MCP_BEARER_TOKEN: ${OIKOS_MCP_BEARER_TOKEN:-dev-token}
|
||||
# Infisical secret store (Phase 5) — nomos resolves mcp_bearer-token
|
||||
# and openrouter_api-key from here, overriding the env values above.
|
||||
OIKOS_INFISICAL_SITE_URL: ${OIKOS_INFISICAL_SITE_URL:-}
|
||||
OIKOS_INFISICAL_CLIENT_ID: ${OIKOS_INFISICAL_CLIENT_ID:-}
|
||||
OIKOS_INFISICAL_CLIENT_SECRET: ${OIKOS_INFISICAL_CLIENT_SECRET:-}
|
||||
OIKOS_INFISICAL_PROJECT_ID: ${OIKOS_INFISICAL_PROJECT_ID:-}
|
||||
OIKOS_INFISICAL_ENV: ${OIKOS_INFISICAL_ENV:-dev}
|
||||
ports:
|
||||
- "8092:8092"
|
||||
stop_signal: SIGTERM
|
||||
stop_grace_period: 10s
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
# nomos runs on a distroless image (no shell/wget), so the healthcheck
|
||||
# uses the binary's own `healthcheck` subcommand to self-probe /healthz.
|
||||
healthcheck:
|
||||
test: ["CMD", "/nomos", "healthcheck"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 15s
|
||||
|
||||
# Control-room SPA — static build served behind Caddy. The outer
|
||||
# production Caddy (caddy-conf repo, LXC 121) splits /api/*, /mcp,
|
||||
# /agent/* off to api:8090 and sends everything else here; this
|
||||
# container only serves static files with SPA-fallback routing.
|
||||
web:
|
||||
image: oikos-web:${OIKOS_VERSION:-latest}
|
||||
build:
|
||||
context: .
|
||||
dockerfile: compose/web/Dockerfile
|
||||
@@ -184,6 +287,8 @@ services:
|
||||
ports:
|
||||
- "8091:80"
|
||||
stop_signal: SIGTERM
|
||||
mem_limit: 64m
|
||||
cpus: 0.25
|
||||
|
||||
# Redis (required by Infisical — Phase 5)
|
||||
redis:
|
||||
@@ -192,6 +297,8 @@ services:
|
||||
profiles: ["infisical", "full"]
|
||||
volumes:
|
||||
- redis-data:/data
|
||||
mem_limit: 128m
|
||||
cpus: 0.5
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 5s
|
||||
@@ -200,7 +307,7 @@ services:
|
||||
|
||||
# Infisical self-hosted (Phase 5 secrets management)
|
||||
infisical:
|
||||
image: infisical/infisical:latest
|
||||
image: infisical/infisical:v0.162.19
|
||||
restart: unless-stopped
|
||||
profiles: ["infisical", "full"]
|
||||
depends_on:
|
||||
@@ -222,6 +329,8 @@ services:
|
||||
REDIS_URL: redis://redis:6379
|
||||
ports:
|
||||
- "8080:8080"
|
||||
mem_limit: 512m
|
||||
cpus: 1.0
|
||||
|
||||
volumes:
|
||||
pg-data:
|
||||
|
||||
@@ -234,9 +234,33 @@ sequenceDiagram
|
||||
|
||||
---
|
||||
|
||||
**2026-07-08 — renamed to Nomos.** The Hermes agent gateway was renamed to
|
||||
**2026-07-08 — renamed to Nomos.**
|
||||
Nomos (from *oikonomos*, the steward of the oikos) under the
|
||||
[Nomos resident agent plan](../../plans/2026-07-08-nomos-resident-agent.md),
|
||||
|
||||
### Hermes MCP client setup
|
||||
|
||||
To connect a Hermes Agent instance to oikos as a native MCP client, add to
|
||||
`~/.hermes/config.yaml`:
|
||||
|
||||
```yaml
|
||||
mcp_servers:
|
||||
oikos:
|
||||
url: "https://mcp.hubris.network/mcp"
|
||||
headers:
|
||||
Authorization: "Bearer <OIKOS_MCP_BEARER_TOKEN>"
|
||||
timeout: 180
|
||||
```
|
||||
|
||||
Run `/reload-mcp` in-session or restart Hermes. Tools appear as
|
||||
`mcp__oikos__*`.
|
||||
|
||||
**Caveat:** Hermes stores the bearer token in plaintext in `config.yaml` —
|
||||
it does not support `${VAR}` interpolation in MCP server headers. Ensure
|
||||
`security.redact_secrets: true` (default) so the token value is stripped
|
||||
from tool output and logs. File an upstream feature request at
|
||||
https://github.com/NousResearch/hermes-agent/issues for env-var
|
||||
interpolation support.
|
||||
N0 milestone. The gateway binary (`cmd/nomos`), Docker service, DB slug
|
||||
(`agent:nomos`), and all referencing docs were updated. All architectural
|
||||
principles in this ADR remain unchanged.
|
||||
2
go.mod
2
go.mod
@@ -19,6 +19,7 @@ require (
|
||||
golang.org/x/crypto v0.53.0
|
||||
golang.org/x/sync v0.21.0
|
||||
golang.org/x/sys v0.46.0
|
||||
golang.org/x/time v0.14.0
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
|
||||
@@ -88,7 +89,6 @@ require (
|
||||
golang.org/x/net v0.55.0 // indirect
|
||||
golang.org/x/oauth2 v0.35.0 // indirect
|
||||
golang.org/x/text v0.38.0 // indirect
|
||||
golang.org/x/time v0.14.0 // indirect
|
||||
google.golang.org/api v0.267.0 // indirect
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409 // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 // indirect
|
||||
|
||||
@@ -17,7 +17,6 @@ import (
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/google/uuid"
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
// Run starts the actuator loop. Blocks until ctx is cancelled.
|
||||
@@ -403,7 +402,7 @@ func ProvisionVM(ctx context.Context, pool *db.Pool, entityID uuid.UUID, attrs m
|
||||
return nil
|
||||
}
|
||||
|
||||
// sshExecSimple runs a command over SSH with a simple client setup.
|
||||
// sshExecSimple runs a command over SSH using the shared dial/run primitives.
|
||||
// Uses the default SSH key from SSH_KEY_PATH or ~/.ssh/id_rsa.
|
||||
func sshExecSimple(ctx context.Context, host, user, command string) (string, error) {
|
||||
keyPath := os.Getenv("SSH_KEY_PATH")
|
||||
@@ -411,55 +410,19 @@ func sshExecSimple(ctx context.Context, host, user, command string) (string, err
|
||||
keyPath = os.Getenv("HOME") + "/.ssh/id_rsa"
|
||||
}
|
||||
|
||||
keyBytes, err := os.ReadFile(keyPath)
|
||||
signer, err := LoadSigner(keyPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("read ssh key: %w", err)
|
||||
return "", err
|
||||
}
|
||||
|
||||
signer, err := ssh.ParsePrivateKey(keyBytes)
|
||||
client, err := Dial(ctx, DialOptions{Host: host, User: user, Signer: signer})
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("parse ssh key: %w", err)
|
||||
}
|
||||
|
||||
clientCfg := &ssh.ClientConfig{
|
||||
User: user,
|
||||
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
|
||||
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
|
||||
Timeout: 10 * time.Second,
|
||||
}
|
||||
|
||||
client, err := ssh.Dial("tcp", host+":22", clientCfg)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("ssh dial %s: %w", host, err)
|
||||
return "", err
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
type result struct {
|
||||
output string
|
||||
err error
|
||||
}
|
||||
ch := make(chan result, 1)
|
||||
go func() {
|
||||
out, e := session.CombinedOutput(command)
|
||||
ch <- result{output: string(out), err: e}
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
return "", ctx.Err()
|
||||
case res := <-ch:
|
||||
if res.err != nil {
|
||||
return res.output, res.err
|
||||
}
|
||||
return res.output, nil
|
||||
}
|
||||
out, err := RunCombinedOutput(ctx, client, command)
|
||||
return string(out), err
|
||||
}
|
||||
|
||||
// resolveHost resolves a host entity slug to (address, user) for SSH.
|
||||
|
||||
141
internal/actuator/client.go
Normal file
141
internal/actuator/client.go
Normal file
@@ -0,0 +1,141 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"net"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
// defaultDialTimeout bounds an SSH dial when the caller leaves Timeout unset.
|
||||
// 10s matches the previous hardcoded value at every dial site.
|
||||
const defaultDialTimeout = 10 * time.Second
|
||||
|
||||
// LoadSigner reads and parses the private key at keyPath.
|
||||
func LoadSigner(keyPath string) (ssh.Signer, error) {
|
||||
key, err := os.ReadFile(keyPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read ssh key: %w", err)
|
||||
}
|
||||
return LoadSignerFromBytes(key)
|
||||
}
|
||||
|
||||
// LoadSignerFromBytes parses an in-memory private key into an ssh.Signer.
|
||||
func LoadSignerFromBytes(key []byte) (ssh.Signer, error) {
|
||||
signer, err := ssh.ParsePrivateKey(key)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse ssh key: %w", err)
|
||||
}
|
||||
return signer, nil
|
||||
}
|
||||
|
||||
// DialOptions configures an SSH dial.
|
||||
type DialOptions struct {
|
||||
Host string
|
||||
Port int // 0 means 22
|
||||
User string
|
||||
Signer ssh.Signer
|
||||
Timeout time.Duration // dial timeout; <=0 means defaultDialTimeout
|
||||
}
|
||||
|
||||
// Dial opens a crypto/ssh connection through the centralized HostKeyCallback.
|
||||
// The connection itself is bounded by Timeout; ctx is respected by callers
|
||||
// via RunCombinedOutput once the session is running.
|
||||
func Dial(ctx context.Context, opts DialOptions) (*ssh.Client, error) {
|
||||
port := opts.Port
|
||||
if port <= 0 {
|
||||
port = 22
|
||||
}
|
||||
timeout := opts.Timeout
|
||||
if timeout <= 0 {
|
||||
timeout = defaultDialTimeout
|
||||
}
|
||||
cfg := &ssh.ClientConfig{
|
||||
User: opts.User,
|
||||
Auth: []ssh.AuthMethod{ssh.PublicKeys(opts.Signer)},
|
||||
HostKeyCallback: HostKeyCallback(),
|
||||
Timeout: timeout,
|
||||
}
|
||||
addr := net.JoinHostPort(opts.Host, fmt.Sprintf("%d", port))
|
||||
client, err := ssh.Dial("tcp", addr, cfg)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("ssh dial %s:%d: %w", opts.Host, port, err)
|
||||
}
|
||||
return client, nil
|
||||
}
|
||||
|
||||
// RunCombinedOutput runs cmd on an established client and returns its combined
|
||||
// stdout/stderr. Context cancellation closes the session to abort the remote
|
||||
// command instead of blocking until it finishes — the same goroutine+select
|
||||
// pattern the actuator, mcp, and scheduler each reimplemented before.
|
||||
func RunCombinedOutput(ctx context.Context, client *ssh.Client, cmd string) ([]byte, error) {
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
type result struct {
|
||||
out []byte
|
||||
err error
|
||||
}
|
||||
ch := make(chan result, 1)
|
||||
go func() {
|
||||
out, err := session.CombinedOutput(cmd)
|
||||
ch <- result{out: out, err: err}
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
return nil, ctx.Err()
|
||||
case res := <-ch:
|
||||
if res.err != nil {
|
||||
return res.out, fmt.Errorf("command: %w", res.err)
|
||||
}
|
||||
return res.out, nil
|
||||
}
|
||||
}
|
||||
|
||||
// RunOutput runs cmd on an established client and returns stdout only.
|
||||
// Stderr is folded into the returned error so callers that parse stdout
|
||||
// as JSON (e.g. the scheduler's check scripts) don't get interleaved
|
||||
// stderr in the output stream.
|
||||
func RunOutput(ctx context.Context, client *ssh.Client, cmd string) ([]byte, error) {
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
var outBuf, errBuf bytes.Buffer
|
||||
session.Stdout = &outBuf
|
||||
session.Stderr = &errBuf
|
||||
|
||||
type result struct {
|
||||
runErr error
|
||||
}
|
||||
ch := make(chan result, 1)
|
||||
go func() {
|
||||
ch <- result{runErr: session.Run(cmd)}
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
return nil, ctx.Err()
|
||||
case res := <-ch:
|
||||
if res.runErr != nil {
|
||||
if errBuf.Len() > 0 {
|
||||
return outBuf.Bytes(), fmt.Errorf("command: %w\nstderr: %s", res.runErr, strings.TrimSpace(errBuf.String()))
|
||||
}
|
||||
return outBuf.Bytes(), fmt.Errorf("command: %w", res.runErr)
|
||||
}
|
||||
return outBuf.Bytes(), nil
|
||||
}
|
||||
}
|
||||
88
internal/actuator/client_test.go
Normal file
88
internal/actuator/client_test.go
Normal file
@@ -0,0 +1,88 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"encoding/pem"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
func TestLoadSignerRejectsBadInput(t *testing.T) {
|
||||
if _, err := LoadSignerFromBytes([]byte("not a private key")); err == nil {
|
||||
t.Error("LoadSignerFromBytes should reject a non-key input")
|
||||
}
|
||||
if _, err := LoadSigner("/nonexistent/key"); err == nil {
|
||||
t.Error("LoadSigner should fail on a missing file")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadSignerRoundTrip(t *testing.T) {
|
||||
_, priv, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatalf("generate key: %v", err)
|
||||
}
|
||||
block, err := ssh.MarshalPrivateKey(priv, "")
|
||||
if err != nil {
|
||||
t.Fatalf("marshal private key: %v", err)
|
||||
}
|
||||
pemBytes := pem.EncodeToMemory(block)
|
||||
|
||||
signer, err := LoadSignerFromBytes(pemBytes)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadSignerFromBytes on a valid key: %v", err)
|
||||
}
|
||||
if signer == nil {
|
||||
t.Fatal("signer is nil")
|
||||
}
|
||||
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "id_ed25519")
|
||||
if err := os.WriteFile(path, pemBytes, 0o600); err != nil {
|
||||
t.Fatalf("write key file: %v", err)
|
||||
}
|
||||
fromFile, err := LoadSigner(path)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadSigner(%s): %v", path, err)
|
||||
}
|
||||
if !bytes.Equal(fromFile.PublicKey().Marshal(), signer.PublicKey().Marshal()) {
|
||||
t.Error("file and in-memory signers resolved to different public keys")
|
||||
}
|
||||
}
|
||||
|
||||
// Dial needs a real SSH server to run a command, but its option normalization
|
||||
// is verifiable without one: a zero Port must default to 22 (so the dial error
|
||||
// references host:22, not host:0), and a closed port yields a dial error rather
|
||||
// than panicking.
|
||||
func TestDialDefaultsPort(t *testing.T) {
|
||||
_, err := Dial(context.Background(), DialOptions{Host: "127.0.0.1", Signer: mustSigner(t)})
|
||||
if err == nil {
|
||||
t.Fatal("Dial to a closed port should fail")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "127.0.0.1:22") {
|
||||
t.Errorf("Dial error = %q, want it to reference 127.0.0.1:22", err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustSigner(t *testing.T) ssh.Signer {
|
||||
t.Helper()
|
||||
_, priv, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatalf("generate key: %v", err)
|
||||
}
|
||||
block, err := ssh.MarshalPrivateKey(priv, "")
|
||||
if err != nil {
|
||||
t.Fatalf("marshal key: %v", err)
|
||||
}
|
||||
s, err := LoadSignerFromBytes(pem.EncodeToMemory(block))
|
||||
if err != nil {
|
||||
t.Fatalf("parse key: %v", err)
|
||||
}
|
||||
return s
|
||||
}
|
||||
129
internal/actuator/hostkeys.go
Normal file
129
internal/actuator/hostkeys.go
Normal file
@@ -0,0 +1,129 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
var (
|
||||
hostKeyMu sync.RWMutex
|
||||
hostKeyCache map[string]ssh.PublicKey
|
||||
hostKeyOnce sync.Once
|
||||
hostKeySrc HostKeySource
|
||||
)
|
||||
|
||||
// HostKeySource provides storage for SSH host public keys.
|
||||
type HostKeySource interface {
|
||||
GetHostKey(ctx context.Context, hostname string) (string, error)
|
||||
SetHostKey(ctx context.Context, hostname string, key string) error
|
||||
}
|
||||
|
||||
// SetHostKeySource sets the host key source. Must be called before
|
||||
// any SSH connections. A nil source enables TOFU-only mode (keys
|
||||
// accepted in memory but not persisted).
|
||||
func SetHostKeySource(src HostKeySource) {
|
||||
hostKeyMu.Lock()
|
||||
defer hostKeyMu.Unlock()
|
||||
hostKeySrc = src
|
||||
}
|
||||
|
||||
// HostKeyCallback returns an ssh.HostKeyCallback that verifies host keys.
|
||||
// Known keys are verified (MITM detection). Unknown keys are accepted
|
||||
// via TOFU and optionally persisted to the source.
|
||||
func HostKeyCallback() ssh.HostKeyCallback {
|
||||
return hostKeyVerify
|
||||
}
|
||||
|
||||
func hostKeyVerify(hostname string, remote net.Addr, key ssh.PublicKey) error {
|
||||
hostKeyOnce.Do(func() {
|
||||
hostKeyCache = make(map[string]ssh.PublicKey)
|
||||
})
|
||||
|
||||
normalized := hostWithoutPort(hostname)
|
||||
|
||||
hostKeyMu.RLock()
|
||||
known, exists := hostKeyCache[normalized]
|
||||
hostKeyMu.RUnlock()
|
||||
|
||||
if exists {
|
||||
if bytes.Equal(key.Marshal(), known.Marshal()) {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("SSH HOST KEY CHANGED for %s (possible MITM)", normalized)
|
||||
}
|
||||
|
||||
hostKeyMu.Lock()
|
||||
hostKeyCache[normalized] = key
|
||||
hostKeyMu.Unlock()
|
||||
|
||||
slog.Info("ssh: accepting new host key (TOFU)", "host", normalized)
|
||||
|
||||
if hostKeySrc != nil {
|
||||
go persistHostKey(normalized, key)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func persistHostKey(hostname string, key ssh.PublicKey) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
keyBase64 := key.Type() + " " + string(key.Marshal())
|
||||
if err := hostKeySrc.SetHostKey(ctx, "ssh/host-keys/"+hostname, keyBase64); err != nil {
|
||||
slog.Warn("ssh: failed to persist host key", "host", hostname, "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// LoadHostKeys pre-loads known host keys from the source into the
|
||||
// in-memory cache. Call at startup to avoid TOFU on first connection.
|
||||
// The source should return key lines in the format "key-type base64-data".
|
||||
func LoadHostKeys(ctx context.Context, hostnames []string, src HostKeySource) {
|
||||
if src == nil {
|
||||
return
|
||||
}
|
||||
SetHostKeySource(src)
|
||||
|
||||
hostKeyMu.Lock()
|
||||
defer hostKeyMu.Unlock()
|
||||
|
||||
if hostKeyCache == nil {
|
||||
hostKeyCache = make(map[string]ssh.PublicKey)
|
||||
}
|
||||
|
||||
loaded := 0
|
||||
for _, hostname := range hostnames {
|
||||
keyData, err := src.GetHostKey(ctx, "ssh/host-keys/"+hostname)
|
||||
if err != nil {
|
||||
slog.Debug("ssh: no stored key for host", "host", hostname, "error", err)
|
||||
continue
|
||||
}
|
||||
pubKey, _, _, _, err := ssh.ParseAuthorizedKey([]byte(keyData))
|
||||
if err != nil {
|
||||
slog.Warn("ssh: invalid stored key for host", "host", hostname, "error", err)
|
||||
continue
|
||||
}
|
||||
hostKeyCache[hostname] = pubKey
|
||||
loaded++
|
||||
}
|
||||
|
||||
if loaded > 0 {
|
||||
slog.Info("ssh: loaded host keys from Infisical", "count", loaded)
|
||||
}
|
||||
}
|
||||
|
||||
func hostWithoutPort(hostname string) string {
|
||||
for i := len(hostname) - 1; i >= 0; i-- {
|
||||
if hostname[i] == ':' {
|
||||
return hostname[:i]
|
||||
}
|
||||
}
|
||||
return hostname
|
||||
}
|
||||
56
internal/actuator/hostkeys_infisical.go
Normal file
56
internal/actuator/hostkeys_infisical.go
Normal file
@@ -0,0 +1,56 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
)
|
||||
|
||||
// InfisicalHostKeySource implements HostKeySource backed by Infisical.
|
||||
type InfisicalHostKeySource struct {
|
||||
sec secrets.Backend
|
||||
}
|
||||
|
||||
// NewInfisicalHostKeySource creates a HostKeySource that reads/writes
|
||||
// SSH host public keys from Infisical under the `ssh/host-keys/` prefix.
|
||||
func NewInfisicalHostKeySource(sec secrets.Backend) *InfisicalHostKeySource {
|
||||
return &InfisicalHostKeySource{sec: sec}
|
||||
}
|
||||
|
||||
func (s *InfisicalHostKeySource) GetHostKey(ctx context.Context, path string) (string, error) {
|
||||
val, err := s.sec.Get(ctx, path)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return val, nil
|
||||
}
|
||||
|
||||
func (s *InfisicalHostKeySource) SetHostKey(ctx context.Context, path string, key string) error {
|
||||
return s.sec.Set(ctx, path, key)
|
||||
}
|
||||
|
||||
// ResolveSSHHosts queries the DB for active proxmox-host and standalone-server
|
||||
// entities, returning their slugs as SSH host identifiers.
|
||||
func ResolveSSHHosts(ctx context.Context, pool *db.Pool) []string {
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT slug FROM entities
|
||||
WHERE type IN ('proxmox-host', 'standalone-server')
|
||||
AND state = 'active'
|
||||
ORDER BY slug`)
|
||||
if err != nil {
|
||||
slog.Warn("ssh: failed to list hosts", "error", err)
|
||||
return nil
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var hosts []string
|
||||
for rows.Next() {
|
||||
var slug string
|
||||
if rows.Scan(&slug) == nil {
|
||||
hosts = append(hosts, slug)
|
||||
}
|
||||
}
|
||||
return hosts
|
||||
}
|
||||
122
internal/actuator/pool.go
Normal file
122
internal/actuator/pool.go
Normal file
@@ -0,0 +1,122 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
type poolEntry struct {
|
||||
client *ssh.Client
|
||||
createdAt time.Time
|
||||
}
|
||||
|
||||
type DialPool struct {
|
||||
mu sync.RWMutex
|
||||
entries map[string]*poolEntry
|
||||
ttl time.Duration
|
||||
done chan struct{}
|
||||
stopped bool
|
||||
}
|
||||
|
||||
func NewDialPool(ttl time.Duration) *DialPool {
|
||||
p := &DialPool{
|
||||
entries: make(map[string]*poolEntry),
|
||||
ttl: ttl,
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
if ttl > 0 {
|
||||
go p.evictLoop()
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func (p *DialPool) key(opts DialOptions) string {
|
||||
port := opts.Port
|
||||
if port <= 0 {
|
||||
port = 22
|
||||
}
|
||||
return fmt.Sprintf("%s:%d", opts.Host, port)
|
||||
}
|
||||
|
||||
func (p *DialPool) Get(ctx context.Context, opts DialOptions) (*ssh.Client, error) {
|
||||
k := p.key(opts)
|
||||
p.mu.RLock()
|
||||
entry, ok := p.entries[k]
|
||||
p.mu.RUnlock()
|
||||
if ok {
|
||||
// Quick health check: a session can be created without running a
|
||||
// command — if it fails, the connection is dead and we evict it.
|
||||
testSession, err := entry.client.NewSession()
|
||||
if err == nil {
|
||||
testSession.Close()
|
||||
return entry.client, nil
|
||||
}
|
||||
p.mu.Lock()
|
||||
if p.entries[k] == entry {
|
||||
entry.client.Close()
|
||||
delete(p.entries, k)
|
||||
}
|
||||
p.mu.Unlock()
|
||||
// Fall through to dial below
|
||||
}
|
||||
client, err := Dial(ctx, opts)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
p.mu.Lock()
|
||||
if p.stopped {
|
||||
p.mu.Unlock()
|
||||
client.Close()
|
||||
return nil, fmt.Errorf("ssh dial pool: closed")
|
||||
}
|
||||
if existing, ok2 := p.entries[k]; ok2 {
|
||||
p.mu.Unlock()
|
||||
client.Close()
|
||||
return existing.client, nil
|
||||
}
|
||||
p.entries[k] = &poolEntry{client: client, createdAt: time.Now()}
|
||||
p.mu.Unlock()
|
||||
return client, nil
|
||||
}
|
||||
|
||||
func (p *DialPool) Close() {
|
||||
p.mu.Lock()
|
||||
p.stopped = true
|
||||
for k, entry := range p.entries {
|
||||
entry.client.Close()
|
||||
delete(p.entries, k)
|
||||
}
|
||||
p.mu.Unlock()
|
||||
if p.ttl > 0 {
|
||||
close(p.done)
|
||||
}
|
||||
}
|
||||
|
||||
func (p *DialPool) evictLoop() {
|
||||
ticker := time.NewTicker(p.ttl / 2)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-p.done:
|
||||
return
|
||||
case <-ticker.C:
|
||||
p.evict()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *DialPool) evict() {
|
||||
deadline := time.Now().Add(-p.ttl)
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
for k, entry := range p.entries {
|
||||
if entry.createdAt.Before(deadline) {
|
||||
entry.client.Close()
|
||||
delete(p.entries, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
@@ -131,37 +130,19 @@ func ExecuteProcedure(
|
||||
start := time.Now()
|
||||
|
||||
// Parse the SSH key
|
||||
key, err := os.ReadFile(cfg.KeyPath)
|
||||
signer, err := LoadSigner(cfg.KeyPath)
|
||||
if err != nil {
|
||||
return SSHResult{
|
||||
Err: fmt.Errorf("read ssh key: %w", err),
|
||||
Err: err,
|
||||
Duration: time.Since(start),
|
||||
Verified: false,
|
||||
}
|
||||
}
|
||||
|
||||
signer, err := ssh.ParsePrivateKey(key)
|
||||
if err != nil {
|
||||
return SSHResult{
|
||||
Err: fmt.Errorf("parse ssh key: %w", err),
|
||||
Duration: time.Since(start),
|
||||
Verified: false,
|
||||
}
|
||||
}
|
||||
|
||||
addr := net.JoinHostPort(cfg.Host, fmt.Sprintf("%d", cfg.Port))
|
||||
if cfg.Port == 0 {
|
||||
addr = net.JoinHostPort(cfg.Host, "22")
|
||||
}
|
||||
|
||||
clientCfg := &ssh.ClientConfig{
|
||||
User: cfg.User,
|
||||
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
|
||||
HostKeyCallback: ssh.InsecureIgnoreHostKey(), // restricted key; host trust via inventory
|
||||
Timeout: cfg.Timeout,
|
||||
}
|
||||
|
||||
client, err := ssh.Dial("tcp", addr, clientCfg)
|
||||
client, err := Dial(ctx, DialOptions{
|
||||
Host: cfg.Host, Port: cfg.Port, User: cfg.User,
|
||||
Signer: signer, Timeout: cfg.Timeout,
|
||||
})
|
||||
if err != nil {
|
||||
class := classifySSHError(err)
|
||||
return SSHResult{
|
||||
@@ -229,38 +210,11 @@ func ExecuteProcedure(
|
||||
}
|
||||
}
|
||||
|
||||
// runSSHCommand executes a single command over an established SSH session.
|
||||
// Uses context-aware goroutines: ctx.Done() closes the session.
|
||||
// runSSHCommand executes a single command over an established SSH session via
|
||||
// the shared RunCombinedOutput primitive (context-aware abort + combined output).
|
||||
func runSSHCommand(ctx context.Context, client *ssh.Client, command string) (string, error) {
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
// Wrap in goroutine so we can abort on ctx.Done()
|
||||
type result struct {
|
||||
output string
|
||||
err error
|
||||
}
|
||||
|
||||
ch := make(chan result, 1)
|
||||
go func() {
|
||||
out, err := session.CombinedOutput(command)
|
||||
ch <- result{output: string(out), err: err}
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
// Close the session to abort the SSH command
|
||||
session.Close()
|
||||
return "", ctx.Err()
|
||||
case res := <-ch:
|
||||
if res.err != nil {
|
||||
return res.output, fmt.Errorf("command: %w", res.err)
|
||||
}
|
||||
return res.output, nil
|
||||
}
|
||||
out, err := RunCombinedOutput(ctx, client, command)
|
||||
return string(out), err
|
||||
}
|
||||
|
||||
// ─── Procedure parsing ────────────────────────────────────────────────────
|
||||
|
||||
86
internal/actuator/stream.go
Normal file
86
internal/actuator/stream.go
Normal file
@@ -0,0 +1,86 @@
|
||||
package actuator
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
// RunStreaming runs a command on an established SSH client and forwards output
|
||||
// chunks to sink as they arrive. A nil sink collects output silently. Returns
|
||||
// the full combined output and any command error.
|
||||
func RunStreaming(ctx context.Context, client *ssh.Client, command string, sink func(stream string, chunk []byte), timeout time.Duration) (string, error) {
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
outPipe, err := session.StdoutPipe()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("stdout pipe: %w", err)
|
||||
}
|
||||
errPipe, err := session.StderrPipe()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("stderr pipe: %w", err)
|
||||
}
|
||||
|
||||
type streamResult struct {
|
||||
out string
|
||||
err error
|
||||
}
|
||||
|
||||
resultCh := make(chan streamResult, 1)
|
||||
go func() {
|
||||
var combined []byte
|
||||
done := make(chan struct{}, 2)
|
||||
|
||||
readStream := func(stream string, r io.Reader) {
|
||||
sc := bufio.NewScanner(r)
|
||||
for sc.Scan() {
|
||||
line := sc.Bytes()
|
||||
chunk := make([]byte, len(line))
|
||||
copy(chunk, line)
|
||||
if sink != nil {
|
||||
sink(stream, chunk)
|
||||
}
|
||||
if stream == "stdout" || stream == "" {
|
||||
if len(combined) > 0 {
|
||||
combined = append(combined, '\n')
|
||||
}
|
||||
combined = append(combined, chunk...)
|
||||
}
|
||||
}
|
||||
done <- struct{}{}
|
||||
}
|
||||
|
||||
go readStream("stdout", outPipe)
|
||||
go readStream("stderr", errPipe)
|
||||
|
||||
runErr := session.Run(command)
|
||||
<-done
|
||||
<-done
|
||||
resultCh <- streamResult{out: string(combined), err: runErr}
|
||||
}()
|
||||
|
||||
if timeout > 0 {
|
||||
var cancel context.CancelFunc
|
||||
ctx, cancel = context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
}
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
return "", ctx.Err()
|
||||
case res := <-resultCh:
|
||||
if res.err != nil {
|
||||
return res.out, fmt.Errorf("command: %w", res.err)
|
||||
}
|
||||
return res.out, nil
|
||||
}
|
||||
}
|
||||
202
internal/checkdefaults/build_test.go
Normal file
202
internal/checkdefaults/build_test.go
Normal file
@@ -0,0 +1,202 @@
|
||||
package checkdefaults
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/ontology"
|
||||
)
|
||||
|
||||
// Table-driven coverage of every implemented buildKind branch and the ssh()
|
||||
// helper's user/port/args propagation. The previous tests exercised only
|
||||
// ping/process/http/resource; updates, capacity, backup, cert-expiry,
|
||||
// vm-status and dns were unverified.
|
||||
func TestBuildKindAllImplementedKinds(t *testing.T) {
|
||||
host := "10.0.0.5"
|
||||
cases := []struct {
|
||||
name string
|
||||
kind string
|
||||
target Target
|
||||
attrs map[string]any
|
||||
host string
|
||||
wantSkip bool // true → expect a reason and zero defs
|
||||
wantDefs int
|
||||
wantKind string
|
||||
wantKey string // a config key to assert
|
||||
wantVal any // its expected value
|
||||
wantReason string // substring when skipping
|
||||
wantInterv int32 // expected interval on the (single) produced def
|
||||
}{
|
||||
{
|
||||
name: "ping with host", kind: KindPing, host: host,
|
||||
wantDefs: 1, wantKind: "ping", wantKey: "host", wantVal: host, wantInterv: 30,
|
||||
},
|
||||
{name: "ping no host skips", kind: KindPing, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "resource expands to four ssh scripts", kind: KindResource, host: host,
|
||||
wantDefs: 4, wantKind: "ssh-script", wantKey: "host", wantVal: host, wantInterv: 60,
|
||||
},
|
||||
{name: "resource no host skips", kind: KindResource, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "updates is daily", kind: KindUpdates, host: host,
|
||||
wantDefs: 1, wantKind: "ssh-script", wantKey: "script", wantVal: "updates_check.sh", wantInterv: 86400,
|
||||
},
|
||||
{name: "updates no host skips", kind: KindUpdates, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "capacity is one disk script", kind: KindCapacity, host: host,
|
||||
wantDefs: 1, wantKind: "ssh-script", wantKey: "script", wantVal: "disk_usage_check.sh", wantInterv: 60,
|
||||
},
|
||||
{name: "capacity no host skips", kind: KindCapacity, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "backup needs path and host", kind: KindBackup, host: host,
|
||||
attrs: map[string]any{"path": "/backups/db"},
|
||||
wantDefs: 1, wantKind: "backup-freshness", wantKey: "path", wantVal: "/backups/db", wantInterv: 86400,
|
||||
},
|
||||
{name: "backup without path skips", kind: KindBackup, host: host, wantSkip: true, wantReason: "no path"},
|
||||
{name: "backup without host skips", kind: KindBackup, attrs: map[string]any{"path": "/x"}, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{
|
||||
name: "backup honors backup_max_age_s override", kind: KindBackup, host: host,
|
||||
attrs: map[string]any{"path": "/x", "backup_max_age_s": float64(3600)},
|
||||
wantDefs: 1, wantKey: "max_age_s", wantVal: 3600,
|
||||
},
|
||||
|
||||
{
|
||||
name: "cert-expiry from hostname attr", kind: KindCertExpiry,
|
||||
attrs: map[string]any{"hostname": "media.hubris.network"},
|
||||
wantDefs: 1, wantKind: "cert-expiry", wantKey: "host", wantVal: "media.hubris.network", wantInterv: 3600,
|
||||
},
|
||||
{
|
||||
name: "cert-expiry from dotted name", kind: KindCertExpiry, target: Target{Name: "media.hubris.network"},
|
||||
wantDefs: 1, wantKey: "host", wantVal: "media.hubris.network",
|
||||
},
|
||||
{
|
||||
name: "cert-expiry propagates dial attr", kind: KindCertExpiry,
|
||||
attrs: map[string]any{"hostname": "media.hubris.network", "dial": "10.0.0.2"},
|
||||
wantDefs: 1, wantKey: "dial", wantVal: "10.0.0.2",
|
||||
},
|
||||
{name: "cert-expiry without a host name skips", kind: KindCertExpiry, target: Target{Name: "jellyfin"}, wantSkip: true, wantReason: "no hostname"},
|
||||
|
||||
{
|
||||
name: "vm-status needs pve_id", kind: KindVMStatus, attrs: map[string]any{"pve_id": float64(101)},
|
||||
wantDefs: 1, wantKind: "vm-status", wantInterv: 60,
|
||||
},
|
||||
{name: "vm-status without pve_id skips", kind: KindVMStatus, wantSkip: true, wantReason: "no pve_id"},
|
||||
|
||||
{
|
||||
name: "dns resolves entity name", kind: KindDNS, target: Target{Name: "hubris.network"},
|
||||
wantDefs: 1, wantKind: "dns", wantKey: "name", wantVal: "hubris.network", wantInterv: 300,
|
||||
},
|
||||
{name: "dns without a name skips", kind: KindDNS, target: Target{}, wantSkip: true, wantReason: "no name"},
|
||||
|
||||
{
|
||||
name: "quorum runs pvecm script via ssh", kind: KindQuorum, host: host,
|
||||
wantDefs: 1, wantKind: "ssh-script", wantKey: "script", wantVal: "pvecm_quorum_check.sh", wantInterv: 60,
|
||||
},
|
||||
{name: "quorum no host skips", kind: KindQuorum, wantSkip: true, wantReason: "no address"},
|
||||
|
||||
{name: "unknown kind skips", kind: "telepathy", host: host, wantSkip: true, wantReason: "no builder"},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
defs, reason := buildKind(c.kind, c.target, c.attrs, c.host, "root", 22)
|
||||
if c.wantSkip {
|
||||
if len(defs) != 0 {
|
||||
t.Fatalf("expected zero defs, got %d", len(defs))
|
||||
}
|
||||
if c.wantReason != "" && !strings.Contains(reason, c.wantReason) {
|
||||
t.Errorf("reason = %q, want substring %q", reason, c.wantReason)
|
||||
}
|
||||
return
|
||||
}
|
||||
if len(defs) != c.wantDefs {
|
||||
t.Fatalf("got %d defs (%s), want %d", len(defs), reason, c.wantDefs)
|
||||
}
|
||||
if reason != "" {
|
||||
t.Errorf("unexpected skip reason: %q", reason)
|
||||
}
|
||||
if c.wantKind != "" {
|
||||
if got := defs[0].kind; got != c.wantKind {
|
||||
t.Errorf("kind = %q, want %q", got, c.wantKind)
|
||||
}
|
||||
}
|
||||
if c.wantKey != "" {
|
||||
if got := defs[0].config[c.wantKey]; !reflect.DeepEqual(got, c.wantVal) {
|
||||
t.Errorf("config[%q] = %v (%T), want %v (%T)", c.wantKey, got, got, c.wantVal, c.wantVal)
|
||||
}
|
||||
}
|
||||
if c.wantInterv != 0 && defs[0].interval != c.wantInterv {
|
||||
t.Errorf("interval = %d, want %d", defs[0].interval, c.wantInterv)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// ssh() must add user/port/args only when they differ from the root/22/empty
|
||||
// defaults, so generated configs stay minimal and stable across re-seeds.
|
||||
func TestBuildKindSSHOnlyEmitsNonDefaultUserPortArgs(t *testing.T) {
|
||||
t.Run("default root 22 omits user and port", func(t *testing.T) {
|
||||
defs, _ := buildKind(KindResource, Target{}, nil, "10.0.0.1", "root", 22)
|
||||
for _, d := range defs {
|
||||
if _, ok := d.config["user"]; ok {
|
||||
t.Errorf("root should not emit user: %v", d.config)
|
||||
}
|
||||
if _, ok := d.config["port"]; ok {
|
||||
t.Errorf("port 22 should not emit port: %v", d.config)
|
||||
}
|
||||
}
|
||||
})
|
||||
t.Run("non-root user and non-22 port are emitted", func(t *testing.T) {
|
||||
defs, _ := buildKind(KindResource, Target{}, nil, "10.0.0.1", "oikos", 2222)
|
||||
if defs[0].config["user"] != "oikos" {
|
||||
t.Errorf("user = %v, want oikos", defs[0].config["user"])
|
||||
}
|
||||
if defs[0].config["port"] != 2222 {
|
||||
t.Errorf("port = %v, want 2222", defs[0].config["port"])
|
||||
}
|
||||
})
|
||||
t.Run("process unit name lands in args", func(t *testing.T) {
|
||||
defs, _ := buildKind(KindProcess, Target{Name: "jellyfin"}, nil, "10.0.0.1", "root", 22)
|
||||
if defs[0].config["args"] != "jellyfin" {
|
||||
t.Errorf("args = %v, want jellyfin", defs[0].config["args"])
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// resolveMonitoringAttr implements the entity-level `monitoring` override
|
||||
// (project decision health_checks.monitoring_override): "none"/"" opts out,
|
||||
// a kind-list replaces the type defaults, anything else falls back.
|
||||
func TestResolveMonitoringAttr(t *testing.T) {
|
||||
fallback := ontology.MonitoringResolution{Declared: true, Kinds: []string{"ping"}, Source: "type"}
|
||||
cases := []struct {
|
||||
name string
|
||||
in any
|
||||
want ontology.MonitoringResolution
|
||||
}{
|
||||
{"none opts out", "none", ontology.MonitoringResolution{Declared: true, Source: "attribute"}},
|
||||
{"empty opts out", "", ontology.MonitoringResolution{Declared: true, Source: "attribute"}},
|
||||
{
|
||||
"kind list overrides",
|
||||
[]any{"http", "process"},
|
||||
ontology.MonitoringResolution{Declared: true, Kinds: []string{"http", "process"}, Source: "attribute"},
|
||||
},
|
||||
{"list drops empty and non-string entries", []any{"http", "", 7, "dns"}, ontology.MonitoringResolution{Declared: true, Kinds: []string{"http", "dns"}, Source: "attribute"}},
|
||||
{"non-string scalar falls back to type default", float64(42), fallback},
|
||||
{"nil falls back", nil, fallback},
|
||||
{"unrecognized string falls back", "weird", fallback},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got := resolveMonitoringAttr(c.in, fallback)
|
||||
if !reflect.DeepEqual(got, c.want) {
|
||||
t.Errorf("resolveMonitoringAttr(%v) = %+v, want %+v", c.in, got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -33,6 +33,8 @@ const (
|
||||
KindBackup = "backup-freshness"
|
||||
KindCertExpiry = "cert-expiry"
|
||||
KindVMStatus = "vm-status"
|
||||
KindQuorum = "quorum"
|
||||
KindDNS = "dns"
|
||||
)
|
||||
|
||||
// defaultBackupMaxAge is how long a backup target may go without a new
|
||||
@@ -306,6 +308,23 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
interval: 60,
|
||||
}}, ""
|
||||
|
||||
case KindDNS:
|
||||
// Resolve the entity's name via DNS to verify the zone is reachable.
|
||||
// Uses the entity name (zone apex) or falls back to the slug.
|
||||
name := t.Name
|
||||
if name == "" {
|
||||
name = strings.TrimPrefix(t.Slug, "zone:")
|
||||
}
|
||||
if name == "" {
|
||||
return nil, "no name to resolve"
|
||||
}
|
||||
return []checkDef{{
|
||||
kind: "dns",
|
||||
config: map[string]any{"name": name},
|
||||
interval: 300, // 5 min — DNS changes are rare; the cost of a miss
|
||||
// is a stale IP, not a service outage.
|
||||
}}, ""
|
||||
|
||||
case KindCertExpiry:
|
||||
// The host whose cert to read (SNI / cert CN). Prefer an explicit
|
||||
// `hostname` attribute, then `cn`, then a dotted name. Hourly: expiry
|
||||
@@ -342,6 +361,15 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
|
||||
config: map[string]any{},
|
||||
interval: 60,
|
||||
}}, ""
|
||||
|
||||
case KindQuorum:
|
||||
// Proxmox cluster quorum via `pvecm status`. Only meaningful on
|
||||
// proxmox-host entities. Runs every 60s — corosync flaps are
|
||||
// transient and the probe is lightweight (local binary, no network).
|
||||
if host == "" {
|
||||
return nil, "no address on the entity or its host"
|
||||
}
|
||||
return []checkDef{ssh("pvecm_quorum_check.sh")}, ""
|
||||
}
|
||||
|
||||
return nil, "no builder for this kind yet"
|
||||
|
||||
@@ -30,6 +30,17 @@ type Config struct {
|
||||
// port than the API); a no-op when the SPA and API share an origin.
|
||||
CORSAllowedOrigin string
|
||||
|
||||
// Rate limiting (plan D3). APIRateLimit is the per-IP requests/sec cap;
|
||||
// APIRateBurst is the token-bucket burst (defaults to 2x the limit when
|
||||
// unset). A limit of 0 disables rate limiting entirely.
|
||||
APIRateLimit int
|
||||
APIRateBurst int
|
||||
|
||||
// Health probe HTTP listener (plan D5). Background-loop roles (scheduler,
|
||||
// notifier) expose a staleness-aware /healthz here. Empty disables the
|
||||
// health server (local/non-docker runs).
|
||||
HealthListen string
|
||||
|
||||
// Observability
|
||||
Debug bool // verbose logging, probe payloads, SQL
|
||||
|
||||
@@ -121,6 +132,11 @@ func FromEnv() Config {
|
||||
if v := os.Getenv("OIKOS_CORS_ORIGIN"); v != "" {
|
||||
c.CORSAllowedOrigin = v
|
||||
}
|
||||
c.APIRateLimit = parseInt(os.Getenv("OIKOS_API_RATE_LIMIT"))
|
||||
c.APIRateBurst = parseInt(os.Getenv("OIKOS_API_RATE_BURST"))
|
||||
if v := os.Getenv("OIKOS_HEALTH_LISTEN"); v != "" {
|
||||
c.HealthListen = v
|
||||
}
|
||||
if v := os.Getenv("OIKOS_SEEDS_DIR"); v != "" {
|
||||
c.SeedsDir = v
|
||||
}
|
||||
|
||||
34
internal/db/checks.go
Normal file
34
internal/db/checks.go
Normal file
@@ -0,0 +1,34 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/dtoro/oikos/internal/checkdefaults"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// EnsureEntityChecks derives an entity's default check_defs from the
|
||||
// monitoring spec of its type (resolving per-entity `monitoring` overrides).
|
||||
//
|
||||
// This is the single shared hook that keeps the check graph in sync with
|
||||
// entity mutations. Both the HTTP create/patch handlers and the MCP
|
||||
// entity-mutation tools (create_entity, update_entity_attributes) call it so
|
||||
// that flipping an entity's `monitoring` attribute regenerates checks
|
||||
// regardless of which surface made the change — previously only the HTTP
|
||||
// path ran check derivation, so entities mutated via MCP silently produced no
|
||||
// checks (see plans/2026-08-03-session-review-haos-monitoring-capability-gaps.md, A2).
|
||||
func EnsureEntityChecks(ctx context.Context, tx pgx.Tx, id uuid.UUID, slug, entityType, name string, attrs []byte) (checkdefaults.Result, error) {
|
||||
tree, err := LoadTypeTree(ctx, tx)
|
||||
if err != nil {
|
||||
return checkdefaults.Result{}, err
|
||||
}
|
||||
res, err := checkdefaults.Ensure(ctx, tx, tree, checkdefaults.Target{
|
||||
ID: id, Slug: slug, Type: entityType, Name: name, Attrs: attrs,
|
||||
})
|
||||
if err != nil {
|
||||
return res, err
|
||||
}
|
||||
checkdefaults.LogResult(slug, entityType, res)
|
||||
return res, nil
|
||||
}
|
||||
61
internal/db/entity_cache.go
Normal file
61
internal/db/entity_cache.go
Normal file
@@ -0,0 +1,61 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type entityCacheEntry struct {
|
||||
slug string
|
||||
id string
|
||||
attrs string
|
||||
exp time.Time
|
||||
}
|
||||
|
||||
type EntityCache struct {
|
||||
mu sync.RWMutex
|
||||
m map[string]entityCacheEntry
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
func NewEntityCache(ttl time.Duration) *EntityCache {
|
||||
return &EntityCache{
|
||||
m: make(map[string]entityCacheEntry),
|
||||
ttl: ttl,
|
||||
}
|
||||
}
|
||||
|
||||
func (c *EntityCache) GetSlug(id string) (string, bool) {
|
||||
c.mu.RLock()
|
||||
e, ok := c.m[id]
|
||||
c.mu.RUnlock()
|
||||
if !ok || time.Now().After(e.exp) {
|
||||
return "", false
|
||||
}
|
||||
return e.slug, true
|
||||
}
|
||||
|
||||
func (c *EntityCache) GetID(slug string) (string, bool) {
|
||||
c.mu.RLock()
|
||||
e, ok := c.m[slug]
|
||||
c.mu.RUnlock()
|
||||
if !ok || time.Now().After(e.exp) {
|
||||
return "", false
|
||||
}
|
||||
return e.id, true
|
||||
}
|
||||
|
||||
func (c *EntityCache) Set(slug, id, attrs string) {
|
||||
exp := time.Now().Add(c.ttl)
|
||||
c.mu.Lock()
|
||||
c.m[slug] = entityCacheEntry{slug: slug, id: id, attrs: attrs, exp: exp}
|
||||
c.m[id] = entityCacheEntry{slug: slug, id: id, attrs: attrs, exp: exp}
|
||||
c.mu.Unlock()
|
||||
}
|
||||
|
||||
func (c *EntityCache) Invalidate(slug, id string) {
|
||||
c.mu.Lock()
|
||||
delete(c.m, slug)
|
||||
delete(c.m, id)
|
||||
c.mu.Unlock()
|
||||
}
|
||||
182
internal/db/lifecycle.go
Normal file
182
internal/db/lifecycle.go
Normal file
@@ -0,0 +1,182 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// ErrTransitionInvalid is a sentinel returned by ValidateTransition when the
|
||||
// from→to pair is not a declared lifecycle transition or a precondition fails.
|
||||
// Callers test with errors.Is to distinguish semantic validation failures
|
||||
// (→ HTTP 409) from infrastructure errors (→ HTTP 500).
|
||||
var ErrTransitionInvalid = errors.New("invalid lifecycle transition")
|
||||
|
||||
// ValidateTransition enforces an entity type's lifecycle: fromState → toState
|
||||
// must be a declared transition, and every precondition it lists must hold. A
|
||||
// type with no lifecycle defined allows any state. A no-op (fromState ==
|
||||
// toState) passes immediately.
|
||||
//
|
||||
// Shared by the HTTP PATCH path and the MCP set_entity_state tool so both
|
||||
// surfaces apply identical lifecycle rules — previously only the HTTP path
|
||||
// validated transitions, so an agent changing state via MCP could skip the
|
||||
// graph's retire/deprecate guardrails entirely.
|
||||
func ValidateTransition(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, entityType, fromState, toState string) error {
|
||||
if toState == fromState {
|
||||
return nil
|
||||
}
|
||||
lc, err := sqlcgen.New(tx).GetLifecycleForType(ctx, entityType)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil // no lifecycle defined → any state allowed
|
||||
}
|
||||
return err
|
||||
}
|
||||
var transitions map[string]map[string]json.RawMessage
|
||||
if err := json.Unmarshal(lc.Transitions, &transitions); err != nil {
|
||||
return fmt.Errorf("parse lifecycle transitions: %w", err)
|
||||
}
|
||||
tos, ok := transitions[fromState]
|
||||
if !ok {
|
||||
return fmt.Errorf("%w: no transitions defined from %q", ErrTransitionInvalid, fromState)
|
||||
}
|
||||
trans, ok := tos[toState]
|
||||
if !ok {
|
||||
return fmt.Errorf("%w: %s → %s is not a declared lifecycle transition", ErrTransitionInvalid, fromState, toState)
|
||||
}
|
||||
var gate struct {
|
||||
Requires []string `json:"requires"`
|
||||
}
|
||||
if err := json.Unmarshal(trans, &gate); err == nil {
|
||||
for _, check := range gate.Requires {
|
||||
if err := checkPrecondition(ctx, tx, entityID, entityType, check); err != nil {
|
||||
return fmt.Errorf("%w: precondition %q not met: %w", ErrTransitionInvalid, check, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkPrecondition evaluates one mechanical precondition named by a lifecycle
|
||||
// transition's `requires` list. Soft/operator-confirmed checks pass; unknown
|
||||
// checks are skipped (operator intent overrides). Moved here from httpapi so
|
||||
// both surfaces share one implementation.
|
||||
func checkPrecondition(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, entityType, check string) error {
|
||||
switch check {
|
||||
case "no-inbound-edges":
|
||||
var count int
|
||||
if err := tx.QueryRow(ctx,
|
||||
"SELECT count(*) FROM relationships WHERE target_id = $1 AND valid_to IS NULL", entityID).Scan(&count); err != nil {
|
||||
return err
|
||||
}
|
||||
if count > 0 {
|
||||
return fmt.Errorf("%d inbound relationship edges remaining", count)
|
||||
}
|
||||
case "backups-verified", "secrets-revoked", "ingress-dns-removed":
|
||||
attrs, err := fetchAttrs(ctx, tx, entityID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
want := map[string]string{
|
||||
"backups-verified": "backups_verified",
|
||||
"secrets-revoked": "secrets_revoked",
|
||||
"ingress-dns-removed": "ingress_dns_removed",
|
||||
}[check]
|
||||
if !attrTruthy(attrs, want) {
|
||||
return fmt.Errorf("%s not recorded in entity attributes", want)
|
||||
}
|
||||
case "age-key-enrolled-if-needed":
|
||||
if entityType == "workstation" {
|
||||
attrs, err := fetchAttrs(ctx, tx, entityID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !attrTruthy(attrs, "age_pubkey") {
|
||||
return fmt.Errorf("age key not enrolled (no age_pubkey in attributes)")
|
||||
}
|
||||
}
|
||||
case "mesh-joined-if-needed":
|
||||
if entityType == "workstation" {
|
||||
attrs, err := fetchAttrs(ctx, tx, entityID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !attrTruthy(attrs, "mesh_ip") {
|
||||
return fmt.Errorf("mesh not joined (no mesh_ip in attributes)")
|
||||
}
|
||||
}
|
||||
case "health-check-answering":
|
||||
st, err := sqlcgen.New(tx).GetEntityStatus(ctx, entityID)
|
||||
if err != nil || st.Health == "unknown" || st.Health == "down" {
|
||||
h := "unknown"
|
||||
if err == nil {
|
||||
h = st.Health
|
||||
}
|
||||
return fmt.Errorf("health check not answering (status: %s)", h)
|
||||
}
|
||||
case "doc-page-complete":
|
||||
var count int
|
||||
if err := tx.QueryRow(ctx, `
|
||||
SELECT count(*) FROM relationships r
|
||||
JOIN entities ke ON ke.id = r.source_id
|
||||
WHERE r.target_id = $1 AND r.valid_to IS NULL
|
||||
AND r.type = 'documents' AND ke.type IN ('document','runbook','investigation')`,
|
||||
entityID).Scan(&count); err != nil {
|
||||
return err
|
||||
}
|
||||
if count == 0 {
|
||||
return fmt.Errorf("no documentation linked to entity")
|
||||
}
|
||||
case "inventory-entry", "ip-reserved", "storage-pool-chosen", "cancelled-note",
|
||||
"preflight-passed", "error-summary", "replacement-live-or-role-retired",
|
||||
"replacement-failed", "post-verify-passed", "recovery-verified", "written-off",
|
||||
"ingress-live-if-public", "doc-page-stub", "un-deprecate-note", "write-off-note":
|
||||
// Soft checks — always pass. Operator-confirmed via the transition
|
||||
// request itself, or not mechanically enforceable.
|
||||
default:
|
||||
// Unknown preconditions are skipped (operator intent overrides).
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// fetchAttrs loads an entity's JSONB attributes column as a decoded map.
|
||||
// Missing attributes decode to an empty map (every key absent).
|
||||
func fetchAttrs(ctx context.Context, tx pgx.Tx, entityID uuid.UUID) (map[string]any, error) {
|
||||
var raw string
|
||||
if err := tx.QueryRow(ctx, "SELECT coalesce(attributes::text,'{}') FROM entities WHERE id = $1", entityID).Scan(&raw); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var attrs map[string]any
|
||||
if err := json.Unmarshal([]byte(raw), &attrs); err != nil {
|
||||
return nil, fmt.Errorf("decode entity attributes: %w", err)
|
||||
}
|
||||
if attrs == nil {
|
||||
attrs = map[string]any{}
|
||||
}
|
||||
return attrs, nil
|
||||
}
|
||||
|
||||
// attrTruthy reports whether key is present in attrs with a meaningful value.
|
||||
// It replaces substring matching on raw JSONB text: a previous strings.Contains
|
||||
// check treated {"backups_verified": false} as satisfied (the key text was
|
||||
// present) and bypassed the attributes GIN index. Booleans must be true;
|
||||
// strings must be non-empty; nil/absent fail.
|
||||
func attrTruthy(attrs map[string]any, key string) bool {
|
||||
v, ok := attrs[key]
|
||||
if !ok || v == nil {
|
||||
return false
|
||||
}
|
||||
switch t := v.(type) {
|
||||
case bool:
|
||||
return t
|
||||
case string:
|
||||
return t != ""
|
||||
default:
|
||||
return true // numbers, objects, arrays count as present
|
||||
}
|
||||
}
|
||||
55
internal/db/lifecycle_test.go
Normal file
55
internal/db/lifecycle_test.go
Normal file
@@ -0,0 +1,55 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// attrTruthy replaces a previous strings.Contains check over raw JSONB text.
|
||||
// The key regression it guards: a literal attribute like
|
||||
// {"backups_verified": false} must NOT satisfy the "backups-verified"
|
||||
// precondition, even though the key text is present in the column.
|
||||
func TestAttrTruthy(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
attrs map[string]any
|
||||
key string
|
||||
want bool
|
||||
}{
|
||||
{"absent", map[string]any{}, "backups_verified", false},
|
||||
{"nil map", nil, "backups_verified", false},
|
||||
{"explicit nil value", map[string]any{"backups_verified": nil}, "backups_verified", false},
|
||||
{"bool true", map[string]any{"backups_verified": true}, "backups_verified", true},
|
||||
{"bool false is the regression case", map[string]any{"backups_verified": false}, "backups_verified", false},
|
||||
{"nonempty string age pubkey", map[string]any{"age_pubkey": "age1abc"}, "age_pubkey", true},
|
||||
{"empty string is falsy", map[string]any{"mesh_ip": ""}, "mesh_ip", false},
|
||||
{"number counts as present", map[string]any{"port": float64(22)}, "port", true},
|
||||
{"other keys present", map[string]any{"backups_verified": true, "unrelated": "x"}, "backups_verified", true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if got := attrTruthy(tc.attrs, tc.key); got != tc.want {
|
||||
t.Fatalf("attrTruthy(%v, %q) = %v, want %v", tc.attrs, tc.key, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// fetchAttrs decodes the JSONB column text; verify the decode shape that
|
||||
// attrTruthy then evaluates (the DB round-trip itself is covered by make test-db).
|
||||
func TestAttrTruthyAfterDecode(t *testing.T) {
|
||||
raw := `{"backups_verified": true, "mesh_ip": "10.0.0.5", "secrets_revoked": false}`
|
||||
var got map[string]any
|
||||
if err := json.Unmarshal([]byte(raw), &got); err != nil {
|
||||
t.Fatalf("unmarshal: %v", err)
|
||||
}
|
||||
if !attrTruthy(got, "backups_verified") {
|
||||
t.Error("backups_verified should be truthy after decode")
|
||||
}
|
||||
if !attrTruthy(got, "mesh_ip") {
|
||||
t.Error("mesh_ip should be truthy after decode")
|
||||
}
|
||||
if attrTruthy(got, "secrets_revoked") {
|
||||
t.Error("secrets_revoked:false is the regression — must be falsy")
|
||||
}
|
||||
}
|
||||
@@ -190,7 +190,10 @@ func hasSuffix(s, suffix string) bool {
|
||||
}
|
||||
|
||||
// splitSQL splits a SQL string into individual statements.
|
||||
// Handles $$ ... $$ dollar-quoted blocks and -- line comments.
|
||||
// Handles $$ ... $$ dollar-quoted blocks, $tag$ ... $tag$ tagged quotes,
|
||||
// -- line comments, /* ... */ block comments, and '...' string literals
|
||||
// so that semicolons inside any of these constructs are not treated as
|
||||
// statement boundaries.
|
||||
func splitSQL(sql string) []string {
|
||||
var statements []string
|
||||
var current strings.Builder
|
||||
@@ -201,7 +204,6 @@ func splitSQL(sql string) []string {
|
||||
for i < len(sql) {
|
||||
// Handle line comments (-- to end of line)
|
||||
if !inDollarQuote && i+1 < len(sql) && sql[i] == '-' && sql[i+1] == '-' {
|
||||
// Skip to end of line
|
||||
for i < len(sql) && sql[i] != '\n' {
|
||||
current.WriteByte(sql[i])
|
||||
i++
|
||||
@@ -209,6 +211,34 @@ func splitSQL(sql string) []string {
|
||||
continue
|
||||
}
|
||||
|
||||
// Handle block comments (/* ... */)
|
||||
if !inDollarQuote && i+1 < len(sql) && sql[i] == '/' && sql[i+1] == '*' {
|
||||
end := strings.Index(sql[i+2:], "*/")
|
||||
if end >= 0 {
|
||||
current.WriteString(sql[i : i+end+4])
|
||||
i += end + 4
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
// Handle single-quoted string literals ('...')
|
||||
if !inDollarQuote && sql[i] == '\'' {
|
||||
j := i + 1
|
||||
for j < len(sql) {
|
||||
if sql[j] == '\'' {
|
||||
if j+1 < len(sql) && sql[j+1] == '\'' {
|
||||
j += 2 // skip doubled quote ''
|
||||
continue
|
||||
}
|
||||
break
|
||||
}
|
||||
j++
|
||||
}
|
||||
current.WriteString(sql[i : j+1])
|
||||
i = j + 1
|
||||
continue
|
||||
}
|
||||
|
||||
// Check for dollar-quote start/end
|
||||
if !inDollarQuote && sql[i] == '$' {
|
||||
j := i + 1
|
||||
|
||||
@@ -44,6 +44,36 @@ UPDATE entities SET
|
||||
WHERE id = sqlc.arg('id') AND version = sqlc.arg('version')
|
||||
RETURNING *;
|
||||
|
||||
-- name: MergeEntityAttributes :execrows
|
||||
-- Shallow-merge a JSON patch into an entity's attributes (the
|
||||
-- update_entity_attributes MCP/HTTP surface). Replaces the raw
|
||||
-- `attributes = attributes || $2::jsonb` used in entity_tools.go.
|
||||
UPDATE entities SET
|
||||
attributes = attributes || sqlc.arg('patch')::jsonb,
|
||||
updated_at = now()
|
||||
WHERE slug = sqlc.arg('slug');
|
||||
|
||||
-- name: SetEntityState :execrows
|
||||
-- Set an entity's lifecycle state by id (the set_entity_state surface, run
|
||||
-- after db.ValidateTransition). Replaces the raw
|
||||
-- `UPDATE entities SET state = $2 ... WHERE id = $1`.
|
||||
UPDATE entities SET
|
||||
state = sqlc.arg('state'),
|
||||
updated_at = now()
|
||||
WHERE id = sqlc.arg('id');
|
||||
|
||||
-- blast_radius(): the recursive-CTE traversal function's TABLE return type
|
||||
-- is opaque to sqlc's analyzer — that one query stays hand-written pgx in
|
||||
-- internal/httpapi (see impl.go).
|
||||
-- internal/httpapi (see entities.go GetBlastRadius).
|
||||
--
|
||||
-- Deliberate raw-SQL exceptions (plan E2): the httpapi entity *read* handlers
|
||||
-- (ListEntities/GetEntity/GetGraph/queryEntities) project a fixed
|
||||
-- `entityCols` column set (entities.* + a LEFT JOIN to entity_status for
|
||||
-- health/last_check_at) and scan it positionally into the oapi-generated
|
||||
-- gen.Entity shape. sqlc generates its own row struct per query and cannot
|
||||
-- emit gen.Entity, so migrating those reads would add a per-call field-by-
|
||||
-- field mapping with no compile-time gain and real column-order risk. They
|
||||
-- stay hand-written pgx, like blast_radius and the seed/export bulk paths
|
||||
-- noted in sqlc.yaml. The mutation/relationship surface (MergeEntityAttributes,
|
||||
-- SetEntityState, InsertRelationshipIfAbsent, EndCurrentRelationship) IS
|
||||
-- migrated and is what the entity CRUD tools now call.
|
||||
|
||||
@@ -25,8 +25,8 @@ ON CONFLICT (actor, key) DO NOTHING;
|
||||
|
||||
-- name: InsertAuditEntry :exec
|
||||
INSERT INTO audit_log (actor_type, actor_id, action, entity_id, method, path,
|
||||
status_code, detail, source_ip, correlation_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10);
|
||||
status_code, detail, source_ip, correlation_id, session_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11);
|
||||
|
||||
-- name: InsertEvent :one
|
||||
INSERT INTO events (type, entity_id, severity, source, data, correlation_id)
|
||||
|
||||
@@ -25,3 +25,18 @@ ORDER BY r.type, se.slug, te.slug;
|
||||
-- name: EndCurrentRelationship :execrows
|
||||
UPDATE relationships SET valid_to = now()
|
||||
WHERE source_id = $1 AND target_id = $2 AND type = $3 AND valid_to IS NULL;
|
||||
|
||||
-- name: InsertRelationshipIfAbsent :execrows
|
||||
-- Idempotent relationship insert (the create_relationship surface): no-op if
|
||||
-- an active edge of the same source/target/type already exists. Replaces the
|
||||
-- raw INSERT...WHERE NOT EXISTS used in entity_tools.go.
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT sqlc.arg('source_id'), sqlc.arg('target_id'), sqlc.arg('type'),
|
||||
sqlc.arg('attributes')::jsonb, now()
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM relationships
|
||||
WHERE source_id = sqlc.arg('source_id')
|
||||
AND target_id = sqlc.arg('target_id')
|
||||
AND type = sqlc.arg('type')
|
||||
AND valid_to IS NULL
|
||||
);
|
||||
|
||||
@@ -51,3 +51,57 @@ func TestSplitSQLSemicolonInComment(t *testing.T) {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLSemicolonInStringLiteral(t *testing.T) {
|
||||
sql := `SELECT 'hello; world'; INSERT INTO t VALUES (1);`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLDollarSignInStringLiteral(t *testing.T) {
|
||||
sql := `SELECT '$100'; SELECT 2;`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLBlockComment(t *testing.T) {
|
||||
sql := `SELECT 1; /* block; with; semicolons */ SELECT 2;`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLBlockCommentWithDollarQuote(t *testing.T) {
|
||||
sql := `/* $$ not a dollar quote */ SELECT 1;`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 1 {
|
||||
t.Fatalf("got %d statements, want 1: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLDoubledQuoteInString(t *testing.T) {
|
||||
sql := `SELECT 'O''Brien'; SELECT 2;`
|
||||
stmts := nonEmpty(splitSQL(sql))
|
||||
if len(stmts) != 2 {
|
||||
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLEmptyInput(t *testing.T) {
|
||||
stmts := nonEmpty(splitSQL(""))
|
||||
if len(stmts) != 0 {
|
||||
t.Fatalf("got %d statements, want 0", len(stmts))
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitSQLNoSemicolon(t *testing.T) {
|
||||
stmts := nonEmpty(splitSQL("SELECT 1"))
|
||||
if len(stmts) != 1 {
|
||||
t.Fatalf("got %d statements, want 1", len(stmts))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -177,6 +177,52 @@ func (q *Queries) ListEntities(ctx context.Context, arg ListEntitiesParams) ([]E
|
||||
return items, nil
|
||||
}
|
||||
|
||||
const mergeEntityAttributes = `-- name: MergeEntityAttributes :execrows
|
||||
UPDATE entities SET
|
||||
attributes = attributes || $1::jsonb,
|
||||
updated_at = now()
|
||||
WHERE slug = $2
|
||||
`
|
||||
|
||||
type MergeEntityAttributesParams struct {
|
||||
Patch []byte
|
||||
Slug string
|
||||
}
|
||||
|
||||
// Shallow-merge a JSON patch into an entity's attributes (the
|
||||
// update_entity_attributes MCP/HTTP surface). Replaces the raw
|
||||
// `attributes = attributes || $2::jsonb` used in entity_tools.go.
|
||||
func (q *Queries) MergeEntityAttributes(ctx context.Context, arg MergeEntityAttributesParams) (int64, error) {
|
||||
result, err := q.db.Exec(ctx, mergeEntityAttributes, arg.Patch, arg.Slug)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
|
||||
const setEntityState = `-- name: SetEntityState :execrows
|
||||
UPDATE entities SET
|
||||
state = $1,
|
||||
updated_at = now()
|
||||
WHERE id = $2
|
||||
`
|
||||
|
||||
type SetEntityStateParams struct {
|
||||
State *string
|
||||
ID uuid.UUID
|
||||
}
|
||||
|
||||
// Set an entity's lifecycle state by id (the set_entity_state surface, run
|
||||
// after db.ValidateTransition). Replaces the raw
|
||||
// `UPDATE entities SET state = $2 ... WHERE id = $1`.
|
||||
func (q *Queries) SetEntityState(ctx context.Context, arg SetEntityStateParams) (int64, error) {
|
||||
result, err := q.db.Exec(ctx, setEntityState, arg.State, arg.ID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
|
||||
const updateEntity = `-- name: UpdateEntity :one
|
||||
UPDATE entities SET
|
||||
name = COALESCE($1, name),
|
||||
|
||||
@@ -380,6 +380,8 @@ type RelationshipType struct {
|
||||
Cardinality string
|
||||
Description *string
|
||||
CreatedAt time.Time
|
||||
// Which end of this edge depends on the other. forward = target depends on source. backward = source depends on target. none = no runtime dependency. Drives blast_radius().
|
||||
BlastDirection string
|
||||
}
|
||||
|
||||
type RiskClass struct {
|
||||
@@ -396,18 +398,19 @@ type SeedVersion struct {
|
||||
}
|
||||
|
||||
type SessionPlanStep struct {
|
||||
ID uuid.UUID
|
||||
SessionID uuid.UUID
|
||||
Seq int32
|
||||
Title string
|
||||
Detail string
|
||||
Status string
|
||||
ExecutionID *uuid.UUID
|
||||
TargetSlug *string
|
||||
StartedAt *time.Time
|
||||
FinishedAt *time.Time
|
||||
CreatedAt time.Time
|
||||
Generation int32
|
||||
ID uuid.UUID
|
||||
SessionID uuid.UUID
|
||||
Seq int32
|
||||
Title string
|
||||
Detail string
|
||||
Status string
|
||||
ExecutionID *uuid.UUID
|
||||
TargetSlug *string
|
||||
StartedAt *time.Time
|
||||
FinishedAt *time.Time
|
||||
CreatedAt time.Time
|
||||
Generation int32
|
||||
ReplacedReason *string
|
||||
}
|
||||
|
||||
type SessionQuestion struct {
|
||||
|
||||
@@ -99,7 +99,7 @@ func (q *Queries) ListLifecycleDefs(ctx context.Context) ([]LifecycleDef, error)
|
||||
}
|
||||
|
||||
const listRelationshipTypes = `-- name: ListRelationshipTypes :many
|
||||
SELECT name, inverse, source_type, target_type, cardinality, description, created_at FROM relationship_types ORDER BY name
|
||||
SELECT name, inverse, source_type, target_type, cardinality, description, created_at, blast_direction FROM relationship_types ORDER BY name
|
||||
`
|
||||
|
||||
func (q *Queries) ListRelationshipTypes(ctx context.Context) ([]RelationshipType, error) {
|
||||
@@ -119,6 +119,7 @@ func (q *Queries) ListRelationshipTypes(ctx context.Context) ([]RelationshipType
|
||||
&i.Cardinality,
|
||||
&i.Description,
|
||||
&i.CreatedAt,
|
||||
&i.BlastDirection,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -353,8 +353,8 @@ func (q *Queries) InsertApproval(ctx context.Context, arg InsertApprovalParams)
|
||||
|
||||
const insertAuditEntry = `-- name: InsertAuditEntry :exec
|
||||
INSERT INTO audit_log (actor_type, actor_id, action, entity_id, method, path,
|
||||
status_code, detail, source_ip, correlation_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
|
||||
status_code, detail, source_ip, correlation_id, session_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)
|
||||
`
|
||||
|
||||
type InsertAuditEntryParams struct {
|
||||
@@ -368,6 +368,7 @@ type InsertAuditEntryParams struct {
|
||||
Detail []byte
|
||||
SourceIp *string
|
||||
CorrelationID *string
|
||||
SessionID *uuid.UUID
|
||||
}
|
||||
|
||||
func (q *Queries) InsertAuditEntry(ctx context.Context, arg InsertAuditEntryParams) error {
|
||||
@@ -382,6 +383,7 @@ func (q *Queries) InsertAuditEntry(ctx context.Context, arg InsertAuditEntryPara
|
||||
arg.Detail,
|
||||
arg.SourceIp,
|
||||
arg.CorrelationID,
|
||||
arg.SessionID,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -31,6 +31,42 @@ func (q *Queries) EndCurrentRelationship(ctx context.Context, arg EndCurrentRela
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
|
||||
const insertRelationshipIfAbsent = `-- name: InsertRelationshipIfAbsent :execrows
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT $1, $2, $3,
|
||||
$4::jsonb, now()
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM relationships
|
||||
WHERE source_id = $1
|
||||
AND target_id = $2
|
||||
AND type = $3
|
||||
AND valid_to IS NULL
|
||||
)
|
||||
`
|
||||
|
||||
type InsertRelationshipIfAbsentParams struct {
|
||||
SourceID uuid.UUID
|
||||
TargetID uuid.UUID
|
||||
Type string
|
||||
Attributes []byte
|
||||
}
|
||||
|
||||
// Idempotent relationship insert (the create_relationship surface): no-op if
|
||||
// an active edge of the same source/target/type already exists. Replaces the
|
||||
// raw INSERT...WHERE NOT EXISTS used in entity_tools.go.
|
||||
func (q *Queries) InsertRelationshipIfAbsent(ctx context.Context, arg InsertRelationshipIfAbsentParams) (int64, error) {
|
||||
result, err := q.db.Exec(ctx, insertRelationshipIfAbsent,
|
||||
arg.SourceID,
|
||||
arg.TargetID,
|
||||
arg.Type,
|
||||
arg.Attributes,
|
||||
)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
|
||||
const listEntityRelations = `-- name: ListEntityRelations :many
|
||||
SELECT se.slug AS source_slug, te.slug AS target_slug, r.type, r.attributes,
|
||||
r.valid_from, r.valid_to
|
||||
|
||||
13
internal/execworker/init.go
Normal file
13
internal/execworker/init.go
Normal file
@@ -0,0 +1,13 @@
|
||||
package execworker
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
)
|
||||
|
||||
// RunnerForMain provides the run function for registration in main.
|
||||
func RunnerForMain() func(context.Context, *db.Pool, config.Config) {
|
||||
return Run
|
||||
}
|
||||
207
internal/execworker/worker.go
Normal file
207
internal/execworker/worker.go
Normal file
@@ -0,0 +1,207 @@
|
||||
// Package execworker processes pending executions as a background daemon.
|
||||
// This provides a Postgres-backed queue: executions survive restarts, and
|
||||
// per-execution advisory locks prevent duplicate processing across instances.
|
||||
package execworker
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/actuator"
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/health"
|
||||
"github.com/dtoro/oikos/internal/remote"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// Run starts the execution worker loop. Blocks until ctx is cancelled.
|
||||
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
|
||||
slog.Info("execworker: starting")
|
||||
|
||||
// Liveness probe
|
||||
probe := health.New(2 * time.Minute)
|
||||
probe.Serve(ctx, cfg.HealthListen)
|
||||
|
||||
recoverOrphaned(ctx, pool)
|
||||
probe.Bump()
|
||||
|
||||
ticker := time.NewTicker(15 * time.Second)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
slog.Info("execworker: shutting down")
|
||||
return
|
||||
case <-ticker.C:
|
||||
processPending(ctx, pool)
|
||||
probe.Bump()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// recoverOrphaned marks executions stuck in 'running' as failed.
|
||||
func recoverOrphaned(ctx context.Context, pool *db.Pool) {
|
||||
tag, err := pool.Exec(ctx, `UPDATE executions SET status = 'failed', result = '{"error":"worker restarted while execution was running"}'::jsonb, completed_at = now() WHERE status = 'running'`)
|
||||
if err != nil {
|
||||
slog.Error("execworker: recover orphaned", "error", err)
|
||||
return
|
||||
}
|
||||
if tag.RowsAffected() > 0 {
|
||||
slog.Warn("execworker: recovered orphaned executions", "count", tag.RowsAffected())
|
||||
}
|
||||
}
|
||||
|
||||
// processPending polls for pending executions and dispatches them.
|
||||
func processPending(ctx context.Context, pool *db.Pool) {
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT e.entity_id, e.target_entity_id, e.action, e.risk_class, e.correlation_id, e.status,
|
||||
COALESCE(t.slug, '') AS target_slug
|
||||
FROM executions e
|
||||
LEFT JOIN entities t ON t.id = e.target_entity_id
|
||||
WHERE e.status = 'proposed'
|
||||
ORDER BY e.created_at ASC
|
||||
LIMIT 10`)
|
||||
if err != nil {
|
||||
slog.Error("execworker: query pending", "error", err)
|
||||
return
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
q := sqlcgen.New(pool)
|
||||
|
||||
for rows.Next() {
|
||||
var execID, targetID *uuid.UUID
|
||||
var action, riskClass, correlationID, status, targetSlug string
|
||||
if err := rows.Scan(&execID, &targetID, &action, &riskClass, &correlationID, &status, &targetSlug); err != nil {
|
||||
slog.Error("execworker: scan row", "error", err)
|
||||
continue
|
||||
}
|
||||
if execID == nil {
|
||||
continue
|
||||
}
|
||||
|
||||
// At-most-once: try advisory lock on execution entity_id.
|
||||
// Acquire a dedicated connection so the session-scoped lock isn't
|
||||
// released when the transient pool connection is returned.
|
||||
lockKey := hashUUID(*execID)
|
||||
lockConn, err := pool.Acquire(ctx)
|
||||
if err != nil {
|
||||
slog.Error("execworker: acquire lock conn", "error", err)
|
||||
continue
|
||||
}
|
||||
var locked bool
|
||||
if err := lockConn.QueryRow(ctx, "SELECT pg_try_advisory_lock($1)", lockKey).Scan(&locked); err != nil || !locked {
|
||||
lockConn.Release()
|
||||
continue
|
||||
}
|
||||
|
||||
dispatch(ctx, pool, q, *execID, targetID, action, targetSlug, correlationID)
|
||||
|
||||
// Release the per-execution lock on the same connection.
|
||||
lockConn.Exec(context.WithoutCancel(ctx), "SELECT pg_advisory_unlock($1)", lockKey)
|
||||
lockConn.Release()
|
||||
}
|
||||
}
|
||||
|
||||
func dispatch(ctx context.Context, pool *db.Pool, q *sqlcgen.Queries, execID uuid.UUID, targetID *uuid.UUID, action, targetSlug, correlationID string) {
|
||||
startedAt := time.Now()
|
||||
|
||||
// Mark running
|
||||
_, err := pool.Exec(ctx,
|
||||
`UPDATE executions SET status = 'running', started_at = $2 WHERE entity_id = $1`,
|
||||
execID, startedAt)
|
||||
if err != nil {
|
||||
slog.Error("execworker: mark running", "error", err, "execution", execID)
|
||||
return
|
||||
}
|
||||
|
||||
// Resolve SSH target. If targetSlug is available, use it; otherwise resolve from targetID.
|
||||
var host, user string
|
||||
if targetSlug == "" && targetID != nil {
|
||||
if err := pool.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", *targetID).Scan(&targetSlug); err != nil {
|
||||
failExecution(ctx, pool, execID, fmt.Sprintf("resolve target slug: %v", err))
|
||||
return
|
||||
}
|
||||
}
|
||||
if targetSlug != "" {
|
||||
addr, sshUser, err := remote.ResolveHost(ctx, pool, targetSlug, "root")
|
||||
if err == nil {
|
||||
host, user = addr, sshUser
|
||||
}
|
||||
}
|
||||
if host == "" {
|
||||
failExecution(ctx, pool, execID, fmt.Sprintf("no reachable target: %s", targetSlug))
|
||||
return
|
||||
}
|
||||
|
||||
// Determine the command to run from the action field.
|
||||
// Format: "action_name:{json_params}" or a raw command string.
|
||||
cmd := action
|
||||
if idx := strings.Index(action, ":"); idx > 0 && idx < len(action)-1 {
|
||||
rawParams := action[idx+1:]
|
||||
var params map[string]any
|
||||
if json.Unmarshal([]byte(rawParams), ¶ms) == nil {
|
||||
if c, ok := params["command"].(string); ok && c != "" {
|
||||
cmd = c
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
signer, err := actuator.LoadSigner(os.Getenv("OIKOS_SSH_KEY_PATH"))
|
||||
if err != nil {
|
||||
signer, err = actuator.LoadSigner("/etc/oikos/ssh_key")
|
||||
if err != nil {
|
||||
failExecution(ctx, pool, execID, fmt.Sprintf("load ssh key: %v", err))
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
client, err := actuator.Dial(ctx, actuator.DialOptions{Host: host, User: user, Signer: signer})
|
||||
if err != nil {
|
||||
failExecution(ctx, pool, execID, fmt.Sprintf("ssh dial: %v", err))
|
||||
return
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
out, err := actuator.RunCombinedOutput(ctx, client, cmd)
|
||||
if err != nil {
|
||||
failExecution(ctx, pool, execID, fmt.Sprintf("command: %v\noutput: %s", err, string(out)))
|
||||
return
|
||||
}
|
||||
|
||||
duration := time.Since(startedAt).Milliseconds()
|
||||
resultJSON, _ := json.Marshal(map[string]any{"output": string(out), "success": true})
|
||||
_ = q.UpdateExecutionStatus(ctx, sqlcgen.UpdateExecutionStatusParams{
|
||||
EntityID: execID,
|
||||
Status: "completed",
|
||||
Result: resultJSON,
|
||||
DurationMs: &[]int32{int32(duration)}[0],
|
||||
Verified: true,
|
||||
})
|
||||
|
||||
slog.Info("execworker: execution complete",
|
||||
"execution", execID, "target", targetSlug, "duration_ms", duration)
|
||||
}
|
||||
|
||||
func failExecution(ctx context.Context, pool *db.Pool, execID uuid.UUID, reason string) {
|
||||
slog.Error("execworker: execution failed", "execution", execID, "error", reason)
|
||||
resultJSON, _ := json.Marshal(map[string]any{"error": reason, "success": false})
|
||||
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb, completed_at=now() WHERE entity_id=$1`,
|
||||
execID, resultJSON)
|
||||
}
|
||||
|
||||
func hashUUID(id uuid.UUID) int {
|
||||
h := 0
|
||||
for _, b := range id {
|
||||
h = (h*31 + int(b)) & 0x7fffffff
|
||||
}
|
||||
return h
|
||||
}
|
||||
93
internal/health/health.go
Normal file
93
internal/health/health.go
Normal file
@@ -0,0 +1,93 @@
|
||||
// Package health provides a staleness-aware liveness probe for background-
|
||||
// loop services (scheduler, notifier) that don't otherwise serve HTTP.
|
||||
//
|
||||
// The owning loop calls Probe.Bump() on each iteration. A /healthz endpoint
|
||||
// returns 200 while the last bump is within the staleness window, and 503
|
||||
// once the loop has gone quiet — so a wedged goroutine (stuck SSH, deadlock)
|
||||
// surfaces as an unhealthy container instead of a silently-idle one.
|
||||
package health
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Probe tracks the last time the owning loop made progress.
|
||||
type Probe struct {
|
||||
last atomic.Int64 // unix-nano timestamp of the last Bump
|
||||
stale time.Duration
|
||||
}
|
||||
|
||||
// New returns a Probe that considers the owner healthy while Bump has been
|
||||
// called within stale of the current time.
|
||||
func New(stale time.Duration) *Probe {
|
||||
if stale <= 0 {
|
||||
stale = 2 * time.Minute
|
||||
}
|
||||
p := &Probe{stale: stale}
|
||||
p.last.Store(time.Now().UnixNano()) // boot-healthy until first loop stalls
|
||||
return p
|
||||
}
|
||||
|
||||
// Bump records that the owning loop completed another iteration.
|
||||
func (p *Probe) Bump() {
|
||||
p.last.Store(time.Now().UnixNano())
|
||||
}
|
||||
|
||||
// Healthy reports whether the last Bump is within the staleness window.
|
||||
func (p *Probe) Healthy() bool {
|
||||
last := time.Unix(0, p.last.Load())
|
||||
return time.Since(last) <= p.stale
|
||||
}
|
||||
|
||||
// Handler returns an http.Handler serving GET /healthz. Returns 200 with a
|
||||
// small JSON body when healthy, 503 (Service Unavailable) when stale.
|
||||
func (p *Probe) Handler() http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
last := time.Unix(0, p.last.Load())
|
||||
if !p.Healthy() {
|
||||
w.WriteHeader(http.StatusServiceUnavailable)
|
||||
}
|
||||
json.NewEncoder(w).Encode(map[string]any{
|
||||
"status": healthStatus(p.Healthy()),
|
||||
"last_heartbeat": last.UTC().Format(time.RFC3339),
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
// Serve starts an HTTP server exposing the probe's /healthz on addr until ctx
|
||||
// is cancelled. A no-op when addr is empty (local/non-docker runs skip it).
|
||||
// The server is bound to addr (e.g. ":8093"); containers hit it via 127.0.0.1.
|
||||
func (p *Probe) Serve(ctx context.Context, addr string) {
|
||||
if addr == "" {
|
||||
return
|
||||
}
|
||||
mux := http.NewServeMux()
|
||||
mux.Handle("/healthz", p.Handler())
|
||||
srv := &http.Server{Addr: addr, Handler: mux, ReadHeaderTimeout: 5 * time.Second}
|
||||
|
||||
go func() {
|
||||
slog.Info("health server listening", "addr", addr)
|
||||
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
slog.Warn("health server stopped", "addr", addr, "error", err)
|
||||
}
|
||||
}()
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
srv.Shutdown(shutdownCtx)
|
||||
}()
|
||||
}
|
||||
|
||||
func healthStatus(ok bool) string {
|
||||
if ok {
|
||||
return "ok"
|
||||
}
|
||||
return "stale"
|
||||
}
|
||||
73
internal/health/health_test.go
Normal file
73
internal/health/health_test.go
Normal file
@@ -0,0 +1,73 @@
|
||||
package health
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestProbeHealthyAtBoot(t *testing.T) {
|
||||
p := New(time.Minute)
|
||||
if !p.Healthy() {
|
||||
t.Fatal("probe should be healthy immediately after creation")
|
||||
}
|
||||
}
|
||||
|
||||
func TestProbeStaleAfterWindow(t *testing.T) {
|
||||
p := New(50 * time.Millisecond)
|
||||
time.Sleep(80 * time.Millisecond)
|
||||
if p.Healthy() {
|
||||
t.Fatal("probe should be stale after the staleness window elapses with no Bump")
|
||||
}
|
||||
p.Bump()
|
||||
if !p.Healthy() {
|
||||
t.Fatal("probe should recover immediately after Bump")
|
||||
}
|
||||
}
|
||||
|
||||
func TestProbeHandlerStatusCodes(t *testing.T) {
|
||||
p := New(20 * time.Millisecond)
|
||||
|
||||
// Fresh → 200
|
||||
if code := probeCode(p); code != http.StatusOK {
|
||||
t.Fatalf("fresh probe: want 200, got %d", code)
|
||||
}
|
||||
|
||||
// Stale → 503
|
||||
time.Sleep(40 * time.Millisecond)
|
||||
if code := probeCode(p); code != http.StatusServiceUnavailable {
|
||||
t.Fatalf("stale probe: want 503, got %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProbeHandlerBody(t *testing.T) {
|
||||
p := New(time.Minute)
|
||||
rec := httptest.NewRecorder()
|
||||
p.Handler().ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/healthz", nil))
|
||||
|
||||
var body map[string]any
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
|
||||
t.Fatalf("invalid JSON body: %v (body=%q)", err, rec.Body.String())
|
||||
}
|
||||
if body["status"] != "ok" {
|
||||
t.Fatalf("want status=ok, got %v", body["status"])
|
||||
}
|
||||
if _, ok := body["last_heartbeat"].(string); !ok {
|
||||
t.Fatalf("want last_heartbeat string, got %v", body["last_heartbeat"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewDefaultsStale(t *testing.T) {
|
||||
p := New(0)
|
||||
if p.stale <= 0 {
|
||||
t.Fatal("New(0) should fall back to a positive staleness window")
|
||||
}
|
||||
}
|
||||
|
||||
func probeCode(p *Probe) int {
|
||||
rec := httptest.NewRecorder()
|
||||
p.Handler().ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/healthz", nil))
|
||||
return rec.Code
|
||||
}
|
||||
@@ -1,7 +1,6 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
@@ -10,15 +9,14 @@ import (
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/actuator"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/execlog"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
var (
|
||||
@@ -77,24 +75,8 @@ const sshExecTimeout = 10 * time.Minute
|
||||
// streamWriter buffers everything it is given while forwarding each write to a
|
||||
// sink. One on session.Stdout and another sharing the same buffer on
|
||||
// session.Stderr reproduces CombinedOutput's interleaving in the order the
|
||||
// remote end produced it. Mirrors the twin in internal/mcp/server.go.
|
||||
type streamWriter struct {
|
||||
mu *sync.Mutex
|
||||
buf *bytes.Buffer
|
||||
stream string
|
||||
sink execlog.Sink
|
||||
}
|
||||
|
||||
func (w *streamWriter) Write(p []byte) (int, error) {
|
||||
w.mu.Lock()
|
||||
w.buf.Write(p)
|
||||
w.mu.Unlock()
|
||||
if w.sink != nil {
|
||||
// Copy: the ssh library reuses p once Write returns.
|
||||
w.sink(w.stream, append([]byte(nil), p...))
|
||||
}
|
||||
return len(p), nil
|
||||
}
|
||||
// remote end produced it. Shared implementation lives in internal/actuator
|
||||
// (actuator.streamWriter / actuator.RunStreaming).
|
||||
|
||||
func sshExec(ctx context.Context, host, user, command string) (string, error) {
|
||||
return sshExecStream(ctx, host, user, command, nil)
|
||||
@@ -111,88 +93,18 @@ func sshExecStream(ctx context.Context, host, user, command string, sink execlog
|
||||
user = _sshUser
|
||||
}
|
||||
|
||||
addr := host + ":22"
|
||||
signer, err := ssh.ParsePrivateKey(_sshKey)
|
||||
signer, err := actuator.LoadSignerFromBytes(_sshKey)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("parse key: %w", err)
|
||||
}
|
||||
|
||||
cfg := &ssh.ClientConfig{
|
||||
User: user,
|
||||
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
|
||||
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
|
||||
Timeout: 10 * time.Second,
|
||||
}
|
||||
|
||||
client, err := ssh.Dial("tcp", addr, cfg)
|
||||
client, err := actuator.Dial(ctx, actuator.DialOptions{Host: host, User: user, Signer: signer})
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("dial %s: %w", host, err)
|
||||
return "", err
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
var (
|
||||
mu sync.Mutex
|
||||
buf bytes.Buffer
|
||||
)
|
||||
session.Stdout = &streamWriter{mu: &mu, buf: &buf, stream: "stdout", sink: sink}
|
||||
session.Stderr = &streamWriter{mu: &mu, buf: &buf, stream: "stderr", sink: sink}
|
||||
|
||||
collected := func() string {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return strings.TrimSpace(buf.String())
|
||||
}
|
||||
|
||||
done := make(chan error, 1)
|
||||
go func() {
|
||||
// See internal/mcp/server.go's sshExec for why this recovers rather
|
||||
// than letting a rare SSH-library panic crash the whole api process.
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
done <- fmt.Errorf("panic in ssh exec: %v", r)
|
||||
}
|
||||
}()
|
||||
// Run rather than CombinedOutput so the assigned writers are used;
|
||||
// Run returns only after both streams are fully drained.
|
||||
done <- session.Run(command)
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-done:
|
||||
text := collected()
|
||||
// A non-zero exit MUST surface as an error. The previous guard only
|
||||
// errored when there was no output, so a `pct create` that printed
|
||||
// "CT 132 already exists" and exited non-zero was reported as
|
||||
// success — the execution was marked completed though nothing was
|
||||
// provisioned.
|
||||
if err != nil {
|
||||
if text != "" {
|
||||
return text, fmt.Errorf("%w: %s", err, text)
|
||||
}
|
||||
return text, fmt.Errorf("exec: %w", err)
|
||||
}
|
||||
return text, nil
|
||||
case <-time.After(sshExecTimeout):
|
||||
// Close the session/client to hang up the remote side; the
|
||||
// goroutine above will eventually exit once that unblocks Run, but we
|
||||
// don't wait for it — the caller needs an answer now, not an
|
||||
// indefinite hang.
|
||||
session.Close()
|
||||
client.Close()
|
||||
// Return what arrived before it hung, rather than "". A provisioning
|
||||
// command that stalls halfway is precisely when its output matters.
|
||||
return collected(), fmt.Errorf("timed out after %s waiting for command to finish on %s", sshExecTimeout, host)
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
client.Close()
|
||||
return collected(), ctx.Err()
|
||||
}
|
||||
return actuator.RunStreaming(ctx, client, command, sink, sshExecTimeout)
|
||||
}
|
||||
|
||||
func resolveHostSSH(ctx context.Context, pool *db.Pool, entitySlug string) (string, string, error) {
|
||||
|
||||
@@ -89,6 +89,7 @@ func (s *Server) CreateApprovalRule(ctx context.Context, req gen.CreateApprovalR
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "create",
|
||||
&id, "POST", "/api/v1/policy/approval-rules", "",
|
||||
nil,
|
||||
map[string]any{"action": req.Body.Action, "risk_class": req.Body.RiskClass}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
@@ -148,6 +149,7 @@ func (s *Server) PatchApprovalRule(ctx context.Context, req gen.PatchApprovalRul
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "patch",
|
||||
&id, "PATCH", "/api/v1/policy/approval-rules/"+req.Id, "",
|
||||
nil,
|
||||
map[string]any{"action": req.Body.Action}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
@@ -154,6 +154,7 @@ func (s *Server) DecideApproval(ctx context.Context, req gen.DecideApprovalReque
|
||||
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "decide",
|
||||
&id, "POST", "/api/v1/approvals/"+req.Id+"/decision", "",
|
||||
nil,
|
||||
map[string]any{"decision": status}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
@@ -81,6 +81,7 @@ func (s *Server) PatchAutonomySettings(ctx context.Context, req gen.PatchAutonom
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "patch",
|
||||
nil, "PATCH", "/api/v1/policy/autonomy", "",
|
||||
nil,
|
||||
map[string]any{"keys": keysOfMap(*req.Body)}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
@@ -182,6 +182,7 @@ func (s *Server) CreateCheck(ctx context.Context, req gen.CreateCheckRequestObje
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "create",
|
||||
&id, "POST", "/api/v1/checks", "",
|
||||
nil,
|
||||
map[string]any{"kind": req.Body.Kind, "slug": slug}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
@@ -269,6 +270,7 @@ func (s *Server) PatchCheck(ctx context.Context, req gen.PatchCheckRequestObject
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "patch",
|
||||
&id, "PATCH", "/api/v1/checks/"+req.Id, "",
|
||||
nil,
|
||||
map[string]any{"enabled": updated.Enabled}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
88
internal/httpapi/client_context.go
Normal file
88
internal/httpapi/client_context.go
Normal file
@@ -0,0 +1,88 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
)
|
||||
|
||||
func (s *Server) GetClientContext(ctx context.Context, req gen.GetClientContextRequestObject) (gen.GetClientContextResponseObject, error) {
|
||||
slug := string(req.Slug)
|
||||
_, err := s.resolveEntityID(ctx, slug)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var version int64
|
||||
_ = s.pool.QueryRow(ctx,
|
||||
"SELECT version FROM context_version WHERE singleton = true").Scan(&version)
|
||||
|
||||
var filesChanged, toolsChanged []string
|
||||
var sopsChanged bool
|
||||
if req.Params.Since != nil {
|
||||
rows, qErr := s.pool.Query(ctx,
|
||||
"SELECT path FROM context_files WHERE last_changed > $1", *req.Params.Since)
|
||||
if qErr == nil {
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var p string
|
||||
if scanErr := rows.Scan(&p); scanErr == nil {
|
||||
// Matches tools/setup-*.sh (the auto-setup convention —
|
||||
// see tools/post-pull.sh). Was tools/*.setup.sh until
|
||||
// 2026-07-12, which never matched any real filename.
|
||||
if strings.HasPrefix(p, "tools/setup-") && strings.HasSuffix(p, ".sh") {
|
||||
toolsChanged = append(toolsChanged, p)
|
||||
} else if p == ".sops.yaml" {
|
||||
sopsChanged = true
|
||||
} else {
|
||||
filesChanged = append(filesChanged, p)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if filesChanged == nil {
|
||||
filesChanged = []string{}
|
||||
}
|
||||
if toolsChanged == nil {
|
||||
toolsChanged = []string{}
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
|
||||
return gen.GetClientContext200JSONResponse{
|
||||
AgentFilesChanged: &filesChanged,
|
||||
SopsConfigChanged: &sopsChanged,
|
||||
ToolsChanged: &toolsChanged,
|
||||
Version: int(version),
|
||||
Since: &now,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *Server) GetClientSecrets(ctx context.Context, req gen.GetClientSecretsRequestObject) (gen.GetClientSecretsResponseObject, error) {
|
||||
slug := string(req.Slug)
|
||||
_, err := s.resolveEntityID(ctx, slug)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var keys []string
|
||||
if s.secretsManager != nil {
|
||||
list, listErr := s.secretsManager.List(ctx)
|
||||
if listErr == nil {
|
||||
prefix := "clients/" + slug + "/"
|
||||
for _, k := range list {
|
||||
if strings.HasPrefix(k, prefix) || strings.HasPrefix(k, "shared/") {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if keys == nil {
|
||||
keys = []string{}
|
||||
}
|
||||
|
||||
return gen.GetClientSecrets200JSONResponse{Keys: keys}, nil
|
||||
}
|
||||
310
internal/httpapi/client_lifecycle.go
Normal file
310
internal/httpapi/client_lifecycle.go
Normal file
@@ -0,0 +1,310 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math/big"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
openapi_types "github.com/oapi-codegen/runtime/types"
|
||||
)
|
||||
|
||||
func (s *Server) EnrollClient(ctx context.Context, req gen.EnrollClientRequestObject) (gen.EnrollClientResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := s.resolveEntityID(ctx, req.Body.Slug)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
current, err := sqlcgen.New(s.pool).GetEntityByID(ctx, id)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %s", domain.ErrNotFound, req.Body.Slug)
|
||||
}
|
||||
|
||||
currentState := ""
|
||||
if current.State != nil {
|
||||
currentState = *current.State
|
||||
}
|
||||
if currentState != "planned" && currentState != "provisioning" {
|
||||
return nil, fmt.Errorf("%w: entity %s is in state %q, expected planned or provisioning",
|
||||
domain.ErrInvalidTransition, req.Body.Slug, currentState)
|
||||
}
|
||||
|
||||
meshIP := ""
|
||||
if req.Body.MeshIp != nil {
|
||||
meshIP = *req.Body.MeshIp
|
||||
}
|
||||
if meshIP == "" {
|
||||
return nil, fmt.Errorf("%w: mesh_ip is required for enrollment", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
agePubKey, agePrivKey, err := generateAgeKeypair()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("age key generation: %w", err)
|
||||
}
|
||||
|
||||
if s.secretsManager != nil {
|
||||
keyPath := "clients/" + req.Body.Slug + "/age-key"
|
||||
_ = s.secretsManager.Set(ctx, keyPath, agePrivKey)
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
var attrs map[string]any
|
||||
if len(current.Attributes) > 0 {
|
||||
json.Unmarshal(current.Attributes, &attrs)
|
||||
}
|
||||
if attrs == nil {
|
||||
attrs = map[string]any{}
|
||||
}
|
||||
attrs["age_pubkey"] = agePubKey
|
||||
attrs["mesh_ip"] = meshIP
|
||||
attrs["enrolled_at"] = time.Now().UTC().Format(time.RFC3339)
|
||||
if req.Body.Hostname != nil {
|
||||
attrs["hostname"] = *req.Body.Hostname
|
||||
}
|
||||
attrsJSON, _ := json.Marshal(attrs)
|
||||
|
||||
q := sqlcgen.New(tx)
|
||||
provisioning := "provisioning"
|
||||
now := time.Now().UTC()
|
||||
_, err = q.UpdateEntity(ctx, sqlcgen.UpdateEntityParams{
|
||||
State: &provisioning,
|
||||
Attributes: attrsJSON,
|
||||
ID: id,
|
||||
Version: current.Version,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
_, _ = tx.Exec(ctx,
|
||||
"UPDATE entities SET enrolled_at = $1 WHERE id = $2", now, id)
|
||||
|
||||
_, actor := actorInfo(ctx)
|
||||
entityID := id
|
||||
_ = observability.Audit(ctx, q, "operator", actor, "enroll",
|
||||
&entityID, "POST", "/api/v1/clients/enroll", "",
|
||||
nil,
|
||||
map[string]any{"slug": req.Body.Slug, "mesh_ip": meshIP})
|
||||
_ = observability.Event(ctx, q, "client.enrolled", &entityID,
|
||||
"info", "oikos-api", "",
|
||||
map[string]any{"slug": req.Body.Slug, "type": current.Type})
|
||||
|
||||
if err := ensureDefaultChecks(ctx, tx, id, req.Body.Slug, current.Type, current.Name, attrsJSON); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Store age key in Infisical when backend is available.
|
||||
if s.secretsManager != nil {
|
||||
keyPath := "clients/" + req.Body.Slug + "/age-key"
|
||||
_ = s.secretsManager.Set(ctx, keyPath, agePrivKey)
|
||||
}
|
||||
|
||||
resp := gen.EnrollResponse{
|
||||
AgePublicKey: agePubKey,
|
||||
AgePrivateKey: agePrivKey,
|
||||
}
|
||||
|
||||
return gen.EnrollClient200JSONResponse(resp), nil
|
||||
}
|
||||
|
||||
func (s *Server) ProvisionEntity(ctx context.Context, req gen.ProvisionEntityRequestObject) (gen.ProvisionEntityResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
hostSlug := req.Body.Host
|
||||
hostID, err := s.resolveEntityID(ctx, hostSlug)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: host %q not found", domain.ErrNotFound, hostSlug)
|
||||
}
|
||||
|
||||
var existingID uuid.UUID
|
||||
err = s.pool.QueryRow(ctx,
|
||||
"SELECT id FROM entities WHERE slug = $1", req.Body.Slug).Scan(&existingID)
|
||||
if err == nil {
|
||||
return nil, fmt.Errorf("%w: entity slug %q already exists", domain.ErrConflict, req.Body.Slug)
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
entityID := uuid.Must(uuid.NewV7())
|
||||
var attrsJSON []byte
|
||||
if req.Body.Attributes != nil {
|
||||
attrsJSON, _ = json.Marshal(req.Body.Attributes)
|
||||
}
|
||||
if len(attrsJSON) == 0 {
|
||||
attrsJSON = []byte("{}")
|
||||
}
|
||||
|
||||
plannedState := "planned"
|
||||
q := sqlcgen.New(tx)
|
||||
inserted, err := q.InsertEntity(ctx, sqlcgen.InsertEntityParams{
|
||||
ID: entityID,
|
||||
Slug: req.Body.Slug,
|
||||
Type: req.Body.Type,
|
||||
Name: req.Body.Name,
|
||||
State: &plannedState,
|
||||
Attributes: attrsJSON,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
execID := uuid.Must(uuid.NewV7())
|
||||
corrID := "provision_" + entityID.String()[:8]
|
||||
if err := q.InsertExecution(ctx, sqlcgen.InsertExecutionParams{
|
||||
EntityID: entityID,
|
||||
Action: "provision",
|
||||
RiskClass: "config_mutation",
|
||||
CorrelationID: corrID,
|
||||
}); err != nil {
|
||||
return nil, fmt.Errorf("create execution: %w", err)
|
||||
}
|
||||
|
||||
type stepDef struct {
|
||||
order int
|
||||
name string
|
||||
}
|
||||
steps := []stepDef{
|
||||
{1, "validate-constraints"},
|
||||
{2, "create-container"},
|
||||
{3, "configure-network"},
|
||||
{4, "install-services"},
|
||||
{5, "configure-mounts"},
|
||||
{6, "health-check"},
|
||||
}
|
||||
for _, st := range steps {
|
||||
_, err = tx.Exec(ctx,
|
||||
`INSERT INTO provisioning_steps (id, entity_id, execution_id, step_order, step_name)
|
||||
VALUES ($1, $2, $3, $4, $5)`,
|
||||
uuid.Must(uuid.NewV7()), entityID, entityID /* executions PK is entity_id */, st.order, st.name)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("insert provisioning step: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
_, err = tx.Exec(ctx,
|
||||
`INSERT INTO relationships (source_id, target_id, type)
|
||||
VALUES ($1, $2, 'hosts')`, hostID, entityID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("insert relationship: %w", err)
|
||||
}
|
||||
|
||||
_, actor := actorInfo(ctx)
|
||||
_ = observability.Audit(ctx, q, "operator", actor, "provision",
|
||||
&entityID, "POST", "/api/v1/entities/provision", "",
|
||||
nil,
|
||||
map[string]any{"slug": req.Body.Slug, "host": hostSlug})
|
||||
_ = observability.Event(ctx, q, "entity.provisioned", &entityID,
|
||||
"info", "oikos-api", "",
|
||||
map[string]any{"slug": req.Body.Slug, "type": req.Body.Type, "host": hostSlug})
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
entity := sqlcEntityToGen(inserted)
|
||||
return gen.ProvisionEntity201JSONResponse{
|
||||
Body: gen.ProvisionResponse{
|
||||
Entity: entity,
|
||||
ExecutionId: openapi_types.UUID(execID),
|
||||
},
|
||||
Headers: gen.ProvisionEntity201ResponseHeaders{ETag: `"` + strconv.Itoa(int(inserted.Version)) + `"`},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *Server) GetProvisionStatus(ctx context.Context, req gen.GetProvisionStatusRequestObject) (gen.GetProvisionStatusResponseObject, error) {
|
||||
slug := string(req.Slug)
|
||||
id, err := s.resolveEntityID(ctx, slug)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var state string
|
||||
if err := s.pool.QueryRow(ctx,
|
||||
"SELECT state FROM entities WHERE id = $1", id).Scan(&state); err != nil {
|
||||
return nil, fmt.Errorf("%w: %s", domain.ErrNotFound, slug)
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT step_name, status, error_message, started_at, finished_at
|
||||
FROM provisioning_steps WHERE entity_id = $1 ORDER BY step_order`, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var provSteps []struct {
|
||||
ErrorMessage *string `json:"error_message"`
|
||||
FinishedAt *time.Time `json:"finished_at"`
|
||||
StartedAt *time.Time `json:"started_at"`
|
||||
Status gen.ProvisionStatusStepsStatus `json:"status"`
|
||||
Step string `json:"step"`
|
||||
}
|
||||
for rows.Next() {
|
||||
var stepName, status string
|
||||
var errMsg *string
|
||||
var started, finished *time.Time
|
||||
if scanErr := rows.Scan(&stepName, &status, &errMsg, &started, &finished); scanErr != nil {
|
||||
return nil, scanErr
|
||||
}
|
||||
provSteps = append(provSteps, struct {
|
||||
ErrorMessage *string `json:"error_message"`
|
||||
FinishedAt *time.Time `json:"finished_at"`
|
||||
StartedAt *time.Time `json:"started_at"`
|
||||
Status gen.ProvisionStatusStepsStatus `json:"status"`
|
||||
Step string `json:"step"`
|
||||
}{
|
||||
Step: stepName,
|
||||
Status: gen.ProvisionStatusStepsStatus(status),
|
||||
ErrorMessage: errMsg,
|
||||
StartedAt: started,
|
||||
FinishedAt: finished,
|
||||
})
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
return gen.GetProvisionStatus200JSONResponse{
|
||||
Slug: slug,
|
||||
State: state,
|
||||
Steps: provSteps,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func generateAgeKeypair() (pubKey, privKey string, err error) {
|
||||
seed := make([]byte, 32)
|
||||
if _, err := rand.Read(seed); err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
n := new(big.Int).SetBytes(seed)
|
||||
pub := fmt.Sprintf("age1%064x", n)
|
||||
priv := fmt.Sprintf("AGE-SECRET-KEY-1%064x", n)
|
||||
return pub, priv, nil
|
||||
}
|
||||
@@ -3,31 +3,22 @@ package httpapi
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/dtoro/oikos/internal/checkdefaults"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// ensureDefaultChecks derives an entity's default checks from the monitoring
|
||||
// kinds its type declares.
|
||||
// kinds its type declares. Thin wrapper over the shared db.EnsureEntityChecks
|
||||
// hook so the HTTP create/patch paths and the MCP entity-mutation tools stay
|
||||
// in lockstep.
|
||||
//
|
||||
// Note the ordering caveat: an entity created through the API usually has no
|
||||
// edges yet, so a type whose address comes from its host (a service) will
|
||||
// produce no checks on this pass. That gap is real and deliberately visible —
|
||||
// coverageSweep reports it, and the next inventory ingest fills it in once
|
||||
// the hosting edge exists.
|
||||
// Note the ordering caveat (carried from db.LoadTypeTree / checkdefaults.Ensure):
|
||||
// an entity created through the API usually has no edges yet, so a type whose
|
||||
// address comes from its host (a service) will produce no checks on this pass.
|
||||
// That gap is real and deliberately visible — coverageSweep reports it, and
|
||||
// the next inventory ingest fills it in once the hosting edge exists.
|
||||
func ensureDefaultChecks(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, slug, entityType, name string, attrsJSON []byte) error {
|
||||
tree, err := db.LoadTypeTree(ctx, tx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
res, err := checkdefaults.Ensure(ctx, tx, tree, checkdefaults.Target{
|
||||
ID: entityID, Slug: slug, Type: entityType, Name: name, Attrs: attrsJSON,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
checkdefaults.LogResult(slug, entityType, res)
|
||||
return nil
|
||||
_, err := db.EnsureEntityChecks(ctx, tx, entityID, slug, entityType, name, attrsJSON)
|
||||
return err
|
||||
}
|
||||
|
||||
354
internal/httpapi/entities.go
Normal file
354
internal/httpapi/entities.go
Normal file
@@ -0,0 +1,354 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
func (s *Server) ListEntities(ctx context.Context, req gen.ListEntitiesRequestObject) (gen.ListEntitiesResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
|
||||
// Type filter includes descendants via the parent hierarchy (R3-1).
|
||||
query := `
|
||||
WITH RECURSIVE tt AS (
|
||||
SELECT name FROM entity_types WHERE $1::text IS NULL OR name = $1
|
||||
UNION
|
||||
SELECT et.name FROM entity_types et JOIN tt ON et.parent_type = tt.name
|
||||
WHERE $1::text IS NOT NULL
|
||||
)
|
||||
SELECT ` + entityCols + ` FROM entities e
|
||||
JOIN entity_types et ON et.name = e.type
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.type IN (SELECT name FROM tt)
|
||||
AND ($2::text IS NULL OR e.state = $2)
|
||||
AND ($3::text IS NULL OR et.domain = $3)
|
||||
AND ($4::text IS NULL OR et.layer = $4)
|
||||
AND ($5::text IS NULL OR e.slug ILIKE '%'||$5||'%' OR e.name ILIKE '%'||$5||'%')
|
||||
AND ($6::text IS NULL OR e.slug > $6)
|
||||
ORDER BY e.slug
|
||||
LIMIT $7`
|
||||
|
||||
rows, err := s.pool.Query(ctx, query,
|
||||
req.Params.Type, req.Params.State, req.Params.Domain, req.Params.Layer,
|
||||
req.Params.Q, req.Params.Cursor, limit+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var items []gen.Entity
|
||||
for rows.Next() {
|
||||
e, err := scanEntity(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, e)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
var next *string
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
next = &items[len(items)-1].Slug
|
||||
}
|
||||
if items == nil {
|
||||
items = []gen.Entity{}
|
||||
}
|
||||
return gen.ListEntities200JSONResponse{Items: items, NextCursor: next}, nil
|
||||
}
|
||||
|
||||
func (s *Server) GetEntity(ctx context.Context, req gen.GetEntityRequestObject) (gen.GetEntityResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
e, err := scanEntity(s.pool.QueryRow(ctx,
|
||||
"SELECT "+entityCols+" FROM entities e LEFT JOIN entity_status st ON st.entity_id = e.id WHERE e.id = $1", id))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gen.GetEntity200JSONResponse{
|
||||
Body: e,
|
||||
Headers: gen.GetEntity200ResponseHeaders{ETag: `"` + strconv.Itoa(e.Version) + `"`},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *Server) GetEntityRelations(ctx context.Context, req gen.GetEntityRelationsRequestObject) (gen.GetEntityRelationsResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
dir := "both"
|
||||
if req.Params.Direction != nil {
|
||||
dir = string(*req.Params.Direction)
|
||||
}
|
||||
relType := req.Params.RelType
|
||||
rows, err := sqlcgen.New(s.pool).ListEntityRelations(ctx, sqlcgen.ListEntityRelationsParams{
|
||||
Direction: dir,
|
||||
ID: id,
|
||||
RelType: relType,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items := []gen.Relationship{}
|
||||
for _, r := range rows {
|
||||
var attrs *map[string]any
|
||||
if len(r.Attributes) > 0 {
|
||||
var m map[string]any
|
||||
if json.Unmarshal(r.Attributes, &m) == nil && len(m) > 0 {
|
||||
attrs = &m
|
||||
}
|
||||
}
|
||||
validTo := r.ValidTo
|
||||
items = append(items, gen.Relationship{
|
||||
Source: r.SourceSlug,
|
||||
Target: r.TargetSlug,
|
||||
Type: r.Type,
|
||||
Attributes: attrs,
|
||||
ValidFrom: r.ValidFrom,
|
||||
ValidTo: validTo,
|
||||
})
|
||||
}
|
||||
return gen.GetEntityRelations200JSONResponse{Items: items}, nil
|
||||
}
|
||||
|
||||
func (s *Server) GetBlastRadius(ctx context.Context, req gen.GetBlastRadiusRequestObject) (gen.GetBlastRadiusResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
depth := 3
|
||||
if req.Params.Depth != nil {
|
||||
depth = *req.Params.Depth
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT `+entityCols+`, b.depth
|
||||
FROM blast_radius($1, $2) b
|
||||
JOIN entities e ON e.id = b.entity_id
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY b.depth, e.slug`, id, depth)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
resp := gen.GetBlastRadius200JSONResponse{Items: []struct {
|
||||
Depth int `json:"depth"`
|
||||
Entity gen.Entity `json:"entity"`
|
||||
}{}}
|
||||
for rows.Next() {
|
||||
var e gen.Entity
|
||||
var state *string
|
||||
var attrsJSON []byte
|
||||
var maint *time.Time
|
||||
var health *string
|
||||
var lastCheckAt *time.Time
|
||||
var d int
|
||||
if err := rows.Scan(&e.Id, &e.Slug, &e.Type, &e.Name, &state, &attrsJSON,
|
||||
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt, &health, &lastCheckAt, &d); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
e.State = state
|
||||
e.MaintenanceUntil = maint
|
||||
if health != nil {
|
||||
h := gen.EntityHealth(*health)
|
||||
e.Health = &h
|
||||
}
|
||||
e.LastCheckAt = lastCheckAt
|
||||
var attrs map[string]any
|
||||
if len(attrsJSON) > 0 && json.Unmarshal(attrsJSON, &attrs) == nil && len(attrs) > 0 {
|
||||
e.Attributes = &attrs
|
||||
}
|
||||
resp.Items = append(resp.Items, struct {
|
||||
Depth int `json:"depth"`
|
||||
Entity gen.Entity `json:"entity"`
|
||||
}{Depth: d, Entity: e})
|
||||
}
|
||||
return resp, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Server) GetGraph(ctx context.Context, req gen.GetGraphRequestObject) (gen.GetGraphResponseObject, error) {
|
||||
depth := 2
|
||||
if req.Params.Depth != nil {
|
||||
depth = *req.Params.Depth
|
||||
}
|
||||
|
||||
var nodes []gen.Entity
|
||||
var err error
|
||||
truncated := false
|
||||
|
||||
// pgx can't infer the array element type from a nil *[]string (the
|
||||
// param is absent from the request, not an empty list), so dereference
|
||||
// to a plain []string first — nil there still encodes as SQL NULL, but
|
||||
// pgx has a concrete type to work with.
|
||||
var relTypes []string
|
||||
if req.Params.RelType != nil {
|
||||
relTypes = *req.Params.RelType
|
||||
}
|
||||
|
||||
if req.Params.Root != nil && *req.Params.Root != "" {
|
||||
rootID, rerr := s.resolveEntityID(ctx, *req.Params.Root)
|
||||
if rerr != nil {
|
||||
return nil, rerr
|
||||
}
|
||||
nodes, err = s.queryEntities(ctx, `
|
||||
SELECT `+entityCols+`
|
||||
FROM blast_radius($1, $2, $3) b JOIN entities e ON e.id = b.entity_id
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
ORDER BY e.slug`, rootID, depth, relTypes)
|
||||
} else {
|
||||
// Whole-graph view: pick the most-connected entities first so the
|
||||
// graph shows actual topology, not just whatever sorts first
|
||||
// alphabetically. Without this the cap fills with exec:* rows and
|
||||
// drops every host/lxc/service/vm — and every edge those entities
|
||||
// connect — because edges require both endpoints in the node set.
|
||||
// Exclude the cognition transactional types (execution/task): they
|
||||
// are audit records rather than topology, and at ~380 rows they
|
||||
// consumed most of the old 500-node cap.
|
||||
nodes, err = s.queryEntities(ctx, `
|
||||
SELECT `+entityCols+`
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.type NOT IN ('execution','task')
|
||||
AND e.id IN (
|
||||
SELECT e2.id FROM entities e2
|
||||
LEFT JOIN relationships r ON r.valid_to IS NULL
|
||||
AND (r.source_id = e2.id OR r.target_id = e2.id)
|
||||
WHERE e2.type NOT IN ('execution','task')
|
||||
GROUP BY e2.id
|
||||
ORDER BY count(r.type) DESC, e2.slug
|
||||
LIMIT $1
|
||||
)
|
||||
ORDER BY e.slug`,
|
||||
graphNodeCap+1)
|
||||
if err == nil && len(nodes) > graphNodeCap {
|
||||
nodes = nodes[:graphNodeCap]
|
||||
truncated = true
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
ids := make([]uuid.UUID, len(nodes))
|
||||
for i, n := range nodes {
|
||||
ids[i] = uuid.UUID(n.Id)
|
||||
}
|
||||
edgeRows, err := sqlcgen.New(s.pool).ListGraphEdges(ctx, sqlcgen.ListGraphEdgesParams{
|
||||
Ids: ids,
|
||||
RelTypes: relTypes,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
edges := []gen.Relationship{}
|
||||
for _, r := range edgeRows {
|
||||
var attrs *map[string]any
|
||||
if len(r.Attributes) > 0 {
|
||||
var m map[string]any
|
||||
if json.Unmarshal(r.Attributes, &m) == nil && len(m) > 0 {
|
||||
attrs = &m
|
||||
}
|
||||
}
|
||||
validTo := r.ValidTo
|
||||
edges = append(edges, gen.Relationship{
|
||||
Source: r.SourceSlug,
|
||||
Target: r.TargetSlug,
|
||||
Type: r.Type,
|
||||
Attributes: attrs,
|
||||
ValidFrom: r.ValidFrom,
|
||||
ValidTo: validTo,
|
||||
})
|
||||
}
|
||||
|
||||
resp := gen.GetGraph200JSONResponse{Nodes: nodes, Edges: edges}
|
||||
if truncated {
|
||||
resp.Truncated = &truncated
|
||||
}
|
||||
|
||||
if req.Params.Include != nil {
|
||||
for _, inc := range *req.Params.Include {
|
||||
if inc == gen.Status {
|
||||
health, herr := s.entityHealthByID(ctx, ids)
|
||||
if herr != nil {
|
||||
return nil, herr
|
||||
}
|
||||
resp.Health = &health
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
// entityHealthByID returns entity_status.health keyed by entity id, for the
|
||||
// given id set (used by GetGraph's include=status).
|
||||
func (s *Server) entityHealthByID(ctx context.Context, ids []uuid.UUID) (map[string]gen.GraphViewHealth, error) {
|
||||
health := make(map[string]gen.GraphViewHealth, len(ids))
|
||||
rows, err := s.pool.Query(ctx,
|
||||
`SELECT entity_id, health FROM entity_status WHERE entity_id = ANY($1)`, ids)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var id uuid.UUID
|
||||
var h string
|
||||
if err := rows.Scan(&id, &h); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
health[id.String()] = gen.GraphViewHealth(h)
|
||||
}
|
||||
return health, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Server) queryEntities(ctx context.Context, query string, args ...any) ([]gen.Entity, error) {
|
||||
rows, err := s.pool.Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
items := []gen.Entity{}
|
||||
for rows.Next() {
|
||||
e, err := scanEntity(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, e)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
// sqlcEntityToGen converts a sqlcgen.Entity to a gen.Entity.
|
||||
func sqlcEntityToGen(e sqlcgen.Entity) gen.Entity {
|
||||
out := gen.Entity{
|
||||
Id: e.ID,
|
||||
Slug: e.Slug,
|
||||
Type: e.Type,
|
||||
Name: e.Name,
|
||||
State: e.State,
|
||||
Version: int(e.Version),
|
||||
CreatedAt: e.CreatedAt,
|
||||
UpdatedAt: e.UpdatedAt,
|
||||
}
|
||||
if e.MaintenanceUntil != nil {
|
||||
out.MaintenanceUntil = e.MaintenanceUntil
|
||||
}
|
||||
if len(e.Attributes) > 0 {
|
||||
var attrs map[string]any
|
||||
if json.Unmarshal(e.Attributes, &attrs) == nil && len(attrs) > 0 {
|
||||
out.Attributes = &attrs
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
290
internal/httpapi/entity_mutations.go
Normal file
290
internal/httpapi/entity_mutations.go
Normal file
@@ -0,0 +1,290 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
func (s *Server) CreateEntity(ctx context.Context, req gen.CreateEntityRequestObject) (gen.CreateEntityResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
// Check idempotency if a key was provided. The idempotency scope is the
|
||||
// calling actor, so replays are per-caller.
|
||||
actorType, actorLabel := actorInfo(ctx)
|
||||
actor := actorLabel
|
||||
var bodyHash string
|
||||
if req.Params.IdempotencyKey != nil && *req.Params.IdempotencyKey != "" {
|
||||
key := *req.Params.IdempotencyKey
|
||||
q := sqlcgen.New(s.pool)
|
||||
cached, err := q.GetIdempotentResponse(ctx, sqlcgen.GetIdempotentResponseParams{
|
||||
Actor: actor,
|
||||
Key: key,
|
||||
})
|
||||
if err == nil {
|
||||
// Verify the request body hasn't changed.
|
||||
bodyJSON, _ := json.Marshal(req.Body)
|
||||
bodyHash = fmt.Sprintf("%x", sha256.Sum256(bodyJSON))
|
||||
if cached.RequestHash != bodyHash {
|
||||
return nil, fmt.Errorf("%w: idempotency key %s used with different request body", domain.ErrConflict, key)
|
||||
}
|
||||
// Replay the cached response.
|
||||
if cached.ResponseCode != nil && *cached.ResponseCode == 201 {
|
||||
var entity gen.Entity
|
||||
if len(cached.ResponseBody) > 0 {
|
||||
if err := json.Unmarshal(cached.ResponseBody, &entity); err != nil {
|
||||
return nil, fmt.Errorf("unmarshal cached response: %w", err)
|
||||
}
|
||||
}
|
||||
return gen.CreateEntity201JSONResponse{
|
||||
Body: entity,
|
||||
Headers: gen.CreateEntity201ResponseHeaders{ETag: `"` + strconv.Itoa(entity.Version) + `"`},
|
||||
}, nil
|
||||
}
|
||||
// Forward cached error response.
|
||||
return gen.CreateEntitydefaultApplicationProblemPlusJSONResponse{
|
||||
Body: gen.Problem{Status: int(*cached.ResponseCode), Title: "replayed error"},
|
||||
StatusCode: int(*cached.ResponseCode),
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
|
||||
id, err := uuid.NewV7()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
slug := req.Body.Slug
|
||||
if slug == "" {
|
||||
slug = req.Body.Type + ":" + req.Body.Name
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
q := sqlcgen.New(tx)
|
||||
|
||||
// Validate type exists and is NOT abstract.
|
||||
var isAbstract bool
|
||||
if err := tx.QueryRow(ctx, `SELECT is_abstract FROM entity_types WHERE name = $1`, req.Body.Type).Scan(&isAbstract); err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: entity type %q", domain.ErrNotFound, req.Body.Type)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if isAbstract {
|
||||
return nil, fmt.Errorf("%w: %s", domain.ErrAbstractType, req.Body.Type)
|
||||
}
|
||||
|
||||
// Get default state from lifecycle.
|
||||
var defaultState *string
|
||||
var lcDefault string
|
||||
if err := tx.QueryRow(ctx, `SELECT ld.default_state FROM lifecycle_defs ld
|
||||
JOIN entity_types et ON et.lifecycle_id = ld.id
|
||||
WHERE et.name = $1`, req.Body.Type).Scan(&lcDefault); err == nil {
|
||||
defaultState = &lcDefault
|
||||
}
|
||||
|
||||
state := req.Body.State
|
||||
if state == nil && defaultState != nil {
|
||||
state = defaultState
|
||||
}
|
||||
|
||||
// attributes is NOT NULL; the column default only applies when omitted,
|
||||
// not when an explicit NULL is bound — so default to an empty object.
|
||||
attrsJSON := []byte("{}")
|
||||
if req.Body.Attributes != nil {
|
||||
attrsJSON, _ = json.Marshal(req.Body.Attributes)
|
||||
}
|
||||
|
||||
// Insert the entity.
|
||||
inserted, err := q.InsertEntity(ctx, sqlcgen.InsertEntityParams{
|
||||
ID: id,
|
||||
Slug: slug,
|
||||
Type: req.Body.Type,
|
||||
Name: req.Body.Name,
|
||||
State: state,
|
||||
Attributes: attrsJSON,
|
||||
})
|
||||
if err != nil {
|
||||
// Duplicate slug.
|
||||
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
|
||||
return nil, fmt.Errorf("%w: slug %q already exists", domain.ErrAlreadyExists, slug)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Convert sqlcgen.Entity → gen.Entity.
|
||||
entity := sqlcEntityToGen(inserted)
|
||||
|
||||
// Cache idempotent response.
|
||||
if req.Params.IdempotencyKey != nil && *req.Params.IdempotencyKey != "" {
|
||||
respBody, _ := json.Marshal(entity)
|
||||
code := int32(201)
|
||||
if bodyHash == "" {
|
||||
bodyJSON, _ := json.Marshal(req.Body)
|
||||
bodyHash = fmt.Sprintf("%x", sha256.Sum256(bodyJSON))
|
||||
}
|
||||
if putErr := q.PutIdempotentResponse(ctx, sqlcgen.PutIdempotentResponseParams{
|
||||
Actor: actor,
|
||||
Key: *req.Params.IdempotencyKey,
|
||||
RequestHash: bodyHash,
|
||||
ResponseCode: &code,
|
||||
ResponseBody: respBody,
|
||||
}); putErr != nil {
|
||||
return nil, putErr
|
||||
}
|
||||
}
|
||||
|
||||
// Audit.
|
||||
entityID := inserted.ID
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "create",
|
||||
&entityID, "POST", "/api/v1/entities", "",
|
||||
nil,
|
||||
map[string]any{"type": req.Body.Type, "slug": slug}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if eventErr := observability.Event(ctx, q, "entity.created", &entityID,
|
||||
"info", "oikos-api", "",
|
||||
map[string]any{"slug": slug, "type": req.Body.Type}); eventErr != nil {
|
||||
return nil, eventErr
|
||||
}
|
||||
|
||||
if err := ensureDefaultChecks(ctx, tx, inserted.ID, slug, req.Body.Type, inserted.Name, attrsJSON); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return gen.CreateEntity201JSONResponse{
|
||||
Body: entity,
|
||||
Headers: gen.CreateEntity201ResponseHeaders{ETag: `"` + strconv.Itoa(entity.Version) + `"`},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *Server) PatchEntity(ctx context.Context, req gen.PatchEntityRequestObject) (gen.PatchEntityResponseObject, error) {
|
||||
if req.Body == nil {
|
||||
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
|
||||
}
|
||||
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Parse If-Match header (quoted version string).
|
||||
ifMatch := strings.Trim(req.Params.IfMatch, `"`)
|
||||
expectedVersion, err := strconv.Atoi(ifMatch)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: invalid If-Match header %q", domain.ErrInvalidInput, req.Params.IfMatch)
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
// Get current entity for version check + lifecycle validation.
|
||||
current, err := sqlcgen.New(tx).GetEntityByID(ctx, id)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: %s", domain.ErrNotFound, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if int(current.Version) != expectedVersion {
|
||||
return nil, fmt.Errorf("%w: expected version %d, current version %d",
|
||||
domain.ErrConflict, expectedVersion, current.Version)
|
||||
}
|
||||
|
||||
// Validate lifecycle transition if state is being changed.
|
||||
if req.Body.State != nil && *req.Body.State != "" {
|
||||
fromState := ""
|
||||
if current.State != nil {
|
||||
fromState = *current.State
|
||||
}
|
||||
if err := db.ValidateTransition(ctx, tx, id, current.Type, fromState, *req.Body.State); err != nil {
|
||||
if errors.Is(err, db.ErrTransitionInvalid) {
|
||||
return nil, fmt.Errorf("%w: %v", domain.ErrInvalidTransition, err)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
// Check idempotency (note: the spec doesn't define Idempotency-Key for PATCH,
|
||||
// but we handle it if the generated code ever adds it).
|
||||
// For now, no idempotency check on PATCH.
|
||||
|
||||
// Marshal attributes if provided.
|
||||
var attrsJSON []byte
|
||||
if req.Body.Attributes != nil {
|
||||
attrsJSON, _ = json.Marshal(req.Body.Attributes)
|
||||
}
|
||||
|
||||
// Perform the update via sqlcgen.
|
||||
q := sqlcgen.New(tx)
|
||||
updated, err := q.UpdateEntity(ctx, sqlcgen.UpdateEntityParams{
|
||||
Name: req.Body.Name,
|
||||
State: req.Body.State,
|
||||
Attributes: attrsJSON,
|
||||
SetMaintenance: req.Body.MaintenanceUntil != nil,
|
||||
MaintenanceUntil: req.Body.MaintenanceUntil,
|
||||
ID: id,
|
||||
Version: int32(expectedVersion),
|
||||
})
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
// Version mismatch or entity not found.
|
||||
return nil, fmt.Errorf("%w: entity was modified concurrently", domain.ErrConflict)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
entity := sqlcEntityToGen(updated)
|
||||
|
||||
// Audit.
|
||||
patchActorType, patchActor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, patchActorType, patchActor, "patch",
|
||||
&id, "PATCH", "/api/v1/entities/"+req.Id, "",
|
||||
nil,
|
||||
map[string]any{"version": expectedVersion}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
if eventErr := observability.Event(ctx, q, "entity.updated", &id,
|
||||
"info", "oikos-api", "",
|
||||
map[string]any{"slug": entity.Slug, "type": entity.Type, "version": updated.Version}); eventErr != nil {
|
||||
return nil, eventErr
|
||||
}
|
||||
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
s.entityCache.Invalidate(entity.Slug, entity.Id.String())
|
||||
|
||||
return gen.PatchEntity200JSONResponse{
|
||||
Body: entity,
|
||||
Headers: gen.PatchEntity200ResponseHeaders{ETag: `"` + strconv.Itoa(entity.Version) + `"`},
|
||||
}, nil
|
||||
}
|
||||
@@ -67,6 +67,7 @@ func (s *Server) CreateEntityType(ctx context.Context, req gen.CreateEntityTypeR
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "create",
|
||||
nil, "POST", "/api/v1/ontology/entity-types", "",
|
||||
nil,
|
||||
map[string]any{"name": req.Body.Name, "domain": req.Body.Domain}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
@@ -148,6 +149,7 @@ func (s *Server) PatchEntityType(ctx context.Context, req gen.PatchEntityTypeReq
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "patch",
|
||||
nil, "PATCH", "/api/v1/ontology/entity-types/"+req.Name, "",
|
||||
nil,
|
||||
map[string]any{"status": req.Body.Status}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
60
internal/httpapi/events.go
Normal file
60
internal/httpapi/events.go
Normal file
@@ -0,0 +1,60 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
)
|
||||
|
||||
func (s *Server) QueryEvents(ctx context.Context, req gen.QueryEventsRequestObject) (gen.QueryEventsResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
var eventType, entityID, severity, correlationID *string
|
||||
if req.Params.Type != nil {
|
||||
eventType = req.Params.Type
|
||||
}
|
||||
if req.Params.EntityId != nil {
|
||||
entityID = req.Params.EntityId
|
||||
}
|
||||
if req.Params.Severity != nil {
|
||||
severity = req.Params.Severity
|
||||
}
|
||||
if req.Params.CorrelationId != nil {
|
||||
correlationID = req.Params.CorrelationId
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT id, ts, type, entity_id::text, severity, source, data, correlation_id
|
||||
FROM events
|
||||
WHERE ($1::text IS NULL OR type = $1)
|
||||
AND ($2::text IS NULL OR entity_id::text = $2)
|
||||
AND ($3::text IS NULL OR severity = $3)
|
||||
AND ($4::text IS NULL OR correlation_id = $4)
|
||||
AND ($5::timestamptz IS NULL OR ts >= $5)
|
||||
AND ($6::timestamptz IS NULL OR ts <= $6)
|
||||
ORDER BY ts DESC
|
||||
LIMIT $7`,
|
||||
eventType, entityID, severity, correlationID, req.Params.From, req.Params.To, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.Event{}
|
||||
for rows.Next() {
|
||||
var e gen.Event
|
||||
var dataBytes []byte
|
||||
var entID, corrID *string
|
||||
if err := rows.Scan(&e.Id, &e.Ts, &e.Type, &entID, &e.Severity, &e.Source, &dataBytes, &corrID); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
e.EntityId = entID
|
||||
e.CorrelationId = corrID
|
||||
var data map[string]any
|
||||
if json.Unmarshal(dataBytes, &data) == nil {
|
||||
e.Data = &data
|
||||
}
|
||||
items = append(items, e)
|
||||
}
|
||||
return gen.QueryEvents200JSONResponse{Items: items}, rows.Err()
|
||||
}
|
||||
@@ -240,6 +240,7 @@ func (s *Server) RequestExecution(ctx context.Context, req gen.RequestExecutionR
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "create",
|
||||
&id, "POST", "/api/v1/executions", "",
|
||||
nil,
|
||||
map[string]any{"action": req.Body.Action, "target": req.Body.Target}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
@@ -307,6 +308,7 @@ func (s *Server) CancelExecution(ctx context.Context, req gen.CancelExecutionReq
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "cancel",
|
||||
&id, "POST", "/api/v1/executions/"+req.Id+"/cancel", "",
|
||||
nil,
|
||||
map[string]any{"status": "cancelled"}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
80
internal/httpapi/fleet_health.go
Normal file
80
internal/httpapi/fleet_health.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
)
|
||||
|
||||
func (s *Server) GetFleetHealth(ctx context.Context, req gen.GetFleetHealthRequestObject) (gen.GetFleetHealthResponseObject, error) {
|
||||
resp := gen.GetFleetHealth200JSONResponse{}
|
||||
resp.Entities = []struct {
|
||||
Health gen.HealthSummaryEntitiesHealth `json:"health"`
|
||||
LastCheckAt *time.Time `json:"last_check_at"`
|
||||
Slug string `json:"slug"`
|
||||
Trend *gen.HealthSummaryEntitiesTrend `json:"trend"`
|
||||
Type string `json:"type"`
|
||||
}{}
|
||||
|
||||
// Exclude 'check' entities (internal probes) — only entities actually
|
||||
// being monitored should count toward fleet health.
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check'
|
||||
ORDER BY e.slug`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
stale := 0
|
||||
for rows.Next() {
|
||||
var slug, typ, health string
|
||||
var lastCheck *time.Time
|
||||
if err := rows.Scan(&slug, &typ, &health, &lastCheck); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch health {
|
||||
case "healthy":
|
||||
resp.Summary.Healthy++
|
||||
case "degraded":
|
||||
resp.Summary.Degraded++
|
||||
case "down":
|
||||
resp.Summary.Down++
|
||||
case "stale":
|
||||
stale++
|
||||
default:
|
||||
resp.Summary.Unknown++
|
||||
}
|
||||
resp.Entities = append(resp.Entities, struct {
|
||||
Health gen.HealthSummaryEntitiesHealth `json:"health"`
|
||||
LastCheckAt *time.Time `json:"last_check_at"`
|
||||
Slug string `json:"slug"`
|
||||
Trend *gen.HealthSummaryEntitiesTrend `json:"trend"`
|
||||
Type string `json:"type"`
|
||||
}{
|
||||
Health: gen.HealthSummaryEntitiesHealth(health),
|
||||
LastCheckAt: lastCheck,
|
||||
Slug: slug,
|
||||
Type: typ,
|
||||
})
|
||||
}
|
||||
if stale > 0 {
|
||||
resp.Summary.Stale = &stale
|
||||
}
|
||||
return resp, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Server) ExportSeeds(ctx context.Context, req gen.ExportSeedsRequestObject) (gen.ExportSeedsResponseObject, error) {
|
||||
exports, err := db.ExportToYAML(ctx, s.pool)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gen.ExportSeeds200JSONResponse{
|
||||
Ontology: string(exports["ontology.yaml"]),
|
||||
Inventory: string(exports["inventory.yaml"]),
|
||||
Policy: string(exports["policy.yaml"]),
|
||||
}, nil
|
||||
}
|
||||
@@ -616,13 +616,10 @@ type EnrollResponse struct {
|
||||
AgePublicKey string `json:"age_public_key"`
|
||||
|
||||
// InfisicalClientId Infisical UniversalAuth client ID
|
||||
InfisicalClientId string `json:"infisical_client_id"`
|
||||
InfisicalClientId *string `json:"infisical_client_id,omitempty"`
|
||||
|
||||
// InfisicalClientSecret Infisical UniversalAuth client secret
|
||||
InfisicalClientSecret string `json:"infisical_client_secret"`
|
||||
|
||||
// MachineIdentityToken Infisical machine identity access token
|
||||
MachineIdentityToken *string `json:"machine_identity_token,omitempty"`
|
||||
InfisicalClientSecret *string `json:"infisical_client_secret,omitempty"`
|
||||
}
|
||||
|
||||
// Entity defines model for Entity.
|
||||
@@ -8225,177 +8222,177 @@ func (sh *strictHandler) GetTrends(w http.ResponseWriter, r *http.Request, entit
|
||||
// Base64 encoded, gzipped, json marshaled Swagger object
|
||||
var swaggerSpec = []string{
|
||||
|
||||
"H4sIAAAAAAAC/+x963IbudXgq6C4WzXUpClq7JlkI9f3Q5E1thM71lqafJuKXBTYfUhihAZ6ADQlxuWq",
|
||||
"/NoH2MoT5km2cO1uEk02L7KcqfyxJTUaDZwLcO7nUy/lecEZMCV7p596M8AZCPPjxTWe6v8zkKkghSKc",
|
||||
"9U57H0DyUqSA5iAk4QxNuEBvJoN3WKWzXtKT6QxyrN9TiwJ6pz2pBGHT3ufPn5NegQXOQbkPnJdCcrH6",
|
||||
"ifcF/qUElJrHaCJ4jjAqBMwJLyUSIAvOJHwjEYMHNbLDekmP6Hd/KUEsekmP4Vx/PDxsX1bSu2CKqMWb",
|
||||
"bHUlP/305iXiAklaTlEfjqfH6HbGpTqdlWNB5O2R/2yB1az6Ksl6SU/ALyURkPVOlSihywquaBkBuH3W",
|
||||
"WMK9PM1xOsgJI7cJuqUP6WmKs2zRth797pYr+lHw/Jrotz9FAaux0gDrhIscq95pL8MKBkq/mkTmfZNB",
|
||||
"XnAFLF38CRaruz2nBJgaTIGBwAoydAeLF0hAQfFConuiZoShZ9/PkABVCobUDBAXZEoYpoE0PBQsMVeL",
|
||||
"rn18oL9eX3+OH94Cm6pZ7/S7Z/8ruvSJpfFVDF3jaUWmhAv06uL6Bfr+u2eIs8AnOZG545H44ioe2gZR",
|
||||
"b0lOVBuWqHlYnyCDCS6p6p3+cJLoPZO8zHunz070b4TZ374LuydMwRSE+dA1X0cPim9PDZ/1Ti3GzHlw",
|
||||
"CSwjbHpWFILPMdV/SjlTwMz+cFFQkmIN8+HPUgP+U+2D/1PApHfa+x/D6jgb2qdyGCY0n1yitxlmU0BS",
|
||||
"4SlkLxBGOSg8wO4NdI8lSgUYSuxnJaYDvSLB6VHvc9K7FHxMIV+z0MKO+M12C/bzRtZ7IQQXqP/hx3P0",
|
||||
"++9/+J1ZxhWZMkx/KjSss4NBzc4aW4P7EpJ+hMe8weLZFJg6SxWZE2UYvBC8AKGIRTJ2T0aWGj71gGma",
|
||||
"+1tPcU5HKabUMACWnGkq0d9OiWagXtLL02Lk6Q5kiqnZV+/jCmklPaxXMSJZhGmSXsqFAPuyG8JKSvGY",
|
||||
"gue4lVeyUtjxuVwzPvBL0gNzbHedvrHQ2iyEFaUayTLPsVh0momXattXJEi5BShkmaYg14FhzDkFzPRg",
|
||||
"xe+AjVJeWnLcDDdDBvZQ6bAWK7R0vHuqY/Vv9opWslejlGSJNiuy4uOfIVX6e/WzaZWuLXutkps9QEZY",
|
||||
"dV2spfps/TubadbNMe5GBvBQEAFyq2Vakgljy9LCdXnYHWFZndfhAdJSWaYuOCXpYpCag1j/jpUCwQYG",
|
||||
"Ge0MXuAF5TgisxkRSeOGS8iQnR1lZDJpB1mFX0Hk3Sil2JL3Kuk7CW31gcKqlPUtFvYy01RlaAYyc5Yx",
|
||||
"Yn6wsLZi4pzfQRbdoyztwtbKhFoCCvdVylkKgsnN5BHjBycnOlJuQMPhMOy0QS4NEl/HNh9KCluxDi4V",
|
||||
"ZzxfjCjMgdbhq59U14DhB5iDiMLRncX+xmnC8h1eoDEgPJZK4FShPmEzEERJlPF7dtSF0TpywSbiSnkB",
|
||||
"I7vW9SjXKlcBYmDHIj4HIUgGsstanTgau25iJBGnhSW0VLNuQv65oZOnJ4F9cbOemToBLQqqMiPqgimx",
|
||||
"2A5EqeKi6/VtBy9LX+YW7CU9/UWsrMq8kAq8kpeVtAWyuwhToDChta1UEDiM2JSDmvFuUxhNuZPYY8we",
|
||||
"I1J0G22OyVHKM+go9xxAkqkwGxg3TmWWEK9AKT3jCqndWc0cHnBe6DX3ppSPMT3WFDzCqYodbqVVCraS",
|
||||
"HuaYlnF27H5K3Rk93s60/hw6n0F6t7rZlLMJidhd/oIpsXqOPmrd7RehV43WOhnWhN+O94LemphjOpJx",
|
||||
"al6WnmZKFXqeVP+bEXmnL2AQamCuZA2OTJCJxlJhJRApZwO7tSgHUyzVaAaYqohx43pGJEo16L6RiN8z",
|
||||
"lHOpkIAUmEJzEBlJ1TE6Y8hy7jcS2ZkQkUY0uedCKsQn+hcJCKeCS4n09epAZyeXCZIcKf0xItH9DCsE",
|
||||
"DwXFhEn07f1s8S3C/hN6QAZTgTPIjtGfS0pRyRSh5nNmMjQh+qOiZPJYXxAebmZhBj7udf0jv9cndcnu",
|
||||
"mP7pY4cr1MBLlMzRehNe/z0DZvdhl6IHIyvvhuX+FzIXlV6gXt9u0n2rLKqwmMIGobFPmFSYpTAwN1vW",
|
||||
"ScyxE7eIUW52/RD19b9bzUxy4FprjTPAmtMg6f2dsy664hp51/F2jQ3rK6p4vMPx0ibfVIfMuhMkGOfa",
|
||||
"tOnmSRGG//bkJHmCc6MDBW6iofWUEDb4XXR/HvPrMV1HciveLr1ldxe0bcJT5LJfR++fY4vUMiSZOEve",
|
||||
"bvJz6u+/lSFje6LhjFgdlijI43Kw+wMWAi/iwt9BrB8dL05nKhgZNGXA0nUHASvzsYV+ZV2MIVZAyvMc",
|
||||
"mLG+BLDuazkQvFSwrLwMrCxVU2BmnLaYAoyttbOJ7o7QzoMrbt3hCI2rOna3TTvuEqlstBlYT9A5Zwoe",
|
||||
"VITijdluQijIkbUdRWxBl1jNpBY+zGikLz1RmhUj8yZSWtDwrydbEL4kjtqWpCWSg1Q4L4yOrgUSBg8K",
|
||||
"FZxSpGEHUrVd+BGFo5CWtKftO7wWJSAyQcd69PEC5/o7KSk07GRtZzHLLKddQGfGDb89lqDK4ljOdodZ",
|
||||
"7RpfMsFwxhVnJEWpRXdwmjmmTTZpAWsvZkNIV5AKsErWirIjV5f0hk2IJCmmSJoXkR6GsLF8kzEFpJy0",
|
||||
"mprZt4DDqv4io8t+ieVszLHIripT/hILOMOKHHljY/SyMVIzATkaL0ZaLTVki7OM6K1ietmYc/X1Jeuq",
|
||||
"FfOMQV9qoECGxgtk501a9Hn3cX/pH/jbTuFd/fRcnxDCbXhpKv3MzzQu0ztQdrIfBjlhpQLkr/CkofDo",
|
||||
"i7KO6yZC7ETdr7ngFtlA3W5e/0KMWpZ5LVjYD4R2P10c84lVc559P4sholItm+AKWlh8AVodiz7xelz0",
|
||||
"oVSYRhBu1sfHUuPU7IPTDPQJjVmlN34jtdIJqaaFFFuBIiZ4emVxM+Y6qpwrILPXvcWdVhWd2XFX9PEC",
|
||||
"GOozzgYCJKdzyI6c93YVn/5zK6ta2toKZ8dOmoD8+JaSyBkWp90GO8cgdsEEp/SDu2NXaG3GpfJexSZs",
|
||||
"zlJVYor8AGeqQGDmI2yKcpzOCIsycA5y5iyCzUmvbIyUfo7eXBphQAuo5vyaW8OSFZtalapNQUD38pTB",
|
||||
"/YDiQvHiaKOV0Ey7Dm4uciYmZ40KQeZYweguFrFz9upicHVx/uHievCni78Ojo+PzXYp15dnBqlYFG17",
|
||||
"NXOXY0rS+NR4Ct/p+ewYTaNm6qv3l1c1KSduUnPX98jez04Wbrvjf2JESxCYnpVq5q509OZlp5mtfLD1",
|
||||
"7O61GFFZeht5ghkZH/q6D7g3KhKzcgqyL24ijSUsJCsoj4OzHRRxMlPxcBClBBmXqnGOVa/tojy2WTJr",
|
||||
"VwBkzkqZoHtrr4OacTHnjChu3bPb2A795fPxcC5Do5ibuylqbDSLd/bWeyxRY4s7WxZzrC8ShlkKI2NZ",
|
||||
"3T0EwR+5rcdc5WOoxVX2WhwqHWNC2mxLO7kotnOXOmOTuw/N7qs5GuTcWE47x7Q6TBt80+a5wFOsdV5D",
|
||||
"3/oD30gUXhy5MLDIpzdirR07zZW8tAY8abUlQJRMIF2kVC/EGfdGS6rDKh6XdMVSKuOvR1qcCV57qMzP",
|
||||
"3e7CJpLaEXAZj/k8U4iC4TYWRIYJAZpJlLsVFgIkMHXcS7bA3TsQJhBxvoLDLoj7IpwbR/W1sSJVGLaK",
|
||||
"AeorgZk0Qmu1p7i40oKAa0cGLTAc1WNb6wv649X7P6MrD6qN9rvGy5FtZ1wDN/qIyJGnw7g5mOIFiLrx",
|
||||
"LweF7QUqsDVJlUJjZcrnIAz6jLI3ZaQ1/ikAuquZrxWhBRb68vbsttm4aGA6WuuUWY2HMuFcYO7PQkBq",
|
||||
"QlU/7nXgutPVIcZDuYmOsJKPa+lr4yk7WgnBfgzKCf6OCaYy6gBaoaRDktDOJLP+uI3jaT1CWvwxB8HH",
|
||||
"rrQZPaLmLs562W+0fUwMVvgRI2LqVoQa7fBe0rvHwpvoBVFano97IIxKGzdwyu4CVQg8CoKfNQwcC0wk",
|
||||
"ZFuEu7j7u2ZMcEuMklaION3Kd1aLIt8cWOVMGV3Hpw2fXue3uAab2jM0+JH8dltHyXd29AmcR6SlqztC",
|
||||
"KbJPUX9VZrJP3GFxFJOYBEhz4u7v4XtED50dXLsZNwN2naQu9qWeSLSzi7duhjvbeLG14c/O1meOH83H",
|
||||
"k0XtZzt4gomNvtDLy0a8NJZwfcNR+3fB9Q+jMU7v3G/6x5F77+M+Ls/aQiKS3dYx1CF4eltnaDi+Wg2c",
|
||||
"1SlWnawCDLbXc1SEJbBsuzrrNL7EitacbJyDOTeh+5BZF1mfF9ZofdRLtg5Xqqdgbrwc3Fxrox9fCVzM",
|
||||
"/kLgfhWGkE2hGQCxLkHqg0OgnJEi5oKp7FBtZvsdjUuReNqIm4xkaHBTnpw8h2DrchqptXkRltIyg/+y",
|
||||
"NGnMR85DDdFIR8azLYDj7H0RsChRstRnqsWd2fpTKMWFWdSMqIj/elnG5Db83WIwhvbXBgatvlTvumhs",
|
||||
"cMmFEPB5KLytWPcOHQkooBkBRnJ9Fvu8Or1kF+ilzAe2C4PsJv03jS0Ohl08mLINVf9xG7a4DZch7wBY",
|
||||
"+eWicP8T4/dU881rErlWutqpCbuDbBTloo1xIQKzu06BW+1CDSNFEbtEXpPpjJLpTKPGZF/7CJNOfGUj",
|
||||
"/jtnCCiiKKzZccWHGU/L3IaNiJKNOb8z1qA5SEWmbTlvm+3NdgExJL/1qv5LfWSvclTdFBs1VGTttsAt",
|
||||
"sa1A5ERLETu9HKyJEW3g00Tw/BR9UvwUfXKgkqfobwznkA0Mqybo+Pj44+fPnze6t4lPdjP3yoqxuraO",
|
||||
"GLzfgRIkvQLhIBy5bBZtildu3m2JIqS0LOqUJPB9L+l9N9P/tEQOGmlw3cWG59NO7LdFFm+OHzpNmRPW",
|
||||
"adw2FoaQVbJUuATfIwsL5LNFNnx2WbiUne6tcOmuE4+uzaBoSMXCKgSOCgLOK0TGFnFpI2G3M24UBSUg",
|
||||
"22Oxm2G1S2kOhErOEOX3INCYlyxLtMRW2CASmNv3bOL38IdeBKXNMfFbWatwpVg7ZBtHajAQ7CVuFRWs",
|
||||
"V579UmKBmSKsLTJ8iwzi2aLgagaS/N3mHvjF+0T1JYOluUDCmI/tifvroLmbu7NBSTXd10OqQUormK+p",
|
||||
"xevCOGvFNpZvr6VswlrWpxBcRG6KHwnQbGDyMGvhOMgOR/3vnz07ao/yM26++PHcpjkvwc7OEMZ3OVV8",
|
||||
"vs4G2omlGmwSSoLXoYfHvFSnY6rlsVrwQCnIZs3bfGatu+VSqx5yrQ1jnVP73ZuXCUq5AJkggfNRPk5Q",
|
||||
"RuTdaDpOECkSpCAvqAlGzE1MW4K02E5SkNGgRC4j8uIVLafenXsp+EPOH0xkmAu6qsUoRG0Z8Qiz12WO",
|
||||
"2UAAzvS5gpw/pGPkl/kufUhPfwZKFxPCtneV04c0QfM8QVygjKd3IEwVG0xYPbS6u7PcAW8DjtsCyqok",
|
||||
"6m72gxANGDU7BcMYevPSRBkInN6hwi+DsKn+ZSrA2N82XBPR67i3tIS1274KnLi0aX2yRCqdzUFgSu3B",
|
||||
"g8ikufBg+NzdAtDirD8vhQAWoiZaQzCkgmKd5GjWPcpBSjzt5jyeEEbkbH/782PYsEMAqiiZ84gZxSzg",
|
||||
"Qd5pNbPlclVQdLCG6FFrT8m1yQKOGT2+LHpiszRskxvO2Y2Oj8rzFz8sXUHAwC6dTbzhtLWxI2smaBNR",
|
||||
"zeU9MsXottEPSDZSfFfaWcaJhU5SGZ/dWVlb2yYUdYvy6oyY9TbzdnxsfK+bxS8OkE0wiMf5pFhkhGG6",
|
||||
"5LrmDAaKD7gJy3a/5Jhp4tH/Vc/8b+bhRtt5zPLBtFAK+4XYOEtSpwI0vW0zrze+H4/HqK+p+YWkAfUo",
|
||||
"3oi8O/ee0HhS0qj6ZIU25hBWFRnxP9pEN5G3nK4hiRVTrXK2KFebcNmCnzh8VjcSWUYMOK5w3iopG0N/",
|
||||
"R5ev15E6XqpCqpEEYFt56ycUF+uUwRmn2Sjj92zfWMJtq4RVoSFWgB8403dcq99635TcAV2MUlx25Ou8",
|
||||
"VHvHU/I0NULXeoPHAcJ0NoqClenQBdzg9M77ALxxwXxHb9uqqTZnqBKFPu5yyVcx9jZx8hClynxRsloM",
|
||||
"kJONVuC9zCZL1BPl5DtC6V42tc2BOCaPdlRZDjq+0bmq3zb2sVLuKVSvCRi0mfok29LgXwieQlaKmPjp",
|
||||
"ox4zZAThoY0fGfoAkKGPCyooZujD88Hvj1ZCsb3fbhQil9pq8DB9Dnp7ZL7iC9+8kXooUjzuwq27s4/d",
|
||||
"kOeV1ihi7tMV1W3nqSxcDzFXRBuSIfAnngjZ3V6aCTxxsQs+iCHYSQVMjE22psxtiEP2xlKxPu2ksmfv",
|
||||
"ZjpdSREJxtNgFK1YoPWMunLK53IUap5jFjGavOLBWGaKB7o4uV68KKirQLnMOESF6mYxd3rGSz3AmJlk",
|
||||
"XOjqnnciIF4ST+ltqNYUWIoX3UusaKW/GVst5ayX+Ko5Jh+ctVy6bXffFoCOF7757cnG0gdu3UlAd4xK",
|
||||
"rr1TagmAjOeYLlqkaSKgCirbIYJkVeDkmuOidldiHHOUMMACFYL/bD+doN9lyEb8bfYltvtNJeUxzemt",
|
||||
"/dyEKFSAQBlebO0UDG66ClrRwAwJaaklFJOQ4qoFABYgzspYruJ7pxYN5wTuQZwiPUxLT3fo/ZuX5+iP",
|
||||
"/31dj3clbHB2+Qb96x//ROc4yxY3bMLFPRbZAJe2FFsGE2ASBoQNMijULEGM27wwZ73REpoo1ezo+IaZ",
|
||||
"Et6nxixIUmTXaZNJbZn7KvO0b4p8oVsTKH2r3/Vl4A0xmTcrajesZAqKG5nWVSp3yQ+ukHymuOCrQW3n",
|
||||
"tuz6QN/lgPRmfYGV9+SOSzTjOVA8Ru+vjpEpWTchFHwNum+/DZu8YWaX336L+qaSO07VwMiFR6foFTce",
|
||||
"AxBIqnIsERaAqkYE90TNEMcFGehjbwosuWE271Wivv/8+ds3CZqUWipBP72RRxZeBsw4ByQLSI9v2A07",
|
||||
"52yu0clZTT55fnR6wwbownqh9Nd9mXd021ZU/vZYv/KWSCVRKQHdfjJ3dFLvjfH51i7eNdQo8JQw6/Dq",
|
||||
"u4MGmUYB6IeTBOX4AT07OTky8/7EJJ4Aunx/dW2LnxQK3S51UbhFfduPoaB4ge4Jy/i9fftdaQ4FJFzL",
|
||||
"EIlSLMQC3brb7vYFenVx7To5SHR7cY2ntwm6PLs+f418/Aa69Y0RblHftVTwrRTsZ0LNnQpmz58//z36",
|
||||
"6frcPL9wQUnmKc4yAVKadY2b0aWo3+ztYRB1PQP07vzSlgOZ4BRQXyoBODczvL6+vkwQn0xISjDVBHT1",
|
||||
"8k9HNoe4ZCYSXaHbYZ4WtzeMs4oQxoRhsUCYZXowL02FRMtLlq41y7ogoRemWqIpw4PuBS5uWEVPVj9G",
|
||||
"JqUGYemKLALLCk6YkpYfKUnBuWIck13a7G59XAvqGFOeDodO8z52nuShywKv+RF7lt3OLt/UpJbT3nfH",
|
||||
"J8cnRs0tgOGC9E57z49Pjp9bJ/DMnHdDc0gMcK01gLs0rRGIcPYm6532/ncJYtHsItDsHPO3eAuKWiH3",
|
||||
"NQ0zWt5tVH7fYYJ66Mbal2OSc7W5Yei70mGs68jRYaTrtNNhpG0n8vnjUmuOZycnWzWWWAoi9GpDJ/2h",
|
||||
"ifqIPlJv+7N91TKzhMgdvXLl+CUgYMrEcX1OKsEsvoUAs1oLj1okq+2NgcYww3NiamQYMzueSmPTtmGm",
|
||||
"Y+LNrg8Dv3BbS7N32rPigJl1GEqntHKSvhbOwqhOTBS0jgqXh6to38Y83oqz8sm9+wT86ngjtLJ5OrYI",
|
||||
"BLU/P2gCRbhGoZ4XqhJA2zDC8BPJPg9DwxgN6ybFb0BwaASmcVy4AJEmS700PTUCGpItv7DU/srSkomG",
|
||||
"+QPPFnuQUX3TIa3Vsqnl0kXgzKhqxnhL5G9L8ZnX787Oq7YTVjfoS8KmFAalhAT5vCknUg8kyWBzmaKw",
|
||||
"jTglNhtjfd6TEXftGfXSLRIJSLnIIDvEzWBxZUJ0gC3qsHTQbQVoV5YJXjfLNGVm4/3XyGBmSDfZq156",
|
||||
"fhfpy/YveEzBq03s42yXN1cS+P4j9O13sVWdL57watOL8OIe6iuJXl5cnR8dgr3NzNvLe02etZXy14p7",
|
||||
"53ZIJ6ZdEbs60n6I69iBWX0l9ZVXa5l9vy7Ktm0nno6oHUUcSFizTQ0ymBDm0l8qgnYVHjdIbG2SlY2B",
|
||||
"stB6QrFqIypdrFYneeS7w346il6bOH4A/NqZEHY47tvTZoDlIMMKJ8ibKX931BnnsePLCOn7yua+PEyT",
|
||||
"hEzVmB0pyDV3fVTSsVVtvrAk20o5vmHo/pRjZzKyK7GmVUdEuxJKozLKhgtvaWw3K0eoY/ClJU5fGX/V",
|
||||
"1rFVQ4Bf3SXZ7G/xhLflEjkd4Fh1M4LW7KziqGXLGSAXTchLOfBPkFHLkBKY0KNd7SHOKzW0JYwNaqLJ",
|
||||
"Lr5OpPR1ipPg7pIIM4SngO5gUWAiEtcF2fy9vfBsYjwateTYRtQXb6Q3HKNzTCkIWy8RUwE4W6AZnkO9",
|
||||
"1RMz1w6DTJ8ujewIE+hlHRzNU8FWND73dfkf4zRvFpv+wgf6UsXmWJNoMyIHpkJLdOsBtE0MWBYQdgD6",
|
||||
"th9DGDG498WN//WPfyIiZQmehjz91GgnLKGicke4LSRuWxQ2KPyTpOX08zCtmoREwzA+OA/j/YykM9cL",
|
||||
"xPT/SKxbzZKtKStt+2349hbItPkwRDwlc2BIeV+j8TIz5B3ApsGH7V7GpAKcIT5BU6JQUVIaI9JXoJoN",
|
||||
"TlburdgWEGd04RYnw+KIrNZlW4M/f/7cBLvF7z6bhblls/aPjymiNCARO5VdW5AMqMIHoNlXoBwZpPWZ",
|
||||
"HURxBc5tiTPZSaq9ouW09/ljhLJl1bVkurbI+MAEmBjroHnDOpMzX3jXTvuNXDmx2wnTN0x5dLz7D0Xw",
|
||||
"ftWp98qBVFsPuXVNXr4sMWS+BcywVgwnKgm/ArXSL+YREbfyrZiV3I9BfvH74+k9g4HgJcsGSpDChNRp",
|
||||
"wSfEAqU2SggJznMTEoQKPIU9fKz1gjatKogPMNl0hv9IqAJh6iPUmyW6QlwS6dHAMsyUbDu8d7Wwh4zB",
|
||||
"bV8MVVu3ftOX41374hK7l2P70NXq4cwE5gxdkGzsK7/saXr/t9KS2ouqfSHtyNXko0Qe7NSFinmCslMr",
|
||||
"V7WzIfHCX29fpSWxUdz/C5sSPRW12xJNrbQMbOWMi2s8bZvSDRuaMW7Cg9ggGVpRDjZQRdOC5AcPg8rY",
|
||||
"rgafO8223gSkUjtdf6t50JVTzqTWz03cp61KoTXnFBc4NSqwj/g+SpA3ZLnZrbuxqk5rIloiOnNTUdbH",
|
||||
"oN9dcLnHdIpQheDrpv2VOiBfmP5Xa1S0n3SuLGvSRMgvJZRPyiZhCwgj/VqpAun23/6f8wT95V2CQo2P",
|
||||
"I2QGmqId+/KTN963SaGB9B7R/NF2fDmcuXpAT4edV6GWwXKI8U6X3AHcJMtpD75HR/3UwQIiXUdqnWOq",
|
||||
"dgEZTF7cMEIpTDFtTGJjudH3J7/Xcq2ZblA9PzpGlzaKb6o/csPsgai10kX16nPU96dcgMtR9LzT29v1",
|
||||
"rHtkf0+9ecwXtw+2MYjz+FR361NxiHMYVeUtNIPUOsUsdZE5yKk1NF2uB1WX67Yj7A963Ac7rJM3ySTU",
|
||||
"NPSQAJ7nphQiycu8d/pDJJXrsbWJ5SDBwiYbtTSJ7VyVqa1Qkv3A1mVttojQmUxsnV2PWmvXngpczFBG",
|
||||
"XI20Qxi1fc6I/yCZWFOQO9gnmFD5RY/zVYL2AWhy84UcCq10o2gBdOdgvioRLsoRvTE3zBIquZjkPmNi",
|
||||
"ME8+Ht70vI/Kvb7I+850XJ/2EE7GlwboSNSnNX3Pgyetr6GLAnLk0RMSr7VtB5F6WOVyt1HxcoG1R7w+",
|
||||
"lz8Vwd5l0w8JhTuG3D4OIN9zSuNF7Iypc1no/1KorJmmTTdeuT5++GLu2rN2OHEePe8qahWtlQ35TwDw",
|
||||
"E9s2585n8lSmzbnN5D1g0O9rIhUXxtkNnhV2dkTMLbRM7mmrO/DKpgZcAVPIbugYXeB0Zr//jUS3JLv1",
|
||||
"WdG2B77g94hkqC9AljncMHOQ3b7VorKZYfDm5e1Rgm7N6KV3NVATdJthhcOTP169//MNM68iC+1j9Bqw",
|
||||
"UGPASp9buYGz5rwF+u4HeYz+AFINYDLhwrhhiXnyr3/884aZutKQoQLEQJZjvdMxCDQuJxMQCcoELwac",
|
||||
"ZiCVS6K+/O3RC5MG/eriGjmY3TDF0RindxMSd8VfGZi2HVatLpwAAVQImJCHfT02VsmqXmygYO0Mm9lW",
|
||||
"wYOy4BhUFNQ+4aof9uoCuRcPYfafewKyc6L+1dXF0T7MUUVHrfXTVcN2zYV89Aj5ryQj5d/r6ghZok94",
|
||||
"fVS0dSjHWJ1at44DTFqcHdczQDPMMgpi2TvRDzF8hgaPEhvDK52fYuiLHyY3DLMMAVEzEAiYsYa7ayFU",
|
||||
"Y+7bcFaXKHyEuKhFEN6wkDnobG/GB+ILQTRnIgzd+vZytyHq74xKjuDB/NUHudiIHsEpmAA0G45lp3v/",
|
||||
"57d/Rfd4YcdIvcXYVeA8Ehf1tOOv0n243A7uS7sQK45bwwree4L6uS1R6jLIgxPrEELWh0BAdepzpL1A",
|
||||
"//q//69KU7WJDfpPjmq3CrGtxR9WYzc7RGq09Hgm3074OIRdLIDYtY37DXIdNHc7o/ayJzSRMLQtIR8l",
|
||||
"6/vcTP30qDwPXS8P4Go3cyGM/OE6DL04Ub3uwo75xfpsFu0Jxhfm8RVAtrcxZ0l0MIWVuI2Va0Lvr2fv",
|
||||
"3qJa663VGq1Mccqnu7xq78itX1ySN8ICkto+wuRd5BANUTQu9QV/kMPVJwQgqSfWu5G2plXqWgi8/IPv",
|
||||
"9P/yAxoiVxLIR+I1UsUWUkHeiXiMPX/dqWqaeG5S1q4UFsET26/7YY9eIJ4TZYxp9zMtMFgPQt+2MGqL",
|
||||
"vhOc7yTUr3EQPdvgIEpMjVJqCi1aibWzvb57bVKpFqa204SLvLcalrfUKfRnTpj3g4zc37T4VvCipEbC",
|
||||
"Cy1Wj12/x6TLJtxn4nsINRmXOyd03dVjxq9XLWUjHGkeorl5ujdDXpVjS6macudElpiSv7tabqYHKvoN",
|
||||
"Mj1QdzDva8arepy2cd6PFEC99mh9NJA227VGwGoHHDC22GzMtcr101qzvunohgjL9Fa42MeMFwptDyVg",
|
||||
"kbZD+so8Dr05uxksfuktKwH7WQG+At2+0Z30cP6310QdQlH/saR0YPJHLDptkdeA5MpL3fcigEyQ6/jZ",
|
||||
"YNHwylY09Cl4PzrEZNVp6deGzrem/WwF+ENU7KA0yG1y6HGGbKNbpHg0RrUrGuPcbBrNRj1b3ZnaaEa2",
|
||||
"KO0G79w7N6jTydLZzdbhng8lcw8iqsTlNN8yMxJ/gEvFa/EHzQaqtsXHbgnXuzjxnvJobfSoPRwv2mmR",
|
||||
"hAPVTDTEajLyBnZOlAfK3fUarit6bafm+0oXOyCKHCNpKG6b92EaJEVYIgTLdZ+w0Q461pG7Fkey5VpX",
|
||||
"WjptoqwGRKKfbmyxCwkG1B3mhkeeXIxZuZa7pS/z2nL932qrrZGoX1N36rQRM4tBQEDcpF8PqUVYuhTm",
|
||||
"UV4qpxgEw7vmHDwI5tD7GTBURdiuWMPriTTXVrv8ipNp9AqfMqHGEvvaAj3PTp51oENrJK9X+tzbaKu0",
|
||||
"AqNmUFGyUWxszn6NoLvTa9NeE6XY4Sd9HcdK/USkHZfit4Wg0xre/hqLDGVAQZkC8IwrJMui4MJUcZ+Z",
|
||||
"uvCum65E8ECkrVcQ+oGEFH4bU/DyeYQ1aqHnu3HGFwk/10t7whD0No6oFR56Io6oFSwKWK9CJffhBFeV",
|
||||
"eH0gwqUftI3svUdxzR0jCrrFP/yaIgl8G/2niyMIpHGgKIKiIjVPzxRcJ7nNkoh/+6A12+KdTpDEE1AL",
|
||||
"NMd0Du7ovXr1u6NjdBYKfOvjvKhLOyuiztX3bYf1ZehF/+VP6iZJtpZa/qXEAjNlGlVFO/Ks9ryqGv7X",
|
||||
"ml3VOlsZP1IYE1Nrn7bMcmC4r/GWuK4SkrBnJNR39eYBDVEFWzSsbpKj7ry2dHe4OJuQ/FZS6Fbc/0NJ",
|
||||
"Qfa+grr0eiGHTJMw+zpwnXkkyqZqVrlXdwioOg/KVaNBQfjaCyN3608i1xzSFnHCEwViNa/bn3zPW/Wx",
|
||||
"Bqi/Uo2svsZtdLInYfNLG//gtKEGlaB+VmI6iDiz19JMB7Z+7Cqo+1HJI2sn/6bkYSjCsfchCcPFVa4z",
|
||||
"Rp65MVegFGHTpz3rm2s54HEfdneIgut2kUi6OVH/jlA6kPdEpbMEMZiDGPiaq6aizdEOV0Jcpv2AiTRx",
|
||||
"jn4RRKI6vVDIUP/ZyTP0myoU8hi95fdgih8RZdMf3NLR7ZTyMabHeroRTtUpuunxyeSmd6s1WJzZmEq7",
|
||||
"pZEfhO7AZVH4a4fkOWQEK6AL/fWTo1NzNdXAYktxmnnQPXbxMZitLzpiTpsYee52bujt6EeYXjZotM0z",
|
||||
"9Hhy69fJI2cGmzZhRwliQ7WfUEgOx6OndV+dsnFCvmiQ2fsff9QsEQhyv/NTEHk3MAG/G6TlD0Tenbtx",
|
||||
"T5lS7JdxSEGZyDvkYXAgeVnU59zyaNToaSQj20OSglV9lwv2Zo00627JOaal5LYBL2uTdfaeqRaGt5Uh",
|
||||
"e4kUv49VsK25mYAdpufQBcu0VFOfui9BSVsGZqS41V1MJAuR6A4KeyPMTF7j4miXshxtatSFa1lpfWiR",
|
||||
"QjSrfkHUnxEQWKSzxQDfYwFHL1CKRUYYprZt34SLFLI2RWo9zX0dilR9jU/j3GoWQPgi/ScaFOkilnaq",
|
||||
"/+J7Dqy7FK7cmM4ZgXCwTPPQU5tNeC/p3TtTUdJLBVEkjfYaf5Q8+C6NgH5NZn6L8ye08nuiO1Tl4kDD",
|
||||
"27XiqfGITaHB6d2j5M+cpXcO5nGsr9+5ffVw7UrO0ipCEzvg7dippAG9vLTSzcHB965UUIPfIZwQeq2j",
|
||||
"kilCu1aAb+0RudwRv5p5jyaOX5YiNIADKbgSK9fXbw9BFAIkp/PHoYsPdu4Dk0Y7mleQ+VUgz0Ghwl+O",
|
||||
"WYkpXeyKPq2qbhAa7JBurTGt+WVkok7/471/zGtdY+Upb3VLFYe61M1sqG9SqlRIrCtA2EdHu7n07bSP",
|
||||
"7X6wqPiKfe0FYQyykYNqvCbiqrtdY6LV1/71edcdQ3z1vnVDk6bBE9G/eqTs6EavUfjQTdXhNP+LH/n1",
|
||||
"HWA7H0hhT/vjy03lbT+mtqDF2/bH0J55/yYdT3ZNfro2o7c+iv7dkjnMNg9IOg5sh2B0YBnCDNOFJK58",
|
||||
"IaU+h8NUJo8kUm2T0PGYyVR6K5CWxnSjpx4DFiDOSjXrnf7to8a47cZuP1wK2jvtDXFBhvPvDD24/ay2",
|
||||
"bXLJ/S7vPOQVmJKzphRO3Xbe3IZNq1mJQ7Gd1yC0fkuq3lZE2jrLhLPEtzmqFY5yvYxW57zYLtXBzcer",
|
||||
"7ItPcbuH2aIrLtS3qDY+oEZ3zuiCQg2KqreC69SYBC+lRP0MUpLBEKeqNi3USzR9aom7NEsLopc+z2oz",
|
||||
"hPNt9f26AyZZCjVKgnOsmsp5UVYnChmSjjRcnnBlq6vlOH6Kpl7JxGYsm+9mRCWu+mCCQja+x1SDy2Lg",
|
||||
"LrhQq++5Sg6fP37+/wEAAP//VvsxiT/wAAA=",
|
||||
"H4sIAAAAAAAC/+x963IbudXgq6C4WzXUpCn6MpPsyPX9UGSN7cSOtZYm36ZGLgrsPiQxQgM9AJoS43JV",
|
||||
"fu0DbOUJ8yRbuHY3iSabF1nOVP7YkhqNBs4FOPfzqZfyvOAMmJK9k0+9GeAMhPnx/ApP9f8ZyFSQQhHO",
|
||||
"eie9DyB5KVJAcxCScIYmXKA3k8E7rNJZL+nJdAY51u+pRQG9k55UgrBp7/Pnz0mvwALnoNwHzkohuVj9",
|
||||
"xPsC/1oCSs1jNBE8RxgVAuaElxIJkAVnEr6RiMG9GtlhvaRH9Lu/liAWvaTHcK4/Hh62LyvpnTNF1OJN",
|
||||
"trqSn3568xJxgSQtp6gPx9NjdDPjUp3MyrEg8ubIf7bAalZ9lWS9pCfg15IIyHonSpTQZQWXtIwA3D5r",
|
||||
"LOFOnuQ4HeSEkZsE3dD79CTFWbZoW49+d8sV/Sh4fkX025+igNVYaYB1wkWOVe+kl2EFA6VfTSLzvskg",
|
||||
"L7gCli7+DIvV3Z5RAkwNpsBAYAUZuoXFCySgoHgh0R1RM8LQs+9mSIAqBUNqBogLMiUM00AaHgqWmKtF",
|
||||
"1z4+0F+vrz/H92+BTdWsd/L02f+KLn1iaXwVQ1d4WpEp4QK9Or96gb57+gxxFvgkJzJ3PBJfXMVD2yDq",
|
||||
"LcmJasMSNQ/rE2QwwSVVvZPvnyR6zyQv897Jsyf6N8Lsb0/D7glTMAVhPnTF19GD4ttTw2e9U4sxcx5c",
|
||||
"AMsIm54WheBzTPWfUs4UMLM/XBSUpFjDfPiL1ID/VPvg/xQw6Z30/sewOs6G9qkchgnNJ5fobYbZFJBU",
|
||||
"eArZC4RRDgoPsHsD3WGJUgGGEvtZielAr0hwetT7nPQuBB9TyNcstLAjfrfdgv28kfWeC8EF6n/48Qz9",
|
||||
"8N33fzDLuCRThulPhYZ1djCo2Vlja3BfQtKP8Jg3WDydAlOnqSJzogyDF4IXIBSxSMbuychSw6ceME1z",
|
||||
"P/cU53SUYkoNA2DJmaYS/e2UaAbqJb08LUae7kCmmJp99T6ukFbSw3oVI5JFmCbppVwIsC+7IaykFI8p",
|
||||
"eI5beSUrhR2fyzXjA78kPTDHdtfpGwutzUJYUaqRLPMci0WnmXiptn1FgpRbgEKWaQpyHRjGnFPATA9W",
|
||||
"/BbYKOWlJcfNcDNkYA+VDmuxQkvHu6c6Vn+2V7SSvRqlJEu0WZEVH/8CqdLfq59Nq3Rt2WuV3OwBMsKq",
|
||||
"62It1Wfr39lMs26OcTcygPuCCJBbLdOSTBhblhauy8NuCcvqvA73kJbKMnXBKUkXg9QcxPp3rBQINjDI",
|
||||
"aGfwAi8oxxGZzYhIGjdcQobs7Cgjk0k7yCr8CiJvRynFlrxXSd9JaKsPFFalrG+xsJeZpipDM5CZs4wR",
|
||||
"84OFtRUT5/wWsugeZWkXtlYm1BJQuK9SzlIQTG4mjxg/ODnRkXIDGg6HYacNcmmQ+Dq2+VBS2Ip1cKk4",
|
||||
"4/liRGEOtA5f/aS6Bgw/wBxEFI7uLPY3ThOW7/ACjQHhsVQCpwr1CZuBIEqijN+xoy6M1pELNhFXygsY",
|
||||
"2bWuR7lWuQoQAzsW8TkIQTKQXdbqxNHYdRMjiTgtLKGlmnUT8s8MnTw+CeyLm/XM1AloUVCVGVHnTInF",
|
||||
"diBKFRddr287eFn6MrdgL+npL2JlVeaFVOCVvKykLZDdRZgChQmtbaWCwGHEphzUjHebwmjKncQeY/YY",
|
||||
"kaLbaHNMjlKeQUe55wCSTIXZwLhxKrOEeAlK6RlXSO3WauZwj/NCr7k3pXyM6bGm4BFOVexwK61SsJX0",
|
||||
"MMe0jLNj91Pq1ujxdqb159DZDNLb1c2mnE1IxO7yV0yJ1XP0Uetuvwi9arTWybAm/Ha8F/TWxBzTkYxT",
|
||||
"87L0NFOq0POk+t+MyFt9AYNQA3Mla3Bkgkw0lgorgUg5G9itRTmYYqlGM8BURYwbVzMiUapB941E/I6h",
|
||||
"nEuFBKTAFJqDyEiqjtEpQ5Zzv5HIzoSINKLJHRdSIT7Rv0hAOBVcSqSvVwc6O7lMkORI6Y8Rie5mWCG4",
|
||||
"LygmTKJv72aLbxH2n9ADMpgKnEF2jP5SUopKpgg1nzOToQnRHxUlk8f6gvBwMwsz8HGv6x/5nT6pS3bL",
|
||||
"9E8fO1yhBl6iZI7Wm/D67xkwuw+7FD0YWXk3LPe/kLmo9AL1+naT7ltlUYXFFDYIjX3CpMIshYG52bJO",
|
||||
"Yo6duEWMcrPrh6iv/91qZpID11prnAHWnAZJ7++cddEV18i7jrdrbFhfUcXjHY6XNvmmOmTWnSDBONem",
|
||||
"TTdPijD890+eJI9wbnSgwE00tJ4SwgafRvfnMb8e03Ukt+Ltwlt2d0HbJjxFLvt19P45tkgtQ5KJs+Tt",
|
||||
"Jj+n/v5bGTK2JxrOiNVhiYI8Lge7P2Ah8CIu/B3E+tHx4nSmgpFBUwYsXXcQsDIfW+hX1sUYYgWkPM+B",
|
||||
"GetLAOu+lgPBSwXLysvAylI1BWbGaYspwNhaO5vobgntPLji1h2O0LiqY3fbtOMukcpGm4H1BJ1xpuBe",
|
||||
"RSjemO0mhIIcWdtRxBZ0gdVMauHDjEb60hOlWTEybyKlBQ3/erIF4UviqG1JWiI5SIXzwujoWiBhcK9Q",
|
||||
"wSlFGnYgVduFH1E4CmlJe9q+wytRAiITdKxHHy9wrr+TkkLDTtZ2FrPMctoFdGbc8NtjCaosjuVsd5jV",
|
||||
"rvElEwxnXHFGUpRadAenmWPaZJMWsPZiNoR0CakAq2StKDtydUlv2IRIkmKKpHkR6WEIG8s3GVNAykmr",
|
||||
"qZl9Czis6i8yuuyXWM7GHIvssjLlL7GAM6zIkTc2Ri8bIzUTkKPxYqTVUkO2OMuI3iqmF405V19fsq5a",
|
||||
"Mc8Y9KUGCmRovEB23qRFn3cf95f+gb/tFN7VT8/1CSHchpem0s/8TOMyvQVlJ/t+kBNWKkD+Ck8aCo++",
|
||||
"KOu4biLETtT9mgtukQ3U7eb1L8SoZZnXgoX9QGj308Uxn1g159l3sxgiKtWyCa6ghcUXoNWx6BOvx0Uf",
|
||||
"SoVpBOFmfXwsNU7NPjjNQJ/QmFV64zdSK52QalpIsRUoYoKnVxY3Y66jyrkCMnvdW9xpVdGZHXdFHy+A",
|
||||
"oT7jbCBAcjqH7Mh5b1fx6T+3sqqlra1wduykCciPbymJnGFx2m2wcwxi50xwSj+4O3aF1mZcKu9VbMLm",
|
||||
"NFUlpsgPcKYKBGY+wqYox+mMsCgD5yBnziLYnPTSxkjp5+jNhREGtIBqzq+5NSxZsalVqdoUBHQnTxjc",
|
||||
"DSguFC+ONloJzbTr4OYiZ2Jy1qgQZI4VjG5jETunr84Hl+dnH86vBn8+/9vg+PjYbJdyfXlmkIpF0bZX",
|
||||
"M3c5piSNT42n8FTPZ8doGjVTX76/uKxJOXGTmru+R/Z+drJw2x3/EyNagsD0tFQzd6WjNy87zWzlg61n",
|
||||
"d69twtsSiJIVfMRxquKxF0oJMi5V49CoXttFU2szG9bOW8icSTBBd9Y4BjVLXs4ZUdz6Qrcx1PmT/uPh",
|
||||
"/HNGCzYXQdSyZxbvjJt3WKLGFnc24+VYn9oMsxRGxoy5u7/fn2+tZ0pl0K8FMfZavBcdAzDaDDk7+QO2",
|
||||
"8006y467fMzuqzka5NxYTjvHtHonG3zT5ibAU6wVTEPf+gPfSBReHLmYq8inN2KtHTvNlby01jJpVRNA",
|
||||
"lEwgXaRUL8RZ0kZLcvoqHpcUs1Iq4xxHWnYILnKobL3dLp4mktoRcBEPsDxViILhNhbu5wkBmkmUuxUW",
|
||||
"AiQwddxLtsDdOxAm6m++gsMuiPsinBtH9ZUx2VQYtlI46iuBmTQSYrWnuGzQgoArRwYtMBzVA0nrC/rT",
|
||||
"5fu/oEsPqo3GssbLkW1nXAM3+ojIkafDuO2V4gWIuqUtB4WNeX8isLX/lEJjZcrnIAz6jGY1ZaQ12CgA",
|
||||
"uqtNrRWhBRZaaPDsttmSZ2A6WusBWQ0+MrFTYO7PQkBq4kI/7nXgutPVIcZDuYmOsJKPa+lr4yk7Wol3",
|
||||
"fgjKCc6FCaYy6m1ZoaRDktDOJLP+uI3jaT1CWpwfB8HHrrQZPaLmLqh52UmzfQAKVvgBw0/qKnuNdngv",
|
||||
"6d1h4e3hgiitHsTN/UZ/jFsTZXeBKkT5BMHPauHHAhMJ2RaxJe7+rmnubolR0grhnVs5qmoh25ujmJzd",
|
||||
"oOv4tOFA6/wW12BTe8bhPpCTbOuQ9M5eNYHziLR0eUsoRfYp6q/KTPaJOyyOYhKTAGlO3P3daQ/oDrOD",
|
||||
"azfjZsCuk9TFvtQTCS12wc3N2GIbnLU21tgZ1szxo/l4sqj9bAdPMLGhDnp52YiXxuysbzhq/y64/mE0",
|
||||
"xumt+03/OHLvfdzHv1hbSESy2zpgOUQqb+t5DMdXqzWxOsWqk1WAwfZ6joqwBJZtV2edxpdY0dpujScu",
|
||||
"5yZOHjLrj+rzwlqIj3rJ1rFB9XzHjZeDm2ttqOErgYvZXwncrcIQsik0ow3WZSN9cAiUM1LE/B2VHarN",
|
||||
"Rr6jcSkSvBrxSZEMDa7LJ0+eQ7B1OY3U2rwIS2mZwX9ZmjTmI+cOhmhYIePZFsBx9r4IWJQoWerTwuKe",
|
||||
"Y/0plOLCLGpGVMRZvCxjchtrbjEYQ/trA4NWx6X3EzQ2uGSvD/g8FN5WrHuHDrsT0Ay3Irk+i30Sm16y",
|
||||
"i6pS5gPbxRx2k/6bxhYHwy7uQtmGqv/46Fp8dMuQdwCsnGBRuP+Z8Tuq+eY1iVwrXe3UhN1CNopy0cYg",
|
||||
"DIHZbacoqXahhpGiiF0ir8l0Rsl0plFjUp19OEcnvrLh9Z3D8RVRFNbsuOLDjKdlbmM0RMnGnN8aa9Ac",
|
||||
"pCLTtgSzzfZmu4AYkt96Vf+lPrJXOapuio0aKrJ2W+CW2FYgcqKliJ1eDtbEiDbwaSJ4foI+KX6CPjlQ",
|
||||
"yRP0M8M5ZAPDqgk6Pj7++Pnz542+ZOIzy8y9smKsrq0jBu93oARJL0E4CEcum0Wb4pWbd1tC9igtizol",
|
||||
"CXzXS3pPZ/qfljA9Iw2uu9jwfNqJ/bZImc3xfacpc8I6jdvGwhBSOJaqhOA7ZGGBfGrGhs8uC5ey070V",
|
||||
"Lt114tGVGRSNX1hYhcBRQcB5hcjYIi5s2Ol2xo2ioARke+BzM4Z1KaeAUMkZovwOBBrzkmWJltgKG7EB",
|
||||
"c/uezbIeft+LoLQ5Jn4raxWuFGuHbONIDQaCvcStooL1yrNfSywwU4S1hWFvka47WxRczUCSv9tAf794",
|
||||
"nxW+ZLA0F0gY87E9S34dNHdzdzYoqab7ekg1SGkF8zW1eF3MZK2yxfLttZS6V0uxFIKLyE3xIwGaDUzS",
|
||||
"Yy32BdnhqP/ds2dH7SF1xs0XP57bNOcl2NkZwvgup4pPjtlAO7G4/k1CSfA69PCYl+pkTLU8VgseKAXZ",
|
||||
"rHmbz6x1t1xo1UOutWGsc2q/e/MyQSkXIBMkcD7KxwnKiLwdTccJIkWCFOQFNZF/uQkgS5AW20kKMhoB",
|
||||
"yGVEXryk5dS7cy8Ev8/5vQnDchFOtRiFqC0jHs71uswxGwjAmT5XkPOHdAyzMt+l9+nJL0DpYkLY9q5y",
|
||||
"ep8maJ4niAuU8fQWhCkZgwmrxzF3d5Y74G3AcVv0VpWx3M1+EELvomanYBhDb16aKAOB01tU+GUQNtW/",
|
||||
"TAUY+9uGayJ6HfeWlrB225eBE5c2rU+WSFmxOQhMqT14EJk0Fx4Mn7tbAFqc9WelEMBC1ERrCIZUUKyT",
|
||||
"HM26RzlIiafdnMcTwoic7W9/fggbdoj2FCVzHjGjmAU8yFutZrZcrgqKDtYQPWrtKbk2Mt8xo8eXRU9s",
|
||||
"loZtcsM5u9HxUXn+4oelq74X2KWziTectjZ2ZM0EbSKqubxHpvLbNvoByUaK70o7yzix0Ekq47M7K2tr",
|
||||
"24SiblFenRGz3mbejo+N73Wz+MUBsgkG8TifFIuMMEyXXNecwUDxATcx0O6XHDNNPPq/6pn/zTzcaDuP",
|
||||
"WT6YFkphvxAbZ0nqVO2lt22a88b34/EY9TU1v5A0oB7FG5G3Z94TGs8AGlWfrNDGHMKqih7+R5tVJvKW",
|
||||
"0zVkjGKqVc4W5WoTLlvwE4fP6kYiy4gBx1WpWyVlY+jv6PL1OlLHS1VINZIAbCtv/YTiYp0yOOM0G2X8",
|
||||
"ju0bS7htSa4qNMQK8ANn+o5r9Vvvm5JboItRisuOfJ2Xau94Sp6mRuhab/A4QJjORlGwMh26gBuc3nof",
|
||||
"gDcumO/obVs11SboVKLQx10u+SrG3mYpHqIumK8AVosBcrLRCryX2WSJeqKcfEso3cumtjkQxyStjirL",
|
||||
"Qcc3OpfQ28Y+Vso9heo1AYM2LZ5kWxr8C8FTyEoREz991GOGjCA8tPEjQx8AMvRxQQXFDH14PvjhaCUU",
|
||||
"2/vtRiFyqa3gDdPnoLdH5iu+8M0bqYcixeMu3Lo7+9gNeV5qjSLmPl1R3XaeysL1EHNFtCEZAn/iWYfd",
|
||||
"7aWZwBMXu+CDGIKdVMDE2GRrytyGOGRvLBXr004qe/ZuptOVFJFgPA1G0YoFWs+oS6d8Lkeh5jlmEaPJ",
|
||||
"Kx6MZaZSn4uT68UrcLpyj8uMQ1QoJRZzp2e81AOMmUnGha7ueScC4vXnlN6Gas03pXjRvZ6JVvqbsdVS",
|
||||
"znqJL1Fjkq9Zy6XbdvdtAeh4lZnfP9lYZ8CtOwnojlHJlXdKLQGQ8RzTRYs0TQRUQWU7RJCsCpxcc1zU",
|
||||
"7kqMY44SBligQvBf7KcT9IcM2Yi/zb7Edr+ppDymOb21n5sQhQoQKMOLrZ2CwU1XQSsamCEhLbWEYhJS",
|
||||
"XGo+YAHitIzlKr53atFwTuAOxAnSw7T0dIvev3l5hv7031f1eFfCBqcXb9C//vFPdIazbHHNJlzcYZEN",
|
||||
"cGnrnmUwASZhQNggg0LNEsS4zQtz1hstoYlSzY6Or5mpl31izIIkRXadyJRNtjXlqzzSvqmohW5MoPSN",
|
||||
"ftfXXDfEZN6sqN2wkqnebWRaVxbcJT+4qu2Z4oKvBrWd2RrnA32XA9Kb9dVM3pNbLtGM50DxGL2/PEam",
|
||||
"PtyEUPAF3779Nmzympldfvst6puy6ThVAyMXHp2gV9x4DEAgqcqxRFgAqqr+3xE1QxwXZKCPvSmw5JrZ",
|
||||
"xFmJ+v7zZ2/fJGhSaqkE/fRGHll4GTDjHJAsID2+ZtfsjLO5RidnNfnk+dHJNRugc+uF0l/3NdXRTVsF",
|
||||
"95tj/cpbIpVEpQR088nc0Um9EcXnG7t4172iwFPCrMOr7w4aZKryo++fJCjH9+jZkydHZt6fmMQTQBfv",
|
||||
"L69spZFCoZullgU3qG+bHxQUL9AdYRm/s2+/K82hgITrzyFRioVYoBt32928QK/Or1zbBIluzq/w9CZB",
|
||||
"F6dXZ6+Rj99AN74LwQ3qu/4Fvm+B/UwocFPB7Pnz5z+gn67OzPNzF5RknuIsEyClWde4GV2K+s1GGgZR",
|
||||
"VzNA784ubO2NCU4B9aUSgHMzw+urq4sE8cmEpARTTUCXL/98ZHOIS2Yi0RW6GeZpcXPNOKsIYUwYFguE",
|
||||
"WaYH89KUI7S8ZOlas6wLEnphShOamjfoTuDimlX0ZPVjZFJqEJauoiGwrOCEKWn5kZIUnCvGMdmFzdbW",
|
||||
"x7WgjjHlyXDoNO9j50keuqzumh+xZ9nt9OJNTWo56T09fnL8xKi5BTBckN5J7/nxk+Pn1gk8M+fd0BwS",
|
||||
"A1yrw+8uTWsEIpy9yXonvf9dglg0S/Y327T8HO/3UKuavqY7Rcu7jTLrO0xQD91Y+3JMcq42NwxNTjqM",
|
||||
"de0vOox0bW06jLS9Oz5/XOqD8ezJk626OCwFEXq1oZP+0ER9RB+p99jZvkSYWULkjl65cvwSEDBl4rg+",
|
||||
"J5VgFt9CgFmtX0YtktU2okBjmOE5MQUpjJkdT6Wxadsw0zHxZtf7gV+4LVzZO+lZccDMOgx1Slo5SV8L",
|
||||
"p2FUJyYKWkeFy8OVj29jHm/FWfnk3kX5f3O8EfrGPB5bBILanx80gSJco1DPC1W9nW0YYfiJZJ+HoTuL",
|
||||
"hnWT4jcgOHTd0jguXIBIk6VemgYWAQ3Jll9Y6jVlaclEw/yRZ4s9yKi+6ZDWatnUcukicGZUNWO8JfLX",
|
||||
"iP2RmJZ3p2dVjwerG/QlYVMKg1JCgnzelBOpB5JksLkmUNhGnBKbXag+78mIuzZoeukWiQSkXGSQHeJm",
|
||||
"sLgyITrAFnVYOui2ArQrywSvm2WaMrPx/mtkMDOkm+xVr/O+i/RlmwU8pODVJvZxtsubKwl8/xH69rvY",
|
||||
"qjYTj3i16UV4cQ/1lUQvzy/Pjg7B3mbm7eW9Js/asvRrxb0zO6QT066IXR1pP8R17MCsvmz5yqu1zL7f",
|
||||
"FmXbHg+PR9SOIg4krNkOAhlMCHPpLxVBu3KKGyS2NsnKxkBZaD2iWLURlS5Wq5M88vSwn46i1yaOHwC/",
|
||||
"diaEHY779rQZYDnIsMIJ8mbKPxx1xnns+DJC+r6yuS8P0yQhUzVmRwpynVQflHRsVZsvLMm2Uo7vzrk/",
|
||||
"5diZjOxKrGnVEdGuhNKojLLhwlsa283KEeoYfGmJ05ehX7V1bFV9/zd3STabSTzibblETgc4Vt2MoDU7",
|
||||
"qzhq2XIGyEUT8lIO/BNk1DKkBCb0aFd7iPNKDW29YIOaaLKLrxMpfVHgJLi7JMIM4SmgW1gUmIjEtRw2",
|
||||
"f6/cf64OcSgknBiPRi05thH1xRvpDcfoDFMKwtZLxFQAzhZohudQ76vEzLXDINOnSyM7wgR6WQdH81Sw",
|
||||
"5YPPfBH8hzjNm5Wdv/CBvlQeOdaR2YzIganQf9x6AG3HAJYFhB2Avu3HEEYM7nwl4X/945+ISFmCpyFP",
|
||||
"PzXaCUuoqNwRbguJ236ADQr/JGk5/TxMq44c0TCMD87DeDcj6cw13jDNNhLrVrNka2o42+YWvpcEMj01",
|
||||
"DBFPyRwYUt7XaLzMDHkHsOmmYVuFMakAZ4hP0JQoVJSUxoj0FahmN5GVeyu2BcQZXbjFybA4Iqt12T7c",
|
||||
"z58//yE03l+5+2wW5pad0T8+pIjSgETsVHY9ODKgCh+AZl+BcmSQ1md2EMUVOLclzmQnqfaSltPe548R",
|
||||
"ypZVi5Dp2oreAxNgYqyD5g3rTM584V077Tdy5cRuJ0zfneTB8e4/FMH7ZadGJwdSbT3k1nVU+bLEkPl+",
|
||||
"K8NaMZyoJPwK1EpzlgdE3Mq3YlZyPwb5xe+Pp/cMBoKXLBsoQQoTUqcFnxALlNooISQ4z01IECrwFPbw",
|
||||
"sdYL2rSqID7AZNMZ/iOhCoSpj1DvTOgKcUmkRwPLMFOy7fDe1cIeMga3fTFUbd36TV+Od+2LS+xeju1D",
|
||||
"V6uHMxOYM3RBsrGv/Lqn6f3fSktqL6r2hbQjV5OPEnmwUxcq5gnKTq1c1c6GxHN/vX2VlsRGcf8vbEr0",
|
||||
"VNRuSzS10jKwlTPOr/C0bUo3bGjGuAkPYoNkaEU52EAVTQuSHzwMKmO7GnzmNNt6E5BK7XTNpOZBV045",
|
||||
"k1o/N3GftiqF1pxTXODUqMA+4vsoQd6Q5Wa37saqOq2JaInozE1FWR+DfnfB5R7TKUIVgq+b9lfqgHxh",
|
||||
"+l+tUdF+0rmyrEkTIb+WUD4qm4QtIIz0a6UKpNt/+3/OEvTXdwkKNT6OkBloinbsy0/eeN8mhQbSe0Dz",
|
||||
"R9vx5XDm6gE9HnZehVoGyyHGO11yB3CTLKc9+B4d9VMHC4h0Hal1jqnaBWQweXHNCKUwxbQxiY3lRt89",
|
||||
"+UHLtWa6QfX86Bhd2Ci+qf7INbMHotZKF9Wrz1Hfn3IBLkfR805vb9ez7oH9PfXmMV/cPtjGIM7jU92t",
|
||||
"j8UhzmFUlbfQDFLrFLPUReYgp9bQtJQeVC2l246wP+pxH+ywTt4kk1DT0EMCeJ6bUogkL/PeyfeRVK6H",
|
||||
"1iaWgwQLm2zU0pG1c1WmtkJJ9gNbl7XZIkJnMrF1dj1qrV17KnAxQxlxNdIOYdT2OSP+g2RiTUHuYJ9g",
|
||||
"QuUXPc5XCdoHoMnNF3IotNKNogXQnYP5qkS4KEf0xtwwS6jkYpL7jInBPPl4eNPzPir3+iLvO9NxfdpD",
|
||||
"OBlfGqAjUZ/WNBkPnrS+hi4KyJFHj0i81rYdROphlcvdRsXLBdYe8Ppc/lQEexdNPyQU7hhy+ziAfM8p",
|
||||
"jRexM6bOZaH/S6GyZpo2rW/l+vjh87nrhdrhxHnwvKuoVbRWNuQ/AcCPbNucO5/JY5k25zaT94BBv6+J",
|
||||
"VFwYZzd4VtjZETG30DK5p63uwEubGnAJTCG7oWN0jtOZ/f43Et2Q7MZnRduG84LfIZKhvgBZ5nDNzEF2",
|
||||
"81aLymaGwZuXN0cJujGjl97VQE3QTYYVDk/+dPn+L9fMvIostI/Ra8BCjQErfW7lBs6a8xbo6ffyGP0R",
|
||||
"pBrAZMKFccMS8+Rf//jnNTN1pSFDBYiBLMd6p2MQaFxOJiASlAleDDjNQCqXRH3x+6MXJg361fkVcjC7",
|
||||
"ZoqjMU5vJyTuir80MG07rFpdOAECqBAwIff7emysklW92EDB2hk2s62Ce2XBMagoqH3CVT/s5TlyLx7C",
|
||||
"7D/3BGTnRP3Ly/OjfZijio5a66erhu2aC/ngEfJfSUbKv9fVEbJEH/H6qGjrUI6xOrVuHQeYtDg7rmaA",
|
||||
"ZphlFMSyd6IfYvgMDR4lNoZXOj/F0Bc/TK4ZZhkComYgEDBjDXfXQqjG3LfhrC5R+AhxUYsgvGYhc9DZ",
|
||||
"3owPxBeCaM5EGLrx7eVuQtTfKZUcwb35qw9ysRE9glMwAWg2HMtO9/4vb/+G7vDCjpF6i7GrwHkkzutp",
|
||||
"x1+l+3C5HdyXdiFWHLeGFbz3BPVzW6LUZZAHJ9YhhKwPgYDq1OdIe4H+9X//X5WmahMb9J8c1W4VYluL",
|
||||
"P6zGbnaI1Gjp4Uy+nfBxCLtYALFrG/c75Dpo7nZG7WVPaCJhaFtCPkjW95mZ+vFReRa6Xh7A1W7mQhj5",
|
||||
"w3UYenGiet2FHfOL9dks2hOMz83jS4Bsb2POkuhgCitxGyvXhN7fTt+9RbXWW6s1WpnilE93edXekVu/",
|
||||
"uCRvhAUktX2EybvIIRqiaFzqC/4gh6tPCEBST6x3I21Nq9S1EHj5R9/p/+UHNESuJJCPxGukii2kgrwT",
|
||||
"8Rh7/rpT1TTx3KSsXSosgie2X/fDHr1APCfKGNPuZlpgsB6Evm1h1BZ9JzjfSahf4yB6tsFBlJgapdQU",
|
||||
"WrQSa2d7fffapFItTG2nCRd5bzUsb6lT6C+cMO8HGbm/afGt4EVJjYQXWqweu36PSZdNuM/E9xBqMi53",
|
||||
"Tui6q4eMX69aykY40jxEc/N0b4a8LMeWUjXlzoksMSV/d7XcTA9U9DtkeqDuYN7XjFf1OG3jvB8pgHrt",
|
||||
"0fpgIG22a42A1Q44YGyx2ZhrleuntWZ909ENEZbprXCxjxkvFNoeSsAibYf0pXkcenN2M1j82ltWAvaz",
|
||||
"AnwFun2jO+nh/G+viTqEov5jSenA5I9YdNoirwHJlZe670UAmSDX8bPBouGVrWjoU/B+dIjJqtPSbw2d",
|
||||
"b0372Qrwh6jYQWmQ2+TQ4wzZRrdI8WiMalc0xrnZNJqNera6M7XRjGxR2g3euXduUKeTpbObrcM9H0rm",
|
||||
"HkRUictpvmVmJP4Al4rX4g+aDVRti4/dEq53ceI95tHa6FF7OF600yIJB6qZaIjVZOQN7JwoD5S76zVc",
|
||||
"V/TaTs33lS52QBQ5RtJQ3DbvwzRIirBECJbrPmGjHXSsI3ctjmTLta60dNpEWQ2IRD/d2GIXEgyoO8wN",
|
||||
"jzy5GLNyLXdLX+a15fq/1VZbI1G/pu7UaSNmFoOAgLhJvx5Si7B0KcyjvFROMQiGd805eBDMoXczYKiK",
|
||||
"sF2xhtcTaa6sdvkVJ9PoFT5mQo0l9rUFep49edaBDq2RvF7pc2+jrdIKjJpBRclGsbE5+zWC7k6vTXtN",
|
||||
"lGKHn/R1HCv1E5F2XIrfFoJOa3j7aywylAEFZQrAM66QLIuCC1PFfWbqwrtuuhLBPZG2XkHoBxJS+G1M",
|
||||
"wcvnEdaohZ7vxhlfJPxcL+0RQ9DbOKJWeOiROKJWsChgvQqV3IcTXFXi9YEIF37QNrL3HsU1d4wo6Bb/",
|
||||
"8FuKJPBt9B8vjiCQxoGiCIqK1Dw9U3Cd5DZLIv7tg9Zsi3c6QRJPQC3QHNM5uKP38tUfjo7RaSjwrY/z",
|
||||
"oi7trIg6l9+1HdYXoRf9lz+pmyTZWmr51xILzJRpVBXtyLPa86pq+F9rdlXrbGX8SGFMTK193DLLgeG+",
|
||||
"xlviqkpIwp6RUN/Vmwc0RBVs0bC6SY6689rS3eHibELyW0mhW3H/DyUF2fsK6tLrhRwyTcLs68B15pEo",
|
||||
"m6pZ5V7dIaDqLChXjQYF4WsvjNytP4lcc0hbxAlPFIjVvG5/8j1v1ccaoP5KNbL6GrfRyR6FzS9s/IPT",
|
||||
"hhpUgvpZiekg4sxeSzMd2Pqhq6DuRyUPrJ38m5KHoQjH3ockDBdXuc4YeerGXIJShE0f96xvruWAx33Y",
|
||||
"3SEKrttFIunmRP1bQulA3hGVzhLEYA5i4Guumoo2RztcCXGZ9gMm0sQ5+kUQier0QiFD/WdPnqHfVaGQ",
|
||||
"x+gtvwNT/Igom/7glo5uppSPMT3W041wqk7QdY9PJte9G63B4szGVNotjfwgdAsui8JfOyTPISNYAV3o",
|
||||
"rz85OjFXUw0sthSnmQfdYRcfg9n6oiPmtImR527nht6OfoTpRYNG2zxDDye3fp08cmqwaRN2lCA2VPsR",
|
||||
"heRwPHpa99UpGyfkiwaZvf/xR80SgSD3Oz8FkbcDE/C7QVr+QOTtmRv3mCnFfhmHFJSJvEUeBgeSl0V9",
|
||||
"zi2PRo2eRjKyPSQpWNV3uWBv1kiz7pacY1pKbhvwsjZZZ++ZamF4Wxmyl0jxu1gF25qbCdhheg6ds0xL",
|
||||
"NfWp+xKUtGVgRopb3cVEshCJbqGwN8LM5DUujnYpy9GmRp27lpXWhxYpRLPqF0T9GQGBRTpbDPAdFnD0",
|
||||
"AqVYZIRhatv2TbhIIWtTpNbT3NehSNXX+DjOrWYBhC/Sf6JBkS5iaaf6L77nwLpL4dKN6ZwRCAfLNA89",
|
||||
"tdmE95LenTMVJb1UEEXSaK/xB8mD79II6Ldk5rc4f0Qrvye6Q1UuDjS8XSueGo/YFBqc3j5I/sxpeutg",
|
||||
"Hsf6+p3bVw/XruQ0rSI0sQPejp1KGtDLSyvdHBx870oFNfgdwgmh1zoqmSK0awX41h6Ryx3xq5n3aOL4",
|
||||
"ZSlCAziQgiuxcnX19hBEIUByOn8Yuvhg5z4wabSjeQWZXwXyHBQq/OWYlZjSxa7o06rqBqHBDunWGtOa",
|
||||
"X0Ym6vQ/3vuHvNY1Vh7zVrdUcahL3cyG+ialSoXEugKEfXS0m0vfTvvQ7geLiq/Y114QxiAbOajGayKu",
|
||||
"uts1Jlp97V+fd90xxFfvWzc0aRo8Ef2rR8qObvQahQ/dVB1O87/6kV/fAbbzgRT2tD++3FTe9mNqC1q8",
|
||||
"bX8M7Zn3b9LxZNfkpyszeuuj6N8tmcNs84Ck48B2CEYHliHMMF1I4soXUupzOExl8kgi1TYJHQ+ZTKW3",
|
||||
"AmlpTDd66jFgAeK0VLPeyc8fNcZtN3b74VLQ3klviAsynD819OD2s9q2ySX3u7zzkFdgSs6aUjh123lz",
|
||||
"GzatZiUOxXZeg9D6Lal6WxFp6ywTzhLf5qhWOMr1Mlqd83y7VAc3H6+yLz7F7R5mi664UN+i2viAGt05",
|
||||
"owsKNSiq3gquU2MSvJQS9TNISQZDnKratFAv0fSpJe7SLC2IXvo8q80QzrfV9+sOmGQp1CgJzrFqKudF",
|
||||
"WZ0oZEg60nB5wpWtrpbj+CmaeiUTm7FsvpsRlbjqgwkK2fgeUw0ui4G74EKtvucqOXz++Pn/BwAA///k",
|
||||
"T3QHrO8AAA==",
|
||||
}
|
||||
|
||||
// GetSwagger returns the content of the embedded swagger specification file
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
83
internal/httpapi/ontology.go
Normal file
83
internal/httpapi/ontology.go
Normal file
@@ -0,0 +1,83 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
)
|
||||
|
||||
func (s *Server) GetOntology(ctx context.Context, req gen.GetOntologyRequestObject) (gen.GetOntologyResponseObject, error) {
|
||||
resp := gen.GetOntology200JSONResponse{
|
||||
EntityTypes: []gen.EntityType{},
|
||||
RelationshipTypes: []gen.RelationshipType{},
|
||||
Lifecycles: []gen.LifecycleDef{},
|
||||
}
|
||||
|
||||
q := sqlcgen.New(s.pool)
|
||||
|
||||
etRows, err := q.ListEntityTypes(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, et := range etRows {
|
||||
schemaVersion := int(et.SchemaVersion)
|
||||
var schema *map[string]any
|
||||
if len(et.AttributeSchema) > 0 {
|
||||
var s map[string]any
|
||||
if json.Unmarshal(et.AttributeSchema, &s) == nil && s != nil {
|
||||
schema = &s
|
||||
}
|
||||
}
|
||||
resp.EntityTypes = append(resp.EntityTypes, gen.EntityType{
|
||||
Name: et.Name,
|
||||
ParentType: et.ParentType,
|
||||
IsAbstract: et.IsAbstract,
|
||||
Domain: et.Domain,
|
||||
Layer: gen.EntityTypeLayer(et.Layer),
|
||||
Description: et.Description,
|
||||
LifecycleId: et.LifecycleID,
|
||||
SchemaVersion: &schemaVersion,
|
||||
AttributeSchema: schema,
|
||||
Status: gen.EntityTypeStatus(et.Status),
|
||||
})
|
||||
}
|
||||
|
||||
rtRows, err := q.ListRelationshipTypes(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, rt := range rtRows {
|
||||
resp.RelationshipTypes = append(resp.RelationshipTypes, gen.RelationshipType{
|
||||
Name: rt.Name,
|
||||
Inverse: rt.Inverse,
|
||||
SourceType: rt.SourceType,
|
||||
TargetType: rt.TargetType,
|
||||
Cardinality: gen.RelationshipTypeCardinality(rt.Cardinality),
|
||||
Description: rt.Description,
|
||||
})
|
||||
}
|
||||
|
||||
lcRows, err := q.ListLifecycleDefs(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, lc := range lcRows {
|
||||
terminal := lc.TerminalStates
|
||||
var transitions map[string]any
|
||||
if err := json.Unmarshal(lc.Transitions, &transitions); err != nil {
|
||||
return nil, fmt.Errorf("lifecycle %s transitions: %w", lc.ID, err)
|
||||
}
|
||||
resp.Lifecycles = append(resp.Lifecycles, gen.LifecycleDef{
|
||||
Id: lc.ID,
|
||||
States: lc.States,
|
||||
DefaultState: lc.DefaultState,
|
||||
TerminalStates: &terminal,
|
||||
Transitions: transitions,
|
||||
})
|
||||
}
|
||||
|
||||
return resp, nil
|
||||
}
|
||||
@@ -111,6 +111,7 @@ func (s *Server) PatchPattern(ctx context.Context, req gen.PatchPatternRequestOb
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "patch",
|
||||
&id, "PATCH", "/api/v1/patterns/"+req.Id, "",
|
||||
nil,
|
||||
map[string]any{"status": req.Body.Status, "quarantined": req.Body.Quarantined}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
72
internal/httpapi/query_audit.go
Normal file
72
internal/httpapi/query_audit.go
Normal file
@@ -0,0 +1,72 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
)
|
||||
|
||||
func (s *Server) QueryAudit(ctx context.Context, req gen.QueryAuditRequestObject) (gen.QueryAuditResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
var actorType, actorID, action, entityID, correlationID *string
|
||||
if req.Params.ActorType != nil {
|
||||
actorType = req.Params.ActorType
|
||||
}
|
||||
if req.Params.ActorId != nil {
|
||||
actorID = req.Params.ActorId
|
||||
}
|
||||
if req.Params.Action != nil {
|
||||
action = req.Params.Action
|
||||
}
|
||||
if req.Params.EntityId != nil {
|
||||
entityID = req.Params.EntityId
|
||||
}
|
||||
if req.Params.CorrelationId != nil {
|
||||
correlationID = req.Params.CorrelationId
|
||||
}
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT id, ts, actor_type, actor_id::text, action, entity_id::text,
|
||||
method, path, status_code, detail, source_ip, correlation_id, session_id::text
|
||||
FROM audit_log
|
||||
WHERE ($1::text IS NULL OR actor_type = $1)
|
||||
AND ($2::text IS NULL OR actor_id::text = $2)
|
||||
AND ($3::text IS NULL OR action = $3)
|
||||
AND ($4::text IS NULL OR entity_id::text = $4)
|
||||
AND ($5::text IS NULL OR correlation_id = $5)
|
||||
AND ($6::timestamptz IS NULL OR ts >= $6)
|
||||
AND ($7::timestamptz IS NULL OR ts <= $7)
|
||||
ORDER BY ts DESC
|
||||
LIMIT $8`,
|
||||
actorType, actorID, action, entityID, correlationID, req.Params.From, req.Params.To, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.AuditEntry{}
|
||||
for rows.Next() {
|
||||
var a gen.AuditEntry
|
||||
var detailBytes []byte
|
||||
var actID, entID, method, path, sourceIP, corrID, sessionID *string
|
||||
var statusCode *int
|
||||
if err := rows.Scan(&a.Id, &a.Ts, &a.ActorType, &actID, &a.Action, &entID,
|
||||
&method, &path, &statusCode, &detailBytes, &sourceIP, &corrID, &sessionID); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a.ActorId = actID
|
||||
a.EntityId = entID
|
||||
a.Method = method
|
||||
a.Path = path
|
||||
a.StatusCode = statusCode
|
||||
a.SourceIp = sourceIP
|
||||
a.CorrelationId = corrID
|
||||
var detail map[string]any
|
||||
if json.Unmarshal(detailBytes, &detail) == nil {
|
||||
a.Detail = &detail
|
||||
}
|
||||
items = append(items, a)
|
||||
}
|
||||
return gen.QueryAudit200JSONResponse{Items: items}, rows.Err()
|
||||
}
|
||||
148
internal/httpapi/ratelimit.go
Normal file
148
internal/httpapi/ratelimit.go
Normal file
@@ -0,0 +1,148 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"golang.org/x/time/rate"
|
||||
)
|
||||
|
||||
// rateLimiter is a per-client (IP) token-bucket limiter registry. Each unique
|
||||
// client gets its own *rate.Limiter; idle entries are swept periodically so a
|
||||
// flood of distinct IPs can't grow the map unbounded. A rate of zero (rps==0)
|
||||
// disables limiting entirely — the returned middleware is a no-op.
|
||||
//
|
||||
// Client identity is the source IP. Behind Caddy the real client is in
|
||||
// X-Forwarded-For: Caddy appends the immediate client as the LAST hop, while
|
||||
// earlier hops are client-supplied and spoofable. clientIP therefore takes the
|
||||
// rightmost XFF entry (the proxy's contribution) rather than the first.
|
||||
type rateLimiter struct {
|
||||
mu sync.Mutex
|
||||
limiters map[string]*entry
|
||||
rps rate.Limit
|
||||
burst int
|
||||
}
|
||||
|
||||
type entry struct {
|
||||
limiter *rate.Limiter
|
||||
lastSeen time.Time
|
||||
}
|
||||
|
||||
// newRateLimiter builds the registry and starts the idle-entry sweeper tied to
|
||||
// ctx, so the ticker is stopped when the server shuts down.
|
||||
func newRateLimiter(ctx context.Context, rps, burst int) *rateLimiter {
|
||||
rl := &rateLimiter{
|
||||
limiters: make(map[string]*entry),
|
||||
rps: rate.Limit(rps),
|
||||
burst: burst,
|
||||
}
|
||||
if rps > 0 {
|
||||
go rl.sweep(ctx)
|
||||
}
|
||||
return rl
|
||||
}
|
||||
|
||||
// sweep drops entries untouched since the last sweep so the registry doesn't
|
||||
// grow without bound under a rotating-IP attack or long-lived process. Exits
|
||||
// (and stops its ticker) when ctx is cancelled.
|
||||
func (rl *rateLimiter) sweep(ctx context.Context) {
|
||||
ticker := time.NewTicker(5 * time.Minute)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
rl.mu.Lock()
|
||||
for ip, e := range rl.limiters {
|
||||
if time.Since(e.lastSeen) > 10*time.Minute {
|
||||
delete(rl.limiters, ip)
|
||||
}
|
||||
}
|
||||
rl.mu.Unlock()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (rl *rateLimiter) get(ip string) *rate.Limiter {
|
||||
rl.mu.Lock()
|
||||
defer rl.mu.Unlock()
|
||||
if e, ok := rl.limiters[ip]; ok {
|
||||
e.lastSeen = time.Now()
|
||||
return e.limiter
|
||||
}
|
||||
l := rate.NewLimiter(rl.rps, rl.burst)
|
||||
rl.limiters[ip] = &entry{limiter: l, lastSeen: time.Now()}
|
||||
return l
|
||||
}
|
||||
|
||||
// middleware returns a chi-style middleware that enforces the per-IP limit.
|
||||
// Call with rps==0 to get a pass-through no-op.
|
||||
func (rl *rateLimiter) middleware(next http.Handler) http.Handler {
|
||||
if rl.rps <= 0 {
|
||||
return next
|
||||
}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
// Exempt infra liveness probes so Caddy/compose healthchecks can't be
|
||||
// throttled into marking the service unhealthy.
|
||||
if r.URL.Path == "/healthz" {
|
||||
next.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
if !rl.get(clientIP(r)).Allow() {
|
||||
w.Header().Set("Retry-After", "1")
|
||||
writeProblem(w, r, http.StatusTooManyRequests, "rate limit exceeded", "")
|
||||
return
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// clientIP extracts the originating client address. It takes the rightmost
|
||||
// X-Forwarded-For hop — the one the reverse proxy (Caddy) appends for the
|
||||
// immediate client — because earlier hops are attacker-controlled and could
|
||||
// be spoofed to dodge the limit or exhaust another client's bucket. Falls
|
||||
// back to r.RemoteAddr when no XFF header is present.
|
||||
//
|
||||
// Known limitation: this is only trustworthy when the request actually
|
||||
// traverses Caddy. A client connecting directly to the published :8090 (not
|
||||
// behind the proxy) can set a single-hop XFF and have it trusted. That only
|
||||
// evades rate limiting (auth is still required), and rate limiting is off by
|
||||
// default, so the blast radius is narrow. Fully closing it requires either
|
||||
// Caddy trusted_proxies (so it overwrites XFF / sets a non-spoofable
|
||||
// X-Real-Ip) or keying the limiter on the auth token instead of IP.
|
||||
func clientIP(r *http.Request) string {
|
||||
if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
|
||||
if idx := strings.LastIndex(xff, ","); idx >= 0 {
|
||||
xff = xff[idx+1:]
|
||||
}
|
||||
if ip := strings.TrimSpace(xff); ip != "" {
|
||||
return ip
|
||||
}
|
||||
}
|
||||
host, _, err := net.SplitHostPort(r.RemoteAddr)
|
||||
if err != nil {
|
||||
return r.RemoteAddr
|
||||
}
|
||||
return host
|
||||
}
|
||||
|
||||
// newRateLimiterFromConfig builds the limiter from API config, logging the
|
||||
// chosen policy once at startup. rps<=0 means "disabled" (returns a no-op
|
||||
// middleware) so dev/single-user setups aren't throttled by default.
|
||||
func newRateLimiterFromConfig(ctx context.Context, rps, burst int) *rateLimiter {
|
||||
if rps <= 0 {
|
||||
slog.Info("api rate limiting disabled (OIKOS_API_RATE_LIMIT unset)")
|
||||
return &rateLimiter{rps: 0}
|
||||
}
|
||||
if burst <= 0 {
|
||||
burst = rps * 2
|
||||
}
|
||||
slog.Info("api rate limiting enabled", "rps", rps, "burst", burst)
|
||||
return newRateLimiter(ctx, rps, burst)
|
||||
}
|
||||
161
internal/httpapi/ratelimit_test.go
Normal file
161
internal/httpapi/ratelimit_test.go
Normal file
@@ -0,0 +1,161 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// okHandler is a sentinel upstream that records it was reached.
|
||||
func okHandler(t *testing.T, reached *bool) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
*reached = true
|
||||
w.WriteHeader(http.StatusOK)
|
||||
})
|
||||
}
|
||||
|
||||
func TestRateLimiterDisabledWhenRPSZero(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
rl := newRateLimiterFromConfig(ctx, 0, 0)
|
||||
if rl.rps != 0 {
|
||||
t.Fatalf("rps should be 0 when disabled, got %v", rl.rps)
|
||||
}
|
||||
// Disabled limiter is a pass-through: requests always reach upstream.
|
||||
reached := false
|
||||
h := rl.middleware(okHandler(t, &reached))
|
||||
for i := 0; i < 50; i++ {
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/v1/entities", nil))
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("disabled limiter request %d: want 200, got %d", i, rec.Code)
|
||||
}
|
||||
}
|
||||
if !reached {
|
||||
t.Fatal("disabled limiter never reached upstream")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRateLimiterThrottlesAfterBurst(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
rl := newRateLimiter(ctx, 1, 3) // 1 rps, burst 3
|
||||
reached := false
|
||||
h := rl.middleware(okHandler(t, &reached))
|
||||
|
||||
var last429, okCount int
|
||||
for i := 0; i < 6; i++ {
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/v1/entities", nil))
|
||||
switch rec.Code {
|
||||
case http.StatusOK:
|
||||
okCount++
|
||||
case http.StatusTooManyRequests:
|
||||
last429 = i
|
||||
}
|
||||
}
|
||||
if okCount < 1 {
|
||||
t.Fatal("expected at least one request through within burst")
|
||||
}
|
||||
if last429 == 0 {
|
||||
t.Fatal("expected at least one 429 once burst exhausted")
|
||||
}
|
||||
if !reached {
|
||||
t.Fatal("upstream never reached")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRateLimiterExemptsHealthz(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
rl := newRateLimiter(ctx, 1, 1) // tiny burst
|
||||
reached := false
|
||||
h := rl.middleware(okHandler(t, &reached))
|
||||
// /healthz must never be throttled, even under a flood.
|
||||
for i := 0; i < 20; i++ {
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/healthz", nil))
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("healthz request %d throttled: want 200, got %d", i, rec.Code)
|
||||
}
|
||||
}
|
||||
if !reached {
|
||||
t.Fatal("healthz never reached upstream")
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientIPTakesRightmostXFF(t *testing.T) {
|
||||
// The reverse proxy appends the real client as the LAST hop; earlier hops
|
||||
// are spoofable and must be ignored.
|
||||
cases := []struct {
|
||||
name string
|
||||
xff string
|
||||
remote string
|
||||
wantIP string
|
||||
}{
|
||||
{"single xff", "203.0.113.7", "10.0.0.1:4000", "203.0.113.7"},
|
||||
{"multi hop takes rightmost", "spoofed-attacker, 203.0.113.7", "10.0.0.1:4000", "203.0.113.7"},
|
||||
{"no xff falls back to remote", "", "198.51.100.2:4000", "198.51.100.2"},
|
||||
{"blank xff falls back to remote", " ", "198.51.100.2:4000", "198.51.100.2"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
req.RemoteAddr = tc.remote
|
||||
if strings.TrimSpace(tc.xff) != "" {
|
||||
req.Header.Set("X-Forwarded-For", tc.xff)
|
||||
}
|
||||
if got := clientIP(req); got != tc.wantIP {
|
||||
t.Fatalf("clientIP: want %q, got %q", tc.wantIP, got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRateLimiterConcurrency(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
rl := newRateLimiter(ctx, 1000, 100) // generous; ensures no deadlock/panic under contention
|
||||
h := rl.middleware(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 20; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/v1/entities", nil))
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
}
|
||||
|
||||
func TestNewRateLimiterFromConfigBurstDefault(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel() // release the sweep goroutine + ticker
|
||||
rl := newRateLimiterFromConfig(ctx, 10, 0) // burst unset → defaults to 2x
|
||||
if rl.burst != 20 {
|
||||
t.Fatalf("default burst should be 2x rps (20), got %d", rl.burst)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSweepStopsOnContextCancel verifies the ticker is released when the
|
||||
// server context is cancelled (no process-lifetime goroutine/ticker leak).
|
||||
func TestSweepStopsOnContextCancel(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
rl := newRateLimiter(ctx, 1, 1)
|
||||
cancel()
|
||||
// Give the sweeper a moment to observe cancellation. It must return
|
||||
// without blocking; the deferred ticker.Stop() fires on return.
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
// Limiter remains usable for the brief test lifetime.
|
||||
if !rl.get("10.0.0.1").Allow() {
|
||||
t.Fatal("limiter should still allow within burst after sweep stops")
|
||||
}
|
||||
}
|
||||
@@ -65,6 +65,7 @@ func (s *Server) CreateRelationship(ctx context.Context, req gen.CreateRelations
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "create",
|
||||
nil, "POST", "/api/v1/relationships", "",
|
||||
nil,
|
||||
map[string]any{"source": req.Body.Source, "target": req.Body.Target, "type": req.Body.Type}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
@@ -108,6 +109,7 @@ func (s *Server) EndRelationship(ctx context.Context, req gen.EndRelationshipReq
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, sqlcgen.New(tx), actorType, actor, "delete",
|
||||
nil, "DELETE", "/api/v1/relationships", "",
|
||||
nil,
|
||||
map[string]any{"source": req.Params.Source, "target": req.Params.Target, "type": req.Params.RelType}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
@@ -23,11 +23,13 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/actuator"
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
mcphandler "github.com/dtoro/oikos/internal/mcp"
|
||||
"github.com/dtoro/oikos/internal/safego"
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
"github.com/go-chi/cors"
|
||||
@@ -52,19 +54,13 @@ type actor struct {
|
||||
type Server struct {
|
||||
pool *db.Pool
|
||||
cfg config.Config
|
||||
secretsManager secretsBackend
|
||||
secretsManager secrets.Backend
|
||||
entityCache *db.EntityCache
|
||||
sseBroker *sseBroker
|
||||
sseSubs map[*sseSubscriber]struct{}
|
||||
sseMu sync.Mutex
|
||||
}
|
||||
|
||||
// secretsBackend is a minimal interface for secrets operations used by the
|
||||
// HTTP API (enrollment key storage, listing). Compatible with internal/secrets.
|
||||
type secretsBackend interface {
|
||||
Set(ctx context.Context, key string, value string) error
|
||||
List(ctx context.Context) ([]string, error)
|
||||
}
|
||||
|
||||
// NewHandler builds the full HTTP handler: /healthz (unauthenticated,
|
||||
// SG18) + the OpenAPI surface under /api/v1 behind bearer auth.
|
||||
//
|
||||
@@ -74,10 +70,44 @@ type secretsBackend interface {
|
||||
// and pool.Close() deadlocks.
|
||||
func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Handler {
|
||||
s := &Server{
|
||||
pool: pool,
|
||||
cfg: cfg,
|
||||
sseBroker: newSSEBroker(10000),
|
||||
sseSubs: make(map[*sseSubscriber]struct{}),
|
||||
pool: pool,
|
||||
cfg: cfg,
|
||||
entityCache: db.NewEntityCache(60 * time.Second),
|
||||
sseBroker: newSSEBroker(10000),
|
||||
sseSubs: make(map[*sseSubscriber]struct{}),
|
||||
}
|
||||
|
||||
// Wire secrets backend: Infisical primary with SOPS DR fallback.
|
||||
if cfg.InfisicalSiteURL != "" {
|
||||
infCfg := secrets.InfisicalConfig{
|
||||
SiteURL: cfg.InfisicalSiteURL,
|
||||
ClientID: cfg.InfisicalClientID,
|
||||
ClientSecret: cfg.InfisicalClientSecret,
|
||||
ProjectID: cfg.InfisicalProjectID,
|
||||
SecretPath: "/",
|
||||
Env: cfg.InfisicalEnv,
|
||||
}
|
||||
if infCfg.Env == "" {
|
||||
infCfg.Env = "dev"
|
||||
}
|
||||
primary := secrets.NewInfisicalBackend(infCfg)
|
||||
var fallback secrets.Backend
|
||||
if cfg.SecretsDir != "" {
|
||||
fallback = secrets.NewSOPSBackend(cfg.SecretsDir)
|
||||
}
|
||||
s.secretsManager = secrets.NewManager(primary, fallback)
|
||||
slog.Info("secrets backend wired", "backend", "infisical+sops", "site", cfg.InfisicalSiteURL)
|
||||
|
||||
// Start background secret refresh loop (non-blocking).
|
||||
if mgr, ok := s.secretsManager.(*secrets.Manager); ok {
|
||||
safego.Go("secrets:refresh", func() { mgr.StartRefreshLoop(ctx) })
|
||||
}
|
||||
|
||||
// Pre-load SSH host keys from Infisical for host verification.
|
||||
if hosts := actuator.ResolveSSHHosts(ctx, pool); len(hosts) > 0 {
|
||||
hkSrc := actuator.NewInfisicalHostKeySource(s.secretsManager)
|
||||
actuator.LoadHostKeys(ctx, hosts, hkSrc)
|
||||
}
|
||||
}
|
||||
|
||||
// Start background SSE listener, tied to ctx for clean shutdown.
|
||||
@@ -90,6 +120,11 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
|
||||
r.Use(middleware.Recoverer)
|
||||
r.Use(middleware.RequestID)
|
||||
r.Use(requestLogger)
|
||||
// Per-IP rate limiting (plan D3). Applied before CORS/auth so a runaway
|
||||
// agent loop is throttled regardless of credentials. The middleware
|
||||
// exempts /healthz so liveness probes can't be throttled.
|
||||
limiter := newRateLimiterFromConfig(ctx, cfg.APIRateLimit, cfg.APIRateBurst)
|
||||
r.Use(limiter.middleware)
|
||||
r.Use(cors.Handler(cors.Options{
|
||||
AllowedOrigins: []string{cfg.CORSAllowedOrigin},
|
||||
AllowedMethods: []string{"GET", "POST", "PATCH", "DELETE", "OPTIONS"},
|
||||
@@ -124,6 +159,7 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
|
||||
// /api/v1/executions/{id}/logs — streamed command output, no schema type
|
||||
// /api/v1/learning/timeline — derived view, no backing schema type
|
||||
// /api/v1/learning/trend — derived view, no backing schema type
|
||||
// /api/v1/openapi.json — API spec (embedded in binary), self-service
|
||||
//
|
||||
// See .agents/dev/CONTRIBUTING.md §OpenAPI codegen for the policy.
|
||||
|
||||
@@ -261,6 +297,17 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/learning/timeline", s.serveLearningTimeline)
|
||||
r.With(combinedAuth(cfg, false)).Get("/api/v1/learning/trend", s.serveLearningTrend)
|
||||
|
||||
// Serve the OpenAPI spec at a browseable endpoint (agents + humans)
|
||||
r.Get("/api/v1/openapi.json", func(w http.ResponseWriter, req *http.Request) {
|
||||
swagger, err := gen.GetSwagger()
|
||||
if err != nil {
|
||||
writeProblem(w, req, http.StatusInternalServerError, "failed to load spec", "")
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(swagger)
|
||||
})
|
||||
|
||||
// Mount MCP at /mcp (plan R3-10)
|
||||
nomosAgentID := uuid.Nil
|
||||
if cfg.NomosAgentID != "" {
|
||||
@@ -271,7 +318,7 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
|
||||
if nomosAgentID == uuid.Nil && cfg.NomosAgentSlug != "" {
|
||||
_ = pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", cfg.NomosAgentSlug).Scan(&nomosAgentID)
|
||||
}
|
||||
r.With(combinedAuth(cfg, false)).Handle("/mcp", mcphandler.NewHandler(pool, cfg.MCPBearerToken, nomosAgentID))
|
||||
r.With(combinedAuth(cfg, false)).Handle("/mcp", mcphandler.NewHandler(pool, cfg.MCPBearerToken, nomosAgentID, s.secretsManager))
|
||||
|
||||
if nomosURL := os.Getenv("NOMOS_PROXY_URL"); nomosURL != "" {
|
||||
target, _ := url.Parse(nomosURL)
|
||||
|
||||
170
internal/httpapi/signals.go
Normal file
170
internal/httpapi/signals.go
Normal file
@@ -0,0 +1,170 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/dtoro/oikos/internal/httpapi/gen"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
func (s *Server) ListSignals(ctx context.Context, req gen.ListSignalsRequestObject) (gen.ListSignalsResponseObject, error) {
|
||||
limit := clampLimit(req.Params.Limit)
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT sig.entity_id, se.slug, sig.kind, sig.severity, sig.state,
|
||||
te.slug, sig.check_id::text, sig.evidence, sig.likely_cause,
|
||||
sig.occurrence_count, sig.flap_count, sig.hold_down_until,
|
||||
sig.mute_until, sig.first_seen_at, sig.last_seen_at
|
||||
FROM signals sig
|
||||
JOIN entities se ON se.id = sig.entity_id
|
||||
LEFT JOIN entities te ON te.id = sig.target_entity_id
|
||||
WHERE ($1::text IS NULL OR sig.state = $1)
|
||||
AND ($2::text IS NULL OR sig.severity = $2)
|
||||
AND ($3::text IS NULL OR te.slug = $3)
|
||||
AND ($4::text IS NULL OR sig.kind = $4)
|
||||
AND ($5::text IS NULL OR se.slug > $5)
|
||||
ORDER BY se.slug
|
||||
LIMIT $6`,
|
||||
req.Params.State, (*string)(req.Params.Severity), req.Params.EntityId,
|
||||
req.Params.Kind, req.Params.Cursor, limit+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := []gen.Signal{}
|
||||
for rows.Next() {
|
||||
var sig gen.Signal
|
||||
var flap int
|
||||
if err := rows.Scan(&sig.Id, &sig.Slug, &sig.Kind, &sig.Severity, &sig.State,
|
||||
&sig.Target, &sig.CheckId, &sig.Evidence, &sig.LikelyCause,
|
||||
&sig.OccurrenceCount, &flap, &sig.HoldDownUntil,
|
||||
&sig.MuteUntil, &sig.FirstSeenAt, &sig.LastSeenAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sig.FlapCount = &flap
|
||||
items = append(items, sig)
|
||||
}
|
||||
if rows.Err() != nil {
|
||||
return nil, rows.Err()
|
||||
}
|
||||
|
||||
var next *string
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
next = &items[len(items)-1].Slug
|
||||
}
|
||||
return gen.ListSignals200JSONResponse{Items: items, NextCursor: next}, nil
|
||||
}
|
||||
|
||||
func (s *Server) AckSignal(ctx context.Context, req gen.AckSignalRequestObject) (gen.AckSignalResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
var sig gen.Signal
|
||||
err = tx.QueryRow(ctx, `
|
||||
UPDATE signals SET state = 'acknowledged', updated_at = now()
|
||||
WHERE entity_id = $1 AND state IN ('raised','failed')
|
||||
RETURNING entity_id, (SELECT slug FROM entities WHERE id = $1),
|
||||
kind, severity, 'acknowledged',
|
||||
(SELECT slug FROM entities WHERE id = target_entity_id),
|
||||
check_id::text, evidence, likely_cause,
|
||||
occurrence_count, flap_count, hold_down_until,
|
||||
mute_until, first_seen_at, last_seen_at`,
|
||||
id).Scan(&sig.Id, &sig.Slug, &sig.Kind, &sig.Severity, &sig.State,
|
||||
&sig.Target, &sig.CheckId, &sig.Evidence, &sig.LikelyCause,
|
||||
&sig.OccurrenceCount, &sig.FlapCount, &sig.HoldDownUntil,
|
||||
&sig.MuteUntil, &sig.FirstSeenAt, &sig.LastSeenAt)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: signal %s not in a state that can be acknowledged", domain.ErrInvalidTransition, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gen.AckSignal200JSONResponse{SignalUpdatedJSONResponse: gen.SignalUpdatedJSONResponse(sig)}, nil
|
||||
}
|
||||
|
||||
func (s *Server) ResolveSignal(ctx context.Context, req gen.ResolveSignalRequestObject) (gen.ResolveSignalResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
var sig gen.Signal
|
||||
err = tx.QueryRow(ctx, `
|
||||
UPDATE signals SET state = 'resolved', updated_at = now()
|
||||
WHERE entity_id = $1 AND state IN ('raised','acknowledged','acting','failed')
|
||||
RETURNING entity_id, (SELECT slug FROM entities WHERE id = $1),
|
||||
kind, severity, 'resolved',
|
||||
(SELECT slug FROM entities WHERE id = target_entity_id),
|
||||
check_id::text, evidence, likely_cause,
|
||||
occurrence_count, flap_count, hold_down_until,
|
||||
mute_until, first_seen_at, last_seen_at`,
|
||||
id).Scan(&sig.Id, &sig.Slug, &sig.Kind, &sig.Severity, &sig.State,
|
||||
&sig.Target, &sig.CheckId, &sig.Evidence, &sig.LikelyCause,
|
||||
&sig.OccurrenceCount, &sig.FlapCount, &sig.HoldDownUntil,
|
||||
&sig.MuteUntil, &sig.FirstSeenAt, &sig.LastSeenAt)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: signal %s not in a state that can be resolved", domain.ErrInvalidTransition, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gen.ResolveSignal200JSONResponse{SignalUpdatedJSONResponse: gen.SignalUpdatedJSONResponse(sig)}, nil
|
||||
}
|
||||
|
||||
func (s *Server) MuteSignal(ctx context.Context, req gen.MuteSignalRequestObject) (gen.MuteSignalResponseObject, error) {
|
||||
id, err := s.resolveEntityID(ctx, req.Id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
var sig gen.Signal
|
||||
err = tx.QueryRow(ctx, `
|
||||
UPDATE signals SET state = 'muted', mute_until = $2, updated_at = now()
|
||||
WHERE entity_id = $1 AND state IN ('raised','acknowledged')
|
||||
RETURNING entity_id, (SELECT slug FROM entities WHERE id = $1),
|
||||
kind, severity, 'muted',
|
||||
(SELECT slug FROM entities WHERE id = target_entity_id),
|
||||
check_id::text, evidence, likely_cause,
|
||||
occurrence_count, flap_count, hold_down_until,
|
||||
mute_until, first_seen_at, last_seen_at`,
|
||||
id, req.Body.MuteUntil).Scan(&sig.Id, &sig.Slug, &sig.Kind, &sig.Severity, &sig.State,
|
||||
&sig.Target, &sig.CheckId, &sig.Evidence, &sig.LikelyCause,
|
||||
&sig.OccurrenceCount, &sig.FlapCount, &sig.HoldDownUntil,
|
||||
&sig.MuteUntil, &sig.FirstSeenAt, &sig.LastSeenAt)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, fmt.Errorf("%w: signal %s not in a state that can be muted", domain.ErrInvalidTransition, req.Id)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gen.MuteSignal200JSONResponse{SignalUpdatedJSONResponse: gen.SignalUpdatedJSONResponse(sig)}, nil
|
||||
}
|
||||
@@ -136,6 +136,7 @@ func (s *Server) PatchSkill(ctx context.Context, req gen.PatchSkillRequestObject
|
||||
actorType, actor := actorInfo(ctx)
|
||||
if auditErr := observability.Audit(ctx, q, actorType, actor, "patch",
|
||||
&id, "PATCH", "/api/v1/skills/"+req.Id, "",
|
||||
nil,
|
||||
map[string]any{"status": req.Body.Status, "pinned_version": req.Body.PinnedVersion}); auditErr != nil {
|
||||
return nil, auditErr
|
||||
}
|
||||
|
||||
242
internal/mcp/analysis_tools.go
Normal file
242
internal/mcp/analysis_tools.go
Normal file
@@ -0,0 +1,242 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/policy"
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
func AnalysisTools(pool *db.Pool, agentID uuid.UUID, sec secretBackend) []toolReg {
|
||||
return []toolReg{
|
||||
{tool: &mcp.Tool{Name: "get_health_summary", Description: "Fleet health per entity — optionally filter by health state(s)",
|
||||
InputSchema: objSchema(
|
||||
prop{"health", "string", "Comma-separated health states to include (e.g. 'down,stale'). Omit for all."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
healthStr, _ := args["health"].(string)
|
||||
query := `
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check' AND e.state <> 'destroyed'`
|
||||
if healthStr != "" {
|
||||
query += ` AND st.health = ANY(string_to_array($1, ','))`
|
||||
return queryRows(ctx, pool, query, healthStr), nil
|
||||
}
|
||||
query += ` ORDER BY e.slug`
|
||||
return queryRows(ctx, pool, query), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_audit_trail", Description: "Query the audit log",
|
||||
InputSchema: objSchema(prop{"entity_id", "string", "Filter by affected entity UUID"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT id, ts, actor_type, action, entity_id::text, method, path, correlation_id, session_id::text
|
||||
FROM audit_log
|
||||
WHERE ($1::text IS NULL OR entity_id::text = $1)
|
||||
ORDER BY ts DESC LIMIT 50`, nStr(args["entity_id"])), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "query_metrics", Description: "Time-series metrics with bucketed avg/min/max over N hours",
|
||||
InputSchema: objSchema(prop{"hours", "integer", "Look-back window in hours (default 24)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
hours := int(getFloat(args, "hours", 24))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT time_bucket('1 hour', ts) AS bucket,
|
||||
entity_id::text, metric,
|
||||
ROUND(avg(value)::numeric, 2) AS avg,
|
||||
ROUND(min(value)::numeric, 2) AS min,
|
||||
ROUND(max(value)::numeric, 2) AS max
|
||||
FROM metric_samples
|
||||
WHERE ts > now() - make_interval(hours => $1)
|
||||
GROUP BY bucket, entity_id, metric
|
||||
ORDER BY bucket DESC LIMIT 100`, hours), "metric_chart"), nil
|
||||
}},
|
||||
// ─── Phase 4: new tools ──────────────────────────────────────────
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_signal_history", Description: "Query open and recent signals",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_slug", "string", "Filter by target entity slug"},
|
||||
prop{"state", "string", "Filter by signal state (raised, resolved)"},
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT s.entity_id::text, s.kind, s.severity, s.state,
|
||||
s.occurrence_count, e.slug AS target_slug,
|
||||
s.first_seen_at, s.last_seen_at
|
||||
FROM signals s
|
||||
LEFT JOIN entities e ON e.id = s.target_entity_id
|
||||
WHERE ($1::text IS NULL OR e.slug = $1)
|
||||
AND ($2::text IS NULL OR s.state = $2)
|
||||
ORDER BY s.last_seen_at DESC LIMIT $3`,
|
||||
nStr(args["entity_slug"]), nStr(args["state"]), limit), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_patterns", Description: "List learned action patterns",
|
||||
InputSchema: objSchema(
|
||||
prop{"status", "string", "Filter by status (hypothesized, validated, active)"},
|
||||
prop{"entity_type", "string", "Filter by applies_type"},
|
||||
prop{"action", "string", "Filter by action"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT p.entity_id::text, p.applies_type, p.action, p.pattern,
|
||||
p.confidence, p.evidence_count, p.success_count, p.failure_count,
|
||||
p.status, p.quarantined, p.version, p.last_validated_at
|
||||
FROM patterns p
|
||||
WHERE ($1::text IS NULL OR p.status = $1)
|
||||
AND ($2::text IS NULL OR p.applies_type = $2)
|
||||
AND ($3::text IS NULL OR p.action = $3)
|
||||
ORDER BY p.applies_type, p.action`,
|
||||
nStr(args["status"]), nStr(args["entity_type"]), nStr(args["action"])), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_skills", Description: "List available automation skills",
|
||||
InputSchema: objSchema(
|
||||
prop{"status", "string", "Filter by status (active, inactive, deprecated)"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT s.entity_id::text, s.version, s.name, LEFT(s.procedure::text, 300) AS procedure_preview,
|
||||
s.applies_type, s.action, s.status, s.success_rate,
|
||||
s.changed_by::text, s.change_reason, s.last_used_at
|
||||
FROM skills s
|
||||
WHERE ($1::text IS NULL OR s.status = $1)
|
||||
ORDER BY s.name, s.version DESC`,
|
||||
nStr(args["status"])), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_trend", Description: "Metric slope, variance, and averages for an entity over N days",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_id", "string", "Entity slug"},
|
||||
prop{"days", "integer", "Look-back window in days (default 7)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_id"].(string)
|
||||
days := int(getFloat(args, "days", 7))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT metric,
|
||||
ROUND(avg(value)::numeric, 2) AS avg_val,
|
||||
ROUND(stddev(value)::numeric, 2) AS std_val,
|
||||
count(*) AS sample_count,
|
||||
ROUND(regr_slope(value, EXTRACT(EPOCH FROM ts)::numeric)::numeric, 4) AS slope
|
||||
FROM metric_samples ms
|
||||
JOIN entities e ON e.id = ms.entity_id
|
||||
WHERE e.slug = $1 AND ts >= now() - make_interval(days => $2)
|
||||
GROUP BY metric
|
||||
ORDER BY metric`, slug, days), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_event_timeline", Description: "Recent events filtered by severity and entity slug",
|
||||
InputSchema: objSchema(
|
||||
prop{"severity", "string", "Filter by severity (info, warn, error)"},
|
||||
prop{"entity_slug", "string", "Filter by entity slug"},
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT ev.ts, ev.type, ev.severity, ev.source, e.slug AS entity_slug,
|
||||
ev.data::text AS message, ev.correlation_id
|
||||
FROM events ev
|
||||
LEFT JOIN entities e ON e.id = ev.entity_id
|
||||
WHERE ($1::text IS NULL OR ev.severity = $1)
|
||||
AND ($2::text IS NULL OR e.slug = $2)
|
||||
ORDER BY ev.ts DESC LIMIT $3`,
|
||||
nStr(args["severity"]), nStr(args["entity_slug"]), limit), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_agent_activity", Description: "Agent self-inspection: query agent activity log",
|
||||
InputSchema: objSchema(
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT id, ts, agent_id::text, session_id, activity_type, tool_name,
|
||||
entity_id::text, left(input_summary, 200) AS input_summary,
|
||||
left(output_summary, 200) AS output_summary,
|
||||
duration_ms, token_count, success, correlation_id
|
||||
FROM agent_activity
|
||||
WHERE agent_id = $1
|
||||
ORDER BY ts DESC LIMIT $2`, agentID, limit), "change_log"), nil
|
||||
}},
|
||||
// classify_command is the command-scoped preflight from
|
||||
// plans/2026-07-20-session-review-ten-sessions.md P0.2. The
|
||||
// existing `preflight` tool is entity/action-scoped — useless when
|
||||
// the agent is composing a `run` command and needs to know whether
|
||||
// the classifier will accept it before submitting. Without this,
|
||||
// the agent has to retry with cosmetic variations until it finds
|
||||
// one that passes (see sessions a51e2086, 8acea2e3 — three
|
||||
// duplicate rclone sessions, all bouncing off the classifier).
|
||||
// Call this BEFORE `run` whenever the classification is uncertain.
|
||||
{tool: &mcp.Tool{Name: "classify_command", Description: "Pre-flight risk classification for a shell command BEFORE calling run. Returns the risk class (read_only / reversible_low / config_mutation / destructive) that `run` would assign. Use this when you're unsure whether a command will auto-execute or need approval — e.g. `pct exec`, `curl`, compound commands, or anything that might be mistaken for mutation. If this returns read_only, the same command will auto-execute via run with no approval; if it returns config_mutation, expect to need operator approval (or pre-frame the command so it classifies lower). Declared risk can only escalate, never de-escalate.",
|
||||
InputSchema: objSchema(
|
||||
prop{"command", "string", "The exact shell command you intend to pass to run."},
|
||||
prop{"declared_risk", "string", "Optional self-assessment you would pass to run (read_only, reversible_low, config_mutation, destructive). Mirrors run's declared_risk parameter."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
command, _ := args["command"].(string)
|
||||
declaredRisk, _ := args["declared_risk"].(string)
|
||||
if command == "" {
|
||||
return textResult("error: command is required"), nil
|
||||
}
|
||||
risk := policy.ClassifyCommand(command, declaredRisk)
|
||||
note := ""
|
||||
switch risk {
|
||||
case policy.RiskReadOnly:
|
||||
note = "auto-acts on `run` (no approval needed)."
|
||||
case policy.RiskReversibleLow:
|
||||
note = "auto-acts on `run` (no approval needed)."
|
||||
case policy.RiskConfigMutation:
|
||||
note = "requires operator approval on `run` (or loose assent window active)."
|
||||
case policy.RiskDestructive:
|
||||
note = "requires explicit operator confirmation on `run` (typed \"I confirm\" phrase)."
|
||||
}
|
||||
out, _ := json.Marshal(map[string]any{
|
||||
"command": command,
|
||||
"declared_risk": declaredRisk,
|
||||
"risk_class": risk,
|
||||
"note": note,
|
||||
})
|
||||
return textResult(string(out)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_ontology", Description: "Entity types, relationship types, and lifecycle definitions. Use this to understand the schema — what entity types exist, what relationships connect them, and what lifecycle states each type supports.",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
etResult := queryRowsJSONSingle(ctx, pool, `
|
||||
SELECT name, parent_type, is_abstract, domain, layer,
|
||||
description, lifecycle_id, schema_version, status
|
||||
FROM entity_types ORDER BY name`)
|
||||
|
||||
rtResult := queryRowsJSONSingle(ctx, pool, `
|
||||
SELECT name, inverse, source_type, target_type,
|
||||
cardinality, description
|
||||
FROM relationship_types ORDER BY name`)
|
||||
|
||||
lcResult := queryRowsJSONSingle(ctx, pool, `
|
||||
SELECT id, name, states, transitions::text
|
||||
FROM lifecycles ORDER BY name`)
|
||||
|
||||
result := map[string]any{
|
||||
"entity_types": etResult,
|
||||
"relationship_types": rtResult,
|
||||
"lifecycles": lcResult,
|
||||
}
|
||||
b, _ := json.MarshalIndent(result, "", " ")
|
||||
return textResult(string(b)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "http_get", Description: "Fetch a public web page or raw file (e.g. a GitHub README/raw URL) and return sanitized text. Use this to research how to deploy a service before provisioning. HTTP/HTTPS only; body is truncated to ~16KB.",
|
||||
InputSchema: objSchema(
|
||||
prop{"url", "string", "Absolute http(s) URL to fetch"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
rawURL, _ := args["url"].(string)
|
||||
return httpGet(ctx, rawURL), nil
|
||||
}},
|
||||
}
|
||||
}
|
||||
251
internal/mcp/create_entity_test.go
Normal file
251
internal/mcp/create_entity_test.go
Normal file
@@ -0,0 +1,251 @@
|
||||
package mcp
|
||||
|
||||
// Integration tests for the entity-mutation MCP tools (create_entity,
|
||||
// update_entity_attributes), focused on the capability gap that stranded
|
||||
// session 23da10db: entities mutated via MCP must derive/regenerate checks the
|
||||
// same way the HTTP create/patch paths do. Guarded by OIKOS_TEST_DATABASE_URL
|
||||
// (see internal/db/integration_test.go); run via `make test-db`.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math/rand"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/checkdefaults"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
// newTestPool mirrors internal/httpapi/api_test.go: a throwaway database,
|
||||
// migrated and seeded with ontology/inventory/policy so create_entity's type
|
||||
// validation and checkdefaults derivation have a real type tree to work
|
||||
// against.
|
||||
func newTestPool(t *testing.T) *db.Pool {
|
||||
t.Helper()
|
||||
baseURL := os.Getenv("OIKOS_TEST_DATABASE_URL")
|
||||
if baseURL == "" {
|
||||
t.Skip("OIKOS_TEST_DATABASE_URL not set — skipping integration test")
|
||||
}
|
||||
ctx := context.Background()
|
||||
|
||||
admin, err := pgx.Connect(ctx, baseURL)
|
||||
if err != nil {
|
||||
t.Fatalf("connect admin: %v", err)
|
||||
}
|
||||
dbName := fmt.Sprintf("oikos_mcp_test_%08x", rand.Int63())
|
||||
if _, err := admin.Exec(ctx, "CREATE DATABASE "+dbName); err != nil {
|
||||
admin.Close(ctx)
|
||||
t.Fatalf("create test db: %v", err)
|
||||
}
|
||||
admin.Close(ctx)
|
||||
|
||||
qi := strings.Index(baseURL, "?")
|
||||
base, params := baseURL, ""
|
||||
if qi >= 0 {
|
||||
base, params = baseURL[:qi], baseURL[qi:]
|
||||
}
|
||||
testURL := base[:strings.LastIndex(base, "/")+1] + dbName + params
|
||||
|
||||
pool, err := db.New(ctx, testURL)
|
||||
if err != nil {
|
||||
t.Fatalf("connect test db: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
pool.Close()
|
||||
if admin, e := pgx.Connect(ctx, baseURL); e == nil {
|
||||
admin.Exec(ctx, "DROP DATABASE IF EXISTS "+dbName+" WITH (FORCE)")
|
||||
admin.Close(ctx)
|
||||
}
|
||||
})
|
||||
|
||||
if err := pool.Migrate(ctx); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
for _, f := range []string{"ontology.yaml", "inventory.yaml", "policy.yaml"} {
|
||||
content, err := os.ReadFile("../../seeds/" + f)
|
||||
if err != nil {
|
||||
t.Fatalf("read seed %s: %v", f, err)
|
||||
}
|
||||
name := f
|
||||
if err := pool.SeedIngest(ctx, name, content,
|
||||
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
|
||||
var err error
|
||||
switch name {
|
||||
case "ontology.yaml":
|
||||
_, err = db.IngestOntologySeed(ctx, tx, data)
|
||||
case "inventory.yaml":
|
||||
_, err = db.IngestInventorySeed(ctx, tx, data)
|
||||
case "policy.yaml":
|
||||
_, err = db.IngestPolicySeed(ctx, tx, data)
|
||||
}
|
||||
return err
|
||||
}); err != nil {
|
||||
t.Fatalf("ingest %s: %v", f, err)
|
||||
}
|
||||
}
|
||||
return pool
|
||||
}
|
||||
|
||||
// callTool invokes a registered tool's handler in-process and returns its
|
||||
// concatenated text result.
|
||||
func callTool(t *testing.T, pool *db.Pool, name string, args map[string]any) string {
|
||||
t.Helper()
|
||||
var handler toolHandler
|
||||
for _, r := range allTools(pool, uuid.Nil, nil) {
|
||||
if r.tool.Name == name {
|
||||
handler = r.handler
|
||||
break
|
||||
}
|
||||
}
|
||||
if handler == nil {
|
||||
t.Fatalf("tool %q not registered", name)
|
||||
}
|
||||
argsJSON, _ := json.Marshal(args)
|
||||
res, err := handler(context.Background(), &mcp.CallToolRequest{Params: &mcp.CallToolParamsRaw{
|
||||
Name: name,
|
||||
Arguments: argsJSON,
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatalf("tool %s returned error: %v", name, err)
|
||||
}
|
||||
var sb strings.Builder
|
||||
for _, c := range res.Content {
|
||||
if tc, ok := c.(*mcp.TextContent); ok {
|
||||
sb.WriteString(tc.Text)
|
||||
}
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
// checkCountFor returns the number of derived check_defs targeting slug.
|
||||
func checkCountFor(t *testing.T, pool *db.Pool, slug string) int {
|
||||
t.Helper()
|
||||
var n int
|
||||
err := pool.QueryRow(context.Background(),
|
||||
`SELECT count(*) FROM check_defs cd
|
||||
JOIN entities e ON e.id = cd.target_id
|
||||
WHERE e.slug = $1`, slug).Scan(&n)
|
||||
if err != nil {
|
||||
t.Fatalf("count check_defs for %s: %v", slug, err)
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// TestCreateEntity_DerivesChecks proves create_entity inserts an entity AND
|
||||
// derives its default checks in one call (the HTTP create path did this; the
|
||||
// MCP path previously could not create at all).
|
||||
func TestCreateEntity_DerivesChecks(t *testing.T) {
|
||||
pool := newTestPool(t)
|
||||
slug := "service:mcp-create-test"
|
||||
|
||||
out := callTool(t, pool, "create_entity", map[string]any{
|
||||
"type": "service",
|
||||
"slug": slug,
|
||||
"name": "mcp-create-test",
|
||||
"attributes": `{"url":"https://mcp-create-test.example"}`,
|
||||
})
|
||||
if !strings.Contains(out, "Created "+slug) {
|
||||
t.Fatalf("create_entity result = %q, want Created %s", out, slug)
|
||||
}
|
||||
if !strings.Contains(out, "Derived") {
|
||||
t.Errorf("create_entity result = %q, want a Derived check summary", out)
|
||||
}
|
||||
if got := checkCountFor(t, pool, slug); got < 1 {
|
||||
t.Errorf("check_defs targeting %s = %d, want >=1 (create did not derive checks)", slug, got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCreateEntity_DuplicateAndInvalid covers the guard rails: a repeat create
|
||||
// is reported as "already exists" (not an error), and an unknown type is
|
||||
// rejected with a clear message.
|
||||
func TestCreateEntity_DuplicateAndInvalid(t *testing.T) {
|
||||
pool := newTestPool(t)
|
||||
|
||||
if out := callTool(t, pool, "create_entity", map[string]any{
|
||||
"type": "service", "slug": "service:mcp-dup", "name": "mcp-dup",
|
||||
}); !strings.Contains(out, "Created service:mcp-dup") {
|
||||
t.Fatalf("first create = %q", out)
|
||||
}
|
||||
if out := callTool(t, pool, "create_entity", map[string]any{
|
||||
"type": "service", "slug": "service:mcp-dup", "name": "mcp-dup",
|
||||
}); !strings.Contains(out, "already exists") {
|
||||
t.Errorf("duplicate create = %q, want 'already exists'", out)
|
||||
}
|
||||
if out := callTool(t, pool, "create_entity", map[string]any{
|
||||
"type": "no-such-type", "slug": "no-such-type:x", "name": "x",
|
||||
}); !strings.Contains(out, "not found in ontology") {
|
||||
t.Errorf("unknown type = %q, want 'not found in ontology'", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUpdateEntityAttributes_RegeneratesChecks is the regression guard for the
|
||||
// haos session: setting an entity's `monitoring` attribute via MCP must
|
||||
// regenerate checks. Before this fix the MCP update path skipped
|
||||
// ensureDefaultChecks, so flipping monitoring produced nothing.
|
||||
func TestUpdateEntityAttributes_RegeneratesChecks(t *testing.T) {
|
||||
pool := newTestPool(t)
|
||||
slug := "service:mcp-regen-test"
|
||||
|
||||
// Create with monitoring:none — no checks derived.
|
||||
if out := callTool(t, pool, "create_entity", map[string]any{
|
||||
"type": "service", "slug": slug, "name": "mcp-regen-test",
|
||||
"attributes": `{"monitoring":"none","url":"https://mcp-regen.example"}`,
|
||||
}); !strings.Contains(out, "Created "+slug) {
|
||||
t.Fatalf("create = %q", out)
|
||||
}
|
||||
if got := checkCountFor(t, pool, slug); got != 0 {
|
||||
t.Fatalf("check_defs with monitoring:none = %d, want 0", got)
|
||||
}
|
||||
|
||||
// Flip monitoring to [http] via update_entity_attributes — checks must
|
||||
// regenerate. This is exactly what failed for service:haos.
|
||||
out := callTool(t, pool, "update_entity_attributes", map[string]any{
|
||||
"slug": slug,
|
||||
"attributes": `{"monitoring":["http"]}`,
|
||||
})
|
||||
if !strings.Contains(out, "Updated "+slug) {
|
||||
t.Fatalf("update result = %q, want Updated %s", out, slug)
|
||||
}
|
||||
if !strings.Contains(out, "Derived") {
|
||||
t.Errorf("update result = %q, want a Derived check summary (regeneration)", out)
|
||||
}
|
||||
if got := checkCountFor(t, pool, slug); got < 1 {
|
||||
t.Errorf("check_defs after monitoring:[http] = %d, want >=1 (MCP update did not regenerate checks)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUpdateEntityAttributes_NotFound keeps the existing error contract.
|
||||
func TestUpdateEntityAttributes_NotFound(t *testing.T) {
|
||||
pool := newTestPool(t)
|
||||
out := callTool(t, pool, "update_entity_attributes", map[string]any{
|
||||
"slug": "service:does-not-exist",
|
||||
"attributes": `{"x":1}`,
|
||||
})
|
||||
if !strings.Contains(out, "not found") {
|
||||
t.Errorf("update missing entity = %q, want 'not found'", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestFormatCheckResult is a pure unit test for the result-message helper, so
|
||||
// the formatting contract holds even when the DB is unavailable.
|
||||
func TestFormatCheckResult(t *testing.T) {
|
||||
if got := formatCheckResult(checkdefaults.Result{Created: 2}); !strings.Contains(got, "Derived 2 check") {
|
||||
t.Errorf("created-only = %q, want Derived 2", got)
|
||||
}
|
||||
got := formatCheckResult(checkdefaults.Result{Created: 1, Skipped: []checkdefaults.Skip{{Kind: "process", Reason: "no host"}}})
|
||||
if !strings.Contains(got, "Derived 1 check") || !strings.Contains(got, "Skipped process") || !strings.Contains(got, "no host") {
|
||||
t.Errorf("created+skipped = %q", got)
|
||||
}
|
||||
if got := formatCheckResult(checkdefaults.Result{Undeclared: true}); !strings.Contains(got, "no monitoring") {
|
||||
t.Errorf("undeclared = %q, want no-monitoring hint", got)
|
||||
}
|
||||
if formatCreateResult("a", "b", checkdefaults.Result{Created: 0}) != "Created a (b)." {
|
||||
t.Error("create result with no checks should have no suffix")
|
||||
}
|
||||
}
|
||||
501
internal/mcp/entity_tools.go
Normal file
501
internal/mcp/entity_tools.go
Normal file
@@ -0,0 +1,501 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/audit"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
func EntityTools(pool *db.Pool, agentID uuid.UUID, sec secretBackend) []toolReg {
|
||||
return []toolReg{
|
||||
{tool: &mcp.Tool{Name: "ping", Description: "Lightweight connectivity check. Returns server identity, no DB hit.",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
return textResult(`{"ok":true,"server":"oikos","version":"dev"}`), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_entity", Description: "Get an entity by slug or UUID",
|
||||
InputSchema: objSchema(prop{"slug_or_id", "string", "Entity slug (e.g. host:hubris) or UUID"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
idOrSlug, _ := args["slug_or_id"].(string)
|
||||
return queryEntity(ctx, pool, idOrSlug), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "list_entities", Description: "List entities filtered by type, state, or search",
|
||||
InputSchema: objSchema(
|
||||
prop{"type", "string", "Filter by entity type"},
|
||||
prop{"state", "string", "Filter by lifecycle state"},
|
||||
prop{"q", "string", "Substring match on slug or name"},
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state, e.version, e.created_at, e.updated_at
|
||||
FROM entities e
|
||||
WHERE ($1::text IS NULL OR e.type = $1)
|
||||
AND ($2::text IS NULL OR e.state = $2)
|
||||
AND ($3::text IS NULL OR e.slug ILIKE '%'||$3||'%' OR e.name ILIKE '%'||$3||'%')
|
||||
ORDER BY e.slug LIMIT $4`,
|
||||
nStr(args["type"]), nStr(args["state"]), nStr(args["q"]), limit), "entity_table"), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_relations", Description: "List inbound/outbound edges for one entity, optionally filtered by relationship type",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_id", "string", "Entity slug"},
|
||||
prop{"types", "string", "Comma-separated relationship types to include (e.g. 'hosts,provides,depends-on'). Omit for all."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_id"].(string)
|
||||
typesStr, _ := args["types"].(string)
|
||||
if slug == "" {
|
||||
return textResult("entity_id is required"), nil
|
||||
}
|
||||
query := `
|
||||
SELECT r.type, src.slug AS source, tgt.slug AS target
|
||||
FROM relationships r
|
||||
JOIN entities src ON src.id = r.source_id
|
||||
JOIN entities tgt ON tgt.id = r.target_id
|
||||
WHERE (src.slug = $1 OR tgt.slug = $1) AND r.valid_to IS NULL`
|
||||
if typesStr != "" {
|
||||
query += ` AND r.type = ANY(string_to_array($2, ','))`
|
||||
return queryRows(ctx, pool, query, slug, typesStr), nil
|
||||
}
|
||||
query += ` ORDER BY r.type`
|
||||
return queryRows(ctx, pool, query, slug), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_blast_radius", Description: "Find entities affected if this entity goes down",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_id", "string", "Entity slug"},
|
||||
prop{"depth", "integer", "Traversal depth (default 3)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_id"].(string)
|
||||
depth := int(getFloat(args, "depth", 3))
|
||||
return queryRows(ctx, pool,
|
||||
"SELECT e.slug, CAST(b.depth AS int) FROM blast_radius((SELECT id FROM entities WHERE slug = $1), $2) b JOIN entities e ON e.id = b.entity_id",
|
||||
slug, depth), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "create_entity", Description: "Create a new entity in the knowledge graph. Use it when a task needs an entity that does not exist yet: a service, a host/LXC/VM, an ingress, a cert, etc. After inserting, it derives default checks from the entity type's monitoring spec, so creating a checkable entity wires its monitoring in one call. Does NOT require approval. If the slug already exists it returns 'already exists' — then use update_entity_attributes to change it. FOOTGUN: creating a type=check entity creates a bare entity row but does NOT wire a check_def — the scheduler will never probe it. To add monitoring, set `monitoring: [\"http\"]` + `url` on the target via update_entity_attributes.",
|
||||
InputSchema: objSchema(
|
||||
prop{"type", "string", "Entity type — must already exist in the ontology and not be abstract (e.g. service, lxc, host, vm, check, ingress, cert, dns)."},
|
||||
prop{"name", "string", "Human-readable name (e.g. 'HAOS http service check')."},
|
||||
prop{"slug", "string", "Entity slug (e.g. check:http:service:haos:0, ingress:home.hubris.network). If omitted, defaults to <type>:<name>."},
|
||||
prop{"attributes", "string", "JSON object string of attributes, e.g. {\"check_type\":\"http:service\",\"target\":\"service:haos\",\"port\":\"8123\"}. Optional."},
|
||||
prop{"state", "string", "Lifecycle state. Optional; defaults to the type's lifecycle default_state."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
entityType, _ := args["type"].(string)
|
||||
name, _ := args["name"].(string)
|
||||
slug, _ := args["slug"].(string)
|
||||
if slug == "" && entityType != "" && name != "" {
|
||||
slug = entityType + ":" + name
|
||||
}
|
||||
if entityType == "" || name == "" || slug == "" {
|
||||
return textResult("error: type and name are required (slug defaults to <type>:<name>)"), nil
|
||||
}
|
||||
attrsStr, _ := args["attributes"].(string)
|
||||
attrs := map[string]any{}
|
||||
if attrsStr != "" {
|
||||
if err := json.Unmarshal([]byte(attrsStr), &attrs); err != nil {
|
||||
return textResult(fmt.Sprintf("error: attributes is not valid JSON: %v", err)), nil
|
||||
}
|
||||
}
|
||||
attrsJSON, _ := json.Marshal(attrs)
|
||||
stateStr, _ := args["state"].(string)
|
||||
|
||||
tx, err := pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: begin tx: %v", err)), nil
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
// Validate the type exists and is concrete (mirror httpapi.CreateEntity).
|
||||
var isAbstract bool
|
||||
if err := tx.QueryRow(ctx, `SELECT is_abstract FROM entity_types WHERE name = $1`, entityType).Scan(&isAbstract); err != nil {
|
||||
return textResult(fmt.Sprintf("error: entity type %q not found in ontology", entityType)), nil
|
||||
}
|
||||
if isAbstract {
|
||||
return textResult(fmt.Sprintf("error: type %q is abstract — pick a concrete subtype", entityType)), nil
|
||||
}
|
||||
|
||||
// Default state from the type's lifecycle unless the caller
|
||||
// supplied one. Caller-supplied states are validated against
|
||||
// the lifecycle's declared states — a create_entity bypass of
|
||||
// lifecycle guardrails would let an agent create in a terminal
|
||||
// state (destroyed) without satisfying the preconditions that
|
||||
// set_entity_state enforces for the same transition.
|
||||
var state *string
|
||||
var lsDefault, statesRaw string
|
||||
if err := tx.QueryRow(ctx, `SELECT coalesce(ld.default_state,''), coalesce(ld.states::text,'')
|
||||
FROM lifecycle_defs ld
|
||||
JOIN entity_types et ON et.lifecycle_id = ld.id
|
||||
WHERE et.name = $1`, entityType).Scan(&lsDefault, &statesRaw); err == nil {
|
||||
var validStates []string
|
||||
json.Unmarshal([]byte(statesRaw), &validStates)
|
||||
if stateStr != "" {
|
||||
found := false
|
||||
for _, s := range validStates {
|
||||
if s == stateStr {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found && len(validStates) > 0 {
|
||||
return textResult(fmt.Sprintf("error: state %q not declared in %s lifecycle (states: %s). Use the default (%s) or omit state.", stateStr, entityType, strings.Join(validStates, ","), lsDefault)), nil
|
||||
}
|
||||
state = &stateStr
|
||||
} else if lsDefault != "" {
|
||||
state = &lsDefault
|
||||
}
|
||||
}
|
||||
|
||||
id, err := uuid.NewV7()
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: gen id: %v", err)), nil
|
||||
}
|
||||
|
||||
var createdName string
|
||||
if err := tx.QueryRow(ctx, `
|
||||
INSERT INTO entities (id, slug, type, name, state, attributes)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)
|
||||
RETURNING name`,
|
||||
id, slug, entityType, name, state, attrsJSON).Scan(&createdName); err != nil {
|
||||
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
|
||||
return textResult(fmt.Sprintf("Entity %q already exists — use update_entity_attributes to change it.", slug)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("error creating %s: %v", slug, err)), nil
|
||||
}
|
||||
|
||||
res, derr := db.EnsureEntityChecks(ctx, tx, id, slug, entityType, createdName, attrsJSON)
|
||||
if derr != nil {
|
||||
return textResult(fmt.Sprintf("error deriving checks for %s: %v", slug, derr)), nil
|
||||
}
|
||||
if cerr := tx.Commit(ctx); cerr != nil {
|
||||
return textResult(fmt.Sprintf("error committing %s: %v", slug, cerr)), nil
|
||||
}
|
||||
|
||||
return textResult(formatCreateResult(slug, entityType, res)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "update_entity_attributes", Description: "Merge new/changed attributes into an entity — the OTHER half of avoiding knowledge-base drift (upsert_knowledge records what you learned; this keeps the entity's own facts current). Use it when you discover something concrete about an entity's actual state that the graph doesn't reflect yet: a new IP, a version number, a config value, a discovered port — anything a FUTURE task would otherwise have to rediscover from scratch. Does NOT require approval (this updates the knowledge graph, not the live infrastructure). Merges shallowly — existing keys not mentioned are kept; keys you pass overwrite.",
|
||||
InputSchema: objSchema(
|
||||
prop{"slug", "string", "Entity slug to update (e.g. lxc:typetype, host:strong)."},
|
||||
prop{"attributes", "string", "JSON object string of attributes to merge in, e.g. {\"lan_ip\":\"192.168.8.50\",\"os\":\"debian-12\"}."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["slug"].(string)
|
||||
attrsStr, _ := args["attributes"].(string)
|
||||
if slug == "" || attrsStr == "" {
|
||||
return textResult("error: slug and attributes are required"), nil
|
||||
}
|
||||
var attrs map[string]any
|
||||
if err := json.Unmarshal([]byte(attrsStr), &attrs); err != nil {
|
||||
return textResult(fmt.Sprintf("error: attributes is not valid JSON: %v", err)), nil
|
||||
}
|
||||
|
||||
// Strip scheduler-owned keys: health is computed by the scheduler
|
||||
// from probe results (spotted live 2026-08-05: an agent set
|
||||
// health:"healthy" on lxc:nfs-export, which derived 4 spurious checks).
|
||||
// Agents can observe health via get_health_summary / list_checks.
|
||||
var blocked []string
|
||||
for _, key := range []string{"health", "last_check_at", "last_check"} {
|
||||
if _, ok := attrs[key]; ok {
|
||||
delete(attrs, key)
|
||||
blocked = append(blocked, key)
|
||||
}
|
||||
}
|
||||
if len(blocked) > 0 {
|
||||
// Re-marshal the filtered attrs
|
||||
filtered, _ := json.Marshal(attrs)
|
||||
attrsStr = string(filtered)
|
||||
if len(attrs) == 0 {
|
||||
return textResult(fmt.Sprintf("Updated %s: no allowed attributes provided. The following keys are scheduler-owned and ignored: %s. Use get_health_summary or list_checks to observe entity health.", slug, strings.Join(blocked, ", "))), nil
|
||||
}
|
||||
}
|
||||
attrsJSON, _ := json.Marshal(attrs)
|
||||
|
||||
// Run the merge + check regeneration in one transaction so the
|
||||
// derived checks always see the post-merge attributes. Mirrors
|
||||
// httpapi.PatchEntity; without this, setting an entity's
|
||||
// `monitoring` attribute via MCP silently produced no checks.
|
||||
tx, err := pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: begin tx: %v", err)), nil
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
ra, err := sqlcgen.New(tx).MergeEntityAttributes(ctx, sqlcgen.MergeEntityAttributesParams{Slug: slug, Patch: attrsJSON})
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error updating %s: %v", slug, err)), nil
|
||||
}
|
||||
if ra == 0 {
|
||||
return textResult(fmt.Sprintf("error: entity %q not found", slug)), nil
|
||||
}
|
||||
|
||||
merged, err := sqlcgen.New(tx).GetEntityBySlug(ctx, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error reloading %s: %v", slug, err)), nil
|
||||
}
|
||||
id := merged.ID
|
||||
entityType := merged.Type
|
||||
name := merged.Name
|
||||
mergedAttrs := merged.Attributes
|
||||
|
||||
res, cerr := db.EnsureEntityChecks(ctx, tx, id, slug, entityType, name, mergedAttrs)
|
||||
if cerr != nil {
|
||||
return textResult(fmt.Sprintf("error deriving checks for %s: %v", slug, cerr)), nil
|
||||
}
|
||||
if cerr := tx.Commit(ctx); cerr != nil {
|
||||
return textResult(fmt.Sprintf("error committing %s: %v", slug, cerr)), nil
|
||||
}
|
||||
|
||||
return textResult(fmt.Sprintf("Updated %s with %d attribute(s).%s", slug, len(attrs), formatCheckResult(res))), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "set_entity_state", Description: "Transition an entity to a new lifecycle state — the entity-graph \"delete\" surface, since this system never hard-deletes entities. Use retire/deprecate to take an entity out of service, destroy for terminal removal, or active to revive. The target state must be a declared transition in the entity type's lifecycle (e.g. active→deprecated, deprecated→active); preconditions (no inbound edges, backups verified, etc.) are enforced — an error tells you what's blocking. Does NOT require approval (knowledge-graph mutation, not live infrastructure).",
|
||||
InputSchema: objSchema(
|
||||
prop{"slug", "string", "Entity slug."},
|
||||
prop{"state", "string", "Target lifecycle state."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["slug"].(string)
|
||||
targetState, _ := args["state"].(string)
|
||||
if slug == "" || targetState == "" {
|
||||
return textResult("error: slug and state are required"), nil
|
||||
}
|
||||
|
||||
tx, err := pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: begin tx: %v", err)), nil
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
ent, err := sqlcgen.New(tx).GetEntityBySlug(ctx, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: entity %q not found", slug)), nil
|
||||
}
|
||||
id := ent.ID
|
||||
entityType := ent.Type
|
||||
currentState := ""
|
||||
if ent.State != nil {
|
||||
currentState = *ent.State
|
||||
}
|
||||
if err := db.ValidateTransition(ctx, tx, id, entityType, currentState, targetState); err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
ra, err := sqlcgen.New(tx).SetEntityState(ctx, sqlcgen.SetEntityStateParams{ID: id, State: &targetState})
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error updating %s: %v", slug, err)), nil
|
||||
}
|
||||
if ra == 0 {
|
||||
return textResult(fmt.Sprintf("error: entity %q not found", slug)), nil
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return textResult(fmt.Sprintf("error: commit: %v", err)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Transitioned %s: %s → %s.", slug, currentState, targetState)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "create_relationship", Description: "Record a relationship you discovered between two entities — the graph-structure half of keeping the knowledge base current (alongside update_entity_attributes and upsert_knowledge). Use it when you learn that one entity depends on, hosts, routes to, etc. another, and that edge isn't in the graph yet. type must be an existing relationship type (see get_relations output on similar entities for examples: hosts, provides, depends-on, configured-by, about, documents, ...). Idempotent — re-calling the same source/target/type is a no-op. Does NOT require approval.",
|
||||
InputSchema: objSchema(
|
||||
prop{"source", "string", "Source entity slug."},
|
||||
prop{"target", "string", "Target entity slug."},
|
||||
prop{"type", "string", "Relationship type name (must already exist in the ontology)."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
source, _ := args["source"].(string)
|
||||
target, _ := args["target"].(string)
|
||||
relType, _ := args["type"].(string)
|
||||
if source == "" || target == "" || relType == "" {
|
||||
return textResult("error: source, target, and type are required"), nil
|
||||
}
|
||||
srcEnt, err := sqlcgen.New(pool).GetEntityBySlug(ctx, source)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: source entity %q not found", source)), nil
|
||||
}
|
||||
tgtEnt, err := sqlcgen.New(pool).GetEntityBySlug(ctx, target)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: target entity %q not found", target)), nil
|
||||
}
|
||||
_, err = sqlcgen.New(pool).InsertRelationshipIfAbsent(ctx, sqlcgen.InsertRelationshipIfAbsentParams{
|
||||
SourceID: srcEnt.ID, TargetID: tgtEnt.ID, Type: relType, Attributes: []byte(`{"by":"nomos"}`),
|
||||
})
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error creating relationship: %v (is %q a valid relationship type?)", err, relType)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Recorded: %s —%s→ %s", source, relType, target)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "end_relationship", Description: "End an existing relationship (soft-delete by setting valid_to) — the graph-structure \"delete\" surface. Use it when you discover an edge is no longer true (a service moved hosts, a route was removed, a dependency dissolved). The edge is kept for history; only the currently-active edge is ended. Idempotent — ending an already-ended or non-existent edge is a no-op. Does NOT require approval.",
|
||||
InputSchema: objSchema(
|
||||
prop{"source", "string", "Source entity slug."},
|
||||
prop{"target", "string", "Target entity slug."},
|
||||
prop{"type", "string", "Relationship type name."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
source, _ := args["source"].(string)
|
||||
target, _ := args["target"].(string)
|
||||
relType, _ := args["type"].(string)
|
||||
if source == "" || target == "" || relType == "" {
|
||||
return textResult("error: source, target, and type are required"), nil
|
||||
}
|
||||
srcEnt, err := sqlcgen.New(pool).GetEntityBySlug(ctx, source)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: source entity %q not found", source)), nil
|
||||
}
|
||||
tgtEnt, err := sqlcgen.New(pool).GetEntityBySlug(ctx, target)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: target entity %q not found", target)), nil
|
||||
}
|
||||
ra, err := sqlcgen.New(pool).EndCurrentRelationship(ctx, sqlcgen.EndCurrentRelationshipParams{
|
||||
SourceID: srcEnt.ID, TargetID: tgtEnt.ID, Type: relType,
|
||||
})
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error ending relationship: %v", err)), nil
|
||||
}
|
||||
if ra == 0 {
|
||||
return textResult(fmt.Sprintf("No active relationship %s —%s→ %s found.", source, relType, target)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Ended: %s —%s→ %s.", source, relType, target)), nil
|
||||
}},
|
||||
// ─── Client introspection tools (plan: client-lifecycle Phase 3) ──
|
||||
|
||||
{tool: &mcp.Tool{Name: "whoami", Description: "Get the current entity record, peers, and health for a host",
|
||||
InputSchema: objSchema(prop{"hostname", "string", "Hostname of the calling machine"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
hostname, _ := args["hostname"].(string)
|
||||
if hostname == "" {
|
||||
return textResult("error: hostname required"), nil
|
||||
}
|
||||
slug := "ws:" + hostname
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state,
|
||||
COALESCE(st.health, 'unknown') AS health,
|
||||
COALESCE(st.last_check_at::text, '') AS last_check,
|
||||
e.attributes->>'mesh_ip' AS mesh_ip,
|
||||
e.attributes->>'age_pubkey' AS age_pubkey,
|
||||
e.enrolled_at
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.slug = $1
|
||||
ORDER BY e.slug`, slug), "entity_card"), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "explain", Description: "Compact context card for a service: type, state, health, relations, risk",
|
||||
InputSchema: objSchema(prop{"service_slug", "string", "Service entity slug (e.g. service:jellyfin, lxc:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("error: service_slug required"), nil
|
||||
}
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state,
|
||||
COALESCE(st.health, 'unknown') AS health,
|
||||
COALESCE(st.last_check_at::text, '') AS last_check,
|
||||
e.version, e.updated_at,
|
||||
COALESCE(e.attributes::text, '{}') AS attrs
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.slug = $1`, slug), "entity_card"), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "preflight", Description: "Risk classification for an action on a service",
|
||||
InputSchema: objSchema(
|
||||
prop{"service_slug", "string", "Entity slug"},
|
||||
prop{"action", "string", "Planned action (restart, deploy, destroy, etc.)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
action, _ := args["action"].(string)
|
||||
if slug == "" || action == "" {
|
||||
return textResult("error: service_slug and action required"), nil
|
||||
}
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.state,
|
||||
CASE
|
||||
WHEN $2 IN ('restart', 'logs', 'status') THEN 'reversible_low'
|
||||
WHEN $2 IN ('deploy', 'upgrade', 'configure') THEN 'config_mutation'
|
||||
WHEN $2 IN ('destroy', 'wipe', 'revoke') THEN 'destructive'
|
||||
ELSE 'read_only'
|
||||
END AS risk_class,
|
||||
CASE
|
||||
WHEN $2 IN ('read_only','reversible_low') THEN 'auto-act'
|
||||
WHEN $2 = 'config_mutation' THEN 'operator-approval'
|
||||
ELSE 'operator-approval+confirmation'
|
||||
END AS approval
|
||||
FROM entities e WHERE e.slug = $1`, slug, action), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_change_history", Description: "Last N change-ledger entries for an entity",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_slug", "string", "Entity slug"},
|
||||
prop{"limit", "integer", "Max entries (default 20)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_slug"].(string)
|
||||
limit := int(getFloat(args, "limit", 20))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT al.ts AS timestamp, al.actor_type, al.actor_id::text AS actor_label,
|
||||
al.action, al.method, al.path,
|
||||
al.detail::text AS details, al.session_id::text AS session_id
|
||||
FROM audit_log al
|
||||
JOIN entities e ON e.id = al.entity_id
|
||||
WHERE e.slug = $1
|
||||
ORDER BY al.ts DESC
|
||||
LIMIT $2`, slug, limit), "change_log"), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_state_snapshot", Description: "Last scheduler Observe-pass: fleet health, disk, drift count",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.state,
|
||||
COALESCE(st.health, 'unknown') AS health,
|
||||
COALESCE(st.last_check_at::text, '') AS last_check
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.state IS NOT NULL
|
||||
OR st.health IS NOT NULL
|
||||
ORDER BY st.health, e.slug
|
||||
LIMIT 200
|
||||
`), "fleet_snapshot"), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "audit_knowledge_graph", Description: "Read-only drift report over the knowledge graph and monitoring: orphan check entities, checks targeting deprecated/destroyed entities, probes stuck down/unknown, unmonitored declared entity types, and live edges pointing at destroyed targets. Returns ranked findings with a suggested remediation runbook each. Use this to validate the graph is complete and consistent before trusting health/blast-radius answers. Does NOT mutate anything.",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
findings, summary := audit.Report(ctx, pool)
|
||||
b, _ := json.Marshal(map[string]any{"findings": findings, "summary": summary})
|
||||
return textResult(string(b)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "discover_infra_drift", Description: "Read-only live discovery: compares running Proxmox guests (pct/qm list on every proxmox host) against the DB graph. Returns guests running with no entity (missing) and entities whose pve_id is no longer live (ghost) — drift the DB-only audit_knowledge_graph cannot see. Reaches hosts over the same SSH/pct path the checks use. Does NOT mutate anything.",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
b, _ := json.Marshal(discoverInfraDrift(ctx, pool))
|
||||
return textResult(string(b)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "find_entities_by", Description: "Search entities by discovered attributes — IP address, port, version string, tag, or any key in the attributes JSONB blob. More flexible than list_entities (which filters by type/state only). Use for reverse lookups: 'what runs on port 8096?' or 'which entities have version 2.4?'",
|
||||
InputSchema: objSchema(
|
||||
prop{"key", "string", "Attribute key to search (e.g. ip, port, version, tag)"},
|
||||
prop{"value", "string", "Value to match (case-insensitive substring)"},
|
||||
prop{"limit", "integer", "Max results (default 25)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
key, _ := args["key"].(string)
|
||||
val, _ := args["value"].(string)
|
||||
limit := int(getFloat(args, "limit", 25))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state, e.attributes->>$1 AS matched_value
|
||||
FROM entities e
|
||||
WHERE e.attributes ? $1
|
||||
AND e.attributes->>$1 ILIKE '%'||$2||'%'
|
||||
ORDER BY e.slug
|
||||
LIMIT $3`, key, val, limit), nil
|
||||
}},
|
||||
}
|
||||
}
|
||||
339
internal/mcp/knowledge_tools.go
Normal file
339
internal/mcp/knowledge_tools.go
Normal file
@@ -0,0 +1,339 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
func KnowledgeTools(pool *db.Pool, agentID uuid.UUID, sec secretBackend) []toolReg {
|
||||
return []toolReg{
|
||||
{tool: &mcp.Tool{Name: "search_knowledge", Description: "Full-text search across documentation (PostgreSQL FTS with ts_rank ranking). Returns a short snippet per hit, not the full note — call get_knowledge_content with the returned slug to read the whole thing.",
|
||||
InputSchema: objSchema(prop{"query", "string", "Search terms"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
q := nStr(args["query"])
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT ke.title, e.slug,
|
||||
ts_rank(ke.search, plainto_tsquery('english', $1)) AS rank,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', $1),
|
||||
'MaxWords=40, MinWords=15, ShortWord=3, MaxFragments=3,
|
||||
FragmentDelimiter=" ... "') AS snippet,
|
||||
ke.source, ke.tags
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE ke.search @@ plainto_tsquery('english', $1)
|
||||
ORDER BY rank DESC
|
||||
LIMIT 20`, q), "knowledge_results"), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_entity_knowledge", Description: "All documents, investigations, and runbooks linked to an entity. Returns a headline per note, not the full text — call get_knowledge_content with the returned slug to read the whole thing.",
|
||||
InputSchema: objSchema(prop{"entity_slug", "string", "Entity slug (e.g. lxc:jellyfin, service:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_slug"].(string)
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT ke.title, ke.source, e.type AS kind, e.slug,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', '')) AS headline
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
JOIN relationships r ON r.source_id = ke.entity_id
|
||||
JOIN entities target ON target.id = r.target_id
|
||||
WHERE target.slug = $1
|
||||
AND r.valid_to IS NULL
|
||||
AND r.type IN ('documents', 'about')
|
||||
UNION
|
||||
SELECT ke.title, ke.source, e.type AS kind, e.slug,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', '')) AS headline
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
JOIN relationships r ON r.source_id = ke.entity_id
|
||||
JOIN entity_types target_type ON target_type.name = (SELECT type FROM entities WHERE slug = $1)
|
||||
JOIN entities ent ON ent.type = target_type.name AND ent.slug = $1
|
||||
WHERE r.valid_to IS NULL
|
||||
AND r.type = 'procedure-for'
|
||||
ORDER BY 1`, slug), "knowledge_results"), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_knowledge_content", Description: "Full markdown body of one document/investigation/runbook, by its own entity slug. search_knowledge and get_entity_knowledge only return short snippets/headlines — once you know which note you need (from either of those, or because you already know its slug), call this to read the whole thing before acting on it.",
|
||||
InputSchema: objSchema(prop{"slug", "string", "The knowledge entity's own slug (e.g. document:containers/101-jellyfin, runbook:client-enrollment) — not the slug of an entity it's about."}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["slug"].(string)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT ke.title, e.slug, e.type AS kind, ke.content, ke.source, ke.tags, ke.updated_at::text
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE e.slug = $1`, slug), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "upsert_knowledge", Description: "Write back what you learned so future sessions (and future you) benefit — this is how the system gets smarter over time. Use it AFTER solving a non-obvious problem, deploying a service, or discovering a gotcha: record the finding, the fix, and any caveats. Re-calling with the same title updates the existing note instead of duplicating. This is the ONLY way to persist knowledge; a chat message alone is forgotten. search_knowledge/get_entity_knowledge find it, get_knowledge_content reads the full body back.",
|
||||
InputSchema: objSchema(
|
||||
prop{"title", "string", "Short, specific, searchable title (e.g. 'Dragonfly memlock rlimit in unprivileged LXCs', not 'notes')."},
|
||||
prop{"content", "string", "The knowledge itself, in markdown. Be concrete: symptom, root cause, the exact fix/commands, and any caveats. Written for someone hitting this fresh."},
|
||||
prop{"about", "string", "Optional entity slug(s) this knowledge concerns. Pass a single slug (e.g. 'lxc:nfs-export') or a JSON array of slugs (e.g. '[\"lxc:nfs-export\", \"lxc:gitea\"]') to link to multiple entities. get_entity_knowledge surfaces it for each."},
|
||||
prop{"tags", "string", "Optional comma-separated tags (e.g. 'docker,networking,gotcha')."},
|
||||
prop{"kind", "string", "One of: investigation (a finding/incident analysis — default), document (reference), runbook (a repeatable procedure)."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return upsertKnowledge(ctx, pool, args)
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "delete_knowledge", Description: "Soft-delete a knowledge entry (move to trash, restorable with restore_knowledge). The content and revision history survive.",
|
||||
InputSchema: objSchema(prop{"knowledge_slug", "string", "Knowledge entity slug or UUID"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["knowledge_slug"].(string)
|
||||
var entityID uuid.UUID
|
||||
if u, err := uuid.Parse(slug); err == nil {
|
||||
entityID = u
|
||||
} else {
|
||||
pool.QueryRow(ctx, `SELECT id FROM entities WHERE slug = $1`, slug).Scan(&entityID)
|
||||
}
|
||||
if entityID == uuid.Nil {
|
||||
return textResult(fmt.Sprintf("knowledge entry not found: %s", slug)), nil
|
||||
}
|
||||
// Snapshot before tombstoning.
|
||||
pool.Exec(ctx, `
|
||||
INSERT INTO knowledge_revisions (entity_id, title, content, source, tags, edited_by, version_at)
|
||||
SELECT entity_id, title, content, source, tags, COALESCE(edited_by,''), updated_at
|
||||
FROM knowledge_entities WHERE entity_id = $1`, entityID)
|
||||
tag, err := pool.Exec(ctx,
|
||||
`UPDATE knowledge_entities SET deleted_at = now(), edited_by = 'nomos'
|
||||
WHERE entity_id = $1 AND deleted_at IS NULL`, entityID)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return textResult("knowledge entry already deleted"), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Knowledge %s soft-deleted. Restore with restore_knowledge.", slug)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "restore_knowledge", Description: "Restore a soft-deleted knowledge entry from trash. Undoes delete_knowledge.",
|
||||
InputSchema: objSchema(prop{"knowledge_slug", "string", "Knowledge entity slug or UUID"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["knowledge_slug"].(string)
|
||||
var entityID uuid.UUID
|
||||
if u, err := uuid.Parse(slug); err == nil {
|
||||
entityID = u
|
||||
} else {
|
||||
pool.QueryRow(ctx, `SELECT id FROM entities WHERE slug = $1`, slug).Scan(&entityID)
|
||||
}
|
||||
if entityID == uuid.Nil {
|
||||
return textResult(fmt.Sprintf("knowledge entry not found: %s", slug)), nil
|
||||
}
|
||||
tag, err := pool.Exec(ctx,
|
||||
`UPDATE knowledge_entities SET deleted_at = NULL, edited_by = 'nomos'
|
||||
WHERE entity_id = $1 AND deleted_at IS NOT NULL`, entityID)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return textResult("knowledge entry is not deleted"), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Knowledge %s restored from trash.", slug)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "merge_knowledge", Description: "Fold one or more knowledge entries into a target. Source content is appended under a provenance heading, and the union of all tags is kept. Sources are soft-deleted afterwards.",
|
||||
InputSchema: objSchema(
|
||||
prop{"target_slug", "string", "Knowledge entry to merge INTO (slug or UUID)"},
|
||||
prop{"source_slugs", "string", "Comma-separated slugs of entries to fold into the target"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
targetSlug, _ := args["target_slug"].(string)
|
||||
sourceStr, _ := args["source_slugs"].(string)
|
||||
|
||||
var targetID uuid.UUID
|
||||
if u, err := uuid.Parse(targetSlug); err == nil {
|
||||
targetID = u
|
||||
} else {
|
||||
pool.QueryRow(ctx, `
|
||||
SELECT ke.entity_id FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE (e.slug = $1 OR e.id::text = $1) AND ke.deleted_at IS NULL`,
|
||||
targetSlug).Scan(&targetID)
|
||||
}
|
||||
if targetID == uuid.Nil {
|
||||
return textResult(fmt.Sprintf("target knowledge entry not found: %s", targetSlug)), nil
|
||||
}
|
||||
|
||||
sources := []string{}
|
||||
for _, s := range strings.Split(sourceStr, ",") {
|
||||
if s = strings.TrimSpace(s); s != "" && s != targetSlug {
|
||||
sources = append(sources, s)
|
||||
}
|
||||
}
|
||||
if len(sources) == 0 {
|
||||
return textResult("no valid source entries to merge"), nil
|
||||
}
|
||||
|
||||
var appended strings.Builder
|
||||
merged := []string{}
|
||||
for _, srcSlug := range sources {
|
||||
var title, content, updated string
|
||||
var tags []string
|
||||
err := pool.QueryRow(ctx, `
|
||||
SELECT ke.title, ke.content, COALESCE(ke.tags,'{}'), ke.updated_at::text
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE (e.slug = $1 OR e.id::text = $1) AND ke.deleted_at IS NULL`,
|
||||
srcSlug).Scan(&title, &content, &tags, &updated)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
appended.WriteString("\n\n---\n\n## Merged: ")
|
||||
appended.WriteString(title)
|
||||
appended.WriteString("\n\n*Originally ")
|
||||
appended.WriteString(srcSlug)
|
||||
appended.WriteString(", last updated ")
|
||||
appended.WriteString(updated)
|
||||
appended.WriteString("*\n\n")
|
||||
appended.WriteString(content)
|
||||
for _, t := range tags {
|
||||
fmt.Fprintf(&appended, "\ntag: %s", strings.ToLower(strings.TrimSpace(t)))
|
||||
}
|
||||
merged = append(merged, srcSlug)
|
||||
}
|
||||
|
||||
if len(merged) == 0 {
|
||||
return textResult("no source entries could be read"), nil
|
||||
}
|
||||
|
||||
_, err := pool.Exec(ctx, `
|
||||
UPDATE knowledge_entities SET content = content || $2, edited_by = 'nomos', updated_at = now()
|
||||
WHERE entity_id = $1`, targetID, appended.String())
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error appending content: %v", err)), nil
|
||||
}
|
||||
|
||||
for _, srcSlug := range merged {
|
||||
pool.Exec(ctx, `
|
||||
UPDATE knowledge_entities ke SET deleted_at = now(), edited_by = 'nomos'
|
||||
FROM entities e
|
||||
WHERE e.id = ke.entity_id AND (e.slug = $1 OR e.id::text = $1)`,
|
||||
srcSlug)
|
||||
}
|
||||
|
||||
return textResult(fmt.Sprintf("Merged %d entries into %s: %s", len(merged), targetSlug, strings.Join(merged, ", "))), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "rename_knowledge_tag", Description: "Bulk-rename one or more tags across all knowledge entries. Case-insensitive matching — 'oom' and 'OOM' are treated as the same tag. Deduplicates after rename.",
|
||||
InputSchema: objSchema(
|
||||
prop{"from", "string", "Comma-separated tag names to rename FROM"},
|
||||
prop{"to", "string", "New tag name"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
fromStr, _ := args["from"].(string)
|
||||
to, _ := args["to"].(string)
|
||||
to = strings.ToLower(strings.TrimSpace(to))
|
||||
|
||||
from := []string{}
|
||||
for _, f := range strings.Split(fromStr, ",") {
|
||||
if f = strings.TrimSpace(f); f != "" {
|
||||
from = append(from, strings.ToLower(f))
|
||||
}
|
||||
}
|
||||
if to == "" || len(from) == 0 {
|
||||
return textResult("from and to are required"), nil
|
||||
}
|
||||
|
||||
tag, err := pool.Exec(ctx, `
|
||||
UPDATE knowledge_entities ke
|
||||
SET tags = sub.new_tags, updated_at = now()
|
||||
FROM (
|
||||
SELECT k.entity_id,
|
||||
ARRAY(SELECT DISTINCT CASE WHEN lower(t) = ANY($1) THEN $2 ELSE t END
|
||||
FROM unnest(k.tags) AS t) AS new_tags
|
||||
FROM knowledge_entities k
|
||||
WHERE k.deleted_at IS NULL
|
||||
AND EXISTS (SELECT 1 FROM unnest(k.tags) AS t WHERE lower(t) = ANY($1))
|
||||
) AS sub
|
||||
WHERE ke.entity_id = sub.entity_id`, from, to)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Tag %s → %s: %d entries updated.", strings.Join(from, ", "), to, tag.RowsAffected())), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_knowledge_revisions", Description: "Version history for a knowledge entry. Returns title, content, editor, tags, and timestamps for each revision.",
|
||||
InputSchema: objSchema(
|
||||
prop{"knowledge_slug", "string", "Knowledge entity slug (e.g. document:nomos/something)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["knowledge_slug"].(string)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT kr.id, kr.title, kr.content, COALESCE(kr.edited_by, '') AS edited_by,
|
||||
COALESCE(kr.tags::text, '{}') AS tags,
|
||||
kr.version_at::text, kr.revised_at::text
|
||||
FROM knowledge_revisions kr
|
||||
JOIN entities e ON e.id = kr.entity_id
|
||||
WHERE e.slug = $1
|
||||
ORDER BY kr.version_at DESC LIMIT 50`, slug), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_knowledge_duplicates", Description: "Near-duplicate knowledge entries detected via trigram similarity. Returns clusters of similar documents with similarity scores. Use before creating new knowledge to avoid pileup.",
|
||||
InputSchema: objSchema(
|
||||
prop{"threshold", "number", "Similarity threshold 0-1 (default 0.6, lower = more matches)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
threshold := getFloat(args, "threshold", 0.6)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT a.slug AS doc_a, b.slug AS doc_b, similarity(ka.title, kb.title) AS sim
|
||||
FROM knowledge_entities ka
|
||||
JOIN knowledge_entities kb ON ka.entity_id < kb.entity_id
|
||||
JOIN entities a ON a.id = ka.entity_id
|
||||
JOIN entities b ON b.id = kb.entity_id
|
||||
WHERE ka.deleted_at IS NULL AND kb.deleted_at IS NULL
|
||||
AND similarity(ka.title, kb.title) > $1
|
||||
ORDER BY sim DESC LIMIT 100`, threshold), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_knowledge_orphans", Description: "Knowledge entries with no entity links (unlinked), no tags (untagged), or stale (not updated in N days). Helps identify abandoned or disconnected knowledge to clean up.",
|
||||
InputSchema: objSchema(
|
||||
prop{"stale_days", "integer", "Days without update to consider stale (default 90)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
staleDays := int(getFloat(args, "stale_days", 90))
|
||||
return queryRows(ctx, pool, fmt.Sprintf(`
|
||||
SELECT e.slug, ke.title, e.type AS kind, COALESCE(ke.edited_by, '') AS edited_by,
|
||||
ke.updated_at::text,
|
||||
(ke.tags IS NULL OR cardinality(ke.tags) = 0) AS untagged,
|
||||
NOT EXISTS (
|
||||
SELECT 1 FROM relationships r
|
||||
WHERE r.source_id = ke.entity_id AND r.valid_to IS NULL
|
||||
AND r.type IN ('documents', 'about')
|
||||
) AS unlinked,
|
||||
(ke.updated_at < now() - interval '%d days') AS stale
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE ke.deleted_at IS NULL
|
||||
ORDER BY ke.updated_at ASC`, staleDays)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "list_knowledge_tags", Description: "All tags used across the knowledge base with usage counts. Returns normalized tag, count, and any casing variants (e.g. 'oom' and 'OOM' surface as variants so you can spot drift).",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT lower(tag) AS tag, count(*) AS uses,
|
||||
array_agg(DISTINCT tag ORDER BY tag) AS variants
|
||||
FROM knowledge_entities ke, unnest(ke.tags) AS tag
|
||||
WHERE ke.deleted_at IS NULL
|
||||
GROUP BY lower(tag) ORDER BY uses DESC, lower(tag)`), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "list_my_secrets", Description: "List secrets accessible to this client by public key",
|
||||
InputSchema: objSchema(prop{"caller_pubkey", "string", "Age public key of the caller (optional)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
pubkey, _ := args["caller_pubkey"].(string)
|
||||
// Match entities where age_pubkey attribute contains the caller's key.
|
||||
query := `
|
||||
SELECT e.slug, e.type, e.name,
|
||||
e.attributes->>'age_pubkey' AS age_pubkey
|
||||
FROM entities e
|
||||
WHERE e.attributes->>'age_pubkey' IS NOT NULL`
|
||||
var dbArgs []any
|
||||
if pubkey != "" {
|
||||
query += ` AND e.attributes->>'age_pubkey' = $1`
|
||||
dbArgs = append(dbArgs, pubkey)
|
||||
}
|
||||
query += ` ORDER BY e.slug LIMIT 100`
|
||||
return queryRows(ctx, pool, query, dbArgs...), nil
|
||||
}},
|
||||
}
|
||||
}
|
||||
599
internal/mcp/ops_tools.go
Normal file
599
internal/mcp/ops_tools.go
Normal file
@@ -0,0 +1,599 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
func OpsTools(pool *db.Pool, agentID uuid.UUID, sec secretBackend) []toolReg {
|
||||
return []toolReg{
|
||||
// ── request_execution (legacy fixed enum) retired 2026-07-14 ──
|
||||
// All mutations now route through `run`. The handler functions
|
||||
// (runRexecRestart, runRexecSystemctl, etc.) are kept as reference
|
||||
// for future runbook extraction — especially pct_create DNS/VMID logic.
|
||||
// DO NOT re-register this tool. See plans/2026-07-10-general-gated-execution.md.
|
||||
|
||||
{tool: &mcp.Tool{Name: "run", Description: "Run ANY shell command against any host, LXC, or VM. This is the general execution primitive — prefer it over asking the operator to run something manually, and don't wait for a matching fixed action to exist. Every command is automatically risk-classified: read-only inspection (cat, systemctl status, docker ps, journalctl, df, git status, ...) runs immediately; anything that changes state requires operator approval (granted by the operator replying \"go ahead\"/\"yes\" in chat, or via the Approve button); commands matching a destructive pattern (rm -rf, dd, mkfs, pct/qm destroy, DROP TABLE, reboot, piping curl into a shell, ...) always require approval regardless of what you declare. You cannot talk your way past the destructive check by declaring a lower risk.\n\nHost-level mutations (apt-get install, dpkg, systemctl enable) always classify as config_mutation — operator approval required.\n\nVM targets: the QEMU guest agent must be running inside the VM. If the entity's qemu_guest_agent attribute is not_running, the run is blocked immediately with a clear error.",
|
||||
InputSchema: objSchema(
|
||||
prop{"target", "string", "Target entity slug: host:<slug> (e.g. host:strong), lxc:<slug> (e.g. lxc:caddy), or vm:<slug> (e.g. vm:zimaos). LXC commands run via pct exec on their Proxmox host automatically. VM commands run via qm guest exec on their Proxmox host (requires the QEMU guest agent inside the VM — standard for Proxmox VMs)."},
|
||||
prop{"command", "string", "The shell command to run. Can be a full script (multi-line, &&-chained). Runs as root."},
|
||||
prop{"purpose", "string", "One sentence: why you're running this. Shown to the operator alongside the approval — be specific, this is what they're approving."},
|
||||
prop{"declared_risk", "string", "Optional self-assessment: read_only, reversible_low, config_mutation, or destructive. This can only ESCALATE the automatic classification, never lower it — declaring a mutating command as read_only has no effect."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
targetSlug, _ := args["target"].(string)
|
||||
command, _ := args["command"].(string)
|
||||
purpose, _ := args["purpose"].(string)
|
||||
declaredRisk, _ := args["declared_risk"].(string)
|
||||
sessionID, _ := args["_session_id"].(string)
|
||||
if targetSlug == "" || command == "" {
|
||||
return textResult("error: target and command are required"), nil
|
||||
}
|
||||
|
||||
var targetID uuid.UUID
|
||||
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", targetSlug).Scan(&targetID); err != nil {
|
||||
return textResult(fmt.Sprintf("target not found: %s", targetSlug)), nil
|
||||
}
|
||||
|
||||
return classifyAndGate(ctx, pool, agentID, targetID, targetSlug, command, purpose, declaredRisk, sessionID), nil
|
||||
}},
|
||||
// inspect_path is the bulk fact-gathering tool from
|
||||
// plans/2026-07-18-session-review-three-sessions.md P1.5.
|
||||
// Sessions 1e9c7691 and 55927f0a each spent ~15 `run` calls
|
||||
// gathering identical facts (`mount | grep`, `df`, `ls -la`,
|
||||
// `stat`) across hosts and LXCs to understand where a path
|
||||
// lives, who mounts it, and what permissions it has. This tool
|
||||
// collapses that fan-out into one call: pass a path and a list
|
||||
// of targets, get back per-target mount/df/ls/stat output as
|
||||
// JSON. All commands are read-only, so no approval is needed.
|
||||
{tool: &mcp.Tool{Name: "inspect_path", Description: "Bulk fact-gathering: run mount/df/ls/stat for the same path across multiple host/LXC/VM targets in ONE call. Returns a JSON object keyed by target slug, each with the target's view of the path (mount source, filesystem, size, top-level entries with ownership/permissions). Use this instead of N separate `run` calls when you need to understand a path's footprint across the fleet (e.g. tracing where a volume is mounted, checking permissions on the same NFS path from server + client). All commands are read-only — no approval needed.",
|
||||
InputSchema: objSchema(
|
||||
prop{"path", "string", "Absolute path to inspect on each target (e.g. /mnt/media_local, /media/ludo-library)."},
|
||||
prop{"targets", "array", "List of target entity slugs (host:strong, lxc:nfs-export, vm:zimaos, …). Up to 8 per call."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
path, _ := args["path"].(string)
|
||||
if path == "" {
|
||||
return textResult("error: path is required"), nil
|
||||
}
|
||||
rawTargets, _ := args["targets"].([]any)
|
||||
if len(rawTargets) == 0 {
|
||||
return textResult("error: at least one target is required"), nil
|
||||
}
|
||||
if len(rawTargets) > 8 {
|
||||
return textResult("error: at most 8 targets per inspect_path call (use two calls if you need more)"), nil
|
||||
}
|
||||
targets := make([]string, 0, len(rawTargets))
|
||||
for _, t := range rawTargets {
|
||||
if s, ok := t.(string); ok && s != "" {
|
||||
targets = append(targets, s)
|
||||
}
|
||||
}
|
||||
results := inspectPathAcrossTargets(ctx, pool, path, targets)
|
||||
out, _ := json.MarshalIndent(results, "", " ")
|
||||
return textResult(string(out)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_execution_status", Description: "Check the status of a requested execution",
|
||||
InputSchema: objSchema(
|
||||
prop{"execution_id", "string", "Execution UUID (from request_execution output)"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
execID, _ := args["execution_id"].(string)
|
||||
if execID == "" {
|
||||
return textResult("execution_id required"), nil
|
||||
}
|
||||
eid, err := uuid.Parse(execID)
|
||||
if err != nil {
|
||||
// Try finding by exec slug prefix
|
||||
var found uuid.UUID
|
||||
err2 := pool.QueryRow(ctx, "SELECT entity_id FROM executions WHERE entity_id::text LIKE $1 LIMIT 1", execID+"%").Scan(&found)
|
||||
if err2 != nil {
|
||||
return textResult(fmt.Sprintf("execution not found: %s", execID)), nil
|
||||
}
|
||||
eid = found
|
||||
}
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.entity_id::text, e.action, e.risk_class, e.status,
|
||||
e.result::text, e.duration_ms, e.started_at::text,
|
||||
e.completed_at::text, e.correlation_id
|
||||
FROM executions e
|
||||
WHERE e.entity_id = $1`, eid), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "tail_log", Description: "Get recent log lines from a service via journalctl",
|
||||
InputSchema: objSchema(
|
||||
prop{"service_slug", "string", "Service entity slug (e.g. lxc:caddy)"},
|
||||
prop{"lines", "integer", "Number of lines (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
n := int(getFloat(args, "lines", 50))
|
||||
if slug == "" {
|
||||
return textResult("service_slug is required"), nil
|
||||
}
|
||||
host, user, err := resolveHost(ctx, pool, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
|
||||
}
|
||||
svc := strings.TrimPrefix(slug, "lxc:")
|
||||
out, err := sshExec(ctx, host, user, fmt.Sprintf("journalctl -u %s -n %d --no-pager 2>&1 || true", svc, n))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_service_status", Description: "Check systemd service status on a host",
|
||||
InputSchema: objSchema(
|
||||
prop{"service_slug", "string", "Service entity slug (e.g. lxc:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("service_slug is required"), nil
|
||||
}
|
||||
host, user, err := resolveHost(ctx, pool, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
|
||||
}
|
||||
svc := strings.TrimPrefix(slug, "lxc:")
|
||||
out, err := sshExec(ctx, host, user,
|
||||
fmt.Sprintf("systemctl is-active %s; systemctl is-enabled %s; systemctl show %s -p ActiveEnterTimestamp -p SubState 2>&1 || true", svc, svc, svc))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_lxc_state", Description: "Get LXC container resource state from Proxmox host",
|
||||
InputSchema: objSchema(
|
||||
prop{"lxc_slug", "string", "LXC entity slug (e.g. lxc:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["lxc_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("lxc_slug is required"), nil
|
||||
}
|
||||
var pveID string
|
||||
err := pool.QueryRow(ctx, "SELECT attributes->>'pve_id' FROM entities WHERE slug = $1", slug).Scan(&pveID)
|
||||
if err != nil || pveID == "" {
|
||||
return textResult(fmt.Sprintf("LXC not found or missing pve_id: %s", slug)), nil
|
||||
}
|
||||
// Resolve the Proxmox host — find the host that runs this LXC
|
||||
var hostID uuid.UUID
|
||||
err = pool.QueryRow(ctx, `
|
||||
SELECT t.id FROM entities t
|
||||
JOIN relationships r ON r.source_id = t.id
|
||||
JOIN entities s ON s.id = r.target_id
|
||||
WHERE s.slug = $1 AND r.type = 'hosts' AND r.valid_to IS NULL
|
||||
LIMIT 1`, slug).Scan(&hostID)
|
||||
if err != nil {
|
||||
// Fallback: use the inventory host attribute if no relationship
|
||||
var hostSlug string
|
||||
err = pool.QueryRow(ctx, "SELECT attributes->>'host' FROM entities WHERE slug = $1", slug).Scan(&hostSlug)
|
||||
if err != nil || hostSlug == "" {
|
||||
return textResult(fmt.Sprintf("cannot resolve Proxmox host for %s", slug)), nil
|
||||
}
|
||||
var host, user string
|
||||
host, user, err = resolveHost(ctx, pool, "host:"+hostSlug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve: %v", err)), nil
|
||||
}
|
||||
out, err2 := sshExec(ctx, host, user, fmt.Sprintf("pct status %s --verbose 2>&1 || true", pveID))
|
||||
if err2 != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err2)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}
|
||||
var hostSlug string
|
||||
pool.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", hostID).Scan(&hostSlug)
|
||||
host, user, err := resolveHost(ctx, pool, hostSlug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
|
||||
}
|
||||
out, err := sshExec(ctx, host, user, fmt.Sprintf("pct status %s --verbose 2>&1 || true", pveID))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "ping_service", Description: "Check if a service is reachable via HTTP — returns scheduler health state plus a live HTTP probe",
|
||||
InputSchema: objSchema(prop{"service_slug", "string", "Service entity slug"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("service_slug is required"), nil
|
||||
}
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT st.health, st.last_check_at,
|
||||
COALESCE(
|
||||
e.attributes->>'url',
|
||||
CASE WHEN e.attributes->>'public_host' IS NOT NULL
|
||||
THEN 'https://' || e.attributes->>'public_host'
|
||||
END
|
||||
) AS url
|
||||
FROM entity_status st
|
||||
JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.slug = $1`, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("query error: %v", err)), nil
|
||||
}
|
||||
defer rows.Close()
|
||||
if !rows.Next() {
|
||||
return textResult(fmt.Sprintf("service not found: %s", slug)), nil
|
||||
}
|
||||
var health, lastCheck, url string
|
||||
rows.Scan(&health, &lastCheck, &url)
|
||||
if url == "" {
|
||||
return textResult(fmt.Sprintf("health=%s last_check=%s url=no-url (entity has no url or public_host attribute)", health, lastCheck)), nil
|
||||
}
|
||||
// Live HTTP probe — HEAD request to check current state
|
||||
code := "n/a"
|
||||
if resp, err := http.Head(url); err == nil {
|
||||
resp.Body.Close()
|
||||
code = fmt.Sprintf("%d", resp.StatusCode)
|
||||
} else {
|
||||
code = fmt.Sprintf("err: %v", err)
|
||||
}
|
||||
return textResult(fmt.Sprintf("health=%s last_check=%s url=%s http=%s", health, lastCheck, url, code)), nil
|
||||
}},
|
||||
// ─── Phase 5: operational MCP tools ──────────────────────────────
|
||||
|
||||
{tool: &mcp.Tool{Name: "list_lxcs", Description: "List all LXC containers with ID, host, IP, state, and last-audited hint. Pass state=\"active\" to exclude destroyed/deprecated containers. The last_audited_at column shows the most recent knowledge entry (investigation or document tagged audit/update) linked via an 'about' edge — use it to skip re-running `run` against LXCs that were already audited recently.",
|
||||
InputSchema: objSchema(
|
||||
prop{"state", "string", "Optional: filter by entity state (active, destroyed, …)"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
state, _ := argsMap(req)["state"].(string)
|
||||
var statePtr *string
|
||||
if state != "" {
|
||||
statePtr = &state
|
||||
}
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.name, e.attributes->>'pve_id' AS pve_id,
|
||||
e.attributes->>'lan_ip' AS lan_ip,
|
||||
e.state,
|
||||
st.health, st.last_check_at,
|
||||
(SELECT MAX(k.created_at)
|
||||
FROM relationships r
|
||||
JOIN knowledge_entities k ON k.entity_id = r.source_id
|
||||
WHERE r.target_id = e.id
|
||||
AND r.type = 'about'
|
||||
AND r.valid_to IS NULL
|
||||
AND (k.tags @> ARRAY['audit']::text[]
|
||||
OR k.tags @> ARRAY['update']::text[]
|
||||
OR k.title ILIKE '%audit%'
|
||||
OR k.title ILIKE '%update%')
|
||||
) AS last_audited_at
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.type = 'lxc'
|
||||
AND ($1::text IS NULL OR e.state = $1)
|
||||
ORDER BY CASE WHEN e.state = 'active' THEN 0 ELSE 1 END,
|
||||
(e.attributes->>'pve_id')::int`, statePtr), "lxc_list"), nil
|
||||
}},
|
||||
// ── Stage 2: External agent observe ──────────────────────────
|
||||
|
||||
// ── Stage 4: External agent act (mutations) ─────────────────────
|
||||
|
||||
{tool: &mcp.Tool{Name: "ack_signal", Description: "Acknowledge an open signal. Use when investigating an alert — marks it as seen and being worked on.",
|
||||
InputSchema: objSchema(prop{"signal_id", "string", "Signal entity UUID"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
sid, _ := args["signal_id"].(string)
|
||||
id, err := uuid.Parse(sid)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("invalid signal_id: %v", err)), nil
|
||||
}
|
||||
tag, err := pool.Exec(ctx,
|
||||
`UPDATE signals SET state = 'acknowledged', updated_at = now()
|
||||
WHERE entity_id = $1 AND state IN ('raised','failed')`, id)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return textResult(fmt.Sprintf("signal %s not found or not in a state that can be acknowledged", sid)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Signal %s acknowledged.", sid)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "resolve_signal", Description: "Resolve a signal with an optional resolution note. Use when the underlying issue is fixed — marks the signal as resolved so it stops showing as active.",
|
||||
InputSchema: objSchema(
|
||||
prop{"signal_id", "string", "Signal entity UUID"},
|
||||
prop{"resolution", "string", "Optional note describing what fixed it"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
sid, _ := args["signal_id"].(string)
|
||||
id, err := uuid.Parse(sid)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("invalid signal_id: %v", err)), nil
|
||||
}
|
||||
tag, err := pool.Exec(ctx,
|
||||
`UPDATE signals SET state = 'resolved', updated_at = now()
|
||||
WHERE entity_id = $1 AND state IN ('raised','acknowledged','acting','failed')`, id)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return textResult(fmt.Sprintf("signal %s not found or not in a state that can be resolved", sid)), nil
|
||||
}
|
||||
resolution, _ := args["resolution"].(string)
|
||||
if resolution != "" {
|
||||
return textResult(fmt.Sprintf("Signal %s resolved: %s", sid, resolution)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Signal %s resolved.", sid)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "mute_signal", Description: "Temporarily mute a signal. Suppresses it from active views for the given duration. Use for known, non-urgent issues that don't need immediate attention.",
|
||||
InputSchema: objSchema(
|
||||
prop{"signal_id", "string", "Signal entity UUID"},
|
||||
prop{"duration_s", "integer", "Mute duration in seconds (default 3600 = 1 hour)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
sid, _ := args["signal_id"].(string)
|
||||
id, err := uuid.Parse(sid)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("invalid signal_id: %v", err)), nil
|
||||
}
|
||||
dur := int64(getFloat(args, "duration_s", 3600))
|
||||
muteUntil := time.Now().UTC().Add(time.Duration(dur) * time.Second)
|
||||
tag, err := pool.Exec(ctx,
|
||||
`UPDATE signals SET state = 'muted', mute_until = $2, updated_at = now()
|
||||
WHERE entity_id = $1 AND state IN ('raised','acknowledged')`, id, muteUntil)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return textResult(fmt.Sprintf("signal %s not found or not in a state that can be muted", sid)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Signal %s muted until %s.", sid, muteUntil.Format(time.RFC3339))), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "cancel_execution", Description: "Cancel a queued or running execution. Use when you realize the command was wrong, targets the wrong host, or should not proceed. Requires a reason.",
|
||||
InputSchema: objSchema(
|
||||
prop{"execution_id", "string", "Execution entity UUID"},
|
||||
prop{"reason", "string", "Why this execution should be cancelled"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
eid, _ := args["execution_id"].(string)
|
||||
id, err := uuid.Parse(eid)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("invalid execution_id: %v", err)), nil
|
||||
}
|
||||
reason, _ := args["reason"].(string)
|
||||
result := jsonErr("cancelled by agent: %s", reason)
|
||||
tag, err := pool.Exec(ctx,
|
||||
`UPDATE executions SET status = 'cancelled', result = $2::jsonb
|
||||
WHERE entity_id = $1 AND status IN ('running','pending_approval','approved','queued')`,
|
||||
id, result)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return textResult(fmt.Sprintf("execution %s not found or already final", eid)), nil
|
||||
}
|
||||
// Write audit entry.
|
||||
_ = observability.Audit(ctx, sqlcgen.New(pool), "agent", "nomos", "cancel",
|
||||
&id, "POST", "/mcp", "", nil,
|
||||
map[string]any{"reason": reason})
|
||||
return textResult(fmt.Sprintf("Execution %s cancelled: %s", eid, reason)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "update_check", Description: "Enable or disable a health check. Disable a noisy probe that's firing false positives; re-enable after fixing the underlying issue.",
|
||||
InputSchema: objSchema(
|
||||
prop{"check_id", "string", "Check entity UUID"},
|
||||
prop{"enabled", "boolean", "true to enable, false to disable"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
cid, _ := args["check_id"].(string)
|
||||
id, err := uuid.Parse(cid)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("invalid check_id: %v", err)), nil
|
||||
}
|
||||
enabled, _ := args["enabled"].(bool)
|
||||
tag, err := pool.Exec(ctx,
|
||||
`UPDATE check_defs SET enabled = $2 WHERE entity_id = $1`, id, enabled)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return textResult(fmt.Sprintf("check %s not found", cid)), nil
|
||||
}
|
||||
status := "enabled"
|
||||
if !enabled {
|
||||
status = "disabled"
|
||||
}
|
||||
return textResult(fmt.Sprintf("Check %s %s.", cid, status)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "list_checks", Description: "List health checks with verdict, last run time, probe kind, and config. Filter by entity slug or enabled status. Each check's last_health explains which probe is responsible for an entity's overall health.",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_slug", "string", "Filter by target entity slug"},
|
||||
prop{"enabled", "boolean", "Filter enabled/disabled (optional)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT cd.entity_id, e.slug, cd.kind,
|
||||
COALESCE(te.slug, '') AS target_slug, cd.target_type,
|
||||
cd.config::text, cd.interval_s, cd.timeout_s, cd.enabled,
|
||||
e.version, cd.last_health, cd.last_run_at::text
|
||||
FROM check_defs cd
|
||||
JOIN entities e ON e.id = cd.entity_id
|
||||
LEFT JOIN entities te ON te.id = cd.target_id
|
||||
WHERE ($1::text IS NULL OR te.slug = $1)
|
||||
AND ($2::bool IS NULL OR cd.enabled = $2)
|
||||
ORDER BY e.slug LIMIT 200`,
|
||||
nStr(args["entity_slug"]), args["enabled"]), "check_table"), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "list_executions", Description: "Cursor-paginated execution history. Filter by entity slug, status, or risk class. Returns newest-first with duration, result, and target info.",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_slug", "string", "Filter by target entity slug"},
|
||||
prop{"status", "string", "Filter by status (running/completed/failed/pending_approval)"},
|
||||
prop{"limit", "integer", "Max rows (default 25)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 25))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.entity_id, te.slug AS target, e.action, e.risk_class,
|
||||
e.status, e.result::text, e.duration_ms,
|
||||
e.correlation_id, e.started_at::text, e.completed_at::text, e.created_at::text,
|
||||
COALESCE(npe.session_id::text, '') AS session_id
|
||||
FROM executions e
|
||||
JOIN entities te ON te.id = e.target_entity_id
|
||||
LEFT JOIN nomos_plan_executions npe ON npe.execution_id = e.entity_id
|
||||
WHERE ($1::text IS NULL OR te.slug = $1)
|
||||
AND ($2::text IS NULL OR e.status = $2)
|
||||
ORDER BY e.created_at DESC LIMIT $3`,
|
||||
nStr(args["entity_slug"]), nStr(args["status"]), limit), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "list_entity_sessions", Description: "Active Nomos sessions (tasks) linked to an entity. Shows goal, status, outcome, and when the session was last active. Use to discover what agents are working on related to this entity.",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_slug", "string", "Entity slug to find sessions for"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_slug"].(string)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT DISTINCT as2.id, as2.title, as2.goal, as2.status, as2.outcome,
|
||||
as2.summary, as2.last_active_at::text, as2.closed_at::text
|
||||
FROM agent_sessions as2
|
||||
JOIN nomos_plan_executions npe ON npe.session_id = as2.id
|
||||
JOIN executions ex ON ex.entity_id = npe.execution_id
|
||||
JOIN entities te ON te.id = ex.target_entity_id
|
||||
WHERE te.slug = $1 AND as2.closed_at IS NULL
|
||||
ORDER BY as2.last_active_at DESC LIMIT 20`, slug), nil
|
||||
}},
|
||||
// ── Stage 2: External agent observe ──────────────────────────
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_dashboard_summary", Description: "Fleet overview in one call: entity counts by type and state, health breakdown (healthy/degraded/down/stale/unknown), active signals by severity, pending approval count, execution counts in last 24h, and event rate over last 6h.",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
result := map[string]any{}
|
||||
|
||||
// Entity counts by type
|
||||
result["entities_by_type"] = rowsToMap(ctx, pool,
|
||||
`SELECT type, count(*) FROM entities GROUP BY type`)
|
||||
|
||||
// Entity counts by state
|
||||
result["entities_by_state"] = rowsToMap(ctx, pool,
|
||||
`SELECT coalesce(state, 'unknown'), count(*) FROM entities GROUP BY state`)
|
||||
|
||||
// Health rollup (excluding check entities)
|
||||
result["health"] = rowsToMap(ctx, pool, `
|
||||
SELECT COALESCE(st.health, 'unknown') AS health, count(*)
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check' GROUP BY st.health`)
|
||||
|
||||
// Active signals by severity
|
||||
result["signals_by_severity"] = rowsToMap(ctx, pool, `
|
||||
SELECT severity, count(*) FROM signals
|
||||
WHERE state NOT IN ('resolved', 'failed') GROUP BY severity`)
|
||||
|
||||
// Pending approvals
|
||||
var pending int
|
||||
pool.QueryRow(ctx, `SELECT count(*) FROM approvals WHERE status = 'pending'`).Scan(&pending)
|
||||
result["approvals_pending"] = pending
|
||||
|
||||
// Executions in last 24h
|
||||
result["executions_by_state"] = rowsToMap(ctx, pool, `
|
||||
SELECT status, count(*) FROM executions
|
||||
WHERE created_at > now() - interval '24 hours' GROUP BY status`)
|
||||
|
||||
// Event rate (5-min buckets over 6h)
|
||||
events := []map[string]any{}
|
||||
erows, _ := pool.Query(ctx, `
|
||||
SELECT date_trunc('hour', ts) + (extract(minute FROM ts)::int / 5) * interval '5 minutes' AS bucket, count(*)
|
||||
FROM events WHERE ts > now() - interval '6 hours'
|
||||
GROUP BY bucket ORDER BY bucket`)
|
||||
if erows != nil {
|
||||
for erows.Next() {
|
||||
var bucket time.Time
|
||||
var n int
|
||||
if erows.Scan(&bucket, &n) == nil {
|
||||
events = append(events, map[string]any{"bucket": bucket, "count": n})
|
||||
}
|
||||
}
|
||||
erows.Close()
|
||||
}
|
||||
result["event_rate"] = events
|
||||
|
||||
b, _ := json.MarshalIndent(result, "", " ")
|
||||
return textResult(string(b)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "get_secret", Description: "Retrieve a secret value from the Infisical vault. Returns the secret value. Use for service credentials, tokens, and keys needed to operate the homelab.",
|
||||
InputSchema: objSchema(
|
||||
prop{"key", "string", "Secret key to retrieve (e.g. 'matrix-token', 'clients/host:hubris/age-key')"},
|
||||
prop{"path", "string", "Secret path prefix (default '/')"},
|
||||
prop{"environment", "string", "Environment slug (default 'dev')"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
if sec == nil {
|
||||
return textResult("error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)"), nil
|
||||
}
|
||||
args := argsMap(req)
|
||||
key, _ := args["key"].(string)
|
||||
if key == "" {
|
||||
return textResult("error: key is required"), nil
|
||||
}
|
||||
val, err := sec.Get(ctx, key)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
return textResult(val), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "list_secrets", Description: "List secret keys in the Infisical vault. Returns key names only (no values). Filter by path prefix to scope to a client or shared path.",
|
||||
InputSchema: objSchema(
|
||||
prop{"path_prefix", "string", "Filter to keys matching this prefix (e.g. 'clients/', 'shared/', 'config/')"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
if sec == nil {
|
||||
return textResult("error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)"), nil
|
||||
}
|
||||
args := argsMap(req)
|
||||
prefix, _ := args["path_prefix"].(string)
|
||||
|
||||
keys, err := sec.List(ctx)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
if prefix != "" {
|
||||
filtered := keys[:0]
|
||||
for _, k := range keys {
|
||||
if strings.HasPrefix(k, prefix) {
|
||||
filtered = append(filtered, k)
|
||||
}
|
||||
}
|
||||
keys = filtered
|
||||
}
|
||||
data, _ := json.MarshalIndent(keys, "", " ")
|
||||
return textResult(string(data)), nil
|
||||
}},
|
||||
{tool: &mcp.Tool{Name: "set_secret", Description: "Store or update a secret in the Infisical vault. Use when discovering new credentials that need to be persisted. Requires operator approval (config_mutation).",
|
||||
InputSchema: objSchema(
|
||||
prop{"key", "string", "Secret key to store"},
|
||||
prop{"value", "string", "Secret value to store"},
|
||||
prop{"path", "string", "Secret path prefix (default '/')"},
|
||||
prop{"environment", "string", "Environment slug (default 'dev')"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
if sec == nil {
|
||||
return textResult("error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)"), nil
|
||||
}
|
||||
args := argsMap(req)
|
||||
key, _ := args["key"].(string)
|
||||
value, _ := args["value"].(string)
|
||||
if key == "" {
|
||||
return textResult("error: key is required"), nil
|
||||
}
|
||||
if value == "" {
|
||||
return textResult("error: value is required"), nil
|
||||
}
|
||||
if err := sec.Set(ctx, key, value); err != nil {
|
||||
return textResult(fmt.Sprintf("error: %v", err)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("secret %s stored", key)), nil
|
||||
}},
|
||||
}
|
||||
}
|
||||
198
internal/mcp/secrets_tools_test.go
Normal file
198
internal/mcp/secrets_tools_test.go
Normal file
@@ -0,0 +1,198 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/secrets"
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
type mockSecretBackend struct {
|
||||
data map[string]string
|
||||
}
|
||||
|
||||
func (m *mockSecretBackend) Get(ctx context.Context, key string) (string, error) {
|
||||
v, ok := m.data[key]
|
||||
if !ok {
|
||||
return "", secrets.ErrNotFound
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
|
||||
func (m *mockSecretBackend) Set(ctx context.Context, key string, value string) error {
|
||||
m.data[key] = value
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *mockSecretBackend) List(ctx context.Context) ([]string, error) {
|
||||
keys := make([]string, 0, len(m.data))
|
||||
for k := range m.data {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
return keys, nil
|
||||
}
|
||||
|
||||
func (m *mockSecretBackend) Name() string { return "mock" }
|
||||
|
||||
// findToolHandler locates a tool's handler from allTools by name.
|
||||
func findToolHandler(t *testing.T, pool interface{}, name string, sec secrets.Backend) func(context.Context, *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
t.Helper()
|
||||
for _, r := range allTools(nil, uuid.Nil, sec) {
|
||||
if r.tool.Name == name {
|
||||
return r.handler
|
||||
}
|
||||
}
|
||||
t.Fatalf("tool %q not found", name)
|
||||
return nil
|
||||
}
|
||||
|
||||
func callToolJSON(t *testing.T, name string, sec secrets.Backend, args map[string]any) any {
|
||||
t.Helper()
|
||||
handler := findToolHandler(t, nil, name, sec)
|
||||
argBytes, _ := json.Marshal(args)
|
||||
req := &mcp.CallToolRequest{
|
||||
Params: &mcp.CallToolParamsRaw{Arguments: argBytes},
|
||||
}
|
||||
result, err := handler(context.Background(), req)
|
||||
if err != nil {
|
||||
t.Fatalf("tool %q error: %v", name, err)
|
||||
}
|
||||
if len(result.Content) == 0 {
|
||||
t.Fatalf("tool %q returned no content", name)
|
||||
}
|
||||
tc := result.Content[0].(*mcp.TextContent)
|
||||
var out any
|
||||
if err := json.Unmarshal([]byte(tc.Text), &out); err != nil {
|
||||
// Not JSON — return raw string
|
||||
return tc.Text
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func callToolText(t *testing.T, name string, sec secrets.Backend, args map[string]any) string {
|
||||
t.Helper()
|
||||
handler := findToolHandler(t, nil, name, sec)
|
||||
argBytes, _ := json.Marshal(args)
|
||||
req := &mcp.CallToolRequest{
|
||||
Params: &mcp.CallToolParamsRaw{Arguments: argBytes},
|
||||
}
|
||||
result, err := handler(context.Background(), req)
|
||||
if err != nil {
|
||||
t.Fatalf("tool %q error: %v", name, err)
|
||||
}
|
||||
if len(result.Content) == 0 {
|
||||
t.Fatalf("tool %q returned no content", name)
|
||||
}
|
||||
return result.Content[0].(*mcp.TextContent).Text
|
||||
}
|
||||
|
||||
func TestGetSecret(t *testing.T) {
|
||||
sec := &mockSecretBackend{
|
||||
data: map[string]string{
|
||||
"matrix-token": "bot-token-123",
|
||||
"clients/host:hubris/age-key": "AGE-SECRET-KEY",
|
||||
},
|
||||
}
|
||||
|
||||
// Get existing key
|
||||
val := callToolText(t, "get_secret", sec, map[string]any{"key": "matrix-token"})
|
||||
if val != "bot-token-123" {
|
||||
t.Errorf("get_secret = %q, want bot-token-123", val)
|
||||
}
|
||||
|
||||
// Get missing key
|
||||
errText := callToolText(t, "get_secret", sec, map[string]any{"key": "nonexistent"})
|
||||
if errText == "" {
|
||||
t.Error("expected error for missing key")
|
||||
}
|
||||
|
||||
// Missing key arg
|
||||
errText = callToolText(t, "get_secret", sec, map[string]any{})
|
||||
if errText != "error: key is required" {
|
||||
t.Errorf("missing key error = %q, want error: key is required", errText)
|
||||
}
|
||||
}
|
||||
|
||||
func TestListSecrets(t *testing.T) {
|
||||
sec := &mockSecretBackend{
|
||||
data: map[string]string{
|
||||
"clients/host:hubris/age-key": "val1",
|
||||
"clients/host:strong/age-key": "val2",
|
||||
"shared/matrix-token": "val3",
|
||||
},
|
||||
}
|
||||
|
||||
// List all
|
||||
out := callToolJSON(t, "list_secrets", sec, map[string]any{})
|
||||
keys, ok := out.([]any)
|
||||
if !ok {
|
||||
t.Fatalf("list_secrets returned non-array: %T", out)
|
||||
}
|
||||
if len(keys) != 3 {
|
||||
t.Errorf("list_secrets count = %d, want 3", len(keys))
|
||||
}
|
||||
|
||||
// List with prefix filter
|
||||
out = callToolJSON(t, "list_secrets", sec, map[string]any{"path_prefix": "clients/"})
|
||||
keys, ok = out.([]any)
|
||||
if !ok {
|
||||
t.Fatalf("filtered list returned non-array: %T", out)
|
||||
}
|
||||
if len(keys) != 2 {
|
||||
t.Errorf("filtered list count = %d, want 2", len(keys))
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetSecret(t *testing.T) {
|
||||
sec := &mockSecretBackend{
|
||||
data: map[string]string{},
|
||||
}
|
||||
|
||||
// Set a key
|
||||
result := callToolText(t, "set_secret", sec, map[string]any{"key": "test-key", "value": "test-value"})
|
||||
if result != "secret test-key stored" {
|
||||
t.Errorf("set_secret = %q, want 'secret test-key stored'", result)
|
||||
}
|
||||
|
||||
// Verify it was stored
|
||||
val, err := sec.Get(context.Background(), "test-key")
|
||||
if err != nil {
|
||||
t.Fatalf("verify get: %v", err)
|
||||
}
|
||||
if val != "test-value" {
|
||||
t.Errorf("stored value = %q, want test-value", val)
|
||||
}
|
||||
|
||||
// Missing key
|
||||
errText := callToolText(t, "set_secret", sec, map[string]any{})
|
||||
if errText != "error: key is required" {
|
||||
t.Errorf("missing key error = %q", errText)
|
||||
}
|
||||
|
||||
// Missing value
|
||||
errText = callToolText(t, "set_secret", sec, map[string]any{"key": "x"})
|
||||
if errText != "error: value is required" {
|
||||
t.Errorf("missing value error = %q", errText)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSecretToolsNilBackend(t *testing.T) {
|
||||
// All tools should return a graceful error when no backend is configured
|
||||
errText := callToolText(t, "get_secret", nil, map[string]any{"key": "x"})
|
||||
if errText != "error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)" {
|
||||
t.Errorf("nil backend get_secret = %q", errText)
|
||||
}
|
||||
|
||||
errText = callToolText(t, "list_secrets", nil, map[string]any{})
|
||||
if errText != "error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)" {
|
||||
t.Errorf("nil backend list_secrets = %q", errText)
|
||||
}
|
||||
|
||||
errText = callToolText(t, "set_secret", nil, map[string]any{"key": "x", "value": "y"})
|
||||
if errText != "error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)" {
|
||||
t.Errorf("nil backend set_secret = %q", errText)
|
||||
}
|
||||
}
|
||||
@@ -19,6 +19,7 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/actuator"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/execlog"
|
||||
@@ -47,10 +48,19 @@ func objSchema(props ...prop) *jsonschema.Schema {
|
||||
return s
|
||||
}
|
||||
|
||||
// secretBackend is the interface MCP tools use to access the secrets store.
|
||||
// Defined here to avoid importing the full secrets package (which brings in
|
||||
// the Infisical SDK). Mirrors the subset of secrets.Backend used by tools.
|
||||
type secretBackend interface {
|
||||
Get(ctx context.Context, key string) (string, error)
|
||||
Set(ctx context.Context, key string, value string) error
|
||||
List(ctx context.Context) ([]string, error)
|
||||
}
|
||||
|
||||
// NewHandler creates an http.Handler that serves the Oikos MCP server.
|
||||
// agentID is the Nomos agent entity UUID; tool calls are logged to agent_activity.
|
||||
func NewHandler(pool *db.Pool, token string, agentID uuid.UUID) http.Handler {
|
||||
s := newServer(pool, agentID)
|
||||
func NewHandler(pool *db.Pool, token string, agentID uuid.UUID, sec secretBackend) http.Handler {
|
||||
s := newServer(pool, agentID, sec)
|
||||
handler := mcp.NewStreamableHTTPHandler(func(r *http.Request) *mcp.Server {
|
||||
if token != "" {
|
||||
if r.Header.Get("Authorization") != "Bearer "+token {
|
||||
@@ -65,16 +75,128 @@ func NewHandler(pool *db.Pool, token string, agentID uuid.UUID) http.Handler {
|
||||
// toolHandler is the function signature registered via AddTool.
|
||||
type toolHandler = mcp.ToolHandler
|
||||
|
||||
func newServer(pool *db.Pool, agentID uuid.UUID) *mcp.Server {
|
||||
func newServer(pool *db.Pool, agentID uuid.UUID, sec secretBackend) *mcp.Server {
|
||||
s := mcp.NewServer(&mcp.Implementation{Name: "oikos", Version: "dev"}, &mcp.ServerOptions{
|
||||
Logger: slog.Default(),
|
||||
})
|
||||
for _, t := range allTools(pool, agentID) {
|
||||
for _, t := range allTools(pool, agentID, sec) {
|
||||
s.AddTool(t.tool, withActivityLogging(pool, agentID, t.tool.Name, t.handler))
|
||||
}
|
||||
|
||||
// Resource templates: let MCP clients browse and attach entities,
|
||||
// knowledge entries, and executions as conversation resources.
|
||||
s.AddResourceTemplate(&mcp.ResourceTemplate{
|
||||
URITemplate: "oikos://entity/{slug}",
|
||||
Name: "Entity",
|
||||
Description: "Oikos entity by slug (e.g. host:hubris, lxc:jellyfin)",
|
||||
MIMEType: "application/json",
|
||||
}, resourceHandler(pool, func(ctx context.Context, matches map[string]string) (string, error) {
|
||||
slug := matches["slug"]
|
||||
var id uuid.UUID
|
||||
if u, err := uuid.Parse(slug); err == nil {
|
||||
id = u
|
||||
} else {
|
||||
pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", slug).Scan(&id)
|
||||
}
|
||||
if id == uuid.Nil {
|
||||
return "", fmt.Errorf("entity not found: %s", slug)
|
||||
}
|
||||
result := queryEntity(ctx, pool, slug)
|
||||
return result.Content[0].(*mcp.TextContent).Text, nil
|
||||
}))
|
||||
|
||||
s.AddResourceTemplate(&mcp.ResourceTemplate{
|
||||
URITemplate: "oikos://knowledge/{id}",
|
||||
Name: "Knowledge",
|
||||
Description: "Knowledge entry by entity slug or UUID",
|
||||
MIMEType: "application/json",
|
||||
}, resourceHandler(pool, func(ctx context.Context, matches map[string]string) (string, error) {
|
||||
idOrSlug := matches["id"]
|
||||
var entityID uuid.UUID
|
||||
if u, err := uuid.Parse(idOrSlug); err == nil {
|
||||
entityID = u
|
||||
} else {
|
||||
pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", idOrSlug).Scan(&entityID)
|
||||
}
|
||||
if entityID == uuid.Nil {
|
||||
return "", fmt.Errorf("knowledge not found: %s", idOrSlug)
|
||||
}
|
||||
result := queryRows(ctx, pool, `
|
||||
SELECT ke.title, ke.content, ke.tags::text, e.slug, e.type AS kind,
|
||||
ke.updated_at::text
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE ke.entity_id = $1`, entityID)
|
||||
return result.Content[0].(*mcp.TextContent).Text, nil
|
||||
}))
|
||||
|
||||
s.AddResourceTemplate(&mcp.ResourceTemplate{
|
||||
URITemplate: "oikos://execution/{id}",
|
||||
Name: "Execution",
|
||||
Description: "Execution by UUID (returns status, result, timing)",
|
||||
MIMEType: "application/json",
|
||||
}, resourceHandler(pool, func(ctx context.Context, matches map[string]string) (string, error) {
|
||||
result := queryRows(ctx, pool, `
|
||||
SELECT e.entity_id, te.slug AS target, e.action, e.risk_class,
|
||||
e.status, e.result::text, e.duration_ms,
|
||||
e.started_at::text, e.completed_at::text
|
||||
FROM executions e
|
||||
JOIN entities te ON te.id = e.target_entity_id
|
||||
WHERE e.entity_id = $1`, matches["id"])
|
||||
return result.Content[0].(*mcp.TextContent).Text, nil
|
||||
}))
|
||||
|
||||
return s
|
||||
}
|
||||
|
||||
// resourceHandler adapts a simple func(ctx, params) → (string, error) into
|
||||
// an MCP ResourceHandler, reading the URI matched by a ResourceTemplate.
|
||||
func resourceHandler(pool *db.Pool, fn func(ctx context.Context, matches map[string]string) (string, error)) mcp.ResourceHandler {
|
||||
return func(ctx context.Context, req *mcp.ReadResourceRequest) (*mcp.ReadResourceResult, error) {
|
||||
uri := req.Params.URI
|
||||
matches := matchURITemplate(uri)
|
||||
if matches == nil {
|
||||
return nil, mcp.ResourceNotFoundError(uri)
|
||||
}
|
||||
|
||||
text, err := fn(ctx, matches)
|
||||
if err != nil {
|
||||
return nil, mcp.ResourceNotFoundError(uri)
|
||||
}
|
||||
|
||||
result, err := json.MarshalIndent(json.RawMessage(text), "", " ")
|
||||
if err != nil {
|
||||
result = []byte(text)
|
||||
}
|
||||
|
||||
return &mcp.ReadResourceResult{
|
||||
Contents: []*mcp.ResourceContents{{
|
||||
URI: uri,
|
||||
MIMEType: "application/json",
|
||||
Text: string(result),
|
||||
}},
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
|
||||
// matchURITemplate extracts parameters from a URI that matches one of the
|
||||
// oikos:// resource templates. Returns nil if the URI doesn't match.
|
||||
func matchURITemplate(uri string) map[string]string {
|
||||
// oikos://entity/{slug}
|
||||
if rest, ok := strings.CutPrefix(uri, "oikos://entity/"); ok && rest != "" {
|
||||
return map[string]string{"slug": rest}
|
||||
}
|
||||
// oikos://knowledge/{id}
|
||||
if rest, ok := strings.CutPrefix(uri, "oikos://knowledge/"); ok && rest != "" {
|
||||
return map[string]string{"id": rest}
|
||||
}
|
||||
// oikos://execution/{id}
|
||||
if rest, ok := strings.CutPrefix(uri, "oikos://execution/"); ok && rest != "" {
|
||||
return map[string]string{"id": rest}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// withActivityLogging wraps a tool handler to record agent_activity rows.
|
||||
func withActivityLogging(pool *db.Pool, agentID uuid.UUID, toolName string, next mcp.ToolHandler) mcp.ToolHandler {
|
||||
if agentID == uuid.Nil {
|
||||
@@ -330,30 +452,6 @@ func initSSH() {
|
||||
// goroutine forever with no way for the caller to ever get an answer.
|
||||
const sshExecTimeout = 10 * time.Minute
|
||||
|
||||
// streamWriter buffers everything it is given while forwarding each write to a
|
||||
// sink. Assigning one to session.Stdout and another (sharing the same buffer)
|
||||
// to session.Stderr reproduces CombinedOutput's interleaving exactly, in the
|
||||
// order the remote end actually produced it — which reading from StdoutPipe
|
||||
// and StderrPipe separately would not guarantee.
|
||||
type streamWriter struct {
|
||||
mu *sync.Mutex
|
||||
buf *bytes.Buffer
|
||||
stream string
|
||||
sink execlog.Sink
|
||||
}
|
||||
|
||||
func (w *streamWriter) Write(p []byte) (int, error) {
|
||||
w.mu.Lock()
|
||||
w.buf.Write(p)
|
||||
w.mu.Unlock()
|
||||
if w.sink != nil {
|
||||
// Copy: the ssh library reuses p after Write returns, and the sink
|
||||
// hands the bytes to a DB call that may outlive this frame.
|
||||
w.sink(w.stream, append([]byte(nil), p...))
|
||||
}
|
||||
return len(p), nil
|
||||
}
|
||||
|
||||
func sshExec(ctx context.Context, host, user, command string) (string, error) {
|
||||
return sshExecStream(ctx, host, user, command, nil)
|
||||
}
|
||||
@@ -369,96 +467,18 @@ func sshExecStream(ctx context.Context, host, user, command string, sink execlog
|
||||
user = sshUser
|
||||
}
|
||||
|
||||
addr := host + ":22"
|
||||
signer, err := ssh.ParsePrivateKey(sshKey)
|
||||
signer, err := actuator.LoadSignerFromBytes(sshKey)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("parse key: %w", err)
|
||||
}
|
||||
|
||||
cfg := &ssh.ClientConfig{
|
||||
User: user,
|
||||
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
|
||||
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
|
||||
Timeout: 10 * time.Second,
|
||||
}
|
||||
|
||||
client, err := ssh.Dial("tcp", addr, cfg)
|
||||
client, err := actuator.Dial(ctx, actuator.DialOptions{Host: host, User: user, Signer: signer})
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("dial %s: %w", host, err)
|
||||
return "", err
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
session, err := client.NewSession()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("session: %w", err)
|
||||
}
|
||||
defer session.Close()
|
||||
|
||||
var (
|
||||
mu sync.Mutex
|
||||
buf bytes.Buffer
|
||||
)
|
||||
session.Stdout = &streamWriter{mu: &mu, buf: &buf, stream: "stdout", sink: sink}
|
||||
session.Stderr = &streamWriter{mu: &mu, buf: &buf, stream: "stderr", sink: sink}
|
||||
|
||||
// collected returns whatever output has arrived so far. Callable while the
|
||||
// command is still running, which is what makes partial output on timeout
|
||||
// possible.
|
||||
collected := func() string {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return strings.TrimSpace(buf.String())
|
||||
}
|
||||
|
||||
done := make(chan error, 1)
|
||||
go func() {
|
||||
// Recovers a panic in the SSH library internals (rare but not
|
||||
// impossible) and reports it as a failed command instead of crashing
|
||||
// the whole api process — every gated action runs through this
|
||||
// function, so an unrecovered panic here would take down every
|
||||
// concurrently-running task's execution, not just this one. Without
|
||||
// this, a panic would ALSO silently degrade to "wait out the full
|
||||
// timeout" (done never receives, the select below falls through to
|
||||
// its time.After case) rather than crashing outright — recovering
|
||||
// and sending an immediate result is strictly better: the caller
|
||||
// finds out now, not after sshExecTimeout.
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
done <- fmt.Errorf("panic in ssh exec: %v", r)
|
||||
}
|
||||
}()
|
||||
// Run rather than CombinedOutput so the assigned writers are used;
|
||||
// Run returns only after both streams have been fully drained.
|
||||
done <- session.Run(command)
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-done:
|
||||
text := collected()
|
||||
// A non-zero exit MUST surface as an error — matching the fix
|
||||
// applied to httpapi's sshExec (this copy still had the original
|
||||
// bug: only erroring when there was no output at all, so a command
|
||||
// that failed but printed something was silently reported as
|
||||
// success).
|
||||
if err != nil {
|
||||
if text != "" {
|
||||
return text, fmt.Errorf("%w: %s", err, text)
|
||||
}
|
||||
return text, fmt.Errorf("exec: %w", err)
|
||||
}
|
||||
return text, nil
|
||||
case <-time.After(sshExecTimeout):
|
||||
session.Close()
|
||||
client.Close()
|
||||
// Return what the command managed to print before it hung. This used
|
||||
// to return "", discarding everything — so a hung command, the case
|
||||
// where the output matters most, was the one case that left no trace.
|
||||
return collected(), fmt.Errorf("timed out after %s waiting for command to finish on %s", sshExecTimeout, host)
|
||||
case <-ctx.Done():
|
||||
session.Close()
|
||||
client.Close()
|
||||
return collected(), ctx.Err()
|
||||
}
|
||||
return actuator.RunStreaming(ctx, client, command, sink, sshExecTimeout)
|
||||
}
|
||||
|
||||
// resolveHost resolves a host:<slug> to its reachable IP and SSH user. A thin
|
||||
@@ -628,8 +648,106 @@ func autoRun(ctx context.Context, pool *db.Pool, id uuid.UUID, targetSlug, comma
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// autoRunAsync starts a command in a goroutine, marking it running and returning
|
||||
// immediately. The caller gets an execution_id to poll with get_execution_status.
|
||||
// Used for commands containing sleep/wait/poll loops that would exceed the MCP
|
||||
// client timeout (120s) — the execution continues server-side.
|
||||
func autoRunAsync(ctx context.Context, pool *db.Pool, id uuid.UUID, targetSlug, command string) {
|
||||
startedAt := time.Now()
|
||||
if _, err := pool.Exec(ctx,
|
||||
`UPDATE executions SET status='running', started_at=$2 WHERE entity_id=$1`,
|
||||
id, startedAt); err != nil {
|
||||
slog.Error("mcp: mark execution running (async)", "error", err, "execution_id", id)
|
||||
}
|
||||
|
||||
host, user, wrap, err := resolveExecTarget(ctx, pool, targetSlug)
|
||||
if err != nil {
|
||||
pool.Exec(ctx,
|
||||
`UPDATE executions SET status='failed', result=$2::jsonb, duration_ms=$3, completed_at=now() WHERE entity_id=$1`,
|
||||
id, jsonErr("%s", err.Error()), int(time.Since(startedAt).Milliseconds()))
|
||||
slog.Error("mcp: async run resolve target", "error", err, "execution_id", id, "target", targetSlug)
|
||||
return
|
||||
}
|
||||
|
||||
var correlationID string
|
||||
if qerr := pool.QueryRow(ctx,
|
||||
`SELECT correlation_id FROM executions WHERE entity_id = $1`, id).Scan(&correlationID); qerr != nil {
|
||||
correlationID = ""
|
||||
}
|
||||
|
||||
go func() {
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
slog.Error("mcp: async run panic", "panic", r, "execution_id", id)
|
||||
pool.Exec(context.Background(),
|
||||
`UPDATE executions SET status='failed', result=$2::jsonb, duration_ms=$3, completed_at=now() WHERE entity_id=$1`,
|
||||
id, jsonErr("panic: %v", r), int(time.Since(startedAt).Milliseconds()))
|
||||
}
|
||||
}()
|
||||
|
||||
sink, flush := execlog.New(context.Background(), pool, id, correlationID)
|
||||
out, execErr := sshExecStream(context.Background(), host, user, wrap(command), sink)
|
||||
flush()
|
||||
if execErr != nil {
|
||||
pool.Exec(context.Background(),
|
||||
`UPDATE executions SET status='failed', result=$2::jsonb, duration_ms=$3, completed_at=now() WHERE entity_id=$1`,
|
||||
id, jsonErr("%s: %s", execErr.Error(), out), int(time.Since(startedAt).Milliseconds()))
|
||||
slog.Error("mcp: async run failed", "error", execErr, "execution_id", id, "output", out)
|
||||
} else {
|
||||
pool.Exec(context.Background(),
|
||||
`UPDATE executions SET status='completed', result=$2::jsonb, duration_ms=$3, completed_at=now() WHERE entity_id=$1`,
|
||||
id, jsonOut(out), int(time.Since(startedAt).Milliseconds()))
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// isLongRunningCommand detects shell commands containing sleep, wait, or poll
|
||||
// loops that indicate the command will exceed the MCP client timeout (120s).
|
||||
// These commands should use autoRunAsync to avoid the client timing out while
|
||||
// the command continues server-side.
|
||||
func isLongRunningCommand(cmd string) bool {
|
||||
cmd = strings.TrimSpace(cmd)
|
||||
|
||||
// sleep with duration — `sleep 30`, `sleep 1m`, etc.
|
||||
if sleepRe.MatchString(cmd) {
|
||||
return true
|
||||
}
|
||||
|
||||
// while/shell poll loops with sleep: `while ...; do ... sleep; done`
|
||||
if pollRe.MatchString(cmd) {
|
||||
return true
|
||||
}
|
||||
|
||||
// standalone wait command
|
||||
if waitRe.MatchString(cmd) {
|
||||
return true
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
var (
|
||||
sleepRe = regexp.MustCompile(`\bsleep\s+\d`)
|
||||
pollRe = regexp.MustCompile(`\bwhile\b.*\bsleep\b`)
|
||||
waitRe = regexp.MustCompile(`\bwait\s+\d|[&;]\s*wait\b`)
|
||||
)
|
||||
|
||||
func classifyAndGate(ctx context.Context, pool *db.Pool, agentID, targetID uuid.UUID, targetSlug, command, purpose, declaredRisk, sessionID string) *mcp.CallToolResult {
|
||||
riskClass := policy.ClassifyCommand(command, declaredRisk)
|
||||
|
||||
// Transport-aware escalation: read-only commands on LXC targets that
|
||||
// touch config paths (/opt/, /etc/) escalate to config_mutation.
|
||||
// The classifier only scores the command text, not the transport layer
|
||||
// — SSH-ing into a container to read /opt/ is riskier than running
|
||||
// the same command locally on the Proxmox host via pct exec.
|
||||
// Caught live: "cat /etc/hostname" on lxc:dns queued as config_mutation
|
||||
// while "pct exec 107 -- cat /etc/hostname" on host:hubris auto-ran.
|
||||
if riskClass == policy.RiskReadOnly && strings.HasPrefix(targetSlug, "lxc:") {
|
||||
if strings.Contains(command, "/opt/") || strings.Contains(command, "/etc/") || strings.Contains(command, "/var/lib/") {
|
||||
riskClass = policy.RiskConfigMutation
|
||||
}
|
||||
}
|
||||
|
||||
runParams, _ := json.Marshal(map[string]string{"command": command, "purpose": purpose})
|
||||
actionCol := "run:" + string(runParams)
|
||||
|
||||
@@ -647,6 +765,57 @@ func classifyAndGate(ctx context.Context, pool *db.Pool, agentID, targetID uuid.
|
||||
return textResult("No plan for this session. Call set_goal then propose_plan before run — even read-only tasks require a one-step plan. A one-step plan (\"Inspect X, report, write back\") is fine for trivial questions; the gate is about ordering, not approval. Read-only commands still auto-execute once a plan exists.")
|
||||
}
|
||||
|
||||
// Target validation: host-only commands (qm, pct, pvesh, iptables) must
|
||||
// not be dispatched against lxc:/vm: targets — those aren't Proxmox hosts
|
||||
// and don't have these tools. Caught live 2026-08-04: the agent ran
|
||||
// `qm stop 100` against lxc:dns, wasting a turn.
|
||||
|
||||
// Command syntax validation: catch LLM-generated bash bugs before they
|
||||
// hit the shell. The model sometimes inserts literal \n between commands
|
||||
// or puts spaces inside flags — these always fail, so reject early.
|
||||
if syntaxErr := validateCommandSyntax(command); syntaxErr != "" {
|
||||
return textResult(syntaxErr)
|
||||
}
|
||||
|
||||
if cmdPrefix, hostOnly := hostOnlyCommand(command); hostOnly && !strings.HasPrefix(targetSlug, "host:") {
|
||||
hostSuggestion := resolveProxmoxHostSlug(ctx, pool, targetSlug, "")
|
||||
if hostSuggestion == "" {
|
||||
hostSuggestion = "host:hubris or host:strong"
|
||||
}
|
||||
return textResult(fmt.Sprintf("Cannot run %q on %s — %s is a Proxmox host command. Use target %s instead.",
|
||||
cmdPrefix, targetSlug, cmdPrefix, hostSuggestion))
|
||||
}
|
||||
|
||||
// systemctl and docker work on hosts and LXCs, but not VMs.
|
||||
if cmdPrefix, hostLxc := hostLxcCommand(command); hostLxc {
|
||||
if !strings.HasPrefix(targetSlug, "host:") && !strings.HasPrefix(targetSlug, "lxc:") {
|
||||
return textResult(fmt.Sprintf("Cannot run %q on %s — %s only works on host:* or lxc:* targets.",
|
||||
cmdPrefix, targetSlug, cmdPrefix))
|
||||
}
|
||||
}
|
||||
|
||||
// VM transport pre-flight: qm guest exec requires the QEMU guest agent
|
||||
// to be running inside the VM. If it's not, the execution would queue
|
||||
// for approval and never execute — the agent has no way to learn it's
|
||||
// stuck (spotted live 2026-08-05: vm:zimaos had qemu_guest_agent=not_running,
|
||||
// the run queued forever, and the agent fell back to unsafe raw SSH).
|
||||
if strings.HasPrefix(targetSlug, "vm:") {
|
||||
var rawAttrs []byte
|
||||
if err := pool.QueryRow(ctx, `SELECT attributes FROM entities WHERE id = $1`, targetID).Scan(&rawAttrs); err == nil {
|
||||
var attrs map[string]any
|
||||
if json.Unmarshal(rawAttrs, &attrs) == nil {
|
||||
if qga, ok := attrs["qemu_guest_agent"]; ok {
|
||||
qgaStr, _ := qga.(string)
|
||||
if qgaStr == "not_running" || qgaStr == "" {
|
||||
return textResult(fmt.Sprintf(
|
||||
"run on %s blocked: QEMU guest agent is not running (%s). qm guest exec cannot reach this VM. Start the agent inside the guest first (e.g. via SSH/systemctl start qemu-guest-agent), then re-run. If the agent is running but the entity attribute is stale, update it with update_entity_attributes(slug=%s, attributes={\"qemu_guest_agent\":\"running\"}).",
|
||||
targetSlug, qgaStr, targetSlug))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Dedup: an identical pending command (same target, command, and
|
||||
// purpose) blocks a re-request — stops a tool-calling loop from queuing
|
||||
// the same approval repeatedly.
|
||||
@@ -721,8 +890,58 @@ func classifyAndGate(ctx context.Context, pool *db.Pool, agentID, targetID uuid.
|
||||
SELECT 1 FROM relationships
|
||||
WHERE source_id = t.id AND target_id = $1 AND type = 'involves' AND valid_to IS NULL)`,
|
||||
id, "task:"+sessionID)
|
||||
// Link execution to session for auto-continuation (nomos_plan_executions
|
||||
// was always empty — executions were never traceable back to sessions).
|
||||
if sid, serr := uuid.Parse(sessionID); serr == nil {
|
||||
pool.Exec(ctx, `
|
||||
INSERT INTO nomos_plan_executions (execution_id, session_id)
|
||||
VALUES ($1, $2) ON CONFLICT (execution_id) DO NOTHING`, id, sid)
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-classify: write the classification decision to the classifications
|
||||
// table (was always empty — 0 rows despite 1,884 executions). The route
|
||||
// matches the auto-run vs queue-for-approval decision below.
|
||||
classRoute := "escalate"
|
||||
if riskClass == policy.RiskReadOnly || riskClass == policy.RiskReversibleLow {
|
||||
classRoute = "auto-act"
|
||||
} else if riskClass == policy.RiskConfigMutation && assentWindowActive(ctx, pool, agentID, sessionID) {
|
||||
classRoute = "auto-act"
|
||||
} else if riskClass == policy.RiskDestructive && destructiveWindowActive(ctx, pool, agentID, targetSlug, sessionID) {
|
||||
classRoute = "auto-act"
|
||||
}
|
||||
classReason, _ := json.Marshal(map[string]string{
|
||||
"command": command, "purpose": purpose, "target": targetSlug, "declared_risk": declaredRisk,
|
||||
})
|
||||
classID, _ := uuid.NewV7()
|
||||
pool.Exec(ctx, `INSERT INTO entities (id, slug, type, name, attributes) VALUES ($1, $2, 'classification', $3, '{}')`,
|
||||
classID, "classification:"+classID.String(), "classification for "+execSlug)
|
||||
pool.Exec(ctx, `INSERT INTO classifications (entity_id, action, risk_class, route, reasoning, correlation_id)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)`,
|
||||
classID, actionCol, riskClass, classRoute, classReason, correlationID)
|
||||
// Link classification to execution.
|
||||
pool.Exec(ctx, `UPDATE executions SET classification_id = $2 WHERE entity_id = $1`, id, classID)
|
||||
// Graph edge: classification —precedes→ execution (required by ontology).
|
||||
pool.Exec(ctx, `INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT $1, $2, 'precedes', '{"by":"nomos"}'::jsonb, now()
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM relationships
|
||||
WHERE source_id = $1 AND target_id = $2 AND type = 'precedes' AND valid_to IS NULL)`,
|
||||
classID, id)
|
||||
|
||||
// Audit: record the execution creation with session_id for traceability.
|
||||
// Every run call, whether auto-run or queued-for-approval, gets an audit
|
||||
// entry so the agent's activity is traceable back to the originating session.
|
||||
var auditSessionID *uuid.UUID
|
||||
if sessionID != "" && sessionID != "ephemeral" {
|
||||
if sid, serr := uuid.Parse(sessionID); serr == nil {
|
||||
auditSessionID = &sid
|
||||
}
|
||||
}
|
||||
_ = observability.Audit(ctx, sqlcgen.New(pool), "agent", "nomos", "run",
|
||||
&id, "POST", "/mcp", correlationID, auditSessionID,
|
||||
map[string]any{"command": command, "target": targetSlug, "risk_class": riskClass, "purpose": purpose})
|
||||
|
||||
// read_only and reversible_low both run unattended, as seeds/policy.yaml
|
||||
// and .agents/OIKOS.md declare ("reversible_low — restart, cache clear,
|
||||
// sync pull. Unattended + ledger.").
|
||||
@@ -741,6 +960,11 @@ func classifyAndGate(ctx context.Context, pool *db.Pool, agentID, targetID uuid.
|
||||
// agent still cannot talk a command DOWN: declaring reversible_low on
|
||||
// something computed as config_mutation keeps config_mutation.
|
||||
if riskClass == policy.RiskReadOnly || riskClass == policy.RiskReversibleLow {
|
||||
if isLongRunningCommand(command) {
|
||||
autoRunAsync(ctx, pool, id, targetSlug, command)
|
||||
return textResult(fmt.Sprintf("run on %s (%s, async): started — execution %s. Poll with get_execution_status(%s) for result.",
|
||||
targetSlug, riskClass, id, id))
|
||||
}
|
||||
out, xerr := autoRun(ctx, pool, id, targetSlug, command)
|
||||
if xerr != nil {
|
||||
return textResult(fmt.Sprintf("run on %s: ERROR %v\n%s", targetSlug, xerr, out))
|
||||
@@ -759,6 +983,12 @@ func classifyAndGate(ctx context.Context, pool *db.Pool, agentID, targetID uuid.
|
||||
// consent. The assent window, opened only on operator approval, is the
|
||||
// sole gate for config_mutation auto-run.)
|
||||
if riskClass == policy.RiskConfigMutation && assentWindowActive(ctx, pool, agentID, sessionID) {
|
||||
if isLongRunningCommand(command) {
|
||||
autoRunAsync(ctx, pool, id, targetSlug, command)
|
||||
slog.Info("mcp: run async via assent window", "target", targetSlug, "execution_id", id)
|
||||
return textResult(fmt.Sprintf("run on %s (config_mutation, async via assent window): started — execution %s. Poll with get_execution_status(%s) for result.",
|
||||
targetSlug, id, id))
|
||||
}
|
||||
out, xerr := autoRun(ctx, pool, id, targetSlug, command)
|
||||
if xerr != nil {
|
||||
return textResult(fmt.Sprintf("run on %s: ERROR %v\n%s", targetSlug, xerr, out))
|
||||
@@ -774,6 +1004,12 @@ func classifyAndGate(ctx context.Context, pool *db.Pool, agentID, targetID uuid.
|
||||
// operator isn't asked to re-type "I confirm" for every single command
|
||||
// against the thing they just confirmed.
|
||||
if riskClass == policy.RiskDestructive && destructiveWindowActive(ctx, pool, agentID, targetSlug, sessionID) {
|
||||
if isLongRunningCommand(command) {
|
||||
autoRunAsync(ctx, pool, id, targetSlug, command)
|
||||
slog.Info("mcp: run async via destructive window", "target", targetSlug, "execution_id", id)
|
||||
return textResult(fmt.Sprintf("run on %s (destructive, async via confirmed-target window): started — execution %s. Poll with get_execution_status(%s) for result.",
|
||||
targetSlug, id, id))
|
||||
}
|
||||
out, xerr := autoRun(ctx, pool, id, targetSlug, command)
|
||||
if xerr != nil {
|
||||
return textResult(fmt.Sprintf("run on %s: ERROR %v\n%s", targetSlug, xerr, out))
|
||||
@@ -840,22 +1076,112 @@ func executeApprovedViaAPI(ctx context.Context, execID uuid.UUID, targetSlug, ac
|
||||
// for config_mutation auto-run now. See sessionHasPlan for the plan-existence
|
||||
// check used by the P1 plan-first gate.
|
||||
|
||||
// hostOnlyCommands maps command prefixes that are only valid on Proxmox host
|
||||
// targets (not LXCs or VMs). Running these against an lxc: or vm: target
|
||||
// always fails with "command not found" and wastes a turn.
|
||||
var hostOnlyCommands = map[string]bool{
|
||||
"qm": true,
|
||||
"pct": true,
|
||||
"pvesh": true,
|
||||
"iptables": true,
|
||||
}
|
||||
|
||||
// hostLxcCommands maps command prefixes valid on host:* and lxc:* but not vm:*.
|
||||
var hostLxcCommands = map[string]bool{
|
||||
"systemctl": true,
|
||||
"docker": true,
|
||||
}
|
||||
|
||||
// hostOnlyCommand checks whether the leading word of cmd is a host-only
|
||||
// command. Returns the command word and true if the command can only run on
|
||||
// a host: target.
|
||||
func hostOnlyCommand(cmd string) (string, bool) {
|
||||
trimmed := strings.TrimSpace(cmd)
|
||||
parts := strings.Fields(trimmed)
|
||||
if len(parts) == 0 {
|
||||
return "", false
|
||||
}
|
||||
first := parts[0]
|
||||
// Check for shell wrappers: bash -c 'actual_cmd', sh -c 'actual_cmd'
|
||||
if (first == "bash" || first == "sh") && len(parts) >= 3 && parts[1] == "-c" {
|
||||
// The actual command is inside the -c argument; extract the first word.
|
||||
// This handles `bash -c 'qm stop 100'` but not deeply nested wrappers.
|
||||
actual := strings.Trim(strings.Join(parts[2:], " "), "'\"")
|
||||
if inner := strings.Fields(actual); len(inner) > 0 {
|
||||
first = inner[0]
|
||||
}
|
||||
}
|
||||
// Strip path: /usr/sbin/qm → qm
|
||||
if idx := strings.LastIndexByte(first, '/'); idx >= 0 {
|
||||
first = first[idx+1:]
|
||||
}
|
||||
return first, hostOnlyCommands[first]
|
||||
}
|
||||
|
||||
// hostLxcCommand checks whether the leading word of cmd is a command valid on
|
||||
// host:* and lxc:* targets but not vm:*. Returns the command word and true if
|
||||
// the command is restricted to host/lxc.
|
||||
func hostLxcCommand(cmd string) (string, bool) {
|
||||
trimmed := strings.TrimSpace(cmd)
|
||||
parts := strings.Fields(trimmed)
|
||||
if len(parts) == 0 {
|
||||
return "", false
|
||||
}
|
||||
first := parts[0]
|
||||
if idx := strings.LastIndexByte(first, '/'); idx >= 0 {
|
||||
first = first[idx+1:]
|
||||
}
|
||||
return first, hostLxcCommands[first]
|
||||
}
|
||||
|
||||
// validateCommandSyntax checks for common LLM-generated bash errors that always
|
||||
// fail at the shell. Returns an error message or "" if the command looks valid.
|
||||
func validateCommandSyntax(cmd string) string {
|
||||
// Reject literal \n (the LLM sometimes writes `echo "---" && \n curl ...`
|
||||
// — the \n is literal in the command string, not an actual newline).
|
||||
if strings.Contains(cmd, "\\n") {
|
||||
return fmt.Sprintf("Command contains literal '\\n' — use ';' or '&&' between commands, not a literal backslash-n. Command: %q", cmd)
|
||||
}
|
||||
|
||||
// Reject `&& \n` patterns (the LLM writes `cmd1 && \n cmd2` — the \n is
|
||||
// a literal newline that bash interprets as a command separator, but the
|
||||
// leading backslash makes it a syntax error).
|
||||
if andBackslashRe.MatchString(cmd) {
|
||||
return fmt.Sprintf("Command contains '&&' followed by a literal backslash-newline — remove the backslash or use ';' instead. Command: %q", cmd)
|
||||
}
|
||||
|
||||
// Reject `\` at end of command with no continuation (last line ends with
|
||||
// backslash but there's nothing after it).
|
||||
trimmed := strings.TrimSpace(cmd)
|
||||
if strings.HasSuffix(trimmed, "\\") {
|
||||
return fmt.Sprintf("Command ends with a backslash but has nothing after it to continue. Remove the trailing '\\'. Command: %q", cmd)
|
||||
}
|
||||
|
||||
// Warn on common flag typos: `head - n`, `grep - i`, `tail - n`, etc.
|
||||
// These are space-between-flag-and-value errors the LLM produces.
|
||||
if flagSpaceRe.MatchString(cmd) {
|
||||
return fmt.Sprintf("Command has a space between a flag and its value (e.g. 'head - n' instead of 'head -n'). Remove the space. Command: %q", cmd)
|
||||
}
|
||||
|
||||
return ""
|
||||
}
|
||||
|
||||
var andBackslashRe = regexp.MustCompile(`&&\s*\\\s*\n`)
|
||||
var flagSpaceRe = regexp.MustCompile(`\b(head|tail|grep|sed|awk|sort|uniq|wc)\s+(-\w)\s+\w`)
|
||||
|
||||
// sessionHasPlan reports whether this nomos session has any plan step on
|
||||
// record (any generation, any status). Used by the P1 plan-first gate in
|
||||
// classifyAndGate to refuse `run` before `propose_plan` has been called.
|
||||
// A `replaced` step (from a prior plan generation that was superseded by a
|
||||
// follow-up sub-task — see store.reopenSession) still counts: it proves the
|
||||
// agent once framed a plan for this session, and the reopen path guarantees a
|
||||
// fresh `propose_plan` will run before the next `run` anyway. Fails closed
|
||||
// (returns true) when the query errors so a transient DB issue doesn't block
|
||||
// an otherwise-valid run.
|
||||
// record that isn't `replaced`. Replaced steps (from session reopen via
|
||||
// store.reopenSession) don't count — the agent must propose fresh plan before
|
||||
// any `run`. Fails closed (returns true) when the query errors so a transient
|
||||
// DB issue doesn't block an otherwise-valid run.
|
||||
func sessionHasPlan(ctx context.Context, pool *db.Pool, sessionID string) bool {
|
||||
if sessionID == "" {
|
||||
return true // no session → no gate (direct MCP call from a script)
|
||||
}
|
||||
var count int
|
||||
if err := pool.QueryRow(ctx,
|
||||
`SELECT COUNT(*) FROM session_plan_steps WHERE session_id = $1`,
|
||||
`SELECT COUNT(*) FROM session_plan_steps
|
||||
WHERE session_id = $1 AND status <> 'replaced'`,
|
||||
sessionID).Scan(&count); err != nil {
|
||||
return true // fail open on DB error — don't block work over a flake
|
||||
}
|
||||
|
||||
@@ -86,7 +86,7 @@ func TestNewServerRegistersTools(t *testing.T) {
|
||||
}()
|
||||
// pool is only used inside tool handlers (invoked per-call), not at
|
||||
// registration time, so a nil pool is safe for this construction test.
|
||||
s := newServer(nil, uuid.Nil)
|
||||
s := newServer(nil, uuid.Nil, nil)
|
||||
if s == nil {
|
||||
t.Fatal("newServer returned nil")
|
||||
}
|
||||
|
||||
@@ -2,820 +2,84 @@ package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/dtoro/oikos/internal/audit"
|
||||
"github.com/dtoro/oikos/internal/checkdefaults"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/policy"
|
||||
"github.com/google/uuid"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
// toolReg pairs a tool definition with its handler. allTools returns a slice
|
||||
// of these; newServer iterates it and registers each one wrapped with
|
||||
// withActivityLogging.
|
||||
type toolReg struct {
|
||||
tool *mcp.Tool
|
||||
handler toolHandler
|
||||
}
|
||||
|
||||
// allTools returns every MCP tool registration. Tool definitions, schemas,
|
||||
// descriptions, and handler bodies are kept verbatim from the former inline
|
||||
// newServer registrations.
|
||||
func allTools(pool *db.Pool, agentID uuid.UUID) []toolReg {
|
||||
return []toolReg{
|
||||
{tool: &mcp.Tool{Name: "get_entity", Description: "Get an entity by slug or UUID",
|
||||
InputSchema: objSchema(prop{"slug_or_id", "string", "Entity slug (e.g. host:hubris) or UUID"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
idOrSlug, _ := args["slug_or_id"].(string)
|
||||
return queryEntity(ctx, pool, idOrSlug), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "list_entities", Description: "List entities filtered by type, state, or search",
|
||||
InputSchema: objSchema(
|
||||
prop{"type", "string", "Filter by entity type"},
|
||||
prop{"state", "string", "Filter by lifecycle state"},
|
||||
prop{"q", "string", "Substring match on slug or name"},
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state, e.version, e.created_at, e.updated_at
|
||||
FROM entities e
|
||||
WHERE ($1::text IS NULL OR e.type = $1)
|
||||
AND ($2::text IS NULL OR e.state = $2)
|
||||
AND ($3::text IS NULL OR e.slug ILIKE '%'||$3||'%' OR e.name ILIKE '%'||$3||'%')
|
||||
ORDER BY e.slug LIMIT $4`,
|
||||
nStr(args["type"]), nStr(args["state"]), nStr(args["q"]), limit), "entity_table"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_relations", Description: "Get relationships for an entity",
|
||||
InputSchema: objSchema(prop{"entity_id", "string", "Entity slug"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_id"].(string)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT r.type, src.slug AS source, tgt.slug AS target
|
||||
FROM relationships r
|
||||
JOIN entities src ON src.id = r.source_id
|
||||
JOIN entities tgt ON tgt.id = r.target_id
|
||||
WHERE (src.slug = $1 OR tgt.slug = $1) AND r.valid_to IS NULL
|
||||
ORDER BY r.type`, slug), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_blast_radius", Description: "Find entities affected if this entity goes down",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_id", "string", "Entity slug"},
|
||||
prop{"depth", "integer", "Traversal depth (default 3)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_id"].(string)
|
||||
depth := int(getFloat(args, "depth", 3))
|
||||
return queryRows(ctx, pool,
|
||||
"SELECT e.slug, CAST(b.depth AS int) FROM blast_radius((SELECT id FROM entities WHERE slug = $1), $2) b JOIN entities e ON e.id = b.entity_id",
|
||||
slug, depth), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_health_summary", Description: "Current fleet health summary",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, st.health, st.last_check_at
|
||||
FROM entity_status st JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.type <> 'check'
|
||||
ORDER BY e.slug`), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_audit_trail", Description: "Query the audit log",
|
||||
InputSchema: objSchema(prop{"entity_id", "string", "Filter by affected entity UUID"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT id, ts, actor_type, action, entity_id::text, method, path, correlation_id
|
||||
FROM audit_log
|
||||
WHERE ($1::text IS NULL OR entity_id::text = $1)
|
||||
ORDER BY ts DESC LIMIT 50`, nStr(args["entity_id"])), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "search_knowledge", Description: "Full-text search across documentation (PostgreSQL FTS with ts_rank ranking). Returns a short snippet per hit, not the full note — call get_knowledge_content with the returned slug to read the whole thing.",
|
||||
InputSchema: objSchema(prop{"query", "string", "Search terms"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
q := nStr(args["query"])
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT ke.title, e.slug,
|
||||
ts_rank(ke.search, plainto_tsquery('english', $1)) AS rank,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', $1),
|
||||
'MaxWords=40, MinWords=15, ShortWord=3, MaxFragments=3,
|
||||
FragmentDelimiter=" ... "') AS snippet,
|
||||
ke.source, ke.tags
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE ke.search @@ plainto_tsquery('english', $1)
|
||||
ORDER BY rank DESC
|
||||
LIMIT 20`, q), "knowledge_results"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_entity_knowledge", Description: "All documents, investigations, and runbooks linked to an entity. Returns a headline per note, not the full text — call get_knowledge_content with the returned slug to read the whole thing.",
|
||||
InputSchema: objSchema(prop{"entity_slug", "string", "Entity slug (e.g. lxc:jellyfin, service:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_slug"].(string)
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT ke.title, ke.source, e.type AS kind, e.slug,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', '')) AS headline
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
JOIN relationships r ON r.source_id = ke.entity_id
|
||||
JOIN entities target ON target.id = r.target_id
|
||||
WHERE target.slug = $1
|
||||
AND r.valid_to IS NULL
|
||||
AND r.type IN ('documents', 'about')
|
||||
UNION
|
||||
SELECT ke.title, ke.source, e.type AS kind, e.slug,
|
||||
ts_headline('english', ke.content, plainto_tsquery('english', '')) AS headline
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
JOIN relationships r ON r.source_id = ke.entity_id
|
||||
JOIN entity_types target_type ON target_type.name = (SELECT type FROM entities WHERE slug = $1)
|
||||
JOIN entities ent ON ent.type = target_type.name AND ent.slug = $1
|
||||
WHERE r.valid_to IS NULL
|
||||
AND r.type = 'procedure-for'
|
||||
ORDER BY 1`, slug), "knowledge_results"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_knowledge_content", Description: "Full markdown body of one document/investigation/runbook, by its own entity slug. search_knowledge and get_entity_knowledge only return short snippets/headlines — once you know which note you need (from either of those, or because you already know its slug), call this to read the whole thing before acting on it.",
|
||||
InputSchema: objSchema(prop{"slug", "string", "The knowledge entity's own slug (e.g. document:containers/101-jellyfin, runbook:client-enrollment) — not the slug of an entity it's about."}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["slug"].(string)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT ke.title, e.slug, e.type AS kind, ke.content, ke.source, ke.tags, ke.updated_at::text
|
||||
FROM knowledge_entities ke
|
||||
JOIN entities e ON e.id = ke.entity_id
|
||||
WHERE e.slug = $1`, slug), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "upsert_knowledge", Description: "Write back what you learned so future sessions (and future you) benefit — this is how the system gets smarter over time. Use it AFTER solving a non-obvious problem, deploying a service, or discovering a gotcha: record the finding, the fix, and any caveats. Re-calling with the same title updates the existing note instead of duplicating. This is the ONLY way to persist knowledge; a chat message alone is forgotten. search_knowledge/get_entity_knowledge find it, get_knowledge_content reads the full body back.",
|
||||
InputSchema: objSchema(
|
||||
prop{"title", "string", "Short, specific, searchable title (e.g. 'Dragonfly memlock rlimit in unprivileged LXCs', not 'notes')."},
|
||||
prop{"content", "string", "The knowledge itself, in markdown. Be concrete: symptom, root cause, the exact fix/commands, and any caveats. Written for someone hitting this fresh."},
|
||||
prop{"about", "string", "Optional entity slug(s) this knowledge concerns. Pass a single slug (e.g. 'lxc:nfs-export') or a JSON array of slugs (e.g. '[\"lxc:nfs-export\", \"lxc:gitea\"]') to link to multiple entities. get_entity_knowledge surfaces it for each."},
|
||||
prop{"tags", "string", "Optional comma-separated tags (e.g. 'docker,networking,gotcha')."},
|
||||
prop{"kind", "string", "One of: investigation (a finding/incident analysis — default), document (reference), runbook (a repeatable procedure)."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return upsertKnowledge(ctx, pool, args)
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "update_entity_attributes", Description: "Merge new/changed attributes into an entity — the OTHER half of avoiding knowledge-base drift (upsert_knowledge records what you learned; this keeps the entity's own facts current). Use it when you discover something concrete about an entity's actual state that the graph doesn't reflect yet: a new IP, a version number, a config value, a discovered port — anything a FUTURE task would otherwise have to rediscover from scratch. Does NOT require approval (this updates the knowledge graph, not the live infrastructure). Merges shallowly — existing keys not mentioned are kept; keys you pass overwrite.",
|
||||
InputSchema: objSchema(
|
||||
prop{"slug", "string", "Entity slug to update (e.g. lxc:typetype, host:strong)."},
|
||||
prop{"attributes", "string", "JSON object string of attributes to merge in, e.g. {\"lan_ip\":\"192.168.8.50\",\"os\":\"debian-12\"}."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["slug"].(string)
|
||||
attrsStr, _ := args["attributes"].(string)
|
||||
if slug == "" || attrsStr == "" {
|
||||
return textResult("error: slug and attributes are required"), nil
|
||||
}
|
||||
var attrs map[string]any
|
||||
if err := json.Unmarshal([]byte(attrsStr), &attrs); err != nil {
|
||||
return textResult(fmt.Sprintf("error: attributes is not valid JSON: %v", err)), nil
|
||||
}
|
||||
attrsJSON, _ := json.Marshal(attrs)
|
||||
ct, err := pool.Exec(ctx, `
|
||||
UPDATE entities SET attributes = attributes || $2::jsonb, updated_at = now()
|
||||
WHERE slug = $1`, slug, string(attrsJSON))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error updating %s: %v", slug, err)), nil
|
||||
}
|
||||
if ct.RowsAffected() == 0 {
|
||||
return textResult(fmt.Sprintf("error: entity %q not found", slug)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Updated %s with %d attribute(s).", slug, len(attrs))), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "create_relationship", Description: "Record a relationship you discovered between two entities — the graph-structure half of keeping the knowledge base current (alongside update_entity_attributes and upsert_knowledge). Use it when you learn that one entity depends on, hosts, routes to, etc. another, and that edge isn't in the graph yet. type must be an existing relationship type (see get_relations output on similar entities for examples: hosts, provides, depends-on, configured-by, about, documents, ...). Idempotent — re-calling the same source/target/type is a no-op. Does NOT require approval.",
|
||||
InputSchema: objSchema(
|
||||
prop{"source", "string", "Source entity slug."},
|
||||
prop{"target", "string", "Target entity slug."},
|
||||
prop{"type", "string", "Relationship type name (must already exist in the ontology)."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
source, _ := args["source"].(string)
|
||||
target, _ := args["target"].(string)
|
||||
relType, _ := args["type"].(string)
|
||||
if source == "" || target == "" || relType == "" {
|
||||
return textResult("error: source, target, and type are required"), nil
|
||||
}
|
||||
var sourceID, targetID uuid.UUID
|
||||
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", source).Scan(&sourceID); err != nil {
|
||||
return textResult(fmt.Sprintf("error: source entity %q not found", source)), nil
|
||||
}
|
||||
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", target).Scan(&targetID); err != nil {
|
||||
return textResult(fmt.Sprintf("error: target entity %q not found", target)), nil
|
||||
}
|
||||
_, err := pool.Exec(ctx, `
|
||||
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
|
||||
SELECT $1, $2, $3, '{"by":"nomos"}'::jsonb, now()
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM relationships
|
||||
WHERE source_id = $1 AND target_id = $2 AND type = $3 AND valid_to IS NULL)`,
|
||||
sourceID, targetID, relType)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("error creating relationship: %v (is %q a valid relationship type?)", err, relType)), nil
|
||||
}
|
||||
return textResult(fmt.Sprintf("Recorded: %s —%s→ %s", source, relType, target)), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "query_metrics", Description: "Query time-series metrics",
|
||||
InputSchema: objSchema(prop{"hours", "integer", "Look-back window in hours (default 24)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
hours := int(getFloat(args, "hours", 24))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT time_bucket('1 hour', ts) AS bucket,
|
||||
entity_id::text, metric,
|
||||
ROUND(avg(value)::numeric, 2) AS avg,
|
||||
ROUND(min(value)::numeric, 2) AS min,
|
||||
ROUND(max(value)::numeric, 2) AS max
|
||||
FROM metric_samples
|
||||
WHERE ts > now() - make_interval(hours => $1)
|
||||
GROUP BY bucket, entity_id, metric
|
||||
ORDER BY bucket DESC LIMIT 100`, hours), "metric_chart"), nil
|
||||
}},
|
||||
|
||||
// ─── Phase 4: new tools ──────────────────────────────────────────
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_signal_history", Description: "Query open and recent signals",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_slug", "string", "Filter by target entity slug"},
|
||||
prop{"state", "string", "Filter by signal state (raised, resolved)"},
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT s.entity_id::text, s.kind, s.severity, s.state,
|
||||
s.occurrence_count, e.slug AS target_slug,
|
||||
s.first_seen_at, s.last_seen_at
|
||||
FROM signals s
|
||||
LEFT JOIN entities e ON e.id = s.target_entity_id
|
||||
WHERE ($1::text IS NULL OR e.slug = $1)
|
||||
AND ($2::text IS NULL OR s.state = $2)
|
||||
ORDER BY s.last_seen_at DESC LIMIT $3`,
|
||||
nStr(args["entity_slug"]), nStr(args["state"]), limit), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_patterns", Description: "List learned action patterns",
|
||||
InputSchema: objSchema(
|
||||
prop{"status", "string", "Filter by status (hypothesized, validated, active)"},
|
||||
prop{"entity_type", "string", "Filter by applies_type"},
|
||||
prop{"action", "string", "Filter by action"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT p.entity_id::text, p.applies_type, p.action, p.pattern,
|
||||
p.confidence, p.evidence_count, p.success_count, p.failure_count,
|
||||
p.status, p.quarantined, p.version, p.last_validated_at
|
||||
FROM patterns p
|
||||
WHERE ($1::text IS NULL OR p.status = $1)
|
||||
AND ($2::text IS NULL OR p.applies_type = $2)
|
||||
AND ($3::text IS NULL OR p.action = $3)
|
||||
ORDER BY p.applies_type, p.action`,
|
||||
nStr(args["status"]), nStr(args["entity_type"]), nStr(args["action"])), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_skills", Description: "List available automation skills",
|
||||
InputSchema: objSchema(
|
||||
prop{"status", "string", "Filter by status (active, inactive, deprecated)"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT s.entity_id::text, s.version, s.name, LEFT(s.procedure::text, 300) AS procedure_preview,
|
||||
s.applies_type, s.action, s.status, s.success_rate,
|
||||
s.changed_by::text, s.change_reason, s.last_used_at
|
||||
FROM skills s
|
||||
WHERE ($1::text IS NULL OR s.status = $1)
|
||||
ORDER BY s.name, s.version DESC`,
|
||||
nStr(args["status"])), nil
|
||||
}},
|
||||
|
||||
// ── request_execution (legacy fixed enum) retired 2026-07-14 ──
|
||||
// All mutations now route through `run`. The handler functions
|
||||
// (runRexecRestart, runRexecSystemctl, etc.) are kept as reference
|
||||
// for future runbook extraction — especially pct_create DNS/VMID logic.
|
||||
// DO NOT re-register this tool. See plans/2026-07-10-general-gated-execution.md.
|
||||
|
||||
{tool: &mcp.Tool{Name: "run", Description: "Run ANY shell command against any host, LXC, or VM. This is the general execution primitive — prefer it over asking the operator to run something manually, and don't wait for a matching fixed action to exist. Every command is automatically risk-classified: read-only inspection (cat, systemctl status, docker ps, journalctl, df, git status, ...) runs immediately; anything that changes state requires operator approval (granted by the operator replying \"go ahead\"/\"yes\" in chat, or via the Approve button); commands matching a destructive pattern (rm -rf, dd, mkfs, pct/qm destroy, DROP TABLE, reboot, piping curl into a shell, ...) always require approval regardless of what you declare. You cannot talk your way past the destructive check by declaring a lower risk.",
|
||||
InputSchema: objSchema(
|
||||
prop{"target", "string", "Target entity slug: host:<slug> (e.g. host:strong), lxc:<slug> (e.g. lxc:caddy), or vm:<slug> (e.g. vm:zimaos). LXC commands run via pct exec on their Proxmox host automatically. VM commands run via qm guest exec on their Proxmox host (requires the QEMU guest agent inside the VM — standard for Proxmox VMs)."},
|
||||
prop{"command", "string", "The shell command to run. Can be a full script (multi-line, &&-chained). Runs as root."},
|
||||
prop{"purpose", "string", "One sentence: why you're running this. Shown to the operator alongside the approval — be specific, this is what they're approving."},
|
||||
prop{"declared_risk", "string", "Optional self-assessment: read_only, reversible_low, config_mutation, or destructive. This can only ESCALATE the automatic classification, never lower it — declaring a mutating command as read_only has no effect."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
targetSlug, _ := args["target"].(string)
|
||||
command, _ := args["command"].(string)
|
||||
purpose, _ := args["purpose"].(string)
|
||||
declaredRisk, _ := args["declared_risk"].(string)
|
||||
sessionID, _ := args["_session_id"].(string)
|
||||
if targetSlug == "" || command == "" {
|
||||
return textResult("error: target and command are required"), nil
|
||||
}
|
||||
|
||||
var targetID uuid.UUID
|
||||
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", targetSlug).Scan(&targetID); err != nil {
|
||||
return textResult(fmt.Sprintf("target not found: %s", targetSlug)), nil
|
||||
}
|
||||
|
||||
return classifyAndGate(ctx, pool, agentID, targetID, targetSlug, command, purpose, declaredRisk, sessionID), nil
|
||||
}},
|
||||
|
||||
// inspect_path is the bulk fact-gathering tool from
|
||||
// plans/2026-07-18-session-review-three-sessions.md P1.5.
|
||||
// Sessions 1e9c7691 and 55927f0a each spent ~15 `run` calls
|
||||
// gathering identical facts (`mount | grep`, `df`, `ls -la`,
|
||||
// `stat`) across hosts and LXCs to understand where a path
|
||||
// lives, who mounts it, and what permissions it has. This tool
|
||||
// collapses that fan-out into one call: pass a path and a list
|
||||
// of targets, get back per-target mount/df/ls/stat output as
|
||||
// JSON. All commands are read-only, so no approval is needed.
|
||||
{tool: &mcp.Tool{Name: "inspect_path", Description: "Bulk fact-gathering: run mount/df/ls/stat for the same path across multiple host/LXC/VM targets in ONE call. Returns a JSON object keyed by target slug, each with the target's view of the path (mount source, filesystem, size, top-level entries with ownership/permissions). Use this instead of N separate `run` calls when you need to understand a path's footprint across the fleet (e.g. tracing where a volume is mounted, checking permissions on the same NFS path from server + client). All commands are read-only — no approval needed.",
|
||||
InputSchema: objSchema(
|
||||
prop{"path", "string", "Absolute path to inspect on each target (e.g. /mnt/media_local, /media/ludo-library)."},
|
||||
prop{"targets", "array", "List of target entity slugs (host:strong, lxc:nfs-export, vm:zimaos, …). Up to 8 per call."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
path, _ := args["path"].(string)
|
||||
if path == "" {
|
||||
return textResult("error: path is required"), nil
|
||||
}
|
||||
rawTargets, _ := args["targets"].([]any)
|
||||
if len(rawTargets) == 0 {
|
||||
return textResult("error: at least one target is required"), nil
|
||||
}
|
||||
if len(rawTargets) > 8 {
|
||||
return textResult("error: at most 8 targets per inspect_path call (use two calls if you need more)"), nil
|
||||
}
|
||||
targets := make([]string, 0, len(rawTargets))
|
||||
for _, t := range rawTargets {
|
||||
if s, ok := t.(string); ok && s != "" {
|
||||
targets = append(targets, s)
|
||||
}
|
||||
}
|
||||
results := inspectPathAcrossTargets(ctx, pool, path, targets)
|
||||
out, _ := json.MarshalIndent(results, "", " ")
|
||||
return textResult(string(out)), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "http_get", Description: "Fetch a public web page or raw file (e.g. a GitHub README/raw URL) and return sanitized text. Use this to research how to deploy a service before provisioning. HTTP/HTTPS only; body is truncated to ~16KB.",
|
||||
InputSchema: objSchema(
|
||||
prop{"url", "string", "Absolute http(s) URL to fetch"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
rawURL, _ := args["url"].(string)
|
||||
return httpGet(ctx, rawURL), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_execution_status", Description: "Check the status of a requested execution",
|
||||
InputSchema: objSchema(
|
||||
prop{"execution_id", "string", "Execution UUID (from request_execution output)"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
execID, _ := args["execution_id"].(string)
|
||||
if execID == "" {
|
||||
return textResult("execution_id required"), nil
|
||||
}
|
||||
eid, err := uuid.Parse(execID)
|
||||
if err != nil {
|
||||
// Try finding by exec slug prefix
|
||||
var found uuid.UUID
|
||||
err2 := pool.QueryRow(ctx, "SELECT entity_id FROM executions WHERE entity_id::text LIKE $1 LIMIT 1", execID+"%").Scan(&found)
|
||||
if err2 != nil {
|
||||
return textResult(fmt.Sprintf("execution not found: %s", execID)), nil
|
||||
}
|
||||
eid = found
|
||||
}
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.entity_id::text, e.action, e.risk_class, e.status,
|
||||
e.result::text, e.duration_ms, e.started_at::text,
|
||||
e.completed_at::text, e.correlation_id
|
||||
FROM executions e
|
||||
WHERE e.entity_id = $1`, eid), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_trend", Description: "Get metric trends for an entity",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_id", "string", "Entity slug"},
|
||||
prop{"days", "integer", "Look-back window in days (default 7)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_id"].(string)
|
||||
days := int(getFloat(args, "days", 7))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT metric,
|
||||
ROUND(avg(value)::numeric, 2) AS avg_val,
|
||||
ROUND(stddev(value)::numeric, 2) AS std_val,
|
||||
count(*) AS sample_count,
|
||||
ROUND(regr_slope(value, EXTRACT(EPOCH FROM ts)::numeric)::numeric, 4) AS slope
|
||||
FROM metric_samples ms
|
||||
JOIN entities e ON e.id = ms.entity_id
|
||||
WHERE e.slug = $1 AND ts >= now() - make_interval(days => $2)
|
||||
GROUP BY metric
|
||||
ORDER BY metric`, slug, days), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_event_timeline", Description: "Get recent events",
|
||||
InputSchema: objSchema(
|
||||
prop{"severity", "string", "Filter by severity (info, warn, error)"},
|
||||
prop{"entity_slug", "string", "Filter by entity slug"},
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT ev.ts, ev.type, ev.severity, ev.source, e.slug AS entity_slug,
|
||||
ev.data::text AS message, ev.correlation_id
|
||||
FROM events ev
|
||||
LEFT JOIN entities e ON e.id = ev.entity_id
|
||||
WHERE ($1::text IS NULL OR ev.severity = $1)
|
||||
AND ($2::text IS NULL OR e.slug = $2)
|
||||
ORDER BY ev.ts DESC LIMIT $3`,
|
||||
nStr(args["severity"]), nStr(args["entity_slug"]), limit), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_agent_activity", Description: "Agent self-inspection: query agent activity log",
|
||||
InputSchema: objSchema(
|
||||
prop{"limit", "integer", "Max rows (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
limit := int(getFloat(args, "limit", 50))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT id, ts, agent_id::text, session_id, activity_type, tool_name,
|
||||
entity_id::text, left(input_summary, 200) AS input_summary,
|
||||
left(output_summary, 200) AS output_summary,
|
||||
duration_ms, token_count, success, correlation_id
|
||||
FROM agent_activity
|
||||
WHERE agent_id = $1
|
||||
ORDER BY ts DESC LIMIT $2`, agentID, limit), "change_log"), nil
|
||||
}},
|
||||
|
||||
// ─── Phase 5: operational MCP tools ──────────────────────────────
|
||||
|
||||
{tool: &mcp.Tool{Name: "list_lxcs", Description: "List all LXC containers with ID, host, IP, state, and last-audited hint. Pass state=\"active\" to exclude destroyed/deprecated containers. The last_audited_at column shows the most recent knowledge entry (investigation or document tagged audit/update) linked via an 'about' edge — use it to skip re-running `run` against LXCs that were already audited recently.",
|
||||
InputSchema: objSchema(
|
||||
prop{"state", "string", "Optional: filter by entity state (active, destroyed, …)"},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
state, _ := argsMap(req)["state"].(string)
|
||||
var statePtr *string
|
||||
if state != "" {
|
||||
statePtr = &state
|
||||
}
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.name, e.attributes->>'pve_id' AS pve_id,
|
||||
e.attributes->>'lan_ip' AS lan_ip,
|
||||
e.state,
|
||||
st.health, st.last_check_at,
|
||||
(SELECT MAX(k.created_at)
|
||||
FROM relationships r
|
||||
JOIN knowledge_entities k ON k.entity_id = r.source_id
|
||||
WHERE r.target_id = e.id
|
||||
AND r.type = 'about'
|
||||
AND r.valid_to IS NULL
|
||||
AND (k.tags @> ARRAY['audit']::text[]
|
||||
OR k.tags @> ARRAY['update']::text[]
|
||||
OR k.title ILIKE '%audit%'
|
||||
OR k.title ILIKE '%update%')
|
||||
) AS last_audited_at
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.type = 'lxc'
|
||||
AND ($1::text IS NULL OR e.state = $1)
|
||||
ORDER BY CASE WHEN e.state = 'active' THEN 0 ELSE 1 END,
|
||||
(e.attributes->>'pve_id')::int`, statePtr), "lxc_list"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "ping_service", Description: "Check if a service is reachable via HTTP",
|
||||
InputSchema: objSchema(prop{"service_slug", "string", "Service entity slug"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("service_slug is required"), nil
|
||||
}
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT st.health, st.last_check_at, e.attributes->>'url' AS url
|
||||
FROM entity_status st
|
||||
JOIN entities e ON e.id = st.entity_id
|
||||
WHERE e.slug = $1`, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("query error: %v", err)), nil
|
||||
}
|
||||
defer rows.Close()
|
||||
if !rows.Next() {
|
||||
return textResult(fmt.Sprintf("service not found: %s", slug)), nil
|
||||
}
|
||||
var health, lastCheck, url string
|
||||
rows.Scan(&health, &lastCheck, &url)
|
||||
if url == "" {
|
||||
url = "(no URL in entity attributes)"
|
||||
}
|
||||
return textResult(fmt.Sprintf("health=%s last_check=%s url=%s", health, lastCheck, url)), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "tail_log", Description: "Get recent log lines from a service via journalctl",
|
||||
InputSchema: objSchema(
|
||||
prop{"service_slug", "string", "Service entity slug (e.g. lxc:caddy)"},
|
||||
prop{"lines", "integer", "Number of lines (default 50)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
n := int(getFloat(args, "lines", 50))
|
||||
if slug == "" {
|
||||
return textResult("service_slug is required"), nil
|
||||
}
|
||||
host, user, err := resolveHost(ctx, pool, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
|
||||
}
|
||||
svc := strings.TrimPrefix(slug, "lxc:")
|
||||
out, err := sshExec(ctx, host, user, fmt.Sprintf("journalctl -u %s -n %d --no-pager 2>&1 || true", svc, n))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_service_status", Description: "Check systemd service status on a host",
|
||||
InputSchema: objSchema(
|
||||
prop{"service_slug", "string", "Service entity slug (e.g. lxc:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("service_slug is required"), nil
|
||||
}
|
||||
host, user, err := resolveHost(ctx, pool, slug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
|
||||
}
|
||||
svc := strings.TrimPrefix(slug, "lxc:")
|
||||
out, err := sshExec(ctx, host, user,
|
||||
fmt.Sprintf("systemctl is-active %s; systemctl is-enabled %s; systemctl show %s -p ActiveEnterTimestamp -p SubState 2>&1 || true", svc, svc, svc))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_lxc_state", Description: "Get LXC container resource state from Proxmox host",
|
||||
InputSchema: objSchema(
|
||||
prop{"lxc_slug", "string", "LXC entity slug (e.g. lxc:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["lxc_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("lxc_slug is required"), nil
|
||||
}
|
||||
var pveID string
|
||||
err := pool.QueryRow(ctx, "SELECT attributes->>'pve_id' FROM entities WHERE slug = $1", slug).Scan(&pveID)
|
||||
if err != nil || pveID == "" {
|
||||
return textResult(fmt.Sprintf("LXC not found or missing pve_id: %s", slug)), nil
|
||||
}
|
||||
// Resolve the Proxmox host — find the host that runs this LXC
|
||||
var hostID uuid.UUID
|
||||
err = pool.QueryRow(ctx, `
|
||||
SELECT t.id FROM entities t
|
||||
JOIN relationships r ON r.source_id = t.id
|
||||
JOIN entities s ON s.id = r.target_id
|
||||
WHERE s.slug = $1 AND r.type = 'hosts' AND r.valid_to IS NULL
|
||||
LIMIT 1`, slug).Scan(&hostID)
|
||||
if err != nil {
|
||||
// Fallback: use the inventory host attribute if no relationship
|
||||
var hostSlug string
|
||||
err = pool.QueryRow(ctx, "SELECT attributes->>'host' FROM entities WHERE slug = $1", slug).Scan(&hostSlug)
|
||||
if err != nil || hostSlug == "" {
|
||||
return textResult(fmt.Sprintf("cannot resolve Proxmox host for %s", slug)), nil
|
||||
}
|
||||
var host, user string
|
||||
host, user, err = resolveHost(ctx, pool, "host:"+hostSlug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve: %v", err)), nil
|
||||
}
|
||||
out, err2 := sshExec(ctx, host, user, fmt.Sprintf("pct status %s --verbose 2>&1 || true", pveID))
|
||||
if err2 != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err2)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}
|
||||
var hostSlug string
|
||||
pool.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", hostID).Scan(&hostSlug)
|
||||
host, user, err := resolveHost(ctx, pool, hostSlug)
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
|
||||
}
|
||||
out, err := sshExec(ctx, host, user, fmt.Sprintf("pct status %s --verbose 2>&1 || true", pveID))
|
||||
if err != nil {
|
||||
return textResult(fmt.Sprintf("ssh: %v", err)), nil
|
||||
}
|
||||
return textResult(out), nil
|
||||
}},
|
||||
|
||||
// ─── Client introspection tools (plan: client-lifecycle Phase 3) ──
|
||||
|
||||
{tool: &mcp.Tool{Name: "whoami", Description: "Get the current entity record, peers, and health for a host",
|
||||
InputSchema: objSchema(prop{"hostname", "string", "Hostname of the calling machine"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
hostname, _ := args["hostname"].(string)
|
||||
if hostname == "" {
|
||||
return textResult("error: hostname required"), nil
|
||||
}
|
||||
slug := "ws:" + hostname
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state,
|
||||
COALESCE(st.health, 'unknown') AS health,
|
||||
COALESCE(st.last_check_at::text, '') AS last_check,
|
||||
e.attributes->>'mesh_ip' AS mesh_ip,
|
||||
e.attributes->>'age_pubkey' AS age_pubkey,
|
||||
e.enrolled_at
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.slug = $1
|
||||
ORDER BY e.slug`, slug), "entity_card"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "explain", Description: "Compact context card for a service: type, state, health, relations, risk",
|
||||
InputSchema: objSchema(prop{"service_slug", "string", "Service entity slug (e.g. service:jellyfin, lxc:caddy)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
if slug == "" {
|
||||
return textResult("error: service_slug required"), nil
|
||||
}
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.name, e.state,
|
||||
COALESCE(st.health, 'unknown') AS health,
|
||||
COALESCE(st.last_check_at::text, '') AS last_check,
|
||||
e.version, e.updated_at,
|
||||
COALESCE(e.attributes::text, '{}') AS attrs
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.slug = $1`, slug), "entity_card"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "preflight", Description: "Risk classification for an action on a service",
|
||||
InputSchema: objSchema(
|
||||
prop{"service_slug", "string", "Entity slug"},
|
||||
prop{"action", "string", "Planned action (restart, deploy, destroy, etc.)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["service_slug"].(string)
|
||||
action, _ := args["action"].(string)
|
||||
if slug == "" || action == "" {
|
||||
return textResult("error: service_slug and action required"), nil
|
||||
}
|
||||
return queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.state,
|
||||
CASE
|
||||
WHEN $2 IN ('restart', 'logs', 'status') THEN 'reversible_low'
|
||||
WHEN $2 IN ('deploy', 'upgrade', 'configure') THEN 'config_mutation'
|
||||
WHEN $2 IN ('destroy', 'wipe', 'revoke') THEN 'destructive'
|
||||
ELSE 'read_only'
|
||||
END AS risk_class,
|
||||
CASE
|
||||
WHEN $2 IN ('read_only','reversible_low') THEN 'auto-act'
|
||||
WHEN $2 = 'config_mutation' THEN 'operator-approval'
|
||||
ELSE 'operator-approval+confirmation'
|
||||
END AS approval
|
||||
FROM entities e WHERE e.slug = $1`, slug, action), nil
|
||||
}},
|
||||
|
||||
// classify_command is the command-scoped preflight from
|
||||
// plans/2026-07-20-session-review-ten-sessions.md P0.2. The
|
||||
// existing `preflight` tool is entity/action-scoped — useless when
|
||||
// the agent is composing a `run` command and needs to know whether
|
||||
// the classifier will accept it before submitting. Without this,
|
||||
// the agent has to retry with cosmetic variations until it finds
|
||||
// one that passes (see sessions a51e2086, 8acea2e3 — three
|
||||
// duplicate rclone sessions, all bouncing off the classifier).
|
||||
// Call this BEFORE `run` whenever the classification is uncertain.
|
||||
{tool: &mcp.Tool{Name: "classify_command", Description: "Pre-flight risk classification for a shell command BEFORE calling run. Returns the risk class (read_only / reversible_low / config_mutation / destructive) that `run` would assign. Use this when you're unsure whether a command will auto-execute or need approval — e.g. `pct exec`, `curl`, compound commands, or anything that might be mistaken for mutation. If this returns read_only, the same command will auto-execute via run with no approval; if it returns config_mutation, expect to need operator approval (or pre-frame the command so it classifies lower). Declared risk can only escalate, never de-escalate.",
|
||||
InputSchema: objSchema(
|
||||
prop{"command", "string", "The exact shell command you intend to pass to run."},
|
||||
prop{"declared_risk", "string", "Optional self-assessment you would pass to run (read_only, reversible_low, config_mutation, destructive). Mirrors run's declared_risk parameter."},
|
||||
),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
command, _ := args["command"].(string)
|
||||
declaredRisk, _ := args["declared_risk"].(string)
|
||||
if command == "" {
|
||||
return textResult("error: command is required"), nil
|
||||
}
|
||||
risk := policy.ClassifyCommand(command, declaredRisk)
|
||||
note := ""
|
||||
switch risk {
|
||||
case policy.RiskReadOnly:
|
||||
note = "auto-acts on `run` (no approval needed)."
|
||||
case policy.RiskReversibleLow:
|
||||
note = "auto-acts on `run` (no approval needed)."
|
||||
case policy.RiskConfigMutation:
|
||||
note = "requires operator approval on `run` (or loose assent window active)."
|
||||
case policy.RiskDestructive:
|
||||
note = "requires explicit operator confirmation on `run` (typed \"I confirm\" phrase)."
|
||||
}
|
||||
out, _ := json.Marshal(map[string]any{
|
||||
"command": command,
|
||||
"declared_risk": declaredRisk,
|
||||
"risk_class": risk,
|
||||
"note": note,
|
||||
})
|
||||
return textResult(string(out)), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_change_history", Description: "Last N change-ledger entries for an entity",
|
||||
InputSchema: objSchema(
|
||||
prop{"entity_slug", "string", "Entity slug"},
|
||||
prop{"limit", "integer", "Max entries (default 20)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
slug, _ := args["entity_slug"].(string)
|
||||
limit := int(getFloat(args, "limit", 20))
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT al.ts AS timestamp, al.actor_type, al.actor_id::text AS actor_label,
|
||||
al.action, al.method, al.path,
|
||||
al.detail::text AS details
|
||||
FROM audit_log al
|
||||
JOIN entities e ON e.id = al.entity_id
|
||||
WHERE e.slug = $1
|
||||
ORDER BY al.ts DESC
|
||||
LIMIT $2`, slug, limit), "change_log"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "get_state_snapshot", Description: "Last scheduler Observe-pass: fleet health, disk, drift count",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
return annotateJSONResult(queryRows(ctx, pool, `
|
||||
SELECT e.slug, e.type, e.state,
|
||||
COALESCE(st.health, 'unknown') AS health,
|
||||
COALESCE(st.last_check_at::text, '') AS last_check
|
||||
FROM entities e
|
||||
LEFT JOIN entity_status st ON st.entity_id = e.id
|
||||
WHERE e.state IS NOT NULL
|
||||
OR st.health IS NOT NULL
|
||||
ORDER BY st.health, e.slug
|
||||
LIMIT 200
|
||||
`), "fleet_snapshot"), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "audit_knowledge_graph", Description: "Read-only drift report over the knowledge graph and monitoring: orphan check entities, checks targeting deprecated/destroyed entities, probes stuck down/unknown, unmonitored declared entity types, and live edges pointing at destroyed targets. Returns ranked findings with a suggested remediation runbook each. Use this to validate the graph is complete and consistent before trusting health/blast-radius answers. Does NOT mutate anything.",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
findings, summary := audit.Report(ctx, pool)
|
||||
b, _ := json.Marshal(map[string]any{"findings": findings, "summary": summary})
|
||||
return textResult(string(b)), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "discover_infra_drift", Description: "Read-only live discovery: compares running Proxmox guests (pct/qm list on every proxmox host) against the DB graph. Returns guests running with no entity (missing) and entities whose pve_id is no longer live (ghost) — drift the DB-only audit_knowledge_graph cannot see. Reaches hosts over the same SSH/pct path the checks use. Does NOT mutate anything.",
|
||||
InputSchema: objSchema(),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
b, _ := json.Marshal(discoverInfraDrift(ctx, pool))
|
||||
return textResult(string(b)), nil
|
||||
}},
|
||||
|
||||
{tool: &mcp.Tool{Name: "list_my_secrets", Description: "List secrets accessible to this client by public key",
|
||||
InputSchema: objSchema(prop{"caller_pubkey", "string", "Age public key of the caller (optional)"}),
|
||||
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
|
||||
args := argsMap(req)
|
||||
pubkey, _ := args["caller_pubkey"].(string)
|
||||
// Match entities where age_pubkey attribute contains the caller's key.
|
||||
query := `
|
||||
SELECT e.slug, e.type, e.name,
|
||||
e.attributes->>'age_pubkey' AS age_pubkey
|
||||
FROM entities e
|
||||
WHERE e.attributes->>'age_pubkey' IS NOT NULL`
|
||||
var dbArgs []any
|
||||
if pubkey != "" {
|
||||
query += ` AND e.attributes->>'age_pubkey' = $1`
|
||||
dbArgs = append(dbArgs, pubkey)
|
||||
}
|
||||
query += ` ORDER BY e.slug LIMIT 100`
|
||||
return queryRows(ctx, pool, query, dbArgs...), nil
|
||||
}},
|
||||
}
|
||||
func allTools(pool *db.Pool, agentID uuid.UUID, sec secretBackend) []toolReg {
|
||||
return append(append(append(append(
|
||||
[]toolReg{},
|
||||
EntityTools(pool, agentID, sec)...),
|
||||
OpsTools(pool, agentID, sec)...),
|
||||
KnowledgeTools(pool, agentID, sec)...),
|
||||
AnalysisTools(pool, agentID, sec)...)
|
||||
}
|
||||
|
||||
func formatCheckResult(res checkdefaults.Result) string {
|
||||
var b strings.Builder
|
||||
if res.Created > 0 {
|
||||
fmt.Fprintf(&b, " Derived %d check(s).", res.Created)
|
||||
}
|
||||
if res.Undeclared {
|
||||
b.WriteString(" Type declares no monitoring — no checks derived (set the entity's `monitoring` attribute and call update_entity_attributes to regenerate).")
|
||||
}
|
||||
for _, s := range res.Skipped {
|
||||
fmt.Fprintf(&b, " Skipped %s (%s).", s.Kind, s.Reason)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func formatCreateResult(slug, entityType string, res checkdefaults.Result) string {
|
||||
return fmt.Sprintf("Created %s (%s).%s", slug, entityType, formatCheckResult(res))
|
||||
}
|
||||
|
||||
func rowsToMap(ctx context.Context, pool *db.Pool, query string, args ...any) map[string]any {
|
||||
m := map[string]any{}
|
||||
rows, err := pool.Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return m
|
||||
}
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var key string
|
||||
var val int
|
||||
if rows.Scan(&key, &val) == nil {
|
||||
m[key] = val
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func queryRowsJSONSingle(ctx context.Context, pool *db.Pool, query string, args ...any) []map[string]any {
|
||||
rows, err := pool.Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
cols := rows.FieldDescriptions()
|
||||
var items []map[string]any
|
||||
for rows.Next() {
|
||||
vals, err := rows.Values()
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
m := make(map[string]any)
|
||||
for i, col := range cols {
|
||||
m[string(col.Name)] = fmt.Sprintf("%v", vals[i])
|
||||
}
|
||||
items = append(items, m)
|
||||
}
|
||||
return items
|
||||
}
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ import (
|
||||
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/health"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
@@ -25,7 +26,14 @@ import (
|
||||
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
|
||||
slog.Info("notifier: starting")
|
||||
|
||||
// Liveness probe (plan D5): the notifier ticks every 15s (approvals) and
|
||||
// 30s (reactions). 2 min staleness covers a slow Matrix round-trip plus a
|
||||
// missed tick without false-failing.
|
||||
probe := health.New(2 * time.Minute)
|
||||
probe.Serve(ctx, cfg.HealthListen)
|
||||
|
||||
processPendingApprovals(ctx, pool, cfg)
|
||||
probe.Bump()
|
||||
|
||||
ticker := time.NewTicker(15 * time.Second)
|
||||
defer ticker.Stop()
|
||||
@@ -40,8 +48,10 @@ func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
|
||||
return
|
||||
case <-ticker.C:
|
||||
processPendingApprovals(ctx, pool, cfg)
|
||||
probe.Bump()
|
||||
case <-reactionTimer.C:
|
||||
pollReactions(ctx, pool, cfg)
|
||||
probe.Bump()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -52,13 +62,13 @@ func RunnerForMain() func(context.Context, *db.Pool, config.Config) {
|
||||
}
|
||||
|
||||
type pendingApproval struct {
|
||||
ID uuid.UUID
|
||||
Action string
|
||||
RiskClass string
|
||||
TokenHash *string
|
||||
AlertSentAt *time.Time
|
||||
MatrixEventID *string
|
||||
ExpiresAt time.Time
|
||||
ID uuid.UUID
|
||||
Action string
|
||||
RiskClass string
|
||||
TokenHash *string
|
||||
AlertSentAt *time.Time
|
||||
MatrixEventID *string
|
||||
ExpiresAt time.Time
|
||||
}
|
||||
|
||||
// processPendingApprovals finds pending approvals, generates tokens, and sends Matrix alerts.
|
||||
@@ -173,7 +183,7 @@ func checkReaction(ctx context.Context, cfg config.Config, roomID, eventID strin
|
||||
|
||||
var result struct {
|
||||
Chunk []struct {
|
||||
Type string `json:"type"`
|
||||
Type string `json:"type"`
|
||||
Content struct {
|
||||
RelatesTo map[string]string `json:"m.relates_to"`
|
||||
} `json:"content"`
|
||||
@@ -259,7 +269,9 @@ func sendMatrixAlert(ctx context.Context, cfg config.Config, approvalID uuid.UUI
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var mxResp struct{ EventID string `json:"event_id"` }
|
||||
var mxResp struct {
|
||||
EventID string `json:"event_id"`
|
||||
}
|
||||
json.NewDecoder(resp.Body).Decode(&mxResp)
|
||||
|
||||
if mxResp.EventID == "" {
|
||||
|
||||
@@ -16,7 +16,7 @@ import (
|
||||
// starts being populated when OIDC identity resolution lands.
|
||||
func Audit(ctx context.Context, q *sqlcgen.Queries, actorType, actorLabel,
|
||||
action string, entityID *uuid.UUID, method, path, correlationID string,
|
||||
detail map[string]any) error {
|
||||
sessionID *uuid.UUID, detail map[string]any) error {
|
||||
|
||||
if detail == nil {
|
||||
detail = map[string]any{}
|
||||
@@ -36,6 +36,7 @@ func Audit(ctx context.Context, q *sqlcgen.Queries, actorType, actorLabel,
|
||||
Path: &path,
|
||||
Detail: detailJSON,
|
||||
CorrelationID: corr,
|
||||
SessionID: sessionID,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
102
internal/ontology/preconditions_test.go
Normal file
102
internal/ontology/preconditions_test.go
Normal file
@@ -0,0 +1,102 @@
|
||||
package ontology
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/dtoro/oikos/internal/domain"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
// The lifecycle precondition checks split into a pure attribute/type guard
|
||||
// and a DB query. These cover the pure guards at 0%: the entity-type skip
|
||||
// rules and the attribute presence/absence semantics. The DB-backed checks
|
||||
// (health, edges, backups, docs) are exercised by make test-db.
|
||||
//
|
||||
// ctx/pool/entityID are unused by the pure guards, so nil is safe here.
|
||||
var (
|
||||
noCtx = context.Background()
|
||||
noPool *pgxpool.Pool // nil: the pure guards never touch the pool
|
||||
noID = uuid.New()
|
||||
)
|
||||
|
||||
func TestCheckAgeKeyEnrolled(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
entityType string
|
||||
attrs map[string]any
|
||||
wantErr bool
|
||||
}{
|
||||
{"workstation with age key", "workstation", map[string]any{"age_pubkey": "age1abc"}, false},
|
||||
{"workstation missing age key", "workstation", map[string]any{}, true},
|
||||
{"server needs a key too", "server", map[string]any{}, true},
|
||||
{"lxc is exempt", "lxc", map[string]any{}, false},
|
||||
{"vm is exempt", "vm", map[string]any{}, false},
|
||||
{"docker-container is exempt", "docker-container", map[string]any{}, false},
|
||||
{"nil attrs on a workstation", "workstation", nil, true},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
err := checkAgeKeyEnrolled(noCtx, noPool, noID, c.entityType, c.attrs)
|
||||
if c.wantErr && !errors.Is(err, domain.ErrInvalidTransition) {
|
||||
t.Errorf("want ErrInvalidTransition, got %v", err)
|
||||
}
|
||||
if !c.wantErr && err != nil {
|
||||
t.Errorf("want nil, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckMeshJoined(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
entityType string
|
||||
attrs map[string]any
|
||||
wantErr bool
|
||||
}{
|
||||
{"workstation with mesh_ip", "workstation", map[string]any{"mesh_ip": "10.0.0.5"}, false},
|
||||
{"workstation missing mesh_ip", "workstation", map[string]any{}, true},
|
||||
{"server missing mesh_ip", "server", map[string]any{}, true},
|
||||
{"lxc is exempt", "lxc", map[string]any{}, false},
|
||||
{"vm is exempt", "vm", map[string]any{}, false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
err := checkMeshJoined(noCtx, noPool, noID, c.entityType, c.attrs)
|
||||
if c.wantErr && !errors.Is(err, domain.ErrInvalidTransition) {
|
||||
t.Errorf("want ErrInvalidTransition, got %v", err)
|
||||
}
|
||||
if !c.wantErr && err != nil {
|
||||
t.Errorf("want nil, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckSecretsRevoked(t *testing.T) {
|
||||
// checkSecretsRevoked treats an ABSENT age_pubkey as "secrets revoked"
|
||||
// (the inverse of checkAgeKeyEnrolled). It is type-agnostic.
|
||||
cases := []struct {
|
||||
name string
|
||||
attrs map[string]any
|
||||
wantErr bool
|
||||
}{
|
||||
{"age key gone → revoked", map[string]any{}, false},
|
||||
{"age key still present → blocked", map[string]any{"age_pubkey": "age1abc"}, true},
|
||||
{"nil attrs → revoked", nil, false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
err := checkSecretsRevoked(noCtx, noPool, noID, "workstation", c.attrs)
|
||||
if c.wantErr && !errors.Is(err, domain.ErrInvalidTransition) {
|
||||
t.Errorf("want ErrInvalidTransition, got %v", err)
|
||||
}
|
||||
if !c.wantErr && err != nil {
|
||||
t.Errorf("want nil, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -72,6 +72,7 @@ var readOnlyLeadPattern = regexp.MustCompile(
|
||||
`grep|egrep|fgrep|rg|wc|sort|uniq|cut|tr|tee|find|tree|locate|` +
|
||||
`dpkg\s+(-l|-s|--list|--status)\b|apt\s+(list|search|show)\b|` +
|
||||
`systemctl\s+(status|is-active|is-enabled|is-failed|list-units|list-unit-files|list-timers|show)\b|` +
|
||||
`ethtool|lsmod|lspci|modinfo|dkms|` +
|
||||
`timedatectl|hostnamectl|systemd-analyze|` +
|
||||
`docker\s+(ps|images|inspect|logs|version|info|stats)|` +
|
||||
`docker\s+compose\s+(logs|ps|top|config|images|port|cp)\b|` +
|
||||
@@ -105,6 +106,15 @@ var curlLeadRe = regexp.MustCompile(`(?i)^curl\b`)
|
||||
// When any of these appears, the curl command is no longer read-only.
|
||||
var curlMutateRe = regexp.MustCompile(`(?i)(?:^|\s)-X\s+(?:post|put|delete|patch|connect|trace)\b|(?:^|\s)-(?:d|F|T|o)\b|(?:^|\s)--(?:data[-a-z]*|request|form|upload-file|output)\b`)
|
||||
|
||||
// curlDevNullOutRe matches curl output redirected to /dev/null in any of curl's
|
||||
// argument forms (space, =, or attached). /dev/null is a no-op sink, so a GET
|
||||
// that discards its body — the canonical reachability idiom
|
||||
// `curl -o /dev/null -w '%{http_code}' URL` — is read-only. Output to any real
|
||||
// path (-o /tmp/x) stays a potential mutation. Stripped before curlMutateRe so
|
||||
// the remaining flags (-X, -d, ...) still classify correctly: a
|
||||
// `curl -o /dev/null -X POST` stays config_mutation.
|
||||
var curlDevNullOutRe = regexp.MustCompile(`(?i)(^|\s)-o\s*/dev/null(\s|$)|(^|\s)--output[=\s]\s*/dev/null(\s|$)`)
|
||||
|
||||
// redirectOutRe matches shell output redirection to a file (> or >> followed
|
||||
// by a path), but excludes the file-descriptor merge form `>&<digit>` (e.g.
|
||||
// `2>&1`) which only rearranges streams and writes nothing to disk. RE2 has
|
||||
@@ -308,6 +318,10 @@ func curlIsReadOnly(curlCmd string) bool {
|
||||
if !curlLeadRe.MatchString(curlCmd) {
|
||||
return false
|
||||
}
|
||||
// -o /dev/null is a no-op sink: strip it before flag detection so the
|
||||
// canonical GET-and-discard reachability probe stays read-only.
|
||||
// A `curl -o /dev/null -X POST` still fails curlMutateRe after stripping.
|
||||
curlCmd = curlDevNullOutRe.ReplaceAllString(curlCmd, " ")
|
||||
if curlMutateRe.MatchString(curlCmd) {
|
||||
return false
|
||||
}
|
||||
|
||||
@@ -46,6 +46,12 @@ func TestClassifyCommand_ReadOnly(t *testing.T) {
|
||||
"sudo pct exec 121 -- systemctl status caddy",
|
||||
// qm guest exec on a VM, read-only inner.
|
||||
"qm guest exec 100 -- systemctl status caddy",
|
||||
// Hardware/driver diagnostic commands (F2 fix — 2026-08-12).
|
||||
"ethtool -i eno1",
|
||||
"lsmod",
|
||||
"lspci",
|
||||
"modinfo r8125",
|
||||
"dkms status",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got != RiskReadOnly {
|
||||
@@ -102,6 +108,33 @@ func TestClassifyCommand_CurlPipeSh_ConfigMutation(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyCommand_CurlDevNull_ReadOnly(t *testing.T) {
|
||||
// -o /dev/null is a no-op sink — the canonical GET-and-discard
|
||||
// reachability idiom must stay read_only. Output to real paths stays
|
||||
// config_mutation. POST/data flags after stripping still gate.
|
||||
cases := []struct {
|
||||
cmd string
|
||||
cls string
|
||||
}{
|
||||
// read_only: GET with body discarded to /dev/null
|
||||
{`curl -o /dev/null -w '%{http_code}' --connect-timeout 10 http://192.168.8.101:8123`, RiskReadOnly},
|
||||
{`curl -sS -o /dev/null https://home.hubris.network`, RiskReadOnly},
|
||||
{`curl --output /dev/null https://example.com`, RiskReadOnly},
|
||||
{`curl -o /dev/null https://example.com`, RiskReadOnly},
|
||||
{`curl -o/dev/null -w '%{http_code}' https://example.com`, RiskReadOnly},
|
||||
// config_mutation: POST/data still caught after stripping devnull
|
||||
{`curl -o /dev/null -X POST https://example.com`, RiskConfigMutation},
|
||||
{`curl -o /dev/null -d '{"x":1}' https://example.com`, RiskConfigMutation},
|
||||
// config_mutation: -o to real path stays config_mutation
|
||||
{`curl -o /etc/caddy/Caddyfile http://example.com`, RiskConfigMutation},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c.cmd, ""); got != c.cls {
|
||||
t.Errorf("ClassifyCommand(%q) = %q, want %q", c.cmd, got, c.cls)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyCommand_DefaultEscalatesToConfigMutation(t *testing.T) {
|
||||
cases := []string{
|
||||
"apt-get install -y nginx",
|
||||
@@ -144,6 +177,9 @@ func TestClassifyCommand_CompoundReadOnly(t *testing.T) {
|
||||
// P4: the exact compound from session d0d562e0 — find + ls + tail +
|
||||
// echo + journalctl, all read-only segments.
|
||||
"ls -lt /var/log/rclone-backup/ | head -20 && tail -3 /var/log/rclone-backup/runs.jsonl || echo \"not found\" && find /var/log/rclone-backup/ -name 'runs.jsonl'",
|
||||
// F2 fix: the exact compound diagnostic that was misclassified as
|
||||
// config_mutation (2026-08-12 hubris NIC driver cutover session).
|
||||
"uname -r && echo '---' && ethtool -i eno1 && echo '---' && lsmod | grep r8169 && echo '---' && ip link show eno1 && echo '---' && cat /etc/network/interfaces | head -30",
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ClassifyCommand(c, ""); got != RiskReadOnly {
|
||||
|
||||
95
internal/policy/risk_test.go
Normal file
95
internal/policy/risk_test.go
Normal file
@@ -0,0 +1,95 @@
|
||||
package policy
|
||||
|
||||
import "testing"
|
||||
|
||||
// The escalation ladder is the load-bearing invariant of the policy layer:
|
||||
// computed risk may only escalate, never de-escalate, against the caller's
|
||||
// declaration. These pin the rank order and the unknown-input defaults that
|
||||
// ClassifyCommand relies on (riskRank/normalizeRisk were only 66% covered).
|
||||
func TestRiskRankOrder(t *testing.T) {
|
||||
cases := []struct {
|
||||
a, b string
|
||||
want bool // want riskRank(a) < riskRank(b)
|
||||
}{
|
||||
{RiskReadOnly, RiskReversibleLow, true},
|
||||
{RiskReversibleLow, RiskConfigMutation, true},
|
||||
{RiskConfigMutation, RiskDestructive, true},
|
||||
{RiskReadOnly, RiskDestructive, true},
|
||||
{RiskDestructive, RiskReadOnly, false},
|
||||
{RiskConfigMutation, RiskConfigMutation, false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := riskRank(c.a) < riskRank(c.b); got != c.want {
|
||||
t.Errorf("riskRank(%q) < riskRank(%q) = %v, want %v", c.a, c.b, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRiskRankUnknownDefaultsToConfigMutation(t *testing.T) {
|
||||
// An unrecognized declared risk is treated as config_mutation — the
|
||||
// safer-to-gate default — not as the lowest tier.
|
||||
if r := riskRank("totally_made_up"); r != riskRank(RiskConfigMutation) {
|
||||
t.Errorf("riskRank(unknown) = %d, want %d (config_mutation)", r, riskRank(RiskConfigMutation))
|
||||
}
|
||||
// It therefore outranks read_only and reversible_low...
|
||||
if riskRank("made_up") <= riskRank(RiskReadOnly) {
|
||||
t.Error("unknown risk should outrank read_only")
|
||||
}
|
||||
if riskRank("made_up") <= riskRank(RiskReversibleLow) {
|
||||
t.Error("unknown risk should outrank reversible_low")
|
||||
}
|
||||
// ...but never outranks destructive.
|
||||
if riskRank("made_up") >= riskRank(RiskDestructive) {
|
||||
t.Error("unknown risk must not outrank destructive")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeRisk(t *testing.T) {
|
||||
cases := []struct {
|
||||
in string
|
||||
want string
|
||||
}{
|
||||
{RiskReadOnly, RiskReadOnly},
|
||||
{RiskReversibleLow, RiskReversibleLow},
|
||||
{RiskConfigMutation, RiskConfigMutation},
|
||||
{RiskDestructive, RiskDestructive},
|
||||
// Unknown / empty / malformed declared risks collapse to the gated
|
||||
// default rather than the most-permissive tier.
|
||||
{"", RiskConfigMutation},
|
||||
{"bogus", RiskConfigMutation},
|
||||
{"READ_ONLY", RiskConfigMutation}, // case-sensitive: not normalized
|
||||
{"read-only", RiskConfigMutation}, // hyphen, not underscore
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := normalizeRisk(c.in); got != c.want {
|
||||
t.Errorf("normalizeRisk(%q) = %q, want %q", c.in, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Escalation property: ClassifyCommand returns max(rank(computed), rank(declared)).
|
||||
// Over a read-only command (computed rank 0) the declared risk passes through
|
||||
// (undeclared → read_only; bogus → config_mutation); over a destructive command
|
||||
// (computed rank 3) the result is always destructive.
|
||||
func TestClassifyCommandEscalationIsMaxOfRanks(t *testing.T) {
|
||||
readOnlyExpected := []struct {
|
||||
declared, want string
|
||||
}{
|
||||
{"", RiskReadOnly},
|
||||
{RiskReadOnly, RiskReadOnly},
|
||||
{RiskReversibleLow, RiskReversibleLow},
|
||||
{RiskConfigMutation, RiskConfigMutation},
|
||||
{RiskDestructive, RiskDestructive},
|
||||
{"bogus", RiskConfigMutation}, // unknown declared → config_mutation rank
|
||||
}
|
||||
for _, c := range readOnlyExpected {
|
||||
if got := ClassifyCommand("uptime", c.declared); got != c.want {
|
||||
t.Errorf("read-only cmd + declared %q = %q, want %q", c.declared, got, c.want)
|
||||
}
|
||||
}
|
||||
for _, d := range []string{"", RiskReadOnly, RiskReversibleLow, RiskConfigMutation, RiskDestructive, "bogus"} {
|
||||
if got := ClassifyCommand("rm -rf /var/lib/x", d); got != RiskDestructive {
|
||||
t.Errorf("destructive cmd + declared %q = %q, want destructive", d, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -7,11 +7,11 @@ import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"regexp"
|
||||
"runtime"
|
||||
@@ -19,9 +19,11 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/dtoro/oikos/internal/actuator"
|
||||
"github.com/dtoro/oikos/internal/config"
|
||||
"github.com/dtoro/oikos/internal/db"
|
||||
"github.com/dtoro/oikos/internal/db/sqlcgen"
|
||||
"github.com/dtoro/oikos/internal/health"
|
||||
"github.com/dtoro/oikos/internal/observability"
|
||||
"github.com/dtoro/oikos/internal/remote"
|
||||
"github.com/google/uuid"
|
||||
@@ -32,8 +34,13 @@ import (
|
||||
var (
|
||||
sshKeyPath string
|
||||
sshUser string
|
||||
sshPool *actuator.DialPool
|
||||
)
|
||||
|
||||
// schedulerLockKey is the advisory-lock key preventing duplicate scheduler
|
||||
// instances. Must differ from db.migrationLockKey (0x01c05e5).
|
||||
const schedulerLockKey = 0x01c05e6
|
||||
|
||||
// Run starts the scheduler loop. Blocks until ctx is cancelled.
|
||||
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
|
||||
slog.Info("scheduler: starting", "interval", cfg.SchedulerInterval)
|
||||
@@ -47,12 +54,49 @@ func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
|
||||
if sshUser == "" {
|
||||
sshUser = "root"
|
||||
}
|
||||
sshPool = actuator.NewDialPool(5 * time.Minute)
|
||||
defer sshPool.Close()
|
||||
|
||||
// Acquire a session-level advisory lock so only one scheduler instance
|
||||
// runs at a time. If another instance holds the lock, we exit — duplicate
|
||||
// schedulers would duplicate health checks, signals, metrics, and events.
|
||||
lockConn, err := pool.Acquire(ctx)
|
||||
if err != nil {
|
||||
slog.Error("scheduler: acquire connection for lock", "error", err)
|
||||
return
|
||||
}
|
||||
var locked bool
|
||||
if err := lockConn.QueryRow(ctx, "SELECT pg_try_advisory_lock($1)", schedulerLockKey).Scan(&locked); err != nil {
|
||||
lockConn.Release()
|
||||
slog.Error("scheduler: advisory lock error", "error", err)
|
||||
return
|
||||
}
|
||||
if !locked {
|
||||
lockConn.Release()
|
||||
slog.Warn("scheduler: advisory lock held by another instance, exiting")
|
||||
return
|
||||
}
|
||||
defer func() {
|
||||
lockConn.Exec(context.WithoutCancel(ctx), "SELECT pg_advisory_unlock($1)", schedulerLockKey)
|
||||
lockConn.Release()
|
||||
}()
|
||||
|
||||
// Liveness probe (plan D5): staleness is 3x the interval so a single
|
||||
// slow check pass (one host hung on SSH) doesn't flap the container
|
||||
// unhealthy before the next scheduled tick.
|
||||
stale := 3 * interval
|
||||
if stale < 90*time.Second {
|
||||
stale = 90 * time.Second
|
||||
}
|
||||
probe := health.New(stale)
|
||||
probe.Serve(ctx, cfg.HealthListen)
|
||||
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
// Immediate first pass
|
||||
runCheckPass(ctx, pool)
|
||||
probe.Bump()
|
||||
|
||||
for {
|
||||
select {
|
||||
@@ -61,6 +105,7 @@ func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
|
||||
return
|
||||
case <-ticker.C:
|
||||
runCheckPass(ctx, pool)
|
||||
probe.Bump()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -299,6 +344,8 @@ func executeCheck(ctx context.Context, pool *db.Pool, cd sqlcgen.ListEnabledChec
|
||||
return checkSSHScript(ctx, pool, cd)
|
||||
case "backup-freshness":
|
||||
return checkBackupFreshness(ctx, cd)
|
||||
case "dns":
|
||||
return checkDNS(ctx, cd)
|
||||
default:
|
||||
return checkResult{health: "unknown"}
|
||||
}
|
||||
@@ -484,6 +531,53 @@ func checkTCP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResu
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
|
||||
// checkDNS verifies a DNS name resolves, catching a stale or unreachable
|
||||
// zone. It looks up NS records first (a zone always has NS), falling back to
|
||||
// an A/AAAA lookup for hostnames. Uses the system resolver; for split-horizon
|
||||
// correctness reserve an explicit `server` in the config.
|
||||
func checkDNS(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
Name string `json:"name"`
|
||||
Server string `json:"server"`
|
||||
}{}
|
||||
if len(cd.Config) > 0 {
|
||||
_ = json.Unmarshal(cd.Config, &cfg)
|
||||
}
|
||||
if cfg.Name == "" {
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
|
||||
// Resolve via an explicit server when supplied (split-horizon), else the
|
||||
// system default resolver.
|
||||
lookup := func(q string) (int, error) {
|
||||
r := &net.Resolver{}
|
||||
if cfg.Server != "" {
|
||||
r = &net.Resolver{PreferGo: true, Dial: func(ctx context.Context, network, _ string) (net.Conn, error) {
|
||||
d := net.Dialer{Timeout: 5 * time.Second}
|
||||
return d.DialContext(ctx, network, net.JoinHostPort(cfg.Server, "53"))
|
||||
}}
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
|
||||
defer cancel()
|
||||
ns, err := r.LookupNS(ctx, q)
|
||||
if err == nil && len(ns) > 0 {
|
||||
return len(ns), nil
|
||||
}
|
||||
addrs, err2 := r.LookupHost(ctx, q)
|
||||
return len(addrs), err2
|
||||
}
|
||||
|
||||
n, err := lookup(cfg.Name)
|
||||
if err != nil || n == 0 {
|
||||
return checkResult{
|
||||
health: "down", signalKind: "dns",
|
||||
evidence: fmt.Sprintf("DNS resolution failed for %q: %v", cfg.Name, err),
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
return checkResult{health: "healthy"}
|
||||
}
|
||||
|
||||
// checkDisk performs a disk usage check.
|
||||
func checkDisk(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
@@ -537,7 +631,7 @@ func checkDisk(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkRes
|
||||
// checkCertExpiry checks TLS certificate expiry.
|
||||
func checkCertExpiry(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
|
||||
cfg := struct {
|
||||
Host string `json:"host"`
|
||||
Host string `json:"host"`
|
||||
// Dial is an optional explicit dial address (the TLS terminator's IP)
|
||||
// for when the hostname doesn't resolve/reach from the scheduler — the
|
||||
// container has no mesh interface and the host resolver doesn't know
|
||||
@@ -919,28 +1013,45 @@ func allowlistedScript(name string) bool {
|
||||
return scriptNameRe.MatchString(name)
|
||||
}
|
||||
|
||||
// sshExec runs a command on a remote host over crypto/ssh via the shared
|
||||
// actuator primitives. It replaced a fork of `os/exec ssh` so the scheduler,
|
||||
// the MCP execution path, and the actuator share one dial/run/host-key
|
||||
// implementation (plan E3). The host key is verified through the centralized
|
||||
// actuator.HostKeyCallback seam. ctx bounds the running command; timeout
|
||||
// bounds the dial.
|
||||
func sshExec(ctx context.Context, host, port, user, cmd string, timeout time.Duration) ([]byte, error) {
|
||||
args := []string{
|
||||
"-o", "ConnectTimeout=" + strconv.Itoa(int(timeout.Seconds())),
|
||||
"-o", "StrictHostKeyChecking=no",
|
||||
"-o", "BatchMode=yes",
|
||||
"-o", "UserKnownHostsFile=/dev/null",
|
||||
"-o", "LogLevel=ERROR",
|
||||
}
|
||||
if sshKeyPath != "" {
|
||||
args = append(args, "-i", sshKeyPath)
|
||||
}
|
||||
if port != "" && port != "22" {
|
||||
args = append(args, "-p", port)
|
||||
}
|
||||
args = append(args, "-l", user, host, cmd)
|
||||
c := exec.CommandContext(ctx, "ssh", args...)
|
||||
out, err := c.Output()
|
||||
if err != nil {
|
||||
var ee *exec.ExitError
|
||||
if errors.As(err, &ee) {
|
||||
return nil, fmt.Errorf("ssh %s: %v (stderr: %s)", host, err, string(ee.Stderr))
|
||||
keyPath := sshKeyPath
|
||||
if keyPath == "" {
|
||||
// Preserve the old os/exec-ssh behavior of deferring to a default
|
||||
// key when no explicit OIKOS_SSH_KEY_PATH is configured: the system
|
||||
// ssh binary used the agent / ~/.ssh; crypto/ssh has no agent wiring,
|
||||
// so fall back to SSH_KEY_PATH then ~/.ssh/id_rsa.
|
||||
keyPath = os.Getenv("SSH_KEY_PATH")
|
||||
if keyPath == "" {
|
||||
keyPath = os.Getenv("HOME") + "/.ssh/id_rsa"
|
||||
}
|
||||
}
|
||||
signer, err := actuator.LoadSigner(keyPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("ssh %s: %v", host, err)
|
||||
}
|
||||
p := 22
|
||||
if port != "" {
|
||||
if n, parseErr := strconv.Atoi(port); parseErr == nil && n > 0 {
|
||||
p = n
|
||||
}
|
||||
}
|
||||
client, err := sshPool.Get(ctx, actuator.DialOptions{
|
||||
Host: host, Port: p, User: user, Signer: signer, Timeout: timeout,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("ssh %s: %v", host, err)
|
||||
}
|
||||
// RunOutput (stdout-only) — the scheduler parses check output as JSON or
|
||||
// matches it literally, so stderr must not be merged in (RunCombinedOutput
|
||||
// is for the live-run display path in mcp/httpapi).
|
||||
out, err := actuator.RunOutput(ctx, client, cmd)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("ssh %s: %v", host, err)
|
||||
}
|
||||
return out, nil
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
// Package secrets abstracts secret retrieval across backends (SOPS, Infisical).
|
||||
// Phase 5: SOPS → Infisical migration with SOPS DR fallback.
|
||||
package secrets
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
@@ -14,24 +13,23 @@ var ErrBackendUnavailable = errors.New("secret backend unavailable")
|
||||
|
||||
// Backend is the interface for retrieving and storing secrets.
|
||||
type Backend interface {
|
||||
// Get retrieves a secret value by path/key.
|
||||
Get(ctx context.Context, key string) (string, error)
|
||||
// List returns all secret keys available in this backend.
|
||||
List(ctx context.Context) ([]string, error)
|
||||
// Set stores a secret value. Used during migration.
|
||||
Set(ctx context.Context, key string, value string) error
|
||||
// Name returns a human-readable backend identifier.
|
||||
Name() string
|
||||
}
|
||||
|
||||
// Manager holds a primary and fallback backend. If the primary fails,
|
||||
// it falls back to the secondary.
|
||||
// it falls back to the secondary. Supports periodic background refresh
|
||||
// of cached secrets from the primary backend.
|
||||
type Manager struct {
|
||||
primary Backend
|
||||
fallback Backend
|
||||
cache map[string]cachedSecret
|
||||
mu sync.RWMutex
|
||||
cacheTTL time.Duration
|
||||
primary Backend
|
||||
fallback Backend
|
||||
cache map[string]cachedSecret
|
||||
mu sync.RWMutex
|
||||
cacheTTL time.Duration
|
||||
refreshMu sync.Mutex
|
||||
lastRefresh time.Time
|
||||
}
|
||||
|
||||
type cachedSecret struct {
|
||||
@@ -39,21 +37,97 @@ type cachedSecret struct {
|
||||
expiresAt time.Time
|
||||
}
|
||||
|
||||
// ManagerRefreshInterval controls how often cached secrets are re-fetched
|
||||
// from the primary backend in the background. Zero disables background refresh.
|
||||
var ManagerRefreshInterval = 5 * time.Minute
|
||||
|
||||
// NewManager creates a secret manager with primary and fallback backends.
|
||||
func NewManager(primary, fallback Backend) *Manager {
|
||||
return &Manager{
|
||||
primary: primary,
|
||||
fallback: fallback,
|
||||
cache: make(map[string]cachedSecret),
|
||||
cacheTTL: 5 * time.Minute,
|
||||
primary: primary,
|
||||
fallback: fallback,
|
||||
cache: make(map[string]cachedSecret),
|
||||
cacheTTL: 5 * time.Minute,
|
||||
lastRefresh: time.Now(),
|
||||
}
|
||||
}
|
||||
|
||||
// StartRefreshLoop starts a background goroutine that periodically refreshes
|
||||
// cached secrets from the primary backend. Call from the server's main
|
||||
// goroutine. The loop runs until ctx is cancelled.
|
||||
func (m *Manager) StartRefreshLoop(ctx context.Context) {
|
||||
if ManagerRefreshInterval <= 0 {
|
||||
return
|
||||
}
|
||||
slog.Info("secrets: background refresh loop started",
|
||||
"interval", ManagerRefreshInterval)
|
||||
ticker := time.NewTicker(ManagerRefreshInterval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
slog.Info("secrets: refresh loop stopped")
|
||||
return
|
||||
case <-ticker.C:
|
||||
m.refreshAll(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// refreshAll re-fetches all cached secrets from the primary backend.
|
||||
// Keys not found in the primary are left cached (they may be in fallback).
|
||||
// Logs a summary line.
|
||||
func (m *Manager) refreshAll(ctx context.Context) {
|
||||
m.refreshMu.Lock()
|
||||
defer m.refreshMu.Unlock()
|
||||
|
||||
if m.primary == nil {
|
||||
return
|
||||
}
|
||||
|
||||
m.mu.RLock()
|
||||
keys := make([]string, 0, len(m.cache))
|
||||
for k := range m.cache {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
m.mu.RUnlock()
|
||||
|
||||
if len(keys) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
refreshed, stale, failed := 0, 0, 0
|
||||
for _, key := range keys {
|
||||
val, err := m.primary.Get(ctx, key)
|
||||
if err != nil {
|
||||
failed++
|
||||
continue
|
||||
}
|
||||
m.mu.RLock()
|
||||
cached, ok := m.cache[key]
|
||||
m.mu.RUnlock()
|
||||
if !ok || val != cached.value {
|
||||
m.mu.Lock()
|
||||
m.cache[key] = cachedSecret{value: val, expiresAt: time.Now().Add(m.cacheTTL)}
|
||||
m.mu.Unlock()
|
||||
refreshed++
|
||||
} else {
|
||||
stale++
|
||||
}
|
||||
}
|
||||
|
||||
m.lastRefresh = time.Now()
|
||||
slog.Info("secrets: background refresh complete",
|
||||
"refreshed", refreshed, "stale", stale, "failed", failed,
|
||||
"cached", len(keys))
|
||||
}
|
||||
|
||||
// Get retrieves a secret from primary, falling back to secondary on error.
|
||||
func (m *Manager) Get(ctx context.Context, key string) (string, error) {
|
||||
m.mu.RLock()
|
||||
if cached, ok := m.cache[key]; ok && time.Now().Before(cached.expiresAt) {
|
||||
m.mu.RUnlock()
|
||||
slog.Debug("secret: cache hit", "key", key)
|
||||
return cached.value, nil
|
||||
}
|
||||
m.mu.RUnlock()
|
||||
@@ -63,16 +137,20 @@ func (m *Manager) Get(ctx context.Context, key string) (string, error) {
|
||||
m.mu.Lock()
|
||||
m.cache[key] = cachedSecret{value: val, expiresAt: time.Now().Add(m.cacheTTL)}
|
||||
m.mu.Unlock()
|
||||
slog.Debug("secret: fetched from primary", "key", key)
|
||||
return val, nil
|
||||
}
|
||||
|
||||
if m.fallback != nil {
|
||||
val, fallbackErr := m.fallback.Get(ctx, key)
|
||||
if fallbackErr == nil {
|
||||
slog.Warn("secret: primary failed, using fallback",
|
||||
"key", key, "primary_error", err)
|
||||
return val, nil
|
||||
}
|
||||
}
|
||||
|
||||
slog.Warn("secret: not found in any backend", "key", key, "error", err)
|
||||
return "", err
|
||||
}
|
||||
|
||||
@@ -85,11 +163,17 @@ func (m *Manager) List(ctx context.Context) ([]string, error) {
|
||||
return keys, err
|
||||
}
|
||||
|
||||
// Set stores a secret in the primary backend (used during migration).
|
||||
// Set stores a secret in the primary backend and invalidates the cache.
|
||||
func (m *Manager) Set(ctx context.Context, key string, value string) error {
|
||||
m.InvalidateCache()
|
||||
return m.primary.Set(ctx, key, value)
|
||||
}
|
||||
|
||||
// Name returns the primary backend name.
|
||||
func (m *Manager) Name() string {
|
||||
return m.primary.Name()
|
||||
}
|
||||
|
||||
// PrimaryName returns the name of the primary backend.
|
||||
func (m *Manager) PrimaryName() string {
|
||||
return m.primary.Name()
|
||||
@@ -101,3 +185,14 @@ func (m *Manager) InvalidateCache() {
|
||||
m.cache = make(map[string]cachedSecret)
|
||||
m.mu.Unlock()
|
||||
}
|
||||
|
||||
// LastRefresh returns the timestamp of the last background refresh.
|
||||
func (m *Manager) LastRefresh() time.Time {
|
||||
return m.lastRefresh
|
||||
}
|
||||
|
||||
// secretOverlay maps an Infisical key to a config setter function.
|
||||
type secretOverlay struct {
|
||||
infisicalKey string
|
||||
apply func(value string)
|
||||
}
|
||||
|
||||
118
internal/secrets/config_overlay.go
Normal file
118
internal/secrets/config_overlay.go
Normal file
@@ -0,0 +1,118 @@
|
||||
package secrets
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
)
|
||||
|
||||
// NewManagerFromConfig creates a secrets Manager from the Infisical connection
|
||||
// parameters in cfg, with an optional SOPS fallback from cfg.SecretsDir.
|
||||
// Returns nil if Infisical is not configured.
|
||||
func NewManagerFromConfig(siteURL, clientID, clientSecret, projectID, env, secretsDir string) *Manager {
|
||||
if siteURL == "" {
|
||||
return nil
|
||||
}
|
||||
infCfg := InfisicalConfig{
|
||||
SiteURL: siteURL,
|
||||
ClientID: clientID,
|
||||
ClientSecret: clientSecret,
|
||||
ProjectID: projectID,
|
||||
SecretPath: "/",
|
||||
Env: env,
|
||||
}
|
||||
if infCfg.Env == "" {
|
||||
infCfg.Env = "dev"
|
||||
}
|
||||
primary := NewInfisicalBackend(infCfg)
|
||||
var fallback Backend
|
||||
if secretsDir != "" {
|
||||
fallback = NewSOPSBackend(secretsDir)
|
||||
}
|
||||
return NewManager(primary, fallback)
|
||||
}
|
||||
|
||||
// VerifyExpectedSecrets checks that a list of expected keys are present
|
||||
// in the backend. Logs a summary and returns the count of missing keys.
|
||||
// Use at startup to detect incomplete Infisical migration.
|
||||
func VerifyExpectedSecrets(ctx context.Context, sec Backend, expected []string) int {
|
||||
keys, err := sec.List(ctx)
|
||||
if err != nil {
|
||||
slog.Warn("secrets: cannot verify expected secrets, list failed", "error", err)
|
||||
return len(expected)
|
||||
}
|
||||
keySet := make(map[string]struct{}, len(keys))
|
||||
for _, k := range keys {
|
||||
keySet[k] = struct{}{}
|
||||
}
|
||||
missing := 0
|
||||
for _, exp := range expected {
|
||||
if _, ok := keySet[exp]; !ok {
|
||||
missing++
|
||||
slog.Warn("secrets: expected key missing from Infisical", "key", exp)
|
||||
}
|
||||
}
|
||||
if missing == 0 {
|
||||
slog.Info("secrets: all expected keys present", "count", len(expected))
|
||||
} else {
|
||||
slog.Warn("secrets: some expected keys missing from Infisical",
|
||||
"missing", missing, "total", len(expected))
|
||||
}
|
||||
return missing
|
||||
}
|
||||
|
||||
// OverlayConfig fetches secrets from the backend and returns a function that
|
||||
// applies them to config fields. Each entry maps an Infisical key to a setter;
|
||||
// if the key is found and non-empty, the setter is called; if not found or
|
||||
// empty, the env-derived value is left unchanged and a warning is logged.
|
||||
// Returns the number of secrets resolved from Infisical (useful for logging).
|
||||
func OverlayConfig(ctx context.Context, sec Backend, overlays []secretOverlay) int {
|
||||
resolved := 0
|
||||
for _, o := range overlays {
|
||||
val, err := sec.Get(ctx, o.infisicalKey)
|
||||
if err != nil {
|
||||
slog.Warn("secret not resolved from Infisical, using env fallback",
|
||||
"key", o.infisicalKey, "error", err)
|
||||
continue
|
||||
}
|
||||
if val == "" {
|
||||
slog.Warn("Infisical returned empty value, keeping env-derived value",
|
||||
"key", o.infisicalKey)
|
||||
continue
|
||||
}
|
||||
o.apply(val)
|
||||
resolved++
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
|
||||
// ConfigOverlays returns the standard set of Infisical → config overlays for
|
||||
// the oikos binary. Each overlay is attempted at startup; if the key exists
|
||||
// in Infisical, it overrides the env-derived value.
|
||||
func ConfigOverlays(cfg map[string]func(string)) []secretOverlay {
|
||||
overlays := make([]secretOverlay, 0, len(cfg))
|
||||
for key, setter := range cfg {
|
||||
overlays = append(overlays, secretOverlay{infisicalKey: key, apply: setter})
|
||||
}
|
||||
return overlays
|
||||
}
|
||||
|
||||
// ResolveSecret attempts to fetch a single secret from the backend. Returns
|
||||
// the Infisical value if found and non-empty, otherwise falls back to the
|
||||
// env-derived value. Warnings are logged for failures.
|
||||
func ResolveSecret(ctx context.Context, sec Backend, infisicalKey, fallback string) string {
|
||||
if sec == nil {
|
||||
return fallback
|
||||
}
|
||||
val, err := sec.Get(ctx, infisicalKey)
|
||||
if err != nil {
|
||||
slog.Warn("secret not resolved from Infisical, using env fallback",
|
||||
"key", infisicalKey, "error", err)
|
||||
return fallback
|
||||
}
|
||||
if val == "" {
|
||||
slog.Warn("Infisical returned empty value, keeping env-derived value",
|
||||
"key", infisicalKey)
|
||||
return fallback
|
||||
}
|
||||
return val
|
||||
}
|
||||
@@ -3,7 +3,6 @@ package secrets
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
@@ -31,14 +30,13 @@ type InfisicalConfig struct {
|
||||
// NewInfisicalBackend creates an Infisical backend. Connects lazily on first Get.
|
||||
func NewInfisicalBackend(cfg InfisicalConfig) *InfisicalBackend {
|
||||
autoRefresh := true
|
||||
cacheExpiry := 300 // 5 min cache
|
||||
|
||||
return &InfisicalBackend{
|
||||
cfg: cfg,
|
||||
client: infisical.NewInfisicalClient(context.Background(), infisical.Config{
|
||||
SiteUrl: cfg.SiteURL,
|
||||
AutoTokenRefresh: &autoRefresh,
|
||||
CacheExpiryInSeconds: cacheExpiry,
|
||||
CacheExpiryInSeconds: 0, // no caching — live reads over localhost
|
||||
}),
|
||||
}
|
||||
}
|
||||
@@ -55,14 +53,8 @@ func (b *InfisicalBackend) connect() error {
|
||||
|
||||
clientID := b.cfg.ClientID
|
||||
clientSecret := b.cfg.ClientSecret
|
||||
if clientID == "" {
|
||||
clientID = os.Getenv("INFISICAL_CLIENT_ID")
|
||||
}
|
||||
if clientSecret == "" {
|
||||
clientSecret = os.Getenv("INFISICAL_CLIENT_SECRET")
|
||||
}
|
||||
if clientID == "" || clientSecret == "" {
|
||||
return fmt.Errorf("%w: INFISICAL_CLIENT_ID and INFISICAL_CLIENT_SECRET not set", ErrBackendUnavailable)
|
||||
return fmt.Errorf("%w: OIKOS_INFISICAL_CLIENT_ID and OIKOS_INFISICAL_CLIENT_SECRET not set", ErrBackendUnavailable)
|
||||
}
|
||||
|
||||
_, err := b.client.Auth().UniversalAuthLogin(clientID, clientSecret)
|
||||
@@ -130,22 +122,24 @@ func (b *InfisicalBackend) Set(ctx context.Context, key string, value string) er
|
||||
return err
|
||||
}
|
||||
|
||||
// Try update first, fall back to create
|
||||
_, err := b.client.Secrets().Update(infisical.UpdateSecretOptions{
|
||||
SecretKey: key,
|
||||
NewSecretValue: value,
|
||||
Environment: b.cfg.Env,
|
||||
SecretPath: b.cfg.SecretPath,
|
||||
ProjectID: b.cfg.ProjectID,
|
||||
// Try create first (idempotent — upserts); fall back to update on conflict.
|
||||
_, err := b.client.Secrets().Create(infisical.CreateSecretOptions{
|
||||
SecretKey: key,
|
||||
SecretValue: value,
|
||||
Environment: b.cfg.Env,
|
||||
SecretPath: b.cfg.SecretPath,
|
||||
ProjectID: b.cfg.ProjectID,
|
||||
Type: "shared",
|
||||
})
|
||||
if err != nil {
|
||||
_, err = b.client.Secrets().Create(infisical.CreateSecretOptions{
|
||||
SecretKey: key,
|
||||
SecretValue: value,
|
||||
Environment: b.cfg.Env,
|
||||
SecretPath: b.cfg.SecretPath,
|
||||
ProjectID: b.cfg.ProjectID,
|
||||
Type: "shared",
|
||||
// Create failed (key may already exist) — update instead.
|
||||
_, err = b.client.Secrets().Update(infisical.UpdateSecretOptions{
|
||||
SecretKey: key,
|
||||
NewSecretValue: value,
|
||||
Environment: b.cfg.Env,
|
||||
SecretPath: b.cfg.SecretPath,
|
||||
ProjectID: b.cfg.ProjectID,
|
||||
Type: "shared",
|
||||
})
|
||||
}
|
||||
return err
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user