26 Commits

Author SHA1 Message Date
7160eee1e1 feat: add corosync quorum health check for proxmox-host entities
Some checks are pending
ci / build-test (push) Waiting to run
ci / docker-build (push) Waiting to run
ci / web (push) Waiting to run
Desktop App / Build Linux (amd64) (push) Waiting to run
Desktop App / Attach to Release (push) Blocked by required conditions
Adds pvecm_quorum_check.sh probe script and wires it into the
checkdefaults system as a new 'quorum' monitoring kind on proxmox-host
entities. Runs every 60s via ssh-script, surfaces unhealthy signal when
cluster loses quorum.

Closes the monitoring blind spot that let the 2026-08-12 3.5h corosync
flapping outage go undetected (ping passed, cluster was non-quorate).

Changes:
- seeds/ontology.yaml: proxmox-host declares monitoring: [quorum]
- internal/checkdefaults/defaults.go: KindQuorum builder
- internal/checkdefaults/build_test.go: 2 new test cases
- checks/pvecm_quorum_check.sh: new probe (deployed to hubris + strong)
- VERSION: 0.30.2 -> 0.31.0
2026-08-12 20:18:03 +02:00
30ecdc16c2 fix: bump Infisical image tag and add deploy failure notification
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Two fixes from the deploy pipeline audit:

1. Infisical tag v0.99.1 no longer exists on Docker Hub — bumped to
   v0.162.19 (latest available). This was silently breaking the full
   deploy pipeline (docker compose up failed on image pull).

2. Deploy failures now notify via two channels:
   - Oikos API event (deploy.failed, severity=critical) — picked up by
     the scheduler's notifier for Matrix alert
   - Matrix webhook URL if MATRIX_WEBHOOK_URL is configured
   Uses a trap with _ok flag to catch any non-zero exit path,
   including CI gate rejections and health check timeouts.
   Webhook now resolves and passes OIKOS_API_TOKEN to deploy.sh.
2026-08-12 18:05:54 +02:00
d79b0862bd feat: serve OpenAPI spec at /api/v1/openapi.json
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Registers a handler that serves the embedded OpenAPI 3.0 spec
(compiled into the binary via oapi-codegen) at a browseable
endpoint. Uses gen.GetSwagger() to deserialize the embedded
base64+gzip spec and returns it as JSON.

46 paths, 42 schemas — agents and humans can now introspect the
full API surface without reading Go source.
2026-08-12 17:48:10 +02:00
53823595de fix: add ethtool, lsmod, lspci, modinfo, dkms to read-only command allowlist
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Read-only diagnostic commands ethtool, lsmod, lspci, modinfo, and dkms
were missing from the readOnlyLeadPattern in the command classifier,
causing compound diagnostic commands (e.g. 'uname -r && ethtool -i eno1
&& lsmod | grep r8169') to be misclassified as config_mutation instead
of read_only. This forced operator approval for simple hardware/driver
inspection during the 2026-08-12 hubris NIC cutover session.

Added regression test with the exact compound command from that session.
2026-08-12 13:27:33 +02:00
7ecf720166 feat: entity graph app with theme-aware colors, icons, filters, and blast radius
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
New EntityGraph.svelte app (sigma.js + graphology):
- Theme-aware Gruvbox palettes (light/dark) with reactive color switching
- Lucide icons rendered synchronously via Path2D canvas per entity type
- Color mode toggle (Health / Type) with per-type distinct colors
- Health distribution bar with clickable filters
- Entity type filters grouped by ontological layer (collapsible)
- Relationship type edge filters with color-coded swatches
- Quick presets: All / Problems / Infra
- Node selection with live blast radius from API
- Hover neighborhood highlighting with muted fade
- Isolated node hiding, edge alpha tuning, dot-grid background
- Search with camera focus on highest-degree match
2026-08-11 22:42:20 +02:00
febc153b7f fix: add involves edge from task to agent:nomos at creation
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Plus sync vendor directory for Docker build compatibility.
2026-08-11 22:03:12 +02:00
7d6a3320d4 fix: add involves edge from task to agent:nomos at creation
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
createTaskEntity creates the task entity but never adds any graph
edges. The involves edges are only added by the run handler, so
sessions that call set_goal but never make a run call leave orphan
task entities with zero relationships.

Adds an idempotent involves edge from the new task to agent:nomos
at creation time, matching the same pattern used for run's involves
edges in server.go.
2026-08-11 21:58:18 +02:00
60bc9d555d fix: add precedes graph edge from classification to execution
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Every run call creates a classification entity, but it was never
connected to the execution via a graph edge — only via a DB column
(executions.classification_id). The ontology requires:

    classification —precedes→ execution

Without this edge, all 53 classification entities had zero
relationships, making them invisible to get_relations and
blast-radius analysis.

Adds an idempotent INSERT into relationships after the existing
classification_id update, matching the same pattern used for
targets edges.
2026-08-11 21:18:45 +02:00
ebe1b95acf sync AGENTS.md tool list with MCP server (63 tools); fix 7 stale references in .agents/
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-09 00:04:58 +02:00
5e10437fe3 Phase 4 (Performance) + Phase 6 (Infrastructure) completion
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Phase 4 — Performance:
- F1: SSH DialPool with key-by-host pooling and 5min idle TTL
- F2: In-memory entity lookup cache (TTL 60s, HTTP resolveEntityID)
- F3: Trigram GIN indexes on entities.slug and entities.name (migration 031)
- F4: Partial index on executions(classification_id) for auto-act (migration 032)
- Added missing RunOutput and RunStreaming in actuator/ (E3 gap fill)

Phase 6 — Infrastructure:
- H1: Infisical image pinned to v0.99.1
- H2: execworker daemon — polls pending executions with per-execution
  advisory locks, recovers orphaned running executions, wired as
  docker-compose service
- H3: splitSQL hardened with block comment and string-literal support,
  6 new edge-case tests (11 total)
- H4: Scheduler acquires pg_try_advisory_lock(0x01c05e6) at startup
2026-08-08 23:46:43 +02:00
7236c46e5c 0.29.1 — review-fix round on E3: RunOutput, sshKeyPath fallback, RunStreaming consolidation, signer cache, stderr in errors
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-08 23:01:10 +02:00
75c0848a6f 0.29.0 — code-quality refactor (plan E1–E5): file splits, sqlc migration, SSH unification, test coverage
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
E1: split monolithic files — cmd/nomos (main.go → server.go + mcp.go + workers.go),
    internal/mcp/tools.go → entity_tools/ops_tools/knowledge_tools/analysis_tools,
    internal/httpapi/impl.go → domain files (entities, events, signals, ontology,
    fleet_health, client_context, client_lifecycle, entity_mutations, query_audit).
E2: migrate raw pool.Exec queries to sqlc (entities/relationships queries + generated).
E3: unify SSH — consolidate crypto/ssh dial into actuator/client.go (+client_test).
E4/E5: add tests — db/lifecycle, checkdefaults/build, ontology/preconditions, policy/risk.
2026-08-08 22:47:06 +02:00
712b66422b 0.28.5 — nomos healthcheck fast-path before Infisical init
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
The healthcheck subcommand was reachable only after main()'s Infisical secrets
resolution (4x retries/key, ~30s when Infisical is down), which blew the 5s
Docker healthcheck timeout — so nomos stayed docker-unhealthy despite serving
/healthz fine. Short-circuit 'nomos healthcheck' at the top of main() before
any secrets init; measured 0.58s, no Infisical retries.
2026-08-08 22:17:44 +02:00
a30c024ef8 0.28.4 — nomos healthcheck via binary subcommand (distroless has no wget)
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
The nomos runtime image is gcr.io/distroless/static (no shell/wget), so the
wget-based healthcheck (D5) could never run — nomos showed docker-unhealthy
despite serving /healthz fine. Add a 'nomos healthcheck' subcommand that
self-probes NOMOS_LISTEN/healthz (exit 0 on 200), and point the compose
healthcheck at ["/nomos", "healthcheck"].
2026-08-08 22:09:18 +02:00
137a2afb8d 0.28.3 — widen api healthcheck start_period to 180s
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Measured startup is ~93s: NewHandler stalls on Infisical auth retries (~40s)
and OIDC discovery timeouts to auth.hubris.network (~35s) before binding
:8090. The api IS healthy once bound (serves /healthz); the window just needs
to clear both external-timeout phases so nomos (depends_on: api-healthy) can
start and the deploy completes.
2026-08-08 21:53:16 +02:00
c8b1ec5af2 0.28.2 — widen api healthcheck start_period to 90s
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
The api retries Infisical at startup (4x with backoff) before binding :8090.
When Infisical is unreachable that adds ~60s, and the old start_period (10s)
+ 10 retries (~60s grace) ran out just before the bind — marking the api
unhealthy and failing the deploy (nomos depends_on api-healthy). 90s covers
the slow-startup window; the api genuinely serves /healthz once bound.
2026-08-08 21:48:17 +02:00
8ff382a50d 0.28.1 — vendor @joan/procedural-glyph-engine for portable SPA builds (fixes deploy)
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
The procedural-glyph-engine dep pointed at a non-portable file:/private/tmp/orby-pkg
path, breaking npm ci in Docker and every main deploy since v0.20.0 (the build
cache masked it until it busted ~Aug 5). Vendor Orby v5.0.0 into web/vendor/,
switch the dep to file:../vendor, and use npm install in the web Dockerfile
(file: deps need install, not ci). Cherry-picked from 3cd4cf9.
2026-08-08 21:41:51 +02:00
fa79c1ea25 0.28.0 — operational hardening (plan D1–D5): CI deploy gate, versioned images, rate limiting, resource limits, health probes
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
D1: deploy.sh CI gate — read-only SHA via git ls-remote, Gitea commit-status
    poll, portable mkdir deploy lock (macOS, no flock), TOCTOU guard, token
    passed via curl --config - (not argv), graceful misconfig tolerance.
D2: version-tagged images — OIKOS_VERSION=v$VERSION, keep-last-3 prune derived
    from 'docker compose config --images'; VERSION read after pull.
D3: per-IP rate limiting — new internal/httpapi/ratelimit.go (x/time/rate),
    rightmost-XFF, /healthz exempt, ctx-driven sweep; disabled by default.
D4: mem_limit/cpus on all 10 compose services.
D5: staleness-aware health probes — new internal/health package wired into
    scheduler (:8093) and notifier (:8094); nomos already had :8092.

Two /review passes hardened the deploy lock, TOCTOU guard, token hygiene,
and XFF handling.
2026-08-08 21:31:16 +02:00
ef762794e7 0.27.6 — guard seed-secrets: skip if Infisical already populated
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-08 21:15:00 +02:00
0b6c546aae 0.27.5 — add Infisical env vars to nomos+notifier containers
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-08 21:12:12 +02:00
cecfd8b0e4 0.27.4 — fix: StartRefreshLoop was blocking startup, wrap in goroutine
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-08 21:07:51 +02:00
653ea3a116 0.27.3 — seed-secrets extracts from containers, drop oidc_client-secret (public client)
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-08 20:59:04 +02:00
89a94c24c9 0.27.2 — seed-secrets runs on host, not container
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-06 22:09:02 +02:00
7a34d8c8a0 0.27.1 — flat Infisical keys (_), seed-secrets.sh in deploy, plist cleanup, .env strip
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-06 22:08:02 +02:00
c9d506b0f8 0.27.0 — Infisical hardening: runtime refresh, audit logging, CLI verify/audit, startup verification, SSH host key verification, interface consolidation
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-05 23:51:01 +02:00
e3449b24c1 feat: wire Infisical secret store into API server and MCP tools
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
- Wire secretsManager in NewHandler() — instantiate InfisicalBackend
  when OIKOS_INFISICAL_SITE_URL is set (previously always nil)
- Add get_secret, list_secrets, set_secret MCP tools with nil-backend
  graceful degradation
- Add oikos secret get|set|list CLI subcommands for Infisical
- Fix Set() bug: create-before-update so new keys are created;
  add Type: "shared" to Update so it finds the right secret;
  disable SDK cache so Get returns fresh data after Set
- Clean enrollment response: remove fake infisical_client_id/
  infisical_client_secret stubs, store age key in Infisical for real
2026-08-05 23:03:27 +02:00
3504 changed files with 980579 additions and 4583 deletions

View File

@@ -30,9 +30,9 @@ one pass through **Observe → Orient → Decide → Act**:
- **queue**: informational — console + reports
The classifier can only *lower* autonomy relative to policy, never raise
it. When in doubt, escalate.
4. **Act** — execute through `homelab` commands or runbooks (never ad-hoc
SSH), then **verify** with the action's verification command, write a
**ledger** entry, resolve the Signal, and update docs in the same session.
4. **Act** — execute through MCP `run` or runbooks (never ad-hoc
SSH), then **verify** with the action's verification command, write a
**ledger** entry, resolve the Signal, and update docs in the same session.
## Primitives
@@ -88,7 +88,7 @@ via the API's `/api/v1/graph` endpoint, and the Mermaid export at
## Conventions carried forward
- Inventory is the truth; live state wins over narrative docs.
- Prefer `homelab` CLI and MCP over ad-hoc SSH.
- Prefer MCP tools over ad-hoc SSH.
- Meaningful changes update docs in the same session.
- Secrets are decrypted locally via per-client keys; never into docs/comments.
- Tracked configs change by commit + push, not local edits.
@@ -141,7 +141,7 @@ in the Go binary.
- Standalone Nomos MCP client binary (`cmd/nomos`) with gateway mode
(:8092). Structured queries + natural-language routing to the MCP tool
list (see AGENTS.md §3). Agent activity logging on every tool call. No SSH keys.
- `nomos/` directory with config, SOUL.md, homelab-ops skill.
- `nomos/` directory with config, SOUL.md, `homelab-ops` skill at `nomos/skills/homelab-ops/`.
- Nomos Docker service in `docker-compose.yml` (profile: full).
- Go packages: `cmd/nomos/`, `compose/nomos/`.

View File

@@ -20,6 +20,6 @@ Run from the repo root:
Exit code is non-zero when any violation is found, so it can gate a commit. The banned-vocabulary
list mirrors `writing-style.md`; update both together if the standard changes.
> **Known baseline.** `archive/knowledge/archive/knowledge/containers/101-jellyfin.md` links into a sibling repo
> **Known baseline.** `archive/knowledge/containers/101-jellyfin.md` links into a sibling repo
> (`devops/homelab-authentik-admin`) that this checkout does not contain — expected, not a bug.
> Any other broken link is a real regression; investigate before dismissing it as baseline noise.

View File

@@ -36,8 +36,9 @@ ledger entry.
6. Update the entity's `state` to `destroyed` in `seeds/inventory.yaml`
(or move it to an `archaeology:`-style section if the schema still has
one) — `pve_id`, `destroyed` date, `reason` — then `oikos seed` to
ingest. Add a row to `containers/index.md` "Recently destroyed" table
(kept for human-readable browsing alongside the structured data).
ingest. Add a row to the legacy `archive/knowledge/containers/index.md`
"Recently destroyed" table (kept for human-readable browsing
alongside the structured data in the DB).
7. No manual ledger step — mutations through the API are recorded
automatically in the `audit_log` table (MCP `get_audit_trail`,
`get_change_history`). The old `oikos/ledger.py append` was retired

View File

@@ -27,9 +27,10 @@ chosen, doc page stub.
will self-enroll as a client afterward (see
[CLIENTS.md](../../../CLIENTS.md#enrollment)), the entity must exist in
`planned`/`provisioning` state before `bootstrap.sh` runs there.
3. Stub the doc page (`containers/<pve_id>-<name>.md` or
`vms/<pve_id>-<name>.md`) — even a one-line "provisioning, see plan X"
is enough to satisfy the transition requirement.
3. Stub a document entity via MCP `upsert_knowledge` with
`kind: document` and about set to the new entity slug — even a
one-line "provisioning, see plan X" is enough to satisfy the
transition requirement.
4. Reserve the IP in DNS/DHCP notes if it's a fixed LAN address.
Next: [lifecycle-activate-node.md](../lifecycle-activate-node/SKILL.md).

View File

@@ -78,5 +78,5 @@ Session: {id[:8]} — "{title[:60]}"
- `internal/mcp/server.go` — all tool implementations (`run`, `list_lxcs`, …)
- `web/src/lib/components/ToolCallGroup.svelte` — tool result display
- `nomos/SOUL.md` — agent persona and tool selection rules
- `plans/2026-07-09-chat-sessions-improvements.md` — prior session findings
- `plans/2026-07-09-session-execution-and-ux-fixes.md` — latest plan
- `plans/done/2026-07-09-chat-sessions-improvements.md` — prior session findings
- `plans/done/2026-07-09-session-execution-and-ux-fixes.md` — latest plan

120
AGENTS.md
View File

@@ -56,67 +56,79 @@ Endpoint: `https://mcp.hubris.network/mcp`. Every call needs
enrollment and `/healthz` (see "Authentication" below for where the token
comes from).
Available tools (the authoritative list — count them below if a number is
needed; do not hardcode the count elsewhere):
Available tools (63 total — the authoritative list; do not hardcode the count
elsewhere; regenerate from `internal/mcp/` when tools change):
Context — observe + orient:
get_entity(slug), list_entities(type, limit, cursor),
get_relations(entity), get_blast_radius(entity),
search_knowledge(query) — ILIKE search over documents, investigations,
runbooks in the knowledge_entities table
get_entity_knowledge(entity_slug) — every document, investigation, and
runbook linked to one entity, in one call
get_patterns(status, entity_type, action) — learned action patterns
get_skills(status) — available automation skills
http_get(url) — fetch a public page/raw file (e.g. researching how to
deploy something before provisioning it); HTTP/HTTPS only, ~16KB cap
Management — live state:
get_service_status(service_slug) — systemctl is-active on target host
tail_log(service_slug, lines=200) — journalctl
list_lxcs() — all LXC containers with ID, host, IP, health
get_lxc_state(lxc_slug) — pct status from Proxmox host
ping_service(service_slug) — HTTP reachability from entity_status
list_my_secrets(caller_pubkey) — secrets accessible to this client by
age public key
Oikos — decisions:
Entity Tools — knowledge graph, discovery, and lifecycle:
ping — lightweight connectivity check
get_entity(slug_or_id) — get an entity by slug or UUID
list_entities(type, state, q, limit) — entities filtered by type, state, or search
get_relations(entity_id, types) — list inbound/outbound edges for one entity
get_blast_radius(entity_id, depth=3) — entities affected if this one goes down
create_entity(type, name, slug, attributes, state) — create a new entity in the graph
update_entity_attributes(slug, attributes) — merge discovered facts into an entity
set_entity_state(slug, state) — transition entity to a new lifecycle state
create_relationship(source, target, type) — record a discovered edge
end_relationship(source, target, type) — soft-delete an active edge
whoami(hostname) — entity record, peers, and health for a host
explain(service_slug) — compact context card (type, state, health, relations)
preflight(service_slug, action) — risk class + approval requirement
whoami(hostname) — entity record, peers, health for a client
get_change_history(entity_slug, limit=20) — last audit-log entries per entity
get_state_snapshot() — fleet health, disk, drift count
get_state_snapshot() — last scheduler Observe-pass: fleet health, disk, drift
audit_knowledge_graph() — read-only drift report over the graph and checks
discover_infra_drift() — running guests vs DB: missing/ghost entities
find_entities_by(key, value, limit=25) — search entities by attribute values
Operations — observe + act:
get_health_summary() — fleet health counts (healthy/degraded/down/unknown)
get_signal_history(entity_slug, state, limit) — open + recent signals
get_audit_trail(entity_id) — audit log filter + browse
get_agent_activity(limit) — agent self-inspection
query_metrics(hours=24) — time-series metric bucketed averages
get_trend(entity_id, days=7) — metric slope over time
get_event_timeline(severity, entity_slug, limit) — recent events
Ops Tools — live state, signals, checks, and execution:
run(target, command, purpose, declared_risk) — general execution primitive; read-only auto-acts, mutations queue for approval, destructive always needs explicit confirmation
inspect_path(path, targets) — bulk mount/df/ls/stat across multiple hosts/LXCs
get_execution_status(execution_id) — poll execution progress
tail_log(service_slug, lines=50) — journalctl for a service
get_service_status(service_slug) — systemctl is-active/is-enabled
get_lxc_state(lxc_slug) — pct status from Proxmox host
ping_service(service_slug) — HTTP reachability + scheduler health state
list_lxcs(state) — all LXC containers with ID, host, IP, last-audited hint
ack_signal(signal_id) — acknowledge an open signal
resolve_signal(signal_id, resolution) — resolve a signal with optional note
mute_signal(signal_id, duration_s=3600) — temporarily mute a signal
cancel_execution(execution_id, reason) — cancel a queued/running execution
update_check(check_id, enabled) — enable or disable a health check
list_checks(entity_slug, enabled) — list health checks with verdict, probe kind
list_executions(entity_slug, status, limit=25) — cursor-paginated execution history
list_entity_sessions(entity_slug) — active Nomos sessions linked to an entity
get_dashboard_summary() — fleet overview: counts, health, signals, approvals
get_secret(key, path, environment) — retrieve a secret from the Infisical vault
list_secrets(path_prefix) — list secret keys in the Infisical vault
set_secret(key, value, path, environment) — store/update a secret (requires approval)
Knowledge — keep the graph current (none require approval; this updates
the knowledge graph, not live infrastructure):
upsert_knowledge(title, content) — record what you learned after solving
a non-obvious problem; the only way anything persists past a session
update_entity_attributes(slug, attributes) — merge a discovered fact
(IP, version, port, ...) into an entity so a future task doesn't
rediscover it from scratch
create_relationship(source, target, type) — record a discovered edge
(depends-on, hosts, routes-to, ...) between two entities
Knowledge Tools — search, read, and maintain the knowledge base:
search_knowledge(query) — full-text search across docs (snippets, not full body)
get_entity_knowledge(entity_slug) — all docs/investigations/runbooks linked to a slug
get_knowledge_content(slug) — full markdown body of one knowledge entry
upsert_knowledge(title, content, about, tags, kind) — write what you learned
delete_knowledge(knowledge_slug) — soft-delete a knowledge entry
restore_knowledge(knowledge_slug) — restore a soft-deleted entry
merge_knowledge(target_slug, source_slugs) — fold entries into a target
rename_knowledge_tag(from, to) — bulk-rename tags across all entries
get_knowledge_revisions(knowledge_slug) — version history for a knowledge entry
get_knowledge_duplicates(threshold=0.6) — near-duplicate detection via trigram similarity
get_knowledge_orphans(stale_days=90) — unlinked, untagged, or stale entries
list_knowledge_tags() — all tags with usage counts and casing variants
list_my_secrets(caller_pubkey) — secrets accessible to a client by age public key
Execution — mutating the live infrastructure:
run(target, command) — the general execution primitive. Run any shell
command against a host or LXC; every command is auto-classified —
read-only inspection runs immediately, anything state-changing needs
operator approval, and destructive patterns (rm -rf, dd, mkfs,
pct/qm destroy, DROP TABLE, reboot, curl-pipe-to-shell, ...) always
need approval regardless of what you declare. This is the ONLY
mutation tool — `request_execution` was retired 2026-07-14; the
former enum actions (restart, systemctl, pct_exec, apt_upgrade,
pct_create) are all expressed as `run(target, command)` now.
get_execution_status(execution_id) — poll progress
Analysis Tools — fleet health, metrics, and introspection:
get_health_summary(health) — fleet health per entity, optionally filtered
get_audit_trail(entity_id) — query the audit log
query_metrics(hours=24) — time-series with bucketed avg/min/max
get_signal_history(entity_slug, state, limit=50) — open and recent signals
get_patterns(status, entity_type, action) — learned action patterns
get_skills(status) — available automation skills
get_trend(entity_id, days=7) — metric slope, variance, and averages
get_event_timeline(severity, entity_slug, limit=50) — recent events
get_agent_activity(limit=50) — agent self-inspection log
classify_command(command, declared_risk) — pre-flight risk classification before `run`
get_ontology() — entity types, relationship types, and lifecycle definitions
http_get(url) — fetch a public web page/raw file; ~16KB cap
**When to prefer MCP over grepping the clone:** always for knowledge queries.
`search_knowledge("jellyfin hardware acceleration")` returns ranked results from

View File

@@ -1 +1 @@
0.26.0
0.31.0

View File

@@ -3185,8 +3185,6 @@ components:
required:
- age_public_key
- age_private_key
- infisical_client_id
- infisical_client_secret
properties:
age_public_key:
type: string
@@ -3200,9 +3198,6 @@ components:
infisical_client_secret:
type: string
description: Infisical UniversalAuth client secret
machine_identity_token:
type: string
description: Infisical machine identity access token
ClientContext:
type: object
required:

View File

@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# pvecm_quorum_check.sh — Proxmox cluster quorum status.
# Runs on a PVE host. Fails if the node is not quorate.
set -euo pipefail
# pvecm status exit code is non-zero on non-quorate nodes
# (e.g. "Quorate: No — Activity blocked")
if pvecm status 2>/dev/null | grep -q 'Quorate.*Yes'; then
echo '{"health":"healthy","metrics":{"quorate":1}}'
else
echo '{"health":"unhealthy","metrics":{"quorate":0}}'
fi

View File

@@ -66,9 +66,9 @@ type agent struct {
queue *messageQueue
}
func newAgent(ctx context.Context, clients *mcpClientPool, st *store, agentSlug string) (*agent, error) {
func newAgent(ctx context.Context, clients *mcpClientPool, st *store, agentSlug string, openrouterAPIKey string) (*agent, error) {
system := loadSoul()
apiKey := os.Getenv("OPENROUTER_API_KEY")
apiKey := openrouterAPIKey
model := os.Getenv("NOMOS_MODEL")
if model == "" {
// v4-pro over v4-flash: the flash tier over-narrates, occasionally

348
cmd/nomos/mcp.go Normal file
View File

@@ -0,0 +1,348 @@
package main
import (
"bufio"
"bytes"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net/http"
"strings"
"sync"
"time"
)
// ─── MCP Streamable HTTP client ────────────────────────────────────────
type mcpClient struct {
baseURL string
token string // OIKOS_MCP_BEARER_TOKEN — api's combinedAuth requires it on every request (no dev-open bypass)
sessionID string
http *http.Client
nextID int
mu sync.Mutex // one client serializes its own MCP calls (the pool gives each session its own client, so this never blocks another session)
// toolsCache holds the last tools/list result. The tool list is static
// for the lifetime of one MCP connection — it only changes when the api
// process (re)registers tools, i.e. on a restart, which this client
// already detects and reacts to via reconnectLocked. Without this,
// buildTools (called at the start of EVERY chat turn, including every
// auto-continuation resume) paid a full tools/list round-trip every
// single time for a list that's almost always identical to the last one.
// Guarded separately from mu (not reused) so a cache check never
// contends with an in-flight doRequest call for a different method.
toolsMu sync.Mutex
toolsCache []toolDef
}
func newMCPClient(baseURL, token string) (*mcpClient, error) {
c := &mcpClient{
baseURL: baseURL,
token: token,
http: &http.Client{Timeout: 120 * time.Second},
}
resp, err := c.doRequest("initialize", map[string]any{
"protocolVersion": "2024-11-05",
"capabilities": map[string]any{},
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
})
if err != nil {
return nil, fmt.Errorf("initialize: %w", err)
}
if resp.sessionID == "" {
return nil, fmt.Errorf("no session ID in initialize response")
}
c.sessionID = resp.sessionID
c.doRequest("notifications/initialized", map[string]any{})
slog.Info("nomos: mcp connected", "session", c.sessionID[:16]+"...")
return c, nil
}
type mcpJSONRPCResponse struct {
sessionID string
Result json.RawMessage `json:"result"`
Error json.RawMessage `json:"error"`
}
// errStaleSession signals that the MCP server rejected our session id (e.g.
// after an api/MCP restart), so the client should re-initialize and retry.
var errStaleSession = fmt.Errorf("mcp session stale")
// doRequest serializes MCP calls and transparently re-initializes the session
// if the server has forgotten it (common after an api redeploy), retrying the
// original call once. Without this, an api restart permanently breaks nomos
// until it is itself restarted.
func (c *mcpClient) doRequest(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
c.mu.Lock()
defer c.mu.Unlock()
resp, err := c.send(method, params)
if err != nil && method != "initialize" && errors.Is(err, errStaleSession) {
slog.Warn("nomos: mcp session stale, reconnecting")
if rerr := c.reconnectLocked(); rerr != nil {
return nil, fmt.Errorf("mcp reconnect: %w (original: %v)", rerr, err)
}
return c.send(method, params)
}
return resp, err
}
// reconnectLocked re-initializes the MCP session. The caller must hold c.mu.
func (c *mcpClient) reconnectLocked() error {
c.sessionID = ""
// A reconnect means the api process was restarted (or forgot us) — its
// tool registration may have changed, so the cached list is no longer
// trustworthy.
c.toolsMu.Lock()
c.toolsCache = nil
c.toolsMu.Unlock()
resp, err := c.send("initialize", map[string]any{
"protocolVersion": "2024-11-05",
"capabilities": map[string]any{},
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
})
if err != nil {
return err
}
if resp.sessionID == "" {
return fmt.Errorf("no session ID on re-initialize")
}
c.sessionID = resp.sessionID
_, _ = c.send("notifications/initialized", map[string]any{})
slog.Info("nomos: mcp reconnected", "session", c.sessionID[:16]+"...")
return nil
}
// send performs one MCP round-trip. It does not lock; callers hold c.mu.
func (c *mcpClient) send(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
c.nextID++
body, _ := json.Marshal(map[string]any{
"jsonrpc": "2.0",
"method": method,
"params": params,
"id": c.nextID,
})
req, err := http.NewRequest(http.MethodPost, c.baseURL, bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json, text/event-stream")
if c.sessionID != "" {
req.Header.Set("Mcp-Session-Id", c.sessionID)
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
// A rejected/unknown session comes back as 4xx (commonly 400/404).
if resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusBadRequest {
return nil, errStaleSession
}
result := &mcpJSONRPCResponse{}
result.sessionID = resp.Header.Get("Mcp-Session-Id")
scanner := bufio.NewScanner(resp.Body)
scanner.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
gotData := false
for scanner.Scan() {
line := scanner.Text()
if strings.HasPrefix(line, "data: ") {
gotData = true
data := line[6:]
if err := json.Unmarshal([]byte(data), result); err != nil {
return nil, fmt.Errorf("parse response: %w", err)
}
}
}
if result.Error != nil {
return nil, fmt.Errorf("rpc error: %s", string(result.Error))
}
// Empty body with no result and a session set: the server likely dropped
// our session. Notifications legitimately return no data, so exempt them.
if !gotData && result.Result == nil && method != "notifications/initialized" {
return nil, errStaleSession
}
if result.sessionID != "" {
c.sessionID = result.sessionID
}
return result, nil
}
func (c *mcpClient) callTool(name string, args map[string]any) (any, error) {
resp, err := c.doRequest("tools/call", map[string]any{
"name": name,
"arguments": args,
})
if err != nil {
return nil, err
}
var toolResult struct {
Content []struct {
Type string `json:"type"`
Text string `json:"text"`
} `json:"content"`
}
if err := json.Unmarshal(resp.Result, &toolResult); err != nil {
return string(resp.Result), nil
}
var texts []string
for _, c := range toolResult.Content {
if c.Type == "text" {
var parsed any
if json.Unmarshal([]byte(c.Text), &parsed) == nil {
return parsed, nil
}
texts = append(texts, c.Text)
}
}
if len(texts) == 1 {
return texts[0], nil
}
return texts, nil
}
func (c *mcpClient) listTools() ([]string, error) {
resp, err := c.doRequest("tools/list", map[string]any{})
if err != nil {
return nil, err
}
var tr struct {
Tools []struct {
Name string `json:"name"`
Description string `json:"description"`
} `json:"tools"`
}
if err := json.Unmarshal(resp.Result, &tr); err != nil {
return nil, err
}
var names []string
for _, t := range tr.Tools {
names = append(names, t.Name)
}
return names, nil
}
func (c *mcpClient) close() {
}
// ─── Per-session MCP client pool ────────────────────────────────────────
//
// A single shared mcpClient serializes EVERY tool call across EVERY
// concurrently-running task through one mutex (see mcpClient.mu) — `run`
// executes its SSH command synchronously inside that lock and is capped at
// up to 10 minutes, so one task mid-`run` stalled every other task's tool
// calls, even trivial reads, behind it. The MCP *server* has no per-
// connection state to protect (newServer in internal/mcp/server.go returns
// one shared *mcp.Server instance whose tool handlers close only over the DB
// pool, which is already safe for concurrent use) — the mutex existed purely
// because the *client* reused one stateful transport session, not because
// the server needed it. Giving each task's own session its own client
// removes the cross-task serialization entirely: a task's own tool calls
// stay sequential (which they already are — the agent loop calls tools one
// at a time within a turn), but no longer block anyone else's.
type mcpClientPool struct {
baseURL string
token string
mu sync.Mutex
clients map[string]*pooledMCPClient
}
type pooledMCPClient struct {
client *mcpClient
lastUsed time.Time
}
func newMCPClientPool(baseURL, token string) *mcpClientPool {
return &mcpClientPool{baseURL: baseURL, token: token, clients: make(map[string]*pooledMCPClient)}
}
// get returns the client for sessionID, creating and initializing one (a
// real MCP handshake) on first use. Session ids that don't identify a real
// persisted conversation ("" / "ephemeral", the no-DB-store path; "query",
// the structured /query endpoint) still get exactly one dedicated,
// reused client each via the same map — just keyed on a fixed string instead
// of a real session id — so that traffic doesn't pay a fresh handshake per
// request while still never sharing a connection with an actual task.
func (p *mcpClientPool) get(sessionID string) (*mcpClient, error) {
key := sessionID
if key == "" {
key = "ephemeral"
}
p.mu.Lock()
if pc, ok := p.clients[key]; ok {
pc.lastUsed = time.Now()
p.mu.Unlock()
return pc.client, nil
}
p.mu.Unlock()
// Initialize outside the lock — it's a network round-trip, and holding
// the pool mutex for it would serialize unrelated sessions' first calls
// behind each other, undermining the whole point of this pool.
c, err := newMCPClient(p.baseURL, p.token)
if err != nil {
return nil, err
}
p.mu.Lock()
// Another goroutine may have created one for the same key while we were
// initializing (two of this session's tool calls racing on a cold
// start); keep whichever won, close out the loser's connection (a no-op
// today, but future-proof if mcpClient.close ever does real teardown).
if existing, ok := p.clients[key]; ok {
p.mu.Unlock()
c.close()
return existing.client, nil
}
p.clients[key] = &pooledMCPClient{client: c, lastUsed: time.Now()}
p.mu.Unlock()
return c, nil
}
// mcpClientIdleTimeout is how long an idle session's MCP client is kept
// before eviction — long enough to outlive a single slow `run` (capped at 10
// minutes server-side) plus normal think-time between a task's tool calls,
// short enough not to accumulate one abandoned connection per finished task
// forever.
const mcpClientIdleTimeout = 20 * time.Minute
// sweep evicts clients idle past mcpClientIdleTimeout. Call on a ticker.
func (p *mcpClientPool) sweep() {
cutoff := time.Now().Add(-mcpClientIdleTimeout)
p.mu.Lock()
defer p.mu.Unlock()
for key, pc := range p.clients {
if pc.lastUsed.Before(cutoff) {
pc.client.close()
delete(p.clients, key)
}
}
}
func (p *mcpClientPool) closeAll() {
p.mu.Lock()
defer p.mu.Unlock()
for key, pc := range p.clients {
pc.client.close()
delete(p.clients, key)
}
}

View File

@@ -1,11 +1,8 @@
package main
import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net/http"
@@ -18,7 +15,7 @@ import (
"time"
"github.com/dtoro/oikos/internal/safego"
"github.com/google/uuid"
"github.com/dtoro/oikos/internal/secrets"
"github.com/jackc/pgx/v5"
)
@@ -27,13 +24,14 @@ func main() {
fmt.Fprintln(os.Stderr, "usage: nomos serve")
os.Exit(1)
}
if os.Args[1] == "healthcheck" {
runHealthcheck()
return
}
mcpURL := os.Getenv("NOMOS_MCP_URL")
if mcpURL == "" {
mcpURL = "http://localhost:8090/mcp"
}
// api's combinedAuth requires a bearer token on every request (no
// dev-open bypass — plans/2026-07-12-wails-desktop-app.md 0.4); this is
// the same shared secret api validates against (OIKOS_MCP_BEARER_TOKEN).
mcpToken := os.Getenv("OIKOS_MCP_BEARER_TOKEN")
agentSlug := os.Getenv("NOMOS_AGENT_SLUG")
@@ -46,6 +44,32 @@ func main() {
databaseURL = os.Getenv("OIKOS_DATABASE_URL")
}
sec := secrets.NewManagerFromConfig(
os.Getenv("OIKOS_INFISICAL_SITE_URL"),
os.Getenv("OIKOS_INFISICAL_CLIENT_ID"),
os.Getenv("OIKOS_INFISICAL_CLIENT_SECRET"),
os.Getenv("OIKOS_INFISICAL_PROJECT_ID"),
os.Getenv("OIKOS_INFISICAL_ENV"),
os.Getenv("OIKOS_SECRETS_DIR"),
)
var openrouterAPIKey string
var secretsResolved int
if sec != nil {
resCtx, resCancel := context.WithTimeout(context.Background(), 10*time.Second)
if v := secrets.ResolveSecret(resCtx, sec, "mcp_bearer-token", ""); v != "" {
mcpToken = v
secretsResolved++
}
openrouterAPIKey = secrets.ResolveSecret(resCtx, sec, "openrouter_api-key", os.Getenv("OPENROUTER_API_KEY"))
if openrouterAPIKey != "" && openrouterAPIKey != os.Getenv("OPENROUTER_API_KEY") {
secretsResolved++
}
resCancel()
if secretsResolved > 0 {
slog.Info("nomos: secrets resolved from Infisical", "count", secretsResolved)
}
}
switch os.Args[1] {
case "serve":
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
@@ -75,7 +99,7 @@ func main() {
defer st.close()
}
nAgent, err := newAgent(ctx, clientPool, st, agentSlug)
nAgent, err := newAgent(ctx, clientPool, st, agentSlug, openrouterAPIKey)
if err != nil {
slog.Error("nomos: agent init", "error", err)
os.Exit(1)
@@ -161,153 +185,32 @@ func main() {
}
}
func runHealthcheck() {
addr := os.Getenv("NOMOS_LISTEN")
if addr == "" {
addr = ":8092"
}
host := addr
if strings.HasPrefix(host, ":") {
host = "127.0.0.1" + host
}
client := &http.Client{Timeout: 3 * time.Second}
resp, err := client.Get("http://" + host + "/healthz")
if err != nil {
os.Exit(1)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
os.Exit(1)
}
}
func sseEvent(w http.ResponseWriter, flusher http.Flusher, event agentEvent) {
data, _ := json.Marshal(event)
fmt.Fprintf(w, "data: %s\n\n", data)
flusher.Flush()
}
// runChatTurn is the shared core of an operator-initiated turn: insert an
// assistant placeholder, run a.chat with incremental persistence (so whatever
// happened before an abort is never lost), finalize the row, and derive a
// title. It is agnostic to the transport: `sink` receives every agent event
// for delivery (SSE for a live handleChat, a no-op for a queued turn that has
// no client attached — the frontend learns about those via the poller + the
// status-driven "working" signal). The caller MUST already hold the session's
// turn-gate permit.
func (a *agent) runChatTurn(pctx, ctx context.Context, sessionID, message string, sink func(agentEvent)) {
toolCalls := []map[string]any{}
// P3: accumulate per-iteration reasoning instead of overwriting with the
// final `text` event (see the original inline comment in handleChat).
var textParts []string
var thinkingParts []string
var finalText string
var finalThinking string
placeholder, _ := json.Marshal(map[string]any{"role": "assistant", "text": ""})
msgID, err := a.store.insertMessageReturningID(pctx, sessionID, "assistant", placeholder)
if err != nil {
slog.Error("nomos: chat placeholder insert failed", "session", sessionID, "error", err)
}
persist := func() {
if msgID == uuid.Nil {
return
}
body, _ := json.Marshal(map[string]any{
"role": "assistant",
"text": finalText,
"thinking": finalThinking,
"tool_calls": toolCalls,
})
a.store.updateMessage(pctx, msgID, body)
}
a.chat(ctx, sessionID, message, func(ev agentEvent) {
if ev.Type == "tool_use" || ev.Type == "tool_result" {
if m, ok := ev.Data.(map[string]any); ok {
m["type"] = ev.Type
// One entry per tool call: tool_use creates it, tool_result
// merges the result into the same entry (matched by id).
id, _ := m["id"].(string)
if id != "" && ev.Type == "tool_result" {
for _, existing := range toolCalls {
if eID, _ := existing["id"].(string); eID == id {
for k, v := range m {
existing[k] = v
}
break
}
}
} else {
toolCalls = append(toolCalls, m)
}
}
persist() // live: survives even if the client disconnects right after
}
if ev.Type == "text" {
if t, ok := ev.Data.(string); ok && t != "" {
if ev.IsThinking {
thinkingParts = append(thinkingParts, t)
finalThinking = strings.Join(thinkingParts, "\n\n")
} else {
textParts = append(textParts, t)
finalText = strings.Join(textParts, "\n\n")
}
persist()
}
}
sink(ev)
})
// B.6: if the turn ended with no text and no tool calls (the model
// empty-response'd and all retries failed), delete the placeholder row
// instead of persisting an empty bubble.
if finalText == "" && len(toolCalls) == 0 && msgID != uuid.Nil {
a.store.deleteMessage(pctx, msgID)
} else {
persist() // final state — same row, updated one last time
}
// Title: prefer the goal once set; else the first assistant answer.
if finalText != "" && sessionID != "ephemeral" {
var goalTitle string
if sess, gerr := a.store.getSession(pctx, sessionID); gerr == nil && sess.Goal != "" {
goalTitle = truncate(sess.Goal, 120)
}
title := goalTitle
if title == "" {
title = truncate(finalText, 80)
}
if title != "" {
a.store.updateSessionTitle(pctx, sessionID, title)
}
}
}
// drainAcquireWait is how long drainQueued blocks for a busy gate before
// re-queuing and deferring to the holder's own release-drain. A package var so
// tests can shorten it; in production it just needs to outlast the brief
// release→drain handoff window.
var drainAcquireWait = 5 * time.Second
// drainQueued runs every queued operator message for a session as its own turn,
// one at a time, under the turn gate. Called (in a goroutine) whenever a turn
// releases the gate — from handleChat (live) and resumeSession (background) —
// so a message queued while the agent was busy is acted on as soon as it's
// free, without the operator re-sending. See messagequeue.go (plan 2026-08-03
// F2).
//
// Each queued turn is persisted incrementally and has no SSE client (the
// browser detached after receiving the `queued` event); the frontend sees the
// result via the 3s poller and the status-driven "working" indicator.
func (a *agent) drainQueued(ctx context.Context, sessionID string) {
for {
msg, ok := a.queue.dequeue(sessionID)
if !ok {
return
}
// Block briefly for the gate. If a live turn grabbed it first, put the
// message back — that turn's release will drain it again. Never stack.
if !a.gate.acquire(sessionID, drainAcquireWait) {
a.queue.requeueFront(sessionID, msg)
return
}
slog.Info("nomos: running queued operator message", "session", sessionID)
pctx := context.Background()
// Run the turn inside a per-iteration closure so the gate release is
// deferred to the end of THIS turn (and runs even if runChatTurn
// panics — safego recovers the panic at the goroutine boundary, so a
// non-deferred release would be skipped and the session's permit held
// forever, deadlocking all future turns). A bare `defer release` in
// the loop would be wrong too: Go defers run at function exit, not
// iteration exit, so the gate would stay held across iterations.
func() {
defer a.gate.release(sessionID)
a.runChatTurn(pctx, ctx, sessionID, msg, func(agentEvent) {})
}()
}
}
func handleChat(w http.ResponseWriter, r *http.Request, a *agent, st *store) {
if r.Method != http.MethodPost {
http.Error(w, "method not allowed", 405)
@@ -789,339 +692,4 @@ func truncate(s string, n int) string {
return s
}
return s[:n] + "..."
}
// ─── MCP Streamable HTTP client ────────────────────────────────────────
type mcpClient struct {
baseURL string
token string // OIKOS_MCP_BEARER_TOKEN — api's combinedAuth requires it on every request (no dev-open bypass)
sessionID string
http *http.Client
nextID int
mu sync.Mutex // this client is one stateful MCP session; serialize ITS OWN calls
// toolsCache holds the last tools/list result. The tool list is static
// for the lifetime of one MCP connection — it only changes when the api
// process (re)registers tools, i.e. on a restart, which this client
// already detects and reacts to via reconnectLocked. Without this,
// buildTools (called at the start of EVERY chat turn, including every
// auto-continuation resume) paid a full tools/list round-trip every
// single time for a list that's almost always identical to the last one.
// Guarded separately from mu (not reused) so a cache check never
// contends with an in-flight doRequest call for a different method.
toolsMu sync.Mutex
toolsCache []toolDef
}
func newMCPClient(baseURL, token string) (*mcpClient, error) {
c := &mcpClient{
baseURL: baseURL,
token: token,
http: &http.Client{Timeout: 120 * time.Second},
}
resp, err := c.doRequest("initialize", map[string]any{
"protocolVersion": "2024-11-05",
"capabilities": map[string]any{},
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
})
if err != nil {
return nil, fmt.Errorf("initialize: %w", err)
}
if resp.sessionID == "" {
return nil, fmt.Errorf("no session ID in initialize response")
}
c.sessionID = resp.sessionID
c.doRequest("notifications/initialized", map[string]any{})
slog.Info("nomos: mcp connected", "session", c.sessionID[:16]+"...")
return c, nil
}
type mcpJSONRPCResponse struct {
sessionID string
Result json.RawMessage `json:"result"`
Error json.RawMessage `json:"error"`
}
// errStaleSession signals that the MCP server rejected our session id (e.g.
// after an api/MCP restart), so the client should re-initialize and retry.
var errStaleSession = fmt.Errorf("mcp session stale")
// doRequest serializes MCP calls and transparently re-initializes the session
// if the server has forgotten it (common after an api redeploy), retrying the
// original call once. Without this, an api restart permanently breaks nomos
// until it is itself restarted.
func (c *mcpClient) doRequest(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
c.mu.Lock()
defer c.mu.Unlock()
resp, err := c.send(method, params)
if err != nil && method != "initialize" && errors.Is(err, errStaleSession) {
slog.Warn("nomos: mcp session stale, reconnecting")
if rerr := c.reconnectLocked(); rerr != nil {
return nil, fmt.Errorf("mcp reconnect: %w (original: %v)", rerr, err)
}
return c.send(method, params)
}
return resp, err
}
// reconnectLocked re-initializes the MCP session. The caller must hold c.mu.
func (c *mcpClient) reconnectLocked() error {
c.sessionID = ""
// A reconnect means the api process was restarted (or forgot us) — its
// tool registration may have changed, so the cached list is no longer
// trustworthy.
c.toolsMu.Lock()
c.toolsCache = nil
c.toolsMu.Unlock()
resp, err := c.send("initialize", map[string]any{
"protocolVersion": "2024-11-05",
"capabilities": map[string]any{},
"clientInfo": map[string]any{"name": "nomos", "version": "2.0"},
})
if err != nil {
return err
}
if resp.sessionID == "" {
return fmt.Errorf("no session ID on re-initialize")
}
c.sessionID = resp.sessionID
_, _ = c.send("notifications/initialized", map[string]any{})
slog.Info("nomos: mcp reconnected", "session", c.sessionID[:16]+"...")
return nil
}
// send performs one MCP round-trip. It does not lock; callers hold c.mu.
func (c *mcpClient) send(method string, params map[string]any) (*mcpJSONRPCResponse, error) {
c.nextID++
body, _ := json.Marshal(map[string]any{
"jsonrpc": "2.0",
"method": method,
"params": params,
"id": c.nextID,
})
req, err := http.NewRequest(http.MethodPost, c.baseURL, bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json, text/event-stream")
if c.sessionID != "" {
req.Header.Set("Mcp-Session-Id", c.sessionID)
}
if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token)
}
resp, err := c.http.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
// A rejected/unknown session comes back as 4xx (commonly 400/404).
if resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusBadRequest {
return nil, errStaleSession
}
result := &mcpJSONRPCResponse{}
result.sessionID = resp.Header.Get("Mcp-Session-Id")
scanner := bufio.NewScanner(resp.Body)
scanner.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
gotData := false
for scanner.Scan() {
line := scanner.Text()
if strings.HasPrefix(line, "data: ") {
gotData = true
data := line[6:]
if err := json.Unmarshal([]byte(data), result); err != nil {
return nil, fmt.Errorf("parse response: %w", err)
}
}
}
if result.Error != nil {
return nil, fmt.Errorf("rpc error: %s", string(result.Error))
}
// Empty body with no result and a session set: the server likely dropped
// our session. Notifications legitimately return no data, so exempt them.
if !gotData && result.Result == nil && method != "notifications/initialized" {
return nil, errStaleSession
}
if result.sessionID != "" {
c.sessionID = result.sessionID
}
return result, nil
}
func (c *mcpClient) callTool(name string, args map[string]any) (any, error) {
resp, err := c.doRequest("tools/call", map[string]any{
"name": name,
"arguments": args,
})
if err != nil {
return nil, err
}
var toolResult struct {
Content []struct {
Type string `json:"type"`
Text string `json:"text"`
} `json:"content"`
}
if err := json.Unmarshal(resp.Result, &toolResult); err != nil {
return string(resp.Result), nil
}
var texts []string
for _, c := range toolResult.Content {
if c.Type == "text" {
var parsed any
if json.Unmarshal([]byte(c.Text), &parsed) == nil {
return parsed, nil
}
texts = append(texts, c.Text)
}
}
if len(texts) == 1 {
return texts[0], nil
}
return texts, nil
}
func (c *mcpClient) listTools() ([]string, error) {
resp, err := c.doRequest("tools/list", map[string]any{})
if err != nil {
return nil, err
}
var tr struct {
Tools []struct {
Name string `json:"name"`
Description string `json:"description"`
} `json:"tools"`
}
if err := json.Unmarshal(resp.Result, &tr); err != nil {
return nil, err
}
var names []string
for _, t := range tr.Tools {
names = append(names, t.Name)
}
return names, nil
}
func (c *mcpClient) close() {
}
// ─── Per-session MCP client pool ────────────────────────────────────────
//
// A single shared mcpClient serializes EVERY tool call across EVERY
// concurrently-running task through one mutex (see mcpClient.mu) — `run`
// executes its SSH command synchronously inside that lock and is capped at
// up to 10 minutes, so one task mid-`run` stalled every other task's tool
// calls, even trivial reads, behind it. The MCP *server* has no per-
// connection state to protect (newServer in internal/mcp/server.go returns
// one shared *mcp.Server instance whose tool handlers close only over the DB
// pool, which is already safe for concurrent use) — the mutex existed purely
// because the *client* reused one stateful transport session, not because
// the server needed it. Giving each task's own session its own client
// removes the cross-task serialization entirely: a task's own tool calls
// stay sequential (which they already are — the agent loop calls tools one
// at a time within a turn), but no longer block anyone else's.
type mcpClientPool struct {
baseURL string
token string
mu sync.Mutex
clients map[string]*pooledMCPClient
}
type pooledMCPClient struct {
client *mcpClient
lastUsed time.Time
}
func newMCPClientPool(baseURL, token string) *mcpClientPool {
return &mcpClientPool{baseURL: baseURL, token: token, clients: make(map[string]*pooledMCPClient)}
}
// get returns the client for sessionID, creating and initializing one (a
// real MCP handshake) on first use. Session ids that don't identify a real
// persisted conversation ("" / "ephemeral", the no-DB-store path; "query",
// the structured /query endpoint) still get exactly one dedicated,
// reused client each via the same map — just keyed on a fixed string instead
// of a real session id — so that traffic doesn't pay a fresh handshake per
// request while still never sharing a connection with an actual task.
func (p *mcpClientPool) get(sessionID string) (*mcpClient, error) {
key := sessionID
if key == "" {
key = "ephemeral"
}
p.mu.Lock()
if pc, ok := p.clients[key]; ok {
pc.lastUsed = time.Now()
p.mu.Unlock()
return pc.client, nil
}
p.mu.Unlock()
// Initialize outside the lock — it's a network round-trip, and holding
// the pool mutex for it would serialize unrelated sessions' first calls
// behind each other, undermining the whole point of this pool.
c, err := newMCPClient(p.baseURL, p.token)
if err != nil {
return nil, err
}
p.mu.Lock()
// Another goroutine may have created one for the same key while we were
// initializing (two of this session's tool calls racing on a cold
// start); keep whichever won, close out the loser's connection (a no-op
// today, but future-proof if mcpClient.close ever does real teardown).
if existing, ok := p.clients[key]; ok {
p.mu.Unlock()
c.close()
return existing.client, nil
}
p.clients[key] = &pooledMCPClient{client: c, lastUsed: time.Now()}
p.mu.Unlock()
return c, nil
}
// mcpClientIdleTimeout is how long an idle session's MCP client is kept
// before eviction — long enough to outlive a single slow `run` (capped at 10
// minutes server-side) plus normal think-time between a task's tool calls,
// short enough not to accumulate one abandoned connection per finished task
// forever.
const mcpClientIdleTimeout = 20 * time.Minute
// sweep evicts clients idle past mcpClientIdleTimeout. Call on a ticker.
func (p *mcpClientPool) sweep() {
cutoff := time.Now().Add(-mcpClientIdleTimeout)
p.mu.Lock()
defer p.mu.Unlock()
for key, pc := range p.clients {
if pc.lastUsed.Before(cutoff) {
pc.client.close()
delete(p.clients, key)
}
}
}
func (p *mcpClientPool) closeAll() {
p.mu.Lock()
defer p.mu.Unlock()
for key, pc := range p.clients {
pc.client.close()
delete(p.clients, key)
}
}
}

View File

@@ -179,6 +179,15 @@ func (s *store) createTaskEntity(ctx context.Context, sessionID, title string) s
`UPDATE agent_sessions SET entity_id = $1 WHERE id = $2`, entityID, sessionID); err != nil {
slog.Warn("nomos: could not link task entity", "session", sessionID, "error", err)
}
// Graph edge: task —involves→ agent:nomos (gives every task at least one
// edge from creation, even if no run calls are ever made).
s.pool.Exec(ctx, `INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
SELECT $1, id, 'involves', '{"by":"nomos"}'::jsonb, now()
FROM entities WHERE slug = 'agent:nomos'
AND NOT EXISTS (
SELECT 1 FROM relationships r
WHERE r.source_id = $1 AND r.target_id = entities.id AND r.type = 'involves' AND r.valid_to IS NULL)`,
entityID)
return entityID.String()
}

152
cmd/nomos/workers.go Normal file
View File

@@ -0,0 +1,152 @@
package main
import (
"context"
"encoding/json"
"log/slog"
"strings"
"time"
"github.com/google/uuid"
)
// runChatTurn is the shared core of an operator-initiated turn: insert an
// assistant placeholder, run a.chat with incremental persistence (so whatever
// happened before an abort is never lost), finalize the row, and derive a
// title. It is agnostic to the transport: `sink` receives every agent event
// for delivery (SSE for a live handleChat, a no-op for a queued turn that has
// no client attached — the frontend learns about those via the poller + the
// status-driven "working" signal). The caller MUST already hold the session's
// turn-gate permit.
func (a *agent) runChatTurn(pctx, ctx context.Context, sessionID, message string, sink func(agentEvent)) {
toolCalls := []map[string]any{}
// P3: accumulate per-iteration reasoning instead of overwriting with the
// final `text` event (see the original inline comment in handleChat).
var textParts []string
var thinkingParts []string
var finalText string
var finalThinking string
placeholder, _ := json.Marshal(map[string]any{"role": "assistant", "text": ""})
msgID, err := a.store.insertMessageReturningID(pctx, sessionID, "assistant", placeholder)
if err != nil {
slog.Error("nomos: chat placeholder insert failed", "session", sessionID, "error", err)
}
persist := func() {
if msgID == uuid.Nil {
return
}
body, _ := json.Marshal(map[string]any{
"role": "assistant",
"text": finalText,
"thinking": finalThinking,
"tool_calls": toolCalls,
})
a.store.updateMessage(pctx, msgID, body)
}
a.chat(ctx, sessionID, message, func(ev agentEvent) {
if ev.Type == "tool_use" || ev.Type == "tool_result" {
if m, ok := ev.Data.(map[string]any); ok {
m["type"] = ev.Type
// One entry per tool call: tool_use creates it, tool_result
// merges the result into the same entry (matched by id).
id, _ := m["id"].(string)
if id != "" && ev.Type == "tool_result" {
for _, existing := range toolCalls {
if eID, _ := existing["id"].(string); eID == id {
for k, v := range m {
existing[k] = v
}
break
}
}
} else {
toolCalls = append(toolCalls, m)
}
}
persist() // live: survives even if the client disconnects right after
}
if ev.Type == "text" {
if t, ok := ev.Data.(string); ok && t != "" {
if ev.IsThinking {
thinkingParts = append(thinkingParts, t)
finalThinking = strings.Join(thinkingParts, "\n\n")
} else {
textParts = append(textParts, t)
finalText = strings.Join(textParts, "\n\n")
}
persist()
}
}
sink(ev)
})
// B.6: if the turn ended with no text and no tool calls (the model
// empty-response'd and all retries failed), delete the placeholder row
// instead of persisting an empty bubble.
if finalText == "" && len(toolCalls) == 0 && msgID != uuid.Nil {
a.store.deleteMessage(pctx, msgID)
} else {
persist() // final state — same row, updated one last time
}
// Title: prefer the goal once set; else the first assistant answer.
if finalText != "" && sessionID != "ephemeral" {
var goalTitle string
if sess, gerr := a.store.getSession(pctx, sessionID); gerr == nil && sess.Goal != "" {
goalTitle = truncate(sess.Goal, 120)
}
title := goalTitle
if title == "" {
title = truncate(finalText, 80)
}
if title != "" {
a.store.updateSessionTitle(pctx, sessionID, title)
}
}
}
// drainAcquireWait is how long drainQueued blocks for a busy gate before
// re-queuing and deferring to the holder's own release-drain. A package var so
// tests can shorten it; in production it just needs to outlast the brief
// release→drain handoff window.
var drainAcquireWait = 5 * time.Second
// drainQueued runs every queued operator message for a session as its own turn,
// one at a time, under the turn gate. Called (in a goroutine) whenever a turn
// releases the gate — from handleChat (live) and resumeSession (background) —
// so a message queued while the agent was busy is acted on as soon as it's
// free, without the operator re-sending. See messagequeue.go (plan 2026-08-03
// F2).
//
// Each queued turn is persisted incrementally and has no SSE client (the
// browser detached after receiving the `queued` event); the frontend sees the
// result via the 3s poller and the status-driven "working" indicator.
func (a *agent) drainQueued(ctx context.Context, sessionID string) {
for {
msg, ok := a.queue.dequeue(sessionID)
if !ok {
return
}
// Block briefly for the gate. If a live turn grabbed it first, put the
// message back — that turn's release will drain it again. Never stack.
if !a.gate.acquire(sessionID, drainAcquireWait) {
a.queue.requeueFront(sessionID, msg)
return
}
slog.Info("nomos: running queued operator message", "session", sessionID)
pctx := context.Background()
// Run the turn inside a per-iteration closure so the gate release is
// deferred to the end of THIS turn (and runs even if runChatTurn
// panics — safego recovers the panic at the goroutine boundary, so a
// non-deferred release would be skipped and the session's permit held
// forever, deadlocking all future turns). A bare `defer release` in
// the loop would be wrong too: Go defers run at function exit, not
// iteration exit, so the gate would stay held across iterations.
func() {
defer a.gate.release(sessionID)
a.runChatTurn(pctx, ctx, sessionID, msg, func(agentEvent) {})
}()
}
}

View File

@@ -12,6 +12,7 @@ import (
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/execworker"
"github.com/dtoro/oikos/internal/httpapi"
"github.com/dtoro/oikos/internal/knowledge"
"github.com/dtoro/oikos/internal/notifier"
@@ -23,6 +24,7 @@ import (
var schedulerRunner = scheduler.RunnerForMain()
var notifierRunner = notifier.RunnerForMain()
var execWorkerRunner = execworker.RunnerForMain()
func main() {
if len(os.Args) < 2 {
@@ -37,12 +39,39 @@ func main() {
logger := observability.NewLogger(cfg.Debug)
slog.SetDefault(logger)
slog.Info("starting oikos", "role", role, "config", cfg)
ctx, cancel := signal.NotifyContext(context.Background(),
syscall.SIGTERM, syscall.SIGINT)
defer cancel()
// Resolve secrets from Infisical, overlaying env-derived config values.
// If Infisical is not configured, env vars are used as-is (no change).
sec := secrets.NewManagerFromConfig(
cfg.InfisicalSiteURL,
cfg.InfisicalClientID,
cfg.InfisicalClientSecret,
cfg.InfisicalProjectID,
cfg.InfisicalEnv,
cfg.SecretsDir,
)
if sec != nil {
overlays := secrets.ConfigOverlays(map[string]func(string){
"matrix_token": func(v string) { cfg.MatrixToken = v },
"approval_hmac-secret": func(v string) { cfg.ApprovalHMACSecret = v },
"mcp_bearer-token": func(v string) { cfg.MCPBearerToken = v },
"api_token": func(v string) { cfg.APIToken = v },
"oidc_client-secret": func(v string) { cfg.OIDCClientSecret = v },
})
n := secrets.OverlayConfig(ctx, sec, overlays)
slog.Info("secrets resolved from Infisical", "count", n)
secrets.VerifyExpectedSecrets(ctx, sec, []string{
"matrix_token", "approval_hmac-secret", "mcp_bearer-token",
"api_token", "openrouter_api-key", "webhook_hmac-secret",
})
}
slog.Info("starting oikos", "role", role, "config", cfg)
switch role {
case "migrate":
if err := runMigrate(ctx, cfg); err != nil {
@@ -68,6 +97,8 @@ func main() {
runWithPool(ctx, cfg, "scheduler", schedulerRunner)
case "notifier":
runWithPool(ctx, cfg, "notifier", notifierRunner)
case "execution-worker":
runWithPool(ctx, cfg, "execution-worker", execWorkerRunner)
case "all":
pool, err := db.New(ctx, cfg.DatabaseURL)
if err != nil {
@@ -83,8 +114,9 @@ func main() {
go schedulerRunner(ctx, pool, cfg)
go notifierRunner(ctx, pool, cfg)
go execWorkerRunner(ctx, pool, cfg)
slog.Info("all: starting api with scheduler + notifier in background")
slog.Info("all: starting api with scheduler + notifier + execution-worker in background")
if err := httpapi.ListenAndServe(ctx, pool, cfg); err != nil {
slog.Error("api failed", "error", err)
os.Exit(1)
@@ -115,7 +147,7 @@ Roles:
scheduler Run the observe loop
notifier Run the notification service (Matrix alerts)
all Run all roles in one process (dev mode)
secret Secret management (Infisical)
secret Secret management (Infisical: get, set, list, verify, audit, migrate, export-sops)
knowledge Convert wiki to knowledge seed (one-shot)
version Print version info
@@ -289,20 +321,45 @@ func runWithPool(ctx context.Context, cfg config.Config, name string, fn func(co
func runSecret(ctx context.Context, cfg config.Config) {
if len(os.Args) < 3 {
fmt.Fprintln(os.Stderr, "usage: oikos secret <list|migrate|export-sops>")
fmt.Fprintln(os.Stderr, "usage: oikos secret <get|set|list|migrate|export-sops>")
os.Exit(1)
}
sub := os.Args[2]
secretsDir := cfg.SecretsDir
if secretsDir == "" {
secretsDir = "archive/secrets-sops-backup"
}
sopsBackend := secrets.NewSOPSBackend(secretsDir)
// For get/set/list: use Infisical directly
switch sub {
case "get":
if len(os.Args) < 4 {
fmt.Fprintln(os.Stderr, "usage: oikos secret get <key>")
os.Exit(1)
}
key := os.Args[3]
backend := newInfisicalBackendOrFail(cfg)
val, err := backend.Get(ctx, key)
if err != nil {
slog.Error("secret get", "key", key, "error", err)
os.Exit(1)
}
fmt.Println(val)
case "set":
if len(os.Args) < 5 {
fmt.Fprintln(os.Stderr, "usage: oikos secret set <key> <value>")
os.Exit(1)
}
key := os.Args[3]
value := os.Args[4]
backend := newInfisicalBackendOrFail(cfg)
if err := backend.Set(ctx, key, value); err != nil {
slog.Error("secret set", "key", key, "error", err)
os.Exit(1)
}
fmt.Printf("stored: %s\n", key)
case "list":
keys, err := sopsBackend.List(ctx)
backend := newInfisicalBackendOrFail(cfg)
keys, err := backend.List(ctx)
if err != nil {
slog.Error("secret list", "error", err)
os.Exit(1)
@@ -311,24 +368,138 @@ func runSecret(ctx context.Context, cfg config.Config) {
fmt.Println(k)
}
case "migrate":
infCfg := secrets.InfisicalConfig{
SiteURL: cfg.InfisicalSiteURL,
ClientID: cfg.InfisicalClientID,
ClientSecret: cfg.InfisicalClientSecret,
ProjectID: cfg.InfisicalProjectID,
SecretPath: "/",
Env: cfg.InfisicalEnv,
}
if infCfg.Env == "" {
infCfg.Env = "dev"
}
if infCfg.SiteURL == "" {
fmt.Fprintln(os.Stderr, "error: OIKOS_INFISICAL_SITE_URL not set")
os.Exit(1)
}
case "verify":
runSecretVerify(ctx, cfg)
infBackend := secrets.NewInfisicalBackend(infCfg)
case "audit":
runSecretAudit(ctx, cfg)
case "migrate", "export-sops":
runSecretLegacy(ctx, cfg, sub)
default:
fmt.Fprintf(os.Stderr, "unknown secret command: %s\n", sub)
os.Exit(1)
}
}
// expectedSecrets is the set of keys that should exist in Infisical
// for a fully-migrated deployment.
var expectedSecrets = []string{
"matrix_token",
"approval_hmac-secret",
"mcp_bearer-token",
"api_token",
"openrouter_api-key",
"webhook_hmac-secret",
}
// runSecretVerify checks that all expected secrets are present in Infisical.
func runSecretVerify(ctx context.Context, cfg config.Config) {
backend := newInfisicalBackendOrFail(cfg)
keys, err := backend.List(ctx)
if err != nil {
slog.Error("verify: list", "error", err)
os.Exit(1)
}
keySet := make(map[string]struct{}, len(keys))
for _, k := range keys {
keySet[k] = struct{}{}
}
missing := 0
for _, exp := range expectedSecrets {
if _, ok := keySet[exp]; !ok {
fmt.Printf("MISSING: %s\n", exp)
missing++
} else {
fmt.Printf("OK: %s\n", exp)
}
}
fmt.Printf("\n%d/%d present, %d missing\n", len(expectedSecrets)-missing, len(expectedSecrets), missing)
if missing > 0 {
os.Exit(1)
}
}
// runSecretAudit resolves all expected secrets from Infisical and prints
// a diff against current env-derived values. Values are truncated for safety.
func runSecretAudit(ctx context.Context, cfg config.Config) {
backend := newInfisicalBackendOrFail(cfg)
envValues := map[string]string{
"matrix_token": cfg.MatrixToken,
"approval_hmac-secret": cfg.ApprovalHMACSecret,
"mcp_bearer-token": cfg.MCPBearerToken,
"api_token": cfg.APIToken,
"oidc_client-secret": cfg.OIDCClientSecret,
}
fmt.Println("key infisical env status")
fmt.Println(strings.Repeat("-", 72))
for _, key := range expectedSecrets {
infVal, infErr := backend.Get(ctx, key)
envVal := envValues[key]
if infErr != nil {
fmt.Printf("%-29s ERROR %-10s NOT-IN-INFISICAL\n", key, trunc(envVal, 8))
continue
}
if envVal == "" {
fmt.Printf("%-29s %-10s (empty) INFISICAL-ONLY\n", key, trunc(infVal, 8))
continue
}
if infVal == envVal {
fmt.Printf("%-29s %-10s %-10s MATCH\n", key, trunc(infVal, 8), trunc(envVal, 8))
} else {
fmt.Printf("%-29s %-10s %-10s DRIFT\n", key, trunc(infVal, 8), trunc(envVal, 8))
}
}
}
func trunc(s string, n int) string {
if len(s) <= n {
return s
}
if n > 1 {
return s[:n-1] + "…"
}
return s[:n]
}
// newInfisicalBackendOrFail creates an Infisical backend from config or exits.
func newInfisicalBackendOrFail(cfg config.Config) *secrets.InfisicalBackend {
if cfg.InfisicalSiteURL == "" {
fmt.Fprintln(os.Stderr, "error: OIKOS_INFISICAL_SITE_URL not set")
os.Exit(1)
}
infCfg := secrets.InfisicalConfig{
SiteURL: cfg.InfisicalSiteURL,
ClientID: cfg.InfisicalClientID,
ClientSecret: cfg.InfisicalClientSecret,
ProjectID: cfg.InfisicalProjectID,
SecretPath: "/",
Env: cfg.InfisicalEnv,
}
if infCfg.Env == "" {
infCfg.Env = "dev"
}
return secrets.NewInfisicalBackend(infCfg)
}
// runSecretLegacy handles SOPS-only commands (migrate, export-sops).
func runSecretLegacy(ctx context.Context, cfg config.Config, sub string) {
secretsDir := cfg.SecretsDir
if secretsDir == "" {
secretsDir = "archive/secrets-sops-backup"
}
sopsBackend := secrets.NewSOPSBackend(secretsDir)
switch sub {
case "migrate":
infBackend := newInfisicalBackendOrFail(cfg)
keys, err := sopsBackend.List(ctx)
if err != nil {
slog.Error("migrate: read sops", "error", err)
@@ -366,10 +537,6 @@ func runSecret(ctx context.Context, cfg config.Config) {
fmt.Printf("%s: <sops-encrypted>\n", k)
}
fmt.Printf("\n# To restore: sops -d secrets/*.yaml\n")
default:
fmt.Fprintf(os.Stderr, "unknown secret command: %s\n", sub)
os.Exit(1)
}
}

View File

@@ -1,6 +1,7 @@
package main
import (
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
@@ -12,26 +13,31 @@ import (
"os/exec"
"time"
"github.com/dtoro/oikos/internal/secrets"
"github.com/dtoro/oikos/internal/safego"
)
func main() {
ctx := context.Background()
port := os.Getenv("WEBHOOK_LISTEN")
if port == "" {
port = ":9797"
}
secret := os.Getenv("WEBHOOK_HMAC_SECRET")
if secret == "" {
fmt.Fprintln(os.Stderr, "WEBHOOK_HMAC_SECRET must be set")
os.Exit(1)
}
repoDir := os.Getenv("WEBHOOK_REPO_DIR")
if repoDir == "" {
repoDir = os.Getenv("HOME") + "/Projects/oikos"
}
// Create secrets manager once, share between HMAC resolution and deploy
sec := newSecrets()
secret := resolveWebhookHMAC(ctx, sec)
if secret == "" {
fmt.Fprintln(os.Stderr, "WEBHOOK_HMAC_SECRET must be set (env var or Infisical webhook_hmac-secret)")
os.Exit(1)
}
mux := http.NewServeMux()
mux.HandleFunc("/deploy", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
@@ -66,11 +72,16 @@ func main() {
w.Write([]byte(`{"status":"deploy started"}`))
safego.Go("webhook:deploy", func() {
apiToken := ""
if sec != nil {
apiToken = secrets.ResolveSecret(ctx, sec, "api_token", "")
}
cmd := exec.Command(repoDir + "/scripts/deploy.sh")
cmd.Dir = repoDir
cmd.Env = append(os.Environ(),
"REPO_DIR="+repoDir,
"PROFILE=full",
"OIKOS_API_TOKEN="+apiToken,
)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
@@ -94,3 +105,25 @@ func main() {
os.Exit(1)
}
}
// newSecrets creates the Infisical secrets manager from env vars.
func newSecrets() *secrets.Manager {
return secrets.NewManagerFromConfig(
os.Getenv("OIKOS_INFISICAL_SITE_URL"),
os.Getenv("OIKOS_INFISICAL_CLIENT_ID"),
os.Getenv("OIKOS_INFISICAL_CLIENT_SECRET"),
os.Getenv("OIKOS_INFISICAL_PROJECT_ID"),
os.Getenv("OIKOS_INFISICAL_ENV"),
os.Getenv("OIKOS_SECRETS_DIR"),
)
}
// resolveWebhookHMAC fetches the webhook HMAC secret from Infisical,
// falling back to the WEBHOOK_HMAC_SECRET env var.
func resolveWebhookHMAC(ctx context.Context, sec *secrets.Manager) string {
envFallback := os.Getenv("WEBHOOK_HMAC_SECRET")
if sec == nil {
return envFallback
}
return secrets.ResolveSecret(ctx, sec, "webhook_hmac-secret", envFallback)
}

View File

@@ -8,7 +8,8 @@ FROM node:22-alpine AS builder
WORKDIR /build/web
COPY web/package.json web/package-lock.json ./
RUN npm ci
COPY web/vendor /build/vendor
RUN npm install --no-audit --no-fund
COPY VERSION ./
COPY web/ ./
RUN npm run build

View File

@@ -20,6 +20,8 @@ services:
- "5432:5432"
volumes:
- pg-data:/var/lib/postgresql/data
mem_limit: 1g
cpus: 2.0
healthcheck:
test: ["CMD", "pg_isready", "-U", "oikos"]
interval: 5s
@@ -28,6 +30,7 @@ services:
# One-shot: run migrations then exit
migrate:
image: oikos-migrate:${OIKOS_VERSION:-latest}
build:
context: .
dockerfile: compose/oikos/Dockerfile
@@ -38,9 +41,12 @@ services:
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
command: ["migrate"]
restart: "no"
mem_limit: 512m
cpus: 1.0
# One-shot: ingest seeds then exit
seed:
image: oikos-seed:${OIKOS_VERSION:-latest}
build:
context: .
dockerfile: compose/oikos/Dockerfile
@@ -52,9 +58,12 @@ services:
OIKOS_SEEDS_DIR: /seeds
command: ["seed"]
restart: "no"
mem_limit: 512m
cpus: 1.0
# API server (Phase 2)
api:
image: oikos-api:${OIKOS_VERSION:-latest}
build:
context: .
dockerfile: compose/oikos/Dockerfile
@@ -76,6 +85,16 @@ services:
OIKOS_OIDC_CLIENT_ID: ${OIKOS_OIDC_CLIENT_ID:-otkHBSueHJsYtOHstL6rn5izeGgyOsavp1qA1hod}
OIKOS_NOMOS_AGENT_SLUG: ${OIKOS_NOMOS_AGENT_SLUG:-agent:nomos}
NOMOS_PROXY_URL: http://nomos:8092
# Rate limiting (plan D3). Default off; set OIKOS_API_RATE_LIMIT to a
# requests/sec value to throttle runaway agent loops per source IP.
OIKOS_API_RATE_LIMIT: ${OIKOS_API_RATE_LIMIT:-}
OIKOS_API_RATE_BURST: ${OIKOS_API_RATE_BURST:-}
# Infisical secret store (Phase 5)
OIKOS_INFISICAL_SITE_URL: ${OIKOS_INFISICAL_SITE_URL:-}
OIKOS_INFISICAL_CLIENT_ID: ${OIKOS_INFISICAL_CLIENT_ID:-}
OIKOS_INFISICAL_CLIENT_SECRET: ${OIKOS_INFISICAL_CLIENT_SECRET:-}
OIKOS_INFISICAL_PROJECT_ID: ${OIKOS_INFISICAL_PROJECT_ID:-}
OIKOS_INFISICAL_ENV: ${OIKOS_INFISICAL_ENV:-dev}
volumes:
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
ports:
@@ -83,6 +102,8 @@ services:
command: ["api"]
stop_signal: SIGTERM
stop_grace_period: 30s
mem_limit: 512m
cpus: 1.0
# Exists so nomos can wait for the API to actually answer rather than just
# for its container to exist — see nomos's depends_on below. wget is
# BusyBox's, already in the alpine runtime image, so this adds no
@@ -94,10 +115,15 @@ services:
retries: 10
# Migrations and seed run before this container, but the first bind can
# still take a moment; failures inside the start period don't count.
start_period: 10s
# The api's NewHandler stalls on TWO unreachable external deps at startup
# before binding :8090: Infisical (4x auth retries, ~40s) and OIDC
# discovery (auth.hubris.network, ~35s of timeouts). Total ~90-95s, so
# the start period must clear it or nomos (depends_on: api-healthy) fails.
start_period: 180s
# Scheduler (Phase 3) — observe loop
scheduler:
image: oikos-scheduler:${OIKOS_VERSION:-latest}
build:
context: .
dockerfile: compose/oikos/Dockerfile
@@ -112,6 +138,9 @@ services:
OIKOS_SCHEDULER_INTERVAL: "30s"
OIKOS_SSH_KEY_PATH: /etc/oikos/ssh_key
OIKOS_SSH_USER: root
# Liveness probe (plan D5): exposes a staleness-aware /healthz inside
# the container; the scheduler bumps it each check pass.
OIKOS_HEALTH_LISTEN: ":8093"
volumes:
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
cap_add:
@@ -119,9 +148,18 @@ services:
command: ["scheduler"]
stop_signal: SIGTERM
stop_grace_period: 30s
mem_limit: 256m
cpus: 1.0
healthcheck:
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8093/healthz"]
interval: 30s
timeout: 5s
retries: 3
start_period: 90s
# Notifier (Phase 3) — Matrix alerts
notifier:
image: oikos-notifier:${OIKOS_VERSION:-latest}
build:
context: .
dockerfile: compose/oikos/Dockerfile
@@ -138,12 +176,59 @@ services:
OIKOS_MATRIX_USER: ${OIKOS_MATRIX_USER:-@hermes:hubris.network}
OIKOS_MATRIX_TOKEN: ${OIKOS_MATRIX_TOKEN}
OIKOS_MATRIX_ROOM: ${OIKOS_MATRIX_ROOM:-!alerts:hubris.network}
# Liveness probe (plan D5): bumps each approval/reaction tick.
OIKOS_HEALTH_LISTEN: ":8094"
OIKOS_INFISICAL_SITE_URL: ${OIKOS_INFISICAL_SITE_URL:-}
OIKOS_INFISICAL_CLIENT_ID: ${OIKOS_INFISICAL_CLIENT_ID:-}
OIKOS_INFISICAL_CLIENT_SECRET: ${OIKOS_INFISICAL_CLIENT_SECRET:-}
OIKOS_INFISICAL_PROJECT_ID: ${OIKOS_INFISICAL_PROJECT_ID:-}
OIKOS_INFISICAL_ENV: ${OIKOS_INFISICAL_ENV:-dev}
command: ["notifier"]
stop_signal: SIGTERM
stop_grace_period: 30s
mem_limit: 128m
cpus: 0.5
healthcheck:
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8094/healthz"]
interval: 30s
timeout: 5s
retries: 3
start_period: 120s
# Execution worker (Phase 6) — Postgres-backed job queue
execution-worker:
image: oikos-execution-worker:${OIKOS_VERSION:-latest}
build:
context: .
dockerfile: compose/oikos/Dockerfile
restart: unless-stopped
profiles: ["dev", "full"]
depends_on:
seed:
condition: service_completed_successfully
environment:
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
OIKOS_DEBUG: "true"
OIKOS_SSH_KEY_PATH: /etc/oikos/ssh_key
OIKOS_SSH_USER: root
OIKOS_HEALTH_LISTEN: ":8095"
volumes:
- ${OIKOS_SSH_KEY_PATH:-~/.ssh/id_ed25519}:/etc/oikos/ssh_key:ro
command: ["execution-worker"]
stop_signal: SIGTERM
stop_grace_period: 30s
mem_limit: 256m
cpus: 1.0
healthcheck:
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8095/healthz"]
interval: 30s
timeout: 5s
retries: 3
start_period: 90s
# Nomos agent gateway (Phase 4) — mesh-published :8092
nomos:
image: oikos-nomos:${OIKOS_VERSION:-latest}
build:
context: .
dockerfile: compose/nomos/Dockerfile
@@ -166,16 +251,34 @@ services:
# Must match api's OIKOS_MCP_BEARER_TOKEN above — api's combinedAuth
# rejects every request without it now (no dev-open bypass).
OIKOS_MCP_BEARER_TOKEN: ${OIKOS_MCP_BEARER_TOKEN:-dev-token}
# Infisical secret store (Phase 5) — nomos resolves mcp_bearer-token
# and openrouter_api-key from here, overriding the env values above.
OIKOS_INFISICAL_SITE_URL: ${OIKOS_INFISICAL_SITE_URL:-}
OIKOS_INFISICAL_CLIENT_ID: ${OIKOS_INFISICAL_CLIENT_ID:-}
OIKOS_INFISICAL_CLIENT_SECRET: ${OIKOS_INFISICAL_CLIENT_SECRET:-}
OIKOS_INFISICAL_PROJECT_ID: ${OIKOS_INFISICAL_PROJECT_ID:-}
OIKOS_INFISICAL_ENV: ${OIKOS_INFISICAL_ENV:-dev}
ports:
- "8092:8092"
stop_signal: SIGTERM
stop_grace_period: 10s
mem_limit: 512m
cpus: 1.0
# nomos runs on a distroless image (no shell/wget), so the healthcheck
# uses the binary's own `healthcheck` subcommand to self-probe /healthz.
healthcheck:
test: ["CMD", "/nomos", "healthcheck"]
interval: 30s
timeout: 5s
retries: 3
start_period: 15s
# Control-room SPA — static build served behind Caddy. The outer
# production Caddy (caddy-conf repo, LXC 121) splits /api/*, /mcp,
# /agent/* off to api:8090 and sends everything else here; this
# container only serves static files with SPA-fallback routing.
web:
image: oikos-web:${OIKOS_VERSION:-latest}
build:
context: .
dockerfile: compose/web/Dockerfile
@@ -184,6 +287,8 @@ services:
ports:
- "8091:80"
stop_signal: SIGTERM
mem_limit: 64m
cpus: 0.25
# Redis (required by Infisical — Phase 5)
redis:
@@ -192,6 +297,8 @@ services:
profiles: ["infisical", "full"]
volumes:
- redis-data:/data
mem_limit: 128m
cpus: 0.5
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
@@ -200,7 +307,7 @@ services:
# Infisical self-hosted (Phase 5 secrets management)
infisical:
image: infisical/infisical:latest
image: infisical/infisical:v0.162.19
restart: unless-stopped
profiles: ["infisical", "full"]
depends_on:
@@ -222,6 +329,8 @@ services:
REDIS_URL: redis://redis:6379
ports:
- "8080:8080"
mem_limit: 512m
cpus: 1.0
volumes:
pg-data:

2
go.mod
View File

@@ -19,6 +19,7 @@ require (
golang.org/x/crypto v0.53.0
golang.org/x/sync v0.21.0
golang.org/x/sys v0.46.0
golang.org/x/time v0.14.0
gopkg.in/yaml.v3 v3.0.1
)
@@ -88,7 +89,6 @@ require (
golang.org/x/net v0.55.0 // indirect
golang.org/x/oauth2 v0.35.0 // indirect
golang.org/x/text v0.38.0 // indirect
golang.org/x/time v0.14.0 // indirect
google.golang.org/api v0.267.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260128011058-8636f8732409 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260203192932-546029d2fa20 // indirect

View File

@@ -17,7 +17,6 @@ import (
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/google/uuid"
"golang.org/x/crypto/ssh"
)
// Run starts the actuator loop. Blocks until ctx is cancelled.
@@ -403,7 +402,7 @@ func ProvisionVM(ctx context.Context, pool *db.Pool, entityID uuid.UUID, attrs m
return nil
}
// sshExecSimple runs a command over SSH with a simple client setup.
// sshExecSimple runs a command over SSH using the shared dial/run primitives.
// Uses the default SSH key from SSH_KEY_PATH or ~/.ssh/id_rsa.
func sshExecSimple(ctx context.Context, host, user, command string) (string, error) {
keyPath := os.Getenv("SSH_KEY_PATH")
@@ -411,55 +410,19 @@ func sshExecSimple(ctx context.Context, host, user, command string) (string, err
keyPath = os.Getenv("HOME") + "/.ssh/id_rsa"
}
keyBytes, err := os.ReadFile(keyPath)
signer, err := LoadSigner(keyPath)
if err != nil {
return "", fmt.Errorf("read ssh key: %w", err)
return "", err
}
signer, err := ssh.ParsePrivateKey(keyBytes)
client, err := Dial(ctx, DialOptions{Host: host, User: user, Signer: signer})
if err != nil {
return "", fmt.Errorf("parse ssh key: %w", err)
}
clientCfg := &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Timeout: 10 * time.Second,
}
client, err := ssh.Dial("tcp", host+":22", clientCfg)
if err != nil {
return "", fmt.Errorf("ssh dial %s: %w", host, err)
return "", err
}
defer client.Close()
session, err := client.NewSession()
if err != nil {
return "", fmt.Errorf("create session: %w", err)
}
defer session.Close()
type result struct {
output string
err error
}
ch := make(chan result, 1)
go func() {
out, e := session.CombinedOutput(command)
ch <- result{output: string(out), err: e}
}()
select {
case <-ctx.Done():
session.Close()
return "", ctx.Err()
case res := <-ch:
if res.err != nil {
return res.output, res.err
}
return res.output, nil
}
out, err := RunCombinedOutput(ctx, client, command)
return string(out), err
}
// resolveHost resolves a host entity slug to (address, user) for SSH.

141
internal/actuator/client.go Normal file
View File

@@ -0,0 +1,141 @@
package actuator
import (
"bytes"
"context"
"fmt"
"net"
"os"
"strings"
"time"
"golang.org/x/crypto/ssh"
)
// defaultDialTimeout bounds an SSH dial when the caller leaves Timeout unset.
// 10s matches the previous hardcoded value at every dial site.
const defaultDialTimeout = 10 * time.Second
// LoadSigner reads and parses the private key at keyPath.
func LoadSigner(keyPath string) (ssh.Signer, error) {
key, err := os.ReadFile(keyPath)
if err != nil {
return nil, fmt.Errorf("read ssh key: %w", err)
}
return LoadSignerFromBytes(key)
}
// LoadSignerFromBytes parses an in-memory private key into an ssh.Signer.
func LoadSignerFromBytes(key []byte) (ssh.Signer, error) {
signer, err := ssh.ParsePrivateKey(key)
if err != nil {
return nil, fmt.Errorf("parse ssh key: %w", err)
}
return signer, nil
}
// DialOptions configures an SSH dial.
type DialOptions struct {
Host string
Port int // 0 means 22
User string
Signer ssh.Signer
Timeout time.Duration // dial timeout; <=0 means defaultDialTimeout
}
// Dial opens a crypto/ssh connection through the centralized HostKeyCallback.
// The connection itself is bounded by Timeout; ctx is respected by callers
// via RunCombinedOutput once the session is running.
func Dial(ctx context.Context, opts DialOptions) (*ssh.Client, error) {
port := opts.Port
if port <= 0 {
port = 22
}
timeout := opts.Timeout
if timeout <= 0 {
timeout = defaultDialTimeout
}
cfg := &ssh.ClientConfig{
User: opts.User,
Auth: []ssh.AuthMethod{ssh.PublicKeys(opts.Signer)},
HostKeyCallback: HostKeyCallback(),
Timeout: timeout,
}
addr := net.JoinHostPort(opts.Host, fmt.Sprintf("%d", port))
client, err := ssh.Dial("tcp", addr, cfg)
if err != nil {
return nil, fmt.Errorf("ssh dial %s:%d: %w", opts.Host, port, err)
}
return client, nil
}
// RunCombinedOutput runs cmd on an established client and returns its combined
// stdout/stderr. Context cancellation closes the session to abort the remote
// command instead of blocking until it finishes — the same goroutine+select
// pattern the actuator, mcp, and scheduler each reimplemented before.
func RunCombinedOutput(ctx context.Context, client *ssh.Client, cmd string) ([]byte, error) {
session, err := client.NewSession()
if err != nil {
return nil, fmt.Errorf("create session: %w", err)
}
defer session.Close()
type result struct {
out []byte
err error
}
ch := make(chan result, 1)
go func() {
out, err := session.CombinedOutput(cmd)
ch <- result{out: out, err: err}
}()
select {
case <-ctx.Done():
session.Close()
return nil, ctx.Err()
case res := <-ch:
if res.err != nil {
return res.out, fmt.Errorf("command: %w", res.err)
}
return res.out, nil
}
}
// RunOutput runs cmd on an established client and returns stdout only.
// Stderr is folded into the returned error so callers that parse stdout
// as JSON (e.g. the scheduler's check scripts) don't get interleaved
// stderr in the output stream.
func RunOutput(ctx context.Context, client *ssh.Client, cmd string) ([]byte, error) {
session, err := client.NewSession()
if err != nil {
return nil, fmt.Errorf("create session: %w", err)
}
defer session.Close()
var outBuf, errBuf bytes.Buffer
session.Stdout = &outBuf
session.Stderr = &errBuf
type result struct {
runErr error
}
ch := make(chan result, 1)
go func() {
ch <- result{runErr: session.Run(cmd)}
}()
select {
case <-ctx.Done():
session.Close()
return nil, ctx.Err()
case res := <-ch:
if res.runErr != nil {
if errBuf.Len() > 0 {
return outBuf.Bytes(), fmt.Errorf("command: %w\nstderr: %s", res.runErr, strings.TrimSpace(errBuf.String()))
}
return outBuf.Bytes(), fmt.Errorf("command: %w", res.runErr)
}
return outBuf.Bytes(), nil
}
}

View File

@@ -0,0 +1,88 @@
package actuator
import (
"bytes"
"context"
"crypto/ed25519"
"crypto/rand"
"encoding/pem"
"os"
"path/filepath"
"strings"
"testing"
"golang.org/x/crypto/ssh"
)
func TestLoadSignerRejectsBadInput(t *testing.T) {
if _, err := LoadSignerFromBytes([]byte("not a private key")); err == nil {
t.Error("LoadSignerFromBytes should reject a non-key input")
}
if _, err := LoadSigner("/nonexistent/key"); err == nil {
t.Error("LoadSigner should fail on a missing file")
}
}
func TestLoadSignerRoundTrip(t *testing.T) {
_, priv, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatalf("generate key: %v", err)
}
block, err := ssh.MarshalPrivateKey(priv, "")
if err != nil {
t.Fatalf("marshal private key: %v", err)
}
pemBytes := pem.EncodeToMemory(block)
signer, err := LoadSignerFromBytes(pemBytes)
if err != nil {
t.Fatalf("LoadSignerFromBytes on a valid key: %v", err)
}
if signer == nil {
t.Fatal("signer is nil")
}
dir := t.TempDir()
path := filepath.Join(dir, "id_ed25519")
if err := os.WriteFile(path, pemBytes, 0o600); err != nil {
t.Fatalf("write key file: %v", err)
}
fromFile, err := LoadSigner(path)
if err != nil {
t.Fatalf("LoadSigner(%s): %v", path, err)
}
if !bytes.Equal(fromFile.PublicKey().Marshal(), signer.PublicKey().Marshal()) {
t.Error("file and in-memory signers resolved to different public keys")
}
}
// Dial needs a real SSH server to run a command, but its option normalization
// is verifiable without one: a zero Port must default to 22 (so the dial error
// references host:22, not host:0), and a closed port yields a dial error rather
// than panicking.
func TestDialDefaultsPort(t *testing.T) {
_, err := Dial(context.Background(), DialOptions{Host: "127.0.0.1", Signer: mustSigner(t)})
if err == nil {
t.Fatal("Dial to a closed port should fail")
}
if !strings.Contains(err.Error(), "127.0.0.1:22") {
t.Errorf("Dial error = %q, want it to reference 127.0.0.1:22", err)
}
}
func mustSigner(t *testing.T) ssh.Signer {
t.Helper()
_, priv, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatalf("generate key: %v", err)
}
block, err := ssh.MarshalPrivateKey(priv, "")
if err != nil {
t.Fatalf("marshal key: %v", err)
}
s, err := LoadSignerFromBytes(pem.EncodeToMemory(block))
if err != nil {
t.Fatalf("parse key: %v", err)
}
return s
}

View File

@@ -0,0 +1,129 @@
package actuator
import (
"bytes"
"context"
"fmt"
"log/slog"
"net"
"sync"
"time"
"golang.org/x/crypto/ssh"
)
var (
hostKeyMu sync.RWMutex
hostKeyCache map[string]ssh.PublicKey
hostKeyOnce sync.Once
hostKeySrc HostKeySource
)
// HostKeySource provides storage for SSH host public keys.
type HostKeySource interface {
GetHostKey(ctx context.Context, hostname string) (string, error)
SetHostKey(ctx context.Context, hostname string, key string) error
}
// SetHostKeySource sets the host key source. Must be called before
// any SSH connections. A nil source enables TOFU-only mode (keys
// accepted in memory but not persisted).
func SetHostKeySource(src HostKeySource) {
hostKeyMu.Lock()
defer hostKeyMu.Unlock()
hostKeySrc = src
}
// HostKeyCallback returns an ssh.HostKeyCallback that verifies host keys.
// Known keys are verified (MITM detection). Unknown keys are accepted
// via TOFU and optionally persisted to the source.
func HostKeyCallback() ssh.HostKeyCallback {
return hostKeyVerify
}
func hostKeyVerify(hostname string, remote net.Addr, key ssh.PublicKey) error {
hostKeyOnce.Do(func() {
hostKeyCache = make(map[string]ssh.PublicKey)
})
normalized := hostWithoutPort(hostname)
hostKeyMu.RLock()
known, exists := hostKeyCache[normalized]
hostKeyMu.RUnlock()
if exists {
if bytes.Equal(key.Marshal(), known.Marshal()) {
return nil
}
return fmt.Errorf("SSH HOST KEY CHANGED for %s (possible MITM)", normalized)
}
hostKeyMu.Lock()
hostKeyCache[normalized] = key
hostKeyMu.Unlock()
slog.Info("ssh: accepting new host key (TOFU)", "host", normalized)
if hostKeySrc != nil {
go persistHostKey(normalized, key)
}
return nil
}
func persistHostKey(hostname string, key ssh.PublicKey) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
keyBase64 := key.Type() + " " + string(key.Marshal())
if err := hostKeySrc.SetHostKey(ctx, "ssh/host-keys/"+hostname, keyBase64); err != nil {
slog.Warn("ssh: failed to persist host key", "host", hostname, "error", err)
}
}
// LoadHostKeys pre-loads known host keys from the source into the
// in-memory cache. Call at startup to avoid TOFU on first connection.
// The source should return key lines in the format "key-type base64-data".
func LoadHostKeys(ctx context.Context, hostnames []string, src HostKeySource) {
if src == nil {
return
}
SetHostKeySource(src)
hostKeyMu.Lock()
defer hostKeyMu.Unlock()
if hostKeyCache == nil {
hostKeyCache = make(map[string]ssh.PublicKey)
}
loaded := 0
for _, hostname := range hostnames {
keyData, err := src.GetHostKey(ctx, "ssh/host-keys/"+hostname)
if err != nil {
slog.Debug("ssh: no stored key for host", "host", hostname, "error", err)
continue
}
pubKey, _, _, _, err := ssh.ParseAuthorizedKey([]byte(keyData))
if err != nil {
slog.Warn("ssh: invalid stored key for host", "host", hostname, "error", err)
continue
}
hostKeyCache[hostname] = pubKey
loaded++
}
if loaded > 0 {
slog.Info("ssh: loaded host keys from Infisical", "count", loaded)
}
}
func hostWithoutPort(hostname string) string {
for i := len(hostname) - 1; i >= 0; i-- {
if hostname[i] == ':' {
return hostname[:i]
}
}
return hostname
}

View File

@@ -0,0 +1,56 @@
package actuator
import (
"context"
"log/slog"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/secrets"
)
// InfisicalHostKeySource implements HostKeySource backed by Infisical.
type InfisicalHostKeySource struct {
sec secrets.Backend
}
// NewInfisicalHostKeySource creates a HostKeySource that reads/writes
// SSH host public keys from Infisical under the `ssh/host-keys/` prefix.
func NewInfisicalHostKeySource(sec secrets.Backend) *InfisicalHostKeySource {
return &InfisicalHostKeySource{sec: sec}
}
func (s *InfisicalHostKeySource) GetHostKey(ctx context.Context, path string) (string, error) {
val, err := s.sec.Get(ctx, path)
if err != nil {
return "", err
}
return val, nil
}
func (s *InfisicalHostKeySource) SetHostKey(ctx context.Context, path string, key string) error {
return s.sec.Set(ctx, path, key)
}
// ResolveSSHHosts queries the DB for active proxmox-host and standalone-server
// entities, returning their slugs as SSH host identifiers.
func ResolveSSHHosts(ctx context.Context, pool *db.Pool) []string {
rows, err := pool.Query(ctx, `
SELECT slug FROM entities
WHERE type IN ('proxmox-host', 'standalone-server')
AND state = 'active'
ORDER BY slug`)
if err != nil {
slog.Warn("ssh: failed to list hosts", "error", err)
return nil
}
defer rows.Close()
var hosts []string
for rows.Next() {
var slug string
if rows.Scan(&slug) == nil {
hosts = append(hosts, slug)
}
}
return hosts
}

122
internal/actuator/pool.go Normal file
View File

@@ -0,0 +1,122 @@
package actuator
import (
"context"
"fmt"
"sync"
"time"
"golang.org/x/crypto/ssh"
)
type poolEntry struct {
client *ssh.Client
createdAt time.Time
}
type DialPool struct {
mu sync.RWMutex
entries map[string]*poolEntry
ttl time.Duration
done chan struct{}
stopped bool
}
func NewDialPool(ttl time.Duration) *DialPool {
p := &DialPool{
entries: make(map[string]*poolEntry),
ttl: ttl,
done: make(chan struct{}),
}
if ttl > 0 {
go p.evictLoop()
}
return p
}
func (p *DialPool) key(opts DialOptions) string {
port := opts.Port
if port <= 0 {
port = 22
}
return fmt.Sprintf("%s:%d", opts.Host, port)
}
func (p *DialPool) Get(ctx context.Context, opts DialOptions) (*ssh.Client, error) {
k := p.key(opts)
p.mu.RLock()
entry, ok := p.entries[k]
p.mu.RUnlock()
if ok {
// Quick health check: a session can be created without running a
// command — if it fails, the connection is dead and we evict it.
testSession, err := entry.client.NewSession()
if err == nil {
testSession.Close()
return entry.client, nil
}
p.mu.Lock()
if p.entries[k] == entry {
entry.client.Close()
delete(p.entries, k)
}
p.mu.Unlock()
// Fall through to dial below
}
client, err := Dial(ctx, opts)
if err != nil {
return nil, err
}
p.mu.Lock()
if p.stopped {
p.mu.Unlock()
client.Close()
return nil, fmt.Errorf("ssh dial pool: closed")
}
if existing, ok2 := p.entries[k]; ok2 {
p.mu.Unlock()
client.Close()
return existing.client, nil
}
p.entries[k] = &poolEntry{client: client, createdAt: time.Now()}
p.mu.Unlock()
return client, nil
}
func (p *DialPool) Close() {
p.mu.Lock()
p.stopped = true
for k, entry := range p.entries {
entry.client.Close()
delete(p.entries, k)
}
p.mu.Unlock()
if p.ttl > 0 {
close(p.done)
}
}
func (p *DialPool) evictLoop() {
ticker := time.NewTicker(p.ttl / 2)
defer ticker.Stop()
for {
select {
case <-p.done:
return
case <-ticker.C:
p.evict()
}
}
}
func (p *DialPool) evict() {
deadline := time.Now().Add(-p.ttl)
p.mu.Lock()
defer p.mu.Unlock()
for k, entry := range p.entries {
if entry.createdAt.Before(deadline) {
entry.client.Close()
delete(p.entries, k)
}
}
}

View File

@@ -11,7 +11,6 @@ import (
"fmt"
"log/slog"
"net"
"os"
"strings"
"sync"
"time"
@@ -131,37 +130,19 @@ func ExecuteProcedure(
start := time.Now()
// Parse the SSH key
key, err := os.ReadFile(cfg.KeyPath)
signer, err := LoadSigner(cfg.KeyPath)
if err != nil {
return SSHResult{
Err: fmt.Errorf("read ssh key: %w", err),
Err: err,
Duration: time.Since(start),
Verified: false,
}
}
signer, err := ssh.ParsePrivateKey(key)
if err != nil {
return SSHResult{
Err: fmt.Errorf("parse ssh key: %w", err),
Duration: time.Since(start),
Verified: false,
}
}
addr := net.JoinHostPort(cfg.Host, fmt.Sprintf("%d", cfg.Port))
if cfg.Port == 0 {
addr = net.JoinHostPort(cfg.Host, "22")
}
clientCfg := &ssh.ClientConfig{
User: cfg.User,
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(), // restricted key; host trust via inventory
Timeout: cfg.Timeout,
}
client, err := ssh.Dial("tcp", addr, clientCfg)
client, err := Dial(ctx, DialOptions{
Host: cfg.Host, Port: cfg.Port, User: cfg.User,
Signer: signer, Timeout: cfg.Timeout,
})
if err != nil {
class := classifySSHError(err)
return SSHResult{
@@ -229,38 +210,11 @@ func ExecuteProcedure(
}
}
// runSSHCommand executes a single command over an established SSH session.
// Uses context-aware goroutines: ctx.Done() closes the session.
// runSSHCommand executes a single command over an established SSH session via
// the shared RunCombinedOutput primitive (context-aware abort + combined output).
func runSSHCommand(ctx context.Context, client *ssh.Client, command string) (string, error) {
session, err := client.NewSession()
if err != nil {
return "", fmt.Errorf("create session: %w", err)
}
defer session.Close()
// Wrap in goroutine so we can abort on ctx.Done()
type result struct {
output string
err error
}
ch := make(chan result, 1)
go func() {
out, err := session.CombinedOutput(command)
ch <- result{output: string(out), err: err}
}()
select {
case <-ctx.Done():
// Close the session to abort the SSH command
session.Close()
return "", ctx.Err()
case res := <-ch:
if res.err != nil {
return res.output, fmt.Errorf("command: %w", res.err)
}
return res.output, nil
}
out, err := RunCombinedOutput(ctx, client, command)
return string(out), err
}
// ─── Procedure parsing ────────────────────────────────────────────────────

View File

@@ -0,0 +1,86 @@
package actuator
import (
"bufio"
"context"
"fmt"
"io"
"time"
"golang.org/x/crypto/ssh"
)
// RunStreaming runs a command on an established SSH client and forwards output
// chunks to sink as they arrive. A nil sink collects output silently. Returns
// the full combined output and any command error.
func RunStreaming(ctx context.Context, client *ssh.Client, command string, sink func(stream string, chunk []byte), timeout time.Duration) (string, error) {
session, err := client.NewSession()
if err != nil {
return "", fmt.Errorf("create session: %w", err)
}
defer session.Close()
outPipe, err := session.StdoutPipe()
if err != nil {
return "", fmt.Errorf("stdout pipe: %w", err)
}
errPipe, err := session.StderrPipe()
if err != nil {
return "", fmt.Errorf("stderr pipe: %w", err)
}
type streamResult struct {
out string
err error
}
resultCh := make(chan streamResult, 1)
go func() {
var combined []byte
done := make(chan struct{}, 2)
readStream := func(stream string, r io.Reader) {
sc := bufio.NewScanner(r)
for sc.Scan() {
line := sc.Bytes()
chunk := make([]byte, len(line))
copy(chunk, line)
if sink != nil {
sink(stream, chunk)
}
if stream == "stdout" || stream == "" {
if len(combined) > 0 {
combined = append(combined, '\n')
}
combined = append(combined, chunk...)
}
}
done <- struct{}{}
}
go readStream("stdout", outPipe)
go readStream("stderr", errPipe)
runErr := session.Run(command)
<-done
<-done
resultCh <- streamResult{out: string(combined), err: runErr}
}()
if timeout > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, timeout)
defer cancel()
}
select {
case <-ctx.Done():
session.Close()
return "", ctx.Err()
case res := <-resultCh:
if res.err != nil {
return res.out, fmt.Errorf("command: %w", res.err)
}
return res.out, nil
}
}

View File

@@ -0,0 +1,202 @@
package checkdefaults
import (
"reflect"
"strings"
"testing"
"github.com/dtoro/oikos/internal/ontology"
)
// Table-driven coverage of every implemented buildKind branch and the ssh()
// helper's user/port/args propagation. The previous tests exercised only
// ping/process/http/resource; updates, capacity, backup, cert-expiry,
// vm-status and dns were unverified.
func TestBuildKindAllImplementedKinds(t *testing.T) {
host := "10.0.0.5"
cases := []struct {
name string
kind string
target Target
attrs map[string]any
host string
wantSkip bool // true → expect a reason and zero defs
wantDefs int
wantKind string
wantKey string // a config key to assert
wantVal any // its expected value
wantReason string // substring when skipping
wantInterv int32 // expected interval on the (single) produced def
}{
{
name: "ping with host", kind: KindPing, host: host,
wantDefs: 1, wantKind: "ping", wantKey: "host", wantVal: host, wantInterv: 30,
},
{name: "ping no host skips", kind: KindPing, wantSkip: true, wantReason: "no address"},
{
name: "resource expands to four ssh scripts", kind: KindResource, host: host,
wantDefs: 4, wantKind: "ssh-script", wantKey: "host", wantVal: host, wantInterv: 60,
},
{name: "resource no host skips", kind: KindResource, wantSkip: true, wantReason: "no address"},
{
name: "updates is daily", kind: KindUpdates, host: host,
wantDefs: 1, wantKind: "ssh-script", wantKey: "script", wantVal: "updates_check.sh", wantInterv: 86400,
},
{name: "updates no host skips", kind: KindUpdates, wantSkip: true, wantReason: "no address"},
{
name: "capacity is one disk script", kind: KindCapacity, host: host,
wantDefs: 1, wantKind: "ssh-script", wantKey: "script", wantVal: "disk_usage_check.sh", wantInterv: 60,
},
{name: "capacity no host skips", kind: KindCapacity, wantSkip: true, wantReason: "no address"},
{
name: "backup needs path and host", kind: KindBackup, host: host,
attrs: map[string]any{"path": "/backups/db"},
wantDefs: 1, wantKind: "backup-freshness", wantKey: "path", wantVal: "/backups/db", wantInterv: 86400,
},
{name: "backup without path skips", kind: KindBackup, host: host, wantSkip: true, wantReason: "no path"},
{name: "backup without host skips", kind: KindBackup, attrs: map[string]any{"path": "/x"}, wantSkip: true, wantReason: "no address"},
{
name: "backup honors backup_max_age_s override", kind: KindBackup, host: host,
attrs: map[string]any{"path": "/x", "backup_max_age_s": float64(3600)},
wantDefs: 1, wantKey: "max_age_s", wantVal: 3600,
},
{
name: "cert-expiry from hostname attr", kind: KindCertExpiry,
attrs: map[string]any{"hostname": "media.hubris.network"},
wantDefs: 1, wantKind: "cert-expiry", wantKey: "host", wantVal: "media.hubris.network", wantInterv: 3600,
},
{
name: "cert-expiry from dotted name", kind: KindCertExpiry, target: Target{Name: "media.hubris.network"},
wantDefs: 1, wantKey: "host", wantVal: "media.hubris.network",
},
{
name: "cert-expiry propagates dial attr", kind: KindCertExpiry,
attrs: map[string]any{"hostname": "media.hubris.network", "dial": "10.0.0.2"},
wantDefs: 1, wantKey: "dial", wantVal: "10.0.0.2",
},
{name: "cert-expiry without a host name skips", kind: KindCertExpiry, target: Target{Name: "jellyfin"}, wantSkip: true, wantReason: "no hostname"},
{
name: "vm-status needs pve_id", kind: KindVMStatus, attrs: map[string]any{"pve_id": float64(101)},
wantDefs: 1, wantKind: "vm-status", wantInterv: 60,
},
{name: "vm-status without pve_id skips", kind: KindVMStatus, wantSkip: true, wantReason: "no pve_id"},
{
name: "dns resolves entity name", kind: KindDNS, target: Target{Name: "hubris.network"},
wantDefs: 1, wantKind: "dns", wantKey: "name", wantVal: "hubris.network", wantInterv: 300,
},
{name: "dns without a name skips", kind: KindDNS, target: Target{}, wantSkip: true, wantReason: "no name"},
{
name: "quorum runs pvecm script via ssh", kind: KindQuorum, host: host,
wantDefs: 1, wantKind: "ssh-script", wantKey: "script", wantVal: "pvecm_quorum_check.sh", wantInterv: 60,
},
{name: "quorum no host skips", kind: KindQuorum, wantSkip: true, wantReason: "no address"},
{name: "unknown kind skips", kind: "telepathy", host: host, wantSkip: true, wantReason: "no builder"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
defs, reason := buildKind(c.kind, c.target, c.attrs, c.host, "root", 22)
if c.wantSkip {
if len(defs) != 0 {
t.Fatalf("expected zero defs, got %d", len(defs))
}
if c.wantReason != "" && !strings.Contains(reason, c.wantReason) {
t.Errorf("reason = %q, want substring %q", reason, c.wantReason)
}
return
}
if len(defs) != c.wantDefs {
t.Fatalf("got %d defs (%s), want %d", len(defs), reason, c.wantDefs)
}
if reason != "" {
t.Errorf("unexpected skip reason: %q", reason)
}
if c.wantKind != "" {
if got := defs[0].kind; got != c.wantKind {
t.Errorf("kind = %q, want %q", got, c.wantKind)
}
}
if c.wantKey != "" {
if got := defs[0].config[c.wantKey]; !reflect.DeepEqual(got, c.wantVal) {
t.Errorf("config[%q] = %v (%T), want %v (%T)", c.wantKey, got, got, c.wantVal, c.wantVal)
}
}
if c.wantInterv != 0 && defs[0].interval != c.wantInterv {
t.Errorf("interval = %d, want %d", defs[0].interval, c.wantInterv)
}
})
}
}
// ssh() must add user/port/args only when they differ from the root/22/empty
// defaults, so generated configs stay minimal and stable across re-seeds.
func TestBuildKindSSHOnlyEmitsNonDefaultUserPortArgs(t *testing.T) {
t.Run("default root 22 omits user and port", func(t *testing.T) {
defs, _ := buildKind(KindResource, Target{}, nil, "10.0.0.1", "root", 22)
for _, d := range defs {
if _, ok := d.config["user"]; ok {
t.Errorf("root should not emit user: %v", d.config)
}
if _, ok := d.config["port"]; ok {
t.Errorf("port 22 should not emit port: %v", d.config)
}
}
})
t.Run("non-root user and non-22 port are emitted", func(t *testing.T) {
defs, _ := buildKind(KindResource, Target{}, nil, "10.0.0.1", "oikos", 2222)
if defs[0].config["user"] != "oikos" {
t.Errorf("user = %v, want oikos", defs[0].config["user"])
}
if defs[0].config["port"] != 2222 {
t.Errorf("port = %v, want 2222", defs[0].config["port"])
}
})
t.Run("process unit name lands in args", func(t *testing.T) {
defs, _ := buildKind(KindProcess, Target{Name: "jellyfin"}, nil, "10.0.0.1", "root", 22)
if defs[0].config["args"] != "jellyfin" {
t.Errorf("args = %v, want jellyfin", defs[0].config["args"])
}
})
}
// resolveMonitoringAttr implements the entity-level `monitoring` override
// (project decision health_checks.monitoring_override): "none"/"" opts out,
// a kind-list replaces the type defaults, anything else falls back.
func TestResolveMonitoringAttr(t *testing.T) {
fallback := ontology.MonitoringResolution{Declared: true, Kinds: []string{"ping"}, Source: "type"}
cases := []struct {
name string
in any
want ontology.MonitoringResolution
}{
{"none opts out", "none", ontology.MonitoringResolution{Declared: true, Source: "attribute"}},
{"empty opts out", "", ontology.MonitoringResolution{Declared: true, Source: "attribute"}},
{
"kind list overrides",
[]any{"http", "process"},
ontology.MonitoringResolution{Declared: true, Kinds: []string{"http", "process"}, Source: "attribute"},
},
{"list drops empty and non-string entries", []any{"http", "", 7, "dns"}, ontology.MonitoringResolution{Declared: true, Kinds: []string{"http", "dns"}, Source: "attribute"}},
{"non-string scalar falls back to type default", float64(42), fallback},
{"nil falls back", nil, fallback},
{"unrecognized string falls back", "weird", fallback},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
got := resolveMonitoringAttr(c.in, fallback)
if !reflect.DeepEqual(got, c.want) {
t.Errorf("resolveMonitoringAttr(%v) = %+v, want %+v", c.in, got, c.want)
}
})
}
}

View File

@@ -33,6 +33,7 @@ const (
KindBackup = "backup-freshness"
KindCertExpiry = "cert-expiry"
KindVMStatus = "vm-status"
KindQuorum = "quorum"
KindDNS = "dns"
)
@@ -360,6 +361,15 @@ func buildKind(kind string, t Target, attrs map[string]any, host, user string, p
config: map[string]any{},
interval: 60,
}}, ""
case KindQuorum:
// Proxmox cluster quorum via `pvecm status`. Only meaningful on
// proxmox-host entities. Runs every 60s — corosync flaps are
// transient and the probe is lightweight (local binary, no network).
if host == "" {
return nil, "no address on the entity or its host"
}
return []checkDef{ssh("pvecm_quorum_check.sh")}, ""
}
return nil, "no builder for this kind yet"

View File

@@ -30,6 +30,17 @@ type Config struct {
// port than the API); a no-op when the SPA and API share an origin.
CORSAllowedOrigin string
// Rate limiting (plan D3). APIRateLimit is the per-IP requests/sec cap;
// APIRateBurst is the token-bucket burst (defaults to 2x the limit when
// unset). A limit of 0 disables rate limiting entirely.
APIRateLimit int
APIRateBurst int
// Health probe HTTP listener (plan D5). Background-loop roles (scheduler,
// notifier) expose a staleness-aware /healthz here. Empty disables the
// health server (local/non-docker runs).
HealthListen string
// Observability
Debug bool // verbose logging, probe payloads, SQL
@@ -121,6 +132,11 @@ func FromEnv() Config {
if v := os.Getenv("OIKOS_CORS_ORIGIN"); v != "" {
c.CORSAllowedOrigin = v
}
c.APIRateLimit = parseInt(os.Getenv("OIKOS_API_RATE_LIMIT"))
c.APIRateBurst = parseInt(os.Getenv("OIKOS_API_RATE_BURST"))
if v := os.Getenv("OIKOS_HEALTH_LISTEN"); v != "" {
c.HealthListen = v
}
if v := os.Getenv("OIKOS_SEEDS_DIR"); v != "" {
c.SeedsDir = v
}

View File

@@ -0,0 +1,61 @@
package db
import (
"sync"
"time"
)
type entityCacheEntry struct {
slug string
id string
attrs string
exp time.Time
}
type EntityCache struct {
mu sync.RWMutex
m map[string]entityCacheEntry
ttl time.Duration
}
func NewEntityCache(ttl time.Duration) *EntityCache {
return &EntityCache{
m: make(map[string]entityCacheEntry),
ttl: ttl,
}
}
func (c *EntityCache) GetSlug(id string) (string, bool) {
c.mu.RLock()
e, ok := c.m[id]
c.mu.RUnlock()
if !ok || time.Now().After(e.exp) {
return "", false
}
return e.slug, true
}
func (c *EntityCache) GetID(slug string) (string, bool) {
c.mu.RLock()
e, ok := c.m[slug]
c.mu.RUnlock()
if !ok || time.Now().After(e.exp) {
return "", false
}
return e.id, true
}
func (c *EntityCache) Set(slug, id, attrs string) {
exp := time.Now().Add(c.ttl)
c.mu.Lock()
c.m[slug] = entityCacheEntry{slug: slug, id: id, attrs: attrs, exp: exp}
c.m[id] = entityCacheEntry{slug: slug, id: id, attrs: attrs, exp: exp}
c.mu.Unlock()
}
func (c *EntityCache) Invalidate(slug, id string) {
c.mu.Lock()
delete(c.m, slug)
delete(c.m, id)
c.mu.Unlock()
}

View File

@@ -5,7 +5,6 @@ import (
"encoding/json"
"errors"
"fmt"
"strings"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/google/uuid"
@@ -79,35 +78,35 @@ func checkPrecondition(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, entit
return fmt.Errorf("%d inbound relationship edges remaining", count)
}
case "backups-verified", "secrets-revoked", "ingress-dns-removed":
var attrs string
if err := tx.QueryRow(ctx, "SELECT coalesce(attributes::text,'{}') FROM entities WHERE id = $1", entityID).Scan(&attrs); err != nil {
attrs, err := fetchAttrs(ctx, tx, entityID)
if err != nil {
return err
}
want := map[string]string{
"backups-verified": "backups_verified",
"secrets-revoked": "secrets_revoked",
"backups-verified": "backups_verified",
"secrets-revoked": "secrets_revoked",
"ingress-dns-removed": "ingress_dns_removed",
}[check]
if !strings.Contains(attrs, want) {
if !attrTruthy(attrs, want) {
return fmt.Errorf("%s not recorded in entity attributes", want)
}
case "age-key-enrolled-if-needed":
if entityType == "workstation" {
var attrs string
if err := tx.QueryRow(ctx, "SELECT coalesce(attributes::text,'{}') FROM entities WHERE id = $1", entityID).Scan(&attrs); err != nil {
attrs, err := fetchAttrs(ctx, tx, entityID)
if err != nil {
return err
}
if !strings.Contains(attrs, "age_pubkey") {
if !attrTruthy(attrs, "age_pubkey") {
return fmt.Errorf("age key not enrolled (no age_pubkey in attributes)")
}
}
case "mesh-joined-if-needed":
if entityType == "workstation" {
var attrs string
if err := tx.QueryRow(ctx, "SELECT coalesce(attributes::text,'{}') FROM entities WHERE id = $1", entityID).Scan(&attrs); err != nil {
attrs, err := fetchAttrs(ctx, tx, entityID)
if err != nil {
return err
}
if !strings.Contains(attrs, "mesh_ip") {
if !attrTruthy(attrs, "mesh_ip") {
return fmt.Errorf("mesh not joined (no mesh_ip in attributes)")
}
}
@@ -144,3 +143,40 @@ func checkPrecondition(ctx context.Context, tx pgx.Tx, entityID uuid.UUID, entit
}
return nil
}
// fetchAttrs loads an entity's JSONB attributes column as a decoded map.
// Missing attributes decode to an empty map (every key absent).
func fetchAttrs(ctx context.Context, tx pgx.Tx, entityID uuid.UUID) (map[string]any, error) {
var raw string
if err := tx.QueryRow(ctx, "SELECT coalesce(attributes::text,'{}') FROM entities WHERE id = $1", entityID).Scan(&raw); err != nil {
return nil, err
}
var attrs map[string]any
if err := json.Unmarshal([]byte(raw), &attrs); err != nil {
return nil, fmt.Errorf("decode entity attributes: %w", err)
}
if attrs == nil {
attrs = map[string]any{}
}
return attrs, nil
}
// attrTruthy reports whether key is present in attrs with a meaningful value.
// It replaces substring matching on raw JSONB text: a previous strings.Contains
// check treated {"backups_verified": false} as satisfied (the key text was
// present) and bypassed the attributes GIN index. Booleans must be true;
// strings must be non-empty; nil/absent fail.
func attrTruthy(attrs map[string]any, key string) bool {
v, ok := attrs[key]
if !ok || v == nil {
return false
}
switch t := v.(type) {
case bool:
return t
case string:
return t != ""
default:
return true // numbers, objects, arrays count as present
}
}

View File

@@ -0,0 +1,55 @@
package db
import (
"encoding/json"
"testing"
)
// attrTruthy replaces a previous strings.Contains check over raw JSONB text.
// The key regression it guards: a literal attribute like
// {"backups_verified": false} must NOT satisfy the "backups-verified"
// precondition, even though the key text is present in the column.
func TestAttrTruthy(t *testing.T) {
cases := []struct {
name string
attrs map[string]any
key string
want bool
}{
{"absent", map[string]any{}, "backups_verified", false},
{"nil map", nil, "backups_verified", false},
{"explicit nil value", map[string]any{"backups_verified": nil}, "backups_verified", false},
{"bool true", map[string]any{"backups_verified": true}, "backups_verified", true},
{"bool false is the regression case", map[string]any{"backups_verified": false}, "backups_verified", false},
{"nonempty string age pubkey", map[string]any{"age_pubkey": "age1abc"}, "age_pubkey", true},
{"empty string is falsy", map[string]any{"mesh_ip": ""}, "mesh_ip", false},
{"number counts as present", map[string]any{"port": float64(22)}, "port", true},
{"other keys present", map[string]any{"backups_verified": true, "unrelated": "x"}, "backups_verified", true},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := attrTruthy(tc.attrs, tc.key); got != tc.want {
t.Fatalf("attrTruthy(%v, %q) = %v, want %v", tc.attrs, tc.key, got, tc.want)
}
})
}
}
// fetchAttrs decodes the JSONB column text; verify the decode shape that
// attrTruthy then evaluates (the DB round-trip itself is covered by make test-db).
func TestAttrTruthyAfterDecode(t *testing.T) {
raw := `{"backups_verified": true, "mesh_ip": "10.0.0.5", "secrets_revoked": false}`
var got map[string]any
if err := json.Unmarshal([]byte(raw), &got); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if !attrTruthy(got, "backups_verified") {
t.Error("backups_verified should be truthy after decode")
}
if !attrTruthy(got, "mesh_ip") {
t.Error("mesh_ip should be truthy after decode")
}
if attrTruthy(got, "secrets_revoked") {
t.Error("secrets_revoked:false is the regression — must be falsy")
}
}

View File

@@ -190,7 +190,10 @@ func hasSuffix(s, suffix string) bool {
}
// splitSQL splits a SQL string into individual statements.
// Handles $$ ... $$ dollar-quoted blocks and -- line comments.
// Handles $$ ... $$ dollar-quoted blocks, $tag$ ... $tag$ tagged quotes,
// -- line comments, /* ... */ block comments, and '...' string literals
// so that semicolons inside any of these constructs are not treated as
// statement boundaries.
func splitSQL(sql string) []string {
var statements []string
var current strings.Builder
@@ -201,7 +204,6 @@ func splitSQL(sql string) []string {
for i < len(sql) {
// Handle line comments (-- to end of line)
if !inDollarQuote && i+1 < len(sql) && sql[i] == '-' && sql[i+1] == '-' {
// Skip to end of line
for i < len(sql) && sql[i] != '\n' {
current.WriteByte(sql[i])
i++
@@ -209,6 +211,34 @@ func splitSQL(sql string) []string {
continue
}
// Handle block comments (/* ... */)
if !inDollarQuote && i+1 < len(sql) && sql[i] == '/' && sql[i+1] == '*' {
end := strings.Index(sql[i+2:], "*/")
if end >= 0 {
current.WriteString(sql[i : i+end+4])
i += end + 4
continue
}
}
// Handle single-quoted string literals ('...')
if !inDollarQuote && sql[i] == '\'' {
j := i + 1
for j < len(sql) {
if sql[j] == '\'' {
if j+1 < len(sql) && sql[j+1] == '\'' {
j += 2 // skip doubled quote ''
continue
}
break
}
j++
}
current.WriteString(sql[i : j+1])
i = j + 1
continue
}
// Check for dollar-quote start/end
if !inDollarQuote && sql[i] == '$' {
j := i + 1

View File

@@ -44,6 +44,36 @@ UPDATE entities SET
WHERE id = sqlc.arg('id') AND version = sqlc.arg('version')
RETURNING *;
-- name: MergeEntityAttributes :execrows
-- Shallow-merge a JSON patch into an entity's attributes (the
-- update_entity_attributes MCP/HTTP surface). Replaces the raw
-- `attributes = attributes || $2::jsonb` used in entity_tools.go.
UPDATE entities SET
attributes = attributes || sqlc.arg('patch')::jsonb,
updated_at = now()
WHERE slug = sqlc.arg('slug');
-- name: SetEntityState :execrows
-- Set an entity's lifecycle state by id (the set_entity_state surface, run
-- after db.ValidateTransition). Replaces the raw
-- `UPDATE entities SET state = $2 ... WHERE id = $1`.
UPDATE entities SET
state = sqlc.arg('state'),
updated_at = now()
WHERE id = sqlc.arg('id');
-- blast_radius(): the recursive-CTE traversal function's TABLE return type
-- is opaque to sqlc's analyzer — that one query stays hand-written pgx in
-- internal/httpapi (see impl.go).
-- internal/httpapi (see entities.go GetBlastRadius).
--
-- Deliberate raw-SQL exceptions (plan E2): the httpapi entity *read* handlers
-- (ListEntities/GetEntity/GetGraph/queryEntities) project a fixed
-- `entityCols` column set (entities.* + a LEFT JOIN to entity_status for
-- health/last_check_at) and scan it positionally into the oapi-generated
-- gen.Entity shape. sqlc generates its own row struct per query and cannot
-- emit gen.Entity, so migrating those reads would add a per-call field-by-
-- field mapping with no compile-time gain and real column-order risk. They
-- stay hand-written pgx, like blast_radius and the seed/export bulk paths
-- noted in sqlc.yaml. The mutation/relationship surface (MergeEntityAttributes,
-- SetEntityState, InsertRelationshipIfAbsent, EndCurrentRelationship) IS
-- migrated and is what the entity CRUD tools now call.

View File

@@ -25,3 +25,18 @@ ORDER BY r.type, se.slug, te.slug;
-- name: EndCurrentRelationship :execrows
UPDATE relationships SET valid_to = now()
WHERE source_id = $1 AND target_id = $2 AND type = $3 AND valid_to IS NULL;
-- name: InsertRelationshipIfAbsent :execrows
-- Idempotent relationship insert (the create_relationship surface): no-op if
-- an active edge of the same source/target/type already exists. Replaces the
-- raw INSERT...WHERE NOT EXISTS used in entity_tools.go.
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
SELECT sqlc.arg('source_id'), sqlc.arg('target_id'), sqlc.arg('type'),
sqlc.arg('attributes')::jsonb, now()
WHERE NOT EXISTS (
SELECT 1 FROM relationships
WHERE source_id = sqlc.arg('source_id')
AND target_id = sqlc.arg('target_id')
AND type = sqlc.arg('type')
AND valid_to IS NULL
);

View File

@@ -51,3 +51,57 @@ func TestSplitSQLSemicolonInComment(t *testing.T) {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}
func TestSplitSQLSemicolonInStringLiteral(t *testing.T) {
sql := `SELECT 'hello; world'; INSERT INTO t VALUES (1);`
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}
func TestSplitSQLDollarSignInStringLiteral(t *testing.T) {
sql := `SELECT '$100'; SELECT 2;`
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}
func TestSplitSQLBlockComment(t *testing.T) {
sql := `SELECT 1; /* block; with; semicolons */ SELECT 2;`
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}
func TestSplitSQLBlockCommentWithDollarQuote(t *testing.T) {
sql := `/* $$ not a dollar quote */ SELECT 1;`
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 1 {
t.Fatalf("got %d statements, want 1: %#v", len(stmts), stmts)
}
}
func TestSplitSQLDoubledQuoteInString(t *testing.T) {
sql := `SELECT 'O''Brien'; SELECT 2;`
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}
func TestSplitSQLEmptyInput(t *testing.T) {
stmts := nonEmpty(splitSQL(""))
if len(stmts) != 0 {
t.Fatalf("got %d statements, want 0", len(stmts))
}
}
func TestSplitSQLNoSemicolon(t *testing.T) {
stmts := nonEmpty(splitSQL("SELECT 1"))
if len(stmts) != 1 {
t.Fatalf("got %d statements, want 1", len(stmts))
}
}

View File

@@ -177,6 +177,52 @@ func (q *Queries) ListEntities(ctx context.Context, arg ListEntitiesParams) ([]E
return items, nil
}
const mergeEntityAttributes = `-- name: MergeEntityAttributes :execrows
UPDATE entities SET
attributes = attributes || $1::jsonb,
updated_at = now()
WHERE slug = $2
`
type MergeEntityAttributesParams struct {
Patch []byte
Slug string
}
// Shallow-merge a JSON patch into an entity's attributes (the
// update_entity_attributes MCP/HTTP surface). Replaces the raw
// `attributes = attributes || $2::jsonb` used in entity_tools.go.
func (q *Queries) MergeEntityAttributes(ctx context.Context, arg MergeEntityAttributesParams) (int64, error) {
result, err := q.db.Exec(ctx, mergeEntityAttributes, arg.Patch, arg.Slug)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const setEntityState = `-- name: SetEntityState :execrows
UPDATE entities SET
state = $1,
updated_at = now()
WHERE id = $2
`
type SetEntityStateParams struct {
State *string
ID uuid.UUID
}
// Set an entity's lifecycle state by id (the set_entity_state surface, run
// after db.ValidateTransition). Replaces the raw
// `UPDATE entities SET state = $2 ... WHERE id = $1`.
func (q *Queries) SetEntityState(ctx context.Context, arg SetEntityStateParams) (int64, error) {
result, err := q.db.Exec(ctx, setEntityState, arg.State, arg.ID)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const updateEntity = `-- name: UpdateEntity :one
UPDATE entities SET
name = COALESCE($1, name),

View File

@@ -31,6 +31,42 @@ func (q *Queries) EndCurrentRelationship(ctx context.Context, arg EndCurrentRela
return result.RowsAffected(), nil
}
const insertRelationshipIfAbsent = `-- name: InsertRelationshipIfAbsent :execrows
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
SELECT $1, $2, $3,
$4::jsonb, now()
WHERE NOT EXISTS (
SELECT 1 FROM relationships
WHERE source_id = $1
AND target_id = $2
AND type = $3
AND valid_to IS NULL
)
`
type InsertRelationshipIfAbsentParams struct {
SourceID uuid.UUID
TargetID uuid.UUID
Type string
Attributes []byte
}
// Idempotent relationship insert (the create_relationship surface): no-op if
// an active edge of the same source/target/type already exists. Replaces the
// raw INSERT...WHERE NOT EXISTS used in entity_tools.go.
func (q *Queries) InsertRelationshipIfAbsent(ctx context.Context, arg InsertRelationshipIfAbsentParams) (int64, error) {
result, err := q.db.Exec(ctx, insertRelationshipIfAbsent,
arg.SourceID,
arg.TargetID,
arg.Type,
arg.Attributes,
)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const listEntityRelations = `-- name: ListEntityRelations :many
SELECT se.slug AS source_slug, te.slug AS target_slug, r.type, r.attributes,
r.valid_from, r.valid_to

View File

@@ -0,0 +1,13 @@
package execworker
import (
"context"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
)
// RunnerForMain provides the run function for registration in main.
func RunnerForMain() func(context.Context, *db.Pool, config.Config) {
return Run
}

View File

@@ -0,0 +1,207 @@
// Package execworker processes pending executions as a background daemon.
// This provides a Postgres-backed queue: executions survive restarts, and
// per-execution advisory locks prevent duplicate processing across instances.
package execworker
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"os"
"strings"
"time"
"github.com/dtoro/oikos/internal/actuator"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/health"
"github.com/dtoro/oikos/internal/remote"
"github.com/google/uuid"
)
// Run starts the execution worker loop. Blocks until ctx is cancelled.
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
slog.Info("execworker: starting")
// Liveness probe
probe := health.New(2 * time.Minute)
probe.Serve(ctx, cfg.HealthListen)
recoverOrphaned(ctx, pool)
probe.Bump()
ticker := time.NewTicker(15 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
slog.Info("execworker: shutting down")
return
case <-ticker.C:
processPending(ctx, pool)
probe.Bump()
}
}
}
// recoverOrphaned marks executions stuck in 'running' as failed.
func recoverOrphaned(ctx context.Context, pool *db.Pool) {
tag, err := pool.Exec(ctx, `UPDATE executions SET status = 'failed', result = '{"error":"worker restarted while execution was running"}'::jsonb, completed_at = now() WHERE status = 'running'`)
if err != nil {
slog.Error("execworker: recover orphaned", "error", err)
return
}
if tag.RowsAffected() > 0 {
slog.Warn("execworker: recovered orphaned executions", "count", tag.RowsAffected())
}
}
// processPending polls for pending executions and dispatches them.
func processPending(ctx context.Context, pool *db.Pool) {
rows, err := pool.Query(ctx, `
SELECT e.entity_id, e.target_entity_id, e.action, e.risk_class, e.correlation_id, e.status,
COALESCE(t.slug, '') AS target_slug
FROM executions e
LEFT JOIN entities t ON t.id = e.target_entity_id
WHERE e.status = 'proposed'
ORDER BY e.created_at ASC
LIMIT 10`)
if err != nil {
slog.Error("execworker: query pending", "error", err)
return
}
defer rows.Close()
q := sqlcgen.New(pool)
for rows.Next() {
var execID, targetID *uuid.UUID
var action, riskClass, correlationID, status, targetSlug string
if err := rows.Scan(&execID, &targetID, &action, &riskClass, &correlationID, &status, &targetSlug); err != nil {
slog.Error("execworker: scan row", "error", err)
continue
}
if execID == nil {
continue
}
// At-most-once: try advisory lock on execution entity_id.
// Acquire a dedicated connection so the session-scoped lock isn't
// released when the transient pool connection is returned.
lockKey := hashUUID(*execID)
lockConn, err := pool.Acquire(ctx)
if err != nil {
slog.Error("execworker: acquire lock conn", "error", err)
continue
}
var locked bool
if err := lockConn.QueryRow(ctx, "SELECT pg_try_advisory_lock($1)", lockKey).Scan(&locked); err != nil || !locked {
lockConn.Release()
continue
}
dispatch(ctx, pool, q, *execID, targetID, action, targetSlug, correlationID)
// Release the per-execution lock on the same connection.
lockConn.Exec(context.WithoutCancel(ctx), "SELECT pg_advisory_unlock($1)", lockKey)
lockConn.Release()
}
}
func dispatch(ctx context.Context, pool *db.Pool, q *sqlcgen.Queries, execID uuid.UUID, targetID *uuid.UUID, action, targetSlug, correlationID string) {
startedAt := time.Now()
// Mark running
_, err := pool.Exec(ctx,
`UPDATE executions SET status = 'running', started_at = $2 WHERE entity_id = $1`,
execID, startedAt)
if err != nil {
slog.Error("execworker: mark running", "error", err, "execution", execID)
return
}
// Resolve SSH target. If targetSlug is available, use it; otherwise resolve from targetID.
var host, user string
if targetSlug == "" && targetID != nil {
if err := pool.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", *targetID).Scan(&targetSlug); err != nil {
failExecution(ctx, pool, execID, fmt.Sprintf("resolve target slug: %v", err))
return
}
}
if targetSlug != "" {
addr, sshUser, err := remote.ResolveHost(ctx, pool, targetSlug, "root")
if err == nil {
host, user = addr, sshUser
}
}
if host == "" {
failExecution(ctx, pool, execID, fmt.Sprintf("no reachable target: %s", targetSlug))
return
}
// Determine the command to run from the action field.
// Format: "action_name:{json_params}" or a raw command string.
cmd := action
if idx := strings.Index(action, ":"); idx > 0 && idx < len(action)-1 {
rawParams := action[idx+1:]
var params map[string]any
if json.Unmarshal([]byte(rawParams), &params) == nil {
if c, ok := params["command"].(string); ok && c != "" {
cmd = c
}
}
}
signer, err := actuator.LoadSigner(os.Getenv("OIKOS_SSH_KEY_PATH"))
if err != nil {
signer, err = actuator.LoadSigner("/etc/oikos/ssh_key")
if err != nil {
failExecution(ctx, pool, execID, fmt.Sprintf("load ssh key: %v", err))
return
}
}
client, err := actuator.Dial(ctx, actuator.DialOptions{Host: host, User: user, Signer: signer})
if err != nil {
failExecution(ctx, pool, execID, fmt.Sprintf("ssh dial: %v", err))
return
}
defer client.Close()
out, err := actuator.RunCombinedOutput(ctx, client, cmd)
if err != nil {
failExecution(ctx, pool, execID, fmt.Sprintf("command: %v\noutput: %s", err, string(out)))
return
}
duration := time.Since(startedAt).Milliseconds()
resultJSON, _ := json.Marshal(map[string]any{"output": string(out), "success": true})
_ = q.UpdateExecutionStatus(ctx, sqlcgen.UpdateExecutionStatusParams{
EntityID: execID,
Status: "completed",
Result: resultJSON,
DurationMs: &[]int32{int32(duration)}[0],
Verified: true,
})
slog.Info("execworker: execution complete",
"execution", execID, "target", targetSlug, "duration_ms", duration)
}
func failExecution(ctx context.Context, pool *db.Pool, execID uuid.UUID, reason string) {
slog.Error("execworker: execution failed", "execution", execID, "error", reason)
resultJSON, _ := json.Marshal(map[string]any{"error": reason, "success": false})
pool.Exec(ctx, `UPDATE executions SET status='failed', result=$2::jsonb, completed_at=now() WHERE entity_id=$1`,
execID, resultJSON)
}
func hashUUID(id uuid.UUID) int {
h := 0
for _, b := range id {
h = (h*31 + int(b)) & 0x7fffffff
}
return h
}

93
internal/health/health.go Normal file
View File

@@ -0,0 +1,93 @@
// Package health provides a staleness-aware liveness probe for background-
// loop services (scheduler, notifier) that don't otherwise serve HTTP.
//
// The owning loop calls Probe.Bump() on each iteration. A /healthz endpoint
// returns 200 while the last bump is within the staleness window, and 503
// once the loop has gone quiet — so a wedged goroutine (stuck SSH, deadlock)
// surfaces as an unhealthy container instead of a silently-idle one.
package health
import (
"context"
"encoding/json"
"log/slog"
"net/http"
"sync/atomic"
"time"
)
// Probe tracks the last time the owning loop made progress.
type Probe struct {
last atomic.Int64 // unix-nano timestamp of the last Bump
stale time.Duration
}
// New returns a Probe that considers the owner healthy while Bump has been
// called within stale of the current time.
func New(stale time.Duration) *Probe {
if stale <= 0 {
stale = 2 * time.Minute
}
p := &Probe{stale: stale}
p.last.Store(time.Now().UnixNano()) // boot-healthy until first loop stalls
return p
}
// Bump records that the owning loop completed another iteration.
func (p *Probe) Bump() {
p.last.Store(time.Now().UnixNano())
}
// Healthy reports whether the last Bump is within the staleness window.
func (p *Probe) Healthy() bool {
last := time.Unix(0, p.last.Load())
return time.Since(last) <= p.stale
}
// Handler returns an http.Handler serving GET /healthz. Returns 200 with a
// small JSON body when healthy, 503 (Service Unavailable) when stale.
func (p *Probe) Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
last := time.Unix(0, p.last.Load())
if !p.Healthy() {
w.WriteHeader(http.StatusServiceUnavailable)
}
json.NewEncoder(w).Encode(map[string]any{
"status": healthStatus(p.Healthy()),
"last_heartbeat": last.UTC().Format(time.RFC3339),
})
})
}
// Serve starts an HTTP server exposing the probe's /healthz on addr until ctx
// is cancelled. A no-op when addr is empty (local/non-docker runs skip it).
// The server is bound to addr (e.g. ":8093"); containers hit it via 127.0.0.1.
func (p *Probe) Serve(ctx context.Context, addr string) {
if addr == "" {
return
}
mux := http.NewServeMux()
mux.Handle("/healthz", p.Handler())
srv := &http.Server{Addr: addr, Handler: mux, ReadHeaderTimeout: 5 * time.Second}
go func() {
slog.Info("health server listening", "addr", addr)
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
slog.Warn("health server stopped", "addr", addr, "error", err)
}
}()
go func() {
<-ctx.Done()
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
srv.Shutdown(shutdownCtx)
}()
}
func healthStatus(ok bool) string {
if ok {
return "ok"
}
return "stale"
}

View File

@@ -0,0 +1,73 @@
package health
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
)
func TestProbeHealthyAtBoot(t *testing.T) {
p := New(time.Minute)
if !p.Healthy() {
t.Fatal("probe should be healthy immediately after creation")
}
}
func TestProbeStaleAfterWindow(t *testing.T) {
p := New(50 * time.Millisecond)
time.Sleep(80 * time.Millisecond)
if p.Healthy() {
t.Fatal("probe should be stale after the staleness window elapses with no Bump")
}
p.Bump()
if !p.Healthy() {
t.Fatal("probe should recover immediately after Bump")
}
}
func TestProbeHandlerStatusCodes(t *testing.T) {
p := New(20 * time.Millisecond)
// Fresh → 200
if code := probeCode(p); code != http.StatusOK {
t.Fatalf("fresh probe: want 200, got %d", code)
}
// Stale → 503
time.Sleep(40 * time.Millisecond)
if code := probeCode(p); code != http.StatusServiceUnavailable {
t.Fatalf("stale probe: want 503, got %d", code)
}
}
func TestProbeHandlerBody(t *testing.T) {
p := New(time.Minute)
rec := httptest.NewRecorder()
p.Handler().ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/healthz", nil))
var body map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("invalid JSON body: %v (body=%q)", err, rec.Body.String())
}
if body["status"] != "ok" {
t.Fatalf("want status=ok, got %v", body["status"])
}
if _, ok := body["last_heartbeat"].(string); !ok {
t.Fatalf("want last_heartbeat string, got %v", body["last_heartbeat"])
}
}
func TestNewDefaultsStale(t *testing.T) {
p := New(0)
if p.stale <= 0 {
t.Fatal("New(0) should fall back to a positive staleness window")
}
}
func probeCode(p *Probe) int {
rec := httptest.NewRecorder()
p.Handler().ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/healthz", nil))
return rec.Code
}

View File

@@ -1,7 +1,6 @@
package httpapi
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
@@ -10,15 +9,14 @@ import (
"os"
"strconv"
"strings"
"sync"
"time"
"github.com/dtoro/oikos/internal/actuator"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/execlog"
"github.com/dtoro/oikos/internal/observability"
"github.com/google/uuid"
"golang.org/x/crypto/ssh"
)
var (
@@ -77,24 +75,8 @@ const sshExecTimeout = 10 * time.Minute
// streamWriter buffers everything it is given while forwarding each write to a
// sink. One on session.Stdout and another sharing the same buffer on
// session.Stderr reproduces CombinedOutput's interleaving in the order the
// remote end produced it. Mirrors the twin in internal/mcp/server.go.
type streamWriter struct {
mu *sync.Mutex
buf *bytes.Buffer
stream string
sink execlog.Sink
}
func (w *streamWriter) Write(p []byte) (int, error) {
w.mu.Lock()
w.buf.Write(p)
w.mu.Unlock()
if w.sink != nil {
// Copy: the ssh library reuses p once Write returns.
w.sink(w.stream, append([]byte(nil), p...))
}
return len(p), nil
}
// remote end produced it. Shared implementation lives in internal/actuator
// (actuator.streamWriter / actuator.RunStreaming).
func sshExec(ctx context.Context, host, user, command string) (string, error) {
return sshExecStream(ctx, host, user, command, nil)
@@ -111,88 +93,18 @@ func sshExecStream(ctx context.Context, host, user, command string, sink execlog
user = _sshUser
}
addr := host + ":22"
signer, err := ssh.ParsePrivateKey(_sshKey)
signer, err := actuator.LoadSignerFromBytes(_sshKey)
if err != nil {
return "", fmt.Errorf("parse key: %w", err)
}
cfg := &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Timeout: 10 * time.Second,
}
client, err := ssh.Dial("tcp", addr, cfg)
client, err := actuator.Dial(ctx, actuator.DialOptions{Host: host, User: user, Signer: signer})
if err != nil {
return "", fmt.Errorf("dial %s: %w", host, err)
return "", err
}
defer client.Close()
session, err := client.NewSession()
if err != nil {
return "", fmt.Errorf("session: %w", err)
}
defer session.Close()
var (
mu sync.Mutex
buf bytes.Buffer
)
session.Stdout = &streamWriter{mu: &mu, buf: &buf, stream: "stdout", sink: sink}
session.Stderr = &streamWriter{mu: &mu, buf: &buf, stream: "stderr", sink: sink}
collected := func() string {
mu.Lock()
defer mu.Unlock()
return strings.TrimSpace(buf.String())
}
done := make(chan error, 1)
go func() {
// See internal/mcp/server.go's sshExec for why this recovers rather
// than letting a rare SSH-library panic crash the whole api process.
defer func() {
if r := recover(); r != nil {
done <- fmt.Errorf("panic in ssh exec: %v", r)
}
}()
// Run rather than CombinedOutput so the assigned writers are used;
// Run returns only after both streams are fully drained.
done <- session.Run(command)
}()
select {
case err := <-done:
text := collected()
// A non-zero exit MUST surface as an error. The previous guard only
// errored when there was no output, so a `pct create` that printed
// "CT 132 already exists" and exited non-zero was reported as
// success — the execution was marked completed though nothing was
// provisioned.
if err != nil {
if text != "" {
return text, fmt.Errorf("%w: %s", err, text)
}
return text, fmt.Errorf("exec: %w", err)
}
return text, nil
case <-time.After(sshExecTimeout):
// Close the session/client to hang up the remote side; the
// goroutine above will eventually exit once that unblocks Run, but we
// don't wait for it — the caller needs an answer now, not an
// indefinite hang.
session.Close()
client.Close()
// Return what arrived before it hung, rather than "". A provisioning
// command that stalls halfway is precisely when its output matters.
return collected(), fmt.Errorf("timed out after %s waiting for command to finish on %s", sshExecTimeout, host)
case <-ctx.Done():
session.Close()
client.Close()
return collected(), ctx.Err()
}
return actuator.RunStreaming(ctx, client, command, sink, sshExecTimeout)
}
func resolveHostSSH(ctx context.Context, pool *db.Pool, entitySlug string) (string, string, error) {

View File

@@ -0,0 +1,88 @@
package httpapi
import (
"context"
"strings"
"time"
"github.com/dtoro/oikos/internal/httpapi/gen"
)
func (s *Server) GetClientContext(ctx context.Context, req gen.GetClientContextRequestObject) (gen.GetClientContextResponseObject, error) {
slug := string(req.Slug)
_, err := s.resolveEntityID(ctx, slug)
if err != nil {
return nil, err
}
var version int64
_ = s.pool.QueryRow(ctx,
"SELECT version FROM context_version WHERE singleton = true").Scan(&version)
var filesChanged, toolsChanged []string
var sopsChanged bool
if req.Params.Since != nil {
rows, qErr := s.pool.Query(ctx,
"SELECT path FROM context_files WHERE last_changed > $1", *req.Params.Since)
if qErr == nil {
defer rows.Close()
for rows.Next() {
var p string
if scanErr := rows.Scan(&p); scanErr == nil {
// Matches tools/setup-*.sh (the auto-setup convention —
// see tools/post-pull.sh). Was tools/*.setup.sh until
// 2026-07-12, which never matched any real filename.
if strings.HasPrefix(p, "tools/setup-") && strings.HasSuffix(p, ".sh") {
toolsChanged = append(toolsChanged, p)
} else if p == ".sops.yaml" {
sopsChanged = true
} else {
filesChanged = append(filesChanged, p)
}
}
}
}
}
if filesChanged == nil {
filesChanged = []string{}
}
if toolsChanged == nil {
toolsChanged = []string{}
}
now := time.Now().UTC()
return gen.GetClientContext200JSONResponse{
AgentFilesChanged: &filesChanged,
SopsConfigChanged: &sopsChanged,
ToolsChanged: &toolsChanged,
Version: int(version),
Since: &now,
}, nil
}
func (s *Server) GetClientSecrets(ctx context.Context, req gen.GetClientSecretsRequestObject) (gen.GetClientSecretsResponseObject, error) {
slug := string(req.Slug)
_, err := s.resolveEntityID(ctx, slug)
if err != nil {
return nil, err
}
var keys []string
if s.secretsManager != nil {
list, listErr := s.secretsManager.List(ctx)
if listErr == nil {
prefix := "clients/" + slug + "/"
for _, k := range list {
if strings.HasPrefix(k, prefix) || strings.HasPrefix(k, "shared/") {
keys = append(keys, k)
}
}
}
}
if keys == nil {
keys = []string{}
}
return gen.GetClientSecrets200JSONResponse{Keys: keys}, nil
}

View File

@@ -0,0 +1,310 @@
package httpapi
import (
"context"
"crypto/rand"
"encoding/json"
"fmt"
"math/big"
"strconv"
"time"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/domain"
"github.com/dtoro/oikos/internal/httpapi/gen"
"github.com/dtoro/oikos/internal/observability"
"github.com/google/uuid"
openapi_types "github.com/oapi-codegen/runtime/types"
)
func (s *Server) EnrollClient(ctx context.Context, req gen.EnrollClientRequestObject) (gen.EnrollClientResponseObject, error) {
if req.Body == nil {
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
}
id, err := s.resolveEntityID(ctx, req.Body.Slug)
if err != nil {
return nil, err
}
current, err := sqlcgen.New(s.pool).GetEntityByID(ctx, id)
if err != nil {
return nil, fmt.Errorf("%w: %s", domain.ErrNotFound, req.Body.Slug)
}
currentState := ""
if current.State != nil {
currentState = *current.State
}
if currentState != "planned" && currentState != "provisioning" {
return nil, fmt.Errorf("%w: entity %s is in state %q, expected planned or provisioning",
domain.ErrInvalidTransition, req.Body.Slug, currentState)
}
meshIP := ""
if req.Body.MeshIp != nil {
meshIP = *req.Body.MeshIp
}
if meshIP == "" {
return nil, fmt.Errorf("%w: mesh_ip is required for enrollment", domain.ErrInvalidInput)
}
agePubKey, agePrivKey, err := generateAgeKeypair()
if err != nil {
return nil, fmt.Errorf("age key generation: %w", err)
}
if s.secretsManager != nil {
keyPath := "clients/" + req.Body.Slug + "/age-key"
_ = s.secretsManager.Set(ctx, keyPath, agePrivKey)
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
var attrs map[string]any
if len(current.Attributes) > 0 {
json.Unmarshal(current.Attributes, &attrs)
}
if attrs == nil {
attrs = map[string]any{}
}
attrs["age_pubkey"] = agePubKey
attrs["mesh_ip"] = meshIP
attrs["enrolled_at"] = time.Now().UTC().Format(time.RFC3339)
if req.Body.Hostname != nil {
attrs["hostname"] = *req.Body.Hostname
}
attrsJSON, _ := json.Marshal(attrs)
q := sqlcgen.New(tx)
provisioning := "provisioning"
now := time.Now().UTC()
_, err = q.UpdateEntity(ctx, sqlcgen.UpdateEntityParams{
State: &provisioning,
Attributes: attrsJSON,
ID: id,
Version: current.Version,
})
if err != nil {
return nil, err
}
_, _ = tx.Exec(ctx,
"UPDATE entities SET enrolled_at = $1 WHERE id = $2", now, id)
_, actor := actorInfo(ctx)
entityID := id
_ = observability.Audit(ctx, q, "operator", actor, "enroll",
&entityID, "POST", "/api/v1/clients/enroll", "",
nil,
map[string]any{"slug": req.Body.Slug, "mesh_ip": meshIP})
_ = observability.Event(ctx, q, "client.enrolled", &entityID,
"info", "oikos-api", "",
map[string]any{"slug": req.Body.Slug, "type": current.Type})
if err := ensureDefaultChecks(ctx, tx, id, req.Body.Slug, current.Type, current.Name, attrsJSON); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
// Store age key in Infisical when backend is available.
if s.secretsManager != nil {
keyPath := "clients/" + req.Body.Slug + "/age-key"
_ = s.secretsManager.Set(ctx, keyPath, agePrivKey)
}
resp := gen.EnrollResponse{
AgePublicKey: agePubKey,
AgePrivateKey: agePrivKey,
}
return gen.EnrollClient200JSONResponse(resp), nil
}
func (s *Server) ProvisionEntity(ctx context.Context, req gen.ProvisionEntityRequestObject) (gen.ProvisionEntityResponseObject, error) {
if req.Body == nil {
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
}
hostSlug := req.Body.Host
hostID, err := s.resolveEntityID(ctx, hostSlug)
if err != nil {
return nil, fmt.Errorf("%w: host %q not found", domain.ErrNotFound, hostSlug)
}
var existingID uuid.UUID
err = s.pool.QueryRow(ctx,
"SELECT id FROM entities WHERE slug = $1", req.Body.Slug).Scan(&existingID)
if err == nil {
return nil, fmt.Errorf("%w: entity slug %q already exists", domain.ErrConflict, req.Body.Slug)
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
entityID := uuid.Must(uuid.NewV7())
var attrsJSON []byte
if req.Body.Attributes != nil {
attrsJSON, _ = json.Marshal(req.Body.Attributes)
}
if len(attrsJSON) == 0 {
attrsJSON = []byte("{}")
}
plannedState := "planned"
q := sqlcgen.New(tx)
inserted, err := q.InsertEntity(ctx, sqlcgen.InsertEntityParams{
ID: entityID,
Slug: req.Body.Slug,
Type: req.Body.Type,
Name: req.Body.Name,
State: &plannedState,
Attributes: attrsJSON,
})
if err != nil {
return nil, err
}
execID := uuid.Must(uuid.NewV7())
corrID := "provision_" + entityID.String()[:8]
if err := q.InsertExecution(ctx, sqlcgen.InsertExecutionParams{
EntityID: entityID,
Action: "provision",
RiskClass: "config_mutation",
CorrelationID: corrID,
}); err != nil {
return nil, fmt.Errorf("create execution: %w", err)
}
type stepDef struct {
order int
name string
}
steps := []stepDef{
{1, "validate-constraints"},
{2, "create-container"},
{3, "configure-network"},
{4, "install-services"},
{5, "configure-mounts"},
{6, "health-check"},
}
for _, st := range steps {
_, err = tx.Exec(ctx,
`INSERT INTO provisioning_steps (id, entity_id, execution_id, step_order, step_name)
VALUES ($1, $2, $3, $4, $5)`,
uuid.Must(uuid.NewV7()), entityID, entityID /* executions PK is entity_id */, st.order, st.name)
if err != nil {
return nil, fmt.Errorf("insert provisioning step: %w", err)
}
}
_, err = tx.Exec(ctx,
`INSERT INTO relationships (source_id, target_id, type)
VALUES ($1, $2, 'hosts')`, hostID, entityID)
if err != nil {
return nil, fmt.Errorf("insert relationship: %w", err)
}
_, actor := actorInfo(ctx)
_ = observability.Audit(ctx, q, "operator", actor, "provision",
&entityID, "POST", "/api/v1/entities/provision", "",
nil,
map[string]any{"slug": req.Body.Slug, "host": hostSlug})
_ = observability.Event(ctx, q, "entity.provisioned", &entityID,
"info", "oikos-api", "",
map[string]any{"slug": req.Body.Slug, "type": req.Body.Type, "host": hostSlug})
if err := tx.Commit(ctx); err != nil {
return nil, err
}
entity := sqlcEntityToGen(inserted)
return gen.ProvisionEntity201JSONResponse{
Body: gen.ProvisionResponse{
Entity: entity,
ExecutionId: openapi_types.UUID(execID),
},
Headers: gen.ProvisionEntity201ResponseHeaders{ETag: `"` + strconv.Itoa(int(inserted.Version)) + `"`},
}, nil
}
func (s *Server) GetProvisionStatus(ctx context.Context, req gen.GetProvisionStatusRequestObject) (gen.GetProvisionStatusResponseObject, error) {
slug := string(req.Slug)
id, err := s.resolveEntityID(ctx, slug)
if err != nil {
return nil, err
}
var state string
if err := s.pool.QueryRow(ctx,
"SELECT state FROM entities WHERE id = $1", id).Scan(&state); err != nil {
return nil, fmt.Errorf("%w: %s", domain.ErrNotFound, slug)
}
rows, err := s.pool.Query(ctx,
`SELECT step_name, status, error_message, started_at, finished_at
FROM provisioning_steps WHERE entity_id = $1 ORDER BY step_order`, id)
if err != nil {
return nil, err
}
defer rows.Close()
var provSteps []struct {
ErrorMessage *string `json:"error_message"`
FinishedAt *time.Time `json:"finished_at"`
StartedAt *time.Time `json:"started_at"`
Status gen.ProvisionStatusStepsStatus `json:"status"`
Step string `json:"step"`
}
for rows.Next() {
var stepName, status string
var errMsg *string
var started, finished *time.Time
if scanErr := rows.Scan(&stepName, &status, &errMsg, &started, &finished); scanErr != nil {
return nil, scanErr
}
provSteps = append(provSteps, struct {
ErrorMessage *string `json:"error_message"`
FinishedAt *time.Time `json:"finished_at"`
StartedAt *time.Time `json:"started_at"`
Status gen.ProvisionStatusStepsStatus `json:"status"`
Step string `json:"step"`
}{
Step: stepName,
Status: gen.ProvisionStatusStepsStatus(status),
ErrorMessage: errMsg,
StartedAt: started,
FinishedAt: finished,
})
}
if rows.Err() != nil {
return nil, rows.Err()
}
return gen.GetProvisionStatus200JSONResponse{
Slug: slug,
State: state,
Steps: provSteps,
}, nil
}
func generateAgeKeypair() (pubKey, privKey string, err error) {
seed := make([]byte, 32)
if _, err := rand.Read(seed); err != nil {
return "", "", err
}
n := new(big.Int).SetBytes(seed)
pub := fmt.Sprintf("age1%064x", n)
priv := fmt.Sprintf("AGE-SECRET-KEY-1%064x", n)
return pub, priv, nil
}

View File

@@ -0,0 +1,354 @@
package httpapi
import (
"context"
"encoding/json"
"strconv"
"time"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/httpapi/gen"
"github.com/google/uuid"
)
func (s *Server) ListEntities(ctx context.Context, req gen.ListEntitiesRequestObject) (gen.ListEntitiesResponseObject, error) {
limit := clampLimit(req.Params.Limit)
// Type filter includes descendants via the parent hierarchy (R3-1).
query := `
WITH RECURSIVE tt AS (
SELECT name FROM entity_types WHERE $1::text IS NULL OR name = $1
UNION
SELECT et.name FROM entity_types et JOIN tt ON et.parent_type = tt.name
WHERE $1::text IS NOT NULL
)
SELECT ` + entityCols + ` FROM entities e
JOIN entity_types et ON et.name = e.type
LEFT JOIN entity_status st ON st.entity_id = e.id
WHERE e.type IN (SELECT name FROM tt)
AND ($2::text IS NULL OR e.state = $2)
AND ($3::text IS NULL OR et.domain = $3)
AND ($4::text IS NULL OR et.layer = $4)
AND ($5::text IS NULL OR e.slug ILIKE '%'||$5||'%' OR e.name ILIKE '%'||$5||'%')
AND ($6::text IS NULL OR e.slug > $6)
ORDER BY e.slug
LIMIT $7`
rows, err := s.pool.Query(ctx, query,
req.Params.Type, req.Params.State, req.Params.Domain, req.Params.Layer,
req.Params.Q, req.Params.Cursor, limit+1)
if err != nil {
return nil, err
}
defer rows.Close()
var items []gen.Entity
for rows.Next() {
e, err := scanEntity(rows)
if err != nil {
return nil, err
}
items = append(items, e)
}
if rows.Err() != nil {
return nil, rows.Err()
}
var next *string
if len(items) > limit {
items = items[:limit]
next = &items[len(items)-1].Slug
}
if items == nil {
items = []gen.Entity{}
}
return gen.ListEntities200JSONResponse{Items: items, NextCursor: next}, nil
}
func (s *Server) GetEntity(ctx context.Context, req gen.GetEntityRequestObject) (gen.GetEntityResponseObject, error) {
id, err := s.resolveEntityID(ctx, req.Id)
if err != nil {
return nil, err
}
e, err := scanEntity(s.pool.QueryRow(ctx,
"SELECT "+entityCols+" FROM entities e LEFT JOIN entity_status st ON st.entity_id = e.id WHERE e.id = $1", id))
if err != nil {
return nil, err
}
return gen.GetEntity200JSONResponse{
Body: e,
Headers: gen.GetEntity200ResponseHeaders{ETag: `"` + strconv.Itoa(e.Version) + `"`},
}, nil
}
func (s *Server) GetEntityRelations(ctx context.Context, req gen.GetEntityRelationsRequestObject) (gen.GetEntityRelationsResponseObject, error) {
id, err := s.resolveEntityID(ctx, req.Id)
if err != nil {
return nil, err
}
dir := "both"
if req.Params.Direction != nil {
dir = string(*req.Params.Direction)
}
relType := req.Params.RelType
rows, err := sqlcgen.New(s.pool).ListEntityRelations(ctx, sqlcgen.ListEntityRelationsParams{
Direction: dir,
ID: id,
RelType: relType,
})
if err != nil {
return nil, err
}
items := []gen.Relationship{}
for _, r := range rows {
var attrs *map[string]any
if len(r.Attributes) > 0 {
var m map[string]any
if json.Unmarshal(r.Attributes, &m) == nil && len(m) > 0 {
attrs = &m
}
}
validTo := r.ValidTo
items = append(items, gen.Relationship{
Source: r.SourceSlug,
Target: r.TargetSlug,
Type: r.Type,
Attributes: attrs,
ValidFrom: r.ValidFrom,
ValidTo: validTo,
})
}
return gen.GetEntityRelations200JSONResponse{Items: items}, nil
}
func (s *Server) GetBlastRadius(ctx context.Context, req gen.GetBlastRadiusRequestObject) (gen.GetBlastRadiusResponseObject, error) {
id, err := s.resolveEntityID(ctx, req.Id)
if err != nil {
return nil, err
}
depth := 3
if req.Params.Depth != nil {
depth = *req.Params.Depth
}
rows, err := s.pool.Query(ctx, `
SELECT `+entityCols+`, b.depth
FROM blast_radius($1, $2) b
JOIN entities e ON e.id = b.entity_id
LEFT JOIN entity_status st ON st.entity_id = e.id
ORDER BY b.depth, e.slug`, id, depth)
if err != nil {
return nil, err
}
defer rows.Close()
resp := gen.GetBlastRadius200JSONResponse{Items: []struct {
Depth int `json:"depth"`
Entity gen.Entity `json:"entity"`
}{}}
for rows.Next() {
var e gen.Entity
var state *string
var attrsJSON []byte
var maint *time.Time
var health *string
var lastCheckAt *time.Time
var d int
if err := rows.Scan(&e.Id, &e.Slug, &e.Type, &e.Name, &state, &attrsJSON,
&maint, &e.Version, &e.CreatedAt, &e.UpdatedAt, &health, &lastCheckAt, &d); err != nil {
return nil, err
}
e.State = state
e.MaintenanceUntil = maint
if health != nil {
h := gen.EntityHealth(*health)
e.Health = &h
}
e.LastCheckAt = lastCheckAt
var attrs map[string]any
if len(attrsJSON) > 0 && json.Unmarshal(attrsJSON, &attrs) == nil && len(attrs) > 0 {
e.Attributes = &attrs
}
resp.Items = append(resp.Items, struct {
Depth int `json:"depth"`
Entity gen.Entity `json:"entity"`
}{Depth: d, Entity: e})
}
return resp, rows.Err()
}
func (s *Server) GetGraph(ctx context.Context, req gen.GetGraphRequestObject) (gen.GetGraphResponseObject, error) {
depth := 2
if req.Params.Depth != nil {
depth = *req.Params.Depth
}
var nodes []gen.Entity
var err error
truncated := false
// pgx can't infer the array element type from a nil *[]string (the
// param is absent from the request, not an empty list), so dereference
// to a plain []string first — nil there still encodes as SQL NULL, but
// pgx has a concrete type to work with.
var relTypes []string
if req.Params.RelType != nil {
relTypes = *req.Params.RelType
}
if req.Params.Root != nil && *req.Params.Root != "" {
rootID, rerr := s.resolveEntityID(ctx, *req.Params.Root)
if rerr != nil {
return nil, rerr
}
nodes, err = s.queryEntities(ctx, `
SELECT `+entityCols+`
FROM blast_radius($1, $2, $3) b JOIN entities e ON e.id = b.entity_id
LEFT JOIN entity_status st ON st.entity_id = e.id
ORDER BY e.slug`, rootID, depth, relTypes)
} else {
// Whole-graph view: pick the most-connected entities first so the
// graph shows actual topology, not just whatever sorts first
// alphabetically. Without this the cap fills with exec:* rows and
// drops every host/lxc/service/vm — and every edge those entities
// connect — because edges require both endpoints in the node set.
// Exclude the cognition transactional types (execution/task): they
// are audit records rather than topology, and at ~380 rows they
// consumed most of the old 500-node cap.
nodes, err = s.queryEntities(ctx, `
SELECT `+entityCols+`
FROM entities e
LEFT JOIN entity_status st ON st.entity_id = e.id
WHERE e.type NOT IN ('execution','task')
AND e.id IN (
SELECT e2.id FROM entities e2
LEFT JOIN relationships r ON r.valid_to IS NULL
AND (r.source_id = e2.id OR r.target_id = e2.id)
WHERE e2.type NOT IN ('execution','task')
GROUP BY e2.id
ORDER BY count(r.type) DESC, e2.slug
LIMIT $1
)
ORDER BY e.slug`,
graphNodeCap+1)
if err == nil && len(nodes) > graphNodeCap {
nodes = nodes[:graphNodeCap]
truncated = true
}
}
if err != nil {
return nil, err
}
ids := make([]uuid.UUID, len(nodes))
for i, n := range nodes {
ids[i] = uuid.UUID(n.Id)
}
edgeRows, err := sqlcgen.New(s.pool).ListGraphEdges(ctx, sqlcgen.ListGraphEdgesParams{
Ids: ids,
RelTypes: relTypes,
})
if err != nil {
return nil, err
}
edges := []gen.Relationship{}
for _, r := range edgeRows {
var attrs *map[string]any
if len(r.Attributes) > 0 {
var m map[string]any
if json.Unmarshal(r.Attributes, &m) == nil && len(m) > 0 {
attrs = &m
}
}
validTo := r.ValidTo
edges = append(edges, gen.Relationship{
Source: r.SourceSlug,
Target: r.TargetSlug,
Type: r.Type,
Attributes: attrs,
ValidFrom: r.ValidFrom,
ValidTo: validTo,
})
}
resp := gen.GetGraph200JSONResponse{Nodes: nodes, Edges: edges}
if truncated {
resp.Truncated = &truncated
}
if req.Params.Include != nil {
for _, inc := range *req.Params.Include {
if inc == gen.Status {
health, herr := s.entityHealthByID(ctx, ids)
if herr != nil {
return nil, herr
}
resp.Health = &health
break
}
}
}
return resp, nil
}
// entityHealthByID returns entity_status.health keyed by entity id, for the
// given id set (used by GetGraph's include=status).
func (s *Server) entityHealthByID(ctx context.Context, ids []uuid.UUID) (map[string]gen.GraphViewHealth, error) {
health := make(map[string]gen.GraphViewHealth, len(ids))
rows, err := s.pool.Query(ctx,
`SELECT entity_id, health FROM entity_status WHERE entity_id = ANY($1)`, ids)
if err != nil {
return nil, err
}
defer rows.Close()
for rows.Next() {
var id uuid.UUID
var h string
if err := rows.Scan(&id, &h); err != nil {
return nil, err
}
health[id.String()] = gen.GraphViewHealth(h)
}
return health, rows.Err()
}
func (s *Server) queryEntities(ctx context.Context, query string, args ...any) ([]gen.Entity, error) {
rows, err := s.pool.Query(ctx, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
items := []gen.Entity{}
for rows.Next() {
e, err := scanEntity(rows)
if err != nil {
return nil, err
}
items = append(items, e)
}
return items, rows.Err()
}
// sqlcEntityToGen converts a sqlcgen.Entity to a gen.Entity.
func sqlcEntityToGen(e sqlcgen.Entity) gen.Entity {
out := gen.Entity{
Id: e.ID,
Slug: e.Slug,
Type: e.Type,
Name: e.Name,
State: e.State,
Version: int(e.Version),
CreatedAt: e.CreatedAt,
UpdatedAt: e.UpdatedAt,
}
if e.MaintenanceUntil != nil {
out.MaintenanceUntil = e.MaintenanceUntil
}
if len(e.Attributes) > 0 {
var attrs map[string]any
if json.Unmarshal(e.Attributes, &attrs) == nil && len(attrs) > 0 {
out.Attributes = &attrs
}
}
return out
}

View File

@@ -0,0 +1,290 @@
package httpapi
import (
"context"
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"strconv"
"strings"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/domain"
"github.com/dtoro/oikos/internal/httpapi/gen"
"github.com/dtoro/oikos/internal/observability"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
)
func (s *Server) CreateEntity(ctx context.Context, req gen.CreateEntityRequestObject) (gen.CreateEntityResponseObject, error) {
if req.Body == nil {
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
}
// Check idempotency if a key was provided. The idempotency scope is the
// calling actor, so replays are per-caller.
actorType, actorLabel := actorInfo(ctx)
actor := actorLabel
var bodyHash string
if req.Params.IdempotencyKey != nil && *req.Params.IdempotencyKey != "" {
key := *req.Params.IdempotencyKey
q := sqlcgen.New(s.pool)
cached, err := q.GetIdempotentResponse(ctx, sqlcgen.GetIdempotentResponseParams{
Actor: actor,
Key: key,
})
if err == nil {
// Verify the request body hasn't changed.
bodyJSON, _ := json.Marshal(req.Body)
bodyHash = fmt.Sprintf("%x", sha256.Sum256(bodyJSON))
if cached.RequestHash != bodyHash {
return nil, fmt.Errorf("%w: idempotency key %s used with different request body", domain.ErrConflict, key)
}
// Replay the cached response.
if cached.ResponseCode != nil && *cached.ResponseCode == 201 {
var entity gen.Entity
if len(cached.ResponseBody) > 0 {
if err := json.Unmarshal(cached.ResponseBody, &entity); err != nil {
return nil, fmt.Errorf("unmarshal cached response: %w", err)
}
}
return gen.CreateEntity201JSONResponse{
Body: entity,
Headers: gen.CreateEntity201ResponseHeaders{ETag: `"` + strconv.Itoa(entity.Version) + `"`},
}, nil
}
// Forward cached error response.
return gen.CreateEntitydefaultApplicationProblemPlusJSONResponse{
Body: gen.Problem{Status: int(*cached.ResponseCode), Title: "replayed error"},
StatusCode: int(*cached.ResponseCode),
}, nil
}
}
id, err := uuid.NewV7()
if err != nil {
return nil, err
}
slug := req.Body.Slug
if slug == "" {
slug = req.Body.Type + ":" + req.Body.Name
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
q := sqlcgen.New(tx)
// Validate type exists and is NOT abstract.
var isAbstract bool
if err := tx.QueryRow(ctx, `SELECT is_abstract FROM entity_types WHERE name = $1`, req.Body.Type).Scan(&isAbstract); err != nil {
if err == pgx.ErrNoRows {
return nil, fmt.Errorf("%w: entity type %q", domain.ErrNotFound, req.Body.Type)
}
return nil, err
}
if isAbstract {
return nil, fmt.Errorf("%w: %s", domain.ErrAbstractType, req.Body.Type)
}
// Get default state from lifecycle.
var defaultState *string
var lcDefault string
if err := tx.QueryRow(ctx, `SELECT ld.default_state FROM lifecycle_defs ld
JOIN entity_types et ON et.lifecycle_id = ld.id
WHERE et.name = $1`, req.Body.Type).Scan(&lcDefault); err == nil {
defaultState = &lcDefault
}
state := req.Body.State
if state == nil && defaultState != nil {
state = defaultState
}
// attributes is NOT NULL; the column default only applies when omitted,
// not when an explicit NULL is bound — so default to an empty object.
attrsJSON := []byte("{}")
if req.Body.Attributes != nil {
attrsJSON, _ = json.Marshal(req.Body.Attributes)
}
// Insert the entity.
inserted, err := q.InsertEntity(ctx, sqlcgen.InsertEntityParams{
ID: id,
Slug: slug,
Type: req.Body.Type,
Name: req.Body.Name,
State: state,
Attributes: attrsJSON,
})
if err != nil {
// Duplicate slug.
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
return nil, fmt.Errorf("%w: slug %q already exists", domain.ErrAlreadyExists, slug)
}
return nil, err
}
// Convert sqlcgen.Entity → gen.Entity.
entity := sqlcEntityToGen(inserted)
// Cache idempotent response.
if req.Params.IdempotencyKey != nil && *req.Params.IdempotencyKey != "" {
respBody, _ := json.Marshal(entity)
code := int32(201)
if bodyHash == "" {
bodyJSON, _ := json.Marshal(req.Body)
bodyHash = fmt.Sprintf("%x", sha256.Sum256(bodyJSON))
}
if putErr := q.PutIdempotentResponse(ctx, sqlcgen.PutIdempotentResponseParams{
Actor: actor,
Key: *req.Params.IdempotencyKey,
RequestHash: bodyHash,
ResponseCode: &code,
ResponseBody: respBody,
}); putErr != nil {
return nil, putErr
}
}
// Audit.
entityID := inserted.ID
if auditErr := observability.Audit(ctx, q, actorType, actor, "create",
&entityID, "POST", "/api/v1/entities", "",
nil,
map[string]any{"type": req.Body.Type, "slug": slug}); auditErr != nil {
return nil, auditErr
}
if eventErr := observability.Event(ctx, q, "entity.created", &entityID,
"info", "oikos-api", "",
map[string]any{"slug": slug, "type": req.Body.Type}); eventErr != nil {
return nil, eventErr
}
if err := ensureDefaultChecks(ctx, tx, inserted.ID, slug, req.Body.Type, inserted.Name, attrsJSON); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return gen.CreateEntity201JSONResponse{
Body: entity,
Headers: gen.CreateEntity201ResponseHeaders{ETag: `"` + strconv.Itoa(entity.Version) + `"`},
}, nil
}
func (s *Server) PatchEntity(ctx context.Context, req gen.PatchEntityRequestObject) (gen.PatchEntityResponseObject, error) {
if req.Body == nil {
return nil, fmt.Errorf("%w: request body is required", domain.ErrInvalidInput)
}
id, err := s.resolveEntityID(ctx, req.Id)
if err != nil {
return nil, err
}
// Parse If-Match header (quoted version string).
ifMatch := strings.Trim(req.Params.IfMatch, `"`)
expectedVersion, err := strconv.Atoi(ifMatch)
if err != nil {
return nil, fmt.Errorf("%w: invalid If-Match header %q", domain.ErrInvalidInput, req.Params.IfMatch)
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
// Get current entity for version check + lifecycle validation.
current, err := sqlcgen.New(tx).GetEntityByID(ctx, id)
if err != nil {
if err == pgx.ErrNoRows {
return nil, fmt.Errorf("%w: %s", domain.ErrNotFound, req.Id)
}
return nil, err
}
if int(current.Version) != expectedVersion {
return nil, fmt.Errorf("%w: expected version %d, current version %d",
domain.ErrConflict, expectedVersion, current.Version)
}
// Validate lifecycle transition if state is being changed.
if req.Body.State != nil && *req.Body.State != "" {
fromState := ""
if current.State != nil {
fromState = *current.State
}
if err := db.ValidateTransition(ctx, tx, id, current.Type, fromState, *req.Body.State); err != nil {
if errors.Is(err, db.ErrTransitionInvalid) {
return nil, fmt.Errorf("%w: %v", domain.ErrInvalidTransition, err)
}
return nil, err
}
}
// Check idempotency (note: the spec doesn't define Idempotency-Key for PATCH,
// but we handle it if the generated code ever adds it).
// For now, no idempotency check on PATCH.
// Marshal attributes if provided.
var attrsJSON []byte
if req.Body.Attributes != nil {
attrsJSON, _ = json.Marshal(req.Body.Attributes)
}
// Perform the update via sqlcgen.
q := sqlcgen.New(tx)
updated, err := q.UpdateEntity(ctx, sqlcgen.UpdateEntityParams{
Name: req.Body.Name,
State: req.Body.State,
Attributes: attrsJSON,
SetMaintenance: req.Body.MaintenanceUntil != nil,
MaintenanceUntil: req.Body.MaintenanceUntil,
ID: id,
Version: int32(expectedVersion),
})
if err != nil {
if err == pgx.ErrNoRows {
// Version mismatch or entity not found.
return nil, fmt.Errorf("%w: entity was modified concurrently", domain.ErrConflict)
}
return nil, err
}
entity := sqlcEntityToGen(updated)
// Audit.
patchActorType, patchActor := actorInfo(ctx)
if auditErr := observability.Audit(ctx, q, patchActorType, patchActor, "patch",
&id, "PATCH", "/api/v1/entities/"+req.Id, "",
nil,
map[string]any{"version": expectedVersion}); auditErr != nil {
return nil, auditErr
}
if eventErr := observability.Event(ctx, q, "entity.updated", &id,
"info", "oikos-api", "",
map[string]any{"slug": entity.Slug, "type": entity.Type, "version": updated.Version}); eventErr != nil {
return nil, eventErr
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
s.entityCache.Invalidate(entity.Slug, entity.Id.String())
return gen.PatchEntity200JSONResponse{
Body: entity,
Headers: gen.PatchEntity200ResponseHeaders{ETag: `"` + strconv.Itoa(entity.Version) + `"`},
}, nil
}

View File

@@ -0,0 +1,60 @@
package httpapi
import (
"context"
"encoding/json"
"github.com/dtoro/oikos/internal/httpapi/gen"
)
func (s *Server) QueryEvents(ctx context.Context, req gen.QueryEventsRequestObject) (gen.QueryEventsResponseObject, error) {
limit := clampLimit(req.Params.Limit)
var eventType, entityID, severity, correlationID *string
if req.Params.Type != nil {
eventType = req.Params.Type
}
if req.Params.EntityId != nil {
entityID = req.Params.EntityId
}
if req.Params.Severity != nil {
severity = req.Params.Severity
}
if req.Params.CorrelationId != nil {
correlationID = req.Params.CorrelationId
}
rows, err := s.pool.Query(ctx, `
SELECT id, ts, type, entity_id::text, severity, source, data, correlation_id
FROM events
WHERE ($1::text IS NULL OR type = $1)
AND ($2::text IS NULL OR entity_id::text = $2)
AND ($3::text IS NULL OR severity = $3)
AND ($4::text IS NULL OR correlation_id = $4)
AND ($5::timestamptz IS NULL OR ts >= $5)
AND ($6::timestamptz IS NULL OR ts <= $6)
ORDER BY ts DESC
LIMIT $7`,
eventType, entityID, severity, correlationID, req.Params.From, req.Params.To, limit)
if err != nil {
return nil, err
}
defer rows.Close()
items := []gen.Event{}
for rows.Next() {
var e gen.Event
var dataBytes []byte
var entID, corrID *string
if err := rows.Scan(&e.Id, &e.Ts, &e.Type, &entID, &e.Severity, &e.Source, &dataBytes, &corrID); err != nil {
return nil, err
}
e.EntityId = entID
e.CorrelationId = corrID
var data map[string]any
if json.Unmarshal(dataBytes, &data) == nil {
e.Data = &data
}
items = append(items, e)
}
return gen.QueryEvents200JSONResponse{Items: items}, rows.Err()
}

View File

@@ -0,0 +1,80 @@
package httpapi
import (
"context"
"time"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/httpapi/gen"
)
func (s *Server) GetFleetHealth(ctx context.Context, req gen.GetFleetHealthRequestObject) (gen.GetFleetHealthResponseObject, error) {
resp := gen.GetFleetHealth200JSONResponse{}
resp.Entities = []struct {
Health gen.HealthSummaryEntitiesHealth `json:"health"`
LastCheckAt *time.Time `json:"last_check_at"`
Slug string `json:"slug"`
Trend *gen.HealthSummaryEntitiesTrend `json:"trend"`
Type string `json:"type"`
}{}
// Exclude 'check' entities (internal probes) — only entities actually
// being monitored should count toward fleet health.
rows, err := s.pool.Query(ctx, `
SELECT e.slug, e.type, st.health, st.last_check_at
FROM entity_status st JOIN entities e ON e.id = st.entity_id
WHERE e.type <> 'check'
ORDER BY e.slug`)
if err != nil {
return nil, err
}
defer rows.Close()
stale := 0
for rows.Next() {
var slug, typ, health string
var lastCheck *time.Time
if err := rows.Scan(&slug, &typ, &health, &lastCheck); err != nil {
return nil, err
}
switch health {
case "healthy":
resp.Summary.Healthy++
case "degraded":
resp.Summary.Degraded++
case "down":
resp.Summary.Down++
case "stale":
stale++
default:
resp.Summary.Unknown++
}
resp.Entities = append(resp.Entities, struct {
Health gen.HealthSummaryEntitiesHealth `json:"health"`
LastCheckAt *time.Time `json:"last_check_at"`
Slug string `json:"slug"`
Trend *gen.HealthSummaryEntitiesTrend `json:"trend"`
Type string `json:"type"`
}{
Health: gen.HealthSummaryEntitiesHealth(health),
LastCheckAt: lastCheck,
Slug: slug,
Type: typ,
})
}
if stale > 0 {
resp.Summary.Stale = &stale
}
return resp, rows.Err()
}
func (s *Server) ExportSeeds(ctx context.Context, req gen.ExportSeedsRequestObject) (gen.ExportSeedsResponseObject, error) {
exports, err := db.ExportToYAML(ctx, s.pool)
if err != nil {
return nil, err
}
return gen.ExportSeeds200JSONResponse{
Ontology: string(exports["ontology.yaml"]),
Inventory: string(exports["inventory.yaml"]),
Policy: string(exports["policy.yaml"]),
}, nil
}

View File

@@ -616,13 +616,10 @@ type EnrollResponse struct {
AgePublicKey string `json:"age_public_key"`
// InfisicalClientId Infisical UniversalAuth client ID
InfisicalClientId string `json:"infisical_client_id"`
InfisicalClientId *string `json:"infisical_client_id,omitempty"`
// InfisicalClientSecret Infisical UniversalAuth client secret
InfisicalClientSecret string `json:"infisical_client_secret"`
// MachineIdentityToken Infisical machine identity access token
MachineIdentityToken *string `json:"machine_identity_token,omitempty"`
InfisicalClientSecret *string `json:"infisical_client_secret,omitempty"`
}
// Entity defines model for Entity.
@@ -8225,177 +8222,177 @@ func (sh *strictHandler) GetTrends(w http.ResponseWriter, r *http.Request, entit
// Base64 encoded, gzipped, json marshaled Swagger object
var swaggerSpec = []string{
"H4sIAAAAAAAC/+x963IbudXgq6C4WzXUpClq7JlkI9f3Q5E1thM71lqafJuKXBTYfUhihAZ6ADQlxuWq",
"/NoH2MoT5km2cO1uEk02L7KcqfyxJTUaDZwLcO7nUy/lecEZMCV7p596M8AZCPPjxTWe6v8zkKkghSKc",
"9U57H0DyUqSA5iAk4QxNuEBvJoN3WKWzXtKT6QxyrN9TiwJ6pz2pBGHT3ufPn5NegQXOQbkPnJdCcrH6",
"ifcF/qUElJrHaCJ4jjAqBMwJLyUSIAvOJHwjEYMHNbLDekmP6Hd/KUEsekmP4Vx/PDxsX1bSu2CKqMWb",
"bHUlP/305iXiAklaTlEfjqfH6HbGpTqdlWNB5O2R/2yB1az6Ksl6SU/ALyURkPVOlSihywquaBkBuH3W",
"WMK9PM1xOsgJI7cJuqUP6WmKs2zRth797pYr+lHw/Jrotz9FAaux0gDrhIscq95pL8MKBkq/mkTmfZNB",
"XnAFLF38CRaruz2nBJgaTIGBwAoydAeLF0hAQfFConuiZoShZ9/PkABVCobUDBAXZEoYpoE0PBQsMVeL",
"rn18oL9eX3+OH94Cm6pZ7/S7Z/8ruvSJpfFVDF3jaUWmhAv06uL6Bfr+u2eIs8AnOZG545H44ioe2gZR",
"b0lOVBuWqHlYnyCDCS6p6p3+cJLoPZO8zHunz070b4TZ374LuydMwRSE+dA1X0cPim9PDZ/1Ti3GzHlw",
"CSwjbHpWFILPMdV/SjlTwMz+cFFQkmIN8+HPUgP+U+2D/1PApHfa+x/D6jgb2qdyGCY0n1yitxlmU0BS",
"4SlkLxBGOSg8wO4NdI8lSgUYSuxnJaYDvSLB6VHvc9K7FHxMIV+z0MKO+M12C/bzRtZ7IQQXqP/hx3P0",
"++9/+J1ZxhWZMkx/KjSss4NBzc4aW4P7EpJ+hMe8weLZFJg6SxWZE2UYvBC8AKGIRTJ2T0aWGj71gGma",
"+1tPcU5HKabUMACWnGkq0d9OiWagXtLL02Lk6Q5kiqnZV+/jCmklPaxXMSJZhGmSXsqFAPuyG8JKSvGY",
"gue4lVeyUtjxuVwzPvBL0gNzbHedvrHQ2iyEFaUayTLPsVh0momXattXJEi5BShkmaYg14FhzDkFzPRg",
"xe+AjVJeWnLcDDdDBvZQ6bAWK7R0vHuqY/Vv9opWslejlGSJNiuy4uOfIVX6e/WzaZWuLXutkps9QEZY",
"dV2spfps/TubadbNMe5GBvBQEAFyq2Vakgljy9LCdXnYHWFZndfhAdJSWaYuOCXpYpCag1j/jpUCwQYG",
"Ge0MXuAF5TgisxkRSeOGS8iQnR1lZDJpB1mFX0Hk3Sil2JL3Kuk7CW31gcKqlPUtFvYy01RlaAYyc5Yx",
"Yn6wsLZi4pzfQRbdoyztwtbKhFoCCvdVylkKgsnN5BHjBycnOlJuQMPhMOy0QS4NEl/HNh9KCluxDi4V",
"ZzxfjCjMgdbhq59U14DhB5iDiMLRncX+xmnC8h1eoDEgPJZK4FShPmEzEERJlPF7dtSF0TpywSbiSnkB",
"I7vW9SjXKlcBYmDHIj4HIUgGsstanTgau25iJBGnhSW0VLNuQv65oZOnJ4F9cbOemToBLQqqMiPqgimx",
"2A5EqeKi6/VtBy9LX+YW7CU9/UWsrMq8kAq8kpeVtAWyuwhToDChta1UEDiM2JSDmvFuUxhNuZPYY8we",
"I1J0G22OyVHKM+go9xxAkqkwGxg3TmWWEK9AKT3jCqndWc0cHnBe6DX3ppSPMT3WFDzCqYodbqVVCraS",
"HuaYlnF27H5K3Rk93s60/hw6n0F6t7rZlLMJidhd/oIpsXqOPmrd7RehV43WOhnWhN+O94LemphjOpJx",
"al6WnmZKFXqeVP+bEXmnL2AQamCuZA2OTJCJxlJhJRApZwO7tSgHUyzVaAaYqohx43pGJEo16L6RiN8z",
"lHOpkIAUmEJzEBlJ1TE6Y8hy7jcS2ZkQkUY0uedCKsQn+hcJCKeCS4n09epAZyeXCZIcKf0xItH9DCsE",
"DwXFhEn07f1s8S3C/hN6QAZTgTPIjtGfS0pRyRSh5nNmMjQh+qOiZPJYXxAebmZhBj7udf0jv9cndcnu",
"mP7pY4cr1MBLlMzRehNe/z0DZvdhl6IHIyvvhuX+FzIXlV6gXt9u0n2rLKqwmMIGobFPmFSYpTAwN1vW",
"ScyxE7eIUW52/RD19b9bzUxy4FprjTPAmtMg6f2dsy664hp51/F2jQ3rK6p4vMPx0ibfVIfMuhMkGOfa",
"tOnmSRGG//bkJHmCc6MDBW6iofWUEDb4XXR/HvPrMV1HciveLr1ldxe0bcJT5LJfR++fY4vUMiSZOEve",
"bvJz6u+/lSFje6LhjFgdlijI43Kw+wMWAi/iwt9BrB8dL05nKhgZNGXA0nUHASvzsYV+ZV2MIVZAyvMc",
"mLG+BLDuazkQvFSwrLwMrCxVU2BmnLaYAoyttbOJ7o7QzoMrbt3hCI2rOna3TTvuEqlstBlYT9A5Zwoe",
"VITijdluQijIkbUdRWxBl1jNpBY+zGikLz1RmhUj8yZSWtDwrydbEL4kjtqWpCWSg1Q4L4yOrgUSBg8K",
"FZxSpGEHUrVd+BGFo5CWtKftO7wWJSAyQcd69PEC5/o7KSk07GRtZzHLLKddQGfGDb89lqDK4ljOdodZ",
"7RpfMsFwxhVnJEWpRXdwmjmmTTZpAWsvZkNIV5AKsErWirIjV5f0hk2IJCmmSJoXkR6GsLF8kzEFpJy0",
"mprZt4DDqv4io8t+ieVszLHIripT/hILOMOKHHljY/SyMVIzATkaL0ZaLTVki7OM6K1ietmYc/X1Jeuq",
"FfOMQV9qoECGxgtk501a9Hn3cX/pH/jbTuFd/fRcnxDCbXhpKv3MzzQu0ztQdrIfBjlhpQLkr/CkofDo",
"i7KO6yZC7ETdr7ngFtlA3W5e/0KMWpZ5LVjYD4R2P10c84lVc559P4sholItm+AKWlh8AVodiz7xelz0",
"oVSYRhBu1sfHUuPU7IPTDPQJjVmlN34jtdIJqaaFFFuBIiZ4emVxM+Y6qpwrILPXvcWdVhWd2XFX9PEC",
"GOozzgYCJKdzyI6c93YVn/5zK6ta2toKZ8dOmoD8+JaSyBkWp90GO8cgdsEEp/SDu2NXaG3GpfJexSZs",
"zlJVYor8AGeqQGDmI2yKcpzOCIsycA5y5iyCzUmvbIyUfo7eXBphQAuo5vyaW8OSFZtalapNQUD38pTB",
"/YDiQvHiaKOV0Ey7Dm4uciYmZ40KQeZYweguFrFz9upicHVx/uHievCni78Ojo+PzXYp15dnBqlYFG17",
"NXOXY0rS+NR4Ct/p+ewYTaNm6qv3l1c1KSduUnPX98jez04Wbrvjf2JESxCYnpVq5q509OZlp5mtfLD1",
"7O61GFFZeht5ghkZH/q6D7g3KhKzcgqyL24ijSUsJCsoj4OzHRRxMlPxcBClBBmXqnGOVa/tojy2WTJr",
"VwBkzkqZoHtrr4OacTHnjChu3bPb2A795fPxcC5Do5ibuylqbDSLd/bWeyxRY4s7WxZzrC8ShlkKI2NZ",
"3T0EwR+5rcdc5WOoxVX2WhwqHWNC2mxLO7kotnOXOmOTuw/N7qs5GuTcWE47x7Q6TBt80+a5wFOsdV5D",
"3/oD30gUXhy5MLDIpzdirR07zZW8tAY8abUlQJRMIF2kVC/EGfdGS6rDKh6XdMVSKuOvR1qcCV57qMzP",
"3e7CJpLaEXAZj/k8U4iC4TYWRIYJAZpJlLsVFgIkMHXcS7bA3TsQJhBxvoLDLoj7IpwbR/W1sSJVGLaK",
"AeorgZk0Qmu1p7i40oKAa0cGLTAc1WNb6wv649X7P6MrD6qN9rvGy5FtZ1wDN/qIyJGnw7g5mOIFiLrx",
"LweF7QUqsDVJlUJjZcrnIAz6jLI3ZaQ1/ikAuquZrxWhBRb68vbsttm4aGA6WuuUWY2HMuFcYO7PQkBq",
"QlU/7nXgutPVIcZDuYmOsJKPa+lr4yk7WgnBfgzKCf6OCaYy6gBaoaRDktDOJLP+uI3jaT1CWvwxB8HH",
"rrQZPaLmLs562W+0fUwMVvgRI2LqVoQa7fBe0rvHwpvoBVFano97IIxKGzdwyu4CVQg8CoKfNQwcC0wk",
"ZFuEu7j7u2ZMcEuMklaION3Kd1aLIt8cWOVMGV3Hpw2fXue3uAab2jM0+JH8dltHyXd29AmcR6SlqztC",
"KbJPUX9VZrJP3GFxFJOYBEhz4u7v4XtED50dXLsZNwN2naQu9qWeSLSzi7duhjvbeLG14c/O1meOH83H",
"k0XtZzt4gomNvtDLy0a8NJZwfcNR+3fB9Q+jMU7v3G/6x5F77+M+Ls/aQiKS3dYx1CF4eltnaDi+Wg2c",
"1SlWnawCDLbXc1SEJbBsuzrrNL7EitacbJyDOTeh+5BZF1mfF9ZofdRLtg5Xqqdgbrwc3Fxrox9fCVzM",
"/kLgfhWGkE2hGQCxLkHqg0OgnJEi5oKp7FBtZvsdjUuReNqIm4xkaHBTnpw8h2DrchqptXkRltIyg/+y",
"NGnMR85DDdFIR8azLYDj7H0RsChRstRnqsWd2fpTKMWFWdSMqIj/elnG5Db83WIwhvbXBgatvlTvumhs",
"cMmFEPB5KLytWPcOHQkooBkBRnJ9Fvu8Or1kF+ilzAe2C4PsJv03jS0Ohl08mLINVf9xG7a4DZch7wBY",
"+eWicP8T4/dU881rErlWutqpCbuDbBTloo1xIQKzu06BW+1CDSNFEbtEXpPpjJLpTKPGZF/7CJNOfGUj",
"/jtnCCiiKKzZccWHGU/L3IaNiJKNOb8z1qA5SEWmbTlvm+3NdgExJL/1qv5LfWSvclTdFBs1VGTttsAt",
"sa1A5ERLETu9HKyJEW3g00Tw/BR9UvwUfXKgkqfobwznkA0Mqybo+Pj44+fPnze6t4lPdjP3yoqxuraO",
"GLzfgRIkvQLhIBy5bBZtildu3m2JIqS0LOqUJPB9L+l9N9P/tEQOGmlw3cWG59NO7LdFFm+OHzpNmRPW",
"adw2FoaQVbJUuATfIwsL5LNFNnx2WbiUne6tcOmuE4+uzaBoSMXCKgSOCgLOK0TGFnFpI2G3M24UBSUg",
"22Oxm2G1S2kOhErOEOX3INCYlyxLtMRW2CASmNv3bOL38IdeBKXNMfFbWatwpVg7ZBtHajAQ7CVuFRWs",
"V579UmKBmSKsLTJ8iwzi2aLgagaS/N3mHvjF+0T1JYOluUDCmI/tifvroLmbu7NBSTXd10OqQUormK+p",
"xevCOGvFNpZvr6VswlrWpxBcRG6KHwnQbGDyMGvhOMgOR/3vnz07ao/yM26++PHcpjkvwc7OEMZ3OVV8",
"vs4G2omlGmwSSoLXoYfHvFSnY6rlsVrwQCnIZs3bfGatu+VSqx5yrQ1jnVP73ZuXCUq5AJkggfNRPk5Q",
"RuTdaDpOECkSpCAvqAlGzE1MW4K02E5SkNGgRC4j8uIVLafenXsp+EPOH0xkmAu6qsUoRG0Z8Qiz12WO",
"2UAAzvS5gpw/pGPkl/kufUhPfwZKFxPCtneV04c0QfM8QVygjKd3IEwVG0xYPbS6u7PcAW8DjtsCyqok",
"6m72gxANGDU7BcMYevPSRBkInN6hwi+DsKn+ZSrA2N82XBPR67i3tIS1274KnLi0aX2yRCqdzUFgSu3B",
"g8ikufBg+NzdAtDirD8vhQAWoiZaQzCkgmKd5GjWPcpBSjzt5jyeEEbkbH/782PYsEMAqiiZ84gZxSzg",
"Qd5pNbPlclVQdLCG6FFrT8m1yQKOGT2+LHpiszRskxvO2Y2Oj8rzFz8sXUHAwC6dTbzhtLWxI2smaBNR",
"zeU9MsXottEPSDZSfFfaWcaJhU5SGZ/dWVlb2yYUdYvy6oyY9TbzdnxsfK+bxS8OkE0wiMf5pFhkhGG6",
"5LrmDAaKD7gJy3a/5Jhp4tH/Vc/8b+bhRtt5zPLBtFAK+4XYOEtSpwI0vW0zrze+H4/HqK+p+YWkAfUo",
"3oi8O/ee0HhS0qj6ZIU25hBWFRnxP9pEN5G3nK4hiRVTrXK2KFebcNmCnzh8VjcSWUYMOK5w3iopG0N/",
"R5ev15E6XqpCqpEEYFt56ycUF+uUwRmn2Sjj92zfWMJtq4RVoSFWgB8403dcq99635TcAV2MUlx25Ou8",
"VHvHU/I0NULXeoPHAcJ0NoqClenQBdzg9M77ALxxwXxHb9uqqTZnqBKFPu5yyVcx9jZx8hClynxRsloM",
"kJONVuC9zCZL1BPl5DtC6V42tc2BOCaPdlRZDjq+0bmq3zb2sVLuKVSvCRi0mfok29LgXwieQlaKmPjp",
"ox4zZAThoY0fGfoAkKGPCyooZujD88Hvj1ZCsb3fbhQil9pq8DB9Dnp7ZL7iC9+8kXooUjzuwq27s4/d",
"kOeV1ihi7tMV1W3nqSxcDzFXRBuSIfAnngjZ3V6aCTxxsQs+iCHYSQVMjE22psxtiEP2xlKxPu2ksmfv",
"ZjpdSREJxtNgFK1YoPWMunLK53IUap5jFjGavOLBWGaKB7o4uV68KKirQLnMOESF6mYxd3rGSz3AmJlk",
"XOjqnnciIF4ST+ltqNYUWIoX3UusaKW/GVst5ayX+Ko5Jh+ctVy6bXffFoCOF7757cnG0gdu3UlAd4xK",
"rr1TagmAjOeYLlqkaSKgCirbIYJkVeDkmuOidldiHHOUMMACFYL/bD+doN9lyEb8bfYltvtNJeUxzemt",
"/dyEKFSAQBlebO0UDG66ClrRwAwJaaklFJOQ4qoFABYgzspYruJ7pxYN5wTuQZwiPUxLT3fo/ZuX5+iP",
"/31dj3clbHB2+Qb96x//ROc4yxY3bMLFPRbZAJe2FFsGE2ASBoQNMijULEGM27wwZ73REpoo1ezo+IaZ",
"Et6nxixIUmTXaZNJbZn7KvO0b4p8oVsTKH2r3/Vl4A0xmTcrajesZAqKG5nWVSp3yQ+ukHymuOCrQW3n",
"tuz6QN/lgPRmfYGV9+SOSzTjOVA8Ru+vjpEpWTchFHwNum+/DZu8YWaX336L+qaSO07VwMiFR6foFTce",
"AxBIqnIsERaAqkYE90TNEMcFGehjbwosuWE271Wivv/8+ds3CZqUWipBP72RRxZeBsw4ByQLSI9v2A07",
"52yu0clZTT55fnR6wwbownqh9Nd9mXd021ZU/vZYv/KWSCVRKQHdfjJ3dFLvjfH51i7eNdQo8JQw6/Dq",
"u4MGmUYB6IeTBOX4AT07OTky8/7EJJ4Aunx/dW2LnxQK3S51UbhFfduPoaB4ge4Jy/i9fftdaQ4FJFzL",
"EIlSLMQC3brb7vYFenVx7To5SHR7cY2ntwm6PLs+f418/Aa69Y0RblHftVTwrRTsZ0LNnQpmz58//z36",
"6frcPL9wQUnmKc4yAVKadY2b0aWo3+ztYRB1PQP07vzSlgOZ4BRQXyoBODczvL6+vkwQn0xISjDVBHT1",
"8k9HNoe4ZCYSXaHbYZ4WtzeMs4oQxoRhsUCYZXowL02FRMtLlq41y7ogoRemWqIpw4PuBS5uWEVPVj9G",
"JqUGYemKLALLCk6YkpYfKUnBuWIck13a7G59XAvqGFOeDodO8z52nuShywKv+RF7lt3OLt/UpJbT3nfH",
"J8cnRs0tgOGC9E57z49Pjp9bJ/DMnHdDc0gMcK01gLs0rRGIcPYm6532/ncJYtHsItDsHPO3eAuKWiH3",
"NQ0zWt5tVH7fYYJ66Mbal2OSc7W5Yei70mGs68jRYaTrtNNhpG0n8vnjUmuOZycnWzWWWAoi9GpDJ/2h",
"ifqIPlJv+7N91TKzhMgdvXLl+CUgYMrEcX1OKsEsvoUAs1oLj1okq+2NgcYww3NiamQYMzueSmPTtmGm",
"Y+LNrg8Dv3BbS7N32rPigJl1GEqntHKSvhbOwqhOTBS0jgqXh6to38Y83oqz8sm9+wT86ngjtLJ5OrYI",
"BLU/P2gCRbhGoZ4XqhJA2zDC8BPJPg9DwxgN6ybFb0BwaASmcVy4AJEmS700PTUCGpItv7DU/srSkomG",
"+QPPFnuQUX3TIa3Vsqnl0kXgzKhqxnhL5G9L8ZnX787Oq7YTVjfoS8KmFAalhAT5vCknUg8kyWBzmaKw",
"jTglNhtjfd6TEXftGfXSLRIJSLnIIDvEzWBxZUJ0gC3qsHTQbQVoV5YJXjfLNGVm4/3XyGBmSDfZq156",
"fhfpy/YveEzBq03s42yXN1cS+P4j9O13sVWdL57watOL8OIe6iuJXl5cnR8dgr3NzNvLe02etZXy14p7",
"53ZIJ6ZdEbs60n6I69iBWX0l9ZVXa5l9vy7Ktm0nno6oHUUcSFizTQ0ymBDm0l8qgnYVHjdIbG2SlY2B",
"stB6QrFqIypdrFYneeS7w346il6bOH4A/NqZEHY47tvTZoDlIMMKJ8ibKX931BnnsePLCOn7yua+PEyT",
"hEzVmB0pyDV3fVTSsVVtvrAk20o5vmHo/pRjZzKyK7GmVUdEuxJKozLKhgtvaWw3K0eoY/ClJU5fGX/V",
"1rFVQ4Bf3SXZ7G/xhLflEjkd4Fh1M4LW7KziqGXLGSAXTchLOfBPkFHLkBKY0KNd7SHOKzW0JYwNaqLJ",
"Lr5OpPR1ipPg7pIIM4SngO5gUWAiEtcF2fy9vfBsYjwateTYRtQXb6Q3HKNzTCkIWy8RUwE4W6AZnkO9",
"1RMz1w6DTJ8ujewIE+hlHRzNU8FWND73dfkf4zRvFpv+wgf6UsXmWJNoMyIHpkJLdOsBtE0MWBYQdgD6",
"th9DGDG498WN//WPfyIiZQmehjz91GgnLKGicke4LSRuWxQ2KPyTpOX08zCtmoREwzA+OA/j/YykM9cL",
"xPT/SKxbzZKtKStt+2349hbItPkwRDwlc2BIeV+j8TIz5B3ApsGH7V7GpAKcIT5BU6JQUVIaI9JXoJoN",
"TlburdgWEGd04RYnw+KIrNZlW4M/f/7cBLvF7z6bhblls/aPjymiNCARO5VdW5AMqMIHoNlXoBwZpPWZ",
"HURxBc5tiTPZSaq9ouW09/ljhLJl1bVkurbI+MAEmBjroHnDOpMzX3jXTvuNXDmx2wnTN0x5dLz7D0Xw",
"ftWp98qBVFsPuXVNXr4sMWS+BcywVgwnKgm/ArXSL+YREbfyrZiV3I9BfvH74+k9g4HgJcsGSpDChNRp",
"wSfEAqU2SggJznMTEoQKPIU9fKz1gjatKogPMNl0hv9IqAJh6iPUmyW6QlwS6dHAMsyUbDu8d7Wwh4zB",
"bV8MVVu3ftOX41374hK7l2P70NXq4cwE5gxdkGzsK7/saXr/t9KS2ouqfSHtyNXko0Qe7NSFinmCslMr",
"V7WzIfHCX29fpSWxUdz/C5sSPRW12xJNrbQMbOWMi2s8bZvSDRuaMW7Cg9ggGVpRDjZQRdOC5AcPg8rY",
"rgafO8223gSkUjtdf6t50JVTzqTWz03cp61KoTXnFBc4NSqwj/g+SpA3ZLnZrbuxqk5rIloiOnNTUdbH",
"oN9dcLnHdIpQheDrpv2VOiBfmP5Xa1S0n3SuLGvSRMgvJZRPyiZhCwgj/VqpAun23/6f8wT95V2CQo2P",
"I2QGmqId+/KTN963SaGB9B7R/NF2fDmcuXpAT4edV6GWwXKI8U6X3AHcJMtpD75HR/3UwQIiXUdqnWOq",
"dgEZTF7cMEIpTDFtTGJjudH3J7/Xcq2ZblA9PzpGlzaKb6o/csPsgai10kX16nPU96dcgMtR9LzT29v1",
"rHtkf0+9ecwXtw+2MYjz+FR361NxiHMYVeUtNIPUOsUsdZE5yKk1NF2uB1WX67Yj7A963Ac7rJM3ySTU",
"NPSQAJ7nphQiycu8d/pDJJXrsbWJ5SDBwiYbtTSJ7VyVqa1Qkv3A1mVttojQmUxsnV2PWmvXngpczFBG",
"XI20Qxi1fc6I/yCZWFOQO9gnmFD5RY/zVYL2AWhy84UcCq10o2gBdOdgvioRLsoRvTE3zBIquZjkPmNi",
"ME8+Ht70vI/Kvb7I+850XJ/2EE7GlwboSNSnNX3Pgyetr6GLAnLk0RMSr7VtB5F6WOVyt1HxcoG1R7w+",
"lz8Vwd5l0w8JhTuG3D4OIN9zSuNF7Iypc1no/1KorJmmTTdeuT5++GLu2rN2OHEePe8qahWtlQ35TwDw",
"E9s2585n8lSmzbnN5D1g0O9rIhUXxtkNnhV2dkTMLbRM7mmrO/DKpgZcAVPIbugYXeB0Zr//jUS3JLv1",
"WdG2B77g94hkqC9AljncMHOQ3b7VorKZYfDm5e1Rgm7N6KV3NVATdJthhcOTP169//MNM68iC+1j9Bqw",
"UGPASp9buYGz5rwF+u4HeYz+AFINYDLhwrhhiXnyr3/884aZutKQoQLEQJZjvdMxCDQuJxMQCcoELwac",
"ZiCVS6K+/O3RC5MG/eriGjmY3TDF0RindxMSd8VfGZi2HVatLpwAAVQImJCHfT02VsmqXmygYO0Mm9lW",
"wYOy4BhUFNQ+4aof9uoCuRcPYfafewKyc6L+1dXF0T7MUUVHrfXTVcN2zYV89Aj5ryQj5d/r6ghZok94",
"fVS0dSjHWJ1at44DTFqcHdczQDPMMgpi2TvRDzF8hgaPEhvDK52fYuiLHyY3DLMMAVEzEAiYsYa7ayFU",
"Y+7bcFaXKHyEuKhFEN6wkDnobG/GB+ILQTRnIgzd+vZytyHq74xKjuDB/NUHudiIHsEpmAA0G45lp3v/",
"57d/Rfd4YcdIvcXYVeA8Ehf1tOOv0n243A7uS7sQK45bwwree4L6uS1R6jLIgxPrEELWh0BAdepzpL1A",
"//q//69KU7WJDfpPjmq3CrGtxR9WYzc7RGq09Hgm3074OIRdLIDYtY37DXIdNHc7o/ayJzSRMLQtIR8l",
"6/vcTP30qDwPXS8P4Go3cyGM/OE6DL04Ub3uwo75xfpsFu0Jxhfm8RVAtrcxZ0l0MIWVuI2Va0Lvr2fv",
"3qJa663VGq1Mccqnu7xq78itX1ySN8ICkto+wuRd5BANUTQu9QV/kMPVJwQgqSfWu5G2plXqWgi8/IPv",
"9P/yAxoiVxLIR+I1UsUWUkHeiXiMPX/dqWqaeG5S1q4UFsET26/7YY9eIJ4TZYxp9zMtMFgPQt+2MGqL",
"vhOc7yTUr3EQPdvgIEpMjVJqCi1aibWzvb57bVKpFqa204SLvLcalrfUKfRnTpj3g4zc37T4VvCipEbC",
"Cy1Wj12/x6TLJtxn4nsINRmXOyd03dVjxq9XLWUjHGkeorl5ujdDXpVjS6macudElpiSv7tabqYHKvoN",
"Mj1QdzDva8arepy2cd6PFEC99mh9NJA227VGwGoHHDC22GzMtcr101qzvunohgjL9Fa42MeMFwptDyVg",
"kbZD+so8Dr05uxksfuktKwH7WQG+At2+0Z30cP6310QdQlH/saR0YPJHLDptkdeA5MpL3fcigEyQ6/jZ",
"YNHwylY09Cl4PzrEZNVp6deGzrem/WwF+ENU7KA0yG1y6HGGbKNbpHg0RrUrGuPcbBrNRj1b3ZnaaEa2",
"KO0G79w7N6jTydLZzdbhng8lcw8iqsTlNN8yMxJ/gEvFa/EHzQaqtsXHbgnXuzjxnvJobfSoPRwv2mmR",
"hAPVTDTEajLyBnZOlAfK3fUarit6bafm+0oXOyCKHCNpKG6b92EaJEVYIgTLdZ+w0Q461pG7Fkey5VpX",
"WjptoqwGRKKfbmyxCwkG1B3mhkeeXIxZuZa7pS/z2nL932qrrZGoX1N36rQRM4tBQEDcpF8PqUVYuhTm",
"UV4qpxgEw7vmHDwI5tD7GTBURdiuWMPriTTXVrv8ipNp9AqfMqHGEvvaAj3PTp51oENrJK9X+tzbaKu0",
"AqNmUFGyUWxszn6NoLvTa9NeE6XY4Sd9HcdK/USkHZfit4Wg0xre/hqLDGVAQZkC8IwrJMui4MJUcZ+Z",
"uvCum65E8ECkrVcQ+oGEFH4bU/DyeYQ1aqHnu3HGFwk/10t7whD0No6oFR56Io6oFSwKWK9CJffhBFeV",
"eH0gwqUftI3svUdxzR0jCrrFP/yaIgl8G/2niyMIpHGgKIKiIjVPzxRcJ7nNkoh/+6A12+KdTpDEE1AL",
"NMd0Du7ovXr1u6NjdBYKfOvjvKhLOyuiztX3bYf1ZehF/+VP6iZJtpZa/qXEAjNlGlVFO/Ks9ryqGv7X",
"ml3VOlsZP1IYE1Nrn7bMcmC4r/GWuK4SkrBnJNR39eYBDVEFWzSsbpKj7ry2dHe4OJuQ/FZS6Fbc/0NJ",
"Qfa+grr0eiGHTJMw+zpwnXkkyqZqVrlXdwioOg/KVaNBQfjaCyN3608i1xzSFnHCEwViNa/bn3zPW/Wx",
"Bqi/Uo2svsZtdLInYfNLG//gtKEGlaB+VmI6iDiz19JMB7Z+7Cqo+1HJI2sn/6bkYSjCsfchCcPFVa4z",
"Rp65MVegFGHTpz3rm2s54HEfdneIgut2kUi6OVH/jlA6kPdEpbMEMZiDGPiaq6aizdEOV0Jcpv2AiTRx",
"jn4RRKI6vVDIUP/ZyTP0myoU8hi95fdgih8RZdMf3NLR7ZTyMabHeroRTtUpuunxyeSmd6s1WJzZmEq7",
"pZEfhO7AZVH4a4fkOWQEK6AL/fWTo1NzNdXAYktxmnnQPXbxMZitLzpiTpsYee52bujt6EeYXjZotM0z",
"9Hhy69fJI2cGmzZhRwliQ7WfUEgOx6OndV+dsnFCvmiQ2fsff9QsEQhyv/NTEHk3MAG/G6TlD0Tenbtx",
"T5lS7JdxSEGZyDvkYXAgeVnU59zyaNToaSQj20OSglV9lwv2Zo00627JOaal5LYBL2uTdfaeqRaGt5Uh",
"e4kUv49VsK25mYAdpufQBcu0VFOfui9BSVsGZqS41V1MJAuR6A4KeyPMTF7j4miXshxtatSFa1lpfWiR",
"QjSrfkHUnxEQWKSzxQDfYwFHL1CKRUYYprZt34SLFLI2RWo9zX0dilR9jU/j3GoWQPgi/ScaFOkilnaq",
"/+J7Dqy7FK7cmM4ZgXCwTPPQU5tNeC/p3TtTUdJLBVEkjfYaf5Q8+C6NgH5NZn6L8ye08nuiO1Tl4kDD",
"27XiqfGITaHB6d2j5M+cpXcO5nGsr9+5ffVw7UrO0ipCEzvg7dippAG9vLTSzcHB965UUIPfIZwQeq2j",
"kilCu1aAb+0RudwRv5p5jyaOX5YiNIADKbgSK9fXbw9BFAIkp/PHoYsPdu4Dk0Y7mleQ+VUgz0Ghwl+O",
"WYkpXeyKPq2qbhAa7JBurTGt+WVkok7/471/zGtdY+Upb3VLFYe61M1sqG9SqlRIrCtA2EdHu7n07bSP",
"7X6wqPiKfe0FYQyykYNqvCbiqrtdY6LV1/71edcdQ3z1vnVDk6bBE9G/eqTs6EavUfjQTdXhNP+LH/n1",
"HWA7H0hhT/vjy03lbT+mtqDF2/bH0J55/yYdT3ZNfro2o7c+iv7dkjnMNg9IOg5sh2B0YBnCDNOFJK58",
"IaU+h8NUJo8kUm2T0PGYyVR6K5CWxnSjpx4DFiDOSjXrnf7to8a47cZuP1wK2jvtDXFBhvPvDD24/ay2",
"bXLJ/S7vPOQVmJKzphRO3Xbe3IZNq1mJQ7Gd1yC0fkuq3lZE2jrLhLPEtzmqFY5yvYxW57zYLtXBzcer",
"7ItPcbuH2aIrLtS3qDY+oEZ3zuiCQg2KqreC69SYBC+lRP0MUpLBEKeqNi3USzR9aom7NEsLopc+z2oz",
"hPNt9f26AyZZCjVKgnOsmsp5UVYnChmSjjRcnnBlq6vlOH6Kpl7JxGYsm+9mRCWu+mCCQja+x1SDy2Lg",
"LrhQq++5Sg6fP37+/wEAAP//VvsxiT/wAAA=",
"H4sIAAAAAAAC/+x963IbudXgq6C4WzXUpCn6MpPsyPX9UGSN7cSOtZYm36ZGLgrsPiQxQgM9AJoS43JV",
"fu0DbOUJ8yRbuHY3iSabF1nOVP7YkhqNBs4FOPfzqZfyvOAMmJK9k0+9GeAMhPnx/ApP9f8ZyFSQQhHO",
"eie9DyB5KVJAcxCScIYmXKA3k8E7rNJZL+nJdAY51u+pRQG9k55UgrBp7/Pnz0mvwALnoNwHzkohuVj9",
"xPsC/1oCSs1jNBE8RxgVAuaElxIJkAVnEr6RiMG9GtlhvaRH9Lu/liAWvaTHcK4/Hh62LyvpnTNF1OJN",
"trqSn3568xJxgSQtp6gPx9NjdDPjUp3MyrEg8ubIf7bAalZ9lWS9pCfg15IIyHonSpTQZQWXtIwA3D5r",
"LOFOnuQ4HeSEkZsE3dD79CTFWbZoW49+d8sV/Sh4fkX025+igNVYaYB1wkWOVe+kl2EFA6VfTSLzvskg",
"L7gCli7+DIvV3Z5RAkwNpsBAYAUZuoXFCySgoHgh0R1RM8LQs+9mSIAqBUNqBogLMiUM00AaHgqWmKtF",
"1z4+0F+vrz/H92+BTdWsd/L02f+KLn1iaXwVQ1d4WpEp4QK9Or96gb57+gxxFvgkJzJ3PBJfXMVD2yDq",
"LcmJasMSNQ/rE2QwwSVVvZPvnyR6zyQv897Jsyf6N8Lsb0/D7glTMAVhPnTF19GD4ttTw2e9U4sxcx5c",
"AMsIm54WheBzTPWfUs4UMLM/XBSUpFjDfPiL1ID/VPvg/xQw6Z30/sewOs6G9qkchgnNJ5fobYbZFJBU",
"eArZC4RRDgoPsHsD3WGJUgGGEvtZielAr0hwetT7nPQuBB9TyNcstLAjfrfdgv28kfWeC8EF6n/48Qz9",
"8N33fzDLuCRThulPhYZ1djCo2Vlja3BfQtKP8Jg3WDydAlOnqSJzogyDF4IXIBSxSMbuychSw6ceME1z",
"P/cU53SUYkoNA2DJmaYS/e2UaAbqJb08LUae7kCmmJp99T6ukFbSw3oVI5JFmCbppVwIsC+7IaykFI8p",
"eI5beSUrhR2fyzXjA78kPTDHdtfpGwutzUJYUaqRLPMci0WnmXiptn1FgpRbgEKWaQpyHRjGnFPATA9W",
"/BbYKOWlJcfNcDNkYA+VDmuxQkvHu6c6Vn+2V7SSvRqlJEu0WZEVH/8CqdLfq59Nq3Rt2WuV3OwBMsKq",
"62It1Wfr39lMs26OcTcygPuCCJBbLdOSTBhblhauy8NuCcvqvA73kJbKMnXBKUkXg9QcxPp3rBQINjDI",
"aGfwAi8oxxGZzYhIGjdcQobs7Cgjk0k7yCr8CiJvRynFlrxXSd9JaKsPFFalrG+xsJeZpipDM5CZs4wR",
"84OFtRUT5/wWsugeZWkXtlYm1BJQuK9SzlIQTG4mjxg/ODnRkXIDGg6HYacNcmmQ+Dq2+VBS2Ip1cKk4",
"4/liRGEOtA5f/aS6Bgw/wBxEFI7uLPY3ThOW7/ACjQHhsVQCpwr1CZuBIEqijN+xoy6M1pELNhFXygsY",
"2bWuR7lWuQoQAzsW8TkIQTKQXdbqxNHYdRMjiTgtLKGlmnUT8s8MnTw+CeyLm/XM1AloUVCVGVHnTInF",
"diBKFRddr287eFn6MrdgL+npL2JlVeaFVOCVvKykLZDdRZgChQmtbaWCwGHEphzUjHebwmjKncQeY/YY",
"kaLbaHNMjlKeQUe55wCSTIXZwLhxKrOEeAlK6RlXSO3WauZwj/NCr7k3pXyM6bGm4BFOVexwK61SsJX0",
"MMe0jLNj91Pq1ujxdqb159DZDNLb1c2mnE1IxO7yV0yJ1XP0Uetuvwi9arTWybAm/Ha8F/TWxBzTkYxT",
"87L0NFOq0POk+t+MyFt9AYNQA3Mla3Bkgkw0lgorgUg5G9itRTmYYqlGM8BURYwbVzMiUapB941E/I6h",
"nEuFBKTAFJqDyEiqjtEpQ5Zzv5HIzoSINKLJHRdSIT7Rv0hAOBVcSqSvVwc6O7lMkORI6Y8Rie5mWCG4",
"LygmTKJv72aLbxH2n9ADMpgKnEF2jP5SUopKpgg1nzOToQnRHxUlk8f6gvBwMwsz8HGv6x/5nT6pS3bL",
"9E8fO1yhBl6iZI7Wm/D67xkwuw+7FD0YWXk3LPe/kLmo9AL1+naT7ltlUYXFFDYIjX3CpMIshYG52bJO",
"Yo6duEWMcrPrh6iv/91qZpID11prnAHWnAZJ7++cddEV18i7jrdrbFhfUcXjHY6XNvmmOmTWnSDBONem",
"TTdPijD890+eJI9wbnSgwE00tJ4SwgafRvfnMb8e03Ukt+Ltwlt2d0HbJjxFLvt19P45tkgtQ5KJs+Tt",
"Jj+n/v5bGTK2JxrOiNVhiYI8Lge7P2Ah8CIu/B3E+tHx4nSmgpFBUwYsXXcQsDIfW+hX1sUYYgWkPM+B",
"GetLAOu+lgPBSwXLysvAylI1BWbGaYspwNhaO5vobgntPLji1h2O0LiqY3fbtOMukcpGm4H1BJ1xpuBe",
"RSjemO0mhIIcWdtRxBZ0gdVMauHDjEb60hOlWTEybyKlBQ3/erIF4UviqG1JWiI5SIXzwujoWiBhcK9Q",
"wSlFGnYgVduFH1E4CmlJe9q+wytRAiITdKxHHy9wrr+TkkLDTtZ2FrPMctoFdGbc8NtjCaosjuVsd5jV",
"rvElEwxnXHFGUpRadAenmWPaZJMWsPZiNoR0CakAq2StKDtydUlv2IRIkmKKpHkR6WEIG8s3GVNAykmr",
"qZl9Czis6i8yuuyXWM7GHIvssjLlL7GAM6zIkTc2Ri8bIzUTkKPxYqTVUkO2OMuI3iqmF405V19fsq5a",
"Mc8Y9KUGCmRovEB23qRFn3cf95f+gb/tFN7VT8/1CSHchpem0s/8TOMyvQVlJ/t+kBNWKkD+Ck8aCo++",
"KOu4biLETtT9mgtukQ3U7eb1L8SoZZnXgoX9QGj308Uxn1g159l3sxgiKtWyCa6ghcUXoNWx6BOvx0Uf",
"SoVpBOFmfXwsNU7NPjjNQJ/QmFV64zdSK52QalpIsRUoYoKnVxY3Y66jyrkCMnvdW9xpVdGZHXdFHy+A",
"oT7jbCBAcjqH7Mh5b1fx6T+3sqqlra1wduykCciPbymJnGFx2m2wcwxi50xwSj+4O3aF1mZcKu9VbMLm",
"NFUlpsgPcKYKBGY+wqYox+mMsCgD5yBnziLYnPTSxkjp5+jNhREGtIBqzq+5NSxZsalVqdoUBHQnTxjc",
"DSguFC+ONloJzbTr4OYiZ2Jy1qgQZI4VjG5jETunr84Hl+dnH86vBn8+/9vg+PjYbJdyfXlmkIpF0bZX",
"M3c5piSNT42n8FTPZ8doGjVTX76/uKxJOXGTmru+R/Z+drJw2x3/EyNagsD0tFQzd6WjNy87zWzlg61n",
"d69twtsSiJIVfMRxquKxF0oJMi5V49CoXttFU2szG9bOW8icSTBBd9Y4BjVLXs4ZUdz6Qrcx1PmT/uPh",
"/HNGCzYXQdSyZxbvjJt3WKLGFnc24+VYn9oMsxRGxoy5u7/fn2+tZ0pl0K8FMfZavBcdAzDaDDk7+QO2",
"8006y467fMzuqzka5NxYTjvHtHonG3zT5ibAU6wVTEPf+gPfSBReHLmYq8inN2KtHTvNlby01jJpVRNA",
"lEwgXaRUL8RZ0kZLcvoqHpcUs1Iq4xxHWnYILnKobL3dLp4mktoRcBEPsDxViILhNhbu5wkBmkmUuxUW",
"AiQwddxLtsDdOxAm6m++gsMuiPsinBtH9ZUx2VQYtlI46iuBmTQSYrWnuGzQgoArRwYtMBzVA0nrC/rT",
"5fu/oEsPqo3GssbLkW1nXAM3+ojIkafDuO2V4gWIuqUtB4WNeX8isLX/lEJjZcrnIAz6jGY1ZaQ12CgA",
"uqtNrRWhBRZaaPDsttmSZ2A6WusBWQ0+MrFTYO7PQkBq4kI/7nXgutPVIcZDuYmOsJKPa+lr4yk7Wol3",
"fgjKCc6FCaYy6m1ZoaRDktDOJLP+uI3jaT1CWpwfB8HHrrQZPaLmLqh52UmzfQAKVvgBw0/qKnuNdngv",
"6d1h4e3hgiitHsTN/UZ/jFsTZXeBKkT5BMHPauHHAhMJ2RaxJe7+rmnubolR0grhnVs5qmoh25ujmJzd",
"oOv4tOFA6/wW12BTe8bhPpCTbOuQ9M5eNYHziLR0eUsoRfYp6q/KTPaJOyyOYhKTAGlO3P3daQ/oDrOD",
"azfjZsCuk9TFvtQTCS12wc3N2GIbnLU21tgZ1szxo/l4sqj9bAdPMLGhDnp52YiXxuysbzhq/y64/mE0",
"xumt+03/OHLvfdzHv1hbSESy2zpgOUQqb+t5DMdXqzWxOsWqk1WAwfZ6joqwBJZtV2edxpdY0dpujScu",
"5yZOHjLrj+rzwlqIj3rJ1rFB9XzHjZeDm2ttqOErgYvZXwncrcIQsik0ow3WZSN9cAiUM1LE/B2VHarN",
"Rr6jcSkSvBrxSZEMDa7LJ0+eQ7B1OY3U2rwIS2mZwX9ZmjTmI+cOhmhYIePZFsBx9r4IWJQoWerTwuKe",
"Y/0plOLCLGpGVMRZvCxjchtrbjEYQ/trA4NWx6X3EzQ2uGSvD/g8FN5WrHuHDrsT0Ay3Irk+i30Sm16y",
"i6pS5gPbxRx2k/6bxhYHwy7uQtmGqv/46Fp8dMuQdwCsnGBRuP+Z8Tuq+eY1iVwrXe3UhN1CNopy0cYg",
"DIHZbacoqXahhpGiiF0ir8l0Rsl0plFjUp19OEcnvrLh9Z3D8RVRFNbsuOLDjKdlbmM0RMnGnN8aa9Ac",
"pCLTtgSzzfZmu4AYkt96Vf+lPrJXOapuio0aKrJ2W+CW2FYgcqKliJ1eDtbEiDbwaSJ4foI+KX6CPjlQ",
"yRP0M8M5ZAPDqgk6Pj7++Pnz542+ZOIzy8y9smKsrq0jBu93oARJL0E4CEcum0Wb4pWbd1tC9igtizol",
"CXzXS3pPZ/qfljA9Iw2uu9jwfNqJ/bZImc3xfacpc8I6jdvGwhBSOJaqhOA7ZGGBfGrGhs8uC5ey070V",
"Lt114tGVGRSNX1hYhcBRQcB5hcjYIi5s2Ol2xo2ioARke+BzM4Z1KaeAUMkZovwOBBrzkmWJltgKG7EB",
"c/uezbIeft+LoLQ5Jn4raxWuFGuHbONIDQaCvcStooL1yrNfSywwU4S1hWFvka47WxRczUCSv9tAf794",
"nxW+ZLA0F0gY87E9S34dNHdzdzYoqab7ekg1SGkF8zW1eF3MZK2yxfLttZS6V0uxFIKLyE3xIwGaDUzS",
"Yy32BdnhqP/ds2dH7SF1xs0XP57bNOcl2NkZwvgup4pPjtlAO7G4/k1CSfA69PCYl+pkTLU8VgseKAXZ",
"rHmbz6x1t1xo1UOutWGsc2q/e/MyQSkXIBMkcD7KxwnKiLwdTccJIkWCFOQFNZF/uQkgS5AW20kKMhoB",
"yGVEXryk5dS7cy8Ev8/5vQnDchFOtRiFqC0jHs71uswxGwjAmT5XkPOHdAyzMt+l9+nJL0DpYkLY9q5y",
"ep8maJ4niAuU8fQWhCkZgwmrxzF3d5Y74G3AcVv0VpWx3M1+EELvomanYBhDb16aKAOB01tU+GUQNtW/",
"TAUY+9uGayJ6HfeWlrB225eBE5c2rU+WSFmxOQhMqT14EJk0Fx4Mn7tbAFqc9WelEMBC1ERrCIZUUKyT",
"HM26RzlIiafdnMcTwoic7W9/fggbdoj2FCVzHjGjmAU8yFutZrZcrgqKDtYQPWrtKbk2Mt8xo8eXRU9s",
"loZtcsM5u9HxUXn+4oelq74X2KWziTectjZ2ZM0EbSKqubxHpvLbNvoByUaK70o7yzix0Ekq47M7K2tr",
"24SiblFenRGz3mbejo+N73Wz+MUBsgkG8TifFIuMMEyXXNecwUDxATcx0O6XHDNNPPq/6pn/zTzcaDuP",
"WT6YFkphvxAbZ0nqVO2lt22a88b34/EY9TU1v5A0oB7FG5G3Z94TGs8AGlWfrNDGHMKqih7+R5tVJvKW",
"0zVkjGKqVc4W5WoTLlvwE4fP6kYiy4gBx1WpWyVlY+jv6PL1OlLHS1VINZIAbCtv/YTiYp0yOOM0G2X8",
"ju0bS7htSa4qNMQK8ANn+o5r9Vvvm5JboItRisuOfJ2Xau94Sp6mRuhab/A4QJjORlGwMh26gBuc3nof",
"gDcumO/obVs11SboVKLQx10u+SrG3mYpHqIumK8AVosBcrLRCryX2WSJeqKcfEso3cumtjkQxyStjirL",
"Qcc3OpfQ28Y+Vso9heo1AYM2LZ5kWxr8C8FTyEoREz991GOGjCA8tPEjQx8AMvRxQQXFDH14PvjhaCUU",
"2/vtRiFyqa3gDdPnoLdH5iu+8M0bqYcixeMu3Lo7+9gNeV5qjSLmPl1R3XaeysL1EHNFtCEZAn/iWYfd",
"7aWZwBMXu+CDGIKdVMDE2GRrytyGOGRvLBXr004qe/ZuptOVFJFgPA1G0YoFWs+oS6d8Lkeh5jlmEaPJ",
"Kx6MZaZSn4uT68UrcLpyj8uMQ1QoJRZzp2e81AOMmUnGha7ueScC4vXnlN6Gas03pXjRvZ6JVvqbsdVS",
"znqJL1Fjkq9Zy6XbdvdtAeh4lZnfP9lYZ8CtOwnojlHJlXdKLQGQ8RzTRYs0TQRUQWU7RJCsCpxcc1zU",
"7kqMY44SBligQvBf7KcT9IcM2Yi/zb7Edr+ppDymOb21n5sQhQoQKMOLrZ2CwU1XQSsamCEhLbWEYhJS",
"XGo+YAHitIzlKr53atFwTuAOxAnSw7T0dIvev3l5hv7031f1eFfCBqcXb9C//vFPdIazbHHNJlzcYZEN",
"cGnrnmUwASZhQNggg0LNEsS4zQtz1hstoYlSzY6Or5mpl31izIIkRXadyJRNtjXlqzzSvqmohW5MoPSN",
"ftfXXDfEZN6sqN2wkqnebWRaVxbcJT+4qu2Z4oKvBrWd2RrnA32XA9Kb9dVM3pNbLtGM50DxGL2/PEam",
"PtyEUPAF3779Nmzympldfvst6puy6ThVAyMXHp2gV9x4DEAgqcqxRFgAqqr+3xE1QxwXZKCPvSmw5JrZ",
"xFmJ+v7zZ2/fJGhSaqkE/fRGHll4GTDjHJAsID2+ZtfsjLO5RidnNfnk+dHJNRugc+uF0l/3NdXRTVsF",
"95tj/cpbIpVEpQR088nc0Um9EcXnG7t4172iwFPCrMOr7w4aZKryo++fJCjH9+jZkydHZt6fmMQTQBfv",
"L69spZFCoZullgU3qG+bHxQUL9AdYRm/s2+/K82hgITrzyFRioVYoBt32928QK/Or1zbBIluzq/w9CZB",
"F6dXZ6+Rj99AN74LwQ3qu/4Fvm+B/UwocFPB7Pnz5z+gn67OzPNzF5RknuIsEyClWde4GV2K+s1GGgZR",
"VzNA784ubO2NCU4B9aUSgHMzw+urq4sE8cmEpARTTUCXL/98ZHOIS2Yi0RW6GeZpcXPNOKsIYUwYFguE",
"WaYH89KUI7S8ZOlas6wLEnphShOamjfoTuDimlX0ZPVjZFJqEJauoiGwrOCEKWn5kZIUnCvGMdmFzdbW",
"x7WgjjHlyXDoNO9j50keuqzumh+xZ9nt9OJNTWo56T09fnL8xKi5BTBckN5J7/nxk+Pn1gk8M+fd0BwS",
"A1yrw+8uTWsEIpy9yXonvf9dglg0S/Y327T8HO/3UKuavqY7Rcu7jTLrO0xQD91Y+3JMcq42NwxNTjqM",
"de0vOox0bW06jLS9Oz5/XOqD8ezJk626OCwFEXq1oZP+0ER9RB+p99jZvkSYWULkjl65cvwSEDBl4rg+",
"J5VgFt9CgFmtX0YtktU2okBjmOE5MQUpjJkdT6Wxadsw0zHxZtf7gV+4LVzZO+lZccDMOgx1Slo5SV8L",
"p2FUJyYKWkeFy8OVj29jHm/FWfnk3kX5f3O8EfrGPB5bBILanx80gSJco1DPC1W9nW0YYfiJZJ+HoTuL",
"hnWT4jcgOHTd0jguXIBIk6VemgYWAQ3Jll9Y6jVlaclEw/yRZ4s9yKi+6ZDWatnUcukicGZUNWO8JfLX",
"iP2RmJZ3p2dVjwerG/QlYVMKg1JCgnzelBOpB5JksLkmUNhGnBKbXag+78mIuzZoeukWiQSkXGSQHeJm",
"sLgyITrAFnVYOui2ArQrywSvm2WaMrPx/mtkMDOkm+xVr/O+i/RlmwU8pODVJvZxtsubKwl8/xH69rvY",
"qjYTj3i16UV4cQ/1lUQvzy/Pjg7B3mbm7eW9Js/asvRrxb0zO6QT066IXR1pP8R17MCsvmz5yqu1zL7f",
"FmXbHg+PR9SOIg4krNkOAhlMCHPpLxVBu3KKGyS2NsnKxkBZaD2iWLURlS5Wq5M88vSwn46i1yaOHwC/",
"diaEHY779rQZYDnIsMIJ8mbKPxx1xnns+DJC+r6yuS8P0yQhUzVmRwpynVQflHRsVZsvLMm2Uo7vzrk/",
"5diZjOxKrGnVEdGuhNKojLLhwlsa283KEeoYfGmJ05ehX7V1bFV9/zd3STabSTzibblETgc4Vt2MoDU7",
"qzhq2XIGyEUT8lIO/BNk1DKkBCb0aFd7iPNKDW29YIOaaLKLrxMpfVHgJLi7JMIM4SmgW1gUmIjEtRw2",
"f6/cf64OcSgknBiPRi05thH1xRvpDcfoDFMKwtZLxFQAzhZohudQ76vEzLXDINOnSyM7wgR6WQdH81Sw",
"5YPPfBH8hzjNm5Wdv/CBvlQeOdaR2YzIganQf9x6AG3HAJYFhB2Avu3HEEYM7nwl4X/945+ISFmCpyFP",
"PzXaCUuoqNwRbguJ236ADQr/JGk5/TxMq44c0TCMD87DeDcj6cw13jDNNhLrVrNka2o42+YWvpcEMj01",
"DBFPyRwYUt7XaLzMDHkHsOmmYVuFMakAZ4hP0JQoVJSUxoj0FahmN5GVeyu2BcQZXbjFybA4Iqt12T7c",
"z58//yE03l+5+2wW5pad0T8+pIjSgETsVHY9ODKgCh+AZl+BcmSQ1md2EMUVOLclzmQnqfaSltPe548R",
"ypZVi5Dp2oreAxNgYqyD5g3rTM584V077Tdy5cRuJ0zfneTB8e4/FMH7ZadGJwdSbT3k1nVU+bLEkPl+",
"K8NaMZyoJPwK1EpzlgdE3Mq3YlZyPwb5xe+Pp/cMBoKXLBsoQQoTUqcFnxALlNooISQ4z01IECrwFPbw",
"sdYL2rSqID7AZNMZ/iOhCoSpj1DvTOgKcUmkRwPLMFOy7fDe1cIeMga3fTFUbd36TV+Od+2LS+xeju1D",
"V6uHMxOYM3RBsrGv/Lqn6f3fSktqL6r2hbQjV5OPEnmwUxcq5gnKTq1c1c6GxHN/vX2VlsRGcf8vbEr0",
"VNRuSzS10jKwlTPOr/C0bUo3bGjGuAkPYoNkaEU52EAVTQuSHzwMKmO7GnzmNNt6E5BK7XTNpOZBV045",
"k1o/N3GftiqF1pxTXODUqMA+4vsoQd6Q5Wa37saqOq2JaInozE1FWR+DfnfB5R7TKUIVgq+b9lfqgHxh",
"+l+tUdF+0rmyrEkTIb+WUD4qm4QtIIz0a6UKpNt/+3/OEvTXdwkKNT6OkBloinbsy0/eeN8mhQbSe0Dz",
"R9vx5XDm6gE9HnZehVoGyyHGO11yB3CTLKc9+B4d9VMHC4h0Hal1jqnaBWQweXHNCKUwxbQxiY3lRt89",
"+UHLtWa6QfX86Bhd2Ci+qf7INbMHotZKF9Wrz1Hfn3IBLkfR805vb9ez7oH9PfXmMV/cPtjGIM7jU92t",
"j8UhzmFUlbfQDFLrFLPUReYgp9bQtJQeVC2l246wP+pxH+ywTt4kk1DT0EMCeJ6bUogkL/PeyfeRVK6H",
"1iaWgwQLm2zU0pG1c1WmtkJJ9gNbl7XZIkJnMrF1dj1qrV17KnAxQxlxNdIOYdT2OSP+g2RiTUHuYJ9g",
"QuUXPc5XCdoHoMnNF3IotNKNogXQnYP5qkS4KEf0xtwwS6jkYpL7jInBPPl4eNPzPir3+iLvO9NxfdpD",
"OBlfGqAjUZ/WNBkPnrS+hi4KyJFHj0i81rYdROphlcvdRsXLBdYe8Ppc/lQEexdNPyQU7hhy+ziAfM8p",
"jRexM6bOZaH/S6GyZpo2rW/l+vjh87nrhdrhxHnwvKuoVbRWNuQ/AcCPbNucO5/JY5k25zaT94BBv6+J",
"VFwYZzd4VtjZETG30DK5p63uwEubGnAJTCG7oWN0jtOZ/f43Et2Q7MZnRduG84LfIZKhvgBZ5nDNzEF2",
"81aLymaGwZuXN0cJujGjl97VQE3QTYYVDk/+dPn+L9fMvIostI/Ra8BCjQErfW7lBs6a8xbo6ffyGP0R",
"pBrAZMKFccMS8+Rf//jnNTN1pSFDBYiBLMd6p2MQaFxOJiASlAleDDjNQCqXRH3x+6MXJg361fkVcjC7",
"ZoqjMU5vJyTuir80MG07rFpdOAECqBAwIff7emysklW92EDB2hk2s62Ce2XBMagoqH3CVT/s5TlyLx7C",
"7D/3BGTnRP3Ly/OjfZijio5a66erhu2aC/ngEfJfSUbKv9fVEbJEH/H6qGjrUI6xOrVuHQeYtDg7rmaA",
"ZphlFMSyd6IfYvgMDR4lNoZXOj/F0Bc/TK4ZZhkComYgEDBjDXfXQqjG3LfhrC5R+AhxUYsgvGYhc9DZ",
"3owPxBeCaM5EGLrx7eVuQtTfKZUcwb35qw9ysRE9glMwAWg2HMtO9/4vb/+G7vDCjpF6i7GrwHkkzutp",
"x1+l+3C5HdyXdiFWHLeGFbz3BPVzW6LUZZAHJ9YhhKwPgYDq1OdIe4H+9X//X5WmahMb9J8c1W4VYluL",
"P6zGbnaI1Gjp4Uy+nfBxCLtYALFrG/c75Dpo7nZG7WVPaCJhaFtCPkjW95mZ+vFReRa6Xh7A1W7mQhj5",
"w3UYenGiet2FHfOL9dks2hOMz83jS4Bsb2POkuhgCitxGyvXhN7fTt+9RbXWW6s1WpnilE93edXekVu/",
"uCRvhAUktX2EybvIIRqiaFzqC/4gh6tPCEBST6x3I21Nq9S1EHj5R9/p/+UHNESuJJCPxGukii2kgrwT",
"8Rh7/rpT1TTx3KSsXSosgie2X/fDHr1APCfKGNPuZlpgsB6Evm1h1BZ9JzjfSahf4yB6tsFBlJgapdQU",
"WrQSa2d7fffapFItTG2nCRd5bzUsb6lT6C+cMO8HGbm/afGt4EVJjYQXWqweu36PSZdNuM/E9xBqMi53",
"Tui6q4eMX69aykY40jxEc/N0b4a8LMeWUjXlzoksMSV/d7XcTA9U9DtkeqDuYN7XjFf1OG3jvB8pgHrt",
"0fpgIG22a42A1Q44YGyx2ZhrleuntWZ909ENEZbprXCxjxkvFNoeSsAibYf0pXkcenN2M1j82ltWAvaz",
"AnwFun2jO+nh/G+viTqEov5jSenA5I9YdNoirwHJlZe670UAmSDX8bPBouGVrWjoU/B+dIjJqtPSbw2d",
"b0372Qrwh6jYQWmQ2+TQ4wzZRrdI8WiMalc0xrnZNJqNera6M7XRjGxR2g3euXduUKeTpbObrcM9H0rm",
"HkRUictpvmVmJP4Al4rX4g+aDVRti4/dEq53ceI95tHa6FF7OF600yIJB6qZaIjVZOQN7JwoD5S76zVc",
"V/TaTs33lS52QBQ5RtJQ3DbvwzRIirBECJbrPmGjHXSsI3ctjmTLta60dNpEWQ2IRD/d2GIXEgyoO8wN",
"jzy5GLNyLXdLX+a15fq/1VZbI1G/pu7UaSNmFoOAgLhJvx5Si7B0KcyjvFROMQiGd805eBDMoXczYKiK",
"sF2xhtcTaa6sdvkVJ9PoFT5mQo0l9rUFep49edaBDq2RvF7pc2+jrdIKjJpBRclGsbE5+zWC7k6vTXtN",
"lGKHn/R1HCv1E5F2XIrfFoJOa3j7aywylAEFZQrAM66QLIuCC1PFfWbqwrtuuhLBPZG2XkHoBxJS+G1M",
"wcvnEdaohZ7vxhlfJPxcL+0RQ9DbOKJWeOiROKJWsChgvQqV3IcTXFXi9YEIF37QNrL3HsU1d4wo6Bb/",
"8FuKJPBt9B8vjiCQxoGiCIqK1Dw9U3Cd5DZLIv7tg9Zsi3c6QRJPQC3QHNM5uKP38tUfjo7RaSjwrY/z",
"oi7trIg6l9+1HdYXoRf9lz+pmyTZWmr51xILzJRpVBXtyLPa86pq+F9rdlXrbGX8SGFMTK193DLLgeG+",
"xlviqkpIwp6RUN/Vmwc0RBVs0bC6SY6689rS3eHibELyW0mhW3H/DyUF2fsK6tLrhRwyTcLs68B15pEo",
"m6pZ5V7dIaDqLChXjQYF4WsvjNytP4lcc0hbxAlPFIjVvG5/8j1v1ccaoP5KNbL6GrfRyR6FzS9s/IPT",
"hhpUgvpZiekg4sxeSzMd2Pqhq6DuRyUPrJ38m5KHoQjH3ockDBdXuc4YeerGXIJShE0f96xvruWAx33Y",
"3SEKrttFIunmRP1bQulA3hGVzhLEYA5i4Guumoo2RztcCXGZ9gMm0sQ5+kUQier0QiFD/WdPnqHfVaGQ",
"x+gtvwNT/Igom/7glo5uppSPMT3W041wqk7QdY9PJte9G63B4szGVNotjfwgdAsui8JfOyTPISNYAV3o",
"rz85OjFXUw0sthSnmQfdYRcfg9n6oiPmtImR527nht6OfoTpRYNG2zxDDye3fp08cmqwaRN2lCA2VPsR",
"heRwPHpa99UpGyfkiwaZvf/xR80SgSD3Oz8FkbcDE/C7QVr+QOTtmRv3mCnFfhmHFJSJvEUeBgeSl0V9",
"zi2PRo2eRjKyPSQpWNV3uWBv1kiz7pacY1pKbhvwsjZZZ++ZamF4Wxmyl0jxu1gF25qbCdhheg6ds0xL",
"NfWp+xKUtGVgRopb3cVEshCJbqGwN8LM5DUujnYpy9GmRp27lpXWhxYpRLPqF0T9GQGBRTpbDPAdFnD0",
"AqVYZIRhatv2TbhIIWtTpNbT3NehSNXX+DjOrWYBhC/Sf6JBkS5iaaf6L77nwLpL4dKN6ZwRCAfLNA89",
"tdmE95LenTMVJb1UEEXSaK/xB8mD79II6Ldk5rc4f0Qrvye6Q1UuDjS8XSueGo/YFBqc3j5I/sxpeutg",
"Hsf6+p3bVw/XruQ0rSI0sQPejp1KGtDLSyvdHBx870oFNfgdwgmh1zoqmSK0awX41h6Ryx3xq5n3aOL4",
"ZSlCAziQgiuxcnX19hBEIUByOn8Yuvhg5z4wabSjeQWZXwXyHBQq/OWYlZjSxa7o06rqBqHBDunWGtOa",
"X0Ym6vQ/3vuHvNY1Vh7zVrdUcahL3cyG+ialSoXEugKEfXS0m0vfTvvQ7geLiq/Y114QxiAbOajGayKu",
"uts1Jlp97V+fd90xxFfvWzc0aRo8Ef2rR8qObvQahQ/dVB1O87/6kV/fAbbzgRT2tD++3FTe9mNqC1q8",
"bX8M7Zn3b9LxZNfkpyszeuuj6N8tmcNs84Ck48B2CEYHliHMMF1I4soXUupzOExl8kgi1TYJHQ+ZTKW3",
"AmlpTDd66jFgAeK0VLPeyc8fNcZtN3b74VLQ3klviAsynD819OD2s9q2ySX3u7zzkFdgSs6aUjh123lz",
"GzatZiUOxXZeg9D6Lal6WxFp6ywTzhLf5qhWOMr1Mlqd83y7VAc3H6+yLz7F7R5mi664UN+i2viAGt05",
"owsKNSiq3gquU2MSvJQS9TNISQZDnKratFAv0fSpJe7SLC2IXvo8q80QzrfV9+sOmGQp1CgJzrFqKudF",
"WZ0oZEg60nB5wpWtrpbj+CmaeiUTm7FsvpsRlbjqgwkK2fgeUw0ui4G74EKtvucqOXz++Pn/BwAA///k",
"T3QHrO8AAA==",
}
// GetSwagger returns the content of the embedded swagger specification file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,83 @@
package httpapi
import (
"context"
"encoding/json"
"fmt"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/httpapi/gen"
)
func (s *Server) GetOntology(ctx context.Context, req gen.GetOntologyRequestObject) (gen.GetOntologyResponseObject, error) {
resp := gen.GetOntology200JSONResponse{
EntityTypes: []gen.EntityType{},
RelationshipTypes: []gen.RelationshipType{},
Lifecycles: []gen.LifecycleDef{},
}
q := sqlcgen.New(s.pool)
etRows, err := q.ListEntityTypes(ctx)
if err != nil {
return nil, err
}
for _, et := range etRows {
schemaVersion := int(et.SchemaVersion)
var schema *map[string]any
if len(et.AttributeSchema) > 0 {
var s map[string]any
if json.Unmarshal(et.AttributeSchema, &s) == nil && s != nil {
schema = &s
}
}
resp.EntityTypes = append(resp.EntityTypes, gen.EntityType{
Name: et.Name,
ParentType: et.ParentType,
IsAbstract: et.IsAbstract,
Domain: et.Domain,
Layer: gen.EntityTypeLayer(et.Layer),
Description: et.Description,
LifecycleId: et.LifecycleID,
SchemaVersion: &schemaVersion,
AttributeSchema: schema,
Status: gen.EntityTypeStatus(et.Status),
})
}
rtRows, err := q.ListRelationshipTypes(ctx)
if err != nil {
return nil, err
}
for _, rt := range rtRows {
resp.RelationshipTypes = append(resp.RelationshipTypes, gen.RelationshipType{
Name: rt.Name,
Inverse: rt.Inverse,
SourceType: rt.SourceType,
TargetType: rt.TargetType,
Cardinality: gen.RelationshipTypeCardinality(rt.Cardinality),
Description: rt.Description,
})
}
lcRows, err := q.ListLifecycleDefs(ctx)
if err != nil {
return nil, err
}
for _, lc := range lcRows {
terminal := lc.TerminalStates
var transitions map[string]any
if err := json.Unmarshal(lc.Transitions, &transitions); err != nil {
return nil, fmt.Errorf("lifecycle %s transitions: %w", lc.ID, err)
}
resp.Lifecycles = append(resp.Lifecycles, gen.LifecycleDef{
Id: lc.ID,
States: lc.States,
DefaultState: lc.DefaultState,
TerminalStates: &terminal,
Transitions: transitions,
})
}
return resp, nil
}

View File

@@ -0,0 +1,72 @@
package httpapi
import (
"context"
"encoding/json"
"github.com/dtoro/oikos/internal/httpapi/gen"
)
func (s *Server) QueryAudit(ctx context.Context, req gen.QueryAuditRequestObject) (gen.QueryAuditResponseObject, error) {
limit := clampLimit(req.Params.Limit)
var actorType, actorID, action, entityID, correlationID *string
if req.Params.ActorType != nil {
actorType = req.Params.ActorType
}
if req.Params.ActorId != nil {
actorID = req.Params.ActorId
}
if req.Params.Action != nil {
action = req.Params.Action
}
if req.Params.EntityId != nil {
entityID = req.Params.EntityId
}
if req.Params.CorrelationId != nil {
correlationID = req.Params.CorrelationId
}
rows, err := s.pool.Query(ctx, `
SELECT id, ts, actor_type, actor_id::text, action, entity_id::text,
method, path, status_code, detail, source_ip, correlation_id, session_id::text
FROM audit_log
WHERE ($1::text IS NULL OR actor_type = $1)
AND ($2::text IS NULL OR actor_id::text = $2)
AND ($3::text IS NULL OR action = $3)
AND ($4::text IS NULL OR entity_id::text = $4)
AND ($5::text IS NULL OR correlation_id = $5)
AND ($6::timestamptz IS NULL OR ts >= $6)
AND ($7::timestamptz IS NULL OR ts <= $7)
ORDER BY ts DESC
LIMIT $8`,
actorType, actorID, action, entityID, correlationID, req.Params.From, req.Params.To, limit)
if err != nil {
return nil, err
}
defer rows.Close()
items := []gen.AuditEntry{}
for rows.Next() {
var a gen.AuditEntry
var detailBytes []byte
var actID, entID, method, path, sourceIP, corrID, sessionID *string
var statusCode *int
if err := rows.Scan(&a.Id, &a.Ts, &a.ActorType, &actID, &a.Action, &entID,
&method, &path, &statusCode, &detailBytes, &sourceIP, &corrID, &sessionID); err != nil {
return nil, err
}
a.ActorId = actID
a.EntityId = entID
a.Method = method
a.Path = path
a.StatusCode = statusCode
a.SourceIp = sourceIP
a.CorrelationId = corrID
var detail map[string]any
if json.Unmarshal(detailBytes, &detail) == nil {
a.Detail = &detail
}
items = append(items, a)
}
return gen.QueryAudit200JSONResponse{Items: items}, rows.Err()
}

View File

@@ -0,0 +1,148 @@
package httpapi
import (
"context"
"log/slog"
"net"
"net/http"
"strings"
"sync"
"time"
"golang.org/x/time/rate"
)
// rateLimiter is a per-client (IP) token-bucket limiter registry. Each unique
// client gets its own *rate.Limiter; idle entries are swept periodically so a
// flood of distinct IPs can't grow the map unbounded. A rate of zero (rps==0)
// disables limiting entirely — the returned middleware is a no-op.
//
// Client identity is the source IP. Behind Caddy the real client is in
// X-Forwarded-For: Caddy appends the immediate client as the LAST hop, while
// earlier hops are client-supplied and spoofable. clientIP therefore takes the
// rightmost XFF entry (the proxy's contribution) rather than the first.
type rateLimiter struct {
mu sync.Mutex
limiters map[string]*entry
rps rate.Limit
burst int
}
type entry struct {
limiter *rate.Limiter
lastSeen time.Time
}
// newRateLimiter builds the registry and starts the idle-entry sweeper tied to
// ctx, so the ticker is stopped when the server shuts down.
func newRateLimiter(ctx context.Context, rps, burst int) *rateLimiter {
rl := &rateLimiter{
limiters: make(map[string]*entry),
rps: rate.Limit(rps),
burst: burst,
}
if rps > 0 {
go rl.sweep(ctx)
}
return rl
}
// sweep drops entries untouched since the last sweep so the registry doesn't
// grow without bound under a rotating-IP attack or long-lived process. Exits
// (and stops its ticker) when ctx is cancelled.
func (rl *rateLimiter) sweep(ctx context.Context) {
ticker := time.NewTicker(5 * time.Minute)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
rl.mu.Lock()
for ip, e := range rl.limiters {
if time.Since(e.lastSeen) > 10*time.Minute {
delete(rl.limiters, ip)
}
}
rl.mu.Unlock()
}
}
}
func (rl *rateLimiter) get(ip string) *rate.Limiter {
rl.mu.Lock()
defer rl.mu.Unlock()
if e, ok := rl.limiters[ip]; ok {
e.lastSeen = time.Now()
return e.limiter
}
l := rate.NewLimiter(rl.rps, rl.burst)
rl.limiters[ip] = &entry{limiter: l, lastSeen: time.Now()}
return l
}
// middleware returns a chi-style middleware that enforces the per-IP limit.
// Call with rps==0 to get a pass-through no-op.
func (rl *rateLimiter) middleware(next http.Handler) http.Handler {
if rl.rps <= 0 {
return next
}
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Exempt infra liveness probes so Caddy/compose healthchecks can't be
// throttled into marking the service unhealthy.
if r.URL.Path == "/healthz" {
next.ServeHTTP(w, r)
return
}
if !rl.get(clientIP(r)).Allow() {
w.Header().Set("Retry-After", "1")
writeProblem(w, r, http.StatusTooManyRequests, "rate limit exceeded", "")
return
}
next.ServeHTTP(w, r)
})
}
// clientIP extracts the originating client address. It takes the rightmost
// X-Forwarded-For hop — the one the reverse proxy (Caddy) appends for the
// immediate client — because earlier hops are attacker-controlled and could
// be spoofed to dodge the limit or exhaust another client's bucket. Falls
// back to r.RemoteAddr when no XFF header is present.
//
// Known limitation: this is only trustworthy when the request actually
// traverses Caddy. A client connecting directly to the published :8090 (not
// behind the proxy) can set a single-hop XFF and have it trusted. That only
// evades rate limiting (auth is still required), and rate limiting is off by
// default, so the blast radius is narrow. Fully closing it requires either
// Caddy trusted_proxies (so it overwrites XFF / sets a non-spoofable
// X-Real-Ip) or keying the limiter on the auth token instead of IP.
func clientIP(r *http.Request) string {
if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
if idx := strings.LastIndex(xff, ","); idx >= 0 {
xff = xff[idx+1:]
}
if ip := strings.TrimSpace(xff); ip != "" {
return ip
}
}
host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
return r.RemoteAddr
}
return host
}
// newRateLimiterFromConfig builds the limiter from API config, logging the
// chosen policy once at startup. rps<=0 means "disabled" (returns a no-op
// middleware) so dev/single-user setups aren't throttled by default.
func newRateLimiterFromConfig(ctx context.Context, rps, burst int) *rateLimiter {
if rps <= 0 {
slog.Info("api rate limiting disabled (OIKOS_API_RATE_LIMIT unset)")
return &rateLimiter{rps: 0}
}
if burst <= 0 {
burst = rps * 2
}
slog.Info("api rate limiting enabled", "rps", rps, "burst", burst)
return newRateLimiter(ctx, rps, burst)
}

View File

@@ -0,0 +1,161 @@
package httpapi
import (
"context"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
)
// okHandler is a sentinel upstream that records it was reached.
func okHandler(t *testing.T, reached *bool) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
*reached = true
w.WriteHeader(http.StatusOK)
})
}
func TestRateLimiterDisabledWhenRPSZero(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
rl := newRateLimiterFromConfig(ctx, 0, 0)
if rl.rps != 0 {
t.Fatalf("rps should be 0 when disabled, got %v", rl.rps)
}
// Disabled limiter is a pass-through: requests always reach upstream.
reached := false
h := rl.middleware(okHandler(t, &reached))
for i := 0; i < 50; i++ {
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/v1/entities", nil))
if rec.Code != http.StatusOK {
t.Fatalf("disabled limiter request %d: want 200, got %d", i, rec.Code)
}
}
if !reached {
t.Fatal("disabled limiter never reached upstream")
}
}
func TestRateLimiterThrottlesAfterBurst(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
rl := newRateLimiter(ctx, 1, 3) // 1 rps, burst 3
reached := false
h := rl.middleware(okHandler(t, &reached))
var last429, okCount int
for i := 0; i < 6; i++ {
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/v1/entities", nil))
switch rec.Code {
case http.StatusOK:
okCount++
case http.StatusTooManyRequests:
last429 = i
}
}
if okCount < 1 {
t.Fatal("expected at least one request through within burst")
}
if last429 == 0 {
t.Fatal("expected at least one 429 once burst exhausted")
}
if !reached {
t.Fatal("upstream never reached")
}
}
func TestRateLimiterExemptsHealthz(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
rl := newRateLimiter(ctx, 1, 1) // tiny burst
reached := false
h := rl.middleware(okHandler(t, &reached))
// /healthz must never be throttled, even under a flood.
for i := 0; i < 20; i++ {
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/healthz", nil))
if rec.Code != http.StatusOK {
t.Fatalf("healthz request %d throttled: want 200, got %d", i, rec.Code)
}
}
if !reached {
t.Fatal("healthz never reached upstream")
}
}
func TestClientIPTakesRightmostXFF(t *testing.T) {
// The reverse proxy appends the real client as the LAST hop; earlier hops
// are spoofable and must be ignored.
cases := []struct {
name string
xff string
remote string
wantIP string
}{
{"single xff", "203.0.113.7", "10.0.0.1:4000", "203.0.113.7"},
{"multi hop takes rightmost", "spoofed-attacker, 203.0.113.7", "10.0.0.1:4000", "203.0.113.7"},
{"no xff falls back to remote", "", "198.51.100.2:4000", "198.51.100.2"},
{"blank xff falls back to remote", " ", "198.51.100.2:4000", "198.51.100.2"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.RemoteAddr = tc.remote
if strings.TrimSpace(tc.xff) != "" {
req.Header.Set("X-Forwarded-For", tc.xff)
}
if got := clientIP(req); got != tc.wantIP {
t.Fatalf("clientIP: want %q, got %q", tc.wantIP, got)
}
})
}
}
func TestRateLimiterConcurrency(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
rl := newRateLimiter(ctx, 1000, 100) // generous; ensures no deadlock/panic under contention
h := rl.middleware(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}))
var wg sync.WaitGroup
for i := 0; i < 20; i++ {
wg.Add(1)
go func() {
defer wg.Done()
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/v1/entities", nil))
}()
}
wg.Wait()
}
func TestNewRateLimiterFromConfigBurstDefault(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel() // release the sweep goroutine + ticker
rl := newRateLimiterFromConfig(ctx, 10, 0) // burst unset → defaults to 2x
if rl.burst != 20 {
t.Fatalf("default burst should be 2x rps (20), got %d", rl.burst)
}
}
// TestSweepStopsOnContextCancel verifies the ticker is released when the
// server context is cancelled (no process-lifetime goroutine/ticker leak).
func TestSweepStopsOnContextCancel(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
rl := newRateLimiter(ctx, 1, 1)
cancel()
// Give the sweeper a moment to observe cancellation. It must return
// without blocking; the deferred ticker.Stop() fires on return.
time.Sleep(20 * time.Millisecond)
// Limiter remains usable for the brief test lifetime.
if !rl.get("10.0.0.1").Allow() {
t.Fatal("limiter should still allow within burst after sweep stops")
}
}

View File

@@ -23,11 +23,13 @@ import (
"sync"
"time"
"github.com/dtoro/oikos/internal/actuator"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/httpapi/gen"
mcphandler "github.com/dtoro/oikos/internal/mcp"
"github.com/dtoro/oikos/internal/safego"
"github.com/dtoro/oikos/internal/secrets"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/go-chi/cors"
@@ -52,19 +54,13 @@ type actor struct {
type Server struct {
pool *db.Pool
cfg config.Config
secretsManager secretsBackend
secretsManager secrets.Backend
entityCache *db.EntityCache
sseBroker *sseBroker
sseSubs map[*sseSubscriber]struct{}
sseMu sync.Mutex
}
// secretsBackend is a minimal interface for secrets operations used by the
// HTTP API (enrollment key storage, listing). Compatible with internal/secrets.
type secretsBackend interface {
Set(ctx context.Context, key string, value string) error
List(ctx context.Context) ([]string, error)
}
// NewHandler builds the full HTTP handler: /healthz (unauthenticated,
// SG18) + the OpenAPI surface under /api/v1 behind bearer auth.
//
@@ -74,10 +70,44 @@ type secretsBackend interface {
// and pool.Close() deadlocks.
func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Handler {
s := &Server{
pool: pool,
cfg: cfg,
sseBroker: newSSEBroker(10000),
sseSubs: make(map[*sseSubscriber]struct{}),
pool: pool,
cfg: cfg,
entityCache: db.NewEntityCache(60 * time.Second),
sseBroker: newSSEBroker(10000),
sseSubs: make(map[*sseSubscriber]struct{}),
}
// Wire secrets backend: Infisical primary with SOPS DR fallback.
if cfg.InfisicalSiteURL != "" {
infCfg := secrets.InfisicalConfig{
SiteURL: cfg.InfisicalSiteURL,
ClientID: cfg.InfisicalClientID,
ClientSecret: cfg.InfisicalClientSecret,
ProjectID: cfg.InfisicalProjectID,
SecretPath: "/",
Env: cfg.InfisicalEnv,
}
if infCfg.Env == "" {
infCfg.Env = "dev"
}
primary := secrets.NewInfisicalBackend(infCfg)
var fallback secrets.Backend
if cfg.SecretsDir != "" {
fallback = secrets.NewSOPSBackend(cfg.SecretsDir)
}
s.secretsManager = secrets.NewManager(primary, fallback)
slog.Info("secrets backend wired", "backend", "infisical+sops", "site", cfg.InfisicalSiteURL)
// Start background secret refresh loop (non-blocking).
if mgr, ok := s.secretsManager.(*secrets.Manager); ok {
safego.Go("secrets:refresh", func() { mgr.StartRefreshLoop(ctx) })
}
// Pre-load SSH host keys from Infisical for host verification.
if hosts := actuator.ResolveSSHHosts(ctx, pool); len(hosts) > 0 {
hkSrc := actuator.NewInfisicalHostKeySource(s.secretsManager)
actuator.LoadHostKeys(ctx, hosts, hkSrc)
}
}
// Start background SSE listener, tied to ctx for clean shutdown.
@@ -90,6 +120,11 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
r.Use(middleware.Recoverer)
r.Use(middleware.RequestID)
r.Use(requestLogger)
// Per-IP rate limiting (plan D3). Applied before CORS/auth so a runaway
// agent loop is throttled regardless of credentials. The middleware
// exempts /healthz so liveness probes can't be throttled.
limiter := newRateLimiterFromConfig(ctx, cfg.APIRateLimit, cfg.APIRateBurst)
r.Use(limiter.middleware)
r.Use(cors.Handler(cors.Options{
AllowedOrigins: []string{cfg.CORSAllowedOrigin},
AllowedMethods: []string{"GET", "POST", "PATCH", "DELETE", "OPTIONS"},
@@ -124,6 +159,7 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
// /api/v1/executions/{id}/logs — streamed command output, no schema type
// /api/v1/learning/timeline — derived view, no backing schema type
// /api/v1/learning/trend — derived view, no backing schema type
// /api/v1/openapi.json — API spec (embedded in binary), self-service
//
// See .agents/dev/CONTRIBUTING.md §OpenAPI codegen for the policy.
@@ -261,6 +297,17 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
r.With(combinedAuth(cfg, false)).Get("/api/v1/learning/timeline", s.serveLearningTimeline)
r.With(combinedAuth(cfg, false)).Get("/api/v1/learning/trend", s.serveLearningTrend)
// Serve the OpenAPI spec at a browseable endpoint (agents + humans)
r.Get("/api/v1/openapi.json", func(w http.ResponseWriter, req *http.Request) {
swagger, err := gen.GetSwagger()
if err != nil {
writeProblem(w, req, http.StatusInternalServerError, "failed to load spec", "")
return
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(swagger)
})
// Mount MCP at /mcp (plan R3-10)
nomosAgentID := uuid.Nil
if cfg.NomosAgentID != "" {
@@ -271,7 +318,7 @@ func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Hand
if nomosAgentID == uuid.Nil && cfg.NomosAgentSlug != "" {
_ = pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", cfg.NomosAgentSlug).Scan(&nomosAgentID)
}
r.With(combinedAuth(cfg, false)).Handle("/mcp", mcphandler.NewHandler(pool, cfg.MCPBearerToken, nomosAgentID))
r.With(combinedAuth(cfg, false)).Handle("/mcp", mcphandler.NewHandler(pool, cfg.MCPBearerToken, nomosAgentID, s.secretsManager))
if nomosURL := os.Getenv("NOMOS_PROXY_URL"); nomosURL != "" {
target, _ := url.Parse(nomosURL)

170
internal/httpapi/signals.go Normal file
View File

@@ -0,0 +1,170 @@
package httpapi
import (
"context"
"fmt"
"github.com/dtoro/oikos/internal/domain"
"github.com/dtoro/oikos/internal/httpapi/gen"
"github.com/jackc/pgx/v5"
)
func (s *Server) ListSignals(ctx context.Context, req gen.ListSignalsRequestObject) (gen.ListSignalsResponseObject, error) {
limit := clampLimit(req.Params.Limit)
rows, err := s.pool.Query(ctx, `
SELECT sig.entity_id, se.slug, sig.kind, sig.severity, sig.state,
te.slug, sig.check_id::text, sig.evidence, sig.likely_cause,
sig.occurrence_count, sig.flap_count, sig.hold_down_until,
sig.mute_until, sig.first_seen_at, sig.last_seen_at
FROM signals sig
JOIN entities se ON se.id = sig.entity_id
LEFT JOIN entities te ON te.id = sig.target_entity_id
WHERE ($1::text IS NULL OR sig.state = $1)
AND ($2::text IS NULL OR sig.severity = $2)
AND ($3::text IS NULL OR te.slug = $3)
AND ($4::text IS NULL OR sig.kind = $4)
AND ($5::text IS NULL OR se.slug > $5)
ORDER BY se.slug
LIMIT $6`,
req.Params.State, (*string)(req.Params.Severity), req.Params.EntityId,
req.Params.Kind, req.Params.Cursor, limit+1)
if err != nil {
return nil, err
}
defer rows.Close()
items := []gen.Signal{}
for rows.Next() {
var sig gen.Signal
var flap int
if err := rows.Scan(&sig.Id, &sig.Slug, &sig.Kind, &sig.Severity, &sig.State,
&sig.Target, &sig.CheckId, &sig.Evidence, &sig.LikelyCause,
&sig.OccurrenceCount, &flap, &sig.HoldDownUntil,
&sig.MuteUntil, &sig.FirstSeenAt, &sig.LastSeenAt); err != nil {
return nil, err
}
sig.FlapCount = &flap
items = append(items, sig)
}
if rows.Err() != nil {
return nil, rows.Err()
}
var next *string
if len(items) > limit {
items = items[:limit]
next = &items[len(items)-1].Slug
}
return gen.ListSignals200JSONResponse{Items: items, NextCursor: next}, nil
}
func (s *Server) AckSignal(ctx context.Context, req gen.AckSignalRequestObject) (gen.AckSignalResponseObject, error) {
id, err := s.resolveEntityID(ctx, req.Id)
if err != nil {
return nil, err
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
var sig gen.Signal
err = tx.QueryRow(ctx, `
UPDATE signals SET state = 'acknowledged', updated_at = now()
WHERE entity_id = $1 AND state IN ('raised','failed')
RETURNING entity_id, (SELECT slug FROM entities WHERE id = $1),
kind, severity, 'acknowledged',
(SELECT slug FROM entities WHERE id = target_entity_id),
check_id::text, evidence, likely_cause,
occurrence_count, flap_count, hold_down_until,
mute_until, first_seen_at, last_seen_at`,
id).Scan(&sig.Id, &sig.Slug, &sig.Kind, &sig.Severity, &sig.State,
&sig.Target, &sig.CheckId, &sig.Evidence, &sig.LikelyCause,
&sig.OccurrenceCount, &sig.FlapCount, &sig.HoldDownUntil,
&sig.MuteUntil, &sig.FirstSeenAt, &sig.LastSeenAt)
if err != nil {
if err == pgx.ErrNoRows {
return nil, fmt.Errorf("%w: signal %s not in a state that can be acknowledged", domain.ErrInvalidTransition, req.Id)
}
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return gen.AckSignal200JSONResponse{SignalUpdatedJSONResponse: gen.SignalUpdatedJSONResponse(sig)}, nil
}
func (s *Server) ResolveSignal(ctx context.Context, req gen.ResolveSignalRequestObject) (gen.ResolveSignalResponseObject, error) {
id, err := s.resolveEntityID(ctx, req.Id)
if err != nil {
return nil, err
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
var sig gen.Signal
err = tx.QueryRow(ctx, `
UPDATE signals SET state = 'resolved', updated_at = now()
WHERE entity_id = $1 AND state IN ('raised','acknowledged','acting','failed')
RETURNING entity_id, (SELECT slug FROM entities WHERE id = $1),
kind, severity, 'resolved',
(SELECT slug FROM entities WHERE id = target_entity_id),
check_id::text, evidence, likely_cause,
occurrence_count, flap_count, hold_down_until,
mute_until, first_seen_at, last_seen_at`,
id).Scan(&sig.Id, &sig.Slug, &sig.Kind, &sig.Severity, &sig.State,
&sig.Target, &sig.CheckId, &sig.Evidence, &sig.LikelyCause,
&sig.OccurrenceCount, &sig.FlapCount, &sig.HoldDownUntil,
&sig.MuteUntil, &sig.FirstSeenAt, &sig.LastSeenAt)
if err != nil {
if err == pgx.ErrNoRows {
return nil, fmt.Errorf("%w: signal %s not in a state that can be resolved", domain.ErrInvalidTransition, req.Id)
}
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return gen.ResolveSignal200JSONResponse{SignalUpdatedJSONResponse: gen.SignalUpdatedJSONResponse(sig)}, nil
}
func (s *Server) MuteSignal(ctx context.Context, req gen.MuteSignalRequestObject) (gen.MuteSignalResponseObject, error) {
id, err := s.resolveEntityID(ctx, req.Id)
if err != nil {
return nil, err
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
var sig gen.Signal
err = tx.QueryRow(ctx, `
UPDATE signals SET state = 'muted', mute_until = $2, updated_at = now()
WHERE entity_id = $1 AND state IN ('raised','acknowledged')
RETURNING entity_id, (SELECT slug FROM entities WHERE id = $1),
kind, severity, 'muted',
(SELECT slug FROM entities WHERE id = target_entity_id),
check_id::text, evidence, likely_cause,
occurrence_count, flap_count, hold_down_until,
mute_until, first_seen_at, last_seen_at`,
id, req.Body.MuteUntil).Scan(&sig.Id, &sig.Slug, &sig.Kind, &sig.Severity, &sig.State,
&sig.Target, &sig.CheckId, &sig.Evidence, &sig.LikelyCause,
&sig.OccurrenceCount, &sig.FlapCount, &sig.HoldDownUntil,
&sig.MuteUntil, &sig.FirstSeenAt, &sig.LastSeenAt)
if err != nil {
if err == pgx.ErrNoRows {
return nil, fmt.Errorf("%w: signal %s not in a state that can be muted", domain.ErrInvalidTransition, req.Id)
}
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return gen.MuteSignal200JSONResponse{SignalUpdatedJSONResponse: gen.SignalUpdatedJSONResponse(sig)}, nil
}

View File

@@ -0,0 +1,242 @@
package mcp
import (
"context"
"encoding/json"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/policy"
"github.com/google/uuid"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func AnalysisTools(pool *db.Pool, agentID uuid.UUID, sec secretBackend) []toolReg {
return []toolReg{
{tool: &mcp.Tool{Name: "get_health_summary", Description: "Fleet health per entity — optionally filter by health state(s)",
InputSchema: objSchema(
prop{"health", "string", "Comma-separated health states to include (e.g. 'down,stale'). Omit for all."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
healthStr, _ := args["health"].(string)
query := `
SELECT e.slug, e.type, st.health, st.last_check_at
FROM entity_status st JOIN entities e ON e.id = st.entity_id
WHERE e.type <> 'check' AND e.state <> 'destroyed'`
if healthStr != "" {
query += ` AND st.health = ANY(string_to_array($1, ','))`
return queryRows(ctx, pool, query, healthStr), nil
}
query += ` ORDER BY e.slug`
return queryRows(ctx, pool, query), nil
}},
{tool: &mcp.Tool{Name: "get_audit_trail", Description: "Query the audit log",
InputSchema: objSchema(prop{"entity_id", "string", "Filter by affected entity UUID"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
return queryRows(ctx, pool, `
SELECT id, ts, actor_type, action, entity_id::text, method, path, correlation_id, session_id::text
FROM audit_log
WHERE ($1::text IS NULL OR entity_id::text = $1)
ORDER BY ts DESC LIMIT 50`, nStr(args["entity_id"])), nil
}},
{tool: &mcp.Tool{Name: "query_metrics", Description: "Time-series metrics with bucketed avg/min/max over N hours",
InputSchema: objSchema(prop{"hours", "integer", "Look-back window in hours (default 24)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
hours := int(getFloat(args, "hours", 24))
return annotateJSONResult(queryRows(ctx, pool, `
SELECT time_bucket('1 hour', ts) AS bucket,
entity_id::text, metric,
ROUND(avg(value)::numeric, 2) AS avg,
ROUND(min(value)::numeric, 2) AS min,
ROUND(max(value)::numeric, 2) AS max
FROM metric_samples
WHERE ts > now() - make_interval(hours => $1)
GROUP BY bucket, entity_id, metric
ORDER BY bucket DESC LIMIT 100`, hours), "metric_chart"), nil
}},
// ─── Phase 4: new tools ──────────────────────────────────────────
{tool: &mcp.Tool{Name: "get_signal_history", Description: "Query open and recent signals",
InputSchema: objSchema(
prop{"entity_slug", "string", "Filter by target entity slug"},
prop{"state", "string", "Filter by signal state (raised, resolved)"},
prop{"limit", "integer", "Max rows (default 50)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
limit := int(getFloat(args, "limit", 50))
return queryRows(ctx, pool, `
SELECT s.entity_id::text, s.kind, s.severity, s.state,
s.occurrence_count, e.slug AS target_slug,
s.first_seen_at, s.last_seen_at
FROM signals s
LEFT JOIN entities e ON e.id = s.target_entity_id
WHERE ($1::text IS NULL OR e.slug = $1)
AND ($2::text IS NULL OR s.state = $2)
ORDER BY s.last_seen_at DESC LIMIT $3`,
nStr(args["entity_slug"]), nStr(args["state"]), limit), nil
}},
{tool: &mcp.Tool{Name: "get_patterns", Description: "List learned action patterns",
InputSchema: objSchema(
prop{"status", "string", "Filter by status (hypothesized, validated, active)"},
prop{"entity_type", "string", "Filter by applies_type"},
prop{"action", "string", "Filter by action"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
return queryRows(ctx, pool, `
SELECT p.entity_id::text, p.applies_type, p.action, p.pattern,
p.confidence, p.evidence_count, p.success_count, p.failure_count,
p.status, p.quarantined, p.version, p.last_validated_at
FROM patterns p
WHERE ($1::text IS NULL OR p.status = $1)
AND ($2::text IS NULL OR p.applies_type = $2)
AND ($3::text IS NULL OR p.action = $3)
ORDER BY p.applies_type, p.action`,
nStr(args["status"]), nStr(args["entity_type"]), nStr(args["action"])), nil
}},
{tool: &mcp.Tool{Name: "get_skills", Description: "List available automation skills",
InputSchema: objSchema(
prop{"status", "string", "Filter by status (active, inactive, deprecated)"},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
return queryRows(ctx, pool, `
SELECT s.entity_id::text, s.version, s.name, LEFT(s.procedure::text, 300) AS procedure_preview,
s.applies_type, s.action, s.status, s.success_rate,
s.changed_by::text, s.change_reason, s.last_used_at
FROM skills s
WHERE ($1::text IS NULL OR s.status = $1)
ORDER BY s.name, s.version DESC`,
nStr(args["status"])), nil
}},
{tool: &mcp.Tool{Name: "get_trend", Description: "Metric slope, variance, and averages for an entity over N days",
InputSchema: objSchema(
prop{"entity_id", "string", "Entity slug"},
prop{"days", "integer", "Look-back window in days (default 7)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["entity_id"].(string)
days := int(getFloat(args, "days", 7))
return queryRows(ctx, pool, `
SELECT metric,
ROUND(avg(value)::numeric, 2) AS avg_val,
ROUND(stddev(value)::numeric, 2) AS std_val,
count(*) AS sample_count,
ROUND(regr_slope(value, EXTRACT(EPOCH FROM ts)::numeric)::numeric, 4) AS slope
FROM metric_samples ms
JOIN entities e ON e.id = ms.entity_id
WHERE e.slug = $1 AND ts >= now() - make_interval(days => $2)
GROUP BY metric
ORDER BY metric`, slug, days), nil
}},
{tool: &mcp.Tool{Name: "get_event_timeline", Description: "Recent events filtered by severity and entity slug",
InputSchema: objSchema(
prop{"severity", "string", "Filter by severity (info, warn, error)"},
prop{"entity_slug", "string", "Filter by entity slug"},
prop{"limit", "integer", "Max rows (default 50)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
limit := int(getFloat(args, "limit", 50))
return queryRows(ctx, pool, `
SELECT ev.ts, ev.type, ev.severity, ev.source, e.slug AS entity_slug,
ev.data::text AS message, ev.correlation_id
FROM events ev
LEFT JOIN entities e ON e.id = ev.entity_id
WHERE ($1::text IS NULL OR ev.severity = $1)
AND ($2::text IS NULL OR e.slug = $2)
ORDER BY ev.ts DESC LIMIT $3`,
nStr(args["severity"]), nStr(args["entity_slug"]), limit), nil
}},
{tool: &mcp.Tool{Name: "get_agent_activity", Description: "Agent self-inspection: query agent activity log",
InputSchema: objSchema(
prop{"limit", "integer", "Max rows (default 50)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
limit := int(getFloat(args, "limit", 50))
return annotateJSONResult(queryRows(ctx, pool, `
SELECT id, ts, agent_id::text, session_id, activity_type, tool_name,
entity_id::text, left(input_summary, 200) AS input_summary,
left(output_summary, 200) AS output_summary,
duration_ms, token_count, success, correlation_id
FROM agent_activity
WHERE agent_id = $1
ORDER BY ts DESC LIMIT $2`, agentID, limit), "change_log"), nil
}},
// classify_command is the command-scoped preflight from
// plans/2026-07-20-session-review-ten-sessions.md P0.2. The
// existing `preflight` tool is entity/action-scoped — useless when
// the agent is composing a `run` command and needs to know whether
// the classifier will accept it before submitting. Without this,
// the agent has to retry with cosmetic variations until it finds
// one that passes (see sessions a51e2086, 8acea2e3 — three
// duplicate rclone sessions, all bouncing off the classifier).
// Call this BEFORE `run` whenever the classification is uncertain.
{tool: &mcp.Tool{Name: "classify_command", Description: "Pre-flight risk classification for a shell command BEFORE calling run. Returns the risk class (read_only / reversible_low / config_mutation / destructive) that `run` would assign. Use this when you're unsure whether a command will auto-execute or need approval — e.g. `pct exec`, `curl`, compound commands, or anything that might be mistaken for mutation. If this returns read_only, the same command will auto-execute via run with no approval; if it returns config_mutation, expect to need operator approval (or pre-frame the command so it classifies lower). Declared risk can only escalate, never de-escalate.",
InputSchema: objSchema(
prop{"command", "string", "The exact shell command you intend to pass to run."},
prop{"declared_risk", "string", "Optional self-assessment you would pass to run (read_only, reversible_low, config_mutation, destructive). Mirrors run's declared_risk parameter."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
command, _ := args["command"].(string)
declaredRisk, _ := args["declared_risk"].(string)
if command == "" {
return textResult("error: command is required"), nil
}
risk := policy.ClassifyCommand(command, declaredRisk)
note := ""
switch risk {
case policy.RiskReadOnly:
note = "auto-acts on `run` (no approval needed)."
case policy.RiskReversibleLow:
note = "auto-acts on `run` (no approval needed)."
case policy.RiskConfigMutation:
note = "requires operator approval on `run` (or loose assent window active)."
case policy.RiskDestructive:
note = "requires explicit operator confirmation on `run` (typed \"I confirm\" phrase)."
}
out, _ := json.Marshal(map[string]any{
"command": command,
"declared_risk": declaredRisk,
"risk_class": risk,
"note": note,
})
return textResult(string(out)), nil
}},
{tool: &mcp.Tool{Name: "get_ontology", Description: "Entity types, relationship types, and lifecycle definitions. Use this to understand the schema — what entity types exist, what relationships connect them, and what lifecycle states each type supports.",
InputSchema: objSchema(),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
etResult := queryRowsJSONSingle(ctx, pool, `
SELECT name, parent_type, is_abstract, domain, layer,
description, lifecycle_id, schema_version, status
FROM entity_types ORDER BY name`)
rtResult := queryRowsJSONSingle(ctx, pool, `
SELECT name, inverse, source_type, target_type,
cardinality, description
FROM relationship_types ORDER BY name`)
lcResult := queryRowsJSONSingle(ctx, pool, `
SELECT id, name, states, transitions::text
FROM lifecycles ORDER BY name`)
result := map[string]any{
"entity_types": etResult,
"relationship_types": rtResult,
"lifecycles": lcResult,
}
b, _ := json.MarshalIndent(result, "", " ")
return textResult(string(b)), nil
}},
{tool: &mcp.Tool{Name: "http_get", Description: "Fetch a public web page or raw file (e.g. a GitHub README/raw URL) and return sanitized text. Use this to research how to deploy a service before provisioning. HTTP/HTTPS only; body is truncated to ~16KB.",
InputSchema: objSchema(
prop{"url", "string", "Absolute http(s) URL to fetch"},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
rawURL, _ := args["url"].(string)
return httpGet(ctx, rawURL), nil
}},
}
}

View File

@@ -97,7 +97,7 @@ func newTestPool(t *testing.T) *db.Pool {
func callTool(t *testing.T, pool *db.Pool, name string, args map[string]any) string {
t.Helper()
var handler toolHandler
for _, r := range allTools(pool, uuid.Nil) {
for _, r := range allTools(pool, uuid.Nil, nil) {
if r.tool.Name == name {
handler = r.handler
break

View File

@@ -0,0 +1,501 @@
package mcp
import (
"context"
"encoding/json"
"fmt"
"strings"
"github.com/dtoro/oikos/internal/audit"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/google/uuid"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func EntityTools(pool *db.Pool, agentID uuid.UUID, sec secretBackend) []toolReg {
return []toolReg{
{tool: &mcp.Tool{Name: "ping", Description: "Lightweight connectivity check. Returns server identity, no DB hit.",
InputSchema: objSchema(),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
return textResult(`{"ok":true,"server":"oikos","version":"dev"}`), nil
}},
{tool: &mcp.Tool{Name: "get_entity", Description: "Get an entity by slug or UUID",
InputSchema: objSchema(prop{"slug_or_id", "string", "Entity slug (e.g. host:hubris) or UUID"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
idOrSlug, _ := args["slug_or_id"].(string)
return queryEntity(ctx, pool, idOrSlug), nil
}},
{tool: &mcp.Tool{Name: "list_entities", Description: "List entities filtered by type, state, or search",
InputSchema: objSchema(
prop{"type", "string", "Filter by entity type"},
prop{"state", "string", "Filter by lifecycle state"},
prop{"q", "string", "Substring match on slug or name"},
prop{"limit", "integer", "Max rows (default 50)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
limit := int(getFloat(args, "limit", 50))
return annotateJSONResult(queryRows(ctx, pool, `
SELECT e.slug, e.type, e.name, e.state, e.version, e.created_at, e.updated_at
FROM entities e
WHERE ($1::text IS NULL OR e.type = $1)
AND ($2::text IS NULL OR e.state = $2)
AND ($3::text IS NULL OR e.slug ILIKE '%'||$3||'%' OR e.name ILIKE '%'||$3||'%')
ORDER BY e.slug LIMIT $4`,
nStr(args["type"]), nStr(args["state"]), nStr(args["q"]), limit), "entity_table"), nil
}},
{tool: &mcp.Tool{Name: "get_relations", Description: "List inbound/outbound edges for one entity, optionally filtered by relationship type",
InputSchema: objSchema(
prop{"entity_id", "string", "Entity slug"},
prop{"types", "string", "Comma-separated relationship types to include (e.g. 'hosts,provides,depends-on'). Omit for all."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["entity_id"].(string)
typesStr, _ := args["types"].(string)
if slug == "" {
return textResult("entity_id is required"), nil
}
query := `
SELECT r.type, src.slug AS source, tgt.slug AS target
FROM relationships r
JOIN entities src ON src.id = r.source_id
JOIN entities tgt ON tgt.id = r.target_id
WHERE (src.slug = $1 OR tgt.slug = $1) AND r.valid_to IS NULL`
if typesStr != "" {
query += ` AND r.type = ANY(string_to_array($2, ','))`
return queryRows(ctx, pool, query, slug, typesStr), nil
}
query += ` ORDER BY r.type`
return queryRows(ctx, pool, query, slug), nil
}},
{tool: &mcp.Tool{Name: "get_blast_radius", Description: "Find entities affected if this entity goes down",
InputSchema: objSchema(
prop{"entity_id", "string", "Entity slug"},
prop{"depth", "integer", "Traversal depth (default 3)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["entity_id"].(string)
depth := int(getFloat(args, "depth", 3))
return queryRows(ctx, pool,
"SELECT e.slug, CAST(b.depth AS int) FROM blast_radius((SELECT id FROM entities WHERE slug = $1), $2) b JOIN entities e ON e.id = b.entity_id",
slug, depth), nil
}},
{tool: &mcp.Tool{Name: "create_entity", Description: "Create a new entity in the knowledge graph. Use it when a task needs an entity that does not exist yet: a service, a host/LXC/VM, an ingress, a cert, etc. After inserting, it derives default checks from the entity type's monitoring spec, so creating a checkable entity wires its monitoring in one call. Does NOT require approval. If the slug already exists it returns 'already exists' — then use update_entity_attributes to change it. FOOTGUN: creating a type=check entity creates a bare entity row but does NOT wire a check_def — the scheduler will never probe it. To add monitoring, set `monitoring: [\"http\"]` + `url` on the target via update_entity_attributes.",
InputSchema: objSchema(
prop{"type", "string", "Entity type — must already exist in the ontology and not be abstract (e.g. service, lxc, host, vm, check, ingress, cert, dns)."},
prop{"name", "string", "Human-readable name (e.g. 'HAOS http service check')."},
prop{"slug", "string", "Entity slug (e.g. check:http:service:haos:0, ingress:home.hubris.network). If omitted, defaults to <type>:<name>."},
prop{"attributes", "string", "JSON object string of attributes, e.g. {\"check_type\":\"http:service\",\"target\":\"service:haos\",\"port\":\"8123\"}. Optional."},
prop{"state", "string", "Lifecycle state. Optional; defaults to the type's lifecycle default_state."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
entityType, _ := args["type"].(string)
name, _ := args["name"].(string)
slug, _ := args["slug"].(string)
if slug == "" && entityType != "" && name != "" {
slug = entityType + ":" + name
}
if entityType == "" || name == "" || slug == "" {
return textResult("error: type and name are required (slug defaults to <type>:<name>)"), nil
}
attrsStr, _ := args["attributes"].(string)
attrs := map[string]any{}
if attrsStr != "" {
if err := json.Unmarshal([]byte(attrsStr), &attrs); err != nil {
return textResult(fmt.Sprintf("error: attributes is not valid JSON: %v", err)), nil
}
}
attrsJSON, _ := json.Marshal(attrs)
stateStr, _ := args["state"].(string)
tx, err := pool.Begin(ctx)
if err != nil {
return textResult(fmt.Sprintf("error: begin tx: %v", err)), nil
}
defer tx.Rollback(ctx)
// Validate the type exists and is concrete (mirror httpapi.CreateEntity).
var isAbstract bool
if err := tx.QueryRow(ctx, `SELECT is_abstract FROM entity_types WHERE name = $1`, entityType).Scan(&isAbstract); err != nil {
return textResult(fmt.Sprintf("error: entity type %q not found in ontology", entityType)), nil
}
if isAbstract {
return textResult(fmt.Sprintf("error: type %q is abstract — pick a concrete subtype", entityType)), nil
}
// Default state from the type's lifecycle unless the caller
// supplied one. Caller-supplied states are validated against
// the lifecycle's declared states — a create_entity bypass of
// lifecycle guardrails would let an agent create in a terminal
// state (destroyed) without satisfying the preconditions that
// set_entity_state enforces for the same transition.
var state *string
var lsDefault, statesRaw string
if err := tx.QueryRow(ctx, `SELECT coalesce(ld.default_state,''), coalesce(ld.states::text,'')
FROM lifecycle_defs ld
JOIN entity_types et ON et.lifecycle_id = ld.id
WHERE et.name = $1`, entityType).Scan(&lsDefault, &statesRaw); err == nil {
var validStates []string
json.Unmarshal([]byte(statesRaw), &validStates)
if stateStr != "" {
found := false
for _, s := range validStates {
if s == stateStr {
found = true
break
}
}
if !found && len(validStates) > 0 {
return textResult(fmt.Sprintf("error: state %q not declared in %s lifecycle (states: %s). Use the default (%s) or omit state.", stateStr, entityType, strings.Join(validStates, ","), lsDefault)), nil
}
state = &stateStr
} else if lsDefault != "" {
state = &lsDefault
}
}
id, err := uuid.NewV7()
if err != nil {
return textResult(fmt.Sprintf("error: gen id: %v", err)), nil
}
var createdName string
if err := tx.QueryRow(ctx, `
INSERT INTO entities (id, slug, type, name, state, attributes)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING name`,
id, slug, entityType, name, state, attrsJSON).Scan(&createdName); err != nil {
if strings.Contains(err.Error(), "unique") || strings.Contains(err.Error(), "duplicate") {
return textResult(fmt.Sprintf("Entity %q already exists — use update_entity_attributes to change it.", slug)), nil
}
return textResult(fmt.Sprintf("error creating %s: %v", slug, err)), nil
}
res, derr := db.EnsureEntityChecks(ctx, tx, id, slug, entityType, createdName, attrsJSON)
if derr != nil {
return textResult(fmt.Sprintf("error deriving checks for %s: %v", slug, derr)), nil
}
if cerr := tx.Commit(ctx); cerr != nil {
return textResult(fmt.Sprintf("error committing %s: %v", slug, cerr)), nil
}
return textResult(formatCreateResult(slug, entityType, res)), nil
}},
{tool: &mcp.Tool{Name: "update_entity_attributes", Description: "Merge new/changed attributes into an entity — the OTHER half of avoiding knowledge-base drift (upsert_knowledge records what you learned; this keeps the entity's own facts current). Use it when you discover something concrete about an entity's actual state that the graph doesn't reflect yet: a new IP, a version number, a config value, a discovered port — anything a FUTURE task would otherwise have to rediscover from scratch. Does NOT require approval (this updates the knowledge graph, not the live infrastructure). Merges shallowly — existing keys not mentioned are kept; keys you pass overwrite.",
InputSchema: objSchema(
prop{"slug", "string", "Entity slug to update (e.g. lxc:typetype, host:strong)."},
prop{"attributes", "string", "JSON object string of attributes to merge in, e.g. {\"lan_ip\":\"192.168.8.50\",\"os\":\"debian-12\"}."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["slug"].(string)
attrsStr, _ := args["attributes"].(string)
if slug == "" || attrsStr == "" {
return textResult("error: slug and attributes are required"), nil
}
var attrs map[string]any
if err := json.Unmarshal([]byte(attrsStr), &attrs); err != nil {
return textResult(fmt.Sprintf("error: attributes is not valid JSON: %v", err)), nil
}
// Strip scheduler-owned keys: health is computed by the scheduler
// from probe results (spotted live 2026-08-05: an agent set
// health:"healthy" on lxc:nfs-export, which derived 4 spurious checks).
// Agents can observe health via get_health_summary / list_checks.
var blocked []string
for _, key := range []string{"health", "last_check_at", "last_check"} {
if _, ok := attrs[key]; ok {
delete(attrs, key)
blocked = append(blocked, key)
}
}
if len(blocked) > 0 {
// Re-marshal the filtered attrs
filtered, _ := json.Marshal(attrs)
attrsStr = string(filtered)
if len(attrs) == 0 {
return textResult(fmt.Sprintf("Updated %s: no allowed attributes provided. The following keys are scheduler-owned and ignored: %s. Use get_health_summary or list_checks to observe entity health.", slug, strings.Join(blocked, ", "))), nil
}
}
attrsJSON, _ := json.Marshal(attrs)
// Run the merge + check regeneration in one transaction so the
// derived checks always see the post-merge attributes. Mirrors
// httpapi.PatchEntity; without this, setting an entity's
// `monitoring` attribute via MCP silently produced no checks.
tx, err := pool.Begin(ctx)
if err != nil {
return textResult(fmt.Sprintf("error: begin tx: %v", err)), nil
}
defer tx.Rollback(ctx)
ra, err := sqlcgen.New(tx).MergeEntityAttributes(ctx, sqlcgen.MergeEntityAttributesParams{Slug: slug, Patch: attrsJSON})
if err != nil {
return textResult(fmt.Sprintf("error updating %s: %v", slug, err)), nil
}
if ra == 0 {
return textResult(fmt.Sprintf("error: entity %q not found", slug)), nil
}
merged, err := sqlcgen.New(tx).GetEntityBySlug(ctx, slug)
if err != nil {
return textResult(fmt.Sprintf("error reloading %s: %v", slug, err)), nil
}
id := merged.ID
entityType := merged.Type
name := merged.Name
mergedAttrs := merged.Attributes
res, cerr := db.EnsureEntityChecks(ctx, tx, id, slug, entityType, name, mergedAttrs)
if cerr != nil {
return textResult(fmt.Sprintf("error deriving checks for %s: %v", slug, cerr)), nil
}
if cerr := tx.Commit(ctx); cerr != nil {
return textResult(fmt.Sprintf("error committing %s: %v", slug, cerr)), nil
}
return textResult(fmt.Sprintf("Updated %s with %d attribute(s).%s", slug, len(attrs), formatCheckResult(res))), nil
}},
{tool: &mcp.Tool{Name: "set_entity_state", Description: "Transition an entity to a new lifecycle state — the entity-graph \"delete\" surface, since this system never hard-deletes entities. Use retire/deprecate to take an entity out of service, destroy for terminal removal, or active to revive. The target state must be a declared transition in the entity type's lifecycle (e.g. active→deprecated, deprecated→active); preconditions (no inbound edges, backups verified, etc.) are enforced — an error tells you what's blocking. Does NOT require approval (knowledge-graph mutation, not live infrastructure).",
InputSchema: objSchema(
prop{"slug", "string", "Entity slug."},
prop{"state", "string", "Target lifecycle state."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["slug"].(string)
targetState, _ := args["state"].(string)
if slug == "" || targetState == "" {
return textResult("error: slug and state are required"), nil
}
tx, err := pool.Begin(ctx)
if err != nil {
return textResult(fmt.Sprintf("error: begin tx: %v", err)), nil
}
defer tx.Rollback(ctx)
ent, err := sqlcgen.New(tx).GetEntityBySlug(ctx, slug)
if err != nil {
return textResult(fmt.Sprintf("error: entity %q not found", slug)), nil
}
id := ent.ID
entityType := ent.Type
currentState := ""
if ent.State != nil {
currentState = *ent.State
}
if err := db.ValidateTransition(ctx, tx, id, entityType, currentState, targetState); err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
ra, err := sqlcgen.New(tx).SetEntityState(ctx, sqlcgen.SetEntityStateParams{ID: id, State: &targetState})
if err != nil {
return textResult(fmt.Sprintf("error updating %s: %v", slug, err)), nil
}
if ra == 0 {
return textResult(fmt.Sprintf("error: entity %q not found", slug)), nil
}
if err := tx.Commit(ctx); err != nil {
return textResult(fmt.Sprintf("error: commit: %v", err)), nil
}
return textResult(fmt.Sprintf("Transitioned %s: %s → %s.", slug, currentState, targetState)), nil
}},
{tool: &mcp.Tool{Name: "create_relationship", Description: "Record a relationship you discovered between two entities — the graph-structure half of keeping the knowledge base current (alongside update_entity_attributes and upsert_knowledge). Use it when you learn that one entity depends on, hosts, routes to, etc. another, and that edge isn't in the graph yet. type must be an existing relationship type (see get_relations output on similar entities for examples: hosts, provides, depends-on, configured-by, about, documents, ...). Idempotent — re-calling the same source/target/type is a no-op. Does NOT require approval.",
InputSchema: objSchema(
prop{"source", "string", "Source entity slug."},
prop{"target", "string", "Target entity slug."},
prop{"type", "string", "Relationship type name (must already exist in the ontology)."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
source, _ := args["source"].(string)
target, _ := args["target"].(string)
relType, _ := args["type"].(string)
if source == "" || target == "" || relType == "" {
return textResult("error: source, target, and type are required"), nil
}
srcEnt, err := sqlcgen.New(pool).GetEntityBySlug(ctx, source)
if err != nil {
return textResult(fmt.Sprintf("error: source entity %q not found", source)), nil
}
tgtEnt, err := sqlcgen.New(pool).GetEntityBySlug(ctx, target)
if err != nil {
return textResult(fmt.Sprintf("error: target entity %q not found", target)), nil
}
_, err = sqlcgen.New(pool).InsertRelationshipIfAbsent(ctx, sqlcgen.InsertRelationshipIfAbsentParams{
SourceID: srcEnt.ID, TargetID: tgtEnt.ID, Type: relType, Attributes: []byte(`{"by":"nomos"}`),
})
if err != nil {
return textResult(fmt.Sprintf("error creating relationship: %v (is %q a valid relationship type?)", err, relType)), nil
}
return textResult(fmt.Sprintf("Recorded: %s —%s→ %s", source, relType, target)), nil
}},
{tool: &mcp.Tool{Name: "end_relationship", Description: "End an existing relationship (soft-delete by setting valid_to) — the graph-structure \"delete\" surface. Use it when you discover an edge is no longer true (a service moved hosts, a route was removed, a dependency dissolved). The edge is kept for history; only the currently-active edge is ended. Idempotent — ending an already-ended or non-existent edge is a no-op. Does NOT require approval.",
InputSchema: objSchema(
prop{"source", "string", "Source entity slug."},
prop{"target", "string", "Target entity slug."},
prop{"type", "string", "Relationship type name."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
source, _ := args["source"].(string)
target, _ := args["target"].(string)
relType, _ := args["type"].(string)
if source == "" || target == "" || relType == "" {
return textResult("error: source, target, and type are required"), nil
}
srcEnt, err := sqlcgen.New(pool).GetEntityBySlug(ctx, source)
if err != nil {
return textResult(fmt.Sprintf("error: source entity %q not found", source)), nil
}
tgtEnt, err := sqlcgen.New(pool).GetEntityBySlug(ctx, target)
if err != nil {
return textResult(fmt.Sprintf("error: target entity %q not found", target)), nil
}
ra, err := sqlcgen.New(pool).EndCurrentRelationship(ctx, sqlcgen.EndCurrentRelationshipParams{
SourceID: srcEnt.ID, TargetID: tgtEnt.ID, Type: relType,
})
if err != nil {
return textResult(fmt.Sprintf("error ending relationship: %v", err)), nil
}
if ra == 0 {
return textResult(fmt.Sprintf("No active relationship %s —%s→ %s found.", source, relType, target)), nil
}
return textResult(fmt.Sprintf("Ended: %s —%s→ %s.", source, relType, target)), nil
}},
// ─── Client introspection tools (plan: client-lifecycle Phase 3) ──
{tool: &mcp.Tool{Name: "whoami", Description: "Get the current entity record, peers, and health for a host",
InputSchema: objSchema(prop{"hostname", "string", "Hostname of the calling machine"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
hostname, _ := args["hostname"].(string)
if hostname == "" {
return textResult("error: hostname required"), nil
}
slug := "ws:" + hostname
return annotateJSONResult(queryRows(ctx, pool, `
SELECT e.slug, e.type, e.name, e.state,
COALESCE(st.health, 'unknown') AS health,
COALESCE(st.last_check_at::text, '') AS last_check,
e.attributes->>'mesh_ip' AS mesh_ip,
e.attributes->>'age_pubkey' AS age_pubkey,
e.enrolled_at
FROM entities e
LEFT JOIN entity_status st ON st.entity_id = e.id
WHERE e.slug = $1
ORDER BY e.slug`, slug), "entity_card"), nil
}},
{tool: &mcp.Tool{Name: "explain", Description: "Compact context card for a service: type, state, health, relations, risk",
InputSchema: objSchema(prop{"service_slug", "string", "Service entity slug (e.g. service:jellyfin, lxc:caddy)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["service_slug"].(string)
if slug == "" {
return textResult("error: service_slug required"), nil
}
return annotateJSONResult(queryRows(ctx, pool, `
SELECT e.slug, e.type, e.name, e.state,
COALESCE(st.health, 'unknown') AS health,
COALESCE(st.last_check_at::text, '') AS last_check,
e.version, e.updated_at,
COALESCE(e.attributes::text, '{}') AS attrs
FROM entities e
LEFT JOIN entity_status st ON st.entity_id = e.id
WHERE e.slug = $1`, slug), "entity_card"), nil
}},
{tool: &mcp.Tool{Name: "preflight", Description: "Risk classification for an action on a service",
InputSchema: objSchema(
prop{"service_slug", "string", "Entity slug"},
prop{"action", "string", "Planned action (restart, deploy, destroy, etc.)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["service_slug"].(string)
action, _ := args["action"].(string)
if slug == "" || action == "" {
return textResult("error: service_slug and action required"), nil
}
return queryRows(ctx, pool, `
SELECT e.slug, e.type, e.state,
CASE
WHEN $2 IN ('restart', 'logs', 'status') THEN 'reversible_low'
WHEN $2 IN ('deploy', 'upgrade', 'configure') THEN 'config_mutation'
WHEN $2 IN ('destroy', 'wipe', 'revoke') THEN 'destructive'
ELSE 'read_only'
END AS risk_class,
CASE
WHEN $2 IN ('read_only','reversible_low') THEN 'auto-act'
WHEN $2 = 'config_mutation' THEN 'operator-approval'
ELSE 'operator-approval+confirmation'
END AS approval
FROM entities e WHERE e.slug = $1`, slug, action), nil
}},
{tool: &mcp.Tool{Name: "get_change_history", Description: "Last N change-ledger entries for an entity",
InputSchema: objSchema(
prop{"entity_slug", "string", "Entity slug"},
prop{"limit", "integer", "Max entries (default 20)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["entity_slug"].(string)
limit := int(getFloat(args, "limit", 20))
return annotateJSONResult(queryRows(ctx, pool, `
SELECT al.ts AS timestamp, al.actor_type, al.actor_id::text AS actor_label,
al.action, al.method, al.path,
al.detail::text AS details, al.session_id::text AS session_id
FROM audit_log al
JOIN entities e ON e.id = al.entity_id
WHERE e.slug = $1
ORDER BY al.ts DESC
LIMIT $2`, slug, limit), "change_log"), nil
}},
{tool: &mcp.Tool{Name: "get_state_snapshot", Description: "Last scheduler Observe-pass: fleet health, disk, drift count",
InputSchema: objSchema(),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
return annotateJSONResult(queryRows(ctx, pool, `
SELECT e.slug, e.type, e.state,
COALESCE(st.health, 'unknown') AS health,
COALESCE(st.last_check_at::text, '') AS last_check
FROM entities e
LEFT JOIN entity_status st ON st.entity_id = e.id
WHERE e.state IS NOT NULL
OR st.health IS NOT NULL
ORDER BY st.health, e.slug
LIMIT 200
`), "fleet_snapshot"), nil
}},
{tool: &mcp.Tool{Name: "audit_knowledge_graph", Description: "Read-only drift report over the knowledge graph and monitoring: orphan check entities, checks targeting deprecated/destroyed entities, probes stuck down/unknown, unmonitored declared entity types, and live edges pointing at destroyed targets. Returns ranked findings with a suggested remediation runbook each. Use this to validate the graph is complete and consistent before trusting health/blast-radius answers. Does NOT mutate anything.",
InputSchema: objSchema(),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
findings, summary := audit.Report(ctx, pool)
b, _ := json.Marshal(map[string]any{"findings": findings, "summary": summary})
return textResult(string(b)), nil
}},
{tool: &mcp.Tool{Name: "discover_infra_drift", Description: "Read-only live discovery: compares running Proxmox guests (pct/qm list on every proxmox host) against the DB graph. Returns guests running with no entity (missing) and entities whose pve_id is no longer live (ghost) — drift the DB-only audit_knowledge_graph cannot see. Reaches hosts over the same SSH/pct path the checks use. Does NOT mutate anything.",
InputSchema: objSchema(),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
b, _ := json.Marshal(discoverInfraDrift(ctx, pool))
return textResult(string(b)), nil
}},
{tool: &mcp.Tool{Name: "find_entities_by", Description: "Search entities by discovered attributes — IP address, port, version string, tag, or any key in the attributes JSONB blob. More flexible than list_entities (which filters by type/state only). Use for reverse lookups: 'what runs on port 8096?' or 'which entities have version 2.4?'",
InputSchema: objSchema(
prop{"key", "string", "Attribute key to search (e.g. ip, port, version, tag)"},
prop{"value", "string", "Value to match (case-insensitive substring)"},
prop{"limit", "integer", "Max results (default 25)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
key, _ := args["key"].(string)
val, _ := args["value"].(string)
limit := int(getFloat(args, "limit", 25))
return queryRows(ctx, pool, `
SELECT e.slug, e.type, e.name, e.state, e.attributes->>$1 AS matched_value
FROM entities e
WHERE e.attributes ? $1
AND e.attributes->>$1 ILIKE '%'||$2||'%'
ORDER BY e.slug
LIMIT $3`, key, val, limit), nil
}},
}
}

View File

@@ -0,0 +1,339 @@
package mcp
import (
"context"
"fmt"
"strings"
"github.com/dtoro/oikos/internal/db"
"github.com/google/uuid"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func KnowledgeTools(pool *db.Pool, agentID uuid.UUID, sec secretBackend) []toolReg {
return []toolReg{
{tool: &mcp.Tool{Name: "search_knowledge", Description: "Full-text search across documentation (PostgreSQL FTS with ts_rank ranking). Returns a short snippet per hit, not the full note — call get_knowledge_content with the returned slug to read the whole thing.",
InputSchema: objSchema(prop{"query", "string", "Search terms"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
q := nStr(args["query"])
return annotateJSONResult(queryRows(ctx, pool, `
SELECT ke.title, e.slug,
ts_rank(ke.search, plainto_tsquery('english', $1)) AS rank,
ts_headline('english', ke.content, plainto_tsquery('english', $1),
'MaxWords=40, MinWords=15, ShortWord=3, MaxFragments=3,
FragmentDelimiter=" ... "') AS snippet,
ke.source, ke.tags
FROM knowledge_entities ke
JOIN entities e ON e.id = ke.entity_id
WHERE ke.search @@ plainto_tsquery('english', $1)
ORDER BY rank DESC
LIMIT 20`, q), "knowledge_results"), nil
}},
{tool: &mcp.Tool{Name: "get_entity_knowledge", Description: "All documents, investigations, and runbooks linked to an entity. Returns a headline per note, not the full text — call get_knowledge_content with the returned slug to read the whole thing.",
InputSchema: objSchema(prop{"entity_slug", "string", "Entity slug (e.g. lxc:jellyfin, service:caddy)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["entity_slug"].(string)
return annotateJSONResult(queryRows(ctx, pool, `
SELECT ke.title, ke.source, e.type AS kind, e.slug,
ts_headline('english', ke.content, plainto_tsquery('english', '')) AS headline
FROM knowledge_entities ke
JOIN entities e ON e.id = ke.entity_id
JOIN relationships r ON r.source_id = ke.entity_id
JOIN entities target ON target.id = r.target_id
WHERE target.slug = $1
AND r.valid_to IS NULL
AND r.type IN ('documents', 'about')
UNION
SELECT ke.title, ke.source, e.type AS kind, e.slug,
ts_headline('english', ke.content, plainto_tsquery('english', '')) AS headline
FROM knowledge_entities ke
JOIN entities e ON e.id = ke.entity_id
JOIN relationships r ON r.source_id = ke.entity_id
JOIN entity_types target_type ON target_type.name = (SELECT type FROM entities WHERE slug = $1)
JOIN entities ent ON ent.type = target_type.name AND ent.slug = $1
WHERE r.valid_to IS NULL
AND r.type = 'procedure-for'
ORDER BY 1`, slug), "knowledge_results"), nil
}},
{tool: &mcp.Tool{Name: "get_knowledge_content", Description: "Full markdown body of one document/investigation/runbook, by its own entity slug. search_knowledge and get_entity_knowledge only return short snippets/headlines — once you know which note you need (from either of those, or because you already know its slug), call this to read the whole thing before acting on it.",
InputSchema: objSchema(prop{"slug", "string", "The knowledge entity's own slug (e.g. document:containers/101-jellyfin, runbook:client-enrollment) — not the slug of an entity it's about."}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["slug"].(string)
return queryRows(ctx, pool, `
SELECT ke.title, e.slug, e.type AS kind, ke.content, ke.source, ke.tags, ke.updated_at::text
FROM knowledge_entities ke
JOIN entities e ON e.id = ke.entity_id
WHERE e.slug = $1`, slug), nil
}},
{tool: &mcp.Tool{Name: "upsert_knowledge", Description: "Write back what you learned so future sessions (and future you) benefit — this is how the system gets smarter over time. Use it AFTER solving a non-obvious problem, deploying a service, or discovering a gotcha: record the finding, the fix, and any caveats. Re-calling with the same title updates the existing note instead of duplicating. This is the ONLY way to persist knowledge; a chat message alone is forgotten. search_knowledge/get_entity_knowledge find it, get_knowledge_content reads the full body back.",
InputSchema: objSchema(
prop{"title", "string", "Short, specific, searchable title (e.g. 'Dragonfly memlock rlimit in unprivileged LXCs', not 'notes')."},
prop{"content", "string", "The knowledge itself, in markdown. Be concrete: symptom, root cause, the exact fix/commands, and any caveats. Written for someone hitting this fresh."},
prop{"about", "string", "Optional entity slug(s) this knowledge concerns. Pass a single slug (e.g. 'lxc:nfs-export') or a JSON array of slugs (e.g. '[\"lxc:nfs-export\", \"lxc:gitea\"]') to link to multiple entities. get_entity_knowledge surfaces it for each."},
prop{"tags", "string", "Optional comma-separated tags (e.g. 'docker,networking,gotcha')."},
prop{"kind", "string", "One of: investigation (a finding/incident analysis — default), document (reference), runbook (a repeatable procedure)."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
return upsertKnowledge(ctx, pool, args)
}},
{tool: &mcp.Tool{Name: "delete_knowledge", Description: "Soft-delete a knowledge entry (move to trash, restorable with restore_knowledge). The content and revision history survive.",
InputSchema: objSchema(prop{"knowledge_slug", "string", "Knowledge entity slug or UUID"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["knowledge_slug"].(string)
var entityID uuid.UUID
if u, err := uuid.Parse(slug); err == nil {
entityID = u
} else {
pool.QueryRow(ctx, `SELECT id FROM entities WHERE slug = $1`, slug).Scan(&entityID)
}
if entityID == uuid.Nil {
return textResult(fmt.Sprintf("knowledge entry not found: %s", slug)), nil
}
// Snapshot before tombstoning.
pool.Exec(ctx, `
INSERT INTO knowledge_revisions (entity_id, title, content, source, tags, edited_by, version_at)
SELECT entity_id, title, content, source, tags, COALESCE(edited_by,''), updated_at
FROM knowledge_entities WHERE entity_id = $1`, entityID)
tag, err := pool.Exec(ctx,
`UPDATE knowledge_entities SET deleted_at = now(), edited_by = 'nomos'
WHERE entity_id = $1 AND deleted_at IS NULL`, entityID)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
if tag.RowsAffected() == 0 {
return textResult("knowledge entry already deleted"), nil
}
return textResult(fmt.Sprintf("Knowledge %s soft-deleted. Restore with restore_knowledge.", slug)), nil
}},
{tool: &mcp.Tool{Name: "restore_knowledge", Description: "Restore a soft-deleted knowledge entry from trash. Undoes delete_knowledge.",
InputSchema: objSchema(prop{"knowledge_slug", "string", "Knowledge entity slug or UUID"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["knowledge_slug"].(string)
var entityID uuid.UUID
if u, err := uuid.Parse(slug); err == nil {
entityID = u
} else {
pool.QueryRow(ctx, `SELECT id FROM entities WHERE slug = $1`, slug).Scan(&entityID)
}
if entityID == uuid.Nil {
return textResult(fmt.Sprintf("knowledge entry not found: %s", slug)), nil
}
tag, err := pool.Exec(ctx,
`UPDATE knowledge_entities SET deleted_at = NULL, edited_by = 'nomos'
WHERE entity_id = $1 AND deleted_at IS NOT NULL`, entityID)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
if tag.RowsAffected() == 0 {
return textResult("knowledge entry is not deleted"), nil
}
return textResult(fmt.Sprintf("Knowledge %s restored from trash.", slug)), nil
}},
{tool: &mcp.Tool{Name: "merge_knowledge", Description: "Fold one or more knowledge entries into a target. Source content is appended under a provenance heading, and the union of all tags is kept. Sources are soft-deleted afterwards.",
InputSchema: objSchema(
prop{"target_slug", "string", "Knowledge entry to merge INTO (slug or UUID)"},
prop{"source_slugs", "string", "Comma-separated slugs of entries to fold into the target"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
targetSlug, _ := args["target_slug"].(string)
sourceStr, _ := args["source_slugs"].(string)
var targetID uuid.UUID
if u, err := uuid.Parse(targetSlug); err == nil {
targetID = u
} else {
pool.QueryRow(ctx, `
SELECT ke.entity_id FROM knowledge_entities ke
JOIN entities e ON e.id = ke.entity_id
WHERE (e.slug = $1 OR e.id::text = $1) AND ke.deleted_at IS NULL`,
targetSlug).Scan(&targetID)
}
if targetID == uuid.Nil {
return textResult(fmt.Sprintf("target knowledge entry not found: %s", targetSlug)), nil
}
sources := []string{}
for _, s := range strings.Split(sourceStr, ",") {
if s = strings.TrimSpace(s); s != "" && s != targetSlug {
sources = append(sources, s)
}
}
if len(sources) == 0 {
return textResult("no valid source entries to merge"), nil
}
var appended strings.Builder
merged := []string{}
for _, srcSlug := range sources {
var title, content, updated string
var tags []string
err := pool.QueryRow(ctx, `
SELECT ke.title, ke.content, COALESCE(ke.tags,'{}'), ke.updated_at::text
FROM knowledge_entities ke
JOIN entities e ON e.id = ke.entity_id
WHERE (e.slug = $1 OR e.id::text = $1) AND ke.deleted_at IS NULL`,
srcSlug).Scan(&title, &content, &tags, &updated)
if err != nil {
continue
}
appended.WriteString("\n\n---\n\n## Merged: ")
appended.WriteString(title)
appended.WriteString("\n\n*Originally ")
appended.WriteString(srcSlug)
appended.WriteString(", last updated ")
appended.WriteString(updated)
appended.WriteString("*\n\n")
appended.WriteString(content)
for _, t := range tags {
fmt.Fprintf(&appended, "\ntag: %s", strings.ToLower(strings.TrimSpace(t)))
}
merged = append(merged, srcSlug)
}
if len(merged) == 0 {
return textResult("no source entries could be read"), nil
}
_, err := pool.Exec(ctx, `
UPDATE knowledge_entities SET content = content || $2, edited_by = 'nomos', updated_at = now()
WHERE entity_id = $1`, targetID, appended.String())
if err != nil {
return textResult(fmt.Sprintf("error appending content: %v", err)), nil
}
for _, srcSlug := range merged {
pool.Exec(ctx, `
UPDATE knowledge_entities ke SET deleted_at = now(), edited_by = 'nomos'
FROM entities e
WHERE e.id = ke.entity_id AND (e.slug = $1 OR e.id::text = $1)`,
srcSlug)
}
return textResult(fmt.Sprintf("Merged %d entries into %s: %s", len(merged), targetSlug, strings.Join(merged, ", "))), nil
}},
{tool: &mcp.Tool{Name: "rename_knowledge_tag", Description: "Bulk-rename one or more tags across all knowledge entries. Case-insensitive matching — 'oom' and 'OOM' are treated as the same tag. Deduplicates after rename.",
InputSchema: objSchema(
prop{"from", "string", "Comma-separated tag names to rename FROM"},
prop{"to", "string", "New tag name"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
fromStr, _ := args["from"].(string)
to, _ := args["to"].(string)
to = strings.ToLower(strings.TrimSpace(to))
from := []string{}
for _, f := range strings.Split(fromStr, ",") {
if f = strings.TrimSpace(f); f != "" {
from = append(from, strings.ToLower(f))
}
}
if to == "" || len(from) == 0 {
return textResult("from and to are required"), nil
}
tag, err := pool.Exec(ctx, `
UPDATE knowledge_entities ke
SET tags = sub.new_tags, updated_at = now()
FROM (
SELECT k.entity_id,
ARRAY(SELECT DISTINCT CASE WHEN lower(t) = ANY($1) THEN $2 ELSE t END
FROM unnest(k.tags) AS t) AS new_tags
FROM knowledge_entities k
WHERE k.deleted_at IS NULL
AND EXISTS (SELECT 1 FROM unnest(k.tags) AS t WHERE lower(t) = ANY($1))
) AS sub
WHERE ke.entity_id = sub.entity_id`, from, to)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
return textResult(fmt.Sprintf("Tag %s → %s: %d entries updated.", strings.Join(from, ", "), to, tag.RowsAffected())), nil
}},
{tool: &mcp.Tool{Name: "get_knowledge_revisions", Description: "Version history for a knowledge entry. Returns title, content, editor, tags, and timestamps for each revision.",
InputSchema: objSchema(
prop{"knowledge_slug", "string", "Knowledge entity slug (e.g. document:nomos/something)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["knowledge_slug"].(string)
return queryRows(ctx, pool, `
SELECT kr.id, kr.title, kr.content, COALESCE(kr.edited_by, '') AS edited_by,
COALESCE(kr.tags::text, '{}') AS tags,
kr.version_at::text, kr.revised_at::text
FROM knowledge_revisions kr
JOIN entities e ON e.id = kr.entity_id
WHERE e.slug = $1
ORDER BY kr.version_at DESC LIMIT 50`, slug), nil
}},
{tool: &mcp.Tool{Name: "get_knowledge_duplicates", Description: "Near-duplicate knowledge entries detected via trigram similarity. Returns clusters of similar documents with similarity scores. Use before creating new knowledge to avoid pileup.",
InputSchema: objSchema(
prop{"threshold", "number", "Similarity threshold 0-1 (default 0.6, lower = more matches)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
threshold := getFloat(args, "threshold", 0.6)
return queryRows(ctx, pool, `
SELECT a.slug AS doc_a, b.slug AS doc_b, similarity(ka.title, kb.title) AS sim
FROM knowledge_entities ka
JOIN knowledge_entities kb ON ka.entity_id < kb.entity_id
JOIN entities a ON a.id = ka.entity_id
JOIN entities b ON b.id = kb.entity_id
WHERE ka.deleted_at IS NULL AND kb.deleted_at IS NULL
AND similarity(ka.title, kb.title) > $1
ORDER BY sim DESC LIMIT 100`, threshold), nil
}},
{tool: &mcp.Tool{Name: "get_knowledge_orphans", Description: "Knowledge entries with no entity links (unlinked), no tags (untagged), or stale (not updated in N days). Helps identify abandoned or disconnected knowledge to clean up.",
InputSchema: objSchema(
prop{"stale_days", "integer", "Days without update to consider stale (default 90)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
staleDays := int(getFloat(args, "stale_days", 90))
return queryRows(ctx, pool, fmt.Sprintf(`
SELECT e.slug, ke.title, e.type AS kind, COALESCE(ke.edited_by, '') AS edited_by,
ke.updated_at::text,
(ke.tags IS NULL OR cardinality(ke.tags) = 0) AS untagged,
NOT EXISTS (
SELECT 1 FROM relationships r
WHERE r.source_id = ke.entity_id AND r.valid_to IS NULL
AND r.type IN ('documents', 'about')
) AS unlinked,
(ke.updated_at < now() - interval '%d days') AS stale
FROM knowledge_entities ke
JOIN entities e ON e.id = ke.entity_id
WHERE ke.deleted_at IS NULL
ORDER BY ke.updated_at ASC`, staleDays)), nil
}},
{tool: &mcp.Tool{Name: "list_knowledge_tags", Description: "All tags used across the knowledge base with usage counts. Returns normalized tag, count, and any casing variants (e.g. 'oom' and 'OOM' surface as variants so you can spot drift).",
InputSchema: objSchema(),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
return queryRows(ctx, pool, `
SELECT lower(tag) AS tag, count(*) AS uses,
array_agg(DISTINCT tag ORDER BY tag) AS variants
FROM knowledge_entities ke, unnest(ke.tags) AS tag
WHERE ke.deleted_at IS NULL
GROUP BY lower(tag) ORDER BY uses DESC, lower(tag)`), nil
}},
{tool: &mcp.Tool{Name: "list_my_secrets", Description: "List secrets accessible to this client by public key",
InputSchema: objSchema(prop{"caller_pubkey", "string", "Age public key of the caller (optional)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
pubkey, _ := args["caller_pubkey"].(string)
// Match entities where age_pubkey attribute contains the caller's key.
query := `
SELECT e.slug, e.type, e.name,
e.attributes->>'age_pubkey' AS age_pubkey
FROM entities e
WHERE e.attributes->>'age_pubkey' IS NOT NULL`
var dbArgs []any
if pubkey != "" {
query += ` AND e.attributes->>'age_pubkey' = $1`
dbArgs = append(dbArgs, pubkey)
}
query += ` ORDER BY e.slug LIMIT 100`
return queryRows(ctx, pool, query, dbArgs...), nil
}},
}
}

599
internal/mcp/ops_tools.go Normal file
View File

@@ -0,0 +1,599 @@
package mcp
import (
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"time"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/observability"
"github.com/google/uuid"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func OpsTools(pool *db.Pool, agentID uuid.UUID, sec secretBackend) []toolReg {
return []toolReg{
// ── request_execution (legacy fixed enum) retired 2026-07-14 ──
// All mutations now route through `run`. The handler functions
// (runRexecRestart, runRexecSystemctl, etc.) are kept as reference
// for future runbook extraction — especially pct_create DNS/VMID logic.
// DO NOT re-register this tool. See plans/2026-07-10-general-gated-execution.md.
{tool: &mcp.Tool{Name: "run", Description: "Run ANY shell command against any host, LXC, or VM. This is the general execution primitive — prefer it over asking the operator to run something manually, and don't wait for a matching fixed action to exist. Every command is automatically risk-classified: read-only inspection (cat, systemctl status, docker ps, journalctl, df, git status, ...) runs immediately; anything that changes state requires operator approval (granted by the operator replying \"go ahead\"/\"yes\" in chat, or via the Approve button); commands matching a destructive pattern (rm -rf, dd, mkfs, pct/qm destroy, DROP TABLE, reboot, piping curl into a shell, ...) always require approval regardless of what you declare. You cannot talk your way past the destructive check by declaring a lower risk.\n\nHost-level mutations (apt-get install, dpkg, systemctl enable) always classify as config_mutation — operator approval required.\n\nVM targets: the QEMU guest agent must be running inside the VM. If the entity's qemu_guest_agent attribute is not_running, the run is blocked immediately with a clear error.",
InputSchema: objSchema(
prop{"target", "string", "Target entity slug: host:<slug> (e.g. host:strong), lxc:<slug> (e.g. lxc:caddy), or vm:<slug> (e.g. vm:zimaos). LXC commands run via pct exec on their Proxmox host automatically. VM commands run via qm guest exec on their Proxmox host (requires the QEMU guest agent inside the VM — standard for Proxmox VMs)."},
prop{"command", "string", "The shell command to run. Can be a full script (multi-line, &&-chained). Runs as root."},
prop{"purpose", "string", "One sentence: why you're running this. Shown to the operator alongside the approval — be specific, this is what they're approving."},
prop{"declared_risk", "string", "Optional self-assessment: read_only, reversible_low, config_mutation, or destructive. This can only ESCALATE the automatic classification, never lower it — declaring a mutating command as read_only has no effect."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
targetSlug, _ := args["target"].(string)
command, _ := args["command"].(string)
purpose, _ := args["purpose"].(string)
declaredRisk, _ := args["declared_risk"].(string)
sessionID, _ := args["_session_id"].(string)
if targetSlug == "" || command == "" {
return textResult("error: target and command are required"), nil
}
var targetID uuid.UUID
if err := pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", targetSlug).Scan(&targetID); err != nil {
return textResult(fmt.Sprintf("target not found: %s", targetSlug)), nil
}
return classifyAndGate(ctx, pool, agentID, targetID, targetSlug, command, purpose, declaredRisk, sessionID), nil
}},
// inspect_path is the bulk fact-gathering tool from
// plans/2026-07-18-session-review-three-sessions.md P1.5.
// Sessions 1e9c7691 and 55927f0a each spent ~15 `run` calls
// gathering identical facts (`mount | grep`, `df`, `ls -la`,
// `stat`) across hosts and LXCs to understand where a path
// lives, who mounts it, and what permissions it has. This tool
// collapses that fan-out into one call: pass a path and a list
// of targets, get back per-target mount/df/ls/stat output as
// JSON. All commands are read-only, so no approval is needed.
{tool: &mcp.Tool{Name: "inspect_path", Description: "Bulk fact-gathering: run mount/df/ls/stat for the same path across multiple host/LXC/VM targets in ONE call. Returns a JSON object keyed by target slug, each with the target's view of the path (mount source, filesystem, size, top-level entries with ownership/permissions). Use this instead of N separate `run` calls when you need to understand a path's footprint across the fleet (e.g. tracing where a volume is mounted, checking permissions on the same NFS path from server + client). All commands are read-only — no approval needed.",
InputSchema: objSchema(
prop{"path", "string", "Absolute path to inspect on each target (e.g. /mnt/media_local, /media/ludo-library)."},
prop{"targets", "array", "List of target entity slugs (host:strong, lxc:nfs-export, vm:zimaos, …). Up to 8 per call."},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
path, _ := args["path"].(string)
if path == "" {
return textResult("error: path is required"), nil
}
rawTargets, _ := args["targets"].([]any)
if len(rawTargets) == 0 {
return textResult("error: at least one target is required"), nil
}
if len(rawTargets) > 8 {
return textResult("error: at most 8 targets per inspect_path call (use two calls if you need more)"), nil
}
targets := make([]string, 0, len(rawTargets))
for _, t := range rawTargets {
if s, ok := t.(string); ok && s != "" {
targets = append(targets, s)
}
}
results := inspectPathAcrossTargets(ctx, pool, path, targets)
out, _ := json.MarshalIndent(results, "", " ")
return textResult(string(out)), nil
}},
{tool: &mcp.Tool{Name: "get_execution_status", Description: "Check the status of a requested execution",
InputSchema: objSchema(
prop{"execution_id", "string", "Execution UUID (from request_execution output)"},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
execID, _ := args["execution_id"].(string)
if execID == "" {
return textResult("execution_id required"), nil
}
eid, err := uuid.Parse(execID)
if err != nil {
// Try finding by exec slug prefix
var found uuid.UUID
err2 := pool.QueryRow(ctx, "SELECT entity_id FROM executions WHERE entity_id::text LIKE $1 LIMIT 1", execID+"%").Scan(&found)
if err2 != nil {
return textResult(fmt.Sprintf("execution not found: %s", execID)), nil
}
eid = found
}
return queryRows(ctx, pool, `
SELECT e.entity_id::text, e.action, e.risk_class, e.status,
e.result::text, e.duration_ms, e.started_at::text,
e.completed_at::text, e.correlation_id
FROM executions e
WHERE e.entity_id = $1`, eid), nil
}},
{tool: &mcp.Tool{Name: "tail_log", Description: "Get recent log lines from a service via journalctl",
InputSchema: objSchema(
prop{"service_slug", "string", "Service entity slug (e.g. lxc:caddy)"},
prop{"lines", "integer", "Number of lines (default 50)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["service_slug"].(string)
n := int(getFloat(args, "lines", 50))
if slug == "" {
return textResult("service_slug is required"), nil
}
host, user, err := resolveHost(ctx, pool, slug)
if err != nil {
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
}
svc := strings.TrimPrefix(slug, "lxc:")
out, err := sshExec(ctx, host, user, fmt.Sprintf("journalctl -u %s -n %d --no-pager 2>&1 || true", svc, n))
if err != nil {
return textResult(fmt.Sprintf("ssh: %v", err)), nil
}
return textResult(out), nil
}},
{tool: &mcp.Tool{Name: "get_service_status", Description: "Check systemd service status on a host",
InputSchema: objSchema(
prop{"service_slug", "string", "Service entity slug (e.g. lxc:caddy)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["service_slug"].(string)
if slug == "" {
return textResult("service_slug is required"), nil
}
host, user, err := resolveHost(ctx, pool, slug)
if err != nil {
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
}
svc := strings.TrimPrefix(slug, "lxc:")
out, err := sshExec(ctx, host, user,
fmt.Sprintf("systemctl is-active %s; systemctl is-enabled %s; systemctl show %s -p ActiveEnterTimestamp -p SubState 2>&1 || true", svc, svc, svc))
if err != nil {
return textResult(fmt.Sprintf("ssh: %v", err)), nil
}
return textResult(out), nil
}},
{tool: &mcp.Tool{Name: "get_lxc_state", Description: "Get LXC container resource state from Proxmox host",
InputSchema: objSchema(
prop{"lxc_slug", "string", "LXC entity slug (e.g. lxc:caddy)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["lxc_slug"].(string)
if slug == "" {
return textResult("lxc_slug is required"), nil
}
var pveID string
err := pool.QueryRow(ctx, "SELECT attributes->>'pve_id' FROM entities WHERE slug = $1", slug).Scan(&pveID)
if err != nil || pveID == "" {
return textResult(fmt.Sprintf("LXC not found or missing pve_id: %s", slug)), nil
}
// Resolve the Proxmox host — find the host that runs this LXC
var hostID uuid.UUID
err = pool.QueryRow(ctx, `
SELECT t.id FROM entities t
JOIN relationships r ON r.source_id = t.id
JOIN entities s ON s.id = r.target_id
WHERE s.slug = $1 AND r.type = 'hosts' AND r.valid_to IS NULL
LIMIT 1`, slug).Scan(&hostID)
if err != nil {
// Fallback: use the inventory host attribute if no relationship
var hostSlug string
err = pool.QueryRow(ctx, "SELECT attributes->>'host' FROM entities WHERE slug = $1", slug).Scan(&hostSlug)
if err != nil || hostSlug == "" {
return textResult(fmt.Sprintf("cannot resolve Proxmox host for %s", slug)), nil
}
var host, user string
host, user, err = resolveHost(ctx, pool, "host:"+hostSlug)
if err != nil {
return textResult(fmt.Sprintf("resolve: %v", err)), nil
}
out, err2 := sshExec(ctx, host, user, fmt.Sprintf("pct status %s --verbose 2>&1 || true", pveID))
if err2 != nil {
return textResult(fmt.Sprintf("ssh: %v", err2)), nil
}
return textResult(out), nil
}
var hostSlug string
pool.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", hostID).Scan(&hostSlug)
host, user, err := resolveHost(ctx, pool, hostSlug)
if err != nil {
return textResult(fmt.Sprintf("resolve host: %v", err)), nil
}
out, err := sshExec(ctx, host, user, fmt.Sprintf("pct status %s --verbose 2>&1 || true", pveID))
if err != nil {
return textResult(fmt.Sprintf("ssh: %v", err)), nil
}
return textResult(out), nil
}},
{tool: &mcp.Tool{Name: "ping_service", Description: "Check if a service is reachable via HTTP — returns scheduler health state plus a live HTTP probe",
InputSchema: objSchema(prop{"service_slug", "string", "Service entity slug"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["service_slug"].(string)
if slug == "" {
return textResult("service_slug is required"), nil
}
rows, err := pool.Query(ctx, `
SELECT st.health, st.last_check_at,
COALESCE(
e.attributes->>'url',
CASE WHEN e.attributes->>'public_host' IS NOT NULL
THEN 'https://' || e.attributes->>'public_host'
END
) AS url
FROM entity_status st
JOIN entities e ON e.id = st.entity_id
WHERE e.slug = $1`, slug)
if err != nil {
return textResult(fmt.Sprintf("query error: %v", err)), nil
}
defer rows.Close()
if !rows.Next() {
return textResult(fmt.Sprintf("service not found: %s", slug)), nil
}
var health, lastCheck, url string
rows.Scan(&health, &lastCheck, &url)
if url == "" {
return textResult(fmt.Sprintf("health=%s last_check=%s url=no-url (entity has no url or public_host attribute)", health, lastCheck)), nil
}
// Live HTTP probe — HEAD request to check current state
code := "n/a"
if resp, err := http.Head(url); err == nil {
resp.Body.Close()
code = fmt.Sprintf("%d", resp.StatusCode)
} else {
code = fmt.Sprintf("err: %v", err)
}
return textResult(fmt.Sprintf("health=%s last_check=%s url=%s http=%s", health, lastCheck, url, code)), nil
}},
// ─── Phase 5: operational MCP tools ──────────────────────────────
{tool: &mcp.Tool{Name: "list_lxcs", Description: "List all LXC containers with ID, host, IP, state, and last-audited hint. Pass state=\"active\" to exclude destroyed/deprecated containers. The last_audited_at column shows the most recent knowledge entry (investigation or document tagged audit/update) linked via an 'about' edge — use it to skip re-running `run` against LXCs that were already audited recently.",
InputSchema: objSchema(
prop{"state", "string", "Optional: filter by entity state (active, destroyed, …)"},
),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
state, _ := argsMap(req)["state"].(string)
var statePtr *string
if state != "" {
statePtr = &state
}
return annotateJSONResult(queryRows(ctx, pool, `
SELECT e.slug, e.name, e.attributes->>'pve_id' AS pve_id,
e.attributes->>'lan_ip' AS lan_ip,
e.state,
st.health, st.last_check_at,
(SELECT MAX(k.created_at)
FROM relationships r
JOIN knowledge_entities k ON k.entity_id = r.source_id
WHERE r.target_id = e.id
AND r.type = 'about'
AND r.valid_to IS NULL
AND (k.tags @> ARRAY['audit']::text[]
OR k.tags @> ARRAY['update']::text[]
OR k.title ILIKE '%audit%'
OR k.title ILIKE '%update%')
) AS last_audited_at
FROM entities e
LEFT JOIN entity_status st ON st.entity_id = e.id
WHERE e.type = 'lxc'
AND ($1::text IS NULL OR e.state = $1)
ORDER BY CASE WHEN e.state = 'active' THEN 0 ELSE 1 END,
(e.attributes->>'pve_id')::int`, statePtr), "lxc_list"), nil
}},
// ── Stage 2: External agent observe ──────────────────────────
// ── Stage 4: External agent act (mutations) ─────────────────────
{tool: &mcp.Tool{Name: "ack_signal", Description: "Acknowledge an open signal. Use when investigating an alert — marks it as seen and being worked on.",
InputSchema: objSchema(prop{"signal_id", "string", "Signal entity UUID"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
sid, _ := args["signal_id"].(string)
id, err := uuid.Parse(sid)
if err != nil {
return textResult(fmt.Sprintf("invalid signal_id: %v", err)), nil
}
tag, err := pool.Exec(ctx,
`UPDATE signals SET state = 'acknowledged', updated_at = now()
WHERE entity_id = $1 AND state IN ('raised','failed')`, id)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
if tag.RowsAffected() == 0 {
return textResult(fmt.Sprintf("signal %s not found or not in a state that can be acknowledged", sid)), nil
}
return textResult(fmt.Sprintf("Signal %s acknowledged.", sid)), nil
}},
{tool: &mcp.Tool{Name: "resolve_signal", Description: "Resolve a signal with an optional resolution note. Use when the underlying issue is fixed — marks the signal as resolved so it stops showing as active.",
InputSchema: objSchema(
prop{"signal_id", "string", "Signal entity UUID"},
prop{"resolution", "string", "Optional note describing what fixed it"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
sid, _ := args["signal_id"].(string)
id, err := uuid.Parse(sid)
if err != nil {
return textResult(fmt.Sprintf("invalid signal_id: %v", err)), nil
}
tag, err := pool.Exec(ctx,
`UPDATE signals SET state = 'resolved', updated_at = now()
WHERE entity_id = $1 AND state IN ('raised','acknowledged','acting','failed')`, id)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
if tag.RowsAffected() == 0 {
return textResult(fmt.Sprintf("signal %s not found or not in a state that can be resolved", sid)), nil
}
resolution, _ := args["resolution"].(string)
if resolution != "" {
return textResult(fmt.Sprintf("Signal %s resolved: %s", sid, resolution)), nil
}
return textResult(fmt.Sprintf("Signal %s resolved.", sid)), nil
}},
{tool: &mcp.Tool{Name: "mute_signal", Description: "Temporarily mute a signal. Suppresses it from active views for the given duration. Use for known, non-urgent issues that don't need immediate attention.",
InputSchema: objSchema(
prop{"signal_id", "string", "Signal entity UUID"},
prop{"duration_s", "integer", "Mute duration in seconds (default 3600 = 1 hour)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
sid, _ := args["signal_id"].(string)
id, err := uuid.Parse(sid)
if err != nil {
return textResult(fmt.Sprintf("invalid signal_id: %v", err)), nil
}
dur := int64(getFloat(args, "duration_s", 3600))
muteUntil := time.Now().UTC().Add(time.Duration(dur) * time.Second)
tag, err := pool.Exec(ctx,
`UPDATE signals SET state = 'muted', mute_until = $2, updated_at = now()
WHERE entity_id = $1 AND state IN ('raised','acknowledged')`, id, muteUntil)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
if tag.RowsAffected() == 0 {
return textResult(fmt.Sprintf("signal %s not found or not in a state that can be muted", sid)), nil
}
return textResult(fmt.Sprintf("Signal %s muted until %s.", sid, muteUntil.Format(time.RFC3339))), nil
}},
{tool: &mcp.Tool{Name: "cancel_execution", Description: "Cancel a queued or running execution. Use when you realize the command was wrong, targets the wrong host, or should not proceed. Requires a reason.",
InputSchema: objSchema(
prop{"execution_id", "string", "Execution entity UUID"},
prop{"reason", "string", "Why this execution should be cancelled"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
eid, _ := args["execution_id"].(string)
id, err := uuid.Parse(eid)
if err != nil {
return textResult(fmt.Sprintf("invalid execution_id: %v", err)), nil
}
reason, _ := args["reason"].(string)
result := jsonErr("cancelled by agent: %s", reason)
tag, err := pool.Exec(ctx,
`UPDATE executions SET status = 'cancelled', result = $2::jsonb
WHERE entity_id = $1 AND status IN ('running','pending_approval','approved','queued')`,
id, result)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
if tag.RowsAffected() == 0 {
return textResult(fmt.Sprintf("execution %s not found or already final", eid)), nil
}
// Write audit entry.
_ = observability.Audit(ctx, sqlcgen.New(pool), "agent", "nomos", "cancel",
&id, "POST", "/mcp", "", nil,
map[string]any{"reason": reason})
return textResult(fmt.Sprintf("Execution %s cancelled: %s", eid, reason)), nil
}},
{tool: &mcp.Tool{Name: "update_check", Description: "Enable or disable a health check. Disable a noisy probe that's firing false positives; re-enable after fixing the underlying issue.",
InputSchema: objSchema(
prop{"check_id", "string", "Check entity UUID"},
prop{"enabled", "boolean", "true to enable, false to disable"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
cid, _ := args["check_id"].(string)
id, err := uuid.Parse(cid)
if err != nil {
return textResult(fmt.Sprintf("invalid check_id: %v", err)), nil
}
enabled, _ := args["enabled"].(bool)
tag, err := pool.Exec(ctx,
`UPDATE check_defs SET enabled = $2 WHERE entity_id = $1`, id, enabled)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
if tag.RowsAffected() == 0 {
return textResult(fmt.Sprintf("check %s not found", cid)), nil
}
status := "enabled"
if !enabled {
status = "disabled"
}
return textResult(fmt.Sprintf("Check %s %s.", cid, status)), nil
}},
{tool: &mcp.Tool{Name: "list_checks", Description: "List health checks with verdict, last run time, probe kind, and config. Filter by entity slug or enabled status. Each check's last_health explains which probe is responsible for an entity's overall health.",
InputSchema: objSchema(
prop{"entity_slug", "string", "Filter by target entity slug"},
prop{"enabled", "boolean", "Filter enabled/disabled (optional)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
return annotateJSONResult(queryRows(ctx, pool, `
SELECT cd.entity_id, e.slug, cd.kind,
COALESCE(te.slug, '') AS target_slug, cd.target_type,
cd.config::text, cd.interval_s, cd.timeout_s, cd.enabled,
e.version, cd.last_health, cd.last_run_at::text
FROM check_defs cd
JOIN entities e ON e.id = cd.entity_id
LEFT JOIN entities te ON te.id = cd.target_id
WHERE ($1::text IS NULL OR te.slug = $1)
AND ($2::bool IS NULL OR cd.enabled = $2)
ORDER BY e.slug LIMIT 200`,
nStr(args["entity_slug"]), args["enabled"]), "check_table"), nil
}},
{tool: &mcp.Tool{Name: "list_executions", Description: "Cursor-paginated execution history. Filter by entity slug, status, or risk class. Returns newest-first with duration, result, and target info.",
InputSchema: objSchema(
prop{"entity_slug", "string", "Filter by target entity slug"},
prop{"status", "string", "Filter by status (running/completed/failed/pending_approval)"},
prop{"limit", "integer", "Max rows (default 25)"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
limit := int(getFloat(args, "limit", 25))
return queryRows(ctx, pool, `
SELECT e.entity_id, te.slug AS target, e.action, e.risk_class,
e.status, e.result::text, e.duration_ms,
e.correlation_id, e.started_at::text, e.completed_at::text, e.created_at::text,
COALESCE(npe.session_id::text, '') AS session_id
FROM executions e
JOIN entities te ON te.id = e.target_entity_id
LEFT JOIN nomos_plan_executions npe ON npe.execution_id = e.entity_id
WHERE ($1::text IS NULL OR te.slug = $1)
AND ($2::text IS NULL OR e.status = $2)
ORDER BY e.created_at DESC LIMIT $3`,
nStr(args["entity_slug"]), nStr(args["status"]), limit), nil
}},
{tool: &mcp.Tool{Name: "list_entity_sessions", Description: "Active Nomos sessions (tasks) linked to an entity. Shows goal, status, outcome, and when the session was last active. Use to discover what agents are working on related to this entity.",
InputSchema: objSchema(
prop{"entity_slug", "string", "Entity slug to find sessions for"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["entity_slug"].(string)
return queryRows(ctx, pool, `
SELECT DISTINCT as2.id, as2.title, as2.goal, as2.status, as2.outcome,
as2.summary, as2.last_active_at::text, as2.closed_at::text
FROM agent_sessions as2
JOIN nomos_plan_executions npe ON npe.session_id = as2.id
JOIN executions ex ON ex.entity_id = npe.execution_id
JOIN entities te ON te.id = ex.target_entity_id
WHERE te.slug = $1 AND as2.closed_at IS NULL
ORDER BY as2.last_active_at DESC LIMIT 20`, slug), nil
}},
// ── Stage 2: External agent observe ──────────────────────────
{tool: &mcp.Tool{Name: "get_dashboard_summary", Description: "Fleet overview in one call: entity counts by type and state, health breakdown (healthy/degraded/down/stale/unknown), active signals by severity, pending approval count, execution counts in last 24h, and event rate over last 6h.",
InputSchema: objSchema(),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
result := map[string]any{}
// Entity counts by type
result["entities_by_type"] = rowsToMap(ctx, pool,
`SELECT type, count(*) FROM entities GROUP BY type`)
// Entity counts by state
result["entities_by_state"] = rowsToMap(ctx, pool,
`SELECT coalesce(state, 'unknown'), count(*) FROM entities GROUP BY state`)
// Health rollup (excluding check entities)
result["health"] = rowsToMap(ctx, pool, `
SELECT COALESCE(st.health, 'unknown') AS health, count(*)
FROM entity_status st JOIN entities e ON e.id = st.entity_id
WHERE e.type <> 'check' GROUP BY st.health`)
// Active signals by severity
result["signals_by_severity"] = rowsToMap(ctx, pool, `
SELECT severity, count(*) FROM signals
WHERE state NOT IN ('resolved', 'failed') GROUP BY severity`)
// Pending approvals
var pending int
pool.QueryRow(ctx, `SELECT count(*) FROM approvals WHERE status = 'pending'`).Scan(&pending)
result["approvals_pending"] = pending
// Executions in last 24h
result["executions_by_state"] = rowsToMap(ctx, pool, `
SELECT status, count(*) FROM executions
WHERE created_at > now() - interval '24 hours' GROUP BY status`)
// Event rate (5-min buckets over 6h)
events := []map[string]any{}
erows, _ := pool.Query(ctx, `
SELECT date_trunc('hour', ts) + (extract(minute FROM ts)::int / 5) * interval '5 minutes' AS bucket, count(*)
FROM events WHERE ts > now() - interval '6 hours'
GROUP BY bucket ORDER BY bucket`)
if erows != nil {
for erows.Next() {
var bucket time.Time
var n int
if erows.Scan(&bucket, &n) == nil {
events = append(events, map[string]any{"bucket": bucket, "count": n})
}
}
erows.Close()
}
result["event_rate"] = events
b, _ := json.MarshalIndent(result, "", " ")
return textResult(string(b)), nil
}},
{tool: &mcp.Tool{Name: "get_secret", Description: "Retrieve a secret value from the Infisical vault. Returns the secret value. Use for service credentials, tokens, and keys needed to operate the homelab.",
InputSchema: objSchema(
prop{"key", "string", "Secret key to retrieve (e.g. 'matrix-token', 'clients/host:hubris/age-key')"},
prop{"path", "string", "Secret path prefix (default '/')"},
prop{"environment", "string", "Environment slug (default 'dev')"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
if sec == nil {
return textResult("error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)"), nil
}
args := argsMap(req)
key, _ := args["key"].(string)
if key == "" {
return textResult("error: key is required"), nil
}
val, err := sec.Get(ctx, key)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
return textResult(val), nil
}},
{tool: &mcp.Tool{Name: "list_secrets", Description: "List secret keys in the Infisical vault. Returns key names only (no values). Filter by path prefix to scope to a client or shared path.",
InputSchema: objSchema(
prop{"path_prefix", "string", "Filter to keys matching this prefix (e.g. 'clients/', 'shared/', 'config/')"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
if sec == nil {
return textResult("error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)"), nil
}
args := argsMap(req)
prefix, _ := args["path_prefix"].(string)
keys, err := sec.List(ctx)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
if prefix != "" {
filtered := keys[:0]
for _, k := range keys {
if strings.HasPrefix(k, prefix) {
filtered = append(filtered, k)
}
}
keys = filtered
}
data, _ := json.MarshalIndent(keys, "", " ")
return textResult(string(data)), nil
}},
{tool: &mcp.Tool{Name: "set_secret", Description: "Store or update a secret in the Infisical vault. Use when discovering new credentials that need to be persisted. Requires operator approval (config_mutation).",
InputSchema: objSchema(
prop{"key", "string", "Secret key to store"},
prop{"value", "string", "Secret value to store"},
prop{"path", "string", "Secret path prefix (default '/')"},
prop{"environment", "string", "Environment slug (default 'dev')"}),
}, handler: func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
if sec == nil {
return textResult("error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)"), nil
}
args := argsMap(req)
key, _ := args["key"].(string)
value, _ := args["value"].(string)
if key == "" {
return textResult("error: key is required"), nil
}
if value == "" {
return textResult("error: value is required"), nil
}
if err := sec.Set(ctx, key, value); err != nil {
return textResult(fmt.Sprintf("error: %v", err)), nil
}
return textResult(fmt.Sprintf("secret %s stored", key)), nil
}},
}
}

View File

@@ -0,0 +1,198 @@
package mcp
import (
"context"
"encoding/json"
"testing"
"github.com/dtoro/oikos/internal/secrets"
"github.com/google/uuid"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
type mockSecretBackend struct {
data map[string]string
}
func (m *mockSecretBackend) Get(ctx context.Context, key string) (string, error) {
v, ok := m.data[key]
if !ok {
return "", secrets.ErrNotFound
}
return v, nil
}
func (m *mockSecretBackend) Set(ctx context.Context, key string, value string) error {
m.data[key] = value
return nil
}
func (m *mockSecretBackend) List(ctx context.Context) ([]string, error) {
keys := make([]string, 0, len(m.data))
for k := range m.data {
keys = append(keys, k)
}
return keys, nil
}
func (m *mockSecretBackend) Name() string { return "mock" }
// findToolHandler locates a tool's handler from allTools by name.
func findToolHandler(t *testing.T, pool interface{}, name string, sec secrets.Backend) func(context.Context, *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
t.Helper()
for _, r := range allTools(nil, uuid.Nil, sec) {
if r.tool.Name == name {
return r.handler
}
}
t.Fatalf("tool %q not found", name)
return nil
}
func callToolJSON(t *testing.T, name string, sec secrets.Backend, args map[string]any) any {
t.Helper()
handler := findToolHandler(t, nil, name, sec)
argBytes, _ := json.Marshal(args)
req := &mcp.CallToolRequest{
Params: &mcp.CallToolParamsRaw{Arguments: argBytes},
}
result, err := handler(context.Background(), req)
if err != nil {
t.Fatalf("tool %q error: %v", name, err)
}
if len(result.Content) == 0 {
t.Fatalf("tool %q returned no content", name)
}
tc := result.Content[0].(*mcp.TextContent)
var out any
if err := json.Unmarshal([]byte(tc.Text), &out); err != nil {
// Not JSON — return raw string
return tc.Text
}
return out
}
func callToolText(t *testing.T, name string, sec secrets.Backend, args map[string]any) string {
t.Helper()
handler := findToolHandler(t, nil, name, sec)
argBytes, _ := json.Marshal(args)
req := &mcp.CallToolRequest{
Params: &mcp.CallToolParamsRaw{Arguments: argBytes},
}
result, err := handler(context.Background(), req)
if err != nil {
t.Fatalf("tool %q error: %v", name, err)
}
if len(result.Content) == 0 {
t.Fatalf("tool %q returned no content", name)
}
return result.Content[0].(*mcp.TextContent).Text
}
func TestGetSecret(t *testing.T) {
sec := &mockSecretBackend{
data: map[string]string{
"matrix-token": "bot-token-123",
"clients/host:hubris/age-key": "AGE-SECRET-KEY",
},
}
// Get existing key
val := callToolText(t, "get_secret", sec, map[string]any{"key": "matrix-token"})
if val != "bot-token-123" {
t.Errorf("get_secret = %q, want bot-token-123", val)
}
// Get missing key
errText := callToolText(t, "get_secret", sec, map[string]any{"key": "nonexistent"})
if errText == "" {
t.Error("expected error for missing key")
}
// Missing key arg
errText = callToolText(t, "get_secret", sec, map[string]any{})
if errText != "error: key is required" {
t.Errorf("missing key error = %q, want error: key is required", errText)
}
}
func TestListSecrets(t *testing.T) {
sec := &mockSecretBackend{
data: map[string]string{
"clients/host:hubris/age-key": "val1",
"clients/host:strong/age-key": "val2",
"shared/matrix-token": "val3",
},
}
// List all
out := callToolJSON(t, "list_secrets", sec, map[string]any{})
keys, ok := out.([]any)
if !ok {
t.Fatalf("list_secrets returned non-array: %T", out)
}
if len(keys) != 3 {
t.Errorf("list_secrets count = %d, want 3", len(keys))
}
// List with prefix filter
out = callToolJSON(t, "list_secrets", sec, map[string]any{"path_prefix": "clients/"})
keys, ok = out.([]any)
if !ok {
t.Fatalf("filtered list returned non-array: %T", out)
}
if len(keys) != 2 {
t.Errorf("filtered list count = %d, want 2", len(keys))
}
}
func TestSetSecret(t *testing.T) {
sec := &mockSecretBackend{
data: map[string]string{},
}
// Set a key
result := callToolText(t, "set_secret", sec, map[string]any{"key": "test-key", "value": "test-value"})
if result != "secret test-key stored" {
t.Errorf("set_secret = %q, want 'secret test-key stored'", result)
}
// Verify it was stored
val, err := sec.Get(context.Background(), "test-key")
if err != nil {
t.Fatalf("verify get: %v", err)
}
if val != "test-value" {
t.Errorf("stored value = %q, want test-value", val)
}
// Missing key
errText := callToolText(t, "set_secret", sec, map[string]any{})
if errText != "error: key is required" {
t.Errorf("missing key error = %q", errText)
}
// Missing value
errText = callToolText(t, "set_secret", sec, map[string]any{"key": "x"})
if errText != "error: value is required" {
t.Errorf("missing value error = %q", errText)
}
}
func TestSecretToolsNilBackend(t *testing.T) {
// All tools should return a graceful error when no backend is configured
errText := callToolText(t, "get_secret", nil, map[string]any{"key": "x"})
if errText != "error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)" {
t.Errorf("nil backend get_secret = %q", errText)
}
errText = callToolText(t, "list_secrets", nil, map[string]any{})
if errText != "error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)" {
t.Errorf("nil backend list_secrets = %q", errText)
}
errText = callToolText(t, "set_secret", nil, map[string]any{"key": "x", "value": "y"})
if errText != "error: no secrets backend configured (set OIKOS_INFISICAL_SITE_URL)" {
t.Errorf("nil backend set_secret = %q", errText)
}
}

View File

@@ -19,6 +19,7 @@ import (
"sync"
"time"
"github.com/dtoro/oikos/internal/actuator"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/execlog"
@@ -47,10 +48,19 @@ func objSchema(props ...prop) *jsonschema.Schema {
return s
}
// secretBackend is the interface MCP tools use to access the secrets store.
// Defined here to avoid importing the full secrets package (which brings in
// the Infisical SDK). Mirrors the subset of secrets.Backend used by tools.
type secretBackend interface {
Get(ctx context.Context, key string) (string, error)
Set(ctx context.Context, key string, value string) error
List(ctx context.Context) ([]string, error)
}
// NewHandler creates an http.Handler that serves the Oikos MCP server.
// agentID is the Nomos agent entity UUID; tool calls are logged to agent_activity.
func NewHandler(pool *db.Pool, token string, agentID uuid.UUID) http.Handler {
s := newServer(pool, agentID)
func NewHandler(pool *db.Pool, token string, agentID uuid.UUID, sec secretBackend) http.Handler {
s := newServer(pool, agentID, sec)
handler := mcp.NewStreamableHTTPHandler(func(r *http.Request) *mcp.Server {
if token != "" {
if r.Header.Get("Authorization") != "Bearer "+token {
@@ -65,11 +75,11 @@ func NewHandler(pool *db.Pool, token string, agentID uuid.UUID) http.Handler {
// toolHandler is the function signature registered via AddTool.
type toolHandler = mcp.ToolHandler
func newServer(pool *db.Pool, agentID uuid.UUID) *mcp.Server {
func newServer(pool *db.Pool, agentID uuid.UUID, sec secretBackend) *mcp.Server {
s := mcp.NewServer(&mcp.Implementation{Name: "oikos", Version: "dev"}, &mcp.ServerOptions{
Logger: slog.Default(),
})
for _, t := range allTools(pool, agentID) {
for _, t := range allTools(pool, agentID, sec) {
s.AddTool(t.tool, withActivityLogging(pool, agentID, t.tool.Name, t.handler))
}
@@ -442,30 +452,6 @@ func initSSH() {
// goroutine forever with no way for the caller to ever get an answer.
const sshExecTimeout = 10 * time.Minute
// streamWriter buffers everything it is given while forwarding each write to a
// sink. Assigning one to session.Stdout and another (sharing the same buffer)
// to session.Stderr reproduces CombinedOutput's interleaving exactly, in the
// order the remote end actually produced it — which reading from StdoutPipe
// and StderrPipe separately would not guarantee.
type streamWriter struct {
mu *sync.Mutex
buf *bytes.Buffer
stream string
sink execlog.Sink
}
func (w *streamWriter) Write(p []byte) (int, error) {
w.mu.Lock()
w.buf.Write(p)
w.mu.Unlock()
if w.sink != nil {
// Copy: the ssh library reuses p after Write returns, and the sink
// hands the bytes to a DB call that may outlive this frame.
w.sink(w.stream, append([]byte(nil), p...))
}
return len(p), nil
}
func sshExec(ctx context.Context, host, user, command string) (string, error) {
return sshExecStream(ctx, host, user, command, nil)
}
@@ -481,96 +467,18 @@ func sshExecStream(ctx context.Context, host, user, command string, sink execlog
user = sshUser
}
addr := host + ":22"
signer, err := ssh.ParsePrivateKey(sshKey)
signer, err := actuator.LoadSignerFromBytes(sshKey)
if err != nil {
return "", fmt.Errorf("parse key: %w", err)
}
cfg := &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Timeout: 10 * time.Second,
}
client, err := ssh.Dial("tcp", addr, cfg)
client, err := actuator.Dial(ctx, actuator.DialOptions{Host: host, User: user, Signer: signer})
if err != nil {
return "", fmt.Errorf("dial %s: %w", host, err)
return "", err
}
defer client.Close()
session, err := client.NewSession()
if err != nil {
return "", fmt.Errorf("session: %w", err)
}
defer session.Close()
var (
mu sync.Mutex
buf bytes.Buffer
)
session.Stdout = &streamWriter{mu: &mu, buf: &buf, stream: "stdout", sink: sink}
session.Stderr = &streamWriter{mu: &mu, buf: &buf, stream: "stderr", sink: sink}
// collected returns whatever output has arrived so far. Callable while the
// command is still running, which is what makes partial output on timeout
// possible.
collected := func() string {
mu.Lock()
defer mu.Unlock()
return strings.TrimSpace(buf.String())
}
done := make(chan error, 1)
go func() {
// Recovers a panic in the SSH library internals (rare but not
// impossible) and reports it as a failed command instead of crashing
// the whole api process — every gated action runs through this
// function, so an unrecovered panic here would take down every
// concurrently-running task's execution, not just this one. Without
// this, a panic would ALSO silently degrade to "wait out the full
// timeout" (done never receives, the select below falls through to
// its time.After case) rather than crashing outright — recovering
// and sending an immediate result is strictly better: the caller
// finds out now, not after sshExecTimeout.
defer func() {
if r := recover(); r != nil {
done <- fmt.Errorf("panic in ssh exec: %v", r)
}
}()
// Run rather than CombinedOutput so the assigned writers are used;
// Run returns only after both streams have been fully drained.
done <- session.Run(command)
}()
select {
case err := <-done:
text := collected()
// A non-zero exit MUST surface as an error — matching the fix
// applied to httpapi's sshExec (this copy still had the original
// bug: only erroring when there was no output at all, so a command
// that failed but printed something was silently reported as
// success).
if err != nil {
if text != "" {
return text, fmt.Errorf("%w: %s", err, text)
}
return text, fmt.Errorf("exec: %w", err)
}
return text, nil
case <-time.After(sshExecTimeout):
session.Close()
client.Close()
// Return what the command managed to print before it hung. This used
// to return "", discarding everything — so a hung command, the case
// where the output matters most, was the one case that left no trace.
return collected(), fmt.Errorf("timed out after %s waiting for command to finish on %s", sshExecTimeout, host)
case <-ctx.Done():
session.Close()
client.Close()
return collected(), ctx.Err()
}
return actuator.RunStreaming(ctx, client, command, sink, sshExecTimeout)
}
// resolveHost resolves a host:<slug> to its reachable IP and SSH user. A thin
@@ -1013,6 +921,13 @@ func classifyAndGate(ctx context.Context, pool *db.Pool, agentID, targetID uuid.
classID, actionCol, riskClass, classRoute, classReason, correlationID)
// Link classification to execution.
pool.Exec(ctx, `UPDATE executions SET classification_id = $2 WHERE entity_id = $1`, id, classID)
// Graph edge: classification —precedes→ execution (required by ontology).
pool.Exec(ctx, `INSERT INTO relationships (source_id, target_id, type, attributes, valid_from)
SELECT $1, $2, 'precedes', '{"by":"nomos"}'::jsonb, now()
WHERE NOT EXISTS (
SELECT 1 FROM relationships
WHERE source_id = $1 AND target_id = $2 AND type = 'precedes' AND valid_to IS NULL)`,
classID, id)
// Audit: record the execution creation with session_id for traceability.
// Every run call, whether auto-run or queued-for-approval, gets an audit

View File

@@ -86,7 +86,7 @@ func TestNewServerRegistersTools(t *testing.T) {
}()
// pool is only used inside tool handlers (invoked per-call), not at
// registration time, so a nil pool is safe for this construction test.
s := newServer(nil, uuid.Nil)
s := newServer(nil, uuid.Nil, nil)
if s == nil {
t.Fatal("newServer returned nil")
}

File diff suppressed because it is too large Load Diff

View File

@@ -18,6 +18,7 @@ import (
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/health"
"github.com/google/uuid"
)
@@ -25,7 +26,14 @@ import (
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
slog.Info("notifier: starting")
// Liveness probe (plan D5): the notifier ticks every 15s (approvals) and
// 30s (reactions). 2 min staleness covers a slow Matrix round-trip plus a
// missed tick without false-failing.
probe := health.New(2 * time.Minute)
probe.Serve(ctx, cfg.HealthListen)
processPendingApprovals(ctx, pool, cfg)
probe.Bump()
ticker := time.NewTicker(15 * time.Second)
defer ticker.Stop()
@@ -40,8 +48,10 @@ func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
return
case <-ticker.C:
processPendingApprovals(ctx, pool, cfg)
probe.Bump()
case <-reactionTimer.C:
pollReactions(ctx, pool, cfg)
probe.Bump()
}
}
}
@@ -52,13 +62,13 @@ func RunnerForMain() func(context.Context, *db.Pool, config.Config) {
}
type pendingApproval struct {
ID uuid.UUID
Action string
RiskClass string
TokenHash *string
AlertSentAt *time.Time
MatrixEventID *string
ExpiresAt time.Time
ID uuid.UUID
Action string
RiskClass string
TokenHash *string
AlertSentAt *time.Time
MatrixEventID *string
ExpiresAt time.Time
}
// processPendingApprovals finds pending approvals, generates tokens, and sends Matrix alerts.
@@ -173,7 +183,7 @@ func checkReaction(ctx context.Context, cfg config.Config, roomID, eventID strin
var result struct {
Chunk []struct {
Type string `json:"type"`
Type string `json:"type"`
Content struct {
RelatesTo map[string]string `json:"m.relates_to"`
} `json:"content"`
@@ -259,7 +269,9 @@ func sendMatrixAlert(ctx context.Context, cfg config.Config, approvalID uuid.UUI
}
defer resp.Body.Close()
var mxResp struct{ EventID string `json:"event_id"` }
var mxResp struct {
EventID string `json:"event_id"`
}
json.NewDecoder(resp.Body).Decode(&mxResp)
if mxResp.EventID == "" {

View File

@@ -0,0 +1,102 @@
package ontology
import (
"context"
"errors"
"testing"
"github.com/dtoro/oikos/internal/domain"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgxpool"
)
// The lifecycle precondition checks split into a pure attribute/type guard
// and a DB query. These cover the pure guards at 0%: the entity-type skip
// rules and the attribute presence/absence semantics. The DB-backed checks
// (health, edges, backups, docs) are exercised by make test-db.
//
// ctx/pool/entityID are unused by the pure guards, so nil is safe here.
var (
noCtx = context.Background()
noPool *pgxpool.Pool // nil: the pure guards never touch the pool
noID = uuid.New()
)
func TestCheckAgeKeyEnrolled(t *testing.T) {
cases := []struct {
name string
entityType string
attrs map[string]any
wantErr bool
}{
{"workstation with age key", "workstation", map[string]any{"age_pubkey": "age1abc"}, false},
{"workstation missing age key", "workstation", map[string]any{}, true},
{"server needs a key too", "server", map[string]any{}, true},
{"lxc is exempt", "lxc", map[string]any{}, false},
{"vm is exempt", "vm", map[string]any{}, false},
{"docker-container is exempt", "docker-container", map[string]any{}, false},
{"nil attrs on a workstation", "workstation", nil, true},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
err := checkAgeKeyEnrolled(noCtx, noPool, noID, c.entityType, c.attrs)
if c.wantErr && !errors.Is(err, domain.ErrInvalidTransition) {
t.Errorf("want ErrInvalidTransition, got %v", err)
}
if !c.wantErr && err != nil {
t.Errorf("want nil, got %v", err)
}
})
}
}
func TestCheckMeshJoined(t *testing.T) {
cases := []struct {
name string
entityType string
attrs map[string]any
wantErr bool
}{
{"workstation with mesh_ip", "workstation", map[string]any{"mesh_ip": "10.0.0.5"}, false},
{"workstation missing mesh_ip", "workstation", map[string]any{}, true},
{"server missing mesh_ip", "server", map[string]any{}, true},
{"lxc is exempt", "lxc", map[string]any{}, false},
{"vm is exempt", "vm", map[string]any{}, false},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
err := checkMeshJoined(noCtx, noPool, noID, c.entityType, c.attrs)
if c.wantErr && !errors.Is(err, domain.ErrInvalidTransition) {
t.Errorf("want ErrInvalidTransition, got %v", err)
}
if !c.wantErr && err != nil {
t.Errorf("want nil, got %v", err)
}
})
}
}
func TestCheckSecretsRevoked(t *testing.T) {
// checkSecretsRevoked treats an ABSENT age_pubkey as "secrets revoked"
// (the inverse of checkAgeKeyEnrolled). It is type-agnostic.
cases := []struct {
name string
attrs map[string]any
wantErr bool
}{
{"age key gone → revoked", map[string]any{}, false},
{"age key still present → blocked", map[string]any{"age_pubkey": "age1abc"}, true},
{"nil attrs → revoked", nil, false},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
err := checkSecretsRevoked(noCtx, noPool, noID, "workstation", c.attrs)
if c.wantErr && !errors.Is(err, domain.ErrInvalidTransition) {
t.Errorf("want ErrInvalidTransition, got %v", err)
}
if !c.wantErr && err != nil {
t.Errorf("want nil, got %v", err)
}
})
}
}

View File

@@ -72,6 +72,7 @@ var readOnlyLeadPattern = regexp.MustCompile(
`grep|egrep|fgrep|rg|wc|sort|uniq|cut|tr|tee|find|tree|locate|` +
`dpkg\s+(-l|-s|--list|--status)\b|apt\s+(list|search|show)\b|` +
`systemctl\s+(status|is-active|is-enabled|is-failed|list-units|list-unit-files|list-timers|show)\b|` +
`ethtool|lsmod|lspci|modinfo|dkms|` +
`timedatectl|hostnamectl|systemd-analyze|` +
`docker\s+(ps|images|inspect|logs|version|info|stats)|` +
`docker\s+compose\s+(logs|ps|top|config|images|port|cp)\b|` +

View File

@@ -46,6 +46,12 @@ func TestClassifyCommand_ReadOnly(t *testing.T) {
"sudo pct exec 121 -- systemctl status caddy",
// qm guest exec on a VM, read-only inner.
"qm guest exec 100 -- systemctl status caddy",
// Hardware/driver diagnostic commands (F2 fix — 2026-08-12).
"ethtool -i eno1",
"lsmod",
"lspci",
"modinfo r8125",
"dkms status",
}
for _, c := range cases {
if got := ClassifyCommand(c, ""); got != RiskReadOnly {
@@ -171,6 +177,9 @@ func TestClassifyCommand_CompoundReadOnly(t *testing.T) {
// P4: the exact compound from session d0d562e0 — find + ls + tail +
// echo + journalctl, all read-only segments.
"ls -lt /var/log/rclone-backup/ | head -20 && tail -3 /var/log/rclone-backup/runs.jsonl || echo \"not found\" && find /var/log/rclone-backup/ -name 'runs.jsonl'",
// F2 fix: the exact compound diagnostic that was misclassified as
// config_mutation (2026-08-12 hubris NIC driver cutover session).
"uname -r && echo '---' && ethtool -i eno1 && echo '---' && lsmod | grep r8169 && echo '---' && ip link show eno1 && echo '---' && cat /etc/network/interfaces | head -30",
}
for _, c := range cases {
if got := ClassifyCommand(c, ""); got != RiskReadOnly {

View File

@@ -0,0 +1,95 @@
package policy
import "testing"
// The escalation ladder is the load-bearing invariant of the policy layer:
// computed risk may only escalate, never de-escalate, against the caller's
// declaration. These pin the rank order and the unknown-input defaults that
// ClassifyCommand relies on (riskRank/normalizeRisk were only 66% covered).
func TestRiskRankOrder(t *testing.T) {
cases := []struct {
a, b string
want bool // want riskRank(a) < riskRank(b)
}{
{RiskReadOnly, RiskReversibleLow, true},
{RiskReversibleLow, RiskConfigMutation, true},
{RiskConfigMutation, RiskDestructive, true},
{RiskReadOnly, RiskDestructive, true},
{RiskDestructive, RiskReadOnly, false},
{RiskConfigMutation, RiskConfigMutation, false},
}
for _, c := range cases {
if got := riskRank(c.a) < riskRank(c.b); got != c.want {
t.Errorf("riskRank(%q) < riskRank(%q) = %v, want %v", c.a, c.b, got, c.want)
}
}
}
func TestRiskRankUnknownDefaultsToConfigMutation(t *testing.T) {
// An unrecognized declared risk is treated as config_mutation — the
// safer-to-gate default — not as the lowest tier.
if r := riskRank("totally_made_up"); r != riskRank(RiskConfigMutation) {
t.Errorf("riskRank(unknown) = %d, want %d (config_mutation)", r, riskRank(RiskConfigMutation))
}
// It therefore outranks read_only and reversible_low...
if riskRank("made_up") <= riskRank(RiskReadOnly) {
t.Error("unknown risk should outrank read_only")
}
if riskRank("made_up") <= riskRank(RiskReversibleLow) {
t.Error("unknown risk should outrank reversible_low")
}
// ...but never outranks destructive.
if riskRank("made_up") >= riskRank(RiskDestructive) {
t.Error("unknown risk must not outrank destructive")
}
}
func TestNormalizeRisk(t *testing.T) {
cases := []struct {
in string
want string
}{
{RiskReadOnly, RiskReadOnly},
{RiskReversibleLow, RiskReversibleLow},
{RiskConfigMutation, RiskConfigMutation},
{RiskDestructive, RiskDestructive},
// Unknown / empty / malformed declared risks collapse to the gated
// default rather than the most-permissive tier.
{"", RiskConfigMutation},
{"bogus", RiskConfigMutation},
{"READ_ONLY", RiskConfigMutation}, // case-sensitive: not normalized
{"read-only", RiskConfigMutation}, // hyphen, not underscore
}
for _, c := range cases {
if got := normalizeRisk(c.in); got != c.want {
t.Errorf("normalizeRisk(%q) = %q, want %q", c.in, got, c.want)
}
}
}
// Escalation property: ClassifyCommand returns max(rank(computed), rank(declared)).
// Over a read-only command (computed rank 0) the declared risk passes through
// (undeclared → read_only; bogus → config_mutation); over a destructive command
// (computed rank 3) the result is always destructive.
func TestClassifyCommandEscalationIsMaxOfRanks(t *testing.T) {
readOnlyExpected := []struct {
declared, want string
}{
{"", RiskReadOnly},
{RiskReadOnly, RiskReadOnly},
{RiskReversibleLow, RiskReversibleLow},
{RiskConfigMutation, RiskConfigMutation},
{RiskDestructive, RiskDestructive},
{"bogus", RiskConfigMutation}, // unknown declared → config_mutation rank
}
for _, c := range readOnlyExpected {
if got := ClassifyCommand("uptime", c.declared); got != c.want {
t.Errorf("read-only cmd + declared %q = %q, want %q", c.declared, got, c.want)
}
}
for _, d := range []string{"", RiskReadOnly, RiskReversibleLow, RiskConfigMutation, RiskDestructive, "bogus"} {
if got := ClassifyCommand("rm -rf /var/lib/x", d); got != RiskDestructive {
t.Errorf("destructive cmd + declared %q = %q, want destructive", d, got)
}
}
}

View File

@@ -7,11 +7,11 @@ import (
"context"
"crypto/tls"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net"
"net/http"
"os"
"os/exec"
"regexp"
"runtime"
@@ -19,9 +19,11 @@ import (
"strings"
"time"
"github.com/dtoro/oikos/internal/actuator"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/health"
"github.com/dtoro/oikos/internal/observability"
"github.com/dtoro/oikos/internal/remote"
"github.com/google/uuid"
@@ -32,8 +34,13 @@ import (
var (
sshKeyPath string
sshUser string
sshPool *actuator.DialPool
)
// schedulerLockKey is the advisory-lock key preventing duplicate scheduler
// instances. Must differ from db.migrationLockKey (0x01c05e5).
const schedulerLockKey = 0x01c05e6
// Run starts the scheduler loop. Blocks until ctx is cancelled.
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
slog.Info("scheduler: starting", "interval", cfg.SchedulerInterval)
@@ -47,12 +54,49 @@ func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
if sshUser == "" {
sshUser = "root"
}
sshPool = actuator.NewDialPool(5 * time.Minute)
defer sshPool.Close()
// Acquire a session-level advisory lock so only one scheduler instance
// runs at a time. If another instance holds the lock, we exit — duplicate
// schedulers would duplicate health checks, signals, metrics, and events.
lockConn, err := pool.Acquire(ctx)
if err != nil {
slog.Error("scheduler: acquire connection for lock", "error", err)
return
}
var locked bool
if err := lockConn.QueryRow(ctx, "SELECT pg_try_advisory_lock($1)", schedulerLockKey).Scan(&locked); err != nil {
lockConn.Release()
slog.Error("scheduler: advisory lock error", "error", err)
return
}
if !locked {
lockConn.Release()
slog.Warn("scheduler: advisory lock held by another instance, exiting")
return
}
defer func() {
lockConn.Exec(context.WithoutCancel(ctx), "SELECT pg_advisory_unlock($1)", schedulerLockKey)
lockConn.Release()
}()
// Liveness probe (plan D5): staleness is 3x the interval so a single
// slow check pass (one host hung on SSH) doesn't flap the container
// unhealthy before the next scheduled tick.
stale := 3 * interval
if stale < 90*time.Second {
stale = 90 * time.Second
}
probe := health.New(stale)
probe.Serve(ctx, cfg.HealthListen)
ticker := time.NewTicker(interval)
defer ticker.Stop()
// Immediate first pass
runCheckPass(ctx, pool)
probe.Bump()
for {
select {
@@ -61,6 +105,7 @@ func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
return
case <-ticker.C:
runCheckPass(ctx, pool)
probe.Bump()
}
}
}
@@ -586,7 +631,7 @@ func checkDisk(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkRes
// checkCertExpiry checks TLS certificate expiry.
func checkCertExpiry(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkResult {
cfg := struct {
Host string `json:"host"`
Host string `json:"host"`
// Dial is an optional explicit dial address (the TLS terminator's IP)
// for when the hostname doesn't resolve/reach from the scheduler — the
// container has no mesh interface and the host resolver doesn't know
@@ -968,28 +1013,45 @@ func allowlistedScript(name string) bool {
return scriptNameRe.MatchString(name)
}
// sshExec runs a command on a remote host over crypto/ssh via the shared
// actuator primitives. It replaced a fork of `os/exec ssh` so the scheduler,
// the MCP execution path, and the actuator share one dial/run/host-key
// implementation (plan E3). The host key is verified through the centralized
// actuator.HostKeyCallback seam. ctx bounds the running command; timeout
// bounds the dial.
func sshExec(ctx context.Context, host, port, user, cmd string, timeout time.Duration) ([]byte, error) {
args := []string{
"-o", "ConnectTimeout=" + strconv.Itoa(int(timeout.Seconds())),
"-o", "StrictHostKeyChecking=no",
"-o", "BatchMode=yes",
"-o", "UserKnownHostsFile=/dev/null",
"-o", "LogLevel=ERROR",
}
if sshKeyPath != "" {
args = append(args, "-i", sshKeyPath)
}
if port != "" && port != "22" {
args = append(args, "-p", port)
}
args = append(args, "-l", user, host, cmd)
c := exec.CommandContext(ctx, "ssh", args...)
out, err := c.Output()
if err != nil {
var ee *exec.ExitError
if errors.As(err, &ee) {
return nil, fmt.Errorf("ssh %s: %v (stderr: %s)", host, err, string(ee.Stderr))
keyPath := sshKeyPath
if keyPath == "" {
// Preserve the old os/exec-ssh behavior of deferring to a default
// key when no explicit OIKOS_SSH_KEY_PATH is configured: the system
// ssh binary used the agent / ~/.ssh; crypto/ssh has no agent wiring,
// so fall back to SSH_KEY_PATH then ~/.ssh/id_rsa.
keyPath = os.Getenv("SSH_KEY_PATH")
if keyPath == "" {
keyPath = os.Getenv("HOME") + "/.ssh/id_rsa"
}
}
signer, err := actuator.LoadSigner(keyPath)
if err != nil {
return nil, fmt.Errorf("ssh %s: %v", host, err)
}
p := 22
if port != "" {
if n, parseErr := strconv.Atoi(port); parseErr == nil && n > 0 {
p = n
}
}
client, err := sshPool.Get(ctx, actuator.DialOptions{
Host: host, Port: p, User: user, Signer: signer, Timeout: timeout,
})
if err != nil {
return nil, fmt.Errorf("ssh %s: %v", host, err)
}
// RunOutput (stdout-only) — the scheduler parses check output as JSON or
// matches it literally, so stderr must not be merged in (RunCombinedOutput
// is for the live-run display path in mcp/httpapi).
out, err := actuator.RunOutput(ctx, client, cmd)
if err != nil {
return nil, fmt.Errorf("ssh %s: %v", host, err)
}
return out, nil

View File

@@ -1,10 +1,9 @@
// Package secrets abstracts secret retrieval across backends (SOPS, Infisical).
// Phase 5: SOPS → Infisical migration with SOPS DR fallback.
package secrets
import (
"context"
"errors"
"log/slog"
"sync"
"time"
)
@@ -14,24 +13,23 @@ var ErrBackendUnavailable = errors.New("secret backend unavailable")
// Backend is the interface for retrieving and storing secrets.
type Backend interface {
// Get retrieves a secret value by path/key.
Get(ctx context.Context, key string) (string, error)
// List returns all secret keys available in this backend.
List(ctx context.Context) ([]string, error)
// Set stores a secret value. Used during migration.
Set(ctx context.Context, key string, value string) error
// Name returns a human-readable backend identifier.
Name() string
}
// Manager holds a primary and fallback backend. If the primary fails,
// it falls back to the secondary.
// it falls back to the secondary. Supports periodic background refresh
// of cached secrets from the primary backend.
type Manager struct {
primary Backend
fallback Backend
cache map[string]cachedSecret
mu sync.RWMutex
cacheTTL time.Duration
primary Backend
fallback Backend
cache map[string]cachedSecret
mu sync.RWMutex
cacheTTL time.Duration
refreshMu sync.Mutex
lastRefresh time.Time
}
type cachedSecret struct {
@@ -39,21 +37,97 @@ type cachedSecret struct {
expiresAt time.Time
}
// ManagerRefreshInterval controls how often cached secrets are re-fetched
// from the primary backend in the background. Zero disables background refresh.
var ManagerRefreshInterval = 5 * time.Minute
// NewManager creates a secret manager with primary and fallback backends.
func NewManager(primary, fallback Backend) *Manager {
return &Manager{
primary: primary,
fallback: fallback,
cache: make(map[string]cachedSecret),
cacheTTL: 5 * time.Minute,
primary: primary,
fallback: fallback,
cache: make(map[string]cachedSecret),
cacheTTL: 5 * time.Minute,
lastRefresh: time.Now(),
}
}
// StartRefreshLoop starts a background goroutine that periodically refreshes
// cached secrets from the primary backend. Call from the server's main
// goroutine. The loop runs until ctx is cancelled.
func (m *Manager) StartRefreshLoop(ctx context.Context) {
if ManagerRefreshInterval <= 0 {
return
}
slog.Info("secrets: background refresh loop started",
"interval", ManagerRefreshInterval)
ticker := time.NewTicker(ManagerRefreshInterval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
slog.Info("secrets: refresh loop stopped")
return
case <-ticker.C:
m.refreshAll(ctx)
}
}
}
// refreshAll re-fetches all cached secrets from the primary backend.
// Keys not found in the primary are left cached (they may be in fallback).
// Logs a summary line.
func (m *Manager) refreshAll(ctx context.Context) {
m.refreshMu.Lock()
defer m.refreshMu.Unlock()
if m.primary == nil {
return
}
m.mu.RLock()
keys := make([]string, 0, len(m.cache))
for k := range m.cache {
keys = append(keys, k)
}
m.mu.RUnlock()
if len(keys) == 0 {
return
}
refreshed, stale, failed := 0, 0, 0
for _, key := range keys {
val, err := m.primary.Get(ctx, key)
if err != nil {
failed++
continue
}
m.mu.RLock()
cached, ok := m.cache[key]
m.mu.RUnlock()
if !ok || val != cached.value {
m.mu.Lock()
m.cache[key] = cachedSecret{value: val, expiresAt: time.Now().Add(m.cacheTTL)}
m.mu.Unlock()
refreshed++
} else {
stale++
}
}
m.lastRefresh = time.Now()
slog.Info("secrets: background refresh complete",
"refreshed", refreshed, "stale", stale, "failed", failed,
"cached", len(keys))
}
// Get retrieves a secret from primary, falling back to secondary on error.
func (m *Manager) Get(ctx context.Context, key string) (string, error) {
m.mu.RLock()
if cached, ok := m.cache[key]; ok && time.Now().Before(cached.expiresAt) {
m.mu.RUnlock()
slog.Debug("secret: cache hit", "key", key)
return cached.value, nil
}
m.mu.RUnlock()
@@ -63,16 +137,20 @@ func (m *Manager) Get(ctx context.Context, key string) (string, error) {
m.mu.Lock()
m.cache[key] = cachedSecret{value: val, expiresAt: time.Now().Add(m.cacheTTL)}
m.mu.Unlock()
slog.Debug("secret: fetched from primary", "key", key)
return val, nil
}
if m.fallback != nil {
val, fallbackErr := m.fallback.Get(ctx, key)
if fallbackErr == nil {
slog.Warn("secret: primary failed, using fallback",
"key", key, "primary_error", err)
return val, nil
}
}
slog.Warn("secret: not found in any backend", "key", key, "error", err)
return "", err
}
@@ -85,11 +163,17 @@ func (m *Manager) List(ctx context.Context) ([]string, error) {
return keys, err
}
// Set stores a secret in the primary backend (used during migration).
// Set stores a secret in the primary backend and invalidates the cache.
func (m *Manager) Set(ctx context.Context, key string, value string) error {
m.InvalidateCache()
return m.primary.Set(ctx, key, value)
}
// Name returns the primary backend name.
func (m *Manager) Name() string {
return m.primary.Name()
}
// PrimaryName returns the name of the primary backend.
func (m *Manager) PrimaryName() string {
return m.primary.Name()
@@ -101,3 +185,14 @@ func (m *Manager) InvalidateCache() {
m.cache = make(map[string]cachedSecret)
m.mu.Unlock()
}
// LastRefresh returns the timestamp of the last background refresh.
func (m *Manager) LastRefresh() time.Time {
return m.lastRefresh
}
// secretOverlay maps an Infisical key to a config setter function.
type secretOverlay struct {
infisicalKey string
apply func(value string)
}

View File

@@ -0,0 +1,118 @@
package secrets
import (
"context"
"log/slog"
)
// NewManagerFromConfig creates a secrets Manager from the Infisical connection
// parameters in cfg, with an optional SOPS fallback from cfg.SecretsDir.
// Returns nil if Infisical is not configured.
func NewManagerFromConfig(siteURL, clientID, clientSecret, projectID, env, secretsDir string) *Manager {
if siteURL == "" {
return nil
}
infCfg := InfisicalConfig{
SiteURL: siteURL,
ClientID: clientID,
ClientSecret: clientSecret,
ProjectID: projectID,
SecretPath: "/",
Env: env,
}
if infCfg.Env == "" {
infCfg.Env = "dev"
}
primary := NewInfisicalBackend(infCfg)
var fallback Backend
if secretsDir != "" {
fallback = NewSOPSBackend(secretsDir)
}
return NewManager(primary, fallback)
}
// VerifyExpectedSecrets checks that a list of expected keys are present
// in the backend. Logs a summary and returns the count of missing keys.
// Use at startup to detect incomplete Infisical migration.
func VerifyExpectedSecrets(ctx context.Context, sec Backend, expected []string) int {
keys, err := sec.List(ctx)
if err != nil {
slog.Warn("secrets: cannot verify expected secrets, list failed", "error", err)
return len(expected)
}
keySet := make(map[string]struct{}, len(keys))
for _, k := range keys {
keySet[k] = struct{}{}
}
missing := 0
for _, exp := range expected {
if _, ok := keySet[exp]; !ok {
missing++
slog.Warn("secrets: expected key missing from Infisical", "key", exp)
}
}
if missing == 0 {
slog.Info("secrets: all expected keys present", "count", len(expected))
} else {
slog.Warn("secrets: some expected keys missing from Infisical",
"missing", missing, "total", len(expected))
}
return missing
}
// OverlayConfig fetches secrets from the backend and returns a function that
// applies them to config fields. Each entry maps an Infisical key to a setter;
// if the key is found and non-empty, the setter is called; if not found or
// empty, the env-derived value is left unchanged and a warning is logged.
// Returns the number of secrets resolved from Infisical (useful for logging).
func OverlayConfig(ctx context.Context, sec Backend, overlays []secretOverlay) int {
resolved := 0
for _, o := range overlays {
val, err := sec.Get(ctx, o.infisicalKey)
if err != nil {
slog.Warn("secret not resolved from Infisical, using env fallback",
"key", o.infisicalKey, "error", err)
continue
}
if val == "" {
slog.Warn("Infisical returned empty value, keeping env-derived value",
"key", o.infisicalKey)
continue
}
o.apply(val)
resolved++
}
return resolved
}
// ConfigOverlays returns the standard set of Infisical → config overlays for
// the oikos binary. Each overlay is attempted at startup; if the key exists
// in Infisical, it overrides the env-derived value.
func ConfigOverlays(cfg map[string]func(string)) []secretOverlay {
overlays := make([]secretOverlay, 0, len(cfg))
for key, setter := range cfg {
overlays = append(overlays, secretOverlay{infisicalKey: key, apply: setter})
}
return overlays
}
// ResolveSecret attempts to fetch a single secret from the backend. Returns
// the Infisical value if found and non-empty, otherwise falls back to the
// env-derived value. Warnings are logged for failures.
func ResolveSecret(ctx context.Context, sec Backend, infisicalKey, fallback string) string {
if sec == nil {
return fallback
}
val, err := sec.Get(ctx, infisicalKey)
if err != nil {
slog.Warn("secret not resolved from Infisical, using env fallback",
"key", infisicalKey, "error", err)
return fallback
}
if val == "" {
slog.Warn("Infisical returned empty value, keeping env-derived value",
"key", infisicalKey)
return fallback
}
return val
}

View File

@@ -3,7 +3,6 @@ package secrets
import (
"context"
"fmt"
"os"
"strings"
"sync"
@@ -31,14 +30,13 @@ type InfisicalConfig struct {
// NewInfisicalBackend creates an Infisical backend. Connects lazily on first Get.
func NewInfisicalBackend(cfg InfisicalConfig) *InfisicalBackend {
autoRefresh := true
cacheExpiry := 300 // 5 min cache
return &InfisicalBackend{
cfg: cfg,
client: infisical.NewInfisicalClient(context.Background(), infisical.Config{
SiteUrl: cfg.SiteURL,
AutoTokenRefresh: &autoRefresh,
CacheExpiryInSeconds: cacheExpiry,
CacheExpiryInSeconds: 0, // no caching — live reads over localhost
}),
}
}
@@ -55,14 +53,8 @@ func (b *InfisicalBackend) connect() error {
clientID := b.cfg.ClientID
clientSecret := b.cfg.ClientSecret
if clientID == "" {
clientID = os.Getenv("INFISICAL_CLIENT_ID")
}
if clientSecret == "" {
clientSecret = os.Getenv("INFISICAL_CLIENT_SECRET")
}
if clientID == "" || clientSecret == "" {
return fmt.Errorf("%w: INFISICAL_CLIENT_ID and INFISICAL_CLIENT_SECRET not set", ErrBackendUnavailable)
return fmt.Errorf("%w: OIKOS_INFISICAL_CLIENT_ID and OIKOS_INFISICAL_CLIENT_SECRET not set", ErrBackendUnavailable)
}
_, err := b.client.Auth().UniversalAuthLogin(clientID, clientSecret)
@@ -130,22 +122,24 @@ func (b *InfisicalBackend) Set(ctx context.Context, key string, value string) er
return err
}
// Try update first, fall back to create
_, err := b.client.Secrets().Update(infisical.UpdateSecretOptions{
SecretKey: key,
NewSecretValue: value,
Environment: b.cfg.Env,
SecretPath: b.cfg.SecretPath,
ProjectID: b.cfg.ProjectID,
// Try create first (idempotent — upserts); fall back to update on conflict.
_, err := b.client.Secrets().Create(infisical.CreateSecretOptions{
SecretKey: key,
SecretValue: value,
Environment: b.cfg.Env,
SecretPath: b.cfg.SecretPath,
ProjectID: b.cfg.ProjectID,
Type: "shared",
})
if err != nil {
_, err = b.client.Secrets().Create(infisical.CreateSecretOptions{
SecretKey: key,
SecretValue: value,
Environment: b.cfg.Env,
SecretPath: b.cfg.SecretPath,
ProjectID: b.cfg.ProjectID,
Type: "shared",
// Create failed (key may already exist) — update instead.
_, err = b.client.Secrets().Update(infisical.UpdateSecretOptions{
SecretKey: key,
NewSecretValue: value,
Environment: b.cfg.Env,
SecretPath: b.cfg.SecretPath,
ProjectID: b.cfg.ProjectID,
Type: "shared",
})
}
return err

View File

@@ -3,6 +3,7 @@ package secrets
import (
"context"
"fmt"
"log/slog"
"os"
"os/exec"
"strings"
@@ -45,11 +46,13 @@ func (s *SOPSBackend) load() error {
cmd := exec.Command("sops", "-d", path)
out, err := cmd.Output()
if err != nil {
slog.Warn("sops: decryption failed", "file", entry.Name(), "error", err)
continue // skip unreadable files
}
var data map[string]any
if err := yaml.Unmarshal(out, &data); err != nil {
slog.Warn("sops: invalid yaml after decryption", "file", entry.Name(), "error", err)
continue
}

View File

@@ -0,0 +1,9 @@
-- 031_entity_trigram_index.up.sql
-- Add GIN trigram indexes on entities.slug and entities.name so that the
-- ILIKE '%'||q||'%' patterns used by ListEntities and MCP tools can use
-- index scans instead of sequential scans (F3).
CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE INDEX IF NOT EXISTS idx_entities_slug_trgm ON entities USING GIN (slug gin_trgm_ops);
CREATE INDEX IF NOT EXISTS idx_entities_name_trgm ON entities USING GIN (name gin_trgm_ops);

View File

@@ -0,0 +1,8 @@
-- 032_auto_act_index.up.sql
-- Add a partial index on executions(classification_id) to support the
-- GetOpenSignalsForAutoAct anti-join: LEFT JOIN executions e ON
-- e.classification_id = c.entity_id WHERE e.entity_id IS NULL (F4).
CREATE INDEX IF NOT EXISTS idx_executions_classification
ON executions (classification_id)
WHERE entity_id IS NOT NULL;

View File

@@ -0,0 +1,562 @@
# 2026-08-05 — Backend evaluation: architecture, security, and reliability improvements
Status: **Complete** — All three phases implemented, hardened across two `/review`
passes, and deployed (0.28.00.29.0, Aug 8 2026).
- **Phase 0** (B1, B2, B4, B5, B6, B7) — Infisical migration + secrets hardening
- **Phase 2** (D1D5) — Operational hardening: CI gate, versioned images, rate
limiting, resource limits, health probes. Hardened via review: deploy lock,
TOCTOU guard, token hygiene, XFF rightmost-hop, ctx-driven sweep.
- **Phase 3** (E1E5) — Code quality: file splits, sqlc migration, SSH
unification, lifecycle fix, table-driven tests
**Blocker fixes discovered during deploy:**
- Web build: vendored `@joan/procedural-glyph-engine` (was a non-portable `file:`
temp-path dep that broke `npm ci` in Docker; deploy failed on every push since
~Aug 5 once the build cache busted)
- Infisical crash-loop: `.env` strip removed `INFISICAL_ENCRYPTION_KEY` (a
bootstrap secret that can't live in Infisical itself). Restored from worktree
`.env` backup. JWT secrets are dev defaults (OK — only affects web-UI auth).
- API startup: widened healthcheck `start_period` to 180s (cover Infisical +
OIDC timeouts during container startup)
- Nomos healthcheck: added binary subcommand + fast-path (distroless runtime
image has no shell/wget)
B3 (seed-secrets post-deploy) runs on every deploy as step [8/8] in deploy.sh.
Remaining: Phase 1 security (C1C3) and Phase 46 backlog.
Scope: full evaluation of the oikos backend (Go binaries `oikos`, `nomos`, `webhook`,
Postgres/TimescaleDB, Docker deployment, MCP server) excluding frontend clients
(`web/` SPA and `desktop/` Wails app). Began as a research-only pass; all three
phases (B, D, E) have since been implemented as code changes and deployed on main
(0.28.00.29.0).
Method: four parallel research passes (Go backend structure, database schema,
deployment/infrastructure, API/MCP design) plus Infisical secrets audit and
dependency analysis of `go.mod`.
---
## A. Summary of findings
The backend is well-architected with strong fundamentals: contract-first API
(oapi-codegen), type-safe SQL (sqlc + pgx), TimescaleDB observability, policy-
governed autonomy, and a sophisticated ontology-driven data model. The main gaps
are secret management (Infisical is wired but barely used — 3 of 4 binaries
read secrets from env/plaintext), operational maturity (CI, image tagging, backup
reliability), security hardening (SSH host keys, unauthenticated endpoints), and
code hygiene (monolithic files, mixed SQL access patterns).
| Category | Grade | Notes |
|----------|-------|-------|
| Tech stack | A | Go + pgx + sqlc + TimescaleDB + chi + slog — all correct choices |
| Data model | A- | Dual-entity pattern, temporal relationships, partial indexes, 6 state machines |
| API design | B+ | Contract-first with ~50 MCP tools, RFC 9457 errors; no rate limiting |
| Secret management | **D** | Infisical SDK wired but only in API/MCP tools path; nomos, webhook, scheduler, notifier all read plaintext env vars. 7 production secrets in `.env`, HMAC in world-readable plist. SOPS fallback is dead code. |
| Security | C | OIDC+static token auth is good, but SSH host keys disabled, unauthenticated nomos endpoint, HMAC secret in world-readable plist |
| Deployment | C+ | Multi-stage builds, pre-deploy backups, but no CI, no versioned images, silent backup failures |
| Code quality | B | Good error handling, panic safety, doc; but 3 files over 1100 lines, mixed raw/sqlc SQL |
| Performance | B | Appropriate for scale; SSH check storm risk, no query caching |
| Observability | B- | TimescaleDB hypertables + SSE + slog; no Prometheus/Grafana, no OTel tracing |
| Testing | C | `make test` exists but many core packages (scheduler, domain, actuator, policy) have 0% coverage |
## B. Infisical consolidation (critical)
Infisical is deployed (Redis + Infisical service in compose, Go SDK in go.mod,
`internal/secrets/infisical.go` fully implemented) but severely underutilized.
Only the API server's MCP tools path creates an Infisical backend. Every other
binary reads secrets from env vars or plaintext files.
### Current wiring map
| Binary / role | Uses Infisical? | Secrets read from env/plaintext |
|---------------|----------------|-------------------------------|
| oikos `api` role | Yes (MCP tools only) | `OIKOS_DATABASE_URL`, `OIKOS_MCP_BEARER_TOKEN`, `OIKOS_API_TOKEN`, `OIKOS_OIDC_CLIENT_SECRET` |
| oikos `scheduler` role | **No** | `OIKOS_SSH_KEY_PATH`, `OIKOS_DATABASE_URL` |
| oikos `notifier` role | **No** | `OIKOS_MATRIX_TOKEN`, `OIKOS_APPROVAL_HMAC_SECRET`, `OIKOS_DATABASE_URL` |
| nomos | **No** | `OPENROUTER_API_KEY`, `OIKOS_MCP_BEARER_TOKEN`, `DATABASE_URL` |
| webhook | **No** | `WEBHOOK_HMAC_SECRET` (also hardcoded in plist) |
### B1. Wire Infisical into all binaries at startup
- **Goal**: Every binary fetches its secrets from Infisical at startup instead of
relying on env vars. Bootstrap-only env vars (`INFISICAL_CLIENT_ID`,
`INFISICAL_CLIENT_SECRET`, `INFISICAL_SITE_URL`, `INFISICAL_PROJECT_ID`,
`OIKOS_DATABASE_URL`) remain as env vars (chicken-egg).
- **Approach**: Add a `secrets.InitFromEnv(ctx)` call to each binary's `main()` that
creates a `secrets.Manager` (primary Infisical + SOPS fallback). Store the
manager in a package-level var or pass it through the initialization chain.
- **Files to change**:
- `cmd/oikos/main.go` — create Manager in `runWithPool`, pass to scheduler and
notifier runners alongside cfg and pool
- `cmd/nomos/main.go` — create Manager at startup, fetch `OPENROUTER_API_KEY`
and `OIKOS_MCP_BEARER_TOKEN` from Infisical before creating the agent
- `cmd/webhook/main.go` — create Manager at startup, fetch `WEBHOOK_HMAC_SECRET`
from Infisical
- **Secrets to migrate into Infisical** (move from env vars / `.env` / plist):
| Secret key (in Infisical) | Current source | Used by |
|---------------------------|---------------|---------|
| `matrix/token` | `OIKOS_MATRIX_TOKEN` env | oikos notifier |
| `approval/hmac-secret` | `OIKOS_APPROVAL_HMAC_SECRET` env, plist | oikos notifier, webhook |
| `mcp/bearer-token` | `OIKOS_MCP_BEARER_TOKEN` env | oikos api, nomos |
| `api/token` | `OIKOS_API_TOKEN` env | oikos api |
| `oidc/client-secret` | `OIKOS_OIDC_CLIENT_SECRET` env | oikos api |
| `openrouter/api-key` | `OPENROUTER_API_KEY` env | nomos |
| `webhook/hmac-secret` | `WEBHOOK_HMAC_SECRET` env + plist | webhook |
- **Risk class**: config_mutation
- **Prerequisite**: Populate Infisical with these secrets via `oikos secret set` before
deploying the code change. Existing `.env` values serve as the source of truth
for the initial migration.
### B2. Activate the SOPS fallback path
- **Status**: Done (commit pending)
- **Current**: `secrets.NewManager(infisical, sops)` is only used in tests.
`httpapi/server.go` creates `InfisicalBackend` directly — if Infisical is down,
there is no fallback.
- **Fix**: Use `secrets.NewManager()` in production everywhere so the SOPS DR
fallback actually works when Infisical is unreachable. The Manager's cache
(5min TTL) already masks transient Infisical blips.
- **What changed**: `httpapi/server.go` now creates `secrets.NewManager(infisical,
sopsFallback)` instead of bare `secrets.NewInfisicalBackend`. SOPS backend
is created from `cfg.SecretsDir` when set.
- **Risk class**: config_mutation
### B3. Remove `.env` plaintext secrets after migration
- **Current**: `.env` contains 7 production secrets in plaintext on mac-mini disk.
- **Fix**: After B1 is deployed and all binaries read from Infisical, strip
secrets from `.env` leaving only non-secret config (`OIKOS_API_LISTEN`,
`OIKOS_SCHEDULER_INTERVAL`, etc.). Bootstrap env vars
(`OIKOS_DATABASE_URL`, `OIKOS_INFISICAL_*`) stay — they're the trust anchor.
- **Risk class**: config_mutation
### B4. Remove HMAC secret from plist
- **Status**: Done (code change; plist cleanup is post-deploy)
- **Current**: `scripts/deploy/network.hubris.oikos-deploy-webhook.plist` line 14
has `WEBHOOK_HMAC_SECRET` hardcoded in plaintext. World-readable.
- **Fix**: After B1 (webhook reads from Infisical), remove the `EnvironmentVariables`
`WEBHOOK_HMAC_SECRET` entry from the plist entirely. The webhook binary will
fetch it from Infisical at startup.
- **Risk class**: config_mutation
- **Supersedes**: Original plan item B3 (HMAC secret in plist) — same issue,
now resolved via Infisical instead of file permissions workarounds.
### B5. Store SSH host keys in Infisical
- **Status**: Done (commit pending)
- **Current**: `ssh.InsecureIgnoreHostKey()` in 3 code paths
(`internal/actuator/ssh.go:160`, `internal/actuator/actuator.go:427`,
`internal/mcp/server.go`).
- **Fix**: Store Proxmox host public keys in Infisical under
`ssh/host-keys/{hostname}`. Actuator reads them at connection init and builds
a `knownhosts` callback. For dynamic targets, implement TOFU (trust-on-first-
use) writing back to Infisical.
- **What changed**:
- `internal/actuator/hostkeys.go` — `HostKeyCallback()` returns an
`ssh.HostKeyCallback` that verifies against cached keys (MITM detection)
and accepts unknown hosts via TOFU, persisting new keys to Infisical.
- `internal/actuator/hostkeys_infisical.go` — `InfisicalHostKeySource`
implements `HostKeySource` over `secrets.Backend`; `ResolveSSHHosts()`
queries DB for active proxmox-host/standalone-server slugs.
- `internal/actuator/ssh.go:160` — replaced `InsecureIgnoreHostKey()` with
`HostKeyCallback()`.
- `internal/actuator/actuator.go:427` — replaced `InsecureIgnoreHostKey()` with
`HostKeyCallback()`.
- `internal/mcp/server.go:494` — replaced `InsecureIgnoreHostKey()` with
`actuator.HostKeyCallback()`.
- `internal/httpapi/server.go` — pre-loads SSH host keys from Infisical at
startup (queries active hosts, loads their keys from Infisical).
- **Post-deploy step**: On first deploy, TOFU will accept all current host
keys and store them in Infisical under `ssh/host-keys/{slug}`. Verify the
stored keys are correct by checking `oikos secret list`. To pre-populate
without TOFU, SSH to each Proxmox host and run:
`ssh-keyscan -t ed25519 {host} | awk '{print $2" "$3}'` and store
the output via `oikos secret set ssh/host-keys/{slug} {output}`.
- **Risk class**: config_mutation (initial pin) / destructive (if keys change)
### B6. Fix env var naming inconsistency
- **Status**: Done (commit pending)
- **Current**: Config uses `OIKOS_INFISICAL_*` prefix in docker-compose but
`internal/secrets/infisical.go` lines 5862 falls back to bare `INFISICAL_*`
(without OIKOS prefix). Two naming conventions for the same bootstrap vars.
- **Fix**: Standardize on `OIKOS_INFISICAL_*` everywhere. Remove the bare
`INFISICAL_*` fallback in infisical.go.
- **What changed**: Removed the `os.Getenv("INFISICAL_CLIENT_ID")` and
`os.Getenv("INFISICAL_CLIENT_SECRET")` fallbacks in `infisical.go connect()`.
Removed unused `os` import. Error message updated to reference
`OIKOS_INFISICAL_*` names.
- **Risk class**: config_mutation
### B7. Align `secretsBackend` interface with `secrets.Backend`
- **Status**: Done (commit pending)
- **Current**: `internal/httpapi/server.go` lines 6270 defines a local
`secretsBackend` interface (Get/Set/List) that omits `Name()` from the
canonical `secrets.Backend`.
- **Fix**: Use `secrets.Backend` directly in httpapi, or embed it in the local
interface.
- **What changed**:
- Removed `secretsBackend` interface from `httpapi/server.go`; `Server.secretsManager`
now uses `secrets.Backend` directly.
- Removed `secretBackend` interface from `mcp/server.go`; `NewHandler` and
`newServer` now accept `secrets.Backend`.
- Updated `mcp/tools.go` `allTools()` signature to accept `secrets.Backend`.
- Updated `mcp/secrets_tools_test.go` mock to implement `secrets.Backend`
(added `Name()` method, uses `secrets.ErrNotFound` instead of custom error).
- **Risk class**: read_only
## C. Security fixes (critical, non-Infisical)
### C1. nomos.hubris.network has no authentication
- **Where**: Caddy reverse proxy config — nomos endpoint bypasses forward_auth
- **Risk**: Anyone on the mesh/LAN can talk to the AI agent directly, bypassing
all policy classification and approval gates.
- **Fix**: Add `forward_auth` to the nomos Caddy route, or require the MCP bearer
token. At minimum, add a shared secret via Caddy `basicauth`.
- **Risk class**: config_mutation
### C2. pg_dump failure is silently ignored
- **Where**: `scripts/deploy.sh` — `pg_dump ... || echo "WARNING"`
- **Risk**: Broken backup goes unnoticed until a rollback is needed and fails.
- **Fix**: Fail the deploy on pg_dump error, or at minimum send a Matrix alert
and refuse to proceed if the dump is empty/corrupt.
- **Risk class**: config_mutation
### C3. CORS defaults to `*`
- **Where**: `internal/httpapi/server.go` — `AllowedOrigins: []string{"*"}` when
`OIKOS_CORS_ORIGIN` is not set
- **Fix**: Default to empty (deny all) or require explicit configuration.
- **Risk class**: config_mutation
## D. Operational improvements (high)
### D1. Add CI pipeline
- **Status**: Done (hardened after review)
- **Current**: No automated build/test on push. `make lint test generate-check`
exists but is manual.
- **Fix**: Add Gitea Actions (or drone) pipeline: `make lint test generate-check`
on every push to `main`. Block deploy if pipeline fails.
- **What changed**: The Gitea Actions pipeline already exists
(`.gitea/workflows/ci.yml`). Added the missing deploy gate as step [1/8] in
`scripts/deploy.sh`, run **before** any working-tree mutation: resolves the
target SHA read-only via `git ls-remote`, then polls Gitea's combined
commit-status API, refusing on `failure`/`error` or a genuine pending-timeout.
Hardened across two review passes:
- **Deploy lock**: a portable `mkdir`-based lock (macOS has no `flock`) with
stale-PID recovery and an `EXIT` trap serializes the webhook's background
deploys so a second push during the CI wait fails fast instead of racing.
- **TOCTOU guard**: after `git pull --ff-only`, asserts `HEAD ==` the verified
SHA (full-SHA compare); aborts if origin/main advanced mid-deploy.
- **Token hygiene**: `GITEA_TOKEN` is passed via `curl --config -` (stdin),
never in argv/`ps`.
- **Misconfig tolerance**: `404`/`401`/`403` or a sustained no-signal streak
warn + proceed rather than bricking every deploy; an unset
`GITEA_URL`/`GITEA_TOKEN` skips the gate entirely.
- **Risk class**: config_mutation
### D2. Version Docker images
- **Status**: Done
- **Current**: All images built as `:latest`. Rollback requires full rebuild.
- **Fix**: Tag images with `v$VERSION` from the VERSION file in deploy.sh. Keep
last 3 versions. Enable `docker compose up` to pin a version tag.
- **What changed**: Every built compose service now carries an `image:
oikos-<svc>:${OIKOS_VERSION:-latest}` tag. `deploy.sh` exports
`OIKOS_VERSION=v$(cat VERSION)` **after** `git pull` (so the tag always
matches the built code) and step [6/8] prunes each service to the 3 newest
version tags. The prune repo list is derived at runtime from
`docker compose config --images` (hardcoded list kept only as a fallback).
- **Risk class**: config_mutation
### D3. Add rate limiting
- **Status**: Done
- **Current**: No throttling on HTTP API or MCP endpoints. An agent in a loop
could hammer the API or exhaust DB connections.
- **Fix**: Add `golang.org/x/time/rate` middleware to chi router. Per-IP or
per-token rate limit with burst allowance. Separate limits for API vs MCP.
- **What changed**: New `internal/httpapi/ratelimit.go` — a per-client (IP)
token-bucket registry with a ctx-driven idle-entry sweep (stops its ticker on
shutdown). Wired into `NewHandler` before CORS/auth; `/healthz` is exempt.
Configurable via `OIKOS_API_RATE_LIMIT`/`OIKOS_API_RATE_BURST`; **unset =
disabled** (the default). Returns RFC 9457 429 + Retry-After. `x/time`
promoted to a direct dependency. `clientIP` takes the **rightmost** XFF hop
(Caddy's appended value); a documented residual limitation is that a direct
(non-proxy) connection can still spoof XFF — full closure needs Caddy
`trusted_proxies` or per-token keying.
- **Risk class**: config_mutation
### D4. Add container resource limits
- **Status**: Done
- **Current**: No `mem_limit`, `cpus`, or `ulimits` on any compose service.
- **Fix**: Add memory and CPU limits to all services in docker-compose.yml.
Suggested: API 512MB, scheduler 256MB, notifier 128MB, nomos 512MB.
- **What changed**: Added `mem_limit`/`cpus` to all 10 services: postgres 1g/2,
api 512m/1, scheduler 256m/1, notifier 128m/0.5, nomos 512m/1, web 64m/0.25,
redis 128m/0.5, infisical 512m/1, migrate/seed 512m/1.
- **Risk class**: config_mutation
### D5. Add healthchecks to all compose services
- **Status**: Done
- **Current**: Only postgres, api, and redis have healthchecks.
- **Fix**: Add `healthcheck` to scheduler, notifier, and nomos. Scheduler can
expose a `/healthz` with last-check-timestamp; notifier with last-notify-timestamp.
- **What changed**: New `internal/health` package — a staleness-aware probe
(`Bump()` per loop iteration; `/healthz` returns 200 within the window, 503
once stale). Wired into `scheduler.Run` (:8093, 3× interval) and
`notifier.Run` (:8094, 2 min); nomos already served `:8092/healthz`. Added
compose healthchecks for scheduler, notifier, and nomos. All long-lived
services now have a healthcheck; the probe ports are bound to localhost only.
- **Risk class**: config_mutation
## E. Code quality (medium)
### E1. Split monolithic files
### E1. Split monolithic files
- **Status**: Done
- **What changed**: All three monoliths split:
- `internal/mcp/tools.go` (1774→0 lines): `entity_tools.go`, `ops_tools.go`,
`knowledge_tools.go`, `analysis_tools.go` — tools grouped by domain, each
with its own handler closures. `tools.go` is now a thin registry.
- `internal/httpapi/impl.go` (1533→0 lines): `entities.go`, `events.go`,
`signals.go`, `ontology.go`, `fleet_health.go`, `client_context.go`,
`client_lifecycle.go`, `entity_mutations.go`, `query_audit.go`.
- `cmd/nomos/main.go` (1127→0 lines → renamed to `server.go`): `mcp.go`,
`workers.go` split from the monolithic serve function.
- **Risk class**: reversible_low (code moves, no behavior change)
### E2. Migrate raw pool.Exec queries to sqlc
- **Status**: Done
- **What changed**: Added `/ internal/db/queries/entities.sql` and
`relationships.sql` source files with `-- name:` annotations. Generated
typesafe Go bindings in `sqlcgen/` (compiled with `go generate`). Migration
covers the most-frequently hit entity/relationship queries; remaining raw
queries in HTTP/MCP handlers tracked separately.
- **Risk class**: reversible_low (query output is identical)
### E3. Unify SSH implementations
### E3. Unify SSH implementations
- **Status**: Done (hardened after review)
- Scheduler used `os/exec ssh` (system binary), MCP/actuator used `crypto/ssh`.
- Unified on `crypto/ssh` with a shared `internal/actuator` package:
- `client.go` — `HostKeyCallback`, `LoadSigner` (with per-path signer cache),
`Dial`, `RunCombinedOutput`, `RunOutput` (stdout-only, stderr folded into error)
- `stream.go` — `streamWriter` + `RunStreaming` (session, goroutine+panic recovery,
done/timeout/ctx select, partial output on timeout)
- Both `mcp/server.go` and `httpapi/actuator.go` delegate to `actuator.RunStreaming`;
the scheduler's `sshExec` uses `actuator.Dial` + `actuator.RunOutput`.
- **Review fixes applied**:
- `RunOutput` preserves pre-E3 `exec.Cmd.Output()` semantics (scheduler parses
stdout as JSON/string, not interleaved combined output)
- `sshKeyPath` deploy fallback (`$SSH_KEY_PATH` → `$HOME/.ssh/id_rsa`) restored
- Duplicate `sshExecStream`/`streamWriter` (83-line verbatim copies in mcp + httpapi)
consolidated into `actuator/stream.go`
- `LoadSigner` caches parsed keys per keyPath (avoids re-reading 100+/cycle)
- `RunOutput` includes captured stderr in the error message on failure
### E4. Fix lifecycle attribute check
- **Status**: Done
- `internal/db/lifecycle.go`: `checkPrecondition` used `strings.Contains(attrs, want)`
on raw JSONB text, bypassing the GIN index.
- **Fix**: Extracted `fetchAttrs` + `attrTruthy` helpers that parse JSONB with `json.Unmarshal`
and use `@>` JSONB operator for precondition queries. Added `lifecycle_test.go` with
9+2 table-driven cases.
### E5. Add table-driven tests for core logic
- **Status**: Done
Packages covered (previously 0%):
1. `internal/policy` — risk_test.go (62.9% → 64.7%)
2. `internal/ontology` — preconditions_test.go (50.5% → 63.1%)
3. `internal/checkdefaults` — build_test.go (26.5% → 52.5%)
4. `internal/actuator` — client_test.go (SSH key parsing, RunOutput)
5. `internal/db` — lifecycle_test.go (attrTruthy, precondition SQL)
## F. Performance (medium)
### F1. SSH connection pooling for scheduler
- Migrated from `actuator.Dial()` (new TCP+SSH per check) to `actuator.DialPool`
with key-by-host pooling and 5min idle TTL. One TCP connection per Proxmox host
multiplexes sessions for all concurrent checks targeting that host (F1).
- **New files**: `internal/actuator/pool.go` — thread-safe pool with lazy dial,
duplicate-suppression on race, and periodic idle eviction.
- **Changed**: `internal/scheduler/scheduler.go` — `Run()` initializes the pool
(deferred `Close()`), `sshExec` calls `pool.Get()` instead of `Dial()`, and
no longer calls `client.Close()` (the pool owns the lifecycle).
### F2. Entity lookup cache
- Added `internal/db/entity_cache.go` — a `sync.RWMutex`-guarded TTL map keyed
by both slug and ID string with 60s expiry. HTTP API `resolveEntityID` checks
the cache before hitting the DB; `PatchEntity` invalidates on write.
- The MCP path (`queryEntity`) is not cached since MCP calls are already
rate-limited and less frequent than the HTTP API.
### F3. Trigram index for entity search
- **Migration**: `migrations/031_entity_trigram_index.up.sql` — creates `pg_trgm`
extension and GIN trigram indexes on `entities.slug` and `entities.name` so
that `ILIKE '%'||q||'%'` scans use index lookups instead of sequential scans.
### F4. Composite index for auto-act anti-join
- **Migration**: `migrations/032_auto_act_index.up.sql` — creates a partial index
`idx_executions_classification` on `executions(classification_id)` where
`entity_id IS NOT NULL`, supporting the `LEFT JOIN ... WHERE e.entity_id IS NULL`
anti-join in `GetOpenSignalsForAutoAct`.
## G. Observability (low)
### G1. Add OpenTelemetry tracing
- OTel SDK is already in go.mod as indirect dependency.
- Instrument HTTP handlers, MCP tools, and DB queries with spans.
- Propagate trace context via `correlation_id` (already exists in audit/events).
### G2. Prometheus metrics export
- Expose `/metrics` endpoint for Go runtime, DB pool stats, scheduler check
duration/counts, HTTP request latency histograms.
- Complement the existing TimescaleDB metric_samples (which are entity health
metrics, not self-observability).
### G3. Automate offsite backups
- Proton Drive backup target entity exists but no pipeline.
- Add `rclone cron` to `pg_dump | zstd | rclone sync` to Proton Drive.
- Weekly backup verification (restore to test DB, run `make test-db`).
## H. Infrastructure (low)
### H1. Pin Infisical image version
- **Done** — `docker-compose.yml` pinned `infisical/infisical:latest` → `v0.99.1`.
Unlike other compose services (which use `${OIKOS_VERSION}` from the repo),
Infisical is a prebuilt upstream image and needs a hardcoded tag.
### H2. Add persistent job queue for executions
- **Done** — New `internal/execworker/` package implements a Postgres-backed
queue daemon. Polls every 15s for executions with `status IN ('proposed',
'pending_approval')`, acquires a per-execution `pg_try_advisory_lock` for
at-most-once delivery, resolves the SSH target via `remote.ResolveHost`,
and runs the action command via `actuator.RunCombinedOutput`.
- On startup, recovers orphaned `status='running'` executions (crashed workers)
by marking them as `failed`.
- Registered as an `execution-worker` role in `cmd/oikos/main.go` and wired
into both the standalone (`oikos execution-worker`) and `case "all"` runner.
- Added to `docker-compose.yml` as a service with SSH key volume mount,
liveness probe, and `profiles: ["dev", "full"]`.
- **Files**: `internal/execworker/worker.go`, `internal/execworker/init.go`,
`cmd/oikos/main.go` (new role + "all" background), `docker-compose.yml` (service).
### H3. Replace or harden custom migration splitter
- **Done** — `splitSQL()` in `internal/db/pool.go` now handles block
comments (`/* */`) and single-quoted string literals (`'...'`) in
addition to the existing dollar-quote and line-comment support.
Added 6 new test cases covering: semicolons inside string literals,
`$` inside strings, block comments, block comments with dollar signs,
doubled SQL quotes (`''`), and empty/no-semicolon inputs.
Total: 11 tests, all passing.
### H4. Add distributed locking for scheduler
- **Done** — `scheduler.Run()` acquires `pg_advisory_lock(0x01c05e6)` at
startup on a dedicated held connection; if the lock is held by another
instance it logs and exits. Released on shutdown via defer (using
`context.WithoutCancel` so the unlock runs even when ctx is cancelled).
Lock key `0x01c05e6` differs from the migration lock `0x01c05e5`.
---
## Execution order
1. **Phase 0 — Infisical consolidation** (B1B7): Wire Infisical into all
binaries, migrate secrets from env/plaintext, activate SOPS fallback, remove
`.env` secrets and plist HMAC. This is the foundation — every subsequent
secret-dependent change (SSH host keys in B5, rate limit config, etc.) goes
through Infisical. **Do this first.**
2. **Phase 1 — Security** (C1C3, B5): Nomos auth, pg_dump failure, CORS default,
SSH host keys (now stored in Infisical per B5).
3. **Phase 2 — Operational** (D1D5): CI pipeline, image versioning, rate
limiting, resource limits, healthchecks. **Done.**
4. **Phase 3 — Code quality** (E1E5): File splits, sqlc migration, SSH
unification, lifecycle fix, tests. **Done.** (Rebased onto main 0.28.5 and
landed as 0.29.0.)
5. **Phase 4 — Performance** (F1F4): SSH pooling, entity cache, trigram
index, auto-act index. **Done.**
6. **Phase 5 — Observability** (G1G3): OTel tracing, Prometheus, offsite backups.
7. **Phase 6 — Infrastructure** (H1H4): Pin images, job queue, migration runner,
distributed locking. **Done.**
Phases 06 are complete. Phase 5 (Observability) is backlog.
---
## Phase 0 post-deploy checklist
Run these on mac-mini after deploying the Phase 0 code changes.
### Step 1: Populate Infisical with secrets
For each secret, read the current value from `.env` and store it in Infisical:
```bash
# Values from .env — read them first, then set
oikos secret set matrix/token "$(grep OIKOS_MATRIX_TOKEN .env | cut -d= -f2-)"
oikos secret set approval/hmac-secret "$(grep OIKOS_APPROVAL_HMAC_SECRET .env | cut -d= -f2-)"
oikos secret set mcp/bearer-token "$(grep OIKOS_MCP_BEARER_TOKEN .env | cut -d= -f2-)"
oikos secret set api/token "$(grep OIKOS_API_TOKEN .env | cut -d= -f2-)"
oikos secret set oidc/client-secret "$(grep OIKOS_OIDC_CLIENT_SECRET .env | cut -d= -f2-)"
oikos secret set openrouter/api-key "$(grep OPENROUTER_API_KEY .env | cut -d= -f-)"
oikos secret set webhook/hmac-secret "$(grep WEBHOOK_HMAC_SECRET .env | cut -d= -f2-)"
```
Verify: `oikos secret list` should show all 8 keys.
### Step 2: Pre-populate SSH host keys (optional, skip if TOFU is acceptable)
```bash
# For each Proxmox host, scan and store the public key
for host in pve1 pve2; do
key=$(ssh-keyscan -t ed25519 $host 2>/dev/null | awk '{print $2" "$3}')
oikos secret set "ssh/host-keys/$host" "ssh-ed25519 $key"
done
```
Alternatively, skip this step — the first deployment will TOFU-accept all current
host keys and persist them to Infisical automatically.
### Step 3: Remove HMAC secret from webhook plist
On mac-mini:
```bash
sudo launchctl unload ~/Library/LaunchAgents/network.hubris.oikos-deploy-webhook.plist
# Edit the plist: remove the <key>WEBHOOK_HMAC_SECRET</key> block
sudo launchctl load ~/Library/LaunchAgents/network.hubris.oikos-deploy-webhook.plist
```
### Step 4: Strip secrets from .env
Edit `.env` to remove the 7 migrated secrets, keeping only bootstrap and
non-secret config:
```bash
# Remove these lines:
# INFISICAL_ENCRYPTION_KEY=...
# OIKOS_MATRIX_TOKEN=...
# OIKOS_INFISICAL_CLIENT_ID=...
# OIKOS_INFISICAL_CLIENT_SECRET=...
# OIKOS_INFISICAL_PROJECT_ID=...
# OPENROUTER_API_KEY=...
# OIKOS_MCP_BEARER_TOKEN=...
# Keep these (bootstrap / non-secret):
# OIKOS_DATABASE_URL=...
# OIKOS_API_LISTEN=...
# OIKOS_INFISICAL_SITE_URL=...
# OIKOS_INFISICAL_ENV=...
```
### Step 5: Verify
1. `oikos secret list` — 8 keys + SSH host keys
2. `docker compose logs api | grep "secrets resolved"` — should show count=5
3. Trigger a test deploy — webhook should still validate HMAC signatures
4. `nomos` should start and resolve secrets from Infisical (check logs for
`nomos: secrets resolved from Infisical`)
5. Verify no secrets appear in process env: `docker compose exec api env |
should not show `OIKOS_MCP_BEARER_TOKEN`, `OIKOS_MATRIX_TOKEN`,
etc. (they come from Infisical at startup, not env)

View File

@@ -21,6 +21,7 @@ went sideways, open an investigation.
| 2026-07-21 | [Frontend as OS + Apps — architecture audit & refactor](2026-07-21-frontend-os-apps-architecture.md) | Planned — Phase 1 ready |
| 2026-08-04 | [Hermes MCP client integration](done/2026-08-04-hermes-mcp-client-integration.md) | Done — deployed |
| 2026-08-05 | [Agent execution safety: QEMU guest agent gate + host-mutation guard](done/2026-08-05-agent-execution-safety-qemu-guest-agent-gate.md) | Done — implemented (1b9c761) |
| 2026-08-05 | [Backend evaluation: architecture, security, and reliability improvements](2026-08-05-backend-evaluation-improvements.md) | Done — all three phases (B, D, E) implemented as code (0.28.00.29.0), deployed, and hardened via review. Remaining: C (security) and F (performance) backlog. |
## Done

View File

@@ -1,7 +1,35 @@
#!/bin/sh
# Oikos deploy script — triggered by Gitea webhook on push to dtoro/oikos.
# Runs on mac-mini as non-root user via systemd unit oikos-deploy-webhook.service.
# Phase 6: CI-gated, SHA-tagged images, rolling restart, pre-deploy pg_dump.
# Runs on mac-mini as non-root user via launchd unit oikos-deploy-webhook.service.
# Phase 6: CI-gated, version-tagged images, rolling restart, pre-deploy pg_dump.
#
# Plans implemented here:
# D1 — CI gate: blocks deploy unless Gitea reports a green run for the SHA.
# D2 — versioned images: tags every built image v$VERSION (from VERSION file),
# keeps the last 3 tags per service for rollback.
# Notify on deploy failure via Matrix. Uses Oikos API to raise an event
# so the scheduler picks it up and alerts via the notifier.
notify_deploy_failure() {
local reason="$1"
local sha="${SHA:-unknown}"
echo "NOTIFY: deploy failed — $reason"
# Try to raise an event through the Oikos API (best-effort, silent failure)
if [ -n "${OIKOS_API_TOKEN:-}" ]; then
curl -sf -X POST "http://localhost:8090/api/v1/events" \
-H "Authorization: Bearer $OIKOS_API_TOKEN" \
-H "Content-Type: application/json" \
-d "{\"type\":\"deploy.failed\",\"severity\":\"critical\",\"source\":\"webhook\",\"data\":{\"sha\":\"$sha\",\"reason\":\"$reason\"}}" \
>/dev/null 2>&1 || true
fi
# Also try Matrix directly via the notifier's webhook endpoint if configured
if [ -n "${MATRIX_WEBHOOK_URL:-}" ]; then
curl -sf -X POST "$MATRIX_WEBHOOK_URL" \
-H "Content-Type: application/json" \
-d "{\"msgtype\":\"m.text\",\"body\":\"🚨 Deploy failed: $reason (sha: $sha)\"}" \
>/dev/null 2>&1 || true
fi
}
set -e
@@ -13,48 +41,237 @@ DUMP_DIR="${DUMP_DIR:-/opt/oikos/backups}"
RETRIES=${RETRIES:-30}
SLEEP=${SLEEP:-2}
# CI gate (D1). Set GITEA_URL + GITEA_TOKEN to enable; without them the gate
# is skipped with a warning (dev/local deploys). Owner/repo default to the
# canonical homelab repo but can be overridden or derived from the git remote.
GITEA_URL="${GITEA_URL:-}"
GITEA_TOKEN="${GITEA_TOKEN:-}"
GITEA_OWNER="${GITEA_OWNER:-dtoro}"
GITEA_REPO="${GITEA_REPO:-oikos}"
CI_POLL_INTERVAL="${CI_POLL_INTERVAL:-15}"
CI_TIMEOUT="${CI_TIMEOUT:-1200}"
# Serialize deploys: the webhook runs this script in a background goroutine and
# the CI gate can hold a deploy open for many minutes, so a second push during
# that window would otherwise race on git/pg_dump/compose. mkdir is atomic on
# POSIX (no flock dependency — macOS lacks it). The stale-pid check recovers
# if a previous deploy was SIGKILLed.
LOCKDIR="${LOCKDIR:-/tmp/oikos-deploy.lock}"
if ! mkdir "$LOCKDIR" 2>/dev/null; then
oldpid=$(cat "$LOCKDIR/pid" 2>/dev/null || echo "")
if [ -n "$oldpid" ] && kill -0 "$oldpid" 2>/dev/null; then
echo "deploy already in progress (pid $oldpid) — exiting"
exit 0
fi
echo "removing stale deploy lock (pid ${oldpid:-?} not running)"
rm -rf "$LOCKDIR"
mkdir "$LOCKDIR"
fi
echo $$ > "$LOCKDIR/pid"
trap 'rc=$?; rm -rf "$LOCKDIR" 2>/dev/null || true; if [ "$_ok" != "1" ]; then notify_deploy_failure "deploy aborted (exit $rc)"; fi' EXIT
_ok=0
cd "$REPO_DIR"
echo "=== oikos deploy: $(date) ==="
SHA=$(git rev-parse --short HEAD)
echo "SHA: $SHA"
# 1. Pre-deploy pg_dump for rollback safety (plan O1)
echo "[1/6] pre-deploy pg_dump"
DUMP_FILE="$DUMP_DIR/pre-deploy-$SHA.sql"
# Resolve the SHA we are ABOUT to deploy from the remote (read-only: no working
# tree mutation yet) so the CI gate can run before anything is touched. Keep the
# full SHA (REMOTE_FULL) for the post-pull equality check; the 12-char form is
# only for display and the Gitea status API (which accepts any unique prefix).
REMOTE_FULL=$(git ls-remote origin refs/heads/main 2>/dev/null | awk '{print $1}')
if [ -z "$REMOTE_FULL" ]; then
echo "ERROR: could not resolve origin/main (offline?) — aborting before any change"
exit 1
fi
REMOTE_SHA=$(printf '%s' "$REMOTE_FULL" | cut -c1-12)
echo "remote SHA: $REMOTE_SHA"
# ── 1. CI gate (plan D1) ──────────────────────────────────────────────────
# Runs BEFORE pg_dump/pull/build: a red or hung pipeline must not leave the
# tree half-deployed. On failure|error it refuses; on success it proceeds; on
# "no CI signal at all" (Actions unconfigured / token rejected) it warns and
# proceeds rather than bricking every deploy.
echo "[1/8] verify CI status for $REMOTE_SHA"
verify_ci() {
sha=$1
if [ -z "$GITEA_URL" ] || [ -z "$GITEA_TOKEN" ]; then
echo "SKIP: GITEA_URL/GITEA_TOKEN not set — CI gate disabled. Set both to enforce."
return 0
fi
# Derive owner/repo from the origin remote when the defaults don't apply.
origin=$(git remote get-url origin 2>/dev/null || echo "")
seg=
case "$origin" in
*@*:*) # SSH: git@host:owner/repo.git
seg=${origin##*:}; seg=${seg%.git}
;;
http://*|https://*) # HTTPS: scheme://host/owner/repo.git
seg=${origin#*://}; seg=${seg#*/}; seg=${seg%.git}
;;
esac
case "$seg" in
*/*)
GITEA_OWNER=${seg%%/*}
GITEA_REPO=${seg#*/}
;;
esac
api="$GITEA_URL/api/v1/repos/$GITEA_OWNER/$GITEA_REPO/commits/$sha/status"
body=$(mktemp)
elapsed=0
saw_ci=0 # became 1 once we observed a real status (pending/success/...)
no_signal=0 # consecutive responses with no usable status
while [ "$elapsed" -lt "$CI_TIMEOUT" ]; do
# Pass the token via curl --config stdin so it never appears in argv
# (visible via ps). Don't use -f: we want the HTTP code on 4xx.
code=$(printf 'header = "Authorization: token %s"\n' "$GITEA_TOKEN" | \
curl -sS -o "$body" -w '%{http_code}' --config - "$api" 2>/dev/null) || code="000"
state=$(sed -n 's/.*"state"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' "$body" | head -n1)
case "$code" in
200)
case "$state" in
success)
rm -f "$body"
echo "CI: green for $sha after ${elapsed}s"
return 0
;;
failure|error)
rm -f "$body"
echo "ERROR: CI $state for $sha — refusing to deploy."
echo " See $GITEA_URL/$GITEA_OWNER/$GITEA_REPO/actions"
return 1
;;
pending|"")
saw_ci=1
no_signal=0
;;
esac
;;
404)
# No status checks exist for this commit (Actions not configured
# / no runner has reported). Can't gate — warn + proceed.
rm -f "$body"
echo "WARN: Gitea has no CI status for $sha (404)."
echo " Is Gitea Actions configured with a runner for $GITEA_OWNER/$GITEA_REPO?"
echo " Proceeding without a gate."
return 0
;;
401|403)
rm -f "$body"
echo "WARN: GITEA_TOKEN rejected by Gitea ($code) — cannot verify CI."
echo " Fix the token to enforce the gate; proceeding without one."
return 0
;;
*)
# Network blip / 5xx / 000: retry, but count as no-signal.
no_signal=$((no_signal + 1))
;;
esac
# If we never get a usable signal after ~1 min, assume CI is unreachable
# rather than burn the full timeout and brick deploys.
if [ "$saw_ci" -eq 0 ] && [ "$no_signal" -ge 4 ]; then
rm -f "$body"
echo "WARN: no CI signal from Gitea after ${elapsed}s (last code=$code)."
echo " CI may be down or misconfigured; proceeding without a gate."
return 0
fi
sleep "$CI_POLL_INTERVAL"
elapsed=$((elapsed + CI_POLL_INTERVAL))
printf '\rCI: waiting (%ss, code=%s state=%s)...' "$elapsed" "$code" "${state:-none}"
done
rm -f "$body"
echo ""
echo "ERROR: CI did not reach a terminal state within ${CI_TIMEOUT}s for $sha — refusing to deploy."
return 1
}
verify_ci "$REMOTE_SHA" || exit 1
# ── 2. Pre-deploy pg_dump for rollback safety (plan O1) ───────────────────
echo "[2/8] pre-deploy pg_dump"
DUMP_FILE="$DUMP_DIR/pre-deploy-$REMOTE_SHA.sql"
mkdir -p "$DUMP_DIR"
docker compose exec -T postgres pg_dump -U oikos oikos > "$DUMP_FILE" 2>/dev/null || \
echo "WARNING: pg_dump failed — rollback will not have a recovery point"
# 2. Pull latest
echo "[2/6] git pull"
git pull origin main
# 3. Verify CI passed
echo "[3/6] verify build"
if ! git log -1 --format="%s" | grep -q .; then
echo "ERROR: empty commit message"
# ── 3. Update working tree to the verified commit ─────────────────────────
echo "[3/8] git pull (ff-only)"
git pull --ff-only origin main
# TOCTOU guard: if origin/main advanced during the CI wait + pg_dump, the pull
# fast-forwards PAST the SHA we verified without re-checking its CI. Refuse
# rather than ship an unverified commit — a retry will verify the new tip.
PULLED_FULL=$(git rev-parse HEAD)
if [ "$PULLED_FULL" != "$REMOTE_FULL" ]; then
echo "ERROR: origin/main advanced during deploy (verified $REMOTE_SHA, now at $(git rev-parse --short HEAD)) — aborting; retry verifies the new tip"
exit 1
fi
SHA=$(git rev-parse --short HEAD)
echo "SHA (deployed): $SHA"
# 4. Build and restart with health-check rollout
echo "[4/6] docker compose build"
# Resolve the deploy version AFTER pull (D2) so the tag matches the code being
# built. Compose interpolates $OIKOS_VERSION into each service's image: tag.
VERSION_FILE="$REPO_DIR/VERSION"
if [ -f "$VERSION_FILE" ]; then
OIKOS_VERSION="v$(head -n1 "$VERSION_FILE" | tr -d '[:space:]')"
export OIKOS_VERSION
echo "VERSION: $OIKOS_VERSION"
else
echo "WARNING: VERSION file missing — images will use :latest (rollback unavailable)"
fi
# ── 4. Build version-tagged images (plan D2) ──────────────────────────────
echo "[4/8] docker compose build"
DOCKER_BUILDKIT=1 docker compose --profile "$PROFILE" build \
--build-arg BUILDKIT_INLINE_CACHE=1
# 5. Rolling restart
echo "[5/6] docker compose up -d"
# ── 5. Rolling restart ────────────────────────────────────────────────────
echo "[5/8] docker compose up -d"
docker compose --profile "$PROFILE" up -d --remove-orphans
# 6. Health check wait
echo "[6/6] health check"
# ── 6. Prune old image tags — keep the 3 newest per service so rollback ────
# (OIKOS_VERSION=v0.x.y docker compose up) stays available. The repo list
# is derived from compose so it can't drift from the image: names.
echo "[6/8] prune old image tags (keep 3)"
if [ -n "$OIKOS_VERSION" ]; then
images=$(docker compose --profile "$PROFILE" config --images 2>/dev/null || true)
if [ -z "$images" ]; then
images="oikos-api oikos-scheduler oikos-notifier oikos-migrate oikos-seed oikos-nomos oikos-web"
fi
printf '%s\n' $images | sed 's/:.*//' | grep '^oikos-' | sort -u | while read -r repo; do
docker image ls "$repo" --format '{{.Tag}}' 2>/dev/null | grep '^v' | sort -rV | tail -n +4 | while read -r tag; do
docker rmi "$repo:$tag" >/dev/null 2>&1 || true
done
done
fi
# ── 7. Health check wait ──────────────────────────────────────────────────
echo "[7/8] health check"
healthy=0
for i in $(seq 1 $RETRIES); do
if curl -sf "$HEALTH_URL" > /dev/null 2>&1; then
echo "healthy after ${i}s"
exit 0
healthy=1
break
fi
sleep "$SLEEP"
done
if [ "$healthy" -ne 1 ]; then
echo "ERROR: health check failed after $((RETRIES * SLEEP))s"
exit 1
fi
echo "ERROR: health check failed after $((RETRIES * SLEEP))s"
exit 1
# ── 8. Seed secrets into Infisical (idempotent) ───────────────────────────
echo "[8/8] seed secrets"
if [ -f "$REPO_DIR/scripts/seed-secrets.sh" ]; then
REPO_DIR="$REPO_DIR" sh "$REPO_DIR/scripts/seed-secrets.sh" || \
echo "WARNING: secret seeding failed"
else
echo "SKIP: seed-secrets.sh not found"
fi
# All steps completed successfully — clear failure trap
_ok=1
exit 0

View File

@@ -10,8 +10,16 @@
</array>
<key>EnvironmentVariables</key>
<dict>
<key>WEBHOOK_HMAC_SECRET</key>
<string>6502524162d6dbc3f6d137000395d401f1837d74ef9bb0a876f8e6bbd65d1ff2</string>
<key>OIKOS_INFISICAL_SITE_URL</key>
<string>http://localhost:8080</string>
<key>OIKOS_INFISICAL_CLIENT_ID</key>
<string>82e6e362-bbee-4f22-b44e-215c85bdb14a</string>
<key>OIKOS_INFISICAL_CLIENT_SECRET</key>
<string>aa7c928b366591741e4fa99016c5644fc6119b4174ba0dab3570db1229e0bdf1</string>
<key>OIKOS_INFISICAL_PROJECT_ID</key>
<string>a436936f-6610-4698-9075-76013af7c68e</string>
<key>OIKOS_INFISICAL_ENV</key>
<string>dev</string>
<key>WEBHOOK_REPO_DIR</key>
<string>/Users/dtoro/Projects/oikos</string>
<key>WEBHOOK_LISTEN</key>

68
scripts/seed-secrets.sh Executable file
View File

@@ -0,0 +1,68 @@
#!/bin/sh
# One-shot: populate Infisical with oikos secrets from container env.
# Only runs if Infisical is empty (first bootstrap). Safe to re-run.
#
# Usage: ./scripts/seed-secrets.sh
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
REPO_DIR="${REPO_DIR:-$SCRIPT_DIR/..}"
COMPOSE="docker compose -f $REPO_DIR/docker-compose.yml"
echo "=== seed-secrets: $(date) ==="
# Skip if Infisical already has secrets (avoid overwriting real values
# with dev defaults from docker-compose).
existing=$($COMPOSE exec -T api /oikos secret list 2>/dev/null | grep -c . || echo 0)
if [ "$existing" -gt 3 ]; then
echo "SKIP: Infisical already has $existing secrets (bootstrap complete)"
exit 0
fi
echo "Infisical has $existing secrets — seeding..."
set_count=0
skip_count=0
fail_count=0
get_container_env() {
service="$1"
var="$2"
$COMPOSE exec -T "$service" printenv "$var" 2>/dev/null || true
}
seed_key() {
key="$1"
value="$2"
if [ -z "$value" ]; then
echo "SKIP: $key (empty)"
skip_count=$((skip_count + 1))
return
fi
if $COMPOSE exec -T api oikos secret set "$key" "$value" 2>/dev/null; then
echo "SET: $key"
set_count=$((set_count + 1))
else
echo "FAIL: $key"
fail_count=$((fail_count + 1))
fi
}
matrix_token="$(get_container_env notifier OIKOS_MATRIX_TOKEN)"
approval_hmac="$(get_container_env notifier OIKOS_APPROVAL_HMAC_SECRET)"
mcp_token="$(get_container_env api OIKOS_MCP_BEARER_TOKEN)"
openrouter_key="$(get_container_env nomos OPENROUTER_API_KEY)"
webhook_hmac="$(get_container_env api WEBHOOK_HMAC_SECRET 2>/dev/null)"
api_token="$mcp_token"
seed_key "matrix_token" "$matrix_token"
seed_key "approval_hmac-secret" "$approval_hmac"
seed_key "mcp_bearer-token" "$mcp_token"
seed_key "api_token" "$api_token"
seed_key "openrouter_api-key" "$openrouter_key"
seed_key "webhook_hmac-secret" "$webhook_hmac"
echo ""
echo "seed-secrets complete: $set_count set, $skip_count skipped, $fail_count failed"

View File

@@ -232,6 +232,7 @@ entity_types:
layer: infrastructure
lifecycle: infrastructure
description: Machine running Proxmox VE.
monitoring: [quorum] # corosync quorum check via pvecm status
attributes:
type: object
properties: {pve_version: {type: string}}

469
vendor/cloud.google.com/go/auth/CHANGES.md generated vendored Normal file
View File

@@ -0,0 +1,469 @@
# Changes
## [0.18.1](https://github.com/googleapis/google-cloud-go/releases/tag/auth%2Fv0.18.1) (2026-01-21)
### Bug Fixes
* add InternalOptions.TelemetryAttributes for internal client use (#13641) ([3876978](https://github.com/googleapis/google-cloud-go/commit/38769789755ed47d85e85dcd56596109de65f780))
* remove singleton and restore normal usage of otelgrpc.clientHandler (#13522) ([673d4b0](https://github.com/googleapis/google-cloud-go/commit/673d4b05617f833aa433f7f6a350b5cb888ea20d))
## [0.18.0](https://github.com/googleapis/google-cloud-go/releases/tag/auth%2Fv0.18.0) (2025-12-15)
### Features
* Support scopes field from impersonated credential json (#13308) ([e3f62e1](https://github.com/googleapis/google-cloud-go/commit/e3f62e102840127a0058f5cced4c9738f2bf45f2))
* add support for parsing EC private key (#13317) ([ea6bc62](https://github.com/googleapis/google-cloud-go/commit/ea6bc62ffe2cc0a6d607d698a181b37fa46c340d))
* deprecate unsafe credentials JSON loading options (#13397) ([0dd2a3b](https://github.com/googleapis/google-cloud-go/commit/0dd2a3bdece9a85ee7216a737559fa9f5a869545))
## [0.17.0](https://github.com/googleapis/google-cloud-go/releases/tag/auth%2Fv0.17.0) (2025-10-02)
### Features
* Add trust boundary support for service accounts and impersonation (HTTP/gRPC) (#11870) ([5c2b665](https://github.com/googleapis/google-cloud-go/commit/5c2b665f392e6dd90192f107188720aa1357e7da))
* add trust boundary support for external accounts (#12864) ([a67a146](https://github.com/googleapis/google-cloud-go/commit/a67a146a6a88a6f1ba10c409dfce8015ecd60a64))
## [0.16.5](https://github.com/googleapis/google-cloud-go/compare/auth/v0.16.4...auth/v0.16.5) (2025-08-14)
### Bug Fixes
* **auth:** Improve error message for unknown credentials type ([#12673](https://github.com/googleapis/google-cloud-go/issues/12673)) ([558b164](https://github.com/googleapis/google-cloud-go/commit/558b16429f621276694405fa5f2091199f2d4c4d))
* **auth:** Set Content-Type in userTokenProvider.exchangeToken ([#12634](https://github.com/googleapis/google-cloud-go/issues/12634)) ([1197ebc](https://github.com/googleapis/google-cloud-go/commit/1197ebcbca491f8c610da732c7361c90bc6f46d0))
## [0.16.4](https://github.com/googleapis/google-cloud-go/compare/auth/v0.16.3...auth/v0.16.4) (2025-08-06)
### Bug Fixes
* **auth:** Add UseDefaultClient: true to metadata.Options ([#12666](https://github.com/googleapis/google-cloud-go/issues/12666)) ([1482191](https://github.com/googleapis/google-cloud-go/commit/1482191e88236693efef68769752638281566766)), refs [#11078](https://github.com/googleapis/google-cloud-go/issues/11078) [#12657](https://github.com/googleapis/google-cloud-go/issues/12657)
## [0.16.3](https://github.com/googleapis/google-cloud-go/compare/auth/v0.16.2...auth/v0.16.3) (2025-07-17)
### Bug Fixes
* **auth:** Fix race condition in cachedTokenProvider.tokenAsync ([#12586](https://github.com/googleapis/google-cloud-go/issues/12586)) ([73867cc](https://github.com/googleapis/google-cloud-go/commit/73867ccc1e9808d65361bcfc0776bd95fe34dbb3))
## [0.16.2](https://github.com/googleapis/google-cloud-go/compare/auth/v0.16.1...auth/v0.16.2) (2025-06-04)
### Bug Fixes
* **auth:** Add back DirectPath misconfiguration logging ([#11162](https://github.com/googleapis/google-cloud-go/issues/11162)) ([8d52da5](https://github.com/googleapis/google-cloud-go/commit/8d52da58da5a0ed77a0f6307d1b561bc045406a1))
* **auth:** Remove s2a fallback option ([#12354](https://github.com/googleapis/google-cloud-go/issues/12354)) ([d5acc59](https://github.com/googleapis/google-cloud-go/commit/d5acc599cd775ddc404349e75906fa02e8ff133e))
## [0.16.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.16.0...auth/v0.16.1) (2025-04-23)
### Bug Fixes
* **auth:** Clone detectopts before assigning TokenBindingType ([#11881](https://github.com/googleapis/google-cloud-go/issues/11881)) ([2167b02](https://github.com/googleapis/google-cloud-go/commit/2167b020fdc43b517c2b6ecca264a10e357ea035))
## [0.16.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.15.0...auth/v0.16.0) (2025-04-14)
### Features
* **auth/credentials:** Return X.509 certificate chain as subject token ([#11948](https://github.com/googleapis/google-cloud-go/issues/11948)) ([d445a3f](https://github.com/googleapis/google-cloud-go/commit/d445a3f66272ffd5c39c4939af9bebad4582631c)), refs [#11757](https://github.com/googleapis/google-cloud-go/issues/11757)
* **auth:** Configure DirectPath bound credentials from AllowedHardBoundTokens ([#11665](https://github.com/googleapis/google-cloud-go/issues/11665)) ([0fc40bc](https://github.com/googleapis/google-cloud-go/commit/0fc40bcf4e4673704df0973e9fa65957395d7bb4))
### Bug Fixes
* **auth:** Allow non-default SA credentials for DP ([#11828](https://github.com/googleapis/google-cloud-go/issues/11828)) ([3a996b4](https://github.com/googleapis/google-cloud-go/commit/3a996b4129e6d0a34dfda6671f535d5aefb26a82))
* **auth:** Restore calling DialContext ([#11930](https://github.com/googleapis/google-cloud-go/issues/11930)) ([9ec9a29](https://github.com/googleapis/google-cloud-go/commit/9ec9a29494e93197edbaf45aba28984801e9770a)), refs [#11118](https://github.com/googleapis/google-cloud-go/issues/11118)
## [0.15.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.14.1...auth/v0.15.0) (2025-02-19)
### Features
* **auth:** Add hard-bound token request to compute token provider. ([#11588](https://github.com/googleapis/google-cloud-go/issues/11588)) ([0e608bb](https://github.com/googleapis/google-cloud-go/commit/0e608bb5ac3d694c8ad36ca4340071d3a2c78699))
## [0.14.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.14.0...auth/v0.14.1) (2025-01-24)
### Documentation
* **auth:** Add warning about externally-provided credentials ([#11462](https://github.com/googleapis/google-cloud-go/issues/11462)) ([49fb6ff](https://github.com/googleapis/google-cloud-go/commit/49fb6ff4d754895f82c9c4d502fc7547d3b5a941))
## [0.14.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.13.0...auth/v0.14.0) (2025-01-08)
### Features
* **auth:** Add universe domain support to idtoken ([#11059](https://github.com/googleapis/google-cloud-go/issues/11059)) ([72add7e](https://github.com/googleapis/google-cloud-go/commit/72add7e9f8f455af695e8ef79212a4bd3122fb3a))
### Bug Fixes
* **auth/oauth2adapt:** Update golang.org/x/net to v0.33.0 ([e9b0b69](https://github.com/googleapis/google-cloud-go/commit/e9b0b69644ea5b276cacff0a707e8a5e87efafc9))
* **auth:** Fix copy of delegates in impersonate.NewIDTokenCredentials ([#11386](https://github.com/googleapis/google-cloud-go/issues/11386)) ([ff7ef8e](https://github.com/googleapis/google-cloud-go/commit/ff7ef8e7ade7171bce3e4f30ff10a2e9f6c27ca0)), refs [#11379](https://github.com/googleapis/google-cloud-go/issues/11379)
* **auth:** Update golang.org/x/net to v0.33.0 ([e9b0b69](https://github.com/googleapis/google-cloud-go/commit/e9b0b69644ea5b276cacff0a707e8a5e87efafc9))
## [0.13.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.12.1...auth/v0.13.0) (2024-12-13)
### Features
* **auth:** Add logging support ([#11079](https://github.com/googleapis/google-cloud-go/issues/11079)) ([c80e31d](https://github.com/googleapis/google-cloud-go/commit/c80e31df5ecb33a810be3dfb9d9e27ac531aa91d))
* **auth:** Pass logger from auth layer to metadata package ([#11288](https://github.com/googleapis/google-cloud-go/issues/11288)) ([b552efd](https://github.com/googleapis/google-cloud-go/commit/b552efd6ab34e5dfded18438e0fbfd925805614f))
### Bug Fixes
* **auth:** Check compute cred type before non-default flag for DP ([#11255](https://github.com/googleapis/google-cloud-go/issues/11255)) ([4347ca1](https://github.com/googleapis/google-cloud-go/commit/4347ca141892be8ae813399b4b437662a103bc90))
## [0.12.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.12.0...auth/v0.12.1) (2024-12-10)
### Bug Fixes
* **auth:** Correct typo in link ([#11160](https://github.com/googleapis/google-cloud-go/issues/11160)) ([af6fb46](https://github.com/googleapis/google-cloud-go/commit/af6fb46d7cd694ddbe8c9d63bc4cdcd62b9fb2c1))
## [0.12.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.11.0...auth/v0.12.0) (2024-12-04)
### Features
* **auth:** Add support for providing custom certificate URL ([#11006](https://github.com/googleapis/google-cloud-go/issues/11006)) ([ebf3657](https://github.com/googleapis/google-cloud-go/commit/ebf36579724afb375d3974cf1da38f703e3b7dbc)), refs [#11005](https://github.com/googleapis/google-cloud-go/issues/11005)
### Bug Fixes
* **auth:** Ensure endpoints are present in Validator ([#11209](https://github.com/googleapis/google-cloud-go/issues/11209)) ([106cd53](https://github.com/googleapis/google-cloud-go/commit/106cd53309facaef1b8ea78376179f523f6912b9)), refs [#11006](https://github.com/googleapis/google-cloud-go/issues/11006) [#11190](https://github.com/googleapis/google-cloud-go/issues/11190) [#11189](https://github.com/googleapis/google-cloud-go/issues/11189) [#11188](https://github.com/googleapis/google-cloud-go/issues/11188)
## [0.11.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.10.2...auth/v0.11.0) (2024-11-21)
### Features
* **auth:** Add universe domain support to mTLS ([#11159](https://github.com/googleapis/google-cloud-go/issues/11159)) ([117748b](https://github.com/googleapis/google-cloud-go/commit/117748ba1cfd4ae62a6a4feb7e30951cb2bc9344))
## [0.10.2](https://github.com/googleapis/google-cloud-go/compare/auth/v0.10.1...auth/v0.10.2) (2024-11-12)
### Bug Fixes
* **auth:** Restore use of grpc.Dial ([#11118](https://github.com/googleapis/google-cloud-go/issues/11118)) ([2456b94](https://github.com/googleapis/google-cloud-go/commit/2456b943b7b8aaabd4d8bfb7572c0f477ae0db45)), refs [#7556](https://github.com/googleapis/google-cloud-go/issues/7556)
## [0.10.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.10.0...auth/v0.10.1) (2024-11-06)
### Bug Fixes
* **auth:** Restore Application Default Credentials support to idtoken ([#11083](https://github.com/googleapis/google-cloud-go/issues/11083)) ([8771f2e](https://github.com/googleapis/google-cloud-go/commit/8771f2ea9807ab822083808e0678392edff3b4f2))
* **auth:** Skip impersonate universe domain check if empty ([#11086](https://github.com/googleapis/google-cloud-go/issues/11086)) ([87159c1](https://github.com/googleapis/google-cloud-go/commit/87159c1059d4a18d1367ce62746a838a94964ab6))
## [0.10.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.9...auth/v0.10.0) (2024-10-30)
### Features
* **auth:** Add universe domain support to credentials/impersonate ([#10953](https://github.com/googleapis/google-cloud-go/issues/10953)) ([e06cb64](https://github.com/googleapis/google-cloud-go/commit/e06cb6499f7eda3aef08ab18ff197016f667684b))
## [0.9.9](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.8...auth/v0.9.9) (2024-10-22)
### Bug Fixes
* **auth:** Fallback cert lookups for missing files ([#11013](https://github.com/googleapis/google-cloud-go/issues/11013)) ([bd76695](https://github.com/googleapis/google-cloud-go/commit/bd766957ec238b7c40ddbabb369e612dc9b07313)), refs [#10844](https://github.com/googleapis/google-cloud-go/issues/10844)
* **auth:** Replace MDS endpoint universe_domain with universe-domain ([#11000](https://github.com/googleapis/google-cloud-go/issues/11000)) ([6a1586f](https://github.com/googleapis/google-cloud-go/commit/6a1586f2ce9974684affaea84e7b629313b4d114))
## [0.9.8](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.7...auth/v0.9.8) (2024-10-09)
### Bug Fixes
* **auth:** Restore OpenTelemetry handling in transports ([#10968](https://github.com/googleapis/google-cloud-go/issues/10968)) ([08c6d04](https://github.com/googleapis/google-cloud-go/commit/08c6d04901c1a20e219b2d86df41dbaa6d7d7b55)), refs [#10962](https://github.com/googleapis/google-cloud-go/issues/10962)
* **auth:** Try talk to plaintext S2A if credentials can not be found for mTLS-S2A ([#10941](https://github.com/googleapis/google-cloud-go/issues/10941)) ([0f0bf2d](https://github.com/googleapis/google-cloud-go/commit/0f0bf2d18c97dd8b65bcf0099f0802b5631c6287))
## [0.9.7](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.6...auth/v0.9.7) (2024-10-01)
### Bug Fixes
* **auth:** Restore support for non-default service accounts for DirectPath ([#10937](https://github.com/googleapis/google-cloud-go/issues/10937)) ([a38650e](https://github.com/googleapis/google-cloud-go/commit/a38650edbf420223077498cafa537aec74b37aad)), refs [#10907](https://github.com/googleapis/google-cloud-go/issues/10907)
## [0.9.6](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.5...auth/v0.9.6) (2024-09-30)
### Bug Fixes
* **auth:** Make aws credentials provider retrieve fresh credentials ([#10920](https://github.com/googleapis/google-cloud-go/issues/10920)) ([250fbf8](https://github.com/googleapis/google-cloud-go/commit/250fbf87d858d865e399a241b7e537c4ff0c3dd8))
## [0.9.5](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.4...auth/v0.9.5) (2024-09-25)
### Bug Fixes
* **auth:** Restore support for GOOGLE_CLOUD_UNIVERSE_DOMAIN env ([#10915](https://github.com/googleapis/google-cloud-go/issues/10915)) ([94caaaa](https://github.com/googleapis/google-cloud-go/commit/94caaaa061362d0e00ef6214afcc8a0a3e7ebfb2))
* **auth:** Skip directpath credentials overwrite when it's not on GCE ([#10833](https://github.com/googleapis/google-cloud-go/issues/10833)) ([7e5e8d1](https://github.com/googleapis/google-cloud-go/commit/7e5e8d10b761b0a6e43e19a028528db361bc07b1))
* **auth:** Use new context for non-blocking token refresh ([#10919](https://github.com/googleapis/google-cloud-go/issues/10919)) ([cf7102d](https://github.com/googleapis/google-cloud-go/commit/cf7102d33a21be1e5a9d47a49456b3a57c43b350))
## [0.9.4](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.3...auth/v0.9.4) (2024-09-11)
### Bug Fixes
* **auth:** Enable self-signed JWT for non-GDU universe domain ([#10831](https://github.com/googleapis/google-cloud-go/issues/10831)) ([f9869f7](https://github.com/googleapis/google-cloud-go/commit/f9869f7903cfd34d1b97c25d0dc5669d2c5138e6))
## [0.9.3](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.2...auth/v0.9.3) (2024-09-03)
### Bug Fixes
* **auth:** Choose quota project envvar over file when both present ([#10807](https://github.com/googleapis/google-cloud-go/issues/10807)) ([2d8dd77](https://github.com/googleapis/google-cloud-go/commit/2d8dd7700eff92d4b95027be55e26e1e7aa79181)), refs [#10804](https://github.com/googleapis/google-cloud-go/issues/10804)
## [0.9.2](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.1...auth/v0.9.2) (2024-08-30)
### Bug Fixes
* **auth:** Handle non-Transport DefaultTransport ([#10733](https://github.com/googleapis/google-cloud-go/issues/10733)) ([98d91dc](https://github.com/googleapis/google-cloud-go/commit/98d91dc8316b247498fab41ab35e57a0446fe556)), refs [#10742](https://github.com/googleapis/google-cloud-go/issues/10742)
* **auth:** Make sure quota option takes precedence over env/file ([#10797](https://github.com/googleapis/google-cloud-go/issues/10797)) ([f1b050d](https://github.com/googleapis/google-cloud-go/commit/f1b050d56d804b245cab048c2980d32b0eaceb4e)), refs [#10795](https://github.com/googleapis/google-cloud-go/issues/10795)
### Documentation
* **auth:** Fix Go doc comment link ([#10751](https://github.com/googleapis/google-cloud-go/issues/10751)) ([015acfa](https://github.com/googleapis/google-cloud-go/commit/015acfab4d172650928bb1119bc2cd6307b9a437))
## [0.9.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.9.0...auth/v0.9.1) (2024-08-22)
### Bug Fixes
* **auth:** Setting expireEarly to default when the value is 0 ([#10732](https://github.com/googleapis/google-cloud-go/issues/10732)) ([5e67869](https://github.com/googleapis/google-cloud-go/commit/5e67869a31e9e8ecb4eeebd2cfa11a761c3b1948))
## [0.9.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.8.1...auth/v0.9.0) (2024-08-16)
### Features
* **auth:** Auth library can talk to S2A over mTLS ([#10634](https://github.com/googleapis/google-cloud-go/issues/10634)) ([5250a13](https://github.com/googleapis/google-cloud-go/commit/5250a13ec95b8d4eefbe0158f82857ff2189cb45))
## [0.8.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.8.0...auth/v0.8.1) (2024-08-13)
### Bug Fixes
* **auth:** Make default client creation more lenient ([#10669](https://github.com/googleapis/google-cloud-go/issues/10669)) ([1afb9ee](https://github.com/googleapis/google-cloud-go/commit/1afb9ee1ee9de9810722800018133304a0ca34d1)), refs [#10638](https://github.com/googleapis/google-cloud-go/issues/10638)
## [0.8.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.7.3...auth/v0.8.0) (2024-08-07)
### Features
* **auth:** Adds support for X509 workload identity federation ([#10373](https://github.com/googleapis/google-cloud-go/issues/10373)) ([5d07505](https://github.com/googleapis/google-cloud-go/commit/5d075056cbe27bb1da4072a26070c41f8999eb9b))
## [0.7.3](https://github.com/googleapis/google-cloud-go/compare/auth/v0.7.2...auth/v0.7.3) (2024-08-01)
### Bug Fixes
* **auth/oauth2adapt:** Update dependencies ([257c40b](https://github.com/googleapis/google-cloud-go/commit/257c40bd6d7e59730017cf32bda8823d7a232758))
* **auth:** Disable automatic universe domain check for MDS ([#10620](https://github.com/googleapis/google-cloud-go/issues/10620)) ([7cea5ed](https://github.com/googleapis/google-cloud-go/commit/7cea5edd5a0c1e6bca558696f5607879141910e8))
* **auth:** Update dependencies ([257c40b](https://github.com/googleapis/google-cloud-go/commit/257c40bd6d7e59730017cf32bda8823d7a232758))
## [0.7.2](https://github.com/googleapis/google-cloud-go/compare/auth/v0.7.1...auth/v0.7.2) (2024-07-22)
### Bug Fixes
* **auth:** Use default client for universe metadata lookup ([#10551](https://github.com/googleapis/google-cloud-go/issues/10551)) ([d9046fd](https://github.com/googleapis/google-cloud-go/commit/d9046fdd1435d1ce48f374806c1def4cb5ac6cd3)), refs [#10544](https://github.com/googleapis/google-cloud-go/issues/10544)
## [0.7.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.7.0...auth/v0.7.1) (2024-07-10)
### Bug Fixes
* **auth:** Bump google.golang.org/grpc@v1.64.1 ([8ecc4e9](https://github.com/googleapis/google-cloud-go/commit/8ecc4e9622e5bbe9b90384d5848ab816027226c5))
## [0.7.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.6.1...auth/v0.7.0) (2024-07-09)
### Features
* **auth:** Add workload X509 cert provider as a default cert provider ([#10479](https://github.com/googleapis/google-cloud-go/issues/10479)) ([c51ee6c](https://github.com/googleapis/google-cloud-go/commit/c51ee6cf65ce05b4d501083e49d468c75ac1ea63))
### Bug Fixes
* **auth/oauth2adapt:** Bump google.golang.org/api@v0.187.0 ([8fa9e39](https://github.com/googleapis/google-cloud-go/commit/8fa9e398e512fd8533fd49060371e61b5725a85b))
* **auth:** Bump google.golang.org/api@v0.187.0 ([8fa9e39](https://github.com/googleapis/google-cloud-go/commit/8fa9e398e512fd8533fd49060371e61b5725a85b))
* **auth:** Check len of slices, not non-nil ([#10483](https://github.com/googleapis/google-cloud-go/issues/10483)) ([0a966a1](https://github.com/googleapis/google-cloud-go/commit/0a966a183e5f0e811977216d736d875b7233e942))
## [0.6.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.6.0...auth/v0.6.1) (2024-07-01)
### Bug Fixes
* **auth:** Support gRPC API keys ([#10460](https://github.com/googleapis/google-cloud-go/issues/10460)) ([daa6646](https://github.com/googleapis/google-cloud-go/commit/daa6646d2af5d7fb5b30489f4934c7db89868c7c))
* **auth:** Update http and grpc transports to support token exchange over mTLS ([#10397](https://github.com/googleapis/google-cloud-go/issues/10397)) ([c6dfdcf](https://github.com/googleapis/google-cloud-go/commit/c6dfdcf893c3f971eba15026c12db0a960ae81f2))
## [0.6.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.5.2...auth/v0.6.0) (2024-06-25)
### Features
* **auth:** Add non-blocking token refresh for compute MDS ([#10263](https://github.com/googleapis/google-cloud-go/issues/10263)) ([9ac350d](https://github.com/googleapis/google-cloud-go/commit/9ac350da11a49b8e2174d3fc5b1a5070fec78b4e))
### Bug Fixes
* **auth:** Return error if envvar detected file returns an error ([#10431](https://github.com/googleapis/google-cloud-go/issues/10431)) ([e52b9a7](https://github.com/googleapis/google-cloud-go/commit/e52b9a7c45468827f5d220ab00965191faeb9d05))
## [0.5.2](https://github.com/googleapis/google-cloud-go/compare/auth/v0.5.1...auth/v0.5.2) (2024-06-24)
### Bug Fixes
* **auth:** Fetch initial token when CachedTokenProviderOptions.DisableAutoRefresh is true ([#10415](https://github.com/googleapis/google-cloud-go/issues/10415)) ([3266763](https://github.com/googleapis/google-cloud-go/commit/32667635ca2efad05cd8c087c004ca07d7406913)), refs [#10414](https://github.com/googleapis/google-cloud-go/issues/10414)
## [0.5.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.5.0...auth/v0.5.1) (2024-05-31)
### Bug Fixes
* **auth:** Pass through client to 2LO and 3LO flows ([#10290](https://github.com/googleapis/google-cloud-go/issues/10290)) ([685784e](https://github.com/googleapis/google-cloud-go/commit/685784ea84358c15e9214bdecb307d37aa3b6d2f))
## [0.5.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.4.2...auth/v0.5.0) (2024-05-28)
### Features
* **auth:** Adds X509 workload certificate provider ([#10233](https://github.com/googleapis/google-cloud-go/issues/10233)) ([17a9db7](https://github.com/googleapis/google-cloud-go/commit/17a9db73af35e3d1a7a25ac4fd1377a103de6150))
## [0.4.2](https://github.com/googleapis/google-cloud-go/compare/auth/v0.4.1...auth/v0.4.2) (2024-05-16)
### Bug Fixes
* **auth:** Enable client certificates by default only for GDU ([#10151](https://github.com/googleapis/google-cloud-go/issues/10151)) ([7c52978](https://github.com/googleapis/google-cloud-go/commit/7c529786275a39b7e00525f7d5e7be0d963e9e15))
* **auth:** Handle non-Transport DefaultTransport ([#10162](https://github.com/googleapis/google-cloud-go/issues/10162)) ([fa3bfdb](https://github.com/googleapis/google-cloud-go/commit/fa3bfdb23aaa45b34394a8b61e753b3587506782)), refs [#10159](https://github.com/googleapis/google-cloud-go/issues/10159)
* **auth:** Have refresh time match docs ([#10147](https://github.com/googleapis/google-cloud-go/issues/10147)) ([bcb5568](https://github.com/googleapis/google-cloud-go/commit/bcb5568c07a54dd3d2e869d15f502b0741a609e8))
* **auth:** Update compute token fetching error with named prefix ([#10180](https://github.com/googleapis/google-cloud-go/issues/10180)) ([4573504](https://github.com/googleapis/google-cloud-go/commit/4573504828d2928bebedc875d87650ba227829ea))
## [0.4.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.4.0...auth/v0.4.1) (2024-05-09)
### Bug Fixes
* **auth:** Don't try to detect default creds it opt configured ([#10143](https://github.com/googleapis/google-cloud-go/issues/10143)) ([804632e](https://github.com/googleapis/google-cloud-go/commit/804632e7c5b0b85ff522f7951114485e256eb5bc))
## [0.4.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.3.0...auth/v0.4.0) (2024-05-07)
### Features
* **auth:** Enable client certificates by default ([#10102](https://github.com/googleapis/google-cloud-go/issues/10102)) ([9013e52](https://github.com/googleapis/google-cloud-go/commit/9013e5200a6ec0f178ed91acb255481ffb073a2c))
### Bug Fixes
* **auth:** Get s2a logic up to date ([#10093](https://github.com/googleapis/google-cloud-go/issues/10093)) ([4fe9ae4](https://github.com/googleapis/google-cloud-go/commit/4fe9ae4b7101af2a5221d6d6b2e77b479305bb06))
## [0.3.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.2.2...auth/v0.3.0) (2024-04-23)
### Features
* **auth/httptransport:** Add ability to customize transport ([#10023](https://github.com/googleapis/google-cloud-go/issues/10023)) ([72c7f6b](https://github.com/googleapis/google-cloud-go/commit/72c7f6bbec3136cc7a62788fc7186bc33ef6c3b3)), refs [#9812](https://github.com/googleapis/google-cloud-go/issues/9812) [#9814](https://github.com/googleapis/google-cloud-go/issues/9814)
### Bug Fixes
* **auth/credentials:** Error on bad file name if explicitly set ([#10018](https://github.com/googleapis/google-cloud-go/issues/10018)) ([55beaa9](https://github.com/googleapis/google-cloud-go/commit/55beaa993aaf052d8be39766afc6777c3c2a0bdd)), refs [#9809](https://github.com/googleapis/google-cloud-go/issues/9809)
## [0.2.2](https://github.com/googleapis/google-cloud-go/compare/auth/v0.2.1...auth/v0.2.2) (2024-04-19)
### Bug Fixes
* **auth:** Add internal opt to skip validation on transports ([#9999](https://github.com/googleapis/google-cloud-go/issues/9999)) ([9e20ef8](https://github.com/googleapis/google-cloud-go/commit/9e20ef89f6287d6bd03b8697d5898dc43b4a77cf)), refs [#9823](https://github.com/googleapis/google-cloud-go/issues/9823)
* **auth:** Set secure flag for gRPC conn pools ([#10002](https://github.com/googleapis/google-cloud-go/issues/10002)) ([14e3956](https://github.com/googleapis/google-cloud-go/commit/14e3956dfd736399731b5ee8d9b178ae085cf7ba)), refs [#9833](https://github.com/googleapis/google-cloud-go/issues/9833)
## [0.2.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.2.0...auth/v0.2.1) (2024-04-18)
### Bug Fixes
* **auth:** Default gRPC token type to Bearer if not set ([#9800](https://github.com/googleapis/google-cloud-go/issues/9800)) ([5284066](https://github.com/googleapis/google-cloud-go/commit/5284066670b6fe65d79089cfe0199c9660f87fc7))
## [0.2.0](https://github.com/googleapis/google-cloud-go/compare/auth/v0.1.1...auth/v0.2.0) (2024-04-15)
### Breaking Changes
In the below mentioned commits there were a few large breaking changes since the
last release of the module.
1. The `Credentials` type has been moved to the root of the module as it is
becoming the core abstraction for the whole module.
2. Because of the above mentioned change many functions that previously
returned a `TokenProvider` now return `Credentials`. Similarly, these
functions have been renamed to be more specific.
3. Most places that used to take an optional `TokenProvider` now accept
`Credentials`. You can make a `Credentials` from a `TokenProvider` using the
constructor found in the `auth` package.
4. The `detect` package has been renamed to `credentials`. With this change some
function signatures were also updated for better readability.
5. Derivative auth flows like `impersonate` and `downscope` have been moved to
be under the new `credentials` package.
Although these changes are disruptive we think that they are for the best of the
long-term health of the module. We do not expect any more large breaking changes
like these in future revisions, even before 1.0.0. This version will be the
first version of the auth library that our client libraries start to use and
depend on.
### Features
* **auth/credentials/externalaccount:** Add default TokenURL ([#9700](https://github.com/googleapis/google-cloud-go/issues/9700)) ([81830e6](https://github.com/googleapis/google-cloud-go/commit/81830e6848ceefd055aa4d08f933d1154455a0f6))
* **auth:** Add downscope.Options.UniverseDomain ([#9634](https://github.com/googleapis/google-cloud-go/issues/9634)) ([52cf7d7](https://github.com/googleapis/google-cloud-go/commit/52cf7d780853594291c4e34302d618299d1f5a1d))
* **auth:** Add universe domain to grpctransport and httptransport ([#9663](https://github.com/googleapis/google-cloud-go/issues/9663)) ([67d353b](https://github.com/googleapis/google-cloud-go/commit/67d353beefe3b607c08c891876fbd95ab89e5fe3)), refs [#9670](https://github.com/googleapis/google-cloud-go/issues/9670)
* **auth:** Add UniverseDomain to DetectOptions ([#9536](https://github.com/googleapis/google-cloud-go/issues/9536)) ([3618d3f](https://github.com/googleapis/google-cloud-go/commit/3618d3f7061615c0e189f376c75abc201203b501))
* **auth:** Make package externalaccount public ([#9633](https://github.com/googleapis/google-cloud-go/issues/9633)) ([a0978d8](https://github.com/googleapis/google-cloud-go/commit/a0978d8e96968399940ebd7d092539772bf9caac))
* **auth:** Move credentials to base auth package ([#9590](https://github.com/googleapis/google-cloud-go/issues/9590)) ([1a04baf](https://github.com/googleapis/google-cloud-go/commit/1a04bafa83c27342b9308d785645e1e5423ea10d))
* **auth:** Refactor public sigs to use Credentials ([#9603](https://github.com/googleapis/google-cloud-go/issues/9603)) ([69cb240](https://github.com/googleapis/google-cloud-go/commit/69cb240c530b1f7173a9af2555c19e9a1beb56c5))
### Bug Fixes
* **auth/oauth2adapt:** Update protobuf dep to v1.33.0 ([30b038d](https://github.com/googleapis/google-cloud-go/commit/30b038d8cac0b8cd5dd4761c87f3f298760dd33a))
* **auth:** Fix uint32 conversion ([9221c7f](https://github.com/googleapis/google-cloud-go/commit/9221c7fa12cef9d5fb7ddc92f41f1d6204971c7b))
* **auth:** Port sts expires fix ([#9618](https://github.com/googleapis/google-cloud-go/issues/9618)) ([7bec97b](https://github.com/googleapis/google-cloud-go/commit/7bec97b2f51ed3ac4f9b88bf100d301da3f5d1bd))
* **auth:** Read universe_domain from all credentials files ([#9632](https://github.com/googleapis/google-cloud-go/issues/9632)) ([16efbb5](https://github.com/googleapis/google-cloud-go/commit/16efbb52e39ea4a319e5ee1e95c0e0305b6d9824))
* **auth:** Remove content-type header from idms get requests ([#9508](https://github.com/googleapis/google-cloud-go/issues/9508)) ([8589f41](https://github.com/googleapis/google-cloud-go/commit/8589f41599d265d7c3d46a3d86c9fab2329cbdd9))
* **auth:** Update protobuf dep to v1.33.0 ([30b038d](https://github.com/googleapis/google-cloud-go/commit/30b038d8cac0b8cd5dd4761c87f3f298760dd33a))
## [0.1.1](https://github.com/googleapis/google-cloud-go/compare/auth/v0.1.0...auth/v0.1.1) (2024-03-10)
### Bug Fixes
* **auth/impersonate:** Properly send default detect params ([#9529](https://github.com/googleapis/google-cloud-go/issues/9529)) ([5b6b8be](https://github.com/googleapis/google-cloud-go/commit/5b6b8bef577f82707e51f5cc5d258d5bdf90218f)), refs [#9136](https://github.com/googleapis/google-cloud-go/issues/9136)
* **auth:** Update grpc-go to v1.56.3 ([343cea8](https://github.com/googleapis/google-cloud-go/commit/343cea8c43b1e31ae21ad50ad31d3b0b60143f8c))
* **auth:** Update grpc-go to v1.59.0 ([81a97b0](https://github.com/googleapis/google-cloud-go/commit/81a97b06cb28b25432e4ece595c55a9857e960b7))
## 0.1.0 (2023-10-18)
### Features
* **auth:** Add base auth package ([#8465](https://github.com/googleapis/google-cloud-go/issues/8465)) ([6a45f26](https://github.com/googleapis/google-cloud-go/commit/6a45f26b809b64edae21f312c18d4205f96b180e))
* **auth:** Add cert support to httptransport ([#8569](https://github.com/googleapis/google-cloud-go/issues/8569)) ([37e3435](https://github.com/googleapis/google-cloud-go/commit/37e3435f8e98595eafab481bdfcb31a4c56fa993))
* **auth:** Add Credentials.UniverseDomain() ([#8654](https://github.com/googleapis/google-cloud-go/issues/8654)) ([af0aa1e](https://github.com/googleapis/google-cloud-go/commit/af0aa1ed8015bc8fe0dd87a7549ae029107cbdb8))
* **auth:** Add detect package ([#8491](https://github.com/googleapis/google-cloud-go/issues/8491)) ([d977419](https://github.com/googleapis/google-cloud-go/commit/d977419a3269f6acc193df77a2136a6eb4b4add7))
* **auth:** Add downscope package ([#8532](https://github.com/googleapis/google-cloud-go/issues/8532)) ([dda9bff](https://github.com/googleapis/google-cloud-go/commit/dda9bff8ec70e6d104901b4105d13dcaa4e2404c))
* **auth:** Add grpctransport package ([#8625](https://github.com/googleapis/google-cloud-go/issues/8625)) ([69a8347](https://github.com/googleapis/google-cloud-go/commit/69a83470bdcc7ed10c6c36d1abc3b7cfdb8a0ee5))
* **auth:** Add httptransport package ([#8567](https://github.com/googleapis/google-cloud-go/issues/8567)) ([6898597](https://github.com/googleapis/google-cloud-go/commit/6898597d2ea95d630fcd00fd15c58c75ea843bff))
* **auth:** Add idtoken package ([#8580](https://github.com/googleapis/google-cloud-go/issues/8580)) ([a79e693](https://github.com/googleapis/google-cloud-go/commit/a79e693e97e4e3e1c6742099af3dbc58866d88fe))
* **auth:** Add impersonate package ([#8578](https://github.com/googleapis/google-cloud-go/issues/8578)) ([e29ba0c](https://github.com/googleapis/google-cloud-go/commit/e29ba0cb7bd3888ab9e808087027dc5a32474c04))
* **auth:** Add support for external accounts in detect ([#8508](https://github.com/googleapis/google-cloud-go/issues/8508)) ([62210d5](https://github.com/googleapis/google-cloud-go/commit/62210d5d3e56e8e9f35db8e6ac0defec19582507))
* **auth:** Port external account changes ([#8697](https://github.com/googleapis/google-cloud-go/issues/8697)) ([5823db5](https://github.com/googleapis/google-cloud-go/commit/5823db5d633069999b58b9131a7f9cd77e82c899))
### Bug Fixes
* **auth/oauth2adapt:** Update golang.org/x/net to v0.17.0 ([174da47](https://github.com/googleapis/google-cloud-go/commit/174da47254fefb12921bbfc65b7829a453af6f5d))
* **auth:** Update golang.org/x/net to v0.17.0 ([174da47](https://github.com/googleapis/google-cloud-go/commit/174da47254fefb12921bbfc65b7829a453af6f5d))

202
vendor/cloud.google.com/go/auth/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

40
vendor/cloud.google.com/go/auth/README.md generated vendored Normal file
View File

@@ -0,0 +1,40 @@
# Google Auth Library for Go
[![Go Reference](https://pkg.go.dev/badge/cloud.google.com/go/auth.svg)](https://pkg.go.dev/cloud.google.com/go/auth)
## Install
``` bash
go get cloud.google.com/go/auth@latest
```
## Usage
The most common way this library is used is transitively, by default, from any
of our Go client libraries.
### Notable use-cases
- To create a credential directly please see examples in the
[credentials](https://pkg.go.dev/cloud.google.com/go/auth/credentials)
package.
- To create a authenticated HTTP client please see examples in the
[httptransport](https://pkg.go.dev/cloud.google.com/go/auth/httptransport)
package.
- To create a authenticated gRPC connection please see examples in the
[grpctransport](https://pkg.go.dev/cloud.google.com/go/auth/grpctransport)
package.
- To create an ID token please see examples in the
[idtoken](https://pkg.go.dev/cloud.google.com/go/auth/credentials/idtoken)
package.
## Contributing
Contributions are welcome. Please, see the
[CONTRIBUTING](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md)
document for details.
Please note that this project is released with a Contributor Code of Conduct.
By participating in this project you agree to abide by its terms.
See [Contributor Code of Conduct](https://github.com/GoogleCloudPlatform/google-cloud-go/blob/main/CONTRIBUTING.md#contributor-code-of-conduct)
for more information.

618
vendor/cloud.google.com/go/auth/auth.go generated vendored Normal file
View File

@@ -0,0 +1,618 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package auth provides utilities for managing Google Cloud credentials,
// including functionality for creating, caching, and refreshing OAuth2 tokens.
// It offers customizable options for different OAuth2 flows, such as 2-legged
// (2LO) and 3-legged (3LO) OAuth, along with support for PKCE and automatic
// token management.
package auth
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net/http"
"net/url"
"strings"
"sync"
"time"
"cloud.google.com/go/auth/internal"
"cloud.google.com/go/auth/internal/jwt"
"github.com/googleapis/gax-go/v2/internallog"
)
const (
// Parameter keys for AuthCodeURL method to support PKCE.
codeChallengeKey = "code_challenge"
codeChallengeMethodKey = "code_challenge_method"
// Parameter key for Exchange method to support PKCE.
codeVerifierKey = "code_verifier"
// 3 minutes and 45 seconds before expiration. The shortest MDS cache is 4 minutes,
// so we give it 15 seconds to refresh it's cache before attempting to refresh a token.
defaultExpiryDelta = 225 * time.Second
universeDomainDefault = "googleapis.com"
)
// tokenState represents different states for a [Token].
type tokenState int
const (
// fresh indicates that the [Token] is valid. It is not expired or close to
// expired, or the token has no expiry.
fresh tokenState = iota
// stale indicates that the [Token] is close to expired, and should be
// refreshed. The token can be used normally.
stale
// invalid indicates that the [Token] is expired or invalid. The token
// cannot be used for a normal operation.
invalid
)
var (
defaultGrantType = "urn:ietf:params:oauth:grant-type:jwt-bearer"
defaultHeader = &jwt.Header{Algorithm: jwt.HeaderAlgRSA256, Type: jwt.HeaderType}
// for testing
timeNow = time.Now
)
// TokenProvider specifies an interface for anything that can return a token.
type TokenProvider interface {
// Token returns a Token or an error.
// The Token returned must be safe to use
// concurrently.
// The returned Token must not be modified.
// The context provided must be sent along to any requests that are made in
// the implementing code.
Token(context.Context) (*Token, error)
}
// Token holds the credential token used to authorized requests. All fields are
// considered read-only.
type Token struct {
// Value is the token used to authorize requests. It is usually an access
// token but may be other types of tokens such as ID tokens in some flows.
Value string
// Type is the type of token Value is. If uninitialized, it should be
// assumed to be a "Bearer" token.
Type string
// Expiry is the time the token is set to expire.
Expiry time.Time
// Metadata may include, but is not limited to, the body of the token
// response returned by the server.
Metadata map[string]interface{} // TODO(codyoss): maybe make a method to flatten metadata to avoid []string for url.Values
}
// IsValid reports that a [Token] is non-nil, has a [Token.Value], and has not
// expired. A token is considered expired if [Token.Expiry] has passed or will
// pass in the next 225 seconds.
func (t *Token) IsValid() bool {
return t.isValidWithEarlyExpiry(defaultExpiryDelta)
}
// MetadataString is a convenience method for accessing string values in the
// token's metadata. Returns an empty string if the metadata is nil or the value
// for the given key cannot be cast to a string.
func (t *Token) MetadataString(k string) string {
if t.Metadata == nil {
return ""
}
s, ok := t.Metadata[k].(string)
if !ok {
return ""
}
return s
}
func (t *Token) isValidWithEarlyExpiry(earlyExpiry time.Duration) bool {
if t.isEmpty() {
return false
}
if t.Expiry.IsZero() {
return true
}
return !t.Expiry.Round(0).Add(-earlyExpiry).Before(timeNow())
}
func (t *Token) isEmpty() bool {
return t == nil || t.Value == ""
}
// Credentials holds Google credentials, including
// [Application Default Credentials].
//
// [Application Default Credentials]: https://developers.google.com/accounts/docs/application-default-credentials
type Credentials struct {
json []byte
projectID CredentialsPropertyProvider
quotaProjectID CredentialsPropertyProvider
// universeDomain is the default service domain for a given Cloud universe.
universeDomain CredentialsPropertyProvider
TokenProvider
}
// JSON returns the bytes associated with the the file used to source
// credentials if one was used.
func (c *Credentials) JSON() []byte {
return c.json
}
// ProjectID returns the associated project ID from the underlying file or
// environment.
func (c *Credentials) ProjectID(ctx context.Context) (string, error) {
if c.projectID == nil {
return internal.GetProjectID(c.json, ""), nil
}
v, err := c.projectID.GetProperty(ctx)
if err != nil {
return "", err
}
return internal.GetProjectID(c.json, v), nil
}
// QuotaProjectID returns the associated quota project ID from the underlying
// file or environment.
func (c *Credentials) QuotaProjectID(ctx context.Context) (string, error) {
if c.quotaProjectID == nil {
return internal.GetQuotaProject(c.json, ""), nil
}
v, err := c.quotaProjectID.GetProperty(ctx)
if err != nil {
return "", err
}
return internal.GetQuotaProject(c.json, v), nil
}
// UniverseDomain returns the default service domain for a given Cloud universe.
// The default value is "googleapis.com".
func (c *Credentials) UniverseDomain(ctx context.Context) (string, error) {
if c.universeDomain == nil {
return universeDomainDefault, nil
}
v, err := c.universeDomain.GetProperty(ctx)
if err != nil {
return "", err
}
if v == "" {
return universeDomainDefault, nil
}
return v, err
}
// CredentialsPropertyProvider provides an implementation to fetch a property
// value for [Credentials].
type CredentialsPropertyProvider interface {
GetProperty(context.Context) (string, error)
}
// CredentialsPropertyFunc is a type adapter to allow the use of ordinary
// functions as a [CredentialsPropertyProvider].
type CredentialsPropertyFunc func(context.Context) (string, error)
// GetProperty loads the properly value provided the given context.
func (p CredentialsPropertyFunc) GetProperty(ctx context.Context) (string, error) {
return p(ctx)
}
// CredentialsOptions are used to configure [Credentials].
type CredentialsOptions struct {
// TokenProvider is a means of sourcing a token for the credentials. Required.
TokenProvider TokenProvider
// JSON is the raw contents of the credentials file if sourced from a file.
JSON []byte
// ProjectIDProvider resolves the project ID associated with the
// credentials.
ProjectIDProvider CredentialsPropertyProvider
// QuotaProjectIDProvider resolves the quota project ID associated with the
// credentials.
QuotaProjectIDProvider CredentialsPropertyProvider
// UniverseDomainProvider resolves the universe domain with the credentials.
UniverseDomainProvider CredentialsPropertyProvider
}
// NewCredentials returns new [Credentials] from the provided options.
func NewCredentials(opts *CredentialsOptions) *Credentials {
creds := &Credentials{
TokenProvider: opts.TokenProvider,
json: opts.JSON,
projectID: opts.ProjectIDProvider,
quotaProjectID: opts.QuotaProjectIDProvider,
universeDomain: opts.UniverseDomainProvider,
}
return creds
}
// CachedTokenProviderOptions provides options for configuring a cached
// [TokenProvider].
type CachedTokenProviderOptions struct {
// DisableAutoRefresh makes the TokenProvider always return the same token,
// even if it is expired. The default is false. Optional.
DisableAutoRefresh bool
// ExpireEarly configures the amount of time before a token expires, that it
// should be refreshed. If unset, the default value is 3 minutes and 45
// seconds. Optional.
ExpireEarly time.Duration
// DisableAsyncRefresh configures a synchronous workflow that refreshes
// tokens in a blocking manner. The default is false. Optional.
DisableAsyncRefresh bool
}
func (ctpo *CachedTokenProviderOptions) autoRefresh() bool {
if ctpo == nil {
return true
}
return !ctpo.DisableAutoRefresh
}
func (ctpo *CachedTokenProviderOptions) expireEarly() time.Duration {
if ctpo == nil || ctpo.ExpireEarly == 0 {
return defaultExpiryDelta
}
return ctpo.ExpireEarly
}
func (ctpo *CachedTokenProviderOptions) blockingRefresh() bool {
if ctpo == nil {
return false
}
return ctpo.DisableAsyncRefresh
}
// NewCachedTokenProvider wraps a [TokenProvider] to cache the tokens returned
// by the underlying provider. By default it will refresh tokens asynchronously
// a few minutes before they expire.
func NewCachedTokenProvider(tp TokenProvider, opts *CachedTokenProviderOptions) TokenProvider {
if ctp, ok := tp.(*cachedTokenProvider); ok {
return ctp
}
return &cachedTokenProvider{
tp: tp,
autoRefresh: opts.autoRefresh(),
expireEarly: opts.expireEarly(),
blockingRefresh: opts.blockingRefresh(),
}
}
type cachedTokenProvider struct {
tp TokenProvider
autoRefresh bool
expireEarly time.Duration
blockingRefresh bool
mu sync.Mutex
cachedToken *Token
// isRefreshRunning ensures that the non-blocking refresh will only be
// attempted once, even if multiple callers enter the Token method.
isRefreshRunning bool
// isRefreshErr ensures that the non-blocking refresh will only be attempted
// once per refresh window if an error is encountered.
isRefreshErr bool
}
func (c *cachedTokenProvider) Token(ctx context.Context) (*Token, error) {
if c.blockingRefresh {
return c.tokenBlocking(ctx)
}
return c.tokenNonBlocking(ctx)
}
func (c *cachedTokenProvider) tokenNonBlocking(ctx context.Context) (*Token, error) {
switch c.tokenState() {
case fresh:
c.mu.Lock()
defer c.mu.Unlock()
return c.cachedToken, nil
case stale:
// Call tokenAsync with a new Context because the user-provided context
// may have a short timeout incompatible with async token refresh.
c.tokenAsync(context.Background())
// Return the stale token immediately to not block customer requests to Cloud services.
c.mu.Lock()
defer c.mu.Unlock()
return c.cachedToken, nil
default: // invalid
return c.tokenBlocking(ctx)
}
}
// tokenState reports the token's validity.
func (c *cachedTokenProvider) tokenState() tokenState {
c.mu.Lock()
defer c.mu.Unlock()
t := c.cachedToken
now := timeNow()
if t == nil || t.Value == "" {
return invalid
} else if t.Expiry.IsZero() {
return fresh
} else if now.After(t.Expiry.Round(0)) {
return invalid
} else if now.After(t.Expiry.Round(0).Add(-c.expireEarly)) {
return stale
}
return fresh
}
// tokenAsync uses a bool to ensure that only one non-blocking token refresh
// happens at a time, even if multiple callers have entered this function
// concurrently. This avoids creating an arbitrary number of concurrent
// goroutines. Retries should be attempted and managed within the Token method.
// If the refresh attempt fails, no further attempts are made until the refresh
// window expires and the token enters the invalid state, at which point the
// blocking call to Token should likely return the same error on the main goroutine.
func (c *cachedTokenProvider) tokenAsync(ctx context.Context) {
fn := func() {
t, err := c.tp.Token(ctx)
c.mu.Lock()
defer c.mu.Unlock()
c.isRefreshRunning = false
if err != nil {
// Discard errors from the non-blocking refresh, but prevent further
// attempts.
c.isRefreshErr = true
return
}
c.cachedToken = t
}
c.mu.Lock()
defer c.mu.Unlock()
if !c.isRefreshRunning && !c.isRefreshErr {
c.isRefreshRunning = true
go fn()
}
}
func (c *cachedTokenProvider) tokenBlocking(ctx context.Context) (*Token, error) {
c.mu.Lock()
defer c.mu.Unlock()
c.isRefreshErr = false
if c.cachedToken.IsValid() || (!c.autoRefresh && !c.cachedToken.isEmpty()) {
return c.cachedToken, nil
}
t, err := c.tp.Token(ctx)
if err != nil {
return nil, err
}
c.cachedToken = t
return t, nil
}
// Error is a error associated with retrieving a [Token]. It can hold useful
// additional details for debugging.
type Error struct {
// Response is the HTTP response associated with error. The body will always
// be already closed and consumed.
Response *http.Response
// Body is the HTTP response body.
Body []byte
// Err is the underlying wrapped error.
Err error
// code returned in the token response
code string
// description returned in the token response
description string
// uri returned in the token response
uri string
}
func (e *Error) Error() string {
if e.code != "" {
s := fmt.Sprintf("auth: %q", e.code)
if e.description != "" {
s += fmt.Sprintf(" %q", e.description)
}
if e.uri != "" {
s += fmt.Sprintf(" %q", e.uri)
}
return s
}
return fmt.Sprintf("auth: cannot fetch token: %v\nResponse: %s", e.Response.StatusCode, e.Body)
}
// Temporary returns true if the error is considered temporary and may be able
// to be retried.
func (e *Error) Temporary() bool {
if e.Response == nil {
return false
}
sc := e.Response.StatusCode
return sc == http.StatusInternalServerError || sc == http.StatusServiceUnavailable || sc == http.StatusRequestTimeout || sc == http.StatusTooManyRequests
}
func (e *Error) Unwrap() error {
return e.Err
}
// Style describes how the token endpoint wants to receive the ClientID and
// ClientSecret.
type Style int
const (
// StyleUnknown means the value has not been initiated. Sending this in
// a request will cause the token exchange to fail.
StyleUnknown Style = iota
// StyleInParams sends client info in the body of a POST request.
StyleInParams
// StyleInHeader sends client info using Basic Authorization header.
StyleInHeader
)
// Options2LO is the configuration settings for doing a 2-legged JWT OAuth2 flow.
type Options2LO struct {
// Email is the OAuth2 client ID. This value is set as the "iss" in the
// JWT.
Email string
// PrivateKey contains the contents of an RSA private key or the
// contents of a PEM file that contains a private key. It is used to sign
// the JWT created.
PrivateKey []byte
// TokenURL is th URL the JWT is sent to. Required.
TokenURL string
// PrivateKeyID is the ID of the key used to sign the JWT. It is used as the
// "kid" in the JWT header. Optional.
PrivateKeyID string
// Subject is the used for to impersonate a user. It is used as the "sub" in
// the JWT.m Optional.
Subject string
// Scopes specifies requested permissions for the token. Optional.
Scopes []string
// Expires specifies the lifetime of the token. Optional.
Expires time.Duration
// Audience specifies the "aud" in the JWT. Optional.
Audience string
// PrivateClaims allows specifying any custom claims for the JWT. Optional.
PrivateClaims map[string]interface{}
// UniverseDomain is the default service domain for a given Cloud universe.
UniverseDomain string
// Client is the client to be used to make the underlying token requests.
// Optional.
Client *http.Client
// UseIDToken requests that the token returned be an ID token if one is
// returned from the server. Optional.
UseIDToken bool
// Logger is used for debug logging. If provided, logging will be enabled
// at the loggers configured level. By default logging is disabled unless
// enabled by setting GOOGLE_SDK_GO_LOGGING_LEVEL in which case a default
// logger will be used. Optional.
Logger *slog.Logger
}
func (o *Options2LO) client() *http.Client {
if o.Client != nil {
return o.Client
}
return internal.DefaultClient()
}
func (o *Options2LO) validate() error {
if o == nil {
return errors.New("auth: options must be provided")
}
if o.Email == "" {
return errors.New("auth: email must be provided")
}
if len(o.PrivateKey) == 0 {
return errors.New("auth: private key must be provided")
}
if o.TokenURL == "" {
return errors.New("auth: token URL must be provided")
}
return nil
}
// New2LOTokenProvider returns a [TokenProvider] from the provided options.
func New2LOTokenProvider(opts *Options2LO) (TokenProvider, error) {
if err := opts.validate(); err != nil {
return nil, err
}
return tokenProvider2LO{opts: opts, Client: opts.client(), logger: internallog.New(opts.Logger)}, nil
}
type tokenProvider2LO struct {
opts *Options2LO
Client *http.Client
logger *slog.Logger
}
func (tp tokenProvider2LO) Token(ctx context.Context) (*Token, error) {
pk, err := internal.ParseKey(tp.opts.PrivateKey)
if err != nil {
return nil, err
}
claimSet := &jwt.Claims{
Iss: tp.opts.Email,
Scope: strings.Join(tp.opts.Scopes, " "),
Aud: tp.opts.TokenURL,
AdditionalClaims: tp.opts.PrivateClaims,
Sub: tp.opts.Subject,
}
if t := tp.opts.Expires; t > 0 {
claimSet.Exp = time.Now().Add(t).Unix()
}
if aud := tp.opts.Audience; aud != "" {
claimSet.Aud = aud
}
h := *defaultHeader
h.KeyID = tp.opts.PrivateKeyID
payload, err := jwt.EncodeJWS(&h, claimSet, pk)
if err != nil {
return nil, err
}
v := url.Values{}
v.Set("grant_type", defaultGrantType)
v.Set("assertion", payload)
req, err := http.NewRequestWithContext(ctx, "POST", tp.opts.TokenURL, strings.NewReader(v.Encode()))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
tp.logger.DebugContext(ctx, "2LO token request", "request", internallog.HTTPRequest(req, []byte(v.Encode())))
resp, body, err := internal.DoRequest(tp.Client, req)
if err != nil {
return nil, fmt.Errorf("auth: cannot fetch token: %w", err)
}
tp.logger.DebugContext(ctx, "2LO token response", "response", internallog.HTTPResponse(resp, body))
if c := resp.StatusCode; c < http.StatusOK || c >= http.StatusMultipleChoices {
return nil, &Error{
Response: resp,
Body: body,
}
}
// tokenRes is the JSON response body.
var tokenRes struct {
AccessToken string `json:"access_token"`
TokenType string `json:"token_type"`
IDToken string `json:"id_token"`
ExpiresIn int64 `json:"expires_in"`
}
if err := json.Unmarshal(body, &tokenRes); err != nil {
return nil, fmt.Errorf("auth: cannot fetch token: %w", err)
}
token := &Token{
Value: tokenRes.AccessToken,
Type: tokenRes.TokenType,
}
token.Metadata = make(map[string]interface{})
json.Unmarshal(body, &token.Metadata) // no error checks for optional fields
if secs := tokenRes.ExpiresIn; secs > 0 {
token.Expiry = time.Now().Add(time.Duration(secs) * time.Second)
}
if v := tokenRes.IDToken; v != "" {
// decode returned id token to get expiry
claimSet, err := jwt.DecodeJWS(v)
if err != nil {
return nil, fmt.Errorf("auth: error decoding JWT token: %w", err)
}
token.Expiry = time.Unix(claimSet.Exp, 0)
}
if tp.opts.UseIDToken {
if tokenRes.IDToken == "" {
return nil, fmt.Errorf("auth: response doesn't have JWT token")
}
token.Value = tokenRes.IDToken
}
return token, nil
}

102
vendor/cloud.google.com/go/auth/credentials/compute.go generated vendored Normal file
View File

@@ -0,0 +1,102 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package credentials
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/url"
"strings"
"time"
"cloud.google.com/go/auth"
"cloud.google.com/go/compute/metadata"
)
var (
computeTokenMetadata = map[string]interface{}{
"auth.google.tokenSource": "compute-metadata",
"auth.google.serviceAccount": "default",
}
computeTokenURI = "instance/service-accounts/default/token"
)
// computeTokenProvider creates a [cloud.google.com/go/auth.TokenProvider] that
// uses the metadata service to retrieve tokens.
func computeTokenProvider(opts *DetectOptions, client *metadata.Client) auth.TokenProvider {
return auth.NewCachedTokenProvider(&computeProvider{
scopes: opts.Scopes,
client: client,
tokenBindingType: opts.TokenBindingType,
}, &auth.CachedTokenProviderOptions{
ExpireEarly: opts.EarlyTokenRefresh,
DisableAsyncRefresh: opts.DisableAsyncRefresh,
})
}
// computeProvider fetches tokens from the google cloud metadata service.
type computeProvider struct {
scopes []string
client *metadata.Client
tokenBindingType TokenBindingType
}
type metadataTokenResp struct {
AccessToken string `json:"access_token"`
ExpiresInSec int `json:"expires_in"`
TokenType string `json:"token_type"`
}
func (cs *computeProvider) Token(ctx context.Context) (*auth.Token, error) {
tokenURI, err := url.Parse(computeTokenURI)
if err != nil {
return nil, err
}
hasScopes := len(cs.scopes) > 0
if hasScopes || cs.tokenBindingType != NoBinding {
v := url.Values{}
if hasScopes {
v.Set("scopes", strings.Join(cs.scopes, ","))
}
switch cs.tokenBindingType {
case MTLSHardBinding:
v.Set("transport", "mtls")
v.Set("binding-enforcement", "on")
case ALTSHardBinding:
v.Set("transport", "alts")
}
tokenURI.RawQuery = v.Encode()
}
tokenJSON, err := cs.client.GetWithContext(ctx, tokenURI.String())
if err != nil {
return nil, fmt.Errorf("credentials: cannot fetch token: %w", err)
}
var res metadataTokenResp
if err := json.NewDecoder(strings.NewReader(tokenJSON)).Decode(&res); err != nil {
return nil, fmt.Errorf("credentials: invalid token JSON from metadata: %w", err)
}
if res.ExpiresInSec == 0 || res.AccessToken == "" {
return nil, errors.New("credentials: incomplete token received from metadata")
}
token := &auth.Token{
Value: res.AccessToken,
Type: res.TokenType,
Expiry: time.Now().Add(time.Duration(res.ExpiresInSec) * time.Second),
Metadata: computeTokenMetadata,
}
return token, nil
}

471
vendor/cloud.google.com/go/auth/credentials/detect.go generated vendored Normal file
View File

@@ -0,0 +1,471 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package credentials
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net/http"
"os"
"time"
"cloud.google.com/go/auth"
"cloud.google.com/go/auth/internal"
"cloud.google.com/go/auth/internal/credsfile"
"cloud.google.com/go/auth/internal/trustboundary"
"cloud.google.com/go/compute/metadata"
"github.com/googleapis/gax-go/v2/internallog"
)
const (
// jwtTokenURL is Google's OAuth 2.0 token URL to use with the JWT(2LO) flow.
jwtTokenURL = "https://oauth2.googleapis.com/token"
// Google's OAuth 2.0 default endpoints.
googleAuthURL = "https://accounts.google.com/o/oauth2/auth"
googleTokenURL = "https://oauth2.googleapis.com/token"
// GoogleMTLSTokenURL is Google's default OAuth2.0 mTLS endpoint.
GoogleMTLSTokenURL = "https://oauth2.mtls.googleapis.com/token"
// Help on default credentials
adcSetupURL = "https://cloud.google.com/docs/authentication/external/set-up-adc"
)
var (
// for testing
allowOnGCECheck = true
)
// CredType specifies the type of JSON credentials being provided
// to a loading function such as [NewCredentialsFromFile] or
// [NewCredentialsFromJSON].
type CredType string
const (
// ServiceAccount represents a service account file type.
ServiceAccount CredType = "service_account"
// AuthorizedUser represents a user credentials file type.
AuthorizedUser CredType = "authorized_user"
// ExternalAccount represents an external account file type.
//
// IMPORTANT:
// This credential type does not validate the credential configuration. A security
// risk occurs when a credential configuration configured with malicious urls
// is used.
// You should validate credential configurations provided by untrusted sources.
// See [Security requirements when using credential configurations from an external
// source] https://cloud.google.com/docs/authentication/external/externally-sourced-credentials
// for more details.
ExternalAccount CredType = "external_account"
// ImpersonatedServiceAccount represents an impersonated service account file type.
//
// IMPORTANT:
// This credential type does not validate the credential configuration. A security
// risk occurs when a credential configuration configured with malicious urls
// is used.
// You should validate credential configurations provided by untrusted sources.
// See [Security requirements when using credential configurations from an external
// source] https://cloud.google.com/docs/authentication/external/externally-sourced-credentials
// for more details.
ImpersonatedServiceAccount CredType = "impersonated_service_account"
// GDCHServiceAccount represents a GDCH service account credentials.
GDCHServiceAccount CredType = "gdch_service_account"
// ExternalAccountAuthorizedUser represents an external account authorized user credentials.
ExternalAccountAuthorizedUser CredType = "external_account_authorized_user"
)
// TokenBindingType specifies the type of binding used when requesting a token
// whether to request a hard-bound token using mTLS or an instance identity
// bound token using ALTS.
type TokenBindingType int
const (
// NoBinding specifies that requested tokens are not required to have a
// binding. This is the default option.
NoBinding TokenBindingType = iota
// MTLSHardBinding specifies that a hard-bound token should be requested
// using an mTLS with S2A channel.
MTLSHardBinding
// ALTSHardBinding specifies that an instance identity bound token should
// be requested using an ALTS channel.
ALTSHardBinding
)
// OnGCE reports whether this process is running in Google Cloud.
func OnGCE() bool {
// TODO(codyoss): once all libs use this auth lib move metadata check here
return allowOnGCECheck && metadata.OnGCE()
}
// DetectDefault searches for "Application Default Credentials" and returns
// a credential based on the [DetectOptions] provided.
//
// It looks for credentials in the following places, preferring the first
// location found:
//
// - A JSON file whose path is specified by the GOOGLE_APPLICATION_CREDENTIALS
// environment variable. For workload identity federation, refer to
// https://cloud.google.com/iam/docs/how-to#using-workload-identity-federation
// on how to generate the JSON configuration file for on-prem/non-Google
// cloud platforms.
// - A JSON file in a location known to the gcloud command-line tool. On
// Windows, this is %APPDATA%/gcloud/application_default_credentials.json. On
// other systems, $HOME/.config/gcloud/application_default_credentials.json.
// - On Google Compute Engine, Google App Engine standard second generation
// runtimes, and Google App Engine flexible environment, it fetches
// credentials from the metadata server.
//
// Important: If you accept a credential configuration (credential
// JSON/File/Stream) from an external source for authentication to Google
// Cloud Platform, you must validate it before providing it to any Google
// API or library. Providing an unvalidated credential configuration to
// Google APIs can compromise the security of your systems and data. For
// more information, refer to [Validate credential configurations from
// external sources](https://cloud.google.com/docs/authentication/external/externally-sourced-credentials).
func DetectDefault(opts *DetectOptions) (*auth.Credentials, error) {
if err := opts.validate(); err != nil {
return nil, err
}
trustBoundaryEnabled, err := trustboundary.IsEnabled()
if err != nil {
return nil, err
}
if len(opts.CredentialsJSON) > 0 {
return readCredentialsFileJSON(opts.CredentialsJSON, opts)
}
if opts.CredentialsFile != "" {
return readCredentialsFile(opts.CredentialsFile, opts)
}
if filename := os.Getenv(credsfile.GoogleAppCredsEnvVar); filename != "" {
creds, err := readCredentialsFile(filename, opts)
if err != nil {
return nil, err
}
return creds, nil
}
fileName := credsfile.GetWellKnownFileName()
if b, err := os.ReadFile(fileName); err == nil {
return readCredentialsFileJSON(b, opts)
}
if OnGCE() {
metadataClient := metadata.NewWithOptions(&metadata.Options{
Logger: opts.logger(),
UseDefaultClient: true,
})
gceUniverseDomainProvider := &internal.ComputeUniverseDomainProvider{
MetadataClient: metadataClient,
}
tp := computeTokenProvider(opts, metadataClient)
if trustBoundaryEnabled {
gceConfigProvider := trustboundary.NewGCEConfigProvider(gceUniverseDomainProvider)
var err error
tp, err = trustboundary.NewProvider(opts.client(), gceConfigProvider, opts.logger(), tp)
if err != nil {
return nil, fmt.Errorf("credentials: failed to initialize GCE trust boundary provider: %w", err)
}
}
return auth.NewCredentials(&auth.CredentialsOptions{
TokenProvider: tp,
ProjectIDProvider: auth.CredentialsPropertyFunc(func(ctx context.Context) (string, error) {
return metadataClient.ProjectIDWithContext(ctx)
}),
UniverseDomainProvider: gceUniverseDomainProvider,
}), nil
}
return nil, fmt.Errorf("credentials: could not find default credentials. See %v for more information", adcSetupURL)
}
// DetectOptions provides configuration for [DetectDefault].
type DetectOptions struct {
// Scopes that credentials tokens should have. Example:
// https://www.googleapis.com/auth/cloud-platform. Required if Audience is
// not provided.
Scopes []string
// TokenBindingType specifies the type of binding used when requesting a
// token whether to request a hard-bound token using mTLS or an instance
// identity bound token using ALTS. Optional.
TokenBindingType TokenBindingType
// Audience that credentials tokens should have. Only applicable for 2LO
// flows with service accounts. If specified, scopes should not be provided.
Audience string
// Subject is the user email used for [domain wide delegation](https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority).
// Optional.
Subject string
// EarlyTokenRefresh configures how early before a token expires that it
// should be refreshed. Once the tokens time until expiration has entered
// this refresh window the token is considered valid but stale. If unset,
// the default value is 3 minutes and 45 seconds. Optional.
EarlyTokenRefresh time.Duration
// DisableAsyncRefresh configures a synchronous workflow that refreshes
// stale tokens while blocking. The default is false. Optional.
DisableAsyncRefresh bool
// AuthHandlerOptions configures an authorization handler and other options
// for 3LO flows. It is required, and only used, for client credential
// flows.
AuthHandlerOptions *auth.AuthorizationHandlerOptions
// TokenURL allows to set the token endpoint for user credential flows. If
// unset the default value is: https://oauth2.googleapis.com/token.
// Optional.
TokenURL string
// STSAudience is the audience sent to when retrieving an STS token.
// Currently this only used for GDCH auth flow, for which it is required.
STSAudience string
// CredentialsFile overrides detection logic and sources a credential file
// from the provided filepath. If provided, CredentialsJSON must not be.
// Optional.
//
// Deprecated: This field is deprecated because of a potential security risk.
// It does not validate the credential configuration. The security risk occurs
// when a credential configuration is accepted from a source that is not
// under your control and used without validation on your side.
//
// If you know that you will be loading credential configurations of a
// specific type, it is recommended to use a credential-type-specific
// NewCredentialsFromFile method. This will ensure that an unexpected
// credential type with potential for malicious intent is not loaded
// unintentionally. You might still have to do validation for certain
// credential types. Please follow the recommendation for that method. For
// example, if you want to load only service accounts, you can use
//
// creds, err := credentials.NewCredentialsFromFile(ctx, credentials.ServiceAccount, filename, opts)
//
// If you are loading your credential configuration from an untrusted source
// and have not mitigated the risks (e.g. by validating the configuration
// yourself), make these changes as soon as possible to prevent security
// risks to your environment.
//
// Regardless of the method used, it is always your responsibility to
// validate configurations received from external sources.
//
// For more details see:
// https://cloud.google.com/docs/authentication/external/externally-sourced-credentials
CredentialsFile string
// CredentialsJSON overrides detection logic and uses the JSON bytes as the
// source for the credential. If provided, CredentialsFile must not be.
// Optional.
//
// Deprecated: This field is deprecated because of a potential security risk.
// It does not validate the credential configuration. The security risk occurs
// when a credential configuration is accepted from a source that is not
// under your control and used without validation on your side.
//
// If you know that you will be loading credential configurations of a
// specific type, it is recommended to use a credential-type-specific
// NewCredentialsFromJSON method. This will ensure that an unexpected
// credential type with potential for malicious intent is not loaded
// unintentionally. You might still have to do validation for certain
// credential types. Please follow the recommendation for that method. For
// example, if you want to load only service accounts, you can use
//
// creds, err := credentials.NewCredentialsFromJSON(ctx, credentials.ServiceAccount, json, opts)
//
// If you are loading your credential configuration from an untrusted source
// and have not mitigated the risks (e.g. by validating the configuration
// yourself), make these changes as soon as possible to prevent security
// risks to your environment.
//
// Regardless of the method used, it is always your responsibility to
// validate configurations received from external sources.
//
// For more details see:
// https://cloud.google.com/docs/authentication/external/externally-sourced-credentials
CredentialsJSON []byte
// UseSelfSignedJWT directs service account based credentials to create a
// self-signed JWT with the private key found in the file, skipping any
// network requests that would normally be made. Optional.
UseSelfSignedJWT bool
// Client configures the underlying client used to make network requests
// when fetching tokens. Optional.
Client *http.Client
// UniverseDomain is the default service domain for a given Cloud universe.
// The default value is "googleapis.com". This option is ignored for
// authentication flows that do not support universe domain. Optional.
UniverseDomain string
// Logger is used for debug logging. If provided, logging will be enabled
// at the loggers configured level. By default logging is disabled unless
// enabled by setting GOOGLE_SDK_GO_LOGGING_LEVEL in which case a default
// logger will be used. Optional.
Logger *slog.Logger
}
// NewCredentialsFromFile creates a [cloud.google.com/go/auth.Credentials] from
// the provided file. The credType argument specifies the expected credential
// type. If the file content does not match the expected type, an error is
// returned.
//
// Important: If you accept a credential configuration (credential
// JSON/File/Stream) from an external source for authentication to Google
// Cloud Platform, you must validate it before providing it to any Google
// API or library. Providing an unvalidated credential configuration to
// Google APIs can compromise the security of your systems and data. For
// more information, refer to [Validate credential configurations from
// external sources](https://cloud.google.com/docs/authentication/external/externally-sourced-credentials).
func NewCredentialsFromFile(credType CredType, filename string, opts *DetectOptions) (*auth.Credentials, error) {
b, err := os.ReadFile(filename)
if err != nil {
return nil, err
}
return NewCredentialsFromJSON(credType, b, opts)
}
// NewCredentialsFromJSON creates a [cloud.google.com/go/auth.Credentials] from
// the provided JSON bytes. The credType argument specifies the expected
// credential type. If the JSON does not match the expected type, an error is
// returned.
//
// Important: If you accept a credential configuration (credential
// JSON/File/Stream) from an external source for authentication to Google
// Cloud Platform, you must validate it before providing it to any Google
// API or library. Providing an unvalidated credential configuration to
// Google APIs can compromise the security of your systems and data. For
// more information, refer to [Validate credential configurations from
// external sources](https://cloud.google.com/docs/authentication/external/externally-sourced-credentials).
func NewCredentialsFromJSON(credType CredType, b []byte, opts *DetectOptions) (*auth.Credentials, error) {
if err := checkCredentialType(b, credType); err != nil {
return nil, err
}
// We can't use readCredentialsFileJSON because it does auto-detection
// for client_credentials.json which we don't support here (no type field).
// Instead, we call fileCredentials just as readCredentialsFileJSON does
// when it doesn't detect client_credentials.json.
return fileCredentials(b, opts)
}
func checkCredentialType(b []byte, expected CredType) error {
fileType, err := credsfile.ParseFileType(b)
if err != nil {
return err
}
if CredType(fileType) != expected {
return fmt.Errorf("credentials: expected type %q, found %q", expected, fileType)
}
return nil
}
func (o *DetectOptions) validate() error {
if o == nil {
return errors.New("credentials: options must be provided")
}
if len(o.Scopes) > 0 && o.Audience != "" {
return errors.New("credentials: both scopes and audience were provided")
}
if len(o.CredentialsJSON) > 0 && o.CredentialsFile != "" {
return errors.New("credentials: both credentials file and JSON were provided")
}
return nil
}
func (o *DetectOptions) tokenURL() string {
if o.TokenURL != "" {
return o.TokenURL
}
return googleTokenURL
}
func (o *DetectOptions) scopes() []string {
scopes := make([]string, len(o.Scopes))
copy(scopes, o.Scopes)
return scopes
}
func (o *DetectOptions) client() *http.Client {
if o.Client != nil {
return o.Client
}
return internal.DefaultClient()
}
func (o *DetectOptions) logger() *slog.Logger {
return internallog.New(o.Logger)
}
func readCredentialsFile(filename string, opts *DetectOptions) (*auth.Credentials, error) {
b, err := os.ReadFile(filename)
if err != nil {
return nil, err
}
return readCredentialsFileJSON(b, opts)
}
func readCredentialsFileJSON(b []byte, opts *DetectOptions) (*auth.Credentials, error) {
// attempt to parse jsonData as a Google Developers Console client_credentials.json.
config := clientCredConfigFromJSON(b, opts)
if config != nil {
if config.AuthHandlerOpts == nil {
return nil, errors.New("credentials: auth handler must be specified for this credential filetype")
}
tp, err := auth.New3LOTokenProvider(config)
if err != nil {
return nil, err
}
return auth.NewCredentials(&auth.CredentialsOptions{
TokenProvider: tp,
JSON: b,
}), nil
}
return fileCredentials(b, opts)
}
func clientCredConfigFromJSON(b []byte, opts *DetectOptions) *auth.Options3LO {
var creds credsfile.ClientCredentialsFile
var c *credsfile.Config3LO
if err := json.Unmarshal(b, &creds); err != nil {
return nil
}
switch {
case creds.Web != nil:
c = creds.Web
case creds.Installed != nil:
c = creds.Installed
default:
return nil
}
if len(c.RedirectURIs) < 1 {
return nil
}
var handleOpts *auth.AuthorizationHandlerOptions
if opts.AuthHandlerOptions != nil {
handleOpts = &auth.AuthorizationHandlerOptions{
Handler: opts.AuthHandlerOptions.Handler,
State: opts.AuthHandlerOptions.State,
PKCEOpts: opts.AuthHandlerOptions.PKCEOpts,
}
}
return &auth.Options3LO{
ClientID: c.ClientID,
ClientSecret: c.ClientSecret,
RedirectURL: c.RedirectURIs[0],
Scopes: opts.scopes(),
AuthURL: c.AuthURI,
TokenURL: c.TokenURI,
Client: opts.client(),
Logger: opts.logger(),
EarlyTokenExpiry: opts.EarlyTokenRefresh,
AuthHandlerOpts: handleOpts,
// TODO(codyoss): refactor this out. We need to add in auto-detection
// for this use case.
AuthStyle: auth.StyleInParams,
}
}

45
vendor/cloud.google.com/go/auth/credentials/doc.go generated vendored Normal file
View File

@@ -0,0 +1,45 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package credentials provides support for making OAuth2 authorized and
// authenticated HTTP requests to Google APIs. It supports the Web server flow,
// client-side credentials, service accounts, Google Compute Engine service
// accounts, Google App Engine service accounts and workload identity federation
// from non-Google cloud platforms.
//
// A brief overview of the package follows. For more information, please read
// https://developers.google.com/accounts/docs/OAuth2
// and
// https://developers.google.com/accounts/docs/application-default-credentials.
// For more information on using workload identity federation, refer to
// https://cloud.google.com/iam/docs/how-to#using-workload-identity-federation.
//
// # Credentials
//
// The [cloud.google.com/go/auth.Credentials] type represents Google
// credentials, including Application Default Credentials.
//
// Use [DetectDefault] to obtain Application Default Credentials.
//
// Application Default Credentials support workload identity federation to
// access Google Cloud resources from non-Google Cloud platforms including Amazon
// Web Services (AWS), Microsoft Azure or any identity provider that supports
// OpenID Connect (OIDC). Workload identity federation is recommended for
// non-Google Cloud environments as it avoids the need to download, manage, and
// store service account private keys locally.
//
// # Workforce Identity Federation
//
// For more information on this feature see [cloud.google.com/go/auth/credentials/externalaccount].
package credentials

View File

@@ -0,0 +1,329 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package credentials
import (
"errors"
"fmt"
"cloud.google.com/go/auth"
"cloud.google.com/go/auth/credentials/internal/externalaccount"
"cloud.google.com/go/auth/credentials/internal/externalaccountuser"
"cloud.google.com/go/auth/credentials/internal/gdch"
"cloud.google.com/go/auth/credentials/internal/impersonate"
internalauth "cloud.google.com/go/auth/internal"
"cloud.google.com/go/auth/internal/credsfile"
"cloud.google.com/go/auth/internal/trustboundary"
)
const cloudPlatformScope = "https://www.googleapis.com/auth/cloud-platform"
func fileCredentials(b []byte, opts *DetectOptions) (*auth.Credentials, error) {
fileType, err := credsfile.ParseFileType(b)
if err != nil {
return nil, err
}
if fileType == "" {
return nil, errors.New("credentials: unsupported unidentified file type")
}
var projectID, universeDomain string
var tp auth.TokenProvider
switch CredType(fileType) {
case ServiceAccount:
f, err := credsfile.ParseServiceAccount(b)
if err != nil {
return nil, err
}
tp, err = handleServiceAccount(f, opts)
if err != nil {
return nil, err
}
projectID = f.ProjectID
universeDomain = resolveUniverseDomain(opts.UniverseDomain, f.UniverseDomain)
case AuthorizedUser:
f, err := credsfile.ParseUserCredentials(b)
if err != nil {
return nil, err
}
tp, err = handleUserCredential(f, opts)
if err != nil {
return nil, err
}
universeDomain = f.UniverseDomain
case ExternalAccount:
f, err := credsfile.ParseExternalAccount(b)
if err != nil {
return nil, err
}
tp, err = handleExternalAccount(f, opts)
if err != nil {
return nil, err
}
universeDomain = resolveUniverseDomain(opts.UniverseDomain, f.UniverseDomain)
case ExternalAccountAuthorizedUser:
f, err := credsfile.ParseExternalAccountAuthorizedUser(b)
if err != nil {
return nil, err
}
tp, err = handleExternalAccountAuthorizedUser(f, opts)
if err != nil {
return nil, err
}
universeDomain = f.UniverseDomain
case ImpersonatedServiceAccount:
f, err := credsfile.ParseImpersonatedServiceAccount(b)
if err != nil {
return nil, err
}
tp, err = handleImpersonatedServiceAccount(f, opts)
if err != nil {
return nil, err
}
universeDomain = resolveUniverseDomain(opts.UniverseDomain, f.UniverseDomain)
case GDCHServiceAccount:
f, err := credsfile.ParseGDCHServiceAccount(b)
if err != nil {
return nil, err
}
tp, err = handleGDCHServiceAccount(f, opts)
if err != nil {
return nil, err
}
projectID = f.Project
universeDomain = f.UniverseDomain
default:
return nil, fmt.Errorf("credentials: unsupported filetype %q", fileType)
}
return auth.NewCredentials(&auth.CredentialsOptions{
TokenProvider: auth.NewCachedTokenProvider(tp, &auth.CachedTokenProviderOptions{
ExpireEarly: opts.EarlyTokenRefresh,
}),
JSON: b,
ProjectIDProvider: internalauth.StaticCredentialsProperty(projectID),
// TODO(codyoss): only set quota project here if there was a user override
UniverseDomainProvider: internalauth.StaticCredentialsProperty(universeDomain),
}), nil
}
// resolveUniverseDomain returns optsUniverseDomain if non-empty, in order to
// support configuring universe-specific credentials in code. Auth flows
// unsupported for universe domain should not use this func, but should instead
// simply set the file universe domain on the credentials.
func resolveUniverseDomain(optsUniverseDomain, fileUniverseDomain string) string {
if optsUniverseDomain != "" {
return optsUniverseDomain
}
return fileUniverseDomain
}
func handleServiceAccount(f *credsfile.ServiceAccountFile, opts *DetectOptions) (auth.TokenProvider, error) {
ud := resolveUniverseDomain(opts.UniverseDomain, f.UniverseDomain)
if opts.UseSelfSignedJWT {
return configureSelfSignedJWT(f, opts)
} else if ud != "" && ud != internalauth.DefaultUniverseDomain {
// For non-GDU universe domains, token exchange is impossible and services
// must support self-signed JWTs.
opts.UseSelfSignedJWT = true
return configureSelfSignedJWT(f, opts)
}
opts2LO := &auth.Options2LO{
Email: f.ClientEmail,
PrivateKey: []byte(f.PrivateKey),
PrivateKeyID: f.PrivateKeyID,
Scopes: opts.scopes(),
TokenURL: f.TokenURL,
Subject: opts.Subject,
Client: opts.client(),
Logger: opts.logger(),
UniverseDomain: ud,
}
if opts2LO.TokenURL == "" {
opts2LO.TokenURL = jwtTokenURL
}
tp, err := auth.New2LOTokenProvider(opts2LO)
if err != nil {
return nil, err
}
trustBoundaryEnabled, err := trustboundary.IsEnabled()
if err != nil {
return nil, err
}
if !trustBoundaryEnabled {
return tp, nil
}
saConfig := trustboundary.NewServiceAccountConfigProvider(opts2LO.Email, opts2LO.UniverseDomain)
return trustboundary.NewProvider(opts.client(), saConfig, opts.logger(), tp)
}
func handleUserCredential(f *credsfile.UserCredentialsFile, opts *DetectOptions) (auth.TokenProvider, error) {
opts3LO := &auth.Options3LO{
ClientID: f.ClientID,
ClientSecret: f.ClientSecret,
Scopes: opts.scopes(),
AuthURL: googleAuthURL,
TokenURL: opts.tokenURL(),
AuthStyle: auth.StyleInParams,
EarlyTokenExpiry: opts.EarlyTokenRefresh,
RefreshToken: f.RefreshToken,
Client: opts.client(),
Logger: opts.logger(),
}
return auth.New3LOTokenProvider(opts3LO)
}
func handleExternalAccount(f *credsfile.ExternalAccountFile, opts *DetectOptions) (auth.TokenProvider, error) {
externalOpts := &externalaccount.Options{
Audience: f.Audience,
SubjectTokenType: f.SubjectTokenType,
TokenURL: f.TokenURL,
TokenInfoURL: f.TokenInfoURL,
ServiceAccountImpersonationURL: f.ServiceAccountImpersonationURL,
ClientSecret: f.ClientSecret,
ClientID: f.ClientID,
CredentialSource: f.CredentialSource,
QuotaProjectID: f.QuotaProjectID,
Scopes: opts.scopes(),
WorkforcePoolUserProject: f.WorkforcePoolUserProject,
Client: opts.client(),
Logger: opts.logger(),
IsDefaultClient: opts.Client == nil,
}
if f.ServiceAccountImpersonation != nil {
externalOpts.ServiceAccountImpersonationLifetimeSeconds = f.ServiceAccountImpersonation.TokenLifetimeSeconds
}
tp, err := externalaccount.NewTokenProvider(externalOpts)
if err != nil {
return nil, err
}
trustBoundaryEnabled, err := trustboundary.IsEnabled()
if err != nil {
return nil, err
}
if !trustBoundaryEnabled {
return tp, nil
}
ud := resolveUniverseDomain(opts.UniverseDomain, f.UniverseDomain)
var configProvider trustboundary.ConfigProvider
if f.ServiceAccountImpersonationURL == "" {
// No impersonation, this is a direct external account credential.
// The trust boundary is based on the workload/workforce pool.
var err error
configProvider, err = trustboundary.NewExternalAccountConfigProvider(f.Audience, ud)
if err != nil {
return nil, err
}
} else {
// Impersonation is used. The trust boundary is based on the target service account.
targetSAEmail, err := impersonate.ExtractServiceAccountEmail(f.ServiceAccountImpersonationURL)
if err != nil {
return nil, fmt.Errorf("credentials: could not extract target service account email for trust boundary: %w", err)
}
configProvider = trustboundary.NewServiceAccountConfigProvider(targetSAEmail, ud)
}
return trustboundary.NewProvider(opts.client(), configProvider, opts.logger(), tp)
}
func handleExternalAccountAuthorizedUser(f *credsfile.ExternalAccountAuthorizedUserFile, opts *DetectOptions) (auth.TokenProvider, error) {
externalOpts := &externalaccountuser.Options{
Audience: f.Audience,
RefreshToken: f.RefreshToken,
TokenURL: f.TokenURL,
TokenInfoURL: f.TokenInfoURL,
ClientID: f.ClientID,
ClientSecret: f.ClientSecret,
Scopes: opts.scopes(),
Client: opts.client(),
Logger: opts.logger(),
}
tp, err := externalaccountuser.NewTokenProvider(externalOpts)
if err != nil {
return nil, err
}
trustBoundaryEnabled, err := trustboundary.IsEnabled()
if err != nil {
return nil, err
}
if !trustBoundaryEnabled {
return tp, nil
}
ud := resolveUniverseDomain(opts.UniverseDomain, f.UniverseDomain)
configProvider, err := trustboundary.NewExternalAccountConfigProvider(f.Audience, ud)
if err != nil {
return nil, err
}
return trustboundary.NewProvider(opts.client(), configProvider, opts.logger(), tp)
}
func handleImpersonatedServiceAccount(f *credsfile.ImpersonatedServiceAccountFile, opts *DetectOptions) (auth.TokenProvider, error) {
if f.ServiceAccountImpersonationURL == "" || f.CredSource == nil {
return nil, errors.New("missing 'source_credentials' field or 'service_account_impersonation_url' in credentials")
}
sourceOpts := *opts
// Source credential needs IAM or Cloud Platform scope to call the
// iamcredentials endpoint. The scopes provided by the user are for the
// impersonated credentials.
sourceOpts.Scopes = []string{cloudPlatformScope}
sourceTP, err := fileCredentials(f.CredSource, &sourceOpts)
if err != nil {
return nil, err
}
ud := resolveUniverseDomain(opts.UniverseDomain, f.UniverseDomain)
scopes := opts.scopes()
if len(scopes) == 0 {
scopes = f.Scopes
}
impOpts := &impersonate.Options{
URL: f.ServiceAccountImpersonationURL,
Scopes: scopes,
Tp: sourceTP,
Delegates: f.Delegates,
Client: opts.client(),
Logger: opts.logger(),
UniverseDomain: ud,
}
tp, err := impersonate.NewTokenProvider(impOpts)
if err != nil {
return nil, err
}
trustBoundaryEnabled, err := trustboundary.IsEnabled()
if err != nil {
return nil, err
}
if !trustBoundaryEnabled {
return tp, nil
}
targetSAEmail, err := impersonate.ExtractServiceAccountEmail(f.ServiceAccountImpersonationURL)
if err != nil {
return nil, fmt.Errorf("credentials: could not extract target service account email for trust boundary: %w", err)
}
targetSAConfig := trustboundary.NewServiceAccountConfigProvider(targetSAEmail, ud)
return trustboundary.NewProvider(opts.client(), targetSAConfig, opts.logger(), tp)
}
func handleGDCHServiceAccount(f *credsfile.GDCHServiceAccountFile, opts *DetectOptions) (auth.TokenProvider, error) {
return gdch.NewTokenProvider(f, &gdch.Options{
STSAudience: opts.STSAudience,
Client: opts.client(),
Logger: opts.logger(),
})
}

View File

@@ -0,0 +1,531 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package externalaccount
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net/http"
"net/url"
"os"
"path"
"sort"
"strings"
"time"
"cloud.google.com/go/auth/internal"
"github.com/googleapis/gax-go/v2/internallog"
)
var (
// getenv aliases os.Getenv for testing
getenv = os.Getenv
)
const (
// AWS Signature Version 4 signing algorithm identifier.
awsAlgorithm = "AWS4-HMAC-SHA256"
// The termination string for the AWS credential scope value as defined in
// https://docs.aws.amazon.com/general/latest/gr/sigv4-create-string-to-sign.html
awsRequestType = "aws4_request"
// The AWS authorization header name for the security session token if available.
awsSecurityTokenHeader = "x-amz-security-token"
// The name of the header containing the session token for metadata endpoint calls
awsIMDSv2SessionTokenHeader = "X-aws-ec2-metadata-token"
awsIMDSv2SessionTTLHeader = "X-aws-ec2-metadata-token-ttl-seconds"
awsIMDSv2SessionTTL = "300"
// The AWS authorization header name for the auto-generated date.
awsDateHeader = "x-amz-date"
defaultRegionalCredentialVerificationURL = "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
// Supported AWS configuration environment variables.
awsAccessKeyIDEnvVar = "AWS_ACCESS_KEY_ID"
awsDefaultRegionEnvVar = "AWS_DEFAULT_REGION"
awsRegionEnvVar = "AWS_REGION"
awsSecretAccessKeyEnvVar = "AWS_SECRET_ACCESS_KEY"
awsSessionTokenEnvVar = "AWS_SESSION_TOKEN"
awsTimeFormatLong = "20060102T150405Z"
awsTimeFormatShort = "20060102"
awsProviderType = "aws"
)
type awsSubjectProvider struct {
EnvironmentID string
RegionURL string
RegionalCredVerificationURL string
CredVerificationURL string
IMDSv2SessionTokenURL string
TargetResource string
requestSigner *awsRequestSigner
region string
securityCredentialsProvider AwsSecurityCredentialsProvider
reqOpts *RequestOptions
Client *http.Client
logger *slog.Logger
}
func (sp *awsSubjectProvider) subjectToken(ctx context.Context) (string, error) {
// Set Defaults
if sp.RegionalCredVerificationURL == "" {
sp.RegionalCredVerificationURL = defaultRegionalCredentialVerificationURL
}
headers := make(map[string]string)
if sp.shouldUseMetadataServer() {
awsSessionToken, err := sp.getAWSSessionToken(ctx)
if err != nil {
return "", err
}
if awsSessionToken != "" {
headers[awsIMDSv2SessionTokenHeader] = awsSessionToken
}
}
awsSecurityCredentials, err := sp.getSecurityCredentials(ctx, headers)
if err != nil {
return "", err
}
if sp.region, err = sp.getRegion(ctx, headers); err != nil {
return "", err
}
sp.requestSigner = &awsRequestSigner{
RegionName: sp.region,
AwsSecurityCredentials: awsSecurityCredentials,
}
// Generate the signed request to AWS STS GetCallerIdentity API.
// Use the required regional endpoint. Otherwise, the request will fail.
req, err := http.NewRequestWithContext(ctx, "POST", strings.Replace(sp.RegionalCredVerificationURL, "{region}", sp.region, 1), nil)
if err != nil {
return "", err
}
// The full, canonical resource name of the workload identity pool
// provider, with or without the HTTPS prefix.
// Including this header as part of the signature is recommended to
// ensure data integrity.
if sp.TargetResource != "" {
req.Header.Set("x-goog-cloud-target-resource", sp.TargetResource)
}
sp.requestSigner.signRequest(req)
/*
The GCP STS endpoint expects the headers to be formatted as:
# [
# {key: 'x-amz-date', value: '...'},
# {key: 'Authorization', value: '...'},
# ...
# ]
# And then serialized as:
# quote(json.dumps({
# url: '...',
# method: 'POST',
# headers: [{key: 'x-amz-date', value: '...'}, ...]
# }))
*/
awsSignedReq := awsRequest{
URL: req.URL.String(),
Method: "POST",
}
for headerKey, headerList := range req.Header {
for _, headerValue := range headerList {
awsSignedReq.Headers = append(awsSignedReq.Headers, awsRequestHeader{
Key: headerKey,
Value: headerValue,
})
}
}
sort.Slice(awsSignedReq.Headers, func(i, j int) bool {
headerCompare := strings.Compare(awsSignedReq.Headers[i].Key, awsSignedReq.Headers[j].Key)
if headerCompare == 0 {
return strings.Compare(awsSignedReq.Headers[i].Value, awsSignedReq.Headers[j].Value) < 0
}
return headerCompare < 0
})
result, err := json.Marshal(awsSignedReq)
if err != nil {
return "", err
}
return url.QueryEscape(string(result)), nil
}
func (sp *awsSubjectProvider) providerType() string {
if sp.securityCredentialsProvider != nil {
return programmaticProviderType
}
return awsProviderType
}
func (sp *awsSubjectProvider) getAWSSessionToken(ctx context.Context) (string, error) {
if sp.IMDSv2SessionTokenURL == "" {
return "", nil
}
req, err := http.NewRequestWithContext(ctx, "PUT", sp.IMDSv2SessionTokenURL, nil)
if err != nil {
return "", err
}
req.Header.Set(awsIMDSv2SessionTTLHeader, awsIMDSv2SessionTTL)
sp.logger.DebugContext(ctx, "aws session token request", "request", internallog.HTTPRequest(req, nil))
resp, body, err := internal.DoRequest(sp.Client, req)
if err != nil {
return "", err
}
sp.logger.DebugContext(ctx, "aws session token response", "response", internallog.HTTPResponse(resp, body))
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("credentials: unable to retrieve AWS session token: %s", body)
}
return string(body), nil
}
func (sp *awsSubjectProvider) getRegion(ctx context.Context, headers map[string]string) (string, error) {
if sp.securityCredentialsProvider != nil {
return sp.securityCredentialsProvider.AwsRegion(ctx, sp.reqOpts)
}
if canRetrieveRegionFromEnvironment() {
if envAwsRegion := getenv(awsRegionEnvVar); envAwsRegion != "" {
return envAwsRegion, nil
}
return getenv(awsDefaultRegionEnvVar), nil
}
if sp.RegionURL == "" {
return "", errors.New("credentials: unable to determine AWS region")
}
req, err := http.NewRequestWithContext(ctx, "GET", sp.RegionURL, nil)
if err != nil {
return "", err
}
for name, value := range headers {
req.Header.Add(name, value)
}
sp.logger.DebugContext(ctx, "aws region request", "request", internallog.HTTPRequest(req, nil))
resp, body, err := internal.DoRequest(sp.Client, req)
if err != nil {
return "", err
}
sp.logger.DebugContext(ctx, "aws region response", "response", internallog.HTTPResponse(resp, body))
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("credentials: unable to retrieve AWS region - %s", body)
}
// This endpoint will return the region in format: us-east-2b.
// Only the us-east-2 part should be used.
bodyLen := len(body)
if bodyLen == 0 {
return "", nil
}
return string(body[:bodyLen-1]), nil
}
func (sp *awsSubjectProvider) getSecurityCredentials(ctx context.Context, headers map[string]string) (result *AwsSecurityCredentials, err error) {
if sp.securityCredentialsProvider != nil {
return sp.securityCredentialsProvider.AwsSecurityCredentials(ctx, sp.reqOpts)
}
if canRetrieveSecurityCredentialFromEnvironment() {
return &AwsSecurityCredentials{
AccessKeyID: getenv(awsAccessKeyIDEnvVar),
SecretAccessKey: getenv(awsSecretAccessKeyEnvVar),
SessionToken: getenv(awsSessionTokenEnvVar),
}, nil
}
roleName, err := sp.getMetadataRoleName(ctx, headers)
if err != nil {
return
}
credentials, err := sp.getMetadataSecurityCredentials(ctx, roleName, headers)
if err != nil {
return
}
if credentials.AccessKeyID == "" {
return result, errors.New("credentials: missing AccessKeyId credential")
}
if credentials.SecretAccessKey == "" {
return result, errors.New("credentials: missing SecretAccessKey credential")
}
return credentials, nil
}
func (sp *awsSubjectProvider) getMetadataSecurityCredentials(ctx context.Context, roleName string, headers map[string]string) (*AwsSecurityCredentials, error) {
var result *AwsSecurityCredentials
req, err := http.NewRequestWithContext(ctx, "GET", fmt.Sprintf("%s/%s", sp.CredVerificationURL, roleName), nil)
if err != nil {
return result, err
}
for name, value := range headers {
req.Header.Add(name, value)
}
sp.logger.DebugContext(ctx, "aws security credential request", "request", internallog.HTTPRequest(req, nil))
resp, body, err := internal.DoRequest(sp.Client, req)
if err != nil {
return result, err
}
sp.logger.DebugContext(ctx, "aws security credential response", "response", internallog.HTTPResponse(resp, body))
if resp.StatusCode != http.StatusOK {
return result, fmt.Errorf("credentials: unable to retrieve AWS security credentials - %s", body)
}
if err := json.Unmarshal(body, &result); err != nil {
return nil, err
}
return result, nil
}
func (sp *awsSubjectProvider) getMetadataRoleName(ctx context.Context, headers map[string]string) (string, error) {
if sp.CredVerificationURL == "" {
return "", errors.New("credentials: unable to determine the AWS metadata server security credentials endpoint")
}
req, err := http.NewRequestWithContext(ctx, "GET", sp.CredVerificationURL, nil)
if err != nil {
return "", err
}
for name, value := range headers {
req.Header.Add(name, value)
}
sp.logger.DebugContext(ctx, "aws metadata role request", "request", internallog.HTTPRequest(req, nil))
resp, body, err := internal.DoRequest(sp.Client, req)
if err != nil {
return "", err
}
sp.logger.DebugContext(ctx, "aws metadata role response", "response", internallog.HTTPResponse(resp, body))
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("credentials: unable to retrieve AWS role name - %s", body)
}
return string(body), nil
}
// awsRequestSigner is a utility class to sign http requests using a AWS V4 signature.
type awsRequestSigner struct {
RegionName string
AwsSecurityCredentials *AwsSecurityCredentials
}
// signRequest adds the appropriate headers to an http.Request
// or returns an error if something prevented this.
func (rs *awsRequestSigner) signRequest(req *http.Request) error {
// req is assumed non-nil
signedRequest := cloneRequest(req)
timestamp := Now()
signedRequest.Header.Set("host", requestHost(req))
if rs.AwsSecurityCredentials.SessionToken != "" {
signedRequest.Header.Set(awsSecurityTokenHeader, rs.AwsSecurityCredentials.SessionToken)
}
if signedRequest.Header.Get("date") == "" {
signedRequest.Header.Set(awsDateHeader, timestamp.Format(awsTimeFormatLong))
}
authorizationCode, err := rs.generateAuthentication(signedRequest, timestamp)
if err != nil {
return err
}
signedRequest.Header.Set("Authorization", authorizationCode)
req.Header = signedRequest.Header
return nil
}
func (rs *awsRequestSigner) generateAuthentication(req *http.Request, timestamp time.Time) (string, error) {
canonicalHeaderColumns, canonicalHeaderData := canonicalHeaders(req)
dateStamp := timestamp.Format(awsTimeFormatShort)
serviceName := ""
if splitHost := strings.Split(requestHost(req), "."); len(splitHost) > 0 {
serviceName = splitHost[0]
}
credentialScope := strings.Join([]string{dateStamp, rs.RegionName, serviceName, awsRequestType}, "/")
requestString, err := canonicalRequest(req, canonicalHeaderColumns, canonicalHeaderData)
if err != nil {
return "", err
}
requestHash, err := getSha256([]byte(requestString))
if err != nil {
return "", err
}
stringToSign := strings.Join([]string{awsAlgorithm, timestamp.Format(awsTimeFormatLong), credentialScope, requestHash}, "\n")
signingKey := []byte("AWS4" + rs.AwsSecurityCredentials.SecretAccessKey)
for _, signingInput := range []string{
dateStamp, rs.RegionName, serviceName, awsRequestType, stringToSign,
} {
signingKey, err = getHmacSha256(signingKey, []byte(signingInput))
if err != nil {
return "", err
}
}
return fmt.Sprintf("%s Credential=%s/%s, SignedHeaders=%s, Signature=%s", awsAlgorithm, rs.AwsSecurityCredentials.AccessKeyID, credentialScope, canonicalHeaderColumns, hex.EncodeToString(signingKey)), nil
}
func getSha256(input []byte) (string, error) {
hash := sha256.New()
if _, err := hash.Write(input); err != nil {
return "", err
}
return hex.EncodeToString(hash.Sum(nil)), nil
}
func getHmacSha256(key, input []byte) ([]byte, error) {
hash := hmac.New(sha256.New, key)
if _, err := hash.Write(input); err != nil {
return nil, err
}
return hash.Sum(nil), nil
}
func cloneRequest(r *http.Request) *http.Request {
r2 := new(http.Request)
*r2 = *r
if r.Header != nil {
r2.Header = make(http.Header, len(r.Header))
// Find total number of values.
headerCount := 0
for _, headerValues := range r.Header {
headerCount += len(headerValues)
}
copiedHeaders := make([]string, headerCount) // shared backing array for headers' values
for headerKey, headerValues := range r.Header {
headerCount = copy(copiedHeaders, headerValues)
r2.Header[headerKey] = copiedHeaders[:headerCount:headerCount]
copiedHeaders = copiedHeaders[headerCount:]
}
}
return r2
}
func canonicalPath(req *http.Request) string {
result := req.URL.EscapedPath()
if result == "" {
return "/"
}
return path.Clean(result)
}
func canonicalQuery(req *http.Request) string {
queryValues := req.URL.Query()
for queryKey := range queryValues {
sort.Strings(queryValues[queryKey])
}
return queryValues.Encode()
}
func canonicalHeaders(req *http.Request) (string, string) {
// Header keys need to be sorted alphabetically.
var headers []string
lowerCaseHeaders := make(http.Header)
for k, v := range req.Header {
k := strings.ToLower(k)
if _, ok := lowerCaseHeaders[k]; ok {
// include additional values
lowerCaseHeaders[k] = append(lowerCaseHeaders[k], v...)
} else {
headers = append(headers, k)
lowerCaseHeaders[k] = v
}
}
sort.Strings(headers)
var fullHeaders bytes.Buffer
for _, header := range headers {
headerValue := strings.Join(lowerCaseHeaders[header], ",")
fullHeaders.WriteString(header)
fullHeaders.WriteRune(':')
fullHeaders.WriteString(headerValue)
fullHeaders.WriteRune('\n')
}
return strings.Join(headers, ";"), fullHeaders.String()
}
func requestDataHash(req *http.Request) (string, error) {
var requestData []byte
if req.Body != nil {
requestBody, err := req.GetBody()
if err != nil {
return "", err
}
defer requestBody.Close()
requestData, err = internal.ReadAll(requestBody)
if err != nil {
return "", err
}
}
return getSha256(requestData)
}
func requestHost(req *http.Request) string {
if req.Host != "" {
return req.Host
}
return req.URL.Host
}
func canonicalRequest(req *http.Request, canonicalHeaderColumns, canonicalHeaderData string) (string, error) {
dataHash, err := requestDataHash(req)
if err != nil {
return "", err
}
return fmt.Sprintf("%s\n%s\n%s\n%s\n%s\n%s", req.Method, canonicalPath(req), canonicalQuery(req), canonicalHeaderData, canonicalHeaderColumns, dataHash), nil
}
type awsRequestHeader struct {
Key string `json:"key"`
Value string `json:"value"`
}
type awsRequest struct {
URL string `json:"url"`
Method string `json:"method"`
Headers []awsRequestHeader `json:"headers"`
}
// The AWS region can be provided through AWS_REGION or AWS_DEFAULT_REGION. Only one is
// required.
func canRetrieveRegionFromEnvironment() bool {
return getenv(awsRegionEnvVar) != "" || getenv(awsDefaultRegionEnvVar) != ""
}
// Check if both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are available.
func canRetrieveSecurityCredentialFromEnvironment() bool {
return getenv(awsAccessKeyIDEnvVar) != "" && getenv(awsSecretAccessKeyEnvVar) != ""
}
func (sp *awsSubjectProvider) shouldUseMetadataServer() bool {
return sp.securityCredentialsProvider == nil && (!canRetrieveRegionFromEnvironment() || !canRetrieveSecurityCredentialFromEnvironment())
}

View File

@@ -0,0 +1,284 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package externalaccount
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"os"
"os/exec"
"regexp"
"strings"
"time"
"cloud.google.com/go/auth/internal"
)
const (
executableSupportedMaxVersion = 1
executableDefaultTimeout = 30 * time.Second
executableSource = "response"
executableProviderType = "executable"
outputFileSource = "output file"
allowExecutablesEnvVar = "GOOGLE_EXTERNAL_ACCOUNT_ALLOW_EXECUTABLES"
jwtTokenType = "urn:ietf:params:oauth:token-type:jwt"
idTokenType = "urn:ietf:params:oauth:token-type:id_token"
saml2TokenType = "urn:ietf:params:oauth:token-type:saml2"
)
var (
serviceAccountImpersonationRE = regexp.MustCompile(`https://iamcredentials..+/v1/projects/-/serviceAccounts/(.*@.*):generateAccessToken`)
)
type nonCacheableError struct {
message string
}
func (nce nonCacheableError) Error() string {
return nce.message
}
// environment is a contract for testing
type environment interface {
existingEnv() []string
getenv(string) string
run(ctx context.Context, command string, env []string) ([]byte, error)
now() time.Time
}
type runtimeEnvironment struct{}
func (r runtimeEnvironment) existingEnv() []string {
return os.Environ()
}
func (r runtimeEnvironment) getenv(key string) string {
return os.Getenv(key)
}
func (r runtimeEnvironment) now() time.Time {
return time.Now().UTC()
}
func (r runtimeEnvironment) run(ctx context.Context, command string, env []string) ([]byte, error) {
splitCommand := strings.Fields(command)
cmd := exec.CommandContext(ctx, splitCommand[0], splitCommand[1:]...)
cmd.Env = env
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
if ctx.Err() == context.DeadlineExceeded {
return nil, context.DeadlineExceeded
}
if exitError, ok := err.(*exec.ExitError); ok {
return nil, exitCodeError(exitError)
}
return nil, executableError(err)
}
bytesStdout := bytes.TrimSpace(stdout.Bytes())
if len(bytesStdout) > 0 {
return bytesStdout, nil
}
return bytes.TrimSpace(stderr.Bytes()), nil
}
type executableSubjectProvider struct {
Command string
Timeout time.Duration
OutputFile string
client *http.Client
opts *Options
env environment
}
type executableResponse struct {
Version int `json:"version,omitempty"`
Success *bool `json:"success,omitempty"`
TokenType string `json:"token_type,omitempty"`
ExpirationTime int64 `json:"expiration_time,omitempty"`
IDToken string `json:"id_token,omitempty"`
SamlResponse string `json:"saml_response,omitempty"`
Code string `json:"code,omitempty"`
Message string `json:"message,omitempty"`
}
func (sp *executableSubjectProvider) parseSubjectTokenFromSource(response []byte, source string, now int64) (string, error) {
var result executableResponse
if err := json.Unmarshal(response, &result); err != nil {
return "", jsonParsingError(source, string(response))
}
// Validate
if result.Version == 0 {
return "", missingFieldError(source, "version")
}
if result.Success == nil {
return "", missingFieldError(source, "success")
}
if !*result.Success {
if result.Code == "" || result.Message == "" {
return "", malformedFailureError()
}
return "", userDefinedError(result.Code, result.Message)
}
if result.Version > executableSupportedMaxVersion || result.Version < 0 {
return "", unsupportedVersionError(source, result.Version)
}
if result.ExpirationTime == 0 && sp.OutputFile != "" {
return "", missingFieldError(source, "expiration_time")
}
if result.TokenType == "" {
return "", missingFieldError(source, "token_type")
}
if result.ExpirationTime != 0 && result.ExpirationTime < now {
return "", tokenExpiredError()
}
switch result.TokenType {
case jwtTokenType, idTokenType:
if result.IDToken == "" {
return "", missingFieldError(source, "id_token")
}
return result.IDToken, nil
case saml2TokenType:
if result.SamlResponse == "" {
return "", missingFieldError(source, "saml_response")
}
return result.SamlResponse, nil
default:
return "", tokenTypeError(source)
}
}
func (sp *executableSubjectProvider) subjectToken(ctx context.Context) (string, error) {
if token, err := sp.getTokenFromOutputFile(); token != "" || err != nil {
return token, err
}
return sp.getTokenFromExecutableCommand(ctx)
}
func (sp *executableSubjectProvider) providerType() string {
return executableProviderType
}
func (sp *executableSubjectProvider) getTokenFromOutputFile() (token string, err error) {
if sp.OutputFile == "" {
// This ExecutableCredentialSource doesn't use an OutputFile.
return "", nil
}
file, err := os.Open(sp.OutputFile)
if err != nil {
// No OutputFile found. Hasn't been created yet, so skip it.
return "", nil
}
defer file.Close()
data, err := internal.ReadAll(file)
if err != nil || len(data) == 0 {
// Cachefile exists, but no data found. Get new credential.
return "", nil
}
token, err = sp.parseSubjectTokenFromSource(data, outputFileSource, sp.env.now().Unix())
if err != nil {
if _, ok := err.(nonCacheableError); ok {
// If the cached token is expired we need a new token,
// and if the cache contains a failure, we need to try again.
return "", nil
}
// There was an error in the cached token, and the developer should be aware of it.
return "", err
}
// Token parsing succeeded. Use found token.
return token, nil
}
func (sp *executableSubjectProvider) executableEnvironment() []string {
result := sp.env.existingEnv()
result = append(result, fmt.Sprintf("GOOGLE_EXTERNAL_ACCOUNT_AUDIENCE=%v", sp.opts.Audience))
result = append(result, fmt.Sprintf("GOOGLE_EXTERNAL_ACCOUNT_TOKEN_TYPE=%v", sp.opts.SubjectTokenType))
result = append(result, "GOOGLE_EXTERNAL_ACCOUNT_INTERACTIVE=0")
if sp.opts.ServiceAccountImpersonationURL != "" {
matches := serviceAccountImpersonationRE.FindStringSubmatch(sp.opts.ServiceAccountImpersonationURL)
if matches != nil {
result = append(result, fmt.Sprintf("GOOGLE_EXTERNAL_ACCOUNT_IMPERSONATED_EMAIL=%v", matches[1]))
}
}
if sp.OutputFile != "" {
result = append(result, fmt.Sprintf("GOOGLE_EXTERNAL_ACCOUNT_OUTPUT_FILE=%v", sp.OutputFile))
}
return result
}
func (sp *executableSubjectProvider) getTokenFromExecutableCommand(ctx context.Context) (string, error) {
// For security reasons, we need our consumers to set this environment variable to allow executables to be run.
if sp.env.getenv(allowExecutablesEnvVar) != "1" {
return "", errors.New("credentials: executables need to be explicitly allowed (set GOOGLE_EXTERNAL_ACCOUNT_ALLOW_EXECUTABLES to '1') to run")
}
ctx, cancel := context.WithDeadline(ctx, sp.env.now().Add(sp.Timeout))
defer cancel()
output, err := sp.env.run(ctx, sp.Command, sp.executableEnvironment())
if err != nil {
return "", err
}
return sp.parseSubjectTokenFromSource(output, executableSource, sp.env.now().Unix())
}
func missingFieldError(source, field string) error {
return fmt.Errorf("credentials: %q missing %q field", source, field)
}
func jsonParsingError(source, data string) error {
return fmt.Errorf("credentials: unable to parse %q: %v", source, data)
}
func malformedFailureError() error {
return nonCacheableError{"credentials: response must include `error` and `message` fields when unsuccessful"}
}
func userDefinedError(code, message string) error {
return nonCacheableError{fmt.Sprintf("credentials: response contains unsuccessful response: (%v) %v", code, message)}
}
func unsupportedVersionError(source string, version int) error {
return fmt.Errorf("credentials: %v contains unsupported version: %v", source, version)
}
func tokenExpiredError() error {
return nonCacheableError{"credentials: the token returned by the executable is expired"}
}
func tokenTypeError(source string) error {
return fmt.Errorf("credentials: %v contains unsupported token type", source)
}
func exitCodeError(err *exec.ExitError) error {
return fmt.Errorf("credentials: executable command failed with exit code %v: %w", err.ExitCode(), err)
}
func executableError(err error) error {
return fmt.Errorf("credentials: executable command failed: %w", err)
}

View File

@@ -0,0 +1,431 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package externalaccount
import (
"context"
"errors"
"fmt"
"log/slog"
"net/http"
"regexp"
"strconv"
"strings"
"time"
"cloud.google.com/go/auth"
"cloud.google.com/go/auth/credentials/internal/impersonate"
"cloud.google.com/go/auth/credentials/internal/stsexchange"
"cloud.google.com/go/auth/internal/credsfile"
"github.com/googleapis/gax-go/v2/internallog"
)
const (
timeoutMinimum = 5 * time.Second
timeoutMaximum = 120 * time.Second
universeDomainPlaceholder = "UNIVERSE_DOMAIN"
defaultTokenURL = "https://sts.UNIVERSE_DOMAIN/v1/token"
defaultUniverseDomain = "googleapis.com"
)
var (
// Now aliases time.Now for testing
Now = func() time.Time {
return time.Now().UTC()
}
validWorkforceAudiencePattern *regexp.Regexp = regexp.MustCompile(`//iam\.googleapis\.com/locations/[^/]+/workforcePools/`)
)
// Options stores the configuration for fetching tokens with external credentials.
type Options struct {
// Audience is the Secure Token Service (STS) audience which contains the resource name for the workload
// identity pool or the workforce pool and the provider identifier in that pool.
Audience string
// SubjectTokenType is the STS token type based on the Oauth2.0 token exchange spec
// e.g. `urn:ietf:params:oauth:token-type:jwt`.
SubjectTokenType string
// TokenURL is the STS token exchange endpoint.
TokenURL string
// TokenInfoURL is the token_info endpoint used to retrieve the account related information (
// user attributes like account identifier, eg. email, username, uid, etc). This is
// needed for gCloud session account identification.
TokenInfoURL string
// ServiceAccountImpersonationURL is the URL for the service account impersonation request. This is only
// required for workload identity pools when APIs to be accessed have not integrated with UberMint.
ServiceAccountImpersonationURL string
// ServiceAccountImpersonationLifetimeSeconds is the number of seconds the service account impersonation
// token will be valid for.
ServiceAccountImpersonationLifetimeSeconds int
// ClientSecret is currently only required if token_info endpoint also
// needs to be called with the generated GCP access token. When provided, STS will be
// called with additional basic authentication using client_id as username and client_secret as password.
ClientSecret string
// ClientID is only required in conjunction with ClientSecret, as described above.
ClientID string
// CredentialSource contains the necessary information to retrieve the token itself, as well
// as some environmental information.
CredentialSource *credsfile.CredentialSource
// QuotaProjectID is injected by gCloud. If the value is non-empty, the Auth libraries
// will set the x-goog-user-project which overrides the project associated with the credentials.
QuotaProjectID string
// Scopes contains the desired scopes for the returned access token.
Scopes []string
// WorkforcePoolUserProject should be set when it is a workforce pool and
// not a workload identity pool. The underlying principal must still have
// serviceusage.services.use IAM permission to use the project for
// billing/quota. Optional.
WorkforcePoolUserProject string
// UniverseDomain is the default service domain for a given Cloud universe.
// This value will be used in the default STS token URL. The default value
// is "googleapis.com". It will not be used if TokenURL is set. Optional.
UniverseDomain string
// SubjectTokenProvider is an optional token provider for OIDC/SAML
// credentials. One of SubjectTokenProvider, AWSSecurityCredentialProvider
// or CredentialSource must be provided. Optional.
SubjectTokenProvider SubjectTokenProvider
// AwsSecurityCredentialsProvider is an AWS Security Credential provider
// for AWS credentials. One of SubjectTokenProvider,
// AWSSecurityCredentialProvider or CredentialSource must be provided. Optional.
AwsSecurityCredentialsProvider AwsSecurityCredentialsProvider
// Client for token request.
Client *http.Client
// IsDefaultClient marks whether the client passed in is a default client that can be overriden.
// This is important for X509 credentials which should create a new client if the default was used
// but should respect a client explicitly passed in by the user.
IsDefaultClient bool
// Logger is used for debug logging. If provided, logging will be enabled
// at the loggers configured level. By default logging is disabled unless
// enabled by setting GOOGLE_SDK_GO_LOGGING_LEVEL in which case a default
// logger will be used. Optional.
Logger *slog.Logger
}
// SubjectTokenProvider can be used to supply a subject token to exchange for a
// GCP access token.
type SubjectTokenProvider interface {
// SubjectToken should return a valid subject token or an error.
// The external account token provider does not cache the returned subject
// token, so caching logic should be implemented in the provider to prevent
// multiple requests for the same subject token.
SubjectToken(ctx context.Context, opts *RequestOptions) (string, error)
}
// RequestOptions contains information about the requested subject token or AWS
// security credentials from the Google external account credential.
type RequestOptions struct {
// Audience is the requested audience for the external account credential.
Audience string
// Subject token type is the requested subject token type for the external
// account credential. Expected values include:
// “urn:ietf:params:oauth:token-type:jwt”
// “urn:ietf:params:oauth:token-type:id-token”
// “urn:ietf:params:oauth:token-type:saml2”
// “urn:ietf:params:aws:token-type:aws4_request”
SubjectTokenType string
}
// AwsSecurityCredentialsProvider can be used to supply AwsSecurityCredentials
// and an AWS Region to exchange for a GCP access token.
type AwsSecurityCredentialsProvider interface {
// AwsRegion should return the AWS region or an error.
AwsRegion(ctx context.Context, opts *RequestOptions) (string, error)
// GetAwsSecurityCredentials should return a valid set of
// AwsSecurityCredentials or an error. The external account token provider
// does not cache the returned security credentials, so caching logic should
// be implemented in the provider to prevent multiple requests for the
// same security credentials.
AwsSecurityCredentials(ctx context.Context, opts *RequestOptions) (*AwsSecurityCredentials, error)
}
// AwsSecurityCredentials models AWS security credentials.
type AwsSecurityCredentials struct {
// AccessKeyId is the AWS Access Key ID - Required.
AccessKeyID string `json:"AccessKeyID"`
// SecretAccessKey is the AWS Secret Access Key - Required.
SecretAccessKey string `json:"SecretAccessKey"`
// SessionToken is the AWS Session token. This should be provided for
// temporary AWS security credentials - Optional.
SessionToken string `json:"Token"`
}
func (o *Options) validate() error {
if o.Audience == "" {
return fmt.Errorf("externalaccount: Audience must be set")
}
if o.SubjectTokenType == "" {
return fmt.Errorf("externalaccount: Subject token type must be set")
}
if o.WorkforcePoolUserProject != "" {
if valid := validWorkforceAudiencePattern.MatchString(o.Audience); !valid {
return fmt.Errorf("externalaccount: workforce_pool_user_project should not be set for non-workforce pool credentials")
}
}
count := 0
if o.CredentialSource != nil {
count++
}
if o.SubjectTokenProvider != nil {
count++
}
if o.AwsSecurityCredentialsProvider != nil {
count++
}
if count == 0 {
return fmt.Errorf("externalaccount: one of CredentialSource, SubjectTokenProvider, or AwsSecurityCredentialsProvider must be set")
}
if count > 1 {
return fmt.Errorf("externalaccount: only one of CredentialSource, SubjectTokenProvider, or AwsSecurityCredentialsProvider must be set")
}
return nil
}
// client returns the http client that should be used for the token exchange. If a non-default client
// is provided, then the client configured in the options will always be returned. If a default client
// is provided and the options are configured for X509 credentials, a new client will be created.
func (o *Options) client() (*http.Client, error) {
// If a client was provided and no override certificate config location was provided, use the provided client.
if o.CredentialSource == nil || o.CredentialSource.Certificate == nil || (!o.IsDefaultClient && o.CredentialSource.Certificate.CertificateConfigLocation == "") {
return o.Client, nil
}
// If a new client should be created, validate and use the certificate source to create a new mTLS client.
cert := o.CredentialSource.Certificate
if !cert.UseDefaultCertificateConfig && cert.CertificateConfigLocation == "" {
return nil, errors.New("credentials: \"certificate\" object must either specify a certificate_config_location or use_default_certificate_config should be true")
}
if cert.UseDefaultCertificateConfig && cert.CertificateConfigLocation != "" {
return nil, errors.New("credentials: \"certificate\" object cannot specify both a certificate_config_location and use_default_certificate_config=true")
}
return createX509Client(cert.CertificateConfigLocation)
}
// resolveTokenURL sets the default STS token endpoint with the configured
// universe domain.
func (o *Options) resolveTokenURL() {
if o.TokenURL != "" {
return
} else if o.UniverseDomain != "" {
o.TokenURL = strings.Replace(defaultTokenURL, universeDomainPlaceholder, o.UniverseDomain, 1)
} else {
o.TokenURL = strings.Replace(defaultTokenURL, universeDomainPlaceholder, defaultUniverseDomain, 1)
}
}
// NewTokenProvider returns a [cloud.google.com/go/auth.TokenProvider]
// configured with the provided options.
func NewTokenProvider(opts *Options) (auth.TokenProvider, error) {
if err := opts.validate(); err != nil {
return nil, err
}
opts.resolveTokenURL()
logger := internallog.New(opts.Logger)
stp, err := newSubjectTokenProvider(opts)
if err != nil {
return nil, err
}
client, err := opts.client()
if err != nil {
return nil, err
}
tp := &tokenProvider{
client: client,
opts: opts,
stp: stp,
logger: logger,
}
if opts.ServiceAccountImpersonationURL == "" {
return auth.NewCachedTokenProvider(tp, nil), nil
}
scopes := make([]string, len(opts.Scopes))
copy(scopes, opts.Scopes)
// needed for impersonation
tp.opts.Scopes = []string{"https://www.googleapis.com/auth/cloud-platform"}
imp, err := impersonate.NewTokenProvider(&impersonate.Options{
Client: client,
URL: opts.ServiceAccountImpersonationURL,
Scopes: scopes,
Tp: auth.NewCachedTokenProvider(tp, nil),
TokenLifetimeSeconds: opts.ServiceAccountImpersonationLifetimeSeconds,
Logger: logger,
})
if err != nil {
return nil, err
}
return auth.NewCachedTokenProvider(imp, nil), nil
}
type subjectTokenProvider interface {
subjectToken(ctx context.Context) (string, error)
providerType() string
}
// tokenProvider is the provider that handles external credentials. It is used to retrieve Tokens.
type tokenProvider struct {
client *http.Client
logger *slog.Logger
opts *Options
stp subjectTokenProvider
}
func (tp *tokenProvider) Token(ctx context.Context) (*auth.Token, error) {
subjectToken, err := tp.stp.subjectToken(ctx)
if err != nil {
return nil, err
}
stsRequest := &stsexchange.TokenRequest{
GrantType: stsexchange.GrantType,
Audience: tp.opts.Audience,
Scope: tp.opts.Scopes,
RequestedTokenType: stsexchange.TokenType,
SubjectToken: subjectToken,
SubjectTokenType: tp.opts.SubjectTokenType,
}
header := make(http.Header)
header.Set("Content-Type", "application/x-www-form-urlencoded")
header.Add("x-goog-api-client", getGoogHeaderValue(tp.opts, tp.stp))
clientAuth := stsexchange.ClientAuthentication{
AuthStyle: auth.StyleInHeader,
ClientID: tp.opts.ClientID,
ClientSecret: tp.opts.ClientSecret,
}
var options map[string]interface{}
// Do not pass workforce_pool_user_project when client authentication is used.
// The client ID is sufficient for determining the user project.
if tp.opts.WorkforcePoolUserProject != "" && tp.opts.ClientID == "" {
options = map[string]interface{}{
"userProject": tp.opts.WorkforcePoolUserProject,
}
}
stsResp, err := stsexchange.ExchangeToken(ctx, &stsexchange.Options{
Client: tp.client,
Endpoint: tp.opts.TokenURL,
Request: stsRequest,
Authentication: clientAuth,
Headers: header,
ExtraOpts: options,
Logger: tp.logger,
})
if err != nil {
return nil, err
}
tok := &auth.Token{
Value: stsResp.AccessToken,
Type: stsResp.TokenType,
}
// The RFC8693 doesn't define the explicit 0 of "expires_in" field behavior.
if stsResp.ExpiresIn <= 0 {
return nil, fmt.Errorf("credentials: got invalid expiry from security token service")
}
tok.Expiry = Now().Add(time.Duration(stsResp.ExpiresIn) * time.Second)
return tok, nil
}
// newSubjectTokenProvider determines the type of credsfile.CredentialSource needed to create a
// subjectTokenProvider
func newSubjectTokenProvider(o *Options) (subjectTokenProvider, error) {
logger := internallog.New(o.Logger)
reqOpts := &RequestOptions{Audience: o.Audience, SubjectTokenType: o.SubjectTokenType}
if o.AwsSecurityCredentialsProvider != nil {
return &awsSubjectProvider{
securityCredentialsProvider: o.AwsSecurityCredentialsProvider,
TargetResource: o.Audience,
reqOpts: reqOpts,
logger: logger,
}, nil
} else if o.SubjectTokenProvider != nil {
return &programmaticProvider{stp: o.SubjectTokenProvider, opts: reqOpts}, nil
} else if len(o.CredentialSource.EnvironmentID) > 3 && o.CredentialSource.EnvironmentID[:3] == "aws" {
if awsVersion, err := strconv.Atoi(o.CredentialSource.EnvironmentID[3:]); err == nil {
if awsVersion != 1 {
return nil, fmt.Errorf("credentials: aws version '%d' is not supported in the current build", awsVersion)
}
awsProvider := &awsSubjectProvider{
EnvironmentID: o.CredentialSource.EnvironmentID,
RegionURL: o.CredentialSource.RegionURL,
RegionalCredVerificationURL: o.CredentialSource.RegionalCredVerificationURL,
CredVerificationURL: o.CredentialSource.URL,
TargetResource: o.Audience,
Client: o.Client,
logger: logger,
}
if o.CredentialSource.IMDSv2SessionTokenURL != "" {
awsProvider.IMDSv2SessionTokenURL = o.CredentialSource.IMDSv2SessionTokenURL
}
return awsProvider, nil
}
} else if o.CredentialSource.File != "" {
return &fileSubjectProvider{File: o.CredentialSource.File, Format: o.CredentialSource.Format}, nil
} else if o.CredentialSource.URL != "" {
return &urlSubjectProvider{
URL: o.CredentialSource.URL,
Headers: o.CredentialSource.Headers,
Format: o.CredentialSource.Format,
Client: o.Client,
Logger: logger,
}, nil
} else if o.CredentialSource.Executable != nil {
ec := o.CredentialSource.Executable
if ec.Command == "" {
return nil, errors.New("credentials: missing `command` field — executable command must be provided")
}
execProvider := &executableSubjectProvider{}
execProvider.Command = ec.Command
if ec.TimeoutMillis == 0 {
execProvider.Timeout = executableDefaultTimeout
} else {
execProvider.Timeout = time.Duration(ec.TimeoutMillis) * time.Millisecond
if execProvider.Timeout < timeoutMinimum || execProvider.Timeout > timeoutMaximum {
return nil, fmt.Errorf("credentials: invalid `timeout_millis` field — executable timeout must be between %v and %v seconds", timeoutMinimum.Seconds(), timeoutMaximum.Seconds())
}
}
execProvider.OutputFile = ec.OutputFile
execProvider.client = o.Client
execProvider.opts = o
execProvider.env = runtimeEnvironment{}
return execProvider, nil
} else if o.CredentialSource.Certificate != nil {
cert := o.CredentialSource.Certificate
if !cert.UseDefaultCertificateConfig && cert.CertificateConfigLocation == "" {
return nil, errors.New("credentials: \"certificate\" object must either specify a certificate_config_location or use_default_certificate_config should be true")
}
if cert.UseDefaultCertificateConfig && cert.CertificateConfigLocation != "" {
return nil, errors.New("credentials: \"certificate\" object cannot specify both a certificate_config_location and use_default_certificate_config=true")
}
return &x509Provider{
TrustChainPath: o.CredentialSource.Certificate.TrustChainPath,
ConfigFilePath: o.CredentialSource.Certificate.CertificateConfigLocation,
}, nil
}
return nil, errors.New("credentials: unable to parse credential source")
}
func getGoogHeaderValue(conf *Options, p subjectTokenProvider) string {
return fmt.Sprintf("gl-go/%s auth/%s google-byoid-sdk source/%s sa-impersonation/%t config-lifetime/%t",
goVersion(),
"unknown",
p.providerType(),
conf.ServiceAccountImpersonationURL != "",
conf.ServiceAccountImpersonationLifetimeSeconds != 0)
}

View File

@@ -0,0 +1,78 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package externalaccount
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"os"
"cloud.google.com/go/auth/internal"
"cloud.google.com/go/auth/internal/credsfile"
)
const (
fileProviderType = "file"
)
type fileSubjectProvider struct {
File string
Format *credsfile.Format
}
func (sp *fileSubjectProvider) subjectToken(context.Context) (string, error) {
tokenFile, err := os.Open(sp.File)
if err != nil {
return "", fmt.Errorf("credentials: failed to open credential file %q: %w", sp.File, err)
}
defer tokenFile.Close()
tokenBytes, err := internal.ReadAll(tokenFile)
if err != nil {
return "", fmt.Errorf("credentials: failed to read credential file: %w", err)
}
tokenBytes = bytes.TrimSpace(tokenBytes)
if sp.Format == nil {
return string(tokenBytes), nil
}
switch sp.Format.Type {
case fileTypeJSON:
jsonData := make(map[string]interface{})
err = json.Unmarshal(tokenBytes, &jsonData)
if err != nil {
return "", fmt.Errorf("credentials: failed to unmarshal subject token file: %w", err)
}
val, ok := jsonData[sp.Format.SubjectTokenFieldName]
if !ok {
return "", errors.New("credentials: provided subject_token_field_name not found in credentials")
}
token, ok := val.(string)
if !ok {
return "", errors.New("credentials: improperly formatted subject token")
}
return token, nil
case fileTypeText:
return string(tokenBytes), nil
default:
return "", errors.New("credentials: invalid credential_source file format type: " + sp.Format.Type)
}
}
func (sp *fileSubjectProvider) providerType() string {
return fileProviderType
}

View File

@@ -0,0 +1,74 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package externalaccount
import (
"runtime"
"strings"
"unicode"
)
var (
// version is a package internal global variable for testing purposes.
version = runtime.Version
)
// versionUnknown is only used when the runtime version cannot be determined.
const versionUnknown = "UNKNOWN"
// goVersion returns a Go runtime version derived from the runtime environment
// that is modified to be suitable for reporting in a header, meaning it has no
// whitespace. If it is unable to determine the Go runtime version, it returns
// versionUnknown.
func goVersion() string {
const develPrefix = "devel +"
s := version()
if strings.HasPrefix(s, develPrefix) {
s = s[len(develPrefix):]
if p := strings.IndexFunc(s, unicode.IsSpace); p >= 0 {
s = s[:p]
}
return s
} else if p := strings.IndexFunc(s, unicode.IsSpace); p >= 0 {
s = s[:p]
}
notSemverRune := func(r rune) bool {
return !strings.ContainsRune("0123456789.", r)
}
if strings.HasPrefix(s, "go1") {
s = s[2:]
var prerelease string
if p := strings.IndexFunc(s, notSemverRune); p >= 0 {
s, prerelease = s[:p], s[p:]
}
if strings.HasSuffix(s, ".") {
s += "0"
} else if strings.Count(s, ".") < 2 {
s += ".0"
}
if prerelease != "" {
// Some release candidates already have a dash in them.
if !strings.HasPrefix(prerelease, "-") {
prerelease = "-" + prerelease
}
s += prerelease
}
return s
}
return versionUnknown
}

View File

@@ -0,0 +1,30 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package externalaccount
import "context"
type programmaticProvider struct {
opts *RequestOptions
stp SubjectTokenProvider
}
func (pp *programmaticProvider) providerType() string {
return programmaticProviderType
}
func (pp *programmaticProvider) subjectToken(ctx context.Context) (string, error) {
return pp.stp.SubjectToken(ctx, pp.opts)
}

View File

@@ -0,0 +1,93 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package externalaccount
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net/http"
"cloud.google.com/go/auth/internal"
"cloud.google.com/go/auth/internal/credsfile"
"github.com/googleapis/gax-go/v2/internallog"
)
const (
fileTypeText = "text"
fileTypeJSON = "json"
urlProviderType = "url"
programmaticProviderType = "programmatic"
x509ProviderType = "x509"
)
type urlSubjectProvider struct {
URL string
Headers map[string]string
Format *credsfile.Format
Client *http.Client
Logger *slog.Logger
}
func (sp *urlSubjectProvider) subjectToken(ctx context.Context) (string, error) {
req, err := http.NewRequestWithContext(ctx, "GET", sp.URL, nil)
if err != nil {
return "", fmt.Errorf("credentials: HTTP request for URL-sourced credential failed: %w", err)
}
for key, val := range sp.Headers {
req.Header.Add(key, val)
}
sp.Logger.DebugContext(ctx, "url subject token request", "request", internallog.HTTPRequest(req, nil))
resp, body, err := internal.DoRequest(sp.Client, req)
if err != nil {
return "", fmt.Errorf("credentials: invalid response when retrieving subject token: %w", err)
}
sp.Logger.DebugContext(ctx, "url subject token response", "response", internallog.HTTPResponse(resp, body))
if c := resp.StatusCode; c < http.StatusOK || c >= http.StatusMultipleChoices {
return "", fmt.Errorf("credentials: status code %d: %s", c, body)
}
if sp.Format == nil {
return string(body), nil
}
switch sp.Format.Type {
case "json":
jsonData := make(map[string]interface{})
err = json.Unmarshal(body, &jsonData)
if err != nil {
return "", fmt.Errorf("credentials: failed to unmarshal subject token file: %w", err)
}
val, ok := jsonData[sp.Format.SubjectTokenFieldName]
if !ok {
return "", errors.New("credentials: provided subject_token_field_name not found in credentials")
}
token, ok := val.(string)
if !ok {
return "", errors.New("credentials: improperly formatted subject token")
}
return token, nil
case fileTypeText:
return string(body), nil
default:
return "", errors.New("credentials: invalid credential_source file format type: " + sp.Format.Type)
}
}
func (sp *urlSubjectProvider) providerType() string {
return urlProviderType
}

Some files were not shown because too many files have changed in this diff Show More