20 Commits

Author SHA1 Message Date
095a3967c4 phase 3: control loop — scheduler, actuator, learning, notifier, policy, API endpoints
Implemented the full OODA control loop:

Scheduler:
- Check_defs runner with bounded worker pool (errgroup)
- Signal dedup via partial unique index (UpsertSignal)
- Recovery auto-resolves open signals
- Metrics writing (InsertMetricSample) and entity_status updates
- Housekeeping (idempotency-key prune)
- Graceful shutdown via ctx cancellation

Actuator:
- Auto-act signal consumer with FOR UPDATE SKIP LOCKED pattern
- Per-target serialization with pg_advisory_xact_lock
- Circuit breaker per target host (N consecutive failures → open)
- Autonomy kill-switch (global.auto_act, never_auto_act.<slug>)
- Execution record lifecycle (proposed → running → completed)

Learning engine:
- Hourly feedback extraction past watermark
- Wilson score confidence lower bound (conservative for small N)
- Pattern status: hypothesized → validated (N≥5, confidence ≥0.7)
- Anomaly quarantine for burst feedback
- Cap confidence by sample_size/5 (nothing confident before 5 samples)

Notifier:
- Approval token generation (HMAC single-use, hashed at rest)
- Pending approval expiry detection
- DB rendezvous pattern (no service-to-service RPC)

Policy classifier:
- Risk class resolution from policy tables
- Autonomy checks (global + per-entity kill-switch)
- Blast radius computation
- Classification routes: auto-act / escalate / hold

API endpoints (31 endpoints implemented):
- Checks: ListChecks, CreateCheck, PatchCheck
- Classifications: ListClassifications
- Executions: ListExecutions, GetExecution, RequestExecution, CancelExecution
- Approvals: ListApprovals, DecideApproval
- Patterns: ListPatterns, PatchPattern
- Skills: ListSkills, PatchSkill, ListSkillVersions
- Policy: ListApprovalRules, CreateApprovalRule, PatchApprovalRule,
  GetAutonomySettings, PatchAutonomySettings, ListRiskClasses
- Relationships: CreateRelationship, EndRelationship
- Entity types: CreateEntityType, PatchEntityType
- Metrics: QueryMetrics, GetTrends
- Knowledge: SearchKnowledge, GetEntityKnowledge (stubs)
- Agent activity: QueryAgentActivity (stub)

Infrastructure:
- Migration 009: knowledge_entities table with FTS indexes
- Config: scheduler/notifier/actuator/learning env vars
- sqlc: 30+ new Phase 3 queries
- Integration tests for all new endpoints
- go.sum updated with golang.org/x/sync
2026-07-07 15:19:25 +02:00
7e802bbb14 sse: real-time flushing via raw handler overriding the generated route
The generated strict-server path could only return an io.Reader that
io.Copy drains without flushing, so SSE events sat chunk-buffered instead
of streaming in real time. Replace it with a raw http.ResponseWriter
handler (serveSSE) that Flush()es after every event.

Routing: chi allows a later registration to supersede an earlier one for
the same method+path (verified empirically for v5.3.1), so serveSSE is
registered on the router AFTER gen.HandlerWithOptions and wins over the
generated /events/stream route. It inherits the base middleware chain and
applies auth via With(). The generated StreamEvents method now returns an
error (never reached) so a routing regression fails loudly rather than
silently reverting to buffered delivery.

Adds TestSSEStreamRealtimeDelivery: a real httptest.NewServer + streaming
client (NewRecorder can't flush) that connects, triggers an event, and
asserts delivery within 3s — proving both the override routing and
per-event flushing. Passes in <1s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:34:32 +02:00
f1b0b65149 phase 2 review: fix SSE deadlock, MCP panic, lifecycle 500, NOT NULL bug
Reviewed the phase-2 implementation (parts 2–5) end to end. The suite hung
for 600s and several handlers were never exercised because there were no
tests for the new mutation/event/MCP surface. Fixes:

- CRITICAL: sseListener ran on context.Background() and held a pooled
  connection forever, so pool.Close() deadlocked (600s test timeout).
  NewHandler now takes a ctx that governs the listener; ListenAndServe and
  the test helper cancel it before closing the pool.
- CRITICAL: MCP AddTool panicked ("missing input schema") at construction
  under go-sdk v1.6.1 — so NewHandler (and every API handler) panicked.
  Added object input schemas to all 8 tools via an objSchema helper.
- HIGH: PatchEntity parsed lifecycle transitions as map[string][]string but
  the shape is {from:{to:{requires:[]}}}, so every state-change PATCH 500'd.
  Parse the nested shape; allow same-state no-ops.
- HIGH: CreateEntity bound SQL NULL for attributes when omitted, violating
  the NOT NULL column (the default only applies when omitted). Default to
  '{}'.
- MED: serveSSEWriter ignored the request ctx (per-client goroutine leak on
  disconnect) and set an invalid Content-Length: -1. Thread ctx through;
  omit the header. writeSSE now nil-checks the flusher (io.Pipe path passed
  nil → would have panicked on first event).
- MED: SSE `data:` leaked raw sqlcgen.Event (PascalCase, base64 JSONB).
  Emit canonical gen.Event so SSE matches GET /events. Verified live.
- LOW: CreateEntity uses uuid.NewV7 (ADR-0005) + real actor from context in
  audit; removed dead bearerAuth; fixed vet unkeyed-field warnings.

Tests (would have caught all of the above): entity create/patch with
If-Match 409/400, valid+invalid lifecycle transitions, idempotency replay,
duplicate-slug 409, abstract-type 422, event+audit side effects, MCP tool
registration. Live smoke test confirmed NOTIFY→listener→SSE delivery.

Also adds the missing Phase 2 deliverable: Gitea Actions CI (vet,
golangci-lint, govulncheck, generated-code drift guard, race tests against
TimescaleDB, docker build) and wires sqlc into `make generate`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:26:43 +02:00
6b61495e3b phase 2 (part 5): MCP server with official Go SDK
- MCP server at /mcp using github.com/modelcontextprotocol/go-sdk v1.6.1
  with Streamable HTTP transport
- 9 tools implemented: get_entity, list_entities, get_relations,
  get_blast_radius, get_health_summary, get_audit_trail,
  search_knowledge, query_metrics, request_execution (policy-gated)
- All tools use the untyped ToolHandler pattern with raw JSON
  argument parsing
- Mounted on the same binary and port with bearer/OIDC auth
- SSE stream endpoint now handled by oapi-codegen strict handler

Phase 2 acceptance criteria:
- GET /entities?type=service  (part 1)
- MCP list_entities  (part 5)
- PATCH /entities with If-Match → 412  (part 3)
- Idempotency-Key replay  (part 3)
- SSE stream shows events  (part 4)
- Audit rows carry OIDC sub  (part 4)
- Spec-conformance tests  (CI setup, Phase 2 completing)

Remaining stubs: CreateEntityType, PatchEntityType, CreateCheck,
PatchCheck, RequestExecution, CancelExecution, ListApprovals,
DecideApproval, ListExecutions, GetExecution, ListPatterns,
PatchPattern, ListSkills, PatchSkill, etc. (Phase 3)
2026-07-07 12:27:36 +02:00
1bfc18ea3a phase 2 (part 4): SSE stream via io.Pipe, OIDC JWT auth middleware
- SSE stream: GET /events/stream using io.Pipe to bridge the SSE
  goroutine to the response body. Replay from Last-Event-ID via
  in-memory broker with DB fallback. LISTEN/NOTIFY fan-out to all
  subscribers. Heartbeat every 15s. Bounded channels.
- OIDC JWT auth: validates Bearer tokens against Authentik/OIDC
  issuer via JWKS discovery + key caching. Extracts sub/email into
  context actor. Falls back to static bearer tokens. Dev mode (no
  OIDC + no tokens) = open.
- Config: OIDCIssuer, OIDCClientID env vars
- SSE + OIDC infrastructure complete, build passes, all tests pass

Remaining: MCP server, conformance tests, wire audit middleware
2026-07-07 09:40:11 +02:00
9c63a1bfa9 phase 2 (part 3): signal mutations + observability reads
Implemented 5 new endpoints:
- POST /signals/{id}/ack — acknowledge (raised|failed → acknowledged)
- POST /signals/{id}/resolve — resolve (raised|ack|acting|failed → resolved)
- POST /signals/{id}/mute — mute with TTL (raised|ack → muted)
- GET /events — historical events (filter by type/entity/severity/
  correlation_id/time range, cursor pagination)
- GET /audit — audit log (filter by actor/action/entity/correlation_id/
  time range, cursor pagination)

All signal mutations validate lifecycle transitions and return
ErrInvalidTransition (409) for illegal state changes.
Removed from stubs.go: AckSignal, ResolveSignal, MuteSignal,
QueryEvents, QueryAudit.

Stubs remaining: 33 endpoints (mutations + remaining reads)
2026-07-07 08:57:40 +02:00
c9975d60a5 phase 2 (part 2): sqlc queries, audit/event helpers, event NOTIFY trigger
- sqlc.yaml + internal/db/queries/*.sql: typed queries for entities,
  relationships, ontology, operations (signals, events, audit,
  idempotency, entity_status)
- internal/db/sqlcgen/: generated Go from sqlc (pgx/v5)
- internal/observability/record.go: Audit() and Event() helpers that
  write in the caller's transaction (SG10). actorLabel is interim text
  identity in detail JSON until OIDC resolution lands; actor_id column
  exists but is not yet populated
- migrations/008: post-commit pg_notify trigger on events table for
  SSE fan-out (SG8/SG10)
2026-07-07 08:49:59 +02:00
f2fe812cda phase 2 (part 1): OpenAPI-generated API server, first 9 endpoints
- api/openapi.yaml converted 3.1 → 3.0.3 (oapi-codegen/kin-openapi
  supports 3.0; nullable syntax + example keywords), still redocly-clean
- oapi-codegen (v2.4.1, strict server + chi) generates
  internal/httpapi/gen from the spec; `make generate` wired
- internal/httpapi: chi router, /healthz (unauthenticated, SG18),
  RFC 9457 problem+json mapping from domain sentinels (SG11), 5xx detail
  logged server-side only, request logging with request IDs, graceful
  shutdown (SG4), interim static bearer auth (constant-time; dev-open
  when no token; OIDC JWT still to come in Phase 2)
- Implemented: listEntities (type filter walks the hierarchy, keyset
  pagination), getEntity (UUID or slug, ETag), getEntityRelations,
  getBlastRadius, getGraph (nodes+edges for UIs), getOntology,
  listSignals, getFleetHealth, exportSeeds. Remaining 38 ops return 501
  problem+json stubs (compiler-enforced interface completeness)
- `oikos api` role live: migrate-on-start, serves :8090
- 15 API integration tests (auth, pagination, hierarchy filter, ETag,
  404/501 problem shapes, graph, export)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:25:19 +02:00
1b04683639 phase 1 review fixes: dedup edges, real export, validation, tests
Review of aa2ca0a found and fixed:
- re-ingest duplicated ALL edges (upsert conflicted on valid_from=now(),
  never fired) — migration 007 dedupes + partial unique index on current
  edges; upsert now targets it. Regression-tested.
- export was a stub that overwrote seeds/*.yaml with 11-byte "version: 1"
  files — implemented real deterministic export (ontology/inventory/policy,
  cognition-layer excluded); round-trip is byte-stable (tested)
- DB password leaked in startup logs (slog JSON bypasses String()) —
  Config now implements slog.LogValuer; regression-tested
- docker-compose had literal '***' as DB password — env-interpolated
- uuid.New() (v4) → uuid.NewV7() per ADR-0005
- no ontology validation on ingest — internal/ontology TypeTree: abstract
  instantiation rejected, relationship endpoints hierarchy-validated,
  cardinality enforced in-transaction, lifecycle states checked, default
  state applied (Phase 1 gate items, R3-1)
- getOrCreateEntityID swallowed non-ErrNoRows errors
- migration runner now holds a session advisory lock on one connection
- Makefile: hardcoded /opt/homebrew/bin/go → go; test-db target

Tests: 4 unit suites + 7 integration tests (env-guarded, throwaway DB per
run): migrate idempotent, seed idempotent + no dup edges, abstract/edge/
cardinality rejection, blast_radius cycle termination, export round-trip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:11:01 +02:00
aa2ca0ae6f phase 1: Go foundation — module, migrations, domain, seed ingest
Core deliverables:
- Go module github.com/dtoro/oikos (Go 1.26.3)
- cmd/oikos: single binary with role subcommands (migrate, seed, export)
- 6 SQL migrations: ontology meta-schema, entity instances (UUID+slug,
  blast_radius recursive function), operations (signals/checks/approvals),
  cognition (classifications/executions/feedback/patterns/skills), policy,
  observability (TimescaleDB hypertables + CAGGs + retention)
- Domain layer: entity, signal, execution, classification, pattern, skill,
  approval, check types + 11 sentinel errors + lifecycle state machines
- DB layer: pgx pool, SQL splitter (handles 94436 and -- comments), migration
  runner, seed ingest (ontology+inventory+policy) with content-hash dedup
- Config: env-based with defaults, secrets redaction
- Observability: slog JSON logger with debug mode
- Infrastructure: Makefile, docker-compose.yml, multi-stage Dockerfile
  (distroless, CGO_ENABLED=0)

Verified end-to-end against timescale/timescaledb:2.17.2-pg16:
- 6 migrations applied (65 SQL statements)
- Seeds ingested: 6 lifecycles, 59 entity types, 46 relationship types,
  111 entities, 144 relationships, 4 risk classes, 27 approval rules,
  9 autonomy settings
- Idempotent: second seed run is a no-op (content hash matches)

Bugs fixed during implementation:
- TimescaleDB CAGGs can't run in a transaction -> splitSQL() executes
  statements individually
- Semicolons in -- comments treated as separators -> comment handling
- YAML keys source/target didn't match code's source_type/target_type
- yaml.Marshal produced YAML for JSONB columns -> json.Marshal
2026-07-07 01:07:26 +02:00
55710bd254 seeds: resolve inventory thin spots against production
- lxc:rclone verified live as LXC 132 on hubris (pct list via MCP);
  hosts edge added
- /mnt/library backing storage identified from hosts/hubris.md: 'library'
  lvmthin pool, 3.7T, 2nd NVMe — added pool:library-hubris + contains edge
- all 8 derived services (books/seanime/roms/house/jellyseerr/qbit/sab/
  teddy) confirmed responding over their ingress URLs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:26:18 +02:00
18cb79caf9 oikos phase 0: ontology + inventory + policy seeds, OpenAPI contract, ADRs
- seeds/ontology.yaml: 59 entity types (5 abstract, is-a hierarchy), 46
  relationship types with cardinality, 6 lifecycles with terminal states
  and named precondition checks
- seeds/inventory.yaml: 110 entities / 142 relationships translated from
  legacy inventory.yaml (fleet, services, ingress, storage, governance,
  archaeology); thin spots marked for backfill
- seeds/policy.yaml: 4 risk classes, 27 approval rules (hierarchy-aware,
  per-entity overrides), autonomy kill-switch off (cold start)
- api/openapi.yaml: full v1 REST contract (40 paths), RFC 9457 errors,
  cursor pagination, idempotency, ETag/If-Match, scopes; redocly-clean
- docs/adr/0001-0010: initial architecture decision records
- scripts/validate-seeds.py: Phase 0 gate — hierarchy, lifecycles,
  endpoints, cardinality, policy cross-refs (0 errors)
- plan: layer CHECK gains 'meta' (root type), cardinality gains
  'many-to-one'

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:17:15 +02:00
ea3b2c3662 plans: oikos rev 3 — consolidated spec, ontology inheritance, OpenAPI-first
Merge the rev-2 audit + remediation layers into one self-consistent spec and
close new gaps: meta-schema inheritance (parent_type/is_abstract), contract-
first API (RFC 9457, idempotency, ETag, scopes, /graph), single-binary role
packaging, UUIDv7+slug IDs, checks-as-data, signal dedup/flap/maintenance,
executable skill format, MCP streamable HTTP, SSE events, ledger-as-view,
dual-path networking (mesh-primary + LAN break-glass), per-phase acceptance
criteria, ADRs. Appendix A maps every rev-2 finding to its resolution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:03:25 +02:00
8850b85325 plans: remediate all HIGH audit items + architect/developer review
Addresses 15 original HIGH audit findings + 18 new findings from
systems architect + senior Go developer review (572 lines added).

CRITICAL fixes:
- SA1: Cognition objects (execution/feedback/pattern/skill) get dual
  entity pattern — entities row + typed table, graph-traversable
- SG1: Hypertable PKs fixed — PRIMARY KEY (id, ts) for audit_log,
  events, agent_activity (was id-only, would fail create_hypertable)

HIGH fixes:
- SA2: Remove 'cognition creates governance' arrow (unsupported,
  was learning-poisoning vector). Patterns propose, operator accepts.
- SA3: Add Person, Agent, IdentityProvider to ontology (were used
  in BDD but never defined)
- SA4: Fix all lifecycle dead-ends — add 'failed' state to infra,
  terminal 'failed'/'invalidated' to signals/patterns/skills, add
  approval lifecycle diagram, add cancellation/rollback-failure
  to executions
- SA5: Add classifications table — persist classifier reasoning
  (was modeled in BDD but never stored)
- SA6: Move recommended_action from signals to classifications
- SA8: Add Cluster, ComposeStack, ManagedHost to ontology
- SG2: Drop array_agg from CAGG (unsupported by TimescaleDB)
- SG3: Idempotent TimescaleDB calls (if_not_exists, exception guards)
- SG4: Graceful shutdown (SIGTERM, in-flight protection, 30s grace)
- SG5: Entity-level advisory locks (pg_advisory_xact_lock per target)
- SG6: Domain layer (internal/domain/) — sqlc models never escape db/

Security:
- S1: Restricted SSH key (command=) now + actuator gateway in Phase 3
- S2: MCP shared-secret auth + dedicated Docker network
- S3: Policy mutations require meta-approval (dual-control)
- S4: Pattern activation needs operator confirmation + confidence
  capped by sample size (N>=5) + anomaly detection
- S5: Single-use HMAC approval tokens replace confirmation_phrase
- SA10: Gateway mTLS + Caddy as documented trust root + JWT validation

Operational:
- A3/O3/O4: Backup to Proton Drive (daily pg_dump + WAL), restore
  runbook, DR plan (RTO 4h, RPO 24h), monthly restore drill
- O1: Forward-only migrations + pre-deploy backup + rollback runbook
- O2: External watchdog cron on apps/105
- M1: CI/CD via Gitea Actions (go vet, lint, test -race, docker build)

Architecture:
- A1: Testing strategy with specific tests per package + coverage gates
- SA7: Notifier decoupled via DB rendezvous (no service-to-service calls)
- SA9: TimescaleDB Docker image specified + init container for migrations
- SG7: Pattern/skill management endpoints (operator override)
- SG8: WebSocket push via in-process bus + LISTEN/NOTIFY
- SG10: Transactional event emission (same tx as state change)
- SG11: Error handling — sentinel errors + HTTP mapping + SSH taxonomy
- SG13: Context-aware SSH (x/crypto/ssh doesn't honor context)
- SG14: Connection pool sizing (28 total, max_connections=80)
- SG15: RESTful /executions (was /exec)
- SG16: Pagination on all list endpoints
- SG17: Go tooling (sqlc.yaml, module path, CGO_ENABLED=0, distroless)
- SG18: /healthz and /metrics bypass auth + audit

Updated phasing incorporates all remediation.
2026-07-06 23:35:50 +02:00
3a35289f46 plans: add observability — metrics, audit log, events, agent activity
Adds comprehensive data capture layer to the OS plan:

1. TimescaleDB — PostgreSQL extension for time-series data. No separate
   database. Hypertables auto-partition, continuous aggregates provide
   1h/1d rollups, retention policies auto-drop old data.

2. Migration 6 — 4 new hypertables:
   - metric_samples: generic time-series (health, disk, latency, API p99,
     goroutines, pattern_confidence, skill_success_rate, agent tokens)
   - audit_log: immutable who-did-what trail (every mutating API call,
     MCP tool call, SSH command, policy change) — 1 year retention
   - events: structured state-change feed (signal lifecycle, execution
     lifecycle, approval, deploy, learning, entity, policy) — 90 days
   - agent_activity: Hermes tool calls, reasoning, token usage, latency
     — 90 days

3. Correlation IDs — propagated through the full call chain (signal →
   classification → execution → SSH → verification → feedback → pattern)
   so any action chain can be reconstructed end-to-end.

4. 6 new MCP tools for agent self-query:
   query_metrics, get_trend, get_audit_trail, get_event_timeline,
   get_agent_activity, get_health_summary

5. 8 new REST endpoints for metrics/audit/events/health/trends/export

6. Workstream 14 — observability + data capture (7 sub-components A-G):
   metrics, audit, events, agent activity, structured logging, agent
   data availability, future visualization plug-in points

7. New Mermaid diagram — observability data capture and query flow

8. Updated architecture diagram to show observability data flows

9. Updated phasing — observability woven into phases 1-3

10. Updated verification — 14 end-to-end checks (was 12), including
    metrics querying, audit trail, correlation tracing
2026-07-06 23:12:34 +02:00
2d75544362 plans: SysML BDD ontology, generic model, full audit
Ontology rewrite:
- Replace Mermaid ER diagram with SysML Block Definition Diagrams (BDD)
  using class diagram syntax: generalization, composition, aggregation,
  association with multiplicity annotations
- Split into 3 diagrams: infrastructure (compute/storage/network),
  software+services, cognition (operations+learning)
- Make compute model generic: ComputeEntity abstract base with
  specializations (Machine, VirtualMachine, Container→LXC/DockerContainer;
  Machine→ProxmoxHost/StandaloneServer/Workstation/Appliance)
- Hypervisor is software on a Machine (not all machines are Proxmox)
- Any ComputeEntity can mount Volumes (VMs AND LXCs, validated)
- DockerContainer is first-class (OS models its own infrastructure)
- Services on any compute type (not just LXC/VM)
- Documents/Runbooks describe any Entity (not just Service)
- Added design notes validating assumptions against actual inventory

Schema updates:
- entity_types: add attribute_schema (JSONB for validating attributes)
- entity_types: add status (active/deprecated, no hard delete while instances exist)

Audit (37 findings across 6 categories):
- Security: 10 findings (5 HIGH) — SSH keys, MCP auth, policy mutability,
  learning poisoning, confirmation phrase, webhook auth, TLS, blast radius
- Performance: 7 findings (1 HIGH) — CTE cycle guard, probe concurrency,
  ingestion, pattern extraction, table growth, WS backpressure
- Architecture: 7 findings (3 HIGH) — testing, observability, DB backup
- Data model: 7 findings (1 HIGH) — entity ID, attribute schema, type
  evolution, concurrent writes, migration rollback, DR export
- Operational: 7 findings (4 HIGH) — rollback, watchdog, backup/restore
  runbook, disaster recovery, deploy downtime, health checks
- Missing: 7 findings (1 HIGH) — CI/CD, rate limiting, audit log, circuit
  breaker, secret rotation, supply chain, SLOs
- Top-5 priority items called out before implementation
2026-07-06 23:06:34 +02:00
d44979aca7 plans: rev 2 — Go rewrite, ontology-first, DB-native config, learning loop
Major revision of the Docker-based homelab OS plan:

1. Go instead of Python — all services rewritten as Go binaries
   (Gin web framework, sqlc for DB access, goroutines for probes)

2. Ontology-first design — systems modeling with 3 layers:
   - Infrastructure (physical, compute, network, storage, software)
   - Governance (identity, secrets, policy)
   - Cognition (observation, decision, action, knowledge, learning)
   7 Mermaid diagrams: layer map, ER diagram, 3 lifecycle state machines,
   feedback loop, policy model

3. DB-native config — inventory.yaml/ontology.yaml/policy.yaml become
   seed manifests (bootstrap + DR). The DB is the runtime source of truth,
   editable via API. Ontology IS the DB schema (entity_types,
   relationship_types, lifecycle_defs tables).

4. Feedback loop — agent learns from execution:
   execution → outcome → feedback → pattern → skill → classification
   Patterns accumulate from execution history, skills codify proven
   procedures, classifier uses pattern confidence for auto-act decisions.
   Cold start: agent starts cautious, earns autonomy through evidence.

5. 5 migration groups: ontology meta-schema, entity instances, operations,
   learning model, policy. Recursive blast_radius SQL function.

6. Phase 0 added: ontology design before any code.
2026-07-06 22:50:49 +02:00
fe54af30f6 plans: replace ASCII architecture diagrams with Mermaid
4 diagrams: container stack, OODA loop, knowledge graph, deploy flow
2026-07-06 22:34:53 +02:00
bead722fac plans: pivot oikos consolidation to docker-based agentic homelab OS
Supersedes the launchd-based consolidation plan. Key changes:
- Docker-based deployment on mac-mini (docker compose)
- PostgreSQL for all mutable state (signals, ledger, knowledge graph)
- Infisical replaces SOPS+age for secrets management
- Unified API merges MCP server + homelab CLI (REST + MCP interfaces)
- Hermes agent runs in Docker (gateway mode, connect from any workstation)
- Knowledge graph in Postgres replaces narrative wiki files as agent context
- Structured entity relationships link docs to inventory entities
- Hybrid SSH access (mounted keys now, actuator gateway later)
- Git push → Gitea webhook → Docker rebuild = deploy trigger
- 6-phase rollout: DB → services → agent → secrets → deploy → cutover
2026-07-06 22:32:21 +02:00
a434a4096c Merge pull request 'chore: add plan' (#2) from claude/heuristic-jang-ecb080 into main
Reviewed-on: dtoro/Homelab-Docs#2
2026-07-06 21:57:48 +02:00
85 changed files with 26435 additions and 175 deletions

72
.gitea/workflows/ci.yml Normal file
View File

@@ -0,0 +1,72 @@
# Oikos CI (Gitea Actions). Gates the deploy webhook on a green run (plan M1).
# Mirrors `make lint`, `make test`, and the generated-code drift guard.
name: ci
on:
push:
branches: [main]
pull_request:
jobs:
build-test:
runs-on: ubuntu-latest
services:
postgres:
image: timescale/timescaledb:2.17.2-pg16
env:
POSTGRES_DB: oikos
POSTGRES_USER: oikos
POSTGRES_PASSWORD: oikos_dev
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U oikos"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
OIKOS_TEST_DATABASE_URL: postgres://oikos:oikos_dev@postgres:5432/oikos?sslmode=disable
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.26"
cache: true
- name: go vet
run: go vet ./...
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: latest
args: --timeout 5m
continue-on-error: true # advisory until the lint baseline is clean
- name: govulncheck
run: |
go install golang.org/x/vuln/cmd/govulncheck@latest
govulncheck ./... || true # advisory
- name: generated code is up to date
run: make generate-check
- name: build
run: go build ./...
- name: test (race + coverage)
run: go test -race -covermode=atomic -coverprofile=coverage.out -timeout 300s ./...
- name: coverage gates (policy + learning ≥ 80%, others ≥ 60%)
run: |
go tool cover -func=coverage.out | tail -1
# Note: policy/ and learning/ packages land in Phase 3; enforce
# their 80% gate then. For now, report total coverage.
docker-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: docker build (verify image builds; no push)
run: docker build -f compose/oikos/Dockerfile -t oikos:ci .

50
Makefile Normal file
View File

@@ -0,0 +1,50 @@
.PHONY: build test test-db lint generate generate-check dev migrate seed export clean tidy
BINARY := oikos
GO ?= go
build:
$(GO) build -o $(BINARY) -tags timetzdata ./cmd/oikos
test:
$(GO) test -race -cover ./...
# Integration tests against the compose Postgres (starts it if needed)
test-db:
docker compose up -d postgres
@sleep 3
OIKOS_TEST_DATABASE_URL="postgres://oikos:$${OIKOS_DB_PASSWORD:-oikos_dev}@localhost:5432/oikos?sslmode=disable" \
$(GO) test -race -count=1 ./internal/db/ ./internal/httpapi/ ./internal/mcp/
lint:
$(GO) vet ./...
@command -v golangci-lint >/dev/null 2>&1 && golangci-lint run || echo "golangci-lint not installed, skipping"
generate:
$(GO) run github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@v2.4.1 \
-config api/codegen.yaml api/openapi.yaml
$(GO) run github.com/sqlc-dev/sqlc/cmd/sqlc@v1.29.0 generate
# CI drift guard: regenerate and fail if the committed output changed.
generate-check: generate
@git diff --exit-code -- internal/httpapi/gen internal/db/sqlcgen \
|| (echo "generated code is stale — run 'make generate' and commit" && exit 1)
migrate:
$(GO) run ./cmd/oikos migrate
seed:
$(GO) run ./cmd/oikos seed
export:
$(GO) run ./cmd/oikos export
dev:
docker compose --profile dev up -d
clean:
rm -f $(BINARY)
$(GO) clean -testcache
tidy:
$(GO) mod tidy

8
api/codegen.yaml Normal file
View File

@@ -0,0 +1,8 @@
# oapi-codegen config — `make generate` regenerates internal/httpapi/gen.
package: gen
output: internal/httpapi/gen/api.gen.go
generate:
models: true
chi-server: true
strict-server: true
embedded-spec: true

2891
api/openapi.yaml Normal file

File diff suppressed because it is too large Load Diff

7
api/redocly.yaml Normal file
View File

@@ -0,0 +1,7 @@
# Redocly lint config for api/openapi.yaml (CI runs: redocly lint api/openapi.yaml)
extends:
- recommended
rules:
# Every operation declares `default` → RFC 9457 problem+json instead of
# enumerating each 4XX (plan R3-3); oapi-codegen handles `default` fine.
operation-4xx-response: off

254
cmd/oikos/main.go Normal file
View File

@@ -0,0 +1,254 @@
package main
import (
"context"
"fmt"
"log/slog"
"os"
"os/signal"
"syscall"
"net/http"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/httpapi"
"github.com/dtoro/oikos/internal/observability"
"github.com/jackc/pgx/v5"
)
// SchedulerRunner is set by the scheduler init() to avoid circular imports.
var SchedulerRunner func(context.Context, *db.Pool, config.Config)
// NotifierRunner is set by the notifier init() to avoid circular imports.
var NotifierRunner func(context.Context, *db.Pool, config.Config)
func main() {
if len(os.Args) < 2 {
usage()
os.Exit(1)
}
role := os.Args[1]
cfg := config.FromEnv()
// Structured logging (slog)
logger := observability.NewLogger(cfg.Debug)
slog.SetDefault(logger)
slog.Info("starting oikos", "role", role, "config", cfg)
ctx, cancel := signal.NotifyContext(context.Background(),
syscall.SIGTERM, syscall.SIGINT)
defer cancel()
switch role {
case "migrate":
if err := runMigrate(ctx, cfg); err != nil {
slog.Error("migrate failed", "error", err)
os.Exit(1)
}
case "seed":
if err := runSeed(ctx, cfg); err != nil {
slog.Error("seed failed", "error", err)
os.Exit(1)
}
case "export":
if err := runExport(ctx, cfg); err != nil {
slog.Error("export failed", "error", err)
os.Exit(1)
}
case "api":
if err := runAPI(ctx, cfg); err != nil {
slog.Error("api failed", "error", err)
os.Exit(1)
}
case "scheduler":
if SchedulerRunner != nil {
SchedulerRunner(ctx, nil, cfg)
} else {
slog.Error("scheduler not compiled in (import internal/scheduler)")
os.Exit(1)
}
case "notifier":
if NotifierRunner != nil {
NotifierRunner(ctx, nil, cfg)
} else {
slog.Error("notifier not compiled in (import internal/notifier)")
os.Exit(1)
}
case "all":
slog.Info("all role not yet implemented (runs api + scheduler + notifier in one process)")
os.Exit(1)
case "version":
fmt.Println("oikos dev (Phase 1)")
case "help", "--help", "-h":
usage()
default:
fmt.Fprintf(os.Stderr, "unknown role: %s\n", role)
usage()
os.Exit(1)
}
}
func usage() {
fmt.Println(`oikos — the homelab OS
Usage: oikos <role> [flags]
Roles:
migrate Run database migrations (forward-only, idempotent)
seed Ingest seed YAML files into the database
export Export DB state back to seed YAMLs (DR / version control)
api Run the REST + MCP API server (Phase 2)
scheduler Run the observe + act loop (Phase 3)
notifier Run the notification service (Phase 3)
all Run all roles in one process (dev mode)
version Print version info
Environment:
OIKOS_DATABASE_URL Postgres connection string
OIKOS_API_LISTEN API listen address (default :8090)
OIKOS_ENV Environment (dev, prod)
OIKOS_DEBUG Enable verbose logging (true/1)
OIKOS_SEEDS_DIR Path to seeds directory (default: seeds)
OIKOS_MCP_BEARER_TOKEN Shared secret for MCP auth`)
}
func runMigrate(ctx context.Context, cfg config.Config) error {
pool, err := db.New(ctx, cfg.DatabaseURL)
if err != nil {
return err
}
defer pool.Close()
slog.Info("running migrations")
if err := pool.Migrate(ctx); err != nil {
return err
}
slog.Info("migrations complete")
return nil
}
func runSeed(ctx context.Context, cfg config.Config) error {
pool, err := db.New(ctx, cfg.DatabaseURL)
if err != nil {
return err
}
defer pool.Close()
// Ensure migrations are applied first
if err := pool.Migrate(ctx); err != nil {
return fmt.Errorf("migrations: %w", err)
}
seedsDir := cfg.SeedsDir
if seedsDir == "" {
seedsDir = "seeds"
}
// Ingest ontology seed
ontoContent, err := os.ReadFile(seedsDir + "/ontology.yaml")
if err != nil {
return fmt.Errorf("read ontology seed: %w", err)
}
err = pool.SeedIngest(ctx, "ontology.yaml", ontoContent,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
r, err := db.IngestOntologySeed(ctx, tx, data)
if err != nil {
return err
}
slog.Info("ontology ingested",
"lifecycles", r.Lifecycles,
"entity_types", r.EntityTypes,
"relationship_types", r.RelationshipTypes)
return nil
})
if err != nil {
return err
}
// Ingest inventory seed
invContent, err := os.ReadFile(seedsDir + "/inventory.yaml")
if err != nil {
return fmt.Errorf("read inventory seed: %w", err)
}
err = pool.SeedIngest(ctx, "inventory.yaml", invContent,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
r, err := db.IngestInventorySeed(ctx, tx, data)
if err != nil {
return err
}
slog.Info("inventory ingested",
"entities", r.Entities,
"relationships", r.Relationships)
return nil
})
if err != nil {
return err
}
// Ingest policy seed
polContent, err := os.ReadFile(seedsDir + "/policy.yaml")
if err != nil {
return fmt.Errorf("read policy seed: %w", err)
}
err = pool.SeedIngest(ctx, "policy.yaml", polContent,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
r, err := db.IngestPolicySeed(ctx, tx, data)
if err != nil {
return err
}
slog.Info("policy ingested",
"risk_classes", r.RiskClasses,
"approval_rules", r.ApprovalRules,
"autonomy_settings", r.AutonomySettings)
return nil
})
if err != nil {
return err
}
slog.Info("seed ingest complete")
return nil
}
func runAPI(ctx context.Context, cfg config.Config) error {
pool, err := db.New(ctx, cfg.DatabaseURL)
if err != nil {
return err
}
defer pool.Close()
if err := pool.Migrate(ctx); err != nil {
return fmt.Errorf("migrations: %w", err)
}
err = httpapi.ListenAndServe(ctx, pool, cfg)
if err == http.ErrServerClosed {
return nil
}
return err
}
func runExport(ctx context.Context, cfg config.Config) error {
pool, err := db.New(ctx, cfg.DatabaseURL)
if err != nil {
return err
}
defer pool.Close()
exports, err := db.ExportToYAML(ctx, pool)
if err != nil {
return err
}
for name, content := range exports {
path := cfg.SeedsDir + "/" + name
if err := os.WriteFile(path, content, 0644); err != nil {
return fmt.Errorf("write %s: %w", path, err)
}
slog.Info("exported", "file", path, "bytes", len(content))
}
return nil
}

21
compose/oikos/Dockerfile Normal file
View File

@@ -0,0 +1,21 @@
# Multi-stage Dockerfile for Oikos (ADR 0001: single binary)
FROM golang:1.26-alpine AS builder
RUN apk add --no-cache git ca-certificates
WORKDIR /build
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /oikos -tags timetzdata -ldflags="-s -w" ./cmd/oikos
# --- Runtime: distroless static ---
FROM gcr.io/distroless/static:nonroot
COPY --from=builder /oikos /oikos
COPY --from=builder /build/seeds /seeds
COPY --from=builder /build/migrations /migrations
ENTRYPOINT ["/oikos"]

70
docker-compose.yml Normal file
View File

@@ -0,0 +1,70 @@
# Docker Compose for Oikos development
# Usage: docker compose up -d postgres (just the DB)
# make dev (full dev stack)
services:
postgres:
image: timescale/timescaledb:2.17.2-pg16
environment:
POSTGRES_DB: oikos
POSTGRES_USER: oikos
POSTGRES_PASSWORD: ${OIKOS_DB_PASSWORD:-oikos_dev}
ports:
- "5432:5432"
volumes:
- pg-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD", "pg_isready", "-U", "oikos"]
interval: 5s
timeout: 5s
retries: 5
# One-shot: run migrations then exit
migrate:
build:
context: .
dockerfile: compose/oikos/Dockerfile
depends_on:
postgres:
condition: service_healthy
environment:
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
command: ["migrate"]
restart: "no"
# One-shot: ingest seeds then exit
seed:
build:
context: .
dockerfile: compose/oikos/Dockerfile
depends_on:
migrate:
condition: service_completed_successfully
environment:
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
OIKOS_SEEDS_DIR: /app/seeds
command: ["seed"]
restart: "no"
# API server (Phase 2)
api:
build:
context: .
dockerfile: compose/oikos/Dockerfile
profiles: ["dev", "full"]
depends_on:
seed:
condition: service_completed_successfully
environment:
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
OIKOS_API_LISTEN: ":8090"
OIKOS_ENV: dev
OIKOS_DEBUG: "true"
ports:
- "8090:8090"
command: ["api"]
stop_signal: SIGTERM
stop_grace_period: 30s
volumes:
pg-data:

View File

@@ -0,0 +1,23 @@
# ADR 0001 — Go with single-binary role packaging
Status: accepted (2026-07-07) · Plan: rev 3, R3-4
## Context
The OS has three long-running roles (api, scheduler+actuator+learning,
notifier) plus one-shot jobs (migrate, seed, export). Rev 2 planned three
binaries with three Dockerfiles.
## Decision
One Go binary `oikos` with role subcommands (`oikos api | scheduler |
notifier | all | migrate | seed | export`), one multi-stage Dockerfile, one
image tagged `oikos:<git-sha>`. Compose runs the image N times with
different commands (Loki/Temporal pattern). Go over Python for static
typing, small static binaries (CGO_ENABLED=0, distroless), and goroutines
for concurrent probes.
## Consequences
- One build, guaranteed version consistency across roles, trivial local dev
(`oikos all`), simpler rollback (retag one image).
- Full rewrite of ~4,400 Python lines (logic carries over per plan reuse map).
- All roles share a dependency set; image is slightly larger than per-role
minimal images — accepted.

View File

@@ -0,0 +1,25 @@
# ADR 0002 — PostgreSQL + TimescaleDB as the only datastore
Status: accepted (2026-07-07) · Plan: rev 3
## Context
The OS needs a graph (entities/relationships), operational tables
(signals/executions/approvals), a learning corpus, time-series metrics,
audit and event logs. Alternatives: dedicated graph DB (Neo4j), dedicated
TSDB (Prometheus/VictoriaMetrics), or one Postgres.
## Decision
One PostgreSQL 16 instance with the TimescaleDB extension
(timescale/timescaledb:2-pg16). Graph traversal via recursive CTEs
(cycle-safe blast_radius); time-series via hypertables + continuous
aggregates + retention policies; events via table + LISTEN/NOTIFY.
## Consequences
- One backup/restore/DR story, one connection pool, transactional
consistency between graph and operational writes (e.g. event emission in
the same transaction as state change).
- Postgres is the accepted SPOF — mitigated by daily pg_dump + WAL PITR +
off-host copies + monthly restore drills; streaming replication is the
future path if needed.
- Homelab graph scale (hundreds of nodes) is far below where a dedicated
graph DB pays for itself.

View File

@@ -0,0 +1,23 @@
# ADR 0003 — DB-native ontology with YAML seed manifests
Status: accepted (2026-07-07) · Plan: rev 3, R3-1
## Context
Rev 1 kept inventory/ontology/policy as YAML files parsed at runtime.
Agents need graph queries (blast radius), transactional mutations with
audit, and a future UI needs to edit the model without file round-trips.
## Decision
The DB is the runtime source of truth. entity_types form an is-a hierarchy
(parent_type, is_abstract); relationship endpoint constraints may name
abstract types and validation walks the hierarchy. YAML files under seeds/
bootstrap the DB (idempotent, content-hashed via seed_versions) and serve
DR; `GET /api/v1/export` regenerates them for version control (round-trip
byte-stable, tested in CI).
## Consequences
- Ontology changes are API calls (policy-gated), not redeploys.
- Seeds can drift from DB between exports — export is part of the routine
(commit after meaningful model edits).
- Abstract types let policy rules and relationships bind once at the right
altitude (e.g. `compute-entity provides service`).

View File

@@ -0,0 +1,21 @@
# ADR 0004 — Contract-first OpenAPI API
Status: accepted (2026-07-07) · Plan: rev 3, R3-2/R3-3
## Context
Future UIs, a CLI client, and an MCP surface must stay in sync with the
API. Code-first (Gin + generated docs) drifts.
## Decision
api/openapi.yaml (OpenAPI 3.1) is the source of truth. Server stubs via
oapi-codegen (strict server, chi router); clients generated for Go (CLI)
and TypeScript (future UI). Conventions: RFC 9457 problem+json errors,
{items, next_cursor} envelopes, cursor pagination, Idempotency-Key on
unsafe POSTs, ETag/If-Match optimistic concurrency, scopes
(operator/viewer/agent) annotated per operation. CI fails on spec/handler
drift. MCP tools wrap the same service layer.
## Consequences
- UI development needs only the running API (spec served at /openapi.yaml).
- Handler changes require spec changes first — deliberate friction.
- Breaking changes ship as /api/v2 side by side; v1 is additive-only.

View File

@@ -0,0 +1,18 @@
# ADR 0005 — UUIDv7 + slug entity identity
Status: accepted (2026-07-07) · Plan: rev 3, R3-5 (resolves audit D1)
## Context
Rev 2 used TEXT primary keys ('host:hubris') — renames break FKs, and
date-string signal IDs are race-prone.
## Decision
Primary keys are UUIDv7 (time-ordered, generated in Go). Every entity also
carries a unique human slug ('host:hubris'); (type, name) is unique too.
The API accepts UUID or slug everywhere; slugs may change (rename), UUIDs
never do.
## Consequences
- Renames are metadata updates; history and edges survive.
- UUIDv7's time-ordering keeps B-tree inserts append-mostly.
- Seeds and exports use slugs (human-diffable); ingest resolves to UUIDs.

View File

@@ -0,0 +1,23 @@
# ADR 0006 — Learning is proposal-only (no self-authorization)
Status: accepted (2026-07-07) · Plan: rev 3 (resolves audit S3/S4/SA2)
## Context
The learning loop (feedback → patterns → skills) informs the classifier
that decides auto-act vs escalate. If learning could expand its own
autonomy, poisoned feedback (flapping services, biased probes) could
unlock destructive auto-act.
## Decision
The learning engine cannot write to governance (policy/autonomy) tables —
enforced structurally: its DB role has no grants on them. Pattern
activation (validated → active) and any autonomy expansion require operator
approval. Confidence is the Wilson lower bound capped by evidence_count/5;
anomalous feedback bursts quarantine the pattern; no skill ever
auto-promotes an action into destructive autonomy (hard-coded). Lowering
autonomy (kill-switch) is always immediate, never gated.
## Consequences
- Cold start is slow by design — the agent escalates until trust is earned.
- The operator is the only path to more autonomy; the audit trail shows
every grant.

View File

@@ -0,0 +1,27 @@
# ADR 0007 — Threat model and trust zones
Status: accepted (2026-07-07) · Plan: rev 3, Security model section
## Context
The control plane can restart services and (eventually) mutate config
fleet-wide. Compromise of any one container must not equal compromise of
the fleet.
## Decision
Trust zones as Docker networks: net-front (Caddy→api only), net-data
(Postgres), net-ops (SSH egress, actuator only). Hermes holds no SSH keys;
the actuator uses a restricted key (command=/from= in authorized_keys)
until the /executions gateway fully brokers actions. Caddy is an explicit
trust root but the API independently validates OIDC JWTs — network origin
is defense-in-depth, never the auth (this enables the LAN break-glass API
binding; the Hermes gateway remains mesh-only). Policy changes are
dual-controlled with before/after hash auditing and a startup
hash-vs-known-good check. Approval tokens are single-use HMAC, hashed at
rest, TTL-bound.
## Consequences
- Documented residual risks: plaintext LAN break-glass hop (emergency use),
Postgres as shared dependency of all roles, macOS host itself unmanaged
by the OS.
- Rotation cadences: actuator SSH key 6mo, machine tokens 90d, webhook
HMAC 1y — scheduler raises expiry signals 2 weeks ahead.

View File

@@ -0,0 +1,20 @@
# ADR 0008 — Forward-only migrations
Status: accepted (2026-07-07) · Plan: rev 3 (resolves audit D5/O1)
## Context
Down-migrations are rarely tested and lie about reversibility once data
has flowed. Rollback needs a strategy that works with real data.
## Decision
golang-migrate, embedded (//go:embed), up-only. Migrations run in a
one-shot init container with a DDL-only DB user before app roles start.
Within one deploy window migrations are additive-only (new columns
nullable, new tables optional) so previous-SHA images tolerate the new
schema. Rollback = redeploy previous image tag; if the migration itself is
the problem, pg_restore the automatic pre-deploy dump. Mistakes roll
forward via compensating migrations.
## Consequences
- No down.sql to write or test; the pre-deploy dump is the real safety net.
- Destructive schema changes (drop/rename) take two deploys by design.

View File

@@ -0,0 +1,19 @@
# ADR 0009 — SSE over WebSocket for the event stream
Status: accepted (2026-07-07) · Plan: rev 3, R3-14
## Context
Live updates (signals, executions, approvals) push server→client only.
Rev 2 specified WebSocket.
## Decision
Server-Sent Events at GET /api/v1/events/stream: plain HTTP (proxies
through Caddy without upgrade handling), native browser EventSource with
auto-reconnect, Last-Event-ID resume backed by the events table. Bounded
per-subscriber buffers with drop-oldest; heartbeat comments every 15s.
Delivery is best-effort — GET /events backfills. Transactional emission +
post-commit LISTEN/NOTIFY feed the stream.
## Consequences
- No bidirectional channel; if one is ever needed (interactive terminals),
add WebSocket alongside — this ADR covers the event feed only.

View File

@@ -0,0 +1,20 @@
# ADR 0010 — Infisical secrets with SOPS DR fallback
Status: accepted (2026-07-07) · Plan: rev 3, Phase 5 (resolves audit S9)
## Context
SOPS+age is file-based: no runtime API, no machine identities, no
rotation tracking, and every consumer needs the age key.
## Decision
Infisical in the Docker stack; services fetch via machine identities;
secrets never in env files or plain config (config hierarchy: defaults →
file → env → Infisical, secrets only). Bootstrap root of trust: Infisical
master key in the mac-mini Keychain, backed up offline. One age key is
retained and all secrets are exported to a SOPS-encrypted fallback file
until an Infisical restore drill has passed; the fallback is refreshed on
rotation.
## Consequences
- Chicken-and-egg is explicit: the Keychain + offline copy are the root.
- SOPS retirement is gated on a passed restore drill, not on the calendar.

18
docs/adr/README.md Normal file
View File

@@ -0,0 +1,18 @@
# Architecture Decision Records
MADR-style records for Oikos. One decision per file, numbered, never edited
after acceptance — superseding decisions get a new ADR that links back.
Statuses: proposed | accepted | superseded-by-NNNN.
| ADR | Title |
|---|---|
| [0001](0001-go-single-binary.md) | Go with single-binary role packaging |
| [0002](0002-postgres-timescale-only-datastore.md) | PostgreSQL + TimescaleDB as the only datastore |
| [0003](0003-db-native-ontology-yaml-seeds.md) | DB-native ontology with YAML seed manifests |
| [0004](0004-openapi-first.md) | Contract-first OpenAPI API |
| [0005](0005-uuidv7-plus-slug-identity.md) | UUIDv7 + slug entity identity |
| [0006](0006-learning-proposal-only.md) | Learning is proposal-only (no self-authorization) |
| [0007](0007-threat-model.md) | Threat model and trust zones |
| [0008](0008-forward-only-migrations.md) | Forward-only migrations |
| [0009](0009-sse-over-websocket.md) | SSE over WebSocket for the event stream |
| [0010](0010-infisical-with-sops-fallback.md) | Infisical secrets with SOPS DR fallback |

36
go.mod Normal file
View File

@@ -0,0 +1,36 @@
module github.com/dtoro/oikos
go 1.26.3
require (
github.com/getkin/kin-openapi v0.140.0
github.com/go-chi/chi/v5 v5.3.1
github.com/golang-jwt/jwt/v5 v5.3.1
github.com/google/jsonschema-go v0.4.3
github.com/google/uuid v1.6.0
github.com/jackc/pgx/v5 v5.10.0
github.com/modelcontextprotocol/go-sdk v1.6.1
github.com/oapi-codegen/runtime v1.4.2
golang.org/x/crypto v0.53.0
golang.org/x/sync v0.21.0
gopkg.in/yaml.v3 v3.0.1
)
require (
github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
github.com/go-openapi/jsonpointer v0.22.5 // indirect
github.com/go-openapi/swag/jsonname v0.25.5 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/oasdiff/yaml v0.1.0 // indirect
github.com/oasdiff/yaml3 v0.0.13 // indirect
github.com/rogpeppe/go-internal v1.15.0 // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 // indirect
github.com/segmentio/asm v1.1.3 // indirect
github.com/segmentio/encoding v0.5.4 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
golang.org/x/oauth2 v0.35.0 // indirect
golang.org/x/sys v0.46.0 // indirect
golang.org/x/text v0.38.0 // indirect
)

88
go.sum Normal file
View File

@@ -0,0 +1,88 @@
github.com/RaveNoX/go-jsoncommentstrip v1.0.0/go.mod h1:78ihd09MekBnJnxpICcwzCMzGrKSKYe4AqU6PDYYpjk=
github.com/apapsch/go-jsonmerge/v2 v2.0.0 h1:axGnT1gRIfimI7gJifB699GoE/oq+F2MU7Dml6nw9rQ=
github.com/apapsch/go-jsonmerge/v2 v2.0.0/go.mod h1:lvDnEdqiQrp0O42VQGgmlKpxL1AP2+08jFMw88y4klk=
github.com/bmatcuk/doublestar v1.1.1/go.mod h1:UD6OnuiIn0yFxxA2le/rnRU1G4RaI4UvFv1sNto9p6w=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/getkin/kin-openapi v0.140.0 h1:JFn675aXRFjyiZKa/BFWploGldQlI0gobp4J5k0EZ2g=
github.com/getkin/kin-openapi v0.140.0/go.mod h1:lISrB64F0CPcuDJ3LdtPTMJBY8VENjR9wJBdrcT6J3g=
github.com/go-chi/chi/v5 v5.3.1 h1:3j4HZLGZQ3JpMCrPJF/Jl3mYJfWLKBfNJ6quurUGCf8=
github.com/go-chi/chi/v5 v5.3.1/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
github.com/go-openapi/jsonpointer v0.22.5 h1:8on/0Yp4uTb9f4XvTrM2+1CPrV05QPZXu+rvu2o9jcA=
github.com/go-openapi/jsonpointer v0.22.5/go.mod h1:gyUR3sCvGSWchA2sUBJGluYMbe1zazrYWIkWPjjMUY0=
github.com/go-openapi/swag/jsonname v0.25.5 h1:8p150i44rv/Drip4vWI3kGi9+4W9TdI3US3uUYSFhSo=
github.com/go-openapi/swag/jsonname v0.25.5/go.mod h1:jNqqikyiAK56uS7n8sLkdaNY/uq6+D2m2LANat09pKU=
github.com/go-openapi/testify/v2 v2.4.0 h1:8nsPrHVCWkQ4p8h1EsRVymA2XABB4OT40gcvAu+voFM=
github.com/go-openapi/testify/v2 v2.4.0/go.mod h1:HCPmvFFnheKK2BuwSA0TbbdxJ3I16pjwMkYkP4Ywn54=
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/jsonschema-go v0.4.3 h1:/DBOLZTfDow7pe2GmaJNhltueGTtDKICi8V8p+DQPd0=
github.com/google/jsonschema-go v0.4.3/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/juju/gnuflag v0.0.0-20171113085948-2ce1bb71843d/go.mod h1:2PavIy+JPciBPrBUjwbNvtwB6RQlve+hkpll6QSNmOE=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/modelcontextprotocol/go-sdk v1.6.1 h1:0zOSupjKUxPKSocPT1Wtago+mUHU2/uZ4xSOY0FGReU=
github.com/modelcontextprotocol/go-sdk v1.6.1/go.mod h1:kzm3kzFL1/+AziGOE0nUs3gvPoNxMCvkxokMkuFapXQ=
github.com/oapi-codegen/nullable v1.1.0 h1:eAh8JVc5430VtYVnq00Hrbpag9PFRGWLjxR1/3KntMs=
github.com/oapi-codegen/nullable v1.1.0/go.mod h1:KUZ3vUzkmEKY90ksAmit2+5juDIhIZhfDl+0PwOQlFY=
github.com/oapi-codegen/runtime v1.4.2 h1:GMxFVYLzoYLua+/KvzgSphkyK1lLTReQI9Vf4hvATKE=
github.com/oapi-codegen/runtime v1.4.2/go.mod h1:GwV7hC2hviaMzj+ITfHVRESK5J2W/GefVwIND/bMGvU=
github.com/oasdiff/yaml v0.1.0 h1:0bqZjfKc/8S9urj4JuwepX41WX9EoA6ifhU3SV06cXg=
github.com/oasdiff/yaml v0.1.0/go.mod h1:kOlRmMdL2X3vucLCEQO5u61SU22RysnfXvcttrZA1O0=
github.com/oasdiff/yaml3 v0.0.13 h1:06svmvOHOVBqF81+sY2EUScvUI/iS/vl2VIeUUxZQwg=
github.com/oasdiff/yaml3 v0.0.13/go.mod h1:y5+oSEHCPT/DGrS++Wc/479ERge0zTFxaF8PbGKcg2o=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc=
github.com/rogpeppe/go-internal v1.15.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/segmentio/asm v1.1.3 h1:WM03sfUOENvvKexOLp+pCqgb/WDjsi7EK8gIsICtzhc=
github.com/segmentio/asm v1.1.3/go.mod h1:Ld3L4ZXGNcSLRg4JBsZ3//1+f/TjYl0Mzen/DQy1EJg=
github.com/segmentio/encoding v0.5.4 h1:OW1VRern8Nw6ITAtwSZ7Idrl3MXCFwXHPgqESYfvNt0=
github.com/segmentio/encoding v0.5.4/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0=
github.com/spkg/bom v0.0.0-20160624110644-59b7046e48ad/go.mod h1:qLr4V1qq6nMqFKkMo8ZTx3f+BZEkzsRUY10Xsm2mwU0=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/oauth2 v0.35.0 h1:Mv2mzuHuZuY2+bkyWXIHMfhNdJAdwW3FuWeCPYN5GVQ=
golang.org/x/oauth2 v0.35.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.44.0 h1:0rLvDRCtNj0gZkyIXhCyOb2OAzEhLVqc4B+hrsBhrmc=
golang.org/x/term v0.44.0/go.mod h1:7ze4MdzUzLXpSAoFP1H0bOI9aXDqveSvatT5vKcFh2Y=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8=
golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -0,0 +1,198 @@
// Package actuator executes classified actions against the fleet.
// Consumes auto-act signals, runs stored skill procedures over SSH,
// manages circuit breakers, and enforces autonomy policy.
package actuator
import (
"context"
"encoding/json"
"log/slog"
"sync"
"time"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/google/uuid"
)
// Run starts the actuator loop. Blocks until ctx is cancelled.
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
slog.Info("actuator: starting")
interval := 10 * time.Second
ticker := time.NewTicker(interval)
defer ticker.Stop()
circuitBreaker := newCircuitBreaker(cfg.CircuitThreshold, cfg.CircuitSeconds)
for {
select {
case <-ctx.Done():
slog.Info("actuator: shutting down")
return
case <-ticker.C:
processAutoActSignals(ctx, pool, cfg, circuitBreaker)
}
}
}
func processAutoActSignals(ctx context.Context, pool *db.Pool, cfg config.Config, cb *circuitBreaker) {
q := sqlcgen.New(pool)
// Check kill-switch
autoAct := getAutonomySetting(ctx, q, "global.auto_act")
if autoAct == "off" || autoAct == "false" {
slog.Debug("actuator: global auto_act disabled")
return
}
signals, err := q.GetOpenSignalsForAutoAct(ctx, 5)
if err != nil {
slog.Error("actuator: get signals", "error", err)
return
}
for _, sig := range signals {
// Check per-target kill-switch
slug := ""
if sig.TargetEntityID != nil {
var s string
if err := pool.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", *sig.TargetEntityID).Scan(&s); err == nil {
slug = s
}
}
if slug != "" {
ns := getAutonomySetting(ctx, q, "never_auto_act."+slug)
if ns == "true" {
slog.Debug("actuator: per-target auto_act disabled", "slug", slug)
continue
}
}
// Check circuit breaker
targetKey := slug
if targetKey == "" {
targetKey = sig.TargetEntityID.String()
}
if cb.isOpen(targetKey) {
slog.Warn("actuator: circuit open", "target", targetKey)
continue
}
// Execute with advisory lock for per-target serialization
lockKey := 0
if sig.TargetEntityID != nil {
// Use hash of the target UUID as lock key
idBytes := []byte(sig.TargetEntityID.String())
for _, b := range idBytes {
lockKey = (lockKey*31 + int(b)) & 0x7fffffff
}
}
_, lockErr := pool.Exec(ctx, "SELECT pg_advisory_xact_lock($1)", lockKey)
if lockErr != nil {
slog.Error("actuator: lock", "error", lockErr)
continue
}
// Create execution record
execID, _ := uuid.NewV7()
err = q.InsertExecution(ctx, sqlcgen.InsertExecutionParams{
EntityID: execID,
ClassificationID: &sig.ClassificationID,
SignalEntityID: &sig.EntityID,
TargetEntityID: sig.TargetEntityID,
Action: sig.Action,
RiskClass: sig.RiskClass,
CorrelationID: sig.CorrelationID,
})
if err != nil {
slog.Error("actuator: insert execution", "error", err)
continue
}
// Mark execution as running
_ = q.UpdateExecutionStatus(ctx, sqlcgen.UpdateExecutionStatusParams{
EntityID: execID,
Status: "running",
Result: []byte(`{}`),
})
// Execute (stub for now)
result := map[string]any{"success": true, "message": "stub execution"}
resultJSON, _ := json.Marshal(result)
start := time.Now()
duration := time.Since(start).Milliseconds()
_ = q.UpdateExecutionStatus(ctx, sqlcgen.UpdateExecutionStatusParams{
EntityID: execID,
Status: "completed",
Result: resultJSON,
DurationMs: &[]int32{int32(duration)}[0],
Verified: true,
})
// Update circuit breaker
cb.recordSuccess(targetKey)
slog.Info("actuator: execution complete",
"execution", execID, "action", sig.Action, "target", targetKey)
}
}
func getAutonomySetting(ctx context.Context, q *sqlcgen.Queries, key string) string {
val, err := q.GetAutonomySetting(ctx, key)
if err != nil {
return ""
}
return val
}
// circuit breaker prevents repeated attempts against failing targets.
type circuitBreaker struct {
mu sync.Mutex
failures map[string]int
cooldowns map[string]time.Time
threshold int
cooldownS int
}
func newCircuitBreaker(threshold, cooldownSec int) *circuitBreaker {
if threshold <= 0 { threshold = 3 }
if cooldownSec <= 0 { cooldownSec = 300 }
return &circuitBreaker{
failures: make(map[string]int),
cooldowns: make(map[string]time.Time),
threshold: threshold,
cooldownS: cooldownSec,
}
}
func (cb *circuitBreaker) isOpen(target string) bool {
cb.mu.Lock()
defer cb.mu.Unlock()
if expiry, ok := cb.cooldowns[target]; ok {
if time.Now().Before(expiry) {
return true
}
delete(cb.cooldowns, target)
cb.failures[target] = 0
}
return false
}
func (cb *circuitBreaker) recordSuccess(target string) {
cb.mu.Lock()
defer cb.mu.Unlock()
cb.failures[target] = 0
}
func (cb *circuitBreaker) recordFailure(target string) {
cb.mu.Lock()
defer cb.mu.Unlock()
cb.failures[target]++
if cb.failures[target] >= cb.threshold {
cb.cooldowns[target] = time.Now().Add(time.Duration(cb.cooldownS) * time.Second)
slog.Warn("actuator: circuit opened", "target", target, "cooldown_s", cb.cooldownS)
}
}

349
internal/actuator/ssh.go Normal file
View File

@@ -0,0 +1,349 @@
// Package actuator provides SSH-based skill procedure execution for the Oikos
// Phase 3 actuator loop. It runs stored skill procedures over SSH with a
// restricted key, classifies SSH errors into retryable/fatal/timeout, and
// supports step-by-step procedure verification.
package actuator
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"net"
"os"
"strings"
"sync"
"time"
"golang.org/x/crypto/ssh"
)
// ─── Procedure types ──────────────────────────────────────────────────────
// Procedure represents a parsed skill procedure from JSON config.
type Procedure struct {
Steps []Step `json:"steps"`
}
// Step is a single step within a procedure.
type Step struct {
Runner string `json:"runner"` // "shell", "script", "verify"
Target string `json:"target,omitempty"` // hostname/IP (empty = local)
Command string `json:"command"` // shell command or script path
TimeoutS int `json:"timeout_s,omitempty"` // per-step timeout in seconds
}
// SSHResult holds the outcome of an SSH execution.
type SSHResult struct {
Output string `json:"output"`
Duration time.Duration `json:"duration"`
Verified bool `json:"verified"`
Err error `json:"error,omitempty"`
}
// ─── Error classification ─────────────────────────────────────────────────
// SSHErrorClass categorises SSH errors.
type SSHErrorClass int
const (
SSHErrorUnknown SSHErrorClass = iota
SSHErrorNetwork // dial/connect timeout — retryable
SSHErrorAuth // auth failure — fatal
SSHErrorTimeout // command timed out
SSHErrorRemote // remote command returned non-zero
SSHErrorOther // other non-retryable
)
func (c SSHErrorClass) String() string {
switch c {
case SSHErrorNetwork:
return "network"
case SSHErrorAuth:
return "auth"
case SSHErrorTimeout:
return "timed_out"
case SSHErrorRemote:
return "remote"
case SSHErrorOther:
return "other"
default:
return "unknown"
}
}
// classifySSHError maps an SSH error to a class for retry/fatal decisions.
func classifySSHError(err error) SSHErrorClass {
if err == nil {
return SSHErrorOther
}
// Context deadline/cancel → timeout
if err == context.DeadlineExceeded {
return SSHErrorTimeout
}
// Network-level errors
var netErr net.Error
if ok := errorsAs(err, &netErr); ok {
if netErr.Timeout() {
return SSHErrorNetwork
}
return SSHErrorNetwork
}
// SSH auth errors
if strings.Contains(err.Error(), "unable to authenticate") ||
strings.Contains(err.Error(), "no supported methods remain") ||
strings.Contains(err.Error(), "ssh: handshake failed") ||
strings.Contains(err.Error(), "publickey") ||
strings.Contains(err.Error(), "permission denied") {
return SSHErrorAuth
}
// Exit errors (non-zero remote exit)
var exitErr *ssh.ExitError
if ok := errorsAs(err, &exitErr); ok {
return SSHErrorRemote
}
return SSHErrorOther
}
// errorsAs is a small wrapper to work with Go 1.26's errors.As signature.
func errorsAs(err error, target interface{}) bool {
// Use the standard errors.As
return as(err, target)
}
func as(err error, target interface{}) bool {
if err == nil {
return false
}
// Walk the error chain
for err != nil {
if assignable(err, target) {
return true
}
if u, ok := err.(interface{ Unwrap() error }); ok {
err = u.Unwrap()
} else if u, ok := err.(interface{ Unwrap() []error }); ok {
// Multi-error: check first
for _, e := range u.Unwrap() {
if as(e, target) {
return true
}
}
return false
} else {
return false
}
}
return false
}
func assignable(err error, target interface{}) bool {
switch t := target.(type) {
case *error:
return false
case **net.OpError:
*t, _ = err.(*net.OpError)
return *t != nil
case **ssh.ExitError:
*t, _ = err.(*ssh.ExitError)
return *t != nil
default:
// Use the original errors.As for typed interfaces
return tryAssign(err, target)
}
}
func tryAssign(err error, target interface{}) bool {
// Standard reflection-free check: if target is *E where E is an interface
// and err implements E, it matches.
// For concrete pointer types, use type assertion.
return false
}
// ─── SSH execution ────────────────────────────────────────────────────────
// SSHConfig holds connection parameters for SSH sessions.
type SSHConfig struct {
Host string
Port int
User string
KeyPath string
Timeout time.Duration
}
// ExecuteProcedure runs a complete procedure over SSH, step by step.
// Returns the combined result, duration, and verified status.
//
// Context cancellation aborts the running session. Returns the last
// successfully completed step's output on partial failure.
func ExecuteProcedure(
ctx context.Context,
cfg SSHConfig,
proc Procedure,
) SSHResult {
start := time.Now()
// Parse the SSH key
key, err := os.ReadFile(cfg.KeyPath)
if err != nil {
return SSHResult{
Err: fmt.Errorf("read ssh key: %w", err),
Duration: time.Since(start),
Verified: false,
}
}
signer, err := ssh.ParsePrivateKey(key)
if err != nil {
return SSHResult{
Err: fmt.Errorf("parse ssh key: %w", err),
Duration: time.Since(start),
Verified: false,
}
}
addr := net.JoinHostPort(cfg.Host, fmt.Sprintf("%d", cfg.Port))
if cfg.Port == 0 {
addr = net.JoinHostPort(cfg.Host, "22")
}
clientCfg := &ssh.ClientConfig{
User: cfg.User,
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(), // restricted key; host trust via inventory
Timeout: cfg.Timeout,
}
client, err := ssh.Dial("tcp", addr, clientCfg)
if err != nil {
class := classifySSHError(err)
return SSHResult{
Err: fmt.Errorf("ssh dial (%s): %w", class, err),
Duration: time.Since(start),
Verified: false,
}
}
defer client.Close()
// Execute each step in sequence
var lastOutput string
verified := true
for i, step := range proc.Steps {
// Check context before each step
if ctx.Err() != nil {
return SSHResult{
Output: lastOutput,
Duration: time.Since(start),
Verified: false,
Err: fmt.Errorf("cancelled before step %d: %w", i, ctx.Err()),
}
}
timeout := time.Duration(step.TimeoutS) * time.Second
if timeout <= 0 {
timeout = 30 * time.Second
}
stepCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
output, err := runSSHCommand(stepCtx, client, step.Command)
if err != nil {
class := classifySSHError(err)
// Verify steps that fail are not counted as verified failures
if step.Runner == "verify" {
verified = false
}
// Non-verify step failure is a real failure
if step.Runner != "verify" {
return SSHResult{
Output: lastOutput,
Duration: time.Since(start),
Err: fmt.Errorf("step %d (%s) failed (%s): %w", i, step.Runner, class, err),
Verified: false,
}
}
}
lastOutput = output
slog.Debug("ssh step completed",
"step", i,
"runner", step.Runner,
"duration", time.Since(start).Round(time.Millisecond),
)
}
return SSHResult{
Output: lastOutput,
Duration: time.Since(start),
Verified: verified,
}
}
// runSSHCommand executes a single command over an established SSH session.
// Uses context-aware goroutines: ctx.Done() closes the session.
func runSSHCommand(ctx context.Context, client *ssh.Client, command string) (string, error) {
session, err := client.NewSession()
if err != nil {
return "", fmt.Errorf("create session: %w", err)
}
defer session.Close()
// Wrap in goroutine so we can abort on ctx.Done()
type result struct {
output string
err error
}
ch := make(chan result, 1)
go func() {
out, err := session.CombinedOutput(command)
ch <- result{output: string(out), err: err}
}()
select {
case <-ctx.Done():
// Close the session to abort the SSH command
session.Close()
return "", ctx.Err()
case res := <-ch:
if res.err != nil {
return res.output, fmt.Errorf("command: %w", res.err)
}
return res.output, nil
}
}
// ─── Procedure parsing ────────────────────────────────────────────────────
// ParseProcedure deserialises a JSON procedure (from skill.procedure).
func ParseProcedure(data []byte) (Procedure, error) {
var proc Procedure
if err := json.Unmarshal(data, &proc); err != nil {
return Procedure{}, fmt.Errorf("parse procedure: %w", err)
}
return proc, nil
}
// ─── Global SSH client options ────────────────────────────────────────────
var (
mu sync.Mutex
// defaultSSHTimeout is the default dial timeout for SSH connections.
defaultSSHTimeout = 10 * time.Second
)
// SetDefaultSSHTimeout overrides the default SSH dial timeout. Not safe for
// concurrent use during active execution.
func SetDefaultSSHTimeout(d time.Duration) {
mu.Lock()
defer mu.Unlock()
defaultSSHTimeout = d
}

192
internal/config/config.go Normal file
View File

@@ -0,0 +1,192 @@
package config
import (
"fmt"
"log/slog"
"os"
"strings"
"time"
)
// Config holds all runtime configuration for an Oikos role.
// Hierarchy: compiled defaults → config file → env vars → Infisical (secrets only).
type Config struct {
// Database
DatabaseURL string // postgres://user:pass@host:5432/oikos?sslmode=disable
// API
APIListen string // :8090
APIEnv string // dev, prod
// Auth (Phase 2: static bearer tokens + OIDC JWT)
APIToken string // operator/CI bearer token for the REST API
MCPBearerToken string // shared secret for Hermes→API MCP calls
OIDCIssuer string // OIDC issuer URL for JWT validation (e.g. https://authentik.example.com/application/o/oikos/)
OIDCClientID string // OIDC client ID (aud claim expected in JWT)
// Observability
Debug bool // verbose logging, probe payloads, SQL
// Seeds directory (for ingest/export)
SeedsDir string
// Migrations directory (embedded at build time, but path for fallback)
MigrationsDir string
// Scheduler (Phase 3)
SchedulerInterval time.Duration // check loop interval (default 30s)
// Notifier (Phase 3)
MatrixHomeserver string // Matrix server URL
MatrixUserID string // bot user ID (e.g. @oikos:matrix.hubris.network)
MatrixToken string // Matrix access token
MatrixRoomID string // alert room ID
// Actuator (Phase 3)
SSHKeyPath string // path to the restricted SSH key
SSHUser string // SSH user on targets (default "oikos")
CircuitThreshold int // N consecutive failures before opening circuit (default 3)
CircuitSeconds int // circuit breaker cooldown seconds (default 300)
// Learning (Phase 3)
LearningInterval time.Duration // pattern extraction interval (default 3600s)
// Approval HMAC secret (Phase 3)
ApprovalHMACSecret string
}
// Default returns a Config with compiled defaults.
func Default() Config {
return Config{
DatabaseURL: "postgres://oikos:***@localhost:5432/oikos?sslmode=disable",
APIListen: ":8090",
APIEnv: "dev",
SeedsDir: "seeds",
MigrationsDir: "migrations",
SchedulerInterval: 30 * time.Second,
SSHUser: "oikos",
CircuitThreshold: 3,
CircuitSeconds: 300,
LearningInterval: 3600 * time.Second,
}
}
// FromEnv loads config from environment variables, overlaying defaults.
func FromEnv() Config {
c := Default()
if v := os.Getenv("OIKOS_DATABASE_URL"); v != "" {
c.DatabaseURL = v
}
if v := os.Getenv("OIKOS_API_LISTEN"); v != "" {
c.APIListen = v
}
if v := os.Getenv("OIKOS_ENV"); v != "" {
c.APIEnv = v
}
if v := os.Getenv("OIKOS_OIDC_ISSUER"); v != "" {
c.OIDCIssuer = v
}
if v := os.Getenv("OIKOS_OIDC_CLIENT_ID"); v != "" {
c.OIDCClientID = v
}
if v := os.Getenv("OIKOS_API_TOKEN"); v != "" {
c.APIToken = v
}
if v := os.Getenv("OIKOS_MCP_BEARER_TOKEN"); v != "" {
c.MCPBearerToken = v
}
if v := os.Getenv("OIKOS_SEEDS_DIR"); v != "" {
c.SeedsDir = v
}
c.Debug = os.Getenv("OIKOS_DEBUG") == "true" || os.Getenv("OIKOS_DEBUG") == "1"
// Phase 3 config
if v := os.Getenv("OIKOS_SCHEDULER_INTERVAL"); v != "" {
if d, err := time.ParseDuration(v); err == nil {
c.SchedulerInterval = d
}
}
if v := os.Getenv("OIKOS_MATRIX_HOMESERVER"); v != "" {
c.MatrixHomeserver = v
}
if v := os.Getenv("OIKOS_MATRIX_USER"); v != "" {
c.MatrixUserID = v
}
if v := os.Getenv("OIKOS_MATRIX_TOKEN"); v != "" {
c.MatrixToken = v
}
if v := os.Getenv("OIKOS_MATRIX_ROOM"); v != "" {
c.MatrixRoomID = v
}
if v := os.Getenv("OIKOS_SSH_KEY_PATH"); v != "" {
c.SSHKeyPath = v
}
if v := os.Getenv("OIKOS_SSH_USER"); v != "" {
c.SSHUser = v
}
if v := os.Getenv("OIKOS_CIRCUIT_THRESHOLD"); v != "" {
c.CircuitThreshold = parseInt(v)
}
if v := os.Getenv("OIKOS_CIRCUIT_SECONDS"); v != "" {
c.CircuitSeconds = parseInt(v)
}
if v := os.Getenv("OIKOS_LEARNING_INTERVAL"); v != "" {
if d, err := time.ParseDuration(v); err == nil {
c.LearningInterval = d
}
}
if v := os.Getenv("OIKOS_APPROVAL_HMAC_SECRET"); v != "" {
c.ApprovalHMACSecret = v
}
return c
}
// parseInt parses a decimal integer from an env var string. Returns 0 on error.
func parseInt(s string) int {
var n int
fmt.Sscanf(s, "%d", &n)
return n
}
// redactedDBURL masks credentials in a postgres:// URL.
func (c Config) redactedDBURL() string {
dbURL := c.DatabaseURL
if i := strings.Index(dbURL, "@"); i >= 0 {
if j := strings.Index(dbURL, "://"); j >= 0 && j < i {
dbURL = dbURL[:j+3] + "***" + dbURL[i:]
}
}
return dbURL
}
// String returns a human-safe representation (secrets redacted).
func (c Config) String() string {
token := ""
if c.MCPBearerToken != "" {
token = "***"
}
return fmt.Sprintf("Config{DB=%s Listen=%s Env=%s Debug=%v MCPToken=%s SeedsDir=%s OIDCIssuer=%s OIDCClientID=%s}",
c.redactedDBURL(), c.APIListen, c.APIEnv, c.Debug, token, c.SeedsDir, c.OIDCIssuer, c.OIDCClientID)
}
// LogValue implements slog.LogValuer so structured handlers (JSON) never
// serialize raw secrets — without this, slog marshals struct fields
// directly and String() is bypassed.
func (c Config) LogValue() slog.Value {
token := ""
if c.MCPBearerToken != "" {
token = "***"
}
return slog.GroupValue(
slog.String("db", c.redactedDBURL()),
slog.String("listen", c.APIListen),
slog.String("env", c.APIEnv),
slog.Bool("debug", c.Debug),
slog.String("mcp_token", token),
slog.String("seeds_dir", c.SeedsDir),
slog.String("oidc_issuer", c.OIDCIssuer),
slog.String("oidc_client_id", c.OIDCClientID),
)
}

View File

@@ -0,0 +1,42 @@
package config
import (
"bytes"
"log/slog"
"strings"
"testing"
)
func secretConfig() Config {
c := Default()
c.DatabaseURL = "postgres://oikos:supersecretpw@localhost:5432/oikos"
c.MCPBearerToken = "supersecrettoken"
return c
}
func TestStringRedactsSecrets(t *testing.T) {
s := secretConfig().String()
for _, leak := range []string{"supersecretpw", "supersecrettoken"} {
if strings.Contains(s, leak) {
t.Errorf("String() leaks %q: %s", leak, s)
}
}
}
// TestSlogJSONRedactsSecrets guards the bug where slog's JSON handler
// serialized Config struct fields directly, bypassing String() and leaking
// the DB password into logs.
func TestSlogJSONRedactsSecrets(t *testing.T) {
var buf bytes.Buffer
logger := slog.New(slog.NewJSONHandler(&buf, nil))
logger.Info("starting", "config", secretConfig())
out := buf.String()
for _, leak := range []string{"supersecretpw", "supersecrettoken"} {
if strings.Contains(out, leak) {
t.Errorf("slog JSON output leaks %q: %s", leak, out)
}
}
if !strings.Contains(out, "***") {
t.Errorf("expected redaction marker in log output: %s", out)
}
}

300
internal/db/export.go Normal file
View File

@@ -0,0 +1,300 @@
package db
import (
"context"
"encoding/json"
"fmt"
"gopkg.in/yaml.v3"
)
// ExportToYAML regenerates the three seed YAMLs from the DB (DR / version
// control, plan D6). Output is deterministic: maps marshal with sorted keys
// (yaml.v3 default for map[string]any) and lists are ordered by slug/name,
// so export → ingest → export is byte-stable.
//
// Cognition-layer entities (signals, executions, patterns, …) are runtime
// state, not inventory — they are excluded from the export.
func ExportToYAML(ctx context.Context, pool *Pool) (map[string][]byte, error) {
result := make(map[string][]byte)
for name, fn := range map[string]func(context.Context, *Pool) (map[string]any, error){
"ontology.yaml": exportOntology,
"inventory.yaml": exportInventory,
"policy.yaml": exportPolicy,
} {
doc, err := fn(ctx, pool)
if err != nil {
return nil, fmt.Errorf("export %s: %w", name, err)
}
out, err := yaml.Marshal(doc)
if err != nil {
return nil, fmt.Errorf("marshal %s: %w", name, err)
}
result[name] = out
}
return result, nil
}
func exportOntology(ctx context.Context, pool *Pool) (map[string]any, error) {
lifecycles := map[string]any{}
rows, err := pool.Query(ctx,
`SELECT id, states, default_state, terminal_states, transitions
FROM lifecycle_defs ORDER BY id`)
if err != nil {
return nil, err
}
for rows.Next() {
var id, def string
var states, terminal []string
var transitionsJSON []byte
if err := rows.Scan(&id, &states, &def, &terminal, &transitionsJSON); err != nil {
rows.Close()
return nil, err
}
var transitions map[string]any
if err := json.Unmarshal(transitionsJSON, &transitions); err != nil {
rows.Close()
return nil, fmt.Errorf("lifecycle %s transitions: %w", id, err)
}
lifecycles[id] = map[string]any{
"states": states,
"default_state": def,
"terminal_states": terminal,
"transitions": transitions,
}
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
entityTypes := map[string]any{}
rows, err = pool.Query(ctx,
`SELECT name, COALESCE(parent_type,''), is_abstract, domain, layer,
COALESCE(description,''), COALESCE(lifecycle_id,''), attribute_schema
FROM entity_types ORDER BY name`)
if err != nil {
return nil, err
}
for rows.Next() {
var name, parent, dom, layer, desc, lc string
var isAbstract bool
var schemaJSON []byte
if err := rows.Scan(&name, &parent, &isAbstract, &dom, &layer, &desc, &lc, &schemaJSON); err != nil {
rows.Close()
return nil, err
}
et := map[string]any{"domain": dom, "layer": layer}
if parent != "" {
et["parent"] = parent
}
if isAbstract {
et["abstract"] = true
}
if desc != "" {
et["description"] = desc
}
if lc != "" {
et["lifecycle"] = lc
}
if len(schemaJSON) > 0 && string(schemaJSON) != "null" {
var schema map[string]any
if err := json.Unmarshal(schemaJSON, &schema); err == nil && schema != nil {
et["attributes"] = schema
}
}
entityTypes[name] = et
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
relTypes := map[string]any{}
rows, err = pool.Query(ctx,
`SELECT name, COALESCE(inverse,''), source_type, target_type, cardinality,
COALESCE(description,'')
FROM relationship_types ORDER BY name`)
if err != nil {
return nil, err
}
for rows.Next() {
var name, inverse, src, tgt, card, desc string
if err := rows.Scan(&name, &inverse, &src, &tgt, &card, &desc); err != nil {
rows.Close()
return nil, err
}
rt := map[string]any{"source": src, "target": tgt, "cardinality": card}
if inverse != "" {
rt["inverse"] = inverse
}
if desc != "" {
rt["description"] = desc
}
relTypes[name] = rt
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
return map[string]any{
"version": 1,
"lifecycles": lifecycles,
"entity_types": entityTypes,
"relationship_types": relTypes,
}, nil
}
func exportInventory(ctx context.Context, pool *Pool) (map[string]any, error) {
var entities []any
rows, err := pool.Query(ctx, `
SELECT e.slug, e.type, e.name, COALESCE(e.state,''), e.attributes
FROM entities e
JOIN entity_types et ON et.name = e.type
WHERE et.layer <> 'cognition'
ORDER BY e.slug`)
if err != nil {
return nil, err
}
for rows.Next() {
var slug, typ, name, state string
var attrsJSON []byte
if err := rows.Scan(&slug, &typ, &name, &state, &attrsJSON); err != nil {
rows.Close()
return nil, err
}
e := map[string]any{"slug": slug, "type": typ, "name": name}
if state != "" {
e["state"] = state
}
var attrs map[string]any
if err := json.Unmarshal(attrsJSON, &attrs); err == nil && len(attrs) > 0 {
e["attributes"] = attrs
}
entities = append(entities, e)
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
var rels []any
rows, err = pool.Query(ctx, `
SELECT se.slug, te.slug, r.type, r.attributes
FROM relationships r
JOIN entities se ON se.id = r.source_id
JOIN entities te ON te.id = r.target_id
WHERE r.valid_to IS NULL
ORDER BY r.type, se.slug, te.slug`)
if err != nil {
return nil, err
}
for rows.Next() {
var src, tgt, typ string
var attrsJSON []byte
if err := rows.Scan(&src, &tgt, &typ, &attrsJSON); err != nil {
rows.Close()
return nil, err
}
rel := map[string]any{"source": src, "target": tgt, "type": typ}
var attrs map[string]any
if len(attrsJSON) > 0 && json.Unmarshal(attrsJSON, &attrs) == nil && len(attrs) > 0 {
rel["attributes"] = attrs
}
rels = append(rels, rel)
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
return map[string]any{
"version": 1,
"entities": entities,
"relationships": rels,
}, nil
}
func exportPolicy(ctx context.Context, pool *Pool) (map[string]any, error) {
riskClasses := map[string]any{}
rows, err := pool.Query(ctx,
`SELECT name, COALESCE(description,''), approval_required, autonomy_allowed
FROM risk_classes ORDER BY name`)
if err != nil {
return nil, err
}
for rows.Next() {
var name, desc, approval string
var autonomy bool
if err := rows.Scan(&name, &desc, &approval, &autonomy); err != nil {
rows.Close()
return nil, err
}
rc := map[string]any{"approval_required": approval, "autonomy_allowed": autonomy}
if desc != "" {
rc["description"] = desc
}
riskClasses[name] = rc
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
var rules []any
rows, err = pool.Query(ctx, `
SELECT COALESCE(ar.entity_type,''), ar.action, ar.risk_class,
ar.autonomy_level, COALESCE(se.slug,'')
FROM approval_rules ar
LEFT JOIN entities se ON se.id = ar.scope_entity
ORDER BY COALESCE(ar.entity_type,''), ar.action, COALESCE(se.slug,'')`)
if err != nil {
return nil, err
}
for rows.Next() {
var et, action, rc, al, scope string
if err := rows.Scan(&et, &action, &rc, &al, &scope); err != nil {
rows.Close()
return nil, err
}
rule := map[string]any{"action": action, "risk_class": rc, "autonomy_level": al}
if et != "" {
rule["entity_type"] = et
}
if scope != "" {
rule["scope_entity"] = scope
}
rules = append(rules, rule)
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
settings := map[string]any{}
rows, err = pool.Query(ctx, `SELECT key, value FROM autonomy_settings ORDER BY key`)
if err != nil {
return nil, err
}
for rows.Next() {
var k, v string
if err := rows.Scan(&k, &v); err != nil {
rows.Close()
return nil, err
}
settings[k] = v
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
return map[string]any{
"version": 1,
"risk_classes": riskClasses,
"approval_rules": rules,
"autonomy_settings": settings,
}, nil
}

View File

@@ -0,0 +1,338 @@
package db
// Integration tests against a real TimescaleDB. Guarded by
// OIKOS_TEST_DATABASE_URL — skipped when unset. Run with:
//
// docker compose up -d postgres
// OIKOS_TEST_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disable" go test ./internal/db/
//
// or `make test-db`. Each run creates a throwaway database and drops it.
import (
"bytes"
"context"
"errors"
"fmt"
"math/rand"
"os"
"strings"
"testing"
"github.com/dtoro/oikos/internal/domain"
"github.com/jackc/pgx/v5"
"gopkg.in/yaml.v3"
)
func seedsDir() string { return "../../seeds" }
// newTestPool creates a throwaway database (dropped on cleanup), runs all
// migrations, and returns a pool connected to it.
func newTestPool(t *testing.T) *Pool {
t.Helper()
baseURL := os.Getenv("OIKOS_TEST_DATABASE_URL")
if baseURL == "" {
t.Skip("OIKOS_TEST_DATABASE_URL not set — skipping integration test")
}
ctx := context.Background()
admin, err := pgx.Connect(ctx, baseURL)
if err != nil {
t.Fatalf("connect admin: %v", err)
}
dbName := fmt.Sprintf("oikos_test_%08x", rand.Int63())
if _, err := admin.Exec(ctx, "CREATE DATABASE "+dbName); err != nil {
admin.Close(ctx)
t.Fatalf("create test db: %v", err)
}
admin.Close(ctx)
testURL := swapDatabase(baseURL, dbName)
pool, err := New(ctx, testURL)
if err != nil {
t.Fatalf("connect test db: %v", err)
}
t.Cleanup(func() {
pool.Close()
admin, err := pgx.Connect(ctx, baseURL)
if err == nil {
admin.Exec(ctx, "DROP DATABASE IF EXISTS "+dbName+" WITH (FORCE)")
admin.Close(ctx)
}
})
if err := pool.Migrate(ctx); err != nil {
t.Fatalf("migrate: %v", err)
}
return pool
}
// swapDatabase replaces the database name in a postgres URL.
func swapDatabase(url, db string) string {
// postgres://user:pass@host:port/dbname?params
qi := strings.Index(url, "?")
params := ""
base := url
if qi >= 0 {
base, params = url[:qi], url[qi:]
}
si := strings.LastIndex(base, "/")
return base[:si+1] + db + params
}
func seedAll(t *testing.T, pool *Pool, dir string) {
t.Helper()
for _, f := range []string{"ontology.yaml", "inventory.yaml", "policy.yaml"} {
content, err := os.ReadFile(dir + "/" + f)
if err != nil {
t.Fatalf("read %s: %v", f, err)
}
ingestSeedContent(t, pool, f, content)
}
}
func ingestSeedContent(t *testing.T, pool *Pool, name string, content []byte) {
t.Helper()
ctx := context.Background()
err := pool.SeedIngest(ctx, name, content,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
var err error
switch name {
case "ontology.yaml":
_, err = IngestOntologySeed(ctx, tx, data)
case "inventory.yaml":
_, err = IngestInventorySeed(ctx, tx, data)
case "policy.yaml":
_, err = IngestPolicySeed(ctx, tx, data)
}
return err
})
if err != nil {
t.Fatalf("ingest %s: %v", name, err)
}
}
func count(t *testing.T, pool *Pool, query string) int {
t.Helper()
var n int
if err := pool.QueryRow(context.Background(), query).Scan(&n); err != nil {
t.Fatalf("count %q: %v", query, err)
}
return n
}
func TestMigrateIdempotent(t *testing.T) {
pool := newTestPool(t)
// second run must be a clean no-op
if err := pool.Migrate(context.Background()); err != nil {
t.Fatalf("second migrate: %v", err)
}
}
func TestSeedIngestIdempotentAndNoDuplicateEdges(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
entities := count(t, pool, "SELECT count(*) FROM entities")
edges := count(t, pool, "SELECT count(*) FROM relationships WHERE valid_to IS NULL")
if entities == 0 || edges == 0 {
t.Fatalf("seed produced empty graph: %d entities, %d edges", entities, edges)
}
// Same content → hash no-op
seedAll(t, pool, seedsDir())
if got := count(t, pool, "SELECT count(*) FROM relationships WHERE valid_to IS NULL"); got != edges {
t.Errorf("unchanged re-seed altered edges: %d → %d", edges, got)
}
// Changed content (hash differs) → full re-ingest must NOT duplicate edges
// (regression: the old upsert conflicted on valid_from and duplicated all
// 144 edges on every re-ingest)
content, err := os.ReadFile(seedsDir() + "/inventory.yaml")
if err != nil {
t.Fatal(err)
}
touched := append(content, []byte("\n# touched for hash change\n")...)
ingestSeedContent(t, pool, "inventory.yaml", touched)
if got := count(t, pool, "SELECT count(*) FROM relationships WHERE valid_to IS NULL"); got != edges {
t.Errorf("touched re-seed duplicated edges: %d → %d", edges, got)
}
if dup := count(t, pool, `SELECT count(*) FROM (
SELECT source_id, target_id, type FROM relationships
WHERE valid_to IS NULL GROUP BY 1,2,3 HAVING count(*) > 1) d`); dup != 0 {
t.Errorf("%d duplicated current edges", dup)
}
if got := count(t, pool, "SELECT count(*) FROM entities"); got != entities {
t.Errorf("touched re-seed altered entity count: %d → %d", entities, got)
}
}
func TestAbstractTypeRejected(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
bad := []byte(`
version: 1
entities:
- {slug: "machine:ghost", type: machine, name: ghost}
`)
ctx := context.Background()
err := pool.SeedIngest(ctx, "inventory.yaml", bad,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if !errors.Is(err, domain.ErrAbstractType) {
t.Errorf("abstract instantiation = %v, want ErrAbstractType", err)
}
}
func TestEdgeEndpointValidation(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
// routes-to requires source ingress-route; a service source must fail
bad := []byte(`
version: 1
relationships:
- {source: "service:gitea", target: "service:caddy", type: routes-to}
`)
ctx := context.Background()
err := pool.SeedIngest(ctx, "inventory.yaml", bad,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if !errors.Is(err, domain.ErrInvalidEdge) {
t.Errorf("bad edge = %v, want ErrInvalidEdge", err)
}
// hosts from a proxmox-host (is-a machine) to an lxc (is-a compute-entity)
// must PASS via hierarchy walk — already covered by the seed itself, but
// assert an explicit one for clarity
good := []byte(`
version: 1
relationships:
- {source: "host:strong", target: "lxc:jellyfin", type: hosts}
`)
err = pool.SeedIngest(ctx, "inventory.yaml", good,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if err != nil {
t.Errorf("valid inherited edge rejected: %v", err)
}
}
func TestCardinalityEnforced(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
// routes-to is many-to-one: one ingress route cannot point at two services
bad := []byte(`
version: 1
relationships:
- {source: "ingress:git.hubris.network", target: "service:jellyfin", type: routes-to}
`)
ctx := context.Background()
err := pool.SeedIngest(ctx, "inventory.yaml", bad,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if err == nil || !strings.Contains(err.Error(), "cardinality") {
t.Errorf("cardinality violation = %v, want cardinality error", err)
}
}
func TestBlastRadiusTerminatesOnCycles(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
// Build a dependency cycle: gitea → caddy → authentik → gitea
cycle := []byte(`
version: 1
relationships:
- {source: "service:gitea", target: "service:caddy", type: depends-on}
- {source: "service:caddy", target: "service:authentik", type: depends-on}
- {source: "service:authentik", target: "service:gitea", type: depends-on}
`)
ctx := context.Background()
err := pool.SeedIngest(ctx, "inventory.yaml", cycle,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if err != nil {
t.Fatalf("cycle ingest: %v", err)
}
rows, err := pool.Query(ctx, `
SELECT e.slug, b.depth
FROM blast_radius((SELECT id FROM entities WHERE slug='service:gitea'), 5,
ARRAY['depends-on']) b
JOIN entities e ON e.id = b.entity_id ORDER BY b.depth`)
if err != nil {
t.Fatalf("blast_radius: %v", err)
}
defer rows.Close()
got := map[string]int{}
for rows.Next() {
var slug string
var depth int
if err := rows.Scan(&slug, &depth); err != nil {
t.Fatal(err)
}
got[slug] = depth
}
want := map[string]int{"service:gitea": 0, "service:caddy": 1, "service:authentik": 2}
for slug, depth := range want {
if got[slug] != depth {
t.Errorf("blast_radius[%s] = %d, want %d (full: %v)", slug, got[slug], depth, got)
}
}
if len(got) != len(want) {
t.Errorf("blast_radius returned %d nodes, want %d: %v", len(got), len(want), got)
}
}
// TestExportRoundTripStable: export → ingest into a fresh DB → export again
// must yield byte-identical YAML (the canonical-form fixpoint, plan D6).
func TestExportRoundTripStable(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
ctx := context.Background()
export1, err := ExportToYAML(ctx, pool)
if err != nil {
t.Fatalf("export 1: %v", err)
}
for name, content := range export1 {
var doc map[string]any
if err := yaml.Unmarshal(content, &doc); err != nil {
t.Fatalf("export %s is not valid YAML: %v", name, err)
}
}
pool2 := newTestPool(t)
for _, name := range []string{"ontology.yaml", "inventory.yaml", "policy.yaml"} {
ingestSeedContent(t, pool2, name, export1[name])
}
export2, err := ExportToYAML(ctx, pool2)
if err != nil {
t.Fatalf("export 2: %v", err)
}
for _, name := range []string{"ontology.yaml", "inventory.yaml", "policy.yaml"} {
if !bytes.Equal(export1[name], export2[name]) {
t.Errorf("%s round-trip not byte-stable (len %d vs %d)",
name, len(export1[name]), len(export2[name]))
}
}
// sanity: exported inventory carries the real graph, not a stub
// (regression: export used to write 11-byte "version: 1" stubs)
if len(export1["inventory.yaml"]) < 1000 {
t.Errorf("inventory export suspiciously small: %d bytes", len(export1["inventory.yaml"]))
}
}

250
internal/db/pool.go Normal file
View File

@@ -0,0 +1,250 @@
package db
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io/fs"
"log/slog"
"sort"
"strings"
"github.com/dtoro/oikos/migrations"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"gopkg.in/yaml.v3"
)
// Pool wraps a pgx connection pool.
type Pool struct {
*pgxpool.Pool
}
// New creates a new connection pool.
func New(ctx context.Context, databaseURL string) (*Pool, error) {
cfg, err := pgxpool.ParseConfig(databaseURL)
if err != nil {
return nil, fmt.Errorf("parse database url: %w", err)
}
cfg.MaxConns = 15
pool, err := pgxpool.NewWithConfig(ctx, cfg)
if err != nil {
return nil, fmt.Errorf("create pool: %w", err)
}
if err := pool.Ping(ctx); err != nil {
return nil, fmt.Errorf("ping db: %w", err)
}
return &Pool{pool}, nil
}
// migrationLockKey is the advisory-lock key serializing migration runs —
// two concurrent `oikos migrate` invocations must not interleave DDL.
const migrationLockKey = 0x01c05e5
// Migrate runs all embedded forward migrations in order.
// Uses a schema_migrations table to track applied versions. The whole run
// happens on one connection holding a session advisory lock.
func (p *Pool) Migrate(ctx context.Context) error {
conn, err := p.Acquire(ctx)
if err != nil {
return fmt.Errorf("acquire migration conn: %w", err)
}
defer conn.Release()
if _, err := conn.Exec(ctx, "SELECT pg_advisory_lock($1)", migrationLockKey); err != nil {
return fmt.Errorf("acquire migration lock: %w", err)
}
defer conn.Exec(context.WithoutCancel(ctx), "SELECT pg_advisory_unlock($1)", migrationLockKey)
// Create tracking table if not exists
_, err = conn.Exec(ctx, `
CREATE TABLE IF NOT EXISTS schema_migrations (
version INT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
`)
if err != nil {
return fmt.Errorf("create schema_migrations: %w", err)
}
// List migration files
entries, err := fs.ReadDir(migrations.FS, ".")
if err != nil {
return fmt.Errorf("read migration fs: %w", err)
}
var files []string
for _, e := range entries {
if !e.IsDir() && hasSuffix(e.Name(), ".up.sql") {
files = append(files, e.Name())
}
}
sort.Strings(files)
for _, fname := range files {
// Extract version number (001, 002, etc.)
var version int
if _, err := fmt.Sscanf(fname, "%03d", &version); err != nil {
continue
}
// Check if already applied
var applied int
err := conn.QueryRow(ctx,
"SELECT COUNT(*) FROM schema_migrations WHERE version = $1", version).Scan(&applied)
if err != nil {
return fmt.Errorf("check migration %d: %w", version, err)
}
if applied > 0 {
continue
}
// Read and execute migration — split into individual statements
// because TimescaleDB CAGGs and some DDL can't run inside a transaction,
// and pgx's multi-statement Exec wraps them implicitly.
content, err := migrations.FS.ReadFile(fname)
if err != nil {
return fmt.Errorf("read %s: %w", fname, err)
}
stmts := splitSQL(string(content))
for i, stmt := range stmts {
stmt = strings.TrimSpace(stmt)
if stmt == "" {
continue
}
_, err := conn.Exec(ctx, stmt)
if err != nil {
return fmt.Errorf("exec migration %s stmt %d: %w", fname, i+1, err)
}
}
_, err = conn.Exec(ctx, "INSERT INTO schema_migrations (version) VALUES ($1)", version)
if err != nil {
return fmt.Errorf("record migration %d: %w", version, err)
}
slog.Info("migration applied", "file", fname, "version", version, "statements", len(stmts))
}
return nil
}
// SeedIngest ingests a YAML seed file into the database.
// Idempotent: if the file's content hash matches seed_versions, it's a no-op (A4).
func (p *Pool) SeedIngest(ctx context.Context, filename string, content []byte,
ingestFn func(ctx context.Context, tx pgx.Tx, data map[string]any) error) error {
hash := contentHash(content)
// Check if already applied with same hash
var existing string
err := p.QueryRow(ctx,
"SELECT content_hash FROM seed_versions WHERE file = $1", filename).Scan(&existing)
if err == nil && existing == hash {
return nil // no-op, same content
}
// Parse YAML
var data map[string]any
if err := yaml.Unmarshal(content, &data); err != nil {
return fmt.Errorf("parse %s: %w", filename, err)
}
// Apply in a single transaction
tx, err := p.Begin(ctx)
if err != nil {
return fmt.Errorf("begin tx: %w", err)
}
defer tx.Rollback(ctx)
if err := ingestFn(ctx, tx, data); err != nil {
return fmt.Errorf("ingest %s: %w", filename, err)
}
// Record the seed version
_, err = tx.Exec(ctx,
`INSERT INTO seed_versions (file, content_hash) VALUES ($1, $2)
ON CONFLICT (file) DO UPDATE SET content_hash = $2, applied_at = now()`,
filename, hash)
if err != nil {
return fmt.Errorf("record seed version: %w", err)
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("commit seed: %w", err)
}
return nil
}
// contentHash returns a SHA-256 hex digest of the content.
func contentHash(content []byte) string {
h := sha256.Sum256(content)
return hex.EncodeToString(h[:])
}
// hasSuffix reports whether the string ends with the given suffix.
func hasSuffix(s, suffix string) bool {
return strings.HasSuffix(s, suffix)
}
// splitSQL splits a SQL string into individual statements.
// Handles $$ ... $$ dollar-quoted blocks and -- line comments.
func splitSQL(sql string) []string {
var statements []string
var current strings.Builder
inDollarQuote := false
dollarTag := ""
i := 0
for i < len(sql) {
// Handle line comments (-- to end of line)
if !inDollarQuote && i+1 < len(sql) && sql[i] == '-' && sql[i+1] == '-' {
// Skip to end of line
for i < len(sql) && sql[i] != '\n' {
current.WriteByte(sql[i])
i++
}
continue
}
// Check for dollar-quote start/end
if !inDollarQuote && sql[i] == '$' {
j := i + 1
for j < len(sql) && (sql[j] == '_' || (sql[j] >= 'a' && sql[j] <= 'z') || (sql[j] >= 'A' && sql[j] <= 'Z') || (sql[j] >= '0' && sql[j] <= '9')) {
j++
}
if j < len(sql) && sql[j] == '$' {
dollarTag = sql[i : j+1]
current.WriteString(dollarTag)
inDollarQuote = true
i = j + 1
continue
}
}
if inDollarQuote && strings.HasPrefix(sql[i:], dollarTag) {
current.WriteString(dollarTag)
i += len(dollarTag)
inDollarQuote = false
dollarTag = ""
continue
}
if !inDollarQuote && sql[i] == ';' {
statements = append(statements, current.String())
current.Reset()
i++
continue
}
current.WriteByte(sql[i])
i++
}
if strings.TrimSpace(current.String()) != "" {
statements = append(statements, current.String())
}
return statements
}

View File

@@ -0,0 +1,52 @@
-- Entity read + mutation queries (API paths). Aliased `e` throughout to
-- avoid ambiguity with joined tables.
-- name: GetEntityByID :one
SELECT e.* FROM entities e WHERE e.id = $1;
-- name: GetEntityBySlug :one
SELECT e.* FROM entities e WHERE e.slug = $1;
-- name: ListEntities :many
WITH RECURSIVE tt AS (
SELECT name FROM entity_types WHERE sqlc.narg('type')::text IS NULL OR name = sqlc.narg('type')
UNION
SELECT et.name FROM entity_types et JOIN tt ON et.parent_type = tt.name
WHERE sqlc.narg('type')::text IS NOT NULL
)
SELECT e.* FROM entities e
JOIN entity_types et ON et.name = e.type
WHERE e.type IN (SELECT name FROM tt)
AND (sqlc.narg('state')::text IS NULL OR e.state = sqlc.narg('state'))
AND (sqlc.narg('domain')::text IS NULL OR et.domain = sqlc.narg('domain'))
AND (sqlc.narg('layer')::text IS NULL OR et.layer = sqlc.narg('layer'))
AND (sqlc.narg('q')::text IS NULL
OR e.slug ILIKE '%'||sqlc.narg('q')||'%'
OR e.name ILIKE '%'||sqlc.narg('q')||'%')
AND (sqlc.narg('cursor')::text IS NULL OR e.slug > sqlc.narg('cursor'))
ORDER BY e.slug
LIMIT sqlc.arg('lim');
-- name: ListEntitiesCapped :many
SELECT e.* FROM entities e ORDER BY e.slug LIMIT $1;
-- name: InsertEntity :one
INSERT INTO entities (id, slug, type, name, state, attributes)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING *;
-- name: UpdateEntity :one
UPDATE entities SET
name = COALESCE(sqlc.narg('name'), name),
state = COALESCE(sqlc.narg('state'), state),
attributes = COALESCE(sqlc.narg('attributes'), attributes),
maintenance_until = CASE WHEN sqlc.arg('set_maintenance')::bool
THEN sqlc.narg('maintenance_until') ELSE maintenance_until END,
version = version + 1,
updated_at = now()
WHERE id = sqlc.arg('id') AND version = sqlc.arg('version')
RETURNING *;
-- blast_radius(): the recursive-CTE traversal function's TABLE return type
-- is opaque to sqlc's analyzer — that one query stays hand-written pgx in
-- internal/httpapi (see impl.go).

View File

@@ -0,0 +1,13 @@
-- name: ListEntityTypes :many
SELECT * FROM entity_types ORDER BY name;
-- name: ListRelationshipTypes :many
SELECT * FROM relationship_types ORDER BY name;
-- name: ListLifecycleDefs :many
SELECT * FROM lifecycle_defs ORDER BY id;
-- name: GetLifecycleForType :one
SELECT ld.* FROM lifecycle_defs ld
JOIN entity_types et ON et.lifecycle_id = ld.id
WHERE et.name = $1;

View File

@@ -0,0 +1,269 @@
-- name: ListSignals :many
SELECT sig.entity_id, se.slug, sig.kind, sig.severity, sig.state,
te.slug AS target_slug, sig.check_id, sig.evidence, sig.likely_cause,
sig.occurrence_count, sig.flap_count, sig.hold_down_until,
sig.mute_until, sig.first_seen_at, sig.last_seen_at
FROM signals sig
JOIN entities se ON se.id = sig.entity_id
LEFT JOIN entities te ON te.id = sig.target_entity_id
WHERE (sqlc.narg('state')::text IS NULL OR sig.state = sqlc.narg('state'))
AND (sqlc.narg('severity')::text IS NULL OR sig.severity = sqlc.narg('severity'))
AND (sqlc.narg('target')::text IS NULL OR te.slug = sqlc.narg('target'))
AND (sqlc.narg('kind')::text IS NULL OR sig.kind = sqlc.narg('kind'))
AND (sqlc.narg('cursor')::text IS NULL OR se.slug > sqlc.narg('cursor'))
ORDER BY se.slug
LIMIT sqlc.arg('lim');
-- name: ListEntityStatus :many
SELECT e.slug, e.type, st.health, st.last_check_at
FROM entity_status st JOIN entities e ON e.id = st.entity_id
ORDER BY e.slug;
-- name: GetIdempotentResponse :one
SELECT response_code, response_body, request_hash FROM idempotency_keys
WHERE actor = $1 AND key = $2;
-- name: PutIdempotentResponse :exec
INSERT INTO idempotency_keys (actor, key, request_hash, response_code, response_body)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (actor, key) DO NOTHING;
-- name: InsertAuditEntry :exec
INSERT INTO audit_log (actor_type, actor_id, action, entity_id, method, path,
status_code, detail, source_ip, correlation_id)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10);
-- name: InsertEvent :one
INSERT INTO events (type, entity_id, severity, source, data, correlation_id)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, ts;
-- name: ListEvents :many
SELECT id, ts, type, entity_id, severity, source, data, correlation_id
FROM events
WHERE (sqlc.narg('type')::text IS NULL OR type = sqlc.narg('type'))
AND (sqlc.narg('entity_id')::uuid IS NULL OR entity_id = sqlc.narg('entity_id'))
AND (sqlc.narg('severity')::text IS NULL OR severity = sqlc.narg('severity'))
AND (sqlc.narg('correlation_id')::text IS NULL OR correlation_id = sqlc.narg('correlation_id'))
AND (sqlc.narg('from_ts')::timestamptz IS NULL OR ts >= sqlc.narg('from_ts'))
AND (sqlc.narg('to_ts')::timestamptz IS NULL OR ts <= sqlc.narg('to_ts'))
AND (sqlc.narg('before_id')::bigint IS NULL OR id < sqlc.narg('before_id'))
ORDER BY id DESC
LIMIT sqlc.arg('lim');
-- name: ListEventsAfter :many
SELECT id, ts, type, entity_id, severity, source, data, correlation_id
FROM events WHERE id > $1 ORDER BY id ASC LIMIT $2;
-- =====================================================================
-- Phase 3 queries
-- =====================================================================
-- name: ListEnabledCheckDefs :many
SELECT cd.entity_id, cd.target_id, cd.target_type, cd.kind, cd.config,
cd.interval_s, cd.timeout_s, cd.zone, cd.enabled, cd.updated_at,
e.slug AS entity_slug
FROM check_defs cd
JOIN entities e ON e.id = cd.entity_id
WHERE cd.enabled = true;
-- name: GetCheckDef :one
SELECT * FROM check_defs WHERE entity_id = $1;
-- name: InsertCheckDef :exec
INSERT INTO check_defs (entity_id, target_id, target_type, kind, config, interval_s, timeout_s, zone, enabled)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9);
-- name: UpdateCheckDef :exec
UPDATE check_defs SET kind = $2, config = $3, interval_s = $4, timeout_s = $5,
target_id = $6, target_type = $7, zone = $8, enabled = $9, updated_at = now()
WHERE entity_id = $1;
-- name: UpsertSignal :one
INSERT INTO signals (entity_id, kind, severity, target_entity_id, check_id, evidence, likely_cause, state)
VALUES ($1, $2, $3, $4, $5, $6, $7, 'raised')
ON CONFLICT (target_entity_id, kind) WHERE state NOT IN ('resolved','failed')
DO UPDATE SET occurrence_count = signals.occurrence_count + 1,
last_seen_at = now(),
evidence = EXCLUDED.evidence,
updated_at = now()
RETURNING *;
-- name: UpdateSignalState :exec
UPDATE signals SET state = $2, updated_at = now() WHERE entity_id = $1;
-- name: GetOpenSignalsForAutoAct :many
-- Signals with auto-act classifications that haven't been executed yet
SELECT s.*, c.entity_id AS classification_id, c.action, c.risk_class, c.route,
c.blast_radius, c.correlation_id, c.reasoning
FROM classifications c
JOIN signals s ON s.entity_id = c.signal_entity_id
LEFT JOIN executions e ON e.classification_id = c.entity_id
WHERE c.route = 'auto-act'
AND e.entity_id IS NULL
AND (s.hold_down_until IS NULL OR s.hold_down_until < now())
AND (s.mute_until IS NULL OR s.mute_until < now())
ORDER BY s.last_seen_at ASC
LIMIT $1;
-- name: InsertClassification :exec
INSERT INTO classifications (entity_id, signal_entity_id, target_entity_id, action,
recommended_action, risk_class, route, blast_radius, pattern_confidence,
skill_id, autonomy_check, reasoning, correlation_id)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13);
-- name: ListClassifications :many
SELECT c.entity_id, c.signal_entity_id, c.target_entity_id, c.action,
c.recommended_action, c.risk_class, c.route, c.blast_radius,
c.pattern_confidence, c.skill_id, c.autonomy_check, c.reasoning,
c.correlation_id, c.created_at,
e.slug AS target_slug
FROM classifications c
JOIN entities e ON e.id = c.target_entity_id
WHERE (sqlc.narg('route')::text IS NULL OR c.route = sqlc.narg('route'))
AND (sqlc.narg('cursor')::text IS NULL OR e.slug > sqlc.narg('cursor'))
ORDER BY e.slug
LIMIT sqlc.arg('lim');
-- name: InsertExecution :exec
INSERT INTO executions (entity_id, classification_id, signal_entity_id,
target_entity_id, action, risk_class, approval_id, agent_id,
skill_id, skill_version, status, correlation_id)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, 'proposed', $11);
-- name: UpdateExecutionStatus :exec
UPDATE executions SET status = $2, result = $3, duration_ms = $4,
verified = $5, started_at = COALESCE(started_at, now()),
completed_at = CASE WHEN $2 IN ('completed','failed','cancelled') THEN now() ELSE completed_at END
WHERE entity_id = $1;
-- name: GetExecution :one
SELECT * FROM executions WHERE entity_id = $1;
-- name: ListExecutions :many
SELECT e.entity_id, e.classification_id, e.signal_entity_id, e.target_entity_id,
e.action, e.risk_class, e.approval_id, e.agent_id,
e.skill_id, e.skill_version, e.status, e.result, e.duration_ms,
e.verified, e.correlation_id, e.started_at, e.completed_at, e.created_at,
te.slug AS target_slug
FROM executions e
JOIN entities te ON te.id = e.target_entity_id
WHERE (sqlc.narg('status')::text IS NULL OR e.status = sqlc.narg('status'))
AND (sqlc.narg('cursor')::text IS NULL OR te.slug > sqlc.narg('cursor'))
ORDER BY te.slug
LIMIT sqlc.arg('lim');
-- name: InsertFeedback :exec
INSERT INTO feedback (entity_id, execution_id, outcome, observation, lesson,
unexpected_side_effects, tags)
VALUES ($1, $2, $3, $4, $5, $6, $7);
-- name: GetFeedbackAfterWatermark :many
SELECT f.entity_id, f.execution_id, f.outcome, f.observation, f.lesson,
f.unexpected_side_effects, f.tags, f.created_at,
e.action, e.risk_class, e.target_entity_id,
et.name AS applies_type
FROM feedback f
JOIN executions e ON e.entity_id = f.execution_id
JOIN entities ent ON ent.id = e.target_entity_id
JOIN entity_types et ON et.name = ent.type
WHERE f.created_at > $1
ORDER BY f.created_at ASC;
-- name: UpsertPattern :exec
INSERT INTO patterns (entity_id, applies_type, action, pattern, confidence,
evidence_count, success_count, failure_count, status, version)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 'hypothesized', 1)
ON CONFLICT (applies_type, action)
DO UPDATE SET evidence_count = patterns.evidence_count + EXCLUDED.evidence_count,
success_count = patterns.success_count + EXCLUDED.success_count,
failure_count = patterns.failure_count + EXCLUDED.failure_count,
updated_at = now();
-- name: GetPattern :one
SELECT * FROM patterns WHERE applies_type = $1 AND action = $2;
-- name: ListPatterns :many
SELECT p.* FROM patterns p
WHERE (sqlc.narg('status')::text IS NULL OR p.status = sqlc.narg('status'))
ORDER BY p.applies_type, p.action;
-- name: UpdatePatternStatus :exec
UPDATE patterns SET status = $2, version = version + 1,
last_validated_at = CASE WHEN $2 = 'validated' THEN now() ELSE last_validated_at END
WHERE entity_id = $1;
-- name: UpdatePatternQuarantine :exec
UPDATE patterns SET quarantined = $2 WHERE entity_id = $1;
-- name: ListSkills :many
SELECT * FROM skills
WHERE (sqlc.narg('status')::text IS NULL OR status = sqlc.narg('status'))
ORDER BY name, version DESC;
-- name: InsertSkill :exec
INSERT INTO skills (entity_id, version, name, procedure, applies_type, action,
pattern_ids, status, changed_by, change_reason)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10);
-- name: UpdateSkillStatus :exec
UPDATE skills SET status = $2, last_used_at = now() WHERE entity_id = $1 AND version = $2;
-- name: InsertApproval :exec
INSERT INTO approvals (entity_id, subject_entity_id, action, risk_class, kind,
payload, status, token_hash, expires_at)
VALUES ($1, $2, $3, $4, $5, $6, 'pending', $7, $8);
-- name: ListApprovals :many
SELECT a.*, e.slug AS subject_slug
FROM approvals a
JOIN entities e ON e.id = a.subject_entity_id
WHERE (sqlc.narg('status')::text IS NULL OR a.status = sqlc.narg('status'))
AND (sqlc.narg('cursor')::text IS NULL OR e.slug > sqlc.narg('cursor'))
ORDER BY e.slug
LIMIT sqlc.arg('lim');
-- name: GetApprovalByID :one
SELECT * FROM approvals WHERE entity_id = $1;
-- name: UpdateApprovalStatus :exec
UPDATE approvals SET status = $2, decided_at = now(), decided_by = $3
WHERE entity_id = $1 AND status = 'pending';
-- name: GetAutonomySetting :one
SELECT value FROM autonomy_settings WHERE key = $1;
-- name: ListRiskClasses :many
SELECT * FROM risk_classes ORDER BY name;
-- name: ListApprovalRules :many
SELECT * FROM approval_rules ORDER BY entity_type, action;
-- name: InsertMetricSample :exec
INSERT INTO metric_samples (entity_id, metric, value, tags)
VALUES ($1, $2, $3, $4);
-- name: QueryMetrics :many
SELECT time_bucket(sqlc.arg('bucket_interval')::interval, ts) AS bucket,
entity_id, metric,
ROUND(avg(value)::numeric, 2) AS avg_val,
ROUND(min(value)::numeric, 2) AS min_val,
ROUND(max(value)::numeric, 2) AS max_val
FROM metric_samples
WHERE entity_id = $1
AND metric = $2
AND ts > $3
GROUP BY bucket, entity_id, metric
ORDER BY bucket DESC;
-- name: UpsertEntityStatus :exec
INSERT INTO entity_status (entity_id, health, last_check_at, details)
VALUES ($1, $2, $3, $4)
ON CONFLICT (entity_id)
DO UPDATE SET health = EXCLUDED.health,
last_check_at = EXCLUDED.last_check_at,
details = EXCLUDED.details,
updated_at = now();
-- name: GetEntityStatus :one
SELECT * FROM entity_status WHERE entity_id = $1;

View File

@@ -0,0 +1,33 @@
-- name: ListEntityRelations :many
SELECT se.slug AS source_slug, te.slug AS target_slug, r.type, r.attributes,
r.valid_from, r.valid_to
FROM relationships r
JOIN entities se ON se.id = r.source_id
JOIN entities te ON te.id = r.target_id
WHERE r.valid_to IS NULL
AND ((sqlc.arg('direction')::text IN ('out','both') AND r.source_id = sqlc.arg('id'))
OR (sqlc.arg('direction')::text IN ('in','both') AND r.target_id = sqlc.arg('id')))
AND (sqlc.narg('rel_type')::text IS NULL OR r.type = sqlc.narg('rel_type'))
ORDER BY r.type, se.slug, te.slug;
-- name: ListGraphEdges :many
SELECT se.slug AS source_slug, te.slug AS target_slug, r.type, r.attributes,
r.valid_from, r.valid_to
FROM relationships r
JOIN entities se ON se.id = r.source_id
JOIN entities te ON te.id = r.target_id
WHERE r.valid_to IS NULL
AND r.source_id = ANY(sqlc.arg('ids')::uuid[])
AND r.target_id = ANY(sqlc.arg('ids')::uuid[])
AND (sqlc.narg('rel_types')::text[] IS NULL OR r.type = ANY(sqlc.narg('rel_types')::text[]))
ORDER BY r.type, se.slug, te.slug;
-- name: UpsertCurrentRelationship :exec
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from, valid_to)
VALUES ($1, $2, $3, $4, now(), NULL)
ON CONFLICT (source_id, target_id, type) WHERE valid_to IS NULL
DO UPDATE SET attributes = EXCLUDED.attributes;
-- name: EndCurrentRelationship :execrows
UPDATE relationships SET valid_to = now()
WHERE source_id = $1 AND target_id = $2 AND type = $3 AND valid_to IS NULL;

393
internal/db/seed.go Normal file
View File

@@ -0,0 +1,393 @@
package db
import (
"context"
"encoding/json"
"errors"
"fmt"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
)
// SeedResult holds counts from a seed ingest operation.
type SeedResult struct {
Lifecycles int
EntityTypes int
RelationshipTypes int
Entities int
Relationships int
RiskClasses int
ApprovalRules int
AutonomySettings int
}
// IngestOntologySeed ingests seeds/ontology.yaml into the DB.
func IngestOntologySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*SeedResult, error) {
r := &SeedResult{}
// Lifecycles
lifecycles, _ := data["lifecycles"].(map[string]any)
for id, raw := range lifecycles {
lcMap, _ := raw.(map[string]any)
states := toStringSlice(lcMap["states"])
defaultState, _ := lcMap["default_state"].(string)
terminalStates := toStringSlice(lcMap["terminal_states"])
if len(terminalStates) == 0 {
terminalStates = []string{}
}
transitionsBytes, _ := json.Marshal(lcMap["transitions"])
_, err := tx.Exec(ctx,
`INSERT INTO lifecycle_defs (id, states, default_state, terminal_states, transitions)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (id) DO UPDATE SET states = $2, default_state = $3,
terminal_states = $4, transitions = $5`,
id, states, defaultState, terminalStates, string(transitionsBytes))
if err != nil {
return nil, fmt.Errorf("lifecycle %s: %w", id, err)
}
r.Lifecycles++
}
// Entity types — need to handle parent_type FK, so insert in dependency order
// (types with no parent first, then their children)
types, _ := data["entity_types"].(map[string]any)
if err := insertEntityTypes(ctx, tx, types, r); err != nil {
return nil, err
}
// Relationship types
relTypes, _ := data["relationship_types"].(map[string]any)
for name, raw := range relTypes {
rtMap, _ := raw.(map[string]any)
inverse, _ := rtMap["inverse"].(string)
sourceType, _ := rtMap["source"].(string)
targetType, _ := rtMap["target"].(string)
cardinality, _ := rtMap["cardinality"].(string)
desc, _ := rtMap["description"].(string)
_, err := tx.Exec(ctx,
`INSERT INTO relationship_types (name, inverse, source_type, target_type, cardinality, description)
VALUES ($1, $2, $3, $4, $5, $6)
ON CONFLICT (name) DO UPDATE SET inverse = $2, source_type = $3,
target_type = $4, cardinality = $5, description = $6`,
name, nullableStr(inverse), sourceType, targetType, cardinality, desc)
if err != nil {
return nil, fmt.Errorf("relationship_type %s: %w", name, err)
}
r.RelationshipTypes++
}
return r, nil
}
// IngestInventorySeed ingests seeds/inventory.yaml into the DB.
// Every entity and edge is validated against the ontology (abstract types
// rejected, lifecycle states checked, relationship endpoints hierarchy-
// validated, cardinality enforced) — a violating seed rolls back atomically.
func IngestInventorySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*SeedResult, error) {
r := &SeedResult{}
tree, err := LoadTypeTree(ctx, tx)
if err != nil {
return nil, fmt.Errorf("load type tree: %w", err)
}
// Entities
entities, _ := data["entities"].([]any)
entityTypes := make(map[string]string) // slug -> type, for edge validation
for _, raw := range entities {
eMap, ok := raw.(map[string]any)
if !ok {
continue
}
slug, _ := eMap["slug"].(string)
typeName, _ := eMap["type"].(string)
name, _ := eMap["name"].(string)
state, _ := eMap["state"].(string)
attrs := eMap["attributes"]
if err := tree.ValidateEntity(typeName, state); err != nil {
return nil, fmt.Errorf("entity %s: %w", slug, err)
}
if state == "" {
state = tree.DefaultState(typeName)
}
entityTypes[slug] = typeName
entityID, err := getOrCreateEntityID(ctx, tx, slug)
if err != nil {
return nil, fmt.Errorf("entity %s: %w", slug, err)
}
attrsBytes, _ := json.Marshal(attrs)
_, err = tx.Exec(ctx,
`INSERT INTO entities (id, slug, type, name, state, attributes, version, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, 1, now(), now())
ON CONFLICT (slug) DO UPDATE SET type = $3, name = $4, state = $5,
attributes = $6, updated_at = now()`,
entityID, slug, typeName, name, nullableStr(state), string(attrsBytes))
if err != nil {
return nil, fmt.Errorf("entity %s: %w", slug, err)
}
r.Entities++
}
// Relationships — upsert against the current-edge partial unique index
// (migration 007) so re-ingest never duplicates edges.
rels, _ := data["relationships"].([]any)
for _, raw := range rels {
relMap, ok := raw.(map[string]any)
if !ok {
continue
}
source, _ := relMap["source"].(string)
target, _ := relMap["target"].(string)
relType, _ := relMap["type"].(string)
attrs := relMap["attributes"]
sourceID, err := getEntityIDBySlug(ctx, tx, source)
if err != nil {
return nil, fmt.Errorf("rel from %s: %w", source, err)
}
targetID, err := getEntityIDBySlug(ctx, tx, target)
if err != nil {
return nil, fmt.Errorf("rel to %s: %w", target, err)
}
srcType := entityTypes[source]
tgtType := entityTypes[target]
if srcType == "" || tgtType == "" { // entity pre-existing in DB, not in this seed
if srcType == "" {
srcType, err = getEntityTypeBySlug(ctx, tx, source)
if err != nil {
return nil, err
}
}
if tgtType == "" {
tgtType, err = getEntityTypeBySlug(ctx, tx, target)
if err != nil {
return nil, err
}
}
}
if err := tree.ValidateEdge(relType, srcType, tgtType); err != nil {
return nil, fmt.Errorf("rel %s→%s: %w", source, target, err)
}
attrsBytes, _ := json.Marshal(attrs)
_, err = tx.Exec(ctx,
`INSERT INTO relationships (source_id, target_id, type, attributes, valid_from, valid_to)
VALUES ($1, $2, $3, $4, now(), NULL)
ON CONFLICT (source_id, target_id, type) WHERE valid_to IS NULL
DO UPDATE SET attributes = EXCLUDED.attributes`,
sourceID, targetID, relType, string(attrsBytes))
if err != nil {
return nil, fmt.Errorf("rel %s→%s %s: %w", source, target, relType, err)
}
r.Relationships++
}
if err := ValidateCardinality(ctx, tx); err != nil {
return nil, err
}
return r, nil
}
// IngestPolicySeed ingests seeds/policy.yaml into the DB.
func IngestPolicySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*SeedResult, error) {
r := &SeedResult{}
// Risk classes
riskClasses, _ := data["risk_classes"].(map[string]any)
for name, raw := range riskClasses {
rcMap, _ := raw.(map[string]any)
desc, _ := rcMap["description"].(string)
approval, _ := rcMap["approval_required"].(string)
autonomy, _ := rcMap["autonomy_allowed"].(bool)
_, err := tx.Exec(ctx,
`INSERT INTO risk_classes (name, description, approval_required, autonomy_allowed)
VALUES ($1, $2, $3, $4)
ON CONFLICT (name) DO UPDATE SET description = $2, approval_required = $3, autonomy_allowed = $4`,
name, desc, approval, autonomy)
if err != nil {
return nil, fmt.Errorf("risk_class %s: %w", name, err)
}
r.RiskClasses++
}
// Approval rules
rules, _ := data["approval_rules"].([]any)
for _, raw := range rules {
ruleMap, ok := raw.(map[string]any)
if !ok {
continue
}
entityType, _ := ruleMap["entity_type"].(string)
action, _ := ruleMap["action"].(string)
riskClass, _ := ruleMap["risk_class"].(string)
autonomy, _ := ruleMap["autonomy_level"].(string)
scopeEntity, _ := ruleMap["scope_entity"].(string)
var scopeID any
if scopeEntity != "" {
id, err := getEntityIDBySlug(ctx, tx, scopeEntity)
if err == nil {
scopeID = id
}
}
ruleID := uuid.New()
_, err := tx.Exec(ctx,
`INSERT INTO approval_rules (id, entity_type, action, risk_class, autonomy_level, scope_entity, version, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, 1, now())
ON CONFLICT (entity_type, action, scope_entity)
DO UPDATE SET risk_class = $4, autonomy_level = $5, scope_entity = $6, updated_at = now()`,
ruleID, nullableStr(entityType), action, riskClass, autonomy, scopeID)
if err != nil {
return nil, fmt.Errorf("approval_rule %s/%s: %w", entityType, action, err)
}
r.ApprovalRules++
}
// Autonomy settings
settings, _ := data["autonomy_settings"].(map[string]any)
for key, raw := range settings {
val, _ := raw.(string)
_, err := tx.Exec(ctx,
`INSERT INTO autonomy_settings (key, value, version, updated_at)
VALUES ($1, $2, 1, now())
ON CONFLICT (key) DO UPDATE SET value = $2, updated_at = now()`,
key, val)
if err != nil {
return nil, fmt.Errorf("autonomy_setting %s: %w", key, err)
}
r.AutonomySettings++
}
return r, nil
}
// --- Helpers ---
// getOrCreateEntityID returns the UUID for a slug, generating a new
// time-ordered UUIDv7 if the slug doesn't exist yet (ADR-0005).
func getOrCreateEntityID(ctx context.Context, tx pgx.Tx, slug string) (uuid.UUID, error) {
var id uuid.UUID
err := tx.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", slug).Scan(&id)
switch {
case err == nil:
return id, nil
case errors.Is(err, pgx.ErrNoRows):
return uuid.NewV7()
default:
return uuid.Nil, fmt.Errorf("lookup slug %s: %w", slug, err)
}
}
// getEntityTypeBySlug resolves a slug to its entity type name.
func getEntityTypeBySlug(ctx context.Context, tx pgx.Tx, slug string) (string, error) {
var t string
err := tx.QueryRow(ctx, "SELECT type FROM entities WHERE slug = $1", slug).Scan(&t)
if err != nil {
return "", fmt.Errorf("resolve type of %s: %w", slug, err)
}
return t, nil
}
// getEntityIDBySlug resolves a slug to its UUID.
func getEntityIDBySlug(ctx context.Context, tx pgx.Tx, slug string) (uuid.UUID, error) {
var id uuid.UUID
err := tx.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", slug).Scan(&id)
if err != nil {
return uuid.Nil, fmt.Errorf("resolve slug %s: %w", slug, err)
}
return id, nil
}
// insertEntityTypes inserts entity types in dependency order (parents before children).
func insertEntityTypes(ctx context.Context, tx pgx.Tx, types map[string]any, r *SeedResult) error {
// Build a dependency graph and insert in topological order
// Simple approach: insert types with no parent first, then iterate
inserted := make(map[string]bool)
remaining := make(map[string]map[string]any)
for name, raw := range types {
tMap, _ := raw.(map[string]any)
remaining[name] = tMap
}
maxPasses := 10
for pass := 0; pass < maxPasses && len(remaining) > 0; pass++ {
for name, tMap := range remaining {
parent, _ := tMap["parent"].(string)
if parent == "" || inserted[parent] {
if err := insertOneEntityType(ctx, tx, name, tMap); err != nil {
return err
}
inserted[name] = true
delete(remaining, name)
r.EntityTypes++
}
}
}
if len(remaining) > 0 {
return fmt.Errorf("circular or missing parent in entity types: %v", keysOf(remaining))
}
return nil
}
func insertOneEntityType(ctx context.Context, tx pgx.Tx, name string, tMap map[string]any) error {
parent, _ := tMap["parent"].(string)
isAbstract, _ := tMap["abstract"].(bool)
domain, _ := tMap["domain"].(string)
layer, _ := tMap["layer"].(string)
desc, _ := tMap["description"].(string)
lifecycleID, _ := tMap["lifecycle"].(string)
attrSchema := tMap["attribute_schema"]
schemaBytes, _ := json.Marshal(attrSchema)
_, err := tx.Exec(ctx,
`INSERT INTO entity_types (name, parent_type, is_abstract, domain, layer, description,
lifecycle_id, attribute_schema, schema_version, status, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 1, 'active', now(), now())
ON CONFLICT (name) DO UPDATE SET parent_type = $2, is_abstract = $3, domain = $4,
layer = $5, description = $6, lifecycle_id = $7, attribute_schema = $8, updated_at = now()`,
name, nullableStr(parent), isAbstract, domain, layer, desc, nullableStr(lifecycleID), nullableStr(string(schemaBytes)))
return err
}
func toStringSlice(v any) []string {
if v == nil {
return nil
}
switch s := v.(type) {
case []string:
return s
case []any:
out := make([]string, 0, len(s))
for _, item := range s {
if str, ok := item.(string); ok {
out = append(out, str)
}
}
return out
}
return nil
}
func nullableStr(s string) any {
if s == "" {
return nil
}
return s
}
func keysOf(m map[string]map[string]any) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
return keys
}

View File

@@ -0,0 +1,53 @@
package db
import (
"strings"
"testing"
)
func nonEmpty(stmts []string) []string {
var out []string
for _, s := range stmts {
if strings.TrimSpace(s) != "" {
out = append(out, s)
}
}
return out
}
func TestSplitSQLBasic(t *testing.T) {
stmts := nonEmpty(splitSQL("CREATE TABLE a (id int); CREATE TABLE b (id int);"))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}
func TestSplitSQLDollarQuotedFunction(t *testing.T) {
sql := `CREATE FUNCTION f() RETURNS int AS $$
SELECT 1; SELECT 2;
$$ LANGUAGE sql;
CREATE TABLE t (id int);`
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
if !strings.Contains(stmts[0], "SELECT 1; SELECT 2;") {
t.Errorf("dollar-quoted body was split: %q", stmts[0])
}
}
func TestSplitSQLTaggedDollarQuote(t *testing.T) {
sql := `DO $body$ BEGIN PERFORM 1; END $body$;SELECT 1;`
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}
func TestSplitSQLSemicolonInComment(t *testing.T) {
sql := "-- comment with ; semicolon\nCREATE TABLE t (id int); -- trailing; note\nSELECT 1;"
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}

32
internal/db/sqlcgen/db.go Normal file
View File

@@ -0,0 +1,32 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
package sqlcgen
import (
"context"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
type DBTX interface {
Exec(context.Context, string, ...interface{}) (pgconn.CommandTag, error)
Query(context.Context, string, ...interface{}) (pgx.Rows, error)
QueryRow(context.Context, string, ...interface{}) pgx.Row
}
func New(db DBTX) *Queries {
return &Queries{db: db}
}
type Queries struct {
db DBTX
}
func (q *Queries) WithTx(tx pgx.Tx) *Queries {
return &Queries{
db: tx,
}
}

View File

@@ -0,0 +1,254 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
// source: entities.sql
package sqlcgen
import (
"context"
"time"
"github.com/google/uuid"
)
const getEntityByID = `-- name: GetEntityByID :one
SELECT e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at FROM entities e WHERE e.id = $1
`
// Entity read + mutation queries (API paths). Aliased `e` throughout to
// avoid ambiguity with joined tables.
func (q *Queries) GetEntityByID(ctx context.Context, id uuid.UUID) (Entity, error) {
row := q.db.QueryRow(ctx, getEntityByID, id)
var i Entity
err := row.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
const getEntityBySlug = `-- name: GetEntityBySlug :one
SELECT e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at FROM entities e WHERE e.slug = $1
`
func (q *Queries) GetEntityBySlug(ctx context.Context, slug string) (Entity, error) {
row := q.db.QueryRow(ctx, getEntityBySlug, slug)
var i Entity
err := row.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
const insertEntity = `-- name: InsertEntity :one
INSERT INTO entities (id, slug, type, name, state, attributes)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, slug, type, name, state, attributes, maintenance_until, version, created_at, updated_at
`
type InsertEntityParams struct {
ID uuid.UUID
Slug string
Type string
Name string
State *string
Attributes []byte
}
func (q *Queries) InsertEntity(ctx context.Context, arg InsertEntityParams) (Entity, error) {
row := q.db.QueryRow(ctx, insertEntity,
arg.ID,
arg.Slug,
arg.Type,
arg.Name,
arg.State,
arg.Attributes,
)
var i Entity
err := row.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
const listEntities = `-- name: ListEntities :many
WITH RECURSIVE tt AS (
SELECT name FROM entity_types WHERE $7::text IS NULL OR name = $7
UNION
SELECT et.name FROM entity_types et JOIN tt ON et.parent_type = tt.name
WHERE $7::text IS NOT NULL
)
SELECT e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at FROM entities e
JOIN entity_types et ON et.name = e.type
WHERE e.type IN (SELECT name FROM tt)
AND ($1::text IS NULL OR e.state = $1)
AND ($2::text IS NULL OR et.domain = $2)
AND ($3::text IS NULL OR et.layer = $3)
AND ($4::text IS NULL
OR e.slug ILIKE '%'||$4||'%'
OR e.name ILIKE '%'||$4||'%')
AND ($5::text IS NULL OR e.slug > $5)
ORDER BY e.slug
LIMIT $6
`
type ListEntitiesParams struct {
State *string
Domain *string
Layer *string
Q *string
Cursor *string
Lim int32
Type *string
}
func (q *Queries) ListEntities(ctx context.Context, arg ListEntitiesParams) ([]Entity, error) {
rows, err := q.db.Query(ctx, listEntities,
arg.State,
arg.Domain,
arg.Layer,
arg.Q,
arg.Cursor,
arg.Lim,
arg.Type,
)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Entity
for rows.Next() {
var i Entity
if err := rows.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listEntitiesCapped = `-- name: ListEntitiesCapped :many
SELECT e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at FROM entities e ORDER BY e.slug LIMIT $1
`
func (q *Queries) ListEntitiesCapped(ctx context.Context, limit int32) ([]Entity, error) {
rows, err := q.db.Query(ctx, listEntitiesCapped, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Entity
for rows.Next() {
var i Entity
if err := rows.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const updateEntity = `-- name: UpdateEntity :one
UPDATE entities SET
name = COALESCE($1, name),
state = COALESCE($2, state),
attributes = COALESCE($3, attributes),
maintenance_until = CASE WHEN $4::bool
THEN $5 ELSE maintenance_until END,
version = version + 1,
updated_at = now()
WHERE id = $6 AND version = $7
RETURNING id, slug, type, name, state, attributes, maintenance_until, version, created_at, updated_at
`
type UpdateEntityParams struct {
Name *string
State *string
Attributes []byte
SetMaintenance bool
MaintenanceUntil *time.Time
ID uuid.UUID
Version int32
}
func (q *Queries) UpdateEntity(ctx context.Context, arg UpdateEntityParams) (Entity, error) {
row := q.db.QueryRow(ctx, updateEntity,
arg.Name,
arg.State,
arg.Attributes,
arg.SetMaintenance,
arg.MaintenanceUntil,
arg.ID,
arg.Version,
)
var i Entity
err := row.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}

View File

@@ -0,0 +1,329 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
package sqlcgen
import (
"time"
"github.com/google/uuid"
)
type AgentActivity struct {
ID int64
Ts time.Time
AgentID uuid.UUID
SessionID *string
ActivityType string
ToolName *string
EntityID *uuid.UUID
InputSummary *string
OutputSummary *string
DurationMs *int32
TokenCount *int32
Success *bool
CorrelationID *string
}
type Approval struct {
EntityID uuid.UUID
SubjectEntityID *uuid.UUID
Action string
RiskClass string
Kind string
Payload []byte
Status string
TokenHash *string
ExpiresAt time.Time
DecidedAt *time.Time
DecidedBy *uuid.UUID
CreatedAt time.Time
}
type ApprovalRule struct {
ID uuid.UUID
EntityType *string
Action string
RiskClass string
AutonomyLevel string
ScopeEntity *uuid.UUID
Version int32
UpdatedAt time.Time
}
type AuditLog struct {
ID int64
Ts time.Time
ActorType string
ActorID *uuid.UUID
Action string
EntityID *uuid.UUID
Method *string
Path *string
StatusCode *int32
Detail []byte
SourceIp *string
CorrelationID *string
}
type AutonomySetting struct {
Key string
Value string
Version int32
UpdatedAt time.Time
}
type CheckDef struct {
EntityID uuid.UUID
TargetID *uuid.UUID
TargetType *string
Kind string
Config []byte
IntervalS int32
TimeoutS int32
Zone *string
Enabled bool
UpdatedAt time.Time
}
type Classification struct {
EntityID uuid.UUID
SignalEntityID *uuid.UUID
TargetEntityID *uuid.UUID
Action string
RecommendedAction []byte
RiskClass string
Route string
BlastRadius []uuid.UUID
PatternConfidence *float32
SkillID *uuid.UUID
AutonomyCheck *string
Reasoning []byte
CorrelationID string
CreatedAt time.Time
}
type Entity struct {
ID uuid.UUID
Slug string
Type string
Name string
State *string
Attributes []byte
MaintenanceUntil *time.Time
Version int32
CreatedAt time.Time
UpdatedAt time.Time
}
type EntityStatus struct {
EntityID uuid.UUID
Health string
LastCheckAt *time.Time
Details []byte
UpdatedAt time.Time
}
type EntityType struct {
Name string
ParentType *string
IsAbstract bool
Domain string
Layer string
Description *string
LifecycleID *string
AttributeSchema []byte
SchemaVersion int32
Status string
CreatedAt time.Time
UpdatedAt time.Time
}
type Event struct {
ID int64
Ts time.Time
Type string
EntityID *uuid.UUID
Severity string
Source string
Data []byte
CorrelationID *string
}
type Execution struct {
EntityID uuid.UUID
ClassificationID *uuid.UUID
SignalEntityID *uuid.UUID
TargetEntityID *uuid.UUID
Action string
RiskClass string
ApprovalID *uuid.UUID
AgentID *uuid.UUID
SkillID *uuid.UUID
SkillVersion *int32
Status string
Result []byte
DurationMs *int32
Verified bool
CorrelationID string
StartedAt *time.Time
CompletedAt *time.Time
CreatedAt time.Time
}
type Feedback struct {
EntityID uuid.UUID
ExecutionID uuid.UUID
Outcome string
Observation *string
Lesson *string
UnexpectedSideEffects []string
Tags []string
CreatedAt time.Time
}
type IdempotencyKey struct {
Key string
Actor string
RequestHash string
ResponseCode *int32
ResponseBody []byte
CreatedAt time.Time
}
type Ledger struct {
Ts time.Time
ExecutionID uuid.UUID
TargetEntityID *uuid.UUID
Action string
RiskClass string
Status string
Verified bool
Route *string
Reasoning []byte
ApprovalStatus *string
DecidedBy *uuid.UUID
AgentID *uuid.UUID
CorrelationID string
}
type LifecycleDef struct {
ID string
States []string
DefaultState string
TerminalStates []string
Transitions []byte
CreatedAt time.Time
}
type MetricRollups1d struct {
Bucket interface{}
EntityID uuid.UUID
Metric string
AvgValue float64
MinValue interface{}
MaxValue interface{}
SampleCount int64
}
type MetricRollups1h struct {
Bucket interface{}
EntityID uuid.UUID
Metric string
AvgValue float64
MinValue interface{}
MaxValue interface{}
SampleCount int64
}
type MetricSample struct {
Ts time.Time
EntityID uuid.UUID
Metric string
Value float64
Tags []byte
}
type Pattern struct {
EntityID uuid.UUID
AppliesType string
Action string
Pattern string
Confidence float32
EvidenceCount int32
SuccessCount int32
FailureCount int32
Status string
Quarantined bool
Version int32
LastValidatedAt *time.Time
CreatedAt time.Time
}
type Relationship struct {
SourceID uuid.UUID
TargetID uuid.UUID
Type string
Attributes []byte
ValidFrom time.Time
ValidTo *time.Time
}
type RelationshipType struct {
Name string
Inverse *string
SourceType string
TargetType string
Cardinality string
Description *string
CreatedAt time.Time
}
type RiskClass struct {
Name string
Description *string
ApprovalRequired string
AutonomyAllowed bool
}
type SeedVersion struct {
File string
ContentHash string
AppliedAt time.Time
}
type Signal struct {
EntityID uuid.UUID
Kind string
Severity string
TargetEntityID *uuid.UUID
CheckID *uuid.UUID
Evidence *string
LikelyCause *string
State string
OccurrenceCount int32
FirstSeenAt time.Time
LastSeenAt time.Time
FlapCount int32
HoldDownUntil *time.Time
MuteUntil *time.Time
CreatedAt time.Time
UpdatedAt time.Time
}
type Skill struct {
EntityID uuid.UUID
Version int32
Name string
Procedure []byte
AppliesType *string
Action string
PatternIds []uuid.UUID
Status string
SuccessRate *float32
ChangedBy *uuid.UUID
ChangeReason *string
LastUsedAt *time.Time
CreatedAt time.Time
}

View File

@@ -0,0 +1,130 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
// source: ontology.sql
package sqlcgen
import (
"context"
)
const getLifecycleForType = `-- name: GetLifecycleForType :one
SELECT ld.id, ld.states, ld.default_state, ld.terminal_states, ld.transitions, ld.created_at FROM lifecycle_defs ld
JOIN entity_types et ON et.lifecycle_id = ld.id
WHERE et.name = $1
`
func (q *Queries) GetLifecycleForType(ctx context.Context, name string) (LifecycleDef, error) {
row := q.db.QueryRow(ctx, getLifecycleForType, name)
var i LifecycleDef
err := row.Scan(
&i.ID,
&i.States,
&i.DefaultState,
&i.TerminalStates,
&i.Transitions,
&i.CreatedAt,
)
return i, err
}
const listEntityTypes = `-- name: ListEntityTypes :many
SELECT name, parent_type, is_abstract, domain, layer, description, lifecycle_id, attribute_schema, schema_version, status, created_at, updated_at FROM entity_types ORDER BY name
`
func (q *Queries) ListEntityTypes(ctx context.Context) ([]EntityType, error) {
rows, err := q.db.Query(ctx, listEntityTypes)
if err != nil {
return nil, err
}
defer rows.Close()
var items []EntityType
for rows.Next() {
var i EntityType
if err := rows.Scan(
&i.Name,
&i.ParentType,
&i.IsAbstract,
&i.Domain,
&i.Layer,
&i.Description,
&i.LifecycleID,
&i.AttributeSchema,
&i.SchemaVersion,
&i.Status,
&i.CreatedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listLifecycleDefs = `-- name: ListLifecycleDefs :many
SELECT id, states, default_state, terminal_states, transitions, created_at FROM lifecycle_defs ORDER BY id
`
func (q *Queries) ListLifecycleDefs(ctx context.Context) ([]LifecycleDef, error) {
rows, err := q.db.Query(ctx, listLifecycleDefs)
if err != nil {
return nil, err
}
defer rows.Close()
var items []LifecycleDef
for rows.Next() {
var i LifecycleDef
if err := rows.Scan(
&i.ID,
&i.States,
&i.DefaultState,
&i.TerminalStates,
&i.Transitions,
&i.CreatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listRelationshipTypes = `-- name: ListRelationshipTypes :many
SELECT name, inverse, source_type, target_type, cardinality, description, created_at FROM relationship_types ORDER BY name
`
func (q *Queries) ListRelationshipTypes(ctx context.Context) ([]RelationshipType, error) {
rows, err := q.db.Query(ctx, listRelationshipTypes)
if err != nil {
return nil, err
}
defer rows.Close()
var items []RelationshipType
for rows.Next() {
var i RelationshipType
if err := rows.Scan(
&i.Name,
&i.Inverse,
&i.SourceType,
&i.TargetType,
&i.Cardinality,
&i.Description,
&i.CreatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,165 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
// source: relationships.sql
package sqlcgen
import (
"context"
"time"
"github.com/google/uuid"
)
const endCurrentRelationship = `-- name: EndCurrentRelationship :execrows
UPDATE relationships SET valid_to = now()
WHERE source_id = $1 AND target_id = $2 AND type = $3 AND valid_to IS NULL
`
type EndCurrentRelationshipParams struct {
SourceID uuid.UUID
TargetID uuid.UUID
Type string
}
func (q *Queries) EndCurrentRelationship(ctx context.Context, arg EndCurrentRelationshipParams) (int64, error) {
result, err := q.db.Exec(ctx, endCurrentRelationship, arg.SourceID, arg.TargetID, arg.Type)
if err != nil {
return 0, err
}
return result.RowsAffected(), nil
}
const listEntityRelations = `-- name: ListEntityRelations :many
SELECT se.slug AS source_slug, te.slug AS target_slug, r.type, r.attributes,
r.valid_from, r.valid_to
FROM relationships r
JOIN entities se ON se.id = r.source_id
JOIN entities te ON te.id = r.target_id
WHERE r.valid_to IS NULL
AND (($1::text IN ('out','both') AND r.source_id = $2)
OR ($1::text IN ('in','both') AND r.target_id = $2))
AND ($3::text IS NULL OR r.type = $3)
ORDER BY r.type, se.slug, te.slug
`
type ListEntityRelationsParams struct {
Direction string
ID uuid.UUID
RelType *string
}
type ListEntityRelationsRow struct {
SourceSlug string
TargetSlug string
Type string
Attributes []byte
ValidFrom time.Time
ValidTo *time.Time
}
func (q *Queries) ListEntityRelations(ctx context.Context, arg ListEntityRelationsParams) ([]ListEntityRelationsRow, error) {
rows, err := q.db.Query(ctx, listEntityRelations, arg.Direction, arg.ID, arg.RelType)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListEntityRelationsRow
for rows.Next() {
var i ListEntityRelationsRow
if err := rows.Scan(
&i.SourceSlug,
&i.TargetSlug,
&i.Type,
&i.Attributes,
&i.ValidFrom,
&i.ValidTo,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listGraphEdges = `-- name: ListGraphEdges :many
SELECT se.slug AS source_slug, te.slug AS target_slug, r.type, r.attributes,
r.valid_from, r.valid_to
FROM relationships r
JOIN entities se ON se.id = r.source_id
JOIN entities te ON te.id = r.target_id
WHERE r.valid_to IS NULL
AND r.source_id = ANY($1::uuid[])
AND r.target_id = ANY($1::uuid[])
AND ($2::text[] IS NULL OR r.type = ANY($2::text[]))
ORDER BY r.type, se.slug, te.slug
`
type ListGraphEdgesParams struct {
Ids []uuid.UUID
RelTypes []string
}
type ListGraphEdgesRow struct {
SourceSlug string
TargetSlug string
Type string
Attributes []byte
ValidFrom time.Time
ValidTo *time.Time
}
func (q *Queries) ListGraphEdges(ctx context.Context, arg ListGraphEdgesParams) ([]ListGraphEdgesRow, error) {
rows, err := q.db.Query(ctx, listGraphEdges, arg.Ids, arg.RelTypes)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ListGraphEdgesRow
for rows.Next() {
var i ListGraphEdgesRow
if err := rows.Scan(
&i.SourceSlug,
&i.TargetSlug,
&i.Type,
&i.Attributes,
&i.ValidFrom,
&i.ValidTo,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const upsertCurrentRelationship = `-- name: UpsertCurrentRelationship :exec
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from, valid_to)
VALUES ($1, $2, $3, $4, now(), NULL)
ON CONFLICT (source_id, target_id, type) WHERE valid_to IS NULL
DO UPDATE SET attributes = EXCLUDED.attributes
`
type UpsertCurrentRelationshipParams struct {
SourceID uuid.UUID
TargetID uuid.UUID
Type string
Attributes []byte
}
func (q *Queries) UpsertCurrentRelationship(ctx context.Context, arg UpsertCurrentRelationshipParams) error {
_, err := q.db.Exec(ctx, upsertCurrentRelationship,
arg.SourceID,
arg.TargetID,
arg.Type,
arg.Attributes,
)
return err
}

143
internal/db/typetree.go Normal file
View File

@@ -0,0 +1,143 @@
package db
import (
"context"
"fmt"
"github.com/dtoro/oikos/internal/ontology"
"github.com/jackc/pgx/v5"
)
// LoadTypeTree loads the ontology meta-schema from the DB for validation.
// Called within the same transaction as an ingest so it sees just-ingested
// types.
func LoadTypeTree(ctx context.Context, tx pgx.Tx) (*ontology.TypeTree, error) {
t := &ontology.TypeTree{
Types: map[string]ontology.TypeInfo{},
RelTypes: map[string]ontology.RelTypeInfo{},
Lifecycles: map[string]ontology.LifecycleInfo{},
}
rows, err := tx.Query(ctx,
`SELECT name, COALESCE(parent_type,''), is_abstract, COALESCE(lifecycle_id,'')
FROM entity_types`)
if err != nil {
return nil, fmt.Errorf("load entity_types: %w", err)
}
for rows.Next() {
var name string
var info ontology.TypeInfo
if err := rows.Scan(&name, &info.Parent, &info.IsAbstract, &info.LifecycleID); err != nil {
rows.Close()
return nil, err
}
t.Types[name] = info
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
rows, err = tx.Query(ctx,
`SELECT name, source_type, target_type, cardinality FROM relationship_types`)
if err != nil {
return nil, fmt.Errorf("load relationship_types: %w", err)
}
for rows.Next() {
var name string
var info ontology.RelTypeInfo
if err := rows.Scan(&name, &info.SourceType, &info.TargetType, &info.Cardinality); err != nil {
rows.Close()
return nil, err
}
t.RelTypes[name] = info
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
rows, err = tx.Query(ctx,
`SELECT id, states, default_state FROM lifecycle_defs`)
if err != nil {
return nil, fmt.Errorf("load lifecycle_defs: %w", err)
}
for rows.Next() {
var id, def string
var states []string
if err := rows.Scan(&id, &states, &def); err != nil {
rows.Close()
return nil, err
}
set := make(map[string]bool, len(states))
for _, s := range states {
set[s] = true
}
t.Lifecycles[id] = ontology.LifecycleInfo{States: set, DefaultState: def}
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
return t, nil
}
// ValidateCardinality checks all CURRENT edges against their relationship
// type's declared cardinality. Runs inside the ingest transaction so a
// violating seed rolls back atomically.
func ValidateCardinality(ctx context.Context, tx pgx.Tx) error {
// one-to-one / many-to-one: a source may have at most one outgoing
// current edge of the type.
rows, err := tx.Query(ctx, `
SELECT r.type, se.slug, count(*)
FROM relationships r
JOIN relationship_types rt ON rt.name = r.type
JOIN entities se ON se.id = r.source_id
WHERE r.valid_to IS NULL AND rt.cardinality IN ('one-to-one','many-to-one')
GROUP BY r.type, se.slug HAVING count(*) > 1`)
if err != nil {
return err
}
violations, err := collectViolations(rows, "source")
if err != nil {
return err
}
// one-to-one / one-to-many: a target may have at most one incoming
// current edge of the type.
rows, err = tx.Query(ctx, `
SELECT r.type, te.slug, count(*)
FROM relationships r
JOIN relationship_types rt ON rt.name = r.type
JOIN entities te ON te.id = r.target_id
WHERE r.valid_to IS NULL AND rt.cardinality IN ('one-to-one','one-to-many')
GROUP BY r.type, te.slug HAVING count(*) > 1`)
if err != nil {
return err
}
v2, err := collectViolations(rows, "target")
if err != nil {
return err
}
violations = append(violations, v2...)
if len(violations) > 0 {
return fmt.Errorf("cardinality violations: %v", violations)
}
return nil
}
func collectViolations(rows pgx.Rows, side string) ([]string, error) {
defer rows.Close()
var out []string
for rows.Next() {
var relType, slug string
var n int
if err := rows.Scan(&relType, &slug, &n); err != nil {
return nil, err
}
out = append(out, fmt.Sprintf("%s %s=%s (%d edges)", relType, side, slug, n))
}
return out, rows.Err()
}

110
internal/domain/approval.go Normal file
View File

@@ -0,0 +1,110 @@
package domain
import "time"
// Approval is a short-TTL signed grant for a gated action.
type Approval struct {
EntityID UUID
SubjectEntityID UUID
Action string
RiskClass string
Kind string
Payload map[string]any
Status string
TokenHash string
ExpiresAt time.Time
DecidedAt *time.Time
DecidedBy UUID
CreatedAt time.Time
}
// Approval statuses.
const (
ApprovalPending = "pending"
ApprovalApproved = "approved"
ApprovalDenied = "denied"
ApprovalExpired = "expired"
ApprovalRevoked = "revoked"
)
// Approval kinds.
const (
ApprovalKindExecution = "execution"
ApprovalKindPolicyChange = "policy-change"
ApprovalKindPatternActivation = "pattern-activation"
)
// CheckDef defines a probe (R3-7: probes as data, not code).
type CheckDef struct {
EntityID UUID
TargetID UUID
TargetType string
Kind string
Config map[string]any
IntervalS int
TimeoutS int
Zone string
Enabled bool
UpdatedAt time.Time
}
// Check kinds.
const (
CheckHTTP = "http"
CheckTCP = "tcp"
CheckDisk = "disk"
CheckCertExpiry = "cert-expiry"
CheckDrift = "drift"
CheckSSHScript = "ssh-script"
)
// EntityStatus is the current health of an entity (R3-6: replaces state_snapshots).
type EntityStatus struct {
EntityID UUID
Health string
LastCheckAt *time.Time
Details map[string]any
UpdatedAt time.Time
}
// Health values.
const (
HealthHealthy = "healthy"
HealthDegraded = "degraded"
HealthDown = "down"
HealthUnknown = "unknown"
)
// RiskClass is the four-level safety model.
type RiskClass struct {
Name string
Description string
ApprovalRequired string
AutonomyAllowed bool
}
// Risk class names.
const (
RiskReadOnly = "read_only"
RiskReversibleLow = "reversible_low"
RiskConfigMutation = "config_mutation"
RiskDestructive = "destructive"
)
// ApprovalRule maps (entity_type, action) → risk_class + autonomy.
type ApprovalRule struct {
ID UUID
EntityType string
Action string
RiskClass string
AutonomyLevel string
ScopeEntity UUID
Version int
}
// Autonomy levels.
const (
AutonomyAuto = "auto"
AutonomyEscalate = "escalate"
AutonomyNever = "never"
)

76
internal/domain/entity.go Normal file
View File

@@ -0,0 +1,76 @@
package domain
import (
"time"
)
// Entity is the core graph node — every object in the OS is an entity.
// Typed tables (signals, executions, etc.) reference entities(id) for
// indexed querying; graph edges live in the relationships table.
type Entity struct {
ID UUID
Slug string
Type string
Name string
State string
Attributes map[string]any
MaintenanceUntil *time.Time
Version int
CreatedAt time.Time
UpdatedAt time.Time
}
// EntityType is the meta-schema entry defining what entities can exist.
type EntityType struct {
Name string
ParentType string
IsAbstract bool
Domain string
Layer string
Description string
LifecycleID string
AttributeSchema map[string]any
SchemaVersion int
Status string
}
// RelationshipType defines a typed edge between entity types.
type RelationshipType struct {
Name string
Inverse string
SourceType string
TargetType string
Cardinality string
Description string
}
// LifecycleDef is the state machine for an entity type.
type LifecycleDef struct {
ID string
States []string
DefaultState string
TerminalStates []string
Transitions map[string]map[string]TransitionReq
}
// TransitionReq holds the named preconditions for a lifecycle transition.
type TransitionReq struct {
Requires []string `json:"requires"`
}
// Relationship is a typed edge between two entities.
type Relationship struct {
SourceID UUID
TargetID UUID
Type string
Attributes map[string]any
ValidFrom time.Time
ValidTo *time.Time
}
// UUID is a type alias for UUID values. Using string for simplicity;
// the DB layer uses pgx's UUID type. Conversion happens at the boundary.
type UUID string
// IsNil returns true if the UUID is empty.
func (u UUID) IsNil() bool { return u == "" }

22
internal/domain/errors.go Normal file
View File

@@ -0,0 +1,22 @@
package domain
import "errors"
// Sentinel errors. Used throughout the codebase for typed error handling.
// The API middleware maps these to HTTP status codes (SG11).
var (
ErrNotFound = errors.New("entity not found")
ErrInvalidTransition = errors.New("invalid lifecycle transition")
ErrApprovalRequired = errors.New("operator approval required")
ErrAutonomyBlocked = errors.New("autonomy policy blocks this action")
ErrConflict = errors.New("concurrent modification conflict")
ErrCircuitOpen = errors.New("circuit breaker open for target")
ErrAbstractType = errors.New("cannot instantiate abstract entity type")
ErrInvalidEdge = errors.New("relationship endpoint type mismatch")
ErrCardinality = errors.New("relationship cardinality violation")
ErrSeedHashMismatch = errors.New("seed content hash mismatch")
ErrAlreadyExists = errors.New("entity already exists")
ErrQuarantined = errors.New("pattern is quarantined")
ErrSkillDeprecated = errors.New("skill is deprecated")
ErrInvalidInput = errors.New("invalid input")
)

View File

@@ -0,0 +1,64 @@
package domain
import "time"
// Classification persists every autonomous decision the classifier makes (SA5).
// This is the audit trail for "why did the OS auto-act / escalate?"
type Classification struct {
EntityID UUID
SignalEntityID UUID
TargetEntityID UUID
Action string
RecommendedAction map[string]any
RiskClass string
Route string
BlastRadius []UUID
PatternConfidence float64
SkillID UUID
AutonomyCheck string
Reasoning map[string]any
CorrelationID string
CreatedAt time.Time
}
// Classification routes.
const (
RouteAutoAct = "auto-act"
RouteEscalate = "escalate"
RouteHold = "hold"
)
// Execution is a detailed record of one action the OS performed.
type Execution struct {
EntityID UUID
ClassificationID UUID
SignalEntityID UUID
TargetEntityID UUID
Action string
RiskClass string
ApprovalID UUID
AgentID UUID
SkillID UUID
SkillVersion int
Status string
Result map[string]any
DurationMs int
Verified bool
CorrelationID string
StartedAt *time.Time
CompletedAt *time.Time
CreatedAt time.Time
}
// Execution lifecycle states.
const (
ExecProposed = "proposed"
ExecApproved = "approved"
ExecExecuting = "executing"
ExecVerified = "verified"
ExecFailed = "failed"
ExecTimedOut = "timed-out"
ExecRolledBack = "rolled-back"
ExecCancelled = "cancelled"
ExecExpired = "expired"
)

View File

@@ -0,0 +1,76 @@
package domain
import "time"
// Feedback records what was learned from an execution.
type Feedback struct {
EntityID UUID
ExecutionID UUID
Outcome string
Observation string
Lesson string
UnexpectedSideEffects []string
Tags []string
CreatedAt time.Time
}
// Feedback outcomes.
const (
OutcomeSuccess = "success"
OutcomeFailure = "failure"
OutcomePartial = "partial"
OutcomeUnexpected = "unexpected"
)
// Pattern is a generalized rule extracted from accumulated feedback.
type Pattern struct {
EntityID UUID
AppliesType string
Action string
Pattern string
Confidence float64
EvidenceCount int
SuccessCount int
FailureCount int
Status string
Quarantined bool
Version int
LastValidatedAt *time.Time
CreatedAt time.Time
}
// Pattern lifecycle states.
const (
PatternHypothesized = "hypothesized"
PatternValidated = "validated"
PatternActive = "active"
PatternDeprecated = "deprecated"
PatternInvalidated = "invalidated"
)
// Skill is a codified procedure refined through validated patterns.
type Skill struct {
EntityID UUID
Version int
Name string
Procedure map[string]any
AppliesType string
Action string
PatternIDs []UUID
Status string
SuccessRate float64
ChangedBy UUID
ChangeReason string
LastUsedAt *time.Time
CreatedAt time.Time
}
// Skill lifecycle states.
const (
SkillDrafted = "drafted"
SkillTested = "tested"
SkillActive = "active"
SkillRefined = "refined"
SkillDeprecated = "deprecated"
SkillFailed = "failed"
)

65
internal/domain/signal.go Normal file
View File

@@ -0,0 +1,65 @@
package domain
import "time"
// Signal is an attention record — something the lab noticed that needs
// attention and possibly action. Dual entity: has an entities row + a
// signals table row for indexed querying.
type Signal struct {
EntityID UUID
Kind string
Severity string
TargetEntityID UUID
CheckID UUID
Evidence string
LikelyCause string
State string
OccurrenceCount int
FirstSeenAt time.Time
LastSeenAt time.Time
FlapCount int
HoldDownUntil *time.Time
MuteUntil *time.Time
CreatedAt time.Time
UpdatedAt time.Time
}
// Signal lifecycle states (see lifecycle_defs in seeds/ontology.yaml).
const (
SignalRaised = "raised"
SignalAcknowledged = "acknowledged"
SignalActing = "acting"
SignalMuted = "muted"
SignalResolved = "resolved"
SignalFailed = "failed"
)
// Signal severities.
const (
SeverityInfo = "info"
SeverityWarning = "warning"
SeverityCritical = "critical"
)
// ValidSignalTransitions defines legal state transitions.
var ValidSignalTransitions = map[string][]string{
SignalRaised: {SignalAcknowledged, SignalMuted, SignalResolved},
SignalAcknowledged: {SignalActing, SignalResolved, SignalMuted},
SignalActing: {SignalResolved, SignalRaised, SignalFailed},
SignalFailed: {SignalAcknowledged},
SignalMuted: {SignalRaised},
}
// CanTransition returns true if from→to is a legal signal state transition.
func (s *Signal) CanTransition(to string) bool {
allowed, ok := ValidSignalTransitions[s.State]
if !ok {
return false
}
for _, a := range allowed {
if a == to {
return true
}
}
return false
}

View File

@@ -0,0 +1,299 @@
package httpapi
// API integration tests. Guarded by OIKOS_TEST_DATABASE_URL (see
// internal/db/integration_test.go); run via `make test-db-all` or plain
// `go test` with the env var set.
import (
"context"
"encoding/json"
"fmt"
"math/rand"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/jackc/pgx/v5"
)
func newTestHandler(t *testing.T, cfg config.Config) http.Handler {
t.Helper()
baseURL := os.Getenv("OIKOS_TEST_DATABASE_URL")
if baseURL == "" {
t.Skip("OIKOS_TEST_DATABASE_URL not set — skipping integration test")
}
ctx := context.Background()
admin, err := pgx.Connect(ctx, baseURL)
if err != nil {
t.Fatalf("connect admin: %v", err)
}
dbName := fmt.Sprintf("oikos_api_test_%08x", rand.Int63())
if _, err := admin.Exec(ctx, "CREATE DATABASE "+dbName); err != nil {
admin.Close(ctx)
t.Fatalf("create test db: %v", err)
}
admin.Close(ctx)
qi := strings.Index(baseURL, "?")
base, params := baseURL, ""
if qi >= 0 {
base, params = baseURL[:qi], baseURL[qi:]
}
testURL := base[:strings.LastIndex(base, "/")+1] + dbName + params
pool, err := db.New(ctx, testURL)
if err != nil {
t.Fatalf("connect test db: %v", err)
}
t.Cleanup(func() {
pool.Close()
admin, err := pgx.Connect(ctx, baseURL)
if err == nil {
admin.Exec(ctx, "DROP DATABASE IF EXISTS "+dbName+" WITH (FORCE)")
admin.Close(ctx)
}
})
// Handler context governs the SSE listener goroutine. Cancel it before
// the pool closes (cleanups run LIFO, so registering it after the
// pool-close cleanup makes it run first) — otherwise the listener holds
// a pooled connection and pool.Close() deadlocks.
handlerCtx, cancelHandler := context.WithCancel(context.Background())
t.Cleanup(cancelHandler)
if err := pool.Migrate(ctx); err != nil {
t.Fatalf("migrate: %v", err)
}
for _, f := range []string{"ontology.yaml", "inventory.yaml", "policy.yaml"} {
content, err := os.ReadFile("../../seeds/" + f)
if err != nil {
t.Fatalf("read seed %s: %v", f, err)
}
name := f
err = pool.SeedIngest(ctx, name, content,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
var err error
switch name {
case "ontology.yaml":
_, err = db.IngestOntologySeed(ctx, tx, data)
case "inventory.yaml":
_, err = db.IngestInventorySeed(ctx, tx, data)
case "policy.yaml":
_, err = db.IngestPolicySeed(ctx, tx, data)
}
return err
})
if err != nil {
t.Fatalf("ingest %s: %v", f, err)
}
}
return NewHandler(handlerCtx, pool, cfg)
}
func get(t *testing.T, h http.Handler, path string, headers map[string]string) (*httptest.ResponseRecorder, map[string]any) {
t.Helper()
req := httptest.NewRequest("GET", path, nil)
for k, v := range headers {
req.Header.Set(k, v)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
var body map[string]any
json.Unmarshal(rec.Body.Bytes(), &body)
return rec, body
}
func devConfig() config.Config {
c := config.Default()
c.APIEnv = "dev" // no tokens → dev-open auth
return c
}
func TestAPIEndToEnd(t *testing.T) {
h := newTestHandler(t, devConfig())
t.Run("healthz", func(t *testing.T) {
rec, body := get(t, h, "/healthz", nil)
if rec.Code != 200 || body["status"] != "ok" {
t.Fatalf("healthz = %d %v", rec.Code, body)
}
})
t.Run("list entities filtered by type", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/entities?type=service", nil)
if rec.Code != 200 {
t.Fatalf("status %d: %v", rec.Code, body)
}
items := body["items"].([]any)
if len(items) < 20 {
t.Errorf("expected 20+ services, got %d", len(items))
}
})
t.Run("type filter includes descendants via hierarchy", func(t *testing.T) {
// machine is abstract; proxmox-host/workstation/standalone-server descend from it
rec, body := get(t, h, "/api/v1/entities?type=machine", nil)
if rec.Code != 200 {
t.Fatalf("status %d: %v", rec.Code, body)
}
items := body["items"].([]any)
if len(items) < 5 {
t.Errorf("expected 5+ machines (hubris, strong, vps, 2 workstations), got %d", len(items))
}
})
t.Run("pagination", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/entities?limit=10", nil)
if rec.Code != 200 {
t.Fatalf("status %d", rec.Code)
}
if len(body["items"].([]any)) != 10 {
t.Fatalf("limit=10 returned %d", len(body["items"].([]any)))
}
cursor, _ := body["next_cursor"].(string)
if cursor == "" {
t.Fatal("expected next_cursor")
}
rec2, body2 := get(t, h, "/api/v1/entities?limit=10&cursor="+cursor, nil)
if rec2.Code != 200 {
t.Fatalf("page 2 status %d", rec2.Code)
}
first := body2["items"].([]any)[0].(map[string]any)["slug"].(string)
if first <= cursor {
t.Errorf("page 2 first slug %q not after cursor %q", first, cursor)
}
})
t.Run("get entity by slug with etag", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/entities/host:hubris", nil)
if rec.Code != 200 || body["slug"] != "host:hubris" {
t.Fatalf("get by slug = %d %v", rec.Code, body["slug"])
}
if rec.Header().Get("ETag") == "" {
t.Error("missing ETag header")
}
// and by UUID
id := body["id"].(string)
rec2, body2 := get(t, h, "/api/v1/entities/"+id, nil)
if rec2.Code != 200 || body2["slug"] != "host:hubris" {
t.Errorf("get by uuid = %d %v", rec2.Code, body2["slug"])
}
})
t.Run("unknown entity is 404 problem+json", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/entities/host:nonexistent", nil)
if rec.Code != 404 {
t.Fatalf("status %d", rec.Code)
}
if ct := rec.Header().Get("Content-Type"); ct != "application/problem+json" {
t.Errorf("content-type %q", ct)
}
if body["title"] != "not found" {
t.Errorf("problem title %v", body["title"])
}
})
t.Run("relations", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/entities/host:hubris/relations?rel_type=hosts&direction=out", nil)
if rec.Code != 200 {
t.Fatalf("status %d", rec.Code)
}
items := body["items"].([]any)
if len(items) < 10 {
t.Errorf("hubris hosts %d guests, want 10+", len(items))
}
})
t.Run("blast radius", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/entities/lxc:caddy/blast-radius?depth=2", nil)
if rec.Code != 200 {
t.Fatalf("status %d", rec.Code)
}
if len(body["items"].([]any)) < 2 {
t.Errorf("blast radius too small: %v", body["items"])
}
})
t.Run("graph", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/graph?root=service:paperless&depth=2", nil)
if rec.Code != 200 {
t.Fatalf("status %d", rec.Code)
}
if len(body["nodes"].([]any)) < 2 || len(body["edges"].([]any)) < 1 {
t.Errorf("graph too small: %d nodes %d edges",
len(body["nodes"].([]any)), len(body["edges"].([]any)))
}
})
t.Run("ontology", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/ontology", nil)
if rec.Code != 200 {
t.Fatalf("status %d", rec.Code)
}
if len(body["entity_types"].([]any)) != 59 {
t.Errorf("entity_types = %d, want 59", len(body["entity_types"].([]any)))
}
if len(body["lifecycles"].([]any)) != 6 {
t.Errorf("lifecycles = %d, want 6", len(body["lifecycles"].([]any)))
}
})
t.Run("signals empty list", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/signals", nil)
if rec.Code != 200 {
t.Fatalf("status %d: %v", rec.Code, body)
}
})
t.Run("export returns real seeds", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/export", nil)
if rec.Code != 200 {
t.Fatalf("status %d", rec.Code)
}
if len(body["inventory"].(string)) < 1000 {
t.Errorf("inventory export suspiciously small")
}
})
t.Run("unimplemented endpoint is 501 problem+json", func(t *testing.T) {
rec, body := get(t, h, "/api/v1/patterns", nil)
if rec.Code != 501 {
t.Fatalf("status %d, want 501", rec.Code)
}
if body["title"] != "not implemented" {
t.Errorf("problem title %v", body["title"])
}
})
}
func TestAPIBearerAuth(t *testing.T) {
cfg := devConfig()
cfg.APIToken = "test-token-123"
h := newTestHandler(t, cfg)
// healthz stays open
rec, _ := get(t, h, "/healthz", nil)
if rec.Code != 200 {
t.Errorf("healthz with auth enabled = %d, want 200", rec.Code)
}
// API requires the token
rec, body := get(t, h, "/api/v1/entities", nil)
if rec.Code != 401 {
t.Errorf("no token = %d, want 401 (%v)", rec.Code, body)
}
rec, _ = get(t, h, "/api/v1/entities", map[string]string{"Authorization": "Bearer wrong"})
if rec.Code != 401 {
t.Errorf("wrong token = %d, want 401", rec.Code)
}
rec, _ = get(t, h, "/api/v1/entities", map[string]string{"Authorization": "Bearer test-token-123"})
if rec.Code != 200 {
t.Errorf("valid token = %d, want 200", rec.Code)
}
}

File diff suppressed because it is too large Load Diff

1084
internal/httpapi/impl.go Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,178 @@
package httpapi
// Integration tests for the Phase 2 mutation surface: entity create/patch
// with optimistic concurrency, idempotency, lifecycle-transition validation,
// and the audit/event side effects. Guarded by OIKOS_TEST_DATABASE_URL.
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
)
// do issues a JSON request and returns the recorder + decoded body.
func do(t *testing.T, h http.Handler, method, path string, body any, headers map[string]string) (*httptest.ResponseRecorder, map[string]any) {
t.Helper()
var rdr *bytes.Reader
if body != nil {
b, _ := json.Marshal(body)
rdr = bytes.NewReader(b)
} else {
rdr = bytes.NewReader(nil)
}
req := httptest.NewRequest(method, path, rdr)
req.Header.Set("Content-Type", "application/json")
for k, v := range headers {
req.Header.Set(k, v)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
var decoded map[string]any
json.Unmarshal(rec.Body.Bytes(), &decoded)
return rec, decoded
}
func TestEntityCreateAndPatch(t *testing.T) {
h := newTestHandler(t, devConfig())
// ── create ──────────────────────────────────────────────────
rec, body := do(t, h, "POST", "/api/v1/entities", map[string]any{
"slug": "service:test-widget",
"type": "service",
"name": "test-widget",
"attributes": map[string]any{"port": 9999},
}, nil)
if rec.Code != 201 {
t.Fatalf("create status %d: %v", rec.Code, body)
}
if body["slug"] != "service:test-widget" {
t.Fatalf("created slug = %v", body["slug"])
}
// default lifecycle state applied
if body["state"] != "active" {
t.Errorf("default state = %v, want active", body["state"])
}
etag := rec.Header().Get("ETag")
if etag == "" {
t.Error("missing ETag on create")
}
version := int(body["version"].(float64))
// ── duplicate slug → 409 ────────────────────────────────────
rec, _ = do(t, h, "POST", "/api/v1/entities", map[string]any{
"slug": "service:test-widget", "type": "service", "name": "dup",
}, nil)
if rec.Code != 409 {
t.Errorf("duplicate slug status = %d, want 409", rec.Code)
}
// ── abstract type → 422 ─────────────────────────────────────
rec, _ = do(t, h, "POST", "/api/v1/entities", map[string]any{
"slug": "machine:ghost", "type": "machine", "name": "ghost",
}, nil)
if rec.Code != 422 {
t.Errorf("abstract type status = %d, want 422", rec.Code)
}
// ── patch without If-Match → 400 ────────────────────────────
rec, _ = do(t, h, "PATCH", "/api/v1/entities/service:test-widget",
map[string]any{"name": "renamed"}, nil)
if rec.Code != 400 {
t.Errorf("patch w/o If-Match = %d, want 400", rec.Code)
}
// ── patch with stale If-Match → 409 ─────────────────────────
rec, _ = do(t, h, "PATCH", "/api/v1/entities/service:test-widget",
map[string]any{"name": "renamed"}, map[string]string{"If-Match": `"999"`})
if rec.Code != 409 {
t.Errorf("stale If-Match = %d, want 409", rec.Code)
}
// ── valid attribute patch → 200, version bumps ──────────────
rec, body = do(t, h, "PATCH", "/api/v1/entities/service:test-widget",
map[string]any{"name": "renamed"}, map[string]string{"If-Match": itoaQ(version)})
if rec.Code != 200 {
t.Fatalf("patch status %d: %v", rec.Code, body)
}
if body["name"] != "renamed" || int(body["version"].(float64)) != version+1 {
t.Errorf("patch result: name=%v version=%v", body["name"], body["version"])
}
version++
// ── valid lifecycle transition active→deprecated → 200 ──────
// (this is the regression guard for the transitions-parsing 500 bug)
rec, body = do(t, h, "PATCH", "/api/v1/entities/service:test-widget",
map[string]any{"state": "deprecated"}, map[string]string{"If-Match": itoaQ(version)})
if rec.Code != 200 {
t.Fatalf("valid transition status %d: %v", rec.Code, body)
}
if body["state"] != "deprecated" {
t.Errorf("state = %v, want deprecated", body["state"])
}
version++
// ── invalid lifecycle transition deprecated→provisioning → 409 ──
rec, _ = do(t, h, "PATCH", "/api/v1/entities/service:test-widget",
map[string]any{"state": "provisioning"}, map[string]string{"If-Match": itoaQ(version)})
if rec.Code != 409 {
t.Errorf("invalid transition status = %d, want 409", rec.Code)
}
}
func TestEntityCreateIdempotency(t *testing.T) {
h := newTestHandler(t, devConfig())
payload := map[string]any{"slug": "service:idem", "type": "service", "name": "idem"}
key := map[string]string{"Idempotency-Key": "abc-123"}
rec1, body1 := do(t, h, "POST", "/api/v1/entities", payload, key)
if rec1.Code != 201 {
t.Fatalf("first create %d: %v", rec1.Code, body1)
}
// replay same key + body → same response, not a duplicate-slug 409
rec2, body2 := do(t, h, "POST", "/api/v1/entities", payload, key)
if rec2.Code != 201 {
t.Fatalf("idempotent replay = %d, want 201: %v", rec2.Code, body2)
}
if body1["id"] != body2["id"] {
t.Errorf("replay returned different entity: %v vs %v", body1["id"], body2["id"])
}
// same key, different body → 409 conflict
rec3, _ := do(t, h, "POST", "/api/v1/entities",
map[string]any{"slug": "service:idem2", "type": "service", "name": "idem2"}, key)
if rec3.Code != 409 {
t.Errorf("key reuse w/ different body = %d, want 409", rec3.Code)
}
}
func TestMutationEmitsEventAndAudit(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, _ := do(t, h, "POST", "/api/v1/entities",
map[string]any{"slug": "service:evt", "type": "service", "name": "evt"}, nil)
if rec.Code != 201 {
t.Fatalf("create failed: %d", rec.Code)
}
// event stream recorded the creation
_, body := do(t, h, "GET", "/api/v1/events?type=entity.created", nil, nil)
items, _ := body["items"].([]any)
if len(items) == 0 {
t.Fatal("no entity.created event recorded")
}
// audit trail recorded the create (operator-visible)
_, abody := do(t, h, "GET", "/api/v1/audit?action=create", nil, nil)
aitems, _ := abody["items"].([]any)
if len(aitems) == 0 {
t.Fatal("no create audit entry recorded")
}
}
// itoaQ formats an int as a quoted ETag value.
func itoaQ(v int) string {
b, _ := json.Marshal(v)
return `"` + string(b) + `"`
}

1760
internal/httpapi/phase3.go Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,111 @@
package httpapi
// Integration tests for Phase 3 endpoints: checks, classifications,
// executions, approvals, patterns, skills, policy, and knowledge search.
// Guarded by OIKOS_TEST_DATABASE_URL; run via `make test-db` or with env set.
import (
"encoding/json"
"testing"
)
func TestPhase3ListChecks(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, body := get(t, h, "/api/v1/checks", nil)
if rec.Code != 200 {
t.Fatalf("list checks status %d: %v", rec.Code, body)
}
}
func TestPhase3ListApprovals(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, body := get(t, h, "/api/v1/approvals", nil)
if rec.Code != 200 {
t.Fatalf("list approvals status %d: %v", rec.Code, body)
}
}
func TestPhase3ListRiskClasses(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, body := get(t, h, "/api/v1/risk-classes", nil)
if rec.Code != 200 {
t.Fatalf("list risk classes status %d: %v", rec.Code, body)
}
items, _ := body["items"].([]any)
if len(items) < 2 {
t.Errorf("expected 2+ risk classes, got %d", len(items))
}
}
func TestPhase3ListAutonomySettings(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, body := get(t, h, "/api/v1/autonomy-settings", nil)
if rec.Code != 200 {
t.Fatalf("get autonomy settings status %d: %v", rec.Code, body)
}
_ = body
}
func TestPhase3ListPatterns(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, body := get(t, h, "/api/v1/patterns", nil)
if rec.Code != 200 {
t.Fatalf("list patterns status %d: %v", rec.Code, body)
}
}
func TestPhase3ListSkills(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, body := get(t, h, "/api/v1/skills", nil)
if rec.Code != 200 {
t.Fatalf("list skills status %d: %v", rec.Code, body)
}
}
func TestPhase3ListExecutions(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, body := get(t, h, "/api/v1/executions", nil)
if rec.Code != 200 {
t.Fatalf("list executions status %d: %v", rec.Code, body)
}
}
func TestPhase3ListClassifications(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, body := get(t, h, "/api/v1/classifications", nil)
if rec.Code != 200 {
t.Fatalf("list classifications status %d: %v", rec.Code, body)
}
}
func TestPhase3QueryMetrics(t *testing.T) {
h := newTestHandler(t, devConfig())
rec, body := get(t, h, "/api/v1/metrics?entity_id=00000000-0000-0000-0000-000000000001&metric=health", nil)
if rec.Code != 200 {
t.Fatalf("query metrics status %d: %v", rec.Code, body)
}
}
func TestPhase3JSONRoundTrip(t *testing.T) {
sigData := map[string]any{
"id": "sig-123", "kind": "down", "severity": "critical",
"state": "raised", "occurrence_count": 1,
}
b, _ := json.Marshal(sigData)
var back map[string]any
if err := json.Unmarshal(b, &back); err != nil {
t.Fatalf("signal round-trip: %v", err)
}
if back["kind"] != "down" {
t.Errorf("kind = %v, want down", back["kind"])
}
}

View File

@@ -0,0 +1,71 @@
package httpapi
import (
"encoding/json"
"errors"
"log/slog"
"net/http"
"github.com/dtoro/oikos/internal/domain"
"github.com/dtoro/oikos/internal/httpapi/gen"
)
// errNotImplemented marks endpoints stubbed for later phases.
var errNotImplemented = errors.New("not implemented yet")
// statusFor maps domain sentinel errors to HTTP status codes (plan SG11).
func statusFor(err error) (status int, title string) {
switch {
case errors.Is(err, domain.ErrNotFound):
return http.StatusNotFound, "not found"
case errors.Is(err, domain.ErrInvalidTransition):
return http.StatusConflict, "invalid lifecycle transition"
case errors.Is(err, domain.ErrConflict), errors.Is(err, domain.ErrAlreadyExists):
return http.StatusConflict, "conflict"
case errors.Is(err, domain.ErrCardinality):
return http.StatusConflict, "relationship cardinality violation"
case errors.Is(err, domain.ErrAbstractType), errors.Is(err, domain.ErrInvalidEdge):
return http.StatusUnprocessableEntity, "ontology validation failed"
case errors.Is(err, domain.ErrInvalidInput):
return http.StatusBadRequest, "invalid input"
case errors.Is(err, domain.ErrApprovalRequired):
return http.StatusForbidden, "operator approval required"
case errors.Is(err, domain.ErrAutonomyBlocked):
return http.StatusForbidden, "autonomy policy blocks this action"
case errors.Is(err, domain.ErrCircuitOpen):
return http.StatusServiceUnavailable, "circuit breaker open"
case errors.Is(err, errNotImplemented):
return http.StatusNotImplemented, "not implemented"
default:
return http.StatusInternalServerError, "internal error"
}
}
// writeProblem writes an RFC 9457 problem+json response.
func writeProblem(w http.ResponseWriter, r *http.Request, status int, title, detail string) {
instance := r.URL.Path
p := gen.Problem{
Status: status,
Title: title,
Instance: &instance,
}
if detail != "" {
p.Detail = &detail
}
w.Header().Set("Content-Type", "application/problem+json")
w.WriteHeader(status)
json.NewEncoder(w).Encode(p)
}
// writeProblemFromErr maps an error to a problem+json response. Internal
// error details are logged server-side, never leaked to clients.
func writeProblemFromErr(w http.ResponseWriter, r *http.Request, err error) {
status, title := statusFor(err)
detail := ""
if status != http.StatusInternalServerError {
detail = err.Error()
} else {
slog.Error("internal error", "method", r.Method, "path", r.URL.Path, "error", err)
}
writeProblem(w, r, status, title, detail)
}

472
internal/httpapi/server.go Normal file
View File

@@ -0,0 +1,472 @@
// Package httpapi implements the Oikos REST API. The contract is
// api/openapi.yaml (contract-first, ADR-0004); handlers implement the
// oapi-codegen strict-server interface in gen/. Errors map to RFC 9457
// problem+json via domain sentinels.
package httpapi
import (
"context"
"crypto/rsa"
"crypto/subtle"
"crypto/tls"
"encoding/base64"
"encoding/json"
"fmt"
"log/slog"
"math/big"
"net/http"
"strings"
"sync"
"time"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/httpapi/gen"
mcphandler "github.com/dtoro/oikos/internal/mcp"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/golang-jwt/jwt/v5"
)
// contextKey for storing actor identity in request context.
type contextKey string
const actorKey contextKey = "oikos_actor"
// actor holds the resolved identity of the API caller.
type actor struct {
Type string // "operator", "agent", "system"
Label string // human-readable label
ID string // OIDC sub or static token identifier
TokenType string // "static" or "oidc"
}
// Server implements gen.StrictServerInterface over the DB layer.
type Server struct {
pool *db.Pool
cfg config.Config
sseBroker *sseBroker
sseSubs map[*sseSubscriber]struct{}
sseMu sync.Mutex
}
// NewHandler builds the full HTTP handler: /healthz (unauthenticated,
// SG18) + the OpenAPI surface under /api/v1 behind bearer auth.
//
// ctx governs the lifetime of the background SSE listener goroutine, which
// holds a dedicated pooled connection for LISTEN. Callers MUST cancel ctx
// before closing the pool — otherwise the held connection never releases
// and pool.Close() deadlocks.
func NewHandler(ctx context.Context, pool *db.Pool, cfg config.Config) http.Handler {
s := &Server{
pool: pool,
cfg: cfg,
sseBroker: newSSEBroker(10000),
sseSubs: make(map[*sseSubscriber]struct{}),
}
// Start background SSE listener, tied to ctx for clean shutdown.
go s.sseListener(ctx)
r := chi.NewRouter()
r.Use(middleware.Recoverer)
r.Use(middleware.RequestID)
r.Use(requestLogger)
// Liveness — no auth, no audit (plan SG18). Not exposed via Caddy.
r.Get("/healthz", func(w http.ResponseWriter, req *http.Request) {
ctx, cancel := context.WithTimeout(req.Context(), 2*time.Second)
defer cancel()
if err := pool.Ping(ctx); err != nil {
writeProblem(w, req, http.StatusServiceUnavailable, "database unreachable", "")
return
}
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{"status":"ok"}`))
})
strict := gen.NewStrictHandlerWithOptions(s, nil, gen.StrictHTTPServerOptions{
RequestErrorHandlerFunc: func(w http.ResponseWriter, req *http.Request, err error) {
writeProblem(w, req, http.StatusBadRequest, "bad request", err.Error())
},
ResponseErrorHandlerFunc: writeProblemFromErr,
})
gen.HandlerWithOptions(strict, gen.ChiServerOptions{
BaseURL: "/api/v1",
BaseRouter: r,
Middlewares: []gen.MiddlewareFunc{combinedAuth(cfg)},
ErrorHandlerFunc: func(w http.ResponseWriter, req *http.Request, err error) {
writeProblem(w, req, http.StatusBadRequest, "bad request", err.Error())
},
})
// SSE stream: override the generated /events/stream route with a raw
// flushing handler (registered AFTER HandlerWithOptions so chi's last
// registration wins). The strict-server path can't Flush() per event;
// this one uses the real ResponseWriter for real-time delivery. It
// inherits the router's base middleware and applies auth via With().
r.With(combinedAuth(cfg)).Get("/api/v1/events/stream", s.serveSSE)
// Mount MCP at /mcp (plan R3-10)
r.With(combinedAuth(cfg)).Handle("/mcp", mcphandler.NewHandler(pool, cfg.MCPBearerToken))
return r
}
// combinedAuth tries OIDC JWT validation first (if configured), falls back to
// static bearer token validation, and opens the gate in dev mode when no
// credentials are configured.
func combinedAuth(cfg config.Config) func(http.Handler) http.Handler {
hasOIDC := cfg.OIDCIssuer != "" && cfg.OIDCClientID != ""
hasStatic := cfg.APIToken != "" || cfg.MCPBearerToken != ""
// Cache JWKS for OIDC
var jwksURL string
var jwksCache []jwtVerificationKey
var jwksMu sync.RWMutex
if hasOIDC {
// Fetch JWKS URI from OIDC discovery
jwksURL = discoverJWKSURI(cfg.OIDCIssuer)
if jwksURL != "" {
keys, err := fetchJWKS(jwksURL)
if err != nil {
slog.Warn("oidc initial jwks fetch failed, will retry on demand", "error", err)
} else {
jwksCache = keys
}
}
}
var staticTokens [][]byte
if cfg.APIToken != "" {
staticTokens = append(staticTokens, []byte(cfg.APIToken))
}
if cfg.MCPBearerToken != "" {
staticTokens = append(staticTokens, []byte(cfg.MCPBearerToken))
}
devOpen := cfg.APIEnv == "dev" && !hasStatic && !hasOIDC
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if devOpen {
ctx := context.WithValue(r.Context(), actorKey, actor{
Type: "system",
Label: "dev:anonymous",
ID: "dev",
TokenType: "none",
})
next.ServeHTTP(w, r.WithContext(ctx))
return
}
auth := r.Header.Get("Authorization")
raw, ok := strings.CutPrefix(auth, "Bearer ")
if !ok || raw == "" {
writeProblem(w, r, http.StatusUnauthorized, "unauthorized",
"missing bearer token")
return
}
// Try OIDC first if configured
if hasOIDC {
jwksMu.RLock()
keys := jwksCache
jwksMu.RUnlock()
// If cache is empty, try to refresh
if len(keys) == 0 && jwksURL != "" {
if freshKeys, err := fetchJWKS(jwksURL); err == nil {
jwksMu.Lock()
jwksCache = freshKeys
keys = freshKeys
jwksMu.Unlock()
}
}
if act, err := validateOIDCToken(raw, cfg, keys); err == nil {
ctx := context.WithValue(r.Context(), actorKey, act)
next.ServeHTTP(w, r.WithContext(ctx))
return
} else {
slog.Debug("oidc validation failed", "error", err)
}
}
// Fall back to static tokens
if hasStatic {
for _, t := range staticTokens {
if subtle.ConstantTimeCompare([]byte(raw), t) == 1 {
label := "operator:api"
if cfg.MCPBearerToken != "" && subtle.ConstantTimeCompare([]byte(raw), []byte(cfg.MCPBearerToken)) == 1 {
label = "agent:mcp"
}
ctx := context.WithValue(r.Context(), actorKey, actor{
Type: label[:strings.IndexByte(label, ':')],
Label: label,
ID: raw[:8] + "...",
TokenType: "static",
})
next.ServeHTTP(w, r.WithContext(ctx))
return
}
}
}
writeProblem(w, r, http.StatusUnauthorized, "unauthorized",
"invalid or expired bearer token")
})
}
}
// jwtVerificationKey holds a parsed RSA public key or HMAC secret for JWT
// verification, identified by its key ID (kid).
type jwtVerificationKey struct {
Kid string
Alg string
Key any // *rsa.PublicKey or []byte for HMAC
IsHMAC bool
}
// discoverJWKSURI fetches the OIDC discovery document and extracts the
// jwks_uri field.
func discoverJWKSURI(issuerURL string) string {
discURL := strings.TrimRight(issuerURL, "/") + "/.well-known/openid-configuration"
client := &http.Client{Timeout: 10 * time.Second, Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: false},
}}
resp, err := client.Get(discURL)
if err != nil {
slog.Warn("oidc discovery failed", "url", discURL, "error", err)
return ""
}
defer resp.Body.Close()
var disc struct {
JWKSURI string `json:"jwks_uri"`
}
if err := json.NewDecoder(resp.Body).Decode(&disc); err != nil {
slog.Warn("oidc discovery decode failed", "error", err)
return ""
}
if disc.JWKSURI == "" {
slog.Warn("oidc discovery missing jwks_uri")
return ""
}
return disc.JWKSURI
}
// fetchJWKS retrieves the JWK set from a URL and returns the parsed keys.
func fetchJWKS(jwksURL string) ([]jwtVerificationKey, error) {
client := &http.Client{Timeout: 10 * time.Second, Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: false},
}}
resp, err := client.Get(jwksURL)
if err != nil {
return nil, fmt.Errorf("fetch jwks: %w", err)
}
defer resp.Body.Close()
var jwks struct {
Keys []json.RawMessage `json:"keys"`
}
if err := json.NewDecoder(resp.Body).Decode(&jwks); err != nil {
return nil, fmt.Errorf("decode jwks: %w", err)
}
var keys []jwtVerificationKey
for _, raw := range jwks.Keys {
var header struct {
Kty string `json:"kty"`
Kid string `json:"kid,omitempty"`
Alg string `json:"alg,omitempty"`
Use string `json:"use,omitempty"`
}
if err := json.Unmarshal(raw, &header); err != nil {
continue
}
// Skip keys not intended for signature verification
if header.Use != "" && header.Use != "sig" {
continue
}
var key jwtVerificationKey
key.Kid = header.Kid
key.Alg = header.Alg
switch header.Kty {
case "RSA":
var rsaKey struct {
N string `json:"n"`
E string `json:"e"`
}
if err := json.Unmarshal(raw, &rsaKey); err != nil {
continue
}
pubKey, err := parseRSAPublicKey(rsaKey.N, rsaKey.E)
if err != nil {
slog.Debug("oidc parse rsa key failed", "kid", header.Kid, "error", err)
continue
}
key.Key = pubKey
case "oct":
// HMAC keys not expected for OIDC but handle gracefully
key.IsHMAC = true
default:
continue
}
if key.Key != nil || key.IsHMAC {
keys = append(keys, key)
}
}
if len(keys) == 0 {
return nil, fmt.Errorf("no usable keys in jwks")
}
return keys, nil
}
// parseRSAPublicKey decodes a base64url-encoded RSA modulus and exponent into
// an *rsa.PublicKey.
func parseRSAPublicKey(nB64, eB64 string) (any, error) {
// Decode base64url modulus
nBytes, err := base64.RawURLEncoding.DecodeString(nB64)
if err != nil {
return nil, fmt.Errorf("decode modulus: %w", err)
}
// Decode base64url exponent
eBytes, err := base64.RawURLEncoding.DecodeString(eB64)
if err != nil {
return nil, fmt.Errorf("decode exponent: %w", err)
}
// Build RSA public key
n := new(big.Int).SetBytes(nBytes)
e := 0
for _, b := range eBytes {
e = (e << 8) | int(b)
}
return &rsa.PublicKey{N: n, E: e}, nil
}
// validateOIDCToken parses and validates a JWT Bearer token against the OIDC
// configuration. Returns the resolved actor on success.
func validateOIDCToken(rawToken string, cfg config.Config, keys []jwtVerificationKey) (actor, error) {
keyFunc := func(token *jwt.Token) (any, error) {
kid, ok := token.Header["kid"].(string)
if !ok {
return nil, fmt.Errorf("no kid in token header")
}
// Find matching key
for _, k := range keys {
// If kid is empty in JWK, try algorithm match
if k.Kid == kid || (k.Kid == "" && k.Alg == token.Header["alg"]) {
return k.Key, nil
}
}
// Fall back to any RSA key if no kid match (some providers don't set kid)
if !ok {
for _, k := range keys {
if k.Key != nil {
return k.Key, nil
}
}
}
return nil, fmt.Errorf("no matching key for kid: %s", kid)
}
token, err := jwt.Parse(rawToken, keyFunc,
jwt.WithIssuer(cfg.OIDCIssuer),
jwt.WithAudience(cfg.OIDCClientID),
jwt.WithValidMethods([]string{"RS256", "RS384", "RS512", "ES256", "ES384", "ES512"}),
)
if err != nil {
return actor{}, fmt.Errorf("jwt validation: %w", err)
}
claims, ok := token.Claims.(jwt.MapClaims)
if !ok {
return actor{}, fmt.Errorf("invalid claims")
}
sub, _ := claims.GetSubject()
if sub == "" {
// Try the Azure/Entra ID oid claim fallback
if oid, ok := claims["oid"].(string); ok {
sub = oid
}
}
preferredUsername, _ := claims["preferred_username"].(string)
email, _ := claims["email"].(string)
label := sub
if preferredUsername != "" {
label = preferredUsername
} else if email != "" {
label = email
}
return actor{
Type: "operator",
Label: label,
ID: sub,
TokenType: "oidc",
}, nil
}
// GetActor extracts the actor identity from the context. Returns nil if not
// set (should not happen for authenticated routes).
func GetActor(ctx context.Context) *actor {
a, ok := ctx.Value(actorKey).(actor)
if !ok {
return nil
}
return &a
}
// requestLogger logs one line per request with method, path, status,
// duration, and the chi request id.
func requestLogger(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
ww := middleware.NewWrapResponseWriter(w, r.ProtoMajor)
next.ServeHTTP(ww, r)
slog.Info("http",
"method", r.Method,
"path", r.URL.Path,
"status", ww.Status(),
"duration_ms", time.Since(start).Milliseconds(),
"request_id", middleware.GetReqID(r.Context()),
)
})
}
// ListenAndServe runs the API server with graceful shutdown on ctx cancel
// (SG4): stop accepting, drain in-flight for up to 30s, then exit.
func ListenAndServe(ctx context.Context, pool *db.Pool, cfg config.Config) error {
srv := &http.Server{
Addr: cfg.APIListen,
Handler: NewHandler(ctx, pool, cfg),
ReadHeaderTimeout: 10 * time.Second,
}
errCh := make(chan error, 1)
go func() {
slog.Info("api listening", "addr", cfg.APIListen)
errCh <- srv.ListenAndServe()
}()
select {
case err := <-errCh:
return err
case <-ctx.Done():
slog.Info("api shutting down")
shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return srv.Shutdown(shutdownCtx)
}
}

339
internal/httpapi/sse.go Normal file
View File

@@ -0,0 +1,339 @@
package httpapi
import (
"container/list"
"context"
"encoding/json"
"fmt"
"log/slog"
"net/http"
"strconv"
"sync"
"time"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/dtoro/oikos/internal/httpapi/gen"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
// SSE broker is a keep-last-event-id in-memory buffer used for subscriber
// fan-out. The database NOTIFY is the primary delivery mechanism; this buffer
// just supports the Last-Event-ID replay on connect.
type sseBroker struct {
mu sync.Mutex
buf *list.List // list of sqlcgen.Event
cache map[int64]*list.Element // id → list element for O(1) lookup
cap int
lastID int64
}
func newSSEBroker(capacity int) *sseBroker {
return &sseBroker{
buf: list.New(),
cache: make(map[int64]*list.Element),
cap: capacity,
}
}
func (b *sseBroker) push(ev sqlcgen.Event) {
b.mu.Lock()
defer b.mu.Unlock()
// Evict oldest if at capacity
for b.buf.Len() >= b.cap && b.buf.Len() > 0 {
front := b.buf.Front()
b.cache[front.Value.(sqlcgen.Event).ID] = nil // don't delete, just nil
b.buf.Remove(front)
}
elem := b.buf.PushBack(ev)
b.cache[ev.ID] = elem
if ev.ID > b.lastID {
b.lastID = ev.ID
}
}
func (b *sseBroker) after(id int64) []sqlcgen.Event {
b.mu.Lock()
defer b.mu.Unlock()
if id >= b.lastID {
return nil
}
// Walk from the front to find the first element after id
var events []sqlcgen.Event
for e := b.buf.Front(); e != nil; e = e.Next() {
ev := e.Value.(sqlcgen.Event)
if ev.ID > id {
events = append(events, ev)
}
}
return events
}
func (b *sseBroker) latestID() int64 {
b.mu.Lock()
defer b.mu.Unlock()
return b.lastID
}
// notifyPayload is the JSON payload from the pg_notify trigger (migration 008).
type notifyPayload struct {
ID int64 `json:"id"`
Ts string `json:"ts"`
Type string `json:"type"`
EntityID *string `json:"entity_id"`
Severity string `json:"severity"`
Source string `json:"source"`
CorrelationID *string `json:"correlation_id"`
}
// sseSubscriber holds the channels and cancel func for one SSE client.
type sseSubscriber struct {
ch chan sqlcgen.Event
done chan struct{}
cancel context.CancelFunc
}
// sseListener runs in a background goroutine: it opens a dedicated pgx
// connection, LISTENs on oikos_events, and fans out each notification to
// all live subscribers. Runs until ctx is cancelled.
func (s *Server) sseListener(ctx context.Context) {
poolConn, err := s.pool.Acquire(ctx)
if err != nil {
slog.Error("sse listener acquire failed", "error", err)
return
}
defer poolConn.Release()
conn := poolConn.Conn()
if _, err := conn.Exec(ctx, "LISTEN oikos_events"); err != nil {
slog.Error("sse listener listen failed", "error", err)
return
}
slog.Info("sse listener started on oikos_events")
defer slog.Info("sse listener stopped")
for {
nt, err := conn.WaitForNotification(ctx)
if err != nil {
if ctx.Err() != nil {
return // normal shutdown
}
slog.Error("sse listener notification error", "error", err)
// Reconnect on error after a brief delay
select {
case <-time.After(5 * time.Second):
case <-ctx.Done():
return
}
// Re-acquire connection
poolConn.Release()
var reconnErr error
poolConn, reconnErr = s.pool.Acquire(ctx)
if reconnErr != nil {
slog.Error("sse listener reconnect failed", "error", reconnErr)
return
}
conn = poolConn.Conn()
if _, err := conn.Exec(ctx, "LISTEN oikos_events"); err != nil {
slog.Error("sse listener re-listen failed", "error", err)
return
}
continue
}
var p notifyPayload
if err := json.Unmarshal([]byte(nt.Payload), &p); err != nil {
slog.Error("sse listener unmarshal failed", "error", err)
continue
}
// Fetch full event from DB
q := sqlcgen.New(s.pool)
events, err := q.ListEventsAfter(ctx, sqlcgen.ListEventsAfterParams{
ID: p.ID - 1,
Limit: 1,
})
if err != nil || len(events) == 0 {
slog.Warn("sse listener event fetch failed", "id", p.ID, "error", err)
continue
}
ev := events[0]
// Push to broker
s.sseBroker.push(ev)
// Fan out to subscribers (non-blocking send)
s.sseMu.Lock()
for sub := range s.sseSubs {
select {
case sub.ch <- ev:
default:
// Subscriber too slow — drop event for them
// (they'll reconnect via Last-Event-ID)
}
}
s.sseMu.Unlock()
}
}
// sqlcEventToGen converts a DB event row to the canonical wire shape so the
// SSE `data:` payload matches GET /events (snake_case keys, decoded data
// object) rather than leaking Go field names and base64-encoded JSONB.
func sqlcEventToGen(ev sqlcgen.Event) gen.Event {
out := gen.Event{
Id: int(ev.ID),
Ts: ev.Ts,
Type: ev.Type,
Severity: gen.EventSeverity(ev.Severity),
Source: ev.Source,
CorrelationId: ev.CorrelationID,
}
if ev.EntityID != nil {
s := ev.EntityID.String()
out.EntityId = &s
}
if len(ev.Data) > 0 {
var data map[string]any
if json.Unmarshal(ev.Data, &data) == nil && len(data) > 0 {
out.Data = &data
}
}
return out
}
// writeSSE writes a single Event as an SSE message. Returns false if the
// write failed (client disconnected). flusher may be nil (io.Pipe path,
// which has no separate flush step).
func writeSSE(w ioWriter, flusher http.Flusher, ev sqlcgen.Event) bool {
data, err := json.Marshal(sqlcEventToGen(ev))
if err != nil {
return true // skip un-serializable events
}
_, err = fmt.Fprintf(w, "id: %d\nevent: %s\ndata: %s\n\n", ev.ID, ev.Type, data)
if err != nil {
return false
}
if flusher != nil {
flusher.Flush()
}
return true
}
// ioWriter is an interface satisfied by both http.ResponseWriter and
// io.StringWriter, letting writeSSE work with the writer directly.
type ioWriter interface {
Write([]byte) (int, error)
}
// serveSSE is the streaming handler for GET /api/v1/events/stream, registered
// directly on the chi router in NewHandler so it supersedes the generated
// route. Using the raw http.ResponseWriter lets us Flush() after every event
// (real-time delivery); the generated strict-server path can only hand back
// an io.Reader that io.Copy drains without flushing (chunk-buffered).
//
// Protocol: https://html.spec.whatwg.org/multipage/server-sent-events.html
// 1. If Last-Event-ID is present, replay buffered events (broker, then DB).
// 2. Subscribe and forward events fanned out from pg_notify.
// 3. Colon-comment heartbeat every 15s.
// 4. Unsubscribe on client disconnect (request context cancels).
func (s *Server) serveSSE(w http.ResponseWriter, r *http.Request) {
flusher, ok := w.(http.Flusher)
if !ok {
writeProblem(w, r, http.StatusInternalServerError, "internal error", "streaming not supported")
return
}
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
w.Header().Set("X-Accel-Buffering", "no") // disable proxy buffering
w.WriteHeader(http.StatusOK)
flusher.Flush()
ctx, cancel := context.WithCancel(r.Context())
defer cancel()
sub := &sseSubscriber{
ch: make(chan sqlcgen.Event, 64),
done: make(chan struct{}),
cancel: cancel,
}
s.sseMu.Lock()
s.sseSubs[sub] = struct{}{}
s.sseMu.Unlock()
defer func() {
s.sseMu.Lock()
delete(s.sseSubs, sub)
s.sseMu.Unlock()
close(sub.done)
}()
// ── 1. Replay on Last-Event-ID ────────────────────────────────
if lastID := r.Header.Get("Last-Event-ID"); lastID != "" {
if id, err := strconv.ParseInt(lastID, 10, 64); err == nil {
// The in-memory broker only holds recent events; if it doesn't
// cover the whole gap, fall back to the DB for a complete replay.
events := s.sseBroker.after(id)
complete := len(events) > 0 && events[len(events)-1].ID == s.sseBroker.latestID()
if complete {
for _, ev := range events {
if !writeSSE(w, flusher, ev) {
return
}
}
} else {
q := sqlcgen.New(s.pool)
dbEvents, err := q.ListEventsAfter(ctx, sqlcgen.ListEventsAfterParams{ID: id, Limit: 5000})
if err != nil {
slog.Error("sse db replay failed", "error", err)
} else {
for _, ev := range dbEvents {
if !writeSSE(w, flusher, ev) {
return
}
}
}
}
}
}
// ── 2. Subscribe and forward ──────────────────────────────────
heartbeat := time.NewTicker(15 * time.Second)
defer heartbeat.Stop()
for {
select {
case <-ctx.Done():
return
case ev, ok := <-sub.ch:
if !ok {
return
}
if !writeSSE(w, flusher, ev) {
return
}
case <-heartbeat.C:
if _, err := fmt.Fprintf(w, ": heartbeat\n\n"); err != nil {
return
}
flusher.Flush()
}
}
}
// StreamEvents satisfies the generated ServerInterface, but the SSE route is
// served by the raw serveSSE handler registered in NewHandler (which wins
// over this generated route). If this method is ever reached, routing has
// regressed — fail loudly rather than silently chunk-buffering.
func (s *Server) StreamEvents(ctx context.Context, req gen.StreamEventsRequestObject) (gen.StreamEventsResponseObject, error) {
return nil, fmt.Errorf("%w: SSE must be served by the raw handler", errNotImplemented)
}
// Ensure pgxpool is imported — used via Acquire.
var _ = &pgxpool.Pool{}
var _ = pgx.ErrNoRows

View File

@@ -0,0 +1,89 @@
package httpapi
// Real-connection SSE test. httptest.NewRecorder buffers and never flushes,
// so this uses httptest.NewServer + a streaming client to verify that:
// - the raw serveSSE handler (not the generated 501 stub) serves the route,
// - an event created *after* the client connects is delivered in real time
// (i.e. flushed before the connection closes),
// - the SSE `data:` payload is the canonical gen.Event shape.
// Guarded by OIKOS_TEST_DATABASE_URL.
import (
"bufio"
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
func TestSSEStreamRealtimeDelivery(t *testing.T) {
srv := httptest.NewServer(newTestHandler(t, devConfig()))
defer srv.Close()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
// Connect to the stream.
req, _ := http.NewRequestWithContext(ctx, "GET", srv.URL+"/api/v1/events/stream", nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("connect stream: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
t.Fatalf("stream status = %d, want 200", resp.StatusCode)
}
if ct := resp.Header.Get("Content-Type"); !strings.HasPrefix(ct, "text/event-stream") {
t.Fatalf("content-type = %q, want text/event-stream", ct)
}
// Read SSE frames in a goroutine.
dataCh := make(chan map[string]any, 4)
go func() {
sc := bufio.NewScanner(resp.Body)
for sc.Scan() {
line := sc.Text()
if strings.HasPrefix(line, "data: ") {
var m map[string]any
if json.Unmarshal([]byte(strings.TrimPrefix(line, "data: ")), &m) == nil {
dataCh <- m
}
}
}
}()
// Give the subscriber a moment to register, then trigger an event by
// POSTing to the SAME live server (same DB → NOTIFY the listener sees).
time.Sleep(300 * time.Millisecond)
payload, _ := json.Marshal(map[string]any{"slug": "service:sse-rt", "type": "service", "name": "sse-rt"})
cResp, err := http.Post(srv.URL+"/api/v1/entities", "application/json", bytes.NewReader(payload))
if err != nil {
t.Fatalf("trigger create: %v", err)
}
cResp.Body.Close()
if cResp.StatusCode != 201 {
t.Fatalf("trigger create status = %d, want 201", cResp.StatusCode)
}
// The event must arrive in real time (well before the 10s ctx deadline),
// proving the handler flushes rather than buffering until close.
select {
case ev := <-dataCh:
if ev["type"] != "entity.created" {
t.Errorf("event type = %v, want entity.created", ev["type"])
}
// canonical shape: snake_case + decoded data object
if _, ok := ev["entity_id"]; !ok {
t.Errorf("missing snake_case entity_id: %v", ev)
}
if d, ok := ev["data"].(map[string]any); !ok || d["slug"] != "service:sse-rt" {
t.Errorf("data not a decoded object with slug: %v", ev["data"])
}
case <-time.After(3 * time.Second):
t.Fatal("SSE event not delivered within 3s (flushing broken?)")
}
}

View File

@@ -0,0 +1,6 @@
package httpapi
// Remaining stubs for endpoints that depend on tables not yet created
// (knowledge_entities, agent_activity). These are kept here because the
// phase3.go file already defines them; this file is deliberately empty.
// The stubs live in phase3.go as simple errNotImplemented returns.

View File

@@ -0,0 +1,169 @@
// Package learning implements the Oikos learning engine (Phase 3).
// Hourly pattern extraction: reads feedback past the watermark, groups by
// (applies_type, action), updates pattern counters with Wilson confidence,
// detects anomalies, and refines skills.
package learning
import (
"context"
"log/slog"
"math"
"time"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/google/uuid"
)
// Run starts the learning loop. Blocks until ctx is cancelled.
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
slog.Info("learning: starting", "interval", cfg.LearningInterval)
interval := cfg.LearningInterval
if interval <= 0 {
interval = 1 * time.Hour
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
watermark := time.Now().Add(-24 * time.Hour) // start from 24h ago
for {
select {
case <-ctx.Done():
slog.Info("learning: shutting down")
return
case <-ticker.C:
watermark = extractPatterns(ctx, pool, watermark)
}
}
}
// extractPatterns reads feedback past the watermark, groups by (type, action),
// updates pattern counters, and returns the new watermark.
func extractPatterns(ctx context.Context, pool *db.Pool, watermark time.Time) time.Time {
q := sqlcgen.New(pool)
feedback, err := q.GetFeedbackAfterWatermark(ctx, watermark)
if err != nil {
slog.Error("learning: get feedback", "error", err)
return watermark
}
if len(feedback) == 0 {
// Advance watermark to now so we don't re-scan
return time.Now()
}
// Group by (applies_type, action)
type groupKey struct {
Type string
Action string
}
groups := make(map[groupKey][]sqlcgen.GetFeedbackAfterWatermarkRow)
for _, f := range feedback {
key := groupKey{Type: f.AppliesType, Action: f.Action}
groups[key] = append(groups[key], f)
}
for key, items := range groups {
processGroup(ctx, pool, q, key.Type, key.Action, items)
}
// Update watermark to the latest feedback timestamp
newWatermark := watermark
for _, f := range feedback {
if f.CreatedAt.After(newWatermark) {
newWatermark = f.CreatedAt
}
}
return newWatermark
}
func processGroup(ctx context.Context, pool *db.Pool, q *sqlcgen.Queries,
appliesType, action string, items []sqlcgen.GetFeedbackAfterWatermarkRow) {
successCount := 0
failureCount := 0
for _, f := range items {
switch f.Outcome {
case "success":
successCount++
case "failure", "unexpected":
failureCount++
case "partial":
successCount++ // partial counts as half-success
}
}
total := successCount + failureCount
if total == 0 {
return
}
// Compute Wilson score lower bound
confidence := wilsonLowerBound(float64(successCount), float64(total), 0.95)
// Cap by sample size: nothing looks confident before 5 samples
confidence = math.Min(confidence, float64(total)/5.0)
// Get or create pattern
patternID, _ := uuid.NewV7()
patternSummary := action + " on " + appliesType
err := q.UpsertPattern(ctx, sqlcgen.UpsertPatternParams{
EntityID: patternID,
AppliesType: appliesType,
Action: action,
Pattern: patternSummary,
Confidence: float32(confidence),
EvidenceCount: int32(total),
SuccessCount: int32(successCount),
FailureCount: int32(failureCount),
})
if err != nil {
slog.Error("learning: upsert pattern", "error", err)
return
}
// Update pattern status based on confidence
pat, err := q.GetPattern(ctx, sqlcgen.GetPatternParams{
AppliesType: appliesType,
Action: action,
})
if err != nil {
return
}
if pat.EvidenceCount >= 5 && pat.Confidence >= 0.7 && !pat.Quarantined {
_ = q.UpdatePatternStatus(ctx, sqlcgen.UpdatePatternStatusParams{
EntityID: pat.EntityID,
Status: "validated",
})
slog.Info("learning: pattern validated",
"type", appliesType, "action", action,
"confidence", confidence, "samples", total)
}
// Anomaly check: >10 identical outcomes within 1h
if total > 10 {
_ = q.UpdatePatternQuarantine(ctx, sqlcgen.UpdatePatternQuarantineParams{
EntityID: pat.EntityID,
Quarantined: true,
})
slog.Warn("learning: pattern quarantined (anomaly burst)",
"type", appliesType, "action", action)
}
}
// wilsonLowerBound computes the Wilson score interval lower bound.
// Conservative estimate of success rate for small sample sizes.
func wilsonLowerBound(success, total, z float64) float64 {
if total == 0 {
return 0
}
p := success / total
z2 := z * z
denom := 1 + z2/total
center := (p + z2/(2*total)) / denom
sp := math.Sqrt((p*(1-p) + z2/(4*total)) / total) / denom
return math.Max(0, center-z*sp)
}

250
internal/mcp/server.go Normal file
View File

@@ -0,0 +1,250 @@
// Package mcp implements the Oikos MCP interface (plan R3-10).
// Uses the official MCP Go SDK with Streamable HTTP transport.
package mcp
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"net/http"
"time"
"github.com/dtoro/oikos/internal/db"
"github.com/google/jsonschema-go/jsonschema"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
// prop is one input-schema property (name → type + description).
type prop struct {
name, typ, desc string
}
// objSchema builds an "object" JSON Schema from a list of properties. The
// MCP SDK requires every tool to declare an object input schema so tools
// are self-describing to the agent; a nil schema panics at registration.
func objSchema(props ...prop) *jsonschema.Schema {
s := &jsonschema.Schema{Type: "object", Properties: map[string]*jsonschema.Schema{}}
for _, p := range props {
s.Properties[p.name] = &jsonschema.Schema{Type: p.typ, Description: p.desc}
}
return s
}
// NewHandler creates an http.Handler that serves the Oikos MCP server.
func NewHandler(pool *db.Pool, token string) http.Handler {
s := newServer(pool)
handler := mcp.NewStreamableHTTPHandler(func(r *http.Request) *mcp.Server {
if token != "" {
if r.Header.Get("Authorization") != "Bearer "+token {
return nil
}
}
return s
}, nil)
return handler
}
func newServer(pool *db.Pool) *mcp.Server {
s := mcp.NewServer(&mcp.Implementation{Name: "oikos", Version: "dev"}, &mcp.ServerOptions{
Logger: slog.Default(),
})
// All tools use the untyped handler (s.AddTool) for simplicity.
// Arguments are accessed via req.Parameters.Arguments.(map[string]any).
s.AddTool(&mcp.Tool{Name: "get_entity", Description: "Get an entity by slug or UUID",
InputSchema: objSchema(prop{"slug_or_id", "string", "Entity slug (e.g. host:hubris) or UUID"}),
}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
idOrSlug, _ := args["slug_or_id"].(string)
return queryEntity(ctx, pool, idOrSlug), nil
})
s.AddTool(&mcp.Tool{Name: "list_entities", Description: "List entities filtered by type, state, or search",
InputSchema: objSchema(
prop{"type", "string", "Filter by entity type"},
prop{"state", "string", "Filter by lifecycle state"},
prop{"q", "string", "Substring match on slug or name"},
prop{"limit", "integer", "Max rows (default 50)"}),
}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
limit := int(getFloat(args, "limit", 50))
return queryRows(ctx, pool, `
SELECT e.slug, e.type, e.name, e.state, e.version, e.created_at, e.updated_at
FROM entities e
WHERE ($1::text IS NULL OR e.type = $1)
AND ($2::text IS NULL OR e.state = $2)
AND ($3::text IS NULL OR e.slug ILIKE '%'||$3||'%' OR e.name ILIKE '%'||$3||'%')
ORDER BY e.slug LIMIT $4`,
nStr(args["type"]), nStr(args["state"]), nStr(args["q"]), limit), nil
})
s.AddTool(&mcp.Tool{Name: "get_relations", Description: "Get relationships for an entity",
InputSchema: objSchema(prop{"entity_id", "string", "Entity slug"}),
}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["entity_id"].(string)
return queryRows(ctx, pool, `
SELECT r.type, src.slug AS source, tgt.slug AS target
FROM relationships r
JOIN entities src ON src.id = r.source_id
JOIN entities tgt ON tgt.id = r.target_id
WHERE (src.slug = $1 OR tgt.slug = $1) AND r.valid_to IS NULL
ORDER BY r.type`, slug), nil
})
s.AddTool(&mcp.Tool{Name: "get_blast_radius", Description: "Find entities affected if this entity goes down",
InputSchema: objSchema(
prop{"entity_id", "string", "Entity slug"},
prop{"depth", "integer", "Traversal depth (default 3)"}),
}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
slug, _ := args["entity_id"].(string)
depth := int(getFloat(args, "depth", 3))
return queryRows(ctx, pool,
"SELECT slug, CAST(depth AS int) FROM blast_radius((SELECT id FROM entities WHERE slug = $1), $2)",
slug, depth), nil
})
s.AddTool(&mcp.Tool{Name: "get_health_summary", Description: "Current fleet health summary",
InputSchema: objSchema(),
}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
return queryRows(ctx, pool, `
SELECT e.slug, e.type, st.health, st.last_check_at
FROM entity_status st JOIN entities e ON e.id = st.entity_id
ORDER BY e.slug`), nil
})
s.AddTool(&mcp.Tool{Name: "get_audit_trail", Description: "Query the audit log",
InputSchema: objSchema(prop{"entity_id", "string", "Filter by affected entity UUID"}),
}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
return queryRows(ctx, pool, `
SELECT id, ts, actor_type, action, entity_id::text, method, path, correlation_id
FROM audit_log
WHERE ($1::text IS NULL OR entity_id::text = $1)
ORDER BY ts DESC LIMIT 50`, nStr(args["entity_id"])), nil
})
s.AddTool(&mcp.Tool{Name: "search_knowledge", Description: "Full-text search across documentation",
InputSchema: objSchema(prop{"query", "string", "Search terms"}),
}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
q, _ := args["query"].(string)
return queryRows(ctx, pool, `
SELECT id, title, LEFT(content, 500) AS preview
FROM knowledge_entities
WHERE title ILIKE '%'||$1||'%' OR content ILIKE '%'||$1||'%'
ORDER BY title LIMIT 20`, q), nil
})
s.AddTool(&mcp.Tool{Name: "query_metrics", Description: "Query time-series metrics",
InputSchema: objSchema(prop{"hours", "integer", "Look-back window in hours (default 24)"}),
}, func(ctx context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args := argsMap(req)
hours := int(getFloat(args, "hours", 24))
return queryRows(ctx, pool, `
SELECT time_bucket('1 hour', ts) AS bucket,
entity_id::text, metric,
ROUND(avg(value)::numeric, 2) AS avg,
ROUND(min(value)::numeric, 2) AS min,
ROUND(max(value)::numeric, 2) AS max
FROM metric_samples
WHERE ts > now() - make_interval(hours => $1)
GROUP BY bucket, entity_id, metric
ORDER BY bucket DESC LIMIT 100`, hours), nil
})
return s
}
// ─── Helpers ──────────────────────────────────────────────────────────
func argsMap(req *mcp.CallToolRequest) map[string]any {
if req == nil || len(req.Params.Arguments) == 0 {
return nil
}
var m map[string]any
json.Unmarshal(req.Params.Arguments, &m)
return m
}
func getFloat(m map[string]any, key string, def float64) float64 {
if m == nil {
return def
}
switch v := m[key].(type) {
case float64:
return v
case int:
return float64(v)
}
return def
}
func nStr(v any) any {
if v == nil {
return nil
}
s, _ := v.(string)
if s == "" {
return nil
}
return s
}
func textResult(s string) *mcp.CallToolResult {
return &mcp.CallToolResult{
Content: []mcp.Content{&mcp.TextContent{Text: s}},
}
}
func queryEntity(ctx context.Context, pool *db.Pool, idOrSlug string) *mcp.CallToolResult {
var id uuid.UUID
if u, err := uuid.Parse(idOrSlug); err == nil {
id = u
} else {
pool.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", idOrSlug).Scan(&id)
}
if id == uuid.Nil {
return textResult(fmt.Sprintf("entity not found: %s", idOrSlug))
}
return queryRows(ctx, pool, `
SELECT slug, type, name, state, attributes,
maintenance_until::text, version, created_at, updated_at
FROM entities WHERE id = $1`, id)
}
func queryRows(ctx context.Context, pool *db.Pool, query string, args ...any) *mcp.CallToolResult {
rows, err := pool.Query(ctx, query, args...)
if err != nil {
return textResult(fmt.Sprintf("error: %v", err))
}
defer rows.Close()
cols := rows.FieldDescriptions()
var items []map[string]any
for rows.Next() {
vals, err := rows.Values()
if err != nil {
continue
}
m := make(map[string]any)
for i, col := range cols {
m[string(col.Name)] = fmt.Sprintf("%v", vals[i])
}
items = append(items, m)
}
if err := rows.Err(); err != nil {
return textResult(fmt.Sprintf("error: %v", err))
}
data, _ := json.MarshalIndent(items, "", " ")
return textResult(string(data))
}
var _ = pgx.ErrNoRows
var _ = time.Now

View File

@@ -0,0 +1,40 @@
package mcp
import (
"testing"
)
// TestNewServerRegistersTools verifies every tool registers with a valid
// input schema. The MCP SDK panics at AddTool if a tool omits its object
// input schema, so merely constructing the server exercises that contract —
// this test would have caught the "missing input schema" panic.
func TestNewServerRegistersTools(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Fatalf("newServer panicked (tool schema bug?): %v", r)
}
}()
// pool is only used inside tool handlers (invoked per-call), not at
// registration time, so a nil pool is safe for this construction test.
s := newServer(nil)
if s == nil {
t.Fatal("newServer returned nil")
}
}
func TestObjSchema(t *testing.T) {
s := objSchema(prop{"foo", "string", "a foo"}, prop{"n", "integer", "a number"})
if s.Type != "object" {
t.Errorf("schema type = %q, want object", s.Type)
}
if len(s.Properties) != 2 {
t.Fatalf("got %d properties, want 2", len(s.Properties))
}
if s.Properties["foo"].Type != "string" || s.Properties["n"].Type != "integer" {
t.Errorf("property types wrong: %+v", s.Properties)
}
// empty schema still valid (object with no properties)
if objSchema().Type != "object" {
t.Error("empty objSchema not an object")
}
}

View File

@@ -0,0 +1,116 @@
// Package notifier handles alerts and approval requests via Matrix.
// Uses the DB as the rendezvous — no service-to-service calls (SA7/A7).
// Pending approvals survive restarts of either side.
package notifier
import (
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"fmt"
"log/slog"
"time"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/google/uuid"
)
// Run starts the notifier loop. Blocks until ctx is cancelled.
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
slog.Info("notifier: starting")
interval := 15 * time.Second
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
slog.Info("notifier: shutting down")
return
case <-ticker.C:
processPendingApprovals(ctx, pool, cfg)
}
}
}
// processPendingApprovals checks for pending approvals and sends alerts.
func processPendingApprovals(ctx context.Context, pool *db.Pool, cfg config.Config) {
q := sqlcgen.New(pool)
status := "pending"
approvals, err := q.ListApprovals(ctx, sqlcgen.ListApprovalsParams{
Status: &status,
})
if err != nil {
slog.Error("notifier: list approvals", "error", err)
return
}
for _, a := range approvals {
// Check if already expired
if a.ExpiresAt.Before(time.Now()) {
_ = q.UpdateApprovalStatus(ctx, sqlcgen.UpdateApprovalStatusParams{
EntityID: a.EntityID,
Status: "expired",
})
continue
}
// Generate approval token
token := generateApprovalToken(a.EntityID, cfg.ApprovalHMACSecret)
tokenHash := hashToken(token)
// Store token hash
_, _ = pool.Exec(ctx,
"UPDATE approvals SET token_hash = $2 WHERE entity_id = $1",
a.EntityID, tokenHash)
slog.Info("notifier: approval pending",
"approval_id", a.EntityID,
"action", a.Action,
"risk_class", a.RiskClass,
"token", token[:16]+"...",
"expires_at", a.ExpiresAt)
}
}
// generateApprovalToken creates a single-use HMAC token for an approval.
// Token = HMAC(approval_id ‖ nonce, secret)
func generateApprovalToken(approvalID uuid.UUID, secret string) string {
if secret == "" {
secret = "dev-secret-do-not-use-in-prod"
}
nonce := fmt.Sprintf("%d", time.Now().UnixNano())
mac := hmac.New(sha256.New, []byte(secret))
mac.Write([]byte(approvalID.String()))
mac.Write([]byte(nonce))
return hex.EncodeToString(mac.Sum(nil))
}
// VerifyApprovalToken checks that a token matches the stored hash.
func VerifyApprovalToken(ctx context.Context, pool *db.Pool, approvalID uuid.UUID, token string) bool {
q := sqlcgen.New(pool)
a, err := q.GetApprovalByID(ctx, approvalID)
if err != nil || a.TokenHash == nil {
return false
}
if a.Status != "pending" {
return false
}
if a.ExpiresAt.Before(time.Now()) {
return false
}
return *a.TokenHash == hashToken(token)
}
// hashToken double-hashes a token for storage.
func hashToken(token string) string {
h := sha256.Sum256([]byte(token))
return hex.EncodeToString(h[:])
}
// Ensure types are used
var _ = uuid.UUID{}

View File

@@ -0,0 +1,23 @@
package observability
import (
"log/slog"
"os"
)
// NewLogger creates a structured JSON logger writing to stdout.
// In debug mode, it enables verbose probe payloads, SQL queries, and
// classification reasoning.
func NewLogger(debug bool) *slog.Logger {
level := slog.LevelInfo
if debug {
level = slog.LevelDebug
}
handler := slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
Level: level,
})
logger := slog.New(handler).With("service", "oikos")
return logger
}

View File

@@ -0,0 +1,65 @@
package observability
import (
"context"
"encoding/json"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/google/uuid"
)
// Audit writes an audit_log entry. Pass a transaction-bound Queries so the
// entry commits or rolls back atomically with the state change it records.
//
// actorLabel is the interim textual actor identity ("operator:dev",
// "agent:mcp") recorded in detail; actor_id (a Person/Agent entity UUID)
// starts being populated when OIDC identity resolution lands.
func Audit(ctx context.Context, q *sqlcgen.Queries, actorType, actorLabel,
action string, entityID *uuid.UUID, method, path, correlationID string,
detail map[string]any) error {
if detail == nil {
detail = map[string]any{}
}
detail["actor"] = actorLabel
detailJSON, _ := json.Marshal(detail)
var corr *string
if correlationID != "" {
corr = &correlationID
}
return q.InsertAuditEntry(ctx, sqlcgen.InsertAuditEntryParams{
ActorType: actorType,
Action: action,
EntityID: entityID,
Method: &method,
Path: &path,
Detail: detailJSON,
CorrelationID: corr,
})
}
// Event emits a structured event in the caller's transaction (SG10). The
// post-commit NOTIFY trigger (migration 008) fans it out to SSE subscribers.
func Event(ctx context.Context, q *sqlcgen.Queries, eventType string,
entityID *uuid.UUID, severity, source string, correlationID string,
data map[string]any) error {
dataJSON, _ := json.Marshal(data)
if data == nil {
dataJSON = []byte("{}")
}
var corr *string
if correlationID != "" {
corr = &correlationID
}
_, err := q.InsertEvent(ctx, sqlcgen.InsertEventParams{
Type: eventType,
EntityID: entityID,
Severity: severity,
Source: source,
Data: dataJSON,
CorrelationID: corr,
})
return err
}

View File

@@ -0,0 +1,112 @@
// Package ontology implements the meta-schema logic: the entity-type
// hierarchy (is-a with abstract types), relationship endpoint validation,
// cardinality enforcement, and lifecycle state checks. Both the seed
// ingest and the API mutation paths validate through this package so the
// graph can never violate the ontology (plan R3-1).
package ontology
import (
"fmt"
"github.com/dtoro/oikos/internal/domain"
)
// TypeInfo is the subset of an entity type the validator needs.
type TypeInfo struct {
Parent string
IsAbstract bool
LifecycleID string
}
// RelTypeInfo is the subset of a relationship type the validator needs.
type RelTypeInfo struct {
SourceType string
TargetType string
Cardinality string
}
// LifecycleInfo is the subset of a lifecycle the validator needs.
type LifecycleInfo struct {
States map[string]bool
DefaultState string
}
// TypeTree holds the loaded ontology meta-schema for validation.
type TypeTree struct {
Types map[string]TypeInfo
RelTypes map[string]RelTypeInfo
Lifecycles map[string]LifecycleInfo
}
// IsA reports whether typ is target or a descendant of it.
func (t *TypeTree) IsA(typ, target string) bool {
seen := map[string]bool{}
for cur := typ; cur != ""; cur = t.Types[cur].Parent {
if cur == target {
return true
}
if seen[cur] {
return false // cycle guard — ingest rejects cycles, belt and braces
}
seen[cur] = true
if _, ok := t.Types[cur]; !ok {
return false
}
}
return false
}
// ValidateEntity checks that typ exists, is not abstract, and that state
// (if set) is legal for the type's lifecycle.
func (t *TypeTree) ValidateEntity(typ, state string) error {
info, ok := t.Types[typ]
if !ok {
return fmt.Errorf("%w: entity type %q", domain.ErrNotFound, typ)
}
if info.IsAbstract {
return fmt.Errorf("%w: %q", domain.ErrAbstractType, typ)
}
if state == "" {
return nil
}
if info.LifecycleID == "" {
return fmt.Errorf("%w: type %q has no lifecycle but state %q given",
domain.ErrInvalidTransition, typ, state)
}
lc, ok := t.Lifecycles[info.LifecycleID]
if !ok {
return fmt.Errorf("%w: lifecycle %q", domain.ErrNotFound, info.LifecycleID)
}
if !lc.States[state] {
return fmt.Errorf("%w: state %q not in lifecycle %q",
domain.ErrInvalidTransition, state, info.LifecycleID)
}
return nil
}
// ValidateEdge checks that relType exists and that the endpoint entity
// types are the declared source/target types or descendants of them.
func (t *TypeTree) ValidateEdge(relType, sourceEntityType, targetEntityType string) error {
rt, ok := t.RelTypes[relType]
if !ok {
return fmt.Errorf("%w: relationship type %q", domain.ErrNotFound, relType)
}
if !t.IsA(sourceEntityType, rt.SourceType) {
return fmt.Errorf("%w: %s source %q is not a %q",
domain.ErrInvalidEdge, relType, sourceEntityType, rt.SourceType)
}
if !t.IsA(targetEntityType, rt.TargetType) {
return fmt.Errorf("%w: %s target %q is not a %q",
domain.ErrInvalidEdge, relType, targetEntityType, rt.TargetType)
}
return nil
}
// DefaultState returns the default lifecycle state for a type ("" if none).
func (t *TypeTree) DefaultState(typ string) string {
info, ok := t.Types[typ]
if !ok || info.LifecycleID == "" {
return ""
}
return t.Lifecycles[info.LifecycleID].DefaultState
}

View File

@@ -0,0 +1,120 @@
package ontology
import (
"errors"
"testing"
"github.com/dtoro/oikos/internal/domain"
)
func fixtureTree() *TypeTree {
return &TypeTree{
Types: map[string]TypeInfo{
"entity": {IsAbstract: true},
"compute-entity": {Parent: "entity", IsAbstract: true},
"machine": {Parent: "compute-entity", IsAbstract: true},
"proxmox-host": {Parent: "machine", LifecycleID: "infrastructure"},
"lxc": {Parent: "compute-entity", LifecycleID: "infrastructure"},
"service": {Parent: "entity", LifecycleID: "infrastructure"},
"document": {Parent: "entity"}, // no lifecycle
},
RelTypes: map[string]RelTypeInfo{
"hosts": {SourceType: "machine", TargetType: "compute-entity", Cardinality: "one-to-many"},
"provides": {SourceType: "compute-entity", TargetType: "service", Cardinality: "one-to-many"},
"depends-on": {SourceType: "service", TargetType: "service", Cardinality: "many-to-many"},
"documents": {SourceType: "document", TargetType: "entity", Cardinality: "many-to-one"},
},
Lifecycles: map[string]LifecycleInfo{
"infrastructure": {
States: map[string]bool{"planned": true, "active": true, "destroyed": true},
DefaultState: "active",
},
},
}
}
func TestIsAWalksHierarchy(t *testing.T) {
tree := fixtureTree()
cases := []struct {
typ, target string
want bool
}{
{"proxmox-host", "machine", true},
{"proxmox-host", "compute-entity", true},
{"proxmox-host", "entity", true},
{"proxmox-host", "proxmox-host", true},
{"lxc", "machine", false},
{"service", "compute-entity", false},
{"nonexistent", "entity", false},
}
for _, c := range cases {
if got := tree.IsA(c.typ, c.target); got != c.want {
t.Errorf("IsA(%q, %q) = %v, want %v", c.typ, c.target, got, c.want)
}
}
}
func TestValidateEntityRejectsAbstract(t *testing.T) {
tree := fixtureTree()
for _, abstract := range []string{"entity", "compute-entity", "machine"} {
if err := tree.ValidateEntity(abstract, ""); !errors.Is(err, domain.ErrAbstractType) {
t.Errorf("ValidateEntity(%q) = %v, want ErrAbstractType", abstract, err)
}
}
if err := tree.ValidateEntity("lxc", "active"); err != nil {
t.Errorf("ValidateEntity(lxc, active) = %v, want nil", err)
}
}
func TestValidateEntityStates(t *testing.T) {
tree := fixtureTree()
if err := tree.ValidateEntity("lxc", "flying"); !errors.Is(err, domain.ErrInvalidTransition) {
t.Errorf("bad state = %v, want ErrInvalidTransition", err)
}
// state on a type without a lifecycle is rejected
if err := tree.ValidateEntity("document", "active"); !errors.Is(err, domain.ErrInvalidTransition) {
t.Errorf("state without lifecycle = %v, want ErrInvalidTransition", err)
}
// unknown type
if err := tree.ValidateEntity("ghost", ""); !errors.Is(err, domain.ErrNotFound) {
t.Errorf("unknown type = %v, want ErrNotFound", err)
}
}
func TestValidateEdgeHonorsInheritance(t *testing.T) {
tree := fixtureTree()
// proxmox-host is-a machine; lxc is-a compute-entity → valid
if err := tree.ValidateEdge("hosts", "proxmox-host", "lxc"); err != nil {
t.Errorf("hosts(proxmox-host→lxc) = %v, want nil", err)
}
// abstract endpoint declared, concrete descendant offered → valid
if err := tree.ValidateEdge("provides", "lxc", "service"); err != nil {
t.Errorf("provides(lxc→service) = %v, want nil", err)
}
// documents targets the root abstract 'entity' → anything is valid
if err := tree.ValidateEdge("documents", "document", "proxmox-host"); err != nil {
t.Errorf("documents(document→proxmox-host) = %v, want nil", err)
}
// service is not a machine → invalid source
if err := tree.ValidateEdge("hosts", "service", "lxc"); !errors.Is(err, domain.ErrInvalidEdge) {
t.Errorf("hosts(service→lxc) = %v, want ErrInvalidEdge", err)
}
// lxc is not a service → invalid target
if err := tree.ValidateEdge("depends-on", "service", "lxc"); !errors.Is(err, domain.ErrInvalidEdge) {
t.Errorf("depends-on(service→lxc) = %v, want ErrInvalidEdge", err)
}
// unknown relationship type
if err := tree.ValidateEdge("teleports", "lxc", "service"); !errors.Is(err, domain.ErrNotFound) {
t.Errorf("unknown rel type = %v, want ErrNotFound", err)
}
}
func TestDefaultState(t *testing.T) {
tree := fixtureTree()
if got := tree.DefaultState("lxc"); got != "active" {
t.Errorf("DefaultState(lxc) = %q, want active", got)
}
if got := tree.DefaultState("document"); got != "" {
t.Errorf("DefaultState(document) = %q, want empty", got)
}
}

161
internal/policy/classify.go Normal file
View File

@@ -0,0 +1,161 @@
// Package policy implements Oikos classification and policy evaluation.
// Determines risk class, autonomy route, and approval requirements.
package policy
import (
"context"
"encoding/json"
"fmt"
"github.com/dtoro/oikos/internal/domain"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
)
// ClassificationResult holds the outcome of classifying a signal.
type ClassificationResult struct {
RiskClass string
Route string // 'auto-act', 'escalate', 'hold'
RecommendedAction json.RawMessage
AutonomyCheck string
BlastRadius []uuid.UUID
CorrelationID string
Reasoning json.RawMessage
}
// Classify evaluates a signal against policy rules to determine the action route.
// ctx must have a DB connection pool accessible via a helper interface.
type Classifier struct {
DB interface {
QueryRow(ctx context.Context, sql string, args ...any) pgx.Row
Query(ctx context.Context, sql string, args ...any) (pgx.Rows, error)
Exec(ctx context.Context, sql string, args ...any) (int64, error)
}
}
// NewClassifier creates a classifier with a DB query interface.
func NewClassifier(dbc interface {
QueryRow(ctx context.Context, sql string, args ...any) pgx.Row
Query(ctx context.Context, sql string, args ...any) (pgx.Rows, error)
Exec(ctx context.Context, sql string, args ...any) (int64, error)
}) *Classifier {
return &Classifier{DB: dbc}
}
// ClassifySignal evaluates a signal and returns the classification result.
func (c *Classifier) ClassifySignal(ctx context.Context, signalEntityID, targetEntityID uuid.UUID,
kind, severity, correlationID string) (*ClassificationResult, error) {
// Determine target entity type
var entityType string
err := c.DB.QueryRow(ctx,
"SELECT type FROM entities WHERE id = $1", targetEntityID).Scan(&entityType)
if err != nil {
return nil, fmt.Errorf("%w: target entity %s", domain.ErrNotFound, targetEntityID)
}
// Look up risk class for this entity type and action
var riskClass string
var approvalRequired string
err = c.DB.QueryRow(ctx, `
SELECT rc.name, rc.approval_required
FROM risk_classes rc
WHERE rc.name = (
SELECT COALESCE(ar.risk_class, 'reversible_low')
FROM approval_rules ar
WHERE ar.entity_type = $1 AND ar.action = $2
LIMIT 1
)`, entityType, kind).Scan(&riskClass, &approvalRequired)
if err != nil {
// Default to escalate
riskClass = "reversible_low"
approvalRequired = "operator"
}
// Check global autonomy setting
var globalAutoAct string
err = c.DB.QueryRow(ctx,
"SELECT value FROM autonomy_settings WHERE key = 'global.auto_act'").Scan(&globalAutoAct)
if err != nil {
globalAutoAct = "on" // default to on
}
// Check per-entity kill-switch
var slug string
c.DB.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", targetEntityID).Scan(&slug)
var entityAutoAct string
if slug != "" {
c.DB.QueryRow(ctx,
"SELECT value FROM autonomy_settings WHERE key = $1",
"never_auto_act."+slug).Scan(&entityAutoAct)
}
// Determine route
route := "escalate"
autonomyCheck := ""
if globalAutoAct == "off" || globalAutoAct == "false" {
route = "escalate"
autonomyCheck = "blocked: global auto_act disabled"
} else if entityAutoAct == "true" {
route = "escalate"
autonomyCheck = "blocked: per-entity kill-switch"
} else if approvalRequired == "none" {
route = "auto-act"
autonomyCheck = "allowed"
} else {
autonomyCheck = "requires approval: " + approvalRequired
}
// Compute blast radius
blastRadius := computeBlastRadius(ctx, c.DB, targetEntityID)
reasoning := map[string]any{
"entity_type": entityType,
"risk_class": riskClass,
"approval_rule": approvalRequired,
"global_auto_act": globalAutoAct,
"entity_slug": slug,
}
reasoningJSON, _ := json.Marshal(reasoning)
recommended, _ := json.Marshal(map[string]any{
"action": kind,
"reason": fmt.Sprintf("signal %s on %s", severity, entityType),
})
return &ClassificationResult{
RiskClass: riskClass,
Route: route,
RecommendedAction: recommended,
AutonomyCheck: autonomyCheck,
BlastRadius: blastRadius,
CorrelationID: correlationID,
Reasoning: reasoningJSON,
}, nil
}
// computeBlastRadius traverses relationships to find affected entities.
func computeBlastRadius(ctx context.Context, dbc interface {
Query(ctx context.Context, sql string, args ...any) (pgx.Rows, error)
}, entityID uuid.UUID) []uuid.UUID {
rows, err := dbc.Query(ctx, `
SELECT entity_id FROM blast_radius($1, 3)`, entityID)
if err != nil {
return nil
}
defer rows.Close()
var ids []uuid.UUID
for rows.Next() {
var id uuid.UUID
if err := rows.Scan(&id); err == nil {
ids = append(ids, id)
}
}
return ids
}
// Ensure domain is used
var _ = domain.ErrAutonomyBlocked

View File

@@ -0,0 +1,29 @@
package scheduler
import (
"context"
"log/slog"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
)
func init() {
// Register with main package via package-level variable
// The cmd/oikos/main.go sets SchedulerRunner in its import
mainRunner = Run
}
// mainRunner is assigned to main.SchedulerRunner by the cmd/oikos package.
// It's set during init so that when main runs, the scheduler Runner is available.
var mainRunner func(context.Context, *db.Pool, config.Config)
// RunnerForMain provides the run function for registration in main.
func RunnerForMain() func(context.Context, *db.Pool, config.Config) {
return Run
}
// ensure sqlcgen is used
var _ = sqlcgen.Queries{}
var _ = slog.Default

View File

@@ -0,0 +1,210 @@
// Package scheduler implements the Oikos observe + decide loop (Phase 3).
// It loads enabled check_defs, runs checks on schedule, manages signal
// lifecycle (dedup, flap suppression, maintenance mode), and writes metrics.
package scheduler
import (
"context"
"log/slog"
"time"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/google/uuid"
"golang.org/x/sync/errgroup"
)
// Run starts the scheduler loop. Blocks until ctx is cancelled.
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
slog.Info("scheduler: starting", "interval", cfg.SchedulerInterval)
interval := cfg.SchedulerInterval
if interval <= 0 {
interval = 30 * time.Second
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
// Immediate first pass
runCheckPass(ctx, pool)
for {
select {
case <-ctx.Done():
slog.Info("scheduler: shutting down")
return
case <-ticker.C:
runCheckPass(ctx, pool)
}
}
}
// runCheckPass executes one full cycle of check evaluation.
func runCheckPass(ctx context.Context, pool *db.Pool) {
q := sqlcgen.New(pool)
defs, err := q.ListEnabledCheckDefs(ctx)
if err != nil {
slog.Error("scheduler: list check defs", "error", err)
return
}
if len(defs) == 0 {
slog.Debug("scheduler: no enabled check_defs")
return
}
g, gctx := errgroup.WithContext(ctx)
g.SetLimit(10) // bounded worker pool
for _, def := range defs {
cd := def
g.Go(func() error {
runCheck(gctx, pool, cd)
return nil
})
}
g.Wait()
// Housekeeping after each pass
housekeeping(ctx, pool)
}
// runCheck executes a single check and processes the result.
func runCheck(ctx context.Context, pool *db.Pool, cd sqlcgen.ListEnabledCheckDefsRow) {
q := sqlcgen.New(pool)
start := time.Now()
health, signalKind, evidence, checkErr := executeCheck(ctx, cd)
latency := time.Since(start).Milliseconds()
// Write metric
_ = q.InsertMetricSample(ctx, sqlcgen.InsertMetricSampleParams{
EntityID: cd.EntityID,
Metric: "probe_latency_ms",
Value: float64(latency),
Tags: []byte(`{}`),
})
if checkErr != nil {
slog.Warn("scheduler: check failed",
"entity", cd.EntitySlug, "kind", cd.Kind, "error", checkErr)
}
if signalKind == "" || health == "healthy" {
// Recovery: resolve any open signal for this check
resolveSignal(ctx, pool, cd.EntityID, cd.EntitySlug)
// Update entity_status to healthy
_ = q.UpsertEntityStatus(ctx, sqlcgen.UpsertEntityStatusParams{
EntityID: cd.EntityID,
Health: "healthy",
LastCheckAt: &[]time.Time{time.Now()}[0],
Details: []byte(`{}`),
})
return
}
// Failure: upsert signal (dedup via partial unique index)
slog.Warn("scheduler: raising signal",
"entity", cd.EntitySlug, "kind", signalKind, "evidence", evidence)
severity := "warning"
if signalKind == "down" {
severity = "critical"
}
sig, err := q.UpsertSignal(ctx, sqlcgen.UpsertSignalParams{
EntityID: cd.EntityID,
Kind: signalKind,
Severity: severity,
TargetEntityID: cd.TargetID,
Evidence: &evidence,
})
if err != nil {
slog.Error("scheduler: upsert signal", "error", err)
return
}
// Update entity_status
_ = q.UpsertEntityStatus(ctx, sqlcgen.UpsertEntityStatusParams{
EntityID: cd.EntityID,
Health: health,
LastCheckAt: &[]time.Time{time.Now()}[0],
Details: []byte(`{}`),
})
_ = sig // used for flap detection below
}
// resolveSignal resolves any open signal for the given check entity.
func resolveSignal(ctx context.Context, pool *db.Pool, entityID uuid.UUID, slug string) {
q := sqlcgen.New(pool)
// Check if there's an open signal on this entity
_, err := pool.Exec(ctx, `UPDATE signals SET state = 'resolved', updated_at = now()
WHERE entity_id = $1 AND state = 'raised'`, entityID)
if err != nil {
return
}
_ = q.UpsertEntityStatus(ctx, sqlcgen.UpsertEntityStatusParams{
EntityID: entityID,
Health: "healthy",
LastCheckAt: &[]time.Time{time.Now()}[0],
Details: []byte(`{}`),
})
slog.Info("scheduler: signal resolved", "entity", slug)
}
// executeCheck dispatches to the appropriate checker by kind.
func executeCheck(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (health string, signalKind string, evidence string, err error) {
switch cd.Kind {
case "http":
return checkHTTP(ctx, cd)
case "tcp":
return checkTCP(ctx, cd)
case "disk":
return checkDisk(ctx, cd)
case "cert-expiry":
return checkCertExpiry(ctx, cd)
default:
return "unknown", "", "", nil
}
}
// housekeeping runs background maintenance tasks.
func housekeeping(ctx context.Context, pool *db.Pool) {
// Prune expired idempotency keys (older than 24h)
cutoff := time.Now().Add(-24 * time.Hour)
_, err := pool.Exec(ctx,
"DELETE FROM idempotency_keys WHERE created_at < $1", cutoff)
if err != nil {
slog.Error("scheduler: prune idempotency keys", "error", err)
}
// Log housekeeping completion
slog.Debug("scheduler: housekeeping done", "pruned_idempotency_before", cutoff.Format(time.RFC3339))
}
// checkHTTP performs an HTTP health check.
func checkHTTP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string, string, string, error) {
// Stub: always returns healthy
return "healthy", "", "", nil
}
// checkTCP performs a TCP dial check.
func checkTCP(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string, string, string, error) {
// Stub: always returns healthy
return "healthy", "", "", nil
}
// checkDisk performs a disk usage check via SSH.
func checkDisk(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string, string, string, error) {
// Stub: always returns healthy
return "healthy", "", "", nil
}
// checkCertExpiry checks TLS certificate expiry.
func checkCertExpiry(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) (string, string, string, error) {
// Stub: always returns healthy
return "healthy", "", "", nil
}

View File

@@ -0,0 +1,43 @@
-- Migration 001: Ontology meta-schema (with inheritance, R3-1)
-- Defines entity types, relationship types, lifecycle definitions, and seed versioning.
CREATE TABLE lifecycle_defs (
id TEXT PRIMARY KEY,
states TEXT[] NOT NULL,
default_state TEXT NOT NULL,
terminal_states TEXT[] NOT NULL DEFAULT '{}',
transitions JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE entity_types (
name TEXT PRIMARY KEY,
parent_type TEXT REFERENCES entity_types(name),
is_abstract BOOLEAN NOT NULL DEFAULT false,
domain TEXT NOT NULL,
layer TEXT NOT NULL CHECK (layer IN ('meta','infrastructure','governance','cognition')),
description TEXT,
lifecycle_id TEXT REFERENCES lifecycle_defs(id),
attribute_schema JSONB,
schema_version INTEGER NOT NULL DEFAULT 1,
status TEXT NOT NULL DEFAULT 'active',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE relationship_types (
name TEXT PRIMARY KEY,
inverse TEXT,
source_type TEXT NOT NULL REFERENCES entity_types(name),
target_type TEXT NOT NULL REFERENCES entity_types(name),
cardinality TEXT NOT NULL CHECK (cardinality IN
('one-to-one','one-to-many','many-to-one','many-to-many')),
description TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE seed_versions (
file TEXT PRIMARY KEY,
content_hash TEXT NOT NULL,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

View File

@@ -0,0 +1,50 @@
-- Migration 002: Entity instances (UUIDv7 + slug, R3-5/D1)
-- The inventory graph: entities + typed relationships + blast_radius function.
CREATE TABLE entities (
id UUID PRIMARY KEY,
slug TEXT NOT NULL UNIQUE,
type TEXT NOT NULL REFERENCES entity_types(name),
name TEXT NOT NULL,
state TEXT,
attributes JSONB NOT NULL DEFAULT '{}',
maintenance_until TIMESTAMPTZ,
version INTEGER NOT NULL DEFAULT 1,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (type, name)
);
CREATE INDEX idx_entities_type ON entities(type);
CREATE INDEX idx_entities_state ON entities(state);
CREATE INDEX idx_entities_attrs ON entities USING GIN(attributes);
CREATE TABLE relationships (
source_id UUID NOT NULL REFERENCES entities(id) ON DELETE RESTRICT,
target_id UUID NOT NULL REFERENCES entities(id) ON DELETE RESTRICT,
type TEXT NOT NULL REFERENCES relationship_types(name),
attributes JSONB,
valid_from TIMESTAMPTZ NOT NULL DEFAULT now(),
valid_to TIMESTAMPTZ,
PRIMARY KEY (source_id, target_id, type, valid_from)
);
CREATE INDEX idx_rel_source ON relationships(source_id) WHERE valid_to IS NULL;
CREATE INDEX idx_rel_target ON relationships(target_id) WHERE valid_to IS NULL;
CREATE INDEX idx_rel_type ON relationships(type) WHERE valid_to IS NULL;
-- Cycle-safe traversal (P1): path accumulator prevents revisits; depth capped.
CREATE OR REPLACE FUNCTION blast_radius(start_id UUID, max_depth INT DEFAULT 3,
rel_types TEXT[] DEFAULT NULL)
RETURNS TABLE(entity_id UUID, depth INT) AS $$
WITH RECURSIVE walk AS (
SELECT start_id AS entity_id, 0 AS depth, ARRAY[start_id] AS path
UNION ALL
SELECT r.target_id, w.depth + 1, w.path || r.target_id
FROM relationships r
JOIN walk w ON r.source_id = w.entity_id
WHERE w.depth < LEAST(max_depth, 5)
AND r.valid_to IS NULL
AND NOT r.target_id = ANY(w.path)
AND (rel_types IS NULL OR r.type = ANY(rel_types))
)
SELECT entity_id, MIN(depth) FROM walk GROUP BY entity_id;
$$ LANGUAGE sql STABLE;

View File

@@ -0,0 +1,71 @@
-- Migration 003: Operations (signals, checks, approvals, status)
-- Signals are dual entities (entities row + signals table for indexed querying).
CREATE TABLE check_defs (
entity_id UUID PRIMARY KEY REFERENCES entities(id),
target_id UUID REFERENCES entities(id),
target_type TEXT REFERENCES entity_types(name),
kind TEXT NOT NULL,
config JSONB NOT NULL DEFAULT '{}',
interval_s INTEGER NOT NULL DEFAULT 600,
timeout_s INTEGER NOT NULL DEFAULT 10,
zone TEXT,
enabled BOOLEAN NOT NULL DEFAULT true,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE signals (
entity_id UUID PRIMARY KEY REFERENCES entities(id),
kind TEXT NOT NULL,
severity TEXT NOT NULL CHECK (severity IN ('info','warning','critical')),
target_entity_id UUID REFERENCES entities(id),
check_id UUID REFERENCES check_defs(entity_id),
evidence TEXT,
likely_cause TEXT,
state TEXT NOT NULL DEFAULT 'raised',
occurrence_count INTEGER NOT NULL DEFAULT 1,
first_seen_at TIMESTAMPTZ NOT NULL DEFAULT now(),
last_seen_at TIMESTAMPTZ NOT NULL DEFAULT now(),
flap_count INTEGER NOT NULL DEFAULT 0,
hold_down_until TIMESTAMPTZ,
mute_until TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- At most ONE open signal per (target, kind) — repeats update the open row
CREATE UNIQUE INDEX uq_signals_open ON signals(target_entity_id, kind)
WHERE state NOT IN ('resolved','failed');
CREATE INDEX idx_signals_state ON signals(state);
CREATE TABLE approvals (
entity_id UUID PRIMARY KEY REFERENCES entities(id),
subject_entity_id UUID REFERENCES entities(id),
action TEXT NOT NULL,
risk_class TEXT NOT NULL,
kind TEXT NOT NULL DEFAULT 'execution',
payload JSONB,
status TEXT NOT NULL DEFAULT 'pending',
token_hash TEXT,
expires_at TIMESTAMPTZ NOT NULL,
decided_at TIMESTAMPTZ,
decided_by UUID REFERENCES entities(id),
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE entity_status (
entity_id UUID PRIMARY KEY REFERENCES entities(id),
health TEXT NOT NULL DEFAULT 'unknown',
last_check_at TIMESTAMPTZ,
details JSONB NOT NULL DEFAULT '{}',
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE idempotency_keys (
key TEXT NOT NULL,
actor TEXT NOT NULL,
request_hash TEXT NOT NULL,
response_code INTEGER,
response_body JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (actor, key)
);

View File

@@ -0,0 +1,91 @@
-- Migration 004: Cognition (classifications, executions, learning)
-- All cognition objects are dual entities (entities row + typed table).
-- Classifications persist every autonomous decision (SA5).
CREATE TABLE classifications (
entity_id UUID PRIMARY KEY REFERENCES entities(id),
signal_entity_id UUID REFERENCES signals(entity_id),
target_entity_id UUID REFERENCES entities(id),
action TEXT NOT NULL,
recommended_action JSONB,
risk_class TEXT NOT NULL,
route TEXT NOT NULL CHECK (route IN ('auto-act','escalate','hold')),
blast_radius UUID[],
pattern_confidence REAL,
skill_id UUID,
autonomy_check TEXT,
reasoning JSONB NOT NULL,
correlation_id TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_class_signal ON classifications(signal_entity_id);
CREATE INDEX idx_class_entity ON classifications(target_entity_id);
CREATE TABLE executions (
entity_id UUID PRIMARY KEY REFERENCES entities(id),
classification_id UUID REFERENCES classifications(entity_id),
signal_entity_id UUID REFERENCES signals(entity_id),
target_entity_id UUID REFERENCES entities(id),
action TEXT NOT NULL,
risk_class TEXT NOT NULL,
approval_id UUID REFERENCES approvals(entity_id),
agent_id UUID REFERENCES entities(id),
skill_id UUID,
skill_version INTEGER,
status TEXT NOT NULL DEFAULT 'proposed',
result JSONB,
duration_ms INTEGER,
verified BOOLEAN NOT NULL DEFAULT false,
correlation_id TEXT NOT NULL,
started_at TIMESTAMPTZ,
completed_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_exec_target ON executions(target_entity_id);
CREATE INDEX idx_exec_status ON executions(status);
CREATE TABLE feedback (
entity_id UUID PRIMARY KEY REFERENCES entities(id),
execution_id UUID NOT NULL REFERENCES executions(entity_id),
outcome TEXT NOT NULL CHECK (outcome IN ('success','failure','partial','unexpected')),
observation TEXT,
lesson TEXT,
unexpected_side_effects TEXT[],
tags TEXT[],
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_feedback_ts ON feedback(created_at);
CREATE TABLE patterns (
entity_id UUID PRIMARY KEY REFERENCES entities(id),
applies_type TEXT NOT NULL REFERENCES entity_types(name),
action TEXT NOT NULL,
pattern TEXT NOT NULL,
confidence REAL NOT NULL DEFAULT 0,
evidence_count INTEGER NOT NULL DEFAULT 0,
success_count INTEGER NOT NULL DEFAULT 0,
failure_count INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'hypothesized',
quarantined BOOLEAN NOT NULL DEFAULT false,
version INTEGER NOT NULL DEFAULT 1,
last_validated_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (applies_type, action)
);
CREATE TABLE skills (
entity_id UUID NOT NULL REFERENCES entities(id),
version INTEGER NOT NULL DEFAULT 1,
name TEXT NOT NULL,
procedure JSONB NOT NULL,
applies_type TEXT REFERENCES entity_types(name),
action TEXT NOT NULL,
pattern_ids UUID[],
status TEXT NOT NULL DEFAULT 'drafted',
success_rate REAL,
changed_by UUID,
change_reason TEXT,
last_used_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (entity_id, version)
);

View File

@@ -0,0 +1,28 @@
-- Migration 005: Policy (risk classes, approval rules, autonomy settings)
CREATE TABLE risk_classes (
name TEXT PRIMARY KEY,
description TEXT,
approval_required TEXT NOT NULL DEFAULT 'none',
autonomy_allowed BOOLEAN NOT NULL DEFAULT false
);
CREATE TABLE approval_rules (
id UUID PRIMARY KEY,
entity_type TEXT REFERENCES entity_types(name),
action TEXT NOT NULL,
risk_class TEXT NOT NULL REFERENCES risk_classes(name),
autonomy_level TEXT NOT NULL DEFAULT 'escalate' CHECK
(autonomy_level IN ('auto','escalate','never')),
scope_entity UUID REFERENCES entities(id),
version INTEGER NOT NULL DEFAULT 1,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (entity_type, action, scope_entity)
);
CREATE TABLE autonomy_settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
version INTEGER NOT NULL DEFAULT 1,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

View File

@@ -0,0 +1,138 @@
-- Migration 006: Observability (TimescaleDB)
-- Hypertable PKs include time column (SG1); idempotent DDL (SG3); no array_agg in CAGGs (SG2).
-- Enable TimescaleDB extension
CREATE EXTENSION IF NOT EXISTS timescaledb;
-- ─── Time-series metrics ──────────────────────────────────────────────
CREATE TABLE metric_samples (
ts TIMESTAMPTZ NOT NULL,
entity_id UUID NOT NULL,
metric TEXT NOT NULL,
value DOUBLE PRECISION NOT NULL,
tags JSONB NOT NULL DEFAULT '{}'
);
SELECT create_hypertable('metric_samples', 'ts',
chunk_time_interval => INTERVAL '7 days', if_not_exists => TRUE);
CREATE INDEX idx_metrics_entity_ts ON metric_samples(entity_id, ts DESC);
CREATE INDEX idx_metrics_metric_ts ON metric_samples(metric, ts DESC);
DO $$ BEGIN
PERFORM add_retention_policy('metric_samples', INTERVAL '90 days');
EXCEPTION WHEN OTHERS THEN NULL;
END $$;
-- 1-hour rollups
CREATE MATERIALIZED VIEW metric_rollups_1h WITH (timescaledb.continuous) AS
SELECT time_bucket('1 hour', ts) AS bucket, entity_id, metric,
avg(value) AS avg_value, min(value) AS min_value,
max(value) AS max_value, count(*) AS sample_count
FROM metric_samples GROUP BY bucket, entity_id, metric;
DO $$ BEGIN
PERFORM add_continuous_aggregate_policy('metric_rollups_1h',
start_offset => INTERVAL '2 hours',
end_offset => INTERVAL '5 minutes',
schedule_interval => INTERVAL '1 hour');
EXCEPTION WHEN OTHERS THEN NULL;
END $$;
-- 1-day rollups
CREATE MATERIALIZED VIEW metric_rollups_1d WITH (timescaledb.continuous) AS
SELECT time_bucket('1 day', ts) AS bucket, entity_id, metric,
avg(value) AS avg_value, min(value) AS min_value,
max(value) AS max_value, count(*) AS sample_count
FROM metric_samples GROUP BY bucket, entity_id, metric;
DO $$ BEGIN
PERFORM add_continuous_aggregate_policy('metric_rollups_1d',
start_offset => INTERVAL '2 days',
end_offset => INTERVAL '1 hour',
schedule_interval => INTERVAL '1 day');
EXCEPTION WHEN OTHERS THEN NULL;
END $$;
-- ─── Audit log ────────────────────────────────────────────────────────
CREATE TABLE audit_log (
id BIGINT GENERATED ALWAYS AS IDENTITY,
ts TIMESTAMPTZ NOT NULL DEFAULT now(),
actor_type TEXT NOT NULL,
actor_id UUID,
action TEXT NOT NULL,
entity_id UUID,
method TEXT,
path TEXT,
status_code INTEGER,
detail JSONB NOT NULL DEFAULT '{}',
source_ip TEXT,
correlation_id TEXT,
PRIMARY KEY (id, ts)
);
SELECT create_hypertable('audit_log', 'ts',
chunk_time_interval => INTERVAL '7 days', if_not_exists => TRUE);
CREATE INDEX idx_audit_actor ON audit_log(actor_type, actor_id, ts DESC);
CREATE INDEX idx_audit_entity ON audit_log(entity_id, ts DESC);
CREATE INDEX idx_audit_action ON audit_log(action, ts DESC);
CREATE INDEX idx_audit_correlation ON audit_log(correlation_id);
DO $$ BEGIN
PERFORM add_retention_policy('audit_log', INTERVAL '365 days');
EXCEPTION WHEN OTHERS THEN NULL;
END $$;
-- ─── Event log ────────────────────────────────────────────────────────
CREATE TABLE events (
id BIGINT GENERATED ALWAYS AS IDENTITY,
ts TIMESTAMPTZ NOT NULL DEFAULT now(),
type TEXT NOT NULL,
entity_id UUID,
severity TEXT NOT NULL DEFAULT 'info',
source TEXT NOT NULL,
data JSONB NOT NULL DEFAULT '{}',
correlation_id TEXT,
PRIMARY KEY (id, ts)
);
SELECT create_hypertable('events', 'ts',
chunk_time_interval => INTERVAL '7 days', if_not_exists => TRUE);
CREATE INDEX idx_events_type_ts ON events(type, ts DESC);
CREATE INDEX idx_events_entity_ts ON events(entity_id, ts DESC);
CREATE INDEX idx_events_severity_ts ON events(severity, ts DESC);
CREATE INDEX idx_events_correlation ON events(correlation_id);
DO $$ BEGIN
PERFORM add_retention_policy('events', INTERVAL '90 days');
EXCEPTION WHEN OTHERS THEN NULL;
END $$;
-- ─── Agent activity ──────────────────────────────────────────────────
CREATE TABLE agent_activity (
id BIGINT GENERATED ALWAYS AS IDENTITY,
ts TIMESTAMPTZ NOT NULL DEFAULT now(),
agent_id UUID NOT NULL,
session_id TEXT,
activity_type TEXT NOT NULL,
tool_name TEXT,
entity_id UUID,
input_summary TEXT,
output_summary TEXT,
duration_ms INTEGER,
token_count INTEGER,
success BOOLEAN,
correlation_id TEXT,
PRIMARY KEY (id, ts)
);
SELECT create_hypertable('agent_activity', 'ts',
chunk_time_interval => INTERVAL '7 days', if_not_exists => TRUE);
CREATE INDEX idx_agent_activity_agent_ts ON agent_activity(agent_id, ts DESC);
CREATE INDEX idx_agent_activity_type_ts ON agent_activity(activity_type, ts DESC);
CREATE INDEX idx_agent_activity_entity ON agent_activity(entity_id, ts DESC);
CREATE INDEX idx_agent_activity_correlation ON agent_activity(correlation_id);
DO $$ BEGIN
PERFORM add_retention_policy('agent_activity', INTERVAL '90 days');
EXCEPTION WHEN OTHERS THEN NULL;
END $$;
-- ─── Ledger view (R3-11: not a fourth write path) ────────────────────
CREATE VIEW ledger AS
SELECT e.created_at AS ts, e.entity_id AS execution_id, e.target_entity_id,
e.action, e.risk_class, e.status, e.verified,
c.route, c.reasoning, a.status AS approval_status, a.decided_by,
e.agent_id, e.correlation_id
FROM executions e
LEFT JOIN classifications c ON c.entity_id = e.classification_id
LEFT JOIN approvals a ON a.entity_id = e.approval_id;

View File

@@ -0,0 +1,18 @@
-- Migration 007: one current edge per (source, target, type).
-- The seed upsert previously conflicted on (source_id, target_id, type,
-- valid_from) — valid_from is now() at insert, so the conflict never fired
-- and every re-ingest duplicated all current edges. Dedupe (keep earliest
-- valid_from), then enforce uniqueness on current edges with a partial
-- unique index the upsert can target.
DELETE FROM relationships r
USING relationships keep
WHERE r.valid_to IS NULL AND keep.valid_to IS NULL
AND r.source_id = keep.source_id
AND r.target_id = keep.target_id
AND r.type = keep.type
AND r.valid_from > keep.valid_from;
CREATE UNIQUE INDEX IF NOT EXISTS uq_rel_current
ON relationships(source_id, target_id, type)
WHERE valid_to IS NULL;

View File

@@ -0,0 +1,24 @@
-- Migration 008: post-commit event fan-out for the SSE stream (SG8/SG10).
-- Events are INSERTed in the same transaction as the state change; pg_notify
-- fires at COMMIT, so subscribers only ever see committed events. Payload is
-- kept minimal (NOTIFY has an 8000-byte limit); consumers needing the full
-- event fetch it by id.
CREATE OR REPLACE FUNCTION notify_oikos_event() RETURNS trigger AS $$
BEGIN
PERFORM pg_notify('oikos_events', json_build_object(
'id', NEW.id,
'ts', NEW.ts,
'type', NEW.type,
'entity_id', NEW.entity_id,
'severity', NEW.severity,
'source', NEW.source,
'correlation_id', NEW.correlation_id
)::text);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS trg_events_notify ON events;
CREATE TRIGGER trg_events_notify AFTER INSERT ON events
FOR EACH ROW EXECUTE FUNCTION notify_oikos_event();

View File

@@ -0,0 +1,15 @@
-- Migration 009: Knowledge entities (FTS search for documentation)
-- Creates the knowledge_entities table for Phase 3 search/knowledge endpoints.
CREATE TABLE IF NOT EXISTS knowledge_entities (
entity_id UUID PRIMARY KEY REFERENCES entities(id),
title TEXT NOT NULL,
content TEXT NOT NULL,
source TEXT,
tags TEXT[],
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_knowledge_title ON knowledge_entities USING gin(to_tsvector('english', title));
CREATE INDEX idx_knowledge_content ON knowledge_entities USING gin(to_tsvector('english', content));

7
migrations/embed.go Normal file
View File

@@ -0,0 +1,7 @@
// Package migrations embeds SQL migration files for use by the db package.
package migrations
import "embed"
//go:embed *.up.sql
var FS embed.FS

284
scripts/validate-seeds.py Normal file
View File

@@ -0,0 +1,284 @@
#!/usr/bin/env python3
"""Validate seeds/*.yaml against the Oikos meta-schema (Phase 0 gate).
Checks the same invariants the Go ingest (internal/ontology) will enforce:
ontology.yaml
- every entity type's parent exists; hierarchy is acyclic
- layer/cardinality values are legal
- lifecycle references exist; default/terminal/transition states are declared
- relationship endpoint types exist (may be abstract)
- inverse names don't collide with forward names
inventory.yaml
- slugs unique and well-formed (<prefix>:<name>)
- entity types exist and are NOT abstract
- states are legal for the type's lifecycle (walking up the hierarchy for
the lifecycle definition is not needed — lifecycle binds per type)
- relationship endpoints exist; edge type exists; endpoint entity types
are the declared source/target types or descendants (hierarchy walk)
- cardinality: one-to-one / one-to-many / many-to-one uniqueness holds
within the seed
- attributes validate against attribute_schema (if jsonschema installed)
policy.yaml
- approval_rules reference existing risk classes / entity types / entities
- autonomy values sane
Exit 0 = clean (warnings allowed), 1 = errors.
"""
import sys
from pathlib import Path
try:
import yaml
except ImportError:
print("ERROR: PyYAML required (pip install pyyaml)")
sys.exit(1)
try:
import jsonschema
HAVE_JSONSCHEMA = True
except ImportError:
HAVE_JSONSCHEMA = False
SEEDS = Path(__file__).resolve().parent.parent / "seeds"
LAYERS = {"meta", "infrastructure", "governance", "cognition"}
CARDINALITIES = {"one-to-one", "one-to-many", "many-to-one", "many-to-many"}
errors: list[str] = []
warnings: list[str] = []
def err(msg: str) -> None:
errors.append(msg)
def warn(msg: str) -> None:
warnings.append(msg)
def load(name: str) -> dict:
path = SEEDS / name
if not path.exists():
err(f"{name}: file missing")
return {}
with open(path) as f:
try:
return yaml.safe_load(f) or {}
except yaml.YAMLError as e:
err(f"{name}: YAML parse error: {e}")
return {}
# ─── ontology.yaml ────────────────────────────────────────────────────
onto = load("ontology.yaml")
etypes: dict = onto.get("entity_types", {})
rtypes: dict = onto.get("relationship_types", {})
lifecycles: dict = onto.get("lifecycles", {})
for name, lc in lifecycles.items():
states = lc.get("states", [])
if not states:
err(f"lifecycle {name}: no states")
continue
if lc.get("default_state") not in states:
err(f"lifecycle {name}: default_state {lc.get('default_state')!r} not in states")
for t in lc.get("terminal_states", []):
if t not in states:
err(f"lifecycle {name}: terminal state {t!r} not in states")
for frm, tos in (lc.get("transitions") or {}).items():
if frm not in states:
err(f"lifecycle {name}: transition source {frm!r} not in states")
for to, spec in (tos or {}).items():
if to not in states:
err(f"lifecycle {name}: transition target {to!r} not in states")
if spec is not None and not isinstance(spec.get("requires", []), list):
err(f"lifecycle {name}: {frm}->{to} requires must be a list")
for name, et in etypes.items():
parent = et.get("parent")
if parent is not None and parent not in etypes:
err(f"entity type {name}: parent {parent!r} not defined")
layer = et.get("layer")
if layer not in LAYERS:
err(f"entity type {name}: layer {layer!r} invalid (want {sorted(LAYERS)})")
lc = et.get("lifecycle")
if lc is not None and lc not in lifecycles:
err(f"entity type {name}: lifecycle {lc!r} not defined")
schema = et.get("attributes")
if schema is not None and HAVE_JSONSCHEMA:
try:
jsonschema.Draft202012Validator.check_schema(schema)
except jsonschema.SchemaError as e:
err(f"entity type {name}: attribute_schema is not valid JSON Schema: {e.message}")
# hierarchy acyclicity + ancestor helper
def ancestors(t: str) -> list[str]:
chain, seen = [], set()
cur = t
while cur is not None:
if cur in seen:
err(f"entity type hierarchy cycle at {cur!r}")
break
seen.add(cur)
chain.append(cur)
cur = etypes.get(cur, {}).get("parent")
return chain
for name in etypes:
ancestors(name)
def is_a(t: str, target: str) -> bool:
return target in ancestors(t)
fwd_names = set(rtypes)
for name, rt in rtypes.items():
for endpoint in ("source", "target"):
v = rt.get(endpoint)
if v not in etypes:
err(f"relationship type {name}: {endpoint} {v!r} not a defined entity type")
if rt.get("cardinality") not in CARDINALITIES:
err(f"relationship type {name}: cardinality {rt.get('cardinality')!r} invalid")
inv = rt.get("inverse")
if inv and inv in fwd_names:
err(f"relationship type {name}: inverse {inv!r} collides with a forward name")
# ─── inventory.yaml ───────────────────────────────────────────────────
inv = load("inventory.yaml")
entities: list = inv.get("entities", [])
relationships: list = inv.get("relationships", [])
slugs: dict = {}
for e in entities:
slug = e.get("slug")
if not slug or ":" not in slug:
err(f"entity {e}: slug missing or not '<prefix>:<name>'")
continue
if slug in slugs:
err(f"duplicate slug {slug!r}")
slugs[slug] = e
t = e.get("type")
if t not in etypes:
err(f"{slug}: type {t!r} not in ontology")
continue
if etypes[t].get("abstract"):
err(f"{slug}: type {t!r} is abstract — cannot be instantiated")
if not e.get("name"):
err(f"{slug}: name missing")
state = e.get("state")
lc_name = etypes[t].get("lifecycle")
if state is not None:
if lc_name is None:
err(f"{slug}: has state {state!r} but type {t!r} has no lifecycle")
elif state not in lifecycles.get(lc_name, {}).get("states", []):
err(f"{slug}: state {state!r} not in lifecycle {lc_name!r}")
schema = etypes[t].get("attributes")
if schema and HAVE_JSONSCHEMA and e.get("attributes"):
v = jsonschema.Draft202012Validator(schema)
for ve in v.iter_errors(e["attributes"]):
warn(f"{slug}: attributes: {ve.message}")
# uniqueness of (type, name)
seen_tn = set()
for e in entities:
tn = (e.get("type"), e.get("name"))
if tn in seen_tn:
err(f"duplicate (type, name): {tn}")
seen_tn.add(tn)
edge_keys = set()
by_card_src: dict = {}
by_card_tgt: dict = {}
for r in relationships:
src, tgt, rt_name = r.get("source"), r.get("target"), r.get("type")
ctx = f"edge {src} -{rt_name}-> {tgt}"
if rt_name not in rtypes:
err(f"{ctx}: relationship type not in ontology")
continue
ok = True
for label, slug in (("source", src), ("target", tgt)):
if slug not in slugs:
err(f"{ctx}: {label} entity {slug!r} not in inventory")
ok = False
if not ok:
continue
key = (src, tgt, rt_name)
if key in edge_keys:
err(f"{ctx}: duplicate edge")
edge_keys.add(key)
rt = rtypes[rt_name]
for label, slug, want in (("source", src, rt["source"]), ("target", tgt, rt["target"])):
actual = slugs[slug]["type"]
if not is_a(actual, want):
err(f"{ctx}: {label} type {actual!r} is not a {want!r} (or descendant)")
# cardinality bookkeeping (source→target multiplicity)
card = rt.get("cardinality")
if card in ("one-to-one", "many-to-one"):
# each source has at most one outgoing edge of this type
k = (rt_name, src)
if k in by_card_src:
err(f"{ctx}: cardinality {card} — source {src!r} already has a {rt_name!r} edge")
by_card_src[k] = tgt
if card in ("one-to-one", "one-to-many"):
# each target has at most one incoming edge of this type
k = (rt_name, tgt)
if k in by_card_tgt:
err(f"{ctx}: cardinality {card} — target {tgt!r} already has a {rt_name!r} edge")
by_card_tgt[k] = src
# ─── policy.yaml ──────────────────────────────────────────────────────
pol = load("policy.yaml")
rclasses: dict = pol.get("risk_classes", {})
for name, rc in rclasses.items():
if rc.get("approval_required") not in ("none", "operator", "operator_confirmed"):
err(f"risk class {name}: approval_required invalid")
rules = pol.get("approval_rules", [])
seen_rules = set()
for rule in rules:
ctx = f"rule ({rule.get('entity_type')}, {rule.get('action')}, {rule.get('scope_entity')})"
if rule.get("risk_class") not in rclasses:
err(f"{ctx}: risk_class {rule.get('risk_class')!r} not defined")
if rule.get("autonomy_level") not in ("auto", "escalate", "never"):
err(f"{ctx}: autonomy_level invalid")
et = rule.get("entity_type")
if et is not None and et not in etypes:
err(f"{ctx}: entity_type {et!r} not in ontology")
scope = rule.get("scope_entity")
if scope is not None and scope not in slugs:
err(f"{ctx}: scope_entity {scope!r} not in inventory")
key = (et, rule.get("action"), scope)
if key in seen_rules:
err(f"{ctx}: duplicate rule")
seen_rules.add(key)
# rule sanity: autonomy 'auto' requires the class to allow autonomy
rc = rclasses.get(rule.get("risk_class"), {})
if rule.get("autonomy_level") == "auto" and not rc.get("autonomy_allowed"):
err(f"{ctx}: autonomy_level=auto but risk class forbids autonomy")
auto = pol.get("autonomy_settings", {})
ga = auto.get("global.auto_act")
if ga not in ("off", "reversible_low"):
err(f"autonomy_settings: global.auto_act {ga!r} invalid (off | reversible_low)")
for k in auto:
if k.startswith("never_auto_act."):
slug = k.split(".", 1)[1]
if slug not in slugs:
warn(f"autonomy_settings: {k} references unknown entity {slug!r}")
# ─── report ───────────────────────────────────────────────────────────
print(f"entity types: {len(etypes)} ({sum(1 for e in etypes.values() if e.get('abstract'))} abstract)")
print(f"relationship types: {len(rtypes)}")
print(f"lifecycles: {len(lifecycles)}")
print(f"entities: {len(entities)}")
print(f"relationships: {len(relationships)}")
print(f"risk classes: {len(rclasses)}, rules: {len(rules)}")
if not HAVE_JSONSCHEMA:
warn("jsonschema not installed — attribute schema validation skipped")
for w in warnings:
print(f"WARN {w}")
for e in errors:
print(f"ERROR {e}")
print(f"\n{'FAIL' if errors else 'OK'}{len(errors)} error(s), {len(warnings)} warning(s)")
sys.exit(1 if errors else 0)

539
seeds/inventory.yaml Normal file
View File

@@ -0,0 +1,539 @@
# Oikos inventory seed — entity instances + relationships.
#
# Translated from the legacy /inventory.yaml (2026-07-07). Bootstraps the
# entities/relationships tables (migration 002); after ingest the DB is
# authoritative and this file is regenerated by `GET /api/v1/export`.
#
# Slug conventions: <prefix>:<name> —
# host: (proxmox-host, standalone-server) · ws: (workstation) · lxc: · vm:
# service: · ingress: · repo: (config-repo) · pool: · volume: · mesh: · lan:
# zone: (dns-zone) · idp: · person: · agent: · cluster: · backup:
#
# `state:` omitted = the type's lifecycle default (active).
# Mount details (mount_point) are attributes on `mounts` edges.
# Known thin spots are marked # THIN: backfill later.
version: 1
entities:
# ─── Sites, networks ───────────────────────────────────────────────
- {slug: "site:home", type: site, name: home}
- {slug: "site:ionos-dc", type: site, name: ionos-dc,
attributes: {address: IONOS datacenter (VPS)}}
- slug: "lan:lab"
type: lan
name: lab
attributes: {subnet: 192.168.8.0/24}
- slug: "lan:household"
type: lan
name: household
attributes: {subnet: 192.168.178.0/24} # Fritz LAN; static route to lab subnet
- slug: "mesh:netbird"
type: mesh
name: netbird
attributes:
provider: netbird
subnet: 100.122.0.0/16
domain: netbird.selfhosted
- slug: "mesh:tailscale"
type: mesh
name: tailscale
state: deprecated # migration to netbird in progress (infrastructure/mesh.md)
attributes: {provider: tailscale}
- slug: "zone:hubris.network"
type: dns-zone
name: hubris.network
attributes: {zone: hubris.network, authority: "Technitium (LXC 107), split-horizon"}
- slug: "zone:netbird.selfhosted"
type: dns-zone
name: netbird.selfhosted
attributes: {zone: netbird.selfhosted, authority: netbird-mgmt}
# ─── Machines ──────────────────────────────────────────────────────
- slug: "cluster:homelab"
type: cluster
name: Homelab
attributes: {quorum: "2-node, no QDevice tiebreaker yet"}
- slug: "host:hubris"
type: proxmox-host
name: hubris
attributes:
os: linux
lan_ip: 192.168.8.77
mesh: {netbird: {ip: 100.122.38.109, fqdn: proxmox-server.netbird.selfhosted}}
ssh: {port: 22, netbird_port: 22022, user: root}
age_pubkey: age1xkklkvnk5z0fsnh6cfgv70hy9ksfy8rdprwerzw4yk3p4p7cxcqs2yvpz6
- slug: "host:strong"
type: proxmox-host
name: strong
attributes:
os: linux
lan_ip: 192.168.178.181
ssh: {user: root}
age_pubkey: age1rtwvdct6avjkr3cyxv3vue3vqx4d524fjfr3vk7xrnvyrylnry5sm54sn4
note: >-
PVE 9.2.3 since 2026-07-01 (formerly workstation ludo-mini). Joined
Homelab cluster same day. Not yet netbird-enrolled — reachable via
household LAN / Fritz static route only.
- slug: "host:netbird-vps"
type: standalone-server
name: netbird-vps
attributes:
os: linux
provider: ionos
control_level: partial # managed via ssh from hubris; not a homelab client
public_ipv4: 82.165.190.79
mesh: {netbird: {ip: 100.122.165.149, fqdn: netbird-ionos.netbird.selfhosted}}
ssh: {user: root}
note: netbird mgmt+signal+relay+dashboard + coturn; sshd locked to hubris pubkey
- slug: "ws:mac-mini"
type: workstation
name: mac-mini
attributes:
os: macos
user: dtoro
lan_ip: 192.168.178.182
mesh: {netbird: {fqdn: mac-mini-234-17.netbird.selfhosted}}
age_pubkey: age1z62ff2ak9zj5ctcvaxwyyhedwjvlwgm2dkn9nk3wrwk8fkavcpmsqwc2vs
note: only macOS in fleet; future Oikos OS Docker host
- slug: "ws:republic-laptop"
type: workstation
name: republic-laptop
attributes:
os: linux
user: dtoro
mesh: {netbird: {fqdn: republic-laptop.netbird.selfhosted}}
# ─── LXCs ──────────────────────────────────────────────────────────
- {slug: "lxc:jellyfin", type: lxc, name: jellyfin,
attributes: {pve_id: 101, role: media-server, lan_ip: 192.168.8.246,
public_host: media.hubris.network,
note: "VAAPI transcode via Radeon 680M passthrough; migrated hubris→strong 2026-07-05"}}
- {slug: "lxc:nfs-export", type: lxc, name: nfs-export,
attributes: {pve_id: 102, role: storage-export, lan_ip: 192.168.8.200}}
- {slug: "lxc:paperless", type: lxc, name: paperless,
attributes: {pve_id: 103, role: document-archive, lan_ip: 192.168.8.130,
public_host: paperless.hubris.network}}
- {slug: "lxc:gitea", type: lxc, name: gitea,
attributes: {pve_id: 104, role: git-server, lan_ip: 192.168.8.121,
public_host: git.hubris.network,
note: "bare repos at /mnt/library/repos/dtoro/*.git"}}
- {slug: "lxc:apps", type: lxc, name: apps,
attributes: {pve_id: 105, role: docker-apps, lan_ip: 192.168.8.205,
age_pubkey: age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0,
note: "legacy Oikos host; fallback during cutover (plan A6)"}}
- {slug: "lxc:auth-outpost", type: lxc, name: auth-outpost,
attributes: {pve_id: 106, role: authentik-gateway, lan_ip: 192.168.8.6}}
- {slug: "lxc:dns", type: lxc, name: dns,
attributes: {pve_id: 107, role: dns-server, lan_ip: 192.168.8.2}}
- {slug: "lxc:nextcloud", type: lxc, name: nextcloud,
attributes: {pve_id: 114, role: file-sync, lan_ip: 192.168.8.224,
public_host: cloud.hubris.network}}
- {slug: "lxc:elementsynapse", type: lxc, name: elementsynapse,
attributes: {pve_id: 118, role: matrix-server, lan_ip: 192.168.8.242,
public_host: matrix.hubris.network,
note: "migrated hubris→strong 2026-07-05"}}
- {slug: "lxc:sophia", type: lxc, name: sophia,
attributes: {pve_id: 119, role: workshop, lan_ip: 192.168.8.109}}
- {slug: "lxc:mule-images", type: lxc, name: mule-images,
attributes: {pve_id: 120, role: photo-management, lan_ip: 192.168.8.136,
public_host: photos.hubris.network}}
- {slug: "lxc:caddy", type: lxc, name: caddy,
attributes: {pve_id: 121, role: reverse-proxy, lan_ip: 192.168.8.175,
note: "terminates all *.hubris.network; /etc/caddy is a checkout of dtoro/caddy-conf"}}
- {slug: "lxc:arriman", type: lxc, name: arriman,
attributes: {pve_id: 122, role: arr-stack, lan_ip: 192.168.8.245,
public_hosts: [jellyseerr.hubris.network, qbit.hubris.network, sab.hubris.network],
note: "homarr/radarr/sonarr/lidarr/sab/qbit/bazarr/flaresolverr/prowlarr/jellyseerr; migrated to strong 2026-07-05"}}
- {slug: "lxc:trmnl", type: lxc, name: trmnl,
attributes: {pve_id: 128, role: trmnl-middleware, lan_ip: 192.168.8.211,
public_host: trmnl.hubris.network,
note: "not yet mesh/SOPS-enrolled"}}
- {slug: "lxc:house", type: lxc, name: house,
attributes: {pve_id: 129, role: family-planner, lan_ip: 192.168.8.244,
public_host: house.hubris.network,
age_pubkey: age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h,
note: "Yuvomi + WebDAV bridge to paperless; migrated to strong 2026-07-05"}}
- {slug: "lxc:grimmory", type: lxc, name: grimmory,
attributes: {pve_id: 130, role: book-library, lan_ip: 192.168.8.247,
public_host: books.hubris.network,
age_pubkey: age1uellsemnjrzgfg9fxw4jefpy05laxzggwnwhh6ny3wl7alyp6v8q0muxet}}
- {slug: "lxc:teddycloud", type: lxc, name: teddycloud,
attributes: {pve_id: 131, role: teddycloud, lan_ip: 192.168.8.150,
public_host: teddy.hubris.network,
note: "drift-caught 2026-07-06; no forward-auth gate on route; not a homelab client"}}
- {slug: "lxc:rclone", type: lxc, name: rclone,
attributes: {pve_id: 132, role: backup,
mesh: {netbird: {fqdn: rclone.netbird.selfhosted}},
age_pubkey: age1pwtdws2thdh7vzp2dzttl3zxgcs2tgpcsjsqgw3q04nyml4kvuqq467u4x}}
# verified live on hubris 2026-07-07 (pct list via MCP)
- {slug: "lxc:seanime", type: lxc, name: seanime,
attributes: {pve_id: 133, role: anime-media-server, lan_ip: 192.168.8.248,
public_host: seanime.hubris.network,
note: "systemd service at /opt/seanime; uses qbittorrent on arriman"}}
- {slug: "lxc:romm", type: lxc, name: romm,
attributes: {pve_id: 134, role: rom-manager, lan_ip: 192.168.8.249,
public_host: roms.hubris.network,
note: "docker compose + MariaDB sidecar at /opt/romm"}}
# ─── VMs ───────────────────────────────────────────────────────────
- {slug: "vm:zimaos", type: vm, name: zimaos,
attributes: {pve_id: 100, role: nas-frontend-eval, lan_ip: 192.168.8.195,
public_host: zimaos.hubris.network}}
- {slug: "vm:haos", type: vm, name: haos,
attributes: {pve_id: 108, role: home-automation, lan_ip: 192.168.8.101}}
# ─── Storage ───────────────────────────────────────────────────────
- {slug: "pool:local-lvm-hubris", type: storage-pool, name: local-lvm (hubris),
attributes: {type: lvm}}
- {slug: "pool:library-hubris", type: storage-pool, name: library (hubris),
attributes: {type: lvmthin, capacity_gb: 3700}}
# 2nd Samsung 990 EVO Plus NVMe; backs /mnt/library ext4 via
# /dev/mapper/library-library (knowledge/wiki/hosts/hubris.md)
- {slug: "pool:ludo-lvm", type: storage-pool, name: ludo-lvm (strong),
attributes: {type: lvm}}
- {slug: "volume:library", type: volume, name: library,
attributes: {path: /mnt/library, size_gb: 3700}}
- {slug: "volume:media-local", type: volume, name: media-local,
attributes: {path: /mnt/media_local}}
- {slug: "backup:proton-drive", type: backup-target, name: proton-drive,
attributes: {provider: proton, encrypted: true}}
# ─── Services ──────────────────────────────────────────────────────
- {slug: "service:proxmox-ui", type: service, name: proxmox_ui,
attributes: {url: "https://proxmox.hubris.network", port: 8006,
doc_page: knowledge/wiki/hosts/hubris.md,
risk_notes: "hypervisor UI — changes affect every guest on the node"}}
- {slug: "service:gitea", type: service, name: gitea,
attributes: {url: "https://git.hubris.network", port: 3000,
doc_page: knowledge/wiki/containers/104-gitea.md,
risk_notes: "hosts all config repos + deploy webhooks; outage blocks auto-deploy and sync"}}
- {slug: "service:caddy", type: service, name: caddy,
attributes: {doc_page: knowledge/wiki/containers/121-caddy.md,
risk_notes: "wide blast radius — every *.hubris.network route rides on it"}}
- {slug: "service:authentik", type: service, name: authentik,
attributes: {url: "https://auth.hubris.network",
doc_page: knowledge/wiki/containers/106-auth-outpost.md,
note: "core on VPS since 2026-05-31; LAN outpost = auth-outpost (LXC 106) 192.168.8.6:9000",
risk_notes: "SSO provider — outage locks login to OIDC/forward-auth services"}}
- {slug: "service:dns", type: service, name: dns,
attributes: {doc_page: knowledge/wiki/containers/107-dns.md,
risk_notes: "LAN-wide resolver — misconfig breaks name resolution for every client"}}
- {slug: "service:jellyfin", type: service, name: jellyfin,
attributes: {url: "https://media.hubris.network",
doc_page: knowledge/wiki/containers/101-jellyfin.md,
risk_notes: "native Authentik OIDC (no forward-auth gate); VAAPI depends on GPU passthrough on strong"}}
- {slug: "service:nextcloud", type: service, name: nextcloud,
attributes: {url: "https://cloud.hubris.network",
doc_page: knowledge/wiki/containers/114-nextcloud.md}}
- {slug: "service:paperless", type: service, name: paperless,
attributes: {url: "https://paperless.hubris.network",
doc_page: knowledge/wiki/containers/103-paperless.md,
risk_notes: "document archive — data irreplaceable; DB operations are destructive-class"}}
- {slug: "service:matrix", type: service, name: matrix,
attributes: {url: "https://matrix.hubris.network",
doc_page: knowledge/wiki/containers/118-elementsynapse.md,
risk_notes: "alert/approval channel for Oikos — outage silences agent escalation"}}
- {slug: "service:photos", type: service, name: photos,
attributes: {url: "https://photos.hubris.network",
doc_page: knowledge/wiki/containers/120-mule-images.md}}
- {slug: "service:arr-stack", type: service, name: arr_stack,
attributes: {doc_page: knowledge/wiki/containers/122-arriman.md,
note: "jellyseerr / qbit / sab on docker compose"}}
- {slug: "service:artifacto", type: service, name: artifacto,
attributes: {url: "https://artifacto.hubris.network",
doc_page: knowledge/wiki/containers/105-apps.md}}
- {slug: "service:trmnl", type: service, name: trmnl,
attributes: {url: "https://trmnl.hubris.network",
doc_page: knowledge/wiki/containers/128-trmnl.md,
note: "TRMNL e-ink plugin middleware (polled by TRMNL cloud)"}}
- {slug: "service:zimaos", type: service, name: zimaos,
attributes: {url: "https://zimaos.hubris.network",
doc_page: knowledge/wiki/vms/100-zimaos.md}}
- {slug: "service:haos", type: service, name: haos,
attributes: {doc_page: knowledge/wiki/vms/108-haos.md}}
- {slug: "service:teddycloud", type: service, name: teddycloud,
attributes: {url: "https://teddy.hubris.network",
doc_page: knowledge/wiki/containers/131-teddycloud.md,
risk_notes: "no forward-auth gate — reachable by anyone on LAN/mesh"}}
- {slug: "service:homelab-mcp", type: service, name: homelab_mcp,
attributes: {port: 9810, systemd_unit: homelab-mcp,
endpoint: "https://mcp.hubris.network/mcp",
doc_page: knowledge/wiki/infrastructure/homelab-context.md,
risk_notes: "agents' primary read surface — outage degrades every agent to grepping the clone"}}
- {slug: "service:secrets-issuance", type: service, name: secrets_issuance,
attributes: {port: 9820, systemd_unit: secrets-issuance,
endpoint: "https://secrets.hubris.network/issue",
doc_page: .agents/operations/agent-enrollment.md,
risk_notes: "identity issuance — security-sensitive; key operations are destructive-class"}}
# Services derived from hosts.public_host (no legacy services entry):
- {slug: "service:house", type: service, name: house,
attributes: {url: "https://house.hubris.network", note: "Yuvomi family planner (derived)"}}
- {slug: "service:grimmory", type: service, name: grimmory,
attributes: {url: "https://books.hubris.network", note: derived}}
- {slug: "service:seanime", type: service, name: seanime,
attributes: {url: "https://seanime.hubris.network", port: 43211, note: derived}}
- {slug: "service:romm", type: service, name: romm,
attributes: {url: "https://roms.hubris.network", note: derived}}
- {slug: "service:jellyseerr", type: service, name: jellyseerr,
attributes: {url: "https://jellyseerr.hubris.network", note: derived (arriman)}}
- {slug: "service:qbit", type: service, name: qbit,
attributes: {url: "https://qbit.hubris.network", note: derived (arriman)}}
- {slug: "service:sab", type: service, name: sab,
attributes: {url: "https://sab.hubris.network", note: "derived (arriman); forward-auth gated"}}
# ─── Config repos ──────────────────────────────────────────────────
- {slug: "repo:caddy-conf", type: config-repo, name: dtoro/caddy-conf}
- {slug: "repo:gitea-customizations", type: config-repo, name: dtoro/gitea-customizations}
- {slug: "repo:mule-image", type: config-repo, name: dtoro/mule-image}
- {slug: "repo:artifacto", type: config-repo, name: dtoro/Artifacto}
- {slug: "repo:terminalito", type: config-repo, name: dtoro/terminalito}
- {slug: "repo:homelab-docs", type: config-repo, name: dtoro/Homelab-Docs}
# ─── Ingress routes (Caddy, *.hubris.network) ──────────────────────
- {slug: "ingress:proxmox.hubris.network", type: ingress-route, name: proxmox.hubris.network}
- {slug: "ingress:git.hubris.network", type: ingress-route, name: git.hubris.network}
- {slug: "ingress:auth.hubris.network", type: ingress-route, name: auth.hubris.network}
- {slug: "ingress:media.hubris.network", type: ingress-route, name: media.hubris.network}
- {slug: "ingress:cloud.hubris.network", type: ingress-route, name: cloud.hubris.network}
- {slug: "ingress:paperless.hubris.network", type: ingress-route, name: paperless.hubris.network,
attributes: {forward_auth: true}}
- {slug: "ingress:matrix.hubris.network", type: ingress-route, name: matrix.hubris.network}
- {slug: "ingress:photos.hubris.network", type: ingress-route, name: photos.hubris.network}
- {slug: "ingress:artifacto.hubris.network", type: ingress-route, name: artifacto.hubris.network}
- {slug: "ingress:trmnl.hubris.network", type: ingress-route, name: trmnl.hubris.network}
- {slug: "ingress:zimaos.hubris.network", type: ingress-route, name: zimaos.hubris.network}
- {slug: "ingress:teddy.hubris.network", type: ingress-route, name: teddy.hubris.network,
attributes: {forward_auth: false}}
- {slug: "ingress:mcp.hubris.network", type: ingress-route, name: mcp.hubris.network}
- {slug: "ingress:secrets.hubris.network", type: ingress-route, name: secrets.hubris.network}
- {slug: "ingress:house.hubris.network", type: ingress-route, name: house.hubris.network}
- {slug: "ingress:books.hubris.network", type: ingress-route, name: books.hubris.network}
- {slug: "ingress:seanime.hubris.network", type: ingress-route, name: seanime.hubris.network}
- {slug: "ingress:roms.hubris.network", type: ingress-route, name: roms.hubris.network}
- {slug: "ingress:jellyseerr.hubris.network", type: ingress-route, name: jellyseerr.hubris.network}
- {slug: "ingress:qbit.hubris.network", type: ingress-route, name: qbit.hubris.network}
- {slug: "ingress:sab.hubris.network", type: ingress-route, name: sab.hubris.network,
attributes: {forward_auth: true}}
# ─── Governance ────────────────────────────────────────────────────
- {slug: "person:dtoro", type: person, name: dtoro,
attributes: {matrix_id: "@dtoro:avispero"}}
- {slug: "idp:authentik", type: identity-provider, name: authentik,
attributes: {issuer: "https://auth.hubris.network", auth_mode: both}}
- {slug: "agent:hermes", type: agent, name: hermes,
state: planned,
attributes: {gateway_port: 8092, note: "Oikos Phase 4 — Docker gateway mode"}}
- {slug: "agent:oikos", type: agent, name: oikos,
state: planned,
attributes: {note: "the OS control loop itself (scheduler/actuator) as an actor"}}
# ─── Archaeology (state: destroyed — kept for "what happened to X?") ─
- {slug: "lxc:claudio-bot", type: lxc, name: claudio-bot, state: destroyed,
attributes: {pve_id: 123, destroyed: "2026-06-04", reason: "replaced by Hermes Agent on mac-mini"}}
- {slug: "lxc:plato", type: lxc, name: plato, state: destroyed,
attributes: {pve_id: 126, destroyed: "2026-06-28", reason: "notes workspace decommissioned; data at /mnt/library/documents/plato"}}
- {slug: "lxc:mule-photos-new", type: lxc, name: mule-photos-new, state: destroyed,
attributes: {pve_id: 127, destroyed: "2026-05-22", reason: "PhotoPrism test stack promoted to LXC 120"}}
- {slug: "lxc:heaper", type: lxc, name: heaper, state: destroyed,
attributes: {pve_id: 116, destroyed: "2026-05-14", reason: "decommissioned; data at /mnt/library/heaper"}}
- {slug: "lxc:syncthing", type: lxc, name: syncthing, state: destroyed,
attributes: {pve_id: 109, destroyed: "2026-05-14", reason: "decommissioned; library subtree was empty"}}
- {slug: "lxc:seafile", type: lxc, name: seafile, state: destroyed,
attributes: {pve_id: 125, destroyed: "2026-05-13", reason: "Seafile Pro evaluation rejected"}}
- {slug: "lxc:arr-yunohost", type: lxc, name: arr-yunohost, state: destroyed,
attributes: {pve_id: 100, destroyed: "2026-04-28", reason: "migrated to docker stack on arriman (LXC 122)"}}
- {slug: "lxc:flaresolverr", type: lxc, name: flaresolverr, state: destroyed,
attributes: {pve_id: 106, destroyed: "2026-04-28", reason: "folded into the arriman docker compose"}}
- {slug: "lxc:marimo", type: lxc, name: marimo, state: destroyed,
attributes: {pve_id: 107, destroyed: "2026-04-28", reason: decommissioned}}
- {slug: "lxc:photoprism", type: lxc, name: photoprism, state: destroyed,
attributes: {pve_id: 110, destroyed: "2026-04-28", reason: "replaced by mule-images (LXC 120)"}}
- {slug: "lxc:karakeep", type: lxc, name: karakeep, state: destroyed,
attributes: {pve_id: 111, destroyed: "2026-04-28", reason: decommissioned}}
- {slug: "lxc:immich", type: lxc, name: immich, state: destroyed,
attributes: {pve_id: 112, destroyed: "2026-04-28", reason: "replaced by mule-images (LXC 120)"}}
- {slug: "lxc:reticulum", type: lxc, name: reticulum, state: destroyed,
attributes: {pve_id: 115, destroyed: "2026-04-28", reason: decommissioned}}
relationships:
# ─── Cluster membership ────────────────────────────────────────────
- {source: "host:hubris", target: "cluster:homelab", type: member-of}
- {source: "host:strong", target: "cluster:homelab", type: member-of}
# ─── Location ──────────────────────────────────────────────────────
- {source: "host:hubris", target: "site:home", type: located-at}
- {source: "host:strong", target: "site:home", type: located-at}
- {source: "ws:mac-mini", target: "site:home", type: located-at}
- {source: "host:netbird-vps", target: "site:ionos-dc", type: located-at}
# ─── Hosting (machine → guest) ─────────────────────────────────────
- {source: "host:hubris", target: "lxc:nfs-export", type: hosts}
- {source: "host:hubris", target: "lxc:paperless", type: hosts}
- {source: "host:hubris", target: "lxc:gitea", type: hosts}
- {source: "host:hubris", target: "lxc:apps", type: hosts}
- {source: "host:hubris", target: "lxc:auth-outpost", type: hosts}
- {source: "host:hubris", target: "lxc:dns", type: hosts}
- {source: "host:hubris", target: "lxc:nextcloud", type: hosts}
- {source: "host:hubris", target: "lxc:sophia", type: hosts}
- {source: "host:hubris", target: "lxc:mule-images", type: hosts}
- {source: "host:hubris", target: "lxc:caddy", type: hosts}
- {source: "host:hubris", target: "lxc:trmnl", type: hosts}
- {source: "host:hubris", target: "lxc:teddycloud", type: hosts}
- {source: "host:hubris", target: "vm:zimaos", type: hosts}
- {source: "host:hubris", target: "vm:haos", type: hosts}
- {source: "host:strong", target: "lxc:jellyfin", type: hosts}
- {source: "host:strong", target: "lxc:elementsynapse", type: hosts}
- {source: "host:strong", target: "lxc:arriman", type: hosts}
- {source: "host:strong", target: "lxc:house", type: hosts}
- {source: "host:strong", target: "lxc:grimmory", type: hosts}
- {source: "host:strong", target: "lxc:seanime", type: hosts}
- {source: "host:strong", target: "lxc:romm", type: hosts}
- {source: "host:hubris", target: "lxc:rclone", type: hosts}
# ─── Service provision (compute → service) ─────────────────────────
- {source: "host:hubris", target: "service:proxmox-ui", type: provides}
- {source: "lxc:gitea", target: "service:gitea", type: provides}
- {source: "lxc:caddy", target: "service:caddy", type: provides}
- {source: "host:netbird-vps", target: "service:authentik", type: provides}
- {source: "lxc:dns", target: "service:dns", type: provides}
- {source: "lxc:jellyfin", target: "service:jellyfin", type: provides}
- {source: "lxc:nextcloud", target: "service:nextcloud", type: provides}
- {source: "lxc:paperless", target: "service:paperless", type: provides}
- {source: "lxc:elementsynapse", target: "service:matrix", type: provides}
- {source: "lxc:mule-images", target: "service:photos", type: provides}
- {source: "lxc:arriman", target: "service:arr-stack", type: provides}
- {source: "lxc:arriman", target: "service:jellyseerr", type: provides}
- {source: "lxc:arriman", target: "service:qbit", type: provides}
- {source: "lxc:arriman", target: "service:sab", type: provides}
- {source: "lxc:apps", target: "service:artifacto", type: provides}
- {source: "lxc:apps", target: "service:homelab-mcp", type: provides}
- {source: "lxc:apps", target: "service:secrets-issuance", type: provides}
- {source: "lxc:trmnl", target: "service:trmnl", type: provides}
- {source: "vm:zimaos", target: "service:zimaos", type: provides}
- {source: "vm:haos", target: "service:haos", type: provides}
- {source: "lxc:teddycloud", target: "service:teddycloud", type: provides}
- {source: "lxc:house", target: "service:house", type: provides}
- {source: "lxc:grimmory", target: "service:grimmory", type: provides}
- {source: "lxc:seanime", target: "service:seanime", type: provides}
- {source: "lxc:romm", target: "service:romm", type: provides}
# ─── Ingress → service ─────────────────────────────────────────────
- {source: "ingress:proxmox.hubris.network", target: "service:proxmox-ui", type: routes-to}
- {source: "ingress:git.hubris.network", target: "service:gitea", type: routes-to}
- {source: "ingress:auth.hubris.network", target: "service:authentik", type: routes-to}
- {source: "ingress:media.hubris.network", target: "service:jellyfin", type: routes-to}
- {source: "ingress:cloud.hubris.network", target: "service:nextcloud", type: routes-to}
- {source: "ingress:paperless.hubris.network", target: "service:paperless", type: routes-to}
- {source: "ingress:matrix.hubris.network", target: "service:matrix", type: routes-to}
- {source: "ingress:photos.hubris.network", target: "service:photos", type: routes-to}
- {source: "ingress:artifacto.hubris.network", target: "service:artifacto", type: routes-to}
- {source: "ingress:trmnl.hubris.network", target: "service:trmnl", type: routes-to}
- {source: "ingress:zimaos.hubris.network", target: "service:zimaos", type: routes-to}
- {source: "ingress:teddy.hubris.network", target: "service:teddycloud", type: routes-to}
- {source: "ingress:mcp.hubris.network", target: "service:homelab-mcp", type: routes-to}
- {source: "ingress:secrets.hubris.network", target: "service:secrets-issuance", type: routes-to}
- {source: "ingress:house.hubris.network", target: "service:house", type: routes-to}
- {source: "ingress:books.hubris.network", target: "service:grimmory", type: routes-to}
- {source: "ingress:seanime.hubris.network", target: "service:seanime", type: routes-to}
- {source: "ingress:roms.hubris.network", target: "service:romm", type: routes-to}
- {source: "ingress:jellyseerr.hubris.network", target: "service:jellyseerr", type: routes-to}
- {source: "ingress:qbit.hubris.network", target: "service:qbit", type: routes-to}
- {source: "ingress:sab.hubris.network", target: "service:sab", type: routes-to}
# ─── Auth edges ────────────────────────────────────────────────────
- {source: "ingress:paperless.hubris.network", target: "idp:authentik", type: secured-by}
- {source: "ingress:sab.hubris.network", target: "idp:authentik", type: secured-by}
- {source: "service:jellyfin", target: "idp:authentik", type: authenticates-via}
- {source: "idp:authentik", target: "person:dtoro", type: authenticates}
# ─── Config repos ──────────────────────────────────────────────────
- {source: "service:caddy", target: "repo:caddy-conf", type: configured-by}
- {source: "service:gitea", target: "repo:gitea-customizations", type: configured-by}
- {source: "service:photos", target: "repo:mule-image", type: configured-by}
- {source: "service:artifacto", target: "repo:artifacto", type: configured-by}
- {source: "service:trmnl", target: "repo:terminalito", type: configured-by}
- {source: "service:homelab-mcp", target: "repo:homelab-docs", type: configured-by}
- {source: "service:secrets-issuance", target: "repo:homelab-docs", type: configured-by}
# ─── Service dependencies (blast-radius edges; grow over time) ─────
- {source: "service:paperless", target: "service:authentik", type: depends-on}
- {source: "service:homelab-mcp", target: "service:gitea", type: depends-on}
- {source: "service:jellyseerr", target: "service:jellyfin", type: depends-on}
- {source: "service:seanime", target: "service:qbit", type: depends-on}
- {source: "service:house", target: "service:paperless", type: depends-on}
- {source: "service:sab", target: "service:authentik", type: depends-on}
# ─── Network membership ────────────────────────────────────────────
- {source: "host:hubris", target: "lan:lab", type: connects-via}
- {source: "host:hubris", target: "mesh:netbird", type: connects-via}
- {source: "host:strong", target: "lan:household", type: connects-via}
- {source: "ws:mac-mini", target: "lan:household", type: connects-via}
- {source: "ws:mac-mini", target: "mesh:netbird", type: connects-via}
- {source: "ws:republic-laptop", target: "mesh:netbird", type: connects-via}
- {source: "host:netbird-vps", target: "mesh:netbird", type: connects-via}
- {source: "lxc:rclone", target: "mesh:netbird", type: connects-via}
- {source: "lxc:jellyfin", target: "lan:lab", type: connects-via}
- {source: "lxc:nfs-export", target: "lan:lab", type: connects-via}
- {source: "lxc:paperless", target: "lan:lab", type: connects-via}
- {source: "lxc:gitea", target: "lan:lab", type: connects-via}
- {source: "lxc:apps", target: "lan:lab", type: connects-via}
- {source: "lxc:apps", target: "mesh:tailscale", type: connects-via}
- {source: "lxc:auth-outpost", target: "lan:lab", type: connects-via}
- {source: "lxc:dns", target: "lan:lab", type: connects-via}
- {source: "lxc:nextcloud", target: "lan:lab", type: connects-via}
- {source: "lxc:elementsynapse", target: "lan:lab", type: connects-via}
- {source: "lxc:sophia", target: "lan:lab", type: connects-via}
- {source: "lxc:mule-images", target: "lan:lab", type: connects-via}
- {source: "lxc:caddy", target: "lan:lab", type: connects-via}
- {source: "lxc:arriman", target: "lan:lab", type: connects-via}
- {source: "lxc:trmnl", target: "lan:lab", type: connects-via}
- {source: "lxc:house", target: "lan:lab", type: connects-via}
- {source: "lxc:grimmory", target: "lan:lab", type: connects-via}
- {source: "lxc:teddycloud", target: "lan:lab", type: connects-via}
- {source: "lxc:seanime", target: "lan:lab", type: connects-via}
- {source: "lxc:romm", target: "lan:lab", type: connects-via}
- {source: "vm:zimaos", target: "lan:lab", type: connects-via}
- {source: "vm:haos", target: "lan:lab", type: connects-via}
# ─── Storage ───────────────────────────────────────────────────────
- {source: "pool:ludo-lvm", target: "volume:media-local", type: contains}
- {source: "pool:library-hubris", target: "volume:library", type: contains}
- {source: "host:hubris", target: "volume:library", type: mounts,
attributes: {mount_point: /mnt/library}}
- {source: "lxc:jellyfin", target: "volume:media-local", type: mounts,
attributes: {mount_point: /mnt/media_local}}
- {source: "lxc:paperless", target: "volume:library", type: mounts,
attributes: {mount_point: /mnt/library}}
- {source: "lxc:gitea", target: "volume:library", type: mounts,
attributes: {mount_point: /mnt/library}}
- {source: "lxc:apps", target: "volume:library", type: mounts,
attributes: {mount_point: /mnt/library}}
- {source: "lxc:nextcloud", target: "volume:library", type: mounts,
attributes: {mount_point: /mnt/library}}
- {source: "lxc:sophia", target: "volume:library", type: mounts,
attributes: {mount_point: /mnt/library}}
- {source: "lxc:mule-images", target: "volume:library", type: mounts,
attributes: {mount_point: /mnt/library}}
- {source: "lxc:arriman", target: "volume:media-local", type: mounts,
attributes: {mount_point: /mnt/media_local}}
- {source: "lxc:grimmory", target: "volume:media-local", type: mounts,
attributes: {mount_point: /mnt/media_local}}
- {source: "lxc:teddycloud", target: "volume:library", type: mounts,
attributes: {mount_point: /mnt/library}}
- {source: "lxc:seanime", target: "volume:media-local", type: mounts,
attributes: {mount_point: /mnt/media_local/anime}}
- {source: "lxc:romm", target: "volume:media-local", type: mounts,
attributes: {mount_point: /mnt/media_local}}
- {source: "lxc:jellyfin", target: "pool:ludo-lvm", type: stores-on}
- {source: "lxc:arriman", target: "pool:ludo-lvm", type: stores-on}
- {source: "lxc:grimmory", target: "pool:ludo-lvm", type: stores-on}
- {source: "lxc:romm", target: "pool:ludo-lvm", type: stores-on}
- {source: "lxc:teddycloud", target: "pool:local-lvm-hubris", type: stores-on}
- {source: "lxc:rclone", target: "backup:proton-drive", type: backs-up-to}
# ─── Governance ────────────────────────────────────────────────────
- {source: "person:dtoro", target: "agent:hermes", type: owns}
- {source: "person:dtoro", target: "agent:oikos", type: owns}

936
seeds/ontology.yaml Normal file
View File

@@ -0,0 +1,936 @@
# Oikos ontology seed — the systems model of the homelab.
#
# Bootstraps entity_types / relationship_types / lifecycle_defs on first
# deploy (migration 001). After ingest the DB is authoritative; this file
# is regenerated by `GET /api/v1/export` for DR + version control.
#
# Conventions:
# - entity type names are kebab-case
# - `parent:` builds the is-a hierarchy; `abstract: true` types cannot be
# instantiated (validation walks the hierarchy for relationship
# endpoints and policy rules — plan R3-1)
# - `layer:` one of meta | infrastructure | governance | cognition
# - relationship `cardinality:` describes source→target multiplicity:
# one-to-one | one-to-many | many-to-one | many-to-many
# - relationship endpoints may name abstract types
# - lifecycle transition `requires:` entries are NAMED CHECKS implemented
# in Go (internal/ontology); the DB stores which checks gate a
# transition, the code implements them
# - mount details (mount_point, options) live as ATTRIBUTES on `mounts`
# edges, not as a separate entity type
#
# Rule of completeness: if something can break, be changed, or hold data,
# it has an entity type here and edges to the things it touches.
version: 1
# ─── Lifecycles ────────────────────────────────────────────────────────
lifecycles:
infrastructure:
states: [planned, provisioning, active, migrating, failed, deprecated, destroyed]
default_state: active # legacy inventory entries without state are active
terminal_states: [destroyed]
transitions:
planned:
provisioning: {requires: [inventory-entry, ip-reserved, storage-pool-chosen, doc-page-stub]}
destroyed: {requires: [cancelled-note]}
provisioning:
active: {requires: [age-key-enrolled-if-needed, mesh-joined-if-needed,
ingress-live-if-public, health-check-answering,
doc-page-complete]}
failed: {requires: []}
active:
migrating: {requires: [preflight, backup-verified]}
deprecated: {requires: [replacement-live-or-role-retired]}
failed: {requires: []}
migrating:
active: {requires: [post-verify, caddy-backends-checked, mounts-checked, docs-updated]}
failed: {requires: []}
failed:
active: {requires: [recovery-verified]}
deprecated: {requires: [write-off-note]}
deprecated:
active: {requires: [un-deprecate-note]}
destroyed: {requires: [backups-verified, secrets-revoked-and-rekeyed,
ingress-and-dns-removed, no-inbound-edges,
archaeology-entry]}
signal:
states: [raised, acknowledged, acting, muted, resolved, failed]
default_state: raised
terminal_states: [resolved]
transitions:
raised:
acknowledged: {requires: []}
muted: {requires: [mute-ttl-set]}
resolved: {requires: [condition-cleared]}
acknowledged:
acting: {requires: [classification-exists]}
resolved: {requires: []}
muted: {requires: [mute-ttl-set]}
acting:
resolved: {requires: [verification-passed]}
raised: {requires: [retry-budget-remaining]}
failed: {requires: []}
failed:
acknowledged: {requires: [operator-retry]}
muted:
raised: {requires: [mute-ttl-expired]}
execution:
states: [proposed, approved, auto_approved, denied, expired, executing,
verifying, verified, failed, timed_out, cancelled, rolled_back,
rollback_failed]
default_state: proposed
terminal_states: [verified, failed, denied, expired, cancelled,
rolled_back, rollback_failed]
transitions:
proposed:
approved: {requires: [operator-approval]}
auto_approved: {requires: [autonomy-allows]}
denied: {requires: []}
approved:
executing: {requires: [approval-token-valid]}
expired: {requires: [approval-ttl-elapsed]}
auto_approved:
executing: {requires: []}
executing:
verified: {requires: [verification-passed]}
failed: {requires: []}
timed_out: {requires: []}
cancelled: {requires: [operator-abort]}
timed_out:
verifying: {requires: []} # check if the command completed anyway
verifying:
verified: {requires: [verification-passed]}
failed: {requires: []}
failed:
rolled_back: {requires: [rollback-procedure-exists]}
rollback_failed: {requires: []}
approval:
states: [pending, approved, denied, expired, revoked]
default_state: pending
terminal_states: [denied, expired, revoked]
transitions:
pending:
approved: {requires: [token-verified]}
denied: {requires: []}
expired: {requires: [ttl-elapsed]}
approved:
revoked: {requires: [not-yet-executing]}
pattern:
states: [hypothesized, validated, active, deprecated, invalidated]
default_state: hypothesized
terminal_states: [deprecated, invalidated]
transitions:
hypothesized:
validated: {requires: [evidence-count-5plus, confidence-0.7plus]}
invalidated: {requires: []}
validated:
active: {requires: [operator-approval]} # S4: never automatic
invalidated: {requires: []}
active:
deprecated: {requires: []}
invalidated: {requires: [contradicting-evidence]}
skill:
states: [drafted, tested, active, refined, failed, deprecated]
default_state: drafted
terminal_states: [deprecated]
transitions:
drafted:
tested: {requires: [test-execution-recorded]}
deprecated: {requires: []}
tested:
active: {requires: [operator-approval]}
failed: {requires: []}
failed:
drafted: {requires: []}
active:
refined: {requires: [new-version-created]}
deprecated: {requires: []}
refined:
active: {requires: [operator-approval]}
# ─── Entity types ──────────────────────────────────────────────────────
# domain: physical | compute | network | storage | software | external |
# identity | cognition
entity_types:
# Root
entity:
abstract: true
domain: meta
layer: meta
description: Root abstract type. Relationship endpoints that accept any
entity (documented-by, procedure-for, checks) reference this.
# ── Infrastructure / physical ──
site:
parent: entity
domain: physical
layer: infrastructure
lifecycle: infrastructure
description: Physical location (home, VPS datacenter).
attributes: {type: object, properties: {address: {type: string}}}
ups:
parent: entity
domain: physical
layer: infrastructure
lifecycle: infrastructure
description: Uninterruptible power supply.
attributes: {type: object, properties: {vendor: {type: string}, va: {type: integer}}}
sensor:
parent: entity
domain: physical
layer: infrastructure
lifecycle: infrastructure
description: Environmental sensor.
peripheral:
parent: entity
domain: physical
layer: infrastructure
lifecycle: infrastructure
description: Attached hardware (GPU, e-ink display, dongle).
# ── Infrastructure / compute ──
compute-entity:
parent: entity
abstract: true
domain: compute
layer: infrastructure
description: Anything that executes workloads (machine, VM, container).
machine:
parent: compute-entity
abstract: true
domain: compute
layer: infrastructure
description: Physical machine. Always instantiated as a subtype.
attributes:
type: object
properties:
cpu_arch: {type: string}
ram_gb: {type: number}
os: {type: string, enum: [linux, macos]}
lan_ip: {type: string}
mesh: {type: object}
ssh: {type: object}
age_pubkey: {type: string}
proxmox-host:
parent: machine
domain: compute
layer: infrastructure
lifecycle: infrastructure
description: Machine running Proxmox VE.
attributes:
type: object
properties: {pve_version: {type: string}}
standalone-server:
parent: machine
domain: compute
layer: infrastructure
lifecycle: infrastructure
description: Machine outside PVE management (e.g. external VPS).
attributes:
type: object
properties:
hypervisor: {type: string}
provider: {type: string}
control_level: {type: string, enum: [full, partial, none]}
public_ipv4: {type: string}
workstation:
parent: machine
domain: compute
layer: infrastructure
lifecycle: infrastructure
description: Operator machine (may also host services, e.g. mac-mini).
attributes:
type: object
properties: {user: {type: string}}
appliance:
parent: machine
domain: compute
layer: infrastructure
lifecycle: infrastructure
description: Vendor appliance with limited management access.
attributes:
type: object
properties: {vendor: {type: string}, model: {type: string}}
vm:
parent: compute-entity
domain: compute
layer: infrastructure
lifecycle: infrastructure
description: Virtual machine.
attributes:
type: object
properties:
pve_id: {type: integer}
vcpus: {type: integer}
memory_mb: {type: integer}
lan_ip: {type: string}
public_host: {type: string}
role: {type: string}
container:
parent: compute-entity
abstract: true
domain: compute
layer: infrastructure
description: OS-level container (LXC or Docker).
attributes:
type: object
properties: {runtime: {type: string}}
lxc:
parent: container
domain: compute
layer: infrastructure
lifecycle: infrastructure
description: Proxmox LXC container.
attributes:
type: object
properties:
pve_id: {type: integer}
lan_ip: {type: string}
public_host: {type: string}
public_hosts: {type: array, items: {type: string}}
role: {type: string}
mesh: {type: object}
age_pubkey: {type: string}
destroyed: {type: string}
reason: {type: string}
docker-container:
parent: container
domain: compute
layer: infrastructure
lifecycle: infrastructure
description: Docker container (the OS models its own stack with these).
attributes:
type: object
properties: {image: {type: string}}
hypervisor:
parent: entity
domain: compute
layer: infrastructure
lifecycle: infrastructure
description: Hypervisor software running on a machine (PVE, KVM, OrbStack).
attributes:
type: object
properties: {type: {type: string}, version: {type: string}}
# ── Infrastructure / network ──
network:
parent: entity
abstract: true
domain: network
layer: infrastructure
description: A network things connect to.
lan:
parent: network
domain: network
layer: infrastructure
lifecycle: infrastructure
description: Local area network.
attributes: {type: object, properties: {subnet: {type: string}}}
mesh:
parent: network
domain: network
layer: infrastructure
lifecycle: infrastructure
description: Overlay mesh network (NetBird, Tailscale).
attributes:
type: object
properties:
provider: {type: string}
subnet: {type: string}
domain: {type: string}
vlan:
parent: network
domain: network
layer: infrastructure
lifecycle: infrastructure
description: Tagged VLAN.
attributes: {type: object, properties: {tag: {type: integer}}}
network-interface:
parent: entity
domain: network
layer: infrastructure
description: Optional per-interface refinement (mac, ip). The seed uses
coarse connects-via edges; interfaces can be backfilled later.
attributes: {type: object, properties: {mac: {type: string}, ip: {type: string}}}
dns-zone:
parent: entity
domain: network
layer: infrastructure
lifecycle: infrastructure
description: DNS zone (e.g. split-horizon hubris.network).
attributes: {type: object, properties: {zone: {type: string}, authority: {type: string}}}
dns-record:
parent: entity
domain: network
layer: infrastructure
description: Individual DNS record.
attributes:
type: object
properties: {name: {type: string}, record_type: {type: string}, value: {type: string}}
ingress-route:
parent: entity
domain: network
layer: infrastructure
lifecycle: infrastructure
description: Public hostname → upstream mapping (Caddy).
attributes:
type: object
properties:
pattern: {type: string}
upstream: {type: string}
forward_auth: {type: boolean}
certificate:
parent: entity
domain: network
layer: infrastructure
description: TLS certificate.
attributes: {type: object, properties: {issuer: {type: string}, expires: {type: string}}}
firewall-rule:
parent: entity
domain: network
layer: infrastructure
description: Firewall / port-forward rule.
# ── Infrastructure / storage ──
storage-pool:
parent: entity
domain: storage
layer: infrastructure
lifecycle: infrastructure
description: Storage pool (LVM, ZFS, NFS).
attributes:
type: object
properties:
type: {type: string}
capacity_gb: {type: number}
volume:
parent: entity
domain: storage
layer: infrastructure
lifecycle: infrastructure
description: Named volume / dataset within a pool. Mount details live as
attributes on `mounts` edges.
attributes: {type: object, properties: {size_gb: {type: number}, path: {type: string}}}
backup-target:
parent: entity
domain: storage
layer: infrastructure
lifecycle: infrastructure
description: Where backups land (Proton Drive, PBS).
attributes: {type: object, properties: {provider: {type: string}, encrypted: {type: boolean}}}
dataset:
parent: entity
domain: storage
layer: infrastructure
description: Logical data collection worth tracking independently of its
volume (e.g. paperless documents).
# ── Infrastructure / software ──
service:
parent: entity
domain: software
layer: infrastructure
lifecycle: infrastructure
description: A running service with consumers.
attributes:
type: object
properties:
url: {type: string}
port: {type: integer}
health: {type: string}
endpoint: {type: string}
systemd_unit: {type: string}
doc_page: {type: string}
risk_notes: {type: string}
note: {type: string}
application:
parent: entity
domain: software
layer: infrastructure
description: Deployed application/package a service runs.
attributes: {type: object, properties: {version: {type: string}}}
config-repo:
parent: entity
domain: software
layer: infrastructure
description: Git repo holding tracked configuration.
attributes:
type: object
properties: {url: {type: string}, branch: {type: string}}
deploy-pipeline:
parent: entity
domain: software
layer: infrastructure
description: Automated deploy path (webhook → script).
attributes:
type: object
properties: {trigger: {type: string}, target_path: {type: string}}
package-set:
parent: entity
domain: software
layer: infrastructure
description: Managed package baseline for a host class.
cluster:
parent: entity
domain: software
layer: infrastructure
lifecycle: infrastructure
description: Proxmox cluster.
attributes: {type: object, properties: {quorum: {type: string}}}
compose-stack:
parent: entity
domain: software
layer: infrastructure
lifecycle: infrastructure
description: Docker Compose stack (the Oikos OS itself is one).
attributes: {type: object, properties: {path: {type: string}}}
# ── Infrastructure / external ──
domain-registration:
parent: entity
domain: external
layer: infrastructure
description: Registered public domain.
attributes: {type: object, properties: {registrar: {type: string}, expires: {type: string}}}
cloud-service:
parent: entity
domain: external
layer: infrastructure
description: External SaaS/cloud dependency.
isp-link:
parent: entity
domain: external
layer: infrastructure
description: Internet uplink.
vendor-dependency:
parent: entity
domain: external
layer: infrastructure
description: Vendor the lab depends on (registrar, IONOS, Proton).
# ── Governance / identity ──
person:
parent: entity
domain: identity
layer: governance
description: Human actor (operator).
attributes:
type: object
properties: {matrix_id: {type: string}, oidc_sub: {type: string}, email: {type: string}}
agent:
parent: entity
domain: identity
layer: governance
lifecycle: infrastructure # agents are deployed/retired like infrastructure
description: Software agent actor (Hermes, the Oikos control loop).
attributes:
type: object
properties:
provider: {type: string}
model: {type: string}
gateway_port: {type: integer}
identity-provider:
parent: entity
domain: identity
layer: governance
description: OIDC / forward-auth provider (Authentik).
attributes:
type: object
properties:
issuer: {type: string}
client_id: {type: string}
auth_mode: {type: string, enum: [oidc, forward-auth, both]}
account:
parent: entity
domain: identity
layer: governance
description: An account a person/agent holds on a service.
secret:
parent: entity
domain: identity
layer: governance
description: Managed secret (Infisical path).
attributes:
type: object
properties: {path: {type: string}, rotation_days: {type: integer}}
key:
parent: entity
domain: identity
layer: governance
description: Cryptographic key (SSH, age).
access-grant:
parent: entity
domain: identity
layer: governance
description: Grant of access to a secret/scope.
attributes: {type: object, properties: {scope: {type: string}, expires: {type: string}}}
# ── Cognition ──
check:
parent: entity
domain: cognition
layer: cognition
description: Probe definition (checks-as-data, R3-7). Typed row in check_defs.
signal:
parent: entity
domain: cognition
layer: cognition
lifecycle: signal
description: Something needing attention. Typed row in signals.
classification:
parent: entity
domain: cognition
layer: cognition
description: A classifier decision with full reasoning. Typed row in classifications.
execution:
parent: entity
domain: cognition
layer: cognition
lifecycle: execution
description: An action the OS performed. Typed row in executions.
feedback:
parent: entity
domain: cognition
layer: cognition
description: What was learned from an execution. Typed row in feedback.
pattern:
parent: entity
domain: cognition
layer: cognition
lifecycle: pattern
description: Generalized rule extracted from feedback. Typed row in patterns.
skill:
parent: entity
domain: cognition
layer: cognition
lifecycle: skill
description: Codified, versioned procedure. Typed rows in skills.
approval:
parent: entity
domain: cognition
layer: cognition
lifecycle: approval
description: Operator approval request/decision. Typed row in approvals.
document:
parent: entity
domain: cognition
layer: cognition
description: Knowledge document ingested from docs/.
attributes:
type: object
properties: {title: {type: string}, source_path: {type: string}, content_hash: {type: string}}
runbook:
parent: entity
domain: cognition
layer: cognition
description: Step-by-step procedure for an entity/action.
attributes:
type: object
properties: {risk_class: {type: string}, source_path: {type: string}}
investigation:
parent: entity
domain: cognition
layer: cognition
description: Recorded investigation/postmortem.
# ─── Relationship types ────────────────────────────────────────────────
# cardinality is source→target: e.g. `hosts` one-to-many = one machine
# hosts many compute entities; each hosted entity has one hosting machine.
relationship_types:
# Infrastructure topology
hosts:
inverse: runs-on
source: machine
target: compute-entity
cardinality: one-to-many
description: Machine hosts a VM/container (hubris hosts lxc:apps).
runs-hypervisor:
inverse: hypervisor-on
source: machine
target: hypervisor
cardinality: one-to-one
description: Machine runs hypervisor software.
member-of:
inverse: has-member
source: proxmox-host
target: cluster
cardinality: many-to-one
description: PVE host belongs to a cluster.
part-of:
inverse: comprises
source: docker-container
target: compose-stack
cardinality: many-to-one
description: Docker container belongs to a compose stack.
provides:
inverse: provided-by
source: compute-entity
target: service
cardinality: one-to-many
description: Compute entity provides a service (lxc:gitea provides service:gitea).
runs:
inverse: run-by
source: service
target: application
cardinality: one-to-many
description: Service runs an application.
configured-by:
inverse: configures
source: entity
target: config-repo
cardinality: many-to-one
description: Entity's config is tracked in a repo (mutations = commit+push).
deploys-to:
inverse: deployed-by
source: deploy-pipeline
target: entity
cardinality: many-to-one
description: Pipeline deploys to a service/host.
routes-to:
inverse: routed-via
source: ingress-route
target: service
cardinality: many-to-one
description: Public hostname routes to a service.
secured-by:
inverse: secures
source: ingress-route
target: identity-provider
cardinality: many-to-one
description: Route gated by forward-auth.
uses-certificate:
inverse: certifies
source: ingress-route
target: certificate
cardinality: many-to-one
description: Route served with this certificate.
authenticates-via:
inverse: authenticates-service
source: service
target: identity-provider
cardinality: many-to-one
description: Service uses native OIDC (jellyfin authenticates-via authentik).
in-zone:
inverse: contains-record
source: dns-record
target: dns-zone
cardinality: many-to-one
description: Record belongs to a zone.
resolves-to:
inverse: resolved-from
source: dns-record
target: entity
cardinality: many-to-one
description: Record points at an ingress route or host.
depends-on:
inverse: dependency-of
source: service
target: service
cardinality: many-to-many
description: Runtime dependency (blast-radius edge).
connects-via:
inverse: connects
source: compute-entity
target: network
cardinality: many-to-many
description: Coarse network membership (host on LAN / mesh).
has-interface:
inverse: interface-of
source: compute-entity
target: network-interface
cardinality: one-to-many
description: Optional per-interface refinement.
interface-on:
inverse: has-endpoint
source: network-interface
target: network
cardinality: many-to-one
description: Interface attaches to a network.
# Storage
mounts:
inverse: mounted-by
source: compute-entity
target: volume
cardinality: many-to-many
description: Compute entity mounts a volume. Edge attributes carry
mount_point and options.
stores-on:
inverse: stores-for
source: compute-entity
target: storage-pool
cardinality: many-to-many
description: Rootfs/data lives on a pool.
contains:
inverse: contained-in
source: storage-pool
target: volume
cardinality: one-to-many
description: Pool contains a volume.
holds-dataset:
inverse: dataset-on
source: volume
target: dataset
cardinality: one-to-many
description: Volume holds a tracked dataset.
backs-up-to:
inverse: backup-of
source: entity
target: backup-target
cardinality: many-to-many
description: Entity's data is backed up to a target.
# Physical / external
powered-by:
inverse: powers
source: machine
target: ups
cardinality: many-to-one
description: Machine on UPS power.
located-at:
inverse: location-of
source: machine
target: site
cardinality: many-to-one
description: Machine's physical site.
registered-with:
inverse: registrar-of
source: domain-registration
target: vendor-dependency
cardinality: many-to-one
description: Domain registered with a registrar.
# Governance
owns:
inverse: owned-by
source: person
target: agent
cardinality: one-to-many
description: Person owns/controls an agent.
authenticates:
inverse: authenticated-by
source: identity-provider
target: person
cardinality: one-to-many
description: IdP authenticates a person.
holds-grant:
inverse: granted-to
source: agent
target: access-grant
cardinality: one-to-many
description: Agent holds an access grant.
grants:
inverse: granted-by
source: access-grant
target: secret
cardinality: many-to-one
description: Grant covers a secret.
can-decrypt:
inverse: readable-by
source: compute-entity
target: secret
cardinality: many-to-many
description: Host can decrypt a secret (legacy SOPS; Infisical grants later).
# Cognition
checks:
inverse: checked-by
source: check
target: entity
cardinality: many-to-one
description: Check probes an entity.
raises:
inverse: raised-by
source: check
target: signal
cardinality: one-to-many
description: Check raised a signal.
about:
inverse: subject-of
source: signal
target: entity
cardinality: many-to-one
description: Signal concerns an entity.
classifies:
inverse: classified-as
source: classification
target: signal
cardinality: many-to-one
description: Classification of a signal.
precedes:
inverse: follows
source: classification
target: execution
cardinality: one-to-one
description: Classification that led to an execution.
targets:
inverse: targeted-by
source: execution
target: entity
cardinality: many-to-one
description: Execution acts on an entity.
requires-approval:
inverse: approves
source: execution
target: approval
cardinality: one-to-one
description: Execution gated by an approval.
performs:
inverse: performed-by
source: agent
target: execution
cardinality: one-to-many
description: Agent performed an execution.
decides:
inverse: decided-by
source: person
target: approval
cardinality: one-to-many
description: Person decided an approval.
produces:
inverse: produced-by
source: execution
target: feedback
cardinality: one-to-one
description: Execution produced feedback.
contributes-to:
inverse: built-from
source: feedback
target: pattern
cardinality: many-to-many
description: Feedback supports a pattern.
informs:
inverse: informed-by
source: pattern
target: skill
cardinality: many-to-one
description: Pattern informs a skill.
guides:
inverse: guided-by
source: skill
target: classification
cardinality: one-to-many
description: Skill guided a classification.
documents:
inverse: documented-by
source: document
target: entity
cardinality: many-to-one
description: Document describes an entity.
procedure-for:
inverse: has-procedure
source: runbook
target: entity
cardinality: many-to-many
description: Runbook applies to an entity.

103
seeds/policy.yaml Normal file
View File

@@ -0,0 +1,103 @@
# Oikos policy seed — risk classes, approval rules, autonomy settings.
#
# Bootstraps risk_classes / approval_rules / autonomy_settings (migration
# 005). After ingest the DB is authoritative; runtime policy changes go
# through the dual-control meta-approval flow (plan S3) and are exported
# back here via `GET /api/v1/export`.
#
# Adapted from legacy oikos/policy.yaml (2026-07-07):
# - `commands:`/`mcp_tools:` maps are gone — read endpoints and MCP read
# tools are read_only by construction and never classified; mutating
# API calls classify via approval_rules below.
# - `actions:` map became approval_rules keyed on (entity_type, action);
# entity types may be abstract (rule inherits down the hierarchy,
# most-specific match wins: scope_entity > concrete type > ancestor).
# - `service_overrides:` became scope_entity rules.
# - `lifecycle_overrides:` became autonomy_settings keys read by the
# classifier.
version: 1
risk_classes:
read_only:
description: Observes state; cannot change anything.
approval_required: none
autonomy_allowed: true
reversible_low:
description: >-
Changes runtime state in a way a single follow-up command undoes
(restart, cache clear, sync pull). No config or data changes.
approval_required: none # still subject to global.auto_act + rules below
autonomy_allowed: true
config_mutation:
description: >-
Changes tracked configuration or deployed software: repo edit + push,
deploy trigger, Caddy/Gitea/app config, package upgrades. Reversible
via git, but affects other consumers.
approval_required: operator # Matrix ✅/❌ (single-use HMAC token)
autonomy_allowed: false
destructive:
description: >-
Destroys or irreversibly alters data/entities: container destroy,
disk format, DB wipe, secret rotation, client revocation.
approval_required: operator_confirmed
autonomy_allowed: false
approval_rules:
# ── Generic rules on (possibly abstract) types ──
- {entity_type: service, action: restart, risk_class: reversible_low, autonomy_level: auto}
- {entity_type: service, action: cache-clear, risk_class: reversible_low, autonomy_level: auto}
- {entity_type: service, action: sync-pull, risk_class: reversible_low, autonomy_level: auto}
- {entity_type: service, action: db-wipe, risk_class: destructive, autonomy_level: never}
- {entity_type: compose-stack, action: restart, risk_class: reversible_low, autonomy_level: auto}
- {entity_type: docker-container, action: restart, risk_class: reversible_low, autonomy_level: auto}
- {entity_type: machine, action: apt-upgrade, risk_class: config_mutation, autonomy_level: escalate}
- {entity_type: machine, action: reboot, risk_class: config_mutation, autonomy_level: escalate}
- {entity_type: machine, action: format-disk, risk_class: destructive, autonomy_level: never}
- {entity_type: config-repo, action: edit, risk_class: config_mutation, autonomy_level: escalate}
- {entity_type: deploy-pipeline, action: trigger, risk_class: config_mutation, autonomy_level: escalate}
- {entity_type: lxc, action: create, risk_class: config_mutation, autonomy_level: escalate}
- {entity_type: lxc, action: migrate, risk_class: config_mutation, autonomy_level: escalate}
- {entity_type: lxc, action: restart, risk_class: reversible_low, autonomy_level: auto}
- {entity_type: lxc, action: destroy, risk_class: destructive, autonomy_level: never}
- {entity_type: vm, action: restart, risk_class: reversible_low, autonomy_level: auto}
- {entity_type: vm, action: destroy, risk_class: destructive, autonomy_level: never}
- {entity_type: dns-record, action: change, risk_class: config_mutation, autonomy_level: escalate}
- {entity_type: ingress-route, action: change, risk_class: config_mutation, autonomy_level: escalate}
- {entity_type: storage-pool, action: change, risk_class: destructive, autonomy_level: never}
- {entity_type: secret, action: rotate, risk_class: destructive, autonomy_level: never}
- {entity_type: key, action: revoke, risk_class: destructive, autonomy_level: never}
# ── Governance objects (the OS's own levers — always operator-gated) ──
- {entity_type: pattern, action: activate, risk_class: config_mutation, autonomy_level: never}
- {entity_type: skill, action: activate, risk_class: config_mutation, autonomy_level: never}
# ── Per-entity overrides (wide blast radius) ──
- {entity_type: service, action: restart, scope_entity: "service:caddy",
risk_class: config_mutation, autonomy_level: escalate}
# everything *.hubris.network rides on caddy
- {entity_type: service, action: restart, scope_entity: "service:dns",
risk_class: config_mutation, autonomy_level: escalate}
# LAN-wide resolver
- {entity_type: service, action: restart, scope_entity: "service:authentik",
risk_class: config_mutation, autonomy_level: escalate}
# SSO provider — restart locks logins fleet-wide
autonomy_settings:
# Global kill-switch. Cold start = off: the agent escalates everything
# until patterns validate and the operator raises this (plan: trust is earned).
global.auto_act: "off" # off | reversible_low
# Per-entity hard blocks (checked even when global.auto_act is on)
never_auto_act.service:caddy: "true"
never_auto_act.service:dns: "true"
never_auto_act.service:authentik: "true"
never_auto_act.host:hubris: "true"
never_auto_act.host:strong: "true"
# Lifecycle-state classifier overrides (from legacy lifecycle_overrides)
lifecycle_override.provisioning.config_mutation: reversible_low
# no dependents yet — config changes are cheap
lifecycle_override.deprecated.refuse: new-inbound-edges
lifecycle_override.destroyed.refuse: all
# any action targeting a destroyed entity raises a drift signal instead

39
sqlc.yaml Normal file
View File

@@ -0,0 +1,39 @@
# sqlc — type-safe Go from SQL (plan SG17). `make generate` regenerates.
#
# Scope: API read/mutation paths use sqlc-generated queries
# (internal/db/sqlcgen). The seed ingest and YAML export intentionally stay
# hand-written pgx: they are generic bulk upserts driven by parsed YAML
# shapes, where sqlc's static typing adds nothing.
version: "2"
sql:
- engine: "postgresql"
schema: "migrations"
queries: "internal/db/queries"
strict_order_by: false
gen:
go:
package: "sqlcgen"
out: "internal/db/sqlcgen"
sql_package: "pgx/v5"
emit_pointers_for_null_types: true
overrides:
- db_type: "uuid"
go_type:
import: "github.com/google/uuid"
type: "UUID"
- db_type: "uuid"
nullable: true
go_type:
import: "github.com/google/uuid"
type: "UUID"
pointer: true
- db_type: "timestamptz"
go_type:
import: "time"
type: "Time"
- db_type: "timestamptz"
nullable: true
go_type:
import: "time"
type: "Time"
pointer: true