Commit Graph

5 Commits

Author SHA1 Message Date
095a3967c4 phase 3: control loop — scheduler, actuator, learning, notifier, policy, API endpoints
Implemented the full OODA control loop:

Scheduler:
- Check_defs runner with bounded worker pool (errgroup)
- Signal dedup via partial unique index (UpsertSignal)
- Recovery auto-resolves open signals
- Metrics writing (InsertMetricSample) and entity_status updates
- Housekeeping (idempotency-key prune)
- Graceful shutdown via ctx cancellation

Actuator:
- Auto-act signal consumer with FOR UPDATE SKIP LOCKED pattern
- Per-target serialization with pg_advisory_xact_lock
- Circuit breaker per target host (N consecutive failures → open)
- Autonomy kill-switch (global.auto_act, never_auto_act.<slug>)
- Execution record lifecycle (proposed → running → completed)

Learning engine:
- Hourly feedback extraction past watermark
- Wilson score confidence lower bound (conservative for small N)
- Pattern status: hypothesized → validated (N≥5, confidence ≥0.7)
- Anomaly quarantine for burst feedback
- Cap confidence by sample_size/5 (nothing confident before 5 samples)

Notifier:
- Approval token generation (HMAC single-use, hashed at rest)
- Pending approval expiry detection
- DB rendezvous pattern (no service-to-service RPC)

Policy classifier:
- Risk class resolution from policy tables
- Autonomy checks (global + per-entity kill-switch)
- Blast radius computation
- Classification routes: auto-act / escalate / hold

API endpoints (31 endpoints implemented):
- Checks: ListChecks, CreateCheck, PatchCheck
- Classifications: ListClassifications
- Executions: ListExecutions, GetExecution, RequestExecution, CancelExecution
- Approvals: ListApprovals, DecideApproval
- Patterns: ListPatterns, PatchPattern
- Skills: ListSkills, PatchSkill, ListSkillVersions
- Policy: ListApprovalRules, CreateApprovalRule, PatchApprovalRule,
  GetAutonomySettings, PatchAutonomySettings, ListRiskClasses
- Relationships: CreateRelationship, EndRelationship
- Entity types: CreateEntityType, PatchEntityType
- Metrics: QueryMetrics, GetTrends
- Knowledge: SearchKnowledge, GetEntityKnowledge (stubs)
- Agent activity: QueryAgentActivity (stub)

Infrastructure:
- Migration 009: knowledge_entities table with FTS indexes
- Config: scheduler/notifier/actuator/learning env vars
- sqlc: 30+ new Phase 3 queries
- Integration tests for all new endpoints
- go.sum updated with golang.org/x/sync
2026-07-07 15:19:25 +02:00
1bfc18ea3a phase 2 (part 4): SSE stream via io.Pipe, OIDC JWT auth middleware
- SSE stream: GET /events/stream using io.Pipe to bridge the SSE
  goroutine to the response body. Replay from Last-Event-ID via
  in-memory broker with DB fallback. LISTEN/NOTIFY fan-out to all
  subscribers. Heartbeat every 15s. Bounded channels.
- OIDC JWT auth: validates Bearer tokens against Authentik/OIDC
  issuer via JWKS discovery + key caching. Extracts sub/email into
  context actor. Falls back to static bearer tokens. Dev mode (no
  OIDC + no tokens) = open.
- Config: OIDCIssuer, OIDCClientID env vars
- SSE + OIDC infrastructure complete, build passes, all tests pass

Remaining: MCP server, conformance tests, wire audit middleware
2026-07-07 09:40:11 +02:00
f2fe812cda phase 2 (part 1): OpenAPI-generated API server, first 9 endpoints
- api/openapi.yaml converted 3.1 → 3.0.3 (oapi-codegen/kin-openapi
  supports 3.0; nullable syntax + example keywords), still redocly-clean
- oapi-codegen (v2.4.1, strict server + chi) generates
  internal/httpapi/gen from the spec; `make generate` wired
- internal/httpapi: chi router, /healthz (unauthenticated, SG18),
  RFC 9457 problem+json mapping from domain sentinels (SG11), 5xx detail
  logged server-side only, request logging with request IDs, graceful
  shutdown (SG4), interim static bearer auth (constant-time; dev-open
  when no token; OIDC JWT still to come in Phase 2)
- Implemented: listEntities (type filter walks the hierarchy, keyset
  pagination), getEntity (UUID or slug, ETag), getEntityRelations,
  getBlastRadius, getGraph (nodes+edges for UIs), getOntology,
  listSignals, getFleetHealth, exportSeeds. Remaining 38 ops return 501
  problem+json stubs (compiler-enforced interface completeness)
- `oikos api` role live: migrate-on-start, serves :8090
- 15 API integration tests (auth, pagination, hierarchy filter, ETag,
  404/501 problem shapes, graph, export)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:25:19 +02:00
1b04683639 phase 1 review fixes: dedup edges, real export, validation, tests
Review of aa2ca0a found and fixed:
- re-ingest duplicated ALL edges (upsert conflicted on valid_from=now(),
  never fired) — migration 007 dedupes + partial unique index on current
  edges; upsert now targets it. Regression-tested.
- export was a stub that overwrote seeds/*.yaml with 11-byte "version: 1"
  files — implemented real deterministic export (ontology/inventory/policy,
  cognition-layer excluded); round-trip is byte-stable (tested)
- DB password leaked in startup logs (slog JSON bypasses String()) —
  Config now implements slog.LogValuer; regression-tested
- docker-compose had literal '***' as DB password — env-interpolated
- uuid.New() (v4) → uuid.NewV7() per ADR-0005
- no ontology validation on ingest — internal/ontology TypeTree: abstract
  instantiation rejected, relationship endpoints hierarchy-validated,
  cardinality enforced in-transaction, lifecycle states checked, default
  state applied (Phase 1 gate items, R3-1)
- getOrCreateEntityID swallowed non-ErrNoRows errors
- migration runner now holds a session advisory lock on one connection
- Makefile: hardcoded /opt/homebrew/bin/go → go; test-db target

Tests: 4 unit suites + 7 integration tests (env-guarded, throwaway DB per
run): migrate idempotent, seed idempotent + no dup edges, abstract/edge/
cardinality rejection, blast_radius cycle termination, export round-trip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:11:01 +02:00
aa2ca0ae6f phase 1: Go foundation — module, migrations, domain, seed ingest
Core deliverables:
- Go module github.com/dtoro/oikos (Go 1.26.3)
- cmd/oikos: single binary with role subcommands (migrate, seed, export)
- 6 SQL migrations: ontology meta-schema, entity instances (UUID+slug,
  blast_radius recursive function), operations (signals/checks/approvals),
  cognition (classifications/executions/feedback/patterns/skills), policy,
  observability (TimescaleDB hypertables + CAGGs + retention)
- Domain layer: entity, signal, execution, classification, pattern, skill,
  approval, check types + 11 sentinel errors + lifecycle state machines
- DB layer: pgx pool, SQL splitter (handles 94436 and -- comments), migration
  runner, seed ingest (ontology+inventory+policy) with content-hash dedup
- Config: env-based with defaults, secrets redaction
- Observability: slog JSON logger with debug mode
- Infrastructure: Makefile, docker-compose.yml, multi-stage Dockerfile
  (distroless, CGO_ENABLED=0)

Verified end-to-end against timescale/timescaledb:2.17.2-pg16:
- 6 migrations applied (65 SQL statements)
- Seeds ingested: 6 lifecycles, 59 entity types, 46 relationship types,
  111 entities, 144 relationships, 4 risk classes, 27 approval rules,
  9 autonomy settings
- Idempotent: second seed run is a no-op (content hash matches)

Bugs fixed during implementation:
- TimescaleDB CAGGs can't run in a transaction -> splitSQL() executes
  statements individually
- Semicolons in -- comments treated as separators -> comment handling
- YAML keys source/target didn't match code's source_type/target_type
- yaml.Marshal produced YAML for JSONB columns -> json.Marshal
2026-07-07 01:07:26 +02:00