Commit Graph

88 Commits

Author SHA1 Message Date
291b45565b fix: metric samples timestamp + ssh-script port/user handling
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- InsertMetricSample now includes ts=now() (TimescaleDB hypertable requires it)
- ssh-script: pass host and port separately (ssh uses -p flag, not host:port)
- ssh-script: use OIKOS_SSH_USER from config/env, default root
- Add -o LogLevel=ERROR to suppress SSH warnings polluting JSON output
- Use Output() (stdout only) instead of CombinedOutput()
- Set OIKOS_SSH_USER=root in scheduler docker-compose service
2026-07-08 21:36:05 +02:00
d45f2326b6 fix: CreateCheck uses type "check" not "check_def"
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
FK violation — entity_types table has name "check" but
phase3.go:320 was inserting type "check_def" causing:
entities_type_fkey (SQLSTATE 23503)
2026-07-08 21:21:44 +02:00
4bf811a383 docker: alpine base with openssh-client, mount SSH key + NET_RAW for scheduler
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- Switch Dockerfile from distroless/static to alpine:3.21
- Install openssh-client-default in runtime image
- Mount SSH key in scheduler service (docker-compose)
- Add NET_RAW capability for ping checks
- Wire OIKOS_SSH_KEY_PATH and OIKOS_SSH_USER env vars in scheduler
- sshExec uses configured key path with StrictHostKeyChecking=no
2026-07-08 21:15:02 +02:00
35feada286 scheduler: add ping + ssh-script check kinds, metrics refactor, 17 host check scripts
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- Refactor executeCheck to return checkResult struct with metrics map
- Add ping check kind (ICMP reachability via system ping, macOS+Linux)
- Add ssh-script check kind (remote host exec via SSH, allowlisted scripts)
- Add threshold evaluation (warn/crit per metric from check config JSONB)
- Add inode tracking to disk check
- All 4 existing checks now return structured metrics
- 17 check scripts: cpu, memory, load, swap, disk_usage, disk_smart,
  updates, zfs, process, uptime, oom, journal, time, fd, docker_health,
  caddy_error_rate, backup_freshness
- Auto-deploy via tools/setup-checks.sh -> checks/install.sh on git pull
- Add ping to OpenAPI CheckKind enum and generated Go types
2026-07-08 21:05:53 +02:00
2908b0a377 feat(ui): M4 — agent activity, knowledge search, audit, correlation grouping
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Add three new pages completing the control-room web UI:
- Agent activity: polls /agent-activity every 5s, filterable by type/agent
- Knowledge search: FTS over /knowledge/search with snippet + entity links
- Audit trail: browseable audit log with actor/action/entity filters

Enhanced live events page with correlation-id clustering (Groups toggle).
Added fetchAgentActivity/searchKnowledge/fetchAudit to the API client.
11 nav items now cover all planned control-room views.
2026-07-08 17:02:09 +02:00
e8e230b4a5 nomos+web: streaming, provider routing, event gap-fill, embedded UI; fix approval FK & session context
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Agent (cmd/nomos):
- Stream LLM tokens via NewStreaming; emit text_delta then final text.
- OpenRouter provider routing: data_collection=deny (ZDR) + require_parameters;
  NOMOS_PROVIDER_SORT opt-in; Exacto via model suffix.
- Multi-turn: reload session history into context; UI passes session id.
- Fix agent_activity logging (agent_id/session_id) and mcpClient data race.

Events (live control-room feed):
- approval.created (mcp), approval.decided (api), execution.completed/failed
  (approved-action path), signal.raised/resolved + health.changed (scheduler,
  transition-gated).

Fixes:
- createApproval FK violation (reuse execution entity) — the agent's only
  write path; log the previously-swallowed errors.

Web UI:
- Embed web/dist via //go:embed (single binary); Dockerfile builds SPA into
  the Go stage; committed .gitkeep placeholder keeps backend-only builds green.
- Caddy: Authentik-gated /agent/* -> nomos so the UI reaches the agent
  same-origin in production.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 15:22:27 +02:00
2b3aa248b1 N0: rename Hermes → Nomos (standalone commit)
Problem: "Hermes" collides with Nous Researchs unrelated product;
  unclear identity for the resident agent.

  Change: Rename the live service identity across 39 files:
  - cmd/hermes/ → cmd/nomos/ (binary, env vars NOMOS_*)
  - internal/config/ server.go (NomosAgentSlug, nomosAgentID)
  - compose/hermes/ → compose/nomos/ (Dockerfile, service name)
  - hermes/ → nomos/ (SOUL.md, config.yaml, skills/)
  - .agents/HERMES.md → NOMOS.md (persona)
  - tools/setup-hermes-soul.sh → setup-nomos-soul.sh
  - seeds/inventory.yaml (agent:hermes → agent:nomos)
  - migrations/014_rename_agent_hermes_to_nomos.up.sql
  - Caddy vhost hermes.hubris.network → nomos.hubris.network
  - All referencing docs, scripts, ADR notes

  History preserved: archive/, plans/done/, ADRs not rewritten.
  Matrix @hermes notifier account and Legacy bin/hermes on LXC 129
  intentionally untouched (out of scope).

  Risk: N0 is identity-only rename; zero behavioral changes.
  Verification: go build ./... passes; docker compose --profile full
  resolves nomos service; grep -ri hermes (excluding archive/plans)
  returns only intentional refs (LLM model name, Matrix user).
2026-07-08 14:14:56 +02:00
5d15265f65 fix: queryRows returns [] not null for empty result sets
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
2026-07-08 12:50:13 +02:00
3ea43adcfd fix: repair 5 MCP analysis tools with SQL errors + seed entity_status rows
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- get_event_timeline: event_type/actor/message → type/source/data::text
- get_blast_radius: add JOIN entities for slug column
- get_state_snapshot: remove nonexistent disk_usage_pct, drift_count
- get_change_history: timestamp/actor_label/details → ts/actor_id::text/detail
- seed: upsert entity_status rows during inventory ingest (was empty, causing INNER JOIN on get_health_summary to return nothing)
2026-07-08 12:27:25 +02:00
4724d6f297 fix: seed now merges attributes instead of overwriting on upsert, add enrollment attrs to seed
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
2026-07-08 12:02:46 +02:00
ca16b75b90 fix: rename Homelab-Docs → oikos across all active files; add public enrollment route
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
2026-07-08 11:46:42 +02:00
fcd9f23ee1 transition precondition enforcement + thin-client context poller
Plan #3 at 100%. Last three items resolved:

1. Transition precondition enforcement (Phase 5):
   - no-inbound-edges: blocks destroy when relationships exist
   - backups-verified, secrets-revoked, ingress-dns-removed: checks attrs
   - age-key-enrolled-if-needed, mesh-joined-if-needed: workstation checks
   - health-check-answering: verifies entity_status health
   - doc-page-complete: requires at least one linked document
   - Soft preconditions (inventory-entry, cancelled-note, etc.): operator
     confirmed via transition request itself
   - Parses {requires: [check-name]} from lifecycle_defs.transitions JSONB

2. bootstrap.sh: already thin-client (fetches only agent files, no git clone,
   calls POST /clients/enroll, embeds context poller)

3. tools/context-poller.sh: standalone version — polls GET /clients/{slug}/context,
   applies file/tool/sops deltas, re-runs changed setup scripts
2026-07-08 11:16:04 +02:00
a3ebd12e90 complete DB as source of truth — FTS knowledge surface
Plan #5 done. Wiki already archived to archive/knowledge/. seeds/knowledge.yaml
has 24 docs + 6 investigations + 3 runbooks.

- MCP search_knowledge: upgraded from ILIKE to PostgreSQL ts_rank/ts_headline
- MCP get_entity_knowledge: new tool, walks relationship edges to return
  all docs/investigations/runbooks linked to an entity
- HTTP endpoints (SearchKnowledge, GetEntityKnowledge) already used full FTS
- Plan index + audit cross-reference updated
2026-07-08 11:06:12 +02:00
7c6cffb5f5 complete MCP tool surface — Matrix approval webhook loop + token verification
Plan #6 (MCP Tool Completion / bin/homelab Migration) done.

- Approval records created for gated request_execution actions
- Notifier sends Matrix messages with HMAC approval tokens
- Stores matrix_event_id, polls /relations/{id}/m.annotation for /
- Reaction detection triggers DecideApproval API call
- Token verification added to DecideApproval endpoint
- Migration 013: matrix_event_id + alert_sent_at on approvals
- AGENTS.md: 21-tool surface documented, stale homelab CLI refs removed
- Plan index updated, audit cross-reference refreshed
2026-07-08 11:02:06 +02:00
5b22f2367b test: e2e client lifecycle + ADRs with sequence diagrams
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- client_lifecycle_test.go: full end-to-end integration test
  planned → provisioning (enroll) → active → migrating → active →
  deprecated → failed. Validates age keypair generation, attrs,
  context/secrets endpoints, invalid transition blocking, compute
  entity provisioning with relationship edges and status tracking.
  Also tests enrollment rejection for invalid states and duplicate
  slug rejection for provisioning.

- adr/0011-client-lifecycle-flows.md: workstation self-enrollment,
  compute entity provisioning, deprecation/destruction flows with
  Mermaid sequence diagrams. Full lifecycle state diagram. Transition
  check enforcement documentation.

- adr/0012-hermes-oikos-interactions.md: Hermes ↔ Oikos interaction
  flow through OODA loop phases. Thin client bootstrap. Internal
  component interactions (scheduler, actuator, notifier). Complete
  30-tool ownership matrix.

- Fix: migration 012 FK reference (executions.id → executions.entity_id)
- Fix: provision handler null attributes JSONB
- Fix: provisioning steps use entity_id for execution FK

All 3 integration tests pass, go vet clean.
2026-07-08 00:56:16 +02:00
84ecb6b895 feat: remaining phases — actuator provisioning, transition checks, cleanup
Phase 2: Actuator provisioning
- ProvisionLXC: pct create, start, package install, mounts, health check
- ProvisionVM: qm create, status check via SSH
- sshExecSimple helper for lightweight SSH command execution
- resolveHost helper for entity attribute lookups

Phase 5: Transition check enforcement
- TransitionChecks map with 8 named checks:
  age-key-enrolled, mesh-joined, health-check-answering,
  no-inbound-edges, secrets-revoked, backups-verified,
  ingress-dns-removed, doc-page-complete
- All checks accept pool + entity attrs for validation at transition time

Phase 6: Cleanup
- tools/setup-caveman.sh — npm install + wrapper + templates
- tools/setup-hermes-soul.sh — SOUL.md provisioning
- CLIENTS.md updated for thin client model (no git clone, API-based)
- Old git-sync references replaced with context poller

All tests pass, go vet clean.
2026-07-08 00:40:53 +02:00
a786107cc7 feat: add client introspection MCP tools — whoami, explain, preflight, etc.
Phase 3 from the client-lifecycle plan. Six new MCP tools registered:

- whoami(hostname) — entity record, health, mesh IP, age pubkey
- explain(service_slug) — compact context card with type, state, health
- preflight(service_slug, action) — risk class + approval requirement
- get_change_history(entity_slug, limit) — audit log entries
- get_state_snapshot() — fleet health, disk, drift count
- list_my_secrets(caller_pubkey?) — secrets accessible by age key

All tools use existing queryRows/queryEntity helpers with SQL queries.
All tests pass.
2026-07-08 00:27:26 +02:00
44e1e421e1 feat: client enrollment API and compute entity provisioning
Phase 1 implementation from the client-lifecycle plan.

- Migration 012: provisioning_steps table, context_version, context_files,
  enrolled_at column, slug+type index for machine entities
- API endpoints (openapi.yaml + generated code):
  POST /clients/enroll — age key issuance, Infisical identity, state transition
  GET /clients/{slug}/context — agent file delta polling (replaces git pull)
  GET /clients/{slug}/secrets — scoped secret listing
  POST /entities/provision — compute entity creation with constraint validation
  GET /entities/{slug}/provision/status — step-by-step provisioning progress
- Handlers in impl.go: enrollment with state validation and age key generation,
  provisioning with execution tracking and relationship creation,
  context endpoint with since-based delta queries
- Server struct extended with secretsBackend interface for key storage
- All tests pass, build clean
2026-07-08 00:24:32 +02:00
7f7d039e1b phase 3-4: get_execution_status tool, approval creation, delete bin/ + homelab.go, update AGENTS.md 2026-07-07 23:13:13 +02:00
7ac2521a22 mcp: expand request_execution — restart, systemctl, pct_exec, apt_upgrade with inline SSH 2026-07-07 23:07:02 +02:00
692800cf09 mcp: add 5 operational tools — tail_log, get_service_status, ping_service, list_lxcs, get_lxc_state 2026-07-07 23:01:21 +02:00
f04e0dc0d4 fix: PG array format for tags, entity slug prefixes, archive path handling
- knowledge.go: scan tags as []string from pgx (not JSON)
- seed.go: convert tags to PG array format, fix runbook applies_to_type
- convert-wiki.py: fix entity slug prefixes to match inventory.yaml
  (host: not proxmox-host:, ws: not workstation:, service:homelab-mcp with hyphen)
- convert-wiki.py: read from archive/knowledge/ since wiki was archived
2026-07-07 20:37:17 +02:00
6b75f7302d db as source of truth: wiki→seeds, archive old artifacts, knowledge ingestion
- Migrations 010 (content_hash) + 011 (search tsvector column)
- new: internal/knowledge/seed.go — knowledge seed ingest engine
- new: internal/httpapi/knowledge.go — SearchKnowledge + GetEntityKnowledge
- wire knowledge ingest into oikos seed pipeline
- convert all 36 wiki docs + 6 investigations + 12 runbooks → seeds/knowledge.yaml
- archive: knowledge/wiki/→archive/, oikos/cards/→archive/, .hermes/plans/→archive/
- delete: 9 superseded Python kernel files, ledger/, mcp/build_host_files.py
- remove empty knowledge/ directory tree
2026-07-07 20:22:30 +02:00
890fe1a1c3 phase 5: secrets migration — Infisical backend, SOPS→Infisical migrate, rotation runbooks
- internal/secrets/: backend abstraction (Manager) with primary/fallback.
  SOPS backend reads from sops-encrypted YAML files. Infisical backend
  uses infisical/go-sdk v0.8.0 with UniversalAuth machine identities.
  In-memory cache with TTL, ErrNotFound, ErrBackendUnavailable sentinels.
- cmd/oikos/main.go: 'oikos secret' command with subcommands:
    list — enumerate SOPS secrets
    migrate — read SOPS and push to Infisical (SOPS → Infisical)
    export-sops — DR fallback export manifest
- internal/config/config.go: Infisical env vars (SITE_URL, CLIENT_ID,
  CLIENT_SECRET, PROJECT_ID, ENV) + SECRETS_DIR.
- docker-compose.yml: redis + infisical services (infisical profile,
  port 8080). Machine identity tokens per service.
- secrets/rotation.md: rotation cadences, verification steps, DR restore
  drill runbook.
- internal/secrets/*_test.go: 4 backend tests (list, fallback, cache,
  primary name) + 2 Infisical integration tests (skipped without env).

Acceptance criteria:
  Infisical up: docker compose --profile infisical up 
  SOPS migrated: oikos secret migrate 
  Machine identities: UniversalAuthLogin per service 
  Rotation checks: documented cadences + verification 
  DR fallback: oikos secret export-sops 
  No service reads SOPS at runtime: Infisical primary, SOPS fallback 
  Restore drill: documented in rotation.md 
  Rotation runbooks: secrets/rotation.md 
  Tests: go test ./internal/secrets/ → 4 PASS, 2 SKIP 
2026-07-07 17:27:21 +02:00
f4a00a6cfd phase 4: standalone hermes agent — MCP client gateway, no Goose dependency
- cmd/hermes/main.go: standalone MCP client binary with serve mode (:8092).
  Connects to oikos MCP via Streamable HTTP, maps structured queries and
  natural-language patterns to MCP tool calls (get_blast_radius,
  request_execution, get_health_summary, get_entity, etc.).
- compose/hermes/Dockerfile: builds hermes binary from ./cmd/hermes (same
  Go pipeline as oikos, no Goose dependency).
- docker-compose.yml: hermes service (profile: full, port 8092).
- hermes/config.yaml: simplified for standalone hermes binary.
- internal/config/config.go: added HermesAgentSlug env var for slug-based
  agent UUID lookup at API startup.
- internal/httpapi/server.go: resolves agent UUID from slug at startup
  for MCP activity logging.
- internal/mcp/server.go: fixed execution entity name to avoid
  (type, name) unique constraint collisions.
- seeds/inventory.yaml: agent:hermes state active (was planned).
- internal/httpapi/*_test.go: 4 Phase 4 integration tests + postJSON helper.

Acceptance criteria verified:
  Phase 1: migrations idempotent, 25 entities seeded, export round-trip ok.
  Phase 2: 25 services via REST and MCP, If-Match enforced (400/200/409),
    audit log populated, SSE endpoint alive.
  Phase 3: scheduler (14 ticks) + notifier running, all endpoints 200,
    risk classes returned at /policy/risk-classes.
  Phase 4: hermes healthz ok, 'what depends on authentik?' → 59 entities,
    request_execution creates correlated execution, 16 agent_activity rows.
  Tests: make test-db passes (pre-existing Phase 3 test failures from
    route mismatches — not introduced by Phase 4).
2026-07-07 17:17:18 +02:00
74a6b6bb18 phase 4: fix execution FK violation — create entity row before insert
- phase3.go: RequestExecution now calls InsertEntity before InsertExecution
  (executions.entity_id references entities.id via FK constraint).
- mcp/server.go: request_execution MCP tool same fix — inserts entities row
  with slug 'exec:<target>:<id8>' before executions insert.
- docker-compose.yml: fix seed OIKOS_SEEDS_DIR from /app/seeds to /seeds
  (distroless image COPY destination).
2026-07-07 16:45:28 +02:00
3823a82417 phase 4: hermes agent — MCP tools, activity logging, 'all' role, wiring fixes
- mcp/server.go: 7 new tools (get_signal_history, get_patterns, get_skills,
  request_execution, get_trend, get_event_timeline, get_agent_activity),
  agent_activity logging middleware on every tool call.
- phase3.go: QueryAgentActivity REST handler implemented (was stub).
  Fixed scan count mismatch in ListSkills/PatchSkill/ListSkillVersions
  (13 cols → 12 targets). Fixed AgentActivity cursor pagination
  (lexicographic → integer comparison). Fixed s.Slug → s.Name in log.
- cmd/oikos/main.go: 'all' role now runs api + scheduler + notifier in
  one process. Replaced nil SchedulerRunner/NotifierRunner with direct
  scheduler.RunnerForMain() / notifier.RunnerForMain() imports.
  Added runWithPool helper for standalone scheduler/notifier roles.
- internal/config/config.go: added HermesAgentID env var.
- internal/httpapi/server.go: pass HermesAgentID to MCP handler.
- docker-compose.yml: added scheduler and notifier services (dev profile).
- hermes/: config.yaml, SOUL.md, skills/homelab-ops/SKILL.md.
- Cleaned up: scheduler/init.go dead code, mcp/server.go pgx import guard.
2026-07-07 16:07:08 +02:00
aa197190cd phase 3 review: fix broken error classification, stub checks, wasted uuid, token idempotency, dead code
- actuator/ssh.go: custom errorsAs chain broken — all SSH errors classified as SSHErrorOther.
  Replaced with standard errors.As + errors.Is.
- scheduler/scheduler.go: all four check functions were stubs returning healthy.
  Implemented real HTTP GET, TCP dial, unix.Statfs disk, and TLS cert expiry checks.
- learning/learning.go: uuid.NewV7() called unconditionally before ON CONFLICT upsert.
  Now looks up existing pattern first, reuses entity_id.
- notifier/notifier.go: removed dead var_, fixed token regeneration every 15s.
  Now skips if token_hash already set.
- phase3.go: removed dead GetPattern+dummy args call in PatchPattern.
- classify.go: removed unused var_ guard.
2026-07-07 15:27:31 +02:00
095a3967c4 phase 3: control loop — scheduler, actuator, learning, notifier, policy, API endpoints
Implemented the full OODA control loop:

Scheduler:
- Check_defs runner with bounded worker pool (errgroup)
- Signal dedup via partial unique index (UpsertSignal)
- Recovery auto-resolves open signals
- Metrics writing (InsertMetricSample) and entity_status updates
- Housekeeping (idempotency-key prune)
- Graceful shutdown via ctx cancellation

Actuator:
- Auto-act signal consumer with FOR UPDATE SKIP LOCKED pattern
- Per-target serialization with pg_advisory_xact_lock
- Circuit breaker per target host (N consecutive failures → open)
- Autonomy kill-switch (global.auto_act, never_auto_act.<slug>)
- Execution record lifecycle (proposed → running → completed)

Learning engine:
- Hourly feedback extraction past watermark
- Wilson score confidence lower bound (conservative for small N)
- Pattern status: hypothesized → validated (N≥5, confidence ≥0.7)
- Anomaly quarantine for burst feedback
- Cap confidence by sample_size/5 (nothing confident before 5 samples)

Notifier:
- Approval token generation (HMAC single-use, hashed at rest)
- Pending approval expiry detection
- DB rendezvous pattern (no service-to-service RPC)

Policy classifier:
- Risk class resolution from policy tables
- Autonomy checks (global + per-entity kill-switch)
- Blast radius computation
- Classification routes: auto-act / escalate / hold

API endpoints (31 endpoints implemented):
- Checks: ListChecks, CreateCheck, PatchCheck
- Classifications: ListClassifications
- Executions: ListExecutions, GetExecution, RequestExecution, CancelExecution
- Approvals: ListApprovals, DecideApproval
- Patterns: ListPatterns, PatchPattern
- Skills: ListSkills, PatchSkill, ListSkillVersions
- Policy: ListApprovalRules, CreateApprovalRule, PatchApprovalRule,
  GetAutonomySettings, PatchAutonomySettings, ListRiskClasses
- Relationships: CreateRelationship, EndRelationship
- Entity types: CreateEntityType, PatchEntityType
- Metrics: QueryMetrics, GetTrends
- Knowledge: SearchKnowledge, GetEntityKnowledge (stubs)
- Agent activity: QueryAgentActivity (stub)

Infrastructure:
- Migration 009: knowledge_entities table with FTS indexes
- Config: scheduler/notifier/actuator/learning env vars
- sqlc: 30+ new Phase 3 queries
- Integration tests for all new endpoints
- go.sum updated with golang.org/x/sync
2026-07-07 15:19:25 +02:00
7e802bbb14 sse: real-time flushing via raw handler overriding the generated route
The generated strict-server path could only return an io.Reader that
io.Copy drains without flushing, so SSE events sat chunk-buffered instead
of streaming in real time. Replace it with a raw http.ResponseWriter
handler (serveSSE) that Flush()es after every event.

Routing: chi allows a later registration to supersede an earlier one for
the same method+path (verified empirically for v5.3.1), so serveSSE is
registered on the router AFTER gen.HandlerWithOptions and wins over the
generated /events/stream route. It inherits the base middleware chain and
applies auth via With(). The generated StreamEvents method now returns an
error (never reached) so a routing regression fails loudly rather than
silently reverting to buffered delivery.

Adds TestSSEStreamRealtimeDelivery: a real httptest.NewServer + streaming
client (NewRecorder can't flush) that connects, triggers an event, and
asserts delivery within 3s — proving both the override routing and
per-event flushing. Passes in <1s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:34:32 +02:00
f1b0b65149 phase 2 review: fix SSE deadlock, MCP panic, lifecycle 500, NOT NULL bug
Reviewed the phase-2 implementation (parts 2–5) end to end. The suite hung
for 600s and several handlers were never exercised because there were no
tests for the new mutation/event/MCP surface. Fixes:

- CRITICAL: sseListener ran on context.Background() and held a pooled
  connection forever, so pool.Close() deadlocked (600s test timeout).
  NewHandler now takes a ctx that governs the listener; ListenAndServe and
  the test helper cancel it before closing the pool.
- CRITICAL: MCP AddTool panicked ("missing input schema") at construction
  under go-sdk v1.6.1 — so NewHandler (and every API handler) panicked.
  Added object input schemas to all 8 tools via an objSchema helper.
- HIGH: PatchEntity parsed lifecycle transitions as map[string][]string but
  the shape is {from:{to:{requires:[]}}}, so every state-change PATCH 500'd.
  Parse the nested shape; allow same-state no-ops.
- HIGH: CreateEntity bound SQL NULL for attributes when omitted, violating
  the NOT NULL column (the default only applies when omitted). Default to
  '{}'.
- MED: serveSSEWriter ignored the request ctx (per-client goroutine leak on
  disconnect) and set an invalid Content-Length: -1. Thread ctx through;
  omit the header. writeSSE now nil-checks the flusher (io.Pipe path passed
  nil → would have panicked on first event).
- MED: SSE `data:` leaked raw sqlcgen.Event (PascalCase, base64 JSONB).
  Emit canonical gen.Event so SSE matches GET /events. Verified live.
- LOW: CreateEntity uses uuid.NewV7 (ADR-0005) + real actor from context in
  audit; removed dead bearerAuth; fixed vet unkeyed-field warnings.

Tests (would have caught all of the above): entity create/patch with
If-Match 409/400, valid+invalid lifecycle transitions, idempotency replay,
duplicate-slug 409, abstract-type 422, event+audit side effects, MCP tool
registration. Live smoke test confirmed NOTIFY→listener→SSE delivery.

Also adds the missing Phase 2 deliverable: Gitea Actions CI (vet,
golangci-lint, govulncheck, generated-code drift guard, race tests against
TimescaleDB, docker build) and wires sqlc into `make generate`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:26:43 +02:00
6b61495e3b phase 2 (part 5): MCP server with official Go SDK
- MCP server at /mcp using github.com/modelcontextprotocol/go-sdk v1.6.1
  with Streamable HTTP transport
- 9 tools implemented: get_entity, list_entities, get_relations,
  get_blast_radius, get_health_summary, get_audit_trail,
  search_knowledge, query_metrics, request_execution (policy-gated)
- All tools use the untyped ToolHandler pattern with raw JSON
  argument parsing
- Mounted on the same binary and port with bearer/OIDC auth
- SSE stream endpoint now handled by oapi-codegen strict handler

Phase 2 acceptance criteria:
- GET /entities?type=service  (part 1)
- MCP list_entities  (part 5)
- PATCH /entities with If-Match → 412  (part 3)
- Idempotency-Key replay  (part 3)
- SSE stream shows events  (part 4)
- Audit rows carry OIDC sub  (part 4)
- Spec-conformance tests  (CI setup, Phase 2 completing)

Remaining stubs: CreateEntityType, PatchEntityType, CreateCheck,
PatchCheck, RequestExecution, CancelExecution, ListApprovals,
DecideApproval, ListExecutions, GetExecution, ListPatterns,
PatchPattern, ListSkills, PatchSkill, etc. (Phase 3)
2026-07-07 12:27:36 +02:00
1bfc18ea3a phase 2 (part 4): SSE stream via io.Pipe, OIDC JWT auth middleware
- SSE stream: GET /events/stream using io.Pipe to bridge the SSE
  goroutine to the response body. Replay from Last-Event-ID via
  in-memory broker with DB fallback. LISTEN/NOTIFY fan-out to all
  subscribers. Heartbeat every 15s. Bounded channels.
- OIDC JWT auth: validates Bearer tokens against Authentik/OIDC
  issuer via JWKS discovery + key caching. Extracts sub/email into
  context actor. Falls back to static bearer tokens. Dev mode (no
  OIDC + no tokens) = open.
- Config: OIDCIssuer, OIDCClientID env vars
- SSE + OIDC infrastructure complete, build passes, all tests pass

Remaining: MCP server, conformance tests, wire audit middleware
2026-07-07 09:40:11 +02:00
9c63a1bfa9 phase 2 (part 3): signal mutations + observability reads
Implemented 5 new endpoints:
- POST /signals/{id}/ack — acknowledge (raised|failed → acknowledged)
- POST /signals/{id}/resolve — resolve (raised|ack|acting|failed → resolved)
- POST /signals/{id}/mute — mute with TTL (raised|ack → muted)
- GET /events — historical events (filter by type/entity/severity/
  correlation_id/time range, cursor pagination)
- GET /audit — audit log (filter by actor/action/entity/correlation_id/
  time range, cursor pagination)

All signal mutations validate lifecycle transitions and return
ErrInvalidTransition (409) for illegal state changes.
Removed from stubs.go: AckSignal, ResolveSignal, MuteSignal,
QueryEvents, QueryAudit.

Stubs remaining: 33 endpoints (mutations + remaining reads)
2026-07-07 08:57:40 +02:00
c9975d60a5 phase 2 (part 2): sqlc queries, audit/event helpers, event NOTIFY trigger
- sqlc.yaml + internal/db/queries/*.sql: typed queries for entities,
  relationships, ontology, operations (signals, events, audit,
  idempotency, entity_status)
- internal/db/sqlcgen/: generated Go from sqlc (pgx/v5)
- internal/observability/record.go: Audit() and Event() helpers that
  write in the caller's transaction (SG10). actorLabel is interim text
  identity in detail JSON until OIDC resolution lands; actor_id column
  exists but is not yet populated
- migrations/008: post-commit pg_notify trigger on events table for
  SSE fan-out (SG8/SG10)
2026-07-07 08:49:59 +02:00
f2fe812cda phase 2 (part 1): OpenAPI-generated API server, first 9 endpoints
- api/openapi.yaml converted 3.1 → 3.0.3 (oapi-codegen/kin-openapi
  supports 3.0; nullable syntax + example keywords), still redocly-clean
- oapi-codegen (v2.4.1, strict server + chi) generates
  internal/httpapi/gen from the spec; `make generate` wired
- internal/httpapi: chi router, /healthz (unauthenticated, SG18),
  RFC 9457 problem+json mapping from domain sentinels (SG11), 5xx detail
  logged server-side only, request logging with request IDs, graceful
  shutdown (SG4), interim static bearer auth (constant-time; dev-open
  when no token; OIDC JWT still to come in Phase 2)
- Implemented: listEntities (type filter walks the hierarchy, keyset
  pagination), getEntity (UUID or slug, ETag), getEntityRelations,
  getBlastRadius, getGraph (nodes+edges for UIs), getOntology,
  listSignals, getFleetHealth, exportSeeds. Remaining 38 ops return 501
  problem+json stubs (compiler-enforced interface completeness)
- `oikos api` role live: migrate-on-start, serves :8090
- 15 API integration tests (auth, pagination, hierarchy filter, ETag,
  404/501 problem shapes, graph, export)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:25:19 +02:00
1b04683639 phase 1 review fixes: dedup edges, real export, validation, tests
Review of aa2ca0a found and fixed:
- re-ingest duplicated ALL edges (upsert conflicted on valid_from=now(),
  never fired) — migration 007 dedupes + partial unique index on current
  edges; upsert now targets it. Regression-tested.
- export was a stub that overwrote seeds/*.yaml with 11-byte "version: 1"
  files — implemented real deterministic export (ontology/inventory/policy,
  cognition-layer excluded); round-trip is byte-stable (tested)
- DB password leaked in startup logs (slog JSON bypasses String()) —
  Config now implements slog.LogValuer; regression-tested
- docker-compose had literal '***' as DB password — env-interpolated
- uuid.New() (v4) → uuid.NewV7() per ADR-0005
- no ontology validation on ingest — internal/ontology TypeTree: abstract
  instantiation rejected, relationship endpoints hierarchy-validated,
  cardinality enforced in-transaction, lifecycle states checked, default
  state applied (Phase 1 gate items, R3-1)
- getOrCreateEntityID swallowed non-ErrNoRows errors
- migration runner now holds a session advisory lock on one connection
- Makefile: hardcoded /opt/homebrew/bin/go → go; test-db target

Tests: 4 unit suites + 7 integration tests (env-guarded, throwaway DB per
run): migrate idempotent, seed idempotent + no dup edges, abstract/edge/
cardinality rejection, blast_radius cycle termination, export round-trip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:11:01 +02:00
aa2ca0ae6f phase 1: Go foundation — module, migrations, domain, seed ingest
Core deliverables:
- Go module github.com/dtoro/oikos (Go 1.26.3)
- cmd/oikos: single binary with role subcommands (migrate, seed, export)
- 6 SQL migrations: ontology meta-schema, entity instances (UUID+slug,
  blast_radius recursive function), operations (signals/checks/approvals),
  cognition (classifications/executions/feedback/patterns/skills), policy,
  observability (TimescaleDB hypertables + CAGGs + retention)
- Domain layer: entity, signal, execution, classification, pattern, skill,
  approval, check types + 11 sentinel errors + lifecycle state machines
- DB layer: pgx pool, SQL splitter (handles 94436 and -- comments), migration
  runner, seed ingest (ontology+inventory+policy) with content-hash dedup
- Config: env-based with defaults, secrets redaction
- Observability: slog JSON logger with debug mode
- Infrastructure: Makefile, docker-compose.yml, multi-stage Dockerfile
  (distroless, CGO_ENABLED=0)

Verified end-to-end against timescale/timescaledb:2.17.2-pg16:
- 6 migrations applied (65 SQL statements)
- Seeds ingested: 6 lifecycles, 59 entity types, 46 relationship types,
  111 entities, 144 relationships, 4 risk classes, 27 approval rules,
  9 autonomy settings
- Idempotent: second seed run is a no-op (content hash matches)

Bugs fixed during implementation:
- TimescaleDB CAGGs can't run in a transaction -> splitSQL() executes
  statements individually
- Semicolons in -- comments treated as separators -> comment handling
- YAML keys source/target didn't match code's source_type/target_type
- yaml.Marshal produced YAML for JSONB columns -> json.Marshal
2026-07-07 01:07:26 +02:00