Files
oikos/.agents/dev/CONTRIBUTING.md
dtoro d2950dd09d refactor: sqlc vs raw SQL — hybrid approach (R3)
Deleted 8 genuinely unused sqlc queries (no inline equivalent):
- UpsertCurrentRelationship, ListEntitiesCapped, ListEntityStatus,
  UpdateSignalState, InsertClassification, InsertFeedback, InsertSkill,
  UpsertCurrentRelationship — all had zero call sites.

Migrated 9 inline raw SQL sites to use sqlc queries:
- GetOntology (impl.go): ListEntityTypes, ListRelationshipTypes,
  ListLifecycleDefs — replaces 3 raw pool.Query blocks with typed sqlcgen
  calls, eliminating manual row scanning.
- EndRelationship (phase3.go): EndCurrentRelationship — replaces tx.Exec
  with sqlcgen.New(tx).EndCurrentRelationship.
- checkPrecondition (impl.go): GetEntityStatus — replaces tx.QueryRow +
  manual Scan with sqlcgen.New(tx).GetEntityStatus.
- GetEntityRelations (impl.go): ListEntityRelations — replaces raw pool.Query
  + scanRelationships helper (now deleted).
- GetGraph (impl.go): ListGraphEdges — replaces raw pool.Query +
  scanRelationships.
- resolveEntityID (impl.go): GetEntityBySlug/GetEntityByID — replaces
  raw pool.QueryRow + Scan.
- createApproval (mcp/server.go): InsertApproval — replaces raw pool.Exec
  with sqlcgen.InsertApproval.

Deleted scanRelationships helper (was only used by the two migrated
graph queries above).

Regenerated sqlcgen — also picks up stale model updates (AgentSession,
SessionPlanStep, SessionQuestion, etc. from recent migrations).

Documented the carve-out in .agents/dev/CONTRIBUTING.md §SQL conventions:
sqlc is the default; raw pool.Query/Exec is reserved for LISTEN/NOTIFY,
dynamic WHERE builders, blast_radius(), and COPY.

go vet, build, httpapi/mcp/db tests all pass. -383/+170 lines.
2026-07-17 22:24:23 +02:00

8.8 KiB

Agent developer guide

Instructions for AI agents working on the Oikos codebase. Read this after AGENTS.md and OIKOS.md. Human developers: see CONTRIBUTING.md for a human-friendly version.

Codebase map

cmd/oikos/main.go          Entry point. Subcommands: api, scheduler, notifier, migrate,
                             seed, export, secret, all
cmd/nomos/main.go          Nomos MCP client gateway (standalone binary, formerly Hermes)
cmd/webhook/main.go        Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
internal/httpapi/           REST + MCP server. Chi router. OpenAPI-generated types from
                             internal/httpapi/gen/api.gen.go. Strict server in impl.go.
internal/mcp/               MCP tool implementations (get_entity, search_knowledge, etc.)
internal/db/                Connection pool (pool.go), seed ingestion (seed.go), DB→YAML
                             export (export.go), type hierarchy (typetree.go)
internal/db/queries/        SQL query files → sqlc generates internal/db/sqlcgen/
internal/scheduler/         Observe loop: probes, signals, check_defs
internal/actuator/          SSH execution with circuit breaker + retry
internal/learning/          Pattern extraction, anomaly detection
internal/notifier/          Matrix notification + approval token generation
internal/policy/            Risk classifier (read policy.yaml → classify action)
internal/secrets/           Backend abstraction: Infisical (primary) + SOPS (fallback)
internal/domain/            Core types: entities, approvals, executions, signals, patterns
internal/ontology/          Type hierarchy validation, relationship checks
internal/knowledge/         Knowledge YAML seed ingestion
internal/config/            Config loading from env vars
web/                         Control-room SPA (Svelte 5) — standalone static build, not
                             embedded in the oikos binary (plans/2026-07-12-wails-desktop-app.md)
api/openapi.yaml            REST API contract. Source of truth for endpoints.
api/codegen.yaml            oapi-codegen config → generates internal/httpapi/gen/
migrations/                 Forward-only SQL. Format: NNN_name.up.sql. No down migrations.
seeds/                      Bootstrap YAML. ontology.yaml, inventory.yaml, policy.yaml,
                             knowledge.yaml. Regenerated from DB via oikos export.
compose/                    Dockerfiles. oikos/ (2-stage, Go only — SPA is built/deployed
                             separately), nomos/ (distroless).
                             Caddy config at compose/caddy/Caddyfile.oikos.
scripts/                     Deploy, rollback, watchdog, verification, cutover checklist.
checks/                      Host health-check scripts run over SSH by the scheduler.
tools/                       Client auto-setup scripts (checks).
nomos/                       Nomos config.yaml, SOUL.md, skills.
.agents/                     Agent instruction files, domains, shared conventions, skills.
plans/                       Design documents. active/ + done/.
docs/adr/                    Architecture decision records. Numbered, prefix-sorted.

Development loop

# Start dependencies
make dev

# Generate code after API/SQL changes
make generate

# Build
make build

# Run tests
make test           # all unit tests
make test-db        # integration tests (needs compose Postgres)

# Lint
make lint

# CI drift guard (run before commit)
make generate-check

Adding a feature or phase

Oikos features follow a phase model (read OIKOS.md for the current phase status). To add a new capability:

  1. ADR first. Write an architecture decision record in docs/adr/ with the next sequence number. Document the decision, context, alternatives considered, and consequences.
  2. Plan. If the change is non-trivial, create a plan in plans/ following the template in page-templates.md.
  3. Schema. If the feature needs new DB tables, write a forward-only migration in migrations/. Use IF NOT EXISTS for idempotency.
  4. API. If the feature exposes endpoints, define them in api/openapi.yaml first, then run make generate, then implement.
  5. Domain. Add types to internal/domain/ before adding logic.
  6. Tests. Write tests alongside implementation. Integration tests go in *_test.go in the relevant package, using the compose Postgres.
  7. Policy. If the feature introduces new mutation types, update seeds/policy.yaml and the classifier in internal/policy/.
  8. Run make generate-check before commit to ensure generated code is current.

SQL conventions

  • Queries live in internal/db/queries/*.sql with -- name: FuncName :exec annotations for sqlc. Generated code in internal/db/sqlcgen/ — never hand-edit. Call via sqlcgen.New(pool).QueryName(ctx, params).
  • sqlc is the default for all DB access. Raw pool.Query/Exec with inline SQL is a documented carve-out for cases sqlc can't express: LISTEN/NOTIFY, dynamic WHERE-clause builders, blast_radius() (opaque return type), and COPY. All other DB access should go through sqlc queries.
  • Use pgx/v5 driver. UUIDs use pgtype.UUID, timestamps use time.Time
  • CTEs for graph traversals (blast radius, dependency chains)
  • CAGGs and retention policies for TimescaleDB hypertables
  • FTS via tsvector + tsquery for knowledge search (migration 011)

OpenAPI codegen

  • Config: api/codegen.yaml. Uses oapi-codegen/v2 with Chi server template
  • Generated output: internal/httpapi/gen/api.gen.go — never hand-edit
  • Strict server interface: api.gen.go generates the StrictServerInterface; implement it in internal/httpapi/impl.go
  • Problem+JSON errors via internal/httpapi/problem.go — RFC 9457 format
  • Cursor pagination, If-Match/ETag, idempotency keys, SSE streaming

Testing philosophy

  • Race detector always on. make test runs go test -race -cover ./...
  • Integration tests use the compose Postgres. Run with make test-db. Each test creates + tears down its own schema namespace.
  • Coverage gates in CI: policy + learning ≥ 80%, others ≥ 60%
  • Tests use testing.T directly, no assertion library
  • Table-driven tests for validation and classification logic

Migration rules

  • Forward-only. No down migrations (ADR 0008)
  • Idempotent: use IF NOT EXISTS, DO $$ BEGIN ... END $$ blocks
  • Sequence numbers are sequential integers (001, 002, ...)
  • Each migration file is NNN_name.up.sql
  • Migrations are embedded in the binary via migrations/embed.go

Seed files

  • seeds/ontology.yaml — entity types, relationship types, lifecycles (validated against schema in internal/ontology/)
  • seeds/inventory.yaml — hosts, services, entities (the topology)
  • seeds/policy.yaml — risk classes, approval rules, autonomy settings
  • seeds/knowledge.yaml — documents, investigations, runbooks (DB is source of truth; this file is the DR export)
  • After DB changes via the API, run make export to regenerate seeds

Secrets handling

  • No secrets in code, config, or commits
  • Dev secrets in .env (gitignored)
  • Primary: Infisical (internal/secrets/infisical.go)
  • Fallback: SOPS + age (internal/secrets/sops.go)
  • Backend interface: internal/secrets/backend.go
  • Machine identities via Infisical UniversalAuth
  • In-memory cache with TTL for performance

Staging and deployment

  • CI pipeline: .gitea/workflows/ci.yml — lint, vet, vulncheck, test, docker build
  • Deploy: scripts/deploy.sh — git pull → docker build → compose up → health check
  • Watchdog: scripts/watchdog.sh — 2-minute cron, Matrix alert on failure
  • Rollback: scripts/rollback.sh — checkout SHA + pg_restore
  • Cutover checklist: scripts/cutover-checklist.md

Writing conventions

Apply writing-style.md for all committed prose. Terse, reference-style, no marketing vocabulary. Code comments explain intent and trade-offs, not mechanics.

Apply caveman.md for agent communication. The caveman standard applies to agent chat responses, not committed documentation.

Skills

Agent skills live under .agents/skills/<name>/SKILL.md. Each skill has a frontmatter description that tools match against tasks. To add a skill:

  1. Create .agents/skills/<name>/SKILL.md
  2. Include frontmatter with description field
  3. Document the procedure following the runbook template
  4. Reference relevant files, commands, and policy classes

Skills that require code (e.g. linting) may include companion scripts in the same directory.

When in doubt

  • Query MCP tools first (search_knowledge, get_entity)
  • Read the relevant ADR in docs/adr/
  • Grep the codebase: rg <symbol> internal/
  • Check plans/ for in-progress work that may conflict
  • Classify any new mutation against seeds/policy.yaml before suggesting it