Deleted 8 genuinely unused sqlc queries (no inline equivalent): - UpsertCurrentRelationship, ListEntitiesCapped, ListEntityStatus, UpdateSignalState, InsertClassification, InsertFeedback, InsertSkill, UpsertCurrentRelationship — all had zero call sites. Migrated 9 inline raw SQL sites to use sqlc queries: - GetOntology (impl.go): ListEntityTypes, ListRelationshipTypes, ListLifecycleDefs — replaces 3 raw pool.Query blocks with typed sqlcgen calls, eliminating manual row scanning. - EndRelationship (phase3.go): EndCurrentRelationship — replaces tx.Exec with sqlcgen.New(tx).EndCurrentRelationship. - checkPrecondition (impl.go): GetEntityStatus — replaces tx.QueryRow + manual Scan with sqlcgen.New(tx).GetEntityStatus. - GetEntityRelations (impl.go): ListEntityRelations — replaces raw pool.Query + scanRelationships helper (now deleted). - GetGraph (impl.go): ListGraphEdges — replaces raw pool.Query + scanRelationships. - resolveEntityID (impl.go): GetEntityBySlug/GetEntityByID — replaces raw pool.QueryRow + Scan. - createApproval (mcp/server.go): InsertApproval — replaces raw pool.Exec with sqlcgen.InsertApproval. Deleted scanRelationships helper (was only used by the two migrated graph queries above). Regenerated sqlcgen — also picks up stale model updates (AgentSession, SessionPlanStep, SessionQuestion, etc. from recent migrations). Documented the carve-out in .agents/dev/CONTRIBUTING.md §SQL conventions: sqlc is the default; raw pool.Query/Exec is reserved for LISTEN/NOTIFY, dynamic WHERE builders, blast_radius(), and COPY. go vet, build, httpapi/mcp/db tests all pass. -383/+170 lines.
8.8 KiB
Agent developer guide
Instructions for AI agents working on the Oikos codebase. Read this after AGENTS.md and OIKOS.md. Human developers: see CONTRIBUTING.md for a human-friendly version.
Codebase map
cmd/oikos/main.go Entry point. Subcommands: api, scheduler, notifier, migrate,
seed, export, secret, all
cmd/nomos/main.go Nomos MCP client gateway (standalone binary, formerly Hermes)
cmd/webhook/main.go Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
internal/httpapi/ REST + MCP server. Chi router. OpenAPI-generated types from
internal/httpapi/gen/api.gen.go. Strict server in impl.go.
internal/mcp/ MCP tool implementations (get_entity, search_knowledge, etc.)
internal/db/ Connection pool (pool.go), seed ingestion (seed.go), DB→YAML
export (export.go), type hierarchy (typetree.go)
internal/db/queries/ SQL query files → sqlc generates internal/db/sqlcgen/
internal/scheduler/ Observe loop: probes, signals, check_defs
internal/actuator/ SSH execution with circuit breaker + retry
internal/learning/ Pattern extraction, anomaly detection
internal/notifier/ Matrix notification + approval token generation
internal/policy/ Risk classifier (read policy.yaml → classify action)
internal/secrets/ Backend abstraction: Infisical (primary) + SOPS (fallback)
internal/domain/ Core types: entities, approvals, executions, signals, patterns
internal/ontology/ Type hierarchy validation, relationship checks
internal/knowledge/ Knowledge YAML seed ingestion
internal/config/ Config loading from env vars
web/ Control-room SPA (Svelte 5) — standalone static build, not
embedded in the oikos binary (plans/2026-07-12-wails-desktop-app.md)
api/openapi.yaml REST API contract. Source of truth for endpoints.
api/codegen.yaml oapi-codegen config → generates internal/httpapi/gen/
migrations/ Forward-only SQL. Format: NNN_name.up.sql. No down migrations.
seeds/ Bootstrap YAML. ontology.yaml, inventory.yaml, policy.yaml,
knowledge.yaml. Regenerated from DB via oikos export.
compose/ Dockerfiles. oikos/ (2-stage, Go only — SPA is built/deployed
separately), nomos/ (distroless).
Caddy config at compose/caddy/Caddyfile.oikos.
scripts/ Deploy, rollback, watchdog, verification, cutover checklist.
checks/ Host health-check scripts run over SSH by the scheduler.
tools/ Client auto-setup scripts (checks).
nomos/ Nomos config.yaml, SOUL.md, skills.
.agents/ Agent instruction files, domains, shared conventions, skills.
plans/ Design documents. active/ + done/.
docs/adr/ Architecture decision records. Numbered, prefix-sorted.
Development loop
# Start dependencies
make dev
# Generate code after API/SQL changes
make generate
# Build
make build
# Run tests
make test # all unit tests
make test-db # integration tests (needs compose Postgres)
# Lint
make lint
# CI drift guard (run before commit)
make generate-check
Adding a feature or phase
Oikos features follow a phase model (read OIKOS.md for the current phase status). To add a new capability:
- ADR first. Write an architecture decision record in
docs/adr/with the next sequence number. Document the decision, context, alternatives considered, and consequences. - Plan. If the change is non-trivial, create a plan in
plans/following the template in page-templates.md. - Schema. If the feature needs new DB tables, write a forward-only
migration in
migrations/. UseIF NOT EXISTSfor idempotency. - API. If the feature exposes endpoints, define them in
api/openapi.yamlfirst, then runmake generate, then implement. - Domain. Add types to
internal/domain/before adding logic. - Tests. Write tests alongside implementation. Integration tests go in
*_test.goin the relevant package, using the compose Postgres. - Policy. If the feature introduces new mutation types, update
seeds/policy.yamland the classifier ininternal/policy/. - Run
make generate-checkbefore commit to ensure generated code is current.
SQL conventions
- Queries live in
internal/db/queries/*.sqlwith-- name: FuncName :execannotations for sqlc. Generated code ininternal/db/sqlcgen/— never hand-edit. Call viasqlcgen.New(pool).QueryName(ctx, params). - sqlc is the default for all DB access. Raw
pool.Query/Execwith inline SQL is a documented carve-out for cases sqlc can't express:LISTEN/NOTIFY, dynamic WHERE-clause builders,blast_radius()(opaque return type), andCOPY. All other DB access should go through sqlc queries. - Use
pgx/v5driver. UUIDs usepgtype.UUID, timestamps usetime.Time - CTEs for graph traversals (blast radius, dependency chains)
- CAGGs and retention policies for TimescaleDB hypertables
- FTS via
tsvector+tsqueryfor knowledge search (migration 011)
OpenAPI codegen
- Config:
api/codegen.yaml. Usesoapi-codegen/v2with Chi server template - Generated output:
internal/httpapi/gen/api.gen.go— never hand-edit - Strict server interface:
api.gen.gogenerates theStrictServerInterface; implement it ininternal/httpapi/impl.go - Problem+JSON errors via
internal/httpapi/problem.go— RFC 9457 format - Cursor pagination, If-Match/ETag, idempotency keys, SSE streaming
Testing philosophy
- Race detector always on.
make testrunsgo test -race -cover ./... - Integration tests use the compose Postgres. Run with
make test-db. Each test creates + tears down its own schema namespace. - Coverage gates in CI: policy + learning ≥ 80%, others ≥ 60%
- Tests use
testing.Tdirectly, no assertion library - Table-driven tests for validation and classification logic
Migration rules
- Forward-only. No down migrations (ADR 0008)
- Idempotent: use
IF NOT EXISTS,DO $$ BEGIN ... END $$blocks - Sequence numbers are sequential integers (001, 002, ...)
- Each migration file is
NNN_name.up.sql - Migrations are embedded in the binary via
migrations/embed.go
Seed files
seeds/ontology.yaml— entity types, relationship types, lifecycles (validated against schema ininternal/ontology/)seeds/inventory.yaml— hosts, services, entities (the topology)seeds/policy.yaml— risk classes, approval rules, autonomy settingsseeds/knowledge.yaml— documents, investigations, runbooks (DB is source of truth; this file is the DR export)- After DB changes via the API, run
make exportto regenerate seeds
Secrets handling
- No secrets in code, config, or commits
- Dev secrets in
.env(gitignored) - Primary: Infisical (
internal/secrets/infisical.go) - Fallback: SOPS + age (
internal/secrets/sops.go) - Backend interface:
internal/secrets/backend.go - Machine identities via Infisical UniversalAuth
- In-memory cache with TTL for performance
Staging and deployment
- CI pipeline:
.gitea/workflows/ci.yml— lint, vet, vulncheck, test, docker build - Deploy:
scripts/deploy.sh— git pull → docker build → compose up → health check - Watchdog:
scripts/watchdog.sh— 2-minute cron, Matrix alert on failure - Rollback:
scripts/rollback.sh— checkout SHA + pg_restore - Cutover checklist:
scripts/cutover-checklist.md
Writing conventions
Apply writing-style.md for all committed prose. Terse, reference-style, no marketing vocabulary. Code comments explain intent and trade-offs, not mechanics.
Apply caveman.md for agent communication. The caveman standard applies to agent chat responses, not committed documentation.
Skills
Agent skills live under .agents/skills/<name>/SKILL.md. Each skill has a
frontmatter description that tools match against tasks. To add a skill:
- Create
.agents/skills/<name>/SKILL.md - Include frontmatter with description field
- Document the procedure following the runbook template
- Reference relevant files, commands, and policy classes
Skills that require code (e.g. linting) may include companion scripts in the same directory.
When in doubt
- Query MCP tools first (search_knowledge, get_entity)
- Read the relevant ADR in
docs/adr/ - Grep the codebase:
rg <symbol> internal/ - Check
plans/for in-progress work that may conflict - Classify any new mutation against
seeds/policy.yamlbefore suggesting it