dtoro
d80a394b7f
Documentation and repo-hygiene pass following the client/server split:
Plan drift (audited all other active plans against current code):
- oikos-gaps-and-improvements.md: mark Section C and D.5 resolved (both
described cmd/hermes, renamed to cmd/nomos with a real LLM loop since);
refresh ~10 stale file:line citations; fix tool-count (33, not 28).
- liveness-drift-and-ux-cohesion.md: fix stale default-model claim (now
deepseek-v4-pro since 2026-07-10) and "not yet deployed" status.
- nomos-agent-code-review.md: fix C1's citation (one unauthenticated route
to nomos now, not two, after the client/server split).
- wails-desktop-app.md: record the production deploy outcome.
Repo structure: added missing directories to README/CONTRIBUTING layout
tables (checks/, tools/, cmd/webhook/, docs/operations/), fixed a broken
link, added ADR 0015 documenting the auth/CORS/client-split model (there
wasn't one despite CONTRIBUTING's own process requiring it), normalized
ADR 0013/0014's format drift, added an Authentication section to
AGENTS.md/CLIENTS.md (every example call was missing the now-required
bearer header).
Retired the Goose+Nomos workstation flow (bootstrap.sh --with-nomos,
tools/setup-nomos-soul.sh, .agents/operations/nomos-agent.md) and the
Caveman auto-install tooling (tools/setup-caveman.sh, tools/caveman/) —
both superseded by the production containerized Nomos agent, which has
never used either. Kept .agents/shared/caveman.md itself (the terse
writing-style convention agents still follow by reading it).
Deleted the orphaned legacy Python oikos/ directory — nothing imports it,
and bin/homelab (the CLI it was kept for) no longer exists in the repo.
Rewrote .agents/operations/agent-enrollment.md (365 -> ~110 lines) and
commands.md to match the current architecture instead of the retired
`homelab` CLI; migrated the still-true networking prerequisites (Netbird,
split-horizon DNS, SSH key distribution) into the knowledge base as a
runbook via upsert_knowledge rather than duplicating them in markdown.
Updated all 10 .agents/skills/ runbooks referencing the dead CLI with
their real MCP tool / REST API equivalents, or flagged them as needing
verification where no equivalent is confirmed yet.
Two real bugs found and fixed, not just docs:
- The tools/setup-*.sh auto-setup glob was tools/*.setup.sh in THREE
places (tools/post-pull.sh, bootstrap.sh, and internal/httpapi/impl.go's
GetClientContext handler) since the mechanism's introduction on
2026-06-02 — never matched any real filename, so no client has ever
picked up an auto-setup script via git-pull or the context-poller sync.
Fixed all three; the Go server-side fix is the one that actually matters
since it's what the current context-poller mechanism depends on.
- bootstrap.sh removed dead vestigial --gitea-token/--gitea-user flags
(parsed, never consumed) left over from an earlier clone-based model.
Also flagged, not fixed (documented as an open gap in
client-enrollment/SKILL.md): bootstrap.sh tells a freshly-enrolled client
to call POST /api/v1/clients/{slug}/activate to finish enrollment, but
that route doesn't exist in api/openapi.yaml — EnrollClient sets entities
to provisioning and nothing currently transitions them to active.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
185 lines
8.4 KiB
Markdown
185 lines
8.4 KiB
Markdown
# Agent developer guide
|
|
|
|
Instructions for AI agents working on the Oikos codebase. Read this after
|
|
[AGENTS.md](../../AGENTS.md) and [OIKOS.md](../OIKOS.md). Human developers:
|
|
see [CONTRIBUTING.md](../../CONTRIBUTING.md) for a human-friendly version.
|
|
|
|
## Codebase map
|
|
|
|
```
|
|
cmd/oikos/main.go Entry point. Subcommands: api, scheduler, notifier, migrate,
|
|
seed, export, secret, all
|
|
cmd/nomos/main.go Nomos MCP client gateway (standalone binary, formerly Hermes)
|
|
cmd/webhook/main.go Gitea deploy-webhook receiver (push-to-deploy on mac-mini)
|
|
internal/httpapi/ REST + MCP server. Chi router. OpenAPI-generated types from
|
|
internal/httpapi/gen/api.gen.go. Strict server in impl.go.
|
|
internal/mcp/ MCP tool implementations (get_entity, search_knowledge, etc.)
|
|
internal/db/ Connection pool (pool.go), seed ingestion (seed.go), DB→YAML
|
|
export (export.go), type hierarchy (typetree.go)
|
|
internal/db/queries/ SQL query files → sqlc generates internal/db/sqlcgen/
|
|
internal/scheduler/ Observe loop: probes, signals, check_defs
|
|
internal/actuator/ SSH execution with circuit breaker + retry
|
|
internal/learning/ Pattern extraction, anomaly detection
|
|
internal/notifier/ Matrix notification + approval token generation
|
|
internal/policy/ Risk classifier (read policy.yaml → classify action)
|
|
internal/secrets/ Backend abstraction: Infisical (primary) + SOPS (fallback)
|
|
internal/domain/ Core types: entities, approvals, executions, signals, patterns
|
|
internal/ontology/ Type hierarchy validation, relationship checks
|
|
internal/knowledge/ Knowledge YAML seed ingestion
|
|
internal/config/ Config loading from env vars
|
|
web/ Control-room SPA (Svelte 5) — standalone static build, not
|
|
embedded in the oikos binary (plans/2026-07-12-wails-desktop-app.md)
|
|
api/openapi.yaml REST API contract. Source of truth for endpoints.
|
|
api/codegen.yaml oapi-codegen config → generates internal/httpapi/gen/
|
|
migrations/ Forward-only SQL. Format: NNN_name.up.sql. No down migrations.
|
|
seeds/ Bootstrap YAML. ontology.yaml, inventory.yaml, policy.yaml,
|
|
knowledge.yaml. Regenerated from DB via oikos export.
|
|
compose/ Dockerfiles. oikos/ (2-stage, Go only — SPA is built/deployed
|
|
separately), nomos/ (distroless).
|
|
Caddy config at compose/caddy/Caddyfile.oikos.
|
|
scripts/ Deploy, rollback, watchdog, verification, cutover checklist.
|
|
checks/ Host health-check scripts run over SSH by the scheduler.
|
|
tools/ Client auto-setup scripts (checks).
|
|
nomos/ Nomos config.yaml, SOUL.md, skills.
|
|
.agents/ Agent instruction files, domains, shared conventions, skills.
|
|
plans/ Design documents. active/ + done/.
|
|
docs/adr/ Architecture decision records. Numbered, prefix-sorted.
|
|
```
|
|
|
|
## Development loop
|
|
|
|
```bash
|
|
# Start dependencies
|
|
make dev
|
|
|
|
# Generate code after API/SQL changes
|
|
make generate
|
|
|
|
# Build
|
|
make build
|
|
|
|
# Run tests
|
|
make test # all unit tests
|
|
make test-db # integration tests (needs compose Postgres)
|
|
|
|
# Lint
|
|
make lint
|
|
|
|
# CI drift guard (run before commit)
|
|
make generate-check
|
|
```
|
|
|
|
## Adding a feature or phase
|
|
|
|
Oikos features follow a phase model (read [OIKOS.md](../OIKOS.md) for the
|
|
current phase status). To add a new capability:
|
|
|
|
1. **ADR first.** Write an architecture decision record in `docs/adr/` with
|
|
the next sequence number. Document the decision, context, alternatives
|
|
considered, and consequences.
|
|
2. **Plan.** If the change is non-trivial, create a plan in `plans/` following
|
|
the template in [page-templates.md](../shared/page-templates.md).
|
|
3. **Schema.** If the feature needs new DB tables, write a forward-only
|
|
migration in `migrations/`. Use `IF NOT EXISTS` for idempotency.
|
|
4. **API.** If the feature exposes endpoints, define them in
|
|
`api/openapi.yaml` first, then run `make generate`, then implement.
|
|
5. **Domain.** Add types to `internal/domain/` before adding logic.
|
|
6. **Tests.** Write tests alongside implementation. Integration tests go in
|
|
`*_test.go` in the relevant package, using the compose Postgres.
|
|
7. **Policy.** If the feature introduces new mutation types, update
|
|
`seeds/policy.yaml` and the classifier in `internal/policy/`.
|
|
8. **Run `make generate-check`** before commit to ensure generated code is
|
|
current.
|
|
|
|
## SQL conventions
|
|
|
|
- Queries live in `internal/db/queries/*.sql` with `-- name: FuncName :exec`
|
|
annotations for sqlc
|
|
- Use `pgx/v5` driver. UUIDs use `pgtype.UUID`, timestamps use `time.Time`
|
|
- CTEs for graph traversals (blast radius, dependency chains)
|
|
- CAGGs and retention policies for TimescaleDB hypertables
|
|
- FTS via `tsvector` + `tsquery` for knowledge search (migration 011)
|
|
|
|
## OpenAPI codegen
|
|
|
|
- Config: `api/codegen.yaml`. Uses `oapi-codegen/v2` with Chi server template
|
|
- Generated output: `internal/httpapi/gen/api.gen.go` — never hand-edit
|
|
- Strict server interface: `api.gen.go` generates the `StrictServerInterface`;
|
|
implement it in `internal/httpapi/impl.go`
|
|
- Problem+JSON errors via `internal/httpapi/problem.go` — RFC 9457 format
|
|
- Cursor pagination, If-Match/ETag, idempotency keys, SSE streaming
|
|
|
|
## Testing philosophy
|
|
|
|
- **Race detector always on.** `make test` runs `go test -race -cover ./...`
|
|
- **Integration tests** use the compose Postgres. Run with `make test-db`.
|
|
Each test creates + tears down its own schema namespace.
|
|
- **Coverage gates** in CI: policy + learning ≥ 80%, others ≥ 60%
|
|
- Tests use `testing.T` directly, no assertion library
|
|
- Table-driven tests for validation and classification logic
|
|
|
|
## Migration rules
|
|
|
|
- Forward-only. No down migrations (ADR 0008)
|
|
- Idempotent: use `IF NOT EXISTS`, `DO $$ BEGIN ... END $$` blocks
|
|
- Sequence numbers are sequential integers (001, 002, ...)
|
|
- Each migration file is `NNN_name.up.sql`
|
|
- Migrations are embedded in the binary via `migrations/embed.go`
|
|
|
|
## Seed files
|
|
|
|
- `seeds/ontology.yaml` — entity types, relationship types, lifecycles
|
|
(validated against schema in `internal/ontology/`)
|
|
- `seeds/inventory.yaml` — hosts, services, entities (the topology)
|
|
- `seeds/policy.yaml` — risk classes, approval rules, autonomy settings
|
|
- `seeds/knowledge.yaml` — documents, investigations, runbooks (DB is source
|
|
of truth; this file is the DR export)
|
|
- After DB changes via the API, run `make export` to regenerate seeds
|
|
|
|
## Secrets handling
|
|
|
|
- No secrets in code, config, or commits
|
|
- Dev secrets in `.env` (gitignored)
|
|
- Primary: Infisical (`internal/secrets/infisical.go`)
|
|
- Fallback: SOPS + age (`internal/secrets/sops.go`)
|
|
- Backend interface: `internal/secrets/backend.go`
|
|
- Machine identities via Infisical UniversalAuth
|
|
- In-memory cache with TTL for performance
|
|
|
|
## Staging and deployment
|
|
|
|
- CI pipeline: `.gitea/workflows/ci.yml` — lint, vet, vulncheck, test, docker build
|
|
- Deploy: `scripts/deploy.sh` — git pull → docker build → compose up → health check
|
|
- Watchdog: `scripts/watchdog.sh` — 2-minute cron, Matrix alert on failure
|
|
- Rollback: `scripts/rollback.sh` — checkout SHA + pg_restore
|
|
- Cutover checklist: `scripts/cutover-checklist.md`
|
|
|
|
## Writing conventions
|
|
|
|
Apply [writing-style.md](../shared/writing-style.md) for all committed prose.
|
|
Terse, reference-style, no marketing vocabulary. Code comments explain intent
|
|
and trade-offs, not mechanics.
|
|
|
|
Apply [caveman.md](../shared/caveman.md) for agent communication. The caveman
|
|
standard applies to agent *chat responses*, not committed documentation.
|
|
|
|
## Skills
|
|
|
|
Agent skills live under `.agents/skills/<name>/SKILL.md`. Each skill has a
|
|
frontmatter description that tools match against tasks. To add a skill:
|
|
|
|
1. Create `.agents/skills/<name>/SKILL.md`
|
|
2. Include frontmatter with description field
|
|
3. Document the procedure following the runbook template
|
|
4. Reference relevant files, commands, and policy classes
|
|
|
|
Skills that require code (e.g. linting) may include companion scripts in the
|
|
same directory.
|
|
|
|
## When in doubt
|
|
|
|
- Query MCP tools first (search_knowledge, get_entity)
|
|
- Read the relevant ADR in `docs/adr/`
|
|
- Grep the codebase: `rg <symbol> internal/`
|
|
- Check `plans/` for in-progress work that may conflict
|
|
- Classify any new mutation against `seeds/policy.yaml` before suggesting it |