docs: add client lifecycle plan, cleanup stale files, document repo for 3 audiences
Problem: Repo had no developer guide, no client onboarding doc, no agent dev instructions. Stale files (675KB SQL dump, one-off convert script, legacy MCP builder) cluttered the tree. Client enrollment was a documented intention with no Go implementation. Changes: - New docs: CONTRIBUTING.md (dev setup), CLIENTS.md (client onboarding), .agents/dev/CONTRIBUTING.md (agent codebase map) - New plan: plans/2026-07-07-client-lifecycle-in-go.md — full client lifecycle (planned→provisioning→active→deprecated→destroyed) in Go, replacing archived Python secrets-issuance, adding client API endpoints and 6 missing MCP tools - Cleanup: deleted archive/convert-wiki.py (one-off), archive/mcp/ build_host_files.py (legacy), backups/pre-deploy-7f7d039.sql (local) - Fixes: plans/index.md duplicate row removed, README.md repo layout updated for current state, AGENTS.md header points to new guides Risk: low. Docs only + stale file deletion. No code changes. New plan is proposal, not implementation. Verification: git diff reviewed, all changes are prose/docs/plans.
This commit is contained in:
179
.agents/dev/CONTRIBUTING.md
Normal file
179
.agents/dev/CONTRIBUTING.md
Normal file
@@ -0,0 +1,179 @@
|
||||
# Agent developer guide
|
||||
|
||||
Instructions for AI agents working on the Oikos codebase. Read this after
|
||||
[AGENTS.md](../../AGENTS.md) and [OIKOS.md](../OIKOS.md). Human developers:
|
||||
see [CONTRIBUTING.md](../../CONTRIBUTING.md) for a human-friendly version.
|
||||
|
||||
## Codebase map
|
||||
|
||||
```
|
||||
cmd/oikos/main.go Entry point. Subcommands: api, scheduler, notifier, migrate,
|
||||
seed, export, secret, all
|
||||
cmd/hermes/main.go Hermes MCP client gateway (standalone binary)
|
||||
internal/httpapi/ REST + MCP server. Chi router. OpenAPI-generated types from
|
||||
internal/httpapi/gen/api.gen.go. Strict server in impl.go.
|
||||
internal/mcp/ MCP tool implementations (get_entity, search_knowledge, etc.)
|
||||
internal/db/ Connection pool (pool.go), seed ingestion (seed.go), DB→YAML
|
||||
export (export.go), type hierarchy (typetree.go)
|
||||
internal/db/queries/ SQL query files → sqlc generates internal/db/sqlcgen/
|
||||
internal/scheduler/ Observe loop: probes, signals, check_defs
|
||||
internal/actuator/ SSH execution with circuit breaker + retry
|
||||
internal/learning/ Pattern extraction, anomaly detection
|
||||
internal/notifier/ Matrix notification + approval token generation
|
||||
internal/policy/ Risk classifier (read policy.yaml → classify action)
|
||||
internal/secrets/ Backend abstraction: Infisical (primary) + SOPS (fallback)
|
||||
internal/domain/ Core types: entities, approvals, executions, signals, patterns
|
||||
internal/ontology/ Type hierarchy validation, relationship checks
|
||||
internal/knowledge/ Knowledge YAML seed ingestion
|
||||
internal/config/ Config loading from env vars
|
||||
api/openapi.yaml REST API contract. Source of truth for endpoints.
|
||||
api/codegen.yaml oapi-codegen config → generates internal/httpapi/gen/
|
||||
migrations/ Forward-only SQL. Format: NNN_name.up.sql. No down migrations.
|
||||
seeds/ Bootstrap YAML. ontology.yaml, inventory.yaml, policy.yaml,
|
||||
knowledge.yaml. Regenerated from DB via oikos export.
|
||||
compose/ Dockerfiles. oikos/ (multi-stage), hermes/ (distroless).
|
||||
Caddy config at compose/caddy/Caddyfile.oikos.
|
||||
scripts/ Deploy, rollback, watchdog, verification, cutover checklist.
|
||||
hermes/ Hermes config.yaml, SOUL.md, skills.
|
||||
.agents/ Agent instruction files, domains, shared conventions, skills.
|
||||
plans/ Design documents. active/ + done/.
|
||||
docs/adr/ Architecture decision records. Numbered, prefix-sorted.
|
||||
```
|
||||
|
||||
## Development loop
|
||||
|
||||
```bash
|
||||
# Start dependencies
|
||||
make dev
|
||||
|
||||
# Generate code after API/SQL changes
|
||||
make generate
|
||||
|
||||
# Build
|
||||
make build
|
||||
|
||||
# Run tests
|
||||
make test # all unit tests
|
||||
make test-db # integration tests (needs compose Postgres)
|
||||
|
||||
# Lint
|
||||
make lint
|
||||
|
||||
# CI drift guard (run before commit)
|
||||
make generate-check
|
||||
```
|
||||
|
||||
## Adding a feature or phase
|
||||
|
||||
Oikos features follow a phase model (read [OIKOS.md](../OIKOS.md) for the
|
||||
current phase status). To add a new capability:
|
||||
|
||||
1. **ADR first.** Write an architecture decision record in `docs/adr/` with
|
||||
the next sequence number. Document the decision, context, alternatives
|
||||
considered, and consequences.
|
||||
2. **Plan.** If the change is non-trivial, create a plan in `plans/` following
|
||||
the template in [page-templates.md](../shared/page-templates.md).
|
||||
3. **Schema.** If the feature needs new DB tables, write a forward-only
|
||||
migration in `migrations/`. Use `IF NOT EXISTS` for idempotency.
|
||||
4. **API.** If the feature exposes endpoints, define them in
|
||||
`api/openapi.yaml` first, then run `make generate`, then implement.
|
||||
5. **Domain.** Add types to `internal/domain/` before adding logic.
|
||||
6. **Tests.** Write tests alongside implementation. Integration tests go in
|
||||
`*_test.go` in the relevant package, using the compose Postgres.
|
||||
7. **Policy.** If the feature introduces new mutation types, update
|
||||
`seeds/policy.yaml` and the classifier in `internal/policy/`.
|
||||
8. **Run `make generate-check`** before commit to ensure generated code is
|
||||
current.
|
||||
|
||||
## SQL conventions
|
||||
|
||||
- Queries live in `internal/db/queries/*.sql` with `-- name: FuncName :exec`
|
||||
annotations for sqlc
|
||||
- Use `pgx/v5` driver. UUIDs use `pgtype.UUID`, timestamps use `time.Time`
|
||||
- CTEs for graph traversals (blast radius, dependency chains)
|
||||
- CAGGs and retention policies for TimescaleDB hypertables
|
||||
- FTS via `tsvector` + `tsquery` for knowledge search (migration 011)
|
||||
|
||||
## OpenAPI codegen
|
||||
|
||||
- Config: `api/codegen.yaml`. Uses `oapi-codegen/v2` with Chi server template
|
||||
- Generated output: `internal/httpapi/gen/api.gen.go` — never hand-edit
|
||||
- Strict server interface: `api.gen.go` generates the `StrictServerInterface`;
|
||||
implement it in `internal/httpapi/impl.go`
|
||||
- Problem+JSON errors via `internal/httpapi/problem.go` — RFC 9457 format
|
||||
- Cursor pagination, If-Match/ETag, idempotency keys, SSE streaming
|
||||
|
||||
## Testing philosophy
|
||||
|
||||
- **Race detector always on.** `make test` runs `go test -race -cover ./...`
|
||||
- **Integration tests** use the compose Postgres. Run with `make test-db`.
|
||||
Each test creates + tears down its own schema namespace.
|
||||
- **Coverage gates** in CI: policy + learning ≥ 80%, others ≥ 60%
|
||||
- Tests use `testing.T` directly, no assertion library
|
||||
- Table-driven tests for validation and classification logic
|
||||
|
||||
## Migration rules
|
||||
|
||||
- Forward-only. No down migrations (ADR 0008)
|
||||
- Idempotent: use `IF NOT EXISTS`, `DO $$ BEGIN ... END $$` blocks
|
||||
- Sequence numbers are sequential integers (001, 002, ...)
|
||||
- Each migration file is `NNN_name.up.sql`
|
||||
- Migrations are embedded in the binary via `migrations/embed.go`
|
||||
|
||||
## Seed files
|
||||
|
||||
- `seeds/ontology.yaml` — entity types, relationship types, lifecycles
|
||||
(validated against schema in `internal/ontology/`)
|
||||
- `seeds/inventory.yaml` — hosts, services, entities (the topology)
|
||||
- `seeds/policy.yaml` — risk classes, approval rules, autonomy settings
|
||||
- `seeds/knowledge.yaml` — documents, investigations, runbooks (DB is source
|
||||
of truth; this file is the DR export)
|
||||
- After DB changes via the API, run `make export` to regenerate seeds
|
||||
|
||||
## Secrets handling
|
||||
|
||||
- No secrets in code, config, or commits
|
||||
- Dev secrets in `.env` (gitignored)
|
||||
- Primary: Infisical (`internal/secrets/infisical.go`)
|
||||
- Fallback: SOPS + age (`internal/secrets/sops.go`)
|
||||
- Backend interface: `internal/secrets/backend.go`
|
||||
- Machine identities via Infisical UniversalAuth
|
||||
- In-memory cache with TTL for performance
|
||||
|
||||
## Staging and deployment
|
||||
|
||||
- CI pipeline: `.gitea/workflows/ci.yml` — lint, vet, vulncheck, test, docker build
|
||||
- Deploy: `scripts/deploy.sh` — git pull → docker build → compose up → health check
|
||||
- Watchdog: `scripts/watchdog.sh` — 2-minute cron, Matrix alert on failure
|
||||
- Rollback: `scripts/rollback.sh` — checkout SHA + pg_restore
|
||||
- Cutover checklist: `scripts/cutover-checklist.md`
|
||||
|
||||
## Writing conventions
|
||||
|
||||
Apply [writing-style.md](../shared/writing-style.md) for all committed prose.
|
||||
Terse, reference-style, no marketing vocabulary. Code comments explain intent
|
||||
and trade-offs, not mechanics.
|
||||
|
||||
Apply [caveman.md](../shared/caveman.md) for agent communication. The caveman
|
||||
standard applies to agent *chat responses*, not committed documentation.
|
||||
|
||||
## Skills
|
||||
|
||||
Agent skills live under `.agents/skills/<name>/SKILL.md`. Each skill has a
|
||||
frontmatter description that tools match against tasks. To add a skill:
|
||||
|
||||
1. Create `.agents/skills/<name>/SKILL.md`
|
||||
2. Include frontmatter with description field
|
||||
3. Document the procedure following the runbook template
|
||||
4. Reference relevant files, commands, and policy classes
|
||||
|
||||
Skills that require code (e.g. linting) may include companion scripts in the
|
||||
same directory.
|
||||
|
||||
## When in doubt
|
||||
|
||||
- Query MCP tools first (search_knowledge, get_entity)
|
||||
- Read the relevant ADR in `docs/adr/`
|
||||
- Grep the codebase: `rg <symbol> internal/`
|
||||
- Check `plans/` for in-progress work that may conflict
|
||||
- Classify any new mutation against `seeds/policy.yaml` before suggesting it
|
||||
Reference in New Issue
Block a user