docs: add client lifecycle plan, cleanup stale files, document repo for 3 audiences
Problem: Repo had no developer guide, no client onboarding doc, no agent dev instructions. Stale files (675KB SQL dump, one-off convert script, legacy MCP builder) cluttered the tree. Client enrollment was a documented intention with no Go implementation. Changes: - New docs: CONTRIBUTING.md (dev setup), CLIENTS.md (client onboarding), .agents/dev/CONTRIBUTING.md (agent codebase map) - New plan: plans/2026-07-07-client-lifecycle-in-go.md — full client lifecycle (planned→provisioning→active→deprecated→destroyed) in Go, replacing archived Python secrets-issuance, adding client API endpoints and 6 missing MCP tools - Cleanup: deleted archive/convert-wiki.py (one-off), archive/mcp/ build_host_files.py (legacy), backups/pre-deploy-7f7d039.sql (local) - Fixes: plans/index.md duplicate row removed, README.md repo layout updated for current state, AGENTS.md header points to new guides Risk: low. Docs only + stale file deletion. No code changes. New plan is proposal, not implementation. Verification: git diff reviewed, all changes are prose/docs/plans.
This commit is contained in:
179
.agents/dev/CONTRIBUTING.md
Normal file
179
.agents/dev/CONTRIBUTING.md
Normal file
@@ -0,0 +1,179 @@
|
||||
# Agent developer guide
|
||||
|
||||
Instructions for AI agents working on the Oikos codebase. Read this after
|
||||
[AGENTS.md](../../AGENTS.md) and [OIKOS.md](../OIKOS.md). Human developers:
|
||||
see [CONTRIBUTING.md](../../CONTRIBUTING.md) for a human-friendly version.
|
||||
|
||||
## Codebase map
|
||||
|
||||
```
|
||||
cmd/oikos/main.go Entry point. Subcommands: api, scheduler, notifier, migrate,
|
||||
seed, export, secret, all
|
||||
cmd/hermes/main.go Hermes MCP client gateway (standalone binary)
|
||||
internal/httpapi/ REST + MCP server. Chi router. OpenAPI-generated types from
|
||||
internal/httpapi/gen/api.gen.go. Strict server in impl.go.
|
||||
internal/mcp/ MCP tool implementations (get_entity, search_knowledge, etc.)
|
||||
internal/db/ Connection pool (pool.go), seed ingestion (seed.go), DB→YAML
|
||||
export (export.go), type hierarchy (typetree.go)
|
||||
internal/db/queries/ SQL query files → sqlc generates internal/db/sqlcgen/
|
||||
internal/scheduler/ Observe loop: probes, signals, check_defs
|
||||
internal/actuator/ SSH execution with circuit breaker + retry
|
||||
internal/learning/ Pattern extraction, anomaly detection
|
||||
internal/notifier/ Matrix notification + approval token generation
|
||||
internal/policy/ Risk classifier (read policy.yaml → classify action)
|
||||
internal/secrets/ Backend abstraction: Infisical (primary) + SOPS (fallback)
|
||||
internal/domain/ Core types: entities, approvals, executions, signals, patterns
|
||||
internal/ontology/ Type hierarchy validation, relationship checks
|
||||
internal/knowledge/ Knowledge YAML seed ingestion
|
||||
internal/config/ Config loading from env vars
|
||||
api/openapi.yaml REST API contract. Source of truth for endpoints.
|
||||
api/codegen.yaml oapi-codegen config → generates internal/httpapi/gen/
|
||||
migrations/ Forward-only SQL. Format: NNN_name.up.sql. No down migrations.
|
||||
seeds/ Bootstrap YAML. ontology.yaml, inventory.yaml, policy.yaml,
|
||||
knowledge.yaml. Regenerated from DB via oikos export.
|
||||
compose/ Dockerfiles. oikos/ (multi-stage), hermes/ (distroless).
|
||||
Caddy config at compose/caddy/Caddyfile.oikos.
|
||||
scripts/ Deploy, rollback, watchdog, verification, cutover checklist.
|
||||
hermes/ Hermes config.yaml, SOUL.md, skills.
|
||||
.agents/ Agent instruction files, domains, shared conventions, skills.
|
||||
plans/ Design documents. active/ + done/.
|
||||
docs/adr/ Architecture decision records. Numbered, prefix-sorted.
|
||||
```
|
||||
|
||||
## Development loop
|
||||
|
||||
```bash
|
||||
# Start dependencies
|
||||
make dev
|
||||
|
||||
# Generate code after API/SQL changes
|
||||
make generate
|
||||
|
||||
# Build
|
||||
make build
|
||||
|
||||
# Run tests
|
||||
make test # all unit tests
|
||||
make test-db # integration tests (needs compose Postgres)
|
||||
|
||||
# Lint
|
||||
make lint
|
||||
|
||||
# CI drift guard (run before commit)
|
||||
make generate-check
|
||||
```
|
||||
|
||||
## Adding a feature or phase
|
||||
|
||||
Oikos features follow a phase model (read [OIKOS.md](../OIKOS.md) for the
|
||||
current phase status). To add a new capability:
|
||||
|
||||
1. **ADR first.** Write an architecture decision record in `docs/adr/` with
|
||||
the next sequence number. Document the decision, context, alternatives
|
||||
considered, and consequences.
|
||||
2. **Plan.** If the change is non-trivial, create a plan in `plans/` following
|
||||
the template in [page-templates.md](../shared/page-templates.md).
|
||||
3. **Schema.** If the feature needs new DB tables, write a forward-only
|
||||
migration in `migrations/`. Use `IF NOT EXISTS` for idempotency.
|
||||
4. **API.** If the feature exposes endpoints, define them in
|
||||
`api/openapi.yaml` first, then run `make generate`, then implement.
|
||||
5. **Domain.** Add types to `internal/domain/` before adding logic.
|
||||
6. **Tests.** Write tests alongside implementation. Integration tests go in
|
||||
`*_test.go` in the relevant package, using the compose Postgres.
|
||||
7. **Policy.** If the feature introduces new mutation types, update
|
||||
`seeds/policy.yaml` and the classifier in `internal/policy/`.
|
||||
8. **Run `make generate-check`** before commit to ensure generated code is
|
||||
current.
|
||||
|
||||
## SQL conventions
|
||||
|
||||
- Queries live in `internal/db/queries/*.sql` with `-- name: FuncName :exec`
|
||||
annotations for sqlc
|
||||
- Use `pgx/v5` driver. UUIDs use `pgtype.UUID`, timestamps use `time.Time`
|
||||
- CTEs for graph traversals (blast radius, dependency chains)
|
||||
- CAGGs and retention policies for TimescaleDB hypertables
|
||||
- FTS via `tsvector` + `tsquery` for knowledge search (migration 011)
|
||||
|
||||
## OpenAPI codegen
|
||||
|
||||
- Config: `api/codegen.yaml`. Uses `oapi-codegen/v2` with Chi server template
|
||||
- Generated output: `internal/httpapi/gen/api.gen.go` — never hand-edit
|
||||
- Strict server interface: `api.gen.go` generates the `StrictServerInterface`;
|
||||
implement it in `internal/httpapi/impl.go`
|
||||
- Problem+JSON errors via `internal/httpapi/problem.go` — RFC 9457 format
|
||||
- Cursor pagination, If-Match/ETag, idempotency keys, SSE streaming
|
||||
|
||||
## Testing philosophy
|
||||
|
||||
- **Race detector always on.** `make test` runs `go test -race -cover ./...`
|
||||
- **Integration tests** use the compose Postgres. Run with `make test-db`.
|
||||
Each test creates + tears down its own schema namespace.
|
||||
- **Coverage gates** in CI: policy + learning ≥ 80%, others ≥ 60%
|
||||
- Tests use `testing.T` directly, no assertion library
|
||||
- Table-driven tests for validation and classification logic
|
||||
|
||||
## Migration rules
|
||||
|
||||
- Forward-only. No down migrations (ADR 0008)
|
||||
- Idempotent: use `IF NOT EXISTS`, `DO $$ BEGIN ... END $$` blocks
|
||||
- Sequence numbers are sequential integers (001, 002, ...)
|
||||
- Each migration file is `NNN_name.up.sql`
|
||||
- Migrations are embedded in the binary via `migrations/embed.go`
|
||||
|
||||
## Seed files
|
||||
|
||||
- `seeds/ontology.yaml` — entity types, relationship types, lifecycles
|
||||
(validated against schema in `internal/ontology/`)
|
||||
- `seeds/inventory.yaml` — hosts, services, entities (the topology)
|
||||
- `seeds/policy.yaml` — risk classes, approval rules, autonomy settings
|
||||
- `seeds/knowledge.yaml` — documents, investigations, runbooks (DB is source
|
||||
of truth; this file is the DR export)
|
||||
- After DB changes via the API, run `make export` to regenerate seeds
|
||||
|
||||
## Secrets handling
|
||||
|
||||
- No secrets in code, config, or commits
|
||||
- Dev secrets in `.env` (gitignored)
|
||||
- Primary: Infisical (`internal/secrets/infisical.go`)
|
||||
- Fallback: SOPS + age (`internal/secrets/sops.go`)
|
||||
- Backend interface: `internal/secrets/backend.go`
|
||||
- Machine identities via Infisical UniversalAuth
|
||||
- In-memory cache with TTL for performance
|
||||
|
||||
## Staging and deployment
|
||||
|
||||
- CI pipeline: `.gitea/workflows/ci.yml` — lint, vet, vulncheck, test, docker build
|
||||
- Deploy: `scripts/deploy.sh` — git pull → docker build → compose up → health check
|
||||
- Watchdog: `scripts/watchdog.sh` — 2-minute cron, Matrix alert on failure
|
||||
- Rollback: `scripts/rollback.sh` — checkout SHA + pg_restore
|
||||
- Cutover checklist: `scripts/cutover-checklist.md`
|
||||
|
||||
## Writing conventions
|
||||
|
||||
Apply [writing-style.md](../shared/writing-style.md) for all committed prose.
|
||||
Terse, reference-style, no marketing vocabulary. Code comments explain intent
|
||||
and trade-offs, not mechanics.
|
||||
|
||||
Apply [caveman.md](../shared/caveman.md) for agent communication. The caveman
|
||||
standard applies to agent *chat responses*, not committed documentation.
|
||||
|
||||
## Skills
|
||||
|
||||
Agent skills live under `.agents/skills/<name>/SKILL.md`. Each skill has a
|
||||
frontmatter description that tools match against tasks. To add a skill:
|
||||
|
||||
1. Create `.agents/skills/<name>/SKILL.md`
|
||||
2. Include frontmatter with description field
|
||||
3. Document the procedure following the runbook template
|
||||
4. Reference relevant files, commands, and policy classes
|
||||
|
||||
Skills that require code (e.g. linting) may include companion scripts in the
|
||||
same directory.
|
||||
|
||||
## When in doubt
|
||||
|
||||
- Query MCP tools first (search_knowledge, get_entity)
|
||||
- Read the relevant ADR in `docs/adr/`
|
||||
- Grep the codebase: `rg <symbol> internal/`
|
||||
- Check `plans/` for in-progress work that may conflict
|
||||
- Classify any new mutation against `seeds/policy.yaml` before suggesting it
|
||||
@@ -4,6 +4,9 @@ You are running on a machine that is part of the **hubris** homelab. The full
|
||||
context is in this checkout at `/opt/homelab-context/`. This file is the entry
|
||||
point. Read it once at start, then keep working.
|
||||
|
||||
- **New client?** Read [CLIENTS.md](CLIENTS.md) first.
|
||||
- **Developing on this repo?** Also read [.agents/dev/CONTRIBUTING.md](.agents/dev/CONTRIBUTING.md).
|
||||
|
||||
The operating model — OODA loop, risk classes, approval rules, the ontology,
|
||||
and node lifecycle — is defined in [OIKOS.md](.agents/OIKOS.md). Before any mutation,
|
||||
classify the action against `seeds/policy.yaml`; when the class requires
|
||||
|
||||
117
CLIENTS.md
Normal file
117
CLIENTS.md
Normal file
@@ -0,0 +1,117 @@
|
||||
# Oikos — client guide
|
||||
|
||||
If you are a homelab machine, this is what Oikos is and what it gives you.
|
||||
If you are an AI agent running on this machine, also read [AGENTS.md](AGENTS.md).
|
||||
|
||||
## What is Oikos?
|
||||
|
||||
Oikos is the agentic operating system for the **hubris** Proxmox homelab. It
|
||||
observes state, classifies actions against policy, executes approved
|
||||
procedures, learns from outcomes, and notifies the operator. It runs as a
|
||||
Docker stack on mac-mini and exposes an MCP server + REST API.
|
||||
|
||||
## What Oikos provides
|
||||
|
||||
| Capability | How you access it |
|
||||
|------------|-------------------|
|
||||
| Entity query (topology, blast radius) | MCP `get_entity`, `get_blast_radius` |
|
||||
| Full-text knowledge search | MCP `search_knowledge` |
|
||||
| Service status + logs | MCP `get_service_status`, `tail_log` |
|
||||
| LXC inventory + state | MCP `list_lxcs`, `get_lxc_state` |
|
||||
| Context cards | MCP `explain` |
|
||||
| Pre-flight risk classification | MCP `preflight` |
|
||||
| Change history | MCP `get_change_history` |
|
||||
| State snapshot (health, disk, drift) | MCP `get_state_snapshot` |
|
||||
| Secrets (Infisical) | REST API + `oikos secret` CLI |
|
||||
| Approval tokens | Matrix via notifier |
|
||||
|
||||
All MCP tools are read-only. Mutations use the `homelab` CLI with operator
|
||||
approval.
|
||||
|
||||
## Enrollment
|
||||
|
||||
Enrolled clients have a checkout at `/opt/homelab-context/`. If this
|
||||
directory does not exist, the client is not enrolled.
|
||||
|
||||
To enroll:
|
||||
```bash
|
||||
# Run from an existing enrolled client
|
||||
homelab client add <hostname>
|
||||
```
|
||||
|
||||
This runs `bootstrap.sh` on the target, which:
|
||||
1. Clones the repo to `/opt/homelab-context/`
|
||||
2. Configures the auto-sync timer
|
||||
3. Provisions agent persona from `hermes/SOUL.md` (on Hermes agents)
|
||||
4. Installs Caveman tooling for terse communication
|
||||
|
||||
## After enrollment
|
||||
|
||||
### What changes on your machine
|
||||
|
||||
- `/opt/homelab-context/` — the repo checkout, your source of truth
|
||||
- `/opt/homelab-context/inventory.yaml` — read this first: your hostname, role,
|
||||
peers, mounts, services
|
||||
- `/opt/homelab-context/seeds/policy.yaml` — rules for what actions you can
|
||||
take autonomously vs. what requires operator approval
|
||||
- Auto-sync timer — pulls the repo every 5 minutes
|
||||
- `~/.hermes/SOUL.md` — agent persona (on Hermes agents)
|
||||
- `~/bin/caveman_wrapper.sh` — terse communication tooling
|
||||
|
||||
### Your identity
|
||||
|
||||
Your identity in the homelab is defined in `inventory.yaml`. Run `hostname`
|
||||
(Linux) or `scutil --get LocalHostName` (macOS), then look up your entry.
|
||||
It tells you your role, what services you host, what's mounted, and your
|
||||
mesh address.
|
||||
|
||||
### Source of truth hierarchy
|
||||
|
||||
1. **Postgres database** (runtime) — authoritative for entities, knowledge,
|
||||
signals, ledger. Query via MCP or REST API.
|
||||
2. **Repo at `/opt/homelab-context/`** — bootstrap seeds + documentation.
|
||||
On disk, available offline.
|
||||
3. **Never guess.** If data is missing, query MCP. If MCP is down, grep the
|
||||
clone.
|
||||
|
||||
## The sync timer
|
||||
|
||||
Every 5 minutes, `systemd` (Linux) or `launchd` (macOS) runs:
|
||||
|
||||
1. `git pull` via `tools/post-pull.sh`
|
||||
2. Any `tools/*.setup.sh` scripts that need to run
|
||||
|
||||
This keeps your checkout current and applies tooling updates automatically.
|
||||
To trigger sync manually: `sudo homelab sync`.
|
||||
|
||||
## Making changes
|
||||
|
||||
- **Read state**: use MCP tools or the repo checkout
|
||||
- **Mutate state** (restart, edit config, deploy): classify the action against
|
||||
`seeds/policy.yaml`:
|
||||
- `read_only` / `reversible_low` — execute directly
|
||||
- `config_mutation` / `destructive` — request operator approval via the
|
||||
`homelab` CLI
|
||||
- **Secrets**: use Infisical (primary) or SOPS (fallback). Never hardcode.
|
||||
- **Knowledge**: if you observe a discrepancy between docs and live state,
|
||||
update the DB via the API in the same session. Run `oikos export` to
|
||||
regenerate seeds.
|
||||
|
||||
## MCP endpoint
|
||||
|
||||
```
|
||||
https://mcp.hubris.network/mcp
|
||||
```
|
||||
|
||||
Available tools are listed in [AGENTS.md](AGENTS.md#3-the-mcp-server).
|
||||
|
||||
## Communication mode
|
||||
|
||||
Apply [.agents/shared/caveman.md](.agents/shared/caveman.md) — terse,
|
||||
fragment-heavy communication. Drop filler, keep substance.
|
||||
|
||||
## Related
|
||||
|
||||
- [AGENTS.md](AGENTS.md) — full agent orientation (read this first)
|
||||
- [OIKOS.md](.agents/OIKOS.md) — operating model, OODA loop, risk classes
|
||||
- [CONTRIBUTING.md](CONTRIBUTING.md) — developer guide for the Oikos codebase
|
||||
155
CONTRIBUTING.md
Normal file
155
CONTRIBUTING.md
Normal file
@@ -0,0 +1,155 @@
|
||||
# Contributing to Oikos
|
||||
|
||||
Developer guide for the Oikos codebase. If you are a homelab client consuming
|
||||
Oikos, see [CLIENTS.md](CLIENTS.md). If you are an AI agent working on the
|
||||
repo, see [.agents/dev/CONTRIBUTING.md](.agents/dev/CONTRIBUTING.md).
|
||||
|
||||
## Dev setup
|
||||
|
||||
- **Go 1.26+** (see `go.mod` for pinned version)
|
||||
- **PostgreSQL with TimescaleDB** — the compose stack includes `timescale/timescaledb:2.17.2-pg16`
|
||||
- **Docker** for the full dev stack
|
||||
|
||||
```bash
|
||||
# Start dependencies (Postgres + Redis)
|
||||
docker compose --profile dev up -d
|
||||
|
||||
# Run all tests
|
||||
make test
|
||||
|
||||
# Run integration tests (needs compose Postgres)
|
||||
make test-db
|
||||
|
||||
# Build the binary
|
||||
make build
|
||||
```
|
||||
|
||||
## Project structure
|
||||
|
||||
```
|
||||
cmd/oikos/ Single-binary entry point
|
||||
cmd/hermes/ Hermes MCP client gateway
|
||||
internal/ All Go packages
|
||||
httpapi/ REST + MCP server (OpenAPI-generated)
|
||||
mcp/ MCP tool implementations
|
||||
db/ Connection pool, migrations, seeds, sqlc queries
|
||||
scheduler/ Observe loop, probes, signals
|
||||
actuator/ SSH execution
|
||||
learning/ Pattern recognition, anomaly detection
|
||||
notifier/ Matrix notifications, approval tokens
|
||||
policy/ Risk classifier
|
||||
secrets/ Infisical + SOPS backend
|
||||
domain/ Core types: entities, approvals, signals, patterns
|
||||
ontology/ Type hierarchy, relationship validation
|
||||
knowledge/ Knowledge YAML seed ingestion
|
||||
api/openapi.yaml API contract — the source of truth for endpoints
|
||||
migrations/ Forward-only SQL migrations (TimescaleDB)
|
||||
seeds/ Bootstrap YAML: ontology, inventory, policy, knowledge
|
||||
compose/ Dockerfiles + Caddy config
|
||||
scripts/ Deploy, watchdog, rollback
|
||||
hermes/ Hermes config, persona, skills
|
||||
.agents/ Agent instruction files + skills
|
||||
plans/ Design documents
|
||||
docs/adr/ Architecture decision records
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `make build` | Build `oikos` binary |
|
||||
| `make test` | Run all tests with race detection |
|
||||
| `make test-db` | Run integration tests against compose Postgres |
|
||||
| `make lint` | `go vet` + `golangci-lint` |
|
||||
| `make generate` | Regenerate OpenAPI + sqlc code |
|
||||
| `make generate-check` | CI drift guard — fail if generated code is stale |
|
||||
| `make migrate` | Apply DB migrations |
|
||||
| `make seed` | Ingest seeds into DB |
|
||||
| `make export` | Export DB state to YAML seeds |
|
||||
| `make dev` | Start compose dev stack |
|
||||
| `make clean` | Remove binary + test cache |
|
||||
|
||||
## Conventions
|
||||
|
||||
### APIs are OpenAPI-first
|
||||
|
||||
The REST API is defined in `api/openapi.yaml`. Server code is generated with
|
||||
`oapi-codegen` into `internal/httpapi/gen/`. To add an endpoint:
|
||||
|
||||
1. Add the path + schema to `api/openapi.yaml`
|
||||
2. Run `make generate`
|
||||
3. Implement the handler in `internal/httpapi/impl.go`
|
||||
4. Add tests in `internal/httpapi/api_test.go`
|
||||
|
||||
Never hand-edit `internal/httpapi/gen/api.gen.go`.
|
||||
|
||||
### Database access is sqlc-first
|
||||
|
||||
SQL queries live in `internal/db/queries/*.sql`. Go code is generated with
|
||||
`sqlc` into `internal/db/sqlcgen/`. Config in `sqlc.yaml`.
|
||||
|
||||
- Queries target pgx/v5 with UUID + timestamptz overrides
|
||||
- Never hand-edit generated sqlc code
|
||||
|
||||
### Migrations are forward-only
|
||||
|
||||
SQL migrations live in `migrations/` as `NNN_name.up.sql`. There are no down
|
||||
migrations (see [ADR 0008](docs/adr/0008-forward-only-migrations.md)).
|
||||
Migrations are idempotent where possible (`IF NOT EXISTS`, `DO $$` blocks).
|
||||
|
||||
To add a migration:
|
||||
|
||||
1. Create `migrations/NNN_name.up.sql` with the next sequence number
|
||||
2. Write the DDL
|
||||
3. Run `make migrate` to apply
|
||||
|
||||
### Seeds are DB-generated
|
||||
|
||||
`seeds/*.yaml` are the bootstrap files used by `oikos seed`. After making
|
||||
changes via the API, run `make export` to regenerate the seed files. These
|
||||
files are version-controlled and serve as DR fallback.
|
||||
|
||||
### Writing style
|
||||
|
||||
Follow [.agents/shared/writing-style.md](.agents/shared/writing-style.md).
|
||||
Documentation is reference prose, not marketing. Banned vocabulary includes
|
||||
"robust", "seamless", "leverage", "utilize", "delve", "cutting-edge".
|
||||
|
||||
### Risk classification
|
||||
|
||||
Every mutation is classified against `seeds/policy.yaml` before execution.
|
||||
Four risk classes: `read_only`, `reversible_low`, `config_mutation`,
|
||||
`destructive`. The classifier can only lower autonomy relative to policy,
|
||||
never raise it. When in doubt, escalate.
|
||||
|
||||
## CI
|
||||
|
||||
Gitea Actions runs on push to `main` and pull requests (`ci.yml`):
|
||||
|
||||
1. `go vet` + `golangci-lint` + `govulncheck`
|
||||
2. Generated code drift check (`make generate-check`)
|
||||
3. Build (`go build ./...`)
|
||||
4. Test with race detector + coverage
|
||||
5. Docker build verification (no push)
|
||||
|
||||
Coverage gates: policy + learning packages ≥ 80%, others ≥ 60%.
|
||||
|
||||
## PR workflow
|
||||
|
||||
1. Create a branch from `main`
|
||||
2. Make changes, write tests
|
||||
3. Run `make lint test generate-check`
|
||||
4. Commit with a message following: problem → change → risk → verification
|
||||
5. Push to Gitea; CI gates PRs on green
|
||||
|
||||
## Secrets
|
||||
|
||||
Secrets are managed by Infisical (primary) with SOPS as DR fallback. Never
|
||||
hardcode secrets. Use environment variables from `.env` for local dev.
|
||||
The `.env` and `.infisical-credentials` files are gitignored.
|
||||
|
||||
## Related
|
||||
|
||||
- [OIKOS.md](.agents/OIKOS.md) — operating model, OODA loop, ontology
|
||||
- [CLIENTS.md](CLIENTS.md) — for homelab clients consuming Oikos
|
||||
- [docs/adr/](docs/adr/) — architecture decision records
|
||||
23
README.md
23
README.md
@@ -7,6 +7,8 @@ state, classifies actions against policy, executes approved procedures over SSH,
|
||||
learns from outcomes, and escalates when uncertain.
|
||||
|
||||
**For agents running on enrolled clients:** start with [AGENTS.md](AGENTS.md).
|
||||
**For client machines:** see [CLIENTS.md](CLIENTS.md).
|
||||
**For developers:** see [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
|
||||
## Quick start
|
||||
|
||||
@@ -101,29 +103,34 @@ oikos secret migrate # SOPS → Infisical
|
||||
cmd/oikos/ Go entry point — single binary
|
||||
cmd/hermes/ Hermes MCP client gateway
|
||||
internal/ Go packages (httpapi, mcp, scheduler, actuator, learning,
|
||||
notifier, policy, secrets, db, config, ontology, domain)
|
||||
notifier, policy, secrets, db, config, ontology, domain,
|
||||
knowledge)
|
||||
api/openapi.yaml API contract (OpenAPI 3.1)
|
||||
migrations/ Forward-only SQL migrations (TimescaleDB)
|
||||
seeds/ Bootstrap YAML (ontology, inventory, policy)
|
||||
seeds/ Bootstrap YAML (ontology, inventory, policy, knowledge)
|
||||
compose/ Dockerfiles + Caddy config
|
||||
scripts/ Deploy, watchdog, verification, rollback
|
||||
hermes/ Hermes config, persona, skills
|
||||
archive/knowledge/ Narrative documentation (containers, hosts, infrastructure)
|
||||
.agents/ Agent instruction files + skills
|
||||
plans/ Design documents
|
||||
.agents/ Agent instruction files, shared conventions, skills
|
||||
archive/ Historical reference (legacy wiki, plans, SOPS backups)
|
||||
plans/ Design documents (active + done)
|
||||
docs/adr/ Architecture decision records
|
||||
```
|
||||
|
||||
## For agents
|
||||
|
||||
See [AGENTS.md](AGENTS.md) for the full orientation. Quick reference:
|
||||
|
||||
- **Source of truth:** DB (runtime) then repo (bootstrap + docs)
|
||||
- **Mutations:** classify against policy, request approval for `destructive`/`config_mutation`
|
||||
- **Wiki:** files under `archive/knowledge/`, changelog at bottom of each page
|
||||
- **Source of truth:** DB (runtime) then seeds (bootstrap). Old wiki is
|
||||
archived at `archive/knowledge/` — use MCP `search_knowledge` instead.
|
||||
- **Mutations:** classify against policy, request approval for
|
||||
`destructive`/`config_mutation`
|
||||
- **Secrets:** Infisical (primary) or SOPS (fallback) — never hardcode
|
||||
|
||||
## Related
|
||||
|
||||
- [OIKOS.md](.agents/OIKOS.md) — operating model, OODA loop, ontology
|
||||
- [CLIENTS.md](CLIENTS.md) — client onboarding guide
|
||||
- [CONTRIBUTING.md](CONTRIBUTING.md) — developer guide
|
||||
- [plans/](plans/) — design documents and cutover checklist
|
||||
- [docs/adr/](docs/adr/) — architecture decision records
|
||||
|
||||
@@ -1,392 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""One-shot: convert knowledge/wiki/ to seeds/knowledge.yaml."""
|
||||
|
||||
import os, re, yaml
|
||||
from pathlib import Path
|
||||
from hashlib import sha256
|
||||
|
||||
REPO = Path("/Users/dtoro/Projects/oikos")
|
||||
WIKI = REPO / "archive" / "knowledge"
|
||||
SOURCES = REPO / "archive" / "knowledge"
|
||||
GLOSSARY = REPO / "archive" / "knowledge" / "GLOSSARY.md"
|
||||
|
||||
# Maps wiki path components to entity slugs
|
||||
# Format: (path_pattern, entity_slug)
|
||||
PATH_TO_ENTITY = {
|
||||
# Containers
|
||||
"containers/101-jellyfin": "lxc:jellyfin",
|
||||
"containers/102-nfs-export": "lxc:nfs-export",
|
||||
"containers/103-paperless": "lxc:paperless",
|
||||
"containers/104-gitea": "lxc:gitea",
|
||||
"containers/105-apps": "lxc:apps",
|
||||
"containers/106-auth-outpost": "lxc:auth-outpost",
|
||||
"containers/107-dns": "lxc:dns",
|
||||
"containers/114-nextcloud": "lxc:nextcloud",
|
||||
"containers/118-elementsynapse": "lxc:elementsynapse",
|
||||
"containers/119-sophia": "lxc:sophia",
|
||||
"containers/120-mule-images": "lxc:mule-images",
|
||||
"containers/121-caddy": "lxc:caddy",
|
||||
"containers/122-arriman": "lxc:arriman",
|
||||
"containers/128-trmnl": "lxc:trmnl",
|
||||
"containers/129-house": "lxc:house",
|
||||
"containers/130-grimmory": "lxc:grimmory",
|
||||
"containers/131-teddycloud": "lxc:teddycloud",
|
||||
"containers/132-rclone": "lxc:rclone",
|
||||
"containers/133-seanime": "lxc:seanime",
|
||||
"containers/134-romm": "lxc:romm",
|
||||
# Hosts
|
||||
"hosts/hubris": "host:hubris",
|
||||
"hosts/strong": "host:strong",
|
||||
# VMs
|
||||
"vms/100-zimaos": "vm:zimaos",
|
||||
"vms/108-haos": "vm:haos",
|
||||
# Infrastructure → services
|
||||
"infrastructure/auto-deploy": None,
|
||||
"infrastructure/backups": None,
|
||||
"infrastructure/dns": "service:dns",
|
||||
"infrastructure/homelab-context": "service:homelab-mcp",
|
||||
"infrastructure/ingress": "service:caddy",
|
||||
"infrastructure/media-permissions": "service:jellyfin",
|
||||
"infrastructure/mesh": None,
|
||||
"infrastructure/monitoring": None,
|
||||
"infrastructure/network": None,
|
||||
"infrastructure/ssh-access": None,
|
||||
"infrastructure/topology": None,
|
||||
"infrastructure/vps-hardening": "host:netbird-vps",
|
||||
}
|
||||
|
||||
def parse_page(path):
|
||||
"""Parse a wiki page into structured sections."""
|
||||
if not path.exists():
|
||||
return None
|
||||
text = path.read_text()
|
||||
lines = text.split('\n')
|
||||
|
||||
# Title is first H1
|
||||
title = ""
|
||||
for line in lines:
|
||||
if line.startswith('# ') and not line.startswith('## '):
|
||||
title = line[2:].strip()
|
||||
break
|
||||
|
||||
# Find sections by H2 headings
|
||||
sections = {}
|
||||
current_heading = "_preamble"
|
||||
current_content = []
|
||||
|
||||
for line in lines:
|
||||
if line.startswith('## ') and not line.startswith('### '):
|
||||
if current_content:
|
||||
sections[current_heading] = '\n'.join(current_content).strip()
|
||||
current_heading = line[3:].strip().lower()
|
||||
current_content = []
|
||||
else:
|
||||
current_content.append(line)
|
||||
if current_content:
|
||||
sections[current_heading] = '\n'.join(current_content).strip()
|
||||
|
||||
# Parse at-a-glance
|
||||
at_glance = {}
|
||||
ag_text = sections.get('at a glance', '')
|
||||
for line in ag_text.split('\n'):
|
||||
line = line.strip()
|
||||
# Strip leading bullet
|
||||
line = re.sub(r'^[-*]\s+', '', line)
|
||||
# Match **Key:** value or **Key Word:** value
|
||||
m = re.match(r'\*\*([^*]+?):?\*\*\s+(.+)', line)
|
||||
if not m:
|
||||
m = re.match(r'([A-Z][a-z]+(?:\s+[A-Z][a-z]+)*):\s+(.+)', line)
|
||||
if m:
|
||||
key = m.group(1).lower().strip().replace(' ', '_').replace('/', '_')
|
||||
val = m.group(2).strip()
|
||||
# Strip trailing parenthetical notes
|
||||
val = re.sub(r'\s*\([^)]*\)$', '', val)
|
||||
# Strip markdown formatting from value
|
||||
val = re.sub(r'\*\*([^*]+)\*\*', r'\1', val)
|
||||
val = re.sub(r'`([^`]+)`', r'\1', val)
|
||||
# Simplify link text
|
||||
val = re.sub(r'\[([^\]]+)\]\([^)]+\)', r'\1', val)
|
||||
val = re.sub(r'↔', '', val).strip()
|
||||
# Normalize keys
|
||||
key_map = {
|
||||
'cores': 'cores', 'core': 'cores',
|
||||
'ram': 'ram', 'memory': 'ram',
|
||||
'mounts': 'mounts', 'mount': 'mounts',
|
||||
'host': 'host', 'ip': 'ip',
|
||||
'public_host': 'public_host', 'public_hostname': 'public_host',
|
||||
'lan_ip': 'lan_ip',
|
||||
'os': 'os', 'kind': 'kind',
|
||||
'runtime': 'runtime', 'role': 'role',
|
||||
'pve_id': 'pve_id', 'privilege': 'privileged',
|
||||
'resources': 'resources', 'gpu': 'gpu',
|
||||
'swap': 'swap', 'rootfs': 'rootfs',
|
||||
'version': 'version', 'hardware': 'hardware',
|
||||
}
|
||||
key = key_map.get(key, key)
|
||||
at_glance[key] = val
|
||||
|
||||
# Parse changelog
|
||||
changelog = []
|
||||
cl_text = sections.get('changelog', '')
|
||||
current_entry = None
|
||||
for line in cl_text.split('\n'):
|
||||
m = re.match(r'###\s+(\d{4}-\d{2}-\d{2})\s+[—–-]\s+(.+)', line)
|
||||
if m:
|
||||
if current_entry:
|
||||
changelog.append(current_entry)
|
||||
current_entry = {'date': m.group(1), 'title': m.group(2).strip(), 'body': ''}
|
||||
elif current_entry is not None:
|
||||
stripped = line.strip()
|
||||
if stripped and not stripped.startswith('#'):
|
||||
if current_entry['body']:
|
||||
current_entry['body'] += ' '
|
||||
current_entry['body'] += stripped
|
||||
if current_entry:
|
||||
changelog.append(current_entry)
|
||||
|
||||
# Tags from path
|
||||
parts = path.relative_to(REPO).parts
|
||||
tags = []
|
||||
if 'containers' in parts:
|
||||
tags.append('container')
|
||||
elif 'hosts' in parts:
|
||||
tags.append('host')
|
||||
elif 'vms' in parts:
|
||||
tags.append('vm')
|
||||
elif 'infrastructure' in parts:
|
||||
tags.append('infrastructure')
|
||||
|
||||
# Determine slug from relative path
|
||||
rel = str(path.relative_to(WIKI))
|
||||
slug = rel.replace('.md', '')
|
||||
|
||||
# Entity mapping
|
||||
entity_slug = PATH_TO_ENTITY.get(slug, None)
|
||||
|
||||
return {
|
||||
'slug': slug,
|
||||
'title': title,
|
||||
'content': text,
|
||||
'entity_slug': entity_slug,
|
||||
'tags': tags,
|
||||
'at_glance': at_glance,
|
||||
'changelog': changelog,
|
||||
'is_investigation': 'investigations' in rel,
|
||||
}
|
||||
|
||||
|
||||
def parse_investigation(path):
|
||||
"""Parse an investigation page."""
|
||||
if not path.exists():
|
||||
return None
|
||||
text = path.read_text()
|
||||
lines = text.split('\n')
|
||||
|
||||
title = ""
|
||||
for line in lines:
|
||||
if line.startswith('# '):
|
||||
title = line[2:].strip()
|
||||
break
|
||||
|
||||
# Extract date from title or filename
|
||||
date = ""
|
||||
status = "resolved"
|
||||
duration = ""
|
||||
|
||||
for line in lines[:30]:
|
||||
m = re.search(r'(\d{4}-\d{2}-\d{2})', line)
|
||||
if m:
|
||||
date = m.group(1)
|
||||
break
|
||||
|
||||
for line in lines:
|
||||
if '**Status:**' in line:
|
||||
status = line.split('**Status:**')[-1].strip().lower()
|
||||
if '**Duration:**' in line:
|
||||
duration = line.split('**Duration:**')[-1].strip()
|
||||
|
||||
# Extract entity references for about_slugs
|
||||
about_slugs = []
|
||||
entity_patterns = [
|
||||
(r'\bcaddy\b', 'service:caddy'),
|
||||
(r'\bauthentik\b', 'service:authentik'),
|
||||
(r'\bdns\b', 'service:dns'),
|
||||
(r'\bgitea\b', 'service:gitea'),
|
||||
(r'\bjellyfin\b', 'service:jellyfin'),
|
||||
(r'\bmatrix\b', 'service:matrix'),
|
||||
(r'\bpaperless\b', 'service:paperless'),
|
||||
(r'\bnextcloud\b', 'service:nextcloud'),
|
||||
(r'\bartifacto\b', 'service:artifacto'),
|
||||
(r'\barriman\b', 'lxc:arriman'),
|
||||
(r'\btrmnl\b', 'service:trmnl'),
|
||||
(r'\bmac-mini\b', 'ws:mac-mini'),
|
||||
(r'\bhubris\b', 'host:hubris'),
|
||||
(r'\bstrong\b', 'host:strong'),
|
||||
]
|
||||
for pattern, slug in entity_patterns:
|
||||
if re.search(pattern, text, re.IGNORECASE):
|
||||
about_slugs.append(slug)
|
||||
|
||||
rel = str(path.relative_to(WIKI))
|
||||
slug = rel.replace('.md', '')
|
||||
|
||||
return {
|
||||
'slug': slug,
|
||||
'title': title,
|
||||
'date': date,
|
||||
'status': status,
|
||||
'duration': duration,
|
||||
'content': text,
|
||||
'about_slugs': about_slugs,
|
||||
'tags': ['investigation'],
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
documents = []
|
||||
investigations = []
|
||||
runbooks = []
|
||||
|
||||
# Container pages
|
||||
containers_dir = WIKI / "containers"
|
||||
for f in sorted(containers_dir.glob("*.md")):
|
||||
if 'index' in f.name:
|
||||
continue
|
||||
if f.parent.name == 'archive':
|
||||
continue
|
||||
result = parse_page(f)
|
||||
if result and result['title']:
|
||||
documents.append(result)
|
||||
print(f" document: {result['slug']} → {result['entity_slug']}")
|
||||
|
||||
# Host pages
|
||||
hosts_dir = WIKI / "hosts"
|
||||
for f in sorted(hosts_dir.glob("*.md")):
|
||||
if 'index' in f.name:
|
||||
continue
|
||||
result = parse_page(f)
|
||||
if result and result['title']:
|
||||
documents.append(result)
|
||||
print(f" document: {result['slug']} → {result['entity_slug']}")
|
||||
|
||||
# VM pages
|
||||
vms_dir = WIKI / "vms"
|
||||
for f in sorted(vms_dir.glob("*.md")):
|
||||
if 'index' in f.name:
|
||||
continue
|
||||
result = parse_page(f)
|
||||
if result and result['title']:
|
||||
documents.append(result)
|
||||
print(f" document: {result['slug']} → {result['entity_slug']}")
|
||||
|
||||
# Infrastructure pages
|
||||
infra_dir = WIKI / "infrastructure"
|
||||
for f in sorted(infra_dir.glob("*.md")):
|
||||
if 'index' in f.name:
|
||||
continue
|
||||
result = parse_page(f)
|
||||
if result and result['title']:
|
||||
documents.append(result)
|
||||
print(f" document: {result['slug']} → {result['entity_slug']}")
|
||||
|
||||
# Investigation pages
|
||||
inv_dir = SOURCES / "investigations"
|
||||
for f in sorted(inv_dir.glob("*.md")):
|
||||
if 'index' in f.name:
|
||||
continue
|
||||
result = parse_investigation(f)
|
||||
if result and result['title']:
|
||||
investigations.append(result)
|
||||
print(f" investigation: {result['slug']} → {result['about_slugs']}")
|
||||
|
||||
# Archive investigations too
|
||||
inv_archive = inv_dir / "archive"
|
||||
if inv_archive.exists():
|
||||
for f in sorted(inv_archive.glob("*.md")):
|
||||
result = parse_investigation(f)
|
||||
if result and result['title']:
|
||||
investigations.append(result)
|
||||
print(f" investigation: {result['slug']} → {result['about_slugs']}")
|
||||
|
||||
# Runbooks from .agents/skills/
|
||||
skills_dir = REPO / ".agents" / "skills"
|
||||
for skill_dir in sorted(skills_dir.iterdir()):
|
||||
if not skill_dir.is_dir():
|
||||
continue
|
||||
skill_file = skill_dir / "SKILL.md"
|
||||
if not skill_file.exists():
|
||||
continue
|
||||
text = skill_file.read_text()
|
||||
lines = text.split('\n')
|
||||
title = ""
|
||||
for line in lines:
|
||||
if line.startswith('# '):
|
||||
title = line[2:].strip()
|
||||
break
|
||||
|
||||
# Extract risk_class and entity_type from frontmatter
|
||||
risk_class = "read_only"
|
||||
entity_type = "service"
|
||||
for line in lines[:30]:
|
||||
m = re.match(r'\*\*risk_class:\*\*\s*(\w+)', line, re.IGNORECASE)
|
||||
if m:
|
||||
risk_class = m.group(1)
|
||||
m = re.match(r'\*\*applies_to:\*\*\s*(\w[\w-]*)', line, re.IGNORECASE)
|
||||
if m:
|
||||
entity_type = m.group(1)
|
||||
|
||||
name = skill_dir.name
|
||||
runbooks.append({
|
||||
'slug': name,
|
||||
'name': title or name,
|
||||
'risk_class': risk_class,
|
||||
'entity_type': entity_type,
|
||||
'procedure': {}, # SKILL.md is narrative, not structured yet
|
||||
'content': text,
|
||||
'tags': ['skill', 'runbook'],
|
||||
})
|
||||
print(f" runbook: {name}")
|
||||
|
||||
# Build seed YAML
|
||||
seed = {
|
||||
'version': 1,
|
||||
'documents': [{
|
||||
'slug': d['slug'],
|
||||
'title': d['title'],
|
||||
'content': d['content'],
|
||||
'entity_slug': d['entity_slug'],
|
||||
'tags': d['tags'],
|
||||
'at_glance': d['at_glance'],
|
||||
'changelog': d['changelog'],
|
||||
} for d in documents],
|
||||
'investigations': [{
|
||||
'slug': i['slug'],
|
||||
'title': i['title'],
|
||||
'date': i['date'],
|
||||
'status': i['status'],
|
||||
'duration': i['duration'],
|
||||
'content': i['content'],
|
||||
'about_slugs': i['about_slugs'],
|
||||
'tags': i['tags'],
|
||||
} for i in investigations],
|
||||
'runbooks': [{
|
||||
'slug': r['slug'],
|
||||
'name': r['name'],
|
||||
'risk_class': r['risk_class'],
|
||||
'entity_type': r['entity_type'],
|
||||
'procedure': r['procedure'],
|
||||
'content': r['content'],
|
||||
'tags': r['tags'],
|
||||
} for r in runbooks],
|
||||
}
|
||||
|
||||
out_path = REPO / "seeds" / "knowledge.yaml"
|
||||
out_path.write_text(yaml.dump(seed, allow_unicode=True, width=120, sort_keys=False))
|
||||
print(f"\nWrote {out_path}")
|
||||
print(f" {len(documents)} documents")
|
||||
print(f" {len(investigations)} investigations")
|
||||
print(f" {len(runbooks)} runbooks")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,162 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate hosts/<name>.yaml from inventory.yaml.
|
||||
|
||||
Run from the repo root:
|
||||
python3 mcp/build_host_files.py # writes files, exits non-zero on diff
|
||||
python3 mcp/build_host_files.py --check # exits non-zero if any output differs
|
||||
|
||||
Designed to be wired into a pre-commit hook or Gitea Action so generated
|
||||
hosts/*.yaml never drift from inventory.yaml.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import difflib
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError: # pragma: no cover
|
||||
print("PyYAML is required: pip install pyyaml", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
INVENTORY = REPO / "inventory.yaml"
|
||||
HOSTS_DIR = REPO / "hosts"
|
||||
|
||||
GENERATED_BANNER = (
|
||||
"# Generated by mcp/build_host_files.py from inventory.yaml.\n"
|
||||
"# Do NOT edit by hand — your changes will be overwritten.\n"
|
||||
"# Source of truth: ../inventory.yaml\n"
|
||||
)
|
||||
|
||||
|
||||
def narrative_page(name: str, kind: str, pve_id: int | None) -> str | None:
|
||||
"""Best-guess path to the human-authored narrative page for this host."""
|
||||
if kind == "proxmox-host":
|
||||
candidate = REPO / "hosts" / f"{name}.md"
|
||||
elif kind == "lxc":
|
||||
candidate = REPO / "containers" / f"{pve_id}-{name}.md"
|
||||
elif kind == "vm":
|
||||
candidate = REPO / "vms" / f"{pve_id}-{name}.md"
|
||||
else:
|
||||
return None
|
||||
if candidate.exists():
|
||||
return str(candidate.relative_to(REPO))
|
||||
return None
|
||||
|
||||
|
||||
def build_one(name: str, entry: dict, inventory: dict) -> dict:
|
||||
"""Project the entry for a single host into a per-host yaml record."""
|
||||
services = inventory.get("services", {})
|
||||
mesh = inventory.get("mesh", {})
|
||||
pve_id = entry.get("pve_id")
|
||||
|
||||
# Services this host runs: scan inventory.services for matching backend.
|
||||
runs_services = sorted(
|
||||
svc for svc, sentry in services.items()
|
||||
if isinstance(sentry, dict) and sentry.get("backend") == name
|
||||
)
|
||||
|
||||
record = {
|
||||
"name": name,
|
||||
"kind": entry.get("kind"),
|
||||
"os": entry.get("os"),
|
||||
"role": entry.get("role"),
|
||||
# Oikos lifecycle (oikos/ontology.yaml); absent in inventory = active
|
||||
"state": entry.get("state", "active"),
|
||||
"host": entry.get("host"),
|
||||
"pve_id": pve_id,
|
||||
"storage": entry.get("storage"),
|
||||
"depends_on": entry.get("depends_on", []),
|
||||
"lan_ip": entry.get("lan_ip"),
|
||||
"mesh": entry.get("mesh", {}),
|
||||
"mesh_globals": {
|
||||
"primary": mesh.get("primary"),
|
||||
"accepted": mesh.get("accepted"),
|
||||
},
|
||||
"peers": entry.get("peers", []),
|
||||
"mounts": entry.get("mounts", []),
|
||||
"public_host": entry.get("public_host"),
|
||||
"public_hosts": entry.get("public_hosts", []),
|
||||
"ssh": entry.get("ssh", {}),
|
||||
"runs": entry.get("runs", []) + runs_services,
|
||||
"services_hosted": [
|
||||
{"name": svc, **services[svc]} for svc in runs_services
|
||||
],
|
||||
"notes": entry.get("notes", []),
|
||||
"age_pubkey": entry.get("age_pubkey", ""),
|
||||
"see_also": [
|
||||
page for page in [narrative_page(name, entry.get("kind", ""), pve_id)]
|
||||
if page
|
||||
],
|
||||
"mcp_endpoint": services.get("homelab_mcp", {}).get("endpoint"),
|
||||
"secrets_issuance_endpoint": (
|
||||
services.get("secrets_issuance", {}).get("endpoint")
|
||||
),
|
||||
}
|
||||
# Strip None and empty containers so the file stays readable.
|
||||
return {k: v for k, v in record.items() if v not in (None, {}, [], "")}
|
||||
|
||||
|
||||
def serialize(record: dict) -> str:
|
||||
return GENERATED_BANNER + yaml.safe_dump(
|
||||
record, sort_keys=False, default_flow_style=False, width=100
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--check", action="store_true",
|
||||
help="exit 1 if any output would change (don't write)")
|
||||
args = parser.parse_args()
|
||||
|
||||
inventory = yaml.safe_load(INVENTORY.read_text())
|
||||
hosts = inventory.get("hosts", {})
|
||||
|
||||
HOSTS_DIR.mkdir(exist_ok=True)
|
||||
desired: dict[Path, str] = {}
|
||||
for name, entry in hosts.items():
|
||||
desired[HOSTS_DIR / f"{name}.yaml"] = serialize(build_one(name, entry, inventory))
|
||||
|
||||
diff_count = 0
|
||||
for path, content in desired.items():
|
||||
existing = path.read_text() if path.exists() else ""
|
||||
if existing != content:
|
||||
diff_count += 1
|
||||
if args.check:
|
||||
diff = difflib.unified_diff(
|
||||
existing.splitlines(keepends=True),
|
||||
content.splitlines(keepends=True),
|
||||
fromfile=str(path),
|
||||
tofile=str(path) + " (generated)",
|
||||
)
|
||||
sys.stdout.writelines(diff)
|
||||
else:
|
||||
path.write_text(content)
|
||||
print(f"wrote {path.relative_to(REPO)}")
|
||||
|
||||
# Clean up orphans (file exists but host removed from inventory).
|
||||
for existing_path in HOSTS_DIR.glob("*.yaml"):
|
||||
if existing_path not in desired:
|
||||
diff_count += 1
|
||||
if args.check:
|
||||
print(f"orphan: {existing_path.relative_to(REPO)} (would delete)")
|
||||
else:
|
||||
existing_path.unlink()
|
||||
print(f"deleted orphan {existing_path.relative_to(REPO)}")
|
||||
|
||||
if args.check and diff_count > 0:
|
||||
print(f"\n{diff_count} file(s) would change. Run without --check to write.",
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
451
plans/2026-07-07-client-lifecycle-in-go.md
Normal file
451
plans/2026-07-07-client-lifecycle-in-go.md
Normal file
@@ -0,0 +1,451 @@
|
||||
# Plan: Client lifecycle — enrollment through deprecation in Oikos Go
|
||||
|
||||
**Status:** Planned (2026-07-07)
|
||||
|
||||
## Goal
|
||||
|
||||
Define and implement the complete lifecycle of a homelab client in the Oikos Go
|
||||
runtime: how a new machine is provisioned, enrolled, given secrets, synced,
|
||||
operated, and eventually deprecated (or decommissioned or destroyed). Every
|
||||
state transition feeds the Postgres DB as the authoritative source of truth.
|
||||
No step depends on the archived Python `secrets-issuance` server or the
|
||||
non-existent `bin/homelab` CLI.
|
||||
|
||||
## Current state — what exists vs. what runs
|
||||
|
||||
| Component | Exists? | Runs? | Notes |
|
||||
|-----------|---------|-------|-------|
|
||||
| `bootstrap.sh` (684 lines) | ✅ repo | ⚠️ references dead endpoints | Calls `https://secrets.hubris.network/issue` (Python server, stopped per Phase 6). Symlinks `bin/homelab` (file absent). References `tools/*.setup.sh` (files absent). |
|
||||
| `archive/secrets-issuance/server.py` | ✅ archive | ❌ stopped (Phase 6) | Issued age keys, validated mesh IP. No Go replacement. |
|
||||
| `archive/mcp/` (build_host_files.py, deleted) | ❌ deleted | ❌ | Legacy host file builder. |
|
||||
| `inventory.yaml` (root) | ✅ | ⚠️ edited manually | Flat `hosts:` + `services:` layout. Diverges from `seeds/inventory.yaml` entity-relationship format. |
|
||||
| `seeds/inventory.yaml` | ✅ | ✅ ingested into DB | Entity-relationship format with slugs (`host:hubris`, `ws:mac-mini`). No translation path from root format. |
|
||||
| Go entity API (`POST/GET/PATCH /entities`) | ✅ | ✅ | Generic CRUD. No client-specific validation, no key issuance, no lifecycle gating. |
|
||||
| Go MCP server | ✅ | ✅ | 15 tools. Missing `whoami`, `explain`, `preflight`, `get_change_history`, `get_state_snapshot`. |
|
||||
| `oikos secret` CLI (Infisical/SOPS) | ✅ | ✅ | Secrets read/migrate/export. No client-key provisioning. |
|
||||
| Sync timer (post-pull.sh) | ✅ | ⚠️ partially broken | References `tools/*.setup.sh` (glob returns zero files). `setup-caveman.sh` and `setup-hermes-soul.sh` documented but absent. |
|
||||
|
||||
**Takeaway**: Enrollment today runs on shell scripts calling archived Python
|
||||
services. The Go runtime has zero awareness of client lifecycle. This plan
|
||||
closes that gap — the DB becomes the sole engine for client identity,
|
||||
secrets, state, and lifecycle transitions.
|
||||
|
||||
## Target architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ NEW CLIENT (bare machine) │
|
||||
│ │
|
||||
│ 1. curl bootstrap.sh | sudo bash │
|
||||
│ → clones repo, installs sync timer │
|
||||
│ → calls POST /api/v1/clients/enroll (new endpoint) │
|
||||
│ → receives age keypair from Oikos API │
|
||||
│ → writes /etc/age/key.txt │
|
||||
│ → sync timer starts pulling every 5 min │
|
||||
└──────────────────────────┬───────────────────────────────────┘
|
||||
│ POST /api/v1/clients/enroll
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ OIKOS API (Go, :8090) │
|
||||
│ │
|
||||
│ POST /api/v1/clients/enroll — issue age key, set state │
|
||||
│ POST /api/v1/clients/{slug}/activate — provisioning→active │
|
||||
│ POST /api/v1/clients/{slug}/deprecate — active→deprecated │
|
||||
│ POST /api/v1/clients/{slug}/destroy — deprecated→destroyed │
|
||||
│ GET /api/v1/clients/{slug}/secrets — client's accessible │
|
||||
│ secrets (Infisical lookup by machine identity) │
|
||||
│ MCP whoami(hostname) — client self-introspection │
|
||||
│ MCP explain(service) — compact context card │
|
||||
│ MCP preflight(service) — risk classification │
|
||||
│ MCP get_change_history(entity) — ledger entries │
|
||||
│ MCP get_state_snapshot() — last scheduler pass │
|
||||
└──────────────────────────┬───────────────────────────────────┘
|
||||
│ writes
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ POSTGRES (TimescaleDB) │
|
||||
│ │
|
||||
│ entities table: slug, type, name, state, attributes (JSONB) │
|
||||
│ entity_status: health, disk, drift count (scheduler) │
|
||||
│ audit_log: every state transition, enrollment, revocation │
|
||||
│ executions: approved actions, results │
|
||||
│ secrets (via Infisical): age keys, API tokens │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Client lifecycle: state machine
|
||||
|
||||
```
|
||||
[planned] ──→ provisioning ──→ active ──→ migrating ──→ active
|
||||
│ │ │
|
||||
│ │ ├──→ deprecated ──→ destroyed
|
||||
│ │ │
|
||||
│ └──→ failed └──→ failed
|
||||
│
|
||||
└──→ destroyed (cancelled)
|
||||
```
|
||||
|
||||
### State: `planned`
|
||||
|
||||
The operator declares intent. A client entity exists in the DB with state
|
||||
`planned` but has no host, no keys, no sync.
|
||||
|
||||
**Entry condition**: Operator creates the entity via API or seed file.
|
||||
|
||||
**Required attributes**:
|
||||
- `slug` — `ws:<hostname>` for workstations, `host:<hostname>` for servers
|
||||
- `type` — `workstation`, `standalone-server`, or `proxmox-host`
|
||||
- `name` — human-readable name
|
||||
- `lan_ip` — expected LAN IP (reserved in DHCP)
|
||||
- `os` — `linux` or `macos`
|
||||
- `role` — free-text description of what this machine does
|
||||
- `mesh.expected_type` — `netbird` or `tailscale` (which mesh it will join)
|
||||
- `ssh.user` — login user (default `root`)
|
||||
|
||||
**Allowed transitions**: `→ provisioning` (operator triggers), `→ destroyed` (cancelled).
|
||||
|
||||
### State: `provisioning`
|
||||
|
||||
The machine has been declared. Operator runs `bootstrap.sh` on the target,
|
||||
which calls the enrollment API. The API validates identity (mesh IP matches
|
||||
expected subnet, hostname matches slug), issues an age keypair, and records
|
||||
the public key. The sync timer starts pulling the repo.
|
||||
|
||||
**Go API**: `POST /api/v1/clients/enroll`
|
||||
```json
|
||||
{
|
||||
"slug": "ws:new-laptop",
|
||||
"hostname": "new-laptop",
|
||||
"mesh_ip": "100.122.x.x"
|
||||
}
|
||||
```
|
||||
|
||||
**What the enrollment endpoint does**:
|
||||
1. Looks up entity by slug — must exist, must be in state `planned` or `provisioning`
|
||||
2. Validates mesh IP is in `100.122.0.0/16` (Netbird) or `100.64.0.0/10` (Tailscale) or `192.168.8.0/24` (LAN)
|
||||
3. Validates hostname has no conflicting mesh IP already recorded
|
||||
4. Generates an age keypair (`age-keygen`)
|
||||
5. Stores the private key in Infisical under path `/clients/<slug>/age-key`
|
||||
6. Creates an Infisical machine identity for the client (UniversalAuth)
|
||||
7. Updates entity `attributes` with `age_pubkey`, `mesh_ip`, `enrolled_at`
|
||||
8. Writes audit log: `client.enrolled`
|
||||
9. Returns the age private key, Infisical client ID + secret, and machine identity token
|
||||
|
||||
**Response** (to bootstrap.sh, over mesh — TLS + mesh IP validation):
|
||||
```json
|
||||
{
|
||||
"age_private_key": "AGE-SECRET-KEY-...",
|
||||
"age_public_key": "age1...",
|
||||
"infisical_client_id": "...",
|
||||
"infisical_client_secret": "...",
|
||||
"machine_identity_token": "..."
|
||||
}
|
||||
```
|
||||
|
||||
**Bootstrap script changes**:
|
||||
- Remove call to `https://secrets.hubris.network/issue`
|
||||
- Replace with `POST /api/v1/clients/enroll` to `https://oikos.hubris.network`
|
||||
- Remove `--no-secrets` / `--no-mesh` flags (or keep as escape hatches with degraded state)
|
||||
- Remove symlink to `bin/homelab` (file doesn't exist)
|
||||
- After receiving keys, bootstrap.sh writes `/etc/age/key.txt` (0600) and `/etc/infisical/identity` (0600)
|
||||
|
||||
**Pre-built bootstrap**: The bootstrap.sh is served from the Gitea repo raw URL
|
||||
(already the case). After this plan, it calls Oikos API instead of the dead
|
||||
Python service.
|
||||
|
||||
**Allowed transition**: `→ active` (when `age-key-enrolled`, `mesh-joined`,
|
||||
`doc-page-complete` checks pass).
|
||||
|
||||
### State: `active`
|
||||
|
||||
Normal operation. The client pulls the repo every 5 minutes, uses its age key
|
||||
to decrypt SOPS secrets (fallback), and authenticates to Infisical via its
|
||||
machine identity (primary). The MCP `whoami(hostname)` tool returns its
|
||||
entity record, peer list, accessible secrets, and current health.
|
||||
|
||||
**Go enforcement of transition checks** (`provisioning → active`):
|
||||
- `age-key-enrolled-if-needed`: entity.attributes.age_pubkey is non-empty
|
||||
- `mesh-joined-if-needed`: entity.attributes.mesh_ip is non-empty
|
||||
- `ingress-live-if-public`: skipped for workstations (no public ingress)
|
||||
- `health-check-answering`: scheduler probe passes for this entity
|
||||
- `doc-page-complete`: entity has at least one `documents` edge
|
||||
- `inventory-in-db`: entity exists in DB with all required attributes
|
||||
|
||||
**API**: `POST /api/v1/clients/{slug}/activate`
|
||||
- Validates all `provisioning → active` transition checks
|
||||
- Sets state to `active`
|
||||
- Writes audit log: `client.activated`
|
||||
|
||||
**MCP tools active clients get**:
|
||||
- `whoami(hostname)` — returns entity record, peers, secrets list, health
|
||||
- `list_my_secrets(caller_pubkey?)` — secrets this client can decrypt
|
||||
|
||||
**Allowed transitions**: `→ migrating`, `→ deprecated`, `→ failed`.
|
||||
|
||||
### State: `migrating`
|
||||
|
||||
Client is being moved — OS reinstall, hardware swap, role change. Inbound edges
|
||||
still exist; no deprovisioning has started.
|
||||
|
||||
**Allowed transition**: `→ active` (migration complete, post-verify passes).
|
||||
|
||||
**API**: `POST /api/v1/clients/{slug}/migrate` (sets state, links migration plan).
|
||||
|
||||
### State: `deprecated`
|
||||
|
||||
Client is being phased out. Services moved off, mesh disconnected, secrets
|
||||
rotation started. The deprecation gate (`no-inbound-edges`) blocks `→ destroyed`
|
||||
until all `depends-on`, `hosts`, `provides`, and `mounts` edges are gone.
|
||||
|
||||
**API**: `POST /api/v1/clients/{slug}/deprecate`
|
||||
- Validates `replacement-live-or-role-retired`: operator confirms replacement exists or role is no longer needed
|
||||
- Sets state to `deprecated`
|
||||
- Writes audit log: `client.deprecated`
|
||||
|
||||
**Allowed transitions**: `→ active` (un-deprecate), `→ destroyed`.
|
||||
|
||||
### State: `destroyed`
|
||||
|
||||
Client is gone. All edges removed, secrets revoked, archaeology entry written.
|
||||
|
||||
**API**: `POST /api/v1/clients/{slug}/destroy`
|
||||
- Validates all `deprecated → destroyed` transition checks:
|
||||
- `backups-verified`: any data on this client was backed up
|
||||
- `secrets-revoked-and-rekeyed`: age key removed from Infisical, SOPS recipients updated, machine identity deleted
|
||||
- `ingress-and-dns-removed`: no remaining DNS records or Caddy backends
|
||||
- `no-inbound-edges`: zero `depends-on`, `hosts`, `provides`, `mounts` edges pointing to this entity
|
||||
- `archaeology-entry`: writes a record explaining why and when
|
||||
- Sets state to `destroyed`
|
||||
- Revokes Infisical machine identity
|
||||
- Removes age public key from `.sops.yaml`
|
||||
- Writes audit log: `client.destroyed`
|
||||
|
||||
### State: `failed`
|
||||
|
||||
Something went wrong during provisioning or operation. Requires operator
|
||||
intervention. Treated as informational — no automatic recovery.
|
||||
|
||||
**API**: `POST /api/v1/clients/{slug}/fail`
|
||||
- Sets state to `failed`
|
||||
- Requires `reason` field explaining what broke
|
||||
- Writes audit log: `client.failed`
|
||||
|
||||
## Secrets integration
|
||||
|
||||
### Age key lifecycle
|
||||
|
||||
```
|
||||
planned ────────────→ no key exists
|
||||
provisioning ───────→ keypair generated, pubkey stored in entity attributes,
|
||||
private key delivered to client via enroll response,
|
||||
private key stored in Infisical under /clients/<slug>/age-key
|
||||
active ─────────────→ key used for SOPS decryption fallback, authenticated
|
||||
to Infisical via machine identity for primary secrets
|
||||
deprecated ─────────→ key still valid, but rotation initiated
|
||||
destroyed ──────────→ key revoked from Infisical, removed from .sops.yaml,
|
||||
machine identity deleted
|
||||
```
|
||||
|
||||
### Infisical machine identity
|
||||
|
||||
Each client gets an Infisical machine identity during enrollment. This is the
|
||||
primary secrets path — the age key is fallback for SOPS-encrypted DR files.
|
||||
|
||||
- **Client ID + Secret** returned in enroll response
|
||||
- **Scoped to paths**: `/clients/<slug>/*`, `/shared/*`
|
||||
- **Revoked on destroy**: identity deleted, access gone
|
||||
|
||||
### SOPS fallback
|
||||
|
||||
The age public key is added to `.sops.yaml` recipients during enrollment.
|
||||
On destroy, it is removed via `oikos secret export-sops` regeneration.
|
||||
|
||||
### bootstrap.sh changes
|
||||
|
||||
```diff
|
||||
- # calls https://secrets.hubris.network/issue (Python, dead)
|
||||
- AGE_KEY=$(curl -s -X POST "$ISSUANCE_URL" ...)
|
||||
-
|
||||
+ # calls Oikos API enrollment endpoint
|
||||
+ ENROLL_RESP=$(curl -s -X POST "$OIKOS_URL/api/v1/clients/enroll" \
|
||||
+ -H "Content-Type: application/json" \
|
||||
+ -d "{\"slug\":\"ws:$HNAME\",\"hostname\":\"$HNAME\",\"mesh_ip\":\"$MESH_IP\"}")
|
||||
+ AGE_PRIVKEY=$(echo "$ENROLL_RESP" | jq -r '.age_private_key')
|
||||
```
|
||||
|
||||
## DB integration
|
||||
|
||||
### New migration
|
||||
|
||||
`012_client_enrollment.up.sql`:
|
||||
```sql
|
||||
-- No new tables needed — entities table already holds clients.
|
||||
-- Add enrollment-specific attributes validation via check constraints
|
||||
-- or application-level validation.
|
||||
|
||||
-- Enforce slug format for machine entities
|
||||
-- ws:<hostname> for workstations, host:<hostname> for servers
|
||||
-- (application-level validation in Go, not a DB constraint)
|
||||
|
||||
-- Add index for slug-based client lookups
|
||||
CREATE INDEX IF NOT EXISTS idx_entities_slug_type
|
||||
ON entities (slug, type)
|
||||
WHERE type IN ('workstation', 'standalone-server', 'proxmox-host');
|
||||
```
|
||||
|
||||
### Entity attributes schema (for `machine` types)
|
||||
|
||||
```json
|
||||
{
|
||||
"cpu_arch": "arm64",
|
||||
"ram_gb": 16,
|
||||
"os": "macos",
|
||||
"lan_ip": "192.168.8.175",
|
||||
"mesh": {
|
||||
"netbird": {"ip": "100.122.x.x", "fqdn": "hostname.netbird.selfhosted"}
|
||||
},
|
||||
"ssh": {"user": "dtoro"},
|
||||
"age_pubkey": "age1...",
|
||||
"enrolled_at": "2026-07-07T12:00:00Z",
|
||||
"enrolled_by": "ws:mac-mini",
|
||||
"infisical_identity_id": "identity_abc123"
|
||||
}
|
||||
```
|
||||
|
||||
All attributes are stored in the `attributes` JSONB column on the `entities`
|
||||
table. Validation happens at the application layer (Go) using the schema
|
||||
defined in `seeds/ontology.yaml`.
|
||||
|
||||
## MCP tools to add
|
||||
|
||||
These are documented in AGENTS.md section 3 but not implemented in the Go MCP
|
||||
server. Implementation: register in `internal/mcp/server.go`.
|
||||
|
||||
| Tool | Input | Output | Implementation |
|
||||
|------|-------|--------|----------------|
|
||||
| `whoami` | `hostname` | Entity record, peers, accessible secrets, health | DB lookup by slug derived from hostname |
|
||||
| `list_my_secrets` | `caller_pubkey?` | Secrets this client can decrypt | Infisical list + SOPS `.sops.yaml` match |
|
||||
| `explain` | `service_slug` | Compact context card: type, state, health, relations, last change | DB join: entity + entity_status + audit_log |
|
||||
| `preflight` | `service_slug` | Risk class, approval requirement, verification command | Policy classifier on the entity's type |
|
||||
| `get_change_history` | `entity_slug`, `limit` | Last N audit_log entries for entity | DB query on audit_log table |
|
||||
| `get_state_snapshot` | none | Last scheduler Observe pass: health, disk, drift count | DB query on entity_status + signals |
|
||||
|
||||
## API endpoints to add
|
||||
|
||||
Add to `api/openapi.yaml`, regenerate with `make generate`, implement in
|
||||
`internal/httpapi/impl.go`.
|
||||
|
||||
| Method | Path | Scope | Purpose |
|
||||
|--------|------|-------|---------|
|
||||
| `POST` | `/api/v1/clients/enroll` | agent | Issue age key, validate mesh, set state → provisioning |
|
||||
| `POST` | `/api/v1/clients/{slug}/activate` | operator | Run transition checks, state → active |
|
||||
| `POST` | `/api/v1/clients/{slug}/deprecate` | operator | State → deprecated |
|
||||
| `POST` | `/api/v1/clients/{slug}/destroy` | operator | Run destroy checks, revoke secrets, state → destroyed |
|
||||
| `POST` | `/api/v1/clients/{slug}/fail` | operator | State → failed with reason |
|
||||
| `GET` | `/api/v1/clients/{slug}/secrets` | agent | List secrets this client can access |
|
||||
|
||||
## Files changed
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `bootstrap.sh` | Replace `secrets.hubris.network/issue` call with `POST /api/v1/clients/enroll`. Remove dead symlinks. |
|
||||
| `api/openapi.yaml` | Add client enrollment, lifecycle, and secret endpoints |
|
||||
| `internal/httpapi/impl.go` | Implement client lifecycle handlers |
|
||||
| `internal/db/queries/clients.sql` | Add client-specific sqlc queries |
|
||||
| `internal/mcp/server.go` | Register whoami, explain, preflight, get_change_history, get_state_snapshot, list_my_secrets |
|
||||
| `internal/secrets/infisical.go` | Add `CreateMachineIdentity`, `DeleteMachineIdentity`, `StoreClientKey` |
|
||||
| `internal/ontology/validate.go` | Implement lifecycle transition checks for infrastructure lifecycle |
|
||||
| `seeds/ontology.yaml` | Add client-specific attributes schema for machine types |
|
||||
| `migrations/012_client_enrollment.up.sql` | Index for slug+type lookups |
|
||||
| `AGENTS.md` | Update MCP tool list to match actual implementation |
|
||||
| `CLIENTS.md` | Update enrollment flow to reference Oikos API, not Python issuance |
|
||||
| `CONTRIBUTING.md` | Add client lifecycle as a documented extension point |
|
||||
|
||||
## Files deleted or deprecated
|
||||
|
||||
| File | Disposition |
|
||||
|------|-------------|
|
||||
| `archive/secrets-issuance/` | Already archived. Add deprecation notice referencing this plan. |
|
||||
| `archive/secrets-sops-backup/` | Keep for DR. Add note that new clients use Infisical, SOPS is fallback. |
|
||||
| Any reference to `bin/homelab` | Delete or comment out in bootstrap.sh; CLI doesn't exist. |
|
||||
| `tools/*.setup.sh` references | Either create the files or remove the auto-setup convention from post-pull.sh. |
|
||||
|
||||
## Phased implementation
|
||||
|
||||
### Phase 1 — API + DB (P0, this week)
|
||||
|
||||
1. Write `migrations/012_client_enrollment.up.sql`
|
||||
2. Add client endpoints to `api/openapi.yaml`
|
||||
3. Run `make generate`
|
||||
4. Implement enrollment handler (`POST /api/v1/clients/enroll`):
|
||||
- Age key generation
|
||||
- Infisical machine identity creation
|
||||
- Entity attribute update
|
||||
- Audit log write
|
||||
5. Implement lifecycle transition handlers (activate, deprecate, destroy, fail)
|
||||
6. Implement `GET /api/v1/clients/{slug}/secrets`
|
||||
7. Update `seeds/ontology.yaml` with client attribute schemas
|
||||
8. Add sqlc queries in `internal/db/queries/clients.sql`
|
||||
|
||||
### Phase 2 — MCP tools (P1, next week)
|
||||
|
||||
1. Register `whoami(hostname)` in `internal/mcp/server.go`
|
||||
2. Register `explain(service)` — compact context card from DB
|
||||
3. Register `preflight(service)` — risk classification
|
||||
4. Register `get_change_history(entity, limit)`
|
||||
5. Register `get_state_snapshot()`
|
||||
6. Register `list_my_secrets(caller_pubkey?)`
|
||||
|
||||
### Phase 3 — Bootstrap script cleanup (P1, next week)
|
||||
|
||||
1. Replace secrets issuance URL with Oikos API endpoint
|
||||
2. Remove `--no-secrets` / `--no-mesh` or rewire them to degraded modes
|
||||
3. Remove `bin/homelab` symlink
|
||||
4. Update Infisical identity file creation
|
||||
5. Test full enrollment on a fresh machine
|
||||
|
||||
### Phase 4 — Transition check enforcement (P2, within 2 weeks)
|
||||
|
||||
1. Implement all `provisioning → active` checks in `internal/ontology/validate.go`
|
||||
2. Implement all `deprecated → destroyed` checks
|
||||
3. Wire checks into lifecycle transition handlers
|
||||
4. Test that `POST /activate` fails when checks don't pass
|
||||
5. Test that `POST /destroy` fails when inbound edges exist
|
||||
|
||||
### Phase 5 — Cleanup (P2, within 2 weeks)
|
||||
|
||||
1. Delete or comment-out dead code in bootstrap.sh
|
||||
2. Recreate `tools/setup-caveman.sh` and `tools/setup-hermes-soul.sh` (or remove references)
|
||||
3. Update AGENTS.md MCP tool list
|
||||
4. Update CLIENTS.md enrollment flow
|
||||
5. Archive Python secrets-issuance with final deprecation note
|
||||
6. Run `make generate-check` and full test suite
|
||||
|
||||
## Verification
|
||||
|
||||
- Fresh machine with no prior state: `curl bootstrap.sh | sudo bash` → machine
|
||||
shows up in DB as `provisioning` with age pubkey, Infisical identity, and sync
|
||||
timer running
|
||||
- `POST /api/v1/clients/ws:test-machine/activate` → state → `active`, all checks pass
|
||||
- `POST /api/v1/clients/ws:test-machine/deprecate` → state → `deprecated`
|
||||
- `POST /api/v1/clients/ws:test-machine/destroy` → fails if edges exist; succeeds
|
||||
after edges removed, secrets revoked
|
||||
- MCP `whoami(ws:test-machine)` returns client record with peers and health
|
||||
- MCP `explain(service:caddy)` returns context card with relations and risk class
|
||||
- `GET /api/v1/clients/ws:test-machine/secrets` returns secrets list scoped to client
|
||||
- Existing clients continue working through the sync timer (no regression)
|
||||
- `make test test-db generate-check` passes
|
||||
|
||||
## Related
|
||||
|
||||
- [2026-07-07-migrate-bin-homelab-to-go.md](2026-07-07-migrate-bin-homelab-to-go.md) — MCP tool completion plan (whoami, explain, preflight)
|
||||
- [seeds/ontology.yaml](../seeds/ontology.yaml) — lifecycle definitions, entity type hierarchy
|
||||
- [seeds/policy.yaml](../seeds/policy.yaml) — risk classes, approval rules
|
||||
- [CLIENTS.md](../CLIENTS.md) — client onboarding guide (update after this plan)
|
||||
- [bootstrap.sh](../bootstrap.sh) — current enrollment script (rewrite in Phase 3)
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2026-07-07 — initial plan. Replaces Python secrets-issuance, defines full
|
||||
lifecycle in Go, adds client API endpoints, MCP tools, and Infisical
|
||||
machine identity integration.
|
||||
@@ -10,10 +10,10 @@ went sideways, open an investigation.
|
||||
| ---- | ----- | ------ |
|
||||
| 2026-07-05 | [Oikos Prometheus LXC](2026-07-05-oikos-prometheus-lxc.md) | Planned |
|
||||
| 2026-07-06 | [Consolidate Oikos control plane onto mac-mini](2026-07-06-consolidate-oikos-control-plane-onto-mac-mini.md) | In Progress (Phase 1-6 implemented, pending cutover) |
|
||||
| 2026-07-07 | [Client lifecycle in Go — enrollment through deprecation](2026-07-07-client-lifecycle-in-go.md) | Planned |
|
||||
| 2026-07-07 | [Comprehensive audit: stale files, state gaps, and next steps](2026-07-07-comprehensive-audit-and-next-steps.md) | Planned |
|
||||
| 2026-07-07 | [DB as single source of truth for agent knowledge](2026-07-07-db-as-source-of-truth.md) | Proposed |
|
||||
| 2026-07-07 | [Migrate bin/homelab CLI to Go oikos homelab](2026-07-07-migrate-bin-homelab-to-go.md) | Planned |
|
||||
| 2026-07-07 | [Migrate bin/homelab CLI to Go oikos homelab](2026-07-07-migrate-bin-homelab-to-go.md) | Planned |
|
||||
|
||||
## Done
|
||||
|
||||
|
||||
Reference in New Issue
Block a user