dtoro
5b22f2367b
- client_lifecycle_test.go: full end-to-end integration test planned → provisioning (enroll) → active → migrating → active → deprecated → failed. Validates age keypair generation, attrs, context/secrets endpoints, invalid transition blocking, compute entity provisioning with relationship edges and status tracking. Also tests enrollment rejection for invalid states and duplicate slug rejection for provisioning. - adr/0011-client-lifecycle-flows.md: workstation self-enrollment, compute entity provisioning, deprecation/destruction flows with Mermaid sequence diagrams. Full lifecycle state diagram. Transition check enforcement documentation. - adr/0012-hermes-oikos-interactions.md: Hermes ↔ Oikos interaction flow through OODA loop phases. Thin client bootstrap. Internal component interactions (scheduler, actuator, notifier). Complete 30-tool ownership matrix. - Fix: migration 012 FK reference (executions.id → executions.entity_id) - Fix: provision handler null attributes JSONB - Fix: provisioning steps use entity_id for execution FK All 3 integration tests pass, go vet clean.
205 lines
7.7 KiB
Markdown
205 lines
7.7 KiB
Markdown
# ADR 0011 — Client lifecycle sequence diagrams
|
|
|
|
**Status:** Accepted
|
|
**Date:** 2026-07-08
|
|
|
|
## Context
|
|
|
|
The Oikos client lifecycle spans two distinct onboarding paths — workstation
|
|
self-enrollment and compute entity (LXC/VM) provisioning — sharing the same
|
|
infrastructure lifecycle state machine (`planned → provisioning → active →
|
|
migrating → deprecated → destroyed`). These flows must be documented and
|
|
validated against the Go implementation.
|
|
|
|
## Decision
|
|
|
|
All machines in the homelab follow the same lifecycle state machine defined
|
|
in `seeds/ontology.yaml` under the `infrastructure` lifecycle. The
|
|
implementation lives in `internal/httpapi/impl.go` (handlers),
|
|
`internal/actuator/actuator.go` (SSH provisioning), and
|
|
`internal/ontology/validate.go` (transition check enforcement).
|
|
|
|
## Workstation self-enrollment flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Op as Operator
|
|
participant BS as bootstrap.sh
|
|
participant API as Oikos API (:8090)
|
|
participant DB as Postgres
|
|
participant Inf as Infisical
|
|
|
|
Op->>API: POST /entities {slug:"ws:laptop", type:"workstation", state:"planned"}
|
|
API->>DB: INSERT entities (planned)
|
|
API-->>Op: 201 + ETag
|
|
|
|
Op->>BS: curl bootstrap.sh | sudo bash
|
|
BS->>BS: detect hostname, mesh IP
|
|
BS->>BS: fetch CLIENTS.md, AGENTS.md, OIKOS.md, tools/
|
|
BS->>BS: install age, curl, jq
|
|
|
|
BS->>API: POST /clients/enroll {slug, hostname, mesh_ip}
|
|
API->>DB: validate state = planned|provisioning
|
|
API->>API: generateAgeKeypair()
|
|
API->>API: store age key in Infisical (best-effort)
|
|
API->>DB: UPDATE state→provisioning, set age_pubkey, mesh_ip, enrolled_at
|
|
API->>DB: INSERT audit_log (client.enrolled)
|
|
API-->>BS: {age_private_key, age_public_key, infisical_client_id, infisical_client_secret}
|
|
|
|
BS->>BS: write /etc/age/key.txt (0600)
|
|
BS->>BS: write /etc/infisical/identity (0600)
|
|
BS->>BS: install context-poller.sh (launchd/systemd, 5min)
|
|
|
|
loop Every 5 minutes
|
|
BS->>API: GET /clients/ws:laptop/context?since={timestamp}
|
|
API->>DB: SELECT context_files, context_version
|
|
API-->>BS: {agent_files_changed, sops_config_changed, tools_changed}
|
|
end
|
|
|
|
Op->>API: PATCH /entities/ws:laptop {state:"active"}
|
|
API->>DB: validate lifecyle transition provisioning→active
|
|
API->>DB: UPDATE state→active, version+1
|
|
API->>DB: INSERT audit_log (client.activated)
|
|
API-->>Op: 200 + ETag
|
|
```
|
|
|
|
### Enforced transition checks
|
|
|
|
Before `provisioning → active`, the `TransitionChecks` map in
|
|
`internal/ontology/validate.go` validates:
|
|
|
|
| Check | Workstation | LXC/VM |
|
|
|-------|------------|--------|
|
|
| `age-key-enrolled-if-needed` | Requires `age_pubkey` in attrs | Skipped (no age key for compute entities) |
|
|
| `mesh-joined-if-needed` | Requires `mesh_ip` in attrs | Skipped |
|
|
| `health-check-answering` | Queries `entity_status.health != 'down'` | Same |
|
|
| `doc-page-complete` | Requires at least 1 `documents` edge | Same |
|
|
|
|
### Before `deprecated → destroyed`:
|
|
|
|
| Check | Description |
|
|
|-------|-------------|
|
|
| `no-inbound-edges` | Zero `depends-on`, `hosts`, `provides`, `mounts`, `routes-to`, `stores-on` edges |
|
|
| `secrets-revoked-and-rekeyed` | `age_pubkey` must be removed from attributes |
|
|
| `backups-verified` | Audit log must have a `backup-verified` entry in last 30 days |
|
|
| `ingress-and-dns-removed` | No remaining `routes-to`/`provides`/`hosts` edges |
|
|
|
|
## Compute entity provisioning flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Op as Operator/Hermes
|
|
participant API as Oikos API (:8090)
|
|
participant DB as Postgres
|
|
participant Act as Actuator
|
|
participant PVE as Proxmox Host
|
|
|
|
Op->>API: POST /entities/provision {slug:"lxc:jellyfin", host:"host:hubris", attrs:{vmid,cores,...}}
|
|
API->>DB: validate host exists, slug not taken
|
|
API->>DB: INSERT entities (planned)
|
|
API->>DB: INSERT executions (provision)
|
|
API->>DB: INSERT provisioning_steps (6 steps, all pending)
|
|
API->>DB: INSERT relationships (host:hubris hosts lxc:jellyfin)
|
|
API->>DB: INSERT audit_log (entity.provisioned)
|
|
API-->>Op: 201 {entity, execution_id}
|
|
|
|
Note over Act,PVE: Actuator loop picks up provisioning execution
|
|
|
|
Act->>API: GET execution for lxc:jellyfin
|
|
Act->>DB: resolve host:hubris → (mesh IP, ssh user)
|
|
|
|
par Provisioning steps
|
|
Act->>DB: UPDATE provisioning_step[1] (validate-constraints)
|
|
Act->>PVE: ssh: pct status {vmid}
|
|
PVE-->>Act: "does not exist" → ok
|
|
Act->>DB: UPDATE provisioning_step[1] (ok)
|
|
|
|
Act->>DB: UPDATE provisioning_step[2] (create-container)
|
|
Act->>PVE: ssh: pct create {vmid} --cores N --memory M --rootfs ...
|
|
PVE-->>Act: container created
|
|
Act->>DB: UPDATE provisioning_step[2] (ok)
|
|
|
|
Act->>PVE: ssh: pct exec {vmid} -- apt install -y service1 service2
|
|
Act->>DB: UPDATE provisioning_step[4] (ok)
|
|
|
|
Act->>PVE: ssh: pct set {vmid} -mp0 /mnt/library,/mnt/library
|
|
Act->>DB: UPDATE provisioning_step[5] (ok)
|
|
|
|
Act->>PVE: ssh: pct exec {vmid} -- systemctl is-system-running
|
|
PVE-->>Act: "running" → ok
|
|
Act->>DB: UPDATE provisioning_step[6] (ok)
|
|
end
|
|
|
|
Act->>DB: UPDATE entity state→active
|
|
Act->>DB: UPDATE execution status→completed
|
|
```
|
|
|
|
## Deprecation and destruction flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Op as Operator
|
|
participant API as Oikos API
|
|
participant DB as Postgres
|
|
|
|
Note over Op,DB: Active entity → Deprecated
|
|
|
|
Op->>API: PATCH /entities/lxc:jellyfin {state:"deprecated"}
|
|
API->>DB: validate lifecycle transition active→deprecated
|
|
API->>DB: UPDATE state→deprecated
|
|
API-->>Op: 200
|
|
|
|
Note over Op,DB: Deprecated → Destroyed (with gate checks)
|
|
|
|
Op->>API: PATCH /entities/lxc:jellyfin {state:"destroyed"}
|
|
alt Inbound edges exist
|
|
API->>DB: TransitionChecks["no-inbound-edges"] → count > 0
|
|
API-->>Op: 409 (inbound edges still exist)
|
|
else All checks pass
|
|
API->>DB: TransitionChecks → all pass
|
|
API->>DB: UPDATE state→destroyed
|
|
API->>DB: INSERT audit_log (client.destroyed)
|
|
API-->>Op: 200
|
|
end
|
|
```
|
|
|
|
## Full lifecycle state diagram
|
|
|
|
```mermaid
|
|
stateDiagram-v2
|
|
[*] --> planned : POST /entities
|
|
|
|
planned --> provisioning : POST /clients/enroll (workstation)<br/>POST /entities/provision (compute)
|
|
planned --> destroyed : cancelled via PATCH
|
|
|
|
provisioning --> active : PATCH state="active"<br/>checks: age-key, mesh, health
|
|
provisioning --> failed : error during provisioning
|
|
|
|
active --> migrating : PATCH state="migrating"
|
|
active --> deprecated : PATCH state="deprecated"
|
|
active --> failed : PATCH state="failed"
|
|
|
|
migrating --> active : PATCH state="active"
|
|
migrating --> failed : PATCH state="failed"
|
|
|
|
failed --> active : PATCH state="active"<br/>check: recovery-verified
|
|
|
|
deprecated --> active : PATCH state="active" (un-deprecate)
|
|
deprecated --> destroyed : PATCH state="destroyed"<br/>checks: no-inbound-edges,<br/>secrets-revoked, backups-verified,<br/>ingress-dns-removed
|
|
|
|
destroyed --> [*]
|
|
```
|
|
|
|
## Consequences
|
|
|
|
- **Workstations** self-enroll via `bootstrap.sh` → `POST /clients/enroll`.
|
|
The age keypair is generated server-side and delivered once.
|
|
- **Compute entities** are provisioned by the actuator over SSH. The operator
|
|
declares intent via `POST /entities/provision`; the actuator executes
|
|
step-by-step with DB-tracked progress.
|
|
- **Transition gates** are enforced by named checks in
|
|
`internal/ontology/validate.go`. The `deprecated → destroyed` gate blocks
|
|
until all inbound edges are severed — preventing orphan references.
|
|
- **Thin clients** poll `GET /clients/{slug}/context` for agent file deltas
|
|
instead of `git pull`. The control plane host (mac-mini) keeps the full
|
|
repo clone. |