oikos/plans/2026-07-07-client-lifecycle-in-go.md

Plan: Client lifecycle — enrollment through deprecation in Oikos Go

Status: Planned (2026-07-07)

Goal

Define and implement the complete lifecycle of a homelab client in the Oikos Go runtime: how a new machine is provisioned, enrolled, given secrets, synced, operated, and eventually deprecated (or decommissioned or destroyed). Every state transition feeds the Postgres DB as the authoritative source of truth. No step depends on the archived Python secrets-issuance server or the non-existent bin/homelab CLI.

Current state — what exists vs. what runs

Component	Exists?	Runs?	Notes
bootstrap.sh (684 lines)	✅ repo	⚠️ references dead endpoints	Calls https://secrets.hubris.network/issue (Python server, stopped per Phase 6). Symlinks bin/homelab (file absent). References tools/*.setup.sh (files absent).
archive/secrets-issuance/server.py	✅ archive	❌ stopped (Phase 6)	Issued age keys, validated mesh IP. No Go replacement.
archive/mcp/ (build_host_files.py, deleted)	❌ deleted	❌	Legacy host file builder.
inventory.yaml (root)	✅	⚠️ edited manually	Flat hosts: + services: layout. Diverges from seeds/inventory.yaml entity-relationship format.
seeds/inventory.yaml	✅	✅ ingested into DB	Entity-relationship format with slugs (host:hubris, ws:mac-mini). No translation path from root format.
Go entity API (POST/GET/PATCH /entities)	✅	✅	Generic CRUD. No client-specific validation, no key issuance, no lifecycle gating.
Go MCP server	✅	✅	15 tools. Missing whoami, explain, preflight, get_change_history, get_state_snapshot.
oikos secret CLI (Infisical/SOPS)	✅	✅	Secrets read/migrate/export. No client-key provisioning.
Sync timer (post-pull.sh)	✅	⚠️ partially broken	References tools/*.setup.sh (glob returns zero files). setup-caveman.sh and setup-hermes-soul.sh documented but absent.

Takeaway: Enrollment today runs on shell scripts calling archived Python services. The Go runtime has zero awareness of client lifecycle. This plan closes that gap — the DB becomes the sole engine for client identity, secrets, state, and lifecycle transitions.

Target architecture

┌──────────────────────────────────────────────────────────────┐
│                    NEW CLIENT (bare machine)                  │
│                                                              │
│  1. curl bootstrap.sh | sudo bash                            │
│     → clones repo, installs sync timer                        │
│     → calls POST /api/v1/clients/enroll (new endpoint)        │
│     → receives age keypair from Oikos API                     │
│     → writes /etc/age/key.txt                                 │
│     → sync timer starts pulling every 5 min                   │
└──────────────────────────┬───────────────────────────────────┘
                           │ POST /api/v1/clients/enroll
                           ▼
┌──────────────────────────────────────────────────────────────┐
│                    OIKOS API (Go, :8090)                      │
│                                                              │
│  POST /api/v1/clients/enroll     — issue age key, set state   │
│  POST /api/v1/clients/{slug}/activate — provisioning→active  │
│  POST /api/v1/clients/{slug}/deprecate — active→deprecated   │
│  POST /api/v1/clients/{slug}/destroy  — deprecated→destroyed │
│  GET  /api/v1/clients/{slug}/secrets — client's accessible   │
│        secrets (Infisical lookup by machine identity)         │
│  MCP  whoami(hostname)           — client self-introspection │
│  MCP  explain(service)           — compact context card      │
│  MCP  preflight(service)         — risk classification       │
│  MCP  get_change_history(entity) — ledger entries            │
│  MCP  get_state_snapshot()       — last scheduler pass       │
└──────────────────────────┬───────────────────────────────────┘
                           │ writes
                           ▼
┌──────────────────────────────────────────────────────────────┐
│                    POSTGRES (TimescaleDB)                     │
│                                                              │
│  entities table: slug, type, name, state, attributes (JSONB) │
│  entity_status: health, disk, drift count (scheduler)        │
│  audit_log: every state transition, enrollment, revocation   │
│  executions: approved actions, results                       │
│  secrets (via Infisical): age keys, API tokens               │
└──────────────────────────────────────────────────────────────┘

Client lifecycle: state machine

  [planned] ──→ provisioning ──→ active ──→ migrating ──→ active
      │              │               │
      │              │               ├──→ deprecated ──→ destroyed
      │              │               │
      │              └──→ failed     └──→ failed
      │
      └──→ destroyed (cancelled)

State: planned

The operator declares intent. A client entity exists in the DB with state planned but has no host, no keys, no sync.

Entry condition: Operator creates the entity via API or seed file.

Required attributes:

• slug — ws:<hostname> for workstations, host:<hostname> for servers
• type — workstation, standalone-server, or proxmox-host
• name — human-readable name
• lan_ip — expected LAN IP (reserved in DHCP)
• os — linux or macos
• role — free-text description of what this machine does
• mesh.expected_type — netbird or tailscale (which mesh it will join)
• ssh.user — login user (default root)

Allowed transitions: → provisioning (operator triggers), → destroyed (cancelled).

State: provisioning

The machine has been declared. Operator runs bootstrap.sh on the target, which calls the enrollment API. The API validates identity (mesh IP matches expected subnet, hostname matches slug), issues an age keypair, and records the public key. The sync timer starts pulling the repo.

Go API: POST /api/v1/clients/enroll

{
  "slug": "ws:new-laptop",
  "hostname": "new-laptop",
  "mesh_ip": "100.122.x.x"
}

What the enrollment endpoint does:

1. Looks up entity by slug — must exist, must be in state planned or provisioning
2. Validates mesh IP is in 100.122.0.0/16 (Netbird) or 100.64.0.0/10 (Tailscale) or 192.168.8.0/24 (LAN)
3. Validates hostname has no conflicting mesh IP already recorded
4. Generates an age keypair (age-keygen)
5. Stores the private key in Infisical under path /clients/<slug>/age-key
6. Creates an Infisical machine identity for the client (UniversalAuth)
7. Updates entity attributes with age_pubkey, mesh_ip, enrolled_at
8. Writes audit log: client.enrolled
9. Returns the age private key, Infisical client ID + secret, and machine identity token

Response (to bootstrap.sh, over mesh — TLS + mesh IP validation):

{
  "age_private_key": "AGE-SECRET-KEY-...",
  "age_public_key": "age1...",
  "infisical_client_id": "...",
  "infisical_client_secret": "...",
  "machine_identity_token": "..."
}

Bootstrap script changes:

• Remove call to https://secrets.hubris.network/issue
• Replace with POST /api/v1/clients/enroll to https://oikos.hubris.network
• Remove --no-secrets / --no-mesh flags (or keep as escape hatches with degraded state)
• Remove symlink to bin/homelab (file doesn't exist)
• After receiving keys, bootstrap.sh writes /etc/age/key.txt (0600) and /etc/infisical/identity (0600)

Pre-built bootstrap: The bootstrap.sh is served from the Gitea repo raw URL (already the case). After this plan, it calls Oikos API instead of the dead Python service.

Allowed transition: → active (when age-key-enrolled, mesh-joined, doc-page-complete checks pass).

State: active

Normal operation. The client pulls the repo every 5 minutes, uses its age key to decrypt SOPS secrets (fallback), and authenticates to Infisical via its machine identity (primary). The MCP whoami(hostname) tool returns its entity record, peer list, accessible secrets, and current health.

Go enforcement of transition checks (provisioning → active):

• age-key-enrolled-if-needed: entity.attributes.age_pubkey is non-empty
• mesh-joined-if-needed: entity.attributes.mesh_ip is non-empty
• ingress-live-if-public: skipped for workstations (no public ingress)
• health-check-answering: scheduler probe passes for this entity
• doc-page-complete: entity has at least one documents edge
• inventory-in-db: entity exists in DB with all required attributes

API: POST /api/v1/clients/{slug}/activate

• Validates all provisioning → active transition checks
• Sets state to active
• Writes audit log: client.activated

MCP tools active clients get:

• whoami(hostname) — returns entity record, peers, secrets list, health
• list_my_secrets(caller_pubkey?) — secrets this client can decrypt

Allowed transitions: → migrating, → deprecated, → failed.

State: migrating

Client is being moved — OS reinstall, hardware swap, role change. Inbound edges still exist; no deprovisioning has started.

Allowed transition: → active (migration complete, post-verify passes).

API: POST /api/v1/clients/{slug}/migrate (sets state, links migration plan).

State: deprecated

Client is being phased out. Services moved off, mesh disconnected, secrets rotation started. The deprecation gate (no-inbound-edges) blocks → destroyed until all depends-on, hosts, provides, and mounts edges are gone.

API: POST /api/v1/clients/{slug}/deprecate

• Validates replacement-live-or-role-retired: operator confirms replacement exists or role is no longer needed
• Sets state to deprecated
• Writes audit log: client.deprecated

Allowed transitions: → active (un-deprecate), → destroyed.

State: destroyed

Client is gone. All edges removed, secrets revoked, archaeology entry written.

API: POST /api/v1/clients/{slug}/destroy

• Validates all deprecated → destroyed transition checks:
  • backups-verified: any data on this client was backed up
  • secrets-revoked-and-rekeyed: age key removed from Infisical, SOPS recipients updated, machine identity deleted
  • ingress-and-dns-removed: no remaining DNS records or Caddy backends
  • no-inbound-edges: zero depends-on, hosts, provides, mounts edges pointing to this entity
  • archaeology-entry: writes a record explaining why and when
• Sets state to destroyed
• Revokes Infisical machine identity
• Removes age public key from .sops.yaml
• Writes audit log: client.destroyed

State: failed

Something went wrong during provisioning or operation. Requires operator intervention. Treated as informational — no automatic recovery.

API: POST /api/v1/clients/{slug}/fail

• Sets state to failed
• Requires reason field explaining what broke
• Writes audit log: client.failed

Secrets integration

Age key lifecycle

planned ────────────→ no key exists
provisioning ───────→ keypair generated, pubkey stored in entity attributes,
                      private key delivered to client via enroll response,
                      private key stored in Infisical under /clients/<slug>/age-key
active ─────────────→ key used for SOPS decryption fallback, authenticated
                      to Infisical via machine identity for primary secrets
deprecated ─────────→ key still valid, but rotation initiated
destroyed ──────────→ key revoked from Infisical, removed from .sops.yaml,
                      machine identity deleted

Infisical machine identity

Each client gets an Infisical machine identity during enrollment. This is the primary secrets path — the age key is fallback for SOPS-encrypted DR files.

• Client ID + Secret returned in enroll response
• Scoped to paths: /clients/<slug>/*, /shared/*
• Revoked on destroy: identity deleted, access gone

SOPS fallback

The age public key is added to .sops.yaml recipients during enrollment. On destroy, it is removed via oikos secret export-sops regeneration.

bootstrap.sh changes

- # calls https://secrets.hubris.network/issue (Python, dead)
- AGE_KEY=$(curl -s -X POST "$ISSUANCE_URL" ...)
- 
+ # calls Oikos API enrollment endpoint
+ ENROLL_RESP=$(curl -s -X POST "$OIKOS_URL/api/v1/clients/enroll" \
+   -H "Content-Type: application/json" \
+   -d "{\"slug\":\"ws:$HNAME\",\"hostname\":\"$HNAME\",\"mesh_ip\":\"$MESH_IP\"}")
+ AGE_PRIVKEY=$(echo "$ENROLL_RESP" | jq -r '.age_private_key')

DB integration

New migration

012_client_enrollment.up.sql:

-- No new tables needed — entities table already holds clients.
-- Add enrollment-specific attributes validation via check constraints
-- or application-level validation.

-- Enforce slug format for machine entities
-- ws:<hostname> for workstations, host:<hostname> for servers
-- (application-level validation in Go, not a DB constraint)

-- Add index for slug-based client lookups
CREATE INDEX IF NOT EXISTS idx_entities_slug_type
  ON entities (slug, type)
  WHERE type IN ('workstation', 'standalone-server', 'proxmox-host');

Entity attributes schema (for machine types)

{
  "cpu_arch": "arm64",
  "ram_gb": 16,
  "os": "macos",
  "lan_ip": "192.168.8.175",
  "mesh": {
    "netbird": {"ip": "100.122.x.x", "fqdn": "hostname.netbird.selfhosted"}
  },
  "ssh": {"user": "dtoro"},
  "age_pubkey": "age1...",
  "enrolled_at": "2026-07-07T12:00:00Z",
  "enrolled_by": "ws:mac-mini",
  "infisical_identity_id": "identity_abc123"
}

All attributes are stored in the attributes JSONB column on the entities table. Validation happens at the application layer (Go) using the schema defined in seeds/ontology.yaml.

MCP tools to add

These are documented in AGENTS.md section 3 but not implemented in the Go MCP server. Implementation: register in internal/mcp/server.go.

Tool	Input	Output	Implementation
whoami	hostname	Entity record, peers, accessible secrets, health	DB lookup by slug derived from hostname
list_my_secrets	caller_pubkey?	Secrets this client can decrypt	Infisical list + SOPS .sops.yaml match
explain	service_slug	Compact context card: type, state, health, relations, last change	DB join: entity + entity_status + audit_log
preflight	service_slug	Risk class, approval requirement, verification command	Policy classifier on the entity's type
get_change_history	entity_slug, limit	Last N audit_log entries for entity	DB query on audit_log table
get_state_snapshot	none	Last scheduler Observe pass: health, disk, drift count	DB query on entity_status + signals

API endpoints to add

Add to api/openapi.yaml, regenerate with make generate, implement in internal/httpapi/impl.go.

Method	Path	Scope	Purpose
POST	/api/v1/clients/enroll	agent	Issue age key, validate mesh, set state → provisioning
POST	/api/v1/clients/{slug}/activate	operator	Run transition checks, state → active
POST	/api/v1/clients/{slug}/deprecate	operator	State → deprecated
POST	/api/v1/clients/{slug}/destroy	operator	Run destroy checks, revoke secrets, state → destroyed
POST	/api/v1/clients/{slug}/fail	operator	State → failed with reason
GET	/api/v1/clients/{slug}/secrets	agent	List secrets this client can access

Files changed

File	Change
bootstrap.sh	Replace secrets.hubris.network/issue call with POST /api/v1/clients/enroll. Remove dead symlinks.
api/openapi.yaml	Add client enrollment, lifecycle, and secret endpoints
internal/httpapi/impl.go	Implement client lifecycle handlers
internal/db/queries/clients.sql	Add client-specific sqlc queries
internal/mcp/server.go	Register whoami, explain, preflight, get_change_history, get_state_snapshot, list_my_secrets
internal/secrets/infisical.go	Add CreateMachineIdentity, DeleteMachineIdentity, StoreClientKey
internal/ontology/validate.go	Implement lifecycle transition checks for infrastructure lifecycle
seeds/ontology.yaml	Add client-specific attributes schema for machine types
migrations/012_client_enrollment.up.sql	Index for slug+type lookups
AGENTS.md	Update MCP tool list to match actual implementation
CLIENTS.md	Update enrollment flow to reference Oikos API, not Python issuance
CONTRIBUTING.md	Add client lifecycle as a documented extension point

Files deleted or deprecated

File	Disposition
archive/secrets-issuance/	Already archived. Add deprecation notice referencing this plan.
archive/secrets-sops-backup/	Keep for DR. Add note that new clients use Infisical, SOPS is fallback.
Any reference to bin/homelab	Delete or comment out in bootstrap.sh; CLI doesn't exist.
tools/*.setup.sh references	Either create the files or remove the auto-setup convention from post-pull.sh.

Phased implementation

Phase 1 — API + DB (P0, this week)

1. Write migrations/012_client_enrollment.up.sql
2. Add client endpoints to api/openapi.yaml
3. Run make generate
4. Implement enrollment handler (POST /api/v1/clients/enroll):
   • Age key generation
   • Infisical machine identity creation
   • Entity attribute update
   • Audit log write
5. Implement lifecycle transition handlers (activate, deprecate, destroy, fail)
6. Implement GET /api/v1/clients/{slug}/secrets
7. Update seeds/ontology.yaml with client attribute schemas
8. Add sqlc queries in internal/db/queries/clients.sql

Phase 2 — MCP tools (P1, next week)

1. Register whoami(hostname) in internal/mcp/server.go
2. Register explain(service) — compact context card from DB
3. Register preflight(service) — risk classification
4. Register get_change_history(entity, limit)
5. Register get_state_snapshot()
6. Register list_my_secrets(caller_pubkey?)

Phase 3 — Bootstrap script cleanup (P1, next week)

1. Replace secrets issuance URL with Oikos API endpoint
2. Remove --no-secrets / --no-mesh or rewire them to degraded modes
3. Remove bin/homelab symlink
4. Update Infisical identity file creation
5. Test full enrollment on a fresh machine

Phase 4 — Transition check enforcement (P2, within 2 weeks)

1. Implement all provisioning → active checks in internal/ontology/validate.go
2. Implement all deprecated → destroyed checks
3. Wire checks into lifecycle transition handlers
4. Test that POST /activate fails when checks don't pass
5. Test that POST /destroy fails when inbound edges exist

Phase 5 — Cleanup (P2, within 2 weeks)

1. Delete or comment-out dead code in bootstrap.sh
2. Recreate tools/setup-caveman.sh and tools/setup-hermes-soul.sh (or remove references)
3. Update AGENTS.md MCP tool list
4. Update CLIENTS.md enrollment flow
5. Archive Python secrets-issuance with final deprecation note
6. Run make generate-check and full test suite

Verification

• Fresh machine with no prior state: curl bootstrap.sh | sudo bash → machine shows up in DB as provisioning with age pubkey, Infisical identity, and sync timer running
• POST /api/v1/clients/ws:test-machine/activate → state → active, all checks pass
• POST /api/v1/clients/ws:test-machine/deprecate → state → deprecated
• POST /api/v1/clients/ws:test-machine/destroy → fails if edges exist; succeeds after edges removed, secrets revoked
• MCP whoami(ws:test-machine) returns client record with peers and health
• MCP explain(service:caddy) returns context card with relations and risk class
• GET /api/v1/clients/ws:test-machine/secrets returns secrets list scoped to client
• Existing clients continue working through the sync timer (no regression)
• make test test-db generate-check passes

Related

• 2026-07-07-migrate-bin-homelab-to-go.md — MCP tool completion plan (whoami, explain, preflight)
• seeds/ontology.yaml — lifecycle definitions, entity type hierarchy
• seeds/policy.yaml — risk classes, approval rules
• CLIENTS.md — client onboarding guide (update after this plan)
• bootstrap.sh — current enrollment script (rewrite in Phase 3)

Changelog

• 2026-07-07 — initial plan. Replaces Python secrets-issuance, defines full lifecycle in Go, adds client API endpoints, MCP tools, and Infisical machine identity integration.