dtoro/oikos
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

adr: full entity model — types, relationships, state machines, OODA loop
Some checks failed
ci / build-test (push) Has been cancelled
Details
ci / docker-build (push) Has been cancelled
Details

Browse Source 
Covers:
- 56 entity types with full hierarchy (abstract/concrete, domain, layer)
- 34 relationship types with cardinality and OODA phase mapping
- 88 concrete entity instances with key attributes
- Lifecycle state machines (infrastructure, signal, execution, approval,
  pattern, skill) with which preconditions are code-real vs schema-only
- Sequence diagrams: machine onboarding, OODA loop, thermals query
- What's fully implemented vs schema-defined-but-not-wired
- Database physical schema with FK relationships
- Policy: risk classes, approval rules, per-entity overrides, autonomy
- Blast radius via recursive CTE over depends-on/hosts/routes-to edges
This commit is contained in:
authentik Default Admin
2026-07-08 22:15:11 +02:00
parent ef00e5b8e4
commit 4a5e68bafe
1 changed files with 486 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

486
adr/entity-model.md Normal file
View File

@@ -0,0 +1,486 @@
# Oikos Entity Model — Types, Relationships & Interactions
**Status:** Adopted
**Date:** 2026-07-08
**Scope:** Full inventory of every entity type, relationship, state machine, and
cognition pipeline — with clear markers for what is **code-real** vs **schema-only**.
---
## 1. Entity Type Hierarchy (56 types)
```
layer: meta
entity ★ (abstract root)
layer: infrastructure ──────────────────────────────────────────────────
domain: physical
site ups sensor peripheral
domain: compute
compute-entity ★ (abstract)
machine ★ (abstract)
proxmox-host standalone-server workstation appliance
vm
container ★ (abstract)
lxc docker-container
hypervisor
domain: network
network ★ (abstract)
lan mesh vlan
network-interface dns-zone dns-record
ingress-route certificate firewall-rule
domain: storage
storage-pool volume backup-target dataset
domain: software
service application config-repo deploy-pipeline
package-set cluster compose-stack
domain: external
domain-registration cloud-service isp-link vendor-dependency
layer: governance ──────────────────────────────────────────────────────
domain: identity
person agent identity-provider account
secret key access-grant
layer: cognition ── the OODA loop ──────────────────────────────────────
domain: cognition
check signal classification execution feedback
pattern skill approval
document runbook investigation
★ = abstract (cannot be instantiated; acts as polymorphic target for relationships)
```
### Concrete instances (88 active entities)
| Type | Count | Examples |
|------|-------|---------|
| `lxc` | 19 | jellyfin, caddy, dns, gitea, nextcloud, matrix, arriman… |
| `service` | 25 | caddy, authentik, dns, jellyfin, paperless, matrix… |
| `ingress-route` | 21 | *.hubris.network |
| `config-repo` | 6 | caddy-conf, gitea-customizations, mule-image… |
| `proxmox-host` | 2 | hubris, strong |
| `workstation` | 2 | mac-mini, republic-laptop |
| `standalone-server` | 1 | netbird-vps |
| `vm` | 2 | zimaos, haos |
| `storage-pool` | 3 | local-lvm-hubris, library-hubris, ludo-lvm |
| `volume` | 2 | library, media-local |
| + sites, networks, agents, documents, destroyed… | | |
---
## 2. Core Sequence: Machine Onboarding
```
Operator Oikos API DB Scheduler Target Machine
┌────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ POST │ │ │ │ │ │ │ │ │
│/entities│────>│Create │ │ │ │ │ │ │
│ │ │Entity() │ │ │ │ │ │ │
│ │ │ │────>│INSERT │ │ │ │ │
│ │ │ │ │entities │ │ │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ensure │ │ │ │ │ │ │
│ │ │Default │────>│INSERT │ │ │ │ │
│ │ │Checks() │ │check_defs│ │ │ │ │
│ │ │→ ping │ │×6 │ │ │ │ │
│ │ │→ cpu │ │ │ │ │ │ │
│ │ │→ memory │ │ │ │ │ │ │
│ │ │→ load │ │(target_id│ │ │ │ │
│ │ │→ disk │ │ set) │ │ │ │ │
│ │ │→ updates │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │
│ │<────│201 │ │ │ │ │ │ │
│ │ │Created │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │
│ │ │ │ │ │ ── 30s tick ─>│ │ │
│ │ │ │ │ │ loads │ │ │
│ │ │ │ │ │ check_defs │ │ │
│ │ │ │ │ │ │──SSH────>│ │
│ │ │ │ │ │ │ /opt/ │ │
│ │ │ │ │ │ │ oikos/ │ │
│ │ │ │ │ │ │ checks/ │ │
│ │ │ │ │ │ │ cpu.sh │ │
│ │ │ │ │ │ │<──JSON───│ │
│ │ │ │ │<─────────│INSERT │ │ │
│ │ │ │ │metric │metric_samples │ │ │
│ │ │ │ │samples │ │ │ │
│ │ │ │ │ │ │ │ │
│ │ │ │ │<─────────│UPSERT │ │ │
│ │ │ │ │entity │entity_status │ │ │
│ │ │ │ │status │(health) │ │ │
│ │ │ │ │ │ │ │ │
```
**What's code-real here:**
- `CreateEntity()` at `internal/httpapi/impl.go:811` — handles POST, validates type, calls `ensureDefaultChecks()`
- `ensureDefaultChecks()` → `internal/checkdefaults/defaults.go:144` — resolves host IP, SSH user, creates 6 check_defs rows with target_id
- Scheduler at `internal/scheduler/scheduler.go:26` — loads `ListEnabledCheckDefs`, dispatches by kind, writes metrics + signals
---
## 3. Core Sequence: The OODA Loop (observe → orient → decide → act)
```
┌─────────────────────────────────────────────────────────────────────┐
│ OBSERVE (Scheduler) │
│ │
│ Every 30s: │
│ ┌──────────┐ ListEnabledCheckDefs ┌──────────┐ │
│ │scheduler │─────────────────────────>│ Postgres │ │
│ │.go:54 │ │ │ │
│ └──────────┘ └──────────┘ │
│ │ │
│ ├── ping ──> exec.Command("ping", host) │
│ ├── http ──> http.Get(url) │
│ ├── tcp ──> net.DialTimeout("tcp", addr) │
│ ├── disk ──> unix.Statfs(path) │
│ ├── cert-expiry ──> tls.Dial + cert.NotAfter │
│ └── ssh-script ──> exec.Command("ssh", host, script) │
│ │ │
│ ┌───────┘ │
│ ▼ │
│ ┌─────────────┐ │
│ │ checkResult │ {health, signalKind, evidence, metrics}│
│ └─────────────┘ │
│ │ │
│ ┌──────────┼──────────┐ │
│ ▼ ▼ ▼ │
│ metric_samples signals entity_status │
│ INSERT UPSERT UPSERT │
│ (every cycle) (dedup by (health + last_check_at) │
│ target+kind) │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ ORIENT (Classification) │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ For each open signal: │ │
│ │ │ │
│ │ classify_by_policy(signal, entity, blast_radius) │ │
│ │ │ │ │
│ │ ├── read_only ───────────> route: auto_act │ │
│ │ ├── reversible_low ──────> route: auto_act │ │
│ │ │ (if global.auto_act=on + not in never_auto_act)│ │
│ │ ├── config_mutation ─────> route: escalate │ │
│ │ └── destructive ─────────> route: hold │ │
│ │ │ │
│ │ INSERT INTO classifications │ │
│ │ edge: classifies → signal │ │
│ └──────────────────────────────────────────────────────┘ │
│ │
│ ⚠ classification creation: schema defined, NOT yet wired │
│ (policy.ClassifySignal exists but scheduler doesn't call it) │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ DECIDE (Approval Gate) │
│ │
│ For route=auto_act: │
│ skip approval, execute immediately │
│ │
│ For route=escalate (config_mutation): │
│ POST /api/v1/executions ──> INSERT approval (status=pending) │
│ notifier.go sends Matrix alert with HMAC token │
│ operator replies ✅ or ❌ │
│ DecideApproval() → systemctl restart / apt upgrade │
│ │
│ For route=hold (destructive): │
│ queued for operator, requires explicit confirmation │
│ (never auto-executed even with global.auto_act=on) │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ ACT (Execution) │
│ │
│ ┌──────────┐ request_execution ┌──────────┐ │
│ │ Nomos │───────────────────────>│ MCP tool │ │
│ │ (agent) │ │ server.go │ │
│ └──────────┘ └──────────┘ │
│ │ │
│ ┌───────────┼───────────┐ │
│ ▼ ▼ ▼ │
│ reversible config_ destructive │
│ _low mutation │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ immediate approval hold │
│ execute queue (never auto) │
│ │ │ │
│ ▼ ▼ │
│ actuator. Matrix │
│ Execute() alert → │
│ (SSH exec) operator │
│ → approves │
│ → actuator.Execute() │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ LEARN (Patterns + Skills) │
│ │
│ execution ──produces──> feedback ──contributes-to──> pattern │
│ │ │
│ informs │
│ ▼ │
│ skill │
│ │
│ ⚠ Schema defined, NOT yet wired: │
│ - No code writes feedback records │
│ - No code transitions patterns hypothesized→validated │
│ - Skill execution against JSON procedure definitions not built │
└─────────────────────────────────────────────────────────────────────┘
```
### What's code-real in the OODA loop
| Phase | Table | Code | Status |
|-------|-------|------|--------|
| Observe | `check_defs`, `metric_samples` | `scheduler.go:26-215` | ✅ fully wired, 6 probe kinds |
| Observe → Orient | `signals` | `scheduler.go:130-141` (UpsertSignal) | ✅ dedup, severity, events |
| Orient | `classifications` | `policy/classify.go` (function exists) | ⚠ function defined but scheduler never calls it |
| Decide | `approvals` | `server.go:311-367` (request_execution) | ✅ escalation gate works |
| Act | `executions` | `actuator/exec.go` (SSH exec) | ✅ systemctl, apt, pct |
| Learn | `feedback`, `patterns`, `skills` | tables + list endpoints only | ⚠ schema only, no write path |
---
## 4. Relationship Types — The Edge Catalog (34 edges)
### Infrastructure Topology
```
host:hubris ──hosts──> lxc:jellyfin, lxc:caddy, lxc:dns, ... (machine provisions LXCs)
host:strong ──hosts──> lxc:jellyfin, lxc:arriman, ... (migrated LXCs)
host:hubris ──member-of──> cluster:homelab
host:strong ──member-of──> cluster:homelab
lxc:caddy ──provides──> service:caddy
lxc:gitea ──provides──> service:gitea
lxc:dns ──provides──> service:dns
host:hubris ──mounts──> volume:library (attrs: mount_point=/mnt/library)
host:hubris ──stores-on──> pool:library-hubris
```
### Network
```
ingress:paperless.hubris.network ──routes-to──> service:paperless
ingress:paperless.hubris.network ──secured-by──> idp:authentik
ingress:paperless.hubris.network ──uses-certificate──> cert:*.hubris.network
service:jellyfin ──authenticates-via──> idp:authentik (OIDC)
dns:paperless ──in-zone──> zone:hubris.network
dns:paperless ──resolves-to──> lxc:caddy (caddy terminates)
host:hubris ──connects-via──> lan:lab
host:strong ──connects-via──> lan:household
```
### Service Dependencies
```
service:jellyfin ──depends-on──> service:authentik (OIDC auth)
service:paperless ──depends-on──> service:authentik
service:arr-stack ──depends-on──> service:jellyfin
(depends-on edges feed blast_radius() — recursive CTE)
```
### Cognition (OODA edges)
```
check:ssh-script:d419257d ──checks──> host:hubris
check:ssh-script:d419257d ──raises──> signal:cpu-pressure (when unhealthy)
signal:cpu-pressure ──about──> host:hubris
classification:xyz ──classifies──> signal:cpu-pressure
classification:xyz ──precedes──> execution:restart-xyz
execution:restart-xyz ──targets──> host:hubris
execution:restart-xyz ──performs──> agent:nomos
```
### Governance
```
person:dtoro ──owns──> agent:nomos
person:dtoro ──decides──> approval:xyz
idp:authentik ──authenticates──> person:dtoro
```
---
## 5. Lifecycle State Machines
### Infrastructure (15 concrete types use this)
```
planned ──> provisioning ──> active ──> migrating ──> active
│ │ │ │
│ │ └── failed ──┘
│ │ └── deprecated ──> destroyed
│ │
│ └── failed ──> active (recovery)
│
└── destroyed (cancelled)
Terminal: [destroyed]
Default: active
```
**Real precondition checks** (code in `impl.go:1494-1579`):
| Transition | Precondition | How it's checked |
|------------|-------------|-----------------|
| provisioning→active | `health-check-answering` | `SELECT health FROM entity_status WHERE entity_id=$1` — must be healthy |
| provisioning→active | `age-key-enrolled-if-needed` | Checks `attributes->>'age_pubkey'` (workstation only) |
| provisioning→active | `mesh-joined-if-needed` | Checks `attributes->>'mesh_ip'` (workstation only) |
| provisioning→active | `doc-page-complete` | `SELECT count(*) FROM relationships WHERE target_id=$1 AND type='documents'` |
| deprecated→destroyed | `no-inbound-edges` | `SELECT count(*) FROM relationships WHERE target_id=$1 AND valid_to IS NULL` |
| any → terminated | `backups-verified` | Checks flag in entity attributes |
| any → terminated | `secrets-revoked` | Checks flag in entity attributes |
**Soft preconditions** (always pass — operator-confirmed): `inventory-entry`, `ip-reserved`, `preflight-passed`, `backup-verified`, `replacement-live`, `caddy-backends-checked`, `un-deprecate-note`, etc.
### Signal
```
raised ──> acknowledged ──> acting ──> resolved
│ │ │
├── muted ├── muted ├── raised (retry budget)
│ │ │
└── resolved└── resolved └── failed ──> acknowledged (operator-retry)
Terminal: [resolved]
Default: raised
```
**Implemented preconditions:**
- `raised → muted`: requires `mute_until` set (MuteSignal handler, `impl.go:615-689`)
- `acting → resolved`: requires `verification-passed` (soft — operator confirms)
**Dedup mechanism:** `UNIQUE INDEX uq_signals_open ON signals(target_entity_id, kind) WHERE state NOT IN ('resolved','failed')` — at most one open signal per (entity, kind). Repeated failures call `UpsertSignal` which increments `occurrence_count` on the existing row.
---
## 6. What's Code-Real vs Schema-Only
### ✅ Fully Implemented (code exists, running in production)
| Component | File(s) | What it does |
|-----------|---------|-------------|
| Entity CRUD | `impl.go:811-966` | Create, read, patch, list entities |
| Lifecycle transitions | `impl.go:1494-1579` | Precondition checks + state transitions |
| Relationship management | `seed.go` (ingest) | Create edges with `valid_from/valid_to` |
| Client enrollment | `impl.go:1134-1236` | `POST /clients/enroll` — age keypair, Infisical, state: provisioning |
| Check definitions | `phase3.go:265-376` | CreateCheck, ListChecks, PatchCheck |
| Scheduler observe | `scheduler.go:26-215` | 6 probe kinds, metric_samples, signals, entity_status |
| Signals | `scheduler.go:81-161` | UpsertSignal (dedup), ResolveSignal, severity evaluation |
| Executions | `server.go:286-411` | request_execution MCP tool — reversible_low/config_mutation/destructive |
| Approvals | `server.go:970-1052` | createApproval, DecideApproval → executeApprovedAction |
| Notifier | `notifier/notifier.go` | Matrix alerts for pending approvals |
| Patterns | `phase3.go:1100+` | ListPatterns, PatchPattern (status/quarantine) |
| Skills | `phase3.go:1300+` | ListSkills, PatchSkill, ListSkillVersions |
| Default checks | `checkdefaults/defaults.go` | Auto-create checks on entity creation/enrollment/seed |
| TimescaleDB metrics | `metric_samples` table | Hypertable with 1h/1d continuous aggregates, 90-day retention |
| Events + SSE | `events` table + pg_notify | Real-time UI updates via SSE endpoint |
| Audit log | `audit_log` hypertable | Every mutation with actor + action |
| Knowledge entities | `knowledge_entities` | Documents, runbooks, investigations with FTS |
| MCP tools | `server.go` | 24 tools for observe/orient/decide/act |
| Blast radius | `blast_radius()` fn | Recursive CTE — depends-on + hosts + routes-to edges |
### ⚠ Schema Defined, Not Yet Wired (table exists, no active code path creates rows)
| Component | What's Missing |
|-----------|---------------|
| `classifications` auto-creation | `policy.ClassifySignal()` exists but scheduler never calls it. Signals are raised but never automatically classified. The `GetOpenSignalsForAutoAct` query would return signals with auto-act classification, but the classify step is manual-only. |
| `feedback` records | No code writes to the `feedback` table. Execution results are not analyzed for patterns. |
| Pattern auto-learning | No code transitions patterns from `hypothesized → validated`. The lifecycle requires `evidence-count≥5 + confidence≥0.7` but no aggregation runs. |
| Skill execution | Skill entities carry a JSON `procedure` field but no execution engine reads or runs it. |
| `drift` check kind | Defined in OpenAPI and `check_defs.kind` enum, but no scheduler implementation exists. |
### 📋 Defined in Seeds Only (ontology.yaml references, no DB schema)
| Item | Notes |
|------|-------|
| Relationship type `cluster` | Mentioned in inventory but not in ontology relationship_types |
| `certificate` entity type | Referenced in `uses-certificate` edges but no concrete certificates in inventory |
| Relationship type `powers` / `monitors` | Not defined in relationship_types |
---
## 7. Database Physical Schema (Key Tables)
```
entity_types ──FK──> lifecycle_defs
│
│ FK (entities.type)
▼
entities ──FK──> entity_types
│
├──FK──> entity_status (dual)
├──FK──> check_defs (dual; check_defs.target_id → entities)
├──FK──> signals (dual; signals.target_entity_id → entities)
├──FK──> classifications (dual)
├──FK──> executions (dual; executions.target_entity_id → entities)
├──FK──> feedback (dual)
├──FK──> patterns (dual)
├──FK──> skills (dual)
├──FK──> approvals (dual; approvals.subject_entity_id → entities)
├──FK──> knowledge_entities (dual)
└──>→ relationships (source_id, target_id → entities)
relationship_types ──FK──> entity_types (source_type, target_type)
│
│ FK (relationships.type)
▼
relationships ──FK──> entities (source_id, target_id)
│
└── unique index: (source_id, target_id, type) WHERE valid_to IS NULL
approval_rules ──FK──> entity_types (entity_type)
autonomy_settings (key/value, no FKs)
risk_classes (standalone)
metric_samples (TimescaleDB hypertable — ts dimension)
events (TimescaleDB hypertable — ts dimension, pg_notify trigger for SSE)
audit_log (TimescaleDB hypertable — ts dimension)
agent_activity (TimescaleDB hypertable — ts dimension)
```
**Key architectural patterns:**
- **Dual entities:** `check_defs`, `signals`, `classifications`, `executions`, `feedback`, `patterns`, `skills`, `approvals`, `knowledge_entities` — all have `entity_id UUID PK REFERENCES entities(id)`. Every row is also an entity.
- **Partial unique indexes:** `relationships` (current edges), `signals` (open signals), `patterns` (per-type action) — all use `WHERE` clauses for snapshot semantics.
- **TimescaleDB:** 4 hypertables with continuous aggregates and retention policies.
- **SSE fan-out:** `pg_notify('oikos_events', ...)` trigger on `events` INSERT → Go listener fan-out → SSE connections.
---
## 8. How Nomos Queries Thermals — End-to-End Trace
```
User: "what are the thermals of hubris?"
│
▼
Nomos calls MCP: query_metrics(metric=["cpu_pct","cpu_temp"])
│
▼
server.go:getMetricHistory()
│
▼
SELECT time_bucket('1h', ts) AS bucket,
avg(value), min(value), max(value)
FROM metric_samples
WHERE metric IN ('cpu_pct', 'cpu_temp')
AND entity_id = (SELECT id FROM entities WHERE slug = 'host:hubris')
GROUP BY bucket
│
▼
Returns: cpu_pct ≈ 15%, cpu_temp ≈ 48°C (from TimescaleDB continuous aggregate)
│
▼
Nomos formats and presents results to user
```
**What made this possible (chronologically):**
1. `scheduler.go` refactored to return metrics map → `checkResult{metrics}`
2. `ssh-script` check kind implemented → SSH exec to remote host
3. `cpu_check.sh` deployed to hubris → returns `{"metrics":{"cpu_pct":2.5,"cpu_temp":48}}`
4. Check created: `POST /checks {"kind":"ssh-script","target":"host:hubris","config":{"host":"192.168.8.77","script":"cpu_check.sh"}}`
5. Fixed: `InsertMetricSample` missing `ts` column → `now()` literal
6. Fixed: SSH port/user parsing bugs
7. Fixed: SSH warnings polluting JSON output
8. Scheduler loop → metrics written to TimescaleDB every 60s
9. MCP `query_metrics` reads from TimescaleDB → Nomos gets live data