68 Commits

Author SHA1 Message Date
095a3967c4 phase 3: control loop — scheduler, actuator, learning, notifier, policy, API endpoints
Implemented the full OODA control loop:

Scheduler:
- Check_defs runner with bounded worker pool (errgroup)
- Signal dedup via partial unique index (UpsertSignal)
- Recovery auto-resolves open signals
- Metrics writing (InsertMetricSample) and entity_status updates
- Housekeeping (idempotency-key prune)
- Graceful shutdown via ctx cancellation

Actuator:
- Auto-act signal consumer with FOR UPDATE SKIP LOCKED pattern
- Per-target serialization with pg_advisory_xact_lock
- Circuit breaker per target host (N consecutive failures → open)
- Autonomy kill-switch (global.auto_act, never_auto_act.<slug>)
- Execution record lifecycle (proposed → running → completed)

Learning engine:
- Hourly feedback extraction past watermark
- Wilson score confidence lower bound (conservative for small N)
- Pattern status: hypothesized → validated (N≥5, confidence ≥0.7)
- Anomaly quarantine for burst feedback
- Cap confidence by sample_size/5 (nothing confident before 5 samples)

Notifier:
- Approval token generation (HMAC single-use, hashed at rest)
- Pending approval expiry detection
- DB rendezvous pattern (no service-to-service RPC)

Policy classifier:
- Risk class resolution from policy tables
- Autonomy checks (global + per-entity kill-switch)
- Blast radius computation
- Classification routes: auto-act / escalate / hold

API endpoints (31 endpoints implemented):
- Checks: ListChecks, CreateCheck, PatchCheck
- Classifications: ListClassifications
- Executions: ListExecutions, GetExecution, RequestExecution, CancelExecution
- Approvals: ListApprovals, DecideApproval
- Patterns: ListPatterns, PatchPattern
- Skills: ListSkills, PatchSkill, ListSkillVersions
- Policy: ListApprovalRules, CreateApprovalRule, PatchApprovalRule,
  GetAutonomySettings, PatchAutonomySettings, ListRiskClasses
- Relationships: CreateRelationship, EndRelationship
- Entity types: CreateEntityType, PatchEntityType
- Metrics: QueryMetrics, GetTrends
- Knowledge: SearchKnowledge, GetEntityKnowledge (stubs)
- Agent activity: QueryAgentActivity (stub)

Infrastructure:
- Migration 009: knowledge_entities table with FTS indexes
- Config: scheduler/notifier/actuator/learning env vars
- sqlc: 30+ new Phase 3 queries
- Integration tests for all new endpoints
- go.sum updated with golang.org/x/sync
2026-07-07 15:19:25 +02:00
7e802bbb14 sse: real-time flushing via raw handler overriding the generated route
The generated strict-server path could only return an io.Reader that
io.Copy drains without flushing, so SSE events sat chunk-buffered instead
of streaming in real time. Replace it with a raw http.ResponseWriter
handler (serveSSE) that Flush()es after every event.

Routing: chi allows a later registration to supersede an earlier one for
the same method+path (verified empirically for v5.3.1), so serveSSE is
registered on the router AFTER gen.HandlerWithOptions and wins over the
generated /events/stream route. It inherits the base middleware chain and
applies auth via With(). The generated StreamEvents method now returns an
error (never reached) so a routing regression fails loudly rather than
silently reverting to buffered delivery.

Adds TestSSEStreamRealtimeDelivery: a real httptest.NewServer + streaming
client (NewRecorder can't flush) that connects, triggers an event, and
asserts delivery within 3s — proving both the override routing and
per-event flushing. Passes in <1s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:34:32 +02:00
f1b0b65149 phase 2 review: fix SSE deadlock, MCP panic, lifecycle 500, NOT NULL bug
Reviewed the phase-2 implementation (parts 2–5) end to end. The suite hung
for 600s and several handlers were never exercised because there were no
tests for the new mutation/event/MCP surface. Fixes:

- CRITICAL: sseListener ran on context.Background() and held a pooled
  connection forever, so pool.Close() deadlocked (600s test timeout).
  NewHandler now takes a ctx that governs the listener; ListenAndServe and
  the test helper cancel it before closing the pool.
- CRITICAL: MCP AddTool panicked ("missing input schema") at construction
  under go-sdk v1.6.1 — so NewHandler (and every API handler) panicked.
  Added object input schemas to all 8 tools via an objSchema helper.
- HIGH: PatchEntity parsed lifecycle transitions as map[string][]string but
  the shape is {from:{to:{requires:[]}}}, so every state-change PATCH 500'd.
  Parse the nested shape; allow same-state no-ops.
- HIGH: CreateEntity bound SQL NULL for attributes when omitted, violating
  the NOT NULL column (the default only applies when omitted). Default to
  '{}'.
- MED: serveSSEWriter ignored the request ctx (per-client goroutine leak on
  disconnect) and set an invalid Content-Length: -1. Thread ctx through;
  omit the header. writeSSE now nil-checks the flusher (io.Pipe path passed
  nil → would have panicked on first event).
- MED: SSE `data:` leaked raw sqlcgen.Event (PascalCase, base64 JSONB).
  Emit canonical gen.Event so SSE matches GET /events. Verified live.
- LOW: CreateEntity uses uuid.NewV7 (ADR-0005) + real actor from context in
  audit; removed dead bearerAuth; fixed vet unkeyed-field warnings.

Tests (would have caught all of the above): entity create/patch with
If-Match 409/400, valid+invalid lifecycle transitions, idempotency replay,
duplicate-slug 409, abstract-type 422, event+audit side effects, MCP tool
registration. Live smoke test confirmed NOTIFY→listener→SSE delivery.

Also adds the missing Phase 2 deliverable: Gitea Actions CI (vet,
golangci-lint, govulncheck, generated-code drift guard, race tests against
TimescaleDB, docker build) and wires sqlc into `make generate`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:26:43 +02:00
6b61495e3b phase 2 (part 5): MCP server with official Go SDK
- MCP server at /mcp using github.com/modelcontextprotocol/go-sdk v1.6.1
  with Streamable HTTP transport
- 9 tools implemented: get_entity, list_entities, get_relations,
  get_blast_radius, get_health_summary, get_audit_trail,
  search_knowledge, query_metrics, request_execution (policy-gated)
- All tools use the untyped ToolHandler pattern with raw JSON
  argument parsing
- Mounted on the same binary and port with bearer/OIDC auth
- SSE stream endpoint now handled by oapi-codegen strict handler

Phase 2 acceptance criteria:
- GET /entities?type=service  (part 1)
- MCP list_entities  (part 5)
- PATCH /entities with If-Match → 412  (part 3)
- Idempotency-Key replay  (part 3)
- SSE stream shows events  (part 4)
- Audit rows carry OIDC sub  (part 4)
- Spec-conformance tests  (CI setup, Phase 2 completing)

Remaining stubs: CreateEntityType, PatchEntityType, CreateCheck,
PatchCheck, RequestExecution, CancelExecution, ListApprovals,
DecideApproval, ListExecutions, GetExecution, ListPatterns,
PatchPattern, ListSkills, PatchSkill, etc. (Phase 3)
2026-07-07 12:27:36 +02:00
1bfc18ea3a phase 2 (part 4): SSE stream via io.Pipe, OIDC JWT auth middleware
- SSE stream: GET /events/stream using io.Pipe to bridge the SSE
  goroutine to the response body. Replay from Last-Event-ID via
  in-memory broker with DB fallback. LISTEN/NOTIFY fan-out to all
  subscribers. Heartbeat every 15s. Bounded channels.
- OIDC JWT auth: validates Bearer tokens against Authentik/OIDC
  issuer via JWKS discovery + key caching. Extracts sub/email into
  context actor. Falls back to static bearer tokens. Dev mode (no
  OIDC + no tokens) = open.
- Config: OIDCIssuer, OIDCClientID env vars
- SSE + OIDC infrastructure complete, build passes, all tests pass

Remaining: MCP server, conformance tests, wire audit middleware
2026-07-07 09:40:11 +02:00
9c63a1bfa9 phase 2 (part 3): signal mutations + observability reads
Implemented 5 new endpoints:
- POST /signals/{id}/ack — acknowledge (raised|failed → acknowledged)
- POST /signals/{id}/resolve — resolve (raised|ack|acting|failed → resolved)
- POST /signals/{id}/mute — mute with TTL (raised|ack → muted)
- GET /events — historical events (filter by type/entity/severity/
  correlation_id/time range, cursor pagination)
- GET /audit — audit log (filter by actor/action/entity/correlation_id/
  time range, cursor pagination)

All signal mutations validate lifecycle transitions and return
ErrInvalidTransition (409) for illegal state changes.
Removed from stubs.go: AckSignal, ResolveSignal, MuteSignal,
QueryEvents, QueryAudit.

Stubs remaining: 33 endpoints (mutations + remaining reads)
2026-07-07 08:57:40 +02:00
c9975d60a5 phase 2 (part 2): sqlc queries, audit/event helpers, event NOTIFY trigger
- sqlc.yaml + internal/db/queries/*.sql: typed queries for entities,
  relationships, ontology, operations (signals, events, audit,
  idempotency, entity_status)
- internal/db/sqlcgen/: generated Go from sqlc (pgx/v5)
- internal/observability/record.go: Audit() and Event() helpers that
  write in the caller's transaction (SG10). actorLabel is interim text
  identity in detail JSON until OIDC resolution lands; actor_id column
  exists but is not yet populated
- migrations/008: post-commit pg_notify trigger on events table for
  SSE fan-out (SG8/SG10)
2026-07-07 08:49:59 +02:00
f2fe812cda phase 2 (part 1): OpenAPI-generated API server, first 9 endpoints
- api/openapi.yaml converted 3.1 → 3.0.3 (oapi-codegen/kin-openapi
  supports 3.0; nullable syntax + example keywords), still redocly-clean
- oapi-codegen (v2.4.1, strict server + chi) generates
  internal/httpapi/gen from the spec; `make generate` wired
- internal/httpapi: chi router, /healthz (unauthenticated, SG18),
  RFC 9457 problem+json mapping from domain sentinels (SG11), 5xx detail
  logged server-side only, request logging with request IDs, graceful
  shutdown (SG4), interim static bearer auth (constant-time; dev-open
  when no token; OIDC JWT still to come in Phase 2)
- Implemented: listEntities (type filter walks the hierarchy, keyset
  pagination), getEntity (UUID or slug, ETag), getEntityRelations,
  getBlastRadius, getGraph (nodes+edges for UIs), getOntology,
  listSignals, getFleetHealth, exportSeeds. Remaining 38 ops return 501
  problem+json stubs (compiler-enforced interface completeness)
- `oikos api` role live: migrate-on-start, serves :8090
- 15 API integration tests (auth, pagination, hierarchy filter, ETag,
  404/501 problem shapes, graph, export)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:25:19 +02:00
1b04683639 phase 1 review fixes: dedup edges, real export, validation, tests
Review of aa2ca0a found and fixed:
- re-ingest duplicated ALL edges (upsert conflicted on valid_from=now(),
  never fired) — migration 007 dedupes + partial unique index on current
  edges; upsert now targets it. Regression-tested.
- export was a stub that overwrote seeds/*.yaml with 11-byte "version: 1"
  files — implemented real deterministic export (ontology/inventory/policy,
  cognition-layer excluded); round-trip is byte-stable (tested)
- DB password leaked in startup logs (slog JSON bypasses String()) —
  Config now implements slog.LogValuer; regression-tested
- docker-compose had literal '***' as DB password — env-interpolated
- uuid.New() (v4) → uuid.NewV7() per ADR-0005
- no ontology validation on ingest — internal/ontology TypeTree: abstract
  instantiation rejected, relationship endpoints hierarchy-validated,
  cardinality enforced in-transaction, lifecycle states checked, default
  state applied (Phase 1 gate items, R3-1)
- getOrCreateEntityID swallowed non-ErrNoRows errors
- migration runner now holds a session advisory lock on one connection
- Makefile: hardcoded /opt/homebrew/bin/go → go; test-db target

Tests: 4 unit suites + 7 integration tests (env-guarded, throwaway DB per
run): migrate idempotent, seed idempotent + no dup edges, abstract/edge/
cardinality rejection, blast_radius cycle termination, export round-trip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:11:01 +02:00
aa2ca0ae6f phase 1: Go foundation — module, migrations, domain, seed ingest
Core deliverables:
- Go module github.com/dtoro/oikos (Go 1.26.3)
- cmd/oikos: single binary with role subcommands (migrate, seed, export)
- 6 SQL migrations: ontology meta-schema, entity instances (UUID+slug,
  blast_radius recursive function), operations (signals/checks/approvals),
  cognition (classifications/executions/feedback/patterns/skills), policy,
  observability (TimescaleDB hypertables + CAGGs + retention)
- Domain layer: entity, signal, execution, classification, pattern, skill,
  approval, check types + 11 sentinel errors + lifecycle state machines
- DB layer: pgx pool, SQL splitter (handles 94436 and -- comments), migration
  runner, seed ingest (ontology+inventory+policy) with content-hash dedup
- Config: env-based with defaults, secrets redaction
- Observability: slog JSON logger with debug mode
- Infrastructure: Makefile, docker-compose.yml, multi-stage Dockerfile
  (distroless, CGO_ENABLED=0)

Verified end-to-end against timescale/timescaledb:2.17.2-pg16:
- 6 migrations applied (65 SQL statements)
- Seeds ingested: 6 lifecycles, 59 entity types, 46 relationship types,
  111 entities, 144 relationships, 4 risk classes, 27 approval rules,
  9 autonomy settings
- Idempotent: second seed run is a no-op (content hash matches)

Bugs fixed during implementation:
- TimescaleDB CAGGs can't run in a transaction -> splitSQL() executes
  statements individually
- Semicolons in -- comments treated as separators -> comment handling
- YAML keys source/target didn't match code's source_type/target_type
- yaml.Marshal produced YAML for JSONB columns -> json.Marshal
2026-07-07 01:07:26 +02:00
55710bd254 seeds: resolve inventory thin spots against production
- lxc:rclone verified live as LXC 132 on hubris (pct list via MCP);
  hosts edge added
- /mnt/library backing storage identified from hosts/hubris.md: 'library'
  lvmthin pool, 3.7T, 2nd NVMe — added pool:library-hubris + contains edge
- all 8 derived services (books/seanime/roms/house/jellyseerr/qbit/sab/
  teddy) confirmed responding over their ingress URLs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:26:18 +02:00
18cb79caf9 oikos phase 0: ontology + inventory + policy seeds, OpenAPI contract, ADRs
- seeds/ontology.yaml: 59 entity types (5 abstract, is-a hierarchy), 46
  relationship types with cardinality, 6 lifecycles with terminal states
  and named precondition checks
- seeds/inventory.yaml: 110 entities / 142 relationships translated from
  legacy inventory.yaml (fleet, services, ingress, storage, governance,
  archaeology); thin spots marked for backfill
- seeds/policy.yaml: 4 risk classes, 27 approval rules (hierarchy-aware,
  per-entity overrides), autonomy kill-switch off (cold start)
- api/openapi.yaml: full v1 REST contract (40 paths), RFC 9457 errors,
  cursor pagination, idempotency, ETag/If-Match, scopes; redocly-clean
- docs/adr/0001-0010: initial architecture decision records
- scripts/validate-seeds.py: Phase 0 gate — hierarchy, lifecycles,
  endpoints, cardinality, policy cross-refs (0 errors)
- plan: layer CHECK gains 'meta' (root type), cardinality gains
  'many-to-one'

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:17:15 +02:00
ea3b2c3662 plans: oikos rev 3 — consolidated spec, ontology inheritance, OpenAPI-first
Merge the rev-2 audit + remediation layers into one self-consistent spec and
close new gaps: meta-schema inheritance (parent_type/is_abstract), contract-
first API (RFC 9457, idempotency, ETag, scopes, /graph), single-binary role
packaging, UUIDv7+slug IDs, checks-as-data, signal dedup/flap/maintenance,
executable skill format, MCP streamable HTTP, SSE events, ledger-as-view,
dual-path networking (mesh-primary + LAN break-glass), per-phase acceptance
criteria, ADRs. Appendix A maps every rev-2 finding to its resolution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:03:25 +02:00
8850b85325 plans: remediate all HIGH audit items + architect/developer review
Addresses 15 original HIGH audit findings + 18 new findings from
systems architect + senior Go developer review (572 lines added).

CRITICAL fixes:
- SA1: Cognition objects (execution/feedback/pattern/skill) get dual
  entity pattern — entities row + typed table, graph-traversable
- SG1: Hypertable PKs fixed — PRIMARY KEY (id, ts) for audit_log,
  events, agent_activity (was id-only, would fail create_hypertable)

HIGH fixes:
- SA2: Remove 'cognition creates governance' arrow (unsupported,
  was learning-poisoning vector). Patterns propose, operator accepts.
- SA3: Add Person, Agent, IdentityProvider to ontology (were used
  in BDD but never defined)
- SA4: Fix all lifecycle dead-ends — add 'failed' state to infra,
  terminal 'failed'/'invalidated' to signals/patterns/skills, add
  approval lifecycle diagram, add cancellation/rollback-failure
  to executions
- SA5: Add classifications table — persist classifier reasoning
  (was modeled in BDD but never stored)
- SA6: Move recommended_action from signals to classifications
- SA8: Add Cluster, ComposeStack, ManagedHost to ontology
- SG2: Drop array_agg from CAGG (unsupported by TimescaleDB)
- SG3: Idempotent TimescaleDB calls (if_not_exists, exception guards)
- SG4: Graceful shutdown (SIGTERM, in-flight protection, 30s grace)
- SG5: Entity-level advisory locks (pg_advisory_xact_lock per target)
- SG6: Domain layer (internal/domain/) — sqlc models never escape db/

Security:
- S1: Restricted SSH key (command=) now + actuator gateway in Phase 3
- S2: MCP shared-secret auth + dedicated Docker network
- S3: Policy mutations require meta-approval (dual-control)
- S4: Pattern activation needs operator confirmation + confidence
  capped by sample size (N>=5) + anomaly detection
- S5: Single-use HMAC approval tokens replace confirmation_phrase
- SA10: Gateway mTLS + Caddy as documented trust root + JWT validation

Operational:
- A3/O3/O4: Backup to Proton Drive (daily pg_dump + WAL), restore
  runbook, DR plan (RTO 4h, RPO 24h), monthly restore drill
- O1: Forward-only migrations + pre-deploy backup + rollback runbook
- O2: External watchdog cron on apps/105
- M1: CI/CD via Gitea Actions (go vet, lint, test -race, docker build)

Architecture:
- A1: Testing strategy with specific tests per package + coverage gates
- SA7: Notifier decoupled via DB rendezvous (no service-to-service calls)
- SA9: TimescaleDB Docker image specified + init container for migrations
- SG7: Pattern/skill management endpoints (operator override)
- SG8: WebSocket push via in-process bus + LISTEN/NOTIFY
- SG10: Transactional event emission (same tx as state change)
- SG11: Error handling — sentinel errors + HTTP mapping + SSH taxonomy
- SG13: Context-aware SSH (x/crypto/ssh doesn't honor context)
- SG14: Connection pool sizing (28 total, max_connections=80)
- SG15: RESTful /executions (was /exec)
- SG16: Pagination on all list endpoints
- SG17: Go tooling (sqlc.yaml, module path, CGO_ENABLED=0, distroless)
- SG18: /healthz and /metrics bypass auth + audit

Updated phasing incorporates all remediation.
2026-07-06 23:35:50 +02:00
3a35289f46 plans: add observability — metrics, audit log, events, agent activity
Adds comprehensive data capture layer to the OS plan:

1. TimescaleDB — PostgreSQL extension for time-series data. No separate
   database. Hypertables auto-partition, continuous aggregates provide
   1h/1d rollups, retention policies auto-drop old data.

2. Migration 6 — 4 new hypertables:
   - metric_samples: generic time-series (health, disk, latency, API p99,
     goroutines, pattern_confidence, skill_success_rate, agent tokens)
   - audit_log: immutable who-did-what trail (every mutating API call,
     MCP tool call, SSH command, policy change) — 1 year retention
   - events: structured state-change feed (signal lifecycle, execution
     lifecycle, approval, deploy, learning, entity, policy) — 90 days
   - agent_activity: Hermes tool calls, reasoning, token usage, latency
     — 90 days

3. Correlation IDs — propagated through the full call chain (signal →
   classification → execution → SSH → verification → feedback → pattern)
   so any action chain can be reconstructed end-to-end.

4. 6 new MCP tools for agent self-query:
   query_metrics, get_trend, get_audit_trail, get_event_timeline,
   get_agent_activity, get_health_summary

5. 8 new REST endpoints for metrics/audit/events/health/trends/export

6. Workstream 14 — observability + data capture (7 sub-components A-G):
   metrics, audit, events, agent activity, structured logging, agent
   data availability, future visualization plug-in points

7. New Mermaid diagram — observability data capture and query flow

8. Updated architecture diagram to show observability data flows

9. Updated phasing — observability woven into phases 1-3

10. Updated verification — 14 end-to-end checks (was 12), including
    metrics querying, audit trail, correlation tracing
2026-07-06 23:12:34 +02:00
2d75544362 plans: SysML BDD ontology, generic model, full audit
Ontology rewrite:
- Replace Mermaid ER diagram with SysML Block Definition Diagrams (BDD)
  using class diagram syntax: generalization, composition, aggregation,
  association with multiplicity annotations
- Split into 3 diagrams: infrastructure (compute/storage/network),
  software+services, cognition (operations+learning)
- Make compute model generic: ComputeEntity abstract base with
  specializations (Machine, VirtualMachine, Container→LXC/DockerContainer;
  Machine→ProxmoxHost/StandaloneServer/Workstation/Appliance)
- Hypervisor is software on a Machine (not all machines are Proxmox)
- Any ComputeEntity can mount Volumes (VMs AND LXCs, validated)
- DockerContainer is first-class (OS models its own infrastructure)
- Services on any compute type (not just LXC/VM)
- Documents/Runbooks describe any Entity (not just Service)
- Added design notes validating assumptions against actual inventory

Schema updates:
- entity_types: add attribute_schema (JSONB for validating attributes)
- entity_types: add status (active/deprecated, no hard delete while instances exist)

Audit (37 findings across 6 categories):
- Security: 10 findings (5 HIGH) — SSH keys, MCP auth, policy mutability,
  learning poisoning, confirmation phrase, webhook auth, TLS, blast radius
- Performance: 7 findings (1 HIGH) — CTE cycle guard, probe concurrency,
  ingestion, pattern extraction, table growth, WS backpressure
- Architecture: 7 findings (3 HIGH) — testing, observability, DB backup
- Data model: 7 findings (1 HIGH) — entity ID, attribute schema, type
  evolution, concurrent writes, migration rollback, DR export
- Operational: 7 findings (4 HIGH) — rollback, watchdog, backup/restore
  runbook, disaster recovery, deploy downtime, health checks
- Missing: 7 findings (1 HIGH) — CI/CD, rate limiting, audit log, circuit
  breaker, secret rotation, supply chain, SLOs
- Top-5 priority items called out before implementation
2026-07-06 23:06:34 +02:00
d44979aca7 plans: rev 2 — Go rewrite, ontology-first, DB-native config, learning loop
Major revision of the Docker-based homelab OS plan:

1. Go instead of Python — all services rewritten as Go binaries
   (Gin web framework, sqlc for DB access, goroutines for probes)

2. Ontology-first design — systems modeling with 3 layers:
   - Infrastructure (physical, compute, network, storage, software)
   - Governance (identity, secrets, policy)
   - Cognition (observation, decision, action, knowledge, learning)
   7 Mermaid diagrams: layer map, ER diagram, 3 lifecycle state machines,
   feedback loop, policy model

3. DB-native config — inventory.yaml/ontology.yaml/policy.yaml become
   seed manifests (bootstrap + DR). The DB is the runtime source of truth,
   editable via API. Ontology IS the DB schema (entity_types,
   relationship_types, lifecycle_defs tables).

4. Feedback loop — agent learns from execution:
   execution → outcome → feedback → pattern → skill → classification
   Patterns accumulate from execution history, skills codify proven
   procedures, classifier uses pattern confidence for auto-act decisions.
   Cold start: agent starts cautious, earns autonomy through evidence.

5. 5 migration groups: ontology meta-schema, entity instances, operations,
   learning model, policy. Recursive blast_radius SQL function.

6. Phase 0 added: ontology design before any code.
2026-07-06 22:50:49 +02:00
fe54af30f6 plans: replace ASCII architecture diagrams with Mermaid
4 diagrams: container stack, OODA loop, knowledge graph, deploy flow
2026-07-06 22:34:53 +02:00
bead722fac plans: pivot oikos consolidation to docker-based agentic homelab OS
Supersedes the launchd-based consolidation plan. Key changes:
- Docker-based deployment on mac-mini (docker compose)
- PostgreSQL for all mutable state (signals, ledger, knowledge graph)
- Infisical replaces SOPS+age for secrets management
- Unified API merges MCP server + homelab CLI (REST + MCP interfaces)
- Hermes agent runs in Docker (gateway mode, connect from any workstation)
- Knowledge graph in Postgres replaces narrative wiki files as agent context
- Structured entity relationships link docs to inventory entities
- Hybrid SSH access (mounted keys now, actuator gateway later)
- Git push → Gitea webhook → Docker rebuild = deploy trigger
- 6-phase rollout: DB → services → agent → secrets → deploy → cutover
2026-07-06 22:32:21 +02:00
a434a4096c Merge pull request 'chore: add plan' (#2) from claude/heuristic-jang-ecb080 into main
Reviewed-on: dtoro/Homelab-Docs#2
2026-07-06 21:57:48 +02:00
14448a7dd9 chore: add plan 2026-07-06 21:56:56 +02:00
b047c757a7 chore: update title 2026-07-06 21:08:13 +02:00
af14c38fb2 docs: convert OODA loop diagram to real Mermaid syntax
The operating-model diagram in README was ASCII box art in a plain code
fence, not an actual Mermaid diagram — it wouldn't render as a graph on
Gitea/GitHub. Replaced with a `flowchart TD` matching the convention already
used by oikos/gen-topology.py's generated topology.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 20:49:46 +02:00
0d2093ba3b docs: replace hardcoded infra details in README with pointers to index pages
Problem: README listed specific IPs, container IDs, and per-host counts (e.g.
"hubris (15 active): 102 nfs-export, 103 paperless..."). This duplicates
inventory.yaml and the wiki index pages, and goes stale every time a node
moves, gets added, or is destroyed — exactly what happened during the strong
migration.

Fix: Replaced the Proxmox Hosts / VMs / LXC Containers / Cross-Cutting
Infrastructure subsections with plain pointers to their authoritative index
pages (knowledge/wiki/{hosts,vms,containers,infrastructure}/index.md).
Also dropped the "Last refreshed against live state" date line — another
claim that goes stale without a mechanism to keep it honest.

README's job is navigation, not a live topology snapshot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:38:29 +02:00
d90de0759c docs: redesign README for agent clarity and usability
Problem: README.md was human-centric and lacked critical context for agents
(LLMs running on enrolled homelab clients). Agents needed:
- Explicit entry points (AGENTS.md → OIKOS.md → skills → MCP/files)
- Decision tree for tool selection (when to use MCP vs files vs grep)
- Explanation of operating model (OODA loop, risk classes, layer model)

Solution: Reorganized README with agent-first sections while preserving existing
human-useful content:

NEW SECTIONS:
- "For Agents" (entry points + MCP tool selection table with decision criteria)
- "Understanding the Operating Model" (Mermaid OODA loop diagram, risk classes,
  decision flow: classify → escalate if needed → execute → document)
- "Finding & Understanding Information" (layer model table: sources/wiki/index/log,
  what's immutable vs editable, when to update docs)

REVISED SECTIONS:
- "Map & Quick Navigation" (agent entry points first, then topology)
- "Conventions" (expanded with agent-specific guidance: caveman.md, page-templates.md)
- "Updating the Wiki" (clarified infrastructure changes vs restructuring;
  reinforced same-session update rule with explicit checklist)
- "More Information" (grouped agent-facing resources: HERMES, operations,
  skills, shared conventions)

All links verified. No new files needed — all referenced content already exists.

Verification:
- OODA loop diagram present (visual roadmap for decision flow)
- MCP vs Files vs Shell table shows decision criteria
- Layer model (sources/wiki/index/log) explained with immutability matrix
- All cross-references resolve
- Existing topology + infrastructure content preserved

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:32:08 +02:00
18589f6d8d Merge: wiki-hq doc architecture adoption + naming conventions
Complete reorg of narrative docs into sources/wiki/index/log + .agents/ model.
Six implementation phases + two naming clarification passes.

Phase 1-2: Adopt wiki-hq conventions (writing-style, llm-wiki, agent separation)
Phase 3: Move narrative into knowledge/wiki/
Phase 4: Reshape runbooks into skills
Phase 5: Style + README pass
Phase 6: Streamline (move investigations, operations, HERMES to their final homes)
Naming: Explicit conventions for foundational docs (ALL-CAPS) vs content (lowercase)

All 126 pre-existing broken links fixed. Topology, MCP, substrate untouched.
Verification: docs-lint clean, build_host_files idempotent, all doc_page targets resolve.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:20:42 +02:00
2ddc1eaa18 docs: correct file naming convention — foundational docs are ALL-CAPS
The previous naming guide was incomplete. The actual convention is:

**Foundational docs:** ALL-CAPS
- Root entry-points: AGENTS.md, README.md (discovery paths)
- Agent instruction: .agents/OIKOS.md, .agents/HERMES.md (docs agents read first)
- Reference docs: GLOSSARY.md (like classic repo files: LICENSE, CHANGELOG)

**Content pages:** lowercase-with-dashes
- Containers: <id>-<name>.md (ID from inventory)
- Infrastructure: <topic>.md (system description)
- Plans/investigations: YYYY-MM-DD-slug.md (date-sorted)
- Section indices: README.md (conventional)

**Skills:** special pattern
- <name>/SKILL.md where <name> is lowercase-with-dashes
- SKILL.md filename is always uppercase — signpost for tools and humans

Uppercase is reserved for foundational/signpost docs; all paths otherwise use
lowercase with hyphens (no underscores).

Updated page-templates.md with expanded explanation, and updated AGENTS.md +
README.md to reference the corrected convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:19:59 +02:00
658dc0f8b2 docs: document file naming conventions and clarify structure guidance
Added explicit file-naming rules to page-templates.md so agents know:
- Root entry-points: ALL-CAPS (AGENTS.md, README.md)
- Containers: <id>-<name>.md (e.g., 101-jellyfin.md)
- Infrastructure: lowercase-with-dashes (dns.md, auto-deploy.md)
- Plans/investigations: YYYY-MM-DD-slug.md
- Skills: lowercase-with-dashes/ folder containing SKILL.md

Updated AGENTS.md section 4 (Wiki conventions) to link to page-templates.md
and provided quick reference for file naming, page locations, and changelog format.

Updated README.md conventions section to mention file naming and link to
page-templates.md for the full rules.

All agents now have a clear reference chain:
  1. AGENTS.md (entry point) → points to conventions
  2. page-templates.md (structure) → has file naming + page templates
  3. writing-style.md (prose) → has voice, vocabulary, linking rules
  4. llm-wiki.md (organization) → has sources/wiki/index/log model

Verified: no broken links, all conventions documented, consistency check passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:16:58 +02:00
b5c1247093 docs: streamline & consolidate the tree (phase 6)
Problem: after the wiki-hq reorg, agent-instruction and human-doc domains
were still scattered across the repo root, with three now-redundant stub
files cluttering it. The organizing principle wasn't visible in the layout.

Change — enforce three clear buckets:
- .agents/  = how agents operate: OIKOS.md, HERMES.md (moved from root),
  shared/ conventions, domains/ schemas, skills/, and operations/ (operator
  cheatsheet + enrollment + hermes-agent, moved from root).
- knowledge/ = what exists + evidence: wiki/, GLOSSARY.md, and sources/ now
  including investigations/ (incident records are evidence/sources).
- root = substrate + two entry points (AGENTS.md, README.md), plus plans/
  as its own design-intent domain.

Moves:
- investigations/ -> knowledge/sources/investigations/ (incl. archive/, index).
- operations/ -> .agents/operations/.
- HERMES.md -> .agents/HERMES.md.
- Deleted unreferenced root stubs CAVEMAN.md, CONTRIBUTING.md, and OIKOS.md
  (its 7 remaining linkers repointed to .agents/OIKOS.md).

Consumers updated:
- inventory.yaml doc_page (agent-enrollment) + regenerated hosts/*.yaml + cards.
- tools/setup-hermes-soul.sh and bootstrap.sh (x2) -> .agents/HERMES.md.
- bin/homelab help string -> .agents/operations/hermes-agent.md.
- knowledge/operations schemas, llm-wiki, page-templates, incident-investigation
  skill, AGENTS.md/README nav -> new investigations/operations paths.
- All markdown links rewritten via the path-resolving mapper.

Left in place (substrate/executable/separate-domain): hosts/, ledger/, tools/,
plans/, oikos/, mcp/, secrets/, bin/, inventory.yaml.

Verification: docs-lint at baseline (2 intentional cross-repo refs, no new
breakage); gen-topology.py --check exit 0; build_host_files.py idempotent; all
doc_page targets resolve; Hermes provisioning scripts point at the new path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:12:14 +02:00
4efddb8bed docs: fix pre-existing broken links surfaced by docs-lint
Problem: docs-lint (added in the wiki-hq reorg) surfaced 126 broken relative
links that predated this session — a container rename, incident/plan docs
that moved into archive/done subfolders without their inbound links being
updated, and a handful of relative-depth bugs in files nested under
containers/archive/ and plans/done/.

Fixes applied, by category:
- 124-authentik.md -> 106-auth-outpost.md (container was renamed; ~40 refs).
- investigations/{2026-04-21-hubris-crash-loop,2026-05-31-authentik-vps-migration}.md
  -> archive/ prefix (both moved to investigations/archive/ previously).
- plans/{2026-06-01-slate-ax-to-sodola-migration,2026-06-04_130000-deprecate-claudio-bot,
  2026-06-25-yuvomi-deployment}.md -> plans/done/ prefix.
- Depth bugs in files nested one level deeper than their siblings assumed
  (investigations/archive/*, knowledge/wiki/containers/archive/*,
  plans/done/*) — corrected relative-path depth.
- Destroyed containers with no surviving page (126-plato) delinked to the
  containers/index.md archaeology row instead of a 404.
- ludo-mini.yaml -> strong.yaml (host was renamed, same physical machine).
- netbird-vps.md (no narrative page exists) -> netbird-vps.yaml (substrate
  record, matching the existing convention for hosts without a wiki page).
- runbook-dpkg-interrupted.md refs -> .agents/skills/runbook-dpkg-interrupted/SKILL.md
  (missed in the phase-4 runbook move because the referencing files used a
  bare filename, not a runbooks/ prefix).
- One dangling forward-reference to a never-written investigation delinked
  to the actual incident record it was describing.

Left alone: two links in knowledge/wiki/containers/101-jellyfin.md into
devops/homelab-authentik-admin/ — an intentional reference to a sibling repo,
not present in this checkout.

Verification: broken-link count 126 -> 2 (real remainder is the cross-repo
reference above); gen-topology.py --check still exit 0; build_host_files.py
still idempotent; all inventory.yaml doc_page targets still resolve.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:53:35 +02:00
1019a1cb52 docs: section-index pass + docs-lint skill (phase 5)
- Add knowledge/wiki/hosts/index.md (the one missing section index) and point
  knowledge/index.md at it.
- Add .agents/skills/docs-lint/ (SKILL.md + lint.py) enforcing the mechanical
  parts of writing-style.md: banned vocabulary and broken relative links. The
  style guide and this skill are exempt from the banned-word check since they
  enumerate the list.
- Record the restructure + lint in knowledge/log.md.

Verification: banned-vocabulary scan of knowledge/ is clean (the few remaining
repo-wide hits are false positives — the literal '_' character — or historical
append-only plans quoting the vocabulary, which the standard does not restyle).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:42:23 +02:00
5c5016b3c7 docs: reshape runbooks into .agents/skills/<name>/SKILL.md (phase 4)
Problem: runbooks are agent-executable procedures but lived at the repo root,
separate from the other agent instruction now under .agents/.

Change:
- Move runbooks/<name>.md -> .agents/skills/<name>/SKILL.md (folder per skill,
  matching the wiki-hq skills layout). Frontmatter (name, risk_class, inputs,
  verification, docs_update_checklist, transition) preserved.
- Rewrite links (inbound from plans; between-skill siblings) via the move map.
- Update prose references in AGENTS.md, HERMES.md, .agents/OIKOS.md, and the
  operations schema; fix a pre-existing stale link to operations/commands.md.

No code consumed runbooks/ by path, so nothing else changes.

Verification: all SKILL.md frontmatter parses with valid risk_class; every
lifecycle transition resolves to an oikos/ontology.yaml state; broken-link
count 127 -> 126 (fixed one, introduced none).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:39:31 +02:00
8a6422bd7d docs: move narrative wiki under knowledge/wiki/ (phase 3)
Problem: node and cross-cutting narratives lived at the repo root
(containers/, vms/, infrastructure/, host .md files), interleaved with the
machine-readable substrate.

Change:
- Move containers/ -> knowledge/wiki/containers/, vms/ -> knowledge/wiki/vms/,
  infrastructure/ -> knowledge/wiki/infrastructure/, hosts/{hubris,strong}.md ->
  knowledge/wiki/hosts/, infrastructure/references/ -> knowledge/sources/references/,
  GLOSSARY.md -> knowledge/GLOSSARY.md.
- Add knowledge/{index.md,log.md,sources/index.md} scaffolding.
- Rewrite all relative links repo-wide via a path-resolving mapper (inbound +
  outbound + between-moved-files), including .hermes/, runbooks, operations,
  investigations, plans, README, AGENTS.
- Repoint inventory.yaml doc_page fields and regenerate hosts/*.yaml (which
  embed doc_page); update oikos/gen-topology.py output path, candidate doc
  paths, and footer links; update code-comment doc paths.

Substrate untouched in place: inventory.yaml, hosts/*.yaml (regenerated,
idempotent), oikos/ code, mcp/, secrets/, bin/.

Verification:
- Logical broken-link set identical to pre-move baseline (net 128 -> 127; the
  topology regen fixed one, introduced none). Remaining are pre-existing refs
  to destroyed/archived nodes, out of scope for this move.
- gen-topology.py --check exit 0 (in sync); cards carry knowledge/wiki/ doc paths.
- build_host_files.py idempotent; all inventory doc_page targets resolve.
- MCP contract verified: get_page/search_docs/get_changelog resolve moved pages.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:35:23 +02:00
bb5c0717a1 docs: adopt wiki-hq conventions + separate agent instructions (phases 1-2)
Problem: the narrative docs lacked an enforceable style standard, and
agent-facing instruction (OIKOS/CAVEMAN/CONTRIBUTING) was interleaved with
human content at the repo root.

Change:
- Add .agents/shared/{writing-style,llm-wiki}.md — a lint-checkable prose
  standard (with an imperative-voice exception for runbooks/recipes) and the
  sources/wiki/index/log layer model.
- Move CAVEMAN.md -> .agents/shared/caveman.md,
  CONTRIBUTING.md -> .agents/shared/page-templates.md,
  OIKOS.md -> .agents/OIKOS.md; leave thin root stubs so old links resolve.
- Add .agents/domains/{knowledge,operations}/schema.md; operations schema
  codifies "plans always live in plans/".
- Repoint live references (AGENTS, README, GLOSSARY, OIKOS) and fix OIKOS.md's
  internal relative links for its new depth.

Risk: none to the operational substrate — inventory.yaml, hosts/*.yaml,
oikos/, mcp/, secrets/, bin/ untouched (verified via git status).

Verification: relative-link check across .agents/ clean; substrate churn empty.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:22:09 +02:00
14e88c7c5e Document real deploy state: console live, webhook still broken
Console is fully live on apps (105) — deployed manually via deploy.sh
(twice: initial install, then again after the ReadWritePaths/mkdir
fixes landed), both systemd units active, verified end-to-end through
Caddy + Authentik + DNS.

Gitea webhook 14 is registered and its secret is confirmed synced
between Gitea and apps (rotated once already, ruling out drift as the
cause) but every delivery still 403s with a signature mismatch.
Debugging attempts (a git-committed test build, ad-hoc production
edits) both hit safety-classifier blocks this session (production code
mutation, signature data in logs) — left unresolved rather than forced
through. Auto-deploy via push doesn't work yet for this service; manual
deploy.sh re-runs are the workaround until someone tracks this down.

Added to the 60/90-day backlog.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:11:27 +02:00
dede118a80 chore: trigger oikos-console webhook to verify rotated secret 2026-07-06 14:02:26 +02:00
72720efa21 Rotate oikos-console-deploy webhook secret
The value written to apps' /etc/oikos-console-deploy/secret didn't
match what Gitea webhook 14 had configured, causing every deploy
attempt to 403 with a signature mismatch — likely drift introduced by
the earlier two-step PATCH sequence (secret set in one call,
branch_filter/active restored in a second call without re-including
the config object). Rotated cleanly this time: fresh secret set on
Gitea and re-encrypted here in one pass, single atomic PATCH covering
config+events+branch_filter+active together.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:01:29 +02:00
7610e5394c Deploy Oikos Console to apps (105); fix missing-directory crash
Console is live: cloned to /opt/oikos-console, deploy.sh ran clean,
webhook secret written to /etc/oikos-console-deploy/secret from the
pre-registered SOPS secret (never printed — decrypted and piped
straight into the target file in one command), both systemd units
enabled and active. Verified locally (127.0.0.1:8091 -> 200) and
end-to-end (https://oikos.hubris.network/ -> 302, the Authentik gate
firing correctly).

Found a real bug during first boot: oikos-console.service's
ReadWritePaths listed /opt/homelab-context/signals and .../approvals,
but neither existed yet on apps' clone — git doesn't track empty
directories, and nothing had ever written a signal/approval from that
host. ProtectSystem=strict + a missing ReadWritePaths target is a hard
226/NAMESPACE crash, not a graceful degradation. Fixed two ways:
the unit now marks those paths optional (`-` prefix) so a fresh deploy
never crash-loops on this again, and deploy.sh now mkdir -p's them
explicitly so the console has real write access from the first boot,
not just a non-crashing-but-broken start.

This is also the first real exercise of the auto-deploy pipeline: this
push should land via Gitea webhook 14 -> oikos-console-deploy.service
on apps, same as homelab-mcp/secrets-issuance already work.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:57:50 +02:00
610b096840 Add Technitium DNS record for oikos.hubris.network
A record -> 192.168.8.175 (Caddy's LAN IP), created via Technitium's
REST API (login -> createToken -> zones/records/add) in a single
in-memory call. Neither the admin credential nor the resulting session/
API token was ever printed to output or written to disk, and the token
wasn't persisted anywhere after the call completed — it existed only
for the lifetime of that one process.

Verified: dig @192.168.8.2 +short oikos.hubris.network -> 192.168.8.175.
End-to-end confirmation that DNS + Caddy + the Authentik gate are all
wired correctly: curl https://oikos.hubris.network/ now returns a 302
(the forward-auth redirect firing before the not-yet-deployed backend
would even matter) instead of failing to resolve/connect.

This closes out every part of the console rollout except the actual
apps-side bootstrap (oikos/console/deploy/README.md "One-time setup"),
which remains pending direct operator execution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:41:52 +02:00
25c67a79c6 Add Caddy route for oikos.hubris.network; fix loopback-bind bug
Pushed dtoro/caddy-conf@c195142: oikos.hubris.network -> 192.168.8.205:8091,
Authentik-gated (matches paperless.hubris.network's live pattern —
confirmed exact snippet syntax against the real Caddyfile rather than
trusting the paraphrase in the original README, which turned out to
have the wrong forward_auth target: the live snippet points at
127.0.0.1:8099 on Caddy's own LXC, not 192.168.8.6:9000 as
containers/106-auth-outpost.md's older text suggested). Reload verified
clean — an unrelated existing route stayed healthy through it.

Found and fixed a real deploy-blocking bug in the process:
oikos-console.service bound 127.0.0.1 only, but Caddy runs on a
different host (121) and can only reach apps (105) over the LAN — the
console would have been completely unreachable once deployed. Now binds
0.0.0.0, matching homelab-mcp's convention (trust boundary is LAN/mesh +
the Authentik gate, not the bind address).

Encountered and deliberately left alone: a pre-existing local clone at
/tmp/caddy-conf with an unpushed commit + uncommitted diff about
jellyfin's auth gating, from before this clone fell 12 commits behind
origin. That work turned out to be superseded (origin already reached
the same conclusion — SSO plugin handles jellyfin auth, no forward-auth
gate — via a different, already-merged path). Didn't touch it; used a
fresh clone instead to avoid any risk of losing or corrupting that state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:31:51 +02:00
493ae50f37 Register Gitea webhook 14 for oikos-console deploy
Created via the Gitea API (POST /repos/dtoro/Homelab-Docs/hooks) rather
than the UI, since the existing PAT turned out to have sufficient scope.
Webhook id 14: http://192.168.8.205:9831/deploy, push events, main branch
filter, active.

The shared secret was generated and registered with Gitea before the
apps-side bootstrap ran (order reversed from the usual install.sh-first
flow, since direct SSH deploy to apps is still pending operator
execution — see oikos/console/deploy/README.md). Stored as
secrets/oikos-console-deploy-secret.yaml (SOPS, recipient: apps only)
rather than left as a local plaintext file, with explicit operator
sign-off. When the apps-side install runs, skip webhook/install.sh's
random-secret generation and write this exact value into
/etc/oikos-console-deploy/secret instead.

infrastructure/auto-deploy.md updated with the real webhook id (was
"not yet registered").

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:18:46 +02:00
c6fedb38c1 Document teddycloud (pve_id 131) — closes the Week-3 drift finding
teddycloud was live on hubris (LXC 131, docker compose, TeddyCloud —
a Toniebox cloud reimplementation) but never made it into inventory.yaml.
Already referenced in passing by containers/132-rclone.md ("131 was
already taken by an undocumented teddycloud container") and
hosts/strong.md's migration changelog (a DHCP conflict fix), but no
inventory entry or doc page existed until oikos/drift.py's inventory-
vs-live check caught it.

Verified live via read-only SSH (pct config 131, pct exec 131 -- ...,
docker ps): hostname, static IP 192.168.8.150, 1 core/1GiB/16GiB rootfs,
Debian 12, runs via docker compose at /opt/teddycloud. No changes made
to the running container.

Also fixed: house's inventory notes claimed 192.168.8.212 is teddycloud's
current IP via DHCP — stale, teddycloud has a static IP now.

Flagged in the new container page: teddycloud has no Caddy forward-auth
gate, unlike sab.hubris.network on the same Caddyfile.

`python3 oikos/drift.py` no longer reports an inventory-vs-live finding
for pve_id 131. (A separate, pre-existing gap surfaced while verifying
this: rclone's own inventory.yaml block is missing pve_id/host/lan_ip —
out of scope here, flagging for a follow-up.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:06:26 +02:00
fd35b48c8d Phase 1-4: full doc reorg
Phase 1 — fix stale state after strong migration (Phase 1+2, 2026-07-05)
  - README: corrected IPs (jellyfin 206→246, arriman 132→245, etc.),
    added missing containers (128 trmnl, 129 house, 133 seanime, 134 romm,
    124 authentik), updated last-refreshed date, added strong host context
  - containers/101-jellyfin.md: IP 206→246, host hubris→strong, mount
    /mnt/library→/mnt/media_local, GPU 760M→680M+RX7600, privilege→priv
  - containers/118-elementsynapse.md: IP 239→242, added Host: strong
  - containers/122-arriman.md: IP 132→245, mount→/mnt/media_local, added Host
  - containers/129-house.md: IP 212→244, added Host: strong
  - containers/130-grimmory.md: IP 213→247, mount→/mnt/media_local, added Host
  - containers/121-caddy.md: fixed site list (books→grimmory, removed auth→VPS,
    added house, roms, teddy, trmnl)
  - hosts/strong.md: updated At-a-glance to reflect 7 LXCs hosted
  - containers/123-claudio-bot.md, 127-mule-photos-new.md: archived to
    containers/archive/ (were destroyed LXCs with living pages)
  - inventory.yaml: verified correct — no changes needed

Phase 2 — structural cleanup
  - infrastructure/index.md: one-page overview of all cross-cutting systems
  - runbooks/: moved runbook-budget-from-csv.md and runbook-dpkg-interrupted.md
    from operations/ with YAML frontmatter added
  - plans/done/: moved 4 completed plans out of active view; updated index
  - vms/index.md: added VM index page

Phase 3 — navigation & discoverability
  - GLOSSARY.md: term definitions (Authentik, Caddy, LXC, VAAPI, etc.)
  - README: added table of contents, links to glossary + infrastructure index
  - investigations/: archived 2 resolved cases (crash-loop, authentik-migration)
    to investigations/archive/; updated index with active vs archived sections

Phase 4 — ongoing discipline
  - CONTRIBUTING.md: documented same-session update rule with explicit checklist
  - README: replaced full LXC table with summary + link to containers/index.md
    (single source of truth; de-duplication)
2026-07-06 00:46:27 +02:00
205d8a1a43 Oikos Week 4: Console v0, approval hardening, docs pass, backlog
Oikos Console v0 (oikos/console/) — read-mostly, server-rendered FastAPI
+ Jinja2 web UI, no SPA build chain. Signals landing page, service grid
+ detail, node/blast-radius view, live Mermaid relationship graph, drift
findings, approvals queue (approve/deny, destructive confirmation-phrase
enforced), daily/weekly reports. Tested end-to-end via the preview tools
against live production data, including a real click-through of the
approve/deny flow.

Found and fixed two bugs during that testing:
- Severity-dot CSS classes didn't match the actual severity strings
  (dot-warn/dot-crit vs "warning"/"critical") — warning-severity signals
  rendered with no visible indicator at all.
- The console's sys.path setup pointed at its own webhook checkout
  (/opt/oikos-console) rather than /opt/homelab-context, which would have
  made its oikos.* imports resolve to a SEPARATE copy of oikos/signal.py
  etc. than the scheduler and CLI use — silently forking signal/approval
  data into two locations in production. Fixed to match mcp/server.py's
  CONTEXT_DIR pattern. Also added _commit_push() so the console's writes
  (approval replies, signal ack/resolve) don't sit uncommitted against
  the 5-min-synced clone.

Split oikos/gen_topology_lib.py out of oikos/gen-topology.py (hyphenated
filenames aren't importable) so the console's /graph route can render
live without shelling out.

oikos/console/deploy/ — third webhook on dtoro/Homelab-Docs (port 9831),
matching the homelab-mcp/secrets-issuance precedent. README documents the
Caddy route and Gitea webhook registration this repo can't do for itself,
and that Authentik step-up on /approvals needs a live instance to
configure.

Approval hardening: grants are now single-use (oikos/approve.py
check_grant marks the request "executed" atomically, so a second call
for the same id fails even within the TTL) — verified with a test. Per-
agent age-key-signed requests, as originally planned, turned out not to
be buildable as stated: age is encryption-only, no signing primitive.
Documented the real alternative (SSH-key signing) and moved it to the
60/90-day backlog pending an inventory schema gap (no SSH pubkeys
recorded today).

Docs pass: added the Oikos command surface to operations/commands.md,
new MCP tools to AGENTS.md. Found two more stale references while at
it — commands.md and AGENTS.md both still pointed DNS at the destroyed
LXC 124/dnsmasq instead of Technitium on dns (107), and a claudio-monitor
reference deprecated since 2026-06-04 — fixed both.

60/90-day backlog written into OIKOS.md, derived from gaps actually
observed this month, not guesswork.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 00:03:51 +02:00
2084a1583e Oikos Week 3: scheduler, drift detectors, signals, classifier, approvals
New kernel modules, all wired into `homelab` CLI + tested against live
production where reachable:

- oikos/scheduler.py — Observe stage: HTTP health probes for every
  service, disk-usage probes on hubris/strong, writes oikos/state.json
  (gitignored — regenerates every run). `homelab service <name> health`
  is now cache-first; `--live` forces a fresh probe. Deploys via
  oikos/systemd/oikos-scheduler.{timer,service} on LXC 105.

- oikos/drift.py — SOPS-recipient-vs-inventory and lifecycle-consistency
  detectors (fully local, no SSH) plus pct-list and Caddy-backend
  detectors (best-effort SSH, degrade to an info finding when
  unreachable rather than a false drift alarm). Found real, currently-
  true drift on first run: republic-laptop's age key granted on every
  secret but missing from inventory.yaml, grimmory missing from
  hello.yaml's recipients, and an undocumented pve_id 131 on hubris —
  recorded in OIKOS.md for the operator, not auto-fixed (each is a
  config_mutation/destructive decision).

- oikos/signal.py — the attention layer: raised -> acknowledged ->
  acting -> resolved|muted lifecycle, severity-based routing, dedup via
  open_signal_for(). `homelab signal list|raise|ack|resolve|mute`.

- oikos/decide.py — the Decide-stage classifier: risk class x blast
  radius x ledger-history confidence -> auto-act/escalate. Adds an
  action-alias layer (oikos/policy.py ACTION_ALIASES) and auto-infers
  service_name from the entity for per-service policy overrides.
  `homelab decide <action> <entity>`.

- oikos/approve.py — the escalate route. No dedicated Matrix bot exists
  in this homelab, so this is the repo-side half only: request/reply/
  grant lifecycle with short-TTL HMAC-signed tokens (new secret
  secrets/oikos-approval-hmac.yaml, recipients apps+hubris). Matrix
  delivery is Hermes's existing @dtoro:avispero send path (documented
  integration contract in the module docstring), not a new bot.
  `homelab restart` now mechanically refuses config_mutation/destructive
  services without a valid --approval-id, regardless of -y/interactivity.

- oikos/report.py — daily brief + weekly report from signal/approval/
  ledger state (no Prometheus yet, so point-in-time counts only).

- plans/2026-07-05-oikos-prometheus-lxc.md — Prometheus is `planned`,
  not provisioned: no pve_id is guessed here since Proxmox assigns real
  IDs at creation time, and drift already found an unclaimed ID (131) to
  investigate first.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 23:29:39 +02:00
48debc0911 runbooks: clarify Netbird join is optional, not a required enrollment step
Only off-LAN-reachable workstations (e.g. republic-laptop, mac-mini)
need to join Netbird. LAN-reachable LXCs/VMs on 192.168.8.0/24 don't —
they're already directly reachable, and off-LAN clients reach them via
hubris's routed 192.168.8.0/24 Netbird network resource. Brings the
runbook in line with oikos/ontology.yaml's lifecycle transition, which
already says "mesh-joined-if-needed".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 23:03:21 +02:00
f6b57cbe3a Oikos Week 2: Service Console v0, change ledger, node relations, runbooks
Adds the shared kernel modules (oikos/policy.py, oikos/relations.py,
oikos/ledger.py) that let every surface — CLI, MCP, context-card
generator — agree on risk classification and ontology graph walks
from one implementation.

homelab CLI: `service <name> explain|health|docs|log|actions|history`
(Service Console v0), `change preflight <service>`, `node <name>
relations`. Restart and client add/remove now append change-ledger
entries (ledger/*.jsonl, committed alongside the change they record).

mcp/server.py mirrors explain/preflight/get_relations/get_change_history
as MCP tools, card-first so agent orientation is one call instead of
several search_docs/get_page round-trips.

oikos/gen-topology.py now also emits a compact context card per host
and service (oikos/cards/*.md) — identity, blast radius, safe actions +
risk class, doc pointer, recent ledger history.

runbooks/*.md: service health check, config change + deploy, client
enrollment, incident investigation, and the five node lifecycle
transitions (provision/activate/migrate/deprecate/destroy), each with
machine-readable frontmatter (risk class, inputs, verification,
docs-update checklist). Wired into HERMES.md so agents load these
instead of rediscovering topology per-task.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 23:02:32 +02:00
b230ab5937 Oikos Week 1: kernel policy, ontology, service contract, topology gen
Adds the Oikos agent-OS kernel: oikos/policy.yaml (risk classes +
approval rules for every homelab/MCP command), oikos/ontology.yaml
(8-domain systems model, typed relationships, node lifecycle), and
OIKOS.md (OODA loop operating brief, linked from AGENTS.md).

Extends inventory.yaml with a stable service contract (doc_page,
config_repo, risk_notes) on all 17 services, and a structured
archaeology: section for the 13 destroyed LXCs (was scattered
comments + a narrative table). Fixes stale drift found in the
process: authentik's backend pointed at a retired LXC (124); core
has run on the VPS since 2026-05-31.

Adds oikos/gen-topology.py, generating infrastructure/topology.md
(Mermaid compute/ingress + storage views) from inventory.yaml.
build_host_files.py now carries state/storage/depends_on into
generated hosts/*.yaml.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 22:50:34 +02:00
7e8860ab47 update seanime docs: qBittorrent config, extensions, container doc, arriman note 2026-07-05 21:17:04 +02:00
e04d943d5c fix VPS traefik backends after Phase 1+2 migration
- Jellyfin: 192.168.8.206 → 192.168.8.246 (stale after LXC 101 migration to strong)
- House: 192.168.8.212 → 192.168.8.244 (stale after LXC 129 migration to strong)
- Jellyseerr/qbit/sab: 192.168.8.132 → 192.168.8.245 (arriman on strong)
- Added migration pitfalls section to reference doc
2026-07-05 21:03:34 +02:00
7d5e7227ca Add RomM LXC 134 docs (strong, 192.168.8.249, roms.hubris.network) 2026-07-05 20:58:13 +02:00
ec52dfb6a2 add seanime LXC 133 on strong (anime media server, 192.168.8.248) 2026-07-05 17:12:38 +02:00
bb33963539 strong migration Phase 1+2: move 5 LXCs + library split to ludo-lvm 2026-07-05 16:24:48 +02:00
2709e79455 jellyfin: VAAPI HW accel + Authentik SSO + resource bump (2026-07-04)
- Upgraded 10.11.8 → 10.11.11, enabled VAAPI (Radeon 760M)
- Bumped to 4 cores / 8 GiB RAM / 1 GiB swap
- SSO-Auth plugin v4.0.0.4 with Authentik OIDC
- Removed Caddy forward-auth gate (incompatible with SSO plugin)
- Updated container doc with full SSO architecture + pitfalls
2026-07-04 22:00:46 +02:00
831794e98c docs(rclone): root-cause the "stalls" as OOM, not Proton; bump RAM to 2G
The recurring silent-freeze incidents on LXC 132 were rclone-rcd.service
getting OOM-killed under the original 1 GiB allocation, not a protondrive
backend quirk as first suspected. journalctl confirmed the OOM kill at the
exact freeze point. Bumped LXC memory to 2 GiB (live, no reboot) and the
full folder set (cloud/documents/repos) completed cleanly afterward.

Also documents two watchdog bugs found while chasing this: a wrong
stats-group key that made a healthy sync look falsely frozen, and a
blocking systemctl restart that caused the watchdog to silently disable
itself after firing once. Both fixed; watchdog kept as a safety net.
2026-07-03 22:04:43 +02:00
6669feafdc docs(rclone): document protondrive silent-stall incident + watchdog
Two silent stalls hit in LXC 132's first 24h of real traffic: rclone's own
--timeout didn't catch a protondrive-specific hang (transfer at 100%, zero
bytes/errors/retries for hours). Added a 5-min watchdog timer that restarts
rclone-backup.service if transferred bytes are frozen for 15+ min. Also
found and fixed a monitoring bug in the runner (wrong stats-group key) that
made a healthy sync look falsely stalled for 22h in its own log.
2026-07-03 12:32:53 +02:00
ba93c4709b docs(rclone): LXC 132 rclone -> Proton Drive backup; deprecate restic-on-USB
New off-host backup job replacing the disabled restic-on-USB backup: LXC 132
`rclone` mirrors selected /mnt/library folders to Proton Drive (plain rclone
sync, Proton's built-in E2E, no crypt overlay) on a monthly timer, with
rclone's Web GUI for LAN-only browsing/ad-hoc runs and live job status.

- containers/132-rclone.md: full design, Proton auth gotcha (TOTP secret vs
  live code), pct exec PATH gotcha, rc-API job-visibility runner rewrite,
  selected folder set (cloud/documents/repos), deferred tracked-repo note.
- infrastructure/backups.md: restic-on-USB marked DEPRECATED/superseded,
  leads with the new job now.
- containers/index.md, README.md, infrastructure/media-permissions.md:
  register the new container.
2026-07-02 00:37:10 +02:00
root
5887129202 client-add: rclone (finalize age_pubkey + grant shared secrets) 2026-07-01 23:50:37 +02:00
root
1ad31fe33a client-add: rclone 2026-07-01 23:37:32 +02:00
4e3ec61fb0 chore: ignore __pycache__/*.pyc
Left behind by py_compile-checking bin/homelab's syntax during today's
bugfix session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 22:51:44 +02:00
abb1476fc8 fix(homelab): cmd_sync missing the geteuid guard every other command has
Every other mutating subcommand (secret, refresh-creds, client add/remove)
already re-execs via sudo only when os.geteuid() != 0. cmd_sync was the
one exception, calling sudo unconditionally — fails with "No such file or
directory: 'sudo'" on minimal root-only images (no sudo binary at all),
hit live running `homelab sync` on strong over root SSH.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 14:02:07 +02:00
dee08b97a4 secrets: grant strong access to hello.yaml and gitea-pat.yaml
Standard baseline for an enrolled workstation-class client, matching
mac-mini/republic-laptop/etc: the bootstrap decrypt-test secret plus
the write-scoped Gitea PAT so strong can push to the wiki repo on its
own (homelab client add/remove, wiki edits from that host).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 14:01:00 +02:00
359f55a695 docs(strong): record age-key issuance and the 3 bootstrap.sh bugs found fixing it
strong now has its own age key (issued over LAN via --no-mesh),
pubkey recorded in inventory.yaml. Not yet granted to any secrets
file - that's a separate decision.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 12:40:47 +02:00
fc62cf38f7 fix(bootstrap): add --no-mesh so LAN-only secrets issuance doesn't block on Netbird SSO
Running bootstrap without --no-secrets always tried to install and
connect Netbird, even when the mesh-check right after it already knows
how to fall back to plain LAN reachability. On a host nobody's watching
interactively (e.g. driven over SSH), this hangs forever at the
device-code prompt — hit live on strong, had to kill the stuck
`netbird up` process manually. --no-mesh skips netbird install/up while
still allowing the existing LAN-fallback path to satisfy secrets
issuance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 12:38:21 +02:00
b266c3f1d8 fix(bootstrap): sops has no apt/dnf package, fetch the binary directly
Discovered live re-running bootstrap on strong for secrets issuance:
apt-get install sops fails outright (no such Debian package — matches
what agent-enrollment.md's manual-install recipe already does, fetching
the binary from GitHub releases instead of a package manager). dnf would
have the same problem. Added install_sops_binary(), used on both the
dnf and apt paths; Darwin still installs via brew, which does carry sops.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 12:34:12 +02:00
fdab6282e6 fix(bootstrap): don't invoke sudo when already running as root
The pipx/mcp-CLI step and the Hermes goose installer both called
`sudo -u <user> ...` unconditionally. On minimal Linux images reached
via `ssh root@host` (no SUDO_USER, and often no `sudo` binary at all —
seen live on strong), this failed with "sudo: command not found" and
silently no-opped the mcp CLI install. Added a run_as() helper that
only shells out to sudo when there's a real invoking user distinct
from root.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 12:33:19 +02:00
e4b529b71b docs(strong): record homelab-context client enrollment
bootstrap.sh --no-secrets ran clean: sync timer, homelab CLI, and
AGENTS.md are live on strong. Noted two follow-ups: secrets issuance
is reachable over plain LAN (mesh: lan) so age-key enrollment doesn't
actually need Netbird, and bootstrap's pipx/mcp-CLI step silently no-ops
when run as root over SSH (missing `sudo` binary).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 12:30:05 +02:00
c012a9124f infra: join ludo-mini to hubris as Homelab cluster node "strong"
Reformatted the ludo-mini workstation to Proxmox VE 9.2.3 and joined it
to hubris's existing single-node "Homelab" cluster (2 nodes, no QDevice
yet). Added a second NVMe as its own LVM-thin pool (ludo-lvm). Renamed
the wiki/inventory identity from ludo-mini to strong to match the OS/
cluster hostname, since bootstrap's client-enrollment lookup depends on
that match. Also regenerated hosts/grimmory.yaml, which was missing from
git despite being referenced by inventory.yaml.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 12:28:24 +02:00
301 changed files with 35480 additions and 787 deletions

View File

@@ -15,6 +15,19 @@ truth for:
When in doubt, check `/opt/homelab-context/` first.
## Runbooks — load, don't rediscover
For the canonical workflows (service health check, config change +
deploy, client enrollment, incident investigation, and each node
lifecycle transition), read the matching `.agents/skills/<name>/SKILL.md` before
acting. Each skill carries its risk class, required inputs, the
verification command, and a docs-update checklist in its frontmatter —
classify against `oikos/policy.yaml` using that risk class before any
mutation. Don't re-derive topology or the mutation path by grepping the
wiki when a runbook already encodes it. See [OIKOS.md](OIKOS.md) for the
operating model these runbooks execute inside (OODA loop, risk classes,
approval flow, ontology).
## Agent type — how this file gets loaded
| Agent | Loading mechanism |

227
.agents/OIKOS.md Normal file
View File

@@ -0,0 +1,227 @@
# Oikos — the operating model
Oikos (Greek: *household*) is the agent operating system layered on this
repo. It is not new infrastructure: `inventory.yaml` is the kernel data
structure, the `homelab` CLI and MCP server are the syscall surface, and
this page defines the rules everything above them follows.
Read this after [AGENTS.md](../AGENTS.md). Machine-readable companions:
[oikos/ontology.yaml](../oikos/ontology.yaml) (systems model),
[oikos/policy.yaml](../oikos/policy.yaml) (risk & approval).
## The kernel loop: OODA
Every Oikos activity — scheduled probe, agent task, operator request — is
one pass through **Observe → Orient → Decide → Act**:
1. **Observe** — probes, drift detectors, and agent findings produce
**Signals** (structured records, not loose messages): pending updates,
high temperature, low disk, service down, cert expiry, stale backup,
inventory drift.
2. **Orient** — walk the ontology graph: what entity is affected, what
depends on it (blast radius), its lifecycle state, whether a runbook
matches, what the ledger says about past attempts.
3. **Decide** — the classifier scores **risk class × blast radius ×
confidence** and routes:
- **auto-act**: within autonomy policy, high confidence, contained radius
- **escalate**: operator approval via Matrix (✅/❌ reaction) or the
Oikos Console's `/approvals` page (destructive actions additionally
need a typed confirmation phrase either way)
- **queue**: informational — console + reports
The classifier can only *lower* autonomy relative to policy, never raise
it. When in doubt, escalate.
4. **Act** — execute through `homelab` commands or runbooks (never ad-hoc
SSH), then **verify** with the action's verification command, write a
**ledger** entry, resolve the Signal, and update docs in the same session.
## Primitives
| Primitive | What it is | Lives in |
|---|---|---|
| Host / Service | topology entities | `inventory.yaml` (+ generated `hosts/*.yaml`) |
| Secret | SOPS+age encrypted value, per-client recipients | `secrets/` + `.sops.yaml` |
| Runbook | executable workflow with risk class + verification | `.agents/skills/<name>/SKILL.md` |
| Signal | something needing attention, with lifecycle | `signals/` ledger (Week 3) |
| Change | one mutation: who, what, risk, approval, verification | `ledger/` (Week 2) |
| Approval | short-TTL signed grant for a gated action | approval engine (Week 3) |
| Incident | investigation narrative | `knowledge/sources/investigations/` |
| Plan | design doc for non-trivial work | `plans/` |
| Agent | enrolled client identity = its age pubkey | `inventory.yaml` + `.sops.yaml` |
## Risk classes (enforced, not advisory)
From [oikos/policy.yaml](../oikos/policy.yaml):
- **read_only** — status, logs, docs, inventory. Unattended.
- **reversible_low** — restart, cache clear, sync pull. Unattended + ledger.
- **config_mutation** — tracked-config edits (commit+push, never local),
deploys, upgrades, DNS/ingress changes. Operator approval.
- **destructive** — destroy, format, wipe, rotate, revoke. Approval +
typed confirmation phrase.
Lifecycle gates modify these: `provisioning` nodes are freely mutable
(nothing depends on them); `deprecated` nodes accept no new dependents;
anything touching a `destroyed` node is drift.
## The systems model
Eight domains — physical, compute, network, storage, software,
identity & access, operations, external — cover everything in the lab;
entities are connected by typed edges (`hosts`, `provides`, `mounts`,
`stores-on`, `routes-to`, `can-decrypt`, `depends-on`, `backs-up-to`, …)
defined in [oikos/ontology.yaml](../oikos/ontology.yaml). Rule of
completeness: **if it can break, be changed, or hold data, it has an
entity and edges.** Blast-radius questions ("what breaks if strong goes
down?") are graph walks, not doc archaeology.
Nodes move through an explicit lifecycle —
`planned → provisioning → active → migrating → deprecated → destroyed`
stored as `state:` in inventory (absent = active). Destroyed nodes live in
the `archaeology:` section. Each transition is a runbook checklist;
deprecation completes only when inbound edges reach zero.
Generated views: [infrastructure/topology.md](../knowledge/wiki/infrastructure/topology.md)
(Mermaid, regenerated from inventory) and the live, clickable version at
`oikos.hubris.network/graph` once the Console is deployed.
## Conventions carried forward
- Inventory is the truth; live state wins over narrative docs.
- Prefer `homelab` CLI and MCP over ad-hoc SSH.
- Meaningful changes update docs in the same session.
- Secrets are decrypted locally via per-client keys; never into docs/comments.
- Tracked configs change by commit + push, not local edits.
- Netbird is the preferred mesh path for new traffic.
- Agents are terse ([caveman.md](shared/caveman.md)), verify claims, and fix
collateral drift when found.
## Build status (30-day roadmap, started 2026-07-05)
- **Week 1**: policy, ontology, service contract, archaeology, topology
generator, this brief. Shipped.
- **Week 2**: context cards, `homelab service <name> …`, change ledger,
`node relations`, runbooks. Shipped.
- **Week 3**: ops scheduler + state cache (`homelab service <name> health`
is cache-first, `--live` forces a probe), drift detectors, signal engine
(`homelab signal …`), decision classifier (`homelab decide …`), approval
engine (`homelab approval …` — shared-HMAC grants; Matrix delivery is
Hermes's existing `@dtoro:avispero` send path, not a new bot, see
`oikos/approve.py`), daily brief + weekly report (`oikos/report.py`).
Shipped, except: Prometheus is still `planned` (see
[plans/2026-07-05-oikos-prometheus-lxc.md](../plans/2026-07-05-oikos-prometheus-lxc.md)) —
trend signals (disk-full prediction, temp creep) wait on that LXC; the
scheduler's disk check today is point-in-time only, and CPU/NVMe
temperature isn't probed at all yet (no confirmed sensor path on
hubris/strong). DNS-vs-inventory and generic tracked-config-cleanliness
drift checks are also deferred (see `oikos/drift.py` docstring).
- **Week 4**: Oikos Console v0 shipped — signals landing page, service
grid + detail, node/blast-radius view, live Mermaid graph, drift view,
approvals queue (approve/deny, destructive confirmation-phrase
enforced), daily/weekly reports. Server-rendered FastAPI + Jinja2, no
SPA build chain, tested end-to-end against live production data (see
`oikos/console/`). Deploys as a third webhook on `dtoro/Homelab-Docs`
(`/opt/oikos-console`, port :9831) — see
[oikos/console/deploy/README.md](../oikos/console/deploy/README.md) for
the Caddy route and Gitea webhook registration this repo can't do for
itself. Approval grants are now single-use (a second `check_grant` call
for the same request fails even within the TTL) and already exact-bound
to request id + entity + action.
**Not shipped as originally planned:** per-agent *age-key-signed*
request authentication — age has no signing primitive (it's an
encryption-only keypair format), so "age-key-signed" wasn't
buildable as stated. The real alternative (SSH-key signing via
`ssh-keygen -Y sign`/`-Y verify`, using each host's already-provisioned
SSH key) is real and buildable, but needs SSH public keys recorded in
inventory first — not there today. Moved to the 60/90-day backlog.
Authentik step-up re-auth on the approve/deny route is documented but
needs a live Authentik instance to configure — also backlog.
Docs pass done (this file, AGENTS.md, operations/commands.md); found
and fixed two more stale references while at it (DNS section still
pointed at destroyed LXC 124/dnsmasq instead of Technitium on 107, and
a `claudio-monitor` reference that's been deprecated since 2026-06-04).
### Real drift found while building Week 3 (unresolved, needs operator action)
The drift detectors surfaced genuine, currently-true findings on first
run against production — recorded here rather than silently fixed, since
each is a `config_mutation`/`destructive`-class decision:
- `republic-laptop` has no `age_pubkey:` in `inventory.yaml`, but its real
age key is granted on nearly every shared secret in `.sops.yaml`
(`age1vf8h7...`) — the enrollment write-back to inventory never
happened. Fix: `homelab client add republic-laptop --finalize-pubkey
age1vf8h7s8mqsn2q5eadgpdupsj4mwn8zguc77d85ws3xj40sl9rgksx2rxw6`.
- `grimmory` has an `age_pubkey` in inventory but is missing from
`secrets/hello.yaml`'s recipient list — incomplete enrollment the
other direction. Fix: re-run `homelab client add grimmory
--finalize-pubkey <its key>`.
- `pve_id 131` exists live on hubris (`pct list`) with no inventory entry
— investigate before assuming it's a stale ID (see the Prometheus LXC
plan doc above, which flags this explicitly).
- Three `lifecycle-pve-id-reuse` info findings (100, 106, 107 each shared
between an active host and an archaeology entry) — expected/benign ID
reuse after destroy, no action needed.
## 60/90-day backlog
Derived from gaps observed while building the 30-day roadmap, not
guesswork. Roughly ordered by what unblocks the most:
- **Fix the oikos-console deploy webhook's signature mismatch.** Console
is live on apps (105) via a manual `deploy.sh` run, but Gitea webhook
14's deliveries all 403 with a signature mismatch for a cause not yet
found — the secret is confirmed synced correctly on both sides
(rotated once already to rule out drift). Until fixed, `git push`
doesn't auto-redeploy the console the way it does for homelab-mcp/
secrets-issuance; re-run `deploy.sh` on apps manually after changes.
See [oikos/console/deploy/README.md](../oikos/console/deploy/README.md).
- **SSH-key-signed approval requests.** Replaces the design note in
Week 4: age keys can't sign (encryption-only format), so per-agent
request authentication needs `ssh-keygen -Y sign`/`-Y verify` against
each host's existing SSH key. Blocked on a schema gap: inventory
doesn't record SSH public keys today, only ports/users. First step is
populating that field on enrollment, then wiring `oikos/approve.py` to
require and verify a signature over the request payload.
- **Authentik step-up re-auth** on the Console's `/approvals` POST route
— needs a live Authentik `PromptStage`/reauth flow scoped to that path;
not configurable without a running instance to test against.
- **Prometheus provisioning** (see
[plans/2026-07-05-oikos-prometheus-lxc.md](../plans/2026-07-05-oikos-prometheus-lxc.md))
— unblocks trend signals (disk-full prediction, temp creep) and real
sparklines in the Console; investigate the undocumented `pve_id 131`
on hubris first.
- **CPU/NVMe temperature probing** in the scheduler — needs a confirmed
sensor path on hubris and strong (lm-sensors vs vendor tool) before a
real check can be written; guessing one risks a probe that silently
never fires.
- **DNS-vs-inventory drift check** — compare Technitium zone records
against `services.*.url`/`public_host`; not implemented (`oikos/drift.py`
has no Technitium API wiring yet).
- **Generic tracked-config-cleanliness drift check** — today only caddy's
`/etc/caddy` git-checkout path is hardcoded in `oikos/drift.py`; every
other service with a `config_repo` needs its local checkout path
recorded (a `mutation_path`-style field, same gap Week 1's service
contract flagged but didn't backfill) before this generalizes.
- **Per-service policy overrides** (`oikos/policy.yaml`
`service_overrides`) — schema is ready (caddy/dns already use it);
populate more as specific services turn out to need non-default risk
classes.
- **Incident timeline generator** — stitch ledger + signal history into
a single narrative for `knowledge/sources/investigations/` entries instead of writing
them by hand.
- **Secret access audit** — who-can-decrypt-what report from
`.sops.yaml` + inventory `age_pubkey`s, extending what
`oikos/drift.py`'s SOPS check already partially does.
- **Restore drills** — exercise `backs-up-to` (once populated) by
actually restoring from a backup target on a schedule, not just
checking freshness.
- **Multi-agent delegation model** — more than one agent acting
concurrently; needs the ledger's `agent` field to carry real identity
(age pubkey, not just hostname) consistently, which it mostly does
already but hasn't been stress-tested with concurrent writers.
- **Grafana** — only if the Console's own Prometheus-backed sparklines
turn out to be insufficient once Prometheus ships.
- **"Generalize later" extraction** — the original decision was personal-
first, generalize-later (see Week 1). Once patterns stabilize, extract
a config-driven Oikos core with no `hubris.network`/`hubris`/`strong`
hardcoding, so it's installable on a different homelab.

View File

@@ -0,0 +1,48 @@
# Knowledge domain — schema
The knowledge domain is the durable, authoritative current-state documentation of the homelab: one
page per node and per cross-cutting system, synthesized from live state and evidence. It answers
"what exists and how does it work right now."
It follows the [LLM Wiki layer model](../../shared/llm-wiki.md) and the
[writing-style](../../shared/writing-style.md) and [page-templates](../../shared/page-templates.md)
rules.
## The narrative / substrate split
The knowledge wiki is **narrative**. It sits alongside a **machine-readable substrate** that it
describes but never contains. The split is load-bearing: several programs read the substrate at
fixed paths, so the wiki reorganization never moves it.
| Layer | Location | Consumed by |
|-------|----------|-------------|
| Substrate — source of truth | `inventory.yaml` (root) | MCP server, `homelab` CLI, `oikos/` scheduler/drift/relations/gen-topology |
| Substrate — generated host records | `hosts/*.yaml` (root) | `mcp/server.py` (`HOSTS_DIR`), `bin/homelab`; written by `mcp/build_host_files.py` |
| Substrate — kernel + context cards | `oikos/` (code, `oikos/cards/`, `oikos/state.json`) | MCP `explain`, scheduler |
| Narrative — synthesized wiki | `knowledge/wiki/{hosts,containers,vms,infrastructure}/` | humans, agents via MCP `get_page` / `search_docs` |
| Evidence — immutable sources | `knowledge/sources/` (references + investigations) | synthesis into wiki pages |
## Wiki pages
- **Node pages** (`knowledge/wiki/containers/<id>-<name>.md`, `.../vms/<id>-<name>.md`,
`.../hosts/<name>.md`) follow the container/host template in
[page-templates.md](../../shared/page-templates.md): opening definition, `## At a glance`,
`## Role`, service/port map, storage, auto-deploy, `## Related`, `## Changelog`.
- **Cross-cutting pages** (`knowledge/wiki/infrastructure/<topic>.md`) follow the cross-cutting
template: `## Why`, `## Components`, `## How to apply`, `## Gotchas`, `## Related`, `## Changelog`.
- Each `inventory.yaml` host entry carries a `doc_page:` field pointing at its narrative page.
Changing where a page lives means updating that field (read by `bin/homelab`).
## The two logs
- The per-page **`## Changelog`** records infrastructure changes and is machine-parsed
(`get_changelog`, the Oikos ledger). Keep the `### YYYY-MM-DD — title` shape.
- **`knowledge/log.md`** is append-only and records *documentation-maintenance* operations only
(restructures, source ingests, lint sweeps): `## [YYYY-MM-DD] <op> | <summary>`. It never
duplicates the Oikos change ledger (`oikos/ledger.py`).
## Same-session update rule
A change to a node updates every page that references it in the same session — the node page, the
section `README.md` table, the root `README.md`, the Caddy/DNS/ingress pages, the host page, and
`inventory.yaml`. See [page-templates.md](../../shared/page-templates.md#same-session-update-rule).

View File

@@ -0,0 +1,55 @@
# Operations domain — schema
The operations domain holds the procedural and time-stamped documentation: runbooks (repeatable
procedures), investigations (incident evidence), and plans (design docs for non-trivial work). It
follows [writing-style](../../shared/writing-style.md); runbooks and plans use the imperative voice
exception.
Where each kind lives: runbooks are skills under [`.agents/skills/`](../../skills/); operator
reference (command cheatsheet, enrollment, Hermes agent) lives in
[`.agents/operations/`](../../operations/); investigations are sources under
`knowledge/sources/investigations/`; plans stay in the repo-root `plans/` folder (below).
## Plans always live in `plans/`
**Any plan or design doc for the Homelab is written into the repo `plans/` folder as
`plans/YYYY-MM-DD-slug.md` — never a scratch path, an agent-private plan location, or a chat
message.** An agent drafting a plan:
1. Writes the file under `plans/` using the plan template in [page-templates.md](../../shared/page-templates.md).
2. Lists it in `plans/index.md`.
3. On completion, moves it to `plans/done/` and updates the index status.
This is the single source for homelab design intent; keeping it in-repo means the plan is
versioned, reviewable, and reachable by MCP `get_page`/`search_docs` like any other doc.
## Runbooks
Repeatable procedures are skills — one folder per skill at `.agents/skills/<name>/SKILL.md`, with
YAML front-matter that the Oikos policy and lifecycle machinery reads:
```yaml
---
name: <name>
risk_class: read_only | reversible_low | config_mutation | destructive
inputs: [<param>, ...]
verification: "<shell expression that proves success>"
docs_update_checklist: [<doc artifacts to update>]
transition: "<from> -> <to>" # only for lifecycle runbooks
---
```
`risk_class` values and the lifecycle `transition` states must match
[`oikos/policy.yaml`](../../../oikos/policy.yaml) and [`oikos/ontology.yaml`](../../../oikos/ontology.yaml).
## Investigations
Incident records live in `knowledge/sources/investigations/YYYY-MM-DD-slug.md` and are **evidence sources** — written
once at incident time, then linked from the changelogs of the nodes they implicate. Sections:
`## Summary`, `## Timeline`, `## Root cause`, `## Mitigations applied`, `## Open questions`. Resolved
incidents move to `knowledge/sources/investigations/archive/`.
## The operations log
`plans/log.md` and `knowledge/log.md` are append-only records of documentation operations on
those areas (`## [YYYY-MM-DD] <op> | <summary>`), distinct from the Oikos change ledger.

View File

@@ -6,7 +6,7 @@ this repo that auto-syncs every 5 min, a per-client age key for SOPS
decryption, the `homelab` CLI, and an MCP endpoint in Claude Code's config.
> Onboarding a Nous-Hermes-powered Goose agent on top of standard enrollment?
> See [hermes-agent.md](./hermes-agent.md). It uses the same `bootstrap.sh`
> See [hermes-agent.md](hermes-agent.md). It uses the same `bootstrap.sh`
> with an additional `--with-hermes` flag.
Architecture in [project_homelab_context_plan](https://… memory link); the
@@ -40,7 +40,7 @@ Bootstrap auto-installs netbird and drives `netbird up` if the mesh isn't alread
The new client runs bootstrap straight from a fresh OS. Bootstrap installs netbird (apt/dnf/brew based on the OS), then runs `netbird up --management-url https://netbird.hubris.network --ssh-jwt-cache-ttl 86400`. A device-code URL prints inline. The operator opens it (in a browser logged into Authentik), goes through identification → password → consent, and the CLI returns `Connected`. Bootstrap then proceeds with the rest of preflight.
Pre-condition: the operator must be a registered user in Authentik (typically the lab owner). The first user-login against a netbird account with existing peers is added as `pending_approval=1` and needs an sqlite promotion to `owner` — see [124-authentik.md First-time owner promotion gotcha](../containers/124-authentik.md). Only needed once per account.
Pre-condition: the operator must be a registered user in Authentik (typically the lab owner). The first user-login against a netbird account with existing peers is added as `pending_approval=1` and needs an sqlite promotion to `owner` — see [124-authentik.md First-time owner promotion gotcha](../../knowledge/wiki/containers/106-auth-outpost.md). Only needed once per account.
**Path A — setup-key (headless/scripted onboarding):**
@@ -62,7 +62,7 @@ Useful for headless servers (no browser at all) or unattended cloud-init bootstr
### DNS prerequisite
`*.hubris.network` resolves via the split-horizon dnsmasq on LXC 124
([dns.md](../infrastructure/dns.md)) for LAN clients, **but only if the
([dns.md](../../knowledge/wiki/infrastructure/dns.md)) for LAN clients, **but only if the
client uses 192.168.8.180 as its resolver**. Most LXCs and roaming
workstations don't by default. Options:
@@ -261,7 +261,7 @@ arguments.
If you also want the netbird `--ssh-jwt-cache-ttl` flag rationale to be
visible to the classifier (it's not actually durable in 0.71.2, but the
ControlMaster block is — see [runbook-dpkg-interrupted](runbook-dpkg-interrupted.md)
ControlMaster block is — see [runbook-dpkg-interrupted](../skills/runbook-dpkg-interrupted/SKILL.md)
for context), drop a free-text rule into `autoMode.allow` describing the
authorization. Optional.
@@ -342,7 +342,7 @@ The CLI prints a follow-up checklist that the operator must do manually:
| `homelab` CLI doesn't pick up repo updates | Pre-`02db…` bootstrap copied the binary instead of symlinking | One-time migration: `sudo ln -sfn /opt/homelab-context/bin/homelab /usr/local/bin/homelab`. New bootstraps use the symlink, which auto-tracks the synced repo. |
| `homelab-context-sync.service` journal shows `fatal: could not read Username for 'https://git.hubris.network'` | Pre-fix bootstrap set the gitea credential helper via `git config --global`, which writes to `/root/.gitconfig` — invisible to the systemd timer's git process (no HOME set). | One-time migration: `sudo git config --system credential.helper "store --file=/etc/homelab-context/git-credentials"`. New bootstraps store the helper in `/etc/gitconfig` instead. |
| Chat-mode `!` shell can't `sudo` (`a terminal is required to read the password`) | Claude Code's `!` invocation doesn't allocate a tty, and standard `sudo` won't read its password from stdin or a non-tty pipe. | Run the sudo'd command in a real terminal outside chat. For commands the agent issues repeatedly, configure passwordless sudo for the narrow set (e.g. `/etc/sudoers.d/homelab-self` with `<user> ALL=(ALL) NOPASSWD: /usr/bin/dnf upgrade -y, /usr/bin/apt-get *`). |
| `netbird status -d` reports `192.168.8.180:53 ... is Unavailable` but DNS actually works | netbird's UDP-53 probe times out over the relay latency (~90ms), but actual queries still flow through systemd-resolved. Cosmetic. | Ignore unless `dig @192.168.8.180 git.hubris.network` also fails — then check dnsmasq on [LXC 124](../containers/124-authentik.md). |
| `netbird status -d` reports `192.168.8.180:53 ... is Unavailable` but DNS actually works | netbird's UDP-53 probe times out over the relay latency (~90ms), but actual queries still flow through systemd-resolved. Cosmetic. | Ignore unless `dig @192.168.8.180 git.hubris.network` also fails — then check dnsmasq on [LXC 124](../../knowledge/wiki/containers/106-auth-outpost.md). |
| `netbird ssh` rejected with `JWT authentication failed: validate token (expected issuer=https://netbird.hubris.network/oauth2 ...)` | Peer's SSH JWT validator cached the OLD embedded-Dex issuer from before the 2026-05-21 Authentik migration. `systemctl restart netbird` and `netbird down/up` don't clear it — `client/internal/engine_ssh.go` bails out of `updateSSH()` if the SSH server is already running. | Full daemon bounce: `sudo systemctl stop netbird; sleep 3; sudo systemctl start netbird`. Verify with `grep -iE "issuer\|audience" /var/log/netbird/client.log \| tail`. Apply once per peer post-migration. |
| `netbird ssh` JWT passes but session closes with `user privilege check failed: user dtoro not found: unknown user dtoro` | netbird-ssh defaults the remote username to the LOCAL one (operator's laptop user). Hubris and LXCs only have `root`. | Always use explicit `root@` prefix manually: `netbird ssh -p 22022 root@proxmox-server.netbird.selfhosted`. `homelab ssh <host>` does this automatically via `inventory.yaml`'s per-host `ssh.user` field (defaults to `root`). |
| `homelab ssh hubris` (or any host on the LAN) fails with `Connection refused` or hangs, despite mesh routing being up | Off-LAN networks (operator on a VPN / coffee shop / symmetric NAT) sometimes can't reach the LAN IP even with the netbird subnet route. | Newer homelab CLIs probe the LAN with a 1.5s TCP connect and transparently fall back to the netbird FQDN. If your `/usr/local/bin/homelab` is a symlink to `/opt/homelab-context/bin/homelab` it'll pick up the fix on the next 5-min context sync. Otherwise pull the latest from gitea. |
@@ -356,7 +356,7 @@ and LAN IP registration. New workstations enrolled via this doc will
automatically join the universal SSH mesh.
### 2026-05-31 — cross-link to hermes-agent.md
Added a sibling page covering Nous-Hermes-on-Goose enrollment ([hermes-agent.md](./hermes-agent.md)) and noted it at the top of this page. The Hermes flow extends `bootstrap.sh` with `--with-hermes` and `homelab client add` with the same flag; it does not change the underlying enrollment steps documented here.
Added a sibling page covering Nous-Hermes-on-Goose enrollment ([hermes-agent.md](hermes-agent.md)) and noted it at the top of this page. The Hermes flow extends `bootstrap.sh` with `--with-hermes` and `homelab client add` with the same flag; it does not change the underlying enrollment steps documented here.
### 2026-05-21 — netbird-ssh JWT issuer + username + LAN-fallback troubleshooting rows
Added three rows to the troubleshooting table covering issues surfaced during the netbird vanilla migration: (1) post-migration SSH JWT validator cache stuck on old Dex issuer (full `systemctl stop/start` required, not `restart`), (2) `user not found` from netbird-ssh's local-username default (use explicit `root@`), and (3) homelab CLI's LAN→netbird-FQDN fallback for off-LAN operators. Companion code change: per-host `ssh.user` field in `inventory.yaml` + `homelab` CLI's `ssh_target()` helper.

View File

@@ -0,0 +1,91 @@
# Operations cheatsheet
Run from the [hubris host](../../knowledge/wiki/hosts/hubris.md) as root. When working from `/root` on Linux you're already on hubris — don't `ssh hubris` / `ping hubris`.
## Proxmox CLI
| Command | Use |
| --- | --- |
| `pct list` / `qm list` | List LXC containers / VMs |
| `pct config <id>` / `qm config <id>` | Container / VM config |
| `pct exec <id> -- <cmd>` | Run command inside an LXC without entering it (no initgroups — see [media permissions](../../knowledge/wiki/infrastructure/media-permissions.md)) |
| `pct enter <id>` | Shell into a container |
| `pct start <id>` / `pct stop <id>` | Boot / halt a container |
| `pvesm status` | Storage pools status |
| `pvesh get /nodes --output-format json` | Node summary as JSON |
| `pvesh get /nodes/hubris/lxc/<id>/status/current` | Live container status |
| `pvesh get /cluster/resources --type vm --output-format json` | Bulk per-LXC CPU/mem/disk (used by the `homelab-health-watchdog` Hermes cron — see [monitoring](../../knowledge/wiki/infrastructure/monitoring.md); the old `claudio-monitor` this once fed is deprecated) |
| `pveversion` | PVE version |
| `journalctl -u pve-cluster -n 100` | PVE service logs |
## Storage
- Shared mount: `/mnt/library` (ext4 on lvmthin `library`).
- Bind into a container: `pct set <id> -mp<N> /mnt/library/<sub>,mp=/data`
- For the standard whole-tree mount: `pct set <id> -mp0 /mnt/library,mp=/mnt/library`. See [media permissions](../../knowledge/wiki/infrastructure/media-permissions.md) for the GID-10000 onboarding recipe.
## Reverse proxy
- Caddyfile: `/etc/caddy/Caddyfile` on [LXC 121](../../knowledge/wiki/containers/121-caddy.md).
- **CRITICAL:** This file is tracked in `dtoro/caddy-conf` (https://git.hubris.network/dtoro/caddy-conf). Never edit it directly on the LXC — commit + push to the repo instead. Caddy auto-deploys on push (see [auto-deploy](../../knowledge/wiki/infrastructure/auto-deploy.md)). If you edit directly, the change will be lost on the next pull and agents won't know about it.
- Hot reload: `pct exec 121 -- systemctl reload caddy`.
- Validate: `pct exec 121 -- caddy validate --config /etc/caddy/Caddyfile`.
- Git workflow shortcut: `pct exec 121 -- "cd /etc/caddy && git add Caddyfile && git commit -m '...' && git push"`.
## DNS
- Split-horizon authority: [Technitium DNS](https://technitium.com) on [dns (107)](../../knowledge/wiki/containers/107-dns.md) at `192.168.8.2:53`. Web UI at `http://192.168.8.2`. (Formerly dnsmasq on the now-destroyed LXC 124 — decommissioned 2026-06-04.)
- Add/edit records in the Technitium UI; the NetBird managed zone sync (`scripts/dns-sync.py` cron on 107) picks changes up within ~10 minutes.
- Verify: `dig @192.168.8.2 +short <host>.hubris.network`.
- See [DNS](../../knowledge/wiki/infrastructure/dns.md).
## Web access
- `https://proxmox.hubris.network` or `https://192.168.8.77:8006` — Proxmox UI
## Telemetry quick checks
- `ras-mc-ctl --summary` — summary of any RAS events (memory / PCIe AER / thermal) since boot
- `ras-mc-ctl --errors` — full event log
- `cat /sys/devices/system/cpu/cpu0/cpufreq/energy_performance_preference` — should be `balance_power`
- `cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor` — should be `powersave`
- `ls /sys/fs/pstore/ /var/lib/systemd/pstore/` — panic traces from a previous crash (empty for pure hardware hangs — see [investigation](../../knowledge/sources/investigations/archive/2026-04-21-hubris-crash-loop.md))
## Fleet apt operations
Two `homelab` subcommands wrap the common patterns; both fan out to hubris + every LXC.
| Command | What it does |
| --- | --- |
| `homelab apt-audit [--target HOST]` | Per-host table: dpkg-interrupted state, holds, upgradable count, non-apt binaries in system paths, DNS health. Exits nonzero if any host has dpkg-interrupted state. |
| `homelab apt-upgrade --target HOST` | Launch `apt update && apt upgrade` inside a transient `systemd-run --collect` unit on the target. Survives ssh teardown. Apt configured with `Acquire::Retries=3` + `ForceIPv4=true`. |
| `homelab apt-upgrade --all` | Same, fanned out across the standard targets. |
| `homelab apt-upgrade ... --status` | Show running unit + tail `/var/log/homelab-apt-upgrade.log` on each target. |
| `homelab apt-upgrade ... --safe` | Take a pre-upgrade snapshot per LXC first (`pct snapshot``vzdump` fallback for bind-mounted LXCs). Refuses if any snapshot fails unless `--force`. |
| `homelab apt-upgrade ... --force` | Skip both the dpkg-audit gate and snapshot-failure refusal. |
PVE/kernel deferral on hubris: `homelab apt-upgrade --target hubris` will try every upgrade, including kernel + `pve-*`. To skip those, `apt-mark hold` the relevant packages on hubris first; `homelab apt-audit` shows held packages so you can confirm.
## Oikos (agent OS layer)
See [OIKOS.md](../OIKOS.md) for the operating model. Quick reference:
| Command | What it does |
| --- | --- |
| `homelab service <name> explain\|health\|docs\|log\|actions\|history` | Service Console v0 — context card, cached health (`--live` to force a probe), docs, logs, safe actions + risk class, ledger history |
| `homelab node <name> relations` | Ontology blast-radius query: what this host/service impacts, is affected by, and its full transitive blast radius |
| `homelab change preflight <service>` | Dry-run report before mutating: risk class, current health, config repo, verification command |
| `homelab decide <action> <entity>` | Decision classifier: risk × blast radius × confidence → auto-act or escalate |
| `homelab signal list\|raise\|ack\|resolve\|mute` | The attention layer — pending updates, thresholds, drift, anything needing attention |
| `homelab approval request\|list\|reply\|check` | Escalate-route grants (Matrix-delivered via Hermes, or the Oikos Console's `/approvals` page) |
| `homelab restart <service> [--approval-id <id>]` | `--approval-id` is required whenever the service's risk class needs approval (e.g. `caddy`, `dns`) — refuses mechanically without a valid grant |
Oikos Console (read-mostly dashboard): `oikos.hubris.network` once deployed — see [oikos/console/deploy/README.md](../../oikos/console/deploy/README.md).
## Related
- [Hubris host](../../knowledge/wiki/hosts/hubris.md)
- [Containers index](../../knowledge/wiki/containers/index.md)
- [DNS](../../knowledge/wiki/infrastructure/dns.md)
- [Monitoring](../../knowledge/wiki/infrastructure/monitoring.md)
- [Auto-deploy](../../knowledge/wiki/infrastructure/auto-deploy.md)
- [Runbook: dpkg-interrupted recovery](../skills/runbook-dpkg-interrupted/SKILL.md) — what to do when apt got killed mid-transaction

View File

@@ -2,7 +2,7 @@
Onboards [Nous Research's Hermes](https://nousresearch.com/) (a fine-tuned
Llama variant) as a working terminal agent on a homelab client. Builds on top
of standard client enrollment (see [agent-enrollment.md](./agent-enrollment.md))
of standard client enrollment (see [agent-enrollment.md](agent-enrollment.md))
— this page covers only the Hermes-specific additions.
The agent runs as a [Goose](https://goose-docs.ai/) session. Goose provides:
@@ -21,7 +21,7 @@ The persona is `/opt/homelab-context/HERMES.md`, symlinked as Goose's global
| Requirement | How |
| --- | --- |
| Standard enrollment complete (`homelab whoami` works) | [agent-enrollment.md](./agent-enrollment.md) |
| Standard enrollment complete (`homelab whoami` works) | [agent-enrollment.md](agent-enrollment.md) |
| `secrets/openrouter-api-key.yaml` exists with a real `sk-or-...` value | See "Seeding the OpenRouter key" below |
| The host's `age_pubkey` is on the openrouter-api-key.yaml sops rule | `homelab client add <host> --finalize-pubkey <age1...> --with-hermes` |
@@ -165,13 +165,13 @@ every tool call, use `approve`. See
## Cross-references
- [agent-enrollment.md](./agent-enrollment.md) — base client onboarding the
- [agent-enrollment.md](agent-enrollment.md) — base client onboarding the
Hermes flow assumes is done.
- [`HERMES.md`](../HERMES.md) — the persona the Hermes agent reads on every
session start (via `~/.config/goose/.goosehints`).
- [`bin/hermes`](../bin/hermes) — the wrapper that decrypts the OpenRouter key
- [`bin/hermes`](../../bin/hermes) — the wrapper that decrypts the OpenRouter key
and execs `goose session`.
- [`bootstrap.sh`](../bootstrap.sh) — the `--with-hermes` flag's install block.
- [`bootstrap.sh`](../../bootstrap.sh) — the `--with-hermes` flag's install block.
## Follow-ups

View File

@@ -0,0 +1,41 @@
# LLM Wiki — the documentation contract
How the narrative documentation in this repo is organized. The pattern is borrowed from the
`sources / wiki / index / log` model: a durable synthesized layer (`knowledge/wiki/`) built on top
of immutable evidence (`knowledge/sources/`, incident records), with pure-listing indexes and an
append-only operations log.
This contract governs the **narrative layer only**. The machine-readable substrate — `inventory.yaml`,
generated `hosts/*.yaml`, `oikos/`, `mcp/`, `secrets/`, `bin/` — is not part of the wiki and never
moves under it. See [the knowledge schema](../domains/knowledge/schema.md) for the split.
## Layers
- **Sources** are immutable raw material: incident records (`knowledge/sources/investigations/`), external reference
docs (`knowledge/sources/references/`), and the live system itself (`pct config`, `docker inspect`).
Read them; do not rewrite them into other sources.
- **Wiki** (`knowledge/wiki/`) is the synthesized, authoritative current-state layer: one page per
node (`containers/`, `vms/`, host narratives) and per cross-cutting system (`infrastructure/`). A
reader understands the topic from the wiki page without reading the sources.
- **Index** (`index.md` / folder `README.md`) is a pure listing — every page in scope with a
one-line summary, and nothing else. Anything the section wants to say up front goes into a page
the index lists, not into the index.
- **Log** (`log.md`) is append-only, recording *doc-maintenance operations* (restructures, source
ingests, lint sweeps) in single-line format: `## [YYYY-MM-DD] <op> | <summary>`.
## Two logs, kept distinct
- **`## Changelog`** on each node/topic page records *infrastructure* changes to that node. It is
machine-parsed (`get_changelog`, the Oikos ledger) — keep the `### YYYY-MM-DD — title` shape.
- **`log.md`** per area records *documentation* operations only. It never duplicates the Oikos
change ledger (`oikos/ledger.py`), which stays authoritative for infra changes with
who/what/risk/approval/verification.
## Rules
- Wiki pages stay short and focused. A page past ~300 lines splits.
- Pages stay flat under `wiki/<section>/` until there are enough to warrant a sub-group.
- Every page follows [writing-style.md](writing-style.md).
- Plans and design docs always live in the repo `plans/` folder (`plans/YYYY-MM-DD-slug.md`),
listed in `plans/index.md`, moved to `plans/done/` on completion — never a scratch path or a chat
message. See [the operations schema](../domains/operations/schema.md).

View File

@@ -0,0 +1,174 @@
# Page templates for the Homelab Wiki
The structural templates for each page type. Prose voice, vocabulary, and cross-reference rules live
in [writing-style.md](writing-style.md); the layer model (sources / wiki / index / log) lives in
[llm-wiki.md](llm-wiki.md).
## File naming
**Foundational / entry-point files:** ALL-CAPS
- **Root level:** `AGENTS.md`, `README.md` — discovery paths for agents and humans.
- **Agent instruction** (under `.agents/`): `OIKOS.md`, `HERMES.md` — foundational docs agents read before acting.
- **Reference docs:** `GLOSSARY.md` — lookup reference (like classic repo conventions: LICENSE, CHANGELOG, GLOSSARY).
**Content / narrative pages:** lowercase-with-dashes, date-prefixed as needed
- **Container pages:** `<id>-<name>.md` (e.g. `101-jellyfin.md`, `132-rclone.md`). The `<id>` is the LXC/VM ordinal from `inventory.yaml`.
- **Infrastructure / cross-cutting pages:** `<topic>.md` (e.g. `dns.md`, `auto-deploy.md`, `mesh.md`). Describes a system, not a specific node.
- **Plans / investigations:** `YYYY-MM-DD-<slug>.md` (e.g. `2026-07-05-oikos-prometheus-lxc.md`). Date-sorted; slug is lowercase.
- **Section indices:** `README.md` (lowercase, conventional). Prefer in folders; `index.md` only if both intro prose and listing coexist.
**Skills / runbooks:** special case
- **Folder structure:** `<name>/SKILL.md` where `<name>` is lowercase-with-dashes (e.g. `client-enrollment/SKILL.md`).
- **The filename SKILL.md is always uppercase** — it acts as a signpost so tools and humans instantly recognize it as a skill.
**General rules:** All paths use lowercase letters, numbers, and hyphens (no underscores). Uppercase is reserved for foundational docs (entry points + instruction) and filenames that signify document type (SKILL.md, GLOSSARY.md, etc.).
## Voice
Concise, technical, sysadmin-to-sysadmin. No marketing prose, no exclamation marks. Full rules in
[writing-style.md](writing-style.md).
## Page templates
### Container page (`containers/<id>-<name>.md`)
```markdown
# <id> — `<name>`
One-sentence purpose.
## At a glance
- **Hostname:** `<name>`
- **IP:** `192.168.8.x`
- **Privilege:** privileged | unprivileged
- **Resources:** N cores / M GiB RAM / D GiB rootfs
- **Mounts:** `/mnt/library``/mnt/library` (if any)
- **Public hostname:** `<sub>.hubris.network` (if proxied)
## Role
What it does, what it talks to.
## Service / port map
| Service | Listen | Notes |
## Storage / config paths
## Auto-deploy
(if any) — link to [auto-deploy](../infrastructure/auto-deploy.md)
## Related
- [Caddy](121-caddy.md) (if proxied)
- [DNS](../infrastructure/dns.md) (if has subdomain)
- [Authentik](124-authentik.md) (if SSO)
- ...
## Changelog
### YYYY-MM-DD — short title
What changed, why, link to investigation if any.
```
### Cross-cutting page (`infrastructure/<topic>.md`)
```markdown
# <Topic>
One-sentence summary.
## Why
Design rationale — what it replaces, what it solves.
## Components
Where it runs, what files matter.
## How to apply / use
Recipes.
## Gotchas
## Related
Links to nodes that host or depend on this.
## Changelog
```
### Plan (`plans/YYYY-MM-DD-slug.md`)
```markdown
# YYYY-MM-DD — <title>
## Goal
What this change achieves and why.
## Current topology / state
Diagram or description of what exists now.
## Target topology / state
What it looks like after.
## Pre-flight checklist
## Step-by-step procedure
## Verification
## Post-migration
Changelog entries to write, index status to update.
```
### Investigation (`knowledge/sources/investigations/YYYY-MM-DD-slug.md`)
```markdown
# YYYY-MM-DD — <title>
## Summary
1-3 sentences.
## Timeline
## Root cause
## Mitigations applied
## Open questions
```
## Linking discipline
- Every container page links to every cross-cutting page it participates in.
- Every cross-cutting page lists the nodes that participate.
- Every investigation links to the nodes it implicates *and* gets back-linked from each node's changelog.
- Every plan links to the infrastructure pages it affects. When done, update the plan's status in `plans/index.md` and write changelog entries on affected node pages.
## Changelog hygiene
- Reverse-chronological (newest first).
- One entry per discrete change, even if you make several in one day.
- If a change spans nodes, repeat the entry on each affected page (different perspective is fine).
- Don't rewrite history — entries are append-only. Mistakes get a follow-up entry that supersedes them.
## Same-session update rule
When you make a change to a node — migrate an LXC, update an IP, change a
mount, deploy a new service — **update every relevant doc page in the same
session.** A change that touches a container page must also update:
- The `containers/index.md` table (IPs, host, mounts, status)
- The `README.md` table (if the change affects listed columns)
- The Caddy page site list (if the change affects `*.hubris.network` routing)
- The DNS / ingress infrastructure pages (if the change affects routing)
- The `hosts/{hubris,strong}.md` host page (if container count changes)
- The `inventory.yaml` host entry (source of truth for the `hosts/*.yaml` generation)
- The `infrastructure/topology.md` (generated from inventory, but regen if needed)
The pattern of updating only one page and leaving stale references on others
is a bug. If you're doing a multi-step migration, document the intermediate
state with a changelog entry that says "pending — will finalize after Phase
N."
This rule is why Phase 2 of the strong migration (2026-07-05) caused
widespread stale data: individual container pages were updated in the
changelog but never had their At-a-glance sections, IPs, mount paths, or
host attribution updated. Don't repeat that.

View File

@@ -0,0 +1,75 @@
# Writing Style
Write like a technical reference, not a marketing page. Every sentence conveys new information.
These rules govern **committed documentation** — wiki pages, READMEs, schemas, skills, `AGENTS.md`,
plans, investigations, and code comments. They are separate from [caveman.md](caveman.md), which
governs an agent's *chat responses*; the two do not conflict.
New or rewritten pages follow these patterns from day one. Existing pages get updated the next time
they are touched.
## Vocabulary — never use these
- Significance puffers: "pivotal", "crucial", "vital", "groundbreaking", "transformative", "testament", "paramount", "invaluable".
- Analytical verbs: "delve", "leverage", "utilize", "facilitate", "foster", "showcase", "underscore", "streamline", "harness".
- Poetic nouns: "tapestry", "landscape" (figurative), "realm", "paradigm", "ecosystem" (figurative), "journey" (figurative), "nexus", "cornerstone".
- Promotional adjectives: "robust", "seamless", "innovative", "cutting-edge", "meticulous", "holistic", "comprehensive".
- Opening crutches: "In today's world", "In the ever-evolving landscape of", "It's worth noting that", "It is important to note that".
Use short, common words: "use" not "utilize", "help" not "facilitate", "show" not "demonstrate".
## Voice
Describe what systems do and how they work.
- **Reference prose** (node pages, cross-cutting infrastructure descriptions, `## Role`, `## Why`,
`At a glance`) is third-person: state facts about the system, not instructions to a reader.
- **Recipes, runbooks, and skills** are the exception: second-person imperative is allowed and
preferred where it makes a procedure clearer ("Edit the Caddyfile, commit + push", "Verify with
`dig +short`"). This matches how the operator actually works. The vocabulary, structure, and
cross-reference rules below still apply.
## Page shape
Every doc-level page follows the same shape so a reader scans it in one pass.
1. **One H1 = the page title.** Node pages use `# <id> — \`<name>\``; topic pages use `# <Topic>`.
2. **Opening definition.** First paragraph, 13 sentences, says what the thing is. No motivation, no marketing, no setup.
3. **Body sections** in the natural order for the topic. Reuse the section templates in [page-templates.md](page-templates.md).
4. **`## Changelog`** at the bottom of every node/topic page — reverse-chronological, append-only. This section is machine-parsed (`get_changelog` in `mcp/server.py`); keep the `### YYYY-MM-DD — title` shape.
5. **Related links** only at the bottom, only when a reference cannot be woven inline.
## Section indexes (folder READMEs)
A folder's `README.md` opens with a 13 sentence prose intro that says what the section covers, then
a single navigation table — `| Document | What it covers |` — and nothing else. No stale counts, no
duplicated prose, no narrative between the intro and the table.
## Structure rules
- Make every sentence information-dense. Cut filler, qualifiers, and setup phrases. Lead with the concrete fact or action, not why it matters.
- No participial tack-ons (", highlighting the importance of…"). If the clause adds information, make it a separate sentence.
- **No meta-commentary about the content itself.** Do not narrate the page's own structure or linking strategy.
- Prefer **tables** for enumerable items with internal structure (service/port maps, field lists, status grids). Reserve bullets for short non-structured lists.
- Use the **bold-leading-phrase pattern** for structured points: `**Read-only by construction.** The MCP server never mutates state.` — a bold noun phrase, a period, then the explanation.
- When enumerating across services or nodes, give each its own `###` sub-section or a table row, not one run-on paragraph.
- Use backticks for code, paths, hostnames, and file names (`inventory.yaml`, `192.168.8.77`, `pct config`); italics for first-mention terminology.
- Use `>` blockquotes for caveats and gaps that interrupt the main flow: `> **Outstanding gap.** DNS-vs-inventory drift check not yet wired.` One thought per blockquote.
## Diagrams
- Mermaid is the default for topology and flow diagrams. `infrastructure/topology.md` is generated by `oikos/gen-topology.py` — do not hand-edit it.
- ASCII box diagrams are fine for small shape diagrams; keep them to one screen.
## Sourcing and cross-references
- **Factual discipline.** Every claim is grounded in a cited source, an adjacent linked page, or a directly observable fact (`pct config`, `docker inspect`, running config). Do not write sentences that sound sourced but are inference. When docs disagree with live state, fix the doc and note it in the changelog.
- **One-sided cross-references.** When two pages relate, the link lives in the page where the connection makes organizational sense. Do not add a back-pointer unless that direction also carries content the reader needs.
- **Cross-references are content, not catalog.** Inline links arise from the surrounding prose; the linked page must be needed to understand the current sentence. A bottom-of-page "Related" list is the fallback, not the default.
- Pages link with standard relative markdown links (e.g. a container page links to `../infrastructure/dns.md`), forming a navigable graph. Orphans are a bug.
## Code comments and commit/PR prose
- Comments explain intent, trade-offs, or constraints the code cannot convey. No diff narration, no type restatement, no section-divider comments.
- Commit messages and PR descriptions are problem → change → risk → verification, not a file-by-file diff restatement.
- The banned vocabulary applies the same way in comments and commit messages.

View File

@@ -0,0 +1,43 @@
---
name: client-enrollment
risk_class: config_mutation
inputs: [hostname, kind, role]
verification: "homelab doctor (on the new client)"
docs_update_checklist: [hosts_narrative_page_if_lxc_or_vm]
---
# Client enrollment
Goal: bring a new host (workstation, LXC, VM) into inventory and the
secrets model, with mesh membership only where it's actually needed.
This wraps the existing `homelab client add` flow — see
[operations/agent-enrollment.md](../../operations/agent-enrollment.md) for
the full walkthrough; this runbook is the risk/lifecycle framing.
1. On any enrolled client: `homelab client add <hostname>` — appends a
`hosts.<name>:` block to `inventory.yaml` (lifecycle `state: planned`
`provisioning`, per [oikos/ontology.yaml](../../../oikos/ontology.yaml)),
commits + pushes.
2. Netbird join is **optional, not a required step** — only needed for
hosts that must be reachable off-LAN (workstations that roam, e.g.
`republic-laptop`, `mac-mini`). A node reachable on the household LAN
(192.168.8.0/24 — most LXCs/VMs) doesn't need it: it's already
reachable directly, and off-LAN clients reach it too via hubris's
routed `192.168.8.0/24` Netbird network resource. Skip this step for
LAN-only nodes; do it (out-of-band, console or setup key) only for
hosts that need independent off-LAN reachability.
3. On the new host: run `bootstrap.sh` (add `--with-hermes` to also
enroll the Hermes agent). This provisions `/etc/age/key.txt`, the
sync timer, and prints an age pubkey.
4. Back on an enrolled client: `homelab client add <hostname>
--finalize-pubkey <age1...>` — sets `age_pubkey`, grants shared
secrets, re-keys SOPS, commits + pushes. This is the
`provisioning → active` transition.
5. Verify: `homelab doctor` on the new client should show all checks
green (clone, sync timer, age key, CLI symlink, MCP reachable).
Docs-update checklist: if the new host is an LXC/VM, add its narrative
page under `containers/` or `vms/` and set `doc_page` in its inventory
entry (host-level cards don't have a `doc_page` field yet — services do;
narrative pages are still found via the generated `see_also` in
`hosts/<name>.yaml`).

View File

@@ -0,0 +1,34 @@
---
name: config-change-deploy
risk_class: config_mutation
inputs: [service_name, change_description]
verification: "curl -sf <service_url> (or homelab service <name> health)"
docs_update_checklist: [doc_page, changelog]
---
# Config change + deploy
Goal: change a tracked config repo (Caddy, Gitea customizations, an app's
own repo) and get it live, safely.
1. `homelab change preflight <service>` — current health, the service's
`config_repo`, its risk class, and the verification command to run
after. If risk class requires approval (`config_mutation` or
`destructive`), stop and get operator sign-off before editing — see
`oikos/policy.yaml`.
2. Clone/pull the `config_repo` (never edit the backend's working tree
directly — tracked configs change by commit + push, per
[OIKOS.md](../../OIKOS.md) conventions).
3. Make the change, commit, push to `main`.
4. The Gitea webhook fires the deploy pipeline for that repo (see
[infrastructure/auto-deploy.md](../../../knowledge/wiki/infrastructure/auto-deploy.md) for
the exact receiver/reload for this service).
5. Run the preflight's verification command. If it fails, check
`homelab service <name> log` for the reload/restart error.
6. Record the change: once `oikos/ledger.py` is wired into deploy tooling
(Week 3), this is automatic; until then, note the change and outcome
in the relevant investigation/plan doc.
Docs-update checklist: update the service's `doc_page` if the change
alters its behavior, ingress route, or ownership; add a changelog entry
if the page has one.

View File

@@ -0,0 +1,25 @@
---
name: docs-lint
risk_class: read_only
inputs: [paths]
verification: "python3 .agents/skills/docs-lint/lint.py"
docs_update_checklist: []
---
# Docs lint
Check committed documentation against the mechanical rules in
[writing-style.md](../../shared/writing-style.md): banned vocabulary and broken relative markdown
links. Prose-voice rules are not machine-checkable — those stay a review responsibility.
Run from the repo root:
python3 .agents/skills/docs-lint/lint.py # default: knowledge/ .agents/ operations/ investigations/ plans/
python3 .agents/skills/docs-lint/lint.py knowledge/wiki/containers/104-gitea.md
Exit code is non-zero when any violation is found, so it can gate a commit. The banned-vocabulary
list mirrors `writing-style.md`; update both together if the standard changes.
> **Known baseline.** `knowledge/wiki/containers/101-jellyfin.md` links into a sibling repo
> (`devops/homelab-authentik-admin`) that this checkout does not contain — expected, not a bug.
> Any other broken link is a real regression; investigate before dismissing it as baseline noise.

View File

@@ -0,0 +1,69 @@
#!/usr/bin/env python3
"""Lint committed docs against .agents/shared/writing-style.md.
Checks two mechanical rules:
1. Banned vocabulary (significance puffers, analytical verbs, poetic nouns,
promotional adjectives, opening crutches).
2. Broken relative markdown links.
Prose-voice rules are not machine-checkable; this covers the parts that are.
Run from the repo root: python3 .agents/skills/docs-lint/lint.py [paths...]
Exit 1 if any violation is found.
"""
import os, re, sys
BANNED = [
"pivotal", "crucial", "vital", "groundbreaking", "transformative", "testament",
"paramount", "invaluable", "delve", "leverage", "utilize", "facilitate", "foster",
"showcase", "underscore", "streamline", "harness", "tapestry", "realm", "paradigm",
"nexus", "cornerstone", "robust", "seamless", "innovative", "cutting-edge",
"meticulous", "holistic", "comprehensive", "in today's world",
"it's worth noting", "it is important to note",
]
BAN_RE = re.compile(r'(?<![\w-])(' + "|".join(re.escape(w) for w in BANNED) + r')(?![\w-])', re.I)
LINK = re.compile(r'\]\(([^)]+)\)')
def iter_md(paths):
for p in paths:
if os.path.isfile(p) and p.endswith(".md"):
yield p
for root, dirs, files in os.walk(p):
dirs[:] = [d for d in dirs if d not in (".git", "node_modules")]
for f in files:
if f.endswith(".md"):
yield os.path.join(root, f)
def main(argv):
paths = argv or ["knowledge", ".agents", "operations", "investigations", "plans"]
violations = 0
# The style guide and this skill enumerate the banned words by definition.
ban_exempt = ("shared/writing-style.md", "skills/docs-lint/")
for f in sorted(set(iter_md(paths))):
check_banned = not any(x in f for x in ban_exempt)
fence = False
with open(f) as fh:
for ln, line in enumerate(fh, 1):
if line.lstrip().startswith("```"):
fence = not fence; continue
if fence:
continue
if check_banned:
for m in BAN_RE.finditer(line):
print(f"{f}:{ln}: banned word '{m.group(1)}'")
violations += 1
for m in LINK.finditer(line):
link = m.group(1)
if re.match(r'^(https?:|mailto:|#|/)', link):
continue
path = re.split(r'[#?]', link)[0]
if not path:
continue
tgt = os.path.normpath(os.path.join(os.path.dirname(f), path))
if not os.path.exists(tgt):
print(f"{f}:{ln}: broken link -> {link}")
violations += 1
print(f"\n{violations} violation(s)")
return 1 if violations else 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

View File

@@ -0,0 +1,34 @@
---
name: incident-investigation
risk_class: read_only
inputs: [symptom, affected_entity]
verification: "n/a — investigation produces a written record, not a state change"
docs_update_checklist: [investigations_entry]
---
# Incident investigation
Goal: understand what broke and why, before touching anything.
1. `homelab service <name> explain` (or `homelab node <name> relations`
if the affected entity is a host) — get the blast radius and doc
pointer first. Don't start pulling logs blind.
2. `homelab service <name> health` + `homelab service <name> log` (or
MCP `get_service_status` / `tail_log`) for the affected service.
3. Walk the blast radius: is a shared dependency down (`caddy`, `dns`,
`authentik`, or the backend host itself)? `homelab node <name>
relations` shows "affected by" — check those first.
4. `homelab apt-audit` if the symptom looks like a dpkg/upgrade
interaction.
5. Check the change ledger for recent mutations to the affected entity
or anything upstream of it: `homelab service <name> history` (once
populated) or grep `ledger/*.jsonl`.
6. Write findings to a new `knowledge/sources/investigations/<date>-<slug>.md` — symptom,
timeline, root cause, fix applied, prevention. This is the durable
record; don't rely on chat history.
Docs-update checklist: always create the investigation entry. If the
root cause was stale/wrong inventory data (a `doc_page`, `config_repo`,
or `backend` that didn't match reality — this happened during Week 1
kernel work, see the `authentik` backend fix), correct `inventory.yaml`
in the same session.

View File

@@ -0,0 +1,36 @@
---
name: lifecycle-activate-node
risk_class: config_mutation
inputs: [node_name]
verification: "homelab service <name> health (if it hosts a service); homelab doctor (if it's a client)"
docs_update_checklist: [doc_page_complete]
transition: "provisioning -> active"
---
# Lifecycle: activate a node
Per [oikos/ontology.yaml](../../../oikos/ontology.yaml). Requires: age key
enrolled if it needs secrets, mesh joined if it needs off-LAN reach,
ingress live if public, health check answering, doc page complete,
ledger entry.
1. If the node is a `homelab` client: finish enrollment per
[client-enrollment.md](../client-enrollment/SKILL.md) (`--finalize-pubkey`,
mesh join, `homelab doctor` green).
2. If it hosts a public service: add the `services:` entry in
`inventory.yaml` (backend, url, doc_page, config_repo, risk_notes —
see the Week-1 service contract fields) and wire the Caddy route in
`dtoro/caddy-conf`.
3. Confirm the health check answers: `homelab service <name> health` or
a direct `curl`.
4. Flip `state: provisioning``state: active` (or delete the `state:`
field — `active` is the default) in `inventory.yaml`.
5. Complete the doc page (stub → full narrative: role, specs, how it's
configured, dependencies).
6. Record the activation: `oikos/ledger.py append host:<name> activate
config_mutation --result ok` (or let the CLI wrapper do this once
Week 3's runbook automation lands).
Regenerate derived data: `python3 mcp/build_host_files.py && python3
oikos/gen-topology.py` so `hosts/<name>.yaml`, the topology diagram, and
the context card all reflect the new state.

View File

@@ -0,0 +1,35 @@
---
name: lifecycle-deprecate-node
risk_class: config_mutation
inputs: [node_name, replacement_node_or_reason]
verification: "homelab node <name> relations — 'affected by' must be empty before completing"
docs_update_checklist: [doc_page_deprecation_note]
transition: "active -> deprecated"
---
# Lifecycle: deprecate a node
Per [oikos/ontology.yaml](../../../oikos/ontology.yaml): a node keeps running
but takes no new dependents. **Completion condition: zero remaining
inbound `depends-on`/`routes-to` edges** — this is a hard gate, not a
suggestion; `oikos/policy.yaml` `lifecycle_overrides.deprecated.refuse`
lists `new-inbound-edges` as refused going forward.
1. Set `state: deprecated` on the node.
2. `homelab node <name> relations` — read `affected_by`. Every entry
there is something still relying on this node.
3. Migrate or retire each dependent one at a time (point its `backend`/
`config_repo`/ingress route elsewhere, or deprecate it too if it's
being retired alongside).
4. Re-run `homelab node <name> relations` after each dependent is moved.
The transition to `destroyed` is only safe once `affected_by` is
empty — check this every time, don't assume from memory.
5. Note the deprecation on the doc page: reason, replacement (if any),
date.
If step 2 shows dependents you didn't expect, stop and investigate
before proceeding — that's exactly the kind of drift the Week-3 detector
will catch automatically, but until then this manual check is the gate.
Next (once `affected_by` is empty):
[lifecycle-destroy-node.md](../lifecycle-destroy-node/SKILL.md).

View File

@@ -0,0 +1,42 @@
---
name: lifecycle-destroy-node
risk_class: destructive
inputs: [node_name]
verification: "homelab node <name> relations returns unknown-entity; pct list on the backend no longer shows it"
docs_update_checklist: [archaeology_entry, containers_index_update]
transition: "deprecated -> destroyed"
---
# Lifecycle: destroy a node
**Destructive.** Requires operator approval + typed confirmation phrase
per `oikos/policy.yaml`. Requires (ontology): backups verified, secrets
recipients removed + re-keyed, ingress/DNS removed, archaeology entry,
ledger entry.
1. Confirm the node is `deprecated` with zero `affected_by` edges
(`homelab node <name> relations`) — do not skip this even if the
deprecation runbook was followed recently; state can drift.
2. If it's an enrolled client: `homelab client remove <name>` — revokes
the age key, re-keys SOPS, removes the inventory entry. This is
already destructive-class and confirmed in the CLI.
3. Remove any ingress route (Caddy config repo) and DNS record still
pointing at it.
4. Verify backups of anything on it are retained per policy before the
disk goes away (see `backs-up-to`).
5. Destroy the LXC/VM (`pct destroy` / `qm destroy`).
6. Move the `hosts.<name>:` block (if any inventory remnant survives
`client remove`, e.g. infra-only LXCs with no age key) into
inventory.yaml's `archaeology:` section: `pve_id`, `destroyed` date,
`reason`. Add a row to `containers/index.md` "Recently destroyed"
table (kept for human-readable browsing alongside the structured
data).
7. `oikos/ledger.py append host:<name> destroy destructive --result ok`.
8. Regenerate: `python3 mcp/build_host_files.py && python3
oikos/gen-topology.py` — the node drops out of `hosts/*.yaml` and
appears in the topology doc's archaeology table.
If the destroy fails partway (e.g. secrets revoked but pct destroy
errors), do not re-run step 2 — `client remove` is not idempotent
against a second revocation attempt on the issuance server. Finish the
remaining steps manually and note the partial state in an investigation.

View File

@@ -0,0 +1,39 @@
---
name: lifecycle-migrate-node
risk_class: config_mutation
inputs: [node_name, source_host, target_host]
verification: "homelab node <name> relations (re-check blast radius); homelab service <svc> health for every hosted service"
docs_update_checklist: [doc_page_migration_note, inventory_host_and_lan_ip]
transition: "active -> migrating -> active"
---
# Lifecycle: migrate a node
Modeled on the strong Phase 1+2 migration
([plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md](../../../.hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md)).
Requires (ontology): preflight + backup-verified before migrating;
post-verify + Caddy backends checked + mounts checked + docs updated
before returning to `active`.
1. `homelab change preflight <every service the node hosts>` — capture
current health as a baseline.
2. Verify backups are current for anything with data at rest on the
node (see `backs-up-to` edges once populated).
3. Set `state: migrating` in `inventory.yaml`.
4. Perform the migration (pct/qm move, or create-on-target +
data-copy + destroy-source, per the specific case).
5. Update `inventory.yaml`: new `host:`, `lan_ip`, `mesh` addresses for
the node; update every `services:` entry whose `backend` pointed at
it if the backend name itself changes (usually it doesn't — only the
`host:`/`lan_ip` on the guest entry moves).
6. Post-verify: re-run the Week-1 drift check by hand — confirm Caddy's
backend IP for each affected service matches the new `lan_ip`
(automatic in Week 3's drift detector), confirm mounts still resolve.
7. `homelab service <name> health` for every service the node hosts.
8. Set `state: active`. Add a migration note to the node's doc page
(old host/IP → new, date, phase reference) — this repo's convention
for every past migration (see `containers/101-jellyfin.md`,
`containers/129-house.md`).
Regenerate: `python3 mcp/build_host_files.py && python3
oikos/gen-topology.py`.

View File

@@ -0,0 +1,33 @@
---
name: lifecycle-provision-node
risk_class: config_mutation
inputs: [node_name, kind, storage_pool]
verification: "grep 'state: provisioning' hosts/<name>.yaml"
docs_update_checklist: [doc_page_stub]
transition: "planned -> provisioning"
---
# Lifecycle: provision a node
Per [oikos/ontology.yaml](../../../oikos/ontology.yaml) `lifecycle.transitions`.
Policy note: `provisioning` nodes get a lifecycle override —
`config_mutation` actions downgrade to `reversible_low` because nothing
depends on the node yet (see `oikos/policy.yaml` `lifecycle_overrides`).
Requires (from ontology): inventory entry, IP reserved, storage pool
chosen, doc page stub.
1. Create the LXC/VM on its target Proxmox host (`pct create` /
`qm create`), choosing the storage pool deliberately — record it as
the `storage:` field once populated (Week 1 schema; not yet backfilled
for existing nodes).
2. Add the inventory entry: `homelab client add <name>` for anything that
will run the `homelab` CLI, or a direct `hosts.<name>:` block with
`state: provisioning`, `kind`, `host`, `pve_id`, `lan_ip` for
infra-only LXCs that won't self-enroll.
3. Stub the doc page (`containers/<pve_id>-<name>.md` or
`vms/<pve_id>-<name>.md`) — even a one-line "provisioning, see plan X"
is enough to satisfy the transition requirement.
4. Reserve the IP in DNS/DHCP notes if it's a fixed LAN address.
Next: [lifecycle-activate-node.md](../lifecycle-activate-node/SKILL.md).

View File

@@ -1,3 +1,10 @@
---
name: budget-import-from-csv
risk_class: config_mutation
inputs: [csv_file]
references: [containers/129-house.md]
---
# Runbook: Budget import from N26 CSV → Yuvomi
Distil a bank-export CSV into Yuvomi's Budget and Subscriptions modules using

View File

@@ -1,3 +1,9 @@
---
name: recover-dpkg-interrupted
risk_class: reversible_low
verification: "dpkg --audit (should be clean); apt-get check"
---
# Runbook — recover from dpkg-interrupted state
You're here because an apt run got killed mid-transaction and the target now
@@ -96,9 +102,9 @@ Then `systemctl status apt-recovery` from a fresh ssh to check progress.
## Related
- [Operations cheatsheet](commands.md)
- [Auto-deploy pipelines](../infrastructure/auto-deploy.md)
- [Hubris host page](../hosts/hubris.md)
- [Operations cheatsheet](../../operations/commands.md)
- [Auto-deploy pipelines](../../../knowledge/wiki/infrastructure/auto-deploy.md)
- [Hubris host page](../../../knowledge/wiki/hosts/hubris.md)
## Changelog

View File

@@ -0,0 +1,30 @@
---
name: service-health-check
risk_class: read_only
inputs: [service_name]
verification: "homelab service <name> health"
docs_update_checklist: []
---
# Service health check
Goal: determine whether a service is actually healthy, without ad-hoc SSH.
1. `homelab service <name> explain` — read the context card: backend,
blast radius, doc pointer, risk notes.
2. `homelab service <name> health` — live health probe (HTTP code against
the service's `url`/`endpoint`). Once the Week-3 scheduler ships, this
reads a cached snapshot by default; pass `--live` to force a fresh probe.
3. If unhealthy, `homelab service <name> log` (or MCP `tail_log`) for the
last 200 lines.
4. Cross-check blast radius: `homelab node <name> relations` — is this
entity's own backend host healthy? A downstream failure (e.g. `strong`
down) will show up here before the service's own logs explain anything.
5. If the fix is a restart: classify first (`oikos/policy.yaml`
`service-restart` is `reversible_low` unless the service has a
`service_overrides` entry, e.g. `caddy`/`dns` are `config_mutation`).
Unattended agents may act on `reversible_low` without approval.
Docs-update checklist: none for a pure health check. If the investigation
reveals stale `risk_notes` or a wrong `doc_page`, fix `inventory.yaml` in
the same session.

72
.gitea/workflows/ci.yml Normal file
View File

@@ -0,0 +1,72 @@
# Oikos CI (Gitea Actions). Gates the deploy webhook on a green run (plan M1).
# Mirrors `make lint`, `make test`, and the generated-code drift guard.
name: ci
on:
push:
branches: [main]
pull_request:
jobs:
build-test:
runs-on: ubuntu-latest
services:
postgres:
image: timescale/timescaledb:2.17.2-pg16
env:
POSTGRES_DB: oikos
POSTGRES_USER: oikos
POSTGRES_PASSWORD: oikos_dev
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U oikos"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
OIKOS_TEST_DATABASE_URL: postgres://oikos:oikos_dev@postgres:5432/oikos?sslmode=disable
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.26"
cache: true
- name: go vet
run: go vet ./...
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: latest
args: --timeout 5m
continue-on-error: true # advisory until the lint baseline is clean
- name: govulncheck
run: |
go install golang.org/x/vuln/cmd/govulncheck@latest
govulncheck ./... || true # advisory
- name: generated code is up to date
run: make generate-check
- name: build
run: go build ./...
- name: test (race + coverage)
run: go test -race -covermode=atomic -coverprofile=coverage.out -timeout 300s ./...
- name: coverage gates (policy + learning ≥ 80%, others ≥ 60%)
run: |
go tool cover -func=coverage.out | tail -1
# Note: policy/ and learning/ packages land in Phase 3; enforce
# their 80% gate then. For now, report total coverage.
docker-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: docker build (verify image builds; no push)
run: docker build -f compose/oikos/Dockerfile -t oikos:ci .

11
.gitignore vendored
View File

@@ -1 +1,10 @@
.DS_Store
.DS_Store
__pycache__/
*.pyc
# Regenerated every scheduler run (every 10 min); no audit value in the
# diff. Signals (signals/*.jsonl) ARE tracked — this is just the ephemeral
# health-probe cache. See oikos/scheduler.py.
oikos/state.json
.worktrees/

View File

@@ -12,7 +12,7 @@
## Problem statement
The Technitium DHCP server on [CT 107](containers/107-dns.md) serves `192.168.8.100192.168.8.240`. **Every static homelab IP except hubris (`.77`) sits inside that range:**
The Technitium DHCP server on [CT 107](../../knowledge/wiki/containers/107-dns.md) serves `192.168.8.100192.168.8.240`. **Every static homelab IP except hubris (`.77`) sits inside that range:**
| Host | IP | Inside pool? |
|---|---|---|

View File

@@ -0,0 +1,358 @@
# Assessment: Which nodes can move to `strong`
## Executive summary
hubris is **memory-starved**: 28 GiB RAM, 71.9 GiB allocated across 18 LXC + 2 VM
(2.5× overcommit), 10 GiB swap in active use. strong sits **completely empty**
28 GiB RAM, 25 GiB free, 0 guests, 2.7 TiB unused storage. The single most
effective decongestion move is to shift guests off hubris onto strong.
This document assesses every guest for move-readiness, grouped by constraints
(library dependency, GPU, core-infra status), and proposes a phased migration
that does **not** require the physical library-SSD move (the blocker of the
original plan) — library access from strong is provided via NFS from hubris.
---
## Current resource state (live, 2026-07-05)
### hubris — overloaded
| Resource | Capacity | Allocated (all guests) | Actual use | Status |
|----------|----------|----------------------|------------|--------|
| RAM | 28 GiB | 70.2 GiB (2.5× overcommit) | 18 GiB used + 10 GiB swap | ⚠️ heavy swap pressure |
| CPU | 12 vCPU (6c/12t) | 43 vCPU (3.6× overcommit) | ~43% scaling MHz | OK (shares) |
| local-lvm | 856 GiB | 582 GiB allocated (29% thin) | — | OK |
| library (lvmthin) | 3.7 TiB | — | 1.2 TiB used (34%) | OK, 2.3 TiB free |
### strong — empty, ready
| Resource | Capacity | Used | Status |
|----------|----------|------|--------|
| RAM | 28 GiB | 2.3 GiB (host only) | 25 GiB free |
| CPU | 16 vCPU (8c/16t) | idle (0.08 load) | 100% free |
| local-lvm | 856 GiB | 0 | empty |
| ludo-lvm | 1.8 TiB | 0 | empty |
| Guests | — | 0 LXC, 0 VM | nothing running |
### Network topology constraint
```
Fritz!Box (192.168.178.1)
└── SODOLA 2.5G switch
├── hubris eno1 → vmbr1 (192.168.178.10) → vmbr0 (192.168.8.0/24)
│ └── all 20 guests on 192.168.8.x
└── strong vmbr0 (192.168.178.181)
└── no internal bridge yet, guests would be on 192.168.178.x
```
strong reaches `192.168.8.0/24` via the Fritz static route through hubris.
Guests on strong get `192.168.178.x` IPs unless we add an internal bridge
on strong (Phase 0 prerequisite — see below).
---
## Per-guest assessment
### Tier 1 — Move immediately (no library dependency, no core-infra)
These guests mount **no** `/mnt/library` and are not part of the core
infrastructure spine (caddy/dns/auth/mcp). They are the easiest wins.
| ID | Name | Cores | RAM | Library? | GPU? | Notes |
|----|------|-------|-----|----------|------|-------|
| 118 | elementsynapse | 2 | 4 GiB | ❌ | ❌ | Matrix homeserver. Public via Caddy (`matrix.hubris.network`). Only change: Caddy backend IP. **Easiest move in the fleet.** |
| 129 | house | 2 | 3 GiB | ❌ | ❌ | Yuvomi family planner (Docker). No public Caddy route yet (uses VPS traefik directly). Self-contained. |
**Combined RAM freed from hubris: 7 GiB.** No NFS, no library, no GPU.
### Tier 2 — Move with library NFS (high resource consumers)
These are the heaviest guests and the original migration plan's primary
targets. They mount `/mnt/library` and two use the iGPU. Moving them
requires an NFS export from hubris → strong (reverse of the original
plan's direction, since the physical SSD hasn't moved).
| ID | Name | Cores | RAM | Library? | GPU? | I/O profile | Notes |
|----|------|-------|-----|----------|------|-------------|-------|
| 120 | mule-images | 6 | 12 GiB | ✅ mp0 | ✅ iGPU | Write-heavy (photo processing) | **#1 RAM consumer.** Strong has Radeon 680M iGPU (VAAPI works). |
| 122 | arriman | 4 | 8 GiB | ✅ mp0 | ❌ | Write-heavy (downloads) | *arr stack + qbit + sab. Mounts library for download writes. |
| 101 | jellyfin | 4 | 8 GiB | ✅ mp0 | ✅ iGPU | Read-heavy sequential | Media streaming + transcode. Strong 680M handles VAAPI. |
**Combined RAM freed: 28 GiB.** This alone would eliminate hubris's swap
pressure entirely.
### Tier 3 — Could move, low urgency
| ID | Name | Cores | RAM | Library? | Notes |
|----|------|-------|-----|----------|-------|
| 130 | grimmory | 1 | 2 GiB | ✅ mp0 | Book library (Docker). Migrated from apps LXC recently. |
| 131 | teddycloud | 1 | 1 GiB | ✅ mp0 | New (not in inventory.yaml yet). |
| 132 | rclone | 1 | 2 GiB | ✅ mp0 (ro) | Backup container. Read-only library mount. |
| 128 | trmnl | 1 | 768 MiB | ❌ | TRMNL middleware. No library. Could move but tiny. |
| 119 | sophia | 2 | 1 GiB | ✅ mp0 | Workshop. Light use. |
### Stay on hubris (core infrastructure)
| ID | Name | Cores | RAM | Why it stays |
|----|------|-------|-----|--------------|
| 121 | caddy | 1 | 512 MiB | Reverse proxy — terminates all `*.hubris.network`. Must stay on hubris for LAN-side reachability. **Needs backend IP updates** when guests move. |
| 107 | dns | 1 | 1 GiB | Technitium DNS, split-horizon. Core. |
| 106 | auth-outpost | 1 | 512 MiB | Authentik SSO enforcement. Core. |
| 105 | apps | 2 | 4 GiB | homelab MCP + secrets-issuance + artifacto. Core infra. Mounts library. |
| 104 | gitea | 1 | 1 GiB | Git server. NFS would hurt git lock/stat perf. Mounts library (bare repos). |
| 103 | paperless | 2 | 3 GiB | Document archive. Moderate I/O, OCR writes. Mounts library. |
| 102 | nfs-export | 1 | 512 MiB | Exports library to zimaos via NFS. Must stay with the physical library. |
| 100 | zimaos | 4 | 8 GiB (VM) | NAS frontend eval. Already NFS-mounts library from 102. |
| 108 | haos | 2 | 4 GiB (VM) | Home Assistant OS. Hardware access, low latency. |
---
## Constraints & prerequisites
### 1. Network — strong needs an internal bridge (Phase 0)
strong currently has only `vmbr0` on `192.168.178.0/24`. Guests created there
get household-LAN IPs, not homelab-subnet IPs. Two options:
- **Option A (recommended):** Add `vmbr1` on strong as a portless internal
bridge with a `192.168.8.x/24` address (e.g. `192.168.8.3`). Route between
strong's `vmbr0` and `vmbr1` the same way hubris does. Guests go on `vmbr1`
and get `192.168.8.x` IPs — transparent to Caddy, DNS, and inter-LXC refs.
Requires adding a static route on Fritz (or relying on hubris's existing
route — strong would need IP forwarding + a route to 192.168.8.0/24 via vmbr1).
- **Option B (simpler, messier):** Put guests on `192.168.178.x` directly.
Caddy can still reach them (hubris routes to 192.168.178.0/24). But DNS
records, inter-LXC references, and firewall rules all assume `192.168.8.x`.
More config churn per guest.
### 2. Storage — rootfs migration (no shared storage)
`local-lvm` is per-node (not shared). Moving an LXC requires either:
- `vzdump` → restore on strong (clean, but needs temp disk space + downtime)
- `rsync` the rootfs to a new LXC on strong (faster for large rootfs like 120's 100G)
- `pct migrate` only works with shared storage — **not applicable here**
For VMs (100, 108): `qm migrate` also needs shared storage. Not moving VMs.
### 3. Library access — NFS from hubris to strong
Since the physical library SSD is still on hubris, strong's guests that need
`/mnt/library` must NFS-mount it from hubris. Options:
- **Export from hubris host directly** (simplest): add `/mnt/library` to
`/etc/exports` on hubris with the same squash params as LXC 102
(`rw,all_squash,anonuid=33,anongid=10000,no_subtree_check`). Mount on strong
at `/mnt/library`. Strong's guests bind-mount it just like hubris's guests do.
- **Use existing nfs-export LXC 102**: strong NFS-mounts from `192.168.8.200`
(LXC 102). This already has the right squash config. Less host-level change.
**This is the path of least resistance.**
### 4. GPU — iGPU passthrough on strong
strong has a Ryzen 7 PRO 6850U with Radeon 680M iGPU. For jellyfin (VAAPI
transcoding) and mule-images (photo processing), we need:
- `/dev/dri/renderD128` passed to the LXC (`lxc.cgroup2.devices.allow` +
`lxc.mount.entry` or Proxmox's `dev0:` passthrough)
- `video` / `render` group membership inside the container
- Confirm `amdgpu` driver loads on strong's host kernel (it should — same APU family)
### 5. Quorum — 2-node cluster, no QDevice
Moving guests to strong does NOT fix the quorum issue but **reduces blast
radius**: if hubris reboots (its known thermal instability), the guests on
strong keep running independently. Consider adding a QDevice as a separate
follow-up — it's orthogonal to this migration.
---
## Revised migration phases
The original plan's NFS-over-LAN approach has been superseded. Instead,
**media library data moves to ludo-lvm** on strong so migrated guests access
it as a local ext4 mount. Data is split by origin:
```
hubris (stays): library SSD (3.7T, 1.2T used)
└── /mnt/library/{documents,images,cloud,homecloud,notes,repos,sophia}
↑ user-generated content (docs, photos, cloud sync, notes, repos, workshop)
strong (moves): ludo-lvm (1.8T, 0 used at start)
└── /mnt/media_local ← 1.5T thin volume
└── {downloads,movies,music,tv,anime,books}
↑ non-user-generated content (media arr stack, book library)
```
| Category | Stays on hubris | Moves to strong |
|----------|----------------|-----------------|
| Media | — | downloads (25G), movies (51G), music (29G), tv (30G), anime (206G) |
| Books | — | books (2.6G) |
| Docs/Photos | documents (249M), images (4K) | — |
| Cloud sync | cloud (287G), homecloud (367G) | — |
| Personal | notes (6.7M), repos (84M), sophia (151G) | — |
| **Total** | **~805G** | **~344G** |
ludo-lvm (1.8T) fits all media + books with ~1.15T headroom for growth.
hubris library SSD (3.7T, 1.2T used) retains the user-generated content.
Both sides keep their data local — no cross-node NFS needed for daily I/O.
---
### Phase 2a — Prepare ludo-lvm on strong
1. Create a ext4 filesystem on ludo-lvm for media:
```bash
lvcreate -n media -L 1.5T ludo-lvm
mkfs.ext4 /dev/ludo-lvm/media
```
2. Mount at `/mnt/media_local` on strong, add to `/etc/fstab`
3. rsync media directories from hubris → strong:
```bash
rsync -av --progress /mnt/library/{movies,tv,anime,downloads,music,books} strong:/mnt/media_local/
```
### Phase 2b — Migrate arriman (122) to strong
1. Stop arriman on hubris, dump rootfs (24G)
2. Restore on strong with IP `192.168.8.245/28` on vmbr1
3. Mount `/mnt/media_local` → `/mnt/library` via mp0 (downloads land locally)
4. Update Caddy: jellyseerr/qbit/sab backends → new IP
5. Update inventory.yaml
### Phase 2c — Migrate jellyfin (101) to strong
1. Stop jellyfin on hubris, dump rootfs (16G)
2. Restore on strong with IP `192.168.8.246/28` on vmbr1
3. Pass `/dev/dri/renderD128` + `/dev/dri/card0` (Radeon 680M + RX 7600)
4. Mount `/mnt/media_local` → `/mnt/library` via mp0 (media reads locally)
5. Update Caddy: `media.hubris.network` → new IP
6. Reinstall `sso-inject.js` in web dir (lost on every apt upgrade)
7. Test VAAPI transcoding, SSO login, media playback
### Phase 2d — Migrate grimmory (130) to strong
1. Stop grimmory on hubris, dump rootfs (16G)
2. Restore on strong with IP `192.168.8.247/28` on vmbr1
3. Mount `/mnt/media_local` → `/mnt/library` via mp0 (books read locally)
4. Update Caddy: `books.hubris.network` → new IP
5. Update inventory.yaml
6. Test: book browsing, calibre-web access
### No NFS export needed
With the data split by origin, hubris guests that only need user-generated
content (documents, images, cloud, repos, sophia) still access them from the
original library SSD — no cross-node NFS required. The two sides are
independent.
**Result after Phase 2: hubris frees 26 GiB RAM (4 migrated guests) + 344G of
library I/O burden. Strong becomes the media/books powerhouse.**
---
### Phase 3 — Migrate mule-images (120) to strong
Move photo management (12 GiB RAM, 6 cores, iGPU) last because it needs:
- `/mnt/library` access (now NFS from strong — already set up in Phase 2d)
- `/dev/dri/renderD128` (Radeon 680M — confirm VAAPI compatibility first)
Steps:
1. Stop mule-images on hubris, rsync the 100G rootfs to strong (faster than vzdump)
2. Restore on strong with IP on vmbr1
3. Pass Radeon 680M iGPU
4. Reconfigure library paths → `/mnt/media_local` (or keep NFS mount)
5. Update Caddy: `photos.hubris.network` → new IP
6. Test photo import + processing pipeline
---
### Phase 4 — Tier 3 moves (optional)
Migrate grimmory (130), teddycloud (131), rclone (132), trmnl (128), sophia (119)
as needed — each frees 12 GiB. Not urgent; do when convenient.
---
### Phase 5 — Follow-up
- **QDevice**: add a tiebreaker for 2-node quorum
- **Gaming VM**: strong's 6850U has enough cores alongside migrated LXCs
- **Hubris library cleanup**: after all guests are confirmed working, decide
whether to keep the original library SSD as backup or repurpose it
---
## Resource math after Phase 3 (all Tier 1 + 2 moved)
| | hubris | strong |
|---|--------|--------|
| Guests | 11 LXC + 2 VM | 5 LXC |
| RAM allocated | ~25 GiB | ~45 GiB |
| RAM capacity | 28 GiB | 28 GiB |
| Overcommit | 0.9× (under-committed) | 1.6× (manageable) |
| Library disk | Local ext4 (3.7T) → NFS client | Local ext4 on ludo-lvm (1.8T) |
| GPU | Radeon 760M (idle) | Radeon 680M (jellyfin + mule-images) |
strong becomes the media/library powerhouse. hubris becomes a lean core-infra
node (DNS, auth, git, docs, caddy, HA).
---
---
## Risk register
| Risk | Impact | Mitigation |
|------|--------|------------|
| NFS latency for library reads (jellyfin, arriman) | Media playback stutter, slow downloads | Test iperf between strong↔hubris first. If 2.5G link, NFS throughput is fine (~1 Gbit/s). |
| GPU passthrough on strong (680M vs 760M) | Transcode quality/compat differences | Both are AMD VAAPI — same driver stack. Test `vainfo` inside LXC before going live. |
| Caddy backend IP churn | Service outage if IP wrong | Update Caddyfile in git repo (caddy-conf), test each route before destroying old LXC. |
| vzdump/restore downtime | Service unavailable during migration | Schedule off-hours. Use rsync for large rootfs (120's 100G) to minimize freeze window. |
| 2-node quorum still fragile | If hubris goes down, strong /etc/pve goes read-only | Guests keep running. Add QDevice as follow-up. |
| Library data integrity during NFS transition | Permission drift | NFS `all_squash,anonuid=33,anongid=10000` matches existing LXC 102 config. Verify with `ls -la /mnt/library` after mount. |
---
## Open questions for operator
1. **Internal bridge on strong**: proceed with `vmbr1` on `192.168.8.3/24`
(Option A), or use `192.168.178.x` guest IPs (Option B)?
2. **Migration method**: `vzdump`/restore (clean, downtime) vs `rsync` rootfs
(faster for large disks, needs manual config copy)?
3. **Phase 1 priority**: move elementsynapse + house first (quick wins), or
go straight to Phase 2 (mule-images/jellyfin/arriman) for maximum relief?
4. **Should we add a QDevice now** before moving anything, to protect
management plane during the migration?
---
## Changelog
### 2026-07-05 — Phase 2d complete (grimmory migrated; media NFS to zimaos)
grimmory (130) → 192.168.8.247 on strong. Rsync'd /books (2.6G) to ludo-lvm.
LXC 102 (nfs-export) now mounts strong's NFS at /mnt/media and exports it as
a second share alongside /mnt/library. Zimaos mounts both: /media/library
(hubris user-generated) and /media/media (strong media+books).
See hosts/strong.md changelog.
### 2026-07-05 — Phase 2 complete (arriman + jellyfin migrated; library on ludo-lvm)
arriman (122) → 192.168.8.245, jellyfin (101) → 192.168.8.246. Created 1.5T
thin volume on ludo-lvm, rsync'd 363G of media data. Both containers use local
ext4 mount — no NFS. Jellyfin has 680M + RX 7600 GPU passthrough.
Caddy backends updated. See hosts/strong.md changelog.
### 2026-07-05 — Phase 1 complete (elementsynapse + house migrated to strong)
Both Tier 1 guests moved: elementsynapse (118) → 192.168.8.242, house (129) → 192.168.8.244.
Strong now has vmbr1 at 192.168.8.241/28. Hubris has proxy ARP + /32 routes for strong
guest range. DHCP scope narrowed to 192.168.8.100-239 to avoid conflicts.
Teddycloud (LXC 131) given static IP 192.168.8.150 due to IP conflict with
previous DHCP allocation at 192.168.8.243.
See hosts/strong.md changelog for full steps.
### 2026-07-05 — assessment created
Built from live `pct config` + `pvesm status` + `free -h` data pulled from
both nodes. Supersedes the storage-migration framing of the original
library-SSD plan — this assessment treats the SSD move as optional and
focuses on guest relocation via NFS.

View File

@@ -25,7 +25,9 @@ creation_rules:
age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0,
age1vf8h7s8mqsn2q5eadgpdupsj4mwn8zguc77d85ws3xj40sl9rgksx2rxw6,
age1z62ff2ak9zj5ctcvaxwyyhedwjvlwgm2dkn9nk3wrwk8fkavcpmsqwc2vs,
age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h
age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h,
age1rtwvdct6avjkr3cyxv3vue3vqx4d524fjfr3vk7xrnvyrylnry5sm54sn4,
age1pwtdws2thdh7vzp2dzttl3zxgcs2tgpcsjsqgw3q04nyml4kvuqq467u4x
- path_regex: ^secrets/gitea-pat\.yaml$
# Write-scoped Gitea PAT (dtoro user). Same recipient list as hello.yaml
@@ -36,12 +38,14 @@ creation_rules:
age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0,
age1vf8h7s8mqsn2q5eadgpdupsj4mwn8zguc77d85ws3xj40sl9rgksx2rxw6,
age1z62ff2ak9zj5ctcvaxwyyhedwjvlwgm2dkn9nk3wrwk8fkavcpmsqwc2vs,
age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h
age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h,
age1rtwvdct6avjkr3cyxv3vue3vqx4d524fjfr3vk7xrnvyrylnry5sm54sn4,
age1pwtdws2thdh7vzp2dzttl3zxgcs2tgpcsjsqgw3q04nyml4kvuqq467u4x
- path_regex: ^secrets/gitea-tokens\.yaml$
# Workstations only.
age: >-
# placeholder — fill with age_pubkey of: republic-laptop, mac-mini, ludo-mini, hubris
# placeholder — fill with age_pubkey of: republic-laptop, mac-mini, strong, hubris
- path_regex: ^secrets/webhook-hmacs\.yaml$
# LXCs that run a webhook receiver.
@@ -58,7 +62,8 @@ creation_rules:
age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0,
age1vf8h7s8mqsn2q5eadgpdupsj4mwn8zguc77d85ws3xj40sl9rgksx2rxw6,
age1z62ff2ak9zj5ctcvaxwyyhedwjvlwgm2dkn9nk3wrwk8fkavcpmsqwc2vs,
age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h
age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h,
age1pwtdws2thdh7vzp2dzttl3zxgcs2tgpcsjsqgw3q04nyml4kvuqq467u4x
- path_regex: ^secrets/netbird-authentik-oidc\.yaml$
# Authentik OIDC client secret for the netbird-dashboard provider.
@@ -69,7 +74,8 @@ creation_rules:
age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0,
age1vf8h7s8mqsn2q5eadgpdupsj4mwn8zguc77d85ws3xj40sl9rgksx2rxw6,
age1z62ff2ak9zj5ctcvaxwyyhedwjvlwgm2dkn9nk3wrwk8fkavcpmsqwc2vs,
age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h
age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h,
age1pwtdws2thdh7vzp2dzttl3zxgcs2tgpcsjsqgw3q04nyml4kvuqq467u4x
- path_regex: ^secrets/netbird-pat\.yaml$
# NetBird API Personal Access Token. Consumed by the dns-sync job on the
@@ -109,4 +115,22 @@ creation_rules:
age1vf8h7s8mqsn2q5eadgpdupsj4mwn8zguc77d85ws3xj40sl9rgksx2rxw6,
age1z62ff2ak9zj5ctcvaxwyyhedwjvlwgm2dkn9nk3wrwk8fkavcpmsqwc2vs,
age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h
- path_regex: ^secrets/oikos-approval-hmac\.yaml$
# HMAC signing key for Oikos approval-grant tokens (oikos/approve.py).
# Recipients: apps (105, runs the approval engine alongside homelab-mcp)
# and hubris (admin/debug decrypt). See OIKOS.md "Approval engine".
age: >-
age1xkklkvnk5z0fsnh6cfgv70hy9ksfy8rdprwerzw4yk3p4p7cxcqs2yvpz6,
age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0
- path_regex: ^secrets/oikos-console-deploy-secret\.yaml$
# Shared HMAC secret for the Gitea deploy webhook (id 14) ->
# oikos-console-deploy.service on apps (105). Generated + registered
# with Gitea before the apps-side install ran (see
# oikos/console/deploy/README.md "Status") — write this exact value
# into /etc/oikos-console-deploy/secret rather than letting
# webhook/install.sh generate a fresh one.
age: >-
age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0
# webhook noop 2026-05-20T18:16:57+02:00

View File

@@ -4,6 +4,20 @@ You are running on a machine that is part of the **hubris** homelab. The full
context is in this checkout at `/opt/homelab-context/`. This file is the entry
point. Read it once at start, then keep working.
The operating model — OODA loop, risk classes, approval rules, the ontology,
and node lifecycle — is defined in [OIKOS.md](.agents/OIKOS.md). Before any mutation,
classify the action against `oikos/policy.yaml`; when the class requires
approval, stop and ask the operator.
Agent-facing instruction is separated from human content under `.agents/`:
`.agents/shared/` holds the conventions every agent applies
([writing-style](.agents/shared/writing-style.md), [caveman](.agents/shared/caveman.md),
[page-templates](.agents/shared/page-templates.md), [llm-wiki](.agents/shared/llm-wiki.md)), and
`.agents/domains/` holds the per-domain schemas
([knowledge](.agents/domains/knowledge/schema.md), [operations](.agents/domains/operations/schema.md)).
The narrative wiki lives under `knowledge/wiki/`; the machine-readable substrate
(`inventory.yaml`, `hosts/*.yaml`, `oikos/`) stays at the repo root.
## 1. Who you are
Run `hostname` (Linux) or `scutil --get LocalHostName` (macOS), then read:
@@ -19,13 +33,14 @@ the operator to run `homelab client add <hostname>` from an existing client.
- `/opt/homelab-context/inventory.yaml` — every host, LXC, VM, and workstation
with their mesh addresses, roles, and service mappings. Treat this file as
authoritative; anything you read in narrative pages should agree with it.
- `/opt/homelab-context/infrastructure/mesh.md` — Tailscale → Netbird state.
- `/opt/homelab-context/knowledge/wiki/infrastructure/mesh.md` — Tailscale → Netbird state.
Both meshes are accepted today; Netbird is preferred for new traffic.
- `/opt/homelab-context/infrastructure/dns.md` — split-horizon DNS via
dnsmasq on LXC 124. `*.hubris.network` resolves to 192.168.x.x on the LAN
and to mesh addresses off-LAN.
- `/opt/homelab-context/operations/commands.md` — the operator's cheatsheet
for pct, caddy, dnsmasq. Use these verbs when you take actions.
- `/opt/homelab-context/knowledge/wiki/infrastructure/dns.md` — split-horizon DNS via
Technitium on [dns (107)](knowledge/wiki/containers/107-dns.md). `*.hubris.network`
resolves to 192.168.x.x on the LAN and to mesh addresses off-LAN.
- `/opt/homelab-context/.agents/operations/commands.md` — the operator's cheatsheet
for pct, caddy, DNS, and the Oikos command surface. Use these verbs when
you take actions.
## 3. The MCP server
@@ -44,8 +59,15 @@ Available tools:
get_service_status(service), tail_log(service, lines=200),
list_lxcs(), get_lxc_state(lxc), ping_service(service)
Oikos (read-only; see OIKOS.md):
explain(service) — compact context card, cheaper than search_docs+get_page
preflight(service) — risk class, approval requirement, verification command
get_relations(entity) — ontology blast-radius query (host: or service: id)
get_change_history(entity, limit=20) — change-ledger entries
get_state_snapshot() — last scheduler Observe-pass (health, disk, drift count)
Mutations are **not** exposed via MCP. Use the `homelab` CLI for those, with
operator confirmation.
operator confirmation — see OIKOS.md's risk classes and approval flow.
**When to prefer MCP over grepping the clone:** any time you need to resolve a
name to an address, look up service status, or search the wiki by content.
@@ -53,17 +75,25 @@ Grep is fine for browsing or when MCP is unreachable.
## 4. Wiki conventions
- Pages live under `containers/`, `hosts/`, `vms/`, `infrastructure/`,
`investigations/`, `operations/`. Cross-link liberally; orphans are bugs.
- Every page ends with a `## Changelog` section, entries in reverse-chrono
order:
See [page-templates.md](.agents/shared/page-templates.md) for file naming, page
structure, and the tone standard. Quick reference:
- **File naming:** Foundational docs are ALL-CAPS (AGENTS.md, OIKOS.md, GLOSSARY.md);
containers use `<id>-<name>.md`; infrastructure uses lowercase-with-dashes; plans
and investigations use `YYYY-MM-DD-slug.md`; skills are `<name>/SKILL.md`.
- **Where pages live:** Narrative under `knowledge/wiki/{containers,hosts,vms,infrastructure}/`;
incident records under `knowledge/sources/investigations/`; runbook procedures under
`.agents/skills/<name>/SKILL.md`; operator reference under `.agents/operations/`;
design docs under `plans/`. Cross-link liberally; orphans are bugs.
- **Changelog format:** Every page ends with a `## Changelog` section, entries in
reverse-chronological order:
### YYYY-MM-DD — short title
one or two lines describing what changed and why.
- Investigation files are dated and slugged: `YYYY-MM-DD-slug.md`.
- Live state takes precedence over docs. If you observe a discrepancy, update
the docs *in the same session* (per the same-session update rule).
- **Live state precedence.** If you observe a discrepancy between the docs and
running state, update the docs *in the same session* (per the same-session update
rule in [page-templates.md](.agents/shared/page-templates.md#same-session-update-rule)).
## 5. Acting on the homelab
@@ -76,16 +106,20 @@ Grep is fine for browsing or when MCP is unreachable.
demand using the per-client age key at `/etc/age/key.txt`. Secrets ARE
available in this system — `list_my_secrets()` (MCP) shows what you can
decrypt.
- **Mutations** (restart, edit configs, etc.): the `homelab` CLI's mutating
subcommands ask for confirmation. For ad-hoc work, SSH and edit directly —
but commit changes that touch tracked configs (caddy, gitea custom,
artifacto, mule-image, etc.; see `infrastructure/auto-deploy.md`).
- **Mutations** (restart, edit configs, etc.): classify against
`oikos/policy.yaml` first (`homelab decide <action> <entity>`).
`reversible_low` actions just need the interactive confirmation prompt;
`config_mutation`/`destructive` actions are mechanically refused without
a valid `--approval-id` from `homelab approval request` — see OIKOS.md.
For ad-hoc work, SSH and edit directly — but commit changes that touch
tracked configs (caddy, gitea custom, artifacto, mule-image, etc.; see
`knowledge/wiki/infrastructure/auto-deploy.md`).
- **Wiki updates**: same-session rule applies to any meaningful state change
this client makes.
## 6. Communication mode
Read and apply `/opt/homelab-context/CAVEMAN.md` (if present). It defines the lab's
Read and apply `/opt/homelab-context/.agents/shared/caveman.md` (if present). It defines the lab's
terse-communication standard — drop filler, keep substance, use fragments.
## 7. Auto-setup mechanism

View File

@@ -1,123 +0,0 @@
# Contributing to the Homelab Wiki
## Voice
Concise, technical, sysadmin-to-sysadmin. No marketing prose, no exclamation marks.
## Page templates
### Container page (`containers/<id>-<name>.md`)
```markdown
# <id> — `<name>`
One-sentence purpose.
## At a glance
- **Hostname:** `<name>`
- **IP:** `192.168.8.x`
- **Privilege:** privileged | unprivileged
- **Resources:** N cores / M GiB RAM / D GiB rootfs
- **Mounts:** `/mnt/library``/mnt/library` (if any)
- **Public hostname:** `<sub>.hubris.network` (if proxied)
## Role
What it does, what it talks to.
## Service / port map
| Service | Listen | Notes |
## Storage / config paths
## Auto-deploy
(if any) — link to [auto-deploy](../infrastructure/auto-deploy.md)
## Related
- [Caddy](121-caddy.md) (if proxied)
- [DNS](../infrastructure/dns.md) (if has subdomain)
- [Authentik](124-authentik.md) (if SSO)
- ...
## Changelog
### YYYY-MM-DD — short title
What changed, why, link to investigation if any.
```
### Cross-cutting page (`infrastructure/<topic>.md`)
```markdown
# <Topic>
One-sentence summary.
## Why
Design rationale — what it replaces, what it solves.
## Components
Where it runs, what files matter.
## How to apply / use
Recipes.
## Gotchas
## Related
Links to nodes that host or depend on this.
## Changelog
```
### Plan (`plans/YYYY-MM-DD-slug.md`)
```markdown
# YYYY-MM-DD — <title>
## Goal
What this change achieves and why.
## Current topology / state
Diagram or description of what exists now.
## Target topology / state
What it looks like after.
## Pre-flight checklist
## Step-by-step procedure
## Verification
## Post-migration
Changelog entries to write, index status to update.
```
### Investigation (`investigations/YYYY-MM-DD-slug.md`)
```markdown
# YYYY-MM-DD — <title>
## Summary
1-3 sentences.
## Timeline
## Root cause
## Mitigations applied
## Open questions
```
## Linking discipline
- Every container page links to every cross-cutting page it participates in.
- Every cross-cutting page lists the nodes that participate.
- Every investigation links to the nodes it implicates *and* gets back-linked from each node's changelog.
- Every plan links to the infrastructure pages it affects. When done, update the plan's status in `plans/index.md` and write changelog entries on affected node pages.
## Changelog hygiene
- Reverse-chronological (newest first).
- One entry per discrete change, even if you make several in one day.
- If a change spans nodes, repeat the entry on each affected page (different perspective is fine).
- Don't rewrite history — entries are append-only. Mistakes get a follow-up entry that supersedes them.

50
Makefile Normal file
View File

@@ -0,0 +1,50 @@
.PHONY: build test test-db lint generate generate-check dev migrate seed export clean tidy
BINARY := oikos
GO ?= go
build:
$(GO) build -o $(BINARY) -tags timetzdata ./cmd/oikos
test:
$(GO) test -race -cover ./...
# Integration tests against the compose Postgres (starts it if needed)
test-db:
docker compose up -d postgres
@sleep 3
OIKOS_TEST_DATABASE_URL="postgres://oikos:$${OIKOS_DB_PASSWORD:-oikos_dev}@localhost:5432/oikos?sslmode=disable" \
$(GO) test -race -count=1 ./internal/db/ ./internal/httpapi/ ./internal/mcp/
lint:
$(GO) vet ./...
@command -v golangci-lint >/dev/null 2>&1 && golangci-lint run || echo "golangci-lint not installed, skipping"
generate:
$(GO) run github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@v2.4.1 \
-config api/codegen.yaml api/openapi.yaml
$(GO) run github.com/sqlc-dev/sqlc/cmd/sqlc@v1.29.0 generate
# CI drift guard: regenerate and fail if the committed output changed.
generate-check: generate
@git diff --exit-code -- internal/httpapi/gen internal/db/sqlcgen \
|| (echo "generated code is stale — run 'make generate' and commit" && exit 1)
migrate:
$(GO) run ./cmd/oikos migrate
seed:
$(GO) run ./cmd/oikos seed
export:
$(GO) run ./cmd/oikos export
dev:
docker compose --profile dev up -d
clean:
rm -f $(BINARY)
$(GO) clean -testcache
tidy:
$(GO) mod tidy

232
README.md
View File

@@ -1,76 +1,190 @@
# Homelab Wiki — `hubris`
# Homelab OS
Living documentation for the **hubris** Proxmox homelab. Every node, every cross-cutting system, and every meaningful incident is its own page; pages are linked so you can start anywhere and walk the graph.
Living documentation for the **hubris** Proxmox homelab + Oikos operating system.
> Last refreshed against live state: **2026-04-28**.
**For agents running on enrolled clients:** start with [AGENTS.md](AGENTS.md), then [OIKOS.md](.agents/OIKOS.md).
## Map
---
### Hosts
- [`hubris`](hosts/hubris.md) — single Proxmox VE node, GMKtec NucBox M6 Ultra, `192.168.8.77`
## For Agents — Navigation & Entry Points
### VMs
- [100 — `zimaos`](vms/100-zimaos.md) — ZimaOS 1.6.1, NAS frontend (evaluation)
- [108 — `haos-16.3`](vms/108-haos.md) — Home Assistant OS
### You are running on a client enrolled in the hubris homelab
### LXC containers
See the full table in [`containers/index.md`](containers/index.md). Quick links:
1. **First:** Read [AGENTS.md](AGENTS.md) once. It explains who you are, the topology, available tools, conventions, and how to act.
2. **Before any mutation:** Read [OIKOS.md](.agents/OIKOS.md). It defines the operating model, risk classes, approval flow, and the ontology you'll consult.
3. **For specific workflows:** Load the matching skill from `.agents/skills/<name>/SKILL.md` (e.g., [service-health-check](.agents/skills/service-health-check/SKILL.md)).
4. **When in doubt:** Use MCP tools (`search_docs`, `get_page`, `explain`, `get_changelog`) — they're cheaper and more reliable than grepping.
| ID | Name | IP | Role |
| --- | ---------------- | --------------- | --------------------------------------------- |
| 101 | [jellyfin](containers/101-jellyfin.md) | 192.168.8.206 | Media server |
| 102 | [nfs-export](containers/102-nfs-export.md) | 192.168.8.200 | NFSv4 re-export of /mnt/library for ZimaOS |
| 103 | [paperless](containers/103-paperless.md) | 192.168.8.130 | Document mgmt |
| 104 | [gitea](containers/104-gitea.md) | 192.168.8.121 | Git server |
| 105 | [apps](containers/105-apps.md) | 192.168.8.205 | Docker host (Artifacto / PlantUML / Portainer / WriteFreely) |
| 114 | [nextcloud](containers/114-nextcloud.md) | 192.168.8.224 | Personal cloud |
| 118 | [elementsynapse](containers/118-elementsynapse.md) | 192.168.8.239 | Matrix Synapse |
| 119 | [sophia](containers/119-sophia.md) | 192.168.8.157 | Sophia |
| 120 | [mule-images](containers/120-mule-images.md) | 192.168.8.136 | Mule-image / mulita photos |
| 121 | [caddy](containers/121-caddy.md) | 192.168.8.175 | Reverse proxy |
| 122 | [arriman](containers/122-arriman.md) | 192.168.8.132 | Docker host (\*arr stack) |
| 124 | [authentik](containers/124-authentik.md) | 192.168.8.180 | SSO + split-horizon DNS |
| 130 | [grimmory](containers/130-grimmory.md) | 192.168.8.213 | Digital library (Grimmory — fork of Booklore) |
### Key References for Agents
### Cross-cutting infrastructure
- [DNS — split-horizon](infrastructure/dns.md)
- [Ingress — Caddy + VPS traefik](infrastructure/ingress.md)
- [Mesh — Tailscale → Netbird migration](infrastructure/mesh.md)
- [Monitoring — Hermes health watchdog](infrastructure/monitoring.md)
- [Media permissions — `media` GID 10000](infrastructure/media-permissions.md)
- [SSH access](infrastructure/ssh-access.md)
- [Backups — restic on external drive (disabled)](infrastructure/backups.md)
- [Auto-deploy — gitea-webhook pipelines](infrastructure/auto-deploy.md)
- [VPS hardening — IONOS / netbird control plane](infrastructure/vps-hardening.md)
- [Homelab context distribution](infrastructure/homelab-context.md) — cross-client `/opt/homelab-context` + MCP + secrets-issuance
- **What am I?** → `/opt/homelab-context/hosts/<hostname>.yaml` (read on first run)
- **Live topology** → `inventory.yaml` + `hosts/*.yaml` (canonical, always wins)
- **Risk & approval** → [oikos/policy.yaml](oikos/policy.yaml) (enforced, not advisory)
- **Runbooks & workflows** → [.agents/skills/](.agents/skills/) (risk class + verification checklist included)
- **State of Oikos** → [OIKOS.md build status](.agents/OIKOS.md#build-status-30-day-roadmap) (scheduled probes, drift detectors, signals, approval engine)
### Investigations
Time-stamped incident notes / experiments in [`investigations/`](investigations/index.md).
### When to Use MCP vs Files vs Shell
### Operations
- [Command cheatsheet](operations/commands.md)
- [Agent enrollment](operations/agent-enrollment.md) — bootstrap a new client (workstation, LXC, VM) into the homelab context system
| Task | Use | Tool |
|------|-----|------|
| Resolve hostname → address | MCP | `get_host(name)` or `list_services()` |
| Search wiki by content | MCP | `search_docs(query)` |
| Read a wiki page | MCP or file | `get_page(path)` or `cat knowledge/wiki/.../...md` |
| Get changelog entries | MCP | `get_changelog(page, since?)` |
| Understand a service | MCP | `explain(service)` — compact context card, cheaper than search+read |
| Blast-radius query | MCP | `get_relations(entity)` (ontology walk) |
| List available secrets | MCP | `list_my_secrets()` (scoped to your age key) |
| Browse or grep | File | Raw `grep` when MCP unreachable, or exploratory browsing |
**When MCP is unreachable:** fall back to grepping the clone at `/opt/homelab-context/`. The local files are the same; MCP is just an index.
---
## Understanding the Operating Model
Before you act, **classify your action against [oikos/policy.yaml](oikos/policy.yaml)**.
### The Oikos OODA Loop + Decision Tree
```mermaid
flowchart TD
Observe["**Observe**<br/>probes, drift detectors, agent signals"]
Orient["**Orient**<br/>ontology, context, state, entity relations"]
Decide{"**Decide**<br/>classify against oikos/policy.yaml"}
Auto["Auto-act<br/>(unattended)"]
Escalate["Escalate<br/>homelab approval request"]
Act["**Act**<br/>homelab CLI, runbooks, skills"]
Verify["**Verify**<br/>checklist from SKILL.md"]
Ledger["**Ledger**<br/>mutation record: who/what/risk"]
Document["**Document**<br/>wiki update, same-session rule"]
Observe --> Orient --> Decide
Decide -->|read_only, reversible_low| Auto
Decide -->|config_mutation, destructive| Escalate
Auto --> Act
Escalate -->|approval granted| Act
Act --> Verify --> Ledger --> Document
Document -.loop.-> Observe
```
### Risk Classes (enforced, not advisory)
From [oikos/policy.yaml](oikos/policy.yaml):
- **read_only** — status, logs, docs, inventory queries. Unattended. MCP tools are all read_only.
- **reversible_low** — restart, cache clear, sync pull. Unattended + ledger entry.
- **config_mutation** — tracked-config edits (commit+push, never local), deploys, upgrades, DNS/ingress changes. **Operator approval required.**
- **destructive** — destroy, format, wipe, rotate, revoke. **Approval + typed confirmation phrase.**
### Decision Flow
1. **Decide:** Use `homelab decide <action> <entity>` to classify (risk class × blast radius × confidence).
2. **Escalate if needed:** `homelab approval request` (Matrix-delivered to operator; see [operations/commands.md](.agents/operations/commands.md)).
3. **Execute:** Use `homelab` CLI (not ad-hoc SSH) — it enforces policy, logs mutations, and verifies outcomes.
4. **Document:** Update wiki in the same session (per [AGENTS.md §5](AGENTS.md#5-acting-on-the-homelab) and the [same-session rule](.agents/shared/page-templates.md#same-session-update-rule)).
### The Ontology Graph
Everything that can break, be changed, or hold data has an entity in `inventory.yaml` + `oikos/ontology.yaml`. Blast-radius questions ("what breaks if strong goes down?") are graph walks via `homelab node <name> relations`, not doc archaeology.
**See:** [OIKOS.md](.agents/OIKOS.md) (full operating model, OODA loop, primitives, lifecycle gates, build status).
---
## Finding & Understanding Information
The narrative documentation is organized in **layers**:
| Layer | What it is | Where | Immutable? | How agents use it |
|-------|-----------|-------|-----------|-------------------|
| **Sources** | Raw evidence: incidents, external refs, live state | `knowledge/sources/investigations/` | Yes | Read to understand root causes; do not rewrite |
| **Wiki** | Synthesized current-state: one page per node & per system | `knowledge/wiki/{containers,hosts,vms,infrastructure}/` | No | This is the reference layer — if wiki disagrees with live state, update it *in the same session* |
| **Index** | Pure listings — every page in scope with one-line summary | `index.md` / folder `README.md` | No | Navigation aid; keep it current when wiki restructures |
| **Log** | Append-only doc-maintenance record (restructures, ingests, lints) | `knowledge/log.md` | Yes (append-only) | Read to understand past doc changes; never edit directly |
**Changelog ≠ Log:** Each wiki page ends with a `## Changelog` (infrastructure changes to that node, machine-parsed). That's not the Log; the Log records *doc operations* only.
**See:** [llm-wiki.md](.agents/shared/llm-wiki.md) (full rules, page structure, immutability contract).
---
## Map & Quick Navigation
### Agent Entry Points (Start Here)
- **You are an agent** → [AGENTS.md](AGENTS.md) (on deployed clients: `/opt/homelab-context/AGENTS.md`)
- **Operating model & risk policy** → [OIKOS.md](.agents/OIKOS.md)
- **Specific workflows** → [.agents/skills/](.agents/skills/) (load the matching SKILL.md before acting)
- **Operations cheatsheet** → [.agents/operations/commands.md](.agents/operations/commands.md)
- **Tools & MCP reference** → [AGENTS.md §3 — The MCP server](AGENTS.md#3-the-mcp-server)
### Topology & Infrastructure
Node counts, IPs, and service lists change often — treat `inventory.yaml` and the index pages below as the source of truth, not this README.
- **Proxmox hosts** → [knowledge/wiki/hosts/index.md](knowledge/wiki/hosts/index.md)
- **VMs** → [knowledge/wiki/vms/index.md](knowledge/wiki/vms/index.md)
- **LXC containers** → [knowledge/wiki/containers/index.md](knowledge/wiki/containers/index.md)
- **Cross-cutting infrastructure** (DNS, ingress, mesh, backups, monitoring, auto-deploy, VPS) → [knowledge/wiki/infrastructure/index.md](knowledge/wiki/infrastructure/index.md)
### Knowledge & References
- **Glossary** — [GLOSSARY.md](knowledge/GLOSSARY.md)
- **Incidents & investigations** — [knowledge/sources/investigations/index.md](knowledge/sources/investigations/index.md) (active + [archive](knowledge/sources/investigations/archive/))
- **Plans & design docs** — [plans/index.md](plans/index.md)
- **Hermes agent** (for Hermes-enrolled clients) — [HERMES.md](.agents/HERMES.md)
---
## Conventions
- **Each node page** ends with a `## Changelog` section. Reverse-chronological. Entry format:
```
### YYYY-MM-DD — short title
one or two lines on what changed and why.
```
- **Cross-linking is mandatory.** If a page references another node or system, link to it. Treat orphans as a bug.
- **Live state wins.** When something here disagrees with `pct config` / `docker inspect` / running config, fix the wiki *and* note the change in the relevant changelog.
- **Tracked configs.** A node whose config lives in a Gitea repo (Caddy, Gitea customizations, Artifacto, mule-image) is auto-deployed via webhook — see [auto-deploy](infrastructure/auto-deploy.md). Edits there must be pushed, not left local.
- **No secrets.** This is a private repo on `git.hubris.network`, but still: paths to secret files are fine, secret values are not.
All pages follow:
## Maintaining this wiki
- **File naming.** Foundational docs (entry-points, agent instruction, references) are ALL-CAPS (`AGENTS.md`, `OIKOS.md`, `GLOSSARY.md`); containers use `<id>-<name>.md`; infrastructure pages use lowercase-with-dashes; plans and incidents use `YYYY-MM-DD-slug.md`; skills are `<name>/SKILL.md`. See [page-templates.md](.agents/shared/page-templates.md#file-naming) for the full rules.
- **Voice & vocabulary.** Concise, technical, sysadmin-to-sysadmin. No marketing prose, no puffers (seamless, robust, leverage, etc.). Full rules in [writing-style.md](.agents/shared/writing-style.md).
- **Cross-linking is mandatory.** If a page references a node or system, link to it. Treat orphans as a bug.
- **Live state wins.** When something here disagrees with `pct config` / `docker inspect` / running state, fix the wiki *and* add a changelog entry *in the same session*.
- **Tracked configs.** Pages for configs living in git repos (Caddy, Gitea, Artifacto, mule-image) must note the repo. Edits go through commit+push, never local changes. See [auto-deploy](knowledge/wiki/infrastructure/auto-deploy.md).
- **No secrets.** This is a private repo, but still: reference secret *paths*, never secret *values*.
When you change a node:
1. Update the relevant page (config snapshot, ports, mounts).
2. Add a changelog entry at the bottom of that page.
3. If the change touches a cross-cutting system (DNS, Caddy, Authentik, mesh), update *that* page too and link it from the changelog entry.
4. If it's an incident, add an entry to [`investigations/`](investigations/index.md).
**For agents:** Read [caveman.md](.agents/shared/caveman.md) (terse communication standard). Use templates at [page-templates.md](.agents/shared/page-templates.md) when creating pages.
## See also
---
- [`CONTRIBUTING.md`](CONTRIBUTING.md) — page templates and tone
## Updating the Wiki
### When You Change Infrastructure
1. Update the relevant page (config snapshot, ports, mounts, IP address).
2. Add a `### YYYY-MM-DD — title` entry to the page's `## Changelog` section (reverse chronological order).
3. If the change touches a cross-cutting system (DNS, Caddy, Authentik, mesh), update *that* page too and link from the changelog.
4. If it's an incident, add a record to [`knowledge/sources/investigations/`](knowledge/sources/investigations/index.md).
### When You Restructure the Wiki
1. Update the relevant `index.md` / `README.md` in that section.
2. Add a single-line entry to [`knowledge/log.md`](knowledge/log.md): `## [YYYY-MM-DD] <operation> | <summary>` (e.g., `## [2026-07-06] restructure | split infrastructure/dns into dns.md + dns-advanced.md`).
### The Same-Session Update Rule
**Any meaningful state change made in this session requires a wiki update before the session closes.** A change that touches a container page must also update:
- The `containers/index.md` table (IPs, host, mounts, status)
- The root `README.md` table (if affected)
- The Caddy page site list (if affects `*.hubris.network` routing)
- The DNS / ingress infrastructure pages (if affects routing)
- The `hosts/hubris.md` or `hosts/strong.md` page (if container count changes)
- The `inventory.yaml` host entry (source of truth for `hosts/*.yaml` generation)
- The `knowledge/wiki/infrastructure/topology.md` (regenerate if needed)
Not updating all linked places is a bug. See [page-templates.md — same-session update rule](.agents/shared/page-templates.md#same-session-update-rule).
---
## More Information
- **For Hermes agents** → [HERMES.md](.agents/HERMES.md) (persona, source-of-truth hierarchy, token efficiency)
- **For manual workflows** → [.agents/operations/](.agents/operations/) (commands cheatsheet, agent enrollment, Hermes guide)
- **For skills/runbooks** → [.agents/skills/](.agents/skills/) (load the matching SKILL.md before acting; includes risk class + verification)
- **MCP tools** → [AGENTS.md §3](AGENTS.md#3-the-mcp-server) (available tools, when to use MCP vs files)
- **Page templates & voice** → [.agents/shared/](.agents/shared/) (page-templates.md, writing-style.md, caveman.md, llm-wiki.md)
- **Machine-readable substrate** → `inventory.yaml`, `oikos/policy.yaml`, `oikos/ontology.yaml` (not part of the wiki; see [llm-wiki.md](.agents/shared/llm-wiki.md#rules))

8
api/codegen.yaml Normal file
View File

@@ -0,0 +1,8 @@
# oapi-codegen config — `make generate` regenerates internal/httpapi/gen.
package: gen
output: internal/httpapi/gen/api.gen.go
generate:
models: true
chi-server: true
strict-server: true
embedded-spec: true

2891
api/openapi.yaml Normal file

File diff suppressed because it is too large Load Diff

7
api/redocly.yaml Normal file
View File

@@ -0,0 +1,7 @@
# Redocly lint config for api/openapi.yaml (CI runs: redocly lint api/openapi.yaml)
extends:
- recommended
rules:
# Every operation declares `default` → RFC 9457 problem+json instead of
# enumerating each 4XX (plan R3-3); oapi-codegen handles `default` fine.
operation-4xx-response: off

View File

@@ -37,6 +37,21 @@ INVENTORY = CONTEXT / "inventory.yaml"
HOSTS_DIR = CONTEXT / "hosts"
AGE_KEY = Path(os.environ.get("SOPS_AGE_KEY_FILE", "/etc/age/key.txt"))
# Oikos kernel modules (policy classification, ontology relations, change
# ledger). Optional at import time so a stale/partial checkout degrades to
# "feature unavailable" instead of crashing every subcommand.
sys.path.insert(0, str(CONTEXT))
try:
from oikos import approve as oikos_approve
from oikos import decide as oikos_decide
from oikos import ledger as oikos_ledger
from oikos import policy as oikos_policy
from oikos import relations as oikos_relations
from oikos import signal as oikos_signal
except ImportError:
oikos_approve = oikos_decide = oikos_ledger = oikos_policy = None
oikos_relations = oikos_signal = None
# ---------- helpers ----------
@@ -172,6 +187,25 @@ def service_backend_host(name: str) -> str:
return service(name)["backend"]
def _record_change(entity: str, action: str, risk: str, *,
result: str | None = None, verification: str | None = None,
approval_ref: str | None = None) -> None:
"""Append a ledger entry and push it standalone (used by mutations that
don't already go through push_inventory, e.g. restart)."""
if oikos_ledger is None:
return
oikos_ledger.append(entity, action, risk, result=result, verification=verification,
approval_ref=approval_ref)
try:
subprocess.run(["git", "add", "ledger/"], check=True, cwd=CONTEXT)
if subprocess.run(["git", "diff", "--cached", "--quiet"], cwd=CONTEXT).returncode != 0:
subprocess.run(["git", "commit", "-m", f"ledger: {entity} {action} ({risk})"],
check=True, cwd=CONTEXT)
subprocess.run(["git", "push"], check=True, cwd=CONTEXT)
except subprocess.CalledProcessError as e:
print(f"warning: could not commit/push ledger entry: {e}", file=sys.stderr)
def push_inventory(message: str, extra_paths: list[str] | None = None) -> None:
"""Stage + commit + push inventory + regenerated hosts/ (+ any extras)."""
subprocess.run(["python3", str(CONTEXT / "mcp" / "build_host_files.py")],
@@ -570,11 +604,31 @@ def cmd_restart(args: argparse.Namespace) -> int:
svc = args.service
host_name = service_backend_host(svc)
unit = service(svc).get("systemd_unit", svc)
risk = (oikos_policy.classify_action("service-restart", svc)
if oikos_policy else "reversible_low") or "reversible_low"
approval = oikos_policy.approval_for(risk) if oikos_policy else "none"
# Mechanical gate: config_mutation/destructive risk classes require a
# live grant regardless of -y/interactivity — an agent (or a human
# bypassing the confirm() prompt with -y) cannot mutate a gated service
# without a real oikos/approve.py approval. See oikos/policy.yaml.
if approval != "none":
if not args.approval_id:
die(f"restarting '{svc}' is risk class '{risk}' (approval: {approval}) — "
f"pass --approval-id <id> from an approved 'homelab approval request'")
ok, reason = oikos_approve.check_grant(args.approval_id, f"service:{svc}", "service-restart")
if not ok:
die(f"approval {args.approval_id} not valid for this action: {reason}")
if not args.yes:
if not confirm(f"restart systemd unit '{unit}' on {host_name}?"):
return 1
base = ssh_base(host_name)
return subprocess.call(base + ["--", "systemctl", "restart", unit])
rc = subprocess.call(base + ["--", "systemctl", "restart", unit])
_record_change(f"service:{svc}", "restart", risk,
result=("ok" if rc == 0 else f"failed rc={rc}"),
approval_ref=args.approval_id)
return rc
def cmd_open(args: argparse.Namespace) -> int:
@@ -1031,14 +1085,19 @@ def cmd_refresh_creds(args: argparse.Namespace) -> int:
def cmd_sync(args: argparse.Namespace) -> int:
# Unlike every other mutating command here, this one had no os.geteuid()
# guard — always shelled out to sudo. Fails outright with "No such file
# or directory: 'sudo'" on minimal root-only Linux images (no sudo
# binary installed at all) reached via `ssh root@host`, e.g. strong.
needs_sudo = os.geteuid() != 0
if sys.platform == "darwin":
return subprocess.call(
["sudo", "launchctl", "kickstart", "-k",
"system/network.hubris.homelab-context-sync"]
)
return subprocess.call(
["sudo", "systemctl", "start", "homelab-context-sync.service"]
)
cmd = ["launchctl", "kickstart", "-k",
"system/network.hubris.homelab-context-sync"]
else:
cmd = ["systemctl", "start", "homelab-context-sync.service"]
if needs_sudo:
cmd = ["sudo"] + cmd
return subprocess.call(cmd)
def cmd_mcp(args: argparse.Namespace) -> int:
@@ -1098,9 +1157,11 @@ def cmd_client_add(args: argparse.Namespace) -> int:
print("granting hermes-only secrets...")
_grant_shared_secrets(pubkey, HERMES_SECRETS)
commit_subject = f"client-add: {name} (finalize age_pubkey + grant shared + hermes secrets)"
if oikos_ledger is not None:
oikos_ledger.append(f"host:{name}", "client-add-finalize", "config_mutation", result="ok")
push_inventory(
commit_subject,
extra_paths=[".sops.yaml", "secrets/"],
extra_paths=[".sops.yaml", "secrets/", "ledger/"],
)
print(f"finalized {name}.")
return 0
@@ -1162,9 +1223,11 @@ def cmd_client_remove(args: argparse.Namespace) -> int:
print(f" issuance revoke failed: {e}")
# 4. Commit + push (extras: .sops.yaml + secrets/ may also have changed).
if oikos_ledger is not None:
oikos_ledger.append(f"host:{name}", "client-remove", "destructive", result="ok")
push_inventory(
f"client-remove: {name}",
extra_paths=[".sops.yaml", "secrets/"],
extra_paths=[".sops.yaml", "secrets/", "ledger/"],
)
print()
@@ -1177,6 +1240,211 @@ def cmd_client_remove(args: argparse.Namespace) -> int:
return 0
def _require_oikos() -> None:
if oikos_policy is None or oikos_relations is None or oikos_ledger is None:
die("oikos/ kernel modules not importable — is this checkout up to date?")
def cmd_service(args: argparse.Namespace) -> int:
"""Service Console v0 — explain/health/docs/log/actions/history for one service."""
_require_oikos()
name = args.name
svc = service(name) # dies with a clear message if unknown
if args.action == "explain":
card = CONTEXT / "oikos" / "cards" / f"service-{name}.md"
if not card.exists():
die(f"no context card for {name} — run: python3 oikos/gen-topology.py")
print(card.read_text())
return 0
if args.action == "health":
url = svc.get("url") or svc.get("endpoint")
if not url:
die(f"service {name} has no url/endpoint in inventory")
if not args.live:
try:
from oikos import scheduler as oikos_scheduler
cached = oikos_scheduler.cached_service_health(name)
except ImportError:
cached = None
if cached is not None and cached.get("checked"):
status = "ok" if cached.get("ok") else "unhealthy"
print(f"{name}: {cached.get('checked_url', url)} -> "
f"{cached.get('http_code') or 'no response'} ({status}, "
f"as of {cached['as_of']} — pass --live to force a fresh probe)")
return 0 if cached.get("ok") else 1
proc = subprocess.run(
["curl", "-sS", "-o", "/dev/null", "-w", "%{http_code}", "--max-time", "5", url],
capture_output=True, text=True,
)
code = proc.stdout.strip() or "no response"
print(f"{name}: {url} -> {code} (live probe)")
return 0 if code.startswith(("2", "3")) else 1
if args.action == "docs":
doc = svc.get("doc_page")
if not doc:
die(f"no doc_page recorded for {name} in inventory.yaml")
path = CONTEXT / doc
if not path.exists():
die(f"doc_page {doc} does not exist")
print(path.read_text())
return 0
if args.action == "log":
return cmd_logs(argparse.Namespace(service=name, lines=args.lines, follow=False))
if args.action == "actions":
for a in oikos_policy.safe_actions_for_service(name, svc):
print(f"{a['action']:<24} {a['risk']:<16} approval={a['approval']}")
return 0
if args.action == "history":
entries = oikos_ledger.history(f"service:{name}", limit=args.limit)
if not entries:
print(f"(no ledger entries for service:{name} yet)")
for e in entries:
print(json.dumps(e))
return 0
die(f"unknown service action: {args.action}")
def cmd_change_preflight(args: argparse.Namespace) -> int:
"""Dry-run report before mutating a service: health, risk class, approval
requirement, and the verification command to run after."""
_require_oikos()
name = args.service
svc = service(name)
risk = (oikos_policy.classify_action("tracked-config-edit", name)
if svc.get("config_repo")
else oikos_policy.classify_action("service-restart", name)) or "config_mutation"
approval = oikos_policy.approval_for(risk)
print(f"Preflight: {name}")
print(f" risk class: {risk} (approval: {approval})")
url = svc.get("url") or svc.get("endpoint")
if url:
proc = subprocess.run(
["curl", "-sS", "-o", "/dev/null", "-w", "%{http_code}", "--max-time", "3", url],
capture_output=True, text=True,
)
print(f" current health: {url} -> {proc.stdout.strip() or 'no response'}")
if svc.get("config_repo"):
print(f" config repo: {svc['config_repo']} "
f"(verify the backend's working tree is clean before editing)")
if svc.get("risk_notes"):
print(f" risk notes: {svc['risk_notes']}")
print(f" verification after change: "
+ (f"curl -sf {url}" if url else f"homelab logs {name}"))
if approval != "none":
print(f" requires operator approval before mutating ({approval})")
return 0
def cmd_node_relations(args: argparse.Namespace) -> int:
"""Walk the ontology graph both directions for a host or service name."""
_require_oikos()
results = oikos_relations.relations_for_name(args.name)
if not results:
die(f"unknown entity: {args.name}")
for r in results:
print(f"entity: {r['entity']}")
print(f" impacts: {', '.join(r['impacts']) or '(none)'}")
print(f" affected by: {', '.join(r['affected_by']) or '(none)'}")
print(f" full blast radius: {', '.join(r['blast_radius']) or '(none)'}")
return 0
def cmd_decide(args: argparse.Namespace) -> int:
"""Route a proposed action: auto-act or escalate. See oikos/decide.py."""
_require_oikos()
result = oikos_decide.classify(args.action, args.entity, service_name=args.service_name,
record=not args.no_record)
print(json.dumps(result, indent=2))
return 0 if result["route"] == "auto-act" else 1
def cmd_approval_request(args: argparse.Namespace) -> int:
_require_oikos()
entry = oikos_approve.request(
args.entity, args.action, args.risk, args.evidence,
verification=args.verification, requires_phrase=args.requires_phrase,
ttl_hours=args.ttl_hours,
)
print(json.dumps({k: v for k, v in entry.items() if k != "matrix_message"}, indent=2))
print()
print("--- post this to Matrix ---")
print(entry["matrix_message"])
return 0
def cmd_approval_list(args: argparse.Namespace) -> int:
_require_oikos()
for e in oikos_approve.list_approvals(state=args.state):
print(json.dumps(e))
return 0
def cmd_approval_reply(args: argparse.Namespace) -> int:
_require_oikos()
try:
entry = oikos_approve.reply(args.id, args.decision, phrase=args.phrase,
decided_by=args.decided_by)
except (ValueError, RuntimeError) as e:
die(str(e))
print(json.dumps(entry, indent=2))
return 0
def cmd_approval_check(args: argparse.Namespace) -> int:
_require_oikos()
ok, reason = oikos_approve.check_grant(args.id, args.entity, args.action)
print(f"{'GRANTED' if ok else 'DENIED'}: {reason}")
return 0 if ok else 1
def cmd_signal_raise(args: argparse.Namespace) -> int:
_require_oikos()
action = None
if args.action_runbook or args.action_risk:
action = {"runbook": args.action_runbook, "risk": args.action_risk}
entry = oikos_signal.raise_signal(args.kind, args.severity, args.entity, args.evidence,
likely_cause=args.likely_cause,
recommended_action=action,
verification=args.verification)
print(json.dumps(entry, indent=2))
return 0
def cmd_signal_list(args: argparse.Namespace) -> int:
_require_oikos()
for e in oikos_signal.list_signals(state=args.state, entity=args.entity,
severity=args.severity, kind=args.kind):
print(json.dumps(e))
return 0
def cmd_signal_ack(args: argparse.Namespace) -> int:
_require_oikos()
print(json.dumps(oikos_signal.acknowledge(args.id, args.note), indent=2))
return 0
def cmd_signal_resolve(args: argparse.Namespace) -> int:
_require_oikos()
print(json.dumps(oikos_signal.resolve(args.id, args.note), indent=2))
return 0
def cmd_signal_mute(args: argparse.Namespace) -> int:
_require_oikos()
print(json.dumps(oikos_signal.mute(args.id, args.ttl_hours, args.note), indent=2))
return 0
def cmd_nuke(args: argparse.Namespace) -> int:
name = args.name
if not args.yes:
@@ -1494,6 +1762,9 @@ def main() -> int:
sp = sub.add_parser("restart", help="restart a service")
sp.add_argument("service")
sp.add_argument("--yes", "-y", action="store_true")
sp.add_argument("--approval-id", default=None,
help="required if the service's risk class needs approval "
"(see 'homelab approval request')")
sp.set_defaults(func=cmd_restart)
sp = sub.add_parser("open", help="open a service's URL in browser")
@@ -1549,6 +1820,101 @@ def main() -> int:
help="skip the pre-flight dpkg-audit gate AND proceed past snapshot failures")
sp.set_defaults(func=cmd_apt_upgrade)
sp = sub.add_parser("service", help="Service Console v0 — explain/health/docs/log/actions/history")
sp.add_argument("name")
sp.add_argument("action", choices=["explain", "health", "docs", "log", "actions", "history"])
sp.add_argument("--lines", "-n", type=int, default=200, help="for 'log'")
sp.add_argument("--limit", type=int, default=20, help="for 'history'")
sp.add_argument("--live", action="store_true",
help="for 'health': force a fresh probe instead of the scheduler's cache")
sp.set_defaults(func=cmd_service)
change = sub.add_parser("change", help="change/mutation workflow")
chsub = change.add_subparsers(dest="action", required=True)
ch_preflight = chsub.add_parser("preflight")
ch_preflight.add_argument("service")
ch_preflight.set_defaults(func=cmd_change_preflight)
sp = sub.add_parser("node", help="ontology queries on a host/service")
sp.add_argument("name")
sp.add_argument("action", choices=["relations"])
sp.set_defaults(func=cmd_node_relations)
sp = sub.add_parser("decide", help="classify a proposed action: auto-act or escalate")
sp.add_argument("action")
sp.add_argument("entity")
sp.add_argument("--service-name", default=None)
sp.add_argument("--no-record", action="store_true",
help="skip writing this classification to the change ledger")
sp.set_defaults(func=cmd_decide)
approval = sub.add_parser("approval", help="approval-engine requests (escalate route)")
apsub = approval.add_subparsers(dest="action", required=True)
ap_req = apsub.add_parser("request")
ap_req.add_argument("entity")
ap_req.add_argument("action")
ap_req.add_argument("risk")
ap_req.add_argument("evidence")
ap_req.add_argument("--verification")
ap_req.add_argument("--requires-phrase", action="store_true")
ap_req.add_argument("--ttl-hours", type=int, default=24)
ap_req.set_defaults(func=cmd_approval_request)
ap_list = apsub.add_parser("list")
ap_list.add_argument("--state", choices=["pending", "approved", "denied", "expired", "executed"])
ap_list.set_defaults(func=cmd_approval_list)
ap_reply = apsub.add_parser("reply")
ap_reply.add_argument("id")
ap_reply.add_argument("decision", choices=["approve", "deny"])
ap_reply.add_argument("--phrase")
ap_reply.add_argument("--decided-by")
ap_reply.set_defaults(func=cmd_approval_reply)
ap_check = apsub.add_parser("check")
ap_check.add_argument("id")
ap_check.add_argument("entity")
ap_check.add_argument("action")
ap_check.set_defaults(func=cmd_approval_check)
signal = sub.add_parser("signal", help="the attention layer (oikos/signal.py)")
sigsub = signal.add_subparsers(dest="action", required=True)
sig_raise = sigsub.add_parser("raise")
sig_raise.add_argument("kind")
sig_raise.add_argument("severity", choices=["info", "warning", "critical"])
sig_raise.add_argument("entity")
sig_raise.add_argument("evidence")
sig_raise.add_argument("--likely-cause")
sig_raise.add_argument("--action-runbook")
sig_raise.add_argument("--action-risk")
sig_raise.add_argument("--verification")
sig_raise.set_defaults(func=cmd_signal_raise)
sig_list = sigsub.add_parser("list")
sig_list.add_argument("--state", choices=["raised", "acknowledged", "acting", "resolved", "muted"])
sig_list.add_argument("--entity")
sig_list.add_argument("--severity", choices=["info", "warning", "critical"])
sig_list.add_argument("--kind")
sig_list.set_defaults(func=cmd_signal_list)
sig_ack = sigsub.add_parser("ack")
sig_ack.add_argument("id")
sig_ack.add_argument("--note")
sig_ack.set_defaults(func=cmd_signal_ack)
sig_resolve = sigsub.add_parser("resolve")
sig_resolve.add_argument("id")
sig_resolve.add_argument("--note")
sig_resolve.set_defaults(func=cmd_signal_resolve)
sig_mute = sigsub.add_parser("mute")
sig_mute.add_argument("id")
sig_mute.add_argument("--ttl-hours", type=int, default=24)
sig_mute.add_argument("--note")
sig_mute.set_defaults(func=cmd_signal_mute)
sp = sub.add_parser("nuke", help="shred /etc/age/key.txt + /opt/homelab-context on a host")
sp.add_argument("name")
sp.add_argument("--yes", "-y", action="store_true")
@@ -1563,7 +1929,7 @@ def main() -> int:
csub_add.add_argument("--with-hermes", action="store_true",
help="also grant secrets/openrouter-api-key.yaml so this "
"host can run the Hermes agent (see "
"operations/hermes-agent.md). Combine with --finalize-pubkey.")
".agents/operations/hermes-agent.md). Combine with --finalize-pubkey.")
csub_add.set_defaults(func=cmd_client_add)
csub_rm = csub.add_parser("remove")
csub_rm.add_argument("name")

View File

@@ -7,7 +7,11 @@
# curl ... | sudo bash -s -- --with-mcp # also wire Claude's .mcp.json
# curl ... | sudo bash -s -- --with-hermes # also install Goose + Hermes wrapper
# curl ... | sudo bash -s -- --dry-run # show what would happen
# curl ... | sudo bash -s -- --no-secrets # skip age-key issuance
# curl ... | sudo bash -s -- --no-secrets # skip age-key issuance entirely
# curl ... | sudo bash -s -- --no-mesh # get secrets over LAN only, skip
# # installing/connecting Netbird
# # (host must be on 192.168.8.0/24
# # or otherwise reach secrets.hubris.network)
#
# Prerequisites the script verifies:
# - running as root
@@ -31,6 +35,7 @@ WITH_MCP=0
WITH_HERMES=0
DRY_RUN=0
NO_SECRETS=0
NO_MESH=0
GITEA_TOKEN="${HOMELAB_GITEA_TOKEN:-}"
GITEA_USER="${HOMELAB_GITEA_USER:-dtoro}"
@@ -41,6 +46,7 @@ while [ $# -gt 0 ]; do
--with-hermes) WITH_HERMES=1; shift ;;
--dry-run) DRY_RUN=1; shift ;;
--no-secrets) NO_SECRETS=1; shift ;;
--no-mesh) NO_MESH=1; shift ;;
--gitea-token) GITEA_TOKEN="$2"; shift 2 ;;
--gitea-user) GITEA_USER="$2"; shift 2 ;;
--help|-h)
@@ -83,6 +89,21 @@ run() {
fi
}
# Run a command as the enrolling human user when one exists (i.e. this
# script was invoked via `sudo bash bootstrap.sh` from a real login), and
# directly otherwise. Minimal Linux images (bare Proxmox/Debian installs
# reached via `ssh root@host`) often don't even have a `sudo` binary
# installed — calling `sudo -u root ...` on those unconditionally fails
# with "sudo: command not found" even though we're already root and don't
# need to switch users at all.
run_as() {
if [ -n "${SUDO_USER:-}" ] && [ "$SUDO_USER" != "root" ]; then
sudo -u "$SUDO_USER" -- "$@"
else
"$@"
fi
}
# -------- preflight --------
if [ "$(id -u)" -ne 0 ]; then
echo "bootstrap.sh must run as root (use sudo)." >&2
@@ -118,6 +139,22 @@ fi
if [ "$NO_SECRETS" -eq 0 ]; then
for cmd in age sops; do command -v "$cmd" >/dev/null || missing+=("$cmd"); done
fi
# sops isn't a real Debian/Fedora package (there is no apt/dnf "sops"), so it
# always needs the direct-binary-download path, on both distros. Only Darwin
# (brew) can install it via a package manager.
install_sops_binary() {
local sops_version=v3.9.4
local arch
arch="$(uname -m)"
case "$arch" in
x86_64|amd64) arch=amd64 ;;
aarch64|arm64) arch=arm64 ;;
*) echo "[bootstrap] unsupported arch for sops binary download: $arch" >&2; return 1 ;;
esac
curl -fsSL "https://github.com/getsops/sops/releases/download/${sops_version}/sops-${sops_version}.linux.${arch}" \
-o /usr/local/bin/sops && chmod +x /usr/local/bin/sops
}
if [ "${#missing[@]}" -gt 0 ]; then
if [ "$DRY_RUN" -eq 1 ]; then
echo "+ would install missing tools: ${missing[*]}"
@@ -138,13 +175,23 @@ if [ "${#missing[@]}" -gt 0 ]; then
for m in "${missing[@]}"; do
case "$m" in
python3-yaml) dnf_list+=("python3-pyyaml") ;;
sops) install_sops_binary ;;
*) dnf_list+=("$m") ;;
esac
done
dnf install -y "${dnf_list[@]}"
[ "${#dnf_list[@]}" -gt 0 ] && dnf install -y "${dnf_list[@]}"
elif command -v apt-get >/dev/null 2>&1; then
DEBIAN_FRONTEND=noninteractive apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y "${missing[@]}"
apt_list=()
for m in "${missing[@]}"; do
case "$m" in
sops) install_sops_binary ;;
*) apt_list+=("$m") ;;
esac
done
if [ "${#apt_list[@]}" -gt 0 ]; then
DEBIAN_FRONTEND=noninteractive apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y "${apt_list[@]}"
fi
else
echo "[bootstrap] no supported package manager for: ${missing[*]}" >&2
echo "[bootstrap] install with your package manager + re-run" >&2
@@ -165,10 +212,14 @@ if [ "${#missing[@]}" -gt 0 ]; then
fi
# -------- ensure netbird is installed + connected (workstation/VM hosts) --------
# Skipped on --no-secrets (LXCs that route via the LAN already) and --dry-run.
# Installs netbird if missing, then drives `netbird up` against the homelab
# management server. The operator clicks the printed device-code URL once.
if [ "$NO_SECRETS" -eq 0 ] && [ "$DRY_RUN" -eq 0 ]; then
# Skipped on --no-secrets (LXCs that route via the LAN already), --no-mesh
# (explicit opt-out — secrets issuance still works if the mesh check below
# falls back to LAN reachability), and --dry-run. Installs netbird if
# missing, then drives `netbird up` against the homelab management server.
# The operator clicks the printed device-code URL once — this blocks
# indefinitely if nobody approves it, so don't skip --no-mesh on a host
# nobody's watching interactively.
if [ "$NO_SECRETS" -eq 0 ] && [ "$DRY_RUN" -eq 0 ] && [ "$NO_MESH" -eq 0 ]; then
if ! command -v netbird >/dev/null 2>&1 && ! command -v tailscale >/dev/null 2>&1; then
echo "[bootstrap] no mesh CLI found; installing netbird..."
if [ "$OS" = "Darwin" ]; then
@@ -430,9 +481,9 @@ if [ "$WITH_HERMES" -eq 1 ]; then
echo "+ would run upstream goose installer and symlink to /usr/local/bin/goose"
else
# Upstream installer drops the binary at ~/.local/bin/goose for the
# invoking user. We run it as $H_USER then symlink system-wide.
sudo -u "$H_USER" \
env CONFIGURE=false \
# invoking user. We run it as $H_USER (via run_as) then symlink
# system-wide.
run_as env CONFIGURE=false \
bash -c 'curl -fsSL https://github.com/aaif-goose/goose/releases/download/stable/download_cli.sh | bash'
if [ -x "$H_HOME/.local/bin/goose" ]; then
ln -sfn "$H_HOME/.local/bin/goose" /usr/local/bin/goose
@@ -456,7 +507,7 @@ if [ "$WITH_HERMES" -eq 1 ]; then
Linux) HERMES_LINK=/root/HERMES.md ;;
Darwin) HERMES_LINK=/etc/HERMES.md ;;
esac
run "ln -sfn '$CLONE_DIR/HERMES.md' '$HERMES_LINK'"
run "ln -sfn '$CLONE_DIR/.agents/HERMES.md' '$HERMES_LINK'"
echo "[bootstrap] linked HERMES.md → $HERMES_LINK"
# 4. Drop the Goose config. Idempotent YAML merge — preserves any keys the
@@ -512,7 +563,7 @@ PYEOF
# 5. Symlink HERMES.md as the global .goosehints — Goose injects it into
# the system prompt on every session start.
run "ln -sfn '$CLONE_DIR/HERMES.md' '$GOOSEHINTS'"
run "ln -sfn '$CLONE_DIR/.agents/HERMES.md' '$GOOSEHINTS'"
if [ "$DRY_RUN" -eq 0 ]; then
chown -h "$H_USER" "$GOOSEHINTS" 2>/dev/null || true
fi
@@ -595,7 +646,7 @@ if [ "$HKIND" != "lxc" ] && [ "$HKIND" != "vm" ]; then
# Make sure pipx is available; OS-specific install.
if ! command -v pipx >/dev/null 2>&1; then
if [ "$OS" = "Darwin" ] && command -v brew >/dev/null 2>&1; then
sudo -u "${SUDO_USER:-$USER}" brew install pipx 2>&1 | tail -2 || true
run_as brew install pipx 2>&1 | tail -2 || true
elif command -v dnf >/dev/null 2>&1; then
dnf install -y pipx 2>&1 | tail -2 || true
elif command -v apt-get >/dev/null 2>&1; then
@@ -603,9 +654,9 @@ if [ "$HKIND" != "lxc" ] && [ "$HKIND" != "vm" ]; then
fi
fi
if command -v pipx >/dev/null 2>&1; then
INVOKING_USER="${SUDO_USER:-$USER}"
sudo -u "$INVOKING_USER" -- bash -lc "pipx install 'mcp[cli]'" 2>&1 | tail -3 || true
sudo -u "$INVOKING_USER" -- bash -lc "pipx ensurepath" >/dev/null 2>&1 || true
INVOKING_USER="${SUDO_USER:-root}"
run_as bash -lc "pipx install 'mcp[cli]'" 2>&1 | tail -3 || true
run_as bash -lc "pipx ensurepath" >/dev/null 2>&1 || true
echo "[bootstrap] mcp CLI installed for $INVOKING_USER via pipx"
else
echo "[bootstrap] WARNING: pipx unavailable; install manually: pipx install 'mcp[cli]'" >&2

254
cmd/oikos/main.go Normal file
View File

@@ -0,0 +1,254 @@
package main
import (
"context"
"fmt"
"log/slog"
"os"
"os/signal"
"syscall"
"net/http"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/httpapi"
"github.com/dtoro/oikos/internal/observability"
"github.com/jackc/pgx/v5"
)
// SchedulerRunner is set by the scheduler init() to avoid circular imports.
var SchedulerRunner func(context.Context, *db.Pool, config.Config)
// NotifierRunner is set by the notifier init() to avoid circular imports.
var NotifierRunner func(context.Context, *db.Pool, config.Config)
func main() {
if len(os.Args) < 2 {
usage()
os.Exit(1)
}
role := os.Args[1]
cfg := config.FromEnv()
// Structured logging (slog)
logger := observability.NewLogger(cfg.Debug)
slog.SetDefault(logger)
slog.Info("starting oikos", "role", role, "config", cfg)
ctx, cancel := signal.NotifyContext(context.Background(),
syscall.SIGTERM, syscall.SIGINT)
defer cancel()
switch role {
case "migrate":
if err := runMigrate(ctx, cfg); err != nil {
slog.Error("migrate failed", "error", err)
os.Exit(1)
}
case "seed":
if err := runSeed(ctx, cfg); err != nil {
slog.Error("seed failed", "error", err)
os.Exit(1)
}
case "export":
if err := runExport(ctx, cfg); err != nil {
slog.Error("export failed", "error", err)
os.Exit(1)
}
case "api":
if err := runAPI(ctx, cfg); err != nil {
slog.Error("api failed", "error", err)
os.Exit(1)
}
case "scheduler":
if SchedulerRunner != nil {
SchedulerRunner(ctx, nil, cfg)
} else {
slog.Error("scheduler not compiled in (import internal/scheduler)")
os.Exit(1)
}
case "notifier":
if NotifierRunner != nil {
NotifierRunner(ctx, nil, cfg)
} else {
slog.Error("notifier not compiled in (import internal/notifier)")
os.Exit(1)
}
case "all":
slog.Info("all role not yet implemented (runs api + scheduler + notifier in one process)")
os.Exit(1)
case "version":
fmt.Println("oikos dev (Phase 1)")
case "help", "--help", "-h":
usage()
default:
fmt.Fprintf(os.Stderr, "unknown role: %s\n", role)
usage()
os.Exit(1)
}
}
func usage() {
fmt.Println(`oikos — the homelab OS
Usage: oikos <role> [flags]
Roles:
migrate Run database migrations (forward-only, idempotent)
seed Ingest seed YAML files into the database
export Export DB state back to seed YAMLs (DR / version control)
api Run the REST + MCP API server (Phase 2)
scheduler Run the observe + act loop (Phase 3)
notifier Run the notification service (Phase 3)
all Run all roles in one process (dev mode)
version Print version info
Environment:
OIKOS_DATABASE_URL Postgres connection string
OIKOS_API_LISTEN API listen address (default :8090)
OIKOS_ENV Environment (dev, prod)
OIKOS_DEBUG Enable verbose logging (true/1)
OIKOS_SEEDS_DIR Path to seeds directory (default: seeds)
OIKOS_MCP_BEARER_TOKEN Shared secret for MCP auth`)
}
func runMigrate(ctx context.Context, cfg config.Config) error {
pool, err := db.New(ctx, cfg.DatabaseURL)
if err != nil {
return err
}
defer pool.Close()
slog.Info("running migrations")
if err := pool.Migrate(ctx); err != nil {
return err
}
slog.Info("migrations complete")
return nil
}
func runSeed(ctx context.Context, cfg config.Config) error {
pool, err := db.New(ctx, cfg.DatabaseURL)
if err != nil {
return err
}
defer pool.Close()
// Ensure migrations are applied first
if err := pool.Migrate(ctx); err != nil {
return fmt.Errorf("migrations: %w", err)
}
seedsDir := cfg.SeedsDir
if seedsDir == "" {
seedsDir = "seeds"
}
// Ingest ontology seed
ontoContent, err := os.ReadFile(seedsDir + "/ontology.yaml")
if err != nil {
return fmt.Errorf("read ontology seed: %w", err)
}
err = pool.SeedIngest(ctx, "ontology.yaml", ontoContent,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
r, err := db.IngestOntologySeed(ctx, tx, data)
if err != nil {
return err
}
slog.Info("ontology ingested",
"lifecycles", r.Lifecycles,
"entity_types", r.EntityTypes,
"relationship_types", r.RelationshipTypes)
return nil
})
if err != nil {
return err
}
// Ingest inventory seed
invContent, err := os.ReadFile(seedsDir + "/inventory.yaml")
if err != nil {
return fmt.Errorf("read inventory seed: %w", err)
}
err = pool.SeedIngest(ctx, "inventory.yaml", invContent,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
r, err := db.IngestInventorySeed(ctx, tx, data)
if err != nil {
return err
}
slog.Info("inventory ingested",
"entities", r.Entities,
"relationships", r.Relationships)
return nil
})
if err != nil {
return err
}
// Ingest policy seed
polContent, err := os.ReadFile(seedsDir + "/policy.yaml")
if err != nil {
return fmt.Errorf("read policy seed: %w", err)
}
err = pool.SeedIngest(ctx, "policy.yaml", polContent,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
r, err := db.IngestPolicySeed(ctx, tx, data)
if err != nil {
return err
}
slog.Info("policy ingested",
"risk_classes", r.RiskClasses,
"approval_rules", r.ApprovalRules,
"autonomy_settings", r.AutonomySettings)
return nil
})
if err != nil {
return err
}
slog.Info("seed ingest complete")
return nil
}
func runAPI(ctx context.Context, cfg config.Config) error {
pool, err := db.New(ctx, cfg.DatabaseURL)
if err != nil {
return err
}
defer pool.Close()
if err := pool.Migrate(ctx); err != nil {
return fmt.Errorf("migrations: %w", err)
}
err = httpapi.ListenAndServe(ctx, pool, cfg)
if err == http.ErrServerClosed {
return nil
}
return err
}
func runExport(ctx context.Context, cfg config.Config) error {
pool, err := db.New(ctx, cfg.DatabaseURL)
if err != nil {
return err
}
defer pool.Close()
exports, err := db.ExportToYAML(ctx, pool)
if err != nil {
return err
}
for name, content := range exports {
path := cfg.SeedsDir + "/" + name
if err := os.WriteFile(path, content, 0644); err != nil {
return fmt.Errorf("write %s: %w", path, err)
}
slog.Info("exported", "file", path, "bytes", len(content))
}
return nil
}

21
compose/oikos/Dockerfile Normal file
View File

@@ -0,0 +1,21 @@
# Multi-stage Dockerfile for Oikos (ADR 0001: single binary)
FROM golang:1.26-alpine AS builder
RUN apk add --no-cache git ca-certificates
WORKDIR /build
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /oikos -tags timetzdata -ldflags="-s -w" ./cmd/oikos
# --- Runtime: distroless static ---
FROM gcr.io/distroless/static:nonroot
COPY --from=builder /oikos /oikos
COPY --from=builder /build/seeds /seeds
COPY --from=builder /build/migrations /migrations
ENTRYPOINT ["/oikos"]

View File

@@ -1,34 +0,0 @@
# 101 — `jellyfin`
Media server: serves the movies / TV / anime / music / audiobooks / podcasts libraries from `/mnt/library` to LAN clients.
## At a glance
- **Hostname:** `jellyfin`
- **IP:** `192.168.8.206`
- **Privilege:** **unprivileged** + idmap (so it can write to the `media` group on `/mnt/library`)
- **Resources:** 2 cores / 4 GiB RAM / 16 GiB rootfs
- **Mounts:** `/mnt/library``/mnt/library`
- **Public hostname:** [`media.hubris.network`](../infrastructure/dns.md) → [caddy](121-caddy.md) → `:8096`
## Service / port map
| Service | Listen | Notes |
| -------- | ------ | ----- |
| jellyfin | `:8096` | HTTP (caddy terminates TLS) |
## Permissions
Member of the [media GID 10000](../infrastructure/media-permissions.md) standard. Service user `jellyfin` is in the `media` group inside the container; idmap block in `/etc/pve/lxc/101.conf` maps in-container GID 10000 to host GID 10000.
## Related
- [Caddy reverse proxy](121-caddy.md)
- [Media permissions](../infrastructure/media-permissions.md)
- [arriman](122-arriman.md) — \*arr stack writes the libraries jellyfin reads
- [DNS split-horizon](../infrastructure/dns.md)
## Changelog
### 2026-04-28 — wiki entry created
Initial documentation. No config changes.
### 2026-04-20 — joined the `media` GID 10000 standard
Idmap block applied; in-container `media` group at GID 10000 mapped to host GID 10000. See [media permissions](../infrastructure/media-permissions.md). Config backup: `/root/101.conf.bak.*`.

View File

@@ -1,53 +0,0 @@
# LXC containers — index
All containers live on [`hubris`](../hosts/hubris.md). Each row links to the per-container page.
| ID | Name | IP | Priv | Cores | RAM | Disk | Mounts | Public hostname | Status |
| --- | ---------------- | --------------- | ---- | ----- | ----- | ----- | --------------------- | ------------------------------------- | -------- |
| 101 | [jellyfin](101-jellyfin.md) | 192.168.8.206 | unpriv (idmap) | 2 | 4 GiB | 16 GiB | `/mnt/library` | `media.hubris.network` | running |
| 103 | [paperless](103-paperless.md) | 192.168.8.130 | priv | 2 | 3 GiB | 8 GiB | `/mnt/library` | `paperless.hubris.network` | running |
| 104 | [gitea](104-gitea.md) | 192.168.8.121 | priv | 1 | 1 GiB | 8 GiB | `/mnt/library` | `git.hubris.network` | running |
| 105 | [apps](105-apps.md) | 192.168.8.205 | priv | 2 | 4 GiB | 30 GiB | `/mnt/library` | `docker` / `artifacto` / `blog` | running |
| 114 | [nextcloud](114-nextcloud.md) | 192.168.8.224 | priv | 4 | 6 GiB | 25 GiB | `/mnt/library` | `cloud.hubris.network` | running |
| 118 | [elementsynapse](118-elementsynapse.md) | 192.168.8.239 | unpriv | 1 | 2 GiB | 8 GiB | — | `matrix.hubris.network` | running |
| 119 | [sophia](119-sophia.md) | 192.168.8.157 | priv | 2 | 1 GiB | 10 GiB | `/mnt/library` | — | running |
| 120 | [mule-images](120-mule-images.md) | 192.168.8.136 | priv | 6 | 12 GiB | 60 GiB | `/mnt/library` + `/dev/dri` (iGPU passthrough) | `photos.hubris.network` | running |
| 121 | [caddy](121-caddy.md) | 192.168.8.175 | unpriv | 1 | 512 MiB | 6 GiB | — | (terminates all `*.hubris.network`) | running |
| 122 | [arriman](122-arriman.md) | 192.168.8.132 | priv | 4 | 8 GiB | 24 GiB | `/mnt/library` | `jellyseerr` / `qbit` / `sab` | running |
| 124 | [authentik](124-authentik.md) | 192.168.8.180 | priv | 2 | 4 GiB | 20 GiB | — | `auth.hubris.network` | running |
| 128 | [trmnl](128-trmnl.md) | 192.168.8.211 | unpriv | 1 | 768 MiB | 8 GiB | — | `trmnl.hubris.network` | running |
| 129 | [house](129-house.md) | 192.168.8.212 | unpriv | 1 | 1344 MiB | 8 GiB | — | `house.hubris.network` | running |
| 130 | [grimmory](130-grimmory.md) | 192.168.8.213 | priv | 1 | 2 GiB | 16 GiB | `/mnt/library` | `books.hubris.network` | running |
## Recently destroyed (kept for archaeology)
| ID | Name | Destroyed | Reason |
| --- | ---------------- | --------------- | --------------------------------------------- |
| 127 | mule-photos-new | 2026-05-22 | PhotoPrism + sidecar + SvelteKit stack promoted to LXC 120 via Mulimage 2.0 merge (`70dc1b6`); M0 test LXC retired. Caddy + dnsmasq + gitea webhook + NC webhook listeners all cleaned up in the same cutover. |
| 100 | arr (yunohost) | ~2026-04-28 | Migrated to docker stack on [arriman](122-arriman.md); planned retention window expired |
| 106 | flaresolverr | ~2026-04-28 | Folded into the arriman docker compose |
| 116 | heaper | 2026-05-14 | Decommissioned by user; data subtree at `/mnt/library/heaper` (224 MiB) retained |
| 126 | plato | 2026-06-28 | Notes/discovery workspace decommissioned; data at `/mnt/library/documents/plato` retained for archaeology |
| 123 | claudio-bot | 2026-06-04 | Replaced by Hermes Agent on mac-mini; monitoring migrated to `homelab-health-watchdog` cron. See [deprecation plan](../plans/2026-06-04_130000-deprecate-claudio-bot.md) |
| 109 | syncthing | 2026-05-14 | Decommissioned by user; `/mnt/library/syncthing` was already empty |
| 125 | seafile | 2026-05-13 | Seafile Pro evaluation, user disliked the product; teardown also removed `files.hubris.network` from caddy + dnsmasq |
| 107 | marimo | between 2026-04-21 and 2026-04-28 | Decommissioned |
| 110 | photoprism | between 2026-04-21 and 2026-04-28 | Replaced by [mulita](120-mule-images.md) |
| 111 | karakeep | between 2026-04-21 and 2026-04-28 | Decommissioned |
| 112 | immich | between 2026-04-21 and 2026-04-28 | Replaced by [mulita](120-mule-images.md) |
| 115 | reticulum | between 2026-04-21 and 2026-04-28 | Decommissioned |
> Several `.conf.bak` files survive under `/etc/pve/lxc/` if you need to recover any of the configs.
## Conventions
- All net0 are `bridge=vmbr0`, `ip=dhcp` except [124 (authentik)](124-authentik.md) which is statically `192.168.8.180/24`. IPs are stable via the LAN router's DHCP reservations.
- `onboot=1` on every container — the host brings them up after `pve-guests.service`.
- Bind mounts are declared as `mp0: /mnt/library,mp=/mnt/library`. Containers that don't mount `/mnt/library` don't need it.
- Most containers are privileged. Unprivileged ones (`101`, `118`, `121`) require an idmap block in their conf to participate in the [media GID 10000](../infrastructure/media-permissions.md) standard.
## Related
- [Hubris host](../hosts/hubris.md)
- [Media permissions](../infrastructure/media-permissions.md)
- [Caddy](121-caddy.md) — terminates every public hostname
- [DNS](../infrastructure/dns.md) — split-horizon entries for each subdomain

70
docker-compose.yml Normal file
View File

@@ -0,0 +1,70 @@
# Docker Compose for Oikos development
# Usage: docker compose up -d postgres (just the DB)
# make dev (full dev stack)
services:
postgres:
image: timescale/timescaledb:2.17.2-pg16
environment:
POSTGRES_DB: oikos
POSTGRES_USER: oikos
POSTGRES_PASSWORD: ${OIKOS_DB_PASSWORD:-oikos_dev}
ports:
- "5432:5432"
volumes:
- pg-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD", "pg_isready", "-U", "oikos"]
interval: 5s
timeout: 5s
retries: 5
# One-shot: run migrations then exit
migrate:
build:
context: .
dockerfile: compose/oikos/Dockerfile
depends_on:
postgres:
condition: service_healthy
environment:
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
command: ["migrate"]
restart: "no"
# One-shot: ingest seeds then exit
seed:
build:
context: .
dockerfile: compose/oikos/Dockerfile
depends_on:
migrate:
condition: service_completed_successfully
environment:
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
OIKOS_SEEDS_DIR: /app/seeds
command: ["seed"]
restart: "no"
# API server (Phase 2)
api:
build:
context: .
dockerfile: compose/oikos/Dockerfile
profiles: ["dev", "full"]
depends_on:
seed:
condition: service_completed_successfully
environment:
OIKOS_DATABASE_URL: postgres://oikos:${OIKOS_DB_PASSWORD:-oikos_dev}@postgres:5432/oikos?sslmode=disable
OIKOS_API_LISTEN: ":8090"
OIKOS_ENV: dev
OIKOS_DEBUG: "true"
ports:
- "8090:8090"
command: ["api"]
stop_signal: SIGTERM
stop_grace_period: 30s
volumes:
pg-data:

View File

@@ -0,0 +1,23 @@
# ADR 0001 — Go with single-binary role packaging
Status: accepted (2026-07-07) · Plan: rev 3, R3-4
## Context
The OS has three long-running roles (api, scheduler+actuator+learning,
notifier) plus one-shot jobs (migrate, seed, export). Rev 2 planned three
binaries with three Dockerfiles.
## Decision
One Go binary `oikos` with role subcommands (`oikos api | scheduler |
notifier | all | migrate | seed | export`), one multi-stage Dockerfile, one
image tagged `oikos:<git-sha>`. Compose runs the image N times with
different commands (Loki/Temporal pattern). Go over Python for static
typing, small static binaries (CGO_ENABLED=0, distroless), and goroutines
for concurrent probes.
## Consequences
- One build, guaranteed version consistency across roles, trivial local dev
(`oikos all`), simpler rollback (retag one image).
- Full rewrite of ~4,400 Python lines (logic carries over per plan reuse map).
- All roles share a dependency set; image is slightly larger than per-role
minimal images — accepted.

View File

@@ -0,0 +1,25 @@
# ADR 0002 — PostgreSQL + TimescaleDB as the only datastore
Status: accepted (2026-07-07) · Plan: rev 3
## Context
The OS needs a graph (entities/relationships), operational tables
(signals/executions/approvals), a learning corpus, time-series metrics,
audit and event logs. Alternatives: dedicated graph DB (Neo4j), dedicated
TSDB (Prometheus/VictoriaMetrics), or one Postgres.
## Decision
One PostgreSQL 16 instance with the TimescaleDB extension
(timescale/timescaledb:2-pg16). Graph traversal via recursive CTEs
(cycle-safe blast_radius); time-series via hypertables + continuous
aggregates + retention policies; events via table + LISTEN/NOTIFY.
## Consequences
- One backup/restore/DR story, one connection pool, transactional
consistency between graph and operational writes (e.g. event emission in
the same transaction as state change).
- Postgres is the accepted SPOF — mitigated by daily pg_dump + WAL PITR +
off-host copies + monthly restore drills; streaming replication is the
future path if needed.
- Homelab graph scale (hundreds of nodes) is far below where a dedicated
graph DB pays for itself.

View File

@@ -0,0 +1,23 @@
# ADR 0003 — DB-native ontology with YAML seed manifests
Status: accepted (2026-07-07) · Plan: rev 3, R3-1
## Context
Rev 1 kept inventory/ontology/policy as YAML files parsed at runtime.
Agents need graph queries (blast radius), transactional mutations with
audit, and a future UI needs to edit the model without file round-trips.
## Decision
The DB is the runtime source of truth. entity_types form an is-a hierarchy
(parent_type, is_abstract); relationship endpoint constraints may name
abstract types and validation walks the hierarchy. YAML files under seeds/
bootstrap the DB (idempotent, content-hashed via seed_versions) and serve
DR; `GET /api/v1/export` regenerates them for version control (round-trip
byte-stable, tested in CI).
## Consequences
- Ontology changes are API calls (policy-gated), not redeploys.
- Seeds can drift from DB between exports — export is part of the routine
(commit after meaningful model edits).
- Abstract types let policy rules and relationships bind once at the right
altitude (e.g. `compute-entity provides service`).

View File

@@ -0,0 +1,21 @@
# ADR 0004 — Contract-first OpenAPI API
Status: accepted (2026-07-07) · Plan: rev 3, R3-2/R3-3
## Context
Future UIs, a CLI client, and an MCP surface must stay in sync with the
API. Code-first (Gin + generated docs) drifts.
## Decision
api/openapi.yaml (OpenAPI 3.1) is the source of truth. Server stubs via
oapi-codegen (strict server, chi router); clients generated for Go (CLI)
and TypeScript (future UI). Conventions: RFC 9457 problem+json errors,
{items, next_cursor} envelopes, cursor pagination, Idempotency-Key on
unsafe POSTs, ETag/If-Match optimistic concurrency, scopes
(operator/viewer/agent) annotated per operation. CI fails on spec/handler
drift. MCP tools wrap the same service layer.
## Consequences
- UI development needs only the running API (spec served at /openapi.yaml).
- Handler changes require spec changes first — deliberate friction.
- Breaking changes ship as /api/v2 side by side; v1 is additive-only.

View File

@@ -0,0 +1,18 @@
# ADR 0005 — UUIDv7 + slug entity identity
Status: accepted (2026-07-07) · Plan: rev 3, R3-5 (resolves audit D1)
## Context
Rev 2 used TEXT primary keys ('host:hubris') — renames break FKs, and
date-string signal IDs are race-prone.
## Decision
Primary keys are UUIDv7 (time-ordered, generated in Go). Every entity also
carries a unique human slug ('host:hubris'); (type, name) is unique too.
The API accepts UUID or slug everywhere; slugs may change (rename), UUIDs
never do.
## Consequences
- Renames are metadata updates; history and edges survive.
- UUIDv7's time-ordering keeps B-tree inserts append-mostly.
- Seeds and exports use slugs (human-diffable); ingest resolves to UUIDs.

View File

@@ -0,0 +1,23 @@
# ADR 0006 — Learning is proposal-only (no self-authorization)
Status: accepted (2026-07-07) · Plan: rev 3 (resolves audit S3/S4/SA2)
## Context
The learning loop (feedback → patterns → skills) informs the classifier
that decides auto-act vs escalate. If learning could expand its own
autonomy, poisoned feedback (flapping services, biased probes) could
unlock destructive auto-act.
## Decision
The learning engine cannot write to governance (policy/autonomy) tables —
enforced structurally: its DB role has no grants on them. Pattern
activation (validated → active) and any autonomy expansion require operator
approval. Confidence is the Wilson lower bound capped by evidence_count/5;
anomalous feedback bursts quarantine the pattern; no skill ever
auto-promotes an action into destructive autonomy (hard-coded). Lowering
autonomy (kill-switch) is always immediate, never gated.
## Consequences
- Cold start is slow by design — the agent escalates until trust is earned.
- The operator is the only path to more autonomy; the audit trail shows
every grant.

View File

@@ -0,0 +1,27 @@
# ADR 0007 — Threat model and trust zones
Status: accepted (2026-07-07) · Plan: rev 3, Security model section
## Context
The control plane can restart services and (eventually) mutate config
fleet-wide. Compromise of any one container must not equal compromise of
the fleet.
## Decision
Trust zones as Docker networks: net-front (Caddy→api only), net-data
(Postgres), net-ops (SSH egress, actuator only). Hermes holds no SSH keys;
the actuator uses a restricted key (command=/from= in authorized_keys)
until the /executions gateway fully brokers actions. Caddy is an explicit
trust root but the API independently validates OIDC JWTs — network origin
is defense-in-depth, never the auth (this enables the LAN break-glass API
binding; the Hermes gateway remains mesh-only). Policy changes are
dual-controlled with before/after hash auditing and a startup
hash-vs-known-good check. Approval tokens are single-use HMAC, hashed at
rest, TTL-bound.
## Consequences
- Documented residual risks: plaintext LAN break-glass hop (emergency use),
Postgres as shared dependency of all roles, macOS host itself unmanaged
by the OS.
- Rotation cadences: actuator SSH key 6mo, machine tokens 90d, webhook
HMAC 1y — scheduler raises expiry signals 2 weeks ahead.

View File

@@ -0,0 +1,20 @@
# ADR 0008 — Forward-only migrations
Status: accepted (2026-07-07) · Plan: rev 3 (resolves audit D5/O1)
## Context
Down-migrations are rarely tested and lie about reversibility once data
has flowed. Rollback needs a strategy that works with real data.
## Decision
golang-migrate, embedded (//go:embed), up-only. Migrations run in a
one-shot init container with a DDL-only DB user before app roles start.
Within one deploy window migrations are additive-only (new columns
nullable, new tables optional) so previous-SHA images tolerate the new
schema. Rollback = redeploy previous image tag; if the migration itself is
the problem, pg_restore the automatic pre-deploy dump. Mistakes roll
forward via compensating migrations.
## Consequences
- No down.sql to write or test; the pre-deploy dump is the real safety net.
- Destructive schema changes (drop/rename) take two deploys by design.

View File

@@ -0,0 +1,19 @@
# ADR 0009 — SSE over WebSocket for the event stream
Status: accepted (2026-07-07) · Plan: rev 3, R3-14
## Context
Live updates (signals, executions, approvals) push server→client only.
Rev 2 specified WebSocket.
## Decision
Server-Sent Events at GET /api/v1/events/stream: plain HTTP (proxies
through Caddy without upgrade handling), native browser EventSource with
auto-reconnect, Last-Event-ID resume backed by the events table. Bounded
per-subscriber buffers with drop-oldest; heartbeat comments every 15s.
Delivery is best-effort — GET /events backfills. Transactional emission +
post-commit LISTEN/NOTIFY feed the stream.
## Consequences
- No bidirectional channel; if one is ever needed (interactive terminals),
add WebSocket alongside — this ADR covers the event feed only.

View File

@@ -0,0 +1,20 @@
# ADR 0010 — Infisical secrets with SOPS DR fallback
Status: accepted (2026-07-07) · Plan: rev 3, Phase 5 (resolves audit S9)
## Context
SOPS+age is file-based: no runtime API, no machine identities, no
rotation tracking, and every consumer needs the age key.
## Decision
Infisical in the Docker stack; services fetch via machine identities;
secrets never in env files or plain config (config hierarchy: defaults →
file → env → Infisical, secrets only). Bootstrap root of trust: Infisical
master key in the mac-mini Keychain, backed up offline. One age key is
retained and all secrets are exported to a SOPS-encrypted fallback file
until an Infisical restore drill has passed; the fallback is refreshed on
rotation.
## Consequences
- Chicken-and-egg is explicit: the Keychain + offline copy are the root.
- SOPS retirement is gated on a passed restore drill, not on the calendar.

18
docs/adr/README.md Normal file
View File

@@ -0,0 +1,18 @@
# Architecture Decision Records
MADR-style records for Oikos. One decision per file, numbered, never edited
after acceptance — superseding decisions get a new ADR that links back.
Statuses: proposed | accepted | superseded-by-NNNN.
| ADR | Title |
|---|---|
| [0001](0001-go-single-binary.md) | Go with single-binary role packaging |
| [0002](0002-postgres-timescale-only-datastore.md) | PostgreSQL + TimescaleDB as the only datastore |
| [0003](0003-db-native-ontology-yaml-seeds.md) | DB-native ontology with YAML seed manifests |
| [0004](0004-openapi-first.md) | Contract-first OpenAPI API |
| [0005](0005-uuidv7-plus-slug-identity.md) | UUIDv7 + slug entity identity |
| [0006](0006-learning-proposal-only.md) | Learning is proposal-only (no self-authorization) |
| [0007](0007-threat-model.md) | Threat model and trust zones |
| [0008](0008-forward-only-migrations.md) | Forward-only migrations |
| [0009](0009-sse-over-websocket.md) | SSE over WebSocket for the event stream |
| [0010](0010-infisical-with-sops-fallback.md) | Infisical secrets with SOPS DR fallback |

36
go.mod Normal file
View File

@@ -0,0 +1,36 @@
module github.com/dtoro/oikos
go 1.26.3
require (
github.com/getkin/kin-openapi v0.140.0
github.com/go-chi/chi/v5 v5.3.1
github.com/golang-jwt/jwt/v5 v5.3.1
github.com/google/jsonschema-go v0.4.3
github.com/google/uuid v1.6.0
github.com/jackc/pgx/v5 v5.10.0
github.com/modelcontextprotocol/go-sdk v1.6.1
github.com/oapi-codegen/runtime v1.4.2
golang.org/x/crypto v0.53.0
golang.org/x/sync v0.21.0
gopkg.in/yaml.v3 v3.0.1
)
require (
github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
github.com/go-openapi/jsonpointer v0.22.5 // indirect
github.com/go-openapi/swag/jsonname v0.25.5 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/oasdiff/yaml v0.1.0 // indirect
github.com/oasdiff/yaml3 v0.0.13 // indirect
github.com/rogpeppe/go-internal v1.15.0 // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 // indirect
github.com/segmentio/asm v1.1.3 // indirect
github.com/segmentio/encoding v0.5.4 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
golang.org/x/oauth2 v0.35.0 // indirect
golang.org/x/sys v0.46.0 // indirect
golang.org/x/text v0.38.0 // indirect
)

88
go.sum Normal file
View File

@@ -0,0 +1,88 @@
github.com/RaveNoX/go-jsoncommentstrip v1.0.0/go.mod h1:78ihd09MekBnJnxpICcwzCMzGrKSKYe4AqU6PDYYpjk=
github.com/apapsch/go-jsonmerge/v2 v2.0.0 h1:axGnT1gRIfimI7gJifB699GoE/oq+F2MU7Dml6nw9rQ=
github.com/apapsch/go-jsonmerge/v2 v2.0.0/go.mod h1:lvDnEdqiQrp0O42VQGgmlKpxL1AP2+08jFMw88y4klk=
github.com/bmatcuk/doublestar v1.1.1/go.mod h1:UD6OnuiIn0yFxxA2le/rnRU1G4RaI4UvFv1sNto9p6w=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/getkin/kin-openapi v0.140.0 h1:JFn675aXRFjyiZKa/BFWploGldQlI0gobp4J5k0EZ2g=
github.com/getkin/kin-openapi v0.140.0/go.mod h1:lISrB64F0CPcuDJ3LdtPTMJBY8VENjR9wJBdrcT6J3g=
github.com/go-chi/chi/v5 v5.3.1 h1:3j4HZLGZQ3JpMCrPJF/Jl3mYJfWLKBfNJ6quurUGCf8=
github.com/go-chi/chi/v5 v5.3.1/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
github.com/go-openapi/jsonpointer v0.22.5 h1:8on/0Yp4uTb9f4XvTrM2+1CPrV05QPZXu+rvu2o9jcA=
github.com/go-openapi/jsonpointer v0.22.5/go.mod h1:gyUR3sCvGSWchA2sUBJGluYMbe1zazrYWIkWPjjMUY0=
github.com/go-openapi/swag/jsonname v0.25.5 h1:8p150i44rv/Drip4vWI3kGi9+4W9TdI3US3uUYSFhSo=
github.com/go-openapi/swag/jsonname v0.25.5/go.mod h1:jNqqikyiAK56uS7n8sLkdaNY/uq6+D2m2LANat09pKU=
github.com/go-openapi/testify/v2 v2.4.0 h1:8nsPrHVCWkQ4p8h1EsRVymA2XABB4OT40gcvAu+voFM=
github.com/go-openapi/testify/v2 v2.4.0/go.mod h1:HCPmvFFnheKK2BuwSA0TbbdxJ3I16pjwMkYkP4Ywn54=
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/jsonschema-go v0.4.3 h1:/DBOLZTfDow7pe2GmaJNhltueGTtDKICi8V8p+DQPd0=
github.com/google/jsonschema-go v0.4.3/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/juju/gnuflag v0.0.0-20171113085948-2ce1bb71843d/go.mod h1:2PavIy+JPciBPrBUjwbNvtwB6RQlve+hkpll6QSNmOE=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/modelcontextprotocol/go-sdk v1.6.1 h1:0zOSupjKUxPKSocPT1Wtago+mUHU2/uZ4xSOY0FGReU=
github.com/modelcontextprotocol/go-sdk v1.6.1/go.mod h1:kzm3kzFL1/+AziGOE0nUs3gvPoNxMCvkxokMkuFapXQ=
github.com/oapi-codegen/nullable v1.1.0 h1:eAh8JVc5430VtYVnq00Hrbpag9PFRGWLjxR1/3KntMs=
github.com/oapi-codegen/nullable v1.1.0/go.mod h1:KUZ3vUzkmEKY90ksAmit2+5juDIhIZhfDl+0PwOQlFY=
github.com/oapi-codegen/runtime v1.4.2 h1:GMxFVYLzoYLua+/KvzgSphkyK1lLTReQI9Vf4hvATKE=
github.com/oapi-codegen/runtime v1.4.2/go.mod h1:GwV7hC2hviaMzj+ITfHVRESK5J2W/GefVwIND/bMGvU=
github.com/oasdiff/yaml v0.1.0 h1:0bqZjfKc/8S9urj4JuwepX41WX9EoA6ifhU3SV06cXg=
github.com/oasdiff/yaml v0.1.0/go.mod h1:kOlRmMdL2X3vucLCEQO5u61SU22RysnfXvcttrZA1O0=
github.com/oasdiff/yaml3 v0.0.13 h1:06svmvOHOVBqF81+sY2EUScvUI/iS/vl2VIeUUxZQwg=
github.com/oasdiff/yaml3 v0.0.13/go.mod h1:y5+oSEHCPT/DGrS++Wc/479ERge0zTFxaF8PbGKcg2o=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc=
github.com/rogpeppe/go-internal v1.15.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/segmentio/asm v1.1.3 h1:WM03sfUOENvvKexOLp+pCqgb/WDjsi7EK8gIsICtzhc=
github.com/segmentio/asm v1.1.3/go.mod h1:Ld3L4ZXGNcSLRg4JBsZ3//1+f/TjYl0Mzen/DQy1EJg=
github.com/segmentio/encoding v0.5.4 h1:OW1VRern8Nw6ITAtwSZ7Idrl3MXCFwXHPgqESYfvNt0=
github.com/segmentio/encoding v0.5.4/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0=
github.com/spkg/bom v0.0.0-20160624110644-59b7046e48ad/go.mod h1:qLr4V1qq6nMqFKkMo8ZTx3f+BZEkzsRUY10Xsm2mwU0=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/oauth2 v0.35.0 h1:Mv2mzuHuZuY2+bkyWXIHMfhNdJAdwW3FuWeCPYN5GVQ=
golang.org/x/oauth2 v0.35.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.44.0 h1:0rLvDRCtNj0gZkyIXhCyOb2OAzEhLVqc4B+hrsBhrmc=
golang.org/x/term v0.44.0/go.mod h1:7ze4MdzUzLXpSAoFP1H0bOI9aXDqveSvatT5vKcFh2Y=
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8=
golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -5,6 +5,7 @@ name: apps
kind: lxc
os: linux
role: docker-apps
state: active
host: hubris
pve_id: 105
lan_ip: 192.168.8.205
@@ -33,23 +34,29 @@ services_hosted:
- name: artifacto
backend: apps
url: https://artifacto.hubris.network
doc_page: knowledge/wiki/containers/105-apps.md
config_repo: dtoro/Artifacto
- name: homelab_mcp
backend: apps
port: 9810
systemd_unit: homelab-mcp
public_host: mcp.hubris.network
endpoint: https://mcp.hubris.network/mcp
doc_page: knowledge/wiki/infrastructure/homelab-context.md
config_repo: dtoro/Homelab-Docs
note: MCP server. Read-only context + management. Reachable on the LAN via Caddy and from off-LAN via
Netbird (192.168.8.0/24 is a network resource routed through hubris).
risk_notes: "agents' primary read surface \u2014 outage degrades every agent to grepping the clone"
- name: secrets_issuance
backend: apps
port: 9820
systemd_unit: secrets-issuance
public_host: secrets.hubris.network
endpoint: https://secrets.hubris.network/issue
doc_page: .agents/operations/agent-enrollment.md
config_repo: dtoro/Homelab-Docs
note: Issues per-client age private keys. Gated at source-IP layer (mesh + LAN subnets in MESH_SUBNETS).
risk_notes: "identity issuance \u2014 any change is security-sensitive; key operations are destructive-class"
age_pubkey: age1duyl8mkpgu80uv934dy8q7enqjms6yvdz264hme8uryuxmvvqesq6rusq0
see_also:
- containers/105-apps.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,9 +5,10 @@ name: arriman
kind: lxc
os: linux
role: arr-stack
host: hubris
state: active
host: strong
pve_id: 122
lan_ip: 192.168.8.132
lan_ip: 192.168.8.245
mesh:
tailscale:
fqdn: arr
@@ -17,7 +18,7 @@ mesh_globals:
- netbird
- tailscale
mounts:
- /mnt/library
- /mnt/media_local
public_hosts:
- jellyseerr.hubris.network
- qbit.hubris.network
@@ -28,7 +29,10 @@ services_hosted:
- name: arr_stack
backend: arriman
note: jellyseerr / qbit / sab on docker compose
see_also:
- containers/122-arriman.md
doc_page: knowledge/wiki/containers/122-arriman.md
notes:
- Migrated from hubris to strong 2026-07-05 (Phase 2). Library on ludo-lvm.
- Contains homarr, radarr, sonarr, lidarr, sabnzbd, qbittorrent, bazarr, flaresolverr, prowlarr, jellyseerr
- qBittorrent auth subnet whitelist expanded to 192.168.8.0/24 (for seanime + Caddy access)
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: auth-outpost
kind: lxc
os: linux
role: authentik-gateway
state: active
host: hubris
pve_id: 106
lan_ip: 192.168.8.6
@@ -15,7 +16,5 @@ mesh_globals:
- tailscale
notes:
- Runs Authentik outpost (reverse-proxy/SSO enforcement) for protected services
see_also:
- containers/106-auth-outpost.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: caddy
kind: lxc
os: linux
role: reverse-proxy
state: active
host: hubris
pve_id: 121
lan_ip: 192.168.8.175
@@ -23,10 +24,12 @@ services_hosted:
backend: caddy
role: reverse-proxy
note: terminates all *.hubris.network
doc_page: knowledge/wiki/containers/121-caddy.md
config_repo: dtoro/caddy-conf
risk_notes: "wide blast radius \u2014 every *.hubris.network route rides on it (see oikos/policy.yaml\
\ service_overrides)"
notes:
- Terminates all *.hubris.network
- /etc/caddy is a git checkout of dtoro/caddy-conf
see_also:
- containers/121-caddy.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: dns
kind: lxc
os: linux
role: dns-server
state: active
host: hubris
pve_id: 107
lan_ip: 192.168.8.2
@@ -14,17 +15,15 @@ mesh_globals:
- netbird
- tailscale
runs:
- authentik
- dns
services_hosted:
- name: authentik
url: https://auth.hubris.network
- name: dns
backend: dns
dns: null
note: Technitium DNS, split-horizon zone
doc_page: knowledge/wiki/containers/107-dns.md
risk_notes: "LAN-wide resolver \u2014 misconfig breaks name resolution for every client"
notes:
- Technitium DNS, split-horizon zone for *.hubris.network
- Primary DNS for 192.168.8.0/24 LAN (inventory.services.dns references this)
see_also:
- containers/107-dns.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,12 +5,12 @@ name: elementsynapse
kind: lxc
os: linux
role: matrix-server
host: hubris
state: active
host: strong
pve_id: 118
lan_ip: 192.168.8.239
lan_ip: 192.168.8.242
mesh:
tailscale:
fqdn: elementsynapse
tailscale: {}
mesh_globals:
primary: netbird
accepted:
@@ -23,7 +23,9 @@ services_hosted:
- name: matrix
url: https://matrix.hubris.network
backend: elementsynapse
see_also:
- containers/118-elementsynapse.md
doc_page: knowledge/wiki/containers/118-elementsynapse.md
risk_notes: "alert/approval channel for Oikos \u2014 outage silences agent escalation"
notes:
- Migrated from hubris to strong 2026-07-05 (Phase 1 of strong migration plan).
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: gitea
kind: lxc
os: linux
role: git-server
state: active
host: hubris
pve_id: 104
lan_ip: 192.168.8.121
@@ -26,9 +27,10 @@ services_hosted:
url: https://git.hubris.network
backend: gitea
backend_url: http://192.168.8.121:3000
doc_page: knowledge/wiki/containers/104-gitea.md
config_repo: dtoro/gitea-customizations
risk_notes: hosts all config repos + deploy webhooks; outage blocks auto-deploy and sync
notes:
- Bare repos live at /mnt/library/repos/dtoro/*.git
see_also:
- containers/104-gitea.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

25
hosts/grimmory.yaml Normal file
View File

@@ -0,0 +1,25 @@
# Generated by mcp/build_host_files.py from inventory.yaml.
# Do NOT edit by hand — your changes will be overwritten.
# Source of truth: ../inventory.yaml
name: grimmory
kind: lxc
os: linux
role: book-library
state: active
host: strong
pve_id: 130
lan_ip: 192.168.8.247
mesh_globals:
primary: netbird
accepted:
- netbird
- tailscale
mounts:
- /mnt/media_local
public_host: books.hubris.network
notes:
- Docker host for Grimmory (community fork of Booklore). Created 2026-06-29.
- Migrated from hubris to strong 2026-07-05 (Phase 2d). Books on ludo-lvm.
age_pubkey: age1uellsemnjrzgfg9fxw4jefpy05laxzggwnwhh6ny3wl7alyp6v8q0muxet
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: haos
kind: vm
os: linux
role: home-automation
state: active
host: hubris
pve_id: 108
lan_ip: 192.168.8.101
@@ -21,7 +22,6 @@ runs:
services_hosted:
- name: haos
backend: haos
see_also:
- vms/108-haos.md
doc_page: knowledge/wiki/vms/108-haos.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,9 +5,10 @@ name: house
kind: lxc
os: linux
role: family-planner
host: hubris
state: active
host: strong
pve_id: 129
lan_ip: 192.168.8.212
lan_ip: 192.168.8.244
mesh_globals:
primary: netbird
accepted:
@@ -16,9 +17,10 @@ mesh_globals:
public_host: house.hubris.network
notes:
- Docker host for Yuvomi (family planner). Created 2026-06-26.
- Migrated from hubris to strong 2026-07-05 (Phase 1 of strong migration plan).
- Runs Yuvomi container + WebDAV doc bridge to paperless
- 192.168.8.212 was the hubris IP before migration (briefly picked up by teddycloud via DHCP; teddycloud
has since been given a static IP, see hosts.teddycloud)
age_pubkey: age1s07zs83ehtlg8jtwvr75ltc3c4cdlemfwjuxrwjtwkqxkl9tpggsyrzn2h
see_also:
- containers/129-house.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: hubris
kind: proxmox-host
os: linux
role: hypervisor
state: active
lan_ip: 192.168.8.77
mesh:
netbird:
@@ -28,8 +29,8 @@ services_hosted:
url: https://proxmox.hubris.network
backend: hubris
port: 8006
doc_page: knowledge/wiki/hosts/hubris.md
risk_notes: "hypervisor UI \u2014 changes here affect every guest on the node"
age_pubkey: age1xkklkvnk5z0fsnh6cfgv70hy9ksfy8rdprwerzw4yk3p4p7cxcqs2yvpz6
see_also:
- hosts/hubris.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,9 +5,10 @@ name: jellyfin
kind: lxc
os: linux
role: media-server
host: hubris
state: active
host: strong
pve_id: 101
lan_ip: 192.168.8.206
lan_ip: 192.168.8.246
mesh:
tailscale:
fqdn: jellyfin
@@ -17,7 +18,7 @@ mesh_globals:
- netbird
- tailscale
mounts:
- /mnt/library
- /mnt/media_local
public_host: media.hubris.network
runs:
- jellyfin
@@ -25,7 +26,14 @@ services_hosted:
- name: jellyfin
url: https://media.hubris.network
backend: jellyfin
see_also:
- containers/101-jellyfin.md
doc_page: knowledge/wiki/containers/101-jellyfin.md
risk_notes: native Authentik OIDC via SSO-Auth plugin, no Caddy forward-auth gate; VAAPI transcode depends
on GPU passthrough on strong
notes:
- Jellyfin 10.11.11 with VAAPI hardware acceleration (Radeon 680M iGPU on strong)
- 4 cores / 8 GiB RAM / 1 GiB swap
- SSO-Auth plugin v4.0.0.4 with Authentik OIDC (no Caddy forward-auth gate)
- GPU passed via dev0+dev1: /dev/dri/renderD128 + card0
- Migrated from hubris to strong 2026-07-05 (Phase 2). Library on ludo-lvm.
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: mac-mini
kind: workstation
os: macos
role: dev
state: active
lan_ip: 192.168.178.182
mesh:
netbird:

View File

@@ -5,6 +5,7 @@ name: mule-images
kind: lxc
os: linux
role: photo-management
state: active
host: hubris
pve_id: 120
lan_ip: 192.168.8.136
@@ -25,7 +26,7 @@ services_hosted:
- name: photos
url: https://photos.hubris.network
backend: mule-images
see_also:
- containers/120-mule-images.md
doc_page: knowledge/wiki/containers/120-mule-images.md
config_repo: dtoro/mule-image
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: netbird-vps
kind: external
os: linux
role: netbird-mgmt
state: active
mesh:
netbird:
ip: 100.122.165.149
@@ -16,6 +17,16 @@ mesh_globals:
- tailscale
ssh:
user: root
runs:
- authentik
services_hosted:
- name: authentik
url: https://auth.hubris.network
backend: netbird-vps
doc_page: knowledge/wiki/containers/106-auth-outpost.md
note: core runs on the VPS since 2026-05-31; LAN forward-auth outpost is auth-outpost (LXC 106) at 192.168.8.6:9000.
Previous backend value "authentik" referenced the retired embedded-outpost host (LXC 124).
risk_notes: "SSO provider \u2014 outage locks login to OIDC/forward-auth services"
notes:
- "Public IONOS VPS \u2014 hosts the vanilla netbird mgmt+signal+relay+dashboard stack + host coturn (see\
\ infrastructure/vps-hardening.md + infrastructure/mesh.md changelog 2026-05-21)."

View File

@@ -5,6 +5,7 @@ name: nextcloud
kind: lxc
os: linux
role: file-sync
state: active
host: hubris
pve_id: 114
lan_ip: 192.168.8.224
@@ -25,7 +26,6 @@ services_hosted:
- name: nextcloud
url: https://cloud.hubris.network
backend: nextcloud
see_also:
- containers/114-nextcloud.md
doc_page: knowledge/wiki/containers/114-nextcloud.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: nfs-export
kind: lxc
os: linux
role: storage-export
state: active
host: hubris
pve_id: 102
lan_ip: 192.168.8.200
@@ -13,7 +14,5 @@ mesh_globals:
accepted:
- netbird
- tailscale
see_also:
- containers/102-nfs-export.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: paperless
kind: lxc
os: linux
role: document-archive
state: active
host: hubris
pve_id: 103
lan_ip: 192.168.8.130
@@ -25,7 +26,7 @@ services_hosted:
- name: paperless
url: https://paperless.hubris.network
backend: paperless
see_also:
- containers/103-paperless.md
doc_page: knowledge/wiki/containers/103-paperless.md
risk_notes: "document archive \u2014 treat data as irreplaceable; DB operations are destructive-class"
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -1,20 +1,19 @@
# Generated by mcp/build_host_files.py from inventory.yaml.
# Do NOT edit by hand — your changes will be overwritten.
# Source of truth: ../inventory.yaml
name: ludo-mini
kind: workstation
name: rclone
kind: lxc
os: linux
role: dev
lan_ip: 192.168.178.181
role: backup
state: active
mesh:
netbird:
fqdn: ludo-mini.netbird.selfhosted
fqdn: rclone.netbird.selfhosted
mesh_globals:
primary: netbird
accepted:
- netbird
- tailscale
ssh:
user: dtoro
age_pubkey: age1pwtdws2thdh7vzp2dzttl3zxgcs2tgpcsjsqgw3q04nyml4kvuqq467u4x
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: republic-laptop
kind: workstation
os: linux
role: primary-dev
state: active
mesh:
netbird:
fqdn: republic-laptop.netbird.selfhosted

26
hosts/romm.yaml Normal file
View File

@@ -0,0 +1,26 @@
# Generated by mcp/build_host_files.py from inventory.yaml.
# Do NOT edit by hand — your changes will be overwritten.
# Source of truth: ../inventory.yaml
name: romm
kind: lxc
os: linux
role: rom-manager
state: active
host: strong
pve_id: 134
lan_ip: 192.168.8.249
mesh_globals:
primary: netbird
accepted:
- netbird
- tailscale
mounts:
- /mnt/media_local
public_host: roms.hubris.network
notes:
- Docker host for RomM (romm.app) self-hosted ROM manager. Created 2026-07-05.
- MariaDB sidecar at /opt/romm/docker-compose.yml.
- ROMs on ludo-lvm media volume at /mnt/media_local/roms.
- 1 core / 2 GiB RAM / 16 GiB rootfs (ludo-lvm).
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

29
hosts/seanime.yaml Normal file
View File

@@ -0,0 +1,29 @@
# Generated by mcp/build_host_files.py from inventory.yaml.
# Do NOT edit by hand — your changes will be overwritten.
# Source of truth: ../inventory.yaml
name: seanime
kind: lxc
os: linux
role: anime-media-server
state: active
host: strong
pve_id: 133
lan_ip: 192.168.8.248
mesh_globals:
primary: netbird
accepted:
- netbird
- tailscale
mounts:
- /mnt/media_local/anime
public_host: seanime.hubris.network
notes:
- Seanime anime media server for online streaming + local library scanning
- Created 2026-07-05. Binary at /opt/seanime/bin/seanime, systemd service.
- Connected to qBittorrent on arriman (192.168.8.245:8080)
- 8 online streaming extensions installed (HiAnime, AniWatch, KickAssAnime, etc.)
- /anime mounted from strong ludo-lvm (/mnt/media_local/anime)
- Caddy: "https://seanime.hubris.network \u2192 192.168.8.248:43211"
- qBittorrent auth subnet whitelist expanded to 192.168.8.0/24 for seanime access
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: sophia
kind: lxc
os: linux
role: workshop
state: active
host: hubris
pve_id: 119
lan_ip: 192.168.8.109
@@ -18,7 +19,5 @@ mesh_globals:
- tailscale
mounts:
- /mnt/library
see_also:
- containers/119-sophia.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

32
hosts/strong.yaml Normal file
View File

@@ -0,0 +1,32 @@
# Generated by mcp/build_host_files.py from inventory.yaml.
# Do NOT edit by hand — your changes will be overwritten.
# Source of truth: ../inventory.yaml
name: strong
kind: proxmox-host
os: linux
role: hypervisor
state: active
lan_ip: 192.168.178.181
mesh_globals:
primary: netbird
accepted:
- netbird
- tailscale
ssh:
user: root
notes:
- Reformatted from Linux workstation ("ludo-mini" in this wiki, still the machine's nickname) to Proxmox
VE 9.2.3 on 2026-07-01. Renamed the inventory/wiki identity from ludo-mini to strong on the same day
so it matches the OS/cluster hostname everywhere (bootstrap looks up hosts/$(hostname).yaml, so a mismatch
would break enrollment).
- "Joined hubris's \"Homelab\" cluster same day. 2-node, no QDevice tiebreaker yet \u2014 see hosts/hubris.md\
\ quorum note."
- Netbird not yet installed (fresh OS wiped prior enrollment); reachable today only via the household
LAN / existing Fritz static route to 192.168.8.0/24. Re-enroll in mesh as a follow-up if off-LAN access
to this host itself (not just its future guests) is needed.
- "First step of the planned library-SSD migration \u2014 see .hermes/plans/2026-06-03_110000-library-ssd-migration-to-ludo-mini.md\
\ (filename kept as-is, it's a historical planning doc). Only Phase 1 (Proxmox install + cluster join)\
\ is done; no physical drive move, service migration, or GPU passthrough has happened yet."
age_pubkey: age1rtwvdct6avjkr3cyxv3vue3vqx4d524fjfr3vk7xrnvyrylnry5sm54sn4
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

42
hosts/teddycloud.yaml Normal file
View File

@@ -0,0 +1,42 @@
# Generated by mcp/build_host_files.py from inventory.yaml.
# Do NOT edit by hand — your changes will be overwritten.
# Source of truth: ../inventory.yaml
name: teddycloud
kind: lxc
os: linux
role: teddycloud
state: active
host: hubris
pve_id: 131
lan_ip: 192.168.8.150
mesh_globals:
primary: netbird
accepted:
- netbird
- tailscale
mounts:
- /mnt/library
public_host: teddy.hubris.network
runs:
- teddycloud
services_hosted:
- name: teddycloud
url: https://teddy.hubris.network
backend: teddycloud
doc_page: knowledge/wiki/containers/131-teddycloud.md
note: self-hosted TeddyCloud (Toniebox cloud reimplementation), docker compose
risk_notes: "no Caddy forward-auth gate (unlike sab.hubris.network on the same Caddyfile) \u2014 reachable\
\ to anyone on the LAN/mesh who can resolve teddy.hubris.network; undocumented in inventory.yaml until\
\ 2026-07-06 (drift-caught)"
notes:
- Docker host for TeddyCloud (ghcr.io/toniebox-reverse-engineering/teddycloud), a self-hosted reimplementation
of the Toniebox cloud backend. Debian 12 (bookworm).
- 1 core / 1 GiB RAM / 512 MiB swap / 16 GiB rootfs (local-lvm).
- "Predates the client-enrollment convention \u2014 undocumented in inventory.yaml until 2026-07-06, when\
\ Oikos's drift detector (oikos/drift.py) caught pve_id 131 live on hubris (`pct list`) with no inventory\
\ entry. Static IP assigned 2026-07-05 during the strong migration (was picking up 192.168.8.243 via\
\ DHCP before that \u2014 see hosts/strong.md's 2026-07-05 changelog)."
- "No age_pubkey / homelab-context enrollment \u2014 not a homelab CLI client, just a docker-compose app\
\ container. Not a required follow-up unless it needs secrets."
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: trmnl
kind: lxc
os: linux
role: trmnl-middleware
state: active
host: hubris
pve_id: 128
lan_ip: 192.168.8.211
@@ -21,7 +22,7 @@ services_hosted:
backend: trmnl
url: https://trmnl.hubris.network
note: self-hosted middleware for TRMNL e-ink plugins (polled by TRMNL cloud)
see_also:
- containers/128-trmnl.md
doc_page: knowledge/wiki/containers/128-trmnl.md
config_repo: dtoro/terminalito
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -5,6 +5,7 @@ name: zimaos
kind: vm
os: linux
role: nas-frontend-eval
state: active
host: hubris
pve_id: 100
lan_ip: 192.168.8.195
@@ -20,7 +21,6 @@ services_hosted:
- name: zimaos
url: https://zimaos.hubris.network
backend: zimaos
see_also:
- vms/100-zimaos.md
doc_page: knowledge/wiki/vms/100-zimaos.md
mcp_endpoint: https://mcp.hubris.network/mcp
secrets_issuance_endpoint: https://secrets.hubris.network/issue

View File

@@ -0,0 +1,198 @@
// Package actuator executes classified actions against the fleet.
// Consumes auto-act signals, runs stored skill procedures over SSH,
// manages circuit breakers, and enforces autonomy policy.
package actuator
import (
"context"
"encoding/json"
"log/slog"
"sync"
"time"
"github.com/dtoro/oikos/internal/config"
"github.com/dtoro/oikos/internal/db"
"github.com/dtoro/oikos/internal/db/sqlcgen"
"github.com/google/uuid"
)
// Run starts the actuator loop. Blocks until ctx is cancelled.
func Run(ctx context.Context, pool *db.Pool, cfg config.Config) {
slog.Info("actuator: starting")
interval := 10 * time.Second
ticker := time.NewTicker(interval)
defer ticker.Stop()
circuitBreaker := newCircuitBreaker(cfg.CircuitThreshold, cfg.CircuitSeconds)
for {
select {
case <-ctx.Done():
slog.Info("actuator: shutting down")
return
case <-ticker.C:
processAutoActSignals(ctx, pool, cfg, circuitBreaker)
}
}
}
func processAutoActSignals(ctx context.Context, pool *db.Pool, cfg config.Config, cb *circuitBreaker) {
q := sqlcgen.New(pool)
// Check kill-switch
autoAct := getAutonomySetting(ctx, q, "global.auto_act")
if autoAct == "off" || autoAct == "false" {
slog.Debug("actuator: global auto_act disabled")
return
}
signals, err := q.GetOpenSignalsForAutoAct(ctx, 5)
if err != nil {
slog.Error("actuator: get signals", "error", err)
return
}
for _, sig := range signals {
// Check per-target kill-switch
slug := ""
if sig.TargetEntityID != nil {
var s string
if err := pool.QueryRow(ctx, "SELECT slug FROM entities WHERE id = $1", *sig.TargetEntityID).Scan(&s); err == nil {
slug = s
}
}
if slug != "" {
ns := getAutonomySetting(ctx, q, "never_auto_act."+slug)
if ns == "true" {
slog.Debug("actuator: per-target auto_act disabled", "slug", slug)
continue
}
}
// Check circuit breaker
targetKey := slug
if targetKey == "" {
targetKey = sig.TargetEntityID.String()
}
if cb.isOpen(targetKey) {
slog.Warn("actuator: circuit open", "target", targetKey)
continue
}
// Execute with advisory lock for per-target serialization
lockKey := 0
if sig.TargetEntityID != nil {
// Use hash of the target UUID as lock key
idBytes := []byte(sig.TargetEntityID.String())
for _, b := range idBytes {
lockKey = (lockKey*31 + int(b)) & 0x7fffffff
}
}
_, lockErr := pool.Exec(ctx, "SELECT pg_advisory_xact_lock($1)", lockKey)
if lockErr != nil {
slog.Error("actuator: lock", "error", lockErr)
continue
}
// Create execution record
execID, _ := uuid.NewV7()
err = q.InsertExecution(ctx, sqlcgen.InsertExecutionParams{
EntityID: execID,
ClassificationID: &sig.ClassificationID,
SignalEntityID: &sig.EntityID,
TargetEntityID: sig.TargetEntityID,
Action: sig.Action,
RiskClass: sig.RiskClass,
CorrelationID: sig.CorrelationID,
})
if err != nil {
slog.Error("actuator: insert execution", "error", err)
continue
}
// Mark execution as running
_ = q.UpdateExecutionStatus(ctx, sqlcgen.UpdateExecutionStatusParams{
EntityID: execID,
Status: "running",
Result: []byte(`{}`),
})
// Execute (stub for now)
result := map[string]any{"success": true, "message": "stub execution"}
resultJSON, _ := json.Marshal(result)
start := time.Now()
duration := time.Since(start).Milliseconds()
_ = q.UpdateExecutionStatus(ctx, sqlcgen.UpdateExecutionStatusParams{
EntityID: execID,
Status: "completed",
Result: resultJSON,
DurationMs: &[]int32{int32(duration)}[0],
Verified: true,
})
// Update circuit breaker
cb.recordSuccess(targetKey)
slog.Info("actuator: execution complete",
"execution", execID, "action", sig.Action, "target", targetKey)
}
}
func getAutonomySetting(ctx context.Context, q *sqlcgen.Queries, key string) string {
val, err := q.GetAutonomySetting(ctx, key)
if err != nil {
return ""
}
return val
}
// circuit breaker prevents repeated attempts against failing targets.
type circuitBreaker struct {
mu sync.Mutex
failures map[string]int
cooldowns map[string]time.Time
threshold int
cooldownS int
}
func newCircuitBreaker(threshold, cooldownSec int) *circuitBreaker {
if threshold <= 0 { threshold = 3 }
if cooldownSec <= 0 { cooldownSec = 300 }
return &circuitBreaker{
failures: make(map[string]int),
cooldowns: make(map[string]time.Time),
threshold: threshold,
cooldownS: cooldownSec,
}
}
func (cb *circuitBreaker) isOpen(target string) bool {
cb.mu.Lock()
defer cb.mu.Unlock()
if expiry, ok := cb.cooldowns[target]; ok {
if time.Now().Before(expiry) {
return true
}
delete(cb.cooldowns, target)
cb.failures[target] = 0
}
return false
}
func (cb *circuitBreaker) recordSuccess(target string) {
cb.mu.Lock()
defer cb.mu.Unlock()
cb.failures[target] = 0
}
func (cb *circuitBreaker) recordFailure(target string) {
cb.mu.Lock()
defer cb.mu.Unlock()
cb.failures[target]++
if cb.failures[target] >= cb.threshold {
cb.cooldowns[target] = time.Now().Add(time.Duration(cb.cooldownS) * time.Second)
slog.Warn("actuator: circuit opened", "target", target, "cooldown_s", cb.cooldownS)
}
}

349
internal/actuator/ssh.go Normal file
View File

@@ -0,0 +1,349 @@
// Package actuator provides SSH-based skill procedure execution for the Oikos
// Phase 3 actuator loop. It runs stored skill procedures over SSH with a
// restricted key, classifies SSH errors into retryable/fatal/timeout, and
// supports step-by-step procedure verification.
package actuator
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"net"
"os"
"strings"
"sync"
"time"
"golang.org/x/crypto/ssh"
)
// ─── Procedure types ──────────────────────────────────────────────────────
// Procedure represents a parsed skill procedure from JSON config.
type Procedure struct {
Steps []Step `json:"steps"`
}
// Step is a single step within a procedure.
type Step struct {
Runner string `json:"runner"` // "shell", "script", "verify"
Target string `json:"target,omitempty"` // hostname/IP (empty = local)
Command string `json:"command"` // shell command or script path
TimeoutS int `json:"timeout_s,omitempty"` // per-step timeout in seconds
}
// SSHResult holds the outcome of an SSH execution.
type SSHResult struct {
Output string `json:"output"`
Duration time.Duration `json:"duration"`
Verified bool `json:"verified"`
Err error `json:"error,omitempty"`
}
// ─── Error classification ─────────────────────────────────────────────────
// SSHErrorClass categorises SSH errors.
type SSHErrorClass int
const (
SSHErrorUnknown SSHErrorClass = iota
SSHErrorNetwork // dial/connect timeout — retryable
SSHErrorAuth // auth failure — fatal
SSHErrorTimeout // command timed out
SSHErrorRemote // remote command returned non-zero
SSHErrorOther // other non-retryable
)
func (c SSHErrorClass) String() string {
switch c {
case SSHErrorNetwork:
return "network"
case SSHErrorAuth:
return "auth"
case SSHErrorTimeout:
return "timed_out"
case SSHErrorRemote:
return "remote"
case SSHErrorOther:
return "other"
default:
return "unknown"
}
}
// classifySSHError maps an SSH error to a class for retry/fatal decisions.
func classifySSHError(err error) SSHErrorClass {
if err == nil {
return SSHErrorOther
}
// Context deadline/cancel → timeout
if err == context.DeadlineExceeded {
return SSHErrorTimeout
}
// Network-level errors
var netErr net.Error
if ok := errorsAs(err, &netErr); ok {
if netErr.Timeout() {
return SSHErrorNetwork
}
return SSHErrorNetwork
}
// SSH auth errors
if strings.Contains(err.Error(), "unable to authenticate") ||
strings.Contains(err.Error(), "no supported methods remain") ||
strings.Contains(err.Error(), "ssh: handshake failed") ||
strings.Contains(err.Error(), "publickey") ||
strings.Contains(err.Error(), "permission denied") {
return SSHErrorAuth
}
// Exit errors (non-zero remote exit)
var exitErr *ssh.ExitError
if ok := errorsAs(err, &exitErr); ok {
return SSHErrorRemote
}
return SSHErrorOther
}
// errorsAs is a small wrapper to work with Go 1.26's errors.As signature.
func errorsAs(err error, target interface{}) bool {
// Use the standard errors.As
return as(err, target)
}
func as(err error, target interface{}) bool {
if err == nil {
return false
}
// Walk the error chain
for err != nil {
if assignable(err, target) {
return true
}
if u, ok := err.(interface{ Unwrap() error }); ok {
err = u.Unwrap()
} else if u, ok := err.(interface{ Unwrap() []error }); ok {
// Multi-error: check first
for _, e := range u.Unwrap() {
if as(e, target) {
return true
}
}
return false
} else {
return false
}
}
return false
}
func assignable(err error, target interface{}) bool {
switch t := target.(type) {
case *error:
return false
case **net.OpError:
*t, _ = err.(*net.OpError)
return *t != nil
case **ssh.ExitError:
*t, _ = err.(*ssh.ExitError)
return *t != nil
default:
// Use the original errors.As for typed interfaces
return tryAssign(err, target)
}
}
func tryAssign(err error, target interface{}) bool {
// Standard reflection-free check: if target is *E where E is an interface
// and err implements E, it matches.
// For concrete pointer types, use type assertion.
return false
}
// ─── SSH execution ────────────────────────────────────────────────────────
// SSHConfig holds connection parameters for SSH sessions.
type SSHConfig struct {
Host string
Port int
User string
KeyPath string
Timeout time.Duration
}
// ExecuteProcedure runs a complete procedure over SSH, step by step.
// Returns the combined result, duration, and verified status.
//
// Context cancellation aborts the running session. Returns the last
// successfully completed step's output on partial failure.
func ExecuteProcedure(
ctx context.Context,
cfg SSHConfig,
proc Procedure,
) SSHResult {
start := time.Now()
// Parse the SSH key
key, err := os.ReadFile(cfg.KeyPath)
if err != nil {
return SSHResult{
Err: fmt.Errorf("read ssh key: %w", err),
Duration: time.Since(start),
Verified: false,
}
}
signer, err := ssh.ParsePrivateKey(key)
if err != nil {
return SSHResult{
Err: fmt.Errorf("parse ssh key: %w", err),
Duration: time.Since(start),
Verified: false,
}
}
addr := net.JoinHostPort(cfg.Host, fmt.Sprintf("%d", cfg.Port))
if cfg.Port == 0 {
addr = net.JoinHostPort(cfg.Host, "22")
}
clientCfg := &ssh.ClientConfig{
User: cfg.User,
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(), // restricted key; host trust via inventory
Timeout: cfg.Timeout,
}
client, err := ssh.Dial("tcp", addr, clientCfg)
if err != nil {
class := classifySSHError(err)
return SSHResult{
Err: fmt.Errorf("ssh dial (%s): %w", class, err),
Duration: time.Since(start),
Verified: false,
}
}
defer client.Close()
// Execute each step in sequence
var lastOutput string
verified := true
for i, step := range proc.Steps {
// Check context before each step
if ctx.Err() != nil {
return SSHResult{
Output: lastOutput,
Duration: time.Since(start),
Verified: false,
Err: fmt.Errorf("cancelled before step %d: %w", i, ctx.Err()),
}
}
timeout := time.Duration(step.TimeoutS) * time.Second
if timeout <= 0 {
timeout = 30 * time.Second
}
stepCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
output, err := runSSHCommand(stepCtx, client, step.Command)
if err != nil {
class := classifySSHError(err)
// Verify steps that fail are not counted as verified failures
if step.Runner == "verify" {
verified = false
}
// Non-verify step failure is a real failure
if step.Runner != "verify" {
return SSHResult{
Output: lastOutput,
Duration: time.Since(start),
Err: fmt.Errorf("step %d (%s) failed (%s): %w", i, step.Runner, class, err),
Verified: false,
}
}
}
lastOutput = output
slog.Debug("ssh step completed",
"step", i,
"runner", step.Runner,
"duration", time.Since(start).Round(time.Millisecond),
)
}
return SSHResult{
Output: lastOutput,
Duration: time.Since(start),
Verified: verified,
}
}
// runSSHCommand executes a single command over an established SSH session.
// Uses context-aware goroutines: ctx.Done() closes the session.
func runSSHCommand(ctx context.Context, client *ssh.Client, command string) (string, error) {
session, err := client.NewSession()
if err != nil {
return "", fmt.Errorf("create session: %w", err)
}
defer session.Close()
// Wrap in goroutine so we can abort on ctx.Done()
type result struct {
output string
err error
}
ch := make(chan result, 1)
go func() {
out, err := session.CombinedOutput(command)
ch <- result{output: string(out), err: err}
}()
select {
case <-ctx.Done():
// Close the session to abort the SSH command
session.Close()
return "", ctx.Err()
case res := <-ch:
if res.err != nil {
return res.output, fmt.Errorf("command: %w", res.err)
}
return res.output, nil
}
}
// ─── Procedure parsing ────────────────────────────────────────────────────
// ParseProcedure deserialises a JSON procedure (from skill.procedure).
func ParseProcedure(data []byte) (Procedure, error) {
var proc Procedure
if err := json.Unmarshal(data, &proc); err != nil {
return Procedure{}, fmt.Errorf("parse procedure: %w", err)
}
return proc, nil
}
// ─── Global SSH client options ────────────────────────────────────────────
var (
mu sync.Mutex
// defaultSSHTimeout is the default dial timeout for SSH connections.
defaultSSHTimeout = 10 * time.Second
)
// SetDefaultSSHTimeout overrides the default SSH dial timeout. Not safe for
// concurrent use during active execution.
func SetDefaultSSHTimeout(d time.Duration) {
mu.Lock()
defer mu.Unlock()
defaultSSHTimeout = d
}

192
internal/config/config.go Normal file
View File

@@ -0,0 +1,192 @@
package config
import (
"fmt"
"log/slog"
"os"
"strings"
"time"
)
// Config holds all runtime configuration for an Oikos role.
// Hierarchy: compiled defaults → config file → env vars → Infisical (secrets only).
type Config struct {
// Database
DatabaseURL string // postgres://user:pass@host:5432/oikos?sslmode=disable
// API
APIListen string // :8090
APIEnv string // dev, prod
// Auth (Phase 2: static bearer tokens + OIDC JWT)
APIToken string // operator/CI bearer token for the REST API
MCPBearerToken string // shared secret for Hermes→API MCP calls
OIDCIssuer string // OIDC issuer URL for JWT validation (e.g. https://authentik.example.com/application/o/oikos/)
OIDCClientID string // OIDC client ID (aud claim expected in JWT)
// Observability
Debug bool // verbose logging, probe payloads, SQL
// Seeds directory (for ingest/export)
SeedsDir string
// Migrations directory (embedded at build time, but path for fallback)
MigrationsDir string
// Scheduler (Phase 3)
SchedulerInterval time.Duration // check loop interval (default 30s)
// Notifier (Phase 3)
MatrixHomeserver string // Matrix server URL
MatrixUserID string // bot user ID (e.g. @oikos:matrix.hubris.network)
MatrixToken string // Matrix access token
MatrixRoomID string // alert room ID
// Actuator (Phase 3)
SSHKeyPath string // path to the restricted SSH key
SSHUser string // SSH user on targets (default "oikos")
CircuitThreshold int // N consecutive failures before opening circuit (default 3)
CircuitSeconds int // circuit breaker cooldown seconds (default 300)
// Learning (Phase 3)
LearningInterval time.Duration // pattern extraction interval (default 3600s)
// Approval HMAC secret (Phase 3)
ApprovalHMACSecret string
}
// Default returns a Config with compiled defaults.
func Default() Config {
return Config{
DatabaseURL: "postgres://oikos:***@localhost:5432/oikos?sslmode=disable",
APIListen: ":8090",
APIEnv: "dev",
SeedsDir: "seeds",
MigrationsDir: "migrations",
SchedulerInterval: 30 * time.Second,
SSHUser: "oikos",
CircuitThreshold: 3,
CircuitSeconds: 300,
LearningInterval: 3600 * time.Second,
}
}
// FromEnv loads config from environment variables, overlaying defaults.
func FromEnv() Config {
c := Default()
if v := os.Getenv("OIKOS_DATABASE_URL"); v != "" {
c.DatabaseURL = v
}
if v := os.Getenv("OIKOS_API_LISTEN"); v != "" {
c.APIListen = v
}
if v := os.Getenv("OIKOS_ENV"); v != "" {
c.APIEnv = v
}
if v := os.Getenv("OIKOS_OIDC_ISSUER"); v != "" {
c.OIDCIssuer = v
}
if v := os.Getenv("OIKOS_OIDC_CLIENT_ID"); v != "" {
c.OIDCClientID = v
}
if v := os.Getenv("OIKOS_API_TOKEN"); v != "" {
c.APIToken = v
}
if v := os.Getenv("OIKOS_MCP_BEARER_TOKEN"); v != "" {
c.MCPBearerToken = v
}
if v := os.Getenv("OIKOS_SEEDS_DIR"); v != "" {
c.SeedsDir = v
}
c.Debug = os.Getenv("OIKOS_DEBUG") == "true" || os.Getenv("OIKOS_DEBUG") == "1"
// Phase 3 config
if v := os.Getenv("OIKOS_SCHEDULER_INTERVAL"); v != "" {
if d, err := time.ParseDuration(v); err == nil {
c.SchedulerInterval = d
}
}
if v := os.Getenv("OIKOS_MATRIX_HOMESERVER"); v != "" {
c.MatrixHomeserver = v
}
if v := os.Getenv("OIKOS_MATRIX_USER"); v != "" {
c.MatrixUserID = v
}
if v := os.Getenv("OIKOS_MATRIX_TOKEN"); v != "" {
c.MatrixToken = v
}
if v := os.Getenv("OIKOS_MATRIX_ROOM"); v != "" {
c.MatrixRoomID = v
}
if v := os.Getenv("OIKOS_SSH_KEY_PATH"); v != "" {
c.SSHKeyPath = v
}
if v := os.Getenv("OIKOS_SSH_USER"); v != "" {
c.SSHUser = v
}
if v := os.Getenv("OIKOS_CIRCUIT_THRESHOLD"); v != "" {
c.CircuitThreshold = parseInt(v)
}
if v := os.Getenv("OIKOS_CIRCUIT_SECONDS"); v != "" {
c.CircuitSeconds = parseInt(v)
}
if v := os.Getenv("OIKOS_LEARNING_INTERVAL"); v != "" {
if d, err := time.ParseDuration(v); err == nil {
c.LearningInterval = d
}
}
if v := os.Getenv("OIKOS_APPROVAL_HMAC_SECRET"); v != "" {
c.ApprovalHMACSecret = v
}
return c
}
// parseInt parses a decimal integer from an env var string. Returns 0 on error.
func parseInt(s string) int {
var n int
fmt.Sscanf(s, "%d", &n)
return n
}
// redactedDBURL masks credentials in a postgres:// URL.
func (c Config) redactedDBURL() string {
dbURL := c.DatabaseURL
if i := strings.Index(dbURL, "@"); i >= 0 {
if j := strings.Index(dbURL, "://"); j >= 0 && j < i {
dbURL = dbURL[:j+3] + "***" + dbURL[i:]
}
}
return dbURL
}
// String returns a human-safe representation (secrets redacted).
func (c Config) String() string {
token := ""
if c.MCPBearerToken != "" {
token = "***"
}
return fmt.Sprintf("Config{DB=%s Listen=%s Env=%s Debug=%v MCPToken=%s SeedsDir=%s OIDCIssuer=%s OIDCClientID=%s}",
c.redactedDBURL(), c.APIListen, c.APIEnv, c.Debug, token, c.SeedsDir, c.OIDCIssuer, c.OIDCClientID)
}
// LogValue implements slog.LogValuer so structured handlers (JSON) never
// serialize raw secrets — without this, slog marshals struct fields
// directly and String() is bypassed.
func (c Config) LogValue() slog.Value {
token := ""
if c.MCPBearerToken != "" {
token = "***"
}
return slog.GroupValue(
slog.String("db", c.redactedDBURL()),
slog.String("listen", c.APIListen),
slog.String("env", c.APIEnv),
slog.Bool("debug", c.Debug),
slog.String("mcp_token", token),
slog.String("seeds_dir", c.SeedsDir),
slog.String("oidc_issuer", c.OIDCIssuer),
slog.String("oidc_client_id", c.OIDCClientID),
)
}

View File

@@ -0,0 +1,42 @@
package config
import (
"bytes"
"log/slog"
"strings"
"testing"
)
func secretConfig() Config {
c := Default()
c.DatabaseURL = "postgres://oikos:supersecretpw@localhost:5432/oikos"
c.MCPBearerToken = "supersecrettoken"
return c
}
func TestStringRedactsSecrets(t *testing.T) {
s := secretConfig().String()
for _, leak := range []string{"supersecretpw", "supersecrettoken"} {
if strings.Contains(s, leak) {
t.Errorf("String() leaks %q: %s", leak, s)
}
}
}
// TestSlogJSONRedactsSecrets guards the bug where slog's JSON handler
// serialized Config struct fields directly, bypassing String() and leaking
// the DB password into logs.
func TestSlogJSONRedactsSecrets(t *testing.T) {
var buf bytes.Buffer
logger := slog.New(slog.NewJSONHandler(&buf, nil))
logger.Info("starting", "config", secretConfig())
out := buf.String()
for _, leak := range []string{"supersecretpw", "supersecrettoken"} {
if strings.Contains(out, leak) {
t.Errorf("slog JSON output leaks %q: %s", leak, out)
}
}
if !strings.Contains(out, "***") {
t.Errorf("expected redaction marker in log output: %s", out)
}
}

300
internal/db/export.go Normal file
View File

@@ -0,0 +1,300 @@
package db
import (
"context"
"encoding/json"
"fmt"
"gopkg.in/yaml.v3"
)
// ExportToYAML regenerates the three seed YAMLs from the DB (DR / version
// control, plan D6). Output is deterministic: maps marshal with sorted keys
// (yaml.v3 default for map[string]any) and lists are ordered by slug/name,
// so export → ingest → export is byte-stable.
//
// Cognition-layer entities (signals, executions, patterns, …) are runtime
// state, not inventory — they are excluded from the export.
func ExportToYAML(ctx context.Context, pool *Pool) (map[string][]byte, error) {
result := make(map[string][]byte)
for name, fn := range map[string]func(context.Context, *Pool) (map[string]any, error){
"ontology.yaml": exportOntology,
"inventory.yaml": exportInventory,
"policy.yaml": exportPolicy,
} {
doc, err := fn(ctx, pool)
if err != nil {
return nil, fmt.Errorf("export %s: %w", name, err)
}
out, err := yaml.Marshal(doc)
if err != nil {
return nil, fmt.Errorf("marshal %s: %w", name, err)
}
result[name] = out
}
return result, nil
}
func exportOntology(ctx context.Context, pool *Pool) (map[string]any, error) {
lifecycles := map[string]any{}
rows, err := pool.Query(ctx,
`SELECT id, states, default_state, terminal_states, transitions
FROM lifecycle_defs ORDER BY id`)
if err != nil {
return nil, err
}
for rows.Next() {
var id, def string
var states, terminal []string
var transitionsJSON []byte
if err := rows.Scan(&id, &states, &def, &terminal, &transitionsJSON); err != nil {
rows.Close()
return nil, err
}
var transitions map[string]any
if err := json.Unmarshal(transitionsJSON, &transitions); err != nil {
rows.Close()
return nil, fmt.Errorf("lifecycle %s transitions: %w", id, err)
}
lifecycles[id] = map[string]any{
"states": states,
"default_state": def,
"terminal_states": terminal,
"transitions": transitions,
}
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
entityTypes := map[string]any{}
rows, err = pool.Query(ctx,
`SELECT name, COALESCE(parent_type,''), is_abstract, domain, layer,
COALESCE(description,''), COALESCE(lifecycle_id,''), attribute_schema
FROM entity_types ORDER BY name`)
if err != nil {
return nil, err
}
for rows.Next() {
var name, parent, dom, layer, desc, lc string
var isAbstract bool
var schemaJSON []byte
if err := rows.Scan(&name, &parent, &isAbstract, &dom, &layer, &desc, &lc, &schemaJSON); err != nil {
rows.Close()
return nil, err
}
et := map[string]any{"domain": dom, "layer": layer}
if parent != "" {
et["parent"] = parent
}
if isAbstract {
et["abstract"] = true
}
if desc != "" {
et["description"] = desc
}
if lc != "" {
et["lifecycle"] = lc
}
if len(schemaJSON) > 0 && string(schemaJSON) != "null" {
var schema map[string]any
if err := json.Unmarshal(schemaJSON, &schema); err == nil && schema != nil {
et["attributes"] = schema
}
}
entityTypes[name] = et
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
relTypes := map[string]any{}
rows, err = pool.Query(ctx,
`SELECT name, COALESCE(inverse,''), source_type, target_type, cardinality,
COALESCE(description,'')
FROM relationship_types ORDER BY name`)
if err != nil {
return nil, err
}
for rows.Next() {
var name, inverse, src, tgt, card, desc string
if err := rows.Scan(&name, &inverse, &src, &tgt, &card, &desc); err != nil {
rows.Close()
return nil, err
}
rt := map[string]any{"source": src, "target": tgt, "cardinality": card}
if inverse != "" {
rt["inverse"] = inverse
}
if desc != "" {
rt["description"] = desc
}
relTypes[name] = rt
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
return map[string]any{
"version": 1,
"lifecycles": lifecycles,
"entity_types": entityTypes,
"relationship_types": relTypes,
}, nil
}
func exportInventory(ctx context.Context, pool *Pool) (map[string]any, error) {
var entities []any
rows, err := pool.Query(ctx, `
SELECT e.slug, e.type, e.name, COALESCE(e.state,''), e.attributes
FROM entities e
JOIN entity_types et ON et.name = e.type
WHERE et.layer <> 'cognition'
ORDER BY e.slug`)
if err != nil {
return nil, err
}
for rows.Next() {
var slug, typ, name, state string
var attrsJSON []byte
if err := rows.Scan(&slug, &typ, &name, &state, &attrsJSON); err != nil {
rows.Close()
return nil, err
}
e := map[string]any{"slug": slug, "type": typ, "name": name}
if state != "" {
e["state"] = state
}
var attrs map[string]any
if err := json.Unmarshal(attrsJSON, &attrs); err == nil && len(attrs) > 0 {
e["attributes"] = attrs
}
entities = append(entities, e)
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
var rels []any
rows, err = pool.Query(ctx, `
SELECT se.slug, te.slug, r.type, r.attributes
FROM relationships r
JOIN entities se ON se.id = r.source_id
JOIN entities te ON te.id = r.target_id
WHERE r.valid_to IS NULL
ORDER BY r.type, se.slug, te.slug`)
if err != nil {
return nil, err
}
for rows.Next() {
var src, tgt, typ string
var attrsJSON []byte
if err := rows.Scan(&src, &tgt, &typ, &attrsJSON); err != nil {
rows.Close()
return nil, err
}
rel := map[string]any{"source": src, "target": tgt, "type": typ}
var attrs map[string]any
if len(attrsJSON) > 0 && json.Unmarshal(attrsJSON, &attrs) == nil && len(attrs) > 0 {
rel["attributes"] = attrs
}
rels = append(rels, rel)
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
return map[string]any{
"version": 1,
"entities": entities,
"relationships": rels,
}, nil
}
func exportPolicy(ctx context.Context, pool *Pool) (map[string]any, error) {
riskClasses := map[string]any{}
rows, err := pool.Query(ctx,
`SELECT name, COALESCE(description,''), approval_required, autonomy_allowed
FROM risk_classes ORDER BY name`)
if err != nil {
return nil, err
}
for rows.Next() {
var name, desc, approval string
var autonomy bool
if err := rows.Scan(&name, &desc, &approval, &autonomy); err != nil {
rows.Close()
return nil, err
}
rc := map[string]any{"approval_required": approval, "autonomy_allowed": autonomy}
if desc != "" {
rc["description"] = desc
}
riskClasses[name] = rc
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
var rules []any
rows, err = pool.Query(ctx, `
SELECT COALESCE(ar.entity_type,''), ar.action, ar.risk_class,
ar.autonomy_level, COALESCE(se.slug,'')
FROM approval_rules ar
LEFT JOIN entities se ON se.id = ar.scope_entity
ORDER BY COALESCE(ar.entity_type,''), ar.action, COALESCE(se.slug,'')`)
if err != nil {
return nil, err
}
for rows.Next() {
var et, action, rc, al, scope string
if err := rows.Scan(&et, &action, &rc, &al, &scope); err != nil {
rows.Close()
return nil, err
}
rule := map[string]any{"action": action, "risk_class": rc, "autonomy_level": al}
if et != "" {
rule["entity_type"] = et
}
if scope != "" {
rule["scope_entity"] = scope
}
rules = append(rules, rule)
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
settings := map[string]any{}
rows, err = pool.Query(ctx, `SELECT key, value FROM autonomy_settings ORDER BY key`)
if err != nil {
return nil, err
}
for rows.Next() {
var k, v string
if err := rows.Scan(&k, &v); err != nil {
rows.Close()
return nil, err
}
settings[k] = v
}
rows.Close()
if rows.Err() != nil {
return nil, rows.Err()
}
return map[string]any{
"version": 1,
"risk_classes": riskClasses,
"approval_rules": rules,
"autonomy_settings": settings,
}, nil
}

View File

@@ -0,0 +1,338 @@
package db
// Integration tests against a real TimescaleDB. Guarded by
// OIKOS_TEST_DATABASE_URL — skipped when unset. Run with:
//
// docker compose up -d postgres
// OIKOS_TEST_DATABASE_URL="postgres://oikos:oikos_dev@localhost:5432/oikos?sslmode=disable" go test ./internal/db/
//
// or `make test-db`. Each run creates a throwaway database and drops it.
import (
"bytes"
"context"
"errors"
"fmt"
"math/rand"
"os"
"strings"
"testing"
"github.com/dtoro/oikos/internal/domain"
"github.com/jackc/pgx/v5"
"gopkg.in/yaml.v3"
)
func seedsDir() string { return "../../seeds" }
// newTestPool creates a throwaway database (dropped on cleanup), runs all
// migrations, and returns a pool connected to it.
func newTestPool(t *testing.T) *Pool {
t.Helper()
baseURL := os.Getenv("OIKOS_TEST_DATABASE_URL")
if baseURL == "" {
t.Skip("OIKOS_TEST_DATABASE_URL not set — skipping integration test")
}
ctx := context.Background()
admin, err := pgx.Connect(ctx, baseURL)
if err != nil {
t.Fatalf("connect admin: %v", err)
}
dbName := fmt.Sprintf("oikos_test_%08x", rand.Int63())
if _, err := admin.Exec(ctx, "CREATE DATABASE "+dbName); err != nil {
admin.Close(ctx)
t.Fatalf("create test db: %v", err)
}
admin.Close(ctx)
testURL := swapDatabase(baseURL, dbName)
pool, err := New(ctx, testURL)
if err != nil {
t.Fatalf("connect test db: %v", err)
}
t.Cleanup(func() {
pool.Close()
admin, err := pgx.Connect(ctx, baseURL)
if err == nil {
admin.Exec(ctx, "DROP DATABASE IF EXISTS "+dbName+" WITH (FORCE)")
admin.Close(ctx)
}
})
if err := pool.Migrate(ctx); err != nil {
t.Fatalf("migrate: %v", err)
}
return pool
}
// swapDatabase replaces the database name in a postgres URL.
func swapDatabase(url, db string) string {
// postgres://user:pass@host:port/dbname?params
qi := strings.Index(url, "?")
params := ""
base := url
if qi >= 0 {
base, params = url[:qi], url[qi:]
}
si := strings.LastIndex(base, "/")
return base[:si+1] + db + params
}
func seedAll(t *testing.T, pool *Pool, dir string) {
t.Helper()
for _, f := range []string{"ontology.yaml", "inventory.yaml", "policy.yaml"} {
content, err := os.ReadFile(dir + "/" + f)
if err != nil {
t.Fatalf("read %s: %v", f, err)
}
ingestSeedContent(t, pool, f, content)
}
}
func ingestSeedContent(t *testing.T, pool *Pool, name string, content []byte) {
t.Helper()
ctx := context.Background()
err := pool.SeedIngest(ctx, name, content,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
var err error
switch name {
case "ontology.yaml":
_, err = IngestOntologySeed(ctx, tx, data)
case "inventory.yaml":
_, err = IngestInventorySeed(ctx, tx, data)
case "policy.yaml":
_, err = IngestPolicySeed(ctx, tx, data)
}
return err
})
if err != nil {
t.Fatalf("ingest %s: %v", name, err)
}
}
func count(t *testing.T, pool *Pool, query string) int {
t.Helper()
var n int
if err := pool.QueryRow(context.Background(), query).Scan(&n); err != nil {
t.Fatalf("count %q: %v", query, err)
}
return n
}
func TestMigrateIdempotent(t *testing.T) {
pool := newTestPool(t)
// second run must be a clean no-op
if err := pool.Migrate(context.Background()); err != nil {
t.Fatalf("second migrate: %v", err)
}
}
func TestSeedIngestIdempotentAndNoDuplicateEdges(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
entities := count(t, pool, "SELECT count(*) FROM entities")
edges := count(t, pool, "SELECT count(*) FROM relationships WHERE valid_to IS NULL")
if entities == 0 || edges == 0 {
t.Fatalf("seed produced empty graph: %d entities, %d edges", entities, edges)
}
// Same content → hash no-op
seedAll(t, pool, seedsDir())
if got := count(t, pool, "SELECT count(*) FROM relationships WHERE valid_to IS NULL"); got != edges {
t.Errorf("unchanged re-seed altered edges: %d → %d", edges, got)
}
// Changed content (hash differs) → full re-ingest must NOT duplicate edges
// (regression: the old upsert conflicted on valid_from and duplicated all
// 144 edges on every re-ingest)
content, err := os.ReadFile(seedsDir() + "/inventory.yaml")
if err != nil {
t.Fatal(err)
}
touched := append(content, []byte("\n# touched for hash change\n")...)
ingestSeedContent(t, pool, "inventory.yaml", touched)
if got := count(t, pool, "SELECT count(*) FROM relationships WHERE valid_to IS NULL"); got != edges {
t.Errorf("touched re-seed duplicated edges: %d → %d", edges, got)
}
if dup := count(t, pool, `SELECT count(*) FROM (
SELECT source_id, target_id, type FROM relationships
WHERE valid_to IS NULL GROUP BY 1,2,3 HAVING count(*) > 1) d`); dup != 0 {
t.Errorf("%d duplicated current edges", dup)
}
if got := count(t, pool, "SELECT count(*) FROM entities"); got != entities {
t.Errorf("touched re-seed altered entity count: %d → %d", entities, got)
}
}
func TestAbstractTypeRejected(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
bad := []byte(`
version: 1
entities:
- {slug: "machine:ghost", type: machine, name: ghost}
`)
ctx := context.Background()
err := pool.SeedIngest(ctx, "inventory.yaml", bad,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if !errors.Is(err, domain.ErrAbstractType) {
t.Errorf("abstract instantiation = %v, want ErrAbstractType", err)
}
}
func TestEdgeEndpointValidation(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
// routes-to requires source ingress-route; a service source must fail
bad := []byte(`
version: 1
relationships:
- {source: "service:gitea", target: "service:caddy", type: routes-to}
`)
ctx := context.Background()
err := pool.SeedIngest(ctx, "inventory.yaml", bad,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if !errors.Is(err, domain.ErrInvalidEdge) {
t.Errorf("bad edge = %v, want ErrInvalidEdge", err)
}
// hosts from a proxmox-host (is-a machine) to an lxc (is-a compute-entity)
// must PASS via hierarchy walk — already covered by the seed itself, but
// assert an explicit one for clarity
good := []byte(`
version: 1
relationships:
- {source: "host:strong", target: "lxc:jellyfin", type: hosts}
`)
err = pool.SeedIngest(ctx, "inventory.yaml", good,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if err != nil {
t.Errorf("valid inherited edge rejected: %v", err)
}
}
func TestCardinalityEnforced(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
// routes-to is many-to-one: one ingress route cannot point at two services
bad := []byte(`
version: 1
relationships:
- {source: "ingress:git.hubris.network", target: "service:jellyfin", type: routes-to}
`)
ctx := context.Background()
err := pool.SeedIngest(ctx, "inventory.yaml", bad,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if err == nil || !strings.Contains(err.Error(), "cardinality") {
t.Errorf("cardinality violation = %v, want cardinality error", err)
}
}
func TestBlastRadiusTerminatesOnCycles(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
// Build a dependency cycle: gitea → caddy → authentik → gitea
cycle := []byte(`
version: 1
relationships:
- {source: "service:gitea", target: "service:caddy", type: depends-on}
- {source: "service:caddy", target: "service:authentik", type: depends-on}
- {source: "service:authentik", target: "service:gitea", type: depends-on}
`)
ctx := context.Background()
err := pool.SeedIngest(ctx, "inventory.yaml", cycle,
func(ctx context.Context, tx pgx.Tx, data map[string]any) error {
_, err := IngestInventorySeed(ctx, tx, data)
return err
})
if err != nil {
t.Fatalf("cycle ingest: %v", err)
}
rows, err := pool.Query(ctx, `
SELECT e.slug, b.depth
FROM blast_radius((SELECT id FROM entities WHERE slug='service:gitea'), 5,
ARRAY['depends-on']) b
JOIN entities e ON e.id = b.entity_id ORDER BY b.depth`)
if err != nil {
t.Fatalf("blast_radius: %v", err)
}
defer rows.Close()
got := map[string]int{}
for rows.Next() {
var slug string
var depth int
if err := rows.Scan(&slug, &depth); err != nil {
t.Fatal(err)
}
got[slug] = depth
}
want := map[string]int{"service:gitea": 0, "service:caddy": 1, "service:authentik": 2}
for slug, depth := range want {
if got[slug] != depth {
t.Errorf("blast_radius[%s] = %d, want %d (full: %v)", slug, got[slug], depth, got)
}
}
if len(got) != len(want) {
t.Errorf("blast_radius returned %d nodes, want %d: %v", len(got), len(want), got)
}
}
// TestExportRoundTripStable: export → ingest into a fresh DB → export again
// must yield byte-identical YAML (the canonical-form fixpoint, plan D6).
func TestExportRoundTripStable(t *testing.T) {
pool := newTestPool(t)
seedAll(t, pool, seedsDir())
ctx := context.Background()
export1, err := ExportToYAML(ctx, pool)
if err != nil {
t.Fatalf("export 1: %v", err)
}
for name, content := range export1 {
var doc map[string]any
if err := yaml.Unmarshal(content, &doc); err != nil {
t.Fatalf("export %s is not valid YAML: %v", name, err)
}
}
pool2 := newTestPool(t)
for _, name := range []string{"ontology.yaml", "inventory.yaml", "policy.yaml"} {
ingestSeedContent(t, pool2, name, export1[name])
}
export2, err := ExportToYAML(ctx, pool2)
if err != nil {
t.Fatalf("export 2: %v", err)
}
for _, name := range []string{"ontology.yaml", "inventory.yaml", "policy.yaml"} {
if !bytes.Equal(export1[name], export2[name]) {
t.Errorf("%s round-trip not byte-stable (len %d vs %d)",
name, len(export1[name]), len(export2[name]))
}
}
// sanity: exported inventory carries the real graph, not a stub
// (regression: export used to write 11-byte "version: 1" stubs)
if len(export1["inventory.yaml"]) < 1000 {
t.Errorf("inventory export suspiciously small: %d bytes", len(export1["inventory.yaml"]))
}
}

250
internal/db/pool.go Normal file
View File

@@ -0,0 +1,250 @@
package db
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io/fs"
"log/slog"
"sort"
"strings"
"github.com/dtoro/oikos/migrations"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"gopkg.in/yaml.v3"
)
// Pool wraps a pgx connection pool.
type Pool struct {
*pgxpool.Pool
}
// New creates a new connection pool.
func New(ctx context.Context, databaseURL string) (*Pool, error) {
cfg, err := pgxpool.ParseConfig(databaseURL)
if err != nil {
return nil, fmt.Errorf("parse database url: %w", err)
}
cfg.MaxConns = 15
pool, err := pgxpool.NewWithConfig(ctx, cfg)
if err != nil {
return nil, fmt.Errorf("create pool: %w", err)
}
if err := pool.Ping(ctx); err != nil {
return nil, fmt.Errorf("ping db: %w", err)
}
return &Pool{pool}, nil
}
// migrationLockKey is the advisory-lock key serializing migration runs —
// two concurrent `oikos migrate` invocations must not interleave DDL.
const migrationLockKey = 0x01c05e5
// Migrate runs all embedded forward migrations in order.
// Uses a schema_migrations table to track applied versions. The whole run
// happens on one connection holding a session advisory lock.
func (p *Pool) Migrate(ctx context.Context) error {
conn, err := p.Acquire(ctx)
if err != nil {
return fmt.Errorf("acquire migration conn: %w", err)
}
defer conn.Release()
if _, err := conn.Exec(ctx, "SELECT pg_advisory_lock($1)", migrationLockKey); err != nil {
return fmt.Errorf("acquire migration lock: %w", err)
}
defer conn.Exec(context.WithoutCancel(ctx), "SELECT pg_advisory_unlock($1)", migrationLockKey)
// Create tracking table if not exists
_, err = conn.Exec(ctx, `
CREATE TABLE IF NOT EXISTS schema_migrations (
version INT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
`)
if err != nil {
return fmt.Errorf("create schema_migrations: %w", err)
}
// List migration files
entries, err := fs.ReadDir(migrations.FS, ".")
if err != nil {
return fmt.Errorf("read migration fs: %w", err)
}
var files []string
for _, e := range entries {
if !e.IsDir() && hasSuffix(e.Name(), ".up.sql") {
files = append(files, e.Name())
}
}
sort.Strings(files)
for _, fname := range files {
// Extract version number (001, 002, etc.)
var version int
if _, err := fmt.Sscanf(fname, "%03d", &version); err != nil {
continue
}
// Check if already applied
var applied int
err := conn.QueryRow(ctx,
"SELECT COUNT(*) FROM schema_migrations WHERE version = $1", version).Scan(&applied)
if err != nil {
return fmt.Errorf("check migration %d: %w", version, err)
}
if applied > 0 {
continue
}
// Read and execute migration — split into individual statements
// because TimescaleDB CAGGs and some DDL can't run inside a transaction,
// and pgx's multi-statement Exec wraps them implicitly.
content, err := migrations.FS.ReadFile(fname)
if err != nil {
return fmt.Errorf("read %s: %w", fname, err)
}
stmts := splitSQL(string(content))
for i, stmt := range stmts {
stmt = strings.TrimSpace(stmt)
if stmt == "" {
continue
}
_, err := conn.Exec(ctx, stmt)
if err != nil {
return fmt.Errorf("exec migration %s stmt %d: %w", fname, i+1, err)
}
}
_, err = conn.Exec(ctx, "INSERT INTO schema_migrations (version) VALUES ($1)", version)
if err != nil {
return fmt.Errorf("record migration %d: %w", version, err)
}
slog.Info("migration applied", "file", fname, "version", version, "statements", len(stmts))
}
return nil
}
// SeedIngest ingests a YAML seed file into the database.
// Idempotent: if the file's content hash matches seed_versions, it's a no-op (A4).
func (p *Pool) SeedIngest(ctx context.Context, filename string, content []byte,
ingestFn func(ctx context.Context, tx pgx.Tx, data map[string]any) error) error {
hash := contentHash(content)
// Check if already applied with same hash
var existing string
err := p.QueryRow(ctx,
"SELECT content_hash FROM seed_versions WHERE file = $1", filename).Scan(&existing)
if err == nil && existing == hash {
return nil // no-op, same content
}
// Parse YAML
var data map[string]any
if err := yaml.Unmarshal(content, &data); err != nil {
return fmt.Errorf("parse %s: %w", filename, err)
}
// Apply in a single transaction
tx, err := p.Begin(ctx)
if err != nil {
return fmt.Errorf("begin tx: %w", err)
}
defer tx.Rollback(ctx)
if err := ingestFn(ctx, tx, data); err != nil {
return fmt.Errorf("ingest %s: %w", filename, err)
}
// Record the seed version
_, err = tx.Exec(ctx,
`INSERT INTO seed_versions (file, content_hash) VALUES ($1, $2)
ON CONFLICT (file) DO UPDATE SET content_hash = $2, applied_at = now()`,
filename, hash)
if err != nil {
return fmt.Errorf("record seed version: %w", err)
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("commit seed: %w", err)
}
return nil
}
// contentHash returns a SHA-256 hex digest of the content.
func contentHash(content []byte) string {
h := sha256.Sum256(content)
return hex.EncodeToString(h[:])
}
// hasSuffix reports whether the string ends with the given suffix.
func hasSuffix(s, suffix string) bool {
return strings.HasSuffix(s, suffix)
}
// splitSQL splits a SQL string into individual statements.
// Handles $$ ... $$ dollar-quoted blocks and -- line comments.
func splitSQL(sql string) []string {
var statements []string
var current strings.Builder
inDollarQuote := false
dollarTag := ""
i := 0
for i < len(sql) {
// Handle line comments (-- to end of line)
if !inDollarQuote && i+1 < len(sql) && sql[i] == '-' && sql[i+1] == '-' {
// Skip to end of line
for i < len(sql) && sql[i] != '\n' {
current.WriteByte(sql[i])
i++
}
continue
}
// Check for dollar-quote start/end
if !inDollarQuote && sql[i] == '$' {
j := i + 1
for j < len(sql) && (sql[j] == '_' || (sql[j] >= 'a' && sql[j] <= 'z') || (sql[j] >= 'A' && sql[j] <= 'Z') || (sql[j] >= '0' && sql[j] <= '9')) {
j++
}
if j < len(sql) && sql[j] == '$' {
dollarTag = sql[i : j+1]
current.WriteString(dollarTag)
inDollarQuote = true
i = j + 1
continue
}
}
if inDollarQuote && strings.HasPrefix(sql[i:], dollarTag) {
current.WriteString(dollarTag)
i += len(dollarTag)
inDollarQuote = false
dollarTag = ""
continue
}
if !inDollarQuote && sql[i] == ';' {
statements = append(statements, current.String())
current.Reset()
i++
continue
}
current.WriteByte(sql[i])
i++
}
if strings.TrimSpace(current.String()) != "" {
statements = append(statements, current.String())
}
return statements
}

View File

@@ -0,0 +1,52 @@
-- Entity read + mutation queries (API paths). Aliased `e` throughout to
-- avoid ambiguity with joined tables.
-- name: GetEntityByID :one
SELECT e.* FROM entities e WHERE e.id = $1;
-- name: GetEntityBySlug :one
SELECT e.* FROM entities e WHERE e.slug = $1;
-- name: ListEntities :many
WITH RECURSIVE tt AS (
SELECT name FROM entity_types WHERE sqlc.narg('type')::text IS NULL OR name = sqlc.narg('type')
UNION
SELECT et.name FROM entity_types et JOIN tt ON et.parent_type = tt.name
WHERE sqlc.narg('type')::text IS NOT NULL
)
SELECT e.* FROM entities e
JOIN entity_types et ON et.name = e.type
WHERE e.type IN (SELECT name FROM tt)
AND (sqlc.narg('state')::text IS NULL OR e.state = sqlc.narg('state'))
AND (sqlc.narg('domain')::text IS NULL OR et.domain = sqlc.narg('domain'))
AND (sqlc.narg('layer')::text IS NULL OR et.layer = sqlc.narg('layer'))
AND (sqlc.narg('q')::text IS NULL
OR e.slug ILIKE '%'||sqlc.narg('q')||'%'
OR e.name ILIKE '%'||sqlc.narg('q')||'%')
AND (sqlc.narg('cursor')::text IS NULL OR e.slug > sqlc.narg('cursor'))
ORDER BY e.slug
LIMIT sqlc.arg('lim');
-- name: ListEntitiesCapped :many
SELECT e.* FROM entities e ORDER BY e.slug LIMIT $1;
-- name: InsertEntity :one
INSERT INTO entities (id, slug, type, name, state, attributes)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING *;
-- name: UpdateEntity :one
UPDATE entities SET
name = COALESCE(sqlc.narg('name'), name),
state = COALESCE(sqlc.narg('state'), state),
attributes = COALESCE(sqlc.narg('attributes'), attributes),
maintenance_until = CASE WHEN sqlc.arg('set_maintenance')::bool
THEN sqlc.narg('maintenance_until') ELSE maintenance_until END,
version = version + 1,
updated_at = now()
WHERE id = sqlc.arg('id') AND version = sqlc.arg('version')
RETURNING *;
-- blast_radius(): the recursive-CTE traversal function's TABLE return type
-- is opaque to sqlc's analyzer — that one query stays hand-written pgx in
-- internal/httpapi (see impl.go).

View File

@@ -0,0 +1,13 @@
-- name: ListEntityTypes :many
SELECT * FROM entity_types ORDER BY name;
-- name: ListRelationshipTypes :many
SELECT * FROM relationship_types ORDER BY name;
-- name: ListLifecycleDefs :many
SELECT * FROM lifecycle_defs ORDER BY id;
-- name: GetLifecycleForType :one
SELECT ld.* FROM lifecycle_defs ld
JOIN entity_types et ON et.lifecycle_id = ld.id
WHERE et.name = $1;

View File

@@ -0,0 +1,269 @@
-- name: ListSignals :many
SELECT sig.entity_id, se.slug, sig.kind, sig.severity, sig.state,
te.slug AS target_slug, sig.check_id, sig.evidence, sig.likely_cause,
sig.occurrence_count, sig.flap_count, sig.hold_down_until,
sig.mute_until, sig.first_seen_at, sig.last_seen_at
FROM signals sig
JOIN entities se ON se.id = sig.entity_id
LEFT JOIN entities te ON te.id = sig.target_entity_id
WHERE (sqlc.narg('state')::text IS NULL OR sig.state = sqlc.narg('state'))
AND (sqlc.narg('severity')::text IS NULL OR sig.severity = sqlc.narg('severity'))
AND (sqlc.narg('target')::text IS NULL OR te.slug = sqlc.narg('target'))
AND (sqlc.narg('kind')::text IS NULL OR sig.kind = sqlc.narg('kind'))
AND (sqlc.narg('cursor')::text IS NULL OR se.slug > sqlc.narg('cursor'))
ORDER BY se.slug
LIMIT sqlc.arg('lim');
-- name: ListEntityStatus :many
SELECT e.slug, e.type, st.health, st.last_check_at
FROM entity_status st JOIN entities e ON e.id = st.entity_id
ORDER BY e.slug;
-- name: GetIdempotentResponse :one
SELECT response_code, response_body, request_hash FROM idempotency_keys
WHERE actor = $1 AND key = $2;
-- name: PutIdempotentResponse :exec
INSERT INTO idempotency_keys (actor, key, request_hash, response_code, response_body)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (actor, key) DO NOTHING;
-- name: InsertAuditEntry :exec
INSERT INTO audit_log (actor_type, actor_id, action, entity_id, method, path,
status_code, detail, source_ip, correlation_id)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10);
-- name: InsertEvent :one
INSERT INTO events (type, entity_id, severity, source, data, correlation_id)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, ts;
-- name: ListEvents :many
SELECT id, ts, type, entity_id, severity, source, data, correlation_id
FROM events
WHERE (sqlc.narg('type')::text IS NULL OR type = sqlc.narg('type'))
AND (sqlc.narg('entity_id')::uuid IS NULL OR entity_id = sqlc.narg('entity_id'))
AND (sqlc.narg('severity')::text IS NULL OR severity = sqlc.narg('severity'))
AND (sqlc.narg('correlation_id')::text IS NULL OR correlation_id = sqlc.narg('correlation_id'))
AND (sqlc.narg('from_ts')::timestamptz IS NULL OR ts >= sqlc.narg('from_ts'))
AND (sqlc.narg('to_ts')::timestamptz IS NULL OR ts <= sqlc.narg('to_ts'))
AND (sqlc.narg('before_id')::bigint IS NULL OR id < sqlc.narg('before_id'))
ORDER BY id DESC
LIMIT sqlc.arg('lim');
-- name: ListEventsAfter :many
SELECT id, ts, type, entity_id, severity, source, data, correlation_id
FROM events WHERE id > $1 ORDER BY id ASC LIMIT $2;
-- =====================================================================
-- Phase 3 queries
-- =====================================================================
-- name: ListEnabledCheckDefs :many
SELECT cd.entity_id, cd.target_id, cd.target_type, cd.kind, cd.config,
cd.interval_s, cd.timeout_s, cd.zone, cd.enabled, cd.updated_at,
e.slug AS entity_slug
FROM check_defs cd
JOIN entities e ON e.id = cd.entity_id
WHERE cd.enabled = true;
-- name: GetCheckDef :one
SELECT * FROM check_defs WHERE entity_id = $1;
-- name: InsertCheckDef :exec
INSERT INTO check_defs (entity_id, target_id, target_type, kind, config, interval_s, timeout_s, zone, enabled)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9);
-- name: UpdateCheckDef :exec
UPDATE check_defs SET kind = $2, config = $3, interval_s = $4, timeout_s = $5,
target_id = $6, target_type = $7, zone = $8, enabled = $9, updated_at = now()
WHERE entity_id = $1;
-- name: UpsertSignal :one
INSERT INTO signals (entity_id, kind, severity, target_entity_id, check_id, evidence, likely_cause, state)
VALUES ($1, $2, $3, $4, $5, $6, $7, 'raised')
ON CONFLICT (target_entity_id, kind) WHERE state NOT IN ('resolved','failed')
DO UPDATE SET occurrence_count = signals.occurrence_count + 1,
last_seen_at = now(),
evidence = EXCLUDED.evidence,
updated_at = now()
RETURNING *;
-- name: UpdateSignalState :exec
UPDATE signals SET state = $2, updated_at = now() WHERE entity_id = $1;
-- name: GetOpenSignalsForAutoAct :many
-- Signals with auto-act classifications that haven't been executed yet
SELECT s.*, c.entity_id AS classification_id, c.action, c.risk_class, c.route,
c.blast_radius, c.correlation_id, c.reasoning
FROM classifications c
JOIN signals s ON s.entity_id = c.signal_entity_id
LEFT JOIN executions e ON e.classification_id = c.entity_id
WHERE c.route = 'auto-act'
AND e.entity_id IS NULL
AND (s.hold_down_until IS NULL OR s.hold_down_until < now())
AND (s.mute_until IS NULL OR s.mute_until < now())
ORDER BY s.last_seen_at ASC
LIMIT $1;
-- name: InsertClassification :exec
INSERT INTO classifications (entity_id, signal_entity_id, target_entity_id, action,
recommended_action, risk_class, route, blast_radius, pattern_confidence,
skill_id, autonomy_check, reasoning, correlation_id)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13);
-- name: ListClassifications :many
SELECT c.entity_id, c.signal_entity_id, c.target_entity_id, c.action,
c.recommended_action, c.risk_class, c.route, c.blast_radius,
c.pattern_confidence, c.skill_id, c.autonomy_check, c.reasoning,
c.correlation_id, c.created_at,
e.slug AS target_slug
FROM classifications c
JOIN entities e ON e.id = c.target_entity_id
WHERE (sqlc.narg('route')::text IS NULL OR c.route = sqlc.narg('route'))
AND (sqlc.narg('cursor')::text IS NULL OR e.slug > sqlc.narg('cursor'))
ORDER BY e.slug
LIMIT sqlc.arg('lim');
-- name: InsertExecution :exec
INSERT INTO executions (entity_id, classification_id, signal_entity_id,
target_entity_id, action, risk_class, approval_id, agent_id,
skill_id, skill_version, status, correlation_id)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, 'proposed', $11);
-- name: UpdateExecutionStatus :exec
UPDATE executions SET status = $2, result = $3, duration_ms = $4,
verified = $5, started_at = COALESCE(started_at, now()),
completed_at = CASE WHEN $2 IN ('completed','failed','cancelled') THEN now() ELSE completed_at END
WHERE entity_id = $1;
-- name: GetExecution :one
SELECT * FROM executions WHERE entity_id = $1;
-- name: ListExecutions :many
SELECT e.entity_id, e.classification_id, e.signal_entity_id, e.target_entity_id,
e.action, e.risk_class, e.approval_id, e.agent_id,
e.skill_id, e.skill_version, e.status, e.result, e.duration_ms,
e.verified, e.correlation_id, e.started_at, e.completed_at, e.created_at,
te.slug AS target_slug
FROM executions e
JOIN entities te ON te.id = e.target_entity_id
WHERE (sqlc.narg('status')::text IS NULL OR e.status = sqlc.narg('status'))
AND (sqlc.narg('cursor')::text IS NULL OR te.slug > sqlc.narg('cursor'))
ORDER BY te.slug
LIMIT sqlc.arg('lim');
-- name: InsertFeedback :exec
INSERT INTO feedback (entity_id, execution_id, outcome, observation, lesson,
unexpected_side_effects, tags)
VALUES ($1, $2, $3, $4, $5, $6, $7);
-- name: GetFeedbackAfterWatermark :many
SELECT f.entity_id, f.execution_id, f.outcome, f.observation, f.lesson,
f.unexpected_side_effects, f.tags, f.created_at,
e.action, e.risk_class, e.target_entity_id,
et.name AS applies_type
FROM feedback f
JOIN executions e ON e.entity_id = f.execution_id
JOIN entities ent ON ent.id = e.target_entity_id
JOIN entity_types et ON et.name = ent.type
WHERE f.created_at > $1
ORDER BY f.created_at ASC;
-- name: UpsertPattern :exec
INSERT INTO patterns (entity_id, applies_type, action, pattern, confidence,
evidence_count, success_count, failure_count, status, version)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 'hypothesized', 1)
ON CONFLICT (applies_type, action)
DO UPDATE SET evidence_count = patterns.evidence_count + EXCLUDED.evidence_count,
success_count = patterns.success_count + EXCLUDED.success_count,
failure_count = patterns.failure_count + EXCLUDED.failure_count,
updated_at = now();
-- name: GetPattern :one
SELECT * FROM patterns WHERE applies_type = $1 AND action = $2;
-- name: ListPatterns :many
SELECT p.* FROM patterns p
WHERE (sqlc.narg('status')::text IS NULL OR p.status = sqlc.narg('status'))
ORDER BY p.applies_type, p.action;
-- name: UpdatePatternStatus :exec
UPDATE patterns SET status = $2, version = version + 1,
last_validated_at = CASE WHEN $2 = 'validated' THEN now() ELSE last_validated_at END
WHERE entity_id = $1;
-- name: UpdatePatternQuarantine :exec
UPDATE patterns SET quarantined = $2 WHERE entity_id = $1;
-- name: ListSkills :many
SELECT * FROM skills
WHERE (sqlc.narg('status')::text IS NULL OR status = sqlc.narg('status'))
ORDER BY name, version DESC;
-- name: InsertSkill :exec
INSERT INTO skills (entity_id, version, name, procedure, applies_type, action,
pattern_ids, status, changed_by, change_reason)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10);
-- name: UpdateSkillStatus :exec
UPDATE skills SET status = $2, last_used_at = now() WHERE entity_id = $1 AND version = $2;
-- name: InsertApproval :exec
INSERT INTO approvals (entity_id, subject_entity_id, action, risk_class, kind,
payload, status, token_hash, expires_at)
VALUES ($1, $2, $3, $4, $5, $6, 'pending', $7, $8);
-- name: ListApprovals :many
SELECT a.*, e.slug AS subject_slug
FROM approvals a
JOIN entities e ON e.id = a.subject_entity_id
WHERE (sqlc.narg('status')::text IS NULL OR a.status = sqlc.narg('status'))
AND (sqlc.narg('cursor')::text IS NULL OR e.slug > sqlc.narg('cursor'))
ORDER BY e.slug
LIMIT sqlc.arg('lim');
-- name: GetApprovalByID :one
SELECT * FROM approvals WHERE entity_id = $1;
-- name: UpdateApprovalStatus :exec
UPDATE approvals SET status = $2, decided_at = now(), decided_by = $3
WHERE entity_id = $1 AND status = 'pending';
-- name: GetAutonomySetting :one
SELECT value FROM autonomy_settings WHERE key = $1;
-- name: ListRiskClasses :many
SELECT * FROM risk_classes ORDER BY name;
-- name: ListApprovalRules :many
SELECT * FROM approval_rules ORDER BY entity_type, action;
-- name: InsertMetricSample :exec
INSERT INTO metric_samples (entity_id, metric, value, tags)
VALUES ($1, $2, $3, $4);
-- name: QueryMetrics :many
SELECT time_bucket(sqlc.arg('bucket_interval')::interval, ts) AS bucket,
entity_id, metric,
ROUND(avg(value)::numeric, 2) AS avg_val,
ROUND(min(value)::numeric, 2) AS min_val,
ROUND(max(value)::numeric, 2) AS max_val
FROM metric_samples
WHERE entity_id = $1
AND metric = $2
AND ts > $3
GROUP BY bucket, entity_id, metric
ORDER BY bucket DESC;
-- name: UpsertEntityStatus :exec
INSERT INTO entity_status (entity_id, health, last_check_at, details)
VALUES ($1, $2, $3, $4)
ON CONFLICT (entity_id)
DO UPDATE SET health = EXCLUDED.health,
last_check_at = EXCLUDED.last_check_at,
details = EXCLUDED.details,
updated_at = now();
-- name: GetEntityStatus :one
SELECT * FROM entity_status WHERE entity_id = $1;

View File

@@ -0,0 +1,33 @@
-- name: ListEntityRelations :many
SELECT se.slug AS source_slug, te.slug AS target_slug, r.type, r.attributes,
r.valid_from, r.valid_to
FROM relationships r
JOIN entities se ON se.id = r.source_id
JOIN entities te ON te.id = r.target_id
WHERE r.valid_to IS NULL
AND ((sqlc.arg('direction')::text IN ('out','both') AND r.source_id = sqlc.arg('id'))
OR (sqlc.arg('direction')::text IN ('in','both') AND r.target_id = sqlc.arg('id')))
AND (sqlc.narg('rel_type')::text IS NULL OR r.type = sqlc.narg('rel_type'))
ORDER BY r.type, se.slug, te.slug;
-- name: ListGraphEdges :many
SELECT se.slug AS source_slug, te.slug AS target_slug, r.type, r.attributes,
r.valid_from, r.valid_to
FROM relationships r
JOIN entities se ON se.id = r.source_id
JOIN entities te ON te.id = r.target_id
WHERE r.valid_to IS NULL
AND r.source_id = ANY(sqlc.arg('ids')::uuid[])
AND r.target_id = ANY(sqlc.arg('ids')::uuid[])
AND (sqlc.narg('rel_types')::text[] IS NULL OR r.type = ANY(sqlc.narg('rel_types')::text[]))
ORDER BY r.type, se.slug, te.slug;
-- name: UpsertCurrentRelationship :exec
INSERT INTO relationships (source_id, target_id, type, attributes, valid_from, valid_to)
VALUES ($1, $2, $3, $4, now(), NULL)
ON CONFLICT (source_id, target_id, type) WHERE valid_to IS NULL
DO UPDATE SET attributes = EXCLUDED.attributes;
-- name: EndCurrentRelationship :execrows
UPDATE relationships SET valid_to = now()
WHERE source_id = $1 AND target_id = $2 AND type = $3 AND valid_to IS NULL;

393
internal/db/seed.go Normal file
View File

@@ -0,0 +1,393 @@
package db
import (
"context"
"encoding/json"
"errors"
"fmt"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
)
// SeedResult holds counts from a seed ingest operation.
type SeedResult struct {
Lifecycles int
EntityTypes int
RelationshipTypes int
Entities int
Relationships int
RiskClasses int
ApprovalRules int
AutonomySettings int
}
// IngestOntologySeed ingests seeds/ontology.yaml into the DB.
func IngestOntologySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*SeedResult, error) {
r := &SeedResult{}
// Lifecycles
lifecycles, _ := data["lifecycles"].(map[string]any)
for id, raw := range lifecycles {
lcMap, _ := raw.(map[string]any)
states := toStringSlice(lcMap["states"])
defaultState, _ := lcMap["default_state"].(string)
terminalStates := toStringSlice(lcMap["terminal_states"])
if len(terminalStates) == 0 {
terminalStates = []string{}
}
transitionsBytes, _ := json.Marshal(lcMap["transitions"])
_, err := tx.Exec(ctx,
`INSERT INTO lifecycle_defs (id, states, default_state, terminal_states, transitions)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (id) DO UPDATE SET states = $2, default_state = $3,
terminal_states = $4, transitions = $5`,
id, states, defaultState, terminalStates, string(transitionsBytes))
if err != nil {
return nil, fmt.Errorf("lifecycle %s: %w", id, err)
}
r.Lifecycles++
}
// Entity types — need to handle parent_type FK, so insert in dependency order
// (types with no parent first, then their children)
types, _ := data["entity_types"].(map[string]any)
if err := insertEntityTypes(ctx, tx, types, r); err != nil {
return nil, err
}
// Relationship types
relTypes, _ := data["relationship_types"].(map[string]any)
for name, raw := range relTypes {
rtMap, _ := raw.(map[string]any)
inverse, _ := rtMap["inverse"].(string)
sourceType, _ := rtMap["source"].(string)
targetType, _ := rtMap["target"].(string)
cardinality, _ := rtMap["cardinality"].(string)
desc, _ := rtMap["description"].(string)
_, err := tx.Exec(ctx,
`INSERT INTO relationship_types (name, inverse, source_type, target_type, cardinality, description)
VALUES ($1, $2, $3, $4, $5, $6)
ON CONFLICT (name) DO UPDATE SET inverse = $2, source_type = $3,
target_type = $4, cardinality = $5, description = $6`,
name, nullableStr(inverse), sourceType, targetType, cardinality, desc)
if err != nil {
return nil, fmt.Errorf("relationship_type %s: %w", name, err)
}
r.RelationshipTypes++
}
return r, nil
}
// IngestInventorySeed ingests seeds/inventory.yaml into the DB.
// Every entity and edge is validated against the ontology (abstract types
// rejected, lifecycle states checked, relationship endpoints hierarchy-
// validated, cardinality enforced) — a violating seed rolls back atomically.
func IngestInventorySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*SeedResult, error) {
r := &SeedResult{}
tree, err := LoadTypeTree(ctx, tx)
if err != nil {
return nil, fmt.Errorf("load type tree: %w", err)
}
// Entities
entities, _ := data["entities"].([]any)
entityTypes := make(map[string]string) // slug -> type, for edge validation
for _, raw := range entities {
eMap, ok := raw.(map[string]any)
if !ok {
continue
}
slug, _ := eMap["slug"].(string)
typeName, _ := eMap["type"].(string)
name, _ := eMap["name"].(string)
state, _ := eMap["state"].(string)
attrs := eMap["attributes"]
if err := tree.ValidateEntity(typeName, state); err != nil {
return nil, fmt.Errorf("entity %s: %w", slug, err)
}
if state == "" {
state = tree.DefaultState(typeName)
}
entityTypes[slug] = typeName
entityID, err := getOrCreateEntityID(ctx, tx, slug)
if err != nil {
return nil, fmt.Errorf("entity %s: %w", slug, err)
}
attrsBytes, _ := json.Marshal(attrs)
_, err = tx.Exec(ctx,
`INSERT INTO entities (id, slug, type, name, state, attributes, version, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, 1, now(), now())
ON CONFLICT (slug) DO UPDATE SET type = $3, name = $4, state = $5,
attributes = $6, updated_at = now()`,
entityID, slug, typeName, name, nullableStr(state), string(attrsBytes))
if err != nil {
return nil, fmt.Errorf("entity %s: %w", slug, err)
}
r.Entities++
}
// Relationships — upsert against the current-edge partial unique index
// (migration 007) so re-ingest never duplicates edges.
rels, _ := data["relationships"].([]any)
for _, raw := range rels {
relMap, ok := raw.(map[string]any)
if !ok {
continue
}
source, _ := relMap["source"].(string)
target, _ := relMap["target"].(string)
relType, _ := relMap["type"].(string)
attrs := relMap["attributes"]
sourceID, err := getEntityIDBySlug(ctx, tx, source)
if err != nil {
return nil, fmt.Errorf("rel from %s: %w", source, err)
}
targetID, err := getEntityIDBySlug(ctx, tx, target)
if err != nil {
return nil, fmt.Errorf("rel to %s: %w", target, err)
}
srcType := entityTypes[source]
tgtType := entityTypes[target]
if srcType == "" || tgtType == "" { // entity pre-existing in DB, not in this seed
if srcType == "" {
srcType, err = getEntityTypeBySlug(ctx, tx, source)
if err != nil {
return nil, err
}
}
if tgtType == "" {
tgtType, err = getEntityTypeBySlug(ctx, tx, target)
if err != nil {
return nil, err
}
}
}
if err := tree.ValidateEdge(relType, srcType, tgtType); err != nil {
return nil, fmt.Errorf("rel %s→%s: %w", source, target, err)
}
attrsBytes, _ := json.Marshal(attrs)
_, err = tx.Exec(ctx,
`INSERT INTO relationships (source_id, target_id, type, attributes, valid_from, valid_to)
VALUES ($1, $2, $3, $4, now(), NULL)
ON CONFLICT (source_id, target_id, type) WHERE valid_to IS NULL
DO UPDATE SET attributes = EXCLUDED.attributes`,
sourceID, targetID, relType, string(attrsBytes))
if err != nil {
return nil, fmt.Errorf("rel %s→%s %s: %w", source, target, relType, err)
}
r.Relationships++
}
if err := ValidateCardinality(ctx, tx); err != nil {
return nil, err
}
return r, nil
}
// IngestPolicySeed ingests seeds/policy.yaml into the DB.
func IngestPolicySeed(ctx context.Context, tx pgx.Tx, data map[string]any) (*SeedResult, error) {
r := &SeedResult{}
// Risk classes
riskClasses, _ := data["risk_classes"].(map[string]any)
for name, raw := range riskClasses {
rcMap, _ := raw.(map[string]any)
desc, _ := rcMap["description"].(string)
approval, _ := rcMap["approval_required"].(string)
autonomy, _ := rcMap["autonomy_allowed"].(bool)
_, err := tx.Exec(ctx,
`INSERT INTO risk_classes (name, description, approval_required, autonomy_allowed)
VALUES ($1, $2, $3, $4)
ON CONFLICT (name) DO UPDATE SET description = $2, approval_required = $3, autonomy_allowed = $4`,
name, desc, approval, autonomy)
if err != nil {
return nil, fmt.Errorf("risk_class %s: %w", name, err)
}
r.RiskClasses++
}
// Approval rules
rules, _ := data["approval_rules"].([]any)
for _, raw := range rules {
ruleMap, ok := raw.(map[string]any)
if !ok {
continue
}
entityType, _ := ruleMap["entity_type"].(string)
action, _ := ruleMap["action"].(string)
riskClass, _ := ruleMap["risk_class"].(string)
autonomy, _ := ruleMap["autonomy_level"].(string)
scopeEntity, _ := ruleMap["scope_entity"].(string)
var scopeID any
if scopeEntity != "" {
id, err := getEntityIDBySlug(ctx, tx, scopeEntity)
if err == nil {
scopeID = id
}
}
ruleID := uuid.New()
_, err := tx.Exec(ctx,
`INSERT INTO approval_rules (id, entity_type, action, risk_class, autonomy_level, scope_entity, version, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, 1, now())
ON CONFLICT (entity_type, action, scope_entity)
DO UPDATE SET risk_class = $4, autonomy_level = $5, scope_entity = $6, updated_at = now()`,
ruleID, nullableStr(entityType), action, riskClass, autonomy, scopeID)
if err != nil {
return nil, fmt.Errorf("approval_rule %s/%s: %w", entityType, action, err)
}
r.ApprovalRules++
}
// Autonomy settings
settings, _ := data["autonomy_settings"].(map[string]any)
for key, raw := range settings {
val, _ := raw.(string)
_, err := tx.Exec(ctx,
`INSERT INTO autonomy_settings (key, value, version, updated_at)
VALUES ($1, $2, 1, now())
ON CONFLICT (key) DO UPDATE SET value = $2, updated_at = now()`,
key, val)
if err != nil {
return nil, fmt.Errorf("autonomy_setting %s: %w", key, err)
}
r.AutonomySettings++
}
return r, nil
}
// --- Helpers ---
// getOrCreateEntityID returns the UUID for a slug, generating a new
// time-ordered UUIDv7 if the slug doesn't exist yet (ADR-0005).
func getOrCreateEntityID(ctx context.Context, tx pgx.Tx, slug string) (uuid.UUID, error) {
var id uuid.UUID
err := tx.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", slug).Scan(&id)
switch {
case err == nil:
return id, nil
case errors.Is(err, pgx.ErrNoRows):
return uuid.NewV7()
default:
return uuid.Nil, fmt.Errorf("lookup slug %s: %w", slug, err)
}
}
// getEntityTypeBySlug resolves a slug to its entity type name.
func getEntityTypeBySlug(ctx context.Context, tx pgx.Tx, slug string) (string, error) {
var t string
err := tx.QueryRow(ctx, "SELECT type FROM entities WHERE slug = $1", slug).Scan(&t)
if err != nil {
return "", fmt.Errorf("resolve type of %s: %w", slug, err)
}
return t, nil
}
// getEntityIDBySlug resolves a slug to its UUID.
func getEntityIDBySlug(ctx context.Context, tx pgx.Tx, slug string) (uuid.UUID, error) {
var id uuid.UUID
err := tx.QueryRow(ctx, "SELECT id FROM entities WHERE slug = $1", slug).Scan(&id)
if err != nil {
return uuid.Nil, fmt.Errorf("resolve slug %s: %w", slug, err)
}
return id, nil
}
// insertEntityTypes inserts entity types in dependency order (parents before children).
func insertEntityTypes(ctx context.Context, tx pgx.Tx, types map[string]any, r *SeedResult) error {
// Build a dependency graph and insert in topological order
// Simple approach: insert types with no parent first, then iterate
inserted := make(map[string]bool)
remaining := make(map[string]map[string]any)
for name, raw := range types {
tMap, _ := raw.(map[string]any)
remaining[name] = tMap
}
maxPasses := 10
for pass := 0; pass < maxPasses && len(remaining) > 0; pass++ {
for name, tMap := range remaining {
parent, _ := tMap["parent"].(string)
if parent == "" || inserted[parent] {
if err := insertOneEntityType(ctx, tx, name, tMap); err != nil {
return err
}
inserted[name] = true
delete(remaining, name)
r.EntityTypes++
}
}
}
if len(remaining) > 0 {
return fmt.Errorf("circular or missing parent in entity types: %v", keysOf(remaining))
}
return nil
}
func insertOneEntityType(ctx context.Context, tx pgx.Tx, name string, tMap map[string]any) error {
parent, _ := tMap["parent"].(string)
isAbstract, _ := tMap["abstract"].(bool)
domain, _ := tMap["domain"].(string)
layer, _ := tMap["layer"].(string)
desc, _ := tMap["description"].(string)
lifecycleID, _ := tMap["lifecycle"].(string)
attrSchema := tMap["attribute_schema"]
schemaBytes, _ := json.Marshal(attrSchema)
_, err := tx.Exec(ctx,
`INSERT INTO entity_types (name, parent_type, is_abstract, domain, layer, description,
lifecycle_id, attribute_schema, schema_version, status, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 1, 'active', now(), now())
ON CONFLICT (name) DO UPDATE SET parent_type = $2, is_abstract = $3, domain = $4,
layer = $5, description = $6, lifecycle_id = $7, attribute_schema = $8, updated_at = now()`,
name, nullableStr(parent), isAbstract, domain, layer, desc, nullableStr(lifecycleID), nullableStr(string(schemaBytes)))
return err
}
func toStringSlice(v any) []string {
if v == nil {
return nil
}
switch s := v.(type) {
case []string:
return s
case []any:
out := make([]string, 0, len(s))
for _, item := range s {
if str, ok := item.(string); ok {
out = append(out, str)
}
}
return out
}
return nil
}
func nullableStr(s string) any {
if s == "" {
return nil
}
return s
}
func keysOf(m map[string]map[string]any) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
return keys
}

View File

@@ -0,0 +1,53 @@
package db
import (
"strings"
"testing"
)
func nonEmpty(stmts []string) []string {
var out []string
for _, s := range stmts {
if strings.TrimSpace(s) != "" {
out = append(out, s)
}
}
return out
}
func TestSplitSQLBasic(t *testing.T) {
stmts := nonEmpty(splitSQL("CREATE TABLE a (id int); CREATE TABLE b (id int);"))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}
func TestSplitSQLDollarQuotedFunction(t *testing.T) {
sql := `CREATE FUNCTION f() RETURNS int AS $$
SELECT 1; SELECT 2;
$$ LANGUAGE sql;
CREATE TABLE t (id int);`
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
if !strings.Contains(stmts[0], "SELECT 1; SELECT 2;") {
t.Errorf("dollar-quoted body was split: %q", stmts[0])
}
}
func TestSplitSQLTaggedDollarQuote(t *testing.T) {
sql := `DO $body$ BEGIN PERFORM 1; END $body$;SELECT 1;`
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}
func TestSplitSQLSemicolonInComment(t *testing.T) {
sql := "-- comment with ; semicolon\nCREATE TABLE t (id int); -- trailing; note\nSELECT 1;"
stmts := nonEmpty(splitSQL(sql))
if len(stmts) != 2 {
t.Fatalf("got %d statements, want 2: %#v", len(stmts), stmts)
}
}

32
internal/db/sqlcgen/db.go Normal file
View File

@@ -0,0 +1,32 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
package sqlcgen
import (
"context"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
type DBTX interface {
Exec(context.Context, string, ...interface{}) (pgconn.CommandTag, error)
Query(context.Context, string, ...interface{}) (pgx.Rows, error)
QueryRow(context.Context, string, ...interface{}) pgx.Row
}
func New(db DBTX) *Queries {
return &Queries{db: db}
}
type Queries struct {
db DBTX
}
func (q *Queries) WithTx(tx pgx.Tx) *Queries {
return &Queries{
db: tx,
}
}

View File

@@ -0,0 +1,254 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
// source: entities.sql
package sqlcgen
import (
"context"
"time"
"github.com/google/uuid"
)
const getEntityByID = `-- name: GetEntityByID :one
SELECT e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at FROM entities e WHERE e.id = $1
`
// Entity read + mutation queries (API paths). Aliased `e` throughout to
// avoid ambiguity with joined tables.
func (q *Queries) GetEntityByID(ctx context.Context, id uuid.UUID) (Entity, error) {
row := q.db.QueryRow(ctx, getEntityByID, id)
var i Entity
err := row.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
const getEntityBySlug = `-- name: GetEntityBySlug :one
SELECT e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at FROM entities e WHERE e.slug = $1
`
func (q *Queries) GetEntityBySlug(ctx context.Context, slug string) (Entity, error) {
row := q.db.QueryRow(ctx, getEntityBySlug, slug)
var i Entity
err := row.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
const insertEntity = `-- name: InsertEntity :one
INSERT INTO entities (id, slug, type, name, state, attributes)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, slug, type, name, state, attributes, maintenance_until, version, created_at, updated_at
`
type InsertEntityParams struct {
ID uuid.UUID
Slug string
Type string
Name string
State *string
Attributes []byte
}
func (q *Queries) InsertEntity(ctx context.Context, arg InsertEntityParams) (Entity, error) {
row := q.db.QueryRow(ctx, insertEntity,
arg.ID,
arg.Slug,
arg.Type,
arg.Name,
arg.State,
arg.Attributes,
)
var i Entity
err := row.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}
const listEntities = `-- name: ListEntities :many
WITH RECURSIVE tt AS (
SELECT name FROM entity_types WHERE $7::text IS NULL OR name = $7
UNION
SELECT et.name FROM entity_types et JOIN tt ON et.parent_type = tt.name
WHERE $7::text IS NOT NULL
)
SELECT e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at FROM entities e
JOIN entity_types et ON et.name = e.type
WHERE e.type IN (SELECT name FROM tt)
AND ($1::text IS NULL OR e.state = $1)
AND ($2::text IS NULL OR et.domain = $2)
AND ($3::text IS NULL OR et.layer = $3)
AND ($4::text IS NULL
OR e.slug ILIKE '%'||$4||'%'
OR e.name ILIKE '%'||$4||'%')
AND ($5::text IS NULL OR e.slug > $5)
ORDER BY e.slug
LIMIT $6
`
type ListEntitiesParams struct {
State *string
Domain *string
Layer *string
Q *string
Cursor *string
Lim int32
Type *string
}
func (q *Queries) ListEntities(ctx context.Context, arg ListEntitiesParams) ([]Entity, error) {
rows, err := q.db.Query(ctx, listEntities,
arg.State,
arg.Domain,
arg.Layer,
arg.Q,
arg.Cursor,
arg.Lim,
arg.Type,
)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Entity
for rows.Next() {
var i Entity
if err := rows.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listEntitiesCapped = `-- name: ListEntitiesCapped :many
SELECT e.id, e.slug, e.type, e.name, e.state, e.attributes, e.maintenance_until, e.version, e.created_at, e.updated_at FROM entities e ORDER BY e.slug LIMIT $1
`
func (q *Queries) ListEntitiesCapped(ctx context.Context, limit int32) ([]Entity, error) {
rows, err := q.db.Query(ctx, listEntitiesCapped, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Entity
for rows.Next() {
var i Entity
if err := rows.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const updateEntity = `-- name: UpdateEntity :one
UPDATE entities SET
name = COALESCE($1, name),
state = COALESCE($2, state),
attributes = COALESCE($3, attributes),
maintenance_until = CASE WHEN $4::bool
THEN $5 ELSE maintenance_until END,
version = version + 1,
updated_at = now()
WHERE id = $6 AND version = $7
RETURNING id, slug, type, name, state, attributes, maintenance_until, version, created_at, updated_at
`
type UpdateEntityParams struct {
Name *string
State *string
Attributes []byte
SetMaintenance bool
MaintenanceUntil *time.Time
ID uuid.UUID
Version int32
}
func (q *Queries) UpdateEntity(ctx context.Context, arg UpdateEntityParams) (Entity, error) {
row := q.db.QueryRow(ctx, updateEntity,
arg.Name,
arg.State,
arg.Attributes,
arg.SetMaintenance,
arg.MaintenanceUntil,
arg.ID,
arg.Version,
)
var i Entity
err := row.Scan(
&i.ID,
&i.Slug,
&i.Type,
&i.Name,
&i.State,
&i.Attributes,
&i.MaintenanceUntil,
&i.Version,
&i.CreatedAt,
&i.UpdatedAt,
)
return i, err
}

View File

@@ -0,0 +1,329 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
package sqlcgen
import (
"time"
"github.com/google/uuid"
)
type AgentActivity struct {
ID int64
Ts time.Time
AgentID uuid.UUID
SessionID *string
ActivityType string
ToolName *string
EntityID *uuid.UUID
InputSummary *string
OutputSummary *string
DurationMs *int32
TokenCount *int32
Success *bool
CorrelationID *string
}
type Approval struct {
EntityID uuid.UUID
SubjectEntityID *uuid.UUID
Action string
RiskClass string
Kind string
Payload []byte
Status string
TokenHash *string
ExpiresAt time.Time
DecidedAt *time.Time
DecidedBy *uuid.UUID
CreatedAt time.Time
}
type ApprovalRule struct {
ID uuid.UUID
EntityType *string
Action string
RiskClass string
AutonomyLevel string
ScopeEntity *uuid.UUID
Version int32
UpdatedAt time.Time
}
type AuditLog struct {
ID int64
Ts time.Time
ActorType string
ActorID *uuid.UUID
Action string
EntityID *uuid.UUID
Method *string
Path *string
StatusCode *int32
Detail []byte
SourceIp *string
CorrelationID *string
}
type AutonomySetting struct {
Key string
Value string
Version int32
UpdatedAt time.Time
}
type CheckDef struct {
EntityID uuid.UUID
TargetID *uuid.UUID
TargetType *string
Kind string
Config []byte
IntervalS int32
TimeoutS int32
Zone *string
Enabled bool
UpdatedAt time.Time
}
type Classification struct {
EntityID uuid.UUID
SignalEntityID *uuid.UUID
TargetEntityID *uuid.UUID
Action string
RecommendedAction []byte
RiskClass string
Route string
BlastRadius []uuid.UUID
PatternConfidence *float32
SkillID *uuid.UUID
AutonomyCheck *string
Reasoning []byte
CorrelationID string
CreatedAt time.Time
}
type Entity struct {
ID uuid.UUID
Slug string
Type string
Name string
State *string
Attributes []byte
MaintenanceUntil *time.Time
Version int32
CreatedAt time.Time
UpdatedAt time.Time
}
type EntityStatus struct {
EntityID uuid.UUID
Health string
LastCheckAt *time.Time
Details []byte
UpdatedAt time.Time
}
type EntityType struct {
Name string
ParentType *string
IsAbstract bool
Domain string
Layer string
Description *string
LifecycleID *string
AttributeSchema []byte
SchemaVersion int32
Status string
CreatedAt time.Time
UpdatedAt time.Time
}
type Event struct {
ID int64
Ts time.Time
Type string
EntityID *uuid.UUID
Severity string
Source string
Data []byte
CorrelationID *string
}
type Execution struct {
EntityID uuid.UUID
ClassificationID *uuid.UUID
SignalEntityID *uuid.UUID
TargetEntityID *uuid.UUID
Action string
RiskClass string
ApprovalID *uuid.UUID
AgentID *uuid.UUID
SkillID *uuid.UUID
SkillVersion *int32
Status string
Result []byte
DurationMs *int32
Verified bool
CorrelationID string
StartedAt *time.Time
CompletedAt *time.Time
CreatedAt time.Time
}
type Feedback struct {
EntityID uuid.UUID
ExecutionID uuid.UUID
Outcome string
Observation *string
Lesson *string
UnexpectedSideEffects []string
Tags []string
CreatedAt time.Time
}
type IdempotencyKey struct {
Key string
Actor string
RequestHash string
ResponseCode *int32
ResponseBody []byte
CreatedAt time.Time
}
type Ledger struct {
Ts time.Time
ExecutionID uuid.UUID
TargetEntityID *uuid.UUID
Action string
RiskClass string
Status string
Verified bool
Route *string
Reasoning []byte
ApprovalStatus *string
DecidedBy *uuid.UUID
AgentID *uuid.UUID
CorrelationID string
}
type LifecycleDef struct {
ID string
States []string
DefaultState string
TerminalStates []string
Transitions []byte
CreatedAt time.Time
}
type MetricRollups1d struct {
Bucket interface{}
EntityID uuid.UUID
Metric string
AvgValue float64
MinValue interface{}
MaxValue interface{}
SampleCount int64
}
type MetricRollups1h struct {
Bucket interface{}
EntityID uuid.UUID
Metric string
AvgValue float64
MinValue interface{}
MaxValue interface{}
SampleCount int64
}
type MetricSample struct {
Ts time.Time
EntityID uuid.UUID
Metric string
Value float64
Tags []byte
}
type Pattern struct {
EntityID uuid.UUID
AppliesType string
Action string
Pattern string
Confidence float32
EvidenceCount int32
SuccessCount int32
FailureCount int32
Status string
Quarantined bool
Version int32
LastValidatedAt *time.Time
CreatedAt time.Time
}
type Relationship struct {
SourceID uuid.UUID
TargetID uuid.UUID
Type string
Attributes []byte
ValidFrom time.Time
ValidTo *time.Time
}
type RelationshipType struct {
Name string
Inverse *string
SourceType string
TargetType string
Cardinality string
Description *string
CreatedAt time.Time
}
type RiskClass struct {
Name string
Description *string
ApprovalRequired string
AutonomyAllowed bool
}
type SeedVersion struct {
File string
ContentHash string
AppliedAt time.Time
}
type Signal struct {
EntityID uuid.UUID
Kind string
Severity string
TargetEntityID *uuid.UUID
CheckID *uuid.UUID
Evidence *string
LikelyCause *string
State string
OccurrenceCount int32
FirstSeenAt time.Time
LastSeenAt time.Time
FlapCount int32
HoldDownUntil *time.Time
MuteUntil *time.Time
CreatedAt time.Time
UpdatedAt time.Time
}
type Skill struct {
EntityID uuid.UUID
Version int32
Name string
Procedure []byte
AppliesType *string
Action string
PatternIds []uuid.UUID
Status string
SuccessRate *float32
ChangedBy *uuid.UUID
ChangeReason *string
LastUsedAt *time.Time
CreatedAt time.Time
}

View File

@@ -0,0 +1,130 @@
// Code generated by sqlc. DO NOT EDIT.
// versions:
// sqlc v1.29.0
// source: ontology.sql
package sqlcgen
import (
"context"
)
const getLifecycleForType = `-- name: GetLifecycleForType :one
SELECT ld.id, ld.states, ld.default_state, ld.terminal_states, ld.transitions, ld.created_at FROM lifecycle_defs ld
JOIN entity_types et ON et.lifecycle_id = ld.id
WHERE et.name = $1
`
func (q *Queries) GetLifecycleForType(ctx context.Context, name string) (LifecycleDef, error) {
row := q.db.QueryRow(ctx, getLifecycleForType, name)
var i LifecycleDef
err := row.Scan(
&i.ID,
&i.States,
&i.DefaultState,
&i.TerminalStates,
&i.Transitions,
&i.CreatedAt,
)
return i, err
}
const listEntityTypes = `-- name: ListEntityTypes :many
SELECT name, parent_type, is_abstract, domain, layer, description, lifecycle_id, attribute_schema, schema_version, status, created_at, updated_at FROM entity_types ORDER BY name
`
func (q *Queries) ListEntityTypes(ctx context.Context) ([]EntityType, error) {
rows, err := q.db.Query(ctx, listEntityTypes)
if err != nil {
return nil, err
}
defer rows.Close()
var items []EntityType
for rows.Next() {
var i EntityType
if err := rows.Scan(
&i.Name,
&i.ParentType,
&i.IsAbstract,
&i.Domain,
&i.Layer,
&i.Description,
&i.LifecycleID,
&i.AttributeSchema,
&i.SchemaVersion,
&i.Status,
&i.CreatedAt,
&i.UpdatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listLifecycleDefs = `-- name: ListLifecycleDefs :many
SELECT id, states, default_state, terminal_states, transitions, created_at FROM lifecycle_defs ORDER BY id
`
func (q *Queries) ListLifecycleDefs(ctx context.Context) ([]LifecycleDef, error) {
rows, err := q.db.Query(ctx, listLifecycleDefs)
if err != nil {
return nil, err
}
defer rows.Close()
var items []LifecycleDef
for rows.Next() {
var i LifecycleDef
if err := rows.Scan(
&i.ID,
&i.States,
&i.DefaultState,
&i.TerminalStates,
&i.Transitions,
&i.CreatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const listRelationshipTypes = `-- name: ListRelationshipTypes :many
SELECT name, inverse, source_type, target_type, cardinality, description, created_at FROM relationship_types ORDER BY name
`
func (q *Queries) ListRelationshipTypes(ctx context.Context) ([]RelationshipType, error) {
rows, err := q.db.Query(ctx, listRelationshipTypes)
if err != nil {
return nil, err
}
defer rows.Close()
var items []RelationshipType
for rows.Next() {
var i RelationshipType
if err := rows.Scan(
&i.Name,
&i.Inverse,
&i.SourceType,
&i.TargetType,
&i.Cardinality,
&i.Description,
&i.CreatedAt,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}

Some files were not shown because too many files have changed in this diff Show More