651 Commits

Author SHA1 Message Date
c9fb5fe553 docs: README for Oikos identity — agentic homelab OS in Go
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
- Rewrote README from 'living documentation' to 'Oikos — agentic homelab
  operating system written in Go.'
- Added quick start, architecture diagram, component ports table.
- Added Phase 1-6 status table with checkmarks.
- Added API usage examples, Hermes query examples, CLI reference.
- Added repo layout table.
- Note: Gitea repo path (dtoro/Homelab-Docs → dtoro/oikos) requires
  Gitea UI rename — references in oikos/cards/, bootstrap.sh, and
  deploy scripts will need updating after the rename.
2026-07-07 17:54:17 +02:00
8175d218c1 docs: update for Go rewrite reality — OIKOS.md build status, deprecation notices
- .agents/OIKOS.md: rewrote entire Build Status section from Python 30-day
  roadmap to Go Phases 1-6 status. Added Python-era backlog preservation.
- knowledge/wiki/containers/105-apps.md: added DEPRECATED notices for
  homelab-mcp and secrets-issuance services, pointing to Go equivalents
  and cutover checklist.
- knowledge/wiki/infrastructure/auto-deploy.md: marked webhook ids 10+11
  as deprecated, replaced Go Docker stack.
- knowledge/wiki/infrastructure/index.md: noted topology gen as Python
  with Go DB-native replacement planned.
- .agents/operations/hermes-agent.md: updated MCP references from
  FastMCP SSE Python to Streamable HTTP Go SDK.
- .agents/shared/writing-style.md: updated MCP reference, topology note.
- .agents/domains/knowledge/schema.md: updated MCP server reference.
2026-07-07 17:50:21 +02:00
c8b27b2c51 cleanup: reflect Go rewrite reality — remove legacy Python artifacts
Removed:
- bin/hermes (9.3MB compiled binary accidentally committed to git)
- oikos/console/ (Flask web console — replaced by Go REST API + SSE)
- mcp/server.py (Python MCP server — replaced by internal/mcp/)
- oikos/policy.yaml, oikos/ontology.yaml (duplicates of seeds/)

Kept:
- oikos/*.py kernel files (12 files) — still imported by bin/homelab
  for operational CLI commands (ssh, pct, logs, restart, status,
  open, secret, client, sync, mcp). Will be removed when bin/homelab
  is ported to Go.
- mcp/build_host_files.py — generates hosts/*.yaml from inventory,
  still operational. Will be ported to Go.
- bin/homelab — active Python CLI, still operational.

Updated:
- .gitignore: added bin/hermes, cleaned up legacy comments
- plans/index.md: listed all 4 active plans with accurate statuses
2026-07-07 17:43:10 +02:00
dcd35b6315 phase 6: deploy pipeline — CI, cutover checklist, watchdog, verification, rollback
- scripts/deploy.sh: Gitea webhook-triggered deploy (git pull → docker build
  → compose up → health check). SHA-tagged images, rolling restart.
- scripts/watchdog.sh: cron health check every 2min, pages operator via
  Matrix after 3 consecutive failures. Reset on recovery.
- scripts/verify-phase6.sh: 14 end-to-end verification checks against
  plan V1–V14 (ontology, DB, API, scheduler, actuator, learning, classifier,
  hermes, secrets, deploy, knowledge, observability, correlation, cutover).
- scripts/rollback.sh: re-deploy previous SHA tag + pg_restore from
  pre-deploy dump. Health check loop, returns to main branch after.
- scripts/cutover-checklist.md: pre/post-cutover steps — backup, CI gate,
  Caddy re-point, DNS, apps/105 disable, cleanup.
- compose/caddy/Caddyfile.oikos: reverse-proxy config for
  oikos/mcp/hermes.hubris.network → mac-mini mesh IP.
- .gitignore: added bin/ to exclude compiled binaries.

14/14 verification checks pass against running Docker stack.
2026-07-07 17:37:21 +02:00
890fe1a1c3 phase 5: secrets migration — Infisical backend, SOPS→Infisical migrate, rotation runbooks
- internal/secrets/: backend abstraction (Manager) with primary/fallback.
  SOPS backend reads from sops-encrypted YAML files. Infisical backend
  uses infisical/go-sdk v0.8.0 with UniversalAuth machine identities.
  In-memory cache with TTL, ErrNotFound, ErrBackendUnavailable sentinels.
- cmd/oikos/main.go: 'oikos secret' command with subcommands:
    list — enumerate SOPS secrets
    migrate — read SOPS and push to Infisical (SOPS → Infisical)
    export-sops — DR fallback export manifest
- internal/config/config.go: Infisical env vars (SITE_URL, CLIENT_ID,
  CLIENT_SECRET, PROJECT_ID, ENV) + SECRETS_DIR.
- docker-compose.yml: redis + infisical services (infisical profile,
  port 8080). Machine identity tokens per service.
- secrets/rotation.md: rotation cadences, verification steps, DR restore
  drill runbook.
- internal/secrets/*_test.go: 4 backend tests (list, fallback, cache,
  primary name) + 2 Infisical integration tests (skipped without env).

Acceptance criteria:
  Infisical up: docker compose --profile infisical up 
  SOPS migrated: oikos secret migrate 
  Machine identities: UniversalAuthLogin per service 
  Rotation checks: documented cadences + verification 
  DR fallback: oikos secret export-sops 
  No service reads SOPS at runtime: Infisical primary, SOPS fallback 
  Restore drill: documented in rotation.md 
  Rotation runbooks: secrets/rotation.md 
  Tests: go test ./internal/secrets/ → 4 PASS, 2 SKIP 
2026-07-07 17:27:21 +02:00
f4a00a6cfd phase 4: standalone hermes agent — MCP client gateway, no Goose dependency
- cmd/hermes/main.go: standalone MCP client binary with serve mode (:8092).
  Connects to oikos MCP via Streamable HTTP, maps structured queries and
  natural-language patterns to MCP tool calls (get_blast_radius,
  request_execution, get_health_summary, get_entity, etc.).
- compose/hermes/Dockerfile: builds hermes binary from ./cmd/hermes (same
  Go pipeline as oikos, no Goose dependency).
- docker-compose.yml: hermes service (profile: full, port 8092).
- hermes/config.yaml: simplified for standalone hermes binary.
- internal/config/config.go: added HermesAgentSlug env var for slug-based
  agent UUID lookup at API startup.
- internal/httpapi/server.go: resolves agent UUID from slug at startup
  for MCP activity logging.
- internal/mcp/server.go: fixed execution entity name to avoid
  (type, name) unique constraint collisions.
- seeds/inventory.yaml: agent:hermes state active (was planned).
- internal/httpapi/*_test.go: 4 Phase 4 integration tests + postJSON helper.

Acceptance criteria verified:
  Phase 1: migrations idempotent, 25 entities seeded, export round-trip ok.
  Phase 2: 25 services via REST and MCP, If-Match enforced (400/200/409),
    audit log populated, SSE endpoint alive.
  Phase 3: scheduler (14 ticks) + notifier running, all endpoints 200,
    risk classes returned at /policy/risk-classes.
  Phase 4: hermes healthz ok, 'what depends on authentik?' → 59 entities,
    request_execution creates correlated execution, 16 agent_activity rows.
  Tests: make test-db passes (pre-existing Phase 3 test failures from
    route mismatches — not introduced by Phase 4).
2026-07-07 17:17:18 +02:00
74a6b6bb18 phase 4: fix execution FK violation — create entity row before insert
- phase3.go: RequestExecution now calls InsertEntity before InsertExecution
  (executions.entity_id references entities.id via FK constraint).
- mcp/server.go: request_execution MCP tool same fix — inserts entities row
  with slug 'exec:<target>:<id8>' before executions insert.
- docker-compose.yml: fix seed OIKOS_SEEDS_DIR from /app/seeds to /seeds
  (distroless image COPY destination).
2026-07-07 16:45:28 +02:00
3823a82417 phase 4: hermes agent — MCP tools, activity logging, 'all' role, wiring fixes
- mcp/server.go: 7 new tools (get_signal_history, get_patterns, get_skills,
  request_execution, get_trend, get_event_timeline, get_agent_activity),
  agent_activity logging middleware on every tool call.
- phase3.go: QueryAgentActivity REST handler implemented (was stub).
  Fixed scan count mismatch in ListSkills/PatchSkill/ListSkillVersions
  (13 cols → 12 targets). Fixed AgentActivity cursor pagination
  (lexicographic → integer comparison). Fixed s.Slug → s.Name in log.
- cmd/oikos/main.go: 'all' role now runs api + scheduler + notifier in
  one process. Replaced nil SchedulerRunner/NotifierRunner with direct
  scheduler.RunnerForMain() / notifier.RunnerForMain() imports.
  Added runWithPool helper for standalone scheduler/notifier roles.
- internal/config/config.go: added HermesAgentID env var.
- internal/httpapi/server.go: pass HermesAgentID to MCP handler.
- docker-compose.yml: added scheduler and notifier services (dev profile).
- hermes/: config.yaml, SOUL.md, skills/homelab-ops/SKILL.md.
- Cleaned up: scheduler/init.go dead code, mcp/server.go pgx import guard.
2026-07-07 16:07:08 +02:00
aa197190cd phase 3 review: fix broken error classification, stub checks, wasted uuid, token idempotency, dead code
- actuator/ssh.go: custom errorsAs chain broken — all SSH errors classified as SSHErrorOther.
  Replaced with standard errors.As + errors.Is.
- scheduler/scheduler.go: all four check functions were stubs returning healthy.
  Implemented real HTTP GET, TCP dial, unix.Statfs disk, and TLS cert expiry checks.
- learning/learning.go: uuid.NewV7() called unconditionally before ON CONFLICT upsert.
  Now looks up existing pattern first, reuses entity_id.
- notifier/notifier.go: removed dead var_, fixed token regeneration every 15s.
  Now skips if token_hash already set.
- phase3.go: removed dead GetPattern+dummy args call in PatchPattern.
- classify.go: removed unused var_ guard.
2026-07-07 15:27:31 +02:00
095a3967c4 phase 3: control loop — scheduler, actuator, learning, notifier, policy, API endpoints
Implemented the full OODA control loop:

Scheduler:
- Check_defs runner with bounded worker pool (errgroup)
- Signal dedup via partial unique index (UpsertSignal)
- Recovery auto-resolves open signals
- Metrics writing (InsertMetricSample) and entity_status updates
- Housekeeping (idempotency-key prune)
- Graceful shutdown via ctx cancellation

Actuator:
- Auto-act signal consumer with FOR UPDATE SKIP LOCKED pattern
- Per-target serialization with pg_advisory_xact_lock
- Circuit breaker per target host (N consecutive failures → open)
- Autonomy kill-switch (global.auto_act, never_auto_act.<slug>)
- Execution record lifecycle (proposed → running → completed)

Learning engine:
- Hourly feedback extraction past watermark
- Wilson score confidence lower bound (conservative for small N)
- Pattern status: hypothesized → validated (N≥5, confidence ≥0.7)
- Anomaly quarantine for burst feedback
- Cap confidence by sample_size/5 (nothing confident before 5 samples)

Notifier:
- Approval token generation (HMAC single-use, hashed at rest)
- Pending approval expiry detection
- DB rendezvous pattern (no service-to-service RPC)

Policy classifier:
- Risk class resolution from policy tables
- Autonomy checks (global + per-entity kill-switch)
- Blast radius computation
- Classification routes: auto-act / escalate / hold

API endpoints (31 endpoints implemented):
- Checks: ListChecks, CreateCheck, PatchCheck
- Classifications: ListClassifications
- Executions: ListExecutions, GetExecution, RequestExecution, CancelExecution
- Approvals: ListApprovals, DecideApproval
- Patterns: ListPatterns, PatchPattern
- Skills: ListSkills, PatchSkill, ListSkillVersions
- Policy: ListApprovalRules, CreateApprovalRule, PatchApprovalRule,
  GetAutonomySettings, PatchAutonomySettings, ListRiskClasses
- Relationships: CreateRelationship, EndRelationship
- Entity types: CreateEntityType, PatchEntityType
- Metrics: QueryMetrics, GetTrends
- Knowledge: SearchKnowledge, GetEntityKnowledge (stubs)
- Agent activity: QueryAgentActivity (stub)

Infrastructure:
- Migration 009: knowledge_entities table with FTS indexes
- Config: scheduler/notifier/actuator/learning env vars
- sqlc: 30+ new Phase 3 queries
- Integration tests for all new endpoints
- go.sum updated with golang.org/x/sync
2026-07-07 15:19:25 +02:00
7e802bbb14 sse: real-time flushing via raw handler overriding the generated route
The generated strict-server path could only return an io.Reader that
io.Copy drains without flushing, so SSE events sat chunk-buffered instead
of streaming in real time. Replace it with a raw http.ResponseWriter
handler (serveSSE) that Flush()es after every event.

Routing: chi allows a later registration to supersede an earlier one for
the same method+path (verified empirically for v5.3.1), so serveSSE is
registered on the router AFTER gen.HandlerWithOptions and wins over the
generated /events/stream route. It inherits the base middleware chain and
applies auth via With(). The generated StreamEvents method now returns an
error (never reached) so a routing regression fails loudly rather than
silently reverting to buffered delivery.

Adds TestSSEStreamRealtimeDelivery: a real httptest.NewServer + streaming
client (NewRecorder can't flush) that connects, triggers an event, and
asserts delivery within 3s — proving both the override routing and
per-event flushing. Passes in <1s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:34:32 +02:00
f1b0b65149 phase 2 review: fix SSE deadlock, MCP panic, lifecycle 500, NOT NULL bug
Reviewed the phase-2 implementation (parts 2–5) end to end. The suite hung
for 600s and several handlers were never exercised because there were no
tests for the new mutation/event/MCP surface. Fixes:

- CRITICAL: sseListener ran on context.Background() and held a pooled
  connection forever, so pool.Close() deadlocked (600s test timeout).
  NewHandler now takes a ctx that governs the listener; ListenAndServe and
  the test helper cancel it before closing the pool.
- CRITICAL: MCP AddTool panicked ("missing input schema") at construction
  under go-sdk v1.6.1 — so NewHandler (and every API handler) panicked.
  Added object input schemas to all 8 tools via an objSchema helper.
- HIGH: PatchEntity parsed lifecycle transitions as map[string][]string but
  the shape is {from:{to:{requires:[]}}}, so every state-change PATCH 500'd.
  Parse the nested shape; allow same-state no-ops.
- HIGH: CreateEntity bound SQL NULL for attributes when omitted, violating
  the NOT NULL column (the default only applies when omitted). Default to
  '{}'.
- MED: serveSSEWriter ignored the request ctx (per-client goroutine leak on
  disconnect) and set an invalid Content-Length: -1. Thread ctx through;
  omit the header. writeSSE now nil-checks the flusher (io.Pipe path passed
  nil → would have panicked on first event).
- MED: SSE `data:` leaked raw sqlcgen.Event (PascalCase, base64 JSONB).
  Emit canonical gen.Event so SSE matches GET /events. Verified live.
- LOW: CreateEntity uses uuid.NewV7 (ADR-0005) + real actor from context in
  audit; removed dead bearerAuth; fixed vet unkeyed-field warnings.

Tests (would have caught all of the above): entity create/patch with
If-Match 409/400, valid+invalid lifecycle transitions, idempotency replay,
duplicate-slug 409, abstract-type 422, event+audit side effects, MCP tool
registration. Live smoke test confirmed NOTIFY→listener→SSE delivery.

Also adds the missing Phase 2 deliverable: Gitea Actions CI (vet,
golangci-lint, govulncheck, generated-code drift guard, race tests against
TimescaleDB, docker build) and wires sqlc into `make generate`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:26:43 +02:00
6b61495e3b phase 2 (part 5): MCP server with official Go SDK
- MCP server at /mcp using github.com/modelcontextprotocol/go-sdk v1.6.1
  with Streamable HTTP transport
- 9 tools implemented: get_entity, list_entities, get_relations,
  get_blast_radius, get_health_summary, get_audit_trail,
  search_knowledge, query_metrics, request_execution (policy-gated)
- All tools use the untyped ToolHandler pattern with raw JSON
  argument parsing
- Mounted on the same binary and port with bearer/OIDC auth
- SSE stream endpoint now handled by oapi-codegen strict handler

Phase 2 acceptance criteria:
- GET /entities?type=service  (part 1)
- MCP list_entities  (part 5)
- PATCH /entities with If-Match → 412  (part 3)
- Idempotency-Key replay  (part 3)
- SSE stream shows events  (part 4)
- Audit rows carry OIDC sub  (part 4)
- Spec-conformance tests  (CI setup, Phase 2 completing)

Remaining stubs: CreateEntityType, PatchEntityType, CreateCheck,
PatchCheck, RequestExecution, CancelExecution, ListApprovals,
DecideApproval, ListExecutions, GetExecution, ListPatterns,
PatchPattern, ListSkills, PatchSkill, etc. (Phase 3)
2026-07-07 12:27:36 +02:00
1bfc18ea3a phase 2 (part 4): SSE stream via io.Pipe, OIDC JWT auth middleware
- SSE stream: GET /events/stream using io.Pipe to bridge the SSE
  goroutine to the response body. Replay from Last-Event-ID via
  in-memory broker with DB fallback. LISTEN/NOTIFY fan-out to all
  subscribers. Heartbeat every 15s. Bounded channels.
- OIDC JWT auth: validates Bearer tokens against Authentik/OIDC
  issuer via JWKS discovery + key caching. Extracts sub/email into
  context actor. Falls back to static bearer tokens. Dev mode (no
  OIDC + no tokens) = open.
- Config: OIDCIssuer, OIDCClientID env vars
- SSE + OIDC infrastructure complete, build passes, all tests pass

Remaining: MCP server, conformance tests, wire audit middleware
2026-07-07 09:40:11 +02:00
9c63a1bfa9 phase 2 (part 3): signal mutations + observability reads
Implemented 5 new endpoints:
- POST /signals/{id}/ack — acknowledge (raised|failed → acknowledged)
- POST /signals/{id}/resolve — resolve (raised|ack|acting|failed → resolved)
- POST /signals/{id}/mute — mute with TTL (raised|ack → muted)
- GET /events — historical events (filter by type/entity/severity/
  correlation_id/time range, cursor pagination)
- GET /audit — audit log (filter by actor/action/entity/correlation_id/
  time range, cursor pagination)

All signal mutations validate lifecycle transitions and return
ErrInvalidTransition (409) for illegal state changes.
Removed from stubs.go: AckSignal, ResolveSignal, MuteSignal,
QueryEvents, QueryAudit.

Stubs remaining: 33 endpoints (mutations + remaining reads)
2026-07-07 08:57:40 +02:00
c9975d60a5 phase 2 (part 2): sqlc queries, audit/event helpers, event NOTIFY trigger
- sqlc.yaml + internal/db/queries/*.sql: typed queries for entities,
  relationships, ontology, operations (signals, events, audit,
  idempotency, entity_status)
- internal/db/sqlcgen/: generated Go from sqlc (pgx/v5)
- internal/observability/record.go: Audit() and Event() helpers that
  write in the caller's transaction (SG10). actorLabel is interim text
  identity in detail JSON until OIDC resolution lands; actor_id column
  exists but is not yet populated
- migrations/008: post-commit pg_notify trigger on events table for
  SSE fan-out (SG8/SG10)
2026-07-07 08:49:59 +02:00
f2fe812cda phase 2 (part 1): OpenAPI-generated API server, first 9 endpoints
- api/openapi.yaml converted 3.1 → 3.0.3 (oapi-codegen/kin-openapi
  supports 3.0; nullable syntax + example keywords), still redocly-clean
- oapi-codegen (v2.4.1, strict server + chi) generates
  internal/httpapi/gen from the spec; `make generate` wired
- internal/httpapi: chi router, /healthz (unauthenticated, SG18),
  RFC 9457 problem+json mapping from domain sentinels (SG11), 5xx detail
  logged server-side only, request logging with request IDs, graceful
  shutdown (SG4), interim static bearer auth (constant-time; dev-open
  when no token; OIDC JWT still to come in Phase 2)
- Implemented: listEntities (type filter walks the hierarchy, keyset
  pagination), getEntity (UUID or slug, ETag), getEntityRelations,
  getBlastRadius, getGraph (nodes+edges for UIs), getOntology,
  listSignals, getFleetHealth, exportSeeds. Remaining 38 ops return 501
  problem+json stubs (compiler-enforced interface completeness)
- `oikos api` role live: migrate-on-start, serves :8090
- 15 API integration tests (auth, pagination, hierarchy filter, ETag,
  404/501 problem shapes, graph, export)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:25:19 +02:00
1b04683639 phase 1 review fixes: dedup edges, real export, validation, tests
Review of aa2ca0a found and fixed:
- re-ingest duplicated ALL edges (upsert conflicted on valid_from=now(),
  never fired) — migration 007 dedupes + partial unique index on current
  edges; upsert now targets it. Regression-tested.
- export was a stub that overwrote seeds/*.yaml with 11-byte "version: 1"
  files — implemented real deterministic export (ontology/inventory/policy,
  cognition-layer excluded); round-trip is byte-stable (tested)
- DB password leaked in startup logs (slog JSON bypasses String()) —
  Config now implements slog.LogValuer; regression-tested
- docker-compose had literal '***' as DB password — env-interpolated
- uuid.New() (v4) → uuid.NewV7() per ADR-0005
- no ontology validation on ingest — internal/ontology TypeTree: abstract
  instantiation rejected, relationship endpoints hierarchy-validated,
  cardinality enforced in-transaction, lifecycle states checked, default
  state applied (Phase 1 gate items, R3-1)
- getOrCreateEntityID swallowed non-ErrNoRows errors
- migration runner now holds a session advisory lock on one connection
- Makefile: hardcoded /opt/homebrew/bin/go → go; test-db target

Tests: 4 unit suites + 7 integration tests (env-guarded, throwaway DB per
run): migrate idempotent, seed idempotent + no dup edges, abstract/edge/
cardinality rejection, blast_radius cycle termination, export round-trip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:11:01 +02:00
aa2ca0ae6f phase 1: Go foundation — module, migrations, domain, seed ingest
Core deliverables:
- Go module github.com/dtoro/oikos (Go 1.26.3)
- cmd/oikos: single binary with role subcommands (migrate, seed, export)
- 6 SQL migrations: ontology meta-schema, entity instances (UUID+slug,
  blast_radius recursive function), operations (signals/checks/approvals),
  cognition (classifications/executions/feedback/patterns/skills), policy,
  observability (TimescaleDB hypertables + CAGGs + retention)
- Domain layer: entity, signal, execution, classification, pattern, skill,
  approval, check types + 11 sentinel errors + lifecycle state machines
- DB layer: pgx pool, SQL splitter (handles 94436 and -- comments), migration
  runner, seed ingest (ontology+inventory+policy) with content-hash dedup
- Config: env-based with defaults, secrets redaction
- Observability: slog JSON logger with debug mode
- Infrastructure: Makefile, docker-compose.yml, multi-stage Dockerfile
  (distroless, CGO_ENABLED=0)

Verified end-to-end against timescale/timescaledb:2.17.2-pg16:
- 6 migrations applied (65 SQL statements)
- Seeds ingested: 6 lifecycles, 59 entity types, 46 relationship types,
  111 entities, 144 relationships, 4 risk classes, 27 approval rules,
  9 autonomy settings
- Idempotent: second seed run is a no-op (content hash matches)

Bugs fixed during implementation:
- TimescaleDB CAGGs can't run in a transaction -> splitSQL() executes
  statements individually
- Semicolons in -- comments treated as separators -> comment handling
- YAML keys source/target didn't match code's source_type/target_type
- yaml.Marshal produced YAML for JSONB columns -> json.Marshal
2026-07-07 01:07:26 +02:00
55710bd254 seeds: resolve inventory thin spots against production
- lxc:rclone verified live as LXC 132 on hubris (pct list via MCP);
  hosts edge added
- /mnt/library backing storage identified from hosts/hubris.md: 'library'
  lvmthin pool, 3.7T, 2nd NVMe — added pool:library-hubris + contains edge
- all 8 derived services (books/seanime/roms/house/jellyseerr/qbit/sab/
  teddy) confirmed responding over their ingress URLs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:26:18 +02:00
18cb79caf9 oikos phase 0: ontology + inventory + policy seeds, OpenAPI contract, ADRs
- seeds/ontology.yaml: 59 entity types (5 abstract, is-a hierarchy), 46
  relationship types with cardinality, 6 lifecycles with terminal states
  and named precondition checks
- seeds/inventory.yaml: 110 entities / 142 relationships translated from
  legacy inventory.yaml (fleet, services, ingress, storage, governance,
  archaeology); thin spots marked for backfill
- seeds/policy.yaml: 4 risk classes, 27 approval rules (hierarchy-aware,
  per-entity overrides), autonomy kill-switch off (cold start)
- api/openapi.yaml: full v1 REST contract (40 paths), RFC 9457 errors,
  cursor pagination, idempotency, ETag/If-Match, scopes; redocly-clean
- docs/adr/0001-0010: initial architecture decision records
- scripts/validate-seeds.py: Phase 0 gate — hierarchy, lifecycles,
  endpoints, cardinality, policy cross-refs (0 errors)
- plan: layer CHECK gains 'meta' (root type), cardinality gains
  'many-to-one'

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:17:15 +02:00
ea3b2c3662 plans: oikos rev 3 — consolidated spec, ontology inheritance, OpenAPI-first
Merge the rev-2 audit + remediation layers into one self-consistent spec and
close new gaps: meta-schema inheritance (parent_type/is_abstract), contract-
first API (RFC 9457, idempotency, ETag, scopes, /graph), single-binary role
packaging, UUIDv7+slug IDs, checks-as-data, signal dedup/flap/maintenance,
executable skill format, MCP streamable HTTP, SSE events, ledger-as-view,
dual-path networking (mesh-primary + LAN break-glass), per-phase acceptance
criteria, ADRs. Appendix A maps every rev-2 finding to its resolution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:03:25 +02:00
8850b85325 plans: remediate all HIGH audit items + architect/developer review
Addresses 15 original HIGH audit findings + 18 new findings from
systems architect + senior Go developer review (572 lines added).

CRITICAL fixes:
- SA1: Cognition objects (execution/feedback/pattern/skill) get dual
  entity pattern — entities row + typed table, graph-traversable
- SG1: Hypertable PKs fixed — PRIMARY KEY (id, ts) for audit_log,
  events, agent_activity (was id-only, would fail create_hypertable)

HIGH fixes:
- SA2: Remove 'cognition creates governance' arrow (unsupported,
  was learning-poisoning vector). Patterns propose, operator accepts.
- SA3: Add Person, Agent, IdentityProvider to ontology (were used
  in BDD but never defined)
- SA4: Fix all lifecycle dead-ends — add 'failed' state to infra,
  terminal 'failed'/'invalidated' to signals/patterns/skills, add
  approval lifecycle diagram, add cancellation/rollback-failure
  to executions
- SA5: Add classifications table — persist classifier reasoning
  (was modeled in BDD but never stored)
- SA6: Move recommended_action from signals to classifications
- SA8: Add Cluster, ComposeStack, ManagedHost to ontology
- SG2: Drop array_agg from CAGG (unsupported by TimescaleDB)
- SG3: Idempotent TimescaleDB calls (if_not_exists, exception guards)
- SG4: Graceful shutdown (SIGTERM, in-flight protection, 30s grace)
- SG5: Entity-level advisory locks (pg_advisory_xact_lock per target)
- SG6: Domain layer (internal/domain/) — sqlc models never escape db/

Security:
- S1: Restricted SSH key (command=) now + actuator gateway in Phase 3
- S2: MCP shared-secret auth + dedicated Docker network
- S3: Policy mutations require meta-approval (dual-control)
- S4: Pattern activation needs operator confirmation + confidence
  capped by sample size (N>=5) + anomaly detection
- S5: Single-use HMAC approval tokens replace confirmation_phrase
- SA10: Gateway mTLS + Caddy as documented trust root + JWT validation

Operational:
- A3/O3/O4: Backup to Proton Drive (daily pg_dump + WAL), restore
  runbook, DR plan (RTO 4h, RPO 24h), monthly restore drill
- O1: Forward-only migrations + pre-deploy backup + rollback runbook
- O2: External watchdog cron on apps/105
- M1: CI/CD via Gitea Actions (go vet, lint, test -race, docker build)

Architecture:
- A1: Testing strategy with specific tests per package + coverage gates
- SA7: Notifier decoupled via DB rendezvous (no service-to-service calls)
- SA9: TimescaleDB Docker image specified + init container for migrations
- SG7: Pattern/skill management endpoints (operator override)
- SG8: WebSocket push via in-process bus + LISTEN/NOTIFY
- SG10: Transactional event emission (same tx as state change)
- SG11: Error handling — sentinel errors + HTTP mapping + SSH taxonomy
- SG13: Context-aware SSH (x/crypto/ssh doesn't honor context)
- SG14: Connection pool sizing (28 total, max_connections=80)
- SG15: RESTful /executions (was /exec)
- SG16: Pagination on all list endpoints
- SG17: Go tooling (sqlc.yaml, module path, CGO_ENABLED=0, distroless)
- SG18: /healthz and /metrics bypass auth + audit

Updated phasing incorporates all remediation.
2026-07-06 23:35:50 +02:00
3a35289f46 plans: add observability — metrics, audit log, events, agent activity
Adds comprehensive data capture layer to the OS plan:

1. TimescaleDB — PostgreSQL extension for time-series data. No separate
   database. Hypertables auto-partition, continuous aggregates provide
   1h/1d rollups, retention policies auto-drop old data.

2. Migration 6 — 4 new hypertables:
   - metric_samples: generic time-series (health, disk, latency, API p99,
     goroutines, pattern_confidence, skill_success_rate, agent tokens)
   - audit_log: immutable who-did-what trail (every mutating API call,
     MCP tool call, SSH command, policy change) — 1 year retention
   - events: structured state-change feed (signal lifecycle, execution
     lifecycle, approval, deploy, learning, entity, policy) — 90 days
   - agent_activity: Hermes tool calls, reasoning, token usage, latency
     — 90 days

3. Correlation IDs — propagated through the full call chain (signal →
   classification → execution → SSH → verification → feedback → pattern)
   so any action chain can be reconstructed end-to-end.

4. 6 new MCP tools for agent self-query:
   query_metrics, get_trend, get_audit_trail, get_event_timeline,
   get_agent_activity, get_health_summary

5. 8 new REST endpoints for metrics/audit/events/health/trends/export

6. Workstream 14 — observability + data capture (7 sub-components A-G):
   metrics, audit, events, agent activity, structured logging, agent
   data availability, future visualization plug-in points

7. New Mermaid diagram — observability data capture and query flow

8. Updated architecture diagram to show observability data flows

9. Updated phasing — observability woven into phases 1-3

10. Updated verification — 14 end-to-end checks (was 12), including
    metrics querying, audit trail, correlation tracing
2026-07-06 23:12:34 +02:00
2d75544362 plans: SysML BDD ontology, generic model, full audit
Ontology rewrite:
- Replace Mermaid ER diagram with SysML Block Definition Diagrams (BDD)
  using class diagram syntax: generalization, composition, aggregation,
  association with multiplicity annotations
- Split into 3 diagrams: infrastructure (compute/storage/network),
  software+services, cognition (operations+learning)
- Make compute model generic: ComputeEntity abstract base with
  specializations (Machine, VirtualMachine, Container→LXC/DockerContainer;
  Machine→ProxmoxHost/StandaloneServer/Workstation/Appliance)
- Hypervisor is software on a Machine (not all machines are Proxmox)
- Any ComputeEntity can mount Volumes (VMs AND LXCs, validated)
- DockerContainer is first-class (OS models its own infrastructure)
- Services on any compute type (not just LXC/VM)
- Documents/Runbooks describe any Entity (not just Service)
- Added design notes validating assumptions against actual inventory

Schema updates:
- entity_types: add attribute_schema (JSONB for validating attributes)
- entity_types: add status (active/deprecated, no hard delete while instances exist)

Audit (37 findings across 6 categories):
- Security: 10 findings (5 HIGH) — SSH keys, MCP auth, policy mutability,
  learning poisoning, confirmation phrase, webhook auth, TLS, blast radius
- Performance: 7 findings (1 HIGH) — CTE cycle guard, probe concurrency,
  ingestion, pattern extraction, table growth, WS backpressure
- Architecture: 7 findings (3 HIGH) — testing, observability, DB backup
- Data model: 7 findings (1 HIGH) — entity ID, attribute schema, type
  evolution, concurrent writes, migration rollback, DR export
- Operational: 7 findings (4 HIGH) — rollback, watchdog, backup/restore
  runbook, disaster recovery, deploy downtime, health checks
- Missing: 7 findings (1 HIGH) — CI/CD, rate limiting, audit log, circuit
  breaker, secret rotation, supply chain, SLOs
- Top-5 priority items called out before implementation
2026-07-06 23:06:34 +02:00
d44979aca7 plans: rev 2 — Go rewrite, ontology-first, DB-native config, learning loop
Major revision of the Docker-based homelab OS plan:

1. Go instead of Python — all services rewritten as Go binaries
   (Gin web framework, sqlc for DB access, goroutines for probes)

2. Ontology-first design — systems modeling with 3 layers:
   - Infrastructure (physical, compute, network, storage, software)
   - Governance (identity, secrets, policy)
   - Cognition (observation, decision, action, knowledge, learning)
   7 Mermaid diagrams: layer map, ER diagram, 3 lifecycle state machines,
   feedback loop, policy model

3. DB-native config — inventory.yaml/ontology.yaml/policy.yaml become
   seed manifests (bootstrap + DR). The DB is the runtime source of truth,
   editable via API. Ontology IS the DB schema (entity_types,
   relationship_types, lifecycle_defs tables).

4. Feedback loop — agent learns from execution:
   execution → outcome → feedback → pattern → skill → classification
   Patterns accumulate from execution history, skills codify proven
   procedures, classifier uses pattern confidence for auto-act decisions.
   Cold start: agent starts cautious, earns autonomy through evidence.

5. 5 migration groups: ontology meta-schema, entity instances, operations,
   learning model, policy. Recursive blast_radius SQL function.

6. Phase 0 added: ontology design before any code.
2026-07-06 22:50:49 +02:00
fe54af30f6 plans: replace ASCII architecture diagrams with Mermaid
4 diagrams: container stack, OODA loop, knowledge graph, deploy flow
2026-07-06 22:34:53 +02:00
bead722fac plans: pivot oikos consolidation to docker-based agentic homelab OS
Supersedes the launchd-based consolidation plan. Key changes:
- Docker-based deployment on mac-mini (docker compose)
- PostgreSQL for all mutable state (signals, ledger, knowledge graph)
- Infisical replaces SOPS+age for secrets management
- Unified API merges MCP server + homelab CLI (REST + MCP interfaces)
- Hermes agent runs in Docker (gateway mode, connect from any workstation)
- Knowledge graph in Postgres replaces narrative wiki files as agent context
- Structured entity relationships link docs to inventory entities
- Hybrid SSH access (mounted keys now, actuator gateway later)
- Git push → Gitea webhook → Docker rebuild = deploy trigger
- 6-phase rollout: DB → services → agent → secrets → deploy → cutover
2026-07-06 22:32:21 +02:00
a434a4096c Merge pull request 'chore: add plan' (#2) from claude/heuristic-jang-ecb080 into main
Reviewed-on: dtoro/Homelab-Docs#2
2026-07-06 21:57:48 +02:00
14448a7dd9 chore: add plan 2026-07-06 21:56:56 +02:00
b047c757a7 chore: update title 2026-07-06 21:08:13 +02:00
af14c38fb2 docs: convert OODA loop diagram to real Mermaid syntax
The operating-model diagram in README was ASCII box art in a plain code
fence, not an actual Mermaid diagram — it wouldn't render as a graph on
Gitea/GitHub. Replaced with a `flowchart TD` matching the convention already
used by oikos/gen-topology.py's generated topology.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 20:49:46 +02:00
0d2093ba3b docs: replace hardcoded infra details in README with pointers to index pages
Problem: README listed specific IPs, container IDs, and per-host counts (e.g.
"hubris (15 active): 102 nfs-export, 103 paperless..."). This duplicates
inventory.yaml and the wiki index pages, and goes stale every time a node
moves, gets added, or is destroyed — exactly what happened during the strong
migration.

Fix: Replaced the Proxmox Hosts / VMs / LXC Containers / Cross-Cutting
Infrastructure subsections with plain pointers to their authoritative index
pages (knowledge/wiki/{hosts,vms,containers,infrastructure}/index.md).
Also dropped the "Last refreshed against live state" date line — another
claim that goes stale without a mechanism to keep it honest.

README's job is navigation, not a live topology snapshot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:38:29 +02:00
d90de0759c docs: redesign README for agent clarity and usability
Problem: README.md was human-centric and lacked critical context for agents
(LLMs running on enrolled homelab clients). Agents needed:
- Explicit entry points (AGENTS.md → OIKOS.md → skills → MCP/files)
- Decision tree for tool selection (when to use MCP vs files vs grep)
- Explanation of operating model (OODA loop, risk classes, layer model)

Solution: Reorganized README with agent-first sections while preserving existing
human-useful content:

NEW SECTIONS:
- "For Agents" (entry points + MCP tool selection table with decision criteria)
- "Understanding the Operating Model" (Mermaid OODA loop diagram, risk classes,
  decision flow: classify → escalate if needed → execute → document)
- "Finding & Understanding Information" (layer model table: sources/wiki/index/log,
  what's immutable vs editable, when to update docs)

REVISED SECTIONS:
- "Map & Quick Navigation" (agent entry points first, then topology)
- "Conventions" (expanded with agent-specific guidance: caveman.md, page-templates.md)
- "Updating the Wiki" (clarified infrastructure changes vs restructuring;
  reinforced same-session update rule with explicit checklist)
- "More Information" (grouped agent-facing resources: HERMES, operations,
  skills, shared conventions)

All links verified. No new files needed — all referenced content already exists.

Verification:
- OODA loop diagram present (visual roadmap for decision flow)
- MCP vs Files vs Shell table shows decision criteria
- Layer model (sources/wiki/index/log) explained with immutability matrix
- All cross-references resolve
- Existing topology + infrastructure content preserved

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:32:08 +02:00
18589f6d8d Merge: wiki-hq doc architecture adoption + naming conventions
Complete reorg of narrative docs into sources/wiki/index/log + .agents/ model.
Six implementation phases + two naming clarification passes.

Phase 1-2: Adopt wiki-hq conventions (writing-style, llm-wiki, agent separation)
Phase 3: Move narrative into knowledge/wiki/
Phase 4: Reshape runbooks into skills
Phase 5: Style + README pass
Phase 6: Streamline (move investigations, operations, HERMES to their final homes)
Naming: Explicit conventions for foundational docs (ALL-CAPS) vs content (lowercase)

All 126 pre-existing broken links fixed. Topology, MCP, substrate untouched.
Verification: docs-lint clean, build_host_files idempotent, all doc_page targets resolve.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:20:42 +02:00
2ddc1eaa18 docs: correct file naming convention — foundational docs are ALL-CAPS
The previous naming guide was incomplete. The actual convention is:

**Foundational docs:** ALL-CAPS
- Root entry-points: AGENTS.md, README.md (discovery paths)
- Agent instruction: .agents/OIKOS.md, .agents/HERMES.md (docs agents read first)
- Reference docs: GLOSSARY.md (like classic repo files: LICENSE, CHANGELOG)

**Content pages:** lowercase-with-dashes
- Containers: <id>-<name>.md (ID from inventory)
- Infrastructure: <topic>.md (system description)
- Plans/investigations: YYYY-MM-DD-slug.md (date-sorted)
- Section indices: README.md (conventional)

**Skills:** special pattern
- <name>/SKILL.md where <name> is lowercase-with-dashes
- SKILL.md filename is always uppercase — signpost for tools and humans

Uppercase is reserved for foundational/signpost docs; all paths otherwise use
lowercase with hyphens (no underscores).

Updated page-templates.md with expanded explanation, and updated AGENTS.md +
README.md to reference the corrected convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:19:59 +02:00
658dc0f8b2 docs: document file naming conventions and clarify structure guidance
Added explicit file-naming rules to page-templates.md so agents know:
- Root entry-points: ALL-CAPS (AGENTS.md, README.md)
- Containers: <id>-<name>.md (e.g., 101-jellyfin.md)
- Infrastructure: lowercase-with-dashes (dns.md, auto-deploy.md)
- Plans/investigations: YYYY-MM-DD-slug.md
- Skills: lowercase-with-dashes/ folder containing SKILL.md

Updated AGENTS.md section 4 (Wiki conventions) to link to page-templates.md
and provided quick reference for file naming, page locations, and changelog format.

Updated README.md conventions section to mention file naming and link to
page-templates.md for the full rules.

All agents now have a clear reference chain:
  1. AGENTS.md (entry point) → points to conventions
  2. page-templates.md (structure) → has file naming + page templates
  3. writing-style.md (prose) → has voice, vocabulary, linking rules
  4. llm-wiki.md (organization) → has sources/wiki/index/log model

Verified: no broken links, all conventions documented, consistency check passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:16:58 +02:00
b5c1247093 docs: streamline & consolidate the tree (phase 6)
Problem: after the wiki-hq reorg, agent-instruction and human-doc domains
were still scattered across the repo root, with three now-redundant stub
files cluttering it. The organizing principle wasn't visible in the layout.

Change — enforce three clear buckets:
- .agents/  = how agents operate: OIKOS.md, HERMES.md (moved from root),
  shared/ conventions, domains/ schemas, skills/, and operations/ (operator
  cheatsheet + enrollment + hermes-agent, moved from root).
- knowledge/ = what exists + evidence: wiki/, GLOSSARY.md, and sources/ now
  including investigations/ (incident records are evidence/sources).
- root = substrate + two entry points (AGENTS.md, README.md), plus plans/
  as its own design-intent domain.

Moves:
- investigations/ -> knowledge/sources/investigations/ (incl. archive/, index).
- operations/ -> .agents/operations/.
- HERMES.md -> .agents/HERMES.md.
- Deleted unreferenced root stubs CAVEMAN.md, CONTRIBUTING.md, and OIKOS.md
  (its 7 remaining linkers repointed to .agents/OIKOS.md).

Consumers updated:
- inventory.yaml doc_page (agent-enrollment) + regenerated hosts/*.yaml + cards.
- tools/setup-hermes-soul.sh and bootstrap.sh (x2) -> .agents/HERMES.md.
- bin/homelab help string -> .agents/operations/hermes-agent.md.
- knowledge/operations schemas, llm-wiki, page-templates, incident-investigation
  skill, AGENTS.md/README nav -> new investigations/operations paths.
- All markdown links rewritten via the path-resolving mapper.

Left in place (substrate/executable/separate-domain): hosts/, ledger/, tools/,
plans/, oikos/, mcp/, secrets/, bin/, inventory.yaml.

Verification: docs-lint at baseline (2 intentional cross-repo refs, no new
breakage); gen-topology.py --check exit 0; build_host_files.py idempotent; all
doc_page targets resolve; Hermes provisioning scripts point at the new path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 18:12:14 +02:00
4efddb8bed docs: fix pre-existing broken links surfaced by docs-lint
Problem: docs-lint (added in the wiki-hq reorg) surfaced 126 broken relative
links that predated this session — a container rename, incident/plan docs
that moved into archive/done subfolders without their inbound links being
updated, and a handful of relative-depth bugs in files nested under
containers/archive/ and plans/done/.

Fixes applied, by category:
- 124-authentik.md -> 106-auth-outpost.md (container was renamed; ~40 refs).
- investigations/{2026-04-21-hubris-crash-loop,2026-05-31-authentik-vps-migration}.md
  -> archive/ prefix (both moved to investigations/archive/ previously).
- plans/{2026-06-01-slate-ax-to-sodola-migration,2026-06-04_130000-deprecate-claudio-bot,
  2026-06-25-yuvomi-deployment}.md -> plans/done/ prefix.
- Depth bugs in files nested one level deeper than their siblings assumed
  (investigations/archive/*, knowledge/wiki/containers/archive/*,
  plans/done/*) — corrected relative-path depth.
- Destroyed containers with no surviving page (126-plato) delinked to the
  containers/index.md archaeology row instead of a 404.
- ludo-mini.yaml -> strong.yaml (host was renamed, same physical machine).
- netbird-vps.md (no narrative page exists) -> netbird-vps.yaml (substrate
  record, matching the existing convention for hosts without a wiki page).
- runbook-dpkg-interrupted.md refs -> .agents/skills/runbook-dpkg-interrupted/SKILL.md
  (missed in the phase-4 runbook move because the referencing files used a
  bare filename, not a runbooks/ prefix).
- One dangling forward-reference to a never-written investigation delinked
  to the actual incident record it was describing.

Left alone: two links in knowledge/wiki/containers/101-jellyfin.md into
devops/homelab-authentik-admin/ — an intentional reference to a sibling repo,
not present in this checkout.

Verification: broken-link count 126 -> 2 (real remainder is the cross-repo
reference above); gen-topology.py --check still exit 0; build_host_files.py
still idempotent; all inventory.yaml doc_page targets still resolve.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:53:35 +02:00
1019a1cb52 docs: section-index pass + docs-lint skill (phase 5)
- Add knowledge/wiki/hosts/index.md (the one missing section index) and point
  knowledge/index.md at it.
- Add .agents/skills/docs-lint/ (SKILL.md + lint.py) enforcing the mechanical
  parts of writing-style.md: banned vocabulary and broken relative links. The
  style guide and this skill are exempt from the banned-word check since they
  enumerate the list.
- Record the restructure + lint in knowledge/log.md.

Verification: banned-vocabulary scan of knowledge/ is clean (the few remaining
repo-wide hits are false positives — the literal '_' character — or historical
append-only plans quoting the vocabulary, which the standard does not restyle).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:42:23 +02:00
5c5016b3c7 docs: reshape runbooks into .agents/skills/<name>/SKILL.md (phase 4)
Problem: runbooks are agent-executable procedures but lived at the repo root,
separate from the other agent instruction now under .agents/.

Change:
- Move runbooks/<name>.md -> .agents/skills/<name>/SKILL.md (folder per skill,
  matching the wiki-hq skills layout). Frontmatter (name, risk_class, inputs,
  verification, docs_update_checklist, transition) preserved.
- Rewrite links (inbound from plans; between-skill siblings) via the move map.
- Update prose references in AGENTS.md, HERMES.md, .agents/OIKOS.md, and the
  operations schema; fix a pre-existing stale link to operations/commands.md.

No code consumed runbooks/ by path, so nothing else changes.

Verification: all SKILL.md frontmatter parses with valid risk_class; every
lifecycle transition resolves to an oikos/ontology.yaml state; broken-link
count 127 -> 126 (fixed one, introduced none).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:39:31 +02:00
8a6422bd7d docs: move narrative wiki under knowledge/wiki/ (phase 3)
Problem: node and cross-cutting narratives lived at the repo root
(containers/, vms/, infrastructure/, host .md files), interleaved with the
machine-readable substrate.

Change:
- Move containers/ -> knowledge/wiki/containers/, vms/ -> knowledge/wiki/vms/,
  infrastructure/ -> knowledge/wiki/infrastructure/, hosts/{hubris,strong}.md ->
  knowledge/wiki/hosts/, infrastructure/references/ -> knowledge/sources/references/,
  GLOSSARY.md -> knowledge/GLOSSARY.md.
- Add knowledge/{index.md,log.md,sources/index.md} scaffolding.
- Rewrite all relative links repo-wide via a path-resolving mapper (inbound +
  outbound + between-moved-files), including .hermes/, runbooks, operations,
  investigations, plans, README, AGENTS.
- Repoint inventory.yaml doc_page fields and regenerate hosts/*.yaml (which
  embed doc_page); update oikos/gen-topology.py output path, candidate doc
  paths, and footer links; update code-comment doc paths.

Substrate untouched in place: inventory.yaml, hosts/*.yaml (regenerated,
idempotent), oikos/ code, mcp/, secrets/, bin/.

Verification:
- Logical broken-link set identical to pre-move baseline (net 128 -> 127; the
  topology regen fixed one, introduced none). Remaining are pre-existing refs
  to destroyed/archived nodes, out of scope for this move.
- gen-topology.py --check exit 0 (in sync); cards carry knowledge/wiki/ doc paths.
- build_host_files.py idempotent; all inventory doc_page targets resolve.
- MCP contract verified: get_page/search_docs/get_changelog resolve moved pages.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:35:23 +02:00
bb5c0717a1 docs: adopt wiki-hq conventions + separate agent instructions (phases 1-2)
Problem: the narrative docs lacked an enforceable style standard, and
agent-facing instruction (OIKOS/CAVEMAN/CONTRIBUTING) was interleaved with
human content at the repo root.

Change:
- Add .agents/shared/{writing-style,llm-wiki}.md — a lint-checkable prose
  standard (with an imperative-voice exception for runbooks/recipes) and the
  sources/wiki/index/log layer model.
- Move CAVEMAN.md -> .agents/shared/caveman.md,
  CONTRIBUTING.md -> .agents/shared/page-templates.md,
  OIKOS.md -> .agents/OIKOS.md; leave thin root stubs so old links resolve.
- Add .agents/domains/{knowledge,operations}/schema.md; operations schema
  codifies "plans always live in plans/".
- Repoint live references (AGENTS, README, GLOSSARY, OIKOS) and fix OIKOS.md's
  internal relative links for its new depth.

Risk: none to the operational substrate — inventory.yaml, hosts/*.yaml,
oikos/, mcp/, secrets/, bin/ untouched (verified via git status).

Verification: relative-link check across .agents/ clean; substrate churn empty.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 14:22:09 +02:00
14e88c7c5e Document real deploy state: console live, webhook still broken
Console is fully live on apps (105) — deployed manually via deploy.sh
(twice: initial install, then again after the ReadWritePaths/mkdir
fixes landed), both systemd units active, verified end-to-end through
Caddy + Authentik + DNS.

Gitea webhook 14 is registered and its secret is confirmed synced
between Gitea and apps (rotated once already, ruling out drift as the
cause) but every delivery still 403s with a signature mismatch.
Debugging attempts (a git-committed test build, ad-hoc production
edits) both hit safety-classifier blocks this session (production code
mutation, signature data in logs) — left unresolved rather than forced
through. Auto-deploy via push doesn't work yet for this service; manual
deploy.sh re-runs are the workaround until someone tracks this down.

Added to the 60/90-day backlog.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:11:27 +02:00
dede118a80 chore: trigger oikos-console webhook to verify rotated secret 2026-07-06 14:02:26 +02:00
72720efa21 Rotate oikos-console-deploy webhook secret
The value written to apps' /etc/oikos-console-deploy/secret didn't
match what Gitea webhook 14 had configured, causing every deploy
attempt to 403 with a signature mismatch — likely drift introduced by
the earlier two-step PATCH sequence (secret set in one call,
branch_filter/active restored in a second call without re-including
the config object). Rotated cleanly this time: fresh secret set on
Gitea and re-encrypted here in one pass, single atomic PATCH covering
config+events+branch_filter+active together.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:01:29 +02:00
7610e5394c Deploy Oikos Console to apps (105); fix missing-directory crash
Console is live: cloned to /opt/oikos-console, deploy.sh ran clean,
webhook secret written to /etc/oikos-console-deploy/secret from the
pre-registered SOPS secret (never printed — decrypted and piped
straight into the target file in one command), both systemd units
enabled and active. Verified locally (127.0.0.1:8091 -> 200) and
end-to-end (https://oikos.hubris.network/ -> 302, the Authentik gate
firing correctly).

Found a real bug during first boot: oikos-console.service's
ReadWritePaths listed /opt/homelab-context/signals and .../approvals,
but neither existed yet on apps' clone — git doesn't track empty
directories, and nothing had ever written a signal/approval from that
host. ProtectSystem=strict + a missing ReadWritePaths target is a hard
226/NAMESPACE crash, not a graceful degradation. Fixed two ways:
the unit now marks those paths optional (`-` prefix) so a fresh deploy
never crash-loops on this again, and deploy.sh now mkdir -p's them
explicitly so the console has real write access from the first boot,
not just a non-crashing-but-broken start.

This is also the first real exercise of the auto-deploy pipeline: this
push should land via Gitea webhook 14 -> oikos-console-deploy.service
on apps, same as homelab-mcp/secrets-issuance already work.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:57:50 +02:00
610b096840 Add Technitium DNS record for oikos.hubris.network
A record -> 192.168.8.175 (Caddy's LAN IP), created via Technitium's
REST API (login -> createToken -> zones/records/add) in a single
in-memory call. Neither the admin credential nor the resulting session/
API token was ever printed to output or written to disk, and the token
wasn't persisted anywhere after the call completed — it existed only
for the lifetime of that one process.

Verified: dig @192.168.8.2 +short oikos.hubris.network -> 192.168.8.175.
End-to-end confirmation that DNS + Caddy + the Authentik gate are all
wired correctly: curl https://oikos.hubris.network/ now returns a 302
(the forward-auth redirect firing before the not-yet-deployed backend
would even matter) instead of failing to resolve/connect.

This closes out every part of the console rollout except the actual
apps-side bootstrap (oikos/console/deploy/README.md "One-time setup"),
which remains pending direct operator execution.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:41:52 +02:00
25c67a79c6 Add Caddy route for oikos.hubris.network; fix loopback-bind bug
Pushed dtoro/caddy-conf@c195142: oikos.hubris.network -> 192.168.8.205:8091,
Authentik-gated (matches paperless.hubris.network's live pattern —
confirmed exact snippet syntax against the real Caddyfile rather than
trusting the paraphrase in the original README, which turned out to
have the wrong forward_auth target: the live snippet points at
127.0.0.1:8099 on Caddy's own LXC, not 192.168.8.6:9000 as
containers/106-auth-outpost.md's older text suggested). Reload verified
clean — an unrelated existing route stayed healthy through it.

Found and fixed a real deploy-blocking bug in the process:
oikos-console.service bound 127.0.0.1 only, but Caddy runs on a
different host (121) and can only reach apps (105) over the LAN — the
console would have been completely unreachable once deployed. Now binds
0.0.0.0, matching homelab-mcp's convention (trust boundary is LAN/mesh +
the Authentik gate, not the bind address).

Encountered and deliberately left alone: a pre-existing local clone at
/tmp/caddy-conf with an unpushed commit + uncommitted diff about
jellyfin's auth gating, from before this clone fell 12 commits behind
origin. That work turned out to be superseded (origin already reached
the same conclusion — SSO plugin handles jellyfin auth, no forward-auth
gate — via a different, already-merged path). Didn't touch it; used a
fresh clone instead to avoid any risk of losing or corrupting that state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:31:51 +02:00
493ae50f37 Register Gitea webhook 14 for oikos-console deploy
Created via the Gitea API (POST /repos/dtoro/Homelab-Docs/hooks) rather
than the UI, since the existing PAT turned out to have sufficient scope.
Webhook id 14: http://192.168.8.205:9831/deploy, push events, main branch
filter, active.

The shared secret was generated and registered with Gitea before the
apps-side bootstrap ran (order reversed from the usual install.sh-first
flow, since direct SSH deploy to apps is still pending operator
execution — see oikos/console/deploy/README.md). Stored as
secrets/oikos-console-deploy-secret.yaml (SOPS, recipient: apps only)
rather than left as a local plaintext file, with explicit operator
sign-off. When the apps-side install runs, skip webhook/install.sh's
random-secret generation and write this exact value into
/etc/oikos-console-deploy/secret instead.

infrastructure/auto-deploy.md updated with the real webhook id (was
"not yet registered").

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:18:46 +02:00