Problem: entity mutations (create/update/state) existed as three drifted
copies — HTTP CreateEntity/PatchEntity, MCP create_entity/
update_entity_attributes/set_entity_state — each with its own inline
SQL, its own validation subset (MCP validated lifecycle states, HTTP
did not; HTTP patched attributes without regenerating derived checks,
MCP did; MCP wrote no audit trail), the exact drift ADR 0016's first
vertical slice exists to collapse.
Change:
- internal/core/ports: DerivedCheck, Idempotency (adapter-owned request
hash + cached-body renderer so the replay record commits in the
create's transaction), IdempotentResponse + GetIdempotent read,
AuditEntry gains Method/Path/CorrelationID, Event gains
CorrelationID; EntityUpdateInput carries ExpectedVersion +
RederiveChecks (derivation for updates runs repo-side: the graph
host fallback reads relationships through the open tx).
- internal/adapters/postgres/repositories.go: EntityRepo (Create/
Update/SetState/reads/idempotency) preserving the load-bearing
check-then-act invariants in-tx: version WHERE-clause, declared
transitions + preconditions (ValidateTransition), duplicate-slug
mapping, audit/event/writeCheck all inside one BEGIN…COMMIT.
OntologyRepo: TTL-cached OntologyStore.
- internal/core/app/entities.go: EntityService — ontology validation
(type exists, concrete, state declared — the stricter MCP rule now
governs both surfaces), default-state resolution, id generation,
derivation for creates, audit/event construction, idempotency
pass-through.
- httpapi CreateEntity/PatchEntity rewired to the service; PATCH now
regenerates derived checks (the A2 parity gap). MCP create/update/
set-state tools call the same service — and now write audit + event
rows like the HTTP surface always did.
- Integration-test seed paths fixed for the adapters/postgres package
depth (../../../seeds).
Pre-existing failures documented: TestAPIEndToEnd (entity_types 60 vs
59; 501-endpoint now 200), TestClientLifecycleEndToEnd, TestPhase3*
rows — verified failing identically at ec11956 (scratch approval-
notifier commit test drift), unrelated to this change. All mutation
integration tests (create/patch/idempotency/audit/regeneration) pass.
Verification: make test-db (postgres + mcp green, httpapi green except
the pre-existing set), full non-DB suite (19 pkgs), golangci on new
packages — 0 issues.
Problem: check derivation logic lived in internal/checkdefaults with
the pure decision logic (buildKind, address/user/port resolution)
interleaved with tx I/O (entity_status insert, graph host fallback,
check upserts) — and internal/db importing it was the plan's called-out
inverted dependency.
Change:
- internal/core/app/checkdefaults.go: Derive(tree, target, lookup) —
the full derivation (monitoring overrides, host fallback via an
injected HostLookup thunk, per-kind builders) with zero I/O imports.
Types renamed for the app surface: CheckTarget, CheckDef,
DeriveResult, Skip; LogDeriveResult.
- internal/adapters/postgres/checks.go absorbs the I/O half:
EnsureChecks (entity_status row + upsert loop), writeCheck, and
hostViaGraph. The db→checkdefaults edge is gone — adapters→core is
the ADR 0016 direction (the Phase 7 SeedService note anticipated
this; the inversion is fixed a phase early).
- seed.go pending-checks loop uses app.CheckTarget + EnsureChecks;
mcp formatting/tests follow the renamed types; both test files moved
to internal/core/app.
- Deliberate behavior note: a hostViaGraph read failure inside the
thunk now logs a warning and degrades to 'skipped: no address'
instead of aborting the whole entity-create tx — a monitoring
derivation gap is visible (warn log + coverage sweep) and self-heals
on the next mutation; failing the create over a graph-read blip was
disproportionate.
Verification: go build/vet, full test suite green (app tests exercise
every buildKind branch at their new home).
Problem: the repo carried cruft that predates the Phase 1 client split:
tracked web/node_modules and web/dist content (committed before the
ignore rules existed — ~2.6M lines), seven stale Claude worktrees plus
two stale Agent Manager worktrees (1.5GB on disk, all fully merged),
their 29 merged experiment branches, a pre-DB root inventory.yaml,
Playwright MCP session logs, a config screenshot, and the executed
one-shot apps/105 webhook-cleanup script. The dockerignore's own
comment documents how this cruft once starved the mac-mini disk
mid-build.
Change:
- git rm: web/ + cmd/desktop/ tracked remnants (node_modules, dist),
root inventory.yaml (stale pre-DB copy; seeds/inventory.yaml is
authoritative and what tests read), .playwright-mcp/ logs,
config-screen.png, .claude/launch.json,
scripts/cleanup-apps105-webhooks.sh (job done; pattern lives on in
oikos-web's webhook setup).
- Removed 9 stale worktrees (nested-first) + pruned; deleted 29 fully
merged branches (claude/*, feature/*, frontend-os-apps,
judicious-freckle, code-quality-* pair, impartial-height). The one
branch with an unmerged commit, chore/vendor-orby-engine, vendored
web/vendor — that work moved to dtoro/oikos-web in Phase 1, so it is
superseded.
- Disk cleanup: web/, cmd/desktop/, stray desktop binary, bin/, build
artifacts. Kept: root oikos + webhook binaries (referenced by the
launchd deploy unit and oikos-web's installer), .env bootstrap,
.infisical-credentials.
- .gitignore: .claude/, .playwright-mcp/, config-screen.png now
clone-safe instead of relying on local info/exclude.
Risk: none functional — deletions are either merged history (branches
recoverable from reflog) or content that moved repos; build, vet,
tests, and generate-check all green post-purge.
Verification: go build/vet, make test (19 pkgs ok), generate-check,
git ls-files web cmd/desktop → 0; worktree list → main only.
Problem: the hexagon's Phase 2 (plans/2026-08-15-hexagonal-architecture.md)
must give the use-cases-to-be their contract surface: driven-port
interfaces, test fakes, the secrets interface moved into core, and the
postgres package inside the adapters tree — before the first vertical
slice (Phase 3) can wire a composition root.
Change:
- internal/core/ports: full driven-port catalog per plan §3.3 —
repositories as transaction-scoped aggregates whose inputs carry
derived checks, audit, and events (§3.6), plus CommandExecutor,
TargetResolver, Checker, Secrets, EventPublisher, Provisioner.
Port-local payload types (Event, AuditEntry, CheckDef, KnowledgeEntry,
ExecResult) keep signatures off infrastructure; TypeTree aliases
internal/ontology (pure over domain) until checkdefaults is absorbed.
ReadModels intentionally not declared yet — it materializes with the
Phase 3 slice and grows as report handlers rewire.
- secrets.Backend is now an alias of ports.Secrets; implementations
(Infisical, SOPS, Manager) unchanged. mcp's local secretBackend
subset is deleted; tool constructors take ports.Secrets.
- internal/db → internal/adapters/postgres (mechanical import rewrite;
package identifier stays db until the Phase 3 repository split).
sqlc.yaml, Makefile, golangci exclusions, and docs follow the move;
make generate-check verified.
- internal/adapters/ssh: Executor implements ports.CommandExecutor over
the actuator dial pool + RunStreaming (10-min default timeout carried
over from the httpapi path).
- internal/adapters/remote: Resolver implements ports.TargetResolver
delegating to internal/remote (still pool-based; drops onto
ports.EntityRepository when repositories land in Phase 3 — documented
transitional import).
- internal/core/ports/portstest: importable fakes — in-memory
EntityRepo (with check-then-act SetState, side-effect recording),
RecordingExecutor, FakeChecker, SpyPublisher; port-satisfaction
guards; tests.
Risk: ports are declared ahead of implementations — signatures firm up
per phase as slices land (documented in the package doc); the
remote→postgres transitional import is explicit and dissolves in
Phase 3.
Verification: go vet, make test (race, 19 packages), generate-check,
golangci on core+adapters — 0 issues; full-repo baseline down
365→344.
Problem: the hexagonal refactor churns the backend tree for nine more
phases; the UI delivery stack (web/ SPA, cmd/desktop Wails wrapper,
compose/web image) must move to its own repo first so doc/layout
rewrites land once on a backend-only tree.
Change:
- New repo git.hubris.network/dtoro/oikos-web (v0.33.0): web/, desktop/
(updateURL repointed to oikos-web releases), compose/, own CI (web +
desktop jobs), own deploy script (CI-green gate, TOCTOU guard,
version-tagged images, prune-to-3), own webhook receiver on :9798 +
launchd unit, own compose project publishing the same 8091:80.
- Cutover executed on mac-mini in order: oikos stack's web service
stopped+removed, oikos-web project brought up on 8091; outer Caddy
untouched (targets the published port) — serving + Authentik flow +
/wails 404 quirk verified post-cutover.
- Stripped from oikos: web/, cmd/desktop/, compose/web/, desktop CI
workflow, ci.yml web job, Makefile ui/desktop/desktop-package/install
targets, the compose web service, oikos-web from deploy.sh's fallback
prune list; wails + go-keyring dropped from go.mod, vendor synced.
- README / CONTRIBUTING / AGENTS.md / .agents dev+operations docs now
point at the new repo; mbse + mascot design docs carry a path note.
Risk: production SPA serving depends on the new pipeline now; rollback
is versioned-image re-up of the old web service from a pre-split
checkout (port 8091). Desktop builds installed before the split still
check dtoro/oikos releases — one manual reinstall, noted in the
oikos-web release notes.
Verification: go vet, make test (race), make generate-check, golangci
(no new findings; baseline down 400→365); post-cutover curls —
localhost:8091 200, /wails/runtime.js 404, outer Caddy 302 Authentik.
Problem: the hexagonal-architecture plan (plans/2026-08-15-hexagonal-
architecture.md) needs its foundation — an accepted ADR, the target
directory tree, and machine-checked dependency rules — before any
service extraction starts. Also folds the four outstanding review
findings (F3.1/F5/F6/F7) into the plan: ObservationService owns the
bounded probe-concurrency contract (scheduler.go:133), Phase 9 gates
ExecutionService+PolicyService ≥ 90% with a gating-matrix test,
per-phase abort criteria, and the §3.2 internal/config note.
Change:
- docs/adr/0016-hexagonal-ports-adapters.md records context, decision,
and consequences of the ports & adapters migration.
- internal/domain → internal/core/domain (mechanical import rewrite,
20 files), new internal/core/{ports,app}, internal/adapters trees
with package docs.
- .golangci.yml: depguard rules for §3.1 (core purity, no agent-client
tech in core, nomos isolation — the nomos rules self-activate when
internal/nomos exists in Phase 8). Config migrated to golangci-lint
v2 format so it loads at all (the v1 config errored under v2, masked
by CI's advisory continue-on-error). Verified depguard fires on a
planted openai-go import in internal/core/app.
- CONTRIBUTING.md layout section now shows the core/adapters tree.
Risk: import path churn is mechanical and tests pass unchanged; the
lint config migration surfaces the pre-existing 400-issue baseline
(advisory in CI, unchanged policy) — new/moved packages lint clean.
Verification: go vet ./..., make test (race, core/domain at 100%
coverage), make generate-check, golangci-lint on internal/core/... and
internal/adapters/... — 0 issues; depguard violation probe confirmed.
Removes the entire Matrix-based notifier (internal/notifier/) that polled
for pending approvals, sent Matrix alerts, and checked for reaction-based
approve/deny. Approval decisions now work on any chat platform (Hermes
desktop, Telegram, Discord, WhatsApp, CLI) via two new MCP tools:
- list_approvals — query pending/recent approvals by status or entity
- decide_approval — approve/deny via same API endpoint as UI + nomos
Config fields removed: MatrixHomeserver, MatrixUserID, MatrixToken,
MatrixRoomID, ApprovalHMACSecret. Docker notifier: service removed.
Approval HMAC token generation removed (unused by code).
The existing chat-assent path in nomos (cmd/nomos/assent.go) and the
control-room Approve button keep working unchanged — both call the
shared POST /api/v1/approvals/{id}/decision endpoint.
New tool batches session discoveries into the knowledge graph:
- Creates a session-audit knowledge entry with summary
- Links it to all touched entities via 'documents' relationships
- Creates individual discovery knowledge entries
- Stamps each entity with last_agent_session attribute
- Updates AGENTS.md with tool listing
P0: Transport escalation in classifyAndGate (server.go:745) now exempts
log-inspection commands (tail/head/cat/journalctl on *.log or /logs/)
from the /opt//etc//var/lib/ gating on LXC targets. Fixes the
'tail -3 /opt/seanime/data/logs/seanime.log queued for approval' bug.
P1: queryEntity returns actionable error when slug_or_id param is empty
('slug_or_id is required') instead of silent 'entity not found: '.
P2: Added slugArg() helper (server.go) so get_entity, get_relations,
get_blast_radius, and explain accept 'slug' as an alias for their
declared param name. Solves the discoverability inconsistency where
every tool used a different param name for the same concept.
P3: Two new MCP tools:
- restart_service(target, service) — systemctl restart wrapper,
correctly classified config_mutation (requires approval)
- push_file(target, source_path, dest_path, backup=true) — pct push
from Proxmox host into LXC, with optional backup. Classified
config_mutation. LXC-only for now.
P4: Updated homelab-lxc-ops skill with MCP tools preference table.
Plus: Wails desktop build now uses build-tag approach for frontend embed
(assets_embed.go + assets_stub.go), so go build ./... works on
clean checkout without the frontend built first.
Version: 0.31.0 → 0.32.0
Adds pvecm_quorum_check.sh probe script and wires it into the
checkdefaults system as a new 'quorum' monitoring kind on proxmox-host
entities. Runs every 60s via ssh-script, surfaces unhealthy signal when
cluster loses quorum.
Closes the monitoring blind spot that let the 2026-08-12 3.5h corosync
flapping outage go undetected (ping passed, cluster was non-quorate).
Changes:
- seeds/ontology.yaml: proxmox-host declares monitoring: [quorum]
- internal/checkdefaults/defaults.go: KindQuorum builder
- internal/checkdefaults/build_test.go: 2 new test cases
- checks/pvecm_quorum_check.sh: new probe (deployed to hubris + strong)
- VERSION: 0.30.2 -> 0.31.0
Two fixes from the deploy pipeline audit:
1. Infisical tag v0.99.1 no longer exists on Docker Hub — bumped to
v0.162.19 (latest available). This was silently breaking the full
deploy pipeline (docker compose up failed on image pull).
2. Deploy failures now notify via two channels:
- Oikos API event (deploy.failed, severity=critical) — picked up by
the scheduler's notifier for Matrix alert
- Matrix webhook URL if MATRIX_WEBHOOK_URL is configured
Uses a trap with _ok flag to catch any non-zero exit path,
including CI gate rejections and health check timeouts.
Webhook now resolves and passes OIKOS_API_TOKEN to deploy.sh.
Registers a handler that serves the embedded OpenAPI 3.0 spec
(compiled into the binary via oapi-codegen) at a browseable
endpoint. Uses gen.GetSwagger() to deserialize the embedded
base64+gzip spec and returns it as JSON.
46 paths, 42 schemas — agents and humans can now introspect the
full API surface without reading Go source.
Read-only diagnostic commands ethtool, lsmod, lspci, modinfo, and dkms
were missing from the readOnlyLeadPattern in the command classifier,
causing compound diagnostic commands (e.g. 'uname -r && ethtool -i eno1
&& lsmod | grep r8169') to be misclassified as config_mutation instead
of read_only. This forced operator approval for simple hardware/driver
inspection during the 2026-08-12 hubris NIC cutover session.
Added regression test with the exact compound command from that session.
New EntityGraph.svelte app (sigma.js + graphology):
- Theme-aware Gruvbox palettes (light/dark) with reactive color switching
- Lucide icons rendered synchronously via Path2D canvas per entity type
- Color mode toggle (Health / Type) with per-type distinct colors
- Health distribution bar with clickable filters
- Entity type filters grouped by ontological layer (collapsible)
- Relationship type edge filters with color-coded swatches
- Quick presets: All / Problems / Infra
- Node selection with live blast radius from API
- Hover neighborhood highlighting with muted fade
- Isolated node hiding, edge alpha tuning, dot-grid background
- Search with camera focus on highest-degree match
createTaskEntity creates the task entity but never adds any graph
edges. The involves edges are only added by the run handler, so
sessions that call set_goal but never make a run call leave orphan
task entities with zero relationships.
Adds an idempotent involves edge from the new task to agent:nomos
at creation time, matching the same pattern used for run's involves
edges in server.go.
Every run call creates a classification entity, but it was never
connected to the execution via a graph edge — only via a DB column
(executions.classification_id). The ontology requires:
classification —precedes→ execution
Without this edge, all 53 classification entities had zero
relationships, making them invisible to get_relations and
blast-radius analysis.
Adds an idempotent INSERT into relationships after the existing
classification_id update, matching the same pattern used for
targets edges.
The healthcheck subcommand was reachable only after main()'s Infisical secrets
resolution (4x retries/key, ~30s when Infisical is down), which blew the 5s
Docker healthcheck timeout — so nomos stayed docker-unhealthy despite serving
/healthz fine. Short-circuit 'nomos healthcheck' at the top of main() before
any secrets init; measured 0.58s, no Infisical retries.
The nomos runtime image is gcr.io/distroless/static (no shell/wget), so the
wget-based healthcheck (D5) could never run — nomos showed docker-unhealthy
despite serving /healthz fine. Add a 'nomos healthcheck' subcommand that
self-probes NOMOS_LISTEN/healthz (exit 0 on 200), and point the compose
healthcheck at ["/nomos", "healthcheck"].
Measured startup is ~93s: NewHandler stalls on Infisical auth retries (~40s)
and OIDC discovery timeouts to auth.hubris.network (~35s) before binding
:8090. The api IS healthy once bound (serves /healthz); the window just needs
to clear both external-timeout phases so nomos (depends_on: api-healthy) can
start and the deploy completes.
The api retries Infisical at startup (4x with backoff) before binding :8090.
When Infisical is unreachable that adds ~60s, and the old start_period (10s)
+ 10 retries (~60s grace) ran out just before the bind — marking the api
unhealthy and failing the deploy (nomos depends_on api-healthy). 90s covers
the slow-startup window; the api genuinely serves /healthz once bound.
The procedural-glyph-engine dep pointed at a non-portable file:/private/tmp/orby-pkg
path, breaking npm ci in Docker and every main deploy since v0.20.0 (the build
cache masked it until it busted ~Aug 5). Vendor Orby v5.0.0 into web/vendor/,
switch the dep to file:../vendor, and use npm install in the web Dockerfile
(file: deps need install, not ci). Cherry-picked from 3cd4cf9.
- Wire secretsManager in NewHandler() — instantiate InfisicalBackend
when OIKOS_INFISICAL_SITE_URL is set (previously always nil)
- Add get_secret, list_secrets, set_secret MCP tools with nil-backend
graceful degradation
- Add oikos secret get|set|list CLI subcommands for Infisical
- Fix Set() bug: create-before-update so new keys are created;
add Type: "shared" to Update so it finds the right secret;
disable SDK cache so Get returns fresh data after Set
- Clean enrollment response: remove fake infisical_client_id/
infisical_client_secret stubs, store age key in Infisical for real
- classifyAndGate: escalate read-only commands on lxc: targets that
touch /opt/, /etc/, /var/lib/ to config_mutation. The classifier
scores command text only, not the SSH transport layer — SSH-ing into
a container to read config is riskier than pct exec from the host.
- ontology: standalone-server monitoring override from inherited
[ping, resource, updates] to [http]. VPS-like machines may not be
SSH/ICMP-reachable from the scheduler; HTTP is the LCD liveness
signal. Entities with full SSH can override per-entity.
- get_health_summary: filter out state=destroyed entities (was noise
from 20+ destroyed test LXCs, deprecated services, etc.)
- create_entity: document the monitoring footgun in the tool description
(creating a type=check entity does NOT wire a check_def; the correct
path is update_entity_attributes with monitoring + url attributes)
Adds a new 'dns' semantic monitoring kind that probes whether a DNS name
resolves. Uses net.LookupNS (NS records) with fallback to net.LookupHost
(A/AAAA). Supports an explicit server config for split-horizon resolution.
Changes:
- seeds/ontology.yaml: dns-zone monitoring: none → [dns] (was deferred
since 2026-06 with a comment 'no dns checker exists yet')
- seeds/inventory.yaml: host:netbird-vps monitoring: [http] (was none;
VPS was invisible for 7 days during the 2026-07-29 outage)
- internal/checkdefaults/defaults.go: add KindDNS, buildKind case for 'dns'
that creates a check_def at 5-minute intervals
- internal/scheduler/scheduler.go: add checkDNS probe + wire in executeCheck
The DNS checker catches stale/unreachable zones (e.g. matrix.hubris.network
pointing to a dead VPS IP). The VPS HTTP check probes the public endpoint
every 60s, closing the 7-day monitoring gap.
I — run pre-flights QEMU guest agent before queueing VM execution
classifyAndGate now checks vm: targets for qemu_guest_agent attribute.
If not_running/missing, returns immediate error instead of queuing forever.
II — policy.yaml: documented host-mutation classifier rule
Added comment clarifying that host-level package/kernel mutations
(apt-get install, dpkg, systemctl enable) always classify as
config_mutation and thus need operator approval.
III — health attribute read-only in update_entity_attributes
Strips scheduler-owned keys (health, last_check_at, last_check) from
attribute updates with a clear message directing agents to
get_health_summary / list_checks instead.
IV — Recorded discovered dependency edges
vm:zimaos → depends-on → lxc:nfs-export (NFS /media/library mount)
vm:zimaos → depends-on → host:strong (NFS /media/ludo-library mount)
Also updated the run tool description to mention both guardrails.
get_relations now accepts an optional 'types' (comma-separated) parameter
to filter relationship types — filters out the noisy exec/targets edges
that previously drowned useful host/provides edges.
get_health_summary now accepts an optional 'health' (comma-separated)
parameter to return only entities in specific health states (e.g.
'health=down,stale') instead of the full 100+ entity list.
ping_service now:
- Falls back to e.attributes->>'public_host' when 'url' is not set
(covers LXCs that only have public_host in the graph)
- Performs a live HTTP HEAD probe against the resolved URL, returning
the actual status code instead of just the scheduler's stale health
state
Also: fixed matrix.hubris.network DNS record (was pointing to dead VPS),
pruned 6 dead graph edges, wired url attributes on 7 LXCs, added VPS
HTTP monitoring check, and resolved the 18k-occurrence unmonitored signal.
This session's audit is documented as
document:nomos/2026-08-05-dns-monitoring-improvements-for-strong-hosted-services.
- Add MCP tool — lightweight connectivity check returning server
identity, no DB hit (resolves agent connection-test friction)
- Tighten 6 tool descriptions (get_relations, get_health_summary,
query_metrics, get_trend, get_event_timeline, ping) to be searchable
in the first 8-12 words
- Document Hermes MCP client setup in ADR-0012 with token security caveat
- Move completed plan to plans/done/
Backend:
- Add isThinking flag to agentEvent for text before tool calls
- Separate thinking from response text in runChatTurn and continue.go
- Persist thinking in a dedicated field in message content
Frontend:
- Add thinking field to MessageContent, ChatMessage, ChatTextEvent types
- Create ThinkingBlock.svelte — collapsible block with brain icon
- SSE handler moves text_delta content to thinking on isThinking flag
- Render thinking block between tools and response in ChatThread
- Fix chat window scroll reset on focus change (stable windowKeys order)
- Remove redundant #key id wrapper in WindowLayer
- Enlarge sidebar rail (24→32 default, 40→60 max)
- Remove glyph from sidebar, square graph at top
- Replace AgentTrace/ToolCallCard/UnifiedTimeline with TurnTrace/ToolLine
