Follow-ups found while verifying the approve→provision path end to end:
- vmid is now optional: the early required-field check rejected vmid:0
before the cluster VMID guard could auto-assign a free id. Only hostname
is required now; 0 (or a collision) resolves to `pvesh get /cluster/nextid`.
- net0: use ip=dhcp with no gateway when no static IP is given (Proxmox
rejects gw alongside dhcp); only attach gw for a static CIDR.
- bump post-create settle to 10s so a DHCP lease is up before apt runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Real production failure when the operator clicked Approve in chat: nothing
provisioned, banner never cleared, execution marked completed.
Three root causes:
- sshExec swallowed non-zero exits when the command produced output, so a
`pct create` that printed "CT 132 already exists" and failed was reported
as success and a bogus lxc entity was registered. Now any non-zero exit
returns an error (with output) so the execution is correctly marked failed.
- The LLM reused VMID 132 (belongs to lxc:rclone; VMIDs are cluster-wide).
pct_create now checks in-use VMIDs via `pvesh get /cluster/resources` and
falls back to `pvesh get /cluster/nextid` when the requested id is taken.
- InlineApproval.svelte reset its state on every prop change (done was also
compared against the wrong string), so the banner never cleared and each
click re-POSTed /decision. Rewritten to track outcome per executionId,
clear on success, and block resubmits.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "asks permission but never acts": the approved pct_create
execution failed to parse because the LLM emitted `"privileged":0` /
`"nesting":1` (numbers) into strict `bool` fields, so the container was
never created. Compounded by a hardcoded template name (debian-13.0-1)
that no longer exists on the host, and no way for the agent to read the web.
- flexBool: accept 0/1, "true", bool for privileged/nesting (the exact prod failure)
- pct_create template pre-flight: list host cache, validate/auto-pick newest debian
- pct_create services[] + post_install: one approval provisions a working service
- new http_get MCP tool (sanitized, size-capped, SSRF-guarded) — agent can read repos/sites
- request_execution description: target=host, full JSON schema + example
- SOUL.md: agent CAN fetch the web; prefer one-step provisioning
- default model deepseek-v4-flash -> v4-pro; maxIterations 15 -> 25
- unit tests for flexBool, template resolve, pkg sanitize, HTML sanitize + SSRF block
Verified live on host:strong with a throwaway VMID 999: template auto-resolved,
container created + booted, services installed, post_install ran, then destroyed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Problem: every host/service/lxc/etc. entity_status row was permanently
stuck at 'unknown' since creation. Verified against the live DB:
metric_samples had 17,559 rows, 100% attached to type='check' probe
entities and 0% to any real monitored entity; only 25 check entities
ever had real health written. check_defs.entity_id (the probe's own
bookkeeping entity) and check_defs.target_id (the host/service actually
being observed) were both real fields, but the scheduler wrote
UpsertEntityStatus/InsertMetricSample/emitSchedulerEvent keyed by
entity_id instead of target_id — so every check ran and every result
was real, it just landed on the wrong row. This is the mechanism behind
observed drift: the agent's dashboard/health tools reported the
internal probes' state, never the actual fleet.
Change:
- scheduler.go: runCheck/resolveSignal now resolve targetID from
cd.TargetID (falling back to the check's own id if unset) and write
status/metrics/events there. Signals stay keyed by the check entity,
unchanged, matching their existing resolution logic.
- Added a staleness sweep to housekeeping(): an entity whose last
observation is older than 3x its fastest enabled check's interval
(floor 5m) is marked 'stale' and emits health.stale, so a stalled
scheduler or disabled check_def can no longer look like current data
forever.
- migrations/016: deletes the now-orphaned check-entity entity_status
rows so dashboard/fleet-health rollups stop double-counting probes as
monitored entities. Historical metric_samples on check entities are
left as-is (time-series data, not safe to reattribute).
- openapi.yaml + regenerated gen code: Entity gains health/last_check_at;
'stale' added to the health enum everywhere it's used.
- dashboard.go / GetFleetHealth / nomos's get_health_summary MCP tool:
exclude type='check' entities from rollups.
- nomos/agent.go: replay prior turns' tool_use/tool_result pairs into
the conversation instead of dropping them (previously only final text
was replayed, forcing the agent to re-derive fleet state every turn),
and inject a compact live fleet-health snapshot into the system prompt
each turn so it starts oriented instead of spending an iteration on
discovery.
Risk: config_mutation (schema-adjacent — new migration, no destructive
DDL, additive DELETE only on orphaned rows). No behavior change until
oikos-api/oikos-scheduler/nomos are rebuilt and redeployed.
Verification: go build/vet clean across the repo. Ran this worktree's
own API binary against the live dev Postgres on an alternate port
(read-only from the live containers' perspective) and confirmed
/api/v1/entities now returns health/last_check_at, and the dashboard
health rollup dropped from double-counting to an honest 168 unmonitored
entities (matches reality pre-deploy — the live scheduler hasn't run
the fixed code yet). Confirmed check_defs.target_id correctly maps
multiple checks to host:hubris via direct psql query.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add three new pages completing the control-room web UI:
- Agent activity: polls /agent-activity every 5s, filterable by type/agent
- Knowledge search: FTS over /knowledge/search with snippet + entity links
- Audit trail: browseable audit log with actor/action/entity filters
Enhanced live events page with correlation-id clustering (Groups toggle).
Added fetchAgentActivity/searchKnowledge/fetchAudit to the API client.
11 nav items now cover all planned control-room views.
- phase3.go: RequestExecution now calls InsertEntity before InsertExecution
(executions.entity_id references entities.id via FK constraint).
- mcp/server.go: request_execution MCP tool same fix — inserts entities row
with slug 'exec:<target>:<id8>' before executions insert.
- docker-compose.yml: fix seed OIKOS_SEEDS_DIR from /app/seeds to /seeds
(distroless image COPY destination).
- actuator/ssh.go: custom errorsAs chain broken — all SSH errors classified as SSHErrorOther.
Replaced with standard errors.As + errors.Is.
- scheduler/scheduler.go: all four check functions were stubs returning healthy.
Implemented real HTTP GET, TCP dial, unix.Statfs disk, and TLS cert expiry checks.
- learning/learning.go: uuid.NewV7() called unconditionally before ON CONFLICT upsert.
Now looks up existing pattern first, reuses entity_id.
- notifier/notifier.go: removed dead var_, fixed token regeneration every 15s.
Now skips if token_hash already set.
- phase3.go: removed dead GetPattern+dummy args call in PatchPattern.
- classify.go: removed unused var_ guard.
