Commit Graph

19 Commits

Author SHA1 Message Date
b02f94bfd4 feat: Phase 4 — RelationshipService, postgres RelRepo, converged edges
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Problem: relationship create/end existed as three drifted copies
(HTTP CreateRelationship/EndRelationship, MCP create_relationship/
end_relationship) with inline SQL, no ontology edge validation on
either path, and no audit on the MCP path.

Change:
- Internal/adapters/postgres/repositories.go: RelRepo implements
  ports.RelationshipRepository (Create/End/ListFor) over the pool,
  with in-tx upsert + audit/event side effects on Create.
- Internal/core/app/relationships.go: RelationshipService validates
  edges against the cached ontology TypeTree (tree.ValidateEdge) and
  delegates the tx to the repository. The adapter resolves slug→entity
  and extracts types before calling the service.
- HTTP CreateRelationship: resolves source/target via ReadModels,
  passes resolved types to RelationshipService for edge validation.
  EndRelationship calls the service directly (audit stays in the
  adapter for End — a simple toggle with no ontology check).
- MCP create_relationship/end_relationship: rewired to the service
  (pool resolves entity IDs inline for the tool handlers; the service
  validates edges and writes audit). The MCP path now gets ontology
  validation and audit coverage for the first time.
- Composition root: RelationshipService built with RelRepo + Ontology
  and wired through httpapi.NewHandler, ListenAndServe, and MCP
  constructors.

Verification: go build/vet, full test suite (19 pkgs, DB integration
postgres+mcp green).
2026-08-16 00:09:40 +02:00
9e3783734e feat: Phase 3b — EntityService, postgres EntityRepository, converged mutations
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Problem: entity mutations (create/update/state) existed as three drifted
copies — HTTP CreateEntity/PatchEntity, MCP create_entity/
update_entity_attributes/set_entity_state — each with its own inline
SQL, its own validation subset (MCP validated lifecycle states, HTTP
did not; HTTP patched attributes without regenerating derived checks,
MCP did; MCP wrote no audit trail), the exact drift ADR 0016's first
vertical slice exists to collapse.

Change:
- internal/core/ports: DerivedCheck, Idempotency (adapter-owned request
  hash + cached-body renderer so the replay record commits in the
  create's transaction), IdempotentResponse + GetIdempotent read,
  AuditEntry gains Method/Path/CorrelationID, Event gains
  CorrelationID; EntityUpdateInput carries ExpectedVersion +
  RederiveChecks (derivation for updates runs repo-side: the graph
  host fallback reads relationships through the open tx).
- internal/adapters/postgres/repositories.go: EntityRepo (Create/
  Update/SetState/reads/idempotency) preserving the load-bearing
  check-then-act invariants in-tx: version WHERE-clause, declared
  transitions + preconditions (ValidateTransition), duplicate-slug
  mapping, audit/event/writeCheck all inside one BEGIN…COMMIT.
  OntologyRepo: TTL-cached OntologyStore.
- internal/core/app/entities.go: EntityService — ontology validation
  (type exists, concrete, state declared — the stricter MCP rule now
  governs both surfaces), default-state resolution, id generation,
  derivation for creates, audit/event construction, idempotency
  pass-through.
- httpapi CreateEntity/PatchEntity rewired to the service; PATCH now
  regenerates derived checks (the A2 parity gap). MCP create/update/
  set-state tools call the same service — and now write audit + event
  rows like the HTTP surface always did.
- Integration-test seed paths fixed for the adapters/postgres package
  depth (../../../seeds).

Pre-existing failures documented: TestAPIEndToEnd (entity_types 60 vs
59; 501-endpoint now 200), TestClientLifecycleEndToEnd, TestPhase3*
rows — verified failing identically at ec11956 (scratch approval-
notifier commit test drift), unrelated to this change. All mutation
integration tests (create/patch/idempotency/audit/regeneration) pass.

Verification: make test-db (postgres + mcp green, httpapi green except
the pre-existing set), full non-DB suite (19 pkgs), golangci on new
packages — 0 issues.
2026-08-15 23:38:21 +02:00
d4f5084a6d refactor: Phase 3a — absorb checkdefaults into core/app as pure Derive
Problem: check derivation logic lived in internal/checkdefaults with
the pure decision logic (buildKind, address/user/port resolution)
interleaved with tx I/O (entity_status insert, graph host fallback,
check upserts) — and internal/db importing it was the plan's called-out
inverted dependency.

Change:
- internal/core/app/checkdefaults.go: Derive(tree, target, lookup) —
  the full derivation (monitoring overrides, host fallback via an
  injected HostLookup thunk, per-kind builders) with zero I/O imports.
  Types renamed for the app surface: CheckTarget, CheckDef,
  DeriveResult, Skip; LogDeriveResult.
- internal/adapters/postgres/checks.go absorbs the I/O half:
  EnsureChecks (entity_status row + upsert loop), writeCheck, and
  hostViaGraph. The db→checkdefaults edge is gone — adapters→core is
  the ADR 0016 direction (the Phase 7 SeedService note anticipated
  this; the inversion is fixed a phase early).
- seed.go pending-checks loop uses app.CheckTarget + EnsureChecks;
  mcp formatting/tests follow the renamed types; both test files moved
  to internal/core/app.
- Deliberate behavior note: a hostViaGraph read failure inside the
  thunk now logs a warning and degrades to 'skipped: no address'
  instead of aborting the whole entity-create tx — a monitoring
  derivation gap is visible (warn log + coverage sweep) and self-heals
  on the next mutation; failing the create over a graph-read blip was
  disproportionate.

Verification: go build/vet, full test suite green (app tests exercise
every buildKind branch at their new home).
2026-08-15 23:13:45 +02:00
64f7d54011 feat: Phase 2 — ports package, secrets port move, postgres adapter move
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
Problem: the hexagon's Phase 2 (plans/2026-08-15-hexagonal-architecture.md)
must give the use-cases-to-be their contract surface: driven-port
interfaces, test fakes, the secrets interface moved into core, and the
postgres package inside the adapters tree — before the first vertical
slice (Phase 3) can wire a composition root.

Change:
- internal/core/ports: full driven-port catalog per plan §3.3 —
  repositories as transaction-scoped aggregates whose inputs carry
  derived checks, audit, and events (§3.6), plus CommandExecutor,
  TargetResolver, Checker, Secrets, EventPublisher, Provisioner.
  Port-local payload types (Event, AuditEntry, CheckDef, KnowledgeEntry,
  ExecResult) keep signatures off infrastructure; TypeTree aliases
  internal/ontology (pure over domain) until checkdefaults is absorbed.
  ReadModels intentionally not declared yet — it materializes with the
  Phase 3 slice and grows as report handlers rewire.
- secrets.Backend is now an alias of ports.Secrets; implementations
  (Infisical, SOPS, Manager) unchanged. mcp's local secretBackend
  subset is deleted; tool constructors take ports.Secrets.
- internal/db → internal/adapters/postgres (mechanical import rewrite;
  package identifier stays db until the Phase 3 repository split).
  sqlc.yaml, Makefile, golangci exclusions, and docs follow the move;
  make generate-check verified.
- internal/adapters/ssh: Executor implements ports.CommandExecutor over
  the actuator dial pool + RunStreaming (10-min default timeout carried
  over from the httpapi path).
- internal/adapters/remote: Resolver implements ports.TargetResolver
  delegating to internal/remote (still pool-based; drops onto
  ports.EntityRepository when repositories land in Phase 3 — documented
  transitional import).
- internal/core/ports/portstest: importable fakes — in-memory
  EntityRepo (with check-then-act SetState, side-effect recording),
  RecordingExecutor, FakeChecker, SpyPublisher; port-satisfaction
  guards; tests.

Risk: ports are declared ahead of implementations — signatures firm up
per phase as slices land (documented in the package doc); the
remote→postgres transitional import is explicit and dissolves in
Phase 3.

Verification: go vet, make test (race, 19 packages), generate-check,
golangci on core+adapters — 0 issues; full-repo baseline down
365→344.
2026-08-15 22:56:56 +02:00
7236c46e5c 0.29.1 — review-fix round on E3: RunOutput, sshKeyPath fallback, RunStreaming consolidation, signer cache, stderr in errors
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-08 23:01:10 +02:00
75c0848a6f 0.29.0 — code-quality refactor (plan E1–E5): file splits, sqlc migration, SSH unification, test coverage
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
E1: split monolithic files — cmd/nomos (main.go → server.go + mcp.go + workers.go),
    internal/mcp/tools.go → entity_tools/ops_tools/knowledge_tools/analysis_tools,
    internal/httpapi/impl.go → domain files (entities, events, signals, ontology,
    fleet_health, client_context, client_lifecycle, entity_mutations, query_audit).
E2: migrate raw pool.Exec queries to sqlc (entities/relationships queries + generated).
E3: unify SSH — consolidate crypto/ssh dial into actuator/client.go (+client_test).
E4/E5: add tests — db/lifecycle, checkdefaults/build, ontology/preconditions, policy/risk.
2026-08-08 22:47:06 +02:00
c9d506b0f8 0.27.0 — Infisical hardening: runtime refresh, audit logging, CLI verify/audit, startup verification, SSH host key verification, interface consolidation
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
2026-08-05 23:51:01 +02:00
e3449b24c1 feat: wire Infisical secret store into API server and MCP tools
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
- Wire secretsManager in NewHandler() — instantiate InfisicalBackend
  when OIKOS_INFISICAL_SITE_URL is set (previously always nil)
- Add get_secret, list_secrets, set_secret MCP tools with nil-backend
  graceful degradation
- Add oikos secret get|set|list CLI subcommands for Infisical
- Fix Set() bug: create-before-update so new keys are created;
  add Type: "shared" to Update so it finds the right secret;
  disable SDK cache so Get returns fresh data after Set
- Clean enrollment response: remove fake infisical_client_id/
  infisical_client_secret stubs, store age key in Infisical for real
2026-08-05 23:03:27 +02:00
1d0197da69 0.25.1 — get_health_summary destroyed filter + create_entity footgun doc
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
- get_health_summary: filter out state=destroyed entities (was noise
  from 20+ destroyed test LXCs, deprecated services, etc.)
- create_entity: document the monitoring footgun in the tool description
  (creating a type=check entity does NOT wire a check_def; the correct
  path is update_entity_attributes with monitoring + url attributes)
2026-08-05 16:07:59 +02:00
1b9c761274 implements plan: agent execution safety — QEMU guest agent gate + health guard + policy docs
I — run pre-flights QEMU guest agent before queueing VM execution
  classifyAndGate now checks vm: targets for qemu_guest_agent attribute.
  If not_running/missing, returns immediate error instead of queuing forever.

II — policy.yaml: documented host-mutation classifier rule
  Added comment clarifying that host-level package/kernel mutations
  (apt-get install, dpkg, systemctl enable) always classify as
  config_mutation and thus need operator approval.

III — health attribute read-only in update_entity_attributes
  Strips scheduler-owned keys (health, last_check_at, last_check) from
  attribute updates with a clear message directing agents to
  get_health_summary / list_checks instead.

IV — Recorded discovered dependency edges
  vm:zimaos → depends-on → lxc:nfs-export (NFS /media/library mount)
  vm:zimaos → depends-on → host:strong (NFS /media/ludo-library mount)

Also updated the run tool description to mention both guardrails.
2026-08-05 15:25:14 +02:00
a126cfa710 0.24.0 — MCP tool improvements: type filter for get_relations, health filter for get_health_summary, live HTTP probe for ping_service
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
get_relations now accepts an optional 'types' (comma-separated) parameter
to filter relationship types — filters out the noisy exec/targets edges
that previously drowned useful host/provides edges.

get_health_summary now accepts an optional 'health' (comma-separated)
parameter to return only entities in specific health states (e.g.
'health=down,stale') instead of the full 100+ entity list.

ping_service now:
- Falls back to e.attributes->>'public_host' when 'url' is not set
  (covers LXCs that only have public_host in the graph)
- Performs a live HTTP HEAD probe against the resolved URL, returning
  the actual status code instead of just the scheduler's stale health
  state

Also: fixed matrix.hubris.network DNS record (was pointing to dead VPS),
pruned 6 dead graph edges, wired url attributes on 7 LXCs, added VPS
HTTP monitoring check, and resolved the 18k-occurrence unmonitored signal.

This session's audit is documented as
document:nomos/2026-08-05-dns-monitoring-improvements-for-strong-hosted-services.
2026-08-05 15:12:04 +02:00
0dd8c28815 feat: MCP ping tool, tightened descriptions, and Hermes client docs
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
- Add  MCP tool — lightweight connectivity check returning server
  identity, no DB hit (resolves agent connection-test friction)
- Tighten 6 tool descriptions (get_relations, get_health_summary,
  query_metrics, get_trend, get_event_timeline, ping) to be searchable
  in the first 8-12 words
- Document Hermes MCP client setup in ADR-0012 with token security caveat
- Move completed plan to plans/done/
2026-08-05 00:21:56 +02:00
4e294b3630 0.22.0 — full MCP agent surface: 19 new tools, async run, session reliability
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Stage 1 — Foundation:
- Target validation: iptables + systemctl/docker target-type gates
- Async run for long-running commands (sleep/wait/poll loops)
- audit_log.session_id plumbing (SQL, sqlcgen, 18 call sites)

Stage 2 — External agent observe (11 tools):
- get_dashboard_summary, get_ontology, list_checks, list_executions
- get_knowledge_revisions, get_knowledge_duplicates, get_knowledge_orphans
- list_knowledge_tags, list_entity_sessions, find_entities_by
- 3 resource templates: oikos://entity/{slug}, knowledge/{id}, execution/{id}

Stage 3 — Nomos reliability:
- complete_task(success) refused without verification (upgraded from warn)
- sessionHasPlan excludes replaced steps (forces propose_plan after reopen)
- Bash syntax validation in run() (rejects literal \n, flag-space typos)
- Scope gate in SOUL.md (ask before pivoting to unrelated subsystem)

Stage 4 — External agent act (9 mutation tools):
- ack_signal, resolve_signal, mute_signal, cancel_execution
- update_check, delete_knowledge, restore_knowledge
- merge_knowledge, rename_knowledge_tag
2026-08-04 23:51:55 +02:00
20adb89650 v0.18.0: MCP entity-graph CRUD, lifecycle validation, curl -o /dev/null fix
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
- create_entity, set_entity_state, end_relationship MCP tools
- update_entity_attributes now triggers check derivation via EnsureEntityChecks
- shared db.EnsureEntityChecks + db.ValidateTransition hooks (HTTP + MCP parity)
- curl -o /dev/null now classified read_only (was config_mutation)
- db.ErrTransitionInvalid sentinel for HTTP error-type accuracy
- SOUL.md: capability escalation, self-grounding, exploration budget rules
- Runbook: oikos check lifecycle for agent self-knowledge
2026-08-04 08:52:08 +02:00
0929c17cbb feat(mcp): discover_infra_drift — live Proxmox vs DB guest reconciliation
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
The DB-only audit_knowledge_graph can't see guests running in Proxmox that
have no entity, or entities whose pve_id is no longer live — the drift that
the stray test LXCs were a symptom of. discover_infra_drift enumerates running
guests via pct/qm list on every proxmox host (over the same SSH/pct path the
checks use) and diffs against the DB: returns missing (live, no entity) and
ghost (DB, not live). Read-only.

Companion to audit_knowledge_graph; the skill now runs both and treats the
remaining checks (misplaced parent, undeployed scripts, seed drift) as manual.
2026-07-29 20:36:02 +02:00
1540f74342 feat(audit): read-only knowledge-graph drift report + skill
Adds audit_knowledge_graph (MCP tool) and GET /api/v1/audit/drift (endpoint)
backed by a shared internal/audit package. One pass surfaces the structural
gaps an operator otherwise finds by accident: orphan check entities, checks
targeting deprecated/destroyed entities, probes stuck down/unknown, unmonitored
declared types, and live edges pointing at destroyed targets. Each finding
carries a suggested remediation runbook. Read-only and safe to run unattended.

Ships the knowledge-graph-audit skill (SKILL.md + seeded runbook) that
interprets the report and routes findings to the lifecycle runbooks.
2026-07-29 13:37:16 +02:00
e055a7c6ce feat(nomos): session-review improvements (P0/P1/P2 from 2026-07-20 audit)
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Classifier now unwraps pct exec / qm guest exec / bash -c / sh -c / sudo
and env-var assignments before classification, so read-only inspection
wrapped in pct exec no longer escalates to config_mutation. curl GET
(default method, no -d/-F/-T/-o/>) is read-only. Eliminates the three
duplicate rclone sessions (a51e2086, 8acea2e3, cb8c8a4a) that bounced
off the classifier for the same goal.

New classify_command MCP tool: command-scoped preflight that returns the
exact risk class run would assign. Documented in SOUL.md with guidance
to pre-classify before run when the verdict is uncertain.

set_goal surfaces prior partial/failed sessions from the last 24h so the
agent picks up the thread instead of rediscovering it.

completeTask auto-closes in-flight plan steps (pending/running -> done
on success, skipped on partial/failure), so one-step plans no longer
need the per-step running->done dance right before completion.

Migration 021 adds blocker + closed_at to agent_sessions. completeTask
sets closed_at once and derives a structured blocker reason
(approval_timeout, user_abandoned, classifier_overreach, model_refusal,
tool_error, ...) from the last assistant message.

/sessions list now carries message_count, tool_call_count,
duration_seconds (server-side aggregates — no more N+1 transcript
fetches to audit a fleet). GET /sessions/{id} returns both metadata
and messages. New query params filter + paginate: outcome, status,
entity_id, blocker, since (RFC3339 or Go duration), cursor, limit.

Titles now prefer the goal when set; sessions without a goal fall back
to the first assistant text.

New GET /sessions/{id}/tool_calls flat view for audit scripts.

Plan: plans/2026-07-20-session-review-ten-sessions.md. VERSION 0.7.12 -> 0.7.13.
2026-07-20 11:32:31 +02:00
544afae77f feat(nomos): retry cap, vm: targets, inspect_path, goal supersession, runbooks
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled
Session-review implementation for the three sessions audited in
plans/2026-07-18-session-review-three-sessions.md. v0.7.11 → v0.7.12.

P0.1 — retry cap + investigate-before-retry (cmd/nomos/retrycap.go,
agent.go): after 3 identical failing run calls in a single turn, refuse
to dispatch the call again and return a directive to investigate *why*
(ps/strace/lsof) or surface the blocker. Per-turn scope so a fresh turn
after the operator responds can retry once more. Session 1e9c7691's 20+
identical chown retries (knfsd held a kernel lock on the exported NFS
dir) is the direct motivation.

P0.2 + P1.8 + P2.10 — SOUL.md guidance: hung command is not a failed
command (investigate before retry); ask before proposing a multi-step
migration; multi-goal sessions summarize the arc not just the last goal.

P1.3 — two new runbook entities in seeds/knowledge.yaml:
  - nfs-exported-dir-mutation-hang (the knfsd fchownat lock procedure:
    killall → exportfs -u → mutate → exportfs -a → verify)
  - netbird-mgmt-oidc-race-after-upgrade (docker restart netbird-mgmt
    after ~30s for the traefik/authentik OIDC race)

P1.4 — setGoal emits task.superseded event when prior goal is overwritten
by a different goal (store.go, TestSetGoal_SupersededEvent). Session
55927f0a had two set_goal calls with the first silently abandoned.

P1.5 — inspect_path MCP tool: runs mount/df/ls/stat for one path across
up to 8 targets in one parallel call, replacing the 15+ run-call
fact-gathering fan-out sessions 1 and 2 each spent on cross-target path
tracing (tools.go, server.go: inspectPathAcrossTargets, inspectOneTarget).

P1.6 — vm: target support in run via qm guest exec (no more SSH-hop
with nested quoting). Extracted shared resolveProxmoxHostSlug for
LXC + VM, with hosts-relationship fallback when attributes.host is
absent (server.go, tools.go). Session 55927f0a's SSH-hop workarounds
for vm:zimaos are the direct motivation.

Deferred (documented in plan): P1.7 (approval window auto-extend on
timeout) and P2.9 (long-running command PENDING detection) — both
addressed at lower cost by the retry cap. Session 3's poll-after-timeout
pattern already works; the cap protects against the failure mode.
2026-07-19 00:09:39 +02:00
fb39a48bef refactor: split phase3.go + extract MCP tool registry (R4)
internal/httpapi/phase3.go (2627 lines, 12+ resource domains) split into
15 per-resource files:
- actuator.go: SSH execution machinery (initSSH, sshExec, resolveRunTarget,
  executeApprovedAction, jsonErr, gatewayPreflightPassed, resolveTemplate)
- checks.go, classifications.go, executions.go, approvals.go, patterns.go,
  skills.go, approval_rules.go, autonomy.go, risk_classes.go,
  relationships.go, entity_types.go, metrics.go, agent_activity.go,
  helpers.go — one file per resource domain, each with its own imports.

internal/mcp/server.go: newServer (708 lines, 33 inline tool registrations)
refactored to a registry pattern:
- internal/mcp/tools.go (new): toolReg struct + allTools() returning all 33
  tool definitions. Handler logic moved verbatim — no changes to tool names,
  descriptions, schemas, or behavior.
- server.go: newServer is now 9 lines (iterate registry, AddTool each).
  -699 lines.

No function logic, names, or signatures changed. go vet, build, and all
tests pass (httpapi, mcp, db, policy).
2026-07-17 22:41:40 +02:00