dtoro
1540f74342
Adds audit_knowledge_graph (MCP tool) and GET /api/v1/audit/drift (endpoint) backed by a shared internal/audit package. One pass surfaces the structural gaps an operator otherwise finds by accident: orphan check entities, checks targeting deprecated/destroyed entities, probes stuck down/unknown, unmonitored declared types, and live edges pointing at destroyed targets. Each finding carries a suggested remediation runbook. Read-only and safe to run unattended. Ships the knowledge-graph-audit skill (SKILL.md + seeded runbook) that interprets the report and routes findings to the lifecycle runbooks.
84 lines
3.9 KiB
Markdown
84 lines
3.9 KiB
Markdown
---
|
|
name: knowledge-graph-audit
|
|
risk_class: read_only
|
|
inputs: []
|
|
verification: "audit_knowledge_graph returns a report with summary.total_findings"
|
|
docs_update_checklist: []
|
|
---
|
|
|
|
# Knowledge-graph audit
|
|
|
|
Goal: validate that the knowledge graph (entities, relationships, checks) and
|
|
the monitoring built on it reflect live reality — without mutating anything.
|
|
Read-only. Run this before trusting health, blast-radius, or coverage answers,
|
|
and whenever something feels off (a healthy host reports `down`, a retired
|
|
service still alarms, the graph looks thin).
|
|
|
|
## 1. Run the drift report
|
|
|
|
Call MCP `audit_knowledge_graph` (or `GET /api/v1/audit/drift`). It returns a
|
|
ranked list of findings, each with `{category, severity, count, entities,
|
|
evidence, suggested_runbook}`, plus a `summary` with totals by category.
|
|
|
|
The DB-side categories:
|
|
|
|
- **orphan_checks** — check entities with truncated/random slugs left by the
|
|
old `shortSlug()` collision bug. Remediation: `scripts/cleanup-orphan-checks.sh`.
|
|
- **dead_checks** — enabled `check_defs` whose target entity is `deprecated`/
|
|
`destroyed`. Remediation: `lifecycle-deprecate-node` / `lifecycle-destroy-node`
|
|
(the scheduler already skips these, but the rows should be retired).
|
|
- **down_checks** — enabled probes reporting `down`. Remediation:
|
|
`service-health-check` (then check whether the failure is real or a
|
|
probe-config/routing problem — see step 3).
|
|
- **unknown_checks** — probes that ran but reported `unknown` (usually a
|
|
misconfigured or not-yet-deployed probe script).
|
|
- **unmonitored** — active entities whose type declares monitoring but have no
|
|
enabled `check_def`.
|
|
- **dangling_edges** — live `hosts`/`provides`/`mounts` edges still pointing at
|
|
destroyed/deprecated targets. Remediation: `lifecycle-destroy-node`.
|
|
|
|
## 2. Triage
|
|
|
|
`severity: critical` (down_checks) first. For each finding, read `evidence` and
|
|
open the entities with `get_entity` / `get_relations` to confirm the diagnosis
|
|
before acting — the report is a pointer, not a verdict.
|
|
|
|
## 3. Common probe-failure causes
|
|
|
|
A `down_checks` finding that is NOT a real outage is usually one of:
|
|
|
|
- **Guest reached wrong** — an LXC/VM check SSHed the guest directly instead of
|
|
routing through its Proxmox host. Confirm with `get_relations` that a `hosts`
|
|
edge exists and the guest has `pve_id`; checks route via `pct exec`/`qm guest
|
|
exec` automatically when both are present.
|
|
- **Script not deployed** — the probe script is absent at `/opt/oikos/checks/`
|
|
inside the target. Remediation: redeploy via `tools/deploy-checks.sh`.
|
|
- **macOS host** — a workstation check used the wrong SSH user or a Linux-only
|
|
script flag. The scheduler resolves `user: dtoro` from the entity attribute.
|
|
|
|
## 4. What this audit does NOT cover (follow-ups)
|
|
|
|
Live-infrastructure discovery is out of scope for the DB report and must be done
|
|
manually until that machinery lands:
|
|
|
|
- **Ghost vs missing entities** — cross-check `pct list` / `qm list` (on
|
|
`host:hubris`, `host:strong`) and `docker ps` against `list_entities`. A guest
|
|
with no entity, or an entity with no guest, is drift.
|
|
- **Misplaced parent** — compare each guest's actual Proxmox host against its
|
|
`hosts` edge (migrations leave these stale).
|
|
- **Undeployed scripts** — per-guest `/opt/oikos/checks/` presence.
|
|
- **Unmodeled certs** — Caddy-managed TLS certs with no `certificate` entity.
|
|
- **Seed drift** — run `oikos export` and `git diff seeds/` to find
|
|
runtime-created entities not in version control.
|
|
|
|
## 5. Acting on findings
|
|
|
|
This skill is read-only — make no changes here. Route each confirmed finding to
|
|
its `suggested_runbook`, classify the action against `seeds/policy.yaml`, and
|
|
proceed through the normal lifecycle/approval flow. Re-run the audit afterward
|
|
to confirm the finding cleared.
|
|
|
|
Docs-update checklist: none — the audit reads state; it changes nothing. If a
|
|
finding reveals stale `risk_notes` or a wrong `doc_page`, fix `inventory.yaml`
|
|
in that remediation session.
|