chore: graph view, dns-zone gap, fleet deploy/cleanup tooling
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled

Graph view: raise the node cap 500 -> 2000 and exclude execution/task audit
rows from the default whole-graph view so the cap is spent on actual topology
rather than ~380 cognition records that crowded out every host/lxc/service.

dns-zone monitoring [dns] -> none: no dns checker exists, so the declaration
only produced unresolvable `unmonitored` noise (requires ontology re-ingest;
coverageSweep now auto-clears the stale signals). Flip back to [dns] when a
checker lands.

Operator tooling: tools/deploy-checks.sh pushes check scripts into guests via
pct push (a pct-exec-routed check runs the script INSIDE the guest), wired
into the post-pull setup-checks hook so guests stay in sync on Proxmox hosts;
scripts/cleanup-orphan-checks.sh (dry-run by default) and
report-stray-test-lxcs.sh retire legacy cruft. VERSION 0.13.0 -> 0.14.0.

Plan: plans/2026-07-29-health-check-reality-and-knowledge-graph.md.
This commit is contained in:
2026-07-29 13:37:27 +02:00
parent 1540f74342
commit b87735a111
8 changed files with 637 additions and 6 deletions

94
tools/deploy-checks.sh Executable file
View File

@@ -0,0 +1,94 @@
#!/usr/bin/env bash
# deploy-checks.sh — push the check scripts into every monitored target.
#
# Background: an ssh-script check runs the script INSIDE the target, so the
# script must exist at /opt/oikos/checks/ on the target itself — not just on
# the proxmox host. The scheduler routes LXC/VM checks through the host via
# `pct exec`/`qm guest exec`, so it never SSHes a guest directly, but the
# script still has to be present inside the guest. This script deploys them.
#
# Run from a Proxmox host (it uses `pct`/`qm`) to populate every local guest,
# and/or pass --host to install on a host/workstation over SSH.
#
# Usage:
# deploy-checks.sh # on a proxmox host: push to every LXC/VM here
# deploy-checks.sh --host ws:mac-mini # ssh-install scripts on a host/workstation
# deploy-checks.sh --checks /path # override the source checks dir
#
# Idempotent: skips a script whose deployed copy is byte-identical.
set -euo pipefail
CHECKS_DIR="${OIKOS_CHECK_DIR:-${HOMELAB_CONTEXT_DIR:-/opt/homelab}/checks}"
DEST=/opt/oikos/checks
die() { echo "deploy-checks: $*" >&2; exit 1; }
deploy_to_guest() {
local id="$1" vm="$2" # vm=0 for LXC, 1 for VM
local kind=pct; [ "$vm" = "1" ] && kind=qm
echo "[deploy-checks] $kind $id"
# Ensure the destination dir exists inside the guest.
if [ "$kind" = "pct" ]; then
pct exec "$id" -- mkdir -p "$DEST" 2>/dev/null || { echo " skip (pct exec failed)"; return; }
else
# qm guest exec returns JSON; best-effort for VMs (guest agent required).
qm guest exec "$id" -- mkdir -p "$DEST" >/dev/null 2>&1 || { echo " skip (qm guest exec failed)"; return; }
fi
local pushed=0 skipped=0
for script in "$CHECKS_DIR"/*.sh; do
local name; name=$(basename "$script")
[ "$name" = "deploy-checks.sh" ] && continue
[ "$name" = "install.sh" ] && continue
if [ "$kind" = "pct" ]; then
pct push "$id" "$script" "$DEST/$name" --perms 755 2>/dev/null && pushed=$((pushed+1)) || skipped=$((skipped+1))
else
# qm has no push; copy via the guest agent file write if available.
qm guest exec "$id" -- /bin/sh -c "cat > $DEST/$name" < "$script" >/dev/null 2>&1 && pushed=$((pushed+1)) || skipped=$((skipped+1))
fi
done
echo " pushed=$pushed skipped=$skipped"
}
deploy_to_host() {
local target="$1" # user@ip or slug resolved by caller
echo "[deploy-checks] host $target"
ssh -o BatchMode=yes -o StrictHostKeyChecking=no "$target" "bash -s" < "$CHECKS_DIR/install.sh" \
|| echo " WARNING: install on $target failed"
}
if [ ! -d "$CHECKS_DIR" ]; then die "checks dir not found: $CHECKS_DIR"; fi
# Host/workstation install mode.
if [ "${1:-}" = "--host" ]; then
[ $# -ge 2 ] || die "--host needs a target (user@ip)"
deploy_to_host "$2"
exit 0
fi
# Proxmox-host mode: push to every local LXC and VM.
if command -v pct >/dev/null 2>&1; then
# LXC containers: ID and status. Skip stopped ones.
while IFS= read -r line; do
[ -z "$line" ] && continue
id=$(awk '{print $1}' <<<"$line")
status=$(awk '{print $2}' <<<"$line")
[ "$status" = "running" ] || { echo "[deploy-checks] skip LXC $id ($status)"; continue; }
deploy_to_guest "$id" 0
done < <(pct list 2>/dev/null | tail -n +2)
else
echo "deploy-checks: 'pct' not found — not a Proxmox host."
echo " On a host/workstation, use: deploy-checks.sh --host user@ip"
exit 0
fi
if command -v qm >/dev/null 2>&1; then
while IFS= read -r line; do
[ -z "$line" ] && continue
id=$(awk '{print $1}' <<<"$line")
status=$(awk '{print $2}' <<<"$line")
[ "$status" = "running" ] || continue
deploy_to_guest "$id" 1
done < <(qm list 2>/dev/null | tail -n +2)
fi
echo "[deploy-checks] done"

View File

@@ -1,13 +1,35 @@
#!/usr/bin/env bash
# setup-checks.sh — deploy check scripts to /opt/oikos/checks on each host.
# setup-checks.sh — deploy check scripts to /opt/oikos/checks.
# Auto-setup hook: tools/setup-*.sh runs after every git pull.
#
# On a plain host/workstation this installs the scripts locally (the pulling
# host). On a Proxmox host it ALSO pushes the scripts into every running LXC/VM
# guest, because an ssh-script check runs the script INSIDE the target — a
# script present only on the host does nothing for a guest reached via
# pct/qm exec. Guest deployment is delegated to deploy-checks.sh.
set -euo pipefail
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
CHECK_SETUP="$CLONE_DIR/checks/install.sh"
DEPLOY="$CLONE_DIR/tools/deploy-checks.sh"
if [ -f "$CHECK_SETUP" ]; then
bash "$CHECK_SETUP" || echo "[setup-checks] WARNING: install.sh exited with code $?"
else
echo "[setup-checks] no checks/install.sh found, skipping"
fi
# On a Proxmox host, keep every guest's scripts in sync too. Best-effort: a
# failing push to one guest must not abort the whole hook. Warn (not skip
# silently) if deploy-checks.sh itself is absent — without it guests never get
# scripts and the pct-exec routing reports every guest check down.
if command -v pct >/dev/null 2>&1; then
if [ ! -f "$DEPLOY" ]; then
echo "[setup-checks] WARNING: $DEPLOY missing — guest scripts will go stale. Commit tools/deploy-checks.sh alongside this hook."
elif [ ! -x "$DEPLOY" ]; then
echo "[setup-checks] WARNING: $DEPLOY not executable — running via bash"
bash "$DEPLOY" || echo "[setup-checks] WARNING: guest deploy exited non-zero (scripts may be stale on some guests)"
else
"$DEPLOY" || echo "[setup-checks] WARNING: guest deploy exited non-zero (scripts may be stale on some guests)"
fi
fi