fix: blast radius walks dependency direction, and reachability survives no ICMP
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
ci / web (push) Has been cancelled
Desktop App / Build Linux (amd64) (push) Has been cancelled
Desktop App / Attach to Release (push) Has been cancelled

Two things the entity window redesign surfaced but deliberately left alone.

**blast_radius answered the wrong question.** It walked source→target for every
relationship type, but which end of an edge is the dependent differs per type:
"machine hosts container" means the target breaks, while "service depends-on
service" and "ingress routes-to service" mean the SOURCE breaks. Walking
everything forwards was right for hosts/provides and backwards for everything
else — and swept in 2,800+ documents/involves/targets edges of pure bookkeeping,
so the result contained tasks and executions that cannot break.

Direction is now declared per relationship type in seeds/ontology.yaml
(blast_direction: forward | backward | none), the same shape as the entity
types' monitoring: declaration, and defaults to none so an undeclared edge
contributes nothing rather than a confidently wrong answer.

It also needed a modelling fix: `routes-to` names an ingress's BACKEND, so
nothing recorded that all 21 public hostnames are terminated by caddy. A
`served-by` edge type now says so.

  pool:ludo-lvm    2 -> 23   (every container storing on it, then their services)
  lxc:caddy        4 -> 22   (service:caddy, then all 21 ingress routes)
  service:authentik      7   (what authenticates via it)

**Every ping check was reporting down.** Not a host:strong false positive: all
seven, including ws:mac-mini — the Docker host itself. The scheduler runs in
Docker on macOS, whose VM does not route ICMP to the LAN; loopback pings succeed
and every LAN ping fails. Under health aggregation each broken probe dragged its
entity to down.

The question the check exists to answer is "is it reachable", and ICMP is only
one way to ask it. checkPing now falls back to a TCP connect before concluding
anything, which restores an honest verdict for the four hosts that are genuinely
up while leaving the genuinely unreachable ones down.

TestBlastRadiusTerminatesOnCycles asserted the old direction (caddy=1,
authentik=2 — the cycle walked the wrong way); it now asserts the corrected
depths, and its exact-node-count check is relaxed because walking the right way
also surfaces the seed's own real dependents, which are correct answers.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-07-29 09:44:17 +02:00
parent ad29295c93
commit c10f6920cd
7 changed files with 233 additions and 9 deletions

View File

@@ -605,6 +605,9 @@ func checkPing(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkRes
cfg := struct {
Host string `json:"host"`
Count int `json:"count"`
// Port for the TCP fallback below. Defaults to 22; set it for hosts
// that answer on something else (a Home Assistant VM has no sshd).
Port int `json:"port"`
}{}
if len(cd.Config) > 0 {
_ = json.Unmarshal(cd.Config, &cfg)
@@ -638,9 +641,26 @@ func checkPing(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkRes
output, err := cmd.Output()
if err != nil {
// ICMP failing does not mean the host is down — it may mean ICMP is
// simply unavailable from here. On this deployment the scheduler runs
// in Docker on macOS, whose VM network stack does not route ICMP to
// the LAN: loopback pings succeed, every LAN ping fails, and all seven
// ping checks reported "down" for hosts that were demonstrably up
// (including the Docker host itself). Under health aggregation that one
// broken probe was enough to drag each entity to down.
//
// The question this check exists to answer is "is it reachable", and
// ICMP is only one way to ask. Fall back to a TCP connect before
// concluding anything.
if tcpReachable(ctx, cfg.Host, cfg.Port, timeout) {
return checkResult{
health: "healthy",
metrics: map[string]float64{},
}
}
return checkResult{
health: "down", signalKind: "ping",
evidence: fmt.Sprintf("ping %s: %v", cfg.Host, err),
evidence: fmt.Sprintf("no ICMP or TCP response from %s: %v", cfg.Host, err),
err: err,
}
}
@@ -654,6 +674,24 @@ func checkPing(ctx context.Context, cd sqlcgen.ListEnabledCheckDefsRow) checkRes
return checkResult{health: "healthy", metrics: metrics}
}
// tcpReachable reports whether a TCP handshake completes, used as the
// reachability answer when ICMP is unavailable rather than unanswered.
func tcpReachable(ctx context.Context, host string, port int, timeout time.Duration) bool {
if port == 0 {
port = 22
}
if timeout <= 0 {
timeout = 5 * time.Second
}
d := net.Dialer{Timeout: timeout}
conn, err := d.DialContext(ctx, "tcp", net.JoinHostPort(host, strconv.Itoa(port)))
if err != nil {
return false
}
conn.Close()
return true
}
var pingRttRe = regexp.MustCompile(`(?:rtt\s+min\/avg\/max\/mdev|round-trip\s+min\/avg\/max\/stddev)\s*=\s*[\d.]+\/([\d.]+)\/`)
func parsePingLatency(output []byte) float64 {