Two new subcommands (B1+H1 + C1+C2 from the apt-sweep backlog):
* `homelab apt-audit [--target HOST]`
Per-host pre-flight table reporting dpkg-interrupted state, holds,
upgradable count, non-apt binaries in system paths, and DNS health.
Exits nonzero if any target has unconfigured packages (would catch the
latent dpkg-interrupted state we found on caddy + nextcloud before
touching them).
* `homelab apt-upgrade [--target HOST | --all] [--status] [--force]`
Launches `apt update && apt upgrade` inside a transient systemd unit
(`apt-upgrade-<hostname>`) on each target via `systemd-run --collect`,
so the run survives ssh teardown. Output tee'd to
`/var/log/homelab-apt-upgrade.log`. Apt configured with
`Acquire::Retries=3` + `ForceIPv4=true` to absorb transient mirror
blips (would have prevented the jellyfin ffmpeg7 fetch failure that
rolled back 131 packages mid-Wave-3).
Pre-flight audit gate refuses to run on hosts with dpkg-interrupted
state; override with --force.
`--status` mode prints each target's unit state and tails the upgrade
log; safe to run before, during, and after.
Also fixes a latent bug in `hubris_ssh()`: the function joined hubris's
`lan_ip` (192.168.8.77) with the netbird-only port 22022. Port 22022
listens on hubris's netbird interface ONLY — not on the LAN interface,
not even via the netbird subnet route. Result: every off-LAN client
hitting `homelab pct/logs/restart/status` got "Connection refused".
Now uses the netbird FQDN when netbird_port is in use; falls back to
host_address otherwise.
Targets exclude VMs and workstations (they have their own update flows).
Standard apt targets = hubris + every LXC (kind=lxc with a pve_id).
Validated 2026-05-21 on republic-laptop:
- apt-audit reported clean dpkg + 40 upgradable on hubris (the still-
deferred PVE/kernel packages) + 0 on every LXC, DNS ok across the
fleet
- apt-upgrade --target mule-photos-new launched the unit, ran apt to
completion (0 upgrades pending), log captured with =START= / =END
rc=0= markers, --status reports unit + log correctly
- --all --status fans out across all 16 targets without issue
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dtoro
2ff7263f3d