Files
oikos/bootstrap.sh
dtoro d80a394b7f
Some checks failed
ci / build-test (push) Has been cancelled
ci / docker-build (push) Has been cancelled
docs: fix plan/repo drift, retire dead Goose+Nomos and Caveman tooling
Documentation and repo-hygiene pass following the client/server split:

Plan drift (audited all other active plans against current code):
- oikos-gaps-and-improvements.md: mark Section C and D.5 resolved (both
  described cmd/hermes, renamed to cmd/nomos with a real LLM loop since);
  refresh ~10 stale file:line citations; fix tool-count (33, not 28).
- liveness-drift-and-ux-cohesion.md: fix stale default-model claim (now
  deepseek-v4-pro since 2026-07-10) and "not yet deployed" status.
- nomos-agent-code-review.md: fix C1's citation (one unauthenticated route
  to nomos now, not two, after the client/server split).
- wails-desktop-app.md: record the production deploy outcome.

Repo structure: added missing directories to README/CONTRIBUTING layout
tables (checks/, tools/, cmd/webhook/, docs/operations/), fixed a broken
link, added ADR 0015 documenting the auth/CORS/client-split model (there
wasn't one despite CONTRIBUTING's own process requiring it), normalized
ADR 0013/0014's format drift, added an Authentication section to
AGENTS.md/CLIENTS.md (every example call was missing the now-required
bearer header).

Retired the Goose+Nomos workstation flow (bootstrap.sh --with-nomos,
tools/setup-nomos-soul.sh, .agents/operations/nomos-agent.md) and the
Caveman auto-install tooling (tools/setup-caveman.sh, tools/caveman/) —
both superseded by the production containerized Nomos agent, which has
never used either. Kept .agents/shared/caveman.md itself (the terse
writing-style convention agents still follow by reading it).

Deleted the orphaned legacy Python oikos/ directory — nothing imports it,
and bin/homelab (the CLI it was kept for) no longer exists in the repo.

Rewrote .agents/operations/agent-enrollment.md (365 -> ~110 lines) and
commands.md to match the current architecture instead of the retired
`homelab` CLI; migrated the still-true networking prerequisites (Netbird,
split-horizon DNS, SSH key distribution) into the knowledge base as a
runbook via upsert_knowledge rather than duplicating them in markdown.
Updated all 10 .agents/skills/ runbooks referencing the dead CLI with
their real MCP tool / REST API equivalents, or flagged them as needing
verification where no equivalent is confirmed yet.

Two real bugs found and fixed, not just docs:
- The tools/setup-*.sh auto-setup glob was tools/*.setup.sh in THREE
  places (tools/post-pull.sh, bootstrap.sh, and internal/httpapi/impl.go's
  GetClientContext handler) since the mechanism's introduction on
  2026-06-02 — never matched any real filename, so no client has ever
  picked up an auto-setup script via git-pull or the context-poller sync.
  Fixed all three; the Go server-side fix is the one that actually matters
  since it's what the current context-poller mechanism depends on.
- bootstrap.sh removed dead vestigial --gitea-token/--gitea-user flags
  (parsed, never consumed) left over from an earlier clone-based model.

Also flagged, not fixed (documented as an open gap in
client-enrollment/SKILL.md): bootstrap.sh tells a freshly-enrolled client
to call POST /api/v1/clients/{slug}/activate to finish enrollment, but
that route doesn't exist in api/openapi.yaml — EnrollClient sets entities
to provisioning and nothing currently transitions them to active.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 18:19:41 +02:00

324 lines
13 KiB
Bash
Executable File

#!/usr/bin/env bash
# bootstrap.sh — enroll a new thin client into the homelab via Oikos API.
#
# Thin client model (rev 2): no git clone, no sync timer. Fetches only the
# agent orientation files (CLIENTS.md, AGENTS.md, OIKOS.md) and tooling
# (checks) from the raw Gitea URL. Enrolls via the Oikos API to receive an
# age keypair and Infisical machine identity. A lightweight context poller
# replaces the old 5-minute git pull.
#
# Usage:
# curl -fsSL https://git.hubris.network/dtoro/oikos/raw/main/bootstrap.sh \
# | sudo bash
# curl ... | sudo bash -s -- --with-mcp # wire Claude's .mcp.json
# curl ... | sudo bash -s -- --dry-run # show what would happen
#
# Prerequisites:
# - running as root
# - OS is Linux or macOS
# - age, python3, curl, jq installed
# - at least one mesh (netbird OR tailscale) connected
# - entity must already exist in DB (planned or provisioning state)
set -euo pipefail
# ── defaults ─────────────────────────────────────────────────────────
REPO_RAW_URL="${HOMELAB_RAW_URL:-https://git.hubris.network/dtoro/oikos/raw/main}"
OIKOS_API_URL="${HOMELAB_OIKOS_URL:-https://oikos.hubris.network/api/v1}"
CLONE_DIR="${HOMELAB_CONTEXT_DIR:-/opt/homelab}"
MCP_URL="${HOMELAB_MCP_URL:-https://mcp.hubris.network/mcp}"
WITH_MCP=0
DRY_RUN=0
# ── helpers ──────────────────────────────────────────────────────────
log() { echo "[oikos] $*"; }
warn() { echo "[oikos] WARN: $*" >&2; }
die() { echo "[oikos] FATAL: $*" >&2; exit 1; }
dry() { if [ "$DRY_RUN" -eq 1 ]; then log "[DRY-RUN] $*"; return 0; else "$@"; fi }
resolve_hostname() {
if [ "$(uname -s)" = "Darwin" ]; then
HNAME=$(scutil --get LocalHostName 2>/dev/null) || HNAME=$(hostname -s)
else
HNAME=$(hostname -s)
fi
if [ -z "$HNAME" ]; then
die "cannot resolve hostname"
fi
}
detect_mesh_ip() {
MESH_IP=""
# Try netbird first
if command -v netbird >/dev/null 2>&1; then
MESH_IP=$(netbird status 2>/dev/null | grep -oE '100\.[0-9]+\.[0-9]+\.[0-9]+' | head -1) || true
fi
# Fall back to tailscale
if [ -z "$MESH_IP" ] && command -v tailscale >/dev/null 2>&1; then
MESH_IP=$(tailscale ip -4 2>/dev/null) || true
fi
# Fall back to LAN
if [ -z "$MESH_IP" ]; then
MESH_IP=$(ifconfig 2>/dev/null | grep -Eo 'inet 192\.168\.8\.[0-9]+' | awk '{print $2}' | head -1) || true
fi
if [ -z "$MESH_IP" ]; then
die "cannot detect mesh IP (netbird/tailscale/LAN)"
fi
}
# ── parse args ───────────────────────────────────────────────────────
while [ $# -gt 0 ]; do
case "$1" in
--with-mcp) WITH_MCP=1 ;;
--dry-run) DRY_RUN=1 ;;
*) die "unknown flag: $1" ;;
esac
shift
done
# ── preflight ────────────────────────────────────────────────────────
[ "$(id -u)" -eq 0 ] || die "must run as root"
OS=$(uname -s)
case "$OS" in
Darwin) ;;
Linux) ;;
*) die "unsupported OS: $OS" ;;
esac
resolve_hostname
log "hostname: $HNAME"
# ── install dependencies ─────────────────────────────────────────────
for cmd in curl jq age python3; do
if ! command -v "$cmd" >/dev/null 2>&1; then
case "$OS" in
Darwin) dry brew install "$cmd" 2>/dev/null || die "install $cmd manually" ;;
Linux)
if command -v apt-get >/dev/null 2>&1; then
dry apt-get update -qq && dry apt-get install -y -qq "$cmd"
elif command -v dnf >/dev/null 2>&1; then
dry dnf install -y "$cmd"
else
die "install $cmd manually"
fi
;;
esac
fi
done
# ── mesh connectivity ────────────────────────────────────────────────
log "checking mesh connectivity..."
if command -v netbird >/dev/null 2>&1; then
dry netbird up --management-url https://netbird.hubris.network || true
fi
detect_mesh_ip
log "mesh IP: $MESH_IP"
# ── create homelab directory ─────────────────────────────────────────
dry mkdir -p "$CLONE_DIR"/{.agents/shared,bin}
# ── fetch agent orientation files (thin client — no git clone) ──────
log "fetching agent files..."
for f in CLIENTS.md AGENTS.md .agents/OIKOS.md .agents/shared/caveman.md .agents/shared/writing-style.md; do
url="$REPO_RAW_URL/$f"
dest="$CLONE_DIR/$f"
if curl -fsSL --connect-timeout 10 "$url" -o "$dest.tmp" 2>/dev/null; then
mv "$dest.tmp" "$dest"
log " + $f"
else
warn " - $f (not found at $url)"
fi
done
# Ensure expected path for caveman
[ -f "$CLONE_DIR/.agents/shared/caveman.md" ] || \
cp "$CLONE_DIR/caveman.md" "$CLONE_DIR/.agents/shared/caveman.md" 2>/dev/null || true
# ── fetch tools ──────────────────────────────────────────────────────
log "fetching tools..."
for tool in setup-checks.sh post-pull.sh; do
url="$REPO_RAW_URL/tools/${tool}"
dest="$CLONE_DIR/tools/${tool}"
dry mkdir -p "$(dirname "$dest")"
if curl -fsSL --connect-timeout 10 "$url" -o "$dest.tmp" 2>/dev/null; then
mv "$dest.tmp" "$dest"
chmod +x "$dest" 2>/dev/null || true
log " + tools/$tool"
else
warn " - tools/$tool (not found)"
fi
done
# ── detect control-plane (use localhost if API is reachable directly) ─
if [ -z "${HOMELAB_OIKOS_URL:-}" ]; then
if curl -s --connect-timeout 2 http://localhost:8090/api/v1/health >/dev/null 2>&1; then
OIKOS_API_URL="http://localhost:8090/api/v1"
log "API reachable on localhost, using direct connection"
fi
fi
# ── enroll via Oikos API (replaces archived Python secrets-issuance) ─
log "enrolling via Oikos API..."
ENROLL_RESP=$(curl -s --connect-timeout 10 -X POST "$OIKOS_API_URL/clients/enroll" \
-H "Content-Type: application/json" \
-d "{\"slug\":\"ws:$HNAME\",\"hostname\":\"$HNAME\",\"mesh_ip\":\"$MESH_IP\"}")
if echo "$ENROLL_RESP" | jq -e '.age_private_key' >/dev/null 2>&1; then
AGE_PRIVKEY=$(echo "$ENROLL_RESP" | jq -r '.age_private_key')
AGE_PUBKEY=$(echo "$ENROLL_RESP" | jq -r '.age_public_key')
# Write age keypair
dry mkdir -p /etc/age
dry bash -c "echo '$AGE_PRIVKEY' > /etc/age/key.txt"
dry chmod 600 /etc/age/key.txt
log " + age keypair written to /etc/age/key.txt"
log " pubkey: $AGE_PUBKEY"
# Write Infisical identity if provided
INF_CLIENT=$(echo "$ENROLL_RESP" | jq -r '.infisical_client_id // empty')
INF_SECRET=$(echo "$ENROLL_RESP" | jq -r '.infisical_client_secret // empty')
if [ -n "$INF_CLIENT" ] && [ -n "$INF_SECRET" ]; then
dry mkdir -p /etc/infisical
dry bash -c "echo '{\"clientId\":\"$INF_CLIENT\",\"clientSecret\":\"$INF_SECRET\"}' > /etc/infisical/identity"
dry chmod 600 /etc/infisical/identity
log " + Infisical identity written to /etc/infisical/identity"
fi
else
err_msg=$(echo "$ENROLL_RESP" | jq -r '.detail // .title // "unknown error"')
die "enrollment failed: $err_msg"
fi
# ── install context poller (replaces git sync timer) ─────────────────
log "installing context poller..."
POLLER_SCRIPT="$CLONE_DIR/tools/context-poller.sh"
cat > "$POLLER_SCRIPT" << 'POLLER_EOF'
#!/usr/bin/env bash
# context-poller.sh — lightweight deltas instead of git pull.
# Polls GET /api/v1/clients/{slug}/context?since=<timestamp> every 5 min.
set -euo pipefail
OIKOS_URL="${OIKOS_API_URL:-https://oikos.hubris.network/api/v1}"
HNAME=$(scutil --get LocalHostName 2>/dev/null || hostname -s)
STATE_FILE="${HOMELAB_CONTEXT_DIR:-/opt/homelab}/.context_since"
SINCE=""
[ -f "$STATE_FILE" ] && SINCE=$(cat "$STATE_FILE")
RESP=$(curl -s --connect-timeout 10 "$OIKOS_URL/clients/ws:${HNAME}/context?since=${SINCE}" 2>/dev/null || true)
if [ -n "$RESP" ]; then
NEW_SINCE=$(echo "$RESP" | jq -r '.since // empty')
if [ -n "$NEW_SINCE" ]; then
echo "$NEW_SINCE" > "$STATE_FILE"
CHANGED=$(echo "$RESP" | jq -r '.agent_files_changed // [] | .[]' 2>/dev/null || true)
if [ -n "$CHANGED" ]; then
echo "[oikos] context updated at $(date -u +%Y-%m-%dT%H:%M:%SZ)"
fi
fi
fi
POLLER_EOF
chmod +x "$POLLER_SCRIPT"
case "$OS" in
Darwin)
PLIST="/Library/LaunchDaemons/network.hubris.oikos-context-poller.plist"
cat > "$PLIST" << EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key><string>network.hubris.oikos-context-poller</string>
<key>ProgramArguments</key>
<array><string>$POLLER_SCRIPT</string></array>
<key>StartInterval</key><integer>300</integer>
<key>RunAtLoad</key><true/>
<key>StandardOutPath</key><string>/var/log/oikos-context-poller.log</string>
<key>StandardErrorPath</key><string>/var/log/oikos-context-poller.log</string>
</dict>
</plist>
EOF
dry launchctl bootstrap system "$PLIST" 2>/dev/null || true
dry launchctl kickstart system/network.hubris.oikos-context-poller 2>/dev/null || true
log " + macOS context poller installed (every 5 min)"
;;
Linux)
SERVICE="/etc/systemd/system/oikos-context-poller.service"
TIMER="/etc/systemd/system/oikos-context-poller.timer"
cat > "$SERVICE" << EOF
[Unit]
Description=Oikos context poller
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=$POLLER_SCRIPT
User=root
EOF
cat > "$TIMER" << EOF
[Unit]
Description=Oikos context poller (every 5 min)
[Timer]
OnBootSec=2min
OnUnitActiveSec=5min
[Install]
WantedBy=timers.target
EOF
dry systemctl daemon-reload
dry systemctl enable --now oikos-context-poller.timer
log " + Linux context poller installed (every 5 min)"
;;
esac
# ── symlink AGENTS.md for discovery ──────────────────────────────────
case "$OS" in
Darwin) dry ln -sfn "$CLONE_DIR/AGENTS.md" /etc/AGENTS.md 2>/dev/null || true ;;
Linux) dry ln -sfn "$CLONE_DIR/AGENTS.md" /root/AGENTS.md 2>/dev/null || true ;;
esac
# ── run auto-setup scripts ───────────────────────────────────────────
for setup in "$CLONE_DIR"/tools/setup-*.sh; do
[ -f "$setup" ] || continue
log "running setup: $(basename "$setup")"
dry bash "$setup"
done
# ── --with-mcp: wire Claude's .mcp.json ──────────────────────────────
if [ "$WITH_MCP" -eq 1 ]; then
log "wiring MCP config..."
MCP_JSON="$HOME/.claude/.mcp.json"
if [ -f "$MCP_JSON" ]; then
dry bash -c "jq --arg url '$MCP_URL' '.mcpServers.homelab = {\"type\":\"sse\",\"url\":\$url}' '$MCP_JSON' > '$MCP_JSON.tmp' && mv '$MCP_JSON.tmp' '$MCP_JSON'"
else
dry mkdir -p "$(dirname "$MCP_JSON")"
dry bash -c "echo '{\"mcpServers\":{\"homelab\":{\"type\":\"sse\",\"url\":\"$MCP_URL\"}}}' > '$MCP_JSON'"
fi
log " + MCP wired to $MCP_URL"
fi
# ── netbird SSH JWT cache ────────────────────────────────────────────
if command -v netbird >/dev/null 2>&1; then
dry netbird up --management-url https://netbird.hubris.network --ssh-jwt-cache-ttl 86400 2>/dev/null || true
fi
# ── done ─────────────────────────────────────────────────────────────
cat <<EOF
Oikos thin client bootstrap complete.
Hostname: $HNAME
Mesh IP: $MESH_IP
Age pubkey: $(cat /etc/age/key.txt 2>/dev/null | head -1 || echo "pending")
Context: $CLONE_DIR
Poller: every 5 min via launchd/systemd
Next steps:
1. On an enrolled client, finalize the public key:
POST /api/v1/clients/ws:$HNAME/activate
2. Add age pubkey to .sops.yaml recipients
3. Verify with MCP: whoami($HNAME)
EOF