oikos/infrastructure/auto-deploy.md

Auto-deploy — gitea-webhook pipelines

Several configs and apps in the lab live in dtoro/* repos on gitea (104) and auto-redeploy on push. All pipelines follow one of two shapes.

Two shapes

Shape A — checkout IS the working tree (config repos)

/etc/<thing> or /var/lib/<thing>/... is itself a git clone. Push triggers git pull + a reload command. Used for pure-config repos where re-cloning is cheap.

Shape B — receiver outside the app repo (compose stacks)

The app repo at /opt/<thing> is the working tree, but the deploy tooling (webhook.py, deploy.sh, systemd unit) lives in a sibling /opt/<thing>-deploy/ so the app repo stays portable. Push triggers git pull + docker compose up -d --build. Returns 202 immediately and runs the build in a daemon thread because docker builds exceed gitea's request timeout.

Common

• All receivers validate X-Gitea-Signature HMAC-SHA256 against a per-pipeline secret in /etc/<thing>-deploy/secret.

• All filter to refs/heads/main (or master for older repos). Gitea's "test delivery" button sends ref=main (without refs/heads/) — those will log "ignoring ref main" and 204. Real pushes work. Don't "fix" the ref filter to accept both — it'd also accept PR merges from side branches that got fast-forwarded.

• Gitea's app.ini [webhook] ALLOWED_HOST_LIST must include every receiver IP. Currently:

  • 127.0.0.1 (gitea customizations on LXC 104)
  • 192.168.8.175 (caddy (121))
  • 192.168.8.205 (apps (105) — Artifacto)
  • 192.168.8.230 (claudio-bot (123))
  • 192.168.8.136 (mule-images (120))
  • 192.168.8.77 (hubris host — backup-library)

  Don't strip these when editing app.ini.

• Git creds for root-run deploy services live in /etc/<thing>-deploy/git-credentials (mode 600) and are wired via credential.helper = store --file=/etc/<thing>-deploy/git-credentials in the repo's .git/config. Necessary because the unit typically runs with ProtectHome=true, which blocks /root.

Pipelines

Repo	Target	Shape	Receiver	Webhook id	Reload action
dtoro/caddy-conf	caddy (121) /etc/caddy/	A	http://192.168.8.175:9797/deploy	2	caddy validate + systemctl reload caddy
dtoro/gitea-customizations	gitea (104) /var/lib/gitea/custom/	A	http://127.0.0.1:9797/deploy (loopback)	(orig)	systemctl restart gitea if templates changed
dtoro/mule-image	mule-images (120) /opt/mule-image/	B	http://192.168.8.136:9797/deploy	6	docker compose up -d --build
dtoro/Artifacto	apps (105) /opt/artifacto/	B	http://192.168.8.205:9798/deploy	7	docker compose up -d --build
dtoro/claudio-bot	claudio-bot (123) /opt/claudio-bot/	A	http://192.168.8.230:9797/deploy	(orig)	pip install + systemctl restart claudio-bot
dtoro/backup-library	hubris host /opt/backup-library/	A	http://192.168.8.77:9798/deploy	(orig)	runs deploy.sh (preserves admin-edited /etc/restic/include-*.list)

Not yet wired: dtoro/claudio-monitor (push, then /opt/claudio-monitor/scripts/deploy.sh manually). dtoro/authentik-conf is reserved but the LXC stack is not git-tracked yet. The dnsmasq config on authentik (124) is also not tracked — if it gets a dtoro/dnsmasq-conf, mirror the caddy-conf pattern.

When you change a tracked config

Always commit + push. Local-only edits drift. Common ones:

• /etc/caddy/Caddyfile ↔ dtoro/caddy-conf (auto-deploys)
• /var/lib/gitea/custom/ ↔ dtoro/gitea-customizations (auto-deploys)
• /opt/artifacto/ ↔ dtoro/Artifacto (auto-deploys)
• /opt/mule-image/ ↔ dtoro/mule-image (auto-deploys)
• /opt/claudio-bot/ ↔ dtoro/claudio-bot (auto-deploys)
• /opt/backup-library/ ↔ dtoro/backup-library (auto-deploys)

Per-pipeline notes / gotchas

caddy-conf

• Repo includes scripts/webhook/install.sh. Editing the systemd unit inside the repo does not auto-reinstall — re-run install.sh manually after unit edits.
• The unit has ReadWritePaths=/etc/caddy — load-bearing (ProtectSystem=full would otherwise block git pull).

gitea-customizations

• Receiver is on loopback (127.0.0.1:9797), not the LXC IP.
• Online3DViewer binary assets are NOT tracked; deploy.sh fetches them on first run.

mule-image / Artifacto

• Async deploy (returns 202) — gitea would otherwise time out the request. Logs: pct exec <id> -- journalctl -u <thing>-deploy-webhook -f.
• Cloning from inside the LXC must use the internal gitea IP (http://192.168.8.121:3000/...). https://git.hubris.network hits a connection reset from inside apps (105) (Caddy routing / TLS hairpin not configured for this LXC). Configured origin on the in-LXC checkout is the internal URL.
• Manual deploy: pct exec <id> -- /opt/<thing>-deploy/deploy.sh.
• Health: pct exec <id> -- curl -s http://127.0.0.1:<port>/health → ok.
• Setup tokens used to register the webhook (e.g., artifacto-deploy-setup, artifacto-deploy-setup-2, artifacto-cleanup on user dtoro) need manual revocation in the Gitea UI → Settings → Applications → Manage Access Tokens. Gitea's /users/{u}/tokens endpoints require basic auth (not bearer), so cleanup couldn't be automated.

backup-library

• Currently the only deploy that targets the host directly (192.168.8.77:9798).
• deploy.sh is careful to preserve admin edits to /etc/restic/include-*.list — canonical source is config/ in the repo, but the install path is treated as authoritative once deploy.sh has run.

Related

• Gitea (104) — webhook source for all of these
• Caddy (121), apps (105), mule-images (120), claudio-bot (123), hubris host — webhook targets
• Backups (disabled)

Changelog

2026-04-28 — wiki entry created

Initial documentation. Six active pipelines.

2026-04-22 — Artifacto pipeline added

Webhook id 7 on dtoro/Artifacto (port 9798 on apps). app.ini ALLOWED_HOST_LIST extended.

2026-04-21 — mule-image + claudio-bot pipelines added

Webhook id 6; receiver on apps' sibling /opt/mule-deploy/. Same shape used for claudio-bot.

2026-04-20 — caddy-conf + gitea-customizations + backup-library pipelines shipped

Initial three. Set the conventions everything else follows.