docs(rclone): LXC 132 rclone -> Proton Drive backup; deprecate restic-on-USB
New off-host backup job replacing the disabled restic-on-USB backup: LXC 132 `rclone` mirrors selected /mnt/library folders to Proton Drive (plain rclone sync, Proton's built-in E2E, no crypt overlay) on a monthly timer, with rclone's Web GUI for LAN-only browsing/ad-hoc runs and live job status. - containers/132-rclone.md: full design, Proton auth gotcha (TOTP secret vs live code), pct exec PATH gotcha, rc-API job-visibility runner rewrite, selected folder set (cloud/documents/repos), deferred tracked-repo note. - infrastructure/backups.md: restic-on-USB marked DEPRECATED/superseded, leads with the new job now. - containers/index.md, README.md, infrastructure/media-permissions.md: register the new container.
This commit is contained in:
149
containers/132-rclone.md
Normal file
149
containers/132-rclone.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# 132 — `rclone`
|
||||
|
||||
Off-host backup appliance. Mirrors selected `/mnt/library` folders to **Proton Drive**
|
||||
with a plain `rclone sync` (monthly), and serves rclone's Web GUI on the LAN for browsing
|
||||
and ad-hoc runs. **Replaces** the disabled restic-on-USB job — see [backups](../infrastructure/backups.md).
|
||||
|
||||
Provisioned 2026-07-01. (LXC 131 was already taken by an undocumented `teddycloud` container,
|
||||
so this landed on **132**.)
|
||||
|
||||
## At a glance
|
||||
|
||||
- **Hostname:** `rclone`
|
||||
- **IP:** `192.168.8.214` (static, set in PVE `net0` config — same pattern as grimmory/authentik)
|
||||
- **Privilege:** privileged (root in-container = host root → reads every `/mnt/library` subtree,
|
||||
incl. `homecloud/` and `documents/`, regardless of owner)
|
||||
- **Resources:** 1 core / 1 GiB RAM / 8 GiB rootfs (Debian 13)
|
||||
- **Mounts:** `/mnt/library` **read-only** (`mp0: /mnt/library,mp=/mnt/library,ro=1`) — a backup
|
||||
job must never be able to write into the library
|
||||
- **Public hostname:** none — the UI is **LAN-only, no auth** (by design)
|
||||
|
||||
## Service / port map
|
||||
|
||||
| Service | Listen | Notes |
|
||||
|---------|--------|-------|
|
||||
| rclone Web GUI (`rcd`) | `192.168.8.214:5572` | `rclone-rcd.service`, **`--rc-no-auth`**, LAN-only. Browse `/mnt/library` + `proton:`, run ad-hoc syncs, live job status |
|
||||
| monthly mirror | — | `rclone-backup.service` + `.timer` (`OnCalendar=*-*-01 03:00`) |
|
||||
|
||||
## Backup design
|
||||
|
||||
- **Mode:** plain mirror — `rclone sync` (Proton mirrors local; deletions propagate; **no versioning**).
|
||||
- **Encryption:** Proton Drive's built-in E2E only (no rclone `crypt` overlay → files stay
|
||||
browsable in Proton's web UI).
|
||||
- **Selected set:** `/etc/rclone-backup/folders.list` — one absolute source path per line
|
||||
(`#`/blank ignored). This file *is* the picked set the monthly timer mirrors. Extensible to other
|
||||
disks once bind-mounted into this LXC.
|
||||
- **Path mapping:** source `S` → `proton:library-backup/<S without leading slash>`
|
||||
(e.g. `/mnt/library/notes` → `proton:library-backup/mnt/library/notes`).
|
||||
- **Runner:** `/usr/local/sbin/rclone-backup.sh [folder ...]` (Python, despite the `.sh` name — kept
|
||||
the path stable) — no arg = every enabled line. Submits each folder as an **async job through the
|
||||
rclone rc API** served by `rclone-rcd.service` (the same daemon backing the Web GUI on `:5572`),
|
||||
so scheduled/ad-hoc runs show up live in the GUI's **Jobs panel**, not just in logs. Gentle on
|
||||
Proton's rate limits (`Transfers=4, TPSLimit=8, FastList=true` via the rc `_config` payload). The
|
||||
rc API here requires **POST for every call** including `job/status` and `core/stats` — GET with
|
||||
query params 404s.
|
||||
- **Logs / "past runs":** per-run logs in `/var/log/rclone-backup/<safe>-<ts>.log`; one-line
|
||||
JSON summary per run appended to `/var/log/rclone-backup/runs.jsonl`.
|
||||
- **Failure notify:** `OnFailure=rclone-backup-notify@%n.service` → logs to journal today;
|
||||
**TODO** wire to Hermes `send_message` (Matrix) per [backups](../infrastructure/backups.md).
|
||||
|
||||
## rclone + Proton Drive
|
||||
|
||||
- **rclone** installed from the official binary (not apt) so the `protondrive` backend is present
|
||||
(`rclone v1.74.3`).
|
||||
- Remote **`proton:`** (type `protondrive`). Config at `/root/.config/rclone/rclone.conf`, mode 600.
|
||||
**This file is a secret** (holds the obscured Proton password + TOTP secret + session) — **never
|
||||
commit it.** Escrow the Proton account creds in the password manager.
|
||||
- **Config gotchas** (from rclone docs/forum):
|
||||
- Log into Proton via a **browser at least once** first, or key generation fails.
|
||||
- For unattended runs, store the **TOTP _secret_** (not a 6-digit code) so rclone self-generates
|
||||
codes; obscure with `rclone obscure`.
|
||||
- Passwords with **extended-ASCII** characters are known to break auth.
|
||||
- Proton's API is rate-limited → keep `--transfers`/`--tpslimit` conservative (baked into the runner).
|
||||
- **DR escrow (pending):** store the Proton creds as sops secret `secrets/protondrive.yaml`, granted
|
||||
to this LXC's age key, so the remote can be rebuilt after a re-provision.
|
||||
|
||||
## The UI (rclone Web GUI)
|
||||
|
||||
`rclone rcd --rc-web-gui --rc-no-auth --rc-addr 0.0.0.0:5572` (assets auto-downloaded on first
|
||||
start). Reach it at **http://192.168.8.214:5572** on the LAN.
|
||||
|
||||
> **Security note:** `--rc-no-auth` exposes *full* rclone control — including deleting remote data —
|
||||
> to anyone on the LAN (accepted per the design choice). The container has only a LAN NIC, so it is
|
||||
> not publicly reachable. Harden later by adding `--rc-user/--rc-pass` or fronting it with Authentik.
|
||||
|
||||
## Tracked config (deferred)
|
||||
|
||||
**Not yet tracked.** The runner, systemd units, and `folders.list` currently live as plain files
|
||||
directly on the LXC — fully functional, just not version-controlled or auto-deployed. A
|
||||
`dtoro/rclone` gitea repo package (runner, units, `install.sh`, webhook receiver) is pre-built and
|
||||
staged at `/root/rclone-repo` on the LXC for whenever this gets tracked (Shape A, like
|
||||
[caddy](121-caddy.md)). Gitea `ALLOWED_HOST_LIST` already includes `192.168.8.214` in anticipation.
|
||||
See [auto-deploy](../infrastructure/auto-deploy.md).
|
||||
|
||||
**Selected folders (live in `/etc/rclone-backup/folders.list`):** `/mnt/library/cloud` (287G),
|
||||
`/mnt/library/documents` (249M), `/mnt/library/repos` (83M). `/mnt/library/notes` was synced once as
|
||||
a connectivity test (not in the recurring set). Proton quota checked: 2 TiB plan, ~1.65 TiB free
|
||||
after this set.
|
||||
|
||||
## Enrollment gotcha: `pct exec` PATH
|
||||
|
||||
`pct exec` (lxc-attach) does **not** source `/etc/environment` or run a login shell, so
|
||||
`/usr/local/bin` (where bootstrap installs `sops`) isn't on `$PATH` by default — bootstrap's own
|
||||
`command -v sops` post-install check failed under `pct exec` even though the binary installed fine.
|
||||
Fixed by symlinking `/usr/local/bin/{sops,homelab}` into `/usr/bin` (always on the minimal PATH),
|
||||
rather than relying on `/etc/environment`. Same category as the documented [`pct exec` no-initgroups
|
||||
gotcha](../infrastructure/media-permissions.md#gotchas) — worth adding to
|
||||
[agent-enrollment.md troubleshooting](../operations/agent-enrollment.md#troubleshooting) if it recurs
|
||||
on future LXC bootstraps.
|
||||
|
||||
## Related
|
||||
|
||||
- [Backups](../infrastructure/backups.md) — this job supersedes the disabled restic-on-USB backup
|
||||
- [Hubris host](../hosts/hubris.md) — owns `/mnt/library`
|
||||
- [Media permissions](../infrastructure/media-permissions.md) — read-only consumer of `/mnt/library`
|
||||
- [Containers index](index.md)
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2026-07-02 — runner rewritten to submit jobs via the rc API (GUI job visibility)
|
||||
|
||||
The original runner (`rclone sync` invoked as a standalone CLI subprocess) was invisible to the Web
|
||||
GUI's Jobs panel — the GUI only tracks work submitted through its own `rcd` process. Rewrote
|
||||
`/usr/local/sbin/rclone-backup.sh` in Python, submitting each folder via `POST /sync/sync` with
|
||||
`_async: true` against `http://127.0.0.1:5572` (the running `rclone-rcd.service`), then polling
|
||||
`POST /job/status` + `POST /core/stats` (both **must be POST** — GET-with-querystring 404s on this
|
||||
rc API) until finished, logging periodic progress snapshots and the same `runs.jsonl` summary line
|
||||
as before. Verified live: submitted job visible in `POST /job/list`'s `runningIds` while running,
|
||||
completed cleanly (`success: true`) once done. Deployed via atomic rename (write-then-`mv`) rather
|
||||
than truncating in place, specifically so it wouldn't risk corrupting the still-running original
|
||||
`cloud`+`documents`+`repos` sync mid-flight (verified after the fact: that sync's bash process was
|
||||
unaffected, kept running to completion under the old in-memory script content). The already-running
|
||||
scheduled sync from before this change is a standalone process and won't retroactively appear in the
|
||||
GUI; every run after this point will.
|
||||
|
||||
### 2026-07-02 — Proton Drive auth fixed; real folder set enabled; first live sync
|
||||
|
||||
Initial `rclone config` failed 2FA (`422 ... auth/v4/2fa`) because a live 6-digit TOTP code was
|
||||
entered instead of the TOTP secret — reconfigured with the secret, auth now works
|
||||
(`rclone lsd proton:` lists the Drive). Verified end-to-end with a real sync of `/mnt/library/notes`
|
||||
(219 objects, 5.964 MiB, exit 0) — confirmed files land as plain, browsable objects on Proton (not
|
||||
an opaque archive), matching the plain-mirror + Proton-E2E design. Checked Proton quota (2 TiB
|
||||
plan, 1.945 TiB free) before enabling a large folder. `folders.list` set to the real selection:
|
||||
`cloud` (287G), `documents` (249M), `repos` (83M); a full sync of that set was kicked off via the
|
||||
actual `rclone-backup.service` unit (not an ad-hoc call) to validate the real monthly path early
|
||||
rather than waiting for the Aug 1 timer. Tracked-repo step (`dtoro/rclone` on gitea) deferred by
|
||||
choice — runner/units/`folders.list` remain plain files on the LXC for now; the repo package stays
|
||||
staged at `/root/rclone-repo` for later.
|
||||
|
||||
### 2026-07-01 — provisioned; enrolled
|
||||
|
||||
LXC 132 created (Debian 13, privileged, `192.168.8.214`, `/mnt/library` read-only). rclone v1.74.3
|
||||
installed from the official binary (`protondrive` backend present). Runner + monthly timer +
|
||||
`folders.list` deployed; rclone Web GUI (`rcd`, LAN-only no-auth) live on `:5572`. Enrolled into
|
||||
homelab-context (`--no-mesh`, LAN-only issuance): age key issued, inventory finalized, shared
|
||||
secrets granted, `homelab whoami` + `homelab secret hello` verified. Gitea `ALLOWED_HOST_LIST`
|
||||
updated to include `192.168.8.214`. Hit and fixed a `pct exec` PATH gotcha (see below). Proton Drive
|
||||
remote, `dtoro/rclone` tracked repo + webhook, and the `secrets/protondrive.yaml` escrow remain
|
||||
operator-run follow-ups (credentialed steps — Proton password/2FA, repo creation). Restic-on-USB
|
||||
backup deprecated in the same change.
|
||||
@@ -18,6 +18,7 @@ All containers live on [`hubris`](../hosts/hubris.md). Each row links to the per
|
||||
| 128 | [trmnl](128-trmnl.md) | 192.168.8.211 | unpriv | 1 | 768 MiB | 8 GiB | — | `trmnl.hubris.network` | running |
|
||||
| 129 | [house](129-house.md) | 192.168.8.212 | unpriv | 1 | 1344 MiB | 8 GiB | — | `house.hubris.network` | running |
|
||||
| 130 | [grimmory](130-grimmory.md) | 192.168.8.213 | priv | 1 | 2 GiB | 16 GiB | `/mnt/library` | `books.hubris.network` | running |
|
||||
| 132 | [rclone](132-rclone.md) | 192.168.8.214 | priv | 1 | 1 GiB | 8 GiB | `/mnt/library` (**ro**) | — (LAN-only UI `:5572`) | running |
|
||||
|
||||
## Recently destroyed (kept for archaeology)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user