Allow splitting config, downloads and media paths via .env
Some checks failed
Check running / run (push) Has been cancelled
Some checks failed
Check running / run (push) Has been cancelled
Adds optional CONFIG_DIR, DOWNLOADS_DIR and MEDIA_DIR env variables so
users can spread configs, downloads and media across different drives
without restructuring ROOT_DIR. Each falls back to its ROOT_DIR-based
default when left empty, so existing setups keep working unchanged.
The -arr services (sonarr, radarr, lidarr, mylar3) now use a single
${DOWNLOADS_DIR}:/data parent bind with a ${MEDIA_DIR}:/data/media
overlay. This preserves TRaSH's single-/data-mount semantics inside
the container (hardlinks still work on one filesystem) while letting
the host-side media tree live wherever the user wants.
README documents the new variables, the generated folder structure,
recommended storage layouts, and the same-filesystem requirement for
hardlinks between downloads and media.
This commit is contained in:
101
README.md
101
README.md
@@ -55,9 +55,13 @@ For other Linux distros you may have to use a different package manager or downl
|
||||
### Using the CLI
|
||||
To make things easier, a CLI has been developed. First, clone the repository in a directory of your
|
||||
choosing. You can run it by entering `python3 main.py` and the CLI will guide you through the
|
||||
process. This is the recommended method if you're setting this up for the first time on a new system.
|
||||
Please take a look at [important notes](#important-notes) before you continue.
|
||||
**NOTE: This script will create users for each container with IDs ranging from 13001 to 13014.
|
||||
process. It will ask which services to enable, your timezone, your root directory, and — if you
|
||||
want — independent absolute paths for your configs, downloads and media trees (press enter at each
|
||||
prompt to keep the `ROOT_DIR`-based default). See [Storage layouts](#storage-layouts) below for the
|
||||
trade-offs when splitting those.
|
||||
This is the recommended method if you're setting this up for the first time on a new system.
|
||||
Please take a look at [important notes](#important-notes) before you continue.
|
||||
**NOTE: This script will create users for each container with IDs ranging from 13001 to 13014.
|
||||
If you want to choose your own IDs (or some of them are occupied) you have to go through the manual install.**
|
||||
|
||||
### Manually
|
||||
@@ -65,9 +69,55 @@ If you're installing this for the first time simply follow these steps.
|
||||
If you're coming from an older version or reinstalling with different IDs, run `remove_old_users.sh` to clean up old users and then follow these steps.
|
||||
1. To get started, clone the repository in a directory of your choosing. `git clone https://github.com/Luctia/ezarr.git`
|
||||
2. Copy `.env.sample` to a real `.env` by running `$ cp .env.sample .env`.
|
||||
3. Set the environment variables to your liking. Pay special attention `ROOT_DIR` as this is where everything is going to be stored in.
|
||||
3. Set the environment variables to your liking. Pay special attention `ROOT_DIR` as this is where everything is going to be stored in.
|
||||
The path in this value needs to be **absolute**. If you leave it empty it's going to install in the directory the .env file is currently in.
|
||||
`UID` should be set to the ID of the user that you want to run docker with. You can find this by running `id -u` from that user's shell.
|
||||
If you want to spread configs, downloads and media across different locations (e.g. SSD for configs, bulk HDD for media), set `CONFIG_DIR`, `DOWNLOADS_DIR` and/or `MEDIA_DIR` to absolute paths. Any of them left empty fall back to `${ROOT_DIR}/config`, `${ROOT_DIR}/data` and `${ROOT_DIR}/data/media` respectively. **Note:** hardlinks/atomic moves between downloads and media only work when `DOWNLOADS_DIR` and `MEDIA_DIR` live on the same filesystem — split them only if you understand that you'll be copying instead of hardlinking.
|
||||
|
||||
The generated folder structure on the host looks like this (the three top-level trees can live under one `ROOT_DIR` or be spread across different drives via the `*_DIR` overrides):
|
||||
```
|
||||
${CONFIG_DIR}/ # default: ${ROOT_DIR}/config
|
||||
├── sonarr-config/
|
||||
├── radarr-config/
|
||||
├── lidarr-config/
|
||||
├── mylar-config/
|
||||
├── prowlarr-config/
|
||||
├── jackett-config/
|
||||
├── qbittorrent-config/
|
||||
├── sabnzbd-config/
|
||||
├── bazarr-config/
|
||||
├── overseerr-config/
|
||||
├── jellyseerr-config/
|
||||
├── plex-config/
|
||||
├── jellyfin-config/
|
||||
├── tautulli-config/
|
||||
└── audiobookshelf-config/
|
||||
|
||||
${DOWNLOADS_DIR}/ # default: ${ROOT_DIR}/data
|
||||
├── torrents/ # qBittorrent save path
|
||||
│ ├── tv/
|
||||
│ ├── movies/
|
||||
│ ├── music/
|
||||
│ ├── comics/
|
||||
│ └── ...
|
||||
└── usenet/ # SABnzbd complete/incomplete
|
||||
├── tv/
|
||||
├── movies/
|
||||
├── music/
|
||||
├── comics/
|
||||
└── ...
|
||||
|
||||
${MEDIA_DIR}/ # default: ${ROOT_DIR}/data/media
|
||||
├── tv/ # Sonarr
|
||||
├── movies/ # Radarr
|
||||
├── music/ # Lidarr
|
||||
├── comics/ # Mylar3
|
||||
├── books/
|
||||
├── audiobooks/ # Audiobookshelf
|
||||
├── podcasts/ # Audiobookshelf
|
||||
└── audiobookshelf-metadata/
|
||||
```
|
||||
Inside the -arr containers these are always exposed under a single `/data` tree — `/data/torrents`, `/data/usenet`, `/data/media/{tv,movies,...}` — regardless of how the host paths are split. This preserves the TRaSH single-`/data`-mount pattern and keeps hardlinks working as long as `DOWNLOADS_DIR` and `MEDIA_DIR` share a filesystem. The compose file achieves this by binding `${DOWNLOADS_DIR}:/data` as the parent and overlaying `${MEDIA_DIR}:/data/media` on top.
|
||||
4. Run `setup.sh` as superuser. This will set up your users, a system of directories and ensure permissions are set correctly.
|
||||
5. Copy `docker-compose.yml.sample` to a real `docker-compose.yml` by running `$ cp docker-compose.yml.sample docker-compose.yml`.
|
||||
6. Take a look at the `docker-compose.yml` file. If there are services you would like to ignore
|
||||
@@ -86,6 +136,41 @@ If you're coming from an older version or reinstalling with different IDs, run `
|
||||
That's it! Your containers are now up and you can continue to set up the settings in them. Please
|
||||
take a look at [important notes](#important-notes) before you continue.
|
||||
|
||||
## Storage layouts
|
||||
The -arr services use **hardlinks** to move downloaded files from `torrents/` or `usenet/` into the
|
||||
media library without duplicating bytes. Hardlinks only work within a single filesystem — so
|
||||
`DOWNLOADS_DIR` and `MEDIA_DIR` must resolve to paths on the same filesystem for this to work. If
|
||||
they don't, Sonarr/Radarr/Lidarr/Mylar will silently fall back to copying, which uses twice the
|
||||
disk space and a lot more I/O on every import. `CONFIG_DIR` has no such constraint and can live
|
||||
anywhere.
|
||||
|
||||
Some layouts that work well:
|
||||
|
||||
- **Single-drive / single-filesystem (simplest).** Set `ROOT_DIR` and leave `CONFIG_DIR`,
|
||||
`DOWNLOADS_DIR`, `MEDIA_DIR` empty. Everything lives under one tree, hardlinks just work.
|
||||
- **SSD for configs, HDD for everything else.** Set `CONFIG_DIR=/ssd/ezarr/config` and leave the
|
||||
other two empty (they default under `ROOT_DIR`, which you point at the HDD). Config databases
|
||||
benefit from the faster disk; downloads + media stay together so hardlinks work.
|
||||
- **Pooled storage with [mergerfs](https://github.com/trapexit/mergerfs).** Combine multiple
|
||||
HDDs into one mount point (e.g. `/pool`) and point `ROOT_DIR` or `DOWNLOADS_DIR`+`MEDIA_DIR` at
|
||||
it. With `category.create=epmfs` and `moveonenospc=true`, mergerfs keeps a torrent and its
|
||||
hardlinked media file on the same underlying disk, so `link(2)` succeeds inside the pool.
|
||||
- **CoW filesystems (btrfs / xfs with reflink / bcachefs / ZFS).** Same-filesystem rule still
|
||||
applies, but you also get reflinks (instant, space-sharing copies) as a fallback when hardlinks
|
||||
aren't appropriate.
|
||||
|
||||
Layouts that will **break hardlinks** (they will still run, just less efficiently):
|
||||
- `DOWNLOADS_DIR` on one drive/mount and `MEDIA_DIR` on a different drive/mount.
|
||||
- Downloads on local disk, media on a remote NFS/SMB share (or vice versa).
|
||||
- Either path going through a union filesystem that doesn't co-locate related files.
|
||||
|
||||
If you genuinely need split storage and can't pool it, set "Use Hardlinks" to `false` in each -arr
|
||||
and accept the copy overhead.
|
||||
|
||||
Regardless of how you split the host paths, the compose file always exposes a single `/data` tree
|
||||
inside the -arr containers — `/data/torrents`, `/data/usenet`, `/data/media/{tv,movies,...}` — so
|
||||
the root-folder paths you configure in Sonarr/Radarr/etc. never need to change.
|
||||
|
||||
## Important notes
|
||||
- You probably shouldn't run the python script as root. Ideally you should create a brand new user that's just for these services, but any regular user will do.
|
||||
It will need your password for `sudo` to set up the permissions and folder structures, but you shouldn't run it *as* root.
|
||||
@@ -114,11 +199,13 @@ Note that if you are a group member (and not the owner), having matching group I
|
||||
the users and groups are set up on both sides correctly is to run `setup.sh` on both your NFS server and your client.
|
||||
On your server:
|
||||
- Copy `.env` and `setup.sh` to your NFS server.
|
||||
- You may have to adjust `.env` so that `ROOT_DIR` reflects where it will be stored on your server, which is most likely different from the mapped location on the client.
|
||||
- You may have to adjust `.env` so that `ROOT_DIR` (and `CONFIG_DIR` / `DOWNLOADS_DIR` / `MEDIA_DIR`, if you've set any of them) reflect where each tree will be stored on your server, which is most likely different from the mapped location on the client.
|
||||
- Make sure that the `.env` file is not a .sample. Run `setup.sh`.
|
||||
- Now follow all the same steps but on your client machine. Always double-check that `.env` is set correctly, especially `ROOT_DIR`.
|
||||
You don't have to do this on your server first but it's recommended. If you are running this script on the client **make sure that you temporarily enable -no-root-squash on your NFS server**,
|
||||
- Now follow all the same steps but on your client machine. Always double-check that `.env` is set correctly, especially the path variables above.
|
||||
You don't have to do this on your server first but it's recommended. If you are running this script on the client **make sure that you temporarily enable -no-root-squash on your NFS server**,
|
||||
as the script needs superuser privileges to run and by default on NFS the root user is mapped to nowhere to prevent abuse.
|
||||
|
||||
**Heads-up on hardlinks over NFS:** if your downloads live on the local disk but media lives on an NFS mount (or vice versa), that's two different filesystems and hardlinks will break — see [Storage layouts](#storage-layouts) for the options. Keeping `DOWNLOADS_DIR` and `MEDIA_DIR` both on the NFS mount is fine, as long as they share the same share/filesystem.
|
||||
|
||||
### SABnzbd External internet access denied message
|
||||
When you're trying to access SABnzbd the first time you'll come across the message `External
|
||||
|
||||
Reference in New Issue
Block a user