# 114 — `nextcloud`

Personal cloud / file collaboration. Source-of-truth for the photo libraries surfaced by [mulita (120)](120-mule-images.md).

## At a glance
- **Hostname:** `nextcloud`
- **IP:** `192.168.8.224`
- **Privilege:** privileged
- **Resources:** 4 cores / 6 GiB RAM / 25 GiB rootfs
- **Mounts:** `/mnt/library` ↔ `/mnt/library` (Nextcloud user files under `/mnt/library/homecloud`)
- **Public hostname:** [`cloud.hubris.network`](../infrastructure/dns.md) → [caddy](121-caddy.md)

## Auth
Native OIDC via `user_oidc` app. **Username override pattern**: Authentik user `dtoro` maps to local Nextcloud user `admin` via the `nc_uid` custom-claim scope. Configured via `occ user_oidc:provider <name> --mapping-uid=nc_uid` and `--scope="openid profile email <app>-uid"`. See [Authentik](124-authentik.md#per-app-username-override-pattern-authentik) for the full pattern.

Redirect URI: `/index.php/apps/user_oidc/code` (NOT `/apps/...` — pretty URLs aren't on).

## DNS workaround (load-bearing)
Nextcloud's PHP HTTP client (`OC\Http\Client\DnsPinMiddleware`) calls `dns_get_record()` directly — it bypasses `/etc/hosts`. So the Authentik OIDC discovery URL needs **real DNS** answering with `192.168.8.175` for `auth.hubris.network`. Recipe applied:
- Local `dnsmasq` listening on `127.0.0.1:53`, `/etc/dnsmasq.d/hubris-internal.conf` with `address=/auth.hubris.network/192.168.8.175`, `server=192.168.8.1`, `server=1.1.1.1`, `interface=lo`, `bind-interfaces`, `no-hosts`, `no-resolv`.
- `pct set 114 --nameserver "127.0.0.1 192.168.8.1 1.1.1.1"` so the LXC starts with the right resolver order.
- `/etc/hosts` override kept as belt-and-suspenders.

Also needs `allow_local_remote_servers=true` in `config.php`.

> Once internal DNS at the router level handles `*.hubris.network`, this LXC-local dnsmasq can go away. See [DNS](../infrastructure/dns.md).

## Storage
Files at `/mnt/library/homecloud`. Owned by Nextcloud's own permission model — **deliberately not on the `media` group**, NC manages it itself and would complain. See [media permissions](../infrastructure/media-permissions.md).

## Web stack (Apache event MPM + php-fpm)
Apache 2.4 with **`mpm_event`** as the MPM and PHP served via **php-fpm 8.4** over the Unix socket `/run/php/php8.4-fpm.sock` (mod_php disabled). Routing to FPM is via the distro's `/etc/apache2/conf-available/php8.4-fpm.conf` (gated by `<IfModule !mod_php8.c>`, so just `a2dismod php8.4` flips Apache onto FPM).

Why this matters: with mod_php every Apache worker is forced onto `mpm_prefork` (one heavy process per connection, full PHP interpreter loaded in each). On FPM + event, Apache workers are ~5–8 MB and async; PHP work is done by a separate FPM pool that keeps opcache hot. Concurrent upload throughput jumps significantly because Apache no longer blocks per-connection on a heavy PHP process.

### PHP-FPM pool
- Socket: `/run/php/php8.4-fpm.sock` (owner/group `www-data`)
- `pm = dynamic`, `pm.max_children = 30`, `pm.start_servers = 6`, `pm.min/max_spare_servers = 4/12`, `pm.max_requests = 500`
- Steady-state ~6 idle workers at ~80 MB each — bump `max_children` only if peak concurrency exceeds 30.

### PHP limits (FPM + CLI both)
Set in `/etc/php/8.4/fpm/php.ini` **and** `/etc/php/8.4/cli/php.ini` (the latter so `occ`/cron see the same ceilings):

| key | value |
|---|---|
| `upload_max_filesize` | `16G` |
| `post_max_size` | `16G` |
| `memory_limit` | `512M` |
| `max_execution_time` | `3600` |
| `max_input_time` | `3600` |
| `output_buffering` | `Off` |

After edits: `systemctl reload php8.4-fpm`. Caddy in front (`cloud.hubris.network`) imposes no body-size limit, so these are the effective ceiling.

### Opcache + JIT
`/etc/php/8.4/mods-available/opcache.ini` — full config maintained, not commented stubs:
- `opcache.enable=1`, `opcache.memory_consumption=256`, `opcache.interned_strings_buffer=32`, `opcache.max_accelerated_files=20000`
- `opcache.jit=tracing`, `opcache.jit_buffer_size=64M`

### Apache mod_reqtimeout
`/etc/apache2/mods-available/reqtimeout.conf` — relaxed body trickle so slow-WAN uploads don't get killed:
- `RequestReadTimeout header=20-40,MinRate=500`
- `RequestReadTimeout body=20,MinRate=100` (was `body=10,MinRate=500`)

## MariaDB tuning
Overrides in `/etc/mysql/mariadb.conf.d/99-nextcloud-tuning.cnf`:
- `innodb_buffer_pool_size = 1G` (default was 128M)
- `innodb_log_file_size = 256M`
- `innodb_flush_log_at_trx_commit = 2` (group-commit; trades a few ms of durability for write throughput — acceptable for NC)
- `innodb_flush_method = O_DIRECT`, `innodb_io_capacity = 2000` / `..._max = 4000`
- `max_allowed_packet = 256M`, `tmp_table_size = 64M`, `max_heap_table_size = 64M`

Apply with `systemctl restart mariadb` (not reload — `innodb_log_file_size` needs a clean restart).

## Related
- [mulita (120)](120-mule-images.md) — reads NC user trees + writes back via WebDAV
- [Authentik (124)](124-authentik.md)
- [Caddy (121)](121-caddy.md)
- [DNS](../infrastructure/dns.md)
- [Mesh migration (DNS overrides explained)](../infrastructure/mesh.md)
- [Media permissions](../infrastructure/media-permissions.md)

## Changelog

### 2026-05-13 — throughput tuning: FPM + event MPM + opcache JIT + MariaDB
Investigated upload/download speed. Bottlenecks were inside this LXC — LAN (33.9 Gbit/s caddy→NC iperf3) and `/mnt/library` NVMe (714 MB/s sustained) were not the limit. Changes:
- **Apache mod_php → php-fpm 8.4** (`a2dismod php8.4`). Flips PHP from in-process to FPM-over-Unix-socket. Apache workers now ~5–8 MB instead of ~80 MB each.
- **MPM prefork → event** (`a2dismod mpm_prefork && a2enmod mpm_event`). Async I/O, far better concurrency for parallel uploads.
- **opcache JIT enabled** (`tracing`, 64M buffer), `memory_consumption=256`, `interned_strings_buffer=32`, `max_accelerated_files=20000`.
- **MariaDB**: `innodb_buffer_pool_size 128M → 1G`, `innodb_log_file_size 96M → 256M`, `flush_log_at_trx_commit 1 → 2`, `O_DIRECT`, higher I/O capacity. New file `99-nextcloud-tuning.cnf`.
- **Apache mod_reqtimeout**: body trickle relaxed from `body=10,MinRate=500` → `body=20,MinRate=100` so slow-WAN uploads aren't killed.
- **PHP-FPM ini** synced to match what the apache2/cli inis already had (FPM was stuck at 2M/8M because FPM wasn't serving traffic before, but now it does).

Backups: `/etc/php/8.4/fpm/php.ini.bak.20260513`, `/etc/apache2/mods-available/reqtimeout.conf.bak.20260513`, `/etc/php/8.4/mods-available/opcache.ini.bak.20260513`. MariaDB tuning is a new file (rollback = `rm`).

Verified: `apache2ctl configtest` Syntax OK, `occ status` clean, `occ setupchecks` shows ✓ on memory_limit / output_buffering / DB indices, `status.php` returns 200 in <40 ms via Caddy with HTTP/2.

### 2026-05-10 — PHP upload limits raised
Apache + CLI php.ini bumped from distro defaults (`upload_max_filesize=2M`, `post_max_size=8M`, `memory_limit=128M`, `max_execution_time=30`) to NC-recommended (`16G/16G/512M/3600`). Backups at `/etc/php/8.4/{apache2,cli}/php.ini.bak-20260510-*`. Was causing slow/failing uploads via web UI.

### 2026-04-28 — wiki entry created
Initial documentation.

### 2026-04-26 — Nextcloud-rooted libraries shipped (mulita)
[mulita (120)](120-mule-images.md) now reads photos from `/nextcloud-users/<nc_user>/files/...` and writes mutations back via WebDAV. Pre-migration DB dump kept at `/root/snapshots/mulita-pre-nc-migration-20260426-075132.dump`.

### 2026-04-21 — wired into Authentik (native OIDC + nc_uid override)
Local `dnsmasq` added on this LXC because Guzzle bypasses `/etc/hosts`. Resolver order set to `127.0.0.1 192.168.8.1 1.1.1.1`. Username override `dtoro → admin`.