From c27cd99c6d78a173902f6bfecce743ff7cdd6c3d Mon Sep 17 00:00:00 2001 From: Claudio Date: Wed, 13 May 2026 21:01:32 +0200 Subject: [PATCH] Document Nextcloud throughput tuning (FPM + event MPM + opcache JIT + MariaDB) --- containers/114-nextcloud.md | 51 ++++++++++++++++++++++++++++++++++--- 1 file changed, 47 insertions(+), 4 deletions(-) diff --git a/containers/114-nextcloud.md b/containers/114-nextcloud.md index eade9ed..da7e1c2 100644 --- a/containers/114-nextcloud.md +++ b/containers/114-nextcloud.md @@ -28,8 +28,18 @@ Also needs `allow_local_remote_servers=true` in `config.php`. ## Storage Files at `/mnt/library/homecloud`. Owned by Nextcloud's own permission model — **deliberately not on the `media` group**, NC manages it itself and would complain. See [media permissions](../infrastructure/media-permissions.md). -## PHP upload tuning -Apache uses `mod_php` (not FPM). Limits raised from distro defaults in both `/etc/php/8.4/apache2/php.ini` and `/etc/php/8.4/cli/php.ini` (the latter so `occ`/cron honor the same): +## Web stack (Apache event MPM + php-fpm) +Apache 2.4 with **`mpm_event`** as the MPM and PHP served via **php-fpm 8.4** over the Unix socket `/run/php/php8.4-fpm.sock` (mod_php disabled). Routing to FPM is via the distro's `/etc/apache2/conf-available/php8.4-fpm.conf` (gated by ``, so just `a2dismod php8.4` flips Apache onto FPM). + +Why this matters: with mod_php every Apache worker is forced onto `mpm_prefork` (one heavy process per connection, full PHP interpreter loaded in each). On FPM + event, Apache workers are ~5–8 MB and async; PHP work is done by a separate FPM pool that keeps opcache hot. Concurrent upload throughput jumps significantly because Apache no longer blocks per-connection on a heavy PHP process. + +### PHP-FPM pool +- Socket: `/run/php/php8.4-fpm.sock` (owner/group `www-data`) +- `pm = dynamic`, `pm.max_children = 30`, `pm.start_servers = 6`, `pm.min/max_spare_servers = 4/12`, `pm.max_requests = 500` +- Steady-state ~6 idle workers at ~80 MB each — bump `max_children` only if peak concurrency exceeds 30. + +### PHP limits (FPM + CLI both) +Set in `/etc/php/8.4/fpm/php.ini` **and** `/etc/php/8.4/cli/php.ini` (the latter so `occ`/cron see the same ceilings): | key | value | |---|---| @@ -38,9 +48,29 @@ Apache uses `mod_php` (not FPM). Limits raised from distro defaults in both `/et | `memory_limit` | `512M` | | `max_execution_time` | `3600` | | `max_input_time` | `3600` | -| `output_buffering` | `0` | +| `output_buffering` | `Off` | -Reload Apache after edits: `systemctl reload apache2`. Caddy in front (`cloud.hubris.network`) imposes no body-size limit, so these are the effective ceiling. +After edits: `systemctl reload php8.4-fpm`. Caddy in front (`cloud.hubris.network`) imposes no body-size limit, so these are the effective ceiling. + +### Opcache + JIT +`/etc/php/8.4/mods-available/opcache.ini` — full config maintained, not commented stubs: +- `opcache.enable=1`, `opcache.memory_consumption=256`, `opcache.interned_strings_buffer=32`, `opcache.max_accelerated_files=20000` +- `opcache.jit=tracing`, `opcache.jit_buffer_size=64M` + +### Apache mod_reqtimeout +`/etc/apache2/mods-available/reqtimeout.conf` — relaxed body trickle so slow-WAN uploads don't get killed: +- `RequestReadTimeout header=20-40,MinRate=500` +- `RequestReadTimeout body=20,MinRate=100` (was `body=10,MinRate=500`) + +## MariaDB tuning +Overrides in `/etc/mysql/mariadb.conf.d/99-nextcloud-tuning.cnf`: +- `innodb_buffer_pool_size = 1G` (default was 128M) +- `innodb_log_file_size = 256M` +- `innodb_flush_log_at_trx_commit = 2` (group-commit; trades a few ms of durability for write throughput — acceptable for NC) +- `innodb_flush_method = O_DIRECT`, `innodb_io_capacity = 2000` / `..._max = 4000` +- `max_allowed_packet = 256M`, `tmp_table_size = 64M`, `max_heap_table_size = 64M` + +Apply with `systemctl restart mariadb` (not reload — `innodb_log_file_size` needs a clean restart). ## Related - [mulita (120)](120-mule-images.md) — reads NC user trees + writes back via WebDAV @@ -52,6 +82,19 @@ Reload Apache after edits: `systemctl reload apache2`. Caddy in front (`cloud.hu ## Changelog +### 2026-05-13 — throughput tuning: FPM + event MPM + opcache JIT + MariaDB +Investigated upload/download speed. Bottlenecks were inside this LXC — LAN (33.9 Gbit/s caddy→NC iperf3) and `/mnt/library` NVMe (714 MB/s sustained) were not the limit. Changes: +- **Apache mod_php → php-fpm 8.4** (`a2dismod php8.4`). Flips PHP from in-process to FPM-over-Unix-socket. Apache workers now ~5–8 MB instead of ~80 MB each. +- **MPM prefork → event** (`a2dismod mpm_prefork && a2enmod mpm_event`). Async I/O, far better concurrency for parallel uploads. +- **opcache JIT enabled** (`tracing`, 64M buffer), `memory_consumption=256`, `interned_strings_buffer=32`, `max_accelerated_files=20000`. +- **MariaDB**: `innodb_buffer_pool_size 128M → 1G`, `innodb_log_file_size 96M → 256M`, `flush_log_at_trx_commit 1 → 2`, `O_DIRECT`, higher I/O capacity. New file `99-nextcloud-tuning.cnf`. +- **Apache mod_reqtimeout**: body trickle relaxed from `body=10,MinRate=500` → `body=20,MinRate=100` so slow-WAN uploads aren't killed. +- **PHP-FPM ini** synced to match what the apache2/cli inis already had (FPM was stuck at 2M/8M because FPM wasn't serving traffic before, but now it does). + +Backups: `/etc/php/8.4/fpm/php.ini.bak.20260513`, `/etc/apache2/mods-available/reqtimeout.conf.bak.20260513`, `/etc/php/8.4/mods-available/opcache.ini.bak.20260513`. MariaDB tuning is a new file (rollback = `rm`). + +Verified: `apache2ctl configtest` Syntax OK, `occ status` clean, `occ setupchecks` shows ✓ on memory_limit / output_buffering / DB indices, `status.php` returns 200 in <40 ms via Caddy with HTTP/2. + ### 2026-05-10 — PHP upload limits raised Apache + CLI php.ini bumped from distro defaults (`upload_max_filesize=2M`, `post_max_size=8M`, `memory_limit=128M`, `max_execution_time=30`) to NC-recommended (`16G/16G/512M/3600`). Backups at `/etc/php/8.4/{apache2,cli}/php.ini.bak-20260510-*`. Was causing slow/failing uploads via web UI.