dtoro/mule-image
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(nextcloud): per-user Nextcloud library integration

Browse Source 
Lets each mule-image user (matched via OIDC preferred_username,
overridable in Settings) browse their Nextcloud files/ tree from the
mule-image UI and register subfolders as per-user SourceRoots. Reads
stay direct on the bind-mounted /nextcloud-users path; mutations
(upload, delete, rename, move within NC) dispatch through Nextcloud
WebDAV so oc_filecache, trashbin, comments, and desktop-sync clients
stay coherent.

Backend:
- users.nextcloud_username + nextcloud_app_password_enc (Fernet at rest,
  key derived from SECRET_KEY) — alembic 0016
- services/nextcloud_dav.py: minimal WebDAV client (PUT, MKCOL, DELETE,
  MOVE) with HTTP Basic auth via the per-user app password
- routers/nextcloud.py: GET /browse, /whoami, GET/POST/DELETE
  /source-roots (path-scoped to current_user.nextcloud_username with
  realpath traversal guard)
- PATCH /api/v1/auth/me to update nextcloud_username and app password
- OIDC callback defaults nextcloud_username from preferred_username on
  first login; backfill on existing users; never overwrites a manual
  override
- routers/upload.py: stream upload to NamedTemporaryFile, then PUT to
  WebDAV (with MKCOL chain) when destination is NC-rooted; existing
  Photo row creation runs unchanged
- routers/discard.py empty-trash: WebDAV DELETE for NC files
- routers/photos.py rename + move: WebDAV MOVE for NC paths;
  cross-system move/copy returns a clean error
- routers/folders.py rename + create + permanent-delete: dispatch via
  WebDAV when targeting NC-rooted paths

Frontend:
- AuthUser carries nextcloud_username + has_nextcloud_app_password
- services/api.ts: nextcloud + account namespaces
- components/dialogs/NextcloudFolderPicker.tsx: lazy tree browser, name
  + submit -> POST /source-roots
- SettingsDialog: new "Nextcloud library" card with username override +
  validate, app-password input, list/remove of NC libraries, and the
  picker entry point

docker-compose.yml: NEXTCLOUD_USERS_HOST_PATH bind to /nextcloud-users
on backend + 3 workers; NEXTCLOUD_USERS_ROOT + NEXTCLOUD_BASE_URL env.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claudio
2026-04-26 01:06:37 +02:00
parent 80dd9d0a8b
commit bc0bb44c05
16 changed files with 1635 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
backend/app/main.py
View File

@@ -12,7 +12,7 @@ import os
from app.config import settings
from app.database import init_db
from app.routers import photos, folders, heaps, tags, discard, library, search, auth, admin, sharing, upload, download, features
from app.routers import photos, folders, heaps, tags, discard, library, search, auth, admin, sharing, upload, download, features, nextcloud
from app.services.scanner import start_initial_scan, bootstrap_default_source_root
from app.services.cleanup import cleanup_data_integrity
@@ -112,6 +112,7 @@ app.include_router(search.router, prefix="/api/v1/photos/search", tags=["search"
app.include_router(upload.router, prefix="/api/v1/upload", tags=["upload"])
app.include_router(download.router, prefix="/api/v1/download", tags=["download"])
app.include_router(features.router, prefix="/api/v1/features", tags=["features"])
app.include_router(nextcloud.router, prefix="/api/v1/nextcloud", tags=["nextcloud"])
@app.get("/")
async def root():

12
backend/app/models/user.py
View File

@@ -35,3 +35,15 @@ class User(Base):
# computes the final avatar URL for the frontend.
avatar_url = Column(String, nullable=True)
display_name = Column(String, nullable=True)
# Nextcloud integration. `nextcloud_username` defaults to the
# `preferred_username` OIDC claim on first login but can be overridden
# in Settings (the local mule-image username doesn't always match the
# Nextcloud user — e.g. authentik `dtoro` ↔ Nextcloud `admin`).
# `nextcloud_app_password_enc` is the user's Nextcloud app password
# (created from Nextcloud → Settings → Security), Fernet-encrypted at
# rest with a key derived from settings.secret_key. Used as HTTP Basic
# auth on outgoing WebDAV calls when the user mutates a file under
# their Nextcloud-rooted SourceRoot.
nextcloud_username = Column(String, nullable=True, index=True)
nextcloud_app_password_enc = Column(String, nullable=True)

67
backend/app/routers/auth.py
View File

@@ -54,6 +54,18 @@ class UserResponse(BaseModel):
created_at: Optional[str]
avatar_url: Optional[str] = None
display_name: Optional[str] = None
# Nextcloud integration — username override (defaults to OIDC
# preferred_username) and a flag for whether the user has stored
# an app password. Cleartext passwords are never serialized.
nextcloud_username: Optional[str] = None
has_nextcloud_app_password: bool = False
class UpdateMeRequest(BaseModel):
"""PATCH /me payload. Every field is optional — only what's set
gets touched. Setting `nextcloud_app_password` to "" clears it."""
nextcloud_username: Optional[str] = None
nextcloud_app_password: Optional[str] = None
class SetupRequest(BaseModel):
username: str
@@ -93,6 +105,8 @@ def serialize_user(user: User) -> UserResponse:
created_at=user.created_at.isoformat() if user.created_at else None,
avatar_url=avatar,
display_name=user.display_name,
nextcloud_username=user.nextcloud_username,
has_nextcloud_app_password=bool(user.nextcloud_app_password_enc),
)
@@ -186,6 +200,46 @@ async def get_me(current_user: User = Depends(get_current_user)):
return serialize_user(current_user)
_NC_USERNAME_RE = re.compile(r"^[a-zA-Z0-9._@-]{1,64}$")
@router.patch("/me", response_model=UserResponse)
async def update_me(
body: UpdateMeRequest,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Update the authenticated user's Nextcloud integration settings.
`nextcloud_username` overrides the OIDC `preferred_username` default
so e.g. the local mule-image user `dtoro` can map to Nextcloud user
`admin`. `nextcloud_app_password` is encrypted at rest via the
Fernet helper in `services/secrets.py`; passing an empty string
clears it.
"""
from app.services.secrets import encrypt
changed = False
if body.nextcloud_username is not None:
candidate = body.nextcloud_username.strip()
if candidate and not _NC_USERNAME_RE.match(candidate):
raise HTTPException(status_code=400, detail="Invalid Nextcloud username")
current_user.nextcloud_username = candidate or None
changed = True
if body.nextcloud_app_password is not None:
if body.nextcloud_app_password == "":
current_user.nextcloud_app_password_enc = None
else:
current_user.nextcloud_app_password_enc = encrypt(body.nextcloud_app_password)
changed = True
if changed:
await db.commit()
await db.refresh(current_user)
return serialize_user(current_user)
@router.post("/change-password")
async def change_password(
body: ChangePasswordRequest,
@@ -426,6 +480,11 @@ async def oidc_callback(request: Request, db: AsyncSession = Depends(get_db)):
oidc_sub=sub,
avatar_url=picture,
display_name=display_name,
# Default the Nextcloud username from preferred_username so
# the common case "same name on both sides" needs zero
# configuration. Override is exposed in Settings for the
# mismatch case (e.g. authentik dtoro ↔ Nextcloud admin).
nextcloud_username=(claims.get("preferred_username") or None),
)
db.add(user)
await db.flush()
@@ -455,6 +514,14 @@ async def oidc_callback(request: Request, db: AsyncSession = Depends(get_db)):
if picture and user.avatar_url != picture:
user.avatar_url = picture
changed = True
# Backfill nextcloud_username on first OIDC login for users that
# predate the column. NEVER overwrites a value the user already
# set in Settings — once the override is non-null, it wins.
if not user.nextcloud_username:
preferred = claims.get("preferred_username")
if preferred:
user.nextcloud_username = preferred
changed = True
if admin_groups:
new_role = "admin" if admin_groups.intersection(groups) else "user"
if user.role != new_role:

30
backend/app/routers/discard.py
View File

@@ -11,6 +11,7 @@ from app.database import get_db
from app.models import Photo
from app.models.user import User
from app.dependencies import get_current_user
from app.services.nextcloud_dav import delete_for_user, is_nextcloud_path
logger = logging.getLogger(__name__)
@@ -49,7 +50,7 @@ async def empty_discard(db: AsyncSession = Depends(get_db), current_user: User =
select(Photo).where(Photo.is_discarded == True, Photo.user_id == current_user.id)
)
photos = result.scalars().all()
return await _permanently_delete(db, photos)
return await _permanently_delete(db, photos, current_user)
@router.delete("")
@@ -70,19 +71,38 @@ async def delete_discarded(
)
)
photos = result.scalars().all()
return await _permanently_delete(db, photos)
return await _permanently_delete(db, photos, current_user)
async def _permanently_delete(db: AsyncSession, photos: list[Photo]) -> dict:
async def _permanently_delete(db: AsyncSession, photos: list[Photo], user: User) -> dict:
"""Shared helper: unlink files for the given photos and delete their
rows. Per-file errors are counted but don't abort the batch.
For files inside a Nextcloud-rooted SourceRoot the unlink is dispatched
through Nextcloud's WebDAV `DELETE` so Nextcloud moves the file into
the user's trashbin and updates `oc_filecache`. For everything else we
fall back to plain `os.unlink`.
"""
deleted = 0
file_errors = 0
for photo in photos:
try:
if photo.filepath and os.path.exists(photo.filepath):
os.unlink(photo.filepath)
if photo.filepath:
if is_nextcloud_path(photo.filepath):
# WebDAV DELETE — Nextcloud moves to trashbin and
# updates oc_filecache. The bind mount will then
# reflect the file's absence (Nextcloud writes
# synchronously). 404 from NC is treated as already
# gone (idempotent).
delete_for_user(user, photo.filepath)
elif os.path.exists(photo.filepath):
os.unlink(photo.filepath)
except HTTPException as e:
# WebDAV-side error — surface to caller via the file_errors
# counter rather than aborting the whole batch.
file_errors += 1
logger.error(f"Failed to delete {photo.filepath} via Nextcloud: {e.detail}")
continue
except OSError as e:
file_errors += 1
logger.error(f"Failed to unlink {photo.filepath}: {e}")

75
backend/app/routers/folders.py
View File

@@ -18,6 +18,12 @@ from app.database import get_db
from app.models import Folder, SourceRoot, Photo
from app.models.user import User
from app.dependencies import get_current_user, get_user_folder
from app.services.nextcloud_dav import (
delete_for_user as nc_delete,
is_nextcloud_path,
mkcol_for_user,
move_for_user as nc_move,
)
logger = logging.getLogger(__name__)
@@ -251,10 +257,16 @@ async def rename_folder(
detail=f"A folder named '{name}' already exists here",
)
try:
shutil.move(old_path, new_path)
except OSError as e:
raise HTTPException(status_code=500, detail=f"Rename failed: {e}")
if is_nextcloud_path(old_path):
# WebDAV MOVE keeps Nextcloud's oc_filecache + sharing metadata
# consistent. NC's MOVE is recursive — descendants come along,
# exactly like shutil.move.
nc_move(current_user, old_path, new_path)
else:
try:
shutil.move(old_path, new_path)
except OSError as e:
raise HTTPException(status_code=500, detail=f"Rename failed: {e}")
# Update folder paths: this row + every descendant. SQLite REPLACE
# rewrites the prefix; we use the trailing separator on the LIKE
@@ -309,10 +321,14 @@ async def create_folder(body: FolderCreate, db: AsyncSession = Depends(get_db),
detail=f"A folder named '{name}' already exists here",
)
try:
os.makedirs(new_path, exist_ok=False)
except OSError as e:
raise HTTPException(status_code=500, detail=f"Create failed: {e}")
if is_nextcloud_path(new_path):
# MKCOL via WebDAV so Nextcloud knows about the new collection.
mkcol_for_user(current_user, new_path)
else:
try:
os.makedirs(new_path, exist_ok=False)
except OSError as e:
raise HTTPException(status_code=500, detail=f"Create failed: {e}")
new_folder = Folder(
name=name,
@@ -395,14 +411,28 @@ async def delete_folder(
# mode == 'permanent'
file_errors = 0
for p in photos:
folder_is_nc = is_nextcloud_path(folder_path)
if folder_is_nc:
# One WebDAV DELETE on the folder itself does the recursive
# delete (NC moves the whole tree to trashbin and updates
# oc_filecache for everything inside). Skip per-photo unlinks.
try:
if p.filepath and os.path.exists(p.filepath):
os.unlink(p.filepath)
except OSError as e:
file_errors += 1
logger.error(f"Failed to unlink {p.filepath}: {e}")
await db.delete(p)
nc_delete(current_user, folder_path)
except HTTPException as e:
logger.error(f"Nextcloud DELETE failed for {folder_path}: {e.detail}")
raise
for p in photos:
await db.delete(p)
else:
for p in photos:
try:
if p.filepath and os.path.exists(p.filepath):
os.unlink(p.filepath)
except OSError as e:
file_errors += 1
logger.error(f"Failed to unlink {p.filepath}: {e}")
await db.delete(p)
# Delete this folder + every descendant Folder row.
await db.execute(
@@ -412,13 +442,14 @@ async def delete_folder(
)
)
try:
if os.path.isdir(folder_path):
shutil.rmtree(folder_path)
except OSError as e:
logger.error(f"Failed to rmtree {folder_path}: {e}")
# Don't raise — DB rows are already gone, leaving an orphan
# directory is the lesser evil.
if not folder_is_nc:
try:
if os.path.isdir(folder_path):
shutil.rmtree(folder_path)
except OSError as e:
logger.error(f"Failed to rmtree {folder_path}: {e}")
# Don't raise — DB rows are already gone, leaving an orphan
# directory is the lesser evil.
await db.commit()
return {

323
backend/app/routers/nextcloud.py Normal file
View File

@@ -0,0 +1,323 @@
"""Nextcloud integration router — folder picker + per-user SourceRoots.
Exposes three things:
- GET /api/v1/nextcloud/whoami?candidate=<name>
Validate that a Nextcloud username actually has a files/ tree
on the mounted homecloud volume. Used by the Settings UI to
sanity-check the override field before saving.
- GET /api/v1/nextcloud/browse?path=<rel>
List immediate subdirectories of the current user's Nextcloud
files tree, scoped server-side to their nextcloud_username.
Powers the folder picker.
- POST /api/v1/nextcloud/source-roots {name, nextcloud_path}
DELETE /api/v1/nextcloud/source-roots/{id}
Add or remove a per-user SourceRoot pointing at a Nextcloud
subfolder. Adding kicks off an immediate scan_folder task so
photos start appearing without a full library re-scan.
All paths are normalized with realpath and rejected if they escape the
user's allowed root — defense-in-depth against `..` and symlink
shenanigans.
"""
from __future__ import annotations
import logging
import os
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.dependencies import get_current_user
from app.models.folders import SourceRoot, Folder
from app.models.user import User
from app.services.nextcloud_dav import (
NEXTCLOUD_USERS_ROOT,
is_nextcloud_path,
whoami_dir_exists,
)
logger = logging.getLogger(__name__)
router = APIRouter()
def _user_root(nc_username: str) -> str:
"""Absolute path of `<NEXTCLOUD_USERS_ROOT>/<user>/files`."""
return os.path.join(NEXTCLOUD_USERS_ROOT, nc_username, "files")
def _resolve_under_user_root(nc_username: str, rel: str) -> str:
"""Resolve `rel` (a relative path the client supplied) under the
user's Nextcloud `files/` directory and ensure the result is still
inside that root. Returns the absolute, realpath-normalized path.
Raises 400 on traversal attempts (`..`, absolute paths, symlinks
that point outside the root)."""
rel = (rel or "").lstrip("/")
if any(seg in ("..",) for seg in rel.split("/") if seg):
raise HTTPException(status_code=400, detail="Invalid path")
base = _user_root(nc_username)
candidate = os.path.realpath(os.path.join(base, rel))
base_real = os.path.realpath(base)
if candidate != base_real and not candidate.startswith(base_real + os.sep):
raise HTTPException(status_code=400, detail="Path escapes Nextcloud root")
return candidate
# ---------------------------------------------------------------------------
class BrowseEntry(BaseModel):
name: str
path: str # path relative to the user's files/ root
has_children: bool # True if it contains at least one sub-directory
class BrowseResponse(BaseModel):
nc_username: str
rel_path: str
parent_rel: Optional[str] # None at the root
entries: list[BrowseEntry]
@router.get("/whoami")
async def whoami(
candidate: Optional[str] = Query(None, description="Nextcloud username to validate"),
current_user: User = Depends(get_current_user),
):
"""Public to authenticated users — returns whether a Nextcloud
username has a real files/ directory on the bind mount. Used by
the Settings UI to validate the override field before save."""
name = (candidate or current_user.nextcloud_username or "").strip()
if not name:
return {
"configured": bool(current_user.nextcloud_username),
"candidate": None,
"valid": False,
"reason": "no_username",
}
valid = whoami_dir_exists(name)
return {
"configured": bool(current_user.nextcloud_username),
"candidate": name,
"valid": valid,
"reason": None if valid else "no_files_dir",
}
@router.get("/browse", response_model=BrowseResponse)
async def browse(
path: str = Query("", description="Path relative to the user's NC files/ root"),
current_user: User = Depends(get_current_user),
):
"""List immediate subdirectories of the current user's NC tree."""
nc_user = (current_user.nextcloud_username or "").strip()
if not nc_user:
raise HTTPException(
status_code=412,
detail="Set your Nextcloud username in Settings → Library first.",
)
if not whoami_dir_exists(nc_user):
raise HTTPException(
status_code=404,
detail=(
f"Nextcloud user '{nc_user}' has no files/ directory on the "
"mounted homecloud volume. Check your nextcloud_username override."
),
)
abs_path = _resolve_under_user_root(nc_user, path)
if not os.path.isdir(abs_path):
raise HTTPException(status_code=404, detail="Folder not found")
base = _user_root(nc_user)
entries: list[BrowseEntry] = []
try:
with os.scandir(abs_path) as it:
for de in it:
# Skip hidden and Nextcloud's appdata noise.
if de.name.startswith("."):
continue
if not de.is_dir(follow_symlinks=False):
continue
child_abs = os.path.join(abs_path, de.name)
rel = os.path.relpath(child_abs, base)
# Quick has_children probe: any subdir that's a real
# directory. Cap at first hit so deep trees don't slow
# the picker.
has_children = False
try:
with os.scandir(child_abs) as sub:
for s in sub:
if s.name.startswith("."):
continue
if s.is_dir(follow_symlinks=False):
has_children = True
break
except OSError:
has_children = False
entries.append(BrowseEntry(name=de.name, path=rel, has_children=has_children))
except PermissionError:
raise HTTPException(
status_code=502,
detail=(
"Cannot read Nextcloud folder — backend container lacks "
"filesystem permissions on the mount. Apply ACL fix on the host."
),
)
entries.sort(key=lambda e: e.name.lower())
rel_path = os.path.relpath(abs_path, _user_root(nc_user)) if abs_path != _user_root(nc_user) else ""
if rel_path == ".":
rel_path = ""
parent_rel: Optional[str] = None
if rel_path:
parent = os.path.dirname(rel_path)
parent_rel = parent
return BrowseResponse(
nc_username=nc_user,
rel_path=rel_path,
parent_rel=parent_rel,
entries=entries,
)
# ---------------------------------------------------------------------------
class SourceRootCreate(BaseModel):
name: str
nextcloud_path: str # relative to the user's files/ root
@router.post("/source-roots", status_code=201)
async def create_nextcloud_source_root(
body: SourceRootCreate,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Register a Nextcloud subfolder as a SourceRoot for the current user
and kick off the initial scan."""
nc_user = (current_user.nextcloud_username or "").strip()
if not nc_user:
raise HTTPException(
status_code=412,
detail="Set your Nextcloud username in Settings → Library first.",
)
name = (body.name or "").strip()
if not name:
raise HTTPException(status_code=400, detail="Name is required")
abs_path = _resolve_under_user_root(nc_user, body.nextcloud_path)
if not os.path.isdir(abs_path):
raise HTTPException(status_code=404, detail="Folder not found in Nextcloud tree")
# Don't allow registering the user's `files/` root itself as a
# SourceRoot — it'd index everything they own (Documents, Notes,
# appdata noise). Force them to pick a subfolder.
if abs_path == os.path.realpath(_user_root(nc_user)):
raise HTTPException(
status_code=400,
detail="Pick a subfolder; the whole files/ root is too broad.",
)
# Refuse duplicates — the path is uniquely indexed but a clean error
# beats a 500 from the unique constraint.
existing = await db.execute(
select(SourceRoot).where(SourceRoot.path == abs_path)
)
if existing.scalar_one_or_none() is not None:
raise HTTPException(status_code=409, detail="A SourceRoot for that path already exists")
sr = SourceRoot(
name=name,
path=abs_path,
user_id=current_user.id,
is_active=True,
)
db.add(sr)
await db.flush()
await db.commit()
await db.refresh(sr)
# Kick off the initial scan. Failures here shouldn't block the
# SourceRoot creation — the user can hit "Re-scan source folders"
# from Settings if Celery is wedged.
try:
from app.tasks.celery import celery_app
celery_app.send_task("scan_folder", args=[sr.path, sr.id])
except Exception as exc: # noqa: BLE001
logger.warning("Failed to queue initial scan for new SourceRoot %s: %s", sr.id, exc)
return {
"id": sr.id,
"name": sr.name,
"path": sr.path,
"user_id": sr.user_id,
"is_active": sr.is_active,
"is_nextcloud": True,
}
@router.delete("/source-roots/{source_root_id}", status_code=204)
async def delete_nextcloud_source_root(
source_root_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Soft-delete: deactivate the SourceRoot and orphan its photo rows
(mark them is_discarded). The actual files in Nextcloud are NOT
touched — this is just unwiring the user's view of the library.
The user can re-add the same path later to restart scanning."""
sr = (await db.execute(
select(SourceRoot).where(
SourceRoot.id == source_root_id,
SourceRoot.user_id == current_user.id,
)
)).scalar_one_or_none()
if sr is None:
raise HTTPException(status_code=404, detail="SourceRoot not found")
if not is_nextcloud_path(sr.path):
raise HTTPException(
status_code=400,
detail="This endpoint only manages Nextcloud-rooted SourceRoots",
)
# Soft-deactivate. Existing photos stay in the DB but won't appear
# in active queries (folders.py filters by is_active=true).
sr.is_active = False
await db.commit()
return None
# ---------------------------------------------------------------------------
@router.get("/source-roots")
async def list_nextcloud_source_roots(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""All Nextcloud-rooted SourceRoots owned by the current user.
Used by the Settings panel to render the manage list."""
rows = (await db.execute(
select(SourceRoot).where(SourceRoot.user_id == current_user.id)
)).scalars().all()
return [
{
"id": r.id,
"name": r.name,
"path": r.path,
"is_active": r.is_active,
"is_nextcloud": True,
}
for r in rows
if is_nextcloud_path(r.path)
]

72
backend/app/routers/photos.py
View File

@@ -31,8 +31,22 @@ from app.dependencies import (
get_user_or_shared_heap, get_user_or_shared_folder,
can_access_photo_via_share,
)
from app.services.nextcloud_dav import is_nextcloud_path, move_for_user
from app.config import settings
def _nc_unsupported(detail: str) -> HTTPException:
"""Returns a 501 with a UI-friendly hint to use Nextcloud's web UI
for an op we haven't routed through WebDAV yet. Centralised so the
message stays consistent."""
return HTTPException(
status_code=501,
detail=(
f"{detail} For Nextcloud-managed libraries, do this from "
"Nextcloud's web UI; mule-image will pick up the change."
),
)
router = APIRouter()
@router.get("")
@@ -843,11 +857,18 @@ async def update_photo(
if os.path.exists(new_path):
raise HTTPException(status_code=409, detail="A file with that name already exists")
try:
os.rename(photo.filepath, new_path)
except OSError as e:
logger.error(f"Failed to rename {photo.filepath} -> {new_path}: {e}")
raise HTTPException(status_code=500, detail=f"Rename failed: {e}")
if is_nextcloud_path(photo.filepath):
# WebDAV MOVE within the same directory == rename. Lands
# the new name in oc_filecache so Nextcloud's web UI and
# sync clients see it; the bind mount reflects the
# rename for our own watcher.
move_for_user(current_user, photo.filepath, new_path)
else:
try:
os.rename(photo.filepath, new_path)
except OSError as e:
logger.error(f"Failed to rename {photo.filepath} -> {new_path}: {e}")
raise HTTPException(status_code=500, detail=f"Rename failed: {e}")
photo.filename = new_name
photo.filepath = new_path
@@ -974,6 +995,15 @@ async def copy_photos(
return candidate
return None
target_nc = is_nextcloud_path(target_dir)
if target_nc:
# Copy into a Nextcloud-managed folder isn't routed through
# WebDAV PUT yet — would need a download from the source path
# plus an upload, doubling IO. Defer until someone needs it.
raise _nc_unsupported(
"Copy into a Nextcloud-managed folder isn't supported yet."
)
for photo in photos_to_copy:
if not os.path.exists(photo.filepath):
errors.append({"id": photo.id, "error": "source file missing"})
@@ -986,6 +1016,17 @@ async def copy_photos(
new_path = os.path.join(target_dir, new_name)
if is_nextcloud_path(photo.filepath):
# Source is NC, dest is local. We could `shutil.copy2`
# since NC files are readable on the bind mount, but the
# local /photos tree has different ownership semantics —
# leave this off until we've thought about it.
errors.append({
"id": photo.id,
"error": "Cross-system copy (Nextcloud ↔ local) not supported yet",
})
continue
try:
shutil.copy2(photo.filepath, new_path)
except OSError as e:
@@ -1099,8 +1140,27 @@ async def move_photos(
errors.append({"id": photo.id, "error": f"name already exists in target: {photo.filename}"})
continue
src_nc = is_nextcloud_path(photo.filepath)
dst_nc = is_nextcloud_path(new_path)
if src_nc != dst_nc:
# Cross-system moves (NC ↔ local /photos) aren't supported
# in v1 — the user can copy via Nextcloud's web UI or use
# the desktop sync client to move the file, and mule-image
# will pick up both sides via inotify.
errors.append({
"id": photo.id,
"error": "Cross-system move (Nextcloud ↔ local) not supported yet",
})
continue
try:
shutil.move(photo.filepath, new_path)
if src_nc:
move_for_user(current_user, photo.filepath, new_path)
else:
shutil.move(photo.filepath, new_path)
except HTTPException as e:
errors.append({"id": photo.id, "error": str(e.detail)})
continue
except OSError as e:
errors.append({"id": photo.id, "error": str(e)})
continue

84
backend/app/routers/upload.py
View File

@@ -18,6 +18,7 @@ the same thumb + metadata pipeline that the scanner uses. An optional
import hashlib
import logging
import os
import tempfile
from pathlib import Path
from datetime import datetime
from typing import Optional
@@ -32,6 +33,11 @@ from app.models import Folder, Heap, Photo, SourceRoot
from app.models.heaps import heap_photos
from app.models.user import User
from app.services.date_guess import has_date_warning
from app.services.nextcloud_dav import (
ensure_parents_for_user,
is_nextcloud_path,
put_for_user,
)
from app.tasks.scan import SUPPORTED_EXTENSIONS, get_media_type
from app.tasks.thumbs import generate_thumbnails
from app.services.metadata import extract_metadata
@@ -122,8 +128,11 @@ async def _ensure_subfolder(
) -> Folder:
"""Return (or create) a Folder row named `name` under `parent`.
Also mkdirs the directory on disk. Idempotent — safe to call for a
path segment that already exists as a Folder row or directory.
Also mkdirs the directory on disk. For Nextcloud-rooted paths the
directory is created via WebDAV MKCOL so Nextcloud's `oc_filecache`
knows about it; otherwise plain `os.makedirs`. Idempotent — safe
to call for a path segment that already exists as a Folder row or
directory.
"""
child_path = os.path.normpath(os.path.join(parent.path, name))
@@ -134,10 +143,23 @@ async def _ensure_subfolder(
)
)).scalar_one_or_none()
if existing is not None:
os.makedirs(child_path, exist_ok=True)
# Materialise the directory if it wasn't already.
if is_nextcloud_path(child_path):
ensure_parents_for_user(user, child_path)
# Also ensure the leaf collection exists; ensure_parents
# only handles intermediate dirs.
from app.services.nextcloud_dav import mkcol_for_user
mkcol_for_user(user, child_path)
else:
os.makedirs(child_path, exist_ok=True)
return existing
os.makedirs(child_path, exist_ok=True)
if is_nextcloud_path(child_path):
ensure_parents_for_user(user, child_path)
from app.services.nextcloud_dav import mkcol_for_user
mkcol_for_user(user, child_path)
else:
os.makedirs(child_path, exist_ok=True)
child = Folder(
name=name,
path=child_path,
@@ -213,38 +235,74 @@ async def upload_file(
target_folder = await _ensure_subfolder(target_folder, seg, current_user, db)
target_dir = target_folder.path
os.makedirs(target_dir, exist_ok=True)
nc_managed = is_nextcloud_path(target_dir)
if not nc_managed:
os.makedirs(target_dir, exist_ok=True)
# else: target_dir was created via WebDAV MKCOL by _ensure_subfolder
filepath, final_name = _unique_path(target_dir, leaf)
# --- stream to disk, hash as we go ----------------------------------
# --- stream the upload to a tempfile, hashing as we go ---------------
# For Nextcloud-managed destinations we then PUT the tempfile to
# WebDAV so Nextcloud's oc_filecache + sharing/comment metadata stay
# consistent. For local destinations we just rename the tempfile
# into place. Either way the hash + size are computed once.
hasher = hashlib.sha256()
total = 0
tmp_dir = os.path.dirname(filepath) if not nc_managed else None
# NamedTemporaryFile in the same directory as filepath when local
# (so the final rename is atomic on the same filesystem). For NC,
# use the system tmpdir — we re-upload via HTTP either way.
tmp = tempfile.NamedTemporaryFile(
delete=False, dir=tmp_dir, suffix=".part"
)
tmp_path = tmp.name
try:
with open(filepath, 'wb') as out:
try:
while True:
chunk = await file.read(1024 * 1024)
if not chunk:
break
total += len(chunk)
if total > MAX_UPLOAD_BYTES:
out.close()
os.unlink(filepath)
raise HTTPException(
status_code=413,
detail=f"File exceeds {MAX_UPLOAD_BYTES // (1024*1024)}MB limit",
)
hasher.update(chunk)
out.write(chunk)
tmp.write(chunk)
finally:
tmp.close()
if nc_managed:
# PUT to Nextcloud WebDAV. The PUT lands the bytes on
# `/mnt/library/homecloud/<nc_user>/files/<rel>` AND
# registers the file in oc_filecache, so the desktop sync
# client and Nextcloud's web UI both see it.
with open(tmp_path, "rb") as body:
put_for_user(current_user, filepath, body)
else:
os.replace(tmp_path, filepath)
tmp_path = None # consumed
except HTTPException:
raise
except Exception as e:
logger.error(f"Upload write failed for {filepath}: {e}")
if os.path.exists(filepath):
raise HTTPException(status_code=500, detail=f"Upload failed: {e}")
finally:
if tmp_path and os.path.exists(tmp_path):
try:
os.unlink(filepath)
os.unlink(tmp_path)
except OSError:
pass
raise HTTPException(status_code=500, detail=f"Upload failed: {e}")
if not os.path.exists(filepath):
# WebDAV wrote it; the bind mount should reflect it. If it
# doesn't, surface a clean error rather than building a Photo
# row that points at a missing file.
raise HTTPException(
status_code=502,
detail="Nextcloud accepted the upload but the file isn't visible on the mount yet.",
)
file_hash = hasher.hexdigest()

302
backend/app/services/nextcloud_dav.py Normal file
View File

@@ -0,0 +1,302 @@
"""Nextcloud WebDAV client — only the verbs we actually need.
Outgoing mutations (upload, delete, rename/move) on files that live
under a user's Nextcloud-rooted SourceRoot route through this client
instead of touching the filesystem directly. That way Nextcloud's
oc_filecache, trashbin, sharing/comments metadata, and desktop sync
clients all stay coherent — the price of bypassing it is a stale
Nextcloud and resurrected files when sync clients re-upload.
Reads (scanning, hashing, EXIF, ML pipelines) keep using the bind
mount at NEXTCLOUD_USERS_ROOT. WebDAV is far too slow for every byte
of every photo, and the read side has no consistency cost — Nextcloud
is the writer, the bind mount is the reader, that's it.
Auth: HTTP Basic with the user's Nextcloud app password (set via the
Settings UI, stored Fernet-encrypted at rest). OIDC bearer reuse is a
later optimization; app passwords work today and are well-supported.
"""
from __future__ import annotations
import logging
import os
from typing import BinaryIO, Optional, Tuple
import httpx
from fastapi import HTTPException, status
from app.config import settings
from app.models.user import User
from app.services.secrets import decrypt
logger = logging.getLogger(__name__)
# Top-level mount inside the backend container. The Nextcloud user tree
# `/mnt/library/homecloud/<nc_user>/files/...` shows up here as
# `/nextcloud-users/<nc_user>/files/...`.
NEXTCLOUD_USERS_ROOT = os.environ.get("NEXTCLOUD_USERS_ROOT", "/nextcloud-users")
def is_nextcloud_path(path: str) -> bool:
"""True iff `path` resolves under the configured NC users mount."""
if not path:
return False
norm = os.path.normpath(path)
root = os.path.normpath(NEXTCLOUD_USERS_ROOT)
return norm == root or norm.startswith(root + os.sep)
def split_nextcloud_path(path: str) -> Tuple[str, str]:
"""Return (nc_username, rel_path) for a file/dir under the NC mount.
rel_path is the path relative to `<NEXTCLOUD_USERS_ROOT>/<user>/files/`,
suitable for appending to the WebDAV base URL. Raises if `path`
isn't a Nextcloud-rooted path or doesn't sit under a `files/`
directory.
"""
norm = os.path.normpath(path)
root = os.path.normpath(NEXTCLOUD_USERS_ROOT)
if not (norm == root or norm.startswith(root + os.sep)):
raise ValueError(f"Not a Nextcloud-rooted path: {path!r}")
rest = norm[len(root):].lstrip(os.sep) # "<user>/files/foo/bar.jpg"
parts = rest.split(os.sep, 2)
if len(parts) < 3 or parts[1] != "files":
# Either we got just /<user>, /<user>/files (no rel), or a
# different second segment — only the user's `files/` tree is
# safe to mutate via WebDAV.
if len(parts) == 2 and parts[1] == "files":
return parts[0], ""
raise ValueError(
f"Path doesn't live under <user>/files/: {path!r}"
)
return parts[0], parts[2]
# ---------------------------------------------------------------------------
# Client
# ---------------------------------------------------------------------------
class NextcloudCredentialsMissing(HTTPException):
"""The user hasn't set their Nextcloud app password yet, but the
request needs it to mutate a Nextcloud-managed file. 412 because
the precondition (credentials) is missing rather than the request
itself being malformed."""
def __init__(self) -> None:
super().__init__(
status_code=status.HTTP_412_PRECONDITION_FAILED,
detail=(
"Set your Nextcloud app password in Settings → Library "
"before mutating files in your Nextcloud library."
),
)
def _credentials_for(user: User) -> tuple[str, str]:
"""Resolve the (nc_username, app_password) pair for a user.
Raises NextcloudCredentialsMissing when either is missing."""
nc_user = (user.nextcloud_username or "").strip()
app_pw = decrypt(user.nextcloud_app_password_enc)
if not nc_user or not app_pw:
raise NextcloudCredentialsMissing()
return nc_user, app_pw
def _base_url() -> str:
"""The Nextcloud WebDAV base URL (without trailing slash, without
user-suffixed path). Resolved per-call so a config reload picks up
a new value without restarting workers."""
base = (
os.environ.get("NEXTCLOUD_BASE_URL")
or getattr(settings, "nextcloud_base_url", None)
or ""
).rstrip("/")
if not base:
raise HTTPException(
status_code=500,
detail="NEXTCLOUD_BASE_URL is not configured on the backend",
)
return base
def _dav_url(nc_username: str, rel_path: str) -> str:
"""Compose the absolute WebDAV URL for a relative path under the
user's `files/` collection."""
base = _base_url()
rel = (rel_path or "").lstrip("/")
# Each segment must be URL-encoded. httpx encodes path segments at
# request time, so we hand it the raw join — but we explicitly drop
# `..` traversals here as defense in depth.
if any(seg in ("", "..") for seg in rel.split("/") if seg):
raise HTTPException(status_code=400, detail="Invalid relative path")
parts = [base, "remote.php/dav/files", nc_username]
if rel:
parts.append(rel)
return "/".join(parts)
# httpx Client TTL: short, since a single request is the unit of work.
_TIMEOUT = httpx.Timeout(30.0, connect=10.0)
def _client(auth: tuple[str, str]) -> httpx.Client:
return httpx.Client(timeout=_TIMEOUT, auth=httpx.BasicAuth(*auth), follow_redirects=False)
def _async_client(auth: tuple[str, str]) -> httpx.AsyncClient:
return httpx.AsyncClient(timeout=_TIMEOUT, auth=httpx.BasicAuth(*auth), follow_redirects=False)
def _raise_for_dav(resp: httpx.Response, action: str) -> None:
"""Translate Nextcloud WebDAV errors into FastAPI HTTPExceptions
the frontend can show. We surface Nextcloud's body verbatim when
it's small enough, since it tends to carry the actually-useful
detail (quota, permission denied, etc.)."""
if resp.is_success:
return
body = resp.text or ""
if len(body) > 400:
body = body[:400] + "…"
logger.warning("Nextcloud %s failed: %s %s — %s", action, resp.status_code, resp.reason_phrase, body[:200])
if resp.status_code in (401, 403):
raise HTTPException(
status_code=status.HTTP_502_BAD_GATEWAY,
detail=f"Nextcloud rejected the {action}: {resp.reason_phrase}. "
f"Check your app password under Settings → Library.",
)
if resp.status_code == 404:
raise HTTPException(status_code=404, detail=f"Not found in Nextcloud during {action}")
if resp.status_code == 507:
raise HTTPException(
status_code=status.HTTP_507_INSUFFICIENT_STORAGE,
detail="Nextcloud quota exceeded",
)
raise HTTPException(
status_code=status.HTTP_502_BAD_GATEWAY,
detail=f"Nextcloud error during {action}: {resp.status_code} {resp.reason_phrase}",
)
# ---------------------------------------------------------------------------
# Verbs
# ---------------------------------------------------------------------------
def delete_for_user(user: User, abs_path: str) -> None:
"""WebDAV DELETE — moves the file/dir into the user's NC trashbin.
`abs_path` is the absolute filesystem path under the bind mount."""
nc_user, app_pw = _credentials_for(user)
expected_user, rel = split_nextcloud_path(abs_path)
if expected_user != nc_user:
raise HTTPException(
status_code=403,
detail="Path belongs to a different Nextcloud user",
)
url = _dav_url(nc_user, rel)
with _client((nc_user, app_pw)) as c:
resp = c.request("DELETE", url)
# 204 = deleted. 404 = already gone (treat as success, idempotent).
if resp.status_code == 404:
logger.info("Nextcloud DELETE %s already gone, treating as success", rel)
return
_raise_for_dav(resp, "delete")
def move_for_user(user: User, src_abs: str, dst_abs: str) -> None:
"""WebDAV MOVE — rename or move within the same Nextcloud user."""
nc_user, app_pw = _credentials_for(user)
src_user, src_rel = split_nextcloud_path(src_abs)
dst_user, dst_rel = split_nextcloud_path(dst_abs)
if src_user != nc_user or dst_user != nc_user:
raise HTTPException(
status_code=403,
detail="MOVE across Nextcloud users is not supported",
)
src_url = _dav_url(nc_user, src_rel)
dst_url = _dav_url(nc_user, dst_rel)
with _client((nc_user, app_pw)) as c:
resp = c.request(
"MOVE",
src_url,
headers={"Destination": dst_url, "Overwrite": "F"},
)
_raise_for_dav(resp, "move")
def mkcol_for_user(user: User, abs_path: str) -> None:
"""WebDAV MKCOL — create a directory. Idempotent: a 405 (Method Not
Allowed) means the collection already exists, treat as success."""
nc_user, app_pw = _credentials_for(user)
expected_user, rel = split_nextcloud_path(abs_path)
if expected_user != nc_user:
raise HTTPException(
status_code=403,
detail="Path belongs to a different Nextcloud user",
)
url = _dav_url(nc_user, rel)
with _client((nc_user, app_pw)) as c:
resp = c.request("MKCOL", url)
if resp.status_code == 405:
return
_raise_for_dav(resp, "mkcol")
def put_for_user(
user: User,
abs_path: str,
fileobj: BinaryIO,
content_type: Optional[str] = None,
) -> None:
"""WebDAV PUT — upload `fileobj` to `abs_path`. Caller is
responsible for ensuring intermediate collections exist via
`mkcol_for_user`. Streams the body, no in-memory copy."""
nc_user, app_pw = _credentials_for(user)
expected_user, rel = split_nextcloud_path(abs_path)
if expected_user != nc_user:
raise HTTPException(
status_code=403,
detail="Path belongs to a different Nextcloud user",
)
url = _dav_url(nc_user, rel)
headers = {}
if content_type:
headers["Content-Type"] = content_type
with _client((nc_user, app_pw)) as c:
resp = c.request("PUT", url, content=fileobj, headers=headers)
_raise_for_dav(resp, "upload")
def ensure_parents_for_user(user: User, abs_path: str) -> None:
"""Walk the parent chain of `abs_path` under the user's NC root and
`mkcol` any missing collection. Stops at the user's `files/`
directory — never tries to create that, which is owned by Nextcloud
itself."""
nc_user, _ = _credentials_for(user)
expected_user, rel = split_nextcloud_path(abs_path)
if expected_user != nc_user:
raise HTTPException(
status_code=403,
detail="Path belongs to a different Nextcloud user",
)
if not rel:
return
parts = rel.split("/")
if len(parts) <= 1:
return # no intermediate dirs to make
accum: list[str] = []
for seg in parts[:-1]:
accum.append(seg)
sub_rel = "/".join(accum)
sub_abs = os.path.join(NEXTCLOUD_USERS_ROOT, nc_user, "files", sub_rel)
mkcol_for_user(user, sub_abs)
def whoami_dir_exists(nc_username: str) -> bool:
"""True iff the bind-mounted `<NEXTCLOUD_USERS_ROOT>/<user>/files`
directory exists. Used by the UI to validate the override field
without round-tripping to Nextcloud — the bind mount is enough to
confirm Nextcloud actually has that user."""
if not nc_username or "/" in nc_username or nc_username in (".", ".."):
return False
target = os.path.join(NEXTCLOUD_USERS_ROOT, nc_username, "files")
return os.path.isdir(target)

45
backend/app/services/secrets.py Normal file
View File

@@ -0,0 +1,45 @@
"""Symmetric encryption for credentials we have to store.
Used today for the per-user Nextcloud app password — we need the
plaintext to put it in an outgoing HTTP Basic header, so a one-way
hash won't do. Key is derived from `settings.secret_key` via SHA-256
so existing deployments don't need a separate KMS dance, and a stable
SECRET_KEY rotates these credentials automatically.
Fernet is symmetric AES-128-CBC + HMAC-SHA256 with a versioned
ciphertext envelope; good enough for column-level secrecy in a
single-host homelab. Rotate by setting a new SECRET_KEY and asking
users to re-enter their app password.
"""
import base64
import hashlib
from typing import Optional
from cryptography.fernet import Fernet, InvalidToken
from app.config import settings
def _fernet() -> Fernet:
# Fernet requires a 32-byte url-safe base64 key. SHA-256 of the
# configured secret gives us exactly 32 bytes; b64-urlsafe-encode
# to fit the API contract.
digest = hashlib.sha256(settings.secret_key.encode("utf-8")).digest()
return Fernet(base64.urlsafe_b64encode(digest))
def encrypt(plaintext: str) -> str:
"""Return a base64 token that can be stored in a VARCHAR column."""
return _fernet().encrypt(plaintext.encode("utf-8")).decode("ascii")
def decrypt(token: Optional[str]) -> Optional[str]:
"""Inverse of encrypt. Returns None for None / empty input. Raises
on tampered or wrong-key tokens — callers should treat that as
"credential unset" rather than crashing the request."""
if not token:
return None
try:
return _fernet().decrypt(token.encode("ascii")).decode("utf-8")
except InvalidToken:
return None