8 Commits

Author SHA1 Message Date
root
5c531f11da feat: runtime feature flags, upload/download, RAW decoding
Adds Redis-backed feature flags for vision stages with admin UI toggles
and manual backfill trigger, photo upload and download routers with
frontend upload modal, and rawpy-based RAW decoding with JPEG fallback
for misnamed DNGs. Fixes pgvector serialization, is_trashed filter, and
naive-datetime bind in incremental duplicate regrouping; bumps Celery
time limits on regroup tasks beyond the 5-minute default.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 21:31:52 +02:00
root
800ee447ad perf: native canvas markers for map view, drop clustering library
Replace 3K+ React CircleMarker components + MarkerClusterGroup with
native Leaflet L.circleMarker on a shared L.canvas() renderer added
in a single useEffect. Zero React components per marker — canvas
draws all points in one paint (<50ms vs multi-second freeze).

Also drops react-leaflet-cluster from the bundle (-46KB gzipped).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 16:17:32 +02:00
root
7c68e1400b fix: map view freezing browser with thousands of markers
Replace DivIcon thumbnail markers with lightweight CircleMarkers.
Each DivIcon created a DOM element with an <img> tag, so 3K+
geotagged photos meant 3K DOM nodes and 3K thumbnail requests
hitting simultaneously — freezing the browser during clustering.

CircleMarkers are SVG-rendered on Leaflet's canvas layer with no
DOM nodes per marker. Photos still open in preview on click.

Also: bump cluster radius 50→80, enable removeOutsideVisibleBounds,
disable clustering at max zoom, increase staleTime to 5 min.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 15:53:58 +02:00
root
ecd8bbe61d fix: HEIC vips fallback, DB pool exhaustion, auth session persistence
HEIC thumbnails:
- Switch fallback from ffmpeg to vips for HEIC files that pillow-heif
  rejects. ffmpeg decoded gain map tiles instead of the primary image,
  producing inverted/negative thumbnails. vips uses libheif's item
  references correctly and extracts the full-resolution primary image.

Database pool exhaustion:
- Add idle_in_transaction_session_timeout=60s so Postgres auto-kills
  leaked connections from disconnected thumbnail requests.
- Add pool_timeout=10 so new requests fail fast instead of hanging.
- Bump pool from 5+5 to 10+10 for thumbnail concurrency headroom.
- get_db rolls back on exception before closing.

Auth session persistence:
- Narrow 401 interceptor exclusion to only /auth/refresh and /auth/login
  (was excluding all /auth/* including /auth/me, preventing token refresh
  on boot).
- fetchMe only clears tokens on 401/403, not network errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 15:25:55 +02:00
root
2adaaf18a1 fix: ffmpeg fallback for HEIC files that libheif rejects
iPhone photos with depth maps or gain maps have too many auxiliary
image references for libheif 1.17, causing pillow-heif to throw
"Too many auxiliary image references". process_heic_image() now
falls back to ffmpeg when pillow-heif fails — ffmpeg's own HEIC
decoder handles these files without issue. Fixes 27/35 HEIC photos
that were stuck in failed state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 11:08:31 +02:00
root
edd569d095 feat: share heaps and folders with other users, fix auth and vision pipeline
Sharing:
- New HeapShare and FolderShare models with read/write permissions
- Sharing API router (CRUD for heap and folder shares)
- Heap endpoints accept shared access (photo_ids, add/remove with write)
- Photo list drops user_id filter in shared context, adds owner_username
- Media serving (thumb/original/proxy) falls back to share check on 404
- ShareDialog component for managing shares from kebab menus
- HeapsPanel shows "Shared with me" section for shared heaps
- LeftSidebar shows "Shared with me" section for shared folders
- Owner badge on PhotoThumbnail for photos from other users

Auth:
- Access token default bumped to 1 year, refresh to 10 years
- Refresh token persisted in localStorage (survives page reload)
- Timer-based refresh replaced with 401 axios interceptor

Vision pipeline fixes:
- Bootstrap sets Redis ready key even on partial export failure
- Export functions run conditionally (only for actually missing models)
- _load_thumb handles multi-user path (/data/thumbs/{user_id}/{photo_id}/)
- can_access_photo_via_share uses single subquery instead of N+1 loop

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 10:59:07 +02:00
root
f090a809a9 fix: harden pipeline — retries, acks_late, time limits, session safety
Addresses 16 robustness, transparency, and performance issues across
the Celery media processing pipeline:

Critical:
- Singleton DB engine in vision tasks (was leaking one per task call)
- acks_late + task_reject_on_worker_lost so crashed workers don't lose tasks
- Global soft/hard time limits (5/10 min) to prevent hung worker slots
- Thumbnail copy-before-resize (in-place mutation degraded larger sizes)
- backfill_vision now checks each task type independently (OCR, faces, etc.)
- Parameterized LIMIT in backfill_vision (was f-string SQL injection)

High:
- try/except + retry(max=3) on all vision inference tasks
- extract_metadata writes processing_error on exiftool failure
- PIL Image handles closed in _load_thumb/_load_original
- Scan progress Redis keys auto-expire after 1 hour
- Watcher lock renewal is wall-clock based (30s) not event-count based
- worker_process_init signal warms up vision models on startup

Medium:
- Explicit task_routes for every task name (wildcards never matched)
- app.services.metadata added to Celery include list
- POST /maintenance/recover-stuck endpoint for photos stuck in processing
- Docker healthchecks for worker-light, worker-vision, and Redis
- Task ID in vision log lines for distributed tracing
- Bare except:pass narrowed to specific exceptions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 08:57:10 +02:00
root
e974ffbfd2 feat: show source directories in library stats and settings UI
Add active source root directories to the library stats endpoint and
display them in the settings page. Hardcode container PHOTO_DIRS to
/photos since the volume mount handles host path mapping. Add .env to
.gitignore to prevent committing secrets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 08:09:08 +02:00
39 changed files with 3980 additions and 334 deletions

1
.gitignore vendored
View File

@@ -34,6 +34,7 @@ dist-ssr/
.DS_Store
# Environment
.env
.env.local
.env.*.local

View File

@@ -0,0 +1,54 @@
"""Add sharing tables for heaps and folders
Revision ID: 0011_sharing
Revises: 0010_embeddings_768d
Create Date: 2026-04-13
Adds heap_shares and folder_shares tables so users can share
heaps and folders with other users (read or read+write).
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "0011_sharing"
down_revision: Union[str, None] = "0010_embeddings_768d"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.execute("""
CREATE TABLE IF NOT EXISTS heap_shares (
id VARCHAR NOT NULL PRIMARY KEY,
heap_id VARCHAR NOT NULL REFERENCES heaps(id) ON DELETE CASCADE,
owner_id VARCHAR NOT NULL REFERENCES users(id),
shared_with_id VARCHAR NOT NULL REFERENCES users(id),
permission VARCHAR NOT NULL DEFAULT 'read',
created_at TIMESTAMP DEFAULT now(),
CONSTRAINT uq_heap_share UNIQUE (heap_id, shared_with_id)
)
""")
op.execute("CREATE INDEX IF NOT EXISTS ix_heap_shares_shared_with ON heap_shares(shared_with_id)")
op.execute("CREATE INDEX IF NOT EXISTS ix_heap_shares_heap_id ON heap_shares(heap_id)")
op.execute("""
CREATE TABLE IF NOT EXISTS folder_shares (
id VARCHAR NOT NULL PRIMARY KEY,
folder_id VARCHAR NOT NULL,
folder_type VARCHAR NOT NULL DEFAULT 'folder',
owner_id VARCHAR NOT NULL REFERENCES users(id),
shared_with_id VARCHAR NOT NULL REFERENCES users(id),
permission VARCHAR NOT NULL DEFAULT 'read',
created_at TIMESTAMP DEFAULT now(),
CONSTRAINT uq_folder_share UNIQUE (folder_id, shared_with_id)
)
""")
op.execute("CREATE INDEX IF NOT EXISTS ix_folder_shares_shared_with ON folder_shares(shared_with_id)")
op.execute("CREATE INDEX IF NOT EXISTS ix_folder_shares_folder_id ON folder_shares(folder_id)")
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS folder_shares")
op.execute("DROP TABLE IF EXISTS heap_shares")

View File

@@ -26,8 +26,8 @@ class PerformanceSettings(BaseModel):
"""Performance tuning settings"""
max_concurrent_thumbnails: int = 10
cache_ttl: int = 3600
db_pool_size: int = 5
db_pool_max_overflow: int = 5
db_pool_size: int = 10
db_pool_max_overflow: int = 10
db_pool_recycle: int = 3600
class EmbedderSettings(BaseModel):
@@ -141,8 +141,8 @@ class Settings(BaseSettings):
default="mulita-dev-secret-change-me",
env="SECRET_KEY",
)
access_token_expire_minutes: int = Field(default=60, env="ACCESS_TOKEN_EXPIRE_MINUTES")
refresh_token_expire_days: int = Field(default=30, env="REFRESH_TOKEN_EXPIRE_DAYS")
access_token_expire_minutes: int = Field(default=525600, env="ACCESS_TOKEN_EXPIRE_MINUTES") # 1 year
refresh_token_expire_days: int = Field(default=3650, env="REFRESH_TOKEN_EXPIRE_DAYS") # 10 years
@property
def cors_origins(self) -> list[str]:

View File

@@ -76,6 +76,10 @@ else:
max_overflow=settings.performance.db_pool_max_overflow,
pool_recycle=settings.performance.db_pool_recycle,
pool_pre_ping=True,
pool_timeout=10, # fail fast if pool exhausted (default 30)
# Kill connections idle in a transaction for >60s. Prevents leaked
# sessions from thumbnail requests that disconnect mid-flight.
connect_args={"server_settings": {"idle_in_transaction_session_timeout": "60000"}},
)
# Create async session factory
@@ -89,10 +93,17 @@ AsyncSessionLocal = async_sessionmaker(
Base = declarative_base()
async def get_db() -> AsyncSession:
"""Dependency to get database session"""
"""Dependency to get database session.
Rolls back any uncommitted transaction before closing so a client
disconnect doesn't leave idle-in-transaction connections in the pool.
"""
async with AsyncSessionLocal() as session:
try:
yield session
except Exception:
await session.rollback()
raise
finally:
await session.close()

View File

@@ -15,8 +15,9 @@ from app.database import get_db
from app.models.user import User
from app.models.photos import Photo
from app.models.folders import Folder, SourceRoot
from app.models.heaps import Heap
from app.models.heaps import Heap, heap_photos
from app.models.tags import Tag
from app.models.sharing import HeapShare, FolderShare
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
@@ -180,3 +181,175 @@ async def get_user_tag(
if tag is None:
raise HTTPException(status_code=404, detail="Tag not found")
return tag
# ---------------------------------------------------------------------------
# Sharing helpers
# ---------------------------------------------------------------------------
async def get_user_or_shared_heap(
heap_id: str,
user: User,
db: AsyncSession,
) -> tuple:
"""Fetch a heap the user owns OR has a share for.
Returns ``(heap, permission)`` where *permission* is
``'owner'``, ``'read'``, or ``'write'``. Raises 404 if no access.
"""
# Fast path: owned by current user.
result = await db.execute(
select(Heap).where(Heap.id == heap_id, Heap.user_id == user.id)
)
heap = result.scalar_one_or_none()
if heap:
return heap, "owner"
# Shared path.
result = await db.execute(
select(HeapShare).where(
HeapShare.heap_id == heap_id,
HeapShare.shared_with_id == user.id,
)
)
share = result.scalar_one_or_none()
if share:
result = await db.execute(select(Heap).where(Heap.id == heap_id))
heap = result.scalar_one_or_none()
if heap:
return heap, share.permission
raise HTTPException(status_code=404, detail="Heap not found")
async def get_user_or_shared_folder(
folder_id: str,
user: User,
db: AsyncSession,
) -> tuple:
"""Fetch a folder (or source root) the user owns OR has a share for.
Returns ``(entity, permission)`` where *entity* is a Folder or
SourceRoot and *permission* is ``'owner'``, ``'read'``, or ``'write'``.
"""
# Try owned folder first.
result = await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == user.id)
)
folder = result.scalar_one_or_none()
if folder:
return folder, "owner"
# Try owned source root.
result = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == user.id)
)
sr = result.scalar_one_or_none()
if sr:
return sr, "owner"
# Shared path.
result = await db.execute(
select(FolderShare).where(
FolderShare.folder_id == folder_id,
FolderShare.shared_with_id == user.id,
)
)
share = result.scalar_one_or_none()
if share:
if share.folder_type == "source_root":
result = await db.execute(select(SourceRoot).where(SourceRoot.id == folder_id))
else:
result = await db.execute(select(Folder).where(Folder.id == folder_id))
entity = result.scalar_one_or_none()
if entity:
return entity, share.permission
raise HTTPException(status_code=404, detail="Folder not found")
async def resolve_username(
username: str,
db: AsyncSession,
) -> User:
"""Look up an active user by username. Raises 404 if not found."""
result = await db.execute(
select(User).where(User.username == username, User.is_active.is_(True))
)
user = result.scalar_one_or_none()
if user is None:
raise HTTPException(status_code=404, detail="User not found")
return user
async def can_access_photo_via_share(
photo_id: str,
user: User,
db: AsyncSession,
) -> bool:
"""Check whether *user* can access *photo_id* through any share.
Returns True if the photo belongs to a heap or folder that has been
shared with the user. Used as a fallback in media-serving endpoints
after the direct ownership check fails.
"""
import os
# Check heap shares: photo in any heap shared with user?
result = await db.execute(
select(heap_photos.c.photo_id).where(
heap_photos.c.photo_id == photo_id,
heap_photos.c.heap_id.in_(
select(HeapShare.heap_id).where(HeapShare.shared_with_id == user.id)
),
).limit(1)
)
if result.scalar_one_or_none() is not None:
return True
# Check folder shares: photo in any folder (or descendant) shared with user?
result = await db.execute(
select(Photo.folder_id).where(Photo.id == photo_id)
)
photo_folder_id = result.scalar_one_or_none()
if photo_folder_id is None:
return False
# Get the photo's folder path for prefix matching.
result = await db.execute(
select(Folder.path, Folder.source_root_id).where(Folder.id == photo_folder_id)
)
row = result.one_or_none()
if row is None:
return False
photo_path, photo_sr_id = row
# Check source root shares — photo's source root matches a shared root?
result = await db.execute(
select(FolderShare.folder_id).where(
FolderShare.shared_with_id == user.id,
FolderShare.folder_type == "source_root",
FolderShare.folder_id == photo_sr_id,
).limit(1)
)
if result.scalar_one_or_none() is not None:
return True
# Check folder shares — photo's folder is at or below a shared folder?
# Single query: join folder_shares → folders to get shared paths, then
# check if the photo's path starts with any of them.
result = await db.execute(
select(Folder.path).where(
Folder.id.in_(
select(FolderShare.folder_id).where(
FolderShare.shared_with_id == user.id,
FolderShare.folder_type == "folder",
)
)
)
)
for (shared_path,) in result.all():
if photo_path == shared_path or photo_path.startswith(shared_path + os.sep):
return True
return False

View File

@@ -11,7 +11,7 @@ import os
from app.config import settings
from app.database import init_db
from app.routers import photos, folders, heaps, tags, discard, library, search, auth, admin
from app.routers import photos, folders, heaps, tags, discard, library, search, auth, admin, sharing, upload, download, features
from app.services.scanner import start_initial_scan, bootstrap_default_source_root
from app.services.cleanup import cleanup_data_integrity
@@ -87,6 +87,7 @@ if os.path.exists("/data/thumbs"):
# Include routers
app.include_router(auth.router, prefix="/api/v1/auth", tags=["auth"])
app.include_router(admin.router, prefix="/api/v1/admin", tags=["admin"])
app.include_router(sharing.router, prefix="/api/v1", tags=["sharing"])
app.include_router(photos.router, prefix="/api/v1/photos", tags=["photos"])
app.include_router(folders.router, prefix="/api/v1/folders", tags=["folders"])
app.include_router(heaps.router, prefix="/api/v1/heaps", tags=["heaps"])
@@ -94,6 +95,9 @@ app.include_router(tags.router, prefix="/api/v1/tags", tags=["tags"])
app.include_router(discard.router, prefix="/api/v1/discard", tags=["discard"])
app.include_router(library.router, prefix="/api/v1/library", tags=["library"])
app.include_router(search.router, prefix="/api/v1/photos/search", tags=["search"])
app.include_router(upload.router, prefix="/api/v1/upload", tags=["upload"])
app.include_router(download.router, prefix="/api/v1/download", tags=["download"])
app.include_router(features.router, prefix="/api/v1/features", tags=["features"])
@app.get("/")
async def root():

View File

@@ -9,6 +9,7 @@ from app.models.heaps import Heap, HeapPhoto
from app.models.embeddings import Embedding
from app.models.ocr_text import OCRText
from app.models.face_embedding import FaceEmbedding
from app.models.sharing import HeapShare, FolderShare
__all__ = [
'User',
@@ -22,4 +23,6 @@ __all__ = [
'Embedding',
'OCRText',
'FaceEmbedding',
'HeapShare',
'FolderShare',
]

View File

@@ -0,0 +1,52 @@
"""
Sharing models — cross-user access to heaps and folders.
HeapShare grants another user read or read+write access to a heap.
FolderShare does the same for a folder (or source root).
"""
import uuid
from sqlalchemy import (
Column, DateTime, ForeignKey, Index, String, UniqueConstraint, func,
)
from app.database import Base
class HeapShare(Base):
__tablename__ = "heap_shares"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
heap_id = Column(
String, ForeignKey("heaps.id", ondelete="CASCADE"), nullable=False,
)
# Denormalized from heap.user_id for fast "shares I own" lookups.
owner_id = Column(String, ForeignKey("users.id"), nullable=False)
shared_with_id = Column(String, ForeignKey("users.id"), nullable=False)
permission = Column(String, nullable=False, default="read") # 'read' | 'write'
created_at = Column(DateTime, server_default=func.now())
__table_args__ = (
UniqueConstraint("heap_id", "shared_with_id", name="uq_heap_share"),
Index("ix_heap_shares_shared_with", "shared_with_id"),
Index("ix_heap_shares_heap_id", "heap_id"),
)
class FolderShare(Base):
__tablename__ = "folder_shares"
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
# Can reference either a Folder.id or a SourceRoot.id.
folder_id = Column(String, nullable=False)
folder_type = Column(String, nullable=False, default="folder") # 'folder' | 'source_root'
owner_id = Column(String, ForeignKey("users.id"), nullable=False)
shared_with_id = Column(String, ForeignKey("users.id"), nullable=False)
permission = Column(String, nullable=False, default="read") # 'read' | 'write'
created_at = Column(DateTime, server_default=func.now())
__table_args__ = (
UniqueConstraint("folder_id", "shared_with_id", name="uq_folder_share"),
Index("ix_folder_shares_shared_with", "shared_with_id"),
Index("ix_folder_shares_folder_id", "folder_id"),
)

View File

@@ -18,6 +18,14 @@ from app.models.user import User
from app.models.photos import Photo
from app.models.folders import SourceRoot
from app.config import settings
from app.services.feature_flags import (
ALL_FLAGS,
snapshot as flags_snapshot,
set_flag,
reset_flag,
is_enabled,
FLAG_VISION_ENABLED,
)
logger = logging.getLogger(__name__)
@@ -258,3 +266,141 @@ async def delete_user(
logger.info(f"Admin '{admin.username}' deactivated user '{user.username}'")
return {"status": "ok", "detail": f"User '{user.username}' deactivated"}
# ---------------------------------------------------------------------------
# AI / vision feature flags + manual triggers
# ---------------------------------------------------------------------------
class FeatureFlagUpdate(BaseModel):
"""PATCH body for toggling a feature flag.
``value`` sets an explicit override (true/false); omitting it clears
the override and reverts the flag to its YAML default.
"""
value: Optional[bool] = None
@router.get("/feature-flags")
async def get_feature_flags(admin: User = Depends(require_admin)):
"""Return every tunable feature flag with its current effective
value, YAML default, and whether an admin override is in effect."""
return {"flags": flags_snapshot()}
@router.patch("/feature-flags/{flag_name}")
async def update_feature_flag(
flag_name: str,
body: FeatureFlagUpdate,
admin: User = Depends(require_admin),
):
"""Set or clear an override for one flag. With ``value`` set, the
flag is pinned to that boolean; without it, the override is deleted
and the YAML default takes over again.
New value is observed by vision tasks on their next invocation —
there's no worker restart required.
"""
if flag_name not in ALL_FLAGS:
raise HTTPException(status_code=404, detail=f"Unknown flag: {flag_name}")
try:
if body.value is None:
reset_flag(flag_name)
action = "cleared override"
else:
set_flag(flag_name, bool(body.value))
action = f"set to {body.value}"
except RuntimeError as e:
# Redis unreachable — surface as 503 so the UI doesn't think it
# succeeded silently.
raise HTTPException(status_code=503, detail=str(e))
logger.info(f"Admin '{admin.username}' {action} for flag '{flag_name}'")
return {"flags": flags_snapshot()}
class BackfillVisionBody(BaseModel):
"""POST body for triggering a vision backfill. ``task`` picks a
specific stage (``embed`` / ``ocr`` / ``detect`` / ``faces`` /
``classify``); leaving it null runs every enabled stage. ``limit``
caps how many photos per stage are queued — useful for smoke-
testing a newly-enabled feature before committing a full run.
"""
task: Optional[str] = None
limit: Optional[int] = None
@router.post("/ai/backfill")
async def trigger_ai_backfill(
body: BackfillVisionBody,
admin: User = Depends(require_admin),
):
"""Queue a vision backfill pass. Identical code path as the automatic
post-scan backfill — just triggered manually from the UI."""
if not is_enabled(FLAG_VISION_ENABLED):
raise HTTPException(
status_code=400,
detail="Vision is currently disabled; enable it before running a backfill.",
)
valid_tasks = {'embed', 'ocr', 'detect', 'faces', 'classify'}
if body.task is not None and body.task not in valid_tasks:
raise HTTPException(
status_code=400,
detail=f"task must be one of {sorted(valid_tasks)} or null",
)
if body.limit is not None and body.limit <= 0:
raise HTTPException(status_code=400, detail="limit must be positive")
# Import lazily so importing admin.py doesn't pull in the whole
# vision stack on startup (Celery task module loads numpy etc.).
from app.tasks.vision import backfill_vision
result = backfill_vision.apply_async(
kwargs={'task': body.task, 'limit': body.limit}
)
logger.info(
f"Admin '{admin.username}' queued vision backfill "
f"(task={body.task}, limit={body.limit}, celery_id={result.id})"
)
return {
"status": "queued",
"task_id": result.id,
"task": body.task,
"limit": body.limit,
}
@router.post("/ai/recluster-faces")
async def trigger_face_recluster(admin: User = Depends(require_admin)):
"""Kick off face recluster. Normally auto-fires after a scan via a
debounced scheduler; this endpoint is for admins who want to force
a fresh clustering pass (e.g. after tweaking ``cluster_eps`` in
the YAML config)."""
if not is_enabled(FLAG_VISION_ENABLED):
raise HTTPException(
status_code=400,
detail="Vision is currently disabled; enable it before reclustering.",
)
from app.tasks.vision import recluster_faces
result = recluster_faces.apply_async()
logger.info(
f"Admin '{admin.username}' queued face recluster (celery_id={result.id})"
)
return {"status": "queued", "task_id": result.id}
@router.post("/ai/rescan")
async def trigger_full_rescan(admin: User = Depends(require_admin)):
"""Dispatch the same scan_all_source_roots job the backend runs at
startup. Picks up any new files on disk and, through the
post-scan hook, queues a vision backfill for whatever still lacks
embeddings / OCR / etc.
"""
from app.tasks.scan import scan_all_source_roots
result = scan_all_source_roots.apply_async()
logger.info(
f"Admin '{admin.username}' queued full rescan (celery_id={result.id})"
)
return {"status": "queued", "task_id": result.id}

View File

@@ -0,0 +1,247 @@
"""
Download router — streams a .zip of every photo in a folder (recursively)
or a heap back to the browser.
Auth: both endpoints accept the regular Authorization header *or* a
``?token=JWT`` query string, mirroring the media endpoints. That lets the
frontend trigger a download with a plain ``<a href>`` (which can't set a
header), keeping the client side a one-liner.
Implementation: we build the zip into a ``NamedTemporaryFile`` and then
stream its bytes back, deleting the temp file on the way out. Stored
(uncompressed) mode because photos and videos are already compressed —
deflating them again just burns CPU for a fraction of a percent. For
very large libraries the temp-file route is mildly wasteful vs. a true
streaming zip (zipstream-ng etc), but it avoids a new dependency and
handles arbitrary folder sizes without blowing out RAM.
"""
import logging
import os
import re
import tempfile
import zipfile
from typing import List
from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import StreamingResponse
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.dependencies import get_current_user_media
from app.models import Folder, Heap, Photo, SourceRoot
from app.models.heaps import heap_photos
from app.models.user import User
logger = logging.getLogger(__name__)
router = APIRouter()
def _safe_filename(name: str) -> str:
"""Strip characters that Content-Disposition or Windows filesystems
would choke on. Keeps the download's filename readable without
needing any escaping on the client side."""
cleaned = re.sub(r'[\\/:*?"<>|\r\n\t]', '_', name).strip().strip('.')
return cleaned or 'download'
async def _collect_folder_photos(
folder_id: str,
user: User,
db: AsyncSession,
) -> tuple[str, str, List[Photo]]:
"""Resolve a folder or source-root id → (base_path, display_name,
photos). ``base_path`` is the prefix we strip off each photo's
filepath when naming zip entries, so the archive mirrors the user's
on-disk structure under that folder.
"""
folder = (await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == user.id)
)).scalar_one_or_none()
base_path: str
display_name: str
if folder is not None:
base_path = os.path.normpath(folder.path)
display_name = folder.name or os.path.basename(base_path)
else:
sr = (await db.execute(
select(SourceRoot).where(
SourceRoot.id == folder_id,
SourceRoot.user_id == user.id,
)
)).scalar_one_or_none()
if sr is None:
raise HTTPException(status_code=404, detail="Folder not found")
base_path = os.path.normpath(sr.path)
display_name = sr.name or os.path.basename(base_path)
# Every photo whose filepath is at or below the base path — matches
# the same prefix convention folders.py uses for recursive deletes.
descendant_prefix = base_path.rstrip(os.sep) + os.sep
result = await db.execute(
select(Photo).where(
Photo.user_id == user.id,
Photo.is_discarded == False, # noqa: E712
(Photo.filepath == base_path) | (Photo.filepath.like(descendant_prefix + '%')),
)
)
photos = list(result.scalars().all())
return base_path, display_name, photos
def _build_zip(
photos: List[Photo],
arcname_fn,
) -> tempfile.NamedTemporaryFile:
"""Write ``photos`` into a fresh ZIP_STORED temp file.
``arcname_fn(photo, used_names)`` returns the entry name to use for
the given photo; the caller supplies it because folder downloads
want path-preserving names while heap downloads flatten to bare
filenames (with a collision suffix).
"""
tmp = tempfile.NamedTemporaryFile(delete=False, suffix='.zip')
try:
used: set[str] = set()
with zipfile.ZipFile(tmp, 'w', zipfile.ZIP_STORED, allowZip64=True) as zf:
for p in photos:
if not p.filepath or not os.path.exists(p.filepath):
# Silent skip: the scanner may have indexed files
# that have since been moved / unlinked by a shell.
continue
name = arcname_fn(p, used)
used.add(name)
try:
zf.write(p.filepath, name)
except OSError as e:
logger.warning(f"Skipping {p.filepath} in zip: {e}")
tmp.close()
return tmp
except Exception:
tmp.close()
try:
os.unlink(tmp.name)
except OSError:
pass
raise
def _stream_and_cleanup(path: str):
"""Yield the temp zip in 1 MiB chunks and unlink it when the
iterator is exhausted (or GC'd, if the client disconnects early)."""
try:
with open(path, 'rb') as f:
while True:
chunk = f.read(1024 * 1024)
if not chunk:
break
yield chunk
finally:
try:
os.unlink(path)
except OSError as e:
logger.debug(f"Temp zip cleanup failed for {path}: {e}")
def _dedupe(name: str, used: set[str]) -> str:
"""Return ``name`` (or ``name (2)``, ``name (3)`` ...) such that the
result doesn't collide with anything in ``used``. Needed for heap
downloads where two members can have identical filenames from
different folders."""
if name not in used:
return name
stem, ext = os.path.splitext(name)
n = 2
while True:
cand = f"{stem} ({n}){ext}"
if cand not in used:
return cand
n += 1
@router.get("/folders/{folder_id}")
async def download_folder(
folder_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user_media),
):
"""Zip every (non-discarded) photo under a folder/source-root and
stream it back. Entries preserve the folder structure relative to
the downloaded root so the resulting archive is a faithful snapshot.
"""
base_path, display_name, photos = await _collect_folder_photos(
folder_id, current_user, db
)
if not photos:
raise HTTPException(status_code=404, detail="No photos to download")
def arcname(p: Photo, _used: set[str]) -> str:
# Relative path from the download root, falling back to the
# bare filename if the photo somehow lives outside base_path.
abs_path = os.path.normpath(p.filepath)
if abs_path.startswith(base_path + os.sep):
rel = abs_path[len(base_path) + 1:]
elif abs_path == base_path:
rel = os.path.basename(abs_path)
else:
rel = p.filename or os.path.basename(abs_path)
# Nest everything under display_name so users see one top-level
# folder inside the zip rather than loose files.
return os.path.join(_safe_filename(display_name), rel)
tmp = _build_zip(photos, arcname)
filename = _safe_filename(display_name) + '.zip'
return StreamingResponse(
_stream_and_cleanup(tmp.name),
media_type='application/zip',
headers={
'Content-Disposition': f'attachment; filename="{filename}"',
'Content-Length': str(os.path.getsize(tmp.name)),
},
)
@router.get("/heaps/{heap_id}")
async def download_heap(
heap_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user_media),
):
"""Zip every photo in a heap. Heaps are flat collections, so entries
use the original filename (with a ``(2)`` collision suffix when
two members share a name)."""
heap = (await db.execute(
select(Heap).where(Heap.id == heap_id, Heap.user_id == current_user.id)
)).scalar_one_or_none()
if heap is None:
raise HTTPException(status_code=404, detail="Heap not found")
result = await db.execute(
select(Photo)
.join(heap_photos, heap_photos.c.photo_id == Photo.id)
.where(
heap_photos.c.heap_id == heap_id,
Photo.is_discarded == False, # noqa: E712
)
)
photos = list(result.scalars().all())
if not photos:
raise HTTPException(status_code=404, detail="Heap is empty")
def arcname(p: Photo, used: set[str]) -> str:
bare = p.filename or os.path.basename(p.filepath or 'photo')
entry = os.path.join(_safe_filename(heap.name), _dedupe(bare, used))
return entry
tmp = _build_zip(photos, arcname)
filename = _safe_filename(heap.name) + '.zip'
return StreamingResponse(
_stream_and_cleanup(tmp.name),
media_type='application/zip',
headers={
'Content-Disposition': f'attachment; filename="{filename}"',
'Content-Length': str(os.path.getsize(tmp.name)),
},
)

View File

@@ -0,0 +1,23 @@
"""
Public feature-flag read API — lets the authenticated frontend know
which AI-powered sections to render.
This is NOT the admin mutation endpoint (that's in ``admin.py`` and
gated by ``require_admin``). Here we only expose the effective boolean
state so the UI can hide things like the People view, Tags view, or
text-search affordances when the underlying pipeline stage is off.
"""
from fastapi import APIRouter, Depends
from app.dependencies import get_current_user
from app.models.user import User
from app.services.feature_flags import ALL_FLAGS, is_enabled
router = APIRouter()
@router.get("")
async def get_enabled_features(_: User = Depends(get_current_user)):
"""Return ``{flag_name: bool}`` for every known flag, reflecting
the currently effective value (admin override or YAML default)."""
return {name: is_enabled(name) for name in ALL_FLAGS}

View File

@@ -15,7 +15,7 @@ from app.models import Heap, Photo, Folder
from app.models.folders import SourceRoot
from app.models.heaps import heap_photos
from app.models.user import User
from app.dependencies import get_current_user, get_user_heap
from app.dependencies import get_current_user, get_user_heap, get_user_or_shared_heap
logger = logging.getLogger(__name__)
@@ -216,7 +216,7 @@ async def get_heap_photo_ids(
to maintain a fast client-side membership lookup for the active heap
(for the basket affordance on thumbnails) without fetching full photo
records."""
await get_user_heap(heap_id, current_user, db)
await get_user_or_shared_heap(heap_id, current_user, db)
result = await db.execute(
select(heap_photos.c.photo_id).where(heap_photos.c.heap_id == heap_id)
)
@@ -231,8 +231,11 @@ async def add_photos_to_heap(
current_user: User = Depends(get_current_user),
):
"""Add photos to a heap. Idempotent: re-adding existing members is a
no-op (handled by an INSERT OR IGNORE-style filter on duplicates)."""
await get_user_heap(heap_id, current_user, db)
no-op (handled by an INSERT OR IGNORE-style filter on duplicates).
Shared users with write permission can add their own photos."""
_heap, permission = await get_user_or_shared_heap(heap_id, current_user, db)
if permission == "read":
raise HTTPException(status_code=403, detail="Read-only access to this heap")
if not body.photo_ids:
return {"status": "success", "added": 0}
@@ -426,8 +429,11 @@ async def remove_photos_from_heap(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Remove photos from a heap. Removing a non-member is a no-op."""
await get_user_heap(heap_id, current_user, db)
"""Remove photos from a heap. Removing a non-member is a no-op.
Shared users with write permission can remove photos."""
_heap, permission = await get_user_or_shared_heap(heap_id, current_user, db)
if permission == "read":
raise HTTPException(status_code=403, detail="Read-only access to this heap")
if not body.photo_ids:
return {"status": "success", "removed": 0}

View File

@@ -18,6 +18,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models import Photo
from app.models.folders import SourceRoot
from app.models.user import User
from app.dependencies import get_current_user
@@ -105,6 +106,13 @@ async def get_library_stats(
size = (await db.execute(select(func.sum(Photo.file_size)).where(owner))).scalar() or 0
# Source root directories (active ones only).
roots = (
await db.execute(
select(SourceRoot.path).where(SourceRoot.is_active.is_(True)).order_by(SourceRoot.path)
)
).scalars().all()
return {
"all_photos": all_photos_count,
"rated": rated_count,
@@ -116,6 +124,7 @@ async def get_library_stats(
"total_videos": video_count,
"total_size": size,
"total_size_gb": round(size / (1024**3), 2) if size else 0,
"source_dirs": roots,
}
@router.post("/scan")
@@ -128,6 +137,35 @@ async def trigger_scan(current_user: User = Depends(get_current_user)):
return {"status": "success", "message": "Library scan started"}
@router.post("/maintenance/recover-stuck")
async def recover_stuck_photos(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Reset photos stuck in 'processing' for more than 30 minutes back to
'pending' so the pipeline can retry them. Returns the count of recovered
photos."""
from datetime import datetime, timedelta, timezone
cutoff = datetime.now(timezone.utc) - timedelta(minutes=30)
result = await db.execute(
update(Photo)
.where(
Photo.processing_status == 'processing',
Photo.updated_at < cutoff,
)
.values(
processing_status='pending',
processing_error='Auto-recovered from stuck processing state',
)
)
await db.commit()
count = result.rowcount
if count:
logger.info("Recovered %d stuck photos back to pending", count)
return {"status": "success", "recovered": count}
@router.post("/backfill-gps")
async def trigger_backfill_gps(current_user: User = Depends(get_current_user)):
"""Re-run EXIF metadata extraction on every photo that's still missing

View File

@@ -26,7 +26,11 @@ from app.models.tags import photo_tags
from app.schemas.photos import PhotoResponse, PhotoUpdate, PhotoListResponse, BulkAction
from app.services.exif_writer import ExifWriteError, write_taken_at
from app.services.date_guess import has_date_warning as compute_date_warning
from app.dependencies import get_current_user, get_current_user_media, get_user_photo
from app.dependencies import (
get_current_user, get_current_user_media, get_user_photo,
get_user_or_shared_heap, get_user_or_shared_folder,
can_access_photo_via_share,
)
from app.config import settings
router = APIRouter()
@@ -56,9 +60,33 @@ async def list_photos(
):
"""List photos with filters and pagination"""
# Determine if this request is for shared content. When viewing a
# shared heap or folder the user_id filter is replaced by the
# heap/folder join that already encodes authorization.
is_shared_context = False
if heap_id:
try:
_heap, _perm = await get_user_or_shared_heap(heap_id, current_user, db)
if _perm != "owner":
is_shared_context = True
except Exception:
raise HTTPException(status_code=404, detail="Heap not found")
if folder_id and not is_shared_context:
try:
_folder, _perm = await get_user_or_shared_folder(folder_id, current_user, db)
if _perm != "owner":
is_shared_context = True
except Exception:
raise HTTPException(status_code=404, detail="Folder not found")
# Build query — eager-load tags so the response can include them
# without an N+1 round-trip per photo. Scoped to the current user.
query = select(Photo).options(selectinload(Photo.tags)).where(Photo.user_id == current_user.id)
# without an N+1 round-trip per photo. Scoped to the current user
# unless we're in a shared context (scoped by heap/folder instead).
query = select(Photo).options(selectinload(Photo.tags))
if not is_shared_context:
query = query.where(Photo.user_id == current_user.id)
# Apply filters
filters = []
@@ -299,6 +327,17 @@ async def list_photos(
# Convert to response, attaching tags inline so the frontend can group
# client-side without a second round-trip.
# In shared context, resolve owner usernames for photos from other users.
owner_cache: dict[str, str] = {} # user_id → username
if is_shared_context:
other_user_ids = {p.user_id for p in photos if p.user_id != current_user.id}
if other_user_ids:
from app.models.user import User as UserModel
user_result = await db.execute(
select(UserModel.id, UserModel.username).where(UserModel.id.in_(other_user_ids))
)
owner_cache = {uid: uname for uid, uname in user_result.all()}
photo_dicts = []
for photo in photos:
d = PhotoResponse.from_orm(photo).dict()
@@ -306,6 +345,8 @@ async def list_photos(
{"id": t.id, "name": t.name, "color": t.color}
for t in (photo.tags or [])
]
if is_shared_context and photo.user_id != current_user.id:
d["owner_username"] = owner_cache.get(photo.user_id)
photo_dicts.append(d)
response = {
@@ -500,6 +541,33 @@ async def remove_photo_tag(
await db.commit()
return None
async def _get_photo_with_share_fallback(
photo_id: str, user: User, db: AsyncSession,
) -> Photo:
"""Fetch a photo the user owns, or one they can access via a share.
The fast path (owned photo) does a single indexed query. The share
fallback only runs when the first query returns nothing — this
happens only for shared photos, not during normal browsing.
"""
# Fast path — owned photo (single indexed query, no extra joins).
result = await db.execute(
select(Photo).where(Photo.id == photo_id, Photo.user_id == user.id)
)
photo = result.scalar_one_or_none()
if photo:
return photo
# Slow path — check share access (only for shared photos).
if await can_access_photo_via_share(photo_id, user, db):
result = await db.execute(
select(Photo).where(Photo.id == photo_id)
)
photo = result.scalar_one_or_none()
if photo:
return photo
raise HTTPException(status_code=404, detail="Photo not found")
@router.get("/{photo_id}/thumb/{size}")
async def get_thumbnail(
photo_id: str,
@@ -512,7 +580,7 @@ async def get_thumbnail(
if size not in ['small', 'medium', 'large']:
raise HTTPException(status_code=400, detail="Invalid thumbnail size")
photo = await get_user_photo(photo_id, current_user, db)
photo = await _get_photo_with_share_fallback(photo_id, current_user, db)
# Check if thumbnail exists, generate if not.
# User-prefixed path for isolation.
@@ -596,7 +664,7 @@ async def get_original(
current_user: User = Depends(get_current_user_media),
):
"""Serve original file (download for RAW, inline for web-safe formats)"""
photo = await get_user_photo(photo_id, current_user, db)
photo = await _get_photo_with_share_fallback(photo_id, current_user, db)
if not os.path.exists(photo.filepath):
raise HTTPException(status_code=404, detail="File not found")
@@ -699,7 +767,7 @@ async def get_proxy(
current_user: User = Depends(get_current_user_media),
):
"""Serve a full-resolution WebP proxy for non-web-safe formats."""
photo = await get_user_photo(photo_id, current_user, db)
photo = await _get_photo_with_share_fallback(photo_id, current_user, db)
if not os.path.exists(photo.filepath):
raise HTTPException(status_code=404, detail="File not found")

View File

@@ -0,0 +1,364 @@
"""
Sharing API router — manage cross-user access to heaps and folders.
"""
import logging
from typing import Literal, Optional
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy import select, func
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.heaps import Heap, heap_photos
from app.models.folders import Folder, SourceRoot
from app.models.photos import Photo
from app.models.sharing import HeapShare, FolderShare
from app.models.user import User
from app.dependencies import (
get_current_user,
get_user_heap,
get_user_folder,
resolve_username,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/sharing", tags=["sharing"])
# ── Schemas ──────────────────────────────────────────────────────────────
class ShareCreate(BaseModel):
username: str
permission: Literal["read", "write"] = "read"
class ShareResponse(BaseModel):
id: str
shared_with_id: str
shared_with_username: str
permission: str
created_at: str
class SharedHeapResponse(BaseModel):
id: str
name: str
owner_username: str
permission: str
photo_count: int
class SharedFolderResponse(BaseModel):
id: str
name: str
folder_type: str
owner_username: str
permission: str
photo_count: int
# ── Heap sharing ─────────────────────────────────────────────────────────
@router.get("/heaps/shared-with-me")
async def list_shared_heaps(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all heaps that have been shared with the current user."""
result = await db.execute(
select(HeapShare, Heap, User)
.join(Heap, HeapShare.heap_id == Heap.id)
.join(User, HeapShare.owner_id == User.id)
.where(HeapShare.shared_with_id == current_user.id)
)
rows = result.all()
items = []
for share, heap, owner in rows:
# Count photos in this heap.
count_result = await db.execute(
select(func.count()).select_from(heap_photos).where(
heap_photos.c.heap_id == heap.id
)
)
count = count_result.scalar() or 0
items.append(SharedHeapResponse(
id=heap.id,
name=heap.name,
owner_username=owner.username,
permission=share.permission,
photo_count=count,
))
return items
@router.get("/heaps/{heap_id}")
async def list_heap_shares(
heap_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all shares for a heap (owner only)."""
heap = await get_user_heap(heap_id, current_user, db)
result = await db.execute(
select(HeapShare, User)
.join(User, HeapShare.shared_with_id == User.id)
.where(HeapShare.heap_id == heap.id)
)
return [
ShareResponse(
id=share.id,
shared_with_id=user.id,
shared_with_username=user.username,
permission=share.permission,
created_at=share.created_at.isoformat() if share.created_at else "",
)
for share, user in result.all()
]
@router.post("/heaps/{heap_id}", status_code=201)
async def share_heap(
heap_id: str,
body: ShareCreate,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Share a heap with another user (owner only)."""
heap = await get_user_heap(heap_id, current_user, db)
target_user = await resolve_username(body.username, db)
if target_user.id == current_user.id:
raise HTTPException(status_code=400, detail="Cannot share with yourself")
# Check for existing share.
existing = await db.execute(
select(HeapShare).where(
HeapShare.heap_id == heap.id,
HeapShare.shared_with_id == target_user.id,
)
)
if existing.scalar_one_or_none():
raise HTTPException(status_code=409, detail="Already shared with this user")
share = HeapShare(
heap_id=heap.id,
owner_id=current_user.id,
shared_with_id=target_user.id,
permission=body.permission,
)
db.add(share)
await db.commit()
logger.info("Heap %s shared with %s (%s)", heap.name, target_user.username, body.permission)
return {"status": "shared", "share_id": share.id}
@router.delete("/heaps/{heap_id}/{share_id}", status_code=204)
async def revoke_heap_share(
heap_id: str,
share_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Revoke a heap share. The owner can revoke any share; a recipient
can revoke their own share (i.e. leave)."""
result = await db.execute(
select(HeapShare).where(HeapShare.id == share_id, HeapShare.heap_id == heap_id)
)
share = result.scalar_one_or_none()
if share is None:
raise HTTPException(status_code=404, detail="Share not found")
# Must be the owner or the recipient themselves.
if share.owner_id != current_user.id and share.shared_with_id != current_user.id:
raise HTTPException(status_code=403, detail="Not authorized")
await db.delete(share)
await db.commit()
# ── Folder sharing ───────────────────────────────────────────────────────
@router.get("/folders/shared-with-me")
async def list_shared_folders(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all folders/source roots shared with the current user."""
result = await db.execute(
select(FolderShare, User)
.join(User, FolderShare.owner_id == User.id)
.where(FolderShare.shared_with_id == current_user.id)
)
rows = result.all()
items = []
for share, owner in rows:
# Resolve the folder/source root name and photo count.
if share.folder_type == "source_root":
sr_result = await db.execute(
select(SourceRoot).where(SourceRoot.id == share.folder_id)
)
entity = sr_result.scalar_one_or_none()
if not entity:
continue
name = entity.name
# Count all photos under this source root's folders.
count_result = await db.execute(
select(func.count()).select_from(Photo).where(
Photo.folder_id.in_(
select(Folder.id).where(Folder.source_root_id == entity.id)
),
Photo.is_discarded.is_(False),
)
)
else:
folder_result = await db.execute(
select(Folder).where(Folder.id == share.folder_id)
)
entity = folder_result.scalar_one_or_none()
if not entity:
continue
name = entity.name
import os
target_path = os.path.normpath(entity.path).rstrip(os.sep)
count_result = await db.execute(
select(func.count()).select_from(Photo).where(
Photo.folder_id.in_(
select(Folder.id).where(
(Folder.path == target_path)
| (Folder.path.like(target_path + os.sep + "%"))
)
),
Photo.is_discarded.is_(False),
)
)
count = count_result.scalar() or 0
items.append(SharedFolderResponse(
id=share.folder_id,
name=name,
folder_type=share.folder_type,
owner_username=owner.username,
permission=share.permission,
photo_count=count,
))
return items
@router.get("/folders/{folder_id}")
async def list_folder_shares(
folder_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""List all shares for a folder (owner only)."""
# Verify ownership — try folder then source root.
owned = False
result = await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == current_user.id)
)
if result.scalar_one_or_none():
owned = True
else:
result = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == current_user.id)
)
if result.scalar_one_or_none():
owned = True
if not owned:
raise HTTPException(status_code=404, detail="Folder not found")
result = await db.execute(
select(FolderShare, User)
.join(User, FolderShare.shared_with_id == User.id)
.where(FolderShare.folder_id == folder_id)
)
return [
ShareResponse(
id=share.id,
shared_with_id=user.id,
shared_with_username=user.username,
permission=share.permission,
created_at=share.created_at.isoformat() if share.created_at else "",
)
for share, user in result.all()
]
@router.post("/folders/{folder_id}", status_code=201)
async def share_folder(
folder_id: str,
body: ShareCreate,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Share a folder or source root with another user (owner only)."""
# Determine folder_type and verify ownership.
folder_type = "folder"
result = await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == current_user.id)
)
entity = result.scalar_one_or_none()
if entity is None:
result = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == current_user.id)
)
entity = result.scalar_one_or_none()
if entity is None:
raise HTTPException(status_code=404, detail="Folder not found")
folder_type = "source_root"
target_user = await resolve_username(body.username, db)
if target_user.id == current_user.id:
raise HTTPException(status_code=400, detail="Cannot share with yourself")
existing = await db.execute(
select(FolderShare).where(
FolderShare.folder_id == folder_id,
FolderShare.shared_with_id == target_user.id,
)
)
if existing.scalar_one_or_none():
raise HTTPException(status_code=409, detail="Already shared with this user")
share = FolderShare(
folder_id=folder_id,
folder_type=folder_type,
owner_id=current_user.id,
shared_with_id=target_user.id,
permission=body.permission,
)
db.add(share)
await db.commit()
logger.info("Folder %s shared with %s (%s)", entity.name, target_user.username, body.permission)
return {"status": "shared", "share_id": share.id}
@router.delete("/folders/{folder_id}/{share_id}", status_code=204)
async def revoke_folder_share(
folder_id: str,
share_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Revoke a folder share (owner or self-remove)."""
result = await db.execute(
select(FolderShare).where(FolderShare.id == share_id, FolderShare.folder_id == folder_id)
)
share = result.scalar_one_or_none()
if share is None:
raise HTTPException(status_code=404, detail="Share not found")
if share.owner_id != current_user.id and share.shared_with_id != current_user.id:
raise HTTPException(status_code=403, detail="Not authorized")
await db.delete(share)
await db.commit()

View File

@@ -0,0 +1,303 @@
"""
Upload router — lets users drop files (or whole folders) from their
desktop into a destination Folder, preserving any sub-folder structure
they bring with them.
Each POST handles one file. The frontend fans out many parallel requests
per drop, giving it per-file progress without the server having to
invent a chunking protocol. For folder uploads, the browser passes
`webkitRelativePath` under the `relative_path` field; any leading
sub-directories there are materialised on disk (and as Folder rows)
under the destination.
Uploaded files are placed under the destination folder on the owner's
media mount, indexed immediately (Photo row created), and queued for
the same thumb + metadata pipeline that the scanner uses. An optional
`heap_id` also drops them into a heap in the same request.
"""
import hashlib
import logging
import os
from pathlib import Path
from datetime import datetime
from typing import Optional
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
from sqlalchemy import insert, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.dependencies import get_current_user
from app.models import Folder, Heap, Photo, SourceRoot
from app.models.heaps import heap_photos
from app.models.user import User
from app.services.date_guess import has_date_warning
from app.tasks.scan import SUPPORTED_EXTENSIONS, get_media_type
from app.tasks.thumbs import generate_thumbnails
from app.services.metadata import extract_metadata
logger = logging.getLogger(__name__)
router = APIRouter()
MAX_UPLOAD_BYTES = 500 * 1024 * 1024 # 500 MB per file cap.
def _validate_segment(segment: str) -> str:
"""Reject path segments that would escape the destination directory."""
segment = segment.strip()
if not segment or segment in ('.', '..') or '/' in segment or '\\' in segment:
raise HTTPException(status_code=400, detail=f"Invalid path segment: {segment!r}")
return segment
def _sanitize_relative_path(rel: Optional[str]) -> list[str]:
"""Split `relative_path` into safe segments (dirs + filename).
Empty or missing → []. Any absolute path, backslash, or `..` segment
raises 400 — we never want an upload to escape the destination.
"""
if not rel:
return []
# Normalise backslashes to forward slashes; browsers on Windows send
# webkitRelativePath with forward slashes anyway, but defend in depth.
rel = rel.replace('\\', '/').strip('/')
if not rel:
return []
segs = [_validate_segment(s) for s in rel.split('/') if s]
return segs
async def _resolve_destination(
folder_id: str,
user: User,
db: AsyncSession,
) -> Folder:
"""Resolve `folder_id` to a concrete Folder row the user owns.
Accepts both Folder ids and SourceRoot ids (for source roots, we
return the Folder row at the mount path — the scanner creates one
for every source root it walks). Raises 404 if neither matches.
"""
folder = (await db.execute(
select(Folder).where(Folder.id == folder_id, Folder.user_id == user.id)
)).scalar_one_or_none()
if folder is not None:
return folder
sr = (await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id, SourceRoot.user_id == user.id)
)).scalar_one_or_none()
if sr is None:
raise HTTPException(status_code=404, detail="Destination folder not found")
root_folder = (await db.execute(
select(Folder).where(
Folder.source_root_id == sr.id,
Folder.user_id == user.id,
Folder.path == os.path.normpath(sr.path),
)
)).scalar_one_or_none()
if root_folder is None:
# First-time source root with no walk yet — create the row now so
# uploads work even before the initial scan has run.
root_folder = Folder(
name=sr.name or os.path.basename(sr.path),
path=os.path.normpath(sr.path),
source_root_id=sr.id,
user_id=user.id,
)
os.makedirs(root_folder.path, exist_ok=True)
db.add(root_folder)
await db.flush()
return root_folder
async def _ensure_subfolder(
parent: Folder,
name: str,
user: User,
db: AsyncSession,
) -> Folder:
"""Return (or create) a Folder row named `name` under `parent`.
Also mkdirs the directory on disk. Idempotent — safe to call for a
path segment that already exists as a Folder row or directory.
"""
child_path = os.path.normpath(os.path.join(parent.path, name))
existing = (await db.execute(
select(Folder).where(
Folder.path == child_path,
Folder.user_id == user.id,
)
)).scalar_one_or_none()
if existing is not None:
os.makedirs(child_path, exist_ok=True)
return existing
os.makedirs(child_path, exist_ok=True)
child = Folder(
name=name,
path=child_path,
parent_id=parent.id,
source_root_id=parent.source_root_id,
user_id=user.id,
is_hidden=parent.is_hidden,
)
db.add(child)
await db.flush()
return child
def _unique_path(target_dir: str, filename: str) -> tuple[str, str]:
"""Return a (filepath, filename) that doesn't collide with an
existing file on disk. Suffixes " (2)", " (3)", ... until a free
slot is found. Prevents upload-over-existing and keeps the user's
original file intact.
"""
base, ext = os.path.splitext(filename)
candidate = os.path.join(target_dir, filename)
n = 2
while os.path.exists(candidate):
new_name = f"{base} ({n}){ext}"
candidate = os.path.join(target_dir, new_name)
n += 1
return candidate, os.path.basename(candidate)
@router.post("")
async def upload_file(
file: UploadFile = File(...),
destination_folder_id: str = Form(...),
relative_path: Optional[str] = Form(None),
heap_id: Optional[str] = Form(None),
db: AsyncSession = Depends(get_db),
current_user: User = Depends(get_current_user),
):
"""Upload a single file into a destination folder (and optionally a
heap). For folder uploads, `relative_path` carries the sub-folder
chain from the browser's `webkitRelativePath`, and we materialise
it under the destination on disk + as Folder rows.
Returns the created photo's id on success. 4xx on unsupported file
type, bad path, missing destination, or too-large file.
"""
# --- validate inputs -------------------------------------------------
raw_name = file.filename or ''
if not raw_name:
raise HTTPException(status_code=400, detail="Missing filename")
# Prefer the leaf of relative_path when present (it contains the
# original filename as the browser saw it inside the picked folder).
segs = _sanitize_relative_path(relative_path)
if segs:
leaf = segs[-1]
subdirs = segs[:-1]
else:
leaf = _validate_segment(os.path.basename(raw_name))
subdirs = []
ext = Path(leaf).suffix.lower()
if ext not in SUPPORTED_EXTENSIONS:
raise HTTPException(
status_code=400,
detail=f"Unsupported file type: {ext or '(none)'}",
)
dest_folder = await _resolve_destination(destination_folder_id, current_user, db)
target_folder = dest_folder
for seg in subdirs:
target_folder = await _ensure_subfolder(target_folder, seg, current_user, db)
target_dir = target_folder.path
os.makedirs(target_dir, exist_ok=True)
filepath, final_name = _unique_path(target_dir, leaf)
# --- stream to disk, hash as we go ----------------------------------
hasher = hashlib.sha256()
total = 0
try:
with open(filepath, 'wb') as out:
while True:
chunk = await file.read(1024 * 1024)
if not chunk:
break
total += len(chunk)
if total > MAX_UPLOAD_BYTES:
out.close()
os.unlink(filepath)
raise HTTPException(
status_code=413,
detail=f"File exceeds {MAX_UPLOAD_BYTES // (1024*1024)}MB limit",
)
hasher.update(chunk)
out.write(chunk)
except HTTPException:
raise
except Exception as e:
logger.error(f"Upload write failed for {filepath}: {e}")
if os.path.exists(filepath):
try:
os.unlink(filepath)
except OSError:
pass
raise HTTPException(status_code=500, detail=f"Upload failed: {e}")
file_hash = hasher.hexdigest()
# --- validate heap before committing the DB row ---------------------
if heap_id:
heap = (await db.execute(
select(Heap).where(Heap.id == heap_id, Heap.user_id == current_user.id)
)).scalar_one_or_none()
if heap is None:
# Destination heap vanished — still keep the file + photo row,
# but tell the caller so the UI can surface the mismatch.
heap_id = None
# --- create Photo row ------------------------------------------------
mtime_dt = datetime.fromtimestamp(os.stat(filepath).st_mtime)
photo = Photo(
filepath=filepath,
filename=final_name,
folder_id=target_folder.id,
user_id=current_user.id,
file_hash=file_hash,
media_type=get_media_type(filepath),
original_format=Path(filepath).suffix.upper()[1:],
file_size=total,
taken_at=mtime_dt,
taken_at_source='filesystem',
has_date_warning=has_date_warning(filepath, mtime_dt),
is_hidden=bool(target_folder.is_hidden),
processing_status='pending',
)
db.add(photo)
await db.flush()
if heap_id:
await db.execute(
insert(heap_photos),
[{"heap_id": heap_id, "photo_id": photo.id}],
)
await db.commit()
# Queue the same background work the scanner does so thumbnails +
# EXIF show up without the user having to trigger a rescan.
try:
generate_thumbnails.delay(photo.id)
extract_metadata.delay(photo.id)
except Exception as e:
logger.warning(f"Failed to queue post-upload tasks for {photo.id}: {e}")
return {
"photo_id": photo.id,
"filename": final_name,
"folder_id": target_folder.id,
"folder_path": target_folder.path,
"heap_id": heap_id,
}

View File

@@ -41,6 +41,7 @@ class PhotoResponse(PhotoBase):
is_duplicate: bool = False
has_date_warning: bool = False
live_photo_video_id: Optional[str] = None
owner_username: Optional[str] = None
# tags: List[Dict[str, Any]] = [] # TODO: Enable when using eager loading
class Config:

View File

@@ -221,6 +221,13 @@ async def incremental_regroup(
from datetime import timedelta
since = datetime.now(timezone.utc) - timedelta(hours=1)
# Photo.added_at is stored as TIMESTAMP WITHOUT TIME ZONE, so
# asyncpg rejects aware datetimes with "can't subtract offset-naive
# and offset-aware". Normalise: if `since` has a tzinfo, convert
# it to UTC and drop the tzinfo so the bind parameter is naive.
if since.tzinfo is not None:
since = since.astimezone(timezone.utc).replace(tzinfo=None)
# Get newly added photos (the "new" set).
new_rows = (
await session.execute(
@@ -349,6 +356,14 @@ async def _clip_neighbor_scan(
for photo_id, vector in target_embeddings:
# pgvector cosine distance: <=> operator
# Find top 20 nearest neighbors within threshold.
# Serialize the vector as "[a,b,c,...]" — pgvector's text
# format uses commas; numpy's default str() joins with spaces
# which Postgres rejects with "invalid input syntax for vector".
if hasattr(vector, 'tolist'):
vec_seq = vector.tolist()
else:
vec_seq = list(vector)
vec_text = '[' + ','.join(f'{float(x):.8f}' for x in vec_seq) + ']'
result = await session.execute(
text("""
SELECT e.photo_id, (e.vector <=> :vec) AS distance
@@ -356,14 +371,14 @@ async def _clip_neighbor_scan(
JOIN photos p ON p.id = e.photo_id
WHERE e.model = :model
AND e.photo_id != :pid
AND p.is_discarded = false
AND p.is_trashed = false
AND p.is_hidden = false
AND (e.vector <=> :vec) < :threshold
ORDER BY e.vector <=> :vec
LIMIT 20
"""),
{
'vec': str(vector),
'vec': vec_text,
'pid': photo_id,
'model': embedder_model,
'threshold': threshold,

View File

@@ -0,0 +1,301 @@
"""
Runtime feature flags for expensive pipeline stages.
The YAML config (``mulita.yml``) ships reasonable defaults. Admins can
toggle these at runtime from the Settings → AI Features tab without
rebuilding the image or editing the bind-mounted YAML; the overrides
live in Redis so both the FastAPI backend and the Celery workers see
the same value within ~1s of the write.
The key namespace is:
mulita:flags:<name> → "true" | "false"
An unset key means "fall back to the YAML default" — so an admin who
has never touched the tab sees exactly the config-file behaviour.
Only bool flags live here. Thresholds, batch sizes, model names etc.
stay in the YAML file because flipping them safely requires restarting
the vision workers (model reload, ONNX session re-init); that's not
something a single admin click should do.
"""
from __future__ import annotations
import logging
from typing import Optional
import redis
from app.config import settings
logger = logging.getLogger(__name__)
# Feature identifiers. The public name is what the admin UI sends; the
# ``yaml_default`` getter returns the value the YAML would have set.
# Keep these in sync with the VisionSettings fields in ``config.py``.
FLAG_VISION_ENABLED = 'vision.enabled'
FLAG_OCR_ENABLED = 'vision.ocr.enabled'
FLAG_DETECTOR_ENABLED = 'vision.detector.enabled'
FLAG_FACES_ENABLED = 'vision.faces.enabled'
FLAG_CLASSIFIER_ENABLED = 'vision.classifier.enabled'
ALL_FLAGS = (
FLAG_VISION_ENABLED,
FLAG_OCR_ENABLED,
FLAG_DETECTOR_ENABLED,
FLAG_FACES_ENABLED,
FLAG_CLASSIFIER_ENABLED,
)
_REDIS: Optional[redis.Redis] = None
def _redis() -> Optional[redis.Redis]:
"""Lazy Redis client. Returns None if the broker is unreachable so
callers can fall back to YAML defaults instead of crashing."""
global _REDIS
if _REDIS is None:
try:
_REDIS = redis.Redis.from_url(
settings.celery_broker_url, decode_responses=True
)
_REDIS.ping()
except Exception as e:
logger.warning(f"feature_flags: Redis unavailable, using YAML defaults ({e})")
_REDIS = None
return _REDIS
def _yaml_default(name: str) -> bool:
"""Return the YAML-configured default for a flag. Used when Redis
has no value set (fresh install or admin never touched the tab)."""
v = settings.vision
if name == FLAG_VISION_ENABLED:
return bool(v.enabled)
if name == FLAG_OCR_ENABLED:
return bool(v.ocr.enabled)
if name == FLAG_DETECTOR_ENABLED:
return bool(v.detector.enabled)
if name == FLAG_FACES_ENABLED:
return bool(v.faces.enabled)
if name == FLAG_CLASSIFIER_ENABLED:
return bool(v.classifier.enabled)
raise ValueError(f"Unknown feature flag: {name!r}")
def _redis_key(name: str) -> str:
return f"mulita:flags:{name}"
def is_enabled(name: str) -> bool:
"""Return True if feature ``name`` is currently enabled.
Order of precedence:
1. Redis override (set by PATCH /admin/feature-flags)
2. YAML default
Reads are cheap (~ms) and we intentionally do NOT add a local
process cache — the whole point of runtime flags is that a toggle
takes effect on the next task without a worker restart.
"""
r = _redis()
if r is not None:
try:
raw = r.get(_redis_key(name))
if raw is not None:
return raw.lower() == 'true'
except Exception as e:
logger.warning(f"feature_flags: Redis read failed for {name} ({e})")
return _yaml_default(name)
def set_flag(name: str, value: bool) -> None:
"""Persist a flag override to Redis. No-op if Redis is unreachable
(we don't silently pretend to have written; raise so the admin
request returns a 500 instead of misleading success)."""
if name not in ALL_FLAGS:
raise ValueError(f"Unknown feature flag: {name!r}")
r = _redis()
if r is None:
raise RuntimeError("Redis unavailable; cannot update feature flags")
r.set(_redis_key(name), 'true' if value else 'false')
_apply_worker_side_effects(name)
def reset_flag(name: str) -> None:
"""Delete the Redis override so the flag falls back to its YAML
default. Useful if an admin wants a clean slate without guessing
what the config defaults are."""
if name not in ALL_FLAGS:
raise ValueError(f"Unknown feature flag: {name!r}")
r = _redis()
if r is None:
raise RuntimeError("Redis unavailable; cannot reset feature flags")
r.delete(_redis_key(name))
_apply_worker_side_effects(name)
# ---------------------------------------------------------------------------
# Worker-level side effects: when the admin flips a flag we don't just want
# gating at task-start (which still executes the message, it just returns
# 'skipped'). We also want queued work gone and the vision worker genuinely
# idle when the master switch is off.
# ---------------------------------------------------------------------------
_VISION_QUEUE = 'vision'
# Flag → celery task name(s) whose queued messages should be dropped when
# the flag goes off. Keeps the queue from replaying yesterday's work the
# moment someone re-enables the stage.
_TASKS_BY_FLAG: dict[str, tuple[str, ...]] = {
FLAG_VISION_ENABLED: (
'embed_photo', 'ocr_photo', 'detect_objects', 'extract_faces',
'classify_content', 'vision_fanout', 'recluster_faces',
),
FLAG_OCR_ENABLED: ('ocr_photo',),
FLAG_DETECTOR_ENABLED: ('detect_objects',),
FLAG_FACES_ENABLED: ('extract_faces', 'recluster_faces'),
FLAG_CLASSIFIER_ENABLED: ('classify_content',),
}
def _apply_worker_side_effects(name: str) -> None:
"""Bring the live workers in line with the new flag value.
For the master ``vision.enabled`` flag we go beyond task gating and
actually stop consumption from the ``vision`` queue — flipping it
off puts the vision worker to sleep (no CPU, no model memory
churn) until it's flipped back on. For per-feature flags, the
running tasks already skip via ``is_enabled``; we just purge any
messages already sitting in the queue so the admin doesn't pay for
a backlog on re-enable.
All operations are best-effort — if control messaging or a Redis
op fails, we log and return; the flag state itself is already
persisted so the gating path continues to work.
"""
try:
# Lazy import: avoids a circular dependency between the services
# module (imported from tasks.vision) and the celery app config.
from app.tasks.celery import celery_app
except Exception as e:
logger.warning(f"feature_flags: celery app unavailable for side effects ({e})")
return
try:
if name == FLAG_VISION_ENABLED:
if is_enabled(FLAG_VISION_ENABLED):
# Re-attach the vision consumer so workers pick up tasks
# again. broadcast=True ensures every running worker
# receives the command.
celery_app.control.add_consumer(_VISION_QUEUE, reply=False)
logger.info("feature_flags: vision re-enabled; consumer added")
else:
celery_app.control.cancel_consumer(_VISION_QUEUE, reply=False)
_purge_queue(_VISION_QUEUE)
logger.info(
"feature_flags: vision disabled; consumer cancelled "
"and queue purged"
)
return
# Per-feature flag going off → drop pending tasks of its types.
if not is_enabled(name):
targets = _TASKS_BY_FLAG.get(name, ())
if targets:
removed = _purge_queue_by_task_names(_VISION_QUEUE, targets)
logger.info(
f"feature_flags: {name} disabled; removed {removed} "
f"pending messages from {_VISION_QUEUE}"
)
except Exception as e:
logger.warning(f"feature_flags: worker side effects failed for {name}: {e}")
def _purge_queue(queue: str) -> int:
"""Drop every pending message from ``queue``. Returns the count
deleted. Celery's control.purge() purges the default queue only,
so we delete the Redis key directly (the broker's queue list)."""
r = _redis()
if r is None:
return 0
try:
removed = r.delete(queue)
return int(removed or 0)
except Exception as e:
logger.warning(f"feature_flags: purge {queue} failed: {e}")
return 0
def _purge_queue_by_task_names(queue: str, task_names: tuple[str, ...]) -> int:
"""Walk ``queue`` and drop any message whose Celery task name is in
``task_names``. Other messages are preserved (pushed back in order)
so we don't flush embed tasks when the admin disabled only OCR.
Celery stores each message as a JSON blob in a Redis list; the
task name lives at ``headers.task``.
"""
import json
r = _redis()
if r is None:
return 0
try:
# Snapshot the queue, then rebuild it without the filtered names.
# Done inside a Redis transaction so a concurrent enqueue doesn't
# race with us (worst case it gets re-delivered after we release,
# which is the normal enqueue path anyway).
pipe = r.pipeline()
pipe.lrange(queue, 0, -1)
pipe.delete(queue)
raw_items, _ = pipe.execute()
kept: list[bytes | str] = []
removed = 0
for raw in raw_items or []:
try:
# Messages can be bytes or str depending on decode_responses.
payload = raw.decode() if isinstance(raw, bytes) else raw
msg = json.loads(payload)
task = (
msg.get('headers', {}).get('task')
or msg.get('task')
)
if task in task_names:
removed += 1
continue
except Exception:
# Unparseable message — keep it, better to leak than
# to silently drop a message we can't identify.
pass
kept.append(raw)
if kept:
r.rpush(queue, *kept)
return removed
except Exception as e:
logger.warning(f"feature_flags: selective purge failed on {queue}: {e}")
return 0
def snapshot() -> dict[str, dict[str, object]]:
"""Return every flag's current effective value, YAML default, and
whether it's overridden. Powers the admin UI tab.
"""
r = _redis()
out: dict[str, dict[str, object]] = {}
for name in ALL_FLAGS:
default = _yaml_default(name)
override = None
if r is not None:
try:
raw = r.get(_redis_key(name))
if raw is not None:
override = raw.lower() == 'true'
except Exception:
pass
out[name] = {
'effective': override if override is not None else default,
'default': default,
'overridden': override is not None,
}
return out

View File

@@ -209,11 +209,14 @@ async def _extract_metadata_async(photo_id: str):
cmd,
capture_output=True,
text=True,
timeout=30
timeout=30,
stdin=subprocess.DEVNULL,
)
if result.returncode != 0:
logger.error(f"ExifTool error: {result.stderr}")
photo.processing_error = f"ExifTool: {result.stderr[:500]}"
await session.commit()
return {'status': 'error', 'message': result.stderr}
# Parse JSON output
@@ -263,9 +266,6 @@ async def _extract_metadata_async(photo_id: str):
# Extract and store key metadata for search
key_metadata = extract_key_metadata(exif_data)
# Update FTS table (would be done via trigger in production)
# For now, we'll store it in a comment
await session.commit()
logger.info(f"Metadata extracted for photo {photo_id}")
@@ -277,9 +277,13 @@ async def _extract_metadata_async(photo_id: str):
except subprocess.TimeoutExpired:
logger.error(f"ExifTool timeout for {photo.filepath}")
photo.processing_error = 'ExifTool timeout'
await session.commit()
return {'status': 'error', 'message': 'ExifTool timeout'}
except json.JSONDecodeError as e:
logger.error(f"Failed to parse ExifTool output: {e}")
photo.processing_error = f"Invalid ExifTool output: {e}"
await session.commit()
return {'status': 'error', 'message': 'Invalid ExifTool output'}
except Exception as e:

View File

@@ -73,21 +73,31 @@ def bootstrap(models_dir: str | None = None):
for rel_path, desc in missing:
logger.warning(" %s%s", base / rel_path, desc)
try:
from app.services.vision import export_models
export_models.export_openclip(base)
export_models.export_siglip2(base)
export_models.export_yolov8n(base)
missing_paths = {rel for rel, _ in missing}
# Only run the export functions whose outputs are actually missing.
# Each function is mapped to the file(s) it produces.
export_map = [
(export_models.export_openclip, ["embed/visual.onnx", "embed/textual.onnx"]),
(export_models.export_siglip2, ["embed_siglip2/visual.onnx", "embed_siglip2/textual.onnx"]),
(export_models.export_yolov8n, ["detect/yolov8n.onnx"]),
]
for export_fn, outputs in export_map:
if not any(o in missing_paths for o in outputs):
continue
try:
export_fn(base)
except Exception as e:
logger.error(
"Automatic export failed: %s. "
"Export step %s failed: %s. "
"Run `python -m app.services.vision.export_models "
"--models-dir %s` manually before starting the worker.",
"--models-dir %s` manually to retry.",
export_fn.__name__,
e,
base,
)
return
# Re-check what's still missing after the export pass.
still_missing = [
@@ -103,6 +113,16 @@ def bootstrap(models_dir: str | None = None):
else:
logger.info("All model files present in %s", base)
# Signal readiness via Redis so the scan pipeline knows the vision
# worker can accept tasks.
try:
import redis as _redis
r = _redis.from_url(settings.redis_url)
r.set("mulita:vision:ready", "1")
logger.info("Set mulita:vision:ready in Redis")
except Exception as e:
logger.warning("Could not set vision readiness flag in Redis: %s", e)
if __name__ == "__main__":
logging.basicConfig(level=logging.INFO)

View File

@@ -1,15 +1,26 @@
"""
Celery configuration and app initialization
"""
import logging
import os
from celery import Celery
from celery.signals import worker_process_init
from app.config import settings
logger = logging.getLogger(__name__)
# Create Celery app
celery_app = Celery(
'mulita',
broker=settings.celery_broker_url,
backend=settings.celery_result_backend,
include=['app.tasks.scan', 'app.tasks.thumbs', 'app.tasks.vision']
include=[
'app.tasks.scan',
'app.tasks.thumbs',
'app.tasks.vision',
'app.services.metadata', # extract_metadata lives here
]
)
# Configure Celery
@@ -19,16 +30,34 @@ celery_app.conf.update(
result_serializer='json',
timezone='UTC',
enable_utc=True,
# Robust acknowledgment: keep message in broker until task succeeds.
task_acks_late=True,
task_reject_on_worker_lost=True,
# Global time limits — individual tasks can override via decorator.
task_soft_time_limit=300, # 5 min — raises SoftTimeLimitExceeded
task_time_limit=600, # 10 min — SIGKILL
# Explicit routes for every task name. Wildcard patterns don't match
# short names produced by @shared_task(name='...').
task_routes={
'app.tasks.thumbs.*': {'queue': 'high'},
'app.tasks.scan.*': {'queue': 'low'},
'app.tasks.vision.*': {'queue': 'vision'},
# Vision queue — GPU/CPU-bound inference
'embed_photo': {'queue': 'vision'},
'ocr_photo': {'queue': 'vision'},
'detect_objects': {'queue': 'vision'},
'extract_faces': {'queue': 'vision'},
'classify_content': {'queue': 'vision'},
'vision_fanout': {'queue': 'vision'},
'recluster_faces': {'queue': 'vision'},
# High-priority queue — thumbnails & duplicates
'generate_thumbnails': {'queue': 'high'},
'regenerate_all_thumbnails': {'queue': 'high'},
'backfill_phashes': {'queue': 'high'},
'regroup_duplicates': {'queue': 'high'},
'incremental_regroup_duplicates': {'queue': 'high'},
# Low-priority queue — scans
'scan_folder': {'queue': 'low'},
'scan_all_source_roots': {'queue': 'low'},
'backfill_gps': {'queue': 'low'},
# Dedicated watcher queue
'watch_folders': {'queue': 'watcher'},
},
task_default_queue='default',
@@ -37,3 +66,21 @@ celery_app.conf.update(
task_default_routing_key='default',
broker_connection_retry_on_startup=True,
)
@worker_process_init.connect
def _warmup_vision_models(**kwargs):
"""Pre-load vision models in the worker process so the first task
doesn't pay cold-start latency. Only runs on the vision queue."""
# The worker name contains the queue — only warm up vision workers.
worker_queues = os.environ.get("CELERY_QUEUES", "")
if "vision" not in worker_queues:
# Heuristic: check the celery command line for -Q vision
import sys
if "vision" not in " ".join(sys.argv):
return
try:
from app.services.vision.registry import registry
registry.warmup()
except Exception:
logger.exception("Vision model warmup failed")

View File

@@ -95,11 +95,13 @@ async def _scan_folder_async(folder_path: str, source_root_id: Optional[str], ta
r = _get_redis()
PROGRESS_TTL = 3600 # 1 hour — auto-expire if scan crashes
def progress_set(key: str, value) -> None:
if r is None:
return
try:
r.set(key, str(value))
r.set(key, str(value), ex=PROGRESS_TTL)
except Exception as e:
logger.debug(f"scan progress set failed: {e}")
@@ -467,12 +469,20 @@ async def _scan_all_source_roots_async():
except Exception as e:
logger.warning(f"Could not queue post-scan face recluster: {e}")
# Re-extract metadata for photos missing GPS coordinates.
# Runs on every startup so photos scanned before the GPS fix
# eventually get their coordinates populated.
try:
backfill_gps.apply_async(countdown=30)
except Exception as e:
logger.warning(f"Could not queue post-scan GPS backfill: {e}")
WATCHER_LOCK_KEY = "mulita:watch_folders:lock"
WATCHER_LOCK_TTL = 300 # 5 min — renewed every 60s
WATCHER_LOCK_TTL = 60 # 1 min — renewed every event batch via wall-clock check
@shared_task(name='watch_folders', bind=True)
@shared_task(name='watch_folders', bind=True, soft_time_limit=None, time_limit=None)
def watch_folders(self):
"""
Watch folders for changes using watchfiles. Long-running task that
@@ -527,16 +537,20 @@ def watch_folders(self):
return root_id
return None
renew_counter = 0
for changes in watch(*paths):
# Renew the Redis lock periodically so it doesn't expire
# while the watcher is idle between events.
renew_counter += 1
if renew_counter % 10 == 0:
import time
last_renew = time.monotonic()
for changes in watch(*paths, rust_timeout=30_000):
# Renew the Redis lock on a wall-clock schedule (every 30s)
# instead of every N events, so quiet directories don't let
# the lock expire. watchfiles' rust_timeout ensures we wake
# at least every 30s even with no FS events.
now = time.monotonic()
if now - last_renew >= 30:
try:
lock.extend(WATCHER_LOCK_TTL)
last_renew = now
except Exception:
pass
logger.warning("watch_folders: failed to renew Redis lock")
for change_type, filepath in changes:
filepath = str(filepath)
@@ -557,7 +571,7 @@ def watch_folders(self):
try:
lock.release()
except Exception:
pass
logger.warning("watch_folders: could not release Redis lock (may have expired)")
async def handle_file_deletion(filepath: str):
"""Handle deletion of a file from the filesystem"""

View File

@@ -102,17 +102,40 @@ def extract_raw_preview(filepath: str) -> Optional[Image.Image]:
return None
def process_heic_image(filepath: str) -> Image.Image:
"""Process HEIC/HEIF image formats"""
"""Process HEIC/HEIF image formats.
Tries pillow-heif first (fast, native). Falls back to ffmpeg for
files that libheif rejects — e.g. iPhone photos with too many
auxiliary image references (depth maps, gain maps).
"""
try:
# Use pillow-heif to open the image
img = Image.open(filepath)
# Convert to RGB if needed
if img.mode != 'RGB':
img = img.convert('RGB')
return img
except Exception as e:
logger.error(f"Error processing HEIC file {filepath}: {e}")
raise
logger.warning(f"pillow-heif failed for {filepath}: {e} — trying vips")
# vips fallback: handles tiled Apple HEIC files (bursts, HDR gain
# maps, depth maps) that pillow-heif/libheif rejects due to too many
# auxiliary image references.
import subprocess, tempfile
try:
with tempfile.NamedTemporaryFile(suffix='.png', delete=False) as tmp:
tmp_path = tmp.name
result = subprocess.run(
['vips', 'heifload', filepath, tmp_path],
capture_output=True, timeout=60, stdin=subprocess.DEVNULL,
)
if result.returncode == 0:
img = Image.open(tmp_path).convert('RGB')
os.unlink(tmp_path)
return img
logger.error(f"vips HEIC decode failed for {filepath}: {result.stderr.decode()[-200:]}")
os.unlink(tmp_path)
except Exception as e2:
logger.error(f"vips fallback failed for {filepath}: {e2}")
raise RuntimeError(f"Cannot decode HEIC: {filepath}")
def process_video_thumbnail(filepath: str) -> Image.Image:
"""Extract a still frame from a video file as a PIL Image."""
@@ -218,23 +241,28 @@ def auto_rotate_image(image: Image.Image) -> Image.Image:
if orientation in rotation_map:
image = image.rotate(rotation_map[orientation], expand=True)
except:
except (AttributeError, KeyError, TypeError):
pass # No orientation data available
return image
def generate_thumbnail(image: Image.Image, size: int, output_path: str):
"""Generate a thumbnail of the specified size"""
# Maintain aspect ratio
image.thumbnail((size, size), Image.Resampling.LANCZOS)
"""Generate a thumbnail of the specified size.
# Save as WebP with specified quality
image.save(
Works on a copy so the caller's image is never mutated — this is
critical because the thumbnail loop iterates multiple sizes and
in-place shrinking would degrade later (larger) sizes.
"""
img = image.copy()
img.thumbnail((size, size), Image.Resampling.LANCZOS)
img.save(
output_path,
'WEBP',
quality=settings.thumbnails.quality,
method=4 # Balance between speed and compression
)
img.close()
@shared_task(bind=True, name='generate_thumbnails')
def generate_thumbnails(self, photo_id: str):
@@ -285,6 +313,24 @@ async def _generate_thumbnails_async(photo_id: str, task):
logger.error(f"Unsupported media type: {photo.media_type}")
image = create_placeholder_thumbnail(photo.media_type)
# Fallback: some files wear a RAW/HEIC extension but are actually
# plain JPEGs — e.g. iPhones that write ProRAW-style .DNG for
# images where no RAW sensor data was captured, or re-exports
# that kept the original suffix. Pillow can open them directly,
# so before giving up, try reading the file as a standard image.
if not image and photo.media_type in ('raw', 'heic'):
try:
image = process_standard_image(photo.filepath)
if image is not None:
logger.info(
f"{photo.filepath}: {photo.media_type} decode failed "
f"but file opens as a standard image — using fallback"
)
except Exception as e:
logger.debug(
f"Standard-image fallback failed for {photo.filepath}: {e}"
)
if not image:
raise Exception("Failed to process image")
@@ -328,8 +374,8 @@ async def _generate_thumbnails_async(photo_id: str, task):
logger.info(f"Thumbnails generated for photo {photo_id}")
# Dispatch vision pipeline (embedding, OCR, detection, faces)
# after thumbs are ready so vision tasks have images to read.
# Dispatch vision pipeline only after thumbnails succeeded —
# vision tasks need the generated thumbnails to run inference.
try:
from app.tasks.vision import vision_fanout
vision_fanout.delay(photo_id)
@@ -465,7 +511,16 @@ async def _backfill_phashes_async():
}
@shared_task(name='regroup_duplicates')
@shared_task(
name='regroup_duplicates',
# Full regroup scales with O(N²) on phash plus one pgvector query per
# embedded photo. On a 16k-photo library that's comfortably past the
# default 5-minute soft limit — bump to 2h / 2h30m. (Passing None here
# does NOT disable limits; Celery falls back to the worker default
# of 300s/600s. An explicit number overrides.)
soft_time_limit=7200,
time_limit=9000,
)
def regroup_duplicates_task():
"""Full recompute of duplicate groups (pHash + CLIP similarity).
@@ -474,7 +529,15 @@ def regroup_duplicates_task():
return asyncio.run(regroup_duplicates())
@shared_task(name='incremental_regroup_duplicates')
@shared_task(
name='incremental_regroup_duplicates',
# O(new × N); still cheaper than a full regroup but can easily exceed
# the 5-minute default after a big batch import. Same caveat as
# regroup_duplicates above — None would just re-inherit the worker
# default, so we pass explicit values.
soft_time_limit=3600,
time_limit=4200,
)
def incremental_regroup_duplicates_task(since_iso: str | None = None):
"""Incremental duplicate detection for newly added photos.

View File

@@ -20,40 +20,91 @@ from PIL import Image
from app.models.embeddings import Embedding
from app.config import settings
from app.services.feature_flags import (
is_enabled,
FLAG_VISION_ENABLED,
FLAG_OCR_ENABLED,
FLAG_DETECTOR_ENABLED,
FLAG_FACES_ENABLED,
FLAG_CLASSIFIER_ENABLED,
)
logger = logging.getLogger(__name__)
VISION_READY_KEY = "mulita:vision:ready"
def _vision_worker_ready() -> bool:
"""Check whether the vision worker has finished model bootstrap."""
try:
import redis as _redis
return bool(_redis.from_url(settings.redis_url).exists(VISION_READY_KEY))
except Exception:
return False
_sync_engine = None
def _get_sync_engine():
"""Return a module-level singleton engine (one per worker process)."""
global _sync_engine
if _sync_engine is None:
sync_url = settings.database_url.replace("+asyncpg", "+psycopg2").replace("+aiosqlite", "")
_sync_engine = create_engine(sync_url, pool_pre_ping=True, pool_size=3, max_overflow=5)
return _sync_engine
def _get_sync_session() -> Session:
"""Create a sync DB session for use in Celery workers."""
sync_url = settings.database_url.replace("+asyncpg", "+psycopg2").replace("+aiosqlite", "")
engine = create_engine(sync_url, pool_pre_ping=True)
return sessionmaker(bind=engine)()
"""Create a sync DB session backed by the shared engine."""
return sessionmaker(bind=_get_sync_engine())()
def _load_thumb(photo_id: str, size: str = "medium") -> np.ndarray | None:
"""Load a thumbnail as an RGB numpy array."""
thumb_path = Path(f"/data/thumbs/{photo_id}/{size}.webp")
"""Load a thumbnail as an RGB numpy array.
Thumbnails may live at ``/data/thumbs/{photo_id}/`` (legacy) or
``/data/thumbs/{user_id}/{photo_id}/`` (multi-user). Try both.
"""
thumb_base = Path("/data/thumbs")
# Try legacy flat path first.
thumb_path = thumb_base / photo_id / f"{size}.webp"
if not thumb_path.exists():
# Try user-prefixed paths: /data/thumbs/*/photo_id/size.webp
matches = list(thumb_base.glob(f"*/{photo_id}/{size}.webp"))
if matches:
thumb_path = matches[0]
else:
logger.warning("Thumbnail not found: %s", thumb_path)
return None
try:
img = Image.open(thumb_path).convert("RGB")
return np.array(img)
img.load() # force decode to catch corruption early
arr = np.array(img)
img.close()
return arr
except Exception as e:
logger.warning("Corrupt or unreadable thumbnail for %s: %s", photo_id, e)
return None
@shared_task(name='embed_photo', queue='vision')
def embed_photo(photo_id: str):
@shared_task(name='embed_photo', queue='vision', bind=True, max_retries=3)
def embed_photo(self, photo_id: str):
"""Generate CLIP embedding for a photo and store in pgvector."""
if not settings.vision.enabled:
if not is_enabled(FLAG_VISION_ENABLED):
return {'status': 'skipped', 'reason': 'vision disabled'}
image = _load_thumb(photo_id, "medium") # 640px
if image is None:
return {'status': 'error', 'message': 'thumbnail not found'}
try:
from app.services.vision.registry import registry
embedder = registry.get_embedder()
vector = embedder.embed_image(image)
except Exception as exc:
logger.exception("embed_photo failed for %s", photo_id)
raise self.retry(exc=exc, countdown=60)
model_name = settings.vision.embedder.name
@@ -72,46 +123,53 @@ def embed_photo(photo_id: str):
)
session.add(emb)
session.commit()
except Exception:
session.rollback()
raise
finally:
session.close()
logger.info("Embedded photo %s with %s", photo_id, model_name)
logger.info("[%s] Embedded photo %s with %s", self.request.id, photo_id, model_name)
return {'status': 'success', 'photo_id': photo_id}
@shared_task(name='vision_fanout', queue='vision')
def vision_fanout(photo_id: str):
"""Dispatch all enabled vision tasks for a photo."""
if not settings.vision.enabled:
if not is_enabled(FLAG_VISION_ENABLED):
return {'status': 'skipped', 'reason': 'vision disabled'}
embed_photo.delay(photo_id)
if settings.vision.ocr.enabled:
if is_enabled(FLAG_OCR_ENABLED):
ocr_photo.delay(photo_id)
if settings.vision.detector.enabled:
if is_enabled(FLAG_DETECTOR_ENABLED):
detect_objects.delay(photo_id)
if settings.vision.faces.enabled:
if is_enabled(FLAG_FACES_ENABLED):
extract_faces.delay(photo_id)
if settings.vision.classifier.enabled:
if is_enabled(FLAG_CLASSIFIER_ENABLED):
classify_content.delay(photo_id)
return {'status': 'dispatched', 'photo_id': photo_id}
@shared_task(name='ocr_photo', queue='vision')
def ocr_photo(photo_id: str):
@shared_task(name='ocr_photo', queue='vision', bind=True, max_retries=3)
def ocr_photo(self, photo_id: str):
"""Run OCR on a photo and store text regions."""
if not settings.vision.enabled or not settings.vision.ocr.enabled:
if not is_enabled(FLAG_VISION_ENABLED) or not is_enabled(FLAG_OCR_ENABLED):
return {'status': 'skipped', 'reason': 'OCR disabled'}
image = _load_thumb(photo_id, "large") # 1280px for better OCR accuracy
if image is None:
return {'status': 'error', 'message': 'thumbnail not found'}
try:
from app.services.vision.registry import registry
ocr_engine = registry.get_ocr()
results = ocr_engine.run(image)
except Exception as exc:
logger.exception("ocr_photo failed for %s", photo_id)
raise self.retry(exc=exc, countdown=60)
if not results:
logger.info("No OCR text found for photo %s", photo_id)
@@ -131,32 +189,40 @@ def ocr_photo(photo_id: str):
bbox=r.bbox,
))
session.commit()
except Exception:
session.rollback()
raise
finally:
session.close()
logger.info("OCR: %d text regions for photo %s", len(results), photo_id)
logger.info("[%s] OCR: %d text regions for photo %s", self.request.id, len(results), photo_id)
return {'status': 'success', 'photo_id': photo_id, 'regions': len(results)}
@shared_task(name='detect_objects', queue='vision')
def detect_objects(photo_id: str):
@shared_task(name='detect_objects', queue='vision', bind=True, max_retries=3)
def detect_objects(self, photo_id: str):
"""Detect objects in a photo, create Tag(kind=object) rows, and
link via photo_tags with confidence/bbox/source."""
if not settings.vision.enabled or not settings.vision.detector.enabled:
if not is_enabled(FLAG_VISION_ENABLED) or not is_enabled(FLAG_DETECTOR_ENABLED):
return {'status': 'skipped', 'reason': 'detection disabled'}
image = _load_thumb(photo_id, "medium") # 640px
if image is None:
return {'status': 'error', 'message': 'thumbnail not found'}
try:
from app.services.vision.registry import registry
detector = registry.get_detector()
detections = detector.detect(image)
except Exception as exc:
logger.exception("detect_objects failed for %s", photo_id)
raise self.retry(exc=exc, countdown=60)
if not detections:
logger.info("No objects detected for photo %s", photo_id)
return {'status': 'success', 'photo_id': photo_id, 'objects': 0}
from app.models import Photo
from app.models.tags import Tag, photo_tags
source_name = "vision:yolov8n"
@@ -206,33 +272,41 @@ def detect_objects(photo_id: str):
)
session.commit()
except Exception:
session.rollback()
raise
finally:
session.close()
labels = [d.label for d in detections]
logger.info("Detected %d objects in photo %s: %s", len(detections), photo_id, labels)
logger.info("[%s] Detected %d objects in photo %s: %s", self.request.id, len(detections), photo_id, labels)
return {'status': 'success', 'photo_id': photo_id, 'objects': len(detections)}
@shared_task(name='classify_content', queue='vision')
def classify_content(photo_id: str):
@shared_task(name='classify_content', queue='vision', bind=True, max_retries=3)
def classify_content(self, photo_id: str):
"""Classify image content type (screenshot, document, artwork, etc.)
using CLIP zero-shot classification. Writes Tag(kind=content_type)."""
if not settings.vision.enabled or not settings.vision.classifier.enabled:
if not is_enabled(FLAG_VISION_ENABLED) or not is_enabled(FLAG_CLASSIFIER_ENABLED):
return {'status': 'skipped', 'reason': 'classifier disabled'}
image = _load_thumb(photo_id, "medium")
if image is None:
return {'status': 'error', 'message': 'thumbnail not found'}
try:
from app.services.vision.registry import registry
classifier = registry.get_classifier()
results = classifier.classify(image)
except Exception as exc:
logger.exception("classify_content failed for %s", photo_id)
raise self.retry(exc=exc, countdown=60)
if not results:
logger.info("No confident classification for photo %s", photo_id)
return {'status': 'success', 'photo_id': photo_id, 'content_type': None}
from app.models import Photo
from app.models.tags import Tag, photo_tags
source_name = "vision:clip_classifier"
@@ -273,10 +347,13 @@ def classify_content(photo_id: str):
)
)
session.commit()
except Exception:
session.rollback()
raise
finally:
session.close()
logger.info("Classified photo %s as '%s' (%.2f)", photo_id, best.label, best.confidence)
logger.info("[%s] Classified photo %s as '%s' (%.2f)", self.request.id, photo_id, best.label, best.confidence)
return {'status': 'success', 'photo_id': photo_id, 'content_type': best.label}
@@ -309,19 +386,23 @@ def _load_original(photo_id: str) -> np.ndarray | None:
w, h = img.size
if max(w, h) > max_dim:
scale = max_dim / max(w, h)
img = img.resize((int(w * scale), int(h * scale)), Image.BICUBIC)
return np.array(img)
resized = img.resize((int(w * scale), int(h * scale)), Image.BICUBIC)
img.close()
img = resized
arr = np.array(img)
img.close()
return arr
except Exception as e:
logger.warning("Failed to load original %s: %s", filepath, e)
return None
@shared_task(name='extract_faces', queue='vision')
def extract_faces(photo_id: str):
@shared_task(name='extract_faces', queue='vision', bind=True, max_retries=3)
def extract_faces(self, photo_id: str):
"""Detect faces and store recognition embeddings using InsightFace
(RetinaFace + ArcFace). No YOLO workaround needed — RetinaFace has
strong human-vs-non-human precision on its own."""
if not settings.vision.enabled or not settings.vision.faces.enabled:
if not is_enabled(FLAG_VISION_ENABLED) or not is_enabled(FLAG_FACES_ENABLED):
return {'status': 'skipped', 'reason': 'faces disabled'}
image = _load_original(photo_id)
@@ -330,9 +411,13 @@ def extract_faces(photo_id: str):
if image is None:
return {'status': 'error', 'message': 'no image available'}
try:
from app.services.vision.registry import registry
face_proc = registry.get_face_processor()
faces = face_proc.process(image)
except Exception as exc:
logger.exception("extract_faces failed for %s", photo_id)
raise self.retry(exc=exc, countdown=60)
if not faces:
logger.info("No faces detected for photo %s", photo_id)
@@ -355,6 +440,9 @@ def _save_faces(photo_id: str, faces) -> dict:
cluster_id=None,
))
session.commit()
except Exception:
session.rollback()
raise
finally:
session.close()
@@ -383,8 +471,8 @@ def _schedule_recluster_debounced():
logger.debug("recluster debounce check failed: %s", e)
@shared_task(name='recluster_faces', queue='vision')
def recluster_faces():
@shared_task(name='recluster_faces', queue='vision', bind=True, max_retries=10)
def recluster_faces(self):
"""Run DBSCAN clustering over all face embeddings and assign/create
Tag(kind=face_cluster) entries."""
# Clear debounce key so new face extractions can schedule another round.
@@ -394,9 +482,14 @@ def recluster_faces():
except Exception:
pass
if not settings.vision.enabled or not settings.vision.faces.enabled:
if not _vision_worker_ready():
logger.info("Vision worker not ready yet — retrying in 30s")
raise self.retry(countdown=30)
if not is_enabled(FLAG_VISION_ENABLED) or not is_enabled(FLAG_FACES_ENABLED):
return {'status': 'skipped', 'reason': 'faces disabled'}
from app.models import Photo
from app.models.face_embedding import FaceEmbedding
from app.models.tags import Tag, photo_tags
from app.services.vision.clustering import cluster_faces
@@ -477,6 +570,9 @@ def recluster_faces():
)
session.commit()
except Exception:
session.rollback()
raise
finally:
session.close()
@@ -485,45 +581,98 @@ def recluster_faces():
return {'status': 'success', 'clusters': n_clusters, 'faces': len(face_rows)}
@shared_task(name='backfill_vision')
def backfill_vision(task: str | None = None, limit: int | None = None):
@shared_task(name='backfill_vision', bind=True, max_retries=10)
def backfill_vision(self, task: str | None = None, limit: int | None = None):
"""Queue vision tasks for photos that haven't been processed yet.
Uses a sync DB connection to avoid asyncpg conflicts in Celery."""
if not _vision_worker_ready():
logger.info("Vision worker not ready yet — retrying in 30s")
raise self.retry(countdown=30)
model_name = settings.vision.embedder.name
# Newest-first ordering — matches regenerate_all_thumbnails so the
# whole ingestion pipeline sweeps the library top-down and the user
# sees recent photos fully-indexed long before the backlog drains.
# `taken_at` is the canonical capture timestamp (from EXIF, falls
# back to filesystem mtime in scan); `added_at` is the tie-breaker
# when taken_at is null.
sql = """
SELECT p.id FROM photos p
LEFT JOIN embeddings e ON e.photo_id = p.id AND e.model = :model
WHERE e.photo_id IS NULL
AND p.processing_status = 'completed'
ORDER BY p.taken_at DESC NULLS LAST, p.added_at DESC NULLS LAST
"""
ordering = "ORDER BY p.taken_at DESC NULLS LAST, p.added_at DESC NULLS LAST"
limit_clause = " LIMIT :lim" if limit else ""
params: dict = {"model": model_name}
if limit:
sql += f" LIMIT {limit}"
params["lim"] = int(limit)
session = _get_sync_session()
try:
result = session.execute(sa_text(sql), {"model": model_name})
photo_ids = [row[0] for row in result.fetchall()]
# Each query finds photos missing a specific pipeline output so
# enabling a new processor after import still back-fills.
embed_ids = []
if task in ('embed', None):
sql = f"""
SELECT p.id FROM photos p
LEFT JOIN embeddings e ON e.photo_id = p.id AND e.model = :model
WHERE e.photo_id IS NULL AND p.processing_status = 'completed'
{ordering}{limit_clause}
"""
embed_ids = [r[0] for r in session.execute(sa_text(sql), params).fetchall()]
ocr_ids = []
if task in ('ocr', None) and is_enabled(FLAG_OCR_ENABLED):
sql = f"""
SELECT p.id FROM photos p
LEFT JOIN ocr_text o ON o.photo_id = p.id
WHERE o.photo_id IS NULL AND p.processing_status = 'completed'
{ordering}{limit_clause}
"""
ocr_ids = [r[0] for r in session.execute(sa_text(sql), params).fetchall()]
detect_ids = []
if task in ('detect', None) and is_enabled(FLAG_DETECTOR_ENABLED):
sql = f"""
SELECT p.id FROM photos p
WHERE p.processing_status = 'completed'
AND NOT EXISTS (
SELECT 1 FROM photo_tags pt WHERE pt.photo_id = p.id
AND pt.source = 'vision:yolov8n'
)
{ordering}{limit_clause}
"""
detect_ids = [r[0] for r in session.execute(sa_text(sql), params).fetchall()]
face_ids = []
if task in ('faces', None) and is_enabled(FLAG_FACES_ENABLED):
sql = f"""
SELECT p.id FROM photos p
LEFT JOIN face_embeddings fe ON fe.photo_id = p.id
WHERE fe.photo_id IS NULL AND p.processing_status = 'completed'
{ordering}{limit_clause}
"""
face_ids = [r[0] for r in session.execute(sa_text(sql), params).fetchall()]
classify_ids = []
if task in ('classify', None) and is_enabled(FLAG_CLASSIFIER_ENABLED):
sql = f"""
SELECT p.id FROM photos p
WHERE p.processing_status = 'completed'
AND NOT EXISTS (
SELECT 1 FROM photo_tags pt WHERE pt.photo_id = p.id
AND pt.source = 'vision:clip_classifier'
)
{ordering}{limit_clause}
"""
classify_ids = [r[0] for r in session.execute(sa_text(sql), params).fetchall()]
except Exception:
session.rollback()
raise
finally:
session.close()
count = 0
for pid in photo_ids:
if task == 'embed' or task is None:
# Dispatch — deduplicate across query results.
all_ids = set(embed_ids) | set(ocr_ids) | set(detect_ids) | set(face_ids) | set(classify_ids)
for pid in embed_ids:
embed_photo.delay(pid)
if task == 'ocr' or task is None:
for pid in ocr_ids:
ocr_photo.delay(pid)
if task == 'detect' or task is None:
for pid in detect_ids:
detect_objects.delay(pid)
if task == 'faces' or task is None:
for pid in face_ids:
extract_faces.delay(pid)
count += 1
for pid in classify_ids:
classify_content.delay(pid)
logger.info("Backfill queued %d photos for vision processing", count)
return {'status': 'queued', 'count': count}
logger.info("Backfill queued %d photos for vision processing", len(all_ids))
return {'status': 'queued', 'count': len(all_ids)}

View File

@@ -18,7 +18,12 @@ flower==2.0.1
# Image processing
# pyvips==2.2.1 # Optional - having compatibility issues, using Pillow as fallback
# rawpy==0.19.0 # Optional - numpy compatibility issues, using Pillow as fallback
rawpy==0.26.1 # RAW decoder (CR2/NEF/ARW/DNG/…). cp312 wheels
# ship with libraw bundled; the older 0.19 pin
# had numpy 2.x incompatibilities — 0.26 is fine
# with our numpy 1.26. iPhone ProRAW-style DNGs
# that aren't real RAW still fail here; thumbs.py
# falls back to opening them as JPEG in that case.
pillow==10.2.0
pillow-heif==0.15.0
imagehash==4.3.1 # perceptual hash for duplicate detection

View File

@@ -48,7 +48,7 @@ services:
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=${PHOTO_DIRS:-/photos}
- PHOTO_DIRS=/photos
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-*}
- SECRET_KEY=${SECRET_KEY:-mulita-dev-secret-change-me}
- ACCESS_TOKEN_EXPIRE_MINUTES=${ACCESS_TOKEN_EXPIRE_MINUTES:-60}
@@ -104,7 +104,7 @@ services:
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=${PHOTO_DIRS:-/photos}
- PHOTO_DIRS=/photos
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
# NullPool — see app/database.py for rationale.
@@ -116,6 +116,12 @@ services:
condition: service_started
db:
condition: service_healthy
healthcheck:
test: ["CMD-SHELL", "celery -A app.tasks.celery inspect ping -d light@$$HOSTNAME 2>/dev/null | grep -q OK"]
interval: 30s
timeout: 10s
retries: 3
start_period: 120s
networks:
- mulita-network
restart: unless-stopped
@@ -138,7 +144,7 @@ services:
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=${PHOTO_DIRS:-/photos}
- PHOTO_DIRS=/photos
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
- MULITA_CELERY_WORKER=1
@@ -170,7 +176,7 @@ services:
- REDIS_URL=redis://redis:6379
- CELERY_BROKER_URL=redis://redis:6379
- CELERY_RESULT_BACKEND=redis://redis:6379
- PHOTO_DIRS=${PHOTO_DIRS:-/photos}
- PHOTO_DIRS=/photos
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- TZ=${TZ:-UTC}
- MULITA_CELERY_WORKER=1
@@ -197,6 +203,12 @@ services:
# - driver: nvidia
# count: all
# capabilities: [gpu]
healthcheck:
test: ["CMD-SHELL", "celery -A app.tasks.celery inspect ping -d vision@$$HOSTNAME 2>/dev/null | grep -q OK"]
interval: 30s
timeout: 10s
retries: 3
start_period: 300s
depends_on:
redis:
condition: service_started
@@ -239,6 +251,11 @@ services:
- mulita-network
restart: unless-stopped
command: redis-server --appendonly yes
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
networks:
mulita-network:

View File

@@ -15,15 +15,23 @@ import {
Activity,
FolderSearch,
Shield,
Brain,
ScanText,
UserSquare2,
Boxes,
Tags as TagsIcon,
RotateCcw,
} from 'lucide-react'
import clsx from 'clsx'
import { useQuery, useQueryClient } from '@tanstack/react-query'
import {
library,
admin as adminApi,
type MediaType,
type PipelineStage,
type ScanStatus,
type WorkerStatus,
type FeatureFlagSnapshot,
} from '../../services/api'
import { toast } from '../ToastContainer'
import { useAuth } from '../../contexts/AuthContext'
@@ -42,13 +50,16 @@ const SETTINGS_SCAN_STATUS_KEY = ['settings', 'scan-status'] as const
// the grid renders from. Imported via the canonical hook key.
import { DUPLICATE_GROUPS_QUERY_KEY } from '../../hooks/useDuplicateGroupsQuery'
type SettingsTab = 'library' | 'users'
type SettingsTab = 'library' | 'ai' | 'users'
const TABS: { id: SettingsTab; label: string; adminOnly?: boolean }[] = [
{ id: 'library', label: 'Library Management' },
{ id: 'ai', label: 'AI Features', adminOnly: true },
{ id: 'users', label: 'Users', adminOnly: true },
]
const SETTINGS_FEATURE_FLAGS_KEY = ['settings', 'feature-flags'] as const
/**
* Full-page settings view with tabbed navigation. Replaces the old
* modal dialog — renders as a top-level section in the main content
@@ -258,6 +269,17 @@ export function SettingsPage() {
/>
</div>
{(libStats?.source_dirs?.length ?? 0) > 0 && (
<div className="mt-3 text-xs text-text-muted">
<span className="font-medium text-text">Source folders</span>
<div className="mt-1 space-y-0.5">
{libStats!.source_dirs.map((dir: string) => (
<div key={dir} className="truncate font-mono" title={dir}>{dir}</div>
))}
</div>
</div>
)}
{/* Inline scan progress. Rendered as a full-width sub-card
when a scan is active so the user sees the same info
they'd get from the floating widget without leaving
@@ -831,6 +853,13 @@ export function SettingsPage() {
</Section>
</>)}
{activeTab === 'ai' && isAdmin && (
<AiFeaturesTab
busy={busy}
runAction={runAction}
/>
)}
{activeTab === 'users' && isAdmin && (
<Section
icon={<Shield className="h-4 w-4" />}
@@ -1062,3 +1091,267 @@ function ActionButton({
</button>
)
}
// ---------------------------------------------------------------------------
// AI Features admin tab
// ---------------------------------------------------------------------------
interface AiFeaturesTabProps {
busy: Record<string, boolean>
runAction: <T>(
key: string,
fn: () => Promise<T>,
successTitle: string,
describe?: (result: T) => string | undefined,
) => Promise<void>
}
// Flags are keyed by the backend's canonical name ("vision.enabled",
// "vision.ocr.enabled", ...). The metadata here just adds presentation
// (label, short description, icon) so the tab layout stays data-driven.
const FLAG_META: Array<{
id: string
label: string
description: string
icon: React.ReactNode
// Optional "run this backfill" hook — lets the user kick off a stage's
// backfill right from the toggle row without hopping to a separate UI.
backfillTask?: 'embed' | 'ocr' | 'detect' | 'faces' | 'classify'
}> = [
{
id: 'vision.enabled',
label: 'Vision pipeline (master switch)',
description:
'When off, every AI stage below is skipped — including newly uploaded photos. ' +
'Existing results stay intact.',
icon: <Sparkles className="h-3.5 w-3.5" />,
},
{
id: 'vision.ocr.enabled',
label: 'Text recognition (OCR)',
description: 'Extract printed / handwritten text from photos so it becomes searchable.',
icon: <ScanText className="h-3.5 w-3.5" />,
backfillTask: 'ocr',
},
{
id: 'vision.detector.enabled',
label: 'Object detection',
description: 'Tag photos with detected objects (person, car, dog, …) via YOLOv8n.',
icon: <Boxes className="h-3.5 w-3.5" />,
backfillTask: 'detect',
},
{
id: 'vision.faces.enabled',
label: 'Face recognition',
description:
'Find and cluster faces across the library (RetinaFace + ArcFace). ' +
'Expensive on big libraries — disable if you don\'t need the People view.',
icon: <UserSquare2 className="h-3.5 w-3.5" />,
backfillTask: 'faces',
},
{
id: 'vision.classifier.enabled',
label: 'Content classification',
description: 'Zero-shot CLIP tags for scenes / activities (beach, wedding, …).',
icon: <TagsIcon className="h-3.5 w-3.5" />,
backfillTask: 'classify',
},
]
function AiFeaturesTab({ busy, runAction }: AiFeaturesTabProps) {
const queryClient = useQueryClient()
const flagsQuery = useQuery<{ flags: FeatureFlagSnapshot }>({
queryKey: SETTINGS_FEATURE_FLAGS_KEY,
queryFn: adminApi.listFeatureFlags,
staleTime: 5_000,
})
const flags = flagsQuery.data?.flags ?? {}
const masterOff = flags['vision.enabled'] && !flags['vision.enabled'].effective
const applyFlag = async (name: string, value: boolean | null) => {
await runAction(
`flag:${name}`,
() => adminApi.setFeatureFlag(name, value),
value === null ? 'Override cleared' : `Feature ${value ? 'enabled' : 'disabled'}`,
)
queryClient.invalidateQueries({ queryKey: SETTINGS_FEATURE_FLAGS_KEY })
// Non-admin feature map drives sidebar gating — invalidate so the
// People / Tags entries appear / disappear immediately without a
// page reload.
queryClient.invalidateQueries({ queryKey: ['features'] })
}
type BackfillTask = 'embed' | 'ocr' | 'detect' | 'faces' | 'classify' | null
const runBackfill = (task: BackfillTask) =>
runAction(
`ai-backfill:${task ?? 'all'}`,
() => adminApi.triggerAiBackfill({ task }),
task ? `Backfill queued for ${task}` : 'Full backfill queued',
(r) => `Celery task ${r.task_id}`,
)
return (
<>
<Section icon={<Brain className="h-4 w-4" />} title="AI feature flags">
<p className="text-xs text-text-muted">
Toggle each stage at runtime. Changes are observed by Celery
workers on the next task no restart needed. "Default" means
the flag hasn\'t been overridden and is tracking the YAML config;
an overridden flag is pinned to the value shown until cleared.
</p>
{flagsQuery.isLoading && (
<div className="mt-3 flex items-center gap-2 text-xs text-text-muted">
<Loader2 className="h-3 w-3 animate-spin" />
Loading feature flags…
</div>
)}
{flagsQuery.error && (
<ErrorBanner
title="Could not load feature flags"
detail={String((flagsQuery.error as Error).message || flagsQuery.error)}
/>
)}
{!flagsQuery.isLoading && !flagsQuery.error && (
<div className="mt-3 space-y-2">
{FLAG_META.map((meta) => {
const state = flags[meta.id]
if (!state) return null
const busyKey = `flag:${meta.id}`
const isBusy = !!busy[busyKey]
const isMaster = meta.id === 'vision.enabled'
const dimmed = !isMaster && masterOff
return (
<div
key={meta.id}
className={clsx(
'rounded border border-border bg-surface p-3 text-xs',
dimmed && 'opacity-60',
)}
>
<div className="flex items-start justify-between gap-3">
<div className="min-w-0 flex-1">
<div className="flex items-center gap-1.5 text-text">
{meta.icon}
<span className="font-medium">{meta.label}</span>
{state.overridden && (
<span className="rounded bg-primary/20 px-1 py-0.5 text-[9px] font-semibold uppercase tracking-wide text-primary">
overridden
</span>
)}
</div>
<p className="mt-1 text-[11px] text-text-muted">{meta.description}</p>
<p className="mt-1 text-[10px] text-text-faint">
Default: {state.default ? 'on' : 'off'} · Currently:{' '}
<span className={state.effective ? 'text-pick' : 'text-reject'}>
{state.effective ? 'on' : 'off'}
</span>
</p>
</div>
<div className="flex shrink-0 items-center gap-2">
<button
role="switch"
aria-checked={state.effective}
onClick={() => applyFlag(meta.id, !state.effective)}
disabled={isBusy || (dimmed && !isMaster)}
className={clsx(
'relative inline-flex h-6 w-11 shrink-0 items-center rounded-full transition-colors',
state.effective ? 'bg-primary' : 'bg-surface-2 border border-border',
(isBusy || (dimmed && !isMaster)) && 'cursor-not-allowed opacity-50',
)}
title={state.effective ? 'Click to disable' : 'Click to enable'}
>
<span
aria-hidden="true"
className={clsx(
'inline-block h-5 w-5 transform rounded-full bg-white shadow transition-transform',
state.effective ? 'translate-x-[22px]' : 'translate-x-0.5',
)}
/>
</button>
{state.overridden && (
<button
onClick={() => applyFlag(meta.id, null)}
disabled={isBusy}
className="rounded border border-border p-1 text-text-muted hover:bg-surface-2 hover:text-text disabled:opacity-50"
title="Reset to YAML default"
aria-label="Reset override"
>
<RotateCcw className="h-3 w-3" />
</button>
)}
</div>
</div>
{meta.backfillTask && state.effective && !masterOff && (
<div className="mt-2">
<ActionButton
loading={!!busy[`ai-backfill:${meta.backfillTask}`]}
onClick={() => runBackfill(meta.backfillTask!)}
>
<RefreshCw className="h-3.5 w-3.5" />
Run {meta.backfillTask} backfill
</ActionButton>
</div>
)}
</div>
)
})}
</div>
)}
</Section>
<Section icon={<Cpu className="h-4 w-4" />} title="Manual pipeline triggers">
<p className="text-xs text-text-muted">
Run a full pass across the enabled stages, recompute face
clusters, or force a fresh filesystem scan. All three are safe
to run repeatedly — the backfill only touches photos that
don\'t yet have a given output, and the rescan skips files
that are already indexed.
</p>
<div className="mt-3 flex flex-wrap gap-2">
<ActionButton
loading={!!busy['ai-backfill:all']}
onClick={() => runBackfill(null)}
disabled={masterOff}
>
<Sparkles className="h-4 w-4" />
Run full vision backfill
</ActionButton>
<ActionButton
loading={!!busy['recluster']}
onClick={() =>
runAction(
'recluster',
() => adminApi.triggerFaceRecluster(),
'Face recluster queued',
(r) => `Celery task ${r.task_id}`,
)
}
disabled={masterOff || !flags['vision.faces.enabled']?.effective}
>
<UserSquare2 className="h-4 w-4" />
Recluster faces
</ActionButton>
<ActionButton
loading={!!busy['rescan-full']}
onClick={() =>
runAction(
'rescan-full',
() => adminApi.triggerFullRescan(),
'Rescan queued',
(r) => `Celery task ${r.task_id}`,
)
}
>
<RefreshCw className="h-4 w-4" />
Rescan all source roots
</ActionButton>
</div>
</Section>
</>
)
}

View File

@@ -10,15 +10,19 @@ import {
Pencil,
Copy,
Trash2,
Users,
Download as DownloadIcon,
} from 'lucide-react'
import clsx from 'clsx'
import { useMutation, useQueryClient } from '@tanstack/react-query'
import { useHeapsQuery, HEAPS_QUERY_KEY } from '../../hooks/useHeapsQuery'
import { heaps as heapsApi, type Heap } from '../../services/api'
import { useSharedHeapsQuery } from '../../hooks/useSharingQueries'
import { heaps as heapsApi, downloads, type Heap } from '../../services/api'
import { useFilterStore } from '../../store/filterStore'
import { toast } from '../ToastContainer'
import { PHOTO_DRAG_MIME } from '../timeline/PhotoThumbnail'
import { HeapConvertDialog } from './HeapConvertDialog'
import { ShareDialog } from '../sharing/ShareDialog'
/**
* Heaps panel for the left sidebar. Renders the list of heaps with the
@@ -43,6 +47,8 @@ export function HeapsPanel() {
// the drop highlight ring. Only one heap can be the target at a time.
const [dropTargetId, setDropTargetId] = useState<string | null>(null)
const [convertingHeap, setConvertingHeap] = useState<Heap | null>(null)
const [sharingHeap, setSharingHeap] = useState<Heap | null>(null)
const { data: sharedHeaps = [] } = useSharedHeapsQuery()
// Inline rename state for heap rows: stores the heap id being edited and
// the draft name. Mirrors the folder rename pattern in LeftSidebar.
const [renamingId, setRenamingId] = useState<string | null>(null)
@@ -435,6 +441,22 @@ export function HeapsPanel() {
setConvertingHeap(heap)
}}
/>
<MenuItem
icon={<Users className="h-3.5 w-3.5" />}
label="Share…"
onClick={() => {
setOpenMenuId(null)
setSharingHeap(heap)
}}
/>
<MenuItem
icon={<DownloadIcon className="h-3.5 w-3.5" />}
label="Download as zip"
onClick={() => {
setOpenMenuId(null)
downloads.trigger(downloads.heapUrl(heap.id))
}}
/>
<div className="my-1 h-px bg-border" />
<MenuItem
icon={<Trash2 className="h-3.5 w-3.5" />}
@@ -460,10 +482,63 @@ export function HeapsPanel() {
</div>
)}
{/* Shared with me */}
{sharedHeaps.length > 0 && expanded && (
<div className="mt-1">
<div className="px-3 py-0.5 text-[9px] font-semibold uppercase tracking-[0.14em] text-text-faint">
Shared with me
</div>
{sharedHeaps.map((sh) => {
const isFiltered = currentSection === `heap-${sh.id}`
return (
<div
key={sh.id}
className={clsx(
'group flex h-[24px] cursor-pointer items-center gap-1 rounded px-2 text-[12px] leading-none',
isFiltered ? 'bg-primary/20 text-primary' : 'text-text hover:bg-surface-2',
)}
style={{ paddingLeft: '20px' }}
onClick={() =>
navigateToSection(`heap-${sh.id}`, { heapId: sh.id })
}
>
<Users
className={clsx(
'h-3.5 w-3.5 flex-shrink-0',
isFiltered ? 'text-primary' : 'text-text-muted'
)}
/>
<span className="truncate" title={sh.name}>
{sh.name}
</span>
<span className="ml-0.5 truncate text-[10px] text-text-faint">
{sh.owner_username}
</span>
<span className="ml-auto rounded bg-surface-offset px-1 text-[9px] uppercase text-text-faint">
{sh.permission}
</span>
{sh.photo_count > 0 && (
<span className="flex h-4 min-w-[20px] flex-shrink-0 items-center justify-center rounded bg-surface-offset px-1 text-[10px] tabular-nums text-text-muted">
{sh.photo_count}
</span>
)}
</div>
)
})}
</div>
)}
<HeapConvertDialog
heap={convertingHeap}
onClose={() => setConvertingHeap(null)}
/>
<ShareDialog
isOpen={!!sharingHeap}
type="heap"
targetId={sharingHeap?.id ?? ''}
targetName={sharingHeap?.name ?? ''}
onClose={() => setSharingHeap(null)}
/>
</div>
)
}

View File

@@ -24,9 +24,11 @@ import {
LogOut,
Shield,
Clock,
Upload as UploadIcon,
Download as DownloadIcon,
} from 'lucide-react'
import clsx from 'clsx'
import { sourceFolders, photos as photosApi, type FolderTreeNode } from '../../services/api'
import { sourceFolders, photos as photosApi, downloads, type FolderTreeNode } from '../../services/api'
import { useMutation, useQueryClient } from '@tanstack/react-query'
import { toast } from '../ToastContainer'
import { useFilterStore } from '../../store/filterStore'
@@ -44,7 +46,11 @@ import {
import { registerUndoable } from '../../store/undoStore'
import type { Photo } from '../../types/photo'
import { DeleteFolderDialog } from '../dialogs/DeleteFolderDialog'
import { ShareDialog } from '../sharing/ShareDialog'
import { UploadModal } from '../upload/UploadModal'
import { useSharedFoldersQuery } from '../../hooks/useSharingQueries'
import { useAuth } from '../../contexts/AuthContext'
import { useFeaturesQuery } from '../../hooks/useFeaturesQuery'
interface TreeItem {
id: string
@@ -76,6 +82,15 @@ export function LeftSidebar({ onCollapse }: LeftSidebarProps) {
const { data: allTags = [] } = useTagsQuery()
const { data: faceClusters = [] } = useTagsQuery('face_cluster')
const { data: stats } = useLibraryStatsQuery()
const { data: featuresMap } = useFeaturesQuery()
const visionOn = featuresMap ? featuresMap['vision.enabled'] !== false : true
const facesOn = visionOn && (featuresMap ? featuresMap['vision.faces.enabled'] !== false : true)
const tagsOn =
visionOn &&
(featuresMap
? featuresMap['vision.detector.enabled'] !== false ||
featuresMap['vision.classifier.enabled'] !== false
: true)
const [dropTargetId, setDropTargetId] = useState<string | null>(null)
// Per-folder kebab menu open state. Stores the tree-item id ("folder-..."
@@ -111,6 +126,19 @@ export function LeftSidebar({ onCollapse }: LeftSidebarProps) {
name: string
photoCount?: number
} | null>(null)
const [sharingFolder, setSharingFolder] = useState<{
id: string
name: string
} | null>(null)
// Upload modal state. `uploadTarget` stores the pre-selected
// destination Folder/SourceRoot id so "Upload here…" on a folder row
// drops files straight into that folder; null means the general
// Library-level button (defaults to the first source root).
const [uploadTarget, setUploadTarget] = useState<{ open: boolean; folderId: string | null }>({
open: false,
folderId: null,
})
const { data: sharedFolders = [] } = useSharedFoldersQuery()
// Bulk discard mutation for the drag-onto-Discarded interaction.
const discardDropMutation = useMutation({
@@ -407,8 +435,8 @@ export function LeftSidebar({ onCollapse }: LeftSidebarProps) {
children: [
{ id: 'all-photos', label: 'All Photos', icon: <Image className="h-4 w-4" />, count: stats?.all_photos ?? 0 },
{ id: 'rated', label: 'Rated', icon: <Star className="h-4 w-4" />, count: stats?.rated ?? 0 },
{ id: 'tags', label: 'Tags', icon: <TagIcon className="h-4 w-4" />, count: tagsTotalCount },
{ id: 'people', label: 'People', icon: <Users className="h-4 w-4" />, count: peopleTotalCount },
...(tagsOn ? [{ id: 'tags', label: 'Tags', icon: <TagIcon className="h-4 w-4" />, count: tagsTotalCount }] : []),
...(facesOn ? [{ id: 'people', label: 'People', icon: <Users className="h-4 w-4" />, count: peopleTotalCount }] : []),
{ id: 'colors', label: 'Colors', icon: <Palette className="h-4 w-4" />, count: stats?.colored ?? 0 },
{ id: 'map', label: 'Map', icon: <MapPin className="h-4 w-4" />, count: stats?.with_gps ?? 0 },
{ id: 'memories', label: 'Memories', icon: <Clock className="h-4 w-4" /> },
@@ -663,6 +691,14 @@ export function LeftSidebar({ onCollapse }: LeftSidebarProps) {
onClick={(e) => e.stopPropagation()}
className="absolute right-0 top-full z-30 mt-1 min-w-[180px] overflow-hidden rounded-lg border border-border bg-surface py-1 text-sm shadow-xl"
>
<FolderMenuItem
icon={<UploadIcon className="h-3.5 w-3.5" />}
label="Upload here…"
onClick={() => {
setOpenMenuId(null)
setUploadTarget({ open: true, folderId })
}}
/>
<FolderMenuItem
icon={<FolderPlus className="h-3.5 w-3.5" />}
label="New sub-folder"
@@ -703,6 +739,22 @@ export function LeftSidebar({ onCollapse }: LeftSidebarProps) {
})
}}
/>
<FolderMenuItem
icon={<Users className="h-3.5 w-3.5" />}
label="Share…"
onClick={() => {
setOpenMenuId(null)
setSharingFolder({ id: folderId, name: item.label })
}}
/>
<FolderMenuItem
icon={<DownloadIcon className="h-3.5 w-3.5" />}
label="Download as zip"
onClick={() => {
setOpenMenuId(null)
downloads.trigger(downloads.folderUrl(folderId))
}}
/>
<div className="my-1 h-px bg-border" />
<FolderMenuItem
icon={<Trash2 className="h-3.5 w-3.5" />}
@@ -786,6 +838,15 @@ export function LeftSidebar({ onCollapse }: LeftSidebarProps) {
<h2 className="text-[11px] font-semibold uppercase tracking-[0.14em] text-text-muted">
Library
</h2>
<div className="flex items-center gap-1">
<button
onClick={() => setUploadTarget({ open: true, folderId: null })}
className="rounded p-0.5 text-text-muted hover:bg-surface-2 hover:text-text"
title="Upload photos"
aria-label="Upload photos"
>
<UploadIcon className="h-3.5 w-3.5" />
</button>
<button
onClick={onCollapse}
className="rounded p-0.5 text-text-muted hover:bg-surface-2 hover:text-text"
@@ -795,6 +856,7 @@ export function LeftSidebar({ onCollapse }: LeftSidebarProps) {
<PanelLeftClose className="h-3.5 w-3.5" />
</button>
</div>
</div>
{/* Active heap card — pinned just below the Library header so
* toasts (bottom-left fixed) can't cover it. Returns null when
* no heap is active, so the layout collapses cleanly. */}
@@ -803,6 +865,53 @@ export function LeftSidebar({ onCollapse }: LeftSidebarProps) {
{/* Tree View */}
<div className="flex-1 overflow-y-auto pb-2">
{libraryTree.map((item) => renderTreeItem(item))}
{/* Shared with me — folders shared by other users */}
{sharedFolders.length > 0 && (
<div className="mt-1">
<div className="px-3 py-1 text-[10px] font-semibold uppercase tracking-[0.14em] text-text-muted">
Shared with me
</div>
{sharedFolders.map((sf) => {
const isSelected = currentSection === `folder-${sf.id}`
return (
<div
key={sf.id}
className={clsx(
'flex h-[24px] cursor-pointer items-center gap-1 rounded px-2 text-[12px] leading-none',
isSelected ? 'bg-primary/20 text-primary' : 'text-text hover:bg-surface-2',
)}
style={{ paddingLeft: '20px' }}
onClick={() =>
navigateToSection(`folder-${sf.id}`, { folderId: sf.id })
}
>
<Users
className={clsx(
'h-3.5 w-3.5 flex-shrink-0',
isSelected ? 'text-primary' : 'text-text-muted'
)}
/>
<span className="truncate" title={sf.name}>
{sf.name}
</span>
<span className="ml-0.5 truncate text-[10px] text-text-faint">
{sf.owner_username}
</span>
<span className="ml-auto rounded bg-surface-offset px-1 text-[9px] uppercase text-text-faint">
{sf.permission}
</span>
{sf.photo_count > 0 && (
<span className="flex h-4 min-w-[20px] flex-shrink-0 items-center justify-center rounded bg-surface-offset px-1 text-[10px] tabular-nums text-text-muted">
{sf.photo_count}
</span>
)}
</div>
)
})}
</div>
)}
<HeapsPanel />
</div>
@@ -853,6 +962,18 @@ export function LeftSidebar({ onCollapse }: LeftSidebarProps) {
}
}}
/>
<ShareDialog
isOpen={!!sharingFolder}
type="folder"
targetId={sharingFolder?.id ?? ''}
targetName={sharingFolder?.name ?? ''}
onClose={() => setSharingFolder(null)}
/>
<UploadModal
isOpen={uploadTarget.open}
initialFolderId={uploadTarget.folderId}
onClose={() => setUploadTarget({ open: false, folderId: null })}
/>
</div>
)
}

View File

@@ -1,11 +1,6 @@
import { useEffect, useMemo, useRef } from 'react'
import { useQuery } from '@tanstack/react-query'
import { MapContainer, TileLayer, Marker, useMap } from 'react-leaflet'
// react-leaflet-cluster has no own .d.ts that survives strict mode in
// every project, so we let TS infer from its runtime export.
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore — package ships JS with no bundled types
import MarkerClusterGroup from 'react-leaflet-cluster'
import { MapContainer, TileLayer, useMap } from 'react-leaflet'
import L from 'leaflet'
import { photos as photosApi } from '../../services/api'
import { usePhotoStore } from '../../store/photoStore'
@@ -18,33 +13,65 @@ interface MapPoint {
taken_at: string | null
}
/** Build the divIcon used for each photo marker. The HTML is a tiny
* square thumbnail with a white border + drop shadow so it reads on
* any tile background. Memoised per-photo so we don't re-create the
* L.DivIcon on every re-render. */
function buildPhotoIcon(photoId: string): L.DivIcon {
const url = photosApi.getThumbnailUrl(photoId, 'small')
return L.divIcon({
className: 'photo-map-marker',
html: `<div class="pmm-frame"><img src="${url}" loading="lazy" alt="" /></div>`,
iconSize: [56, 56],
iconAnchor: [28, 28],
popupAnchor: [0, -28],
})
/** Adds all points as native Leaflet circleMarkers on a single canvas
* layer — no React components per marker, no clustering library.
* 3K+ markers render in <50ms on canvas. */
function CanvasMarkers({
points,
onClickId,
}: {
points: MapPoint[]
onClickId: (id: string) => void
}) {
const map = useMap()
const layerRef = useRef<L.LayerGroup | null>(null)
const fittedRef = useRef(false)
useEffect(() => {
if (!map || points.length === 0) return
// Remove previous layer if data changed.
if (layerRef.current) {
map.removeLayer(layerRef.current)
}
/** Pans/zooms the map to fit the supplied points the first time they
* arrive. Subsequent loads (e.g. cache refresh) leave the user's pan
* alone — they're probably mid-investigation. */
function FitBoundsOnce({ points }: { points: MapPoint[] }) {
const map = useMap()
const fittedRef = useRef(false)
useEffect(() => {
if (fittedRef.current || points.length === 0) return
const bounds = L.latLngBounds(points.map((p) => [p.latitude, p.longitude]))
const group = L.layerGroup()
const renderer = L.canvas({ padding: 0.5 })
for (const p of points) {
const marker = L.circleMarker([p.latitude, p.longitude], {
renderer,
radius: 5,
fillColor: '#3b82f6',
fillOpacity: 0.85,
color: '#ffffff',
weight: 1.5,
opacity: 0.9,
})
marker.on('click', () => onClickId(p.id))
group.addLayer(marker)
}
group.addTo(map)
layerRef.current = group
// Fit bounds once on first data load.
if (!fittedRef.current) {
const bounds = L.latLngBounds(
points.map((p) => [p.latitude, p.longitude] as [number, number])
)
map.fitBounds(bounds, { padding: [40, 40], maxZoom: 14 })
fittedRef.current = true
}, [points, map])
}
return () => {
if (layerRef.current) {
map.removeLayer(layerRef.current)
layerRef.current = null
}
}
}, [map, points, onClickId])
return null
}
@@ -52,27 +79,17 @@ export function MapView() {
const { data: points = [], isLoading, error } = useQuery({
queryKey: ['photos', 'map'],
queryFn: () => photosApi.mapPoints(),
staleTime: 60 * 1000,
staleTime: 5 * 60 * 1000,
})
// Marker click hands off to the same PreviewView the timeline uses, so
// the user gets the full preview UI (large image, info panel with
// location, filmstrip nav between map photos) instead of a bespoke
// map-only lightbox. We pass the map's own point order as the visible
// sequence so left/right arrows step through neighboring markers.
const openPreview = usePhotoStore((s) => s.openPreview)
// Stable per-marker icons. Re-created only when the set of point ids
// changes — the underlying L.DivIcon objects are pure HTML so reusing
// them is safe across re-renders.
const iconsById = useMemo(() => {
const map = new Map<string, L.DivIcon>()
for (const p of points) map.set(p.id, buildPhotoIcon(p.id))
return map
}, [points])
const visibleSequence = useMemo(() => points.map((p) => p.id), [points])
const handleClick = useMemo(
() => (id: string) => openPreview(id, visibleSequence),
[openPreview, visibleSequence],
)
if (isLoading) {
return (
<div className="flex h-full items-center justify-center text-sm text-muted-foreground">
@@ -109,61 +126,15 @@ export function MapView() {
zoom={2}
minZoom={2}
worldCopyJump
preferCanvas
style={{ height: '100%', width: '100%' }}
>
<TileLayer
attribution='&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors'
url="https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png"
/>
<FitBoundsOnce points={points} />
<MarkerClusterGroup chunkedLoading maxClusterRadius={50}>
{points.map((p) => {
const icon = iconsById.get(p.id)
if (!icon) return null
return (
<PhotoMarker
key={p.id}
point={p}
icon={icon}
onClick={() => openPreview(p.id, visibleSequence)}
/>
)
})}
</MarkerClusterGroup>
<CanvasMarkers points={points} onClickId={handleClick} />
</MapContainer>
<style>{`
.photo-map-marker { background: transparent; border: none; }
.photo-map-marker .pmm-frame {
width: 56px; height: 56px;
border-radius: 8px;
overflow: hidden;
border: 2px solid white;
box-shadow: 0 2px 6px rgba(0,0,0,0.4);
background: #1f2937;
}
.photo-map-marker .pmm-frame img {
width: 100%; height: 100%; object-fit: cover; display: block;
}
`}</style>
</div>
)
}
function PhotoMarker({
point,
icon,
onClick,
}: {
point: MapPoint
icon: L.DivIcon
onClick: () => void
}) {
return (
<Marker
position={[point.latitude, point.longitude]}
icon={icon}
eventHandlers={{ click: onClick }}
/>
)
}

View File

@@ -0,0 +1,177 @@
import { useEffect, useState } from 'react'
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
import { Users, Trash2, X } from 'lucide-react'
import clsx from 'clsx'
import { sharing, type ShareInfo } from '../../services/api'
import { SHARED_HEAPS_KEY, SHARED_FOLDERS_KEY } from '../../hooks/useSharingQueries'
interface ShareDialogProps {
isOpen: boolean
type: 'heap' | 'folder'
targetId: string
targetName: string
onClose: () => void
}
export function ShareDialog({ isOpen, type, targetId, targetName, onClose }: ShareDialogProps) {
const [username, setUsername] = useState('')
const [permission, setPermission] = useState<'read' | 'write'>('read')
const [error, setError] = useState<string | null>(null)
const queryClient = useQueryClient()
const sharesQueryKey = [type, 'shares', targetId]
const { data: shares = [], isLoading } = useQuery<ShareInfo[]>({
queryKey: sharesQueryKey,
queryFn: () =>
type === 'heap'
? sharing.heapShares(targetId)
: sharing.folderShares(targetId),
enabled: isOpen,
})
const addMutation = useMutation({
mutationFn: () =>
type === 'heap'
? sharing.shareHeap(targetId, username, permission)
: sharing.shareFolder(targetId, username, permission),
onSuccess: () => {
setUsername('')
setPermission('read')
setError(null)
queryClient.invalidateQueries({ queryKey: sharesQueryKey })
queryClient.invalidateQueries({ queryKey: type === 'heap' ? SHARED_HEAPS_KEY : SHARED_FOLDERS_KEY })
},
onError: (err: any) => {
setError(err?.response?.data?.detail || 'Failed to share')
},
})
const revokeMutation = useMutation({
mutationFn: (shareId: string) =>
type === 'heap'
? sharing.revokeHeapShare(targetId, shareId)
: sharing.revokeFolderShare(targetId, shareId),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: sharesQueryKey })
queryClient.invalidateQueries({ queryKey: type === 'heap' ? SHARED_HEAPS_KEY : SHARED_FOLDERS_KEY })
},
})
useEffect(() => {
if (!isOpen) return
const handler = (e: KeyboardEvent) => {
if (e.key === 'Escape') onClose()
}
window.addEventListener('keydown', handler)
return () => window.removeEventListener('keydown', handler)
}, [isOpen, onClose])
useEffect(() => {
if (isOpen) {
setUsername('')
setPermission('read')
setError(null)
}
}, [isOpen])
if (!isOpen) return null
return (
<div className="fixed inset-0 z-50">
<div className="absolute inset-0 bg-black/60 backdrop-blur-sm" onClick={onClose} />
<div className="absolute left-1/2 top-1/2 -translate-x-1/2 -translate-y-1/2">
<div className="relative z-10 w-[420px] rounded-lg border border-border bg-surface p-5 shadow-2xl">
<div className="mb-4 flex items-center justify-between">
<div className="flex items-center gap-2">
<Users className="h-4 w-4 text-text-muted" />
<h2 className="text-base font-semibold text-text">
Share {type === 'heap' ? 'heap' : 'folder'}
</h2>
</div>
<button onClick={onClose} className="text-text-muted hover:text-text">
<X className="h-4 w-4" />
</button>
</div>
<div className="mb-3 text-sm text-text-muted">
Sharing <span className="font-medium text-text">{targetName}</span>
</div>
{/* Existing shares */}
{shares.length > 0 && (
<div className="mb-4 space-y-1.5">
{shares.map((share) => (
<div
key={share.id}
className="flex items-center justify-between rounded border border-border bg-surface-2 px-3 py-1.5 text-sm"
>
<div className="flex items-center gap-2">
<span className="font-medium text-text">{share.shared_with_username}</span>
<span className="rounded bg-surface px-1.5 py-0.5 text-[10px] font-medium uppercase text-text-muted">
{share.permission}
</span>
</div>
<button
onClick={() => revokeMutation.mutate(share.id)}
className="text-text-muted hover:text-reject"
title="Revoke access"
>
<Trash2 className="h-3.5 w-3.5" />
</button>
</div>
))}
</div>
)}
{isLoading && <div className="mb-4 text-xs text-text-muted">Loading shares...</div>}
{/* Add new share */}
<form
onSubmit={(e) => {
e.preventDefault()
if (username.trim()) addMutation.mutate()
}}
className="space-y-3"
>
<div className="flex gap-2">
<input
type="text"
value={username}
onChange={(e) => {
setUsername(e.target.value)
setError(null)
}}
placeholder="Username"
className="flex-1 rounded border border-border bg-surface-2 px-3 py-1.5 text-sm text-text placeholder:text-text-muted focus:border-primary focus:outline-none"
autoFocus
/>
<select
value={permission}
onChange={(e) => setPermission(e.target.value as 'read' | 'write')}
className="rounded border border-border bg-surface-2 px-2 py-1.5 text-sm text-text"
>
<option value="read">Read</option>
<option value="write">Read + Write</option>
</select>
</div>
{error && <div className="text-xs text-reject">{error}</div>}
<div className="flex justify-end">
<button
type="submit"
disabled={!username.trim() || addMutation.isPending}
className={clsx(
'rounded px-3 py-1.5 text-sm font-medium text-white',
'bg-primary hover:bg-primary/80 disabled:opacity-50'
)}
>
{addMutation.isPending ? 'Sharing...' : 'Share'}
</button>
</div>
</form>
</div>
</div>
</div>
)
}

View File

@@ -7,6 +7,7 @@ import {
Check,
Copy,
AlertTriangle,
Users,
} from 'lucide-react'
import clsx from 'clsx'
import { photos as photosApi } from '../../services/api'
@@ -328,6 +329,24 @@ export function PhotoThumbnail({
</div>
)}
{/* TL offset — owner badge for shared photos. Sits below the
* selection check so both can show simultaneously. */}
{photo.owner_username && (
<div
className={clsx(
'absolute left-1',
isSelected ? 'top-7' : 'top-1',
THUMB_BADGE_BASE,
THUMB_BADGE_NEUTRAL,
'max-w-[90px]'
)}
title={`Photo owned by ${photo.owner_username}`}
>
<Users className={THUMB_BADGE_ICON} strokeWidth={2.5} />
<span className="truncate">{photo.owner_username}</span>
</div>
)}
{/* BL — color label + rating. Color comes first (left of rating)
* so the swatch reads as a "category dot" prefixing the stars. */}
{(photo.color_label || photo.rating > 0) && (

View File

@@ -0,0 +1,608 @@
import { useEffect, useMemo, useRef, useState } from 'react'
import {
ChevronDown,
ChevronRight,
Folder as FolderIcon,
Upload as UploadIcon,
X,
FileImage,
CheckCircle2,
AlertCircle,
} from 'lucide-react'
import clsx from 'clsx'
import { useQueryClient } from '@tanstack/react-query'
import { uploads, type FolderTreeNode } from '../../services/api'
import { useFolderTreeQuery } from '../../hooks/useFolderTreeQuery'
import { useHeapsQuery, HEAPS_QUERY_KEY } from '../../hooks/useHeapsQuery'
import { FOLDER_TREE_QUERY_KEY } from '../../hooks/useFolderTreeQuery'
import { LIBRARY_STATS_QUERY_KEY } from '../../hooks/useLibraryStatsQuery'
import { toast } from '../ToastContainer'
interface UploadModalProps {
isOpen: boolean
onClose: () => void
/** Optional pre-selected destination. Accepts a Folder id or SourceRoot
* id — the backend resolves source roots to their root Folder row. */
initialFolderId?: string | null
}
interface QueuedFile {
/** Stable key; the browser may give us multiple files with the same
* name from different subfolders, so we key on index + path. */
key: string
file: File
relativePath: string
status: 'pending' | 'uploading' | 'done' | 'error'
progress: number
error?: string
}
const SUPPORTED_EXTENSIONS = new Set([
'.jpg', '.jpeg', '.png', '.tiff', '.tif', '.webp', '.bmp',
'.cr2', '.cr3', '.nef', '.arw', '.raf', '.dng', '.orf', '.rw2', '.pef', '.srw',
'.heic', '.heif',
'.mp4', '.mov', '.avi', '.mkv', '.mts', '.m2ts', '.3gp', '.wmv', '.flv',
])
function extOf(name: string): string {
const i = name.lastIndexOf('.')
return i === -1 ? '' : name.slice(i).toLowerCase()
}
function isSupported(name: string): boolean {
return SUPPORTED_EXTENSIONS.has(extOf(name))
}
const MAX_PARALLEL = 4
/** Walk the tree to collect the ids on the path from root to `targetId`,
* excluding the target itself — used to expand ancestor rows so a
* pre-selected destination is visible. */
function collectAncestors(tree: FolderTreeNode[], targetId: string): string[] {
const path: string[] = []
const walk = (nodes: FolderTreeNode[], chain: string[]): boolean => {
for (const n of nodes) {
if (n.id === targetId) {
path.push(...chain)
return true
}
if (n.children && walk(n.children, [...chain, n.id])) return true
}
return false
}
walk(tree, [])
return path
}
/**
* Upload from desktop. Supports:
* - Dropping files or folders onto the drop zone
* - Picking files with "Select files"
* - Picking a whole folder with "Select folder" (webkitdirectory);
* every file's webkitRelativePath is sent to the backend so sub-
* folder structure is preserved under the chosen destination.
*
* Destination is a Folder row (or a source root, which the backend
* resolves to its root folder). An optional heap can also be chosen —
* uploaded photos are added to that heap in the same request.
*/
export function UploadModal({ isOpen, onClose, initialFolderId }: UploadModalProps) {
const queryClient = useQueryClient()
const { data: folderTree } = useFolderTreeQuery()
const { data: allHeaps = [] } = useHeapsQuery()
const [queue, setQueue] = useState<QueuedFile[]>([])
const [destFolderId, setDestFolderId] = useState<string | null>(null)
const [destHeapId, setDestHeapId] = useState<string | null>(null)
const [isUploading, setIsUploading] = useState(false)
const [dragOver, setDragOver] = useState(false)
const [expandedFolders, setExpandedFolders] = useState<Set<string>>(new Set())
const fileInputRef = useRef<HTMLInputElement>(null)
const dirInputRef = useRef<HTMLInputElement>(null)
const abortRef = useRef<AbortController | null>(null)
// Default destination: caller-provided initialFolderId wins; otherwise
// first source root in the tree. Re-runs when the modal is re-opened
// with a different initial target so the right row is highlighted.
useEffect(() => {
if (!isOpen) return
if (initialFolderId) {
setDestFolderId(initialFolderId)
// Expand every ancestor of the pre-selected folder so the row is
// actually visible in the tree.
if (folderTree) {
const ancestors = collectAncestors(folderTree, initialFolderId)
setExpandedFolders((prev) => new Set([...prev, ...ancestors]))
}
return
}
if (!destFolderId && folderTree && folderTree.length > 0) {
setDestFolderId(folderTree[0].id)
setExpandedFolders(new Set([folderTree[0].id]))
}
}, [isOpen, initialFolderId, folderTree, destFolderId])
// Esc closes (unless mid-upload — don't orphan in-flight requests).
useEffect(() => {
if (!isOpen) return
const handler = (e: KeyboardEvent) => {
if (e.key === 'Escape' && !isUploading) onClose()
}
window.addEventListener('keydown', handler)
return () => window.removeEventListener('keydown', handler)
}, [isOpen, isUploading, onClose])
// Reset transient state on open so a previous session's queue doesn't
// bleed into a fresh one.
useEffect(() => {
if (isOpen) {
setQueue([])
setIsUploading(false)
}
}, [isOpen])
const addFiles = (incoming: File[], relPathFn?: (f: File) => string) => {
const next: QueuedFile[] = []
let skipped = 0
for (const file of incoming) {
const relPath = (relPathFn?.(file) ?? '').replace(/\\/g, '/').replace(/^\/+/, '')
const filename = relPath || file.name
if (!isSupported(filename)) {
skipped++
continue
}
next.push({
key: `${relPath || file.name}::${file.size}::${file.lastModified}::${next.length}`,
file,
relativePath: relPath,
status: 'pending',
progress: 0,
})
}
if (skipped > 0) {
toast.info?.(`Skipped ${skipped} unsupported file${skipped === 1 ? '' : 's'}`)
}
setQueue((prev) => [...prev, ...next])
}
const handleFilePick = (e: React.ChangeEvent<HTMLInputElement>) => {
const files = Array.from(e.target.files ?? [])
addFiles(files, (f) => f.name) // no relative path for single files
e.target.value = '' // allow re-picking the same file
}
const handleDirPick = (e: React.ChangeEvent<HTMLInputElement>) => {
const files = Array.from(e.target.files ?? [])
addFiles(files, (f) => (f as File & { webkitRelativePath?: string }).webkitRelativePath || f.name)
e.target.value = ''
}
// Drag-and-drop handler. We walk the DataTransferItemList recursively
// with webkitGetAsEntry so dropped folders contribute every nested
// file, with relative paths reconstructed from the entry chain.
const handleDrop = async (e: React.DragEvent) => {
e.preventDefault()
setDragOver(false)
const items = Array.from(e.dataTransfer.items)
const collected: { file: File; relativePath: string }[] = []
const walkEntry = (entry: any, pathPrefix: string): Promise<void> => {
return new Promise((resolve) => {
if (!entry) return resolve()
if (entry.isFile) {
entry.file((f: File) => {
collected.push({
file: f,
relativePath: pathPrefix ? `${pathPrefix}/${entry.name}` : '',
})
resolve()
}, () => resolve())
} else if (entry.isDirectory) {
const reader = entry.createReader()
const readBatch = () => {
reader.readEntries(async (entries: any[]) => {
if (!entries.length) return resolve()
const childPrefix = pathPrefix ? `${pathPrefix}/${entry.name}` : entry.name
await Promise.all(entries.map((c) => walkEntry(c, childPrefix)))
// readEntries only returns a batch at a time; loop until empty.
readBatch()
}, () => resolve())
}
readBatch()
} else {
resolve()
}
})
}
await Promise.all(
items.map((it) => {
const entry = (it as DataTransferItem & { webkitGetAsEntry?: () => any }).webkitGetAsEntry?.()
return walkEntry(entry, '')
})
)
if (collected.length === 0) {
// Fallback for browsers without webkitGetAsEntry — use plain files.
const files = Array.from(e.dataTransfer.files)
addFiles(files, (f) => f.name)
return
}
const files = collected.map((c) => c.file)
const pathMap = new Map<File, string>(collected.map((c) => [c.file, c.relativePath]))
addFiles(files, (f) => pathMap.get(f) || f.name)
}
const removeFromQueue = (key: string) => {
setQueue((prev) => prev.filter((q) => q.key !== key))
}
const startUpload = async () => {
if (!destFolderId || queue.length === 0) return
setIsUploading(true)
const ctrl = new AbortController()
abortRef.current = ctrl
// Simple worker-pool: up to MAX_PARALLEL concurrent uploads.
const pending = queue.filter((q) => q.status === 'pending' || q.status === 'error')
let cursor = 0
let successCount = 0
let failCount = 0
const uploadOne = async (item: QueuedFile) => {
setQueue((prev) =>
prev.map((q) => (q.key === item.key ? { ...q, status: 'uploading', progress: 0, error: undefined } : q))
)
try {
await uploads.uploadFile(item.file, destFolderId, {
relativePath: item.relativePath || undefined,
heapId: destHeapId,
signal: ctrl.signal,
onProgress: (loaded, total) => {
const pct = total > 0 ? loaded / total : 0
setQueue((prev) =>
prev.map((q) => (q.key === item.key ? { ...q, progress: pct } : q))
)
},
})
successCount++
setQueue((prev) =>
prev.map((q) => (q.key === item.key ? { ...q, status: 'done', progress: 1 } : q))
)
} catch (err: any) {
failCount++
const msg = err?.response?.data?.detail || err?.message || 'Upload failed'
setQueue((prev) =>
prev.map((q) => (q.key === item.key ? { ...q, status: 'error', error: msg } : q))
)
}
}
const workers: Promise<void>[] = []
for (let i = 0; i < Math.min(MAX_PARALLEL, pending.length); i++) {
workers.push(
(async () => {
while (cursor < pending.length && !ctrl.signal.aborted) {
const idx = cursor++
await uploadOne(pending[idx])
}
})()
)
}
await Promise.all(workers)
setIsUploading(false)
abortRef.current = null
// Refresh everything affected by new photos.
queryClient.invalidateQueries({ queryKey: FOLDER_TREE_QUERY_KEY })
queryClient.invalidateQueries({ queryKey: HEAPS_QUERY_KEY })
queryClient.invalidateQueries({ queryKey: LIBRARY_STATS_QUERY_KEY })
queryClient.invalidateQueries({ queryKey: ['photos'] })
if (successCount > 0) {
toast.success(
`Uploaded ${successCount} file${successCount === 1 ? '' : 's'}`,
failCount > 0 ? `${failCount} failed — see list` : undefined
)
}
if (successCount === 0 && failCount > 0) {
toast.error('Upload failed', `${failCount} file${failCount === 1 ? '' : 's'} errored`)
}
}
const cancelUpload = () => {
abortRef.current?.abort()
}
const totalBytes = useMemo(() => queue.reduce((s, q) => s + q.file.size, 0), [queue])
const uploadedBytes = useMemo(
() => queue.reduce((s, q) => s + q.file.size * (q.status === 'done' ? 1 : q.progress), 0),
[queue]
)
const overallPct = totalBytes > 0 ? Math.round((uploadedBytes / totalBytes) * 100) : 0
if (!isOpen) return null
return (
<div className="fixed inset-0 z-50">
<div
className="absolute inset-0 bg-black/60 backdrop-blur-sm"
onClick={!isUploading ? onClose : undefined}
/>
<div className="absolute left-1/2 top-1/2 -translate-x-1/2 -translate-y-1/2">
<div className="relative z-10 flex max-h-[85vh] w-[760px] flex-col rounded-lg border border-border bg-surface shadow-2xl">
{/* Header */}
<div className="flex items-center justify-between border-b border-border px-5 py-3">
<div className="flex items-center gap-2">
<UploadIcon className="h-4 w-4 text-text-muted" />
<h2 className="text-base font-semibold text-text">Upload photos</h2>
</div>
<button
onClick={onClose}
disabled={isUploading}
className="rounded p-1 text-text-muted hover:bg-surface-2 hover:text-text disabled:opacity-40"
aria-label="Close"
>
<X className="h-4 w-4" />
</button>
</div>
{/* Body */}
<div className="flex min-h-0 flex-1 gap-4 overflow-hidden p-5">
{/* Left: destination picker */}
<div className="flex w-64 flex-col gap-4 overflow-hidden">
<div className="flex flex-col gap-2 overflow-hidden">
<label className="text-xs font-medium uppercase tracking-wide text-text-muted">
Destination folder
</label>
<div className="flex-1 overflow-auto rounded border border-border bg-bg p-1 text-sm">
{folderTree && folderTree.length > 0 ? (
folderTree.map((n) => (
<FolderTreeRow
key={n.id}
node={n}
depth={0}
selectedId={destFolderId}
onSelect={setDestFolderId}
expanded={expandedFolders}
onToggle={(id) =>
setExpandedFolders((prev) => {
const next = new Set(prev)
if (next.has(id)) next.delete(id)
else next.add(id)
return next
})
}
/>
))
) : (
<div className="p-3 text-xs text-text-muted">No folders yet.</div>
)}
</div>
</div>
<div className="flex flex-col gap-2">
<label className="text-xs font-medium uppercase tracking-wide text-text-muted">
Also add to heap (optional)
</label>
<select
value={destHeapId ?? ''}
onChange={(e) => setDestHeapId(e.target.value || null)}
className="rounded border border-border bg-bg px-2 py-1.5 text-sm text-text"
>
<option value=""> none </option>
{allHeaps.map((h) => (
<option key={h.id} value={h.id}>
{h.name}
</option>
))}
</select>
</div>
</div>
{/* Right: drop zone + queue */}
<div className="flex min-w-0 flex-1 flex-col gap-3 overflow-hidden">
<div
onDragOver={(e) => {
e.preventDefault()
setDragOver(true)
}}
onDragLeave={() => setDragOver(false)}
onDrop={handleDrop}
className={clsx(
'flex flex-col items-center justify-center rounded border-2 border-dashed px-4 py-6 text-center transition-colors',
dragOver
? 'border-primary bg-primary/10'
: 'border-border bg-bg'
)}
>
<UploadIcon className="mb-2 h-6 w-6 text-text-muted" />
<div className="text-sm text-text">
Drop files or folders here
</div>
<div className="mt-1 text-xs text-text-muted">
Folder structure is preserved under the destination.
</div>
<div className="mt-3 flex gap-2">
<button
type="button"
onClick={() => fileInputRef.current?.click()}
className="rounded border border-border px-3 py-1 text-xs text-text hover:bg-surface-2"
>
Select files
</button>
<button
type="button"
onClick={() => dirInputRef.current?.click()}
className="rounded border border-border px-3 py-1 text-xs text-text hover:bg-surface-2"
>
Select folder
</button>
</div>
<input
ref={fileInputRef}
type="file"
multiple
accept="image/*,video/*,.heic,.heif,.cr2,.cr3,.nef,.arw,.raf,.dng,.orf,.rw2,.pef,.srw"
className="hidden"
onChange={handleFilePick}
/>
<input
ref={dirInputRef}
type="file"
multiple
// @ts-expect-error — non-standard but supported in Chromium/WebKit
webkitdirectory=""
directory=""
className="hidden"
onChange={handleDirPick}
/>
</div>
{/* Queue */}
<div className="min-h-0 flex-1 overflow-auto rounded border border-border bg-bg">
{queue.length === 0 ? (
<div className="flex h-full items-center justify-center text-xs text-text-muted">
No files added yet.
</div>
) : (
<ul className="divide-y divide-border">
{queue.map((item) => (
<li key={item.key} className="flex items-center gap-2 px-3 py-2 text-sm">
<FileImage className="h-4 w-4 shrink-0 text-text-muted" />
<div className="min-w-0 flex-1">
<div className="truncate text-text">
{item.relativePath || item.file.name}
</div>
{item.status === 'uploading' && (
<div className="mt-1 h-1 w-full overflow-hidden rounded bg-surface-2">
<div
className="h-full bg-primary transition-all"
style={{ width: `${Math.round(item.progress * 100)}%` }}
/>
</div>
)}
{item.status === 'error' && item.error && (
<div className="mt-0.5 truncate text-xs text-reject">{item.error}</div>
)}
</div>
<div className="shrink-0">
{item.status === 'done' && <CheckCircle2 className="h-4 w-4 text-green-500" />}
{item.status === 'error' && <AlertCircle className="h-4 w-4 text-reject" />}
{item.status !== 'done' && !isUploading && (
<button
onClick={() => removeFromQueue(item.key)}
className="rounded p-0.5 text-text-muted hover:bg-surface-2 hover:text-text"
aria-label="Remove"
>
<X className="h-3.5 w-3.5" />
</button>
)}
</div>
</li>
))}
</ul>
)}
</div>
{queue.length > 0 && (
<div className="text-xs text-text-muted">
{queue.length} file{queue.length === 1 ? '' : 's'} {' '}
{(totalBytes / 1024 / 1024).toFixed(1)} MB
{isUploading && `${overallPct}% uploaded`}
</div>
)}
</div>
</div>
{/* Footer */}
<div className="flex items-center justify-end gap-2 border-t border-border px-5 py-3">
<button
onClick={isUploading ? cancelUpload : onClose}
className="rounded border border-border px-3 py-1.5 text-sm text-text hover:bg-surface-2"
>
{isUploading ? 'Cancel' : 'Close'}
</button>
<button
onClick={startUpload}
disabled={isUploading || queue.length === 0 || !destFolderId}
className="flex items-center gap-1.5 rounded bg-primary px-3 py-1.5 text-sm font-medium text-white hover:bg-primary/80 disabled:cursor-not-allowed disabled:opacity-50"
>
<UploadIcon className="h-3.5 w-3.5" />
{isUploading ? 'Uploading…' : `Upload ${queue.length || ''}`.trim()}
</button>
</div>
</div>
</div>
</div>
)
}
interface FolderTreeRowProps {
node: FolderTreeNode
depth: number
selectedId: string | null
onSelect: (id: string) => void
expanded: Set<string>
onToggle: (id: string) => void
}
function FolderTreeRow({
node,
depth,
selectedId,
onSelect,
expanded,
onToggle,
}: FolderTreeRowProps) {
const isExpanded = expanded.has(node.id)
const hasChildren = node.children && node.children.length > 0
const isSelected = selectedId === node.id
return (
<>
<div
className={clsx(
'flex cursor-pointer items-center gap-1 rounded px-1 py-1 text-sm',
isSelected ? 'bg-primary/20 text-text' : 'text-text-muted hover:bg-surface-2 hover:text-text'
)}
style={{ paddingLeft: `${depth * 12 + 4}px` }}
onClick={() => onSelect(node.id)}
>
<button
onClick={(e) => {
e.stopPropagation()
if (hasChildren) onToggle(node.id)
}}
className="flex h-4 w-4 items-center justify-center"
aria-label={isExpanded ? 'Collapse' : 'Expand'}
>
{hasChildren ? (
isExpanded ? (
<ChevronDown className="h-3 w-3" />
) : (
<ChevronRight className="h-3 w-3" />
)
) : null}
</button>
<FolderIcon className="h-3.5 w-3.5 shrink-0" />
<span className="truncate">{node.name}</span>
</div>
{isExpanded &&
node.children?.map((c) => (
<FolderTreeRow
key={c.id}
node={c}
depth={depth + 1}
selectedId={selectedId}
onSelect={onSelect}
expanded={expanded}
onToggle={onToggle}
/>
))}
</>
)
}

View File

@@ -4,7 +4,6 @@ import {
useState,
useEffect,
useCallback,
useRef,
type ReactNode,
} from 'react'
import api from '../services/api'
@@ -47,8 +46,17 @@ function storeToken(token: string) {
localStorage.setItem('access_token', token)
}
function clearToken() {
function getStoredRefreshToken(): string | null {
return localStorage.getItem('refresh_token')
}
function storeRefreshToken(token: string) {
localStorage.setItem('refresh_token', token)
}
function clearTokens() {
localStorage.removeItem('access_token')
localStorage.removeItem('refresh_token')
}
// ── Provider ───────────────────────────────────────────────────────────
@@ -57,50 +65,22 @@ export function AuthProvider({ children }: { children: ReactNode }) {
const [user, setUser] = useState<AuthUser | null>(null)
const [isLoading, setIsLoading] = useState(true)
const [needsSetup, setNeedsSetup] = useState(false)
// Keep refresh token in memory only (not localStorage).
const refreshTokenRef = useRef<string | null>(null)
const refreshTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null)
const isAdmin = user?.role === 'admin'
// Schedule a token refresh ~5 min before expiry.
const scheduleRefresh = useCallback((accessToken: string) => {
try {
const payload = JSON.parse(atob(accessToken.split('.')[1]))
const expiresAt = payload.exp * 1000
const refreshIn = Math.max(expiresAt - Date.now() - 5 * 60 * 1000, 10_000)
if (refreshTimerRef.current) clearTimeout(refreshTimerRef.current)
refreshTimerRef.current = setTimeout(async () => {
if (!refreshTokenRef.current) return
try {
const res = await api.post('/auth/refresh', {
refresh_token: refreshTokenRef.current,
})
const { access_token, refresh_token } = res.data
storeToken(access_token)
refreshTokenRef.current = refresh_token
scheduleRefresh(access_token)
} catch {
// Refresh failed — force re-login.
clearToken()
refreshTokenRef.current = null
setUser(null)
}
}, refreshIn)
} catch {
// Malformed token — ignore.
}
}, [])
const fetchMe = useCallback(async () => {
try {
const res = await api.get('/auth/me')
setUser(res.data)
} catch {
clearToken()
} catch (err: any) {
// Only clear tokens on auth failure (401/403), not network errors.
if (err?.response?.status === 401 || err?.response?.status === 403) {
clearTokens()
setUser(null)
}
// Network errors: leave tokens in place, user stays on login screen
// but can retry without re-entering credentials.
}
}, [])
// Boot: check if setup is needed, then try to restore session.
@@ -120,46 +100,72 @@ export function AuthProvider({ children }: { children: ReactNode }) {
const token = getStoredToken()
if (token) {
await fetchMe()
scheduleRefresh(token)
}
setIsLoading(false)
})()
return () => {
if (refreshTimerRef.current) clearTimeout(refreshTimerRef.current)
}
}, [fetchMe, scheduleRefresh])
}, [fetchMe])
const login = useCallback(
async (username: string, password: string) => {
const res = await api.post('/auth/login', { username, password })
const { access_token, refresh_token } = res.data
storeToken(access_token)
refreshTokenRef.current = refresh_token
scheduleRefresh(access_token)
storeRefreshToken(refresh_token)
await fetchMe()
},
[fetchMe, scheduleRefresh],
[fetchMe],
)
const logout = useCallback(() => {
clearToken()
refreshTokenRef.current = null
if (refreshTimerRef.current) clearTimeout(refreshTimerRef.current)
clearTokens()
setUser(null)
}, [])
const onSetupComplete = useCallback(
async (accessToken: string, refreshToken: string) => {
storeToken(accessToken)
refreshTokenRef.current = refreshToken
storeRefreshToken(refreshToken)
setNeedsSetup(false)
scheduleRefresh(accessToken)
await fetchMe()
},
[fetchMe, scheduleRefresh],
[fetchMe],
)
// Axios interceptor: on 401, try to refresh once using the stored
// refresh token. If that fails, sign out.
useEffect(() => {
const interceptor = api.interceptors.response.use(
(res) => res,
async (error) => {
const original = error.config
if (
error.response?.status === 401 &&
!original._retry &&
!original.url?.includes('/auth/refresh') &&
!original.url?.includes('/auth/login')
) {
original._retry = true
const rt = getStoredRefreshToken()
if (rt) {
try {
const res = await api.post('/auth/refresh', { refresh_token: rt })
const { access_token, refresh_token } = res.data
storeToken(access_token)
storeRefreshToken(refresh_token)
original.headers['Authorization'] = `Bearer ${access_token}`
return api(original)
} catch {
clearTokens()
setUser(null)
}
}
}
return Promise.reject(error)
},
)
return () => api.interceptors.response.eject(interceptor)
}, [])
return (
<AuthContext.Provider
value={{ user, isAdmin, isLoading, needsSetup, login, logout, onSetupComplete }}

View File

@@ -0,0 +1,39 @@
import { useQuery, useQueryClient } from '@tanstack/react-query'
import { features, type FeaturesMap } from '../services/api'
export const FEATURES_QUERY_KEY = ['features'] as const
/** Read the effective feature-flag state (admin override or YAML
* default). Powers conditional rendering of pipeline-dependent UI —
* People view, Tags view, OCR snippets, etc. */
export function useFeaturesQuery() {
return useQuery<FeaturesMap>({
queryKey: FEATURES_QUERY_KEY,
queryFn: features.list,
// Re-read every minute so admin toggles reflect without a page
// reload. The admin tab also invalidates this key on write so the
// refresh can be immediate for the admin who just flipped it.
staleTime: 60_000,
refetchInterval: 60_000,
})
}
export function useIsFeatureEnabled(
name:
| 'vision.enabled'
| 'vision.ocr.enabled'
| 'vision.detector.enabled'
| 'vision.faces.enabled'
| 'vision.classifier.enabled',
): boolean {
const { data } = useFeaturesQuery()
// Default to enabled while loading so we don't flash "feature off"
// during a first-paint fetch. The backend is the source of truth;
// any gated UI that slipped through just returns empty data anyway.
if (!data) return true
return !!data[name]
}
export function invalidateFeaturesQuery(queryClient: ReturnType<typeof useQueryClient>) {
queryClient.invalidateQueries({ queryKey: FEATURES_QUERY_KEY })
}

View File

@@ -0,0 +1,21 @@
import { useQuery } from '@tanstack/react-query'
import { sharing, type SharedHeap, type SharedFolder } from '../services/api'
export const SHARED_HEAPS_KEY = ['sharing', 'heaps'] as const
export const SHARED_FOLDERS_KEY = ['sharing', 'folders'] as const
export function useSharedHeapsQuery() {
return useQuery<SharedHeap[]>({
queryKey: SHARED_HEAPS_KEY,
queryFn: sharing.sharedHeaps,
staleTime: 30_000,
})
}
export function useSharedFoldersQuery() {
return useQuery<SharedFolder[]>({
queryKey: SHARED_FOLDERS_KEY,
queryFn: sharing.sharedFolders,
staleTime: 30_000,
})
}

View File

@@ -625,6 +625,7 @@ export interface LibraryStats {
total_videos: number
total_size: number
total_size_gb: number
source_dirs: string[]
}
// Heaps API
@@ -712,6 +713,135 @@ export const heaps = {
},
}
// Upload API — single file per request so the browser can fan out many
// POSTs in parallel with per-file progress. For folder uploads the
// caller passes each File's webkitRelativePath so the backend can
// materialise the folder structure under the destination.
export const uploads = {
uploadFile: async (
file: File,
destinationFolderId: string,
opts: {
relativePath?: string
heapId?: string | null
onProgress?: (loadedBytes: number, totalBytes: number) => void
signal?: AbortSignal
} = {}
) => {
const form = new FormData()
form.append('file', file)
form.append('destination_folder_id', destinationFolderId)
if (opts.relativePath) form.append('relative_path', opts.relativePath)
if (opts.heapId) form.append('heap_id', opts.heapId)
const response = await api.post('/upload', form, {
headers: { 'Content-Type': 'multipart/form-data' },
signal: opts.signal,
onUploadProgress: (evt) => {
if (opts.onProgress && evt.total) opts.onProgress(evt.loaded, evt.total)
},
})
return response.data as {
photo_id: string
filename: string
folder_id: string
folder_path: string
heap_id: string | null
}
},
}
// Download helpers — build a URL the browser can pull directly via an
// <a href>. The backend accepts `?token=` so an <a> works without a
// custom fetch + save-blob dance; the Authorization header is not
// settable on a plain link click.
export const downloads = {
folderUrl: (folderId: string): string => {
const token = localStorage.getItem('access_token') || ''
return `${API_BASE_URL}/download/folders/${folderId}?token=${encodeURIComponent(token)}`
},
heapUrl: (heapId: string): string => {
const token = localStorage.getItem('access_token') || ''
return `${API_BASE_URL}/download/heaps/${heapId}?token=${encodeURIComponent(token)}`
},
trigger: (url: string) => {
// Kicking off a download via a transient <a> click keeps the
// browser in charge of the file dialog + progress indicator. We
// use target=_blank so the current SPA route isn't replaced if
// the server returns an error mid-stream.
const a = document.createElement('a')
a.href = url
a.rel = 'noopener'
a.target = '_blank'
document.body.appendChild(a)
a.click()
document.body.removeChild(a)
},
}
// Sharing API
export interface SharedHeap {
id: string
name: string
owner_username: string
permission: 'read' | 'write'
photo_count: number
}
export interface SharedFolder {
id: string
name: string
folder_type: 'folder' | 'source_root'
owner_username: string
permission: 'read' | 'write'
photo_count: number
}
export interface ShareInfo {
id: string
shared_with_id: string
shared_with_username: string
permission: string
created_at: string
}
export const sharing = {
// Heap shares
shareHeap: async (heapId: string, username: string, permission: 'read' | 'write' = 'read') => {
const response = await api.post(`/sharing/heaps/${heapId}`, { username, permission })
return response.data
},
heapShares: async (heapId: string): Promise<ShareInfo[]> => {
const response = await api.get(`/sharing/heaps/${heapId}`)
return response.data
},
revokeHeapShare: async (heapId: string, shareId: string) => {
await api.delete(`/sharing/heaps/${heapId}/${shareId}`)
},
sharedHeaps: async (): Promise<SharedHeap[]> => {
const response = await api.get('/sharing/heaps/shared-with-me')
return response.data
},
// Folder shares
shareFolder: async (folderId: string, username: string, permission: 'read' | 'write' = 'read') => {
const response = await api.post(`/sharing/folders/${folderId}`, { username, permission })
return response.data
},
folderShares: async (folderId: string): Promise<ShareInfo[]> => {
const response = await api.get(`/sharing/folders/${folderId}`)
return response.data
},
revokeFolderShare: async (folderId: string, shareId: string) => {
await api.delete(`/sharing/folders/${folderId}/${shareId}`)
},
sharedFolders: async (): Promise<SharedFolder[]> => {
const response = await api.get('/sharing/folders/shared-with-me')
return response.data
},
}
// Tags API
export type TagKind = 'user' | 'object' | 'scene' | 'face_cluster'
@@ -864,6 +994,61 @@ export const admin = {
const response = await api.delete(`/admin/users/${userId}`)
return response.data
},
// --- AI / vision feature flags + manual pipeline triggers -----------
listFeatureFlags: async (): Promise<{ flags: FeatureFlagSnapshot }> => {
const response = await api.get('/admin/feature-flags')
return response.data
},
/** Set ``value`` to toggle; pass ``null`` to clear the override and
* fall back to the YAML default. */
setFeatureFlag: async (
name: string,
value: boolean | null,
): Promise<{ flags: FeatureFlagSnapshot }> => {
const response = await api.patch(`/admin/feature-flags/${encodeURIComponent(name)}`, { value })
return response.data
},
triggerAiBackfill: async (body: {
task?: 'embed' | 'ocr' | 'detect' | 'faces' | 'classify' | null
limit?: number | null
}): Promise<{ status: string; task_id: string }> => {
const response = await api.post('/admin/ai/backfill', body)
return response.data
},
triggerFaceRecluster: async (): Promise<{ status: string; task_id: string }> => {
const response = await api.post('/admin/ai/recluster-faces')
return response.data
},
triggerFullRescan: async (): Promise<{ status: string; task_id: string }> => {
const response = await api.post('/admin/ai/rescan')
return response.data
},
}
export interface FeatureFlagState {
effective: boolean
default: boolean
overridden: boolean
}
export type FeatureFlagSnapshot = Record<string, FeatureFlagState>
export type FeaturesMap = Record<string, boolean>
// Public read of effective feature flags. Available to any signed-in
// user so the frontend can hide sections that depend on a disabled
// pipeline stage (e.g. People when faces are off).
export const features = {
list: async (): Promise<FeaturesMap> => {
const response = await api.get('/features')
return response.data
},
}
export default api

View File

@@ -27,6 +27,8 @@ export interface Photo {
latitude?: number | null
longitude?: number | null
tags?: PhotoTagSummary[]
/** Present when viewing shared content — the username of the photo owner. */
owner_username?: string | null
}
/** Minimal payload returned by GET /api/v1/photos/map — only what the