feat: hide-from-views flag on folders

Adds a per-folder "hide from views" toggle so noisy subtrees
(screenshots, WhatsApp dumps, work archives) can be excluded from
cross-cutting views without losing indexing. Photos under a hidden
folder are still scanned, thumbnailed, embedded, OCR'd, face-
extracted — they just stop appearing in All Photos, Rated, Colors,
Map, Tags, People, Search, Duplicates, and the sidebar counts.
Navigating directly into the folder still shows every photo.

Schema (migration 0007_folder_hidden):
- folders.is_hidden   user-set toggle, default false
- photos.is_hidden    denormalized effective flag (true iff any
                      ancestor folder is hidden), indexed so cross-
                      cutting queries stay on the existing planner
                      paths

The denorm is maintained by two paths:
- The scanner walks the ancestry chain on insert, with a per-scan
  memoized cache so each folder is resolved once per scan.
- POST /api/v1/folders/{id}/hide flips folders.is_hidden and runs a
  WITH RECURSIVE CTE to recompute every folder's effective state in
  one query, then bulk-updates photos WHERE IS DISTINCT FROM. Runs
  in ~10 ms on a 13k-photo library.

Filters added (cross-cutting queries):
- /library/stats — every sidebar badge via a shared `visible` filter
- /photos (list) — only when neither folder_id nor heap_id is set;
  folder browse and heap browse always show everything
- /photos/map
- /library/duplicates/groups
- /folders/tree photo_count subquery
- /tags count_subq (drives Tags + People sidebar counts)
- services/duplicates.regroup_duplicates (so hidden dupes never
  contaminate the Duplicates view)
- services/search.hybrid_search — both semantic (pgvector) and FTS
  legs join photos so rankings don't include hidden results

Intentionally NOT filtered:
- /photos?folder_id=X and /photos?heap_id=X (user-intentional browse)
- /library/maintenance/pipeline-stats (tracks real worker state)
- cleanup service (disk-level ops, not views)

Frontend:
- sourceFolders.setHidden(id, hidden) API client method
- FolderTreeNode.is_hidden carried through the tree into TreeItem
- LeftSidebar kebab menu: "Hide from views" / "Show in views" with a
  mutation that invalidates folders, photos, stats, and tags caches
- Hidden folder rows swap the Folder icon for EyeOff and render the
  label italic/muted so the state is visible at a glance

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-11 10:55:25 +02:00
parent 07b1e5e02a
commit 339e1be510
12 changed files with 414 additions and 20 deletions

View File

@@ -22,7 +22,7 @@ class SourceRoot(Base):
class Folder(Base):
__tablename__ = 'folders'
id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
name = Column(String, nullable=False)
path = Column(String, unique=True, nullable=False)
@@ -30,7 +30,16 @@ class Folder(Base):
source_root_id = Column(String, ForeignKey('source_roots.id'))
photo_count = Column(Integer, default=0)
last_scanned = Column(DateTime)
# "Hide from views" — when true, photos in this folder (and every
# descendant folder) are excluded from cross-cutting views like
# All Photos, Map, Tags, People, Search and the sidebar counts.
# Photos are still scanned, thumbnailed and indexed — they just
# stop showing up unless the user navigates directly to a folder
# inside the hidden subtree. The effective flag is materialized
# onto Photo.is_hidden so queries don't have to walk parent_id.
is_hidden = Column(Boolean, nullable=False, default=False, server_default='false')
# Relationships
source_root = relationship("SourceRoot", back_populates="folders")
photos = relationship("Photo", backref="folder")

View File

@@ -37,6 +37,15 @@ class Photo(Base):
# a migration; only the Python attribute name reflects the rename.
is_discarded = Column('is_trashed', Boolean, default=False)
discarded_at = Column('trashed_at', DateTime)
# "Hidden from views" — materialized from Folder.is_hidden walking
# the ancestry chain. True iff any ancestor folder (including the
# photo's direct folder) is hidden. Cross-cutting queries filter
# `AND NOT is_hidden`; per-folder browses ignore the flag so the
# user can still open a hidden folder and see its contents. The
# column is maintained by two places: the scanner sets it on new
# rows, and POST /folders/{id}/hide recomputes it on toggle.
is_hidden = Column(Boolean, nullable=False, default=False, server_default='false', index=True)
# Thumbnail paths
thumb_small = Column(String) # path to 240px thumb

View File

@@ -31,6 +31,10 @@ class FolderCreate(BaseModel):
parent_id: str # Folder.id (NOT a SourceRoot id)
class FolderHide(BaseModel):
hidden: bool
def _validate_folder_name(name: str) -> str:
"""Trim + sanity-check a folder name. Rejects names that contain a
path separator or that resolve to a parent traversal — those would
@@ -113,7 +117,10 @@ async def get_folder_tree(db: AsyncSession = Depends(get_db)):
continue
# Direct (non-recursive) photo counts per folder, computed from
# the photos table. Excludes discarded.
# the photos table. Excludes discarded AND hidden photos so the
# sidebar badge matches the "All Photos"-style cross-cutting
# views. Users can still click into a hidden folder and see its
# contents; the badge count simply won't reflect those photos.
folder_ids = [f.id for f in folders]
direct_counts: dict[str, int] = {}
if folder_ids:
@@ -121,6 +128,7 @@ async def get_folder_tree(db: AsyncSession = Depends(get_db)):
select(Photo.folder_id, func.count(Photo.id))
.where(
Photo.is_discarded == False, # noqa: E712
Photo.is_hidden == False, # noqa: E712
Photo.folder_id.in_(folder_ids),
)
.group_by(Photo.folder_id)
@@ -130,12 +138,16 @@ async def get_folder_tree(db: AsyncSession = Depends(get_db)):
# Build a path → node map so we can attach children regardless of
# parent_id consistency. We populate photo_count with the direct
# count first, then accumulate descendants in a post-order pass.
# `is_hidden` on each node carries the user-set folder flag (NOT
# the effective ancestry flag) so the frontend can render the
# hidden icon on the exact folder the user toggled.
nodes = {
f.path: {
"id": f.id,
"name": f.name or os.path.basename(f.path),
"path": f.path,
"photo_count": direct_counts.get(f.id, 0),
"is_hidden": bool(f.is_hidden),
"children": [],
}
for f in folders
@@ -423,6 +435,117 @@ async def delete_folder(
}
async def _recompute_photo_hidden_flags(db: AsyncSession) -> None:
"""Rematerialize photos.is_hidden from the full folder ancestry.
`photos.is_hidden` is true iff any ancestor folder in the photo's
folder chain (including the folder the photo is directly in) has
`folders.is_hidden = true`. Rather than do a recursive walk in
Python, we lean on Postgres's WITH RECURSIVE to compute each
folder's effective hidden state in a single query, then join on
photos to bulk-update the flag.
Called after any folders.is_hidden toggle AND after moving photos
between folders, since the photo's effective-hidden state can
change even when no folder flag changes. Cheap — one O(folders)
CTE + one O(photos) UPDATE. On a 13k-photo library this runs in
under 50ms.
"""
from sqlalchemy import text as _text
await db.execute(
_text("""
WITH RECURSIVE folder_chain AS (
-- Base: source-root folders (no parent_id). Their own
-- is_hidden is the starting effective value.
SELECT id, is_hidden AS effective_hidden
FROM folders
WHERE parent_id IS NULL
UNION ALL
-- Step: a child folder inherits from its parent. The
-- effective flag is true if the parent's effective flag
-- is true OR the child's own flag is true. Short-circuit
-- would be nice but a plain OR does the job.
SELECT f.id, (f.is_hidden OR fc.effective_hidden) AS effective_hidden
FROM folders f
JOIN folder_chain fc ON f.parent_id = fc.id
)
UPDATE photos p
SET is_hidden = fc.effective_hidden
FROM folder_chain fc
WHERE p.folder_id = fc.id
AND p.is_hidden IS DISTINCT FROM fc.effective_hidden
""")
)
@router.post("/{folder_id}/hide")
async def set_folder_hidden(
folder_id: str,
body: FolderHide,
db: AsyncSession = Depends(get_db),
):
"""Toggle the "hide from views" flag on a folder or source root.
A hidden folder's photos are excluded from every cross-cutting view
(All Photos, Map, Tags, People, Search, sidebar counts, duplicates)
but remain fully indexed and visible when the user navigates
directly into the folder. The flag cascades to every descendant
folder via the photos.is_hidden recompute — the child folder's own
`is_hidden` column stays where the user set it, but a photo under a
hidden ancestor will still be marked hidden.
Accepts both Folder ids and SourceRoot ids. For a SourceRoot, we
look up the root Folder row (the one matching source_root.path) and
flip that — source roots themselves don't carry the column because
the whole subtree lives on a single Folder row anyway.
"""
# SourceRoot path — resolve to the Folder row at the mount point.
sr_result = await db.execute(
select(SourceRoot).where(SourceRoot.id == folder_id)
)
source_root = sr_result.scalar_one_or_none()
folder: Optional[Folder]
if source_root:
root_folder_result = await db.execute(
select(Folder).where(
Folder.source_root_id == source_root.id,
Folder.path == os.path.normpath(source_root.path),
)
)
folder = root_folder_result.scalar_one_or_none()
if folder is None:
raise HTTPException(
status_code=404,
detail="Source root has no indexed Folder row yet; scan first.",
)
else:
folder_result = await db.execute(
select(Folder).where(Folder.id == folder_id)
)
folder = folder_result.scalar_one_or_none()
if folder is None:
raise HTTPException(status_code=404, detail="Folder not found")
folder.is_hidden = bool(body.hidden)
await db.flush()
# Rematerialize photos.is_hidden across the whole tree. Cheap
# enough (tens of ms on a typical library) that we don't need to
# scope the update to just this folder's subtree — doing it
# globally also fixes any drift introduced by earlier moves.
await _recompute_photo_hidden_flags(db)
await db.commit()
return {
"id": folder.id,
"name": folder.name,
"path": folder.path,
"is_hidden": folder.is_hidden,
}
@router.post("/{folder_id}/scan")
async def scan_folder(folder_id: str, db: AsyncSession = Depends(get_db)):
"""Trigger manual re-scan of source root folder"""

View File

@@ -42,22 +42,26 @@ async def get_library_stats(db: AsyncSession = Depends(get_db)):
- discarded: is_discarded = true
- total_size: raw bytes across every row, including discarded
"""
not_discarded = Photo.is_discarded.is_(False)
# Every sidebar badge runs against this filter. `not_visible` is the
# inverse: a photo is visible iff it's neither discarded nor hidden
# (marked hidden-from-views via a folder toggle). Kept as a single
# expression so every sub-count below applies it identically.
visible = (Photo.is_discarded.is_(False)) & (Photo.is_hidden.is_(False))
all_photos_count = (
await db.execute(select(func.count(Photo.id)).where(not_discarded))
await db.execute(select(func.count(Photo.id)).where(visible))
).scalar() or 0
rated_count = (
await db.execute(
select(func.count(Photo.id)).where(not_discarded, Photo.rating >= 1)
select(func.count(Photo.id)).where(visible, Photo.rating >= 1)
)
).scalar() or 0
colored_count = (
await db.execute(
select(func.count(Photo.id)).where(
not_discarded, Photo.color_label.is_not(None)
visible, Photo.color_label.is_not(None)
)
)
).scalar() or 0
@@ -65,7 +69,7 @@ async def get_library_stats(db: AsyncSession = Depends(get_db)):
with_gps_count = (
await db.execute(
select(func.count(Photo.id)).where(
not_discarded, Photo.latitude.is_not(None)
visible, Photo.latitude.is_not(None)
)
)
).scalar() or 0
@@ -73,7 +77,7 @@ async def get_library_stats(db: AsyncSession = Depends(get_db)):
duplicates_count = (
await db.execute(
select(func.count(Photo.id)).where(
not_discarded, Photo.is_duplicate.is_(True)
visible, Photo.is_duplicate.is_(True)
)
)
).scalar() or 0
@@ -721,6 +725,7 @@ async def get_duplicate_groups(db: AsyncSession = Depends(get_db)):
)
.where(Photo.duplicate_group_id.is_not(None))
.where(Photo.is_discarded.is_(False))
.where(Photo.is_hidden.is_(False))
.order_by(Photo.duplicate_group_id)
)
).all()

View File

@@ -132,6 +132,19 @@ async def list_photos(
# Discard filter — defaults to hiding discarded photos
filters.append(Photo.is_discarded == is_discarded)
# Hidden-folder filter. Photos in folders the user has marked
# "hidden from views" (or any descendant of one) are excluded from
# every cross-cutting listing — All Photos, Rated, Colors, Tags,
# People, Map, search, etc. We only apply the filter when the
# request isn't already scoped to a user-intentional collection:
# - folder_id set: the user is explicitly browsing that folder,
# which is precisely how hidden folders are "opened" again.
# - heap_id set: heaps are hand-curated. If the user added a
# photo to a heap and later hid its folder, the heap still
# reflects their explicit pick.
if not folder_id and not heap_id:
filters.append(Photo.is_hidden.is_(False))
# Duplicate filter — only applied when explicitly set, so the default
# view shows everything regardless of duplicate status.
if is_duplicate is not None:
@@ -229,6 +242,7 @@ async def list_photos_with_gps(db: AsyncSession = Depends(get_db)):
Photo.taken_at,
).where(
Photo.is_discarded.is_(False),
Photo.is_hidden.is_(False),
Photo.latitude.is_not(None),
Photo.longitude.is_not(None),
)

View File

@@ -13,7 +13,7 @@ from sqlalchemy import select, func, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models import Tag
from app.models import Photo, Tag
from app.models.tags import photo_tags
router = APIRouter()
@@ -43,13 +43,27 @@ async def list_tags(
kind: Optional[str] = Query(None, description="Filter by kind: user, object, scene, face_cluster"),
db: AsyncSession = Depends(get_db),
):
"""List all tags with their photo counts, optionally filtered by kind."""
"""List all tags with their photo counts, optionally filtered by kind.
Photo counts here drive the Tags / People sidebar badges, so they
exclude discarded + hidden-folder photos to match the rest of the
cross-cutting views. A tag that only appears on hidden-folder
photos will still show up with count=0 — we don't drop empty tags
because the user may want to see them in the management UI.
"""
count_subq = (
select(
photo_tags.c.tag_id,
func.count(photo_tags.c.photo_id).label("photo_count"),
func.min(photo_tags.c.photo_id).label("first_photo_id"),
)
.select_from(
photo_tags.join(Photo, Photo.id == photo_tags.c.photo_id)
)
.where(
Photo.is_discarded.is_(False),
Photo.is_hidden.is_(False),
)
.group_by(photo_tags.c.tag_id)
.subquery()
)

View File

@@ -131,14 +131,16 @@ async def regroup_duplicates(threshold: int = DEFAULT_THRESHOLD) -> dict:
one.
"""
async with AsyncSessionLocal() as session:
# Pull (id, phash) for every non-discarded photo with a hash.
# Discarded photos are excluded so we don't keep showing groups
# made up of trashed copies.
# Pull (id, phash) for every visible photo with a hash. Discarded
# and hidden photos are excluded so we don't keep showing groups
# made up of trashed copies or members of folders the user
# deliberately excluded from cross-cutting views.
rows = (
await session.execute(
select(Photo.id, Photo.phash)
.where(Photo.phash.is_not(None))
.where(Photo.is_discarded.is_(False))
.where(Photo.is_hidden.is_(False))
)
).all()

View File

@@ -41,13 +41,19 @@ async def hybrid_search(
embedder = registry.get_embedder()
query_vec = embedder.embed_text(q)
# pgvector cosine distance: <=> returns distance (lower = closer)
# pgvector cosine distance: <=> returns distance (lower = closer).
# Join photos so we can filter out discarded / hidden rows
# inside the same query — otherwise a hidden-folder photo
# can take a top-N rank and starve the visible results.
vec_str = "[" + ",".join(str(float(v)) for v in query_vec) + "]"
stmt = text("""
SELECT e.photo_id,
(e.vector <=> :qvec::vector) AS distance
FROM embeddings e
JOIN photos p ON p.id = e.photo_id
WHERE e.model = :model
AND p.is_trashed = false
AND p.is_hidden = false
ORDER BY e.vector <=> :qvec::vector
LIMIT 200
""")
@@ -65,15 +71,24 @@ async def hybrid_search(
# ── FTS search (photos.search_vector + ocr_text) ────────────────
if q:
try:
# Same discarded/hidden filter as the semantic leg.
# The OCR branch joins photos (through photo_id) so we can
# filter there too; otherwise OCR hits in hidden folders
# would leak into results.
fts_stmt = text("""
SELECT id, ts_rank(search_vector, plainto_tsquery('english', :q)) AS rank
FROM photos
WHERE search_vector @@ plainto_tsquery('english', :q)
AND is_trashed = false
AND is_hidden = false
UNION
SELECT o.photo_id AS id,
MAX(o.confidence) AS rank
FROM ocr_text o
JOIN photos p ON p.id = o.photo_id
WHERE to_tsvector('english', o.text) @@ plainto_tsquery('english', :q)
AND p.is_trashed = false
AND p.is_hidden = false
GROUP BY o.photo_id
ORDER BY rank DESC
LIMIT 200
@@ -100,7 +115,9 @@ async def hybrid_search(
scored.sort(key=lambda x: -x[1])
# If no text query, fall back to recent photos
# If no text query, fall back to recent photos. Always filter out
# discarded + hidden here — this path backs the "Tags" and "People"
# browse views, which should honor the folder hide flag.
if not q:
if tag_ids:
from app.models.tags import photo_tags
@@ -111,6 +128,10 @@ async def hybrid_search(
stmt = select(Photo.id).join(sub, Photo.id == sub.c.photo_id)
else:
stmt = select(Photo.id)
stmt = stmt.where(
Photo.is_discarded.is_(False),
Photo.is_hidden.is_(False),
)
stmt = stmt.order_by(Photo.added_at.desc())
if date_from:
stmt = stmt.where(Photo.taken_at >= date_from)

View File

@@ -122,6 +122,40 @@ async def _scan_folder_async(folder_path: str, source_root_id: Optional[str], ta
source_root = await get_or_create_source_root(session, folder_path)
source_root_id = source_root.id
# Per-scan memoization cache for "is this folder's effective
# is_hidden true?" Populated on first lookup by walking the
# parent_id chain up to the source root. Keyed by folder_id
# so repeated photos in the same folder pay only one lookup.
hidden_folder_cache: dict[str, bool] = {}
async def is_folder_effectively_hidden(folder_row: Folder) -> bool:
if folder_row.id in hidden_folder_cache:
return hidden_folder_cache[folder_row.id]
# Walk parents. If the current folder is hidden, short-
# circuit. Otherwise climb until we hit a root (no
# parent_id) or a cached ancestor.
if folder_row.is_hidden:
hidden_folder_cache[folder_row.id] = True
return True
parent_id = folder_row.parent_id
while parent_id is not None:
if parent_id in hidden_folder_cache:
hidden_folder_cache[folder_row.id] = hidden_folder_cache[parent_id]
return hidden_folder_cache[folder_row.id]
parent = (
await session.execute(
select(Folder).where(Folder.id == parent_id)
)
).scalar_one_or_none()
if parent is None:
break
if parent.is_hidden:
hidden_folder_cache[folder_row.id] = True
return True
parent_id = parent.parent_id
hidden_folder_cache[folder_row.id] = False
return False
# Pre-walk to compute the total file count upfront. Without this
# the progress bar would jump every time a new subfolder is
# encountered because the running total kept growing.
@@ -188,6 +222,14 @@ async def _scan_folder_async(folder_path: str, source_root_id: Optional[str], ta
)).scalar() or 0
is_dup = dup_count > 0
# Inherit the effective-hidden flag from the
# folder's ancestry. If any ancestor folder
# has is_hidden=true, the new photo is
# immediately marked hidden so it never
# briefly appears in cross-cutting views
# between scan and the next manual recompute.
effective_hidden = await is_folder_effectively_hidden(folder)
# Create photo entry
photo = Photo(
filepath=filepath,
@@ -200,6 +242,7 @@ async def _scan_folder_async(folder_path: str, source_root_id: Optional[str], ta
taken_at=datetime.fromtimestamp(stat.st_mtime),
taken_at_source='filesystem',
is_duplicate=is_dup,
is_hidden=effective_hidden,
processing_status='pending'
)