fix: improvements

PERFORMANCE_IMPROVEMENTS.md

@@ -1,70 +1,108 @@
 # Performance Improvements Plan
 
-## 1. Executive Summary
+## 1. Current State
 
-This document outlines identified performance bottlenecks in the ZUI application and proposes concrete actions to improve render efficiency, state management, and code splitting. The plan is targeted at enabling junior developers to contribute measurable performance gains while maintaining code quality.
+The core rendering and state management architecture is already well-structured for performance:
 
-## 2. Current Performance Issues
+- **Command/reducer store** — all mutations go through a pure reducer; individual selectors can prevent unnecessary re-renders.
+- **Delta-based undo/redo** — only diffs are stored, not full graph snapshots (max 100 entries).
+- **Plugin node registry** — node types are loaded once at app init, not dynamically on each render.
+- **Streaming AI responses** — `POST /api/agent/stream` uses SSE so the UI updates incrementally.
 
-| Issue | Location | Impact | Root Cause |
-|-------|----------|--------|------------|
-| **Large Component Renders** | `frontend/src/app/canvas/CanvasPage.tsx`, `frontend/src/app/recollections/RecollectionPage.tsx` | High CPU usage, frame drops | These components render thousands of nodes without memoization; state updates trigger full re-renders. |
-| **Unmemoized Expensive Calculations** | Various utility functions in `src/lib/*` (e.g., graph layout calculations) | Delayed response to user interactions | Calculations recomputed on every render cycle. |
-| **Frequent State Updates** | `canvasStore` mutations in response to mouse/touch events | Unnecessary re-renders of unrelated nodes | State updates not granular; multiple mutations in quick succession. |
-| **Lack of Code Splitting** | Heavy modules imported globally (e.g., rendering views, graph utilities) | Initial bundle size > 2MB, slow load | All modules loaded upfront even if not used. |
-| **Inefficient List Rendering** | Lists of nodes in sidebar components | Scrolling lag | No virtualization; all items rendered simultaneously. |
-| **Repeated API Calls** | Agent service calls without proper caching | Latency spikes | Calls bypass `cacheRepository` in some paths. |
+The sections below identify remaining bottlenecks and concrete next steps.
+
+---
+
+## 2. Known Bottlenecks
+
+| # | Issue | Location | Impact | Root Cause |
+| --- | --- | --- | --- | --- |
+| 1 | Unguarded re-renders on canvas | `CanvasPage.tsx` | Frame drops on large graphs | Components not subscribed to granular store slices |
+| 2 | Nunjucks template resolution on every render | `rendering.ts` resolve step | Slow Config node updates | No memoization of template output keyed to input hash |
+| 3 | Kroki SVG requests not deduplicated | `rendering.ts` render step | Redundant network calls | No in-flight request deduplication or client-side cache |
+| 4 | Sidebar tree renders all items | `KosmosPage` recollection tree | Scrolling lag with many workspaces | No list virtualization |
+| 5 | Backend cache not wired to agent routes | `agentRoutes.ts` | Repeated identical LLM calls | `InMemoryCache` and `rateLimiter` exist but are unused |
+| 6 | Full graph serialized to localStorage on every change | `useGraphStateWithHistory` | Storage I/O on every keypress | No debounce on the persistence write |
+| 7 | Initial bundle size | Vite build | Slow first load | Heavy deps (BlockNote, React Flow, Nunjucks) loaded eagerly |
+
+---
 
 ## 3. Recommended Improvements
 
-### 3.1 Component Refactoring
+### 3.1 Granular Store Subscriptions
 
-- **Split `CanvasPage` and `RecollectionPage`** into smaller, feature‑specific sub‑components.
-- **Extract pure logic** (e.g., node layout calculations) into standalone utility functions with `useMemo`.
-- **Apply `React.memo`** to pure presentational components that receive static props.
+Zustand supports slice-level subscriptions. Node components should select only their own data slice:
 
-### 3.2 State Management Optimization
+```ts
+// Instead of subscribing to the entire graph:
+const node = useCanvasStore(s => s.graph.nodes.find(n => n.id === id))
+```
 
-- **Granular State Updates**: Use `canvasStore` selectors to update only the affected node slices.
-- **Batch Updates**: Wrap multiple mutations in `runWithTiming` or `unstable_batchedUpdates` to reduce render cycles.
+This prevents all nodes from re-rendering when a single node changes.
 
-### 3.3 Memoization & Lazy Loading
+### 3.2 Memoize Template Resolution
 
-- **Memoize Callbacks**: Replace inline event handlers with `useCallback` references stored in context or hooks.
-- **Dynamic Imports**: Use `import()` for heavy modules (e.g., `RenderingNode`, `AnimatedEdge`) to split the bundle.
-- **Virtualized Lists**: Integrate `react-window` or `react-virtualized` for large node lists in sidebars.
+Cache the Nunjucks resolution output keyed to a hash of the template source plus variable inputs. Invalidate only when those inputs change:
 
-### 3.4 Code Splitting & Bundle Optimization
+```ts
+const resolved = useMemo(
+  () => resolveTemplate(template, variables),
+  [templateHash, variableHash]
+)
+```
 
-- **Remove Unused Dependencies**: Audit `package.json` for deprecated libraries.
-- **Enable `vite-plugin-dynamic-import`** for on‑demand loading of route‑specific components.
-- **Compress Assets**: Configure `gzip`/`brotli` in `nginx.conf` for large SVG and texture assets.
+### 3.3 Deduplicate Kroki Requests
 
-### 3.5 Caching Strategy Enhancements
+Add a simple in-flight map in the render step: if a request for the same PlantUML source is already pending, reuse its promise. Cache successful responses keyed to the source string with a short TTL (e.g. 5 minutes).
 
-- **Centralize Caching**: Ensure all external API calls route through `cacheRepository`.
-- **Add TTL** to cached responses to avoid stale data while still reducing repeat calls.
+### 3.4 Wire Backend Cache and Rate Limiter
 
-### 3.6 Testing & Verification
+`InMemoryCache` and `rateLimiter` middleware are implemented in `backend/src/`. Connect them to `agentRoutes.ts`:
 
-- **Performance Tests**: Add Vitest benchmarks for render times using `performance.now()`.
-- **Profile with React DevTools**: Capture flame graphs before and after each optimization.
-- **CI Gate**: Enforce that PRs must include a performance regression test if changes affect rendering.
+1. Add cache lookup before calling the AI service.
+2. Store the response on cache miss.
+3. Apply rate limiting per IP to prevent abuse.
 
-## 4. Contribution Path for Junior Developers
+### 3.5 Debounce localStorage Writes
 
-1. **Familiarize** with the `canvasStore` architecture and the `CanvasPage` component structure.
-2. **Pick** a low‑risk optimization (e.g., memoizing a utility function).
-3. **Implement** the change, add JSDoc comments, and write a simple benchmark.
-4. **Submit** a PR with:
-   - Description of the performance gain.
-   - Updated tests/benchmarks.
-   - Documentation in `PERFORMANCE_IMPROVEMENTS.md`.
+Wrap the graph persistence call in a debounce (e.g., 300 ms) to avoid a write on every keystroke or node drag. The delta-based history already computes minimal diffs; the bottleneck is the frequency of writes.
 
-## 5. Success Metrics
+### 3.6 Virtualize the Sidebar Tree
 
-- **Target**: Reduce average frame time from ~16 ms to < 10 ms for `CanvasPage`.
-- **Bundle Size**: Decrease initial load by ≥ 15 %.
-- **API Latency**: Cut repeated call overhead by ≥ 30 %.
+Integrate `react-arborist` (already installed) with virtualization enabled for the recollection sidebar when item count exceeds a threshold (~50).
 
-*This plan is living; subsequent sections will track progress and update targets.*
+### 3.7 Code Split Heavy Routes
+
+Add lazy imports for the three heavy route components so the initial bundle only loads what the user navigates to:
+
+```ts
+const FluxRoute    = lazy(() => import('./app/recollections/flux/FluxRoute'))
+const LogosPage    = lazy(() => import('./app/recollections/logos/LogosPage'))
+const KatalogosPage = lazy(() => import('./app/recollections/katalogos/KatalogosPage'))
+```
+
+### 3.8 Enable Brotli Compression in Nginx
+
+Add brotli/gzip compression to `frontend/nginx.conf` for JS, CSS, and SVG assets. This can cut transfer size by 60–70% for the JS bundle.
+
+---
+
+## 4. Success Metrics
+
+| Metric | Current (estimated) | Target |
+| --- | --- | --- |
+| Frame time on 50-node canvas | ~16 ms | < 10 ms |
+| Initial JS bundle (gzipped) | ~800 KB | < 600 KB |
+| Repeated identical LLM calls | uncached | 0 network round-trips |
+| localStorage write frequency | every change | debounced 300 ms |
+
+---
+
+## 5. Contribution Path
+
+1. Read [ARCHITECTURE.md](ARCHITECTURE.md) to understand the module you're optimizing.
+2. Pick one item from section 2.
+3. Add a Vitest benchmark (`performance.now()` before/after) alongside your change.
+4. Submit a PR with the benchmark results in the description and update this file's "Current" column.
+
+*This plan is a living document; update the metrics table when improvements land.*