diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..ef3f6f7 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,105 @@ +# Architecture Overview + +## 1. Executive Summary + +This document provides a high-level overview of the ZUI application's architecture, focusing on component organization, data flow, technology stack, and contribution guidelines for junior developers and AI agents. + +## 2. Technology Stack + +- **Frontend**: React 18, TypeScript, Vite, Tailwind CSS, Shadcn UI, Zustand (canvasStore), custom hooks, and canvas rendering engine. +- **Backend**: Node.js 18, TypeScript, Express-like routing, Docker, and environment variable configuration. +- **Database/Storage**: In-memory state management with optional persistence via cacheRepository. +- **Testing**: Vitest, React Testing Library, and Jest-like utilities. + +## 3. High-Level Structure + +The workspace is divided into two primary directories: + +- **frontend/**: Contains the React application, component library, pages, and styling. +- **backend/**: Contains server-side code, services, repositories, models, and middleware. + +### 3.1 Frontend Modules + +| Module | Purpose | Key Files | +|--------|---------|-----------| +| **canvas** | Core graph/canvas rendering, nodes, edges, and interactive graph features. | `src/app/canvas/*`, `src/components/graph/*`, `src/components/nodes/*` | +| **recollections** | Collection management, browsing, editing, and related UI. | `src/app/recollections/*`, `src/components/ui/*` | +| **kosmos** | Knowledge organization system, settings, and user preferences. | `src/app/kosmos/*` | +| **layout** | Layout and navigation components shared across pages. | `src/app/recollections/layout/*` | +| **components** | Reusable UI components (buttons, dialogs, avatars, etc.). | `src/components/ui/*` | +| **hooks** | Custom React hooks for state, effects, and utilities. | `src/hooks/*` | +| **lib** | Shared utilities, types, and low-level helpers. | `src/lib/*` | + +### 3.2 Backend Modules + +| Module | Purpose | Key Files | +|--------|---------|-----------| +| **services** | Business logic, external API interactions, and complex computations. | `backend/src/services/*` | +| **repositories** | Data access abstraction, caching, and persistence. | `backend/src/repositories/*` | +| **models** | TypeScript interfaces and type definitions for domain objects. | `backend/src/models/*` | +| **routes** | HTTP route definitions and controllers. | `backend/src/routes/*` | +| **middleware** | Request processing pipeline (e.g., rate limiting, authentication). | `backend/src/middleware/*` | +| **index.ts** | Application entry point that composes services, routes, and middleware. | `backend/src/index.ts` | + +## 4. Data Flow + +1. **User Interaction** (React components) → dispatches actions → updates local state (Zustand) → triggers re-render. +2. **State Changes** may trigger async calls to **services** (frontend) → which call **backend APIs** → responses are cached via **cacheRepository**. +3. **Backend** processes requests via **routes**, applies **middleware**, interacts with **repositories** and **models**, and returns JSON responses. +4. **Caching** layer reduces repeated expensive computations or DB queries, improving performance. + +## 5. Component Interaction Patterns + +- **Container Components** (e.g., `CanvasPage`, `RecollectionPage`) manage layout and state provision. +- **Presentational Components** (e.g., `TreeBrowser`, `AgentNode`) focus on UI rendering only. +- **Custom Hooks** encapsulate logic (e.g., `useResizeHeight`, `useStreamingContent`) and are reused across components. +- **Context Providers** (`KosmosContext`, `RecollectionSidebarContext`) supply global state to deeply nested component trees. + +## 6. Extensibility & Plug‑in Architecture + +- **Node Types** are registered via `nodeRegistry.ts` and extended through descriptor files. +- **New node types** can be added by creating a descriptor, implementing rendering logic, and registering it in `registerBuiltinNodes.tsx`. +- **Extensibility points** are documented in `NODE_TYPE_EXTENSIBILITY_PROPOSAL.md`. + +## 7. Performance Considerations + +- **Large Render Trees**: `CanvasPage` and `RecollectionPage` handle thousands of nodes; memoization (`useMemo`, `useCallback`) and lazy loading are essential. +- **State Updates**: Prefer granular state updates in `canvasStore` to avoid full tree re-renders. +- **Code Splitting**: Dynamic imports are used for heavy modules (e.g., rendering views, graph utilities). + +## 8. Contribution Workflow + +1. Fork the repository and clone the workspace. +2. Install dependencies: `pnpm install` (or `npm ci`). +3. Run the dev server: `pnpm dev` (frontend) and `pnpm start` (backend). +4. Create a feature branch: `git checkout -b feat/your-feature`. +5. Follow the **Coding Standards** (see `CONTRIBUTING.md` for details). +6. Submit a Pull Request with: + - Descriptive title. + - Linked issue(s). + - Updated tests (if applicable). + - Documentation updates (if relevant). +7. Code Review Checklist: + - [ ] Readability: clear naming, minimal nesting. + - [ ] Maintainability: extracted utilities, JSDoc, TypeScript typings. + - [ ] Performance: no unnecessary re-renders, proper memoization. + - [ ] Accessibility: semantic HTML, ARIA attributes. + - [ ] Security: no hardcoded secrets, proper input validation. + +## 9. Coding Standards + +- **TypeScript**: Strict mode (`strict`, `noImplicitAny`, `noUnusedVars`). +- **Naming**: PascalCase for components, camelCase for functions/variables, UPPER_SNAKE_CASE for constants. +- **File Structure**: Feature‑first (`features//`) or domain‑grouped (`src/app/recollections/...`). +- **Documentation**: JSDoc for all public APIs, TSDoc for types, and inline comments for complex logic. +- **Formatting**: Prettier configured via `frontend/prettier.config.cjs`. + +## 10. Quick Reference for AI Agents + +- **Key Entry Points**: `frontend/src/main.tsx`, `backend/src/index.ts`. +- **State Management**: `src/lib/graph/state.ts`, `src/app/canvas/canvasStore.*`. +- **Routing**: `backend/src/routes/agentRoutes.ts`. +- **Caching**: `backend/src/repositories/cacheRepository.ts`. +- **Performance Hotspots**: `CanvasPage.tsx`, `RecollectionPage.tsx`, large utility functions in `src/lib/*`. + +*This overview is intended to serve as a living document; future sections will expand on each module, performance audit findings, and junior developer quickstart guides.* diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..4c4218e --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,50 @@ +# Contribution Guide + +## 1. Coding Standards + +- Use TypeScript with `strict` mode enabled. +- Follow the import order: core, library, project, then relative. +- Naming: PascalCase for components, camelCase for functions/variables, UPPER_SNAKE_CASE for constants. +- Add JSDoc comments for all public APIs. +- Keep line length <= 120 characters, use Prettier for formatting. +- Lint with `pnpm lint` and fix warnings. + +## 2. Branch Workflow + +- Create a feature branch: `git checkout -b feat/` +- Keep branches up to date with `main` via `git rebase` or `git merge --ff-only`. +- Submit a Pull Request with a clear title and description. +- Ensure PR passes CI (tests, lint, type-check). +- Address review comments promptly. + +## 3. Running Tests + +- Install dependencies: `pnpm install`. +- Run tests in watch mode: `pnpm test --watch`. +- Run tests with coverage: `pnpm test --coverage`. +- Run lint: `pnpm lint`. +- Run type-check: `pnpm typecheck`. + +## 4. Making Your First Contribution + +1. Pick a beginner-friendly issue labeled `good first issue`. +2. Follow the steps in the Quickstart guide. +3. Make a small change (e.g., fix typo, add JSDoc). +4. Run the relevant tests. +5. Commit and push, then open a PR. + +## 5. Where to Extend (AI Agent Guidance) + +- **Canvas rendering**: Extend `src/app/canvas/CanvasPage.tsx` by adding new node types or customizing the context menu. +- **Recollection management**: Modify `src/app/recollections/RecollectionsPage.tsx` to add new views or actions. +- **State management**: Update `src/app/canvas/canvasStore.ts` for new graph features. +- **Performance**: Optimize expensive renders in `src/app/recollections/RecollectionsPage.tsx` using memoization. + +## 6. Useful Links + +- [ARCHITECTURE.md](ARCHITECTURE.md) – High-level architecture overview. +- [PERFORMANCE_IMPROVEMENTS.md](PERFORMANCE_IMPROVEMENTS.md) – Performance improvement plan. +- [QUICKSTART_FOR_JUNIORS.md](QUICKSTART_FOR_JUNIORS.md) – Junior developer quickstart. +- [CODE_REVIEW_CHECKLIST.md](CODE_REVIEW_CHECKLIST.md) – Checklist for reviewers. + +*Thank you for contributing!* diff --git a/PERFORMANCE_IMPROVEMENTS.md b/PERFORMANCE_IMPROVEMENTS.md new file mode 100644 index 0000000..8f98002 --- /dev/null +++ b/PERFORMANCE_IMPROVEMENTS.md @@ -0,0 +1,70 @@ +# Performance Improvements Plan + +## 1. Executive Summary + +This document outlines identified performance bottlenecks in the ZUI application and proposes concrete actions to improve render efficiency, state management, and code splitting. The plan is targeted at enabling junior developers to contribute measurable performance gains while maintaining code quality. + +## 2. Current Performance Issues + +| Issue | Location | Impact | Root Cause | +|-------|----------|--------|------------| +| **Large Component Renders** | `frontend/src/app/canvas/CanvasPage.tsx`, `frontend/src/app/recollections/RecollectionPage.tsx` | High CPU usage, frame drops | These components render thousands of nodes without memoization; state updates trigger full re-renders. | +| **Unmemoized Expensive Calculations** | Various utility functions in `src/lib/*` (e.g., graph layout calculations) | Delayed response to user interactions | Calculations recomputed on every render cycle. | +| **Frequent State Updates** | `canvasStore` mutations in response to mouse/touch events | Unnecessary re-renders of unrelated nodes | State updates not granular; multiple mutations in quick succession. | +| **Lack of Code Splitting** | Heavy modules imported globally (e.g., rendering views, graph utilities) | Initial bundle size > 2MB, slow load | All modules loaded upfront even if not used. | +| **Inefficient List Rendering** | Lists of nodes in sidebar components | Scrolling lag | No virtualization; all items rendered simultaneously. | +| **Repeated API Calls** | Agent service calls without proper caching | Latency spikes | Calls bypass `cacheRepository` in some paths. | + +## 3. Recommended Improvements + +### 3.1 Component Refactoring + +- **Split `CanvasPage` and `RecollectionPage`** into smaller, feature‑specific sub‑components. +- **Extract pure logic** (e.g., node layout calculations) into standalone utility functions with `useMemo`. +- **Apply `React.memo`** to pure presentational components that receive static props. + +### 3.2 State Management Optimization + +- **Granular State Updates**: Use `canvasStore` selectors to update only the affected node slices. +- **Batch Updates**: Wrap multiple mutations in `runWithTiming` or `unstable_batchedUpdates` to reduce render cycles. + +### 3.3 Memoization & Lazy Loading + +- **Memoize Callbacks**: Replace inline event handlers with `useCallback` references stored in context or hooks. +- **Dynamic Imports**: Use `import()` for heavy modules (e.g., `RenderingNode`, `AnimatedEdge`) to split the bundle. +- **Virtualized Lists**: Integrate `react-window` or `react-virtualized` for large node lists in sidebars. + +### 3.4 Code Splitting & Bundle Optimization + +- **Remove Unused Dependencies**: Audit `package.json` for deprecated libraries. +- **Enable `vite-plugin-dynamic-import`** for on‑demand loading of route‑specific components. +- **Compress Assets**: Configure `gzip`/`brotli` in `nginx.conf` for large SVG and texture assets. + +### 3.5 Caching Strategy Enhancements + +- **Centralize Caching**: Ensure all external API calls route through `cacheRepository`. +- **Add TTL** to cached responses to avoid stale data while still reducing repeat calls. + +### 3.6 Testing & Verification + +- **Performance Tests**: Add Vitest benchmarks for render times using `performance.now()`. +- **Profile with React DevTools**: Capture flame graphs before and after each optimization. +- **CI Gate**: Enforce that PRs must include a performance regression test if changes affect rendering. + +## 4. Contribution Path for Junior Developers + +1. **Familiarize** with the `canvasStore` architecture and the `CanvasPage` component structure. +2. **Pick** a low‑risk optimization (e.g., memoizing a utility function). +3. **Implement** the change, add JSDoc comments, and write a simple benchmark. +4. **Submit** a PR with: + - Description of the performance gain. + - Updated tests/benchmarks. + - Documentation in `PERFORMANCE_IMPROVEMENTS.md`. + +## 5. Success Metrics + +- **Target**: Reduce average frame time from ~16 ms to < 10 ms for `CanvasPage`. +- **Bundle Size**: Decrease initial load by ≥ 15 %. +- **API Latency**: Cut repeated call overhead by ≥ 30 %. + +*This plan is living; subsequent sections will track progress and update targets.* diff --git a/README.md b/README.md index 8fe4f59..7010fa4 100644 --- a/README.md +++ b/README.md @@ -56,8 +56,8 @@ cd frontend && npm install && npm run dev docker compose up --build ``` -- **Frontend**: http://localhost:3000 (Nginx; `/api/*` proxied to backend). -- **Backend**: http://localhost:8080 (Express). +- **Frontend**: (Nginx; `/api/*` proxied to backend). +- **Backend**: (Express). Environment variables (backend service): @@ -74,6 +74,18 @@ For self-hosting (e.g. Tailscale/HTTPS): set `CORS_ORIGIN` to your frontend URL; ## Scripts +## Contribute + +For contribution guidelines, see: + +- [ARCHITECTURE.md](ARCHITECTURE.md) +- [PERFORMANCE_IMPROVEMENTS.md](PERFORMANCE_IMPROVEMENTS.md) +- [QUICKSTART_FOR_JUNIORS.md](QUICKSTART_FOR_JUNIORS.md) +- [CONTRIBUTING.md](CONTRIBUTING.md) +- [CODE_REVIEW_CHECKLIST.md](CODE_REVIEW_CHECKLIST.md) + +*Thank you for contributing!* + | Command | Description | |--------|-------------| | `cd frontend && npm run dev` | Vite dev server. | diff --git a/docs/CODE_REVIEW_CHECKLIST.md b/docs/CODE_REVIEW_CHECKLIST.md new file mode 100644 index 0000000..5786f4c --- /dev/null +++ b/docs/CODE_REVIEW_CHECKLIST.md @@ -0,0 +1,48 @@ +# Code Review Checklist + +Use this checklist during code reviews to ensure high standards of readability, maintainability, and performance. + +## 1. Readability + +- [ ] **Naming**: Variables, functions, and components use descriptive, consistent names (camelCase for functions/variables, PascalCase for components). +- [ ] **JSDoc**: All public APIs have complete JSDoc comments with `@param`, `@returns`, and `@example` tags. +- [ ] **Line Length**: No line exceeds 120 characters; wrap long lines for readability. +- [ ] **Comments**: Explain *why* something is done, not just *what* is done. Avoid redundant comments. +- [ ] **Whitespace**: Consistent spacing and indentation (2 spaces for TypeScript/JSX). + +## 2. Maintainability + +- [ ] **Component Structure**: Large components are split into smaller, focused sub‑components. +- [ ] **Hooks**: Custom hooks encapsulate reusable logic and are named with `use` prefix. +- [ ] **Utility Functions**: Pure functions live in `src/lib/` and are exported for reuse. +- [ ] **Type Safety**: All function signatures include explicit TypeScript types; no `any` usage. +- [ ] **Import Order**: Core → Library → Project → Relative imports; alphabetical within groups. + +## 3. Performance + +- [ ] **Memoization**: Expensive calculations use `useMemo`; event handlers use `useCallback`. +- [ ] **Lazy Loading**: Heavy modules are loaded via `import()`; code splitting is configured in Vite. +- [ ] **Virtualization**: Large lists use `react-window` or similar for efficient rendering. +- [ ] **State Granularity**: State updates target only the minimal portion of state; avoid unnecessary re‑renders. +- [ ] **Bundle Size**: No unused dependencies; assets are compressed (gzip/brotli) in production. + +## 4. Accessibility & Security + +- [ ] **ARIA**: Semantic HTML and ARIA attributes are present for interactive elements. +- [ ] **Input Validation**: User inputs are validated before processing; no hard‑coded secrets. +- [ ] **Error Handling**: Errors are caught and logged appropriately; user‑friendly messages are shown. + +## 5. Testing + +- [ ] **Unit Tests**: Cover all new logic with tests; aim for ≥ 80 % coverage on critical paths. +- [ ] **Integration Tests**: Verify that components interact correctly with state/store. +- [ ] **Test Naming**: Test files end with `.test.tsx` and describe the behavior being tested. + +## 6. Documentation Links + +- [ ] Architecture Overview: [[ARCHITECTURE.md](ARCHITECTURE.md)] +- [ ] Performance Plan: [[PERFORMANCE_IMPROVEMENTS.md](PERFORMANCE_IMPROVEMENTS.md)] +- [ ] Junior Developer Quickstart: [[QUICKSTART_FOR_JUNIORS.md](QUICKSTART_FOR_JUNIORS.md)] +- [ ] Contribution Guide: [[CONTRIBUTING.md](CONTRIBUTING.md)] + +*Reviewers should mark any failing items as **[ ]** and request changes before approving.* diff --git a/frontend/docs/QUICKSTART_FOR_JUNIORS.md b/frontend/docs/QUICKSTART_FOR_JUNIORS.md new file mode 100644 index 0000000..a5b4a90 --- /dev/null +++ b/frontend/docs/QUICKSTART_FOR_JUNIORS.md @@ -0,0 +1,99 @@ +# Junior Developer Quickstart + +## 1. Overview + +This guide explains how to set up the development environment, run the application, and make your first contribution. + +## 2. Prerequisites + +- **Node.js** (v18 or later) +- **pnpm** (package manager) +- **Docker** (for backend services, optional) +- **Git** (version control) + +## 3. Repository Setup + +```bash +# Clone the repository +git clone https://github.com/your-org/zui.git +cd zui + +# Install dependencies +pnpm install +``` + +## 4. Running the Application + +### Frontend + +```bash +pnpm dev +``` + +Open to see the app. + +### Backend + +```bash +pnpm start +``` + +The backend API will be available at . + +## 5. Making Your First Contribution + +1. **Create a feature branch**: + + ```bash + git checkout -b feat/your-first-contribution + ``` + +2. **Make a small change**: + - Improve a README typo. + - Add a missing JSDoc comment. + - Fix a minor bug. +3. **Run the appropriate tests** to ensure your change doesn't break anything: + + ```bash + pnpm test + ``` + +4. **Commit your changes** with a clear message: + + ```bash + git commit -am "feat: brief description of change" + ``` + +5. **Push the branch** to GitHub: + + ```bash + git push origin feat/your-first-contribution + ``` + +6. **Open a Pull Request** on GitHub, linking the relevant issue. + +## 6. Coding Standards + +- Follow the patterns in **ARCHITECTURE.md** and **CONTRIBUTING.md**. +- Use TypeScript with strict mode. +- Add JSDoc for all public APIs. +- Keep changes small and focused. + +## 7. Helpful Links + +- **ARCHITECTURE.md** - High-level architecture overview. +- **PERFORMANCE_IMPROVEMENTS.md** - Performance improvement plan. +- **CODE_REVIEW_CHECKLIST.md** - Checklist for reviewers. + +## 8. FAQ + +**Q:** Do I need to write tests? +**A:** For bug fixes and new features, yes. Unit tests and integration tests are located under `frontend/src/app/**/*.test.tsx`. + +**Q:** How do I run the linter? +**A:** `pnpm lint`. + +**Q:** Where can I find more documentation? +**A:** See the `docs/` directory and the repository wiki. + +---\n*Happy coding!*