Socrates/docs/implementation-plan.md

Socrata — Implementation Plan (Phase 0 + Phase 1 MVP)

Context

Why this exists. Socrata is a structured-thinking platform for product managers. It uses SysML as a formal semantic backbone — taxonomy, ontology, constraints, traceability — but hides that backbone behind two synchronized surfaces (a narrative text canvas and a SysML diagram canvas) and a conversational AI character (Socrates) who mediates all structural change.

Where we start. The repo at /Users/dtoro/Projects/Socrata is empty. A design bundle was extracted to docs/design-source/socrata/ containing three high-fidelity React/HTML prototype variants (Manuscript, Foundry, Atelier) of the dual-canvas editor and the Socrates-led seed onboarding screen. The prototype renders the same dataset (project "Aristotle" — an AI study companion) across all three themes via CSS variables.

Two-phase plan. Before committing 8 weeks to the editor build, we run a 2-week Phase 0 validation experiment that isolates and tests the riskiest assumption: can we actually translate a PM's idea into a useful SysML model, and can Socrates do meaningful analysis on it? Phase 0 uses PlantUML for visualization (not React Flow) so we spend the budget on prompts and evaluation, not on canvas rendering. Phase 1 (MVP, 8 weeks) only begins if Phase 0 passes the rubric — see phase-0-validation.md.

Companion docs. idea.md — full vision and Phase 1–3 spec. phase-0-validation.md — the 2-week validation experiment that gates MVP. sysml-modeling.md — the metamodel, validation rules, dependency graph, and SE discipline this implementation enforces. socrates.md — character, surfaces, modes, prompts, lifecycle, failure modes for the agent that mediates everything. sync.md — bidirectional text↔diagram sync engineering: ops alphabet, applyOps chokepoint, optimistic-local with tempIds, SSE protocol, conflict handling.

Decisions locked.

• Scope: Phase 1 MVP end-to-end (per spec): seed → dual-canvas editor → SysML model + validation → Socrates-mediated proposals → assumptions/risks → requirements traceability → web-search validation → versioning. Out of scope: branching, multi-user co-edit, downstream artifacts.
• Aesthetic: Manuscript only. Parchment + Newsreader serif + ink-blue. Best fit for document-first PM thinking tool. Drop Foundry/Atelier theme files.
• Stack: Next.js 15 (App Router) + TypeScript + React 18, single deployable unit.
• Diagram lib: React Flow with custom nodes/edges styled to match prototype's softened SysML look.
• Backend: Next.js full-stack — route handlers + server actions, Postgres via Prisma, Anthropic SDK called server-side.
• LLM:
  • Phase 0: local model via LM Studio (OpenAI-compatible endpoint at http://localhost:1234/v1). Zero LLM cost during validation; local capability is the bar Phase 0 must clear. Recommended models: Qwen 2.5 72B / Llama 3.3 70B (or smaller-quantized if hardware-constrained). See phase-0-plan.md §9.
  • Phase 1 MVP: user-configurable — LLMGateway interface supports both LM Studio (local, OpenAI-compatible) and Anthropic (hosted: Sonnet 4.6 + Haiku 4.5 with prompt caching). User picks at project setup or via a settings panel. Self-hosters get the privacy story; users who want frontier capability get hosted.
  • Cross-phase: prompts are model-portable in shape but not necessarily in quality. M6 includes a regression run of the Phase 0 corpus against both providers to catch drift in either path.
• Persistence: Postgres + Prisma. No auth, no users in MVP — single-tenant assumption, project owner is implicit.

---

Architecture overview

apps/web                              Next.js app (single package, Turborepo not needed yet)
  app/                                App Router
    (seed)/page.tsx                   Seed screen — Socrates interview, emerging-seed left rail
    (editor)/[projectId]/page.tsx     Main dual-canvas editor
    api/                              Route handlers for streaming LLM, web search
      socrates/route.ts               POST — streamed Socrates turn
      proposals/[id]/analyze/route.ts POST — impact analysis on a pending proposal
      research/route.ts               POST — web search for an assumption/risk
  components/
    editor/                           Shell, TopBar, LeftRail, CanvasHeader, StatusBar
    text-canvas/                      ProseMirror or Lexical editor + chip-rendering decorators
    diagram-canvas/                   React Flow wrapper + custom block/actor/constraint nodes
    socrates/                         Sigil, Dock, Bubble, NumberedOptions, MarginNote
    seed/                             SeedScreen, EmergingSeedRail, MiniGraph, Confidence
    ui/                               Buttons, pills, kbd shortcuts (shared primitives)
  lib/
    sysml/                            Metamodel types + validation engine
      model.ts                        Block, Property, Association, Constraint, Requirement
      validate.ts                     Pure validator → ValidationIssue[]
      diff.ts                         Structural diff between two model versions
    sync/                             Bidirectional text ↔ model sync
      narrative.ts                    Narrative AST (TipTap/ProseMirror doc shape)
      markup.ts                       Chip parse/serialize, model-element references
      reconcile.ts                    Apply text edits → model ops; model ops → narrative patches
    llm/
      gateway.ts                      LLMGateway interface (chat, stream, json mode)
      anthropic.ts                    Anthropic implementation
      prompts/                        Versioned prompt templates (socrates, impact, detect, seed)
    proposals/                        Change-proposal lifecycle + impact analysis orchestration
    research/                         Web-search integration (Brave or Tavily) + citation linking
    confidence/                       Confidence-scoring utilities for auto-generated elements
    db/                               Prisma client + repository functions
  prisma/schema.prisma                Event-sourced model + projection tables
  styles/
    theme-manuscript.css              Ported from design-source, the only theme
    base.css                          Ported from design-source/styles.css (cleaned)

Data model (Prisma)

Event-sourced for the spec's "Activity Changelog with Reasoning" requirement. Every state change writes to ChangelogEntry; SysMLModel, NarrativeDocument, Assumption, etc. are projections rebuilt from the event log.

model Project              { id, name, scope, tagline, ownerLabel, createdAt }
model SysMLModel           { id, projectId, version, json /* serialized graph */ }
model NarrativeDocument    { id, projectId, version, doc /* ProseMirror JSON */ }
model Block                { id, projectId, label, kind, x, y, w, h, properties Json }
model Association          { id, projectId, fromId, toId, label, kind }
model Constraint           { id, projectId, label, expression }
model Requirement          { id, projectId, tag, text, tracedToIds String[] }
model Assumption           { id, projectId, text, status, linkedElementIds String[], confidence }
model Risk                 { id, projectId, text, severity, linkedElementIds String[] }
model Experiment           { id, assumptionId?, riskId?, hypothesis, methodology, results, implications }
model SocratesThread       { id, projectId, anchorElementId?, status }
model SocratesMessage      { id, threadId, role, content Json /* text + options */ }
model Proposal             { id, projectId, threadId?, title, ops Json, status, iterationCount, impactSummary Json }
model ChangelogEntry       { id, projectId, ts, kind, payload Json, reasoning, proposalId? }
model ResearchFinding      { id, projectId, anchorElementId, query, url, snippet, stance }

ops on a Proposal is a list of model mutations (add-block, update-property, add-association, add-constraint…). Applying = appending to the changelog and recomputing projections.

Bidirectional sync model

The narrative is a ProseMirror document with custom chip inline nodes that carry { kind, refId }. The canonical SysML graph lives in Postgres; the narrative is a projection-with-prose: editing prose around a chip is free-form, chips are model references.

• Text → Model: ProseMirror transactions are inspected for chip insert/delete/edit. Each becomes a candidate model op fed through Socrates (or auto-applied for low-impact ops above a confidence threshold).
• Model → Text: When a model element is renamed or deleted, all chips referencing its id are updated/removed in a single ProseMirror transaction. New blocks created from the diagram are appended as a templated narrative paragraph the user can edit.
• Conflict avoidance: All structural mutations route through a single applyOps(projectId, ops, source) function that updates DB + emits patches over a Server-Sent Events stream consumed by both canvases.

LLM orchestration

LLMGateway interface — chat(messages, opts), stream(messages, opts), json(messages, schema). Anthropic implementation uses the official @anthropic-ai/sdk with prompt caching enabled on the system prompts for Socrates/impact templates.

Three model tiers:

• Sonnet 4.6 — Socrates conversational turns, impact analysis on proposals, seed-interview synthesis. Streamed.
• Haiku 4.5 — continuous low-cost detection passes (assumptions/risks/inconsistencies on document save), confidence scoring on auto-generated elements.
• Web search — Tavily API (cheaper than Brave for this volume), called from lib/research, results linked back to model elements with stance classification (supports / contradicts / neutral).

Prompt templates are versioned files under lib/llm/prompts/ so we can A/B them without code changes.

---

Build sequence — gated on Phase 0

The build runs Phase 0 → gate decision → Phase 1 (M1–M8). Phase 0 is detailed in phase-0-validation.md; the summary below covers what it produces and how it feeds into MVP.

Phase 0 — Validation (~2 weeks)

A throwaway harness — CLI + PlantUML rendering, no editor, no DB — that runs a corpus of 10 diverse seed ideas through:

• Seed → SysML model generation (Sonnet)
• Detection (assumptions, risks, inconsistencies)
• Conversational Socrates loop on each generated model
• Mediation (Socrates proposes a change, accept/reject, regenerate)

Each seed is scored on a 7-dimension rubric (model coverage / accuracy / parsimony / constraint capture; Socrates assumption + risk detection quality; voice and character). Pass criteria: ≥8 of 10 seeds average ≥3.5 with no dimension below 3.0.

Phase 0 gate:

• Pass → promote prompts (lib/llm/prompts/socrates/), metamodel types (lib/sysml/model.ts), seed corpus (eval suite), and PlantUML renderer (kept as MVP "export" feature) into Phase 1. Begin M1 with confidence.
• Soft fail → +1 week iteration on prompts/metamodel/model selection, re-run.
• Hard fail → do not start MVP. Sit with user, decide whether to pivot the metamodel, try Opus, or rethink the product hypothesis.

Phase 1 — MVP build sequence (8 milestones, ~8 weeks)

Each milestone is independently demoable. M1–M3 are visual. M4–M8 add engine + intelligence — and import the validated artifacts from Phase 0 rather than authoring from scratch.

M1 — Repo + visual port (week 1).

• pnpm create next-app (TypeScript, App Router, no Tailwind — we use the prototype's CSS).
• Port styles.css and theme-manuscript.css into styles/. Drop foundry + atelier.
• Convert prototype .jsx files (editor-shell, text-canvas, diagram.jsx for static SVG version, socrates, seed-screen) into typed React components under components/. Use static data.js content as a fixture.
• Wire (editor)/[projectId]/page.tsx and (seed)/page.tsx to render the static version pixel-faithful to the prototype.
• Done when: the rendered editor and seed screens match the Manuscript artboard at 1480×920.

M2 — Real text editor (week 2).

• Replace static narrative renderer with TipTap (ProseMirror under the hood) configured with a custom Chip node that renders identically to the prototype's chip styles (pill / color / underline / bracket variants).
• Add a TipTap command for inserting a chip (slash menu: /block, /property, /req, /assoc).
• Persist narrative as ProseMirror JSON in NarrativeDocument.
• Done when: users can type prose, insert chips, and the document round-trips through the DB.

M3 — Real diagram (week 3).

• Replace static SVG with React Flow. Custom nodes for block / actor / constraint mirroring the softened look (rounded rect, stereotype label, divider line, property compartment). Custom edges for association / composition / constraint (dashed for constraint).
• Drag-to-create blocks via a left-side palette. Click to edit label/properties in a sidebar. Drag to draw associations.
• Persist positions and structure to Block / Association tables.
• Done when: users can build the Aristotle example from scratch in the diagram and see it persist on reload.

M4 — SysML metamodel + validation (week 4).

• Import lib/sysml/model.ts from Phase 0 (validated). Implement lib/sysml/validate.ts and lib/sysml/depgraph.ts per the rule codes (S1–S5, M1–M5, T1–T3) in sysml-modeling.md. Pure functions, fully unit-testable.
• Surface issues inline in all four places: rail entry dot, diagram node border, narrative margin note, dock summary of new high-severity issues.
• Build the dependency graph incrementally on each model mutation — this is the substrate for impact analysis in M7.
• Done when: breaking the Aristotle model in known ways (cyclic composition, untraced requirement, dangling association endpoint) produces correct issues with zero false positives.

M5 — Bidirectional sync + change ops (week 5).

• lib/sync/reconcile.ts — single chokepoint applyOps(projectId, ops, source) mutates DB + emits SSE patch.
• ProseMirror plugin watches transactions, derives candidate ops (add chip → resolve to existing element; rename chip → mutate referenced element; delete chip → mark referenced element for removal review).
• React Flow change handlers feed the same applyOps path.
• SSE consumer in both canvases re-renders without losing local cursor state.
• Done when: renaming a block in the diagram updates every chip in the narrative live, and editing chip text in the narrative updates the diagram block label live.

M6 — Socrates dock + LLM gateway (week 6).

• lib/llm/gateway.ts with Anthropic implementation, prompt caching on system prompts.
• Import lib/llm/prompts/socrates/ from Phase 0 (character + interview + review + mediate prompts already validated).
• api/socrates/route.ts streams Socrates turns over SSE. State stored in SocratesThread + SocratesMessage.
• Dock UI with bubbles, numbered-options affordance (matching prototype), reply input, ⌘↵ to send.
• Seed screen wired to the Phase-0-validated interview prompt; field inferences stream into the left rail as they arrive.
• Done when: holding a real conversation with Socrates seeded by the project context produces relevant questions and the seed screen converges to a populated emerging-seed rail end-to-end.

M7 — Proposal workflow + impact analysis (week 7).

• When a structural edit lands (text or diagram), classify by impact heuristic: low-impact (cosmetic rename, position move, property add) auto-applies; high-impact (delete block, change association, add constraint, edit requirement) creates a Proposal and routes through Socrates.
• api/proposals/[id]/analyze/route.ts calls Sonnet with the model snapshot + the proposed ops + linked assumptions/risks/requirements; returns structured impactSummary (impacted blocks, affected requirements, broken assumptions, suggested follow-ups).
• Proposal card in the dock shows summary + Approve/Refine/Reject. Approve → applyOps. Refine → Socrates iteration. Iteration count caps at 3 before Socrates makes a recommendation.
• Every applied op writes a ChangelogEntry with proposal-derived reasoning.
• Done when: deleting a foundational block in the Aristotle model raises a proposal that correctly enumerates affected REQ-001/REQ-002 and produces a sensible recommendation in ≤3 iterations.

M8 — Assumptions, risks, research (week 8).

• Background detection job (debounced on document save) calls Haiku with the narrative + model — using the Phase-0-validated detection prompts (detect-assumptions.md, detect-risks.md). Returns candidate findings with confidence scores and linked element ids. Persist; surface in the rail with confidence indicators.
• Per-assumption/risk "Validate via web" action → api/research/route.ts runs Tavily, stores ResearchFinding rows with stance classification.
• Findings appear inline as margin notes on the linked element; clicking a finding opens a side panel with the snippet + source.
• Experiment logging UI (modal): hypothesis / methodology / results / implications, linked to assumption or risk.
• Run the Phase 0 corpus through the integrated MVP as a regression check — the detection quality should match or exceed the Phase 0 results.
• Done when: running the detector on a fresh Aristotle document surfaces ASM-1/ASM-2/ASM-3 and RSK-1/RSK-2 (or close equivalents) with reasonable confidence, and at least one assumption gets researched with a real-world citation.

After M8: the spec's Phase 1 success metrics are testable end-to-end. Stop, ship to 5–10 beta PMs, measure, then scope Phase 2 (branching).

---

Critical files / functions to reuse from the design source

The following live in docs/design-source/socrata/project/ and should be read and ported, not invented from scratch:

• styles.css (820 lines) → apps/web/styles/base.css. The shell, canvases, chips, dock, seed, and proposal styles are all production-grade and themeable via CSS variables.
• theme-manuscript.css (79 lines) → apps/web/styles/theme-manuscript.css. The full color palette + typography for the chosen aesthetic.
• editor-shell.jsx lines 31–129 → components/editor/LeftRail.tsx. Collapsible-section logic and block-count tooltip behavior.
• editor-shell.jsx lines 1–29 → components/editor/TopBar.tsx.
• editor-shell.jsx lines 143–208 → components/editor/EditorShell.tsx.
• text-canvas.jsx chip-style logic (chip-style-pill | color | underline | bracket) → components/text-canvas/Chip.tsx as a TipTap node-view.
• diagram.jsx → reference for visual fidelity only; the actual implementation is React Flow custom nodes/edges. Match radius, stroke, header-bar treatment, stereotype label position from this file.
• socrates.jsx SocratesSigil (lines 3–30) → components/socrates/Sigil.tsx verbatim. The numbered-options bubble (lines 79–95) → components/socrates/NumberedOptions.tsx.
• seed-screen.jsx whole file → components/seed/* split into EmergingSeedRail, MiniGraph, Confidence, Thread, InputRow.
• data.js → lib/fixtures/aristotle.ts. Used as the seed state for the demo project and as test fixture for SysML validation + LLM eval.

---

Open implementation questions to resolve during build

These are listed as spec "open questions"; the plan defers them to the milestone where they bite:

• Markup syntax — TipTap chip insertion via slash menu in M2 (no user-facing brackets needed). The four prototype rendering styles (pill | color | underline | bracket) ship as a user preference.
• Sync direction — full bidirectional from M5; M2/M3 ship one-way (DB → canvas) only.
• Confidence threshold for auto-apply — start at >= 0.85 for low-impact ops; tune in M7 based on PM feedback.
• Proposal iteration cap — 3 iterations, then Socrates recommends. Hardcoded in M7.

---

Verification

End-to-end flow that exercises the whole stack:

1. pnpm dev, open http://localhost:3000 — lands on seed screen.
2. Conduct a 4–5 turn interview with Socrates about the Aristotle idea; watch the emerging-seed rail populate.
3. Click "Open editor" — lands on dual-canvas with the seed-derived model.
4. Edit a heading in the narrative, insert a [Block: Tutor] chip via slash menu — verify it appears in the diagram.
5. Drag-rename a block in the diagram — verify all matching chips in the narrative update live.
6. Delete a foundational block (Aristotle) — verify a Proposal appears in the dock with impact analysis listing affected requirements; iterate once with Socrates; approve.
7. Open Assumptions in the rail, pick one, click "Validate via web" — verify a real Tavily citation lands as a margin note linked to the assumption.
8. Reload the page — verify everything persists.
9. Run pnpm test — Prisma migration smoke test, SysML validator unit tests (12+ rules), reconcile.ts round-trip property tests.

Local prereqs: DATABASE_URL (Neon dev branch fine), ANTHROPIC_API_KEY, TAVILY_API_KEY. All three documented in .env.example.