dtoro
f1c4566576
Sets up the Socrata project repo with: docs/ — strategy and design documents - idea.md: full product vision - implementation-plan.md: Phase 0 + Phase 1 MVP plan - phase-0-validation.md: 2-week validation experiment strategy - phase-0-plan.md: concrete Phase 0 build plan - phase-0-results.md: Phase 0 gate outcome — GO for MVP - sysml-modeling.md: metamodel + SE discipline + validation rules - socrates.md: agent character, surfaces, modes, prompts, lifecycle - sync.md: bidirectional text↔diagram sync engineering - design-source/: HTML/CSS/JS handoff bundle from Claude Design phase-0/ — validated harness (CLI, no UI, no DB) - LM Studio (local OpenAI-compatible) generation + detection + judge - PlantUML rendering for SysML model visualization - 10-seed corpus (8 working + 2 holdouts) - 5 corpus runs with iteration history in reports/ - Final gate: 10/10 pass, mean 4.32/5, holdouts validated Phase 1 MVP scope and milestones documented in implementation-plan.md.
547 lines
30 KiB
Markdown
547 lines
30 KiB
Markdown
# Bidirectional Sync — Engineering Design
|
||
|
||
This document specifies how the **narrative text canvas** and **SysML diagram canvas** stay in sync with the underlying SysML model. It's the engineering substrate beneath everything in [implementation-plan.md](implementation-plan.md) milestone M5, and the integrity layer that lets [socrates.md](socrates.md)'s impact analysis assume a coherent model snapshot.
|
||
|
||
This is the trickiest piece of Phase 1 engineering. The doc is dense intentionally — every decision below has knock-on consequences and reversing one usually means reversing several.
|
||
|
||
---
|
||
|
||
## 1. The problem
|
||
|
||
Both surfaces display the same underlying SysML model. When the user edits one, the other must reflect the change. This is non-trivial because:
|
||
|
||
1. **Different edit granularities.** Text edits are character-level. Diagram edits are node-level. The two need a common alphabet.
|
||
2. **Mixed content in narrative.** Prose (free-form, no model implication) and chips (model references) are intermingled. Most text edits affect *neither* the model nor the diagram; some affect both.
|
||
3. **In-flight ordering.** Multiple edits can land while a previous one is still applying. We need a deterministic order.
|
||
4. **Mixed apply policies.** Some edits should auto-apply instantly; some must route through Socrates as proposals (per [socrates.md §2.5](socrates.md)).
|
||
5. **Cursor preservation.** A remote-originated change must not disrupt the user's caret or selection.
|
||
6. **Latency tolerance.** Users will not accept network-roundtrip latency on every keystroke.
|
||
|
||
The design below addresses all six. The cost is a non-trivial server-side reconciliation engine; the alternative is a worse UX.
|
||
|
||
---
|
||
|
||
## 2. Core design decisions, at a glance
|
||
|
||
| # | Decision | Rationale |
|
||
|---|---|---|
|
||
| 1 | **Hybrid: optimistic-local for prose & low-impact ops; server-authoritative for high-impact ops.** | Pure server-auth round-trips every keystroke; pure optimistic creates merge nightmares. Prose is local-free; model ops gate on impact class. |
|
||
| 2 | **Common alphabet: `ModelOp`.** | Both surfaces emit and consume the same op type. Single source of truth for what a "change" means. |
|
||
| 3 | **Single chokepoint: `applyOps()`.** | Every model mutation — from text, diagram, proposal approval, or detection — funnels through one function. Validation, classification, persistence, and fan-out happen in one place. |
|
||
| 4 | **Chips reference by `refId` only; labels render from model.** | Eliminates whole categories of sync bugs. A chip never holds stale text. |
|
||
| 5 | **Server-assigned monotonic `version` per project.** | Sequential consistency without CRDT. The single-writer pattern works because we have a single server authority. |
|
||
| 6 | **SSE for fan-out, REST for ops.** | Bidirectional WebSockets are overkill for the actual traffic shape (ops out, patches in). |
|
||
| 7 | **Optimistic-local with temp refIds for instant chip insertion.** | Lets `/block Foo → create new` feel instant; reconciliation maps temp → canonical id. |
|
||
| 8 | **Per-transaction op batching.** | Pasting structured text yields one API call, not five. |
|
||
| 9 | **Per-client `clientId` filters self-echo.** | The originating client doesn't re-apply its own op when the SSE patch echoes back. |
|
||
| 10 | **Impact classification at the server.** | Clients don't decide whether an op auto-applies vs. opens a proposal — server does, with full model + dep-graph context. |
|
||
|
||
---
|
||
|
||
## 3. The vocabulary: `ModelOp`
|
||
|
||
`ModelOp` is the canonical edit unit. Every change to the SysML model is expressible as a sequence of `ModelOp`s.
|
||
|
||
```ts
|
||
// lib/sync/ops.ts
|
||
|
||
export type ModelOp =
|
||
// Block ops
|
||
| { kind: 'add-block'; tempId?: string; block: Block }
|
||
| { kind: 'update-block'; blockId: string; patch: Partial<Block> }
|
||
| { kind: 'remove-block'; blockId: string }
|
||
// Property ops
|
||
| { kind: 'add-property'; blockId: string; tempId?: string; property: Property }
|
||
| { kind: 'update-property'; blockId: string; propertyId: string; patch: Partial<Property> }
|
||
| { kind: 'remove-property'; blockId: string; propertyId: string }
|
||
// Association ops
|
||
| { kind: 'add-association'; tempId?: string; association: Association }
|
||
| { kind: 'update-association'; associationId: string; patch: Partial<Association> }
|
||
| { kind: 'remove-association'; associationId: string }
|
||
// Constraint ops
|
||
| { kind: 'add-constraint'; tempId?: string; constraint: Constraint }
|
||
| { kind: 'update-constraint'; constraintId: string; patch: Partial<Constraint> }
|
||
| { kind: 'remove-constraint'; constraintId: string }
|
||
// Requirement ops
|
||
| { kind: 'add-requirement'; tempId?: string; requirement: Requirement }
|
||
| { kind: 'update-requirement'; requirementId: string; patch: Partial<Requirement> }
|
||
| { kind: 'remove-requirement'; requirementId: string }
|
||
| { kind: 'add-relation'; requirementId: string; relation: RequirementRelation }
|
||
| { kind: 'remove-relation'; requirementId: string; relationIndex: number };
|
||
```
|
||
|
||
**Why `tempId`?** When the client optimistically creates an element before the server has assigned a canonical id, the op carries a `tempId` (e.g., `tmp_a8f3...`). The server may use it as the canonical id (preferred — no rewrite) or generate a new id and return a mapping. Either way, the client uses `tempId` to correlate its local optimistic state with the server's confirmation.
|
||
|
||
**No `move-block`, `reorder-property`, `set-position`?** Those are `update-block` with a `position` patch. Keeping the op set narrow simplifies the validator and the impact classifier.
|
||
|
||
---
|
||
|
||
## 4. The single chokepoint: `applyOps()`
|
||
|
||
```ts
|
||
// lib/sync/applyOps.ts
|
||
|
||
interface ApplyOpsArgs {
|
||
projectId: string;
|
||
ops: ModelOp[];
|
||
source: {
|
||
kind: 'text' | 'diagram' | 'proposal' | 'detection' | 'seed';
|
||
clientId?: string; // for self-echo filtering
|
||
proposalId?: string; // present when source.kind === 'proposal'
|
||
};
|
||
expectedVersion?: number; // optimistic concurrency check
|
||
}
|
||
|
||
interface ApplyOpsResult {
|
||
status: 'applied' | 'proposed' | 'rejected';
|
||
// 'applied' — ops landed, model version bumped, patches fanned out
|
||
appliedOps?: ModelOp[]; // canonical-id-resolved version of the input
|
||
newVersion?: number;
|
||
idMapping?: Record<string, string>; // tempId → canonicalId
|
||
// 'proposed' — ops queued as a Proposal, no model mutation yet
|
||
proposalId?: string;
|
||
// 'rejected' — validation failed
|
||
errors?: Array<{ opIndex: number; code: string; message: string }>;
|
||
}
|
||
|
||
async function applyOps(args: ApplyOpsArgs): Promise<ApplyOpsResult>;
|
||
```
|
||
|
||
**What it does, in order:**
|
||
|
||
1. **Concurrency check.** If `expectedVersion` is present and ≠ current model version, reject with a version-mismatch error. Caller should fetch the latest snapshot and retry (rare).
|
||
2. **Resolve tempIds.** Replace `tempId` with canonical ids in each op (use the tempId itself as canonical when possible — most temp ids are pre-validated UUIDs).
|
||
3. **Validate each op against the current snapshot.** Structural rules (S1–S5 from [sysml-modeling.md §7](sysml-modeling.md)) apply. An op that violates structure is rejected — entire batch fails atomically.
|
||
4. **Classify impact.** Each op gets a `low | medium | high` impact label based on the dep graph + op kind (see §7).
|
||
5. **Branch on the highest impact in the batch:**
|
||
- All low → apply; persist; fan out.
|
||
- Any medium or high → open a Proposal, persist the ops as `Proposal.ops`, **do not mutate the model yet**. Fan out a `proposal-opened` patch only.
|
||
6. **On apply:**
|
||
- Run validators (semantic + traceability rules) against the new snapshot. If any new error appears, **rollback** and rejected.
|
||
- Write a `ChangelogEntry` with payload = ops, source, reasoning (from proposal thread if applicable).
|
||
- Bump model version.
|
||
- Update projection tables (`Block`, `Association`, etc.).
|
||
- Trigger background detection job (debounced).
|
||
- Emit SSE patch.
|
||
7. **Return.** Result tells the caller what happened and supplies any tempId → canonical mapping.
|
||
|
||
`applyOps` is the only function that mutates the model. Period. No backdoor. Detection findings, Socrates suggestions, proposal approvals, seed-screen handoff — all of them call this.
|
||
|
||
---
|
||
|
||
## 5. Text → ops: deriving `ModelOp`s from a ProseMirror transaction
|
||
|
||
The narrative is a TipTap (ProseMirror) document. A custom inline node `chip` carries `{ refId: string, kind: 'block' | 'property' | 'association' | 'requirement' | 'constraint' }`. Crucially, **the chip does not store the displayed label** — labels are looked up live from the model snapshot at render time.
|
||
|
||
### 5.1 Transaction inspection
|
||
|
||
A ProseMirror `Transaction` is a sequence of `Step`s. The relevant ones:
|
||
|
||
- **`ReplaceStep`** — replaces a slice of the doc with another slice. Inserting or deleting a chip node shows up here.
|
||
- **`ReplaceAroundStep`** — block-level structural change (heading promotion, etc.). Rarely produces ops.
|
||
- **Mark steps** — irrelevant for chips.
|
||
- **Attribute steps (`AttrStep`)** — used for in-place chip edits (e.g., changing `refId`). Rare; we generally re-create the chip.
|
||
|
||
The text → ops pipeline lives in `lib/sync/text-to-ops.ts`:
|
||
|
||
```ts
|
||
function transactionToOps(
|
||
tr: ProseMirrorTransaction,
|
||
snapshot: SysMLModel
|
||
): ModelOp[]
|
||
```
|
||
|
||
It walks each Step, detects chip insertions/removals/attr-changes, and emits ops. A pure prose edit (no chip touched) returns `[]`.
|
||
|
||
### 5.2 Chip insertion → ops
|
||
|
||
A new chip lands in the doc. Two cases:
|
||
|
||
- **Reference an existing element.** `refId` resolves in the snapshot. **No op emitted** — inserting a chip is a textual reference, not a model mutation. The dep graph picks up the new reference automatically because chips-in-narrative is a derived projection.
|
||
- **Create a new element.** User typed `/block Foo` and picked "create new" from autocomplete. The slash-menu handler:
|
||
1. Generates a `tempId` (`tmp_${uuid}`)
|
||
2. Inserts the chip with `refId = tempId`
|
||
3. Emits an `add-block` op with `tempId` and the new block's defaults
|
||
4. Sends to server via `applyOps`
|
||
5. On success, replaces the chip's `refId = tempId` with the canonical id from `idMapping`
|
||
|
||
### 5.3 Chip removal → ops (the interesting case)
|
||
|
||
User deletes a chip from prose. Removing the chip ≠ removing the underlying element — the element might still appear in the diagram, or be referenced by other chips, or satisfy a requirement. Two cases:
|
||
|
||
- **Element is still referenced elsewhere** (other chips, diagram, dep graph) → **no op**. The chip removal is a pure text change.
|
||
- **Element has no remaining references** → emit a *candidate* `remove-X` op, but mark it as `medium`-impact in classification. This routes through Socrates: "You just removed the last reference to `Tutor`. Should I delete the block from the model?" User confirms or undoes.
|
||
|
||
This is the right default. Auto-deleting elements when the last chip is removed would surprise users. Asking is cheap.
|
||
|
||
### 5.4 In-place rename
|
||
|
||
A chip's *displayed* label is rendered from the model. To rename, the user clicks the chip, an inline editor pops, edits the label, hits enter. That fires an `update-block` op with `patch: { label: 'New name' }`. Every other chip referencing this block re-renders automatically (label is a model lookup) — no per-chip mutation needed.
|
||
|
||
This is why chip-as-pure-reference is load-bearing: rename is one op; the diagram, the rail, every chip in the narrative all reflect it without per-surface plumbing.
|
||
|
||
### 5.5 Paste behavior
|
||
|
||
User pastes content from another document (or from another part of the same doc) that contains chips. Each pasted chip's `refId` may or may not resolve in this project's model:
|
||
|
||
- Resolves → keep the reference.
|
||
- Doesn't resolve → strip the chip but keep the label as plain text, with a margin note: "Reference to `Foo` couldn't be resolved here." (User can re-create as a new element via slash-menu if desired.)
|
||
|
||
Pasted prose without chips is just prose — no ops.
|
||
|
||
### 5.6 Slash-menu autocomplete
|
||
|
||
Slash commands (`/block`, `/property`, `/req`, `/assoc`, `/constraint`) trigger autocomplete:
|
||
|
||
- Typing `/block ` shows a fuzzy-search list of existing blocks
|
||
- Selecting one inserts a chip with that block's id
|
||
- "Create new" at the bottom inserts a chip with a `tempId` and emits the corresponding `add-X` op
|
||
- Property-chip flow first prompts for the parent block (drilldown), then fuzzy-searches that block's properties
|
||
|
||
Property chips' `refId` is a composite: `${blockId}.${propertyId}`.
|
||
|
||
### 5.7 Batching
|
||
|
||
A single ProseMirror transaction can produce multiple ops (paste of structured text, multi-chip selection delete). All ops from one transaction are sent as one batched `applyOps` call — atomic on the server side.
|
||
|
||
---
|
||
|
||
## 6. Diagram → ops: deriving `ModelOp`s from React Flow events
|
||
|
||
React Flow exposes `onNodesChange`, `onEdgesChange`, `onNodesDelete`, etc. Each callback gives us structured events.
|
||
|
||
| React Flow event | ModelOp |
|
||
|---|---|
|
||
| Node added (drag from palette) | `add-block` (or `add-actor`, `add-constraint` based on palette item) with `tempId` |
|
||
| Node deleted | `remove-block` (medium-impact if dependents exist) |
|
||
| Node position changed (`drag` end) | `update-block` with `patch: { position }` (debounced — only emit on drag end) |
|
||
| Node label edited (inline) | `update-block` with `patch: { label }` |
|
||
| Node properties edited (sidebar) | `update-block` with `patch: { properties }` (or per-property `add-property` / `update-property` / `remove-property`) |
|
||
| Edge added | `add-association` with `tempId`; default `kind: 'association'` until user picks |
|
||
| Edge label edited | `update-association` with `patch: { label }` |
|
||
| Edge kind changed (sidebar) | `update-association` with `patch: { kind }` |
|
||
| Edge deleted | `remove-association` |
|
||
|
||
All of these flow through `lib/sync/diagram-to-ops.ts` and end up at `applyOps`.
|
||
|
||
**Position changes are special.** They're cosmetic, frequent (during drag), and must not block UI. Approach:
|
||
|
||
- Local state updates on every `onNodesChange` (smooth dragging)
|
||
- Server `update-block { position }` only fires on `dragEnd`
|
||
- Position ops are always `low`-impact; they auto-apply
|
||
- The SSE echo of a position op to the originating client is filtered (clientId match)
|
||
|
||
**Edges have no `tempId` round-trip dance** because edges are simple — server-side an `add-association` is just an insert, no race.
|
||
|
||
---
|
||
|
||
## 7. Impact classification
|
||
|
||
`lib/sync/classify.ts` maps each `ModelOp` to `low | medium | high` based on the current model + dep graph.
|
||
|
||
### 7.1 The rules
|
||
|
||
```
|
||
low (auto-apply)
|
||
add-block, add-actor, add-constraint — pure addition; no breakage
|
||
add-property — pure addition
|
||
add-requirement (with no relations) — pure addition; rule T1 will warn but not break
|
||
add-relation — strengthens traceability
|
||
add-association — additive; affects dep graph but doesn't break
|
||
update-block { position } — cosmetic
|
||
update-block { label } — chips re-render; nothing breaks
|
||
update-property { name } — same
|
||
update-requirement { text } — same
|
||
update-association { label } — same
|
||
update-constraint { label } — same
|
||
|
||
medium (propose, single-card iteration ≤ 3)
|
||
update-block { kind } — actor → block changes semantics
|
||
update-property { type | multiplicity } — may invalidate values
|
||
update-association { kind } — association → composition changes semantics
|
||
update-association { fromBlockId | toBlockId } — re-anchors a relationship
|
||
update-constraint { expression | appliesTo } — semantic shift
|
||
update-requirement { relations } — affects traceability
|
||
remove-property — may break chips & validation
|
||
remove-association — affects dep graph
|
||
remove-constraint — relaxes invariants
|
||
remove-requirement — affects traceability
|
||
remove-block (dependents = 0) — clean removal but worth confirming
|
||
|
||
high (propose, modal recommendation card)
|
||
remove-block (dependents > 0) — broad blast radius
|
||
update-block { kind: '*' → 'system' or vice versa } — re-roots the SoI
|
||
remove-relation (last satisfier of a req) — leaves req untraced
|
||
any op explicitly tagged high by a project-specific constraint — Phase 1.5
|
||
```
|
||
|
||
### 7.2 Why server-side?
|
||
|
||
The classifier needs:
|
||
- The current model snapshot
|
||
- The dep graph
|
||
- Sometimes the result of running validators on the post-op snapshot
|
||
|
||
All of which live server-side. Doing classification on the client would require shipping the dep graph to every client, which is wasteful, and would let a buggy or stale client mis-classify. Server is authoritative.
|
||
|
||
### 7.3 Batches
|
||
|
||
For a multi-op batch, the batch's class is `max(class for each op)`. Any single high-impact op promotes the whole batch to high.
|
||
|
||
### 7.4 Override hooks
|
||
|
||
A future user preference ("always confirm even cosmetic edits") can shift the threshold. MVP ships defaults only.
|
||
|
||
---
|
||
|
||
## 8. The SSE patch protocol
|
||
|
||
After `applyOps` succeeds, the server emits patches over an authenticated SSE stream subscribed by every active canvas for the project. (Authentication TBD in Phase 1 since we have no users — placeholder.)
|
||
|
||
```ts
|
||
// lib/sync/patches.ts
|
||
|
||
export type SyncPatch =
|
||
| {
|
||
kind: 'ops-applied';
|
||
ops: ModelOp[]; // canonical ids; tempIds resolved
|
||
version: number;
|
||
sourceClientId?: string;
|
||
}
|
||
| {
|
||
kind: 'ops-rejected';
|
||
sourceClientId: string; // only sent to the originating client
|
||
errors: Array<{ opIndex: number; code: string; message: string }>;
|
||
attemptedOps: ModelOp[];
|
||
}
|
||
| {
|
||
kind: 'proposal-opened';
|
||
proposalId: string;
|
||
title: string;
|
||
ops: ModelOp[];
|
||
impactSummary: ImpactSummary;
|
||
sourceClientId?: string;
|
||
}
|
||
| {
|
||
kind: 'proposal-resolved';
|
||
proposalId: string;
|
||
status: 'approved' | 'rejected';
|
||
}
|
||
| {
|
||
kind: 'validation-issues';
|
||
issues: ValidationIssue[]; // delta — added/removed/changed
|
||
version: number;
|
||
}
|
||
| {
|
||
kind: 'detection-finding';
|
||
finding: Finding;
|
||
}
|
||
| {
|
||
kind: 'snapshot-resync';
|
||
snapshot: SysMLModel; // sent on version skew or manual request
|
||
version: number;
|
||
};
|
||
```
|
||
|
||
### 8.1 What each canvas does
|
||
|
||
On receiving a patch, each canvas:
|
||
|
||
1. **Filter self-echo.** If `sourceClientId === myClientId`, the originating client already applied this op locally and updated its temp→canonical mapping. Drop the patch. (Exception: a sourceless patch — e.g., a detection finding — always applies.)
|
||
2. **Version check.** If `version <= localVersion`, drop (late-arriving). If `version > localVersion + 1`, request snapshot resync.
|
||
3. **Apply ops to local state.** For text canvas: convert the ops to a ProseMirror transaction (chip refIds get re-rendered with new labels; chips for removed elements get stripped). For diagram canvas: feed React Flow a node/edge update.
|
||
4. **Preserve cursor / selection.** See §10.
|
||
5. **Re-render.**
|
||
|
||
### 8.2 Why SSE, not WebSocket?
|
||
|
||
Traffic is asymmetric: high-volume ops out (REST), low-volume patches in (SSE). SSE is one-way server→client, perfect for this shape. WebSockets add complexity for no benefit here. Plus SSE auto-reconnects with `Last-Event-ID` for free.
|
||
|
||
### 8.3 Backpressure
|
||
|
||
Client ignores patches if more than 50 are queued (rare; only on slow CPU). Triggers a snapshot resync to recover. Not a real concern for MVP scale.
|
||
|
||
---
|
||
|
||
## 9. Optimistic-local — making chip insertion feel instant
|
||
|
||
Naive flow for `/block Foo → create new`:
|
||
|
||
1. T+0: user picks "create new"
|
||
2. T+1: send `add-block` to server
|
||
3. T+50–200: server processes
|
||
4. T+50–200: server SSE patch arrives
|
||
5. T+50–200: chip pops in
|
||
|
||
That's 50–200ms of nothing happening between user action and visible result. Unacceptable for a tool used every second.
|
||
|
||
Optimistic-local flow:
|
||
|
||
1. T+0: user picks "create new"
|
||
2. T+1: TipTap inserts chip with `refId = tempId`. Block immediately appears in the diagram (the diagram listens to local state too). Rail updates.
|
||
3. T+1: client sends `add-block` op with `tempId` to server.
|
||
4. T+50–200: server confirms, returns canonical id.
|
||
5. T+50–200: client maps `tempId → canonicalId` in:
|
||
- The chip's `refId` attr in the ProseMirror doc (single attribute change, cursor unaffected)
|
||
- The block's id in local React Flow state
|
||
- The dep graph
|
||
- Any open Socrates thread anchored to this element
|
||
6. T+50–200: SSE patch arrives, but it has `sourceClientId === myClientId` so it's filtered.
|
||
|
||
### 9.1 Failure rollback
|
||
|
||
If the server rejects the op (rare — validation failure):
|
||
- Client receives `ops-rejected` patch
|
||
- Local state rolls back: chip is removed from the doc; if a corresponding diagram node was created locally, it's removed; rail re-renders
|
||
- An inline error appears at the user's cursor: "Couldn't create block — `name already exists` (S5)"
|
||
|
||
### 9.2 Which ops can be optimistic?
|
||
|
||
Only ops that classify as `low`-impact can be optimistic. Medium/high go through proposal flow which is inherently asynchronous and has its own UI (cards, iteration), so latency is expected and tolerable.
|
||
|
||
This gives us the ergonomics of optimistic-local where it matters (typing, dragging) without the merge-hell of optimistic-everywhere.
|
||
|
||
---
|
||
|
||
## 10. Cursor and selection preservation
|
||
|
||
### 10.1 Text canvas
|
||
|
||
When a remote-originated `ops-applied` patch lands while the user has a caret in the doc, we must not disrupt their position. ProseMirror has a built-in mechanism: `Transaction.mapping` maps document positions through a transaction.
|
||
|
||
Approach: convert the incoming ops to a ProseMirror transaction, then map the user's selection through it before re-rendering.
|
||
|
||
- An op affecting only the model (no chip insert/delete) → no transaction needed; just trigger chip re-render via React state update (chip labels are model-derived, so React's reconciliation handles it).
|
||
- An op that removes a chip → ProseMirror transaction deletes the chip node; selection mapped through (caret moves left if it was inside or after).
|
||
- An op that adds a chip from a remote source — rare, only happens with proposal approval — places the chip at the appropriate location (server tells us); selection unchanged unless cursor was at the insertion point.
|
||
|
||
### 10.2 Diagram canvas
|
||
|
||
React Flow tracks selected node ids. After a remote patch:
|
||
|
||
- Selected node still exists → re-select.
|
||
- Selected node was removed → select its nearest dep-graph neighbor, or no selection.
|
||
- Selected node was renamed → no action; React Flow re-renders with new label.
|
||
|
||
### 10.3 Active dock thread
|
||
|
||
If the dock thread's anchor element is removed by a remote patch, the thread is auto-archived with a system message: "Anchor element `Foo` was removed; thread archived." User can unarchive if desired.
|
||
|
||
---
|
||
|
||
## 11. Ordering, versions, clientIds
|
||
|
||
Each browser tab gets a stable `clientId` (UUID generated on connection, persisted in `sessionStorage`).
|
||
|
||
The server maintains a monotonic `version: number` per project. Each successful `applyOps` increments it.
|
||
|
||
Each op-emitting client sends `expectedVersion` with its op batch. If the server's current version differs, the client gets a version-mismatch error and must resync.
|
||
|
||
### 11.1 Single-writer correctness
|
||
|
||
Because all mutations go through `applyOps`, which is a synchronized critical section per project, the version sequence is total and deterministic. No CRDT needed.
|
||
|
||
### 11.2 Lost ops on disconnect
|
||
|
||
If the SSE connection drops mid-edit, the client's local state may diverge from the server. On reconnect:
|
||
|
||
- Client sends its `lastSeenVersion` along with any buffered ops.
|
||
- Server replays patches between `lastSeenVersion + 1` and current version.
|
||
- Client applies them.
|
||
- Then client's buffered ops apply (with `expectedVersion = currentVersion`).
|
||
- Any conflict triggers `ops-rejected` and rollback.
|
||
|
||
If the gap is too large (>200 patches), server sends a full `snapshot-resync` instead of replay. Client replaces local state.
|
||
|
||
### 11.3 Multi-tab
|
||
|
||
Two tabs of the same user editing the same project:
|
||
|
||
- Each tab has its own `clientId`.
|
||
- Each receives the other's patches via SSE.
|
||
- Optimistic-local ops that conflict (e.g., both tabs delete the same block) — server applies first, rejects second, second client rolls back.
|
||
- Realistically rare in single-user MVP; system handles it.
|
||
|
||
---
|
||
|
||
## 12. Detection findings — not part of the model, but on the wire
|
||
|
||
Detection findings (assumptions, risks, validation issues) are *attached* to model elements but don't mutate the model graph. They flow through the same SSE channel as a separate patch kind:
|
||
|
||
```ts
|
||
{ kind: 'detection-finding', finding: Finding }
|
||
```
|
||
|
||
Findings are persisted as projection rows; they have their own lifecycle (open / dismissed / resolved). The sync pipeline's only job is delivery — findings show up in the rail, margin, or dock per the rules in [socrates.md §2](socrates.md).
|
||
|
||
Validation issues from `lib/sysml/validate.ts` work the same way: re-run on every successful apply, broadcast as `validation-issues` patches.
|
||
|
||
---
|
||
|
||
## 13. Failure modes & edge cases
|
||
|
||
| Failure | Behavior |
|
||
|---|---|
|
||
| **LLM down during proposal mediation** | Proposal opens with stub `impactSummary: { ready: false }`. UI shows "analysis pending." User can wait or approve without analysis (modal warns about the risk). Retry on reconnect. |
|
||
| **Network drop mid-edit** | Local optimistic state preserved. Reconnect → replay queue → resolve via §11.2. |
|
||
| **Server rejects an op the user already saw locally** | Roll back; show inline error at the affected element. |
|
||
| **Two devices delete the same element simultaneously** | First wins. Second receives `ops-rejected: element not found`. |
|
||
| **User edits a chip whose underlying element was deleted remotely** | Chip is stripped on patch arrival; user sees their edit "vanish." We surface a margin note: "`Foo` was removed elsewhere." |
|
||
| **Stale snapshot in client** | Detected via version-skew → snapshot resync. |
|
||
| **Validation fails post-apply (rare; means a previously valid model became invalid)** | Apply is rolled back atomically. User sees "couldn't apply: [reason]." Should never happen with a correct validator — it's a defense-in-depth check. |
|
||
| **Proposal approval after underlying model has shifted** | Proposal apply re-validates against current snapshot. If ops no longer valid, proposal is auto-rejected with explanation; user must re-author. |
|
||
| **Hallucinated id in a Socrates-suggested op** | Caught at `applyOps` validation step; op rejected; Socrates re-prompted. |
|
||
| **Op batch partial failure** | Atomic — entire batch fails or succeeds. No partial-apply state. |
|
||
|
||
---
|
||
|
||
## 14. Module layout
|
||
|
||
```
|
||
lib/sync/
|
||
ops.ts ModelOp type, op constructors, op codecs
|
||
applyOps.ts The chokepoint — server-side
|
||
classify.ts Impact classification (low/medium/high)
|
||
text-to-ops.ts ProseMirror transaction → ModelOp[]
|
||
diagram-to-ops.ts React Flow events → ModelOp[]
|
||
ops-to-text.ts ModelOp[] → ProseMirror transaction (for remote patches)
|
||
ops-to-diagram.ts ModelOp[] → React Flow node/edge updates
|
||
patches.ts SyncPatch type, encode/decode
|
||
sse-server.ts Per-project SSE channel; fan-out
|
||
sse-client.ts Client-side subscriber; reconnect; replay
|
||
client-state.ts Optimistic-local state, tempId tracking, rollback
|
||
reconcile.ts Glue: receive patches → apply locally with cursor preservation
|
||
```
|
||
|
||
This layout maps cleanly onto M5 of the build sequence.
|
||
|
||
---
|
||
|
||
## 15. Verification
|
||
|
||
End-to-end tests that exercise the full sync stack:
|
||
|
||
1. **Single-tab smoke:** open editor; insert a chip via slash-menu; verify diagram updates, rail updates, DB has the new block, version bumped.
|
||
2. **Optimistic-local rollback:** simulate a server rejection; verify local state rolls back cleanly; verify error message at affected element.
|
||
3. **Multi-tab convergence:** open same project in two tabs; edit in tab A; verify tab B's canvas updates within 500ms.
|
||
4. **Multi-tab conflict:** delete the same block in both tabs simultaneously; verify one wins, other gets `ops-rejected`, both end in same state.
|
||
5. **Disconnect-replay:** disconnect SSE; make 3 local ops; reconnect; verify all 3 apply.
|
||
6. **Disconnect-snapshot:** disconnect SSE; trigger a 200+-op gap (simulated); reconnect; verify snapshot resync replaces local state.
|
||
7. **Cursor preservation:** caret in middle of paragraph; remote op renames a chip in the same paragraph; verify caret position unchanged.
|
||
8. **Proposal flow:** delete a high-dep block; verify proposal opens, model not yet mutated; approve; verify ops apply atomically; verify changelog reasoning is captured.
|
||
9. **Paste of structured chips:** paste content with 3 valid chip refs and 2 invalid; verify 3 keep, 2 strip with margin notes.
|
||
10. **Position-drag flood:** drag a node continuously for 3 seconds; verify only one `update-block` op fires (on dragEnd); verify all tabs converge on final position.
|
||
|
||
Round-trip property tests on `text-to-ops ∘ ops-to-text` and `diagram-to-ops ∘ ops-to-diagram` for the structural ops we expect in real usage.
|
||
|
||
---
|
||
|
||
## 16. Open questions
|
||
|
||
- **Conflict-free chip-text collisions.** Two tabs simultaneously rename the same block to different labels. First wins; second gets rolled back. Is "last write wins" acceptable, or do we need a merge UI? *MVP: last write wins.*
|
||
- **Op coalescing.** Should consecutive `update-block { position }` ops within N ms be coalesced server-side to reduce changelog noise? *Probably yes, but defer until we see real changelog growth.*
|
||
- **Soft vs. hard validation on apply.** If a soft warning appears post-apply (e.g., T2 — block becomes unused), do we still apply? *Yes — soft warnings don't block.* If a hard error appears, it's a validator bug. *Defense-in-depth rollback.*
|
||
- **Detection-job triggering.** Currently debounced 3s after last apply. Should certain ops (e.g., `add-requirement`) trigger detection immediately? *Maybe — defer to M8 calibration.*
|
||
- **Cross-tab `clientId` reuse.** Each tab gets its own. Is there a use case where two tabs of the same user should share a clientId for self-echo filtering? *No — each is independent.*
|
||
- **History compaction.** The changelog grows monotonically. At what size do we compact (e.g., snapshot-and-prune ops older than 90 days)? *Defer to Phase 2 — branching introduces real history needs.*
|