MVP M8: background detection — assumptions, risks, inconsistencies

The editor now runs Socrates' three Phase-0-validated detection prompts
against the live model, persists the findings, and surfaces them in a
new FindingsPanel beside the IssuesPanel. Click any finding to focus its
linked element across rail + diagram. Re-detect after model edits to
refresh against the new state.

apps/web/lib/llm/prompts/socrates
- detect-assumptions.md / detect-risks.md / detect-inconsistencies.md
  promoted verbatim from phase-0 (Phase 0 corpus validated them 10/10).

apps/web/lib/llm/detect.ts
- Three sequential detection passes (parallel was OOM-prone on 4B local
  models — Phase 0 lesson). Each pass uses the Phase 0 JSON schema with
  jsonObjectMode fallback + chatJSON repair-retry. Fail-soft per pass:
  one busted pass returns [] rather than blowing up the whole detect.
- post-validate strips hallucinated element refs (drops findings whose
  refs ALL fail to resolve; keeps findings with zero refs since some
  inconsistencies are genuinely about absences).

apps/web/prisma/schema.prisma
- Finding table: kind / text / linkedElementIds (JSON) / confidence /
  severity / validationCode / status / modelVersion / provider / model.
- ResearchFinding table reserved for the Tavily integration that comes
  next — schema in place so we don't have to migrate again.

apps/web/lib/db/repo.ts
- listOpenFindings(projectId), replaceFindings(...) — replaceFindings
  wipes prior open findings in a transaction and writes the new set so
  re-detect doesn't accumulate stale findings.

apps/web/app/api/projects/[projectId]/findings/route.ts
- GET returns persisted open findings.
- POST runs detect, persists, returns findings + meta (provider, model,
  durationMs, strippedRefs, droppedFindings).

apps/web/components/editor/FindingsPanel.tsx
- New panel, anchored bottom-right just left of IssuesPanel. Shows
  count summary (asm / risk / inc), detect / re-detect button, list
  grouped by kind (inconsistencies first, then risks, then assumptions),
  per-finding glyph + tag + severity + confidence + linked refs.
- Click a finding row → focus its first linked element via the same
  setFocusBlockId path the rail and IssuesPanel already use.
- "stale" indicator when the model version has advanced past the one the
  findings were detected against.

EditorShell wires version + projectId through to FindingsPanel.

Smoke-tested end-to-end: 12 findings returned (5 asm / 4 risk / 3 inc),
0 hallucinated refs stripped, ~37s on local gemma-4-e4b. Sample
assumption "students are willing to engage with an AI tutor that is
programmed to refuse providing complete solutions" — specific to
Aristotle's refusal_policy, not a generic startup truism.

Deferred to follow-ups: inline rail/diagram badges from findings,
auto-detect-on-save, Tavily research, experiment modal.
This commit is contained in:
2026-04-30 00:43:27 +02:00
parent 4b8e3f04ee
commit 4e725c0b2b
10 changed files with 1010 additions and 1 deletions

View File

@@ -148,6 +148,114 @@ async function ensureSeeded(projectId: string): Promise<{ projectId: string }> {
return { projectId };
}
// ─── Findings (M8) ───────────────────────────────────────────────────────
import type { Finding as DetectedFinding } from "../llm/detect";
export interface StoredFinding {
id: string;
kind: string;
text: string;
linkedElementIds: string[];
confidence: number;
severity: string | null;
validationCode: string | null;
status: string;
modelVersion: number;
provider: string | null;
llmModel: string | null;
createdAt: Date;
}
export async function listOpenFindings(projectId: string): Promise<StoredFinding[]> {
const rows = await prisma.finding.findMany({
where: { projectId, status: "open" },
orderBy: [{ kind: "asc" }, { createdAt: "desc" }],
});
return rows.map(rowToFinding);
}
export async function replaceFindings(
projectId: string,
findings: DetectedFinding[],
modelVersion: number,
provider: string,
llmModel: string
): Promise<StoredFinding[]> {
return prisma.$transaction(async tx => {
// Wipe previous open findings — re-running detection supersedes them.
await tx.finding.deleteMany({ where: { projectId, status: "open" } });
if (findings.length === 0) return [];
const created = await Promise.all(
findings.map(f =>
tx.finding.create({
data: {
projectId,
kind: f.kind,
text: f.text,
linkedElementIds: JSON.stringify(f.linkedElementIds),
confidence: f.confidence,
severity: f.severity ?? null,
validationCode: f.validationCode ?? null,
status: "open",
modelVersion,
provider,
llmModel,
},
})
)
);
return created.map(rowToFinding);
});
}
export async function dismissFinding(findingId: string): Promise<void> {
await prisma.finding.update({
where: { id: findingId },
data: { status: "dismissed" },
});
}
interface FindingRow {
id: string;
kind: string;
text: string;
linkedElementIds: string;
confidence: number;
severity: string | null;
validationCode: string | null;
status: string;
modelVersion: number;
provider: string | null;
llmModel: string | null;
createdAt: Date;
}
function rowToFinding(row: FindingRow): StoredFinding {
let linked: string[] = [];
try {
const parsed = JSON.parse(row.linkedElementIds);
if (Array.isArray(parsed)) linked = parsed.filter((x): x is string => typeof x === "string");
} catch {
linked = [];
}
return {
id: row.id,
kind: row.kind,
text: row.text,
linkedElementIds: linked,
confidence: row.confidence,
severity: row.severity,
validationCode: row.validationCode,
status: row.status,
modelVersion: row.modelVersion,
provider: row.provider,
llmModel: row.llmModel,
createdAt: row.createdAt,
};
}
// ─── Socrates threads ────────────────────────────────────────────────────
export async function getActiveThread(projectId: string): Promise<{ id: string; title: string | null; messages: Array<{ id: string; role: string; content: string; createdAt: Date }> } | null> {