Implement Sophia MVP stages 4-7 (introspection, queries, persistence, polish)

Brings the system from "engrams cluster + form synapses" to a complete
end-to-end demo: ingest text, watch it cluster, ask questions, restart
with state intact.

- Stage 4: birth introspection (taxonomy/goals/open_questions via LLM,
  bounded by the global parallel-op semaphore), per-engram memory log,
  click-to-inspect side panel.
- Stage 5: queries as conversations. POST /api/galaxy/:id/query embeds
  the question, materializes a pinned Query-Engram at the donut center,
  runs broadcast retrieval (global cosine scan + 1-hop synaptic
  expansion with attenuation) and fans out responder LLM calls. The
  integrator runs every 2s on accumulated snippets and streams the
  refining answer back over SSE; responders briefly transition to
  Conversing on the WS bus so the right dots light up.
- Stage 6: snapshot persistence. sled-backed store keyed by galaxy id,
  JSON-encoded values (bincode chokes on internally-tagged enums like
  Manifest/MemoryKind), 60s periodic snapshot task, hydrate-on-boot,
  DELETE /api/galaxy/:id wired through. State survives kill -9.
- Stage 7: HUD additions (sim ticks/sec, LLM queue depth, FPS) via a
  new GET /api/stats polled at 1Hz. `sophia demo` subcommand boots the
  server then auto-ingests a 50-paragraph corpus baked into the binary
  with include_str!. README quickstart added.

Token caps for query_responder/integrator bumped (gemma-4-e4b is a
thinking model — output budget must cover hidden reasoning + visible
answer, otherwise content comes back empty). Pinned engrams skip
physics; their tick scheduling is also skipped at materialization so
they stay perfectly still at the donut center.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

crates/sophia-store/Cargo.toml

@@ -10,5 +10,8 @@ authors.workspace = true
 sophia-core = { workspace = true }
 
 anyhow      = { workspace = true }
+thiserror   = { workspace = true }
 tracing     = { workspace = true }
 serde       = { workspace = true }
+serde_json  = { workspace = true }
+sled        = { workspace = true }

crates/sophia-store/src/lib.rs

@@ -1,7 +1,109 @@
-//! Persistence: sled-backed event log + snapshot.
+//! Persistence: sled-backed snapshot store (Stage 6).
 //!
-//! See `docs/system-analysis.md` §13.3 (event-sourced + graph read model).
-//! Per the doc: "the event log == the canonical memory store" — anything
-//! memorable is persisted; transient state is not.
+//! Each galaxy is one key in a sled `Db`: key = the galaxy's UUID bytes,
+//! value = a JSON-encoded `GalaxySnapshot`. The bin's snapshot task
+//! overwrites the value every 60 s; on boot we iterate the tree, decode
+//! each entry, and hand them to the sim for hydration.
 //!
-//! Stage 0: stub. Real content arrives in Stage 6.
+//! **Why JSON, not bincode**: `Manifest` and `MemoryKind` are
+//! internally-tagged enums (`#[serde(tag = "kind")]`). Internally tagged
+//! enums require a self-describing format on the wire — bincode calls
+//! `deserialize_any` to read the tag, which it doesn't support. JSON is
+//! self-describing and handles them natively. At MVP scale (a few hundred
+//! engrams) the verbosity is irrelevant; if storage grows we can swap in
+//! msgpack/cbor without touching the sim.
+//!
+//! See `docs/system-analysis.md` §13.3: the design calls for an event log
+//! with periodic snapshots, but per the same section "no
+//! event-replay-from-genesis." For MVP that means we only ever need the
+//! snapshot side — no event log, no per-event append. If the model later
+//! needs time-travel debugging we can add the log without disturbing the
+//! snapshot path (different sled tree).
+//!
+//! Sled defaults to `flush_every_ms = 500`, so even a hard kill loses at
+//! most ~half a second of writes — acceptable for snapshot data that's
+//! already 60 s stale by definition.
+
+use std::path::Path;
+
+use sophia_core::{GalaxyId, GalaxySnapshot};
+use thiserror::Error;
+use tracing::{debug, warn};
+
+#[derive(Debug, Error)]
+pub enum StoreError {
+    #[error("sled error: {0}")]
+    Sled(#[from] sled::Error),
+    #[error("json error: {0}")]
+    Json(#[from] serde_json::Error),
+}
+
+/// Cheaply cloneable handle to the on-disk store. Internally holds a
+/// `sled::Db`; cloning shares the same database connection.
+#[derive(Clone)]
+pub struct Store {
+    db: sled::Db,
+}
+
+impl Store {
+    /// Open or create the store at `path`. The path is treated as a
+    /// directory — sled creates it (and the sled internal files inside) if
+    /// needed.
+    pub fn open(path: impl AsRef<Path>) -> Result<Self, StoreError> {
+        let db = sled::open(path.as_ref())?;
+        Ok(Self { db })
+    }
+
+    /// Iterate every stored galaxy snapshot. Decode failures are logged and
+    /// skipped — a corrupt entry shouldn't take down the boot path.
+    pub fn list_galaxies(&self) -> Result<Vec<GalaxySnapshot>, StoreError> {
+        let mut out = Vec::new();
+        for kv in self.db.iter() {
+            let (k, v) = kv?;
+            match serde_json::from_slice::<GalaxySnapshot>(&v) {
+                Ok(snap) => out.push(snap),
+                Err(e) => {
+                    warn!("skipping corrupt galaxy entry (key={} bytes): {e}", k.len());
+                }
+            }
+        }
+        Ok(out)
+    }
+
+    /// Insert or overwrite the stored snapshot for one galaxy. Sled flushes
+    /// asynchronously (`flush_every_ms`); we don't `flush()` here because
+    /// snapshot writes happen on a 60 s cadence and a half-second of in-mem
+    /// buffering is fine.
+    pub fn save_galaxy(&self, snapshot: &GalaxySnapshot) -> Result<(), StoreError> {
+        let bytes = serde_json::to_vec(snapshot)?;
+        self.db.insert(galaxy_key(snapshot.galaxy.id), bytes)?;
+        debug!(
+            "saved galaxy {:?} ({} engrams, {} synapses)",
+            snapshot.galaxy.id,
+            snapshot.engrams.len(),
+            snapshot.synapses.len()
+        );
+        Ok(())
+    }
+
+    /// Remove a galaxy from disk. Idempotent — missing keys are not an error.
+    pub fn delete_galaxy(&self, id: GalaxyId) -> Result<(), StoreError> {
+        self.db.remove(galaxy_key(id))?;
+        Ok(())
+    }
+
+    /// Force a synchronous flush. Called on graceful shutdown (and useful
+    /// in tests). Otherwise sled's background flusher is sufficient.
+    pub fn flush(&self) -> Result<(), StoreError> {
+        self.db.flush()?;
+        Ok(())
+    }
+}
+
+/// 16-byte key for one galaxy. Using the raw UUID bytes (rather than the
+/// hyphenated string) keeps the key compact and lexicographically equivalent
+/// to UUID v7's time order — which makes `Db::iter()` walk galaxies in
+/// creation order without us doing any sorting.
+fn galaxy_key(id: GalaxyId) -> [u8; 16] {
+    *id.0.as_bytes()
+}