fix(agent): bound conversation history replayed to the LLM (A2)
Fix A2 of plans/2026-07-11-nomos-agent-code-review.md. chatWith replayed a
session's ENTIRE message history into the LLM's context on EVERY turn, no
windowing, no token budget — confirmed against a documented production case
(a single turn with 70 tool calls, messages up to 106KB). Every subsequent
turn of a long-running or heavily-autonomous task re-sent that ever-growing
history in full — a real cost/latency/eventual-context-limit risk for
exactly the tasks this system runs longest (many auto-continuation cycles).
Design call (flagged in the review as needing one before implementation):
a fixed-size window for LLM replay specifically, not the UI's own transcript
view. Simplest option that still keeps roughly the current task's working
context; a token-aware trim or LLM-summarize-on-drop are documented as
stretch options if 30 proves insufficient in practice.
- store.go: new getRecentMessages(ctx, sessionID, limit) — last `limit`
messages in chronological order, plus whether older ones were omitted.
getMessages (used by the UI's GET /sessions/{id}) is untouched and stays
unbounded — the operator should still see a task's full history regardless
of length; only what gets sent to the model is bounded.
- agent.go: chatWith uses getRecentMessages(sessionID, historyWindowSize=30)
instead of the unbounded getMessages. When truncated, injects a system
note telling the model explicitly that older turns exist but aren't shown,
so it checks upsert_knowledge/search_knowledge rather than assuming
something wasn't done just because it isn't visible.
New cmd/nomos/store_test.go: real Postgres integration tests (mirroring
internal/db/integration_test.go's throwaway-database pattern, guarded by
OIKOS_TEST_DATABASE_URL). TestGetRecentMessages_Truncation is the direct
proof for this fix (35 messages → 30 returned, correctly ordered,
truncated=true; 5 messages → all 5, truncated=false) — both cases run
against a fully-migrated database, not mocked. Also added
TestProposePlan_AppendVsReplace, closing part of the review's test-coverage
finding (E) by permanently regression-testing the earlier append-vs-replace
plan fix (commit 5384499), which had only been verified manually until now.
Verified live: inflated a real session to 42 persisted messages via direct
SQL, then continued it with a real chat call — the turn proceeded normally
(multiple real tool-call iterations, no crash, no context-length error);
nomos stayed healthy throughout. A3's incremental persistence separately
confirmed to have caught the 7 real tool calls made before the client
connection was cut, cleanly closing out both fixes' interaction.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -215,6 +215,12 @@ func (s *store) listSessions(ctx context.Context) ([]session, error) {
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// getMessages returns a session's ENTIRE message history, unbounded — used
|
||||
// for the UI's own transcript view (GET /sessions/{id}), where the operator
|
||||
// should be able to see everything a task has done regardless of how long
|
||||
// it's run. For LLM replay, see getRecentMessages: sending the operator's
|
||||
// full transcript is fine; sending the model's full transcript on every
|
||||
// single turn is not (see getRecentMessages's doc comment).
|
||||
func (s *store) getMessages(ctx context.Context, sessionID string) ([]message, error) {
|
||||
if s == nil {
|
||||
return nil, nil
|
||||
@@ -238,6 +244,52 @@ func (s *store) getMessages(ctx context.Context, sessionID string) ([]message, e
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// getRecentMessages returns the most recent `limit` messages for sessionID,
|
||||
// in chronological order, plus whether older messages exist beyond that
|
||||
// window. Used specifically for LLM replay (chatWith): without a bound,
|
||||
// every turn re-sent the ENTIRE session history into the model's context,
|
||||
// unconditionally growing with every turn — a real, observed-in-production
|
||||
// cost/latency/eventual-context-limit risk for exactly the long-running,
|
||||
// heavily-autonomous tasks (many auto-continuation cycles) this system is
|
||||
// built to run longest. Fetches limit+1 rows to detect "there's more"
|
||||
// without a separate COUNT query.
|
||||
func (s *store) getRecentMessages(ctx context.Context, sessionID string, limit int) (msgs []message, truncated bool, err error) {
|
||||
if s == nil {
|
||||
return nil, false, nil
|
||||
}
|
||||
rows, qerr := s.pool.Query(ctx,
|
||||
`SELECT id, session_id, role, content, created_at FROM agent_messages
|
||||
WHERE session_id=$1 ORDER BY created_at DESC LIMIT $2`,
|
||||
sessionID, limit+1)
|
||||
if qerr != nil {
|
||||
return nil, false, qerr
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []message
|
||||
for rows.Next() {
|
||||
var m message
|
||||
if err := rows.Scan(&m.ID, &m.SessionID, &m.Role, &m.Content, &m.CreatedAt); err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
out = append(out, m)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
|
||||
truncated = len(out) > limit
|
||||
if truncated {
|
||||
out = out[:limit]
|
||||
}
|
||||
// Rows came back newest-first (for the LIMIT to bound the right end);
|
||||
// reverse to chronological order for replay.
|
||||
for i, j := 0, len(out)-1; i < j; i, j = i+1, j-1 {
|
||||
out[i], out[j] = out[j], out[i]
|
||||
}
|
||||
return out, truncated, nil
|
||||
}
|
||||
|
||||
func (s *store) deleteSession(ctx context.Context, id string) error {
|
||||
if s == nil {
|
||||
return nil
|
||||
|
||||
Reference in New Issue
Block a user