perf(agent): cache the MCP tool list per client (F1)
Fix F1 of plans/2026-07-11-nomos-agent-code-review.md, the last item. buildTools called listToolsFull (a tools/list MCP round-trip) at the start of EVERY chat turn, including every auto-continuation resume — the tool list is static for the lifetime of one MCP connection, changing only when the api process re-registers tools (a restart, which this client already detects and reacts to via reconnectLocked). Re-fetching it every single turn was avoidable network+parsing work on the hot path. mcpClient now caches the parsed tool list after its first fetch, guarded by its own mutex (kept separate from the request-serializing mu so a cache check never contends with an in-flight doRequest call). reconnectLocked clears the cache — an api restart may have changed what's registered, so a stale cache would be wrong, not just slow. fleetSnapshot's get_health_summary call is deliberately left uncached — it's meant to be "as of now." Since each session gets its own client (the per-session pool from the concurrency work), this caches per-task-conversation rather than globally: a task's FIRST turn still pays the round-trip, every turn after reuses the cached list — which is exactly the case that mattered (long-running, heavily-autonomous tasks with many auto-continuation resumes). Verified live via the api's request log: a brand-new session's first turn made 3 MCP calls (initialize, tools/list, get_health_summary); a second turn on the SAME session made exactly 1 (only get_health_summary) — tools/list correctly skipped. This completes the implementation order in plans/2026-07-11-nomos-agent-code-review.md — every A/B/D/E/F finding from the review (excluding C1, explicitly deferred per operator instruction) is now fixed, tested, and verified live. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -721,7 +721,23 @@ func (a *agent) buildTools(sessionID string) ([]openai.ChatCompletionToolParam,
|
||||
return tools, nil
|
||||
}
|
||||
|
||||
// listToolsFull returns the MCP server's tool list, cached on this client
|
||||
// after the first call (see mcpClient.toolsCache). Fix F1 of
|
||||
// plans/2026-07-11-nomos-agent-code-review.md: buildTools calls this at the
|
||||
// start of every chat turn, including every auto-continuation resume — the
|
||||
// tool list is static for the lifetime of one MCP connection, so re-fetching
|
||||
// it every single time was avoidable network+parsing work on the hot path.
|
||||
// Cache invalidates on reconnectLocked (an api restart may change what's
|
||||
// registered).
|
||||
func (c *mcpClient) listToolsFull() ([]toolDef, error) {
|
||||
c.toolsMu.Lock()
|
||||
if c.toolsCache != nil {
|
||||
cached := c.toolsCache
|
||||
c.toolsMu.Unlock()
|
||||
return cached, nil
|
||||
}
|
||||
c.toolsMu.Unlock()
|
||||
|
||||
resp, err := c.doRequest("tools/list", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -744,5 +760,9 @@ func (c *mcpClient) listToolsFull() ([]toolDef, error) {
|
||||
InputSchema: t.InputSchema,
|
||||
}
|
||||
}
|
||||
|
||||
c.toolsMu.Lock()
|
||||
c.toolsCache = out
|
||||
c.toolsMu.Unlock()
|
||||
return out, nil
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user