fix: reuse pi session id as qoder session_id to enable prompt caching

A fresh random session_id on every request meant the Qoder server could
never match a previous cache entry, so cacheRead was ~0% in all agentic
sessions. Ablation testing showed a stable session_id alone raises the
turn-2 cache hit rate from 0% to ~99.6% of prompt tokens (affinity
headers alone contribute nothing). The previously observed "406 Session
blocked" no longer reproduces with a per-pi-session stable id.
This commit is contained in:
liyanan
2026-07-28 10:21:20 +08:00
parent 273368fc0e
commit 85ec02ca8a
+7 -6
View File
@@ -153,12 +153,13 @@ export function streamQoder(
}
}
// Each request gets a fresh server-side session id. Reusing a stable
// session id across back-to-back agentic requests triggers Qoder's
// per-session lock and the server replies `406 Session blocked` mid
// stream. We send full conversation history on every request, so a unique
// session id is safe and avoids the lock entirely.
const sessionID = `${stableHash("qoder-session", userID, qoderModel)}-${crypto.randomUUID()}`;
// Use a stable session id when pi provides one (per agent session) so
// the Qoder server can maintain prompt cache affinity across consecutive
// requests. Fall back to a random id only when no sessionId is available.
const stablePart = stableHash("qoder-session", userID, qoderModel);
const sessionID = options?.sessionId
? `${stablePart}-${options.sessionId}`
: `${stablePart}-${crypto.randomUUID()}`;
let maxTokens = 32768;
if (maxOutputTokens > 0) {