fix: reuse pi session id as qoder session_id to enable prompt caching
A fresh random session_id on every request meant the Qoder server could never match a previous cache entry, so cacheRead was ~0% in all agentic sessions. Ablation testing showed a stable session_id alone raises the turn-2 cache hit rate from 0% to ~99.6% of prompt tokens (affinity headers alone contribute nothing). The previously observed "406 Session blocked" no longer reproduces with a per-pi-session stable id.
This commit is contained in:
+7
-6
@@ -153,12 +153,13 @@ export function streamQoder(
|
||||
}
|
||||
}
|
||||
|
||||
// Each request gets a fresh server-side session id. Reusing a stable
|
||||
// session id across back-to-back agentic requests triggers Qoder's
|
||||
// per-session lock and the server replies `406 Session blocked` mid
|
||||
// stream. We send full conversation history on every request, so a unique
|
||||
// session id is safe and avoids the lock entirely.
|
||||
const sessionID = `${stableHash("qoder-session", userID, qoderModel)}-${crypto.randomUUID()}`;
|
||||
// Use a stable session id when pi provides one (per agent session) so
|
||||
// the Qoder server can maintain prompt cache affinity across consecutive
|
||||
// requests. Fall back to a random id only when no sessionId is available.
|
||||
const stablePart = stableHash("qoder-session", userID, qoderModel);
|
||||
const sessionID = options?.sessionId
|
||||
? `${stablePart}-${options.sessionId}`
|
||||
: `${stablePart}-${crypto.randomUUID()}`;
|
||||
|
||||
let maxTokens = 32768;
|
||||
if (maxOutputTokens > 0) {
|
||||
|
||||
Reference in New Issue
Block a user