Commit Graph
2 Commits
Author SHA1 Message Date
liyanan 37ce615e62 test(stream): assert usage input excludes cached/written tokens
Follow the fix in stream.ts: prompt_tokens (42) includes cached_tokens (5),
so input = 42 - 5 - 10 = 27. Use cache_write_tokens (not cacheable_tokens)
for cacheWrite, and keep cacheable_tokens:99 in the payload to prove it is
no longer mapped to cacheWrite.
2026-07-30 12:42:27 +08:00
liyanan b8980b68b8 fix(stream): stop silently swallowing upstream errors and session-lock stalls
Three stability fixes for the streaming layer. Together they eliminate
the "agent silently stops mid-task" failure mode users hit during agentic
loops.

1. Surface upstream errors instead of swallowing them

The SSE parsing loop wrapped every line in try { ... } catch {}, which
silently dropped the throw raised when Qoder returned an error envelope
(e.g. 406 Session blocked, provider_error). The stream then completed
with empty content and stopReason "stop", so pi saw a clean stop instead
of an error and never retried or informed the user.

The catch now only skips genuinely malformed SSE lines (SyntaxError) and
re-throws everything else so the outer handler can set stopReason "error"
+ errorMessage. A QODER_DEBUG env var logs skipped lines for diagnosis.

2. Stop overwriting the real finish_reason with "stop"

At end-of-stream the code unconditionally did:
    output.stopReason = toolCalls.length ? "toolUse" : "stop"
clobbering any meaningful finish_reason ("length", "content_filter", ...)
captured earlier. Now only toolUse is forced when tool calls are present;
otherwise the upstream finish_reason is preserved (defaults to "stop").
The done event reason cast now includes "length".

3. Give each request a fresh server-side session id

session_id was stableHash("qoder-session", userID, qoderModel) —
constant across all requests for a given user+model. Qoder enforces a
per-session lock and replies 406 Session blocked when a second request
arrives before the previous one fully drains server-side, which happens
routinely in tight agentic tool loops. Appending crypto.randomUUID()
makes every request use a fresh session. We send full conversation
history on each request, so losing server-side session continuity is
safe.

4. Capture upstream usage, responseId and responseModel

The provider reported all-zero usage and no responseId/responseModel.
Now the finish chunk usage (prompt/completion/total tokens, cache
read/write) is mapped into output.usage, and id/model into
responseId/responseModel, so pi sees real token accounting and can
correlate responses.

Add stream.test.ts with regression coverage for the 406 path, the
finish_reason preservation, the toolUse stop reason, and usage/response
metadata capture.
2026-07-25 20:55:40 +08:00