fix: end the stream at [DONE] and drop orphaned tool results
Two independent bugs made Qoder turns hang or fail outright. 1. The SSE read loop never terminated on the sentinel. `data: [DONE]` broke only the inner line loop, so the outer loop kept awaiting reader.read() until the socket closed. Qoder's gateway holds the body open after the sentinel, so a reply that had already streamed in full produced no `done` event: the agent sat there with the complete answer buffered and no error, until something else timed the connection out. The sentinel also arrives wrapped in an envelope (`body: "[DONE]"`), which likewise only did `continue`, so both forms now end the loop. The reader is cancelled once the reply is complete instead of being left holding the connection. 2. Aborting a turn produced an invalid request. Assistant messages with stopReason error/aborted are skipped, but the tool result that followed was still emitted, leaving a `tool` message whose tool_call_id had no matching tool_calls entry. Upstreams reject that with "tool must follow a message with tool_calls" -- the same failure the null-content workaround above it already guards against -- so after one cancelled tool call, every later request in that session failed. Tool calls belonging to a dropped turn are now recorded and their results dropped with them. Both fixes are covered by regression tests that fail on the current code: the two stream tests time out at the 8s limit, and the transform tests produce an orphaned tool message.
This commit is contained in:
+21
-2
@@ -273,7 +273,14 @@ export function streamQoder(
|
||||
|
||||
stream.push({ type: "start", partial: output });
|
||||
|
||||
while (true) {
|
||||
// `data: [DONE]` is the end of the response. Break the read loop too, not
|
||||
// just the line loop: Qoder's gateway keeps the HTTP body open after the
|
||||
// sentinel, so waiting for `done` from reader.read() hung until the
|
||||
// server or the OS eventually closed the socket. The full reply had
|
||||
// already been streamed by then, so the agent looked stuck with no error.
|
||||
let sawDone = false;
|
||||
|
||||
while (!sawDone) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
|
||||
@@ -290,6 +297,7 @@ export function streamQoder(
|
||||
|
||||
const dataStr = line.substring(5).trim();
|
||||
if (dataStr === "[DONE]") {
|
||||
sawDone = true;
|
||||
break;
|
||||
}
|
||||
|
||||
@@ -300,7 +308,14 @@ export function streamQoder(
|
||||
}
|
||||
|
||||
const innerStr = envelope.body;
|
||||
if (!innerStr || innerStr === "[DONE]") continue;
|
||||
// The gateway sends the sentinel wrapped in an envelope
|
||||
// (`body: "[DONE]"`) as well as bare, and both mean the reply is
|
||||
// over, so both have to end the read loop.
|
||||
if (innerStr === "[DONE]") {
|
||||
sawDone = true;
|
||||
break;
|
||||
}
|
||||
if (!innerStr) continue;
|
||||
|
||||
const inner = JSON.parse(innerStr);
|
||||
if (inner.id) output.responseId = inner.id as string;
|
||||
@@ -472,6 +487,10 @@ export function streamQoder(
|
||||
}
|
||||
}
|
||||
|
||||
// Stop reading and let the connection go once the reply is complete.
|
||||
// Without this the body stays open until the server times it out.
|
||||
await reader.cancel().catch(() => {});
|
||||
|
||||
if (thinkingParser) {
|
||||
thinkingParser.finalize();
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user