cat-shark/pi-commandcode-provider
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(e2e): default the GOAT vision phase to gpt-5.6-luna

Browse Source 
google/gemini-3.7-flash currently fails on the Provider API with a
zero-data-retention routing 404 for every request, which made the GOAT
live suite red regardless of provider changes. GPT-5.6 Luna is available
on every plan, accepts image input, and completes the vision request.
This commit is contained in:
Patrick Wozniak
2026-09-02 00:22:26 +02:00
parent d3c9832d28
commit a1bfafa556
2 changed files with 4 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -209,7 +209,7 @@ COMMANDCODE_E2E_GOAT_API_KEY_FILE=/path/to/goat-key \
Each profile runs with an isolated Pi agent directory and asserts transport selection, reasoning across turns, quota plan identity, abort handling, tool calls, and the packed npm artifact. Go must select `generate` and reject unsupported images; GOAT must select `provider` and complete a live vision request. The profile-specific `*_API_KEY` environment variables are also supported for CI secrets, but key files are preferred for local use.
The Go profile defaults to DeepSeek V4 Flash; GOAT defaults to Grok 4.6 because its Provider API stream exposes reasoning consistently across consecutive turns. Override them with `COMMANDCODE_E2E_GO_MODEL`, `COMMANDCODE_E2E_GOAT_MODEL`, or `COMMANDCODE_E2E_PROVIDER_MODEL`. A successful live Anthropic `/provider/v1/messages` test requires a paid account whose plan includes the selected Claude model.
The Go profile defaults to DeepSeek V4 Flash; GOAT defaults to Grok 4.6 because its Provider API stream exposes reasoning consistently across consecutive turns. Override them with `COMMANDCODE_E2E_GO_MODEL`, `COMMANDCODE_E2E_GOAT_MODEL`, or `COMMANDCODE_E2E_PROVIDER_MODEL`. The GOAT vision phase defaults to GPT-5.6 Luna and can be overridden with `COMMANDCODE_E2E_GOAT_VISION_MODEL`. A successful live Anthropic `/provider/v1/messages` test requires a paid account whose plan includes the selected Claude model.
See [CONTRIBUTING.md](CONTRIBUTING.md) for local setup and tests. See [RELEASE.md](RELEASE.md) for the release process.
tests/test-live-e2e.mjs
+3 -1
View File
@@ -40,7 +40,9 @@ const expectedPlan =
: testProfile === "provider"
? "provider"
: undefined
const goatVisionModel = process.env.COMMANDCODE_E2E_GOAT_VISION_MODEL ?? "google/gemini-3.7-flash"
// GPT-5.6 Luna is available on every plan and has a ZDR-capable upstream;
// Gemini 3.7 Flash currently fails with a zero-data-retention routing 404.
const goatVisionModel = process.env.COMMANDCODE_E2E_GOAT_VISION_MODEL ?? "gpt-5.6-luna"
const marker = "commandcode-live-e2e-ok"
function findPiBinary() {