cat-shark/pi-commandcode-provider
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #71 from patlux/integrate/backlog-2026-09

Browse Source 
Integrate contributor PRs and backlog fixes (0.6.1)
This commit is contained in:
Patrick Wozniak
2026-09-02 00:26:23 +02:00
committed by GitHub
parent 0acf311774 a1bfafa556
commit 43fbe55a1c
27 changed files with 1103 additions and 122 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/refresh-model-catalog/SKILL.md
+78
View File
@@ -0,0 +1,78 @@
---
name: refresh-model-catalog
description: Use when adding or removing Command Code models, refreshing the model catalog snapshot (image, reasoning, effort, output-limit metadata), updating display pricing, or refreshing test fixtures in pi-commandcode-provider.
---
# Refresh Model Catalog
Use this skill whenever the Command Code model catalog changes: new or retired models, changed reasoning efforts, output limits, or pricing. All commands run from the repository root and work on Windows and Linux.
## Core rules
- Do not commit, tag, push, or publish unless the user explicitly asks in the current conversation.
- Pricing is manually reviewed: temporary promotions and long-context tiers require explicit review of the official pricing page. Never copy prices blindly from the API.
- Keep the change focused: one refresh per PR, no unrelated refactors.
- Follow [CONTRIBUTING.md](../../../CONTRIBUTING.md) for commit message rules.
## Workflow
### 1. Detect drift
```sh
npm run check:commandcode-catalog
```
This compares the repository snapshot against the latest published `command-code` npm package and reports added/removed models, changed efforts, and version drift. Use the report to scope the work.
### 2. Sync static model metadata
```sh
npm run sync:commandcode-catalog
```
Regenerates `src/commandcode-catalog.ts` and bumps the documented CLI version in `README.md`. Review the diff; the catalog also lists reasoning models without selectable efforts.
Never add efforts to the generated file by hand. Manual effort policy for reasoning models that upstream ships without levels lives in `src/commandcode-catalog-overrides.ts` and is merged at load time. When the sync report lists a model from that file under "New effort metadata", remove its override; `tests/test-models.ts` fails until you do.
### 3. Update display pricing (manual review)
Fetch <https://commandcode.ai/docs/resources/pricing-limits> and compare against `src/pricing.ts`:
- Add entries for new models and remove entries for retired models. Missing models silently display zero cost, so `MODEL_COSTS` must cover the full catalog.
- The pricing page's "Cache Read"/"Cache Write" columns map to `cacheRead`/`cacheWrite`; a "—" column means `0`.
- Update `PRICING_LAST_VERIFIED` to today's date.
- Add or update `TEMPORARY_PRICING` entries for promotions with an end date, so tests fail when they expire.
### 4. Refresh the test fixtures
```sh
node .agents/skills/refresh-model-catalog/scripts/refresh-model-ids.mjs
npx tsx .agents/skills/refresh-model-catalog/scripts/sync-pricing-fixture.ts
```
The first script snapshots the live model-id list into `tests/fixtures/commandcode-model-ids.json`; the second regenerates `tests/fixtures/commandcode-pricing.json` from `MODEL_COSTS`. The pricing test fails until `MODEL_COSTS` matches the catalog snapshot exactly.
### 5. Update test expectations
Adjust the model-specific assertions that the refresh invalidated, typically in:
- `tests/test-pricing.ts`: fixture date assertions, the `freeModels` set, and per-model rate assertions.
- `tests/test-models.ts`: image/reasoning/effort/output-limit assertions and catalog entry counts.
Do not weaken assertions to make them pass; update them to the verified upstream values.
### 6. Validate
```sh
npm run test:models
npm run test:pricing
npm run typecheck
npm run format:check
git diff --check
```
Run the full `npm test` before reporting the work as done when the environment allows it.
### 7. Document
Add entries to the `Unreleased` section of `CHANGELOG.md` covering new/retired models, effort changes, and pricing refreshes.
.agents/skills/refresh-model-catalog/scripts/refresh-model-ids.mjs
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env node
// Refreshes tests/fixtures/commandcode-model-ids.json from the live Command Code
// models API. Run from the repository root:
// node .agents/skills/refresh-model-catalog/scripts/refresh-model-ids.mjs
import { writeFile } from "node:fs/promises"
import { format, resolveConfig } from "prettier"
const MODELS_URL = "https://api.commandcode.ai/provider/v1/models"
const FIXTURE_PATH = new URL(
"../../../../tests/fixtures/commandcode-model-ids.json",
import.meta.url,
)
const response = await fetch(MODELS_URL)
if (!response.ok) {
throw new Error(`Failed to fetch Command Code models: ${response.status} ${response.statusText}`)
}
const body = await response.json()
if (body?.object !== "list" || !Array.isArray(body.data)) {
throw new Error("Expected a Command Code models list response")
}
const modelIds = body.data.map((model) => {
if (typeof model?.id !== "string" || model.id.length === 0) {
throw new Error("Expected each model entry to have a non-empty id")
}
return model.id
})
if (modelIds.length === 0) throw new Error("Command Code returned an empty model catalog")
const fixture = { fetchedAt: new Date().toISOString(), source: MODELS_URL, modelIds }
const options = await resolveConfig(new URL("../../../../.prettierrc.json", import.meta.url))
const contents = await format(JSON.stringify(fixture), {
...options,
filepath: "commandcode-model-ids.json",
})
await writeFile(FIXTURE_PATH, contents, "utf-8")
console.log(`Wrote ${modelIds.length} model ids to tests/fixtures/commandcode-model-ids.json`)
.agents/skills/refresh-model-catalog/scripts/sync-pricing-fixture.ts
+43
View File
@@ -0,0 +1,43 @@
// Regenerates tests/fixtures/commandcode-pricing.json from src/pricing.ts so the
// snapshot always matches MODEL_COSTS. Run from the repository root:
// npx tsx .agents/skills/refresh-model-catalog/scripts/sync-pricing-fixture.ts
import { writeFile } from "node:fs/promises"
import { format, resolveConfig } from "prettier"
import { MODEL_COSTS, PRICING_LAST_VERIFIED, PRICING_SOURCE_URL } from "../../../../src/pricing.ts"
const FIXTURE_PATH = new URL("../../../../tests/fixtures/commandcode-pricing.json", import.meta.url)
const costs: Record<string, [number, number, number, number]> = {}
const tiers: Record<string, [number, number, number, number, number][]> = {}
for (const [modelId, cost] of Object.entries(MODEL_COSTS)) {
costs[modelId] = [cost.input, cost.output, cost.cacheRead, cost.cacheWrite]
if (cost.tiers) {
tiers[modelId] = cost.tiers.map((tier) => [
tier.inputTokensAbove,
tier.input,
tier.output,
tier.cacheRead,
tier.cacheWrite,
])
}
}
const fixture = {
verifiedAt: PRICING_LAST_VERIFIED,
source: PRICING_SOURCE_URL,
tierPolicy:
"Use request-wide input tiers; the highest threshold exceeded by input plus cache tokens applies to the full request.",
tiers,
costs,
}
const options = await resolveConfig(new URL("../../../../.prettierrc.json", import.meta.url))
const contents = await format(JSON.stringify(fixture), {
...options,
filepath: "commandcode-pricing.json",
})
await writeFile(FIXTURE_PATH, contents, "utf-8")
console.log(
`Wrote ${Object.keys(costs).length} model prices to tests/fixtures/commandcode-pricing.json`,
)
.github/scripts/check-commandcode-model-metadata.ts
+22 -4
View File
@@ -18,6 +18,26 @@ const MODELS_REFERENCE_PATH = "dist/bundled/command-code-knowledge/reference/mod
const CLI_BUNDLE_PATH = "dist/cli.mjs"
const TEXT_ONLY_MARKER = ',__name(isKnownTextOnlyModel,"isKnownTextOnlyModel")'
const VALID_EFFORTS = new Set(["minimal", "low", "medium", "high", "xhigh", "max"])
function quoteWindowsArgument(argument: string): string {
if (argument.length === 0) return '""'
if (!/[\s"]/.test(argument)) return argument
return `"${argument.replaceAll('"', '\\"')}"`
}
/**
* Run npm on Windows and POSIX. npm is a `.cmd` shim on Windows, which
* execFile cannot spawn directly, so route it through the shell there with
* shell-safe quoting.
*/
function execNpmFileAsync(
args: readonly string[],
options: { cwd: string; encoding: "utf-8"; maxBuffer?: number },
): Promise<{ stdout: string }> {
if (process.platform !== "win32") return execFileAsync("npm", args, options)
const command = ["npm", ...args.map(quoteWindowsArgument)].join(" ")
return execFileAsync(command, { ...options, shell: true })
}
const CATALOG_SOURCE_PATH = new URL("../../src/commandcode-catalog.ts", import.meta.url)
const README_PATH = new URL("../../README.md", import.meta.url)
@@ -417,8 +437,7 @@ async function resolvePackageSpec(
): Promise<string> {
if (packageSpec !== "command-code@latest") return packageSpec
const { stdout } = await execFileAsync(
"npm",
const { stdout } = await execNpmFileAsync(
["view", packageSpec, "version", "--json", "--prefer-online", "--cache", npmCacheDirectory],
{
cwd: directory,
@@ -437,8 +456,7 @@ async function inspectPackedPackage(packageSpec: string): Promise<{
try {
const resolvedPackageSpec = await resolvePackageSpec(packageSpec, directory, npmCacheDirectory)
const { stdout } = await execFileAsync(
"npm",
const { stdout } = await execNpmFileAsync(
["pack", resolvedPackageSpec, "--json", "--prefer-online", "--cache", npmCacheDirectory],
{
cwd: directory,
CHANGELOG.md
+12
View File
@@ -2,6 +2,18 @@
## Unreleased
- Expose selectable thinking levels (`minimal`, `low`, `medium`, `high`, `xhigh`) for `meta/muse-spark-1.1`, `meta/muse-spark-1.2`, and `meta/muse-spark-1.2-contributor` through a manual catalog override, so `/thinking` and `Shift+Tab` no longer stay locked on `off` for these reasoning models.
- Start from the cached model catalog and refresh it in the background instead of blocking host startup on the catalog request; a first start without a cache still waits for the live catalog.
- Register the `commandcode-custom` API in the `@earendil-works/pi-ai/compat` registry so sibling extensions that stream with the active Command Code model no longer fail with `No API provider registered for api: commandcode-custom` on plain pi.
- Assert structural catalog invariants in the model tests so the daily catalog sync no longer fails on every upstream change.
- Display the monthly renewal date and remaining days in `/commandcode-quota`.
- Stop silently dropping `role: "developer"` messages (for example OMP advisor steering notes, reminders, and nudges). `/alpha/generate` only accepts `user`, `assistant`, and `tool` roles, so developer messages are now forwarded as `user` messages with identical content in the same chronological position instead of disappearing from the request.
- Add `Qwen/Qwen3.8-Flash` and `z-ai/glm-5.3-flash` with their verified reasoning efforts (`low, medium, xhigh` and `low, high, max`) and display pricing.
- Refresh static model capabilities from `command-code@1.40.1`, adding `claude-fable-5-1`, `deepseek/deepseek-v4-flash-fast`, and `tencent/hy4-preview` with their reasoning efforts, adding `moonshotai/Kimi-K3` efforts and the `z-ai/glm-5.3-flash` output limit, and dropping the retired `stealth/ox-alpha` and `minimax/minimax-m3-free`.
- Refresh display pricing for the current 62-model catalog, adding `claude-fable-5-1`, `deepseek/deepseek-v4-flash-fast`, and `tencent/hy4-preview`, removing the retired `stealth/ox-alpha`, `minimax/minimax-m3-free`, and `minimax/minimax-m2.7-free`, and ending the expired Claude Sonnet 5 introductory and Gemini 3.7 Flash promotional windows.
- Fix `npm run sync:commandcode-catalog` and `npm run check:commandcode-catalog` on Windows by spawning npm through the shell.
- Add a `refresh-model-catalog` agent skill with cross-platform helper scripts that snapshot the live model catalog and regenerate the pricing fixture from `MODEL_COSTS`.
## 0.6.0 - 2026-08-25
- Allow switching from a vision-capable model to a text-only model by omitting historical image tool results while preserving their text output; direct image prompts still fail clearly.
README.md
+8 -4
View File
@@ -31,6 +31,10 @@ omp plugin install pi-commandcode-provider
Restart OMP or run `/reload`, then use `/login` and select **Use a subscription** followed by **Command Code**.
## Other extensions
Command Code models are registered under the custom `commandcode-custom` API. The provider also registers that API in the `@earendil-works/pi-ai/compat` registry, so sibling extensions that stream through `streamSimple` from that entrypoint with the active session model (background agents, memory workers, and similar) reach the same Command Code transport instead of failing with `No API provider registered for api: commandcode-custom`. When such a call passes no API key, the provider uses the configured Command Code credentials.
## Authentication
### Login dialog
@@ -84,7 +88,7 @@ Open `/model` and select one of the models provided by Command Code. Model avail
### Reasoning support
Reasoning capability and selectable effort levels follow the official CLI catalog independently. Models can therefore be marked as reasoning-capable even when Command Code chooses their depth automatically. Models with explicit effort support also register a model-specific `thinkingLevelMap`, so pi and OMP expose only valid levels. Pi's native OpenAI- and Anthropic-compatible providers translate the selected level for Provider API accounts; the existing Command Code generate transport sends the matching `reasoning_effort` for Go accounts.
Reasoning capability and selectable effort levels follow the official CLI catalog independently. Models can therefore be marked as reasoning-capable even when Command Code chooses their depth automatically. Models with explicit effort support register a model-specific `thinkingLevelMap`, so pi and OMP expose only valid levels. For a few reasoning models the CLI catalog ships no effort levels although the endpoint accepts `reasoning_effort`; `src/commandcode-catalog-overrides.ts` adds a manual level set for those (currently `meta/muse-spark-1.1`, `meta/muse-spark-1.2`, and `meta/muse-spark-1.2-contributor`) on top of the generated catalog, and the tests fail once upstream publishes its own levels so the override gets removed. Pi's native OpenAI- and Anthropic-compatible providers translate the selected level for Provider API accounts; the existing Command Code generate transport sends the matching `reasoning_effort` for Go accounts.
List Command Code models from the terminal:
@@ -114,7 +118,7 @@ https://api.commandcode.ai/provider/v1/models
The last successful catalog is cached at `<agent-dir>/commandcode-models.json`. For pi this is `~/.pi/agent/commandcode-models.json` by default. Compatible hosts such as OMP use their own agent directory.
If the endpoint is temporarily unavailable, the provider uses the cached catalog. On a first offline start without a cache, pi still loads, but Command Code models remain unavailable until the connection is restored and `/commandcode-refresh` succeeds.
When a valid cache exists, the provider registers the cached catalog immediately and refreshes it from the endpoint in the background, so startup does not wait for the network. The refreshed catalog replaces the cached one as soon as it arrives; `/commandcode-status` reports `source: cache` until then. If the endpoint is temporarily unavailable, the cached catalog stays active. On a first start without a cache, the provider waits for the live catalog; if that fails offline, pi still loads, but Command Code models remain unavailable until the connection is restored and `/commandcode-refresh` succeeds.
While pi is running, use these provider commands without restarting:
@@ -135,7 +139,7 @@ The following environment variables are intended for tests, local mocks, and com
## Image input
The provider advertises image input only for models marked with the `image` input modality in the official Command Code CLI model catalog. The capability snapshot currently follows `command-code@1.32.2`; unknown models default to text-only until their upstream metadata is reviewed. A daily GitHub Actions job synchronizes the CLI version, image capabilities, reasoning flags, reasoning efforts, and model-specific output limits with the latest published CLI package and opens or updates a reviewable pull request when they change. Pricing remains manually reviewed because temporary promotions and long-context tiers require explicit review.
The provider advertises image input only for models marked with the `image` input modality in the official Command Code CLI model catalog. The capability snapshot currently follows `command-code@1.40.1`; unknown models default to text-only until their upstream metadata is reviewed. A daily GitHub Actions job synchronizes the CLI version, image capabilities, reasoning flags, reasoning efforts, and model-specific output limits with the latest published CLI package and opens or updates a reviewable pull request when they change. Pricing remains manually reviewed because temporary promotions and long-context tiers require explicit review.
For vision-capable models, Pi's native provider adapters forward image blocks from user messages and tool results using the documented OpenAI or Anthropic message schema. Unknown and text-only models remain marked text-only in Pi.
@@ -205,7 +209,7 @@ COMMANDCODE_E2E_GOAT_API_KEY_FILE=/path/to/goat-key \
Each profile runs with an isolated Pi agent directory and asserts transport selection, reasoning across turns, quota plan identity, abort handling, tool calls, and the packed npm artifact. Go must select `generate` and reject unsupported images; GOAT must select `provider` and complete a live vision request. The profile-specific `*_API_KEY` environment variables are also supported for CI secrets, but key files are preferred for local use.
The Go profile defaults to DeepSeek V4 Flash; GOAT defaults to Grok 4.6 because its Provider API stream exposes reasoning consistently across consecutive turns. Override them with `COMMANDCODE_E2E_GO_MODEL`, `COMMANDCODE_E2E_GOAT_MODEL`, or `COMMANDCODE_E2E_PROVIDER_MODEL`. A successful live Anthropic `/provider/v1/messages` test requires a paid account whose plan includes the selected Claude model.
The Go profile defaults to DeepSeek V4 Flash; GOAT defaults to Grok 4.6 because its Provider API stream exposes reasoning consistently across consecutive turns. Override them with `COMMANDCODE_E2E_GO_MODEL`, `COMMANDCODE_E2E_GOAT_MODEL`, or `COMMANDCODE_E2E_PROVIDER_MODEL`. The GOAT vision phase defaults to GPT-5.6 Luna and can be overridden with `COMMANDCODE_E2E_GOAT_VISION_MODEL`. A successful live Anthropic `/provider/v1/messages` test requires a paid account whose plan includes the selected Claude model.
See [CONTRIBUTING.md](CONTRIBUTING.md) for local setup and tests. See [RELEASE.md](RELEASE.md) for the release process.
index.ts
+36 -4
View File
@@ -6,7 +6,11 @@
*/
import { AssistantMessageEventStream } from "@earendil-works/pi-ai"
import { streamSimple as streamNativeProvider } from "@earendil-works/pi-ai/compat"
import {
registerApiProvider,
streamSimple as streamNativeProvider,
type ApiStreamSimpleFunction,
} from "@earendil-works/pi-ai/compat"
import {
getAgentDir,
type ExtensionAPI,
@@ -25,6 +29,7 @@ import {
DEFAULT_PROVIDER_API_BASE,
getModelsTimeoutMs,
inputModalitiesForModel,
loadCachedCommandCodeModels,
loadCommandCodeModels,
MODEL_EFFORTS,
thinkingMetadataForModel,
@@ -37,6 +42,9 @@ import { registerCommandCodeQuota } from "./src/quota-command.ts"
import { createCommandCodeRuntime } from "./src/runtime.ts"
import { createCommandCodeTransportRouter } from "./src/transport.ts"
const COMMAND_CODE_API = "commandcode-custom"
const COMPAT_SOURCE_ID = "pi-commandcode-provider"
function commandCodeHeaders(): Record<string, string> | undefined {
if (process.env.CMD_ZDR === "1" || process.env.COMMANDCODE_ZDR === "1") {
return { "x-cmd-zdr": "1" }
@@ -54,7 +62,7 @@ function createProviderConfig(
name: "Command Code",
baseUrl: apiBase,
apiKey: getConfiguredApiKey() ?? "$COMMAND_CODE_API_KEY",
api: "commandcode-custom",
api: COMMAND_CODE_API,
streamSimple: streamCommandCode,
headers,
oauth: {
@@ -66,7 +74,7 @@ function createProviderConfig(
models: models.map((model) => ({
id: model.id,
name: model.name,
api: "commandcode-custom",
api: COMMAND_CODE_API,
baseUrl: baseUrlForModel(apiBase, model.api),
reasoning: model.reasoning,
...(thinkingMetadataForModel(model.id) ?? {}),
@@ -120,6 +128,24 @@ export default async function (pi: ExtensionAPI) {
streamGenerate,
})
// pi dispatches the main chat through the registered provider, but sibling
// extensions that call `streamSimple` from `@earendil-works/pi-ai/compat`
// with a Command Code model resolve `model.api` through the compat
// api-registry, which knows nothing about extension providers. Register the
// custom api there so those calls reach the same transport. The registry
// resolves no credentials for extension providers, so fall back to the
// configured key when the caller passes none.
const compatStream: ApiStreamSimpleFunction = (model, context, options) =>
transport.stream(
model,
context,
options?.apiKey ? options : { ...options, apiKey: getConfiguredApiKey() },
) as AssistantMessageEventStream
registerApiProvider(
{ api: COMMAND_CODE_API, stream: compatStream, streamSimple: compatStream },
COMPAT_SOURCE_ID,
)
pi.on("message_end", async (event, ctx) => {
if (event.message.role !== "assistant") return
const normalized = normalizeCommandCodeMessage(event.message, ctx.model?.provider)
@@ -134,15 +160,21 @@ export default async function (pi: ExtensionAPI) {
const runtime = createCommandCodeRuntime<ProviderConfig, ExtensionCommandContext>(pi, {
endpoint: modelsUrl,
cachePath: modelsCachePath,
loadModels: () =>
loadModels: (signal) =>
loadCommandCodeModels({
url: modelsUrl,
cachePath: modelsCachePath,
timeoutMs: modelsTimeoutMs,
signal,
}),
loadCachedModels: () => loadCachedCommandCodeModels(modelsCachePath),
createProviderConfig: (models) => createProviderConfig(models, apiBase, transport.stream),
getTransport: transport.getTransport,
})
pi.on("session_shutdown", () => {
runtime.dispose()
})
await runtime.initialize()
}
package-lock.json
Generated
+2 -2
View File
@@ -473,7 +473,7 @@
"version": "25.6.0",
"resolved": "https://registry.npmjs.org/@types/node/-/node-25.6.0.tgz",
"integrity": "sha512-+qIYRKdNYJwY3vRCZMdJbPLJAtGjQBudzZzdzwQYkEPQd+PJGixUL5QfvCLDaULoLv+RhT3LDkwEfKaAkgSmNQ==",
"devOptional": true,
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~7.19.0"
@@ -589,7 +589,7 @@
"version": "7.19.2",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.19.2.tgz",
"integrity": "sha512-qYVnV5OEm2AW8cJMCpdV20CDyaN3g0AjDlOGf1OW4iaDEx8MwdtChUp4zu4H0VP3nDRF/8RKWH+IPp9uW0YGZg==",
"devOptional": true,
"dev": true,
"license": "MIT"
}
}
src/commandcode-catalog-overrides.ts
+23
View File
@@ -0,0 +1,23 @@
import type { CommandCodeReasoningEffort } from "./commandcode-catalog.ts"
/**
* Manual reasoning-effort policy for models the official CLI marks as
* reasoning-capable without publishing selectable efforts.
*
* `src/commandcode-catalog.ts` is generated from the CLI package and must stay
* byte-identical to upstream so the daily drift check works. Entries here are
* merged over the generated catalog at load time and are not touched by
* `npm run sync:commandcode-catalog`.
*
* Add a model only when the effort parameter is known to be accepted by the
* Command Code endpoint; remove it once the CLI catalog ships its own efforts.
*/
export const MODEL_EFFORT_OVERRIDES: Readonly<
Record<string, readonly CommandCodeReasoningEffort[]>
> = {
// Meta Muse Spark: the CLI ships no effort levels, but the endpoint accepts
// `reasoning_effort` for these models and other hosts expose the same set.
"meta/muse-spark-1.1": ["minimal", "low", "medium", "high", "xhigh"],
"meta/muse-spark-1.2": ["minimal", "low", "medium", "high", "xhigh"],
"meta/muse-spark-1.2-contributor": ["minimal", "low", "medium", "high", "xhigh"],
}
src/commandcode-catalog.ts
+17 -6
View File
@@ -1,14 +1,15 @@
export const COMMAND_CODE_CLI_VERSION = "1.32.2"
export const COMMAND_CODE_CLI_VERSION = "1.40.1"
export type CommandCodeInputType = "text" | "image"
export type CommandCodeReasoningEffort = "minimal" | "low" | "medium" | "high" | "xhigh" | "max"
/**
* Generated from command-code@1.32.2 by `npm run sync:commandcode-catalog`.
* Generated from command-code@1.40.1 by `npm run sync:commandcode-catalog`.
* Do not edit manually.
*/
export const MODEL_INPUT_MODALITIES: Readonly<Record<string, readonly CommandCodeInputType[]>> = {
"claude-fable-5": ["text", "image"],
"claude-fable-5-1": ["text", "image"],
"claude-haiku-4-5-20251001": ["text", "image"],
"claude-opus-4-7": ["text", "image"],
"claude-opus-4-8": ["text", "image"],
@@ -41,24 +42,27 @@ export const MODEL_INPUT_MODALITIES: Readonly<Record<string, readonly CommandCod
"Qwen/Qwen3.7-Flash": ["text", "image"],
"Qwen/Qwen3.7-Plus": ["text", "image"],
"Qwen/Qwen3.8-27B": ["text", "image"],
"Qwen/Qwen3.8-Flash": ["text", "image"],
"Qwen/Qwen3.8-Max": ["text", "image"],
"sakana/fugu-ultra": ["text", "image"],
"stealth/ox-alpha": ["text", "image"],
"stepfun/Step-3.7-Flash": ["text", "image"],
"thinkingmachines/inkling": ["text", "image"],
"thinkingmachines/inkling-small": ["text", "image"],
"xai/grok-4.5": ["text", "image"],
"xiaomi/mimo-v2.5": ["text", "image"],
"z-ai/glm-5.3-flash": ["text", "image"],
}
export const MODEL_REASONING: Readonly<Record<string, true>> = {
"claude-fable-5": true,
"claude-fable-5-1": true,
"claude-opus-4-7": true,
"claude-opus-4-8": true,
"claude-opus-5": true,
"claude-sonnet-4-6": true,
"claude-sonnet-5": true,
"deepseek/deepseek-v4-flash": true,
"deepseek/deepseek-v4-flash-fast": true,
"deepseek/deepseek-v4-flash-vision-exp": true,
"deepseek/deepseek-v4-pro": true,
"google/gemini-3.1-flash-lite": true,
@@ -88,28 +92,32 @@ export const MODEL_REASONING: Readonly<Record<string, true>> = {
"Qwen/Qwen3.7-Max": true,
"Qwen/Qwen3.7-Plus": true,
"Qwen/Qwen3.8-27B": true,
"Qwen/Qwen3.8-Flash": true,
"Qwen/Qwen3.8-Max": true,
"sakana/fugu-ultra": true,
"stealth/ox-alpha": true,
"stepfun/Step-3.5-Flash": true,
"stepfun/Step-3.7-Flash": true,
"tencent/hy3-paid": true,
"tencent/hy4-preview": true,
"thinkingmachines/inkling": true,
"thinkingmachines/inkling-small": true,
"xai/grok-4.5": true,
"xai/grok-4.6": true,
"z-ai/glm-5.3-flash": true,
"zai-org/GLM-5.2": true,
"zai-org/GLM-5.3": true,
}
export const MODEL_EFFORTS: Readonly<Record<string, readonly CommandCodeReasoningEffort[]>> = {
"claude-fable-5": ["low", "medium", "high", "xhigh", "max"],
"claude-fable-5-1": ["low", "medium", "high", "xhigh", "max"],
"claude-opus-4-7": ["low", "medium", "high", "xhigh", "max"],
"claude-opus-4-8": ["low", "medium", "high", "xhigh", "max"],
"claude-opus-5": ["low", "medium", "high", "xhigh", "max"],
"claude-sonnet-4-6": ["low", "medium", "high", "xhigh", "max"],
"claude-sonnet-5": ["low", "medium", "high", "xhigh", "max"],
"deepseek/deepseek-v4-flash": ["high", "max"],
"deepseek/deepseek-v4-flash-fast": ["low", "high", "max"],
"deepseek/deepseek-v4-flash-vision-exp": ["high", "max"],
"deepseek/deepseek-v4-pro": ["high", "max"],
"google/gemini-3.1-flash-lite": ["low", "medium", "high"],
@@ -124,12 +132,15 @@ export const MODEL_EFFORTS: Readonly<Record<string, readonly CommandCodeReasonin
"gpt-5.6-luna": ["low", "medium", "high", "xhigh", "max"],
"gpt-5.6-sol": ["low", "medium", "high", "xhigh", "max"],
"gpt-5.6-terra": ["low", "medium", "high", "xhigh", "max"],
"moonshotai/Kimi-K3": ["low", "high", "max"],
"Qwen/Qwen3.8-27B": ["low", "medium", "xhigh"],
"Qwen/Qwen3.8-Flash": ["low", "medium", "xhigh"],
"Qwen/Qwen3.8-Max": ["low", "medium", "xhigh"],
"sakana/fugu-ultra": ["high", "xhigh"],
"stealth/ox-alpha": ["low", "high", "max"],
"tencent/hy4-preview": ["low", "medium", "high"],
"xai/grok-4.5": ["low", "medium", "high"],
"xai/grok-4.6": ["low", "medium", "high", "xhigh"],
"z-ai/glm-5.3-flash": ["low", "high", "max"],
"zai-org/GLM-5.2": ["high", "max"],
"zai-org/GLM-5.3": ["low", "high", "max"],
}
@@ -137,5 +148,5 @@ export const MODEL_EFFORTS: Readonly<Record<string, readonly CommandCodeReasonin
export const MODEL_MAX_OUTPUT_TOKENS: Readonly<Record<string, number>> = {
"poolside/laguna-s-2.1-free": 32_768,
"Qwen/Qwen3.8-27B": 32_768,
"stealth/ox-alpha": 131_072,
"z-ai/glm-5.3-flash": 131_072,
}
src/converters.ts
+6 -1
View File
@@ -229,7 +229,12 @@ export function messagesToCC(
const { callIds, resultIds } = toolCallState(messages)
for (const message of messages ?? []) {
if (message.role === "user") {
if (message.role === "user" || message.role === "developer") {
// Hosts such as OMP steer the agent by injecting developer-role messages
// (advisor notes, reminders, nudges) mid-conversation. /alpha/generate
// only accepts user, assistant, and tool roles, so degrade the role to
// user instead of dropping the message. Content and chronological
// position are preserved; system-prompt hoisting would change semantics.
out.push({
role: "user",
content: userContentToCommandCode(message.content, allowImages),
src/models.ts
+20 -2
View File
@@ -1,8 +1,9 @@
import { mkdir, readFile, rename, rm, writeFile } from "node:fs/promises"
import { dirname } from "node:path"
import { MODEL_EFFORT_OVERRIDES } from "./commandcode-catalog-overrides.ts"
import {
MODEL_EFFORTS,
MODEL_EFFORTS as CATALOG_MODEL_EFFORTS,
MODEL_INPUT_MODALITIES,
MODEL_MAX_OUTPUT_TOKENS,
MODEL_REASONING,
@@ -10,7 +11,13 @@ import {
type CommandCodeReasoningEffort,
} from "./commandcode-catalog.ts"
export { MODEL_EFFORTS, MODEL_INPUT_MODALITIES, MODEL_MAX_OUTPUT_TOKENS, MODEL_REASONING }
/** Upstream CLI efforts with the manual overrides merged over them. */
export const MODEL_EFFORTS: Readonly<Record<string, readonly CommandCodeReasoningEffort[]>> = {
...CATALOG_MODEL_EFFORTS,
...MODEL_EFFORT_OVERRIDES,
}
export { MODEL_INPUT_MODALITIES, MODEL_MAX_OUTPUT_TOKENS, MODEL_REASONING }
export type { CommandCodeInputType }
export const DEFAULT_PROVIDER_API_BASE = "https://api.commandcode.ai/provider/v1"
@@ -332,6 +339,17 @@ async function readCommandCodeModelsCache(cachePath: string): Promise<readonly C
return commandCodeModelsFromCache(parsed)
}
/** Reads the cached catalog without touching the network; empty when missing or invalid. */
export async function loadCachedCommandCodeModels(
cachePath: string,
): Promise<readonly CommandCodeModel[]> {
try {
return await readCommandCodeModelsCache(cachePath)
} catch {
return []
}
}
async function writeCommandCodeModelsCache(
cachePath: string,
models: readonly CommandCodeModel[],
src/pricing.ts
+16 -19
View File
@@ -20,7 +20,7 @@ export interface TemporaryPricing {
}
export const PRICING_SOURCE_URL = "https://commandcode.ai/docs/resources/pricing-limits"
export const PRICING_LAST_VERIFIED = "2026-08-25"
export const PRICING_LAST_VERIFIED = "2026-09-01"
export const ZERO_MODEL_COST: CommandCodeModelCost = {
input: 0,
@@ -40,10 +40,10 @@ export const ZERO_MODEL_COST: CommandCodeModelCost = {
export const MODEL_COSTS: Readonly<Record<string, CommandCodeModelCost>> = {
// Free models
"poolside/laguna-s-2.1-free": { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
"stealth/ox-alpha": { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
// Open and open-weight models
"tencent/hy3-paid": { input: 0.14, output: 0.58, cacheRead: 0.035, cacheWrite: 0 },
"tencent/hy4-preview": { input: 0.834, output: 2.501, cacheRead: 0.042, cacheWrite: 0 },
"moonshotai/Kimi-K3": { input: 3, output: 15, cacheRead: 0.3, cacheWrite: 0 },
"moonshotai/Kimi-K2.7-Code": { input: 0.95, output: 4, cacheRead: 0.19, cacheWrite: 0 },
"moonshotai/Kimi-K2.7-Code-Highspeed": {
@@ -54,6 +54,7 @@ export const MODEL_COSTS: Readonly<Record<string, CommandCodeModelCost>> = {
},
"moonshotai/Kimi-K2.6": { input: 0.95, output: 4, cacheRead: 0.16, cacheWrite: 0 },
"moonshotai/Kimi-K2.5": { input: 0.6, output: 3, cacheRead: 0.1, cacheWrite: 0 },
"z-ai/glm-5.3-flash": { input: 0.15, output: 0.5, cacheRead: 0.03, cacheWrite: 0 },
"zai-org/GLM-5.3": { input: 1.4, output: 4.4, cacheRead: 0.26, cacheWrite: 0 },
"zai-org/GLM-5.2": { input: 1.4, output: 4.4, cacheRead: 0.26, cacheWrite: 0 },
"zai-org/GLM-5.2-Fast": { input: 3, output: 10.25, cacheRead: 0.5, cacheWrite: 0 },
@@ -82,8 +83,15 @@ export const MODEL_COSTS: Readonly<Record<string, CommandCodeModelCost>> = {
cacheRead: 0.007,
cacheWrite: 0,
},
"deepseek/deepseek-v4-flash-fast": {
input: 0.28,
output: 0.56,
cacheRead: 0.07,
cacheWrite: 0,
},
"Qwen/Qwen3.8-Max": { input: 2, output: 6, cacheRead: 0.25, cacheWrite: 2.5 },
"Qwen/Qwen3.8-27B": { input: 0.4, output: 3, cacheRead: 0.04, cacheWrite: 0 },
"Qwen/Qwen3.8-Flash": { input: 0.16, output: 0.47, cacheRead: 0.016, cacheWrite: 0 },
"Qwen/Qwen3.7-Max": { input: 2.5, output: 7.5, cacheRead: 0.5, cacheWrite: 3.13 },
"Qwen/Qwen3.7-Plus": {
input: 0.4,
@@ -158,9 +166,9 @@ export const MODEL_COSTS: Readonly<Record<string, CommandCodeModelCost>> = {
},
// Anthropic
// Introductory pricing through 2026-08-31.
"claude-sonnet-5": { input: 2, output: 10, cacheRead: 0.2, cacheWrite: 2.5 },
"claude-sonnet-4-6": { input: 3, output: 15, cacheRead: 0.3, cacheWrite: 3.75 },
"claude-fable-5-1": { input: 10, output: 50, cacheRead: 0.25, cacheWrite: 12.5 },
"claude-fable-5": { input: 10, output: 50, cacheRead: 1, cacheWrite: 12.5 },
"claude-opus-5": { input: 5, output: 25, cacheRead: 0.5, cacheWrite: 6.25 },
"claude-opus-4-8": { input: 5, output: 25, cacheRead: 0.5, cacheWrite: 6.25 },
@@ -183,10 +191,10 @@ export const MODEL_COSTS: Readonly<Record<string, CommandCodeModelCost>> = {
// Google and xAI
"google/gemini-3.7-flash": {
input: 0.75,
output: 3.75,
cacheRead: 0.075,
cacheWrite: 0.04167,
input: 1.5,
output: 7.5,
cacheRead: 0.15,
cacheWrite: 0.08334,
},
"google/gemini-3.6-flash": { input: 1.5, output: 7.5, cacheRead: 0.15, cacheWrite: 0 },
"google/gemini-3.5-flash": { input: 1.5, output: 9, cacheRead: 0.15, cacheWrite: 0 },
@@ -220,15 +228,4 @@ export const MODEL_COSTS: Readonly<Record<string, CommandCodeModelCost>> = {
},
}
export const TEMPORARY_PRICING: readonly TemporaryPricing[] = [
{
models: ["claude-sonnet-5"],
expiresOn: "2026-08-31",
description: "introductory pricing",
},
{
models: ["google/gemini-3.7-flash"],
expiresOn: "2026-12-31",
description: "50% promotional pricing",
},
]
export const TEMPORARY_PRICING: readonly TemporaryPricing[] = []
src/runtime.ts
+39 -3
View File
@@ -26,7 +26,9 @@ export interface CommandCodeRuntimeApi<
export interface CommandCodeRuntimeOptions<TProviderConfig> {
endpoint: string
cachePath: string
loadModels: () => Promise<LoadCommandCodeModelsResult>
loadModels: (signal: AbortSignal) => Promise<LoadCommandCodeModelsResult>
/** Cached catalog only; resolves to an empty list when no valid cache exists. */
loadCachedModels: () => Promise<readonly CommandCodeModel[]>
createProviderConfig: (models: readonly CommandCodeModel[]) => TProviderConfig
getTransport?: () => "unknown" | "provider" | "generate"
now?: () => number
@@ -108,6 +110,7 @@ export class CommandCodeRuntime<TProviderConfig, TContext extends CommandCodeCom
private status: CommandCodeRuntimeStatus
private providerRegistered = false
private refreshPromise: Promise<CommandCodeRefreshResult> | undefined
private readonly shutdown = new AbortController()
constructor(
private readonly pi: CommandCodeRuntimeApi<TProviderConfig, TContext>,
@@ -133,9 +136,34 @@ export class CommandCodeRuntime<TProviderConfig, TContext extends CommandCodeCom
}
}
/**
* Registers the cached catalog immediately and refreshes it in the
* background so host startup does not wait for the network. Without a
* valid cache the live refresh is awaited so models are available at once.
*/
async initialize(): Promise<void> {
this.registerCommands()
await this.refresh()
const cached = await this.options.loadCachedModels()
if (cached.length === 0) {
await this.refresh()
return
}
this.pi.registerProvider("commandcode", this.options.createProviderConfig(cached))
this.providerRegistered = true
this.status = {
...this.status,
source: "cache",
modelCount: cached.length,
lastSuccess: this.now(),
}
void this.refresh()
}
/** Aborts any background refresh so a stopping host does not wait for the network. */
dispose(): void {
this.shutdown.abort(new Error("Command Code provider shut down"))
}
refresh(): Promise<CommandCodeRefreshResult> {
@@ -156,7 +184,7 @@ export class CommandCodeRuntime<TProviderConfig, TContext extends CommandCodeCom
}
try {
const loaded = await this.options.loadModels()
const loaded = await this.options.loadModels(this.shutdown.signal)
const warning = loaded.warning ? redactDiagnosticText(loaded.warning) : undefined
const shouldRegister =
@@ -217,6 +245,14 @@ export class CommandCodeRuntime<TProviderConfig, TContext extends CommandCodeCom
warning: preservedWarning,
}
} catch (error) {
if (this.shutdown.signal.aborted) {
this.status = { ...this.status, refreshing: false }
return {
refreshed: false,
source: this.status.source,
modelCount: this.status.modelCount,
}
}
const warning = redactDiagnosticText(
`Could not refresh the Command Code model catalog: ${errorMessage(error)}`,
)
tests/fixtures/advisory-injector-extension.ts
Vendored
+41
View File
@@ -0,0 +1,41 @@
/**
* Minimal OMP extension used by tests/test-omp-compat.mjs.
*
* Appends a custom advisor message to the session on session start, mirroring
* how the OMP Advisor runtime injects steering notes. OMP converts custom
* messages to `role: "developer"` LLM messages before handing them to the
* provider. Types are declared inline so the fixture has no runtime or
* typecheck dependency on OMP packages.
*/
interface AdvisorInjectorSendMessage {
(
message: {
customType: string
content: string
display: boolean
attribution: string
},
options?: { triggerTurn?: boolean },
): void
}
interface AdvisorInjectorApi {
on: (event: "session_start", handler: () => void | Promise<void>) => void
sendMessage: AdvisorInjectorSendMessage
}
export default function advisoryInjectorExtension(pi: AdvisorInjectorApi): void {
pi.on("session_start", async () => {
pi.sendMessage(
{
customType: "advisor",
content:
'<advisory severity="blocker" guidance="weigh, don\'t blindly obey">\nStop and correct the benchmark.\n</advisory>',
display: true,
attribution: "agent",
},
{ triggerTurn: false },
)
})
}
tests/fixtures/commandcode-model-ids.json
Vendored
+6 -2
View File
@@ -1,9 +1,10 @@
{
"fetchedAt": "2026-08-25T13:32:11.631Z",
"fetchedAt": "2026-09-01T21:28:23.974Z",
"source": "https://api.commandcode.ai/provider/v1/models",
"modelIds": [
"claude-sonnet-5",
"claude-sonnet-4-6",
"claude-fable-5-1",
"claude-fable-5",
"claude-opus-5",
"claude-opus-4-8",
@@ -19,11 +20,13 @@
"deepseek/deepseek-v4-pro",
"deepseek/deepseek-v4-flash",
"deepseek/deepseek-v4-flash-vision-exp",
"deepseek/deepseek-v4-flash-fast",
"moonshotai/Kimi-K3",
"moonshotai/Kimi-K2.7-Code",
"moonshotai/Kimi-K2.7-Code-Highspeed",
"moonshotai/Kimi-K2.6",
"moonshotai/Kimi-K2.5",
"z-ai/glm-5.3-flash",
"zai-org/GLM-5.3",
"zai-org/GLM-5.2",
"zai-org/GLM-5.2-Fast",
@@ -36,6 +39,7 @@
"xiaomi/mimo-v2.5",
"Qwen/Qwen3.8-Max",
"Qwen/Qwen3.8-27B",
"Qwen/Qwen3.8-Flash",
"Qwen/Qwen3.7-Max",
"Qwen/Qwen3.7-Plus",
"Qwen/Qwen3.7-Flash",
@@ -44,6 +48,7 @@
"stepfun/Step-3.7-Flash",
"stepfun/Step-3.5-Flash",
"tencent/hy3-paid",
"tencent/hy4-preview",
"google/gemini-3.7-flash",
"google/gemini-3.6-flash",
"google/gemini-3.5-flash",
@@ -53,7 +58,6 @@
"nvidia/nemotron-3-ultra-550b-a55b",
"thinkingmachines/inkling",
"thinkingmachines/inkling-small",
"stealth/ox-alpha",
"poolside/laguna-s-2.1-free",
"meta/muse-spark-1.1",
"meta/muse-spark-1.2",
tests/fixtures/commandcode-pricing.json
Vendored
+18 -14
View File
@@ -1,5 +1,5 @@
{
"verifiedAt": "2026-08-25",
"verifiedAt": "2026-09-01",
"source": "https://commandcode.ai/docs/resources/pricing-limits",
"tierPolicy": "Use request-wide input tiers; the highest threshold exceeded by input plus cache tokens applies to the full request.",
"tiers": {
@@ -11,14 +11,15 @@
"xai/grok-4.6": [[200000, 4, 12, 1, 0]]
},
"costs": {
"deepseek/deepseek-v4-pro": [0.66, 1.98, 0.022, 0],
"deepseek/deepseek-v4-flash": [0.22, 0.66, 0.007, 0],
"deepseek/deepseek-v4-flash-vision-exp": [0.22, 0.66, 0.007, 0],
"poolside/laguna-s-2.1-free": [0, 0, 0, 0],
"tencent/hy3-paid": [0.14, 0.58, 0.035, 0],
"tencent/hy4-preview": [0.834, 2.501, 0.042, 0],
"moonshotai/Kimi-K3": [3, 15, 0.3, 0],
"moonshotai/Kimi-K2.7-Code": [0.95, 4, 0.19, 0],
"moonshotai/Kimi-K2.7-Code-Highspeed": [1.9, 8, 0.38, 0],
"moonshotai/Kimi-K2.6": [0.95, 4, 0.16, 0],
"moonshotai/Kimi-K2.5": [0.6, 3, 0.1, 0],
"z-ai/glm-5.3-flash": [0.15, 0.5, 0.03, 0],
"zai-org/GLM-5.3": [1.4, 4.4, 0.26, 0],
"zai-org/GLM-5.2": [1.4, 4.4, 0.26, 0],
"zai-org/GLM-5.2-Fast": [3, 10.25, 0.5, 0],
@@ -27,10 +28,13 @@
"MiniMaxAI/MiniMax-M3": [0.3, 1.2, 0.06, 0],
"MiniMaxAI/MiniMax-M2.7": [0.3, 1.2, 0.06, 0],
"MiniMaxAI/MiniMax-M2.5": [0.3, 1.2, 0.03, 0],
"xiaomi/mimo-v2.5-pro": [0.435, 0.87, 0.0036, 0],
"xiaomi/mimo-v2.5": [0.14, 0.28, 0.0028, 0],
"deepseek/deepseek-v4-pro": [0.66, 1.98, 0.022, 0],
"deepseek/deepseek-v4-flash": [0.22, 0.66, 0.007, 0],
"deepseek/deepseek-v4-flash-vision-exp": [0.22, 0.66, 0.007, 0],
"deepseek/deepseek-v4-flash-fast": [0.28, 0.56, 0.07, 0],
"Qwen/Qwen3.8-Max": [2, 6, 0.25, 2.5],
"Qwen/Qwen3.8-27B": [0.4, 3, 0.04, 0],
"Qwen/Qwen3.8-Flash": [0.16, 0.47, 0.016, 0],
"Qwen/Qwen3.7-Max": [2.5, 7.5, 0.5, 3.13],
"Qwen/Qwen3.7-Plus": [0.4, 1.6, 0.08, 0.5],
"Qwen/Qwen3.7-Flash": [0.03, 0.13, 0.006, 0.038],
@@ -38,14 +42,18 @@
"Qwen/Qwen3.6-Plus": [0.5, 3, 0.1, 0],
"stepfun/Step-3.7-Flash": [0.2, 1.15, 0.04, 0],
"stepfun/Step-3.5-Flash": [0.1, 0.3, 0.02, 0],
"tencent/hy3-paid": [0.14, 0.58, 0.035, 0],
"xiaomi/mimo-v2.5-pro": [0.435, 0.87, 0.0036, 0],
"xiaomi/mimo-v2.5": [0.14, 0.28, 0.0028, 0],
"nvidia/nemotron-3-ultra-550b-a55b": [0.6, 2.4, 0.12, 0],
"sakana/fugu-ultra": [5, 30, 0.5, 0],
"thinkingmachines/inkling": [1, 4.05, 0.17, 0],
"thinkingmachines/inkling-small": [0.5, 1.2, 0.1, 0],
"poolside/laguna-s-2.1-free": [0, 0, 0, 0],
"stealth/ox-alpha": [0, 0, 0, 0],
"meta/muse-spark-1.1": [1.25, 4.25, 0.15, 0],
"meta/muse-spark-1.2": [1.25, 4.25, 0.15, 0],
"meta/muse-spark-1.2-contributor": [0.1, 0.2, 0.002, 0],
"claude-sonnet-5": [2, 10, 0.2, 2.5],
"claude-sonnet-4-6": [3, 15, 0.3, 3.75],
"claude-fable-5-1": [10, 50, 0.25, 12.5],
"claude-fable-5": [10, 50, 1, 12.5],
"claude-opus-5": [5, 25, 0.5, 6.25],
"claude-opus-4-8": [5, 25, 0.5, 6.25],
@@ -58,15 +66,11 @@
"gpt-5.4": [2.5, 15, 0.25, 0],
"gpt-5.3-codex": [2, 8, 0.5, 0],
"gpt-5.4-mini": [0.75, 4.5, 0.075, 0],
"google/gemini-3.7-flash": [0.75, 3.75, 0.075, 0.04167],
"google/gemini-3.7-flash": [1.5, 7.5, 0.15, 0.08334],
"google/gemini-3.6-flash": [1.5, 7.5, 0.15, 0],
"google/gemini-3.5-flash": [1.5, 9, 0.15, 0],
"google/gemini-3.5-flash-lite": [0.3, 2.5, 0.03, 0],
"google/gemini-3.1-flash-lite": [0.25, 1.5, 0.03, 0],
"sakana/fugu-ultra": [5, 30, 0.5, 0],
"meta/muse-spark-1.1": [1.25, 4.25, 0.15, 0],
"meta/muse-spark-1.2": [1.25, 4.25, 0.15, 0],
"meta/muse-spark-1.2-contributor": [0.1, 0.2, 0.002, 0],
"xai/grok-4.5": [2, 6, 0.5, 0],
"xai/grok-4.6": [2, 6, 0.5, 0]
}
tests/fixtures/compat-caller-extension.ts
Vendored
+69
View File
@@ -0,0 +1,69 @@
/**
* Test fixture: a sibling extension that streams through the pi-ai compat
* entrypoint with the active session model, the way background-agent
* extensions do. Registers `/compat-call` so the test can drive it over RPC.
*
* Types are declared inline so the fixture has no typecheck dependency on the
* optional pi peer packages; the host's extension loader resolves the import.
*/
// @ts-expect-error pi resolves this peer package at load time.
import * as compatModule from "@earendil-works/pi-ai/compat"
interface CompatTextPart {
type: string
text?: string
}
interface CompatStreamResult {
result(): Promise<{ content: readonly CompatTextPart[] }>
}
interface CompatModule {
streamSimple(model: unknown, context: unknown): CompatStreamResult
}
interface CompatCallerContext {
model?: unknown
ui: { notify(message: string, level: "info" | "error"): void }
}
interface CompatCallerExtensionApi {
registerCommand(
name: string,
command: {
description: string
handler: (args: string, ctx: CompatCallerContext) => Promise<void>
},
): void
}
const compat = compatModule as CompatModule
export default function (pi: CompatCallerExtensionApi) {
pi.registerCommand("compat-call", {
description: "Stream through @earendil-works/pi-ai/compat with the session model",
handler: async (_args, ctx) => {
const model = ctx.model
if (!model) {
ctx.ui.notify("compat-call: no active model", "error")
return
}
try {
const message = await compat
.streamSimple(model, {
messages: [{ role: "user", content: "say mock token", timestamp: Date.now() }],
})
.result()
const text = message.content
.filter((part) => part.type === "text")
.map((part) => part.text ?? "")
.join("")
ctx.ui.notify(`compat-call ok: ${text}`, "info")
} catch (error) {
const detail = error instanceof Error ? error.message : String(error)
ctx.ui.notify(`compat-call failed: ${detail}`, "error")
}
},
})
}
tests/test-live-e2e.mjs
+3 -1
View File
@@ -40,7 +40,9 @@ const expectedPlan =
: testProfile === "provider"
? "provider"
: undefined
const goatVisionModel = process.env.COMMANDCODE_E2E_GOAT_VISION_MODEL ?? "google/gemini-3.7-flash"
// GPT-5.6 Luna is available on every plan and has a ZDR-capable upstream;
// Gemini 3.7 Flash currently fails with a zero-data-retention routing 404.
const goatVisionModel = process.env.COMMANDCODE_E2E_GOAT_VISION_MODEL ?? "gpt-5.6-luna"
const marker = "commandcode-live-e2e-ok"
function findPiBinary() {
tests/test-models.ts
+67 -38
View File
@@ -4,7 +4,11 @@ import { tmpdir } from "node:os"
import { join } from "node:path"
import { describe, it } from "node:test"
import { COMMAND_CODE_CLI_VERSION } from "../src/commandcode-catalog.ts"
import { MODEL_EFFORT_OVERRIDES } from "../src/commandcode-catalog-overrides.ts"
import {
COMMAND_CODE_CLI_VERSION,
MODEL_EFFORTS as CATALOG_MODEL_EFFORTS,
} from "../src/commandcode-catalog.ts"
import {
apiForModelId,
baseUrlForModel,
@@ -104,41 +108,48 @@ describe("commandCodeModelsFromApiResponse()", () => {
})
it(`uses the command-code@${COMMAND_CODE_CLI_VERSION} image capability catalog`, () => {
assert.deepEqual(inputModalitiesForModel("gpt-5.6-luna"), ["text", "image"])
assert.deepEqual(inputModalitiesForModel("meta/muse-spark-1.2"), ["text", "image"])
assert.deepEqual(inputModalitiesForModel("deepseek/deepseek-v4-flash-vision-exp"), [
"text",
"image",
])
assert.deepEqual(inputModalitiesForModel("Qwen/Qwen3.8-27B"), ["text", "image"])
assert.deepEqual(inputModalitiesForModel("google/gemini-3.7-flash"), ["text", "image"])
assert.deepEqual(inputModalitiesForModel("stealth/ox-alpha"), ["text", "image"])
assert.deepEqual(inputModalitiesForModel("deepseek/deepseek-v4-pro"), ["text"])
assert.deepEqual(inputModalitiesForModel("zai-org/GLM-5.3"), ["text"])
assert.deepEqual(inputModalitiesForModel("unknown-new-model"), ["text"])
assert.equal(modelSupportsImageInput("gpt-5.6-luna"), true)
assert.equal(modelSupportsImageInput("deepseek/deepseek-v4-flash-vision-exp"), true)
assert.equal(modelSupportsImageInput("stealth/ox-alpha"), true)
assert.equal(modelSupportsImageInput("deepseek/deepseek-v4-pro"), false)
assert.ok(Object.keys(MODEL_INPUT_MODALITIES).length > 0)
for (const modalities of Object.values(MODEL_INPUT_MODALITIES)) {
assert.deepEqual(modalities, ["text", "image"])
const imageModels = Object.keys(MODEL_INPUT_MODALITIES)
assert.ok(imageModels.length > 0)
for (const modelId of imageModels) {
assert.deepEqual(MODEL_INPUT_MODALITIES[modelId], ["text", "image"], modelId)
assert.deepEqual(inputModalitiesForModel(modelId), ["text", "image"], modelId)
assert.equal(modelSupportsImageInput(modelId), true, modelId)
}
const textOnlyModel = Object.keys(MODEL_REASONING).find(
(modelId) => !(modelId in MODEL_INPUT_MODALITIES),
)
assert.ok(textOnlyModel, "catalog should contain at least one text-only model")
assert.deepEqual(inputModalitiesForModel(textOnlyModel), ["text"])
assert.equal(modelSupportsImageInput(textOnlyModel), false)
assert.deepEqual(inputModalitiesForModel("unknown-new-model"), ["text"])
assert.equal(modelSupportsImageInput("unknown-new-model"), false)
})
it("tracks reasoning independently from selectable effort levels", () => {
const reasoningModels = Object.keys(MODEL_REASONING)
const effortModels = Object.keys(MODEL_EFFORTS)
assert.ok(reasoningModels.length > 0)
assert.ok(effortModels.length > 0)
for (const modelId of effortModels) {
assert.equal(MODEL_REASONING[modelId], true, `${modelId} has efforts but no reasoning flag`)
}
const reasoningWithoutEfforts = reasoningModels.find((modelId) => !(modelId in MODEL_EFFORTS))
assert.ok(reasoningWithoutEfforts, "catalog should contain a reasoning model without efforts")
const models = commandCodeModelsFromApiResponse({
object: "list",
data: [
{ ...API_RESPONSE.data[0], id: "deepseek/deepseek-v4-flash" },
{ ...API_RESPONSE.data[0], id: "moonshotai/Kimi-K3" },
{ ...API_RESPONSE.data[0], id: effortModels[0] },
{ ...API_RESPONSE.data[0], id: reasoningWithoutEfforts },
{ ...API_RESPONSE.data[0], id: "new-model-without-metadata" },
],
})
assert.equal(models[0]?.reasoning, true)
assert.equal(models[1]?.reasoning, true)
assert.deepEqual(thinkingMetadataForModel("moonshotai/Kimi-K3"), {
assert.deepEqual(thinkingMetadataForModel(reasoningWithoutEfforts), {
thinkingLevelMap: {
minimal: null,
low: null,
@@ -149,32 +160,30 @@ describe("commandCodeModelsFromApiResponse()", () => {
},
})
assert.equal(models[2]?.reasoning, false)
assert.equal(Object.keys(MODEL_REASONING).length, 48)
})
it("uses model-specific output limits from the CLI catalog", () => {
const limitedModels = Object.entries(MODEL_MAX_OUTPUT_TOKENS)
assert.ok(limitedModels.length > 0)
for (const [modelId, limit] of limitedModels) {
assert.ok(Number.isInteger(limit) && limit > 0, `${modelId} has an invalid output limit`)
}
const [limitedId, limit] = limitedModels[0]!
const models = commandCodeModelsFromApiResponse({
object: "list",
data: [
{ ...API_RESPONSE.data[0], id: "Qwen/Qwen3.8-27B", context_length: 262_144 },
{ ...API_RESPONSE.data[0], id: "stealth/ox-alpha", context_length: 1_048_576 },
{
...API_RESPONSE.data[0],
id: "poolside/laguna-s-2.1-free",
context_length: 256_000,
},
{ ...API_RESPONSE.data[0], id: limitedId, context_length: limit * 4 },
{ ...API_RESPONSE.data[0], id: limitedId, context_length: Math.floor(limit / 2) },
{ ...API_RESPONSE.data[0], id: "unknown-new-model", context_length: 256_000 },
{ ...API_RESPONSE.data[0], id: "unknown-new-model", context_length: 8_192 },
],
})
assert.deepEqual(
models.map(({ id, maxTokens }) => ({ id, maxTokens })),
[
{ id: "Qwen/Qwen3.8-27B", maxTokens: 32_768 },
{ id: "stealth/ox-alpha", maxTokens: 131_072 },
{ id: "poolside/laguna-s-2.1-free", maxTokens: 32_768 },
],
models.map(({ maxTokens }) => maxTokens),
[limit, Math.floor(limit / 2), 65_536, 8_192],
)
assert.equal(Object.keys(MODEL_MAX_OUTPUT_TOKENS).length, 3)
})
it(`uses the command-code@${COMMAND_CODE_CLI_VERSION} reasoning effort catalog`, () => {
@@ -187,6 +196,26 @@ describe("commandCodeModelsFromApiResponse()", () => {
}
})
it("merges manual effort overrides over the generated catalog", () => {
const validEfforts = new Set(["minimal", "low", "medium", "high", "xhigh", "max"])
assert.ok(Object.keys(MODEL_EFFORT_OVERRIDES).length > 0)
for (const [modelId, efforts] of Object.entries(MODEL_EFFORT_OVERRIDES)) {
assert.equal(MODEL_REASONING[modelId], true, `${modelId} override needs a reasoning flag`)
assert.equal(
CATALOG_MODEL_EFFORTS[modelId],
undefined,
`${modelId} now has upstream efforts; drop the manual override`,
)
assert.ok(efforts.length > 0)
assert.ok(efforts.every((effort) => validEfforts.has(effort)))
assert.deepEqual(MODEL_EFFORTS[modelId], efforts)
assert.deepEqual(thinkingMetadataForModel(modelId)?.thinking?.efforts, efforts)
}
for (const [modelId, efforts] of Object.entries(CATALOG_MODEL_EFFORTS)) {
assert.deepEqual(MODEL_EFFORTS[modelId], efforts, `${modelId} upstream efforts changed`)
}
})
it("builds separate canonical pi and OMP metadata", () => {
for (const [modelId, efforts] of Object.entries(MODEL_EFFORTS)) {
const metadata = thinkingMetadataForModel(modelId)
tests/test-omp-compat.mjs
+124 -14
View File
@@ -19,7 +19,10 @@ import { fileURLToPath } from "node:url"
const __dirname = dirname(fileURLToPath(import.meta.url))
const PROJECT_DIR = resolve(__dirname, "..")
const EXT_PATH = resolve(PROJECT_DIR, "index.ts")
const ADVISORY_EXT_PATH = resolve(PROJECT_DIR, "tests/fixtures/advisory-injector-extension.ts")
const TEST_MODEL = "deepseek/deepseek-v4-flash"
const ADVISORY_XML =
'<advisory severity="blocker" guidance="weigh, don\'t blindly obey">\nStop and correct the benchmark.\n</advisory>'
function findOmpBinary() {
if (process.env.OMP_BIN) return process.env.OMP_BIN
@@ -45,7 +48,11 @@ const tempHome = mkdtempSync(join(tmpdir(), "omp-cc-home-"))
let requestCount = 0
let modelListRequestCount = 0
let lastRequestBody
let requestBodies = []
let lastRequestHeaders = {}
// When true the mock Provider API answers 403 upgrade_required so the
// transport router falls back to the legacy /alpha/generate transport.
let providerUpgradeRequired = false
const server = createServer((req, res) => {
if (req.method === "GET" && req.url === "/provider/v1/models") {
@@ -77,7 +84,52 @@ const server = createServer((req, res) => {
return
}
if (req.method !== "POST" || req.url !== "/provider/v1/chat/completions") {
if (req.method === "POST" && req.url === "/provider/v1/chat/completions") {
requestCount += 1
lastRequestHeaders = Object.fromEntries(
Object.entries(req.headers).map(([key, value]) => [
key,
Array.isArray(value) ? value.join(", ") : (value ?? ""),
]),
)
let body = ""
req.on("data", (chunk) => {
body += chunk.toString("utf-8")
})
req.on("end", () => {
try {
lastRequestBody = JSON.parse(body)
requestBodies.push(lastRequestBody)
} catch {
lastRequestBody = undefined
}
if (providerUpgradeRequired) {
res.writeHead(403, { "Content-Type": "application/json; charset=utf-8" })
res.end(JSON.stringify({ error: { code: "upgrade_required" } }))
return
}
res.writeHead(200, {
"Content-Type": "text/event-stream; charset=utf-8",
"Transfer-Encoding": "chunked",
})
res.write(
`data: ${JSON.stringify({ id: "mock", object: "chat.completion.chunk", choices: [{ index: 0, delta: { role: "assistant", content: "mock-omp-ok" }, finish_reason: null }] })}\n\n`,
)
res.write(
`data: ${JSON.stringify({ id: "mock", object: "chat.completion.chunk", choices: [{ index: 0, delta: {}, finish_reason: "stop" }] })}\n\n`,
)
res.write(
`data: ${JSON.stringify({ id: "mock", object: "chat.completion.chunk", choices: [], usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 } })}\n\n`,
)
res.end("data: [DONE]\n\n")
})
return
}
if (req.method !== "POST" || req.url !== "/alpha/generate") {
res.writeHead(404)
res.end("Not found")
return
@@ -91,31 +143,27 @@ const server = createServer((req, res) => {
]),
)
let body = ""
let generateBody = ""
req.on("data", (chunk) => {
body += chunk.toString("utf-8")
generateBody += chunk.toString("utf-8")
})
req.on("end", () => {
try {
lastRequestBody = JSON.parse(body)
lastRequestBody = JSON.parse(generateBody)
requestBodies.push(lastRequestBody)
} catch {
lastRequestBody = undefined
}
res.writeHead(200, {
"Content-Type": "text/event-stream; charset=utf-8",
"Content-Type": "text/plain; charset=utf-8",
"Transfer-Encoding": "chunked",
})
res.write(`${JSON.stringify({ type: "text-delta", text: "mock-omp-ok" })}\n`)
res.write(
`data: ${JSON.stringify({ id: "mock", object: "chat.completion.chunk", choices: [{ index: 0, delta: { role: "assistant", content: "mock-omp-ok" }, finish_reason: null }] })}\n\n`,
`${JSON.stringify({ type: "finish", finishReason: "stop", totalUsage: { inputTokens: 1, outputTokens: 1 } })}\n`,
)
res.write(
`data: ${JSON.stringify({ id: "mock", object: "chat.completion.chunk", choices: [{ index: 0, delta: {}, finish_reason: "stop" }] })}\n\n`,
)
res.write(
`data: ${JSON.stringify({ id: "mock", object: "chat.completion.chunk", choices: [], usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 } })}\n\n`,
)
res.end("data: [DONE]\n\n")
res.end()
})
})
@@ -176,7 +224,10 @@ try {
const listOutput = result.stdout || result.stderr
assert.match(listOutput, /commandcode/)
assert.match(listOutput, /deepseek\/deepseek-v4-flash/)
assert.equal(modelListRequestCount, 1)
// The failed flag attempt may already load the extension and fetch the
// catalog once before the subcommand fallback runs, so only assert that
// the mock catalog was actually consulted.
assert.ok(modelListRequestCount >= 1)
assert.doesNotThrow(() =>
accessSync(join(tempHome, ".omp", "agent", "commandcode-models.json"), constants.R_OK),
)
@@ -184,6 +235,7 @@ try {
console.log("[omp-compat] print mode through real extension and mock API")
requestCount = 0
requestBodies = []
const print = await runOmp(
["-e", EXT_PATH, "-p", "say mock token", "--model", `commandcode/${TEST_MODEL}`],
30_000,
@@ -199,6 +251,64 @@ try {
assert.equal(lastRequestBody?.model, TEST_MODEL)
assert.ok(Array.isArray(lastRequestBody?.messages))
console.log("[omp-compat] developer advisory reaches the legacy generate request body")
requestCount = 0
requestBodies = []
providerUpgradeRequired = true
const advisoryRun = await runOmp(
[
"-e",
EXT_PATH,
"-e",
ADVISORY_EXT_PATH,
"-p",
"say mock token",
"--model",
`commandcode/${TEST_MODEL}`,
"--no-tools",
"--no-title",
],
30_000,
)
assert.equal(advisoryRun.code, 0, advisoryRun.stderr)
assert.match(advisoryRun.stdout, /mock-omp-ok/)
const promptBodies = requestBodies.filter((body) =>
JSON.stringify(body?.params?.messages ?? []).includes("say mock token"),
)
assert.ok(promptBodies.length >= 1, "expected at least one generate request with the prompt")
for (const body of promptBodies) {
const messages = body?.params?.messages ?? []
const advisoryMessages = messages.filter((message) =>
JSON.stringify(message).includes("Stop and correct the benchmark."),
)
assert.equal(advisoryMessages.length, 1, "the advisory should survive conversion exactly once")
const advisoryMessage = advisoryMessages[0]
assert.equal(advisoryMessage.role, "user")
const advisoryText =
typeof advisoryMessage.content === "string"
? advisoryMessage.content
: (advisoryMessage.content ?? [])
.map((part) => (part?.type === "text" ? part.text : ""))
.join("\n")
assert.equal(advisoryText, ADVISORY_XML, "advisory content must arrive verbatim")
const advisoryIndex = messages.indexOf(advisoryMessage)
const promptIndex = messages.findIndex((message) =>
JSON.stringify(message).includes("say mock token"),
)
assert.ok(
advisoryIndex < promptIndex,
"advisory must keep its chronological position relative to the prompt",
)
assert.doesNotMatch(
String(body?.params?.system ?? ""),
/Stop and correct the benchmark|<advisory/,
"advisory must not be hoisted into the system prompt",
)
}
console.log("[omp-compat] PASS")
} finally {
await new Promise((resolve) => server.close(resolve))
tests/test-pi-local.mjs
+118
View File
@@ -15,6 +15,12 @@ import { fileURLToPath } from "node:url"
const __dirname = dirname(fileURLToPath(import.meta.url))
const PROJECT_DIR = resolve(__dirname, "..")
const EXT_PATH = resolve(PROJECT_DIR, "index.ts")
const COMPAT_CALLER_EXT_PATH = resolve(
PROJECT_DIR,
"tests",
"fixtures",
"compat-caller-extension.ts",
)
const TEST_MODEL = "gpt-5.4"
const CLAUDE_TEST_MODEL = "claude-sonnet-4-6"
@@ -486,6 +492,75 @@ async function runRpcExtensionCommands(timeoutMs = 30_000) {
}
}
async function runRpcCompatCall(timeoutMs = 30_000) {
const child = spawn(
PI_BIN,
[
"--no-extensions",
"--mode",
"rpc",
"-e",
EXT_PATH,
"-e",
COMPAT_CALLER_EXT_PATH,
"--provider",
"commandcode",
"--model",
TEST_MODEL,
],
{
cwd: PROJECT_DIR,
env,
stdio: ["pipe", "pipe", "pipe"],
},
)
let buffer = ""
let stderr = ""
const notification = new Promise((resolve, reject) => {
const timer = setTimeout(
() => reject(new Error(`compat-call timeout. stderr: ${stderr.slice(-500)}`)),
timeoutMs,
)
child.stdout.on("data", (chunk) => {
buffer += chunk.toString("utf-8")
const lines = buffer.split("\n")
buffer = lines.pop() ?? ""
for (const line of lines) {
if (!line.trim()) continue
let event
try {
event = JSON.parse(line)
} catch {
continue
}
if (
event.type === "extension_ui_request" &&
event.method === "notify" &&
typeof event.message === "string" &&
event.message.startsWith("compat-call")
) {
clearTimeout(timer)
resolve(event.message)
}
}
})
child.stderr.on("data", (chunk) => {
stderr += chunk.toString("utf-8")
})
})
try {
child.stdin.write(
`${JSON.stringify({ id: "compat", type: "prompt", message: "/compat-call" })}\n`,
)
return { message: await notification, stderr }
} finally {
child.kill()
}
}
async function runRpcOverflowRecovery(timeoutMs = 60_000) {
const child = spawn(
PI_BIN,
@@ -678,6 +753,37 @@ try {
modelsDelayMs = 0
delete env.COMMANDCODE_MODELS_TIMEOUT_MS
console.log("[pi-local] cached catalog starts without waiting for slow discovery")
const warmup = await runPi(
["--no-extensions", "-e", EXT_PATH, "--list-models", "commandcode"],
20_000,
)
assert.equal(warmup.code, 0, warmup.stderr)
assert.doesNotThrow(() => accessSync(modelsCachePath, constants.R_OK))
modelsDelayMs = 5_000
requestCount = 0
const cachedStartedAt = Date.now()
const cachedPrint = await runPi(
[
"--no-extensions",
"-e",
EXT_PATH,
"-p",
"say mock token",
"--provider",
"commandcode",
"--model",
TEST_MODEL,
],
30_000,
)
const cachedElapsedMs = Date.now() - cachedStartedAt
assert.equal(cachedPrint.code, 0, cachedPrint.stderr)
assert.match(cachedPrint.stdout, /mock-pi-ok/)
assert.equal(requestCount, 1)
assert.ok(cachedElapsedMs < 5_000, `cached start took ${cachedElapsedMs}ms`)
modelsDelayMs = 0
console.log("[pi-local] print mode with reasoning and tool schemas")
requestCount = 0
const print = await runPi(
@@ -797,6 +903,18 @@ try {
JSON.stringify(imageContent),
)
console.log("[pi-local] sibling extension streams through the pi-ai compat registry")
requestCount = 0
const compatCall = await runRpcCompatCall()
assert.equal(compatCall.message, "compat-call ok: mock-pi-ok", compatCall.stderr)
assert.equal(requestCount, 1)
assert.equal(lastRequestBody?.model, TEST_MODEL)
assert.ok(
typeof lastRequestHeaders.authorization === "string" &&
lastRequestHeaders.authorization.startsWith("Bearer "),
"compat call should send a bearer Authorization header",
)
console.log("[pi-local] verify overflow normalization and compaction recovery")
overflowMode = true
overflowRequestCount = 0
tests/test-pricing.ts
+37 -7
View File
@@ -27,7 +27,7 @@ const fixtureUrl = new URL("./fixtures/commandcode-model-ids.json", import.meta.
const fixture = JSON.parse(await readFile(fixtureUrl, "utf-8")) as ModelCatalogSnapshot
const pricingFixtureUrl = new URL("./fixtures/commandcode-pricing.json", import.meta.url)
const pricingFixture = JSON.parse(await readFile(pricingFixtureUrl, "utf-8")) as PricingSnapshot
const freeModels = new Set(["poolside/laguna-s-2.1-free", "stealth/ox-alpha"])
const freeModels = new Set(["poolside/laguna-s-2.1-free"])
function assertCost(
modelId: string,
@@ -50,7 +50,7 @@ function assertCost(
describe("MODEL_COSTS pricing overlay", () => {
it("covers the current Command Code model catalog snapshot", () => {
assert.equal(fixture.source, "https://api.commandcode.ai/provider/v1/models")
assert.match(fixture.fetchedAt, /^2026-08-25T/)
assert.match(fixture.fetchedAt, /^2026-09-01T/)
const catalogIds = [...fixture.modelIds].sort()
const pricedIds = Object.keys(MODEL_COSTS).sort()
@@ -144,11 +144,41 @@ describe("MODEL_COSTS pricing overlay", () => {
cacheRead: 0.04,
cacheWrite: 0,
})
assertCost("Qwen/Qwen3.8-Flash", {
input: 0.16,
output: 0.47,
cacheRead: 0.016,
cacheWrite: 0,
})
assertCost("z-ai/glm-5.3-flash", {
input: 0.15,
output: 0.5,
cacheRead: 0.03,
cacheWrite: 0,
})
assertCost("tencent/hy4-preview", {
input: 0.834,
output: 2.501,
cacheRead: 0.042,
cacheWrite: 0,
})
assertCost("google/gemini-3.7-flash", {
input: 0.75,
output: 3.75,
cacheRead: 0.075,
cacheWrite: 0.04167,
input: 1.5,
output: 7.5,
cacheRead: 0.15,
cacheWrite: 0.08334,
})
assertCost("claude-fable-5-1", {
input: 10,
output: 50,
cacheRead: 0.25,
cacheWrite: 12.5,
})
assertCost("deepseek/deepseek-v4-flash-fast", {
input: 0.28,
output: 0.56,
cacheRead: 0.07,
cacheWrite: 0,
})
assertCost("meta/muse-spark-1.2-contributor", {
input: 0.1,
@@ -196,7 +226,7 @@ describe("MODEL_COSTS pricing overlay", () => {
it("tracks pricing provenance", () => {
assert.equal(PRICING_SOURCE_URL, "https://commandcode.ai/docs/resources/pricing-limits")
assert.equal(PRICING_LAST_VERIFIED, "2026-08-25")
assert.equal(PRICING_LAST_VERIFIED, "2026-09-01")
})
it("fails once temporary pricing needs review", () => {
tests/test-pure-functions.ts
+83
View File
@@ -741,6 +741,89 @@ describe("messagesToCC()", () => {
it("handles empty conversations", () => {
assert.deepEqual(messagesToCC([]), [])
})
it("keeps developer messages instead of dropping them", () => {
const result = messagesToCC([
{ role: "user", content: "start" },
{ role: "developer", content: "mid-conversation steering note" },
])
assert.deepEqual(result, [
{ role: "user", content: "start" },
{ role: "user", content: "mid-conversation steering note" },
])
})
it("converts developer text parts with the same shape as user content", () => {
const result = messagesToCC([
{
role: "developer",
content: [{ type: "text", text: "reminder one" }],
},
])
assert.deepEqual(result, [
{
role: "user",
content: [{ type: "text", text: "reminder one" }],
},
])
})
it("preserves advisory XML verbatim in the serialized request messages", () => {
const advisory =
'<advisory severity="blocker" guidance="weigh, don\'t blindly obey">\nStop and correct the benchmark.\n</advisory>'
const serialized = JSON.stringify(messagesToCC([{ role: "developer", content: advisory }]))
assert.deepEqual(JSON.parse(serialized), [{ role: "user", content: advisory }])
})
it("keeps developer advisories in chronological position without hoisting", () => {
const advisory =
'<advisory severity="blocker" guidance="weigh, don\'t blindly obey">\nStop and correct the benchmark.\n</advisory>'
const result = messagesToCC([
{ role: "user", content: "run the benchmark" },
{
role: "assistant",
content: [
{ type: "text", text: "running it" },
{ type: "toolCall", id: "c1", name: "bash", arguments: { command: "bench" } },
],
},
{
role: "toolResult",
toolCallId: "c1",
toolName: "bash",
content: [{ type: "text", text: "benchmark output" }],
},
{ role: "developer", content: advisory },
{ role: "user", content: "continue" },
])
assert.deepEqual(result, [
{ role: "user", content: "run the benchmark" },
{
role: "assistant",
content: [
{ type: "text", text: "running it" },
{ type: "tool-call", toolCallId: "c1", toolName: "bash", input: { command: "bench" } },
],
},
{
role: "tool",
content: [
{
type: "tool-result",
toolCallId: "c1",
toolName: "bash",
output: { type: "text", value: "benchmark output" },
},
],
},
{ role: "user", content: advisory },
{ role: "user", content: "continue" },
])
})
})
describe("parseStreamEventLine()", () => {
tests/test-runtime.ts
+124
View File
@@ -97,6 +97,7 @@ describe("Command Code runtime", () => {
endpoint: "https://api.commandcode.ai/provider/v1/models?token=user_secret_value",
cachePath: "/tmp/commandcode-models.json",
loadModels: () => firstLoad.promise,
loadCachedModels: async () => [],
createProviderConfig: (models) => ({ models }),
getTransport: () => "provider",
now: () => now,
@@ -105,6 +106,7 @@ describe("Command Code runtime", () => {
const initialization = runtime.initialize()
assert.deepEqual([...pi.commands.keys()], ["commandcode-refresh", "commandcode-status"])
await Promise.resolve()
assert.equal(runtime.getStatus().refreshing, true)
assert.equal(runtime.getStatus().lastAttempt, now)
@@ -142,6 +144,7 @@ describe("Command Code runtime", () => {
if (!next) throw new Error("unexpected refresh")
return next instanceof Promise ? next : next.promise
},
loadCachedModels: async () => [],
createProviderConfig: (models) => ({ models }),
logWarning: (warning) => warnings.push(warning),
})
@@ -189,6 +192,7 @@ describe("Command Code runtime", () => {
if (!result) throw new Error("unexpected refresh")
return result
},
loadCachedModels: async () => [],
createProviderConfig: (models) => ({ models }),
logWarning: () => {},
})
@@ -204,6 +208,123 @@ describe("Command Code runtime", () => {
assert.deepEqual(pi.providers.at(-1)?.models, [FIRST_MODEL, SECOND_MODEL])
})
it("registers the cached catalog immediately and refreshes it in the background", async () => {
const pi = new ExtensionAPITestDouble()
const liveLoad = deferred<LoadCommandCodeModelsResult>()
let now = 1_700_000_000_000
const runtime = createCommandCodeRuntime(pi, {
endpoint: "https://api.commandcode.ai/provider/v1/models",
cachePath: "/tmp/commandcode-models.json",
loadModels: () => liveLoad.promise,
loadCachedModels: async () => [FIRST_MODEL],
createProviderConfig: (models) => ({ models }),
now: () => now,
logWarning: () => {},
})
await runtime.initialize()
assert.equal(pi.providers.length, 1)
assert.deepEqual(pi.providers[0]?.models, [FIRST_MODEL])
assert.equal(runtime.getStatus().source, "cache")
assert.equal(runtime.getStatus().modelCount, 1)
assert.equal(runtime.getStatus().refreshing, true)
now += 1_000
liveLoad.resolve(loaded([FIRST_MODEL, SECOND_MODEL]))
await runtime.refresh()
assert.equal(pi.providers.length, 2)
assert.deepEqual(pi.providers[1]?.models, [FIRST_MODEL, SECOND_MODEL])
assert.equal(runtime.getStatus().source, "live")
assert.equal(runtime.getStatus().modelCount, 2)
assert.equal(runtime.getStatus().refreshing, false)
})
it("keeps the cached catalog when the background refresh fails", async () => {
const pi = new ExtensionAPITestDouble()
const warnings: string[] = []
const runtime = createCommandCodeRuntime(pi, {
endpoint: "https://api.commandcode.ai/provider/v1/models",
cachePath: "/tmp/commandcode-models.json",
loadModels: async () => {
throw new Error("offline")
},
loadCachedModels: async () => [FIRST_MODEL],
createProviderConfig: (models) => ({ models }),
logWarning: (message) => warnings.push(message),
})
await runtime.initialize()
await runtime.refresh()
assert.equal(pi.providers.length, 1)
assert.deepEqual(pi.providers[0]?.models, [FIRST_MODEL])
assert.equal(runtime.getStatus().source, "cache")
assert.equal(runtime.getStatus().modelCount, 1)
assert.match(runtime.getStatus().warning ?? "", /offline/)
assert.equal(warnings.length, 1)
})
it("aborts the background refresh on dispose without reporting a warning", async () => {
const pi = new ExtensionAPITestDouble()
const warnings: string[] = []
let refreshSignal: AbortSignal | undefined
const runtime = createCommandCodeRuntime(pi, {
endpoint: "https://api.commandcode.ai/provider/v1/models",
cachePath: "/tmp/commandcode-models.json",
loadModels: (signal) =>
new Promise((_resolve, reject) => {
refreshSignal = signal
signal.addEventListener("abort", () => reject(signal.reason), { once: true })
}),
loadCachedModels: async () => [FIRST_MODEL],
createProviderConfig: (models) => ({ models }),
logWarning: (message) => warnings.push(message),
})
await runtime.initialize()
const pending = runtime.refresh()
assert.equal(refreshSignal?.aborted, false)
runtime.dispose()
const result = await pending
assert.equal(refreshSignal?.aborted, true)
assert.equal(result.refreshed, false)
assert.equal(runtime.getStatus().refreshing, false)
assert.equal(runtime.getStatus().warning, undefined)
assert.deepEqual(warnings, [])
assert.deepEqual(pi.providers[0]?.models, [FIRST_MODEL])
})
it("awaits the live catalog when no cache exists", async () => {
const pi = new ExtensionAPITestDouble()
const liveLoad = deferred<LoadCommandCodeModelsResult>()
const runtime = createCommandCodeRuntime(pi, {
endpoint: "https://api.commandcode.ai/provider/v1/models",
cachePath: "/tmp/commandcode-models.json",
loadModels: () => liveLoad.promise,
loadCachedModels: async () => [],
createProviderConfig: (models) => ({ models }),
logWarning: () => {},
})
let initialized = false
const initialization = runtime.initialize().then(() => {
initialized = true
})
await Promise.resolve()
assert.equal(pi.providers.length, 0)
assert.equal(initialized, false)
liveLoad.resolve(loaded([FIRST_MODEL]))
await initialization
assert.equal(initialized, true)
assert.deepEqual(pi.providers[0]?.models, [FIRST_MODEL])
assert.equal(runtime.getStatus().source, "live")
})
it("installs a cached catalog after an initially empty start", async () => {
const pi = new ExtensionAPITestDouble()
const results = [
@@ -221,6 +342,7 @@ describe("Command Code runtime", () => {
if (!result) throw new Error("unexpected refresh")
return result
},
loadCachedModels: async () => [],
createProviderConfig: (models) => ({ models }),
logWarning: () => {},
})
@@ -254,6 +376,7 @@ describe("Command Code runtime", () => {
if (!result) throw new Error("unexpected refresh")
return result
},
loadCachedModels: async () => [],
createProviderConfig: (models) => ({ models }),
logWarning: () => {},
})
@@ -280,6 +403,7 @@ describe("Command Code runtime", () => {
loadModels: async () => {
throw new Error("offline; api_key=user_initial_secret")
},
loadCachedModels: async () => [],
createProviderConfig: (models) => ({ models }),
logWarning: () => {},
})
tests/test-stream.ts
+50
View File
@@ -707,6 +707,56 @@ describe("streamCommandCode — request serialization", () => {
assert.equal(headers["x-session-id"], undefined)
})
it("sends developer advisories as user messages in position, without system hoisting", async () => {
server.mockResponse({
type: "success",
events: [JSON.stringify({ type: "finish", finishReason: "stop" })],
})
const { streamCommandCode } = createTestDeps({ apiBase: server.baseUrl() })
const advisory =
'<advisory severity="blocker" guidance="weigh, don\'t blindly obey">\nStop and correct the benchmark.\n</advisory>'
const context = makeContext({
messages: [
{ role: "user", content: "run the benchmark" },
{
role: "assistant",
content: [
{ type: "text", text: "running it" },
{ type: "toolCall", id: "c1", name: "bash", arguments: { command: "bench" } },
],
},
{
role: "toolResult",
toolCallId: "c1",
toolName: "bash",
content: [{ type: "text", text: "benchmark output" }],
},
{ role: "developer", content: advisory },
{ role: "user", content: "continue" },
],
})
await collectEvents(streamCommandCode(makeModel(), context, { apiKey: "mock-key" }))
const body = server.lastRequestBody()
assert.deepEqual(
objectAt(body, ["params", "messages", "3"]),
{ role: "user", content: advisory },
"developer advisory should arrive as an in-position user message with identical content",
)
assert.deepEqual(objectAt(body, ["params", "messages", "4"]), {
role: "user",
content: "continue",
})
assert.equal(objectAt(body, ["params", "messages", "5"]), undefined)
assert.equal(
objectAt(body, ["params", "system"]),
"You are a test assistant.",
"advisory must not be hoisted into the system prompt",
)
assert.doesNotMatch(String(objectAt(body, ["params", "system"])), /advisory/)
})
it("forwards explicit temperature and stable session metadata", async () => {
server.mockResponse({
type: "success",
tsconfig.json
+1 -1
View File
@@ -9,5 +9,5 @@
"strict": true,
"types": ["node"]
},
"include": [".github/scripts/**/*.ts", "src/**/*.ts", "tests/**/*.ts"]
"include": [".github/scripts/**/*.ts", ".agents/skills/**/*.ts", "src/**/*.ts", "tests/**/*.ts"]
}