feat(models): move manual reasoning efforts into a catalog override

PR #69 added efforts for meta/muse-spark-* directly to the generated
catalog, which the drift check flags and the daily sync job reverts.

Keep src/commandcode-catalog.ts byte-identical to upstream and merge a
separate src/commandcode-catalog-overrides.ts over it at load time. A
test fails as soon as upstream publishes efforts for an overridden model
so the override gets removed instead of shadowing the CLI catalog.

Verified against the live endpoint: pi --thinking xhigh sends
reasoning_effort="xhigh" for meta/muse-spark-1.2-contributor and the
request succeeds; --thinking off sends none.

Closes #69
This commit is contained in:
Patrick Wozniak
2026-09-02 00:03:45 +02:00
parent df4f4a6eef
commit d3c9832d28
7 changed files with 61 additions and 7 deletions
@@ -32,6 +32,8 @@ npm run sync:commandcode-catalog
Regenerates `src/commandcode-catalog.ts` and bumps the documented CLI version in `README.md`. Review the diff; the catalog also lists reasoning models without selectable efforts.
Never add efforts to the generated file by hand. Manual effort policy for reasoning models that upstream ships without levels lives in `src/commandcode-catalog-overrides.ts` and is merged at load time. When the sync report lists a model from that file under "New effort metadata", remove its override; `tests/test-models.ts` fails until you do.
### 3. Update display pricing (manual review)
Fetch <https://commandcode.ai/docs/resources/pricing-limits> and compare against `src/pricing.ts`:
+1
View File
@@ -2,6 +2,7 @@
## Unreleased
- Expose selectable thinking levels (`minimal`, `low`, `medium`, `high`, `xhigh`) for `meta/muse-spark-1.1`, `meta/muse-spark-1.2`, and `meta/muse-spark-1.2-contributor` through a manual catalog override, so `/thinking` and `Shift+Tab` no longer stay locked on `off` for these reasoning models.
- Start from the cached model catalog and refresh it in the background instead of blocking host startup on the catalog request; a first start without a cache still waits for the live catalog.
- Register the `commandcode-custom` API in the `@earendil-works/pi-ai/compat` registry so sibling extensions that stream with the active Command Code model no longer fail with `No API provider registered for api: commandcode-custom` on plain pi.
- Assert structural catalog invariants in the model tests so the daily catalog sync no longer fails on every upstream change.
+1 -1
View File
@@ -88,7 +88,7 @@ Open `/model` and select one of the models provided by Command Code. Model avail
### Reasoning support
Reasoning capability and selectable effort levels follow the official CLI catalog independently. Models can therefore be marked as reasoning-capable even when Command Code chooses their depth automatically; for those models (for example `meta/muse-spark-1.1`, `meta/muse-spark-1.2`, and `meta/muse-spark-1.2-contributor`) the thinking level stays `off` because the CLI itself sends no effort parameter and Command Code has not published selectable levels. Models with explicit effort support register a model-specific `thinkingLevelMap`, so pi and OMP expose only valid levels. Pi's native OpenAI- and Anthropic-compatible providers translate the selected level for Provider API accounts; the existing Command Code generate transport sends the matching `reasoning_effort` for Go accounts.
Reasoning capability and selectable effort levels follow the official CLI catalog independently. Models can therefore be marked as reasoning-capable even when Command Code chooses their depth automatically. Models with explicit effort support register a model-specific `thinkingLevelMap`, so pi and OMP expose only valid levels. For a few reasoning models the CLI catalog ships no effort levels although the endpoint accepts `reasoning_effort`; `src/commandcode-catalog-overrides.ts` adds a manual level set for those (currently `meta/muse-spark-1.1`, `meta/muse-spark-1.2`, and `meta/muse-spark-1.2-contributor`) on top of the generated catalog, and the tests fail once upstream publishes its own levels so the override gets removed. Pi's native OpenAI- and Anthropic-compatible providers translate the selected level for Provider API accounts; the existing Command Code generate transport sends the matching `reasoning_effort` for Go accounts.
List Command Code models from the terminal:
+23
View File
@@ -0,0 +1,23 @@
import type { CommandCodeReasoningEffort } from "./commandcode-catalog.ts"
/**
* Manual reasoning-effort policy for models the official CLI marks as
* reasoning-capable without publishing selectable efforts.
*
* `src/commandcode-catalog.ts` is generated from the CLI package and must stay
* byte-identical to upstream so the daily drift check works. Entries here are
* merged over the generated catalog at load time and are not touched by
* `npm run sync:commandcode-catalog`.
*
* Add a model only when the effort parameter is known to be accepted by the
* Command Code endpoint; remove it once the CLI catalog ships its own efforts.
*/
export const MODEL_EFFORT_OVERRIDES: Readonly<
Record<string, readonly CommandCodeReasoningEffort[]>
> = {
// Meta Muse Spark: the CLI ships no effort levels, but the endpoint accepts
// `reasoning_effort` for these models and other hosts expose the same set.
"meta/muse-spark-1.1": ["minimal", "low", "medium", "high", "xhigh"],
"meta/muse-spark-1.2": ["minimal", "low", "medium", "high", "xhigh"],
"meta/muse-spark-1.2-contributor": ["minimal", "low", "medium", "high", "xhigh"],
}
-3
View File
@@ -133,9 +133,6 @@ export const MODEL_EFFORTS: Readonly<Record<string, readonly CommandCodeReasonin
"gpt-5.6-sol": ["low", "medium", "high", "xhigh", "max"],
"gpt-5.6-terra": ["low", "medium", "high", "xhigh", "max"],
"moonshotai/Kimi-K3": ["low", "high", "max"],
"meta/muse-spark-1.1": ["minimal", "low", "medium", "high", "xhigh"],
"meta/muse-spark-1.2": ["minimal", "low", "medium", "high", "xhigh"],
"meta/muse-spark-1.2-contributor": ["minimal", "low", "medium", "high", "xhigh"],
"Qwen/Qwen3.8-27B": ["low", "medium", "xhigh"],
"Qwen/Qwen3.8-Flash": ["low", "medium", "xhigh"],
"Qwen/Qwen3.8-Max": ["low", "medium", "xhigh"],
+9 -2
View File
@@ -1,8 +1,9 @@
import { mkdir, readFile, rename, rm, writeFile } from "node:fs/promises"
import { dirname } from "node:path"
import { MODEL_EFFORT_OVERRIDES } from "./commandcode-catalog-overrides.ts"
import {
MODEL_EFFORTS,
MODEL_EFFORTS as CATALOG_MODEL_EFFORTS,
MODEL_INPUT_MODALITIES,
MODEL_MAX_OUTPUT_TOKENS,
MODEL_REASONING,
@@ -10,7 +11,13 @@ import {
type CommandCodeReasoningEffort,
} from "./commandcode-catalog.ts"
export { MODEL_EFFORTS, MODEL_INPUT_MODALITIES, MODEL_MAX_OUTPUT_TOKENS, MODEL_REASONING }
/** Upstream CLI efforts with the manual overrides merged over them. */
export const MODEL_EFFORTS: Readonly<Record<string, readonly CommandCodeReasoningEffort[]>> = {
...CATALOG_MODEL_EFFORTS,
...MODEL_EFFORT_OVERRIDES,
}
export { MODEL_INPUT_MODALITIES, MODEL_MAX_OUTPUT_TOKENS, MODEL_REASONING }
export type { CommandCodeInputType }
export const DEFAULT_PROVIDER_API_BASE = "https://api.commandcode.ai/provider/v1"
+25 -1
View File
@@ -4,7 +4,11 @@ import { tmpdir } from "node:os"
import { join } from "node:path"
import { describe, it } from "node:test"
import { COMMAND_CODE_CLI_VERSION } from "../src/commandcode-catalog.ts"
import { MODEL_EFFORT_OVERRIDES } from "../src/commandcode-catalog-overrides.ts"
import {
COMMAND_CODE_CLI_VERSION,
MODEL_EFFORTS as CATALOG_MODEL_EFFORTS,
} from "../src/commandcode-catalog.ts"
import {
apiForModelId,
baseUrlForModel,
@@ -192,6 +196,26 @@ describe("commandCodeModelsFromApiResponse()", () => {
}
})
it("merges manual effort overrides over the generated catalog", () => {
const validEfforts = new Set(["minimal", "low", "medium", "high", "xhigh", "max"])
assert.ok(Object.keys(MODEL_EFFORT_OVERRIDES).length > 0)
for (const [modelId, efforts] of Object.entries(MODEL_EFFORT_OVERRIDES)) {
assert.equal(MODEL_REASONING[modelId], true, `${modelId} override needs a reasoning flag`)
assert.equal(
CATALOG_MODEL_EFFORTS[modelId],
undefined,
`${modelId} now has upstream efforts; drop the manual override`,
)
assert.ok(efforts.length > 0)
assert.ok(efforts.every((effort) => validEfforts.has(effort)))
assert.deepEqual(MODEL_EFFORTS[modelId], efforts)
assert.deepEqual(thinkingMetadataForModel(modelId)?.thinking?.efforts, efforts)
}
for (const [modelId, efforts] of Object.entries(CATALOG_MODEL_EFFORTS)) {
assert.deepEqual(MODEL_EFFORTS[modelId], efforts, `${modelId} upstream efforts changed`)
}
})
it("builds separate canonical pi and OMP metadata", () => {
for (const [modelId, efforts] of Object.entries(MODEL_EFFORTS)) {
const metadata = thinkingMetadataForModel(modelId)