feat(models): move manual reasoning efforts into a catalog override
PR #69 added efforts for meta/muse-spark-* directly to the generated catalog, which the drift check flags and the daily sync job reverts. Keep src/commandcode-catalog.ts byte-identical to upstream and merge a separate src/commandcode-catalog-overrides.ts over it at load time. A test fails as soon as upstream publishes efforts for an overridden model so the override gets removed instead of shadowing the CLI catalog. Verified against the live endpoint: pi --thinking xhigh sends reasoning_effort="xhigh" for meta/muse-spark-1.2-contributor and the request succeeds; --thinking off sends none. Closes #69
This commit is contained in:
@@ -32,6 +32,8 @@ npm run sync:commandcode-catalog
|
|||||||
|
|
||||||
Regenerates `src/commandcode-catalog.ts` and bumps the documented CLI version in `README.md`. Review the diff; the catalog also lists reasoning models without selectable efforts.
|
Regenerates `src/commandcode-catalog.ts` and bumps the documented CLI version in `README.md`. Review the diff; the catalog also lists reasoning models without selectable efforts.
|
||||||
|
|
||||||
|
Never add efforts to the generated file by hand. Manual effort policy for reasoning models that upstream ships without levels lives in `src/commandcode-catalog-overrides.ts` and is merged at load time. When the sync report lists a model from that file under "New effort metadata", remove its override; `tests/test-models.ts` fails until you do.
|
||||||
|
|
||||||
### 3. Update display pricing (manual review)
|
### 3. Update display pricing (manual review)
|
||||||
|
|
||||||
Fetch <https://commandcode.ai/docs/resources/pricing-limits> and compare against `src/pricing.ts`:
|
Fetch <https://commandcode.ai/docs/resources/pricing-limits> and compare against `src/pricing.ts`:
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
|
- Expose selectable thinking levels (`minimal`, `low`, `medium`, `high`, `xhigh`) for `meta/muse-spark-1.1`, `meta/muse-spark-1.2`, and `meta/muse-spark-1.2-contributor` through a manual catalog override, so `/thinking` and `Shift+Tab` no longer stay locked on `off` for these reasoning models.
|
||||||
- Start from the cached model catalog and refresh it in the background instead of blocking host startup on the catalog request; a first start without a cache still waits for the live catalog.
|
- Start from the cached model catalog and refresh it in the background instead of blocking host startup on the catalog request; a first start without a cache still waits for the live catalog.
|
||||||
- Register the `commandcode-custom` API in the `@earendil-works/pi-ai/compat` registry so sibling extensions that stream with the active Command Code model no longer fail with `No API provider registered for api: commandcode-custom` on plain pi.
|
- Register the `commandcode-custom` API in the `@earendil-works/pi-ai/compat` registry so sibling extensions that stream with the active Command Code model no longer fail with `No API provider registered for api: commandcode-custom` on plain pi.
|
||||||
- Assert structural catalog invariants in the model tests so the daily catalog sync no longer fails on every upstream change.
|
- Assert structural catalog invariants in the model tests so the daily catalog sync no longer fails on every upstream change.
|
||||||
|
|||||||
@@ -88,7 +88,7 @@ Open `/model` and select one of the models provided by Command Code. Model avail
|
|||||||
|
|
||||||
### Reasoning support
|
### Reasoning support
|
||||||
|
|
||||||
Reasoning capability and selectable effort levels follow the official CLI catalog independently. Models can therefore be marked as reasoning-capable even when Command Code chooses their depth automatically; for those models (for example `meta/muse-spark-1.1`, `meta/muse-spark-1.2`, and `meta/muse-spark-1.2-contributor`) the thinking level stays `off` because the CLI itself sends no effort parameter and Command Code has not published selectable levels. Models with explicit effort support register a model-specific `thinkingLevelMap`, so pi and OMP expose only valid levels. Pi's native OpenAI- and Anthropic-compatible providers translate the selected level for Provider API accounts; the existing Command Code generate transport sends the matching `reasoning_effort` for Go accounts.
|
Reasoning capability and selectable effort levels follow the official CLI catalog independently. Models can therefore be marked as reasoning-capable even when Command Code chooses their depth automatically. Models with explicit effort support register a model-specific `thinkingLevelMap`, so pi and OMP expose only valid levels. For a few reasoning models the CLI catalog ships no effort levels although the endpoint accepts `reasoning_effort`; `src/commandcode-catalog-overrides.ts` adds a manual level set for those (currently `meta/muse-spark-1.1`, `meta/muse-spark-1.2`, and `meta/muse-spark-1.2-contributor`) on top of the generated catalog, and the tests fail once upstream publishes its own levels so the override gets removed. Pi's native OpenAI- and Anthropic-compatible providers translate the selected level for Provider API accounts; the existing Command Code generate transport sends the matching `reasoning_effort` for Go accounts.
|
||||||
|
|
||||||
List Command Code models from the terminal:
|
List Command Code models from the terminal:
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
import type { CommandCodeReasoningEffort } from "./commandcode-catalog.ts"
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Manual reasoning-effort policy for models the official CLI marks as
|
||||||
|
* reasoning-capable without publishing selectable efforts.
|
||||||
|
*
|
||||||
|
* `src/commandcode-catalog.ts` is generated from the CLI package and must stay
|
||||||
|
* byte-identical to upstream so the daily drift check works. Entries here are
|
||||||
|
* merged over the generated catalog at load time and are not touched by
|
||||||
|
* `npm run sync:commandcode-catalog`.
|
||||||
|
*
|
||||||
|
* Add a model only when the effort parameter is known to be accepted by the
|
||||||
|
* Command Code endpoint; remove it once the CLI catalog ships its own efforts.
|
||||||
|
*/
|
||||||
|
export const MODEL_EFFORT_OVERRIDES: Readonly<
|
||||||
|
Record<string, readonly CommandCodeReasoningEffort[]>
|
||||||
|
> = {
|
||||||
|
// Meta Muse Spark: the CLI ships no effort levels, but the endpoint accepts
|
||||||
|
// `reasoning_effort` for these models and other hosts expose the same set.
|
||||||
|
"meta/muse-spark-1.1": ["minimal", "low", "medium", "high", "xhigh"],
|
||||||
|
"meta/muse-spark-1.2": ["minimal", "low", "medium", "high", "xhigh"],
|
||||||
|
"meta/muse-spark-1.2-contributor": ["minimal", "low", "medium", "high", "xhigh"],
|
||||||
|
}
|
||||||
@@ -133,9 +133,6 @@ export const MODEL_EFFORTS: Readonly<Record<string, readonly CommandCodeReasonin
|
|||||||
"gpt-5.6-sol": ["low", "medium", "high", "xhigh", "max"],
|
"gpt-5.6-sol": ["low", "medium", "high", "xhigh", "max"],
|
||||||
"gpt-5.6-terra": ["low", "medium", "high", "xhigh", "max"],
|
"gpt-5.6-terra": ["low", "medium", "high", "xhigh", "max"],
|
||||||
"moonshotai/Kimi-K3": ["low", "high", "max"],
|
"moonshotai/Kimi-K3": ["low", "high", "max"],
|
||||||
"meta/muse-spark-1.1": ["minimal", "low", "medium", "high", "xhigh"],
|
|
||||||
"meta/muse-spark-1.2": ["minimal", "low", "medium", "high", "xhigh"],
|
|
||||||
"meta/muse-spark-1.2-contributor": ["minimal", "low", "medium", "high", "xhigh"],
|
|
||||||
"Qwen/Qwen3.8-27B": ["low", "medium", "xhigh"],
|
"Qwen/Qwen3.8-27B": ["low", "medium", "xhigh"],
|
||||||
"Qwen/Qwen3.8-Flash": ["low", "medium", "xhigh"],
|
"Qwen/Qwen3.8-Flash": ["low", "medium", "xhigh"],
|
||||||
"Qwen/Qwen3.8-Max": ["low", "medium", "xhigh"],
|
"Qwen/Qwen3.8-Max": ["low", "medium", "xhigh"],
|
||||||
|
|||||||
+9
-2
@@ -1,8 +1,9 @@
|
|||||||
import { mkdir, readFile, rename, rm, writeFile } from "node:fs/promises"
|
import { mkdir, readFile, rename, rm, writeFile } from "node:fs/promises"
|
||||||
import { dirname } from "node:path"
|
import { dirname } from "node:path"
|
||||||
|
|
||||||
|
import { MODEL_EFFORT_OVERRIDES } from "./commandcode-catalog-overrides.ts"
|
||||||
import {
|
import {
|
||||||
MODEL_EFFORTS,
|
MODEL_EFFORTS as CATALOG_MODEL_EFFORTS,
|
||||||
MODEL_INPUT_MODALITIES,
|
MODEL_INPUT_MODALITIES,
|
||||||
MODEL_MAX_OUTPUT_TOKENS,
|
MODEL_MAX_OUTPUT_TOKENS,
|
||||||
MODEL_REASONING,
|
MODEL_REASONING,
|
||||||
@@ -10,7 +11,13 @@ import {
|
|||||||
type CommandCodeReasoningEffort,
|
type CommandCodeReasoningEffort,
|
||||||
} from "./commandcode-catalog.ts"
|
} from "./commandcode-catalog.ts"
|
||||||
|
|
||||||
export { MODEL_EFFORTS, MODEL_INPUT_MODALITIES, MODEL_MAX_OUTPUT_TOKENS, MODEL_REASONING }
|
/** Upstream CLI efforts with the manual overrides merged over them. */
|
||||||
|
export const MODEL_EFFORTS: Readonly<Record<string, readonly CommandCodeReasoningEffort[]>> = {
|
||||||
|
...CATALOG_MODEL_EFFORTS,
|
||||||
|
...MODEL_EFFORT_OVERRIDES,
|
||||||
|
}
|
||||||
|
|
||||||
|
export { MODEL_INPUT_MODALITIES, MODEL_MAX_OUTPUT_TOKENS, MODEL_REASONING }
|
||||||
export type { CommandCodeInputType }
|
export type { CommandCodeInputType }
|
||||||
|
|
||||||
export const DEFAULT_PROVIDER_API_BASE = "https://api.commandcode.ai/provider/v1"
|
export const DEFAULT_PROVIDER_API_BASE = "https://api.commandcode.ai/provider/v1"
|
||||||
|
|||||||
+25
-1
@@ -4,7 +4,11 @@ import { tmpdir } from "node:os"
|
|||||||
import { join } from "node:path"
|
import { join } from "node:path"
|
||||||
import { describe, it } from "node:test"
|
import { describe, it } from "node:test"
|
||||||
|
|
||||||
import { COMMAND_CODE_CLI_VERSION } from "../src/commandcode-catalog.ts"
|
import { MODEL_EFFORT_OVERRIDES } from "../src/commandcode-catalog-overrides.ts"
|
||||||
|
import {
|
||||||
|
COMMAND_CODE_CLI_VERSION,
|
||||||
|
MODEL_EFFORTS as CATALOG_MODEL_EFFORTS,
|
||||||
|
} from "../src/commandcode-catalog.ts"
|
||||||
import {
|
import {
|
||||||
apiForModelId,
|
apiForModelId,
|
||||||
baseUrlForModel,
|
baseUrlForModel,
|
||||||
@@ -192,6 +196,26 @@ describe("commandCodeModelsFromApiResponse()", () => {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it("merges manual effort overrides over the generated catalog", () => {
|
||||||
|
const validEfforts = new Set(["minimal", "low", "medium", "high", "xhigh", "max"])
|
||||||
|
assert.ok(Object.keys(MODEL_EFFORT_OVERRIDES).length > 0)
|
||||||
|
for (const [modelId, efforts] of Object.entries(MODEL_EFFORT_OVERRIDES)) {
|
||||||
|
assert.equal(MODEL_REASONING[modelId], true, `${modelId} override needs a reasoning flag`)
|
||||||
|
assert.equal(
|
||||||
|
CATALOG_MODEL_EFFORTS[modelId],
|
||||||
|
undefined,
|
||||||
|
`${modelId} now has upstream efforts; drop the manual override`,
|
||||||
|
)
|
||||||
|
assert.ok(efforts.length > 0)
|
||||||
|
assert.ok(efforts.every((effort) => validEfforts.has(effort)))
|
||||||
|
assert.deepEqual(MODEL_EFFORTS[modelId], efforts)
|
||||||
|
assert.deepEqual(thinkingMetadataForModel(modelId)?.thinking?.efforts, efforts)
|
||||||
|
}
|
||||||
|
for (const [modelId, efforts] of Object.entries(CATALOG_MODEL_EFFORTS)) {
|
||||||
|
assert.deepEqual(MODEL_EFFORTS[modelId], efforts, `${modelId} upstream efforts changed`)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
it("builds separate canonical pi and OMP metadata", () => {
|
it("builds separate canonical pi and OMP metadata", () => {
|
||||||
for (const [modelId, efforts] of Object.entries(MODEL_EFFORTS)) {
|
for (const [modelId, efforts] of Object.entries(MODEL_EFFORTS)) {
|
||||||
const metadata = thinkingMetadataForModel(modelId)
|
const metadata = thinkingMetadataForModel(modelId)
|
||||||
|
|||||||
Reference in New Issue
Block a user