Merge pull request #66 from ThomasByr/feat/refresh-model-catalog-skill
chore(QoL): refresh model catalog skill # Conflicts: # CHANGELOG.md
This commit is contained in:
@@ -0,0 +1,76 @@
|
|||||||
|
---
|
||||||
|
name: refresh-model-catalog
|
||||||
|
description: Use when adding or removing Command Code models, refreshing the model catalog snapshot (image, reasoning, effort, output-limit metadata), updating display pricing, or refreshing test fixtures in pi-commandcode-provider.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Refresh Model Catalog
|
||||||
|
|
||||||
|
Use this skill whenever the Command Code model catalog changes: new or retired models, changed reasoning efforts, output limits, or pricing. All commands run from the repository root and work on Windows and Linux.
|
||||||
|
|
||||||
|
## Core rules
|
||||||
|
|
||||||
|
- Do not commit, tag, push, or publish unless the user explicitly asks in the current conversation.
|
||||||
|
- Pricing is manually reviewed: temporary promotions and long-context tiers require explicit review of the official pricing page. Never copy prices blindly from the API.
|
||||||
|
- Keep the change focused: one refresh per PR, no unrelated refactors.
|
||||||
|
- Follow [CONTRIBUTING.md](../../../CONTRIBUTING.md) for commit message rules.
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
### 1. Detect drift
|
||||||
|
|
||||||
|
```sh
|
||||||
|
npm run check:commandcode-catalog
|
||||||
|
```
|
||||||
|
|
||||||
|
This compares the repository snapshot against the latest published `command-code` npm package and reports added/removed models, changed efforts, and version drift. Use the report to scope the work.
|
||||||
|
|
||||||
|
### 2. Sync static model metadata
|
||||||
|
|
||||||
|
```sh
|
||||||
|
npm run sync:commandcode-catalog
|
||||||
|
```
|
||||||
|
|
||||||
|
Regenerates `src/commandcode-catalog.ts` and bumps the documented CLI version in `README.md`. Review the diff; the catalog also lists reasoning models without selectable efforts.
|
||||||
|
|
||||||
|
### 3. Update display pricing (manual review)
|
||||||
|
|
||||||
|
Fetch <https://commandcode.ai/docs/resources/pricing-limits> and compare against `src/pricing.ts`:
|
||||||
|
|
||||||
|
- Add entries for new models and remove entries for retired models. Missing models silently display zero cost, so `MODEL_COSTS` must cover the full catalog.
|
||||||
|
- The pricing page's "Cache Read"/"Cache Write" columns map to `cacheRead`/`cacheWrite`; a "—" column means `0`.
|
||||||
|
- Update `PRICING_LAST_VERIFIED` to today's date.
|
||||||
|
- Add or update `TEMPORARY_PRICING` entries for promotions with an end date, so tests fail when they expire.
|
||||||
|
|
||||||
|
### 4. Refresh the test fixtures
|
||||||
|
|
||||||
|
```sh
|
||||||
|
node .agents/skills/refresh-model-catalog/scripts/refresh-model-ids.mjs
|
||||||
|
npx tsx .agents/skills/refresh-model-catalog/scripts/sync-pricing-fixture.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
The first script snapshots the live model-id list into `tests/fixtures/commandcode-model-ids.json`; the second regenerates `tests/fixtures/commandcode-pricing.json` from `MODEL_COSTS`. The pricing test fails until `MODEL_COSTS` matches the catalog snapshot exactly.
|
||||||
|
|
||||||
|
### 5. Update test expectations
|
||||||
|
|
||||||
|
Adjust the model-specific assertions that the refresh invalidated, typically in:
|
||||||
|
|
||||||
|
- `tests/test-pricing.ts`: fixture date assertions, the `freeModels` set, and per-model rate assertions.
|
||||||
|
- `tests/test-models.ts`: image/reasoning/effort/output-limit assertions and catalog entry counts.
|
||||||
|
|
||||||
|
Do not weaken assertions to make them pass; update them to the verified upstream values.
|
||||||
|
|
||||||
|
### 6. Validate
|
||||||
|
|
||||||
|
```sh
|
||||||
|
npm run test:models
|
||||||
|
npm run test:pricing
|
||||||
|
npm run typecheck
|
||||||
|
npm run format:check
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
Run the full `npm test` before reporting the work as done when the environment allows it.
|
||||||
|
|
||||||
|
### 7. Document
|
||||||
|
|
||||||
|
Add entries to the `Unreleased` section of `CHANGELOG.md` covering new/retired models, effort changes, and pricing refreshes.
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
// Refreshes tests/fixtures/commandcode-model-ids.json from the live Command Code
|
||||||
|
// models API. Run from the repository root:
|
||||||
|
// node .agents/skills/refresh-model-catalog/scripts/refresh-model-ids.mjs
|
||||||
|
import { writeFile } from "node:fs/promises"
|
||||||
|
|
||||||
|
import { format, resolveConfig } from "prettier"
|
||||||
|
|
||||||
|
const MODELS_URL = "https://api.commandcode.ai/provider/v1/models"
|
||||||
|
const FIXTURE_PATH = new URL(
|
||||||
|
"../../../../tests/fixtures/commandcode-model-ids.json",
|
||||||
|
import.meta.url,
|
||||||
|
)
|
||||||
|
|
||||||
|
const response = await fetch(MODELS_URL)
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`Failed to fetch Command Code models: ${response.status} ${response.statusText}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
const body = await response.json()
|
||||||
|
if (body?.object !== "list" || !Array.isArray(body.data)) {
|
||||||
|
throw new Error("Expected a Command Code models list response")
|
||||||
|
}
|
||||||
|
|
||||||
|
const modelIds = body.data.map((model) => {
|
||||||
|
if (typeof model?.id !== "string" || model.id.length === 0) {
|
||||||
|
throw new Error("Expected each model entry to have a non-empty id")
|
||||||
|
}
|
||||||
|
return model.id
|
||||||
|
})
|
||||||
|
if (modelIds.length === 0) throw new Error("Command Code returned an empty model catalog")
|
||||||
|
|
||||||
|
const fixture = { fetchedAt: new Date().toISOString(), source: MODELS_URL, modelIds }
|
||||||
|
const options = await resolveConfig(new URL("../../../../.prettierrc.json", import.meta.url))
|
||||||
|
const contents = await format(JSON.stringify(fixture), {
|
||||||
|
...options,
|
||||||
|
filepath: "commandcode-model-ids.json",
|
||||||
|
})
|
||||||
|
await writeFile(FIXTURE_PATH, contents, "utf-8")
|
||||||
|
console.log(`Wrote ${modelIds.length} model ids to tests/fixtures/commandcode-model-ids.json`)
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
// Regenerates tests/fixtures/commandcode-pricing.json from src/pricing.ts so the
|
||||||
|
// snapshot always matches MODEL_COSTS. Run from the repository root:
|
||||||
|
// npx tsx .agents/skills/refresh-model-catalog/scripts/sync-pricing-fixture.ts
|
||||||
|
import { writeFile } from "node:fs/promises"
|
||||||
|
|
||||||
|
import { format, resolveConfig } from "prettier"
|
||||||
|
|
||||||
|
import { MODEL_COSTS, PRICING_LAST_VERIFIED, PRICING_SOURCE_URL } from "../../../../src/pricing.ts"
|
||||||
|
|
||||||
|
const FIXTURE_PATH = new URL("../../../../tests/fixtures/commandcode-pricing.json", import.meta.url)
|
||||||
|
|
||||||
|
const costs: Record<string, [number, number, number, number]> = {}
|
||||||
|
const tiers: Record<string, [number, number, number, number, number][]> = {}
|
||||||
|
for (const [modelId, cost] of Object.entries(MODEL_COSTS)) {
|
||||||
|
costs[modelId] = [cost.input, cost.output, cost.cacheRead, cost.cacheWrite]
|
||||||
|
if (cost.tiers) {
|
||||||
|
tiers[modelId] = cost.tiers.map((tier) => [
|
||||||
|
tier.inputTokensAbove,
|
||||||
|
tier.input,
|
||||||
|
tier.output,
|
||||||
|
tier.cacheRead,
|
||||||
|
tier.cacheWrite,
|
||||||
|
])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const fixture = {
|
||||||
|
verifiedAt: PRICING_LAST_VERIFIED,
|
||||||
|
source: PRICING_SOURCE_URL,
|
||||||
|
tierPolicy:
|
||||||
|
"Use request-wide input tiers; the highest threshold exceeded by input plus cache tokens applies to the full request.",
|
||||||
|
tiers,
|
||||||
|
costs,
|
||||||
|
}
|
||||||
|
const options = await resolveConfig(new URL("../../../../.prettierrc.json", import.meta.url))
|
||||||
|
const contents = await format(JSON.stringify(fixture), {
|
||||||
|
...options,
|
||||||
|
filepath: "commandcode-pricing.json",
|
||||||
|
})
|
||||||
|
await writeFile(FIXTURE_PATH, contents, "utf-8")
|
||||||
|
console.log(
|
||||||
|
`Wrote ${Object.keys(costs).length} model prices to tests/fixtures/commandcode-pricing.json`,
|
||||||
|
)
|
||||||
@@ -18,6 +18,26 @@ const MODELS_REFERENCE_PATH = "dist/bundled/command-code-knowledge/reference/mod
|
|||||||
const CLI_BUNDLE_PATH = "dist/cli.mjs"
|
const CLI_BUNDLE_PATH = "dist/cli.mjs"
|
||||||
const TEXT_ONLY_MARKER = ',__name(isKnownTextOnlyModel,"isKnownTextOnlyModel")'
|
const TEXT_ONLY_MARKER = ',__name(isKnownTextOnlyModel,"isKnownTextOnlyModel")'
|
||||||
const VALID_EFFORTS = new Set(["minimal", "low", "medium", "high", "xhigh", "max"])
|
const VALID_EFFORTS = new Set(["minimal", "low", "medium", "high", "xhigh", "max"])
|
||||||
|
|
||||||
|
function quoteWindowsArgument(argument: string): string {
|
||||||
|
if (argument.length === 0) return '""'
|
||||||
|
if (!/[\s"]/.test(argument)) return argument
|
||||||
|
return `"${argument.replaceAll('"', '\\"')}"`
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Run npm on Windows and POSIX. npm is a `.cmd` shim on Windows, which
|
||||||
|
* execFile cannot spawn directly, so route it through the shell there with
|
||||||
|
* shell-safe quoting.
|
||||||
|
*/
|
||||||
|
function execNpmFileAsync(
|
||||||
|
args: readonly string[],
|
||||||
|
options: { cwd: string; encoding: "utf-8"; maxBuffer?: number },
|
||||||
|
): Promise<{ stdout: string }> {
|
||||||
|
if (process.platform !== "win32") return execFileAsync("npm", args, options)
|
||||||
|
const command = ["npm", ...args.map(quoteWindowsArgument)].join(" ")
|
||||||
|
return execFileAsync(command, { ...options, shell: true })
|
||||||
|
}
|
||||||
const CATALOG_SOURCE_PATH = new URL("../../src/commandcode-catalog.ts", import.meta.url)
|
const CATALOG_SOURCE_PATH = new URL("../../src/commandcode-catalog.ts", import.meta.url)
|
||||||
const README_PATH = new URL("../../README.md", import.meta.url)
|
const README_PATH = new URL("../../README.md", import.meta.url)
|
||||||
|
|
||||||
@@ -417,8 +437,7 @@ async function resolvePackageSpec(
|
|||||||
): Promise<string> {
|
): Promise<string> {
|
||||||
if (packageSpec !== "command-code@latest") return packageSpec
|
if (packageSpec !== "command-code@latest") return packageSpec
|
||||||
|
|
||||||
const { stdout } = await execFileAsync(
|
const { stdout } = await execNpmFileAsync(
|
||||||
"npm",
|
|
||||||
["view", packageSpec, "version", "--json", "--prefer-online", "--cache", npmCacheDirectory],
|
["view", packageSpec, "version", "--json", "--prefer-online", "--cache", npmCacheDirectory],
|
||||||
{
|
{
|
||||||
cwd: directory,
|
cwd: directory,
|
||||||
@@ -437,8 +456,7 @@ async function inspectPackedPackage(packageSpec: string): Promise<{
|
|||||||
|
|
||||||
try {
|
try {
|
||||||
const resolvedPackageSpec = await resolvePackageSpec(packageSpec, directory, npmCacheDirectory)
|
const resolvedPackageSpec = await resolvePackageSpec(packageSpec, directory, npmCacheDirectory)
|
||||||
const { stdout } = await execFileAsync(
|
const { stdout } = await execNpmFileAsync(
|
||||||
"npm",
|
|
||||||
["pack", resolvedPackageSpec, "--json", "--prefer-online", "--cache", npmCacheDirectory],
|
["pack", resolvedPackageSpec, "--json", "--prefer-online", "--cache", npmCacheDirectory],
|
||||||
{
|
{
|
||||||
cwd: directory,
|
cwd: directory,
|
||||||
|
|||||||
@@ -6,6 +6,8 @@
|
|||||||
- Add `Qwen/Qwen3.8-Flash` and `z-ai/glm-5.3-flash` with their verified reasoning efforts (`low, medium, xhigh` and `low, high, max`) and display pricing.
|
- Add `Qwen/Qwen3.8-Flash` and `z-ai/glm-5.3-flash` with their verified reasoning efforts (`low, medium, xhigh` and `low, high, max`) and display pricing.
|
||||||
- Refresh static model capabilities from `command-code@1.36.0`, adding the free `minimax/minimax-m3-free` model and the `z-ai/glm-5.3-flash` output limit while dropping the retired `stealth/ox-alpha`.
|
- Refresh static model capabilities from `command-code@1.36.0`, adding the free `minimax/minimax-m3-free` model and the `z-ai/glm-5.3-flash` output limit while dropping the retired `stealth/ox-alpha`.
|
||||||
- Refresh display pricing for the current 62-model catalog, adding the free `minimax/minimax-m3-free` and `minimax/minimax-m2.7-free` promotional variants (free through September 5, 2026) and `tencent/hy4-preview`, and removing the retired `stealth/ox-alpha`.
|
- Refresh display pricing for the current 62-model catalog, adding the free `minimax/minimax-m3-free` and `minimax/minimax-m2.7-free` promotional variants (free through September 5, 2026) and `tencent/hy4-preview`, and removing the retired `stealth/ox-alpha`.
|
||||||
|
- Fix `npm run sync:commandcode-catalog` and `npm run check:commandcode-catalog` on Windows by spawning npm through the shell.
|
||||||
|
- Add a `refresh-model-catalog` agent skill with cross-platform helper scripts that snapshot the live model catalog and regenerate the pricing fixture from `MODEL_COSTS`.
|
||||||
|
|
||||||
## 0.6.0 - 2026-08-25
|
## 0.6.0 - 2026-08-25
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -9,5 +9,5 @@
|
|||||||
"strict": true,
|
"strict": true,
|
||||||
"types": ["node"]
|
"types": ["node"]
|
||||||
},
|
},
|
||||||
"include": [".github/scripts/**/*.ts", "src/**/*.ts", "tests/**/*.ts"]
|
"include": [".github/scripts/**/*.ts", ".agents/skills/**/*.ts", "src/**/*.ts", "tests/**/*.ts"]
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user