Skip to content

Commit b46164e

Browse files
lidge-junkaicotclaude
authored
fix(catalog): widen the dated-variant fold and guard its direction at the merge loop (#3100)
* fix(catalog): recognize YYMMDD/MMDD/YYMM dated model id suffixes (#3024) `isDatedVariantId` only accepted an 8-digit `YYYYMMDD` suffix, so the dated-alias fold in `mergeConfiguredModelsIntoLiveCatalog` never fired for providers that publish shorter release dates. On a real multi-provider install the `\d{8}` rule matched none of the 26 numeric-suffixed ids present, dropping DeepSeek, Kimi, Mistral, Qwen and Solar aliases into `droppedConfiguredIds` even though a live row for the same model existed. Widen the suffix to the formats upstreams actually publish -- `YYYYMMDD`, `YYMMDD`, `MMDD` and `YYMM` -- behind a calendar guard so ordinary numeric suffixes are not read as dates. `-2048`, `-4096` and `-8192` are rejected; `YY` is `2\d` rather than `\d\d` so `1301` is rejected too. `-1024` is a valid `MMDD` and is therefore accepted -- an irreducible collision, pinned by a test so it stays a known cost. The fold stays one-directional (`configured=base` -> `live=dated`). A configured id the provider no longer lists must not be retained on the strength of a format match alone; #1690 is the explicit opt-in for that. Hyphenated ISO suffixes (`-2024-08-06`, `-05-06`) remain out of scope. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit 0a6393d) (cherry picked from commit 6dd0850) * test(catalog): guard the dated fold's direction at the merge loop, not the predicate #3034 widens the dated-suffix matcher and pins it with a predicate test that the fold stays one-way: isDatedVariantId("deepseek-v4-pro", "deepseek-v4-pro-0813") is false. That assertion is true of every implementation, including one whose merge loop calls the predicate a second time with the arguments swapped -- which is exactly what #3041 originally proposed and then withdrew. So the guard is moved to where the resurrection would actually happen. These three drive mergeConfiguredModelsIntoLiveCatalog itself, carried from #3041: - a live base row must not resurrect a configured dated id - a live MMDD dated row still folds onto its configured base - a dated id named in retainConfiguredModelIds survives Both directions were mutation-checked. Adding || isDatedVariantId(candidate.id, live.id) to the merge loop fails only the first test (253 pass / 1 fail); narrowing the suffix back to /^\d{8}$/ fails 13, including the MMDD and YYMM folds. Neither mutation is caught by the predicate test alone. The retention test is labelled for what it actually covers: production fills retainConfiguredModelIds from combo targets, not from providers.*.models, so it pins the OCX-111 path. The operator-facing opt-in is #1690's retainModels, which does not exist yet -- and until it does, the dated id #3024 reports is still dropped. This lands the safe half of #3024 and says so. (cherry picked from commit a909682) --------- Co-authored-by: kaicot <275240300+kaicot@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 330470e commit b46164e

2 files changed

Lines changed: 161 additions & 3 deletions

File tree

src/codex/catalog/provider-fetch.ts

Lines changed: 62 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -936,9 +936,70 @@ export function resolveComboCatalogMember(
936936
};
937937
}
938938

939+
const DATED_VARIANT_YYYYMMDD = /^(\d{4})(\d{2})(\d{2})$/;
940+
const DATED_VARIANT_YYMMDD = /^(2\d)(\d{2})(\d{2})$/;
941+
const DATED_VARIANT_MMDD_OR_YYMM = /^(\d{2})(\d{2})$/;
942+
943+
/** Whether a Gregorian year contains February 29th. */
944+
function isLeapYear(year: number): boolean {
945+
return year % 4 === 0 && (year % 100 !== 0 || year % 400 === 0);
946+
}
947+
948+
/**
949+
* Whether a month/day pair exists in the given year. Without a year, February 29th is
950+
* accepted because it occurs in at least one calendar year.
951+
*/
952+
function isValidCalendarDate(year: number | undefined, month: number, day: number): boolean {
953+
if (year !== undefined && (year < 1 || year > 9999)) return false;
954+
if (month < 1 || month > 12 || day < 1) return false;
955+
const daysInMonth = [
956+
31, year === undefined || isLeapYear(year) ? 29 : 28, 31, 30, 31, 30,
957+
31, 31, 30, 31, 30, 31,
958+
];
959+
return day <= daysInMonth[month - 1]!;
960+
}
961+
962+
/**
963+
* Release-date suffixes providers actually publish: `YYYYMMDD` (`-20251001`), `YYMMDD`
964+
* (`-260806`), `MMDD` (`-0813`) and `YYMM` (`-2512`). A `\d{8}`-only rule matched none of
965+
* the dated ids on a real multi-provider install, so DeepSeek, Kimi, Mistral, Qwen and
966+
* Solar aliases all fell through to `droppedConfiguredIds` (#3024).
967+
*
968+
* Calendar validation rejects impossible month-end and leap-day values as well as ordinary
969+
* numeric suffixes such as `-2048`, `-4096` and `-8192`. `-1024` is the one irreducible
970+
* collision — it is a valid `MMDD` (October 24th) — so it reads as dated. That is a known,
971+
* accepted cost; the test table pins it so it cannot become a surprise later.
972+
*
973+
* Hyphenated ISO suffixes (`-2024-08-06`, `-05-06`) are deliberately out of scope: a
974+
* hyphenated suffix is ambiguous against ordinary name segments and needs its own call.
975+
*/
976+
function isDatedVariantSuffix(suffix: string): boolean {
977+
const yyyyMmDd = DATED_VARIANT_YYYYMMDD.exec(suffix);
978+
if (yyyyMmDd) {
979+
return isValidCalendarDate(
980+
Number(yyyyMmDd[1]), Number(yyyyMmDd[2]), Number(yyyyMmDd[3]),
981+
);
982+
}
983+
984+
const yyMmDd = DATED_VARIANT_YYMMDD.exec(suffix);
985+
if (yyMmDd) {
986+
return isValidCalendarDate(
987+
2000 + Number(yyMmDd[1]), Number(yyMmDd[2]), Number(yyMmDd[3]),
988+
);
989+
}
990+
991+
const mmDdOrYyMm = DATED_VARIANT_MMDD_OR_YYMM.exec(suffix);
992+
if (!mmDdOrYyMm) return false;
993+
const first = Number(mmDdOrYyMm[1]);
994+
const second = Number(mmDdOrYyMm[2]);
995+
return isValidCalendarDate(undefined, first, second)
996+
|| (first >= 20 && first <= 29 && second >= 1 && second <= 12);
997+
}
998+
999+
/** Whether `liveId` is a supported dated release of the configured base id. */
9391000
export function isDatedVariantId(liveId: string, configuredId: string): boolean {
9401001
if (!liveId.startsWith(`${configuredId}-`)) return false;
941-
return /^\d{8}$/.test(liveId.slice(configuredId.length + 1));
1002+
return isDatedVariantSuffix(liveId.slice(configuredId.length + 1));
9421003
}
9431004

9441005
export const lastDropWarnSignature = new Map<string, string>();

tests/codex-catalog.test.ts

Lines changed: 99 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
33
import { tmpdir } from "node:os";
44
import { join } from "node:path";
55
import { applyNativeVisibility, augmentRoutedModelsWithMetadata, augmentRoutedModelsWithRegistryOpenAiApiRows, buildCatalogEntries, buildComboCatalogOmission, catalogModelSlug, clampCatalogModelsToCodexSupport, clampEntryToCodexSupportedEfforts, clampedDefaultEffort, CODEX_ACCOUNT_BOUND_CATALOG_KIND, CODEX_NATIVE_ALIAS_CATALOG_KIND, comboCatalogOmissionReason, deriveComboCatalogModel, exactComboCatalogSlugs, filterCatalogVisibleModels, filterSupportedNativeSlugs, gatherRoutedModels as gatherRoutedModelsDirect, isDatedVariantId, isMediaGenerationModelId, loadBundledCodexCatalog, materializeBundledCodexCatalog, mergeCatalogEntriesForSync, NATIVE_DAYBREAK_BLUE_MODEL, NATIVE_OPENAI_MODELS, nativeDefaultReasoningEffort, nativeInputModalities, nativeOpenAiCapabilitySourceSlug, nativeOpenAiContextWindow, nativeReasoningEfforts, normalizeRoutedCatalogEntry, resetCatalogRuntimeStateForTests, resetOpenAiApiCatalogWarningStateForTests, resolveComboCatalogMember, shouldExposeRoutedModel, upstreamNativeEntry } from "../src/codex/catalog";
6-
import { applyProviderConfigHints } from "../src/codex/catalog/provider-fetch";
6+
import { applyProviderConfigHints, mergeConfiguredModelsIntoLiveCatalog } from "../src/codex/catalog/provider-fetch";
77
import {
88
CODEX_CUSTOM_MODEL_CATALOG_KIND,
99
CODEX_PROVIDER_MODEL_CATALOG_KIND,
@@ -3651,14 +3651,111 @@ describe("Codex catalog routed normalization", () => {
36513651
}
36523652
});
36533653

3654-
test("isDatedVariantId matches only <alias>-YYYYMMDD", () => {
3654+
test("isDatedVariantId matches only <alias>-<date>", () => {
36553655
expect(isDatedVariantId("claude-haiku-4-5-20251001", "claude-haiku-4-5")).toBe(true);
36563656
expect(isDatedVariantId("claude-haiku-4-5-2025", "claude-haiku-4-5")).toBe(false);
36573657
expect(isDatedVariantId("claude-haiku-4-5-latest", "claude-haiku-4-5")).toBe(false);
36583658
expect(isDatedVariantId("claude-haiku-4-5", "claude-haiku-4-5")).toBe(false);
36593659
expect(isDatedVariantId("claude-haiku-4-5-20251001", "claude-haiku-4")).toBe(false);
36603660
});
36613661

3662+
// Real ids from a multi-provider install. A `\d{8}`-only rule matched none of them, so
3663+
// every one of these aliases dropped out of the authoritative live catalog (#3024).
3664+
test.each([
3665+
["YYYYMMDD", "claude-haiku-4-5-20251001", "claude-haiku-4-5"],
3666+
["YYYYMMDD leap day", "acme-model-20240229", "acme-model"],
3667+
["YYMMDD", "solar-pro4-260806", "solar-pro4"],
3668+
["YYMMDD", "syn-pro-251021", "syn-pro"],
3669+
["YYMMDD leap day", "acme-model-240229", "acme-model"],
3670+
["MMDD", "deepseek/deepseek-v4-pro-0813", "deepseek/deepseek-v4-pro"],
3671+
["MMDD", "moonshotai/kimi-k2-0905", "moonshotai/kimi-k2"],
3672+
["MMDD", "openai/gpt-3.5-turbo-0613", "openai/gpt-3.5-turbo"],
3673+
["MMDD leap day", "acme-model-0229", "acme-model"],
3674+
["YYMM", "mistralai/mistral-large-2407", "mistralai/mistral-large"],
3675+
["YYMM", "qwen/qwen3-235b-a22b-2507", "qwen/qwen3-235b-a22b"],
3676+
])("folds a %s dated variant: %s", (_format, liveId, configuredId) => {
3677+
expect(isDatedVariantId(liveId, configuredId)).toBe(true);
3678+
});
3679+
3680+
test.each([
3681+
["a version number", "mistralai/mistral-medium-3-5", "mistralai/mistral-medium-3"],
3682+
["a context size", "openai/gpt-3.5-turbo-16k", "openai/gpt-3.5-turbo"],
3683+
["a variant name", "qwen/qwen3-coder-30b-a3b-instruct", "qwen/qwen3-coder"],
3684+
["a batch lane of a dated id", "deepseek/deepseek-v4-pro-0813:batch", "deepseek/deepseek-v4-pro"],
3685+
["a bare year", "acme-model-2025", "acme-model"],
3686+
["an impossible YYMM", "acme-model-1301", "acme-model"],
3687+
["a non-leap YYYYMMDD", "acme-model-20250229", "acme-model"],
3688+
["a non-leap YYMMDD", "acme-model-250229", "acme-model"],
3689+
["an impossible YYYYMMDD month-end", "acme-model-20240431", "acme-model"],
3690+
["an impossible YYMMDD month-end", "acme-model-240431", "acme-model"],
3691+
["an impossible MMDD month-end", "acme-model-0431", "acme-model"],
3692+
// Hyphenated ISO is ambiguous against ordinary name segments; out of scope for now.
3693+
["a hyphenated ISO date", "openai/gpt-4o-2024-08-06", "openai/gpt-4o"],
3694+
["a hyphenated MM-DD", "google/gemini-2.5-pro-preview-05-06", "google/gemini-2.5-pro-preview"],
3695+
])("does not fold %s: %s", (_label, liveId, configuredId) => {
3696+
expect(isDatedVariantId(liveId, configuredId)).toBe(false);
3697+
});
3698+
3699+
test.each(["2048", "4096", "8192"])("a %s context suffix is not a date", suffix => {
3700+
expect(isDatedVariantId(`acme-model-${suffix}`, "acme-model")).toBe(false);
3701+
});
3702+
3703+
// Known, accepted cost of allowing MMDD: October 24th is a real date, so a `-1024`
3704+
// context suffix is indistinguishable from one. No month/day tightening can exclude it.
3705+
test("a -1024 suffix reads as MMDD and is accepted", () => {
3706+
expect(isDatedVariantId("acme-model-1024", "acme-model")).toBe(true);
3707+
});
3708+
3709+
// The fold stays one-directional: a configured id the provider no longer lists must not
3710+
// be retained on the strength of a format match alone (#1690 is the explicit opt-in for
3711+
// that). `deepseek-v4-pro-0813` configured against a live `deepseek-v4-pro` stays dropped.
3712+
test("does not fold configured=dated against live=base", () => {
3713+
expect(isDatedVariantId("deepseek-v4-pro", "deepseek-v4-pro-0813")).toBe(false);
3714+
});
3715+
3716+
// The predicate test above is necessary and not sufficient: it passes on any
3717+
// implementation, including one whose MERGE LOOP calls the predicate a second time with
3718+
// the arguments swapped. These three drive `mergeConfiguredModelsIntoLiveCatalog` itself,
3719+
// so they fail if the loop ever becomes bidirectional. Carried from #3041, where the
3720+
// reverse fold was proposed and then withdrawn — the guard outlives the proposal.
3721+
test("the merge loop does not infer a configured dated id from a live base id", () => {
3722+
const { models, droppedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({
3723+
name: "deepseek",
3724+
provider: {},
3725+
models: [{ id: "deepseek-v4-pro" } as never],
3726+
configured: [{ id: "deepseek-v4-pro-0813" } as never],
3727+
});
3728+
expect(droppedConfiguredIds).toEqual(["deepseek-v4-pro-0813"]);
3729+
expect(models.map(m => m.id)).not.toContain("deepseek-v4-pro-0813");
3730+
});
3731+
3732+
test("the merge loop folds a live MMDD dated row onto its configured base", () => {
3733+
const { models, droppedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({
3734+
name: "deepseek",
3735+
provider: {},
3736+
models: [{ id: "deepseek-v4-pro-0813" } as never],
3737+
configured: [{ id: "deepseek-v4-pro" } as never],
3738+
});
3739+
expect(droppedConfiguredIds).toEqual([]);
3740+
expect(models.map(m => m.id)).toContain("deepseek-v4-pro");
3741+
});
3742+
3743+
// Retention of a dated id is a decision someone made, not an inference from a name.
3744+
// Note what this set actually is: production fills `retainConfiguredModelIds` from combo
3745+
// targets, not from `providers.*.models`, so this pins the combo-target path (OCX-111).
3746+
// The operator-facing opt-in is #1690's `retainModels`, which does not exist yet.
3747+
test("a dated id named in retainConfiguredModelIds survives the drop", () => {
3748+
const { models, droppedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({
3749+
name: "deepseek",
3750+
provider: {},
3751+
models: [{ id: "deepseek-v4-pro" } as never],
3752+
configured: [{ id: "deepseek-v4-pro-0813" } as never],
3753+
retainConfiguredModelIds: new Set(["deepseek-v4-pro-0813"]),
3754+
});
3755+
expect(droppedConfiguredIds).toEqual([]);
3756+
expect(models.map(m => m.id)).toContain("deepseek-v4-pro-0813");
3757+
});
3758+
36623759
test("disabled providers are excluded from routed model gathering", async () => {
36633760
const models = await gatherRoutedModels({
36643761
port: 10100,

0 commit comments

Comments
 (0)