Repository navigation
chore(golden-al): update Kimi K3.1 Pro DSpark probabilistic-block curve for draft 6a2ef20a - #4851
Conversation
…ve for draft 6a2ef20a Re-measured by SpeedBench AL run 42120504891 on B300 after the Inferact/Kimi-K3.1-Pro-DSpark-preview weights update (HF revision 6a2ef20a). Same image, collector, and collection settings as run 41871192206; only the draft weights changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
| 1: 1.82 | ||
| 2: 2.44 | ||
| 3: 2.88 | ||
| 4: 3.19 | ||
| 5: 3.41 | ||
| 6: 3.57 | ||
| 7: 3.69 | ||
| 8: 3.68 |
There was a problem hiding this comment.
🔴 After merge, the published Kimi K3.1 vLLM DSpark AgentX results keep their old synthetic AL. Nothing re-collects them at the new values, so they no longer match the committed curve. The new values feed synthetic_acceptance_length for kimik3.1-fp4-b300-vllm-agentic-dspark (nst 5/4/3/2) and kimik3.1-fp4-b200-dynamo-vllm-agentic-dspark (nst 7/4). At nst 7 that is 3.21 to 3.69. The PR adds no perf-changelog.yaml entry. Fix: append one changelog block listing both config keys so a sweep collects results at the new curve, as AGENTS.md:87 requires.
Why this was flagged
For AgentX throughput points, synthetic_acceptance.py:200 calls golden_length(...) for vLLM DSpark roles. build_overrides then writes rejection_sample_method=synthetic, synthetic_acceptance_length=<golden> into the served speculative-config. Changing these YAML values therefore changes the speculative-config vLLM is launched with on every point of nvidia-master.yaml:5894. It does the same on nvidia-master.yaml:4812. AGENTS.md:87 says a change that alters what any benchmark runs needs a changelog entry, and that entry is what triggers result collection. infx.matrix.changed only fingerprints configs and recipes, so it cannot flag a golden-curve edit. The diff appends nothing to inferencex-e2e/perf-changelog.yaml, so no sweep or /use ingest happens. The dashboard keeps throughput measured at the old AL, while the curve that future runs inject says something else. On the base branch the curve and the published results agree.
Verification: The trigger is any AgentX throughput point of the two Kimi K3.1 vLLM DSpark configs once this merges. infx/srt_slurm/synthetic_acceptance.py:200 calls golden_length(...), and lines 300-303 set synthetic_acceptance_length=worker_al. The PR adds no changelog entry to re-collect those points, which AGENTS.md:87 requires.
…en-AL revision) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Updates the Kimi K3.1 Pro DSpark golden AL curve (
kimi-k3.1-pro__dspark__kimi-k3.1-pro-dspark-preview__probabilistic-block.yaml). The new values come from SpeedBench AL run 42120504891, measured after theInferact/Kimi-K3.1-Pro-DSpark-previewweights update (HF revision6a2ef20a, "weights update", 2026-11-05 22:12 UTC).Changes:
source_runnow points to run 42120504891.@ 6a2ef20a).thinking_onare replaced with that run's artifactspeedbench-reference-al-kimik3.1, exactly as published.The
collectionblock andidentities.yamlstay the same. The draft repo name, sampling, and verification settings did not change.Only the draft weights changed between the two runs. Both runs used:
vllm/vllm-openai:kimi-k31@sha256:904520f3…a99, vLLM0.1.dev24216+g266e88c05kimik3.1_fp4_b300_vllm_probabilistic_sample_method_block_rejection_sample_method.shonbryan/kimik3.1-speedbench-al{"thinking": true}, temperature 1.0, top_p 0.95), concurrency 64The draft was downloaded at 22:21 UTC, after the 6a2ef20a push at 22:12 UTC.
Checks against the README reviewer list:
infx.golden_al_distribution.curves.load_curve()loads the file with the newsource_run, values, andcollectionblock.Note: nst 8 (3.68) is 0.01 below nst 7 (3.69). The curve looks saturated around nst 7, and this is kept as measured.
Any AgentX point that sets synthetic acceptance from this curve will use the new values once this merges. For example, nst 3 goes from 2.70 to 2.88 and nst 7 from 3.21 to 3.69. The PR #4806 changelog entry quotes the old per-concurrency values, so it would need an update if it lands after this.
🤖 Generated with Claude Code