Skip to content

chore(golden-al): update Kimi K3.1 Pro DSpark probabilistic-block curve for draft 6a2ef20a - #4851

Merged
functionstackx merged 2 commits into
mainfrom
srok/kimik3.1-dspark-6a2ef20a-golden-al
Nov 6, 2026
Merged

functionstackx merged 2 commits into
mainfrom
srok/kimik3.1-dspark-6a2ef20a-golden-al

Conversation

@seungrokj

@seungrokj seungrokj commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Updates the Kimi K3.1 Pro DSpark golden AL curve (kimi-k3.1-pro__dspark__kimi-k3.1-pro-dspark-preview__probabilistic-block.yaml). The new values come from SpeedBench AL run 42120504891, measured after the Inferact/Kimi-K3.1-Pro-DSpark-preview weights update (HF revision 6a2ef20a, "weights update", 2026-11-05 22:12 UTC).

Changes:

  • source_run now points to run 42120504891.
  • The header comment records the draft revision (@ 6a2ef20a).
  • The values under thinking_on are replaced with that run's artifact speedbench-reference-al-kimik3.1, exactly as published.

The collection block and identities.yaml stay the same. The draft repo name, sampling, and verification settings did not change.

num_speculative_tokens 1 2 3 4 5 6 7 8
previous (run 41871192206) 1.79 2.33 2.70 2.92 3.08 3.14 3.21 3.33
new (run 42120504891, draft 6a2ef20a) 1.82 2.44 2.88 3.19 3.41 3.57 3.69 3.68

Only the draft weights changed between the two runs. Both runs used:

  • the same image: vllm/vllm-openai:kimi-k31@sha256:904520f3…a99, vLLM 0.1.dev24216+g266e88c05
  • the same collector: kimik3.1_fp4_b300_vllm_probabilistic_sample_method_block_rejection_sample_method.sh on bryan/kimik3.1-speedbench-al
  • the same settings: B300, coding category, output_len 4096, thinking on ({"thinking": true}, temperature 1.0, top_p 0.95), concurrency 64

The draft was downloaded at 22:21 UTC, after the 6a2ef20a push at 22:12 UTC.

Checks against the README reviewer list:

  • All 8 draft lengths completed with 80/80 successful requests and 0 failed in each cell.
  • The job log has no tracebacks and no draft-disable or chat-template errors.
  • infx.golden_al_distribution.curves.load_curve() loads the file with the new source_run, values, and collection block.

Note: nst 8 (3.68) is 0.01 below nst 7 (3.69). The curve looks saturated around nst 7, and this is kept as measured.

Any AgentX point that sets synthetic acceptance from this curve will use the new values once this merges. For example, nst 3 goes from 2.70 to 2.88 and nst 7 from 3.21 to 3.69. The PR #4806 changelog entry quotes the old per-concurrency values, so it would need an update if it lands after this.

🤖 Generated with Claude Code

…ve for draft 6a2ef20a

Re-measured by SpeedBench AL run 42120504891 on B300 after the
Inferact/Kimi-K3.1-Pro-DSpark-preview weights update (HF revision
6a2ef20a). Same image, collector, and collection settings as run
41871192206; only the draft weights changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

inferencex-e2e/infx diff: +10 -10

Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package.

中文

大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。

Comment on lines +12 to +19
1: 1.82
2: 2.44
3: 2.88
4: 3.19
5: 3.41
6: 3.57
7: 3.69
8: 3.68

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 After merge, the published Kimi K3.1 vLLM DSpark AgentX results keep their old synthetic AL. Nothing re-collects them at the new values, so they no longer match the committed curve. The new values feed synthetic_acceptance_length for kimik3.1-fp4-b300-vllm-agentic-dspark (nst 5/4/3/2) and kimik3.1-fp4-b200-dynamo-vllm-agentic-dspark (nst 7/4). At nst 7 that is 3.21 to 3.69. The PR adds no perf-changelog.yaml entry. Fix: append one changelog block listing both config keys so a sweep collects results at the new curve, as AGENTS.md:87 requires.

Why this was flagged

For AgentX throughput points, synthetic_acceptance.py:200 calls golden_length(...) for vLLM DSpark roles. build_overrides then writes rejection_sample_method=synthetic, synthetic_acceptance_length=<golden> into the served speculative-config. Changing these YAML values therefore changes the speculative-config vLLM is launched with on every point of nvidia-master.yaml:5894. It does the same on nvidia-master.yaml:4812. AGENTS.md:87 says a change that alters what any benchmark runs needs a changelog entry, and that entry is what triggers result collection. infx.matrix.changed only fingerprints configs and recipes, so it cannot flag a golden-curve edit. The diff appends nothing to inferencex-e2e/perf-changelog.yaml, so no sweep or /use ingest happens. The dashboard keeps throughput measured at the old AL, while the curve that future runs inject says something else. On the base branch the curve and the published results agree.

Verification: The trigger is any AgentX throughput point of the two Kimi K3.1 vLLM DSpark configs once this merges. infx/srt_slurm/synthetic_acceptance.py:200 calls golden_length(...), and lines 300-303 set synthetic_acceptance_length=worker_al. The PR adds no changelog entry to re-collect those points, which AGENTS.md:87 requires.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants