[cquil11]

Re-runs kimik3.1-fp4-b300-vllm-agentic-dspark (#4796) now that #4840 removes the Kimi tool and reasoning parsers from vLLM throughput servers that use synthetic acceptance.

#4796's sweep ran with the parsers on, which caused two problems:

• 1.9–2.6% of responses were empty at 5 draft tokens.
• Many other responses were cut short. As a result, its ITL was measured at 0.74–0.83 of the full-response ITL, so its reported ITL and interactivity are 17–26% too optimistic.

This sweep replaces those results.

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

🔴 The changelog check will fail this PR, and the re-run results would be credited to the old PR. The new block's pr-link at line 12109 points to pull/4796. That is the merged PR that first added this config at line 12036, not this PR. validate_added_pr_link accepts only the PR's own URL or an XXX placeholder for a new entry. Fix: set the pr-link to this PR's own URL, or to an XXX placeholder until the URL is known. Keep the mention of #4796 in the description if wanted.

Why this was flagged

The diff appends a new entry whose pr-link is #4796 (perf-changelog.yaml:12109). The same URL already belongs to the original entry for kimik3.1-fp4-b300-vllm-agentic-dspark at perf-changelog.yaml:12036, so this PR is a different, later one. In inferencex-e2e/infx/workflows/validate_perf_changelog.py:112-116, every added entry on a PR run must use https://github.com/SemiAnalysisAI/InferenceX/pull/{pr_number} or a value from PR_LINK_PLACEHOLDERS. Anything else raises ChangelogValidationError with "new PR entry must use ... or an XXX placeholder". The changelog validation for this PR therefore fails. If the entry got through some other way, the re-collected results and metadata would point at #4796, and recover_failed_ingest.py compares pr-link against the expected link, so the new sweep would be mixed up with the superseded one. On the base branch no such mismatched entry exists.

Verification: The trigger is that the PR is merged through the supported path, merge_with_reuse (CONTRIBUTING.md:174). The new entry carries an older PR's link, pull/4796 (perf-changelog.yaml:12109). merge_with_reuse._merge_pr_inner always calls canonicalize_changelog(pr, repo) (merge_with_reuse.py:688), and compare_entries raises. The run-sweep PR check does not reject this.

[claude]

🔴 The PR's re-run of this config never starts, even with the non-canary-full-sweep-enabled label set. The only commit, 33bcb8b, carries [skip-sweep] in its message. run-sweep.yml:87 reads that marker from the head commit and skips the PR benchmark sweep on every event. So no run exists for the required /use <run_id>, and the stale #4796 results stay in place. Fix: push the changelog entry, or a follow-up head commit, without [skip-sweep] so the labelled sweep actually runs. Amending the commit with the same message keeps the sweep suppressed.

Why this was flagged

The PR's head commit is 33bcb8b, "chore(changelog): re-run Kimi-K3.1-Pro vLLM B300 with synthetic parsers off [skip-sweep]". In .github/workflows/run-sweep.yml:86-91, git log -1 --format=%B "$HEAD_SHA" | grep -Fq '[skip-sweep]' sets skip-pr-sweep=true. The sweep job at run-sweep.yml:178 requires needs.check-changelog.outputs.skip-pr-sweep != 'true' before it checks the full-sweep labels at :180-182. That check runs on labeled events too, so the non-canary-full-sweep-enabled label does not start a sweep while this commit is the head. .github/workflows/README.md:268 says [skip-sweep] skips PR benchmark setup only. The entry at perf-changelog.yaml:12103-12109 exists to replace the #4796 results, and per CONTRIBUTING it needs an accepted /use <run_id> before merge. This PR produces no run to accept, so the parsers-off re-run described in the PR body never happens unless someone pushes a new head commit without the marker. Neither the label nor the changelog check prevents this.

Verification: The PR head stays at 33bcb8b, the only commit, whose subject contains the [skip-sweep] marker. .github/workflows/run-sweep.yml:84-91 sets skip-pr-sweep=true, and the setup job requires needs.check-changelog.outputs.skip-pr-sweep != 'true' (run-sweep.yml:~176), including on labeled events. The re-run never starts on this PR.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=42103738354

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=42103738354