Skip to content

[fix] reject video changelog entries without scenario-type / [fix] 拒绝缺少 scenario-type 的视频 changelog 条目 - #4848

Merged
edwingao28 merged 2 commits into
mainfrom
fix/video-changelog-scenario-type
Nov 5, 2026
Merged

edwingao28 merged 2 commits into
mainfrom
fix/video-changelog-scenario-type

Conversation

@edwingao28

@edwingao28 edwingao28 commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Video sweeps are opt-in: a perf-changelog.yaml entry plans video jobs only when its scenario-type lists video-generation. An entry that named a *-video config but left video out, either by omitting scenario-type or by listing only LLM scenarios, still validated and planned an empty sweep, so a contributor would get a green run that measured nothing. build_plan now rejects an entry when a config it names explicitly has video-generation and shares no scenario with the entry's selection. The message names those configs and, when the entry also covers other configs, says to keep fixed-seq-len and agentic-coding so their jobs are not dropped. Wildcard matches such as "*" keep skipping video as before, and every non-video entry plans exactly as on main. The video contribution steps in inferencex-e2e/docs/video-generation.md state the requirement.

Testing: the new test_video_matrix.py case covers an omitted and an LLM-only scenario-type (both fail on main with no error and an empty plan) and a "*" entry that must still plan. The rest of infx/tests/matrix/ passes apart from the three test_revision.py batch-generation cases that fail in this sandbox on every branch. With real configs, #2916's entry still yields its two H100/B200 video jobs and fails with the new message once scenario-type is removed; a "*" entry's plan (1055 single-node rows, no video) and 81 non-video changelog plans are byte-identical to main. Planner and docs only: infx.matrix.changed lists no config keys, so there is no changelog entry and no sweep.

PR checklist and change type

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have completed the AI model disclosure and kept it current
  • I have tested my changes locally
  • I have updated documentation if necessary
  • If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
  • If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
  • Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

inferencex-e2e/infx diff: +34 -0

Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package.

中文

大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline findings, I also checked whether the new master_config[c]["scenarios"] lookup in plan.py can raise KeyError. It can't: the head master config is loaded with validation on (plan.py:354), and scenarios is a required field in both the single-node and multi-node schemas (validation.py:1066 and 1106).

Extended reasoning...

The change adds a scenario-type guard to build_plan in inferencex-e2e/infx/matrix/plan.py, along with a test and a docs note. The direct scenarios lookup was ruled out as a KeyError risk because build_plan validates the head config before it reaches the guard, and the schema requires that field.

Comment thread inferencex-e2e/infx/matrix/plan.py Resolved
Comment thread inferencex-e2e/infx/matrix/plan.py Resolved
@edwingao28
edwingao28 merged commit 8d5a463 into main Nov 5, 2026
4 checks passed
@edwingao28
edwingao28 deleted the fix/video-changelog-scenario-type branch November 5, 2026 23:13
@edwingao28 edwingao28 changed the title [VideoGenX] reject video changelog entries without scenario-type / [VideoGenX] 拒绝缺少 scenario-type 的视频 changelog 条目 [fix] reject video changelog entries without scenario-type / [VideoGenX] 拒绝缺少 scenario-type 的视频 changelog 条目 Nov 5, 2026
@edwingao28 edwingao28 changed the title [fix] reject video changelog entries without scenario-type / [VideoGenX] 拒绝缺少 scenario-type 的视频 changelog 条目 [fix] reject video changelog entries without scenario-type / [fix] 拒绝缺少 scenario-type 的视频 changelog 条目 Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant