Skip to content

[VideoGenX] include video configs in local change detection and rendering / [VideoGenX] 本地变更检测与渲染纳入视频配置 - #4886

Merged
edwingao28 merged 5 commits into
mainfrom
fix/video-changed-detection
Nov 7, 2026
Merged

edwingao28 merged 5 commits into
mainfrom
fix/video-changed-detection

Conversation

@edwingao28

@edwingao28 edwingao28 commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

python -m infx.matrix.changed and infx generate --config-key see no video points. #2916 made video-generation opt-in in generate_config_matrix, so a key's video scenarios expand only when scenario_types names them. changed.head_matrix, changed.base_matrix and generate_recipes call the generator with no scenario selection, so every *-video key has zero points on both sides of the comparison and nothing to render.

One instance, measured on origin/main at 04118aa: after an engine-arg edit to h3/sglang/h100-bf16/serving-client-tp2.yaml, changed --base origin/main --json prints changed: [] across 184 keys, while test-config --scenario-type video-generation moves the g2 fingerprint from f152db5d9170 to c1f805258915. infx generate --config-key h3-bf16-h100-sglang-video exits 2 with has no srt-slurm points. The PR template checklist and the review bot rely on changed, so a video recipe change looks like it needs no perf-changelog.yaml entry and no video sweep runs.

Changes:

  • infx.matrix.generate.SCENARIO_TYPES is the one list of scenario types a changelog entry can select. The --scenario-type choices and plan.py read it from there; plan.DEFAULT_SCENARIO_TYPES is unchanged.
  • changed.config_points generates every scenario type of one key, and head_matrix uses it, so the working tree's video points carry recipe fingerprints.
  • changed.base_matrix keeps the one unscoped test-config request per key for the base revision's LLM scenarios and adds one --scenario-type video-generation request for each key that declares a video scenario, through _base_video_points. The base generator exits when a video recipe cannot be bound, so that request runs alone and its failure lands in the key's errors, where _points records a head failure, instead of aborting the report.
  • generate_recipes renders every scenario of the named key, so infx generate --config-key <video key> writes the video server recipes.
  • signoff_facts.matrix_at is unchanged. The CODEOWNER sign-off generates LLM points only, from config data and trusted code, so it never loads a revision's video-generation/specs/video_spec.py.
  • Unscoped generation is unchanged: generate_config_matrix without scenario_types, full-sweep and test-config without --scenario-type still leave video out.

Results:

Local runs at eb50d8b against origin/main 90bc842, no GPU and no sweep; every row is a local run with no link.

Check origin/main This branch
changed --base origin/main --json after editing the H100 serving-client-tp2.yaml engine args changed: [], 184 keys h3-bf16-h100-sglang-video: 3 -> 3 points, 1 added, 1 removed (g2 f152db5d9170 -> c1f805258915), 184 keys, 0 errors, 6.2 s wall
changed --base origin/main --json on a clean tree changed: [], 184 keys changed: [], 184 keys, 0 errors, 6.2 s wall
infx generate --config-key h3-bf16-h100-sglang-video exit 2, has no srt-slurm points exit 0, 3 recipes (2, 4 and 8 GPUs) and manifest.json
infx.matrix.revision generate <checkout> full-sweep --config-files configs/nvidia-master.yaml configs/amd-master.yaml, with and without --no-evals 1,320,052 B and 1,265,649 B byte-identical by cmp
test-config --config-keys h3-bf16-h100-sglang-video --no-evals without --scenario-type [] []
signoff_facts.matrix_at("HEAD", workdir) on the real configs same file as this branch 184 keys, 0 errors, 1,741 points, 4 video keys with 0 points

Testing:

  • infx/tests/matrix/test_changed.py::test_keys_with_changed_recipes_or_points_are_listed builds a nested inferencex-e2e/ plus video-generation/specs repository with a video key whose recipe changes between base and head and an unbound key whose base recipe lacks VIDEO_RUNTIME_REVISION. It expects video listed with one added and one removed point, unbound listed with one base error, and the text report's warning: line for that error. With origin/main sources it fails at the base-error assertion (not enough values to unpack); without _base_video_points it fails with the base generator's CalledProcessError; with this branch it passes.
  • infx/tests/srt_slurm/test_generate.py::test_a_video_point_renders_its_bound_server_recipe renders a video key through infx generate and checks the bound GPU count, spec path and CONC_LIST. With origin/main sources it fails with infx: error: video has no srt-slurm points (exit 2); with this branch it passes.
  • infx/tests/workflows/test_signoff_facts.py::test_matrix_at_generates_llm_points_only_and_runs_no_revision_code commits a repository whose video-generation/specs/video_spec.py raises on import and calls matrix_at("HEAD", workdir). It expects the LLM key's point, no video points and no errors. It passes on origin/main and on this branch; a matrix_at that generated video points from the revision snapshot returned errors == {"video": "video_spec.py from the revision tree ran"}.
  • The video tests use infx.tests.srt_recipes.write_video_specs, which copies the trusted video_spec.py and writes a one-case schema 1 fixture spec with its own model id and set, so an edit to h3-v2.yaml cannot fail them.
  • Full infx/tests/ suite with -n 4 on macOS: 3,243 passed, 2 skipped, 3 failed. The 3 failures are in infx/tests/matrix/test_revision.py, where the fork server's os.killpg raises PermissionError. They fail the same way on a clean origin/main checkout, and this branch does not touch revision.py or its tests.
  • ruff check and ruff format --check pass.

Limits:

  • A base whose generator rejects --scenario-type video-generation records that error under each of its video keys; its LLM comparison still completes.
  • A base video bind failure yields no base points for that key, so the key is listed as changed with its head points added, while head records the same failure as a point without a fingerprint. The report completes either way.
  • Sign-off check 18(g) still does not cover video points: matrix_at generates LLM scenarios only, as on origin/main.
  • Checked by local tests and local command runs only; no sweep ran, as the change executes no benchmark.

Related: Follows #2916. Pairs with #4885, whose submission guide links the infx generate render step this PR fixes for video.

PR checklist and change type

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have completed the AI model disclosure and kept it current
  • I have tested my changes locally
  • I have updated documentation if necessary
  • If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
  • If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
  • Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

inferencex-e2e/infx diff: +258 -62

Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package.

中文

大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread inferencex-e2e/infx/matrix/changed.py Resolved
@edwingao28
edwingao28 merged commit 518430e into main Nov 7, 2026
4 checks passed
@edwingao28
edwingao28 deleted the fix/video-changed-detection branch November 7, 2026 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant