Skip to content

[skills] docs: codify recipe tuning workflow - #5796

Merged
yaoyu-33 merged 2 commits into
NVIDIA-NeMo:mainfrom
yaoyu-33:codex/recipe-tuning-rule-of-thumb
Aug 25, 2026
Merged

yaoyu-33 merged 2 commits into
NVIDIA-NeMo:mainfrom
yaoyu-33:codex/recipe-tuning-rule-of-thumb

Conversation

@yaoyu-33

Copy link
Copy Markdown
Contributor

Summary

  • codify a fit-first, then fill recipe-tuning loop: keep dense TP as low as capacity allows, use the largest legal EP candidate, and prefer architecture-specific selective recompute
  • document Qwen3.5 GDN handling through gdn_norm_out, with full-layer recompute retained as a measured capacity fallback
  • require post-fit W&B plus per-rank memory review before A/B testing lower TP or higher MBS at fixed GBS
  • keep alltoall as the correctness baseline and treat HybridEP as a matched candidate on NVL8 BF16

The EP8-on-8-GPU and EP32-on-32-GPU guidance is explicitly scoped to legal expert counts and parallel meshes. EP is documented as sharding expert weights only, not dense or activation state.

Validation

  • five updated skills pass the standard skill validator
  • modified JSON/YAML artifacts parse successfully
  • uv run --no-project --active pre-commit run --all-files
  • git diff --check origin/main...HEAD

No training recipe or product code changed, and no performance experiment was rerun. Existing measured dates remain unchanged; cards record a separate 2026-08-25 guidance refresh.

Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33 yaoyu-33 added area:perf Performance optimizations and benchmarking docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. needs-review PR is ready for code review and waiting on a reviewer labels Aug 25, 2026
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test 376a9cb

Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test 32019a2

@yaoyu-33
yaoyu-33 merged commit 6321f35 into NVIDIA-NeMo:main Aug 25, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:perf Performance optimizations and benchmarking docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant