Skip to content

docs(skills): make activation recompute guidance evidence-based - #5570

Merged
cuichenx merged 3 commits into
NVIDIA-NeMo:mainfrom
cuichenx:chcui/sage/update-activation-recompute-guidance
Aug 14, 2026
Merged

cuichenx merged 3 commits into
NVIDIA-NeMo:mainfrom
cuichenx:chcui/sage/update-activation-recompute-guidance

Conversation

@cuichenx

Copy link
Copy Markdown
Contributor

Summary

  • replace universal core_attn guidance with an architecture-, backend-, and peak-aware decision table covering every supported recompute label
  • add matched H100 evidence for Moonlight 16B and Nemotron 3 Nano, with bounded historical Llama 3 70B context
  • update the stable guide, skill metadata, and activation eval cases while clearly marking prior NVSkills results as historical

Why

The previous guidance generalized a qualitative module ranking from an unmatched Llama experiment. It did not account for architecture-specific boundaries, fused-attention behavior, nested boundaries, runtime compatibility, or the distinction between moving an OOM later and making a workload viable.

Evidence

Model Evidence-based conclusion
Moonlight 16B moe_act reduced peak allocation by 1.85% for a 1.99% step-time cost; mla_up_proj reduced it by 1.96% for a 5.19% cost; explicit core_attn produced no material memory reduction under fused attention.
Nemotron 3 Nano Whole-moe was the smallest tested boundary that completed; adding layernorm recovered another 1.014 GB at a 3.07% step-time cost. Narrow moe_act saved 4.194 GB at the intermediate peak but still OOMed after advancing farther.
Llama 3 70B (historical) Whole-mlp recovered about 3.2 GB but cost about 16% throughput in the prior unmatched experiment; this is retained only as contextual evidence.

The cross-model conclusion is to start with the narrowest architecture-specific boundary responsible for the measured per-rank peak, then broaden only when matched measurements show that it is necessary. There is no universal recompute-module ordering.

Validation

  • uv run --no-sync pre-commit run --all-files
  • skill package validation with quick_validate.py
  • JSON and YAML parsing, Markdown link checks, privacy scan, and git diff --check
  • independent pre-publication technical review
  • all successful experimental rows reported finite losses with zero skipped or NaN iterations

Publication note

The existing skill.oms.sig belongs to the previous skill contents. This PR remains draft while a fresh NVSkills evaluation/signature refresh is requested; the historical signature must not be treated as validation of the amended package.

Signed-off-by: Chen Cui <chcui@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 13, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@cuichenx

Copy link
Copy Markdown
Contributor Author

/nvskills-ci

Signed-off-by: Chen Cui <chcui@nvidia.com>
@cuichenx
cuichenx requested a review from a team as a code owner August 14, 2026 05:12
@yaoyu-33 yaoyu-33 added area:misc Cross-cutting utilities, logging, helpers, and other changes docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. needs-review PR is ready for code review and waiting on a reviewer labels Aug 14, 2026
@cuichenx
cuichenx enabled auto-merge (squash) August 14, 2026 18:55
@cuichenx
cuichenx disabled auto-merge August 14, 2026 19:01
@cuichenx
cuichenx merged commit a67b573 into NVIDIA-NeMo:main Aug 14, 2026
39 checks passed
kamran-nvidia added a commit that referenced this pull request Aug 16, 2026
docs/conf.py copies the repo-root skills/ tree into docs/skills/ at build
time, so from docs/training/ the correct relative path is ../skills/...
#5570 used ../../skills/..., which resolves outside the Sphinx source tree.
Sphinx reports "Unknown source document" and the docs job runs with
fail-on-warning: true, so the build fails.

This matches the existing convention in docs/training/moe-optimization.md
and docs/training/communication-overlap.md, whose builds pass.

Signed-off-by: Kamran Jafari <kjafarisadeg@nvidia.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:misc Cross-cutting utilities, logging, helpers, and other changes docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants