@@ -1,85 +1,101 @@
-# Evaluation Report
+# Skill Benchmark: nemo-mbridge-perf-activation-recompute
 
-Evaluation of the `nemo-mbridge-perf-activation-recompute` skill before publication through NVSkills-Eval.
+> ✅ **Overall verdict: PASS — Recommended for publication**
 
-> **Refresh required:** the skill guidance and evaluation task changed on 2026-08-12. The results below are retained as historical evidence for the previous version and do not validate the current version.
+## Publication Recommendation
 
-This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use.
+Recommended for publication based on the completed evaluation evidence in this report.
 
-## Evaluation Summary
+## Evaluation Metadata
 
 - Skill: `nemo-mbridge-perf-activation-recompute`
-- Evaluation date: 2026-06-15
-- NVSkills-Eval profile: `external`
-- Environment: `astra-sandbox`
-- Dataset: 1 evaluation tasks
+- Evaluation date: 2026-08-16
+- Evaluator version: `1.2.7`
+- Agents: Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`), Codex (`openai/openai/gpt-5.5`)
+- Tasks: 3 evaluation tasks (3 positive)
+- Dataset digest: `sha256:b55bb3c51b83cb2245c0d7a8969bbac7066177e342bd9b9c316ac7cf26212ef6` (skill-evaluator-dataset-snapshot/1)
 - Attempts per task: 1
-- Pass threshold: 50%
-- Historical verdict: PASS
-- Current-version verdict: pending a fresh NVSkills-Eval run
+- Environment: `local`
+- Tier 3 evidence: required for publication
 
-## Agents Used
+Tasks ran on the trusted local host; local mode is not sandboxed.
 
-- `claude-code`
-- `codex`
+## What This Report Answers
 
-## Metrics Used
+The three-tier evaluation checks whether the skill:
 
-Reported benchmark dimensions:
+- is safe to use;
+- produces correct answers;
+- is discovered and activated when needed;
+- helps the agent complete the user's goal and expected workflow; and
+- avoids wasted skill and tool usage.
 
-- Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
-- Correctness: checks whether the agent follows the expected workflow and produces the correct final output.
-- Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
-- Effectiveness: checks whether the agent performs measurably better with the skill than without it.
-- Efficiency: checks whether the agent uses fewer tokens and avoids redundant work.
+## Results at a Glance
 
-Underlying evaluation signals used in this run:
+| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
+|---|---:|---:|
+| Overall | 70% → 99% (+29 points) | 53% → 97% (+44 points) |
+| Security | 100% → 100% (±0 points) | 100% → 100% (±0 points) |
+| Correctness | 33% → 100% (+67 points) | 67% → 100% (+33 points) |
+| Discoverability | 100% → 100% (±0 points) | 40% → 90% (+50 points) |
+| Effectiveness | 19% → 94% (+74 points) | 30% → 95% (+65 points) |
+| Efficiency | 96% → 100% (+4 points) | 27% → 100% (+73 points) |
 
-- `security` (Security): checks for unsafe operations, secret leakage, and unauthorized access.
-- `skill_execution` (Skill Execution): verifies that the agent loaded the expected skill and workflow.
-- `skill_efficiency` (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage.
-- `accuracy` (Accuracy): grades final-answer correctness against the reference answer.
-- `goal_accuracy` (Goal Accuracy): checks whether the overall user task completed successfully.
-- `behavior_check` (Behavior Check): verifies expected behavior steps, including safety expectations.
-- `token_efficiency` (Token Efficiency): compares token usage with and without the skill.
+**How to read this table:** baseline is the same task attempted without the target skill. Uplift is `skill score - baseline score`, shown in percentage points.
 
-## Test Tasks
+Example: `47% → 92% (+45 points)` means the skill-assisted run scored 92%, 45 percentage points above its 47% no-skill baseline.
 
-The benchmark dataset contained 1 evaluation tasks:
+## Tier Status
 
-- Positive tasks: 1 tasks where the skill was expected to activate.
-- Negative tasks: 0 tasks where no skill was expected.
-- Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred.
+| Tier | Purpose | Status | Evidence |
+|---|---|---|---|
+| Tier 1 | Static validation | **PASSED WITH OBSERVATIONS** | 1 validator(s); 4 finding(s) |
+| Tier 2 | Semantic deduplication | **NOT RUN** | No result was recorded |
+| Tier 3 | Live agent evaluation | **PASS** | 2 agent(s); 3 task(s) |
 
-Task composition is derived from the evaluation dataset when possible. Entries with `expected_skill` set are treated as positive skill-activation cases, while entries with `expected_skill: null` are treated as negative activation cases.
+## Findings and Observations
 
-## Results
+<details>
+<summary>Show detailed findings and successful checks</summary>
 
-| Dimension | Num | `claude-code` | `codex` |
-|---|---:|---:|---:|
-| Security | 1 | 100% (+0%) | 100% (+0%) |
-| Correctness | 1 | 100% (+100%) | 87% (+40%) |
-| Discoverability | 1 | 100% (+100%) | 97% (+0%) |
-| Effectiveness | 1 | 96% (+80%) | 80% (+54%) |
-| Efficiency | 1 | 94% (+67%) | 96% (-0%) |
+- **MEDIUM** SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/nemo-mbridge-perf-activation-recompute/SKILL.md`)
+- **MEDIUM** SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/nemo-mbridge-perf-activation-recompute/SKILL.md`)
+- **MEDIUM** SCHEMA/author_missing: Author not specified in metadata (`skills/nemo-mbridge-perf-activation-recompute/SKILL.md`)
+- **LOW** SCHEMA/unexpected_file: Unexpected 'card.yaml' in skill root (`skills/nemo-mbridge-perf-activation-recompute/card.yaml`)
 
-Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available.
+</details>
 
-## Tier 1: Static Validation Summary
+## Scoring Methodology
 
-Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 4 total findings.
+<details>
+<summary>Show dimension definitions, source signals, and thresholds</summary>
 
-Top findings:
+| Dimension | Question | Scored signals |
+|---|---|---|
+| Security | Is it safe to use? | `security` (100%) |
+| Correctness | Is the answer correct? | `accuracy` (100%) |
+| Discoverability | Was the right skill loaded when needed? | `skill_execution` (100%) |
+| Effectiveness | Did the skill help complete the task? | `goal_accuracy` (50%) + `behavior_check` (50%) |
+| Efficiency | Did it avoid wasted tool or skill usage? | `skill_efficiency` (100%) |
 
-- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/nemo-mbridge-perf-activation-recompute/SKILL.md`)
-- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/nemo-mbridge-perf-activation-recompute/SKILL.md`)
-- MEDIUM SCHEMA/author_missing: Author not specified in metadata (`skills/nemo-mbridge-perf-activation-recompute/SKILL.md`)
-- LOW SCHEMA/unexpected_file: Unexpected 'card.yaml' in skill root (`skills/nemo-mbridge-perf-activation-recompute/card.yaml`)
+- Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%.
+- Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL.
+- Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate.
+- The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold.
+- Effectiveness is the equal-weight mean of goal completion (`goal_accuracy`) and expected workflow adherence (`behavior_check`).
+- Token efficiency is a separate report-only signal. It does not change a dimension score or the overall verdict.
 
-## Tier 2: Deduplication Summary
+Signals present in this run:
 
-This tier was not run or did not produce findings in this report.
+- `security` (Security): unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution` (Skill Execution): whether the expected skill was found and executed.
+- `skill_efficiency` (Efficiency): routing quality, workspace-aware skill reads, and productive tool use.
+- `accuracy` (Accuracy): final-answer correctness against the reference answer.
+- `goal_accuracy` (Goal Accuracy): whether the user's goal was achieved.
+- `behavior_check` (Behavior Check): whether the expected workflow behavior was followed.
 
-## Historical Publication Recommendation
+</details>
 
-The previous skill version was suitable to proceed toward publication based on this benchmark. Because the evaluation dataset and skill behavior have now changed materially, rerun NVSkills-Eval before treating the current version as benchmark-validated.
+## Freshness
+
+Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes.
@@ -1,75 +1,84 @@
 ## Description: <br>
-Validate and use architecture-aware selective and full activation recompute in Megatron Bridge to reduce GPU memory usage at the cost of extra compute. <br>
+Validate and use selective and full activation recompute in Megatron Bridge to reduce GPU memory usage at the cost of extra compute. <br>
 
 This skill is ready for commercial/non-commercial use. <br>
 
-The skill guidance and evaluation dataset changed materially on 2026-08-12. The evaluation results below are historical results for the previous version and do not validate the current version; a fresh NVSkills-Eval run is pending. <br>
-
 ## Owner
 NVIDIA <br>
 
 ### License/Terms of Use: <br>
 Apache 2.0 <br>
 ## Use Case: <br>
-Developers and engineers reducing GPU memory pressure via activation recompute in Megatron Bridge training workloads, or investigating commits that changed recompute settings and caused OOM errors or performance regressions. <br>
+Developers and engineers diagnosing activation memory OOMs or tuning recompute boundaries (recompute_granularity, recompute_modules, recompute_method) in Megatron Bridge training recipes. <br>
 
 ### Deployment Geography for Use: <br>
 Global <br>
 
+## Requirements / Dependencies: <br>
+**Requires API Key or External Credential:** [No] <br>
+**Credential Type(s):** [None] <br>
+
+Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. <br>
+
 ## Known Risks and Mitigations: <br>
 Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br>
 Mitigation: Review and scan skill before deployment. <br>
 
 ## Reference(s): <br>
-- [Megatron Bridge Performance Tuning Guide](../../docs/performance-guide.md) <br>
-- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/) <br>
+- [Megatron Core Activation Recomputation API Guide](https://docs.nvidia.com/megatron-core/developer-guide/latest/api-guide/index.html) <br>
+- [Activation Recomputation Documentation](docs/training/activation-recomputation.md) <br>
 
 
 ## Skill Output: <br>
-**Output Type(s):** [Configuration instructions, Shell commands, Analysis] <br>
-**Output Format:** [Markdown with inline Python and bash code blocks] <br>
+**Output Type(s):** [Configuration instructions, Analysis] <br>
+**Output Format:** [Markdown with inline Python code blocks] <br>
 **Output Parameters:** [1D] <br>
 **Other Properties Related to Output:** [None] <br>
 
 ## Evaluation Agents Used: <br>
-- Claude Code (`claude-code`) <br>
-- Codex (`codex`) <br>
+- Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`) <br>
+- Codex (`openai/openai/gpt-5.5`) <br>
 
 
 
 ## Evaluation Tasks: <br>
-The current package defines 3 internal skill-activation tasks. The historical results below used 1 previous-version task in the NVSkills-Eval external profile. <br>
+Evaluated against 3 internal skill evaluation tasks (3 positive). <br>
 
 ## Evaluation Metrics Used: <br>
 Reported benchmark dimensions: <br>
-- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. <br>
-- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output. <br>
-- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant. <br>
-- Effectiveness: Checks whether the agent performs measurably better with the skill than without it. <br>
-- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work. <br>
+- Security: Whether the skill is safe to use (unsafe operations, secret leakage, unauthorized access). <br>
+- Correctness: Whether the skill produces correct answers against reference answers. <br>
+- Discoverability: Whether the right skill is loaded and activated when needed. <br>
+- Effectiveness: Whether the skill helps complete the user's goal and expected workflow. <br>
+- Efficiency: Whether the skill avoids wasted tool or skill usage. <br>
 
 Underlying evaluation signals used in this run: <br>
-- `security`: Checks for unsafe operations, secret leakage, and unauthorized access. <br>
-- `skill_execution`: Verifies that the agent loaded the expected skill and workflow. <br>
-- `skill_efficiency`: Checks routing quality, decoy avoidance, and redundant tool usage. <br>
-- `accuracy`: Grades final-answer correctness against the reference answer. <br>
-- `goal_accuracy`: Checks whether the overall user task completed successfully. <br>
-- `behavior_check`: Verifies expected behavior steps, including safety expectations. <br>
-- `token_efficiency`: Compares token usage with and without the skill. <br>
-
-
-
-## Historical Evaluation Results: <br>
-| Dimension | Num | `claude-code` | `codex` |
-|---|---:|---:|---:|
-| Security | 1 | 100% (+0%) | 100% (+0%) |
-| Correctness | 1 | 100% (+100%) | 87% (+40%) |
-| Discoverability | 1 | 100% (+100%) | 97% (+0%) |
-| Effectiveness | 1 | 96% (+80%) | 80% (+54%) |
-| Efficiency | 1 | 94% (+67%) | 96% (-0%) |
+- `security`: Unsafe operations, secret leakage, and unauthorized access. <br>
+- `skill_execution`: Whether the expected skill was found and executed. <br>
+- `skill_efficiency`: Routing quality, workspace-aware skill reads, and productive tool use. <br>
+- `accuracy`: Final-answer correctness against the reference answer. <br>
+- `goal_accuracy`: Whether the user's goal was achieved. <br>
+- `behavior_check`: Whether the expected workflow behavior was followed. <br>
+
+
+
+## Evaluation Results: <br>
+| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
+|---|---:|---:|
+| Overall | 70% → 99% (+29 points) | 53% → 97% (+44 points) |
+| Security | 100% → 100% (±0 points) | 100% → 100% (±0 points) |
+| Correctness | 33% → 100% (+67 points) | 67% → 100% (+33 points) |
+| Discoverability | 100% → 100% (±0 points) | 40% → 90% (+50 points) |
+| Effectiveness | 19% → 94% (+74 points) | 30% → 95% (+65 points) |
+| Efficiency | 96% → 100% (+4 points) | 27% → 100% (+73 points) |
+
+## Testing Completed: <br>
+**[x] Agent Red-Teaming** <br>
+**[ ] Network Security** <br>
+**[ ] Product Security** <br>
 
 ## Skill Version(s): <br>
-Historical evaluated version: v0.2.0rc6-1622-g853062e4 (source: git describe). Current version: evaluation pending. <br>
+1.0.0+b7643bd (source: pyproject.toml) <br>
 
 ## Ethical Considerations: <br>
 NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>