Skip to content

[ckpt] fix: Honor retention for MSC checkpoints (#5559) #20411

[ckpt] fix: Honor retention for MSC checkpoints (#5559)

[ckpt] fix: Honor retention for MSC checkpoints (#5559) #20411

Triggered via push August 13, 2026 16:46
Status Cancelled
Total duration 1h 22m 19s
Artifacts 83

cicd-main.yml

on: push
pre-flight  /  pre-flight
17s
pre-flight / pre-flight
configure
4s
configure
Lint check
25s
Lint check
cicd-wait-in-queue
0s
cicd-wait-in-queue
cicd-compute-build-matrix
3s
cicd-compute-build-matrix
Matrix: cicd-container-build
Matrix: cicd-unit-tests-core
Launch_Unit_Tests_Diffusion
3m 48s
Launch_Unit_Tests_Diffusion
generate-test-matrix
8s
generate-test-matrix
generate-gb200-test-matrix
7s
generate-gb200-test-matrix
Launch_Import_Check
36s
Launch_Import_Check
Matrix: cicd-functional-tests-l0
Matrix: cicd-functional-tests-l1
Matrix: cicd-functional-tests-l2
Matrix: cicd-functional-tests-flaky
Matrix: cicd-functional-tests-gb200-l0
Matrix: cicd-functional-tests-gb200-l1
Matrix: cicd-functional-tests-gb200-l2
Matrix: cicd-functional-tests-gb200-flaky
Nemo_CICD_Test
0s
Nemo_CICD_Test
Coverage_Fake
0s
Coverage_Fake
Matrix: Coverage
Fit to window
Zoom out
Zoom in

Annotations

2 errors, 168 warnings, and 83 notices
L1_Launch_models_glm_vl
Canceling since a higher priority waiting request for CICD NeMo-refs/heads/main-main-push exists
CICD NeMo
Canceling since a higher priority waiting request for CICD NeMo-refs/heads/main-main-push exists
cicd-container-build (gcp, us-east4-docker.pkg.dev/nv-projdgxchipp-20260113193621/megatron-bridge...
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25, docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
cicd-container-build (aws, 766267172432.dkr.ecr.us-east-1.amazonaws.com, nemo-ci-aws-gpu-x2)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25, docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
Launch_Unit_Tests_Diffusion
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
Launch_Unit_Tests_Diffusion
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
Launch_Unit_Tests_Core_0
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
Launch_Unit_Tests_Core_0
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
Launch_Unit_Tests_Core_2
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
Launch_Unit_Tests_Core_2
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
Launch_Unit_Tests_Core_1
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
Launch_Unit_Tests_Core_1
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_perf_config
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_perf_config
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_glm_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_glm_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_deepseek
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_deepseek
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_ministral3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_ministral3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_utils
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_utils
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen35_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen35_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen25_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen25_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen3_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen3_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_gemma3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_gemma3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen2
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen2
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_models_nemotron_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_models_nemotron_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_gemma
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_gemma
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_ministral3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_ministral3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_models_deepseek
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_models_deepseek
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_glm_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_glm_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen3_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_qwen3_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_llama_1b
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_llama_1b
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_deepseek_fsdp
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_deepseek_fsdp
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_kimi_k25_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_kimi_k25_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_gpt_oss
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_gpt_oss
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_vlm_inference
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_vlm_inference
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_gemma4_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_recipes_gemma4_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_data
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_data
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_llama_3b
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_llama_3b
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_gpt_oss
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_gpt_oss
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_stepfun
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_stepfun
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_nemotron_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_nemotron_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_qwen
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_qwen
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_glm
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_models_glm
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_models_qwen3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_models_qwen3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_kimi_k25_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_kimi_k25_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen3_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen3_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen35_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen35_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_converter_fsdp
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_converter_fsdp
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_glm
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_glm
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen2
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen2
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_recipes_perf_config
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_recipes_perf_config
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_converter
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L0_Launch_converter
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_gemma4_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_gemma4_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_deepseek
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_deepseek
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_decentralized_pg
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_decentralized_pg
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_data
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_data
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_flux
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_flux
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_llada15
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_llada15
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_stepfun
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_stepfun
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_gemma
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_gemma
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_gemma3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_gemma3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_gpt_oss
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_gpt_oss
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_ministral3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_ministral3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_glm_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_glm_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen3_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen3_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_nemotron_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_nemotron_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen25_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_qwen25_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_gpt_oss
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_models_gpt_oss
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_callbacks
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_callbacks
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_megatron_fsdp
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_megatron_fsdp
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_recipes_llama_1b
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_recipes_llama_1b
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_recipes_gemma4_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_recipes_gemma4_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_ministral3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_ministral3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_llama_3b
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_llama_3b
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_recipes_deepseek
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_recipes_deepseek
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_distributed_save
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_distributed_save
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_nemotron_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_nemotron_omni
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_qwen
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_qwen
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_converter_generate
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_converter_generate
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_megatron_mimo
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_megatron_mimo
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_local_checkpointing
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_local_checkpointing
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_qwen3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_models_qwen3
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_qwen_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
gb200_L1_Launch_recipes_qwen_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_tensor_inspect
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_tensor_inspect
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_utils
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_utils
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_resiliency
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_resiliency
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_pretrain
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_pretrain
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_finetune
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_training_finetune
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_nemotronh
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_nemotronh
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_vlm_inference
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_vlm_inference
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_converter
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_converter
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_wan_ckpt_conversion
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L0_Launch_wan_ckpt_conversion
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_qwen_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
L1_Launch_recipes_qwen_vl
Input 'app-id' has been deprecated with message: Use 'client-id' instead.
Result
✅ Launch_Unit_Tests_Diffusion — PASSED
Result
✅ Launch_Unit_Tests_Core — PASSED
Result
✅ Launch_Unit_Tests_Core — PASSED
Result
✅ Launch_Unit_Tests_Core — PASSED
Result
✅ L0_Launch_recipes_perf_config — PASSED
Result
✅ L1_Launch_recipes_glm_vl — PASSED
Result
✅ L0_Launch_recipes_deepseek — PASSED
Result
✅ L1_Launch_recipes_ministral3 — PASSED
Result
✅ L0_Launch_utils — PASSED
Result
✅ L1_Launch_models_qwen35_vl — PASSED
Result
✅ L1_Launch_models_qwen25_vl — PASSED
Result
✅ L1_Launch_models_qwen3_omni — PASSED
Result
✅ L1_Launch_models_gemma3 — PASSED
Result
✅ L1_Launch_models_qwen2 — PASSED
Result
✅ L0_Launch_models_nemotron_omni — PASSED
Result
✅ L0_Launch_recipes_gemma — PASSED
Result
✅ L1_Launch_models_ministral3 — PASSED
Result
✅ L0_Launch_models_deepseek — PASSED
Result
✅ L1_Launch_models_glm_vl — PASSED
Result
✅ L1_Launch_models_qwen3_vl — PASSED
Result
✅ L0_Launch_recipes_llama_1b — PASSED
Result
✅ L0_Launch_recipes_deepseek_fsdp — PASSED
Result
✅ L1_Launch_models_kimi_k25_vl — PASSED
Result
✅ L1_Launch_recipes_gpt_oss — PASSED
Result
✅ L1_Launch_vlm_inference — PASSED
Result
✅ L0_Launch_recipes_gemma4_vl — PASSED
Result
✅ L0_Launch_data — PASSED
Result
✅ L1_Launch_recipes_llama_3b — PASSED
Result
✅ L1_Launch_models_gpt_oss — PASSED
Result
✅ L1_Launch_models_stepfun — PASSED
Result
✅ L1_Launch_recipes_nemotron_omni — PASSED
Result
✅ L1_Launch_recipes_qwen — PASSED
Result
✅ L1_Launch_models_glm — PASSED
Result
✅ L0_Launch_models_qwen3 — PASSED
Result
✅ L1_Launch_models_kimi_k25_vl — PASSED
Result
✅ L1_Launch_models_qwen3_omni — PASSED
Result
✅ L1_Launch_models_qwen35_vl — PASSED
Result
✅ L0_Launch_converter_fsdp — PASSED
Result
✅ L1_Launch_models_glm — PASSED
Result
✅ L1_Launch_models_qwen2 — PASSED
Result
✅ L0_Launch_recipes_perf_config — PASSED
Result
✅ L0_Launch_converter — PASSED
Result
✅ L0_Launch_models_gemma4_vl — PASSED
Result
✅ L0_Launch_models_deepseek — PASSED
Result
✅ L0_Launch_training_decentralized_pg — PASSED
Result
✅ L0_Launch_data — PASSED
Result
✅ L0_Launch_models_flux — PASSED
Result
✅ L1_Launch_models_llada15 — PASSED
Result
✅ L1_Launch_models_stepfun — PASSED
Result
✅ L1_Launch_recipes_gemma — PASSED
Result
✅ L1_Launch_models_gemma3 — PASSED
Result
✅ L1_Launch_recipes_gpt_oss — PASSED
Result
✅ L1_Launch_models_ministral3 — PASSED
Result
✅ L1_Launch_recipes_glm_vl — PASSED
Result
✅ L1_Launch_models_qwen3_vl — PASSED
Result
✅ L0_Launch_models_nemotron_omni — PASSED
Result
✅ L1_Launch_models_qwen25_vl — PASSED
Result
✅ L1_Launch_models_gpt_oss — PASSED
Result
✅ L0_Launch_training_callbacks — PASSED
Result
✅ L0_Launch_megatron_fsdp — PASSED
Result
✅ L0_Launch_recipes_llama_1b — PASSED
Result
✅ L0_Launch_recipes_gemma4_vl — PASSED
Result
✅ L1_Launch_recipes_ministral3 — PASSED
Result
✅ L1_Launch_recipes_llama_3b — PASSED
Result
✅ L0_Launch_recipes_deepseek — PASSED
Result
✅ L0_Launch_training_distributed_save — PASSED
Result
✅ L1_Launch_recipes_nemotron_omni — PASSED
Result
✅ L1_Launch_recipes_qwen — PASSED
Result
✅ L0_Launch_converter_generate — PASSED
Result
✅ L0_Launch_training_megatron_mimo — PASSED
Result
✅ L0_Launch_training_local_checkpointing — PASSED
Result
✅ L0_Launch_models_qwen3 — PASSED
Result
✅ L1_Launch_recipes_qwen_vl — PASSED
Result
✅ L0_Launch_training_tensor_inspect — PASSED
Result
✅ L0_Launch_utils — PASSED
Result
✅ L0_Launch_training_resiliency — PASSED
Result
✅ L0_Launch_training_pretrain — PASSED
Result
✅ L0_Launch_training_finetune — PASSED
Result
✅ L1_Launch_recipes_nemotronh — PASSED
Result
✅ L1_Launch_vlm_inference — PASSED
Result
✅ L0_Launch_converter — PASSED
Result
✅ L0_Launch_wan_ckpt_conversion — PASSED
Result
✅ L1_Launch_recipes_qwen_vl — PASSED

Artifacts

Produced during runtime
Name Size Digest
coverage-e2e-31722375079-02b38f6a-6f3d-49ef-9492-eebaf0c460de
348 KB
sha256:dba5036038b9e85c1f47d9e818fd2f93268fefeecec3fb1a0575c57561c20315
coverage-e2e-31722375079-032bc957-15fb-4809-bdb0-eab6fc4a8e3e
773 KB
sha256:879440070f427ea04d361278c935a1eebd1cab65df7ed2ddc7b365fcd003da60
coverage-e2e-31722375079-0868e5c6-e333-43df-a332-267a847cb9ee
379 KB
sha256:453ffff15b12debbccc4ad2bbf0aabb508c370244cecec98a1ae083cb0257e5e
coverage-e2e-31722375079-0bb32b75-f893-4e30-84fd-d5920310cd79
420 KB
sha256:6a1f35e71582b1afadd4bd66760ab4b9f954d4c21fc758ce2d37b1e50f97545c
coverage-e2e-31722375079-0c257a18-b6bb-4cc7-baa4-f56a71f9ee7e
842 KB
sha256:4628ed24de8b1adc730a59320b8f8dd2ed1bbfd6fae51ae09d25c21740cdff44
coverage-e2e-31722375079-0e047ec8-0b77-47a9-8b92-514b93ee566c
420 KB
sha256:8000ae1be4f2b1cec37df568988f095898c979f7f900226289bfc831e472f773
coverage-e2e-31722375079-15e9bb43-f12b-4478-88be-9b3be28df0b0
277 KB
sha256:6b6b624404e09ed4ebc10836a6645e34fa6e95b5d113b7ed82833c1f306a3f0f
coverage-e2e-31722375079-1692ae2e-3b0f-4c38-85e7-45306cc4dfc5
414 KB
sha256:83e77335a11d90b3aeb58eef5f308c4c000c2ac4b704119179391c44a24e55f7
coverage-e2e-31722375079-18bd97da-ec36-4bc0-bef9-8274fce1d884
406 KB
sha256:b3abe0af9429c74ac0991bb49df7e9ac61a1ea7019d206d8cbcc78caf15df474
coverage-e2e-31722375079-240844d8-f12d-4882-bdf2-55a820181940
327 KB
sha256:1375b8bfb24f3dd9cf8567a7bbedb75c3446c0e2a24af051bdfd78b3ad91d70c
coverage-e2e-31722375079-252bba95-2791-40e3-b53d-8f55b72d2ee4
772 KB
sha256:8d5e617b8aaeeec09d3677b2d2ac423083b506d2096fc9f9287e14e9864262da
coverage-e2e-31722375079-2955a4a3-6f16-4c03-aee9-b12c51982bb0
278 KB
sha256:a2e523cb296410837f00bb73e9007894c915469587519585ab94215a5017ed29
coverage-e2e-31722375079-2a3209c4-a905-4cd4-affc-2c3c14f0cc5e
774 KB
sha256:f7f4f64d1bd3e54b61955391a8b8f2636aa10ae0139cae3521c4333720d0d75e
coverage-e2e-31722375079-2e461162-64a1-43a5-9918-2f39971ac42b
550 KB
sha256:c0c502919de2d71a0cf92f5c7c65d9d36a1b169ca77ee81548362640fec310a7
coverage-e2e-31722375079-315b3371-61f3-42f9-b130-10198e8e5252
844 KB
sha256:dac12be4c37c8f21e1adde012862a6a472297ae642366618e3bab521f9a22eff
coverage-e2e-31722375079-38283828-2b34-4d08-a682-445fe26f87fc
793 KB
sha256:ff0ce162251f5fdc34e8784d39f4853aa72fae038de0f85640e0f49e82017916
coverage-e2e-31722375079-38b9cbc1-b327-4735-ba7c-abab9a26c98d
772 KB
sha256:e974979372ea5dde048e648c7f111d218b83beffeb2fac57d9541b267afbc255
coverage-e2e-31722375079-3e6c2592-0b79-41b1-8370-5aa88b52060a
772 KB
sha256:d93ab5c6c631467396d8b7c75c9dc750d6ce4ce504869aa2d5e10be3fe98d29d
coverage-e2e-31722375079-408ac3b7-81f2-40aa-b96e-f9e659d7ec41
409 KB
sha256:68973f54a1d819b428eab17f0b82740e6d31ad496647cf4d19f103f2045deaa5
coverage-e2e-31722375079-4282f98f-ed25-4056-b4c3-1d359400c2e5
369 KB
sha256:23bc18e2bb8a6e99c4a45c9e00806c0fd8aec39414835adb929b5d71441fa7fd
coverage-e2e-31722375079-44dab9b9-e800-429a-b19e-223cd8758c44
791 KB
sha256:909b803425cc3a28b55b75d91d4bb0f7302ac68c2d34defe4fa57c6cceddc33d
coverage-e2e-31722375079-49a1e349-64d3-4f37-9a1a-5162d4ddc818
847 KB
sha256:3a6abe4b0c104db48339ddd9b0be5b3cdc26caf7ae4fc5c21bc8bd25e1c8b6b4
coverage-e2e-31722375079-4ade7e59-49bf-497c-a1f1-28b9af822d34
318 KB
sha256:e3760e407ef04030940dfa89bd956342be53dc8718a569178d7e262b14988d70
coverage-e2e-31722375079-4d36bde8-390a-495e-aa03-f72ca9bf8766
278 KB
sha256:2fd2576b58832da2f8ade4c54ce527d8b30e7c1d7bdf01f7a78f0cc7c683b078
coverage-e2e-31722375079-4ed1af33-9713-4f72-bffe-50a336d75bb8
422 KB
sha256:3d09ec9b751d9d96b15405a427ea2b6826ea60531c4260307f27f4e92332e585
coverage-e2e-31722375079-51c559cb-6693-4701-bf7e-d5478bb1a40f
389 KB
sha256:996b2b9088b53e6d46f95e7d1f4c6b0a28c07d8aec6fd7417d82c522b0c9a0a4
coverage-e2e-31722375079-5238cf32-19ca-4e51-ba52-ad0fc160973b
772 KB
sha256:333e2003c12ef7dd437d6b6032154ce54f6cb2438c577cb447da01dfa6dbde7f
coverage-e2e-31722375079-546e68b4-3ba1-4a4c-b847-d9209737a1ad
302 KB
sha256:8ae8c09c7b834e213fd342f7b1f87f1581a83a8486e122222a82fdbe654be4b4
coverage-e2e-31722375079-598d6852-774a-4e0e-9204-8f9f473a0944
422 KB
sha256:a13297245b985162e462f7b5cb5fe781ec9724ab1136de83e5244ce757cb2e1a
coverage-e2e-31722375079-5a21e5e3-4b5e-4942-8edc-dc43ef63cdf3
855 KB
sha256:71efb54c650fc077b2588e776b2b967df6266842e3cd678626c11dd501f5266a
coverage-e2e-31722375079-5d5e8b98-47d4-4c31-b053-95da82af3b00
378 KB
sha256:7c17a56b86798a9b25b41d5380446193fcad08d045413822a46ccbe19368404b
coverage-e2e-31722375079-5e6eb44e-5bfc-4458-a2c7-87bf6f717e74
278 KB
sha256:e1c37fc95f4e526cfa6aa6ca3991e98f0894250ded9e86ab0e3447a952198da8
coverage-e2e-31722375079-61ae8fbc-1079-4220-9e7f-9f0cf3f5e252
349 KB
sha256:054048eea59d1f9e6131b8177dc87c5c18a1ba5ecf1aaa7837902c7d8831af0a
coverage-e2e-31722375079-7370aea6-e1c3-418f-ad25-a0bc65d7697a
349 KB
sha256:083191992c77e46b8fb7b850f37241e276f2073d1304acc4bb7f7a91ae521e81
coverage-e2e-31722375079-738b87fe-2bd2-41c0-9918-d3df32dfd442
274 KB
sha256:c6c9824225a196e3dfa4de82b014c75b27b9572e58c50f50d98ee4220d4b1c74
coverage-e2e-31722375079-7425e0a6-2c9b-43d6-96df-4e3344cb08c9
774 KB
sha256:485e1a7f0cd9a14f47d14a9ebd58df85224534c713cf83fb05cbc5ef4f2a5b0d
coverage-e2e-31722375079-7468ed96-590f-490d-b0ae-1bd5ae674945
774 KB
sha256:65573a8cf1d01f0c9bb88773d7cb6ddaedc59d06d0612b9498519f619c9cdea8
coverage-e2e-31722375079-840212e3-0872-49aa-9bde-70ca96b4fa34
277 KB
sha256:786f3f15b0d35af872672aa614cf2ac3ff77c9800fcd6d6a2f43df1029f860b7
coverage-e2e-31722375079-84fec89e-88ba-45df-a77e-f3d9ae56353e
844 KB
sha256:a89dbbc859580364b9cd814690bedc5bfc1a219ce1a98ae208ef6d3b47eeaa6c
coverage-e2e-31722375079-95418da2-cd9f-4450-aac5-c094942deecd
277 KB
sha256:6ba094bf53bfd510da85ee9d095526c3896b664bba0a2a1102c53b3130027a4c
coverage-e2e-31722375079-96a0eae9-1dcc-459b-9d46-d0eeb7edf150
422 KB
sha256:cc6ea80cb24bafafc044aed88964c32064327621e530bdbc3f9ae35a7055b4e0
coverage-e2e-31722375079-9d36c7d4-9148-4d70-93d8-037e4bbd8e13
327 KB
sha256:33bc4a050f397327b2275b86f12f4e4ad4dd4128478d1cc56cbf137fdb210308
coverage-e2e-31722375079-9f80bd85-dd7d-4100-95df-896b9ae0fa84
277 KB
sha256:6f31c18d50c3cc427f9f5c697c7da75cd849ad35279102a7ea74717ae7ba2d9e
coverage-e2e-31722375079-a14fe821-8839-4108-836c-11d6c5f11844
277 KB
sha256:14145b06bba3f09a8d6446d2fe7b1381ff5b5b10dd4f837944f9da4500f33f70
coverage-e2e-31722375079-a6b6fc05-fbdb-47bf-ae9f-69892a8694ec
847 KB
sha256:058242e0fb4f00fd4a366074f9fae056bd8e81c1b5003b751fe2de0fd6b86e63
coverage-e2e-31722375079-a916f048-ec53-4023-af2a-57a9c7b483d4
274 KB
sha256:f475aef239806d086c173dd9f66d21a722187396467aecef758808e14bcdea88
coverage-e2e-31722375079-b311871a-99e7-40bf-b1f4-102b3df965f4
319 KB
sha256:7a0667ac8d136e9dca57e567073e862b04f0e8a6c281df7b50d0115f108b05eb
coverage-e2e-31722375079-b60f0424-6090-433c-8197-846b37ebbb66
844 KB
sha256:96d079adcb56e0fb4e0d7baac8c090288192c2c40123632e64f04baecc2f477c
coverage-e2e-31722375079-b6c8921f-a913-4a51-8f6e-c8f77e7a23ae
847 KB
sha256:f4c66f1631ac25f9b5c749939531e196e7ac419b5a4db79144745899d3ec1540
coverage-e2e-31722375079-b9b2c88d-9176-4a83-9aea-909d721614eb
277 KB
sha256:979f513153cb17797121ee83b352aee42f5ea3dca1abfad49262961ac6d5829b
coverage-e2e-31722375079-bab65ab3-fa76-4233-a4f9-f4461cc6d48d
771 KB
sha256:2823feb6de18b03f3849417fcf85020a99055f32268afe68e9b5c85aa6b307ad
coverage-e2e-31722375079-bbaa7ec2-f403-4c8a-b26b-03585280eb3f
376 KB
sha256:ce75888683431f3f1680ecf4ac36ed6cde1e468c1339352dea61d3b89c279cc4
coverage-e2e-31722375079-bbac1f5c-419a-461a-8c74-1e8c420ce15a
312 KB
sha256:3e25029aad7bc17644f6c7faea7867b3114c898311b23d5cb33d74a5722e0c14
coverage-e2e-31722375079-bc1a7dae-3186-432b-bb57-e596ef67c149
775 KB
sha256:7a911ae9ab239275d4de9f7f2ea78d7055bee2e0dd6aa18c236a6ff38356a2fe
coverage-e2e-31722375079-be621dd8-d38a-4f41-b1f6-77251292a231
847 KB
sha256:a9ee3436bdb2e1bdd2553ab0a1efbce471f18b15136ce1030d34419b6686126b
coverage-e2e-31722375079-c3f3a837-ce04-4e63-8599-b721d5f5de6f
278 KB
sha256:51691fa014a36386ad466f0d369874feb0b6dbcf645a3a4cdb22d98a33cf074c
coverage-e2e-31722375079-c8a01314-07fb-48a1-8784-abdb8f41d187
771 KB
sha256:07e22f04b376b0d878606b7a22df1db14e7d9e21c27a000f3c64b1a1cbfa26ea
coverage-e2e-31722375079-cb6ebe2d-7550-4d5a-816d-3f59449fdcca
278 KB
sha256:f04c7303324eb4773444b30118d6af1e95d0563f015ef6a02322263842148602
coverage-e2e-31722375079-cc435647-c9df-48e2-b3db-da6853530ed0
302 KB
sha256:0b67511c431f64040c123c389c0145b468bd56a4a213be51dab596d5e98ffb58
coverage-e2e-31722375079-cc4964d1-1c17-499a-b585-597b117b2ec6
846 KB
sha256:77bbd848c4e2919244ce5985a04f32cdb6948ef99b067a2caa518fd1448309e2
coverage-e2e-31722375079-cd70f2c3-6c21-40d7-8d0d-298a82efcf1b
849 KB
sha256:88d9954a905ab8f719016d2177b5e6371ea251183dcae75067c20197e1e195c0
coverage-e2e-31722375079-d336762f-160e-4b94-8b10-437625e4fb9e
318 KB
sha256:2cfc9d4e5366de2f31bfed4becfd2fcab6cf96832821d6f2393a78ac8c8851d3
coverage-e2e-31722375079-d5e51791-1151-423b-8ec7-574bfd59b02a
772 KB
sha256:1ab14f6218a5c8dd2e83bf35cd0ccb00884796b2dc9d93f48acdd730b42b2161
coverage-e2e-31722375079-d784b2d1-94a9-4dbf-be47-7187e0fb7713
794 KB
sha256:1606df0c24d30fdce368ffbc8ea8cd823209dc2b3982f32cf3b04ebccc2e4add
coverage-e2e-31722375079-d9980ca0-14ed-4387-bec3-9a8b02a91f5d
389 KB
sha256:9a9957ca647fe02b81cd8cf7365d0f78f30e787aa4d4c1fe4f69da6f4413028b
coverage-e2e-31722375079-dae7248c-5bab-4117-94bf-7029de926165
414 KB
sha256:ac11881dbe47961e726f8014c0659fd4ceed2bd18aa0369ca6b1c88b44a51f7e
coverage-e2e-31722375079-db5f4942-d81e-4995-b26f-bcaaa9cf3f3a
423 KB
sha256:e599184e94e5c84ef53b80fb79b246b6749df3f5feb4368b524b061ad15f4d86
coverage-e2e-31722375079-dc01041b-c206-4fab-9fff-d48a0b9a9bba
277 KB
sha256:1cd107e2558b0c49b0575def834395af29c4dd1cab1efbdb13e60fbc899e10af
coverage-e2e-31722375079-df99502a-c11b-426d-9f9b-bb674be9bfd0
369 KB
sha256:11eaf5f3336d41316d5f57eb21b3e51bd9e63d65487aeae8286783b9578709f3
coverage-e2e-31722375079-e499d2d8-101d-44e2-ad03-395f60e98faa
319 KB
sha256:385a92955848e80b522f09f18fcae0108a417c88d28724c739cf2e0de367777b
coverage-e2e-31722375079-e55b38c3-9698-4656-aa04-635213c89ec4
378 KB
sha256:a5c831d785eafde45987c7762234937665de6cb18ba6569c40a79517fa3fd0a1
coverage-e2e-31722375079-eb965b86-2246-497f-9a4a-f8b1b63c8f5d
379 KB
sha256:c50877b75c7fee2b78d7cc87eb6632e9fc3bce7b8b721be7dd687cd6786ade9a
coverage-e2e-31722375079-ef329791-56a7-4b8c-b369-787bfbf978a5
844 KB
sha256:2750759b7f8d89a6a4bd84ebaff2355397e636d7a5bd0fe68120c2564351ad3f
coverage-e2e-31722375079-f168030e-3e31-42cd-aa47-9e1710b56c85
772 KB
sha256:71abb4cf20e2cc6e6f4dd12d7678093b0957330f1e751faab28c691fd9f7e23b
coverage-e2e-31722375079-f5759902-3366-44df-bafb-ee1c1359579f
318 KB
sha256:ffc0227ca745a73ead5b2793425bd1333adf372e3300f96a7599f42351bee4f8
coverage-e2e-31722375079-f64eb92d-fa60-4b27-a46e-0ce9127c7a51
277 KB
sha256:4065a58b11977790d6d9b1bee4b686319b92e442333849662487b73290199106
coverage-e2e-31722375079-f7c2e590-42ea-402c-aa8e-60c4555e39ea
376 KB
sha256:17efe58aa82161c6ad1f10b237c8d85ee61a723582b2087038e7e6917be10525
coverage-e2e-31722375079-f7d9063a-3fa3-40ca-9c75-d095453cc663
492 KB
sha256:51a3cd1355c937d99dcc1c09d76c99a01f470418ae8577357535cebd59ac117a
coverage-e2e-31722375079-fcf85526-5561-4eed-8315-76d3415988ba
1.29 KB
sha256:22d9b87e31e3fee430c8f232ff599aed465f40815ab6a65e932fd9537f1734c0
coverage-unit-test-31722375079-64f20d96-f117-461c-a2d4-42d03a0f8ab8
922 KB
sha256:1cc98a28fd9cf924806f5e9535565def4e4fe30e6017b5f2d882d429a8a87cb4
coverage-unit-test-31722375079-75764506-db05-443c-bc31-6ba5a93b2bad
923 KB
sha256:a65acb590520a1300353d01a0cdb828efe68457327833f1ae6cbf1686f9ef0c0
coverage-unit-test-31722375079-ba94ab1d-40a6-4490-bdcb-859650bdd9cd
921 KB
sha256:4802c518cc1b5041c96698a44e6040bfd941d859be0cd532fe5a325d7f79b2e0
coverage-unit-test-31722375079-d8d2ccd8-d863-4a94-82ef-cbe3227f3da9
375 KB
sha256:70668511e58e910b26eb217353aaceb2c808c016078347434223959ec0a09457