@@ -182,26 +182,23 @@ items:
182182 status : verified
183183 precision : bf16
184184 enabled_features : {}
185- bridge_commit : 07d182e7e8df4421e445906b29e6b5f4a2f4a6f1 # pragma: allowlist secret
185+ bridge_commit : f3130a0e37c78945ecf98b301dfc90505c3c305f # pragma: allowlist secret
186186 command : >
187187 ./scripts/training/train.sh --nodes 1 --gpus-per-node 8
188188 --recipe gemma4_vl_26b_sft_config --mode sft --dataset cord-v2
189189 --pretrained_checkpoint work/model-verification/gemma-4-26b-a4b-it/imported-megatron/iter_0000000
190190 --max_steps 10 --seq_length 4096
191191 'dataset.source.load_kwargs={revision:"7f0115a4b758a71d6473b8d085751692da2fef98"}'
192192 validation.eval_iters=0 validation.eval_interval=0 checkpoint.load=null
193- model.recompute_granularity=full model.recompute_modules=null
194- model.recompute_method=uniform model.recompute_num_layers=1
195- model.cuda_graph_impl=none
196193 --save_dir work/model-verification/gemma-4-26b-a4b-it/sft-checkpoints
197194 --save_interval 10 logger.log_interval=1 logger.log_throughput=true
198195 logger.save_config_filepath=work/model-verification/gemma-4-26b-a4b-it/sft-post-setup.yaml
199196 last_verified : 2026-08-11
200197 metrics :
201198 initial_loss : 1.282612
202- final_loss : 0.07524271
203- last_10_steps_step_time_ms_avg : 15600.68
204- last_10_steps_model_tflops_per_gpu_avg : 32.95
199+ final_loss : 0.09127883
200+ last_10_steps_step_time_ms_avg : 15936.25
201+ last_10_steps_model_tflops_per_gpu_avg : 32.20
205202 peak_allocated_memory_gib : 65.223
206203 peak_reserved_memory_gib : 72.356
207204 expected_result : >
@@ -210,10 +207,11 @@ items:
210207 optimizer steps at MBS1 and GBS32. The persisted post-setup config
211208 confirmed a frozen vision encoder, trainable projection and language
212209 model, expandable CUDA allocator segments, and full-layer uniform
213- recompute with one layer per recompute unit while CUDA graphs and CPU
214- offload remained disabled. Loss moved from 1.282612 to 0.07524271,
215- final grad norm was 2.094, and every step reported zero skipped and zero
216- NaN iterations. After iteration 2, logger-reported ranks returned to
210+ recompute with one layer per recompute unit directly from the recipe
211+ defaults, while CUDA graphs and CPU offload remained disabled. Loss
212+ moved from 1.282612 to 0.09127883, final grad norm was 3.090, and all 10
213+ steps reported zero skipped and zero NaN iterations. After iteration 2,
214+ logger-reported ranks returned to
217215 62.134-62.145 GiB allocated, while cumulative peak allocated and
218216 reserved memory remained at or below 65.223 and 72.356 GiB. The run
219217 saved a complete iter_0000010 torch_dist checkpoint with eight rank
0 commit comments