Skip to content

Commit f3130a0

Browse files
committed
docs(card): refresh Gemma 4 VL SFT evidence
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
1 parent 0ccbb1f commit f3130a0

1 file changed

Lines changed: 28 additions & 23 deletions

File tree

  • examples/model_verification_cards/gemma-4-26b-a4b-it

‎examples/model_verification_cards/gemma-4-26b-a4b-it/card.yaml‎

Lines changed: 28 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,12 @@ summary: >
1111
forward comparison remains unverified because the recorded run compared a
1212
tensor-parallel padding position rather than the prompt's next token.
1313
Decoder-focused SFT is verified on one 8-GPU H100 node with TP4/PP1/EP8/ETP1,
14-
a frozen vision encoder, and selective core-attention and MoE-activation
15-
recompute. The real CORD-v2 run completed 10 optimizer steps with stable
16-
memory and saved iter_0000010. LoRA PEFT is also verified on four H100 GPUs
17-
with TP2/PP1/EP4/ETP1 and a complete adapter checkpoint. Long-context SFT is
18-
pending. The published Bridge recipes do not currently provide VLM pretraining.
14+
a frozen vision encoder, and full-layer uniform activation recompute with one
15+
layer per recompute unit. The real CORD-v2 run completed 10 optimizer steps
16+
with stable memory and saved iter_0000010. LoRA PEFT is also verified on four
17+
H100 GPUs with TP2/PP1/EP4/ETP1 and a complete adapter checkpoint.
18+
Long-context SFT is pending. The published Bridge recipes do not currently
19+
provide VLM pretraining.
1920
verification_index:
2021
model_level:
2122
verified:
@@ -181,39 +182,43 @@ items:
181182
status: verified
182183
precision: bf16
183184
enabled_features: {}
184-
bridge_commit: 81e559ca4d579cf2d9ed5b8f3f48bef82e06451f # pragma: allowlist secret
185+
bridge_commit: 07d182e7e8df4421e445906b29e6b5f4a2f4a6f1 # pragma: allowlist secret
185186
command: >
186187
./scripts/training/train.sh --nodes 1 --gpus-per-node 8
187188
--recipe gemma4_vl_26b_sft_config --mode sft --dataset cord-v2
188189
--pretrained_checkpoint work/model-verification/gemma-4-26b-a4b-it/imported-megatron/iter_0000000
189190
--max_steps 10 --seq_length 4096
190191
'dataset.source.load_kwargs={revision:"7f0115a4b758a71d6473b8d085751692da2fef98"}'
191192
validation.eval_iters=0 validation.eval_interval=0 checkpoint.load=null
193+
model.recompute_granularity=full model.recompute_modules=null
194+
model.recompute_method=uniform model.recompute_num_layers=1
195+
model.cuda_graph_impl=none
192196
--save_dir work/model-verification/gemma-4-26b-a4b-it/sft-checkpoints
193197
--save_interval 10 logger.log_interval=1 logger.log_throughput=true
194198
logger.save_config_filepath=work/model-verification/gemma-4-26b-a4b-it/sft-post-setup.yaml
195-
last_verified: 2026-08-04
199+
last_verified: 2026-08-11
196200
metrics:
197-
initial_loss: 1.268018
198-
final_loss: 0.06543536
199-
last_10_steps_step_time_ms_avg: 18699.99
200-
last_10_steps_model_tflops_per_gpu_avg: 26.47
201-
peak_allocated_memory_gib: 70.624
202-
peak_reserved_memory_gib: 71.021
201+
initial_loss: 1.282612
202+
final_loss: 0.07524271
203+
last_10_steps_step_time_ms_avg: 15600.68
204+
last_10_steps_model_tflops_per_gpu_avg: 32.95
205+
peak_allocated_memory_gib: 65.223
206+
peak_reserved_memory_gib: 72.356
203207
expected_result: >
204-
On 2026-08-04 the single-node TP4/PP1/EP8/ETP1 run loaded the immutable
208+
On 2026-08-11 the single-node TP4/PP1/EP8/ETP1 run loaded the immutable
205209
BF16 checkpoint and real CORD-v2 revision, then completed exactly 10
206210
optimizer steps at MBS1 and GBS32. The persisted post-setup config
207211
confirmed a frozen vision encoder, trainable projection and language
208-
model, expandable CUDA allocator segments, and selective core_attn plus
209-
moe_act recompute with CUDA graphs and CPU offload disabled. Loss moved
210-
from 1.268018 to 0.06543536, final grad norm was 1.561, and every step
211-
reported zero skipped and zero NaN iterations. Across all eight ranks,
212-
allocated memory returned to 56.493 GiB after each step, while
213-
cumulative peak allocated and reserved memory remained at or below
214-
70.624 and 71.021 GiB. The run saved a complete iter_0000010 torch_dist
215-
checkpoint with eight rank shards, finalized metadata, training state,
216-
and a latest-checkpoint tracker containing iteration 10.
212+
model, expandable CUDA allocator segments, and full-layer uniform
213+
recompute with one layer per recompute unit while CUDA graphs and CPU
214+
offload remained disabled. Loss moved from 1.282612 to 0.07524271,
215+
final grad norm was 2.094, and every step reported zero skipped and zero
216+
NaN iterations. After iteration 2, logger-reported ranks returned to
217+
62.134-62.145 GiB allocated, while cumulative peak allocated and
218+
reserved memory remained at or below 65.223 and 72.356 GiB. The run
219+
saved a complete iter_0000010 torch_dist checkpoint with eight rank
220+
shards, finalized metadata, run config, training state, and a
221+
latest-checkpoint tracker containing iteration 10.
217222
218223
sft_export_inference:
219224
H100:

0 commit comments

Comments
 (0)