@@ -9,9 +9,9 @@ summary: >
99 GPT-OSS 120B H100 GPU HF-to-Megatron import, GPU Megatron-to-HF export,
1010 manual HF/Megatron forward-pass correlation, deterministic Megatron
1111 inference, bounded H100 pretraining, H100 SFT, H100 PEFT, and H100
12- checkpoint resume, and H100 SFT export/inference are verified. CPU
13- conversion, long-context SFT, and performance verification remain planned
14- but not yet verified.
12+ checkpoint resume, H100 SFT export/inference, and H100 long-context SFT are
13+ verified. CPU conversion and performance verification remain planned but not
14+ yet verified.
1515verification_index :
1616 model_level :
1717 verified :
@@ -24,8 +24,7 @@ verification_index:
2424 - megatron_to_hf_cpu
2525 training :
2626 H100 :
27- verified : [pretrain, sft, sft_export_inference, peft, checkpoint_resume]
28- unverified : [sft_long_context]
27+ verified : [pretrain, sft, sft_export_inference, sft_long_context, peft, checkpoint_resume]
2928 performance :
3029 GB200 : unverified
3130model :
@@ -276,23 +275,47 @@ items:
276275
277276 sft_long_context :
278277 H100 :
279- status : unverified
278+ status : verified
280279 precision : bf16
281- enabled_features : {}
282- command : null
283- last_verified : null
280+ enabled_features :
281+ sequence_packing : offline
282+ context_parallel_size : 4
283+ command : >
284+ ./scripts/training/train.sh --nodes 6 --gpus-per-node 8
285+ --recipe gpt_oss_120b_sft_48gpu_h100_bf16_32k_config --mode sft
286+ --dataset tulu3
287+ --pretrained_checkpoint work/model-verification/gpt-oss-120b/imported-megatron/iter_0000000
288+ --max_steps 20 --seq_length 32768
289+ --lr 1e-6 --min_lr 0 --warmup_iters 2
290+ 'dataset.hf_dataset.split="train[:10000]"'
291+ 'dataset.hf_dataset.load_kwargs={revision:"b14afda60f1bbebe55d5d2fa1e4df5042f97f8be"}'
292+ '++tokenizer.hf_tokenizer_kwargs.revision="b5c939de8f754692c1647ca79fbf85e8c1e70f8a"'
293+ dataset.hf_output_root=work/data/tulu3/gpt-oss-120b-sft-long-context-32k-cp4-pp6-nofuse-gbs4-b14afda60f1b
294+ dataset.hf_rewrite=true dataset.seed=1234 rng.seed=5678
295+ dataset.do_validation=false dataset.hf_validation_proportion=null
296+ dataset.enable_offline_packing=true
297+ +dataset.offline_packing_specs.pad_seq_to_mult=8
298+ scheduler.lr_decay_iters=20 validation.eval_iters=0 validation.eval_interval=0
299+ checkpoint.load=null checkpoint.save=null
300+ logger.log_interval=1 logger.log_throughput=true
301+ logger.tensorboard_dir=null
302+ last_verified : 2026-08-13
284303 metrics :
285- initial_loss : null
286- final_loss : null
287- last_10_steps_step_time_ms_avg : null
288- last_10_steps_model_tflops_per_gpu_avg : null
304+ initial_loss : 6.666666
305+ final_loss : 1.238030
306+ last_10_steps_step_time_ms_avg : 24388.780
307+ last_10_steps_model_tflops_per_gpu_avg : 5.070
289308 expected_result : >
290- A model-appropriate long-context topology and memory plan must be
291- resolved before a public command is recorded. Future verification must
292- exercise sequence packing with context parallelism greater than one,
293- use the topology-derived packing alignment, complete at least 10 steps
294- with finite losses and zero skipped or NaN iterations, and record all
295- required metrics.
309+ Completed 20 long-context full-SFT optimizer steps from the imported
310+ checkpoint on 48x H100 with 32K sequence length, immutable Tulu3
311+ selection, assistant-token masking, offline packing, TP2/SP, CP4 with
312+ a2a context-parallel communication, PP6, EP4, disabled cross-entropy
313+ and bias-activation fusions, and selective MoE-activation recompute.
314+ All 20 optimizer-step rows are present exactly once with finite losses,
315+ zero skipped iterations, and zero NaN iterations. Loss decreases from
316+ 6.666666 to 1.238030, all four metrics are recorded, and no checkpoint
317+ is saved because this bounded long-context verification run disables
318+ checkpoint output.
296319
297320 peft :
298321 H100 :
0 commit comments