Repository navigation

feat(training): add DPO training with offline-scored reference model - #6092

Merged

cuichenx merged 42 commits into

NVIDIA-NeMo:mainfrom

Josephasafg:dpo_support

Oct 14, 2026

Merged

feat(training): add DPO training with offline-scored reference model#6092
cuichenx merged 42 commits into
NVIDIA-NeMo:mainfrom
Josephasafg:dpo_support

Commits on Sep 8, 2026
- feat(data): allow the chat completion span to start at the assistant turn
  Josephasafg
  committed
  4ad19fc
- feat(data): expose the batch sampler shuffle flag on build_pretraining_data_loader
  Josephasafg
  committed
  05f4c6b
Commits on Sep 9, 2026
Commits on Sep 10, 2026
- feat(training): add the DPO training entry point and config fields
  Josephasafg
  committed
  90ff292
- fix: Remove docstring
  Josephasafg
  committed
  62169c3
- test: Added test for test_config.py
  Josephasafg
  committed
  b5add3e
- feat(data): add the offline reference-logprob scorer
  Josephasafg
  committed
  ecc2e11
- feat(recipe): Added recipes entry point and qwen3-30b recipe
  Josephasafg
  committed
  93fb6cc
- test: Added recipe test for qwen dpo
  Josephasafg
  committed
  499fbfa
Commits on Sep 12, 2026
- fix: Small bugs
  Josephasafg
  committed
  e491eac
Commits on Sep 13, 2026
- refactor(data): rename the preference dataset class
  Josephasafg
  committed
  464657e
- refactor(data): rename the JSONL source config
  Josephasafg
  committed
  07cb850
- test(training): fingerprint JSONL paths the way the loader routes them
  Josephasafg
  committed
  e673668
- feat(training): support pipeline parallelism for DPO
  Josephasafg
  committed
  ce08cf6
- feat(data): shard reference scoring accros dp ranks
  Josephasafg
  committed
  0a172ea
- perf(recipe): move the Qwen3-30B-A3B DPO recipe to tp4/pp2
  Josephasafg
  committed
  1528439
Commits on Sep 14, 2026
- docs(training): trim the DPO README
  Josephasafg
  committed
  a9221b1
- test(training): minimized amount of tests
  Josephasafg
  committed
  69ac318
Commits on Sep 15, 2026
- refactor(training): rename dpo_forward_step
  Josephasafg
  committed
  bd7b0d4
- docs(training): cleanup
  Josephasafg
  committed
  cae2199
Commits on Sep 16, 2026
- Merge branch 'main' into dpo_support
  Josephasafg
  authored
  d9d5158
Commits on Sep 17, 2026
- Merge branch 'main' into dpo_support
  Josephasafg
  authored
  4daa65d
Commits on Sep 19, 2026
- Merge branch 'main' into dpo_support
  Josephasafg
  authored
  d6b405c
Commits on Sep 22, 2026
- Merge branch 'main' into dpo_support
  Josephasafg
  authored
  03676e7
- Merge branch 'main' into dpo_support
  Josephasafg
  authored
  651609b
Commits on Oct 5, 2026
Commits on Oct 11, 2026
- Merge remote-tracking branch 'upstream/main' into dpo_support
  Josephasafg
  committed
  28dc38c
- Merge branch 'main' into dpo_support
  Josephasafg
  authored
  368659c
Commits on Oct 12, 2026

Commits

Commits on Sep 8, 2026

Commits on Sep 9, 2026

Commits on Sep 10, 2026

Commits on Sep 12, 2026

Commits on Sep 13, 2026

Commits on Sep 14, 2026

Commits on Sep 15, 2026

Commits on Sep 16, 2026

Commits on Sep 17, 2026

Commits on Sep 19, 2026

Commits on Sep 22, 2026

Commits on Oct 5, 2026

Commits on Oct 11, 2026

Commits on Oct 12, 2026