Skip to content

Commit a4047be

Browse files
khazicyaoyu-33
authored andcommitted
docs(training): drop the phased plan and open-questions sections
The design page carried a four-phase implementation plan and an open-questions list. Neither describes what this PR (or the repo) actually contains: they promise work that is not scheduled and has no owner, so they go stale the moment priorities move. The sequencing and the open decisions belong in the tracking issue (#5230), which is where they can be updated. The remaining page is the design itself: the algorithm, where it fits in Megatron-Bridge, and the proposed file surface. The frozen-target supervision section keeps both capture options, since the trade-off between them is design content rather than a schedule. Signed-off-by: khazic <khazzz1c@gmail.com>
1 parent 93192ca commit a4047be

1 file changed

Lines changed: 3 additions & 33 deletions

File tree

‎docs/training/dspark-speculative-decoding.md‎

Lines changed: 3 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@ budget.
1717

1818
A working draft-training reference already exists in NeMo AutoModel at
1919
`nemo_automodel/components/speculative/dspark/`. This document specifies how the
20-
same training recipe maps onto Megatron-Bridge and Megatron-Core, and proposes a
21-
phased plan. It intentionally does **not** ship the draft module itself: that is a
20+
same training recipe maps onto Megatron-Bridge and Megatron-Core. It
21+
intentionally does **not** ship the draft module itself: that is a
2222
Megatron-Core (or ModelOpt) neural module whose parallel/loss integration must be
2323
validated on multi-GPU runs, so it belongs in later, separately reviewed PRs.
2424

@@ -140,7 +140,7 @@ The draft trains against features captured from the frozen target: the selected
140140
decoder-layer hidden states (fused by `fc` into the draft's context K/V) and the
141141
final hidden state (which drives the teacher distribution through the shared LM
142142
head). In Megatron-Core, this capture must respect tensor, pipeline, and context
143-
parallelism. Two options, to be decided in Phase 1:
143+
parallelism. Two options carry different trade-offs:
144144

145145
- Colocated capture during the same forward (analogous to how MTP consumes extra
146146
token IDs), keeping the target and draft in one model.
@@ -157,36 +157,6 @@ backbone, plain linears for `fc` and the heads), so trained drafts round-trip
157157
through `AutoBridge` for serving. The `mtp.*` mappings are the closest existing
158158
template.
159159

160-
## Phased implementation plan
161-
162-
- **Phase 0 (this PR): design.** Specify the algorithm, the Bridge mapping, and
163-
the plan.
164-
- **Phase 1: the DSpark draft module.** Implement the semi-autoregressive backbone
165-
plus Markov and confidence heads as a parallel-aware module (Megatron-Core
166-
native, or a ModelOpt speculative module), with the three-term loss. Validate
167-
numerically with L0 unit tests and L1/L2 functional tests within the 2-GPU cap.
168-
This is the load-bearing, separately reviewed piece.
169-
- **Phase 2: Bridge integration.** Add `dspark_provider.py`, `forward_step_dspark`,
170-
`post_training/dspark.py`, the `training/dspark.py` entry, and the `dspark.*`
171-
weight mappings, following the distillation files.
172-
- **Phase 3: recipe, example, and convergence.** Add a recipe/config and an
173-
example, and validate acceptance-length (`tau`) convergence against the AutoModel
174-
reference on a small target (for example Qwen3 dense) on multi-GPU.
175-
176-
## Open questions and risks
177-
178-
- **Module placement.** MCore-native (a Megatron-LM PR, full TP/PP/EP/CP support)
179-
versus ModelOpt-injected (Bridge-only, but tied to ModelOpt). Phase 1 must
180-
decide this first; it drives everything downstream.
181-
- **Parallelism.** The draft heads and the block-attention mask need correct
182-
tensor/pipeline/expert/context-parallel behavior; this is the part that cannot
183-
be validated single-GPU and must go through functional tests.
184-
- **Optional dependency.** If the draft leans on `modelopt.torch.speculative`, it
185-
must be an optional extra and land as a separate dependency PR first, per
186-
`CONTRIBUTING.md`.
187-
- **Target-feature capture cost.** Colocated capture holds the target in memory;
188-
offline precompute adds a cache format. The choice affects the recipe surface.
189-
190160
## References
191161

192162
- DSpark paper: [arXiv:2607.05147](https://arxiv.org/abs/2607.05147),

0 commit comments

Comments
 (0)