Skip to content

Commit c57847e

Browse files
committed
docs(training): drop the phased plan and open-questions sections
The design page carried a four-phase implementation plan and an open-questions list. Neither describes what this PR (or the repo) actually contains: they promise work that is not scheduled and has no owner, so they go stale the moment priorities move. The sequencing and the open decisions belong in the tracking issue (#5230), which is where they can be updated. The remaining page is the design itself: the algorithm, where it fits in Megatron-Bridge, and the proposed file surface. The frozen-target supervision section keeps both capture options, since the trade-off between them is design content rather than a schedule. Signed-off-by: khazic <khazzz1c@gmail.com>
1 parent a46c068 commit c57847e

1 file changed

Lines changed: 3 additions & 33 deletions

File tree

‎docs/training/dspark-speculative-decoding.md‎

Lines changed: 3 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@ budget.
1717

1818
A working draft-training reference already exists in NeMo AutoModel at
1919
`nemo_automodel/components/speculative/dspark/`. This document specifies how the
20-
same training recipe maps onto Megatron-Bridge and Megatron-Core, and proposes a
21-
phased plan. It intentionally does **not** ship the draft module itself: that is a
20+
same training recipe maps onto Megatron-Bridge and Megatron-Core. It
21+
intentionally does **not** ship the draft module itself: that is a
2222
Megatron-Core (or ModelOpt) neural module whose parallel/loss integration must be
2323
validated on multi-GPU runs, so it belongs in later, separately reviewed PRs.
2424

@@ -155,7 +155,7 @@ The draft trains against features captured from the frozen target: the selected
155155
decoder-layer hidden states (fused by `fc` into the draft's context K/V) and the
156156
final hidden state (which drives the teacher distribution through the shared LM
157157
head). In Megatron-Core, this capture must respect tensor, pipeline, and context
158-
parallelism. Two options, to be decided in Phase 1:
158+
parallelism. Two options carry different trade-offs:
159159

160160
- Colocated capture during the same forward (analogous to how MTP consumes extra
161161
token IDs), keeping the target and draft in one model.
@@ -172,36 +172,6 @@ backbone, plain linears for `fc` and the heads), so trained drafts round-trip
172172
through `AutoBridge` for serving. The `mtp.*` mappings are the closest existing
173173
template.
174174

175-
## Phased implementation plan
176-
177-
- **Phase 0 (this PR): design.** Specify the algorithm, the Bridge mapping, and
178-
the plan.
179-
- **Phase 1: the DSpark draft module.** Implement the semi-autoregressive backbone
180-
plus Markov and confidence heads as a parallel-aware module (Megatron-Core
181-
native, or a ModelOpt speculative module), with the three-term loss. Validate
182-
numerically with L0 unit tests and L1/L2 functional tests within the 2-GPU cap.
183-
This is the load-bearing, separately reviewed piece.
184-
- **Phase 2: Bridge integration.** Add `dspark_provider.py`, `forward_step_dspark`,
185-
`post_training/dspark.py`, the `training/dspark.py` entry, and the `dspark.*`
186-
weight mappings, following the distillation files.
187-
- **Phase 3: recipe, example, and convergence.** Add a recipe/config and an
188-
example, and validate acceptance-length (`tau`) convergence against the AutoModel
189-
reference on a small target (for example Qwen3 dense) on multi-GPU.
190-
191-
## Open questions and risks
192-
193-
- **Module placement.** MCore-native (a Megatron-LM PR, full TP/PP/EP/CP support)
194-
versus ModelOpt-injected (Bridge-only, but tied to ModelOpt). Phase 1 must
195-
decide this first; it drives everything downstream.
196-
- **Parallelism.** The draft heads and the block-attention mask need correct
197-
tensor/pipeline/expert/context-parallel behavior; this is the part that cannot
198-
be validated single-GPU and must go through functional tests.
199-
- **Optional dependency.** If the draft leans on `modelopt.torch.speculative`, it
200-
must be an optional extra and land as a separate dependency PR first, per
201-
`CONTRIBUTING.md`.
202-
- **Target-feature capture cost.** Colocated capture holds the target in memory;
203-
offline precompute adds a cache format. The choice affects the recipe surface.
204-
205175
## References
206176

207177
- DSpark paper: [arXiv:2607.05147](https://arxiv.org/abs/2607.05147),

0 commit comments

Comments
 (0)