1- # Hybrid / Hierarchical Context Parallel
1+ # Hierarchical Context Parallel
22
3- This page covers the stable Bridge-facing meaning of hierarchical context
4- parallelism, especially the ` a2a+p2p ` transport path and
3+ This page covers hierarchical context parallelism in Megatron Bridge,
4+ especially the ` a2a+p2p ` transport path and
55` hierarchical_context_parallel_sizes ` .
66
77For operational setup, code anchors, and verification commands, see
8- [ skills/perf-hybrid-context-parallel/SKILL.md ] ( ../skills/perf-hybrid-context-parallel/SKILL.md ) .
8+ [ the hierarchical CP skill ] ( ../skills/perf-hybrid-context-parallel/SKILL.md ) .
99
1010## What It Is
1111
@@ -24,11 +24,6 @@ several CP communication backends:
2424a2a (fast, head-parallel) on intra-node links with p2p (async,
2525sequence-parallel) on inter-node links.
2626
27- It is important to separate this from the upstream boolean
28- ` hybrid_context_parallel ` , which is a different feature for balancing packed or
29- variable-length workloads. The two concepts should not be treated as
30- interchangeable.
31-
3227### Why a2a is limited by KV heads
3328
3429a2a transposes the parallelism dimension: each rank trades its sequence chunk
@@ -109,5 +104,5 @@ all Bridge initialization modes.
109104
110105- [ docs/performance-guide.md] ( ../performance-guide.md )
111106- [ docs/training/communication-overlap.md] ( communication-overlap.md )
112- - [ skills/perf-hybrid-context-parallel/SKILL.md ] ( ../skills/perf-hybrid-context-parallel/SKILL.md )
113- - [ skills/perf-hybrid-context-parallel/ card.yaml ] ( ../skills/perf-hybrid-context-parallel/card.yaml )
107+ - [ Hierarchical CP skill ] ( ../skills/perf-hybrid-context-parallel/SKILL.md )
108+ - [ Hierarchical CP skill card] ( ../skills/perf-hybrid-context-parallel/card.yaml )
0 commit comments