Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/model_support_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ body:
description: Pick the closest fit for the requested model.
options:
- Dense decoder-only (e.g. Llama, Qwen, Gemma, Mistral, Nemotron)
- MoE decoder-only (e.g. DeepSeek V2/V3, OLMoE, Qwen3-MoE, MiniMax-M2)
- MoE decoder-only (e.g. DeepSeek V3, OLMoE, Qwen3-MoE, MiniMax-M2)
- VLM / multimodal (e.g. Qwen-VL, Gemma-VL, Nemotron-VL)
- SSM / hybrid (e.g. Mamba, Nemotron-H)
- SSM / hybrid (e.g. Mamba, Nemotron 3)
- Encoder-decoder (e.g. T5)
- Other / unsure
validations:
Expand Down
16 changes: 5 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,34 +221,28 @@ For a deeper dive into conversion design and advanced usage, see the [models REA

Megatron Bridge provides out-of-the-box bridges and training recipes for a wide range of models, built on top of base model architectures from [Megatron Core](https://github.com/NVIDIA/Megatron-LM/tree/main/megatron/core). Refer to the [models directory](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models) for the full list of model bridges.

> **Deprecation notice:** Gemma 1 (2B/7B), Gemma 2 (2B/9B/27B), Llama 2,
> Mistral 7B, Mistral Small 3 24B, Nemotron H v1 (4B/8B/47B/56B), and the
> legacy `NemotronForCausalLM` bridge previously documented for Nemotron-4 340B
> are no longer actively maintained or tested against current upstream
> checkpoints. They will be removed in Megatron Bridge 0.7.0.

| Family | Supported variants |
|----------------|--------------------|
| [**BAGEL**](docs/models/bagel/bagel.md) | BAGEL-7B-MoT (requires the tested MCore `dev` revision and `bagel` extra) |
| [**Bailing**](docs/models/bailing/index.md) | Ling 2.0 / Ling MoE V2 (Bailing) |
| [**BERT**](docs/models/bert/index.md) | Megatron-Style BERT (`MegatronBertForMaskedLM`) |
| [**DeepSeek**](docs/models/deepseek/index.md) | DeepSeek V2 / V2 Lite (deprecated), DeepSeek V3, DeepSeek V4 / V4 Flash |
| [**DeepSeek**](docs/models/deepseek/index.md) | DeepSeek V3, DeepSeek V4 / V4 Flash |
| [**Diffusion**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/diffusion/models) | FLUX, LLaDA 1.5, Nemotron-Labs Diffusion, WAN |
| **Ernie** | [Ernie 4.5 MoE](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/ernie), [Ernie 4.5 VL MoE](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/ernie_vl) |
| [**EXAONE**](docs/models/exaone/exaone.md) | EXAONE 4.0, EXAONE 4.5 VL, K-EXAONE MoE |
| [**Falcon**](docs/models/falcon/index.md) | Falcon H1 |
| [**Gemma**](docs/models/gemma/index.md) | Gemma / Gemma 2 (deprecated), Gemma 3, Gemma 3-VL, Gemma 4 (26B-A4B MoE / 31B dense), Gemma 4-VL (26B-A4B MoE) |
| [**Gemma**](docs/models/gemma/index.md) | Gemma 3, Gemma 3-VL, Gemma 4 (26B-A4B MoE / 31B dense), Gemma 4-VL (26B-A4B MoE) |
| [**GLM**](docs/models/glm/index.md) | GLM-4.5 / GLM-4.7 / GLM-4.7-Flash, GLM-4.5V, GLM-5 / GLM-5.1 / GLM-5.2 / [GLM-5.3](docs/models/glm/glm5-2.md) |
| [**GPT-OSS**](docs/models/gpt_oss/index.md) | GPT-oss |
| [**HY V3**](https://huggingface.co/tencent/Hy3-preview-Base) | Hy3 preview-Base (HF → Megatron checkpoint conversion) |
| [**Kimi**](docs/models/kimi/index.md) | Kimi K2, Kimi-K2.5-VL, Kimi K3 |
| [**Llama**](docs/models/llama/index.md) | Llama 2 (deprecated), Llama 3 / 3.1 / 3.2 / 3.3 |
| [**Llama**](docs/models/llama/index.md) | Llama 3 / 3.1 / 3.2 / 3.3 |
| [**MiniMax**](docs/models/minimax/index.md) | MiniMax-M2 / M2.5 / M2.7, MiniMax-M3 |
| [**Mistral**](docs/models/mistral/index.md) | Mistral 7B / Small 3 24B (deprecated), Ministral 3 (3B/8B/14B) |
| [**Mistral**](docs/models/mistral/index.md) | Ministral 3 (3B/8B/14B) |
| [**Xiaomi-MiMo**](docs/models/mimo/index.md) | Xiaomi-MiMo, MiMo-V2-Flash |
| [**Moonlight**](docs/models/moonlight/index.md) | Moonlight |
| [**Muse Glimmer**](docs/models/muse_glimmer/muse-glimmer.md) | Muse Glimmer 30B |
| [**Nemotron**](docs/models/nemotron/index.md) | Nemotron H v1 (deprecated), Nemotron Nano v2 (deprecated), Nemotron-3 Nano, Nemotron-3 Super, Nemotron-3 Ultra, Llama Nemotron (deprecated), Nemotron Nano v2 VL (deprecated), Nemotron-3 Nano Omni |
| [**Nemotron**](docs/models/nemotron/index.md) | Nemotron-3 Nano, Nemotron-3 Super, Nemotron-3 Ultra, Nemotron-3 Nano Omni |
| [**OLMoE**](docs/models/olmoe/index.md) | OLMoE |
| [**Qwen**](docs/models/qwen/index.md) | Qwen2 / Qwen2.5, Qwen3, Qwen3-MoE, Qwen3 Next, Qwen3.5 (dense/MoE), Qwen2.5-VL, Qwen3-VL, Qwen3.5-VL, Qwen3.6-VL, Qwen2 Audio, Qwen2.5-Omni, Qwen3-Omni, Qwen3-ASR |
| [**Sarvam**](docs/models/sarvam/index.md) | Sarvam |
Expand Down
2 changes: 1 addition & 1 deletion docs/bridge-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ from megatron.bridge import AutoBridge

# Load with specific settings
bridge = AutoBridge.from_hf_pretrained(
"meta-llama/Llama-2-7b-hf",
"meta-llama/Meta-Llama-3-8B",
torch_dtype=torch.float16,
device_map="auto",
trust_remote_code=True,
Expand Down
10 changes: 1 addition & 9 deletions docs/fern/versions/_nav_order.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,15 @@
"./versions/nightly/pages/modelopt/quantization.mdx": 1
"./versions/nightly/pages/models/README.mdx": 1
"./versions/nightly/pages/models/bailing/ling-2.mdx": 1
"./versions/nightly/pages/models/deepseek/deepseek-v2.mdx": 1
"./versions/nightly/pages/models/falcon/falcon-h1.mdx": 1
"./versions/nightly/pages/models/gemma/gemma.mdx": 1
"./versions/nightly/pages/models/glm/glm45.mdx": 1
"./versions/nightly/pages/models/gpt_oss/gpt-oss-20b.mdx": 1
"./versions/nightly/pages/models/gpt_oss/gpt-oss-120b.mdx": 2
"./versions/nightly/pages/models/kimi/kimi-k2.mdx": 1
"./versions/nightly/pages/models/llama/llama2.mdx": 1
"./versions/nightly/pages/models/mimo/mimo.mdx": 1
"./versions/nightly/pages/models/minimax/minimax-m2.mdx": 1
"./versions/nightly/pages/models/minimax/minimax-m3.mdx": 2
"./versions/nightly/pages/models/mistral/mistral.mdx": 1
"./versions/nightly/pages/models/moonlight/moonlight.mdx": 1
"./versions/nightly/pages/models/nemotron/llama-nemotron.mdx": 1
"./versions/nightly/pages/models/olmoe/olmoe.mdx": 1
"./versions/nightly/pages/models/qwen/qwen3-235b-a22b.mdx": 1
"./versions/nightly/pages/models/qwen/qwen3-30b-a3b.mdx": 2
Expand All @@ -37,12 +32,10 @@
"./versions/nightly/pages/modelopt/distillation.mdx": 2
"./versions/nightly/pages/models/bailing/index.mdx": 2
"./versions/nightly/pages/models/deepseek/deepseek-v3.mdx": 2
"./versions/nightly/pages/models/gemma/gemma2.mdx": 2
"./versions/nightly/pages/models/glm/glm-45v.mdx": 2
"./versions/nightly/pages/models/kimi/kimi-k25-vl.mdx": 2
"./versions/nightly/pages/models/llama/llama3.mdx": 2
"./versions/nightly/pages/models/mistral/ministral3.mdx": 2
"./versions/nightly/pages/models/nemotron/nemotronh.mdx": 2
"./versions/nightly/pages/models/qwen/qwen3-moe.mdx": 4
"./versions/nightly/pages/performance-guide.mdx": 2
"./versions/nightly/pages/performance-summary-archive.mdx": 2
Expand Down Expand Up @@ -75,8 +68,7 @@
"./versions/nightly/pages/megatron-lm-to-megatron-bridge.mdx": 5
"./versions/nightly/pages/models/gemma/gemma4-vl.mdx": 5
"./versions/nightly/pages/models/gemma/index.mdx": 5
"./versions/nightly/pages/models/nemotron/nemotron3.5-lightning.mdx": 8
"./versions/nightly/pages/models/nemotron/nemotron-nano-v2-vl.mdx": 11
"./versions/nightly/pages/models/nemotron/nemotron3.5-lightning.mdx": 6
"./versions/nightly/pages/models/qwen/qwen3-vl.mdx": 7
"./versions/nightly/pages/releases/known-issues.mdx": 5
"./versions/nightly/pages/training/optimizer-scheduler.mdx": 5
Expand Down
16 changes: 0 additions & 16 deletions docs/fern/versions/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,6 @@ navigation:
path: ./nightly/pages/models/bailing/ling-2.mdx
section: Bailing
- contents:
- page: DeepSeek V2
path: ./nightly/pages/models/deepseek/deepseek-v2.mdx
- page: DeepSeek V3
path: ./nightly/pages/models/deepseek/deepseek-v3.mdx
- page: DeepSeek V4
Expand All @@ -58,10 +56,6 @@ navigation:
path: ./nightly/pages/models/falcon/falcon-h1.mdx
section: Falcon
- contents:
- page: Gemma
path: ./nightly/pages/models/gemma/gemma.mdx
- page: Gemma 2
path: ./nightly/pages/models/gemma/gemma2.mdx
- page: Gemma 3
path: ./nightly/pages/models/gemma/gemma3.mdx
- page: Gemma 3 VL (Vision-Language)
Expand Down Expand Up @@ -96,8 +90,6 @@ navigation:
path: ./nightly/pages/models/kimi/kimi-k3.mdx
section: Kimi
- contents:
- page: Llama 2
path: ./nightly/pages/models/llama/llama2.mdx
- page: Llama 3
path: ./nightly/pages/models/llama/llama3.mdx
section: Llama
Expand All @@ -108,8 +100,6 @@ navigation:
path: ./nightly/pages/models/minimax/minimax-m3.mdx
section: MiniMax
- contents:
- page: Mistral
path: ./nightly/pages/models/mistral/mistral.mdx
- page: Ministral 3
path: ./nightly/pages/models/mistral/ministral3.mdx
section: Mistral
Expand All @@ -122,10 +112,6 @@ navigation:
path: ./nightly/pages/models/moonlight/moonlight.mdx
section: Moonlight
- contents:
- page: Llama Nemotron
path: ./nightly/pages/models/nemotron/llama-nemotron.mdx
- page: Nemotron H and Nemotron Nano v2
path: ./nightly/pages/models/nemotron/nemotronh.mdx
- page: Nemotron 3 Nano 30B-A3B
path: ./nightly/pages/models/nemotron/nemotron3-nano.mdx
- page: Nemotron 3 Nano 4B
Expand All @@ -139,8 +125,6 @@ navigation:
- page: Nemotron 3.5 Lightning
path: ./nightly/pages/models/nemotron/nemotron3.5-lightning.mdx
# Super VL guides remain in GitHub, but are excluded from publication.
- page: Nemotron Nano V2 VL
path: ./nightly/pages/models/nemotron/nemotron-nano-v2-vl.mdx
section: Nemotron
- contents:
- page: OLMoE
Expand Down
2 changes: 1 addition & 1 deletion docs/fern/versions/nightly/pages/bridge-guide.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ from megatron.bridge import AutoBridge

# Load with specific settings
bridge = AutoBridge.from_hf_pretrained(
"meta-llama/Llama-2-7b-hf",
"meta-llama/Meta-Llama-3-8B",
torch_dtype=torch.float16,
device_map="auto",
trust_remote_code=True,
Expand Down
10 changes: 5 additions & 5 deletions docs/fern/versions/nightly/pages/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -206,18 +206,18 @@ Megatron Bridge provides out-of-the-box bridges and training recipes for a wide
| Family | Supported variants |
|----------------|--------------------|
| [**Bailing**](docs/models/bailing/index.md) | Ling 2.0 (Bailing) |
| [**DeepSeek**](docs/models/deepseek/index.md) | DeepSeek V2 / V2 Lite, DeepSeek V3, DeepSeek V4 |
| [**DeepSeek**](docs/models/deepseek/index.md) | DeepSeek V3, DeepSeek V4 |
| [**Falcon**](docs/models/falcon/index.md) | Falcon H1 |
| [**Gemma**](docs/models/gemma/index.md) | Gemma / Gemma 2, Gemma 3, Gemma 3-VL, Gemma 4-VL (26B-A4B MoE) |
| [**Gemma**](docs/models/gemma/index.md) | Gemma 3, Gemma 3-VL, Gemma 4-VL (26B-A4B MoE) |
| [**GLM**](docs/models/glm/index.md) | GLM-4.5 / 4.7 / 4.7-Flash, GLM-4.5V, GLM-5 / 5.1 |
| [**GPT-OSS**](docs/models/gpt_oss/index.md) | GPT-oss |
| [**Kimi**](docs/models/kimi/index.md) | Kimi K2, Kimi-K2.5-VL |
| [**Llama**](docs/models/llama/index.md) | Llama 2, Llama 3 / 3.1 / 3.2 / 3.3 |
| [**Llama**](docs/models/llama/index.md) | Llama 3 / 3.1 / 3.2 / 3.3 |
| [**MiniMax**](docs/models/minimax/index.md) | MiniMax-M2 / M2.5 / M2.7 |
| [**Mistral**](docs/models/mistral/index.md) | Mistral, Ministral 3 (3B/8B/14B) |
| [**Mistral**](docs/models/mistral/index.md) | Ministral 3 (3B/8B/14B) |
| [**Xiaomi-MiMo**](docs/models/mimo/index.md) | Xiaomi-MiMo |
| [**Moonlight**](docs/models/moonlight/index.md) | Moonlight |
| [**Nemotron**](docs/models/nemotron/index.md) | Nemotron H, Nemotron Nano v2, Nemotron-3 Nano, Nemotron-3 Super, Llama Nemotron, Nemotron Nano v2 VL, Nemotron-3 Nano Omni |
| [**Nemotron**](docs/models/nemotron/index.md) | Nemotron-3 Nano, Nemotron-3 Super, Nemotron-3 Nano Omni |
| [**OLMoE**](docs/models/olmoe/index.md) | OLMoE |
| [**Qwen**](docs/models/qwen/index.md) | Qwen2 / Qwen2.5, Qwen3, Qwen3-MoE, Qwen3 Next, Qwen2.5-VL, Qwen3-VL, Qwen3.5-VL, Qwen3.6-VL, Qwen2 Audio, Qwen2.5-Omni, Qwen3-Omni, Qwen3-ASR |
| [**Sarvam**](docs/models/sarvam/index.md) | Sarvam |
Expand Down
1 change: 0 additions & 1 deletion docs/fern/versions/nightly/pages/modelopt/quantization.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,6 @@ and ModelOpt scale suffixes, so a pattern can target the logical parameter name
| Llama-3.2-1B | ✅ | ✅ |
| Qwen3-8B | ✅ | ✅ |
| Qwen3-30B-A3B | ✅ | ✅ |
| Nemotron-H-8B-Base-8K | ✅ | ✅ |
| Qwen3-VL-8B-Instruct | ✅ | ✅ |
| Qwen3-VL-30B-A3B-Instruct | ✅ | ✅ |

Expand Down
166 changes: 4 additions & 162 deletions docs/fern/versions/nightly/pages/models/deepseek/deepseek-v2.mdx
Original file line number Diff line number Diff line change
@@ -1,165 +1,7 @@
# DeepSeek V2
# DeepSeek V2 support removed

> **Deprecation notice:** DeepSeek V2 and DeepSeek V2 Lite support is no
> longer actively maintained or tested against current upstream checkpoints and
> will be removed in Megatron Bridge 0.7.0.
{/* Keep this notice for previews that use navigation from main. */}

[DeepSeek-V2](https://github.com/deepseek-ai/DeepSeek-V2) is a Mixture-of-Experts (MoE) language model that uses innovative Multi-head Latent Attention (MLA) for efficient inference and DeepSeekMoE architecture for economical training and inference. The model achieves performance comparable to GPT-4 while using significantly fewer activated parameters. More information is available in the companion paper ["DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model"](https://arxiv.org/abs/2405.04434).
Support for DeepSeek V2 and V2 Lite has been removed from Megatron Bridge. The former conversion and training instructions no longer apply.

DeepSeek V2 models are supported via the Bridge system with auto-detected configuration and weight mapping.

## Available Models

Megatron Bridge supports the following DeepSeek V2 model variants:

- **DeepSeek-V2**: 236B parameters (21B activated per token)
- **DeepSeek-V2-Lite**: 16B parameters (2.4B activated per token)

Both models support pretraining with expert parallelism for efficient MoE training.

## Model Architecture Features

- **Multi-head Latent Attention (MLA)**: Novel attention mechanism that reduces KV cache requirements
- **DeepSeekMoE**: Efficient MoE architecture with routed and shared experts
- **Expert Parallelism**: Distributes experts across GPUs for scalable training
- **RoPE Embeddings**: Rotary position embeddings for position encoding
- **128K Context Length**: Native support for long sequences (DeepSeek-V2)
- **Pre-normalization**: RMSNorm before each transformer sub-layer

## Conversion with 🤗 Hugging Face

### Load HF → Megatron

```python
from megatron.bridge import AutoBridge

# Example: DeepSeek-V2-Lite
bridge = AutoBridge.from_hf_pretrained("deepseek-ai/DeepSeek-V2-Lite", trust_remote_code=True)
provider = bridge.to_megatron_provider()

# Optionally configure parallelism before instantiating the model
provider.tensor_model_parallel_size = 1
provider.pipeline_model_parallel_size = 1
provider.expert_model_parallel_size = 8

model = provider.provide_distributed_model(wrap_with_ddp=False)
```

### Import Checkpoint from HF

```bash
./scripts/conversion/convert.sh import \
--hf-model deepseek-ai/DeepSeek-V2-Lite \
--megatron-path /checkpoints/deepseek_v2_lite_megatron \
--trust-remote-code
```

### Export Megatron → HF

```python
from megatron.bridge import AutoBridge

# Load the bridge from HF model ID
bridge = AutoBridge.from_hf_pretrained("deepseek-ai/DeepSeek-V2-Lite", trust_remote_code=True)

# Export a trained Megatron checkpoint to HF format
bridge.export_ckpt(
megatron_path="/results/deepseek_v2_lite/checkpoints/iter_0000500",
hf_path="/exports/deepseek_v2_lite_hf",
)
```

### Run Inference on Converted Checkpoint

```bash
uv run python examples/conversion/hf_to_megatron_generate_text.py \
--hf_model_path deepseek-ai/DeepSeek-V2-Lite \
--megatron_model_path /checkpoints/deepseek_v2_lite_megatron \
--prompt "What is artificial intelligence?" \
--max_new_tokens 100 \
--ep 8 \
--trust-remote-code
```

For more details, see [examples/conversion/hf_to_megatron_generate_text.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/hf_to_megatron_generate_text.py)

## Recipes

See: [bridge.recipes.deepseek.deepseek_v2](../../apidocs/bridge/bridge.recipes.deepseek.deepseek_v2.md)

### Available Recipes

- **Pretrain recipes**:
- `deepseek_v2_lite_pretrain_config`: Pre-training for DeepSeek-V2-Lite (16B parameters, 2.4B activated per token)
- `deepseek_v2_pretrain_config`: Pre-training for DeepSeek-V2 (236B parameters, 21B activated per token)

### Parallelism Configurations

| Model | TP | PP | EP | Total GPUs | Use Case |
|-------|----|----|----|-----------:|----------|
| **DeepSeek-V2-Lite** | 1 | 1 | 8 | 8 | Pre-training (single node) |
| **DeepSeek-V2** | 1 | 4 | 32 | 128 | Pre-training (16 nodes) |

**Key Features**:
- **Expert Parallelism**: EP=8 (V2-Lite) or EP=32 (V2) for efficient MoE training
- **Selective Recomputation**: Enabled by default for memory optimization
- **Sequence Length**: Default 4096, V2 supports up to 128K tokens

### Pre-training Examples

#### DeepSeek-V2-Lite (16B)

```python
from megatron.bridge.recipes.deepseek import deepseek_v2_lite_pretrain_config

config = deepseek_v2_lite_pretrain_config()
config.dataset.blend = [["/path/to/dataset_text_document"], None]
config.checkpoint.save = "/results/deepseek_v2_lite/checkpoints"
config.checkpoint.load = "/results/deepseek_v2_lite/checkpoints"
config.logger.tensorboard_dir = "/results/deepseek_v2_lite/tb_logs"
config.train.train_iters = 500_000
config.train.global_batch_size = 512
config.model.seq_length = 4096
config.dataset.seq_length = 4096
# Uses TP=1, PP=1, EP=8 (8 GPUs) automatically
```

#### DeepSeek-V2 (236B)

```python
from megatron.bridge.recipes.deepseek import deepseek_v2_pretrain_config

config = deepseek_v2_pretrain_config()
config.dataset.blend = [["/path/to/dataset_text_document"], None]
config.checkpoint.save = "/results/deepseek_v2/checkpoints"
config.checkpoint.load = "/results/deepseek_v2/checkpoints"
config.logger.tensorboard_dir = "/results/deepseek_v2/tb_logs"
config.train.train_iters = 500_000
config.train.global_batch_size = 512
config.model.seq_length = 4096
config.dataset.seq_length = 4096
# Uses TP=1, PP=4, EP=32 (128 GPUs) automatically
```

### Finetuning Recipes

Finetuning recipes for DeepSeek V2 models are not currently available.

## Hugging Face Model Cards & References

### Hugging Face Model Cards
- DeepSeek-V2: https://huggingface.co/deepseek-ai/DeepSeek-V2
- DeepSeek-V2-Lite: https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite
- DeepSeek-V2-Chat: https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat
- DeepSeek-V2-Lite-Chat: https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat

### Technical Papers
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model: [arXiv:2405.04434](https://arxiv.org/abs/2405.04434)

### Additional Resources
- GitHub Repository: https://github.com/deepseek-ai/DeepSeek-V2

## Related Docs
- Recipe usage: [Recipe usage](../../recipe-usage.md)
- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md)
- Training entry points: [Entry points](../../training/entry-points.md)
See the [family overview](index.mdx) for supported models.
1 change: 0 additions & 1 deletion docs/fern/versions/nightly/pages/models/deepseek/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,5 @@ DeepSeek model documentation is organized by model variant.

| Variant | Guide |
|---------|-------|
| DeepSeek V2 | [deepseek-v2.md](deepseek-v2.md) |
| DeepSeek V3 | [deepseek-v3.md](deepseek-v3.md) |
| DeepSeek V4 | [deepseek-v4.md](deepseek-v4.md) |
Loading
Loading