Skip to content

ling-2.md

Latest commit

 

History

History
28 lines (18 loc) · 1.53 KB

ling-2.md

File metadata and controls

28 lines (18 loc) · 1.53 KB
Raw

Ling 2.0

Ling 2.0, documented in this repository under the Bailing family, is a high-sparsity Mixture-of-Experts language model family from inclusionAI. Megatron Bridge supports the Ling 2.0 MoE checkpoints through the Bailing bridge with auto-detected Hugging Face configuration and weight mapping.

Supported Variants

Variant Hugging Face ID Notes
Ling-flash-2.0 inclusionAI/Ling-flash-2.0 100B total, 6.1B active
Ling-flash-base-2.0 inclusionAI/Ling-flash-base-2.0 Base checkpoint, 100B total, 6.1B active
Ling-mini-2.0 inclusionAI/Ling-mini-2.0 16B total, 1.5B active
Ling-mini-base-2.0 inclusionAI/Ling-mini-base-2.0 Base checkpoint, 16B total, 1.5B active

Architecture Notes

  • High-sparsity MoE with 256 routed experts and top-8 routing.
  • Sigmoid routing with QK-Norm and Half RoPE.
  • Custom Hugging Face model code is required, so conversion and inference commands use --trust-remote-code.

Examples

For checkpoint import/export, round-trip validation, and inference commands, see the Bailing examples README.

Related Implementation