Kimi K3 is a large sparse MoE model from Moonshot AI. Megatron Bridge supports the language backbone of the published multimodal checkpoint through the KimiK3Bridge.
Support for this model is in progress. Conversion (HF → Megatron) and Megatron greedy inference are verified; strict full-checkpoint export, exact round-trip parity, and every training workflow are not. See Known Limitations and the machine-readable [verification card](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/model_verification_cards/kimi-k3/card.yaml) before relying on this path.
Choose an exact recorded configuration to see its command and expected result. These selectors are generated from the authoritative verification cards and never synthesize combinations.
Choose a workflow, precision, and exact recorded combination. The command and expected result update below.
- Hardware
- not specified
- Precision
- BF16
- Last verified
- —
No runnable command is recorded for this status.
No CPU Hugging Face-to-Megatron conversion result is claimed. Future verification must import the complete pinned language backbone with the public conversion launcher, persist iter_0000000, reload it, and audit all mapped keys, shapes, dtypes, and values.
- Hardware
- not specified
- Precision
- BF16
- Last verified
- 2026-07-28
./scripts/conversion/convert.sh import --executor slurm --device gpu --nodes 12 --gpus-per-node 4 --hf-model moonshotai/Kimi-K3 --hf-revision 9f62e4e9fffbd0a83ddd60e1c209d828994b3569 --megatron-path work/model-verification/kimi-k3/imported-megatron --torch-dtype bfloat16 --tp 2 --pp 3 --ep 8 --etp 2 --distributed-timeout-minutes 180 --trust-remote-codeOn 48x GB200, the pinned complete checkpoint imports at TP2/PP3/EP8/ETP2 and exits successfully in 27:01. Published MXFP4 routed-expert weights are dequantized to BF16, and iter_0000000 is persisted with 48 distributed-checkpoint shards, metadata, and the recorded 93-layer run configuration. Exact Hugging Face round-trip parity is tracked separately by megatron_to_hf_gpu.
- Hardware
- not specified
- Precision
- BF16
- Last verified
- —
No runnable command is recorded for this status.
No CPU Megatron-to-Hugging-Face conversion result is claimed. Future verification depends on a complete persisted Megatron import and must write, strictly reload, and exactly audit the exported Hugging Face language-backbone weights against the pinned source revision.
- Hardware
- not specified
- Precision
- BF16
- Last verified
- —
No runnable command is recorded for this status.
No GPU Megatron-to-Hugging-Face conversion result is claimed. All 69 source KDA A_log tensors were audited: each stores 96 active heads plus 32 zero-valued inactive entries, for 2,208 checked padding values. Import validation drops only this zero padding, and export unit tests restore it with zeros. A four-layer diagnostic proxy completed low-memory GPU import and distributed streaming export. Its seven safetensors shards and index, totaling 58,482,941,904 bytes, were byte-identical to an export from the previous default-save import; this validates the low-memory I/O behavior, not full-model parity. Against the published proxy source, all 16,402 keys and shapes matched and A_log padding was valid. Three o_norm tensors changed from FP32 to BF16, and 16,128 value mismatches were concentrated in MXFP4 expert packed weights and scales because import dequantizes them to BF16 before export requantizes them. Full verification therefore still requires the pinned complete model to export and strictly reload, exact comparison of lossless tensors, and a declared numerical criterion for lossy MXFP4 reconstruction.
- Hardware
- GB300
- Precision
- BF16
- Last verified
- —
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
No runnable command is recorded for this status.
Training was intentionally excluded from this model-only verification pass. A future public Kimi K3 language-backbone recipe must complete a bounded 100-step pretrain run with finite loss, no skipped or NaN iterations, all five metrics, and a reloadable final checkpoint.
- Hardware
- GB300
- Precision
- BF16
- Last verified
- —
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
No runnable command is recorded for this status.
Training was intentionally excluded from this model-only verification pass. A future public Kimi K3 language-backbone recipe must complete a pinned-data 100-step full-SFT run with finite loss, no skipped or NaN iterations, all five metrics, and a reloadable final checkpoint.
- Hardware
- GB300
- Precision
- BF16
- Last verified
- —
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
No runnable command is recorded for this status.
Training was intentionally excluded from this model-only verification pass. A future public long-context SFT recipe must define supported sequence packing and context parallelism, complete 100 steps with finite loss and no skipped or NaN iterations, and record all five metrics.
- Hardware
- GB300
- Precision
- BF16
- Last verified
- —
- Initial loss
- None
- Final loss
- None
- Step time · last 10 avg
- None ms
- Model throughput · last 10 avg
- None TFLOP/s/GPU
- Token throughput · last 10 avg
- None tokens/s/GPU
No runnable command is recorded for this status.
Training was intentionally excluded from this model-only verification pass. A future Kimi K3 PEFT recipe with an audited adapter target set must complete 100 steps with finite loss, no skipped or NaN iterations, all five metrics, and a reloadable adapter checkpoint.