Skip to content

Commit a398a96

Browse files
larryliu0820claude
andauthored
Add LFM2.5 1.2B support with CI integration (pytorch#17509)
This adds support for exporting LiquidAI's LFM2.5-1.2B-Instruct model, which features improved training (28T tokens vs 10T) and extended context support (32K tokens). Changes: - Add lfm2_5_1_2b model configuration (examples/models/lfm2/config/lfm2_5_1_2b_config.json) - Register lfm2_5_1_2b in EXECUTORCH_DEFINED_MODELS and HUGGING_FACE_REPO_IDS - Add lfm2_5_1_2b to ModelType enum - Update README with LFM2.5 export examples and extended context usage - Add CI test for lfm2_5_1_2b in test_torchao_huggingface_checkpoints.sh - Add lfm2_5_1_2b to trunk.yml test matrix (XNNPACK and TorchAO backends) The model uses the same hybrid architecture as LFM2 (convolution + attention layers) with 16 layers and 8192 FFN dimension. Can be exported with custom max_seq_length and max_context_length for extended context support. Fixes pytorch#17439 --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent 2130a1e commit a398a96

7 files changed

Lines changed: 108 additions & 4 deletions

File tree

‎.ci/scripts/test_torchao_huggingface_checkpoints.sh‎

Lines changed: 27 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ MODEL_NAME=""
1111
# Parse args
1212
if [[ $# -lt 1 ]]; then
1313
echo "Usage: $0 <model_name> [--test_with_runner]"
14-
echo "Supported model_name values: qwen3_4b, phi_4_mini"
14+
echo "Supported model_name values: qwen3_4b, phi_4_mini, lfm2_5_1_2b"
1515
exit 1
1616
fi
1717

@@ -28,7 +28,7 @@ while [[ $# -gt 0 ]]; do
2828
;;
2929
-h|--help)
3030
echo "Usage: $0 <model_name> [--test_with_runner] [--use_torchao_kernels]"
31-
echo " model_name: qwen3_4b | phi_4_mini"
31+
echo " model_name: qwen3_4b | phi_4_mini | lfm2_5_1_2b"
3232
echo " --test_with_runner: build ET + run llama_main to sanity-check the export"
3333
echo " --use_torchao_kernels: use torchao kernels for linear and tied embedding"
3434
exit 0
@@ -101,9 +101,33 @@ case "$MODEL_NAME" in
101101
${BACKEND_ARGS}
102102
;;
103103

104+
lfm2_5_1_2b)
105+
echo "Running LFM2.5-1.2B export..."
106+
HF_MODEL_DIR=$(hf download LiquidAI/LFM2.5-1.2B-Instruct)
107+
EXPECTED_MODEL_SIZE_UPPER_BOUND=$((2500 * 1024 * 1024)) # 2.5GB
108+
$PYTHON_EXECUTABLE -m executorch.examples.models.lfm2.convert_weights \
109+
$HF_MODEL_DIR \
110+
pytorch_model_converted.bin
111+
112+
$PYTHON_EXECUTABLE -m executorch.examples.models.llama.export_llama \
113+
--model "lfm2_5_1_2b" \
114+
--checkpoint pytorch_model_converted.bin \
115+
--params examples/models/lfm2/config/lfm2_5_1_2b_config.json \
116+
--output_name $MODEL_OUT \
117+
-kv \
118+
--use_sdpa_with_kv_cache \
119+
--max_context_length 1024 \
120+
--max_seq_length 1024 \
121+
--metadata '{"get_bos_id":1, "get_eos_ids":[7]}' \
122+
--verbose \
123+
--dtype fp32 \
124+
-qmode 8da4w \
125+
${BACKEND_ARGS}
126+
;;
127+
104128
*)
105129
echo "Error: unsupported model_name '$MODEL_NAME'"
106-
echo "Supported values: qwen3_4b, phi_4_mini"
130+
echo "Supported values: qwen3_4b, phi_4_mini, lfm2_5_1_2b"
107131
exit 1
108132
;;
109133
esac

‎.github/workflows/trunk.yml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -661,7 +661,7 @@ jobs:
661661
contents: read
662662
strategy:
663663
matrix:
664-
model: [qwen3_4b, phi_4_mini]
664+
model: [qwen3_4b, phi_4_mini, lfm2_5_1_2b]
665665
runner: [linux.2xlarge]
666666
docker-image: [executorch-ubuntu-22.04-clang12]
667667
backend: [xnnpack]
@@ -674,6 +674,10 @@ jobs:
674674
runner: linux.arm64.2xlarge
675675
docker-image: executorch-ubuntu-22.04-gcc11-aarch64
676676
backend: torchao
677+
- model: lfm2_5_1_2b
678+
runner: linux.arm64.2xlarge
679+
docker-image: executorch-ubuntu-22.04-gcc11-aarch64
680+
backend: torchao
677681
fail-fast: false
678682
with:
679683
runner: ${{ matrix.runner }}

‎examples/models/lfm2/README.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
## Summary
22
[LFM2](https://huggingface.co/collections/LiquidAI/lfm2-686d721927015b2ad73eaa38) is a new generation of hybrid models developed by [Liquid AI](https://www.liquid.ai/) and available in 3 variants - 350M, 700M, 1.2B.
33

4+
[LFM2.5](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct) is an updated version with improved training (28T tokens vs 10T) and extended context length support (32K tokens).
5+
46
## Instructions
57

68
LFM2 uses the same example code as optimized Llama model, while the checkpoint, model params, and tokenizer are different. Please see the [Llama README page](../llama/README.md) for details.
@@ -35,6 +37,26 @@ python -m extension.llm.export.export_llm \
3537
+base.params="examples/models/lfm2/config/lfm2_1_2b_config.json" \
3638
+export.output_name="lfm2_1_2b_8da4w.pte"
3739
```
40+
41+
Export LFM2.5 1.2B to XNNPack, quantized with 8da4w:
42+
```
43+
python -m extension.llm.export.export_llm \
44+
--config examples/models/lfm2/config/lfm2_xnnpack_q8da4w.yaml \
45+
+base.model_class="lfm2_5_1_2b" \
46+
+base.params="examples/models/lfm2/config/lfm2_5_1_2b_config.json" \
47+
+export.output_name="lfm2_5_1_2b_8da4w.pte"
48+
```
49+
50+
To export with extended context (e.g., 2048 tokens):
51+
```
52+
python -m extension.llm.export.export_llm \
53+
--config examples/models/lfm2/config/lfm2_xnnpack_q8da4w.yaml \
54+
+base.model_class="lfm2_5_1_2b" \
55+
+base.params="examples/models/lfm2/config/lfm2_5_1_2b_config.json" \
56+
+export.max_seq_length=2048 \
57+
+export.max_context_length=2048 \
58+
+export.output_name="lfm2_5_1_2b_8da4w.pte"
59+
```
3860
### Example run
3961
With ExecuTorch pybindings:
4062
```
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
{
2+
"dim": 2048,
3+
"ffn_dim_multiplier": 1,
4+
"hidden_dim": 8192,
5+
"n_heads": 32,
6+
"n_kv_heads": 8,
7+
"n_layers": 16,
8+
"norm_eps": 1e-5,
9+
"rope_theta": 1000000.0,
10+
"use_scaled_rope": false,
11+
"vocab_size": 65536,
12+
"use_hf_rope": true,
13+
"use_qk_norm": true,
14+
"qk_norm_before_rope": true,
15+
"layer_types": [
16+
"conv",
17+
"conv",
18+
"full_attention",
19+
"conv",
20+
"conv",
21+
"full_attention",
22+
"conv",
23+
"conv",
24+
"full_attention",
25+
"conv",
26+
"full_attention",
27+
"conv",
28+
"full_attention",
29+
"conv",
30+
"full_attention",
31+
"conv"
32+
]
33+
}

‎examples/models/lfm2/convert_weights.py‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import argparse
12
import os
23
from typing import Dict
34

@@ -72,3 +73,20 @@ def convert_weights(input_dir: str, output_file: str) -> None:
7273
print("Saving checkpoint...")
7374
torch.save(sd, output_file)
7475
print("Done.")
76+
77+
78+
def main():
79+
parser = argparse.ArgumentParser(description="Convert LFM2 weights to Meta format.")
80+
parser.add_argument(
81+
"input_dir",
82+
type=str,
83+
help="Path to directory containing safetensor checkpoint files.",
84+
)
85+
parser.add_argument("output", type=str, help="Path to the output checkpoint")
86+
87+
args = parser.parse_args()
88+
convert_weights(args.input_dir, args.output)
89+
90+
91+
if __name__ == "__main__":
92+
main()

‎examples/models/llama/export_llama_lib.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,7 @@
108108
"lfm2_350m", # hybrid
109109
"lfm2_700m", # hybrid
110110
"lfm2_1_2b", # hybrid
111+
"lfm2_5_1_2b", # hybrid
111112
]
112113
TORCHTUNE_DEFINED_MODELS = ["llama3_2_vision"]
113114
HUGGING_FACE_REPO_IDS = {
@@ -122,6 +123,7 @@
122123
"lfm2_350m": "LiquidAI/LFM2-350M",
123124
"lfm2_700m": "LiquidAI/LFM2-700M",
124125
"lfm2_1_2b": "LiquidAI/LFM2-1.2B",
126+
"lfm2_5_1_2b": "LiquidAI/LFM2.5-1.2B-Instruct",
125127
}
126128

127129

‎extension/llm/export/config/llm_config.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ class ModelType(str, Enum):
4949
lfm2_350m = "lfm2_350m"
5050
lfm2_700m = "lfm2_700m"
5151
lfm2_1_2b = "lfm2_1_2b"
52+
lfm2_5_1_2b = "lfm2_5_1_2b"
5253

5354

5455
class PreqMode(str, Enum):

0 commit comments

Comments
 (0)