Skip to content

Commit ea59673

Browse files
authored
[examples] fix: update Bailing inference recipe (#4076)
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
1 parent ab1d5e4 commit ea59673

1 file changed

Lines changed: 3 additions & 3 deletions

File tree

‎examples/models/bailing/inference.sh‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ uv run python -m torch.distributed.run --nproc_per_node=8 scripts/inference/text
2626
--hf_model_path inclusionAI/Ling-flash-2.0 \
2727
--prompt "$PROMPT" \
2828
--max_new_tokens "$MAX_NEW_TOKENS" \
29-
--tp 2 --ep 4 \
29+
--tp 1 --ep 8 \
3030
--use-coordinator \
3131
--coordinator-host "${COORDINATOR_HOST}" \
3232
--trust-remote-code
@@ -38,7 +38,7 @@ if [ -d "$MEGATRON_MODEL_PATH" ]; then
3838
--megatron_model_path "$MEGATRON_MODEL_PATH" \
3939
--prompt "$PROMPT" \
4040
--max_new_tokens "$MAX_NEW_TOKENS" \
41-
--tp 2 --ep 4 \
41+
--tp 1 --ep 8 \
4242
--use-coordinator \
4343
--coordinator-host "${COORDINATOR_HOST}" \
4444
--trust-remote-code
@@ -50,7 +50,7 @@ if [ -d "$HF_EXPORT_PATH" ]; then
5050
--hf_model_path "$HF_EXPORT_PATH" \
5151
--prompt "$PROMPT" \
5252
--max_new_tokens "$MAX_NEW_TOKENS" \
53-
--tp 2 --ep 4 \
53+
--tp 1 --ep 8 \
5454
--use-coordinator \
5555
--coordinator-host "${COORDINATOR_HOST}" \
5656
--trust-remote-code

0 commit comments

Comments
 (0)