Skip to content

Commit 9a3b331

Browse files
authored
test(serve): SD/KT-IC integ must use g6.2xlarge, not g4dn (T4) (#6141)
The LMI/vLLM JumpStart container (CUDA 12.x) does not start on ml.g4dn.xlarge (T4/Turing): the Inference Component fails with CannotStartContainerError. Switch INSTANCE_TYPE to ml.g6.2xlarge (L4), the smallest instance on which it starts. Verified live: the deploy() IC path reaches InService on g6.2xlarge with both base_model + draft_model AdditionalModelDataSources intact.
1 parent c4f9b5b commit 9a3b331

1 file changed

Lines changed: 3 additions & 3 deletions

File tree

‎sagemaker-serve/tests/integ/test_ai_inference_recommender_sdkt_ic_integration.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,10 @@
3030
logger = logging.getLogger(__name__)
3131

3232
# Small, ungated, chat-templated model. A GPU instance is required by the
33-
# JumpStart vLLM/LMI container (not by the model size); a single-GPU g4dn.xlarge
34-
# (T4) is ample for a 0.6B model.
33+
# JumpStart vLLM/LMI container (not by the model size); a single-GPU
34+
# g6.2xlarge (L4) is ample for a 0.6B model.
3535
MODEL_ID = "huggingface-reasoning-qwen3-06b"
36-
INSTANCE_TYPE = "ml.g4dn.xlarge"
36+
INSTANCE_TYPE = "ml.g6.2xlarge"
3737
# Right-sized for a 0.6B on a single GPU; an oversized request overflows the
3838
# host and the IC never leaves Creating.
3939
IC_MIN_MEMORY_MB = 4096

0 commit comments

Comments
 (0)