Skip to content

Commit d9ec91c

Browse files
Yadan-WeiYadan Wei
andauthored
docs(vllm-omni): refresh for v1.7.0 (vLLM-Omni 0.28.0) (#6771)
* docs(vllm-omni): refresh for v1.7.0 (vLLM-Omni 0.28.0) - drift: correct s3tokenizer version reference 0.26.0 -> 0.28.0 in index.md - Add changelog entry for v1.7.0 (vLLM-Omni 0.28.0, DLC PR #6680) * docs(vllm-omni): focus v1.7.0 changelog Highlights on container changes - Replace upstream-PR Highlights with the container-side change: PEFT LoRA validated on SageMaker (new endpoint test, PR #6680) - Preserve the upstream v0.26.0...v0.28.0 compare link on the source line * docs(vllm-omni): drop upstream-PR frontend bullet from v1.7.0 Changes * docs(vllm-omni): add v1.7 (0.28.0) entry to README What's New * docs(vllm-omni): add v1.7 (0.28.0) to available_images data --------- Co-authored-by: Yadan Wei <weiyadan@amazon.com>
1 parent 215fdb1 commit d9ec91c

5 files changed

Lines changed: 55 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ ______________________________________________________________________
3131

3232
### 🚀 Release Highlights
3333

34+
- **[2026/09/21]** [vLLM-Omni v1.7 (AL2023)](https://gallery.ecr.aws/deep-learning-containers/vllm) — EC2: `omni-cuda-v1.7` · SageMaker: `omni-sagemaker-cuda-v1.7` · vLLM-Omni `0.28.0` (up from 0.26.0); PEFT LoRA adapters validated on SageMaker — per-request `lora` selection on `/v1/images/generations`; FlashInfer 0.6.16.post3, NCCL pinned 2.30.7, CUDA-13 mooncake wheel.
3435
- **[2026/09/11]** [vLLM v0.29.0 (Ubuntu)](https://gallery.ecr.aws/deep-learning-containers/vllm) — EC2: `0.29.0-gpu-py312-ec2` · SageMaker: `0.29.0-gpu-py312` · New models: Hy4-preview (Tencent 770B/49B-active MoE with Gated DeepSeek Sparse Attention and native MTP), Qwen3.8-Flash-Next (BF16/FP8/NVFP4, MTP), GraniteSWA, GraniteMoeSWA, NemotronH_Omni_Reasoning_V3 (MTP), and Kimi K3 NVFP4 checkpoints.
3536
- **[2026/09/07]** [SGLang v0.5.19 (Ubuntu)](https://gallery.ecr.aws/deep-learning-containers/sglang) — EC2: `0.5.19-gpu-py312-ec2` · SageMaker: `0.5.19-gpu-py312` · Qwen3.8, Ling-3.0, Spark2.5, Granite 4.2; beam search; DeepEP v2 MoE all-to-all.
3637
- **[2026/09/01]** [Ray LLM v1.0 (2.58.0, AL2023)](https://gallery.ecr.aws/deep-learning-containers/ray) — EC2/EKS: `serve-llm-cuda-v1.0` · Initial release: OpenAI-compatible LLM serving with Ray Serve and vLLM 0.26.0 on PyTorch 2.11.0 / CUDA 13.0.2 / Python 3.13; `ray[llm]`'s `build_openai_app` runs vLLM behind Ray Serve — single GPU on EC2, and multi-node serving on EKS via KubeRay.
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
framework: vLLM-Omni
2+
version: "0.28.0"
3+
ecr_repository: vllm
4+
accelerator: gpu
5+
python: py312
6+
cuda: cu130
7+
os: amzn2023
8+
platform: default
9+
public_registry: true
10+
11+
tags:
12+
- "omni-cuda-v1.7"
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
framework: vLLM-Omni
2+
version: "0.28.0"
3+
ecr_repository: vllm
4+
accelerator: gpu
5+
python: py312
6+
cuda: cu130
7+
os: amzn2023
8+
platform: sagemaker
9+
public_registry: true
10+
11+
tags:
12+
- "omni-sagemaker-cuda-v1.7"

‎docs/vllm-omni/changelog/index.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,35 @@ Changelog for the Amazon Linux 2023-based vLLM-Omni images (`omni-cuda`, `omni-s
44

55
* * *
66

7+
## v1.7.0 — 2026-09-21
8+
9+
**Tags:** `omni-cuda-v1.7` · `omni-sagemaker-cuda-v1.7`
10+
11+
**vLLM-Omni source:** [v0.28.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.28.0) (upstream vLLM v0.28.0) —
12+
[full upstream diff](https://github.com/vllm-project/vllm-omni/compare/v0.26.0...v0.28.0)
13+
14+
**DLC PR:** [#6680](https://github.com/aws/deep-learning-containers/pull/6680)
15+
16+
### Highlights
17+
18+
- **PEFT LoRA validated on SageMaker.** A new endpoint test deploys an SD-3.5-medium image model with a baked-in PEFT LoRA adapter registered at
19+
startup (`SM_VLLM_ENABLE_LORA`, `SM_VLLM_LORA_MODULES`, `SM_VLLM_MAX_LORA_RANK`) and confirms that per-request adapter selection via the `lora`
20+
field on `/v1/images/generations` changes the generated image.
21+
22+
### Changes
23+
24+
- **Bundled-library bumps.** FlashInfer 0.6.14 → 0.6.16.post3; DeepEP repinned; NCCL pinned to 2.30.7 because DeepEPv2's GIN backend requires NCCL ≥
25+
2.30.4.
26+
- **CUDA 13 mooncake wheel.** The KV-connector install now swaps `mooncake-transfer-engine` for its `-cuda13` variant to match the CUDA 13 base.
27+
- **`transformers` requirement** widened to `>= 5.10.1, < 5.15` (5.15 introduces a breaking change).
28+
29+
### Notes
30+
31+
- No CUDA change — still CUDA 13.0.2 and Python 3.12. This is a framework bump (v1.6 → v1.7); the EC2 and SageMaker images are rebuilt in lockstep to
32+
keep the paired tags aligned.
33+
34+
* * *
35+
736
## v1.6.0 — 2026-08-25
837

938
**Tags:** `omni-cuda-v1.6` · `omni-sagemaker-cuda-v1.6`

‎docs/vllm-omni/index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ In addition to vLLM-Omni and its core stack (PyTorch, CUDA 13.0, NCCL, Python 3.
3737
- **[EFA](https://aws.amazon.com/hpc/efa/) and [OpenMPI](https://www.open-mpi.org/)** — high-throughput multi-node networking on supported instances
3838
- **espeak-ng and ffmpeg** — system-level dependencies for TTS phonemizer and audio/video encoding
3939
- **[s3tokenizer](https://github.com/xingchensong/S3Tokenizer)** — speech tokenizer required by CosyVoice3 (hard-imported at model load in vLLM-Omni
40-
0.26.0)
40+
0.28.0)
4141

4242
The {{ sm_short }} image additionally includes a routing middleware that dispatches `/invocations` to omni-specific routes (TTS, image, video, etc.)
4343
via the `CustomAttributes` header. See [{{ sagemaker }} Deployment](deployment/sagemaker.md).

0 commit comments

Comments
 (0)