[Yadan-Wei]

What

Bump vLLM-Omni 0.26.0 → 0.28.0 (a two-minor jump; omni tracks upstream vLLM 1:1, so vllm_ref → v0.28.0) across both amzn2023 image configs and the Dockerfile ARG defaults.

Field	0.26.0 → 0.28.0
framework_version / vllm_ref / vllm_omni_version	0.28.0
flashinfer_version	0.6.14 → 0.6.16.post3
deepep_commit_hash	73b6ea4 → d4f41e4e93602a15e95f55f6ee8df8f1aaa0e4bb (full 40-char SHA — upstream now requires it; an abbreviated hash cannot be resolved via git fetch origin <sha>)
dlc_minor_version	6 → 7 (also realigns a Dockerfile ARG default that had drifted to 5)
CUDA / Python	unchanged (13.0.2 / 3.12 — upstream's 13.0.3 + Ubuntu 24 base move is N/A to the amzn2023 base)

Release-notes reconciliation

Added — mooncake CUDA-13 wheel swap (docker/vllm_omni/Dockerfile.amzn2023, KV-connector install step). Upstream's default mooncake-transfer-engine PyPI wheel is a CUDA-12 build (its engine.so links libcudart.so.12, absent from the CUDA-13 runtime). Since 0.28.0, upstream's install-kv-connectors.sh swaps it for the same-version mooncake-transfer-engine-cuda13 variant on CUDA-13 images. We install kv_connectors.txt directly (bypassing that script) and ship install_kv_connectors: true, so this mirrors the swap here — matching the existing nixl-cu* reinstall pattern.

Checked, no change needed:

• NCCL — upstream pins nvidia-nccl-cu13==2.30.7 for DeepEPv2 (needs ≥ 2.30.4); we pull unpinned libnccl from NVIDIA's cuda-rhel9 repo → already current. DeepEP's manylinux syscall + CUDA-stub fixes live inside the ep_kernels script we clone at v0.28.0.
• SageMaker bidi WebSocket routes — WS_ALLOWED_PATHS frozenset unchanged (upstream's 6 routes are identical at v0.28.0).
• s3tokenizer — still dev-only upstream at 0.28.0; the explicit s3tokenizer==0.3.0 pin stays correct.
• GCC ≥ 11.3 / rdma-core / ccache / Rust frontend — amzn2023 already satisfies (gcc 11.4, libibverbs-devel, sccache); Rust build (build_rust.sh, install_protoc.sh) is byte-identical vs 0.26.0.
• Refreshed the stale transformers-cap comment to the new >=5.10.1,<5.15 bound.

Added — PEFT LoRA SageMaker endpoint test

New real-endpoint test test/vllm-omni/sagemaker/test_sm_omni_endpoint.py::test_vllm_omni_peft_lora_endpoint (+ a peft-lora leg in vllm-omni.tests-sagemaker.yml) validates the per-request PEFT LoRA path end-to-end on the SageMaker image, closing the one omni serving mode the suite didn't cover:

• Deploys a realtime endpoint with SM_VLLM_ENABLE_LORA=true and an adapter registered via SM_VLLM_LORA_MODULES, then invokes /v1/images/generations with and without the per-request lora field and asserts the two images differ — proving the adapter is selected per-request through the DLC SageMaker serving middleware.
• Base is stable-diffusion-3.5-medium (~20 GiB, fits a single 24 GB GPU with no offload). It is HF-gated, so the base + PEFT adapter are baked into the S3 model artifact (base at tar root + adapters/sd35-lora/) and loaded from /opt/ml/model — no runtime HF pull and no HF token on the endpoint. Ada-only instance pool (g6.xlarge/2xl/4xl; the omni 0.28 container needs sm_89+, so no g5/Ampere).
• Teardown waits for the endpoint to leave Creating before deleting, so a slow/timed-out deploy cannot leak an endpoint.

SageMaker endpoint test restructure

This PR also reworks how the omni SageMaker endpoint tests run:

• Per-case fan-out. Replaces the old two-group -k split (realtime / async) with one matrix job per endpoint case — tts-realtime, tts-async, bidi-speech-stream, bidi-default-path-rejected, peft-lora, plus a trailing video-async job. max-parallel: 1 keeps at most one endpoint live at a time (respecting the tiny per-type SageMaker endpoint quotas); fail-fast: false lets every case run even after one fails; each leg gets a fresh runner-role session so a failure can be re-run in isolation.
• Native instance pools. All three endpoint fixtures now deploy over SageMaker instance pools via the shared build_instance_pools(instance_type) helper (priority-based capacity fallback across an instance-type ladder), replacing the previous client-side ICE-retry loop — the omni half of the vLLM instance-pools migration. The peft-lora leg uses an Ada-only ladder (g6.xlarge/2xl/4xl) since the 0.28 container needs sm_89+; the tts/bidi/video legs keep their existing ladders.
• video-async runs last as its own job and is allowed to skip (not fail) when SageMaker exhausts every instance-pool candidate, so a capacity skip is not recorded as a full-suite pass.

Test plan

• CI build (first build of a new version is a full source compile — no wheel-cache hit) — passed
• DeepEP/NCCL path and the new mooncake cuda13 swap built cleanly
• EC2 + SageMaker smoke tests — passed (all model smoke-test legs green)
• SageMaker bidi endpoint tests — passed (bidi-speech-stream, bidi-default-path-rejected)
• SageMaker TTS + video-async endpoint tests — passed
• New PEFT LoRA endpoint test — passed (endpoint-test (peft-lora))

Deferred — new-model CI coverage (follow-up)

0.28.0 adds notable families: IndexTTS 2.5, MiniMax Music 3, MiniMax-H3, SANA-Video, Nemotron VoiceChat (full-duplex), dots.tts, π0 VLA, etc. Not added here — each needs model-upload staging + verify-model confirmation before wiring a CI entry, and some require optional-dep extras (indextts2, longcat-video-avatar). Can follow up per customer demand.

The distill LoRA backend (startup-fused LightX2V checkpoints) is validated by config but not yet wired as a CI leg — its base (Qwen-Image, ~60 GiB + cpu-offload) needs a large, capacity-scarce instance; deferred until a reliable slot is available.