Merge remote-tracking branch 'origin/main'

Author: Eric Hartford

AGENTS.md

@@ -120,6 +120,9 @@ behavior:
 
 - Live validation must discover every registry profile compatible with the
   current machine. Do not substitute one tensor-parallel size for another.
+  It covers LLM (`kind: "llm"`) profiles by serving and smoking each one;
+  video profiles are validated by `perf/ltx25_harness/n6_serve_check.py`
+  run on a Mac against the real profile.
 - Every supported profile must run with its registered DSpark drafter and
   registered draft KV configuration. FP8 is the only permitted KV cache
   quantization; TurboQuant is prohibited for targets and drafters.
@@ -128,6 +131,26 @@ behavior:
 - For vision profiles, test both text and image requests. For text-only
   profiles, test text requests.
 
+## Non-LLM generation engines
+
+- SlimServe also hosts non-LLM generation engines. A source declares its
+  engine with `kind`: `"llm"` (the default, served by vLLM) or `"video"`
+  (LTX-2.5 audio+video generation on Metal, `slimserve/video/`). The
+  registry rejects any other kind, and every non-LLM kind must dispatch to
+  a registered non-vLLM engine; a test enforces both.
+- The LLM serving policies (prefix caching, tool calling, thinking,
+  non-greedy sampling, KV cache dtype, default context, drafters, KV tiers)
+  apply to `kind: "llm"` sources and are not meaningful for video sources.
+  The LLM policy tests assert a non-zero count of LLM profiles so a mass
+  exemption is visible.
+- Video profiles still follow every other rule: one record per platform,
+  pinned revisions for every download, no deployment configuration in the
+  repo, and notebook entries with raw-artifact evidence for every measured
+  claim.
+- The LTX-2.5 prompt enhancer (Gemma) decodes greedily because the upstream
+  pipeline does; that is a model-level exception for the enhancer, not a
+  precedent for any chat model.
+
 ## Commit authorship
 
 - AI is never an author. Never attribute authorship to an AI agent: no

CLAUDE.md

@@ -160,6 +160,9 @@ behavior:
 
 - Live validation must discover every registry profile compatible with the
   current machine. Do not substitute one tensor-parallel size for another.
+  It covers LLM (`kind: "llm"`) profiles by serving and smoking each one;
+  video profiles are validated by `perf/ltx25_harness/n6_serve_check.py`
+  run on a Mac against the real profile.
 - Every supported profile must run with its registered DSpark drafter and
   registered draft KV configuration. FP8 is the only permitted KV cache
   quantization; TurboQuant is prohibited for targets and drafters.
@@ -168,6 +171,26 @@ behavior:
 - For vision profiles, test both text and image requests. For text-only
   profiles, test text requests.
 
+## Non-LLM generation engines
+
+- SlimServe also hosts non-LLM generation engines. A source declares its
+  engine with `kind`: `"llm"` (the default, served by vLLM) or `"video"`
+  (LTX-2.5 audio+video generation on Metal, `slimserve/video/`). The
+  registry rejects any other kind, and every non-LLM kind must dispatch to
+  a registered non-vLLM engine; a test enforces both.
+- The LLM serving policies (prefix caching, tool calling, thinking,
+  non-greedy sampling, KV cache dtype, default context, drafters, KV tiers)
+  apply to `kind: "llm"` sources and are not meaningful for video sources.
+  The LLM policy tests assert a non-zero count of LLM profiles so a mass
+  exemption is visible.
+- Video profiles still follow every other rule: one record per platform,
+  pinned revisions for every download, no deployment configuration in the
+  repo, and notebook entries with raw-artifact evidence for every measured
+  claim.
+- The LTX-2.5 prompt enhancer (Gemma) decodes greedily because the upstream
+  pipeline does; that is a model-level exception for the enhancer, not a
+  precedent for any chat model.
+
 ## Commit authorship
 
 - AI is never an author. Never attribute authorship to an AI agent: no
@@ -185,8 +208,9 @@ behavior:
   calling, and thinking enabled, and NEVER uses greedy sampling. These are
   enforced as registry-level `_SERVING_DEFAULTS` plus explicit per-profile
   values and a registry test. Prefix caching admits NO opt-out: every
-  platform record states `enable_prefix_caching: true` and the registry
-  test rejects anything else (operator directive 2026-08-30). For the
+  LLM (`kind: "llm"`) platform record states `enable_prefix_caching: true`
+  and the registry test rejects anything else (operator directive
+  2026-08-30). For the
   others, a profile may opt out only for a model-level impossibility,
   stated in a note.
 - Never rely on engine-layer defaults for these: vLLM silently defaults

CMakeLists.txt

@@ -611,8 +611,17 @@ if(VLLM_GPU_LANG STREQUAL "CUDA" OR VLLM_GPU_LANG STREQUAL "HIP")
     else()
       cuda_archs_loose_intersection(MARLIN_FP8_ARCHS "8.9;12.0a;12.1a" "${CUDA_ARCHS}")
     endif()
-    # marlin arches for other files
-    cuda_archs_loose_intersection(MARLIN_OTHER_ARCHS "7.5;8.0+PTX" "${CUDA_ARCHS}")
+    # marlin arches for other files (marlin.cu, the int4/fp8 preprocess and the
+    # gptq/awq repack kernels). These must carry SASS for every family the
+    # template kernels above are built for: with only 8.0+PTX an SM12x GPU has
+    # to JIT the compute_80 PTX at first launch, which fails with
+    # cudaErrorUnsupportedPtxVersion whenever the toolkit is newer than the
+    # driver (CUDA 13.3 build on an R595/13.2 driver).
+    if(${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL 13.0)
+      cuda_archs_loose_intersection(MARLIN_OTHER_ARCHS "7.5;8.0+PTX;12.0f" "${CUDA_ARCHS}")
+    else()
+      cuda_archs_loose_intersection(MARLIN_OTHER_ARCHS "7.5;8.0+PTX;12.0a;12.1a" "${CUDA_ARCHS}")
+    endif()
 
     if (MARLIN_OTHER_ARCHS)
 
@@ -1098,7 +1107,13 @@ if(VLLM_GPU_LANG STREQUAL "CUDA" OR VLLM_GPU_LANG STREQUAL "HIP")
   endif()
 
   # Hadacore kernels
-  cuda_archs_loose_intersection(HADACORE_ARCHS "8.0+PTX;9.0+PTX" "${CUDA_ARCHS}")
+  # SM12x gets SASS so it does not depend on JIT of the 9.0 PTX (see
+  # MARLIN_OTHER_ARCHS).
+  if(${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL 13.0)
+    cuda_archs_loose_intersection(HADACORE_ARCHS "8.0+PTX;9.0+PTX;12.0f" "${CUDA_ARCHS}")
+  else()
+    cuda_archs_loose_intersection(HADACORE_ARCHS "8.0+PTX;9.0+PTX;12.0a;12.1a" "${CUDA_ARCHS}")
+  endif()
   if(HADACORE_ARCHS)
     set(HADACORE_SRCS "csrc/libtorch_stable/quantization/hadamard/hadacore/hadamard_transform_cuda.cu")
     set_gencode_flags_for_srcs(
@@ -1225,8 +1240,13 @@ if(VLLM_GPU_LANG STREQUAL "CUDA")
   else()
     cuda_archs_loose_intersection(MARLIN_MOE_FP8_ARCHS "8.9;12.0a;12.1a" "${CUDA_ARCHS}")
   endif()
-  # moe marlin arches for other files
-  cuda_archs_loose_intersection(MARLIN_MOE_OTHER_ARCHS "7.5;8.0+PTX" "${CUDA_ARCHS}")
+  # moe marlin arches for other files (ops.cu: dispatch and permute_cols).
+  # SASS for SM12x, not just 8.0 PTX, for the same reason as MARLIN_OTHER_ARCHS.
+  if(${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER_EQUAL 13.0)
+    cuda_archs_loose_intersection(MARLIN_MOE_OTHER_ARCHS "7.5;8.0+PTX;12.0f" "${CUDA_ARCHS}")
+  else()
+    cuda_archs_loose_intersection(MARLIN_MOE_OTHER_ARCHS "7.5;8.0+PTX;12.0a;12.1a" "${CUDA_ARCHS}")
+  endif()
   if (MARLIN_MOE_OTHER_ARCHS)
 
     #