Summary
scaled_mm in tensorrt_llm/_torch/cute_dsl_kernels/blackwell/dense_blockscaled_gemm_persistent.py compiles Sm100BlockScaledPersistentDenseGemmKernel directly. Its positional arguments do not match Sm100BlockScaledPersistentDenseGemmKernel.__call__.
Required change
Update scaled_mm to compile the wrapper interface. Supply the required runtime dimensions and alpha_tensor. Preserve the existing pointer construction, layout mapping, cluster limit, stream, epilogue operation, and compilation options.
Rationale
The direct cute.compile(gemm_obj, ...) call binds the layout tuple where __call__ expects alpha. It then shifts the remaining positional arguments. This can produce an invalid compiled invocation.
Affected area
tensorrt_llm/_torch/cute_dsl_kernels/blackwell/dense_blockscaled_gemm_persistent.py
scaled_mm
Sm100BlockScaledPersistentDenseGemmKernel.wrapper
Acceptance criteria
scaled_mm compiles the wrapper with arguments that match the wrapper signature.
- The compile-time placeholders include the GEMM dimensions and
alpha_tensor.
- Existing layout, pointer-alignment, stream, epilogue, cluster, and
options behavior remains intact.
- Add or update focused coverage for the compile interface if the test environment supports it.
Backlinks
Requested by: @farazkh80
Summary
scaled_mmintensorrt_llm/_torch/cute_dsl_kernels/blackwell/dense_blockscaled_gemm_persistent.pycompilesSm100BlockScaledPersistentDenseGemmKerneldirectly. Its positional arguments do not matchSm100BlockScaledPersistentDenseGemmKernel.__call__.Required change
Update
scaled_mmto compile thewrapperinterface. Supply the required runtime dimensions andalpha_tensor. Preserve the existing pointer construction, layout mapping, cluster limit, stream, epilogue operation, and compilation options.Rationale
The direct
cute.compile(gemm_obj, ...)call binds the layout tuple where__call__expectsalpha. It then shifts the remaining positional arguments. This can produce an invalid compiled invocation.Affected area
tensorrt_llm/_torch/cute_dsl_kernels/blackwell/dense_blockscaled_gemm_persistent.pyscaled_mmSm100BlockScaledPersistentDenseGemmKernel.wrapperAcceptance criteria
scaled_mmcompiles the wrapper with arguments that match the wrapper signature.alpha_tensor.optionsbehavior remains intact.Backlinks
Requested by: @farazkh80