Skip to content

Commit 43f7d9f

Browse files
committed
feat(ci): add CUDA 13 wheel builds
1 parent c7af423 commit 43f7d9f

3 files changed

Lines changed: 13 additions & 5 deletions

File tree

‎.github/workflows/build-wheels-cuda.yaml‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ jobs:
2424
# wheel.py-api = "py3" makes the CUDA wheel interpreter-agnostic,
2525
# so one builder per toolkit version is sufficient.
2626
'pyver' = @("3.9")
27-
'cuda' = @("12.1.1", "12.2.2", "12.3.2", "12.4.1", "12.5.1")
27+
'cuda' = @("12.1.1", "12.2.2", "12.3.2", "12.4.1", "12.5.1", "13.0.2", "13.2.1")
2828
'releasetag' = @("basic")
2929
'exclude' = @(
3030
@{ 'os' = 'windows-2022'; 'cuda' = '12.1.1' },
@@ -169,10 +169,15 @@ jobs:
169169
}
170170
$cudaTagVersion = $nvccVersion.Replace('.','')
171171
$env:VERBOSE = '1'
172-
# Build real cubins for the supported GPUs, including Pascal, and keep
172+
$cudaArchs = "60-real;61-real;70-real;75-real;80-real;86-real;89-real;90-real;90-virtual"
173+
if ([version]$nvccVersion -ge [version]"13.0") {
174+
# CUDA 13 dropped offline compilation support for pre-Turing targets.
175+
$cudaArchs = "75-real;80-real;86-real;89-real;90-real;90-virtual"
176+
}
177+
# Build real cubins for the supported GPUs and keep
173178
# one forward-compatible PTX target instead of embedding PTX for every
174179
# SM. This keeps the wheel under GitHub's 2 GiB release-asset limit.
175-
$env:CMAKE_ARGS = "-DGGML_CUDA_FORCE_MMQ=ON -DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=60-real;61-real;70-real;75-real;80-real;86-real;89-real;90-real;90-virtual -DCMAKE_CUDA_FLAGS=-allow-unsupported-compiler -DCMAKE_CUDA_FLAGS_INIT=-allow-unsupported-compiler $env:CMAKE_ARGS"
180+
$env:CMAKE_ARGS = "-DGGML_CUDA_FORCE_MMQ=ON -DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=$cudaArchs -DCMAKE_CUDA_FLAGS=-allow-unsupported-compiler -DCMAKE_CUDA_FLAGS_INIT=-allow-unsupported-compiler $env:CMAKE_ARGS"
176181
$env:CMAKE_ARGS = $env:CMAKE_ARGS + ' -DGGML_AVX2=off -DGGML_FMA=off -DGGML_F16C=off'
177182
python -m build --wheel
178183
# Publish tags that reflect the actual installed toolkit version.

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
- feat(ci): add CUDA 13.0 and 13.2 wheel builds by @abetlen in #2239
1011
- fix(ci): add Pascal compute capability targets to CUDA wheel builds by @abetlen in #2237
1112

1213
## [0.3.24]

‎README.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -125,8 +125,8 @@ CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python
125125

126126
It is also possible to install a pre-built wheel with CUDA support. As long as your system meets some requirements:
127127

128-
- CUDA Version is 12.1, 12.2, 12.3, 12.4 or 12.5
129-
- NVIDIA GPU compute capability is 6.0 or newer
128+
- CUDA Version is 12.1, 12.2, 12.3, 12.4, 12.5, 13.0 or 13.2
129+
- NVIDIA GPU compute capability is 6.0 or newer for CUDA 12 wheels, or 7.5 or newer for CUDA 13 wheels
130130
- Python Version is 3.10, 3.11 or 3.12
131131

132132
```bash
@@ -140,6 +140,8 @@ Where `<cuda-version>` is one of the following:
140140
- `cu123`: CUDA 12.3
141141
- `cu124`: CUDA 12.4
142142
- `cu125`: CUDA 12.5
143+
- `cu130`: CUDA 13.0
144+
- `cu132`: CUDA 13.2
143145

144146
For example, to install the CUDA 12.1 wheel:
145147

0 commit comments

Comments
 (0)