Skip to content

Commit ba6f7a5

Browse files
committed
Dockerfile: install python3.10-dev + build-essential so Triton can JIT-compile
Uploads fail at inference with: gcc ... cuda_utils.c ... -l:libcuda.so.1 ... returned non-zero exit status 1 bitsandbytes routes the quantized model through Triton, which compiles a cuda_utils.c helper on first inference (needs Python.h) and links against libcuda.so.1 (needs gcc). The base image ships python3.10 and pip but NOT the Python headers, so that gcc step fails and every extraction errors out. This was patched by hand in the live container earlier but never landed in the Dockerfile, so any rebuild/redeploy reproduced the failure. Baking python3.10-dev and build-essential into the image fixes it for good, and independently of which bitsandbytes/triton version pip resolves -- the toolchain being present is what matters, not the version. Note the uv.lock in this repo is NOT a usable pin for this image: it resolves torch 2.11 / transformers 5.5 (CUDA 12.x) against a CUDA 11.8 base. A real dependency freeze has to come from `pip freeze` inside the working container; see the follow-up on pinning.
1 parent 8e83c3d commit ba6f7a5

1 file changed

Lines changed: 10 additions & 1 deletion

File tree

Dockerfile

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,21 @@ ENV DEBIAN_FRONTEND=noninteractive \
77
PYTHONUNBUFFERED=1 \
88
CUDA_VISIBLE_DEVICES=0
99

10-
# Install runtime deps + python
10+
# Install runtime deps + python.
11+
# python3.10-dev (Python.h) and build-essential (gcc/g++/make) are REQUIRED at
12+
# RUNTIME, not just build: bitsandbytes routes the quantized model through
13+
# Triton, which JIT-compiles a cuda_utils.c helper on first inference and links
14+
# -l:libcuda.so.1. Without the Python headers and a compiler that gcc step fails
15+
# with "returned non-zero exit status 1" and every upload errors out. Keeping the
16+
# toolchain in the image makes that compile succeed regardless of which
17+
# bitsandbytes/triton version is resolved.
1118
RUN apt-get update && apt-get install -y --no-install-recommends \
1219
git \
1320
curl \
1421
python3.10 \
22+
python3.10-dev \
1523
python3-pip \
24+
build-essential \
1625
&& rm -rf /var/lib/apt/lists/*
1726

1827
# Create symlink for python

0 commit comments

Comments
 (0)