Skip to content

Commit 1cbc24c

Browse files
Windows wheels: init submodules in pre-build script with OpenSSL (pytorch#18262)
The previous fix (setting sslBackend in pre_build_script.sh) only applied to nested tokenizer submodules. The top-level submodule checkout still used schannel via the reusable workflow's `submodules: true`, causing SEC_E_ILLEGAL_MESSAGE errors when cloning from git.gitlab.arm.com. Move all submodule initialization into the pre-build script where we can control the SSL backend, and disable submodule checkout in the workflow.
1 parent 36c22e9 commit 1cbc24c

2 files changed

Lines changed: 22 additions & 13 deletions

File tree

‎.ci/scripts/wheel/pre_build_script.sh‎

Lines changed: 19 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -9,34 +9,41 @@ set -euxo pipefail
99

1010
# This script is run before building ExecuTorch binaries
1111

12-
if [[ "$(uname -m)" == "aarch64" ]]; then
13-
# On some Linux aarch64 systems, the "atomic" library is not found during linking.
14-
# To work around this, replace "atomic" with the literal ${ATOMIC_LIB} so the
15-
# build system uses the full path to the atomic library.
16-
file="extension/llm/tokenizers/third-party/sentencepiece/src/CMakeLists.txt"
17-
sed 's/list(APPEND SPM_LIBS "atomic")/list(APPEND SPM_LIBS ${ATOMIC_LIB})/' \
18-
"$file" > "${file}.tmp" && mv "${file}.tmp" "$file"
19-
20-
grep -n 'list(APPEND SPM_LIBS ${ATOMIC_LIB})' "$file" && \
21-
echo "the file $file has been modified for atomic to use full path"
12+
# Initialize submodules here instead of during checkout so we can use OpenSSL
13+
# on Windows (schannel fails with SEC_E_ILLEGAL_MESSAGE on some gitlab hosts).
14+
UNAME_S=$(uname -s)
15+
if [[ $UNAME_S == *"MINGW"* || $UNAME_S == *"MSYS"* ]]; then
16+
git -c http.sslBackend=openssl submodule update --init
17+
else
18+
git submodule update --init
2219
fi
2320

2421
# Clone nested submodules for tokenizers - this is a workaround for recursive
2522
# submodule clone failing due to path length limitations on Windows. Eventually,
2623
# we should update the core job in test-infra to enable long paths before
2724
# checkout to avoid needing to do this.
2825
pushd extension/llm/tokenizers
29-
UNAME_S=$(uname -s)
3026
if [[ $UNAME_S == *"MINGW"* || $UNAME_S == *"MSYS"* ]]; then
3127
git -c http.sslBackend=openssl submodule update --init
3228
else
3329
git submodule update --init
3430
fi
3531
popd
3632

33+
if [[ "$(uname -m)" == "aarch64" ]]; then
34+
# On some Linux aarch64 systems, the "atomic" library is not found during linking.
35+
# To work around this, replace "atomic" with the literal ${ATOMIC_LIB} so the
36+
# build system uses the full path to the atomic library.
37+
file="extension/llm/tokenizers/third-party/sentencepiece/src/CMakeLists.txt"
38+
sed 's/list(APPEND SPM_LIBS "atomic")/list(APPEND SPM_LIBS ${ATOMIC_LIB})/' \
39+
"$file" > "${file}.tmp" && mv "${file}.tmp" "$file"
40+
41+
grep -n 'list(APPEND SPM_LIBS ${ATOMIC_LIB})' "$file" && \
42+
echo "the file $file has been modified for atomic to use full path"
43+
fi
44+
3745
# On Windows, enable symlinks and re-checkout the current revision to create
3846
# the symlinked src/ directory. This is needed to build the wheel.
39-
UNAME_S=$(uname -s)
4047
if [[ $UNAME_S == *"MINGW"* || $UNAME_S == *"MSYS"* ]]; then
4148
echo "Enabling symlinks on Windows"
4249
git config core.symlinks true

‎.github/workflows/build-wheels-windows.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,4 +64,6 @@ jobs:
6464
smoke-test-script: ${{ matrix.smoke-test-script }}
6565
trigger-event: ${{ github.event_name }}
6666
wheel-build-params: "--verbose"
67-
submodules: true
67+
# Submodules are initialized in pre_build_script.sh with OpenSSL to avoid
68+
# schannel SSL errors on Windows when cloning from non-GitHub hosts.
69+
submodules: false

0 commit comments

Comments
 (0)