Skip to content

Commit 8390f8b

Browse files
AlbertoDeckowclaude
andcommitted
Build the QUIC stack one configure at a time
Gradle configures arm64-v8a and x86_64 in parallel, and each configure runs build-deps.sh against the same source tree. The second run found the first one's clone half-written, read it as the wrong commit, removed it and fetched over it, and the first clone then failed. With a warm Gradle cache both configures start together, so CI's Assemble and device jobs failed in seconds - intermittently since August. build-deps.sh now takes a lock for any run that fetches or builds. A directory is the lock, since mkdir is atomic everywhere and flock does not ship on macOS; a lock whose holder has exited is taken over. Reproduced with two concurrent fetches into one empty tree: before, one of the two fails with "could not fetch ngtcp2"; after, both succeed and the second waits and reuses the first one's sources. A stale lock left by an exited process is taken over. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 4f446f2 commit 8390f8b

1 file changed

Lines changed: 32 additions & 2 deletions

File tree

‎tools/quic/build-deps.sh‎

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,36 @@ DEPS="${QUIC_DEPS_DIR:-$REPO/.quic-deps}"
3535
SRC="$DEPS/src"
3636
KEY="${NGTCP2_COMMIT:0:12}-${AWSLC_COMMIT:0:12}"
3737

38+
# --- one at a time -----------------------------------------------------------
39+
#
40+
# Gradle configures every ABI in parallel, and each configure runs this script
41+
# against the same source tree. Unserialised, one run saw the other's clone
42+
# half-written, removed it and fetched over it, and both failed; on CI that was
43+
# a build which failed in seconds, intermittently. A directory is the lock
44+
# because mkdir is atomic everywhere and flock is not on macOS. A lock whose
45+
# holder has exited is stale and is taken over.
46+
47+
LOCK="$DEPS/.lock"
48+
49+
lock() {
50+
mkdir -p "$DEPS"
51+
local waited=0
52+
until mkdir "$LOCK" 2>/dev/null; do
53+
local holder; holder=$(cat "$LOCK/pid" 2>/dev/null)
54+
if [ -n "$holder" ] && ! kill -0 "$holder" 2>/dev/null; then
55+
note "taking over a lock left by exited process $holder"
56+
rm -rf "$LOCK"
57+
continue
58+
fi
59+
[ "$waited" -eq 0 ] && note "waiting for another build-deps run${holder:+ (process $holder)}"
60+
waited=$((waited + 1))
61+
[ "$waited" -le 3600 ] || fail "gave up after an hour waiting for $LOCK"
62+
sleep 1
63+
done
64+
echo $$ > "$LOCK/pid"
65+
trap 'rm -rf "$LOCK"' EXIT
66+
}
67+
3868
# --- fetching, and the check that makes a tag mean something ---------------
3969

4070
fetch() {
@@ -178,10 +208,10 @@ build_abi() {
178208
}
179209

180210
case "${1:-}" in
181-
--bundle-source) bundle_source "${2:-}" ;;
211+
--bundle-source) lock; bundle_source "${2:-}" ;;
182212
--prefix-only)
183213
[ -n "${2:-}" ] || fail "--prefix-only needs an ABI"
184214
echo "PREFIX=$DEPS/out/$2" ;;
185-
arm64-v8a|x86_64|host) build_abi "$1" ;;
215+
arm64-v8a|x86_64|host) lock; build_abi "$1" ;;
186216
*) fail "usage: build-deps.sh <arm64-v8a|x86_64|host> | --prefix-only <abi> | --bundle-source <out.tar.gz>" ;;
187217
esac

0 commit comments

Comments
 (0)