wasm/v8: Source wee8 from envoy_toolshed prebuilt via label_flag - #47108
wasm/v8: Source wee8 from envoy_toolshed prebuilt via label_flag#47108dcillera wants to merge 4 commits into
Conversation
|
i think the absl libs have to be added at consumer with current toolshed build - not sure if that is best way - but something like this should probs work ... envoy_cc_library(
name = "wee8_compile_lib",
srcs = ["wee8_compile.cc"],
copts = [
"-Wno-comments",
"-Wno-non-virtual-dtor",
"-Wno-unused-parameter",
],
deps = [
"@com_google_absl//absl/container:flat_hash_map",
"@com_google_absl//absl/container:flat_hash_set",
"@com_google_absl//absl/hash",
"@com_google_absl//absl/synchronization",
"@proxy-wasm-cpp-host//:base_lib",
"@proxy-wasm-cpp-host//:v8_lib",
],
) |
The toolshed prebuilt libwee8.a excludes abseil by design (consumers provide their own to avoid ODR/ABI clashes). The from-source @v8//:wee8 propagates abseil transitively, so wee8_compile linked without listing it; the prebuilt does not, so this minimal v8_lib consumer fails to link with undefined absl::* symbols. Add the abseil libs wee8 references explicitly. Uses @abseil-cpp (the repo name in the current tree) rather than the older @com_google_absl. Addresses phlax review feedback on envoyproxy#47108. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Dario Cillerai <dcillera@redhat.com>
|
Thanks @phlax — done in the latest push. Added the abseil deps to One tweak vs your snippet: the current tree uses On "not sure if that is best way" — I think there are two options and I'd like your call:
I lean towards B as the long-term fix (keeps the "prebuilt == source" contract) with A unblocking us now — but happy to go whichever way you prefer. If B, I'll open it against toolshed. |
|
yeah dont mind either way wrt how we bundle absl - i discussed it with a bot when i was working on it in toolshed - it was on that basis i went this way - but feel free to flip it |
|
fails gcc - that might have been the reason for not bundling the symbols |
|
The previous CI failures should be fixed. |
Inject the envoy_toolshed prebuilt wee8 static library into proxy-wasm-cpp-host's v8_lib through a new //bazel:v8_engine label_flag, instead of building V8 from source (30-120min -> seconds). - bazel/proxy_wasm_cpp_host.patch: add a label_flag //bazel:v8_engine (default :wee8_no_pointer_compression, preserving upstream from-source behaviour) and point v8_lib's deps at it. - .bazelrc: set the flag to @envoy_toolshed//v8:wee8. - bazel/repositories_extra.bzl: setup_wee8_prebuilt() to define the @wee8_prebuilt_* repos. Unlike hardcoding @envoy_toolshed//v8:wee8 inside proxy-wasm's BUILD, a label_flag value is resolved in the root module's repo mapping, so this works in both WORKSPACE and bzlmod modes (strict per-module repo visibility would otherwise hide @envoy_toolshed from proxy-wasm). The @envoy_toolshed//v8:wee8 alias falls back to @v8//:wee8 on platforms without a prebuilt. Requires: envoy_toolshed >= 0.4.9 (already pinned on main). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Dario Cillerai <dcillera@redhat.com>
The toolshed prebuilt libwee8.a excludes abseil by design (consumers provide their own to avoid ODR/ABI clashes). The from-source @v8//:wee8 propagates abseil transitively, so wee8_compile linked without listing it; the prebuilt does not, so this minimal v8_lib consumer fails to link with undefined absl::* symbols. Add the abseil libs wee8 references explicitly. Uses @abseil-cpp (the repo name in the current tree) rather than the older @com_google_absl. Addresses phlax review feedback on envoyproxy#47108. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Dario Cillerai <dcillera@redhat.com>
…rebuilt repos
The published prebuilt wee8 is a clang/libc++, non-instrumented x86_64 build.
CI configs that violate those assumptions were linking/loading the wrong
artifact:
- gcc: undefined references (libstdc++ std:: vs libc++ std::__1:: mangling).
- asan/msan/tsan: wee8_compile_tool aborts building .wasm fixtures because the
prebuilt is not instrumented.
Scope the prebuilt to the config it is valid for and fall back to building wee8
from source elsewhere via the //bazel:v8_engine label_flag:
- build:gcc -> @proxy-wasm-cpp-host//bazel:wee8_no_pointer_compression
- build:sanitizer -> same (asan/msan/tsan all expand to --config=sanitizer)
Also register the prebuilt repos (wee8_prebuilt_x86_64,
wee8_prebuilt_x86_64_libstdcxx, wee8_prebuilt_aarch64) as envoy_toolshed
implied_untracked_deps so the dependency-reachability checker resolves them.
The aarch64 prebuilt is runtime-broken and still selected by the alias; that is
addressed toolshed-side since there is no per-cpu .bazelrc config hook.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Dario Cillerai <dcillera@redhat.com>
…adata merge The deps precheck (//tools/dependency:validate_reachability_test) failed with `wee8_prebuilt_x86_64` unresolved even though the three wee8_prebuilt_* repos were already added to envoy_toolshed.implied_untracked_deps in bazel/deps.yaml. The merged metadata the test consumes is built by bazel/BUILD:legacy_all_repository_locations via jq `$metadata1 * $metadata2` (metadata1 = bazel/deps.yaml, metadata2 = api/bazel/deps.yaml). jq's `*` deep-merges objects but REPLACES arrays with the right-hand operand, so api/bazel/deps.yaml's shorter implied_untracked_deps clobbered the entries. Add the same wee8_prebuilt_* entries to api/bazel/deps.yaml so they survive the merge and resolve back to the envoy_toolshed metadata entry. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Dario Cillerai <dcillera@redhat.com>
2d38855 to
94b4e2d
Compare
|
lets fix the arm artifacts in toolshed |
The prebuilt aarch64 libwee8.a shipped an x86_64 builtins snapshot embedded inside an otherwise-correct AArch64 library, so wee8_compile_tool (and any V8 consumer) SIGSEGVs on the first builtin call on real arm64 (the arm64 CI segfault in envoyproxy/envoy#47108). The wee8 packaging transition set only //command_line_option:platforms for the target arch, not V8's @v8//bazel/config:v8_target_cpu. V8's mksnapshot runs as an exec (build-host) tool but emits code for the ISA its v8_target_cpu_transition resolves, which falls back to mapping[--cpu] when the flag is "none". We never set --cpu, so on an x86_64 build host it resolved to x64 and mksnapshot emitted an x86_64 embedded blob. Set @v8//bazel/config:v8_target_cpu on the transition to the target ISA (aarch64->arm64, x86_64->x64). Validated by rebuilding the aarch64 archive and disassembling v8_Default_embedded_blob_code_ in embedded.pic.o: now AArch64 (2638 ret, 4162 stp x29,x30 prologues, 6435 nops on the 4-byte grid; x86 REX.W byte share fell from 4.3% to 0.26%), versus pure x86_64 before. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Dario Cillerai <dcillera@redhat.com>
The prebuilt aarch64 libwee8.a shipped an x86_64 builtins snapshot embedded inside an otherwise-correct AArch64 library, so wee8_compile_tool (and any V8 consumer) SIGSEGVs on the first builtin call on real arm64 (the arm64 CI segfault in envoyproxy/envoy#47108). The wee8 packaging transition set only //command_line_option:platforms for the target arch, not V8's @v8//bazel/config:v8_target_cpu. V8's mksnapshot runs as an exec (build-host) tool but emits code for the ISA its v8_target_cpu_transition resolves, which falls back to mapping[--cpu] when the flag is "none". We never set --cpu, so on an x86_64 build host it resolved to x64 and mksnapshot emitted an x86_64 embedded blob. Set @v8//bazel/config:v8_target_cpu on the transition to the target ISA (aarch64->arm64, x86_64->x64). Validated by rebuilding the aarch64 archive and disassembling v8_Default_embedded_blob_code_ in embedded.pic.o: now AArch64 (2638 ret, 4162 stp x29,x30 prologues, 6435 nops on the 4-byte grid; x86 REX.W byte share fell from 4.3% to 0.26%), versus pure x86_64 before. Signed-off-by: Dario Cillerai <dcillera@redhat.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Commit Message: wasm/v8: Source wee8 from envoy_toolshed prebuilt via label_flag
Additional Description:
This wires Envoy's proxy-wasm V8 engine to consume the prebuilt
wee8staticlibrary published by
envoy_toolshed, instead of building V8 from source(~30–120 min → seconds), while keeping the change working in both WORKSPACE
and bzlmod modes.
Mechanism — a
label_flag, not a hardcoded rewriteproxy-wasm-cpp-host's
v8_libdepends on//bazel:wee8_no_pointer_compression,which wraps
@v8//:wee8and therefore builds V8 from source. To use the prebuiltthat dep must be redirected to
@envoy_toolshed//v8:wee8.The previous WORKSPACE-only approach hardcoded that redirect inside proxy-wasm's
BUILD— which only works because WORKSPACE has a flat, global repo namespace.Under bzlmod's strict per-module repo visibility, a label inside proxy-wasm
resolves against proxy-wasm's own repo mapping, which does not declare
@envoy_toolshed(fails withNo repository visible as '@envoy_toolshed').override_repo/overrides are root-module-only, so the redirect can't live in theenvoy_toolsheddep either.The portable fix: proxy-wasm's v8 engine dep becomes a
label_flag //bazel:v8_engine(default:wee8_no_pointer_compression, preservingupstream from-source behaviour). The root module (Envoy) sets the flag value to
@envoy_toolshed//v8:wee8. A root-setlabel_flagvalue resolves in the root'srepo mapping — where
@envoy_toolshedis abazel_dep— so the same seam works inWORKSPACE and bzlmod. The patch bakes in no Envoy-specific repo, making it a
candidate to upstream into proxy-wasm-cpp-host.
This is complementary to the existing
//bazel:enginestring_flag(which selectswhich wasm engine): the new
//bazel:v8_enginelabel_flagselects which V8source (prebuilt vs from-source).
@envoy_toolshed//v8:wee8is a platform-agnostic alias that selects the rightprebuilt per platform (linux x86_64/aarch64, libcxx/libstdc++) and falls back to
@v8//:wee8from-source on unsupported platforms — so no platform loses support.Changes
bazel/proxy_wasm_cpp_host.patch: addlabel_flag(name = "v8_engine", build_setting_default = ":wee8_no_pointer_compression"); pointv8_libdeps at//bazel:v8_engine. (Regenerated from pristine proxy-wasm; round-trip verified.).bazelrc:build --@proxy-wasm-cpp-host//bazel:v8_engine=@envoy_toolshed//v8:wee8.bazel/repositories_extra.bzl:setup_wee8_prebuilt()to define the@wee8_prebuilt_*repos. Requiresenvoy_toolshed >= 0.4.9(already pinned on main).Status / follow-ups (draft PR — showcasing the milestone)
migration:
envoy_toolshedas abazel_dep, swappingsetup_wee8_prebuilt()for the
wee8_prebuilt_extensionuse_repo, and proxy-wasm gaining aMODULE.bazel. Thelabel_flagseam is ready for all of that today.label_flagto proxy-wasm-cpp-host.Risk Level: Low — default flag value preserves upstream from-source behaviour;
prebuilt path falls back to source on unsupported platforms; no BUILD/source edits
outside the patch and dependency wiring.
Testing: Full
./ci/do_ci.sh dev(docker, WORKSPACE, publishedenvoy_toolshed0.4.9 prebuilt):
envoy-staticand all test binaries link against the prebuiltlibwee8.awith no undefined symbols. 1695/1702 tests pass; the 7 failures areunrelated env/flaky integration tests (IPv6-less docker on DNS/forward-proxy cases,
a mount-permissions shell test, redis fake-upstream segfaults, one flaky geoip) — no
wasm/v8 targets among them.
cqueryon@proxy-wasm-cpp-host//:v8_libshows@wee8_prebuilt_x86_64//:wee8+lib/libwee8.aand zero@v8//references.Docs Changes: N/A
Release Notes: N/A (build/dependency wiring; no user-facing behaviour change)
Platform Specific Features: Prebuilt wee8 available for linux x86_64 (libcxx +
libstdc++) and aarch64 (libcxx); other platforms transparently fall back to building
V8 from source.