Skip to content

wasm/v8: Source wee8 from envoy_toolshed prebuilt via label_flag - #47108

Draft
dcillera wants to merge 4 commits into
envoyproxy:mainfrom
dcillera:wee8-prebuilt-bzlmod
Draft

wasm/v8: Source wee8 from envoy_toolshed prebuilt via label_flag#47108
dcillera wants to merge 4 commits into
envoyproxy:mainfrom
dcillera:wee8-prebuilt-bzlmod

Conversation

@dcillera

@dcillera dcillera commented Sep 1, 2026

Copy link
Copy Markdown

Commit Message: wasm/v8: Source wee8 from envoy_toolshed prebuilt via label_flag

Additional Description:

This wires Envoy's proxy-wasm V8 engine to consume the prebuilt wee8 static
library
published by envoy_toolshed, instead of building V8 from source
(~30–120 min → seconds), while keeping the change working in both WORKSPACE
and bzlmod
modes.

Mechanism — a label_flag, not a hardcoded rewrite

proxy-wasm-cpp-host's v8_lib depends on //bazel:wee8_no_pointer_compression,
which wraps @v8//:wee8 and therefore builds V8 from source. To use the prebuilt
that dep must be redirected to @envoy_toolshed//v8:wee8.

The previous WORKSPACE-only approach hardcoded that redirect inside proxy-wasm's
BUILD — which only works because WORKSPACE has a flat, global repo namespace.
Under bzlmod's strict per-module repo visibility, a label inside proxy-wasm
resolves against proxy-wasm's own repo mapping, which does not declare
@envoy_toolshed (fails with No repository visible as '@envoy_toolshed').
override_repo/overrides are root-module-only, so the redirect can't live in the
envoy_toolshed dep either.

The portable fix: proxy-wasm's v8 engine dep becomes a
label_flag //bazel:v8_engine (default :wee8_no_pointer_compression, preserving
upstream from-source behaviour). The root module (Envoy) sets the flag value to
@envoy_toolshed//v8:wee8. A root-set label_flag value resolves in the root's
repo mapping — where @envoy_toolshed is a bazel_dep — so the same seam works in
WORKSPACE and bzlmod. The patch bakes in no Envoy-specific repo, making it a
candidate to upstream into proxy-wasm-cpp-host.

This is complementary to the existing //bazel:engine string_flag (which selects
which wasm engine): the new //bazel:v8_engine label_flag selects which V8
source
(prebuilt vs from-source).

@envoy_toolshed//v8:wee8 is a platform-agnostic alias that selects the right
prebuilt per platform (linux x86_64/aarch64, libcxx/libstdc++) and falls back to
@v8//:wee8 from-source on unsupported platforms
— so no platform loses support.

Changes

  • bazel/proxy_wasm_cpp_host.patch: add label_flag(name = "v8_engine", build_setting_default = ":wee8_no_pointer_compression"); point v8_lib deps at
    //bazel:v8_engine. (Regenerated from pristine proxy-wasm; round-trip verified.)
  • .bazelrc: build --@proxy-wasm-cpp-host//bazel:v8_engine=@envoy_toolshed//v8:wee8.
  • bazel/repositories_extra.bzl: setup_wee8_prebuilt() to define the
    @wee8_prebuilt_* repos. Requires envoy_toolshed >= 0.4.9 (already pinned on main).

Status / follow-ups (draft PR — showcasing the milestone)

  • ✅ Works end-to-end in the real Envoy tree today (WORKSPACE mode). See Testing.
  • ⬜ Full Envoy-in-bzlmod wiring is staged follow-up gated on the broader bzlmod
    migration: envoy_toolshed as a bazel_dep, swapping setup_wee8_prebuilt()
    for the wee8_prebuilt_extension use_repo, and proxy-wasm gaining a
    MODULE.bazel. The label_flag seam is ready for all of that today.
  • ⬜ Optionally upstream the generic label_flag to proxy-wasm-cpp-host.

Risk Level: Low — default flag value preserves upstream from-source behaviour;
prebuilt path falls back to source on unsupported platforms; no BUILD/source edits
outside the patch and dependency wiring.

Testing: Full ./ci/do_ci.sh dev (docker, WORKSPACE, published envoy_toolshed
0.4.9 prebuilt): envoy-static and all test binaries link against the prebuilt
libwee8.a with no undefined symbols. 1695/1702 tests pass; the 7 failures are
unrelated env/flaky integration tests (IPv6-less docker on DNS/forward-proxy cases,
a mount-permissions shell test, redis fake-upstream segfaults, one flaky geoip) — no
wasm/v8 targets among them. cquery on @proxy-wasm-cpp-host//:v8_lib shows
@wee8_prebuilt_x86_64//:wee8 + lib/libwee8.a and zero @v8// references.

Docs Changes: N/A

Release Notes: N/A (build/dependency wiring; no user-facing behaviour change)

Platform Specific Features: Prebuilt wee8 available for linux x86_64 (libcxx +
libstdc++) and aarch64 (libcxx); other platforms transparently fall back to building
V8 from source.

@repokitteh-read-only

Copy link
Copy Markdown

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #47108 was opened by dcillera.

see: more, trace.

@dcillera
dcillera deployed to external-contributors September 1, 2026 08:01 — with GitHub Actions Active
@phlax

phlax commented Sep 1, 2026

Copy link
Copy Markdown
Member

i think the absl libs have to be added at consumer with current toolshed build - not sure if that is best way - but something like this should probs work ...

envoy_cc_library(
    name = "wee8_compile_lib",
    srcs = ["wee8_compile.cc"],
    copts = [
        "-Wno-comments",
        "-Wno-non-virtual-dtor",
        "-Wno-unused-parameter",
    ],
    deps = [
        "@com_google_absl//absl/container:flat_hash_map",
        "@com_google_absl//absl/container:flat_hash_set",
        "@com_google_absl//absl/hash",
        "@com_google_absl//absl/synchronization",
        "@proxy-wasm-cpp-host//:base_lib",
        "@proxy-wasm-cpp-host//:v8_lib",
    ],
)

dcillera added a commit to dcillera/envoy-1 that referenced this pull request Sep 1, 2026
The toolshed prebuilt libwee8.a excludes abseil by design (consumers
provide their own to avoid ODR/ABI clashes). The from-source @v8//:wee8
propagates abseil transitively, so wee8_compile linked without listing it;
the prebuilt does not, so this minimal v8_lib consumer fails to link with
undefined absl::* symbols. Add the abseil libs wee8 references explicitly.

Uses @abseil-cpp (the repo name in the current tree) rather than the older
@com_google_absl. Addresses phlax review feedback on envoyproxy#47108.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Dario Cillerai <dcillera@redhat.com>
@dcillera
dcillera deployed to external-contributors September 1, 2026 13:35 — with GitHub Actions Active
@dcillera

dcillera commented Sep 1, 2026

Copy link
Copy Markdown
Author

Thanks @phlax — done in the latest push. Added the abseil deps to test/tools/wee8_compile so the prebuilt (which excludes abseil from libwee8.a by design) links; the from-source @v8//:wee8 was propagating those transitively, which is why the bare v8_lib consumer worked before.

One tweak vs your snippet: the current tree uses @abseil-cpp//absl/... rather than @com_google_absl//..., so I used that.

On "not sure if that is best way" — I think there are two options and I'd like your call:

  • A (this PR): abseil-at-consumer. Works, minimal; in practice wee8_compile is the only bare consumer (envoy-static already links all of abseil, so the full do_ci.sh dev was green). Downside: every future minimal v8_lib consumer has to remember to add the abseil deps.
  • B (drop-in parity): abseil re-exposed by the prebuilt. Make toolshed's prebuilt cc_library wrapper propagate the same abseil deps the source @v8//:wee8 exposes, so no consumer needs changes and the prebuilt is a true drop-in. Needs a toolshed change + release, and has to handle the repo-name difference across modes (@com_google_absl in WORKSPACE vs @abseil-cpp in bzlmod) so the wrapper's dep resolves in both.

I lean towards B as the long-term fix (keeps the "prebuilt == source" contract) with A unblocking us now — but happy to go whichever way you prefer. If B, I'll open it against toolshed.

@phlax

phlax commented Sep 1, 2026

Copy link
Copy Markdown
Member

yeah dont mind either way wrt how we bundle absl - i discussed it with a bot when i was working on it in toolshed - it was on that basis i went this way - but feel free to flip it

@phlax

phlax commented Sep 1, 2026

Copy link
Copy Markdown
Member

fails gcc - that might have been the reason for not bundling the symbols

@dcillera
dcillera deployed to external-contributors September 1, 2026 15:19 — with GitHub Actions Active
@dcillera

dcillera commented Sep 2, 2026

Copy link
Copy Markdown
Author

The previous CI failures should be fixed.
Still outstanding (toolshed-side, not fixable in Envoy): the arm64 wee8_compile_tool segfault from the runtime-broken aarch64 prebuilt — needs the aarch64 artifact repaired or the @envoy_toolshed//v8:wee8 alias to fall back to source for aarch64.

dcillera and others added 4 commits September 2, 2026 10:13
Inject the envoy_toolshed prebuilt wee8 static library into
proxy-wasm-cpp-host's v8_lib through a new //bazel:v8_engine label_flag,
instead of building V8 from source (30-120min -> seconds).

- bazel/proxy_wasm_cpp_host.patch: add a label_flag //bazel:v8_engine
  (default :wee8_no_pointer_compression, preserving upstream from-source
  behaviour) and point v8_lib's deps at it.
- .bazelrc: set the flag to @envoy_toolshed//v8:wee8.
- bazel/repositories_extra.bzl: setup_wee8_prebuilt() to define the
  @wee8_prebuilt_* repos.

Unlike hardcoding @envoy_toolshed//v8:wee8 inside proxy-wasm's BUILD, a
label_flag value is resolved in the root module's repo mapping, so this
works in both WORKSPACE and bzlmod modes (strict per-module repo
visibility would otherwise hide @envoy_toolshed from proxy-wasm). The
@envoy_toolshed//v8:wee8 alias falls back to @v8//:wee8 on platforms
without a prebuilt.

Requires: envoy_toolshed >= 0.4.9 (already pinned on main).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Dario Cillerai <dcillera@redhat.com>
The toolshed prebuilt libwee8.a excludes abseil by design (consumers
provide their own to avoid ODR/ABI clashes). The from-source @v8//:wee8
propagates abseil transitively, so wee8_compile linked without listing it;
the prebuilt does not, so this minimal v8_lib consumer fails to link with
undefined absl::* symbols. Add the abseil libs wee8 references explicitly.

Uses @abseil-cpp (the repo name in the current tree) rather than the older
@com_google_absl. Addresses phlax review feedback on envoyproxy#47108.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Dario Cillerai <dcillera@redhat.com>
…rebuilt repos

The published prebuilt wee8 is a clang/libc++, non-instrumented x86_64 build.
CI configs that violate those assumptions were linking/loading the wrong
artifact:
  - gcc: undefined references (libstdc++ std:: vs libc++ std::__1:: mangling).
  - asan/msan/tsan: wee8_compile_tool aborts building .wasm fixtures because the
    prebuilt is not instrumented.

Scope the prebuilt to the config it is valid for and fall back to building wee8
from source elsewhere via the //bazel:v8_engine label_flag:
  - build:gcc         -> @proxy-wasm-cpp-host//bazel:wee8_no_pointer_compression
  - build:sanitizer   -> same (asan/msan/tsan all expand to --config=sanitizer)

Also register the prebuilt repos (wee8_prebuilt_x86_64,
wee8_prebuilt_x86_64_libstdcxx, wee8_prebuilt_aarch64) as envoy_toolshed
implied_untracked_deps so the dependency-reachability checker resolves them.

The aarch64 prebuilt is runtime-broken and still selected by the alias; that is
addressed toolshed-side since there is no per-cpu .bazelrc config hook.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Dario Cillerai <dcillera@redhat.com>
…adata merge

The deps precheck (//tools/dependency:validate_reachability_test) failed with
`wee8_prebuilt_x86_64` unresolved even though the three wee8_prebuilt_* repos
were already added to envoy_toolshed.implied_untracked_deps in bazel/deps.yaml.

The merged metadata the test consumes is built by
bazel/BUILD:legacy_all_repository_locations via jq `$metadata1 * $metadata2`
(metadata1 = bazel/deps.yaml, metadata2 = api/bazel/deps.yaml). jq's `*`
deep-merges objects but REPLACES arrays with the right-hand operand, so
api/bazel/deps.yaml's shorter implied_untracked_deps clobbered the entries.

Add the same wee8_prebuilt_* entries to api/bazel/deps.yaml so they survive the
merge and resolve back to the envoy_toolshed metadata entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Dario Cillerai <dcillera@redhat.com>
@dcillera
dcillera force-pushed the wee8-prebuilt-bzlmod branch from 2d38855 to 94b4e2d Compare September 2, 2026 08:14
@dcillera
dcillera deployed to external-contributors September 2, 2026 08:14 — with GitHub Actions Active
@phlax

phlax commented Sep 2, 2026

Copy link
Copy Markdown
Member

lets fix the arm artifacts in toolshed

dcillera added a commit to dcillera/toolshed that referenced this pull request Sep 2, 2026
The prebuilt aarch64 libwee8.a shipped an x86_64 builtins snapshot embedded
inside an otherwise-correct AArch64 library, so wee8_compile_tool (and any V8
consumer) SIGSEGVs on the first builtin call on real arm64 (the arm64 CI
segfault in envoyproxy/envoy#47108).

The wee8 packaging transition set only //command_line_option:platforms for the
target arch, not V8's @v8//bazel/config:v8_target_cpu. V8's mksnapshot runs as
an exec (build-host) tool but emits code for the ISA its v8_target_cpu_transition
resolves, which falls back to mapping[--cpu] when the flag is "none". We never
set --cpu, so on an x86_64 build host it resolved to x64 and mksnapshot emitted
an x86_64 embedded blob.

Set @v8//bazel/config:v8_target_cpu on the transition to the target ISA
(aarch64->arm64, x86_64->x64). Validated by rebuilding the aarch64 archive and
disassembling v8_Default_embedded_blob_code_ in embedded.pic.o: now AArch64
(2638 ret, 4162 stp x29,x30 prologues, 6435 nops on the 4-byte grid; x86 REX.W
byte share fell from 4.3% to 0.26%), versus pure x86_64 before.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Dario Cillerai <dcillera@redhat.com>
phlax pushed a commit to envoyproxy/toolshed that referenced this pull request Sep 2, 2026
The prebuilt aarch64 libwee8.a shipped an x86_64 builtins snapshot embedded
inside an otherwise-correct AArch64 library, so wee8_compile_tool (and any V8
consumer) SIGSEGVs on the first builtin call on real arm64 (the arm64 CI
segfault in envoyproxy/envoy#47108).

The wee8 packaging transition set only //command_line_option:platforms for the
target arch, not V8's @v8//bazel/config:v8_target_cpu. V8's mksnapshot runs as
an exec (build-host) tool but emits code for the ISA its v8_target_cpu_transition
resolves, which falls back to mapping[--cpu] when the flag is "none". We never
set --cpu, so on an x86_64 build host it resolved to x64 and mksnapshot emitted
an x86_64 embedded blob.

Set @v8//bazel/config:v8_target_cpu on the transition to the target ISA
(aarch64->arm64, x86_64->x64). Validated by rebuilding the aarch64 archive and
disassembling v8_Default_embedded_blob_code_ in embedded.pic.o: now AArch64
(2638 ret, 4162 stp x29,x30 prologues, 6435 nops on the 4-byte grid; x86 REX.W
byte share fell from 4.3% to 0.26%), versus pure x86_64 before.

Signed-off-by: Dario Cillerai <dcillera@redhat.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants