Skip to content

Bazel.md

Latest commit

 

History

History
775 lines (547 loc) · 34.9 KB

Bazel.md

File metadata and controls

775 lines (547 loc) · 34.9 KB
Raw

Building and testing with Bazel

Bazel is the only supported build system for OpenROAD.
For installation steps, see [Installing OpenROAD](Build.md).

First install Bazelisk or run sudo ./etc/DependencyInstaller.sh -bazel, then you're ready for the main use-case of Bazel, which is to make modifications to OpenROAD and run fast local tests before creating a PR:

bazelisk test --jobs=4 src/...
  • ... means everything below this folder, so use src/gpl/... to run a smaller set of tests.
  • --jobs=4 limits parallel builds to 4 cores, default is use all cores.

For more comprehensive testing locally, including longer OpenROAD integration tests, run:

bazelisk test ...

Note! You'll see bazel in examples and documentation as well as bazelisk. The latter is a wafer thin layer on top of bazel that reads in the .bazelversion file to decide which version of Bazel to use in bazel.

A word on expectations: Bazel is a valuable skill for the future. It is an example of a new generation of build tools(Buck2 is another example) that scales very well, are hermetic and have many features, such as artifacts. Unsurprisingly, this does mean that there is a lot to learn. The OpenROAD documentation makes no attempt at teaching Bazel. Bazel is a very wide topic and it can not be learned in a day or two with intense reading of a well defined document with a start and an end. It is probably best to start as a user running canned commands, such as above, but switch from mechanical repetition of canned command to being curious and following breadcrumbs of interest: read, search, engage with the community and use AI to learn.

Running specific tests or tests below a folder

To list all tests use bazelisk query with Bazel query language:

bazelisk query 'kind(test, ...)'

List all tests below a folder containing asap7:

$ bazelisk query   'filter("asap7", kind(".*_test rule", //src/pdn/...))'
//src/pdn/test:asap7_M1_M3_followpins-tcl_test
...

Run specific test:

bazelisk test --test_output=errors //src/upf/test:levelshifter-tcl_test

To run or list tests below a folder:

bazelisk test src/gpl/...

Build configurations

Bazel build configuration is a big topic as it covers cross compilation as well as multiplatform support.

As a start, browse .bazelrc and also run:

$ bazelisk build
[wait until it starts building then hit ctrl-c, this warms up the cache with information about available configs]
$ bazelisk config

The output lists each configuration with a full hash and a textual shorthand, such as k8-opt or k8-opt-exec-ST-<hash> (exec). The shorthand is used in the names of the bazel-out/ folders.

  • Each configuration has a textual shorthand used in folder names followed by a full hash
  • opt is the default compilation mode (build -c opt in .bazelrc); the other modes are fastbuild and dbg
  • exec means host, appears with ST and an extra hash at the end
  • k8 always there, possibly referring to K8

LTO

The default Bazel build is -c opt, the optimized compilation mode. It does not add -O3 or Link Time Optimization (LTO). --config=opt (build:opt in .bazelrc) adds -O3 and ThinLTO. ThinLTO improves runtime at the cost of a longer link step, which Bazel splits into parallel, cached actions. --config=release includes --config=opt and also stamps the git version. Use --config=opt or --config=release for production binaries shipped to end users; keep the default for the local edit-rebuild loop.

Is it compiling, or hitting the cache?

The console counter advances for cache hits and real compiles alike:

[7,203 / 18,048] Compiling src/dpl/src/objective/detailed_abu.cxx; 20s disk-cache, remote-cache, processwrapper-sandbox ... (16 actions, 12 running)

The strategy list after the elapsed time is the sequence of stages the action has passed through, so the last entry is what is happening right now. Above, the disk cache missed, the remote cache missed, and the compiler has been running for 20s under the process-wrapper sandbox. An action served from a cache never reaches a local strategy (processwrapper-sandbox, linux-sandbox, local) and disappears within a fraction of a second.

The end-of-build line gives the breakdown for the whole build, with no flags needed:

INFO: 2 processes: 1229 action cache hit, 1 disk cache hit, 1 internal.        # nothing compiled
INFO: 2 processes: 1229 action cache hit, 1 internal, 1 processwrapper-sandbox. # one real compile

Three different mechanisms are summarized there:

Runner Meaning
action cache hit Bazel's local action cache; the spawn never ran and never appears in the progress display
disk cache hit, remote cache hit the action ran, but its outputs were fetched from --disk_cache/--remote_cache instead of being built
processwrapper-sandbox, linux-sandbox, local, worker actually compiled here

For which actions those were, and why they were not up to date:

bazelisk build --config=cachelog //src/dpl:dpl
etc/bazel-cache-summary.py
bazel-cachelog.json: 3 spawns

  ran for real: 3 spawns, 16.6s of wall time
    processwrapper-sandbox              3       16.6s

Slowest of the 3 spawns that ran for real:
       5.8s  CppCompile     src/dpl/src/objective/detailed_abu.cxx
       ...

bazel-cachelog-explain.txt: why actions were not up to date
         3  action changed since cached execution
            e.g. Compiling src/dpl/src/objective/detailed_hpwl.cxx

--config=cachelog writes an execution log recording the runner that served every spawn. Each record carries the spawn's full input list -- a few MB per executed action -- so use it on incremental builds rather than on a build of everything. --config=explain is the cheap half on its own: it writes only the per-action reason, a few bytes per action.

Using OpenROAD as a dependency from another project

OpenROAD can be consumed as a Bazel module (bazel_dep) from another project. The public API consists of two targets:

Target Description
@openroad//:openroad The CLI binary
@openroad//:openroad_py Python bindings for scripting

All other targets (e.g. openroad_lib, internal libraries) are restricted to OpenROAD's own subpackages and are not part of the public API.

Minimal MODULE.bazel for a downstream project

module(name = "my-project")

bazel_dep(name = "openroad")
git_override(
    module_name = "openroad",
    commit = "<commit-hash>",
    init_submodules = True,
    remote = "https://github.com/The-OpenROAD-Project/OpenROAD.git",
)

# qt-bazel is not in BCR; git_override is root-module-only,
# so downstream consumers must repeat it.
bazel_dep(name = "qt-bazel")
git_override(
    module_name = "qt-bazel",
    commit = "df022f4ebaa4130713692fffd2f519d49e9d0b97",
    remote = "https://github.com/The-OpenROAD-Project/qt_bazel_prebuilts",
)

Suggested: pin the C++ toolchain for reproducibility

OpenROAD uses hermetic-llvm (BCR module llvm) internally to lock the compiler version and ensure reproducible builds across developers and CI: statically linked LLVM binaries and a zero-sysroot cc_toolchain, so no host compiler, headers or libraries are involved. Downstream consumers can use any C++20-capable compiler, but pinning the same toolchain is recommended to avoid compiler-specific issues:

bazel_dep(name = "llvm", version = "0.8.14")

register_toolchains("@llvm//toolchain:all")

Dev dependencies not leaked to consumers

The following are dev_dependency in OpenROAD and will not be forced on downstream projects via MVS:

  • rules_pkg — only needed for //:install
  • llvm (hermetic-llvm) toolchain registration

The downstream test at test/downstream/ verifies these invariants.

Python API (the openroad wheel)

OpenROAD's Python API ships as a wheel rather than an openroad -python mode. Build it and install it into a virtualenv (CPython 3.10 or later, Linux x86_64):

bazelisk build //python/openroad:openroad_wheel
pip install bazel-bin/python/openroad/openroad-*.whl

Then import it from any Python interpreter:

from openroad import Tech, Design
import odb

tech = Tech()

import openroad also loads the per-tool modules (odb, grt, rsz, ...). Keep each Tech alive while its Design is in use.

Build without testing

bazelisk build :openroad

Version stamping

By default, Bazel dev builds use a fixed placeholder version (bazel-nostamp) so that the build output is deterministic and fully cacheable across commits.

To embed the real git version (e.g. 26Q1-1486-g6fe48208e4), use --config=release:

bazelisk build --config=release :openroad
./bazel-bin/openroad -version

A release build reads the version with git describe, and gives unknown when the tree has no git metadata. Set OPENROAD_VERSION to supply the version directly:

OPENROAD_VERSION=26Q1 bazelisk build --config=release :openroad

The Docker build needs this, because .dockerignore keeps .git out of the build context. Pass the version with --build-arg orVersion=<version>.

etc/Build.sh always passes --config=release and forwards OPENROAD_VERSION, so installs made through it (including ORFS build_openroad.sh and the Docker build) carry the real version.

The version is written into the binary after it is linked (bazel/stamped_binary.bzl). The version therefore changes no compile or link action, and a release build gets them from the remote cache that CI fills; only the copy of the binary that carries the version is built locally.

Platforms

https://bazel.build/extending/platforms

bazelisk build :openroad and the tests use the same cfg=target binary, so building first and testing afterwards does not rebuild OpenROAD. See Bazel targets.

Can I force a rebuild?

You should never have to in Bazel as builds are robust, trust the caching...

That said:

bazelisk clean

Or more forcefully:

bazelisk clean --expunge

Or "nuke it from orbit":

bazelisk shutdown
sudo pkill -9 java
sudo rm -rf ~/.cache/bazel

Run tests with the address sanitizer:

bazelisk test --config=asan --test_tag_filters=-py src/...

Or to get an instrumented binary to run under ORFS:

bazelisk build --config=asan :openroad

The first --config=asan invocation builds compiler-rt's asan runtime from source, so expect a few minutes before any OpenROAD source is compiled.

Instrumented code runs roughly 2x slower and uses several times the memory, so prefer the smallest design that reproduces the problem.

ASan bundles LeakSanitizer, which reports every allocation still reachable at exit. OpenROAD keeps a great deal of state for the life of the process -- pdn::PdnGen holds its grid vias, sta::TimingArcSet its arcs -- and none of it is freed before main returns, so LSan reports all of it. With leak checking on, 2648 of 3997 tests fail that way and 17 on an actual memory error, which is the wrong signal to leave on by default. The config therefore sets detect_leaks=0. To go leak hunting anyway:

bazelisk test --config=asan --test_tag_filters=-py --test_env=ASAN_OPTIONS=detect_leaks=1 src/...

--test_tag_filters=-py skips the Python tests; see "Sanitizers and the Python extension modules" below for why.

Known findings

A --config=asan --test_tag_filters=-py src/... sweep reports 1053 of 3997 tests failing, but all but 19 come from one external binary.

The hierarchy conformance suite and TestLec shell out to kepler-formal, which is a prebuilt, uninstrumented binary. It trips a container-overflow report and exits 1, failing 1034 tests that have nothing wrong with them:

Origin Tests
//src/dbSta/test:dbsta_hier_conformance_case_* 1033
//src/tst:TestLec 1

Both pass under --test_env=ASAN_OPTIONS=detect_leaks=0:detect_container_overflow=0. The config does not set that globally, because the container-overflow asan reports in OpenROAD's own code are real -- see ScanStitch.cpp below.

That leaves 18 tests on a genuine finding, to be fixed separately:

Site Check Tests
eigen Core/arch/SSE/PacketMath.h, Core/functors/AssignmentFunctors.h heap-buffer-overflow 10, via psm
src/dft/src/stitch/ScanStitch.cpp:102 container-overflow 5
src/sta/search/Path.cc:383 heap-use-after-free 1, via dbSta
src/odb/include/odb/dbCommon.h:26 stack-use-after-scope 1, in web
spdlog::logger::should_log reached through utl::Logger SEGV 1, in web

The dft one is a plain read past the end: the loop at ScanStitch.cpp:100 runs while it != scan_cells.end() and dereferences *(it + 1), so the last iteration reads end().

One further test, //src/grt/test:repair_antennas_post_drt_cugr-tcl_test, times out rather than failing -- instrumented code is slow enough to push it past its limit.

Run tests with the thread sanitizer:

bazelisk test --config=tsan --test_tag_filters=-py src/...

Or to get an instrumented binary to run under ORFS:

bazelisk build --config=tsan :openroad

The first --config=tsan invocation builds compiler-rt's tsan runtime from source, so expect a few minutes before any OpenROAD source is compiled.

Instrumented code runs roughly 5-15x slower and uses far more memory, so prefer the smallest design that reproduces the race. A full src/... run at Bazel's default parallelism can exhaust RAM and get the build OOM-killed; throttle it if that happens:

bazelisk test --config=tsan --test_tag_filters=-py --jobs=16 --local_test_jobs=8 src/...

Adjust the runtime via TSAN_OPTIONS, e.g. to keep going past the first report and get the second stack of a lock-order inversion:

bazelisk test --config=tsan --test_tag_filters=-py --test_env=TSAN_OPTIONS="halt_on_error=0 second_deadlock_stack=1" src/...

drt, gpl, grt and ant parallelize with OpenMP. The @openmp runtime is instrumented along with everything else under --config=tsan, but it is not built with OpenMP's TSan annotations (LIBOMP_TSAN_SUPPORT), so races reported inside __kmp_* frames are artifacts of the barrier implementation rather than OpenROAD bugs. Confirm a finding by checking that both stacks land in OpenROAD code.

Known findings

A --config=tsan --test_tag_filters=-py src/... sweep reports 36 of 3980 tests failing, and the OpenMP artifacts above are almost all of it:

Origin Tests
@openmp runtime/src/kmp_runtime.cpp, kmp_wait_release.h 33, across drt, grt, ram, rcx, gpl
src/sta/graph/Graph.cc:1288 1, via rmp
boost::asio scheduler.ipp:187 1, in dst

Silencing the OpenMP group means building @openmp with LIBOMP_TSAN_SUPPORT, which is a change to that module rather than something this repo can pass as a flag. The remaining two are real: the dst one is a test that destroys a stack io_context while its thread still runs it, and the OpenSTA one is upstream.

--test_tag_filters=-py skips the Python tests; see "Sanitizers and the Python extension modules" below for why.

Run tests with the undefined behavior sanitizer:

bazelisk test --config=ubsan --test_tag_filters=-py src/...

Or to get an instrumented binary to run under ORFS:

bazelisk build --config=ubsan :openroad

UBSan is much cheaper than asan/tsan -- a small constant slowdown, no memory overhead -- so it is the cheapest of the three to leave running over a real design.

The config builds with -fno-sanitize-recover=all, so the first finding aborts the process. That is deliberate: -fsanitize=undefined on its own only prints a report and lets the process run on to exit 0, which would leave a test suite green with the reports buried in the logs. To survey everything a run would hit instead of stopping at the first, opt back into recovery:

bazelisk test --config=ubsan --test_tag_filters=-py --copt=-fsanitize-recover=all src/...

Reports name the check that fired (e.g. signed-integer-overflow, misaligned-address). An individual check can be switched off project-wide with a copt, which is preferable to disabling the config wholesale:

bazelisk test --config=ubsan --test_tag_filters=-py --copt=-fno-sanitize=vptr src/...

Known findings

A --config=ubsan --test_tag_filters=-py src/... sweep is not clean yet. As of this writing 36 of 3980 tests fail, in two groups.

Third-party UB reached through headers that are inlined into OpenROAD translation units, 24 tests:

Origin Check Tests
boost.geometry strategies/cartesian/intersection.hpp undefined-behavior 19, via pad::RDLRouter::isEdgeObstructed
coin-or-lemon lemon/network_simplex.h signed-integer-overflow 5, in cts

--per_file_copt=.*external/.*@-fno-sanitize=all above exempts third-party source files, which is why abc, tcl and bliss no longer report. It cannot exempt a third-party header compiled as part of our own translation unit -- the same limitation the -w per_file_copt has. Doing that needs an -fsanitize-ignorelist, which has to be declared as a compile action input, so it belongs in the toolchain rather than in this file.

OpenROAD's own UB, 11 tests, to be fixed separately:

Site Check Tests
src/rcx/src/netRC.cpp:374, :1575, :1576 load of value 8
src/odb/include/odb/geom.h:373 signed-integer-overflow 2
src/grt/src/cugr/src/geo.h:111 signed-integer-overflow 1

Sanitizers and the Python extension modules

--config=asan, --config=tsan and --config=ubsan cover the C++ and Tcl tests, which run the instrumented openroad binary. They do not work for the py-tagged tests, which import OpenROAD as a Python extension module:

ImportError: _openroad.so: undefined symbol: __ubsan_handle_pointer_overflow_abort

Clang links a sanitizer runtime into executables but not into shared libraries, assuming whoever loads the library provides the symbols. That holds for openroad, which statically links the runtime; it fails for _openroad.so, which the uninstrumented system python3 dlopens.

-shared-libsan is the mechanism for this case, but three things in hermetic-llvm block it today:

  1. only the static archives are staged into clang's resource directory, so the driver cannot find libclang_rt.<san>.so (asan stages both, ubsan and tsan stage static only);
  2. the shared runtimes are built by cc_shared_library with sonames like libubsan_standalone.shared.so, so a consumer records a DT_NEEDED on a name that does not match the staged libclang_rt.ubsan_standalone.so;
  3. the runtime arrives via a linkopt rather than a dep, so it lands in neither the test's runfiles nor its RPATH.

Fixing this belongs upstream in hermetic-llvm. Until then, skip them with --test_tag_filters=-py; the Tcl tests cover the same C++ code paths as their Python counterparts.

The Tcl tests do get instrumented: they run the cfg=target openroad, which is what the sanitizer configs instrument. swig, bison and the other exec-configuration build tools stay uninstrumented.

GPU build (--config=gpu)

--config=gpu compiles the Kokkos/CUDA backends of gpl (src/gpl/src/gpu). Kokkos 5.2.2 and KokkosFFT 2.0.0 are downloaded at pinned checksums and built with Bazel's hermetic LLVM toolchain, with the OpenMP, Serial, and CUDA backends enabled (bazel/gpu/kokkos/configure.bzl). Only the CUDA toolkit remains host-provided. Default CPU targets neither download nor compile Kokkos and do not compile or link against CUDA. Because the CUDA repository definition is loaded by shared Bazel configuration, CPU invocations still probe the configured toolkit path.

Choose the configuration matching the device compute capability:

bazelisk test --config=gpu-sm120 //src/gpl/test/...
bazelisk test --config=gpu-sm121 //src/gpl/test/...

nvidia-smi --query-gpu=compute_cap --format=csv,noheader prints the compute capability. Remove the decimal point to form the Bazel value (12.0 -> sm_120, 12.1 -> sm_121). sm_121 needs CUDA 12.9 or newer; with CUDA 13 also set OPENROAD_CUDA_PATH to that toolkit (its libcu++ headers live in include/cccl/, which --config=gpu already adds to the search path).

For another supported architecture, use --config=gpu together with --//:cuda_arch=sm_XX. The accepted values are sm_60, sm_61, sm_70, sm_72, sm_75, sm_80, sm_86, sm_87, sm_89, sm_90, sm_100, sm_103, sm_120, and sm_121. Plain --config=gpu intentionally requires this extra flag so Kokkos and OpenROAD's kernels use one explicit target. If the compiled target differs from the selected device, Kokkos prints a runtime warning; compatible code may still run, usually with reduced portability or performance.

OPENROAD_CUDA_PATH selects the full CUDA toolkit and defaults to /usr/local/cuda-12.8. The toolkit must contain bin/ptxas, nvvm/libdevice, libcudart, and libcufft. A missing or incomplete toolkit never breaks the CPU build: its repository becomes a stub, and only a GPU build fails during analysis with an actionable message. The installed NVIDIA driver must support the CUDA toolkit used for the build.

Dependency model

The Bazel build does not use KOKKOS_ROOT or KOKKOS_FFT_ROOT. Its BUILD overlays compile Kokkos with the same clang, libc++, glibc headers, CUDA target, and C++ standard as OpenROAD. This avoids the C++ ABI and libc symbol mismatches caused by linking a host-built Kokkos into a hermetic OpenROAD binary. The configuration headers that Kokkos's and KokkosFFT's CMake builds generate (KokkosCore_config.h, the backend include lists, desul/atomics/Config.hpp, KokkosFFT_config.hpp) are written at build time from one backend list in bazel/gpu/kokkos/configure.bzl; none is checked in.

Notes

  • The runtime gate is the ENABLE_GPU environment variable, read by gpl::gpuEnabled(). It defaults to on when the GPU code is compiled in, so every regression test pins ENABLE_GPU=0 on a GPU build to stay on its CPU golden logs (test/regression.bzl); the GPU-only tests (region01_gpu, region01_gpu_asym, fft_gpu_test, wl_gpu_test) pin it to 1, are tagged gpu, and are reported as SKIPPED by plain CPU wildcard runs.
  • When the GPU path first initializes Kokkos, its OpenMP backend prints Kokkos::OpenMP::initialize WARNING: OMP_PROC_BIND environment variable not set on stderr unless OMP_PROC_BIND is set (for example to false). It never reaches the golden logs: those tests pin ENABLE_GPU=0, so Kokkos is not initialized, and the GPU-only tests are pass/fail. A Kokkos warning about a CUDA architecture mismatch, by contrast, is worth acting on.
  • Host-side FFTs on the GPU path run on Kokkos::Serial. The Bazel @fftw has no threads library, so FFTW's threaded planner is unavailable.
  • --config=gpu binaries link libcudart/libcufft by absolute path with an rpath into the toolkit; they are not relocatable to another machine.
  • The first GPU build downloads the two pinned source archives and compiles Kokkos's CUDA translation units. Changing cuda_arch rebuilds Kokkos and the OpenROAD GPU sources for the new target.
  • After upgrading CUDA in place, run bazelisk shutdown (or change OPENROAD_CUDA_PATH) so the wrapped repository is refetched with the new file list.

Testing an OpenROAD build with ORFS from within the OpenROAD folder

bazelisk build :openroad
OPENROAD_EXE=$(pwd)/bazel-bin/openroad make --dir ~/OpenROAD-flow-scripts/flow/ DESIGN_CONFIG=designs/asap7/gcd/config.mk clean_floorplan floorplan

bazel-bin/openroad is the optimized binary that bazelisk build :openroad produces. Tests such as bazelisk test ... build OpenROAD from the same sources and flags, but in the execution configuration (see Some OpenROAD and OpenSTA Bazel Specifics).

Testing debug build of OpenROAD with ORFS

Build OpenROAD in debug mode using a workaround:

bazelisk build --cxxopt=-stdlib=libstdc++ --linkopt=-lstdc++ -c dbg :openroad

Run ORFS flow and use debugger as usual for ORFS:

OPENROAD_EXE=$(pwd)/bazel-out/k8-dbg/bin/openroad make --dir ~/OpenROAD-flow-scripts/flow/ DESIGN_CONFIG=designs/asap7/gcd/config.mk clean_floorplan floorplan

Profiling OpenROAD with an ORFS build and your favorite profiling tool

Build an optimized profile binary, using a workaround:

bazelisk build --config=profile --cxxopt=-stdlib=libstdc++ --linkopt=-lstdc++ :openroad

bazel-bin points to the results of the most recent bazelisk build. If you are switching between various builds, the more robust alternative is to point OPENROAD_EXE to the specific build configuration you want in bazel-out.

Start an ORFS job that you want to profile:

OPENROAD_EXE=$(pwd)/bazel-bin/openroad make --dir ~/OpenROAD-flow-scripts/flow/ DESIGN_CONFIG=designs/asap7/gcd/config.mk clean_floorplan floorplan

At this point, use your favorite preformance tool, such as the Linx Perf tool:

perf top

Perhaps attach gdb and use ctrl-c from the command line? Use gdb with an IDE, emacs, or vim?

$ gdb bazel-bin/openroad
[deleted]
(gdb) attach 578603
Attaching to program: /home/oyvind/.cache/bazel/_bazel_oyvind/896cc02f64446168f604c13ad7b60f8b/execroot/_main/bazel-out/k8-fastbuild/bin/openroad, process 578603
#7  0x00005efc3b72b865 in isPolygonCorner () at src/drt/src/gc/FlexGC_init.cpp:705
705	  poly_set.get(polygons);
(gdb) list
700	bool isPolygonCorner(const frCoord x,
701	                     const frCoord y,
702	                     const gtl::polygon_90_set_data<frCoord>& poly_set)
703	{
704	  std::vector<gtl::polygon_90_with_holes_data<frCoord>> polygons;
705	  poly_set.get(polygons);
706	  for (const auto& polygon : polygons) {
707	    for (const auto& pt : polygon) {
708	      if (pt.x() == x && pt.y() == y) {
709	        return true;

Debugging ORFS flow failures

OpenROAD CI does not run ORFS flows; flow-level integration is tested in ORFS and bazel-orfs. Run the ORFS flow with your OpenROAD build as shown above, and use ORFS make <stage>_issue (or bazel-orfs' //:deps workflow) to create a standalone test case.

Some OpenROAD and OpenSTA Bazel Specifics

Bazel distinguishes between host (cfg=exec) and target (cfg=target) configurations, a concept that becomes important when cross-compilation or tool usage is involved.

In the OpenROAD Bazel build:

  • bazelisk build ... builds all targets in the target configuration (cfg=target), assuming you're building for deployment or installation.
  • bazelisk test ..., on the other hand, uses OpenROAD and OpenSTA as host tools, meaning they are built and run in the execution configuration (cfg=exec), to run tests.

⚠️ Avoiding Redundant Builds

By default, bazelisk test would:

  1. First build test dependencies in the target configuration.
  2. Then build tools like OpenROAD/OpenSTA again in the host configuration to actually run the tests.

This causes unnecessary duplication.

To avoid this, .bazelrc includes the following to build only the tests:

test --build_tests_only

Using the OpenROAD project Bazel artifact server to download pre-built results

A single read only artifact server is configured in .bazelrc OpenROAD hosted projects.

This is a read only Bazel artifact server for anonymous access and is normally only updated by OpenROAD CI, though team OpenROAD team members can also update it directly.

OpenROAD team member and CI - configuring write access to artifact server

If you only have a single Google account that you use for Google Cloud locally, you can use --google_default_credentials.

If you are use multiple google accounts, using the default credentials can be cumbersome when switching between projects. To avoid this, you can use the --credential_helper option instead, and pass a script that fetches credentials for the account you want to use. This account needs to have logged in using gcloud auth login and have access to the bucket specified.

.bazelrc is under git version control and it will try to read in user.bazelrc, which is not under git version control, which means that for git checkout or rebase operations will ignore the user configuration in user.bazelrc.

Copy the snippet below into user.bazelrc and specify your username by modifying # user: myname@openroad.tools:

# user: myname@openroad.tools
build --credential_helper=*.googleapis.com=%workspace%/etc/cred_helper.py

cred_helper.py will parse user.bazelrc and look for the username in the comment.

To test, run:

$ ./cred_helper.py test
Running: gcloud auth print-access-token oyvind@openroad.tools
{
  "kind": "storage#testIamPermissionsResponse",
  "permissions": [
    "storage.buckets.get",
    "storage.objects.create"
  ]
}

Note: To test the credential helper, make sure to restart Bazel to avoid using a previous cached authorization:

bazelisk shutdown
bazelisk build BoomTile_final_scripts

To gain write access to the https://storage.googleapis.com/megaboom-bazel-artifacts bucket, reach out to Tom Spyrou, Precision Innovations (https://www.linkedin.com/in/tomspyrou/).

Bisecting OpenSTA or OpenROAD with Bazel

Bisecting OpenROAD or OpenSTA requires finding a good and a bad commit. Normally in bisection, origin/master is bad, but finding a good commit is trickier because most commits are there solely to preserve review history and have not run through any extensive testing.

Fortunately, OpenSTA is a submodule in OpenROAD that is tested before it is updated, so all the submodule commits in OpenROAD of OpenSTA are known to be of good quality. Similarly for OpenROAD and ORFS.

A git/bash incantation will list the commit hashes of the src/sta submodule:

$ git log --pretty=format:'%h' -- src/sta | while read commit; do  git show $commit src/sta| grep "Subproject commit" | awk '{print $3}'; done | head -n 10
5ee1a315141d1c799a0b2532e90ddccf52ddee95
3bff2d218c20adb867fcb3d8ae236f5da9928bed
3bff2d218c20adb867fcb3d8ae236f5da9928bed
5ee1a315141d1c799a0b2532e90ddccf52ddee95
f21d4a3878e2531e3af4930818d9b5968aad9416
3bff2d218c20adb867fcb3d8ae236f5da9928bed
522fc9563f25728f456bf86c2eb665c60d823e74
f21d4a3878e2531e3af4930818d9b5968aad9416
fa0cdd65290843e4e5cbe39d0bb9f2a63d580d1f
522fc9563f25728f456bf86c2eb665c60d823e74

To build OpenSTA, use master of the https://github.com/The-OpenROAD-Project/OpenSTA fork, because it contains Bazel build files:

bazelisk build src/sta:opensta -c opt

Now start the bisection as usual with a bad and good commit from the above list:

$ cd src/sta
$ git bisect start origin/master 6e95d93a44f7c46bb572933f5e2f8a624135820b
HEAD is now at 6e95d93a Merge remote-tracking branch 'parallax/master'
Bisecting: 55 revisions left to test after this (roughly 6 steps)
[03d2a48f462105a39b5850b8f45d6c5db16fd5f0] misc

Use git bisect --skip if the version does not build or otherwise should not be tested.

OpenSTA has an additional challenge in that only the https://github.com/The-OpenROAD-Project/OpenSTA fork has the Bazel BUILD file. To bisect the https://github.com/parallaxsw/OpenSTA branch, check out the branch you want, then check out BUILD from the fork and do a git reset HEAD. This will leave BUILD as a local file, because it is not in the upstream repository and bisection can be done on the upstream master branch.

Testing the GUI with gcd on a pull request by number

To test a PR with the GUI on gcd, build the PR and run ORFS with it:

    $ git fetch origin pull/7856/head
    $ git checkout FETCH_HEAD
    $ bazelisk build //:openroad
    $ OPENROAD_EXE=$(pwd)/bazel-bin/openroad make --dir ~/OpenROAD-flow-scripts/flow/ DESIGN_CONFIG=designs/asap7/gcd/config.mk
    $ OPENROAD_EXE=$(pwd)/bazel-bin/openroad make --dir ~/OpenROAD-flow-scripts/flow/ DESIGN_CONFIG=designs/asap7/gcd/config.mk gui_final

Whittling down .odb files

Global place can take hours to run and to debug an error, the test case has to be whittled down to minutes, or it is probably intractable.

Consider an error such as:

[ERROR GPL-0305] RePlAce diverged during gradient descent calculation, resulting in an invalid step length (Inf or NaN). This is often caused by numerical instability or high placement density. Consider reducing placement density to potentially resolve the issue.

Run the following from the flow/ directory of an ORFS checkout, with OPENROAD_EXE pointing at your build. First run up to the failing stage and stop with ctrl-c on the step that you want to run the whittling down on:

make DESIGN_CONFIG=designs/asap7/gcd/config.mk place

Now run the whittler with stock python3 — no extra packages needed beyond the standard library. You are responsible for having openroad on your PATH first (e.g. after bazelisk run //:install and source env.sh in the ORFS checkout). $OPENROAD below is your OpenROAD checkout:

python3 $OPENROAD/etc/whittle.py --error_string GPL-0305 --base_db_path results/asap7/gcd/base/3_2_place_iop.odb --use_stdout --exit_early_on_error --step "make DESIGN_CONFIG=designs/asap7/gcd/config.mk do-3_3_place_gp"

This should eventually leave you with a whittled down .odb file. Copy the whittled down .odb file into the correct place for 3_2_place_iop.odb, then create a bug report:

make DESIGN_CONFIG=designs/asap7/gcd/config.mk global_place_issue

Monitoring progress

whittle.py prints [whittle] status lines showing the current phase, element counts, .odb file size, and elapsed time. After a step runs for more than 5 minutes, whittle.py also shows the last 10 lines of the step's log output so you can tell what the step is doing.

If the .odb size is not shrinking after 20+ steps, try different parameters (--persistence 1 for a coarser first pass, or a higher --multiplier). If each step takes more than 10 minutes, check that --error_string is specific enough (avoid generic strings like "ERROR").

Using Claude with whittle.py

Point Claude at a GitHub issue that has an attached tarball artifact (from make *_issue). Claude can download the artifact, reproduce the bug with the latest OpenROAD, run whittle.py, and upload a smaller test case.

Scenario Recommended flags
Global placement bugs --error_string GPL-XXXX --persistence 3 --multiplier 2
Fast initial reduction --persistence 1 first, then increase
Large designs (>100K insts) Start with --timeout 600

See the /triage-issue Claude skill in .claude/skills/triage-issue/ for the full step-by-step workflow.