flow: KLayout Python module for GDS, DRC and LVS; deprecate Ruby decks; no KLayout application install

.github/workflows/github-actions-cron-test-installer.yml

@@ -9,6 +9,7 @@ on:
       - '.github/workflows/github-actions-cron-test-installer.yml'
       - 'build_openroad.sh'
       - 'env.sh'
+      - 'setup.sh'
       - 'flow/Makefile'
   pull_request:
     paths:
@@ -17,6 +18,7 @@ on:
       - '.github/workflows/github-actions-cron-test-installer.yml'
       - 'build_openroad.sh'
       - 'env.sh'
+      - 'setup.sh'
       - 'flow/Makefile'
   # Allows you to run this workflow manually from the Actions tab
   workflow_dispatch:
@@ -55,3 +57,41 @@ jobs:
         run: |
           docker rm -f "$INSTALLER_CONTAINER" || true
           docker image rm "openroad/flow-${{ matrix.os }}-builder:$INSTALLER_TAG" "openroad/flow-${{ matrix.os }}-dev:$INSTALLER_TAG" || true
+
+  testSetup:
+    # The documented path: sudo ./setup.sh by a normal user, no VIRTUAL_ENV
+    # set, then env.sh; the flow's PYTHON_EXE must have the klayout module.
+    strategy:
+      fail-fast: false
+      matrix:
+        os: ["ubuntu:26.04", "ubuntu:24.04"]
+    runs-on: ${{ vars.USE_SELF_HOSTED == 'true' && 'self-hosted-ubuntu-26.04' || 'ubuntu-26.04' }}
+    steps:
+      - name: Check out repository code
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1  # v7.0.1
+        with:
+          fetch-depth: 1
+          submodules: recursive
+      - name: Run setup.sh as a user and check PYTHON_EXE
+        run: |
+          script=$(cat <<'EOF'
+          export DEBIAN_FRONTEND=noninteractive
+          apt-get update
+          apt-get install -y --no-install-recommends sudo git ca-certificates
+          useradd -m orfs
+          cp -a /src /home/orfs/orfs
+          chown -R orfs: /home/orfs/orfs
+          git config --system --add safe.directory '*'
+          cd /home/orfs/orfs
+          unset VIRTUAL_ENV
+          SUDO_USER=orfs ./setup.sh
+          sudo -u orfs bash -ex <<'USER'
+          test -z "${VIRTUAL_ENV:-}"
+          source ./env.sh
+          python_exe=$(make -s -C flow print-PYTHON_EXE | sed -n 's/^PYTHON_EXE: //p')
+          test "$python_exe" = /home/orfs/orfs/.venv/bin/python3
+          "$python_exe" -c "import klayout.db"
+          USER
+          EOF
+          )
+          docker run --rm -v "$PWD:/src:ro" "${{ matrix.os }}" bash -exc "$script"

.github/workflows/github-actions-cron-util-test.yml

@@ -6,10 +6,18 @@ on:
     paths:
       - 'flow/util/*.py'
       - 'flow/test/test_*.py'
+      - 'flow/platforms/*/drc/*.py'
+      - 'flow/platforms/*/lvs/*.py'
+      - 'etc/requirements-common_lock.txt'
+      - '.github/workflows/github-actions-cron-util-test.yml'
   pull_request:
     paths:
       - 'flow/util/*.py'
       - 'flow/test/test_*.py'
+      - 'flow/platforms/*/drc/*.py'
+      - 'flow/platforms/*/lvs/*.py'
+      - 'etc/requirements-common_lock.txt'
+      - '.github/workflows/github-actions-cron-util-test.yml'
   # Allows you to run this workflow manually from the Actions tab
   workflow_dispatch:
 
@@ -27,8 +35,14 @@ jobs:
         with:
           fetch-depth: 1
           submodules: recursive
+      - name: Install Python packages
+        run: |
+          python3 -m venv venv
+          venv/bin/python3 -m pip install --no-cache-dir --upgrade --require-hashes -r etc/requirements-pip_lock.txt
+          venv/bin/python3 -m pip install --no-cache-dir -r etc/requirements-common_lock.txt
       - name: Run Tests
         run: |
+          source venv/bin/activate
           cd flow/test
           for file in *.py; do
               python3 "$file"

.gitignore

@@ -107,4 +107,5 @@ MODULE.bazel.lock
 
 # python venv
 venv/
+.venv/
 tmp/

docs/toc.yml

@@ -23,6 +23,8 @@ entries:
     title: Flow Variables
   - file: user/AutoMemories.md
     title: Generated Memory Macros (AUTO_MEMORIES)
+  - file: user/KLayoutPythonDecks.md
+    title: KLayout DRC and LVS Decks in Python
   - file: user/LargeDesigns.md
     title: Tips on building large design
   - file: contrib/Metrics.md

docs/user/BuildLocally.md

@@ -12,6 +12,10 @@ cd OpenROAD-flow-scripts
 sudo ./setup.sh
 ```
 
+`setup.sh` installs the Python packages the flow uses, the `klayout`
+module that writes GDS among them, into a virtual environment in `.venv`,
+which `env.sh` puts first on `PATH`.
+
 ## Using Bazel to build OpenROAD and run the ORFS flow (unsupported)
 
 For ORFS/OpenROAD developers. Most of `./setup.sh` isn't needed when

docs/user/BuildWithPrebuilt.md

@@ -1,10 +1,13 @@
 # Using Pre-built Binaries
 
-## Install Klayout and Yosys
-Please ensure the Klayout version (denoted with `klayoutVersion` variable) is consistent with the one used in [DependencyInstaller script](https://github.com/The-OpenROAD-Project/OpenROAD-flow-scripts/blob/master/etc/DependencyInstaller.sh).
+## Install KLayout and Yosys
+The GDS file, DRC and LVS use KLayout's Python module, installed with the
+other Python packages from `etc/requirements-common_lock.txt`, or alone
+with `pip install klayout`. The KLayout application is needed only for a
+deprecated Ruby DRC/LVS deck (see [KLayoutPythonDecks.md](KLayoutPythonDecks.md)),
+the gallery and the viewers:
+[KLayout](https://www.klayout.de/build.html).
 
-Instructions for installing:
-- [Klayout>=0.28.8](https://www.klayout.de/build.html)
 - [Yosys>=0.58](https://github.com/YosysHQ/oss-cad-suite-build/blob/master/README.md#installation)
 
 ## Install OpenROAD
@@ -44,9 +47,6 @@ Export path variables accordingly.
 export OPENROAD_EXE=$(command -v openroad)
 export YOSYS_EXE=$(command -v yosys)
 
-# only if KLayout is built from source
-export LD_LIBRARY_PATH="<klayout_location>/bin:$PATH"
-
 yosys -help
 yosys -m slang -p "slang_version"
 openroad -help

docs/user/FlowVariables.md

@@ -176,6 +176,8 @@ configuration file.
 | <a name="IO_PLACER_V"></a>IO_PLACER_V| A list of metal layers on which the I/O pins are placed vertically (sides of the die).| |
 | <a name="IR_DROP_LAYER"></a>IR_DROP_LAYER| Default metal layer to report IR drop.| |
 | <a name="KEEP_VARS"></a>KEEP_VARS| Feature toggle to keep intermediate variables during the flow. This is useful for the single-run flow, where all stages of the flow are run in a single OpenROAD instance.| 0|
+| <a name="KLAYOUT_DRC_FILE"></a>KLAYOUT_DRC_FILE| The KLayout DRC deck `make drc` runs. A `.py` deck runs with the klayout Python module; a Ruby deck (`.lydrc`) is deprecated and needs the KLayout application. Empty when the platform has no deck.| |
+| <a name="KLAYOUT_LVS_FILE"></a>KLAYOUT_LVS_FILE| The KLayout LVS deck `make lvs` runs. A `.py` deck runs with the klayout Python module; a Ruby deck (`.lylvs`) is deprecated and needs the KLayout application. Empty when the platform has no deck.| |
 | <a name="KLAYOUT_TECH_FILE"></a>KLAYOUT_TECH_FILE| A mapping from LEF/DEF to GDS using the KLayout tool.| |
 | <a name="LATCH_MAP_FILE"></a>LATCH_MAP_FILE| Optional mapping file supplied to Yosys to map latches| |
 | <a name="LAYER_PARASITICS_FILE"></a>LAYER_PARASITICS_FILE| Path to per layer parasitics file. Defaults to $(PLATFORM_DIR)/setRC.tcl.| |
@@ -632,6 +634,8 @@ configuration file.
 - [CDL_FILE](#CDL_FILE)
 - [GDS_ALLOW_EMPTY](#GDS_ALLOW_EMPTY)
 - [GND_NETS_VOLTAGES](#GND_NETS_VOLTAGES)
+- [KLAYOUT_DRC_FILE](#KLAYOUT_DRC_FILE)
+- [KLAYOUT_LVS_FILE](#KLAYOUT_LVS_FILE)
 - [LEC_AUX_VERILOG_FILES](#LEC_AUX_VERILOG_FILES)
 - [LEC_CHECK](#LEC_CHECK)
 - [MAX_ROUTING_LAYER](#MAX_ROUTING_LAYER)

docs/user/KLayoutPythonDecks.md

@@ -0,0 +1,75 @@
+# KLayout DRC and LVS decks in Python
+
+`make drc` and `make lvs` run a platform's KLayout deck. A deck written in
+Python (`KLAYOUT_DRC_FILE` or `KLAYOUT_LVS_FILE` ending in `.py`) runs with
+the `klayout` Python module from pip, which the ORFS Python requirements
+install. A deck written in KLayout's Ruby DRC language (`.lydrc`, `.lylvs`)
+needs the KLayout application.
+
+The decks ORFS runs are Python: asap7, nangate45, sky130hd and
+ihp-sg13g2 (`sg13g2_minimal`) DRC, and sky130hd and ihp-sg13g2 LVS.
+
+## Ruby decks are deprecated
+
+A Ruby deck still runs, through the KLayout application, and `make drc` and
+`make lvs` print a warning when they run one. The ORFS installer installs
+only the `klayout` Python module, so whoever uses a Ruby deck installs the
+KLayout application themselves ([KLayout](https://www.klayout.de/build.html)).
+To work with the ORFS installer, convert the deck to Python. There is no
+schedule for removing Ruby support; that is up to the ORFS maintainers.
+
+The Python module does not include Ruby, and KLayout will not add it
+([KLayout/klayout#1772](https://github.com/KLayout/klayout/issues/1772)).
+The DRC language is a thin layer over `klayout.db`'s `Region`, `Edges` and
+`EdgePairs`, so a deck translates to Python line by line
+([KLayout/klayout#1980](https://github.com/KLayout/klayout/issues/1980)).
+
+## Converting a deck
+
+`flow/util/klayout_drc.py` is that layer for the operations ORFS's decks
+use, with the DRC language's defaults: `input` and `polygons`, the checks
+(`width`, `space`, `separation`, `enclosing`, `overlap`, `notch`,
+`isolated`), the selections (`with_area`, `with_angle`, `with_density`
+over tiles, `interacting`, `inside`, ...), boolean operations, `sized`,
+`connect` with `antenna_check`, and `output`. `flow/util/klayout_lvs.py`
+adds `connect_global`, `extract_devices`, `same_circuits` and `compare`
+for LVS. The decks in
+`flow/platforms/*/drc/*_drc.py` and `flow/platforms/*/lvs/` are
+worked examples next to the Ruby they replaced (in git history).
+`flow/platforms/ihp-sg13g2/lvs/sg13g2_lvs.py` shows what a deck's own Ruby
+classes become: `RBA::GenericDeviceExtractor`, `RBA::GenericDeviceCombiner`,
+`RBA::NetlistSpiceReaderDelegate` and device class subclasses translate to
+the `klayout.db` classes of the same names, with the same callbacks
+(`setup`, `get_connectivity`, `extract_devices`, `combine_devices`,
+`parse_element`, `element`); `super()` calls the base implementation, as
+Ruby's `super` does.
+
+1. Translate the deck statement by statement into a `*_drc.py` that uses
+   `klayout_drc`; keep the rule names, so the report categories match.
+2. Check it against the Ruby deck with `flow/util/klayout_compare_decks.py`,
+   on the platform's layout and on a copy with shapes added that break the
+   rules (width, space, enclosure, density, antenna):
+
+   ```shell
+   python3 flow/util/klayout_compare_decks.py \
+     --ruby-deck=<deck>.lydrc --python-deck=<deck>_drc.py \
+     --in-gds=results/<platform>/<design>/base/6_final.gds --out-dir=<dir>
+   ```
+
+   It runs the Ruby deck in the KLayout application, the Python deck with
+   the module, and compares the violations per category, count and markers.
+3. Point `KLAYOUT_DRC_FILE` at the Python deck.
+
+The conversion of ORFS's decks found a bug kept as is: in the sky130hd deck,
+`if backend_flow = CU` assigns rather than compares, so its copper-only
+rules never ran; the Python deck keeps `CU = False`.
+
+ihp-sg13g2's Ruby LVS deck never ran in ORFS. Its `config.mk` line was
+commented out and named a file that did not exist; the deck reads its
+inputs from `-rd input=`, `schematic=`, `topcell=` and `report=`, which
+`make lvs` does not pass; and the CDL that `make` wrote left out the decap
+cells that are in the layout. The Python deck gets its inputs from `klayout_lvs.py`, and
+`flow/scripts/cdl.tcl` writes the CDL with `-include_fillers`.
+
+What only the KLayout application has is dropped: `RBA::CellView::active`
+(a deck run from the GUI on the open layout) and file dialogs.

docs/user/UserGuide.md

@@ -6,8 +6,8 @@ The OpenROAD Project uses three tools to perform automated RTL-to-GDS layout gen
     Synthesis
 2.  [OpenROAD](https://github.com/The-OpenROAD-Project/OpenROAD):
     Floorplanning through Detailed Routing
-3.  [KLayout](https://www.klayout.de/): GDS merge, DRC and LVS (public
-    PDKs)
+3.  [KLayout](https://www.klayout.de/): GDS merge (its Python module),
+    DRC and LVS (public PDKs, the application)
 
 To automate RTL-to-GDS we provide
 [OpenROAD Flow](https://github.com/The-OpenROAD-Project/OpenROAD-flow-scripts),

env.sh

@@ -14,9 +14,10 @@ function __setpaths() {
   export PATH=${DIR}/tools/install/OpenROAD/bin:$PATH
   export PATH=${DIR}/tools/install/yosys/bin:$PATH
   export PATH=${DIR}/tools/install/kepler-formal/bin:$PATH
+  # The Python packages setup.sh installs, the klayout module among them
+  export PATH=${DIR}/.venv/bin:$PATH
 
   if [[ "$OSTYPE" == "darwin"* ]]; then
-    export PATH="/Applications/KLayout/klayout.app/Contents/MacOS:$PATH"
     export PATH="$(brew --prefix bison)/bin:$(brew --prefix flex)/bin:$(brew --prefix tcl-tk@8)/bin:$PATH"
     export QT_QPA_PLATFORM=cocoa
   fi