Repository navigation
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the flow to make KLayout an optional dependency by utilizing the klayout Python module for GDS/OAS stream generation, falling back to mocking (writing an empty file with a warning) when KLayout is entirely missing. It updates dependency installers, Makefiles, and documentation to reflect this change, and adds corresponding unit tests. The review feedback correctly identifies a critical issue in flow/util/def2stream.py where a subprocess version check could cause an unnecessary hard failure, and points out that the custom environment dictionary is not being forwarded to the spawned subprocesses.
1ccf595 to
1971407
Compare
|
@maliberty @eder-matheus @vvbandeira TL;DR klayout has been a thorny dependency and it confuses users This is intended to prune as much as possible from ORFS install scripts, etc. while giving each use-case the least chafing experience. We try PyPi klayout first, then local and finally we write an empty .gds file and a warning. The full klayout install is still needed for make lvs/drc and gallery, but I think those are out of the way for most users and PyPi klayout doesn't have ruby, so there's nothing for it, full klayout install is needed. This would be solved by documentation, this is an out of the way use-case you have to install klayout yourself. |
🔍 QoR checkMetrics reflect the PR merge build — i.e. what will land on the target branch. Commit 62 design(s) checked — 0 with regression(s), 0 without a comparable baseline. |
| DRC, LVS, the gallery and the viewers need the KLayout application, | ||
| which you install yourself: [KLayout](https://www.klayout.de/build.html). | ||
| The DRC and LVS rule decks are scripts in KLayout's Ruby-based DSL, | ||
| which the Python module does not include. None of them runs in the | ||
| default `make`. |
There was a problem hiding this comment.
This is arguably a gap that should be fixed rather than a status we should enshrine. I like the simplification but not at the expense of having the possibility of DRC/LVS in the future.
|
Reopened to close the gap rather than document it: the Ruby DRC/LVS decks get rewritten in Python on the PyPI |
def2stream.py merges the GDS file with the klayout Python module from pip (etc/requirements-common), run by PYTHON_EXE, instead of under the KLayout application (klayout -zz -r def2stream.py). The script already used only klayout.db calls; it gets a command line. Without the module it stops with an error and writes no file. The GDS merge no longer depends on check-klayout. DRC and LVS still run the KLayout application, so they depend on check-klayout directly. The wrapped-macro merge passed `-rd config_file=$(FILL_CONFIG)`, which def2stream.py never read; it is not passed any more. test_def2stream.py checks the module path and the error without it; test_klayout_sh.py checks that klayout.sh still stops without KLayout and passes a call through unchanged with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018PFLaCzFtFbnWE4nPTDwpC Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Before, make drc on asap7 needed the KLayout application to run the Ruby deck asap7.lydrc. Now it runs asap7_drc.py with the klayout Python module that already writes the GDS. flow/util/klayout_drc.py is the thin layer KLayout's DRC language is over klayout.db (KLayout/klayout#1980), for the operations ORFS's decks use: each method calls Region, Edges or EdgePairs with the arguments and defaults of _drc_layer.rb, values in microns as floats and in database units as ints, deep mode on a DeepShapeStore, and the report written with rdb's scan_collection into the .lyrdb that make drc counts. The deck is the Ruby deck translated line by line. make drc runs a deck ending in .py with PYTHON_EXE; a .lydrc deck still goes to the KLayout application. RBA::CellView::active, which only the application has, is gone with the Ruby deck: the report file is always given. Checked against the Ruby deck run by the KLayout 0.30.0 application on the same asap7/gcd GDS, Python on klayout 0.30.13: clean GDS: 42 violations in 9 categories, both, same markers seeded GDS: 64 violations in 17 categories, both, same markers (narrow M1/M2, V1 spacing and enclosure, GATE width, ACTIVE without select) runtime: 73 s Ruby, 55 s Python Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Before, make drc on nangate45 needed the KLayout application to run
FreePDK45.lydrc, and KLAYOUT_LVS_FILE named lvs/FreePDK45.lylvs, which
does not exist. Now make drc runs FreePDK45_drc.py on the klayout
Python module, and nangate45 sets no LVS deck: make lvs says LVS is not
supported on this platform instead of failing on a missing file.
The antenna checks go through LayoutToNetlist as the DRC language's
connect and antenna_check do: the layers registered on the deep shape
store, connected, the netlist extracted once and antenna_check called
with the diode layer.
Checked against the Ruby deck run by the KLayout 0.30.0 application on
the same nangate45/gcd GDS, Python on klayout 0.30.13:
clean GDS: 0 violations, both
seeded GDS: 8 violations in 8 categories, both, same markers
(narrow and close metal1, narrow metal2 and poly, via1
without enclosure, ACTIVE.4, a metal1 antenna)
runtime: 27 s Ruby, 2.6 s Python
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Before, make drc on sky130hd needed the KLayout application to run
sky130hd.lydrc. Now it runs sky130hd_drc.py on the klayout Python
module.
The Ruby deck's "if backend_flow = CU" blocks are assignments, not
comparisons: with CU false they never run and the AL blocks always do.
The Python deck says that with one CU = False. Its FEOL rules stay off
as before; the cell selections (source.select) are a Source in
klayout_drc.py with the same select/unselect order on the shape
iterator. The deck's area(layer) is the layer's area in square
microns, as the DRC language's.
Checked against the Ruby deck run by the KLayout 0.30.0 application on
the same sky130hd/gcd GDS, Python on klayout 0.30.13:
clean GDS: 0 violations, both
seeded GDS: 6 violations in 6 categories, both, same markers
(narrow and close m1, narrow li and m2, via without
metal)
FEOL on in both decks, seeded GDS: 6 and 6, same markers
runtime: 4.2 s Ruby, 2.7 s Python (FEOL on: 177 s, 162 s)
sky130hd has no density rules in its deck; the tiled density checks
are in ihp-sg13g2's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Before, make lvs on sky130hd needed the KLayout application to run
sky130hd.lylvs. Now it runs sky130hd_lvs.py on the klayout Python
module.
flow/util/klayout_lvs.py is what the LVS language adds to the DRC
language (_lvs_netter.rb), for what this deck uses: mos4 device
extraction with the layers registered in sorted terminal order, the
CDL schematic, connect_global, device scaling, split_gates and
tolerance applied once the netlist is extracted, same_circuits and
equivalent_pins applied to each NetlistComparer when it is made, align,
compare, and the short-format .lvsdb. The deck's schematic and netlist
edits (0-ohm ties in CONB_1, the stack merges, the VNB/VSS tie) are the
Ruby functions in Python; nets are keyed by name, as the Ruby deck
does, since the binding hands back a new wrapper on each lookup.
The Ruby deck wrote the extracted netlist next to
RBA::CellView::active, which only the application has; the Python deck
writes the .lvsdb only, which holds the extracted netlist.
Checked against the Ruby deck run by the KLayout 0.30.0 application on
the same sky130hd/gcd GDS and CDL, Python on klayout 0.30.13, comparing
the cross references in the two .lvsdb files:
clean GDS: match, both: 59 circuits, 496 devices, 1119 nets,
931 pins, 451 subcircuits matched, both
shorted GDS: no match, both (a met1 bar across the core): 1 circuit,
10 nets and 28 subcircuits mismatched, the rest matched,
the same counts in both
runtime: 2.6 s Ruby, 0.8 s Python
The device abstracts are the same set in both .lvsdb files, in a
different order.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Before, make drc on ihp-sg13g2 needed the KLayout application to run
sg13g2_minimal.lydrc. Now it runs sg13g2_minimal_drc.py on the klayout
Python module.
The Ruby deck adds ext_* methods to the DRC language's layer; the
Python deck adds the same functions to klayout_drc.Layer, line by line:
the space, width and separation checks with their touch-point and
intersecting-edge handling (EdgePairOperator subclasses as in Ruby),
the rectangle and square filters as universal DRC expressions
(CompoundRegionOperationNode), and the global and 800 x 800 um density
windows. klayout_drc.py gets what these need from the DRC language:
with_density/without_density on a TilingProcessor with the same
expression script, tile options, padding_ignore, labels and texts,
raw, merge, dup, select_inside, with_distance, sizing modes and the
ortho edge filter.
What only the application had is gone: the file dialog and
RBA::CellView::active when no GDS is given (make drc always gives
one), and the -rd variables log_file, cell, threads and densityRules,
which the Makefile never set; density rules stay on and the deck runs
single-threaded, as before.
Checked against the Ruby deck run by the KLayout 0.30.0 application on
the same ihp-sg13g2/gcd GDS, Python on klayout 0.30.13:
clean GDS: 8 violations in 8 categories (global and windowed
density), both, same markers
seeded GDS: 24 violations in 20 categories, both, same markers
(narrow and close Metal1, narrow Metal2, GatPoly,
Activ and TopMetal1, Via1 of the wrong size, Cont
spacing with a corner touch, GatPoly spacing, a Metal1
pin outside Metal1, and density)
runtime: 4.3 s Ruby, 2.6 s Python
sg13g2_maximal.lydrc, which no flow uses, and the commented-out LVS
deck stay as they are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
5ac6df2 to
c8e7723
Compare
309fe85 to
63c807e
Compare
Before, ihp-sg13g2 had IHP's Ruby LVS runset (sg13g2.lvs, 39 rule_decks/
files and its driver run_lvs.py), and make lvs did not run it. Now make
lvs runs sg13g2_lvs.py on the klayout Python module, and gcd matches.
What was broken: the runset never ran in ORFS.
- config.mk had KLAYOUT_LVS_FILE commented out, naming
$(PLATFORM).lylvs, a file that does not exist.
- The runset reads -rd input=, schematic=, topcell= and report=; make
lvs passes in_gds=, cdl_file=, top_cell= and report_file=, so run as
make would, it stops at once: "'source': No layout loaded".
- Given its own variables, it runs, and reports no match on gcd: the
CDL from cdl.tcl leaves out the physical-only cells, and
sg13g2_decap_4/8 have transistors, so 1231 decap instances in the
layout have no schematic counterpart. cdl.tcl now writes the CDL with
write_cdl -include_fillers. sky130hd's LVS still matches with it.
sg13g2_lvs.py is the runset translated line by line, in include order:
the layer definitions and derivations on klayout_drc.Layer, the
connections, and the device extraction; the runset's Ruby classes as the
klayout.db classes of the same names: the SPICE reader delegate (its
error fallbacks through super()), the device classes, the device
combiners, and the GenericDeviceExtractor subclasses for resistors,
varactors, inductors and MIM capacitors. The switches are run_lvs.py's
defaults (deep mode, simplify, series and parallel resistors combined).
Kept as the Ruby does it: a Metrics::Projection passed as the second
positional argument of space_check and notch_check is whole_edges, and
the reader deletes [, ], $ and / from element lines. The extracted
netlist (target_netlist and its SPICE writer) is not written; the
.lvsdb holds it, as for sky130hd.
klayout_drc.py gets extent(cell_filter), middle, with_bbox_max and
connect for text layers; klayout_lvs.py gets bjt3, bjt4, diode and
resistor extractors with a device class, a schematic reader, max_res and
min_caps.
Checked against the Ruby runset run by the KLayout 0.30.0 application on
the same ihp-sg13g2/gcd GDS and CDL (with the decaps), Python on klayout
0.30.12 and 0.30.13, comparing the cross references in the .lvsdb files:
clean GDS: match, all: 36 circuits, 232 devices, 837 nets, 239 pins
and 1764 subcircuits matched
shorted GDS: no match, all (a Metal3 bar between the pins req_msg[12]
and req_msg[9]): gcd, 2 nets and 1 subcircuit mismatched,
the layout net named "req_msg[12],req_msg[9]"; the rest
matched, the same counts in all
CDL without the decaps: no match, all, 1231 subcircuits and VDD and
VSS mismatched
runtime: 7.3 s Ruby, 3.9 s Python
The extracted netlists are the same in the Python deck run in the 0.30.0
application and the Ruby runset, but for the pin order of two circuits
and the numbers of unnamed nets. Between 0.30.0, 0.30.12 and 0.30.13 the
split of the source/drain areas (AS, AD) of combined multi-finger
transistors differs; LVS does not compare them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018PFLaCzFtFbnWE4nPTDwpC
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Before, ihp-sg13g2 carried sg13g2_maximal.lydrc, a Ruby deck no flow runs. Now it is gone. No flow sets KLAYOUT_DRC_FILE to it; sg13g2_minimal_drc.py is the deck ORFS runs. IHP's own repository keeps the full deck. sg13g2_update.py does not fetch it: its download line is commented out, since IHP replaced the minimal and maximal decks with a parameterizable sg13g2.drc. test_klayout_sh.py named it only as an argument to klayout.sh, which stops before reading it; the test now names a generic deck.lydrc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018PFLaCzFtFbnWE4nPTDwpC Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Every deck ORFS runs is now Python on the klayout module. A deck in KLayout's Ruby DRC language, a platform outside ORFS for one, still runs through the KLayout application, and make drc / make lvs say it is deprecated: WARNING: <deck> is a Ruby KLayout deck; Ruby decks are deprecated and need a KLayout application you install yourself, see docs/user/KLayoutPythonDecks.md There is no schedule for removing Ruby support; that is up to the ORFS maintainers. docs/user/KLayoutPythonDecks.md says how to convert a deck, with ORFS's decks as worked examples. flow/util/klayout_compare_decks.py runs a Ruby deck and its translation on one GDS file and compares the violations, count and markers per category, as the conversion of ORFS's decks was checked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018PFLaCzFtFbnWE4nPTDwpC Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
The flow writes its GDS file and runs its DRC and LVS decks with KLayout's Python module, from the installer's Python packages, so the installer no longer installs the KLayout application. Deleted: - etc/DependencyInstaller.sh: klayoutVersion, and the KLayout install for each OS: the CentOS 7, CentOS 8 and Rocky Linux 9 RPMs with their version check and reinstall, the Ubuntu .deb with its md5 table per release and the "Unsupported Ubuntu version" exit, the build from source on Ubuntu aarch64 with its Qt, Ruby and libgit2 build dependencies, `klayout` from apt on rodete and the Homebrew cask on macOS. - env.sh: the KLayout.app PATH entry on macOS. - flow/scripts/variables.mk: the lookup of a KLayout built into tools/install/klayout, which nothing installs. KLAYOUT_CMD is the klayout on PATH unless set, and is exported bare. - docs/user/BuildWithPrebuilt.md: the KLayout version note and the LD_LIBRARY_PATH line for a KLayout built from source. A deprecated Ruby DRC/LVS deck, the gallery and the viewers still need the KLayout application, installed by those who use them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018PFLaCzFtFbnWE4nPTDwpC Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
63c807e to
c27524a
Compare
|
@maliberty Rewrote ruby to python, klayout only needed for those designs(confidential/users) that still use Ruby. My thinking is they can install klayout themselves and ORFS can drop klayout install. I don't have any opinion on the deprecation period. |
Before: the flow wrote GDS with the KLayout application, and
make drc/make lvsran Ruby rule decks that need the application.Now: KLayout's Python module,
pip install klayout, writes the GDS and runs every DRC and LVS deck ORFS uses (DRC: asap7, nangate45, sky130hd, ihp-sg13g2; LVS: sky130hd, ihp-sg13g2), rewritten in Python; Ruby decks are deprecated, and the installer installs the module only, not the KLayout application.This continues and reframes the earlier version of this pull request, which dropped the KLayout application from the installer and wrote that DRC/LVS needed the application down as a permanent state. That gap is closed here instead: DRC and LVS run on the Python module, so the application is needed only for a deprecated Ruby deck, and whoever uses one installs it. (#4797, which removed the application install on its own, is folded in as the last commit.)
DRC and LVS decks in Python
KLayout ships no Ruby in the Python module (KLayout/klayout#1772); its DRC language is a thin layer over
klayout.db'sRegion,EdgesandEdgePairs(KLayout/klayout#1980).flow/util/klayout_drc.pyis that layer for the operations ORFS's decks use, with the DRC language's arguments and defaults (_drc_layer.rb), andflow/util/klayout_lvs.pyadds what the LVS language adds (_lvs_netter.rb). Each deck is the Ruby deck translated line by line, one platform per commit, and the Ruby deck is deleted in the same commit.make drc/make lvsrun a deck ending in.pywithPYTHON_EXE. Outputs keep their names:6_drc.lyrdb,6_drc_count.rpt,6_lvs.lvsdb.Each Python deck is checked, on the klayout module at 0.30.12 (the version the lock pins) and at 0.30.13 with identical results, against the Ruby deck run by the KLayout 0.30.0 application on the same GDS from ORFS's
gcdon that platform, clean, and with shapes added outside the design that break a sample of rules. DRC: violations per category and the markers themselves; LVS: the match verdict and the cross-reference counts in the.lvsdb.drc), selectsg13g2_minimal), with densityreq_msg[12]andreq_msg[9]): no match / no match; the same 2 nets and 1 subcircuit mismatched, the merged net namedreq_msg[12],req_msg[9]Also in these commits:
KLAYOUT_LVS_FILE: it namedlvs/FreePDK45.lylvs, which does not exist, somake lvsnow says LVS is not supported instead of failing on a missing file.RBA::CellView::active, which only the application has, is gone with the Ruby decks: the asap7 and sky130hd DRC decks used it when no report file was given, the sky130hd LVS deck to place its extracted netlist (the.lvsdbholds that netlist), the ihp deck for its file dialog.if backend_flow = CUassigns rather than compares, so its copper-only rules never ran; the Python deck keeps that (CU = False)..lvsdbfrom Python lists the same device abstracts as the Ruby one, in a different order.KLAYOUT_LVS_FILEwas commented out and named a file that does not exist, the deck read-rd input=/schematic=/topcell=/report=wheremake lvspassesin_gds=/cdl_file=/top_cell=/report_file=, and run by hand it said "no match" on gcd because the CDL fromcdl.tclleaves out the decap cells, which have transistors (1231 instances). The Python deck takesmake lvs's arguments,cdl.tclnow writes the CDL withwrite_cdl -include_fillers, and ihp-sg13g2 setsKLAYOUT_LVS_FILE. With that CDL both decks match; sky130hd LVS still matches. The-include_fillerschange reaches every platform's CDL.sg13g2.lvs, the 39rule_decks/*.lvsfiles andrun_lvs.py, which only drove it. IHP's vendoring script (sg13g2_update.py) never fetched them.flow/test/test_klayout_drc.pychecks width, space, enclosure, a universaldrccheck, area filters and deep against flat reporting on a generated layout; it is skipped without the klayout module.Ruby decks: deprecated
A
.lydrc/.lylvsdeck, on a platform outside this repository for one, still runs through the KLayout application, andmake drc/make lvswarn:There is no schedule for removing Ruby support; that is up to the maintainers.
docs/user/KLayoutPythonDecks.mdsays how to convert a deck, with ORFS's decks as worked examples.flow/util/klayout_compare_decks.pyruns a Ruby deck and its translation on one GDS file and compares the violations per category, count and markers, as ORFS's decks were checked; on asap7gcdit reports the Ruby and Python decks the same.GDS via the Python module
def2stream.pymerges the GDS file with the module, run byPYTHON_EXE; it already used onlyklayout.dbcalls, and gets a command line. The module is a prebuilt wheel, with no Qt and nothing to compile, for Python 3.10 and up on Linux (x86_64 and aarch64, glibc 2.27 and up) and macOS, added toetc/requirements-common.inand pinned at 0.30.12 in the lock, the version the installer installs. Without the module the merge stops, exit status 1, no file written:lb_32x128,make do-gdsLayoutDiff, 12 cells)Installer: no KLayout application
etc/DependencyInstaller.shloses its KLayout install for every OS; the module comes in with the other Python packages. Deleted:klayoutVersionand the version-check-and-reinstall logic, written three times (CentOS 7, CentOS 8, Rocky Linux 9) around klayout.org RPMs..debdownload, its md5 table per release, and the "Unsupported Ubuntu version" exit, so KLayout's packaging no longer limits which Ubuntu releases the installer supports.klayoutfrom apt on rodete, and the Homebrew cask on macOS.env.sh: the KLayout.appPATHentry on macOS.flow/scripts/variables.mk: the lookup of a KLayout intools/install/klayout, which nothing installs.KLAYOUT_CMDis theklayoutonPATHunless set, looked up once (command -v klayoutruns 21 times formake -n clean_allon asap7/gcd, as on master).docs/user/BuildWithPrebuilt.md: the KLayout version note and theLD_LIBRARY_PATHline for a KLayout built from source.A deprecated Ruby deck, the gallery and the viewers still need the KLayout application, installed by whoever uses them; without it they stop at
check-klayout, as on master.Dogs left lying
flow/scripts/klayout/srcandtest(14 Ruby files): a bring-up tool that generates KLayout tech and layer files from a Cadence Virtuoso tech file. Nothing in the repository runs or references it; it is left as it is.sg13g2_maximal.lydrc, which no flow uses, is deleted rather than converted: ORFS runssg13g2_minimal, and converting IHP's full deck is IHP's to do in its own repository.Commits
flow: write the GDS file with the klayout Python moduleihp-sg13g2: delete the unused Ruby DRC deck sg13g2_maximal.lydrcflow: deprecate Ruby KLayout decksinstaller: no KLayout applicationTests
flow/test/test_def2stream.pychecks that the module gets the arguments, and that without it the merge exits with status 1, prints the error and writes no file.flow/test/test_klayout_sh.pychecks thatklayout.shstops without the KLayout application and passes a call through unchanged with it. Both pass (25 tests).klayout_compare_decks.pyon asap7gcd: same.make -nshows the deprecation warning for a.lydrcdeck and none for a.pydeck.WRAP_LEFS), which only gf12 designs use. It calls the same script with the same arguments as before.🤖 Generated with Claude Code
https://claude.ai/code/session_018PFLaCzFtFbnWE4nPTDwpC