Skip to content

flow: FLOW_INPUT_VARIANT, a stage reads the previous stage's result from another variant - #4812

Draft
oharboe wants to merge 9 commits into
The-OpenROAD-Project:masterfrom
oharboe:flow-input-variant
Draft

oharboe wants to merge 9 commits into
The-OpenROAD-Project:masterfrom
oharboe:flow-input-variant

Conversation

@oharboe

@oharboe oharboe commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Draft: waits for #4751 and #4810. This branch merges both so it can be built and tested; review only the last commit, 371ea26. It is rebased onto master once they merge.

Before: a variant that wanted another variant's synthesis or placement had to copy that variant's results into its own directory first.

Now: FLOW_INPUT_VARIANT names the variant a stage reads the previous stage's result from. One synthesis, floorplan or CTS is shared by many experiments, each forked off it with nothing copied:

make DESIGN_CONFIG=... FLOW_VARIANT=base cts
make DESIGN_CONFIG=... FLOW_INPUT_VARIANT=base FLOW_VARIANT=high_effort do-grt
make DESIGN_CONFIG=... FLOW_VARIANT=high_effort do-route do-final

(do-route running only route is #4811.)

This redoes #4475, which was closed after review: it touched 13 files to resolve every read against a search path (this variant first, then the other), and every reader had to remember to use the helper. #4751 and #4810 remove what made that necessary:

What is left is one explicit read per stage, and no search.

How

  • variables.mk: FLOW_INPUT_VARIANT ?= $(FLOW_VARIANT) and INPUT_RESULTS_DIR. With the default every path is what it was.
  • load.tcl: load_stage_input reads from INPUT_RESULTS_DIR, load_design from RESULTS_DIR; both are load_design_from <dir> <file>.
  • The six reads of a previous stage's result call load_stage_input (floorplan.tcl 1_synth, global_place_skip_io.tcl 2_floorplan, cts.tcl 3_place, global_route.tcl 4_cts, detail_route.tcl 5_1_grt, density_fill.tcl 5_route), and their Makefile prerequisites name $(INPUT_RESULTS_DIR). Every other read, a stage's own earlier steps among them, stays in RESULTS_DIR.
  • variables.yaml/.json, FlowVariables.md: the variable, its stages and its rules.

Fails loudly

When FLOW_INPUT_VARIANT names another variant, load_stage_input stops:

  • The input is in both variants. One invocation that runs two stages, make FLOW_INPUT_VARIANT=base do-grt do-route say, would read route's input, 5_1_grt.odb, from base, not the one grt just wrote; a copy left in the fork from an earlier run is the same ambiguity. The fork reads another variant for its first stage only, and the stages after it with FLOW_INPUT_VARIANT unset.
  • AUTO_MEMORIES=1. The memories are read from the variant that synthesised them, a second location this pull request does not add.

Known limit

After a fork, a file target such as make FLOW_VARIANT=high_effort route follows its prerequisites back through the fork's own directory and reruns the flow from synthesis there: correct, and the cost of the whole flow. do- targets run on the files in the tree and do not. #4811 makes do-route run only route, as the other do-<stage> targets do; until then, a fork at grt runs do-5_2_route do-5_3_fillcell do-5_route after do-grt.

make metadata in a fork reads the fork's own LOG_DIR and REPORTS_DIR: it reports the stages the fork ran, and the shared stages read N/A/ERR, as for any stage that did not run; their metrics are in the input variant's metadata.

copy_endpoint_slack reads the input step's endpoint-slack report from REPORTS_DIR and skips it when it is not there, so a fork whose first step only copies its input (GPL_PLACE_IOS=1 at place) has no endpoint-slack report for that step. That existence check predates this pull request.

Testing

tclfmt/tclint 0.7.0 clean. asap7/gcd with ORFS's make on this branch:

  • base through cts, then a fork: do-5_1_grt with FLOW_INPUT_VARIANT=base, then do-5_2_route do-5_3_fillcell do-5_route do-6_1_fill in the fork. Against base run straight on, the .odb and .sdc of 5_1_grt, 5_2_route, 5_3_fillcell, 5_route and 6_1_fill are byte-identical. The fork's directory holds only those; it read base/4_cts.odb once.
  • do-5_1_grt do-5_2_route in one fork invocation, with base already holding 5_1_grt.odb: route stops at 5_1_grt.odb is in both FLOW_VARIANT=span and FLOW_INPUT_VARIANT=base. .... Without the check it would read base's global route.
  • A stale 4_cts.odb in the fork: do-5_1_grt stops at the same error for 4_cts.odb.
  • grt route file targets in one fork invocation before base has grt: make stops at No rule to make target '.../base/5_1_grt.odb'.
  • AUTO_MEMORIES=1 in a fork: stops at AUTO_MEMORIES=1 reads the memories synthesised in this variant, ....

No CI design sets FLOW_INPUT_VARIANT. With the default, every CI run takes load_stage_input with INPUT_RESULTS_DIR equal to RESULTS_DIR; a fork exercised in CI would need a step in the CI pipeline, which lives outside this repository.

🤖 Generated with Claude Code

oharboe and others added 9 commits October 27, 2026 23:12
Every step that writes N.odb writes N.sdc beside it, and every step
reads the .sdc beside the .odb it loads. Before synthesis the .sdc is
the user's (1_2_yosys.sdc); from 1_synth on, every .odb has its own.

This deletes find_sdc_file, which picked an .sdc by name order and
gave open_*/gui_* another stage's constraints for a .def or a failed
.odb; the second argument of load_design, which named by hand which
earlier step's .sdc a script borrowed; the separate orfs_write_sdc and
flow_write_sdc calls, now part of orfs_write_db, orfs_copy_db and
flow_write_db; and every .sdc rule and prerequisite in the Makefile
except the user's, since a step's input .odb comes with its .sdc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
do-step (for .odb results) and do-copy declare the pair as one grouped
target, X.odb X.sdc &: deps, and 6_final.odb/6_final.sdc depend on
6_report.log again. A missing .sdc now reruns the step that writes it;
a do- target still runs on what is in the tree and stops at read_sdc.

orfs_write_db, orfs_copy_db, flow_write_db and do-copy write the .sdc
before the .odb, so the target make tracks is the last file written.

TclContract: orfs_write_db writes the .odb and its .sdc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>

# Conflicts:
#	flow/scripts/global_route.tcl
Before, every stage after synthesis globbed $(RESULTS_DIR)/memories for
*.lef and *.lib, the klayout tech took $(wildcard memories/*.lef), and
macro placement, regfile dissolve and synthesis read regfiles.txt,
inline.txt and regfile_cores.txt only if they existed. In a tree that
kept files from an earlier run, a .lib or .lef no longer generated was
read with the current ones, and a missing list was skipped.

Now each step reads the <m>.lib/<m>.lef of every module blackboxes.txt
lists, and the lists themselves without an existence check: a file no
list names is not read, and a missing list stops the step at the read.
The readers move from synth_preamble.tcl to util.tcl so OpenROAD and
yosys share them, and generate_klayout_tech.py takes --blackboxes.

On a clean tree the files read, and the results, are the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
…rom another variant

Before, a variant that wanted another variant's synthesis or placement
had to copy that variant's results into its own directory first.

Now FLOW_INPUT_VARIANT names the variant a stage reads the previous
stage's result from: INPUT_RESULTS_DIR. The six places a stage reads
its input say so explicitly, in its script (load_stage_input) and in
its Makefile prerequisite; every other read stays in RESULTS_DIR. It
defaults to FLOW_VARIANT, which leaves every path unchanged.

When it names another variant, load_stage_input stops if the input is
in both variants: one invocation that ran two stages, or a copy left
from an earlier run, where reading the other variant's copy would mix
results. It also stops with AUTO_MEMORIES=1: the memories are read from
the variant that synthesised them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the FLOW_INPUT_VARIANT mechanism, allowing a flow run to read inputs from a shared upstream variant directory while writing outputs to its own variant directory. This is supported by helper functions in Tcl and updated Makefile variables. However, the feedback highlights a critical issue in flow/util/utils.mk where passing only INPUT_*_DIR to genMetrics.py in the metadata-generate and update_metadata_autotuner targets will result in incorrect or incomplete metadata, as metrics for stages run in the current variant will be read from the upstream variant instead.

Comment thread flow/util/utils.mk Resolved
Comment thread flow/util/utils.mk Resolved
@oharboe
oharboe force-pushed the flow-input-variant branch from d426f98 to 371ea26 Compare November 9, 2026 15:30
@openroad-ci

Copy link
Copy Markdown
Member

🔍 QoR check

Metrics reflect the PR merge build — i.e. what will land on the target branch.

Commit 84d336e · Jenkins build #1 · Baseline: build · View build on dashboard

62 design(s) checked — 0 with regression(s), 0 without a comparable baseline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants