Skip to content

Commit e54722b

Browse files
authored
Merge pull request #4647 from The-OpenROAD-Project-staging/endpoint-slack-tracking
Track endpoint slack across flow steps
2 parents 12544aa + 7b9e765 commit e54722b

10 files changed

Lines changed: 1454 additions & 2 deletions

File tree

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# Tracking Endpoint Slack Across Flow Steps
2+
3+
When working on an optimization (resizing, buffering, CTS, repair timing,
4+
...) it is useful to see how the slack of each timing endpoint moves from
5+
one flow step to the next, not just how WNS/TNS change. ORFS can record the
6+
setup and hold slack of every endpoint at the end of every flow step
7+
(`1_synth`, `2_1_floorplan`, ..., `3_3_place_gp`, ..., `6_final`) and compare
8+
any two steps, including a scatter plot of slack (x: earlier step, y: later
9+
step).
10+
11+
## Recording
12+
13+
Set `REPORT_ENDPOINT_SLACK=1`:
14+
15+
``` shell
16+
cd flow
17+
make DESIGN_CONFIG=designs/nangate45/gcd/config.mk REPORT_ENDPOINT_SLACK=1
18+
```
19+
20+
Every step then writes `$(REPORTS_DIR)/<step>_endpoint_slack.json`, e.g.
21+
`3_3_place_gp_endpoint_slack.json`. The snapshot is taken by
22+
`write_endpoint_slack` in `flow/scripts/util.tcl`, called from
23+
`orfs_write_db` so the step name is the name of the `.odb` the step writes.
24+
It works in the single-process `flow/scripts/flow.tcl` flow as well. Steps
25+
that just copy their input (`orfs_copy_db`, e.g. `5_3_fillcell` without fill
26+
cells) re-time the design if it is loaded, or else copy the previous step's
27+
snapshot. Intermediate `.odb` files written through `orfs_write_db` (e.g. CTS
28+
`save_progress`) also get a snapshot; `*-failed` ones don't.
29+
30+
So that all steps are timed the same way, parasitics are estimated before
31+
reporting as in `open.tcl`, based on the step number:
32+
33+
| Steps | Parasitics |
34+
| --- | --- |
35+
| 1, 2 | none (cells are not placed) |
36+
| 3, 4 | `estimate_parasitics -placement` |
37+
| 5, `6_1_fill` | `estimate_parasitics -global_routing` if global routes exist |
38+
| `6_final` | `rcx`: the extracted `6_final.spef`, as in `6_finish.rpt` (`global_routing` on platforms without `RCX_RULES`) |
39+
40+
The parasitics model is recorded in each snapshot and shown by
41+
`endpointSlack.py`. Slack changes between steps with different models (e.g.
42+
`2_4_floorplan_pdn` to `3_1_place_gp_skip_io`, `4_1_cts` to `5_1_grt`, or
43+
`6_1_fill` to `6_final`) include the change of model. Detailed routing is
44+
only extracted in `6_final`, so `5_2_route` to `6_1_fill` show global routing
45+
estimates.
46+
47+
Each file contains `design`, `step`, `parasitics`, `time_unit` (seconds per
48+
time unit of the values) and `endpoints`: a map from every endpoint pin
49+
(`sta::endpoints`) to `[setup slack, hold slack]`, `null` where the endpoint
50+
is unconstrained. The slack is the pin's worst slack over all path groups
51+
and clocks (`get_property $pin slack_max|slack_min`). On asap7/swerv_wrapper
52+
(about 30k endpoints) a snapshot is about 1 MB and takes about 10 s, mostly
53+
the timing update.
54+
55+
## Analysis
56+
57+
`flow/util/endpointSlack.py` only uses the Python standard library.
58+
59+
``` shell
60+
R=reports/nangate45/gcd/base
61+
62+
# Steps in execution order with parasitics, WNS, TNS and violation counts
63+
python3 util/endpointSlack.py list $R
64+
65+
# Compare consecutive steps across the whole run; --html also writes a
66+
# scatter for every step vs. its successor and an index.html to a directory
67+
python3 util/endpointSlack.py track $R --html track
68+
69+
# Compare two steps; -o writes an interactive HTML scatter
70+
python3 util/endpointSlack.py compare \
71+
$R/3_5_place_dp_endpoint_slack.json $R/4_1_cts_endpoint_slack.json \
72+
--unit ps -o compare.html
73+
74+
# Flatten all snapshots into one CSV (one row per endpoint per step)
75+
python3 util/endpointSlack.py export $R -o slack.csv
76+
```
77+
78+
For setup and hold, `compare` reports:
79+
80+
- endpoints in both steps, and how many improved, degraded, stayed the
81+
same, became violating (`newly failing`) or stopped violating
82+
(`newly passing`), with statistics of the slack change (B - A).
83+
Improved, degraded and unchanged exclude endpoints with positive slack in
84+
both steps, which are not being optimized;
85+
- endpoints only in A or only in B, i.e. added or removed from the netlist;
86+
- endpoints in both netlists but constrained in only one of them;
87+
- how well the two steps agree on the critical endpoints (see below).
88+
89+
Unconstrained endpoints, such as the outputs of CTS dummy load cells, are
90+
ignored.
91+
92+
### Agreement metrics
93+
94+
Correlation over all endpoints is dominated by endpoints with large positive
95+
slack and is close to 1 even when the critical endpoints change a lot (0.9997
96+
for setup on asap7/swerv_wrapper from `3_5_place_dp` to `4_1_cts`). The
97+
metrics are therefore measured on a critical set: the worst K endpoints of A
98+
or of B (`--critical K`, default 1000), excluding endpoints with positive
99+
slack in both steps, which are not being optimized. When fewer than K
100+
endpoints violate, the critical set is the endpoints violating in either
101+
step; when nothing violates (often hold) it is empty and the metrics are
102+
reported as `-`.
103+
104+
| Metric | Meaning |
105+
| --- | --- |
106+
| `spearman` | Spearman rank correlation of A and B slack (ties get average ranks): is the order of the critical endpoints preserved? |
107+
| `top agreement` | Fraction of the critical set that is in the worst K of both steps. |
108+
| `bias`, `MAE`, `RMSE`, `max\|delta\|` | Mean, mean absolute, root mean square and maximum absolute slack change B - A, i.e. distance from the y = x line. |
109+
| `violator recall` | Of the endpoints violating in B, the fraction already violating in A (over all endpoints). |
110+
| `violator precision` | Of the endpoints violating in A, the fraction still violating in B (over all endpoints). |
111+
112+
They are printed by `compare` and `track`, shown in the HTML stats, and
113+
the `track` index has Spearman, top agreement and MAE columns for every step
114+
transition.
115+
116+
Times are shown in `--unit` (default ns) with a fixed number of digits after
117+
the decimal point (3 for ns, 1 for ps, or `--digits`), in a fixed-width font
118+
in the HTML.
119+
Slack changes smaller than half the last displayed digit (0.5 ps with the
120+
default ns and 3 digits), or `--tolerance` in `--unit`, count as unchanged
121+
for the improved/degraded counts, the tables of most degraded and improved
122+
endpoints and the `track` row colors. Otherwise numerical noise, e.g. a few
123+
femtoseconds between `5_1_grt` and `5_2_route`, shows up as changes that
124+
display as 0.
125+
126+
`--worst N` restricts the endpoints in both to the union of the worst N
127+
endpoints of A and of B, to focus on the critical part of the design.
128+
129+
The HTML is self-contained (no network access needed). It has:
130+
131+
- WNS, TNS and violation counts of both steps;
132+
- a scatter of endpoint slack with a y = x line, where above the line means
133+
the endpoint improved. It is drawn on a canvas so it handles hundreds of
134+
thousands of endpoints. Drag to zoom, double-click to reset, hover for
135+
the endpoint name;
136+
- a setup/hold toggle;
137+
- tables of the most degraded and most improved endpoints, excluding
138+
endpoints with positive slack in both steps, and of the endpoints only in
139+
one step.
140+
141+
`track --html DIR` writes the same page for every step and its successor
142+
(`<step>__<next step>.html`, with previous/next links) and `DIR/index.html`,
143+
a table of every transition with WNS/TNS and the improved, degraded, newly
144+
failing and newly passing counts for setup and hold. Endpoints with positive
145+
slack in both steps are not considered. Transitions where no considered
146+
endpoint changed (e.g. `2_2_floorplan_macro` to `2_3_floorplan_tapcell`) are
147+
greyed out, and transitions where every considered endpoint improved, for
148+
setup and hold, are green.
149+
150+
For other analyses, import the module (e.g. from a notebook):
151+
`load_snapshot()`, `load_dir()`, `summary()`, `compare()`, `movement()`,
152+
`metrics()` and `spearman()`
153+
return plain Python dicts and lists.
154+
155+
## Testing
156+
157+
`flow/test/test_endpointSlack.py` covers loading and comparison with
158+
synthetic snapshots and runs in the util CI job:
159+
160+
``` shell
161+
python3 flow/test/test_endpointSlack.py
162+
```

‎docs/toc.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,8 @@ entries:
4545
title: Guide to Integrate a New Platform into the OpenROAD Flow
4646
- file: contrib/CI.md
4747
title: CI Guide
48+
- file: contrib/EndpointSlackTracking.md
49+
title: Tracking Endpoint Slack Across Flow Steps
4850

4951
- file: contrib/CODE_OF_CONDUCT.md
5052
title: Code of Conduct

‎docs/user/FlowVariables.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,6 +247,7 @@ configuration file.
247247
| <a name="REMOVE_CELLS_FOR_LEC"></a>REMOVE_CELLS_FOR_LEC| String patterns directly passed to write_verilog -remove_cells <> for lec checks.| |
248248
| <a name="REPAIR_PDN_VIA_LAYER"></a>REPAIR_PDN_VIA_LAYER| Remove power grid vias which generate DRC violations after detailed routing.| |
249249
| <a name="REPORT_CLOCK_SKEW"></a>REPORT_CLOCK_SKEW| Report clock skew as part of reporting metrics, starting at CTS, before which there is no clock skew. This metric can be quite time-consuming, so it can be useful to disable.| 1|
250+
| <a name="REPORT_ENDPOINT_SLACK"></a>REPORT_ENDPOINT_SLACK| If set to 1, write the setup and hold slack of every timing endpoint at the end of every flow step (e.g. 3_3_place_gp) to $REPORTS_DIR/<step>_endpoint_slack.json. These snapshots can be compared across steps with flow/util/endpointSlack.py. See docs/contrib/EndpointSlackTracking.md.| 0|
250251
| <a name="ROUTING_LAYER_ADJUSTMENT"></a>ROUTING_LAYER_ADJUSTMENT| Adjusts routing layer capacities to manage congestion and improve detailed routing. High values ease detailed routing but risk excessive detours and long global routing times, while low values reduce global routing failure but can complicate detailed routing. The global routing running time normally reduces dramatically (entirely design specific, but going from hours to minutes has been observed) when the value is low (such as 0.10). Sometimes, global routing will succeed with lower values and fail with higher values. Exploring results with different values can help shed light on the problem. Start with a too low value, such as 0.10, and bisect to value that works by doing multiple global routing runs. As a last resort, `make global_route_issue` and using the tools/OpenROAD/etc/whittle.py can be useful to debug global routing errors. If there is something specific that is impossible to route, such as a clock line over a macro, global routing will terminate with DRC errors routes that could have been routed were it not for the specific impossible routes. whittle.py should weed out the possible routes and leave a minimal failing case that pinpoints the problem.| 0.5|
251252
| <a name="RTLMP_AREA_WT"></a>RTLMP_AREA_WT| Weight for the area of the current floorplan.| 0.1|
252253
| <a name="RTLMP_ARGS"></a>RTLMP_ARGS| Overrides all other RTL macro placer arguments.| |
@@ -354,6 +355,7 @@ configuration file.
354355
- [POST_SYNTH_TCL](#POST_SYNTH_TCL)
355356
- [POS_CLKGATE_AND_PORTS](#POS_CLKGATE_AND_PORTS)
356357
- [PRE_SYNTH_TCL](#PRE_SYNTH_TCL)
358+
- [REPORT_ENDPOINT_SLACK](#REPORT_ENDPOINT_SLACK)
357359
- [SDC_FILE](#SDC_FILE)
358360
- [SEC_CHECK](#SEC_CHECK)
359361
- [SKIP_REPORT_METRICS](#SKIP_REPORT_METRICS)
@@ -425,6 +427,7 @@ configuration file.
425427
- [PRE_PDN_TCL](#PRE_PDN_TCL)
426428
- [PRE_TAPCELL_TCL](#PRE_TAPCELL_TCL)
427429
- [REMOVE_ABC_BUFFERS](#REMOVE_ABC_BUFFERS)
430+
- [REPORT_ENDPOINT_SLACK](#REPORT_ENDPOINT_SLACK)
428431
- [ROUTING_LAYER_ADJUSTMENT](#ROUTING_LAYER_ADJUSTMENT)
429432
- [RTLMP_AREA_WT](#RTLMP_AREA_WT)
430433
- [RTLMP_ARGS](#RTLMP_ARGS)
@@ -502,6 +505,7 @@ configuration file.
502505
- [PRE_GLOBAL_PLACE_TCL](#PRE_GLOBAL_PLACE_TCL)
503506
- [PRE_IO_PLACEMENT_TCL](#PRE_IO_PLACEMENT_TCL)
504507
- [PRE_RESIZE_TCL](#PRE_RESIZE_TCL)
508+
- [REPORT_ENDPOINT_SLACK](#REPORT_ENDPOINT_SLACK)
505509
- [ROUTING_LAYER_ADJUSTMENT](#ROUTING_LAYER_ADJUSTMENT)
506510
- [SKIP_REPORT_METRICS](#SKIP_REPORT_METRICS)
507511
- [TNS_END_PERCENT](#TNS_END_PERCENT)
@@ -525,6 +529,7 @@ configuration file.
525529
- [POST_CTS_TCL](#POST_CTS_TCL)
526530
- [PRE_CTS_TCL](#PRE_CTS_TCL)
527531
- [REPORT_CLOCK_SKEW](#REPORT_CLOCK_SKEW)
532+
- [REPORT_ENDPOINT_SLACK](#REPORT_ENDPOINT_SLACK)
528533
- [SETUP_MOVE_SEQUENCE](#SETUP_MOVE_SEQUENCE)
529534
- [SETUP_SLACK_MARGIN](#SETUP_SLACK_MARGIN)
530535
- [SKIP_BUFFER_REMOVAL](#SKIP_BUFFER_REMOVAL)
@@ -554,6 +559,7 @@ configuration file.
554559
- [POST_GLOBAL_ROUTE_TCL](#POST_GLOBAL_ROUTE_TCL)
555560
- [PRE_GLOBAL_ROUTE_TCL](#PRE_GLOBAL_ROUTE_TCL)
556561
- [REPORT_CLOCK_SKEW](#REPORT_CLOCK_SKEW)
562+
- [REPORT_ENDPOINT_SLACK](#REPORT_ENDPOINT_SLACK)
557563
- [ROUTING_LAYER_ADJUSTMENT](#ROUTING_LAYER_ADJUSTMENT)
558564
- [SETUP_MOVE_SEQUENCE](#SETUP_MOVE_SEQUENCE)
559565
- [SETUP_SLACK_MARGIN](#SETUP_SLACK_MARGIN)
@@ -587,6 +593,7 @@ configuration file.
587593
- [PRE_DETAIL_ROUTE_TCL](#PRE_DETAIL_ROUTE_TCL)
588594
- [PRE_FILLCELL_TCL](#PRE_FILLCELL_TCL)
589595
- [REPORT_CLOCK_SKEW](#REPORT_CLOCK_SKEW)
596+
- [REPORT_ENDPOINT_SLACK](#REPORT_ENDPOINT_SLACK)
590597
- [ROUTING_LAYER_ADJUSTMENT](#ROUTING_LAYER_ADJUSTMENT)
591598
- [SKIP_ANTENNA_REPAIR_POST_DRT](#SKIP_ANTENNA_REPAIR_POST_DRT)
592599
- [SKIP_DETAILED_ROUTE](#SKIP_DETAILED_ROUTE)
@@ -610,6 +617,7 @@ configuration file.
610617
- [PRE_FINAL_REPORT_TCL](#PRE_FINAL_REPORT_TCL)
611618
- [PWR_NETS_VOLTAGES](#PWR_NETS_VOLTAGES)
612619
- [REPORT_CLOCK_SKEW](#REPORT_CLOCK_SKEW)
620+
- [REPORT_ENDPOINT_SLACK](#REPORT_ENDPOINT_SLACK)
613621
- [ROUTING_LAYER_ADJUSTMENT](#ROUTING_LAYER_ADJUSTMENT)
614622
- [SEC_CHECK](#SEC_CHECK)
615623
- [SKIP_DETAILED_ROUTE](#SKIP_DETAILED_ROUTE)

‎flow/scripts/final_outputs.tcl‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ if {
3535

3636
# Read Spef for OpenSTA
3737
read_spef $::env(RESULTS_DIR)/6_final.spef
38+
set final_parasitics rcx
3839

3940
# Static IR drop analysis
4041
if { [env_var_exists_and_non_empty PWR_NETS_VOLTAGES] } {
@@ -59,7 +60,9 @@ if {
5960
puts "OpenRCX is not enabled for this platform."
6061
puts "Falling back to global route-based estimates."
6162
log_cmd estimate_parasitics -global_routing
63+
set final_parasitics global_routing
6264
}
65+
write_endpoint_slack 6_final $final_parasitics
6366

6467
report_cell_usage
6568

‎flow/scripts/final_report.tcl‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
# (scripts/flow.tcl) can source the parts and perform the write itself
33
# at the same point.
44
source $::env(SCRIPTS_DIR)/final_connect.tcl
5-
orfs_write_db $::env(RESULTS_DIR)/6_final.odb
5+
# final_outputs.tcl writes the endpoint slack after parasitic extraction
6+
orfs_write_db $::env(RESULTS_DIR)/6_final.odb 0
67
orfs_write_sdc $::env(RESULTS_DIR)/6_final.sdc
78
source $::env(SCRIPTS_DIR)/final_outputs.tcl

‎flow/scripts/util.tcl‎

Lines changed: 96 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -302,7 +302,101 @@ proc find_physical_only_masters { } {
302302
return $physical_only_masters
303303
}
304304

305-
proc orfs_write_db { output_file } {
305+
proc endpoint_slack_file { step } {
306+
return $::env(REPORTS_DIR)/${step}_endpoint_slack.json
307+
}
308+
309+
# Write the setup and hold slack of every endpoint at the end of a flow
310+
# step so they can be compared across steps with
311+
# flow/util/endpointSlack.py. Parasitics are estimated the same way as
312+
# open.tcl so that every step is timed consistently, unless the caller
313+
# has already loaded parasitics (e.g. extracted ones) and passes their
314+
# name as parasitics.
315+
proc write_endpoint_slack { step { parasitics "" } } {
316+
if { !$::env(REPORT_ENDPOINT_SLACK) || [string match "*-failed" $step] } {
317+
return
318+
}
319+
set filename [endpoint_slack_file $step]
320+
puts "Writing endpoint slack to $filename"
321+
322+
if { $parasitics == "" } {
323+
set design_stage [lindex [split $step _] 0]
324+
set parasitics none
325+
if { $design_stage >= 5 } {
326+
if { [grt::have_routes] } {
327+
set parasitics global_routing
328+
}
329+
} elseif { $design_stage >= 3 } {
330+
set parasitics placement
331+
}
332+
if { $parasitics != "none" } {
333+
estimate_parasitics -$parasitics
334+
}
335+
}
336+
# Update timing before sta::endpoints: the endpoint set is cached the
337+
# first time it is requested and misses gated clock enable checks if the
338+
# clock network has not been found yet.
339+
sta::find_timing
340+
341+
set fileId [open $filename w]
342+
puts $fileId "{"
343+
puts $fileId "\"design\": \"$::env(DESIGN_NAME)\","
344+
puts $fileId "\"step\": \"$step\","
345+
puts $fileId "\"parasitics\": \"$parasitics\","
346+
# Seconds per reported time unit
347+
puts $fileId "\"time_unit\": [sta::time_ui_sta 1],"
348+
# endpoint: [setup slack, hold slack], null if unconstrained
349+
puts $fileId "\"endpoints\": {"
350+
set sep ""
351+
foreach pin [sta::endpoints] {
352+
set name [string map {\\ \\\\ \" \\\"} [get_full_name $pin]]
353+
set slacks {}
354+
foreach min_max {max min} {
355+
set slack [get_property $pin slack_$min_max]
356+
if { $slack eq "INF" || $slack eq "-INF" } {
357+
set slack null
358+
}
359+
lappend slacks $slack
360+
}
361+
puts -nonewline $fileId "$sep\"$name\": \[[join $slacks ,]\]"
362+
set sep ",\n"
363+
}
364+
puts $fileId "\n}"
365+
puts $fileId "}"
366+
close $fileId
367+
}
368+
369+
# A step that copies its input .odb leaves the design unchanged, so
370+
# re-time it if it is loaded, otherwise copy the input step's snapshot.
371+
proc copy_endpoint_slack { input_file output_file } {
372+
if { !$::env(REPORT_ENDPOINT_SLACK) } {
373+
return
374+
}
375+
set step [file rootname [file tail $output_file]]
376+
set chip [[ord::get_db] getChip]
377+
if { $chip != "NULL" && [$chip getBlock] != "NULL" } {
378+
write_endpoint_slack $step
379+
return
380+
}
381+
set input [endpoint_slack_file [file rootname [file tail $input_file]]]
382+
if { ![file exists $input] } {
383+
return
384+
}
385+
set fileId [open $input r]
386+
set json [read $fileId]
387+
close $fileId
388+
regsub {"step": "[^"]*"} $json "\"step\": \"$step\"" json
389+
set fileId [open [endpoint_slack_file $step] w]
390+
puts -nonewline $fileId $json
391+
close $fileId
392+
}
393+
394+
# endpoint_slack 0 is for callers that write the endpoint slack snapshot
395+
# themselves, e.g. 6_final after parasitic extraction.
396+
proc orfs_write_db { output_file { endpoint_slack 1 } } {
397+
if { $endpoint_slack } {
398+
write_endpoint_slack [file rootname [file tail $output_file]]
399+
}
306400
if { !$::env(WRITE_ODB_AND_SDC_EACH_STAGE) } {
307401
return
308402
}
@@ -326,6 +420,7 @@ proc orfs_write_sdc { output_file } {
326420
# mtime with second resolution, which breaks make's timestamp-based
327421
# up-to-date checks for fast builds.
328422
proc orfs_copy_db { input_file output_file } {
423+
copy_endpoint_slack $input_file $output_file
329424
if { !$::env(WRITE_ODB_AND_SDC_EACH_STAGE) } {
330425
return
331426
}

‎flow/scripts/variables.json‎

Lines changed: 13 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)