|
| 1 | +# Tracking Endpoint Slack Across Flow Steps |
| 2 | + |
| 3 | +When working on an optimization (resizing, buffering, CTS, repair timing, |
| 4 | +...) it is useful to see how the slack of each timing endpoint moves from |
| 5 | +one flow step to the next, not just how WNS/TNS change. ORFS can record the |
| 6 | +setup and hold slack of every endpoint at the end of every flow step |
| 7 | +(`1_synth`, `2_1_floorplan`, ..., `3_3_place_gp`, ..., `6_final`) and compare |
| 8 | +any two steps, including a scatter plot of slack (x: earlier step, y: later |
| 9 | +step). |
| 10 | + |
| 11 | +## Recording |
| 12 | + |
| 13 | +Set `REPORT_ENDPOINT_SLACK=1`: |
| 14 | + |
| 15 | +``` shell |
| 16 | +cd flow |
| 17 | +make DESIGN_CONFIG=designs/nangate45/gcd/config.mk REPORT_ENDPOINT_SLACK=1 |
| 18 | +``` |
| 19 | + |
| 20 | +Every step then writes `$(REPORTS_DIR)/<step>_endpoint_slack.json`, e.g. |
| 21 | +`3_3_place_gp_endpoint_slack.json`. The snapshot is taken by |
| 22 | +`write_endpoint_slack` in `flow/scripts/util.tcl`, called from |
| 23 | +`orfs_write_db` so the step name is the name of the `.odb` the step writes. |
| 24 | +It works in the single-process `flow/scripts/flow.tcl` flow as well. Steps |
| 25 | +that just copy their input (`orfs_copy_db`, e.g. `5_3_fillcell` without fill |
| 26 | +cells) re-time the design if it is loaded, or else copy the previous step's |
| 27 | +snapshot. Intermediate `.odb` files written through `orfs_write_db` (e.g. CTS |
| 28 | +`save_progress`) also get a snapshot; `*-failed` ones don't. |
| 29 | + |
| 30 | +So that all steps are timed the same way, parasitics are estimated before |
| 31 | +reporting as in `open.tcl`, based on the step number: |
| 32 | + |
| 33 | +| Steps | Parasitics | |
| 34 | +| --- | --- | |
| 35 | +| 1, 2 | none (cells are not placed) | |
| 36 | +| 3, 4 | `estimate_parasitics -placement` | |
| 37 | +| 5, `6_1_fill` | `estimate_parasitics -global_routing` if global routes exist | |
| 38 | +| `6_final` | `rcx`: the extracted `6_final.spef`, as in `6_finish.rpt` (`global_routing` on platforms without `RCX_RULES`) | |
| 39 | + |
| 40 | +The parasitics model is recorded in each snapshot and shown by |
| 41 | +`endpointSlack.py`. Slack changes between steps with different models (e.g. |
| 42 | +`2_4_floorplan_pdn` to `3_1_place_gp_skip_io`, `4_1_cts` to `5_1_grt`, or |
| 43 | +`6_1_fill` to `6_final`) include the change of model. Detailed routing is |
| 44 | +only extracted in `6_final`, so `5_2_route` to `6_1_fill` show global routing |
| 45 | +estimates. |
| 46 | + |
| 47 | +Each file contains `design`, `step`, `parasitics`, `time_unit` (seconds per |
| 48 | +time unit of the values) and `endpoints`: a map from every endpoint pin |
| 49 | +(`sta::endpoints`) to `[setup slack, hold slack]`, `null` where the endpoint |
| 50 | +is unconstrained. The slack is the pin's worst slack over all path groups |
| 51 | +and clocks (`get_property $pin slack_max|slack_min`). On asap7/swerv_wrapper |
| 52 | +(about 30k endpoints) a snapshot is about 1 MB and takes about 10 s, mostly |
| 53 | +the timing update. |
| 54 | + |
| 55 | +## Analysis |
| 56 | + |
| 57 | +`flow/util/endpointSlack.py` only uses the Python standard library. |
| 58 | + |
| 59 | +``` shell |
| 60 | +R=reports/nangate45/gcd/base |
| 61 | + |
| 62 | +# Steps in execution order with parasitics, WNS, TNS and violation counts |
| 63 | +python3 util/endpointSlack.py list $R |
| 64 | + |
| 65 | +# Compare consecutive steps across the whole run; --html also writes a |
| 66 | +# scatter for every step vs. its successor and an index.html to a directory |
| 67 | +python3 util/endpointSlack.py track $R --html track |
| 68 | + |
| 69 | +# Compare two steps; -o writes an interactive HTML scatter |
| 70 | +python3 util/endpointSlack.py compare \ |
| 71 | + $R/3_5_place_dp_endpoint_slack.json $R/4_1_cts_endpoint_slack.json \ |
| 72 | + --unit ps -o compare.html |
| 73 | + |
| 74 | +# Flatten all snapshots into one CSV (one row per endpoint per step) |
| 75 | +python3 util/endpointSlack.py export $R -o slack.csv |
| 76 | +``` |
| 77 | + |
| 78 | +For setup and hold, `compare` reports: |
| 79 | + |
| 80 | +- endpoints in both steps, and how many improved, degraded, stayed the |
| 81 | + same, became violating (`newly failing`) or stopped violating |
| 82 | + (`newly passing`), with statistics of the slack change (B - A). |
| 83 | + Improved, degraded and unchanged exclude endpoints with positive slack in |
| 84 | + both steps, which are not being optimized; |
| 85 | +- endpoints only in A or only in B, i.e. added or removed from the netlist; |
| 86 | +- endpoints in both netlists but constrained in only one of them; |
| 87 | +- how well the two steps agree on the critical endpoints (see below). |
| 88 | + |
| 89 | +Unconstrained endpoints, such as the outputs of CTS dummy load cells, are |
| 90 | +ignored. |
| 91 | + |
| 92 | +### Agreement metrics |
| 93 | + |
| 94 | +Correlation over all endpoints is dominated by endpoints with large positive |
| 95 | +slack and is close to 1 even when the critical endpoints change a lot (0.9997 |
| 96 | +for setup on asap7/swerv_wrapper from `3_5_place_dp` to `4_1_cts`). The |
| 97 | +metrics are therefore measured on a critical set: the worst K endpoints of A |
| 98 | +or of B (`--critical K`, default 1000), excluding endpoints with positive |
| 99 | +slack in both steps, which are not being optimized. When fewer than K |
| 100 | +endpoints violate, the critical set is the endpoints violating in either |
| 101 | +step; when nothing violates (often hold) it is empty and the metrics are |
| 102 | +reported as `-`. |
| 103 | + |
| 104 | +| Metric | Meaning | |
| 105 | +| --- | --- | |
| 106 | +| `spearman` | Spearman rank correlation of A and B slack (ties get average ranks): is the order of the critical endpoints preserved? | |
| 107 | +| `top agreement` | Fraction of the critical set that is in the worst K of both steps. | |
| 108 | +| `bias`, `MAE`, `RMSE`, `max\|delta\|` | Mean, mean absolute, root mean square and maximum absolute slack change B - A, i.e. distance from the y = x line. | |
| 109 | +| `violator recall` | Of the endpoints violating in B, the fraction already violating in A (over all endpoints). | |
| 110 | +| `violator precision` | Of the endpoints violating in A, the fraction still violating in B (over all endpoints). | |
| 111 | + |
| 112 | +They are printed by `compare` and `track`, shown in the HTML stats, and |
| 113 | +the `track` index has Spearman, top agreement and MAE columns for every step |
| 114 | +transition. |
| 115 | + |
| 116 | +Times are shown in `--unit` (default ns) with a fixed number of digits after |
| 117 | +the decimal point (3 for ns, 1 for ps, or `--digits`), in a fixed-width font |
| 118 | +in the HTML. |
| 119 | +Slack changes smaller than half the last displayed digit (0.5 ps with the |
| 120 | +default ns and 3 digits), or `--tolerance` in `--unit`, count as unchanged |
| 121 | +for the improved/degraded counts, the tables of most degraded and improved |
| 122 | +endpoints and the `track` row colors. Otherwise numerical noise, e.g. a few |
| 123 | +femtoseconds between `5_1_grt` and `5_2_route`, shows up as changes that |
| 124 | +display as 0. |
| 125 | + |
| 126 | +`--worst N` restricts the endpoints in both to the union of the worst N |
| 127 | +endpoints of A and of B, to focus on the critical part of the design. |
| 128 | + |
| 129 | +The HTML is self-contained (no network access needed). It has: |
| 130 | + |
| 131 | +- WNS, TNS and violation counts of both steps; |
| 132 | +- a scatter of endpoint slack with a y = x line, where above the line means |
| 133 | + the endpoint improved. It is drawn on a canvas so it handles hundreds of |
| 134 | + thousands of endpoints. Drag to zoom, double-click to reset, hover for |
| 135 | + the endpoint name; |
| 136 | +- a setup/hold toggle; |
| 137 | +- tables of the most degraded and most improved endpoints, excluding |
| 138 | + endpoints with positive slack in both steps, and of the endpoints only in |
| 139 | + one step. |
| 140 | + |
| 141 | +`track --html DIR` writes the same page for every step and its successor |
| 142 | +(`<step>__<next step>.html`, with previous/next links) and `DIR/index.html`, |
| 143 | +a table of every transition with WNS/TNS and the improved, degraded, newly |
| 144 | +failing and newly passing counts for setup and hold. Endpoints with positive |
| 145 | +slack in both steps are not considered. Transitions where no considered |
| 146 | +endpoint changed (e.g. `2_2_floorplan_macro` to `2_3_floorplan_tapcell`) are |
| 147 | +greyed out, and transitions where every considered endpoint improved, for |
| 148 | +setup and hold, are green. |
| 149 | + |
| 150 | +For other analyses, import the module (e.g. from a notebook): |
| 151 | +`load_snapshot()`, `load_dir()`, `summary()`, `compare()`, `movement()`, |
| 152 | +`metrics()` and `spearman()` |
| 153 | +return plain Python dicts and lists. |
| 154 | + |
| 155 | +## Testing |
| 156 | + |
| 157 | +`flow/test/test_endpointSlack.py` covers loading and comparison with |
| 158 | +synthetic snapshots and runs in the util CI job: |
| 159 | + |
| 160 | +``` shell |
| 161 | +python3 flow/test/test_endpointSlack.py |
| 162 | +``` |
0 commit comments