|
| 1 | +# tpu: abutted tiles, the parent a macro too |
| 2 | + |
| 3 | +A systolic array of 8 x 8 processing elements (`src/tpu`), built as 17 |
| 4 | +macros and no standard cell at the top: |
| 5 | + |
| 6 | +- **16 identical tiles** (`tpu_block`) of 2 x 2 processing elements, |
| 7 | + carved out of the flat design during synthesis |
| 8 | + (`SYNTH_VERILOG_SURGERY`, `surgery.py` running `flow/util/carve.py`): |
| 9 | + the design's own cells regrouped, the RTL untouched. |
| 10 | +- **the parent as a macro** (`tpu_rest`): everything else of tpu, the |
| 11 | + skew and deskew registers, the clock root, the weights in, is one |
| 12 | + C-shaped macro whose notch holds the tiles (`flow/util/wrap_parent.py`, |
| 13 | + `tpu_io.tcl`). Its abstract obstructs only what it occupies |
| 14 | + (`WRITE_ABSTRACT_LEF_ARGS` empty), so the notch stays free. |
| 15 | +- **abutment**: every tile meets its neighbours and the parent pin on |
| 16 | + pin. Every pin is placed by rule, from the constants in `tpu_io.tcl`, |
| 17 | + before anything is built, so the tile and the parent build side by |
| 18 | + side; the top level places the macros (`place.tcl`) and is placement |
| 19 | + and wires only. |
| 20 | + |
| 21 | +`make DESIGN_CONFIG=./designs/asap7/tpu/config.mk FLOW_VARIANT=flat` |
| 22 | +builds tpu flat from the same RTL, the reference. |
| 23 | + |
| 24 | + |
| 25 | + |
| 26 | +Placement density at global route, each tile's own cells drawn in place: |
| 27 | +the tiles are 80 % full; the parent's three sides around them hold its |
| 28 | +skew and deskew registers sparsely. |
| 29 | + |
| 30 | +## Why |
| 31 | + |
| 32 | +QoR is the best result within a build budget. In a hierarchical flow the |
| 33 | +parent places, repairs and routes around its macros' abstracts: it |
| 34 | +waits for every macro, and every macro change re-places it. Here the |
| 35 | +parent is a macro like the tiles, built against the same pin plan, so |
| 36 | +every macro builds side by side and an RTL edit in a tile rebuilds that |
| 37 | +tile alone. Measured at global route, with the reg2reg minimum period |
| 38 | +stitched across every macro (the study: |
| 39 | +https://github.com/The-OpenROAD-Project/bazel-orfs/pull/1396): |
| 40 | + |
| 41 | +| | tiles in a C | flat | |
| 42 | +|---|---|---| |
| 43 | +| reg2reg minimum period, mean of three placement seeds | 672.6 ps (92 %) | 729.3 ps | |
| 44 | +| a fresh build, critical path | 76 s | 436 s | |
| 45 | +| an RTL edit in a processing element | 99 s | 487 s | |
| 46 | +| die | 1.39 x | 1 x | |
| 47 | + |
| 48 | + |
| 49 | + |
| 50 | +The carved design's first sensible floorplan beats everything flat |
| 51 | +reaches with five times the CPU; tuning either one moves its period by |
| 52 | +less than a placement seed does. |
| 53 | + |
| 54 | + |
| 55 | + |
| 56 | +## The rules of the game |
| 57 | + |
| 58 | +tpu is mesochronous with its surroundings: the same clock frequency, a |
| 59 | +fixed but unknown phase. Its clock reaches each tile through the tiles |
| 60 | +before it, so the phase at its registers is not the clock root's; data |
| 61 | +crosses its boundary through FIFOs, which are assumed to be outside tpu. |
| 62 | +So only reg2reg paths are judged, and the boundary is a `set_max_delay` |
| 63 | +budget for repair, with no input or output delay (`constraint.sdc`, |
| 64 | +`flow/platforms/asap7/constraints.sdc`). tpu has every pin on its bottom |
| 65 | +edge, as a macro abutting its parent would; inside, each macro has its |
| 66 | +pins on the sides its neighbours are on. Every macro routes on M2-M8; |
| 67 | +M9 brings power from above. |
| 68 | + |
| 69 | +## How it is clocked |
| 70 | + |
| 71 | +The clock, reset and load enable enter each row of tiles from the |
| 72 | +parent at its west end and run east through the tiles with the |
| 73 | +activations, one buffer per tile; rows enter together. Each tile sees |
| 74 | +the clock about 77 ps after its west neighbour, so data flowing east is |
| 75 | +captured later than it is launched, and data flowing south sees no |
| 76 | +skew. Only neighbours' skew matters, as Fisher and Kung showed for |
| 77 | +pipelined clocks in systolic arrays: no clock tree is balanced across |
| 78 | +the array. The bottom row passes its clock down to the deskew registers |
| 79 | +in the parent. |
| 80 | + |
| 81 | + |
| 82 | + |
| 83 | + |
| 84 | + |
| 85 | +## Files |
| 86 | + |
| 87 | +| file | what | |
| 88 | +|---|---| |
| 89 | +| `config.mk` | tpu: the top level of macros (base), or flat (`FLOW_VARIANT=flat`) | |
| 90 | +| `tpu_block/config.mk` | the tile, with the settings a placement and clock-tree search found best | |
| 91 | +| `tpu_rest/config.mk` | the parent macro | |
| 92 | +| `surgery.py` | the carve and the parent's module, during synthesis | |
| 93 | +| `tpu_io.tcl` | the pins of the tile and of the parent, and the parent's notch | |
| 94 | +| `place.tcl`, `io_top.tcl`, `pdn_top.tcl`, `tapcell_none.tcl`, `dpl_none.tcl` | the top level of macros only | |
| 95 | +| `pdn_macro.tcl` | every macro's power: rails, M5 straps, M8 straps the top level's M9 lands on | |
| 96 | +| `pins_bottom.tcl` | flat's pins, all on its bottom edge | |
| 97 | +| `*.svg` | the figures above, from the study | |
| 98 | + |
| 99 | +## Limits |
| 100 | + |
| 101 | +- **Hold fails across macros.** Timed with every macro's own cells and |
| 102 | + wires, 143 endpoints from a tile into the parent fail hold, worst |
| 103 | + -417.4 ps, and 75 inside the parent, worst -130.0 ps. The parent's |
| 104 | + deskew registers take their clock from the bottom row of tiles through |
| 105 | + the parent's own tree, about 250 ps after the tile registers that feed |
| 106 | + them, and the parent mixes those clocks with its own; no flow repairs |
| 107 | + hold across macros, each sees only its side. The periods above are |
| 108 | + setup only; with the clock paths derated 5 % either way and common path |
| 109 | + pessimism removed, the carved period is 668.3 ps. |
| 110 | + |
| 111 | +- **Area**: 1.39 x flat's die. The tiles are 80 % full; the parent's |
| 112 | + three sides around them hold its 676 um2 of registers at 5.6 %. |
| 113 | +- **The parent's repair does not see inside the tiles**: it times their |
| 114 | + pins as ports with budgets. tpu's worst path starts and ends inside one |
| 115 | + tile, so this costs nothing here. |
| 116 | +- **Power at the top level reaches the parent only**: pdngen takes the |
| 117 | + parent's whole outline as occupied and draws nothing over the tiles. |
0 commit comments