Skip to content

Commit d128b07

Browse files
oharboeclaude
andcommitted
asap7/tpu: abutted tiles, the parent a macro too; flat as a variant
tpu's 8 x 8 array of processing elements, built as 17 macros and no standard cell at the top: 16 identical tiles of 2 x 2 elements carved out of the flat design during synthesis (SYNTH_VERILOG_SURGERY running flow/util/carve.py; the RTL untouched), and the rest of tpu built as one C-shaped macro whose notch holds them (flow/util/wrap_parent.py). Every pin is placed by rule from the constants in tpu_io.tcl before anything is built, so the tile and the parent build side by side; the top level is placement and wires. FLOW_VARIANT=flat builds tpu flat from the same RTL, the reference. Both use the platform's constraints: tpu is mesochronous with its surroundings, so only reg2reg paths are judged and the boundary is a set_max_delay budget. Every macro routes on M2-M8, tpu's pins on its bottom edge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
1 parent 48256c8 commit d128b07

20 files changed

Lines changed: 3649 additions & 21 deletions

‎flow/designs/asap7/tpu/README.md‎

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# tpu: abutted tiles, the parent a macro too
2+
3+
A systolic array of 8 x 8 processing elements (`src/tpu`), built as 17
4+
macros and no standard cell at the top:
5+
6+
- **16 identical tiles** (`tpu_block`) of 2 x 2 processing elements,
7+
carved out of the flat design during synthesis
8+
(`SYNTH_VERILOG_SURGERY`, `surgery.py` running `flow/util/carve.py`):
9+
the design's own cells regrouped, the RTL untouched.
10+
- **the parent as a macro** (`tpu_rest`): everything else of tpu, the
11+
skew and deskew registers, the clock root, the weights in, is one
12+
C-shaped macro whose notch holds the tiles (`flow/util/wrap_parent.py`,
13+
`tpu_io.tcl`). Its abstract obstructs only what it occupies
14+
(`WRITE_ABSTRACT_LEF_ARGS` empty), so the notch stays free.
15+
- **abutment**: every tile meets its neighbours and the parent pin on
16+
pin. Every pin is placed by rule, from the constants in `tpu_io.tcl`,
17+
before anything is built, so the tile and the parent build side by
18+
side; the top level places the macros (`place.tcl`) and is placement
19+
and wires only.
20+
21+
`make DESIGN_CONFIG=./designs/asap7/tpu/config.mk FLOW_VARIANT=flat`
22+
builds tpu flat from the same RTL, the reference.
23+
24+
![placement density across the parent and its 16 tiles](floorplan.svg)
25+
26+
Placement density at global route, each tile's own cells drawn in place:
27+
the tiles are 80 % full; the parent's three sides around them hold its
28+
skew and deskew registers sparsely.
29+
30+
## Why
31+
32+
QoR is the best result within a build budget. In a hierarchical flow the
33+
parent places, repairs and routes around its macros' abstracts: it
34+
waits for every macro, and every macro change re-places it. Here the
35+
parent is a macro like the tiles, built against the same pin plan, so
36+
every macro builds side by side and an RTL edit in a tile rebuilds that
37+
tile alone. Measured at global route, with the reg2reg minimum period
38+
stitched across every macro (the study:
39+
https://github.com/The-OpenROAD-Project/bazel-orfs/pull/1396):
40+
41+
| | tiles in a C | flat |
42+
|---|---|---|
43+
| reg2reg minimum period, mean of three placement seeds | 672.6 ps (92 %) | 729.3 ps |
44+
| a fresh build, critical path | 76 s | 436 s |
45+
| an RTL edit in a processing element | 99 s | 487 s |
46+
| die | 1.39 x | 1 x |
47+
48+
![best period against the CPU spent searching](period_vs_cpu.svg)
49+
50+
The carved design's first sensible floorplan beats everything flat
51+
reaches with five times the CPU; tuning either one moves its period by
52+
less than a placement seed does.
53+
54+
![an RTL edit's round trip](round_trip.svg)
55+
56+
## The rules of the game
57+
58+
tpu is mesochronous with its surroundings: the same clock frequency, a
59+
fixed but unknown phase. Its clock reaches each tile through the tiles
60+
before it, so the phase at its registers is not the clock root's; data
61+
crosses its boundary through FIFOs, which are assumed to be outside tpu.
62+
So only reg2reg paths are judged, and the boundary is a `set_max_delay`
63+
budget for repair, with no input or output delay (`constraint.sdc`,
64+
`flow/platforms/asap7/constraints.sdc`). tpu has every pin on its bottom
65+
edge, as a macro abutting its parent would; inside, each macro has its
66+
pins on the sides its neighbours are on. Every macro routes on M2-M8;
67+
M9 brings power from above.
68+
69+
## How it is clocked
70+
71+
The clock, reset and load enable enter each row of tiles from the
72+
parent at its west end and run east through the tiles with the
73+
activations, one buffer per tile; rows enter together. Each tile sees
74+
the clock about 77 ps after its west neighbour, so data flowing east is
75+
captured later than it is launched, and data flowing south sees no
76+
skew. Only neighbours' skew matters, as Fisher and Kung showed for
77+
pipelined clocks in systolic arrays: no clock tree is balanced across
78+
the array. The bottom row passes its clock down to the deskew registers
79+
in the parent.
80+
81+
![how the clock runs through the tiles](clocking.svg)
82+
83+
![clock arrival at each tile's registers, measured](clock_skew.svg)
84+
85+
## The worst path at the top level
86+
87+
The worst slack at the top level is not reg2reg but in2reg: tpu's
88+
`a_valid` and `a_data` into the top-left tile.
89+
90+
```
91+
+----+-------------------------------------------+
92+
| | parent, top side |
93+
| +-------------+-------------+-------- ... |
94+
| | u_b0_0 | u_b0_1 | |
95+
| | PE(0,0) | | | row 0
96+
| +----> a_in | | |
97+
| ^ +-------------+-------------+-------- ... |
98+
| | | the other 14 tiles |
99+
| | +-------------------------------------------+
100+
| | | parent, bottom side |
101+
+--^-+-------------------------------------------+
102+
a_valid, a_data[7:0]: tpu's pins, bottom edge
103+
104+
0.0 ps pin a_valid
105+
100.8 ps u_b0_0/a_in: up the parent's west side, through the mux
106+
a_valid ? a_data : 0; row 0 has no skew register
107+
670.8 ps PE(0,0)'s multiply-add into its partial-sum register
108+
300.0 ps the in2reg budget: slack -370.8 ps
109+
```
110+
111+
The RTL delays row i of the activations i cycles so partial sums meet
112+
down each column: row 0 gets none, and the minimum latency with it. The
113+
zeros on idle cycles are injected where the rows enter, and the PE
114+
multiplies its west input as it arrives. So from tpu's inputs to its
115+
first partial sum is one multiply-add, half of a reg2reg path whose
116+
other half is a register outside tpu: no in2reg budget under one
117+
multiply-add holds it, wherever the pins are.
118+
119+
The mesochronous context makes it a reg2reg path. tpu's interface is
120+
valid in (`a_valid`, `w_valid`) and valid out (`c_valid`), with no
121+
back-pressure, and it takes a bubble anywhere. A FIFO at tpu's level,
122+
on tpu's one clock, its pin side on a short branch of the clock tree
123+
and its logic side on the macros' deep one, turns the path into FIFO
124+
register to partial sum, judged with the period; the pins see only the
125+
short branch.
126+
127+
## Files
128+
129+
| file | what |
130+
|---|---|
131+
| `config.mk` | tpu: the top level of macros (base), or flat (`FLOW_VARIANT=flat`) |
132+
| `tpu_block/config.mk` | the tile, with the settings a placement and clock-tree search found best |
133+
| `tpu_rest/config.mk` | the parent macro |
134+
| `surgery.py` | the carve and the parent's module, during synthesis |
135+
| `tpu_io.tcl` | the pins of the tile and of the parent, and the parent's notch |
136+
| `place.tcl`, `io_top.tcl`, `pdn_top.tcl`, `tapcell_none.tcl`, `dpl_none.tcl` | the top level of macros only |
137+
| `pdn_macro.tcl` | every macro's power: rails, M5 straps, M8 straps the top level's M9 lands on |
138+
| `pins_bottom.tcl` | flat's pins, all on its bottom edge |
139+
| `*.svg` | the figures above, from the study |
140+
141+
## Limits
142+
143+
- **Hold fails across macros.** Timed with every macro's own cells and
144+
wires, 143 endpoints from a tile into the parent fail hold, worst
145+
-417.4 ps, and 75 inside the parent, worst -130.0 ps. The parent's
146+
deskew registers take their clock from the bottom row of tiles through
147+
the parent's own tree, about 250 ps after the tile registers that feed
148+
them, and the parent mixes those clocks with its own; no flow repairs
149+
hold across macros, each sees only its side. The clock taken out of
150+
the macros and balanced at a top level of its own closes hold, at a
151+
13 % longer period so far: the chain's skew was free time for every
152+
path flowing east (the study has the numbers). The periods above are
153+
setup only; with the clock paths derated 5 % either way and common path
154+
pessimism removed, the carved period is 668.3 ps.
155+
156+
- **Area**: 1.39 x flat's die. The tiles are 80 % full; the parent's
157+
three sides around them hold its 676 um2 of registers at 5.6 %.
158+
- **The parent's repair does not see inside the tiles**: it times their
159+
pins as ports with budgets. tpu's worst path starts and ends inside one
160+
tile, so this costs nothing here.
161+
- **Power at the top level reaches the parent only**: pdngen takes the
162+
parent's whole outline as occupied and draws nothing over the tiles.
Lines changed: 86 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)