Commit c9b6bcd
authored
feat: plan conservative incremental hashing queries (#11)
## Why
[PR #10](#10) adds a
seedable hash file containing the previous revision's target hashes and
direct dependency graph. Given that seed and a Git diff, the next
problem is deciding which targets must be recomputed.
This PR adds a conservative planner that turns changed file paths into a
reduced Bazel query. “Dirty” here means **must be recomputed**, not
necessarily **known to have a different final hash**. The planner
narrows the query only when the seed contains enough information to
prove the narrower query is safe; otherwise it requests a full
computation.
This remains library functionality in this PR. `hash-persister` starts
using it in [PR
#12](#12).
## From changed files to affected targets
Suppose `lib/value.go` belongs to Bazel package `//lib` and
`//app:binary` depends on `//lib:library`:
1. A change to `lib/value.go` marks `//lib` dirty.
2. Every target previously known in `//lib` is marked directly dirty.
3. The persisted graph is traversed in reverse, marking `//app:binary`
and any further dependents dirty as well.
A source file is assigned to the nearest enclosing package known to the
seed. This follows Bazel package boundaries: a target can only refer to
files in its own package or through labels owned by another package. A
changed file with no enclosing known package cannot be an input of a
seeded target and can be ignored unless it is one of the repository-wide
fallback triggers described below.
The result retains dirty package names as well as target labels. If a
BUILD file changes, targets may have been added or removed since the
seed was written. Querying the whole dirty package discovers additions
and avoids referring to deleted labels; querying only labels remembered
by the seed would do neither.
## When the planner refuses to narrow the query
Some files can change Bazel loading, repository resolution, package
ownership, or hashing behavior without appearing as ordinary target
dependency edges. The planner requests full hashing for these cases,
including:
- Starlark files and workspace, module, or repository metadata;
- Bazel configuration, version, ignore, lock, and repository-state
files;
- files used by rule-class fingerprints;
- deleted or renamed BUILD files;
- a BUILD file that creates a package absent from the seed.
These cases receive a stable, bounded fallback code for automation and
metrics, plus a human-readable detail naming the triggering path.
Keeping paths out of the code prevents unbounded metric dimensions.
## Preserving the caller's target expression
The reduced query contains two kinds of entries:
- dirty packages use package wildcards such as `//lib:all`, so added and
removed targets are handled;
- affected targets in unchanged packages use explicit labels such as
`//app:binary`, because their BUILD files have not changed.
Those entries form a smaller replacement for the repository-wide `//...`
universe in the caller's original Bazel query expression. For example,
filters applied to `//...` remain applied to the reduced universe rather
than being discarded. Only local `//...` occurrences are replaced;
external-repository expressions such as `@repo//...` retain their
original meaning. An expression without a local `//...` cannot be
transformed safely by this mechanism, so orchestration will fall back to
the full query.
## Review guidance
The central correctness property is that every changed path must do one
of two things: reach every target whose hash might be affected, or
request a full computation. Package ownership, BUILD-file boundary
changes, reverse-dependency propagation, and preservation of the
original Bazel query semantics are therefore the most important review
areas.
## Stack
1. [Seedable persistence and compatible cache
seeding](#10)
2. **This PR:** dirty-set planning and scoped queries
3. [`hash-persister` orchestration, verification, fallback handling, and
execution
reporting](#12)5 files changed
Lines changed: 824 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| 14 | + | |
13 | 15 | | |
14 | 16 | | |
15 | 17 | | |
| |||
36 | 38 | | |
37 | 39 | | |
38 | 40 | | |
| 41 | + | |
39 | 42 | | |
40 | 43 | | |
41 | 44 | | |
| 45 | + | |
42 | 46 | | |
43 | 47 | | |
44 | 48 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
0 commit comments