Lock in the self-weighting property of the random walk design - #4
Conversation
allocator's random walk is usable as a survey design only because it is
self-weighting: points spaced evenly along a walk are a uniform sample of the
road network, so a walk-based survey needs no inclusion-probability correction.
Nothing tested that, and it is the property the whole design rests on.
It is provable, not merely observed. generate_walk picks uniformly among a
node's neighbours, so it is a simple random walk, and there pi(v) = deg(v)/2|E|.
The rate of traversing a directed edge (u,v) is therefore
pi(u) * (1/deg u) = deg(u)/(2|E|) * 1/deg(u) = 1/(2|E|)
independent of the edge, its length, and its endpoints' degrees. Every edge is
crossed equally often, and since crossing costs distance proportional to length,
distance-share equals length-share.
Measured on a deliberately irregular graph (degrees 1-4, lengths 0.5-20, so that
"equally often" and "in proportion to length" are very different statements),
over independent walks of 40 network-lengths each:
simple random walk max |t| over 19 edges = 1.88
length-proportional walk max |t| over 19 edges = 269.09
The second is the falsification partner and the reason the first means anything.
Choosing the next edge with probability proportional to its length is a
natural-looking change -- it sounds more length-aware -- and it destroys the
property, because long edges are then favoured in the count of traversals on top
of costing more distance each. test_a_length_proportional_walk_is_caught runs
that variant through the same gate and requires most edges to fail.
Two design points worth stating:
- Replicates are independent walks, not segments of one long walk. A single walk
is a Markov chain, so its successive traversals are correlated and a binomial
standard error from the step count would understate the variance. Measured on
one long walk the largest deviation was 2.85 binomial sigma, which is the sort
of number that looks fine until the dependence is accounted for.
- The band is Bonferroni-corrected for testing all |E| edges at once, derived
from the edge count rather than chosen until the suite went quiet.
Also asserts the mechanism directly, not just the consequence: traversal counts
per edge against 1/|E|. A future change that preserved self-weighting by
accident would still be visible there.
Verified the gates can fail by stubbing every simcheck assertion to a no-op: the
falsification test goes red and the two positive tests stay green, which is the
expected asymmetry.
138 passed, 13 subtests passed, at both simcheck tiers. ruff clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
The new test module imports simcheck and CI could not find it:
ModuleNotFoundError: No module named 'simcheck'
simcheck is declared in [dependency-groups] test, and this repo's CI installs
published extras -- `uv pip install -e ".[test]"` -- which does not read PEP 735
groups. Moving simcheck into [project.optional-dependencies] is not an option:
it is a git reference, and PyPI rejects direct URL references in the metadata of
an uploaded distribution, so it would break allocator's own release.
So the install step now also runs `uv pip install --group test`. Verified the
distinction rather than assuming it, in a throwaway 3.13 venv:
after `uv pip install -e ".[test]"` -> simcheck absent
after `uv pip install --group test` -> Name: simcheck
Applied to both jobs that run pytest.
Note the lint job is still red on this branch, and was red on master before it:
mypy reports `"prange" has no attribute "__iter__"` in
allocator/distances/haversine.py:51. This branch touches only pyproject.toml and
one new test file, so that failure is not from here and is left alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
allocator's lint job has been failing on master since numba 0.66.0 shipped:
allocator/distances/haversine.py:51: error: "prange" has no attribute
"__iter__" (not iterable) [attr-defined]
0.66.0 changed prange's annotation so mypy no longer sees it as iterable. It
went unnoticed locally because uv.lock pins 0.65.0, where mypy is happy, while
CI installs with `uv pip install -e ".[dev]"` and resolves to the latest. Two
environments, two answers, and the one nobody runs is the one that gates.
Reproduced both ways before touching anything:
numba 0.66.0 -> 1 error
numba 0.65.0 -> Success
A `# type: ignore[attr-defined]` cannot fix this. warn_unused_ignores is on, so
the ignore would itself be an error under 0.65 -- it just moves the failure to
the other environment. Instead the module tells the type checker what prange
actually is, which is range with a licence for the compiler to parallelise the
loop. Runtime behaviour is untouched: the else branch still binds numba.prange.
After, in both environments:
numba 0.66.0 -> Success: no issues found in 33 source files
numba 0.65.0 -> Success: no issues found in 33 source files
Verified the jitted loop still computes what it did: 21 distance tests pass, and
against an independently written great-circle formula the largest disagreement
over a 50x40 matrix is 0.000000 m, with self-distances zero.
Unrelated to the self-weighting tests on this branch. It is here because it was
the last red check and it had been red for two months. Happy to split it out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Two follow-on commits, both CI fixes rather than test work, flagged here so they're easy to split out if you'd rather.
A Runtime is untouched: 21 distance tests pass, and against an independently written great-circle formula the largest disagreement over a 50×40 matrix is 0.000000 m, with self-distances zero. |
deptry has been reporting this, behind the mypy failure that stopped the lint
job before it got there:
DEP002 'haversine' defined as a dependency but not used in the codebase
Nothing under allocator/ imports the haversine package. The library computes
haversine distances with its own numba implementation in
allocator/distances/haversine.py, and the only import of the PyPI package in the
whole repo is benchmarks/benchmark_jit.py, which uses it to build the "pure
Python implementation for comparison" that the JIT version is timed against.
deptry excludes benchmarks/ by design, so it saw a runtime dependency nobody
used.
Every user was installing it for a benchmark they do not run. Moved to the dev
extra, which [tool.deptry] optional_dependencies_dev_groups already classifies
as dev.
Checked that the package genuinely does not need it, rather than trusting deptry
to have noticed: installed allocator into a fresh 3.13 environment with no
haversine present, and the distance matrix, the public API and the distance
factory all import and compute.
haversine package present: False
distance matrix works: 157249.381 m
public API imports OK
distance factory imports OK
Why the package is not used, for the record: allocator's numba version is 90x to
145x faster on matrices, which is the whole point of the benchmark. The two
disagree by up to 27.5 m at antipodal range, and that is entirely the Earth
radius constant -- 6,371,000 m here against the package's IUGG mean of
6,371,008.8 m, a ratio of 1.000001381. Rescaling by that ratio leaves a residual
of 0.000000000 m, so the formulas themselves agree to machine precision.
138 passed, 13 subtests passed. deptry, ruff and mypy clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three separate failures surfaced while getting this branch green, and they had
one cause. uv.lock was in .gitignore, so every developer had a lock and CI had
none: locally numba resolved to the pinned 0.65.0, and CI resolved 0.66.0 fresh
on every run. 0.66.0 changed prange's annotation, mypy stopped seeing it as
iterable, and the lint job went red on master and stayed red for two months
because nobody could reproduce it. Behind that failure sat a deptry finding, and
behind that an uninstalled dependency group -- each invisible until the step in
front of it was fixed.
Four changes, which are one change:
- uv.lock is no longer ignored, and is committed. It constrains nobody who
installs allocator: pip resolves from the ranges in pyproject.toml and never
reads the lock. It only fixes what CI and a developer's machine install. Every
other repo in the fleet already commits it -- calibre, fewlab, geoinference,
preen, simcheck, incline -- so this repo was the outlier.
- CI installs with `uv sync --locked` instead of `uv pip install -e ".[...]"`.
Beyond reproducibility, --locked fails when the lock has drifted from
pyproject.toml, which is a mistake made twice on this branch already.
- types-requests and pandas-stubs move into the dev group. They were installed
ad hoc in the lint job, so the versions mypy checked against were whatever
resolved that morning and were recorded nowhere. ortools and scipy were also
installed ad hoc and are simply already runtime dependencies.
- A weekly resolve-latest job. Pinning CI to the lock closes the gap in one
direction and opens it in the other: CI stops noticing when an upstream release
breaks us. This job ignores the lock, resolves the latest of everything, and
runs lint and tests. Upstream breakage then arrives as a run named
resolve-latest on a Monday, rather than ambushing the next unrelated PR.
Each job keeps the extras it had. The obvious tidy-up -- --all-extras everywhere
-- is wrong here: tests/api/test_route_api.py::test_christofides_import_error_handling
asserts that tsp_christofides *raises* ImportError, so installing the algorithms
extra in the plain test job fails it. Tried it, watched it fail, reverted it.
That test's meaning depends on what is installed, which is worth revisiting
separately.
Verified against the committed lock, on every Python in the matrix:
py3.11: 138 passed, 13 subtests
py3.12: 138 passed, 13 subtests
py3.13: 138 passed, 13 subtests
lint on 3.11: mypy clean, deptry clean, ruff clean, vulture clean
And verified the drift job does its job, by running it: it resolves numba 0.66.0
-- the version that caused all this -- and mypy, deptry and all 138 tests pass
under it. It would have reported the regression the Monday after 0.66.0 shipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The docs build went red on this branch the moment the previous commit touched
pyproject.toml:
ImportError: cannot import name 'make_fence_rule' from
'markdown_it.rules_block'
Nothing about that change concerns docs. docs.yml installed with
`uv pip install -e .[docs]`, cached on pyproject.toml, so editing pyproject
invalidated the cache, uv resolved fresh, and picked up a markdown-it-py without
make_fence_rule -- which myst-parser needs. The workflow had been passing on a
cached resolution from whenever it last ran.
That is the same failure as numba 0.66.0, in a different workflow: an unlocked
install that agrees with nobody and changes under you. The lock pins the pair
that works, markdown-it-py 3.0.0 with myst-parser 4.0.1, and the docs build
succeeds under it.
python-publish.yml had it too, in the worst place -- the test gate that runs
before a release resolved fresh, so a release could pass or fail on what PyPI
published that morning.
So all three workflows now install with `uv sync --locked` and cache on uv.lock
rather than pyproject.toml. No `uv pip install` remains in .github/workflows.
Verified each gate locally against the committed lock:
ci lint ruff, mypy, deptry, vulture clean
ci test 138 passed, 13 subtests, on 3.11, 3.12 and 3.13
ci test-algorithms 3 passed
docs build succeeded, build/html/index.html present
publish gate 138 passed, 13 subtests
Also bumped setup-uv from v3 to v7 in docs.yml, matching the other workflows.
Not done here, and worth its own change: actions/checkout is on v4 against v7.0.1
current, codecov-action v5 against v7, and the three pages actions are each one
or two majors behind. Those are version jumps that deserve their own pull request
rather than riding along with a testing branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Why this property
allocator's random walk is usable as a survey design only because it is self-weighting: points spaced evenly along a walk are a uniform sample of the road network, so a walk-based survey needs no inclusion-probability correction. Nothing tested that, and it is the property the whole design rests on.
It's provable, not merely observed.
generate_walkpicks uniformly among a node's neighbours — a simple random walk — whereπ(v) = deg(v)/2|E|. The rate of traversing a directed edge(u,v)is thereforeindependent of the edge, its length, and its endpoints' degrees. Every edge is crossed equally often, and since crossing costs distance proportional to length, distance-share equals length-share.
Measured
On a deliberately irregular graph — degrees 1–4, lengths 0.5–20, so that "equally often" and "in proportion to length" are very different statements — over independent walks of 40 network-lengths each:
The second is the falsification partner and the reason the first means anything. Choosing the next edge with probability proportional to its length is a natural-looking change — it sounds more length-aware — and it destroys the property, because long edges are then favoured in the count of traversals on top of costing more distance each.
Two design points
|E|edges at once, derived from the edge count rather than picked until the suite went quiet.The mechanism is asserted directly too, not just the consequence: traversal counts per edge against
1/|E|. A future change that preserved self-weighting by accident would still be visible there.Verified the gates can fail by stubbing every simcheck assertion to a no-op — the falsification test goes red, the two positive tests stay green.
138 passed, 13 subtests, at both simcheck tiers. ruff clean.
🤖 Generated with Claude Code