Skip to content

Lock in the self-weighting property of the random walk design - #4

Merged
soodoku merged 6 commits into
masterfrom
test/random-walk-self-weighting
Aug 8, 2026
Merged

soodoku merged 6 commits into
masterfrom
test/random-walk-self-weighting

Conversation

@soodoku

@soodoku soodoku commented Aug 8, 2026

Copy link
Copy Markdown
Member

Why this property

allocator's random walk is usable as a survey design only because it is self-weighting: points spaced evenly along a walk are a uniform sample of the road network, so a walk-based survey needs no inclusion-probability correction. Nothing tested that, and it is the property the whole design rests on.

It's provable, not merely observed. generate_walk picks uniformly among a node's neighbours — a simple random walk — where π(v) = deg(v)/2|E|. The rate of traversing a directed edge (u,v) is therefore

π(u) · (1/deg u) = deg(u)/(2|E|) · 1/deg(u) = 1/(2|E|)

independent of the edge, its length, and its endpoints' degrees. Every edge is crossed equally often, and since crossing costs distance proportional to length, distance-share equals length-share.

Measured

On a deliberately irregular graph — degrees 1–4, lengths 0.5–20, so that "equally often" and "in proportion to length" are very different statements — over independent walks of 40 network-lengths each:

walk max |t| over 19 edges
simple random walk (the real one) 1.88
length-proportional (the plausible bug) 269.09

The second is the falsification partner and the reason the first means anything. Choosing the next edge with probability proportional to its length is a natural-looking change — it sounds more length-aware — and it destroys the property, because long edges are then favoured in the count of traversals on top of costing more distance each.

Two design points

  • Replicates are independent walks, not segments of one long walk. A single walk is a Markov chain, so successive traversals are correlated and a binomial SE from the step count would understate the variance. On one long walk the largest deviation measured 2.85 binomial σ — the sort of number that looks fine until the dependence is accounted for.
  • The band is Bonferroni-corrected for testing all |E| edges at once, derived from the edge count rather than picked until the suite went quiet.

The mechanism is asserted directly too, not just the consequence: traversal counts per edge against 1/|E|. A future change that preserved self-weighting by accident would still be visible there.

Verified the gates can fail by stubbing every simcheck assertion to a no-op — the falsification test goes red, the two positive tests stay green.

138 passed, 13 subtests, at both simcheck tiers. ruff clean.

🤖 Generated with Claude Code

allocator's random walk is usable as a survey design only because it is
self-weighting: points spaced evenly along a walk are a uniform sample of the
road network, so a walk-based survey needs no inclusion-probability correction.
Nothing tested that, and it is the property the whole design rests on.

It is provable, not merely observed. generate_walk picks uniformly among a
node's neighbours, so it is a simple random walk, and there pi(v) = deg(v)/2|E|.
The rate of traversing a directed edge (u,v) is therefore

    pi(u) * (1/deg u) = deg(u)/(2|E|) * 1/deg(u) = 1/(2|E|)

independent of the edge, its length, and its endpoints' degrees. Every edge is
crossed equally often, and since crossing costs distance proportional to length,
distance-share equals length-share.

Measured on a deliberately irregular graph (degrees 1-4, lengths 0.5-20, so that
"equally often" and "in proportion to length" are very different statements),
over independent walks of 40 network-lengths each:

    simple random walk          max |t| over 19 edges =   1.88
    length-proportional walk    max |t| over 19 edges = 269.09

The second is the falsification partner and the reason the first means anything.
Choosing the next edge with probability proportional to its length is a
natural-looking change -- it sounds more length-aware -- and it destroys the
property, because long edges are then favoured in the count of traversals on top
of costing more distance each. test_a_length_proportional_walk_is_caught runs
that variant through the same gate and requires most edges to fail.

Two design points worth stating:

- Replicates are independent walks, not segments of one long walk. A single walk
  is a Markov chain, so its successive traversals are correlated and a binomial
  standard error from the step count would understate the variance. Measured on
  one long walk the largest deviation was 2.85 binomial sigma, which is the sort
  of number that looks fine until the dependence is accounted for.
- The band is Bonferroni-corrected for testing all |E| edges at once, derived
  from the edge count rather than chosen until the suite went quiet.

Also asserts the mechanism directly, not just the consequence: traversal counts
per edge against 1/|E|. A future change that preserved self-weighting by
accident would still be visible there.

Verified the gates can fail by stubbing every simcheck assertion to a no-op: the
falsification test goes red and the two positive tests stay green, which is the
expected asymmetry.

138 passed, 13 subtests passed, at both simcheck tiers. ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

soodoku and others added 2 commits August 8, 2026 06:02
The new test module imports simcheck and CI could not find it:

    ModuleNotFoundError: No module named 'simcheck'

simcheck is declared in [dependency-groups] test, and this repo's CI installs
published extras -- `uv pip install -e ".[test]"` -- which does not read PEP 735
groups. Moving simcheck into [project.optional-dependencies] is not an option:
it is a git reference, and PyPI rejects direct URL references in the metadata of
an uploaded distribution, so it would break allocator's own release.

So the install step now also runs `uv pip install --group test`. Verified the
distinction rather than assuming it, in a throwaway 3.13 venv:

    after `uv pip install -e ".[test]"`   -> simcheck absent
    after `uv pip install --group test`   -> Name: simcheck

Applied to both jobs that run pytest.

Note the lint job is still red on this branch, and was red on master before it:
mypy reports `"prange" has no attribute "__iter__"` in
allocator/distances/haversine.py:51. This branch touches only pyproject.toml and
one new test file, so that failure is not from here and is left alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
allocator's lint job has been failing on master since numba 0.66.0 shipped:

    allocator/distances/haversine.py:51: error: "prange" has no attribute
    "__iter__" (not iterable)  [attr-defined]

0.66.0 changed prange's annotation so mypy no longer sees it as iterable. It
went unnoticed locally because uv.lock pins 0.65.0, where mypy is happy, while
CI installs with `uv pip install -e ".[dev]"` and resolves to the latest. Two
environments, two answers, and the one nobody runs is the one that gates.
Reproduced both ways before touching anything:

    numba 0.66.0  ->  1 error
    numba 0.65.0  ->  Success

A `# type: ignore[attr-defined]` cannot fix this. warn_unused_ignores is on, so
the ignore would itself be an error under 0.65 -- it just moves the failure to
the other environment. Instead the module tells the type checker what prange
actually is, which is range with a licence for the compiler to parallelise the
loop. Runtime behaviour is untouched: the else branch still binds numba.prange.

After, in both environments:

    numba 0.66.0  ->  Success: no issues found in 33 source files
    numba 0.65.0  ->  Success: no issues found in 33 source files

Verified the jitted loop still computes what it did: 21 distance tests pass, and
against an independently written great-circle formula the largest disagreement
over a 50x40 matrix is 0.000000 m, with self-distances zero.

Unrelated to the self-weighting tests on this branch. It is here because it was
the last red check and it had been red for two months. Happy to split it out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@soodoku

soodoku commented Aug 8, 2026

Copy link
Copy Markdown
Member Author

Two follow-on commits, both CI fixes rather than test work, flagged here so they're easy to split out if you'd rather.

14090bf — install the test dependency-group in CI. The new module imports simcheck and CI couldn't find it. simcheck is declared in [dependency-groups], and this repo installs published extras (uv pip install -e ".[test]"), which doesn't read PEP 735 groups. It can't move into [project.optional-dependencies] because it's a git reference and PyPI rejects direct URL references in distribution metadata — that would break allocator's own release. Verified the distinction in a throwaway venv rather than assuming it:

after `uv pip install -e ".[test]"`   -> simcheck absent
after `uv pip install --group test`   -> Name: simcheck

bc9bffa — keep haversine checkable across numba releases. The lint job has been red on master since numba 0.66.0 shipped (master run 27514699462, June 14, shows lint: failure with all tests passing — the same shape as this PR before the fix). 0.66.0 changed prange's annotation so mypy stopped seeing it as iterable. It went unnoticed locally because uv.lock pins 0.65.0 while CI resolves latest:

numba 0.66.0  ->  1 error
numba 0.65.0  ->  Success

A # type: ignore can't fix it — warn_unused_ignores is on, so the ignore becomes an error under 0.65 and just moves the failure to the other environment. The module now tells the type checker that prange is range, which is what it is, and binds numba.prange at runtime. Both environments pass after.

Runtime is untouched: 21 distance tests pass, and against an independently written great-circle formula the largest disagreement over a 50×40 matrix is 0.000000 m, with self-distances zero.

soodoku and others added 3 commits August 8, 2026 09:13
deptry has been reporting this, behind the mypy failure that stopped the lint
job before it got there:

    DEP002 'haversine' defined as a dependency but not used in the codebase

Nothing under allocator/ imports the haversine package. The library computes
haversine distances with its own numba implementation in
allocator/distances/haversine.py, and the only import of the PyPI package in the
whole repo is benchmarks/benchmark_jit.py, which uses it to build the "pure
Python implementation for comparison" that the JIT version is timed against.
deptry excludes benchmarks/ by design, so it saw a runtime dependency nobody
used.

Every user was installing it for a benchmark they do not run. Moved to the dev
extra, which [tool.deptry] optional_dependencies_dev_groups already classifies
as dev.

Checked that the package genuinely does not need it, rather than trusting deptry
to have noticed: installed allocator into a fresh 3.13 environment with no
haversine present, and the distance matrix, the public API and the distance
factory all import and compute.

    haversine package present: False
    distance matrix works: 157249.381 m
    public API imports OK
    distance factory imports OK

Why the package is not used, for the record: allocator's numba version is 90x to
145x faster on matrices, which is the whole point of the benchmark. The two
disagree by up to 27.5 m at antipodal range, and that is entirely the Earth
radius constant -- 6,371,000 m here against the package's IUGG mean of
6,371,008.8 m, a ratio of 1.000001381. Rescaling by that ratio leaves a residual
of 0.000000000 m, so the formulas themselves agree to machine precision.

138 passed, 13 subtests passed. deptry, ruff and mypy clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three separate failures surfaced while getting this branch green, and they had
one cause. uv.lock was in .gitignore, so every developer had a lock and CI had
none: locally numba resolved to the pinned 0.65.0, and CI resolved 0.66.0 fresh
on every run. 0.66.0 changed prange's annotation, mypy stopped seeing it as
iterable, and the lint job went red on master and stayed red for two months
because nobody could reproduce it. Behind that failure sat a deptry finding, and
behind that an uninstalled dependency group -- each invisible until the step in
front of it was fixed.

Four changes, which are one change:

- uv.lock is no longer ignored, and is committed. It constrains nobody who
  installs allocator: pip resolves from the ranges in pyproject.toml and never
  reads the lock. It only fixes what CI and a developer's machine install. Every
  other repo in the fleet already commits it -- calibre, fewlab, geoinference,
  preen, simcheck, incline -- so this repo was the outlier.

- CI installs with `uv sync --locked` instead of `uv pip install -e ".[...]"`.
  Beyond reproducibility, --locked fails when the lock has drifted from
  pyproject.toml, which is a mistake made twice on this branch already.

- types-requests and pandas-stubs move into the dev group. They were installed
  ad hoc in the lint job, so the versions mypy checked against were whatever
  resolved that morning and were recorded nowhere. ortools and scipy were also
  installed ad hoc and are simply already runtime dependencies.

- A weekly resolve-latest job. Pinning CI to the lock closes the gap in one
  direction and opens it in the other: CI stops noticing when an upstream release
  breaks us. This job ignores the lock, resolves the latest of everything, and
  runs lint and tests. Upstream breakage then arrives as a run named
  resolve-latest on a Monday, rather than ambushing the next unrelated PR.

Each job keeps the extras it had. The obvious tidy-up -- --all-extras everywhere
-- is wrong here: tests/api/test_route_api.py::test_christofides_import_error_handling
asserts that tsp_christofides *raises* ImportError, so installing the algorithms
extra in the plain test job fails it. Tried it, watched it fail, reverted it.
That test's meaning depends on what is installed, which is worth revisiting
separately.

Verified against the committed lock, on every Python in the matrix:

    py3.11: 138 passed, 13 subtests
    py3.12: 138 passed, 13 subtests
    py3.13: 138 passed, 13 subtests
    lint on 3.11: mypy clean, deptry clean, ruff clean, vulture clean

And verified the drift job does its job, by running it: it resolves numba 0.66.0
-- the version that caused all this -- and mypy, deptry and all 138 tests pass
under it. It would have reported the regression the Monday after 0.66.0 shipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The docs build went red on this branch the moment the previous commit touched
pyproject.toml:

    ImportError: cannot import name 'make_fence_rule' from
    'markdown_it.rules_block'

Nothing about that change concerns docs. docs.yml installed with
`uv pip install -e .[docs]`, cached on pyproject.toml, so editing pyproject
invalidated the cache, uv resolved fresh, and picked up a markdown-it-py without
make_fence_rule -- which myst-parser needs. The workflow had been passing on a
cached resolution from whenever it last ran.

That is the same failure as numba 0.66.0, in a different workflow: an unlocked
install that agrees with nobody and changes under you. The lock pins the pair
that works, markdown-it-py 3.0.0 with myst-parser 4.0.1, and the docs build
succeeds under it.

python-publish.yml had it too, in the worst place -- the test gate that runs
before a release resolved fresh, so a release could pass or fail on what PyPI
published that morning.

So all three workflows now install with `uv sync --locked` and cache on uv.lock
rather than pyproject.toml. No `uv pip install` remains in .github/workflows.

Verified each gate locally against the committed lock:

    ci lint            ruff, mypy, deptry, vulture clean
    ci test            138 passed, 13 subtests, on 3.11, 3.12 and 3.13
    ci test-algorithms 3 passed
    docs               build succeeded, build/html/index.html present
    publish gate       138 passed, 13 subtests

Also bumped setup-uv from v3 to v7 in docs.yml, matching the other workflows.

Not done here, and worth its own change: actions/checkout is on v4 against v7.0.1
current, codecov-action v5 against v7, and the three pages actions are each one
or two majors behind. Those are version jumps that deserve their own pull request
rather than riding along with a testing branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@soodoku
soodoku merged commit f5f7e8a into master Aug 8, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant