How a label's position on the map is computed from where it was placed on a panorama, where that computation runs, what it consumes, and the one invariant any change to the labeling viewport has to keep. The research behind every constant lives in the label-latlng-estimation repo; the production sign-off is its 2026-09-02 report (#5084).
A label is a pixel on an equirectangular panorama, label_point.pano_x/pano_y on a pano of pano_data.width × height
whose center column faces pano_data.camera_heading. Two angles follow directly:
bearing = camera_heading − 180 + pano_x / width × 360 (degrees from true north)
depression = 180 × pano_y / height − 90 (degrees below the horizon)
The distance from the camera is a flat-ground cotangent on one calibrated camera height, blended at 11.25° into a linear tail with matched value and slope, held flat above the horizon and capped:
dep ≥ 11.25°: d = h / tan(dep)
dep < 11.25°: d = min(h / tan(a) + h·(π/180) / sin²(a) · (a − max(dep, 0)), 50 m) a = 11.25°
h = 2.341219672825709 m → largest possible answer 23.848 m (at and above the horizon)
The position is the spherical destination point from the pano's lat/lng along bearing for d on a sphere of
radius 6371 km. Constants are defined once, in PanoDataService.LatLngEstimation (with each value's derivation in
its scaladoc), and reach the browser through mainParam.latLngEstimation in the Explore view, so the client holds no
copy.
Label type is not an input. Panorama resolution is not an input either: both angles are ratios of the pixel to the pano's size, which is what makes the estimate resolution-independent (#4765) and, once the click has been projected through its own frame, viewport-independent (below).
| path | code | when | stamps |
|---|---|---|---|
| Client | Label#toLatLng (public/js/explore/src/label/Label.js), destination via the vendored turf |
every crowd label, at placement, before submit | approximation3 |
| Server | PanoDataService.toLatLng (app/service/PanoDataService.scala) |
AI label submissions (ExploreService) |
approximation3 |
| SQL | evolution 352 (a statement-for-statement port), 366 for the rows 179 had skipped | one-off backfills of stored rows | approximation3 |
Every stored label position is on this estimator except computation_method = 'depth' rows, whose positions were
measured from GSV depth data at label time (2017–2020) and are better than any estimate, and a small, city-dependent
number of approximation2 rows (Teaneck 2, Seattle 499) whose panorama has no usable metadata. There is no "frozen
regression" path for historical labels: evolution 352 recomputed them.
The enum's values, in order of appearance, so the vocabulary in the research reports and the database line up:
computation_method |
introduced | what it is | median error, 2021 held-out split |
|---|---|---|---|
depth |
2017 | position read from Google's per-panorama depth map at label time; the API was withdrawn in Nov 2020 (#2374) | the truth that split is scored against |
approximation1 |
evolution 93, 2020-11-13 | a stopgap: 10 m along the viewport heading, flat-earth offsets, no pitch input | 4.84 m |
approximation2 |
evolution 98, 2021-01-12 | the linear regression of distance on sv_image_y and canvas_y and of heading on canvas_x, per zoom in the client; evolution 98's one-time backfill applied the zoom-2 triple to every row |
1.46 m |
approximation3 |
evolutions 349 and 352, 2026-08 | the geometric estimator above | 1.38 m (0.40 m on modern truth) |
approximation1 rows no longer exist anywhere: evolution 98 rewrote every one of them two months after they were
created, which is why the 2021 analysis's "estimate 1" and the 2026 sign-off's approximation1 row are the same
formula scored on the same split.
The Scala and client implementations are pinned to one fixture, test/fixtures/latLngEstimationParity.json (59 cases
covering the seam, the bearing's 0/360 wrap, the blend angle from both sides, the bounded tail above the horizon, the
nadir, both hemispheres, and random clicks over four resolutions), generated by the research repo's
python/run_signoff.py fixture. The SQL port has no automated check: it was run against the fixture once, by hand.
test/service/LatLngEstimationParitySpec.scala— Scala, 1e-9°; also asserts the fixture's constants equalLatLngEstimation's, so a refit fails here until the fixture is regenerated in the same change. Blocking in CI (the required "Backend tests" check).test/js/latLngEstimationParity.test.js— the client formula with the real vendored turf, 1e-8°. Blocking in CI too, as a step of the required "Frontend (build)" check (#5132).- The SQL formula was run on the fixture by hand (352.sql's CTEs, read-only, on the dev database): 1.4e-14° max disagreement over the 59 cases. Nothing re-runs it.
Destinations are computed on a sphere of mean radius 6371 km, not on the WGS84 ellipsoid. Measured over every
deployed city's latitude and every bearing on a 5° sweep, the sphere sits at most 10.7 cm from the geodesic at the
estimator's 23.85 m largest answer, 5.3 cm at the 11.8 m blend distance and 2.2 cm at 5 m, all well under the
estimator's own 0.4 m median error. Two spheres are actually in play, and the difference is negligible: Scala and SQL
use exactly 6371.0 km (CommonUtils.EARTH_RADIUS_KM); the client's turf uses 6371.0088 km, 1.4 ppm away, which moves
a label 0.03 mm at the largest answer. The parity spec pins the radius so a well-meaning "upgrade" to geodesics on one
path cannot desynchronize it from the others.
Median absolute distance error, the 2021 per-zoom regression → approximation3:
| truth frame | n | regression | approximation3 |
|---|---|---|---|
| fresh GSV depth at the stored click, post-2021 human labels, representative stratum | 1,484 | 1.08 m | 0.40 m (honest held-out: 0.445 m) |
| the regression's own 2017–2020 held-out split | 79,029 | 1.46 m | 1.38 m as shipped; 0.98 m at equal calibration budget in that frame |
On modern truth it wins every zoom, label type, panorama resolution, capture year and city, and a camera height
calibrated on every other city beats the regression in each held-out city. Rig tilt (the panorama's pitch and
roll, which povToPanoCoord ignores, #4784) explains under 5% of the estimator's error variance on modern truth
and is indistinguishable there from road slope, so it is not an input; the sign-off report's §4.4 has the numbers.
On the era frame the shipped constant reads about 17% too near on the subpopulations whose depth truth carries the era
payloads' pinned 2.50 m ground plane (DC, 6656-px panoramas, Newberg); that is the truth's scale, not the click
geometry, and the modern frame is the measured one.
The position above also fixes which side of its street a label is on. Evolution 377 stores it on label_point as
centerline_offset_m, the signed geodesic distance from the estimated position to the centerline of the label's
street_edge_id (the open street nearest the position at submission): positive on the left of the edge's
digitized direction (ST_StartPoint → ST_EndPoint), negative on the right, NULL without a position. The
street_side enum (left/right, mirrored by models.label.StreetSide) is a GENERATED column over it with a 1 m
floor, so it can't drift from the offset. Both are on /v3/api/rawLabels.
The sign is relative to the edge rather than cardinal because that is what "same sidewalk as label X" needs (compare
street_side on a shared street_edge_id), and cardinal sides are undefined on diagonal streets. It comes from a
cross product against the edge's local tangent in Web Mercator (conformal, so no per-city SRID); the magnitude is
geodesic.
The magnitude is measured across the street, not along it. A label whose perpendicular foot falls past the end of its edge -- about 5% of them, since an edge ends at every intersection -- is nearest to that endpoint, so a naive distance-to-the-edge would count the along-street run too, and report a label 30 m off the end and 0.3 m to the side as 30 m of side. Those points keep only their cross-track component; an interior foot is perpendicular by construction and is unaffected to the bit. On Teaneck's 23,911 positioned labels the refinement moves 1,178 of them, changes no label's side, drops 18 inside the 1 m floor where they belong, and takes the largest offset from 126.5 m to 25.1 m.
The one SQL function, label_centerline_offset_m, serves the backfill and the insert path
(LabelPointTable.computeCenterlineOffset, run right after the point insert in ExploreService.insertLabel), so the
stored value always equals a fresh recompute, and StreetSideSpec checks that it does.
Why this design and not the camera heading the issue proposed: scored against Seattle's SDOT sidewalk inventory, the
geometric side is right 93–96% where the two methods disagree, the heading method fails whenever the camera is off
the audited street (18.5% accurate beyond 15 m, and one label in five is shot from more than 5 m off), and no hybrid
beats it. The offset is stored rather than just the enum because accuracy is a monotone function of it alone
(63–70% under 0.5 m, 97–98% at 1.5–2 m, 99%+ from 3 m, the same for every label type), so a consumer can pick its own
floor. Full report: experiments/2026-09-03-street-side-assignment.md.
Recompute contract. Anything that moves label_point.geom, changes label.street_edge_id, or edits
street_edge.geom (a 352/366-style backfill, an estimator refit, an AI reattach, a street re-import) recomputes
centerline_offset_m in the same statement. The reposition in 352 flipped 2.2% of sides; forgetting the recompute
would have left every one of them silently wrong. Re-importing streets is the sharpest edge: reversing an edge's
digitized direction flips the sign of every label on it at once.
The estimator sees only the two angles above, and the angles come from the click through the projection:
util.pano.canvasCoordToCenteredPov(pov, canvasX, canvasY, canvasWidth, canvasHeight) turns a click on the
labeling canvas into the POV that centers it, and util.pano.povToPanoCoord turns that POV into pano_x/pano_y.
The projection models the viewport as a rectilinear camera with focal length (canvasWidth / 2) / tan(hFov / 2),
where the horizontal field of view is a function of zoom alone (#5083).
The invariant: a click must be projected through the frame it was made in. Today every caller passes the
720×480 constants (util.EXPLORE_CANVAS_WIDTH/HEIGHT in JS, LabelPointTable.canvasWidth/Height in Scala's
calculatePovIfCentered), and every stored label was in fact placed on a 720×480 frame, so the constant is correct
for the corpus. The moment the labeling viewport can be another size, the frame has to travel with the label
(label_point.canvas_width/canvas_height, the #5085 plan) and every consumer of canvas_x/canvas_y has to read it.
Measured over 387 label directions on frames from 4:3 to 21:9:
| how the click is interpreted | position error |
|---|---|
| through its own frame | 0 m on every frame |
| scaled axis-by-axis into 720×480 | 0.5–2.1 m at p90 (the vertical field is stretched by the aspect change) |
| scaled by width, read against a 480-px height | 4.9–12.9 m at p90 (a 37 px vertical offset is 5.9° of pitch at zoom 1) |
Uniform scaling is free: a 1280×720 and a 1920×1080 frame give identical results because focal length and both click
offsets scale together, which is why the boxed tool's --ui-scale zoom has never needed a correction. Aspect is
not. Portrait shapes and beyond-21:9 at zoom 3, where GSV clamps the vertical field, change the effective
horizontal FOV rather than this math; #5083's clamp model applies before the projection.
- Refit in the research repo; the new values land in its
data/modern-truth-summary.jsonfinal_coefficients. - Update
PanoDataService.LatLngEstimation(and its scaladoc provenance). - Regenerate
test/fixtures/latLngEstimationParity.jsonwithpython python/run_signoff.py fixture <path>there, and re-pinPanoDataServiceSpecandtest/js/exploreLabelLatLngEstimate.test.js. - Decide, separately, whether stored rows are recomputed: that is an evolution in the shape of 352 (backup table,
type rebuild, SQL port of the new formula), and it is a product decision, not a side effect of a refit. Run the
new SQL port against the fixture before it ships (the research repo's
python/signoff.pyhas the harness): CI has no check for the SQL path.