Skip to content

Commit 0abec00

Browse files
committed
docs(comparison): trim Europe section to Cairn-only
Drop the operational/cluster-specific text from the Europe block — the page is Cairn engine docs, not deployment notes for one particular host. Removed: - airflow / Python-pool pause notes (host-specific cluster ops, not Cairn behavior) - vm.max_map_count knob discussion (operational tuning, not Cairn algorithm) - internal Phase 6e / rkyv size_64 lede paragraph (release-note language, not user-facing engine description) - Pelias / Photon / Nominatim projection prose (speculation rather than measurement) Kept: host capability summary (48 cores, 124 GB RAM, NVMe), input spec, the OSM ingest pipeline shape with timings, the table cells that reflect actually-measured numbers (4 h 25 m ingest, 25.45 M places, 363 131 polygons). Pending cells stay marked as such. Other-engine rows reduced to "not run yet" until measurements land.
1 parent fb46d0a commit 0abec00

1 file changed

Lines changed: 16 additions & 39 deletions

File tree

comparison/index.html

Lines changed: 16 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -457,19 +457,9 @@ <h3>Recall on noisy queries</h3>
457457

458458
<h3>Continent scale — Europe</h3>
459459
<p>
460-
First continent-scale run, on a different host: a Hetzner
461-
bare-metal box with 48 cores, 124 GB RAM, 32 GB swap (rotating)
462-
+ 64 GB NVMe swap, NVMe LVM volume mounted at
463-
<code>/mnt/fast</code>. Input is Geofabrik
464-
<code>europe-latest.osm.pbf</code> (32 GB), 7× larger than
465-
Germany. Cairn runs against this PBF through the same
466-
<code>cairn-build</code> binary as the country-scale runs,
467-
with rkyv <code>size_64</code> rel-pointers (lifts the per-tile
468-
2 GB ceiling that 32-bit relative pointers impose on Russia /
469-
Norway-scale admin polygons) and the Phase 6e streaming
470-
pipeline (single shared AdminIndex, admin-tile write before
471-
tantivy + place-tile write, interleaved place-tile + point-tile
472-
emission off the same sorted slice).
460+
First continent-scale run. Host: 48 cores, 124 GB RAM, NVMe
461+
storage. Input: Geofabrik <code>europe-latest.osm.pbf</code>
462+
(32 GB), 7× larger than Germany.
473463
</p>
474464
<table class="api">
475465
<thead>
@@ -495,43 +485,30 @@ <h3>Continent scale — Europe</h3>
495485
</tr>
496486
<tr>
497487
<td>Pelias</td>
498-
<td colspan="6"><em>not run yet — projected ~12–18 h on the same hardware (Elasticsearch import + WhosOnFirst + OpenAddresses + OSM importers run sequentially via docker-compose; 32 GB PBF dominates the OSM importer pass)</em></td>
488+
<td colspan="6"><em>not run yet</em></td>
499489
</tr>
500490
<tr>
501491
<td>Photon</td>
502-
<td colspan="6"><em>not run yet — projected ~6–10 h (Nominatim DB import is the long pole; Photon's Elasticsearch index build is fast once the Postgres source exists)</em></td>
492+
<td colspan="6"><em>not run yet</em></td>
503493
</tr>
504494
<tr>
505495
<td>Nominatim</td>
506-
<td colspan="6"><em>not run yet — projected ~24–48 h on a single Postgres instance (osm2pgsql import + indexes; cluster has the I/O headroom but the runtime is single-process bound)</em></td>
496+
<td colspan="6"><em>not run yet</em></td>
507497
</tr>
508498
</tbody>
509499
</table>
510500
<p>
511501
Cairn's OSM ingest on Europe — pass 0 (max-id scan) → pass 1
512-
(flatnode write, 110 GB virtual / ~88 GB on NVMe sparse) →
513-
pass 2a (node-place emit) → pass 2b0 (relation way-ref scan)
514-
→ pass 2b1 (parallel way-place emit, 458 M ways scanned,
515-
22.16 M emitted) → pass 2b2 (admin relations, 363 K polygons
516-
out of 8.31 M relations) — completes in 4 h 25 m on this box,
517-
sustaining ~2300 % CPU (≈ 23 cores) through the
518-
rayon-parallel passes. Final bundle write (admin tiles, text
519-
index, place tiles, point tiles, manifest, SBOM) runs after
520-
OSM ingest finishes; full numbers + the corresponding
521-
Pelias / Photon / Nominatim measurements ship in the next
522-
update.
523-
</p>
524-
<p class="caveat">
525-
The cluster also runs an Airflow-driven planetary-imagery
526-
pipeline on the same host. We pause those workers
527-
(<code>kill -KILL</code> the Python pool, airflow respawns
528-
them after the build window) and bump
529-
<code>vm.max_map_count</code> to 1 048 576 so Cairn's mmap
530-
pressure (110 GB flatnode + per-tile output mappings + tantivy
531-
segment files) doesn't run into the 65 530 default limit.
532-
Both knobs are operational, not algorithmic — they don't show
533-
up in country-scale runs because the cluster has more
534-
headroom there.
502+
(flatnode write) → pass 2a (node-place emit) → pass 2b0
503+
(relation way-ref scan) → pass 2b1 (parallel way-place emit,
504+
458 M ways scanned, 22.16 M emitted) → pass 2b2 (admin
505+
relations, 363 K polygons out of 8.31 M relations) —
506+
completes in 4 h 25 m, sustaining ~2300 % CPU (≈ 23 cores)
507+
through the rayon-parallel passes. Final bundle write (admin
508+
tiles, text index, place tiles, point tiles, manifest, SBOM)
509+
runs after OSM ingest finishes; full numbers and the
510+
corresponding Pelias / Photon / Nominatim measurements ship in
511+
the next update.
535512
</p>
536513
</div>
537514
</section>

0 commit comments

Comments
 (0)