Skip to content

Latest commit

 

History

27 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

NSR QA Suite

A browser-based quality-assurance suite for Entur National Stop Register (NSR) data, built on the national NeTEx export. One file load feeds several QA tools, each on its own tab, all sharing a map and a global geographic filter.

It supersedes the earlier single-purpose tools (gosp-qa, alternativename-qa) by consuming the same source file they would and running every check from one upload.


Where is it?

https://entur.github.io/tiamat-qa/


What's in it

Global tools (top bar, apply to every tab):

  • Load NeTEx file — pick the unzipped national export (.xml, ~400 MB). One load, parsed once.
  • Get latest export — opens the current archive for download (unzip, then load).
  • Region filter — county → municipality. Scopes every tab and the map to the area you're responsible for.
  • Map background — selectable base tiles.

Tabs (each a self-contained validator — see its METHOD doc):

Tab Purpose Method
Overview Cross-tool counts for the current region; drill-down county→municipality table that drives the filter
AlternativeNames Flags useless / unhelpful aliases and malformed/colliding names; issue-confidence rating docs/METHOD-altnames.md
Names N801 naming checks: unexpanded abbreviations, disallowed characters, duplicates within a municipality docs/METHOD-names.md
Tags Follow-up on the front-end "tag" post-it notes: open vs resolved, oldest-first, by tag name docs/METHOD-tags.md
GroupOfStopPlaces Validates group members (expired / monomodal-child), all resolved from the file docs/METHOD-gosp.md
Parking Stop-centric parking explorer: completeness of expected fields per vehicle type, plus cross-field consistency docs/METHOD-parking.md
Untouched Stops whose effective version date is June 2017 (the pre-QA import month) — never reviewed since launch docs/METHOD-untouched.md
Recent changes Stops modified within a configurable window (week/month/90 days); age-graded pins, NEW detection, VERSION_COMMENT docs/METHOD-recentchanges.md

Plus one live lookup, on demand:

Feature Purpose Method
Timetable (top bar) Ask the journey planner which lines and service journeys use the selected stop, and on what dates. Available from every tab. docs/METHOD-timetable.md

Validation itself needs no network — the map tiles and the Timetable lookup are the only outbound calls, and the lookup only fires when you click it. Your data is never uploaded.

Deployed at https://entur.github.io/tiamat-qa/, so this is a hosted tool in practice rather than a local file.

Tab counts

Each tab shows how many rows it would list under the current region filterAltNames (24), and AltNames (1) once you narrow to a single municipality. Everything is computed at load time (which is why Overview can show its totals), so the badges cost nothing.

The count comes from each tool's inRegion() — the region-filtered set before the tab's own search box and filter chips. So the badge describes the region, not whatever you happen to have filtered inside the tab.

Parking is counted differently from Overview, on purpose. The tab badge counts stops that have parking (what the tab lists); the Overview card counts parking facilities, of which there can be several per stop. Telemark: Parking (12) vs "21 parking facilities". Both are right for what they measure.

Recent changes recomputes its badge when the week/month/90-day window changes.

Presentation

  • Dark mode — the button in the top bar. Light is the default; the OS prefers-color-scheme setting is deliberately ignored, so the tool never opens dark unexpectedly. The choice persists in localStorage, and the map tiles follow it (Positron light/dark, OSM light / CartoDB dark).
  • LanguageEN / NO toggle. Switching re-renders in place; it never reloads, because reloading would mean re-reading the ~400 MB export. Transmodel terms (GroupOfStopPlaces) and payment-method values are intentionally left untranslated.
  • Welcome panel — feature summary over the map on load, fading out once data is loaded.

Running it

index.html is a single self-contained file — all CSS and JavaScript are inlined. Open it directly:

  • file:// — just double-click index.html in your file manager, or open it via File → Open in the browser.
  • Any static host — e.g. GitHub Pages, npx serve, nginx. No trailing-slash or module-path requirements.

Then: Load NeTEx file… → pick the unzipped .xml → a progress card runs the single-pass scan → the tabs populate.


Configuration

All in the // ── config.js ── section of index.html.

CONFIG.tools — enable/disable validators per deployment

tools: {
  overview:      true,
  altnames:      true,
  names:         true,
  tags:          true,
  gosp:          true,
  parking:       true,
  untouched:     true,
  recentchanges: true,
}

Set a tool to false to remove its tab and skip its work in the parse (e.g. a deployment with no tag validator: tags: false — tag KeyValues are then never collected). The Overview adapts to whichever tools are on. Setting both untouched and recentchanges to false also skips parsing of version, created, changed, fromDate, transportMode, and versionComment.

Other CONFIG

Key Purpose
exportUrl Target of Get latest export.
chunkSize Streaming read chunk size (bytes).
nsrStopUrl / nsrGroupUrl NSR editor deep-link templates ({id} replaced at runtime). Verify against the live editor.
nsrLinkLabel Link button text.

Deep links: do not URL-encode the id. NSR ids must appear in editor links with literal colons (NSR:StopPlace:1234), not percent-encoded — the editor will not resolve an encoded id. This differs from Entur's REST API, where encoding the id is correct. Different targets, different rules.

IDs are otherwise treated as opaque strings throughout: never parsed, split or interpreted to infer meaning.

THEME and DARK_THEME

Colours as CSS custom properties injected at startup by applyTheme(), plus typeColors for AlternativeName types and modeColors for transport modes.

DARK_THEME is Object.assign({}, THEME, { …overrides }), and [data-theme="dark"] on <html> drives the CSS overrides. Every colour must come from a custom property — a hardcoded hex in a rule is invisible to the toggle and will look wrong in one of the two themes. body in particular needs an explicit token background, since a JS error that prevents applyTheme() from running leaves the page white.


Architecture

Everything lives in a single file: index.html. CSS is in a <style> block; all JavaScript is one inline IIFE with comment-delimited sections in dependency order:

// ── config.js ──       CONFIG + THEME (modeColors, typeColors) + applyTheme
// ── util.js ──         shared helpers (text normalisation, edit-distance/LCS/Jaccard, dates, DOM)
// ── parse.js ──        single-pass streaming NeTEx parser → shared model
// ── region.js ──       topographic resolution + the county/municipality filter
// ── map.js ──          shared Leaflet map + background switching
// ── shared mode helpers ──  modeOf / modeColor / orderModes
// ── tools/overview.js ──    dashboard tab
// ── tools/altnames.js ──    AlternativeNames validator
// ── tools/names.js ──       Names (N801) validator
// ── tools/tags.js ──        Tags validator
// ── tools/gosp.js ──        GroupOfStopPlaces validator
// ── tools/parking.js ──     Parking completeness validator
// ── tools/untouched.js ──   Untouched (2017 import) validator
// ── tools/recentchanges.js ── Recent changes validator
// ── app.js ──          orchestrator: global bar, tabs, per-tool lifecycle, i18n, theme

One source file, parsed once. The parser streams the export in chunks (boundary-safe UTF-8) and routes TopographicPlace, GroupOfStopPlaces and StopPlace blocks into a shared model ({ publishedAt, topo, stops, gosps }). Topographic blocks are read by regex; stops and groups are DOM-parsed. Disabled tools skip their extraction. Peak memory is the model plus one in-flight block.

Shared model, per-tab views. Each tool is an object with the contract { id, label, mount(ctx), render() }. ctx gives it the model, the region filter, the shared map, its own map layer, and its sidebar pane. The app owns tab switching (show/hide pane, add/remove layer) and calls render() on activation and whenever the region filter changes.

Geography. IDs are opaque; region is resolved purely by following refs: a stop's TopographicPlaceRef → municipality → (ParentTopographicPlaceRef) county → country, with TopographicPlaceType naming the level.


Adding a tool

  1. Add a // ── tools/yourtool.js ── section to index.html before // ── app.js ──, declaring const yourtool = { id, label, mount(ctx), render() }.
  2. Add yourtool to ALL_TOOLS in app.js.
  3. Add yourtool: true to CONFIG.tools (and, if it needs a new extraction, a flag in the OPTS block of parse.js).
  4. Add a docs/METHOD-yourtool.md.

Internationalisation

STRINGS holds one object per language (en, no); t(key) looks a key up with an English fallback; applyI18n() walks every [data-i18n] element and rewrites its text (or placeholder, or <option> label).

Adding a language means adding one key to STRINGS. Adding a translatable string means adding the key and a data-i18n attribute on the element.

Dynamically built markup is not covered by applyI18n(), so the language toggle also calls activeTab.render() to redraw the current pane. Tab buttons carry data-i18n so they update too.

Beware shadowing: the i18n function is the single character t. Local variables named t (a tag, a theme, a tool) will silently shadow it inside their scope. Existing code renames such locals (tools, tool2) for this reason.


Maintenance traps

Every item below caused a real failure in this codebase.

A JavaScript syntax error blanks the entire page. Everything is one IIFE, so a single bad token means nothing runs — no tabs, no theme, white screen. Two real instances:

  • Curly quotes as string delimiters. Generated code contained 'click' and 'untouched' with U+2018/U+2019 instead of ASCII '. Visually almost identical, fatal to the parser. When a page goes blank, check the console for the line and verify the actual code points, don't eyeball the characters.
  • Identity escapes in a /u regex. In unicode mode only SyntaxCharacters may be backslash-escaped; or \' is a SyntaxError. See docs/METHOD-names.md.

Leaflet tooltips take raw HTML. bindTooltip() sets innerHTML, so passing HTML-escaped text renders visible entities or an empty box. Pass the raw string.

scrollIntoView is unreliable straight after a full list rebuild. Selecting from the map re-renders the sidebar, so the target node is brand new. Use requestAnimationFrame and set scrollTop from offsetTop.

Render caps can hide the selection. Lists cap at a few hundred rows; a map click can select an item beyond the cap. Move the active item to the front of the array before slicing.

Verify in the browser, not by grepping. Confirming that a string is present in the source proves nothing about whether the page runs. Load it and check the console.


Map tiles

CARTO/Positron was removed — it now requires an API key and serves tiles watermarked "API KEY REQUIRED" while still returning HTTP 200, so the failure is invisible to any status check.

Replaced with keyless providers:

Option Light Dark
Grey (pale) (default) Esri World Light Gray Canvas Esri World Dark Gray Canvas
OpenStreetMap standard OSM tiles Esri dark canvas (OSM has no dark style)

Esri splits its canvas basemaps into a base and a reference (labels) layer, so a background is a list of URLs stacked in order — hence BACKGROUNDS[name][variant].urls is an array and each entry becomes a L.layerGroup of tile layers. Attribution is set on the base layer only, to avoid duplicating it.

Esri canvas tiles stop around zoom 16, so maxNativeZoom: 16 with maxZoom: 19 lets Leaflet upscale rather than showing blank tiles when zoomed further.

Note the Esri URL template is {z}/{y}/{x}y before x, unlike the usual {z}/{x}/{y}.


Status

Working tool (v0.8), used against the real ~400 MB national export.

Not calibrated: thresholds in the AlternativeNames and Tags validators are first-pass and not yet checked against bulk manual review. The allowed-abbreviation and permitted-character lists in the Names validator grow in response to false positives on real data.

The standalone tools (gosp-qa, alternativename-qa) remain in the repo until the suite is confirmed at parity.

Planned (v0.8)

  • Quay-level dataDescription on both StopPlace and Quay, and PublicCode on Quay, all of which give passengers identifying markers to compare against what is physically on the stop. Quay elements are already inside the StopPlace blocks the parser reads, so this costs no extra I/O — roughly 5–15% more parse time and 20–50 MB more memory. PublicCode is validatable (it should be a short code — a single character, or something like A21; BC2, AA123 or plattform 2 are not); the descriptions are free text and will initially just be surfaced for review.
  • Quay equipment — shelter and transport sign presence, from placeEquipments.
  • Open question: whether the quay view is stop-grouped (drill into a stop, like Parking) or a flat filterable list. Undecided.

License

MIT — see repository root.

About

No description or website provided.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages