Skip to content

OP#10282: Parse track files off the event loop - #840

Merged
randy-seng merged 5 commits into
mainfrom
task/10282-keep-the-web-ui-responsive-while-loading-track-files
Sep 29, 2026
Merged

randy-seng merged 5 commits into
mainfrom
task/10282-keep-the-web-ui-responsive-while-loading-track-files

Conversation

@randy-seng

@randy-seng randy-seng commented Sep 8, 2026 •

Copy link
Copy Markdown
Member

What this does

Opening an otconfig or adding track files parsed everything on the event loop, so the browser froze for as long as parsing took — on a large project, minutes with no repaint and no way to tell whether anything was happening. LoadTrackFiles.load_async now hands TrackParser.parse_files to asyncio.to_thread and publishes the result back on the event loop, and it opens a progressbar for the duration so the user can see the application is busy. LoadOtconfig gained a matching load_async, and DummyViewModel.load_tracks / _load_otconfig await
it.

Why

Parsing is the expensive part and it is pure — no repository, no observer, no widget — so it is the only part safe to move off the loop.
Everything downstream of it notifies observers that mutate widgets, so _publish stays on the loop. That split is why the use case grew a _parse/_publish pair rather than just wrapping the old __call__ in a thread.

The blocking path could not simply be deleted: PreloadInputFiles loads a --config file at startup, before any loop exists. So both paths stay, and load now raises if it finds a running loop rather than freezing the ui for a caller who picked the wrong one.

ProgressbarBuilder only knew how to decorate a sequence, which cannot track work done on another thread. It gained start(...) -> RunningProgressbar, a handle the caller closes; the default implementation returns NoRunningProgressbar, so command line runs show nothing — their output is already a log.

Notes for the reviewer

Start at OTAnalytics/application/use_cases/load_track_files.py — the load / load_async pair and the _parse / _publish split are the whole change. The docstring on load_async states the invariant the rest depends on.

load_otconfig.py looks like a big diff but is mostly extraction: the old load body became _begin / _publish_before_tracks / _publish_after_tracks / _abort, shared by a sync and an async _apply. The behaviour is unchanged; test_publishes_everything_the_blocking_load_publishes pins that.

Naming: the last commit renamed LoadTrackFiles.__call__ semantics to match LoadOtconfig — load blocks, load_async yields, __call__ delegates to load. load meant opposite things in the two neighbouring use cases before.

The acceptance helper change is the other half of the fix: open_project_otconfig and add_track_via_picker slept a fixed 5s and carried on, which after this change means clicking into an application that has no tracks yet. They now wait for the parsing progressbar to disappear (wait_for_track_loading_to_finish), treating "never appeared within 5s" as "nothing to parse".

Risk and rollout

Anything calling add_tracks_of_files or load_otconfig from inside a running loop now raises RuntimeError instead of silently freezing. That is deliberate and loud, but it is a behaviour change for any call site not updated here — the switch is add_tracks_of_files_async / load_otconfig_async. The desktop and CLI paths still use the blocking ones.

Failure mode to watch for: a progressbar that never closes. close() sits in a finally, so a parser exception still clears it; test_closes_the_progressbar_when_parsing_fails covers that case.

Revert is the four commits — nothing persisted, nothing migrated.

Testing

Still to verify by hand: the acceptance suite (tests/acceptance) — the new wait_for_track_loading_to_finish and the export file polling were
not exercised here, and the 120s timeouts were chosen from local timings rather than measured on CI.

OP#10282

@randy-seng
randy-seng force-pushed the task/10282-keep-the-web-ui-responsive-while-loading-track-files branch from a60313d to 8f2dd1e Compare September 9, 2026 22:01
@randy-seng
randy-seng force-pushed the task/10282-keep-the-web-ui-responsive-while-loading-track-files branch from 8f2dd1e to cacd6ee Compare September 9, 2026 22:22
@randy-seng
randy-seng force-pushed the task/10282-keep-the-web-ui-responsive-while-loading-track-files branch 2 times, most recently from 5426fd9 to 8cfad37 Compare September 14, 2026 16:33
@randy-seng
randy-seng force-pushed the task/10282-keep-the-web-ui-responsive-while-loading-track-files branch from 83d6c56 to 4a3a68c Compare September 16, 2026 11:42
@randy-seng
randy-seng force-pushed the task/10282-keep-the-web-ui-responsive-while-loading-track-files branch from 4a3a68c to 7e4ece5 Compare September 16, 2026 14:06
@randy-seng
randy-seng requested a review from dahelb September 16, 2026 15:33
@randy-seng randy-seng self-assigned this Sep 16, 2026
@randy-seng
randy-seng force-pushed the task/10282-keep-the-web-ui-responsive-while-loading-track-files branch from 7e4ece5 to 34e9e5d Compare September 16, 2026 16:34
@randy-seng
randy-seng requested review from schubsen and removed request for dahelb September 21, 2026 09:42
Comment thread OTAnalytics/application/use_cases/load_otconfig.py Outdated
Base automatically changed from task/10281-show-progress-of-long-operations-in-the-web-ui to main September 21, 2026 12:46
Parsing moved off the event loop, so the click that starts loading returns
long before the tracks are in the repository. Nothing in the browser said so:
no progressbar, no disabled buttons, nothing. A user opening a large otconfig
saw an idle looking application with live buttons that silently did nothing,
and the acceptance tests, which slept a fixed five seconds and then carried on,
clicked "Export track statistics ..." into an application without tracks.

LoadTrackFiles already took a ProgressbarBuilder but never used it. Work handed
to a worker thread cannot be tracked by decorating a sequence, so builders now
also start a progressbar the caller closes again. Builders that can only
decorate a sequence show nothing, which is what a command line run wants.

The acceptance helpers wait for that progressbar to disappear instead of
sleeping. While there, poll for the exported file rather than sleeping five
seconds and asserting it exists: exporting takes about four seconds here, so
the wait was already marginal and failed on a busy machine.
LoadTrackFiles used __call__ for the blocking load and load for the
awaitable one, while LoadOtconfig next to it used load and load_async.
That made load mean the opposite thing in two neighbouring use cases.

LoadTrackFiles now follows LoadOtconfig: load blocks, load_async parses
off the event loop, and __call__ delegates to load so PreloadInputFiles
keeps working.
@randy-seng
randy-seng force-pushed the task/10282-keep-the-web-ui-responsive-while-loading-track-files branch from 34e9e5d to 3c4a720 Compare September 21, 2026 12:46
@randy-seng
randy-seng requested a review from schubsen September 21, 2026 13:00
@briemla
briemla self-requested a review September 28, 2026 18:22
@briemla
briemla removed the request for review from schubsen September 28, 2026 18:22
@briemla
briemla dismissed schubsen’s stale review September 28, 2026 18:24

The requested change has been implemented. A review by another person has been done.

@randy-seng
randy-seng merged commit d040b85 into main Sep 29, 2026
15 checks passed
@randy-seng
randy-seng deleted the task/10282-keep-the-web-ui-responsive-while-loading-track-files branch September 29, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants