Repository navigation
Conversation
Replace spinners with a live step list for extraction and Invenio submission. The server writes progress to the workflow tmpdir, which the loading page polls via a new /progress route. The extraction overlay gains a Cancel button.
jkalayan
deleted the
106-data-submission-to-invenio-times-out-for-large-files-in-production
branch
November 6, 2026 20:21
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This pull request introduces a progress reporting system for uploads and extraction workflows in the BioSimDB interface, improving both backend tracking and frontend user experience. The changes span Python backend modules and frontend JavaScript, enabling cancellation, step visualization, and more informative feedback for users.
Changes
Backend progress tracking and reporting:
ProgressReporterclass inprogress_state.pyto manage file-backed, thread-safe progress reporting, allowing both upload and polling threads to share progress via JSON files. Also added aread_progresshelper.run_record_uploadinclient.pyto accept areportcallback for detailed progress updates at each step (draft creation, file upload, review submission), and integrated calls to this callback throughout the upload process.multipart.py) to accept and propagate anon_updateprogress callback, which is invoked on each progress update, enabling granular progress tracking for both multipart and single PUT uploads.Progressclass inprogress.pyto accept anon_updatecallback and call it after each progress increment, allowing external reporting mechanisms to react to progress changes.Frontend UI improvements for progress and cancellation:
showLoadingOverlayincore/ui.jsto optionally display a stepper panel instead of a spinner, and added a newcreateStepperfunction to render and update step-based progress indicators in the UI.extract/extract.jsto use the new stepper UI: each file upload and the extraction step are shown as progress steps, with real-time updates and error/cancellation handling. Added support for cancelling extraction, which aborts uploads and clears partial workflow state.Other improvements:
schema/webform.pyto clarify session loss and large file handling for users.Testing
Impact
Checklist
pyproject.tomlchanged, the lower bounds were reviewed deliberately