Skip to content

Commit dfa0d77

Browse files
committed
feat(vcs): run repository maintenance in background
Repository operations can exceed proxy timeouts and provide little feedback while requests remain open. Queue browser actions and opt-in API requests as serialized, trackable Celery tasks.
1 parent 2489bd2 commit dfa0d77

22 files changed

Lines changed: 3391 additions & 345 deletions

docs/admin/continuous.rst

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -227,10 +227,17 @@ Availability of individual actions depends on permissions, the configured
227227
version control system, whether pushing is configured, and whether the selected
228228
object can be locked.
229229

230-
The :guilabel:`File management` actions are available only from
231-
:guilabel:`Repository maintenance` for an individual translation. These actions
232-
rewrite that translation file and commit the result; they are not project-wide
233-
or component-wide operations.
230+
Repository actions started from this view are queued for background processing.
231+
For a project, Weblate processes the affected repositories sequentially in one
232+
task and shows its progress. Repeating the same action opens the existing task;
233+
a different action cannot be started for the same repository until that task
234+
finishes.
235+
236+
The :guilabel:`Synchronize` and :guilabel:`Rescan` repository operations are
237+
queued for background processing as described above. The separate
238+
:guilabel:`File management` actions are available only for an individual
239+
translation. They rewrite that translation file and commit the result during
240+
the web request; they are not project-wide or component-wide operations.
234241

235242
Operations that read repository content, such as updating, resetting, or
236243
rescanning, also reconcile translation files in Weblate. Added or removed

docs/api.rst

Lines changed: 44 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1204,10 +1204,19 @@ Projects
12041204
:param project: Project URL slug
12051205
:type project: string
12061206
:<json string operation: Operation to perform: one of ``push``, ``pull``, ``commit``, ``reset``, ``cleanup``, ``file-sync``, ``file-scan``
1207-
:>json boolean result: result of the operation
1207+
:<json boolean background: Schedule the operation as a background task instead of waiting for it to finish. Defaults to ``false``.
1208+
:>json boolean result: result of a synchronous operation
12081209
:>json array included_components: full paths of project components included in the operation
12091210
:>json array skipped_components: full paths of project components omitted from the operation
12101211
:>json array permission_blockers: full paths of components preventing access to omitted repositories
1212+
:>json string detail: Status of a background operation
1213+
:>json string task_url: URL for tracking a background operation; see :http:get:`/api/tasks/(str:uuid)/`
1214+
1215+
With ``background`` set to ``true``, the endpoint returns ``202 Accepted``.
1216+
Repeating an identical queued operation returns the existing task URL. A
1217+
conflicting operation returns ``423 Locked`` and the active task URL when
1218+
available. Eligible project repositories are processed sequentially in one
1219+
task.
12111220

12121221
**CURL example:**
12131222

@@ -1251,6 +1260,33 @@ Projects
12511260
"skipped_components": ["hello/glossary"]
12521261
}
12531262

1263+
**Background JSON request example:**
1264+
1265+
.. sourcecode:: http
1266+
1267+
POST /api/projects/hello/repository/ HTTP/1.1
1268+
Host: example.com
1269+
Accept: application/json
1270+
Content-Type: application/json
1271+
Authorization: Token TOKEN
1272+
1273+
{"operation":"pull","background":true}
1274+
1275+
**Background JSON response example:**
1276+
1277+
.. sourcecode:: http
1278+
1279+
HTTP/1.0 202 Accepted
1280+
Content-Type: application/json
1281+
1282+
{
1283+
"detail": "Repository operation has been queued.",
1284+
"included_components": ["hello/app"],
1285+
"permission_blockers": ["shared/glossary"],
1286+
"skipped_components": ["hello/glossary"],
1287+
"task_url": "https://example.com/api/tasks/01234567-89ab-cdef-0123-456789abcdef/"
1288+
}
1289+
12541290

12551291
.. http:get:: /api/projects/(string:project)/components/
12561292
@@ -3384,6 +3420,13 @@ Tasks
33843420
:>json int progress: Task progress in percent
33853421
:>json object result: Task result or progress details
33863422
:>json string log: Task log
3423+
:>json boolean cancellable: Whether the task can be cancelled
3424+
3425+
.. http:delete:: /api/tasks/(str:uuid)/
3426+
3427+
Cancels a running task when its ``cancellable`` property is ``true``.
3428+
Repository operation tasks cannot be cancelled because interruption can
3429+
leave a repository operation incomplete.
33873430

33883431
.. _api-statistics:
33893432

docs/changes.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ Weblate 2026.9
1111
.. rubric:: Improvements
1212

1313
* Removing the final Weblate workspace connection for a GitHub account, or removing the workspace holding it, now also uninstalls the Weblate GitHub App from GitHub.
14+
* Core repository maintenance actions now run as background tasks, avoiding request and proxy timeouts. The repository API supports the same behavior using ``background: true``. See :ref:`repository-maintenance`.
1415
* :ref:`addon-weblate.gettext.xgettext` now accepts multiple custom keywords (newline-separated) passed to xgettext via ``--keyword``, enabling extraction from different function names.
1516
* Screenshot images are now cached in browsers to reduce repeated downloads.
1617
* Administrators can now find removed accounts by their former e-mail address in the audit log until :setting:`AUDITLOG_EXPIRY`.

docs/security/threat-model.rst

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ documentation; ``*(maintainer)*`` means it was stated by a maintainer during
2525
this threat-model process; ``*(inferred)*`` means it was reasoned from the
2626
current project shape and needs maintainer confirmation.
2727

28-
Provenance summary: 103 documented / 64 maintainer / 0 inferred claims.
28+
Provenance summary: 103 documented / 65 maintainer / 0 inferred claims.
2929

3030
Weblate is a Django-based web localization platform. It accepts work from
3131
browser users, API clients, project-scoped tokens, repository webhooks, VCS
@@ -193,6 +193,13 @@ repository state, background tasks, outbound requests, and rendered UI.
193193
* - Client browser/API client to Weblate
194194
- Untrusted or authenticated requests become permission-checked
195195
application actions. *(documented)* (source: :doc:`/api`, :doc:`/admin/access`)
196+
* - Weblate request process to repository Celery worker
197+
- Permission-checked browser and API repository actions become queued work
198+
carrying the initiating user and affected repository scope. The worker
199+
reacquires the datastore reservation and rechecks the user's current VCS
200+
permission across the current linked-component scope before mutation.
201+
The broker, datastore, and workers are trusted parts of the same Weblate
202+
instance. *(maintainer)*
196203
* - Webhook sender to Weblate
197204
- Public forge notifications can schedule repository synchronization
198205
where hooks are enabled, matching components by exact repository URL
@@ -621,9 +628,13 @@ Security properties Weblate provides
621628
explicit VCS actions cover every component sharing an affected
622629
repository, including linked components in other projects. Project-wide
623630
VCS actions omit repositories where this permission check fails; they do
624-
not partially operate on an individual shared checkout. Weblate's normal
625-
background commit and push of authorized translation changes does not
626-
require the editor to have these VCS permissions. Translation memory
631+
not partially operate on an individual shared checkout. Explicit VCS
632+
actions queued from the browser or API retain the initiating user,
633+
serialize access to the affected repositories, and recheck that user's
634+
permission against the current linked-component scope in the worker
635+
before mutation. Weblate's normal background commit and push of
636+
authorized translation changes does not require the editor to have
637+
these VCS permissions. Translation memory
627638
attributed to an existing restricted component follows that component's
628639
access rules.
629640
Unattributed automatic memory, including unmatched legacy entries and

docs/specs/openapi.yaml

Lines changed: 51 additions & 20 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

weblate/api/serializers.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2831,7 +2831,8 @@ class BooleanResultSerializer(ReadOnlySerializer):
28312831
result = serializers.BooleanField()
28322832

28332833

2834-
class RepositoryOperationSerializer(BooleanResultSerializer):
2834+
class RepositoryOperationSerializer(ReadOnlySerializer):
2835+
result = serializers.BooleanField(required=False)
28352836
detail = serializers.CharField(required=False)
28362837
included_components = serializers.ListField(
28372838
child=serializers.CharField(),
@@ -2848,6 +2849,7 @@ class RepositoryOperationSerializer(BooleanResultSerializer):
28482849
required=False,
28492850
help_text="Full paths of components preventing access to skipped repositories.",
28502851
)
2852+
task_url = serializers.URLField(required=False)
28512853

28522854

28532855
class UploadResultSerializer(BooleanResultSerializer):
@@ -3058,6 +3060,7 @@ class RepoRequestSerializer(ReadOnlySerializer):
30583060
operation = serializers.ChoiceField(
30593061
choices=RepoOperations.choices,
30603062
)
3063+
background = serializers.BooleanField(required=False, default=False)
30613064

30623065

30633066
class CommitInfoSerializer(ReadOnlySerializer):
@@ -4347,6 +4350,7 @@ class TaskSerializer(ReadOnlySerializer):
43474350
progress = serializers.IntegerField(min_value=0, max_value=100)
43484351
result = TaskResultField()
43494352
log = serializers.CharField(allow_blank=True)
4353+
cancellable = serializers.BooleanField()
43504354

43514355

43524356
@extend_schema_serializer(

0 commit comments

Comments
 (0)