Repository navigation
check: --max-age and --max-duration cover the archives check, fixes #10025 - #10504
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #10504 +/- ##
==========================================
+ Coverage 89.12% 89.30% +0.18%
==========================================
Files 103 104 +1
Lines 19783 20027 +244
Branches 3099 3144 +45
==========================================
+ Hits 17631 17886 +255
+ Misses 1490 1482 -8
+ Partials 662 659 -3 ☔ View full report in Codecov by Harness. |
…orgbackup#10025 Archive check results are kept in cache/checked-archives, refs borgbackup#10356.
253f5da to
f0de2d7
Compare
| raise Exception("Unknown archive metadata version") | ||
| items_buffer = ChunkBuffer(self.key) | ||
| items_buffer.write_chunk = add_callback | ||
| complete = True |
There was a problem hiding this comment.
complete handling is a bit weird.
guess this is a good use case for:
complete = False
for item in ...:
...
else:
complete = True
| try: | ||
| super().save() | ||
| except StorePermissionDenied: | ||
| logger.warning(f"Not storing the archive check results: no permission to write {self.NAME}.") |
There was a problem hiding this comment.
guess it would be good to only emit this once per check run.
don't we have the same issue in the packs tracker?
| try: | ||
| super().clear() | ||
| except StorePermissionDenied: | ||
| logger.warning(f"Not clearing the archive check results: no permission to delete {self.NAME}.") |
|
@mr-raj12 ping |
|
The complete flag is now set in the loop's else clause. |
2 participants
Description
Fixes #10025, refs #10356. Implements what was agreed in #10025 (comment).
borg checknow records the result of the archives check per archive incache/checked-archives(archive id -> timestamp, result), in the same format and envelope ascache/checked-packs.--max-agealso skips archives whose ok record is younger than the timespan, and is allowed with--archives-only.--repairand--verify-datacheck every archive.--max-durationis one time budget for the repository check and the archives check, and no longer requires--repository-only. The archives check takes the archives without an ok record first, then the least recently checked ones, and stops between two archives after it analyzed at least one.--repair.Commits:
repository: split CheckTracker out of PackTracker: refactoring only, no behavior change.check: --max-age and --max-duration cover the archives check: the feature, tests and docs.Decisions that go beyond the issue discussion, please object if you disagree:
checked-packs, and when a run without--repairhas a finding before it analyzes the archives (missing pack, dropped content,--verify-dataor--find-lost-archiveserror).--max-duration, the archives check is skipped with an info message. Without--max-age, a repository check that never finishes within the budget therefore never lets the archives check run. The help text says so and suggests--max-ageor separate--repository-only/--archives-onlyruns.--archives-only --max-durationmakes progress in every run.--max-durationis rejected with--verify-dataand--find-lost-archives, as both scan the whole repository.cache/(BORG_REPO_PERMISSIONS=read-only) can still runcheck --archives-only, it logs a warning and stores no records.Timing, 10 archives of 100k files each, aes256-ocb, local repository:
check --archives-onlycheck --archives-only --max-age=1dcheck --max-age=1dcheck --repository-onlycheck --archives-only --max-duration=3docs/changes.rstand the generated usage docs are not touched. This touches the same files as #10194 and #9891, I will rebase when needed.Checklist
master(or maintenance branch if only applicable there)toxor the relevant test subset)