Skip to content

fix(db): treat an unreadable image as proof of corruption - #36

Merged
dandwhelan merged 3 commits into
mainfrom
fix/db-quarantine-unreadable-image
Aug 14, 2026
Merged

dandwhelan merged 3 commits into
mainfrom
fix/db-quarantine-unreadable-image

Conversation

@dandwhelan

Copy link
Copy Markdown
Owner

Fixes the pre-existing db_manager test failure on main (the "corruption" subtest), by fixing the code rather than the test.

What was wrong

6e41ad7 (#35) narrowed quarantine to a positive corruption verdict after two incidents destroyed databases that were never damaged. That narrowing is right for contention, but it went one branch too far: when quick_check could not run at all, the database was kept whatever the reason.

That inverted the policy. Quarantine could only fire for a file that opens fine and grades bad — the milder damage — while a file whose header SQLite cannot even recognise (file is not a database) was kept for ever.

Two places in the codebase already documented the behaviour that had been lost:

  • _cleanup_db_retry's own docstring: quarantine exists because "looping on a permanently corrupt file would just restart Predbat for ever" — which is precisely the file that was being kept.
  • DatabaseManager.recover_from_error calls _cleanup_db_retry so that it "recovers or starts a fresh database rather than looping". It could no longer do either, leaving Predbat running with db_primary set and every history write failing for the life of the process, unnotified.

Both incidents behind 6e41ad7 were contention — a lock, and a file a later integrity_check proved healthy. Contention is already excluded one branch earlier by the OperationalError check, and every benign SQLite failure (BUSY, LOCKED, CANTOPEN, IOERR) is an OperationalError. What still reaches quick_check as a plain DatabaseError carrying a result code is SQLITE_CORRUPT or SQLITE_NOTADB — both positive reports of damage.

The change

Quarantine now also fires when quick_check itself fails with one of those two codes.

The match is on the result code, not the exception class: ProgrammingError ("Cannot operate on a closed database") is a DatabaseError too, and a bug of our own must never cost the user their history. The code is masked to its low byte so extended codes such as SQLITE_CORRUPT_INDEX (779) still match.

Everything 6e41ad7 protects is untouched — contention, a clean verdict, and an error with no corruption code all still keep the database.

Tests

The test asserted the old behaviour and had been failing on main since 6e41ad7, which never updated it. It now pins every branch:

Scenario Expected
Transient malformed error, then success kept
Repeated OperationalError (lock contention) kept
quick_check grades ok, cleanup still failing kept
DatabaseError with no corruption code (ProgrammingError) kept
quick_check grades the image damaged quarantined
Quarantine moves -journal/-wal/-shm with the database sidecars follow
quick_check cannot run: file is not a database quarantined

The positive-verdict case the old test claimed to cover was in fact exercising the unreadable path — it is now tested for real, against a genuinely damaged database image (garbage written over pages 3-6, header left intact) rather than mocks.

Verified: db_manager 9/9 pass, and the new test fails against the pre-change db_engine.py on the unreadable case, so it genuinely pins the semantics. black clean and ruff --select=F401 (as CI runs it) clean.

🤖 Generated with Claude Code

dandwhelan and others added 3 commits August 14, 2026 23:38
6e41ad7 narrowed quarantine to a positive corruption verdict after two
incidents destroyed databases that were never damaged. The narrowing is right
for contention, but it went one branch too far: when quick_check could not run
at all the database was kept, whatever the reason.

That inverted the policy. Quarantine could then only fire for a file that opens
fine and grades bad - the milder damage - while a file whose header SQLite
cannot even recognise was kept for ever. The docstring's reason for quarantining
at all, that "looping on a permanently corrupt file would just restart Predbat
for ever", describes exactly the file that was being kept, and
DatabaseManager.recover_from_error calls _cleanup_db_retry precisely so it
"recovers or starts a fresh database rather than looping" - which it could no
longer do. The result was Predbat running with db_primary set and every history
write failing for the life of the process, with no notification.

Both incidents behind 6e41ad7 were contention: a lock, and a file a later
integrity_check proved healthy. Contention is already excluded one branch
earlier by the OperationalError check, and every benign SQLite failure - BUSY,
LOCKED, CANTOPEN, IOERR - is an OperationalError. What still reaches quick_check
as a plain DatabaseError carrying a result code is SQLITE_CORRUPT or
SQLITE_NOTADB, and both are positive reports of damage.

Quarantine now also fires when quick_check itself fails with one of those two
codes. The match is on the result code rather than the exception class, because
ProgrammingError ("Cannot operate on a closed database") is a DatabaseError too
and a bug of our own must never cost the user their history; the code is masked
to its low byte so extended codes such as SQLITE_CORRUPT_INDEX still match.

The test asserted the old behaviour and had been failing on main since 6e41ad7,
which never updated it. It now pins every branch: transient error, lock
contention, a clean verdict with a stuck cleanup, and a DatabaseError with no
corruption code all keep the database; a graded-bad image and an unreadable one
are both quarantined, sidecars included. The positive-verdict case the old test
claimed to cover was in fact exercising the unreadable path, so it is now tested
for real against a damaged database image.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pre-commit has been failing on main independently of any change here, which
also stops the workflow before it reaches the unit tests. All of it is in files
this branch does not otherwise touch:

- cspell: datasheet, gasbill and unsubtracted were unknown words in
  annual_heat.py, annual.py and web_annual.py
- ruff: two unused imports
- black: three files needed reformatting

Every change is the hooks' own auto-fix, applied with the pinned versions
(ruff 0.11.4, black 23.11.0) so it matches what CI generates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dandwhelan
dandwhelan merged commit 4ce4bf1 into main Aug 14, 2026
2 checks passed
@dandwhelan
dandwhelan deleted the fix/db-quarantine-unreadable-image branch August 14, 2026 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant