test: guard the samples, tighten the qpdf verdict, and stop hiding the interop run - #246
Merged
Merged
Conversation
…e interop run Four open points, all of them a check that was quietly not checking. samples/ went stale for a whole release and nothing noticed. 2.4 made the seal transparent and carried the trailer /ID, and every committed sample still showed the 2.3 shape while the suite read them for chains, timestamps and structure: it was reading old evidence and agreeing with itself. tests/SamplesTest.php asserts they carry what this version writes. Bytes cannot be compared, since signing embeds a time and a throwaway certificate, but the structures can. The qpdf verdict on the samples filtered down to errors, because two fixtures had page objects with no /Resources (ISO 32000-1 §7.7.3.3) and qpdf warned and repaired. One warning was reason enough to ignore warnings on every other sample too, which is how a compromise in one place turns a gate off everywhere. poc/rebuild-stream-fixtures.php regenerates both, preserving object numbering and sizes because tests pin them, and the assertion is now the whole verdict. The live timestamp authority step is continue-on-error, which is the right decision and was hiding its own outcome behind a green tick. That is exactly how three of those tests were committed broken, went green, and were noticed only by someone reading the log. It now writes what happened into the job summary. Non-blocking was the decision; invisible was not. UPGRADE.md said "From 2.4.0 to 2.5.0" for a release that is deliberately not happening yet, and the record and a test comment claimed the version too. Now "Unreleased", and both cite 0028 instead.
lsnepomuceno
added
enhancement
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four of the open points, and they share a shape: a check that was quietly not checking.
samples/went stale for a whole release2.4 made the seal transparent and carried the trailer
/IDinto the revision. Every committed sample still showed the 2.3 shape, and the suite read them the whole time for chains, timestamps and structure. It was reading old evidence and agreeing with itself.Nothing was watching, so
tests/SamplesTest.phpnow does. Bytes cannot be compared, since signing embeds a time and a throwaway certificate, but the structures can: the seal's/ICCBasedcolour space, the/SMaskthat transparency implies, the archive timestamp's appearance dictionary, and no invented/ID.Regenerated in this PR, so the diff carries the seal colour space too.
The qpdf verdict was filtered down to errors
Two fixtures had page objects with no
/Resources, which ISO 32000-1 §7.7.3.3 requires, so qpdf warned and repaired. That one warning was reason enough to ignore warnings on every other sample too: a compromise in one place turning a gate off everywhere.poc/rebuild-stream-fixtures.phprebuilds both, preserving object numbering,/Sizeand structure because tests pin them. qpdf now says "No syntax or stream encoding errors found" about each, and the assertion is the whole verdict rather than a slice of it.The interop run was hiding its own outcome
The live timestamp authority step is
continue-on-error, which is right: an outage on freetsa's side is not a defect here. But it hid the result behind a green tick, and that is exactly how three of those tests were committed broken, went green, and were noticed only by someone reading the log.It now writes what happened into the job summary. Non-blocking was the decision; invisible was not.
No 2.5.0
UPGRADE.mdsaid "From 2.4.0 to 2.5.0" for a release that is deliberately not happening yet, and a decision record and a test comment claimed the version too. Now "Unreleased", both citing 0028 instead.Still open, and not in here: the DSS is not refreshed when extending an archive,
/DocMDPhas no independent verification, and theSupportmutation floor still sits on one measurement.composer checkgreen: 443 passing, PHPStan level max, Pint clean.