Repository navigation
Key Debian and Conan archives by their whole path - #690
Conversation
A Debian package was cached under its repository and file name, but fetched from its full path. Docker's apt repository has a pool under each suite with the same file names, so a bookworm client could get the bullseye build that was fetched first, and fail its hash check. Packages are now cached under their path. Conan joined revision, package reference, package revision and file name with underscores, which revisions can contain, so a recipe file and a package file could share a cache entry. The parts are now joined with slashes. Entries cached under the old names are still served, but only for the URL they were fetched from, so warm caches keep working.
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
andrew
left a comment
There was a problem hiding this comment.
Both collisions are real and keying by path is the right fix. The legacy handling looks good too: old entries are only reused when the recorded upstream URL matches, and the tests cover that through the routes.
One thing to fix before merging. The new Conan keys still collide when a path segment contains an encoded slash. r.PathValue decodes %2F, so
/v2/conans/zlib/1.3/_/_/revisions/R%2Fpackages%2FP%2FQ/files/conan_package.tgz
matches the recipe route with revision set to R/packages/P/Q, and conanStorageFilename returns _/_/R/packages/P/Q/conan_package.tgz. That is the same key as the package file at revisions/R/packages/P/revisions/Q/files/conan_package.tgz. Before this change those two requests had different keys. The upstream would have to serve /revisions/R/packages/P/Q/files/conan_package.tgz for it to matter, which is not a Conan route, so it is narrow, but the point of the change is that the key can't be shared.
In handleRecipeFile and handlePackageFile (internal/handler/conan.go), reject the request with a 400 when any of user, channel, revision, pkgref, pkgrev or filename contains a /, and add a test that requests the %2F URL above. The doc comment on conanStorageFilename says the parts can't contain a slash, which is only true once that check is there.
The Debian side is fine: the key and the download URL are both built from the same decoded path.
Smaller: the doc comment on cacheFilenameKeepingLegacy in internal/handler/handler.go is five lines and the first sentence is hard to follow. Two lines would do: it returns legacy when an artifact recorded under it was fetched from downloadURL, and filename otherwise.
A path value is decoded, so a revision of R%2Fpackages%2FP%2FQ gave a recipe file the cache key of a package file. The recipe and package file routes now answer 400 when user, channel, a revision, the package reference or the filename contains a slash.
|
Fixed in 3e50f45.
|
Two cache keys that more than one URL could share:
dists/bullseye/pool/.../containerd.io_1.7.25-1_amd64.deb,dists/bookworm/pool/...), so a bookworm client got the bullseye build that was fetched first and failed apt's hash check until the entry was evicted. Packages are now cached under their path. deb.debian.org wasn't affected, since its pool file names are unique._, which revisions, user and channel can contain, so.../revisions/R_P_Q/files/conan_package.tgzand.../revisions/R/packages/P/revisions/Q/files/conan_package.tgzshared an entry. They're now joined with/. Exploiting this needs someone who can make the upstream serve the first URL, e.g. upload rights on a shared private remote.Entries cached under the old names are still served, but only for the URL recorded with them (
cacheFilenameKeepingLegacy), so warm caches keep working and a colliding URL fetches its own file. The tests cover both collisions and the old entries.Summary by CodeRabbit