Repository navigation
fix(storage): optimize CRC32C#update using Slicing-by-8 algorithm - #9711
ddelgrosso1 wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request optimizes the CRC32C calculation by implementing a slicing-by-8 lookup table algorithm in the update method, processing 8 bytes at a time, and adds extensive unit tests to verify correctness across various scenarios (RFC test vectors, unaligned views, remainder lengths, etc.). Feedback suggests optimizing the loop condition in the hot path by pre-calculating the loop limit to avoid addition in each iteration.
|
I ran a local benchmark (not against live GCS) stubbing the HTTP layer ( In-memory stream + CRC32C processing speed went from ~216 MiB/s to ~1,070 MiB/s (~5x faster), cutting time per GiB from ~4.7s down to ~0.95s. |
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 🦕