Repository navigation
Conversation
Wrap the interpolation coordinate at half the rotary dimension so both coordinates in a pair use the same angle. Preserve the first-half frequencies and existing correction bounds. Add paired-frequency and compiled-kernel regressions for LLVM, CUDA, and Metal, including partial rotation and equal beta bounds.
Check each known frequency with an independent basis row. Use position 1 for the frequency-1 pair and keep position 4096 for the lower frequencies, so the active angles stay within the accurate range of CUDA fast sin/cos. Keep the comparison tolerances, default compiler options, and long-position norm tests unchanged.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
rope_freq_yarnwraps the base frequency across rotary halves but uses the full coordinate for the interpolation mask. This can give the two coordinates of a rotary pair different angles. I found this while checking Ministral 3 3B Reasoning in WebLLM.For a 128-wide head with theta 1e6, factor 16, original context 16384 and betas 32/1, coordinates 0 and 64 use frequencies 1 and 1/16. At position 1, rotating a unit vector changes its squared norm from 1 to 0.2958277. With this fix, it is 0.99999994 in float32.
Apply
d % (d_range // 2)to the mask so paired coordinates use the same weight, matching the duplicated half-frequency table in the YaRN reference. The first-half frequencies and correction bounds remain unchanged, preserving the earlier fixes in #18576 and #18661.#20104 includes the same mask fix as part of broader KV cache and attention changes. This PR keeps the fix separate and adds regression tests so it can be reviewed independently.
The tests cover paired frequencies, norm preservation, partial and disabled rotation, and known values across the interpolation ramp. They check float16/float32 and a non-power-of-two head width.
On an Apple M4 Pro with LLVM and Metal: 58 passed, 26 skipped because CUDA was unavailable. The original source fails 22 of these cases. Changed-file pre-commit checks pass.
All 26 CUDA cases passed in GPU CI build 2 on commit
dfed4cb.