Skip to content

kev-finetune: reject a fractional score index instead of truncating it - #241

Merged
jaredpalmer merged 2 commits into
mainfrom
cursor/reject-fractional-scores-873c
Oct 15, 2026
Merged

jaredpalmer merged 2 commits into
mainfrom
cursor/reject-fractional-scores-873c

Conversation

@jaredpalmer

Copy link
Copy Markdown
Owner

convert_data.py turns a score column into a level index with int(float(text)). 1.9 becomes 1, so a row that is not on the scale is trained as the level below it. Infinity raises OverflowError, and the handler only catches ValueError, so that one cell aborts the file.

This continues #239. A score label is parsed with Decimal, which keeps 1.9 and 0.99999999999999999 distinct from 1, where float rounds them. A non-finite value or a fraction is skipped and counted by reason. 1, 1.0, 2e0 and -0.0 stay integers, and level text still matches when the token is not a number. The in-range check runs on the Decimal before int(). int(Decimal("1e9999999")) would build a multi-million-digit integer. The test includes that token, and it is skipped as out of range.

The assumption is that a score index is a whole number of levels, and --score-offset only shifts that integer. A level whose text is numeric is unchanged, because a numeric token is still read as an index first.

generate_data.py already asks for an integer index, so it is left alone. Nothing in training or serving changes, so there is no parity run against main.

Test plan

  • uv run --extra serve python -m pytest tests/test_skill_scripts.py -q on this branch: 35 passed
  • Cases for 1.9, NaN, Infinity, 1e9999999 (skipped without building the integer), 1.0, 2e0, -0.0, level text, and a CSV where two bad rows are skipped and the rest of the file is written
Open in Web Open in Cursor 

aakash-kharb and others added 2 commits October 14, 2026 22:20
int(float) truncated 1.9 to 1, and Infinity raised OverflowError past the ValueError handler, so one bad cell aborted the file. Decimal still rejects those, and the range check is its own line so the skip reason is readable.

Co-authored-by: aakash-kharb <akgdsc7@gmail.com>
@jaredpalmer
jaredpalmer marked this pull request as ready for review October 15, 2026 22:40
@jaredpalmer
jaredpalmer merged commit 1f04cfa into main Oct 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants