Repository navigation

perf: speed up top-p sampling for large vocabularies - #1232

Merged

NorbertKlockiewicz merged 4 commits into

mainfrom

@nk/optimize-sampling

Jun 15, 2026

Merged

perf: speed up top-p sampling for large vocabularies#1232
NorbertKlockiewicz merged 4 commits into
mainfrom
@nk/optimize-sampling

Commits on Jun 15, 2026
- perf: speed up top-p sampling for large vocabularies
  
  NorbertKlockiewicz
  and
  claude
  committed
  1fad01e
- refactor: address review feedback in top-p sampler
  
  NorbertKlockiewicz
  and
  claude
  committed
  edcfa37
- refactor: accumulate top-p sums in float, not double
  
  NorbertKlockiewicz
  and
  claude
  committed
  dc4b7cc
- refactor: compute top-p in an upcast-only accumulator type
  
  NorbertKlockiewicz
  and
  claude
  committed
  777bd26