Skip to content

Pull requests: unslothai/llama.cpp

New pull request New
43 Open 172 Closed
43 Open 172 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

glm5next: avoid soft_max gridDim.y overflow in the indexer
#214 opened Sep 11, 2026 by AIalliAI Loading…
1
Pin unslothai/llama.cpp#210 (server-side preemption and exact concurrency) at afd3248b82
#211 opened Sep 9, 2026 by danielhanchen Member Loading…
5
server: preemption notices, asynchronous parks and exact concurrency (pin branch on b10869)
#210 opened Sep 9, 2026 by danielhanchen Member Loading…
32
rpc: do not abort on shutdown when a teardown round trip fails
#203 opened Sep 6, 2026 by danielhanchen Member Loading…
7
server: do not abort a completion when the model emits invalid UTF-8
#202 opened Sep 6, 2026 by danielhanchen Member Loading…
19
server: reuse checkpoint state buffers from a bounded pool
#201 opened Sep 6, 2026 by danielhanchen Member • Draft
17
server: remove two host side stalls in the decode loop at high concurrency
#200 opened Sep 6, 2026 by danielhanchen Member • Draft
14
rpc: cut the protocol cost of uploading weights to a remote server
#199 opened Sep 6, 2026 by danielhanchen Member • Draft
13
CUDA: MMQ tile configuration and FP4 accumulation for GB10 (sm_121)
#198 opened Sep 6, 2026 by danielhanchen Member • Draft
9
server: preemption notices, asynchronous parks and exact concurrency together
#197 opened Sep 6, 2026 by danielhanchen Member • Draft
47
rpc: direct server-to-server tensor transfer for a layer split
#196 opened Sep 6, 2026 by danielhanchen Member • Draft
14
convert: support compressed-tensors mixed-precision NVFP4 checkpoints
#195 opened Sep 6, 2026 by danielhanchen Member • Draft
7
cuda: opt-in exact concurrency, byte-identical output regardless of neighbours or parks
#194 opened Sep 5, 2026 by danielhanchen Member • Draft
30
rpc: one tensor read per decode step and asynchronous split copies
#193 opened Sep 5, 2026 by danielhanchen Member • Draft
42
server: take the park and restore copies off the decode loop
#192 opened Sep 5, 2026 by danielhanchen Member • Draft
24
trace: event tracer for the RPC backend, rpc-server and llama-server
#191 opened Sep 5, 2026 by danielhanchen Member • Draft
46
server: tell a streaming client when its slot is parked and restored
#190 opened Sep 5, 2026 by danielhanchen Member • Draft
19
server: priority-aware preemption with an explicit slot park and unpark API
#189 opened Sep 5, 2026 by danielhanchen Member • Draft
1
server: park unified requests with proactive KV watermarks
#188 opened Sep 5, 2026 by danielhanchen Member • Draft
2
server: --pipeline-groups, run the slots over several contexts of one model
#187 opened Sep 5, 2026 by danielhanchen Member Loading…
52
server: one preemption path, swap under --preempt-ram and recompute above it
#186 opened Sep 5, 2026 by danielhanchen Member • Draft
server: KV preemption by recompute, a zero host RAM alternative to #184
#185 opened Sep 5, 2026 by danielhanchen Member • Draft
2
server: preempt a slot instead of ending every conversation when the KV pool fills
#184 opened Sep 5, 2026 by danielhanchen Member Loading…
40
server: on a full KV cache, terminate one slot instead of every slot
#183 opened Sep 4, 2026 by danielhanchen Member Loading…
4
server: shift the speculative batch indices on the sub-batch retry
#182 opened Sep 4, 2026 by danielhanchen Member Loading…
5
ProTip! Find all pull requests that aren't related to any open issues with -linked:issue.