-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathModelfile.14b
More file actions
23 lines (18 loc) · 805 Bytes
/
Copy pathModelfile.14b
File metadata and controls
23 lines (18 loc) · 805 Bytes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# Modelfile for oxidant-worker-14b
# Base: Qwen2.5-Coder 14B Q5_K_M (~10 GB, fits 4× parallel on 16GB RTX 5080)
#
# Build: ollama create oxidant-worker-14b -f Modelfile.14b
# Verify: ollama run oxidant-worker-14b "fn add(a: i32, b: i32) -> i32 {"
FROM qwen2.5-coder:14b-instruct-q5_K_M
# Context window — always explicit to avoid Bug #14120 (auto-sizing OOM at NUM_PARALLEL=4)
PARAMETER num_ctx 8192
# Max output tokens — function bodies are rarely longer than this
PARAMETER num_predict 1024
# Full GPU residency — fails loudly if layers spill to CPU (better than silent slowdown)
PARAMETER num_gpu 99
# Sampling — deterministic bias for transpilation (not creative writing)
PARAMETER temperature 0.2
PARAMETER top_p 0.8
PARAMETER top_k 20
PARAMETER min_p 0.0
PARAMETER repeat_penalty 1.05