Skip to content

Add prime-rl-based RLM training harness and update the local REPL implementation. - #165

Merged
alexzhang13 merged 8 commits into
mainfrom
training_harness
May 27, 2026
Merged

alexzhang13 merged 8 commits into
mainfrom
training_harness

Conversation

@alexzhang13

@alexzhang13 alexzhang13 commented May 24, 2026 •

Copy link
Copy Markdown
Owner

Add a verifiers-compatible training harness for rlm.RLM at depth=1, designed to plug into prime-rl for end-to-end RL training. Rollouts run through a subprocess-isolated local REPL (no cloud sandboxes), with sub-LM calls proxied back to the trainer's inference server. Minor tweaks to rlm/core/rlm.py, rlm/utils/parsing.py, and rlm/utils/prompts.py so the local REPL behavior stays aligned with the training surface.

  • training/src/rlm_train/ — env, rubric, sub-LM proxy, and subprocess REPL worker
  • training/environments/oolong/ — example OOLONG long-context QA env
  • training/configs/rlm-qwen3-30b-example.toml — example RL config
  • Small alignment fixes to core REPL prompts/parsing so local and training paths match

@alexzhang13
alexzhang13 merged commit 85abec1 into main May 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant