Skip to content

[WIP] Add SearchR1 environment for testing and evaluation - #88

Open
YuRuiii wants to merge 154 commits into
mll-lab-nu:mainfrom
YuRuiii:main
Open

[WIP] Add SearchR1 environment for testing and evaluation#88
YuRuiii wants to merge 154 commits into
mll-lab-nu:mainfrom
YuRuiii:main

Conversation

@YuRuiii

@YuRuiii YuRuiii commented Feb 26, 2026

Copy link
Copy Markdown

Summary by CodeRabbit

  • New Features

    • Added SearchR1 environment with retrieval-based evaluation, HotpotQA dataset preparation, and data-download utilities.
    • Added WebArena environment for browser-driven task evaluation.
    • Added a dense retrieval service (sparse and dense modes), server launch/wait scripts, and evaluation runner scripts.
    • Expanded multi-backend evaluation config with new backend entries (Claude, Gemini).
  • Documentation

    • Added comprehensive SearchR1 README and usage guidance.
  • Chores

    • Removed several FrozenLake example configs and orchestration scripts.

JamesKrW and others added 30 commits January 2, 2026 02:08
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 Updated no-concat gae algorithm, fixed bug in no-concat scripts, updated sokoban env
Removed commented export statement for RENDER_WS_URL.
Removed unnecessary export of ws render url
updated reward variance filtering based on RAGEN
When agent_data.image_data is an empty list, passing it to processor causes
transformers library to attempt accessing images[0], resulting in IndexError.
Changed to pass None instead of empty list when image_data is empty.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants