Skip to content

feat(web): configure judge assertions - #217

Merged
ccarvalho-eng merged 1 commit into
mainfrom
ccarvalho/web-judge-assertions
Sep 5, 2026
Merged

feat(web): configure judge assertions#217
ccarvalho-eng merged 1 commit into
mainfrom
ccarvalho/web-judge-assertions

Conversation

@ccarvalho-eng

Copy link
Copy Markdown
Owner

Motivation

Suite authors could execute rubric judges but had to write assertion JSON. This adds complete dashboard authoring and inspection for built-in and custom judges while preserving the same persisted assertion contract across interfaces.

Test Plan

  • Added parser coverage for built-in and custom judge forms, default thresholds, validation, and typed evidence round trips.
  • Added LiveView coverage for creating, editing, inspecting, and preserving an unavailable judge provider.
  • Verified the full test suite, formatting, static analysis, security checks, production compilation, documentation build, dependency advisories, and Hex package dry run.

Next Steps

Repeated-run sampling controls will follow in a separate focused pull request.

@coderabbitai

coderabbitai Bot commented Sep 5, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 37 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 14aeff1d-e2d3-4721-9807-f3c45e3ec40b

📥 Commits

Reviewing files that changed from the base of the PR and between e7c1e4a and 8caa174.

📒 Files selected for processing (11)
  • README.md
  • guides/evaluations.md
  • guides/features.md
  • lib/aludel/evals/assertion_parser.ex
  • lib/aludel/web/live/suite_live/new.ex
  • lib/aludel/web/live/suite_live/new.html.heex
  • lib/aludel/web/live/suite_live/show.ex
  • lib/aludel/web/live/suite_live/show.html.heex
  • test/aludel/evals/assertion_parser_test.exs
  • test/aludel_web/live/suite_live/new_test.exs
  • test/aludel_web/live/suite_live/show_test.exs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ccarvalho-eng
ccarvalho-eng merged commit 4195c3e into main Sep 5, 2026
11 checks passed
@ccarvalho-eng
ccarvalho-eng deleted the ccarvalho/web-judge-assertions branch September 5, 2026 00:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant