Repository navigation
Commit bfaf911
Benchmark naive Clairic loops that point instead of choosing
Clairic, Cloudflare's extraction model, answers text fields as spans of
the input, so the loop needs no candidate actions: it asks for the next
element ref, the value to type, a direct URL, and a quote proving
completion, and code acts by the element's role. In one interleaved
batch the naive Jev option menu verified 110/150, naive Clairic 42/150,
Clairic with a screenshot 30/150, and Clairic with Clef judging
completion 53/150. Clairic mostly quoted the aim itself as completion
evidence, and without that it did not know when to stop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>1 parent b4ae108 commit bfaf911
6 files changed
Lines changed: 839 additions & 204 deletions
0 commit comments