Repository navigation
fix: vocab consolidation on anthropic-sdk v1, and report failed evaluations as failed - #591
Merged
Merged
Conversation
…c_sampling_kwargs anthropic-sdk v1.0 removed temperature from messages.create(). Every Anthropic call site in ai_providers.py already goes through the helper, but the shared dispatch in llm_utils still passed temperature top-level, so every vocabulary-consolidation evaluation raised TypeError. Refs #590
A merge evaluation whose LLM call or response parsing errored was returned as should_merge=False and counted under Rejected, so a dead provider read as 'the AI judged nothing should merge'. MergeDecision now carries failed=True for those; consolidation (dry-run and live) collects them in a separate 'failed' list that the route, worker and CLI surface. The CLI exits non-zero when every evaluation failed. Tests pin both #590 halves: call_llm_sync sends temperature via extra_body (and drops it for models that reject sampling params), and llm_evaluate_merge marks call and parse errors as failed while a real rejection is not. Refs #590
…d model The key-validation ping called messages.create on the hardcoded claude-3-5-sonnet-20241022, which 404s once the model is retired and rejects valid keys. Match AnthropicProvider.validate_api_key (ADR-800). Refs #590
3 of 4 tasks
…_create Vocabulary consolidation now gets the same per-model adaptation as the other Anthropic call sites: a model that rejects sampling params is retried without them instead of failing the merge evaluation.
This was referenced Oct 2, 2026
Shared agent memory: two-author ingest of the same events merges by framing, drops disagreement
#595
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #590.
Coordination: #586 (`_anthropic_create`, per-model request adaptation) doesn't conflict textually. Whichever lands second should switch `call_llm_sync` to `_anthropic_create` so consolidation gets the same adaptation. Agreed with the session landing #586.
Gates