fix(provider): add Cheaper Inference as a built-in provider - #3893
Open
aiapienthusiast wants to merge 1 commit into
Open
aiapienthusiast wants to merge 1 commit into
aiapienthusiast wants to merge 1 commit into
Conversation
Cheaper Inference is an OpenAI-compatible gateway, so the entry declares "response_type": "OpenAI" and reuses the existing OpenAI request path. Models are discovered from the provider's /models endpoint rather than pinned, as in tailcallhq#3623. - provider.json: the cheaper_inference entry (api key var, chat-completions URL, models URL, api_key auth) - provider.rs: ProviderId::CHEAPER_INFERENCE, built_in_providers(), the FromStr arm and a display_name arm for "Cheaper Inference", plus three tests - provider_repo.rs: test_cheaper_inference_config, which pins the Models::Url variant - README.md: the env-var block, next to the other providers cargo test --lib -p forge_domain: 618 passed, 0 failed. cargo test --lib -p forge_repo: 334 passed, 0 failed, 1 ignored (pre-existing). Both on stable 1.95, because the pinned 1.97 toolchain cannot be downloaded here; the added code adds no imports and is wrapped to 80 columns, but please re-run cargo fmt on the pinned toolchain.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Cheaper Inference as a built-in provider.
Cheaper Inference is an OpenAI-compatible gateway. One endpoint and one API key reach models from several upstream providers. The entry declares
"response_type": "OpenAI"and uses the existing OpenAI request path, so this change needs no new response handling.The change follows #3623 (
fix(provider): add OrcaRouter as a built-in provider), which added OrcaRouter in the same way.Provider details
The repository has a provider-integration issue template that asks for these details:
CHEAPERINFERENCE_API_KEYhttps://api.cheaperinference.com/v1/chat/completionshttps://api.cheaperinference.com/v1/modelsChanges
crates/forge_repo/src/provider/provider.jsoncheaper_inferenceentry: API key variable,OpenAIresponse type, chat-completions URL, models URL,api_keyauth.crates/forge_domain/src/provider.rsProviderId::CHEAPER_INFERENCE, added tobuilt_in_providers()and to theFromStrmatch. Plus adisplay_namearm, because the default conversion givesCheaperInferenceand the display name isCheaper Inference. Three tests mirrortest_orca_router_*. The new id joins the two shared assertion lists.crates/forge_repo/src/provider/provider_repo.rstest_cheaper_inference_config, which mirrorstest_orca_router_config.README.md<details>sections.The model list stays dynamic
provider.jsondoes not enumerate the models. The entry pointsmodelsat the provider's/modelsendpoint. The catalogue and the per-token pricing are discovered at runtime, not pinned in the repository.dto::openai::Pricingalready reads that pricing and treats it as optional.#3623 did the same.
test_cheaper_inference_configasserts theModels::Urlvariant, so a later switch to a hardcoded list fails loudly.What this PR does not do
This PR does not touch
crates/forge_app/src/dto/openai/model.rsand adds no/modelsfixture.In #3623 that part was a schema relaxation, which OrcaRouter's response shape made necessary. Those fields are already optional on
main. This PR changes no deserialisation. No response from this provider is captured, so a fixture would be invented, not recorded.Testing
Run against
7ef8386acaa9564a76526828c53d430e1696730a, the currentmain:cargo test --lib -p forge_domain— 618 passed, 0 failed, with the three newtest_cheaper_inference_*casescargo test --lib -p forge_repo— 334 passed, 0 failed, 1 ignored, with the newtest_cheaper_inference_configcargo clippy -p forge_domain -p forge_repo --lib --all-targets— clean, 0 warningsrustfmt --checkon the two touched Rust files — no diffToolchain limit
The preparation environment cannot install the toolchain that
rust-toolchain.tomlpins (1.97), becausestatic.rust-lang.orgis unreachable there. The checks above ran on stable 1.95. Two points need a maintainer's attention:rustfmtran without the nightly-only options in.rustfmt.toml. Stable rustfmt ignores them with a warning. The new code adds no imports and wraps to 80 columns, which is what those options govern. Please re-runcargo fmton the pinned toolchain.cargo clippyran clean on the two crates, but on the 1.95 lint set, and not across the whole workspace.Not tested through forge
A direct check with a real key confirmed the endpoint.
GET /v1/modelsreturns the catalogue.POST /v1/chat/completionsanswers. The response carries the bare model id that was sent.A reviewer should confirm the path through forge: the provider resolves from
CHEAPERINFERENCE_API_KEY, and a completion succeeds from the app.