Skip to content

Supporting huggingface-tei version 1.8.1 #5279

Closed

Description

@etiennekintzler

The current version of sagemaker (2.251.1) does not support huggingface-tei version 1.8.1 but only the version 1.8.0.

Hugging face TEI version 1.8.1 (https://github.com/huggingface/text-embeddings-inference/releases/tag/v1.8.1) was released 2 weeks ago and provides support for Google's new embedding model, Gemma 300m.

Currently the sagemaker deployment of this model (https://huggingface.co/google/embeddinggemma-300m?sagemaker_deploy=true) is failing because hugging-face tei 1.8.0 does not take into account this model architecture:

"Could not start Candle backend: Could not start backend: Model is not supported\n\nCaused by:\n    unknown variant `gemma3_text`, expected one of `bert`, `xlm-roberta`, `camembert`, `roberta`, `distilbert`, `nomic_bert`, `mistral`, `gte`, `new`, `qwen2`, `qwen3`, `mpnet`, `modernbert` at line 45 column 29"

Activity

  1. sirutBuasai commented on Oct 10, 2025

    @sirutBuasai
    Contributor

    TEI 1.8.2 has been released and is now available on SageMaker PySDK
    Check here for dockerfile

    TEI 1.8.2 supports Gemma 300m and such TEI 1.8.1 will be skipped

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions