Skip to content

Commit 13eda2b

Browse files
authored
document HF TEI sparse embedders + grpc support (#630)
1 parent a73748d commit 13eda2b

1 file changed

Lines changed: 31 additions & 6 deletions

File tree

‎integrations/huggingface-api.md‎

Lines changed: 31 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,11 @@ toc: true
2222
- [Overview](#overview)
2323
- [Installation](#installation)
2424
- [Usage](#usage)
25+
- [Components](#components)
26+
- [Chat Generation](#chat-generation)
27+
- [Dense Embedding Models](#dense-embedding-models)
28+
- [Sparse Embedding Models](#sparse-embedding-models)
29+
- [Ranking Models](#ranking-models)
2530

2631
## Overview
2732

@@ -49,16 +54,17 @@ Unless you are using a self-hosted TGI/TEI server, set your Hugging Face token a
4954

5055
This integration provides several components to interact with Hugging Face APIs:
5156
- [`HuggingFaceAPIChatGenerator`](https://docs.haystack.deepset.ai/docs/huggingfaceapichatgenerator): chat generation with LLMs.
52-
- [`HuggingFaceAPITextEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceapitextembedder): creates an embedding for text (used in query/RAG pipelines).
53-
- [`HuggingFaceAPIDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceapidocumentembedder): enriches documents with embeddings (used in indexing pipelines).
57+
- [`HuggingFaceAPITextEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceapitextembedder): creates a dense embedding for text (used in query/RAG pipelines).
58+
- [`HuggingFaceAPIDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceapidocumentembedder): enriches documents with dense embeddings (used in indexing pipelines).
59+
- [`HuggingFaceTEISparseTextEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceteisparsetextembedder): creates a sparse embedding for text using a TEI endpoint (used in query pipelines).
60+
- [`HuggingFaceTEISparseDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceteisparsedocumentembedder): enriches documents with sparse embeddings using a TEI endpoint (used in indexing pipelines).
5461
- [`HuggingFaceTEIRanker`](https://docs.haystack.deepset.ai/docs/huggingfaceteiranker): ranks documents based on their similarity to the query, using a TEI endpoint.
5562

5663
### Chat Generation
5764

5865
Use [`HuggingFaceAPIChatGenerator`](https://docs.haystack.deepset.ai/docs/huggingfaceapichatgenerator) with the Serverless Inference API (Inference Providers):
5966

6067
```python
61-
from haystack.dataclasses import ChatMessage
6268
from haystack_integrations.components.generators.huggingface_api import HuggingFaceAPIChatGenerator
6369

6470
generator = HuggingFaceAPIChatGenerator(
@@ -79,7 +85,7 @@ generator = HuggingFaceAPIChatGenerator(
7985
)
8086
```
8187

82-
### Embedding Models
88+
### Dense Embedding Models
8389

8490
To create semantic embeddings for documents, use [`HuggingFaceAPIDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceapidocumentembedder) in your indexing pipeline. For generating embeddings for queries, use [`HuggingFaceAPITextEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceapitextembedder).
8591

@@ -95,7 +101,7 @@ print(text_embedder.run("I love pizza!"))
95101
# {'embedding': [0.017020374536514282, -0.023255806416273117, ...]}
96102
```
97103

98-
Both embedders also work with a self-hosted TEI server:
104+
Both embedders also work with a self-hosted TEI server and support HTTP or gRPC:
99105

100106
```python
101107
text_embedder = HuggingFaceAPITextEmbedder(
@@ -104,9 +110,28 @@ text_embedder = HuggingFaceAPITextEmbedder(
104110
)
105111
```
106112

113+
### Sparse Embedding Models
114+
115+
To create sparse embeddings for queries, use [`HuggingFaceTEISparseTextEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceteisparsetextembedder). For documents, use [`HuggingFaceTEISparseDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/huggingfaceteisparsedocumentembedder), which stores the embeddings in each document's `sparse_embedding` field.
116+
117+
Both components support HTTP or gRPC and require a TEI server running a compatible sparse embedding model with SPLADE pooling.
118+
119+
```python
120+
from haystack_integrations.components.embedders.huggingface_api import (
121+
HuggingFaceTEISparseTextEmbedder,
122+
)
123+
124+
embedder = HuggingFaceTEISparseTextEmbedder(
125+
api_base_url="http://localhost:8080",
126+
)
127+
result = embedder.run(text="Who lives in Berlin?")
128+
print(result["sparse_embedding"])
129+
# SparseEmbedding(indices=[1755, 2141, 2160, ...], values=[0.062120, 0.214147, 0.183626, ...])
130+
```
131+
107132
### Ranking Models
108133

109-
Use [`HuggingFaceTEIRanker`](https://docs.haystack.deepset.ai/docs/huggingfaceteiranker) to rank documents with a reranking model served by a TEI endpoint:
134+
Use [`HuggingFaceTEIRanker`](https://docs.haystack.deepset.ai/docs/huggingfaceteiranker) to rank documents with a reranking model served by a TEI endpoint. It supports HTTP or gRPC:
110135

111136
```python
112137
from haystack import Document

0 commit comments

Comments
 (0)