Skip to content

Update ChromaDB tutorial materials for ChromaDB 1.5 and openai 3.x - #831

Open
realpython-bot wants to merge 1 commit into
masterfrom
chromadb-vector-database-update
Open

realpython-bot wants to merge 1 commit into
masterfrom
chromadb-vector-database-update

Conversation

@realpython-bot

Copy link
Copy Markdown
Collaborator

Supporting code for the full update of Embeddings and Vector Databases With ChromaDB (CMS update draft: Post 2390, Trello card https://trello.com/c/e5bIvapv).

Changes

  • Pin all dependencies to the tested versions: chromadb 1.5.9, spacy 3.8.16, sentence-transformers 6.0.1, polars 1.44.2, openai 3.14.0, numpy 2.5.3, python-dotenv 1.2.3 (Python 3.12+, tested on 3.14).
  • chroma_utils.py: use configuration={"hnsw": {"space": ...}} instead of the old metadata={"hnsw:space": ...}, read the batch size from client.get_max_batch_size(), and fix the off-by-one (end_idx = batch[-1] + 1) that dropped the last document of every batch. Use itertools.batched() instead of more-itertools.
  • car_data_etl.py: schema_overrides= (the dtypes= argument is deprecated in Polars 1.x) and maintain_order=True so the printed example review is reproducible.
  • create_car_review_collection.py: load the collection through its persisted embedding-function configuration and print collection.count().
  • llm_car_review_context.py: replace the removed openai.ChatCompletion interface with the OpenAI() client and the Responses API; load OPENAI_API_KEY from .env with python-dotenv (drops config.json).
  • word_vectors.py: use en_core_web_lg, because the 3.8 medium model prunes its vectors so that dog and cat share one vector.
  • README: updated title, install steps, and .env instructions.

Testing

All scripts ran end to end on Python 3.14 with the pinned versions, including the full 5,870-review collection build and live OpenAI calls. The outputs match the updated tutorial draft.

🤖 Generated with Claude Code

- Pin all dependencies to the tested versions (Python 3.12+)
- Use configuration={"hnsw": ...} instead of metadata hnsw:space
- Fix off-by-one that dropped the last document of every batch and
  read the batch size from client.get_max_batch_size()
- Use itertools.batched() instead of more-itertools
- Load the collection from its persisted embedding function config
- Polars: schema_overrides= and maintain_order=True
- Move the OpenAI code to the OpenAI() client and Responses API,
  with the API key loaded from .env via python-dotenv
- Use the spaCy large model, since the 3.8 medium model prunes vectors

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants