Update ChromaDB tutorial materials for ChromaDB 1.5 and openai 3.x - #831
Open
realpython-bot wants to merge 1 commit into
Open
realpython-bot wants to merge 1 commit into
realpython-bot wants to merge 1 commit into
Conversation
- Pin all dependencies to the tested versions (Python 3.12+)
- Use configuration={"hnsw": ...} instead of metadata hnsw:space
- Fix off-by-one that dropped the last document of every batch and
read the batch size from client.get_max_batch_size()
- Use itertools.batched() instead of more-itertools
- Load the collection from its persisted embedding function config
- Polars: schema_overrides= and maintain_order=True
- Move the OpenAI code to the OpenAI() client and Responses API,
with the API key loaded from .env via python-dotenv
- Use the spaCy large model, since the 3.8 medium model prunes vectors
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supporting code for the full update of Embeddings and Vector Databases With ChromaDB (CMS update draft: Post 2390, Trello card https://trello.com/c/e5bIvapv).
Changes
chroma_utils.py: useconfiguration={"hnsw": {"space": ...}}instead of the oldmetadata={"hnsw:space": ...}, read the batch size fromclient.get_max_batch_size(), and fix the off-by-one (end_idx = batch[-1] + 1) that dropped the last document of every batch. Useitertools.batched()instead of more-itertools.car_data_etl.py:schema_overrides=(thedtypes=argument is deprecated in Polars 1.x) andmaintain_order=Trueso the printed example review is reproducible.create_car_review_collection.py: load the collection through its persisted embedding-function configuration and printcollection.count().llm_car_review_context.py: replace the removedopenai.ChatCompletioninterface with theOpenAI()client and the Responses API; loadOPENAI_API_KEYfrom.envwith python-dotenv (dropsconfig.json).word_vectors.py: useen_core_web_lg, because the 3.8 medium model prunes its vectors so that dog and cat share one vector..envinstructions.Testing
All scripts ran end to end on Python 3.14 with the pinned versions, including the full 5,870-review collection build and live OpenAI calls. The outputs match the updated tutorial draft.
🤖 Generated with Claude Code