How we can use open source vector search libraries like FAISS, Annoy for RAG implementation in DSPy?

[KaifAhmad1]

Seeking advice on integrating open-source vector search libraries like FAISS or Annoy for RAG implementation. Any pointers for GitHub?

[tumma72]

Hi there, I am not sure if my question is close to yours in any way, but I am seeking advice on how to best integrate external tools into DSPy, especially for RAG. Let's say I want to download and index documents (PDFs, mostly research papers) and I want to create a vectorized database using ChromaDB or similar. Everything should be running locally on a private network using open source LLMs. Should I use the Retrive interface and build my own retrieval model? Or should I create a simple Module that would call a search method on the vector storage?

[KaifAhmad1]

Hey, @tumma72 Here is the ChromaDB docs you can start with https://docs.trychroma.com/getting-started

[tumma72]

Thanks for the pointer, but I already know how to use ChromaDB, what I wanted to know is if based on DSPy intended architecture I should:

1. Create a Retrieval model for retrieving information from ChromaDB or there is already one built;
2. Just create my own Module in which I would take a query as input, retrieve the results of the query as context and ask the Agent to respond;

What is the preferred way and what are the advantages of each approach provided it is doable?

[KaifAhmad1]

For integrating ChromaDB into the DSPy architecture, you have two main options:

1. Create a custom Retrieval model that interfaces with ChromaDB to retrieve relevant context for a given query. This follows the DSPy architecture and promotes reusability, but requires more effort.

2. Create a custom Module that encapsulates the logic for querying ChromaDB and retrieving relevant context, then passing it to the Agent. This is a simpler approach but may be less reusable across different contexts.

The preferred approach depends on whether you need the ChromaDB retrieval mechanism to be reusable across multiple modules or contexts (favor Retrieval model), or if it's specific to a particular use case (favor custom Module).

[KaifAhmad1]

Curious about your PDF loading process with DSPy, @tumma72. Which tools are you using?

[tumma72]

At the moment I am experimenting with pypdf which does a good job albeit being slow...

[mkotlarz]

You could also use the retriever implementation from Langchain which has broader support, then simply convert the Langchain Document object to DSPY passages

[Deanozk]

Did anyone get FAISS working with DSPy?

[sandu345]

I want an advice from you. Does anyone has an idea about how to use own retriever with DSPy?