Using two LLMs within the same module

[AakashKumarNain]

I didn't find any reference for this in the documentation, so asking it here. Is there a way to configure two LLMs within a single module? For example:

# We only configure the trace globally
dspy.settings.configure(trace=[])


class MyModule(dspy.Module)
    def __init__(self):
        super().__init__()
        self.llm1 = dspy.OpenAI(model='gpt-3.5-turbo-instruct', max_tokens=250)
        self.llm2 = dspy.OpenAI(model='gpt-4', max_tokens=250)

    def forward(self, x):
           with dspy.context(self.llm1):
                ...
            with dspy.context(self.llm2):
                ...

[juicesharp]

to my best knowledge the approach you have shown almost work:

with dspy.context(lm=self.lm1):
     response = dspy.ChainOfThought(Assess)

but partially the issue is module now is unserializable ..... so I ended up to keep different llms inside of global configuration.

[AakashKumarNain]

How can you have two llms in the global config? And not able to serialize with this workflow seems extremely limiting. Any workarounds? @okhat

[juicesharp]

global config - does not mean one belongs to dspy and impossible to use ... you can write your own config just to keep llm instances then use dspy.context.

# your config 
config = {
    "llm_to_ask_questions": dspy.GROQ(model='llama3-8b-8192)
    "llm_to_write_summaries": dsp.Claude(model='claude-3-sonnet-20240229'),
}

# your module
class Program(dspy.Module):
    def __init__(self):
        super().__init__()
        self.ask_question = dspy.Predict(AskQuestionSignature)
        self.write_summary = dspy.Predict(SummarizeAnswerSignature)

    def forward(self, question):
        with dspy.context(lm=config.llm_to_ask_questions): 
            first_answer = self.ask_question(question=question)
        with dspy.context(lm=config.llm_to_write_summaries): 
            return self.write_summary(text=first_answer.answer)


program = Program()
result = program(question="What is the capital of France?")
print(result.summary)

[AakashKumarNain]

Wow! I wasn't aware of this. Though this is very unintuitive, but let me check if this works as expected. Will report back. Thanks