How does dspy work under the hood.

[psych0v0yager]

I have watched the Connor Shorten videos on dspy and read the Building Blocks and parts of the Deep Dive section of the docs.

Still, I am curious how exactly dspy works under the hood. How does dspy go from text in -> text out to optimizers and modules.

I browsed through the filetree and noticed the dsp and dspy folders. There are 2 different version of ChainOfThought, Predict, and multiple other files between dsp and dspy. I read somewhere this is due to scope creep, but what is the main function of the dsp and dspy folder.

Furthermore the docs say BootstrapFewShot gets the LLM to self generate new examples. How exactly is this done? What prompt is passed to the LLM optimizer to instruct it to generate more examples based on the Modules?

Lastly the docs mention "A DSPy module has learnable parameters". If I understand correctly, the learnable parameters are the few shot examples generated by the optimizer + the actual instructions ordering the LLM to "think step by step" (in the case of the CoT module)

Are there resources I can go to that deconstruct how the library works at the lowest level? If so, where can I find them.

Thanks