Repository navigation
Replies: 1 comment
|
As of DSPy 2.5+, here's a comprehensive reference for MIPROv2: What MIPROv2 doesMIPROv2 is a Bayesian prompt optimizer that uses Optuna-based search to find the best combination of:
It evaluates candidates against your metric and returns the best-performing program. Basic usageimport dspy
optimizer = dspy.MIPROv2(metric=my_metric, auto="light")
optimized_program = optimizer.compile(my_program, trainset=trainset)Key parameters
|
| Parameter | Default | Description |
|---|---|---|
metric |
required | A callable metric(example, prediction) -> float that scores predictions |
auto |
"light" |
Preset controlling search budget: "light", "medium", or "heavy". Controls num_candidates and num_trials automatically |
num_candidates |
None |
Number of instruction/demo candidates to generate per predictor. Overrides auto |
max_bootstrapped_demos |
4 |
Max bootstrapped (model-generated) demos per predictor |
max_labeled_demos |
4 |
Max labeled (from trainset) demos per predictor |
prompt_model |
dspy.settings.lm |
LM used to generate instruction candidates |
task_model |
dspy.settings.lm |
LM used to evaluate candidates |
num_threads |
None |
Parallelism for evaluation |
seed |
9 |
Random seed for reproducibility |
verbose |
False |
Print optimization progress |
metric_threshold |
None |
Early-stop if metric exceeds this |
compile() parameters
| Parameter | Default | Description |
|---|---|---|
student |
required | The DSPy program to optimize |
trainset |
required | List of dspy.Example for optimization |
valset |
None |
Separate validation set (if None, splits from trainset) |
teacher |
None |
A pre-optimized program to bootstrap demos from |
num_trials |
None |
Number of Optuna trials (overrides auto) |
minibatch |
True |
Use minibatch evaluation (faster) |
minibatch_size |
35 |
Size of evaluation minibatches |
program_aware_proposer |
True |
Instruction proposer sees program structure |
data_aware_proposer |
True |
Instruction proposer sees data samples |
When to use MIPROv2
- Use MIPROv2 when you want to optimize both instructions and few-shot demos jointly. It's the most powerful optimizer in DSPy.
- Use
auto="light"for quick iteration (fewer trials, ~minutes) - Use
auto="heavy"for production optimization (many trials, ~hours) - For simpler cases where you only need good demos,
BootstrapFewShotis faster.
To your specific question about different datasets
Can I use different datasets for searching for demonstration and evaluation for optimization?
Yes — use the valset parameter in compile():
optimized = optimizer.compile(
my_program,
trainset=demo_search_data, # used for generating demo candidates
valset=evaluation_data, # used for evaluating candidates
)If you don't pass valset, MIPROv2 automatically splits trainset into train/val portions.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I am a new user of dspy. But for some new API within dspy, such as the MIPROv2. I have not found any related documents. I am not sure if I miss them. And here are some specific questions:
Thank you so much!
All reactions