Skip to content

Commit 37f46b8

Browse files
authored
benchmarks: add system prompt to baseline arm so it doesn't ramble (closes #126) (#128)
Co-authored-by: Fato07 <fato07@users.noreply.github.com>
1 parent e782790 commit 37f46b8

1 file changed

Lines changed: 6 additions & 2 deletions

File tree

‎benchmarks/arms/baseline.js‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,6 @@
1-
// Baseline arm: no skill, just the task.
2-
module.exports = ({ vars }) => [{ role: 'user', content: vars.task }];
1+
// Baseline arm: no skill, with a one-line system prompt so the model doesn't ramble.
2+
const system = 'Provide just one example for any given task, and no commentary or usage examples.';
3+
module.exports = ({ vars }) => [
4+
{ role: 'system', content: system },
5+
{ role: 'user', content: vars.task },
6+
];

0 commit comments

Comments
 (0)