Skip to content

Commit e43f7e5

Browse files
committed
[executorch][llm] Make Gemma Apple test deterministic
Pull Request resolved: pytorch#21403 `MultimodalRunnerTest.testGemma` asserted on generated text while inheriting the default sampling temperature of 0.8. Set the temperature to zero for both generation calls so all four TestInfra configurations use deterministic greedy decoding. This commit was authored with Claude. ghstack-source-id: 407467872 @exported-using-ghexport Differential Revision: [D113786993](https://our.internmc.facebook.com/intern/diff/D113786993/)
1 parent 1f2a6f9 commit e43f7e5

1 file changed

Lines changed: 2 additions & 0 deletions

File tree

‎extension/llm/apple/ExecuTorchLLM/__tests__/MultimodalRunnerTest.swift‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,7 @@ class MultimodalRunnerTest: XCTestCase {
123123
MultimodalInput(String(format: chatTemplate, userPrompt)),
124124
], Config {
125125
$0.sequenceLength = sequenceLength
126+
$0.temperature = 0
126127
}) { token in
127128
text += token
128129
if token == "<end_of_turn>" {
@@ -143,6 +144,7 @@ class MultimodalRunnerTest: XCTestCase {
143144
MultimodalInput(String(format: chatTemplate, userPrompt)),
144145
], Config {
145146
$0.sequenceLength = sequenceLength
147+
$0.temperature = 0
146148
}) { token in
147149
text += token
148150
if token == "<end_of_turn>" {

0 commit comments

Comments
 (0)