You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 25fb705
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: docs/docs/04-typescript-api/01-natural-language-processing/LLMModule.md
+17-6Lines changed: 17 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -118,12 +118,12 @@ Model presets expose an optional `generationConfig` that `LLMModule.fromModelNam
118
118
119
119
## Vision-Language Models (VLM)
120
120
121
-
Some models support multimodal input — textand images together. To use them, pass `capabilities` in the model object when calling [`fromModelName`](../../06-api-reference/classes/LLMModule.md#frommodelname):
121
+
Some models support multimodal input — text, images and/or audio together. To use them, pass `capabilities` in the model object when calling [`fromModelName`](../../06-api-reference/classes/LLMModule.md#frommodelname):
Once loaded, pass `imagePath` to [`sendMessage`](../../06-api-reference/classes/LLMModule.md#sendmessage):
144
+
Once loaded, pass `imagePath`Once loaded, pass `imagePath` or `audioBuffer` to [`sendMessage`](../../06-api-reference/classes/LLMModule.md#sendmessage):
145
145
146
146
```typescript
147
147
const response =awaitllm.sendMessage('What is in this image?', {
148
148
imagePath: '/path/to/image.jpg',
149
149
});
150
+
// or
151
+
const response =awaitllm.sendMessage('What can you hear?', {
152
+
audioBuffer: audioRecording, //expected as waveform 16kHz
153
+
});
150
154
```
151
155
152
156
Or use [`generate`](../../06-api-reference/classes/LLMModule.md#generate) with `mediaPath` on the message:
0 commit comments