Skip to content

Commit 0c8a12d

Browse files
committed
Add Grok Voice 3 support and default for new x.ai configurations
1 parent 1d76d9d commit 0c8a12d

5 files changed

Lines changed: 36 additions & 17 deletions

File tree

‎README.md‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,7 @@ A multimodal AI system for the Pepper robot powered by advanced speech-to-speech
155155
- **Real-time Voice Chat** - Natural conversations powered by speech-to-speech (S2S) AI models:
156156
- **OpenAI** - gpt-realtime-2.1, gpt-realtime-2.1-mini, gpt-realtime-2 (GPT-5-class reasoning models with configurable effort levels), gpt-realtime-1.5, gpt-realtime, gpt-realtime-mini, gpt-4o-realtime-preview, gpt-4o-mini-realtime-preview
157157
- **Azure OpenAI** - Same models with network-level isolation and customer-managed encryption for enterprise compliance
158-
- **x.ai** - grok-voice-think-fast-2.0 (default), grok-voice-think-fast-1.0, grok-voice-fast-1.0 via the Voice Agent API with native web/X search
158+
- **x.ai** - grok-voice-3.0 (default for new configurations), grok-voice-think-fast-2.0, grok-voice-think-fast-1.0, grok-voice-fast-1.0 via the Voice Agent API with native web/X search
159159
- **Google** - gemini-3.8-live (default for new configurations), gemini-3.1-flash-live-preview, gemini-2.5-flash-native-audio-preview-12-2025, gemini-2.0-flash-live-001 via the Gemini Live API with native audio and 30 voices
160160
- **Audio Input Modes** - Two optimized modes for voice capture:
161161
- **Direct Audio** - Low-latency streaming with native voice activity detection
@@ -519,7 +519,8 @@ This allows the Android app to automatically start the face recognition server v
519519
```
520520

521521
**Unique Features:**
522-
- **Model Selection**: Defaults to `grok-voice-think-fast-2.0`; `grok-voice-think-fast-1.0` and `grok-voice-fast-1.0` remain selectable in settings
522+
- **Model Selection**: Defaults to `grok-voice-3.0` for new configurations; existing saved selections are preserved. `grok-voice-think-fast-2.0`, `grok-voice-think-fast-1.0`, and `grok-voice-fast-1.0` remain selectable for compatibility with existing configurations. Availability of these legacy IDs depends on x.ai.
523+
- **Grok Voice 3**: Uses the version-pinned `grok-voice-3.0` ID rather than the automatically changing `grok-voice-latest` alias. The WebSocket endpoint, session parameters and events are unchanged; see the [official Speech to Speech documentation](https://docs.x.ai/developers/model-capabilities/audio/speech-to-speech). Reasoning uses the API default; no new reasoning control is added in the app.
523524
- **Explicit Realtime Model URL**: Uses `wss://api.x.ai/v1/realtime?model=<selected-model>`
524525
- **Native Web Search**: Built-in web search without Tavily API (configurable in settings)
525526
- **Native X Search**: Search posts on X/Twitter in real-time (configurable in settings)
@@ -685,7 +686,7 @@ You can switch between different AI providers in the settings:
685686

686687
- **OpenAI Direct** (Realtime API): Supports all eight models (`gpt-realtime-2.1`, `gpt-realtime-2.1-mini`, `gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime`, `gpt-realtime-mini`, `gpt-4o-realtime-preview`, `gpt-4o-mini-realtime-preview`) directly from OpenAI with 10 voices. The reasoning models (`gpt-realtime-2.1`, `gpt-realtime-2.1-mini`, `gpt-realtime-2`) additionally expose a **Reasoning Effort** setting (`minimal`/`low`/`medium`/`high`/`xhigh`) to trade off latency vs. reasoning depth.
687688
- **Azure OpenAI** (Realtime API): Supports all eight models with your Azure deployment, network-level isolation, and customer-managed encryption for enterprise compliance
688-
- **x.ai Grok** (Grok Voice Agent API): `grok-voice-think-fast-2.0` by default, with `grok-voice-think-fast-1.0` and `grok-voice-fast-1.0` also selectable; native web/X search capabilities, configurable server VAD, and 5 voices (`eve`, `ara`, `rex`, `sal`, `leo`)
689+
- **x.ai Grok** (Grok Voice Agent API): `grok-voice-3.0` by default for new configurations, with `grok-voice-think-fast-2.0`, `grok-voice-think-fast-1.0` and `grok-voice-fast-1.0` also selectable; native web/X search capabilities, configurable server VAD, and 5 voices exposed in the app (`eve`, `ara`, `rex`, `sal`, `leo`)
689690
- **Google Gemini** (Live API): Native audio model with end-to-end speech, configurable VAD, thinking budget, and 30 voices
690691

691692
**Note**: Changing the API provider automatically updates available models and voices. Model/voice changes restart the session automatically.
@@ -1488,7 +1489,7 @@ Robot: "Happy Birthday! 🎂"
14881489
```
14891490

14901491
**Key Flow Characteristics:**
1491-
- **Model Flexibility**: Supports OpenAI Realtime models, Google Gemini Live models, and x.ai `grok-voice-think-fast-2.0` / `grok-voice-think-fast-1.0` / `grok-voice-fast-1.0`
1492+
- **Model Flexibility**: Supports OpenAI Realtime models, Google Gemini Live models, and x.ai `grok-voice-3.0` / `grok-voice-think-fast-2.0` / `grok-voice-think-fast-1.0` / `grok-voice-fast-1.0`
14921493
- **Dual Audio Input**: Realtime API (simple) or Azure Speech (dialect quality)
14931494
- **Server-side VAD**: Realtime API handles turn detection automatically
14941495
- **Conditional Tool Calls**: AI decides when tools are needed (not every response uses tools)
@@ -1712,7 +1713,7 @@ app/src/
17121713
- Check your OpenAI, Azure OpenAI, x.ai, or Google API key and endpoint
17131714
- Verify internet connectivity
17141715
- For Azure: Ensure the deployment name matches your Azure setup
1715-
- For x.ai: Ensure `XAI_API_KEY` is set and the selected model is `grok-voice-think-fast-2.0`, `grok-voice-think-fast-1.0`, or `grok-voice-fast-1.0`
1716+
- For x.ai: Ensure `XAI_API_KEY` is set and select `grok-voice-3.0`. If a legacy model selection is rejected by the API, switch to `grok-voice-3.0` in settings.
17161717

17171718
#### "Vision analysis not available"
17181719
- Vision works automatically with `gpt-realtime-2.1`, `gpt-realtime-2.1-mini`, `gpt-realtime-2`, `gpt-realtime-1.5`, `gpt-realtime`, and `gpt-realtime-mini` models (built-in)
@@ -1816,7 +1817,7 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
18161817
## 🙏 Acknowledgments
18171818

18181819
- **OpenAI** - Realtime API
1819-
- **x.ai** - Grok Voice Agent API (`grok-voice-think-fast-2.0`, `grok-voice-think-fast-1.0`, `grok-voice-fast-1.0`)
1820+
- **x.ai** - Grok Voice Agent API (`grok-voice-3.0`, `grok-voice-think-fast-2.0`, `grok-voice-think-fast-1.0`, `grok-voice-fast-1.0`)
18201821
- **SoftBank Robotics** - Pepper robot platform
18211822
- **Microsoft** - Azure Speech and OpenAI services
18221823
- **Groq** - Alternative vision analysis provider

‎app/src/main/java/ch/fhnw/pepper_realtime/manager/SettingsRepository.kt‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ import ch.fhnw.pepper_realtime.R
88
import ch.fhnw.pepper_realtime.network.RealtimeApiProvider
99
import ch.fhnw.pepper_realtime.network.GEMINI_38_LIVE_MODEL
1010
import ch.fhnw.pepper_realtime.network.XAI_LEGACY_MODEL_LABEL
11-
import ch.fhnw.pepper_realtime.network.XAI_THINK_FAST_MODEL
11+
import ch.fhnw.pepper_realtime.network.XAI_DEFAULT_MODEL
1212
import ch.fhnw.pepper_realtime.tools.ToolRegistry
1313
import ch.fhnw.pepper_realtime.ui.settings.LanguageOption
1414
import javax.inject.Inject
@@ -38,7 +38,7 @@ class SettingsRepository @Inject constructor(
3838
var model: String
3939
get() {
4040
val defaultModel = when (apiProviderEnum) {
41-
RealtimeApiProvider.XAI -> XAI_THINK_FAST_MODEL
41+
RealtimeApiProvider.XAI -> XAI_DEFAULT_MODEL
4242
RealtimeApiProvider.GOOGLE_GEMINI -> GEMINI_38_LIVE_MODEL
4343
else -> context.getString(R.string.openai_default_model)
4444
}
@@ -47,7 +47,7 @@ class SettingsRepository @Inject constructor(
4747
?: settings.getString(KEY_MODEL, defaultModel)
4848
?: defaultModel
4949
return if (apiProviderEnum == RealtimeApiProvider.XAI && savedModel == XAI_LEGACY_MODEL_LABEL) {
50-
XAI_THINK_FAST_MODEL
50+
XAI_DEFAULT_MODEL
5151
} else {
5252
savedModel
5353
}

‎app/src/main/java/ch/fhnw/pepper_realtime/network/RealtimeApiProvider.kt‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@ import android.util.Log
44

55
const val XAI_THINK_FAST_2_MODEL = "grok-voice-think-fast-2.0"
66
const val XAI_THINK_FAST_1_MODEL = "grok-voice-think-fast-1.0"
7-
const val XAI_THINK_FAST_MODEL = XAI_THINK_FAST_2_MODEL
7+
const val XAI_VOICE_3_MODEL = "grok-voice-3.0"
8+
const val XAI_DEFAULT_MODEL = XAI_VOICE_3_MODEL
89
const val XAI_FAST_MODEL = "grok-voice-fast-1.0"
910
const val XAI_LEGACY_MODEL_LABEL = "Grok Voice Agent"
1011
const val GEMINI_38_LIVE_MODEL = "models/gemini-3.8-live"
@@ -19,7 +20,7 @@ enum class RealtimeApiProvider(
1920
) {
2021
AZURE_OPENAI("Azure OpenAI", "gpt-4o-realtime-preview"),
2122
OPENAI_DIRECT("OpenAI Direct", "gpt-realtime-1.5"),
22-
XAI("x.ai Grok", XAI_THINK_FAST_MODEL),
23+
XAI("x.ai Grok", XAI_DEFAULT_MODEL),
2324
// Google Live API requires models/ prefix for BidiGenerateContent
2425
GOOGLE_GEMINI("Google Gemini", GEMINI_38_LIVE_MODEL);
2526

@@ -36,7 +37,7 @@ enum class RealtimeApiProvider(
3637
fun getWebSocketUrl(azureEndpoint: String?, model: String?, googleApiKey: String? = null): String {
3738
val requestedModel = if (!model.isNullOrBlank()) model.trim() else modelName
3839
val actualModel = if (this == XAI && requestedModel == XAI_LEGACY_MODEL_LABEL) {
39-
XAI_THINK_FAST_MODEL
40+
XAI_DEFAULT_MODEL
4041
} else {
4142
requestedModel
4243
}

‎app/src/main/java/ch/fhnw/pepper_realtime/ui/compose/settings/SettingsScreen.kt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ import ch.fhnw.pepper_realtime.manager.SettingsRepository
1616
import ch.fhnw.pepper_realtime.network.RealtimeApiProvider
1717
import ch.fhnw.pepper_realtime.network.GEMINI_38_LIVE_MODEL
1818
import ch.fhnw.pepper_realtime.network.XAI_FAST_MODEL
19+
import ch.fhnw.pepper_realtime.network.XAI_VOICE_3_MODEL
1920
import ch.fhnw.pepper_realtime.network.XAI_THINK_FAST_1_MODEL
2021
import ch.fhnw.pepper_realtime.network.XAI_THINK_FAST_2_MODEL
2122
import ch.fhnw.pepper_realtime.tools.ToolRegistry
@@ -70,6 +71,7 @@ fun SettingsScreen(
7071
"gpt-4o-mini-realtime-preview"
7172
)
7273
val xaiModels = listOf(
74+
XAI_VOICE_3_MODEL,
7375
XAI_THINK_FAST_2_MODEL,
7476
XAI_THINK_FAST_1_MODEL,
7577
XAI_FAST_MODEL

‎app/src/test/java/ch/fhnw/pepper_realtime/network/RealtimeApiProviderTest.kt‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,27 +6,42 @@ import org.junit.Test
66
class RealtimeApiProviderTest {
77

88
@Test
9-
fun `xai websocket url defaults to think fast 2 model`() {
9+
fun `xai explicit voice 3 and blank selection use voice 3`() {
10+
listOf(XAI_VOICE_3_MODEL, "", " ").forEach { model ->
11+
assertEquals("wss://api.x.ai/v1/realtime?model=grok-voice-3.0",
12+
RealtimeApiProvider.XAI.getWebSocketUrl(null, model))
13+
}
14+
assertEquals(XAI_VOICE_3_MODEL, XAI_DEFAULT_MODEL)
15+
}
16+
17+
@Test
18+
fun `xai saved think fast 2 selection remains unchanged`() {
19+
assertEquals("wss://api.x.ai/v1/realtime?model=grok-voice-think-fast-2.0",
20+
RealtimeApiProvider.XAI.getWebSocketUrl(null, XAI_THINK_FAST_2_MODEL))
21+
}
22+
23+
@Test
24+
fun `xai websocket url defaults to voice 3 model`() {
1025
val url = RealtimeApiProvider.XAI.getWebSocketUrl(
1126
azureEndpoint = null,
12-
model = XAI_THINK_FAST_MODEL
27+
model = null
1328
)
1429

1530
assertEquals(
16-
"wss://api.x.ai/v1/realtime?model=grok-voice-think-fast-2.0",
31+
"wss://api.x.ai/v1/realtime?model=grok-voice-3.0",
1732
url
1833
)
1934
}
2035

2136
@Test
22-
fun `xai legacy model label maps to think fast 2 model`() {
37+
fun `xai legacy model label maps to voice 3 model`() {
2338
val url = RealtimeApiProvider.XAI.getWebSocketUrl(
2439
azureEndpoint = null,
2540
model = XAI_LEGACY_MODEL_LABEL
2641
)
2742

2843
assertEquals(
29-
"wss://api.x.ai/v1/realtime?model=grok-voice-think-fast-2.0",
44+
"wss://api.x.ai/v1/realtime?model=grok-voice-3.0",
3045
url
3146
)
3247
}

0 commit comments

Comments
 (0)