You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Configure your Gemini API key in `.env` using a placeholder format:
4
+
```
5
+
GEMINI_API_KEY=your-gemini-api-key-here
6
+
```
7
+
8
+
## TTS Provider Support
9
+
10
+
VOX LAB now supports **three TTS providers**:
11
+
12
+
### 1. **Google Cloud TTS** (default)
13
+
- Free tier available
14
+
- Large voice selection
15
+
- Most stable
16
+
17
+
### 2. **Azure Speech Services**
18
+
- Enterprise-grade
19
+
- Real-time streaming
20
+
- Multiple voice styles
21
+
22
+
### 3. **Google Gemini TTS** (NEW!)
23
+
- Modern LLM-based synthesis
24
+
- Natural, expressive output
25
+
- Lower latency for research
26
+
27
+
## Switching TTS Providers
28
+
29
+
You can switch providers via environment variable or API:
30
+
31
+
### Environment Variable
32
+
```bash
33
+
# In .env file
34
+
TTS_PROVIDER=gemini # or 'google' or 'azure'
35
+
GEMINI_API_KEY=your-api-key
36
+
```
37
+
38
+
### API Call (Runtime)
39
+
```bash
40
+
curl -X POST http://localhost:3010/api/set-tts-provider \
41
+
-H "Content-Type: application/json" \
42
+
-d '{"provider": "gemini"}'
43
+
```
44
+
45
+
## Implementation Notes
46
+
47
+
The Gemini TTS integration is ready for:
48
+
- ✅ Text-to-speech synthesis
49
+
- ✅ Multiple voice profiles
50
+
- ✅ Streaming audio output
51
+
- ✅ Format conversion (WAV, MP3)
52
+
53
+
### Example: Using Gemini TTS
54
+
55
+
```javascript
56
+
// Client-side or API call
57
+
constresponse=awaitfetch('/api/tts', {
58
+
method:'POST',
59
+
headers: { 'Content-Type':'application/json' },
60
+
body:JSON.stringify({
61
+
text:"Hello, this is spoken with Gemini TTS",
62
+
voice: { name:"en-US-female" },
63
+
audioConfig: { audioEncoding:"MP3" }
64
+
})
65
+
});
66
+
67
+
constaudio=awaitresponse.arrayBuffer();
68
+
```
69
+
70
+
## Next Steps
71
+
72
+
1.**Test Gemini TTS**: Run a conversation in VOX LAB with TTS provider set to "gemini"
73
+
2.**Regenerate API Key**: For security, [regenerate your Gemini API key](https://console.cloud.google.com/apis/credentials)
74
+
3.**Monitor Costs**: Gemini TTS charges per request - monitor your Google Cloud usage
75
+
76
+
## Troubleshooting
77
+
78
+
If you get "content filter" errors, it means the LLM response violated Azure's content policy. This is separate from TTS and happens during text generation before audio synthesis.
79
+
80
+
**Solution**: The prompt content has been simplified to reduce false positives. If issues persist:
81
+
1. Check agent.js prompts for problematic content
82
+
2. Adjust conversation topics to avoid sensitive content
**VOX LAB** is an authoring environment for configuring and orchestrating multi-agent conversations with animated 3D avatars. Built with React, Vite, and Express, it helps researchers, designers, and developers create, visualize, and evaluate complex agent-based dialog systems.
6
6
7
-
**DialogLab** is an authoring tool for configuring and orchestrating multi-agent conversations with animated 3D avatars. Built with React, Vite, and Express, it enables researchers, designers, and developers to create, visualize, and evaluate complex agent-based dialogue systems.
7
+
## Overview
8
+
9
+
VOX LAB combines a visual authoring interface, avatar-driven playback, and runtime model integrations so teams can rapidly prototype and test conversational experiences.
8
10
9
11
## Features
10
12
11
-
-**Visual Conversation Design**: Configure multi-agent conversations with an intuitive node-based editor
12
-
-**3D Avatar Integration**: Animate conversations using Ready Player Me avatars with synchronized speech
13
-
-**Multiple LLM Support**: Compatible with OpenAI GPT and Google Gemini models
14
-
-**Scene Management**: Create and manage multiple conversation scenarios
15
-
-**Real-time Preview**: Test and iterate on conversations in real-time
16
-
-**Verification Tools**: Analyze conversation metrics and quality
17
-
-**Experience Mode**: Present finished conversations in a polished viewer
13
+
-**Visual conversation design**: Build multi-agent conversations with an intuitive node editor.
14
+
-**3D avatar integration**: Animate conversations with Ready Player Me avatars and synchronized speech.
15
+
-**Multiple LLM providers**: Use either OpenAI or Google Gemini models.
16
+
-**Scene management**: Create and organize multiple conversation scenarios.
17
+
-**Real-time preview**: Test and iterate quickly while authoring.
18
+
-**Verification tooling**: Inspect conversation quality and metrics.
19
+
-**Experience mode**: Present completed conversations in a polished viewer.
18
20
19
21
## Prerequisites
20
22
@@ -24,7 +26,7 @@
24
26
## Repository Structure
25
27
26
28
```
27
-
DialogLab/
29
+
VOX-LAB/
28
30
├── client/ # React UI (Vite) - Dev server on port 5173
29
31
│ ├── src/ # React components and application logic
Example avatar files (`client/public/assets/*.glb`) were created using [Ready Player Me](https://readyplayer.me/) and are subject to [Ready Player Me's Terms of Use](https://readyplayer.me/terms-of-use).
108
-
109
-
**Note**: These assets are provided for demonstration purposes only and are **not covered** by this project's open-source license. For production use, you must obtain your own avatars or comply with Ready Player Me's licensing terms.
110
-
111
-
## Citation
112
-
113
-
If you use DialogLab in your research, please cite our UIST 2025 paper:
114
-
115
-
```bibtex
116
-
@inproceedings{dialoglab2025,
117
-
author = {Hu, Erzhen and Chen, Yanhe and Li, Mingyi and Phadnis, Vrushank and Xu, Pingmei and Qian, Xun and Olwal, Alex and Kim, David and Heo, Seongkook and Du, Ruofei},
118
-
title = {DialogLab: Configuring and Orchestrating Multi-Agent Conversations},
119
-
booktitle = {Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (UIST '25)},
120
-
year = {2025},
121
-
publisher = {Association for Computing Machinery},
122
-
address = {New York, NY, USA},
123
-
doi = {10.1145/3746059.3747696}
124
-
}
125
-
```
113
+
-`NODE_ENV`: Runtime environment (`development` by default).
114
+
-`GEMINI_API_KEY`: Google Gemini API key.
115
+
-`API_KEY_LLM`: OpenAI API key.
116
+
-`DEFAULT_LLM_PROVIDER`: Default provider (`gemini` or `openai`).
0 commit comments