Skip to content
BranchesTags

Repository files navigation

Polyglot Meet 🌐

Real-time multilingual meeting platform — every participant speaks their own language and hears everyone else in their preferred language. AI-powered translation, memory, search, and summaries. Self-hosted, open-source, horizontally scalable.

Built as a cost-effective alternative to Google Meet, Microsoft Teams Translation, and Zoom Translation with added AI capabilities.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                          FRONTEND (Next.js 15)                       │
│  Port 3000  │  React 19  │  Socket.IO Client  │  Zustand  │  RQ     │
└──────────────┬──────────────────────────────┬───────────────────────┘
               │ HTTP REST                     │ WebSocket (Socket.IO)
               ▼                               ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     BACKEND (NestJS + Fastify)                       │
│  Port 4000  │  JWT/OAuth  │  WebRTC Signaling  │  Chat  │  API     │
│  Prisma ORM  │  Redis Cache  │  RabbitMQ Queue  │  MinIO Storage   │
└──────────────┬──────────────────────────────┬───────────────────────┘
               │ RabbitMQ (async)              │ HTTP (sync)
               ▼                               ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     AI SERVICE (FastAPI + Python)                    │
│  Port 8000  │  Gemini 2.0 Flash  │  sentence-transformers           │
│  TurboVec Vector DB  │  STT/TTS  │  Summarization                  │
└─────────────────────────────────────────────────────────────────────┘

Data Flow

  1. Real-time Translation: User speaks → WebRTC audio → STT (Gemini) → Translate (Gemini) → TTS (Gemini) → Target user hears in their language
  2. Transcript Pipeline: Audio text → saved to DB → queued to RabbitMQ → Gemini translates → saved with translations → broadcast via Socket.IO
  3. Memory & Search: Transcript → embedding (sentence-transformers) → stored in TurboVec → semantic search via /api/v1/memory/search
  4. Summaries: Meeting ends → RabbitMQ triggers → Gemini generates structured summary → saved to DB

Tech Stack

Component Technology
Frontend Next.js 15, React 19, TypeScript, Tailwind CSS, shadcn/ui
Backend NestJS 10, Fastify, TypeScript, Prisma ORM
AI Service FastAPI, Python 3.12, Gemini 2.0 Flash, sentence-transformers
Database PostgreSQL 16 (primary), Redis 7 (cache/sessions)
Queue RabbitMQ 3.13 (translation/summary/embedding pipelines)
Storage MinIO (S3-compatible, recording files)
Vector DB TurboVec (self-hosted, cost-effective)
Signaling Socket.IO (WebSocket transport for WebRTC)
WebRTC Native WebRTC (mesh topology, ≤6 participants)
Monitoring Prometheus, Grafana, Loki, OpenTelemetry
CI/CD GitHub Actions + Dependabot
Container Docker, Docker Compose
Orchestration Kubernetes + Helm (optional production deployment)

Directory Structure

polyglot-meet/
├── backend/                          # NestJS + Fastify backend
│   ├── prisma/
│   │   ├── schema.prisma             # 12 models, enums, indexes
│   │   └── seed.ts                   # Test data seeder
│   ├── src/
│   │   ├── auth/                     # JWT + Google OAuth + refresh tokens
│   │   ├── chat/                     # Real-time chat (REST + Socket.IO)
│   │   ├── common/                   # Filters, interceptors, pipes, decorators
│   │   ├── config/                   # Zod-validated env configuration
│   │   ├── database/                 # Prisma + Redis modules
│   │   ├── meeting/                  # Meeting CRUD
│   │   ├── observability/            # OpenTelemetry + Winston logging
│   │   ├── queue/                    # RabbitMQ producer
│   │   ├── recording/                # Recording start/stop/list
│   │   ├── storage/                  # MinIO client
│   │   ├── transcript/               # Transcript + translation pipeline
│   │   ├── user/                     # User profile + language preferences
│   │   └── webrtc/                   # Socket.IO gateway for WebRTC signaling
│   └── .env                          # Backend environment variables
│
├── frontend/                         # Next.js 15 App Router
│   ├── src/
│   │   ├── app/
│   │   │   ├── (auth)/               # Login, register, Google OAuth callback
│   │   │   ├── (dashboard)/          # Meeting list, create, join, settings
│   │   │   └── (meeting)/            # Meeting room with video grid
│   │   ├── components/               # shadcn/ui + custom components
│   │   │   ├── ui/                   # 14 Radix UI primitives
│   │   │   ├── auth/                 # Login/register forms
│   │   │   ├── meeting/              # Video grid, controls, dialogs
│   │   │   ├── chat/                 # Chat panel
│   │   │   ├── transcript/           # Live transcript + translations
│   │   │   ├── summary/              # Summary panel
│   │   │   └── layout/               # Header, sidebar
│   │   ├── hooks/                    # useAuth, useWebRTC, useSocket, useTranslation
│   │   ├── services/                 # API clients (auth, meeting, ai, socket)
│   │   ├── stores/                   # Zustand (auth, meeting, translation, webrtc)
│   │   └── lib/                      # Utilities (cn, formatDate, etc.)
│   └── .env.local                    # Frontend environment variables
│
├── ai-service/                       # FastAPI + Python 3.12 AI microservice
│   ├── app/
│   │   ├── api/v1/                   # REST endpoints (translation, speech, memory, summary)
│   │   ├── core/                     # Config (pydantic-settings), deps
│   │   ├── models/                   # Pydantic schemas
│   │   ├── observability/            # Loguru logging, Prometheus metrics
│   │   ├── services/
│   │   │   ├── translation/          # Gemini translation (20 languages)
│   │   │   ├── speech/               # STT + TTS via Gemini
│   │   │   ├── memory/               # sentence-transformers + TurboVec
│   │   │   ├── search/               # RabbitMQ consumers + search logic
│   │   │   └── summary/              # Meeting summarization via Gemini
│   │   └── storage/                  # asyncpg database service
│   └── scripts/                      # init_db.py
│
├── docker/                           # Multi-stage Dockerfiles
│   └── compose/                      # Per-service compose files (run individually)
├── docker-compose.yml                # 12-service orchestration (all at once)
├── k8s/                              # Kubernetes manifests
│   ├── base/                         # Deployments, services, ingress, HPA, etc.
│   └── overlays/                     # dev + prod Kustomize overlays
├── helm/polyglot-meet/               # Helm chart with Bitnami deps
├── monitoring/                       # Observability configs
│   ├── prometheus/
│   ├── grafana/
│   ├── loki/
│   └── otel-collector/
└── .github/                          # CI/CD workflows + Dependabot

Prerequisites

  • Node.js 20+ (for backend + frontend)
  • Python 3.12+ (for AI service)
  • Docker + Docker Compose (recommended for infrastructure)
  • Gemini API Key (from Google AI Studio)

Environment Variables

Backend (backend/.env)

NODE_ENV=development
PORT=4000

# Database
DATABASE_URL=postgresql://polyglot:polyglot_secret@localhost:5432/polyglot_meet

# Cache
REDIS_URL=redis://localhost:6379

# Message Queue
RABBITMQ_URL=amqp://polyglot:polyglot_secret@localhost:5672

# File Storage (MinIO)
MINIO_ENDPOINT=localhost
MINIO_PORT=9000
MINIO_ACCESS_KEY=polyglot
MINIO_SECRET_KEY=polyglot_secret
MINIO_USE_SSL=false

# Auth
JWT_SECRET=dev-jwt-secret-change-in-production
JWT_REFRESH_SECRET=dev-refresh-secret-change-in-production
JWT_EXPIRES_IN=15m
JWT_REFRESH_EXPIRES_IN=7d

# Service URLs
AI_SERVICE_URL=http://localhost:8000
FRONTEND_URL=http://localhost:3000

Frontend (frontend/.env.local)

NEXT_PUBLIC_API_URL=http://localhost:4000
NEXT_PUBLIC_WS_URL=http://localhost:4000
NEXT_PUBLIC_AI_URL=http://localhost:8000

AI Service (ai-service/.env)

GEMINI_API_KEY=your-gemini-api-key-here
TURBOVEC_URL=http://localhost:8080
DATABASE_URL=postgresql://polyglot:polyglot_secret@localhost:5432/polyglot_meet
REDIS_URL=redis://localhost:6379
RABBITMQ_URL=amqp://polyglot:polyglot_secret@localhost:5672
LOG_LEVEL=INFO

Quick Start (Docker Compose)

Option 1: All services at once

# 1. Clone and navigate
git clone <repo-url>
cd polyglot-meet

# 2. Set required environment variable
export GEMINI_API_KEY="your-key-here"

# 3. Start all services (12 containers)
docker compose up -d

# 4. Run Prisma migrations
docker compose exec backend npx prisma migrate dev --name init

# 5. Seed demo data
docker compose exec backend npx prisma db seed

Option 2: Start services individually

# Infrastructure (start first)
docker compose -f docker\compose\postgres.yml up -d
docker compose -f docker\compose\redis.yml up -d
docker compose -f docker\compose\rabbitmq.yml up -d
docker compose -f docker\compose\minio.yml up -d

# App services
docker compose -f docker\compose\backend.yml up -d
docker compose -f docker\compose\ai-service.yml up -d
docker compose -f docker\compose\frontend.yml up -d

# Monitoring (optional)
docker compose -f docker\compose\prometheus.yml up -d
docker compose -f docker\compose\grafana.yml up -d
docker compose -f docker\compose\loki.yml up -d
docker compose -f docker\compose\otel-collector.yml up -d

Once running:

Service URL
Frontend http://localhost:3000
Backend API http://localhost:4000/api/v1
AI Service http://localhost:8000
Grafana http://localhost:3001
Prometheus http://localhost:9090
Loki http://localhost:3100
MinIO Console http://localhost:9001
RabbitMQ Manager http://localhost:15672

Manual Development Setup

1. Start Infrastructure Services

# Start all infra in one command, or use individual files:
docker compose -f docker\compose\postgres.yml up -d
docker compose -f docker\compose\redis.yml up -d
docker compose -f docker\compose\rabbitmq.yml up -d
docker compose -f docker\compose\minio.yml up -d

2. Backend

cd backend
npm install
npx prisma generate
npx prisma migrate dev --name init
npx prisma db seed
npm run start:dev

3. AI Service

cd ai-service
pip install -r requirements.txt
python scripts/init_db.py
uvicorn app.main:app --reload --port 8000

4. Frontend

cd frontend
npm install
npm run dev

5. Access

Open http://localhost:3000 and log in with seeded credentials:

Email Password Role
admin@polyglotmeet.com Password123 Admin
alice@example.com Password123 Host
bob@example.com Password123 Participant

API Reference

Backend (http://localhost:4000/api/v1)

Auth

Method Endpoint Description
POST /auth/register Register new user
POST /auth/login Login (returns JWT tokens)
POST /auth/refresh Refresh access token
POST /auth/logout Logout (revoke refresh)
POST /auth/google Google OAuth login

Meetings

Method Endpoint Description
GET /meetings List user's meetings
POST /meetings Create meeting
GET /meetings/:id Get meeting details
PATCH /meetings/:id Update meeting
DELETE /meetings/:id Delete meeting
POST /meetings/:id/join Join meeting
POST /meetings/:id/leave Leave meeting

Chat

Method Endpoint Description
POST /meetings/:id/messages Send chat message
GET /meetings/:id/messages Get chat messages

Transcripts

Method Endpoint Description
POST /meetings/:id/transcripts Save transcript
GET /meetings/:id/transcripts Get transcript history

Recordings

Method Endpoint Description
POST /meetings/:id/recordings/start Start recording
POST /meetings/:id/recordings/stop Stop recording
GET /meetings/:id/recordings List recordings

Users

Method Endpoint Description
GET /users/profile Get user profile
GET /users/language Get language preferences
PATCH /users/language Update language preferences

AI Service (http://localhost:8000)

Translation

Method Endpoint Description
POST /api/v1/translation/translate Translate text
POST /api/v1/translation/detect-language Detect language

Speech

Method Endpoint Description
POST /api/v1/speech/stt Speech-to-text
POST /api/v1/speech/tts Text-to-speech

Memory

Method Endpoint Description
POST /api/v1/memory/embeddings Store embedding
POST /api/v1/memory/search Semantic search
DELETE /api/v1/memory/embeddings/:meeting_id Delete embeddings

Summary

Method Endpoint Description
POST /api/v1/summary/summarize Generate summary
GET /api/v1/summary/summaries/:meeting_id Get summary

WebRTC Signaling

The backend provides a Socket.IO gateway (/ws/meeting) for WebRTC signaling:

Event Direction Description
join-room Client → Server Join a meeting room
leave-room Client → Server Leave a meeting room
sdp-offer Bidirectional WebRTC SDP offer exchange
sdp-answer Bidirectional WebRTC SDP answer exchange
ice-candidate Bidirectional ICE candidate exchange
mute-toggle Client → Server Toggle audio/video mute state
user-joined Server → Client Notification when user joins
user-left Server → Client Notification when user leaves

Current topology: Mesh (peer-to-peer) — suitable for ≤6 participants. For larger rooms, the architecture supports migration to mediasoup (Selective Forwarding Unit).

RabbitMQ Queues

Queue TTL Persistent Dead-Letter Description
ai.translation.request 30s Yes Yes Translate transcript text
ai.summary.request 5m Yes Yes Generate meeting summary
ai.embedding.request 30s Yes Yes Generate + store embedding

Consumers run in the AI service (app/services/search/rabbitmq_consumer.py).

Prisma Schema (12 Models)

User ──┬── Meeting (hosted)
       ├── Participant
       ├── LanguagePreference
       ├── ChatMessage
       ├── Transcript ── Translation
       ├── Embedding
       ├── AuditLog
       └── RefreshToken

Meeting ──┬── Participant
          ├── ChatMessage
          ├── Transcript
          ├── Translation
          ├── Summary
          ├── ActionItem
          ├── Recording
          ├── Embedding
          └── AuditLog

Supported Languages (AI Translation)

Code Language Code Language
en English es Spanish
fr French de German
it Italian pt Portuguese
ru Russian zh Chinese
ja Japanese ko Korean
ar Arabic hi Hindi
bn Bengali pa Punjabi
ta Tamil te Telugu
vi Vietnamese th Thai
tr Turkish nl Dutch

Deployment

Production (Kubernetes)

# Apply base manifests
kubectl apply -k k8s/overlays/prod

# Or use Helm
helm install polyglot-meet ./helm/polyglot-meet \
  --set postgresql.auth.password=secure-pass \
  --set global.geminiApiKey=your-key

Key Production Considerations

  • PostgreSQL: Use managed service (Neon, RDS, Cloud SQL) or configure pgbouncer for connection pooling
  • Redis: Use managed service (Upstash, ElastiCache) or configure Redis Sentinel/Cluster
  • RabbitMQ: Use RabbitMQ cluster or managed service (CloudAMQP)
  • MinIO: Use S3 (AWS, DigitalOcean Spaces, Wasabi) for production recordings
  • WebRTC TURN: Deploy a TURN server (coturn) for users behind restrictive NATs
  • AI Service: Set GEMINI_API_KEY and monitor rate limits
  • Turbovec: Deploy with persistent volume; or replace with pgvector if preferred

Database Schema

The Prisma schema (backend/prisma/schema.prisma) defines 12 models:

  • User — authentication, profile, roles
  • Meeting — meetings with status lifecycle (scheduled → active → ended)
  • Participant — meeting membership with audio/video state
  • LanguagePreference — per-user speak/hear language
  • ChatMessage — meeting chat history
  • Transcript — speech-to-text output per utterance
  • Translation — translated versions of transcript entries
  • Embedding — vector embedding references for AI memory
  • Summary — AI-generated meeting summaries with decisions/risks/follow-ups
  • ActionItem — action items extracted from meetings
  • Recording — meeting recording metadata (stored in MinIO)
  • RefreshToken — JWT refresh token rotation
  • AuditLog — immutable audit trail for all actions

Monitoring

Tool Purpose URL
Prometheus Metrics collection (backend:9464, ai:8000) http://localhost:9090
Grafana Dashboards + alerts http://localhost:3001
Loki Log aggregation http://localhost:3100
OpenTelemetry Distributed tracing gRPC :4317 / HTTP :4318

Architecture Decisions

Decision Rationale
Native WebRTC (mesh) over LiveKit/Agora Cost optimization, self-hosted, no per-minute fees. Mediasoup migration path documented for scale.
Fastify over Express 2-3x throughput, built-in helmet/cors/rate-limiter
Python AI Service separate from NestJS Leverages Python ML ecosystem (sentence-transformers, Gemini SDK). Independent scaling.
RabbitMQ for async pipelines Decouples real-time WebRTC from AI processing. Durable queues with dead-letter for reliability.
TurboVec over Pinecone/Milvus Self-hosted, zero cost, simple HTTP API
Socket.IO over raw WebSocket Automatic reconnection, room management, fallback transport
JWT refresh token rotation Security best practice — old refresh tokens are invalidated on use
Zod validation Runtime type safety at API boundary + DTO layer
Winston + OpenTelemetry Structured JSON logging with correlation IDs + distributed tracing

License

MIT — free to use, modify, and distribute.

Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing)
  3. Commit changes (git commit -m 'feat: add amazing feature')
  4. Push to branch (git push origin feature/amazing)
  5. Open a Pull Request

About

Real-time multilingual meeting platform — every participant speaks their own language and hears everyone else in their preferred language. AI-powered translation, memory, search, and summaries. Self-hosted, open-source, horizontally scalable.

Topics

accessibilityai-poweredcollaboration-toolhorizontally-scalableinternational-communicationlanguage-supportmeeting-platformmeeting-summarymeeting-transcriptionmultilingualopen-sourcepythonreal-time-communicationreal-time-translationsearch-meetingsself-hostedtranslation-apitypescriptvideo-conferencingvoice-translation

Resources

Readme
License
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages