Merge pull request chigwell#252 from arnoutvw/feat/transcription-engines

Add OpenAI-compatible and local faster-whisper transcription engines

Author: chigwell

.env.example

@@ -131,9 +131,34 @@ TELEGRAM_SESSION_NAME=telegram_session
 #   leaves Telegram, but empirically drops the last speech segment in
 #   roughly 2 of 3 recordings. Requires Telegram Premium on the account.
 #   Use for chats that should never be sent to a third party.
+# - openai: any OpenAI-compatible /audio/transcriptions endpoint (OpenAI,
+#   a self-hosted Parakeet or speaches server, LocalAI, ...), with an
+#   optional API key.
+# - whisper: local faster-whisper model inside this process. Audio never
+#   leaves the machine. Needs: pip install 'telegram-mcp[whisper]'.
 # TELEGRAM_TRANSCRIBE_ENGINE=groq
 # GROQ_API_KEY=<groq api key, required when engine=groq is ever used>
 #
+# Language hint (ISO-639-1) for groq/openai/whisper. Unset = auto-detect,
+# except on Parakeet servers, which assume English when no language is sent.
+# TELEGRAM_TRANSCRIBE_LANGUAGE=nl
+# Timeout in seconds for the HTTP engines (groq/openai).
+# TELEGRAM_TRANSCRIBE_TIMEOUT=120
+#
+# engine=openai (base URL, like the OpenAI SDK's base_url; a full
+# .../audio/transcriptions URL works too). Parakeet (achetronic/parakeet):
+# http://localhost:5092/v1
+# TELEGRAM_TRANSCRIBE_OPENAI_URL=https://api.openai.com/v1
+# TELEGRAM_TRANSCRIBE_OPENAI_API_KEY=<api key, optional for keyless local servers>
+# TELEGRAM_TRANSCRIBE_OPENAI_MODEL=whisper-1
+# TELEGRAM_TRANSCRIBE_OPENAI_MAX_MB=25
+#
+# engine=whisper (faster-whisper model name or local path; loaded once per process)
+# TELEGRAM_TRANSCRIBE_WHISPER_MODEL=small
+# TELEGRAM_TRANSCRIBE_WHISPER_DEVICE=auto          # auto / cpu / cuda
+# TELEGRAM_TRANSCRIBE_WHISPER_COMPUTE_TYPE=default # e.g. int8 on CPU, float16 on GPU
+# TELEGRAM_TRANSCRIBE_WHISPER_MODEL_DIR=<where downloaded models are stored>
+#
 # Per-call budget for TELEGRAM_TRANSCRIBE=auto prefetch (ignored otherwise;
 # transcribe_voice itself is never budget-limited since it's an explicit call).
 # TELEGRAM_TRANSCRIBE_MAX_VOICES=5

README.md

@@ -74,12 +74,16 @@ Aliases live in `${XDG_STATE_HOME:-~/.local/state}/telegram-mcp/aliases.json` (o
 
 ### Voice transcription
 
-`transcribe_voice(chat_id, message_id, engine=None)` turns a voice message or video note into text. Two engines are available:
+`transcribe_voice(chat_id, message_id, engine=None)` turns a voice message or video note into text. Four engines are available:
 
 - `groq` (default): uploads the recording to Groq's hosted `whisper-large-v3-turbo`. Leaves the server and costs a download+upload per call, but doesn't drop the recording's last few words the way native transcription does. Requires `GROQ_API_KEY`. Groq caps the size of a single upload, so a recording above `TELEGRAM_TRANSCRIBE_GROQ_MAX_MB` (default 25, the free-tier limit) is refused locally with a `too_large` error naming its size instead of being downloaded and rejected by the API. Raise the limit if your Groq tier allows bigger files, or transcribe that message with `engine='telegram'`, which has no such cap.
 - `telegram`: native Telegram Premium transcription (`messages.TranscribeAudioRequest`). Free and never leaves Telegram, but empirically drops the last speech segment in roughly 2 of 3 recordings and requires Telegram Premium on the account. Long recordings come back `pending` and are polled automatically.
+- `openai`: any OpenAI-compatible `/audio/transcriptions` endpoint — OpenAI itself, a self-hosted [Parakeet](https://github.com/achetronic/parakeet) or [speaches](https://github.com/speaches-ai/speaches) server, LocalAI, a vLLM Whisper deployment, and so on. Set `TELEGRAM_TRANSCRIBE_OPENAI_URL` to the API base URL (e.g. `https://api.openai.com/v1`; a full `.../audio/transcriptions` URL also works), `TELEGRAM_TRANSCRIBE_OPENAI_API_KEY` for the bearer token (optional for keyless local servers), and `TELEGRAM_TRANSCRIBE_OPENAI_MODEL` (default `whisper-1`). Size cap: `TELEGRAM_TRANSCRIBE_OPENAI_MAX_MB` (default 25). For Parakeet use `TELEGRAM_TRANSCRIBE_OPENAI_URL=http://localhost:5092/v1`, the API key only if the server sets `PARAKEET_API_KEY`, and set `TELEGRAM_TRANSCRIBE_LANGUAGE` for anything that isn't English — Parakeet assumes `en` when no language is sent.
+- `whisper`: a local [faster-whisper](https://github.com/SYSTRAN/faster-whisper) model loaded inside the MCP server process. The audio never leaves the machine. Install the extra with `pip install 'telegram-mcp[whisper]'` (or `uv sync --extra whisper`). `TELEGRAM_TRANSCRIBE_WHISPER_MODEL` picks the model (default `small`; e.g. `large-v3-turbo` for better quality), `TELEGRAM_TRANSCRIBE_WHISPER_DEVICE` (`auto`/`cpu`/`cuda`), `TELEGRAM_TRANSCRIBE_WHISPER_COMPUTE_TYPE` (e.g. `int8` on CPU) and `TELEGRAM_TRANSCRIBE_WHISPER_MODEL_DIR` (where models are downloaded) tune it. The model is loaded once on first use and recordings are transcribed one at a time. Not available in the Alpine Docker image; use `openai` against a Parakeet or other OpenAI-compatible server next to the container instead.
 
-The engine is chosen per call via the `engine` argument, or otherwise defaults to `TELEGRAM_TRANSCRIBE_ENGINE` (`groq` or `telegram`). Results are cached by `(chat_id, message_id, engine)` in a local SQLite file so repeat reads and repeat listings never re-transcribe the same message. Concurrent requests for the same uncached recording are collapsed too: the second one waits for the first and returns its transcript, so a burst of callers costs one paid call, not one per caller. Every transcript is returned with a `note` marking it as a machine transcript, not a verbatim quote — treat it as a paraphrase, not exact wording.
+`TELEGRAM_TRANSCRIBE_LANGUAGE` (ISO-639-1, e.g. `nl`) is passed as a language hint to every engine except `telegram`; unset, the engines auto-detect. `TELEGRAM_TRANSCRIBE_TIMEOUT` (default 120 seconds) bounds a single request to the HTTP engines (`groq`, `openai`).
+
+The engine is chosen per call via the `engine` argument, or otherwise defaults to `TELEGRAM_TRANSCRIBE_ENGINE` (`groq`, `telegram`, `openai` or `whisper`). Results are cached by `(chat_id, message_id, engine)` in a local SQLite file so repeat reads and repeat listings never re-transcribe the same message. Concurrent requests for the same uncached recording are collapsed too: the second one waits for the first and returns its transcript, so a burst of callers costs one paid call, not one per caller. Every transcript is returned with a `note` marking it as a machine transcript, not a verbatim quote — treat it as a paraphrase, not exact wording.
 
 `get_history`, `get_messages`, and `list_messages` fill in already-cached transcripts for voice messages instead of leaving the text empty, controlled by `TELEGRAM_TRANSCRIBE`:
 
@@ -304,12 +308,13 @@ already-cached transcripts. Enable prefetching or pick an engine explicitly:
 
 ```env
 TELEGRAM_TRANSCRIBE=on-demand       # off / on-demand (default) / auto
-TELEGRAM_TRANSCRIBE_ENGINE=groq     # groq (default) or telegram
+TELEGRAM_TRANSCRIBE_ENGINE=groq     # groq (default), telegram, openai or whisper
 GROQ_API_KEY=your_groq_api_key_here # required whenever engine=groq is used
 ```
 
 `engine=groq` requires `GROQ_API_KEY`; `engine=telegram` requires Telegram
-Premium on the account. `TELEGRAM_TRANSCRIBE_MAX_VOICES` (default 5) and
+Premium on the account; `engine=openai` requires `TELEGRAM_TRANSCRIBE_OPENAI_URL`;
+`engine=whisper` requires the `whisper` extra. `TELEGRAM_TRANSCRIBE_MAX_VOICES` (default 5) and
 `TELEGRAM_TRANSCRIBE_MAX_SECONDS` (default 300) bound how much `auto` mode
 prefetches per listing call; `TELEGRAM_TRANSCRIPT_CACHE_DIR` (default
 `data/transcripts`) sets where the SQLite cache is written; `TELEGRAM_TRANSCRIBE_GROQ_MAX_MB`

pyproject.toml

@@ -38,6 +38,14 @@ dependencies = [
 proxy = [
     "python-socks>=2.4.3",
 ]
+# Required only for the local transcription engine (TELEGRAM_TRANSCRIBE_ENGINE=whisper).
+whisper = [
+    "faster-whisper>=1.1.0",
+    # faster-whisper 1.2.1 passes av.open(metadata_errors=...), which PyAV 19
+    # removed; 18.1.0 is the newest that works. Drop once a faster-whisper
+    # release includes SYSTRAN/faster-whisper#1495.
+    "av<19",
+]
 
 [project.urls]
 "Homepage" = "https://github.com/chigwell/telegram-mcp"

telegram_mcp/tools/messages.py

@@ -1194,31 +1194,37 @@ async def transcribe_voice(
     """
     Transcribe a voice message or video note (video circle) to text.
 
-    Two engines behind one interface:
-    - "groq" (default, override with TELEGRAM_TRANSCRIBE_ENGINE): Groq-hosted
-      whisper-large-v3-turbo. Downloads the audio and sends it to Groq - not
-      free, and leaves the server. Does not drop the recording's last words.
+    Engines (default TELEGRAM_TRANSCRIBE_ENGINE, otherwise "groq"):
+    - "groq": Groq-hosted whisper-large-v3-turbo. Downloads the audio and
+      sends it to Groq - not free, and leaves the server. Does not drop the
+      recording's last words.
     - "telegram": native Telegram Premium transcription. Free, audio never
       leaves Telegram, but empirically drops the last speech segment in
       roughly 2 of 3 recordings (proven with per-segment timestamps). Use for
       chats you don't want sent to a third party, or when Groq is unavailable.
       Requires Telegram Premium on this account; polls briefly (up to ~20s)
       while Telegram finishes a long recording.
+    - "openai": any OpenAI-compatible transcription endpoint
+      (TELEGRAM_TRANSCRIBE_OPENAI_URL, optional API key), e.g. OpenAI or a
+      self-hosted Parakeet/speaches server.
+    - "whisper": a local faster-whisper model on this server. Audio never
+      leaves the machine; slower on CPU.
 
     Results are cached per engine, by (chat_id, message_id, engine) - a
     repeat call with the same engine returns the cached text without
-    hitting either API again. Asking for an engine that has no cached
-    result transcribes with it, even when the other engine's text is
+    hitting any engine again. Asking for an engine that has no cached
+    result transcribes with it, even when another engine's text is
     already cached.
 
     The returned text is a machine transcript, not a verbatim quote: proper
-    names, punctuation and occasional words drift under both engines.
+    names, punctuation and occasional words drift under every engine.
 
     Args:
         chat_id: The chat ID or username.
         message_id: The message ID containing the voice/video-note media.
-        engine: "groq" or "telegram". Defaults to TELEGRAM_TRANSCRIBE_ENGINE
-            (groq unless configured otherwise).
+        engine: "groq", "telegram", "openai" or "whisper".
+            Defaults to TELEGRAM_TRANSCRIBE_ENGINE (groq unless configured
+            otherwise).
     """
     try:
         mode = transcription.transcribe_mode()
@@ -1233,10 +1239,11 @@ async def transcribe_voice(
 
         chosen_engine = (engine or transcription.default_engine()).strip().lower()
         if chosen_engine not in transcription.ENGINES:
-            return f"Invalid engine '{engine}'. Use 'telegram' or 'groq'."
+            accepted = ", ".join(f"'{name}'" for name in sorted(transcription.ENGINES))
+            return f"Invalid engine '{engine}'. Use one of: {accepted}."
 
         # Pinned to the chosen engine on purpose: a cached telegram transcript
-        # must not answer a groq request. The native engine drops the last
+        # must not answer a request for any other engine. The native engine drops the last
         # speech segment and the loss cannot be seen in the text.
         cached = transcription.get_cached_transcript(
             numeric_chat_id, message_id, source=chosen_engine
@@ -1261,11 +1268,9 @@ async def transcribe_voice(
         if not transcription.is_transcribable(msg):
             return f"Message {message_id} has no voice message or video note to transcribe."
 
-        if chosen_engine == "groq" and not os.getenv("GROQ_API_KEY"):
-            return (
-                "GROQ_API_KEY is not configured on this server. "
-                "Use engine='telegram' or set GROQ_API_KEY."
-            )
+        config_error = transcription.engine_config_error(chosen_engine)
+        if config_error:
+            return config_error
 
         duration = transcription.voice_duration(msg)
         # Cache-first and locked by (chat, message, engine): two concurrent