feat: add MOSS diarization backend

Signed-off-by: LauraGPT <18321252+LauraGPT@users.noreply.github.com>

Author: LauraGPT

README.md

@@ -34,6 +34,7 @@
 
 <a name="What's New"></a>
 ## What's New🚀
+- 2026/08/30 FunClip adds the third-party [OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize) model as an opt-in `moss` path. It provides long-form ASR, speaker identity, and segment timestamps without external `vad_model` or `spk_model`, through FunASR's vLLM adapter. OpenMOSS owns and maintains the model; FunClip only integrates its published contract.
 - 2026/08/03 [FunClip v2.1.1](https://github.com/modelscope/FunClip/releases/tag/v2.1.1) fixes fresh Gradio 4 installations by constraining the incompatible Starlette 1.x runtime, keeps `--listen` container startup private unless `--share` is explicitly requested, makes transcript matching case-insensitive, and adds MiniMax M2.7 provider routes.
 - 2026/07/24 [FunClip v2.1.0](https://github.com/modelscope/FunClip/releases/tag/v2.1.0) is the first versioned GitHub release. It packages the current Fun-ASR-Nano, SenseVoice, Paraformer, and LLM-assisted clipping application as checksum-protected source archives for a stable rollback point.
 - 2026/05/20 FunClip now supports [Fun-ASR-Nano](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) and [SenseVoice](https://huggingface.co/FunAudioLLM/SenseVoiceSmall) models. The `fun-asr-nano` option loads the flagship Fun-ASR-Nano-2512 checkpoint for Mandarin, English, Japanese, 7 Chinese dialect groups, and 26 regional accents; it does not load the separate 31-language Fun-ASR-MLT-Nano-2512 checkpoint. SenseVoice adds emotion recognition and audio event detection. Run `python funclip/launch.py -m fun-asr-nano` or `-m sensevoice` to try. For precise text-based clipping, use Paraformer because the released Nano checkpoint does not provide reliable character-level timestamps.
@@ -80,7 +81,7 @@ For a versioned snapshot, download [FunClip-2.1.1.tar.gz](https://github.com/mod
 
 FunClip v2.1.1 supports Gradio 4 with `starlette<1.0`. Existing installations should run `pip install -U -r requirements.txt` before restarting. Container users can pass `--listen` to bind all interfaces; a public Gradio sharing tunnel is created only when `--share` is also supplied.
 
-FunClip's Fun-ASR-Nano, SenseVoice, and subtitle compatibility paths require `funasr>=1.3.29`. This release returns every SenseVoice VAD region through `sentence_info` when token timestamps are unavailable, so clipping and subtitle clients receive segment boundaries instead of an empty timeline. It also includes the real-time final-text and short-tail fixes from 1.3.28. If you installed FunClip before this requirement was updated, run `pip install -U "funasr>=1.3.29"` before starting the Gradio service. [Release notes](https://github.com/modelscope/FunASR/releases/tag/v1.3.29) · [PyPI](https://pypi.org/project/funasr/1.3.29/)
+FunClip's current model and subtitle compatibility paths require `funasr>=1.4.9`. This includes the MOSS vLLM adapter, long-audio generation controls, normalized `sentence_info` speaker segments, and the earlier SenseVoice and realtime fixes. If you installed FunClip before this requirement was updated, run `pip install -U "funasr>=1.4.9"` before starting the Gradio service. [Release notes](https://github.com/modelscope/FunASR/releases/tag/v1.4.9) · [PyPI](https://pypi.org/project/funasr/1.4.9/)
 
 ### imagemagick install (Optional)
 
@@ -119,6 +120,7 @@ python funclip/launch.py
 # '-m fun-asr-nano' for the flagship Fun-ASR-Nano model (Mandarin, English,
 # Japanese, 7 Chinese dialect groups, and 26 regional accents)
 # '-m sensevoice' for SenseVoice model (multilingual ASR + emotion + audio event detection)
+# '--model moss' for OpenMOSS long-form ASR + speaker identity + timestamps
 # '-l en' for English audio recognize
 # '-p xxx' for setting port number
 # '-s True' for establishing service for public accessing
@@ -131,8 +133,23 @@ python funclip/launch.py
 | Default Chinese video clipping with Paraformer | `python funclip/launch.py` |
 | High-accuracy transcription with the flagship Fun-ASR-Nano checkpoint (use Paraformer for precise text-based clipping) | `python funclip/launch.py -m fun-asr-nano` |
 | Multilingual ASR with emotion and audio event tags | `python funclip/launch.py -m sensevoice` |
+| MOSS through a local vLLM transcription service | `python funclip/launch.py --model moss --moss-backend vllm` |
 | English video clipping with the Paraformer English model | `python funclip/launch.py -l en` |
 
+#### MOSS-Transcribe-Diarize backend
+
+[MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize) is a third-party OpenMOSS model, not a FunASR or FunClip model. FunClip pins the published Hugging Face object `OpenMOSS-Team/MOSS-Transcribe-Diarize` at revision `e8681d68e7042738ffca8ac8212bc8fcb1131ab8`. Start and verify the vLLM service using the [bilingual production guide](https://www.funasr.com/en/deploy/moss-transcribe-diarize.html), then run:
+
+```shell
+# vLLM is the default MOSS backend and defaults to http://127.0.0.1:8898/v1
+python funclip/launch.py --model moss --moss-backend vllm
+
+# Optional bearer credentials are read from the environment, not the command line
+MOSS_API_KEY=replace-me python funclip/launch.py --model moss
+```
+
+MOSS performs segmentation and speaker diarization end to end. Do not attach an external `vad_model` or `spk_model`, because chunking would break global speaker identity. Its timestamps are segment-level: SRT, speaker clipping (`spkS01`, `spkS02`, ...), and LLM timestamp clipping are supported, while precise arbitrary text clipping still requires Paraformer's token timestamps. FunClip currently exposes the vLLM path because it is compatible with the standard Transformers 4.x environment and has an end-to-end tested OpenAI transcription contract.
+
 If you only need offline speech transcription on CPU or edge devices and do not need FunClip's video clipping UI, use the FunASR llama.cpp / GGUF runtime instead: [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · [Fun-ASR-Nano-GGUF](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-GGUF) · [SenseVoiceSmall-GGUF](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF).
 
 then visit ```localhost:7860``` you will get a Gradio service like below and you can use FunClip following the steps:

README_zh.md

@@ -36,6 +36,7 @@
 <a name="近期更新"></a>
 ## 近期更新🚀
 
+- 2026/08/30 FunClip 新增第三方 [OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize) 可选路径。它通过 FunASR 的 vLLM 适配器提供长音频 ASR、说话人身份和分段时间戳，不需要外部 `vad_model` 或 `spk_model`。模型归 OpenMOSS 所有并由其维护，FunClip 只集成公开接口。
 - 2026/08/03 [FunClip v2.1.1](https://github.com/modelscope/FunClip/releases/tag/v2.1.1) 修复 Gradio 4 新安装环境与 Starlette 1.x 的不兼容问题；容器使用 `--listen` 时不会自动创建公网分享链接；文本匹配改为大小写不敏感，并新增 MiniMax M2.7 模型路由。
 - 2026/07/24 [FunClip v2.1.0](https://github.com/modelscope/FunClip/releases/tag/v2.1.0) 是首个带版本号的 GitHub Release，将当前支持 Fun-ASR-Nano、SenseVoice、Paraformer 与大模型智能剪辑的应用打包为带 SHA-256 校验的源码归档，提供稳定的下载与回退节点。
 - 2026/05/20 FunClip 现在支持 [Fun-ASR-Nano](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) 与 [SenseVoice](https://huggingface.co/FunAudioLLM/SenseVoiceSmall) 模型。`fun-asr-nano` 选项加载旗舰版 Fun-ASR-Nano-2512，支持普通话、英语、日语、7 类中文方言和 26 种地域口音；该选项不会加载独立的 31 语种 Fun-ASR-MLT-Nano-2512。SenseVoice 支持多语种识别，并额外输出情绪识别与音频事件检测标签。可通过 `python funclip/launch.py -m fun-asr-nano` 或 `python funclip/launch.py -m sensevoice` 启动体验。需要精确按文本裁剪时请使用 Paraformer，因为当前发布的 Nano checkpoint 不提供可靠的字符级时间戳。
@@ -81,7 +82,7 @@ pip install -r ./requirements.txt
 
 FunClip v2.1.1 在 Gradio 4 环境中要求 `starlette<1.0`。已有安装请在重启前执行 `pip install -U -r requirements.txt`。容器用户可用 `--listen` 监听全部网卡；只有同时显式传入 `--share` 才会创建 Gradio 公网分享链接。
 
-FunClip 的 Fun-ASR-Nano、SenseVoice 与字幕兼容路径需要 `funasr>=1.3.29`。当 SenseVoice 没有 token 时间戳时，该版本会通过 `sentence_info` 返回每个 VAD 语音区域，让智能剪辑与字幕客户端获得分段边界，而不再收到空时间线；同时包含 1.3.28 的实时最终文本和短尾语音修复。如果你之前已经安装过 FunClip，请先执行 `pip install -U "funasr>=1.3.29"`，再启动 Gradio 服务。[发布说明](https://github.com/modelscope/FunASR/releases/tag/v1.3.29) · [PyPI](https://pypi.org/project/funasr/1.3.29/)
+FunClip 当前模型与字幕兼容路径需要 `funasr>=1.4.9`，其中包括 MOSS 的 vLLM 适配器、长音频生成上限、归一化的 `sentence_info` 说话人分段，以及此前的 SenseVoice 和实时修复。如果你之前已经安装过 FunClip，请先执行 `pip install -U "funasr>=1.4.9"`，再启动 Gradio 服务。[发布说明](https://github.com/modelscope/FunASR/releases/tag/v1.4.9) · [PyPI](https://pypi.org/project/funasr/1.4.9/)
 
 ### 安装imagemagick（可选）
 
@@ -119,6 +120,7 @@ python funclip/launch.py
 # '-m fun-asr-nano' 使用旗舰版 Fun-ASR-Nano（普通话、英语、日语、
 # 7 类中文方言和 26 种地域口音）
 # '-m sensevoice' 使用 SenseVoice 模型（多语种 ASR + 情绪识别 + 音频事件检测）
+# '--model moss' 使用 OpenMOSS 长音频 ASR + 说话人身份 + 时间戳
 # '-l en' for English audio recognize
 # '-p xxx' for setting port number
 # '-s True' for establishing service for public accessing
@@ -130,8 +132,23 @@ python funclip/launch.py
 | 默认中文视频裁剪，使用 Paraformer | `python funclip/launch.py` |
 | 使用旗舰版 Fun-ASR-Nano 进行高精度转写（精确按文本裁剪请使用 Paraformer） | `python funclip/launch.py -m fun-asr-nano` |
 | 使用 SenseVoice 进行多语种识别、情绪识别和音频事件检测 | `python funclip/launch.py -m sensevoice` |
+| 通过本地 vLLM 转写服务使用 MOSS | `python funclip/launch.py --model moss --moss-backend vllm` |
 | 使用 Paraformer 英文模型裁剪英文视频 | `python funclip/launch.py -l en` |
 
+#### MOSS-Transcribe-Diarize 后端
+
+[MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize) 是 OpenMOSS 维护的第三方模型，不属于 FunASR 或 FunClip。FunClip 固定使用 Hugging Face 模型 `OpenMOSS-Team/MOSS-Transcribe-Diarize` 的 revision `e8681d68e7042738ffca8ac8212bc8fcb1131ab8`。先按[双语生产部署指南](https://www.funasr.com/en/deploy/moss-transcribe-diarize.html)启动并验证 vLLM 服务，再运行：
+
+```shell
+# vLLM 是默认后端，默认地址为 http://127.0.0.1:8898/v1
+python funclip/launch.py --model moss --moss-backend vllm
+
+# 可选凭据只从环境变量读取，不放入命令行参数
+MOSS_API_KEY=replace-me python funclip/launch.py --model moss
+```
+
+MOSS 端到端完成分段与说话人识别，不要再配置外部 `vad_model` 或 `spk_model`，否则切块会破坏全局说话人身份。它提供段级时间戳，适合生成 SRT、按说话人（`spkS01`、`spkS02` 等）剪辑以及 LLM 按时间剪辑；任意文本的精确剪辑仍应使用带 token 时间戳的 Paraformer。FunClip 当前只开放 vLLM 路径，因为它兼容标准 Transformers 4.x 环境，并且已经通过 OpenAI 转写接口的端到端测试。
+
 如果你只需要在 CPU 或边缘设备上离线转写语音，而不需要 FunClip 的视频剪辑界面，请优先使用 FunASR llama.cpp / GGUF 运行时：[funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · [Fun-ASR-Nano-GGUF](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-GGUF) · [SenseVoiceSmall-GGUF](https://huggingface.co/FunAudioLLM/SenseVoiceSmall-GGUF)。
 
 随后在浏览器中访问```localhost:7860```即可看到如下图所示的界面，按如下步骤即可进行视频剪辑

funclip/launch.py

@@ -12,6 +12,7 @@
 import gradio as gr
 from funasr import AutoModel
 from videoclipper import VideoClipper
+from model_selection import create_asr_model as _create_asr_model
 from llm.openai_api import openai_call
 from llm.qwen_api import call_qwen_model
 from llm.g4f_openai_api import g4f_openai_call
@@ -22,48 +23,34 @@
 from launch_config import build_launch_kwargs
 
 
-def create_asr_model(model_name, lang, auto_model_cls=AutoModel):
-    if model_name == "fun-asr-nano":
-        return auto_model_cls(
-            model="FunAudioLLM/Fun-ASR-Nano-2512",
-            trust_remote_code=True,
-            remote_code="./model.py",
-            vad_model="fsmn-vad",
-            vad_kwargs={"max_single_segment_time": 30000},
-            spk_model="cam++",
-            hub="hf",
-        )
-    if model_name == "sensevoice":
-        return auto_model_cls(
-            model="iic/SenseVoiceSmall",
-            vad_model="fsmn-vad",
-            vad_kwargs={"max_single_segment_time": 30000},
-            spk_model="cam++",
-        )
-
-    paraformer_model = (
-        "iic/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch"
-        if lang == "zh"
-        else "iic/speech_paraformer_asr-en-16k-vocab4199-pytorch"
-    )
-    return auto_model_cls(
-        model=paraformer_model,
-        vad_model="damo/speech_fsmn_vad_zh-cn-16k-common-pytorch",
-        punc_model="damo/punc_ct-transformer_zh-cn-common-vocab272727-pytorch",
-        spk_model="damo/speech_campplus_sv_zh-cn_16k-common",
+def create_asr_model(model_name, lang, auto_model_cls=AutoModel, **kwargs):
+    return _create_asr_model(
+        model_name, lang, auto_model_cls=auto_model_cls, **kwargs
     )
 
 
 if __name__ == "__main__":
     parser = argparse.ArgumentParser(description='argparse testing')
     parser.add_argument('--lang', '-l', type=str, default = "zh", help="language mode; selects the Paraformer checkpoint but does not override --model")
-    parser.add_argument('--model', '-m', type=str, default="paraformer", choices=["paraformer", "fun-asr-nano", "sensevoice"], help="ASR model: paraformer, fun-asr-nano, or sensevoice (takes precedence over --lang)")
+    parser.add_argument('--model', '-m', type=str, default="paraformer", choices=["paraformer", "fun-asr-nano", "sensevoice", "moss"], help="ASR model: paraformer, fun-asr-nano, sensevoice, or moss (takes precedence over --lang)")
+    parser.add_argument('--moss-backend', choices=["vllm"], default="vllm", help="MOSS runtime backed by an existing vLLM transcription service")
+    parser.add_argument('--moss-base-url', default="http://127.0.0.1:8898/v1", help="OpenAI-compatible base URL for the MOSS vLLM service")
+    parser.add_argument('--moss-api-key-env', default="MOSS_API_KEY", help="environment variable containing the optional MOSS service API key")
+    parser.add_argument('--moss-max-tokens', type=int, default=8192, help="MOSS generation limit for long recordings")
     parser.add_argument('--share', '-s', action='store_true', help="if to establish gradio share link")
     parser.add_argument('--port', '-p', type=int, default=7860, help='port number')
     parser.add_argument('--listen', action='store_true', help="if to listen to all hosts")
     args = parser.parse_args()
 
-    funasr_model = create_asr_model(args.model, args.lang)
+    moss_api_key = os.environ.get(args.moss_api_key_env) if args.moss_api_key_env else None
+    funasr_model = create_asr_model(
+        args.model,
+        args.lang,
+        moss_backend=args.moss_backend,
+        moss_base_url=args.moss_base_url,
+        moss_api_key=moss_api_key,
+        moss_max_tokens=args.moss_max_tokens,
+    )
     audio_clipper = VideoClipper(funasr_model)
     audio_clipper.lang = args.lang
 

funclip/model_selection.py

@@ -0,0 +1,89 @@
+MOSS_MODEL = "OpenMOSS-Team/MOSS-Transcribe-Diarize"
+MOSS_MODEL_REVISION = "e8681d68e7042738ffca8ac8212bc8fcb1131ab8"
+MOSS_DEFAULT_BASE_URL = "http://127.0.0.1:8898/v1"
+
+
+def _moss_model_kwargs(
+    backend="vllm",
+    base_url=MOSS_DEFAULT_BASE_URL,
+    api_key=None,
+    max_tokens=8192,
+):
+    backend = str(backend).lower()
+    if backend != "vllm":
+        raise ValueError(f"unsupported MOSS backend: {backend}")
+
+    max_tokens = int(max_tokens)
+    if max_tokens <= 0:
+        raise ValueError("MOSS generation token limit must be positive")
+
+    kwargs = {
+        "model": MOSS_MODEL,
+        "model_revision": MOSS_MODEL_REVISION,
+        "backend": backend,
+        "disable_update": True,
+    }
+    base_url = str(base_url or "").rstrip("/")
+    if not base_url:
+        raise ValueError("--moss-base-url is required for remote MOSS backends")
+
+    kwargs.update(
+        {
+            "vllm_base_url": base_url,
+            "vllm_model": "moss-transcribe-diarize",
+            "vllm_response_format": "json",
+            "max_completion_tokens": max_tokens,
+        }
+    )
+    if api_key:
+        kwargs["vllm_api_key"] = api_key
+    return kwargs
+
+
+def create_asr_model(
+    model_name,
+    lang,
+    auto_model_cls,
+    moss_backend="vllm",
+    moss_base_url=MOSS_DEFAULT_BASE_URL,
+    moss_api_key=None,
+    moss_max_tokens=8192,
+):
+    if model_name == "moss":
+        return auto_model_cls(
+            **_moss_model_kwargs(
+                backend=moss_backend,
+                base_url=moss_base_url,
+                api_key=moss_api_key,
+                max_tokens=moss_max_tokens,
+            )
+        )
+    if model_name == "fun-asr-nano":
+        return auto_model_cls(
+            model="FunAudioLLM/Fun-ASR-Nano-2512",
+            trust_remote_code=True,
+            remote_code="./model.py",
+            vad_model="fsmn-vad",
+            vad_kwargs={"max_single_segment_time": 30000},
+            spk_model="cam++",
+            hub="hf",
+        )
+    if model_name == "sensevoice":
+        return auto_model_cls(
+            model="iic/SenseVoiceSmall",
+            vad_model="fsmn-vad",
+            vad_kwargs={"max_single_segment_time": 30000},
+            spk_model="cam++",
+        )
+
+    paraformer_model = (
+        "iic/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch"
+        if lang == "zh"
+        else "iic/speech_paraformer_asr-en-16k-vocab4199-pytorch"
+    )
+    return auto_model_cls(
+        model=paraformer_model,
+        vad_model="damo/speech_fsmn_vad_zh-cn-16k-common-pytorch",
+        punc_model="damo/punc_ct-transformer_zh-cn-common-vocab272727-pytorch",
+        spk_model="damo/speech_campplus_sv_zh-cn_16k-common",
+    )

funclip/utils/trans_utils.py

@@ -33,6 +33,8 @@ def proc(raw_text, timestamp, dest_text, lang='zh'):
     ld = len(dest_text.split())
     normalized_raw_text = raw_text.translate(ASCII_LOWER_TABLE)
     normalized_dest_text = dest_text.translate(ASCII_LOWER_TABLE)
+    if not normalized_dest_text or not timestamp:
+        return []
     mi, ts = [], []
     offset = 0
     while True:
@@ -43,8 +45,11 @@ def proc(raw_text, timestamp, dest_text, lang='zh'):
         if fi == -1:
             break
         offset = fi + len(normalized_dest_text)
+        end_index = ti + ld - 1
+        if ti >= len(timestamp) or end_index >= len(timestamp):
+            continue
         mi.append(fi)
-        ts.append([timestamp[ti][0]*16, timestamp[ti+ld-1][1]*16])
+        ts.append([timestamp[ti][0]*16, timestamp[end_index][1]*16])
     return ts
 
 
@@ -54,7 +59,7 @@ def proc_spk(dest_spk, sd_sentences):
         d_start = d['timestamp'][0][0]
         d_end = d['timestamp'][-1][1]
         spkid=dest_spk[3:]
-        if str(d['spk']) == spkid and d_end-d_start>999:
+        if str(d['spk']) == spkid and d_end > d_start:
             ts.append([d_start*16, d_end*16])
     return ts