Benchmarks

2024 MacBook Pro, 48GB Ram, M4 Pro, Tahoe 26.0

Transcription

https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v3-coreml

swift run fluidaudiocli fleurs-benchmark --languages all --samples all

Language                  | WER%   | CER%   | RTFx    | Duration | Processed | Skipped
-----------------------------------------------------------------------------------------
Bulgarian (Bulgaria)      | 12.8   | 4.1    | 195.2   | 3468.0s  | 350       | -
Croatian (Croatia)        | 14.0   | 4.3    | 204.9   | 3647.0s  | 350       | -
Czech (Czechia)           | 12.0   | 3.8    | 214.2   | 4247.4s  | 350       | -
Danish (Denmark)          | 20.2   | 7.4    | 214.4   | 10579.1s | 930       | -
Dutch (Netherlands)       | 7.8    | 2.6    | 191.7   | 3337.7s  | 350       | -
English (US)              | 5.4    | 2.5    | 207.4   | 3442.9s  | 350       | -
Estonian (Estonia)        | 20.1   | 4.2    | 225.3   | 10825.4s | 893       | -
Finnish (Finland)         | 14.8   | 3.1    | 222.0   | 11894.4s | 918       | -
French (France)           | 5.9    | 2.2    | 199.9   | 3667.3s  | 350       | -
German (Germany)          | 5.9    | 1.9    | 220.9   | 4684.6s  | 350       | -
Greek (Greece)            | 36.9   | 13.7   | 183.0   | 6862.0s  | 650       | -
Hungarian (Hungary)       | 17.6   | 5.2    | 213.6   | 11050.9s | 905       | -
Italian (Italy)           | 4.0    | 1.3    | 236.7   | 5098.7s  | 350       | -
Latvian (Latvia)          | 27.1   | 7.5    | 217.8   | 10218.6s | 851       | -
Lithuanian (Lithuania)    | 25.0   | 6.8    | 202.8   | 10686.5s | 986       | -
Maltese (Malta)           | 25.2   | 9.3    | 217.4   | 12770.6s | 926       | -
Polish (Poland)           | 8.6    | 2.8    | 190.2   | 3409.6s  | 350       | -
Romanian (Romania)        | 14.4   | 4.7    | 200.4   | 9099.4s  | 883       | -
Russian (Russia)          | 7.2    | 2.2    | 209.7   | 3974.6s  | 350       | -
Slovak (Slovakia)         | 12.6   | 4.4    | 227.6   | 4169.6s  | 350       | -
Slovenian (Slovenia)      | 27.4   | 9.2    | 197.1   | 8173.1s  | 834       | -
Spanish (Spain)           | 4.5    | 2.2    | 221.7   | 4258.9s  | 350       | -
Swedish (Sweden)          | 16.8   | 5.0    | 219.5   | 8399.2s  | 759       | -
Ukrainian (Ukraine)       | 7.2    | 2.5    | 201.9   | 3853.7s  | 350       | -
-----------------------------------------------------------------------------------------
AVERAGE                   | 14.7   | 4.7    | 209.8   | 161819.2 | 14085     | -

Dataset: librispeech test-clean
Files processed: 2620
Average WER: 2.5%
Median WER: 0.0%
Average CER: 1.0%
Median RTFx: 139.6x
Overall RTFx: 155.6x (19452.5s / 125.0s)

swift run fluidaudiocli asr-benchmark --max-files all --model-version v2

Use v2 if you only need English, it is a bit more accurate

--- Benchmark Results ---
   Dataset: librispeech test-clean
   Files processed: 2620
   Average WER: 2.1%
   Median WER: 0.0%
   Average CER: 0.7%
   Median RTFx: 128.6x
   Overall RTFx: 145.8x (19452.5s / 133.4s)

ASR Model Compilation

Core ML first-load compile times captured on iPhone 16 Pro Max and iPhone 13 running the parakeet-tdt-0.6b-v3-coreml bundle. Cold-start compilation happens the first time each Core ML model is loaded; subsequent loads hit the cached binaries. Warm compile metrics were collected only on the iPhone 16 Pro Max run, and only for models that were reloaded during the session.

Model	iPhone 16 Pro Max cold (ms)	iPhone 16 Pro Max warm (ms)	iPhone 13 cold (ms)	Compute units
Preprocessor	9.15	-	632.63	MLComputeUnits(rawValue: 2)
Encoder	3361.23	162.05	4396.00	MLComputeUnits(rawValue: 1)
Decoder	88.49	8.11	146.01	MLComputeUnits(rawValue: 1)
JointDecision	48.46	7.97	71.85	MLComputeUnits(rawValue: 1)

Parakeet Unified (English, batch + streaming)

Unified FastConformer-RNNT — one checkpoint serves both offline batch and chunked-attention streaming, English with punctuation and capitalization. Greedy RNNT decode (no TDT duration head); batch and streaming share the same decoder and differ only in the encoder window (offline 15 s full-attention vs streaming 7.68 s chunked).

Model: FluidInference/parakeet-unified-en-0.6b-coreml

Hardware: Apple M5 Pro, macOS 26. Encoder int8 on ANE (.cpuAndNeuralEngine).

iOS note: the int8 encoder is verified on M-series only. On A16 (iPhone 14 Pro) it fails to load on every compute unit — including .cpuOnly — even from an intact download (#828). Use encoderPrecision: .fp16 on iOS; it loads and transcribes on the same device.

LibriSpeech test-clean (2620 files, 5.40h audio)

Mode	WER (Avg)	Aggregate WER	Median WER	Overall RTFx	Median RTFx	Long files (>15s)
Batch	2.15%	1.68%	0.00%	123.3x	111.5x	238
Streaming	2.21%	1.79%	0.00%	29.1x	53.1x	238

Same harness and TextNormalizer as asr-benchmark, so directly comparable to the Transcription numbers above: Parakeet TDT v3 = 2.6% Avg WER / 110x RTFx (multilingual, no punctuation). For English files, Unified batch wins on WER, throughput, and punctuation; TDT v3 remains the multilingual option.

Avg WER is the mean of per-file WER (matches asr-benchmark); Aggregate WER is total errors ÷ total words.
Long files (> 15 s) are not skipped — batch uses overlapping 15 s windows merged on a 2 s overlap; streaming runs them as one continuous session.
Streaming's overall RTFx falls below its median because it re-encodes a 7.68 s window per 1.04 s chunk (the latency tax) — long files amortize that poorly. Batch only re-encodes the 2 s overlap, so throughput stays flat. Use batch for files, streaming for live audio.
int8 encoder is WER-lossless vs fp16 (within noise) at half the size.

# Full benchmark, both modes (auto-downloads dataset + models)
swift run -c release fluidaudiocli unified-benchmark --mode both

# Single mode, limited files, or fp16 encoder
swift run -c release fluidaudiocli unified-benchmark --mode streaming --max-files 100
swift run -c release fluidaudiocli unified-benchmark --mode batch --precision fp16

Nemotron Speech Streaming 0.6B (English)

Cache-aware FastConformer-RNNT streaming, English. Mel features are computed natively in Swift (NemotronMelExtractor → AudioMelSpectrogram, NeMo normalize: NA raw log-mel) — there is no CoreML preprocessor stage. It was removed in the issue #739 fix: the preprocessor's flexible RangeDim audio input was the source of the ios17.slice_by_index: zero shape error ("Skipped adding default_function to entry point: main") ANE warning behind the iPadOS cold-start empty-transcript failure. Encoder int8 on ANE (.cpuAndNeuralEngine).

Model: FluidInference/nemotron-speech-streaming-en-0.6b-coreml

LibriSpeech test-clean (2620 files, 53,120 words, ~5.4h audio)

Chunk tier	Aggregate WER	RTFx	Errors / words
560 ms (lowest latency)	2.71%	40.7x	1442 / 53120
1120 ms (trained chunk)	2.58%	24.3x	1369 / 53120
2240 ms (default)	2.64%	87.4x	1403 / 53120

WER is aggregate (total errors ÷ total words across all 2620 files).
RTFx is end-to-end single-stream (Swift mel + int8 ANE encode + greedy RNN-T), release build, Apple Silicon; absolute RTFx is machine/load-dependent, relative ordering is stable.
Accuracy is essentially flat across tiers (2.58–2.71%). 1120 ms has the best WER but lowest throughput; 2240 ms (default) is the throughput sweet spot, within ~0.06 pp of the best WER.
Parity: NemotronMelExtractor matches NeMo PyTorch raw log-mel to max |Δ| ≈ 9e-3 — the WER here confirms end-to-end correctness (a wrong mel front-end would collapse WER).
Multilingual FLEURS results: see NemotronMultilingual.md.

swift run -c release fluidaudiocli nemotron-benchmark --subset test-clean --chunk <560|1120|2240>

Transcription with Keyword Boosting

CTC-based custom vocabulary boosting system, which enables accurate recognition of domain-specific terms (company names, technical jargon, proper nouns) without retraining the ASR model.

# Download the dataset
swift run fluidaudiocli ctc-earnings-benchmark --auto-download

# Run the benchmark
swift run fluidaudiocli ctc-earnings-benchmark

Earnings Benchmark (TDT transcription + CTC keyword spotting)
  Data directory: /Users/<user>/Library/Application Support/FluidAudio/earnings22-kws/test-dataset
  Output file: ctc_earnings_benchmark.json
  TDT version: v2
  CTC model: /Users/<user>/Library/Application Support/FluidAudio/Models/parakeet-ctc-110m-coreml
Loading TDT models (v2) for transcription...
TDT models loaded successfully
Loading CTC models from: /Users/<user>/Library/Application Support/FluidAudio/Models/parakeet-ctc-110m-coreml
Loaded CTC vocabulary with 1024 tokens, variant: Parakeet CTC 110M (hybrid)
Created CTC spotter with blankId=1024
Processing 773 test files...
[  1/772] 4329526_chunk0            WER:  10.3%  Dict: 1/1
[  2/772] 4329526_chunk109          WER:  12.5%  Dict: 2/2
[  3/772] 4329526_chunk118          WER:   3.1%  Dict: 3/3
[  4/772] 4329526_chunk132          WER:   8.1%  Dict: 1/1
[  5/772] 4329526_chunk135          WER:  25.7%  Dict: 1/1
[  6/772] 4329526_chunk16           WER:   8.6%  Dict: 1/1
...
[767/772] 4485206_chunk_86          WER:   5.0%  Dict: 2/2
[768/772] 4485206_chunk_88          WER:   8.3%  Dict: 2/2
[769/772] 4485206_chunk_92          WER:  14.7%  Dict: 4/4
[770/772] 4485206_chunk_97          WER:  30.5%  Dict: 1/1
[771/772] 4485206_chunk_98          WER:  18.6%  Dict: 4/4
[772/772] 4485206_chunk_99          WER:  22.0%  Dict: 1/1

============================================================
EARNINGS22 BENCHMARK (TDT + CTC)
============================================================
Model: /Users/<user>/Library/Application Support/FluidAudio/Models/parakeet-ctc-110m-coreml
Total tests: 771
Average WER: 15.00%
Dict Pass (Recall): 1299/1308 (99.3%)
Vocab Precision: 99.3% (TP=1068, FP=8)
Vocab Recall: 85.2% (TP=1068, FN=185)
Vocab F-score: 91.7%
Total audio: 11564.5s
Total processing: 182.5s
RTFx: 63.36x
============================================================

Results written to: ctc_earnings_benchmark.json

In context of vocabulary/keyword detection:

Metric	Definition
TP (True Positive)	Word is in reference AND in hypothesis (correctly detected)
FP (False Positive)	Word is in hypothesis but NOT in reference (hallucinated/wrong)
FN (False Negative)	Word is in reference but NOT in hypothesis (missed)

Derived metrics:

Metric	Formula	Meaning
Precision	TP / (TP + FP)	"Of words we output, how many were correct?"
Recall	TP / (TP + FN)	"Of words that should appear, how many did we find?"
F-Score	2 × P × R / (P + R)	Harmonic mean of precision and recall

Text-to-Speech

We generated the same strings with to generate audio between 1s to ~300s in order to test the speed across a range of varying inputs on Pytorch CPU, MPS, and MLX pipeline, and compared it against the native Swift version with Core ML models.

Each pipeline warmed up the models by running through it once with pesudo inputs, and then comparing the raw inference time with the model already loaded. You can see that for the Core ML model, we traded lower memory and very slightly faster inference for longer initial warm-up.

Note that the Pytorch kokoro model in Pytorch has a memory leak issue: hexgrad/kokoro#152

The following tests were ran on M4 Pro, 48GB RAM, Macbook Pro. If you have another device, please do try replicating it as well!

Kokoro-82M PyTorch (CPU)

KPipeline benchmark for voice af_heart (warm-up took 0.175s) using hexgrad/kokoro
Test   Chars    Output (s)   Inf(s)       RTFx       Peak GB
1      42       2.750        0.187        14.737x    1.44
2      129      8.625        0.530        16.264x    1.85
3      254      15.525       0.923        16.814x    2.65
4      93       6.125        0.349        17.566x    2.66
5      104      7.200        0.410        17.567x    2.70
6      130      9.300        0.504        18.443x    2.72
7      197      12.850       0.726        17.711x    2.83
8      6        1.350        0.098        13.823x    2.83
9      1228     76.200       4.342        17.551x    3.19
10     567      35.200       2.069        17.014x    4.85
11     4615     286.525      17.041       16.814x    4.78
Total  -        461.650      27.177       16.987x    4.85

Kokoro-82M PyTorch (MPS)

I wasn't able to run the MPS model for longer durations, even with PYTORCH_ENABLE_MPS_FALLBACK=1 enabled, it kept crashing for the longer strings.

KPipeline benchmark for voice af_heart (warm-up took 0.568s) using pip package
Test   Chars    Output (s)   Inf(s)       RTFx       Peak GB
1      42       2.750        0.414        6.649x     1.41
2      129      8.625        0.729        11.839x    1.54
Total  -        11.375       1.142        9.960x     1.54

Kokoro-82M MLX Pipeline

TTS benchmark for voice af_heart (warm-up took an extra 2.155s) using model prince-canuma/Kokoro-82M
Test   Chars    Output (s)   Inf(s)       RTFx       Peak GB
1      42       2.750        0.347        7.932x     1.12
2      129      8.650        0.597        14.497x    2.47
3      254      15.525       0.825        18.829x    2.65
4      93       6.125        0.306        20.039x    2.65
5      104      7.200        0.343        21.001x    2.65
6      130      9.300        0.560        16.611x    2.65
7      197      12.850       0.596        21.573x    2.65
8      6        1.350        0.364        3.706x     2.65
9      1228     76.200       2.979        25.583x    3.29
10     567      35.200       1.374        25.615x    3.37
11     4615     286.500      11.112       25.783x    3.37
Total  -        461.650      19.401       23.796x    3.37

Swift + Fluid Audio Core ML models

Note that it does take ~15s to compile the model on the first run, subsequent runs are shorter, we expect ~2s to load.

> swift run fluidaudiocli tts --benchmark
...
FluidAudio TTS benchmark for voice af_heart (warm-up took an extra 2.348s)
Test   Chars    Ouput (s)    Inf(s)       RTFx
1      42       2.825        0.440        6.424x
2      129      7.725        0.594        13.014x
3      254      13.400       0.776        17.278x
4      93       5.875        0.587        10.005x
5      104      6.675        0.613        10.889x
6      130      8.075        0.621        13.008x
7      197      10.650       0.627        16.983x
8      6        0.825        0.360        2.290x
9      1228     67.625       2.362        28.625x
10     567      33.025       1.341        24.619x
11     4269     247.600      9.087        27.248x
Total  -        404.300      17.408       23.225

Peak memory usage (process-wide): 1.503 GB

Voice Activity Detection

Model is nearly identical to the base model in terms of quality, performance wise we see an up to ~3.5x improvement compared to the silero Pytorch VAD model with the 256ms batch model (8 chunks of 32ms)

FSMN-VAD (`fsmn-vad-segment`)

Beta: FSMN-VAD is a beta model conversion; results and model artifacts may change.

CoreML FSMN-VAD (FunASR, ~5.2M), an alternative to silero-vad. Model: FluidInference/fsmn-vad-coreml. 2-stage: fbank80+LFR preprocessor (fp32/CPU) → FSMN scorer (fp16/ANE, enumerated buckets) → host decision (port of FunASR FsmnVADStreaming). Hardware: Apple M5 Pro.

Evaluated on the mini50 labeled set via the standard vad-benchmark harness (per-clip speech/non-speech), same metric as the silero baseline:

Backend	Accuracy	Precision	Recall	F1	RTFx
silero (baseline)	82.0%	73.5%	100%	84.7%	1408×
FSMN-VAD	98.0%	96.2%	100%	98.0%	640×

FSMN-VAD is far more precise (96.2% vs 73.5%) at the same 100% recall — many fewer false speech detections — at ~640× real-time. Fidelity vs FunASR's own segments: frame F1 97.4%, boundaries within ~50 ms (vad_bench.py in the conversion repo).

Full FluidInference/musan noise set (774 noise clips) — noise rejection / specificity (correctly classified non-speech):

Backend	Noise rejected (specificity)	False-positive rate	RTFx
silero	69.8%	30.2%	1341×
FSMN-VAD	81.9%	18.1%	571×

On the full MUSAN noise set FSMN-VAD rejects 12 pp more noise as non-speech (18% vs 30% false positives) — consistently more precise than silero on both the balanced (mini50) and noise-heavy (full MUSAN) evaluations.

Long audio is processed in ~30 s chunks (the FSMN's dilated conv needs fixed shapes; RangeDim is rejected by the ANE/BNNS compiler).

swift run -c release fluidaudiocli vad-benchmark --dataset mini50 --backend fsmn
swift run -c release fluidaudiocli fsmn-vad-segment audio.wav

Silero VAD v6.2.1 Core ML preflight benchmark on an Apple M1 MacBook Air (macOS 26.5.1), using the same mini50 dataset and threshold as the VAD CI workflow:

swift run -c release fluidaudiocli vad-benchmark --dataset mini50 --all-files --threshold 0.5 --output musan_vad_results.json

Model	Accuracy	Precision	Recall	F1-Score	Total time	RTFx
Upstream baseline	92.0%	86.2%	100.0%	92.6%	2.19s	1117.2x
Silero VAD v6.2.1	94.0%	89.3%	100.0%	94.3%	2.27s	1077.9x

Dataset: https://github.com/Lab41/VOiCES-subset

swift run fluidaudiocli vad-benchmark --dataset voices-subset --all-files --threshold 0.85
...
Timing Statistics:
[18:56:31.208] [INFO] [VAD]    Total processing time: 0.29s
[18:56:31.208] [INFO] [VAD]    Total audio duration: 351.05s
[18:56:31.208] [INFO] [VAD]    RTFx: 1230.6x faster than real-time
[18:56:31.208] [INFO] [VAD]    Audio loading time: 0.00s (0.6%)
[18:56:31.208] [INFO] [VAD]    VAD inference time: 0.28s (98.7%)
[18:56:31.208] [INFO] [VAD]    Average per file: 0.011s
[18:56:31.208] [INFO] [VAD]    Min per file: 0.001s
[18:56:31.208] [INFO] [VAD]    Max per file: 0.020s
[18:56:31.208] [INFO] [VAD]
VAD Benchmark Results:
[18:56:31.208] [INFO] [VAD]    Accuracy: 96.0%
[18:56:31.208] [INFO] [VAD]    Precision: 100.0%
[18:56:31.208] [INFO] [VAD]    Recall: 95.8%
[18:56:31.208] [INFO] [VAD]    F1-Score: 97.9%
[18:56:31.208] [INFO] [VAD]    Total Time: 0.29s
[18:56:31.208] [INFO] [VAD]    RTFx: 1230.6x faster than real-time
[18:56:31.208] [INFO] [VAD]    Files Processed: 25
[18:56:31.208] [INFO] [VAD]    Avg Time per File: 0.011s

swift run fluidaudiocli vad-benchmark --dataset musan-full --num-files all --threshold 0.8
...
[23:02:35.539] [INFO] [VAD] Total processing time: 322.31s
[23:02:35.539] [INFO] [VAD] Timing Statistics:
[23:02:35.539] [INFO] [VAD] RTFx: 1220.7x faster than real-time
[23:02:35.539] [INFO] [VAD] Audio loading time: 1.20s (0.4%)
[23:02:35.539] [INFO] [VAD] VAD inference time: 319.57s (99.1%)
[23:02:35.539] [INFO] [VAD] Average per file: 0.160s
[23:02:35.539] [INFO] [VAD] Total audio duration: 393442.58s
[23:02:35.539] [INFO] [VAD] Min per file: 0.000s
[23:02:35.539] [INFO] [VAD] Max per file: 0.873s
[23:02:35.711] [INFO] [VAD] VAD Benchmark Results:
[23:02:35.711] [INFO] [VAD] Accuracy: 94.2%
[23:02:35.711] [INFO] [VAD] Precision: 92.6%
[23:02:35.711] [INFO] [VAD] Recall: 78.9%
[23:02:35.711] [INFO] [VAD] F1-Score: 85.2%
[23:02:35.711] [INFO] [VAD] Total Time: 322.31s
[23:02:35.711] [INFO] [VAD] RTFx: 1220.7x faster than real-time
[23:02:35.711] [INFO] [VAD] Files Processed: 2016
[23:02:35.711] [INFO] [VAD] Avg Time per File: 0.160s
[23:02:35.744] [INFO] [VAD] Results saved to: vad_benchmark_results.json

SenseVoice

Non-autoregressive multilingual ASR using SenseVoiceSmall (FunASR, ~234M) converted to CoreML — SANM encoder + single CTC head, all tokens in one forward pass. See ASR/SenseVoice.md for the architecture and conversion notes.

Model: FluidInference/sensevoice-small-coreml

Hardware: Apple M5 Pro, macOS 26. FP16 encoder on the Neural Engine (CPU_AND_NE); FP32 CPU front-end. Full canonical test sets, directly comparable to the published SenseVoice-Small results.

LibriSpeech test-clean (2620 files)

Metric	CoreML (ANE)	Official SenseVoice-Small
WER (Avg)	3.22%	~3.1%
Median RTFx	299x	—

AISHELL-1 Chinese (7176 files)

Metric	CoreML (ANE)	Official SenseVoice-Small
CER (Avg)	3.09%	~2.9%
Median RTFx	382x	—

int8 encoder variant (`--int8`)

Post-training weight quantization of the encoder — ~half the size, accuracy-neutral vs fp16 (run on ANE). Full canonical test sets:

	size	LibriSpeech WER	AISHELL CER	peak RAM
fp16 (default)	447 MB	3.22%	3.09%	0.54 GB
int8	225 MB	3.25%	3.09%	0.32 GB

(Δ +0.03 pp / 0.00 pp on the full LibriSpeech test-clean (2,620) / AISHELL-1 test (7,176), 0 NaN.) int4 per-tensor palettization wrecks accuracy (WER 31%) and is not shipped.

Methodology notes:

CER (character-level, whitespace removed) is the primary metric for Chinese, matching the official SenseVoice chart (AISHELL-1 test).
Both numbers reproduce the published SenseVoice-Small results, confirming the CoreML conversion (front-end + encoder + decode) is faithful.
CoreML↔PyTorch parity additionally verified on FLEURS: en WER Δ +0.00pp, zh CER Δ −0.03pp (100 samples/lang).
The FP16 encoder is correct only on the Neural Engine (NaN on the CPU/GPU FP16 path); non-ANE hardware uses the --fp32 build. See ASR/SenseVoice.md.
AISHELL-1 dataset: TwinkStart/AISHELL-1.

# FLEURS WER/CER (in-repo, multilingual)
swift run -c release fluidaudiocli sensevoice-benchmark --languages en_us,cmn_hans_cn --samples all

Paraformer

Non-autoregressive Mandarin (zh) ASR: SANM encoder + CIF predictor (host integrate-and-fire) + parallel decoder. See ASR/Paraformer.md.

Model: FluidInference/paraformer-large-zh-coreml

Hardware: Apple M5 Pro, macOS 26. Encoder/CifAlphas/decoder on ANE; FP32 CPU front-end.

AISHELL-1 Chinese (7176 files, full test, full-CoreML pipeline)

Precision	size (enc+dec)	CER	median RTFx	peak RAM	Official
fp16 (default)	411 MB	2.12%	85×	0.38 GB	~1.95%
int8	207 MB	2.12%	84×	0.24 GB	~1.95%

Methodology notes:

CER (character-level, whitespace removed) is the primary metric for Chinese, matching the official Paraformer-large AISHELL-1 number.
int8 weight quantization (encoder + decoder) is accuracy-neutral (CER unchanged on the full set), ~half the size/memory.
The ~0.17 pp gap vs official is fp16 + the fixed-shape decoder (enc 512 / tokens 128). RTFx (~85×) is lower than SenseVoice (~400×) because Paraformer runs 3 CoreML predicts/clip + the decoder pads short clips to 512 frames — an enumerated decoder would raise it.
AISHELL-1 dataset: TwinkStart/AISHELL-1.

swift run -c release fluidaudiocli paraformer-transcribe audio.wav         # fp16
swift run -c release fluidaudiocli paraformer-transcribe audio.wav --int8  # half size

Streaming ASR (Parakeet EOU)

Real-time streaming ASR with End-of-Utterance detection using the Parakeet EOU 120M CoreML model.

Model: FluidInference/parakeet-realtime-eou-120m-coreml

Hardware: Apple M2, 2022, macOS 26

LibriSpeech test-clean (2620 files, 5.40h audio)

Chunk Size	WER (Avg)	Median WER	RTFx	Total Time
320ms	4.88%	0.00%	19.25x	1015s (16.9m)
160ms	8.23%	5.26%	5.78x	3387s (56.4m)

# Run 320ms benchmark
swift run -c release fluidaudiocli parakeet-eou --benchmark --chunk-size 320 --use-cache

# Run 160ms benchmark
swift run -c release fluidaudiocli parakeet-eou --benchmark --chunk-size 160 --use-cache

Streaming ASR (Nemotron, English)

NVIDIA's Nemotron Speech Streaming 0.6B for streaming ASR. The default tier is now 2240ms with B1-fused decode (decoder+joint merged into one CoreML call per step) — it trades ~1.1 s of chunk latency for throughput at no accuracy cost. Pass an explicit NemotronChunkSize / --chunk (1120/560/160/80) for lower-latency tiers.

Model: FluidInference/nemotron-speech-streaming-en-0.6b-coreml

Hardware: Apple M5 Pro, macOS 26.5. Encoder int8 on ANE (.cpuAndNeuralEngine).

LibriSpeech test-clean (100 files)

Three tiers, all from one conversion with B1-fused decode (decoder_joint.mlmodelc):

Tier	WER	RTFx	Δ vs 1120ms
560ms	2.28%	42.1	−35%
1120ms	2.28%	65.0	—
2240ms (default)	2.46%	93.6	+44%

WER is neutral across tiers (within n=100 noise). 2240ms = 2× the trained 14-encoder-frame chunk (the chunked-attention mask still tiles cleanly); B1 fusion = one CoreML call per decode step instead of two (~+15% on any tier shipping decoder_joint.mlmodelc). The v1 160ms/80ms tiers were removed (off-tiling, degraded WER).

Encoder optimization notes (M5 Pro):

6-bit palettization beats int8 on every axis — 2.24% WER, +9% RTFx, smaller (422 MB vs 564 MB). Planned follow-up to replace the shipped int8 encoder.
Encoder placement / iOS: ANE gives the fastest inference but slowest load (the iOS ~1.4 GB / ~130 s residency wall for the 24-layer encoder). On iOS, running the encoder on CPU (instant load, ~140 MB, ~66 RTFx) is ~2× faster than 4-way ANE sharding (~33 RTFx) — so CPU, not sharding, is the iOS encoder choice. macOS / plugged-in stays on ANE.

All three tiers are a faithful conversion of the public nvidia/nemotron-speech-streaming-en-0.6b checkpoint (decoder & joint match PyTorch at cos=1.0) and replace the previous v1 tiers. WER parity against NVIDIA's internal tuning of the same model is a tracked follow-up; the ladder above is internally consistent (one conversion for all tiers) and reports the relative gains.

# Default (2240ms + B1)
swift run -c release fluidaudiocli nemotron-benchmark --max-files 100

# Lower-latency tier
swift run -c release fluidaudiocli nemotron-benchmark --chunk 1120 --max-files 100

Streaming ASR (Nemotron Multilingual)

NVIDIA's Nemotron 3.5 ASR Streaming Multilingual 0.6B — real-time streaming RNN-T covering ~40 language-locales, fully on-device. Two models share one encoder per tier: latin (en/es/fr/it/pt/de, 2,828-token script-pruned vocab) and multilingual (zh/ja + 100+ via prompt_id, full 13,087 vocab).

Model: FluidInference/Nemotron-3.5-ASR-Streaming-Multilingual-0.6b-CoreML

Hardware: Apple M5 Pro, macOS 26.5. Encoder/decoder/joint on ANE (.cpuAndNeuralEngine), CoreML iOS 17 target. Per-file sum-aggregate RTFx, 2.24 s (2240 ms) tier, B1 fused decode.

LibriSpeech test-clean (English, 2620 files, 5.40h audio)

Model	Vocab	WER	RTFx
`latin`	2,828	3.6%	124x
`multilingual`	13,087	3.2%	76x

latin is ~1.6× faster than the full-vocab model on the same English audio (smaller per-frame joint matmul) at ~0.4 pp WER. English WER uses the HF EnglishTextNormalizer (Open ASR Leaderboard convention).

FLEURS (full test splits, 2.24 s tier)

Language	Model	WER / CER	RTFx
English (en)	`latin`	8.96%	130x
Spanish (es)	`latin`	4.80%	140x
French (fr)	`latin`	9.52%	130x
Italian (it)	`latin`	5.41%	147x
Portuguese (pt)	`latin`	6.14%	141x
German (de)	`latin`	9.83%	144x
Chinese (zh)	`multilingual`	18.57% CER	89x
Japanese (ja)	`multilingual`	13.79% CER	84x

FLEURS is multi-domain and digit-bearing, so it runs higher than test-clean for the same model. Reference and hypothesis are normalized with text-processing-rs — FluidInference's Rust port of NVIDIA NeMo's (inverse) text-normalization grammars (~98.6% NeMo-suite compatibility) — to match NVIDIA's FLEURS scoring; zh/ja are scored as CER. (The nemotron-multilingual-benchmark CLI's built-in scorer uses a lighter Swift normalizer, so non-English numbers it prints may differ slightly from these.) The full-vocab multilingual model is chunk-sensitive — use the 2 s tier for zh/ja.

# LibriSpeech test-clean (English)
swift run -c release fluidaudiocli nemotron-multilingual-benchmark \
  --dataset librispeech --librispeech-subset test-clean --model-dir <model-dir>

# FLEURS per-language
swift run -c release fluidaudiocli nemotron-multilingual-benchmark \
  --dataset fleurs --languages en_us,es_419,fr_fr,it_it,pt_br,de_de,cmn_hans_cn,ja_jp \
  --model-dir <model-dir>

Speaker Diarization

Both offline and online versions use the community-1 model (via FluidInference/speaker-diarization-coreml).

CAM++ speaker embedding (`campplus-embed`)

Beta: CAM++ is a beta model conversion; results and model artifacts may change.

CoreML CAM++ (FunASR, ~7.2M) speaker-embedding extractor. Model: FluidInference/campplus-coreml. 2-stage: fbank80 preprocessor (fp32/CPU) → CAM++ (RangeDim, CPU/GPU) → 192-d L2-normalized embedding. Hardware: Apple M5 Pro.

Metric	Value
AISHELL-1 EER	0.48%
Same-speaker cosine (mean)	0.805
Different-speaker cosine (mean)	0.256
Trial set	20 speakers, 6000 same / 6000 different pairs

Notes:

EER on AISHELL-1 (clean, read Mandarin) — easier than the official CN-Celeb benchmark (~6–7%); this validates the CoreML embedding discriminates speakers (CoreML↔torch embedding cosine 0.9997–0.99999).
Speaker id parsed from the AISHELL name field (BAC009S0764W... → S0764).

swift run -c release fluidaudiocli campplus-embed a.wav b.wav   # cosine similarity

Offline diarization pipeline

For slightly ~1.2% worse DER we default to a higher step ratio segmentation duration than the baseline community-1 pipeline. This allows us to get nearly ~2x the speed (as expected because we're processing 1/2 of the embeddings). For highly critical use cases, one may should use step ratio = 0.1 and minSegmentDurationSeconds = 0.0

Running on the full voxconverse benchmark:

StepRatio = 0.2, minSegmentDurationSeconds= 1.0
Average DER: 15.07% | Median DER: 10.70% | Average JER: 39.40% | Median JER: 40.95% (collar=0.25s, ignoreOverlap=True)
Average RTFx: 122.06 (from 232 clips)
Completed. New results: 232, Skipped existing: 0, Total attempted: 232
Step Ratio 2, min duration 1.0


StepRatio = 0.1, minSegmentDurationSeconds= 0
Average DER: 13.89% | Median DER: 10.49% | Average JER: 42.84% | Median JER: 43.30% (collar=0.25s, ignoreOverlap=True)
Average RTFx: 64.75 (from 232 clips)
Completed. New results: 232, Skipped existing: 0, Total attempted: 232
Step Ratio 1, min duration 0 (edited)

Note that the baseline pytorch version is ~11% DER, we lost some precision dropping down to fp16 precision in order to run most of the embedding model on neural engine. But as a result, we significantly out perform the baseline mps backend as well. the pyannote-community-1 on cpu is ~1.5-2 RTFx, on mps, it's ~20-25 RTFx.

Running on the full AMI SDM 16-meeting test set (official NeMo/pyannote evaluation split: EN2002, ES2004, IS1009, TS3003 × a-d). Re-run 2026-07-03 on an Apple M5 Pro:

swift run -c release fluidaudiocli diarization-benchmark --mode offline \
    --dataset ami-sdm --auto-download --threshold 0.7

------------------------------------------------------------------------------------------
Meeting        DER %    JER %    Miss %     FA %     SE %   Speakers     RTFx
------------------------------------------------------------------------------------------
IS1009c           5.1      5.9       3.1      1.5      0.6     4/4        335.3
IS1009b           5.4      6.4       2.8      1.4      1.1     4/4        337.1
ES2004b           6.0      7.0       2.7      2.2      1.1     4/4        314.0
ES2004c           6.4      7.3       2.0      3.4      1.0     4/4        304.0
EN2002c           7.8      9.7       5.1      0.5      2.2     3/3        305.0
TS3003b           8.0      7.8       3.6      3.7      0.7     4/4        307.3
TS3003c           9.0      8.7       6.1      1.9      0.9     4/4        350.0
EN2002b           9.1     12.9       4.0      1.9      3.2     5/4        318.5
IS1009d           9.2     11.7       4.5      2.6      2.2     4/4        320.9
IS1009a           9.9     11.9       5.0      2.5      2.4     4/4        342.9
ES2004a          10.4     13.4       7.5      1.6      1.4     4/4        334.0
EN2002a          10.6     15.0       5.4      1.2      4.0     4/4        298.7
ES2004d          11.4     16.4       5.3      2.6      3.5     4/4        329.4
TS3003a          17.2     64.1      13.1      1.3      2.8     2/4        346.4
EN2002d          18.3     38.2       4.6      1.5     12.2     3/4        302.5
TS3003d          26.0     41.6      11.0      2.2     12.8     3/4        324.6
------------------------------------------------------------------------------------------
AVERAGE          10.6     17.4       5.4      2.0      3.3      -        323.2
==========================================================================================

12/16 meetings detect the correct speaker count. Average DER 10.62% matches published pyannote-community-1 offline numbers on this split (~11-12%). Results are fully deterministic (two consecutive runs produce bit-identical metrics).

Clustering threshold matters on this split. The table above predates #801 and uses --threshold 0.7 under the old (inverted) threshold semantics. Since #801 the threshold is a Euclidean cut distance applied directly to the AHC dendrogram (pyannote parity): larger values merge more aggressively and yield fewer speakers. Old values map to new ones via sqrt(2 − 2·old) — the old default 0.6 behaved like a cut at 0.894, and the old --threshold 0.7 from this table behaves like --threshold 0.775 today. The CLI default remains the community-1 preset value (0.6), now interpreted as pyannote does.

Streaming/online Diarization

This is more tricky and honestly a lot more fragile to clustering. Expect substantially worse DER than the offline pipeline — the tail meetings suffer heavy speaker confusion. Only use this when you critically need realtime streaming speaker diarization. In most cases, offline is more than enough for most applications.

All streaming tables below were re-run 2026-07-03 on an Apple M5 Pro against the official AMI SDM 16-meeting test set (same split as the offline table above; earlier revisions of these tables used a different 7-meeting subset, so numbers are not directly comparable to those).

Running a near real-time diarization benchmark for 3s chunks, 1s overlap, and 0.85 clustering threshold:

swift run fluidaudiocli diarization-benchmark --mode streaming \
    --dataset ami-sdm \
    --threshold 0.85 \
    --auto-download \
    --chunk-seconds 3.0 \
    --overlap-seconds 1.0
    
...

------------------------------------------------------------------------------------------
Meeting        DER %    JER %    Miss %     FA %     SE %   Speakers     RTFx
------------------------------------------------------------------------------------------
TS3003a          19.3     37.0       7.3      3.0      9.1     5/4         26.1
ES2004d          21.7     36.9       4.8      3.1     13.8     6/4         24.5
IS1009d          24.4     38.4       2.0      4.6     17.8     10/4        24.7
IS1009a          29.8     46.0       2.8      3.7     23.4     6/4         25.3
TS3003c          31.6     45.9       4.2      3.4     24.0     5/4         25.4
ES2004a          31.7     41.6       6.7      2.1     22.8     7/4         24.9
TS3003b          55.3     63.5       3.0      5.3     47.0     5/4         24.4
EN2002d          60.3     62.8       5.3      2.2     52.8     7/4         23.6
IS1009c          61.0     70.7       1.4      3.8     55.8     6/4         24.7
EN2002a          63.0     62.2       4.8      2.0     56.2     7/4         23.4
EN2002c          67.0     70.9       4.7      2.0     60.4     9/3         23.7
ES2004b          69.5     71.2       3.2      2.6     63.8     9/4         24.2
ES2004c          70.8     68.2       2.5      3.2     65.2     7/4         24.2
TS3003d          74.8     89.2       7.0      3.5     64.3     7/4         24.9
EN2002b          83.9     84.7       3.6      2.3     77.9     11/4        24.2
IS1009b          88.1     90.8       1.3      2.3     84.5     7/4         24.7
------------------------------------------------------------------------------------------
AVERAGE          53.3     61.2       4.0      3.1     46.2      -         24.6
==========================================================================================

Diarization benchmark with 10s chunks, 0s overlap, and 0.7 clustering threshold (best streaming configuration found on this split):

swift run fluidaudiocli diarization-benchmark --mode streaming \
    --dataset ami-sdm \
    --threshold 0.7 \
    --auto-download \
    --chunk-seconds 10.0 \
    --overlap-seconds 0.0

...

------------------------------------------------------------------------------------------
Meeting        DER %    JER %    Miss %     FA %     SE %   Speakers     RTFx
------------------------------------------------------------------------------------------
ES2004a          15.1     24.8       9.2      1.2      4.7     4/4        208.9
ES2004d          17.5     24.5       9.0      1.3      7.2     5/4        199.9
TS3003a          18.8     29.3      13.0      1.2      4.5     5/4        212.2
EN2002b          20.5     28.9       7.1      1.2     12.2     5/4        199.8
TS3003b          20.9     26.0       7.2      2.8     10.8     4/4        214.0
TS3003c          22.1     30.7       9.0      1.3     11.9     4/4        221.3
IS1009c          38.2     45.6       5.7      1.4     31.1     4/4        221.0
EN2002c          39.0     44.5       6.9      0.6     31.4     5/3        200.7
IS1009b          39.3     45.8       4.9      1.0     33.4     6/4        215.6
EN2002d          42.1     47.5       7.0      1.1     34.1     6/4        191.1
IS1009d          44.1     53.5       6.7      2.0     35.3     5/4        210.5
EN2002a          45.5     51.8       8.6      0.8     36.0     7/4        184.8
TS3003d          52.7     68.5      13.5      1.5     37.6     4/4        197.3
ES2004c          55.7     62.6       4.3      2.4     49.1     9/4        211.5
IS1009a          61.0     77.5       6.0      1.9     53.1     5/4        215.5
ES2004b          78.5     86.9       5.1      1.8     71.6     7/4        216.3
------------------------------------------------------------------------------------------
AVERAGE          38.2     46.8       7.7      1.5     29.0      -        207.5
==========================================================================================

Diarization benchmark with 5s chunks, 0s overlap, and 0.8 clustering threshold:

swift run fluidaudiocli diarization-benchmark --mode streaming \
    --dataset ami-sdm \
    --threshold 0.8 \
    --auto-download \
    --chunk-seconds 5.0 \
    --overlap-seconds 0.0

...

------------------------------------------------------------------------------------------
Meeting        DER %    JER %    Miss %     FA %     SE %   Speakers     RTFx
------------------------------------------------------------------------------------------
ES2004a          17.0     26.0       9.0      1.3      6.7     7/4        113.4
IS1009a          18.1     26.5       4.7      2.7     10.8     4/4        113.2
TS3003a          21.0     32.3      12.7      1.4      6.8     2/4        114.4
TS3003b          21.5     26.2       7.1      4.2     10.2     4/4        109.8
ES2004d          22.5     29.6       9.6      1.8     11.0     6/4        108.6
IS1009c          25.5     30.7       3.4      2.5     19.6     5/4        112.9
ES2004c          25.6     29.9       4.8      2.4     18.4     5/4        111.8
TS3003c          33.2     44.3       8.0      2.5     22.7     4/4        114.2
EN2002c          40.6     46.5       8.4      1.5     30.7     5/3         29.7
EN2002b          46.4     57.2       8.2      1.2     36.9     8/4          9.9
IS1009d          47.8     56.7       4.9      3.0     39.9     5/4        111.4
IS1009b          56.1     63.9       2.5      1.5     52.1     5/4        111.7
ES2004b          57.6     64.0       5.9      2.0     49.7     8/4        111.5
EN2002d          62.4     71.3      10.1      1.4     50.9     7/4        103.4
EN2002a          63.4     71.1       9.2      1.1     53.0     7/4        105.2
TS3003d          66.3     83.3      12.7      2.6     51.0     5/4        105.3
------------------------------------------------------------------------------------------
AVERAGE          39.0     47.5       7.6      2.1     29.4      -         99.1
==========================================================================================

Diarization benchmark with 5s chunks, 2s overlap, and 0.8 clustering threshold:

swift run fluidaudiocli diarization-benchmark --mode streaming \
    --dataset ami-sdm \
    --threshold 0.8 \
    --auto-download \
    --chunk-seconds 5.0 \
    --overlap-seconds 2.0

...

------------------------------------------------------------------------------------------
Meeting        DER %    JER %    Miss %     FA %     SE %   Speakers     RTFx
------------------------------------------------------------------------------------------
TS3003a          20.2     45.0       6.7      2.9     10.6     4/4         37.6
ES2004c          26.8     40.7       1.7      3.5     21.6     6/4         34.9
ES2004a          31.6     54.8       6.4      2.3     23.0     5/4         35.2
IS1009c          38.1     39.0       1.3      4.0     32.7     7/4         36.1
EN2002c          38.1     39.3       3.1      1.9     33.1     6/3         33.6
ES2004d          39.0     44.3       4.4      3.1     31.5     5/4         35.1
EN2002b          45.8     59.1       3.3      2.4     40.1     7/4         34.5
IS1009d          55.3     62.1       2.2      4.4     48.7     8/4         35.6
EN2002a          60.9     54.0       3.6      1.9     55.5     7/4         33.8
ES2004b          69.5     74.6       2.4      2.9     64.1     8/4         34.9
TS3003d          71.7     83.4       5.2      4.7     61.8     6/4         34.5
IS1009a          72.2     75.1       1.7      3.5     67.0     5/4         36.9
TS3003c          72.8     85.2       3.2      3.5     66.1     5/4         36.5
IS1009b          77.6     80.3       0.8      2.1     74.6     6/4         35.9
TS3003b          86.4     86.7       2.8      5.5     78.0     5/4         35.5
EN2002d          88.8     92.2       3.7      2.0     83.1     9/4         33.4
------------------------------------------------------------------------------------------
AVERAGE          55.9     63.5       3.3      3.2     49.5      -         35.2
==========================================================================================

Takeaways from the 2026-07-03 sweep: on the official 16-meeting split, no-overlap configs clearly beat overlap configs (10s/0s 38.2% and 5s/0s 39.0% vs 3s/1s 53.3% and 5s/2s 55.9% average DER) — overlap increases chunk count and drives over-clustering (5-11 detected speakers vs 4 truth on the worst meetings). Note: the two overlap configs were measured one meeting per process; running many overlapping-chunk meetings back-to-back in a single process can exhaust IOSurface-backed CoreML buffers on macOS (E5RT Failed to allocate memory IOSurface object).

Sortformer Streaming Diarization

NVIDIA's Sortformer model for streaming speaker diarization, converted to CoreML.

Model: FluidInference/diar-streaming-sortformer-coreml (V2 models for macOS 26+ compatibility)

Hardware: Apple M2, 2022, macOS 26.1

AMI SDM Dataset (NVIDIA High-Latency Config - 30.4s chunks)

swift run fluidaudiocli sortformer-benchmark --nvidia-high-latency --hf --auto-download

================================================================================
SORTFORMER BENCHMARK SUMMARY
================================================================================
Results Sorted by DER:
----------------------------------------------------------------------
Meeting        DER %    Miss %     FA %     SE %   Speakers     RTFx
----------------------------------------------------------------------
IS1009b          16.4     10.6      0.6      5.3 4/4           127.0
ES2004c          23.8     17.8      0.3      5.7 4/4           126.5
ES2004b          23.9     18.7      0.2      5.0 4/4           123.9
IS1009a          26.5     16.0      1.4      9.1 4/4           134.4
ES2004d          28.3     19.7      0.3      8.3 4/4           123.5
IS1009d          29.1     16.5      1.0     11.6 4/4           127.9
TS3003b          31.1     27.1      0.6      3.4 4/4           125.5
EN2002c          31.8     20.1      0.2     11.5 4/3           126.0
ES2004a          33.7     24.6      0.1      9.0 4/4           127.2
EN2002b          34.0     20.2      0.6     13.3 4/4           127.7
TS3003c          34.4     31.1      0.3      3.1 4/4           126.6
EN2002a          35.6     20.0      0.4     15.2 4/4           125.4
EN2002d          37.1     20.1      0.5     16.5 4/4           125.5
IS1009c          38.1     12.8      0.9     24.4 4/4           129.2
TS3003d          41.0     32.0      0.1      8.8 4/4           125.6
TS3003a          41.8     36.8      0.7      4.3 4/4           125.7
----------------------------------------------------------------------
AVERAGE          31.7     21.5      0.5      9.7         -     126.7
======================================================================

Offline throughput (M5 Pro)

A single fused offline graph (mel[1,128,3072] → speaker_preds, 30.72 s window) exported via the NeMo offline path — one CoreML call per window, no streaming state. ComputeUnit.ALL, median of 120 runs after 12 warmup:

variant	model-exec (mel → preds)	RTFx
fp16	10.65 ms	2884×
6-bit palettized	10.93 ms	2809×

End-to-end incl. mel (fp16): 12.49 ms · 2459×. One fused GPU graph — no per-call dispatch or ANE→GPU handoff. Numerical parity vs the PyTorch reference: 100% speaker-argmax agreement (fp16).

Offline diarizer (whole-file, Swift)

OfflineSortformerDiarizer runs the fused offline model end-to-end: mel extraction → fused graph per 30.72 s window (no streaming state) → timeline. Run with fluidaudio sortformer <file> --offline (--palettized for the 6-bit set). Models: FluidInference/diar-streaming-sortformer-coreml v3/{fp16,palettized}/SortformerOffline_v2.1.mlmodelc; NeMo-reference parity = 100% speaker-argmax (fp16), 96.4% (6-bit palettized).

Throughput is excellent — ~1000–1400× RTFx (one fused call per window, no per-chunk state). Voice detection matches streaming exactly (identical Miss/FA on AMI).

Quality caveat — not for long multi-speaker audio. Each 30.72 s window diarizes independently with no speaker cache, so on long meetings with several speakers it produces large speaker confusion. AMI-SDM (collar 0.25, full test set), same harness:

	DER	Miss	FA	Speaker confusion	RTFx
Streaming (`highContextV2_1`)	26.4%	23.4	0.6	2.4	835×
Offline (whole-file)	56.7%	23.4	0.6	32.7	1418×

Detection is identical; the entire gap is speaker confusion the streaming spkcache avoids by construction (accumulating speaker profiles across the whole history). Cross-window re-stitching does not recover it — the confusion is generated within each window — so use offline for short clips (≤ ~30 s), few-speaker audio, or throughput-bound batch jobs, and use the streaming variants for accurate long-form multi-speaker diarization.

LS-EEND Streaming Diarization

A research prototype from Westlake University for streaming speaker diarization.

Model: FluidInference/lseend-coreml.

Hardware: Apple M4 MAX, 2026, macOS 26.1 (CPU only)

AMI SDM Dataset (AMI variant, `--step-size 500ms`)

Each LS-EEND CoreML bundle is keyed by (variant, stepSize). The run below uses the .ami variant with .step500ms, which commits 5 output frames (~500 ms) per CoreML call.

swift run fluidaudiocli lseend-benchmark --variant ami --step-size 500ms --auto-download

================================================================================
LS-EEND BENCHMARK SUMMARY
================================================================================
Results Sorted by DER:
----------------------------------------------------------------------
Meeting        DER %    Miss %     FA %     SE %   Speakers     RTFx
----------------------------------------------------------------------
ES2004c           8.8      7.2      1.3      0.4     4/4        72.4
ES2004b           8.9      7.3      1.0      0.7     4/4        75.6
TS3003c          13.3     11.1      0.9      1.3     4/4        72.2
IS1009d          13.8      7.9      2.1      3.8     4/4        74.8
TS3003b          16.2      5.9      1.6      8.8     4/4        73.0
TS3003a          19.0     16.6      0.8      1.6     4/4        77.0
EN2002b          20.4     16.0      1.7      2.8     4/4        75.9
TS3003d          20.5     14.7      1.9      3.9     4/4        72.1
IS1009c          22.1      6.8      2.0     13.4     4/4        74.4
EN2002c          23.2     16.6      1.9      4.7     4/3        69.9
IS1009a          23.3      7.9      2.7     12.7     4/4        81.8
EN2002a          24.4     19.6      1.1      3.7     4/4        73.2
IS1009b          25.7      4.8      1.7     19.2     4/4        74.9
ES2004d          27.7     15.1      1.5     11.2     4/4        73.1
EN2002d          27.9     21.9      2.2      3.7     4/4        73.2
ES2004a          35.6     13.4     19.1      3.0     4/4        78.1
----------------------------------------------------------------------
AVERAGE          20.7     12.1      2.7      5.9          -     74.5
======================================================================

Multilingual G2P (Grapheme-to-Phoneme)

CharsiuG2P ByT5 encoder-decoder model converted to CoreML for multilingual grapheme-to-phoneme conversion. Used by Kokoro TTS for non-English phonemization.

Model: FluidInference/charsiu-g2p-byt5-coreml

Hardware: Apple M2, 2022, macOS 26

CharsiuG2P Test Set (500 words/language)

swift run -c release fluidaudiocli g2p-benchmark --data-dir /path/to/CharsiuG2P/data/test

Language	PER	WER	ms/word
Spanish	0.1%	0.8%	32.6
French	0.8%	2.0%	26.5
Italian	2.8%	20.0%	20.9
Hindi	4.5%	21.4%	45.4
Japanese	10.5%	23.8%	31.7
Portuguese (BR)	8.9%	43.2%	24.0
British English	13.6%	29.4%	34.0
American English	19.0%	38.8%	28.2
Chinese	86.2%*	95.0%*	53.9
Average	16.3%	30.5%	33.0

*Chinese PER is inflated due to tone notation mismatch between model output and reference data (tone contour marks vs model format), not a model accuracy issue.

PER (Phoneme Error Rate): Character-level Levenshtein distance / reference length, stress marks stripped
WER (Word Error Rate): Fraction of words with any phoneme error

Compute Unit Comparison

Both the English BART G2P and multilingual ByT5 G2P models run fastest on CPU-only due to GPU/ANE dispatch overhead on small autoregressive decoder steps.

Multilingual G2P (ByT5)

Compute Units	ms/word
cpuOnly	38.7
cpuAndGPU	94.7
all (ANE+GPU+CPU)	95.2

English G2P (BART)

Compute Units	ms/word
cpuOnly	13.0
all (ANE+GPU+CPU)	17.3
cpuAndGPU	23.4

TDT Japanese ASR

Parakeet TDT 0.6B Japanese model converted to CoreML for on-device Japanese transcription. Hybrid architecture using CTC preprocessor/encoder with TDT v2 decoder/joint (workaround for CoreML conversion bug).

Model: FluidInference/parakeet-ctc-0.6b-ja-coreml

Hardware: Apple M2, 2022, macOS 26

JSUT-basic5000 Test Set

Full benchmark on the complete JSUT-basic5000 dataset — 5,000 utterances from a single Japanese speaker.

Dataset: JSUT-basic5000

swift run -c release fluidaudiocli ja-benchmark --decoder tdt --dataset jsut --samples 5000 --auto-download

Metric	TDT Decoder
Mean CER	6.88%
Median CER	4.08%
CER < 5%	2,683 (53.7%)
CER < 10%	3,549 (71.0%)
CER < 20%	4,556 (91.1%)
Mean Latency	208.8 ms
Mean RTFx	28.9x

Note: CER calculation includes number normalization (full-width digits → half-width, kanji numbers → Arabic) matching NVIDIA's evaluation methodology. NVIDIA reports 6.4% CER for the same model on JSUT-basic5000.

CUA-S1-FORMS decision scoring

Apple M5 Pro, 24 GB, macOS 27.0. Full published synthetic test: 24,370 decisions, with no filtering or truncation. Batch-1 model-call timing excludes encoding, loading, and UI.

Model / backend	Correct decisions	Accuracy	Median	p95
Upstream PyTorch / CPU	24,359 / 24,370	99.9549%	1.787 ms	3.104 ms
Original FP16 Core ML / CPU + ANE	24,359 / 24,370	99.9549%	1.003 ms	1.133 ms
ANE-gather FP16 Core ML / CPU + ANE	24,359 / 24,370	99.9549%	1.052 ms	1.177 ms

Both Core ML exports match PyTorch's selected option on every row, including the same 11 incorrect fill-for-skip decisions. ANE-gather is 4.8% slower by median; the original remains the default.

Numerical parity fails for both exports: 11 rows exceed the 0.005 probability-error limit (maximum 0.0204874), and one raw output fails the probability-sum check. The Swift runtime fix below handles that output; these accuracy numbers score raw model outputs.

All 11 probability-error cases passed a focused FP32 CPU rerun (max error 0.0000012), pointing to FP16/backend rounding.

Full report · Dataset and reproduction · Per-row trace · Swift API

INT8 weight trial

A matched run over all 24,370 synthetic decisions on the same M5 Pro:

Export	Package size	Accuracy	Median	p95
Original FP16	1.51 MB	99.9549%	0.990 ms	1.102 ms
INT8 weights, FP16 compute	0.81 MB	99.9549%	0.990 ms	1.104 ms

46.2% smaller, with every selected option unchanged and effectively identical latency. Numerical parity still fails: 64 rows exceed the 0.005 probability-error limit (maximum 0.067738), versus 11 for FP16. No INT8 probability-sum violations were observed. The original remains the default.

Batch-1 Core ML CPU+ANE calls, excluding encoding/loading/UI; both models timed in the same process with alternating order. Full report · INT8 reproduction

INT4 weight trial

A matched run over all 24,370 synthetic decisions on M5 Pro, CPU+ANE:

Export	Package size	Accuracy	Median	p95
FP16 control, iOS 18 target	1.51 MB	99.9549%	0.982 ms	1.081 ms
INT4 weights, FP16 compute	0.45 MB	99.9302%	0.982 ms	1.082 ms

70.1% smaller, with essentially unchanged latency. INT4 makes 17 errors versus 11 for FP16: 14 choices change, introducing 10 errors and correcting four. Numerical parity fails: 356 rows exceed the 0.005 probability-error limit (maximum 0.754359); no INT4 probability-sum violations were observed.

Packed INT4 requires iOS 18/macOS 15. Both exports use the same decomposed attention graph and FP16 computation. Batch-1 timing excludes encoding/loading/UI. INT8 preserves all choices at 0.81 MB; the original FP16 remains the default.

Full report · INT4 reproduction

Swift probability fix

Stable softmax in Swift fixes the FP16 probability-sum exception: 73,110/73,110 calls complete, covering all 24,370 decisions for FP16, INT8, and INT4. Every selected option is unchanged; accuracy remains 99.9549%, 99.9549%, and 99.9302%, respectively. Original model scores remain available as rawProbabilities; raw conversion-parity failures above remain. Report and reproduction

Swift runtime and ANE placement

Separate release Swift benchmark: 200/200 correct per variant over 50 demo controls. Timing includes Swift encoding, inference, and decoding; excludes UI. These timings predate the stable-softmax fix; complete SDK latency has not been remeasured.

Export	ANE operations	CPU operations	Swift median	Swift p95
Original	149 / 173 (86.1%)	24	0.912 ms	0.933 ms
ANE-gather	162 / 165 (98.2%)	3	0.961 ms	0.984 ms

Higher ANE placement is 5.4% slower in Swift. Operation counts describe scheduler placement, not utilization or energy use. Swift report · Compute-plan profiles

Browser demonstration: 100/100 decisions across patient, job, and insurance forms, with actual fill/check actions and verified DOM changes. Recording · Browser report

FilesExpand file tree

Benchmarks.md

Latest commit

History

Benchmarks.md

File metadata and controls

Benchmarks

Transcription

ASR Model Compilation

Parakeet Unified (English, batch + streaming)

LibriSpeech test-clean (2620 files, 5.40h audio)

Nemotron Speech Streaming 0.6B (English)

LibriSpeech test-clean (2620 files, 53,120 words, ~5.4h audio)

Transcription with Keyword Boosting

Text-to-Speech

Kokoro-82M PyTorch (CPU)

Kokoro-82M PyTorch (MPS)

Kokoro-82M MLX Pipeline

Swift + Fluid Audio Core ML models

Voice Activity Detection

FSMN-VAD (fsmn-vad-segment)

SenseVoice

LibriSpeech test-clean (2620 files)

AISHELL-1 Chinese (7176 files)

int8 encoder variant (--int8)

Paraformer

AISHELL-1 Chinese (7176 files, full test, full-CoreML pipeline)

Streaming ASR (Parakeet EOU)

LibriSpeech test-clean (2620 files, 5.40h audio)

Streaming ASR (Nemotron, English)

LibriSpeech test-clean (100 files)

Streaming ASR (Nemotron Multilingual)

LibriSpeech test-clean (English, 2620 files, 5.40h audio)

FLEURS (full test splits, 2.24 s tier)

Speaker Diarization

CAM++ speaker embedding (campplus-embed)

Offline diarization pipeline

Streaming/online Diarization

Sortformer Streaming Diarization

AMI SDM Dataset (NVIDIA High-Latency Config - 30.4s chunks)

Offline throughput (M5 Pro)

Offline diarizer (whole-file, Swift)

LS-EEND Streaming Diarization

AMI SDM Dataset (AMI variant, --step-size 500ms)

Multilingual G2P (Grapheme-to-Phoneme)

CharsiuG2P Test Set (500 words/language)

Compute Unit Comparison

TDT Japanese ASR

JSUT-basic5000 Test Set

CUA-S1-FORMS decision scoring

INT8 weight trial

INT4 weight trial

Swift probability fix

Swift runtime and ANE placement

FSMN-VAD (`fsmn-vad-segment`)

int8 encoder variant (`--int8`)

CAM++ speaker embedding (`campplus-embed`)

AMI SDM Dataset (AMI variant, `--step-size 500ms`)