Memory leak in deallocator for ANE

[BrandonWeng]

Why is this change needed?

From a developer

"Thread 19 crashed inside objc_retain while CoreML was trying to access an MLFeatureValue / MLMultiArray during async prediction. The crash is a SIGSEGV / KERN_INVALID_ADDRESS at an address that is not mapped → classic use-after-free / accessing deallocated memory.

Load of the backtrace shows CoreML code paths (- [MLFeatureValue multiArrayValue], MLE5InputPortBinder bindMemoryObjectForFeatureValue:, MLE5ExecutionStreamOperation … prepareAsyncSubmissionForInputFeatures:) — i.e. CoreML is preparing inputs for an async execution and tried to retain an Objective-C object that was already freed.

So: you passed a feature value / multiarray to CoreML, CoreML used it asynchronously, but the Swift/ObjC object backing that data was deallocated earlier (or concurrently mutated in an unsafe way). That triggered an invalid pointer dereference when CoreML tried to retain/read it."

[claude]

Claude finished @BrandonWeng's task —— View job

---

Reviewing PR: Daealloc error fix ✅

Completed comprehensive code review of the memory management fix for CoreML async prediction crashes.

Todo List:

• Create pending GitHub review
• Get PR diff to understand changes
• Read modified file for context
• Analyze memory management fix
• Add inline review comments
• Submit review as COMMENT type

Review Summary

This PR correctly fixes a critical use-after-free bug by changing the deallocator from bytes.deallocate() to Darwin.free(bytes). The fix properly pairs posix_memalign with its required cleanup function, addressing the CoreML async prediction crashes described in the issue.

Key findings:

• ✅ Critical fix: Addresses root cause of the reported crash
• ✅ Technically correct: Proper memory allocation/deallocation pairing
• ✅ Minimal impact: No API changes or performance regressions
• ✅ Well-documented: Clear comment explains the requirement

Recommendation: Ready to merge - this fixes a production crash with no downsides.

Full detailed review submitted as GitHub PR review.

[claude]

Code Review: Memory Management Fix for CoreML Crashes

Overview

This PR correctly addresses the use-after-free crash described in the issue. The fix properly pairs posix_memalign with Darwin.free() instead of the incorrect deallocate() method.

Detailed Analysis

Line 3: Darwin import addition ✅

• Good: Adding import Darwin is necessary for explicit access to Darwin.free()
• Impact: Makes the fix more explicit and avoids potential naming conflicts

Lines 70-71: Critical deallocator fix ✅

• Excellent fix: Changed from bytes.deallocate() to Darwin.free(bytes)
• Root cause addressed: posix_memalign allocates memory that MUST be freed with free(), not Swift's deallocate()
• Comment improvement: The added comment clearly explains why this specific deallocator is required

Technical Correctness

1. Memory allocation pairing: ✅ posix_memalign → Darwin.free() is the correct pair
2. Thread safety: ✅ No concurrency issues introduced
3. API compatibility: ✅ No breaking changes to public interface
4. Error handling: ✅ Existing error handling remains intact

Performance Impact

• Positive: Eliminates crashes during CoreML async prediction
• No regression: Same memory allocation strategy, just correct cleanup
• ANE optimization: Maintains 64-byte alignment benefits

Potential Considerations

• nit: Consider adding a unit test that verifies the deallocator doesn't crash when called multiple times or in edge cases
• nit: The fix could potentially be extracted into a private helper function if this pattern is used elsewhere

Conclusion

This is a critical and correct fix that directly addresses the reported CoreML crash. The change is minimal, targeted, and follows proper memory management practices for POSIX-allocated memory.

Approval recommendation: ✅ Ready to merge - this fixes a production crash with no downsides.

[github-actions]

Speaker Diarization Benchmark Results

Speaker Diarization Performance

Evaluating "who spoke when" detection accuracy

Metric	Value	Target	Status	Description
DER	15.1%	<30%	✅	Diarization Error Rate (lower is better)
JER	24.9%	<25%	✅	Jaccard Error Rate
RTFx	23.66x	>1.0x	✅	Real-Time Factor (higher is faster)

Diarization Pipeline Timing Breakdown

Time spent in each stage of speaker diarization

Stage	Time (s)	%	Description
Model Download	9.124	20.6	Fetching diarization models
Model Compile	3.910	8.8	CoreML compilation
Audio Load	0.047	0.1	Loading audio file
Segmentation	13.298	30.0	Detecting speech regions
Embedding	22.164	50.0	Extracting speaker voices
Clustering	8.866	20.0	Grouping same speakers
Total	44.358	100	Full pipeline

Speaker Diarization Research Comparison

Research baselines typically achieve 18-30% DER on standard datasets

Method	DER	Notes
FluidAudio	15.1%	On-device CoreML
Research baseline	18-30%	Standard dataset performance

Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:

• M2 MacBook Air (2022): Runs at 150 RTFx real-time
• Performance scales with Apple Neural Engine capabilities

_{🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 44.3s diarization time • Test runtime: 1m 19s • 10/09/2025, 03:20 AM EST}

[BrandonWeng]

So IIUC - this is "fine" until its not., as we use posix_memalign earlier in the code, we must all Darwin.free since its using a C API

using bytes.deallocate() on memory that was not allocated by Swift is undefined behavior, and UB often looks “fine” until some combination of runtime/allocator/debugging flags makes it blow up.

[github-actions]

VAD Benchmark Results

Performance Comparison

Dataset	Accuracy	Precision	Recall	F1-Score	RTFx	Files
MUSAN	92.0%	86.2%	100.0%	92.6%	734.8x faster	50
VOiCES	92.0%	86.2%	100.0%	92.6%	692.3x faster	50

Dataset Details

• MUSAN: Music, Speech, and Noise dataset - standard VAD evaluation
• VOiCES: Voices Obscured in Complex Environmental Settings - tests robustness in real-world conditions

✅: Average F1-Score above 70%

[github-actions]

ASR Benchmark Results ✅

Status: All benchmarks passed

Parakeet v3 (multilingual)

Dataset	WER Avg	WER Med	RTFx	Status
test-clean	0.57%	0.00%	2.35x	✅
test-other	1.56%	0.00%	1.52x	✅

Parakeet v2 (English-optimized)

Dataset	WER Avg	WER Med	RTFx	Status
test-clean	0.40%	0.00%	1.79x	✅
test-other	1.16%	0.00%	1.34x	✅

Streaming (v3)

Metric	Value	Description
WER	0.00%	Word Error Rate in streaming mode
RTFx	0.28x	Streaming real-time factor
Avg Chunk Time	3.096s	Average time to process each chunk
Max Chunk Time	4.982s	Maximum chunk processing time
First Token	3.296s	Latency to first transcription token
Total Chunks	31	Number of chunks processed

Streaming (v2)

Metric	Value	Description
WER	0.00%	Word Error Rate in streaming mode
RTFx	0.20x	Streaming real-time factor
Avg Chunk Time	4.634s	Average time to process each chunk
Max Chunk Time	5.741s	Maximum chunk processing time
First Token	4.412s	Latency to first transcription token
Total Chunks	31	Number of chunks processed

_{Streaming tests use 5 files with 0.5s chunks to simulate real-time audio streaming}

_{25 files per dataset • Test runtime: 12m28s • 10/09/2025, 03:29 AM EST}

_{RTFx = Real-Time Factor (higher is better) • Calculated as: Total audio duration ÷ Total processing time
Processing time includes: Model inference on Apple Neural Engine, audio preprocessing, state resets between files, token-to-text conversion, and file I/O
Example: RTFx of 2.0x means 10 seconds of audio processed in 5 seconds (2x faster than real-time)}

Expected RTFx Performance on Physical M1 Hardware:

• M1 Mac: ~28x (clean), ~25x (other)
• CI shows ~0.5-3x due to virtualization limitations

_{Testing methodology follows HuggingFace Open ASR Leaderboard}