Skip to content

Commit 32e5bbc

Browse files
committed
address comments
- .github/workflows/swift-format.yml 2. Documentation/EspeakFramework.md - Reorganized documentation to clarify that both macOS and iOS use the same primary flow (packaged bundle first) - Separated fallback behavior - only macOS falls back to downloading zip if bundle missing - Removed confusion about platform differences in bundling approach 3. sources/FluidAudio/Resources/.gitkeep & espeak-ng/.gitkeep ❌ DELETED - Removed placeholder files (no longer needed if committing actual bundle) 4. Sources/FluidAudio/TextToSpeech/Kokoro/Assets/Lexicon/EspeakG2P.swift - Added configurable espeak voice parameter (defaults to en-us) - Tracks current voice and only calls espeak_SetVoiceByName() when it changes - Enables multi-language support (British English, Spanish, French, etc.) 5. Sources/FluidAudio/TextToSpeech/Kokoro/Assets/TtsResourceDownloader.swift - Removed PyTorch .pt file download logic (58 lines removed) - Simplified voice embedding download to only fetch usable JSON files - Better error messages when embeddings unavailable 6. Sources/FluidAudio/TextToSpeech/Kokoro/Pipeline/Preprocess/KokoroChunker.swift - Removed #if canImport(ESpeakNG) checks (96 lines removed) - Removed #available checks for macOS 10.14/iOS 12.0 (unnecessary with current deployment targets) - Removed NSLinguisticTagger fallback code (always uses NaturalLanguage now) - Cleaner code without redundant platform checks
1 parent 4c7b8b2 commit 32e5bbc

8 files changed

Lines changed: 58 additions & 384 deletions

File tree

.github/workflows/swift-format.yml

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,9 @@ jobs:
2323
- name: Check formatting
2424
run: |
2525
# Create a copy to check if formatting would change files
26-
rm -rf Sources.bak Tests.bak Examples.bak
27-
rsync -a --safe-links --copy-links Sources/ Sources.bak || true
28-
rsync -a --safe-links --copy-links Tests/ Tests.bak || true
29-
rsync -a --safe-links --copy-links Examples/ Examples.bak || true
26+
cp -r Sources Sources.bak
27+
cp -r Tests Tests.bak
28+
cp -r Examples Examples.bak || true
3029
3130
# Format in place
3231
swift format --in-place --recursive --configuration .swift-format Sources/ Tests/ Examples/ || true
@@ -50,4 +49,4 @@ jobs:
5049
exit 1
5150
fi
5251
53-
echo "✅ All files are properly formatted"
52+
echo "✅ All files are properly formatted"

Documentation/EspeakFramework.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,15 @@
22

33
FluidAudio bundles the eSpeak-NG phoneme resources so Kokoro can fall back to G2P lookups when the US lexicons don’t contain a word. The Core ML pipeline expects the resources under `Resources/espeak-ng/espeak-ng-data.bundle` with the canonical `voices/` directory inside.
44

5-
## macOS (and desktop) builds
6-
- `TtsResourceDownloader.ensureEspeakDataBundle` first stages the packaged `espeak-ng-data.bundle` directly from the SwiftPM resources.
7-
- If the packaged copy is removed, macOS falls back to downloading `espeak-ng.zip` and extracts it with `/usr/bin/unzip` into `~/.cache/fluidaudio/Models/kokoro/Resources/`.
8-
- The `voices/` directory is validated after extraction; if it’s missing we raise `TTSError.downloadFailed`.
5+
## All Platforms (Primary Flow)
6+
- `TtsResourceDownloader.ensureEspeakDataBundle` first attempts to stage the packaged `espeak-ng-data.bundle` from SwiftPM resources (`Sources/FluidAudio/Resources/espeak-ng/`).
7+
- The bundle is copied to `~/.cache/fluidaudio/Models/kokoro/Resources/espeak-ng/`.
8+
- The `voices/` directory is validated after staging; if missing, `TTSError.downloadFailed` is raised.
99

10-
## iOS / tvOS / watchOS
11-
- The Swift package now looks for a pre-packaged `espeak-ng-data.bundle` under `Sources/FluidAudio/Resources/espeak-ng/` and stages it into the cache on first use.
12-
- If the bundle is missing, we surface `TTSError.downloadFailed`; iOS builds no longer attempt to shell out or download the ZIP on-device.
13-
- Seed the packaged bundle (or pre-populate the on-device cache) before running TTS on these platforms.
10+
## Fallback Behavior (macOS Only)
11+
- If the packaged bundle is unavailable, **macOS only** falls back to downloading `espeak-ng.zip` from HuggingFace and extracting it with `/usr/bin/unzip`.
12+
- **iOS/tvOS/watchOS** do not support fallback downloads and will throw `TTSError.downloadFailed` if the packaged bundle is missing.
13+
- For mobile platforms, ensure the packaged bundle is present in the Swift package resources before building.
1414

1515
## Best practices
1616
- Keep the `espeak-ng-data.bundle` (packaged copy) and the optional `espeak-ng.zip` fallback in sync with any updates to the Kokoro phoneme mapper.

Sources/FluidAudio/Resources/.gitkeep

Whitespace-only changes.

Sources/FluidAudio/Resources/espeak-ng/.gitkeep

Whitespace-only changes.

Sources/FluidAudio/TextToSpeech/Kokoro/Assets/Lexicon/EspeakG2P.swift

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,9 @@ final class EspeakG2P {
1717
true
1818
}
1919

20-
func phonemize(word: String) -> [String]? {
20+
func phonemize(word: String, espeakVoice: String = "en-us") -> [String]? {
2121
return queue.sync {
22-
guard initializeIfNeeded() else { return nil }
22+
guard initializeIfNeeded(espeakVoice: espeakVoice) else { return nil }
2323
return word.withCString { cstr -> [String]? in
2424
var raw: UnsafeRawPointer? = UnsafeRawPointer(cstr)
2525
let modeIPA = Int32(espeakPHONEMES_IPA)
@@ -39,8 +39,16 @@ final class EspeakG2P {
3939
}
4040
}
4141

42-
private func initializeIfNeeded() -> Bool {
43-
if initialized { return true }
42+
private var currentVoice: String = "en-us"
43+
44+
private func initializeIfNeeded(espeakVoice: String = "en-us") -> Bool {
45+
if initialized {
46+
if espeakVoice != currentVoice {
47+
_ = espeakVoice.withCString { espeak_SetVoiceByName($0) }
48+
currentVoice = espeakVoice
49+
}
50+
return true
51+
}
4452

4553
guard let base = try? TtsModels.cacheDirectoryURL() else {
4654
logger.warning("Unable to resolve TTS cache directory; disabling eSpeak G2P")
@@ -64,7 +72,8 @@ final class EspeakG2P {
6472
logger.error("eSpeak NG initialization failed (rc=\(rc))")
6573
return false
6674
}
67-
_ = "en-us".withCString { espeak_SetVoiceByName($0) }
75+
_ = espeakVoice.withCString { espeak_SetVoiceByName($0) }
76+
currentVoice = espeakVoice
6877
initialized = true
6978
return true
7079
}

Sources/FluidAudio/TextToSpeech/Kokoro/Assets/TtsResourceDownloader.swift

Lines changed: 11 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -46,59 +46,23 @@ public enum TtsResourceDownloader {
4646

4747
/// Download a voice embedding JSON file from HuggingFace
4848
public static func downloadVoiceEmbedding(voice: String) async throws -> Data {
49-
// Try to download pre-converted JSON first
5049
let jsonURL = "\(kokoroBaseURL)/voices/\(voice).json"
5150

52-
if let url = URL(string: jsonURL) {
53-
do {
54-
let data = try await AssetDownloader.fetchData(
55-
from: url,
56-
description: "\(voice) voice embedding JSON",
57-
logger: logger
58-
)
59-
logger.info("Downloaded voice embedding JSON for \(voice)")
60-
return data
61-
} catch {
62-
logger.warning("Could not download \(voice).json: \(error.localizedDescription)")
63-
}
51+
guard let url = URL(string: jsonURL) else {
52+
throw TTSError.modelNotFound("Invalid URL for voice embedding: \(voice)")
6453
}
6554

66-
var downloadedPtPath: String?
67-
68-
// Download the .pt file for future conversion
69-
let ptURL = "\(kokoroBaseURL)/voices/\(voice).pt"
70-
if let url = URL(string: ptURL) {
71-
do {
72-
let ptData = try await AssetDownloader.fetchData(
73-
from: url,
74-
description: "\(voice) voice embedding (.pt)",
75-
logger: logger
76-
)
77-
78-
let cacheDir = try TtsModels.cacheDirectoryURL()
79-
let voicesDir = cacheDir.appendingPathComponent("Models/kokoro/voices")
80-
try FileManager.default.createDirectory(at: voicesDir, withIntermediateDirectories: true)
81-
82-
let ptFileURL = voicesDir.appendingPathComponent("\(voice).pt")
83-
try ptData.write(to: ptFileURL, options: [.atomic])
84-
downloadedPtPath = ptFileURL.path
85-
logger.info(
86-
"Downloaded voice embedding .pt file for \(voice) (\(ptData.count) bytes)")
87-
logger.notice(
88-
"Run 'python3 extract_voice_embeddings.py' to convert \(voice).pt to JSON format"
89-
)
90-
} catch {
91-
logger.warning("Could not download \(voice).pt: \(error.localizedDescription)")
92-
}
93-
}
94-
95-
if let path = downloadedPtPath {
96-
throw TTSError.processingFailed(
97-
"Voice embedding JSON unavailable for \(voice). Downloaded .pt to \(path); run 'python3 extract_voice_embeddings.py' to convert it."
55+
do {
56+
let data = try await AssetDownloader.fetchData(
57+
from: url,
58+
description: "\(voice) voice embedding JSON",
59+
logger: logger
9860
)
61+
logger.info("Downloaded voice embedding JSON for \(voice)")
62+
return data
63+
} catch {
64+
throw TTSError.modelNotFound("Voice embedding JSON unavailable for \(voice): \(error.localizedDescription)")
9965
}
100-
101-
throw TTSError.modelNotFound("Voice embedding JSON for \(voice)")
10266
}
10367

10468
/// Ensure a voice embedding is available in cache

Sources/FluidAudio/TextToSpeech/Kokoro/Pipeline/Preprocess/KokoroChunker.swift

Lines changed: 21 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -385,18 +385,12 @@ enum KokoroChunker {
385385
phonemes = lexicon[normalized]
386386
}
387387

388-
#if canImport(ESpeakNG)
389-
if phonemes == nil {
390-
if #available(macOS 13.0, iOS 16.0, *) {
391-
if let ipa = EspeakG2P.shared.phonemize(word: normalized) {
392-
let mapped = PhonemeMapper.mapIPA(ipa, allowed: allowed)
393-
if !mapped.isEmpty {
394-
phonemes = mapped
395-
}
396-
}
388+
if phonemes == nil, let ipa = EspeakG2P.shared.phonemize(word: normalized) {
389+
let mapped = PhonemeMapper.mapIPA(ipa, allowed: allowed)
390+
if !mapped.isEmpty {
391+
phonemes = mapped
397392
}
398393
}
399-
#endif
400394

401395
if phonemes == nil,
402396
let spelledTokens = spelledOutTokens(for: normalized),
@@ -408,18 +402,12 @@ enum KokoroChunker {
408402
for spelled in spelledTokens {
409403
var segment = lexicon[spelled]
410404

411-
#if canImport(ESpeakNG)
412-
if segment == nil {
413-
if #available(macOS 13.0, iOS 16.0, *) {
414-
if let ipa = EspeakG2P.shared.phonemize(word: spelled) {
415-
let mapped = PhonemeMapper.mapIPA(ipa, allowed: allowed)
416-
if !mapped.isEmpty {
417-
segment = mapped
418-
}
419-
}
405+
if segment == nil, let ipa = EspeakG2P.shared.phonemize(word: spelled) {
406+
let mapped = PhonemeMapper.mapIPA(ipa, allowed: allowed)
407+
if !mapped.isEmpty {
408+
segment = mapped
420409
}
421410
}
422-
#endif
423411

424412
if segment == nil, let fallback = letterPronunciations[spelled] {
425413
let filtered = fallback.filter { allowed.contains($0) }
@@ -590,60 +578,18 @@ enum KokoroChunker {
590578
let breakCharacters = CharacterSet(charactersIn: ",;:")
591579
let separatorTokens = [": ", "; ", ", "]
592580

593-
#if canImport(NaturalLanguage)
594-
if #available(macOS 10.14, iOS 12.0, *) {
595-
let tagger = NLTagger(tagSchemes: [.lexicalClass])
596-
tagger.string = text
597-
tagger.enumerateTags(
598-
in: text.startIndex..<text.endIndex,
599-
unit: .word,
600-
scheme: .lexicalClass,
601-
options: []
602-
) { tag, range in
603-
guard tag == .punctuation else { return true }
604-
let token = text[range]
605-
if token.unicodeScalars.contains(where: { breakCharacters.contains($0) }) {
606-
var endIndex = range.upperBound
607-
for separator in separatorTokens where text[endIndex...].hasPrefix(separator) {
608-
endIndex = text.index(endIndex, offsetBy: separator.count)
609-
break
610-
}
611-
let segment = text[currentStart..<endIndex]
612-
let trimmed = segment.trimmingCharacters(in: .whitespacesAndNewlines)
613-
if !trimmed.isEmpty {
614-
segments.append(trimmed)
615-
}
616-
currentStart = endIndex
617-
}
618-
return true
619-
}
620-
} else {
621-
let tagger = NSLinguisticTagger(tagSchemes: [.lexicalClass], options: 0)
622-
tagger.string = text
623-
let nsRange = NSRange(location: 0, length: (text as NSString).length)
624-
tagger.enumerateTags(in: nsRange, unit: .word, scheme: .lexicalClass, options: []) {
625-
tag, range, _ in
626-
guard tag == .punctuation, let tokenRange = Range(range, in: text) else { return }
627-
let token = text[tokenRange]
628-
if token.unicodeScalars.contains(where: { breakCharacters.contains($0) }) {
629-
var endIndex = tokenRange.upperBound
630-
for separator in separatorTokens where text[endIndex...].hasPrefix(separator) {
631-
endIndex = text.index(endIndex, offsetBy: separator.count)
632-
break
633-
}
634-
let segment = text[currentStart..<endIndex]
635-
let trimmed = segment.trimmingCharacters(in: .whitespacesAndNewlines)
636-
if !trimmed.isEmpty {
637-
segments.append(trimmed)
638-
}
639-
currentStart = endIndex
640-
}
641-
}
642-
}
643-
#else
644-
for (offset, character) in text.enumerated() {
645-
if let scalar = character.unicodeScalars.first, breakCharacters.contains(scalar) {
646-
var endIndex = text.index(text.startIndex, offsetBy: offset + 1)
581+
let tagger = NLTagger(tagSchemes: [.lexicalClass])
582+
tagger.string = text
583+
tagger.enumerateTags(
584+
in: text.startIndex..<text.endIndex,
585+
unit: .word,
586+
scheme: .lexicalClass,
587+
options: []
588+
) { tag, range in
589+
guard tag == .punctuation else { return true }
590+
let token = text[range]
591+
if token.unicodeScalars.contains(where: { breakCharacters.contains($0) }) {
592+
var endIndex = range.upperBound
647593
for separator in separatorTokens where text[endIndex...].hasPrefix(separator) {
648594
endIndex = text.index(endIndex, offsetBy: separator.count)
649595
break
@@ -655,8 +601,8 @@ enum KokoroChunker {
655601
}
656602
currentStart = endIndex
657603
}
604+
return true
658605
}
659-
#endif
660606

661607
if currentStart < text.endIndex {
662608
let tail = text[currentStart..<text.endIndex]

0 commit comments

Comments
 (0)