Repository navigation
feat(audio): bundle original FastMSS for meeting simulation - #2524
timofeeva16tanya wants to merge 5 commits into
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Signed-off-by: Tatiana Timofeeva <49036748+timofeeva16tanya@users.noreply.github.com>
125d4e9 to
7c1cb87
Compare
Signed-off-by: Tatiana Timofeeva <49036748+timofeeva16tanya@users.noreply.github.com>
Vendor original Apache-2.0 FastMSS 81fc2c3 with provenance and attribution. Use bundled sources by default and include them in wheel/sdist packaging. Signed-off-by: Tatiana Timofeeva <49036748+timofeeva16tanya@users.noreply.github.com>
timofeeva16tanya
changed the title
Signed-off-by: Tatiana Timofeeva <49036748+timofeeva16tanya@users.noreply.github.com>
 [Medium impact] The changes reach Curator's audio simulation workflow. Fix the missing ASR transcripts and the two generation failures before merging. Findings
SummaryBundles the original FastMSS source and adds stages that run YAML shards in child processes, then read their meeting manifests into
Diagram%%{init: {'theme': 'neutral'}}%%
flowchart LR
A[YAML configs] --> B[FilePartitioningStage]
B --> C[FastMSSSimulationStage]
C --> D[Bundled FastMSS child process]
D --> E[Audio and RTTM]
D --> F[nemo_manifest.json]
F --> G[ManifestReaderStage]
G --> H[One AudioTask per meeting]
Reviews (1) · Last reviewed commit: "ci(audio): baseline verified public Fast..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| manifest = run_dir / "nemo_manifest.json" | ||
| if not manifest.is_file() or not manifest.stat().st_size: | ||
| msg = ( | ||
| f"FastMSS did not produce a nonempty recording manifest: {manifest}; see {run_dir / 'fastmss.log'}" | ||
| ) | ||
| raise RuntimeError(msg) | ||
| results.append( | ||
| FileGroupTask( | ||
| dataset_name=task.dataset_name, | ||
| data=[str(manifest)], |
There was a problem hiding this comment.
The bundled engine never writes text into nemo_manifest.json, but this stage passes that file straight to ManifestReaderStage. The reader copies each row unchanged, so successful runs produce meetings without the reference transcripts promised for ASR training. Build the meeting transcript from the generated supervisions before returning the manifest, and check it in the real-engine test rather than only the fake entry point.
| else: | ||
| assert self.cfg.save_anechoic == False |
There was a problem hiding this comment.
Setting reverberate=true, save_spk=true and save_anechoic=true while keeping the default reverb_prob=0.5 makes generation fail whenever a meeting skips reverberation. That meeting reaches this assertion with save_anechoic still true, aborting the entire shard. Support dry references for those meetings, or require reverb_prob=1 before launching this combination.
| if prev_speaker is not None: | ||
| available_speakers = [s for s in sampled_spk if s != prev_speaker] | ||
| current_speaker = np.random.choice(available_speakers) |
There was a problem hiding this comment.
With min_max_spk=[1,1], a non-hold transition after the first utterance removes the only speaker from available_speakers. np.random.choice then raises instead of continuing the meeting. The bundled librispeech_1spk.yaml recipes request exactly this case. Treat transitions as holds when only one speaker was sampled, or reject these recipes before generation.
|
|
||
| [Multi-Speaker Simulation with FastMSS](/curate-audio/tutorials/fastmss) | ||
| generates meeting mixtures, reference text and RTTM from prepared single-speaker | ||
| speech using an external FastMSS checkout. |
There was a problem hiding this comment.
This tutorial entry says simulation uses an external FastMSS checkout, which contradicts the bundled default introduced here. Say it uses the bundled source so readers are not directed toward an unnecessary checkout.
| speech using an external FastMSS checkout. | |
| speech using the bundled FastMSS source. |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Description
Generate multi-speaker ASR and diarization training mixtures directly from Curator with the original FastMSS implementation included in the package. A separate FastMSS checkout is no longer required: FastMSSSimulationStage and FastMSS select the bundled source by default. An explicit source-root override remains available.
Original upstream snapshot
nemo_curator/_vendor/fastmss: simulator, HMM, RIR generation, utilities, recipe/defaults, paper configurations, preprocessing scripts and upstream documentation/packaging references.Vendored sources live outside automatic audio-stage discovery. Formatting hooks exclude the copied upstream files to retain exact provenance; safety/YAML checks remain active. Coverage measures the Curator wrapper separately from unchanged third-party sources.
Curator integration
Partition YAML configs into independent simulation shards, execute the original Hydra recipe in child processes, then ingest its recording-level NeMo manifest with the existing ManifestReaderStage. Each output AudioTask represents a complete meeting with audio, duration, reference text, RTTM and additional manifest fields. Preserve overlapping turns.
The wrapper manages output paths, CPU/worker count and manifest generation; retains logs; terminates Linux process groups on timeout; and starts fresh directories on retries. The original defaults resolve from the bundled source. The tutorial documents optional simulation dependency installation in an existing environment; no heavyweight dependencies become unconditional. Include vendored Python, YAML, scripts and attribution in wheels/sdists; exclude bytecode.
Validation
Remaining validation before merge
Checklist