Skip to content

feat(audio-segmentation): in-memory segment handoff + agent-ready contracts - #2338

Open
shubhamNvidia wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
shubhamNvidia:agent/segmentation
Open

feat(audio-segmentation): in-memory segment handoff + agent-ready contracts#2338
shubhamNvidia wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
shubhamNvidia:agent/segmentation

Conversation

@shubhamNvidia

Copy link
Copy Markdown
Contributor

Contracts: describe() + *_key params on speaker-separation and VAD segmentation, with honest cardinality (these produce a nested per-segment list / fan out).

New functionality

  • In-memory handoff (keep_waveform_in_task, keep_segment_waveform_in_task) — hand the cut waveforms straight to the next stage instead of writing every segment to disk and re-reading it. On a diarize→ASR chain this removes a full write+read of the corpus.
  • Explicit disk control (write_to_disk, separated_audio_dir) — writing separated audio is now opt-in and its destination is configurable, rather than implied.

Depends on the agent-ready foundation, #2332 — that must merge first (this branch imports nemo_curator/stages/audio/_agent/, so CI here stays red until it lands).

Adds describe() contracts to the speaker-separation and VAD segmentation stages so a planner can tell what each reads and writes, and how it changes row counts, before running it.

Depends on the agent-ready foundation (nemo_curator/stages/audio/_agent/), which must merge first.

Signed-off-by: Shubham Bhawsar <shbhawsar@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@shubhamNvidia
shubhamNvidia marked this pull request as ready for review August 25, 2026 11:19
@shubhamNvidia
shubhamNvidia requested a review from a team as a code owner August 25, 2026 11:19
@shubhamNvidia
shubhamNvidia requested review from meatybobby and removed request for a team August 25, 2026 11:19
@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds configurable in-memory and file-backed audio handoff to speaker separation and VAD segmentation, together with agent-facing stage contracts and configurable payload keys.

  • Speaker separation can retain per-speaker waveforms, materialize stable WAV files, or do both.
  • VAD supports configurable input residency, output keys, nested output, and optional waveform omission.
  • Fan-out children now copy metadata and stage-performance state to avoid aliasing.
  • Regression tests cover residency choices, output materialization, metadata isolation, and agreement between nested and fan-out VAD boundaries.

Confidence Score: 5/5

The PR appears safe to merge after its explicitly documented agent-ready foundation dependency is present.

No unacknowledged concrete defect with an established reachable failure remains in the reviewed changes.

Important Files Changed

Filename Overview
nemo_curator/stages/audio/segmentation/speaker_separation.py Adds agent contracts, configurable audio keys and residency, optional stable disk output, provenance preservation, and isolated fan-out task state.
nemo_curator/stages/audio/segmentation/vad_segmentation.py Adds agent contracts, configurable residency and output keys, optional waveform retention, and consistent nested or fan-out packaging.
tests/stages/audio/segmentation/test_speaker_separation.py Adds coverage for in-memory and disk outputs, invalid residency configurations, and fan-out metadata isolation.
tests/stages/audio/segmentation/test_vad_segmentation.py Adds CPU-level regression coverage confirming equivalent segment boundaries across nested and fan-out modes.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[AudioTask] --> B{Input residency}
  B -->|Waveform| C[In-memory audio]
  B -->|File| D[Load source file]
  C --> E{Segmentation stage}
  D --> E
  E -->|VAD fan-out| F[Per-segment AudioTasks]
  E -->|VAD nested| G[Task with nested segments]
  E -->|Speaker separation| H[Per-speaker AudioTasks]
  H --> I{Output residency}
  I -->|Keep waveform| J[In-memory speaker waveform]
  I -->|Write to disk| K[Stable per-speaker WAV path]
Loading

Reviews (1): Last reviewed commit: "feat(audio-segmentation): make segmentat..." | Re-trigger Greptile

@shubhamNvidia
shubhamNvidia requested review from ayushdg, mohammadaaftabv and sarahyurick and removed request for meatybobby August 25, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant