[GSmithApps]

Summary

Adds livekit.agents.FaxDetector, which recognizes calls placed by a fax machine.

A calling fax sends a CNG tone (1100 Hz, 0.5 s on / 3 s off, ITU-T T.30) and no speech. Today nothing in the stack catches this:

• AMD transcribes speech and classifies it with an LLM, so a fax hits no_speech_timeout and settles as uncertain, which the docs say to treat as human. AMD also only runs on outbound calls.
• livekit/sip has no tone detection or T.38 handling.

Inbound phone agents (the common case) receive these calls, play a greeting to a fax machine, and the calls show up as short "early end" calls in users' metrics. A user has asked for this

What it does

FaxDetector opens its own rtc.AudioStream on the caller's track, before any noise cancellation the session applies, and confirms a fax after two CNG beeps of the right length (0.35–0.75 s) at the right spacing (3.0–4.0 s). That takes about 4 s of audio. Requiring the cadence is what keeps a single beep, a steady tone, a busy signal, or speech from triggering it.

It only reports (an event and a wait() result). The agent decides what to do: tag the call, hang up, transfer to a fax line.

await session.start(agent=MyAgent(), room=ctx.room)

async with FaxDetector(ctx.room, participant_identity=caller.identity) as detector:
    result = await detector.wait()

if result is not None:
    await ctx.room.local_participant.set_attributes({"call.fax_detected": "true"})
    ctx.delete_room()

For SIP participants, listening starts once sip.callStatus == "active", like AMD. timeout (default 15 s) is wall-clock, counted from when the caller's audio is available.

Why a separate helper rather than an AMD category

• AMD receives audio after the session's noise cancellation, which may remove a pure tone. Reading the raw track avoids the question.
• AMD is outbound-only; faxes are mostly an inbound problem.
• No STT or LLM is needed. The detector is ~100 lines of numpy.

An AMD machine-fax category for outbound calls (2100 Hz CED answer tone) could still be added later.

Open questions (why this is a draft)

1. Should this live in the Agents framework at all, or in livekit/sip, which already decodes the caller audio for every call and could report it in SIPCallInfo? An agent-side helper ships faster and is more flexible; the SIP server would cover every SDK and surface the result in the call record.
2. Naming and location: livekit.agents.FaxDetector in voice/fax/.
3. Does a Node port need to land before this merges?

Testing

• tests/test_fax_detector.py: 12 unit tests under the virtual-time loop. Synthetic audio is generated at 8 kHz, run through G.711 mu-law, and resampled to 16 kHz. Detects CNG (normal, quiet, after leading silence); ignores a single beep, a steady 1100 Hz tone, a US busy signal, beeps at the wrong spacing, synthetic speech, and silence. Covers event emission, timeout, and a track that sends no frames.
• End-to-end against livekit-server --dev: a "caller" participant published the synthetic audio as a real track (so it went through Opus). Fax detected after 4.0 s; a speech-only caller ran to timeout with no detection.
• ruff check, ruff format, and scripts/check_types.py (strict mypy) pass.
• Not yet tested on a real fax recording or a real SIP call. We're asking the user for example calls.

🤖 Generated with Claude Code

[CLAassistant]

All committers have signed the CLA.

[GSmithApps]

@chenghao-mou wondering if you could take a peek at this? I know you worked on the AMD 🙏