Skip to content

Let P4 Connect drain PCM without parking the VM on I2S - #31

Closed
bdbarnett wants to merge 1 commit into
mainfrom
p4-audiodev-pace-factory
Closed

bdbarnett wants to merge 1 commit into
mainfrom
p4-audiodev-pace-factory

Conversation

@bdbarnett

Copy link
Copy Markdown
Contributor

Summary

  • board_config.audio_out / board_peripherals.audio_out is a format factory (FACTORY_ROLES bind the callable, not a constructed object). P4 Connect opens I2S at the app rate, including stereo slots.
  • PaceOutput.write() returns immediately, copies so the caller may reuse a buffer, caps the stash with queue_ms, and exposes space() so a VM pump can gate on DMA room instead of sleep_ms.
  • I2S polls on service()/write(); WASAPI win_audio.audio_out takes an AudioFormat. ES8311 advertises channels = 1. wifi.connect syncs NTP after STA is up (Connect TLS).

Test plan

  • SDL_VIDEODRIVER=dummy SDL_AUDIODRIVER=dummy python3 -m unittest discover -s tests -p 'test_audiodev.py' (43 tests)
  • python3 -m unittest discover -s tests -p 'test_boarddev.py'
  • python3 -m unittest tests.test_es8311 tests.test_esp32_p4_audio
  • On the Waveshare P4: audio_out(AudioFormat(44100, 2, 16), latency="low") then Connect drain; latency="low" must not shrink the output ibuf
  • Headless path imports board_peripherals, not board_config

audio_out is a format factory; PaceOutput.write returns immediately and
exposes space(); I2S polls on service. FACTORY_ROLES bind those callables
instead of constructing on getattr.
bdbarnett added a commit that referenced this pull request Sep 16, 2026
pcm_out/pcm_in return raw PCM devices; audio_out returns an AudioOut sample
player. That holds on hosts and on boards alike, which it did not before:
audiodev.auto.audio_out handed back a PCMOutput while board_peripherals
.audio_out handed back an AudioOut, so an app that could reach either wrote
getattr(pcm, 'transport', pcm) to find out which it held. Both spellings in
audiodev.auto are renamed rather than aliased -- an alias would keep two
names for one thing, which is the quirk being removed.

There is deliberately no audio_in. Output has a player layer; input does not,
and audioif has no input-side node to give it one. A name implying otherwise
is the same vagueness that let audio_out mean two things. pwm_tone's factory
becomes tone_out for the same reason: it returns a ToneOutput, so calling it
pcm_out would have relocated the lie rather than fixed it.

Boards now declare an AudioCapability and audiodev owns negotiate(), instead
of each board hand-writing _require_format() with its own error text. The
capability splits channels (what the wire opens) from native_channels (what
reaches a transducer): the P4's ES8311 clocks two slots into one speaker, so
44.1k stereo opens I2S.STEREO and nothing is mixed down. I2SWire publishes
the pins so a consumer that drives the peripheral itself -- usbif's C pump --
can stop reaching into a board module's private names.

The audiomath import PR #31 added to audiodev/__init__.py is gone; the remix
implementation is injected instead, defaulting to the portable one, with
audiodev.accel as the opt-in seam for the C path. A test now asserts that
layering by walking the AST, because nothing caught it the first time.

boarddev resolves host-vs-board, which audiodev cannot do without importing
board configs and losing its standalone-ness. It resolves to
board_peripherals, never board_config -- importing that on the P4 starts
MIPI DSI and stalls the VM.

Suite: 434 tests, green.
bdbarnett added a commit that referenced this pull request Sep 16, 2026
Brad asked that P4 changes stay on a branch until he has heard them. This is
that branch. Everything measurable without ears is measured; what is left is
named below.

Builds on PR #31, keeping its measured work (the pace factory, the POLL ioctl
that arms I2S non-blocking, the es8311 clock-tree change) and replacing its
hand-written _require_format with the shared capability and negotiate().

That resolves a contradiction PR #31 carried: _MAX_CHANNELS = 1 sat beside an
audio_out that accepted 2 channels, while audio_in raised on the same request
-- one constant asked to mean two different things. Now AUDIO_OUT declares
channels=(1,2) with native_channels=1 (the ES8311 clocks two slots into one
speaker, so stereo needs no mixdown) and AUDIO_IN declares channels=(1,), and
each does what it says.

New role audio_power(enable, volume=): brings the analog path up with NO I2S
stream opened. usbif's C pump opens I2S itself from AUDIO_OUT.wire, and two
owners of one I2S channel is a silent failure, not an error -- but the codec
still has to be powered or every byte moves inaudibly.

A defect the tests caught: pcm_in was ignoring negotiate()'s verdict, so a
caller asking for stereo capture on a mono ADC got a mono stream and no
error. There is no input-side adapter, so it raises now. Fixed on all five
I2S boards, since all five had it.

Measured on the board (non-destructively, under /tst with sys.path ahead of
/lib, so Brad's spremote setup is untouched):

  PHASE_A 24000 mono    measured 23995   err -1 permille
  PHASE_A 44100 stereo  measured 44137   err  0 permille

plus the capability and wire readable without opening, and pcm_in/pcm_out
refusing stereo capture, 32-bit and 3 channels with readable errors.

NOT VERIFIED: that it sounds right. The P4's output and input share I2S(0)
and one set of clocks under AudioSession(duplex=False), so it cannot run the
acoustic gate that the T-Embed can. Drain timing proves the ESP32 clocks at
the rate asked; it cannot prove the ES8311's internal dividers agree, and a
wrong REG06 clocks correctly and sounds like garbage.

Suite: 444 tests, green, stable across four runs.
bdbarnett added a commit that referenced this pull request Sep 16, 2026
pcm_out/pcm_in/audio_out, boards declaring an AudioCapability with
audiodev.negotiate() holding the policy, and channels (what the wire opens)
split from native_channels (what reaches a transducer).

Verified on three boards across three ESP32 families -- T-Embed acoustically
via its own microphones, and the QT Py ESP32 Pico with an Adafruit Audio BFF
by ear. Rates 8 kHz to 48 kHz, mono and stereo, all within 1 permille.

Supersedes PR #31, keeping its measured work.
bdbarnett added a commit that referenced this pull request Sep 16, 2026
Held on a branch until Brad heard it, which he has: two passes of the same
arpeggio at 24 kHz mono and 44.1 kHz stereo, same pitch and clean. That was
the one thing no measurement here could establish -- drain timing proves the
ESP32 clocks at the rate asked but cannot prove the ES8311's own dividers
agree, and 44.1 kHz stereo is a rate this board had never run at.

Also resolves the contradiction PR #31 carried, where _MAX_CHANNELS = 1 sat
beside an audio_out accepting 2 channels while audio_in raised on the same
request.
@bdbarnett

Copy link
Copy Markdown
Contributor Author

Superseded and closed in favour of what landed on main today — but nothing here was discarded. This PR's measured work is all in, and this comment is the handover.

What was kept

  • PaceOutput / pace_output — unchanged. The measurement behind it (I2S write() holding the GIL 88–160 ms on a full DMA, starving the C6 SDIO path) is still the reason it exists, and it turned out to be load-bearing everywhere, not just for Connect: I2SPCMOutput arms I2S non-blocking, so an unpaced PCMOutput.write() raises "audio stream made no write progress" on the first full ring. Every I2S board now returns a paced sink.
  • The POLL ioctl that arms asyncio mode, and try_write.
  • The es8311 clock-tree change (REG06 against the real slot width).
  • The format factory ideaaudio_out taking a format, and FACTORY_ROLES so first attribute access binds the callable rather than constructing.
  • board_peripherals no longer importing board_config for its I2C bus. That fix spread: Tab5 ×2 and CoreS3 were doing the same thing on every audio call, starting a display in headless apps.

What changed, and why

_require_format() became a shared AudioCapability + negotiate(). Boards declare facts; audiodev holds the policy. This resolves a contradiction this PR carried: _MAX_CHANNELS = 1 sat beside an audio_out that accepted 2 channels, while audio_in raised on the same request — one constant asked to mean two things. Now AUDIO_OUT declares channels=(1,2) with native_channels=1 (the ES8311 clocks two slots into one speaker, so stereo needs no mixdown) and AUDIO_IN declares channels=(1,), and each does what it says.

audio_out split into three roles. pcm_outPCMOutput, pcm_inPCMInput, audio_outAudioOut. One name, one return type, on boards and hosts alike — previously audiodev.auto.audio_out returned a raw transport while board_peripherals.audio_out returned a player, so callers wrote getattr(pcm, 'transport', pcm) to find out which they held. There is deliberately no audio_in: output has a player layer, capture does not.

The audiomath import came out of audiodev/__init__.py. That module and every transport must be importable with no DSP package present — it is what lets a headless Connect speaker or USB sound card run on firmware without one. The remix implementation is injected instead, with audiodev.accel as the opt-in seam. A test now asserts the rule by walking the AST, because nothing caught it the first time.

Verified

Three boards, three ESP32 families. ESP32-P4 by ear (24 kHz mono and 44.1 kHz stereo, same pitch, clean); T-Embed acoustically through its own microphones via tools/audio_rig/; QT Py ESP32 Pico with an Adafruit Audio BFF by ear. Rates 8 kHz–48 kHz within 1 permille. USB audio end to end: 5,596,062 bytes at exactly realtime, zero timeouts.

Two gaps recorded rather than papered over: #34 (nothing here has a stereo transducer, so channel order is untested) and PyDevices/usbif#12 (the C pump hard-codes 512fs MCLK and cannot exceed 32 kHz).

Full write-up: docs/audio-format-contract.md and docs/audio-format-contract-results.md in the workspace repo.

@bdbarnett bdbarnett closed this Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant