audiolive: a live rig over the audio pump, and the drum machine keeps time on it - #123
Merged
Merged
Conversation
…h at once over the pump
Two fixes the board found. LiveAudio was passing its 256-frame pull block as the DMA descriptor size, so the smallest ring it could open was 21 ms when the same cushion in 128-frame pieces is 10.7 ms; dma_desc/dma_frame are now their own arguments, read from module globals so a harness can lower them. status() was dividing by the configured block rather than the measured one, which reads ShimmerHall at 113 % of a core when it is using half of one. And the FUZZ pedalboard could not play: Fuzz is 553 % of a block on the P4 and Saturation is 697 %, against Distortion's 36 %. It is a DIRT board now, with the measured costs of all four written down beside them.
Releasing it immediately after the retarget is what a first reading of the lock allows, and on the P4 under a live GUI the audio stopped at the first patch change. It is not the deinit -- deferring it did not fix it -- but one generation of lag is the right shape anyway and costs one spare chain.
… is not a crash Two things the board found. status() was dividing the pull cost by the block size LiveAudio asked for, not the one the graph handed back. Fuzz and Saturation return 512 frames where Overdrive returns 256, so every 512-frame class read exactly twice its real cost: Fuzz at 553 % of a block is 276 %, Saturation's 697 % is 347 %, and CLEAN's 81 % is 41 %. The frames are countable live - bytes over blocks - which wall time is not, because the pump publishes its wall clock once, when its loop ends, and a pump that is behind has a wall period equal to its own pull. And the pump can stop by itself. It has no interpreter thread to raise on, so it breaks out of its loop and leaves a code behind; running() then says False while the task is still registered, and the next spawn() refuses with 'a pump is already spawned'. That pair killed rack_gui with a traceback the first time a patch changed. status() now carries 'why' as a sentence, play() clears a dead pump away before it builds anything, and the readout says 'audio stopped: ... - tap a patch' instead of the app dying. Proven on the P4: fault, then play(), blocks 52 -> 250, load 44 %. DIRT is a darker Overdrive voicing for now rather than a Distortion. Distortion's Character macro crossing the middle of its travel deinits every node the class owns while the pump is pulling them - confirmed on the board, first crossing, err=5 fault=deinited, where its other nine macros are clean over 160 moves and 160 patch swaps are clean over 5224 blocks.
The same chain, 25 seconds each on the P4: 41 % of a block with no display, 52 % with the panel lit and LVGL doing nothing at all, 63 % with sliders being dragged, 66 % with whole-screen repaints forced on top. Eleven of those points are the panel simply being lit - a 720x720 framebuffer is a megabyte read out of PSRAM every frame and the graph lives in PSRAM too - so it is a bandwidth tax on every block, a mean cost and not a tail one. No amount of moving the flush or raising the pump's priority reaches a mean cost; ring does. So rack_gui opens a 12 x 128 ring instead of the module's 4 x 128. Over 190 seconds with three thousand slider moves, forced repaints and pedalboard changes: 4 x 128 starves all the way through, 6 x 128 leaves 381 ms of silence all of it at patch changes, 12 x 128 reads exactly zero in seven of nine readings. 32 ms of latency nobody can hear moving a slider - and still the wrong default for playing an instrument, so it is a constant an app opts into rather than a new default.
…_midi becomes importable rack_all opens the same 12 x 128 ring rack_gui does, for the same measured reason, and its STARVED line says why the audio stopped in words when it has rather than printing a fault number. rack_midi guards main() with __name__ == '__main__'. mpftp run and mpremote exec both execute a file as __main__, so the documented way in is unchanged, and MidiRack is now importable -- which is what a probe and rack_all both wanted. Proven on the P4 by feeding it the bytes a controller sends: four note-ons take the load from 45 % to 49 %, CC 1 moves pedal 0 macro 0 from 64 to 120, program change 2 switches to CLEAN with the chord still held, err=0 fault=0 starved=0 over 829 blocks.
…before the screen goes up
… touchscreen The T-Embed S3's half of rack_gui. The encoder moves the highlighted row and the button moves the highlight; PATCH is the first row, so one knob and one button cover both without inventing a modifier -- appdev's own encoder mapping, the same MOUSEWHEEL / button-2 pair appdev_encoder_test.py uses. audiolive itself gains the two things a board with no codec needs: audio_power is optional now (the MAX98357A has no I2C, no registers and no volume, and board_peripherals raises the rail at import), and LiveAudio(level=) scales every source voice, which on such a board is the only volume there is. LIGHT_PATCHES is the measured patch list for this board: one dear pedal or two cheap ones, 64-73 % of a block, no ShimmerHall and no Reverb.
rack_all's animated 696x240 bar at 30 Hz cost 67.8 ms a repaint against a 10 ms LVGL tick that runs between the interpreter's bytecodes, so repaints ran back to back and a 300-iteration Python loop took 20.3 s; tapping SOURCE took 73 s and looked like a hang. The bar is a 696x40 strip at 8 Hz now, with the measured cost-against-area and cost-against-rate tables above METER_MS and the rule they imply. LiveAudio.source() rebuilt the Mixer, the source and the whole Rack to change which sample one voice plays; it re-points the voices of the mixer it already has when the shape matches, and _make() caches what it builds. 206-352 ms a tap becomes 1-7 ms. Also: audiolive.VOLUME, module-level like DMA_DESC, so a harness can turn the speaker down without editing three files; and rack_all keeps its LVGL timer handles, which is how you take one out of the picture to find what it costs.
It took notes and program changes and silently dropped 0xB, where its sibling rack_midi has had a CC map all along; same map in both files now, so one controller drives either example. feed() is split out of the LVGL timer so a harness can hand the same bytes in with no host attached -- midi_read() puts its bytes there and nowhere else.
i2s_start, i2s_dma_bytes and Input moved to `_audioif` when the pump split on the portability line. The import is guarded: on a build with the engine and no hardware half there is nothing for these examples to play through, and saying so at the call is better than failing on the import line.
The four examples, the board each wants, the one command that starts each, and what you should see, hear and touch -- including which two need headphones. Then the three lessons they paid for, with the numbers: draw meters small and slow (one 696x240 repaint is 67.8 ms against a 10 ms tick, and the app looks hung while the audio is fine -- lvgl-bindings#15); pick the ring for the board (12 x 128 on the P4 with its panel lit, 4 x 128 on the T-Embed, and why the two differ in kind); never write flash while it plays (27-37 ms on the P4, 43-48 ms on the T-Embed; a read is free). Marked as unfinished: rack_all on this branch still draws the wide bar.
The split moved i2s_start, Input and i2s_dma_bytes from audiopump to _audioif, and three hasattr() probes still asked audiopump -- so on a split build on_board read False, the codec never powered and the examples were silent with nothing saying why. hasattr(None, ...) is False, so a build with no driver still takes the desktop path.
Each step goes to audiodev's event queue with the frame it is to sound at, so a screen redraw or a collection cannot move a hit: 2.79 ms of spread against 112-152 ms from the deadline loop, and the loop also drops four to six sixteenths a bar under the same storm. Nothing changes on a firmware without the pump. events() is None, self.seq stays None, and _on_step_timer falls through to the loop that shipped. The look-ahead is 300 ms rather than a number of steps, because what it has to cover is how long the interpreter can be away, and relay() means an edit inside the window is still heard on this pass of that step.
It was dma - written, which is the instantaneous gap, and the panel presents it as silence you heard. It fell again when the pump caught up: 88, 82, 2, 2 ms, which no cumulative count can do. It is the high-water mark of that gap now -- a byte of silence once heard stays heard, and recovery stops the total growing rather than giving any back. starved_now carries the gap for ring tuning.
The driver import was guarded and the engine import was not, so on firmware carrying neither half -- any build without audioif's audiopump in it -- an example died with "ImportError: no module named 'audiopump'" pointing at this file's import block. That tells a reader nothing about what their firmware can do. Both halves are guarded now. available() answers the question and why() gives the sentence to print: no pump at all, a pump with no I2S driver behind it (the desktop: a RAM ring you drain yourself), or the pump owning the channel. LiveAudio() raises why() rather than an AttributeError on None. Proved on three builds: cmods/bin/micropython (no pump) imports and says so; build-splitnodrv (engine, no driver) reads driver() none and gets the ring sentence; build-land2 reads pthread. The rack_all "Not finished" paragraph about the wide meter and the one about starved_ms both describe work that is now on this line, so they go.
A firmware with a pump and an example that is not using it is a different app, and the only thing that said which was one print nobody was watching. CLK AUDIO in green beside the BPM when the sequencer keeps time off the audio clock; CLK TIMER! in amber, with the reason on the console, when a firmware that HAS a pump has fallen back to the old step timer; a quiet grey TIMER on a firmware that has no pump, because there it is the honest answer and not a warning. It earned itself on its first run: a kit change was dropping the player off the pump, in silence.
Building a kit takes a while -- audioinstruments.create plus a pre-warm of every drum -- and LVGL's tick arrives on micropython.schedule, between this function's own bytecodes. _on_step_timer ran INSIDE _load_machine, saw audio_started False after the stop, and called _start_audio() itself; the one at the bottom of _load_machine then asked for an I2S peripheral its own re-entrant copy was holding, and got 'Peripheral in use'. This removes the second _start_audio from the log. It does not make the kit change keep the audio clock -- there is a second cause under it, in audiodev's service timer, written up in live-audio-path-notes.md.
… it keeps time
audiolive's "Not finished" led with "USB MIDI over the wire is unproven from
WSL". It was proven on the Waveshare ESP32-P4: notes, CC 1 and two program
changes reached a live rack_all -- patch: LO-FI, then patch: CRUNCH -- with
err=0 fault=0. What is actually left is one usbipd AutoBind policy per host,
because the identity the board wears with MIDI on is a different device from
the one a CDC+MSC bind covers; with it set, no elevation, no amidi, no sudo,
raw bytes to /dev/snd/midiC0D0 are enough. The two things that ARE unfinished
-- nobody has heard rack_knob on the T-Embed, and a patch change builds a
whole Rack on the UI thread -- have their own lines now.
The night's fourth lesson is with the other three: a timer tick re-enters your
Python between its own bytecodes. Both the kit change ("Peripheral in use"
from its own re-entrant copy) and the full-pattern AttributeError that read
like an exhausted voice pool were this, and a desktop cannot show it to you
without injecting the interrupt.
The drum machine's docstring said it keeps time on the audio clock and stopped
there. It now says what the corner of the screen means (CLK AUDIO / CLK TIMER!
/ TIMER), that the label caught two real bugs, how far ahead it schedules
(AHEAD_MS, 300 ms) and why the number is in milliseconds rather than steps --
the worst gap between two calls of the 15 ms step timer was 100 ms -- plus the
P4's 0 late / 0 dropped / 0 refused at 200 BPM.
This was referenced Sep 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
audioliveis a live audio rig you can run on a board: one object that ownsthe codec, the graph and the pump, and four programs on top of it.
rack_guirack_midirack_knobrack_allpackages/audiolive.jsoninstalls it withmip, and the five files are inpydevices-examples.tomlso a PyScript install gets them too.What it did on the boards
On the Waveshare ESP32-P4 panel,
rack_allran ten minutes on the riff atvolume 50: 11 828 slider moves, 29 patch changes, a 12 × 128 ring,
err=0 fault=0, zero starved bytes at the end, with a worstapp.poll()of2.5 ms. Swapping between the microphone and the riff twenty times had a worst
cost of 7 ms.
rack_guiran 190 s at 12 × 128 witherr=0 fault=0and a worst block of7.4 ms against an 18.3 ms baseline.
USB MIDI reached it over the wire: a held chord, CC 1, program 4 →
patch: LO-FI, program 0 →patch: CRUNCH,err=0 fault=0, withmidi_read()at4 µs.
On the LilyGO T-Embed S3,
rack_knobran 190 s with 4126 encoder turnsand 9 patch changes,
err=0 fault=0.The drum machine keeps time on the audio clock
Twelve lines. Where the firmware has a pump, the step sequencer schedules a
bar ahead onto the audio's own clock instead of firing from a Python timer;
where it does not, the example plays exactly as it shipped. The screen says
which clock it is on —
CLK AUDIOorCLK TIMER!— and that label caught tworeal bugs on its first runs.
Its own bar with its own screen repainting, under a collector storm, on the
desktop: queue, worst spread 2.8 ms and 32 of 32 hits every run; old timer,
112–187 ms and it drops four to six sixteenths a bar. On the Waveshare
ESP32-P4 at 200 BPM with the full pattern and the screen redrawing: 0 late,
0 dropped, 0 refused.
Edits land about a step later and nothing is doubled; a tempo change has no
burst and no gap; a kit change stays on the audio clock, ten in a row.
Three things a board taught this example
An animated meter can starve the interpreter. On MicroPython, LVGL's tick
runs between the interpreter's own bytecodes — it is the app's thread — and
one repaint of a 696 × 240 bar cost 67.8 ms against a 10 ms tick, back to
back. A 300-iteration loop went from 20 340 ms to 0.50 ms with a 696 × 40
strip at 8 Hz. That is what the cheap meter here is.
Re-pointing beats rebuilding.
LiveAudio.source()re-points the Mixer'svoices instead of rebuilding Mixer, source and Rack: 206–352 ms down to
1–7 ms.
A screen costs you ring depth, and the number is per board. On the P4 a lit
720 × 720 panel is a standing PSRAM-bandwidth cost of about eleven points of a
block and a GUI app wants 12 × 128 (32 ms). On the T-Embed's SPI panel a lit
screen costs nothing measurable and the knee is 4 × 128 (10.7 ms). Drawing
often is dearer than drawing big on both.
Never write flash while it plays. A flash erase suspends the pump's core
until it finishes. A 512-byte write costs 27–37 ms of stopped audio on the P4
and 43–48 ms on the T-Embed against a 5.3 ms block; a 4 KB write cost the
T-Embed 661 ms of silence. A read is free, and a first
importis a read. Sothese examples hold their lines and write their results once, after teardown.
It degrades where there is no pump
The import is guarded, so a build with no pump says what is missing rather
than dying on the import line, and
available()andwhy()answer it.audioliveis not in the WebAssembly gallery —# gallery: skipon line 2 —because it wants a board and a pump.
What is NOT proven
Nobody has heard
rack_knobon the T-Embed S3, and it starved 2.5 % ofthe time at a 4 × 128 ring across its 190 s run. 8 × 128 is better, not zero.
A patch change builds a whole Rack on the UI thread. One
app.poll()inten minutes on the P4 hit 222 ms at a patch change. It is a hitch, not a hang.
No unmodified example in this repository has played through
audiodevon aboard.
piano.pyis the one to try and it needs a finger on the screen.This exact line has run on the Waveshare ESP32-P4, and only there.
rack_gui3978 moves under its harness with a worst block of 7.1 ms;rack_all's three gates met; ten drum-machine kit changes on the audio clock; the completely full bar at 200 BPM with nothing refused or dropped and the re-entrancy guard firing three times. The T-Embed S3 has not run this line: its image was built with the wrong partition table and is being rebuilt.CI
Ran 60 tests in 0.032s — OK,ruff check lib tests tools scriptsclean.refresh-requirements.py --checkcurrent;requirements.txtinstalled fromTestPyPI as the README documents; both
pydevices_demoandpalettes_demostill alive at the timeout.
manifests.ymlgreen on both jobs —pydevices-examples.tomlcurrent,install_refresh_manifests.sh --auditwithno differences,
gallery_generator.py --checkover 55 demos,audit_board_config_pairs.pyover 48 pairs.deploy.ymlruns green up to thepush, and the Pages assembly completes clean at 23 MB.
The eight
declared # deps not importedwarnings fromgallery_generatorpre-date this branch and are warnings, not failures.
Could not be run locally: the
peaceiris/actions-gh-pagespush and thepost-deploy Playwright check, which is
continue-on-erroranyway.