Skip to content

Upgrade Flyleaf and FFmpeg to the 9.0 branch - #74

Merged
danielchalmers merged 4 commits into
mainfrom
claude/flyleaf-ffmpeg-upgrade-683143
Aug 30, 2026
Merged

danielchalmers merged 4 commits into
mainfrom
claude/flyleaf-ffmpeg-upgrade-683143

Conversation

@danielchalmers

@danielchalmers danielchalmers commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Moves FlyleafLib 3.10.4 to 3.11.3, FlyleafLib.Controls.WPF 1.6.3 to 1.7.3, Flyleaf.FFmpeg.Bindings 8.0.1 to 9.0.0, and the FFmpeg build the app downloads from the n8.1 branch to n9.0.

Those four are one change: 3.11.3 depends on bindings 9.0.0, and those bindings P/Invoke avcodec-63, avformat-63 and avutil-61, which only the 9.0 shared build ships.

The upgrade turned out to break two things, both of them in FlyleafLib 3.11's reworked audio path. Rather than work around either, this drops audio playback, which the app was never using. That is the whole change: 97 added lines across 7 files, and FlyleafCameraPlayer ends up shorter in the parts that matter.

What the upgrade broke, and why it no longer matters

Neither is visible from the existing suite, because every fixture mp4 in the test project is a bare moov header that nothing ever decodes.

The front camera plays at roughly an eighth of real time, and its speed control silently does nothing. 3.11.1 made avfilter mandatory for audio, and the engine loads LoadProfile.Main, which never loads it. A clip whose front file carries an audio track died inside AudioDecoder's static constructor with DllNotFoundException; the front player advanced 1.0s of media in 8.2s of wall clock while the three audio-free side cameras ran normally, and every speed change on it threw into the debug-level catch in ApplyPlaybackSpeed.

A speed change during a seek freezes the window. AudioDecoder.Flush runs under lockActions/lockCodecCtx and OnSpeedChanged under lockSpeed, and both rebuild the filter graph, so a speed change overlapping a seek wedges the seek thread inside avfilter_graph_free and the next Player.Play spins forever waiting for it on the UI thread. Two process dumps a minute apart showed the same pair of stacks. It is a regression: the same loop runs 8 of 8 iterations clean on 3.10.4 and hangs on iteration 2 on 3.11.3.

Both live behind the audio filter graph. Not opening the audio stream removes them at the root: no LoadProfile.Filters, no seek-settled gate around the speed setter, no workaround to maintain.

Audio was enabled on the primary camera and doing nothing useful. Dashcam recordings carry no audio, there is no volume or mute control anywhere in the UI, and six players on one timeline would each render their own copy of any track that did turn up. Trimmed exports are unaffected because ClipExporter stream-copies whatever the source holds; the harness still finds aac in the exported file.

This was verified against footage that does carry an audio track, so it holds regardless of what a recording turns out to contain: avfilter is never loaded into the process, the audio stream is not opened, playback runs at real time, and the sequence that used to hang on iteration 2 runs 40 of 40 clean.

What the upgrade buys

Measured on both versions, same clip and harness, audio-free footage (on 3.10.4 audio-carrying media is broken outright and would swamp the numbers):

3.10.4 3.11.3
ShowFramePrev moved 0/8 0/8
following ShowFrameNext moved 0/8 8/8
seeks landing within 100ms of target 15/15 15/15
seek latency median / p95 0.0 / 93.9 ms 0.0 / 73.2 ms
end-of-clip drain 7/8, ~1028ms 7/8, ~1030ms
create/dispose 15 player sets 15/15, 3011ms 15/15, 2823ms
private bytes over 20 clip opens +7.9 MB +13.7 MB

Most of the release-note claims are not observable in this app's workload. Seek reliability, end-of-clip draining and disposal are identical within noise, nothing hung on either version, and memory growth is if anything slightly worse on 3.11.3 (single run, also noise). Not dressing those up as wins.

What the measurements do support:

  • Frame stepping is half fixed. A no-op backward step no longer poisons the decoder, so the next forward step advances one frame again. The workaround in StepFrameAsync still has to stay, and its comment now carries the full matrix, because ShowFramePrev does work on 3.11.3 when the sample file happens to carry audio, which makes it look retirable if you test with the wrong clip.
  • No more beta graphics interop in shipped builds. 3.10.4 pulled Vortice Direct3D11/DXGI/XAudio2/D3DCompiler at 3.7.6-beta; 3.11.3 pulls stable 3.8.3.
  • FFmpeg n8.1.2 to n9.0.1 under the code that opens files arriving on a USB stick out of a car.
  • It stops the debt compounding. Five releases and two breaking-change notes' worth of catching up.

Also here

Bumping the branch strands ffmpeg-8.1-bin (~200MB unpacked) next to the app forever, as the 7.1 folder was before it. A successful install now removes superseded ffmpeg-*-bin siblings. This is a direct consequence of the bump, which is why it is in this PR rather than its own.

Verification

The unit suite never touches a decoder, so this was checked against a throwaway WPF harness driving the real stack end to end: the app's actual FFmpeg download, the real engine, real H.264 footage generated with the downloaded ffmpeg.exe, then FfconcatMediaSourceBuilder into VideoPlayerController onto real FlyleafHost surfaces. 79 checks covering install, engine, duration probing, playlist building, playback (open, seek, keyframe scrub, frame-step both ways, speed, end-of-clip, replay), six-camera sync, unreadable and truncated chunks, file-handle release before a clip delete, and export. All green.

Alongside that: 372 unit tests, dotnet format --verify-no-changes clean, and both the x64 and arm64 self-contained single-file publishes CI produces still succeed.

Held back deliberately

The measurement work turned up a pre-existing bug unrelated to the upgrade: PlayAsync sets IsPlaying after its play operation finishes without checking whether the clip is still running, so a play pressed near the end of a clip leaves the transport claiming to play a clip parked on its last frame. It has a fix and two tests ready, but it is not the upgrade and is not in this PR.

On audio-free media, 44 of 60 seeks report SeekCompleted(-1), identically on both versions. That is cosmetic: stepping a frame after each seek showed the decoder lands within 100ms of target 15 out of 15 times, median offset 0.0ms.

FlyleafLib 3.10.4 to 3.11.3, FlyleafLib.Controls.WPF 1.6.3 to 1.7.3, Flyleaf.FFmpeg.Bindings 8.0.1 to 9.0.0, and the build the app downloads from the n8.1 branch to n9.0.

The four move together or not at all.
FlyleafLib 3.11.3 depends on bindings 9.0.0, and those bindings P/Invoke avcodec-63, avformat-63 and avutil-61, which only the 9.0 shared build ships; leaving the download on 8.1 would hand Flyleaf a set of libraries it cannot load.
The About pane and the archive-root constant in the tests name the branch too, so both follow.

Verified against the real download rather than the constant: the app fetches ffmpeg-n9.0-latest-win64-gpl-shared-9.0.zip, extracts avcodec-63, avformat-63, avutil-61, avfilter-12, swresample-7 and swscale-10, and the engine starts on them.
370 tests green, and both the x64 and arm64 self-contained single-file publishes CI produces still succeed.

On its own this bump breaks any clip that carries an audio track; the next commit removes audio playback, which is what makes it safe.
Dashcam recordings carry no audio, the app has no volume or mute control, and four to six players sharing one timeline would each render their own copy of any track that did turn up.
Audio was enabled on the primary camera and had been doing nothing useful.

Turning it off also takes the app out of FlyleafLib 3.11's audio filter graph, which is where both of the defects this upgrade otherwise runs into live.

The first is that 3.11.1 made avfilter mandatory for audio while the engine loads LoadProfile.Main, which never loads it.
A clip whose front file carried an audio track died in AudioDecoder's static constructor with DllNotFoundException, and the damage was silent: the front camera advanced 1.0s of media in 8.2s of wall clock while the three audio-free side cameras ran normally, and every speed change on it threw into the debug-level catch in ApplyPlaybackSpeed.

The second is a deadlock.
AudioDecoder.Flush runs under lockActions and lockCodecCtx, OnSpeedChanged runs under lockSpeed, and both rebuild the filter graph, so a speed change overlapping a seek wedges the seek thread inside avfilter_graph_free and the next Player.Play spins forever waiting for it on the UI thread.
Two process dumps a minute apart showed the same pair of stacks, and it is a regression: the same loop runs 8 of 8 iterations clean on 3.10.4 and hangs on iteration 2 on 3.11.3.

Both were fixed here first, with LoadProfile.Filters and a seek-settled gate around the speed setter.
Not opening the audio stream at all removes them at the root instead, and removes 43 lines of gate with them.

Verified with footage that does carry an audio track, so this holds regardless of what a recording turns out to contain: avfilter is no longer loaded into the process at all, the audio stream is not opened, playback runs at real time, and the sequence that used to hang on the second iteration runs 40 of 40 clean.
Trimmed exports are untouched, because ClipExporter stream-copies whatever the source holds; the harness still sees aac in the exported file.
370 tests green, 79 harness checks green.
Every FFmpeg release branch installs into a folder of its own next to the app, so moving the download from 8.1 to 9.0 leaves ffmpeg-8.1-bin sitting there, about 200MB unpacked, with nothing that will ever load it again and no code path that would ever remove it.
The same was already true of the 7.1 folder before it, so the cost compounds with every branch bump.

A successful install now deletes sibling ffmpeg-*-bin folders that are not the current one.
It runs only after the new binaries are in place, and a folder that refuses to delete is logged rather than failing the install, because the app is fully usable either way and a stale folder is a disk-space problem, not a correctness one.

The match has to stay narrow, since the install root is the app's own folder: a test covers that logs, runtimes and ffmpeg-notes are all left alone, alongside one that covers the delete itself.
The directory list is materialized before the first delete rather than enumerated lazily while entries disappear from under it.

Verified end to end rather than only through the tests: seeded a stale ffmpeg-8.1-bin, deleted the current folder to force a real download, and confirmed only ffmpeg-9.0-bin remained afterwards.
374 tests green.
StepFrameAsync implements backward stepping as a small accurate seek because Flyleaf's own ShowFramePrev is a silent no-op on our ffconcat playlists.
3.11.0 lists a fix for a locking issue between seek and frame stepping, which reads like the workaround can now go.
It cannot, and the comment says so with numbers, because the obvious test to reach for gives the wrong answer.

Measured on the same clip and the same harness, calling the library directly, eight attempts per cell:

  ShowFramePrev moved                3.10.4   3.11.3
    footage with an audio track        0/8      8/8
    footage with no audio track        0/8      0/8

  following ShowFrameNext moved      3.10.4   3.11.3
    footage with no audio track        0/8      8/8

So 3.11 repaired the second half of the old failure: a no-op backward step no longer poisons the decoder, and the next forward step advances exactly one frame again.
The backward step itself still does nothing on footage without an audio track, which is the footage this app opens.

The trap is that it does work on 3.11.3 when the sample file happens to carry audio, so testing against the wrong clip makes the workaround look like dead code.
The comment now carries that explicitly, so the next attempt starts from the measurement instead of the release note.
@danielchalmers
danielchalmers force-pushed the claude/flyleaf-ffmpeg-upgrade-683143 branch from e3ad0ae to 948b62d Compare August 30, 2026 14:30
@danielchalmers
danielchalmers merged commit 86f0afe into main Aug 30, 2026
1 check passed
@danielchalmers
danielchalmers deleted the claude/flyleaf-ffmpeg-upgrade-683143 branch August 30, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant