[alan-george-lk]

This PR introduces a new test binary called memory_tester for Linux, which has the following features:

• Configurable common scenarios
  • Examples: publishing tracks, room joining/closure
• RSS and thread usage tracking
• Configurable repeated iterations, acceptable memory and thread growth bounds

CI is also updated to run this tester and provide a nice markdown table report of the results, and fail jobs that introduce changes that fail this.

Documentation is also updated around testing in general, as it was a bit out of date.

Results

Here are two FFI release results with the following params:

Iterations: 100
Warmup: 20
Threshold: RSS growth ≤ 8 MiB; thread growth ≤ 0.

Prior release livekit-ffi/v0.12.81 results:

Scenario	RSS 0→20	RSS 20→100	Threads 0→20	Threads 20→100	Status
audio-track	+60.63 MiB	-2.64 MiB	+28	0	✅ PASS
video-track	+61.43 MiB	-2.62 MiB	+28	0	✅ PASS
data-track	+53.88 MiB	+0.03 MiB	+2	0	✅ PASS
room-reuse	+53.54 MiB	+3.47 MiB	+2	0	✅ PASS
room-client-leave	+53.66 MiB	+0.12 MiB	+2	0	✅ PASS
room-server-delete	+143.46 MiB	+108.94 MiB	+105	+400	❌ FAIL

Current FFI release results:

Scenario	RSS 0→20	RSS 20→100	Threads 0→20	Threads 20→100	Status
audio-track	+17.36 MiB	+0.19 MiB	+12	0	✅ PASS
video-track	+16.74 MiB	-0.05 MiB	+7	0	✅ PASS
data-track	+12.30 MiB	+0.03 MiB	+1	0	✅ PASS
room-reuse	+12.09 MiB	0.00 MiB	+1	0	✅ PASS
room-client-leave	+12.10 MiB	+0.05 MiB	+1	0	✅ PASS
room-server-delete	+12.11 MiB	+0.08 MiB	+1	0	✅ PASS

[devin-ai-integration]

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

🔍 Devin Review: 2 flags

Not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

[stephen-derosa]

overall looks great, thanks for adding these!

[stephen-derosa]

we are only running these on ubuntu x64 release builds?

[alan-george-lk]

Fair callout, but some specific points:

• The original customer report was on Linux/x64
• The tools I built this with are linux-specific (the way RSS is tracked, and the MALLOC_ARENA stuff), given the original customer report
• Running on multiple platforms will clog up the GitHub summary page/take more CI time for likely negligible coverage differences

So I felt having a single run on the most representative platform for robotics/physical AI (resource constrainted linux) would be most applicable

[stephen-derosa]

these doc strings are great

[devin-ai-integration]

Devin Review found 1 new potential issue.

4 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

[devin-ai-integration]

🟡 All-scenario runs fail on standard builds

After a default release-tests build, --scenario all runs capture scenarios that the SDK cannot execute. Both fail, making the all-scenario run fail even when every standard scenario passes.

Learn more

The build defaults to capture disabled, but still produces the Linux memory tester. The script now appends both capture scenarios to all regardless of that build configuration. In a standard build, runCycle rejects each capture scenario, and the script returns a failure even if every other scenario passes.

Example: Run ./build.sh release-tests followed by source scripts/set-test-tokens.sh and scripts/memory-tester.sh --scenario all. The six standard scenarios can pass, but both capture scenarios fail with a capture-enabled-build error.

Recommended fix: Make all select capture scenarios only when the built SDK supports their respective sources, or provide a standard-only group and retain the existing behavior of all for non-capture builds. Keep explicit capture scenario requests failing clearly when unsupported.

---

Was this helpful? React with 👍 or 👎 to provide feedback.