Repository navigation
ci(android): Run UI tests incl. replay on API 30 in AGP matrix, drop Sauce Labs UI job - #6492
Merged
Merged
Conversation
…Sauce Labs UI job Co-Authored-By: Roman Zavarnitsyn <roman.zavarnitsyn@sentry.io>
📲 Install BuildsAndroid
|
romtsn
marked this pull request as ready for review
November 6, 2026 12:24
romtsn
requested review from
markushi and
runningcode
and removed request for
a team
November 6, 2026 12:24
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
runningcode
approved these changes
Nov 6, 2026
…fail android-emulator-runner runs each script line in its own shell and stops at the first failing line, so a failing connectedReleaseAndroidTest skipped the snapshot pull. Run the tests and the pull from one script that exits with the tests' status. #skip-changelog
Co-Authored-By: Roman Zavarnitsyn <roman.zavarnitsyn@sentry.io> #skip-changelog
API 30 google_apis captures replay frames with PixelCopy (verified in #6495) and avoids the Android 15 ART method-tracing crash that the profiling tests hit on API 35. Co-Authored-By: Roman Zavarnitsyn <roman.zavarnitsyn@sentry.io>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ada4b87. Configure here.
romtsn
changed the title
… timeout The emulator skip was added in #3074 because the timed-out profile's trace file was sometimes missing on emulators. Since #3478 timed-out profiles are discarded and the test asserts no profile is sent, so it no longer reads a trace file. Drop the skip, and add a test-only hook to shorten the 30s profiling timeout so the test waits 5s instead of 35s. Co-Authored-By: Roman Zavarnitsyn <roman.zavarnitsyn@sentry.io>
The test only checks that a timed-out transaction profile is discarded, which AndroidProfilerTest and AndroidTransactionProfilerTest already cover. It was skipped on emulators, so nothing in CI ran it after the Sauce Labs job was removed. Also reverts the test-only profiling timeout hook. Co-Authored-By: Roman Zavarnitsyn <roman.zavarnitsyn@sentry.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📜 Description
Moves the
sentry-uitest-androidsuite fully onto GitHub Actions emulators and drops the Sauce LabsUi testsjob (integration-tests-ui.yml).google_apis(-memory 4096 -gpu auto) instead of API 30aosp_atd, and without-Denvironment=github. SoReplayTest,ReplaySnapshotTestand the replay part ofUserFeedbackUiTestnow run there.ReplaySnapshotTestwrites (scripts/test-ui-android.sh). Only theAGP 9.4.0×integrations: truecell uploads them withsentry-cli snapshots upload, and only for non-fork runs, same as before.BuildConfig.ENVIRONMENTfield and thegithubchecks are gone.ReplayTestandUserFeedbackUiTestnow gate onisReplayAvailable()instead, because theintegrations: falsecells don't shipsentry-android-replay.ReplaySnapshotTestlives inandroidTestReplay, which only compiles with integrations, so it needs no gate..sauce/sentry-uitest-android-ui.yml. The Sauce Labs section of the uitest README is replaced. The macrobenchmark job still uses Sauce Labs and is unchanged.💡 Motivation and Context
The Sauce Labs UI job has a p90 near 50 min and often fails with no device available. With GH emulators,
integration-tests-ui.yml(as proposed in #5639) would duplicate theAGP 9.4.0×integrations: truecell, which runs the same command on the same build config.Replay tests were skipped on GH emulators because "they can't capture screenshots". The experiment in #6490 showed that only holds for ATD images:
google_apis,-gpu autoandswiftshader_indirect: PixelCopy returned a real frame.aosp_atd: PixelCopy failed withERROR_SOURCE_NO_DATA(3).Supersedes #5639.
💚 How did you test it?
CI on this PR. The AGP matrix cells are the test.
ReplayTest(LeakCanary) andUserFeedbackUiTestwith replay haven't run on a GH emulator before, so watch them for flakiness.📝 Checklist
sendDefaultPIIis enabled.🔮 Next steps
Once this is green, #5639 and #6490 can be closed.
#skip-changelog
via roman.
--
View Session [Sentry]