Skip to content

Commit 7ce4e91

Browse files
romtsnclaude
andauthored
feat(replay): Capture SurfaceView content (experimental) (#5333)
* feat(replay): Capture SurfaceView content (experimental) SurfaceView (used by Unity, video players, maps, and similar) renders to a separate Surface that is composited by SurfaceFlinger outside of the View hierarchy. PixelCopy.request(window, ...) only captures the Window surface, so SurfaceView regions appeared as transparent/black holes in Session Replay recordings. When the experimental option options.sessionReplay.isCaptureSurfaceViews is enabled, each visible SurfaceView is now captured separately via PixelCopy.request(surfaceView, ...) and composited onto the screenshot using PorterDuff.DST_OVER, so the SurfaceView content draws behind the Window content (which has transparent holes where the SurfaceViews are). Because SurfaceView redraws do not trigger ViewTreeObserver.OnDrawListener, the recorder bypasses the contentChanged guard when SurfaceViews are present, so subsequent frames are re-captured at the configured frame rate instead of reusing the last screenshot. The option defaults to false to preserve existing behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(replay): Cover SurfaceView capture paths Add unit tests for the new SurfaceView capture support and extract a compositeSurfaceViewInto helper so the drawing contract can be verified with hand-built bitmaps (Robolectric's ShadowPixelCopy cannot produce meaningful SurfaceView pixels because there is no real GL producer). The tests cover: - ViewHierarchyNode.fromView returns SurfaceViewHierarchyNode vs. generic - View.traverse collects SurfaceView nodes when a list is supplied, not when it is null, and skips invisible SurfaceViews - PixelCopyStrategy leaves hasSurfaceViews false when the option is off - PixelCopyStrategy flags hasSurfaceViews true when the option is on - PixelCopyStrategy completes gracefully when a SurfaceView has no valid surface (the common Robolectric case) - compositeSurfaceViewInto fills transparent holes behind existing window content via DST_OVER, and respects both window offset and scale factors Also fixes a latent NPE in captureSurfaceViews when SurfaceHolder.surface is null (not just invalid) — happens before the surface is created. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * formatting * api dump * docs(changelog): Move SurfaceView entry to Unreleased Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(replay): Wire capture-surface-views option through ManifestMetadataReader Allow enabling the experimental SurfaceView capture in Session Replay via the manifest meta-data `io.sentry.session-replay.capture-surface-views`, so users relying on auto-init don't need to switch to manual SentryAndroid.init just to flip the flag. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ref(replay): Inline captureSurfaceViewsEnabled local Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ref(replay): Address review comments - Drop the dedicated hasSurfaceViews flag and ScreenshotStrategy hook; PixelCopyStrategy now signals \"capture again next tick\" via a markContentChanged callback that re-arms the recorder's existing contentChanged gate. One source of truth instead of two booleans. - Inline the trivial submitMaskingAndCallback helper at its single call site. - Bail early in the SurfaceView PixelCopy callback if the strategy has been closed mid-flight, mirroring the Window-capture callback. - Document on SentryReplayOptions.captureSurfaceViews and in CHANGELOG that masking granularity is at the SurfaceView level only — content rendered inside a SurfaceView is opaque to the View masking system. - Simplify ViewsTest: build the test view tree inline instead of via a custom Activity subclass, idle the looper after setContentView. - Drop ViewHierarchyNodeTest — its type-dispatch coverage is implicit in ViewsTest, which only counts non-zero results when SurfaceView instances are correctly mapped to SurfaceViewHierarchyNode. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * tweaks * fix(replay): Detect window size changes on activities with configChanges Activities that declare android:configChanges="orientation|screenSize|..." (e.g. Unity, fullscreen video players) keep the same root view across rotations, so onRootViewsChanged never fires and determineWindowSize was never re-invoked. The recording bitmap stayed at the pre-rotation size, the rotated window content rendered into wrong-dim bitmaps, and SurfaceView captures composited at stale coordinates. Attach an OnLayoutChangeListener to each tracked root so a same-root resize triggers determineWindowSize. The existing size-comparison guard (both width and height must differ) keeps IME/adjustResize relayouts from causing spurious reconfigurations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(replay): Avoid windowLocation race and bitmap leak in SurfaceView capture Address two issues flagged by review: 1. windowLocation race — root.getLocationOnScreen(windowLocation) ran on the main thread, but compositeSurfaceViewsAndMask read windowLocation[0]/[1] later from the executor thread. If a new capture cycle started before the compositor ran, the field was overwritten and SurfaceView pixels would composite at the wrong offset. Snapshot into locals (windowX/windowY) at capture time and pass them through, matching the existing svLocation → capturedX/capturedY pattern. 2. Bitmap leak when isClosed in SurfaceView callback — when the strategy closed mid-capture, the path recycled the in-flight svBitmap but skipped onCaptureComplete(), so remaining never reached zero and any sibling bitmaps already stored in captures[] leaked until GC. Now still drive the completion latch on the closed path, and have the compositor's early-return path recycle leftover captures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(replay): Reconfig on single-dim resizes and recycle SurfaceView bitmap on throw Two review-bot findings: 1. determineWindowSize used && to compare new vs last-known dimensions, so single-dimension resizes (split-screen drag, partial multi-window adjustments, foldable transitions where only one dim shifts) were silently dropped — onWindowSizeChanged only fired when both width AND height differed. The new layout listener already detects single-dim changes with ||, but then delegated to a function that AND'd them away. Switch the existing checks to || so any size delta reconfigs the recorder, matching the listener's intent. 2. In captureSurfaceViews, if PixelCopy.request or getLocationOnScreen threw after svBitmap was allocated, the catch path logged the error but never recycled the bitmap, leaking it until GC. Track the bitmap in a nullable local that the catch block recycles, and clear it after PixelCopy.request returns successfully so ownership transfers to the async callback without double-recycling. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(replay): Ignore layout changes on non-latest root in WindowRecorder rootViews is a stack of windows (dialogs, popups, IME). The recorder binds to the topmost root, so a background activity resizing underneath a dialog must not reconfigure the recorder — we'd otherwise allocate a bitmap sized to the activity while still recording the dialog. The latest root's correct dimensions are already picked up via determineWindowSize in the onRootViewsChanged remove path when the overlaying window dismisses. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent d446e68 commit 7ce4e91

13 files changed

Lines changed: 650 additions & 28 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@
44

55
### Features
66

7+
<<<<<<< rz/feat/replay-capture-surface-views
8+
- Session Replay: experimental support for capturing `SurfaceView` content (e.g. Unity, video players, maps) ([#5333](https://github.com/getsentry/sentry-java/pull/5333))
9+
- To enable, set `options.sessionReplay.isCaptureSurfaceViews = true`
10+
- Or via manifest: `<meta-data android:name="io.sentry.session-replay.capture-surface-views" android:value="true" />`
11+
- **Warning:** masking granularity is at the SurfaceView level only — the SDK cannot mask individual elements rendered inside the SurfaceView (e.g. native Unity UI, map labels, video frames). Only enable for SurfaceViews whose content is safe to record.
12+
=======
713
- Add `Sentry.feedback()` API for `show()` and `capture()` ([#5349](https://github.com/getsentry/sentry-java/pull/5349))
814
- `Sentry.showUserFeedbackDialog()` is deprecated in favor of `Sentry.feedback().show()`
915
- `Sentry.captureFeedback()` is deprecated in favor of `Sentry.feedback().capture()`
@@ -36,6 +42,7 @@
3642
- Fix shake-to-report not triggering on some devices due to high acceleration threshold ([#5366](https://github.com/getsentry/sentry-java/pull/5366))
3743
- Fix feedback form retaining previous message when shown again via shake ([#5366](https://github.com/getsentry/sentry-java/pull/5366))
3844
- Avoid stack overflow when deserializing large flat JSON objects ([#5361](https://github.com/getsentry/sentry-java/pull/5361))
45+
>>>>>>> main
3946

4047
### Dependencies
4148

‎sentry-android-core/src/main/java/io/sentry/android/core/ManifestMetadataReader.java‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,8 @@ final class ManifestMetadataReader {
120120

121121
static final String REPLAYS_DEBUG = "io.sentry.session-replay.debug";
122122
static final String REPLAYS_SCREENSHOT_STRATEGY = "io.sentry.session-replay.screenshot-strategy";
123+
static final String REPLAYS_CAPTURE_SURFACE_VIEWS =
124+
"io.sentry.session-replay.capture-surface-views";
123125

124126
static final String REPLAYS_NETWORK_DETAIL_ALLOW_URLS =
125127
"io.sentry.session-replay.network-detail-allow-urls";
@@ -547,6 +549,15 @@ static void applyMetadata(
547549
}
548550
}
549551

552+
options
553+
.getSessionReplay()
554+
.setCaptureSurfaceViews(
555+
readBool(
556+
metadata,
557+
logger,
558+
REPLAYS_CAPTURE_SURFACE_VIEWS,
559+
options.getSessionReplay().isCaptureSurfaceViews()));
560+
550561
// Network Details Configuration
551562
if (options.getSessionReplay().getNetworkDetailAllowUrls().isEmpty()) {
552563
final @Nullable List<String> allowUrls =

‎sentry-android-core/src/main/java/io/sentry/android/core/ScreenshotEventProcessor.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -201,7 +201,7 @@ private boolean isMaskingEnabled() {
201201

202202
final ViewHierarchyNode rootNode =
203203
ViewHierarchyNode.Companion.fromView(rootView, null, 0, options.getScreenshot());
204-
ViewsKt.traverse(rootView, rootNode, options.getScreenshot(), options.getLogger());
204+
ViewsKt.traverse(rootView, rootNode, options.getScreenshot(), options.getLogger(), null);
205205
return rootNode;
206206
} catch (Throwable e) {
207207
options.getLogger().log(SentryLevel.ERROR, "Failed to build view hierarchy", e);

‎sentry-android-core/src/test/java/io/sentry/android/core/ManifestMetadataReaderTest.kt‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2022,6 +2022,31 @@ class ManifestMetadataReaderTest {
20222022
)
20232023
}
20242024

2025+
@Test
2026+
fun `applyMetadata reads capture-surface-views to options`() {
2027+
// Arrange
2028+
val bundle = bundleOf(ManifestMetadataReader.REPLAYS_CAPTURE_SURFACE_VIEWS to true)
2029+
val context = fixture.getContext(metaData = bundle)
2030+
2031+
// Act
2032+
ManifestMetadataReader.applyMetadata(context, fixture.options, fixture.buildInfoProvider)
2033+
2034+
// Assert
2035+
assertTrue(fixture.options.sessionReplay.isCaptureSurfaceViews)
2036+
}
2037+
2038+
@Test
2039+
fun `applyMetadata reads capture-surface-views and keeps default if not found`() {
2040+
// Arrange
2041+
val context = fixture.getContext()
2042+
2043+
// Act
2044+
ManifestMetadataReader.applyMetadata(context, fixture.options, fixture.buildInfoProvider)
2045+
2046+
// Assert
2047+
assertFalse(fixture.options.sessionReplay.isCaptureSurfaceViews)
2048+
}
2049+
20252050
@Test
20262051
fun `applyMetadata reads anrProfilingSampleRate to options`() {
20272052
// Arrange

‎sentry-android-replay/src/main/java/io/sentry/android/replay/ScreenshotRecorder.kt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ internal class ScreenshotRecorder(
4747
options,
4848
config,
4949
debugOverlayDrawable,
50+
markContentChanged = { contentChanged.set(true) },
5051
)
5152
}
5253

‎sentry-android-replay/src/main/java/io/sentry/android/replay/WindowRecorder.kt‎

Lines changed: 50 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ import io.sentry.android.replay.util.hasSize
1717
import io.sentry.android.replay.util.removeOnPreDrawListenerSafe
1818
import io.sentry.util.AutoClosableReentrantLock
1919
import java.lang.ref.WeakReference
20+
import java.util.WeakHashMap
2021
import java.util.concurrent.ScheduledExecutorService
2122
import java.util.concurrent.atomic.AtomicBoolean
2223

@@ -33,6 +34,7 @@ internal class WindowRecorder(
3334
private val isRecording = AtomicBoolean(false)
3435
private val rootViews = ArrayList<WeakReference<View>>()
3536
private var lastKnownWindowSize: Point = Point()
37+
private val rootLayoutListeners = WeakHashMap<View, View.OnLayoutChangeListener>()
3638
private val rootViewsLock = AutoClosableReentrantLock()
3739
private val capturerLock = AutoClosableReentrantLock()
3840
private val backgroundProcessingHandlerLock = AutoClosableReentrantLock()
@@ -124,24 +126,63 @@ internal class WindowRecorder(
124126
rootViews.add(WeakReference(root))
125127
capturer?.recorder?.bind(root)
126128
determineWindowSize(root)
129+
attachLayoutListener(root)
127130
} else {
131+
detachLayoutListener(root)
128132
capturer?.recorder?.unbind(root)
129133
rootViews.removeAll { it.get() == root }
130134

131135
val newRoot = rootViews.lastOrNull()?.get()
132136
if (newRoot != null && root != newRoot) {
133137
capturer?.recorder?.bind(newRoot)
134138
determineWindowSize(newRoot)
139+
attachLayoutListener(newRoot)
135140
} else {
136141
Unit // synchronized block wants us to return something lol
137142
}
138143
}
139144
}
140145
}
141146

147+
/**
148+
* Activities that handle their own configuration changes (e.g. Unity, video players via
149+
* `android:configChanges="orientation|screenSize|..."`) keep the same root view across rotations,
150+
* so [onRootViewsChanged] never fires and [determineWindowSize] would never re-detect the new
151+
* dimensions. Watch the root for layout-time size changes to catch these cases.
152+
*/
153+
private fun attachLayoutListener(root: View) {
154+
if (rootLayoutListeners.containsKey(root)) return
155+
val listener =
156+
View.OnLayoutChangeListener {
157+
v,
158+
left,
159+
top,
160+
right,
161+
bottom,
162+
oldLeft,
163+
oldTop,
164+
oldRight,
165+
oldBottom ->
166+
val width = right - left
167+
val height = bottom - top
168+
val oldWidth = oldRight - oldLeft
169+
val oldHeight = oldBottom - oldTop
170+
if (width == oldWidth && height == oldHeight) return@OnLayoutChangeListener
171+
// ignore non-latest roots so a dialog stays sized for itself, not its background activity.
172+
if (v != rootViews.lastOrNull()?.get()) return@OnLayoutChangeListener
173+
determineWindowSize(v)
174+
}
175+
rootLayoutListeners[root] = listener
176+
root.addOnLayoutChangeListener(listener)
177+
}
178+
179+
private fun detachLayoutListener(root: View) {
180+
rootLayoutListeners.remove(root)?.let { root.removeOnLayoutChangeListener(it) }
181+
}
182+
142183
fun determineWindowSize(root: View) {
143184
if (root.hasSize()) {
144-
if (root.width != lastKnownWindowSize.x && root.height != lastKnownWindowSize.y) {
185+
if (root.width != lastKnownWindowSize.x || root.height != lastKnownWindowSize.y) {
145186
lastKnownWindowSize.set(root.width, root.height)
146187
windowCallback.onWindowSizeChanged(root.width, root.height)
147188
}
@@ -157,7 +198,7 @@ internal class WindowRecorder(
157198
}
158199
if (root.hasSize()) {
159200
root.removeOnPreDrawListenerSafe(this)
160-
if (root.width != lastKnownWindowSize.x && root.height != lastKnownWindowSize.y) {
201+
if (root.width != lastKnownWindowSize.x || root.height != lastKnownWindowSize.y) {
161202
lastKnownWindowSize.set(root.width, root.height)
162203
windowCallback.onWindowSizeChanged(root.width, root.height)
163204
}
@@ -222,7 +263,13 @@ internal class WindowRecorder(
222263
override fun reset() {
223264
lastKnownWindowSize.set(0, 0)
224265
rootViewsLock.acquire().use {
225-
rootViews.forEach { capturer?.recorder?.unbind(it.get()) }
266+
rootViews.forEach {
267+
val root = it.get()
268+
if (root != null) {
269+
detachLayoutListener(root)
270+
capturer?.recorder?.unbind(root)
271+
}
272+
}
226273
rootViews.clear()
227274
}
228275
}

0 commit comments

Comments
 (0)