Skip to content

Commit 6a1c02c

Browse files
runningcodeclaude
andauthored
ci(macrobenchmark): Run the startup benchmark on Sauce Labs and recover its results (#5909)
* ci(macrobenchmark): Run startup benchmark on Sauce Labs (POC) The sentry-uitest-android-macrobenchmark module currently only runs on a locally connected device. This wires it to Sauce Labs so we can evaluate whether the cold-start timeToInitialDisplay benchmark can run on the real-device cloud already used by our other benchmarks. This is a proof-of-concept, gated behind workflow_dispatch and kept off the per-PR path. Device-guard errors are intentionally not suppressed: on non-rooted, unlocked-clock cloud devices the guards (UNLOCKED, DEBUGGABLE, ...) are expected to fire, and seeing which ones fire is the point of the POC. It also tells us whether timeToInitialDisplay can be retrieved from Sauce artifacts (benchmark JSON) or must be parsed from the device log. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci(macrobenchmark): Call gradle directly instead of a Makefile target Drop the assembleMacrobenchmark Makefile target and invoke the two gradle assemble tasks directly from the workflow. Only one workflow uses them, so the extra indirection isn't worth it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci(macrobenchmark): Pull benchmark results via Real Device Access API saucectl cannot retrieve Macrobenchmark's output. Its artifacts.download.match filters a hardcoded list of assets Sauce hosts for a job -- device.log, junit.xml, video.mp4, network.har, crash.json, screenshots.zip -- and never reads the device filesystem, so benchmarkData.json and the per-iteration perfetto traces stay stranded on the device. The POC's "*-benchmarkData.json" match line could never have matched anything; drop it. Drive the run through the Real Device Access API instead, which exposes adb shell and pullFile: reserve a device, install both APKs, run the instrumentation, then pull the results off. This also recovers the perfetto traces, the only way to resolve sub-millisecond SDK changes that timeToInitialDisplay cannot see. The API is an open-beta add-on, so the workflow probes entitlement before spending a Gradle build on it, and defaults to a one-iteration smoke run. * ci(macrobenchmark): Fix app upload status and drop the iterations override App Storage upload answers 201, not 200, so the strict status check rejected every upload before a device was even reserved. Also drop --iterations: androidx.benchmark.iterations only feeds the microbenchmark path (BenchmarkStateLegacy, MicrobenchmarkPhase). Macrobenchmark reads its iteration count from the test source and ignores the argument, so skipping AOT compilation is the only way to shorten a run that still writes results -- dryRunMode would shorten it but forces outputEnable to false, producing no benchmarkData.json to retrieve. Track installations by installationId rather than by the app reference, which Sauce may echo back normalised. * ci(macrobenchmark): Resolve the device from the catalog before opening a session The first run reached the API but matched zero devices: Real Device Access descriptors are not the ids used in .sauce/*.yml, which carry an OS version and a region suffix (Google_Pixel_9_Pro_XL_15_real_sjc1). Fetch the catalog unfiltered and match client-side against both descriptor id and name, then open the session with the resolved concrete id rather than a pattern. On no match, print every available Android device so a naming mismatch is diagnosable from a single run -- the endpoint's own deviceId filter cannot distinguish "no devices on this account" from "your pattern matched nothing". Also run the no-match check before --probe-only returns. It previously came after, so the probe reported success and the real run failed on it minutes later. * ci(macrobenchmark): Probe every region when the device catalog is empty An empty catalog answers 200 exactly like a populated one, so the first probe reported the API as reachable while there was in fact nothing to run on. Report device availability and the other regions' catalogs too, so an unentitled account is distinguishable from a misconfigured region. * ci(macrobenchmark): Resolve devices from /devices/status, not /devices On a public-cloud account /devices answers 200 with an empty list, which made the probe look like a missing entitlement. /devices/status returns the whole fleet -- 392 devices, all public, 368 available -- and its descriptors are exactly the ids already used in .sauce/*.yml, so the original Google_Pixel_9_Pro_XL_15_real_sjc1 is back as the default. Prefer AVAILABLE devices when a pattern matches several, and report each match's state so an IN_USE device explains itself. * ci(macrobenchmark): Recover results through logcat, not the device API The Real Device Access API can pull files off a device, but it cannot reserve one on a public-cloud account: POST /sessions rejects every request with deviceClasses=[PRIVATE_DEVICE], and the request body has no way to ask for a public device. Everything up to reservation worked -- App Storage upload, device resolution from /devices/status, the session request itself -- so this is a commercial limit, not a technical one. Revert to saucectl and drop the script; git history has it if we ever lease private devices, since that route would also return the per-iteration perfetto traces. Instead have the benchmark echo its own benchmarkData.json into logcat in chunks, which reaches CI as device.log, the one asset Sauce does hand back. scripts/parse-macrobenchmark-log.py reassembles it, validates the JSON, and writes a timeToInitialDisplay table to the job summary. * ci(macrobenchmark): Decode Sauce's JSON-lines device log The recovery worked on the device but not in the parser. Sauce returns device.log as JSON lines -- {"tag", "message", "level", ...} -- so the payload arrives with its quotes escaped and cannot be regexed straight out of the raw line. Decode each entry and match against its message, falling back to the raw line so the same parser still handles `adb logcat` output from a local run. Enable pipefail in the workflow step too. Piping into tee meant the step exited on tee's status, so this very failure reported success while producing no results. Verified against the device.log from run 31117974318: timeToInitialDisplayMs min 443.5 / median 477.3 / max 571.2 over 12 iterations on a Pixel 9 Pro XL. * docs(macrobenchmark): Record how Sauce results are retrieved Document the three routes that do not work -- artifacts.download.match, Macrobenchmark's own reporting channels, and the Real Device Access API on a public-cloud account -- so the next person does not re-derive them. * refactor(macrobenchmark): Collect log chunks per file Chunks were merged across every *.log found, so two devices each emitting the same number of chunks would overwrite each other's indices and reassemble into one bogus document. Collect per file and report when more than one log carries results, which also drops the cross-file total comparison that was standing in for this check. Build the summary table and per-iteration details in one pass while here. * ci(macrobenchmark): Drop unused submodule checkout and encryption key env The repo has no submodules -- no .gitmodules, no gitlink entries -- so submodules: 'recursive' does nothing; sentry-native-ndk is a Maven dependency pinned in gradle/libs.versions.toml. GRADLE_ENCRYPTION_KEY is never read as env.* or $GRADLE_ENCRYPTION_KEY either; setup-gradle takes the secret directly via cache-encryption-key, which stays. * ci(macrobenchmark): Record why the benchmark runs on a high-end device Older hardware would resolve SDK-init regressions better, but Google_Pixel_4_13_real_us could not be allocated in three attempts -- each queued ~61 minutes before Sauce gave up with a concurrency retry-timeout, leaving an empty device log and tests="0". The Pixel 9 Pro XL allocates reliably and is the device this pipeline is proven green on. * ref(macrobenchmark): Narrow the deprecation suppression to the deprecated call @Suppress("DEPRECATION") covered the whole candidateDirs expression, including externalCacheDir, which is not deprecated -- so a future deprecation there would have been silently swallowed. Scope it to getExternalMediaDirs and record why it cannot be migrated: it is deprecated in favour of MediaStore, which returns content URIs rather than the filesystem path androidx.benchmark writes to. * ref(macrobenchmark): Trim comments and drop the cache encryption key The retrieval rationale these comments carried is documented in the module README, so removing the duplicated prose loses nothing. Also drops cache-encryption-key from the Setup Gradle step, leaving this workflow without the Gradle configuration-cache encryption the other workflows still configure. * ci(macrobenchmark): Remove the temporary push trigger (JAVA-679) The push trigger was scaffolding: workflow_dispatch cannot target a workflow that is absent from the default branch, so the workflow had to fire on pushes to the development branch to be exercised at all. Once this merges that constraint is gone, and leaving the trigger in place would start a Sauce Labs run on every push to a same-named branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci(macrobenchmark): Skip result steps on cancellation (JAVA-679) always() keeps a step running even when the job is cancelled, so a run cancelled by the concurrency group would still grind through result recovery and artifact upload instead of stopping promptly. !cancelled() keeps the intended behaviour, running on both success and failure so a failed benchmark still reports what it measured, while letting a cancellation take effect. This matches integration-tests-ui, which already gates its post-Sauce steps the same way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 9a425e5 commit 6a1c02c

5 files changed

Lines changed: 326 additions & 7 deletions

File tree

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
name: 'Integration Tests - Macrobenchmark'
2+
# Runs the sentry-uitest-android-macrobenchmark cold-start benchmark on a Sauce Labs real
3+
# device and recovers timeToInitialDisplay from the device log.
4+
#
5+
on:
6+
workflow_dispatch:
7+
8+
concurrency:
9+
group: ${{ github.workflow }}-${{ github.ref }}
10+
cancel-in-progress: true
11+
12+
jobs:
13+
macrobenchmark:
14+
name: Macrobenchmark
15+
runs-on: ubuntu-latest
16+
17+
# we copy the secret to the env variable in order to access it in the workflow
18+
env:
19+
SAUCE_USERNAME: ${{ secrets.SAUCE_USERNAME }}
20+
21+
steps:
22+
- name: Git checkout
23+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
24+
25+
- name: 'Set up Java: 17'
26+
uses: actions/setup-java@b6effb05e454b25005698d916606bdc6ffcbf961 # v5
27+
with:
28+
distribution: 'temurin'
29+
java-version: '17'
30+
31+
- name: Setup Gradle
32+
uses: gradle/actions/setup-gradle@9c971963bec38e04b3d30dcc455b5382be2fdbfb # v6.3.0
33+
34+
- name: Assemble target app and Macrobenchmark apk
35+
if: env.SAUCE_USERNAME != null
36+
run: ./gradlew :sentry-samples:sentry-samples-android:assembleRelease :sentry-android-integration-tests:sentry-uitest-android-macrobenchmark:assembleBenchmark
37+
38+
- name: Run Macrobenchmark in SauceLab
39+
uses: saucelabs/saucectl-run-action@283660aa934c02723c497efa151d582a3acc5801 # pin@v3
40+
if: env.SAUCE_USERNAME != null
41+
env:
42+
GITHUB_TOKEN: ${{ github.token }}
43+
with:
44+
sauce-username: ${{ secrets.SAUCE_USERNAME }}
45+
sauce-access-key: ${{ secrets.SAUCE_ACCESS_KEY }}
46+
config-file: .sauce/sentry-uitest-android-macrobenchmark.yml
47+
48+
# Runs even when the suite fails: a failed benchmark still logs whatever it managed to
49+
# measure, and the parser's own error explains what was missing.
50+
- name: Recover benchmark results from the device log
51+
if: ${{ !cancelled() && env.SAUCE_USERNAME != null }}
52+
run: |
53+
# Without pipefail the step passes on `tee`'s exit code, so a failed recovery
54+
# would report success while silently producing no results.
55+
set -o pipefail
56+
python3 scripts/parse-macrobenchmark-log.py ./artifacts \
57+
--json-out ./artifacts/benchmarkData.json | tee -a "$GITHUB_STEP_SUMMARY"
58+
59+
- name: Upload Sauce artifacts
60+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
61+
if: ${{ !cancelled() }}
62+
with:
63+
name: macrobenchmark-results
64+
path: ./artifacts/
65+
if-no-files-found: warn
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
apiVersion: v1alpha
2+
kind: espresso
3+
sauce:
4+
region: us-west-1
5+
concurrency: 1
6+
metadata:
7+
build: sentry-uitest-android-macrobenchmark-$GITHUB_REF-$GITHUB_SHA
8+
tags:
9+
- benchmarks
10+
- android
11+
- macrobenchmark
12+
13+
defaults:
14+
timeout: 40m
15+
16+
espresso:
17+
app: ./sentry-samples/sentry-samples-android/build/outputs/apk/release/sentry-samples-android-release.apk
18+
testApp: ./sentry-android-integration-tests/sentry-uitest-android-macrobenchmark/build/outputs/apk/benchmark/sentry-uitest-android-macrobenchmark-benchmark.apk
19+
20+
suites:
21+
22+
- name: "Macrobenchmark startup (api 35)"
23+
# No test orchestrator and no clearPackageData: Macrobenchmark manages its own process
24+
# restarts and AOT compilation, and StartupMode.COLD intentionally keeps app data and
25+
# permissions (it force-stops rather than `pm clear`).
26+
devices:
27+
- id: Google_Pixel_9_Pro_XL_15_real_sjc1 # Google Pixel 9 Pro XL - api 35 (15) - high end
28+
29+
artifacts:
30+
download:
31+
when: always
32+
match:
33+
- junit.xml
34+
- "*.log"
35+
directory: ./artifacts/
Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
#!/usr/bin/env python3
2+
"""Recover Macrobenchmark results from a Sauce Labs device log.
3+
4+
Sauce Labs cannot pull arbitrary files off a real device, so
5+
SentryStartupBenchmark echoes its `<pkg>-benchmarkData.json` into logcat as
6+
numbered chunks. This reassembles those chunks and prints a Markdown summary.
7+
8+
Usage:
9+
parse-macrobenchmark-log.py <artifacts-dir> [--json-out benchmarkData.json]
10+
"""
11+
12+
import argparse
13+
import json
14+
import re
15+
import sys
16+
from pathlib import Path
17+
18+
# Must match SentryStartupBenchmark.LOG_TAG and its "[index/total]" chunk prefix.
19+
CHUNK_RE = re.compile(r"SentryBenchmarkData\s*:\s*\[(\d+)/(\d+)\](.*)$")
20+
21+
22+
def log_messages(log_file):
23+
"""Yields the message text of every log entry.
24+
25+
Sauce hands back device.log as JSON lines -- {"tag", "message", "level", ...} -- which
26+
means the payload arrives with its quotes escaped, so it has to be decoded rather than
27+
regexed out of the raw line. Plain-text lines are passed through unchanged so the same
28+
parser works on `adb logcat` output from a local run.
29+
"""
30+
# Sauce device logs occasionally carry undecodable bytes; don't die on them.
31+
for line in log_file.read_text(errors="replace").splitlines():
32+
line = line.strip()
33+
if line.startswith("{"):
34+
try:
35+
yield json.loads(line).get("message", "")
36+
continue
37+
except json.JSONDecodeError:
38+
pass
39+
yield line
40+
41+
42+
def collect_chunks(log_file):
43+
"""Returns one log's chunk texts keyed by index, plus the expected total."""
44+
chunks, total = {}, None
45+
for message in log_messages(log_file):
46+
match = CHUNK_RE.search(message)
47+
if match:
48+
index, total = int(match.group(1)), int(match.group(2))
49+
chunks[index] = match.group(3)
50+
return chunks, total
51+
52+
53+
def reassemble(chunks, total):
54+
missing = [i for i in range(1, total + 1) if i not in chunks]
55+
if missing:
56+
sys.exit(f"Incomplete benchmark data: missing chunk(s) {missing} of {total}")
57+
return "".join(chunks[i] for i in range(1, total + 1))
58+
59+
60+
def format_summary(data):
61+
context = data["context"]
62+
build = context["build"]
63+
lines = [
64+
"## Macrobenchmark results",
65+
"",
66+
f"**Device:** {build['brand']} {build['model']} "
67+
f"(api {build['version']['sdk']}, {context['cpuCoreCount']} cores) &middot; "
68+
f"**compilation:** {context['compilationMode']} &middot; "
69+
f"**CPU clocks locked:** {context['cpuLocked']}",
70+
"",
71+
]
72+
73+
if not context["cpuLocked"]:
74+
lines += [
75+
"> CPU clocks are unlocked on this device, so run-to-run spread is wide. "
76+
"Treat these numbers as a trend, not a regression gate.",
77+
"",
78+
]
79+
80+
table = [
81+
"| Benchmark | Metric | min | median | max | CoV | iterations |",
82+
"|---|---|--:|--:|--:|--:|--:|",
83+
]
84+
details = []
85+
for benchmark in data["benchmarks"]:
86+
name = f"{benchmark['className'].rsplit('.', 1)[-1]}.{benchmark['name']}"
87+
for metric, result in sorted(benchmark["metrics"].items()):
88+
table.append(
89+
f"| `{name}` | {metric} "
90+
f"| {result['minimum']:.1f} | {result['median']:.1f} | {result['maximum']:.1f} "
91+
f"| {result['coefficientOfVariation'] * 100:.1f}% | {len(result['runs'])} |"
92+
)
93+
runs = ", ".join(f"{run:.1f}" for run in result["runs"])
94+
details += [
95+
"",
96+
f"<details><summary>{metric} per iteration</summary>",
97+
"",
98+
runs,
99+
"",
100+
"</details>",
101+
]
102+
103+
return "\n".join(lines + table + details)
104+
105+
106+
def main():
107+
parser = argparse.ArgumentParser(description=__doc__)
108+
parser.add_argument("artifacts_dir", type=Path, help="directory of downloaded Sauce artifacts")
109+
parser.add_argument("--json-out", type=Path, help="where to write the recovered benchmarkData.json")
110+
args = parser.parse_args()
111+
112+
log_files = sorted(args.artifacts_dir.rglob("*.log"))
113+
if not log_files:
114+
sys.exit(f"No *.log files under {args.artifacts_dir}")
115+
116+
# Keyed by file so chunks from two devices can never be merged into one bogus document.
117+
per_log = {log: collect_chunks(log) for log in log_files}
118+
with_chunks = {log: result for log, (result, total) in per_log.items() if total}
119+
if not with_chunks:
120+
sys.exit(
121+
"No SentryBenchmarkData chunks in the device log. The benchmark most likely "
122+
"failed before reporting — check junit.xml and the log for Macrobenchmark errors."
123+
)
124+
if len(with_chunks) > 1:
125+
sys.exit(
126+
"Chunks from more than one run: "
127+
+ ", ".join(str(log) for log in with_chunks)
128+
+ ". This parser reports a single device."
129+
)
130+
log_file = next(iter(with_chunks))
131+
chunks, total = per_log[log_file]
132+
133+
data = json.loads(reassemble(chunks, total))
134+
135+
if args.json_out:
136+
args.json_out.write_text(json.dumps(data, indent=2))
137+
138+
print(format_summary(data))
139+
140+
141+
if __name__ == "__main__":
142+
main()

‎sentry-android-integration-tests/sentry-uitest-android-macrobenchmark/README.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,34 @@ Connect a device, then:
3434
Results print to the console and are written to
3535
`build/outputs/connected_android_test_additional_output/.../*-benchmarkData.json`.
3636

37+
## Running on Sauce Labs
38+
39+
The `Integration Tests - Macrobenchmark` workflow (manual trigger) runs the same benchmark on a
40+
Sauce Labs real device. It reports numbers only — it is not a PR gate, because cloud devices run
41+
with unlocked CPU clocks and the run-to-run spread swamps most SDK-init changes.
42+
43+
Getting the numbers *back off* the device is the awkward part, so if you are changing this, know
44+
what has already been ruled out:
45+
46+
- **`artifacts.download.match` in `.sauce/*.yml` cannot reach the device.** It filters a
47+
hardcoded list of assets Sauce hosts for the job (`device.log`, `junit.xml`, `video.mp4`,
48+
`network.har`, `crash.json`, `screenshots.zip`); nothing enumerates device storage. A
49+
`*-benchmarkData.json` pattern there matches nothing.
50+
- **Macrobenchmark's own reporting channels don't survive.** The readable summary goes into the
51+
instrumentation status bundle, which only Studio and AGP consume, and `benchmarkData.json` is
52+
written to the app's external media dir, which Sauce never pulls.
53+
- **The Real Device Access API can pull device files, but not on our account.** It offers
54+
`pullFile` and `executeShellCommand`, and `GET /rdc/v2/devices/status` even lists the whole
55+
public fleet — but `POST /sessions` answers `deviceClasses=[PRIVATE_DEVICE]` and there is no
56+
parameter to request a public device. It would need leased private devices. That route would
57+
also return the per-iteration perfetto traces, so it is worth revisiting if we ever get them.
58+
59+
So `SentryStartupBenchmark` echoes its own `benchmarkData.json` into logcat in chunks, which
60+
reaches CI inside `device.log`, and `scripts/parse-macrobenchmark-log.py` reassembles it and
61+
writes a `timeToInitialDisplay` table to the job summary. Note this recovers the metrics only —
62+
the perfetto traces are megabytes each and cannot go through logcat, so sub-millisecond work
63+
still needs a local device.
64+
3765
### Device hygiene (do this for trustworthy numbers)
3866

3967
- **Wake and unlock the device first** — the launch check fails with "Unable to confirm activity

‎sentry-android-integration-tests/sentry-uitest-android-macrobenchmark/src/main/java/io/sentry/uitest/android/macrobenchmark/SentryStartupBenchmark.kt‎

Lines changed: 56 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,16 @@
11
package io.sentry.uitest.android.macrobenchmark
22

3+
import android.util.Log
34
import androidx.benchmark.macro.CompilationMode
45
import androidx.benchmark.macro.ExperimentalMetricApi
56
import androidx.benchmark.macro.StartupMode
67
import androidx.benchmark.macro.StartupTimingMetric
78
import androidx.benchmark.macro.TraceSectionMetric
89
import androidx.benchmark.macro.junit4.MacrobenchmarkRule
910
import androidx.test.ext.junit.runners.AndroidJUnit4
11+
import androidx.test.platform.app.InstrumentationRegistry
12+
import java.io.File
13+
import org.junit.AfterClass
1014
import org.junit.Rule
1115
import org.junit.Test
1216
import org.junit.runner.RunWith
@@ -23,10 +27,6 @@ import org.junit.runner.RunWith
2327
* [android.os.Trace] section the SDK emits, isolating SDK-init cost from the rest of the start.
2428
*
2529
* [CompilationMode.Full] pins ART AOT compilation so dexopt state does not drift between runs.
26-
* Iterations are capped at 12: on an unthrottled Pixel 3, back-to-back cold starts hit thermal
27-
* throttling after ~14 iterations, which inflates the tail of longer runs. This is NOT a CI test;
28-
* it requires a connected device. To A/B an SDK change, see README.md (build the app twice, once
29-
* per SDK variant, in interleaved rounds).
3030
*/
3131
@OptIn(ExperimentalMetricApi::class)
3232
@RunWith(AndroidJUnit4::class)
@@ -47,10 +47,59 @@ class SentryStartupBenchmark {
4747
startActivityAndWait()
4848
}
4949

50-
private companion object {
51-
const val TARGET_PACKAGE = "io.sentry.samples.android"
50+
// Not private: @AfterClass needs a public static method.
51+
companion object {
52+
private const val TARGET_PACKAGE = "io.sentry.samples.android"
5253

5354
// Matches the android.os.Trace section name in SentryAndroid.init.
54-
const val INIT_TRACE_SECTION = "SentryAndroid.init"
55+
private const val INIT_TRACE_SECTION = "SentryAndroid.init"
56+
57+
private const val BENCHMARK_DATA_SUFFIX = "-benchmarkData.json"
58+
59+
/** Kept in sync with `scripts/parse-macrobenchmark-log.py`. */
60+
private const val LOG_TAG = "SentryBenchmarkData"
61+
62+
/** Well under logcat's ~4 KB per-message cap, so a chunk is never silently truncated. */
63+
private const val CHUNK_LENGTH = 2000
64+
65+
/**
66+
* Echoes the benchmark results into logcat so CI can recover them.
67+
*
68+
* Macrobenchmark reports its numbers two ways, and on Sauce Labs neither one arrives: the
69+
* human-readable summary goes into the instrumentation status bundle (which only Studio and AGP
70+
* read), and `<pkg>-benchmarkData.json` is written to the app's external media dir, which Sauce
71+
* cannot pull — it only returns assets it produces itself, and logcat is one of them.
72+
*
73+
* Safe to run at this point because `ResultWriter` writes the file synchronously as each result
74+
* is appended; only its *reporting* is deferred to the end of the run. Locally this is just
75+
* extra logcat noise — Gradle still copies the real file into the build directory.
76+
*/
77+
@JvmStatic
78+
@AfterClass
79+
fun logBenchmarkDataToLogcat() {
80+
val benchmarkData = findBenchmarkData()
81+
if (benchmarkData == null) {
82+
Log.w(LOG_TAG, "No *$BENCHMARK_DATA_SUFFIX found, cannot report results to CI")
83+
return
84+
}
85+
86+
// Dropping the indentation makes the JSON compact enough to survive as a few logcat
87+
// messages. Only structural whitespace is affected: JSON forbids raw newlines inside
88+
// strings, so no value can span lines or start with the indentation being trimmed.
89+
val compactJson = benchmarkData.readText().lineSequence().joinToString("") { it.trimStart() }
90+
val chunks = compactJson.chunked(CHUNK_LENGTH)
91+
chunks.forEachIndexed { index, chunk ->
92+
Log.i(LOG_TAG, "[${index + 1}/${chunks.size}]$chunk")
93+
}
94+
}
95+
96+
private fun findBenchmarkData(): File? {
97+
val context = InstrumentationRegistry.getInstrumentation().targetContext
98+
// This is where Macrobenchmark writes to for some reason.
99+
@Suppress("DEPRECATION") val mediaDirs = context.externalMediaDirs.toList()
100+
return (mediaDirs + context.externalCacheDir).filterNotNull().firstNotNullOfOrNull { dir ->
101+
dir.listFiles()?.firstOrNull { it.name.endsWith(BENCHMARK_DATA_SUFFIX) }
102+
}
103+
}
55104
}
56105
}

0 commit comments

Comments
 (0)