[hyoban]

Summary

E2E job durations currently combine build waiting, environment startup, and test execution, making slow runs difficult to explain. Add phase timings to CI summaries and JSONL log artifacts, plus Next.js compiler durations and restored-cache input comparisons.

Record image pulls separately from container startup, migrations, readiness, auth bootstrap, seed, scenarios, and teardown. Cache diagnostics capture primary/matched keys, cache sizes and inventories by version directory (plus the largest 20 files), Node/Next versions, changed tracked inputs, and an environment-file hash. Dependency preparation reports API, web, browser system dependencies, and browser downloads separately, including failed-phase exit codes. Successful builds save a diagnostic baseline alongside the existing cache; failures preserve their original outcome.

Keep the shared Web build artifact, browser shards, parallel dependency preparation, and cache key policy. Publish the Web artifact before post-build diagnostics so browser jobs can proceed. Nested timing intervals are documented and must not be summed as pipeline wall time.

Fixes #42718

Validation

• vp staged and scoped vp check e2e passed.
• E2E support unit tests: 49 passed.
• Dependency installation, build synchronization, and cache diagnostic tests: 22 passed.
• pnpm -C e2e type-check passed.
• actionlint passed with the existing Depot runner label registered in a temporary config; shellcheck was disabled.
• Diagnostic collection was smoke-tested against this checkout, including its full input manifest.
• Main CI run 35695259680 passed on f11f3f0a85 in 5m06s; all PR checks passed. Full-stack E2E was validated in CI rather than locally.
• Downloaded and verified build diagnostics and all four browser log artifacts: 57 Chromium scenarios and 4 WebKit scenarios passed, with migration, readiness, scenario, and teardown timing records present. All browser jobs reused the Web artifact.
• First observed build: fallback cache, 32.369s cache restore, 6.97 GB restored cache, and 1.060s compilation. This confirms fallback is not sufficient evidence of slow incremental compilation.
• Runtime observations: middleware pulls 25.14–37.14s; migrations 15.01–24.33s; Chromium Cucumber execution including hooks 45.38–73.00s.
• Same-commit rerun, attempt 2 passed: exact cache match, 24.750s restore, 0.812s compilation, and no tracked-input, Node, Next, or environment-hash changes against the saved baseline.
• That rerun exposed a WebKit long tail: APT downloaded 118 MB in 2m59s; the browser download took about 3s. The new installation timing split makes these costs explicit.
• Cache inventory run passed all four E2E browser jobs. Its exact cache restore took 32.903s and compilation 7.2s with unchanged inputs. 6.812 GB of the 6.813 GB restored cache belongs to the current turbopack/v16.3.5-ca2c75eb database. There is no evidence here of redundant old-version directories; cache pruning is not part of this change.
• Latest dependency-timing change (0817d8cb21): Main CI run 35696933079 and all applicable PR checks passed. All 61 E2E scenarios passed (57 Chromium, 4 WebKit). Local tests, shell syntax, staged checks, and actionlint passed.
• Verified the new timing records in all four browser jobs: Chromium system dependencies took 17–25s and browser downloads 4–5s; WebKit system dependencies took 38s and browser download 5s. API installation remained parallel (15–17s); web installation took 7–9s. Every phase reported exit code 0.
• Latest build used a fallback cache: 31.314s restore and 30.6s compiler duration. The tracked-input comparison reported only .github/workflows/web-e2e.yml changed; Node, Next, and environment hash were unchanged. This documents remaining compiler-time variation without attributing it to cache invalidation on insufficient evidence.

Controlled cache experiment

Run 35700562249 passed all three builds on the same merge SHA 8417162dfba4354ba55cd89080452a8043373d39, runner class, and build environment. Each job started with an empty .next; the fresh-cache job used a separate runner and restored only the cold job's isolated cache.

Mode	Restored bytes (GB)	Restore	Compiler	Full build	Restore + full build
Existing fallback	3.117	13.166s	33.4s	39.368s	52.534s
Cold	0	N/A	78s	94.371s	94.371s
Newly generated exact cache	2.185	8.495s	1.17s	11.408s	19.903s

Cold build generated a 2.184 GB compiler database, saved in about 17.1s; compressed archive was 1.447 GB versus the existing cache's 1.873 GB. Fresh restore had identical tracked inputs, Node/Next versions, and environment-file hash. This sample supports retaining incremental caching, not disabling it.

Limitation: the PR merge ref incorporated upstream source and lockfile changes since the earlier 7 GB observations. The existing-cache arm therefore restored a different 3.117 GB dependency-compatible cache, without diagnostic baseline. These results do not isolate cache age/size from input compatibility and cannot prove that pruning the earlier 7 GB cache yields the same improvement. Repeat with pinned cache provenance before introducing cache rotation or narrowing input keys.

Temporary experiment workflows were removed in f9e29dd65f; production cache policy and shared-build/shard structure remain unchanged. That cleanup commit passed all applicable checks.

Pinned-cache repetitions

Run 35701837718 passed all six builds: each cache snapshot was restored on three independent runners, with checkout pinned to 8417162dfba4354ba55cd89080452a8043373d39, identical build environment, no fallback, and no cache writes. Starting cache sizes and exact keys matched across repetitions.

Snapshot / sample	Restore	Compiler	Full build	Restore + full build
Existing / 1	14.750s	53.0s	62.142s	76.892s
Existing / 2	16.840s	37.1s	46.768s	63.608s
Existing / 3	21.420s	49.0s	59.727s	81.147s
Fresh / 1	12.029s	1.342s	11.100s	23.129s
Fresh / 2	9.218s	1.277s	11.236s	20.454s
Fresh / 3	13.984s	1.141s	11.689s	25.673s

Median restore + full build: existing 76.892s (63.608–81.147s), fresh 23.129s (20.454–25.673s). Most of the difference occurs during compilation, not restore. The fresh cache's tracked inputs and runtime/env hashes match the pinned checkout; the existing snapshot has no input baseline. Exact match here verifies the requested snapshot, not compatibility of its original build inputs with this checkout. These repetitions rule out a single unusually slow runner as the sole explanation, but do not isolate input mismatch from accumulated cache state.

Recommended next investigation: warm the old snapshot with the pinned checkout, save it under a separate experiment key, then compare its subsequent restore/build against the cold-generated snapshot. This isolates one-time input adaptation before deciding on any cache rotation. Production caching remains unchanged.

Both temporary experiment workflows were removed again in c1e4081cd5. All applicable final PR checks passed; Main CI run 35702870252 succeeded (duplicate test jobs were skipped). PR remains draft and unmerged.

Screenshots

Not applicable; CI diagnostics only.

Checklist

• This change requires a Dify user-documentation update (not applicable).
• I understand that this PR may be closed in case there was no previous discussion or issues.
• I've verified the change and added or updated tests where meaningful regression risk justifies coverage.
• I've updated the documentation accordingly (e2e/AGENTS.md).
• I ran vp staged; backend checks are not applicable.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.70%. Comparing base (26d4b9e) to head (f01f7c3).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #42719      +/-   ##
==========================================
- Coverage   87.74%   87.70%   -0.04%     
==========================================
  Files        5645     5062     -583     
  Lines      323434   318271    -5163     
  Branches    65381    64285    -1096     
==========================================
- Hits       283784   279132    -4652     
+ Misses      34267    33765     -502     
+ Partials     5383     5374       -9     

Flag	Coverage Δ
api	87.37% <ø> (+0.02%)	⬆️
cli	?
dify-ui	91.98% <ø> (+0.10%)	⬆️
web	88.10% <ø> (-0.07%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.