Repository navigation
Fix: preserve screenshot and raw-HTML settings in config clones - #2373
Open
linjiaye929 wants to merge 2 commits into
Open
linjiaye929 wants to merge 2 commits into
linjiaye929 wants to merge 2 commits into
Conversation
linjiaye929
marked this pull request as ready for review
October 15, 2026 00:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Preserve
force_viewport_screenshotandbase_urlinCrawlerRunConfig.to_dict(), soclone()and dictionary round-trips retain the requested screenshot and raw-HTML link-resolution settings.Both constructor fields were omitted from
to_dict():force_viewport_screenshot=Truesilently reset it to False. Internal deep-crawl clones could then select a full-page screenshot when viewport-only was requested.base_url="https://example.test/docs/"discarded that URL. Real offlineaprocess_htmlprocessing of aguide.htmllink then returned a relative link instead ofhttps://example.test/docs/guide.html, in both prefetch and full Markdown paths.The production changes add those two fields only. Explicit clone overrides, including False and None, still work. The base_url follow-up extends the same missing-field fix within this PR; it does not change class-default or serialized-snapshot semantics.
Validation
280043ee0ec241b7d950f278df0056c104883af8: 6 regression failures / 6 passing controls before; all 12 pass after.aprocess_html; only the unused crawler strategy is mocked. Viewport routing uses the existing mocked-strategy style. No browser launched, real screenshot captured, or live network test performed.git diff --checkpasses. Production-file Black drift exists equally in the baseline and patch. The Black-normalized follow-up difference is only the base_url field; unrelated formatting was left untouched.The original change is based on develop
6192d5e5b59cb21f9ca418fd6b798de47c0741d3. Before the follow-up, develop2efcccc5d1838da1f7718813e9be5177ce579b49differed only in an unrelated HTML documentation template. No rebase or unrelated changes are included. This is separate from #1791 (screenshot routing) and #2030 (URL-joining algorithm).Hosted checks
On current head
36c564821bb09fe294bf10d63f49affcd433eb9e, the Security workflow is completed withaction_required. Hosted tests have not been shown to run; no hosted CI pass is claimed. The earlier head's Security and Discord notification results do not establish coverage of this update.AI assistance
This change and its tests were developed with AI assistance, including independent diff review. Validation scope and limitations are listed above. No human full-diff review is claimed.