You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit f15e8f7
Browse filesBrowse the repository at this point in the historyBrowse files
test: use spawn context for ProcessPoolExecutor in basic crawler tests (#2276)
The state persistence tests in `test_basic_crawler.py` start
`ProcessPoolExecutor` with the default fork start method. On Linux with
Python 3.12 and 3.13, forking a multi-threaded process emits
`DeprecationWarning: This process is multi-threaded, use of fork() may
lead to deadlocks in the child.` The executors now pass
`mp_context=get_context('spawn')`.
Each spawned worker imports `crawlee.crawlers` again, which adds a few
seconds to each of the two tests. Windows (spawn) and Python 3.14
(forkserver) already pay that cost.
Closes: #2273
*✍️ Drafted by Claude Code*
0 commit comments