notify() is just setImmediate(this._maybeRunTask), so calling it from a request handler runs the pool loop in that handler's @apify/timeout context. If the handler has overrun requestHandlerTimeoutSecs, that context is already aborted, and the first tryCancel() in _runTaskFunction sits before its try/catch — so the whole AutoscaledPool rejects instead of one request failing.
import { BasicCrawler, sleep } from 'crawlee';
const crawler = new BasicCrawler({
keepAlive: true, // pool stays idle-but-alive once the queue drains
maxRequestRetries: 0,
requestHandlerTimeoutSecs: 1,
async requestHandler({ request, crawler }) {
if (!request.url.endsWith('/orphan')) return;
await sleep(2000); // overruns the 1s timeout
await crawler.addRequests(['https://x.test/never-crawled']);
await crawler.autoscaledPool.notify(); // pool loop, aborted context
},
failedRequestHandler() {},
});
await crawler.run(['https://x.test/orphan']).catch((e) => console.log(`crawl died: ${e.constructor.name}: ${e.message}`));
const { requestsFinished, requestsFailed } = crawler.stats.state;
console.log(`accounted for ${requestsFinished + requestsFailed}/2 requests`);InternalTimeoutError: Promise handler has been canceled due to a timeout
at tryCancel (@apify/timeout/cjs/index.cjs:43:11)
at BasicCrawler._runTaskFunction (@crawlee/basic/internals/basic-crawler.js:1049:33)
at async AutoscaledPool._maybeRunTask (@crawlee/core/autoscaling/autoscaled_pool.js:535:17)
crawl died: InternalTimeoutError: Promise handler has been canceled due to a timeout
accounted for 1/2 requests
notify()is justsetImmediate(this._maybeRunTask), so calling it from a request handler runs the pool loop in that handler's@apify/timeoutcontext. If the handler has overrunrequestHandlerTimeoutSecs, that context is already aborted, and the firsttryCancel()in_runTaskFunctionsits before its try/catch — so the wholeAutoscaledPoolrejects instead of one request failing.