Skip to content

Request.pushErrorMessage stores null when Error.stack is missing, poisoning persisted requests #4087

Description

@onhate

Which package is this bug report for?

@crawlee/core (observed through @crawlee/cheerio / CheerioCrawler)

Issue description

Request.pushErrorMessage(error) assumes that every Error has a string stack. When an error has stack === null or stack === undefined, it appends that value to request.errorMessages, although RequestOptions.errorMessages only accepts strings.

The invalid request can then fail while Crawlee reclaims, persists, marks handled, or reconstructs it:

ArgumentError: (array `RequestOptions.errorMessages`) Expected values to be of type `string` but received type `null`

We encountered this after an HTTP transport emitted a response-stream/TLS error with a valid message but no stack. The persistence failure prevented the request from settling normally and, in a long-lived shared crawler runtime, blocked later work that was waiting for that runtime to drain.

This appears to be the same underlying problem reported in #1351. That report was closed after maintainers requested a reproduction. The sample below provides a deterministic reproduction and shows where the invalid state originates.

The current v3.18.1 implementation still selects errorOrMessage.stack without verifying that it is a string:

https://github.com/apify/crawlee/blob/v3.18.1/packages/core/src/request.ts#L344-L373

Expected behavior: pushErrorMessage() should always append a string. For an Error without a stack, it could fall back to error.message, then String(error), or omit a non-string value.

Code sample

import { Request } from "@crawlee/cheerio";

const request = new Request({ url: "https://example.com" });
const error = new Error("response stream failed");

// Some transport/runtime errors can arrive without a stack.
Object.defineProperty(error, "stack", { value: null });

request.pushErrorMessage(error);
console.log(request.errorMessages); // [null]

// Reproduces what happens when the invalid request is reconstructed/persisted.
new Request({
  url: request.url,
  errorMessages: request.errorMessages,
});

Result:

ArgumentError: (array `RequestOptions.errorMessages`) Expected values to be of type `string` but received type `null`

Package version

Reproduced with @crawlee/core@3.17.0 through @crawlee/cheerio@3.17.0. The same unchecked stack assignment is present in the latest v3.18.1 source linked above.

Node.js version

Production runtime: Bun 1.4.0. The reproduction does not rely on Bun-specific APIs; it only constructs a Crawlee Request and an Error whose stack is null.

Operating system

macOS arm64 for the minimal reproduction; Linux arm64 in the affected worker runtime.

I have tested this on the next release

Not tested.

Other context

Our downstream workaround uses Crawlee's public errorHandler hook to normalize non-string entries in request.errorMessages before request persistence. We are not patching Crawlee locally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    t-toolingIssues with this label are in the ownership of the tooling team.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions