Skip to content

FileSystemStorageClient writes snake_case data on disk instead of camelCase #1844

Closed

Description

@janbuchar
  • @docs_group('Storage data')
    class RequestQueueMetadata(StorageMetadata):
    """Model for a request queue metadata."""
    model_config = ConfigDict(validate_by_name=True, validate_by_alias=True, from_attributes=True)
    had_multiple_clients: Annotated[bool, Field(alias='hadMultipleClients')]
    """Indicates whether the queue has been accessed by multiple clients (consumers)."""
    handled_request_count: Annotated[int, Field(alias='handledRequestCount')]
    """The number of requests that have been handled from the queue."""
    pending_request_count: Annotated[int, Field(alias='pendingRequestCount')]
    """The number of requests that are still pending in the queue."""
    total_request_count: Annotated[int, Field(alias='totalRequestCount')]
    """The total number of requests that have been added to the queue."""
    - models such as RequestQueueMetadata specify camelCase aliases
    • this suggests that we originally intended to maintain the same on-disk format as crawlee-js
    • FileSystemStorageClient calls model_dump without by_alias=True - that means that the files will be snake_case
  • I found this while working on the Rust file system storage client
  • @vdusek perhaps you remember if this was intentional?

This will be a breaking change.

Activity

  1. added
    t-toolingIssues with this label are in the ownership of the tooling team.
    on Apr 10, 2026
  2. vdusek commented on Apr 13, 2026

    @vdusek
    Collaborator

    @vdusek perhaps you remember if this was intentional?

    I don't think so, we probably just forgot by_alias=True. Maybe a candidate for v2?

  3. janbuchar commented on Apr 13, 2026

    @janbuchar
    CollaboratorAuthor

    @vdusek perhaps you remember if this was intentional?

    I don't think so, we probably just forgot by_alias=True. Maybe a candidate for v2?

    Yeah, I see it the same way. I'll have crawlee_storage use snakeCase internally then - no need to have two different on-disk formats.

  4. added this to the 2.0 milestone on Apr 14, 2026
  5. added theissue type on Apr 14, 2026
  6. linked a pull request that will close this issuefix!: write storage data with camelCase keys #2320on Oct 13, 2026
  7. self-assigned this
    on Oct 13, 2026
  8. added a commit that references this issue on Oct 13, 2026
    24ea410
  9. vdusek commented on Oct 13, 2026

    @vdusek
    Collaborator

    resolved by #2320

  10. added a commit that references this issue on Oct 16, 2026
    0bf67f5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

t-toolingIssues with this label are in the ownership of the tooling team.

Type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions