Skip to content

Commit fbf199c

Browse files
authored
chore!: drop Python 3.10 support (#2295)
First cleanup PR for v2. Python 3.10 reached end of life on 2026-10-04, so Crawlee v2 requires Python 3.11 or newer. - Bumps `requires-python` to `>=3.11` in the package and in the project template, updates the classifiers, the ty target and the CI matrix, and regenerates `uv.lock`. - Applies the ruff fixes the new target enables: `datetime.UTC`, builtin `TimeoutError`, and `typing` imports of `Self` / `NotRequired`. `Event` becomes a `StrEnum`, so `str(Event.X)` returns the value. - Removes the 3.11 skips and the `_restore_crawlers_binding` fixture, which only existed for 3.10. - Adds an "Upgrading to v2" page and updates the Python version in the README and docs. The `apify_client<3.0.0` dev pin and the `async-timeout` dependency (#2052) will go in separate PRs. Closes: #1721 *✍️ Drafted by Claude Code*
1 parent ca71cd2 commit fbf199c

94 files changed

Lines changed: 308 additions & 1352 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/_checks.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,13 +41,13 @@ jobs:
4141
name: Lint check
4242
uses: apify/workflows/.github/workflows/python_lint_check.yaml@main
4343
with:
44-
python_versions: '["3.10", "3.11", "3.12", "3.13", "3.14"]'
44+
python_versions: '["3.11", "3.12", "3.13", "3.14"]'
4545

4646
type_check:
4747
name: Type check
4848
uses: apify/workflows/.github/workflows/python_type_check.yaml@main
4949
with:
50-
python_versions: '["3.10", "3.11", "3.12", "3.13", "3.14"]'
50+
python_versions: '["3.11", "3.12", "3.13", "3.14"]'
5151

5252
markdown_lint_check:
5353
name: Markdown lint check
@@ -135,7 +135,7 @@ jobs:
135135
uses: apify/workflows/.github/workflows/python_unit_tests.yaml@main
136136
secrets: inherit
137137
with:
138-
python_versions: '["3.10", "3.11", "3.12", "3.13", "3.14"]'
138+
python_versions: '["3.11", "3.12", "3.13", "3.14"]'
139139
operating_systems: '["ubuntu-latest", "windows-latest", "macos-latest"]'
140140
python_version_for_codecov: "3.14"
141141
operating_system_for_codecov: ubuntu-latest

‎.rules.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ Note: `uv run poe unit-tests` first runs tests marked `@pytest.mark.run_alone` i
4949
- **Line length**: 120 characters
5050
- **Quotes**: Single quotes (double for docstrings)
5151
- **Docstrings**: Google format (enforced by Ruff)
52-
- **Type checker**: ty (Astral's type checker), target Python 3.10
52+
- **Type checker**: ty (Astral's type checker), target Python 3.11
5353
- **Async mode**: pytest-asyncio in `auto` mode (no need for `@pytest.mark.asyncio`)
5454
- **Commits**: [Conventional Commits](https://www.conventionalcommits.org/) format. Choose the type based on *what* changed, not just *why*:
5555
- `feat:` / `fix:` / `perf:` / `refactor:` / `style:` — **source code only**; these trigger a release and appear in the changelog

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Here you'll find a contributing guide to get started with development.
44

55
## Environment
66

7-
For local development, it is required to have Python 3.10 (or a later version) installed.
7+
For local development, it is required to have Python 3.11 (or a later version) installed.
88

99
We use [uv](https://docs.astral.sh/uv/) for project management. Install it and set up your IDE accordingly.
1010

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
<p align="center">
1717
<a href="https://badge.fury.io/py/crawlee" rel="nofollow"><img src="https://badge.fury.io/py/crawlee.svg" alt="PyPI package version"></a>
1818
<a href="https://pypi.org/project/crawlee/" rel="nofollow"><img src="https://img.shields.io/pypi/dm/crawlee" alt="PyPI package downloads"></a>
19-
<a href="https://pypi.org/project/crawlee/" rel="nofollow"><img src="https://img.shields.io/badge/python-3.10%2B-blue" alt="PyPI Python version"></a>
19+
<a href="https://pypi.org/project/crawlee/" rel="nofollow"><img src="https://img.shields.io/badge/python-3.11%2B-blue" alt="PyPI Python version"></a>
2020
<a href="https://github.com/apify/crawlee-python/actions/workflows/on_master.yaml"><img src="https://github.com/apify/crawlee-python/actions/workflows/on_master.yaml/badge.svg?branch=master" alt="Build status"></a>
2121
<a href="https://codecov.io/gh/apify/crawlee-python"><img src="https://codecov.io/gh/apify/crawlee-python/graph/badge.svg?token=cCju61iPQG" alt="Codecov report"></a>
2222
<a href="https://github.com/apify/crawlee-python/blob/master/LICENSE"><img src="https://img.shields.io/pypi/l/crawlee" alt="License"></a>

‎docs/guides/code_examples/http_crawlers/selectolax_context.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
from dataclasses import dataclass, fields
2+
from typing import Self
23

34
from selectolax.lexbor import LexborHTMLParser
4-
from typing_extensions import Self
55

66
from crawlee.crawlers._abstract_http import ParsedHttpCrawlingContext
77

‎docs/introduction/01_setting_up.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ This guide will help you get started with Crawlee by setting it up on your compu
1414

1515
Before installing Crawlee itself, make sure that your system meets the following requirements:
1616

17-
- **Python 3.10 or higher**: Crawlee requires Python 3.10 or a newer version. You can download Python from the [official website](https://python.org/downloads/).
17+
- **Python 3.11 or higher**: Crawlee requires Python 3.11 or a newer version. You can download Python from the [official website](https://python.org/downloads/).
1818
- **Python package manager**: While this guide uses [pip](https://pip.pypa.io/) (the most common package manager), you can also use any package manager you want. You can download pip from the [official website](https://pip.pypa.io/en/stable/installation/).
1919

2020
### Verifying prerequisites

‎docs/quick-start/index.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Crawlee offers the following main crawler classes: <ApiLink to="class/BeautifulS
2323

2424
:::caution Minimum Python version
2525

26-
Crawlee requires Python 3.10 or higher.
26+
Crawlee requires Python 3.11 or higher.
2727

2828
:::
2929

‎docs/upgrading/upgrading_to_v2.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
id: upgrading-to-v2
3+
title: Upgrading to v2
4+
---
5+
6+
This page summarizes the breaking changes between Crawlee for Python v1 and v2.
7+
8+
## Python version support
9+
10+
We drop support for Python 3.10, which reached its end of life in October 2026. The minimum supported version is now Python 3.11.
11+
12+
## `Event` is a `StrEnum`
13+
14+
`Event` now subclasses `enum.StrEnum`. Calling `str()` on a member or formatting it in an f-string returns its value, for example `'systemInfo'`, where v1 returned `'Event.SYSTEM_INFO'`. Comparisons with plain strings work the same way as in v1.

‎pyproject.toml‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,12 @@ authors = [{ name = "Apify Technologies s.r.o.", email = "support@apify.com" }]
1010
license = "Apache-2.0"
1111
license-files = ["LICENSE"]
1212
readme = "README.md"
13-
requires-python = ">=3.10"
13+
requires-python = ">=3.11"
1414
classifiers = [
1515
"Development Status :: 5 - Production/Stable",
1616
"Environment :: Console",
1717
"Intended Audience :: Developers",
1818
"Operating System :: OS Independent",
19-
"Programming Language :: Python :: 3.10",
2019
"Programming Language :: Python :: 3.11",
2120
"Programming Language :: Python :: 3.12",
2221
"Programming Language :: Python :: 3.13",
@@ -279,7 +278,7 @@ filterwarnings = [
279278
]
280279

281280
[tool.ty.environment]
282-
python-version = "3.10"
281+
python-version = "3.11"
283282

284283
[tool.ty.src]
285284
include = ["src", "tests", "scripts", "docs", "website"]

‎src/crawlee/_autoscaling/_types.py‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
from __future__ import annotations
22

33
from dataclasses import dataclass, field
4-
from datetime import datetime, timedelta, timezone
4+
from datetime import UTC, datetime, timedelta
55
from typing import TYPE_CHECKING, Annotated
66

77
from pydantic import Field
@@ -46,7 +46,7 @@ class SystemInfo:
4646
client_info: LoadRatioInfo
4747
"""The client load ratio."""
4848

49-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
49+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
5050
"""The time at which the system load information was measured."""
5151

5252
@property
@@ -80,7 +80,7 @@ class CpuSnapshot:
8080
max_used_ratio: float
8181
"""The maximum ratio of CPU that is considered acceptable."""
8282

83-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
83+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
8484
"""The time at which the system load information was measured."""
8585

8686
@property
@@ -108,7 +108,7 @@ class MemorySnapshot:
108108
max_used_memory_ratio: float
109109
"""The maximum acceptable ratio of `current_size` to `max_memory_size`."""
110110

111-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
111+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
112112
"""The time at which the system load information was measured."""
113113

114114
@property
@@ -132,7 +132,7 @@ class EventLoopSnapshot:
132132
max_delay: timedelta
133133
"""The maximum delay that is considered acceptable."""
134134

135-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
135+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
136136
"""The time at which the system load information was measured."""
137137

138138
@property
@@ -159,7 +159,7 @@ class ClientSnapshot:
159159
max_error_count: int
160160
"""The maximum number of errors that is considered acceptable."""
161161

162-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
162+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
163163
"""The time at which the system load information was measured."""
164164

165165
@property

0 commit comments

Comments
 (0)